Jul 16 00:38:15.146184 ip-10-0-143-200 systemd[1]: kubelet.service: Failed to load environment files: No such file or directory Jul 16 00:38:15.146196 ip-10-0-143-200 systemd[1]: kubelet.service: Failed to run 'start-pre' task: No such file or directory Jul 16 00:38:15.146205 ip-10-0-143-200 systemd[1]: kubelet.service: Failed with result 'resources'. Jul 16 00:38:15.146549 ip-10-0-143-200 systemd[1]: Failed to start Kubernetes Kubelet. Jul 16 00:38:24.498477 ip-10-0-143-200 systemd[1]: kubelet.service: Failed to schedule restart job: Unit crio.service not found. Jul 16 00:38:24.498492 ip-10-0-143-200 systemd[1]: kubelet.service: Failed with result 'resources'. -- Boot 54d3da2fa85241af9ace16b5da692a63 -- Jul 16 00:39:53.706581 ip-10-0-143-200 systemd[1]: Starting Kubernetes Kubelet... Jul 16 00:39:54.232604 ip-10-0-143-200 kubenswrapper[2572]: Flag --container-runtime-endpoint has been deprecated, This parameter should be set via the config file specified by the Kubelet's --config flag. See https://kubernetes.io/docs/tasks/administer-cluster/kubelet-config-file/ for more information. Jul 16 00:39:54.232604 ip-10-0-143-200 kubenswrapper[2572]: Flag --minimum-container-ttl-duration has been deprecated, Use --eviction-hard or --eviction-soft instead. Will be removed in a future version. Jul 16 00:39:54.232604 ip-10-0-143-200 kubenswrapper[2572]: Flag --volume-plugin-dir has been deprecated, This parameter should be set via the config file specified by the Kubelet's --config flag. See https://kubernetes.io/docs/tasks/administer-cluster/kubelet-config-file/ for more information. Jul 16 00:39:54.232604 ip-10-0-143-200 kubenswrapper[2572]: Flag --pod-infra-container-image has been deprecated, will be removed in 1.35. Image garbage collector will get sandbox image information from CRI. Jul 16 00:39:54.232604 ip-10-0-143-200 kubenswrapper[2572]: Flag --system-reserved has been deprecated, This parameter should be set via the config file specified by the Kubelet's --config flag. See https://kubernetes.io/docs/tasks/administer-cluster/kubelet-config-file/ for more information. Jul 16 00:39:54.232604 ip-10-0-143-200 kubenswrapper[2572]: I0716 00:39:54.165584 2572 server.go:213] "--pod-infra-container-image will not be pruned by the image garbage collector in kubelet and should also be set in the remote runtime" Jul 16 00:39:54.232604 ip-10-0-143-200 kubenswrapper[2572]: W0716 00:39:54.170308 2572 feature_gate.go:328] unrecognized feature gate: AdditionalRoutingCapabilities Jul 16 00:39:54.232604 ip-10-0-143-200 kubenswrapper[2572]: W0716 00:39:54.170323 2572 feature_gate.go:328] unrecognized feature gate: BootImageSkewEnforcement Jul 16 00:39:54.232604 ip-10-0-143-200 kubenswrapper[2572]: W0716 00:39:54.170327 2572 feature_gate.go:328] unrecognized feature gate: MachineAPIOperatorDisableMachineHealthCheckController Jul 16 00:39:54.232604 ip-10-0-143-200 kubenswrapper[2572]: W0716 00:39:54.170331 2572 feature_gate.go:328] unrecognized feature gate: CRDCompatibilityRequirementOperator Jul 16 00:39:54.232604 ip-10-0-143-200 kubenswrapper[2572]: W0716 00:39:54.170334 2572 feature_gate.go:328] unrecognized feature gate: IngressControllerDynamicConfigurationManager Jul 16 00:39:54.232604 ip-10-0-143-200 kubenswrapper[2572]: W0716 00:39:54.170337 2572 feature_gate.go:328] unrecognized feature gate: GatewayAPI Jul 16 00:39:54.232604 ip-10-0-143-200 kubenswrapper[2572]: W0716 00:39:54.170340 2572 feature_gate.go:328] unrecognized feature gate: GatewayAPIWithoutOLM Jul 16 00:39:54.235388 ip-10-0-143-200 kubenswrapper[2572]: W0716 00:39:54.170342 2572 feature_gate.go:328] unrecognized feature gate: HighlyAvailableArbiter Jul 16 00:39:54.235388 ip-10-0-143-200 kubenswrapper[2572]: W0716 00:39:54.170345 2572 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesAzure Jul 16 00:39:54.235388 ip-10-0-143-200 kubenswrapper[2572]: W0716 00:39:54.170348 2572 feature_gate.go:328] unrecognized feature gate: MixedCPUsAllocation Jul 16 00:39:54.235388 ip-10-0-143-200 kubenswrapper[2572]: W0716 00:39:54.170350 2572 feature_gate.go:328] unrecognized feature gate: AWSDualStackInstall Jul 16 00:39:54.235388 ip-10-0-143-200 kubenswrapper[2572]: W0716 00:39:54.170356 2572 feature_gate.go:351] Setting GA feature gate VolumeAttributesClass=true. It will be removed in a future release. Jul 16 00:39:54.235388 ip-10-0-143-200 kubenswrapper[2572]: W0716 00:39:54.170360 2572 feature_gate.go:328] unrecognized feature gate: EtcdBackendQuota Jul 16 00:39:54.235388 ip-10-0-143-200 kubenswrapper[2572]: W0716 00:39:54.170364 2572 feature_gate.go:328] unrecognized feature gate: NewOLMWebhookProviderOpenshiftServiceCA Jul 16 00:39:54.235388 ip-10-0-143-200 kubenswrapper[2572]: W0716 00:39:54.170367 2572 feature_gate.go:328] unrecognized feature gate: KMSEncryption Jul 16 00:39:54.235388 ip-10-0-143-200 kubenswrapper[2572]: W0716 00:39:54.170370 2572 feature_gate.go:328] unrecognized feature gate: AzureDedicatedHosts Jul 16 00:39:54.235388 ip-10-0-143-200 kubenswrapper[2572]: W0716 00:39:54.170373 2572 feature_gate.go:328] unrecognized feature gate: ClusterAPIMachineManagementVSphere Jul 16 00:39:54.235388 ip-10-0-143-200 kubenswrapper[2572]: W0716 00:39:54.170376 2572 feature_gate.go:328] unrecognized feature gate: EventTTL Jul 16 00:39:54.235388 ip-10-0-143-200 kubenswrapper[2572]: W0716 00:39:54.170379 2572 feature_gate.go:328] unrecognized feature gate: ProvisioningRequestAvailable Jul 16 00:39:54.235388 ip-10-0-143-200 kubenswrapper[2572]: W0716 00:39:54.170382 2572 feature_gate.go:328] unrecognized feature gate: NewOLMBoxCutterRuntime Jul 16 00:39:54.235388 ip-10-0-143-200 kubenswrapper[2572]: W0716 00:39:54.170384 2572 feature_gate.go:328] unrecognized feature gate: AWSServiceLBNetworkSecurityGroup Jul 16 00:39:54.235388 ip-10-0-143-200 kubenswrapper[2572]: W0716 00:39:54.170387 2572 feature_gate.go:328] unrecognized feature gate: GCPClusterHostedDNSInstall Jul 16 00:39:54.235388 ip-10-0-143-200 kubenswrapper[2572]: W0716 00:39:54.170390 2572 feature_gate.go:328] unrecognized feature gate: ExternalOIDC Jul 16 00:39:54.235388 ip-10-0-143-200 kubenswrapper[2572]: W0716 00:39:54.170393 2572 feature_gate.go:328] unrecognized feature gate: NetworkSegmentation Jul 16 00:39:54.235388 ip-10-0-143-200 kubenswrapper[2572]: W0716 00:39:54.170396 2572 feature_gate.go:328] unrecognized feature gate: OpenShiftPodSecurityAdmission Jul 16 00:39:54.235388 ip-10-0-143-200 kubenswrapper[2572]: W0716 00:39:54.170399 2572 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesAWS Jul 16 00:39:54.251776 ip-10-0-143-200 kubenswrapper[2572]: W0716 00:39:54.170401 2572 feature_gate.go:328] unrecognized feature gate: AzureMultiDisk Jul 16 00:39:54.251776 ip-10-0-143-200 kubenswrapper[2572]: W0716 00:39:54.170405 2572 feature_gate.go:328] unrecognized feature gate: NetworkLiveMigration Jul 16 00:39:54.251776 ip-10-0-143-200 kubenswrapper[2572]: W0716 00:39:54.170408 2572 feature_gate.go:328] unrecognized feature gate: PreconfiguredUDNAddresses Jul 16 00:39:54.251776 ip-10-0-143-200 kubenswrapper[2572]: W0716 00:39:54.170411 2572 feature_gate.go:328] unrecognized feature gate: ImageStreamImportMode Jul 16 00:39:54.251776 ip-10-0-143-200 kubenswrapper[2572]: W0716 00:39:54.170413 2572 feature_gate.go:328] unrecognized feature gate: MachineAPIMigration Jul 16 00:39:54.251776 ip-10-0-143-200 kubenswrapper[2572]: W0716 00:39:54.170416 2572 feature_gate.go:328] unrecognized feature gate: NewOLM Jul 16 00:39:54.251776 ip-10-0-143-200 kubenswrapper[2572]: W0716 00:39:54.170419 2572 feature_gate.go:328] unrecognized feature gate: VSphereMultiDisk Jul 16 00:39:54.251776 ip-10-0-143-200 kubenswrapper[2572]: W0716 00:39:54.170422 2572 feature_gate.go:328] unrecognized feature gate: Example2 Jul 16 00:39:54.251776 ip-10-0-143-200 kubenswrapper[2572]: W0716 00:39:54.170424 2572 feature_gate.go:328] unrecognized feature gate: AWSClusterHostedDNSInstall Jul 16 00:39:54.251776 ip-10-0-143-200 kubenswrapper[2572]: W0716 00:39:54.170427 2572 feature_gate.go:328] unrecognized feature gate: ExternalOIDCWithUIDAndExtraClaimMappings Jul 16 00:39:54.251776 ip-10-0-143-200 kubenswrapper[2572]: W0716 00:39:54.170429 2572 feature_gate.go:328] unrecognized feature gate: SignatureStores Jul 16 00:39:54.251776 ip-10-0-143-200 kubenswrapper[2572]: W0716 00:39:54.170432 2572 feature_gate.go:328] unrecognized feature gate: SigstoreImageVerificationPKI Jul 16 00:39:54.251776 ip-10-0-143-200 kubenswrapper[2572]: W0716 00:39:54.170435 2572 feature_gate.go:328] unrecognized feature gate: AzureWorkloadIdentity Jul 16 00:39:54.251776 ip-10-0-143-200 kubenswrapper[2572]: W0716 00:39:54.170438 2572 feature_gate.go:328] unrecognized feature gate: ClusterVersionOperatorConfiguration Jul 16 00:39:54.251776 ip-10-0-143-200 kubenswrapper[2572]: W0716 00:39:54.170440 2572 feature_gate.go:328] unrecognized feature gate: SigstoreImageVerification Jul 16 00:39:54.251776 ip-10-0-143-200 kubenswrapper[2572]: W0716 00:39:54.170444 2572 feature_gate.go:328] unrecognized feature gate: InsightsConfig Jul 16 00:39:54.251776 ip-10-0-143-200 kubenswrapper[2572]: W0716 00:39:54.170447 2572 feature_gate.go:328] unrecognized feature gate: BootcNodeManagement Jul 16 00:39:54.251776 ip-10-0-143-200 kubenswrapper[2572]: W0716 00:39:54.170449 2572 feature_gate.go:328] unrecognized feature gate: OnPremDNSRecords Jul 16 00:39:54.251776 ip-10-0-143-200 kubenswrapper[2572]: W0716 00:39:54.170452 2572 feature_gate.go:328] unrecognized feature gate: VolumeGroupSnapshot Jul 16 00:39:54.251776 ip-10-0-143-200 kubenswrapper[2572]: W0716 00:39:54.170454 2572 feature_gate.go:328] unrecognized feature gate: ClusterAPIInstall Jul 16 00:39:54.241905 ip-10-0-143-200 systemd[1]: Started Kubernetes Kubelet. Jul 16 00:39:54.254283 ip-10-0-143-200 kubenswrapper[2572]: W0716 00:39:54.170457 2572 feature_gate.go:328] unrecognized feature gate: ClusterAPIMachineManagement Jul 16 00:39:54.254283 ip-10-0-143-200 kubenswrapper[2572]: W0716 00:39:54.170459 2572 feature_gate.go:328] unrecognized feature gate: NetworkDiagnosticsConfig Jul 16 00:39:54.254283 ip-10-0-143-200 kubenswrapper[2572]: W0716 00:39:54.170462 2572 feature_gate.go:328] unrecognized feature gate: ExternalSnapshotMetadata Jul 16 00:39:54.254283 ip-10-0-143-200 kubenswrapper[2572]: W0716 00:39:54.170465 2572 feature_gate.go:328] unrecognized feature gate: CPMSMachineNamePrefix Jul 16 00:39:54.254283 ip-10-0-143-200 kubenswrapper[2572]: W0716 00:39:54.170467 2572 feature_gate.go:328] unrecognized feature gate: ManagedBootImages Jul 16 00:39:54.254283 ip-10-0-143-200 kubenswrapper[2572]: W0716 00:39:54.170470 2572 feature_gate.go:328] unrecognized feature gate: ImageModeStatusReporting Jul 16 00:39:54.254283 ip-10-0-143-200 kubenswrapper[2572]: W0716 00:39:54.170472 2572 feature_gate.go:328] unrecognized feature gate: MultiArchInstallAzure Jul 16 00:39:54.254283 ip-10-0-143-200 kubenswrapper[2572]: W0716 00:39:54.170475 2572 feature_gate.go:328] unrecognized feature gate: BuildCSIVolumes Jul 16 00:39:54.254283 ip-10-0-143-200 kubenswrapper[2572]: W0716 00:39:54.170478 2572 feature_gate.go:328] unrecognized feature gate: NutanixMultiSubnets Jul 16 00:39:54.254283 ip-10-0-143-200 kubenswrapper[2572]: W0716 00:39:54.170481 2572 feature_gate.go:328] unrecognized feature gate: GCPCustomAPIEndpointsInstall Jul 16 00:39:54.254283 ip-10-0-143-200 kubenswrapper[2572]: W0716 00:39:54.170484 2572 feature_gate.go:328] unrecognized feature gate: NewOLMPreflightPermissionChecks Jul 16 00:39:54.254283 ip-10-0-143-200 kubenswrapper[2572]: W0716 00:39:54.170487 2572 feature_gate.go:328] unrecognized feature gate: AWSClusterHostedDNS Jul 16 00:39:54.254283 ip-10-0-143-200 kubenswrapper[2572]: W0716 00:39:54.170490 2572 feature_gate.go:328] unrecognized feature gate: InsightsOnDemandDataGather Jul 16 00:39:54.254283 ip-10-0-143-200 kubenswrapper[2572]: W0716 00:39:54.170493 2572 feature_gate.go:328] unrecognized feature gate: AutomatedEtcdBackup Jul 16 00:39:54.254283 ip-10-0-143-200 kubenswrapper[2572]: W0716 00:39:54.170497 2572 feature_gate.go:351] Setting GA feature gate ServiceAccountTokenNodeBinding=true. It will be removed in a future release. Jul 16 00:39:54.254283 ip-10-0-143-200 kubenswrapper[2572]: W0716 00:39:54.170501 2572 feature_gate.go:328] unrecognized feature gate: VSphereMultiNetworks Jul 16 00:39:54.254283 ip-10-0-143-200 kubenswrapper[2572]: W0716 00:39:54.170505 2572 feature_gate.go:328] unrecognized feature gate: VSphereHostVMGroupZonal Jul 16 00:39:54.254283 ip-10-0-143-200 kubenswrapper[2572]: W0716 00:39:54.170507 2572 feature_gate.go:328] unrecognized feature gate: UpgradeStatus Jul 16 00:39:54.254283 ip-10-0-143-200 kubenswrapper[2572]: W0716 00:39:54.170558 2572 feature_gate.go:328] unrecognized feature gate: GCPDualStackInstall Jul 16 00:39:54.257332 ip-10-0-143-200 kubenswrapper[2572]: W0716 00:39:54.170561 2572 feature_gate.go:328] unrecognized feature gate: KMSEncryptionProvider Jul 16 00:39:54.257332 ip-10-0-143-200 kubenswrapper[2572]: W0716 00:39:54.170566 2572 feature_gate.go:349] Setting deprecated feature gate KMSv1=true. It will be removed in a future release. Jul 16 00:39:54.257332 ip-10-0-143-200 kubenswrapper[2572]: W0716 00:39:54.170570 2572 feature_gate.go:328] unrecognized feature gate: MultiDiskSetup Jul 16 00:39:54.257332 ip-10-0-143-200 kubenswrapper[2572]: W0716 00:39:54.170573 2572 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesvSphere Jul 16 00:39:54.257332 ip-10-0-143-200 kubenswrapper[2572]: W0716 00:39:54.170576 2572 feature_gate.go:328] unrecognized feature gate: AdminNetworkPolicy Jul 16 00:39:54.257332 ip-10-0-143-200 kubenswrapper[2572]: W0716 00:39:54.170579 2572 feature_gate.go:328] unrecognized feature gate: ClusterMonitoringConfig Jul 16 00:39:54.257332 ip-10-0-143-200 kubenswrapper[2572]: W0716 00:39:54.170582 2572 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesCPMS Jul 16 00:39:54.257332 ip-10-0-143-200 kubenswrapper[2572]: W0716 00:39:54.170585 2572 feature_gate.go:328] unrecognized feature gate: Example Jul 16 00:39:54.257332 ip-10-0-143-200 kubenswrapper[2572]: W0716 00:39:54.170588 2572 feature_gate.go:328] unrecognized feature gate: AWSDedicatedHosts Jul 16 00:39:54.257332 ip-10-0-143-200 kubenswrapper[2572]: W0716 00:39:54.170590 2572 feature_gate.go:328] unrecognized feature gate: DualReplica Jul 16 00:39:54.257332 ip-10-0-143-200 kubenswrapper[2572]: W0716 00:39:54.170593 2572 feature_gate.go:328] unrecognized feature gate: VSphereMixedNodeEnv Jul 16 00:39:54.257332 ip-10-0-143-200 kubenswrapper[2572]: W0716 00:39:54.170596 2572 feature_gate.go:328] unrecognized feature gate: OSStreams Jul 16 00:39:54.257332 ip-10-0-143-200 kubenswrapper[2572]: W0716 00:39:54.170598 2572 feature_gate.go:328] unrecognized feature gate: DyanmicServiceEndpointIBMCloud Jul 16 00:39:54.257332 ip-10-0-143-200 kubenswrapper[2572]: W0716 00:39:54.170601 2572 feature_gate.go:328] unrecognized feature gate: VSphereConfigurableMaxAllowedBlockVolumesPerNode Jul 16 00:39:54.257332 ip-10-0-143-200 kubenswrapper[2572]: W0716 00:39:54.170603 2572 feature_gate.go:328] unrecognized feature gate: ConsolePluginContentSecurityPolicy Jul 16 00:39:54.257332 ip-10-0-143-200 kubenswrapper[2572]: W0716 00:39:54.170606 2572 feature_gate.go:328] unrecognized feature gate: AlibabaPlatform Jul 16 00:39:54.257332 ip-10-0-143-200 kubenswrapper[2572]: W0716 00:39:54.170609 2572 feature_gate.go:328] unrecognized feature gate: AzureClusterHostedDNSInstall Jul 16 00:39:54.257332 ip-10-0-143-200 kubenswrapper[2572]: W0716 00:39:54.170612 2572 feature_gate.go:328] unrecognized feature gate: AzureDualStackInstall Jul 16 00:39:54.257332 ip-10-0-143-200 kubenswrapper[2572]: W0716 00:39:54.170614 2572 feature_gate.go:328] unrecognized feature gate: MetricsCollectionProfiles Jul 16 00:39:54.257332 ip-10-0-143-200 kubenswrapper[2572]: W0716 00:39:54.170617 2572 feature_gate.go:328] unrecognized feature gate: GCPCustomAPIEndpoints Jul 16 00:39:54.260462 ip-10-0-143-200 kubenswrapper[2572]: W0716 00:39:54.170620 2572 feature_gate.go:328] unrecognized feature gate: ShortCertRotation Jul 16 00:39:54.260462 ip-10-0-143-200 kubenswrapper[2572]: W0716 00:39:54.170622 2572 feature_gate.go:328] unrecognized feature gate: HyperShiftOnlyDynamicResourceAllocation Jul 16 00:39:54.260462 ip-10-0-143-200 kubenswrapper[2572]: W0716 00:39:54.170625 2572 feature_gate.go:328] unrecognized feature gate: IrreconcilableMachineConfig Jul 16 00:39:54.260462 ip-10-0-143-200 kubenswrapper[2572]: W0716 00:39:54.170628 2572 feature_gate.go:328] unrecognized feature gate: RouteAdvertisements Jul 16 00:39:54.260462 ip-10-0-143-200 kubenswrapper[2572]: W0716 00:39:54.170630 2572 feature_gate.go:328] unrecognized feature gate: PinnedImages Jul 16 00:39:54.260462 ip-10-0-143-200 kubenswrapper[2572]: W0716 00:39:54.170633 2572 feature_gate.go:328] unrecognized feature gate: GatewayAPIController Jul 16 00:39:54.260462 ip-10-0-143-200 kubenswrapper[2572]: W0716 00:39:54.170636 2572 feature_gate.go:328] unrecognized feature gate: NoRegistryClusterInstall Jul 16 00:39:54.260462 ip-10-0-143-200 kubenswrapper[2572]: W0716 00:39:54.170639 2572 feature_gate.go:328] unrecognized feature gate: DNSNameResolver Jul 16 00:39:54.260462 ip-10-0-143-200 kubenswrapper[2572]: W0716 00:39:54.170641 2572 feature_gate.go:328] unrecognized feature gate: NewOLMOwnSingleNamespace Jul 16 00:39:54.260462 ip-10-0-143-200 kubenswrapper[2572]: W0716 00:39:54.170644 2572 feature_gate.go:328] unrecognized feature gate: OVNObservability Jul 16 00:39:54.260462 ip-10-0-143-200 kubenswrapper[2572]: W0716 00:39:54.170646 2572 feature_gate.go:328] unrecognized feature gate: GCPClusterHostedDNS Jul 16 00:39:54.260462 ip-10-0-143-200 kubenswrapper[2572]: W0716 00:39:54.170649 2572 feature_gate.go:328] unrecognized feature gate: ClusterAPIInstallIBMCloud Jul 16 00:39:54.260462 ip-10-0-143-200 kubenswrapper[2572]: W0716 00:39:54.170651 2572 feature_gate.go:328] unrecognized feature gate: MachineConfigNodes Jul 16 00:39:54.260462 ip-10-0-143-200 kubenswrapper[2572]: W0716 00:39:54.170654 2572 feature_gate.go:328] unrecognized feature gate: NewOLMCatalogdAPIV1Metas Jul 16 00:39:54.260462 ip-10-0-143-200 kubenswrapper[2572]: W0716 00:39:54.171398 2572 feature_gate.go:328] unrecognized feature gate: NetworkDiagnosticsConfig Jul 16 00:39:54.260462 ip-10-0-143-200 kubenswrapper[2572]: W0716 00:39:54.171406 2572 feature_gate.go:328] unrecognized feature gate: AzureDedicatedHosts Jul 16 00:39:54.260462 ip-10-0-143-200 kubenswrapper[2572]: W0716 00:39:54.171409 2572 feature_gate.go:328] unrecognized feature gate: MachineConfigNodes Jul 16 00:39:54.260462 ip-10-0-143-200 kubenswrapper[2572]: W0716 00:39:54.171412 2572 feature_gate.go:328] unrecognized feature gate: Example Jul 16 00:39:54.260462 ip-10-0-143-200 kubenswrapper[2572]: W0716 00:39:54.171414 2572 feature_gate.go:328] unrecognized feature gate: ShortCertRotation Jul 16 00:39:54.260462 ip-10-0-143-200 kubenswrapper[2572]: W0716 00:39:54.171418 2572 feature_gate.go:328] unrecognized feature gate: BootImageSkewEnforcement Jul 16 00:39:54.265750 ip-10-0-143-200 kubenswrapper[2572]: W0716 00:39:54.171421 2572 feature_gate.go:328] unrecognized feature gate: MetricsCollectionProfiles Jul 16 00:39:54.265750 ip-10-0-143-200 kubenswrapper[2572]: W0716 00:39:54.171424 2572 feature_gate.go:328] unrecognized feature gate: SignatureStores Jul 16 00:39:54.265750 ip-10-0-143-200 kubenswrapper[2572]: W0716 00:39:54.171427 2572 feature_gate.go:328] unrecognized feature gate: Example2 Jul 16 00:39:54.265750 ip-10-0-143-200 kubenswrapper[2572]: W0716 00:39:54.171429 2572 feature_gate.go:328] unrecognized feature gate: NetworkLiveMigration Jul 16 00:39:54.265750 ip-10-0-143-200 kubenswrapper[2572]: W0716 00:39:54.171432 2572 feature_gate.go:328] unrecognized feature gate: AlibabaPlatform Jul 16 00:39:54.265750 ip-10-0-143-200 kubenswrapper[2572]: W0716 00:39:54.171435 2572 feature_gate.go:328] unrecognized feature gate: DyanmicServiceEndpointIBMCloud Jul 16 00:39:54.265750 ip-10-0-143-200 kubenswrapper[2572]: W0716 00:39:54.171445 2572 feature_gate.go:328] unrecognized feature gate: GCPCustomAPIEndpoints Jul 16 00:39:54.265750 ip-10-0-143-200 kubenswrapper[2572]: W0716 00:39:54.171448 2572 feature_gate.go:328] unrecognized feature gate: OVNObservability Jul 16 00:39:54.265750 ip-10-0-143-200 kubenswrapper[2572]: W0716 00:39:54.171451 2572 feature_gate.go:328] unrecognized feature gate: InsightsConfig Jul 16 00:39:54.265750 ip-10-0-143-200 kubenswrapper[2572]: W0716 00:39:54.171453 2572 feature_gate.go:328] unrecognized feature gate: MultiArchInstallAzure Jul 16 00:39:54.265750 ip-10-0-143-200 kubenswrapper[2572]: W0716 00:39:54.171456 2572 feature_gate.go:328] unrecognized feature gate: ClusterAPIInstallIBMCloud Jul 16 00:39:54.265750 ip-10-0-143-200 kubenswrapper[2572]: W0716 00:39:54.171458 2572 feature_gate.go:328] unrecognized feature gate: AWSDualStackInstall Jul 16 00:39:54.265750 ip-10-0-143-200 kubenswrapper[2572]: W0716 00:39:54.171461 2572 feature_gate.go:328] unrecognized feature gate: AWSDedicatedHosts Jul 16 00:39:54.265750 ip-10-0-143-200 kubenswrapper[2572]: W0716 00:39:54.171464 2572 feature_gate.go:328] unrecognized feature gate: IngressControllerDynamicConfigurationManager Jul 16 00:39:54.265750 ip-10-0-143-200 kubenswrapper[2572]: W0716 00:39:54.171467 2572 feature_gate.go:328] unrecognized feature gate: GCPClusterHostedDNSInstall Jul 16 00:39:54.265750 ip-10-0-143-200 kubenswrapper[2572]: W0716 00:39:54.171470 2572 feature_gate.go:328] unrecognized feature gate: PreconfiguredUDNAddresses Jul 16 00:39:54.265750 ip-10-0-143-200 kubenswrapper[2572]: W0716 00:39:54.171472 2572 feature_gate.go:328] unrecognized feature gate: SigstoreImageVerification Jul 16 00:39:54.265750 ip-10-0-143-200 kubenswrapper[2572]: W0716 00:39:54.171475 2572 feature_gate.go:328] unrecognized feature gate: GCPDualStackInstall Jul 16 00:39:54.265750 ip-10-0-143-200 kubenswrapper[2572]: W0716 00:39:54.171478 2572 feature_gate.go:328] unrecognized feature gate: AzureWorkloadIdentity Jul 16 00:39:54.265750 ip-10-0-143-200 kubenswrapper[2572]: W0716 00:39:54.171481 2572 feature_gate.go:328] unrecognized feature gate: HyperShiftOnlyDynamicResourceAllocation Jul 16 00:39:54.267862 ip-10-0-143-200 kubenswrapper[2572]: W0716 00:39:54.171488 2572 feature_gate.go:351] Setting GA feature gate VolumeAttributesClass=true. It will be removed in a future release. Jul 16 00:39:54.267862 ip-10-0-143-200 kubenswrapper[2572]: W0716 00:39:54.171492 2572 feature_gate.go:328] unrecognized feature gate: ImageStreamImportMode Jul 16 00:39:54.267862 ip-10-0-143-200 kubenswrapper[2572]: W0716 00:39:54.171495 2572 feature_gate.go:328] unrecognized feature gate: ClusterAPIMachineManagementVSphere Jul 16 00:39:54.267862 ip-10-0-143-200 kubenswrapper[2572]: W0716 00:39:54.171498 2572 feature_gate.go:328] unrecognized feature gate: HighlyAvailableArbiter Jul 16 00:39:54.267862 ip-10-0-143-200 kubenswrapper[2572]: W0716 00:39:54.171502 2572 feature_gate.go:349] Setting deprecated feature gate KMSv1=true. It will be removed in a future release. Jul 16 00:39:54.267862 ip-10-0-143-200 kubenswrapper[2572]: W0716 00:39:54.171505 2572 feature_gate.go:328] unrecognized feature gate: VSphereConfigurableMaxAllowedBlockVolumesPerNode Jul 16 00:39:54.267862 ip-10-0-143-200 kubenswrapper[2572]: W0716 00:39:54.171509 2572 feature_gate.go:328] unrecognized feature gate: AdminNetworkPolicy Jul 16 00:39:54.267862 ip-10-0-143-200 kubenswrapper[2572]: W0716 00:39:54.171511 2572 feature_gate.go:328] unrecognized feature gate: AWSClusterHostedDNSInstall Jul 16 00:39:54.267862 ip-10-0-143-200 kubenswrapper[2572]: W0716 00:39:54.171514 2572 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesAzure Jul 16 00:39:54.267862 ip-10-0-143-200 kubenswrapper[2572]: W0716 00:39:54.171517 2572 feature_gate.go:328] unrecognized feature gate: ExternalOIDCWithUIDAndExtraClaimMappings Jul 16 00:39:54.267862 ip-10-0-143-200 kubenswrapper[2572]: W0716 00:39:54.171520 2572 feature_gate.go:328] unrecognized feature gate: GatewayAPIController Jul 16 00:39:54.267862 ip-10-0-143-200 kubenswrapper[2572]: W0716 00:39:54.171523 2572 feature_gate.go:328] unrecognized feature gate: ExternalOIDC Jul 16 00:39:54.267862 ip-10-0-143-200 kubenswrapper[2572]: W0716 00:39:54.171527 2572 feature_gate.go:328] unrecognized feature gate: BuildCSIVolumes Jul 16 00:39:54.267862 ip-10-0-143-200 kubenswrapper[2572]: W0716 00:39:54.171529 2572 feature_gate.go:328] unrecognized feature gate: AzureDualStackInstall Jul 16 00:39:54.267862 ip-10-0-143-200 kubenswrapper[2572]: W0716 00:39:54.171532 2572 feature_gate.go:328] unrecognized feature gate: OSStreams Jul 16 00:39:54.267862 ip-10-0-143-200 kubenswrapper[2572]: W0716 00:39:54.171535 2572 feature_gate.go:328] unrecognized feature gate: VSphereMixedNodeEnv Jul 16 00:39:54.267862 ip-10-0-143-200 kubenswrapper[2572]: W0716 00:39:54.171537 2572 feature_gate.go:328] unrecognized feature gate: DualReplica Jul 16 00:39:54.267862 ip-10-0-143-200 kubenswrapper[2572]: W0716 00:39:54.171540 2572 feature_gate.go:328] unrecognized feature gate: ManagedBootImages Jul 16 00:39:54.267862 ip-10-0-143-200 kubenswrapper[2572]: W0716 00:39:54.171543 2572 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesAWS Jul 16 00:39:54.269321 ip-10-0-143-200 kubenswrapper[2572]: W0716 00:39:54.171545 2572 feature_gate.go:328] unrecognized feature gate: IrreconcilableMachineConfig Jul 16 00:39:54.269321 ip-10-0-143-200 kubenswrapper[2572]: W0716 00:39:54.171548 2572 feature_gate.go:328] unrecognized feature gate: PinnedImages Jul 16 00:39:54.269321 ip-10-0-143-200 kubenswrapper[2572]: W0716 00:39:54.171550 2572 feature_gate.go:328] unrecognized feature gate: CRDCompatibilityRequirementOperator Jul 16 00:39:54.269321 ip-10-0-143-200 kubenswrapper[2572]: W0716 00:39:54.171553 2572 feature_gate.go:328] unrecognized feature gate: AdditionalRoutingCapabilities Jul 16 00:39:54.269321 ip-10-0-143-200 kubenswrapper[2572]: W0716 00:39:54.171555 2572 feature_gate.go:328] unrecognized feature gate: VSphereMultiDisk Jul 16 00:39:54.269321 ip-10-0-143-200 kubenswrapper[2572]: W0716 00:39:54.171558 2572 feature_gate.go:328] unrecognized feature gate: EtcdBackendQuota Jul 16 00:39:54.269321 ip-10-0-143-200 kubenswrapper[2572]: W0716 00:39:54.171560 2572 feature_gate.go:328] unrecognized feature gate: GCPClusterHostedDNS Jul 16 00:39:54.269321 ip-10-0-143-200 kubenswrapper[2572]: W0716 00:39:54.171563 2572 feature_gate.go:328] unrecognized feature gate: ClusterAPIMachineManagement Jul 16 00:39:54.269321 ip-10-0-143-200 kubenswrapper[2572]: W0716 00:39:54.171565 2572 feature_gate.go:328] unrecognized feature gate: GCPCustomAPIEndpointsInstall Jul 16 00:39:54.269321 ip-10-0-143-200 kubenswrapper[2572]: W0716 00:39:54.171568 2572 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesCPMS Jul 16 00:39:54.269321 ip-10-0-143-200 kubenswrapper[2572]: W0716 00:39:54.171570 2572 feature_gate.go:328] unrecognized feature gate: MachineAPIOperatorDisableMachineHealthCheckController Jul 16 00:39:54.269321 ip-10-0-143-200 kubenswrapper[2572]: W0716 00:39:54.171574 2572 feature_gate.go:351] Setting GA feature gate ServiceAccountTokenNodeBinding=true. It will be removed in a future release. Jul 16 00:39:54.269321 ip-10-0-143-200 kubenswrapper[2572]: W0716 00:39:54.171577 2572 feature_gate.go:328] unrecognized feature gate: ImageModeStatusReporting Jul 16 00:39:54.269321 ip-10-0-143-200 kubenswrapper[2572]: W0716 00:39:54.171583 2572 feature_gate.go:328] unrecognized feature gate: ProvisioningRequestAvailable Jul 16 00:39:54.269321 ip-10-0-143-200 kubenswrapper[2572]: W0716 00:39:54.171586 2572 feature_gate.go:328] unrecognized feature gate: NewOLMWebhookProviderOpenshiftServiceCA Jul 16 00:39:54.269321 ip-10-0-143-200 kubenswrapper[2572]: W0716 00:39:54.171589 2572 feature_gate.go:328] unrecognized feature gate: NetworkSegmentation Jul 16 00:39:54.269321 ip-10-0-143-200 kubenswrapper[2572]: W0716 00:39:54.171592 2572 feature_gate.go:328] unrecognized feature gate: UpgradeStatus Jul 16 00:39:54.269321 ip-10-0-143-200 kubenswrapper[2572]: W0716 00:39:54.171595 2572 feature_gate.go:328] unrecognized feature gate: VolumeGroupSnapshot Jul 16 00:39:54.269321 ip-10-0-143-200 kubenswrapper[2572]: W0716 00:39:54.171597 2572 feature_gate.go:328] unrecognized feature gate: AWSServiceLBNetworkSecurityGroup Jul 16 00:39:54.270623 ip-10-0-143-200 kubenswrapper[2572]: W0716 00:39:54.171600 2572 feature_gate.go:328] unrecognized feature gate: NewOLM Jul 16 00:39:54.270623 ip-10-0-143-200 kubenswrapper[2572]: W0716 00:39:54.171602 2572 feature_gate.go:328] unrecognized feature gate: ClusterMonitoringConfig Jul 16 00:39:54.270623 ip-10-0-143-200 kubenswrapper[2572]: W0716 00:39:54.171605 2572 feature_gate.go:328] unrecognized feature gate: SigstoreImageVerificationPKI Jul 16 00:39:54.270623 ip-10-0-143-200 kubenswrapper[2572]: W0716 00:39:54.171607 2572 feature_gate.go:328] unrecognized feature gate: InsightsOnDemandDataGather Jul 16 00:39:54.270623 ip-10-0-143-200 kubenswrapper[2572]: W0716 00:39:54.171610 2572 feature_gate.go:328] unrecognized feature gate: ClusterVersionOperatorConfiguration Jul 16 00:39:54.270623 ip-10-0-143-200 kubenswrapper[2572]: W0716 00:39:54.171612 2572 feature_gate.go:328] unrecognized feature gate: NoRegistryClusterInstall Jul 16 00:39:54.270623 ip-10-0-143-200 kubenswrapper[2572]: W0716 00:39:54.171615 2572 feature_gate.go:328] unrecognized feature gate: NewOLMPreflightPermissionChecks Jul 16 00:39:54.270623 ip-10-0-143-200 kubenswrapper[2572]: W0716 00:39:54.171619 2572 feature_gate.go:328] unrecognized feature gate: ConsolePluginContentSecurityPolicy Jul 16 00:39:54.270623 ip-10-0-143-200 kubenswrapper[2572]: W0716 00:39:54.171621 2572 feature_gate.go:328] unrecognized feature gate: AutomatedEtcdBackup Jul 16 00:39:54.270623 ip-10-0-143-200 kubenswrapper[2572]: W0716 00:39:54.171624 2572 feature_gate.go:328] unrecognized feature gate: RouteAdvertisements Jul 16 00:39:54.270623 ip-10-0-143-200 kubenswrapper[2572]: W0716 00:39:54.171627 2572 feature_gate.go:328] unrecognized feature gate: MachineAPIMigration Jul 16 00:39:54.270623 ip-10-0-143-200 kubenswrapper[2572]: W0716 00:39:54.171629 2572 feature_gate.go:328] unrecognized feature gate: OpenShiftPodSecurityAdmission Jul 16 00:39:54.270623 ip-10-0-143-200 kubenswrapper[2572]: W0716 00:39:54.171632 2572 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesvSphere Jul 16 00:39:54.270623 ip-10-0-143-200 kubenswrapper[2572]: W0716 00:39:54.171635 2572 feature_gate.go:328] unrecognized feature gate: BootcNodeManagement Jul 16 00:39:54.270623 ip-10-0-143-200 kubenswrapper[2572]: W0716 00:39:54.171637 2572 feature_gate.go:328] unrecognized feature gate: VSphereMultiNetworks Jul 16 00:39:54.270623 ip-10-0-143-200 kubenswrapper[2572]: W0716 00:39:54.171640 2572 feature_gate.go:328] unrecognized feature gate: KMSEncryptionProvider Jul 16 00:39:54.270623 ip-10-0-143-200 kubenswrapper[2572]: W0716 00:39:54.171642 2572 feature_gate.go:328] unrecognized feature gate: VSphereHostVMGroupZonal Jul 16 00:39:54.270623 ip-10-0-143-200 kubenswrapper[2572]: W0716 00:39:54.171646 2572 feature_gate.go:328] unrecognized feature gate: CPMSMachineNamePrefix Jul 16 00:39:54.270623 ip-10-0-143-200 kubenswrapper[2572]: W0716 00:39:54.171648 2572 feature_gate.go:328] unrecognized feature gate: EventTTL Jul 16 00:39:54.270623 ip-10-0-143-200 kubenswrapper[2572]: W0716 00:39:54.171651 2572 feature_gate.go:328] unrecognized feature gate: KMSEncryption Jul 16 00:39:54.277013 ip-10-0-143-200 kubenswrapper[2572]: W0716 00:39:54.171653 2572 feature_gate.go:328] unrecognized feature gate: OnPremDNSRecords Jul 16 00:39:54.277013 ip-10-0-143-200 kubenswrapper[2572]: W0716 00:39:54.171656 2572 feature_gate.go:328] unrecognized feature gate: NewOLMBoxCutterRuntime Jul 16 00:39:54.277013 ip-10-0-143-200 kubenswrapper[2572]: W0716 00:39:54.171659 2572 feature_gate.go:328] unrecognized feature gate: AWSClusterHostedDNS Jul 16 00:39:54.277013 ip-10-0-143-200 kubenswrapper[2572]: W0716 00:39:54.171661 2572 feature_gate.go:328] unrecognized feature gate: ClusterAPIInstall Jul 16 00:39:54.277013 ip-10-0-143-200 kubenswrapper[2572]: W0716 00:39:54.171664 2572 feature_gate.go:328] unrecognized feature gate: NewOLMOwnSingleNamespace Jul 16 00:39:54.277013 ip-10-0-143-200 kubenswrapper[2572]: W0716 00:39:54.171666 2572 feature_gate.go:328] unrecognized feature gate: NutanixMultiSubnets Jul 16 00:39:54.277013 ip-10-0-143-200 kubenswrapper[2572]: W0716 00:39:54.171670 2572 feature_gate.go:328] unrecognized feature gate: GatewayAPI Jul 16 00:39:54.277013 ip-10-0-143-200 kubenswrapper[2572]: W0716 00:39:54.171674 2572 feature_gate.go:328] unrecognized feature gate: ExternalSnapshotMetadata Jul 16 00:39:54.277013 ip-10-0-143-200 kubenswrapper[2572]: W0716 00:39:54.171677 2572 feature_gate.go:328] unrecognized feature gate: MultiDiskSetup Jul 16 00:39:54.277013 ip-10-0-143-200 kubenswrapper[2572]: W0716 00:39:54.171680 2572 feature_gate.go:328] unrecognized feature gate: GatewayAPIWithoutOLM Jul 16 00:39:54.277013 ip-10-0-143-200 kubenswrapper[2572]: W0716 00:39:54.171683 2572 feature_gate.go:328] unrecognized feature gate: AzureClusterHostedDNSInstall Jul 16 00:39:54.277013 ip-10-0-143-200 kubenswrapper[2572]: W0716 00:39:54.171685 2572 feature_gate.go:328] unrecognized feature gate: NewOLMCatalogdAPIV1Metas Jul 16 00:39:54.277013 ip-10-0-143-200 kubenswrapper[2572]: W0716 00:39:54.171688 2572 feature_gate.go:328] unrecognized feature gate: AzureMultiDisk Jul 16 00:39:54.277013 ip-10-0-143-200 kubenswrapper[2572]: W0716 00:39:54.171690 2572 feature_gate.go:328] unrecognized feature gate: MixedCPUsAllocation Jul 16 00:39:54.277013 ip-10-0-143-200 kubenswrapper[2572]: W0716 00:39:54.171693 2572 feature_gate.go:328] unrecognized feature gate: DNSNameResolver Jul 16 00:39:54.277013 ip-10-0-143-200 kubenswrapper[2572]: W0716 00:39:54.172055 2572 feature_gate.go:328] unrecognized feature gate: ProvisioningRequestAvailable Jul 16 00:39:54.277013 ip-10-0-143-200 kubenswrapper[2572]: W0716 00:39:54.172059 2572 feature_gate.go:328] unrecognized feature gate: InsightsOnDemandDataGather Jul 16 00:39:54.277013 ip-10-0-143-200 kubenswrapper[2572]: W0716 00:39:54.172062 2572 feature_gate.go:328] unrecognized feature gate: HighlyAvailableArbiter Jul 16 00:39:54.277013 ip-10-0-143-200 kubenswrapper[2572]: W0716 00:39:54.172065 2572 feature_gate.go:328] unrecognized feature gate: NutanixMultiSubnets Jul 16 00:39:54.277013 ip-10-0-143-200 kubenswrapper[2572]: W0716 00:39:54.172068 2572 feature_gate.go:328] unrecognized feature gate: NewOLMWebhookProviderOpenshiftServiceCA Jul 16 00:39:54.277992 ip-10-0-143-200 kubenswrapper[2572]: W0716 00:39:54.172071 2572 feature_gate.go:328] unrecognized feature gate: GCPClusterHostedDNS Jul 16 00:39:54.277992 ip-10-0-143-200 kubenswrapper[2572]: W0716 00:39:54.172074 2572 feature_gate.go:328] unrecognized feature gate: AWSServiceLBNetworkSecurityGroup Jul 16 00:39:54.277992 ip-10-0-143-200 kubenswrapper[2572]: W0716 00:39:54.172076 2572 feature_gate.go:328] unrecognized feature gate: ExternalSnapshotMetadata Jul 16 00:39:54.277992 ip-10-0-143-200 kubenswrapper[2572]: W0716 00:39:54.172079 2572 feature_gate.go:328] unrecognized feature gate: ExternalOIDC Jul 16 00:39:54.277992 ip-10-0-143-200 kubenswrapper[2572]: W0716 00:39:54.172082 2572 feature_gate.go:328] unrecognized feature gate: DualReplica Jul 16 00:39:54.277992 ip-10-0-143-200 kubenswrapper[2572]: W0716 00:39:54.172084 2572 feature_gate.go:328] unrecognized feature gate: GCPCustomAPIEndpointsInstall Jul 16 00:39:54.277992 ip-10-0-143-200 kubenswrapper[2572]: W0716 00:39:54.172086 2572 feature_gate.go:328] unrecognized feature gate: MetricsCollectionProfiles Jul 16 00:39:54.277992 ip-10-0-143-200 kubenswrapper[2572]: W0716 00:39:54.172089 2572 feature_gate.go:328] unrecognized feature gate: VSphereMultiDisk Jul 16 00:39:54.277992 ip-10-0-143-200 kubenswrapper[2572]: W0716 00:39:54.172092 2572 feature_gate.go:328] unrecognized feature gate: NewOLM Jul 16 00:39:54.277992 ip-10-0-143-200 kubenswrapper[2572]: W0716 00:39:54.172094 2572 feature_gate.go:328] unrecognized feature gate: ManagedBootImages Jul 16 00:39:54.277992 ip-10-0-143-200 kubenswrapper[2572]: W0716 00:39:54.172097 2572 feature_gate.go:328] unrecognized feature gate: MixedCPUsAllocation Jul 16 00:39:54.277992 ip-10-0-143-200 kubenswrapper[2572]: W0716 00:39:54.172099 2572 feature_gate.go:328] unrecognized feature gate: NewOLMOwnSingleNamespace Jul 16 00:39:54.277992 ip-10-0-143-200 kubenswrapper[2572]: W0716 00:39:54.172102 2572 feature_gate.go:328] unrecognized feature gate: SigstoreImageVerification Jul 16 00:39:54.277992 ip-10-0-143-200 kubenswrapper[2572]: W0716 00:39:54.172104 2572 feature_gate.go:328] unrecognized feature gate: AWSClusterHostedDNSInstall Jul 16 00:39:54.277992 ip-10-0-143-200 kubenswrapper[2572]: W0716 00:39:54.172107 2572 feature_gate.go:328] unrecognized feature gate: ImageModeStatusReporting Jul 16 00:39:54.277992 ip-10-0-143-200 kubenswrapper[2572]: W0716 00:39:54.172109 2572 feature_gate.go:328] unrecognized feature gate: SignatureStores Jul 16 00:39:54.277992 ip-10-0-143-200 kubenswrapper[2572]: W0716 00:39:54.172111 2572 feature_gate.go:328] unrecognized feature gate: HyperShiftOnlyDynamicResourceAllocation Jul 16 00:39:54.277992 ip-10-0-143-200 kubenswrapper[2572]: W0716 00:39:54.172114 2572 feature_gate.go:328] unrecognized feature gate: AutomatedEtcdBackup Jul 16 00:39:54.277992 ip-10-0-143-200 kubenswrapper[2572]: W0716 00:39:54.172118 2572 feature_gate.go:328] unrecognized feature gate: ClusterAPIMachineManagementVSphere Jul 16 00:39:54.277992 ip-10-0-143-200 kubenswrapper[2572]: W0716 00:39:54.172123 2572 feature_gate.go:328] unrecognized feature gate: AzureDedicatedHosts Jul 16 00:39:54.278703 ip-10-0-143-200 kubenswrapper[2572]: W0716 00:39:54.172125 2572 feature_gate.go:328] unrecognized feature gate: GatewayAPIWithoutOLM Jul 16 00:39:54.278703 ip-10-0-143-200 kubenswrapper[2572]: W0716 00:39:54.172128 2572 feature_gate.go:328] unrecognized feature gate: EventTTL Jul 16 00:39:54.278703 ip-10-0-143-200 kubenswrapper[2572]: W0716 00:39:54.172131 2572 feature_gate.go:328] unrecognized feature gate: GCPDualStackInstall Jul 16 00:39:54.278703 ip-10-0-143-200 kubenswrapper[2572]: W0716 00:39:54.172133 2572 feature_gate.go:328] unrecognized feature gate: AWSClusterHostedDNS Jul 16 00:39:54.278703 ip-10-0-143-200 kubenswrapper[2572]: W0716 00:39:54.172136 2572 feature_gate.go:328] unrecognized feature gate: MultiDiskSetup Jul 16 00:39:54.278703 ip-10-0-143-200 kubenswrapper[2572]: W0716 00:39:54.172138 2572 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesAzure Jul 16 00:39:54.278703 ip-10-0-143-200 kubenswrapper[2572]: W0716 00:39:54.172140 2572 feature_gate.go:328] unrecognized feature gate: KMSEncryptionProvider Jul 16 00:39:54.278703 ip-10-0-143-200 kubenswrapper[2572]: W0716 00:39:54.172143 2572 feature_gate.go:328] unrecognized feature gate: IngressControllerDynamicConfigurationManager Jul 16 00:39:54.278703 ip-10-0-143-200 kubenswrapper[2572]: W0716 00:39:54.172146 2572 feature_gate.go:328] unrecognized feature gate: NewOLMCatalogdAPIV1Metas Jul 16 00:39:54.278703 ip-10-0-143-200 kubenswrapper[2572]: W0716 00:39:54.172148 2572 feature_gate.go:328] unrecognized feature gate: OVNObservability Jul 16 00:39:54.278703 ip-10-0-143-200 kubenswrapper[2572]: W0716 00:39:54.172151 2572 feature_gate.go:328] unrecognized feature gate: GCPCustomAPIEndpoints Jul 16 00:39:54.278703 ip-10-0-143-200 kubenswrapper[2572]: W0716 00:39:54.172154 2572 feature_gate.go:328] unrecognized feature gate: MachineAPIMigration Jul 16 00:39:54.278703 ip-10-0-143-200 kubenswrapper[2572]: W0716 00:39:54.172157 2572 feature_gate.go:328] unrecognized feature gate: VSphereConfigurableMaxAllowedBlockVolumesPerNode Jul 16 00:39:54.278703 ip-10-0-143-200 kubenswrapper[2572]: W0716 00:39:54.172160 2572 feature_gate.go:328] unrecognized feature gate: AlibabaPlatform Jul 16 00:39:54.278703 ip-10-0-143-200 kubenswrapper[2572]: W0716 00:39:54.172163 2572 feature_gate.go:328] unrecognized feature gate: PinnedImages Jul 16 00:39:54.278703 ip-10-0-143-200 kubenswrapper[2572]: W0716 00:39:54.172165 2572 feature_gate.go:328] unrecognized feature gate: AWSDualStackInstall Jul 16 00:39:54.278703 ip-10-0-143-200 kubenswrapper[2572]: W0716 00:39:54.172167 2572 feature_gate.go:328] unrecognized feature gate: NoRegistryClusterInstall Jul 16 00:39:54.278703 ip-10-0-143-200 kubenswrapper[2572]: W0716 00:39:54.172170 2572 feature_gate.go:328] unrecognized feature gate: VSphereMultiNetworks Jul 16 00:39:54.278703 ip-10-0-143-200 kubenswrapper[2572]: W0716 00:39:54.172172 2572 feature_gate.go:328] unrecognized feature gate: GatewayAPIController Jul 16 00:39:54.278703 ip-10-0-143-200 kubenswrapper[2572]: W0716 00:39:54.172175 2572 feature_gate.go:328] unrecognized feature gate: AdminNetworkPolicy Jul 16 00:39:54.279330 ip-10-0-143-200 kubenswrapper[2572]: W0716 00:39:54.172177 2572 feature_gate.go:328] unrecognized feature gate: KMSEncryption Jul 16 00:39:54.279330 ip-10-0-143-200 kubenswrapper[2572]: W0716 00:39:54.172180 2572 feature_gate.go:328] unrecognized feature gate: DNSNameResolver Jul 16 00:39:54.279330 ip-10-0-143-200 kubenswrapper[2572]: W0716 00:39:54.172184 2572 feature_gate.go:328] unrecognized feature gate: NetworkDiagnosticsConfig Jul 16 00:39:54.279330 ip-10-0-143-200 kubenswrapper[2572]: W0716 00:39:54.172186 2572 feature_gate.go:328] unrecognized feature gate: AzureMultiDisk Jul 16 00:39:54.279330 ip-10-0-143-200 kubenswrapper[2572]: W0716 00:39:54.172189 2572 feature_gate.go:328] unrecognized feature gate: Example2 Jul 16 00:39:54.279330 ip-10-0-143-200 kubenswrapper[2572]: W0716 00:39:54.172191 2572 feature_gate.go:328] unrecognized feature gate: GatewayAPI Jul 16 00:39:54.279330 ip-10-0-143-200 kubenswrapper[2572]: W0716 00:39:54.172194 2572 feature_gate.go:328] unrecognized feature gate: InsightsConfig Jul 16 00:39:54.279330 ip-10-0-143-200 kubenswrapper[2572]: W0716 00:39:54.172196 2572 feature_gate.go:328] unrecognized feature gate: OnPremDNSRecords Jul 16 00:39:54.279330 ip-10-0-143-200 kubenswrapper[2572]: W0716 00:39:54.172199 2572 feature_gate.go:328] unrecognized feature gate: OSStreams Jul 16 00:39:54.279330 ip-10-0-143-200 kubenswrapper[2572]: W0716 00:39:54.172201 2572 feature_gate.go:328] unrecognized feature gate: AzureWorkloadIdentity Jul 16 00:39:54.279330 ip-10-0-143-200 kubenswrapper[2572]: W0716 00:39:54.172205 2572 feature_gate.go:328] unrecognized feature gate: ShortCertRotation Jul 16 00:39:54.279330 ip-10-0-143-200 kubenswrapper[2572]: W0716 00:39:54.172223 2572 feature_gate.go:328] unrecognized feature gate: SigstoreImageVerificationPKI Jul 16 00:39:54.279330 ip-10-0-143-200 kubenswrapper[2572]: W0716 00:39:54.172226 2572 feature_gate.go:328] unrecognized feature gate: MachineConfigNodes Jul 16 00:39:54.279330 ip-10-0-143-200 kubenswrapper[2572]: W0716 00:39:54.172229 2572 feature_gate.go:328] unrecognized feature gate: BootImageSkewEnforcement Jul 16 00:39:54.279330 ip-10-0-143-200 kubenswrapper[2572]: W0716 00:39:54.172232 2572 feature_gate.go:328] unrecognized feature gate: CPMSMachineNamePrefix Jul 16 00:39:54.279330 ip-10-0-143-200 kubenswrapper[2572]: W0716 00:39:54.172235 2572 feature_gate.go:328] unrecognized feature gate: VSphereMixedNodeEnv Jul 16 00:39:54.279330 ip-10-0-143-200 kubenswrapper[2572]: W0716 00:39:54.172237 2572 feature_gate.go:328] unrecognized feature gate: NewOLMBoxCutterRuntime Jul 16 00:39:54.279330 ip-10-0-143-200 kubenswrapper[2572]: W0716 00:39:54.172239 2572 feature_gate.go:328] unrecognized feature gate: VolumeGroupSnapshot Jul 16 00:39:54.279330 ip-10-0-143-200 kubenswrapper[2572]: W0716 00:39:54.172242 2572 feature_gate.go:328] unrecognized feature gate: DyanmicServiceEndpointIBMCloud Jul 16 00:39:54.279330 ip-10-0-143-200 kubenswrapper[2572]: W0716 00:39:54.172245 2572 feature_gate.go:328] unrecognized feature gate: OpenShiftPodSecurityAdmission Jul 16 00:39:54.279955 ip-10-0-143-200 kubenswrapper[2572]: W0716 00:39:54.172247 2572 feature_gate.go:328] unrecognized feature gate: ConsolePluginContentSecurityPolicy Jul 16 00:39:54.279955 ip-10-0-143-200 kubenswrapper[2572]: W0716 00:39:54.172250 2572 feature_gate.go:328] unrecognized feature gate: ExternalOIDCWithUIDAndExtraClaimMappings Jul 16 00:39:54.279955 ip-10-0-143-200 kubenswrapper[2572]: W0716 00:39:54.172252 2572 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesvSphere Jul 16 00:39:54.279955 ip-10-0-143-200 kubenswrapper[2572]: W0716 00:39:54.172255 2572 feature_gate.go:328] unrecognized feature gate: PreconfiguredUDNAddresses Jul 16 00:39:54.279955 ip-10-0-143-200 kubenswrapper[2572]: W0716 00:39:54.172257 2572 feature_gate.go:328] unrecognized feature gate: RouteAdvertisements Jul 16 00:39:54.279955 ip-10-0-143-200 kubenswrapper[2572]: W0716 00:39:54.172261 2572 feature_gate.go:328] unrecognized feature gate: BuildCSIVolumes Jul 16 00:39:54.279955 ip-10-0-143-200 kubenswrapper[2572]: W0716 00:39:54.172264 2572 feature_gate.go:328] unrecognized feature gate: AzureClusterHostedDNSInstall Jul 16 00:39:54.279955 ip-10-0-143-200 kubenswrapper[2572]: W0716 00:39:54.172266 2572 feature_gate.go:328] unrecognized feature gate: ClusterAPIMachineManagement Jul 16 00:39:54.279955 ip-10-0-143-200 kubenswrapper[2572]: W0716 00:39:54.172269 2572 feature_gate.go:328] unrecognized feature gate: NewOLMPreflightPermissionChecks Jul 16 00:39:54.279955 ip-10-0-143-200 kubenswrapper[2572]: W0716 00:39:54.172271 2572 feature_gate.go:328] unrecognized feature gate: AdditionalRoutingCapabilities Jul 16 00:39:54.279955 ip-10-0-143-200 kubenswrapper[2572]: W0716 00:39:54.172274 2572 feature_gate.go:328] unrecognized feature gate: NetworkSegmentation Jul 16 00:39:54.279955 ip-10-0-143-200 kubenswrapper[2572]: W0716 00:39:54.172276 2572 feature_gate.go:328] unrecognized feature gate: Example Jul 16 00:39:54.279955 ip-10-0-143-200 kubenswrapper[2572]: W0716 00:39:54.172279 2572 feature_gate.go:328] unrecognized feature gate: MultiArchInstallAzure Jul 16 00:39:54.279955 ip-10-0-143-200 kubenswrapper[2572]: W0716 00:39:54.172281 2572 feature_gate.go:328] unrecognized feature gate: UpgradeStatus Jul 16 00:39:54.279955 ip-10-0-143-200 kubenswrapper[2572]: W0716 00:39:54.172284 2572 feature_gate.go:328] unrecognized feature gate: VSphereHostVMGroupZonal Jul 16 00:39:54.279955 ip-10-0-143-200 kubenswrapper[2572]: W0716 00:39:54.172287 2572 feature_gate.go:328] unrecognized feature gate: ClusterMonitoringConfig Jul 16 00:39:54.279955 ip-10-0-143-200 kubenswrapper[2572]: W0716 00:39:54.172291 2572 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesAWS Jul 16 00:39:54.279955 ip-10-0-143-200 kubenswrapper[2572]: W0716 00:39:54.172293 2572 feature_gate.go:328] unrecognized feature gate: ClusterVersionOperatorConfiguration Jul 16 00:39:54.279955 ip-10-0-143-200 kubenswrapper[2572]: W0716 00:39:54.172296 2572 feature_gate.go:328] unrecognized feature gate: EtcdBackendQuota Jul 16 00:39:54.280554 ip-10-0-143-200 kubenswrapper[2572]: W0716 00:39:54.172300 2572 feature_gate.go:349] Setting deprecated feature gate KMSv1=true. It will be removed in a future release. Jul 16 00:39:54.280554 ip-10-0-143-200 kubenswrapper[2572]: W0716 00:39:54.172304 2572 feature_gate.go:351] Setting GA feature gate VolumeAttributesClass=true. It will be removed in a future release. Jul 16 00:39:54.280554 ip-10-0-143-200 kubenswrapper[2572]: W0716 00:39:54.172308 2572 feature_gate.go:328] unrecognized feature gate: MachineAPIOperatorDisableMachineHealthCheckController Jul 16 00:39:54.280554 ip-10-0-143-200 kubenswrapper[2572]: W0716 00:39:54.172313 2572 feature_gate.go:328] unrecognized feature gate: ClusterAPIInstallIBMCloud Jul 16 00:39:54.280554 ip-10-0-143-200 kubenswrapper[2572]: W0716 00:39:54.172317 2572 feature_gate.go:328] unrecognized feature gate: BootcNodeManagement Jul 16 00:39:54.280554 ip-10-0-143-200 kubenswrapper[2572]: W0716 00:39:54.172320 2572 feature_gate.go:328] unrecognized feature gate: NetworkLiveMigration Jul 16 00:39:54.280554 ip-10-0-143-200 kubenswrapper[2572]: W0716 00:39:54.172323 2572 feature_gate.go:328] unrecognized feature gate: CRDCompatibilityRequirementOperator Jul 16 00:39:54.280554 ip-10-0-143-200 kubenswrapper[2572]: W0716 00:39:54.172326 2572 feature_gate.go:328] unrecognized feature gate: ImageStreamImportMode Jul 16 00:39:54.280554 ip-10-0-143-200 kubenswrapper[2572]: W0716 00:39:54.172330 2572 feature_gate.go:351] Setting GA feature gate ServiceAccountTokenNodeBinding=true. It will be removed in a future release. Jul 16 00:39:54.280554 ip-10-0-143-200 kubenswrapper[2572]: W0716 00:39:54.172333 2572 feature_gate.go:328] unrecognized feature gate: IrreconcilableMachineConfig Jul 16 00:39:54.280554 ip-10-0-143-200 kubenswrapper[2572]: W0716 00:39:54.172336 2572 feature_gate.go:328] unrecognized feature gate: AzureDualStackInstall Jul 16 00:39:54.280554 ip-10-0-143-200 kubenswrapper[2572]: W0716 00:39:54.172339 2572 feature_gate.go:328] unrecognized feature gate: ClusterAPIInstall Jul 16 00:39:54.280554 ip-10-0-143-200 kubenswrapper[2572]: W0716 00:39:54.172342 2572 feature_gate.go:328] unrecognized feature gate: GCPClusterHostedDNSInstall Jul 16 00:39:54.280554 ip-10-0-143-200 kubenswrapper[2572]: W0716 00:39:54.172344 2572 feature_gate.go:328] unrecognized feature gate: AWSDedicatedHosts Jul 16 00:39:54.280554 ip-10-0-143-200 kubenswrapper[2572]: W0716 00:39:54.172347 2572 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesCPMS Jul 16 00:39:54.280554 ip-10-0-143-200 kubenswrapper[2572]: I0716 00:39:54.172417 2572 flags.go:64] FLAG: --address="0.0.0.0" Jul 16 00:39:54.280554 ip-10-0-143-200 kubenswrapper[2572]: I0716 00:39:54.172424 2572 flags.go:64] FLAG: --allowed-unsafe-sysctls="[]" Jul 16 00:39:54.280554 ip-10-0-143-200 kubenswrapper[2572]: I0716 00:39:54.172430 2572 flags.go:64] FLAG: --anonymous-auth="true" Jul 16 00:39:54.280554 ip-10-0-143-200 kubenswrapper[2572]: I0716 00:39:54.172436 2572 flags.go:64] FLAG: --application-metrics-count-limit="100" Jul 16 00:39:54.281201 ip-10-0-143-200 kubenswrapper[2572]: I0716 00:39:54.172441 2572 flags.go:64] FLAG: --authentication-token-webhook="false" Jul 16 00:39:54.281201 ip-10-0-143-200 kubenswrapper[2572]: I0716 00:39:54.172444 2572 flags.go:64] FLAG: --authentication-token-webhook-cache-ttl="2m0s" Jul 16 00:39:54.281201 ip-10-0-143-200 kubenswrapper[2572]: I0716 00:39:54.172448 2572 flags.go:64] FLAG: --authorization-mode="AlwaysAllow" Jul 16 00:39:54.281201 ip-10-0-143-200 kubenswrapper[2572]: I0716 00:39:54.172453 2572 flags.go:64] FLAG: --authorization-webhook-cache-authorized-ttl="5m0s" Jul 16 00:39:54.281201 ip-10-0-143-200 kubenswrapper[2572]: I0716 00:39:54.172456 2572 flags.go:64] FLAG: --authorization-webhook-cache-unauthorized-ttl="30s" Jul 16 00:39:54.281201 ip-10-0-143-200 kubenswrapper[2572]: I0716 00:39:54.172459 2572 flags.go:64] FLAG: --boot-id-file="/proc/sys/kernel/random/boot_id" Jul 16 00:39:54.281201 ip-10-0-143-200 kubenswrapper[2572]: I0716 00:39:54.172463 2572 flags.go:64] FLAG: --bootstrap-kubeconfig="/etc/kubernetes/kubeconfig" Jul 16 00:39:54.281201 ip-10-0-143-200 kubenswrapper[2572]: I0716 00:39:54.172466 2572 flags.go:64] FLAG: --cert-dir="/var/lib/kubelet/pki" Jul 16 00:39:54.281201 ip-10-0-143-200 kubenswrapper[2572]: I0716 00:39:54.172469 2572 flags.go:64] FLAG: --cgroup-driver="cgroupfs" Jul 16 00:39:54.281201 ip-10-0-143-200 kubenswrapper[2572]: I0716 00:39:54.172472 2572 flags.go:64] FLAG: --cgroup-root="" Jul 16 00:39:54.281201 ip-10-0-143-200 kubenswrapper[2572]: I0716 00:39:54.172475 2572 flags.go:64] FLAG: --cgroups-per-qos="true" Jul 16 00:39:54.281201 ip-10-0-143-200 kubenswrapper[2572]: I0716 00:39:54.172478 2572 flags.go:64] FLAG: --client-ca-file="" Jul 16 00:39:54.281201 ip-10-0-143-200 kubenswrapper[2572]: I0716 00:39:54.172481 2572 flags.go:64] FLAG: --cloud-provider="external" Jul 16 00:39:54.281201 ip-10-0-143-200 kubenswrapper[2572]: I0716 00:39:54.172484 2572 flags.go:64] FLAG: --cluster-dns="[]" Jul 16 00:39:54.281201 ip-10-0-143-200 kubenswrapper[2572]: I0716 00:39:54.172489 2572 flags.go:64] FLAG: --cluster-domain="" Jul 16 00:39:54.281201 ip-10-0-143-200 kubenswrapper[2572]: I0716 00:39:54.172492 2572 flags.go:64] FLAG: --config="/etc/kubernetes/kubelet.conf" Jul 16 00:39:54.281201 ip-10-0-143-200 kubenswrapper[2572]: I0716 00:39:54.172497 2572 flags.go:64] FLAG: --config-dir="/etc/openshift/kubelet.conf.d" Jul 16 00:39:54.281201 ip-10-0-143-200 kubenswrapper[2572]: I0716 00:39:54.172502 2572 flags.go:64] FLAG: --container-hints="/etc/cadvisor/container_hints.json" Jul 16 00:39:54.281201 ip-10-0-143-200 kubenswrapper[2572]: I0716 00:39:54.172505 2572 flags.go:64] FLAG: --container-log-max-files="5" Jul 16 00:39:54.281201 ip-10-0-143-200 kubenswrapper[2572]: I0716 00:39:54.172509 2572 flags.go:64] FLAG: --container-log-max-size="10Mi" Jul 16 00:39:54.281201 ip-10-0-143-200 kubenswrapper[2572]: I0716 00:39:54.172513 2572 flags.go:64] FLAG: --container-runtime-endpoint="/var/run/crio/crio.sock" Jul 16 00:39:54.281201 ip-10-0-143-200 kubenswrapper[2572]: I0716 00:39:54.172516 2572 flags.go:64] FLAG: --containerd="/run/containerd/containerd.sock" Jul 16 00:39:54.281201 ip-10-0-143-200 kubenswrapper[2572]: I0716 00:39:54.172519 2572 flags.go:64] FLAG: --containerd-namespace="k8s.io" Jul 16 00:39:54.282112 ip-10-0-143-200 kubenswrapper[2572]: I0716 00:39:54.172522 2572 flags.go:64] FLAG: --contention-profiling="false" Jul 16 00:39:54.282112 ip-10-0-143-200 kubenswrapper[2572]: I0716 00:39:54.172525 2572 flags.go:64] FLAG: --cpu-cfs-quota="true" Jul 16 00:39:54.282112 ip-10-0-143-200 kubenswrapper[2572]: I0716 00:39:54.172528 2572 flags.go:64] FLAG: --cpu-cfs-quota-period="100ms" Jul 16 00:39:54.282112 ip-10-0-143-200 kubenswrapper[2572]: I0716 00:39:54.172531 2572 flags.go:64] FLAG: --cpu-manager-policy="none" Jul 16 00:39:54.282112 ip-10-0-143-200 kubenswrapper[2572]: I0716 00:39:54.172534 2572 flags.go:64] FLAG: --cpu-manager-policy-options="" Jul 16 00:39:54.282112 ip-10-0-143-200 kubenswrapper[2572]: I0716 00:39:54.172538 2572 flags.go:64] FLAG: --cpu-manager-reconcile-period="10s" Jul 16 00:39:54.282112 ip-10-0-143-200 kubenswrapper[2572]: I0716 00:39:54.172541 2572 flags.go:64] FLAG: --enable-controller-attach-detach="true" Jul 16 00:39:54.282112 ip-10-0-143-200 kubenswrapper[2572]: I0716 00:39:54.172544 2572 flags.go:64] FLAG: --enable-debugging-handlers="true" Jul 16 00:39:54.282112 ip-10-0-143-200 kubenswrapper[2572]: I0716 00:39:54.172547 2572 flags.go:64] FLAG: --enable-load-reader="false" Jul 16 00:39:54.282112 ip-10-0-143-200 kubenswrapper[2572]: I0716 00:39:54.172550 2572 flags.go:64] FLAG: --enable-server="true" Jul 16 00:39:54.282112 ip-10-0-143-200 kubenswrapper[2572]: I0716 00:39:54.172554 2572 flags.go:64] FLAG: --enforce-node-allocatable="[pods]" Jul 16 00:39:54.282112 ip-10-0-143-200 kubenswrapper[2572]: I0716 00:39:54.172559 2572 flags.go:64] FLAG: --event-burst="100" Jul 16 00:39:54.282112 ip-10-0-143-200 kubenswrapper[2572]: I0716 00:39:54.172562 2572 flags.go:64] FLAG: --event-qps="50" Jul 16 00:39:54.282112 ip-10-0-143-200 kubenswrapper[2572]: I0716 00:39:54.172565 2572 flags.go:64] FLAG: --event-storage-age-limit="default=0" Jul 16 00:39:54.282112 ip-10-0-143-200 kubenswrapper[2572]: I0716 00:39:54.172568 2572 flags.go:64] FLAG: --event-storage-event-limit="default=0" Jul 16 00:39:54.282112 ip-10-0-143-200 kubenswrapper[2572]: I0716 00:39:54.172571 2572 flags.go:64] FLAG: --eviction-hard="" Jul 16 00:39:54.282112 ip-10-0-143-200 kubenswrapper[2572]: I0716 00:39:54.172575 2572 flags.go:64] FLAG: --eviction-max-pod-grace-period="0" Jul 16 00:39:54.282112 ip-10-0-143-200 kubenswrapper[2572]: I0716 00:39:54.172578 2572 flags.go:64] FLAG: --eviction-minimum-reclaim="" Jul 16 00:39:54.282112 ip-10-0-143-200 kubenswrapper[2572]: I0716 00:39:54.172581 2572 flags.go:64] FLAG: --eviction-pressure-transition-period="5m0s" Jul 16 00:39:54.282112 ip-10-0-143-200 kubenswrapper[2572]: I0716 00:39:54.172584 2572 flags.go:64] FLAG: --eviction-soft="" Jul 16 00:39:54.282112 ip-10-0-143-200 kubenswrapper[2572]: I0716 00:39:54.172587 2572 flags.go:64] FLAG: --eviction-soft-grace-period="" Jul 16 00:39:54.282112 ip-10-0-143-200 kubenswrapper[2572]: I0716 00:39:54.172590 2572 flags.go:64] FLAG: --exit-on-lock-contention="false" Jul 16 00:39:54.282112 ip-10-0-143-200 kubenswrapper[2572]: I0716 00:39:54.172593 2572 flags.go:64] FLAG: --experimental-allocatable-ignore-eviction="false" Jul 16 00:39:54.282112 ip-10-0-143-200 kubenswrapper[2572]: I0716 00:39:54.172596 2572 flags.go:64] FLAG: --experimental-mounter-path="" Jul 16 00:39:54.282112 ip-10-0-143-200 kubenswrapper[2572]: I0716 00:39:54.172599 2572 flags.go:64] FLAG: --fail-cgroupv1="false" Jul 16 00:39:54.284699 ip-10-0-143-200 kubenswrapper[2572]: I0716 00:39:54.172603 2572 flags.go:64] FLAG: --fail-swap-on="true" Jul 16 00:39:54.284699 ip-10-0-143-200 kubenswrapper[2572]: I0716 00:39:54.172607 2572 flags.go:64] FLAG: --feature-gates="" Jul 16 00:39:54.284699 ip-10-0-143-200 kubenswrapper[2572]: I0716 00:39:54.172611 2572 flags.go:64] FLAG: --file-check-frequency="20s" Jul 16 00:39:54.284699 ip-10-0-143-200 kubenswrapper[2572]: I0716 00:39:54.172614 2572 flags.go:64] FLAG: --global-housekeeping-interval="1m0s" Jul 16 00:39:54.284699 ip-10-0-143-200 kubenswrapper[2572]: I0716 00:39:54.172617 2572 flags.go:64] FLAG: --hairpin-mode="promiscuous-bridge" Jul 16 00:39:54.284699 ip-10-0-143-200 kubenswrapper[2572]: I0716 00:39:54.172621 2572 flags.go:64] FLAG: --healthz-bind-address="127.0.0.1" Jul 16 00:39:54.284699 ip-10-0-143-200 kubenswrapper[2572]: I0716 00:39:54.172624 2572 flags.go:64] FLAG: --healthz-port="10248" Jul 16 00:39:54.284699 ip-10-0-143-200 kubenswrapper[2572]: I0716 00:39:54.172627 2572 flags.go:64] FLAG: --help="false" Jul 16 00:39:54.284699 ip-10-0-143-200 kubenswrapper[2572]: I0716 00:39:54.172630 2572 flags.go:64] FLAG: --hostname-override="ip-10-0-143-200.ec2.internal" Jul 16 00:39:54.284699 ip-10-0-143-200 kubenswrapper[2572]: I0716 00:39:54.172633 2572 flags.go:64] FLAG: --housekeeping-interval="10s" Jul 16 00:39:54.284699 ip-10-0-143-200 kubenswrapper[2572]: I0716 00:39:54.172636 2572 flags.go:64] FLAG: --http-check-frequency="20s" Jul 16 00:39:54.284699 ip-10-0-143-200 kubenswrapper[2572]: I0716 00:39:54.172639 2572 flags.go:64] FLAG: --image-credential-provider-bin-dir="/usr/libexec/kubelet-image-credential-provider-plugins" Jul 16 00:39:54.284699 ip-10-0-143-200 kubenswrapper[2572]: I0716 00:39:54.172642 2572 flags.go:64] FLAG: --image-credential-provider-config="/etc/kubernetes/credential-providers/ecr-credential-provider.yaml" Jul 16 00:39:54.284699 ip-10-0-143-200 kubenswrapper[2572]: I0716 00:39:54.172645 2572 flags.go:64] FLAG: --image-gc-high-threshold="85" Jul 16 00:39:54.284699 ip-10-0-143-200 kubenswrapper[2572]: I0716 00:39:54.172648 2572 flags.go:64] FLAG: --image-gc-low-threshold="80" Jul 16 00:39:54.284699 ip-10-0-143-200 kubenswrapper[2572]: I0716 00:39:54.172651 2572 flags.go:64] FLAG: --image-service-endpoint="" Jul 16 00:39:54.284699 ip-10-0-143-200 kubenswrapper[2572]: I0716 00:39:54.172654 2572 flags.go:64] FLAG: --kernel-memcg-notification="false" Jul 16 00:39:54.284699 ip-10-0-143-200 kubenswrapper[2572]: I0716 00:39:54.172657 2572 flags.go:64] FLAG: --kube-api-burst="100" Jul 16 00:39:54.284699 ip-10-0-143-200 kubenswrapper[2572]: I0716 00:39:54.172661 2572 flags.go:64] FLAG: --kube-api-content-type="application/vnd.kubernetes.protobuf" Jul 16 00:39:54.284699 ip-10-0-143-200 kubenswrapper[2572]: I0716 00:39:54.172664 2572 flags.go:64] FLAG: --kube-api-qps="50" Jul 16 00:39:54.284699 ip-10-0-143-200 kubenswrapper[2572]: I0716 00:39:54.172667 2572 flags.go:64] FLAG: --kube-reserved="" Jul 16 00:39:54.284699 ip-10-0-143-200 kubenswrapper[2572]: I0716 00:39:54.172670 2572 flags.go:64] FLAG: --kube-reserved-cgroup="" Jul 16 00:39:54.284699 ip-10-0-143-200 kubenswrapper[2572]: I0716 00:39:54.172673 2572 flags.go:64] FLAG: --kubeconfig="/var/lib/kubelet/kubeconfig" Jul 16 00:39:54.284699 ip-10-0-143-200 kubenswrapper[2572]: I0716 00:39:54.172676 2572 flags.go:64] FLAG: --kubelet-cgroups="" Jul 16 00:39:54.285801 ip-10-0-143-200 kubenswrapper[2572]: I0716 00:39:54.172679 2572 flags.go:64] FLAG: --local-storage-capacity-isolation="true" Jul 16 00:39:54.285801 ip-10-0-143-200 kubenswrapper[2572]: I0716 00:39:54.172682 2572 flags.go:64] FLAG: --lock-file="" Jul 16 00:39:54.285801 ip-10-0-143-200 kubenswrapper[2572]: I0716 00:39:54.172684 2572 flags.go:64] FLAG: --log-cadvisor-usage="false" Jul 16 00:39:54.285801 ip-10-0-143-200 kubenswrapper[2572]: I0716 00:39:54.172687 2572 flags.go:64] FLAG: --log-flush-frequency="5s" Jul 16 00:39:54.285801 ip-10-0-143-200 kubenswrapper[2572]: I0716 00:39:54.172690 2572 flags.go:64] FLAG: --log-json-info-buffer-size="0" Jul 16 00:39:54.285801 ip-10-0-143-200 kubenswrapper[2572]: I0716 00:39:54.172696 2572 flags.go:64] FLAG: --log-json-split-stream="false" Jul 16 00:39:54.285801 ip-10-0-143-200 kubenswrapper[2572]: I0716 00:39:54.172698 2572 flags.go:64] FLAG: --log-text-info-buffer-size="0" Jul 16 00:39:54.285801 ip-10-0-143-200 kubenswrapper[2572]: I0716 00:39:54.172701 2572 flags.go:64] FLAG: --log-text-split-stream="false" Jul 16 00:39:54.285801 ip-10-0-143-200 kubenswrapper[2572]: I0716 00:39:54.172705 2572 flags.go:64] FLAG: --logging-format="text" Jul 16 00:39:54.285801 ip-10-0-143-200 kubenswrapper[2572]: I0716 00:39:54.172710 2572 flags.go:64] FLAG: --machine-id-file="/etc/machine-id,/var/lib/dbus/machine-id" Jul 16 00:39:54.285801 ip-10-0-143-200 kubenswrapper[2572]: I0716 00:39:54.172713 2572 flags.go:64] FLAG: --make-iptables-util-chains="true" Jul 16 00:39:54.285801 ip-10-0-143-200 kubenswrapper[2572]: I0716 00:39:54.172716 2572 flags.go:64] FLAG: --manifest-url="" Jul 16 00:39:54.285801 ip-10-0-143-200 kubenswrapper[2572]: I0716 00:39:54.172719 2572 flags.go:64] FLAG: --manifest-url-header="" Jul 16 00:39:54.285801 ip-10-0-143-200 kubenswrapper[2572]: I0716 00:39:54.172723 2572 flags.go:64] FLAG: --max-housekeeping-interval="15s" Jul 16 00:39:54.285801 ip-10-0-143-200 kubenswrapper[2572]: I0716 00:39:54.172726 2572 flags.go:64] FLAG: --max-open-files="1000000" Jul 16 00:39:54.285801 ip-10-0-143-200 kubenswrapper[2572]: I0716 00:39:54.172730 2572 flags.go:64] FLAG: --max-pods="110" Jul 16 00:39:54.285801 ip-10-0-143-200 kubenswrapper[2572]: I0716 00:39:54.172733 2572 flags.go:64] FLAG: --maximum-dead-containers="-1" Jul 16 00:39:54.285801 ip-10-0-143-200 kubenswrapper[2572]: I0716 00:39:54.172736 2572 flags.go:64] FLAG: --maximum-dead-containers-per-container="1" Jul 16 00:39:54.285801 ip-10-0-143-200 kubenswrapper[2572]: I0716 00:39:54.172739 2572 flags.go:64] FLAG: --memory-manager-policy="None" Jul 16 00:39:54.285801 ip-10-0-143-200 kubenswrapper[2572]: I0716 00:39:54.172742 2572 flags.go:64] FLAG: --minimum-container-ttl-duration="6m0s" Jul 16 00:39:54.285801 ip-10-0-143-200 kubenswrapper[2572]: I0716 00:39:54.172745 2572 flags.go:64] FLAG: --minimum-image-ttl-duration="2m0s" Jul 16 00:39:54.285801 ip-10-0-143-200 kubenswrapper[2572]: I0716 00:39:54.172748 2572 flags.go:64] FLAG: --node-ip="0.0.0.0" Jul 16 00:39:54.285801 ip-10-0-143-200 kubenswrapper[2572]: I0716 00:39:54.172751 2572 flags.go:64] FLAG: --node-labels="node-role.kubernetes.io/worker=,node.openshift.io/os_id=rhel" Jul 16 00:39:54.285801 ip-10-0-143-200 kubenswrapper[2572]: I0716 00:39:54.172758 2572 flags.go:64] FLAG: --node-status-max-images="50" Jul 16 00:39:54.285801 ip-10-0-143-200 kubenswrapper[2572]: I0716 00:39:54.172761 2572 flags.go:64] FLAG: --node-status-update-frequency="10s" Jul 16 00:39:54.288151 ip-10-0-143-200 kubenswrapper[2572]: I0716 00:39:54.172764 2572 flags.go:64] FLAG: --oom-score-adj="-999" Jul 16 00:39:54.288151 ip-10-0-143-200 kubenswrapper[2572]: I0716 00:39:54.172767 2572 flags.go:64] FLAG: --pod-cidr="" Jul 16 00:39:54.288151 ip-10-0-143-200 kubenswrapper[2572]: I0716 00:39:54.172779 2572 flags.go:64] FLAG: --pod-infra-container-image="quay.io/openshift-release-dev/ocp-v4.0-art-dev@sha256:b04c0b1f73af4d021dd8f0b51bfeb302c24482c093de5f2c43654ab56c506926" Jul 16 00:39:54.288151 ip-10-0-143-200 kubenswrapper[2572]: I0716 00:39:54.172785 2572 flags.go:64] FLAG: --pod-manifest-path="" Jul 16 00:39:54.288151 ip-10-0-143-200 kubenswrapper[2572]: I0716 00:39:54.172788 2572 flags.go:64] FLAG: --pod-max-pids="-1" Jul 16 00:39:54.288151 ip-10-0-143-200 kubenswrapper[2572]: I0716 00:39:54.172791 2572 flags.go:64] FLAG: --pods-per-core="0" Jul 16 00:39:54.288151 ip-10-0-143-200 kubenswrapper[2572]: I0716 00:39:54.172793 2572 flags.go:64] FLAG: --port="10250" Jul 16 00:39:54.288151 ip-10-0-143-200 kubenswrapper[2572]: I0716 00:39:54.172796 2572 flags.go:64] FLAG: --protect-kernel-defaults="false" Jul 16 00:39:54.288151 ip-10-0-143-200 kubenswrapper[2572]: I0716 00:39:54.172799 2572 flags.go:64] FLAG: --provider-id="aws:///us-east-1a/i-0a9bd833ea4532f6a" Jul 16 00:39:54.288151 ip-10-0-143-200 kubenswrapper[2572]: I0716 00:39:54.172803 2572 flags.go:64] FLAG: --qos-reserved="" Jul 16 00:39:54.288151 ip-10-0-143-200 kubenswrapper[2572]: I0716 00:39:54.172805 2572 flags.go:64] FLAG: --read-only-port="10255" Jul 16 00:39:54.288151 ip-10-0-143-200 kubenswrapper[2572]: I0716 00:39:54.172808 2572 flags.go:64] FLAG: --register-node="true" Jul 16 00:39:54.288151 ip-10-0-143-200 kubenswrapper[2572]: I0716 00:39:54.172811 2572 flags.go:64] FLAG: --register-with-taints="" Jul 16 00:39:54.288151 ip-10-0-143-200 kubenswrapper[2572]: I0716 00:39:54.172815 2572 flags.go:64] FLAG: --registry-burst="10" Jul 16 00:39:54.288151 ip-10-0-143-200 kubenswrapper[2572]: I0716 00:39:54.172817 2572 flags.go:64] FLAG: --registry-qps="5" Jul 16 00:39:54.288151 ip-10-0-143-200 kubenswrapper[2572]: I0716 00:39:54.172820 2572 flags.go:64] FLAG: --reserved-cpus="" Jul 16 00:39:54.288151 ip-10-0-143-200 kubenswrapper[2572]: I0716 00:39:54.172826 2572 flags.go:64] FLAG: --reserved-memory="" Jul 16 00:39:54.288151 ip-10-0-143-200 kubenswrapper[2572]: I0716 00:39:54.172830 2572 flags.go:64] FLAG: --resolv-conf="/etc/resolv.conf" Jul 16 00:39:54.288151 ip-10-0-143-200 kubenswrapper[2572]: I0716 00:39:54.172833 2572 flags.go:64] FLAG: --root-dir="/var/lib/kubelet" Jul 16 00:39:54.288151 ip-10-0-143-200 kubenswrapper[2572]: I0716 00:39:54.172836 2572 flags.go:64] FLAG: --rotate-certificates="false" Jul 16 00:39:54.288151 ip-10-0-143-200 kubenswrapper[2572]: I0716 00:39:54.172839 2572 flags.go:64] FLAG: --rotate-server-certificates="false" Jul 16 00:39:54.288151 ip-10-0-143-200 kubenswrapper[2572]: I0716 00:39:54.172842 2572 flags.go:64] FLAG: --runonce="false" Jul 16 00:39:54.288151 ip-10-0-143-200 kubenswrapper[2572]: I0716 00:39:54.172845 2572 flags.go:64] FLAG: --runtime-cgroups="/system.slice/crio.service" Jul 16 00:39:54.288151 ip-10-0-143-200 kubenswrapper[2572]: I0716 00:39:54.172848 2572 flags.go:64] FLAG: --runtime-request-timeout="2m0s" Jul 16 00:39:54.288151 ip-10-0-143-200 kubenswrapper[2572]: I0716 00:39:54.172851 2572 flags.go:64] FLAG: --seccomp-default="false" Jul 16 00:39:54.289299 ip-10-0-143-200 kubenswrapper[2572]: I0716 00:39:54.172854 2572 flags.go:64] FLAG: --serialize-image-pulls="true" Jul 16 00:39:54.289299 ip-10-0-143-200 kubenswrapper[2572]: I0716 00:39:54.172857 2572 flags.go:64] FLAG: --storage-driver-buffer-duration="1m0s" Jul 16 00:39:54.289299 ip-10-0-143-200 kubenswrapper[2572]: I0716 00:39:54.172860 2572 flags.go:64] FLAG: --storage-driver-db="cadvisor" Jul 16 00:39:54.289299 ip-10-0-143-200 kubenswrapper[2572]: I0716 00:39:54.172863 2572 flags.go:64] FLAG: --storage-driver-host="localhost:8086" Jul 16 00:39:54.289299 ip-10-0-143-200 kubenswrapper[2572]: I0716 00:39:54.172866 2572 flags.go:64] FLAG: --storage-driver-password="root" Jul 16 00:39:54.289299 ip-10-0-143-200 kubenswrapper[2572]: I0716 00:39:54.172869 2572 flags.go:64] FLAG: --storage-driver-secure="false" Jul 16 00:39:54.289299 ip-10-0-143-200 kubenswrapper[2572]: I0716 00:39:54.172872 2572 flags.go:64] FLAG: --storage-driver-table="stats" Jul 16 00:39:54.289299 ip-10-0-143-200 kubenswrapper[2572]: I0716 00:39:54.172875 2572 flags.go:64] FLAG: --storage-driver-user="root" Jul 16 00:39:54.289299 ip-10-0-143-200 kubenswrapper[2572]: I0716 00:39:54.172877 2572 flags.go:64] FLAG: --streaming-connection-idle-timeout="4h0m0s" Jul 16 00:39:54.289299 ip-10-0-143-200 kubenswrapper[2572]: I0716 00:39:54.172880 2572 flags.go:64] FLAG: --sync-frequency="1m0s" Jul 16 00:39:54.289299 ip-10-0-143-200 kubenswrapper[2572]: I0716 00:39:54.172884 2572 flags.go:64] FLAG: --system-cgroups="" Jul 16 00:39:54.289299 ip-10-0-143-200 kubenswrapper[2572]: I0716 00:39:54.172887 2572 flags.go:64] FLAG: --system-reserved="cpu=500m,ephemeral-storage=1Gi,memory=1Gi" Jul 16 00:39:54.289299 ip-10-0-143-200 kubenswrapper[2572]: I0716 00:39:54.172892 2572 flags.go:64] FLAG: --system-reserved-cgroup="" Jul 16 00:39:54.289299 ip-10-0-143-200 kubenswrapper[2572]: I0716 00:39:54.172895 2572 flags.go:64] FLAG: --tls-cert-file="" Jul 16 00:39:54.289299 ip-10-0-143-200 kubenswrapper[2572]: I0716 00:39:54.172898 2572 flags.go:64] FLAG: --tls-cipher-suites="[]" Jul 16 00:39:54.289299 ip-10-0-143-200 kubenswrapper[2572]: I0716 00:39:54.172903 2572 flags.go:64] FLAG: --tls-min-version="" Jul 16 00:39:54.289299 ip-10-0-143-200 kubenswrapper[2572]: I0716 00:39:54.172905 2572 flags.go:64] FLAG: --tls-private-key-file="" Jul 16 00:39:54.289299 ip-10-0-143-200 kubenswrapper[2572]: I0716 00:39:54.172909 2572 flags.go:64] FLAG: --topology-manager-policy="none" Jul 16 00:39:54.289299 ip-10-0-143-200 kubenswrapper[2572]: I0716 00:39:54.172912 2572 flags.go:64] FLAG: --topology-manager-policy-options="" Jul 16 00:39:54.289299 ip-10-0-143-200 kubenswrapper[2572]: I0716 00:39:54.172915 2572 flags.go:64] FLAG: --topology-manager-scope="container" Jul 16 00:39:54.289299 ip-10-0-143-200 kubenswrapper[2572]: I0716 00:39:54.172917 2572 flags.go:64] FLAG: --v="2" Jul 16 00:39:54.289299 ip-10-0-143-200 kubenswrapper[2572]: I0716 00:39:54.172926 2572 flags.go:64] FLAG: --version="false" Jul 16 00:39:54.289299 ip-10-0-143-200 kubenswrapper[2572]: I0716 00:39:54.172930 2572 flags.go:64] FLAG: --vmodule="" Jul 16 00:39:54.289299 ip-10-0-143-200 kubenswrapper[2572]: I0716 00:39:54.172937 2572 flags.go:64] FLAG: --volume-plugin-dir="/etc/kubernetes/kubelet-plugins/volume/exec" Jul 16 00:39:54.289299 ip-10-0-143-200 kubenswrapper[2572]: I0716 00:39:54.172940 2572 flags.go:64] FLAG: --volume-stats-agg-period="1m0s" Jul 16 00:39:54.492836 ip-10-0-143-200 kubenswrapper[2572]: W0716 00:39:54.173052 2572 feature_gate.go:328] unrecognized feature gate: MachineAPIOperatorDisableMachineHealthCheckController Jul 16 00:39:54.492836 ip-10-0-143-200 kubenswrapper[2572]: W0716 00:39:54.173056 2572 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesAzure Jul 16 00:39:54.492836 ip-10-0-143-200 kubenswrapper[2572]: W0716 00:39:54.173059 2572 feature_gate.go:328] unrecognized feature gate: BootcNodeManagement Jul 16 00:39:54.492836 ip-10-0-143-200 kubenswrapper[2572]: W0716 00:39:54.173062 2572 feature_gate.go:328] unrecognized feature gate: HyperShiftOnlyDynamicResourceAllocation Jul 16 00:39:54.492836 ip-10-0-143-200 kubenswrapper[2572]: W0716 00:39:54.173065 2572 feature_gate.go:328] unrecognized feature gate: ExternalSnapshotMetadata Jul 16 00:39:54.492836 ip-10-0-143-200 kubenswrapper[2572]: W0716 00:39:54.173068 2572 feature_gate.go:328] unrecognized feature gate: ManagedBootImages Jul 16 00:39:54.492836 ip-10-0-143-200 kubenswrapper[2572]: W0716 00:39:54.173070 2572 feature_gate.go:328] unrecognized feature gate: SigstoreImageVerificationPKI Jul 16 00:39:54.492836 ip-10-0-143-200 kubenswrapper[2572]: W0716 00:39:54.173073 2572 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesvSphere Jul 16 00:39:54.492836 ip-10-0-143-200 kubenswrapper[2572]: W0716 00:39:54.173076 2572 feature_gate.go:328] unrecognized feature gate: KMSEncryptionProvider Jul 16 00:39:54.492836 ip-10-0-143-200 kubenswrapper[2572]: W0716 00:39:54.173078 2572 feature_gate.go:328] unrecognized feature gate: OVNObservability Jul 16 00:39:54.492836 ip-10-0-143-200 kubenswrapper[2572]: W0716 00:39:54.173081 2572 feature_gate.go:328] unrecognized feature gate: AWSServiceLBNetworkSecurityGroup Jul 16 00:39:54.492836 ip-10-0-143-200 kubenswrapper[2572]: W0716 00:39:54.173083 2572 feature_gate.go:328] unrecognized feature gate: NoRegistryClusterInstall Jul 16 00:39:54.492836 ip-10-0-143-200 kubenswrapper[2572]: W0716 00:39:54.173086 2572 feature_gate.go:328] unrecognized feature gate: VSphereMixedNodeEnv Jul 16 00:39:54.492836 ip-10-0-143-200 kubenswrapper[2572]: W0716 00:39:54.173089 2572 feature_gate.go:328] unrecognized feature gate: ClusterMonitoringConfig Jul 16 00:39:54.492836 ip-10-0-143-200 kubenswrapper[2572]: W0716 00:39:54.173091 2572 feature_gate.go:328] unrecognized feature gate: CPMSMachineNamePrefix Jul 16 00:39:54.492836 ip-10-0-143-200 kubenswrapper[2572]: W0716 00:39:54.173094 2572 feature_gate.go:328] unrecognized feature gate: VolumeGroupSnapshot Jul 16 00:39:54.492836 ip-10-0-143-200 kubenswrapper[2572]: W0716 00:39:54.173096 2572 feature_gate.go:328] unrecognized feature gate: AWSClusterHostedDNS Jul 16 00:39:54.492836 ip-10-0-143-200 kubenswrapper[2572]: W0716 00:39:54.173099 2572 feature_gate.go:328] unrecognized feature gate: UpgradeStatus Jul 16 00:39:54.492836 ip-10-0-143-200 kubenswrapper[2572]: W0716 00:39:54.173102 2572 feature_gate.go:328] unrecognized feature gate: GCPClusterHostedDNSInstall Jul 16 00:39:54.507781 ip-10-0-143-200 kubenswrapper[2572]: W0716 00:39:54.173104 2572 feature_gate.go:328] unrecognized feature gate: AzureWorkloadIdentity Jul 16 00:39:54.507781 ip-10-0-143-200 kubenswrapper[2572]: W0716 00:39:54.173107 2572 feature_gate.go:328] unrecognized feature gate: AWSDualStackInstall Jul 16 00:39:54.507781 ip-10-0-143-200 kubenswrapper[2572]: W0716 00:39:54.173110 2572 feature_gate.go:328] unrecognized feature gate: HighlyAvailableArbiter Jul 16 00:39:54.507781 ip-10-0-143-200 kubenswrapper[2572]: W0716 00:39:54.173112 2572 feature_gate.go:328] unrecognized feature gate: VSphereHostVMGroupZonal Jul 16 00:39:54.507781 ip-10-0-143-200 kubenswrapper[2572]: W0716 00:39:54.173114 2572 feature_gate.go:328] unrecognized feature gate: AzureDualStackInstall Jul 16 00:39:54.507781 ip-10-0-143-200 kubenswrapper[2572]: W0716 00:39:54.173117 2572 feature_gate.go:328] unrecognized feature gate: RouteAdvertisements Jul 16 00:39:54.507781 ip-10-0-143-200 kubenswrapper[2572]: W0716 00:39:54.173119 2572 feature_gate.go:328] unrecognized feature gate: ImageStreamImportMode Jul 16 00:39:54.507781 ip-10-0-143-200 kubenswrapper[2572]: W0716 00:39:54.173122 2572 feature_gate.go:328] unrecognized feature gate: ClusterAPIMachineManagement Jul 16 00:39:54.507781 ip-10-0-143-200 kubenswrapper[2572]: W0716 00:39:54.173124 2572 feature_gate.go:328] unrecognized feature gate: AzureDedicatedHosts Jul 16 00:39:54.507781 ip-10-0-143-200 kubenswrapper[2572]: W0716 00:39:54.173127 2572 feature_gate.go:328] unrecognized feature gate: ShortCertRotation Jul 16 00:39:54.507781 ip-10-0-143-200 kubenswrapper[2572]: W0716 00:39:54.173129 2572 feature_gate.go:328] unrecognized feature gate: ImageModeStatusReporting Jul 16 00:39:54.507781 ip-10-0-143-200 kubenswrapper[2572]: W0716 00:39:54.173135 2572 feature_gate.go:328] unrecognized feature gate: ClusterAPIMachineManagementVSphere Jul 16 00:39:54.507781 ip-10-0-143-200 kubenswrapper[2572]: W0716 00:39:54.173137 2572 feature_gate.go:328] unrecognized feature gate: ExternalOIDC Jul 16 00:39:54.507781 ip-10-0-143-200 kubenswrapper[2572]: W0716 00:39:54.173140 2572 feature_gate.go:328] unrecognized feature gate: NewOLMCatalogdAPIV1Metas Jul 16 00:39:54.507781 ip-10-0-143-200 kubenswrapper[2572]: W0716 00:39:54.173142 2572 feature_gate.go:328] unrecognized feature gate: ExternalOIDCWithUIDAndExtraClaimMappings Jul 16 00:39:54.507781 ip-10-0-143-200 kubenswrapper[2572]: W0716 00:39:54.173145 2572 feature_gate.go:328] unrecognized feature gate: NewOLMPreflightPermissionChecks Jul 16 00:39:54.507781 ip-10-0-143-200 kubenswrapper[2572]: W0716 00:39:54.173148 2572 feature_gate.go:328] unrecognized feature gate: OnPremDNSRecords Jul 16 00:39:54.507781 ip-10-0-143-200 kubenswrapper[2572]: W0716 00:39:54.173150 2572 feature_gate.go:328] unrecognized feature gate: AWSDedicatedHosts Jul 16 00:39:54.507781 ip-10-0-143-200 kubenswrapper[2572]: W0716 00:39:54.173153 2572 feature_gate.go:328] unrecognized feature gate: NetworkSegmentation Jul 16 00:39:54.507781 ip-10-0-143-200 kubenswrapper[2572]: W0716 00:39:54.173155 2572 feature_gate.go:328] unrecognized feature gate: CRDCompatibilityRequirementOperator Jul 16 00:39:54.510446 ip-10-0-143-200 kubenswrapper[2572]: W0716 00:39:54.173158 2572 feature_gate.go:328] unrecognized feature gate: NewOLMWebhookProviderOpenshiftServiceCA Jul 16 00:39:54.510446 ip-10-0-143-200 kubenswrapper[2572]: W0716 00:39:54.173161 2572 feature_gate.go:328] unrecognized feature gate: Example2 Jul 16 00:39:54.510446 ip-10-0-143-200 kubenswrapper[2572]: W0716 00:39:54.173163 2572 feature_gate.go:328] unrecognized feature gate: AzureClusterHostedDNSInstall Jul 16 00:39:54.510446 ip-10-0-143-200 kubenswrapper[2572]: W0716 00:39:54.173165 2572 feature_gate.go:328] unrecognized feature gate: GCPCustomAPIEndpointsInstall Jul 16 00:39:54.510446 ip-10-0-143-200 kubenswrapper[2572]: W0716 00:39:54.173169 2572 feature_gate.go:351] Setting GA feature gate VolumeAttributesClass=true. It will be removed in a future release. Jul 16 00:39:54.510446 ip-10-0-143-200 kubenswrapper[2572]: W0716 00:39:54.173173 2572 feature_gate.go:328] unrecognized feature gate: VSphereConfigurableMaxAllowedBlockVolumesPerNode Jul 16 00:39:54.510446 ip-10-0-143-200 kubenswrapper[2572]: W0716 00:39:54.173176 2572 feature_gate.go:328] unrecognized feature gate: NewOLMBoxCutterRuntime Jul 16 00:39:54.510446 ip-10-0-143-200 kubenswrapper[2572]: W0716 00:39:54.173180 2572 feature_gate.go:349] Setting deprecated feature gate KMSv1=true. It will be removed in a future release. Jul 16 00:39:54.510446 ip-10-0-143-200 kubenswrapper[2572]: W0716 00:39:54.173184 2572 feature_gate.go:328] unrecognized feature gate: GatewayAPIWithoutOLM Jul 16 00:39:54.510446 ip-10-0-143-200 kubenswrapper[2572]: W0716 00:39:54.173186 2572 feature_gate.go:328] unrecognized feature gate: VSphereMultiDisk Jul 16 00:39:54.510446 ip-10-0-143-200 kubenswrapper[2572]: W0716 00:39:54.173189 2572 feature_gate.go:328] unrecognized feature gate: MachineConfigNodes Jul 16 00:39:54.510446 ip-10-0-143-200 kubenswrapper[2572]: W0716 00:39:54.173192 2572 feature_gate.go:328] unrecognized feature gate: NewOLM Jul 16 00:39:54.510446 ip-10-0-143-200 kubenswrapper[2572]: W0716 00:39:54.173199 2572 feature_gate.go:328] unrecognized feature gate: GCPClusterHostedDNS Jul 16 00:39:54.510446 ip-10-0-143-200 kubenswrapper[2572]: W0716 00:39:54.173202 2572 feature_gate.go:328] unrecognized feature gate: ClusterVersionOperatorConfiguration Jul 16 00:39:54.510446 ip-10-0-143-200 kubenswrapper[2572]: W0716 00:39:54.173206 2572 feature_gate.go:351] Setting GA feature gate ServiceAccountTokenNodeBinding=true. It will be removed in a future release. Jul 16 00:39:54.510446 ip-10-0-143-200 kubenswrapper[2572]: W0716 00:39:54.173221 2572 feature_gate.go:328] unrecognized feature gate: InsightsOnDemandDataGather Jul 16 00:39:54.510446 ip-10-0-143-200 kubenswrapper[2572]: W0716 00:39:54.173226 2572 feature_gate.go:328] unrecognized feature gate: IrreconcilableMachineConfig Jul 16 00:39:54.510446 ip-10-0-143-200 kubenswrapper[2572]: W0716 00:39:54.173230 2572 feature_gate.go:328] unrecognized feature gate: EventTTL Jul 16 00:39:54.511059 ip-10-0-143-200 kubenswrapper[2572]: W0716 00:39:54.173235 2572 feature_gate.go:328] unrecognized feature gate: MixedCPUsAllocation Jul 16 00:39:54.511059 ip-10-0-143-200 kubenswrapper[2572]: W0716 00:39:54.173239 2572 feature_gate.go:328] unrecognized feature gate: Example Jul 16 00:39:54.511059 ip-10-0-143-200 kubenswrapper[2572]: W0716 00:39:54.173243 2572 feature_gate.go:328] unrecognized feature gate: ClusterAPIInstallIBMCloud Jul 16 00:39:54.511059 ip-10-0-143-200 kubenswrapper[2572]: W0716 00:39:54.173246 2572 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesCPMS Jul 16 00:39:54.511059 ip-10-0-143-200 kubenswrapper[2572]: W0716 00:39:54.173248 2572 feature_gate.go:328] unrecognized feature gate: EtcdBackendQuota Jul 16 00:39:54.511059 ip-10-0-143-200 kubenswrapper[2572]: W0716 00:39:54.173253 2572 feature_gate.go:328] unrecognized feature gate: BootImageSkewEnforcement Jul 16 00:39:54.511059 ip-10-0-143-200 kubenswrapper[2572]: W0716 00:39:54.173256 2572 feature_gate.go:328] unrecognized feature gate: MetricsCollectionProfiles Jul 16 00:39:54.511059 ip-10-0-143-200 kubenswrapper[2572]: W0716 00:39:54.173258 2572 feature_gate.go:328] unrecognized feature gate: AutomatedEtcdBackup Jul 16 00:39:54.511059 ip-10-0-143-200 kubenswrapper[2572]: W0716 00:39:54.173261 2572 feature_gate.go:328] unrecognized feature gate: IngressControllerDynamicConfigurationManager Jul 16 00:39:54.511059 ip-10-0-143-200 kubenswrapper[2572]: W0716 00:39:54.173264 2572 feature_gate.go:328] unrecognized feature gate: MultiArchInstallAzure Jul 16 00:39:54.511059 ip-10-0-143-200 kubenswrapper[2572]: W0716 00:39:54.173266 2572 feature_gate.go:328] unrecognized feature gate: ConsolePluginContentSecurityPolicy Jul 16 00:39:54.511059 ip-10-0-143-200 kubenswrapper[2572]: W0716 00:39:54.173269 2572 feature_gate.go:328] unrecognized feature gate: AWSClusterHostedDNSInstall Jul 16 00:39:54.511059 ip-10-0-143-200 kubenswrapper[2572]: W0716 00:39:54.173272 2572 feature_gate.go:328] unrecognized feature gate: AlibabaPlatform Jul 16 00:39:54.511059 ip-10-0-143-200 kubenswrapper[2572]: W0716 00:39:54.173274 2572 feature_gate.go:328] unrecognized feature gate: PinnedImages Jul 16 00:39:54.511059 ip-10-0-143-200 kubenswrapper[2572]: W0716 00:39:54.173277 2572 feature_gate.go:328] unrecognized feature gate: MultiDiskSetup Jul 16 00:39:54.511059 ip-10-0-143-200 kubenswrapper[2572]: W0716 00:39:54.173280 2572 feature_gate.go:328] unrecognized feature gate: AdditionalRoutingCapabilities Jul 16 00:39:54.511059 ip-10-0-143-200 kubenswrapper[2572]: W0716 00:39:54.173283 2572 feature_gate.go:328] unrecognized feature gate: ProvisioningRequestAvailable Jul 16 00:39:54.511059 ip-10-0-143-200 kubenswrapper[2572]: W0716 00:39:54.173286 2572 feature_gate.go:328] unrecognized feature gate: MachineAPIMigration Jul 16 00:39:54.511059 ip-10-0-143-200 kubenswrapper[2572]: W0716 00:39:54.173288 2572 feature_gate.go:328] unrecognized feature gate: DualReplica Jul 16 00:39:54.511059 ip-10-0-143-200 kubenswrapper[2572]: W0716 00:39:54.173291 2572 feature_gate.go:328] unrecognized feature gate: NetworkLiveMigration Jul 16 00:39:54.514022 ip-10-0-143-200 kubenswrapper[2572]: W0716 00:39:54.173294 2572 feature_gate.go:328] unrecognized feature gate: BuildCSIVolumes Jul 16 00:39:54.514022 ip-10-0-143-200 kubenswrapper[2572]: W0716 00:39:54.173296 2572 feature_gate.go:328] unrecognized feature gate: ClusterAPIInstall Jul 16 00:39:54.514022 ip-10-0-143-200 kubenswrapper[2572]: W0716 00:39:54.173299 2572 feature_gate.go:328] unrecognized feature gate: GCPCustomAPIEndpoints Jul 16 00:39:54.514022 ip-10-0-143-200 kubenswrapper[2572]: W0716 00:39:54.173301 2572 feature_gate.go:328] unrecognized feature gate: GatewayAPI Jul 16 00:39:54.514022 ip-10-0-143-200 kubenswrapper[2572]: W0716 00:39:54.173304 2572 feature_gate.go:328] unrecognized feature gate: OSStreams Jul 16 00:39:54.514022 ip-10-0-143-200 kubenswrapper[2572]: W0716 00:39:54.173306 2572 feature_gate.go:328] unrecognized feature gate: NetworkDiagnosticsConfig Jul 16 00:39:54.514022 ip-10-0-143-200 kubenswrapper[2572]: W0716 00:39:54.173309 2572 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesAWS Jul 16 00:39:54.514022 ip-10-0-143-200 kubenswrapper[2572]: W0716 00:39:54.173312 2572 feature_gate.go:328] unrecognized feature gate: AdminNetworkPolicy Jul 16 00:39:54.514022 ip-10-0-143-200 kubenswrapper[2572]: W0716 00:39:54.173315 2572 feature_gate.go:328] unrecognized feature gate: AzureMultiDisk Jul 16 00:39:54.514022 ip-10-0-143-200 kubenswrapper[2572]: W0716 00:39:54.173318 2572 feature_gate.go:328] unrecognized feature gate: SigstoreImageVerification Jul 16 00:39:54.514022 ip-10-0-143-200 kubenswrapper[2572]: W0716 00:39:54.173320 2572 feature_gate.go:328] unrecognized feature gate: NutanixMultiSubnets Jul 16 00:39:54.514022 ip-10-0-143-200 kubenswrapper[2572]: W0716 00:39:54.173323 2572 feature_gate.go:328] unrecognized feature gate: InsightsConfig Jul 16 00:39:54.514022 ip-10-0-143-200 kubenswrapper[2572]: W0716 00:39:54.173325 2572 feature_gate.go:328] unrecognized feature gate: GatewayAPIController Jul 16 00:39:54.514022 ip-10-0-143-200 kubenswrapper[2572]: W0716 00:39:54.173328 2572 feature_gate.go:328] unrecognized feature gate: OpenShiftPodSecurityAdmission Jul 16 00:39:54.514022 ip-10-0-143-200 kubenswrapper[2572]: W0716 00:39:54.173330 2572 feature_gate.go:328] unrecognized feature gate: SignatureStores Jul 16 00:39:54.514022 ip-10-0-143-200 kubenswrapper[2572]: W0716 00:39:54.173333 2572 feature_gate.go:328] unrecognized feature gate: DNSNameResolver Jul 16 00:39:54.514022 ip-10-0-143-200 kubenswrapper[2572]: W0716 00:39:54.173335 2572 feature_gate.go:328] unrecognized feature gate: DyanmicServiceEndpointIBMCloud Jul 16 00:39:54.514022 ip-10-0-143-200 kubenswrapper[2572]: W0716 00:39:54.173340 2572 feature_gate.go:328] unrecognized feature gate: KMSEncryption Jul 16 00:39:54.514022 ip-10-0-143-200 kubenswrapper[2572]: W0716 00:39:54.173343 2572 feature_gate.go:328] unrecognized feature gate: PreconfiguredUDNAddresses Jul 16 00:39:54.514022 ip-10-0-143-200 kubenswrapper[2572]: W0716 00:39:54.173345 2572 feature_gate.go:328] unrecognized feature gate: VSphereMultiNetworks Jul 16 00:39:54.514660 ip-10-0-143-200 kubenswrapper[2572]: W0716 00:39:54.173348 2572 feature_gate.go:328] unrecognized feature gate: NewOLMOwnSingleNamespace Jul 16 00:39:54.514660 ip-10-0-143-200 kubenswrapper[2572]: W0716 00:39:54.173350 2572 feature_gate.go:328] unrecognized feature gate: GCPDualStackInstall Jul 16 00:39:54.514660 ip-10-0-143-200 kubenswrapper[2572]: I0716 00:39:54.174256 2572 feature_gate.go:384] feature gates: {map[CBORServingAndStorage:false ClientsAllowCBOR:false ClientsPreferCBOR:false EventedPLEG:false ImageVolume:true KMSv1:true MaxUnavailableStatefulSet:false MinimumKubeletVersion:false MutableCSINodeAllocatableCount:false MutatingAdmissionPolicy:false ProcMountType:true RouteExternalCertificate:true SELinuxMount:false ServiceAccountTokenNodeBinding:true StoragePerformantSecurityPolicy:true TranslateStreamCloseWebsocketRequests:false UserNamespacesPodSecurityStandards:true UserNamespacesSupport:true VolumeAttributesClass:true]} Jul 16 00:39:54.514660 ip-10-0-143-200 kubenswrapper[2572]: I0716 00:39:54.181038 2572 server.go:529] "Kubelet version" kubeletVersion="v1.34.9" Jul 16 00:39:54.514660 ip-10-0-143-200 kubenswrapper[2572]: I0716 00:39:54.181054 2572 server.go:531] "Golang settings" GOGC="" GOMAXPROCS="" GOTRACEBACK="" Jul 16 00:39:54.514660 ip-10-0-143-200 kubenswrapper[2572]: I0716 00:39:54.181072 2572 watchdog_linux.go:95] "Systemd watchdog is not enabled" Jul 16 00:39:54.514660 ip-10-0-143-200 kubenswrapper[2572]: I0716 00:39:54.181081 2572 watchdog_linux.go:137] "Systemd watchdog is not enabled or the interval is invalid, so health checking will not be started." Jul 16 00:39:54.514660 ip-10-0-143-200 kubenswrapper[2572]: W0716 00:39:54.181137 2572 feature_gate.go:328] unrecognized feature gate: GCPCustomAPIEndpoints Jul 16 00:39:54.514660 ip-10-0-143-200 kubenswrapper[2572]: W0716 00:39:54.181143 2572 feature_gate.go:328] unrecognized feature gate: ConsolePluginContentSecurityPolicy Jul 16 00:39:54.514660 ip-10-0-143-200 kubenswrapper[2572]: W0716 00:39:54.181147 2572 feature_gate.go:351] Setting GA feature gate ServiceAccountTokenNodeBinding=true. It will be removed in a future release. Jul 16 00:39:54.514660 ip-10-0-143-200 kubenswrapper[2572]: W0716 00:39:54.181152 2572 feature_gate.go:328] unrecognized feature gate: AzureDualStackInstall Jul 16 00:39:54.514660 ip-10-0-143-200 kubenswrapper[2572]: W0716 00:39:54.181155 2572 feature_gate.go:328] unrecognized feature gate: AutomatedEtcdBackup Jul 16 00:39:54.514660 ip-10-0-143-200 kubenswrapper[2572]: W0716 00:39:54.181158 2572 feature_gate.go:328] unrecognized feature gate: GatewayAPI Jul 16 00:39:54.514660 ip-10-0-143-200 kubenswrapper[2572]: W0716 00:39:54.181161 2572 feature_gate.go:328] unrecognized feature gate: Example Jul 16 00:39:54.515260 ip-10-0-143-200 kubenswrapper[2572]: W0716 00:39:54.181164 2572 feature_gate.go:328] unrecognized feature gate: OnPremDNSRecords Jul 16 00:39:54.515260 ip-10-0-143-200 kubenswrapper[2572]: W0716 00:39:54.181168 2572 feature_gate.go:328] unrecognized feature gate: MachineConfigNodes Jul 16 00:39:54.515260 ip-10-0-143-200 kubenswrapper[2572]: W0716 00:39:54.181171 2572 feature_gate.go:328] unrecognized feature gate: ClusterAPIMachineManagement Jul 16 00:39:54.515260 ip-10-0-143-200 kubenswrapper[2572]: W0716 00:39:54.181174 2572 feature_gate.go:328] unrecognized feature gate: ClusterVersionOperatorConfiguration Jul 16 00:39:54.515260 ip-10-0-143-200 kubenswrapper[2572]: W0716 00:39:54.181177 2572 feature_gate.go:328] unrecognized feature gate: GCPCustomAPIEndpointsInstall Jul 16 00:39:54.515260 ip-10-0-143-200 kubenswrapper[2572]: W0716 00:39:54.181181 2572 feature_gate.go:328] unrecognized feature gate: GCPDualStackInstall Jul 16 00:39:54.515260 ip-10-0-143-200 kubenswrapper[2572]: W0716 00:39:54.181184 2572 feature_gate.go:328] unrecognized feature gate: ProvisioningRequestAvailable Jul 16 00:39:54.515260 ip-10-0-143-200 kubenswrapper[2572]: W0716 00:39:54.181187 2572 feature_gate.go:328] unrecognized feature gate: GCPClusterHostedDNS Jul 16 00:39:54.515260 ip-10-0-143-200 kubenswrapper[2572]: W0716 00:39:54.181189 2572 feature_gate.go:328] unrecognized feature gate: AWSClusterHostedDNSInstall Jul 16 00:39:54.515260 ip-10-0-143-200 kubenswrapper[2572]: W0716 00:39:54.181192 2572 feature_gate.go:328] unrecognized feature gate: IngressControllerDynamicConfigurationManager Jul 16 00:39:54.515260 ip-10-0-143-200 kubenswrapper[2572]: W0716 00:39:54.181195 2572 feature_gate.go:328] unrecognized feature gate: AdminNetworkPolicy Jul 16 00:39:54.515260 ip-10-0-143-200 kubenswrapper[2572]: W0716 00:39:54.181197 2572 feature_gate.go:328] unrecognized feature gate: RouteAdvertisements Jul 16 00:39:54.515260 ip-10-0-143-200 kubenswrapper[2572]: W0716 00:39:54.181200 2572 feature_gate.go:328] unrecognized feature gate: AzureDedicatedHosts Jul 16 00:39:54.515260 ip-10-0-143-200 kubenswrapper[2572]: W0716 00:39:54.181203 2572 feature_gate.go:328] unrecognized feature gate: EventTTL Jul 16 00:39:54.515260 ip-10-0-143-200 kubenswrapper[2572]: W0716 00:39:54.181205 2572 feature_gate.go:328] unrecognized feature gate: DyanmicServiceEndpointIBMCloud Jul 16 00:39:54.515260 ip-10-0-143-200 kubenswrapper[2572]: W0716 00:39:54.181225 2572 feature_gate.go:328] unrecognized feature gate: NetworkLiveMigration Jul 16 00:39:54.515260 ip-10-0-143-200 kubenswrapper[2572]: W0716 00:39:54.181230 2572 feature_gate.go:328] unrecognized feature gate: ClusterAPIInstall Jul 16 00:39:54.515260 ip-10-0-143-200 kubenswrapper[2572]: W0716 00:39:54.181233 2572 feature_gate.go:328] unrecognized feature gate: AWSClusterHostedDNS Jul 16 00:39:54.515260 ip-10-0-143-200 kubenswrapper[2572]: W0716 00:39:54.181236 2572 feature_gate.go:328] unrecognized feature gate: NewOLMBoxCutterRuntime Jul 16 00:39:54.515260 ip-10-0-143-200 kubenswrapper[2572]: W0716 00:39:54.181239 2572 feature_gate.go:328] unrecognized feature gate: NewOLMCatalogdAPIV1Metas Jul 16 00:39:54.515890 ip-10-0-143-200 kubenswrapper[2572]: W0716 00:39:54.181242 2572 feature_gate.go:328] unrecognized feature gate: GatewayAPIController Jul 16 00:39:54.515890 ip-10-0-143-200 kubenswrapper[2572]: W0716 00:39:54.181244 2572 feature_gate.go:328] unrecognized feature gate: NetworkDiagnosticsConfig Jul 16 00:39:54.515890 ip-10-0-143-200 kubenswrapper[2572]: W0716 00:39:54.181247 2572 feature_gate.go:328] unrecognized feature gate: PreconfiguredUDNAddresses Jul 16 00:39:54.515890 ip-10-0-143-200 kubenswrapper[2572]: W0716 00:39:54.181250 2572 feature_gate.go:328] unrecognized feature gate: VSphereMultiNetworks Jul 16 00:39:54.515890 ip-10-0-143-200 kubenswrapper[2572]: W0716 00:39:54.181252 2572 feature_gate.go:328] unrecognized feature gate: OSStreams Jul 16 00:39:54.515890 ip-10-0-143-200 kubenswrapper[2572]: W0716 00:39:54.181255 2572 feature_gate.go:328] unrecognized feature gate: BootImageSkewEnforcement Jul 16 00:39:54.515890 ip-10-0-143-200 kubenswrapper[2572]: W0716 00:39:54.181258 2572 feature_gate.go:328] unrecognized feature gate: ExternalOIDCWithUIDAndExtraClaimMappings Jul 16 00:39:54.515890 ip-10-0-143-200 kubenswrapper[2572]: W0716 00:39:54.181260 2572 feature_gate.go:328] unrecognized feature gate: MetricsCollectionProfiles Jul 16 00:39:54.515890 ip-10-0-143-200 kubenswrapper[2572]: W0716 00:39:54.181263 2572 feature_gate.go:328] unrecognized feature gate: ImageStreamImportMode Jul 16 00:39:54.515890 ip-10-0-143-200 kubenswrapper[2572]: W0716 00:39:54.181266 2572 feature_gate.go:328] unrecognized feature gate: VSphereConfigurableMaxAllowedBlockVolumesPerNode Jul 16 00:39:54.515890 ip-10-0-143-200 kubenswrapper[2572]: W0716 00:39:54.181268 2572 feature_gate.go:328] unrecognized feature gate: InsightsConfig Jul 16 00:39:54.515890 ip-10-0-143-200 kubenswrapper[2572]: W0716 00:39:54.181271 2572 feature_gate.go:328] unrecognized feature gate: VSphereMixedNodeEnv Jul 16 00:39:54.515890 ip-10-0-143-200 kubenswrapper[2572]: W0716 00:39:54.181274 2572 feature_gate.go:328] unrecognized feature gate: VolumeGroupSnapshot Jul 16 00:39:54.515890 ip-10-0-143-200 kubenswrapper[2572]: W0716 00:39:54.181277 2572 feature_gate.go:328] unrecognized feature gate: MachineAPIOperatorDisableMachineHealthCheckController Jul 16 00:39:54.515890 ip-10-0-143-200 kubenswrapper[2572]: W0716 00:39:54.181279 2572 feature_gate.go:328] unrecognized feature gate: ImageModeStatusReporting Jul 16 00:39:54.515890 ip-10-0-143-200 kubenswrapper[2572]: W0716 00:39:54.181282 2572 feature_gate.go:328] unrecognized feature gate: OVNObservability Jul 16 00:39:54.515890 ip-10-0-143-200 kubenswrapper[2572]: W0716 00:39:54.181284 2572 feature_gate.go:328] unrecognized feature gate: SignatureStores Jul 16 00:39:54.515890 ip-10-0-143-200 kubenswrapper[2572]: W0716 00:39:54.181287 2572 feature_gate.go:328] unrecognized feature gate: ShortCertRotation Jul 16 00:39:54.515890 ip-10-0-143-200 kubenswrapper[2572]: W0716 00:39:54.181291 2572 feature_gate.go:328] unrecognized feature gate: SigstoreImageVerificationPKI Jul 16 00:39:54.516710 ip-10-0-143-200 kubenswrapper[2572]: W0716 00:39:54.181294 2572 feature_gate.go:328] unrecognized feature gate: AWSDedicatedHosts Jul 16 00:39:54.516710 ip-10-0-143-200 kubenswrapper[2572]: W0716 00:39:54.181296 2572 feature_gate.go:328] unrecognized feature gate: ExternalSnapshotMetadata Jul 16 00:39:54.516710 ip-10-0-143-200 kubenswrapper[2572]: W0716 00:39:54.181299 2572 feature_gate.go:328] unrecognized feature gate: CPMSMachineNamePrefix Jul 16 00:39:54.516710 ip-10-0-143-200 kubenswrapper[2572]: W0716 00:39:54.181302 2572 feature_gate.go:328] unrecognized feature gate: UpgradeStatus Jul 16 00:39:54.516710 ip-10-0-143-200 kubenswrapper[2572]: W0716 00:39:54.181305 2572 feature_gate.go:328] unrecognized feature gate: VSphereMultiDisk Jul 16 00:39:54.516710 ip-10-0-143-200 kubenswrapper[2572]: W0716 00:39:54.181307 2572 feature_gate.go:328] unrecognized feature gate: MixedCPUsAllocation Jul 16 00:39:54.516710 ip-10-0-143-200 kubenswrapper[2572]: W0716 00:39:54.181310 2572 feature_gate.go:328] unrecognized feature gate: HyperShiftOnlyDynamicResourceAllocation Jul 16 00:39:54.516710 ip-10-0-143-200 kubenswrapper[2572]: W0716 00:39:54.181313 2572 feature_gate.go:328] unrecognized feature gate: DNSNameResolver Jul 16 00:39:54.516710 ip-10-0-143-200 kubenswrapper[2572]: W0716 00:39:54.181315 2572 feature_gate.go:328] unrecognized feature gate: NetworkSegmentation Jul 16 00:39:54.516710 ip-10-0-143-200 kubenswrapper[2572]: W0716 00:39:54.181318 2572 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesAWS Jul 16 00:39:54.516710 ip-10-0-143-200 kubenswrapper[2572]: W0716 00:39:54.181320 2572 feature_gate.go:328] unrecognized feature gate: IrreconcilableMachineConfig Jul 16 00:39:54.516710 ip-10-0-143-200 kubenswrapper[2572]: W0716 00:39:54.181323 2572 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesCPMS Jul 16 00:39:54.516710 ip-10-0-143-200 kubenswrapper[2572]: W0716 00:39:54.181326 2572 feature_gate.go:328] unrecognized feature gate: NoRegistryClusterInstall Jul 16 00:39:54.516710 ip-10-0-143-200 kubenswrapper[2572]: W0716 00:39:54.181329 2572 feature_gate.go:328] unrecognized feature gate: NutanixMultiSubnets Jul 16 00:39:54.516710 ip-10-0-143-200 kubenswrapper[2572]: W0716 00:39:54.181332 2572 feature_gate.go:328] unrecognized feature gate: DualReplica Jul 16 00:39:54.516710 ip-10-0-143-200 kubenswrapper[2572]: W0716 00:39:54.181334 2572 feature_gate.go:328] unrecognized feature gate: ClusterAPIMachineManagementVSphere Jul 16 00:39:54.516710 ip-10-0-143-200 kubenswrapper[2572]: W0716 00:39:54.181337 2572 feature_gate.go:328] unrecognized feature gate: MultiDiskSetup Jul 16 00:39:54.516710 ip-10-0-143-200 kubenswrapper[2572]: W0716 00:39:54.181339 2572 feature_gate.go:328] unrecognized feature gate: MachineAPIMigration Jul 16 00:39:54.516710 ip-10-0-143-200 kubenswrapper[2572]: W0716 00:39:54.181342 2572 feature_gate.go:328] unrecognized feature gate: PinnedImages Jul 16 00:39:54.516710 ip-10-0-143-200 kubenswrapper[2572]: W0716 00:39:54.181344 2572 feature_gate.go:328] unrecognized feature gate: AlibabaPlatform Jul 16 00:39:54.517331 ip-10-0-143-200 kubenswrapper[2572]: W0716 00:39:54.181349 2572 feature_gate.go:351] Setting GA feature gate VolumeAttributesClass=true. It will be removed in a future release. Jul 16 00:39:54.517331 ip-10-0-143-200 kubenswrapper[2572]: W0716 00:39:54.181352 2572 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesAzure Jul 16 00:39:54.517331 ip-10-0-143-200 kubenswrapper[2572]: W0716 00:39:54.181355 2572 feature_gate.go:328] unrecognized feature gate: AWSDualStackInstall Jul 16 00:39:54.517331 ip-10-0-143-200 kubenswrapper[2572]: W0716 00:39:54.181358 2572 feature_gate.go:328] unrecognized feature gate: AdditionalRoutingCapabilities Jul 16 00:39:54.517331 ip-10-0-143-200 kubenswrapper[2572]: W0716 00:39:54.181360 2572 feature_gate.go:328] unrecognized feature gate: ExternalOIDC Jul 16 00:39:54.517331 ip-10-0-143-200 kubenswrapper[2572]: W0716 00:39:54.181363 2572 feature_gate.go:328] unrecognized feature gate: GatewayAPIWithoutOLM Jul 16 00:39:54.517331 ip-10-0-143-200 kubenswrapper[2572]: W0716 00:39:54.181366 2572 feature_gate.go:328] unrecognized feature gate: NewOLM Jul 16 00:39:54.517331 ip-10-0-143-200 kubenswrapper[2572]: W0716 00:39:54.181369 2572 feature_gate.go:328] unrecognized feature gate: OpenShiftPodSecurityAdmission Jul 16 00:39:54.517331 ip-10-0-143-200 kubenswrapper[2572]: W0716 00:39:54.181371 2572 feature_gate.go:328] unrecognized feature gate: NewOLMPreflightPermissionChecks Jul 16 00:39:54.517331 ip-10-0-143-200 kubenswrapper[2572]: W0716 00:39:54.181374 2572 feature_gate.go:328] unrecognized feature gate: BootcNodeManagement Jul 16 00:39:54.517331 ip-10-0-143-200 kubenswrapper[2572]: W0716 00:39:54.181377 2572 feature_gate.go:328] unrecognized feature gate: InsightsOnDemandDataGather Jul 16 00:39:54.517331 ip-10-0-143-200 kubenswrapper[2572]: W0716 00:39:54.181379 2572 feature_gate.go:328] unrecognized feature gate: NewOLMOwnSingleNamespace Jul 16 00:39:54.517331 ip-10-0-143-200 kubenswrapper[2572]: W0716 00:39:54.181382 2572 feature_gate.go:328] unrecognized feature gate: AzureMultiDisk Jul 16 00:39:54.517331 ip-10-0-143-200 kubenswrapper[2572]: W0716 00:39:54.181385 2572 feature_gate.go:328] unrecognized feature gate: Example2 Jul 16 00:39:54.517331 ip-10-0-143-200 kubenswrapper[2572]: W0716 00:39:54.181389 2572 feature_gate.go:349] Setting deprecated feature gate KMSv1=true. It will be removed in a future release. Jul 16 00:39:54.517331 ip-10-0-143-200 kubenswrapper[2572]: W0716 00:39:54.181393 2572 feature_gate.go:328] unrecognized feature gate: CRDCompatibilityRequirementOperator Jul 16 00:39:54.517331 ip-10-0-143-200 kubenswrapper[2572]: W0716 00:39:54.181396 2572 feature_gate.go:328] unrecognized feature gate: BuildCSIVolumes Jul 16 00:39:54.517331 ip-10-0-143-200 kubenswrapper[2572]: W0716 00:39:54.181399 2572 feature_gate.go:328] unrecognized feature gate: GCPClusterHostedDNSInstall Jul 16 00:39:54.517331 ip-10-0-143-200 kubenswrapper[2572]: W0716 00:39:54.181401 2572 feature_gate.go:328] unrecognized feature gate: HighlyAvailableArbiter Jul 16 00:39:54.518132 ip-10-0-143-200 kubenswrapper[2572]: W0716 00:39:54.181404 2572 feature_gate.go:328] unrecognized feature gate: SigstoreImageVerification Jul 16 00:39:54.518132 ip-10-0-143-200 kubenswrapper[2572]: W0716 00:39:54.181407 2572 feature_gate.go:328] unrecognized feature gate: AzureWorkloadIdentity Jul 16 00:39:54.518132 ip-10-0-143-200 kubenswrapper[2572]: W0716 00:39:54.181409 2572 feature_gate.go:328] unrecognized feature gate: NewOLMWebhookProviderOpenshiftServiceCA Jul 16 00:39:54.518132 ip-10-0-143-200 kubenswrapper[2572]: W0716 00:39:54.181412 2572 feature_gate.go:328] unrecognized feature gate: KMSEncryptionProvider Jul 16 00:39:54.518132 ip-10-0-143-200 kubenswrapper[2572]: W0716 00:39:54.181415 2572 feature_gate.go:328] unrecognized feature gate: KMSEncryption Jul 16 00:39:54.518132 ip-10-0-143-200 kubenswrapper[2572]: W0716 00:39:54.181418 2572 feature_gate.go:328] unrecognized feature gate: AzureClusterHostedDNSInstall Jul 16 00:39:54.518132 ip-10-0-143-200 kubenswrapper[2572]: W0716 00:39:54.181420 2572 feature_gate.go:328] unrecognized feature gate: EtcdBackendQuota Jul 16 00:39:54.518132 ip-10-0-143-200 kubenswrapper[2572]: W0716 00:39:54.181423 2572 feature_gate.go:328] unrecognized feature gate: AWSServiceLBNetworkSecurityGroup Jul 16 00:39:54.518132 ip-10-0-143-200 kubenswrapper[2572]: W0716 00:39:54.181425 2572 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesvSphere Jul 16 00:39:54.518132 ip-10-0-143-200 kubenswrapper[2572]: W0716 00:39:54.181428 2572 feature_gate.go:328] unrecognized feature gate: ClusterMonitoringConfig Jul 16 00:39:54.518132 ip-10-0-143-200 kubenswrapper[2572]: W0716 00:39:54.181431 2572 feature_gate.go:328] unrecognized feature gate: ManagedBootImages Jul 16 00:39:54.518132 ip-10-0-143-200 kubenswrapper[2572]: W0716 00:39:54.181433 2572 feature_gate.go:328] unrecognized feature gate: MultiArchInstallAzure Jul 16 00:39:54.518132 ip-10-0-143-200 kubenswrapper[2572]: W0716 00:39:54.181436 2572 feature_gate.go:328] unrecognized feature gate: VSphereHostVMGroupZonal Jul 16 00:39:54.518132 ip-10-0-143-200 kubenswrapper[2572]: W0716 00:39:54.181438 2572 feature_gate.go:328] unrecognized feature gate: ClusterAPIInstallIBMCloud Jul 16 00:39:54.518132 ip-10-0-143-200 kubenswrapper[2572]: I0716 00:39:54.181443 2572 feature_gate.go:384] feature gates: {map[CBORServingAndStorage:false ClientsAllowCBOR:false ClientsPreferCBOR:false EventedPLEG:false ImageVolume:true KMSv1:true MaxUnavailableStatefulSet:false MinimumKubeletVersion:false MutableCSINodeAllocatableCount:false MutatingAdmissionPolicy:false ProcMountType:true RouteExternalCertificate:true SELinuxMount:false ServiceAccountTokenNodeBinding:true StoragePerformantSecurityPolicy:true TranslateStreamCloseWebsocketRequests:false UserNamespacesPodSecurityStandards:true UserNamespacesSupport:true VolumeAttributesClass:true]} Jul 16 00:39:54.518617 ip-10-0-143-200 kubenswrapper[2572]: W0716 00:39:54.181555 2572 feature_gate.go:328] unrecognized feature gate: EventTTL Jul 16 00:39:54.518617 ip-10-0-143-200 kubenswrapper[2572]: W0716 00:39:54.181559 2572 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesAzure Jul 16 00:39:54.518617 ip-10-0-143-200 kubenswrapper[2572]: W0716 00:39:54.181562 2572 feature_gate.go:328] unrecognized feature gate: UpgradeStatus Jul 16 00:39:54.518617 ip-10-0-143-200 kubenswrapper[2572]: W0716 00:39:54.181576 2572 feature_gate.go:328] unrecognized feature gate: OnPremDNSRecords Jul 16 00:39:54.518617 ip-10-0-143-200 kubenswrapper[2572]: W0716 00:39:54.181579 2572 feature_gate.go:328] unrecognized feature gate: ConsolePluginContentSecurityPolicy Jul 16 00:39:54.518617 ip-10-0-143-200 kubenswrapper[2572]: W0716 00:39:54.181582 2572 feature_gate.go:328] unrecognized feature gate: SignatureStores Jul 16 00:39:54.518617 ip-10-0-143-200 kubenswrapper[2572]: W0716 00:39:54.181585 2572 feature_gate.go:328] unrecognized feature gate: BootcNodeManagement Jul 16 00:39:54.518617 ip-10-0-143-200 kubenswrapper[2572]: W0716 00:39:54.181588 2572 feature_gate.go:328] unrecognized feature gate: SigstoreImageVerificationPKI Jul 16 00:39:54.518617 ip-10-0-143-200 kubenswrapper[2572]: W0716 00:39:54.181591 2572 feature_gate.go:328] unrecognized feature gate: NewOLM Jul 16 00:39:54.518617 ip-10-0-143-200 kubenswrapper[2572]: W0716 00:39:54.181593 2572 feature_gate.go:328] unrecognized feature gate: ClusterMonitoringConfig Jul 16 00:39:54.518617 ip-10-0-143-200 kubenswrapper[2572]: W0716 00:39:54.181596 2572 feature_gate.go:328] unrecognized feature gate: AutomatedEtcdBackup Jul 16 00:39:54.518617 ip-10-0-143-200 kubenswrapper[2572]: W0716 00:39:54.181600 2572 feature_gate.go:328] unrecognized feature gate: NewOLMBoxCutterRuntime Jul 16 00:39:54.518617 ip-10-0-143-200 kubenswrapper[2572]: W0716 00:39:54.181603 2572 feature_gate.go:328] unrecognized feature gate: IngressControllerDynamicConfigurationManager Jul 16 00:39:54.518617 ip-10-0-143-200 kubenswrapper[2572]: W0716 00:39:54.181606 2572 feature_gate.go:328] unrecognized feature gate: BootImageSkewEnforcement Jul 16 00:39:54.518617 ip-10-0-143-200 kubenswrapper[2572]: W0716 00:39:54.181609 2572 feature_gate.go:328] unrecognized feature gate: BuildCSIVolumes Jul 16 00:39:54.518617 ip-10-0-143-200 kubenswrapper[2572]: W0716 00:39:54.181611 2572 feature_gate.go:328] unrecognized feature gate: Example Jul 16 00:39:54.518617 ip-10-0-143-200 kubenswrapper[2572]: W0716 00:39:54.181614 2572 feature_gate.go:328] unrecognized feature gate: NetworkDiagnosticsConfig Jul 16 00:39:54.518617 ip-10-0-143-200 kubenswrapper[2572]: W0716 00:39:54.181617 2572 feature_gate.go:328] unrecognized feature gate: ExternalOIDCWithUIDAndExtraClaimMappings Jul 16 00:39:54.518617 ip-10-0-143-200 kubenswrapper[2572]: W0716 00:39:54.181619 2572 feature_gate.go:328] unrecognized feature gate: OSStreams Jul 16 00:39:54.518617 ip-10-0-143-200 kubenswrapper[2572]: W0716 00:39:54.181622 2572 feature_gate.go:328] unrecognized feature gate: DualReplica Jul 16 00:39:54.519370 ip-10-0-143-200 kubenswrapper[2572]: W0716 00:39:54.181624 2572 feature_gate.go:328] unrecognized feature gate: AWSClusterHostedDNSInstall Jul 16 00:39:54.519370 ip-10-0-143-200 kubenswrapper[2572]: W0716 00:39:54.181627 2572 feature_gate.go:328] unrecognized feature gate: AdditionalRoutingCapabilities Jul 16 00:39:54.519370 ip-10-0-143-200 kubenswrapper[2572]: W0716 00:39:54.181630 2572 feature_gate.go:328] unrecognized feature gate: GCPCustomAPIEndpoints Jul 16 00:39:54.519370 ip-10-0-143-200 kubenswrapper[2572]: W0716 00:39:54.181632 2572 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesAWS Jul 16 00:39:54.519370 ip-10-0-143-200 kubenswrapper[2572]: W0716 00:39:54.181635 2572 feature_gate.go:328] unrecognized feature gate: InsightsOnDemandDataGather Jul 16 00:39:54.519370 ip-10-0-143-200 kubenswrapper[2572]: W0716 00:39:54.181638 2572 feature_gate.go:328] unrecognized feature gate: VolumeGroupSnapshot Jul 16 00:39:54.519370 ip-10-0-143-200 kubenswrapper[2572]: W0716 00:39:54.181640 2572 feature_gate.go:328] unrecognized feature gate: RouteAdvertisements Jul 16 00:39:54.519370 ip-10-0-143-200 kubenswrapper[2572]: W0716 00:39:54.181643 2572 feature_gate.go:328] unrecognized feature gate: OpenShiftPodSecurityAdmission Jul 16 00:39:54.519370 ip-10-0-143-200 kubenswrapper[2572]: W0716 00:39:54.181645 2572 feature_gate.go:328] unrecognized feature gate: NewOLMCatalogdAPIV1Metas Jul 16 00:39:54.519370 ip-10-0-143-200 kubenswrapper[2572]: W0716 00:39:54.181648 2572 feature_gate.go:328] unrecognized feature gate: AzureWorkloadIdentity Jul 16 00:39:54.519370 ip-10-0-143-200 kubenswrapper[2572]: W0716 00:39:54.181651 2572 feature_gate.go:328] unrecognized feature gate: VSphereConfigurableMaxAllowedBlockVolumesPerNode Jul 16 00:39:54.519370 ip-10-0-143-200 kubenswrapper[2572]: W0716 00:39:54.181653 2572 feature_gate.go:328] unrecognized feature gate: VSphereMixedNodeEnv Jul 16 00:39:54.519370 ip-10-0-143-200 kubenswrapper[2572]: W0716 00:39:54.181656 2572 feature_gate.go:328] unrecognized feature gate: GatewayAPIWithoutOLM Jul 16 00:39:54.519370 ip-10-0-143-200 kubenswrapper[2572]: W0716 00:39:54.181659 2572 feature_gate.go:328] unrecognized feature gate: MachineAPIOperatorDisableMachineHealthCheckController Jul 16 00:39:54.519370 ip-10-0-143-200 kubenswrapper[2572]: W0716 00:39:54.181661 2572 feature_gate.go:328] unrecognized feature gate: MetricsCollectionProfiles Jul 16 00:39:54.519370 ip-10-0-143-200 kubenswrapper[2572]: W0716 00:39:54.181664 2572 feature_gate.go:328] unrecognized feature gate: GCPDualStackInstall Jul 16 00:39:54.519370 ip-10-0-143-200 kubenswrapper[2572]: W0716 00:39:54.181666 2572 feature_gate.go:328] unrecognized feature gate: AWSDualStackInstall Jul 16 00:39:54.519370 ip-10-0-143-200 kubenswrapper[2572]: W0716 00:39:54.181669 2572 feature_gate.go:328] unrecognized feature gate: KMSEncryption Jul 16 00:39:54.519370 ip-10-0-143-200 kubenswrapper[2572]: W0716 00:39:54.181672 2572 feature_gate.go:328] unrecognized feature gate: SigstoreImageVerification Jul 16 00:39:54.519922 ip-10-0-143-200 kubenswrapper[2572]: W0716 00:39:54.181674 2572 feature_gate.go:328] unrecognized feature gate: IrreconcilableMachineConfig Jul 16 00:39:54.519922 ip-10-0-143-200 kubenswrapper[2572]: W0716 00:39:54.181677 2572 feature_gate.go:328] unrecognized feature gate: EtcdBackendQuota Jul 16 00:39:54.519922 ip-10-0-143-200 kubenswrapper[2572]: W0716 00:39:54.181679 2572 feature_gate.go:328] unrecognized feature gate: VSphereMultiDisk Jul 16 00:39:54.519922 ip-10-0-143-200 kubenswrapper[2572]: W0716 00:39:54.181685 2572 feature_gate.go:328] unrecognized feature gate: InsightsConfig Jul 16 00:39:54.519922 ip-10-0-143-200 kubenswrapper[2572]: W0716 00:39:54.181687 2572 feature_gate.go:328] unrecognized feature gate: DyanmicServiceEndpointIBMCloud Jul 16 00:39:54.519922 ip-10-0-143-200 kubenswrapper[2572]: W0716 00:39:54.181690 2572 feature_gate.go:328] unrecognized feature gate: AzureDualStackInstall Jul 16 00:39:54.519922 ip-10-0-143-200 kubenswrapper[2572]: W0716 00:39:54.181693 2572 feature_gate.go:328] unrecognized feature gate: MixedCPUsAllocation Jul 16 00:39:54.519922 ip-10-0-143-200 kubenswrapper[2572]: W0716 00:39:54.181695 2572 feature_gate.go:328] unrecognized feature gate: GCPClusterHostedDNSInstall Jul 16 00:39:54.519922 ip-10-0-143-200 kubenswrapper[2572]: W0716 00:39:54.181698 2572 feature_gate.go:328] unrecognized feature gate: AWSServiceLBNetworkSecurityGroup Jul 16 00:39:54.519922 ip-10-0-143-200 kubenswrapper[2572]: W0716 00:39:54.181700 2572 feature_gate.go:328] unrecognized feature gate: VSphereMultiNetworks Jul 16 00:39:54.519922 ip-10-0-143-200 kubenswrapper[2572]: W0716 00:39:54.181703 2572 feature_gate.go:328] unrecognized feature gate: NoRegistryClusterInstall Jul 16 00:39:54.519922 ip-10-0-143-200 kubenswrapper[2572]: W0716 00:39:54.181706 2572 feature_gate.go:328] unrecognized feature gate: HighlyAvailableArbiter Jul 16 00:39:54.519922 ip-10-0-143-200 kubenswrapper[2572]: W0716 00:39:54.181709 2572 feature_gate.go:351] Setting GA feature gate ServiceAccountTokenNodeBinding=true. It will be removed in a future release. Jul 16 00:39:54.519922 ip-10-0-143-200 kubenswrapper[2572]: W0716 00:39:54.181712 2572 feature_gate.go:328] unrecognized feature gate: NutanixMultiSubnets Jul 16 00:39:54.519922 ip-10-0-143-200 kubenswrapper[2572]: W0716 00:39:54.181715 2572 feature_gate.go:328] unrecognized feature gate: ImageStreamImportMode Jul 16 00:39:54.519922 ip-10-0-143-200 kubenswrapper[2572]: W0716 00:39:54.181718 2572 feature_gate.go:328] unrecognized feature gate: OVNObservability Jul 16 00:39:54.519922 ip-10-0-143-200 kubenswrapper[2572]: W0716 00:39:54.181723 2572 feature_gate.go:328] unrecognized feature gate: GatewayAPI Jul 16 00:39:54.519922 ip-10-0-143-200 kubenswrapper[2572]: W0716 00:39:54.181726 2572 feature_gate.go:328] unrecognized feature gate: AzureClusterHostedDNSInstall Jul 16 00:39:54.519922 ip-10-0-143-200 kubenswrapper[2572]: W0716 00:39:54.181729 2572 feature_gate.go:328] unrecognized feature gate: ClusterAPIInstallIBMCloud Jul 16 00:39:54.519922 ip-10-0-143-200 kubenswrapper[2572]: W0716 00:39:54.181731 2572 feature_gate.go:328] unrecognized feature gate: AlibabaPlatform Jul 16 00:39:55.805393 ip-10-0-143-200 kubenswrapper[2572]: W0716 00:39:54.181734 2572 feature_gate.go:328] unrecognized feature gate: NetworkSegmentation Jul 16 00:39:55.805393 ip-10-0-143-200 kubenswrapper[2572]: W0716 00:39:54.181737 2572 feature_gate.go:328] unrecognized feature gate: ManagedBootImages Jul 16 00:39:55.805393 ip-10-0-143-200 kubenswrapper[2572]: W0716 00:39:54.181740 2572 feature_gate.go:328] unrecognized feature gate: HyperShiftOnlyDynamicResourceAllocation Jul 16 00:39:55.805393 ip-10-0-143-200 kubenswrapper[2572]: W0716 00:39:54.181742 2572 feature_gate.go:328] unrecognized feature gate: AzureDedicatedHosts Jul 16 00:39:55.805393 ip-10-0-143-200 kubenswrapper[2572]: W0716 00:39:54.181745 2572 feature_gate.go:328] unrecognized feature gate: ExternalSnapshotMetadata Jul 16 00:39:55.805393 ip-10-0-143-200 kubenswrapper[2572]: W0716 00:39:54.181748 2572 feature_gate.go:328] unrecognized feature gate: ClusterVersionOperatorConfiguration Jul 16 00:39:55.805393 ip-10-0-143-200 kubenswrapper[2572]: W0716 00:39:54.181751 2572 feature_gate.go:328] unrecognized feature gate: KMSEncryptionProvider Jul 16 00:39:55.805393 ip-10-0-143-200 kubenswrapper[2572]: W0716 00:39:54.181754 2572 feature_gate.go:328] unrecognized feature gate: ExternalOIDC Jul 16 00:39:55.805393 ip-10-0-143-200 kubenswrapper[2572]: W0716 00:39:54.181757 2572 feature_gate.go:328] unrecognized feature gate: PreconfiguredUDNAddresses Jul 16 00:39:55.805393 ip-10-0-143-200 kubenswrapper[2572]: W0716 00:39:54.181760 2572 feature_gate.go:328] unrecognized feature gate: ClusterAPIInstall Jul 16 00:39:55.805393 ip-10-0-143-200 kubenswrapper[2572]: W0716 00:39:54.181762 2572 feature_gate.go:328] unrecognized feature gate: AWSDedicatedHosts Jul 16 00:39:55.805393 ip-10-0-143-200 kubenswrapper[2572]: W0716 00:39:54.181765 2572 feature_gate.go:328] unrecognized feature gate: PinnedImages Jul 16 00:39:55.805393 ip-10-0-143-200 kubenswrapper[2572]: W0716 00:39:54.181767 2572 feature_gate.go:328] unrecognized feature gate: MultiArchInstallAzure Jul 16 00:39:55.805393 ip-10-0-143-200 kubenswrapper[2572]: W0716 00:39:54.181770 2572 feature_gate.go:328] unrecognized feature gate: DNSNameResolver Jul 16 00:39:55.805393 ip-10-0-143-200 kubenswrapper[2572]: W0716 00:39:54.181772 2572 feature_gate.go:328] unrecognized feature gate: MachineConfigNodes Jul 16 00:39:55.805393 ip-10-0-143-200 kubenswrapper[2572]: W0716 00:39:54.181776 2572 feature_gate.go:328] unrecognized feature gate: AWSClusterHostedDNS Jul 16 00:39:55.805393 ip-10-0-143-200 kubenswrapper[2572]: W0716 00:39:54.181779 2572 feature_gate.go:328] unrecognized feature gate: ProvisioningRequestAvailable Jul 16 00:39:55.805393 ip-10-0-143-200 kubenswrapper[2572]: W0716 00:39:54.181781 2572 feature_gate.go:328] unrecognized feature gate: MachineAPIMigration Jul 16 00:39:55.805393 ip-10-0-143-200 kubenswrapper[2572]: W0716 00:39:54.181790 2572 feature_gate.go:349] Setting deprecated feature gate KMSv1=true. It will be removed in a future release. Jul 16 00:39:55.805393 ip-10-0-143-200 kubenswrapper[2572]: W0716 00:39:54.181794 2572 feature_gate.go:328] unrecognized feature gate: AzureMultiDisk Jul 16 00:39:55.806390 ip-10-0-143-200 kubenswrapper[2572]: W0716 00:39:54.181797 2572 feature_gate.go:328] unrecognized feature gate: ClusterAPIMachineManagement Jul 16 00:39:55.806390 ip-10-0-143-200 kubenswrapper[2572]: W0716 00:39:54.181801 2572 feature_gate.go:351] Setting GA feature gate VolumeAttributesClass=true. It will be removed in a future release. Jul 16 00:39:55.806390 ip-10-0-143-200 kubenswrapper[2572]: W0716 00:39:54.181805 2572 feature_gate.go:328] unrecognized feature gate: MultiDiskSetup Jul 16 00:39:55.806390 ip-10-0-143-200 kubenswrapper[2572]: W0716 00:39:54.181807 2572 feature_gate.go:328] unrecognized feature gate: ClusterAPIMachineManagementVSphere Jul 16 00:39:55.806390 ip-10-0-143-200 kubenswrapper[2572]: W0716 00:39:54.181811 2572 feature_gate.go:328] unrecognized feature gate: VSphereHostVMGroupZonal Jul 16 00:39:55.806390 ip-10-0-143-200 kubenswrapper[2572]: W0716 00:39:54.181813 2572 feature_gate.go:328] unrecognized feature gate: GCPCustomAPIEndpointsInstall Jul 16 00:39:55.806390 ip-10-0-143-200 kubenswrapper[2572]: W0716 00:39:54.181816 2572 feature_gate.go:328] unrecognized feature gate: NewOLMWebhookProviderOpenshiftServiceCA Jul 16 00:39:55.806390 ip-10-0-143-200 kubenswrapper[2572]: W0716 00:39:54.181819 2572 feature_gate.go:328] unrecognized feature gate: NewOLMPreflightPermissionChecks Jul 16 00:39:55.806390 ip-10-0-143-200 kubenswrapper[2572]: W0716 00:39:54.181824 2572 feature_gate.go:328] unrecognized feature gate: CPMSMachineNamePrefix Jul 16 00:39:55.806390 ip-10-0-143-200 kubenswrapper[2572]: W0716 00:39:54.181827 2572 feature_gate.go:328] unrecognized feature gate: Example2 Jul 16 00:39:55.806390 ip-10-0-143-200 kubenswrapper[2572]: W0716 00:39:54.181829 2572 feature_gate.go:328] unrecognized feature gate: ShortCertRotation Jul 16 00:39:55.806390 ip-10-0-143-200 kubenswrapper[2572]: W0716 00:39:54.181832 2572 feature_gate.go:328] unrecognized feature gate: NetworkLiveMigration Jul 16 00:39:55.806390 ip-10-0-143-200 kubenswrapper[2572]: W0716 00:39:54.181835 2572 feature_gate.go:328] unrecognized feature gate: GatewayAPIController Jul 16 00:39:55.806390 ip-10-0-143-200 kubenswrapper[2572]: W0716 00:39:54.181838 2572 feature_gate.go:328] unrecognized feature gate: GCPClusterHostedDNS Jul 16 00:39:55.806390 ip-10-0-143-200 kubenswrapper[2572]: W0716 00:39:54.181840 2572 feature_gate.go:328] unrecognized feature gate: ImageModeStatusReporting Jul 16 00:39:55.806390 ip-10-0-143-200 kubenswrapper[2572]: W0716 00:39:54.181843 2572 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesCPMS Jul 16 00:39:55.806390 ip-10-0-143-200 kubenswrapper[2572]: W0716 00:39:54.181846 2572 feature_gate.go:328] unrecognized feature gate: NewOLMOwnSingleNamespace Jul 16 00:39:55.806390 ip-10-0-143-200 kubenswrapper[2572]: W0716 00:39:54.181848 2572 feature_gate.go:328] unrecognized feature gate: CRDCompatibilityRequirementOperator Jul 16 00:39:55.806390 ip-10-0-143-200 kubenswrapper[2572]: W0716 00:39:54.181851 2572 feature_gate.go:328] unrecognized feature gate: AdminNetworkPolicy Jul 16 00:39:55.807265 ip-10-0-143-200 kubenswrapper[2572]: W0716 00:39:54.181854 2572 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesvSphere Jul 16 00:39:55.807265 ip-10-0-143-200 kubenswrapper[2572]: I0716 00:39:54.181859 2572 feature_gate.go:384] feature gates: {map[CBORServingAndStorage:false ClientsAllowCBOR:false ClientsPreferCBOR:false EventedPLEG:false ImageVolume:true KMSv1:true MaxUnavailableStatefulSet:false MinimumKubeletVersion:false MutableCSINodeAllocatableCount:false MutatingAdmissionPolicy:false ProcMountType:true RouteExternalCertificate:true SELinuxMount:false ServiceAccountTokenNodeBinding:true StoragePerformantSecurityPolicy:true TranslateStreamCloseWebsocketRequests:false UserNamespacesPodSecurityStandards:true UserNamespacesSupport:true VolumeAttributesClass:true]} Jul 16 00:39:55.807265 ip-10-0-143-200 kubenswrapper[2572]: I0716 00:39:54.181970 2572 server.go:962] "Client rotation is on, will bootstrap in background" Jul 16 00:39:55.807265 ip-10-0-143-200 kubenswrapper[2572]: I0716 00:39:54.184371 2572 bootstrap.go:101] "Use the bootstrap credentials to request a cert, and set kubeconfig to point to the certificate dir" Jul 16 00:39:55.807265 ip-10-0-143-200 kubenswrapper[2572]: I0716 00:39:54.185980 2572 server.go:1019] "Starting client certificate rotation" Jul 16 00:39:55.807265 ip-10-0-143-200 kubenswrapper[2572]: I0716 00:39:54.186082 2572 certificate_manager.go:422] "Certificate rotation is enabled" logger="kubernetes.io/kube-apiserver-client-kubelet" Jul 16 00:39:55.807265 ip-10-0-143-200 kubenswrapper[2572]: I0716 00:39:54.186128 2572 certificate_manager.go:566] "Rotating certificates" logger="kubernetes.io/kube-apiserver-client-kubelet" Jul 16 00:39:55.807265 ip-10-0-143-200 kubenswrapper[2572]: I0716 00:39:54.191348 2572 dynamic_cafile_content.go:123] "Loaded a new CA Bundle and Verifier" name="client-ca-bundle::/etc/kubernetes/kubelet-ca.crt" Jul 16 00:39:55.807265 ip-10-0-143-200 kubenswrapper[2572]: I0716 00:39:54.195684 2572 dynamic_cafile_content.go:161] "Starting controller" name="client-ca-bundle::/etc/kubernetes/kubelet-ca.crt" Jul 16 00:39:55.807265 ip-10-0-143-200 kubenswrapper[2572]: I0716 00:39:54.200898 2572 log.go:25] "Validated CRI v1 runtime API" Jul 16 00:39:55.807265 ip-10-0-143-200 kubenswrapper[2572]: I0716 00:39:54.203563 2572 log.go:25] "Validated CRI v1 image API" Jul 16 00:39:55.807265 ip-10-0-143-200 kubenswrapper[2572]: I0716 00:39:54.205287 2572 server.go:1429] "Using cgroup driver setting received from the CRI runtime" cgroupDriver="systemd" Jul 16 00:39:55.807719 ip-10-0-143-200 kubenswrapper[2572]: I0716 00:39:54.209392 2572 fs.go:135] Filesystem UUIDs: map[7B77-95E7:/dev/nvme0n1p2 8a712ef9-87c8-494d-a509-ba32b54bcb45:/dev/nvme0n1p3 9bf884b8-ef96-4ee3-aa87-c4128b0345b4:/dev/nvme0n1p4] Jul 16 00:39:55.807719 ip-10-0-143-200 kubenswrapper[2572]: I0716 00:39:54.209408 2572 fs.go:136] Filesystem partitions: map[/dev/nvme0n1p3:{mountpoint:/boot major:259 minor:3 fsType:ext4 blockSize:0} /dev/nvme0n1p4:{mountpoint:/var major:259 minor:4 fsType:xfs blockSize:0} /dev/shm:{mountpoint:/dev/shm major:0 minor:22 fsType:tmpfs blockSize:0} /run:{mountpoint:/run major:0 minor:24 fsType:tmpfs blockSize:0} /tmp:{mountpoint:/tmp major:0 minor:31 fsType:tmpfs blockSize:0} composefs_0-33:{mountpoint:/ major:0 minor:33 fsType:overlay blockSize:0}] Jul 16 00:39:55.808260 ip-10-0-143-200 kubenswrapper[2572]: I0716 00:39:54.215333 2572 manager.go:216] Machine: {Timestamp:2026-07-16 00:39:54.213829483 +0000 UTC m=+0.378999593 CPUVendorID:GenuineIntel NumCores:8 NumPhysicalCores:4 NumSockets:1 CpuFrequency:3100264 MemoryCapacity:33164496896 SwapCapacity:0 MemoryByType:map[] NVMInfo:{MemoryModeCapacity:0 AppDirectModeCapacity:0 AvgPowerBudget:0} HugePages:[{PageSize:1048576 NumPages:0} {PageSize:2048 NumPages:0}] MachineID:ec20c0c3db60c9b90bcf705e90cd780d SystemUUID:ec20c0c3-db60-c9b9-0bcf-705e90cd780d BootID:54d3da2f-a852-41af-9ace-16b5da692a63 Filesystems:[{Device:/dev/shm DeviceMajor:0 DeviceMinor:22 Capacity:16582246400 Type:vfs Inodes:4048400 HasInodes:true} {Device:/run DeviceMajor:0 DeviceMinor:24 Capacity:6632902656 Type:vfs Inodes:819200 HasInodes:true} {Device:/dev/nvme0n1p4 DeviceMajor:259 DeviceMinor:4 Capacity:128243970048 Type:vfs Inodes:62651840 HasInodes:true} {Device:composefs_0-33 DeviceMajor:0 DeviceMinor:33 Capacity:6107136 Type:vfs Inodes:18446744073709551615 HasInodes:true} {Device:/tmp DeviceMajor:0 DeviceMinor:31 Capacity:16582250496 Type:vfs Inodes:1048576 HasInodes:true} {Device:/dev/nvme0n1p3 DeviceMajor:259 DeviceMinor:3 Capacity:366869504 Type:vfs Inodes:98304 HasInodes:true}] DiskMap:map[259:0:{Name:nvme0n1 Major:259 Minor:0 Size:128849018880 Scheduler:none}] NetworkDevices:[{Name:br-ex MacAddress:02:ab:41:2e:65:c7 Speed:0 Mtu:9001} {Name:ens5 MacAddress:02:ab:41:2e:65:c7 Speed:0 Mtu:9001} {Name:ovs-system MacAddress:de:41:47:c7:bd:bb Speed:0 Mtu:1500}] Topology:[{Id:0 Memory:33164496896 HugePages:[{PageSize:1048576 NumPages:0} {PageSize:2048 NumPages:0}] Cores:[{Id:0 Threads:[0 4] Caches:[{Id:0 Size:32768 Type:Data Level:1} {Id:0 Size:32768 Type:Instruction Level:1} {Id:0 Size:1048576 Type:Unified Level:2}] UncoreCaches:[] SocketID:0 BookID: DrawerID:} {Id:1 Threads:[1 5] Caches:[{Id:1 Size:32768 Type:Data Level:1} {Id:1 Size:32768 Type:Instruction Level:1} {Id:1 Size:1048576 Type:Unified Level:2}] UncoreCaches:[] SocketID:0 BookID: DrawerID:} {Id:2 Threads:[2 6] Caches:[{Id:2 Size:32768 Type:Data Level:1} {Id:2 Size:32768 Type:Instruction Level:1} {Id:2 Size:1048576 Type:Unified Level:2}] UncoreCaches:[] SocketID:0 BookID: DrawerID:} {Id:3 Threads:[3 7] Caches:[{Id:3 Size:32768 Type:Data Level:1} {Id:3 Size:32768 Type:Instruction Level:1} {Id:3 Size:1048576 Type:Unified Level:2}] UncoreCaches:[] SocketID:0 BookID: DrawerID:}] Caches:[{Id:0 Size:37486592 Type:Unified Level:3}] Distances:[10]}] CloudProvider:Unknown InstanceType:Unknown InstanceID:None} Jul 16 00:39:55.808260 ip-10-0-143-200 kubenswrapper[2572]: I0716 00:39:54.215438 2572 manager_no_libpfm.go:29] cAdvisor is build without cgo and/or libpfm support. Perf event counters are not available. Jul 16 00:39:55.808260 ip-10-0-143-200 kubenswrapper[2572]: I0716 00:39:54.215516 2572 manager.go:232] Version: {KernelVersion:5.14.0-570.126.1.el9_6.x86_64 ContainerOsVersion:Red Hat Enterprise Linux CoreOS 9.6.20260702-0 (Plow) DockerVersion: DockerAPIVersion: CadvisorVersion: CadvisorRevision:} Jul 16 00:39:55.808260 ip-10-0-143-200 kubenswrapper[2572]: I0716 00:39:54.218817 2572 container_manager_linux.go:273] "Container manager verified user specified cgroup-root exists" cgroupRoot=[] Jul 16 00:39:55.808260 ip-10-0-143-200 kubenswrapper[2572]: I0716 00:39:54.218844 2572 container_manager_linux.go:278] "Creating Container Manager object based on Node Config" nodeConfig={"NodeName":"ip-10-0-143-200.ec2.internal","RuntimeCgroupsName":"/system.slice/crio.service","SystemCgroupsName":"/system.slice","KubeletCgroupsName":"","KubeletOOMScoreAdj":-999,"ContainerRuntime":"","CgroupsPerQOS":true,"CgroupRoot":"/","CgroupDriver":"systemd","KubeletRootDir":"/var/lib/kubelet","ProtectKernelDefaults":true,"KubeReservedCgroupName":"","SystemReservedCgroupName":"","ReservedSystemCPUs":{},"EnforceNodeAllocatable":{"pods":{}},"KubeReserved":null,"SystemReserved":{"cpu":"500m","ephemeral-storage":"1Gi","memory":"1Gi"},"HardEvictionThresholds":[{"Signal":"memory.available","Operator":"LessThan","Value":{"Quantity":"100Mi","Percentage":0},"GracePeriod":0,"MinReclaim":null},{"Signal":"nodefs.available","Operator":"LessThan","Value":{"Quantity":null,"Percentage":0.1},"GracePeriod":0,"MinReclaim":null},{"Signal":"nodefs.inodesFree","Operator":"LessThan","Value":{"Quantity":null,"Percentage":0.05},"GracePeriod":0,"MinReclaim":null},{"Signal":"imagefs.available","Operator":"LessThan","Value":{"Quantity":null,"Percentage":0.15},"GracePeriod":0,"MinReclaim":null},{"Signal":"imagefs.inodesFree","Operator":"LessThan","Value":{"Quantity":null,"Percentage":0.05},"GracePeriod":0,"MinReclaim":null}],"QOSReserved":{},"CPUManagerPolicy":"none","CPUManagerPolicyOptions":null,"TopologyManagerScope":"container","CPUManagerReconcilePeriod":10000000000,"MemoryManagerPolicy":"None","MemoryManagerReservedMemory":null,"PodPidsLimit":4096,"EnforceCPULimits":true,"CPUCFSQuotaPeriod":100000000,"TopologyManagerPolicy":"none","TopologyManagerPolicyOptions":null,"CgroupVersion":2} Jul 16 00:39:55.808260 ip-10-0-143-200 kubenswrapper[2572]: I0716 00:39:54.219800 2572 topology_manager.go:138] "Creating topology manager with none policy" Jul 16 00:39:55.808260 ip-10-0-143-200 kubenswrapper[2572]: I0716 00:39:54.219811 2572 container_manager_linux.go:309] "Creating device plugin manager" Jul 16 00:39:55.808260 ip-10-0-143-200 kubenswrapper[2572]: I0716 00:39:54.219819 2572 manager.go:141] "Creating Device Plugin manager" path="/var/lib/kubelet/device-plugins/kubelet.sock" Jul 16 00:39:55.808260 ip-10-0-143-200 kubenswrapper[2572]: I0716 00:39:54.219829 2572 server.go:74] "Creating device plugin registration server" version="v1beta1" socket="/var/lib/kubelet/device-plugins/kubelet.sock" Jul 16 00:39:55.808260 ip-10-0-143-200 kubenswrapper[2572]: I0716 00:39:54.221788 2572 container_manager_linux.go:318] "Creating Dynamic Resource Allocation (DRA) manager" Jul 16 00:39:55.808260 ip-10-0-143-200 kubenswrapper[2572]: I0716 00:39:54.224943 2572 reflector.go:439] "Caches populated" logger="kubernetes.io/kube-apiserver-client-kubelet" type="*v1.CertificateSigningRequest" reflector="k8s.io/client-go/tools/watch/informerwatcher.go:162" Jul 16 00:39:55.808260 ip-10-0-143-200 kubenswrapper[2572]: I0716 00:39:54.225586 2572 state_mem.go:36] "Initialized new in-memory state store" Jul 16 00:39:55.808260 ip-10-0-143-200 kubenswrapper[2572]: I0716 00:39:54.225712 2572 server.go:1240] "Using root directory" path="/var/lib/kubelet" Jul 16 00:39:55.808260 ip-10-0-143-200 kubenswrapper[2572]: I0716 00:39:54.225742 2572 kubelet.go:486] "Attempting to sync node with API server" Jul 16 00:39:55.808260 ip-10-0-143-200 kubenswrapper[2572]: I0716 00:39:54.225751 2572 kubelet.go:387] "Adding static pod path" path="/etc/kubernetes/manifests" Jul 16 00:39:55.808260 ip-10-0-143-200 kubenswrapper[2572]: I0716 00:39:54.225764 2572 file.go:69] "Watching path" path="/etc/kubernetes/manifests" Jul 16 00:39:55.808260 ip-10-0-143-200 kubenswrapper[2572]: I0716 00:39:54.225772 2572 kubelet.go:398] "Adding apiserver pod source" Jul 16 00:39:55.808837 ip-10-0-143-200 kubenswrapper[2572]: I0716 00:39:54.225781 2572 apiserver.go:42] "Waiting for node sync before watching apiserver pods" Jul 16 00:39:55.808837 ip-10-0-143-200 kubenswrapper[2572]: I0716 00:39:54.226144 2572 state_mem.go:40] "Initialized new in-memory state store for pod resource information tracking" Jul 16 00:39:55.808837 ip-10-0-143-200 kubenswrapper[2572]: I0716 00:39:54.226159 2572 state_mem.go:40] "Initialized new in-memory state store for pod resource information tracking" Jul 16 00:39:55.808837 ip-10-0-143-200 kubenswrapper[2572]: I0716 00:39:54.229982 2572 kuberuntime_manager.go:301] "Container runtime initialized" containerRuntime="cri-o" version="1.34.10-2.rhaos4.21.git83ee25b.el9" apiVersion="v1" Jul 16 00:39:55.808837 ip-10-0-143-200 kubenswrapper[2572]: I0716 00:39:54.233456 2572 kubelet.go:960] "Not starting ClusterTrustBundle informer because we are in static kubelet mode or the ClusterTrustBundleProjection featuregate is disabled" Jul 16 00:39:55.808837 ip-10-0-143-200 kubenswrapper[2572]: I0716 00:39:54.233497 2572 kubelet.go:984] "Not starting PodCertificateRequest manager because we are in static kubelet mode or the PodCertificateProjection feature gate is disabled" Jul 16 00:39:55.808837 ip-10-0-143-200 kubenswrapper[2572]: E0716 00:39:54.236481 2572 reflector.go:205] "Failed to watch" err="failed to list *v1.Node: nodes \"ip-10-0-143-200.ec2.internal\" is forbidden: User \"system:anonymous\" cannot list resource \"nodes\" in API group \"\" at the cluster scope" logger="UnhandledError" reflector="k8s.io/client-go/informers/factory.go:160" type="*v1.Node" Jul 16 00:39:55.808837 ip-10-0-143-200 kubenswrapper[2572]: E0716 00:39:54.236481 2572 reflector.go:205] "Failed to watch" err="failed to list *v1.Service: services is forbidden: User \"system:anonymous\" cannot list resource \"services\" in API group \"\" at the cluster scope" logger="UnhandledError" reflector="k8s.io/client-go/informers/factory.go:160" type="*v1.Service" Jul 16 00:39:55.808837 ip-10-0-143-200 kubenswrapper[2572]: I0716 00:39:54.237016 2572 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/portworx-volume" Jul 16 00:39:55.808837 ip-10-0-143-200 kubenswrapper[2572]: I0716 00:39:54.237037 2572 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/empty-dir" Jul 16 00:39:55.808837 ip-10-0-143-200 kubenswrapper[2572]: I0716 00:39:54.237043 2572 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/git-repo" Jul 16 00:39:55.808837 ip-10-0-143-200 kubenswrapper[2572]: I0716 00:39:54.237049 2572 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/host-path" Jul 16 00:39:55.808837 ip-10-0-143-200 kubenswrapper[2572]: I0716 00:39:54.237058 2572 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/nfs" Jul 16 00:39:55.808837 ip-10-0-143-200 kubenswrapper[2572]: I0716 00:39:54.237064 2572 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/secret" Jul 16 00:39:55.808837 ip-10-0-143-200 kubenswrapper[2572]: I0716 00:39:54.237072 2572 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/iscsi" Jul 16 00:39:55.808837 ip-10-0-143-200 kubenswrapper[2572]: I0716 00:39:54.237078 2572 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/downward-api" Jul 16 00:39:55.808837 ip-10-0-143-200 kubenswrapper[2572]: I0716 00:39:54.237085 2572 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/fc" Jul 16 00:39:55.808837 ip-10-0-143-200 kubenswrapper[2572]: I0716 00:39:54.237091 2572 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/configmap" Jul 16 00:39:55.808837 ip-10-0-143-200 kubenswrapper[2572]: I0716 00:39:54.237099 2572 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/projected" Jul 16 00:39:55.808837 ip-10-0-143-200 kubenswrapper[2572]: I0716 00:39:54.237123 2572 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/local-volume" Jul 16 00:39:55.808837 ip-10-0-143-200 kubenswrapper[2572]: I0716 00:39:54.238114 2572 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/csi" Jul 16 00:39:55.808837 ip-10-0-143-200 kubenswrapper[2572]: I0716 00:39:54.238122 2572 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/image" Jul 16 00:39:55.808837 ip-10-0-143-200 kubenswrapper[2572]: I0716 00:39:54.241130 2572 server.go:1268] "Started kubelet" Jul 16 00:39:55.808837 ip-10-0-143-200 kubenswrapper[2572]: I0716 00:39:54.241207 2572 server.go:180] "Starting to listen" address="0.0.0.0" port=10250 Jul 16 00:39:55.808837 ip-10-0-143-200 kubenswrapper[2572]: I0716 00:39:54.241231 2572 ratelimit.go:56] "Setting rate limiting for endpoint" service="podresources" qps=100 burstTokens=10 Jul 16 00:39:55.808837 ip-10-0-143-200 kubenswrapper[2572]: I0716 00:39:54.241307 2572 server_v1.go:49] "podresources" method="list" useActivePods=true Jul 16 00:39:55.808837 ip-10-0-143-200 kubenswrapper[2572]: I0716 00:39:54.242890 2572 server.go:249] "Starting to serve the podresources API" endpoint="unix:/var/lib/kubelet/pod-resources/kubelet.sock" Jul 16 00:39:55.808837 ip-10-0-143-200 kubenswrapper[2572]: I0716 00:39:54.243198 2572 server.go:310] "Adding debug handlers to kubelet server" Jul 16 00:39:55.808837 ip-10-0-143-200 kubenswrapper[2572]: E0716 00:39:54.247546 2572 kubelet.go:1638] "Image garbage collection failed once. Stats initialization may not have completed yet" err="failed to get imageFs info: unable to find data in memory cache" Jul 16 00:39:55.808837 ip-10-0-143-200 kubenswrapper[2572]: I0716 00:39:54.250122 2572 fs_resource_analyzer.go:67] "Starting FS ResourceAnalyzer" Jul 16 00:39:55.808837 ip-10-0-143-200 kubenswrapper[2572]: I0716 00:39:54.250301 2572 certificate_manager.go:422] "Certificate rotation is enabled" logger="kubernetes.io/kubelet-serving" Jul 16 00:39:55.808837 ip-10-0-143-200 kubenswrapper[2572]: E0716 00:39:54.250570 2572 kubelet_node_status.go:454] "Error getting the current node from lister" err="node \"ip-10-0-143-200.ec2.internal\" not found" Jul 16 00:39:55.808837 ip-10-0-143-200 kubenswrapper[2572]: I0716 00:39:54.251243 2572 volume_manager.go:311] "The desired_state_of_world populator starts" Jul 16 00:39:55.808837 ip-10-0-143-200 kubenswrapper[2572]: I0716 00:39:54.251262 2572 volume_manager.go:313] "Starting Kubelet Volume Manager" Jul 16 00:39:55.808837 ip-10-0-143-200 kubenswrapper[2572]: I0716 00:39:54.251473 2572 desired_state_of_world_populator.go:146] "Desired state populator starts to run" Jul 16 00:39:55.808837 ip-10-0-143-200 kubenswrapper[2572]: I0716 00:39:54.251535 2572 reconstruct.go:97] "Volume reconstruction finished" Jul 16 00:39:55.808837 ip-10-0-143-200 kubenswrapper[2572]: I0716 00:39:54.251542 2572 reconciler.go:29] "Reconciler: start to sync state" Jul 16 00:39:55.808837 ip-10-0-143-200 kubenswrapper[2572]: I0716 00:39:54.252580 2572 factory.go:153] Registering CRI-O factory Jul 16 00:39:55.808837 ip-10-0-143-200 kubenswrapper[2572]: I0716 00:39:54.252594 2572 factory.go:223] Registration of the crio container factory successfully Jul 16 00:39:55.808837 ip-10-0-143-200 kubenswrapper[2572]: I0716 00:39:54.252638 2572 factory.go:221] Registration of the containerd container factory failed: unable to create containerd client: containerd: cannot unix dial containerd api service: dial unix /run/containerd/containerd.sock: connect: no such file or directory Jul 16 00:39:55.808837 ip-10-0-143-200 kubenswrapper[2572]: I0716 00:39:54.252645 2572 factory.go:55] Registering systemd factory Jul 16 00:39:55.808837 ip-10-0-143-200 kubenswrapper[2572]: I0716 00:39:54.252650 2572 factory.go:223] Registration of the systemd container factory successfully Jul 16 00:39:55.808837 ip-10-0-143-200 kubenswrapper[2572]: I0716 00:39:54.252668 2572 factory.go:103] Registering Raw factory Jul 16 00:39:55.808837 ip-10-0-143-200 kubenswrapper[2572]: I0716 00:39:54.252677 2572 manager.go:1196] Started watching for new ooms in manager Jul 16 00:39:55.808837 ip-10-0-143-200 kubenswrapper[2572]: I0716 00:39:54.252960 2572 manager.go:349] Starting recovery of all containers Jul 16 00:39:55.808837 ip-10-0-143-200 kubenswrapper[2572]: E0716 00:39:54.260707 2572 reflector.go:205] "Failed to watch" err="failed to list *v1.CSIDriver: csidrivers.storage.k8s.io is forbidden: User \"system:anonymous\" cannot list resource \"csidrivers\" in API group \"storage.k8s.io\" at the cluster scope" logger="UnhandledError" reflector="k8s.io/client-go/informers/factory.go:160" type="*v1.CSIDriver" Jul 16 00:39:55.810357 ip-10-0-143-200 kubenswrapper[2572]: I0716 00:39:54.260786 2572 csi_plugin.go:990] Failed to contact API server when waiting for CSINode publishing: csinodes.storage.k8s.io "ip-10-0-143-200.ec2.internal" is forbidden: User "system:anonymous" cannot get resource "csinodes" in API group "storage.k8s.io" at the cluster scope Jul 16 00:39:55.810357 ip-10-0-143-200 kubenswrapper[2572]: I0716 00:39:54.268388 2572 manager.go:354] Recovery completed Jul 16 00:39:55.810357 ip-10-0-143-200 kubenswrapper[2572]: E0716 00:39:54.276062 2572 controller.go:145] "Failed to ensure lease exists, will retry" err="leases.coordination.k8s.io \"ip-10-0-143-200.ec2.internal\" is forbidden: User \"system:anonymous\" cannot get resource \"leases\" in API group \"coordination.k8s.io\" in the namespace \"kube-node-lease\"" interval="200ms" Jul 16 00:39:55.810357 ip-10-0-143-200 kubenswrapper[2572]: I0716 00:39:54.276919 2572 csr.go:274] "Certificate signing request is approved, waiting to be issued" logger="kubernetes.io/kube-apiserver-client-kubelet" csr="csr-82gc5" Jul 16 00:39:55.810357 ip-10-0-143-200 kubenswrapper[2572]: E0716 00:39:54.260875 2572 event.go:359] "Server rejected event (will not retry!)" err="events is forbidden: User \"system:anonymous\" cannot create resource \"events\" in API group \"\" in the namespace \"default\"" event="&Event{ObjectMeta:{ip-10-0-143-200.ec2.internal.18c29e3ea259f46a default 0 0001-01-01 00:00:00 +0000 UTC map[] map[] [] [] []},InvolvedObject:ObjectReference{Kind:Node,Namespace:,Name:ip-10-0-143-200.ec2.internal,UID:ip-10-0-143-200.ec2.internal,APIVersion:,ResourceVersion:,FieldPath:,},Reason:Starting,Message:Starting kubelet.,Source:EventSource{Component:kubelet,Host:ip-10-0-143-200.ec2.internal,},FirstTimestamp:2026-07-16 00:39:54.241102954 +0000 UTC m=+0.406273059,LastTimestamp:2026-07-16 00:39:54.241102954 +0000 UTC m=+0.406273059,Count:1,Type:Normal,EventTime:0001-01-01 00:00:00 +0000 UTC,Series:nil,Action:,Related:nil,ReportingController:kubelet,ReportingInstance:ip-10-0-143-200.ec2.internal,}" Jul 16 00:39:55.810357 ip-10-0-143-200 kubenswrapper[2572]: I0716 00:39:54.278197 2572 csr.go:270] "Certificate signing request is issued" logger="kubernetes.io/kube-apiserver-client-kubelet" csr="csr-82gc5" Jul 16 00:39:55.810357 ip-10-0-143-200 kubenswrapper[2572]: I0716 00:39:54.280549 2572 kubelet_node_status.go:400] "Setting node annotation to enable volume controller attach/detach" Jul 16 00:39:55.810357 ip-10-0-143-200 kubenswrapper[2572]: I0716 00:39:54.282560 2572 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-143-200.ec2.internal" event="NodeHasSufficientMemory" Jul 16 00:39:55.810357 ip-10-0-143-200 kubenswrapper[2572]: I0716 00:39:54.282582 2572 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-143-200.ec2.internal" event="NodeHasNoDiskPressure" Jul 16 00:39:55.810357 ip-10-0-143-200 kubenswrapper[2572]: I0716 00:39:54.282592 2572 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-143-200.ec2.internal" event="NodeHasSufficientPID" Jul 16 00:39:55.810357 ip-10-0-143-200 kubenswrapper[2572]: I0716 00:39:54.283082 2572 cpu_manager.go:222] "Starting CPU manager" policy="none" Jul 16 00:39:55.810357 ip-10-0-143-200 kubenswrapper[2572]: I0716 00:39:54.283089 2572 cpu_manager.go:223] "Reconciling" reconcilePeriod="10s" Jul 16 00:39:55.810357 ip-10-0-143-200 kubenswrapper[2572]: I0716 00:39:54.283104 2572 state_mem.go:36] "Initialized new in-memory state store" Jul 16 00:39:55.810357 ip-10-0-143-200 kubenswrapper[2572]: I0716 00:39:54.284155 2572 policy_none.go:49] "None policy: Start" Jul 16 00:39:55.810357 ip-10-0-143-200 kubenswrapper[2572]: I0716 00:39:54.284167 2572 memory_manager.go:187] "Starting memorymanager" policy="None" Jul 16 00:39:55.810357 ip-10-0-143-200 kubenswrapper[2572]: I0716 00:39:54.284176 2572 state_mem.go:36] "Initializing new in-memory state store" logger="Memory Manager state checkpoint" Jul 16 00:39:55.810357 ip-10-0-143-200 kubenswrapper[2572]: I0716 00:39:54.285111 2572 policy_none.go:47] "Start" Jul 16 00:39:55.810357 ip-10-0-143-200 kubenswrapper[2572]: I0716 00:39:54.285830 2572 kubelet_node_status.go:400] "Setting node annotation to enable volume controller attach/detach" Jul 16 00:39:55.810357 ip-10-0-143-200 kubenswrapper[2572]: E0716 00:39:54.286152 2572 event.go:359] "Server rejected event (will not retry!)" err="events is forbidden: User \"system:anonymous\" cannot create resource \"events\" in API group \"\" in the namespace \"default\"" event="&Event{ObjectMeta:{ip-10-0-143-200.ec2.internal.18c29e3ea4d2b8a7 default 0 0001-01-01 00:00:00 +0000 UTC map[] map[] [] [] []},InvolvedObject:ObjectReference{Kind:Node,Namespace:,Name:ip-10-0-143-200.ec2.internal,UID:ip-10-0-143-200.ec2.internal,APIVersion:,ResourceVersion:,FieldPath:,},Reason:NodeHasSufficientMemory,Message:Node ip-10-0-143-200.ec2.internal status is now: NodeHasSufficientMemory,Source:EventSource{Component:kubelet,Host:ip-10-0-143-200.ec2.internal,},FirstTimestamp:2026-07-16 00:39:54.282571943 +0000 UTC m=+0.447742047,LastTimestamp:2026-07-16 00:39:54.282571943 +0000 UTC m=+0.447742047,Count:1,Type:Normal,EventTime:0001-01-01 00:00:00 +0000 UTC,Series:nil,Action:,Related:nil,ReportingController:kubelet,ReportingInstance:ip-10-0-143-200.ec2.internal,}" Jul 16 00:39:55.810357 ip-10-0-143-200 kubenswrapper[2572]: I0716 00:39:54.286686 2572 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-143-200.ec2.internal" event="NodeHasSufficientMemory" Jul 16 00:39:55.810357 ip-10-0-143-200 kubenswrapper[2572]: I0716 00:39:54.286710 2572 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-143-200.ec2.internal" event="NodeHasNoDiskPressure" Jul 16 00:39:55.810357 ip-10-0-143-200 kubenswrapper[2572]: I0716 00:39:54.286730 2572 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-143-200.ec2.internal" event="NodeHasSufficientPID" Jul 16 00:39:55.810357 ip-10-0-143-200 kubenswrapper[2572]: I0716 00:39:54.325868 2572 manager.go:337] "Starting Device Plugin manager" Jul 16 00:39:55.810357 ip-10-0-143-200 kubenswrapper[2572]: E0716 00:39:54.325899 2572 manager.go:513] "Failed to read data from checkpoint" err="checkpoint is not found" checkpoint="kubelet_internal_checkpoint" Jul 16 00:39:55.810357 ip-10-0-143-200 kubenswrapper[2572]: I0716 00:39:54.325909 2572 server.go:87] "Starting device plugin registration server" Jul 16 00:39:55.810357 ip-10-0-143-200 kubenswrapper[2572]: I0716 00:39:54.326130 2572 eviction_manager.go:189] "Eviction manager: starting control loop" Jul 16 00:39:55.810357 ip-10-0-143-200 kubenswrapper[2572]: I0716 00:39:54.326142 2572 container_log_manager.go:146] "Initializing container log rotate workers" workers=1 monitorPeriod="10s" Jul 16 00:39:55.810357 ip-10-0-143-200 kubenswrapper[2572]: I0716 00:39:54.326254 2572 plugin_watcher.go:51] "Plugin Watcher Start" path="/var/lib/kubelet/plugins_registry" Jul 16 00:39:55.810357 ip-10-0-143-200 kubenswrapper[2572]: I0716 00:39:54.326338 2572 plugin_manager.go:116] "The desired_state_of_world populator (plugin watcher) starts" Jul 16 00:39:55.810357 ip-10-0-143-200 kubenswrapper[2572]: I0716 00:39:54.326350 2572 plugin_manager.go:118] "Starting Kubelet Plugin Manager" Jul 16 00:39:55.810357 ip-10-0-143-200 kubenswrapper[2572]: E0716 00:39:54.327343 2572 eviction_manager.go:267] "eviction manager: failed to check if we have separate container filesystem. Ignoring." err="non-existent label \"crio-containers\"" Jul 16 00:39:55.810357 ip-10-0-143-200 kubenswrapper[2572]: E0716 00:39:54.327391 2572 eviction_manager.go:292] "Eviction manager: failed to get summary stats" err="failed to get node info: node \"ip-10-0-143-200.ec2.internal\" not found" Jul 16 00:39:55.811326 ip-10-0-143-200 kubenswrapper[2572]: I0716 00:39:54.348546 2572 container_manager_linux.go:676] "Failed to ensure state" containerName="/system.slice" err="failed to move PID 2608 into the system container \"/system.slice\": " Jul 16 00:39:55.811326 ip-10-0-143-200 kubenswrapper[2572]: I0716 00:39:54.383568 2572 kubelet_network_linux.go:54] "Initialized iptables rules." protocol="IPv4" Jul 16 00:39:55.811326 ip-10-0-143-200 kubenswrapper[2572]: I0716 00:39:54.384829 2572 kubelet_network_linux.go:54] "Initialized iptables rules." protocol="IPv6" Jul 16 00:39:55.811326 ip-10-0-143-200 kubenswrapper[2572]: I0716 00:39:54.384866 2572 status_manager.go:244] "Starting to sync pod status with apiserver" Jul 16 00:39:55.811326 ip-10-0-143-200 kubenswrapper[2572]: I0716 00:39:54.384893 2572 kubelet.go:2448] "Starting kubelet main sync loop" Jul 16 00:39:55.811326 ip-10-0-143-200 kubenswrapper[2572]: E0716 00:39:54.384939 2572 kubelet.go:2472] "Skipping pod synchronization" err="PLEG is not healthy: pleg has yet to be successful" Jul 16 00:39:55.811326 ip-10-0-143-200 kubenswrapper[2572]: I0716 00:39:54.387079 2572 reflector.go:439] "Caches populated" type="*v1.RuntimeClass" reflector="k8s.io/client-go/informers/factory.go:160" Jul 16 00:39:55.811326 ip-10-0-143-200 kubenswrapper[2572]: I0716 00:39:54.426626 2572 kubelet_node_status.go:400] "Setting node annotation to enable volume controller attach/detach" Jul 16 00:39:55.811326 ip-10-0-143-200 kubenswrapper[2572]: I0716 00:39:54.427729 2572 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-143-200.ec2.internal" event="NodeHasSufficientMemory" Jul 16 00:39:55.811326 ip-10-0-143-200 kubenswrapper[2572]: I0716 00:39:54.427763 2572 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-143-200.ec2.internal" event="NodeHasNoDiskPressure" Jul 16 00:39:55.811326 ip-10-0-143-200 kubenswrapper[2572]: I0716 00:39:54.427779 2572 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-143-200.ec2.internal" event="NodeHasSufficientPID" Jul 16 00:39:55.811326 ip-10-0-143-200 kubenswrapper[2572]: I0716 00:39:54.427804 2572 kubelet_node_status.go:78] "Attempting to register node" node="ip-10-0-143-200.ec2.internal" Jul 16 00:39:55.811326 ip-10-0-143-200 kubenswrapper[2572]: I0716 00:39:54.437409 2572 kubelet_node_status.go:81] "Successfully registered node" node="ip-10-0-143-200.ec2.internal" Jul 16 00:39:55.811326 ip-10-0-143-200 kubenswrapper[2572]: E0716 00:39:54.437434 2572 kubelet_node_status.go:536] "Error updating node status, will retry" err="error getting node \"ip-10-0-143-200.ec2.internal\": node \"ip-10-0-143-200.ec2.internal\" not found" Jul 16 00:39:55.811326 ip-10-0-143-200 kubenswrapper[2572]: E0716 00:39:54.464619 2572 kubelet_node_status.go:454] "Error getting the current node from lister" err="node \"ip-10-0-143-200.ec2.internal\" not found" Jul 16 00:39:55.811326 ip-10-0-143-200 kubenswrapper[2572]: I0716 00:39:54.485034 2572 kubelet.go:2535] "SyncLoop ADD" source="file" pods=["openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-143-200.ec2.internal","kube-system/kube-apiserver-proxy-ip-10-0-143-200.ec2.internal"] Jul 16 00:39:55.811326 ip-10-0-143-200 kubenswrapper[2572]: I0716 00:39:54.485100 2572 kubelet_node_status.go:400] "Setting node annotation to enable volume controller attach/detach" Jul 16 00:39:55.811326 ip-10-0-143-200 kubenswrapper[2572]: I0716 00:39:54.486130 2572 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-143-200.ec2.internal" event="NodeHasSufficientMemory" Jul 16 00:39:55.811326 ip-10-0-143-200 kubenswrapper[2572]: I0716 00:39:54.486156 2572 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-143-200.ec2.internal" event="NodeHasNoDiskPressure" Jul 16 00:39:55.811326 ip-10-0-143-200 kubenswrapper[2572]: I0716 00:39:54.486166 2572 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-143-200.ec2.internal" event="NodeHasSufficientPID" Jul 16 00:39:55.811326 ip-10-0-143-200 kubenswrapper[2572]: I0716 00:39:54.487545 2572 kubelet_node_status.go:400] "Setting node annotation to enable volume controller attach/detach" Jul 16 00:39:55.811326 ip-10-0-143-200 kubenswrapper[2572]: I0716 00:39:54.487661 2572 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-143-200.ec2.internal" Jul 16 00:39:55.811326 ip-10-0-143-200 kubenswrapper[2572]: I0716 00:39:54.487690 2572 kubelet_node_status.go:400] "Setting node annotation to enable volume controller attach/detach" Jul 16 00:39:55.811326 ip-10-0-143-200 kubenswrapper[2572]: I0716 00:39:54.488395 2572 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-143-200.ec2.internal" event="NodeHasSufficientMemory" Jul 16 00:39:55.811326 ip-10-0-143-200 kubenswrapper[2572]: I0716 00:39:54.488406 2572 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-143-200.ec2.internal" event="NodeHasSufficientMemory" Jul 16 00:39:55.811326 ip-10-0-143-200 kubenswrapper[2572]: I0716 00:39:54.488418 2572 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-143-200.ec2.internal" event="NodeHasNoDiskPressure" Jul 16 00:39:55.811326 ip-10-0-143-200 kubenswrapper[2572]: I0716 00:39:54.488425 2572 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-143-200.ec2.internal" event="NodeHasNoDiskPressure" Jul 16 00:39:55.811326 ip-10-0-143-200 kubenswrapper[2572]: I0716 00:39:54.488428 2572 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-143-200.ec2.internal" event="NodeHasSufficientPID" Jul 16 00:39:55.811326 ip-10-0-143-200 kubenswrapper[2572]: I0716 00:39:54.488436 2572 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-143-200.ec2.internal" event="NodeHasSufficientPID" Jul 16 00:39:55.811326 ip-10-0-143-200 kubenswrapper[2572]: I0716 00:39:54.496955 2572 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/kube-apiserver-proxy-ip-10-0-143-200.ec2.internal" Jul 16 00:39:55.811326 ip-10-0-143-200 kubenswrapper[2572]: I0716 00:39:54.496977 2572 kubelet_node_status.go:400] "Setting node annotation to enable volume controller attach/detach" Jul 16 00:39:55.811326 ip-10-0-143-200 kubenswrapper[2572]: I0716 00:39:54.497929 2572 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-143-200.ec2.internal" event="NodeHasSufficientMemory" Jul 16 00:39:55.811326 ip-10-0-143-200 kubenswrapper[2572]: I0716 00:39:54.497969 2572 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-143-200.ec2.internal" event="NodeHasNoDiskPressure" Jul 16 00:39:55.811326 ip-10-0-143-200 kubenswrapper[2572]: I0716 00:39:54.497981 2572 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-143-200.ec2.internal" event="NodeHasSufficientPID" Jul 16 00:39:55.811326 ip-10-0-143-200 kubenswrapper[2572]: E0716 00:39:54.522120 2572 kubelet.go:3244] "No need to create a mirror pod, since failed to get node info from the cluster" err="node \"ip-10-0-143-200.ec2.internal\" not found" node="ip-10-0-143-200.ec2.internal" Jul 16 00:39:55.811326 ip-10-0-143-200 kubenswrapper[2572]: E0716 00:39:54.526480 2572 kubelet.go:3244] "No need to create a mirror pod, since failed to get node info from the cluster" err="node \"ip-10-0-143-200.ec2.internal\" not found" node="ip-10-0-143-200.ec2.internal" Jul 16 00:39:55.811326 ip-10-0-143-200 kubenswrapper[2572]: I0716 00:39:54.553050 2572 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"etc-kube\" (UniqueName: \"kubernetes.io/host-path/df298bc82c8f94d4cb31271396017b60-etc-kube\") pod \"kube-rbac-proxy-crio-ip-10-0-143-200.ec2.internal\" (UID: \"df298bc82c8f94d4cb31271396017b60\") " pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-143-200.ec2.internal" Jul 16 00:39:55.812404 ip-10-0-143-200 kubenswrapper[2572]: I0716 00:39:54.553082 2572 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"var-lib-kubelet\" (UniqueName: \"kubernetes.io/host-path/df298bc82c8f94d4cb31271396017b60-var-lib-kubelet\") pod \"kube-rbac-proxy-crio-ip-10-0-143-200.ec2.internal\" (UID: \"df298bc82c8f94d4cb31271396017b60\") " pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-143-200.ec2.internal" Jul 16 00:39:55.812404 ip-10-0-143-200 kubenswrapper[2572]: I0716 00:39:54.553098 2572 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"config\" (UniqueName: \"kubernetes.io/host-path/f0dfd958ab883b381e2b09c1edf12ef3-config\") pod \"kube-apiserver-proxy-ip-10-0-143-200.ec2.internal\" (UID: \"f0dfd958ab883b381e2b09c1edf12ef3\") " pod="kube-system/kube-apiserver-proxy-ip-10-0-143-200.ec2.internal" Jul 16 00:39:55.812404 ip-10-0-143-200 kubenswrapper[2572]: E0716 00:39:54.565480 2572 kubelet_node_status.go:454] "Error getting the current node from lister" err="node \"ip-10-0-143-200.ec2.internal\" not found" Jul 16 00:39:55.812404 ip-10-0-143-200 kubenswrapper[2572]: I0716 00:39:55.710617 2572 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"etc-kube\" (UniqueName: \"kubernetes.io/host-path/df298bc82c8f94d4cb31271396017b60-etc-kube\") pod \"kube-rbac-proxy-crio-ip-10-0-143-200.ec2.internal\" (UID: \"df298bc82c8f94d4cb31271396017b60\") " pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-143-200.ec2.internal" Jul 16 00:39:55.812404 ip-10-0-143-200 kubenswrapper[2572]: I0716 00:39:55.710647 2572 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"etc-kube\" (UniqueName: \"kubernetes.io/host-path/df298bc82c8f94d4cb31271396017b60-etc-kube\") pod \"kube-rbac-proxy-crio-ip-10-0-143-200.ec2.internal\" (UID: \"df298bc82c8f94d4cb31271396017b60\") " pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-143-200.ec2.internal" Jul 16 00:39:55.812404 ip-10-0-143-200 kubenswrapper[2572]: I0716 00:39:55.710688 2572 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"var-lib-kubelet\" (UniqueName: \"kubernetes.io/host-path/df298bc82c8f94d4cb31271396017b60-var-lib-kubelet\") pod \"kube-rbac-proxy-crio-ip-10-0-143-200.ec2.internal\" (UID: \"df298bc82c8f94d4cb31271396017b60\") " pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-143-200.ec2.internal" Jul 16 00:39:55.812404 ip-10-0-143-200 kubenswrapper[2572]: I0716 00:39:55.710726 2572 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"var-lib-kubelet\" (UniqueName: \"kubernetes.io/host-path/df298bc82c8f94d4cb31271396017b60-var-lib-kubelet\") pod \"kube-rbac-proxy-crio-ip-10-0-143-200.ec2.internal\" (UID: \"df298bc82c8f94d4cb31271396017b60\") " pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-143-200.ec2.internal" Jul 16 00:39:55.812404 ip-10-0-143-200 kubenswrapper[2572]: I0716 00:39:55.710731 2572 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"config\" (UniqueName: \"kubernetes.io/host-path/f0dfd958ab883b381e2b09c1edf12ef3-config\") pod \"kube-apiserver-proxy-ip-10-0-143-200.ec2.internal\" (UID: \"f0dfd958ab883b381e2b09c1edf12ef3\") " pod="kube-system/kube-apiserver-proxy-ip-10-0-143-200.ec2.internal" Jul 16 00:39:55.812404 ip-10-0-143-200 kubenswrapper[2572]: I0716 00:39:55.710753 2572 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"config\" (UniqueName: \"kubernetes.io/host-path/f0dfd958ab883b381e2b09c1edf12ef3-config\") pod \"kube-apiserver-proxy-ip-10-0-143-200.ec2.internal\" (UID: \"f0dfd958ab883b381e2b09c1edf12ef3\") " pod="kube-system/kube-apiserver-proxy-ip-10-0-143-200.ec2.internal" Jul 16 00:39:55.812404 ip-10-0-143-200 kubenswrapper[2572]: E0716 00:39:55.722699 2572 kubelet_node_status.go:454] "Error getting the current node from lister" err="node \"ip-10-0-143-200.ec2.internal\" not found" Jul 16 00:39:55.981623 ip-10-0-143-200 kubenswrapper[2572]: E0716 00:39:55.823538 2572 kubelet_node_status.go:454] "Error getting the current node from lister" err="node \"ip-10-0-143-200.ec2.internal\" not found" Jul 16 00:39:55.981623 ip-10-0-143-200 kubenswrapper[2572]: I0716 00:39:55.880737 2572 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-143-200.ec2.internal" Jul 16 00:39:55.981623 ip-10-0-143-200 kubenswrapper[2572]: I0716 00:39:55.885867 2572 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/kube-apiserver-proxy-ip-10-0-143-200.ec2.internal" Jul 16 00:39:55.981623 ip-10-0-143-200 kubenswrapper[2572]: E0716 00:39:55.923826 2572 kubelet_node_status.go:454] "Error getting the current node from lister" err="node \"ip-10-0-143-200.ec2.internal\" not found" Jul 16 00:39:56.195472 ip-10-0-143-200 kubenswrapper[2572]: E0716 00:39:56.024318 2572 kubelet_node_status.go:454] "Error getting the current node from lister" err="node \"ip-10-0-143-200.ec2.internal\" not found" Jul 16 00:39:56.195472 ip-10-0-143-200 kubenswrapper[2572]: E0716 00:39:56.124815 2572 kubelet_node_status.go:454] "Error getting the current node from lister" err="node \"ip-10-0-143-200.ec2.internal\" not found" Jul 16 00:39:56.208386 ip-10-0-143-200 kubenswrapper[2572]: I0716 00:39:56.208362 2572 reflector.go:439] "Caches populated" type="*v1.CSIDriver" reflector="k8s.io/client-go/informers/factory.go:160" Jul 16 00:39:56.225144 ip-10-0-143-200 kubenswrapper[2572]: E0716 00:39:56.225114 2572 kubelet_node_status.go:454] "Error getting the current node from lister" err="node \"ip-10-0-143-200.ec2.internal\" not found" Jul 16 00:39:56.241311 ip-10-0-143-200 kubenswrapper[2572]: I0716 00:39:56.241283 2572 transport.go:147] "Certificate rotation detected, shutting down client connections to start using new credentials" Jul 16 00:39:56.241451 ip-10-0-143-200 kubenswrapper[2572]: I0716 00:39:56.241433 2572 reflector.go:571] "Warning: watch ended with error" reflector="k8s.io/client-go/informers/factory.go:160" type="*v1.RuntimeClass" err="very short watch: k8s.io/client-go/informers/factory.go:160: Unexpected watch close - watch lasted less than a second and no items received" Jul 16 00:39:56.241492 ip-10-0-143-200 kubenswrapper[2572]: I0716 00:39:56.241471 2572 reflector.go:571] "Warning: watch ended with error" reflector="k8s.io/client-go/informers/factory.go:160" type="*v1.CSIDriver" err="very short watch: k8s.io/client-go/informers/factory.go:160: Unexpected watch close - watch lasted less than a second and no items received" Jul 16 00:39:56.325527 ip-10-0-143-200 kubenswrapper[2572]: E0716 00:39:56.325501 2572 kubelet_node_status.go:454] "Error getting the current node from lister" err="node \"ip-10-0-143-200.ec2.internal\" not found" Jul 16 00:39:56.340152 ip-10-0-143-200 kubenswrapper[2572]: I0716 00:39:56.339924 2572 certificate_manager.go:715] "Certificate rotation deadline determined" logger="kubernetes.io/kube-apiserver-client-kubelet" expiration="2028-07-15 00:34:55 +0000 UTC" deadline="2027-12-24 14:18:44.3362252 +0000 UTC" Jul 16 00:39:56.340152 ip-10-0-143-200 kubenswrapper[2572]: I0716 00:39:56.340150 2572 certificate_manager.go:431] "Waiting for next certificate rotation" logger="kubernetes.io/kube-apiserver-client-kubelet" sleep="12637h38m47.996080793s" Jul 16 00:39:56.426088 ip-10-0-143-200 kubenswrapper[2572]: E0716 00:39:56.426058 2572 kubelet_node_status.go:454] "Error getting the current node from lister" err="node \"ip-10-0-143-200.ec2.internal\" not found" Jul 16 00:39:56.517736 ip-10-0-143-200 kubenswrapper[2572]: W0716 00:39:56.517702 2572 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-poddf298bc82c8f94d4cb31271396017b60.slice/crio-335b23f12248d190043b41413c36fc2cde8484dfab250fa09f701f96d41c303e WatchSource:0}: Error finding container 335b23f12248d190043b41413c36fc2cde8484dfab250fa09f701f96d41c303e: Status 404 returned error can't find the container with id 335b23f12248d190043b41413c36fc2cde8484dfab250fa09f701f96d41c303e Jul 16 00:39:56.518163 ip-10-0-143-200 kubenswrapper[2572]: W0716 00:39:56.518140 2572 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podf0dfd958ab883b381e2b09c1edf12ef3.slice/crio-a7f184f793ea7009ffc73ecb0cee71010bb547ba79c1fb80267d0a5934f30e49 WatchSource:0}: Error finding container a7f184f793ea7009ffc73ecb0cee71010bb547ba79c1fb80267d0a5934f30e49: Status 404 returned error can't find the container with id a7f184f793ea7009ffc73ecb0cee71010bb547ba79c1fb80267d0a5934f30e49 Jul 16 00:39:56.522105 ip-10-0-143-200 kubenswrapper[2572]: I0716 00:39:56.522092 2572 provider.go:93] Refreshing cache for provider: *credentialprovider.defaultDockerConfigProvider Jul 16 00:39:56.526868 ip-10-0-143-200 kubenswrapper[2572]: E0716 00:39:56.526852 2572 kubelet_node_status.go:454] "Error getting the current node from lister" err="node \"ip-10-0-143-200.ec2.internal\" not found" Jul 16 00:39:56.531885 ip-10-0-143-200 kubenswrapper[2572]: I0716 00:39:56.531867 2572 reflector.go:439] "Caches populated" type="*v1.Node" reflector="k8s.io/client-go/informers/factory.go:160" Jul 16 00:39:56.592186 ip-10-0-143-200 kubenswrapper[2572]: I0716 00:39:56.592148 2572 reflector.go:439] "Caches populated" type="*v1.Service" reflector="k8s.io/client-go/informers/factory.go:160" Jul 16 00:39:56.608594 ip-10-0-143-200 kubenswrapper[2572]: I0716 00:39:56.608566 2572 kubelet.go:3248] "Creating a mirror pod for static pod" pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-143-200.ec2.internal" Jul 16 00:39:56.622065 ip-10-0-143-200 kubenswrapper[2572]: I0716 00:39:56.622045 2572 warnings.go:110] "Warning: metadata.name: this is used in the Pod's hostname, which can result in surprising behavior; a DNS label is recommended: [must not contain dots]" Jul 16 00:39:56.622993 ip-10-0-143-200 kubenswrapper[2572]: I0716 00:39:56.622980 2572 kubelet.go:3248] "Creating a mirror pod for static pod" pod="kube-system/kube-apiserver-proxy-ip-10-0-143-200.ec2.internal" Jul 16 00:39:56.629856 ip-10-0-143-200 kubenswrapper[2572]: I0716 00:39:56.629839 2572 warnings.go:110] "Warning: metadata.name: this is used in the Pod's hostname, which can result in surprising behavior; a DNS label is recommended: [must not contain dots]" Jul 16 00:39:57.283661 ip-10-0-143-200 kubenswrapper[2572]: I0716 00:39:57.283606 2572 apiserver.go:52] "Watching apiserver" Jul 16 00:39:57.293421 ip-10-0-143-200 kubenswrapper[2572]: I0716 00:39:57.293387 2572 reflector.go:439] "Caches populated" type="*v1.Pod" reflector="pkg/kubelet/config/apiserver.go:66" Jul 16 00:39:57.293989 ip-10-0-143-200 kubenswrapper[2572]: I0716 00:39:57.293953 2572 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-multus/multus-szqbq","openshift-network-diagnostics/network-check-target-52n54","openshift-network-operator/iptables-alerter-gcstg","kube-system/konnectivity-agent-9gnmq","kube-system/kube-apiserver-proxy-ip-10-0-143-200.ec2.internal","openshift-cluster-node-tuning-operator/tuned-w6p89","openshift-dns/node-resolver-6qlgz","openshift-multus/multus-additional-cni-plugins-vfntt","openshift-multus/network-metrics-daemon-5tphg","openshift-ovn-kubernetes/ovnkube-node-fp78g","kube-system/global-pull-secret-syncer-qcbcd","openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-pq88r","openshift-image-registry/node-ca-6msnd","openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-143-200.ec2.internal"] Jul 16 00:39:57.296459 ip-10-0-143-200 kubenswrapper[2572]: I0716 00:39:57.296428 2572 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-dns/node-resolver-6qlgz" Jul 16 00:39:57.299908 ip-10-0-143-200 kubenswrapper[2572]: I0716 00:39:57.299194 2572 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-dns\"/\"openshift-service-ca.crt\"" Jul 16 00:39:57.299908 ip-10-0-143-200 kubenswrapper[2572]: I0716 00:39:57.299444 2572 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-dns\"/\"node-resolver-dockercfg-jdplp\"" Jul 16 00:39:57.299908 ip-10-0-143-200 kubenswrapper[2572]: I0716 00:39:57.299844 2572 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-dns\"/\"kube-root-ca.crt\"" Jul 16 00:39:57.300139 ip-10-0-143-200 kubenswrapper[2572]: I0716 00:39:57.300115 2572 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-qcbcd" Jul 16 00:39:57.300288 ip-10-0-143-200 kubenswrapper[2572]: E0716 00:39:57.300189 2572 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-qcbcd" podUID="a9bdff50-c8ce-4ca4-a810-20be3909b23c" Jul 16 00:39:57.301663 ip-10-0-143-200 kubenswrapper[2572]: I0716 00:39:57.301638 2572 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-operator/iptables-alerter-gcstg" Jul 16 00:39:57.304105 ip-10-0-143-200 kubenswrapper[2572]: I0716 00:39:57.303716 2572 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/konnectivity-agent-9gnmq" Jul 16 00:39:57.304283 ip-10-0-143-200 kubenswrapper[2572]: I0716 00:39:57.304259 2572 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-network-operator\"/\"iptables-alerter-script\"" Jul 16 00:39:57.304663 ip-10-0-143-200 kubenswrapper[2572]: I0716 00:39:57.304612 2572 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-network-operator\"/\"iptables-alerter-dockercfg-grtzs\"" Jul 16 00:39:57.304760 ip-10-0-143-200 kubenswrapper[2572]: I0716 00:39:57.304691 2572 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-network-operator\"/\"kube-root-ca.crt\"" Jul 16 00:39:57.304940 ip-10-0-143-200 kubenswrapper[2572]: I0716 00:39:57.304921 2572 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-network-operator\"/\"openshift-service-ca.crt\"" Jul 16 00:39:57.307121 ip-10-0-143-200 kubenswrapper[2572]: I0716 00:39:57.306604 2572 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"kube-system\"/\"konnectivity-agent\"" Jul 16 00:39:57.307121 ip-10-0-143-200 kubenswrapper[2572]: I0716 00:39:57.306640 2572 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"kube-system\"/\"konnectivity-ca-bundle\"" Jul 16 00:39:57.307121 ip-10-0-143-200 kubenswrapper[2572]: I0716 00:39:57.306662 2572 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"kube-system\"/\"default-dockercfg-pvz4l\"" Jul 16 00:39:57.307121 ip-10-0-143-200 kubenswrapper[2572]: I0716 00:39:57.306998 2572 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/multus-additional-cni-plugins-vfntt" Jul 16 00:39:57.309573 ip-10-0-143-200 kubenswrapper[2572]: I0716 00:39:57.309317 2572 certificate_manager.go:566] "Rotating certificates" logger="kubernetes.io/kubelet-serving" Jul 16 00:39:57.309573 ip-10-0-143-200 kubenswrapper[2572]: I0716 00:39:57.309451 2572 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-multus\"/\"openshift-service-ca.crt\"" Jul 16 00:39:57.309573 ip-10-0-143-200 kubenswrapper[2572]: I0716 00:39:57.309466 2572 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-multus\"/\"default-cni-sysctl-allowlist\"" Jul 16 00:39:57.309573 ip-10-0-143-200 kubenswrapper[2572]: I0716 00:39:57.309540 2572 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-multus\"/\"whereabouts-flatfile-config\"" Jul 16 00:39:57.310085 ip-10-0-143-200 kubenswrapper[2572]: I0716 00:39:57.310068 2572 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-multus\"/\"cni-copy-resources\"" Jul 16 00:39:57.310166 ip-10-0-143-200 kubenswrapper[2572]: I0716 00:39:57.310128 2572 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-multus\"/\"multus-ancillary-tools-dockercfg-6fk4l\"" Jul 16 00:39:57.310225 ip-10-0-143-200 kubenswrapper[2572]: I0716 00:39:57.310073 2572 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-multus\"/\"kube-root-ca.crt\"" Jul 16 00:39:57.310581 ip-10-0-143-200 kubenswrapper[2572]: I0716 00:39:57.310562 2572 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-cluster-node-tuning-operator/tuned-w6p89" Jul 16 00:39:57.313058 ip-10-0-143-200 kubenswrapper[2572]: I0716 00:39:57.313001 2572 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-5tphg" Jul 16 00:39:57.313137 ip-10-0-143-200 kubenswrapper[2572]: E0716 00:39:57.313065 2572 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-5tphg" podUID="4a08cb1b-3bfa-49cf-8ff0-edef0ff944b9" Jul 16 00:39:57.313137 ip-10-0-143-200 kubenswrapper[2572]: I0716 00:39:57.313103 2572 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-cluster-node-tuning-operator\"/\"openshift-service-ca.crt\"" Jul 16 00:39:57.313254 ip-10-0-143-200 kubenswrapper[2572]: I0716 00:39:57.313230 2572 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-cluster-node-tuning-operator\"/\"kube-root-ca.crt\"" Jul 16 00:39:57.313307 ip-10-0-143-200 kubenswrapper[2572]: I0716 00:39:57.313257 2572 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-cluster-node-tuning-operator\"/\"tuned-dockercfg-xk8pw\"" Jul 16 00:39:57.315236 ip-10-0-143-200 kubenswrapper[2572]: I0716 00:39:57.315116 2572 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-image-registry/node-ca-6msnd" Jul 16 00:39:57.317933 ip-10-0-143-200 kubenswrapper[2572]: I0716 00:39:57.317914 2572 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-52n54" Jul 16 00:39:57.318042 ip-10-0-143-200 kubenswrapper[2572]: E0716 00:39:57.317985 2572 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-52n54" podUID="8962f109-ea94-4296-a504-f6709315e9cd" Jul 16 00:39:57.318274 ip-10-0-143-200 kubenswrapper[2572]: I0716 00:39:57.318253 2572 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-image-registry\"/\"openshift-service-ca.crt\"" Jul 16 00:39:57.318474 ip-10-0-143-200 kubenswrapper[2572]: I0716 00:39:57.318454 2572 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-image-registry\"/\"kube-root-ca.crt\"" Jul 16 00:39:57.318643 ip-10-0-143-200 kubenswrapper[2572]: I0716 00:39:57.318453 2572 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-image-registry\"/\"image-registry-certificates\"" Jul 16 00:39:57.318713 ip-10-0-143-200 kubenswrapper[2572]: I0716 00:39:57.318603 2572 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-image-registry\"/\"node-ca-dockercfg-7w298\"" Jul 16 00:39:57.319470 ip-10-0-143-200 kubenswrapper[2572]: I0716 00:39:57.319450 2572 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"agent-certs\" (UniqueName: \"kubernetes.io/secret/3a7e030d-fad8-480d-9dc6-fe5057048cf9-agent-certs\") pod \"konnectivity-agent-9gnmq\" (UID: \"3a7e030d-fad8-480d-9dc6-fe5057048cf9\") " pod="kube-system/konnectivity-agent-9gnmq" Jul 16 00:39:57.319548 ip-10-0-143-200 kubenswrapper[2572]: I0716 00:39:57.319485 2572 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"etc-sysctl-d\" (UniqueName: \"kubernetes.io/host-path/2614f4d9-8dc4-406b-a5aa-7dd2e7446008-etc-sysctl-d\") pod \"tuned-w6p89\" (UID: \"2614f4d9-8dc4-406b-a5aa-7dd2e7446008\") " pod="openshift-cluster-node-tuning-operator/tuned-w6p89" Jul 16 00:39:57.319548 ip-10-0-143-200 kubenswrapper[2572]: I0716 00:39:57.319516 2572 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tmp\" (UniqueName: \"kubernetes.io/empty-dir/2614f4d9-8dc4-406b-a5aa-7dd2e7446008-tmp\") pod \"tuned-w6p89\" (UID: \"2614f4d9-8dc4-406b-a5aa-7dd2e7446008\") " pod="openshift-cluster-node-tuning-operator/tuned-w6p89" Jul 16 00:39:57.319548 ip-10-0-143-200 kubenswrapper[2572]: I0716 00:39:57.319543 2572 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"original-pull-secret\" (UniqueName: \"kubernetes.io/secret/a9bdff50-c8ce-4ca4-a810-20be3909b23c-original-pull-secret\") pod \"global-pull-secret-syncer-qcbcd\" (UID: \"a9bdff50-c8ce-4ca4-a810-20be3909b23c\") " pod="kube-system/global-pull-secret-syncer-qcbcd" Jul 16 00:39:57.319811 ip-10-0-143-200 kubenswrapper[2572]: I0716 00:39:57.319582 2572 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-slash\" (UniqueName: \"kubernetes.io/host-path/0247608f-bb0b-4ace-b071-292d5586fc65-host-slash\") pod \"iptables-alerter-gcstg\" (UID: \"0247608f-bb0b-4ace-b071-292d5586fc65\") " pod="openshift-network-operator/iptables-alerter-gcstg" Jul 16 00:39:57.319985 ip-10-0-143-200 kubenswrapper[2572]: I0716 00:39:57.319949 2572 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"cni-sysctl-allowlist\" (UniqueName: \"kubernetes.io/configmap/a51f2d32-4840-4f0c-804d-c8aa6015ae5a-cni-sysctl-allowlist\") pod \"multus-additional-cni-plugins-vfntt\" (UID: \"a51f2d32-4840-4f0c-804d-c8aa6015ae5a\") " pod="openshift-multus/multus-additional-cni-plugins-vfntt" Jul 16 00:39:57.320055 ip-10-0-143-200 kubenswrapper[2572]: I0716 00:39:57.319997 2572 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"system-cni-dir\" (UniqueName: \"kubernetes.io/host-path/a51f2d32-4840-4f0c-804d-c8aa6015ae5a-system-cni-dir\") pod \"multus-additional-cni-plugins-vfntt\" (UID: \"a51f2d32-4840-4f0c-804d-c8aa6015ae5a\") " pod="openshift-multus/multus-additional-cni-plugins-vfntt" Jul 16 00:39:57.320055 ip-10-0-143-200 kubenswrapper[2572]: I0716 00:39:57.320042 2572 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"etc-sysconfig\" (UniqueName: \"kubernetes.io/host-path/2614f4d9-8dc4-406b-a5aa-7dd2e7446008-etc-sysconfig\") pod \"tuned-w6p89\" (UID: \"2614f4d9-8dc4-406b-a5aa-7dd2e7446008\") " pod="openshift-cluster-node-tuning-operator/tuned-w6p89" Jul 16 00:39:57.320159 ip-10-0-143-200 kubenswrapper[2572]: I0716 00:39:57.320075 2572 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"etc-sysctl-conf\" (UniqueName: \"kubernetes.io/host-path/2614f4d9-8dc4-406b-a5aa-7dd2e7446008-etc-sysctl-conf\") pod \"tuned-w6p89\" (UID: \"2614f4d9-8dc4-406b-a5aa-7dd2e7446008\") " pod="openshift-cluster-node-tuning-operator/tuned-w6p89" Jul 16 00:39:57.320159 ip-10-0-143-200 kubenswrapper[2572]: I0716 00:39:57.320104 2572 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"lib-modules\" (UniqueName: \"kubernetes.io/host-path/2614f4d9-8dc4-406b-a5aa-7dd2e7446008-lib-modules\") pod \"tuned-w6p89\" (UID: \"2614f4d9-8dc4-406b-a5aa-7dd2e7446008\") " pod="openshift-cluster-node-tuning-operator/tuned-w6p89" Jul 16 00:39:57.320337 ip-10-0-143-200 kubenswrapper[2572]: I0716 00:39:57.320314 2572 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"var-lib-kubelet\" (UniqueName: \"kubernetes.io/host-path/2614f4d9-8dc4-406b-a5aa-7dd2e7446008-var-lib-kubelet\") pod \"tuned-w6p89\" (UID: \"2614f4d9-8dc4-406b-a5aa-7dd2e7446008\") " pod="openshift-cluster-node-tuning-operator/tuned-w6p89" Jul 16 00:39:57.320439 ip-10-0-143-200 kubenswrapper[2572]: I0716 00:39:57.320399 2572 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"etc-kubernetes\" (UniqueName: \"kubernetes.io/host-path/2614f4d9-8dc4-406b-a5aa-7dd2e7446008-etc-kubernetes\") pod \"tuned-w6p89\" (UID: \"2614f4d9-8dc4-406b-a5aa-7dd2e7446008\") " pod="openshift-cluster-node-tuning-operator/tuned-w6p89" Jul 16 00:39:57.320439 ip-10-0-143-200 kubenswrapper[2572]: I0716 00:39:57.320431 2572 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-vt9tv\" (UniqueName: \"kubernetes.io/projected/2614f4d9-8dc4-406b-a5aa-7dd2e7446008-kube-api-access-vt9tv\") pod \"tuned-w6p89\" (UID: \"2614f4d9-8dc4-406b-a5aa-7dd2e7446008\") " pod="openshift-cluster-node-tuning-operator/tuned-w6p89" Jul 16 00:39:57.320540 ip-10-0-143-200 kubenswrapper[2572]: I0716 00:39:57.320463 2572 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"hosts-file\" (UniqueName: \"kubernetes.io/host-path/24fd12c1-ddcb-427f-ac7e-4e2877450cb8-hosts-file\") pod \"node-resolver-6qlgz\" (UID: \"24fd12c1-ddcb-427f-ac7e-4e2877450cb8\") " pod="openshift-dns/node-resolver-6qlgz" Jul 16 00:39:57.320540 ip-10-0-143-200 kubenswrapper[2572]: I0716 00:39:57.320498 2572 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"whereabouts-flatfile-configmap\" (UniqueName: \"kubernetes.io/configmap/a51f2d32-4840-4f0c-804d-c8aa6015ae5a-whereabouts-flatfile-configmap\") pod \"multus-additional-cni-plugins-vfntt\" (UID: \"a51f2d32-4840-4f0c-804d-c8aa6015ae5a\") " pod="openshift-multus/multus-additional-cni-plugins-vfntt" Jul 16 00:39:57.320540 ip-10-0-143-200 kubenswrapper[2572]: I0716 00:39:57.320534 2572 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-2k4vw\" (UniqueName: \"kubernetes.io/projected/a51f2d32-4840-4f0c-804d-c8aa6015ae5a-kube-api-access-2k4vw\") pod \"multus-additional-cni-plugins-vfntt\" (UID: \"a51f2d32-4840-4f0c-804d-c8aa6015ae5a\") " pod="openshift-multus/multus-additional-cni-plugins-vfntt" Jul 16 00:39:57.320760 ip-10-0-143-200 kubenswrapper[2572]: I0716 00:39:57.320736 2572 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"etc-systemd\" (UniqueName: \"kubernetes.io/host-path/2614f4d9-8dc4-406b-a5aa-7dd2e7446008-etc-systemd\") pod \"tuned-w6p89\" (UID: \"2614f4d9-8dc4-406b-a5aa-7dd2e7446008\") " pod="openshift-cluster-node-tuning-operator/tuned-w6p89" Jul 16 00:39:57.320827 ip-10-0-143-200 kubenswrapper[2572]: I0716 00:39:57.320775 2572 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-5n9fh\" (UniqueName: \"kubernetes.io/projected/0247608f-bb0b-4ace-b071-292d5586fc65-kube-api-access-5n9fh\") pod \"iptables-alerter-gcstg\" (UID: \"0247608f-bb0b-4ace-b071-292d5586fc65\") " pod="openshift-network-operator/iptables-alerter-gcstg" Jul 16 00:39:57.320827 ip-10-0-143-200 kubenswrapper[2572]: I0716 00:39:57.320807 2572 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"sys\" (UniqueName: \"kubernetes.io/host-path/2614f4d9-8dc4-406b-a5aa-7dd2e7446008-sys\") pod \"tuned-w6p89\" (UID: \"2614f4d9-8dc4-406b-a5aa-7dd2e7446008\") " pod="openshift-cluster-node-tuning-operator/tuned-w6p89" Jul 16 00:39:57.320900 ip-10-0-143-200 kubenswrapper[2572]: I0716 00:39:57.320836 2572 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host\" (UniqueName: \"kubernetes.io/host-path/2614f4d9-8dc4-406b-a5aa-7dd2e7446008-host\") pod \"tuned-w6p89\" (UID: \"2614f4d9-8dc4-406b-a5aa-7dd2e7446008\") " pod="openshift-cluster-node-tuning-operator/tuned-w6p89" Jul 16 00:39:57.320900 ip-10-0-143-200 kubenswrapper[2572]: I0716 00:39:57.320860 2572 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"etc-tuned\" (UniqueName: \"kubernetes.io/empty-dir/2614f4d9-8dc4-406b-a5aa-7dd2e7446008-etc-tuned\") pod \"tuned-w6p89\" (UID: \"2614f4d9-8dc4-406b-a5aa-7dd2e7446008\") " pod="openshift-cluster-node-tuning-operator/tuned-w6p89" Jul 16 00:39:57.321087 ip-10-0-143-200 kubenswrapper[2572]: I0716 00:39:57.321048 2572 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kubelet-config\" (UniqueName: \"kubernetes.io/host-path/a9bdff50-c8ce-4ca4-a810-20be3909b23c-kubelet-config\") pod \"global-pull-secret-syncer-qcbcd\" (UID: \"a9bdff50-c8ce-4ca4-a810-20be3909b23c\") " pod="kube-system/global-pull-secret-syncer-qcbcd" Jul 16 00:39:57.321087 ip-10-0-143-200 kubenswrapper[2572]: I0716 00:39:57.321082 2572 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"etc-modprobe-d\" (UniqueName: \"kubernetes.io/host-path/2614f4d9-8dc4-406b-a5aa-7dd2e7446008-etc-modprobe-d\") pod \"tuned-w6p89\" (UID: \"2614f4d9-8dc4-406b-a5aa-7dd2e7446008\") " pod="openshift-cluster-node-tuning-operator/tuned-w6p89" Jul 16 00:39:57.321204 ip-10-0-143-200 kubenswrapper[2572]: I0716 00:39:57.321116 2572 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"konnectivity-ca\" (UniqueName: \"kubernetes.io/configmap/3a7e030d-fad8-480d-9dc6-fe5057048cf9-konnectivity-ca\") pod \"konnectivity-agent-9gnmq\" (UID: \"3a7e030d-fad8-480d-9dc6-fe5057048cf9\") " pod="kube-system/konnectivity-agent-9gnmq" Jul 16 00:39:57.321204 ip-10-0-143-200 kubenswrapper[2572]: I0716 00:39:57.321147 2572 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"cnibin\" (UniqueName: \"kubernetes.io/host-path/a51f2d32-4840-4f0c-804d-c8aa6015ae5a-cnibin\") pod \"multus-additional-cni-plugins-vfntt\" (UID: \"a51f2d32-4840-4f0c-804d-c8aa6015ae5a\") " pod="openshift-multus/multus-additional-cni-plugins-vfntt" Jul 16 00:39:57.321204 ip-10-0-143-200 kubenswrapper[2572]: I0716 00:39:57.321172 2572 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"cni-binary-copy\" (UniqueName: \"kubernetes.io/configmap/a51f2d32-4840-4f0c-804d-c8aa6015ae5a-cni-binary-copy\") pod \"multus-additional-cni-plugins-vfntt\" (UID: \"a51f2d32-4840-4f0c-804d-c8aa6015ae5a\") " pod="openshift-multus/multus-additional-cni-plugins-vfntt" Jul 16 00:39:57.321351 ip-10-0-143-200 kubenswrapper[2572]: I0716 00:39:57.321205 2572 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-cckw2\" (UniqueName: \"kubernetes.io/projected/24fd12c1-ddcb-427f-ac7e-4e2877450cb8-kube-api-access-cckw2\") pod \"node-resolver-6qlgz\" (UID: \"24fd12c1-ddcb-427f-ac7e-4e2877450cb8\") " pod="openshift-dns/node-resolver-6qlgz" Jul 16 00:39:57.321400 ip-10-0-143-200 kubenswrapper[2572]: I0716 00:39:57.321369 2572 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"dbus\" (UniqueName: \"kubernetes.io/host-path/a9bdff50-c8ce-4ca4-a810-20be3909b23c-dbus\") pod \"global-pull-secret-syncer-qcbcd\" (UID: \"a9bdff50-c8ce-4ca4-a810-20be3909b23c\") " pod="kube-system/global-pull-secret-syncer-qcbcd" Jul 16 00:39:57.321451 ip-10-0-143-200 kubenswrapper[2572]: I0716 00:39:57.321400 2572 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"iptables-alerter-script\" (UniqueName: \"kubernetes.io/configmap/0247608f-bb0b-4ace-b071-292d5586fc65-iptables-alerter-script\") pod \"iptables-alerter-gcstg\" (UID: \"0247608f-bb0b-4ace-b071-292d5586fc65\") " pod="openshift-network-operator/iptables-alerter-gcstg" Jul 16 00:39:57.321451 ip-10-0-143-200 kubenswrapper[2572]: I0716 00:39:57.321429 2572 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"os-release\" (UniqueName: \"kubernetes.io/host-path/a51f2d32-4840-4f0c-804d-c8aa6015ae5a-os-release\") pod \"multus-additional-cni-plugins-vfntt\" (UID: \"a51f2d32-4840-4f0c-804d-c8aa6015ae5a\") " pod="openshift-multus/multus-additional-cni-plugins-vfntt" Jul 16 00:39:57.321539 ip-10-0-143-200 kubenswrapper[2572]: I0716 00:39:57.321455 2572 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tuning-conf-dir\" (UniqueName: \"kubernetes.io/host-path/a51f2d32-4840-4f0c-804d-c8aa6015ae5a-tuning-conf-dir\") pod \"multus-additional-cni-plugins-vfntt\" (UID: \"a51f2d32-4840-4f0c-804d-c8aa6015ae5a\") " pod="openshift-multus/multus-additional-cni-plugins-vfntt" Jul 16 00:39:57.321539 ip-10-0-143-200 kubenswrapper[2572]: I0716 00:39:57.321481 2572 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"run\" (UniqueName: \"kubernetes.io/host-path/2614f4d9-8dc4-406b-a5aa-7dd2e7446008-run\") pod \"tuned-w6p89\" (UID: \"2614f4d9-8dc4-406b-a5aa-7dd2e7446008\") " pod="openshift-cluster-node-tuning-operator/tuned-w6p89" Jul 16 00:39:57.321539 ip-10-0-143-200 kubenswrapper[2572]: I0716 00:39:57.321516 2572 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/24fd12c1-ddcb-427f-ac7e-4e2877450cb8-tmp-dir\") pod \"node-resolver-6qlgz\" (UID: \"24fd12c1-ddcb-427f-ac7e-4e2877450cb8\") " pod="openshift-dns/node-resolver-6qlgz" Jul 16 00:39:57.323615 ip-10-0-143-200 kubenswrapper[2572]: I0716 00:39:57.323594 2572 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ovn-kubernetes/ovnkube-node-fp78g" Jul 16 00:39:57.326834 ip-10-0-143-200 kubenswrapper[2572]: I0716 00:39:57.326806 2572 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/multus-szqbq" Jul 16 00:39:57.327490 ip-10-0-143-200 kubenswrapper[2572]: I0716 00:39:57.327302 2572 reflector.go:439] "Caches populated" logger="kubernetes.io/kubelet-serving" type="*v1.CertificateSigningRequest" reflector="k8s.io/client-go/tools/watch/informerwatcher.go:162" Jul 16 00:39:57.329866 ip-10-0-143-200 kubenswrapper[2572]: I0716 00:39:57.329835 2572 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-pq88r" Jul 16 00:39:57.329959 ip-10-0-143-200 kubenswrapper[2572]: I0716 00:39:57.329869 2572 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ovn-kubernetes\"/\"kube-root-ca.crt\"" Jul 16 00:39:57.330294 ip-10-0-143-200 kubenswrapper[2572]: I0716 00:39:57.330276 2572 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ovn-kubernetes\"/\"ovnkube-script-lib\"" Jul 16 00:39:57.330420 ip-10-0-143-200 kubenswrapper[2572]: I0716 00:39:57.330399 2572 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ovn-kubernetes\"/\"openshift-service-ca.crt\"" Jul 16 00:39:57.330571 ip-10-0-143-200 kubenswrapper[2572]: I0716 00:39:57.330542 2572 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ovn-kubernetes\"/\"ovnkube-config\"" Jul 16 00:39:57.330670 ip-10-0-143-200 kubenswrapper[2572]: I0716 00:39:57.330653 2572 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ovn-kubernetes\"/\"env-overrides\"" Jul 16 00:39:57.331040 ip-10-0-143-200 kubenswrapper[2572]: I0716 00:39:57.331021 2572 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ovn-kubernetes\"/\"ovn-node-metrics-cert\"" Jul 16 00:39:57.331215 ip-10-0-143-200 kubenswrapper[2572]: I0716 00:39:57.331197 2572 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ovn-kubernetes\"/\"ovn-kubernetes-node-dockercfg-smd2h\"" Jul 16 00:39:57.331596 ip-10-0-143-200 kubenswrapper[2572]: I0716 00:39:57.331578 2572 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-multus\"/\"default-dockercfg-5trw6\"" Jul 16 00:39:57.331695 ip-10-0-143-200 kubenswrapper[2572]: I0716 00:39:57.331584 2572 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-multus\"/\"multus-daemon-config\"" Jul 16 00:39:57.333800 ip-10-0-143-200 kubenswrapper[2572]: I0716 00:39:57.332715 2572 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-cluster-csi-drivers\"/\"openshift-service-ca.crt\"" Jul 16 00:39:57.333800 ip-10-0-143-200 kubenswrapper[2572]: I0716 00:39:57.332917 2572 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-cluster-csi-drivers\"/\"aws-ebs-csi-driver-node-metrics-serving-cert\"" Jul 16 00:39:57.333800 ip-10-0-143-200 kubenswrapper[2572]: I0716 00:39:57.333242 2572 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-cluster-csi-drivers\"/\"aws-ebs-csi-driver-node-sa-dockercfg-t4tft\"" Jul 16 00:39:57.333800 ip-10-0-143-200 kubenswrapper[2572]: I0716 00:39:57.333578 2572 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-cluster-csi-drivers\"/\"kube-root-ca.crt\"" Jul 16 00:39:57.344830 ip-10-0-143-200 kubenswrapper[2572]: I0716 00:39:57.344803 2572 csr.go:274] "Certificate signing request is approved, waiting to be issued" logger="kubernetes.io/kubelet-serving" csr="csr-s9jgj" Jul 16 00:39:57.351481 ip-10-0-143-200 kubenswrapper[2572]: I0716 00:39:57.351455 2572 csr.go:270] "Certificate signing request is issued" logger="kubernetes.io/kubelet-serving" csr="csr-s9jgj" Jul 16 00:39:57.409549 ip-10-0-143-200 kubenswrapper[2572]: I0716 00:39:57.409518 2572 desired_state_of_world_populator.go:154] "Finished populating initial desired state of world" Jul 16 00:39:57.422225 ip-10-0-143-200 kubenswrapper[2572]: I0716 00:39:57.422194 2572 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"etc-sysconfig\" (UniqueName: \"kubernetes.io/host-path/2614f4d9-8dc4-406b-a5aa-7dd2e7446008-etc-sysconfig\") pod \"tuned-w6p89\" (UID: \"2614f4d9-8dc4-406b-a5aa-7dd2e7446008\") " pod="openshift-cluster-node-tuning-operator/tuned-w6p89" Jul 16 00:39:57.422404 ip-10-0-143-200 kubenswrapper[2572]: I0716 00:39:57.422237 2572 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"multus-daemon-config\" (UniqueName: \"kubernetes.io/configmap/2b89891b-83eb-48c2-abe4-747d1ca26caf-multus-daemon-config\") pod \"multus-szqbq\" (UID: \"2b89891b-83eb-48c2-abe4-747d1ca26caf\") " pod="openshift-multus/multus-szqbq" Jul 16 00:39:57.422404 ip-10-0-143-200 kubenswrapper[2572]: I0716 00:39:57.422257 2572 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"device-dir\" (UniqueName: \"kubernetes.io/host-path/f82be4fd-cc5f-4ff7-8b98-0f7497e4c802-device-dir\") pod \"aws-ebs-csi-driver-node-pq88r\" (UID: \"f82be4fd-cc5f-4ff7-8b98-0f7497e4c802\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-pq88r" Jul 16 00:39:57.422404 ip-10-0-143-200 kubenswrapper[2572]: I0716 00:39:57.422310 2572 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"hosts-file\" (UniqueName: \"kubernetes.io/host-path/24fd12c1-ddcb-427f-ac7e-4e2877450cb8-hosts-file\") pod \"node-resolver-6qlgz\" (UID: \"24fd12c1-ddcb-427f-ac7e-4e2877450cb8\") " pod="openshift-dns/node-resolver-6qlgz" Jul 16 00:39:57.422404 ip-10-0-143-200 kubenswrapper[2572]: I0716 00:39:57.422335 2572 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-var-lib-cni-multus\" (UniqueName: \"kubernetes.io/host-path/2b89891b-83eb-48c2-abe4-747d1ca26caf-host-var-lib-cni-multus\") pod \"multus-szqbq\" (UID: \"2b89891b-83eb-48c2-abe4-747d1ca26caf\") " pod="openshift-multus/multus-szqbq" Jul 16 00:39:57.422404 ip-10-0-143-200 kubenswrapper[2572]: I0716 00:39:57.422345 2572 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"etc-sysconfig\" (UniqueName: \"kubernetes.io/host-path/2614f4d9-8dc4-406b-a5aa-7dd2e7446008-etc-sysconfig\") pod \"tuned-w6p89\" (UID: \"2614f4d9-8dc4-406b-a5aa-7dd2e7446008\") " pod="openshift-cluster-node-tuning-operator/tuned-w6p89" Jul 16 00:39:57.422404 ip-10-0-143-200 kubenswrapper[2572]: I0716 00:39:57.422358 2572 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"ovnkube-config\" (UniqueName: \"kubernetes.io/configmap/aa66f2b6-0ad8-45d0-8ff5-40fc21966b5a-ovnkube-config\") pod \"ovnkube-node-fp78g\" (UID: \"aa66f2b6-0ad8-45d0-8ff5-40fc21966b5a\") " pod="openshift-ovn-kubernetes/ovnkube-node-fp78g" Jul 16 00:39:57.422404 ip-10-0-143-200 kubenswrapper[2572]: I0716 00:39:57.422384 2572 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"whereabouts-flatfile-configmap\" (UniqueName: \"kubernetes.io/configmap/a51f2d32-4840-4f0c-804d-c8aa6015ae5a-whereabouts-flatfile-configmap\") pod \"multus-additional-cni-plugins-vfntt\" (UID: \"a51f2d32-4840-4f0c-804d-c8aa6015ae5a\") " pod="openshift-multus/multus-additional-cni-plugins-vfntt" Jul 16 00:39:57.422835 ip-10-0-143-200 kubenswrapper[2572]: I0716 00:39:57.422412 2572 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-2k4vw\" (UniqueName: \"kubernetes.io/projected/a51f2d32-4840-4f0c-804d-c8aa6015ae5a-kube-api-access-2k4vw\") pod \"multus-additional-cni-plugins-vfntt\" (UID: \"a51f2d32-4840-4f0c-804d-c8aa6015ae5a\") " pod="openshift-multus/multus-additional-cni-plugins-vfntt" Jul 16 00:39:57.422835 ip-10-0-143-200 kubenswrapper[2572]: I0716 00:39:57.422431 2572 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"hosts-file\" (UniqueName: \"kubernetes.io/host-path/24fd12c1-ddcb-427f-ac7e-4e2877450cb8-hosts-file\") pod \"node-resolver-6qlgz\" (UID: \"24fd12c1-ddcb-427f-ac7e-4e2877450cb8\") " pod="openshift-dns/node-resolver-6qlgz" Jul 16 00:39:57.422835 ip-10-0-143-200 kubenswrapper[2572]: I0716 00:39:57.422497 2572 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"etc-systemd\" (UniqueName: \"kubernetes.io/host-path/2614f4d9-8dc4-406b-a5aa-7dd2e7446008-etc-systemd\") pod \"tuned-w6p89\" (UID: \"2614f4d9-8dc4-406b-a5aa-7dd2e7446008\") " pod="openshift-cluster-node-tuning-operator/tuned-w6p89" Jul 16 00:39:57.422835 ip-10-0-143-200 kubenswrapper[2572]: I0716 00:39:57.422543 2572 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"ovnkube-script-lib\" (UniqueName: \"kubernetes.io/configmap/aa66f2b6-0ad8-45d0-8ff5-40fc21966b5a-ovnkube-script-lib\") pod \"ovnkube-node-fp78g\" (UID: \"aa66f2b6-0ad8-45d0-8ff5-40fc21966b5a\") " pod="openshift-ovn-kubernetes/ovnkube-node-fp78g" Jul 16 00:39:57.422835 ip-10-0-143-200 kubenswrapper[2572]: I0716 00:39:57.422569 2572 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"etc-tuned\" (UniqueName: \"kubernetes.io/empty-dir/2614f4d9-8dc4-406b-a5aa-7dd2e7446008-etc-tuned\") pod \"tuned-w6p89\" (UID: \"2614f4d9-8dc4-406b-a5aa-7dd2e7446008\") " pod="openshift-cluster-node-tuning-operator/tuned-w6p89" Jul 16 00:39:57.422835 ip-10-0-143-200 kubenswrapper[2572]: I0716 00:39:57.422592 2572 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-88fdc\" (UniqueName: \"kubernetes.io/projected/2b89891b-83eb-48c2-abe4-747d1ca26caf-kube-api-access-88fdc\") pod \"multus-szqbq\" (UID: \"2b89891b-83eb-48c2-abe4-747d1ca26caf\") " pod="openshift-multus/multus-szqbq" Jul 16 00:39:57.422835 ip-10-0-143-200 kubenswrapper[2572]: I0716 00:39:57.422620 2572 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"socket-dir\" (UniqueName: \"kubernetes.io/host-path/f82be4fd-cc5f-4ff7-8b98-0f7497e4c802-socket-dir\") pod \"aws-ebs-csi-driver-node-pq88r\" (UID: \"f82be4fd-cc5f-4ff7-8b98-0f7497e4c802\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-pq88r" Jul 16 00:39:57.422835 ip-10-0-143-200 kubenswrapper[2572]: I0716 00:39:57.422660 2572 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-kswdz\" (UniqueName: \"kubernetes.io/projected/f82be4fd-cc5f-4ff7-8b98-0f7497e4c802-kube-api-access-kswdz\") pod \"aws-ebs-csi-driver-node-pq88r\" (UID: \"f82be4fd-cc5f-4ff7-8b98-0f7497e4c802\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-pq88r" Jul 16 00:39:57.422835 ip-10-0-143-200 kubenswrapper[2572]: I0716 00:39:57.422683 2572 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"run-ovn\" (UniqueName: \"kubernetes.io/host-path/aa66f2b6-0ad8-45d0-8ff5-40fc21966b5a-run-ovn\") pod \"ovnkube-node-fp78g\" (UID: \"aa66f2b6-0ad8-45d0-8ff5-40fc21966b5a\") " pod="openshift-ovn-kubernetes/ovnkube-node-fp78g" Jul 16 00:39:57.422835 ip-10-0-143-200 kubenswrapper[2572]: I0716 00:39:57.422707 2572 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"etc-modprobe-d\" (UniqueName: \"kubernetes.io/host-path/2614f4d9-8dc4-406b-a5aa-7dd2e7446008-etc-modprobe-d\") pod \"tuned-w6p89\" (UID: \"2614f4d9-8dc4-406b-a5aa-7dd2e7446008\") " pod="openshift-cluster-node-tuning-operator/tuned-w6p89" Jul 16 00:39:57.422835 ip-10-0-143-200 kubenswrapper[2572]: I0716 00:39:57.422753 2572 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"cnibin\" (UniqueName: \"kubernetes.io/host-path/a51f2d32-4840-4f0c-804d-c8aa6015ae5a-cnibin\") pod \"multus-additional-cni-plugins-vfntt\" (UID: \"a51f2d32-4840-4f0c-804d-c8aa6015ae5a\") " pod="openshift-multus/multus-additional-cni-plugins-vfntt" Jul 16 00:39:57.422835 ip-10-0-143-200 kubenswrapper[2572]: I0716 00:39:57.422782 2572 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"cni-binary-copy\" (UniqueName: \"kubernetes.io/configmap/a51f2d32-4840-4f0c-804d-c8aa6015ae5a-cni-binary-copy\") pod \"multus-additional-cni-plugins-vfntt\" (UID: \"a51f2d32-4840-4f0c-804d-c8aa6015ae5a\") " pod="openshift-multus/multus-additional-cni-plugins-vfntt" Jul 16 00:39:57.422835 ip-10-0-143-200 kubenswrapper[2572]: I0716 00:39:57.422807 2572 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"etc-systemd\" (UniqueName: \"kubernetes.io/host-path/2614f4d9-8dc4-406b-a5aa-7dd2e7446008-etc-systemd\") pod \"tuned-w6p89\" (UID: \"2614f4d9-8dc4-406b-a5aa-7dd2e7446008\") " pod="openshift-cluster-node-tuning-operator/tuned-w6p89" Jul 16 00:39:57.422835 ip-10-0-143-200 kubenswrapper[2572]: I0716 00:39:57.422821 2572 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-cckw2\" (UniqueName: \"kubernetes.io/projected/24fd12c1-ddcb-427f-ac7e-4e2877450cb8-kube-api-access-cckw2\") pod \"node-resolver-6qlgz\" (UID: \"24fd12c1-ddcb-427f-ac7e-4e2877450cb8\") " pod="openshift-dns/node-resolver-6qlgz" Jul 16 00:39:57.423454 ip-10-0-143-200 kubenswrapper[2572]: I0716 00:39:57.422905 2572 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"os-release\" (UniqueName: \"kubernetes.io/host-path/2b89891b-83eb-48c2-abe4-747d1ca26caf-os-release\") pod \"multus-szqbq\" (UID: \"2b89891b-83eb-48c2-abe4-747d1ca26caf\") " pod="openshift-multus/multus-szqbq" Jul 16 00:39:57.423454 ip-10-0-143-200 kubenswrapper[2572]: I0716 00:39:57.422918 2572 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"cnibin\" (UniqueName: \"kubernetes.io/host-path/a51f2d32-4840-4f0c-804d-c8aa6015ae5a-cnibin\") pod \"multus-additional-cni-plugins-vfntt\" (UID: \"a51f2d32-4840-4f0c-804d-c8aa6015ae5a\") " pod="openshift-multus/multus-additional-cni-plugins-vfntt" Jul 16 00:39:57.423454 ip-10-0-143-200 kubenswrapper[2572]: I0716 00:39:57.422938 2572 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"multus-socket-dir-parent\" (UniqueName: \"kubernetes.io/host-path/2b89891b-83eb-48c2-abe4-747d1ca26caf-multus-socket-dir-parent\") pod \"multus-szqbq\" (UID: \"2b89891b-83eb-48c2-abe4-747d1ca26caf\") " pod="openshift-multus/multus-szqbq" Jul 16 00:39:57.423454 ip-10-0-143-200 kubenswrapper[2572]: I0716 00:39:57.422965 2572 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-run-multus-certs\" (UniqueName: \"kubernetes.io/host-path/2b89891b-83eb-48c2-abe4-747d1ca26caf-host-run-multus-certs\") pod \"multus-szqbq\" (UID: \"2b89891b-83eb-48c2-abe4-747d1ca26caf\") " pod="openshift-multus/multus-szqbq" Jul 16 00:39:57.423454 ip-10-0-143-200 kubenswrapper[2572]: I0716 00:39:57.423014 2572 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"etc-modprobe-d\" (UniqueName: \"kubernetes.io/host-path/2614f4d9-8dc4-406b-a5aa-7dd2e7446008-etc-modprobe-d\") pod \"tuned-w6p89\" (UID: \"2614f4d9-8dc4-406b-a5aa-7dd2e7446008\") " pod="openshift-cluster-node-tuning-operator/tuned-w6p89" Jul 16 00:39:57.423454 ip-10-0-143-200 kubenswrapper[2572]: I0716 00:39:57.423019 2572 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"iptables-alerter-script\" (UniqueName: \"kubernetes.io/configmap/0247608f-bb0b-4ace-b071-292d5586fc65-iptables-alerter-script\") pod \"iptables-alerter-gcstg\" (UID: \"0247608f-bb0b-4ace-b071-292d5586fc65\") " pod="openshift-network-operator/iptables-alerter-gcstg" Jul 16 00:39:57.423454 ip-10-0-143-200 kubenswrapper[2572]: I0716 00:39:57.423045 2572 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"multus-conf-dir\" (UniqueName: \"kubernetes.io/host-path/2b89891b-83eb-48c2-abe4-747d1ca26caf-multus-conf-dir\") pod \"multus-szqbq\" (UID: \"2b89891b-83eb-48c2-abe4-747d1ca26caf\") " pod="openshift-multus/multus-szqbq" Jul 16 00:39:57.423454 ip-10-0-143-200 kubenswrapper[2572]: I0716 00:39:57.423099 2572 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"whereabouts-flatfile-configmap\" (UniqueName: \"kubernetes.io/configmap/a51f2d32-4840-4f0c-804d-c8aa6015ae5a-whereabouts-flatfile-configmap\") pod \"multus-additional-cni-plugins-vfntt\" (UID: \"a51f2d32-4840-4f0c-804d-c8aa6015ae5a\") " pod="openshift-multus/multus-additional-cni-plugins-vfntt" Jul 16 00:39:57.423454 ip-10-0-143-200 kubenswrapper[2572]: I0716 00:39:57.423172 2572 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"registration-dir\" (UniqueName: \"kubernetes.io/host-path/f82be4fd-cc5f-4ff7-8b98-0f7497e4c802-registration-dir\") pod \"aws-ebs-csi-driver-node-pq88r\" (UID: \"f82be4fd-cc5f-4ff7-8b98-0f7497e4c802\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-pq88r" Jul 16 00:39:57.423454 ip-10-0-143-200 kubenswrapper[2572]: I0716 00:39:57.423379 2572 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"run-systemd\" (UniqueName: \"kubernetes.io/host-path/aa66f2b6-0ad8-45d0-8ff5-40fc21966b5a-run-systemd\") pod \"ovnkube-node-fp78g\" (UID: \"aa66f2b6-0ad8-45d0-8ff5-40fc21966b5a\") " pod="openshift-ovn-kubernetes/ovnkube-node-fp78g" Jul 16 00:39:57.423454 ip-10-0-143-200 kubenswrapper[2572]: I0716 00:39:57.423412 2572 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-nvl2k\" (UniqueName: \"kubernetes.io/projected/4a08cb1b-3bfa-49cf-8ff0-edef0ff944b9-kube-api-access-nvl2k\") pod \"network-metrics-daemon-5tphg\" (UID: \"4a08cb1b-3bfa-49cf-8ff0-edef0ff944b9\") " pod="openshift-multus/network-metrics-daemon-5tphg" Jul 16 00:39:57.424030 ip-10-0-143-200 kubenswrapper[2572]: I0716 00:39:57.423533 2572 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"run\" (UniqueName: \"kubernetes.io/host-path/2614f4d9-8dc4-406b-a5aa-7dd2e7446008-run\") pod \"tuned-w6p89\" (UID: \"2614f4d9-8dc4-406b-a5aa-7dd2e7446008\") " pod="openshift-cluster-node-tuning-operator/tuned-w6p89" Jul 16 00:39:57.424030 ip-10-0-143-200 kubenswrapper[2572]: I0716 00:39:57.423570 2572 swap_util.go:74] "error creating dir to test if tmpfs noswap is enabled. Assuming not supported" mount path="" error="stat /var/lib/kubelet/plugins/kubernetes.io/empty-dir: no such file or directory" Jul 16 00:39:57.424030 ip-10-0-143-200 kubenswrapper[2572]: I0716 00:39:57.423571 2572 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"cnibin\" (UniqueName: \"kubernetes.io/host-path/2b89891b-83eb-48c2-abe4-747d1ca26caf-cnibin\") pod \"multus-szqbq\" (UID: \"2b89891b-83eb-48c2-abe4-747d1ca26caf\") " pod="openshift-multus/multus-szqbq" Jul 16 00:39:57.424030 ip-10-0-143-200 kubenswrapper[2572]: I0716 00:39:57.423670 2572 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-var-lib-kubelet\" (UniqueName: \"kubernetes.io/host-path/2b89891b-83eb-48c2-abe4-747d1ca26caf-host-var-lib-kubelet\") pod \"multus-szqbq\" (UID: \"2b89891b-83eb-48c2-abe4-747d1ca26caf\") " pod="openshift-multus/multus-szqbq" Jul 16 00:39:57.424030 ip-10-0-143-200 kubenswrapper[2572]: I0716 00:39:57.423718 2572 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"node-log\" (UniqueName: \"kubernetes.io/host-path/aa66f2b6-0ad8-45d0-8ff5-40fc21966b5a-node-log\") pod \"ovnkube-node-fp78g\" (UID: \"aa66f2b6-0ad8-45d0-8ff5-40fc21966b5a\") " pod="openshift-ovn-kubernetes/ovnkube-node-fp78g" Jul 16 00:39:57.424030 ip-10-0-143-200 kubenswrapper[2572]: I0716 00:39:57.423747 2572 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"agent-certs\" (UniqueName: \"kubernetes.io/secret/3a7e030d-fad8-480d-9dc6-fe5057048cf9-agent-certs\") pod \"konnectivity-agent-9gnmq\" (UID: \"3a7e030d-fad8-480d-9dc6-fe5057048cf9\") " pod="kube-system/konnectivity-agent-9gnmq" Jul 16 00:39:57.424030 ip-10-0-143-200 kubenswrapper[2572]: I0716 00:39:57.423756 2572 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"cni-binary-copy\" (UniqueName: \"kubernetes.io/configmap/a51f2d32-4840-4f0c-804d-c8aa6015ae5a-cni-binary-copy\") pod \"multus-additional-cni-plugins-vfntt\" (UID: \"a51f2d32-4840-4f0c-804d-c8aa6015ae5a\") " pod="openshift-multus/multus-additional-cni-plugins-vfntt" Jul 16 00:39:57.424030 ip-10-0-143-200 kubenswrapper[2572]: I0716 00:39:57.423772 2572 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"etc-sysctl-d\" (UniqueName: \"kubernetes.io/host-path/2614f4d9-8dc4-406b-a5aa-7dd2e7446008-etc-sysctl-d\") pod \"tuned-w6p89\" (UID: \"2614f4d9-8dc4-406b-a5aa-7dd2e7446008\") " pod="openshift-cluster-node-tuning-operator/tuned-w6p89" Jul 16 00:39:57.424030 ip-10-0-143-200 kubenswrapper[2572]: I0716 00:39:57.423828 2572 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"original-pull-secret\" (UniqueName: \"kubernetes.io/secret/a9bdff50-c8ce-4ca4-a810-20be3909b23c-original-pull-secret\") pod \"global-pull-secret-syncer-qcbcd\" (UID: \"a9bdff50-c8ce-4ca4-a810-20be3909b23c\") " pod="kube-system/global-pull-secret-syncer-qcbcd" Jul 16 00:39:57.424030 ip-10-0-143-200 kubenswrapper[2572]: I0716 00:39:57.423862 2572 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-slash\" (UniqueName: \"kubernetes.io/host-path/0247608f-bb0b-4ace-b071-292d5586fc65-host-slash\") pod \"iptables-alerter-gcstg\" (UID: \"0247608f-bb0b-4ace-b071-292d5586fc65\") " pod="openshift-network-operator/iptables-alerter-gcstg" Jul 16 00:39:57.424030 ip-10-0-143-200 kubenswrapper[2572]: I0716 00:39:57.423919 2572 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"run-openvswitch\" (UniqueName: \"kubernetes.io/host-path/aa66f2b6-0ad8-45d0-8ff5-40fc21966b5a-run-openvswitch\") pod \"ovnkube-node-fp78g\" (UID: \"aa66f2b6-0ad8-45d0-8ff5-40fc21966b5a\") " pod="openshift-ovn-kubernetes/ovnkube-node-fp78g" Jul 16 00:39:57.424030 ip-10-0-143-200 kubenswrapper[2572]: I0716 00:39:57.423929 2572 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"etc-sysctl-d\" (UniqueName: \"kubernetes.io/host-path/2614f4d9-8dc4-406b-a5aa-7dd2e7446008-etc-sysctl-d\") pod \"tuned-w6p89\" (UID: \"2614f4d9-8dc4-406b-a5aa-7dd2e7446008\") " pod="openshift-cluster-node-tuning-operator/tuned-w6p89" Jul 16 00:39:57.424030 ip-10-0-143-200 kubenswrapper[2572]: I0716 00:39:57.423949 2572 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-var-lib-cni-networks-ovn-kubernetes\" (UniqueName: \"kubernetes.io/host-path/aa66f2b6-0ad8-45d0-8ff5-40fc21966b5a-host-var-lib-cni-networks-ovn-kubernetes\") pod \"ovnkube-node-fp78g\" (UID: \"aa66f2b6-0ad8-45d0-8ff5-40fc21966b5a\") " pod="openshift-ovn-kubernetes/ovnkube-node-fp78g" Jul 16 00:39:57.424030 ip-10-0-143-200 kubenswrapper[2572]: I0716 00:39:57.423970 2572 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"iptables-alerter-script\" (UniqueName: \"kubernetes.io/configmap/0247608f-bb0b-4ace-b071-292d5586fc65-iptables-alerter-script\") pod \"iptables-alerter-gcstg\" (UID: \"0247608f-bb0b-4ace-b071-292d5586fc65\") " pod="openshift-network-operator/iptables-alerter-gcstg" Jul 16 00:39:57.424030 ip-10-0-143-200 kubenswrapper[2572]: I0716 00:39:57.424002 2572 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"run\" (UniqueName: \"kubernetes.io/host-path/2614f4d9-8dc4-406b-a5aa-7dd2e7446008-run\") pod \"tuned-w6p89\" (UID: \"2614f4d9-8dc4-406b-a5aa-7dd2e7446008\") " pod="openshift-cluster-node-tuning-operator/tuned-w6p89" Jul 16 00:39:57.424030 ip-10-0-143-200 kubenswrapper[2572]: I0716 00:39:57.423976 2572 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"cni-sysctl-allowlist\" (UniqueName: \"kubernetes.io/configmap/a51f2d32-4840-4f0c-804d-c8aa6015ae5a-cni-sysctl-allowlist\") pod \"multus-additional-cni-plugins-vfntt\" (UID: \"a51f2d32-4840-4f0c-804d-c8aa6015ae5a\") " pod="openshift-multus/multus-additional-cni-plugins-vfntt" Jul 16 00:39:57.424030 ip-10-0-143-200 kubenswrapper[2572]: I0716 00:39:57.424030 2572 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"host-slash\" (UniqueName: \"kubernetes.io/host-path/0247608f-bb0b-4ace-b071-292d5586fc65-host-slash\") pod \"iptables-alerter-gcstg\" (UID: \"0247608f-bb0b-4ace-b071-292d5586fc65\") " pod="openshift-network-operator/iptables-alerter-gcstg" Jul 16 00:39:57.424754 ip-10-0-143-200 kubenswrapper[2572]: E0716 00:39:57.424104 2572 secret.go:189] Couldn't get secret kube-system/original-pull-secret: object "kube-system"/"original-pull-secret" not registered Jul 16 00:39:57.424754 ip-10-0-143-200 kubenswrapper[2572]: I0716 00:39:57.424153 2572 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"etc-sysctl-conf\" (UniqueName: \"kubernetes.io/host-path/2614f4d9-8dc4-406b-a5aa-7dd2e7446008-etc-sysctl-conf\") pod \"tuned-w6p89\" (UID: \"2614f4d9-8dc4-406b-a5aa-7dd2e7446008\") " pod="openshift-cluster-node-tuning-operator/tuned-w6p89" Jul 16 00:39:57.424754 ip-10-0-143-200 kubenswrapper[2572]: I0716 00:39:57.424181 2572 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"lib-modules\" (UniqueName: \"kubernetes.io/host-path/2614f4d9-8dc4-406b-a5aa-7dd2e7446008-lib-modules\") pod \"tuned-w6p89\" (UID: \"2614f4d9-8dc4-406b-a5aa-7dd2e7446008\") " pod="openshift-cluster-node-tuning-operator/tuned-w6p89" Jul 16 00:39:57.424754 ip-10-0-143-200 kubenswrapper[2572]: I0716 00:39:57.424210 2572 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"var-lib-kubelet\" (UniqueName: \"kubernetes.io/host-path/2614f4d9-8dc4-406b-a5aa-7dd2e7446008-var-lib-kubelet\") pod \"tuned-w6p89\" (UID: \"2614f4d9-8dc4-406b-a5aa-7dd2e7446008\") " pod="openshift-cluster-node-tuning-operator/tuned-w6p89" Jul 16 00:39:57.424754 ip-10-0-143-200 kubenswrapper[2572]: I0716 00:39:57.424257 2572 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-kubelet\" (UniqueName: \"kubernetes.io/host-path/aa66f2b6-0ad8-45d0-8ff5-40fc21966b5a-host-kubelet\") pod \"ovnkube-node-fp78g\" (UID: \"aa66f2b6-0ad8-45d0-8ff5-40fc21966b5a\") " pod="openshift-ovn-kubernetes/ovnkube-node-fp78g" Jul 16 00:39:57.424754 ip-10-0-143-200 kubenswrapper[2572]: I0716 00:39:57.424265 2572 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"etc-sysctl-conf\" (UniqueName: \"kubernetes.io/host-path/2614f4d9-8dc4-406b-a5aa-7dd2e7446008-etc-sysctl-conf\") pod \"tuned-w6p89\" (UID: \"2614f4d9-8dc4-406b-a5aa-7dd2e7446008\") " pod="openshift-cluster-node-tuning-operator/tuned-w6p89" Jul 16 00:39:57.424754 ip-10-0-143-200 kubenswrapper[2572]: I0716 00:39:57.424292 2572 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-slash\" (UniqueName: \"kubernetes.io/host-path/aa66f2b6-0ad8-45d0-8ff5-40fc21966b5a-host-slash\") pod \"ovnkube-node-fp78g\" (UID: \"aa66f2b6-0ad8-45d0-8ff5-40fc21966b5a\") " pod="openshift-ovn-kubernetes/ovnkube-node-fp78g" Jul 16 00:39:57.424754 ip-10-0-143-200 kubenswrapper[2572]: I0716 00:39:57.424316 2572 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-cni-bin\" (UniqueName: \"kubernetes.io/host-path/aa66f2b6-0ad8-45d0-8ff5-40fc21966b5a-host-cni-bin\") pod \"ovnkube-node-fp78g\" (UID: \"aa66f2b6-0ad8-45d0-8ff5-40fc21966b5a\") " pod="openshift-ovn-kubernetes/ovnkube-node-fp78g" Jul 16 00:39:57.424754 ip-10-0-143-200 kubenswrapper[2572]: I0716 00:39:57.424319 2572 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"lib-modules\" (UniqueName: \"kubernetes.io/host-path/2614f4d9-8dc4-406b-a5aa-7dd2e7446008-lib-modules\") pod \"tuned-w6p89\" (UID: \"2614f4d9-8dc4-406b-a5aa-7dd2e7446008\") " pod="openshift-cluster-node-tuning-operator/tuned-w6p89" Jul 16 00:39:57.424754 ip-10-0-143-200 kubenswrapper[2572]: I0716 00:39:57.424342 2572 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"etc-kubernetes\" (UniqueName: \"kubernetes.io/host-path/2614f4d9-8dc4-406b-a5aa-7dd2e7446008-etc-kubernetes\") pod \"tuned-w6p89\" (UID: \"2614f4d9-8dc4-406b-a5aa-7dd2e7446008\") " pod="openshift-cluster-node-tuning-operator/tuned-w6p89" Jul 16 00:39:57.424754 ip-10-0-143-200 kubenswrapper[2572]: E0716 00:39:57.424371 2572 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/a9bdff50-c8ce-4ca4-a810-20be3909b23c-original-pull-secret podName:a9bdff50-c8ce-4ca4-a810-20be3909b23c nodeName:}" failed. No retries permitted until 2026-07-16 00:39:57.924346664 +0000 UTC m=+3.033101329 (durationBeforeRetry 500ms). Error: MountVolume.SetUp failed for volume "original-pull-secret" (UniqueName: "kubernetes.io/secret/a9bdff50-c8ce-4ca4-a810-20be3909b23c-original-pull-secret") pod "global-pull-secret-syncer-qcbcd" (UID: "a9bdff50-c8ce-4ca4-a810-20be3909b23c") : object "kube-system"/"original-pull-secret" not registered Jul 16 00:39:57.424754 ip-10-0-143-200 kubenswrapper[2572]: I0716 00:39:57.424404 2572 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"var-lib-kubelet\" (UniqueName: \"kubernetes.io/host-path/2614f4d9-8dc4-406b-a5aa-7dd2e7446008-var-lib-kubelet\") pod \"tuned-w6p89\" (UID: \"2614f4d9-8dc4-406b-a5aa-7dd2e7446008\") " pod="openshift-cluster-node-tuning-operator/tuned-w6p89" Jul 16 00:39:57.424754 ip-10-0-143-200 kubenswrapper[2572]: I0716 00:39:57.424414 2572 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-vt9tv\" (UniqueName: \"kubernetes.io/projected/2614f4d9-8dc4-406b-a5aa-7dd2e7446008-kube-api-access-vt9tv\") pod \"tuned-w6p89\" (UID: \"2614f4d9-8dc4-406b-a5aa-7dd2e7446008\") " pod="openshift-cluster-node-tuning-operator/tuned-w6p89" Jul 16 00:39:57.424754 ip-10-0-143-200 kubenswrapper[2572]: I0716 00:39:57.424406 2572 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"cni-sysctl-allowlist\" (UniqueName: \"kubernetes.io/configmap/a51f2d32-4840-4f0c-804d-c8aa6015ae5a-cni-sysctl-allowlist\") pod \"multus-additional-cni-plugins-vfntt\" (UID: \"a51f2d32-4840-4f0c-804d-c8aa6015ae5a\") " pod="openshift-multus/multus-additional-cni-plugins-vfntt" Jul 16 00:39:57.424754 ip-10-0-143-200 kubenswrapper[2572]: I0716 00:39:57.424451 2572 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"system-cni-dir\" (UniqueName: \"kubernetes.io/host-path/2b89891b-83eb-48c2-abe4-747d1ca26caf-system-cni-dir\") pod \"multus-szqbq\" (UID: \"2b89891b-83eb-48c2-abe4-747d1ca26caf\") " pod="openshift-multus/multus-szqbq" Jul 16 00:39:57.424754 ip-10-0-143-200 kubenswrapper[2572]: I0716 00:39:57.424475 2572 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"etc-kubernetes\" (UniqueName: \"kubernetes.io/host-path/2614f4d9-8dc4-406b-a5aa-7dd2e7446008-etc-kubernetes\") pod \"tuned-w6p89\" (UID: \"2614f4d9-8dc4-406b-a5aa-7dd2e7446008\") " pod="openshift-cluster-node-tuning-operator/tuned-w6p89" Jul 16 00:39:57.424754 ip-10-0-143-200 kubenswrapper[2572]: I0716 00:39:57.424568 2572 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"cni-binary-copy\" (UniqueName: \"kubernetes.io/configmap/2b89891b-83eb-48c2-abe4-747d1ca26caf-cni-binary-copy\") pod \"multus-szqbq\" (UID: \"2b89891b-83eb-48c2-abe4-747d1ca26caf\") " pod="openshift-multus/multus-szqbq" Jul 16 00:39:57.425321 ip-10-0-143-200 kubenswrapper[2572]: I0716 00:39:57.424603 2572 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"systemd-units\" (UniqueName: \"kubernetes.io/host-path/aa66f2b6-0ad8-45d0-8ff5-40fc21966b5a-systemd-units\") pod \"ovnkube-node-fp78g\" (UID: \"aa66f2b6-0ad8-45d0-8ff5-40fc21966b5a\") " pod="openshift-ovn-kubernetes/ovnkube-node-fp78g" Jul 16 00:39:57.425321 ip-10-0-143-200 kubenswrapper[2572]: I0716 00:39:57.424715 2572 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"env-overrides\" (UniqueName: \"kubernetes.io/configmap/aa66f2b6-0ad8-45d0-8ff5-40fc21966b5a-env-overrides\") pod \"ovnkube-node-fp78g\" (UID: \"aa66f2b6-0ad8-45d0-8ff5-40fc21966b5a\") " pod="openshift-ovn-kubernetes/ovnkube-node-fp78g" Jul 16 00:39:57.425321 ip-10-0-143-200 kubenswrapper[2572]: I0716 00:39:57.424802 2572 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-5n9fh\" (UniqueName: \"kubernetes.io/projected/0247608f-bb0b-4ace-b071-292d5586fc65-kube-api-access-5n9fh\") pod \"iptables-alerter-gcstg\" (UID: \"0247608f-bb0b-4ace-b071-292d5586fc65\") " pod="openshift-network-operator/iptables-alerter-gcstg" Jul 16 00:39:57.425321 ip-10-0-143-200 kubenswrapper[2572]: I0716 00:39:57.424868 2572 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"sys\" (UniqueName: \"kubernetes.io/host-path/2614f4d9-8dc4-406b-a5aa-7dd2e7446008-sys\") pod \"tuned-w6p89\" (UID: \"2614f4d9-8dc4-406b-a5aa-7dd2e7446008\") " pod="openshift-cluster-node-tuning-operator/tuned-w6p89" Jul 16 00:39:57.425321 ip-10-0-143-200 kubenswrapper[2572]: I0716 00:39:57.424988 2572 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host\" (UniqueName: \"kubernetes.io/host-path/2614f4d9-8dc4-406b-a5aa-7dd2e7446008-host\") pod \"tuned-w6p89\" (UID: \"2614f4d9-8dc4-406b-a5aa-7dd2e7446008\") " pod="openshift-cluster-node-tuning-operator/tuned-w6p89" Jul 16 00:39:57.425321 ip-10-0-143-200 kubenswrapper[2572]: I0716 00:39:57.425026 2572 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"sys\" (UniqueName: \"kubernetes.io/host-path/2614f4d9-8dc4-406b-a5aa-7dd2e7446008-sys\") pod \"tuned-w6p89\" (UID: \"2614f4d9-8dc4-406b-a5aa-7dd2e7446008\") " pod="openshift-cluster-node-tuning-operator/tuned-w6p89" Jul 16 00:39:57.425321 ip-10-0-143-200 kubenswrapper[2572]: I0716 00:39:57.425068 2572 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kubelet-config\" (UniqueName: \"kubernetes.io/host-path/a9bdff50-c8ce-4ca4-a810-20be3909b23c-kubelet-config\") pod \"global-pull-secret-syncer-qcbcd\" (UID: \"a9bdff50-c8ce-4ca4-a810-20be3909b23c\") " pod="kube-system/global-pull-secret-syncer-qcbcd" Jul 16 00:39:57.425321 ip-10-0-143-200 kubenswrapper[2572]: I0716 00:39:57.425102 2572 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"host\" (UniqueName: \"kubernetes.io/host-path/2614f4d9-8dc4-406b-a5aa-7dd2e7446008-host\") pod \"tuned-w6p89\" (UID: \"2614f4d9-8dc4-406b-a5aa-7dd2e7446008\") " pod="openshift-cluster-node-tuning-operator/tuned-w6p89" Jul 16 00:39:57.425321 ip-10-0-143-200 kubenswrapper[2572]: I0716 00:39:57.425158 2572 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kubelet-config\" (UniqueName: \"kubernetes.io/host-path/a9bdff50-c8ce-4ca4-a810-20be3909b23c-kubelet-config\") pod \"global-pull-secret-syncer-qcbcd\" (UID: \"a9bdff50-c8ce-4ca4-a810-20be3909b23c\") " pod="kube-system/global-pull-secret-syncer-qcbcd" Jul 16 00:39:57.425321 ip-10-0-143-200 kubenswrapper[2572]: I0716 00:39:57.425166 2572 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-run-netns\" (UniqueName: \"kubernetes.io/host-path/2b89891b-83eb-48c2-abe4-747d1ca26caf-host-run-netns\") pod \"multus-szqbq\" (UID: \"2b89891b-83eb-48c2-abe4-747d1ca26caf\") " pod="openshift-multus/multus-szqbq" Jul 16 00:39:57.425321 ip-10-0-143-200 kubenswrapper[2572]: I0716 00:39:57.425203 2572 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"log-socket\" (UniqueName: \"kubernetes.io/host-path/aa66f2b6-0ad8-45d0-8ff5-40fc21966b5a-log-socket\") pod \"ovnkube-node-fp78g\" (UID: \"aa66f2b6-0ad8-45d0-8ff5-40fc21966b5a\") " pod="openshift-ovn-kubernetes/ovnkube-node-fp78g" Jul 16 00:39:57.425321 ip-10-0-143-200 kubenswrapper[2572]: I0716 00:39:57.425229 2572 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-run-ovn-kubernetes\" (UniqueName: \"kubernetes.io/host-path/aa66f2b6-0ad8-45d0-8ff5-40fc21966b5a-host-run-ovn-kubernetes\") pod \"ovnkube-node-fp78g\" (UID: \"aa66f2b6-0ad8-45d0-8ff5-40fc21966b5a\") " pod="openshift-ovn-kubernetes/ovnkube-node-fp78g" Jul 16 00:39:57.425321 ip-10-0-143-200 kubenswrapper[2572]: I0716 00:39:57.425282 2572 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"konnectivity-ca\" (UniqueName: \"kubernetes.io/configmap/3a7e030d-fad8-480d-9dc6-fe5057048cf9-konnectivity-ca\") pod \"konnectivity-agent-9gnmq\" (UID: \"3a7e030d-fad8-480d-9dc6-fe5057048cf9\") " pod="kube-system/konnectivity-agent-9gnmq" Jul 16 00:39:57.425784 ip-10-0-143-200 kubenswrapper[2572]: I0716 00:39:57.425327 2572 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-run-k8s-cni-cncf-io\" (UniqueName: \"kubernetes.io/host-path/2b89891b-83eb-48c2-abe4-747d1ca26caf-host-run-k8s-cni-cncf-io\") pod \"multus-szqbq\" (UID: \"2b89891b-83eb-48c2-abe4-747d1ca26caf\") " pod="openshift-multus/multus-szqbq" Jul 16 00:39:57.425784 ip-10-0-143-200 kubenswrapper[2572]: I0716 00:39:57.425403 2572 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-var-lib-cni-bin\" (UniqueName: \"kubernetes.io/host-path/2b89891b-83eb-48c2-abe4-747d1ca26caf-host-var-lib-cni-bin\") pod \"multus-szqbq\" (UID: \"2b89891b-83eb-48c2-abe4-747d1ca26caf\") " pod="openshift-multus/multus-szqbq" Jul 16 00:39:57.425784 ip-10-0-143-200 kubenswrapper[2572]: I0716 00:39:57.425488 2572 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"dbus\" (UniqueName: \"kubernetes.io/host-path/a9bdff50-c8ce-4ca4-a810-20be3909b23c-dbus\") pod \"global-pull-secret-syncer-qcbcd\" (UID: \"a9bdff50-c8ce-4ca4-a810-20be3909b23c\") " pod="kube-system/global-pull-secret-syncer-qcbcd" Jul 16 00:39:57.425784 ip-10-0-143-200 kubenswrapper[2572]: I0716 00:39:57.425526 2572 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"hostroot\" (UniqueName: \"kubernetes.io/host-path/2b89891b-83eb-48c2-abe4-747d1ca26caf-hostroot\") pod \"multus-szqbq\" (UID: \"2b89891b-83eb-48c2-abe4-747d1ca26caf\") " pod="openshift-multus/multus-szqbq" Jul 16 00:39:57.425784 ip-10-0-143-200 kubenswrapper[2572]: I0716 00:39:57.425550 2572 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"sys-fs\" (UniqueName: \"kubernetes.io/host-path/f82be4fd-cc5f-4ff7-8b98-0f7497e4c802-sys-fs\") pod \"aws-ebs-csi-driver-node-pq88r\" (UID: \"f82be4fd-cc5f-4ff7-8b98-0f7497e4c802\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-pq88r" Jul 16 00:39:57.425784 ip-10-0-143-200 kubenswrapper[2572]: I0716 00:39:57.425594 2572 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"var-lib-openvswitch\" (UniqueName: \"kubernetes.io/host-path/aa66f2b6-0ad8-45d0-8ff5-40fc21966b5a-var-lib-openvswitch\") pod \"ovnkube-node-fp78g\" (UID: \"aa66f2b6-0ad8-45d0-8ff5-40fc21966b5a\") " pod="openshift-ovn-kubernetes/ovnkube-node-fp78g" Jul 16 00:39:57.425784 ip-10-0-143-200 kubenswrapper[2572]: I0716 00:39:57.425644 2572 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-cni-netd\" (UniqueName: \"kubernetes.io/host-path/aa66f2b6-0ad8-45d0-8ff5-40fc21966b5a-host-cni-netd\") pod \"ovnkube-node-fp78g\" (UID: \"aa66f2b6-0ad8-45d0-8ff5-40fc21966b5a\") " pod="openshift-ovn-kubernetes/ovnkube-node-fp78g" Jul 16 00:39:57.425784 ip-10-0-143-200 kubenswrapper[2572]: I0716 00:39:57.425667 2572 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-8lzb7\" (UniqueName: \"kubernetes.io/projected/aa66f2b6-0ad8-45d0-8ff5-40fc21966b5a-kube-api-access-8lzb7\") pod \"ovnkube-node-fp78g\" (UID: \"aa66f2b6-0ad8-45d0-8ff5-40fc21966b5a\") " pod="openshift-ovn-kubernetes/ovnkube-node-fp78g" Jul 16 00:39:57.425784 ip-10-0-143-200 kubenswrapper[2572]: I0716 00:39:57.425709 2572 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kubelet-dir\" (UniqueName: \"kubernetes.io/host-path/f82be4fd-cc5f-4ff7-8b98-0f7497e4c802-kubelet-dir\") pod \"aws-ebs-csi-driver-node-pq88r\" (UID: \"f82be4fd-cc5f-4ff7-8b98-0f7497e4c802\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-pq88r" Jul 16 00:39:57.425784 ip-10-0-143-200 kubenswrapper[2572]: I0716 00:39:57.425751 2572 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"os-release\" (UniqueName: \"kubernetes.io/host-path/a51f2d32-4840-4f0c-804d-c8aa6015ae5a-os-release\") pod \"multus-additional-cni-plugins-vfntt\" (UID: \"a51f2d32-4840-4f0c-804d-c8aa6015ae5a\") " pod="openshift-multus/multus-additional-cni-plugins-vfntt" Jul 16 00:39:57.425784 ip-10-0-143-200 kubenswrapper[2572]: I0716 00:39:57.425783 2572 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tuning-conf-dir\" (UniqueName: \"kubernetes.io/host-path/a51f2d32-4840-4f0c-804d-c8aa6015ae5a-tuning-conf-dir\") pod \"multus-additional-cni-plugins-vfntt\" (UID: \"a51f2d32-4840-4f0c-804d-c8aa6015ae5a\") " pod="openshift-multus/multus-additional-cni-plugins-vfntt" Jul 16 00:39:57.426170 ip-10-0-143-200 kubenswrapper[2572]: I0716 00:39:57.425806 2572 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/24fd12c1-ddcb-427f-ac7e-4e2877450cb8-tmp-dir\") pod \"node-resolver-6qlgz\" (UID: \"24fd12c1-ddcb-427f-ac7e-4e2877450cb8\") " pod="openshift-dns/node-resolver-6qlgz" Jul 16 00:39:57.426170 ip-10-0-143-200 kubenswrapper[2572]: I0716 00:39:57.425829 2572 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"multus-cni-dir\" (UniqueName: \"kubernetes.io/host-path/2b89891b-83eb-48c2-abe4-747d1ca26caf-multus-cni-dir\") pod \"multus-szqbq\" (UID: \"2b89891b-83eb-48c2-abe4-747d1ca26caf\") " pod="openshift-multus/multus-szqbq" Jul 16 00:39:57.426170 ip-10-0-143-200 kubenswrapper[2572]: I0716 00:39:57.425850 2572 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"etc-kubernetes\" (UniqueName: \"kubernetes.io/host-path/2b89891b-83eb-48c2-abe4-747d1ca26caf-etc-kubernetes\") pod \"multus-szqbq\" (UID: \"2b89891b-83eb-48c2-abe4-747d1ca26caf\") " pod="openshift-multus/multus-szqbq" Jul 16 00:39:57.426170 ip-10-0-143-200 kubenswrapper[2572]: I0716 00:39:57.425890 2572 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"konnectivity-ca\" (UniqueName: \"kubernetes.io/configmap/3a7e030d-fad8-480d-9dc6-fe5057048cf9-konnectivity-ca\") pod \"konnectivity-agent-9gnmq\" (UID: \"3a7e030d-fad8-480d-9dc6-fe5057048cf9\") " pod="kube-system/konnectivity-agent-9gnmq" Jul 16 00:39:57.426170 ip-10-0-143-200 kubenswrapper[2572]: I0716 00:39:57.425973 2572 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-kpfbw\" (UniqueName: \"kubernetes.io/projected/8962f109-ea94-4296-a504-f6709315e9cd-kube-api-access-kpfbw\") pod \"network-check-target-52n54\" (UID: \"8962f109-ea94-4296-a504-f6709315e9cd\") " pod="openshift-network-diagnostics/network-check-target-52n54" Jul 16 00:39:57.426170 ip-10-0-143-200 kubenswrapper[2572]: I0716 00:39:57.426011 2572 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"etc-openvswitch\" (UniqueName: \"kubernetes.io/host-path/aa66f2b6-0ad8-45d0-8ff5-40fc21966b5a-etc-openvswitch\") pod \"ovnkube-node-fp78g\" (UID: \"aa66f2b6-0ad8-45d0-8ff5-40fc21966b5a\") " pod="openshift-ovn-kubernetes/ovnkube-node-fp78g" Jul 16 00:39:57.426170 ip-10-0-143-200 kubenswrapper[2572]: I0716 00:39:57.426051 2572 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tmp\" (UniqueName: \"kubernetes.io/empty-dir/2614f4d9-8dc4-406b-a5aa-7dd2e7446008-tmp\") pod \"tuned-w6p89\" (UID: \"2614f4d9-8dc4-406b-a5aa-7dd2e7446008\") " pod="openshift-cluster-node-tuning-operator/tuned-w6p89" Jul 16 00:39:57.426170 ip-10-0-143-200 kubenswrapper[2572]: I0716 00:39:57.426086 2572 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"dbus\" (UniqueName: \"kubernetes.io/host-path/a9bdff50-c8ce-4ca4-a810-20be3909b23c-dbus\") pod \"global-pull-secret-syncer-qcbcd\" (UID: \"a9bdff50-c8ce-4ca4-a810-20be3909b23c\") " pod="kube-system/global-pull-secret-syncer-qcbcd" Jul 16 00:39:57.426499 ip-10-0-143-200 kubenswrapper[2572]: I0716 00:39:57.426171 2572 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"os-release\" (UniqueName: \"kubernetes.io/host-path/a51f2d32-4840-4f0c-804d-c8aa6015ae5a-os-release\") pod \"multus-additional-cni-plugins-vfntt\" (UID: \"a51f2d32-4840-4f0c-804d-c8aa6015ae5a\") " pod="openshift-multus/multus-additional-cni-plugins-vfntt" Jul 16 00:39:57.426499 ip-10-0-143-200 kubenswrapper[2572]: I0716 00:39:57.426113 2572 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"etc-selinux\" (UniqueName: \"kubernetes.io/host-path/f82be4fd-cc5f-4ff7-8b98-0f7497e4c802-etc-selinux\") pod \"aws-ebs-csi-driver-node-pq88r\" (UID: \"f82be4fd-cc5f-4ff7-8b98-0f7497e4c802\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-pq88r" Jul 16 00:39:57.426499 ip-10-0-143-200 kubenswrapper[2572]: I0716 00:39:57.426222 2572 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host\" (UniqueName: \"kubernetes.io/host-path/ff468ef8-66ba-48f2-8531-da662a5b19d3-host\") pod \"node-ca-6msnd\" (UID: \"ff468ef8-66ba-48f2-8531-da662a5b19d3\") " pod="openshift-image-registry/node-ca-6msnd" Jul 16 00:39:57.426499 ip-10-0-143-200 kubenswrapper[2572]: I0716 00:39:57.426268 2572 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"serviceca\" (UniqueName: \"kubernetes.io/configmap/ff468ef8-66ba-48f2-8531-da662a5b19d3-serviceca\") pod \"node-ca-6msnd\" (UID: \"ff468ef8-66ba-48f2-8531-da662a5b19d3\") " pod="openshift-image-registry/node-ca-6msnd" Jul 16 00:39:57.426499 ip-10-0-143-200 kubenswrapper[2572]: I0716 00:39:57.426296 2572 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-zbq79\" (UniqueName: \"kubernetes.io/projected/ff468ef8-66ba-48f2-8531-da662a5b19d3-kube-api-access-zbq79\") pod \"node-ca-6msnd\" (UID: \"ff468ef8-66ba-48f2-8531-da662a5b19d3\") " pod="openshift-image-registry/node-ca-6msnd" Jul 16 00:39:57.426499 ip-10-0-143-200 kubenswrapper[2572]: I0716 00:39:57.426321 2572 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-run-netns\" (UniqueName: \"kubernetes.io/host-path/aa66f2b6-0ad8-45d0-8ff5-40fc21966b5a-host-run-netns\") pod \"ovnkube-node-fp78g\" (UID: \"aa66f2b6-0ad8-45d0-8ff5-40fc21966b5a\") " pod="openshift-ovn-kubernetes/ovnkube-node-fp78g" Jul 16 00:39:57.426889 ip-10-0-143-200 kubenswrapper[2572]: I0716 00:39:57.426865 2572 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"ovn-node-metrics-cert\" (UniqueName: \"kubernetes.io/secret/aa66f2b6-0ad8-45d0-8ff5-40fc21966b5a-ovn-node-metrics-cert\") pod \"ovnkube-node-fp78g\" (UID: \"aa66f2b6-0ad8-45d0-8ff5-40fc21966b5a\") " pod="openshift-ovn-kubernetes/ovnkube-node-fp78g" Jul 16 00:39:57.427096 ip-10-0-143-200 kubenswrapper[2572]: I0716 00:39:57.426892 2572 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/24fd12c1-ddcb-427f-ac7e-4e2877450cb8-tmp-dir\") pod \"node-resolver-6qlgz\" (UID: \"24fd12c1-ddcb-427f-ac7e-4e2877450cb8\") " pod="openshift-dns/node-resolver-6qlgz" Jul 16 00:39:57.427096 ip-10-0-143-200 kubenswrapper[2572]: I0716 00:39:57.426918 2572 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/4a08cb1b-3bfa-49cf-8ff0-edef0ff944b9-metrics-certs\") pod \"network-metrics-daemon-5tphg\" (UID: \"4a08cb1b-3bfa-49cf-8ff0-edef0ff944b9\") " pod="openshift-multus/network-metrics-daemon-5tphg" Jul 16 00:39:57.427096 ip-10-0-143-200 kubenswrapper[2572]: I0716 00:39:57.426950 2572 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"system-cni-dir\" (UniqueName: \"kubernetes.io/host-path/a51f2d32-4840-4f0c-804d-c8aa6015ae5a-system-cni-dir\") pod \"multus-additional-cni-plugins-vfntt\" (UID: \"a51f2d32-4840-4f0c-804d-c8aa6015ae5a\") " pod="openshift-multus/multus-additional-cni-plugins-vfntt" Jul 16 00:39:57.427096 ip-10-0-143-200 kubenswrapper[2572]: I0716 00:39:57.427089 2572 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"system-cni-dir\" (UniqueName: \"kubernetes.io/host-path/a51f2d32-4840-4f0c-804d-c8aa6015ae5a-system-cni-dir\") pod \"multus-additional-cni-plugins-vfntt\" (UID: \"a51f2d32-4840-4f0c-804d-c8aa6015ae5a\") " pod="openshift-multus/multus-additional-cni-plugins-vfntt" Jul 16 00:39:57.427311 ip-10-0-143-200 kubenswrapper[2572]: I0716 00:39:57.427092 2572 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"tuning-conf-dir\" (UniqueName: \"kubernetes.io/host-path/a51f2d32-4840-4f0c-804d-c8aa6015ae5a-tuning-conf-dir\") pod \"multus-additional-cni-plugins-vfntt\" (UID: \"a51f2d32-4840-4f0c-804d-c8aa6015ae5a\") " pod="openshift-multus/multus-additional-cni-plugins-vfntt" Jul 16 00:39:57.427392 ip-10-0-143-200 kubenswrapper[2572]: I0716 00:39:57.427372 2572 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"etc-tuned\" (UniqueName: \"kubernetes.io/empty-dir/2614f4d9-8dc4-406b-a5aa-7dd2e7446008-etc-tuned\") pod \"tuned-w6p89\" (UID: \"2614f4d9-8dc4-406b-a5aa-7dd2e7446008\") " pod="openshift-cluster-node-tuning-operator/tuned-w6p89" Jul 16 00:39:57.427609 ip-10-0-143-200 kubenswrapper[2572]: I0716 00:39:57.427593 2572 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"agent-certs\" (UniqueName: \"kubernetes.io/secret/3a7e030d-fad8-480d-9dc6-fe5057048cf9-agent-certs\") pod \"konnectivity-agent-9gnmq\" (UID: \"3a7e030d-fad8-480d-9dc6-fe5057048cf9\") " pod="kube-system/konnectivity-agent-9gnmq" Jul 16 00:39:57.429967 ip-10-0-143-200 kubenswrapper[2572]: I0716 00:39:57.429184 2572 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"tmp\" (UniqueName: \"kubernetes.io/empty-dir/2614f4d9-8dc4-406b-a5aa-7dd2e7446008-tmp\") pod \"tuned-w6p89\" (UID: \"2614f4d9-8dc4-406b-a5aa-7dd2e7446008\") " pod="openshift-cluster-node-tuning-operator/tuned-w6p89" Jul 16 00:39:57.431696 ip-10-0-143-200 kubenswrapper[2572]: I0716 00:39:57.431256 2572 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-cckw2\" (UniqueName: \"kubernetes.io/projected/24fd12c1-ddcb-427f-ac7e-4e2877450cb8-kube-api-access-cckw2\") pod \"node-resolver-6qlgz\" (UID: \"24fd12c1-ddcb-427f-ac7e-4e2877450cb8\") " pod="openshift-dns/node-resolver-6qlgz" Jul 16 00:39:57.431810 ip-10-0-143-200 kubenswrapper[2572]: I0716 00:39:57.431726 2572 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-2k4vw\" (UniqueName: \"kubernetes.io/projected/a51f2d32-4840-4f0c-804d-c8aa6015ae5a-kube-api-access-2k4vw\") pod \"multus-additional-cni-plugins-vfntt\" (UID: \"a51f2d32-4840-4f0c-804d-c8aa6015ae5a\") " pod="openshift-multus/multus-additional-cni-plugins-vfntt" Jul 16 00:39:57.433971 ip-10-0-143-200 kubenswrapper[2572]: I0716 00:39:57.433953 2572 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-5n9fh\" (UniqueName: \"kubernetes.io/projected/0247608f-bb0b-4ace-b071-292d5586fc65-kube-api-access-5n9fh\") pod \"iptables-alerter-gcstg\" (UID: \"0247608f-bb0b-4ace-b071-292d5586fc65\") " pod="openshift-network-operator/iptables-alerter-gcstg" Jul 16 00:39:57.434087 ip-10-0-143-200 kubenswrapper[2572]: I0716 00:39:57.434056 2572 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-vt9tv\" (UniqueName: \"kubernetes.io/projected/2614f4d9-8dc4-406b-a5aa-7dd2e7446008-kube-api-access-vt9tv\") pod \"tuned-w6p89\" (UID: \"2614f4d9-8dc4-406b-a5aa-7dd2e7446008\") " pod="openshift-cluster-node-tuning-operator/tuned-w6p89" Jul 16 00:39:57.449114 ip-10-0-143-200 kubenswrapper[2572]: I0716 00:39:57.449044 2572 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-143-200.ec2.internal" event={"ID":"df298bc82c8f94d4cb31271396017b60","Type":"ContainerStarted","Data":"335b23f12248d190043b41413c36fc2cde8484dfab250fa09f701f96d41c303e"} Jul 16 00:39:57.450322 ip-10-0-143-200 kubenswrapper[2572]: I0716 00:39:57.450293 2572 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kube-system/kube-apiserver-proxy-ip-10-0-143-200.ec2.internal" event={"ID":"f0dfd958ab883b381e2b09c1edf12ef3","Type":"ContainerStarted","Data":"a7f184f793ea7009ffc73ecb0cee71010bb547ba79c1fb80267d0a5934f30e49"} Jul 16 00:39:57.465058 ip-10-0-143-200 kubenswrapper[2572]: I0716 00:39:57.465027 2572 reflector.go:439] "Caches populated" type="*v1.RuntimeClass" reflector="k8s.io/client-go/informers/factory.go:160" Jul 16 00:39:57.528133 ip-10-0-143-200 kubenswrapper[2572]: I0716 00:39:57.528095 2572 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-kubelet\" (UniqueName: \"kubernetes.io/host-path/aa66f2b6-0ad8-45d0-8ff5-40fc21966b5a-host-kubelet\") pod \"ovnkube-node-fp78g\" (UID: \"aa66f2b6-0ad8-45d0-8ff5-40fc21966b5a\") " pod="openshift-ovn-kubernetes/ovnkube-node-fp78g" Jul 16 00:39:57.528133 ip-10-0-143-200 kubenswrapper[2572]: I0716 00:39:57.528136 2572 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-slash\" (UniqueName: \"kubernetes.io/host-path/aa66f2b6-0ad8-45d0-8ff5-40fc21966b5a-host-slash\") pod \"ovnkube-node-fp78g\" (UID: \"aa66f2b6-0ad8-45d0-8ff5-40fc21966b5a\") " pod="openshift-ovn-kubernetes/ovnkube-node-fp78g" Jul 16 00:39:57.528404 ip-10-0-143-200 kubenswrapper[2572]: I0716 00:39:57.528160 2572 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-cni-bin\" (UniqueName: \"kubernetes.io/host-path/aa66f2b6-0ad8-45d0-8ff5-40fc21966b5a-host-cni-bin\") pod \"ovnkube-node-fp78g\" (UID: \"aa66f2b6-0ad8-45d0-8ff5-40fc21966b5a\") " pod="openshift-ovn-kubernetes/ovnkube-node-fp78g" Jul 16 00:39:57.528404 ip-10-0-143-200 kubenswrapper[2572]: I0716 00:39:57.528188 2572 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"host-kubelet\" (UniqueName: \"kubernetes.io/host-path/aa66f2b6-0ad8-45d0-8ff5-40fc21966b5a-host-kubelet\") pod \"ovnkube-node-fp78g\" (UID: \"aa66f2b6-0ad8-45d0-8ff5-40fc21966b5a\") " pod="openshift-ovn-kubernetes/ovnkube-node-fp78g" Jul 16 00:39:57.528404 ip-10-0-143-200 kubenswrapper[2572]: I0716 00:39:57.528230 2572 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"system-cni-dir\" (UniqueName: \"kubernetes.io/host-path/2b89891b-83eb-48c2-abe4-747d1ca26caf-system-cni-dir\") pod \"multus-szqbq\" (UID: \"2b89891b-83eb-48c2-abe4-747d1ca26caf\") " pod="openshift-multus/multus-szqbq" Jul 16 00:39:57.528404 ip-10-0-143-200 kubenswrapper[2572]: I0716 00:39:57.528243 2572 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"host-cni-bin\" (UniqueName: \"kubernetes.io/host-path/aa66f2b6-0ad8-45d0-8ff5-40fc21966b5a-host-cni-bin\") pod \"ovnkube-node-fp78g\" (UID: \"aa66f2b6-0ad8-45d0-8ff5-40fc21966b5a\") " pod="openshift-ovn-kubernetes/ovnkube-node-fp78g" Jul 16 00:39:57.528404 ip-10-0-143-200 kubenswrapper[2572]: I0716 00:39:57.528260 2572 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"cni-binary-copy\" (UniqueName: \"kubernetes.io/configmap/2b89891b-83eb-48c2-abe4-747d1ca26caf-cni-binary-copy\") pod \"multus-szqbq\" (UID: \"2b89891b-83eb-48c2-abe4-747d1ca26caf\") " pod="openshift-multus/multus-szqbq" Jul 16 00:39:57.528404 ip-10-0-143-200 kubenswrapper[2572]: I0716 00:39:57.528284 2572 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"systemd-units\" (UniqueName: \"kubernetes.io/host-path/aa66f2b6-0ad8-45d0-8ff5-40fc21966b5a-systemd-units\") pod \"ovnkube-node-fp78g\" (UID: \"aa66f2b6-0ad8-45d0-8ff5-40fc21966b5a\") " pod="openshift-ovn-kubernetes/ovnkube-node-fp78g" Jul 16 00:39:57.528404 ip-10-0-143-200 kubenswrapper[2572]: I0716 00:39:57.528286 2572 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"host-slash\" (UniqueName: \"kubernetes.io/host-path/aa66f2b6-0ad8-45d0-8ff5-40fc21966b5a-host-slash\") pod \"ovnkube-node-fp78g\" (UID: \"aa66f2b6-0ad8-45d0-8ff5-40fc21966b5a\") " pod="openshift-ovn-kubernetes/ovnkube-node-fp78g" Jul 16 00:39:57.528404 ip-10-0-143-200 kubenswrapper[2572]: I0716 00:39:57.528303 2572 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"system-cni-dir\" (UniqueName: \"kubernetes.io/host-path/2b89891b-83eb-48c2-abe4-747d1ca26caf-system-cni-dir\") pod \"multus-szqbq\" (UID: \"2b89891b-83eb-48c2-abe4-747d1ca26caf\") " pod="openshift-multus/multus-szqbq" Jul 16 00:39:57.528404 ip-10-0-143-200 kubenswrapper[2572]: I0716 00:39:57.528307 2572 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"env-overrides\" (UniqueName: \"kubernetes.io/configmap/aa66f2b6-0ad8-45d0-8ff5-40fc21966b5a-env-overrides\") pod \"ovnkube-node-fp78g\" (UID: \"aa66f2b6-0ad8-45d0-8ff5-40fc21966b5a\") " pod="openshift-ovn-kubernetes/ovnkube-node-fp78g" Jul 16 00:39:57.528404 ip-10-0-143-200 kubenswrapper[2572]: I0716 00:39:57.528354 2572 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-run-netns\" (UniqueName: \"kubernetes.io/host-path/2b89891b-83eb-48c2-abe4-747d1ca26caf-host-run-netns\") pod \"multus-szqbq\" (UID: \"2b89891b-83eb-48c2-abe4-747d1ca26caf\") " pod="openshift-multus/multus-szqbq" Jul 16 00:39:57.528404 ip-10-0-143-200 kubenswrapper[2572]: I0716 00:39:57.528376 2572 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"log-socket\" (UniqueName: \"kubernetes.io/host-path/aa66f2b6-0ad8-45d0-8ff5-40fc21966b5a-log-socket\") pod \"ovnkube-node-fp78g\" (UID: \"aa66f2b6-0ad8-45d0-8ff5-40fc21966b5a\") " pod="openshift-ovn-kubernetes/ovnkube-node-fp78g" Jul 16 00:39:57.528404 ip-10-0-143-200 kubenswrapper[2572]: I0716 00:39:57.528401 2572 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-run-ovn-kubernetes\" (UniqueName: \"kubernetes.io/host-path/aa66f2b6-0ad8-45d0-8ff5-40fc21966b5a-host-run-ovn-kubernetes\") pod \"ovnkube-node-fp78g\" (UID: \"aa66f2b6-0ad8-45d0-8ff5-40fc21966b5a\") " pod="openshift-ovn-kubernetes/ovnkube-node-fp78g" Jul 16 00:39:57.528961 ip-10-0-143-200 kubenswrapper[2572]: I0716 00:39:57.528424 2572 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-run-k8s-cni-cncf-io\" (UniqueName: \"kubernetes.io/host-path/2b89891b-83eb-48c2-abe4-747d1ca26caf-host-run-k8s-cni-cncf-io\") pod \"multus-szqbq\" (UID: \"2b89891b-83eb-48c2-abe4-747d1ca26caf\") " pod="openshift-multus/multus-szqbq" Jul 16 00:39:57.528961 ip-10-0-143-200 kubenswrapper[2572]: I0716 00:39:57.528444 2572 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-var-lib-cni-bin\" (UniqueName: \"kubernetes.io/host-path/2b89891b-83eb-48c2-abe4-747d1ca26caf-host-var-lib-cni-bin\") pod \"multus-szqbq\" (UID: \"2b89891b-83eb-48c2-abe4-747d1ca26caf\") " pod="openshift-multus/multus-szqbq" Jul 16 00:39:57.528961 ip-10-0-143-200 kubenswrapper[2572]: I0716 00:39:57.528465 2572 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"hostroot\" (UniqueName: \"kubernetes.io/host-path/2b89891b-83eb-48c2-abe4-747d1ca26caf-hostroot\") pod \"multus-szqbq\" (UID: \"2b89891b-83eb-48c2-abe4-747d1ca26caf\") " pod="openshift-multus/multus-szqbq" Jul 16 00:39:57.528961 ip-10-0-143-200 kubenswrapper[2572]: I0716 00:39:57.528487 2572 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"sys-fs\" (UniqueName: \"kubernetes.io/host-path/f82be4fd-cc5f-4ff7-8b98-0f7497e4c802-sys-fs\") pod \"aws-ebs-csi-driver-node-pq88r\" (UID: \"f82be4fd-cc5f-4ff7-8b98-0f7497e4c802\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-pq88r" Jul 16 00:39:57.528961 ip-10-0-143-200 kubenswrapper[2572]: I0716 00:39:57.528510 2572 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"var-lib-openvswitch\" (UniqueName: \"kubernetes.io/host-path/aa66f2b6-0ad8-45d0-8ff5-40fc21966b5a-var-lib-openvswitch\") pod \"ovnkube-node-fp78g\" (UID: \"aa66f2b6-0ad8-45d0-8ff5-40fc21966b5a\") " pod="openshift-ovn-kubernetes/ovnkube-node-fp78g" Jul 16 00:39:57.528961 ip-10-0-143-200 kubenswrapper[2572]: I0716 00:39:57.528534 2572 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-cni-netd\" (UniqueName: \"kubernetes.io/host-path/aa66f2b6-0ad8-45d0-8ff5-40fc21966b5a-host-cni-netd\") pod \"ovnkube-node-fp78g\" (UID: \"aa66f2b6-0ad8-45d0-8ff5-40fc21966b5a\") " pod="openshift-ovn-kubernetes/ovnkube-node-fp78g" Jul 16 00:39:57.528961 ip-10-0-143-200 kubenswrapper[2572]: I0716 00:39:57.528561 2572 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-8lzb7\" (UniqueName: \"kubernetes.io/projected/aa66f2b6-0ad8-45d0-8ff5-40fc21966b5a-kube-api-access-8lzb7\") pod \"ovnkube-node-fp78g\" (UID: \"aa66f2b6-0ad8-45d0-8ff5-40fc21966b5a\") " pod="openshift-ovn-kubernetes/ovnkube-node-fp78g" Jul 16 00:39:57.528961 ip-10-0-143-200 kubenswrapper[2572]: I0716 00:39:57.528584 2572 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kubelet-dir\" (UniqueName: \"kubernetes.io/host-path/f82be4fd-cc5f-4ff7-8b98-0f7497e4c802-kubelet-dir\") pod \"aws-ebs-csi-driver-node-pq88r\" (UID: \"f82be4fd-cc5f-4ff7-8b98-0f7497e4c802\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-pq88r" Jul 16 00:39:57.528961 ip-10-0-143-200 kubenswrapper[2572]: I0716 00:39:57.528611 2572 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"multus-cni-dir\" (UniqueName: \"kubernetes.io/host-path/2b89891b-83eb-48c2-abe4-747d1ca26caf-multus-cni-dir\") pod \"multus-szqbq\" (UID: \"2b89891b-83eb-48c2-abe4-747d1ca26caf\") " pod="openshift-multus/multus-szqbq" Jul 16 00:39:57.528961 ip-10-0-143-200 kubenswrapper[2572]: I0716 00:39:57.528651 2572 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"etc-kubernetes\" (UniqueName: \"kubernetes.io/host-path/2b89891b-83eb-48c2-abe4-747d1ca26caf-etc-kubernetes\") pod \"multus-szqbq\" (UID: \"2b89891b-83eb-48c2-abe4-747d1ca26caf\") " pod="openshift-multus/multus-szqbq" Jul 16 00:39:57.528961 ip-10-0-143-200 kubenswrapper[2572]: I0716 00:39:57.528677 2572 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-kpfbw\" (UniqueName: \"kubernetes.io/projected/8962f109-ea94-4296-a504-f6709315e9cd-kube-api-access-kpfbw\") pod \"network-check-target-52n54\" (UID: \"8962f109-ea94-4296-a504-f6709315e9cd\") " pod="openshift-network-diagnostics/network-check-target-52n54" Jul 16 00:39:57.528961 ip-10-0-143-200 kubenswrapper[2572]: I0716 00:39:57.528702 2572 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"etc-openvswitch\" (UniqueName: \"kubernetes.io/host-path/aa66f2b6-0ad8-45d0-8ff5-40fc21966b5a-etc-openvswitch\") pod \"ovnkube-node-fp78g\" (UID: \"aa66f2b6-0ad8-45d0-8ff5-40fc21966b5a\") " pod="openshift-ovn-kubernetes/ovnkube-node-fp78g" Jul 16 00:39:57.528961 ip-10-0-143-200 kubenswrapper[2572]: I0716 00:39:57.528728 2572 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"etc-selinux\" (UniqueName: \"kubernetes.io/host-path/f82be4fd-cc5f-4ff7-8b98-0f7497e4c802-etc-selinux\") pod \"aws-ebs-csi-driver-node-pq88r\" (UID: \"f82be4fd-cc5f-4ff7-8b98-0f7497e4c802\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-pq88r" Jul 16 00:39:57.528961 ip-10-0-143-200 kubenswrapper[2572]: I0716 00:39:57.528755 2572 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host\" (UniqueName: \"kubernetes.io/host-path/ff468ef8-66ba-48f2-8531-da662a5b19d3-host\") pod \"node-ca-6msnd\" (UID: \"ff468ef8-66ba-48f2-8531-da662a5b19d3\") " pod="openshift-image-registry/node-ca-6msnd" Jul 16 00:39:57.528961 ip-10-0-143-200 kubenswrapper[2572]: I0716 00:39:57.528779 2572 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"serviceca\" (UniqueName: \"kubernetes.io/configmap/ff468ef8-66ba-48f2-8531-da662a5b19d3-serviceca\") pod \"node-ca-6msnd\" (UID: \"ff468ef8-66ba-48f2-8531-da662a5b19d3\") " pod="openshift-image-registry/node-ca-6msnd" Jul 16 00:39:57.528961 ip-10-0-143-200 kubenswrapper[2572]: I0716 00:39:57.528800 2572 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-zbq79\" (UniqueName: \"kubernetes.io/projected/ff468ef8-66ba-48f2-8531-da662a5b19d3-kube-api-access-zbq79\") pod \"node-ca-6msnd\" (UID: \"ff468ef8-66ba-48f2-8531-da662a5b19d3\") " pod="openshift-image-registry/node-ca-6msnd" Jul 16 00:39:57.528961 ip-10-0-143-200 kubenswrapper[2572]: I0716 00:39:57.528838 2572 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-run-netns\" (UniqueName: \"kubernetes.io/host-path/aa66f2b6-0ad8-45d0-8ff5-40fc21966b5a-host-run-netns\") pod \"ovnkube-node-fp78g\" (UID: \"aa66f2b6-0ad8-45d0-8ff5-40fc21966b5a\") " pod="openshift-ovn-kubernetes/ovnkube-node-fp78g" Jul 16 00:39:57.529585 ip-10-0-143-200 kubenswrapper[2572]: I0716 00:39:57.528862 2572 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"ovn-node-metrics-cert\" (UniqueName: \"kubernetes.io/secret/aa66f2b6-0ad8-45d0-8ff5-40fc21966b5a-ovn-node-metrics-cert\") pod \"ovnkube-node-fp78g\" (UID: \"aa66f2b6-0ad8-45d0-8ff5-40fc21966b5a\") " pod="openshift-ovn-kubernetes/ovnkube-node-fp78g" Jul 16 00:39:57.529585 ip-10-0-143-200 kubenswrapper[2572]: I0716 00:39:57.528865 2572 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"env-overrides\" (UniqueName: \"kubernetes.io/configmap/aa66f2b6-0ad8-45d0-8ff5-40fc21966b5a-env-overrides\") pod \"ovnkube-node-fp78g\" (UID: \"aa66f2b6-0ad8-45d0-8ff5-40fc21966b5a\") " pod="openshift-ovn-kubernetes/ovnkube-node-fp78g" Jul 16 00:39:57.529585 ip-10-0-143-200 kubenswrapper[2572]: I0716 00:39:57.528885 2572 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/4a08cb1b-3bfa-49cf-8ff0-edef0ff944b9-metrics-certs\") pod \"network-metrics-daemon-5tphg\" (UID: \"4a08cb1b-3bfa-49cf-8ff0-edef0ff944b9\") " pod="openshift-multus/network-metrics-daemon-5tphg" Jul 16 00:39:57.529585 ip-10-0-143-200 kubenswrapper[2572]: I0716 00:39:57.528916 2572 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"multus-daemon-config\" (UniqueName: \"kubernetes.io/configmap/2b89891b-83eb-48c2-abe4-747d1ca26caf-multus-daemon-config\") pod \"multus-szqbq\" (UID: \"2b89891b-83eb-48c2-abe4-747d1ca26caf\") " pod="openshift-multus/multus-szqbq" Jul 16 00:39:57.529585 ip-10-0-143-200 kubenswrapper[2572]: I0716 00:39:57.528924 2572 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"systemd-units\" (UniqueName: \"kubernetes.io/host-path/aa66f2b6-0ad8-45d0-8ff5-40fc21966b5a-systemd-units\") pod \"ovnkube-node-fp78g\" (UID: \"aa66f2b6-0ad8-45d0-8ff5-40fc21966b5a\") " pod="openshift-ovn-kubernetes/ovnkube-node-fp78g" Jul 16 00:39:57.529585 ip-10-0-143-200 kubenswrapper[2572]: I0716 00:39:57.528941 2572 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"device-dir\" (UniqueName: \"kubernetes.io/host-path/f82be4fd-cc5f-4ff7-8b98-0f7497e4c802-device-dir\") pod \"aws-ebs-csi-driver-node-pq88r\" (UID: \"f82be4fd-cc5f-4ff7-8b98-0f7497e4c802\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-pq88r" Jul 16 00:39:57.529585 ip-10-0-143-200 kubenswrapper[2572]: I0716 00:39:57.528967 2572 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-var-lib-cni-multus\" (UniqueName: \"kubernetes.io/host-path/2b89891b-83eb-48c2-abe4-747d1ca26caf-host-var-lib-cni-multus\") pod \"multus-szqbq\" (UID: \"2b89891b-83eb-48c2-abe4-747d1ca26caf\") " pod="openshift-multus/multus-szqbq" Jul 16 00:39:57.529585 ip-10-0-143-200 kubenswrapper[2572]: I0716 00:39:57.528988 2572 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"multus-cni-dir\" (UniqueName: \"kubernetes.io/host-path/2b89891b-83eb-48c2-abe4-747d1ca26caf-multus-cni-dir\") pod \"multus-szqbq\" (UID: \"2b89891b-83eb-48c2-abe4-747d1ca26caf\") " pod="openshift-multus/multus-szqbq" Jul 16 00:39:57.529585 ip-10-0-143-200 kubenswrapper[2572]: I0716 00:39:57.528994 2572 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"ovnkube-config\" (UniqueName: \"kubernetes.io/configmap/aa66f2b6-0ad8-45d0-8ff5-40fc21966b5a-ovnkube-config\") pod \"ovnkube-node-fp78g\" (UID: \"aa66f2b6-0ad8-45d0-8ff5-40fc21966b5a\") " pod="openshift-ovn-kubernetes/ovnkube-node-fp78g" Jul 16 00:39:57.529585 ip-10-0-143-200 kubenswrapper[2572]: I0716 00:39:57.529017 2572 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"cni-binary-copy\" (UniqueName: \"kubernetes.io/configmap/2b89891b-83eb-48c2-abe4-747d1ca26caf-cni-binary-copy\") pod \"multus-szqbq\" (UID: \"2b89891b-83eb-48c2-abe4-747d1ca26caf\") " pod="openshift-multus/multus-szqbq" Jul 16 00:39:57.529585 ip-10-0-143-200 kubenswrapper[2572]: I0716 00:39:57.529032 2572 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"ovnkube-script-lib\" (UniqueName: \"kubernetes.io/configmap/aa66f2b6-0ad8-45d0-8ff5-40fc21966b5a-ovnkube-script-lib\") pod \"ovnkube-node-fp78g\" (UID: \"aa66f2b6-0ad8-45d0-8ff5-40fc21966b5a\") " pod="openshift-ovn-kubernetes/ovnkube-node-fp78g" Jul 16 00:39:57.529585 ip-10-0-143-200 kubenswrapper[2572]: I0716 00:39:57.529059 2572 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-88fdc\" (UniqueName: \"kubernetes.io/projected/2b89891b-83eb-48c2-abe4-747d1ca26caf-kube-api-access-88fdc\") pod \"multus-szqbq\" (UID: \"2b89891b-83eb-48c2-abe4-747d1ca26caf\") " pod="openshift-multus/multus-szqbq" Jul 16 00:39:57.529585 ip-10-0-143-200 kubenswrapper[2572]: I0716 00:39:57.529067 2572 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"log-socket\" (UniqueName: \"kubernetes.io/host-path/aa66f2b6-0ad8-45d0-8ff5-40fc21966b5a-log-socket\") pod \"ovnkube-node-fp78g\" (UID: \"aa66f2b6-0ad8-45d0-8ff5-40fc21966b5a\") " pod="openshift-ovn-kubernetes/ovnkube-node-fp78g" Jul 16 00:39:57.529585 ip-10-0-143-200 kubenswrapper[2572]: I0716 00:39:57.529082 2572 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"socket-dir\" (UniqueName: \"kubernetes.io/host-path/f82be4fd-cc5f-4ff7-8b98-0f7497e4c802-socket-dir\") pod \"aws-ebs-csi-driver-node-pq88r\" (UID: \"f82be4fd-cc5f-4ff7-8b98-0f7497e4c802\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-pq88r" Jul 16 00:39:57.529585 ip-10-0-143-200 kubenswrapper[2572]: I0716 00:39:57.529091 2572 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"sys-fs\" (UniqueName: \"kubernetes.io/host-path/f82be4fd-cc5f-4ff7-8b98-0f7497e4c802-sys-fs\") pod \"aws-ebs-csi-driver-node-pq88r\" (UID: \"f82be4fd-cc5f-4ff7-8b98-0f7497e4c802\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-pq88r" Jul 16 00:39:57.529585 ip-10-0-143-200 kubenswrapper[2572]: I0716 00:39:57.529108 2572 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-kswdz\" (UniqueName: \"kubernetes.io/projected/f82be4fd-cc5f-4ff7-8b98-0f7497e4c802-kube-api-access-kswdz\") pod \"aws-ebs-csi-driver-node-pq88r\" (UID: \"f82be4fd-cc5f-4ff7-8b98-0f7497e4c802\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-pq88r" Jul 16 00:39:57.529585 ip-10-0-143-200 kubenswrapper[2572]: I0716 00:39:57.529111 2572 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"var-lib-openvswitch\" (UniqueName: \"kubernetes.io/host-path/aa66f2b6-0ad8-45d0-8ff5-40fc21966b5a-var-lib-openvswitch\") pod \"ovnkube-node-fp78g\" (UID: \"aa66f2b6-0ad8-45d0-8ff5-40fc21966b5a\") " pod="openshift-ovn-kubernetes/ovnkube-node-fp78g" Jul 16 00:39:57.530285 ip-10-0-143-200 kubenswrapper[2572]: I0716 00:39:57.529132 2572 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"run-ovn\" (UniqueName: \"kubernetes.io/host-path/aa66f2b6-0ad8-45d0-8ff5-40fc21966b5a-run-ovn\") pod \"ovnkube-node-fp78g\" (UID: \"aa66f2b6-0ad8-45d0-8ff5-40fc21966b5a\") " pod="openshift-ovn-kubernetes/ovnkube-node-fp78g" Jul 16 00:39:57.530285 ip-10-0-143-200 kubenswrapper[2572]: I0716 00:39:57.529157 2572 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"host-run-ovn-kubernetes\" (UniqueName: \"kubernetes.io/host-path/aa66f2b6-0ad8-45d0-8ff5-40fc21966b5a-host-run-ovn-kubernetes\") pod \"ovnkube-node-fp78g\" (UID: \"aa66f2b6-0ad8-45d0-8ff5-40fc21966b5a\") " pod="openshift-ovn-kubernetes/ovnkube-node-fp78g" Jul 16 00:39:57.530285 ip-10-0-143-200 kubenswrapper[2572]: I0716 00:39:57.529162 2572 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"os-release\" (UniqueName: \"kubernetes.io/host-path/2b89891b-83eb-48c2-abe4-747d1ca26caf-os-release\") pod \"multus-szqbq\" (UID: \"2b89891b-83eb-48c2-abe4-747d1ca26caf\") " pod="openshift-multus/multus-szqbq" Jul 16 00:39:57.530285 ip-10-0-143-200 kubenswrapper[2572]: I0716 00:39:57.529170 2572 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kubelet-dir\" (UniqueName: \"kubernetes.io/host-path/f82be4fd-cc5f-4ff7-8b98-0f7497e4c802-kubelet-dir\") pod \"aws-ebs-csi-driver-node-pq88r\" (UID: \"f82be4fd-cc5f-4ff7-8b98-0f7497e4c802\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-pq88r" Jul 16 00:39:57.530285 ip-10-0-143-200 kubenswrapper[2572]: I0716 00:39:57.529194 2572 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"multus-socket-dir-parent\" (UniqueName: \"kubernetes.io/host-path/2b89891b-83eb-48c2-abe4-747d1ca26caf-multus-socket-dir-parent\") pod \"multus-szqbq\" (UID: \"2b89891b-83eb-48c2-abe4-747d1ca26caf\") " pod="openshift-multus/multus-szqbq" Jul 16 00:39:57.530285 ip-10-0-143-200 kubenswrapper[2572]: I0716 00:39:57.529218 2572 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-run-multus-certs\" (UniqueName: \"kubernetes.io/host-path/2b89891b-83eb-48c2-abe4-747d1ca26caf-host-run-multus-certs\") pod \"multus-szqbq\" (UID: \"2b89891b-83eb-48c2-abe4-747d1ca26caf\") " pod="openshift-multus/multus-szqbq" Jul 16 00:39:57.530285 ip-10-0-143-200 kubenswrapper[2572]: I0716 00:39:57.529229 2572 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"os-release\" (UniqueName: \"kubernetes.io/host-path/2b89891b-83eb-48c2-abe4-747d1ca26caf-os-release\") pod \"multus-szqbq\" (UID: \"2b89891b-83eb-48c2-abe4-747d1ca26caf\") " pod="openshift-multus/multus-szqbq" Jul 16 00:39:57.530285 ip-10-0-143-200 kubenswrapper[2572]: I0716 00:39:57.529264 2572 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"etc-kubernetes\" (UniqueName: \"kubernetes.io/host-path/2b89891b-83eb-48c2-abe4-747d1ca26caf-etc-kubernetes\") pod \"multus-szqbq\" (UID: \"2b89891b-83eb-48c2-abe4-747d1ca26caf\") " pod="openshift-multus/multus-szqbq" Jul 16 00:39:57.530285 ip-10-0-143-200 kubenswrapper[2572]: I0716 00:39:57.529302 2572 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"multus-conf-dir\" (UniqueName: \"kubernetes.io/host-path/2b89891b-83eb-48c2-abe4-747d1ca26caf-multus-conf-dir\") pod \"multus-szqbq\" (UID: \"2b89891b-83eb-48c2-abe4-747d1ca26caf\") " pod="openshift-multus/multus-szqbq" Jul 16 00:39:57.530285 ip-10-0-143-200 kubenswrapper[2572]: I0716 00:39:57.529312 2572 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"device-dir\" (UniqueName: \"kubernetes.io/host-path/f82be4fd-cc5f-4ff7-8b98-0f7497e4c802-device-dir\") pod \"aws-ebs-csi-driver-node-pq88r\" (UID: \"f82be4fd-cc5f-4ff7-8b98-0f7497e4c802\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-pq88r" Jul 16 00:39:57.530285 ip-10-0-143-200 kubenswrapper[2572]: I0716 00:39:57.529326 2572 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"registration-dir\" (UniqueName: \"kubernetes.io/host-path/f82be4fd-cc5f-4ff7-8b98-0f7497e4c802-registration-dir\") pod \"aws-ebs-csi-driver-node-pq88r\" (UID: \"f82be4fd-cc5f-4ff7-8b98-0f7497e4c802\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-pq88r" Jul 16 00:39:57.530285 ip-10-0-143-200 kubenswrapper[2572]: I0716 00:39:57.529352 2572 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"run-systemd\" (UniqueName: \"kubernetes.io/host-path/aa66f2b6-0ad8-45d0-8ff5-40fc21966b5a-run-systemd\") pod \"ovnkube-node-fp78g\" (UID: \"aa66f2b6-0ad8-45d0-8ff5-40fc21966b5a\") " pod="openshift-ovn-kubernetes/ovnkube-node-fp78g" Jul 16 00:39:57.530285 ip-10-0-143-200 kubenswrapper[2572]: I0716 00:39:57.529377 2572 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-nvl2k\" (UniqueName: \"kubernetes.io/projected/4a08cb1b-3bfa-49cf-8ff0-edef0ff944b9-kube-api-access-nvl2k\") pod \"network-metrics-daemon-5tphg\" (UID: \"4a08cb1b-3bfa-49cf-8ff0-edef0ff944b9\") " pod="openshift-multus/network-metrics-daemon-5tphg" Jul 16 00:39:57.530285 ip-10-0-143-200 kubenswrapper[2572]: I0716 00:39:57.529394 2572 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"host\" (UniqueName: \"kubernetes.io/host-path/ff468ef8-66ba-48f2-8531-da662a5b19d3-host\") pod \"node-ca-6msnd\" (UID: \"ff468ef8-66ba-48f2-8531-da662a5b19d3\") " pod="openshift-image-registry/node-ca-6msnd" Jul 16 00:39:57.530285 ip-10-0-143-200 kubenswrapper[2572]: I0716 00:39:57.529402 2572 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"cnibin\" (UniqueName: \"kubernetes.io/host-path/2b89891b-83eb-48c2-abe4-747d1ca26caf-cnibin\") pod \"multus-szqbq\" (UID: \"2b89891b-83eb-48c2-abe4-747d1ca26caf\") " pod="openshift-multus/multus-szqbq" Jul 16 00:39:57.530285 ip-10-0-143-200 kubenswrapper[2572]: I0716 00:39:57.529424 2572 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-var-lib-kubelet\" (UniqueName: \"kubernetes.io/host-path/2b89891b-83eb-48c2-abe4-747d1ca26caf-host-var-lib-kubelet\") pod \"multus-szqbq\" (UID: \"2b89891b-83eb-48c2-abe4-747d1ca26caf\") " pod="openshift-multus/multus-szqbq" Jul 16 00:39:57.530285 ip-10-0-143-200 kubenswrapper[2572]: I0716 00:39:57.529451 2572 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"node-log\" (UniqueName: \"kubernetes.io/host-path/aa66f2b6-0ad8-45d0-8ff5-40fc21966b5a-node-log\") pod \"ovnkube-node-fp78g\" (UID: \"aa66f2b6-0ad8-45d0-8ff5-40fc21966b5a\") " pod="openshift-ovn-kubernetes/ovnkube-node-fp78g" Jul 16 00:39:57.530285 ip-10-0-143-200 kubenswrapper[2572]: I0716 00:39:57.529482 2572 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"host-var-lib-cni-multus\" (UniqueName: \"kubernetes.io/host-path/2b89891b-83eb-48c2-abe4-747d1ca26caf-host-var-lib-cni-multus\") pod \"multus-szqbq\" (UID: \"2b89891b-83eb-48c2-abe4-747d1ca26caf\") " pod="openshift-multus/multus-szqbq" Jul 16 00:39:57.531122 ip-10-0-143-200 kubenswrapper[2572]: I0716 00:39:57.529494 2572 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"run-openvswitch\" (UniqueName: \"kubernetes.io/host-path/aa66f2b6-0ad8-45d0-8ff5-40fc21966b5a-run-openvswitch\") pod \"ovnkube-node-fp78g\" (UID: \"aa66f2b6-0ad8-45d0-8ff5-40fc21966b5a\") " pod="openshift-ovn-kubernetes/ovnkube-node-fp78g" Jul 16 00:39:57.531122 ip-10-0-143-200 kubenswrapper[2572]: I0716 00:39:57.529538 2572 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"run-openvswitch\" (UniqueName: \"kubernetes.io/host-path/aa66f2b6-0ad8-45d0-8ff5-40fc21966b5a-run-openvswitch\") pod \"ovnkube-node-fp78g\" (UID: \"aa66f2b6-0ad8-45d0-8ff5-40fc21966b5a\") " pod="openshift-ovn-kubernetes/ovnkube-node-fp78g" Jul 16 00:39:57.531122 ip-10-0-143-200 kubenswrapper[2572]: I0716 00:39:57.529580 2572 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"host-run-k8s-cni-cncf-io\" (UniqueName: \"kubernetes.io/host-path/2b89891b-83eb-48c2-abe4-747d1ca26caf-host-run-k8s-cni-cncf-io\") pod \"multus-szqbq\" (UID: \"2b89891b-83eb-48c2-abe4-747d1ca26caf\") " pod="openshift-multus/multus-szqbq" Jul 16 00:39:57.531122 ip-10-0-143-200 kubenswrapper[2572]: I0716 00:39:57.529619 2572 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"host-var-lib-cni-bin\" (UniqueName: \"kubernetes.io/host-path/2b89891b-83eb-48c2-abe4-747d1ca26caf-host-var-lib-cni-bin\") pod \"multus-szqbq\" (UID: \"2b89891b-83eb-48c2-abe4-747d1ca26caf\") " pod="openshift-multus/multus-szqbq" Jul 16 00:39:57.531122 ip-10-0-143-200 kubenswrapper[2572]: I0716 00:39:57.529678 2572 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"hostroot\" (UniqueName: \"kubernetes.io/host-path/2b89891b-83eb-48c2-abe4-747d1ca26caf-hostroot\") pod \"multus-szqbq\" (UID: \"2b89891b-83eb-48c2-abe4-747d1ca26caf\") " pod="openshift-multus/multus-szqbq" Jul 16 00:39:57.531122 ip-10-0-143-200 kubenswrapper[2572]: I0716 00:39:57.529714 2572 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"host-cni-netd\" (UniqueName: \"kubernetes.io/host-path/aa66f2b6-0ad8-45d0-8ff5-40fc21966b5a-host-cni-netd\") pod \"ovnkube-node-fp78g\" (UID: \"aa66f2b6-0ad8-45d0-8ff5-40fc21966b5a\") " pod="openshift-ovn-kubernetes/ovnkube-node-fp78g" Jul 16 00:39:57.531122 ip-10-0-143-200 kubenswrapper[2572]: I0716 00:39:57.529807 2572 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"serviceca\" (UniqueName: \"kubernetes.io/configmap/ff468ef8-66ba-48f2-8531-da662a5b19d3-serviceca\") pod \"node-ca-6msnd\" (UID: \"ff468ef8-66ba-48f2-8531-da662a5b19d3\") " pod="openshift-image-registry/node-ca-6msnd" Jul 16 00:39:57.531122 ip-10-0-143-200 kubenswrapper[2572]: I0716 00:39:57.529971 2572 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"multus-socket-dir-parent\" (UniqueName: \"kubernetes.io/host-path/2b89891b-83eb-48c2-abe4-747d1ca26caf-multus-socket-dir-parent\") pod \"multus-szqbq\" (UID: \"2b89891b-83eb-48c2-abe4-747d1ca26caf\") " pod="openshift-multus/multus-szqbq" Jul 16 00:39:57.531122 ip-10-0-143-200 kubenswrapper[2572]: I0716 00:39:57.529988 2572 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"ovnkube-config\" (UniqueName: \"kubernetes.io/configmap/aa66f2b6-0ad8-45d0-8ff5-40fc21966b5a-ovnkube-config\") pod \"ovnkube-node-fp78g\" (UID: \"aa66f2b6-0ad8-45d0-8ff5-40fc21966b5a\") " pod="openshift-ovn-kubernetes/ovnkube-node-fp78g" Jul 16 00:39:57.531122 ip-10-0-143-200 kubenswrapper[2572]: I0716 00:39:57.530032 2572 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"host-run-multus-certs\" (UniqueName: \"kubernetes.io/host-path/2b89891b-83eb-48c2-abe4-747d1ca26caf-host-run-multus-certs\") pod \"multus-szqbq\" (UID: \"2b89891b-83eb-48c2-abe4-747d1ca26caf\") " pod="openshift-multus/multus-szqbq" Jul 16 00:39:57.531122 ip-10-0-143-200 kubenswrapper[2572]: I0716 00:39:57.530062 2572 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-var-lib-cni-networks-ovn-kubernetes\" (UniqueName: \"kubernetes.io/host-path/aa66f2b6-0ad8-45d0-8ff5-40fc21966b5a-host-var-lib-cni-networks-ovn-kubernetes\") pod \"ovnkube-node-fp78g\" (UID: \"aa66f2b6-0ad8-45d0-8ff5-40fc21966b5a\") " pod="openshift-ovn-kubernetes/ovnkube-node-fp78g" Jul 16 00:39:57.531122 ip-10-0-143-200 kubenswrapper[2572]: I0716 00:39:57.530069 2572 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"host-run-netns\" (UniqueName: \"kubernetes.io/host-path/aa66f2b6-0ad8-45d0-8ff5-40fc21966b5a-host-run-netns\") pod \"ovnkube-node-fp78g\" (UID: \"aa66f2b6-0ad8-45d0-8ff5-40fc21966b5a\") " pod="openshift-ovn-kubernetes/ovnkube-node-fp78g" Jul 16 00:39:57.531122 ip-10-0-143-200 kubenswrapper[2572]: I0716 00:39:57.530196 2572 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"etc-selinux\" (UniqueName: \"kubernetes.io/host-path/f82be4fd-cc5f-4ff7-8b98-0f7497e4c802-etc-selinux\") pod \"aws-ebs-csi-driver-node-pq88r\" (UID: \"f82be4fd-cc5f-4ff7-8b98-0f7497e4c802\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-pq88r" Jul 16 00:39:57.531122 ip-10-0-143-200 kubenswrapper[2572]: I0716 00:39:57.530235 2572 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"multus-conf-dir\" (UniqueName: \"kubernetes.io/host-path/2b89891b-83eb-48c2-abe4-747d1ca26caf-multus-conf-dir\") pod \"multus-szqbq\" (UID: \"2b89891b-83eb-48c2-abe4-747d1ca26caf\") " pod="openshift-multus/multus-szqbq" Jul 16 00:39:57.531122 ip-10-0-143-200 kubenswrapper[2572]: I0716 00:39:57.530278 2572 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"registration-dir\" (UniqueName: \"kubernetes.io/host-path/f82be4fd-cc5f-4ff7-8b98-0f7497e4c802-registration-dir\") pod \"aws-ebs-csi-driver-node-pq88r\" (UID: \"f82be4fd-cc5f-4ff7-8b98-0f7497e4c802\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-pq88r" Jul 16 00:39:57.531122 ip-10-0-143-200 kubenswrapper[2572]: I0716 00:39:57.530323 2572 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"run-systemd\" (UniqueName: \"kubernetes.io/host-path/aa66f2b6-0ad8-45d0-8ff5-40fc21966b5a-run-systemd\") pod \"ovnkube-node-fp78g\" (UID: \"aa66f2b6-0ad8-45d0-8ff5-40fc21966b5a\") " pod="openshift-ovn-kubernetes/ovnkube-node-fp78g" Jul 16 00:39:57.531122 ip-10-0-143-200 kubenswrapper[2572]: I0716 00:39:57.529352 2572 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"etc-openvswitch\" (UniqueName: \"kubernetes.io/host-path/aa66f2b6-0ad8-45d0-8ff5-40fc21966b5a-etc-openvswitch\") pod \"ovnkube-node-fp78g\" (UID: \"aa66f2b6-0ad8-45d0-8ff5-40fc21966b5a\") " pod="openshift-ovn-kubernetes/ovnkube-node-fp78g" Jul 16 00:39:57.531122 ip-10-0-143-200 kubenswrapper[2572]: I0716 00:39:57.530386 2572 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"node-log\" (UniqueName: \"kubernetes.io/host-path/aa66f2b6-0ad8-45d0-8ff5-40fc21966b5a-node-log\") pod \"ovnkube-node-fp78g\" (UID: \"aa66f2b6-0ad8-45d0-8ff5-40fc21966b5a\") " pod="openshift-ovn-kubernetes/ovnkube-node-fp78g" Jul 16 00:39:57.531941 ip-10-0-143-200 kubenswrapper[2572]: I0716 00:39:57.530482 2572 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"ovnkube-script-lib\" (UniqueName: \"kubernetes.io/configmap/aa66f2b6-0ad8-45d0-8ff5-40fc21966b5a-ovnkube-script-lib\") pod \"ovnkube-node-fp78g\" (UID: \"aa66f2b6-0ad8-45d0-8ff5-40fc21966b5a\") " pod="openshift-ovn-kubernetes/ovnkube-node-fp78g" Jul 16 00:39:57.531941 ip-10-0-143-200 kubenswrapper[2572]: I0716 00:39:57.529033 2572 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"host-run-netns\" (UniqueName: \"kubernetes.io/host-path/2b89891b-83eb-48c2-abe4-747d1ca26caf-host-run-netns\") pod \"multus-szqbq\" (UID: \"2b89891b-83eb-48c2-abe4-747d1ca26caf\") " pod="openshift-multus/multus-szqbq" Jul 16 00:39:57.531941 ip-10-0-143-200 kubenswrapper[2572]: I0716 00:39:57.530562 2572 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"multus-daemon-config\" (UniqueName: \"kubernetes.io/configmap/2b89891b-83eb-48c2-abe4-747d1ca26caf-multus-daemon-config\") pod \"multus-szqbq\" (UID: \"2b89891b-83eb-48c2-abe4-747d1ca26caf\") " pod="openshift-multus/multus-szqbq" Jul 16 00:39:57.531941 ip-10-0-143-200 kubenswrapper[2572]: I0716 00:39:57.530744 2572 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"socket-dir\" (UniqueName: \"kubernetes.io/host-path/f82be4fd-cc5f-4ff7-8b98-0f7497e4c802-socket-dir\") pod \"aws-ebs-csi-driver-node-pq88r\" (UID: \"f82be4fd-cc5f-4ff7-8b98-0f7497e4c802\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-pq88r" Jul 16 00:39:57.531941 ip-10-0-143-200 kubenswrapper[2572]: I0716 00:39:57.530787 2572 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"run-ovn\" (UniqueName: \"kubernetes.io/host-path/aa66f2b6-0ad8-45d0-8ff5-40fc21966b5a-run-ovn\") pod \"ovnkube-node-fp78g\" (UID: \"aa66f2b6-0ad8-45d0-8ff5-40fc21966b5a\") " pod="openshift-ovn-kubernetes/ovnkube-node-fp78g" Jul 16 00:39:57.531941 ip-10-0-143-200 kubenswrapper[2572]: E0716 00:39:57.530811 2572 secret.go:189] Couldn't get secret openshift-multus/metrics-daemon-secret: object "openshift-multus"/"metrics-daemon-secret" not registered Jul 16 00:39:57.531941 ip-10-0-143-200 kubenswrapper[2572]: I0716 00:39:57.530829 2572 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"cnibin\" (UniqueName: \"kubernetes.io/host-path/2b89891b-83eb-48c2-abe4-747d1ca26caf-cnibin\") pod \"multus-szqbq\" (UID: \"2b89891b-83eb-48c2-abe4-747d1ca26caf\") " pod="openshift-multus/multus-szqbq" Jul 16 00:39:57.531941 ip-10-0-143-200 kubenswrapper[2572]: E0716 00:39:57.530866 2572 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/4a08cb1b-3bfa-49cf-8ff0-edef0ff944b9-metrics-certs podName:4a08cb1b-3bfa-49cf-8ff0-edef0ff944b9 nodeName:}" failed. No retries permitted until 2026-07-16 00:39:58.030849699 +0000 UTC m=+3.139604356 (durationBeforeRetry 500ms). Error: MountVolume.SetUp failed for volume "metrics-certs" (UniqueName: "kubernetes.io/secret/4a08cb1b-3bfa-49cf-8ff0-edef0ff944b9-metrics-certs") pod "network-metrics-daemon-5tphg" (UID: "4a08cb1b-3bfa-49cf-8ff0-edef0ff944b9") : object "openshift-multus"/"metrics-daemon-secret" not registered Jul 16 00:39:57.531941 ip-10-0-143-200 kubenswrapper[2572]: I0716 00:39:57.530862 2572 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"host-var-lib-cni-networks-ovn-kubernetes\" (UniqueName: \"kubernetes.io/host-path/aa66f2b6-0ad8-45d0-8ff5-40fc21966b5a-host-var-lib-cni-networks-ovn-kubernetes\") pod \"ovnkube-node-fp78g\" (UID: \"aa66f2b6-0ad8-45d0-8ff5-40fc21966b5a\") " pod="openshift-ovn-kubernetes/ovnkube-node-fp78g" Jul 16 00:39:57.531941 ip-10-0-143-200 kubenswrapper[2572]: I0716 00:39:57.530972 2572 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"host-var-lib-kubelet\" (UniqueName: \"kubernetes.io/host-path/2b89891b-83eb-48c2-abe4-747d1ca26caf-host-var-lib-kubelet\") pod \"multus-szqbq\" (UID: \"2b89891b-83eb-48c2-abe4-747d1ca26caf\") " pod="openshift-multus/multus-szqbq" Jul 16 00:39:57.533746 ip-10-0-143-200 kubenswrapper[2572]: I0716 00:39:57.533687 2572 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"ovn-node-metrics-cert\" (UniqueName: \"kubernetes.io/secret/aa66f2b6-0ad8-45d0-8ff5-40fc21966b5a-ovn-node-metrics-cert\") pod \"ovnkube-node-fp78g\" (UID: \"aa66f2b6-0ad8-45d0-8ff5-40fc21966b5a\") " pod="openshift-ovn-kubernetes/ovnkube-node-fp78g" Jul 16 00:39:57.540702 ip-10-0-143-200 kubenswrapper[2572]: E0716 00:39:57.540668 2572 projected.go:291] Couldn't get configMap openshift-network-diagnostics/kube-root-ca.crt: object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered Jul 16 00:39:57.540842 ip-10-0-143-200 kubenswrapper[2572]: E0716 00:39:57.540697 2572 projected.go:291] Couldn't get configMap openshift-network-diagnostics/openshift-service-ca.crt: object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered Jul 16 00:39:57.540842 ip-10-0-143-200 kubenswrapper[2572]: E0716 00:39:57.540729 2572 projected.go:196] Error preparing data for projected volume kube-api-access-kpfbw for pod openshift-network-diagnostics/network-check-target-52n54: [object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered, object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered] Jul 16 00:39:57.540842 ip-10-0-143-200 kubenswrapper[2572]: E0716 00:39:57.540804 2572 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/projected/8962f109-ea94-4296-a504-f6709315e9cd-kube-api-access-kpfbw podName:8962f109-ea94-4296-a504-f6709315e9cd nodeName:}" failed. No retries permitted until 2026-07-16 00:39:58.040783217 +0000 UTC m=+3.149537887 (durationBeforeRetry 500ms). Error: MountVolume.SetUp failed for volume "kube-api-access-kpfbw" (UniqueName: "kubernetes.io/projected/8962f109-ea94-4296-a504-f6709315e9cd-kube-api-access-kpfbw") pod "network-check-target-52n54" (UID: "8962f109-ea94-4296-a504-f6709315e9cd") : [object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered, object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered] Jul 16 00:39:57.542621 ip-10-0-143-200 kubenswrapper[2572]: I0716 00:39:57.542595 2572 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-8lzb7\" (UniqueName: \"kubernetes.io/projected/aa66f2b6-0ad8-45d0-8ff5-40fc21966b5a-kube-api-access-8lzb7\") pod \"ovnkube-node-fp78g\" (UID: \"aa66f2b6-0ad8-45d0-8ff5-40fc21966b5a\") " pod="openshift-ovn-kubernetes/ovnkube-node-fp78g" Jul 16 00:39:57.543256 ip-10-0-143-200 kubenswrapper[2572]: I0716 00:39:57.543219 2572 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-88fdc\" (UniqueName: \"kubernetes.io/projected/2b89891b-83eb-48c2-abe4-747d1ca26caf-kube-api-access-88fdc\") pod \"multus-szqbq\" (UID: \"2b89891b-83eb-48c2-abe4-747d1ca26caf\") " pod="openshift-multus/multus-szqbq" Jul 16 00:39:57.544195 ip-10-0-143-200 kubenswrapper[2572]: I0716 00:39:57.544017 2572 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-nvl2k\" (UniqueName: \"kubernetes.io/projected/4a08cb1b-3bfa-49cf-8ff0-edef0ff944b9-kube-api-access-nvl2k\") pod \"network-metrics-daemon-5tphg\" (UID: \"4a08cb1b-3bfa-49cf-8ff0-edef0ff944b9\") " pod="openshift-multus/network-metrics-daemon-5tphg" Jul 16 00:39:57.544834 ip-10-0-143-200 kubenswrapper[2572]: I0716 00:39:57.544541 2572 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-zbq79\" (UniqueName: \"kubernetes.io/projected/ff468ef8-66ba-48f2-8531-da662a5b19d3-kube-api-access-zbq79\") pod \"node-ca-6msnd\" (UID: \"ff468ef8-66ba-48f2-8531-da662a5b19d3\") " pod="openshift-image-registry/node-ca-6msnd" Jul 16 00:39:57.544834 ip-10-0-143-200 kubenswrapper[2572]: I0716 00:39:57.544793 2572 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-kswdz\" (UniqueName: \"kubernetes.io/projected/f82be4fd-cc5f-4ff7-8b98-0f7497e4c802-kube-api-access-kswdz\") pod \"aws-ebs-csi-driver-node-pq88r\" (UID: \"f82be4fd-cc5f-4ff7-8b98-0f7497e4c802\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-pq88r" Jul 16 00:39:57.612195 ip-10-0-143-200 kubenswrapper[2572]: I0716 00:39:57.612149 2572 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-dns/node-resolver-6qlgz" Jul 16 00:39:57.635377 ip-10-0-143-200 kubenswrapper[2572]: I0716 00:39:57.635333 2572 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-operator/iptables-alerter-gcstg" Jul 16 00:39:57.645196 ip-10-0-143-200 kubenswrapper[2572]: I0716 00:39:57.645164 2572 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/konnectivity-agent-9gnmq" Jul 16 00:39:57.653133 ip-10-0-143-200 kubenswrapper[2572]: I0716 00:39:57.653100 2572 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/multus-additional-cni-plugins-vfntt" Jul 16 00:39:57.660132 ip-10-0-143-200 kubenswrapper[2572]: I0716 00:39:57.660108 2572 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-cluster-node-tuning-operator/tuned-w6p89" Jul 16 00:39:57.668999 ip-10-0-143-200 kubenswrapper[2572]: I0716 00:39:57.668972 2572 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-image-registry/node-ca-6msnd" Jul 16 00:39:57.677065 ip-10-0-143-200 kubenswrapper[2572]: I0716 00:39:57.677039 2572 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ovn-kubernetes/ovnkube-node-fp78g" Jul 16 00:39:57.684970 ip-10-0-143-200 kubenswrapper[2572]: I0716 00:39:57.684944 2572 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/multus-szqbq" Jul 16 00:39:57.690085 ip-10-0-143-200 kubenswrapper[2572]: I0716 00:39:57.690058 2572 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-pq88r" Jul 16 00:39:57.933728 ip-10-0-143-200 kubenswrapper[2572]: I0716 00:39:57.933584 2572 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"original-pull-secret\" (UniqueName: \"kubernetes.io/secret/a9bdff50-c8ce-4ca4-a810-20be3909b23c-original-pull-secret\") pod \"global-pull-secret-syncer-qcbcd\" (UID: \"a9bdff50-c8ce-4ca4-a810-20be3909b23c\") " pod="kube-system/global-pull-secret-syncer-qcbcd" Jul 16 00:39:57.933883 ip-10-0-143-200 kubenswrapper[2572]: E0716 00:39:57.933738 2572 secret.go:189] Couldn't get secret kube-system/original-pull-secret: object "kube-system"/"original-pull-secret" not registered Jul 16 00:39:57.933883 ip-10-0-143-200 kubenswrapper[2572]: E0716 00:39:57.933815 2572 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/a9bdff50-c8ce-4ca4-a810-20be3909b23c-original-pull-secret podName:a9bdff50-c8ce-4ca4-a810-20be3909b23c nodeName:}" failed. No retries permitted until 2026-07-16 00:39:58.933792719 +0000 UTC m=+4.042547402 (durationBeforeRetry 1s). Error: MountVolume.SetUp failed for volume "original-pull-secret" (UniqueName: "kubernetes.io/secret/a9bdff50-c8ce-4ca4-a810-20be3909b23c-original-pull-secret") pod "global-pull-secret-syncer-qcbcd" (UID: "a9bdff50-c8ce-4ca4-a810-20be3909b23c") : object "kube-system"/"original-pull-secret" not registered Jul 16 00:39:58.034474 ip-10-0-143-200 kubenswrapper[2572]: I0716 00:39:58.034058 2572 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/4a08cb1b-3bfa-49cf-8ff0-edef0ff944b9-metrics-certs\") pod \"network-metrics-daemon-5tphg\" (UID: \"4a08cb1b-3bfa-49cf-8ff0-edef0ff944b9\") " pod="openshift-multus/network-metrics-daemon-5tphg" Jul 16 00:39:58.034474 ip-10-0-143-200 kubenswrapper[2572]: E0716 00:39:58.034199 2572 secret.go:189] Couldn't get secret openshift-multus/metrics-daemon-secret: object "openshift-multus"/"metrics-daemon-secret" not registered Jul 16 00:39:58.034474 ip-10-0-143-200 kubenswrapper[2572]: E0716 00:39:58.034260 2572 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/4a08cb1b-3bfa-49cf-8ff0-edef0ff944b9-metrics-certs podName:4a08cb1b-3bfa-49cf-8ff0-edef0ff944b9 nodeName:}" failed. No retries permitted until 2026-07-16 00:39:59.034243527 +0000 UTC m=+4.142998195 (durationBeforeRetry 1s). Error: MountVolume.SetUp failed for volume "metrics-certs" (UniqueName: "kubernetes.io/secret/4a08cb1b-3bfa-49cf-8ff0-edef0ff944b9-metrics-certs") pod "network-metrics-daemon-5tphg" (UID: "4a08cb1b-3bfa-49cf-8ff0-edef0ff944b9") : object "openshift-multus"/"metrics-daemon-secret" not registered Jul 16 00:39:58.134671 ip-10-0-143-200 kubenswrapper[2572]: I0716 00:39:58.134615 2572 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-kpfbw\" (UniqueName: \"kubernetes.io/projected/8962f109-ea94-4296-a504-f6709315e9cd-kube-api-access-kpfbw\") pod \"network-check-target-52n54\" (UID: \"8962f109-ea94-4296-a504-f6709315e9cd\") " pod="openshift-network-diagnostics/network-check-target-52n54" Jul 16 00:39:58.134847 ip-10-0-143-200 kubenswrapper[2572]: E0716 00:39:58.134777 2572 projected.go:291] Couldn't get configMap openshift-network-diagnostics/kube-root-ca.crt: object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered Jul 16 00:39:58.134847 ip-10-0-143-200 kubenswrapper[2572]: E0716 00:39:58.134797 2572 projected.go:291] Couldn't get configMap openshift-network-diagnostics/openshift-service-ca.crt: object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered Jul 16 00:39:58.134847 ip-10-0-143-200 kubenswrapper[2572]: E0716 00:39:58.134806 2572 projected.go:196] Error preparing data for projected volume kube-api-access-kpfbw for pod openshift-network-diagnostics/network-check-target-52n54: [object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered, object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered] Jul 16 00:39:58.134980 ip-10-0-143-200 kubenswrapper[2572]: E0716 00:39:58.134873 2572 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/projected/8962f109-ea94-4296-a504-f6709315e9cd-kube-api-access-kpfbw podName:8962f109-ea94-4296-a504-f6709315e9cd nodeName:}" failed. No retries permitted until 2026-07-16 00:39:59.134855423 +0000 UTC m=+4.243610098 (durationBeforeRetry 1s). Error: MountVolume.SetUp failed for volume "kube-api-access-kpfbw" (UniqueName: "kubernetes.io/projected/8962f109-ea94-4296-a504-f6709315e9cd-kube-api-access-kpfbw") pod "network-check-target-52n54" (UID: "8962f109-ea94-4296-a504-f6709315e9cd") : [object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered, object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered] Jul 16 00:39:58.355420 ip-10-0-143-200 kubenswrapper[2572]: I0716 00:39:58.355329 2572 certificate_manager.go:715] "Certificate rotation deadline determined" logger="kubernetes.io/kubelet-serving" expiration="2028-07-15 00:34:57 +0000 UTC" deadline="2028-02-18 00:59:07.355843257 +0000 UTC" Jul 16 00:39:58.355420 ip-10-0-143-200 kubenswrapper[2572]: I0716 00:39:58.355367 2572 certificate_manager.go:431] "Waiting for next certificate rotation" logger="kubernetes.io/kubelet-serving" sleep="13968h19m9.000480833s" Jul 16 00:39:58.401157 ip-10-0-143-200 kubenswrapper[2572]: W0716 00:39:58.401126 2572 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod24fd12c1_ddcb_427f_ac7e_4e2877450cb8.slice/crio-d8a27e16f8c1f243c9d9a795bb3c1c529ce14fb97a11b6133997d31bd4d2dbaf WatchSource:0}: Error finding container d8a27e16f8c1f243c9d9a795bb3c1c529ce14fb97a11b6133997d31bd4d2dbaf: Status 404 returned error can't find the container with id d8a27e16f8c1f243c9d9a795bb3c1c529ce14fb97a11b6133997d31bd4d2dbaf Jul 16 00:39:58.405898 ip-10-0-143-200 kubenswrapper[2572]: W0716 00:39:58.405876 2572 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-poda51f2d32_4840_4f0c_804d_c8aa6015ae5a.slice/crio-bc2884040255035760345adb872ed019c8a33b542c02b792095878d714c71b06 WatchSource:0}: Error finding container bc2884040255035760345adb872ed019c8a33b542c02b792095878d714c71b06: Status 404 returned error can't find the container with id bc2884040255035760345adb872ed019c8a33b542c02b792095878d714c71b06 Jul 16 00:39:58.411547 ip-10-0-143-200 kubenswrapper[2572]: W0716 00:39:58.411513 2572 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod2b89891b_83eb_48c2_abe4_747d1ca26caf.slice/crio-a947fd50a26370ae38af78e8e6a329e8543b94c145a62b6aa276928331cd8e34 WatchSource:0}: Error finding container a947fd50a26370ae38af78e8e6a329e8543b94c145a62b6aa276928331cd8e34: Status 404 returned error can't find the container with id a947fd50a26370ae38af78e8e6a329e8543b94c145a62b6aa276928331cd8e34 Jul 16 00:39:58.412111 ip-10-0-143-200 kubenswrapper[2572]: W0716 00:39:58.412086 2572 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod0247608f_bb0b_4ace_b071_292d5586fc65.slice/crio-4811be7942c676ab41d0a1f144090d1390dd95cfd11468833cf48cc6dd5d3fb4 WatchSource:0}: Error finding container 4811be7942c676ab41d0a1f144090d1390dd95cfd11468833cf48cc6dd5d3fb4: Status 404 returned error can't find the container with id 4811be7942c676ab41d0a1f144090d1390dd95cfd11468833cf48cc6dd5d3fb4 Jul 16 00:39:58.414246 ip-10-0-143-200 kubenswrapper[2572]: W0716 00:39:58.414223 2572 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podaa66f2b6_0ad8_45d0_8ff5_40fc21966b5a.slice/crio-7907c10a0ad9e54fda929c95a29d5a39d5f37febde9ee10403396f03f9fb3824 WatchSource:0}: Error finding container 7907c10a0ad9e54fda929c95a29d5a39d5f37febde9ee10403396f03f9fb3824: Status 404 returned error can't find the container with id 7907c10a0ad9e54fda929c95a29d5a39d5f37febde9ee10403396f03f9fb3824 Jul 16 00:39:58.415379 ip-10-0-143-200 kubenswrapper[2572]: W0716 00:39:58.415350 2572 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod3a7e030d_fad8_480d_9dc6_fe5057048cf9.slice/crio-6fb6b0609a5f1b2d3d0d0317a3fe8fd115bb191f6349c98143bb2fd4eb046929 WatchSource:0}: Error finding container 6fb6b0609a5f1b2d3d0d0317a3fe8fd115bb191f6349c98143bb2fd4eb046929: Status 404 returned error can't find the container with id 6fb6b0609a5f1b2d3d0d0317a3fe8fd115bb191f6349c98143bb2fd4eb046929 Jul 16 00:39:58.416788 ip-10-0-143-200 kubenswrapper[2572]: W0716 00:39:58.416735 2572 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podff468ef8_66ba_48f2_8531_da662a5b19d3.slice/crio-1310cb598d199deba143dc7c8c18f231566f10fbe8e85544eee8e328cbd3cbb5 WatchSource:0}: Error finding container 1310cb598d199deba143dc7c8c18f231566f10fbe8e85544eee8e328cbd3cbb5: Status 404 returned error can't find the container with id 1310cb598d199deba143dc7c8c18f231566f10fbe8e85544eee8e328cbd3cbb5 Jul 16 00:39:58.455907 ip-10-0-143-200 kubenswrapper[2572]: I0716 00:39:58.455879 2572 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-multus/multus-szqbq" event={"ID":"2b89891b-83eb-48c2-abe4-747d1ca26caf","Type":"ContainerStarted","Data":"a947fd50a26370ae38af78e8e6a329e8543b94c145a62b6aa276928331cd8e34"} Jul 16 00:39:58.457052 ip-10-0-143-200 kubenswrapper[2572]: I0716 00:39:58.457026 2572 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-multus/multus-additional-cni-plugins-vfntt" event={"ID":"a51f2d32-4840-4f0c-804d-c8aa6015ae5a","Type":"ContainerStarted","Data":"bc2884040255035760345adb872ed019c8a33b542c02b792095878d714c71b06"} Jul 16 00:39:58.458188 ip-10-0-143-200 kubenswrapper[2572]: I0716 00:39:58.458135 2572 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kube-system/konnectivity-agent-9gnmq" event={"ID":"3a7e030d-fad8-480d-9dc6-fe5057048cf9","Type":"ContainerStarted","Data":"6fb6b0609a5f1b2d3d0d0317a3fe8fd115bb191f6349c98143bb2fd4eb046929"} Jul 16 00:39:58.459431 ip-10-0-143-200 kubenswrapper[2572]: I0716 00:39:58.459406 2572 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-pq88r" event={"ID":"f82be4fd-cc5f-4ff7-8b98-0f7497e4c802","Type":"ContainerStarted","Data":"f5eb67b0579174158c9de6d5092528825e2c0438fdb0cdeaf749953b14a32c7c"} Jul 16 00:39:58.460933 ip-10-0-143-200 kubenswrapper[2572]: I0716 00:39:58.460913 2572 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-cluster-node-tuning-operator/tuned-w6p89" event={"ID":"2614f4d9-8dc4-406b-a5aa-7dd2e7446008","Type":"ContainerStarted","Data":"cf84d8463a5a205f1a69981245eb9efd534ec8260b3a1bd824992cb6fa2a51d2"} Jul 16 00:39:58.461945 ip-10-0-143-200 kubenswrapper[2572]: I0716 00:39:58.461922 2572 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-dns/node-resolver-6qlgz" event={"ID":"24fd12c1-ddcb-427f-ac7e-4e2877450cb8","Type":"ContainerStarted","Data":"d8a27e16f8c1f243c9d9a795bb3c1c529ce14fb97a11b6133997d31bd4d2dbaf"} Jul 16 00:39:58.464281 ip-10-0-143-200 kubenswrapper[2572]: I0716 00:39:58.464261 2572 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-image-registry/node-ca-6msnd" event={"ID":"ff468ef8-66ba-48f2-8531-da662a5b19d3","Type":"ContainerStarted","Data":"1310cb598d199deba143dc7c8c18f231566f10fbe8e85544eee8e328cbd3cbb5"} Jul 16 00:39:58.470141 ip-10-0-143-200 kubenswrapper[2572]: I0716 00:39:58.470119 2572 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ovn-kubernetes/ovnkube-node-fp78g" event={"ID":"aa66f2b6-0ad8-45d0-8ff5-40fc21966b5a","Type":"ContainerStarted","Data":"7907c10a0ad9e54fda929c95a29d5a39d5f37febde9ee10403396f03f9fb3824"} Jul 16 00:39:58.474363 ip-10-0-143-200 kubenswrapper[2572]: I0716 00:39:58.474336 2572 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-network-operator/iptables-alerter-gcstg" event={"ID":"0247608f-bb0b-4ace-b071-292d5586fc65","Type":"ContainerStarted","Data":"4811be7942c676ab41d0a1f144090d1390dd95cfd11468833cf48cc6dd5d3fb4"} Jul 16 00:39:58.941797 ip-10-0-143-200 kubenswrapper[2572]: I0716 00:39:58.941481 2572 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"original-pull-secret\" (UniqueName: \"kubernetes.io/secret/a9bdff50-c8ce-4ca4-a810-20be3909b23c-original-pull-secret\") pod \"global-pull-secret-syncer-qcbcd\" (UID: \"a9bdff50-c8ce-4ca4-a810-20be3909b23c\") " pod="kube-system/global-pull-secret-syncer-qcbcd" Jul 16 00:39:58.941984 ip-10-0-143-200 kubenswrapper[2572]: E0716 00:39:58.941928 2572 secret.go:189] Couldn't get secret kube-system/original-pull-secret: object "kube-system"/"original-pull-secret" not registered Jul 16 00:39:58.942046 ip-10-0-143-200 kubenswrapper[2572]: E0716 00:39:58.942003 2572 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/a9bdff50-c8ce-4ca4-a810-20be3909b23c-original-pull-secret podName:a9bdff50-c8ce-4ca4-a810-20be3909b23c nodeName:}" failed. No retries permitted until 2026-07-16 00:40:00.941982266 +0000 UTC m=+6.050736921 (durationBeforeRetry 2s). Error: MountVolume.SetUp failed for volume "original-pull-secret" (UniqueName: "kubernetes.io/secret/a9bdff50-c8ce-4ca4-a810-20be3909b23c-original-pull-secret") pod "global-pull-secret-syncer-qcbcd" (UID: "a9bdff50-c8ce-4ca4-a810-20be3909b23c") : object "kube-system"/"original-pull-secret" not registered Jul 16 00:39:59.042840 ip-10-0-143-200 kubenswrapper[2572]: I0716 00:39:59.042801 2572 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/4a08cb1b-3bfa-49cf-8ff0-edef0ff944b9-metrics-certs\") pod \"network-metrics-daemon-5tphg\" (UID: \"4a08cb1b-3bfa-49cf-8ff0-edef0ff944b9\") " pod="openshift-multus/network-metrics-daemon-5tphg" Jul 16 00:39:59.043062 ip-10-0-143-200 kubenswrapper[2572]: E0716 00:39:59.042989 2572 secret.go:189] Couldn't get secret openshift-multus/metrics-daemon-secret: object "openshift-multus"/"metrics-daemon-secret" not registered Jul 16 00:39:59.043062 ip-10-0-143-200 kubenswrapper[2572]: E0716 00:39:59.043056 2572 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/4a08cb1b-3bfa-49cf-8ff0-edef0ff944b9-metrics-certs podName:4a08cb1b-3bfa-49cf-8ff0-edef0ff944b9 nodeName:}" failed. No retries permitted until 2026-07-16 00:40:01.043035306 +0000 UTC m=+6.151789974 (durationBeforeRetry 2s). Error: MountVolume.SetUp failed for volume "metrics-certs" (UniqueName: "kubernetes.io/secret/4a08cb1b-3bfa-49cf-8ff0-edef0ff944b9-metrics-certs") pod "network-metrics-daemon-5tphg" (UID: "4a08cb1b-3bfa-49cf-8ff0-edef0ff944b9") : object "openshift-multus"/"metrics-daemon-secret" not registered Jul 16 00:39:59.143439 ip-10-0-143-200 kubenswrapper[2572]: I0716 00:39:59.143394 2572 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-kpfbw\" (UniqueName: \"kubernetes.io/projected/8962f109-ea94-4296-a504-f6709315e9cd-kube-api-access-kpfbw\") pod \"network-check-target-52n54\" (UID: \"8962f109-ea94-4296-a504-f6709315e9cd\") " pod="openshift-network-diagnostics/network-check-target-52n54" Jul 16 00:39:59.143650 ip-10-0-143-200 kubenswrapper[2572]: E0716 00:39:59.143581 2572 projected.go:291] Couldn't get configMap openshift-network-diagnostics/kube-root-ca.crt: object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered Jul 16 00:39:59.143650 ip-10-0-143-200 kubenswrapper[2572]: E0716 00:39:59.143603 2572 projected.go:291] Couldn't get configMap openshift-network-diagnostics/openshift-service-ca.crt: object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered Jul 16 00:39:59.143650 ip-10-0-143-200 kubenswrapper[2572]: E0716 00:39:59.143615 2572 projected.go:196] Error preparing data for projected volume kube-api-access-kpfbw for pod openshift-network-diagnostics/network-check-target-52n54: [object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered, object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered] Jul 16 00:39:59.143817 ip-10-0-143-200 kubenswrapper[2572]: E0716 00:39:59.143690 2572 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/projected/8962f109-ea94-4296-a504-f6709315e9cd-kube-api-access-kpfbw podName:8962f109-ea94-4296-a504-f6709315e9cd nodeName:}" failed. No retries permitted until 2026-07-16 00:40:01.143671104 +0000 UTC m=+6.252425771 (durationBeforeRetry 2s). Error: MountVolume.SetUp failed for volume "kube-api-access-kpfbw" (UniqueName: "kubernetes.io/projected/8962f109-ea94-4296-a504-f6709315e9cd-kube-api-access-kpfbw") pod "network-check-target-52n54" (UID: "8962f109-ea94-4296-a504-f6709315e9cd") : [object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered, object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered] Jul 16 00:39:59.238475 ip-10-0-143-200 kubenswrapper[2572]: I0716 00:39:59.238305 2572 reflector.go:439] "Caches populated" type="*v1.CSIDriver" reflector="k8s.io/client-go/informers/factory.go:160" Jul 16 00:39:59.356032 ip-10-0-143-200 kubenswrapper[2572]: I0716 00:39:59.355945 2572 certificate_manager.go:715] "Certificate rotation deadline determined" logger="kubernetes.io/kubelet-serving" expiration="2028-07-15 00:34:57 +0000 UTC" deadline="2028-01-21 19:04:30.098359064 +0000 UTC" Jul 16 00:39:59.356032 ip-10-0-143-200 kubenswrapper[2572]: I0716 00:39:59.356005 2572 certificate_manager.go:431] "Waiting for next certificate rotation" logger="kubernetes.io/kubelet-serving" sleep="13314h24m30.7423581s" Jul 16 00:39:59.442454 ip-10-0-143-200 kubenswrapper[2572]: I0716 00:39:59.442417 2572 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-5tphg" Jul 16 00:39:59.442669 ip-10-0-143-200 kubenswrapper[2572]: E0716 00:39:59.442588 2572 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-5tphg" podUID="4a08cb1b-3bfa-49cf-8ff0-edef0ff944b9" Jul 16 00:39:59.443146 ip-10-0-143-200 kubenswrapper[2572]: I0716 00:39:59.443123 2572 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-qcbcd" Jul 16 00:39:59.443270 ip-10-0-143-200 kubenswrapper[2572]: E0716 00:39:59.443248 2572 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-qcbcd" podUID="a9bdff50-c8ce-4ca4-a810-20be3909b23c" Jul 16 00:39:59.443334 ip-10-0-143-200 kubenswrapper[2572]: I0716 00:39:59.443324 2572 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-52n54" Jul 16 00:39:59.443415 ip-10-0-143-200 kubenswrapper[2572]: E0716 00:39:59.443397 2572 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-52n54" podUID="8962f109-ea94-4296-a504-f6709315e9cd" Jul 16 00:39:59.496109 ip-10-0-143-200 kubenswrapper[2572]: I0716 00:39:59.494964 2572 generic.go:356] "Generic (PLEG): container finished" podID="df298bc82c8f94d4cb31271396017b60" containerID="9636991f79fc5069bb1d0dc807934a4b159bf3204960a6b08f5db0213c044792" exitCode=0 Jul 16 00:39:59.496109 ip-10-0-143-200 kubenswrapper[2572]: I0716 00:39:59.495066 2572 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-143-200.ec2.internal" event={"ID":"df298bc82c8f94d4cb31271396017b60","Type":"ContainerDied","Data":"9636991f79fc5069bb1d0dc807934a4b159bf3204960a6b08f5db0213c044792"} Jul 16 00:39:59.511884 ip-10-0-143-200 kubenswrapper[2572]: I0716 00:39:59.511850 2572 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kube-system/kube-apiserver-proxy-ip-10-0-143-200.ec2.internal" event={"ID":"f0dfd958ab883b381e2b09c1edf12ef3","Type":"ContainerStarted","Data":"801269c126fbf9f15c24742b5021bf50216dbeb24460ac42885b93d0e4c36715"} Jul 16 00:39:59.530423 ip-10-0-143-200 kubenswrapper[2572]: I0716 00:39:59.530349 2572 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="kube-system/kube-apiserver-proxy-ip-10-0-143-200.ec2.internal" podStartSLOduration=3.53032488 podStartE2EDuration="3.53032488s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=true podCreationTimestamp="2026-07-16 00:39:56 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-16 00:39:59.530186982 +0000 UTC m=+4.638941658" watchObservedRunningTime="2026-07-16 00:39:59.53032488 +0000 UTC m=+4.639079556" Jul 16 00:40:00.543772 ip-10-0-143-200 kubenswrapper[2572]: I0716 00:40:00.543677 2572 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-143-200.ec2.internal" event={"ID":"df298bc82c8f94d4cb31271396017b60","Type":"ContainerStarted","Data":"098f3e465a1be6a047be4f5ce40af4445f77b39809f275fbd188f6c3cda84d50"} Jul 16 00:40:00.959911 ip-10-0-143-200 kubenswrapper[2572]: I0716 00:40:00.959813 2572 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"original-pull-secret\" (UniqueName: \"kubernetes.io/secret/a9bdff50-c8ce-4ca4-a810-20be3909b23c-original-pull-secret\") pod \"global-pull-secret-syncer-qcbcd\" (UID: \"a9bdff50-c8ce-4ca4-a810-20be3909b23c\") " pod="kube-system/global-pull-secret-syncer-qcbcd" Jul 16 00:40:00.960070 ip-10-0-143-200 kubenswrapper[2572]: E0716 00:40:00.959979 2572 secret.go:189] Couldn't get secret kube-system/original-pull-secret: object "kube-system"/"original-pull-secret" not registered Jul 16 00:40:00.960130 ip-10-0-143-200 kubenswrapper[2572]: E0716 00:40:00.960091 2572 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/a9bdff50-c8ce-4ca4-a810-20be3909b23c-original-pull-secret podName:a9bdff50-c8ce-4ca4-a810-20be3909b23c nodeName:}" failed. No retries permitted until 2026-07-16 00:40:04.960040237 +0000 UTC m=+10.068794904 (durationBeforeRetry 4s). Error: MountVolume.SetUp failed for volume "original-pull-secret" (UniqueName: "kubernetes.io/secret/a9bdff50-c8ce-4ca4-a810-20be3909b23c-original-pull-secret") pod "global-pull-secret-syncer-qcbcd" (UID: "a9bdff50-c8ce-4ca4-a810-20be3909b23c") : object "kube-system"/"original-pull-secret" not registered Jul 16 00:40:01.061111 ip-10-0-143-200 kubenswrapper[2572]: I0716 00:40:01.061065 2572 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/4a08cb1b-3bfa-49cf-8ff0-edef0ff944b9-metrics-certs\") pod \"network-metrics-daemon-5tphg\" (UID: \"4a08cb1b-3bfa-49cf-8ff0-edef0ff944b9\") " pod="openshift-multus/network-metrics-daemon-5tphg" Jul 16 00:40:01.061307 ip-10-0-143-200 kubenswrapper[2572]: E0716 00:40:01.061215 2572 secret.go:189] Couldn't get secret openshift-multus/metrics-daemon-secret: object "openshift-multus"/"metrics-daemon-secret" not registered Jul 16 00:40:01.061307 ip-10-0-143-200 kubenswrapper[2572]: E0716 00:40:01.061281 2572 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/4a08cb1b-3bfa-49cf-8ff0-edef0ff944b9-metrics-certs podName:4a08cb1b-3bfa-49cf-8ff0-edef0ff944b9 nodeName:}" failed. No retries permitted until 2026-07-16 00:40:05.061261918 +0000 UTC m=+10.170016588 (durationBeforeRetry 4s). Error: MountVolume.SetUp failed for volume "metrics-certs" (UniqueName: "kubernetes.io/secret/4a08cb1b-3bfa-49cf-8ff0-edef0ff944b9-metrics-certs") pod "network-metrics-daemon-5tphg" (UID: "4a08cb1b-3bfa-49cf-8ff0-edef0ff944b9") : object "openshift-multus"/"metrics-daemon-secret" not registered Jul 16 00:40:01.161903 ip-10-0-143-200 kubenswrapper[2572]: I0716 00:40:01.161865 2572 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-kpfbw\" (UniqueName: \"kubernetes.io/projected/8962f109-ea94-4296-a504-f6709315e9cd-kube-api-access-kpfbw\") pod \"network-check-target-52n54\" (UID: \"8962f109-ea94-4296-a504-f6709315e9cd\") " pod="openshift-network-diagnostics/network-check-target-52n54" Jul 16 00:40:01.162076 ip-10-0-143-200 kubenswrapper[2572]: E0716 00:40:01.162062 2572 projected.go:291] Couldn't get configMap openshift-network-diagnostics/kube-root-ca.crt: object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered Jul 16 00:40:01.162152 ip-10-0-143-200 kubenswrapper[2572]: E0716 00:40:01.162083 2572 projected.go:291] Couldn't get configMap openshift-network-diagnostics/openshift-service-ca.crt: object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered Jul 16 00:40:01.162152 ip-10-0-143-200 kubenswrapper[2572]: E0716 00:40:01.162096 2572 projected.go:196] Error preparing data for projected volume kube-api-access-kpfbw for pod openshift-network-diagnostics/network-check-target-52n54: [object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered, object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered] Jul 16 00:40:01.162247 ip-10-0-143-200 kubenswrapper[2572]: E0716 00:40:01.162157 2572 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/projected/8962f109-ea94-4296-a504-f6709315e9cd-kube-api-access-kpfbw podName:8962f109-ea94-4296-a504-f6709315e9cd nodeName:}" failed. No retries permitted until 2026-07-16 00:40:05.162136796 +0000 UTC m=+10.270891469 (durationBeforeRetry 4s). Error: MountVolume.SetUp failed for volume "kube-api-access-kpfbw" (UniqueName: "kubernetes.io/projected/8962f109-ea94-4296-a504-f6709315e9cd-kube-api-access-kpfbw") pod "network-check-target-52n54" (UID: "8962f109-ea94-4296-a504-f6709315e9cd") : [object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered, object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered] Jul 16 00:40:01.444402 ip-10-0-143-200 kubenswrapper[2572]: I0716 00:40:01.444322 2572 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-5tphg" Jul 16 00:40:01.444556 ip-10-0-143-200 kubenswrapper[2572]: E0716 00:40:01.444476 2572 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-5tphg" podUID="4a08cb1b-3bfa-49cf-8ff0-edef0ff944b9" Jul 16 00:40:01.444876 ip-10-0-143-200 kubenswrapper[2572]: I0716 00:40:01.444857 2572 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-qcbcd" Jul 16 00:40:01.444975 ip-10-0-143-200 kubenswrapper[2572]: E0716 00:40:01.444956 2572 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-qcbcd" podUID="a9bdff50-c8ce-4ca4-a810-20be3909b23c" Jul 16 00:40:01.445045 ip-10-0-143-200 kubenswrapper[2572]: I0716 00:40:01.445018 2572 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-52n54" Jul 16 00:40:01.445114 ip-10-0-143-200 kubenswrapper[2572]: E0716 00:40:01.445096 2572 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-52n54" podUID="8962f109-ea94-4296-a504-f6709315e9cd" Jul 16 00:40:03.442590 ip-10-0-143-200 kubenswrapper[2572]: I0716 00:40:03.442544 2572 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-52n54" Jul 16 00:40:03.442590 ip-10-0-143-200 kubenswrapper[2572]: I0716 00:40:03.442585 2572 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-qcbcd" Jul 16 00:40:03.443119 ip-10-0-143-200 kubenswrapper[2572]: I0716 00:40:03.442544 2572 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-5tphg" Jul 16 00:40:03.443119 ip-10-0-143-200 kubenswrapper[2572]: E0716 00:40:03.442689 2572 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-52n54" podUID="8962f109-ea94-4296-a504-f6709315e9cd" Jul 16 00:40:03.443119 ip-10-0-143-200 kubenswrapper[2572]: E0716 00:40:03.442885 2572 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-5tphg" podUID="4a08cb1b-3bfa-49cf-8ff0-edef0ff944b9" Jul 16 00:40:03.443119 ip-10-0-143-200 kubenswrapper[2572]: E0716 00:40:03.442979 2572 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-qcbcd" podUID="a9bdff50-c8ce-4ca4-a810-20be3909b23c" Jul 16 00:40:04.994039 ip-10-0-143-200 kubenswrapper[2572]: I0716 00:40:04.993996 2572 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"original-pull-secret\" (UniqueName: \"kubernetes.io/secret/a9bdff50-c8ce-4ca4-a810-20be3909b23c-original-pull-secret\") pod \"global-pull-secret-syncer-qcbcd\" (UID: \"a9bdff50-c8ce-4ca4-a810-20be3909b23c\") " pod="kube-system/global-pull-secret-syncer-qcbcd" Jul 16 00:40:04.994513 ip-10-0-143-200 kubenswrapper[2572]: E0716 00:40:04.994140 2572 secret.go:189] Couldn't get secret kube-system/original-pull-secret: object "kube-system"/"original-pull-secret" not registered Jul 16 00:40:04.994513 ip-10-0-143-200 kubenswrapper[2572]: E0716 00:40:04.994235 2572 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/a9bdff50-c8ce-4ca4-a810-20be3909b23c-original-pull-secret podName:a9bdff50-c8ce-4ca4-a810-20be3909b23c nodeName:}" failed. No retries permitted until 2026-07-16 00:40:12.994211186 +0000 UTC m=+18.102965854 (durationBeforeRetry 8s). Error: MountVolume.SetUp failed for volume "original-pull-secret" (UniqueName: "kubernetes.io/secret/a9bdff50-c8ce-4ca4-a810-20be3909b23c-original-pull-secret") pod "global-pull-secret-syncer-qcbcd" (UID: "a9bdff50-c8ce-4ca4-a810-20be3909b23c") : object "kube-system"/"original-pull-secret" not registered Jul 16 00:40:05.095350 ip-10-0-143-200 kubenswrapper[2572]: I0716 00:40:05.095308 2572 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/4a08cb1b-3bfa-49cf-8ff0-edef0ff944b9-metrics-certs\") pod \"network-metrics-daemon-5tphg\" (UID: \"4a08cb1b-3bfa-49cf-8ff0-edef0ff944b9\") " pod="openshift-multus/network-metrics-daemon-5tphg" Jul 16 00:40:05.095542 ip-10-0-143-200 kubenswrapper[2572]: E0716 00:40:05.095458 2572 secret.go:189] Couldn't get secret openshift-multus/metrics-daemon-secret: object "openshift-multus"/"metrics-daemon-secret" not registered Jul 16 00:40:05.095542 ip-10-0-143-200 kubenswrapper[2572]: E0716 00:40:05.095532 2572 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/4a08cb1b-3bfa-49cf-8ff0-edef0ff944b9-metrics-certs podName:4a08cb1b-3bfa-49cf-8ff0-edef0ff944b9 nodeName:}" failed. No retries permitted until 2026-07-16 00:40:13.095514114 +0000 UTC m=+18.204268780 (durationBeforeRetry 8s). Error: MountVolume.SetUp failed for volume "metrics-certs" (UniqueName: "kubernetes.io/secret/4a08cb1b-3bfa-49cf-8ff0-edef0ff944b9-metrics-certs") pod "network-metrics-daemon-5tphg" (UID: "4a08cb1b-3bfa-49cf-8ff0-edef0ff944b9") : object "openshift-multus"/"metrics-daemon-secret" not registered Jul 16 00:40:05.196520 ip-10-0-143-200 kubenswrapper[2572]: I0716 00:40:05.196446 2572 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-kpfbw\" (UniqueName: \"kubernetes.io/projected/8962f109-ea94-4296-a504-f6709315e9cd-kube-api-access-kpfbw\") pod \"network-check-target-52n54\" (UID: \"8962f109-ea94-4296-a504-f6709315e9cd\") " pod="openshift-network-diagnostics/network-check-target-52n54" Jul 16 00:40:05.196733 ip-10-0-143-200 kubenswrapper[2572]: E0716 00:40:05.196606 2572 projected.go:291] Couldn't get configMap openshift-network-diagnostics/kube-root-ca.crt: object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered Jul 16 00:40:05.196733 ip-10-0-143-200 kubenswrapper[2572]: E0716 00:40:05.196644 2572 projected.go:291] Couldn't get configMap openshift-network-diagnostics/openshift-service-ca.crt: object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered Jul 16 00:40:05.196733 ip-10-0-143-200 kubenswrapper[2572]: E0716 00:40:05.196659 2572 projected.go:196] Error preparing data for projected volume kube-api-access-kpfbw for pod openshift-network-diagnostics/network-check-target-52n54: [object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered, object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered] Jul 16 00:40:05.196733 ip-10-0-143-200 kubenswrapper[2572]: E0716 00:40:05.196729 2572 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/projected/8962f109-ea94-4296-a504-f6709315e9cd-kube-api-access-kpfbw podName:8962f109-ea94-4296-a504-f6709315e9cd nodeName:}" failed. No retries permitted until 2026-07-16 00:40:13.19670704 +0000 UTC m=+18.305461740 (durationBeforeRetry 8s). Error: MountVolume.SetUp failed for volume "kube-api-access-kpfbw" (UniqueName: "kubernetes.io/projected/8962f109-ea94-4296-a504-f6709315e9cd-kube-api-access-kpfbw") pod "network-check-target-52n54" (UID: "8962f109-ea94-4296-a504-f6709315e9cd") : [object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered, object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered] Jul 16 00:40:05.443598 ip-10-0-143-200 kubenswrapper[2572]: I0716 00:40:05.443297 2572 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-5tphg" Jul 16 00:40:05.443598 ip-10-0-143-200 kubenswrapper[2572]: E0716 00:40:05.443478 2572 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-5tphg" podUID="4a08cb1b-3bfa-49cf-8ff0-edef0ff944b9" Jul 16 00:40:05.443949 ip-10-0-143-200 kubenswrapper[2572]: I0716 00:40:05.443930 2572 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-qcbcd" Jul 16 00:40:05.444055 ip-10-0-143-200 kubenswrapper[2572]: E0716 00:40:05.444034 2572 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-qcbcd" podUID="a9bdff50-c8ce-4ca4-a810-20be3909b23c" Jul 16 00:40:05.444221 ip-10-0-143-200 kubenswrapper[2572]: I0716 00:40:05.444198 2572 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-52n54" Jul 16 00:40:05.444314 ip-10-0-143-200 kubenswrapper[2572]: E0716 00:40:05.444292 2572 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-52n54" podUID="8962f109-ea94-4296-a504-f6709315e9cd" Jul 16 00:40:07.441834 ip-10-0-143-200 kubenswrapper[2572]: I0716 00:40:07.441759 2572 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-qcbcd" Jul 16 00:40:07.442279 ip-10-0-143-200 kubenswrapper[2572]: I0716 00:40:07.441765 2572 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-5tphg" Jul 16 00:40:07.442279 ip-10-0-143-200 kubenswrapper[2572]: E0716 00:40:07.441894 2572 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-qcbcd" podUID="a9bdff50-c8ce-4ca4-a810-20be3909b23c" Jul 16 00:40:07.442279 ip-10-0-143-200 kubenswrapper[2572]: I0716 00:40:07.441924 2572 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-52n54" Jul 16 00:40:07.442279 ip-10-0-143-200 kubenswrapper[2572]: E0716 00:40:07.442030 2572 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-5tphg" podUID="4a08cb1b-3bfa-49cf-8ff0-edef0ff944b9" Jul 16 00:40:07.442279 ip-10-0-143-200 kubenswrapper[2572]: E0716 00:40:07.442111 2572 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-52n54" podUID="8962f109-ea94-4296-a504-f6709315e9cd" Jul 16 00:40:09.442295 ip-10-0-143-200 kubenswrapper[2572]: I0716 00:40:09.442203 2572 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-qcbcd" Jul 16 00:40:09.442295 ip-10-0-143-200 kubenswrapper[2572]: I0716 00:40:09.442253 2572 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-52n54" Jul 16 00:40:09.442801 ip-10-0-143-200 kubenswrapper[2572]: I0716 00:40:09.442203 2572 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-5tphg" Jul 16 00:40:09.442801 ip-10-0-143-200 kubenswrapper[2572]: E0716 00:40:09.442353 2572 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-qcbcd" podUID="a9bdff50-c8ce-4ca4-a810-20be3909b23c" Jul 16 00:40:09.442801 ip-10-0-143-200 kubenswrapper[2572]: E0716 00:40:09.442457 2572 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-5tphg" podUID="4a08cb1b-3bfa-49cf-8ff0-edef0ff944b9" Jul 16 00:40:09.442801 ip-10-0-143-200 kubenswrapper[2572]: E0716 00:40:09.442553 2572 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-52n54" podUID="8962f109-ea94-4296-a504-f6709315e9cd" Jul 16 00:40:11.442540 ip-10-0-143-200 kubenswrapper[2572]: I0716 00:40:11.442504 2572 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-52n54" Jul 16 00:40:11.443029 ip-10-0-143-200 kubenswrapper[2572]: I0716 00:40:11.442504 2572 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-5tphg" Jul 16 00:40:11.443029 ip-10-0-143-200 kubenswrapper[2572]: E0716 00:40:11.442657 2572 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-52n54" podUID="8962f109-ea94-4296-a504-f6709315e9cd" Jul 16 00:40:11.443029 ip-10-0-143-200 kubenswrapper[2572]: I0716 00:40:11.442504 2572 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-qcbcd" Jul 16 00:40:11.443029 ip-10-0-143-200 kubenswrapper[2572]: E0716 00:40:11.442750 2572 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-5tphg" podUID="4a08cb1b-3bfa-49cf-8ff0-edef0ff944b9" Jul 16 00:40:11.443029 ip-10-0-143-200 kubenswrapper[2572]: E0716 00:40:11.442808 2572 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-qcbcd" podUID="a9bdff50-c8ce-4ca4-a810-20be3909b23c" Jul 16 00:40:13.063237 ip-10-0-143-200 kubenswrapper[2572]: I0716 00:40:13.063197 2572 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"original-pull-secret\" (UniqueName: \"kubernetes.io/secret/a9bdff50-c8ce-4ca4-a810-20be3909b23c-original-pull-secret\") pod \"global-pull-secret-syncer-qcbcd\" (UID: \"a9bdff50-c8ce-4ca4-a810-20be3909b23c\") " pod="kube-system/global-pull-secret-syncer-qcbcd" Jul 16 00:40:13.063851 ip-10-0-143-200 kubenswrapper[2572]: E0716 00:40:13.063379 2572 secret.go:189] Couldn't get secret kube-system/original-pull-secret: object "kube-system"/"original-pull-secret" not registered Jul 16 00:40:13.063851 ip-10-0-143-200 kubenswrapper[2572]: E0716 00:40:13.063465 2572 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/a9bdff50-c8ce-4ca4-a810-20be3909b23c-original-pull-secret podName:a9bdff50-c8ce-4ca4-a810-20be3909b23c nodeName:}" failed. No retries permitted until 2026-07-16 00:40:29.063443343 +0000 UTC m=+34.172198012 (durationBeforeRetry 16s). Error: MountVolume.SetUp failed for volume "original-pull-secret" (UniqueName: "kubernetes.io/secret/a9bdff50-c8ce-4ca4-a810-20be3909b23c-original-pull-secret") pod "global-pull-secret-syncer-qcbcd" (UID: "a9bdff50-c8ce-4ca4-a810-20be3909b23c") : object "kube-system"/"original-pull-secret" not registered Jul 16 00:40:13.164013 ip-10-0-143-200 kubenswrapper[2572]: I0716 00:40:13.163980 2572 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/4a08cb1b-3bfa-49cf-8ff0-edef0ff944b9-metrics-certs\") pod \"network-metrics-daemon-5tphg\" (UID: \"4a08cb1b-3bfa-49cf-8ff0-edef0ff944b9\") " pod="openshift-multus/network-metrics-daemon-5tphg" Jul 16 00:40:13.164198 ip-10-0-143-200 kubenswrapper[2572]: E0716 00:40:13.164102 2572 secret.go:189] Couldn't get secret openshift-multus/metrics-daemon-secret: object "openshift-multus"/"metrics-daemon-secret" not registered Jul 16 00:40:13.164198 ip-10-0-143-200 kubenswrapper[2572]: E0716 00:40:13.164155 2572 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/4a08cb1b-3bfa-49cf-8ff0-edef0ff944b9-metrics-certs podName:4a08cb1b-3bfa-49cf-8ff0-edef0ff944b9 nodeName:}" failed. No retries permitted until 2026-07-16 00:40:29.16414081 +0000 UTC m=+34.272895463 (durationBeforeRetry 16s). Error: MountVolume.SetUp failed for volume "metrics-certs" (UniqueName: "kubernetes.io/secret/4a08cb1b-3bfa-49cf-8ff0-edef0ff944b9-metrics-certs") pod "network-metrics-daemon-5tphg" (UID: "4a08cb1b-3bfa-49cf-8ff0-edef0ff944b9") : object "openshift-multus"/"metrics-daemon-secret" not registered Jul 16 00:40:13.264502 ip-10-0-143-200 kubenswrapper[2572]: I0716 00:40:13.264460 2572 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-kpfbw\" (UniqueName: \"kubernetes.io/projected/8962f109-ea94-4296-a504-f6709315e9cd-kube-api-access-kpfbw\") pod \"network-check-target-52n54\" (UID: \"8962f109-ea94-4296-a504-f6709315e9cd\") " pod="openshift-network-diagnostics/network-check-target-52n54" Jul 16 00:40:13.264718 ip-10-0-143-200 kubenswrapper[2572]: E0716 00:40:13.264670 2572 projected.go:291] Couldn't get configMap openshift-network-diagnostics/kube-root-ca.crt: object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered Jul 16 00:40:13.264718 ip-10-0-143-200 kubenswrapper[2572]: E0716 00:40:13.264696 2572 projected.go:291] Couldn't get configMap openshift-network-diagnostics/openshift-service-ca.crt: object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered Jul 16 00:40:13.264718 ip-10-0-143-200 kubenswrapper[2572]: E0716 00:40:13.264708 2572 projected.go:196] Error preparing data for projected volume kube-api-access-kpfbw for pod openshift-network-diagnostics/network-check-target-52n54: [object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered, object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered] Jul 16 00:40:13.264941 ip-10-0-143-200 kubenswrapper[2572]: E0716 00:40:13.264774 2572 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/projected/8962f109-ea94-4296-a504-f6709315e9cd-kube-api-access-kpfbw podName:8962f109-ea94-4296-a504-f6709315e9cd nodeName:}" failed. No retries permitted until 2026-07-16 00:40:29.26475522 +0000 UTC m=+34.373509890 (durationBeforeRetry 16s). Error: MountVolume.SetUp failed for volume "kube-api-access-kpfbw" (UniqueName: "kubernetes.io/projected/8962f109-ea94-4296-a504-f6709315e9cd-kube-api-access-kpfbw") pod "network-check-target-52n54" (UID: "8962f109-ea94-4296-a504-f6709315e9cd") : [object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered, object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered] Jul 16 00:40:13.442196 ip-10-0-143-200 kubenswrapper[2572]: I0716 00:40:13.442105 2572 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-qcbcd" Jul 16 00:40:13.442196 ip-10-0-143-200 kubenswrapper[2572]: I0716 00:40:13.442134 2572 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-52n54" Jul 16 00:40:13.442407 ip-10-0-143-200 kubenswrapper[2572]: E0716 00:40:13.442235 2572 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-qcbcd" podUID="a9bdff50-c8ce-4ca4-a810-20be3909b23c" Jul 16 00:40:13.442407 ip-10-0-143-200 kubenswrapper[2572]: I0716 00:40:13.442297 2572 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-5tphg" Jul 16 00:40:13.442518 ip-10-0-143-200 kubenswrapper[2572]: E0716 00:40:13.442431 2572 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-52n54" podUID="8962f109-ea94-4296-a504-f6709315e9cd" Jul 16 00:40:13.442577 ip-10-0-143-200 kubenswrapper[2572]: E0716 00:40:13.442532 2572 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-5tphg" podUID="4a08cb1b-3bfa-49cf-8ff0-edef0ff944b9" Jul 16 00:40:15.442769 ip-10-0-143-200 kubenswrapper[2572]: I0716 00:40:15.442727 2572 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-qcbcd" Jul 16 00:40:15.443220 ip-10-0-143-200 kubenswrapper[2572]: I0716 00:40:15.442820 2572 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-5tphg" Jul 16 00:40:15.443220 ip-10-0-143-200 kubenswrapper[2572]: I0716 00:40:15.442849 2572 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-52n54" Jul 16 00:40:15.443220 ip-10-0-143-200 kubenswrapper[2572]: E0716 00:40:15.442890 2572 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-qcbcd" podUID="a9bdff50-c8ce-4ca4-a810-20be3909b23c" Jul 16 00:40:15.443220 ip-10-0-143-200 kubenswrapper[2572]: E0716 00:40:15.442921 2572 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-52n54" podUID="8962f109-ea94-4296-a504-f6709315e9cd" Jul 16 00:40:15.443220 ip-10-0-143-200 kubenswrapper[2572]: E0716 00:40:15.442978 2572 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-5tphg" podUID="4a08cb1b-3bfa-49cf-8ff0-edef0ff944b9" Jul 16 00:40:17.441758 ip-10-0-143-200 kubenswrapper[2572]: I0716 00:40:17.441732 2572 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-qcbcd" Jul 16 00:40:17.442305 ip-10-0-143-200 kubenswrapper[2572]: I0716 00:40:17.441732 2572 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-52n54" Jul 16 00:40:17.442305 ip-10-0-143-200 kubenswrapper[2572]: I0716 00:40:17.441866 2572 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-5tphg" Jul 16 00:40:17.442305 ip-10-0-143-200 kubenswrapper[2572]: E0716 00:40:17.441864 2572 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-qcbcd" podUID="a9bdff50-c8ce-4ca4-a810-20be3909b23c" Jul 16 00:40:17.442305 ip-10-0-143-200 kubenswrapper[2572]: E0716 00:40:17.441996 2572 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-52n54" podUID="8962f109-ea94-4296-a504-f6709315e9cd" Jul 16 00:40:17.442305 ip-10-0-143-200 kubenswrapper[2572]: E0716 00:40:17.442088 2572 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-5tphg" podUID="4a08cb1b-3bfa-49cf-8ff0-edef0ff944b9" Jul 16 00:40:17.576350 ip-10-0-143-200 kubenswrapper[2572]: I0716 00:40:17.576147 2572 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-multus/multus-szqbq" event={"ID":"2b89891b-83eb-48c2-abe4-747d1ca26caf","Type":"ContainerStarted","Data":"9704c4943e6b7aadcd2b50b9a0c846439a96cb48d77b5f6b0c018a900577ad59"} Jul 16 00:40:17.577556 ip-10-0-143-200 kubenswrapper[2572]: I0716 00:40:17.577534 2572 generic.go:356] "Generic (PLEG): container finished" podID="a51f2d32-4840-4f0c-804d-c8aa6015ae5a" containerID="a7ff1ac49339397d4a76b1ee7aed091ef0b680df11bdd657debda05add97681c" exitCode=0 Jul 16 00:40:17.577709 ip-10-0-143-200 kubenswrapper[2572]: I0716 00:40:17.577608 2572 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-multus/multus-additional-cni-plugins-vfntt" event={"ID":"a51f2d32-4840-4f0c-804d-c8aa6015ae5a","Type":"ContainerDied","Data":"a7ff1ac49339397d4a76b1ee7aed091ef0b680df11bdd657debda05add97681c"} Jul 16 00:40:17.579028 ip-10-0-143-200 kubenswrapper[2572]: I0716 00:40:17.579002 2572 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kube-system/konnectivity-agent-9gnmq" event={"ID":"3a7e030d-fad8-480d-9dc6-fe5057048cf9","Type":"ContainerStarted","Data":"65728812c9ae6e275b192dfc2aedd6de1e6809791c6d1478cecc9ad141beeb25"} Jul 16 00:40:17.580431 ip-10-0-143-200 kubenswrapper[2572]: I0716 00:40:17.580405 2572 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-pq88r" event={"ID":"f82be4fd-cc5f-4ff7-8b98-0f7497e4c802","Type":"ContainerStarted","Data":"3a8bd1c774513c320f053ed96a519ffcd8a202f5d98daed9245d945983745d12"} Jul 16 00:40:17.583762 ip-10-0-143-200 kubenswrapper[2572]: I0716 00:40:17.583735 2572 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-cluster-node-tuning-operator/tuned-w6p89" event={"ID":"2614f4d9-8dc4-406b-a5aa-7dd2e7446008","Type":"ContainerStarted","Data":"84de746864ad474a5a83e04cb6baa226e40a2997000850e9b4e5d2cefbbd292f"} Jul 16 00:40:17.585901 ip-10-0-143-200 kubenswrapper[2572]: I0716 00:40:17.585878 2572 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-dns/node-resolver-6qlgz" event={"ID":"24fd12c1-ddcb-427f-ac7e-4e2877450cb8","Type":"ContainerStarted","Data":"28c2bfd42eecf2ec87515ab62a01ce819c474a9e755681e7a6af97e0648c7840"} Jul 16 00:40:17.587248 ip-10-0-143-200 kubenswrapper[2572]: I0716 00:40:17.587229 2572 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-image-registry/node-ca-6msnd" event={"ID":"ff468ef8-66ba-48f2-8531-da662a5b19d3","Type":"ContainerStarted","Data":"0c785474b0f3c23b172bdd708bc45fd7bdcd3157a6961bf2f28fe947c8285f9e"} Jul 16 00:40:17.589463 ip-10-0-143-200 kubenswrapper[2572]: I0716 00:40:17.589419 2572 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ovn-kubernetes_ovnkube-node-fp78g_aa66f2b6-0ad8-45d0-8ff5-40fc21966b5a/ovn-acl-logging/0.log" Jul 16 00:40:17.589773 ip-10-0-143-200 kubenswrapper[2572]: I0716 00:40:17.589745 2572 generic.go:356] "Generic (PLEG): container finished" podID="aa66f2b6-0ad8-45d0-8ff5-40fc21966b5a" containerID="42a467d4b358d7fc43360367ca6a60d32bb57666019a47a40dd51ee3363c7b50" exitCode=1 Jul 16 00:40:17.589834 ip-10-0-143-200 kubenswrapper[2572]: I0716 00:40:17.589779 2572 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ovn-kubernetes/ovnkube-node-fp78g" event={"ID":"aa66f2b6-0ad8-45d0-8ff5-40fc21966b5a","Type":"ContainerStarted","Data":"7ae782f298a67db6bf47714dd645d3b100f315ef51c05d9c3b0abcc608a653cf"} Jul 16 00:40:17.589834 ip-10-0-143-200 kubenswrapper[2572]: I0716 00:40:17.589830 2572 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ovn-kubernetes/ovnkube-node-fp78g" event={"ID":"aa66f2b6-0ad8-45d0-8ff5-40fc21966b5a","Type":"ContainerStarted","Data":"b2cadff5846a01a788446a2d28699efc3d97d4088c2be36b58852ab116ec21c6"} Jul 16 00:40:17.589897 ip-10-0-143-200 kubenswrapper[2572]: I0716 00:40:17.589859 2572 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ovn-kubernetes/ovnkube-node-fp78g" event={"ID":"aa66f2b6-0ad8-45d0-8ff5-40fc21966b5a","Type":"ContainerDied","Data":"42a467d4b358d7fc43360367ca6a60d32bb57666019a47a40dd51ee3363c7b50"} Jul 16 00:40:17.589897 ip-10-0-143-200 kubenswrapper[2572]: I0716 00:40:17.589889 2572 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ovn-kubernetes/ovnkube-node-fp78g" event={"ID":"aa66f2b6-0ad8-45d0-8ff5-40fc21966b5a","Type":"ContainerStarted","Data":"923cde2dbefd770ce5a740164ff154ca28cdbd45e01606b7a621b7e3af037f42"} Jul 16 00:40:17.594665 ip-10-0-143-200 kubenswrapper[2572]: I0716 00:40:17.594616 2572 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-143-200.ec2.internal" podStartSLOduration=21.594604925 podStartE2EDuration="21.594604925s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=true podCreationTimestamp="2026-07-16 00:39:56 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-16 00:40:00.558756618 +0000 UTC m=+5.667511294" watchObservedRunningTime="2026-07-16 00:40:17.594604925 +0000 UTC m=+22.703359650" Jul 16 00:40:17.611514 ip-10-0-143-200 kubenswrapper[2572]: I0716 00:40:17.611476 2572 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-multus/multus-szqbq" podStartSLOduration=3.961962101 podStartE2EDuration="22.611461833s" totalImagesPullingTime="18.649499732s" totalInitContainerRuntime="0s" isStatefulPod=true podCreationTimestamp="2026-07-16 00:39:55 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-16 00:40:17.594736165 +0000 UTC m=+22.703490823" watchObservedRunningTime="2026-07-16 00:40:17.611461833 +0000 UTC m=+22.720216490" Jul 16 00:40:17.611746 ip-10-0-143-200 kubenswrapper[2572]: I0716 00:40:17.611724 2572 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-cluster-node-tuning-operator/tuned-w6p89" podStartSLOduration=4.015678495 podStartE2EDuration="22.61171957s" totalImagesPullingTime="18.596041075s" totalInitContainerRuntime="0s" isStatefulPod=true podCreationTimestamp="2026-07-16 00:39:55 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-16 00:40:17.611418921 +0000 UTC m=+22.720173596" watchObservedRunningTime="2026-07-16 00:40:17.61171957 +0000 UTC m=+22.720474345" Jul 16 00:40:17.627215 ip-10-0-143-200 kubenswrapper[2572]: I0716 00:40:17.627172 2572 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="kube-system/konnectivity-agent-9gnmq" podStartSLOduration=12.037737214 podStartE2EDuration="22.62715683s" totalImagesPullingTime="10.589419616s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-16 00:39:55 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-16 00:40:17.626905465 +0000 UTC m=+22.735660139" watchObservedRunningTime="2026-07-16 00:40:17.62715683 +0000 UTC m=+22.735911504" Jul 16 00:40:17.711246 ip-10-0-143-200 kubenswrapper[2572]: I0716 00:40:17.711202 2572 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-image-registry/node-ca-6msnd" podStartSLOduration=4.126775301 podStartE2EDuration="22.711185854s" totalImagesPullingTime="18.584410553s" totalInitContainerRuntime="0s" isStatefulPod=true podCreationTimestamp="2026-07-16 00:39:55 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-16 00:40:17.711178422 +0000 UTC m=+22.819933099" watchObservedRunningTime="2026-07-16 00:40:17.711185854 +0000 UTC m=+22.819940529" Jul 16 00:40:17.727300 ip-10-0-143-200 kubenswrapper[2572]: I0716 00:40:17.727237 2572 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-dns/node-resolver-6qlgz" podStartSLOduration=4.127134974 podStartE2EDuration="22.727214299s" totalImagesPullingTime="18.600079325s" totalInitContainerRuntime="0s" isStatefulPod=true podCreationTimestamp="2026-07-16 00:39:55 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-16 00:40:17.727001415 +0000 UTC m=+22.835756091" watchObservedRunningTime="2026-07-16 00:40:17.727214299 +0000 UTC m=+22.835968974" Jul 16 00:40:18.600591 ip-10-0-143-200 kubenswrapper[2572]: I0716 00:40:18.600483 2572 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ovn-kubernetes_ovnkube-node-fp78g_aa66f2b6-0ad8-45d0-8ff5-40fc21966b5a/ovn-acl-logging/0.log" Jul 16 00:40:18.601281 ip-10-0-143-200 kubenswrapper[2572]: I0716 00:40:18.600948 2572 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ovn-kubernetes/ovnkube-node-fp78g" event={"ID":"aa66f2b6-0ad8-45d0-8ff5-40fc21966b5a","Type":"ContainerStarted","Data":"1c9142a9e2ff049728db895d0344a6e220d1a0e827f43d88af2debec3fa477d1"} Jul 16 00:40:18.601281 ip-10-0-143-200 kubenswrapper[2572]: I0716 00:40:18.601039 2572 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ovn-kubernetes/ovnkube-node-fp78g" event={"ID":"aa66f2b6-0ad8-45d0-8ff5-40fc21966b5a","Type":"ContainerStarted","Data":"2eeb7c8ca0d7d27e6c48ab5a0e79680f8c85042b8f405e710fdaa5bc81de6bae"} Jul 16 00:40:18.602837 ip-10-0-143-200 kubenswrapper[2572]: I0716 00:40:18.602808 2572 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-network-operator/iptables-alerter-gcstg" event={"ID":"0247608f-bb0b-4ace-b071-292d5586fc65","Type":"ContainerStarted","Data":"fa9f7d389526a86d89b821b6a7bed677c76834b53f9013c7939f185dbab422aa"} Jul 16 00:40:18.610177 ip-10-0-143-200 kubenswrapper[2572]: I0716 00:40:18.610153 2572 plugin_watcher.go:194] "Adding socket path or updating timestamp to desired state cache" path="/var/lib/kubelet/plugins_registry/ebs.csi.aws.com-reg.sock" Jul 16 00:40:19.235827 ip-10-0-143-200 kubenswrapper[2572]: I0716 00:40:19.235783 2572 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="kube-system/konnectivity-agent-9gnmq" Jul 16 00:40:19.236416 ip-10-0-143-200 kubenswrapper[2572]: I0716 00:40:19.236391 2572 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="kube-system/konnectivity-agent-9gnmq" Jul 16 00:40:19.254969 ip-10-0-143-200 kubenswrapper[2572]: I0716 00:40:19.254907 2572 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-network-operator/iptables-alerter-gcstg" podStartSLOduration=5.665590879 podStartE2EDuration="24.254883326s" totalImagesPullingTime="18.589292447s" totalInitContainerRuntime="0s" isStatefulPod=true podCreationTimestamp="2026-07-16 00:39:55 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-16 00:40:18.621248802 +0000 UTC m=+23.730003474" watchObservedRunningTime="2026-07-16 00:40:19.254883326 +0000 UTC m=+24.363638002" Jul 16 00:40:19.393488 ip-10-0-143-200 kubenswrapper[2572]: I0716 00:40:19.393367 2572 reconciler.go:161] "OperationExecutor.RegisterPlugin started" plugin={"SocketPath":"/var/lib/kubelet/plugins_registry/ebs.csi.aws.com-reg.sock","Timestamp":"2026-07-16T00:40:18.610171883Z","UUID":"f98e1147-5f25-4bca-9289-e2d367c425a6","Handler":null,"Name":"","Endpoint":""} Jul 16 00:40:19.395552 ip-10-0-143-200 kubenswrapper[2572]: I0716 00:40:19.395516 2572 csi_plugin.go:106] kubernetes.io/csi: Trying to validate a new CSI Driver with name: ebs.csi.aws.com endpoint: /var/lib/kubelet/plugins/ebs.csi.aws.com/csi.sock versions: 1.0.0 Jul 16 00:40:19.395710 ip-10-0-143-200 kubenswrapper[2572]: I0716 00:40:19.395569 2572 csi_plugin.go:119] kubernetes.io/csi: Register new plugin with name: ebs.csi.aws.com at endpoint: /var/lib/kubelet/plugins/ebs.csi.aws.com/csi.sock Jul 16 00:40:19.441941 ip-10-0-143-200 kubenswrapper[2572]: I0716 00:40:19.441902 2572 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-5tphg" Jul 16 00:40:19.442146 ip-10-0-143-200 kubenswrapper[2572]: E0716 00:40:19.442106 2572 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-5tphg" podUID="4a08cb1b-3bfa-49cf-8ff0-edef0ff944b9" Jul 16 00:40:19.442272 ip-10-0-143-200 kubenswrapper[2572]: I0716 00:40:19.442212 2572 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-qcbcd" Jul 16 00:40:19.442351 ip-10-0-143-200 kubenswrapper[2572]: E0716 00:40:19.442317 2572 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-qcbcd" podUID="a9bdff50-c8ce-4ca4-a810-20be3909b23c" Jul 16 00:40:19.442476 ip-10-0-143-200 kubenswrapper[2572]: I0716 00:40:19.442397 2572 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-52n54" Jul 16 00:40:19.442534 ip-10-0-143-200 kubenswrapper[2572]: E0716 00:40:19.442499 2572 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-52n54" podUID="8962f109-ea94-4296-a504-f6709315e9cd" Jul 16 00:40:19.607019 ip-10-0-143-200 kubenswrapper[2572]: I0716 00:40:19.606918 2572 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-pq88r" event={"ID":"f82be4fd-cc5f-4ff7-8b98-0f7497e4c802","Type":"ContainerStarted","Data":"b16e2287440b635edc74e6bd0dbed2b9e0abd4d5f647d628cd25f4bb73e1346d"} Jul 16 00:40:19.607456 ip-10-0-143-200 kubenswrapper[2572]: I0716 00:40:19.607411 2572 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="kube-system/konnectivity-agent-9gnmq" Jul 16 00:40:19.607761 ip-10-0-143-200 kubenswrapper[2572]: I0716 00:40:19.607738 2572 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="kube-system/konnectivity-agent-9gnmq" Jul 16 00:40:20.612723 ip-10-0-143-200 kubenswrapper[2572]: I0716 00:40:20.612689 2572 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ovn-kubernetes_ovnkube-node-fp78g_aa66f2b6-0ad8-45d0-8ff5-40fc21966b5a/ovn-acl-logging/0.log" Jul 16 00:40:20.613173 ip-10-0-143-200 kubenswrapper[2572]: I0716 00:40:20.613145 2572 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ovn-kubernetes/ovnkube-node-fp78g" event={"ID":"aa66f2b6-0ad8-45d0-8ff5-40fc21966b5a","Type":"ContainerStarted","Data":"253269ffbf2be97b796d0b730957e5e299c49bb2aed09c77494531cc463a9e0f"} Jul 16 00:40:20.615287 ip-10-0-143-200 kubenswrapper[2572]: I0716 00:40:20.615264 2572 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-pq88r" event={"ID":"f82be4fd-cc5f-4ff7-8b98-0f7497e4c802","Type":"ContainerStarted","Data":"5cee9ae16107cc4dba989bf26222b6597d82fa6ffa20d01b2a1cb01622c4c16f"} Jul 16 00:40:20.634173 ip-10-0-143-200 kubenswrapper[2572]: I0716 00:40:20.634109 2572 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-pq88r" podStartSLOduration=4.592070527 podStartE2EDuration="25.634086846s" totalImagesPullingTime="21.042016319s" totalInitContainerRuntime="0s" isStatefulPod=true podCreationTimestamp="2026-07-16 00:39:55 +0000 UTC" imagePullSessionsCount=3 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-16 00:40:20.634039214 +0000 UTC m=+25.742793889" watchObservedRunningTime="2026-07-16 00:40:20.634086846 +0000 UTC m=+25.742841504" Jul 16 00:40:21.442442 ip-10-0-143-200 kubenswrapper[2572]: I0716 00:40:21.442404 2572 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-5tphg" Jul 16 00:40:21.442442 ip-10-0-143-200 kubenswrapper[2572]: I0716 00:40:21.442431 2572 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-qcbcd" Jul 16 00:40:21.442719 ip-10-0-143-200 kubenswrapper[2572]: I0716 00:40:21.442428 2572 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-52n54" Jul 16 00:40:21.442719 ip-10-0-143-200 kubenswrapper[2572]: E0716 00:40:21.442575 2572 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-5tphg" podUID="4a08cb1b-3bfa-49cf-8ff0-edef0ff944b9" Jul 16 00:40:21.442840 ip-10-0-143-200 kubenswrapper[2572]: E0716 00:40:21.442710 2572 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-qcbcd" podUID="a9bdff50-c8ce-4ca4-a810-20be3909b23c" Jul 16 00:40:21.442840 ip-10-0-143-200 kubenswrapper[2572]: E0716 00:40:21.442806 2572 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-52n54" podUID="8962f109-ea94-4296-a504-f6709315e9cd" Jul 16 00:40:22.623803 ip-10-0-143-200 kubenswrapper[2572]: I0716 00:40:22.623774 2572 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ovn-kubernetes_ovnkube-node-fp78g_aa66f2b6-0ad8-45d0-8ff5-40fc21966b5a/ovn-acl-logging/0.log" Jul 16 00:40:22.624434 ip-10-0-143-200 kubenswrapper[2572]: I0716 00:40:22.624144 2572 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ovn-kubernetes/ovnkube-node-fp78g" event={"ID":"aa66f2b6-0ad8-45d0-8ff5-40fc21966b5a","Type":"ContainerStarted","Data":"16b56a3335e7e34e0bc8e85466f6ca5e525896ba4375a717390a3174e958c165"} Jul 16 00:40:22.624434 ip-10-0-143-200 kubenswrapper[2572]: I0716 00:40:22.624366 2572 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ovn-kubernetes/ovnkube-node-fp78g" Jul 16 00:40:22.624434 ip-10-0-143-200 kubenswrapper[2572]: I0716 00:40:22.624426 2572 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ovn-kubernetes/ovnkube-node-fp78g" Jul 16 00:40:22.624697 ip-10-0-143-200 kubenswrapper[2572]: I0716 00:40:22.624624 2572 scope.go:117] "RemoveContainer" containerID="42a467d4b358d7fc43360367ca6a60d32bb57666019a47a40dd51ee3363c7b50" Jul 16 00:40:22.626472 ip-10-0-143-200 kubenswrapper[2572]: I0716 00:40:22.626445 2572 generic.go:356] "Generic (PLEG): container finished" podID="a51f2d32-4840-4f0c-804d-c8aa6015ae5a" containerID="f5f4575c07673877d24b0a92c60f77ff127827508d9a1ac893fd1124c43c84cb" exitCode=0 Jul 16 00:40:22.626587 ip-10-0-143-200 kubenswrapper[2572]: I0716 00:40:22.626486 2572 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-multus/multus-additional-cni-plugins-vfntt" event={"ID":"a51f2d32-4840-4f0c-804d-c8aa6015ae5a","Type":"ContainerDied","Data":"f5f4575c07673877d24b0a92c60f77ff127827508d9a1ac893fd1124c43c84cb"} Jul 16 00:40:22.640777 ip-10-0-143-200 kubenswrapper[2572]: I0716 00:40:22.640756 2572 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ovn-kubernetes/ovnkube-node-fp78g" Jul 16 00:40:22.640922 ip-10-0-143-200 kubenswrapper[2572]: I0716 00:40:22.640907 2572 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ovn-kubernetes/ovnkube-node-fp78g" Jul 16 00:40:23.443322 ip-10-0-143-200 kubenswrapper[2572]: I0716 00:40:23.443278 2572 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-52n54" Jul 16 00:40:23.443583 ip-10-0-143-200 kubenswrapper[2572]: I0716 00:40:23.443410 2572 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-5tphg" Jul 16 00:40:23.443583 ip-10-0-143-200 kubenswrapper[2572]: E0716 00:40:23.443416 2572 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-52n54" podUID="8962f109-ea94-4296-a504-f6709315e9cd" Jul 16 00:40:23.443583 ip-10-0-143-200 kubenswrapper[2572]: I0716 00:40:23.443446 2572 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-qcbcd" Jul 16 00:40:23.443583 ip-10-0-143-200 kubenswrapper[2572]: E0716 00:40:23.443536 2572 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-5tphg" podUID="4a08cb1b-3bfa-49cf-8ff0-edef0ff944b9" Jul 16 00:40:23.443824 ip-10-0-143-200 kubenswrapper[2572]: E0716 00:40:23.443616 2572 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-qcbcd" podUID="a9bdff50-c8ce-4ca4-a810-20be3909b23c" Jul 16 00:40:23.631962 ip-10-0-143-200 kubenswrapper[2572]: I0716 00:40:23.631933 2572 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ovn-kubernetes_ovnkube-node-fp78g_aa66f2b6-0ad8-45d0-8ff5-40fc21966b5a/ovn-acl-logging/0.log" Jul 16 00:40:23.632413 ip-10-0-143-200 kubenswrapper[2572]: I0716 00:40:23.632305 2572 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ovn-kubernetes/ovnkube-node-fp78g" event={"ID":"aa66f2b6-0ad8-45d0-8ff5-40fc21966b5a","Type":"ContainerStarted","Data":"7ea436b6d1a014e7948e86a0a62dd0bbb9a0c9a67eddf8bd429604dbca05ea90"} Jul 16 00:40:23.632413 ip-10-0-143-200 kubenswrapper[2572]: I0716 00:40:23.632374 2572 prober_manager.go:312] "Failed to trigger a manual run" probe="Readiness" Jul 16 00:40:24.610965 ip-10-0-143-200 kubenswrapper[2572]: I0716 00:40:24.610904 2572 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-ovn-kubernetes/ovnkube-node-fp78g" podStartSLOduration=10.956598568 podStartE2EDuration="29.610884018s" totalImagesPullingTime="18.65428545s" totalInitContainerRuntime="0s" isStatefulPod=true podCreationTimestamp="2026-07-16 00:39:55 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-16 00:40:23.717507337 +0000 UTC m=+28.826262012" watchObservedRunningTime="2026-07-16 00:40:24.610884018 +0000 UTC m=+29.719638722" Jul 16 00:40:24.611472 ip-10-0-143-200 kubenswrapper[2572]: I0716 00:40:24.611452 2572 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-multus/network-metrics-daemon-5tphg"] Jul 16 00:40:24.611589 ip-10-0-143-200 kubenswrapper[2572]: I0716 00:40:24.611570 2572 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-5tphg" Jul 16 00:40:24.611720 ip-10-0-143-200 kubenswrapper[2572]: E0716 00:40:24.611702 2572 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-5tphg" podUID="4a08cb1b-3bfa-49cf-8ff0-edef0ff944b9" Jul 16 00:40:24.614987 ip-10-0-143-200 kubenswrapper[2572]: I0716 00:40:24.614960 2572 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-network-diagnostics/network-check-target-52n54"] Jul 16 00:40:24.615092 ip-10-0-143-200 kubenswrapper[2572]: I0716 00:40:24.615061 2572 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-52n54" Jul 16 00:40:24.615322 ip-10-0-143-200 kubenswrapper[2572]: E0716 00:40:24.615158 2572 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-52n54" podUID="8962f109-ea94-4296-a504-f6709315e9cd" Jul 16 00:40:24.630258 ip-10-0-143-200 kubenswrapper[2572]: I0716 00:40:24.630228 2572 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kube-system/global-pull-secret-syncer-qcbcd"] Jul 16 00:40:24.630379 ip-10-0-143-200 kubenswrapper[2572]: I0716 00:40:24.630323 2572 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-qcbcd" Jul 16 00:40:24.630424 ip-10-0-143-200 kubenswrapper[2572]: E0716 00:40:24.630403 2572 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-qcbcd" podUID="a9bdff50-c8ce-4ca4-a810-20be3909b23c" Jul 16 00:40:24.637418 ip-10-0-143-200 kubenswrapper[2572]: I0716 00:40:24.637386 2572 generic.go:356] "Generic (PLEG): container finished" podID="a51f2d32-4840-4f0c-804d-c8aa6015ae5a" containerID="b20059130d5d0ed61aad8f997e4e588bbf631492803ba9ee36e8b141d5e4fce6" exitCode=0 Jul 16 00:40:24.637846 ip-10-0-143-200 kubenswrapper[2572]: I0716 00:40:24.637460 2572 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-multus/multus-additional-cni-plugins-vfntt" event={"ID":"a51f2d32-4840-4f0c-804d-c8aa6015ae5a","Type":"ContainerDied","Data":"b20059130d5d0ed61aad8f997e4e588bbf631492803ba9ee36e8b141d5e4fce6"} Jul 16 00:40:24.637846 ip-10-0-143-200 kubenswrapper[2572]: I0716 00:40:24.637560 2572 prober_manager.go:312] "Failed to trigger a manual run" probe="Readiness" Jul 16 00:40:26.441741 ip-10-0-143-200 kubenswrapper[2572]: I0716 00:40:26.441653 2572 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-5tphg" Jul 16 00:40:26.441741 ip-10-0-143-200 kubenswrapper[2572]: I0716 00:40:26.441676 2572 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-52n54" Jul 16 00:40:26.442164 ip-10-0-143-200 kubenswrapper[2572]: E0716 00:40:26.441771 2572 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-5tphg" podUID="4a08cb1b-3bfa-49cf-8ff0-edef0ff944b9" Jul 16 00:40:26.442164 ip-10-0-143-200 kubenswrapper[2572]: I0716 00:40:26.441825 2572 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-qcbcd" Jul 16 00:40:26.442164 ip-10-0-143-200 kubenswrapper[2572]: E0716 00:40:26.441899 2572 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-52n54" podUID="8962f109-ea94-4296-a504-f6709315e9cd" Jul 16 00:40:26.442164 ip-10-0-143-200 kubenswrapper[2572]: E0716 00:40:26.441971 2572 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-qcbcd" podUID="a9bdff50-c8ce-4ca4-a810-20be3909b23c" Jul 16 00:40:26.643950 ip-10-0-143-200 kubenswrapper[2572]: I0716 00:40:26.643774 2572 generic.go:356] "Generic (PLEG): container finished" podID="a51f2d32-4840-4f0c-804d-c8aa6015ae5a" containerID="8f90e9fc19ef719558b291ccae3d20bfea2341f8624680481940004e4385e443" exitCode=0 Jul 16 00:40:26.644109 ip-10-0-143-200 kubenswrapper[2572]: I0716 00:40:26.643838 2572 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-multus/multus-additional-cni-plugins-vfntt" event={"ID":"a51f2d32-4840-4f0c-804d-c8aa6015ae5a","Type":"ContainerDied","Data":"8f90e9fc19ef719558b291ccae3d20bfea2341f8624680481940004e4385e443"} Jul 16 00:40:27.602957 ip-10-0-143-200 kubenswrapper[2572]: I0716 00:40:27.602921 2572 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ovn-kubernetes/ovnkube-node-fp78g" Jul 16 00:40:27.603466 ip-10-0-143-200 kubenswrapper[2572]: I0716 00:40:27.603197 2572 prober_manager.go:312] "Failed to trigger a manual run" probe="Readiness" Jul 16 00:40:27.622413 ip-10-0-143-200 kubenswrapper[2572]: I0716 00:40:27.622364 2572 prober.go:120] "Probe failed" probeType="Readiness" pod="openshift-ovn-kubernetes/ovnkube-node-fp78g" podUID="aa66f2b6-0ad8-45d0-8ff5-40fc21966b5a" containerName="ovnkube-controller" probeResult="failure" output="" Jul 16 00:40:27.631907 ip-10-0-143-200 kubenswrapper[2572]: I0716 00:40:27.631872 2572 prober.go:120] "Probe failed" probeType="Readiness" pod="openshift-ovn-kubernetes/ovnkube-node-fp78g" podUID="aa66f2b6-0ad8-45d0-8ff5-40fc21966b5a" containerName="ovnkube-controller" probeResult="failure" output="" Jul 16 00:40:28.442155 ip-10-0-143-200 kubenswrapper[2572]: I0716 00:40:28.442116 2572 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-qcbcd" Jul 16 00:40:28.442155 ip-10-0-143-200 kubenswrapper[2572]: I0716 00:40:28.442156 2572 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-52n54" Jul 16 00:40:28.442411 ip-10-0-143-200 kubenswrapper[2572]: I0716 00:40:28.442161 2572 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-5tphg" Jul 16 00:40:28.442411 ip-10-0-143-200 kubenswrapper[2572]: E0716 00:40:28.442255 2572 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-qcbcd" podUID="a9bdff50-c8ce-4ca4-a810-20be3909b23c" Jul 16 00:40:28.442411 ip-10-0-143-200 kubenswrapper[2572]: E0716 00:40:28.442334 2572 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-52n54" podUID="8962f109-ea94-4296-a504-f6709315e9cd" Jul 16 00:40:28.442538 ip-10-0-143-200 kubenswrapper[2572]: E0716 00:40:28.442444 2572 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-5tphg" podUID="4a08cb1b-3bfa-49cf-8ff0-edef0ff944b9" Jul 16 00:40:29.083554 ip-10-0-143-200 kubenswrapper[2572]: I0716 00:40:29.083504 2572 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"original-pull-secret\" (UniqueName: \"kubernetes.io/secret/a9bdff50-c8ce-4ca4-a810-20be3909b23c-original-pull-secret\") pod \"global-pull-secret-syncer-qcbcd\" (UID: \"a9bdff50-c8ce-4ca4-a810-20be3909b23c\") " pod="kube-system/global-pull-secret-syncer-qcbcd" Jul 16 00:40:29.084049 ip-10-0-143-200 kubenswrapper[2572]: E0716 00:40:29.083616 2572 secret.go:189] Couldn't get secret kube-system/original-pull-secret: object "kube-system"/"original-pull-secret" not registered Jul 16 00:40:29.084049 ip-10-0-143-200 kubenswrapper[2572]: E0716 00:40:29.083716 2572 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/a9bdff50-c8ce-4ca4-a810-20be3909b23c-original-pull-secret podName:a9bdff50-c8ce-4ca4-a810-20be3909b23c nodeName:}" failed. No retries permitted until 2026-07-16 00:41:01.083694816 +0000 UTC m=+66.192449483 (durationBeforeRetry 32s). Error: MountVolume.SetUp failed for volume "original-pull-secret" (UniqueName: "kubernetes.io/secret/a9bdff50-c8ce-4ca4-a810-20be3909b23c-original-pull-secret") pod "global-pull-secret-syncer-qcbcd" (UID: "a9bdff50-c8ce-4ca4-a810-20be3909b23c") : object "kube-system"/"original-pull-secret" not registered Jul 16 00:40:29.184203 ip-10-0-143-200 kubenswrapper[2572]: I0716 00:40:29.184170 2572 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/4a08cb1b-3bfa-49cf-8ff0-edef0ff944b9-metrics-certs\") pod \"network-metrics-daemon-5tphg\" (UID: \"4a08cb1b-3bfa-49cf-8ff0-edef0ff944b9\") " pod="openshift-multus/network-metrics-daemon-5tphg" Jul 16 00:40:29.184381 ip-10-0-143-200 kubenswrapper[2572]: E0716 00:40:29.184284 2572 secret.go:189] Couldn't get secret openshift-multus/metrics-daemon-secret: object "openshift-multus"/"metrics-daemon-secret" not registered Jul 16 00:40:29.184381 ip-10-0-143-200 kubenswrapper[2572]: E0716 00:40:29.184339 2572 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/4a08cb1b-3bfa-49cf-8ff0-edef0ff944b9-metrics-certs podName:4a08cb1b-3bfa-49cf-8ff0-edef0ff944b9 nodeName:}" failed. No retries permitted until 2026-07-16 00:41:01.184323389 +0000 UTC m=+66.293078056 (durationBeforeRetry 32s). Error: MountVolume.SetUp failed for volume "metrics-certs" (UniqueName: "kubernetes.io/secret/4a08cb1b-3bfa-49cf-8ff0-edef0ff944b9-metrics-certs") pod "network-metrics-daemon-5tphg" (UID: "4a08cb1b-3bfa-49cf-8ff0-edef0ff944b9") : object "openshift-multus"/"metrics-daemon-secret" not registered Jul 16 00:40:29.284550 ip-10-0-143-200 kubenswrapper[2572]: I0716 00:40:29.284519 2572 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-kpfbw\" (UniqueName: \"kubernetes.io/projected/8962f109-ea94-4296-a504-f6709315e9cd-kube-api-access-kpfbw\") pod \"network-check-target-52n54\" (UID: \"8962f109-ea94-4296-a504-f6709315e9cd\") " pod="openshift-network-diagnostics/network-check-target-52n54" Jul 16 00:40:29.284726 ip-10-0-143-200 kubenswrapper[2572]: E0716 00:40:29.284705 2572 projected.go:291] Couldn't get configMap openshift-network-diagnostics/kube-root-ca.crt: object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered Jul 16 00:40:29.284776 ip-10-0-143-200 kubenswrapper[2572]: E0716 00:40:29.284732 2572 projected.go:291] Couldn't get configMap openshift-network-diagnostics/openshift-service-ca.crt: object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered Jul 16 00:40:29.284776 ip-10-0-143-200 kubenswrapper[2572]: E0716 00:40:29.284741 2572 projected.go:196] Error preparing data for projected volume kube-api-access-kpfbw for pod openshift-network-diagnostics/network-check-target-52n54: [object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered, object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered] Jul 16 00:40:29.284837 ip-10-0-143-200 kubenswrapper[2572]: E0716 00:40:29.284797 2572 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/projected/8962f109-ea94-4296-a504-f6709315e9cd-kube-api-access-kpfbw podName:8962f109-ea94-4296-a504-f6709315e9cd nodeName:}" failed. No retries permitted until 2026-07-16 00:41:01.284778186 +0000 UTC m=+66.393532856 (durationBeforeRetry 32s). Error: MountVolume.SetUp failed for volume "kube-api-access-kpfbw" (UniqueName: "kubernetes.io/projected/8962f109-ea94-4296-a504-f6709315e9cd-kube-api-access-kpfbw") pod "network-check-target-52n54" (UID: "8962f109-ea94-4296-a504-f6709315e9cd") : [object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered, object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered] Jul 16 00:40:30.441926 ip-10-0-143-200 kubenswrapper[2572]: I0716 00:40:30.441882 2572 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-5tphg" Jul 16 00:40:30.441926 ip-10-0-143-200 kubenswrapper[2572]: I0716 00:40:30.441914 2572 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-qcbcd" Jul 16 00:40:30.442428 ip-10-0-143-200 kubenswrapper[2572]: E0716 00:40:30.442032 2572 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-5tphg" podUID="4a08cb1b-3bfa-49cf-8ff0-edef0ff944b9" Jul 16 00:40:30.442428 ip-10-0-143-200 kubenswrapper[2572]: E0716 00:40:30.442107 2572 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-qcbcd" podUID="a9bdff50-c8ce-4ca4-a810-20be3909b23c" Jul 16 00:40:30.442428 ip-10-0-143-200 kubenswrapper[2572]: I0716 00:40:30.442140 2572 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-52n54" Jul 16 00:40:30.442428 ip-10-0-143-200 kubenswrapper[2572]: E0716 00:40:30.442240 2572 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-52n54" podUID="8962f109-ea94-4296-a504-f6709315e9cd" Jul 16 00:40:30.626979 ip-10-0-143-200 kubenswrapper[2572]: I0716 00:40:30.626939 2572 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-143-200.ec2.internal" event="NodeReady" Jul 16 00:40:30.627176 ip-10-0-143-200 kubenswrapper[2572]: I0716 00:40:30.627136 2572 kubelet_node_status.go:489] "Fast updating node status as it just became ready" Jul 16 00:40:30.676325 ip-10-0-143-200 kubenswrapper[2572]: I0716 00:40:30.676285 2572 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-image-registry/image-registry-dd7c989d9-5w25c"] Jul 16 00:40:30.678534 ip-10-0-143-200 kubenswrapper[2572]: I0716 00:40:30.678504 2572 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-image-registry/image-registry-dd7c989d9-5w25c" Jul 16 00:40:30.682849 ip-10-0-143-200 kubenswrapper[2572]: I0716 00:40:30.682717 2572 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-image-registry\"/\"registry-dockercfg-vczkg\"" Jul 16 00:40:30.682849 ip-10-0-143-200 kubenswrapper[2572]: I0716 00:40:30.682750 2572 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-image-registry\"/\"installation-pull-secrets\"" Jul 16 00:40:30.682849 ip-10-0-143-200 kubenswrapper[2572]: I0716 00:40:30.682718 2572 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-image-registry\"/\"image-registry-private-configuration\"" Jul 16 00:40:30.684212 ip-10-0-143-200 kubenswrapper[2572]: I0716 00:40:30.684109 2572 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-image-registry\"/\"image-registry-tls\"" Jul 16 00:40:30.691702 ip-10-0-143-200 kubenswrapper[2572]: I0716 00:40:30.691660 2572 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-image-registry\"/\"trusted-ca\"" Jul 16 00:40:30.698198 ip-10-0-143-200 kubenswrapper[2572]: I0716 00:40:30.698128 2572 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-kube-storage-version-migrator-operator/kube-storage-version-migrator-operator-799cb7f958-tksqh"] Jul 16 00:40:30.700147 ip-10-0-143-200 kubenswrapper[2572]: I0716 00:40:30.700106 2572 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-kube-storage-version-migrator-operator/kube-storage-version-migrator-operator-799cb7f958-tksqh" Jul 16 00:40:30.704311 ip-10-0-143-200 kubenswrapper[2572]: I0716 00:40:30.704286 2572 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-kube-storage-version-migrator-operator\"/\"serving-cert\"" Jul 16 00:40:30.704983 ip-10-0-143-200 kubenswrapper[2572]: I0716 00:40:30.704965 2572 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-kube-storage-version-migrator-operator\"/\"config\"" Jul 16 00:40:30.705100 ip-10-0-143-200 kubenswrapper[2572]: I0716 00:40:30.705046 2572 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-kube-storage-version-migrator-operator\"/\"kube-storage-version-migrator-operator-dockercfg-b4rdl\"" Jul 16 00:40:30.707915 ip-10-0-143-200 kubenswrapper[2572]: I0716 00:40:30.707889 2572 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-cluster-storage-operator/volume-data-source-validator-78495597f6-wffwm"] Jul 16 00:40:30.708492 ip-10-0-143-200 kubenswrapper[2572]: I0716 00:40:30.708471 2572 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-kube-storage-version-migrator-operator\"/\"openshift-service-ca.crt\"" Jul 16 00:40:30.708782 ip-10-0-143-200 kubenswrapper[2572]: I0716 00:40:30.708764 2572 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-kube-storage-version-migrator-operator\"/\"kube-root-ca.crt\"" Jul 16 00:40:30.710324 ip-10-0-143-200 kubenswrapper[2572]: I0716 00:40:30.710304 2572 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-insights/insights-operator-68d9dd94cf-d4nrt"] Jul 16 00:40:30.710436 ip-10-0-143-200 kubenswrapper[2572]: I0716 00:40:30.710416 2572 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-cluster-storage-operator/volume-data-source-validator-78495597f6-wffwm" Jul 16 00:40:30.712408 ip-10-0-143-200 kubenswrapper[2572]: I0716 00:40:30.712387 2572 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-cluster-samples-operator/cluster-samples-operator-65db858894-zf8lz"] Jul 16 00:40:30.712538 ip-10-0-143-200 kubenswrapper[2572]: I0716 00:40:30.712521 2572 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-insights/insights-operator-68d9dd94cf-d4nrt" Jul 16 00:40:30.714292 ip-10-0-143-200 kubenswrapper[2572]: I0716 00:40:30.714250 2572 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-cluster-storage-operator\"/\"volume-data-source-validator-dockercfg-5nkgz\"" Jul 16 00:40:30.714292 ip-10-0-143-200 kubenswrapper[2572]: I0716 00:40:30.714262 2572 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-cluster-storage-operator\"/\"openshift-service-ca.crt\"" Jul 16 00:40:30.714454 ip-10-0-143-200 kubenswrapper[2572]: I0716 00:40:30.714263 2572 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-cluster-storage-operator\"/\"kube-root-ca.crt\"" Jul 16 00:40:30.714688 ip-10-0-143-200 kubenswrapper[2572]: I0716 00:40:30.714671 2572 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-network-diagnostics/network-check-source-869dcfcdcd-r6wkz"] Jul 16 00:40:30.714795 ip-10-0-143-200 kubenswrapper[2572]: I0716 00:40:30.714779 2572 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-cluster-samples-operator/cluster-samples-operator-65db858894-zf8lz" Jul 16 00:40:30.716683 ip-10-0-143-200 kubenswrapper[2572]: I0716 00:40:30.716663 2572 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-ingress/router-default-78f9d48cbb-xwpwf"] Jul 16 00:40:30.716827 ip-10-0-143-200 kubenswrapper[2572]: I0716 00:40:30.716803 2572 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-source-869dcfcdcd-r6wkz" Jul 16 00:40:30.718117 ip-10-0-143-200 kubenswrapper[2572]: I0716 00:40:30.718022 2572 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-insights\"/\"openshift-insights-serving-cert\"" Jul 16 00:40:30.718329 ip-10-0-143-200 kubenswrapper[2572]: I0716 00:40:30.718289 2572 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-insights\"/\"kube-root-ca.crt\"" Jul 16 00:40:30.718910 ip-10-0-143-200 kubenswrapper[2572]: I0716 00:40:30.718723 2572 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-insights\"/\"service-ca-bundle\"" Jul 16 00:40:30.718910 ip-10-0-143-200 kubenswrapper[2572]: I0716 00:40:30.718854 2572 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-insights\"/\"openshift-service-ca.crt\"" Jul 16 00:40:30.719080 ip-10-0-143-200 kubenswrapper[2572]: I0716 00:40:30.718918 2572 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/router-default-78f9d48cbb-xwpwf" Jul 16 00:40:30.719080 ip-10-0-143-200 kubenswrapper[2572]: I0716 00:40:30.718855 2572 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-service-ca-operator/service-ca-operator-7695fc6c8c-7v8cr"] Jul 16 00:40:30.719513 ip-10-0-143-200 kubenswrapper[2572]: I0716 00:40:30.719492 2572 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-cluster-samples-operator\"/\"samples-operator-tls\"" Jul 16 00:40:30.719700 ip-10-0-143-200 kubenswrapper[2572]: I0716 00:40:30.719506 2572 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-cluster-samples-operator\"/\"openshift-service-ca.crt\"" Jul 16 00:40:30.720145 ip-10-0-143-200 kubenswrapper[2572]: I0716 00:40:30.720122 2572 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-cluster-samples-operator\"/\"kube-root-ca.crt\"" Jul 16 00:40:30.721333 ip-10-0-143-200 kubenswrapper[2572]: I0716 00:40:30.720320 2572 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-network-diagnostics\"/\"network-diagnostics-dockercfg-6lcrw\"" Jul 16 00:40:30.721333 ip-10-0-143-200 kubenswrapper[2572]: I0716 00:40:30.720508 2572 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-insights\"/\"operator-dockercfg-pljtd\"" Jul 16 00:40:30.721333 ip-10-0-143-200 kubenswrapper[2572]: I0716 00:40:30.720799 2572 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-console-operator/console-operator-7866497494-pjc6j"] Jul 16 00:40:30.721333 ip-10-0-143-200 kubenswrapper[2572]: I0716 00:40:30.720877 2572 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-network-diagnostics\"/\"kube-root-ca.crt\"" Jul 16 00:40:30.721333 ip-10-0-143-200 kubenswrapper[2572]: I0716 00:40:30.720880 2572 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-network-diagnostics\"/\"openshift-service-ca.crt\"" Jul 16 00:40:30.721333 ip-10-0-143-200 kubenswrapper[2572]: I0716 00:40:30.720880 2572 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-cluster-samples-operator\"/\"cluster-samples-operator-dockercfg-mqqk9\"" Jul 16 00:40:30.721333 ip-10-0-143-200 kubenswrapper[2572]: I0716 00:40:30.721009 2572 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-service-ca-operator/service-ca-operator-7695fc6c8c-7v8cr" Jul 16 00:40:30.723132 ip-10-0-143-200 kubenswrapper[2572]: I0716 00:40:30.723110 2572 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-image-registry/image-registry-dd7c989d9-5w25c"] Jul 16 00:40:30.723241 ip-10-0-143-200 kubenswrapper[2572]: I0716 00:40:30.723148 2572 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-monitoring/cluster-monitoring-operator-5f7f494dd4-dxc26"] Jul 16 00:40:30.723469 ip-10-0-143-200 kubenswrapper[2572]: I0716 00:40:30.723453 2572 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-console-operator/console-operator-7866497494-pjc6j" Jul 16 00:40:30.724598 ip-10-0-143-200 kubenswrapper[2572]: I0716 00:40:30.724579 2572 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ingress\"/\"openshift-service-ca.crt\"" Jul 16 00:40:30.724958 ip-10-0-143-200 kubenswrapper[2572]: I0716 00:40:30.724730 2572 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ingress\"/\"service-ca-bundle\"" Jul 16 00:40:30.724958 ip-10-0-143-200 kubenswrapper[2572]: I0716 00:40:30.724739 2572 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ingress\"/\"kube-root-ca.crt\"" Jul 16 00:40:30.724958 ip-10-0-143-200 kubenswrapper[2572]: I0716 00:40:30.724730 2572 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-service-ca-operator\"/\"serving-cert\"" Jul 16 00:40:30.724958 ip-10-0-143-200 kubenswrapper[2572]: I0716 00:40:30.724805 2572 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-service-ca-operator\"/\"service-ca-operator-dockercfg-lwtmh\"" Jul 16 00:40:30.724958 ip-10-0-143-200 kubenswrapper[2572]: I0716 00:40:30.724858 2572 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress\"/\"default-ingress-cert\"" Jul 16 00:40:30.724958 ip-10-0-143-200 kubenswrapper[2572]: I0716 00:40:30.724872 2572 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress\"/\"router-stats-default\"" Jul 16 00:40:30.724958 ip-10-0-143-200 kubenswrapper[2572]: I0716 00:40:30.724908 2572 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-service-ca-operator\"/\"kube-root-ca.crt\"" Jul 16 00:40:30.724958 ip-10-0-143-200 kubenswrapper[2572]: I0716 00:40:30.724858 2572 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-service-ca-operator\"/\"service-ca-operator-config\"" Jul 16 00:40:30.725353 ip-10-0-143-200 kubenswrapper[2572]: I0716 00:40:30.725054 2572 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-service-ca-operator\"/\"openshift-service-ca.crt\"" Jul 16 00:40:30.725353 ip-10-0-143-200 kubenswrapper[2572]: I0716 00:40:30.725278 2572 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress\"/\"router-metrics-certs-default\"" Jul 16 00:40:30.725353 ip-10-0-143-200 kubenswrapper[2572]: I0716 00:40:30.725296 2572 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress\"/\"router-dockercfg-f6zk6\"" Jul 16 00:40:30.725493 ip-10-0-143-200 kubenswrapper[2572]: I0716 00:40:30.725453 2572 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-dns/dns-default-w6cgc"] Jul 16 00:40:30.725594 ip-10-0-143-200 kubenswrapper[2572]: I0716 00:40:30.725576 2572 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-monitoring/cluster-monitoring-operator-5f7f494dd4-dxc26" Jul 16 00:40:30.726623 ip-10-0-143-200 kubenswrapper[2572]: I0716 00:40:30.726401 2572 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-console-operator\"/\"serving-cert\"" Jul 16 00:40:30.726623 ip-10-0-143-200 kubenswrapper[2572]: I0716 00:40:30.726415 2572 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-console-operator\"/\"console-operator-config\"" Jul 16 00:40:30.726623 ip-10-0-143-200 kubenswrapper[2572]: I0716 00:40:30.726422 2572 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-console-operator\"/\"openshift-service-ca.crt\"" Jul 16 00:40:30.726833 ip-10-0-143-200 kubenswrapper[2572]: I0716 00:40:30.726695 2572 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-console-operator\"/\"console-operator-dockercfg-jkzmx\"" Jul 16 00:40:30.727215 ip-10-0-143-200 kubenswrapper[2572]: I0716 00:40:30.727195 2572 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-insights\"/\"trusted-ca-bundle\"" Jul 16 00:40:30.727875 ip-10-0-143-200 kubenswrapper[2572]: I0716 00:40:30.727372 2572 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-console-operator\"/\"kube-root-ca.crt\"" Jul 16 00:40:30.727875 ip-10-0-143-200 kubenswrapper[2572]: I0716 00:40:30.727573 2572 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-ingress-canary/ingress-canary-hhtd6"] Jul 16 00:40:30.728016 ip-10-0-143-200 kubenswrapper[2572]: I0716 00:40:30.727973 2572 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-dns/dns-default-w6cgc" Jul 16 00:40:30.731858 ip-10-0-143-200 kubenswrapper[2572]: I0716 00:40:30.730086 2572 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-monitoring\"/\"openshift-service-ca.crt\"" Jul 16 00:40:30.731858 ip-10-0-143-200 kubenswrapper[2572]: I0716 00:40:30.731213 2572 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"cluster-monitoring-operator-dockercfg-dh2z4\"" Jul 16 00:40:30.732663 ip-10-0-143-200 kubenswrapper[2572]: I0716 00:40:30.732602 2572 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-kube-storage-version-migrator-operator/kube-storage-version-migrator-operator-799cb7f958-tksqh"] Jul 16 00:40:30.732789 ip-10-0-143-200 kubenswrapper[2572]: I0716 00:40:30.732773 2572 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress-canary/ingress-canary-hhtd6" Jul 16 00:40:30.734992 ip-10-0-143-200 kubenswrapper[2572]: I0716 00:40:30.734966 2572 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-monitoring\"/\"kube-root-ca.crt\"" Jul 16 00:40:30.735318 ip-10-0-143-200 kubenswrapper[2572]: I0716 00:40:30.735300 2572 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-dns\"/\"dns-default-metrics-tls\"" Jul 16 00:40:30.735477 ip-10-0-143-200 kubenswrapper[2572]: I0716 00:40:30.735457 2572 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-dns\"/\"dns-dockercfg-ll89j\"" Jul 16 00:40:30.735691 ip-10-0-143-200 kubenswrapper[2572]: I0716 00:40:30.735669 2572 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-cluster-storage-operator/volume-data-source-validator-78495597f6-wffwm"] Jul 16 00:40:30.735795 ip-10-0-143-200 kubenswrapper[2572]: I0716 00:40:30.735779 2572 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-dns\"/\"dns-default\"" Jul 16 00:40:30.735878 ip-10-0-143-200 kubenswrapper[2572]: I0716 00:40:30.735804 2572 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"cluster-monitoring-operator-tls\"" Jul 16 00:40:30.736649 ip-10-0-143-200 kubenswrapper[2572]: I0716 00:40:30.736532 2572 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-monitoring\"/\"telemetry-config\"" Jul 16 00:40:30.738002 ip-10-0-143-200 kubenswrapper[2572]: I0716 00:40:30.737984 2572 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-insights/insights-operator-68d9dd94cf-d4nrt"] Jul 16 00:40:30.738922 ip-10-0-143-200 kubenswrapper[2572]: I0716 00:40:30.738902 2572 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-network-diagnostics/network-check-source-869dcfcdcd-r6wkz"] Jul 16 00:40:30.740581 ip-10-0-143-200 kubenswrapper[2572]: I0716 00:40:30.740559 2572 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress-canary\"/\"canary-serving-cert\"" Jul 16 00:40:30.740696 ip-10-0-143-200 kubenswrapper[2572]: I0716 00:40:30.740614 2572 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ingress-canary\"/\"kube-root-ca.crt\"" Jul 16 00:40:30.740759 ip-10-0-143-200 kubenswrapper[2572]: I0716 00:40:30.740565 2572 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress-canary\"/\"default-dockercfg-tf6x9\"" Jul 16 00:40:30.740964 ip-10-0-143-200 kubenswrapper[2572]: I0716 00:40:30.740935 2572 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ingress-canary\"/\"openshift-service-ca.crt\"" Jul 16 00:40:30.741040 ip-10-0-143-200 kubenswrapper[2572]: I0716 00:40:30.741029 2572 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-console-operator\"/\"trusted-ca\"" Jul 16 00:40:30.752530 ip-10-0-143-200 kubenswrapper[2572]: I0716 00:40:30.752504 2572 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-cluster-samples-operator/cluster-samples-operator-65db858894-zf8lz"] Jul 16 00:40:30.760934 ip-10-0-143-200 kubenswrapper[2572]: I0716 00:40:30.760908 2572 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-monitoring/cluster-monitoring-operator-5f7f494dd4-dxc26"] Jul 16 00:40:30.761809 ip-10-0-143-200 kubenswrapper[2572]: I0716 00:40:30.761784 2572 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-service-ca-operator/service-ca-operator-7695fc6c8c-7v8cr"] Jul 16 00:40:30.765744 ip-10-0-143-200 kubenswrapper[2572]: I0716 00:40:30.765719 2572 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-dns/dns-default-w6cgc"] Jul 16 00:40:30.774096 ip-10-0-143-200 kubenswrapper[2572]: I0716 00:40:30.774075 2572 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-console-operator/console-operator-7866497494-pjc6j"] Jul 16 00:40:30.784758 ip-10-0-143-200 kubenswrapper[2572]: I0716 00:40:30.784733 2572 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/router-default-78f9d48cbb-xwpwf"] Jul 16 00:40:30.795081 ip-10-0-143-200 kubenswrapper[2572]: I0716 00:40:30.795056 2572 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"bound-sa-token\" (UniqueName: \"kubernetes.io/projected/f103d7a8-2560-409b-801e-701bcc6835c1-bound-sa-token\") pod \"image-registry-dd7c989d9-5w25c\" (UID: \"f103d7a8-2560-409b-801e-701bcc6835c1\") " pod="openshift-image-registry/image-registry-dd7c989d9-5w25c" Jul 16 00:40:30.795195 ip-10-0-143-200 kubenswrapper[2572]: I0716 00:40:30.795087 2572 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"config\" (UniqueName: \"kubernetes.io/configmap/07e0ab9b-1bfe-4335-bac1-8d9d158b0085-config\") pod \"kube-storage-version-migrator-operator-799cb7f958-tksqh\" (UID: \"07e0ab9b-1bfe-4335-bac1-8d9d158b0085\") " pod="openshift-kube-storage-version-migrator-operator/kube-storage-version-migrator-operator-799cb7f958-tksqh" Jul 16 00:40:30.795195 ip-10-0-143-200 kubenswrapper[2572]: I0716 00:40:30.795137 2572 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-kmrt8\" (UniqueName: \"kubernetes.io/projected/f103d7a8-2560-409b-801e-701bcc6835c1-kube-api-access-kmrt8\") pod \"image-registry-dd7c989d9-5w25c\" (UID: \"f103d7a8-2560-409b-801e-701bcc6835c1\") " pod="openshift-image-registry/image-registry-dd7c989d9-5w25c" Jul 16 00:40:30.795195 ip-10-0-143-200 kubenswrapper[2572]: I0716 00:40:30.795157 2572 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"serving-cert\" (UniqueName: \"kubernetes.io/secret/0563669f-9ac5-43d0-a3c0-47c36275ee46-serving-cert\") pod \"insights-operator-68d9dd94cf-d4nrt\" (UID: \"0563669f-9ac5-43d0-a3c0-47c36275ee46\") " pod="openshift-insights/insights-operator-68d9dd94cf-d4nrt" Jul 16 00:40:30.795195 ip-10-0-143-200 kubenswrapper[2572]: I0716 00:40:30.795178 2572 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"image-registry-private-configuration\" (UniqueName: \"kubernetes.io/secret/f103d7a8-2560-409b-801e-701bcc6835c1-image-registry-private-configuration\") pod \"image-registry-dd7c989d9-5w25c\" (UID: \"f103d7a8-2560-409b-801e-701bcc6835c1\") " pod="openshift-image-registry/image-registry-dd7c989d9-5w25c" Jul 16 00:40:30.795383 ip-10-0-143-200 kubenswrapper[2572]: I0716 00:40:30.795202 2572 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"registry-tls\" (UniqueName: \"kubernetes.io/projected/f103d7a8-2560-409b-801e-701bcc6835c1-registry-tls\") pod \"image-registry-dd7c989d9-5w25c\" (UID: \"f103d7a8-2560-409b-801e-701bcc6835c1\") " pod="openshift-image-registry/image-registry-dd7c989d9-5w25c" Jul 16 00:40:30.795383 ip-10-0-143-200 kubenswrapper[2572]: I0716 00:40:30.795229 2572 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"ca-trust-extracted\" (UniqueName: \"kubernetes.io/empty-dir/f103d7a8-2560-409b-801e-701bcc6835c1-ca-trust-extracted\") pod \"image-registry-dd7c989d9-5w25c\" (UID: \"f103d7a8-2560-409b-801e-701bcc6835c1\") " pod="openshift-image-registry/image-registry-dd7c989d9-5w25c" Jul 16 00:40:30.795383 ip-10-0-143-200 kubenswrapper[2572]: I0716 00:40:30.795256 2572 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-snmxw\" (UniqueName: \"kubernetes.io/projected/0563669f-9ac5-43d0-a3c0-47c36275ee46-kube-api-access-snmxw\") pod \"insights-operator-68d9dd94cf-d4nrt\" (UID: \"0563669f-9ac5-43d0-a3c0-47c36275ee46\") " pod="openshift-insights/insights-operator-68d9dd94cf-d4nrt" Jul 16 00:40:30.795383 ip-10-0-143-200 kubenswrapper[2572]: I0716 00:40:30.795298 2572 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"trusted-ca\" (UniqueName: \"kubernetes.io/configmap/f103d7a8-2560-409b-801e-701bcc6835c1-trusted-ca\") pod \"image-registry-dd7c989d9-5w25c\" (UID: \"f103d7a8-2560-409b-801e-701bcc6835c1\") " pod="openshift-image-registry/image-registry-dd7c989d9-5w25c" Jul 16 00:40:30.795383 ip-10-0-143-200 kubenswrapper[2572]: I0716 00:40:30.795328 2572 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"registry-certificates\" (UniqueName: \"kubernetes.io/configmap/f103d7a8-2560-409b-801e-701bcc6835c1-registry-certificates\") pod \"image-registry-dd7c989d9-5w25c\" (UID: \"f103d7a8-2560-409b-801e-701bcc6835c1\") " pod="openshift-image-registry/image-registry-dd7c989d9-5w25c" Jul 16 00:40:30.795383 ip-10-0-143-200 kubenswrapper[2572]: I0716 00:40:30.795343 2572 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"serving-cert\" (UniqueName: \"kubernetes.io/secret/07e0ab9b-1bfe-4335-bac1-8d9d158b0085-serving-cert\") pod \"kube-storage-version-migrator-operator-799cb7f958-tksqh\" (UID: \"07e0ab9b-1bfe-4335-bac1-8d9d158b0085\") " pod="openshift-kube-storage-version-migrator-operator/kube-storage-version-migrator-operator-799cb7f958-tksqh" Jul 16 00:40:30.795686 ip-10-0-143-200 kubenswrapper[2572]: I0716 00:40:30.795394 2572 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"snapshots\" (UniqueName: \"kubernetes.io/empty-dir/0563669f-9ac5-43d0-a3c0-47c36275ee46-snapshots\") pod \"insights-operator-68d9dd94cf-d4nrt\" (UID: \"0563669f-9ac5-43d0-a3c0-47c36275ee46\") " pod="openshift-insights/insights-operator-68d9dd94cf-d4nrt" Jul 16 00:40:30.795686 ip-10-0-143-200 kubenswrapper[2572]: I0716 00:40:30.795446 2572 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"installation-pull-secrets\" (UniqueName: \"kubernetes.io/secret/f103d7a8-2560-409b-801e-701bcc6835c1-installation-pull-secrets\") pod \"image-registry-dd7c989d9-5w25c\" (UID: \"f103d7a8-2560-409b-801e-701bcc6835c1\") " pod="openshift-image-registry/image-registry-dd7c989d9-5w25c" Jul 16 00:40:30.795686 ip-10-0-143-200 kubenswrapper[2572]: I0716 00:40:30.795472 2572 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tmp\" (UniqueName: \"kubernetes.io/empty-dir/0563669f-9ac5-43d0-a3c0-47c36275ee46-tmp\") pod \"insights-operator-68d9dd94cf-d4nrt\" (UID: \"0563669f-9ac5-43d0-a3c0-47c36275ee46\") " pod="openshift-insights/insights-operator-68d9dd94cf-d4nrt" Jul 16 00:40:30.795686 ip-10-0-143-200 kubenswrapper[2572]: I0716 00:40:30.795534 2572 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-df7kv\" (UniqueName: \"kubernetes.io/projected/3cb174c0-4582-44f3-879a-23697944bc47-kube-api-access-df7kv\") pod \"volume-data-source-validator-78495597f6-wffwm\" (UID: \"3cb174c0-4582-44f3-879a-23697944bc47\") " pod="openshift-cluster-storage-operator/volume-data-source-validator-78495597f6-wffwm" Jul 16 00:40:30.795686 ip-10-0-143-200 kubenswrapper[2572]: I0716 00:40:30.795581 2572 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-zbnnk\" (UniqueName: \"kubernetes.io/projected/07e0ab9b-1bfe-4335-bac1-8d9d158b0085-kube-api-access-zbnnk\") pod \"kube-storage-version-migrator-operator-799cb7f958-tksqh\" (UID: \"07e0ab9b-1bfe-4335-bac1-8d9d158b0085\") " pod="openshift-kube-storage-version-migrator-operator/kube-storage-version-migrator-operator-799cb7f958-tksqh" Jul 16 00:40:30.795686 ip-10-0-143-200 kubenswrapper[2572]: I0716 00:40:30.795643 2572 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/0563669f-9ac5-43d0-a3c0-47c36275ee46-trusted-ca-bundle\") pod \"insights-operator-68d9dd94cf-d4nrt\" (UID: \"0563669f-9ac5-43d0-a3c0-47c36275ee46\") " pod="openshift-insights/insights-operator-68d9dd94cf-d4nrt" Jul 16 00:40:30.795686 ip-10-0-143-200 kubenswrapper[2572]: I0716 00:40:30.795681 2572 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"service-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/0563669f-9ac5-43d0-a3c0-47c36275ee46-service-ca-bundle\") pod \"insights-operator-68d9dd94cf-d4nrt\" (UID: \"0563669f-9ac5-43d0-a3c0-47c36275ee46\") " pod="openshift-insights/insights-operator-68d9dd94cf-d4nrt" Jul 16 00:40:30.800447 ip-10-0-143-200 kubenswrapper[2572]: I0716 00:40:30.800414 2572 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress-canary/ingress-canary-hhtd6"] Jul 16 00:40:30.895967 ip-10-0-143-200 kubenswrapper[2572]: I0716 00:40:30.895938 2572 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-df7kv\" (UniqueName: \"kubernetes.io/projected/3cb174c0-4582-44f3-879a-23697944bc47-kube-api-access-df7kv\") pod \"volume-data-source-validator-78495597f6-wffwm\" (UID: \"3cb174c0-4582-44f3-879a-23697944bc47\") " pod="openshift-cluster-storage-operator/volume-data-source-validator-78495597f6-wffwm" Jul 16 00:40:30.896157 ip-10-0-143-200 kubenswrapper[2572]: I0716 00:40:30.895981 2572 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-nt4m9\" (UniqueName: \"kubernetes.io/projected/6f7eb059-755b-47de-82ab-bfaa72b8f899-kube-api-access-nt4m9\") pod \"ingress-canary-hhtd6\" (UID: \"6f7eb059-755b-47de-82ab-bfaa72b8f899\") " pod="openshift-ingress-canary/ingress-canary-hhtd6" Jul 16 00:40:30.896157 ip-10-0-143-200 kubenswrapper[2572]: I0716 00:40:30.896012 2572 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/0563669f-9ac5-43d0-a3c0-47c36275ee46-trusted-ca-bundle\") pod \"insights-operator-68d9dd94cf-d4nrt\" (UID: \"0563669f-9ac5-43d0-a3c0-47c36275ee46\") " pod="openshift-insights/insights-operator-68d9dd94cf-d4nrt" Jul 16 00:40:30.896157 ip-10-0-143-200 kubenswrapper[2572]: I0716 00:40:30.896035 2572 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-t9sc4\" (UniqueName: \"kubernetes.io/projected/5aabb5b2-54bc-4893-91d2-3c0215fabd67-kube-api-access-t9sc4\") pod \"cluster-samples-operator-65db858894-zf8lz\" (UID: \"5aabb5b2-54bc-4893-91d2-3c0215fabd67\") " pod="openshift-cluster-samples-operator/cluster-samples-operator-65db858894-zf8lz" Jul 16 00:40:30.896157 ip-10-0-143-200 kubenswrapper[2572]: I0716 00:40:30.896059 2572 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"service-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/0563669f-9ac5-43d0-a3c0-47c36275ee46-service-ca-bundle\") pod \"insights-operator-68d9dd94cf-d4nrt\" (UID: \"0563669f-9ac5-43d0-a3c0-47c36275ee46\") " pod="openshift-insights/insights-operator-68d9dd94cf-d4nrt" Jul 16 00:40:30.896157 ip-10-0-143-200 kubenswrapper[2572]: I0716 00:40:30.896108 2572 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"bound-sa-token\" (UniqueName: \"kubernetes.io/projected/f103d7a8-2560-409b-801e-701bcc6835c1-bound-sa-token\") pod \"image-registry-dd7c989d9-5w25c\" (UID: \"f103d7a8-2560-409b-801e-701bcc6835c1\") " pod="openshift-image-registry/image-registry-dd7c989d9-5w25c" Jul 16 00:40:30.896417 ip-10-0-143-200 kubenswrapper[2572]: I0716 00:40:30.896177 2572 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"stats-auth\" (UniqueName: \"kubernetes.io/secret/b82b08fd-bf48-4ee3-9d58-b648d6cba972-stats-auth\") pod \"router-default-78f9d48cbb-xwpwf\" (UID: \"b82b08fd-bf48-4ee3-9d58-b648d6cba972\") " pod="openshift-ingress/router-default-78f9d48cbb-xwpwf" Jul 16 00:40:30.896417 ip-10-0-143-200 kubenswrapper[2572]: I0716 00:40:30.896209 2572 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"config\" (UniqueName: \"kubernetes.io/configmap/07e0ab9b-1bfe-4335-bac1-8d9d158b0085-config\") pod \"kube-storage-version-migrator-operator-799cb7f958-tksqh\" (UID: \"07e0ab9b-1bfe-4335-bac1-8d9d158b0085\") " pod="openshift-kube-storage-version-migrator-operator/kube-storage-version-migrator-operator-799cb7f958-tksqh" Jul 16 00:40:30.896417 ip-10-0-143-200 kubenswrapper[2572]: I0716 00:40:30.896233 2572 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"cert\" (UniqueName: \"kubernetes.io/secret/6f7eb059-755b-47de-82ab-bfaa72b8f899-cert\") pod \"ingress-canary-hhtd6\" (UID: \"6f7eb059-755b-47de-82ab-bfaa72b8f899\") " pod="openshift-ingress-canary/ingress-canary-hhtd6" Jul 16 00:40:30.896417 ip-10-0-143-200 kubenswrapper[2572]: I0716 00:40:30.896280 2572 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-snmxw\" (UniqueName: \"kubernetes.io/projected/0563669f-9ac5-43d0-a3c0-47c36275ee46-kube-api-access-snmxw\") pod \"insights-operator-68d9dd94cf-d4nrt\" (UID: \"0563669f-9ac5-43d0-a3c0-47c36275ee46\") " pod="openshift-insights/insights-operator-68d9dd94cf-d4nrt" Jul 16 00:40:30.896417 ip-10-0-143-200 kubenswrapper[2572]: I0716 00:40:30.896310 2572 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"trusted-ca\" (UniqueName: \"kubernetes.io/configmap/a64bcd05-8596-4c12-b88e-e974d676e25f-trusted-ca\") pod \"console-operator-7866497494-pjc6j\" (UID: \"a64bcd05-8596-4c12-b88e-e974d676e25f\") " pod="openshift-console-operator/console-operator-7866497494-pjc6j" Jul 16 00:40:30.896417 ip-10-0-143-200 kubenswrapper[2572]: I0716 00:40:30.896339 2572 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"samples-operator-tls\" (UniqueName: \"kubernetes.io/secret/5aabb5b2-54bc-4893-91d2-3c0215fabd67-samples-operator-tls\") pod \"cluster-samples-operator-65db858894-zf8lz\" (UID: \"5aabb5b2-54bc-4893-91d2-3c0215fabd67\") " pod="openshift-cluster-samples-operator/cluster-samples-operator-65db858894-zf8lz" Jul 16 00:40:30.896417 ip-10-0-143-200 kubenswrapper[2572]: I0716 00:40:30.896364 2572 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"telemetry-config\" (UniqueName: \"kubernetes.io/configmap/265fe067-cade-414b-be13-cb6dbab18057-telemetry-config\") pod \"cluster-monitoring-operator-5f7f494dd4-dxc26\" (UID: \"265fe067-cade-414b-be13-cb6dbab18057\") " pod="openshift-monitoring/cluster-monitoring-operator-5f7f494dd4-dxc26" Jul 16 00:40:30.896417 ip-10-0-143-200 kubenswrapper[2572]: I0716 00:40:30.896388 2572 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-gxtdx\" (UniqueName: \"kubernetes.io/projected/265fe067-cade-414b-be13-cb6dbab18057-kube-api-access-gxtdx\") pod \"cluster-monitoring-operator-5f7f494dd4-dxc26\" (UID: \"265fe067-cade-414b-be13-cb6dbab18057\") " pod="openshift-monitoring/cluster-monitoring-operator-5f7f494dd4-dxc26" Jul 16 00:40:30.896831 ip-10-0-143-200 kubenswrapper[2572]: I0716 00:40:30.896421 2572 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-zsfwq\" (UniqueName: \"kubernetes.io/projected/645fc509-972b-4332-8562-2309cc0eab1b-kube-api-access-zsfwq\") pod \"dns-default-w6cgc\" (UID: \"645fc509-972b-4332-8562-2309cc0eab1b\") " pod="openshift-dns/dns-default-w6cgc" Jul 16 00:40:30.896831 ip-10-0-143-200 kubenswrapper[2572]: I0716 00:40:30.896445 2572 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"config\" (UniqueName: \"kubernetes.io/configmap/1c3985e6-ff56-469e-bdd6-809bf98870ab-config\") pod \"service-ca-operator-7695fc6c8c-7v8cr\" (UID: \"1c3985e6-ff56-469e-bdd6-809bf98870ab\") " pod="openshift-service-ca-operator/service-ca-operator-7695fc6c8c-7v8cr" Jul 16 00:40:30.896831 ip-10-0-143-200 kubenswrapper[2572]: I0716 00:40:30.896494 2572 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"registry-tls\" (UniqueName: \"kubernetes.io/projected/f103d7a8-2560-409b-801e-701bcc6835c1-registry-tls\") pod \"image-registry-dd7c989d9-5w25c\" (UID: \"f103d7a8-2560-409b-801e-701bcc6835c1\") " pod="openshift-image-registry/image-registry-dd7c989d9-5w25c" Jul 16 00:40:30.896831 ip-10-0-143-200 kubenswrapper[2572]: I0716 00:40:30.896553 2572 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"ca-trust-extracted\" (UniqueName: \"kubernetes.io/empty-dir/f103d7a8-2560-409b-801e-701bcc6835c1-ca-trust-extracted\") pod \"image-registry-dd7c989d9-5w25c\" (UID: \"f103d7a8-2560-409b-801e-701bcc6835c1\") " pod="openshift-image-registry/image-registry-dd7c989d9-5w25c" Jul 16 00:40:30.896831 ip-10-0-143-200 kubenswrapper[2572]: I0716 00:40:30.896587 2572 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"default-certificate\" (UniqueName: \"kubernetes.io/secret/b82b08fd-bf48-4ee3-9d58-b648d6cba972-default-certificate\") pod \"router-default-78f9d48cbb-xwpwf\" (UID: \"b82b08fd-bf48-4ee3-9d58-b648d6cba972\") " pod="openshift-ingress/router-default-78f9d48cbb-xwpwf" Jul 16 00:40:30.896831 ip-10-0-143-200 kubenswrapper[2572]: I0716 00:40:30.896617 2572 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-mg2mn\" (UniqueName: \"kubernetes.io/projected/1c3985e6-ff56-469e-bdd6-809bf98870ab-kube-api-access-mg2mn\") pod \"service-ca-operator-7695fc6c8c-7v8cr\" (UID: \"1c3985e6-ff56-469e-bdd6-809bf98870ab\") " pod="openshift-service-ca-operator/service-ca-operator-7695fc6c8c-7v8cr" Jul 16 00:40:30.896831 ip-10-0-143-200 kubenswrapper[2572]: I0716 00:40:30.896708 2572 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"service-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/0563669f-9ac5-43d0-a3c0-47c36275ee46-service-ca-bundle\") pod \"insights-operator-68d9dd94cf-d4nrt\" (UID: \"0563669f-9ac5-43d0-a3c0-47c36275ee46\") " pod="openshift-insights/insights-operator-68d9dd94cf-d4nrt" Jul 16 00:40:30.896831 ip-10-0-143-200 kubenswrapper[2572]: E0716 00:40:30.896592 2572 projected.go:266] Couldn't get secret openshift-image-registry/image-registry-tls: secret "image-registry-tls" not found Jul 16 00:40:30.896831 ip-10-0-143-200 kubenswrapper[2572]: I0716 00:40:30.896743 2572 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"trusted-ca\" (UniqueName: \"kubernetes.io/configmap/f103d7a8-2560-409b-801e-701bcc6835c1-trusted-ca\") pod \"image-registry-dd7c989d9-5w25c\" (UID: \"f103d7a8-2560-409b-801e-701bcc6835c1\") " pod="openshift-image-registry/image-registry-dd7c989d9-5w25c" Jul 16 00:40:30.896831 ip-10-0-143-200 kubenswrapper[2572]: I0716 00:40:30.896784 2572 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"snapshots\" (UniqueName: \"kubernetes.io/empty-dir/0563669f-9ac5-43d0-a3c0-47c36275ee46-snapshots\") pod \"insights-operator-68d9dd94cf-d4nrt\" (UID: \"0563669f-9ac5-43d0-a3c0-47c36275ee46\") " pod="openshift-insights/insights-operator-68d9dd94cf-d4nrt" Jul 16 00:40:30.896831 ip-10-0-143-200 kubenswrapper[2572]: E0716 00:40:30.896750 2572 projected.go:196] Error preparing data for projected volume registry-tls for pod openshift-image-registry/image-registry-dd7c989d9-5w25c: secret "image-registry-tls" not found Jul 16 00:40:30.896831 ip-10-0-143-200 kubenswrapper[2572]: I0716 00:40:30.896814 2572 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-t2kdz\" (UniqueName: \"kubernetes.io/projected/a64bcd05-8596-4c12-b88e-e974d676e25f-kube-api-access-t2kdz\") pod \"console-operator-7866497494-pjc6j\" (UID: \"a64bcd05-8596-4c12-b88e-e974d676e25f\") " pod="openshift-console-operator/console-operator-7866497494-pjc6j" Jul 16 00:40:30.896831 ip-10-0-143-200 kubenswrapper[2572]: I0716 00:40:30.896828 2572 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"config\" (UniqueName: \"kubernetes.io/configmap/07e0ab9b-1bfe-4335-bac1-8d9d158b0085-config\") pod \"kube-storage-version-migrator-operator-799cb7f958-tksqh\" (UID: \"07e0ab9b-1bfe-4335-bac1-8d9d158b0085\") " pod="openshift-kube-storage-version-migrator-operator/kube-storage-version-migrator-operator-799cb7f958-tksqh" Jul 16 00:40:30.897401 ip-10-0-143-200 kubenswrapper[2572]: I0716 00:40:30.896849 2572 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"installation-pull-secrets\" (UniqueName: \"kubernetes.io/secret/f103d7a8-2560-409b-801e-701bcc6835c1-installation-pull-secrets\") pod \"image-registry-dd7c989d9-5w25c\" (UID: \"f103d7a8-2560-409b-801e-701bcc6835c1\") " pod="openshift-image-registry/image-registry-dd7c989d9-5w25c" Jul 16 00:40:30.897401 ip-10-0-143-200 kubenswrapper[2572]: E0716 00:40:30.896886 2572 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/projected/f103d7a8-2560-409b-801e-701bcc6835c1-registry-tls podName:f103d7a8-2560-409b-801e-701bcc6835c1 nodeName:}" failed. No retries permitted until 2026-07-16 00:40:31.396861666 +0000 UTC m=+36.505616328 (durationBeforeRetry 500ms). Error: MountVolume.SetUp failed for volume "registry-tls" (UniqueName: "kubernetes.io/projected/f103d7a8-2560-409b-801e-701bcc6835c1-registry-tls") pod "image-registry-dd7c989d9-5w25c" (UID: "f103d7a8-2560-409b-801e-701bcc6835c1") : secret "image-registry-tls" not found Jul 16 00:40:30.897401 ip-10-0-143-200 kubenswrapper[2572]: I0716 00:40:30.896910 2572 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tmp\" (UniqueName: \"kubernetes.io/empty-dir/0563669f-9ac5-43d0-a3c0-47c36275ee46-tmp\") pod \"insights-operator-68d9dd94cf-d4nrt\" (UID: \"0563669f-9ac5-43d0-a3c0-47c36275ee46\") " pod="openshift-insights/insights-operator-68d9dd94cf-d4nrt" Jul 16 00:40:30.897401 ip-10-0-143-200 kubenswrapper[2572]: I0716 00:40:30.896973 2572 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"config-volume\" (UniqueName: \"kubernetes.io/configmap/645fc509-972b-4332-8562-2309cc0eab1b-config-volume\") pod \"dns-default-w6cgc\" (UID: \"645fc509-972b-4332-8562-2309cc0eab1b\") " pod="openshift-dns/dns-default-w6cgc" Jul 16 00:40:30.897401 ip-10-0-143-200 kubenswrapper[2572]: I0716 00:40:30.896996 2572 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"metrics-tls\" (UniqueName: \"kubernetes.io/secret/645fc509-972b-4332-8562-2309cc0eab1b-metrics-tls\") pod \"dns-default-w6cgc\" (UID: \"645fc509-972b-4332-8562-2309cc0eab1b\") " pod="openshift-dns/dns-default-w6cgc" Jul 16 00:40:30.897401 ip-10-0-143-200 kubenswrapper[2572]: I0716 00:40:30.897011 2572 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"ca-trust-extracted\" (UniqueName: \"kubernetes.io/empty-dir/f103d7a8-2560-409b-801e-701bcc6835c1-ca-trust-extracted\") pod \"image-registry-dd7c989d9-5w25c\" (UID: \"f103d7a8-2560-409b-801e-701bcc6835c1\") " pod="openshift-image-registry/image-registry-dd7c989d9-5w25c" Jul 16 00:40:30.897401 ip-10-0-143-200 kubenswrapper[2572]: I0716 00:40:30.897030 2572 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-zbnnk\" (UniqueName: \"kubernetes.io/projected/07e0ab9b-1bfe-4335-bac1-8d9d158b0085-kube-api-access-zbnnk\") pod \"kube-storage-version-migrator-operator-799cb7f958-tksqh\" (UID: \"07e0ab9b-1bfe-4335-bac1-8d9d158b0085\") " pod="openshift-kube-storage-version-migrator-operator/kube-storage-version-migrator-operator-799cb7f958-tksqh" Jul 16 00:40:30.897401 ip-10-0-143-200 kubenswrapper[2572]: I0716 00:40:30.897053 2572 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/b82b08fd-bf48-4ee3-9d58-b648d6cba972-metrics-certs\") pod \"router-default-78f9d48cbb-xwpwf\" (UID: \"b82b08fd-bf48-4ee3-9d58-b648d6cba972\") " pod="openshift-ingress/router-default-78f9d48cbb-xwpwf" Jul 16 00:40:30.897401 ip-10-0-143-200 kubenswrapper[2572]: I0716 00:40:30.897075 2572 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-4k4r4\" (UniqueName: \"kubernetes.io/projected/9da33896-31b6-4a05-bc7c-b9260876e581-kube-api-access-4k4r4\") pod \"network-check-source-869dcfcdcd-r6wkz\" (UID: \"9da33896-31b6-4a05-bc7c-b9260876e581\") " pod="openshift-network-diagnostics/network-check-source-869dcfcdcd-r6wkz" Jul 16 00:40:30.897401 ip-10-0-143-200 kubenswrapper[2572]: I0716 00:40:30.897101 2572 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"service-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/b82b08fd-bf48-4ee3-9d58-b648d6cba972-service-ca-bundle\") pod \"router-default-78f9d48cbb-xwpwf\" (UID: \"b82b08fd-bf48-4ee3-9d58-b648d6cba972\") " pod="openshift-ingress/router-default-78f9d48cbb-xwpwf" Jul 16 00:40:30.897401 ip-10-0-143-200 kubenswrapper[2572]: I0716 00:40:30.897147 2572 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"config\" (UniqueName: \"kubernetes.io/configmap/a64bcd05-8596-4c12-b88e-e974d676e25f-config\") pod \"console-operator-7866497494-pjc6j\" (UID: \"a64bcd05-8596-4c12-b88e-e974d676e25f\") " pod="openshift-console-operator/console-operator-7866497494-pjc6j" Jul 16 00:40:30.897401 ip-10-0-143-200 kubenswrapper[2572]: I0716 00:40:30.897222 2572 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"serving-cert\" (UniqueName: \"kubernetes.io/secret/1c3985e6-ff56-469e-bdd6-809bf98870ab-serving-cert\") pod \"service-ca-operator-7695fc6c8c-7v8cr\" (UID: \"1c3985e6-ff56-469e-bdd6-809bf98870ab\") " pod="openshift-service-ca-operator/service-ca-operator-7695fc6c8c-7v8cr" Jul 16 00:40:30.897401 ip-10-0-143-200 kubenswrapper[2572]: I0716 00:40:30.897258 2572 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"serving-cert\" (UniqueName: \"kubernetes.io/secret/0563669f-9ac5-43d0-a3c0-47c36275ee46-serving-cert\") pod \"insights-operator-68d9dd94cf-d4nrt\" (UID: \"0563669f-9ac5-43d0-a3c0-47c36275ee46\") " pod="openshift-insights/insights-operator-68d9dd94cf-d4nrt" Jul 16 00:40:30.897401 ip-10-0-143-200 kubenswrapper[2572]: I0716 00:40:30.897286 2572 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"serving-cert\" (UniqueName: \"kubernetes.io/secret/a64bcd05-8596-4c12-b88e-e974d676e25f-serving-cert\") pod \"console-operator-7866497494-pjc6j\" (UID: \"a64bcd05-8596-4c12-b88e-e974d676e25f\") " pod="openshift-console-operator/console-operator-7866497494-pjc6j" Jul 16 00:40:30.897401 ip-10-0-143-200 kubenswrapper[2572]: I0716 00:40:30.897316 2572 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-kmrt8\" (UniqueName: \"kubernetes.io/projected/f103d7a8-2560-409b-801e-701bcc6835c1-kube-api-access-kmrt8\") pod \"image-registry-dd7c989d9-5w25c\" (UID: \"f103d7a8-2560-409b-801e-701bcc6835c1\") " pod="openshift-image-registry/image-registry-dd7c989d9-5w25c" Jul 16 00:40:30.898136 ip-10-0-143-200 kubenswrapper[2572]: I0716 00:40:30.897356 2572 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"image-registry-private-configuration\" (UniqueName: \"kubernetes.io/secret/f103d7a8-2560-409b-801e-701bcc6835c1-image-registry-private-configuration\") pod \"image-registry-dd7c989d9-5w25c\" (UID: \"f103d7a8-2560-409b-801e-701bcc6835c1\") " pod="openshift-image-registry/image-registry-dd7c989d9-5w25c" Jul 16 00:40:30.898136 ip-10-0-143-200 kubenswrapper[2572]: I0716 00:40:30.897375 2572 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"tmp\" (UniqueName: \"kubernetes.io/empty-dir/0563669f-9ac5-43d0-a3c0-47c36275ee46-tmp\") pod \"insights-operator-68d9dd94cf-d4nrt\" (UID: \"0563669f-9ac5-43d0-a3c0-47c36275ee46\") " pod="openshift-insights/insights-operator-68d9dd94cf-d4nrt" Jul 16 00:40:30.898136 ip-10-0-143-200 kubenswrapper[2572]: I0716 00:40:30.897418 2572 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/0563669f-9ac5-43d0-a3c0-47c36275ee46-trusted-ca-bundle\") pod \"insights-operator-68d9dd94cf-d4nrt\" (UID: \"0563669f-9ac5-43d0-a3c0-47c36275ee46\") " pod="openshift-insights/insights-operator-68d9dd94cf-d4nrt" Jul 16 00:40:30.898136 ip-10-0-143-200 kubenswrapper[2572]: I0716 00:40:30.897422 2572 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"cluster-monitoring-operator-tls\" (UniqueName: \"kubernetes.io/secret/265fe067-cade-414b-be13-cb6dbab18057-cluster-monitoring-operator-tls\") pod \"cluster-monitoring-operator-5f7f494dd4-dxc26\" (UID: \"265fe067-cade-414b-be13-cb6dbab18057\") " pod="openshift-monitoring/cluster-monitoring-operator-5f7f494dd4-dxc26" Jul 16 00:40:30.898136 ip-10-0-143-200 kubenswrapper[2572]: I0716 00:40:30.897467 2572 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"registry-certificates\" (UniqueName: \"kubernetes.io/configmap/f103d7a8-2560-409b-801e-701bcc6835c1-registry-certificates\") pod \"image-registry-dd7c989d9-5w25c\" (UID: \"f103d7a8-2560-409b-801e-701bcc6835c1\") " pod="openshift-image-registry/image-registry-dd7c989d9-5w25c" Jul 16 00:40:30.898136 ip-10-0-143-200 kubenswrapper[2572]: I0716 00:40:30.897494 2572 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"serving-cert\" (UniqueName: \"kubernetes.io/secret/07e0ab9b-1bfe-4335-bac1-8d9d158b0085-serving-cert\") pod \"kube-storage-version-migrator-operator-799cb7f958-tksqh\" (UID: \"07e0ab9b-1bfe-4335-bac1-8d9d158b0085\") " pod="openshift-kube-storage-version-migrator-operator/kube-storage-version-migrator-operator-799cb7f958-tksqh" Jul 16 00:40:30.898136 ip-10-0-143-200 kubenswrapper[2572]: I0716 00:40:30.897534 2572 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-psdrt\" (UniqueName: \"kubernetes.io/projected/b82b08fd-bf48-4ee3-9d58-b648d6cba972-kube-api-access-psdrt\") pod \"router-default-78f9d48cbb-xwpwf\" (UID: \"b82b08fd-bf48-4ee3-9d58-b648d6cba972\") " pod="openshift-ingress/router-default-78f9d48cbb-xwpwf" Jul 16 00:40:30.898136 ip-10-0-143-200 kubenswrapper[2572]: I0716 00:40:30.897568 2572 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/645fc509-972b-4332-8562-2309cc0eab1b-tmp-dir\") pod \"dns-default-w6cgc\" (UID: \"645fc509-972b-4332-8562-2309cc0eab1b\") " pod="openshift-dns/dns-default-w6cgc" Jul 16 00:40:30.898136 ip-10-0-143-200 kubenswrapper[2572]: I0716 00:40:30.897602 2572 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"snapshots\" (UniqueName: \"kubernetes.io/empty-dir/0563669f-9ac5-43d0-a3c0-47c36275ee46-snapshots\") pod \"insights-operator-68d9dd94cf-d4nrt\" (UID: \"0563669f-9ac5-43d0-a3c0-47c36275ee46\") " pod="openshift-insights/insights-operator-68d9dd94cf-d4nrt" Jul 16 00:40:30.898565 ip-10-0-143-200 kubenswrapper[2572]: I0716 00:40:30.898230 2572 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"registry-certificates\" (UniqueName: \"kubernetes.io/configmap/f103d7a8-2560-409b-801e-701bcc6835c1-registry-certificates\") pod \"image-registry-dd7c989d9-5w25c\" (UID: \"f103d7a8-2560-409b-801e-701bcc6835c1\") " pod="openshift-image-registry/image-registry-dd7c989d9-5w25c" Jul 16 00:40:30.898565 ip-10-0-143-200 kubenswrapper[2572]: I0716 00:40:30.898378 2572 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"trusted-ca\" (UniqueName: \"kubernetes.io/configmap/f103d7a8-2560-409b-801e-701bcc6835c1-trusted-ca\") pod \"image-registry-dd7c989d9-5w25c\" (UID: \"f103d7a8-2560-409b-801e-701bcc6835c1\") " pod="openshift-image-registry/image-registry-dd7c989d9-5w25c" Jul 16 00:40:30.901362 ip-10-0-143-200 kubenswrapper[2572]: I0716 00:40:30.901329 2572 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"serving-cert\" (UniqueName: \"kubernetes.io/secret/0563669f-9ac5-43d0-a3c0-47c36275ee46-serving-cert\") pod \"insights-operator-68d9dd94cf-d4nrt\" (UID: \"0563669f-9ac5-43d0-a3c0-47c36275ee46\") " pod="openshift-insights/insights-operator-68d9dd94cf-d4nrt" Jul 16 00:40:30.901488 ip-10-0-143-200 kubenswrapper[2572]: I0716 00:40:30.901335 2572 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"installation-pull-secrets\" (UniqueName: \"kubernetes.io/secret/f103d7a8-2560-409b-801e-701bcc6835c1-installation-pull-secrets\") pod \"image-registry-dd7c989d9-5w25c\" (UID: \"f103d7a8-2560-409b-801e-701bcc6835c1\") " pod="openshift-image-registry/image-registry-dd7c989d9-5w25c" Jul 16 00:40:30.901488 ip-10-0-143-200 kubenswrapper[2572]: I0716 00:40:30.901336 2572 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"serving-cert\" (UniqueName: \"kubernetes.io/secret/07e0ab9b-1bfe-4335-bac1-8d9d158b0085-serving-cert\") pod \"kube-storage-version-migrator-operator-799cb7f958-tksqh\" (UID: \"07e0ab9b-1bfe-4335-bac1-8d9d158b0085\") " pod="openshift-kube-storage-version-migrator-operator/kube-storage-version-migrator-operator-799cb7f958-tksqh" Jul 16 00:40:30.901488 ip-10-0-143-200 kubenswrapper[2572]: I0716 00:40:30.901381 2572 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"image-registry-private-configuration\" (UniqueName: \"kubernetes.io/secret/f103d7a8-2560-409b-801e-701bcc6835c1-image-registry-private-configuration\") pod \"image-registry-dd7c989d9-5w25c\" (UID: \"f103d7a8-2560-409b-801e-701bcc6835c1\") " pod="openshift-image-registry/image-registry-dd7c989d9-5w25c" Jul 16 00:40:30.914336 ip-10-0-143-200 kubenswrapper[2572]: I0716 00:40:30.914309 2572 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-df7kv\" (UniqueName: \"kubernetes.io/projected/3cb174c0-4582-44f3-879a-23697944bc47-kube-api-access-df7kv\") pod \"volume-data-source-validator-78495597f6-wffwm\" (UID: \"3cb174c0-4582-44f3-879a-23697944bc47\") " pod="openshift-cluster-storage-operator/volume-data-source-validator-78495597f6-wffwm" Jul 16 00:40:30.914765 ip-10-0-143-200 kubenswrapper[2572]: I0716 00:40:30.914748 2572 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-zbnnk\" (UniqueName: \"kubernetes.io/projected/07e0ab9b-1bfe-4335-bac1-8d9d158b0085-kube-api-access-zbnnk\") pod \"kube-storage-version-migrator-operator-799cb7f958-tksqh\" (UID: \"07e0ab9b-1bfe-4335-bac1-8d9d158b0085\") " pod="openshift-kube-storage-version-migrator-operator/kube-storage-version-migrator-operator-799cb7f958-tksqh" Jul 16 00:40:30.917283 ip-10-0-143-200 kubenswrapper[2572]: I0716 00:40:30.917260 2572 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-snmxw\" (UniqueName: \"kubernetes.io/projected/0563669f-9ac5-43d0-a3c0-47c36275ee46-kube-api-access-snmxw\") pod \"insights-operator-68d9dd94cf-d4nrt\" (UID: \"0563669f-9ac5-43d0-a3c0-47c36275ee46\") " pod="openshift-insights/insights-operator-68d9dd94cf-d4nrt" Jul 16 00:40:30.917644 ip-10-0-143-200 kubenswrapper[2572]: I0716 00:40:30.917607 2572 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-kmrt8\" (UniqueName: \"kubernetes.io/projected/f103d7a8-2560-409b-801e-701bcc6835c1-kube-api-access-kmrt8\") pod \"image-registry-dd7c989d9-5w25c\" (UID: \"f103d7a8-2560-409b-801e-701bcc6835c1\") " pod="openshift-image-registry/image-registry-dd7c989d9-5w25c" Jul 16 00:40:30.918442 ip-10-0-143-200 kubenswrapper[2572]: I0716 00:40:30.918419 2572 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"bound-sa-token\" (UniqueName: \"kubernetes.io/projected/f103d7a8-2560-409b-801e-701bcc6835c1-bound-sa-token\") pod \"image-registry-dd7c989d9-5w25c\" (UID: \"f103d7a8-2560-409b-801e-701bcc6835c1\") " pod="openshift-image-registry/image-registry-dd7c989d9-5w25c" Jul 16 00:40:30.985997 ip-10-0-143-200 kubenswrapper[2572]: I0716 00:40:30.985963 2572 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-network-console/networking-console-plugin-6cf856758c-vc87p"] Jul 16 00:40:30.988441 ip-10-0-143-200 kubenswrapper[2572]: I0716 00:40:30.988421 2572 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-console/networking-console-plugin-6cf856758c-vc87p" Jul 16 00:40:30.992485 ip-10-0-143-200 kubenswrapper[2572]: I0716 00:40:30.992462 2572 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-network-console\"/\"default-dockercfg-tncmn\"" Jul 16 00:40:30.992485 ip-10-0-143-200 kubenswrapper[2572]: I0716 00:40:30.992479 2572 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-network-console\"/\"networking-console-plugin-cert\"" Jul 16 00:40:30.992667 ip-10-0-143-200 kubenswrapper[2572]: I0716 00:40:30.992487 2572 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-network-console\"/\"networking-console-plugin\"" Jul 16 00:40:30.998205 ip-10-0-143-200 kubenswrapper[2572]: I0716 00:40:30.998157 2572 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-nt4m9\" (UniqueName: \"kubernetes.io/projected/6f7eb059-755b-47de-82ab-bfaa72b8f899-kube-api-access-nt4m9\") pod \"ingress-canary-hhtd6\" (UID: \"6f7eb059-755b-47de-82ab-bfaa72b8f899\") " pod="openshift-ingress-canary/ingress-canary-hhtd6" Jul 16 00:40:30.998335 ip-10-0-143-200 kubenswrapper[2572]: I0716 00:40:30.998225 2572 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-t9sc4\" (UniqueName: \"kubernetes.io/projected/5aabb5b2-54bc-4893-91d2-3c0215fabd67-kube-api-access-t9sc4\") pod \"cluster-samples-operator-65db858894-zf8lz\" (UID: \"5aabb5b2-54bc-4893-91d2-3c0215fabd67\") " pod="openshift-cluster-samples-operator/cluster-samples-operator-65db858894-zf8lz" Jul 16 00:40:30.998335 ip-10-0-143-200 kubenswrapper[2572]: I0716 00:40:30.998283 2572 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"stats-auth\" (UniqueName: \"kubernetes.io/secret/b82b08fd-bf48-4ee3-9d58-b648d6cba972-stats-auth\") pod \"router-default-78f9d48cbb-xwpwf\" (UID: \"b82b08fd-bf48-4ee3-9d58-b648d6cba972\") " pod="openshift-ingress/router-default-78f9d48cbb-xwpwf" Jul 16 00:40:30.998335 ip-10-0-143-200 kubenswrapper[2572]: I0716 00:40:30.998310 2572 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"cert\" (UniqueName: \"kubernetes.io/secret/6f7eb059-755b-47de-82ab-bfaa72b8f899-cert\") pod \"ingress-canary-hhtd6\" (UID: \"6f7eb059-755b-47de-82ab-bfaa72b8f899\") " pod="openshift-ingress-canary/ingress-canary-hhtd6" Jul 16 00:40:30.998481 ip-10-0-143-200 kubenswrapper[2572]: I0716 00:40:30.998370 2572 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"trusted-ca\" (UniqueName: \"kubernetes.io/configmap/a64bcd05-8596-4c12-b88e-e974d676e25f-trusted-ca\") pod \"console-operator-7866497494-pjc6j\" (UID: \"a64bcd05-8596-4c12-b88e-e974d676e25f\") " pod="openshift-console-operator/console-operator-7866497494-pjc6j" Jul 16 00:40:30.998481 ip-10-0-143-200 kubenswrapper[2572]: I0716 00:40:30.998399 2572 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"samples-operator-tls\" (UniqueName: \"kubernetes.io/secret/5aabb5b2-54bc-4893-91d2-3c0215fabd67-samples-operator-tls\") pod \"cluster-samples-operator-65db858894-zf8lz\" (UID: \"5aabb5b2-54bc-4893-91d2-3c0215fabd67\") " pod="openshift-cluster-samples-operator/cluster-samples-operator-65db858894-zf8lz" Jul 16 00:40:30.998481 ip-10-0-143-200 kubenswrapper[2572]: I0716 00:40:30.998443 2572 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"telemetry-config\" (UniqueName: \"kubernetes.io/configmap/265fe067-cade-414b-be13-cb6dbab18057-telemetry-config\") pod \"cluster-monitoring-operator-5f7f494dd4-dxc26\" (UID: \"265fe067-cade-414b-be13-cb6dbab18057\") " pod="openshift-monitoring/cluster-monitoring-operator-5f7f494dd4-dxc26" Jul 16 00:40:30.999182 ip-10-0-143-200 kubenswrapper[2572]: E0716 00:40:30.998878 2572 secret.go:189] Couldn't get secret openshift-cluster-samples-operator/samples-operator-tls: secret "samples-operator-tls" not found Jul 16 00:40:30.999182 ip-10-0-143-200 kubenswrapper[2572]: E0716 00:40:30.998986 2572 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/5aabb5b2-54bc-4893-91d2-3c0215fabd67-samples-operator-tls podName:5aabb5b2-54bc-4893-91d2-3c0215fabd67 nodeName:}" failed. No retries permitted until 2026-07-16 00:40:31.498964187 +0000 UTC m=+36.607718843 (durationBeforeRetry 500ms). Error: MountVolume.SetUp failed for volume "samples-operator-tls" (UniqueName: "kubernetes.io/secret/5aabb5b2-54bc-4893-91d2-3c0215fabd67-samples-operator-tls") pod "cluster-samples-operator-65db858894-zf8lz" (UID: "5aabb5b2-54bc-4893-91d2-3c0215fabd67") : secret "samples-operator-tls" not found Jul 16 00:40:30.999481 ip-10-0-143-200 kubenswrapper[2572]: I0716 00:40:30.999460 2572 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-gxtdx\" (UniqueName: \"kubernetes.io/projected/265fe067-cade-414b-be13-cb6dbab18057-kube-api-access-gxtdx\") pod \"cluster-monitoring-operator-5f7f494dd4-dxc26\" (UID: \"265fe067-cade-414b-be13-cb6dbab18057\") " pod="openshift-monitoring/cluster-monitoring-operator-5f7f494dd4-dxc26" Jul 16 00:40:30.999566 ip-10-0-143-200 kubenswrapper[2572]: I0716 00:40:30.999507 2572 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-zsfwq\" (UniqueName: \"kubernetes.io/projected/645fc509-972b-4332-8562-2309cc0eab1b-kube-api-access-zsfwq\") pod \"dns-default-w6cgc\" (UID: \"645fc509-972b-4332-8562-2309cc0eab1b\") " pod="openshift-dns/dns-default-w6cgc" Jul 16 00:40:30.999566 ip-10-0-143-200 kubenswrapper[2572]: I0716 00:40:30.999537 2572 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"config\" (UniqueName: \"kubernetes.io/configmap/1c3985e6-ff56-469e-bdd6-809bf98870ab-config\") pod \"service-ca-operator-7695fc6c8c-7v8cr\" (UID: \"1c3985e6-ff56-469e-bdd6-809bf98870ab\") " pod="openshift-service-ca-operator/service-ca-operator-7695fc6c8c-7v8cr" Jul 16 00:40:30.999691 ip-10-0-143-200 kubenswrapper[2572]: I0716 00:40:30.999577 2572 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"default-certificate\" (UniqueName: \"kubernetes.io/secret/b82b08fd-bf48-4ee3-9d58-b648d6cba972-default-certificate\") pod \"router-default-78f9d48cbb-xwpwf\" (UID: \"b82b08fd-bf48-4ee3-9d58-b648d6cba972\") " pod="openshift-ingress/router-default-78f9d48cbb-xwpwf" Jul 16 00:40:30.999691 ip-10-0-143-200 kubenswrapper[2572]: I0716 00:40:30.999605 2572 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-mg2mn\" (UniqueName: \"kubernetes.io/projected/1c3985e6-ff56-469e-bdd6-809bf98870ab-kube-api-access-mg2mn\") pod \"service-ca-operator-7695fc6c8c-7v8cr\" (UID: \"1c3985e6-ff56-469e-bdd6-809bf98870ab\") " pod="openshift-service-ca-operator/service-ca-operator-7695fc6c8c-7v8cr" Jul 16 00:40:30.999691 ip-10-0-143-200 kubenswrapper[2572]: I0716 00:40:30.999672 2572 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-t2kdz\" (UniqueName: \"kubernetes.io/projected/a64bcd05-8596-4c12-b88e-e974d676e25f-kube-api-access-t2kdz\") pod \"console-operator-7866497494-pjc6j\" (UID: \"a64bcd05-8596-4c12-b88e-e974d676e25f\") " pod="openshift-console-operator/console-operator-7866497494-pjc6j" Jul 16 00:40:30.999836 ip-10-0-143-200 kubenswrapper[2572]: I0716 00:40:30.999700 2572 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"trusted-ca\" (UniqueName: \"kubernetes.io/configmap/a64bcd05-8596-4c12-b88e-e974d676e25f-trusted-ca\") pod \"console-operator-7866497494-pjc6j\" (UID: \"a64bcd05-8596-4c12-b88e-e974d676e25f\") " pod="openshift-console-operator/console-operator-7866497494-pjc6j" Jul 16 00:40:30.999836 ip-10-0-143-200 kubenswrapper[2572]: I0716 00:40:30.999723 2572 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"config-volume\" (UniqueName: \"kubernetes.io/configmap/645fc509-972b-4332-8562-2309cc0eab1b-config-volume\") pod \"dns-default-w6cgc\" (UID: \"645fc509-972b-4332-8562-2309cc0eab1b\") " pod="openshift-dns/dns-default-w6cgc" Jul 16 00:40:31.000915 ip-10-0-143-200 kubenswrapper[2572]: I0716 00:40:30.999932 2572 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-tls\" (UniqueName: \"kubernetes.io/secret/645fc509-972b-4332-8562-2309cc0eab1b-metrics-tls\") pod \"dns-default-w6cgc\" (UID: \"645fc509-972b-4332-8562-2309cc0eab1b\") " pod="openshift-dns/dns-default-w6cgc" Jul 16 00:40:31.000915 ip-10-0-143-200 kubenswrapper[2572]: I0716 00:40:30.999977 2572 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/b82b08fd-bf48-4ee3-9d58-b648d6cba972-metrics-certs\") pod \"router-default-78f9d48cbb-xwpwf\" (UID: \"b82b08fd-bf48-4ee3-9d58-b648d6cba972\") " pod="openshift-ingress/router-default-78f9d48cbb-xwpwf" Jul 16 00:40:31.000915 ip-10-0-143-200 kubenswrapper[2572]: I0716 00:40:31.000033 2572 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-4k4r4\" (UniqueName: \"kubernetes.io/projected/9da33896-31b6-4a05-bc7c-b9260876e581-kube-api-access-4k4r4\") pod \"network-check-source-869dcfcdcd-r6wkz\" (UID: \"9da33896-31b6-4a05-bc7c-b9260876e581\") " pod="openshift-network-diagnostics/network-check-source-869dcfcdcd-r6wkz" Jul 16 00:40:31.000915 ip-10-0-143-200 kubenswrapper[2572]: I0716 00:40:31.000068 2572 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"service-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/b82b08fd-bf48-4ee3-9d58-b648d6cba972-service-ca-bundle\") pod \"router-default-78f9d48cbb-xwpwf\" (UID: \"b82b08fd-bf48-4ee3-9d58-b648d6cba972\") " pod="openshift-ingress/router-default-78f9d48cbb-xwpwf" Jul 16 00:40:31.000915 ip-10-0-143-200 kubenswrapper[2572]: I0716 00:40:31.000065 2572 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"telemetry-config\" (UniqueName: \"kubernetes.io/configmap/265fe067-cade-414b-be13-cb6dbab18057-telemetry-config\") pod \"cluster-monitoring-operator-5f7f494dd4-dxc26\" (UID: \"265fe067-cade-414b-be13-cb6dbab18057\") " pod="openshift-monitoring/cluster-monitoring-operator-5f7f494dd4-dxc26" Jul 16 00:40:31.000915 ip-10-0-143-200 kubenswrapper[2572]: E0716 00:40:31.000159 2572 secret.go:189] Couldn't get secret openshift-dns/dns-default-metrics-tls: secret "dns-default-metrics-tls" not found Jul 16 00:40:31.000915 ip-10-0-143-200 kubenswrapper[2572]: E0716 00:40:31.000167 2572 secret.go:189] Couldn't get secret openshift-ingress/router-metrics-certs-default: secret "router-metrics-certs-default" not found Jul 16 00:40:31.000915 ip-10-0-143-200 kubenswrapper[2572]: E0716 00:40:31.000218 2572 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/645fc509-972b-4332-8562-2309cc0eab1b-metrics-tls podName:645fc509-972b-4332-8562-2309cc0eab1b nodeName:}" failed. No retries permitted until 2026-07-16 00:40:31.500201716 +0000 UTC m=+36.608956391 (durationBeforeRetry 500ms). Error: MountVolume.SetUp failed for volume "metrics-tls" (UniqueName: "kubernetes.io/secret/645fc509-972b-4332-8562-2309cc0eab1b-metrics-tls") pod "dns-default-w6cgc" (UID: "645fc509-972b-4332-8562-2309cc0eab1b") : secret "dns-default-metrics-tls" not found Jul 16 00:40:31.000915 ip-10-0-143-200 kubenswrapper[2572]: E0716 00:40:31.000238 2572 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/b82b08fd-bf48-4ee3-9d58-b648d6cba972-metrics-certs podName:b82b08fd-bf48-4ee3-9d58-b648d6cba972 nodeName:}" failed. No retries permitted until 2026-07-16 00:40:31.500226785 +0000 UTC m=+36.608981444 (durationBeforeRetry 500ms). Error: MountVolume.SetUp failed for volume "metrics-certs" (UniqueName: "kubernetes.io/secret/b82b08fd-bf48-4ee3-9d58-b648d6cba972-metrics-certs") pod "router-default-78f9d48cbb-xwpwf" (UID: "b82b08fd-bf48-4ee3-9d58-b648d6cba972") : secret "router-metrics-certs-default" not found Jul 16 00:40:31.000915 ip-10-0-143-200 kubenswrapper[2572]: I0716 00:40:31.000255 2572 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"config\" (UniqueName: \"kubernetes.io/configmap/a64bcd05-8596-4c12-b88e-e974d676e25f-config\") pod \"console-operator-7866497494-pjc6j\" (UID: \"a64bcd05-8596-4c12-b88e-e974d676e25f\") " pod="openshift-console-operator/console-operator-7866497494-pjc6j" Jul 16 00:40:31.000915 ip-10-0-143-200 kubenswrapper[2572]: E0716 00:40:31.000282 2572 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/configmap/b82b08fd-bf48-4ee3-9d58-b648d6cba972-service-ca-bundle podName:b82b08fd-bf48-4ee3-9d58-b648d6cba972 nodeName:}" failed. No retries permitted until 2026-07-16 00:40:31.500266346 +0000 UTC m=+36.609021034 (durationBeforeRetry 500ms). Error: MountVolume.SetUp failed for volume "service-ca-bundle" (UniqueName: "kubernetes.io/configmap/b82b08fd-bf48-4ee3-9d58-b648d6cba972-service-ca-bundle") pod "router-default-78f9d48cbb-xwpwf" (UID: "b82b08fd-bf48-4ee3-9d58-b648d6cba972") : configmap references non-existent config key: service-ca.crt Jul 16 00:40:31.000915 ip-10-0-143-200 kubenswrapper[2572]: I0716 00:40:31.000329 2572 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"serving-cert\" (UniqueName: \"kubernetes.io/secret/1c3985e6-ff56-469e-bdd6-809bf98870ab-serving-cert\") pod \"service-ca-operator-7695fc6c8c-7v8cr\" (UID: \"1c3985e6-ff56-469e-bdd6-809bf98870ab\") " pod="openshift-service-ca-operator/service-ca-operator-7695fc6c8c-7v8cr" Jul 16 00:40:31.000915 ip-10-0-143-200 kubenswrapper[2572]: I0716 00:40:31.000361 2572 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"serving-cert\" (UniqueName: \"kubernetes.io/secret/a64bcd05-8596-4c12-b88e-e974d676e25f-serving-cert\") pod \"console-operator-7866497494-pjc6j\" (UID: \"a64bcd05-8596-4c12-b88e-e974d676e25f\") " pod="openshift-console-operator/console-operator-7866497494-pjc6j" Jul 16 00:40:31.000915 ip-10-0-143-200 kubenswrapper[2572]: I0716 00:40:31.000394 2572 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"cluster-monitoring-operator-tls\" (UniqueName: \"kubernetes.io/secret/265fe067-cade-414b-be13-cb6dbab18057-cluster-monitoring-operator-tls\") pod \"cluster-monitoring-operator-5f7f494dd4-dxc26\" (UID: \"265fe067-cade-414b-be13-cb6dbab18057\") " pod="openshift-monitoring/cluster-monitoring-operator-5f7f494dd4-dxc26" Jul 16 00:40:31.000915 ip-10-0-143-200 kubenswrapper[2572]: I0716 00:40:31.000443 2572 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-psdrt\" (UniqueName: \"kubernetes.io/projected/b82b08fd-bf48-4ee3-9d58-b648d6cba972-kube-api-access-psdrt\") pod \"router-default-78f9d48cbb-xwpwf\" (UID: \"b82b08fd-bf48-4ee3-9d58-b648d6cba972\") " pod="openshift-ingress/router-default-78f9d48cbb-xwpwf" Jul 16 00:40:31.001842 ip-10-0-143-200 kubenswrapper[2572]: I0716 00:40:31.000469 2572 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/645fc509-972b-4332-8562-2309cc0eab1b-tmp-dir\") pod \"dns-default-w6cgc\" (UID: \"645fc509-972b-4332-8562-2309cc0eab1b\") " pod="openshift-dns/dns-default-w6cgc" Jul 16 00:40:31.001842 ip-10-0-143-200 kubenswrapper[2572]: I0716 00:40:31.000484 2572 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"config\" (UniqueName: \"kubernetes.io/configmap/1c3985e6-ff56-469e-bdd6-809bf98870ab-config\") pod \"service-ca-operator-7695fc6c8c-7v8cr\" (UID: \"1c3985e6-ff56-469e-bdd6-809bf98870ab\") " pod="openshift-service-ca-operator/service-ca-operator-7695fc6c8c-7v8cr" Jul 16 00:40:31.001842 ip-10-0-143-200 kubenswrapper[2572]: E0716 00:40:31.000679 2572 secret.go:189] Couldn't get secret openshift-monitoring/cluster-monitoring-operator-tls: secret "cluster-monitoring-operator-tls" not found Jul 16 00:40:31.001842 ip-10-0-143-200 kubenswrapper[2572]: I0716 00:40:31.000791 2572 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/645fc509-972b-4332-8562-2309cc0eab1b-tmp-dir\") pod \"dns-default-w6cgc\" (UID: \"645fc509-972b-4332-8562-2309cc0eab1b\") " pod="openshift-dns/dns-default-w6cgc" Jul 16 00:40:31.001842 ip-10-0-143-200 kubenswrapper[2572]: I0716 00:40:31.000852 2572 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"config-volume\" (UniqueName: \"kubernetes.io/configmap/645fc509-972b-4332-8562-2309cc0eab1b-config-volume\") pod \"dns-default-w6cgc\" (UID: \"645fc509-972b-4332-8562-2309cc0eab1b\") " pod="openshift-dns/dns-default-w6cgc" Jul 16 00:40:31.001842 ip-10-0-143-200 kubenswrapper[2572]: I0716 00:40:31.000865 2572 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"config\" (UniqueName: \"kubernetes.io/configmap/a64bcd05-8596-4c12-b88e-e974d676e25f-config\") pod \"console-operator-7866497494-pjc6j\" (UID: \"a64bcd05-8596-4c12-b88e-e974d676e25f\") " pod="openshift-console-operator/console-operator-7866497494-pjc6j" Jul 16 00:40:31.001842 ip-10-0-143-200 kubenswrapper[2572]: E0716 00:40:31.000878 2572 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/265fe067-cade-414b-be13-cb6dbab18057-cluster-monitoring-operator-tls podName:265fe067-cade-414b-be13-cb6dbab18057 nodeName:}" failed. No retries permitted until 2026-07-16 00:40:31.500857598 +0000 UTC m=+36.609612266 (durationBeforeRetry 500ms). Error: MountVolume.SetUp failed for volume "cluster-monitoring-operator-tls" (UniqueName: "kubernetes.io/secret/265fe067-cade-414b-be13-cb6dbab18057-cluster-monitoring-operator-tls") pod "cluster-monitoring-operator-5f7f494dd4-dxc26" (UID: "265fe067-cade-414b-be13-cb6dbab18057") : secret "cluster-monitoring-operator-tls" not found Jul 16 00:40:31.002903 ip-10-0-143-200 kubenswrapper[2572]: I0716 00:40:31.002593 2572 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"stats-auth\" (UniqueName: \"kubernetes.io/secret/b82b08fd-bf48-4ee3-9d58-b648d6cba972-stats-auth\") pod \"router-default-78f9d48cbb-xwpwf\" (UID: \"b82b08fd-bf48-4ee3-9d58-b648d6cba972\") " pod="openshift-ingress/router-default-78f9d48cbb-xwpwf" Jul 16 00:40:31.003105 ip-10-0-143-200 kubenswrapper[2572]: I0716 00:40:31.002966 2572 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"default-certificate\" (UniqueName: \"kubernetes.io/secret/b82b08fd-bf48-4ee3-9d58-b648d6cba972-default-certificate\") pod \"router-default-78f9d48cbb-xwpwf\" (UID: \"b82b08fd-bf48-4ee3-9d58-b648d6cba972\") " pod="openshift-ingress/router-default-78f9d48cbb-xwpwf" Jul 16 00:40:31.003462 ip-10-0-143-200 kubenswrapper[2572]: I0716 00:40:31.003443 2572 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"serving-cert\" (UniqueName: \"kubernetes.io/secret/a64bcd05-8596-4c12-b88e-e974d676e25f-serving-cert\") pod \"console-operator-7866497494-pjc6j\" (UID: \"a64bcd05-8596-4c12-b88e-e974d676e25f\") " pod="openshift-console-operator/console-operator-7866497494-pjc6j" Jul 16 00:40:31.003780 ip-10-0-143-200 kubenswrapper[2572]: I0716 00:40:31.003761 2572 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"serving-cert\" (UniqueName: \"kubernetes.io/secret/1c3985e6-ff56-469e-bdd6-809bf98870ab-serving-cert\") pod \"service-ca-operator-7695fc6c8c-7v8cr\" (UID: \"1c3985e6-ff56-469e-bdd6-809bf98870ab\") " pod="openshift-service-ca-operator/service-ca-operator-7695fc6c8c-7v8cr" Jul 16 00:40:31.003877 ip-10-0-143-200 kubenswrapper[2572]: I0716 00:40:31.003851 2572 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"cert\" (UniqueName: \"kubernetes.io/secret/6f7eb059-755b-47de-82ab-bfaa72b8f899-cert\") pod \"ingress-canary-hhtd6\" (UID: \"6f7eb059-755b-47de-82ab-bfaa72b8f899\") " pod="openshift-ingress-canary/ingress-canary-hhtd6" Jul 16 00:40:31.005580 ip-10-0-143-200 kubenswrapper[2572]: I0716 00:40:31.005559 2572 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-network-console/networking-console-plugin-6cf856758c-vc87p"] Jul 16 00:40:31.012895 ip-10-0-143-200 kubenswrapper[2572]: I0716 00:40:31.012872 2572 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-kube-storage-version-migrator-operator/kube-storage-version-migrator-operator-799cb7f958-tksqh" Jul 16 00:40:31.013852 ip-10-0-143-200 kubenswrapper[2572]: I0716 00:40:31.013472 2572 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-gxtdx\" (UniqueName: \"kubernetes.io/projected/265fe067-cade-414b-be13-cb6dbab18057-kube-api-access-gxtdx\") pod \"cluster-monitoring-operator-5f7f494dd4-dxc26\" (UID: \"265fe067-cade-414b-be13-cb6dbab18057\") " pod="openshift-monitoring/cluster-monitoring-operator-5f7f494dd4-dxc26" Jul 16 00:40:31.014007 ip-10-0-143-200 kubenswrapper[2572]: I0716 00:40:31.013898 2572 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-nt4m9\" (UniqueName: \"kubernetes.io/projected/6f7eb059-755b-47de-82ab-bfaa72b8f899-kube-api-access-nt4m9\") pod \"ingress-canary-hhtd6\" (UID: \"6f7eb059-755b-47de-82ab-bfaa72b8f899\") " pod="openshift-ingress-canary/ingress-canary-hhtd6" Jul 16 00:40:31.014130 ip-10-0-143-200 kubenswrapper[2572]: I0716 00:40:31.014107 2572 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-zsfwq\" (UniqueName: \"kubernetes.io/projected/645fc509-972b-4332-8562-2309cc0eab1b-kube-api-access-zsfwq\") pod \"dns-default-w6cgc\" (UID: \"645fc509-972b-4332-8562-2309cc0eab1b\") " pod="openshift-dns/dns-default-w6cgc" Jul 16 00:40:31.014731 ip-10-0-143-200 kubenswrapper[2572]: I0716 00:40:31.014702 2572 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-mg2mn\" (UniqueName: \"kubernetes.io/projected/1c3985e6-ff56-469e-bdd6-809bf98870ab-kube-api-access-mg2mn\") pod \"service-ca-operator-7695fc6c8c-7v8cr\" (UID: \"1c3985e6-ff56-469e-bdd6-809bf98870ab\") " pod="openshift-service-ca-operator/service-ca-operator-7695fc6c8c-7v8cr" Jul 16 00:40:31.015992 ip-10-0-143-200 kubenswrapper[2572]: I0716 00:40:31.015851 2572 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-t2kdz\" (UniqueName: \"kubernetes.io/projected/a64bcd05-8596-4c12-b88e-e974d676e25f-kube-api-access-t2kdz\") pod \"console-operator-7866497494-pjc6j\" (UID: \"a64bcd05-8596-4c12-b88e-e974d676e25f\") " pod="openshift-console-operator/console-operator-7866497494-pjc6j" Jul 16 00:40:31.016597 ip-10-0-143-200 kubenswrapper[2572]: I0716 00:40:31.016568 2572 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-t9sc4\" (UniqueName: \"kubernetes.io/projected/5aabb5b2-54bc-4893-91d2-3c0215fabd67-kube-api-access-t9sc4\") pod \"cluster-samples-operator-65db858894-zf8lz\" (UID: \"5aabb5b2-54bc-4893-91d2-3c0215fabd67\") " pod="openshift-cluster-samples-operator/cluster-samples-operator-65db858894-zf8lz" Jul 16 00:40:31.016778 ip-10-0-143-200 kubenswrapper[2572]: I0716 00:40:31.016760 2572 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-psdrt\" (UniqueName: \"kubernetes.io/projected/b82b08fd-bf48-4ee3-9d58-b648d6cba972-kube-api-access-psdrt\") pod \"router-default-78f9d48cbb-xwpwf\" (UID: \"b82b08fd-bf48-4ee3-9d58-b648d6cba972\") " pod="openshift-ingress/router-default-78f9d48cbb-xwpwf" Jul 16 00:40:31.016846 ip-10-0-143-200 kubenswrapper[2572]: I0716 00:40:31.016796 2572 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-4k4r4\" (UniqueName: \"kubernetes.io/projected/9da33896-31b6-4a05-bc7c-b9260876e581-kube-api-access-4k4r4\") pod \"network-check-source-869dcfcdcd-r6wkz\" (UID: \"9da33896-31b6-4a05-bc7c-b9260876e581\") " pod="openshift-network-diagnostics/network-check-source-869dcfcdcd-r6wkz" Jul 16 00:40:31.025088 ip-10-0-143-200 kubenswrapper[2572]: I0716 00:40:31.025060 2572 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-cluster-storage-operator/volume-data-source-validator-78495597f6-wffwm" Jul 16 00:40:31.037581 ip-10-0-143-200 kubenswrapper[2572]: I0716 00:40:31.037236 2572 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-insights/insights-operator-68d9dd94cf-d4nrt" Jul 16 00:40:31.055336 ip-10-0-143-200 kubenswrapper[2572]: I0716 00:40:31.055302 2572 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-source-869dcfcdcd-r6wkz" Jul 16 00:40:31.074499 ip-10-0-143-200 kubenswrapper[2572]: I0716 00:40:31.074460 2572 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-service-ca-operator/service-ca-operator-7695fc6c8c-7v8cr" Jul 16 00:40:31.081778 ip-10-0-143-200 kubenswrapper[2572]: I0716 00:40:31.081569 2572 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-console-operator/console-operator-7866497494-pjc6j" Jul 16 00:40:31.101357 ip-10-0-143-200 kubenswrapper[2572]: I0716 00:40:31.101260 2572 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"nginx-conf\" (UniqueName: \"kubernetes.io/configmap/605373ad-28e7-4a57-9143-a698a1042a75-nginx-conf\") pod \"networking-console-plugin-6cf856758c-vc87p\" (UID: \"605373ad-28e7-4a57-9143-a698a1042a75\") " pod="openshift-network-console/networking-console-plugin-6cf856758c-vc87p" Jul 16 00:40:31.101571 ip-10-0-143-200 kubenswrapper[2572]: I0716 00:40:31.101529 2572 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"networking-console-plugin-cert\" (UniqueName: \"kubernetes.io/secret/605373ad-28e7-4a57-9143-a698a1042a75-networking-console-plugin-cert\") pod \"networking-console-plugin-6cf856758c-vc87p\" (UID: \"605373ad-28e7-4a57-9143-a698a1042a75\") " pod="openshift-network-console/networking-console-plugin-6cf856758c-vc87p" Jul 16 00:40:31.106417 ip-10-0-143-200 kubenswrapper[2572]: I0716 00:40:31.104120 2572 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress-canary/ingress-canary-hhtd6" Jul 16 00:40:31.202230 ip-10-0-143-200 kubenswrapper[2572]: I0716 00:40:31.202191 2572 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"networking-console-plugin-cert\" (UniqueName: \"kubernetes.io/secret/605373ad-28e7-4a57-9143-a698a1042a75-networking-console-plugin-cert\") pod \"networking-console-plugin-6cf856758c-vc87p\" (UID: \"605373ad-28e7-4a57-9143-a698a1042a75\") " pod="openshift-network-console/networking-console-plugin-6cf856758c-vc87p" Jul 16 00:40:31.202427 ip-10-0-143-200 kubenswrapper[2572]: I0716 00:40:31.202241 2572 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"nginx-conf\" (UniqueName: \"kubernetes.io/configmap/605373ad-28e7-4a57-9143-a698a1042a75-nginx-conf\") pod \"networking-console-plugin-6cf856758c-vc87p\" (UID: \"605373ad-28e7-4a57-9143-a698a1042a75\") " pod="openshift-network-console/networking-console-plugin-6cf856758c-vc87p" Jul 16 00:40:31.202427 ip-10-0-143-200 kubenswrapper[2572]: E0716 00:40:31.202342 2572 secret.go:189] Couldn't get secret openshift-network-console/networking-console-plugin-cert: secret "networking-console-plugin-cert" not found Jul 16 00:40:31.202427 ip-10-0-143-200 kubenswrapper[2572]: E0716 00:40:31.202411 2572 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/605373ad-28e7-4a57-9143-a698a1042a75-networking-console-plugin-cert podName:605373ad-28e7-4a57-9143-a698a1042a75 nodeName:}" failed. No retries permitted until 2026-07-16 00:40:31.702394611 +0000 UTC m=+36.811149268 (durationBeforeRetry 500ms). Error: MountVolume.SetUp failed for volume "networking-console-plugin-cert" (UniqueName: "kubernetes.io/secret/605373ad-28e7-4a57-9143-a698a1042a75-networking-console-plugin-cert") pod "networking-console-plugin-6cf856758c-vc87p" (UID: "605373ad-28e7-4a57-9143-a698a1042a75") : secret "networking-console-plugin-cert" not found Jul 16 00:40:31.202953 ip-10-0-143-200 kubenswrapper[2572]: I0716 00:40:31.202936 2572 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"nginx-conf\" (UniqueName: \"kubernetes.io/configmap/605373ad-28e7-4a57-9143-a698a1042a75-nginx-conf\") pod \"networking-console-plugin-6cf856758c-vc87p\" (UID: \"605373ad-28e7-4a57-9143-a698a1042a75\") " pod="openshift-network-console/networking-console-plugin-6cf856758c-vc87p" Jul 16 00:40:31.226674 ip-10-0-143-200 kubenswrapper[2572]: I0716 00:40:31.226648 2572 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-cluster-storage-operator/volume-data-source-validator-78495597f6-wffwm"] Jul 16 00:40:31.256769 ip-10-0-143-200 kubenswrapper[2572]: I0716 00:40:31.256741 2572 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-kube-storage-version-migrator-operator/kube-storage-version-migrator-operator-799cb7f958-tksqh"] Jul 16 00:40:31.288115 ip-10-0-143-200 kubenswrapper[2572]: I0716 00:40:31.287873 2572 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9gnmq_3a7e030d-fad8-480d-9dc6-fe5057048cf9/konnectivity-agent/0.log" limit=1024 Jul 16 00:40:31.405021 ip-10-0-143-200 kubenswrapper[2572]: I0716 00:40:31.404957 2572 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"registry-tls\" (UniqueName: \"kubernetes.io/projected/f103d7a8-2560-409b-801e-701bcc6835c1-registry-tls\") pod \"image-registry-dd7c989d9-5w25c\" (UID: \"f103d7a8-2560-409b-801e-701bcc6835c1\") " pod="openshift-image-registry/image-registry-dd7c989d9-5w25c" Jul 16 00:40:31.405132 ip-10-0-143-200 kubenswrapper[2572]: E0716 00:40:31.405098 2572 projected.go:266] Couldn't get secret openshift-image-registry/image-registry-tls: secret "image-registry-tls" not found Jul 16 00:40:31.405132 ip-10-0-143-200 kubenswrapper[2572]: E0716 00:40:31.405123 2572 projected.go:196] Error preparing data for projected volume registry-tls for pod openshift-image-registry/image-registry-dd7c989d9-5w25c: secret "image-registry-tls" not found Jul 16 00:40:31.405531 ip-10-0-143-200 kubenswrapper[2572]: E0716 00:40:31.405194 2572 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/projected/f103d7a8-2560-409b-801e-701bcc6835c1-registry-tls podName:f103d7a8-2560-409b-801e-701bcc6835c1 nodeName:}" failed. No retries permitted until 2026-07-16 00:40:32.405170658 +0000 UTC m=+37.513925334 (durationBeforeRetry 1s). Error: MountVolume.SetUp failed for volume "registry-tls" (UniqueName: "kubernetes.io/projected/f103d7a8-2560-409b-801e-701bcc6835c1-registry-tls") pod "image-registry-dd7c989d9-5w25c" (UID: "f103d7a8-2560-409b-801e-701bcc6835c1") : secret "image-registry-tls" not found Jul 16 00:40:31.470620 ip-10-0-143-200 kubenswrapper[2572]: I0716 00:40:31.470531 2572 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress-canary/ingress-canary-hhtd6"] Jul 16 00:40:31.471574 ip-10-0-143-200 kubenswrapper[2572]: I0716 00:40:31.471552 2572 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-console-operator/console-operator-7866497494-pjc6j"] Jul 16 00:40:31.472399 ip-10-0-143-200 kubenswrapper[2572]: I0716 00:40:31.472380 2572 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-service-ca-operator/service-ca-operator-7695fc6c8c-7v8cr"] Jul 16 00:40:31.475080 ip-10-0-143-200 kubenswrapper[2572]: I0716 00:40:31.475011 2572 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9gnmq_3a7e030d-fad8-480d-9dc6-fe5057048cf9/konnectivity-agent/0.log" limit=1024 Jul 16 00:40:31.483227 ip-10-0-143-200 kubenswrapper[2572]: I0716 00:40:31.483202 2572 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-insights/insights-operator-68d9dd94cf-d4nrt"] Jul 16 00:40:31.502170 ip-10-0-143-200 kubenswrapper[2572]: I0716 00:40:31.502136 2572 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-network-diagnostics/network-check-source-869dcfcdcd-r6wkz"] Jul 16 00:40:31.505485 ip-10-0-143-200 kubenswrapper[2572]: I0716 00:40:31.505453 2572 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"samples-operator-tls\" (UniqueName: \"kubernetes.io/secret/5aabb5b2-54bc-4893-91d2-3c0215fabd67-samples-operator-tls\") pod \"cluster-samples-operator-65db858894-zf8lz\" (UID: \"5aabb5b2-54bc-4893-91d2-3c0215fabd67\") " pod="openshift-cluster-samples-operator/cluster-samples-operator-65db858894-zf8lz" Jul 16 00:40:31.505722 ip-10-0-143-200 kubenswrapper[2572]: E0716 00:40:31.505585 2572 secret.go:189] Couldn't get secret openshift-cluster-samples-operator/samples-operator-tls: secret "samples-operator-tls" not found Jul 16 00:40:31.505797 ip-10-0-143-200 kubenswrapper[2572]: E0716 00:40:31.505785 2572 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/5aabb5b2-54bc-4893-91d2-3c0215fabd67-samples-operator-tls podName:5aabb5b2-54bc-4893-91d2-3c0215fabd67 nodeName:}" failed. No retries permitted until 2026-07-16 00:40:32.505762117 +0000 UTC m=+37.614516784 (durationBeforeRetry 1s). Error: MountVolume.SetUp failed for volume "samples-operator-tls" (UniqueName: "kubernetes.io/secret/5aabb5b2-54bc-4893-91d2-3c0215fabd67-samples-operator-tls") pod "cluster-samples-operator-65db858894-zf8lz" (UID: "5aabb5b2-54bc-4893-91d2-3c0215fabd67") : secret "samples-operator-tls" not found Jul 16 00:40:31.505871 ip-10-0-143-200 kubenswrapper[2572]: E0716 00:40:31.505857 2572 secret.go:189] Couldn't get secret openshift-dns/dns-default-metrics-tls: secret "dns-default-metrics-tls" not found Jul 16 00:40:31.505920 ip-10-0-143-200 kubenswrapper[2572]: W0716 00:40:31.505902 2572 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod6f7eb059_755b_47de_82ab_bfaa72b8f899.slice/crio-1daeff1de9377a91b431a853401bd1465540f38c62d1f3d0019d80536f6e9710 WatchSource:0}: Error finding container 1daeff1de9377a91b431a853401bd1465540f38c62d1f3d0019d80536f6e9710: Status 404 returned error can't find the container with id 1daeff1de9377a91b431a853401bd1465540f38c62d1f3d0019d80536f6e9710 Jul 16 00:40:31.505987 ip-10-0-143-200 kubenswrapper[2572]: E0716 00:40:31.505914 2572 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/645fc509-972b-4332-8562-2309cc0eab1b-metrics-tls podName:645fc509-972b-4332-8562-2309cc0eab1b nodeName:}" failed. No retries permitted until 2026-07-16 00:40:32.50589747 +0000 UTC m=+37.614652139 (durationBeforeRetry 1s). Error: MountVolume.SetUp failed for volume "metrics-tls" (UniqueName: "kubernetes.io/secret/645fc509-972b-4332-8562-2309cc0eab1b-metrics-tls") pod "dns-default-w6cgc" (UID: "645fc509-972b-4332-8562-2309cc0eab1b") : secret "dns-default-metrics-tls" not found Jul 16 00:40:31.506427 ip-10-0-143-200 kubenswrapper[2572]: I0716 00:40:31.506058 2572 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-tls\" (UniqueName: \"kubernetes.io/secret/645fc509-972b-4332-8562-2309cc0eab1b-metrics-tls\") pod \"dns-default-w6cgc\" (UID: \"645fc509-972b-4332-8562-2309cc0eab1b\") " pod="openshift-dns/dns-default-w6cgc" Jul 16 00:40:31.506427 ip-10-0-143-200 kubenswrapper[2572]: I0716 00:40:31.506122 2572 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/b82b08fd-bf48-4ee3-9d58-b648d6cba972-metrics-certs\") pod \"router-default-78f9d48cbb-xwpwf\" (UID: \"b82b08fd-bf48-4ee3-9d58-b648d6cba972\") " pod="openshift-ingress/router-default-78f9d48cbb-xwpwf" Jul 16 00:40:31.506427 ip-10-0-143-200 kubenswrapper[2572]: I0716 00:40:31.506156 2572 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"service-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/b82b08fd-bf48-4ee3-9d58-b648d6cba972-service-ca-bundle\") pod \"router-default-78f9d48cbb-xwpwf\" (UID: \"b82b08fd-bf48-4ee3-9d58-b648d6cba972\") " pod="openshift-ingress/router-default-78f9d48cbb-xwpwf" Jul 16 00:40:31.506427 ip-10-0-143-200 kubenswrapper[2572]: I0716 00:40:31.506209 2572 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"cluster-monitoring-operator-tls\" (UniqueName: \"kubernetes.io/secret/265fe067-cade-414b-be13-cb6dbab18057-cluster-monitoring-operator-tls\") pod \"cluster-monitoring-operator-5f7f494dd4-dxc26\" (UID: \"265fe067-cade-414b-be13-cb6dbab18057\") " pod="openshift-monitoring/cluster-monitoring-operator-5f7f494dd4-dxc26" Jul 16 00:40:31.506427 ip-10-0-143-200 kubenswrapper[2572]: E0716 00:40:31.506237 2572 secret.go:189] Couldn't get secret openshift-ingress/router-metrics-certs-default: secret "router-metrics-certs-default" not found Jul 16 00:40:31.506427 ip-10-0-143-200 kubenswrapper[2572]: E0716 00:40:31.506287 2572 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/b82b08fd-bf48-4ee3-9d58-b648d6cba972-metrics-certs podName:b82b08fd-bf48-4ee3-9d58-b648d6cba972 nodeName:}" failed. No retries permitted until 2026-07-16 00:40:32.506271709 +0000 UTC m=+37.615026366 (durationBeforeRetry 1s). Error: MountVolume.SetUp failed for volume "metrics-certs" (UniqueName: "kubernetes.io/secret/b82b08fd-bf48-4ee3-9d58-b648d6cba972-metrics-certs") pod "router-default-78f9d48cbb-xwpwf" (UID: "b82b08fd-bf48-4ee3-9d58-b648d6cba972") : secret "router-metrics-certs-default" not found Jul 16 00:40:31.506427 ip-10-0-143-200 kubenswrapper[2572]: E0716 00:40:31.506320 2572 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/configmap/b82b08fd-bf48-4ee3-9d58-b648d6cba972-service-ca-bundle podName:b82b08fd-bf48-4ee3-9d58-b648d6cba972 nodeName:}" failed. No retries permitted until 2026-07-16 00:40:32.506307185 +0000 UTC m=+37.615061839 (durationBeforeRetry 1s). Error: MountVolume.SetUp failed for volume "service-ca-bundle" (UniqueName: "kubernetes.io/configmap/b82b08fd-bf48-4ee3-9d58-b648d6cba972-service-ca-bundle") pod "router-default-78f9d48cbb-xwpwf" (UID: "b82b08fd-bf48-4ee3-9d58-b648d6cba972") : configmap references non-existent config key: service-ca.crt Jul 16 00:40:31.506427 ip-10-0-143-200 kubenswrapper[2572]: E0716 00:40:31.506363 2572 secret.go:189] Couldn't get secret openshift-monitoring/cluster-monitoring-operator-tls: secret "cluster-monitoring-operator-tls" not found Jul 16 00:40:31.506427 ip-10-0-143-200 kubenswrapper[2572]: E0716 00:40:31.506407 2572 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/265fe067-cade-414b-be13-cb6dbab18057-cluster-monitoring-operator-tls podName:265fe067-cade-414b-be13-cb6dbab18057 nodeName:}" failed. No retries permitted until 2026-07-16 00:40:32.506393891 +0000 UTC m=+37.615148562 (durationBeforeRetry 1s). Error: MountVolume.SetUp failed for volume "cluster-monitoring-operator-tls" (UniqueName: "kubernetes.io/secret/265fe067-cade-414b-be13-cb6dbab18057-cluster-monitoring-operator-tls") pod "cluster-monitoring-operator-5f7f494dd4-dxc26" (UID: "265fe067-cade-414b-be13-cb6dbab18057") : secret "cluster-monitoring-operator-tls" not found Jul 16 00:40:31.506427 ip-10-0-143-200 kubenswrapper[2572]: W0716 00:40:31.506409 2572 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod0563669f_9ac5_43d0_a3c0_47c36275ee46.slice/crio-8a03f7f5e0002fddd9a9887475267d10df95d99d171d180b5c000858669fa397 WatchSource:0}: Error finding container 8a03f7f5e0002fddd9a9887475267d10df95d99d171d180b5c000858669fa397: Status 404 returned error can't find the container with id 8a03f7f5e0002fddd9a9887475267d10df95d99d171d180b5c000858669fa397 Jul 16 00:40:31.623149 ip-10-0-143-200 kubenswrapper[2572]: I0716 00:40:31.623108 2572 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9gnmq_3a7e030d-fad8-480d-9dc6-fe5057048cf9/konnectivity-agent/0.log" limit=1024 Jul 16 00:40:31.657194 ip-10-0-143-200 kubenswrapper[2572]: I0716 00:40:31.657161 2572 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-console-operator/console-operator-7866497494-pjc6j" event={"ID":"a64bcd05-8596-4c12-b88e-e974d676e25f","Type":"ContainerStarted","Data":"c3288ffe72098f666332e4b9c792d2caec91991fcf2ea76736ca5610dc313ec7"} Jul 16 00:40:31.658285 ip-10-0-143-200 kubenswrapper[2572]: I0716 00:40:31.658262 2572 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-network-diagnostics/network-check-source-869dcfcdcd-r6wkz" event={"ID":"9da33896-31b6-4a05-bc7c-b9260876e581","Type":"ContainerStarted","Data":"4baba799dd0acd2c6c4d5636c10f015ede73897e55b5cdd323d2d5dc052c33bd"} Jul 16 00:40:31.659292 ip-10-0-143-200 kubenswrapper[2572]: I0716 00:40:31.659271 2572 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-insights/insights-operator-68d9dd94cf-d4nrt" event={"ID":"0563669f-9ac5-43d0-a3c0-47c36275ee46","Type":"ContainerStarted","Data":"8a03f7f5e0002fddd9a9887475267d10df95d99d171d180b5c000858669fa397"} Jul 16 00:40:31.661783 ip-10-0-143-200 kubenswrapper[2572]: I0716 00:40:31.661760 2572 generic.go:356] "Generic (PLEG): container finished" podID="a51f2d32-4840-4f0c-804d-c8aa6015ae5a" containerID="89b33df1eaeaee7cb3fac66bfd18e399bff20725374a3c3be1d3d5c87560d176" exitCode=0 Jul 16 00:40:31.661849 ip-10-0-143-200 kubenswrapper[2572]: I0716 00:40:31.661820 2572 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-multus/multus-additional-cni-plugins-vfntt" event={"ID":"a51f2d32-4840-4f0c-804d-c8aa6015ae5a","Type":"ContainerDied","Data":"89b33df1eaeaee7cb3fac66bfd18e399bff20725374a3c3be1d3d5c87560d176"} Jul 16 00:40:31.662863 ip-10-0-143-200 kubenswrapper[2572]: I0716 00:40:31.662844 2572 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-service-ca-operator/service-ca-operator-7695fc6c8c-7v8cr" event={"ID":"1c3985e6-ff56-469e-bdd6-809bf98870ab","Type":"ContainerStarted","Data":"b5786a31193a24340fd99ad9e2647db4001e719743834e7e77e6745e3880a197"} Jul 16 00:40:31.664274 ip-10-0-143-200 kubenswrapper[2572]: I0716 00:40:31.664252 2572 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress-canary/ingress-canary-hhtd6" event={"ID":"6f7eb059-755b-47de-82ab-bfaa72b8f899","Type":"ContainerStarted","Data":"1daeff1de9377a91b431a853401bd1465540f38c62d1f3d0019d80536f6e9710"} Jul 16 00:40:31.665363 ip-10-0-143-200 kubenswrapper[2572]: I0716 00:40:31.665340 2572 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-kube-storage-version-migrator-operator/kube-storage-version-migrator-operator-799cb7f958-tksqh" event={"ID":"07e0ab9b-1bfe-4335-bac1-8d9d158b0085","Type":"ContainerStarted","Data":"17866f023566b32618f86917b6f4a574f1f8ffb924b353f6c0e850e482518d8e"} Jul 16 00:40:31.666437 ip-10-0-143-200 kubenswrapper[2572]: I0716 00:40:31.666418 2572 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-cluster-storage-operator/volume-data-source-validator-78495597f6-wffwm" event={"ID":"3cb174c0-4582-44f3-879a-23697944bc47","Type":"ContainerStarted","Data":"1ddba95f6f122d692265aa8676197c58ed58d48641f4c1c3a287e6b1031df812"} Jul 16 00:40:31.707725 ip-10-0-143-200 kubenswrapper[2572]: I0716 00:40:31.707697 2572 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"networking-console-plugin-cert\" (UniqueName: \"kubernetes.io/secret/605373ad-28e7-4a57-9143-a698a1042a75-networking-console-plugin-cert\") pod \"networking-console-plugin-6cf856758c-vc87p\" (UID: \"605373ad-28e7-4a57-9143-a698a1042a75\") " pod="openshift-network-console/networking-console-plugin-6cf856758c-vc87p" Jul 16 00:40:31.707865 ip-10-0-143-200 kubenswrapper[2572]: E0716 00:40:31.707845 2572 secret.go:189] Couldn't get secret openshift-network-console/networking-console-plugin-cert: secret "networking-console-plugin-cert" not found Jul 16 00:40:31.707929 ip-10-0-143-200 kubenswrapper[2572]: E0716 00:40:31.707908 2572 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/605373ad-28e7-4a57-9143-a698a1042a75-networking-console-plugin-cert podName:605373ad-28e7-4a57-9143-a698a1042a75 nodeName:}" failed. No retries permitted until 2026-07-16 00:40:32.70789117 +0000 UTC m=+37.816645828 (durationBeforeRetry 1s). Error: MountVolume.SetUp failed for volume "networking-console-plugin-cert" (UniqueName: "kubernetes.io/secret/605373ad-28e7-4a57-9143-a698a1042a75-networking-console-plugin-cert") pod "networking-console-plugin-6cf856758c-vc87p" (UID: "605373ad-28e7-4a57-9143-a698a1042a75") : secret "networking-console-plugin-cert" not found Jul 16 00:40:31.727750 ip-10-0-143-200 kubenswrapper[2572]: I0716 00:40:31.727677 2572 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9gnmq_3a7e030d-fad8-480d-9dc6-fe5057048cf9/konnectivity-agent/0.log" limit=1024 Jul 16 00:40:32.115716 ip-10-0-143-200 kubenswrapper[2572]: I0716 00:40:32.115435 2572 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9gnmq_3a7e030d-fad8-480d-9dc6-fe5057048cf9/konnectivity-agent/0.log" limit=1024 Jul 16 00:40:32.200985 ip-10-0-143-200 kubenswrapper[2572]: I0716 00:40:32.200947 2572 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9gnmq_3a7e030d-fad8-480d-9dc6-fe5057048cf9/konnectivity-agent/0.log" limit=1024 Jul 16 00:40:32.415718 ip-10-0-143-200 kubenswrapper[2572]: I0716 00:40:32.414935 2572 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"registry-tls\" (UniqueName: \"kubernetes.io/projected/f103d7a8-2560-409b-801e-701bcc6835c1-registry-tls\") pod \"image-registry-dd7c989d9-5w25c\" (UID: \"f103d7a8-2560-409b-801e-701bcc6835c1\") " pod="openshift-image-registry/image-registry-dd7c989d9-5w25c" Jul 16 00:40:32.415718 ip-10-0-143-200 kubenswrapper[2572]: E0716 00:40:32.415167 2572 projected.go:266] Couldn't get secret openshift-image-registry/image-registry-tls: secret "image-registry-tls" not found Jul 16 00:40:32.415718 ip-10-0-143-200 kubenswrapper[2572]: E0716 00:40:32.415183 2572 projected.go:196] Error preparing data for projected volume registry-tls for pod openshift-image-registry/image-registry-dd7c989d9-5w25c: secret "image-registry-tls" not found Jul 16 00:40:32.415718 ip-10-0-143-200 kubenswrapper[2572]: E0716 00:40:32.415281 2572 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/projected/f103d7a8-2560-409b-801e-701bcc6835c1-registry-tls podName:f103d7a8-2560-409b-801e-701bcc6835c1 nodeName:}" failed. No retries permitted until 2026-07-16 00:40:34.415262367 +0000 UTC m=+39.524017025 (durationBeforeRetry 2s). Error: MountVolume.SetUp failed for volume "registry-tls" (UniqueName: "kubernetes.io/projected/f103d7a8-2560-409b-801e-701bcc6835c1-registry-tls") pod "image-registry-dd7c989d9-5w25c" (UID: "f103d7a8-2560-409b-801e-701bcc6835c1") : secret "image-registry-tls" not found Jul 16 00:40:32.444187 ip-10-0-143-200 kubenswrapper[2572]: I0716 00:40:32.442427 2572 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-52n54" Jul 16 00:40:32.444187 ip-10-0-143-200 kubenswrapper[2572]: I0716 00:40:32.442903 2572 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-5tphg" Jul 16 00:40:32.444187 ip-10-0-143-200 kubenswrapper[2572]: I0716 00:40:32.443353 2572 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-qcbcd" Jul 16 00:40:32.446674 ip-10-0-143-200 kubenswrapper[2572]: I0716 00:40:32.446048 2572 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-multus\"/\"metrics-daemon-secret\"" Jul 16 00:40:32.446674 ip-10-0-143-200 kubenswrapper[2572]: I0716 00:40:32.446386 2572 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"kube-system\"/\"original-pull-secret\"" Jul 16 00:40:32.447535 ip-10-0-143-200 kubenswrapper[2572]: I0716 00:40:32.447503 2572 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-network-diagnostics\"/\"default-dockercfg-hn9ps\"" Jul 16 00:40:32.447964 ip-10-0-143-200 kubenswrapper[2572]: I0716 00:40:32.447937 2572 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-multus\"/\"metrics-daemon-sa-dockercfg-vmvq6\"" Jul 16 00:40:32.516700 ip-10-0-143-200 kubenswrapper[2572]: I0716 00:40:32.516660 2572 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"samples-operator-tls\" (UniqueName: \"kubernetes.io/secret/5aabb5b2-54bc-4893-91d2-3c0215fabd67-samples-operator-tls\") pod \"cluster-samples-operator-65db858894-zf8lz\" (UID: \"5aabb5b2-54bc-4893-91d2-3c0215fabd67\") " pod="openshift-cluster-samples-operator/cluster-samples-operator-65db858894-zf8lz" Jul 16 00:40:32.517150 ip-10-0-143-200 kubenswrapper[2572]: I0716 00:40:32.516765 2572 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-tls\" (UniqueName: \"kubernetes.io/secret/645fc509-972b-4332-8562-2309cc0eab1b-metrics-tls\") pod \"dns-default-w6cgc\" (UID: \"645fc509-972b-4332-8562-2309cc0eab1b\") " pod="openshift-dns/dns-default-w6cgc" Jul 16 00:40:32.517150 ip-10-0-143-200 kubenswrapper[2572]: I0716 00:40:32.516798 2572 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/b82b08fd-bf48-4ee3-9d58-b648d6cba972-metrics-certs\") pod \"router-default-78f9d48cbb-xwpwf\" (UID: \"b82b08fd-bf48-4ee3-9d58-b648d6cba972\") " pod="openshift-ingress/router-default-78f9d48cbb-xwpwf" Jul 16 00:40:32.517150 ip-10-0-143-200 kubenswrapper[2572]: I0716 00:40:32.516829 2572 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"service-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/b82b08fd-bf48-4ee3-9d58-b648d6cba972-service-ca-bundle\") pod \"router-default-78f9d48cbb-xwpwf\" (UID: \"b82b08fd-bf48-4ee3-9d58-b648d6cba972\") " pod="openshift-ingress/router-default-78f9d48cbb-xwpwf" Jul 16 00:40:32.517150 ip-10-0-143-200 kubenswrapper[2572]: I0716 00:40:32.516881 2572 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"cluster-monitoring-operator-tls\" (UniqueName: \"kubernetes.io/secret/265fe067-cade-414b-be13-cb6dbab18057-cluster-monitoring-operator-tls\") pod \"cluster-monitoring-operator-5f7f494dd4-dxc26\" (UID: \"265fe067-cade-414b-be13-cb6dbab18057\") " pod="openshift-monitoring/cluster-monitoring-operator-5f7f494dd4-dxc26" Jul 16 00:40:32.517150 ip-10-0-143-200 kubenswrapper[2572]: E0716 00:40:32.517023 2572 secret.go:189] Couldn't get secret openshift-monitoring/cluster-monitoring-operator-tls: secret "cluster-monitoring-operator-tls" not found Jul 16 00:40:32.517150 ip-10-0-143-200 kubenswrapper[2572]: E0716 00:40:32.517091 2572 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/265fe067-cade-414b-be13-cb6dbab18057-cluster-monitoring-operator-tls podName:265fe067-cade-414b-be13-cb6dbab18057 nodeName:}" failed. No retries permitted until 2026-07-16 00:40:34.517072678 +0000 UTC m=+39.625827354 (durationBeforeRetry 2s). Error: MountVolume.SetUp failed for volume "cluster-monitoring-operator-tls" (UniqueName: "kubernetes.io/secret/265fe067-cade-414b-be13-cb6dbab18057-cluster-monitoring-operator-tls") pod "cluster-monitoring-operator-5f7f494dd4-dxc26" (UID: "265fe067-cade-414b-be13-cb6dbab18057") : secret "cluster-monitoring-operator-tls" not found Jul 16 00:40:32.517557 ip-10-0-143-200 kubenswrapper[2572]: E0716 00:40:32.517500 2572 secret.go:189] Couldn't get secret openshift-dns/dns-default-metrics-tls: secret "dns-default-metrics-tls" not found Jul 16 00:40:32.517557 ip-10-0-143-200 kubenswrapper[2572]: E0716 00:40:32.517547 2572 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/645fc509-972b-4332-8562-2309cc0eab1b-metrics-tls podName:645fc509-972b-4332-8562-2309cc0eab1b nodeName:}" failed. No retries permitted until 2026-07-16 00:40:34.517532573 +0000 UTC m=+39.626287225 (durationBeforeRetry 2s). Error: MountVolume.SetUp failed for volume "metrics-tls" (UniqueName: "kubernetes.io/secret/645fc509-972b-4332-8562-2309cc0eab1b-metrics-tls") pod "dns-default-w6cgc" (UID: "645fc509-972b-4332-8562-2309cc0eab1b") : secret "dns-default-metrics-tls" not found Jul 16 00:40:32.517701 ip-10-0-143-200 kubenswrapper[2572]: E0716 00:40:32.517576 2572 secret.go:189] Couldn't get secret openshift-cluster-samples-operator/samples-operator-tls: secret "samples-operator-tls" not found Jul 16 00:40:32.517701 ip-10-0-143-200 kubenswrapper[2572]: E0716 00:40:32.517603 2572 secret.go:189] Couldn't get secret openshift-ingress/router-metrics-certs-default: secret "router-metrics-certs-default" not found Jul 16 00:40:32.517701 ip-10-0-143-200 kubenswrapper[2572]: E0716 00:40:32.517619 2572 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/5aabb5b2-54bc-4893-91d2-3c0215fabd67-samples-operator-tls podName:5aabb5b2-54bc-4893-91d2-3c0215fabd67 nodeName:}" failed. No retries permitted until 2026-07-16 00:40:34.51760685 +0000 UTC m=+39.626361518 (durationBeforeRetry 2s). Error: MountVolume.SetUp failed for volume "samples-operator-tls" (UniqueName: "kubernetes.io/secret/5aabb5b2-54bc-4893-91d2-3c0215fabd67-samples-operator-tls") pod "cluster-samples-operator-65db858894-zf8lz" (UID: "5aabb5b2-54bc-4893-91d2-3c0215fabd67") : secret "samples-operator-tls" not found Jul 16 00:40:32.517701 ip-10-0-143-200 kubenswrapper[2572]: E0716 00:40:32.517657 2572 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/b82b08fd-bf48-4ee3-9d58-b648d6cba972-metrics-certs podName:b82b08fd-bf48-4ee3-9d58-b648d6cba972 nodeName:}" failed. No retries permitted until 2026-07-16 00:40:34.51764768 +0000 UTC m=+39.626402345 (durationBeforeRetry 2s). Error: MountVolume.SetUp failed for volume "metrics-certs" (UniqueName: "kubernetes.io/secret/b82b08fd-bf48-4ee3-9d58-b648d6cba972-metrics-certs") pod "router-default-78f9d48cbb-xwpwf" (UID: "b82b08fd-bf48-4ee3-9d58-b648d6cba972") : secret "router-metrics-certs-default" not found Jul 16 00:40:32.517896 ip-10-0-143-200 kubenswrapper[2572]: E0716 00:40:32.517710 2572 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/configmap/b82b08fd-bf48-4ee3-9d58-b648d6cba972-service-ca-bundle podName:b82b08fd-bf48-4ee3-9d58-b648d6cba972 nodeName:}" failed. No retries permitted until 2026-07-16 00:40:34.517698642 +0000 UTC m=+39.626453309 (durationBeforeRetry 2s). Error: MountVolume.SetUp failed for volume "service-ca-bundle" (UniqueName: "kubernetes.io/configmap/b82b08fd-bf48-4ee3-9d58-b648d6cba972-service-ca-bundle") pod "router-default-78f9d48cbb-xwpwf" (UID: "b82b08fd-bf48-4ee3-9d58-b648d6cba972") : configmap references non-existent config key: service-ca.crt Jul 16 00:40:32.680303 ip-10-0-143-200 kubenswrapper[2572]: I0716 00:40:32.680214 2572 generic.go:356] "Generic (PLEG): container finished" podID="a51f2d32-4840-4f0c-804d-c8aa6015ae5a" containerID="04ba7f1f96e47c894a411dde67b08b97b75fdb8a134ba8bbaa052274a1aca3d6" exitCode=0 Jul 16 00:40:32.680303 ip-10-0-143-200 kubenswrapper[2572]: I0716 00:40:32.680286 2572 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-multus/multus-additional-cni-plugins-vfntt" event={"ID":"a51f2d32-4840-4f0c-804d-c8aa6015ae5a","Type":"ContainerDied","Data":"04ba7f1f96e47c894a411dde67b08b97b75fdb8a134ba8bbaa052274a1aca3d6"} Jul 16 00:40:32.718778 ip-10-0-143-200 kubenswrapper[2572]: I0716 00:40:32.718739 2572 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"networking-console-plugin-cert\" (UniqueName: \"kubernetes.io/secret/605373ad-28e7-4a57-9143-a698a1042a75-networking-console-plugin-cert\") pod \"networking-console-plugin-6cf856758c-vc87p\" (UID: \"605373ad-28e7-4a57-9143-a698a1042a75\") " pod="openshift-network-console/networking-console-plugin-6cf856758c-vc87p" Jul 16 00:40:32.718949 ip-10-0-143-200 kubenswrapper[2572]: E0716 00:40:32.718916 2572 secret.go:189] Couldn't get secret openshift-network-console/networking-console-plugin-cert: secret "networking-console-plugin-cert" not found Jul 16 00:40:32.719022 ip-10-0-143-200 kubenswrapper[2572]: E0716 00:40:32.718980 2572 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/605373ad-28e7-4a57-9143-a698a1042a75-networking-console-plugin-cert podName:605373ad-28e7-4a57-9143-a698a1042a75 nodeName:}" failed. No retries permitted until 2026-07-16 00:40:34.718962319 +0000 UTC m=+39.827716988 (durationBeforeRetry 2s). Error: MountVolume.SetUp failed for volume "networking-console-plugin-cert" (UniqueName: "kubernetes.io/secret/605373ad-28e7-4a57-9143-a698a1042a75-networking-console-plugin-cert") pod "networking-console-plugin-6cf856758c-vc87p" (UID: "605373ad-28e7-4a57-9143-a698a1042a75") : secret "networking-console-plugin-cert" not found Jul 16 00:40:33.690199 ip-10-0-143-200 kubenswrapper[2572]: I0716 00:40:33.690045 2572 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-multus/multus-additional-cni-plugins-vfntt" event={"ID":"a51f2d32-4840-4f0c-804d-c8aa6015ae5a","Type":"ContainerStarted","Data":"ba8f858cd3a3e747f8e7372fd1d2cf0a86b9a89859dae4002e73c0c5f876c05f"} Jul 16 00:40:33.693746 ip-10-0-143-200 kubenswrapper[2572]: I0716 00:40:33.693284 2572 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-cluster-storage-operator/volume-data-source-validator-78495597f6-wffwm" event={"ID":"3cb174c0-4582-44f3-879a-23697944bc47","Type":"ContainerStarted","Data":"50bb1d5ef0667b7b0de8ec471dc19118596167e1b90ab458daf7de6f3d7c9539"} Jul 16 00:40:33.720246 ip-10-0-143-200 kubenswrapper[2572]: I0716 00:40:33.720184 2572 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-multus/multus-additional-cni-plugins-vfntt" podStartSLOduration=9.045646248 podStartE2EDuration="38.720163707s" totalImagesPullingTime="29.674517459s" totalInitContainerRuntime="0s" isStatefulPod=true podCreationTimestamp="2026-07-16 00:39:55 +0000 UTC" imagePullSessionsCount=5 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-16 00:40:33.718120578 +0000 UTC m=+38.826875255" watchObservedRunningTime="2026-07-16 00:40:33.720163707 +0000 UTC m=+38.828918384" Jul 16 00:40:33.735922 ip-10-0-143-200 kubenswrapper[2572]: I0716 00:40:33.735559 2572 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-cluster-storage-operator/volume-data-source-validator-78495597f6-wffwm" podStartSLOduration=6.794115623 podStartE2EDuration="8.735539978s" totalImagesPullingTime="1.941424355s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-16 00:40:25 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-16 00:40:33.734798088 +0000 UTC m=+38.843552764" watchObservedRunningTime="2026-07-16 00:40:33.735539978 +0000 UTC m=+38.844294654" Jul 16 00:40:33.851810 ip-10-0-143-200 kubenswrapper[2572]: I0716 00:40:33.851773 2572 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9gnmq_3a7e030d-fad8-480d-9dc6-fe5057048cf9/konnectivity-agent/0.log" limit=1024 Jul 16 00:40:33.923700 ip-10-0-143-200 kubenswrapper[2572]: I0716 00:40:33.923575 2572 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9gnmq_3a7e030d-fad8-480d-9dc6-fe5057048cf9/konnectivity-agent/0.log" limit=1024 Jul 16 00:40:34.441788 ip-10-0-143-200 kubenswrapper[2572]: I0716 00:40:34.441691 2572 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"registry-tls\" (UniqueName: \"kubernetes.io/projected/f103d7a8-2560-409b-801e-701bcc6835c1-registry-tls\") pod \"image-registry-dd7c989d9-5w25c\" (UID: \"f103d7a8-2560-409b-801e-701bcc6835c1\") " pod="openshift-image-registry/image-registry-dd7c989d9-5w25c" Jul 16 00:40:34.441967 ip-10-0-143-200 kubenswrapper[2572]: E0716 00:40:34.441904 2572 projected.go:266] Couldn't get secret openshift-image-registry/image-registry-tls: secret "image-registry-tls" not found Jul 16 00:40:34.441967 ip-10-0-143-200 kubenswrapper[2572]: E0716 00:40:34.441924 2572 projected.go:196] Error preparing data for projected volume registry-tls for pod openshift-image-registry/image-registry-dd7c989d9-5w25c: secret "image-registry-tls" not found Jul 16 00:40:34.442082 ip-10-0-143-200 kubenswrapper[2572]: E0716 00:40:34.442003 2572 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/projected/f103d7a8-2560-409b-801e-701bcc6835c1-registry-tls podName:f103d7a8-2560-409b-801e-701bcc6835c1 nodeName:}" failed. No retries permitted until 2026-07-16 00:40:38.441983846 +0000 UTC m=+43.550738504 (durationBeforeRetry 4s). Error: MountVolume.SetUp failed for volume "registry-tls" (UniqueName: "kubernetes.io/projected/f103d7a8-2560-409b-801e-701bcc6835c1-registry-tls") pod "image-registry-dd7c989d9-5w25c" (UID: "f103d7a8-2560-409b-801e-701bcc6835c1") : secret "image-registry-tls" not found Jul 16 00:40:34.542354 ip-10-0-143-200 kubenswrapper[2572]: I0716 00:40:34.542318 2572 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"samples-operator-tls\" (UniqueName: \"kubernetes.io/secret/5aabb5b2-54bc-4893-91d2-3c0215fabd67-samples-operator-tls\") pod \"cluster-samples-operator-65db858894-zf8lz\" (UID: \"5aabb5b2-54bc-4893-91d2-3c0215fabd67\") " pod="openshift-cluster-samples-operator/cluster-samples-operator-65db858894-zf8lz" Jul 16 00:40:34.542542 ip-10-0-143-200 kubenswrapper[2572]: E0716 00:40:34.542465 2572 secret.go:189] Couldn't get secret openshift-cluster-samples-operator/samples-operator-tls: secret "samples-operator-tls" not found Jul 16 00:40:34.542542 ip-10-0-143-200 kubenswrapper[2572]: I0716 00:40:34.542489 2572 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-tls\" (UniqueName: \"kubernetes.io/secret/645fc509-972b-4332-8562-2309cc0eab1b-metrics-tls\") pod \"dns-default-w6cgc\" (UID: \"645fc509-972b-4332-8562-2309cc0eab1b\") " pod="openshift-dns/dns-default-w6cgc" Jul 16 00:40:34.542542 ip-10-0-143-200 kubenswrapper[2572]: E0716 00:40:34.542532 2572 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/5aabb5b2-54bc-4893-91d2-3c0215fabd67-samples-operator-tls podName:5aabb5b2-54bc-4893-91d2-3c0215fabd67 nodeName:}" failed. No retries permitted until 2026-07-16 00:40:38.542513505 +0000 UTC m=+43.651268177 (durationBeforeRetry 4s). Error: MountVolume.SetUp failed for volume "samples-operator-tls" (UniqueName: "kubernetes.io/secret/5aabb5b2-54bc-4893-91d2-3c0215fabd67-samples-operator-tls") pod "cluster-samples-operator-65db858894-zf8lz" (UID: "5aabb5b2-54bc-4893-91d2-3c0215fabd67") : secret "samples-operator-tls" not found Jul 16 00:40:34.542731 ip-10-0-143-200 kubenswrapper[2572]: E0716 00:40:34.542574 2572 secret.go:189] Couldn't get secret openshift-dns/dns-default-metrics-tls: secret "dns-default-metrics-tls" not found Jul 16 00:40:34.542731 ip-10-0-143-200 kubenswrapper[2572]: E0716 00:40:34.542641 2572 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/645fc509-972b-4332-8562-2309cc0eab1b-metrics-tls podName:645fc509-972b-4332-8562-2309cc0eab1b nodeName:}" failed. No retries permitted until 2026-07-16 00:40:38.542616205 +0000 UTC m=+43.651370874 (durationBeforeRetry 4s). Error: MountVolume.SetUp failed for volume "metrics-tls" (UniqueName: "kubernetes.io/secret/645fc509-972b-4332-8562-2309cc0eab1b-metrics-tls") pod "dns-default-w6cgc" (UID: "645fc509-972b-4332-8562-2309cc0eab1b") : secret "dns-default-metrics-tls" not found Jul 16 00:40:34.542731 ip-10-0-143-200 kubenswrapper[2572]: E0716 00:40:34.542641 2572 secret.go:189] Couldn't get secret openshift-ingress/router-metrics-certs-default: secret "router-metrics-certs-default" not found Jul 16 00:40:34.542731 ip-10-0-143-200 kubenswrapper[2572]: I0716 00:40:34.542571 2572 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/b82b08fd-bf48-4ee3-9d58-b648d6cba972-metrics-certs\") pod \"router-default-78f9d48cbb-xwpwf\" (UID: \"b82b08fd-bf48-4ee3-9d58-b648d6cba972\") " pod="openshift-ingress/router-default-78f9d48cbb-xwpwf" Jul 16 00:40:34.542731 ip-10-0-143-200 kubenswrapper[2572]: E0716 00:40:34.542679 2572 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/b82b08fd-bf48-4ee3-9d58-b648d6cba972-metrics-certs podName:b82b08fd-bf48-4ee3-9d58-b648d6cba972 nodeName:}" failed. No retries permitted until 2026-07-16 00:40:38.542671237 +0000 UTC m=+43.651425890 (durationBeforeRetry 4s). Error: MountVolume.SetUp failed for volume "metrics-certs" (UniqueName: "kubernetes.io/secret/b82b08fd-bf48-4ee3-9d58-b648d6cba972-metrics-certs") pod "router-default-78f9d48cbb-xwpwf" (UID: "b82b08fd-bf48-4ee3-9d58-b648d6cba972") : secret "router-metrics-certs-default" not found Jul 16 00:40:34.542731 ip-10-0-143-200 kubenswrapper[2572]: I0716 00:40:34.542695 2572 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"service-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/b82b08fd-bf48-4ee3-9d58-b648d6cba972-service-ca-bundle\") pod \"router-default-78f9d48cbb-xwpwf\" (UID: \"b82b08fd-bf48-4ee3-9d58-b648d6cba972\") " pod="openshift-ingress/router-default-78f9d48cbb-xwpwf" Jul 16 00:40:34.542980 ip-10-0-143-200 kubenswrapper[2572]: I0716 00:40:34.542743 2572 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"cluster-monitoring-operator-tls\" (UniqueName: \"kubernetes.io/secret/265fe067-cade-414b-be13-cb6dbab18057-cluster-monitoring-operator-tls\") pod \"cluster-monitoring-operator-5f7f494dd4-dxc26\" (UID: \"265fe067-cade-414b-be13-cb6dbab18057\") " pod="openshift-monitoring/cluster-monitoring-operator-5f7f494dd4-dxc26" Jul 16 00:40:34.542980 ip-10-0-143-200 kubenswrapper[2572]: E0716 00:40:34.542800 2572 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/configmap/b82b08fd-bf48-4ee3-9d58-b648d6cba972-service-ca-bundle podName:b82b08fd-bf48-4ee3-9d58-b648d6cba972 nodeName:}" failed. No retries permitted until 2026-07-16 00:40:38.542792846 +0000 UTC m=+43.651547499 (durationBeforeRetry 4s). Error: MountVolume.SetUp failed for volume "service-ca-bundle" (UniqueName: "kubernetes.io/configmap/b82b08fd-bf48-4ee3-9d58-b648d6cba972-service-ca-bundle") pod "router-default-78f9d48cbb-xwpwf" (UID: "b82b08fd-bf48-4ee3-9d58-b648d6cba972") : configmap references non-existent config key: service-ca.crt Jul 16 00:40:34.542980 ip-10-0-143-200 kubenswrapper[2572]: E0716 00:40:34.542854 2572 secret.go:189] Couldn't get secret openshift-monitoring/cluster-monitoring-operator-tls: secret "cluster-monitoring-operator-tls" not found Jul 16 00:40:34.542980 ip-10-0-143-200 kubenswrapper[2572]: E0716 00:40:34.542900 2572 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/265fe067-cade-414b-be13-cb6dbab18057-cluster-monitoring-operator-tls podName:265fe067-cade-414b-be13-cb6dbab18057 nodeName:}" failed. No retries permitted until 2026-07-16 00:40:38.54288937 +0000 UTC m=+43.651644023 (durationBeforeRetry 4s). Error: MountVolume.SetUp failed for volume "cluster-monitoring-operator-tls" (UniqueName: "kubernetes.io/secret/265fe067-cade-414b-be13-cb6dbab18057-cluster-monitoring-operator-tls") pod "cluster-monitoring-operator-5f7f494dd4-dxc26" (UID: "265fe067-cade-414b-be13-cb6dbab18057") : secret "cluster-monitoring-operator-tls" not found Jul 16 00:40:34.705361 ip-10-0-143-200 kubenswrapper[2572]: I0716 00:40:34.705319 2572 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9gnmq_3a7e030d-fad8-480d-9dc6-fe5057048cf9/konnectivity-agent/0.log" limit=1024 Jul 16 00:40:34.744567 ip-10-0-143-200 kubenswrapper[2572]: I0716 00:40:34.744526 2572 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"networking-console-plugin-cert\" (UniqueName: \"kubernetes.io/secret/605373ad-28e7-4a57-9143-a698a1042a75-networking-console-plugin-cert\") pod \"networking-console-plugin-6cf856758c-vc87p\" (UID: \"605373ad-28e7-4a57-9143-a698a1042a75\") " pod="openshift-network-console/networking-console-plugin-6cf856758c-vc87p" Jul 16 00:40:34.744765 ip-10-0-143-200 kubenswrapper[2572]: E0716 00:40:34.744666 2572 secret.go:189] Couldn't get secret openshift-network-console/networking-console-plugin-cert: secret "networking-console-plugin-cert" not found Jul 16 00:40:34.744765 ip-10-0-143-200 kubenswrapper[2572]: E0716 00:40:34.744735 2572 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/605373ad-28e7-4a57-9143-a698a1042a75-networking-console-plugin-cert podName:605373ad-28e7-4a57-9143-a698a1042a75 nodeName:}" failed. No retries permitted until 2026-07-16 00:40:38.744715981 +0000 UTC m=+43.853470654 (durationBeforeRetry 4s). Error: MountVolume.SetUp failed for volume "networking-console-plugin-cert" (UniqueName: "kubernetes.io/secret/605373ad-28e7-4a57-9143-a698a1042a75-networking-console-plugin-cert") pod "networking-console-plugin-6cf856758c-vc87p" (UID: "605373ad-28e7-4a57-9143-a698a1042a75") : secret "networking-console-plugin-cert" not found Jul 16 00:40:34.805896 ip-10-0-143-200 kubenswrapper[2572]: I0716 00:40:34.805858 2572 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9gnmq_3a7e030d-fad8-480d-9dc6-fe5057048cf9/konnectivity-agent/0.log" limit=1024 Jul 16 00:40:36.050396 ip-10-0-143-200 kubenswrapper[2572]: I0716 00:40:36.050298 2572 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9gnmq_3a7e030d-fad8-480d-9dc6-fe5057048cf9/konnectivity-agent/0.log" limit=1024 Jul 16 00:40:36.150621 ip-10-0-143-200 kubenswrapper[2572]: I0716 00:40:36.150574 2572 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9gnmq_3a7e030d-fad8-480d-9dc6-fe5057048cf9/konnectivity-agent/0.log" limit=1024 Jul 16 00:40:38.484548 ip-10-0-143-200 kubenswrapper[2572]: I0716 00:40:38.484509 2572 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"registry-tls\" (UniqueName: \"kubernetes.io/projected/f103d7a8-2560-409b-801e-701bcc6835c1-registry-tls\") pod \"image-registry-dd7c989d9-5w25c\" (UID: \"f103d7a8-2560-409b-801e-701bcc6835c1\") " pod="openshift-image-registry/image-registry-dd7c989d9-5w25c" Jul 16 00:40:38.484986 ip-10-0-143-200 kubenswrapper[2572]: E0716 00:40:38.484715 2572 projected.go:266] Couldn't get secret openshift-image-registry/image-registry-tls: secret "image-registry-tls" not found Jul 16 00:40:38.484986 ip-10-0-143-200 kubenswrapper[2572]: E0716 00:40:38.484741 2572 projected.go:196] Error preparing data for projected volume registry-tls for pod openshift-image-registry/image-registry-dd7c989d9-5w25c: secret "image-registry-tls" not found Jul 16 00:40:38.484986 ip-10-0-143-200 kubenswrapper[2572]: E0716 00:40:38.484815 2572 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/projected/f103d7a8-2560-409b-801e-701bcc6835c1-registry-tls podName:f103d7a8-2560-409b-801e-701bcc6835c1 nodeName:}" failed. No retries permitted until 2026-07-16 00:40:46.484792229 +0000 UTC m=+51.593546903 (durationBeforeRetry 8s). Error: MountVolume.SetUp failed for volume "registry-tls" (UniqueName: "kubernetes.io/projected/f103d7a8-2560-409b-801e-701bcc6835c1-registry-tls") pod "image-registry-dd7c989d9-5w25c" (UID: "f103d7a8-2560-409b-801e-701bcc6835c1") : secret "image-registry-tls" not found Jul 16 00:40:38.585302 ip-10-0-143-200 kubenswrapper[2572]: I0716 00:40:38.585262 2572 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"samples-operator-tls\" (UniqueName: \"kubernetes.io/secret/5aabb5b2-54bc-4893-91d2-3c0215fabd67-samples-operator-tls\") pod \"cluster-samples-operator-65db858894-zf8lz\" (UID: \"5aabb5b2-54bc-4893-91d2-3c0215fabd67\") " pod="openshift-cluster-samples-operator/cluster-samples-operator-65db858894-zf8lz" Jul 16 00:40:38.585481 ip-10-0-143-200 kubenswrapper[2572]: I0716 00:40:38.585361 2572 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-tls\" (UniqueName: \"kubernetes.io/secret/645fc509-972b-4332-8562-2309cc0eab1b-metrics-tls\") pod \"dns-default-w6cgc\" (UID: \"645fc509-972b-4332-8562-2309cc0eab1b\") " pod="openshift-dns/dns-default-w6cgc" Jul 16 00:40:38.585481 ip-10-0-143-200 kubenswrapper[2572]: I0716 00:40:38.585395 2572 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/b82b08fd-bf48-4ee3-9d58-b648d6cba972-metrics-certs\") pod \"router-default-78f9d48cbb-xwpwf\" (UID: \"b82b08fd-bf48-4ee3-9d58-b648d6cba972\") " pod="openshift-ingress/router-default-78f9d48cbb-xwpwf" Jul 16 00:40:38.585481 ip-10-0-143-200 kubenswrapper[2572]: I0716 00:40:38.585426 2572 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"service-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/b82b08fd-bf48-4ee3-9d58-b648d6cba972-service-ca-bundle\") pod \"router-default-78f9d48cbb-xwpwf\" (UID: \"b82b08fd-bf48-4ee3-9d58-b648d6cba972\") " pod="openshift-ingress/router-default-78f9d48cbb-xwpwf" Jul 16 00:40:38.585481 ip-10-0-143-200 kubenswrapper[2572]: E0716 00:40:38.585444 2572 secret.go:189] Couldn't get secret openshift-cluster-samples-operator/samples-operator-tls: secret "samples-operator-tls" not found Jul 16 00:40:38.585481 ip-10-0-143-200 kubenswrapper[2572]: I0716 00:40:38.585475 2572 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"cluster-monitoring-operator-tls\" (UniqueName: \"kubernetes.io/secret/265fe067-cade-414b-be13-cb6dbab18057-cluster-monitoring-operator-tls\") pod \"cluster-monitoring-operator-5f7f494dd4-dxc26\" (UID: \"265fe067-cade-414b-be13-cb6dbab18057\") " pod="openshift-monitoring/cluster-monitoring-operator-5f7f494dd4-dxc26" Jul 16 00:40:38.585689 ip-10-0-143-200 kubenswrapper[2572]: E0716 00:40:38.585477 2572 secret.go:189] Couldn't get secret openshift-dns/dns-default-metrics-tls: secret "dns-default-metrics-tls" not found Jul 16 00:40:38.585689 ip-10-0-143-200 kubenswrapper[2572]: E0716 00:40:38.585509 2572 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/5aabb5b2-54bc-4893-91d2-3c0215fabd67-samples-operator-tls podName:5aabb5b2-54bc-4893-91d2-3c0215fabd67 nodeName:}" failed. No retries permitted until 2026-07-16 00:40:46.585492572 +0000 UTC m=+51.694247225 (durationBeforeRetry 8s). Error: MountVolume.SetUp failed for volume "samples-operator-tls" (UniqueName: "kubernetes.io/secret/5aabb5b2-54bc-4893-91d2-3c0215fabd67-samples-operator-tls") pod "cluster-samples-operator-65db858894-zf8lz" (UID: "5aabb5b2-54bc-4893-91d2-3c0215fabd67") : secret "samples-operator-tls" not found Jul 16 00:40:38.585689 ip-10-0-143-200 kubenswrapper[2572]: E0716 00:40:38.585532 2572 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/645fc509-972b-4332-8562-2309cc0eab1b-metrics-tls podName:645fc509-972b-4332-8562-2309cc0eab1b nodeName:}" failed. No retries permitted until 2026-07-16 00:40:46.58552076 +0000 UTC m=+51.694275418 (durationBeforeRetry 8s). Error: MountVolume.SetUp failed for volume "metrics-tls" (UniqueName: "kubernetes.io/secret/645fc509-972b-4332-8562-2309cc0eab1b-metrics-tls") pod "dns-default-w6cgc" (UID: "645fc509-972b-4332-8562-2309cc0eab1b") : secret "dns-default-metrics-tls" not found Jul 16 00:40:38.585689 ip-10-0-143-200 kubenswrapper[2572]: E0716 00:40:38.585554 2572 secret.go:189] Couldn't get secret openshift-ingress/router-metrics-certs-default: secret "router-metrics-certs-default" not found Jul 16 00:40:38.585689 ip-10-0-143-200 kubenswrapper[2572]: E0716 00:40:38.585588 2572 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/configmap/b82b08fd-bf48-4ee3-9d58-b648d6cba972-service-ca-bundle podName:b82b08fd-bf48-4ee3-9d58-b648d6cba972 nodeName:}" failed. No retries permitted until 2026-07-16 00:40:46.58557157 +0000 UTC m=+51.694326235 (durationBeforeRetry 8s). Error: MountVolume.SetUp failed for volume "service-ca-bundle" (UniqueName: "kubernetes.io/configmap/b82b08fd-bf48-4ee3-9d58-b648d6cba972-service-ca-bundle") pod "router-default-78f9d48cbb-xwpwf" (UID: "b82b08fd-bf48-4ee3-9d58-b648d6cba972") : configmap references non-existent config key: service-ca.crt Jul 16 00:40:38.585689 ip-10-0-143-200 kubenswrapper[2572]: E0716 00:40:38.585604 2572 secret.go:189] Couldn't get secret openshift-monitoring/cluster-monitoring-operator-tls: secret "cluster-monitoring-operator-tls" not found Jul 16 00:40:38.585689 ip-10-0-143-200 kubenswrapper[2572]: E0716 00:40:38.585614 2572 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/b82b08fd-bf48-4ee3-9d58-b648d6cba972-metrics-certs podName:b82b08fd-bf48-4ee3-9d58-b648d6cba972 nodeName:}" failed. No retries permitted until 2026-07-16 00:40:46.585603981 +0000 UTC m=+51.694358634 (durationBeforeRetry 8s). Error: MountVolume.SetUp failed for volume "metrics-certs" (UniqueName: "kubernetes.io/secret/b82b08fd-bf48-4ee3-9d58-b648d6cba972-metrics-certs") pod "router-default-78f9d48cbb-xwpwf" (UID: "b82b08fd-bf48-4ee3-9d58-b648d6cba972") : secret "router-metrics-certs-default" not found Jul 16 00:40:38.585689 ip-10-0-143-200 kubenswrapper[2572]: E0716 00:40:38.585658 2572 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/265fe067-cade-414b-be13-cb6dbab18057-cluster-monitoring-operator-tls podName:265fe067-cade-414b-be13-cb6dbab18057 nodeName:}" failed. No retries permitted until 2026-07-16 00:40:46.585643761 +0000 UTC m=+51.694398416 (durationBeforeRetry 8s). Error: MountVolume.SetUp failed for volume "cluster-monitoring-operator-tls" (UniqueName: "kubernetes.io/secret/265fe067-cade-414b-be13-cb6dbab18057-cluster-monitoring-operator-tls") pod "cluster-monitoring-operator-5f7f494dd4-dxc26" (UID: "265fe067-cade-414b-be13-cb6dbab18057") : secret "cluster-monitoring-operator-tls" not found Jul 16 00:40:38.786771 ip-10-0-143-200 kubenswrapper[2572]: I0716 00:40:38.786690 2572 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"networking-console-plugin-cert\" (UniqueName: \"kubernetes.io/secret/605373ad-28e7-4a57-9143-a698a1042a75-networking-console-plugin-cert\") pod \"networking-console-plugin-6cf856758c-vc87p\" (UID: \"605373ad-28e7-4a57-9143-a698a1042a75\") " pod="openshift-network-console/networking-console-plugin-6cf856758c-vc87p" Jul 16 00:40:38.786913 ip-10-0-143-200 kubenswrapper[2572]: E0716 00:40:38.786862 2572 secret.go:189] Couldn't get secret openshift-network-console/networking-console-plugin-cert: secret "networking-console-plugin-cert" not found Jul 16 00:40:38.786952 ip-10-0-143-200 kubenswrapper[2572]: E0716 00:40:38.786932 2572 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/605373ad-28e7-4a57-9143-a698a1042a75-networking-console-plugin-cert podName:605373ad-28e7-4a57-9143-a698a1042a75 nodeName:}" failed. No retries permitted until 2026-07-16 00:40:46.786916012 +0000 UTC m=+51.895670665 (durationBeforeRetry 8s). Error: MountVolume.SetUp failed for volume "networking-console-plugin-cert" (UniqueName: "kubernetes.io/secret/605373ad-28e7-4a57-9143-a698a1042a75-networking-console-plugin-cert") pod "networking-console-plugin-6cf856758c-vc87p" (UID: "605373ad-28e7-4a57-9143-a698a1042a75") : secret "networking-console-plugin-cert" not found Jul 16 00:40:39.714267 ip-10-0-143-200 kubenswrapper[2572]: I0716 00:40:39.714224 2572 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-service-ca-operator/service-ca-operator-7695fc6c8c-7v8cr" event={"ID":"1c3985e6-ff56-469e-bdd6-809bf98870ab","Type":"ContainerStarted","Data":"c6b18d98eda073910a1c9ed20d30a469180128d82ef566e74c0fe377242dd123"} Jul 16 00:40:39.717027 ip-10-0-143-200 kubenswrapper[2572]: I0716 00:40:39.716997 2572 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress-canary/ingress-canary-hhtd6" event={"ID":"6f7eb059-755b-47de-82ab-bfaa72b8f899","Type":"ContainerStarted","Data":"a1a9a26790697d3c9b3984f46d3146c9fe689775ddd87e2a42315201ed068665"} Jul 16 00:40:39.719468 ip-10-0-143-200 kubenswrapper[2572]: I0716 00:40:39.718763 2572 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-kube-storage-version-migrator-operator/kube-storage-version-migrator-operator-799cb7f958-tksqh" event={"ID":"07e0ab9b-1bfe-4335-bac1-8d9d158b0085","Type":"ContainerStarted","Data":"5c7cb475bb36e6926df0a29350ca8fd94ac35b5568f1410195dff810ee0e5dcc"} Jul 16 00:40:39.720757 ip-10-0-143-200 kubenswrapper[2572]: I0716 00:40:39.720710 2572 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-console-operator_console-operator-7866497494-pjc6j_a64bcd05-8596-4c12-b88e-e974d676e25f/console-operator/0.log" Jul 16 00:40:39.721030 ip-10-0-143-200 kubenswrapper[2572]: I0716 00:40:39.720953 2572 generic.go:356] "Generic (PLEG): container finished" podID="a64bcd05-8596-4c12-b88e-e974d676e25f" containerID="7a889982c842d53db1cee6288a23d8cd7d51478aa184c8c3b266c3e7d602ff96" exitCode=255 Jul 16 00:40:39.721030 ip-10-0-143-200 kubenswrapper[2572]: I0716 00:40:39.720995 2572 scope.go:117] "RemoveContainer" containerID="7a889982c842d53db1cee6288a23d8cd7d51478aa184c8c3b266c3e7d602ff96" Jul 16 00:40:39.721166 ip-10-0-143-200 kubenswrapper[2572]: I0716 00:40:39.720953 2572 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-console-operator/console-operator-7866497494-pjc6j" event={"ID":"a64bcd05-8596-4c12-b88e-e974d676e25f","Type":"ContainerDied","Data":"7a889982c842d53db1cee6288a23d8cd7d51478aa184c8c3b266c3e7d602ff96"} Jul 16 00:40:39.723253 ip-10-0-143-200 kubenswrapper[2572]: I0716 00:40:39.723229 2572 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-network-diagnostics/network-check-source-869dcfcdcd-r6wkz" event={"ID":"9da33896-31b6-4a05-bc7c-b9260876e581","Type":"ContainerStarted","Data":"edd3fef21f8f92dd648dd0ce4236e6a9d4c4b574bab11a13c94c23733b357570"} Jul 16 00:40:39.725188 ip-10-0-143-200 kubenswrapper[2572]: I0716 00:40:39.725163 2572 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-insights/insights-operator-68d9dd94cf-d4nrt" event={"ID":"0563669f-9ac5-43d0-a3c0-47c36275ee46","Type":"ContainerStarted","Data":"7b90d55e500c9c647f94e7dcee88d09ccfdeb84311ec13c717021fe67c88bb96"} Jul 16 00:40:39.741477 ip-10-0-143-200 kubenswrapper[2572]: I0716 00:40:39.741418 2572 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-service-ca-operator/service-ca-operator-7695fc6c8c-7v8cr" podStartSLOduration=6.917062519 podStartE2EDuration="14.741399918s" totalImagesPullingTime="7.824337399s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-16 00:40:25 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-16 00:40:39.739824625 +0000 UTC m=+44.848579379" watchObservedRunningTime="2026-07-16 00:40:39.741399918 +0000 UTC m=+44.850154594" Jul 16 00:40:39.822232 ip-10-0-143-200 kubenswrapper[2572]: I0716 00:40:39.822172 2572 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-kube-storage-version-migrator-operator/kube-storage-version-migrator-operator-799cb7f958-tksqh" podStartSLOduration=6.76577806 podStartE2EDuration="14.822155471s" totalImagesPullingTime="8.056377411s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-16 00:40:25 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-16 00:40:39.821427183 +0000 UTC m=+44.930181862" watchObservedRunningTime="2026-07-16 00:40:39.822155471 +0000 UTC m=+44.930910146" Jul 16 00:40:39.886866 ip-10-0-143-200 kubenswrapper[2572]: I0716 00:40:39.886810 2572 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-network-diagnostics/network-check-source-869dcfcdcd-r6wkz" podStartSLOduration=7.067800399 podStartE2EDuration="14.88678734s" totalImagesPullingTime="7.818986941s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-16 00:40:25 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-16 00:40:39.864497112 +0000 UTC m=+44.973251788" watchObservedRunningTime="2026-07-16 00:40:39.88678734 +0000 UTC m=+44.995542016" Jul 16 00:40:39.887422 ip-10-0-143-200 kubenswrapper[2572]: I0716 00:40:39.887060 2572 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-ingress-canary/ingress-canary-hhtd6" podStartSLOduration=2.066098217 podStartE2EDuration="9.887050837s" totalImagesPullingTime="7.82095262s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-16 00:40:30 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-16 00:40:39.886514162 +0000 UTC m=+44.995268838" watchObservedRunningTime="2026-07-16 00:40:39.887050837 +0000 UTC m=+44.995805514" Jul 16 00:40:39.908884 ip-10-0-143-200 kubenswrapper[2572]: I0716 00:40:39.907955 2572 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-insights/insights-operator-68d9dd94cf-d4nrt" podStartSLOduration=7.087979332 podStartE2EDuration="14.907931692s" totalImagesPullingTime="7.81995236s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-16 00:40:25 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-16 00:40:39.906851118 +0000 UTC m=+45.015605795" watchObservedRunningTime="2026-07-16 00:40:39.907931692 +0000 UTC m=+45.016686368" Jul 16 00:40:40.730588 ip-10-0-143-200 kubenswrapper[2572]: I0716 00:40:40.730546 2572 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-console-operator_console-operator-7866497494-pjc6j_a64bcd05-8596-4c12-b88e-e974d676e25f/console-operator/1.log" Jul 16 00:40:40.731066 ip-10-0-143-200 kubenswrapper[2572]: I0716 00:40:40.731047 2572 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-console-operator_console-operator-7866497494-pjc6j_a64bcd05-8596-4c12-b88e-e974d676e25f/console-operator/0.log" Jul 16 00:40:40.731135 ip-10-0-143-200 kubenswrapper[2572]: I0716 00:40:40.731093 2572 generic.go:356] "Generic (PLEG): container finished" podID="a64bcd05-8596-4c12-b88e-e974d676e25f" containerID="92d5f7f712643075021fd38b8766ed695855f59e655a80cab7eaabd5675d192e" exitCode=255 Jul 16 00:40:40.731213 ip-10-0-143-200 kubenswrapper[2572]: I0716 00:40:40.731195 2572 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-console-operator/console-operator-7866497494-pjc6j" event={"ID":"a64bcd05-8596-4c12-b88e-e974d676e25f","Type":"ContainerDied","Data":"92d5f7f712643075021fd38b8766ed695855f59e655a80cab7eaabd5675d192e"} Jul 16 00:40:40.731268 ip-10-0-143-200 kubenswrapper[2572]: I0716 00:40:40.731253 2572 scope.go:117] "RemoveContainer" containerID="7a889982c842d53db1cee6288a23d8cd7d51478aa184c8c3b266c3e7d602ff96" Jul 16 00:40:40.731565 ip-10-0-143-200 kubenswrapper[2572]: I0716 00:40:40.731544 2572 scope.go:117] "RemoveContainer" containerID="92d5f7f712643075021fd38b8766ed695855f59e655a80cab7eaabd5675d192e" Jul 16 00:40:40.731777 ip-10-0-143-200 kubenswrapper[2572]: E0716 00:40:40.731755 2572 pod_workers.go:1324] "Error syncing pod, skipping" err="failed to \"StartContainer\" for \"console-operator\" with CrashLoopBackOff: \"back-off 10s restarting failed container=console-operator pod=console-operator-7866497494-pjc6j_openshift-console-operator(a64bcd05-8596-4c12-b88e-e974d676e25f)\"" pod="openshift-console-operator/console-operator-7866497494-pjc6j" podUID="a64bcd05-8596-4c12-b88e-e974d676e25f" Jul 16 00:40:40.876765 ip-10-0-143-200 kubenswrapper[2572]: I0716 00:40:40.876721 2572 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9gnmq_3a7e030d-fad8-480d-9dc6-fe5057048cf9/konnectivity-agent/0.log" limit=1024 Jul 16 00:40:40.897677 ip-10-0-143-200 kubenswrapper[2572]: I0716 00:40:40.896917 2572 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-kube-storage-version-migrator/migrator-7f59b54f6c-m7dk5"] Jul 16 00:40:40.929120 ip-10-0-143-200 kubenswrapper[2572]: I0716 00:40:40.929086 2572 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-kube-storage-version-migrator/migrator-7f59b54f6c-m7dk5" Jul 16 00:40:40.933439 ip-10-0-143-200 kubenswrapper[2572]: I0716 00:40:40.933409 2572 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-kube-storage-version-migrator/migrator-7f59b54f6c-m7dk5"] Jul 16 00:40:40.934979 ip-10-0-143-200 kubenswrapper[2572]: I0716 00:40:40.933762 2572 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-kube-storage-version-migrator\"/\"kube-storage-version-migrator-sa-dockercfg-4vjfd\"" Jul 16 00:40:40.934979 ip-10-0-143-200 kubenswrapper[2572]: I0716 00:40:40.933918 2572 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-kube-storage-version-migrator\"/\"openshift-service-ca.crt\"" Jul 16 00:40:40.934979 ip-10-0-143-200 kubenswrapper[2572]: I0716 00:40:40.934051 2572 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-kube-storage-version-migrator\"/\"kube-root-ca.crt\"" Jul 16 00:40:40.983646 ip-10-0-143-200 kubenswrapper[2572]: I0716 00:40:40.983489 2572 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9gnmq_3a7e030d-fad8-480d-9dc6-fe5057048cf9/konnectivity-agent/0.log" limit=1024 Jul 16 00:40:40.996698 ip-10-0-143-200 kubenswrapper[2572]: I0716 00:40:40.996663 2572 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-insights/insights-runtime-extractor-q8gql"] Jul 16 00:40:41.010837 ip-10-0-143-200 kubenswrapper[2572]: I0716 00:40:41.010802 2572 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-k4kjm\" (UniqueName: \"kubernetes.io/projected/dab79fd4-2058-46eb-8555-d178887d1e28-kube-api-access-k4kjm\") pod \"migrator-7f59b54f6c-m7dk5\" (UID: \"dab79fd4-2058-46eb-8555-d178887d1e28\") " pod="openshift-kube-storage-version-migrator/migrator-7f59b54f6c-m7dk5" Jul 16 00:40:41.020966 ip-10-0-143-200 kubenswrapper[2572]: I0716 00:40:41.020933 2572 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-insights/insights-runtime-extractor-q8gql"] Jul 16 00:40:41.021140 ip-10-0-143-200 kubenswrapper[2572]: I0716 00:40:41.021052 2572 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-insights/insights-runtime-extractor-q8gql" Jul 16 00:40:41.023821 ip-10-0-143-200 kubenswrapper[2572]: I0716 00:40:41.023797 2572 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-insights\"/\"insights-runtime-extractor-tls\"" Jul 16 00:40:41.023983 ip-10-0-143-200 kubenswrapper[2572]: I0716 00:40:41.023909 2572 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-insights\"/\"insights-runtime-extractor-sa-dockercfg-6hgb5\"" Jul 16 00:40:41.024102 ip-10-0-143-200 kubenswrapper[2572]: I0716 00:40:41.023925 2572 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-insights\"/\"kube-rbac-proxy\"" Jul 16 00:40:41.076773 ip-10-0-143-200 kubenswrapper[2572]: I0716 00:40:41.076729 2572 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9gnmq_3a7e030d-fad8-480d-9dc6-fe5057048cf9/konnectivity-agent/0.log" limit=1024 Jul 16 00:40:41.082446 ip-10-0-143-200 kubenswrapper[2572]: I0716 00:40:41.082405 2572 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-console-operator/console-operator-7866497494-pjc6j" Jul 16 00:40:41.082595 ip-10-0-143-200 kubenswrapper[2572]: I0716 00:40:41.082470 2572 kubelet.go:2654] "SyncLoop (probe)" probe="liveness" status="unhealthy" pod="openshift-console-operator/console-operator-7866497494-pjc6j" Jul 16 00:40:41.111982 ip-10-0-143-200 kubenswrapper[2572]: I0716 00:40:41.111944 2572 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-k4kjm\" (UniqueName: \"kubernetes.io/projected/dab79fd4-2058-46eb-8555-d178887d1e28-kube-api-access-k4kjm\") pod \"migrator-7f59b54f6c-m7dk5\" (UID: \"dab79fd4-2058-46eb-8555-d178887d1e28\") " pod="openshift-kube-storage-version-migrator/migrator-7f59b54f6c-m7dk5" Jul 16 00:40:41.112143 ip-10-0-143-200 kubenswrapper[2572]: I0716 00:40:41.111992 2572 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"insights-runtime-extractor-tls\" (UniqueName: \"kubernetes.io/secret/5f736cca-f28c-49b2-9dc9-24f0982e2d24-insights-runtime-extractor-tls\") pod \"insights-runtime-extractor-q8gql\" (UID: \"5f736cca-f28c-49b2-9dc9-24f0982e2d24\") " pod="openshift-insights/insights-runtime-extractor-q8gql" Jul 16 00:40:41.112143 ip-10-0-143-200 kubenswrapper[2572]: I0716 00:40:41.112056 2572 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-rbac-proxy-cm\" (UniqueName: \"kubernetes.io/configmap/5f736cca-f28c-49b2-9dc9-24f0982e2d24-kube-rbac-proxy-cm\") pod \"insights-runtime-extractor-q8gql\" (UID: \"5f736cca-f28c-49b2-9dc9-24f0982e2d24\") " pod="openshift-insights/insights-runtime-extractor-q8gql" Jul 16 00:40:41.112143 ip-10-0-143-200 kubenswrapper[2572]: I0716 00:40:41.112073 2572 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-6nxs6\" (UniqueName: \"kubernetes.io/projected/5f736cca-f28c-49b2-9dc9-24f0982e2d24-kube-api-access-6nxs6\") pod \"insights-runtime-extractor-q8gql\" (UID: \"5f736cca-f28c-49b2-9dc9-24f0982e2d24\") " pod="openshift-insights/insights-runtime-extractor-q8gql" Jul 16 00:40:41.112282 ip-10-0-143-200 kubenswrapper[2572]: I0716 00:40:41.112177 2572 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"crio-socket\" (UniqueName: \"kubernetes.io/host-path/5f736cca-f28c-49b2-9dc9-24f0982e2d24-crio-socket\") pod \"insights-runtime-extractor-q8gql\" (UID: \"5f736cca-f28c-49b2-9dc9-24f0982e2d24\") " pod="openshift-insights/insights-runtime-extractor-q8gql" Jul 16 00:40:41.112282 ip-10-0-143-200 kubenswrapper[2572]: I0716 00:40:41.112204 2572 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"data-volume\" (UniqueName: \"kubernetes.io/empty-dir/5f736cca-f28c-49b2-9dc9-24f0982e2d24-data-volume\") pod \"insights-runtime-extractor-q8gql\" (UID: \"5f736cca-f28c-49b2-9dc9-24f0982e2d24\") " pod="openshift-insights/insights-runtime-extractor-q8gql" Jul 16 00:40:41.122586 ip-10-0-143-200 kubenswrapper[2572]: I0716 00:40:41.122554 2572 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-k4kjm\" (UniqueName: \"kubernetes.io/projected/dab79fd4-2058-46eb-8555-d178887d1e28-kube-api-access-k4kjm\") pod \"migrator-7f59b54f6c-m7dk5\" (UID: \"dab79fd4-2058-46eb-8555-d178887d1e28\") " pod="openshift-kube-storage-version-migrator/migrator-7f59b54f6c-m7dk5" Jul 16 00:40:41.213493 ip-10-0-143-200 kubenswrapper[2572]: I0716 00:40:41.213454 2572 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-rbac-proxy-cm\" (UniqueName: \"kubernetes.io/configmap/5f736cca-f28c-49b2-9dc9-24f0982e2d24-kube-rbac-proxy-cm\") pod \"insights-runtime-extractor-q8gql\" (UID: \"5f736cca-f28c-49b2-9dc9-24f0982e2d24\") " pod="openshift-insights/insights-runtime-extractor-q8gql" Jul 16 00:40:41.213493 ip-10-0-143-200 kubenswrapper[2572]: I0716 00:40:41.213498 2572 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-6nxs6\" (UniqueName: \"kubernetes.io/projected/5f736cca-f28c-49b2-9dc9-24f0982e2d24-kube-api-access-6nxs6\") pod \"insights-runtime-extractor-q8gql\" (UID: \"5f736cca-f28c-49b2-9dc9-24f0982e2d24\") " pod="openshift-insights/insights-runtime-extractor-q8gql" Jul 16 00:40:41.213778 ip-10-0-143-200 kubenswrapper[2572]: I0716 00:40:41.213568 2572 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"crio-socket\" (UniqueName: \"kubernetes.io/host-path/5f736cca-f28c-49b2-9dc9-24f0982e2d24-crio-socket\") pod \"insights-runtime-extractor-q8gql\" (UID: \"5f736cca-f28c-49b2-9dc9-24f0982e2d24\") " pod="openshift-insights/insights-runtime-extractor-q8gql" Jul 16 00:40:41.213778 ip-10-0-143-200 kubenswrapper[2572]: I0716 00:40:41.213588 2572 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"data-volume\" (UniqueName: \"kubernetes.io/empty-dir/5f736cca-f28c-49b2-9dc9-24f0982e2d24-data-volume\") pod \"insights-runtime-extractor-q8gql\" (UID: \"5f736cca-f28c-49b2-9dc9-24f0982e2d24\") " pod="openshift-insights/insights-runtime-extractor-q8gql" Jul 16 00:40:41.213778 ip-10-0-143-200 kubenswrapper[2572]: E0716 00:40:41.213598 2572 configmap.go:193] Couldn't get configMap openshift-insights/kube-rbac-proxy: configmap "kube-rbac-proxy" not found Jul 16 00:40:41.213778 ip-10-0-143-200 kubenswrapper[2572]: I0716 00:40:41.213667 2572 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"insights-runtime-extractor-tls\" (UniqueName: \"kubernetes.io/secret/5f736cca-f28c-49b2-9dc9-24f0982e2d24-insights-runtime-extractor-tls\") pod \"insights-runtime-extractor-q8gql\" (UID: \"5f736cca-f28c-49b2-9dc9-24f0982e2d24\") " pod="openshift-insights/insights-runtime-extractor-q8gql" Jul 16 00:40:41.213778 ip-10-0-143-200 kubenswrapper[2572]: E0716 00:40:41.213700 2572 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/configmap/5f736cca-f28c-49b2-9dc9-24f0982e2d24-kube-rbac-proxy-cm podName:5f736cca-f28c-49b2-9dc9-24f0982e2d24 nodeName:}" failed. No retries permitted until 2026-07-16 00:40:41.713677476 +0000 UTC m=+46.822432149 (durationBeforeRetry 500ms). Error: MountVolume.SetUp failed for volume "kube-rbac-proxy-cm" (UniqueName: "kubernetes.io/configmap/5f736cca-f28c-49b2-9dc9-24f0982e2d24-kube-rbac-proxy-cm") pod "insights-runtime-extractor-q8gql" (UID: "5f736cca-f28c-49b2-9dc9-24f0982e2d24") : configmap "kube-rbac-proxy" not found Jul 16 00:40:41.213778 ip-10-0-143-200 kubenswrapper[2572]: I0716 00:40:41.213728 2572 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"crio-socket\" (UniqueName: \"kubernetes.io/host-path/5f736cca-f28c-49b2-9dc9-24f0982e2d24-crio-socket\") pod \"insights-runtime-extractor-q8gql\" (UID: \"5f736cca-f28c-49b2-9dc9-24f0982e2d24\") " pod="openshift-insights/insights-runtime-extractor-q8gql" Jul 16 00:40:41.213778 ip-10-0-143-200 kubenswrapper[2572]: E0716 00:40:41.213743 2572 secret.go:189] Couldn't get secret openshift-insights/insights-runtime-extractor-tls: secret "insights-runtime-extractor-tls" not found Jul 16 00:40:41.214106 ip-10-0-143-200 kubenswrapper[2572]: E0716 00:40:41.213817 2572 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/5f736cca-f28c-49b2-9dc9-24f0982e2d24-insights-runtime-extractor-tls podName:5f736cca-f28c-49b2-9dc9-24f0982e2d24 nodeName:}" failed. No retries permitted until 2026-07-16 00:40:41.713806039 +0000 UTC m=+46.822560696 (durationBeforeRetry 500ms). Error: MountVolume.SetUp failed for volume "insights-runtime-extractor-tls" (UniqueName: "kubernetes.io/secret/5f736cca-f28c-49b2-9dc9-24f0982e2d24-insights-runtime-extractor-tls") pod "insights-runtime-extractor-q8gql" (UID: "5f736cca-f28c-49b2-9dc9-24f0982e2d24") : secret "insights-runtime-extractor-tls" not found Jul 16 00:40:41.214106 ip-10-0-143-200 kubenswrapper[2572]: I0716 00:40:41.214028 2572 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"data-volume\" (UniqueName: \"kubernetes.io/empty-dir/5f736cca-f28c-49b2-9dc9-24f0982e2d24-data-volume\") pod \"insights-runtime-extractor-q8gql\" (UID: \"5f736cca-f28c-49b2-9dc9-24f0982e2d24\") " pod="openshift-insights/insights-runtime-extractor-q8gql" Jul 16 00:40:41.217753 ip-10-0-143-200 kubenswrapper[2572]: I0716 00:40:41.217728 2572 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9gnmq_3a7e030d-fad8-480d-9dc6-fe5057048cf9/konnectivity-agent/0.log" limit=1024 Jul 16 00:40:41.225521 ip-10-0-143-200 kubenswrapper[2572]: I0716 00:40:41.225498 2572 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-6nxs6\" (UniqueName: \"kubernetes.io/projected/5f736cca-f28c-49b2-9dc9-24f0982e2d24-kube-api-access-6nxs6\") pod \"insights-runtime-extractor-q8gql\" (UID: \"5f736cca-f28c-49b2-9dc9-24f0982e2d24\") " pod="openshift-insights/insights-runtime-extractor-q8gql" Jul 16 00:40:41.245215 ip-10-0-143-200 kubenswrapper[2572]: I0716 00:40:41.245135 2572 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-kube-storage-version-migrator/migrator-7f59b54f6c-m7dk5" Jul 16 00:40:41.288711 ip-10-0-143-200 kubenswrapper[2572]: I0716 00:40:41.288669 2572 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9gnmq_3a7e030d-fad8-480d-9dc6-fe5057048cf9/konnectivity-agent/0.log" limit=1024 Jul 16 00:40:41.407767 ip-10-0-143-200 kubenswrapper[2572]: I0716 00:40:41.407734 2572 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-kube-storage-version-migrator/migrator-7f59b54f6c-m7dk5"] Jul 16 00:40:41.410681 ip-10-0-143-200 kubenswrapper[2572]: W0716 00:40:41.410647 2572 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-poddab79fd4_2058_46eb_8555_d178887d1e28.slice/crio-402ff90854699bb86ed05c93ae709057cf532c87da45109f086e01df3bb47352 WatchSource:0}: Error finding container 402ff90854699bb86ed05c93ae709057cf532c87da45109f086e01df3bb47352: Status 404 returned error can't find the container with id 402ff90854699bb86ed05c93ae709057cf532c87da45109f086e01df3bb47352 Jul 16 00:40:41.690985 ip-10-0-143-200 kubenswrapper[2572]: I0716 00:40:41.690897 2572 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9gnmq_3a7e030d-fad8-480d-9dc6-fe5057048cf9/konnectivity-agent/0.log" limit=1024 Jul 16 00:40:41.719591 ip-10-0-143-200 kubenswrapper[2572]: I0716 00:40:41.719561 2572 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"insights-runtime-extractor-tls\" (UniqueName: \"kubernetes.io/secret/5f736cca-f28c-49b2-9dc9-24f0982e2d24-insights-runtime-extractor-tls\") pod \"insights-runtime-extractor-q8gql\" (UID: \"5f736cca-f28c-49b2-9dc9-24f0982e2d24\") " pod="openshift-insights/insights-runtime-extractor-q8gql" Jul 16 00:40:41.719724 ip-10-0-143-200 kubenswrapper[2572]: I0716 00:40:41.719671 2572 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-rbac-proxy-cm\" (UniqueName: \"kubernetes.io/configmap/5f736cca-f28c-49b2-9dc9-24f0982e2d24-kube-rbac-proxy-cm\") pod \"insights-runtime-extractor-q8gql\" (UID: \"5f736cca-f28c-49b2-9dc9-24f0982e2d24\") " pod="openshift-insights/insights-runtime-extractor-q8gql" Jul 16 00:40:41.719781 ip-10-0-143-200 kubenswrapper[2572]: E0716 00:40:41.719741 2572 secret.go:189] Couldn't get secret openshift-insights/insights-runtime-extractor-tls: secret "insights-runtime-extractor-tls" not found Jul 16 00:40:41.719818 ip-10-0-143-200 kubenswrapper[2572]: E0716 00:40:41.719793 2572 configmap.go:193] Couldn't get configMap openshift-insights/kube-rbac-proxy: configmap "kube-rbac-proxy" not found Jul 16 00:40:41.719855 ip-10-0-143-200 kubenswrapper[2572]: E0716 00:40:41.719819 2572 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/5f736cca-f28c-49b2-9dc9-24f0982e2d24-insights-runtime-extractor-tls podName:5f736cca-f28c-49b2-9dc9-24f0982e2d24 nodeName:}" failed. No retries permitted until 2026-07-16 00:40:42.719804223 +0000 UTC m=+47.828558896 (durationBeforeRetry 1s). Error: MountVolume.SetUp failed for volume "insights-runtime-extractor-tls" (UniqueName: "kubernetes.io/secret/5f736cca-f28c-49b2-9dc9-24f0982e2d24-insights-runtime-extractor-tls") pod "insights-runtime-extractor-q8gql" (UID: "5f736cca-f28c-49b2-9dc9-24f0982e2d24") : secret "insights-runtime-extractor-tls" not found Jul 16 00:40:41.719895 ip-10-0-143-200 kubenswrapper[2572]: E0716 00:40:41.719857 2572 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/configmap/5f736cca-f28c-49b2-9dc9-24f0982e2d24-kube-rbac-proxy-cm podName:5f736cca-f28c-49b2-9dc9-24f0982e2d24 nodeName:}" failed. No retries permitted until 2026-07-16 00:40:42.719842241 +0000 UTC m=+47.828596894 (durationBeforeRetry 1s). Error: MountVolume.SetUp failed for volume "kube-rbac-proxy-cm" (UniqueName: "kubernetes.io/configmap/5f736cca-f28c-49b2-9dc9-24f0982e2d24-kube-rbac-proxy-cm") pod "insights-runtime-extractor-q8gql" (UID: "5f736cca-f28c-49b2-9dc9-24f0982e2d24") : configmap "kube-rbac-proxy" not found Jul 16 00:40:41.737539 ip-10-0-143-200 kubenswrapper[2572]: I0716 00:40:41.737513 2572 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-console-operator_console-operator-7866497494-pjc6j_a64bcd05-8596-4c12-b88e-e974d676e25f/console-operator/1.log" Jul 16 00:40:41.737993 ip-10-0-143-200 kubenswrapper[2572]: I0716 00:40:41.737928 2572 scope.go:117] "RemoveContainer" containerID="92d5f7f712643075021fd38b8766ed695855f59e655a80cab7eaabd5675d192e" Jul 16 00:40:41.738133 ip-10-0-143-200 kubenswrapper[2572]: E0716 00:40:41.738110 2572 pod_workers.go:1324] "Error syncing pod, skipping" err="failed to \"StartContainer\" for \"console-operator\" with CrashLoopBackOff: \"back-off 10s restarting failed container=console-operator pod=console-operator-7866497494-pjc6j_openshift-console-operator(a64bcd05-8596-4c12-b88e-e974d676e25f)\"" pod="openshift-console-operator/console-operator-7866497494-pjc6j" podUID="a64bcd05-8596-4c12-b88e-e974d676e25f" Jul 16 00:40:41.738813 ip-10-0-143-200 kubenswrapper[2572]: I0716 00:40:41.738792 2572 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-kube-storage-version-migrator/migrator-7f59b54f6c-m7dk5" event={"ID":"dab79fd4-2058-46eb-8555-d178887d1e28","Type":"ContainerStarted","Data":"402ff90854699bb86ed05c93ae709057cf532c87da45109f086e01df3bb47352"} Jul 16 00:40:41.762278 ip-10-0-143-200 kubenswrapper[2572]: I0716 00:40:41.762241 2572 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9gnmq_3a7e030d-fad8-480d-9dc6-fe5057048cf9/konnectivity-agent/0.log" limit=1024 Jul 16 00:40:41.917569 ip-10-0-143-200 kubenswrapper[2572]: I0716 00:40:41.917516 2572 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9gnmq_3a7e030d-fad8-480d-9dc6-fe5057048cf9/konnectivity-agent/0.log" limit=1024 Jul 16 00:40:42.019189 ip-10-0-143-200 kubenswrapper[2572]: I0716 00:40:42.019157 2572 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9gnmq_3a7e030d-fad8-480d-9dc6-fe5057048cf9/konnectivity-agent/0.log" limit=1024 Jul 16 00:40:42.730540 ip-10-0-143-200 kubenswrapper[2572]: I0716 00:40:42.730498 2572 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-rbac-proxy-cm\" (UniqueName: \"kubernetes.io/configmap/5f736cca-f28c-49b2-9dc9-24f0982e2d24-kube-rbac-proxy-cm\") pod \"insights-runtime-extractor-q8gql\" (UID: \"5f736cca-f28c-49b2-9dc9-24f0982e2d24\") " pod="openshift-insights/insights-runtime-extractor-q8gql" Jul 16 00:40:42.730763 ip-10-0-143-200 kubenswrapper[2572]: E0716 00:40:42.730649 2572 configmap.go:193] Couldn't get configMap openshift-insights/kube-rbac-proxy: configmap "kube-rbac-proxy" not found Jul 16 00:40:42.730763 ip-10-0-143-200 kubenswrapper[2572]: I0716 00:40:42.730685 2572 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"insights-runtime-extractor-tls\" (UniqueName: \"kubernetes.io/secret/5f736cca-f28c-49b2-9dc9-24f0982e2d24-insights-runtime-extractor-tls\") pod \"insights-runtime-extractor-q8gql\" (UID: \"5f736cca-f28c-49b2-9dc9-24f0982e2d24\") " pod="openshift-insights/insights-runtime-extractor-q8gql" Jul 16 00:40:42.730763 ip-10-0-143-200 kubenswrapper[2572]: E0716 00:40:42.730717 2572 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/configmap/5f736cca-f28c-49b2-9dc9-24f0982e2d24-kube-rbac-proxy-cm podName:5f736cca-f28c-49b2-9dc9-24f0982e2d24 nodeName:}" failed. No retries permitted until 2026-07-16 00:40:44.730697527 +0000 UTC m=+49.839452197 (durationBeforeRetry 2s). Error: MountVolume.SetUp failed for volume "kube-rbac-proxy-cm" (UniqueName: "kubernetes.io/configmap/5f736cca-f28c-49b2-9dc9-24f0982e2d24-kube-rbac-proxy-cm") pod "insights-runtime-extractor-q8gql" (UID: "5f736cca-f28c-49b2-9dc9-24f0982e2d24") : configmap "kube-rbac-proxy" not found Jul 16 00:40:42.730916 ip-10-0-143-200 kubenswrapper[2572]: E0716 00:40:42.730767 2572 secret.go:189] Couldn't get secret openshift-insights/insights-runtime-extractor-tls: secret "insights-runtime-extractor-tls" not found Jul 16 00:40:42.730916 ip-10-0-143-200 kubenswrapper[2572]: E0716 00:40:42.730819 2572 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/5f736cca-f28c-49b2-9dc9-24f0982e2d24-insights-runtime-extractor-tls podName:5f736cca-f28c-49b2-9dc9-24f0982e2d24 nodeName:}" failed. No retries permitted until 2026-07-16 00:40:44.730807822 +0000 UTC m=+49.839562489 (durationBeforeRetry 2s). Error: MountVolume.SetUp failed for volume "insights-runtime-extractor-tls" (UniqueName: "kubernetes.io/secret/5f736cca-f28c-49b2-9dc9-24f0982e2d24-insights-runtime-extractor-tls") pod "insights-runtime-extractor-q8gql" (UID: "5f736cca-f28c-49b2-9dc9-24f0982e2d24") : secret "insights-runtime-extractor-tls" not found Jul 16 00:40:42.743700 ip-10-0-143-200 kubenswrapper[2572]: I0716 00:40:42.743675 2572 scope.go:117] "RemoveContainer" containerID="92d5f7f712643075021fd38b8766ed695855f59e655a80cab7eaabd5675d192e" Jul 16 00:40:42.744162 ip-10-0-143-200 kubenswrapper[2572]: E0716 00:40:42.743866 2572 pod_workers.go:1324] "Error syncing pod, skipping" err="failed to \"StartContainer\" for \"console-operator\" with CrashLoopBackOff: \"back-off 10s restarting failed container=console-operator pod=console-operator-7866497494-pjc6j_openshift-console-operator(a64bcd05-8596-4c12-b88e-e974d676e25f)\"" pod="openshift-console-operator/console-operator-7866497494-pjc6j" podUID="a64bcd05-8596-4c12-b88e-e974d676e25f" Jul 16 00:40:42.785121 ip-10-0-143-200 kubenswrapper[2572]: I0716 00:40:42.785096 2572 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-dns_node-resolver-6qlgz_24fd12c1-ddcb-427f-ac7e-4e2877450cb8/dns-node-resolver/0.log" Jul 16 00:40:43.516688 ip-10-0-143-200 kubenswrapper[2572]: I0716 00:40:43.516596 2572 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-service-ca/service-ca-79f68b9f4f-xwtpt"] Jul 16 00:40:43.537321 ip-10-0-143-200 kubenswrapper[2572]: I0716 00:40:43.537285 2572 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-service-ca/service-ca-79f68b9f4f-xwtpt"] Jul 16 00:40:43.537479 ip-10-0-143-200 kubenswrapper[2572]: I0716 00:40:43.537402 2572 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-service-ca/service-ca-79f68b9f4f-xwtpt" Jul 16 00:40:43.540266 ip-10-0-143-200 kubenswrapper[2572]: I0716 00:40:43.540242 2572 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-service-ca\"/\"service-ca-dockercfg-52hb4\"" Jul 16 00:40:43.540393 ip-10-0-143-200 kubenswrapper[2572]: I0716 00:40:43.540245 2572 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-service-ca\"/\"signing-cabundle\"" Jul 16 00:40:43.541865 ip-10-0-143-200 kubenswrapper[2572]: I0716 00:40:43.541822 2572 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-service-ca\"/\"openshift-service-ca.crt\"" Jul 16 00:40:43.541865 ip-10-0-143-200 kubenswrapper[2572]: I0716 00:40:43.541844 2572 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-service-ca\"/\"kube-root-ca.crt\"" Jul 16 00:40:43.542053 ip-10-0-143-200 kubenswrapper[2572]: I0716 00:40:43.541943 2572 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-service-ca\"/\"signing-key\"" Jul 16 00:40:43.562830 ip-10-0-143-200 kubenswrapper[2572]: I0716 00:40:43.562793 2572 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9gnmq_3a7e030d-fad8-480d-9dc6-fe5057048cf9/konnectivity-agent/0.log" limit=1024 Jul 16 00:40:43.640071 ip-10-0-143-200 kubenswrapper[2572]: I0716 00:40:43.640020 2572 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-w89gj\" (UniqueName: \"kubernetes.io/projected/900da9b6-e9f1-42e9-8e42-0f63dee5638c-kube-api-access-w89gj\") pod \"service-ca-79f68b9f4f-xwtpt\" (UID: \"900da9b6-e9f1-42e9-8e42-0f63dee5638c\") " pod="openshift-service-ca/service-ca-79f68b9f4f-xwtpt" Jul 16 00:40:43.640268 ip-10-0-143-200 kubenswrapper[2572]: I0716 00:40:43.640163 2572 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"signing-cabundle\" (UniqueName: \"kubernetes.io/configmap/900da9b6-e9f1-42e9-8e42-0f63dee5638c-signing-cabundle\") pod \"service-ca-79f68b9f4f-xwtpt\" (UID: \"900da9b6-e9f1-42e9-8e42-0f63dee5638c\") " pod="openshift-service-ca/service-ca-79f68b9f4f-xwtpt" Jul 16 00:40:43.640336 ip-10-0-143-200 kubenswrapper[2572]: I0716 00:40:43.640276 2572 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"signing-key\" (UniqueName: \"kubernetes.io/secret/900da9b6-e9f1-42e9-8e42-0f63dee5638c-signing-key\") pod \"service-ca-79f68b9f4f-xwtpt\" (UID: \"900da9b6-e9f1-42e9-8e42-0f63dee5638c\") " pod="openshift-service-ca/service-ca-79f68b9f4f-xwtpt" Jul 16 00:40:43.662682 ip-10-0-143-200 kubenswrapper[2572]: I0716 00:40:43.662651 2572 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9gnmq_3a7e030d-fad8-480d-9dc6-fe5057048cf9/konnectivity-agent/0.log" limit=1024 Jul 16 00:40:43.741451 ip-10-0-143-200 kubenswrapper[2572]: I0716 00:40:43.741411 2572 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"signing-key\" (UniqueName: \"kubernetes.io/secret/900da9b6-e9f1-42e9-8e42-0f63dee5638c-signing-key\") pod \"service-ca-79f68b9f4f-xwtpt\" (UID: \"900da9b6-e9f1-42e9-8e42-0f63dee5638c\") " pod="openshift-service-ca/service-ca-79f68b9f4f-xwtpt" Jul 16 00:40:43.741721 ip-10-0-143-200 kubenswrapper[2572]: I0716 00:40:43.741506 2572 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-w89gj\" (UniqueName: \"kubernetes.io/projected/900da9b6-e9f1-42e9-8e42-0f63dee5638c-kube-api-access-w89gj\") pod \"service-ca-79f68b9f4f-xwtpt\" (UID: \"900da9b6-e9f1-42e9-8e42-0f63dee5638c\") " pod="openshift-service-ca/service-ca-79f68b9f4f-xwtpt" Jul 16 00:40:43.741721 ip-10-0-143-200 kubenswrapper[2572]: I0716 00:40:43.741564 2572 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"signing-cabundle\" (UniqueName: \"kubernetes.io/configmap/900da9b6-e9f1-42e9-8e42-0f63dee5638c-signing-cabundle\") pod \"service-ca-79f68b9f4f-xwtpt\" (UID: \"900da9b6-e9f1-42e9-8e42-0f63dee5638c\") " pod="openshift-service-ca/service-ca-79f68b9f4f-xwtpt" Jul 16 00:40:43.742246 ip-10-0-143-200 kubenswrapper[2572]: I0716 00:40:43.742228 2572 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"signing-cabundle\" (UniqueName: \"kubernetes.io/configmap/900da9b6-e9f1-42e9-8e42-0f63dee5638c-signing-cabundle\") pod \"service-ca-79f68b9f4f-xwtpt\" (UID: \"900da9b6-e9f1-42e9-8e42-0f63dee5638c\") " pod="openshift-service-ca/service-ca-79f68b9f4f-xwtpt" Jul 16 00:40:43.744026 ip-10-0-143-200 kubenswrapper[2572]: I0716 00:40:43.743999 2572 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"signing-key\" (UniqueName: \"kubernetes.io/secret/900da9b6-e9f1-42e9-8e42-0f63dee5638c-signing-key\") pod \"service-ca-79f68b9f4f-xwtpt\" (UID: \"900da9b6-e9f1-42e9-8e42-0f63dee5638c\") " pod="openshift-service-ca/service-ca-79f68b9f4f-xwtpt" Jul 16 00:40:43.749307 ip-10-0-143-200 kubenswrapper[2572]: I0716 00:40:43.749279 2572 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-kube-storage-version-migrator/migrator-7f59b54f6c-m7dk5" event={"ID":"dab79fd4-2058-46eb-8555-d178887d1e28","Type":"ContainerStarted","Data":"32e52a5e22eb0659a353c7f0d6b5b2817acdf97d404f379d35cd5b2775a6328c"} Jul 16 00:40:43.749437 ip-10-0-143-200 kubenswrapper[2572]: I0716 00:40:43.749376 2572 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-kube-storage-version-migrator/migrator-7f59b54f6c-m7dk5" event={"ID":"dab79fd4-2058-46eb-8555-d178887d1e28","Type":"ContainerStarted","Data":"864618d506dae9e8629c3a4f01b27bbce641e6da8c57176cbc443abdd0cf4c66"} Jul 16 00:40:43.750682 ip-10-0-143-200 kubenswrapper[2572]: I0716 00:40:43.750663 2572 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-w89gj\" (UniqueName: \"kubernetes.io/projected/900da9b6-e9f1-42e9-8e42-0f63dee5638c-kube-api-access-w89gj\") pod \"service-ca-79f68b9f4f-xwtpt\" (UID: \"900da9b6-e9f1-42e9-8e42-0f63dee5638c\") " pod="openshift-service-ca/service-ca-79f68b9f4f-xwtpt" Jul 16 00:40:43.767308 ip-10-0-143-200 kubenswrapper[2572]: I0716 00:40:43.767201 2572 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-kube-storage-version-migrator/migrator-7f59b54f6c-m7dk5" podStartSLOduration=1.971602575 podStartE2EDuration="3.76717912s" totalImagesPullingTime="1.795576545s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-16 00:40:40 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-16 00:40:43.767104007 +0000 UTC m=+48.875858684" watchObservedRunningTime="2026-07-16 00:40:43.76717912 +0000 UTC m=+48.875933797" Jul 16 00:40:43.790785 ip-10-0-143-200 kubenswrapper[2572]: I0716 00:40:43.790755 2572 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-image-registry_node-ca-6msnd_ff468ef8-66ba-48f2-8531-da662a5b19d3/node-ca/0.log" Jul 16 00:40:43.848257 ip-10-0-143-200 kubenswrapper[2572]: I0716 00:40:43.848217 2572 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-service-ca/service-ca-79f68b9f4f-xwtpt" Jul 16 00:40:43.865290 ip-10-0-143-200 kubenswrapper[2572]: I0716 00:40:43.865249 2572 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9gnmq_3a7e030d-fad8-480d-9dc6-fe5057048cf9/konnectivity-agent/0.log" limit=1024 Jul 16 00:40:43.951266 ip-10-0-143-200 kubenswrapper[2572]: I0716 00:40:43.951234 2572 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9gnmq_3a7e030d-fad8-480d-9dc6-fe5057048cf9/konnectivity-agent/0.log" limit=1024 Jul 16 00:40:43.999191 ip-10-0-143-200 kubenswrapper[2572]: I0716 00:40:43.999154 2572 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-service-ca/service-ca-79f68b9f4f-xwtpt"] Jul 16 00:40:44.002308 ip-10-0-143-200 kubenswrapper[2572]: W0716 00:40:44.002281 2572 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod900da9b6_e9f1_42e9_8e42_0f63dee5638c.slice/crio-bc282f905b9c6c5b9059125ccf885684df552b4d0d5f7d1e761b65cd6fcd3a93 WatchSource:0}: Error finding container bc282f905b9c6c5b9059125ccf885684df552b4d0d5f7d1e761b65cd6fcd3a93: Status 404 returned error can't find the container with id bc282f905b9c6c5b9059125ccf885684df552b4d0d5f7d1e761b65cd6fcd3a93 Jul 16 00:40:44.045880 ip-10-0-143-200 kubenswrapper[2572]: I0716 00:40:44.045846 2572 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9gnmq_3a7e030d-fad8-480d-9dc6-fe5057048cf9/konnectivity-agent/0.log" limit=1024 Jul 16 00:40:44.543671 ip-10-0-143-200 kubenswrapper[2572]: I0716 00:40:44.543620 2572 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9gnmq_3a7e030d-fad8-480d-9dc6-fe5057048cf9/konnectivity-agent/0.log" limit=1024 Jul 16 00:40:44.585668 ip-10-0-143-200 kubenswrapper[2572]: I0716 00:40:44.585617 2572 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ingress-canary_ingress-canary-hhtd6_6f7eb059-755b-47de-82ab-bfaa72b8f899/serve-healthcheck-canary/0.log" Jul 16 00:40:44.752938 ip-10-0-143-200 kubenswrapper[2572]: I0716 00:40:44.752903 2572 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"insights-runtime-extractor-tls\" (UniqueName: \"kubernetes.io/secret/5f736cca-f28c-49b2-9dc9-24f0982e2d24-insights-runtime-extractor-tls\") pod \"insights-runtime-extractor-q8gql\" (UID: \"5f736cca-f28c-49b2-9dc9-24f0982e2d24\") " pod="openshift-insights/insights-runtime-extractor-q8gql" Jul 16 00:40:44.753313 ip-10-0-143-200 kubenswrapper[2572]: E0716 00:40:44.753069 2572 secret.go:189] Couldn't get secret openshift-insights/insights-runtime-extractor-tls: secret "insights-runtime-extractor-tls" not found Jul 16 00:40:44.753313 ip-10-0-143-200 kubenswrapper[2572]: I0716 00:40:44.753108 2572 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-rbac-proxy-cm\" (UniqueName: \"kubernetes.io/configmap/5f736cca-f28c-49b2-9dc9-24f0982e2d24-kube-rbac-proxy-cm\") pod \"insights-runtime-extractor-q8gql\" (UID: \"5f736cca-f28c-49b2-9dc9-24f0982e2d24\") " pod="openshift-insights/insights-runtime-extractor-q8gql" Jul 16 00:40:44.753313 ip-10-0-143-200 kubenswrapper[2572]: E0716 00:40:44.753140 2572 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/5f736cca-f28c-49b2-9dc9-24f0982e2d24-insights-runtime-extractor-tls podName:5f736cca-f28c-49b2-9dc9-24f0982e2d24 nodeName:}" failed. No retries permitted until 2026-07-16 00:40:48.75311926 +0000 UTC m=+53.861873934 (durationBeforeRetry 4s). Error: MountVolume.SetUp failed for volume "insights-runtime-extractor-tls" (UniqueName: "kubernetes.io/secret/5f736cca-f28c-49b2-9dc9-24f0982e2d24-insights-runtime-extractor-tls") pod "insights-runtime-extractor-q8gql" (UID: "5f736cca-f28c-49b2-9dc9-24f0982e2d24") : secret "insights-runtime-extractor-tls" not found Jul 16 00:40:44.753313 ip-10-0-143-200 kubenswrapper[2572]: E0716 00:40:44.753158 2572 configmap.go:193] Couldn't get configMap openshift-insights/kube-rbac-proxy: configmap "kube-rbac-proxy" not found Jul 16 00:40:44.753313 ip-10-0-143-200 kubenswrapper[2572]: E0716 00:40:44.753203 2572 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/configmap/5f736cca-f28c-49b2-9dc9-24f0982e2d24-kube-rbac-proxy-cm podName:5f736cca-f28c-49b2-9dc9-24f0982e2d24 nodeName:}" failed. No retries permitted until 2026-07-16 00:40:48.753187614 +0000 UTC m=+53.861942269 (durationBeforeRetry 4s). Error: MountVolume.SetUp failed for volume "kube-rbac-proxy-cm" (UniqueName: "kubernetes.io/configmap/5f736cca-f28c-49b2-9dc9-24f0982e2d24-kube-rbac-proxy-cm") pod "insights-runtime-extractor-q8gql" (UID: "5f736cca-f28c-49b2-9dc9-24f0982e2d24") : configmap "kube-rbac-proxy" not found Jul 16 00:40:44.755081 ip-10-0-143-200 kubenswrapper[2572]: I0716 00:40:44.755059 2572 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-service-ca/service-ca-79f68b9f4f-xwtpt" event={"ID":"900da9b6-e9f1-42e9-8e42-0f63dee5638c","Type":"ContainerStarted","Data":"f282edcf2357d25ae5a20dfa1f432493dda3c4081f2bd04bf45580f765b76733"} Jul 16 00:40:44.755196 ip-10-0-143-200 kubenswrapper[2572]: I0716 00:40:44.755105 2572 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-service-ca/service-ca-79f68b9f4f-xwtpt" event={"ID":"900da9b6-e9f1-42e9-8e42-0f63dee5638c","Type":"ContainerStarted","Data":"bc282f905b9c6c5b9059125ccf885684df552b4d0d5f7d1e761b65cd6fcd3a93"} Jul 16 00:40:44.796515 ip-10-0-143-200 kubenswrapper[2572]: I0716 00:40:44.796431 2572 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-kube-storage-version-migrator_migrator-7f59b54f6c-m7dk5_dab79fd4-2058-46eb-8555-d178887d1e28/migrator/0.log" Jul 16 00:40:44.987709 ip-10-0-143-200 kubenswrapper[2572]: I0716 00:40:44.987677 2572 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-kube-storage-version-migrator_migrator-7f59b54f6c-m7dk5_dab79fd4-2058-46eb-8555-d178887d1e28/graceful-termination/0.log" Jul 16 00:40:45.189503 ip-10-0-143-200 kubenswrapper[2572]: I0716 00:40:45.189405 2572 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-kube-storage-version-migrator-operator_kube-storage-version-migrator-operator-799cb7f958-tksqh_07e0ab9b-1bfe-4335-bac1-8d9d158b0085/kube-storage-version-migrator-operator/0.log" Jul 16 00:40:45.830590 ip-10-0-143-200 kubenswrapper[2572]: I0716 00:40:45.830554 2572 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9gnmq_3a7e030d-fad8-480d-9dc6-fe5057048cf9/konnectivity-agent/0.log" limit=1024 Jul 16 00:40:45.948228 ip-10-0-143-200 kubenswrapper[2572]: I0716 00:40:45.948191 2572 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9gnmq_3a7e030d-fad8-480d-9dc6-fe5057048cf9/konnectivity-agent/0.log" limit=1024 Jul 16 00:40:46.104915 ip-10-0-143-200 kubenswrapper[2572]: I0716 00:40:46.104810 2572 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9gnmq_3a7e030d-fad8-480d-9dc6-fe5057048cf9/konnectivity-agent/0.log" limit=1024 Jul 16 00:40:46.198189 ip-10-0-143-200 kubenswrapper[2572]: I0716 00:40:46.198154 2572 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9gnmq_3a7e030d-fad8-480d-9dc6-fe5057048cf9/konnectivity-agent/0.log" limit=1024 Jul 16 00:40:46.301787 ip-10-0-143-200 kubenswrapper[2572]: I0716 00:40:46.301748 2572 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9gnmq_3a7e030d-fad8-480d-9dc6-fe5057048cf9/konnectivity-agent/0.log" limit=1024 Jul 16 00:40:46.521872 ip-10-0-143-200 kubenswrapper[2572]: I0716 00:40:46.521824 2572 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9gnmq_3a7e030d-fad8-480d-9dc6-fe5057048cf9/konnectivity-agent/0.log" limit=1024 Jul 16 00:40:46.569876 ip-10-0-143-200 kubenswrapper[2572]: I0716 00:40:46.569837 2572 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"registry-tls\" (UniqueName: \"kubernetes.io/projected/f103d7a8-2560-409b-801e-701bcc6835c1-registry-tls\") pod \"image-registry-dd7c989d9-5w25c\" (UID: \"f103d7a8-2560-409b-801e-701bcc6835c1\") " pod="openshift-image-registry/image-registry-dd7c989d9-5w25c" Jul 16 00:40:46.570094 ip-10-0-143-200 kubenswrapper[2572]: E0716 00:40:46.570030 2572 projected.go:266] Couldn't get secret openshift-image-registry/image-registry-tls: secret "image-registry-tls" not found Jul 16 00:40:46.570094 ip-10-0-143-200 kubenswrapper[2572]: E0716 00:40:46.570056 2572 projected.go:196] Error preparing data for projected volume registry-tls for pod openshift-image-registry/image-registry-dd7c989d9-5w25c: secret "image-registry-tls" not found Jul 16 00:40:46.570203 ip-10-0-143-200 kubenswrapper[2572]: E0716 00:40:46.570128 2572 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/projected/f103d7a8-2560-409b-801e-701bcc6835c1-registry-tls podName:f103d7a8-2560-409b-801e-701bcc6835c1 nodeName:}" failed. No retries permitted until 2026-07-16 00:41:02.570107345 +0000 UTC m=+67.678862025 (durationBeforeRetry 16s). Error: MountVolume.SetUp failed for volume "registry-tls" (UniqueName: "kubernetes.io/projected/f103d7a8-2560-409b-801e-701bcc6835c1-registry-tls") pod "image-registry-dd7c989d9-5w25c" (UID: "f103d7a8-2560-409b-801e-701bcc6835c1") : secret "image-registry-tls" not found Jul 16 00:40:46.671041 ip-10-0-143-200 kubenswrapper[2572]: I0716 00:40:46.671006 2572 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"samples-operator-tls\" (UniqueName: \"kubernetes.io/secret/5aabb5b2-54bc-4893-91d2-3c0215fabd67-samples-operator-tls\") pod \"cluster-samples-operator-65db858894-zf8lz\" (UID: \"5aabb5b2-54bc-4893-91d2-3c0215fabd67\") " pod="openshift-cluster-samples-operator/cluster-samples-operator-65db858894-zf8lz" Jul 16 00:40:46.671232 ip-10-0-143-200 kubenswrapper[2572]: I0716 00:40:46.671080 2572 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-tls\" (UniqueName: \"kubernetes.io/secret/645fc509-972b-4332-8562-2309cc0eab1b-metrics-tls\") pod \"dns-default-w6cgc\" (UID: \"645fc509-972b-4332-8562-2309cc0eab1b\") " pod="openshift-dns/dns-default-w6cgc" Jul 16 00:40:46.671232 ip-10-0-143-200 kubenswrapper[2572]: I0716 00:40:46.671100 2572 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/b82b08fd-bf48-4ee3-9d58-b648d6cba972-metrics-certs\") pod \"router-default-78f9d48cbb-xwpwf\" (UID: \"b82b08fd-bf48-4ee3-9d58-b648d6cba972\") " pod="openshift-ingress/router-default-78f9d48cbb-xwpwf" Jul 16 00:40:46.671232 ip-10-0-143-200 kubenswrapper[2572]: I0716 00:40:46.671126 2572 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"service-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/b82b08fd-bf48-4ee3-9d58-b648d6cba972-service-ca-bundle\") pod \"router-default-78f9d48cbb-xwpwf\" (UID: \"b82b08fd-bf48-4ee3-9d58-b648d6cba972\") " pod="openshift-ingress/router-default-78f9d48cbb-xwpwf" Jul 16 00:40:46.671232 ip-10-0-143-200 kubenswrapper[2572]: I0716 00:40:46.671156 2572 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"cluster-monitoring-operator-tls\" (UniqueName: \"kubernetes.io/secret/265fe067-cade-414b-be13-cb6dbab18057-cluster-monitoring-operator-tls\") pod \"cluster-monitoring-operator-5f7f494dd4-dxc26\" (UID: \"265fe067-cade-414b-be13-cb6dbab18057\") " pod="openshift-monitoring/cluster-monitoring-operator-5f7f494dd4-dxc26" Jul 16 00:40:46.671232 ip-10-0-143-200 kubenswrapper[2572]: E0716 00:40:46.671163 2572 secret.go:189] Couldn't get secret openshift-cluster-samples-operator/samples-operator-tls: secret "samples-operator-tls" not found Jul 16 00:40:46.671413 ip-10-0-143-200 kubenswrapper[2572]: E0716 00:40:46.671235 2572 secret.go:189] Couldn't get secret openshift-dns/dns-default-metrics-tls: secret "dns-default-metrics-tls" not found Jul 16 00:40:46.671413 ip-10-0-143-200 kubenswrapper[2572]: E0716 00:40:46.671251 2572 secret.go:189] Couldn't get secret openshift-monitoring/cluster-monitoring-operator-tls: secret "cluster-monitoring-operator-tls" not found Jul 16 00:40:46.671413 ip-10-0-143-200 kubenswrapper[2572]: E0716 00:40:46.671259 2572 secret.go:189] Couldn't get secret openshift-ingress/router-metrics-certs-default: secret "router-metrics-certs-default" not found Jul 16 00:40:46.671413 ip-10-0-143-200 kubenswrapper[2572]: E0716 00:40:46.671244 2572 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/5aabb5b2-54bc-4893-91d2-3c0215fabd67-samples-operator-tls podName:5aabb5b2-54bc-4893-91d2-3c0215fabd67 nodeName:}" failed. No retries permitted until 2026-07-16 00:41:02.671223084 +0000 UTC m=+67.779977755 (durationBeforeRetry 16s). Error: MountVolume.SetUp failed for volume "samples-operator-tls" (UniqueName: "kubernetes.io/secret/5aabb5b2-54bc-4893-91d2-3c0215fabd67-samples-operator-tls") pod "cluster-samples-operator-65db858894-zf8lz" (UID: "5aabb5b2-54bc-4893-91d2-3c0215fabd67") : secret "samples-operator-tls" not found Jul 16 00:40:46.671413 ip-10-0-143-200 kubenswrapper[2572]: E0716 00:40:46.671299 2572 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/265fe067-cade-414b-be13-cb6dbab18057-cluster-monitoring-operator-tls podName:265fe067-cade-414b-be13-cb6dbab18057 nodeName:}" failed. No retries permitted until 2026-07-16 00:41:02.671282468 +0000 UTC m=+67.780037126 (durationBeforeRetry 16s). Error: MountVolume.SetUp failed for volume "cluster-monitoring-operator-tls" (UniqueName: "kubernetes.io/secret/265fe067-cade-414b-be13-cb6dbab18057-cluster-monitoring-operator-tls") pod "cluster-monitoring-operator-5f7f494dd4-dxc26" (UID: "265fe067-cade-414b-be13-cb6dbab18057") : secret "cluster-monitoring-operator-tls" not found Jul 16 00:40:46.671413 ip-10-0-143-200 kubenswrapper[2572]: E0716 00:40:46.671314 2572 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/configmap/b82b08fd-bf48-4ee3-9d58-b648d6cba972-service-ca-bundle podName:b82b08fd-bf48-4ee3-9d58-b648d6cba972 nodeName:}" failed. No retries permitted until 2026-07-16 00:41:02.671307806 +0000 UTC m=+67.780062459 (durationBeforeRetry 16s). Error: MountVolume.SetUp failed for volume "service-ca-bundle" (UniqueName: "kubernetes.io/configmap/b82b08fd-bf48-4ee3-9d58-b648d6cba972-service-ca-bundle") pod "router-default-78f9d48cbb-xwpwf" (UID: "b82b08fd-bf48-4ee3-9d58-b648d6cba972") : configmap references non-existent config key: service-ca.crt Jul 16 00:40:46.671413 ip-10-0-143-200 kubenswrapper[2572]: E0716 00:40:46.671325 2572 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/645fc509-972b-4332-8562-2309cc0eab1b-metrics-tls podName:645fc509-972b-4332-8562-2309cc0eab1b nodeName:}" failed. No retries permitted until 2026-07-16 00:41:02.671319794 +0000 UTC m=+67.780074447 (durationBeforeRetry 16s). Error: MountVolume.SetUp failed for volume "metrics-tls" (UniqueName: "kubernetes.io/secret/645fc509-972b-4332-8562-2309cc0eab1b-metrics-tls") pod "dns-default-w6cgc" (UID: "645fc509-972b-4332-8562-2309cc0eab1b") : secret "dns-default-metrics-tls" not found Jul 16 00:40:46.671413 ip-10-0-143-200 kubenswrapper[2572]: E0716 00:40:46.671335 2572 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/b82b08fd-bf48-4ee3-9d58-b648d6cba972-metrics-certs podName:b82b08fd-bf48-4ee3-9d58-b648d6cba972 nodeName:}" failed. No retries permitted until 2026-07-16 00:41:02.671329724 +0000 UTC m=+67.780084377 (durationBeforeRetry 16s). Error: MountVolume.SetUp failed for volume "metrics-certs" (UniqueName: "kubernetes.io/secret/b82b08fd-bf48-4ee3-9d58-b648d6cba972-metrics-certs") pod "router-default-78f9d48cbb-xwpwf" (UID: "b82b08fd-bf48-4ee3-9d58-b648d6cba972") : secret "router-metrics-certs-default" not found Jul 16 00:40:46.873843 ip-10-0-143-200 kubenswrapper[2572]: I0716 00:40:46.873752 2572 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"networking-console-plugin-cert\" (UniqueName: \"kubernetes.io/secret/605373ad-28e7-4a57-9143-a698a1042a75-networking-console-plugin-cert\") pod \"networking-console-plugin-6cf856758c-vc87p\" (UID: \"605373ad-28e7-4a57-9143-a698a1042a75\") " pod="openshift-network-console/networking-console-plugin-6cf856758c-vc87p" Jul 16 00:40:46.874218 ip-10-0-143-200 kubenswrapper[2572]: E0716 00:40:46.873903 2572 secret.go:189] Couldn't get secret openshift-network-console/networking-console-plugin-cert: secret "networking-console-plugin-cert" not found Jul 16 00:40:46.874218 ip-10-0-143-200 kubenswrapper[2572]: E0716 00:40:46.873969 2572 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/605373ad-28e7-4a57-9143-a698a1042a75-networking-console-plugin-cert podName:605373ad-28e7-4a57-9143-a698a1042a75 nodeName:}" failed. No retries permitted until 2026-07-16 00:41:02.8739482 +0000 UTC m=+67.982702858 (durationBeforeRetry 16s). Error: MountVolume.SetUp failed for volume "networking-console-plugin-cert" (UniqueName: "kubernetes.io/secret/605373ad-28e7-4a57-9143-a698a1042a75-networking-console-plugin-cert") pod "networking-console-plugin-6cf856758c-vc87p" (UID: "605373ad-28e7-4a57-9143-a698a1042a75") : secret "networking-console-plugin-cert" not found Jul 16 00:40:47.724662 ip-10-0-143-200 kubenswrapper[2572]: I0716 00:40:47.724606 2572 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9gnmq_3a7e030d-fad8-480d-9dc6-fe5057048cf9/konnectivity-agent/0.log" limit=1024 Jul 16 00:40:47.992868 ip-10-0-143-200 kubenswrapper[2572]: I0716 00:40:47.992771 2572 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9gnmq_3a7e030d-fad8-480d-9dc6-fe5057048cf9/konnectivity-agent/0.log" limit=1024 Jul 16 00:40:48.153846 ip-10-0-143-200 kubenswrapper[2572]: I0716 00:40:48.153815 2572 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9gnmq_3a7e030d-fad8-480d-9dc6-fe5057048cf9/konnectivity-agent/0.log" limit=1024 Jul 16 00:40:48.320539 ip-10-0-143-200 kubenswrapper[2572]: I0716 00:40:48.320431 2572 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9gnmq_3a7e030d-fad8-480d-9dc6-fe5057048cf9/konnectivity-agent/0.log" limit=1024 Jul 16 00:40:48.512193 ip-10-0-143-200 kubenswrapper[2572]: I0716 00:40:48.512153 2572 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9gnmq_3a7e030d-fad8-480d-9dc6-fe5057048cf9/konnectivity-agent/0.log" limit=1024 Jul 16 00:40:48.732145 ip-10-0-143-200 kubenswrapper[2572]: I0716 00:40:48.732106 2572 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9gnmq_3a7e030d-fad8-480d-9dc6-fe5057048cf9/konnectivity-agent/0.log" limit=1024 Jul 16 00:40:48.793811 ip-10-0-143-200 kubenswrapper[2572]: I0716 00:40:48.793773 2572 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"insights-runtime-extractor-tls\" (UniqueName: \"kubernetes.io/secret/5f736cca-f28c-49b2-9dc9-24f0982e2d24-insights-runtime-extractor-tls\") pod \"insights-runtime-extractor-q8gql\" (UID: \"5f736cca-f28c-49b2-9dc9-24f0982e2d24\") " pod="openshift-insights/insights-runtime-extractor-q8gql" Jul 16 00:40:48.793994 ip-10-0-143-200 kubenswrapper[2572]: I0716 00:40:48.793862 2572 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-rbac-proxy-cm\" (UniqueName: \"kubernetes.io/configmap/5f736cca-f28c-49b2-9dc9-24f0982e2d24-kube-rbac-proxy-cm\") pod \"insights-runtime-extractor-q8gql\" (UID: \"5f736cca-f28c-49b2-9dc9-24f0982e2d24\") " pod="openshift-insights/insights-runtime-extractor-q8gql" Jul 16 00:40:48.793994 ip-10-0-143-200 kubenswrapper[2572]: E0716 00:40:48.793934 2572 secret.go:189] Couldn't get secret openshift-insights/insights-runtime-extractor-tls: secret "insights-runtime-extractor-tls" not found Jul 16 00:40:48.794074 ip-10-0-143-200 kubenswrapper[2572]: E0716 00:40:48.794008 2572 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/5f736cca-f28c-49b2-9dc9-24f0982e2d24-insights-runtime-extractor-tls podName:5f736cca-f28c-49b2-9dc9-24f0982e2d24 nodeName:}" failed. No retries permitted until 2026-07-16 00:40:56.793989596 +0000 UTC m=+61.902744249 (durationBeforeRetry 8s). Error: MountVolume.SetUp failed for volume "insights-runtime-extractor-tls" (UniqueName: "kubernetes.io/secret/5f736cca-f28c-49b2-9dc9-24f0982e2d24-insights-runtime-extractor-tls") pod "insights-runtime-extractor-q8gql" (UID: "5f736cca-f28c-49b2-9dc9-24f0982e2d24") : secret "insights-runtime-extractor-tls" not found Jul 16 00:40:48.794404 ip-10-0-143-200 kubenswrapper[2572]: I0716 00:40:48.794375 2572 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-rbac-proxy-cm\" (UniqueName: \"kubernetes.io/configmap/5f736cca-f28c-49b2-9dc9-24f0982e2d24-kube-rbac-proxy-cm\") pod \"insights-runtime-extractor-q8gql\" (UID: \"5f736cca-f28c-49b2-9dc9-24f0982e2d24\") " pod="openshift-insights/insights-runtime-extractor-q8gql" Jul 16 00:40:49.444093 ip-10-0-143-200 kubenswrapper[2572]: I0716 00:40:49.444052 2572 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9gnmq_3a7e030d-fad8-480d-9dc6-fe5057048cf9/konnectivity-agent/0.log" limit=1024 Jul 16 00:40:49.546254 ip-10-0-143-200 kubenswrapper[2572]: I0716 00:40:49.546216 2572 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9gnmq_3a7e030d-fad8-480d-9dc6-fe5057048cf9/konnectivity-agent/0.log" limit=1024 Jul 16 00:40:49.667091 ip-10-0-143-200 kubenswrapper[2572]: I0716 00:40:49.667064 2572 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9gnmq_3a7e030d-fad8-480d-9dc6-fe5057048cf9/konnectivity-agent/0.log" limit=1024 Jul 16 00:40:49.749353 ip-10-0-143-200 kubenswrapper[2572]: I0716 00:40:49.749323 2572 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9gnmq_3a7e030d-fad8-480d-9dc6-fe5057048cf9/konnectivity-agent/0.log" limit=1024 Jul 16 00:40:49.928397 ip-10-0-143-200 kubenswrapper[2572]: I0716 00:40:49.928351 2572 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9gnmq_3a7e030d-fad8-480d-9dc6-fe5057048cf9/konnectivity-agent/0.log" limit=1024 Jul 16 00:40:50.115385 ip-10-0-143-200 kubenswrapper[2572]: I0716 00:40:50.115299 2572 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9gnmq_3a7e030d-fad8-480d-9dc6-fe5057048cf9/konnectivity-agent/0.log" limit=1024 Jul 16 00:40:50.314891 ip-10-0-143-200 kubenswrapper[2572]: I0716 00:40:50.314856 2572 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9gnmq_3a7e030d-fad8-480d-9dc6-fe5057048cf9/konnectivity-agent/0.log" limit=1024 Jul 16 00:40:50.374247 ip-10-0-143-200 kubenswrapper[2572]: I0716 00:40:50.374147 2572 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9gnmq_3a7e030d-fad8-480d-9dc6-fe5057048cf9/konnectivity-agent/0.log" limit=1024 Jul 16 00:40:50.517461 ip-10-0-143-200 kubenswrapper[2572]: I0716 00:40:50.517414 2572 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9gnmq_3a7e030d-fad8-480d-9dc6-fe5057048cf9/konnectivity-agent/0.log" limit=1024 Jul 16 00:40:51.521200 ip-10-0-143-200 kubenswrapper[2572]: I0716 00:40:51.521169 2572 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9gnmq_3a7e030d-fad8-480d-9dc6-fe5057048cf9/konnectivity-agent/0.log" limit=1024 Jul 16 00:40:51.621644 ip-10-0-143-200 kubenswrapper[2572]: I0716 00:40:51.621601 2572 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9gnmq_3a7e030d-fad8-480d-9dc6-fe5057048cf9/konnectivity-agent/0.log" limit=1024 Jul 16 00:40:51.726467 ip-10-0-143-200 kubenswrapper[2572]: I0716 00:40:51.723445 2572 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9gnmq_3a7e030d-fad8-480d-9dc6-fe5057048cf9/konnectivity-agent/0.log" limit=1024 Jul 16 00:40:51.927467 ip-10-0-143-200 kubenswrapper[2572]: I0716 00:40:51.927370 2572 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9gnmq_3a7e030d-fad8-480d-9dc6-fe5057048cf9/konnectivity-agent/0.log" limit=1024 Jul 16 00:40:52.050979 ip-10-0-143-200 kubenswrapper[2572]: I0716 00:40:52.050945 2572 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9gnmq_3a7e030d-fad8-480d-9dc6-fe5057048cf9/konnectivity-agent/0.log" limit=1024 Jul 16 00:40:52.149621 ip-10-0-143-200 kubenswrapper[2572]: I0716 00:40:52.149571 2572 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9gnmq_3a7e030d-fad8-480d-9dc6-fe5057048cf9/konnectivity-agent/0.log" limit=1024 Jul 16 00:40:52.324798 ip-10-0-143-200 kubenswrapper[2572]: I0716 00:40:52.324757 2572 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9gnmq_3a7e030d-fad8-480d-9dc6-fe5057048cf9/konnectivity-agent/0.log" limit=1024 Jul 16 00:40:52.520789 ip-10-0-143-200 kubenswrapper[2572]: I0716 00:40:52.520749 2572 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9gnmq_3a7e030d-fad8-480d-9dc6-fe5057048cf9/konnectivity-agent/0.log" limit=1024 Jul 16 00:40:53.725114 ip-10-0-143-200 kubenswrapper[2572]: I0716 00:40:53.725071 2572 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9gnmq_3a7e030d-fad8-480d-9dc6-fe5057048cf9/konnectivity-agent/0.log" limit=1024 Jul 16 00:40:53.920934 ip-10-0-143-200 kubenswrapper[2572]: I0716 00:40:53.920888 2572 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9gnmq_3a7e030d-fad8-480d-9dc6-fe5057048cf9/konnectivity-agent/0.log" limit=1024 Jul 16 00:40:54.127656 ip-10-0-143-200 kubenswrapper[2572]: I0716 00:40:54.127539 2572 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9gnmq_3a7e030d-fad8-480d-9dc6-fe5057048cf9/konnectivity-agent/0.log" limit=1024 Jul 16 00:40:54.351243 ip-10-0-143-200 kubenswrapper[2572]: I0716 00:40:54.351213 2572 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9gnmq_3a7e030d-fad8-480d-9dc6-fe5057048cf9/konnectivity-agent/0.log" limit=1024 Jul 16 00:40:54.531282 ip-10-0-143-200 kubenswrapper[2572]: I0716 00:40:54.531237 2572 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9gnmq_3a7e030d-fad8-480d-9dc6-fe5057048cf9/konnectivity-agent/0.log" limit=1024 Jul 16 00:40:55.537231 ip-10-0-143-200 kubenswrapper[2572]: I0716 00:40:55.537194 2572 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9gnmq_3a7e030d-fad8-480d-9dc6-fe5057048cf9/konnectivity-agent/0.log" limit=1024 Jul 16 00:40:55.725445 ip-10-0-143-200 kubenswrapper[2572]: I0716 00:40:55.725407 2572 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9gnmq_3a7e030d-fad8-480d-9dc6-fe5057048cf9/konnectivity-agent/0.log" limit=1024 Jul 16 00:40:56.121732 ip-10-0-143-200 kubenswrapper[2572]: I0716 00:40:56.121700 2572 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9gnmq_3a7e030d-fad8-480d-9dc6-fe5057048cf9/konnectivity-agent/0.log" limit=1024 Jul 16 00:40:56.327125 ip-10-0-143-200 kubenswrapper[2572]: I0716 00:40:56.327085 2572 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9gnmq_3a7e030d-fad8-480d-9dc6-fe5057048cf9/konnectivity-agent/0.log" limit=1024 Jul 16 00:40:56.505292 ip-10-0-143-200 kubenswrapper[2572]: I0716 00:40:56.505254 2572 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9gnmq_3a7e030d-fad8-480d-9dc6-fe5057048cf9/konnectivity-agent/0.log" limit=1024 Jul 16 00:40:56.608489 ip-10-0-143-200 kubenswrapper[2572]: I0716 00:40:56.608450 2572 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9gnmq_3a7e030d-fad8-480d-9dc6-fe5057048cf9/konnectivity-agent/0.log" limit=1024 Jul 16 00:40:56.872059 ip-10-0-143-200 kubenswrapper[2572]: I0716 00:40:56.871970 2572 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"insights-runtime-extractor-tls\" (UniqueName: \"kubernetes.io/secret/5f736cca-f28c-49b2-9dc9-24f0982e2d24-insights-runtime-extractor-tls\") pod \"insights-runtime-extractor-q8gql\" (UID: \"5f736cca-f28c-49b2-9dc9-24f0982e2d24\") " pod="openshift-insights/insights-runtime-extractor-q8gql" Jul 16 00:40:56.874380 ip-10-0-143-200 kubenswrapper[2572]: I0716 00:40:56.874357 2572 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"insights-runtime-extractor-tls\" (UniqueName: \"kubernetes.io/secret/5f736cca-f28c-49b2-9dc9-24f0982e2d24-insights-runtime-extractor-tls\") pod \"insights-runtime-extractor-q8gql\" (UID: \"5f736cca-f28c-49b2-9dc9-24f0982e2d24\") " pod="openshift-insights/insights-runtime-extractor-q8gql" Jul 16 00:40:56.933439 ip-10-0-143-200 kubenswrapper[2572]: I0716 00:40:56.933402 2572 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-insights\"/\"insights-runtime-extractor-sa-dockercfg-6hgb5\"" Jul 16 00:40:56.941154 ip-10-0-143-200 kubenswrapper[2572]: I0716 00:40:56.941130 2572 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-insights/insights-runtime-extractor-q8gql" Jul 16 00:40:57.073970 ip-10-0-143-200 kubenswrapper[2572]: I0716 00:40:57.073909 2572 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-service-ca/service-ca-79f68b9f4f-xwtpt" podStartSLOduration=14.073887708 podStartE2EDuration="14.073887708s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-16 00:40:43 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-16 00:40:44.791886492 +0000 UTC m=+49.900641168" watchObservedRunningTime="2026-07-16 00:40:57.073887708 +0000 UTC m=+62.182642387" Jul 16 00:40:57.075138 ip-10-0-143-200 kubenswrapper[2572]: I0716 00:40:57.075118 2572 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-insights/insights-runtime-extractor-q8gql"] Jul 16 00:40:57.077891 ip-10-0-143-200 kubenswrapper[2572]: W0716 00:40:57.077848 2572 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod5f736cca_f28c_49b2_9dc9_24f0982e2d24.slice/crio-406a8c44485a010af3539ab02c53694baa74d5aee24ad141926653f59772d3cb WatchSource:0}: Error finding container 406a8c44485a010af3539ab02c53694baa74d5aee24ad141926653f59772d3cb: Status 404 returned error can't find the container with id 406a8c44485a010af3539ab02c53694baa74d5aee24ad141926653f59772d3cb Jul 16 00:40:57.442423 ip-10-0-143-200 kubenswrapper[2572]: I0716 00:40:57.442395 2572 scope.go:117] "RemoveContainer" containerID="92d5f7f712643075021fd38b8766ed695855f59e655a80cab7eaabd5675d192e" Jul 16 00:40:57.638269 ip-10-0-143-200 kubenswrapper[2572]: I0716 00:40:57.638226 2572 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ovn-kubernetes/ovnkube-node-fp78g" Jul 16 00:40:57.800752 ip-10-0-143-200 kubenswrapper[2572]: I0716 00:40:57.800665 2572 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-console-operator_console-operator-7866497494-pjc6j_a64bcd05-8596-4c12-b88e-e974d676e25f/console-operator/2.log" Jul 16 00:40:57.803065 ip-10-0-143-200 kubenswrapper[2572]: I0716 00:40:57.801962 2572 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-console-operator_console-operator-7866497494-pjc6j_a64bcd05-8596-4c12-b88e-e974d676e25f/console-operator/1.log" Jul 16 00:40:57.803065 ip-10-0-143-200 kubenswrapper[2572]: I0716 00:40:57.802011 2572 generic.go:356] "Generic (PLEG): container finished" podID="a64bcd05-8596-4c12-b88e-e974d676e25f" containerID="032af45c0cea94209fd5ba66ef9e541cc34af375b80d46c5feea4a54ccac0a0a" exitCode=255 Jul 16 00:40:57.803065 ip-10-0-143-200 kubenswrapper[2572]: I0716 00:40:57.802098 2572 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-console-operator/console-operator-7866497494-pjc6j" event={"ID":"a64bcd05-8596-4c12-b88e-e974d676e25f","Type":"ContainerDied","Data":"032af45c0cea94209fd5ba66ef9e541cc34af375b80d46c5feea4a54ccac0a0a"} Jul 16 00:40:57.803065 ip-10-0-143-200 kubenswrapper[2572]: I0716 00:40:57.802152 2572 scope.go:117] "RemoveContainer" containerID="92d5f7f712643075021fd38b8766ed695855f59e655a80cab7eaabd5675d192e" Jul 16 00:40:57.803065 ip-10-0-143-200 kubenswrapper[2572]: I0716 00:40:57.802582 2572 scope.go:117] "RemoveContainer" containerID="032af45c0cea94209fd5ba66ef9e541cc34af375b80d46c5feea4a54ccac0a0a" Jul 16 00:40:57.803065 ip-10-0-143-200 kubenswrapper[2572]: E0716 00:40:57.802794 2572 pod_workers.go:1324] "Error syncing pod, skipping" err="failed to \"StartContainer\" for \"console-operator\" with CrashLoopBackOff: \"back-off 20s restarting failed container=console-operator pod=console-operator-7866497494-pjc6j_openshift-console-operator(a64bcd05-8596-4c12-b88e-e974d676e25f)\"" pod="openshift-console-operator/console-operator-7866497494-pjc6j" podUID="a64bcd05-8596-4c12-b88e-e974d676e25f" Jul 16 00:40:57.805239 ip-10-0-143-200 kubenswrapper[2572]: I0716 00:40:57.805176 2572 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-insights/insights-runtime-extractor-q8gql" event={"ID":"5f736cca-f28c-49b2-9dc9-24f0982e2d24","Type":"ContainerStarted","Data":"406a8c44485a010af3539ab02c53694baa74d5aee24ad141926653f59772d3cb"} Jul 16 00:40:58.810513 ip-10-0-143-200 kubenswrapper[2572]: I0716 00:40:58.810480 2572 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-console-operator_console-operator-7866497494-pjc6j_a64bcd05-8596-4c12-b88e-e974d676e25f/console-operator/2.log" Jul 16 00:41:01.082649 ip-10-0-143-200 kubenswrapper[2572]: I0716 00:41:01.082587 2572 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-console-operator/console-operator-7866497494-pjc6j" Jul 16 00:41:01.083042 ip-10-0-143-200 kubenswrapper[2572]: I0716 00:41:01.082681 2572 kubelet.go:2654] "SyncLoop (probe)" probe="liveness" status="unhealthy" pod="openshift-console-operator/console-operator-7866497494-pjc6j" Jul 16 00:41:01.083042 ip-10-0-143-200 kubenswrapper[2572]: I0716 00:41:01.083027 2572 scope.go:117] "RemoveContainer" containerID="032af45c0cea94209fd5ba66ef9e541cc34af375b80d46c5feea4a54ccac0a0a" Jul 16 00:41:01.083261 ip-10-0-143-200 kubenswrapper[2572]: E0716 00:41:01.083239 2572 pod_workers.go:1324] "Error syncing pod, skipping" err="failed to \"StartContainer\" for \"console-operator\" with CrashLoopBackOff: \"back-off 20s restarting failed container=console-operator pod=console-operator-7866497494-pjc6j_openshift-console-operator(a64bcd05-8596-4c12-b88e-e974d676e25f)\"" pod="openshift-console-operator/console-operator-7866497494-pjc6j" podUID="a64bcd05-8596-4c12-b88e-e974d676e25f" Jul 16 00:41:01.118343 ip-10-0-143-200 kubenswrapper[2572]: I0716 00:41:01.118295 2572 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"original-pull-secret\" (UniqueName: \"kubernetes.io/secret/a9bdff50-c8ce-4ca4-a810-20be3909b23c-original-pull-secret\") pod \"global-pull-secret-syncer-qcbcd\" (UID: \"a9bdff50-c8ce-4ca4-a810-20be3909b23c\") " pod="kube-system/global-pull-secret-syncer-qcbcd" Jul 16 00:41:01.121623 ip-10-0-143-200 kubenswrapper[2572]: I0716 00:41:01.121591 2572 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"kube-system\"/\"original-pull-secret\"" Jul 16 00:41:01.131729 ip-10-0-143-200 kubenswrapper[2572]: I0716 00:41:01.131703 2572 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"original-pull-secret\" (UniqueName: \"kubernetes.io/secret/a9bdff50-c8ce-4ca4-a810-20be3909b23c-original-pull-secret\") pod \"global-pull-secret-syncer-qcbcd\" (UID: \"a9bdff50-c8ce-4ca4-a810-20be3909b23c\") " pod="kube-system/global-pull-secret-syncer-qcbcd" Jul 16 00:41:01.219028 ip-10-0-143-200 kubenswrapper[2572]: I0716 00:41:01.218980 2572 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/4a08cb1b-3bfa-49cf-8ff0-edef0ff944b9-metrics-certs\") pod \"network-metrics-daemon-5tphg\" (UID: \"4a08cb1b-3bfa-49cf-8ff0-edef0ff944b9\") " pod="openshift-multus/network-metrics-daemon-5tphg" Jul 16 00:41:01.222280 ip-10-0-143-200 kubenswrapper[2572]: I0716 00:41:01.222257 2572 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-multus\"/\"metrics-daemon-secret\"" Jul 16 00:41:01.231969 ip-10-0-143-200 kubenswrapper[2572]: I0716 00:41:01.231942 2572 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/4a08cb1b-3bfa-49cf-8ff0-edef0ff944b9-metrics-certs\") pod \"network-metrics-daemon-5tphg\" (UID: \"4a08cb1b-3bfa-49cf-8ff0-edef0ff944b9\") " pod="openshift-multus/network-metrics-daemon-5tphg" Jul 16 00:41:01.273390 ip-10-0-143-200 kubenswrapper[2572]: I0716 00:41:01.273352 2572 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-multus\"/\"metrics-daemon-sa-dockercfg-vmvq6\"" Jul 16 00:41:01.278920 ip-10-0-143-200 kubenswrapper[2572]: I0716 00:41:01.278892 2572 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-qcbcd" Jul 16 00:41:01.280644 ip-10-0-143-200 kubenswrapper[2572]: I0716 00:41:01.280609 2572 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-5tphg" Jul 16 00:41:01.319995 ip-10-0-143-200 kubenswrapper[2572]: I0716 00:41:01.319958 2572 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-kpfbw\" (UniqueName: \"kubernetes.io/projected/8962f109-ea94-4296-a504-f6709315e9cd-kube-api-access-kpfbw\") pod \"network-check-target-52n54\" (UID: \"8962f109-ea94-4296-a504-f6709315e9cd\") " pod="openshift-network-diagnostics/network-check-target-52n54" Jul 16 00:41:01.322807 ip-10-0-143-200 kubenswrapper[2572]: I0716 00:41:01.322782 2572 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-kpfbw\" (UniqueName: \"kubernetes.io/projected/8962f109-ea94-4296-a504-f6709315e9cd-kube-api-access-kpfbw\") pod \"network-check-target-52n54\" (UID: \"8962f109-ea94-4296-a504-f6709315e9cd\") " pod="openshift-network-diagnostics/network-check-target-52n54" Jul 16 00:41:01.564593 ip-10-0-143-200 kubenswrapper[2572]: I0716 00:41:01.564561 2572 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-network-diagnostics\"/\"default-dockercfg-hn9ps\"" Jul 16 00:41:01.571983 ip-10-0-143-200 kubenswrapper[2572]: I0716 00:41:01.571941 2572 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-52n54" Jul 16 00:41:01.903080 ip-10-0-143-200 kubenswrapper[2572]: I0716 00:41:01.902981 2572 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9gnmq_3a7e030d-fad8-480d-9dc6-fe5057048cf9/konnectivity-agent/0.log" limit=1024 Jul 16 00:41:02.142703 ip-10-0-143-200 kubenswrapper[2572]: I0716 00:41:02.142677 2572 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-network-diagnostics/network-check-target-52n54"] Jul 16 00:41:02.146258 ip-10-0-143-200 kubenswrapper[2572]: W0716 00:41:02.146149 2572 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod8962f109_ea94_4296_a504_f6709315e9cd.slice/crio-7a70385409e2a2bf1332792992a0d264fe3717458e6cd26622e7648ec555e9e1 WatchSource:0}: Error finding container 7a70385409e2a2bf1332792992a0d264fe3717458e6cd26622e7648ec555e9e1: Status 404 returned error can't find the container with id 7a70385409e2a2bf1332792992a0d264fe3717458e6cd26622e7648ec555e9e1 Jul 16 00:41:02.162938 ip-10-0-143-200 kubenswrapper[2572]: I0716 00:41:02.162796 2572 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-multus/network-metrics-daemon-5tphg"] Jul 16 00:41:02.164944 ip-10-0-143-200 kubenswrapper[2572]: W0716 00:41:02.164919 2572 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod4a08cb1b_3bfa_49cf_8ff0_edef0ff944b9.slice/crio-4a68e2e5a3a0d54f727424d0b288310a696217435275fbae9e231fdf9af8fff4 WatchSource:0}: Error finding container 4a68e2e5a3a0d54f727424d0b288310a696217435275fbae9e231fdf9af8fff4: Status 404 returned error can't find the container with id 4a68e2e5a3a0d54f727424d0b288310a696217435275fbae9e231fdf9af8fff4 Jul 16 00:41:02.180095 ip-10-0-143-200 kubenswrapper[2572]: I0716 00:41:02.180046 2572 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kube-system/global-pull-secret-syncer-qcbcd"] Jul 16 00:41:02.360571 ip-10-0-143-200 kubenswrapper[2572]: W0716 00:41:02.360530 2572 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-poda9bdff50_c8ce_4ca4_a810_20be3909b23c.slice/crio-ca87a05238f7b33b146a69b60463679128f7e09b895518e18686e6fce77163e7 WatchSource:0}: Error finding container ca87a05238f7b33b146a69b60463679128f7e09b895518e18686e6fce77163e7: Status 404 returned error can't find the container with id ca87a05238f7b33b146a69b60463679128f7e09b895518e18686e6fce77163e7 Jul 16 00:41:02.631353 ip-10-0-143-200 kubenswrapper[2572]: I0716 00:41:02.631320 2572 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"registry-tls\" (UniqueName: \"kubernetes.io/projected/f103d7a8-2560-409b-801e-701bcc6835c1-registry-tls\") pod \"image-registry-dd7c989d9-5w25c\" (UID: \"f103d7a8-2560-409b-801e-701bcc6835c1\") " pod="openshift-image-registry/image-registry-dd7c989d9-5w25c" Jul 16 00:41:02.634212 ip-10-0-143-200 kubenswrapper[2572]: I0716 00:41:02.634182 2572 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"registry-tls\" (UniqueName: \"kubernetes.io/projected/f103d7a8-2560-409b-801e-701bcc6835c1-registry-tls\") pod \"image-registry-dd7c989d9-5w25c\" (UID: \"f103d7a8-2560-409b-801e-701bcc6835c1\") " pod="openshift-image-registry/image-registry-dd7c989d9-5w25c" Jul 16 00:41:02.732188 ip-10-0-143-200 kubenswrapper[2572]: I0716 00:41:02.732144 2572 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"samples-operator-tls\" (UniqueName: \"kubernetes.io/secret/5aabb5b2-54bc-4893-91d2-3c0215fabd67-samples-operator-tls\") pod \"cluster-samples-operator-65db858894-zf8lz\" (UID: \"5aabb5b2-54bc-4893-91d2-3c0215fabd67\") " pod="openshift-cluster-samples-operator/cluster-samples-operator-65db858894-zf8lz" Jul 16 00:41:02.732399 ip-10-0-143-200 kubenswrapper[2572]: I0716 00:41:02.732226 2572 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-tls\" (UniqueName: \"kubernetes.io/secret/645fc509-972b-4332-8562-2309cc0eab1b-metrics-tls\") pod \"dns-default-w6cgc\" (UID: \"645fc509-972b-4332-8562-2309cc0eab1b\") " pod="openshift-dns/dns-default-w6cgc" Jul 16 00:41:02.732399 ip-10-0-143-200 kubenswrapper[2572]: I0716 00:41:02.732260 2572 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/b82b08fd-bf48-4ee3-9d58-b648d6cba972-metrics-certs\") pod \"router-default-78f9d48cbb-xwpwf\" (UID: \"b82b08fd-bf48-4ee3-9d58-b648d6cba972\") " pod="openshift-ingress/router-default-78f9d48cbb-xwpwf" Jul 16 00:41:02.732399 ip-10-0-143-200 kubenswrapper[2572]: I0716 00:41:02.732290 2572 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"service-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/b82b08fd-bf48-4ee3-9d58-b648d6cba972-service-ca-bundle\") pod \"router-default-78f9d48cbb-xwpwf\" (UID: \"b82b08fd-bf48-4ee3-9d58-b648d6cba972\") " pod="openshift-ingress/router-default-78f9d48cbb-xwpwf" Jul 16 00:41:02.732399 ip-10-0-143-200 kubenswrapper[2572]: I0716 00:41:02.732378 2572 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"cluster-monitoring-operator-tls\" (UniqueName: \"kubernetes.io/secret/265fe067-cade-414b-be13-cb6dbab18057-cluster-monitoring-operator-tls\") pod \"cluster-monitoring-operator-5f7f494dd4-dxc26\" (UID: \"265fe067-cade-414b-be13-cb6dbab18057\") " pod="openshift-monitoring/cluster-monitoring-operator-5f7f494dd4-dxc26" Jul 16 00:41:02.733290 ip-10-0-143-200 kubenswrapper[2572]: I0716 00:41:02.733259 2572 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"service-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/b82b08fd-bf48-4ee3-9d58-b648d6cba972-service-ca-bundle\") pod \"router-default-78f9d48cbb-xwpwf\" (UID: \"b82b08fd-bf48-4ee3-9d58-b648d6cba972\") " pod="openshift-ingress/router-default-78f9d48cbb-xwpwf" Jul 16 00:41:02.735177 ip-10-0-143-200 kubenswrapper[2572]: I0716 00:41:02.735144 2572 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/b82b08fd-bf48-4ee3-9d58-b648d6cba972-metrics-certs\") pod \"router-default-78f9d48cbb-xwpwf\" (UID: \"b82b08fd-bf48-4ee3-9d58-b648d6cba972\") " pod="openshift-ingress/router-default-78f9d48cbb-xwpwf" Jul 16 00:41:02.735265 ip-10-0-143-200 kubenswrapper[2572]: I0716 00:41:02.735222 2572 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"metrics-tls\" (UniqueName: \"kubernetes.io/secret/645fc509-972b-4332-8562-2309cc0eab1b-metrics-tls\") pod \"dns-default-w6cgc\" (UID: \"645fc509-972b-4332-8562-2309cc0eab1b\") " pod="openshift-dns/dns-default-w6cgc" Jul 16 00:41:02.735309 ip-10-0-143-200 kubenswrapper[2572]: I0716 00:41:02.735277 2572 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"samples-operator-tls\" (UniqueName: \"kubernetes.io/secret/5aabb5b2-54bc-4893-91d2-3c0215fabd67-samples-operator-tls\") pod \"cluster-samples-operator-65db858894-zf8lz\" (UID: \"5aabb5b2-54bc-4893-91d2-3c0215fabd67\") " pod="openshift-cluster-samples-operator/cluster-samples-operator-65db858894-zf8lz" Jul 16 00:41:02.735646 ip-10-0-143-200 kubenswrapper[2572]: I0716 00:41:02.735615 2572 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"cluster-monitoring-operator-tls\" (UniqueName: \"kubernetes.io/secret/265fe067-cade-414b-be13-cb6dbab18057-cluster-monitoring-operator-tls\") pod \"cluster-monitoring-operator-5f7f494dd4-dxc26\" (UID: \"265fe067-cade-414b-be13-cb6dbab18057\") " pod="openshift-monitoring/cluster-monitoring-operator-5f7f494dd4-dxc26" Jul 16 00:41:02.798456 ip-10-0-143-200 kubenswrapper[2572]: I0716 00:41:02.798418 2572 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-image-registry\"/\"registry-dockercfg-vczkg\"" Jul 16 00:41:02.805835 ip-10-0-143-200 kubenswrapper[2572]: I0716 00:41:02.805792 2572 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-image-registry/image-registry-dd7c989d9-5w25c" Jul 16 00:41:02.828324 ip-10-0-143-200 kubenswrapper[2572]: I0716 00:41:02.828292 2572 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-insights/insights-runtime-extractor-q8gql" event={"ID":"5f736cca-f28c-49b2-9dc9-24f0982e2d24","Type":"ContainerStarted","Data":"8c1e6454766833aceb8e11342f05001a5465886fb7921142b3796ef010e47f81"} Jul 16 00:41:02.829507 ip-10-0-143-200 kubenswrapper[2572]: I0716 00:41:02.829477 2572 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-multus/network-metrics-daemon-5tphg" event={"ID":"4a08cb1b-3bfa-49cf-8ff0-edef0ff944b9","Type":"ContainerStarted","Data":"4a68e2e5a3a0d54f727424d0b288310a696217435275fbae9e231fdf9af8fff4"} Jul 16 00:41:02.830833 ip-10-0-143-200 kubenswrapper[2572]: I0716 00:41:02.830810 2572 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-network-diagnostics/network-check-target-52n54" event={"ID":"8962f109-ea94-4296-a504-f6709315e9cd","Type":"ContainerStarted","Data":"8cf351fddea59fa3fd50a5e44862a90e7c44c2cad8527a5e03161730efc4b589"} Jul 16 00:41:02.830987 ip-10-0-143-200 kubenswrapper[2572]: I0716 00:41:02.830845 2572 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-network-diagnostics/network-check-target-52n54" event={"ID":"8962f109-ea94-4296-a504-f6709315e9cd","Type":"ContainerStarted","Data":"7a70385409e2a2bf1332792992a0d264fe3717458e6cd26622e7648ec555e9e1"} Jul 16 00:41:02.830987 ip-10-0-143-200 kubenswrapper[2572]: I0716 00:41:02.830882 2572 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-network-diagnostics/network-check-target-52n54" Jul 16 00:41:02.832040 ip-10-0-143-200 kubenswrapper[2572]: I0716 00:41:02.832017 2572 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kube-system/global-pull-secret-syncer-qcbcd" event={"ID":"a9bdff50-c8ce-4ca4-a810-20be3909b23c","Type":"ContainerStarted","Data":"ca87a05238f7b33b146a69b60463679128f7e09b895518e18686e6fce77163e7"} Jul 16 00:41:02.849389 ip-10-0-143-200 kubenswrapper[2572]: I0716 00:41:02.849326 2572 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-network-diagnostics/network-check-target-52n54" podStartSLOduration=67.849309951 podStartE2EDuration="1m7.849309951s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-16 00:39:55 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-16 00:41:02.848557274 +0000 UTC m=+67.957311950" watchObservedRunningTime="2026-07-16 00:41:02.849309951 +0000 UTC m=+67.958064625" Jul 16 00:41:02.850910 ip-10-0-143-200 kubenswrapper[2572]: I0716 00:41:02.850882 2572 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-cluster-samples-operator\"/\"cluster-samples-operator-dockercfg-mqqk9\"" Jul 16 00:41:02.858607 ip-10-0-143-200 kubenswrapper[2572]: I0716 00:41:02.858577 2572 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-cluster-samples-operator/cluster-samples-operator-65db858894-zf8lz" Jul 16 00:41:02.867793 ip-10-0-143-200 kubenswrapper[2572]: I0716 00:41:02.867762 2572 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress\"/\"router-dockercfg-f6zk6\"" Jul 16 00:41:02.875757 ip-10-0-143-200 kubenswrapper[2572]: I0716 00:41:02.875688 2572 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/router-default-78f9d48cbb-xwpwf" Jul 16 00:41:02.892719 ip-10-0-143-200 kubenswrapper[2572]: I0716 00:41:02.892689 2572 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"cluster-monitoring-operator-dockercfg-dh2z4\"" Jul 16 00:41:02.899885 ip-10-0-143-200 kubenswrapper[2572]: I0716 00:41:02.899855 2572 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-monitoring/cluster-monitoring-operator-5f7f494dd4-dxc26" Jul 16 00:41:02.900154 ip-10-0-143-200 kubenswrapper[2572]: I0716 00:41:02.900139 2572 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-dns\"/\"dns-dockercfg-ll89j\"" Jul 16 00:41:02.907744 ip-10-0-143-200 kubenswrapper[2572]: I0716 00:41:02.907713 2572 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-dns/dns-default-w6cgc" Jul 16 00:41:02.934196 ip-10-0-143-200 kubenswrapper[2572]: I0716 00:41:02.934163 2572 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"networking-console-plugin-cert\" (UniqueName: \"kubernetes.io/secret/605373ad-28e7-4a57-9143-a698a1042a75-networking-console-plugin-cert\") pod \"networking-console-plugin-6cf856758c-vc87p\" (UID: \"605373ad-28e7-4a57-9143-a698a1042a75\") " pod="openshift-network-console/networking-console-plugin-6cf856758c-vc87p" Jul 16 00:41:02.936950 ip-10-0-143-200 kubenswrapper[2572]: I0716 00:41:02.936924 2572 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"networking-console-plugin-cert\" (UniqueName: \"kubernetes.io/secret/605373ad-28e7-4a57-9143-a698a1042a75-networking-console-plugin-cert\") pod \"networking-console-plugin-6cf856758c-vc87p\" (UID: \"605373ad-28e7-4a57-9143-a698a1042a75\") " pod="openshift-network-console/networking-console-plugin-6cf856758c-vc87p" Jul 16 00:41:02.972436 ip-10-0-143-200 kubenswrapper[2572]: I0716 00:41:02.972389 2572 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-image-registry/image-registry-dd7c989d9-5w25c"] Jul 16 00:41:03.109843 ip-10-0-143-200 kubenswrapper[2572]: I0716 00:41:03.109758 2572 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-network-console\"/\"default-dockercfg-tncmn\"" Jul 16 00:41:03.117486 ip-10-0-143-200 kubenswrapper[2572]: I0716 00:41:03.117456 2572 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-console/networking-console-plugin-6cf856758c-vc87p" Jul 16 00:41:03.154061 ip-10-0-143-200 kubenswrapper[2572]: W0716 00:41:03.153989 2572 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podf103d7a8_2560_409b_801e_701bcc6835c1.slice/crio-8930e9612ebbcaeb1380f49c8578c122926b656baaeb1217fba519c550b0cb5a WatchSource:0}: Error finding container 8930e9612ebbcaeb1380f49c8578c122926b656baaeb1217fba519c550b0cb5a: Status 404 returned error can't find the container with id 8930e9612ebbcaeb1380f49c8578c122926b656baaeb1217fba519c550b0cb5a Jul 16 00:41:03.384790 ip-10-0-143-200 kubenswrapper[2572]: I0716 00:41:03.384732 2572 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-network-console/networking-console-plugin-6cf856758c-vc87p"] Jul 16 00:41:03.449653 ip-10-0-143-200 kubenswrapper[2572]: I0716 00:41:03.449610 2572 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-dns/dns-default-w6cgc"] Jul 16 00:41:03.459750 ip-10-0-143-200 kubenswrapper[2572]: I0716 00:41:03.459574 2572 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-monitoring/cluster-monitoring-operator-5f7f494dd4-dxc26"] Jul 16 00:41:03.592623 ip-10-0-143-200 kubenswrapper[2572]: W0716 00:41:03.592579 2572 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod605373ad_28e7_4a57_9143_a698a1042a75.slice/crio-93716e872de6dec19e793df0389069143a7a065b896450d3f8a6dc626a5ada95 WatchSource:0}: Error finding container 93716e872de6dec19e793df0389069143a7a065b896450d3f8a6dc626a5ada95: Status 404 returned error can't find the container with id 93716e872de6dec19e793df0389069143a7a065b896450d3f8a6dc626a5ada95 Jul 16 00:41:03.593532 ip-10-0-143-200 kubenswrapper[2572]: W0716 00:41:03.593498 2572 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod645fc509_972b_4332_8562_2309cc0eab1b.slice/crio-c9bc1a9b3c27a774db1a53d5e0fac2e508719778104aaf699005f56f021dc81a WatchSource:0}: Error finding container c9bc1a9b3c27a774db1a53d5e0fac2e508719778104aaf699005f56f021dc81a: Status 404 returned error can't find the container with id c9bc1a9b3c27a774db1a53d5e0fac2e508719778104aaf699005f56f021dc81a Jul 16 00:41:03.594132 ip-10-0-143-200 kubenswrapper[2572]: W0716 00:41:03.594099 2572 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod265fe067_cade_414b_be13_cb6dbab18057.slice/crio-cf1514f0e61c272b642473cc1fc58cbce8933aa48bcba67da2d355a0003a7005 WatchSource:0}: Error finding container cf1514f0e61c272b642473cc1fc58cbce8933aa48bcba67da2d355a0003a7005: Status 404 returned error can't find the container with id cf1514f0e61c272b642473cc1fc58cbce8933aa48bcba67da2d355a0003a7005 Jul 16 00:41:03.612982 ip-10-0-143-200 kubenswrapper[2572]: I0716 00:41:03.612957 2572 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-cluster-samples-operator/cluster-samples-operator-65db858894-zf8lz"] Jul 16 00:41:03.616234 ip-10-0-143-200 kubenswrapper[2572]: I0716 00:41:03.616206 2572 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/router-default-78f9d48cbb-xwpwf"] Jul 16 00:41:03.620822 ip-10-0-143-200 kubenswrapper[2572]: W0716 00:41:03.620789 2572 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podb82b08fd_bf48_4ee3_9d58_b648d6cba972.slice/crio-8303157b22fb3c32c61bcf7c91aaf4db5dd3680be72798a85292c0e1ca546740 WatchSource:0}: Error finding container 8303157b22fb3c32c61bcf7c91aaf4db5dd3680be72798a85292c0e1ca546740: Status 404 returned error can't find the container with id 8303157b22fb3c32c61bcf7c91aaf4db5dd3680be72798a85292c0e1ca546740 Jul 16 00:41:03.842265 ip-10-0-143-200 kubenswrapper[2572]: I0716 00:41:03.842008 2572 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/cluster-monitoring-operator-5f7f494dd4-dxc26" event={"ID":"265fe067-cade-414b-be13-cb6dbab18057","Type":"ContainerStarted","Data":"cf1514f0e61c272b642473cc1fc58cbce8933aa48bcba67da2d355a0003a7005"} Jul 16 00:41:03.845098 ip-10-0-143-200 kubenswrapper[2572]: I0716 00:41:03.845018 2572 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-network-console/networking-console-plugin-6cf856758c-vc87p" event={"ID":"605373ad-28e7-4a57-9143-a698a1042a75","Type":"ContainerStarted","Data":"93716e872de6dec19e793df0389069143a7a065b896450d3f8a6dc626a5ada95"} Jul 16 00:41:03.847121 ip-10-0-143-200 kubenswrapper[2572]: I0716 00:41:03.847089 2572 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-cluster-samples-operator/cluster-samples-operator-65db858894-zf8lz" event={"ID":"5aabb5b2-54bc-4893-91d2-3c0215fabd67","Type":"ContainerStarted","Data":"542f3bc3540052a807e32f8b44d387e169ececa2cb8807549a6dd90b9c678908"} Jul 16 00:41:03.849115 ip-10-0-143-200 kubenswrapper[2572]: I0716 00:41:03.849089 2572 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-dns/dns-default-w6cgc" event={"ID":"645fc509-972b-4332-8562-2309cc0eab1b","Type":"ContainerStarted","Data":"c9bc1a9b3c27a774db1a53d5e0fac2e508719778104aaf699005f56f021dc81a"} Jul 16 00:41:03.853316 ip-10-0-143-200 kubenswrapper[2572]: I0716 00:41:03.853238 2572 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-multus/network-metrics-daemon-5tphg" event={"ID":"4a08cb1b-3bfa-49cf-8ff0-edef0ff944b9","Type":"ContainerStarted","Data":"fbe4f1ddabdfedfb04f0e9ae33e677e5aea46fa514c650367767be7c0b78d113"} Jul 16 00:41:03.855909 ip-10-0-143-200 kubenswrapper[2572]: I0716 00:41:03.855650 2572 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/router-default-78f9d48cbb-xwpwf" event={"ID":"b82b08fd-bf48-4ee3-9d58-b648d6cba972","Type":"ContainerStarted","Data":"1cacd7c4ee4d4f93d8234fc8f74cd4af3dd88853a8b6ad5d1f28331fe03a7e57"} Jul 16 00:41:03.855909 ip-10-0-143-200 kubenswrapper[2572]: I0716 00:41:03.855696 2572 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/router-default-78f9d48cbb-xwpwf" event={"ID":"b82b08fd-bf48-4ee3-9d58-b648d6cba972","Type":"ContainerStarted","Data":"8303157b22fb3c32c61bcf7c91aaf4db5dd3680be72798a85292c0e1ca546740"} Jul 16 00:41:03.857808 ip-10-0-143-200 kubenswrapper[2572]: I0716 00:41:03.857785 2572 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-image-registry/image-registry-dd7c989d9-5w25c" event={"ID":"f103d7a8-2560-409b-801e-701bcc6835c1","Type":"ContainerStarted","Data":"7b74de616c853f60ff635e09a7ab38a0bbeb4e38be86d83574484fe3fe2ae387"} Jul 16 00:41:03.857909 ip-10-0-143-200 kubenswrapper[2572]: I0716 00:41:03.857826 2572 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-image-registry/image-registry-dd7c989d9-5w25c" event={"ID":"f103d7a8-2560-409b-801e-701bcc6835c1","Type":"ContainerStarted","Data":"8930e9612ebbcaeb1380f49c8578c122926b656baaeb1217fba519c550b0cb5a"} Jul 16 00:41:03.857978 ip-10-0-143-200 kubenswrapper[2572]: I0716 00:41:03.857922 2572 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-image-registry/image-registry-dd7c989d9-5w25c" Jul 16 00:41:03.876114 ip-10-0-143-200 kubenswrapper[2572]: I0716 00:41:03.876074 2572 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="openshift-ingress/router-default-78f9d48cbb-xwpwf" Jul 16 00:41:03.880572 ip-10-0-143-200 kubenswrapper[2572]: I0716 00:41:03.880472 2572 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-ingress/router-default-78f9d48cbb-xwpwf" podStartSLOduration=38.880451847 podStartE2EDuration="38.880451847s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-16 00:40:25 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-16 00:41:03.879315349 +0000 UTC m=+68.988070025" watchObservedRunningTime="2026-07-16 00:41:03.880451847 +0000 UTC m=+68.989206523" Jul 16 00:41:03.880767 ip-10-0-143-200 kubenswrapper[2572]: I0716 00:41:03.880681 2572 patch_prober.go:28] interesting pod/router-default-78f9d48cbb-xwpwf container/router namespace/openshift-ingress: Startup probe status=failure output="HTTP probe failed with statuscode: 500" start-of-body=[-]backend-proxy-http failed: reason withheld Jul 16 00:41:03.880767 ip-10-0-143-200 kubenswrapper[2572]: [-]has-synced failed: reason withheld Jul 16 00:41:03.880767 ip-10-0-143-200 kubenswrapper[2572]: [+]process-running ok Jul 16 00:41:03.880767 ip-10-0-143-200 kubenswrapper[2572]: healthz check failed Jul 16 00:41:03.880767 ip-10-0-143-200 kubenswrapper[2572]: I0716 00:41:03.880734 2572 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/router-default-78f9d48cbb-xwpwf" podUID="b82b08fd-bf48-4ee3-9d58-b648d6cba972" containerName="router" probeResult="failure" output="HTTP probe failed with statuscode: 500" Jul 16 00:41:03.909304 ip-10-0-143-200 kubenswrapper[2572]: I0716 00:41:03.908386 2572 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-image-registry/image-registry-dd7c989d9-5w25c" podStartSLOduration=68.908369614 podStartE2EDuration="1m8.908369614s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-16 00:39:55 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-16 00:41:03.906846874 +0000 UTC m=+69.015601554" watchObservedRunningTime="2026-07-16 00:41:03.908369614 +0000 UTC m=+69.017124288" Jul 16 00:41:04.605241 ip-10-0-143-200 kubenswrapper[2572]: I0716 00:41:04.603924 2572 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["open-cluster-management-agent-addon/klusterlet-addon-workmgr-f5fff84cd-z7htr"] Jul 16 00:41:04.606704 ip-10-0-143-200 kubenswrapper[2572]: I0716 00:41:04.606551 2572 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["open-cluster-management-agent-addon/managed-serviceaccount-addon-agent-f9cd9dfd6-jh8np"] Jul 16 00:41:04.607505 ip-10-0-143-200 kubenswrapper[2572]: I0716 00:41:04.606683 2572 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="open-cluster-management-agent-addon/klusterlet-addon-workmgr-f5fff84cd-z7htr" Jul 16 00:41:04.609912 ip-10-0-143-200 kubenswrapper[2572]: I0716 00:41:04.609898 2572 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="open-cluster-management-agent-addon/managed-serviceaccount-addon-agent-f9cd9dfd6-jh8np" Jul 16 00:41:04.615131 ip-10-0-143-200 kubenswrapper[2572]: I0716 00:41:04.614664 2572 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"open-cluster-management-agent-addon\"/\"openshift-service-ca.crt\"" Jul 16 00:41:04.618432 ip-10-0-143-200 kubenswrapper[2572]: I0716 00:41:04.616855 2572 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"open-cluster-management-agent-addon\"/\"kube-root-ca.crt\"" Jul 16 00:41:04.618432 ip-10-0-143-200 kubenswrapper[2572]: I0716 00:41:04.617160 2572 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"open-cluster-management-agent-addon\"/\"work-manager-hub-kubeconfig\"" Jul 16 00:41:04.618432 ip-10-0-143-200 kubenswrapper[2572]: I0716 00:41:04.617368 2572 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"open-cluster-management-agent-addon\"/\"open-cluster-management-image-pull-credentials\"" Jul 16 00:41:04.618432 ip-10-0-143-200 kubenswrapper[2572]: I0716 00:41:04.617564 2572 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"open-cluster-management-agent-addon\"/\"managed-serviceaccount-hub-kubeconfig\"" Jul 16 00:41:04.618432 ip-10-0-143-200 kubenswrapper[2572]: I0716 00:41:04.617756 2572 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"open-cluster-management-agent-addon\"/\"managed-serviceaccount-dockercfg-5nz6c\"" Jul 16 00:41:04.637812 ip-10-0-143-200 kubenswrapper[2572]: I0716 00:41:04.637786 2572 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["open-cluster-management-agent-addon/cluster-proxy-proxy-agent-599f598f85-4q8hh"] Jul 16 00:41:04.644051 ip-10-0-143-200 kubenswrapper[2572]: I0716 00:41:04.642915 2572 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["open-cluster-management-agent-addon/managed-serviceaccount-addon-agent-f9cd9dfd6-jh8np"] Jul 16 00:41:04.644051 ip-10-0-143-200 kubenswrapper[2572]: I0716 00:41:04.642954 2572 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["open-cluster-management-agent-addon/klusterlet-addon-workmgr-f5fff84cd-z7htr"] Jul 16 00:41:04.644051 ip-10-0-143-200 kubenswrapper[2572]: I0716 00:41:04.643067 2572 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="open-cluster-management-agent-addon/cluster-proxy-proxy-agent-599f598f85-4q8hh" Jul 16 00:41:04.648106 ip-10-0-143-200 kubenswrapper[2572]: I0716 00:41:04.647034 2572 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"open-cluster-management-agent-addon\"/\"cluster-proxy-ca\"" Jul 16 00:41:04.648106 ip-10-0-143-200 kubenswrapper[2572]: I0716 00:41:04.647380 2572 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"open-cluster-management-agent-addon\"/\"cluster-proxy-hub-kubeconfig\"" Jul 16 00:41:04.648106 ip-10-0-143-200 kubenswrapper[2572]: I0716 00:41:04.647717 2572 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"open-cluster-management-agent-addon\"/\"cluster-proxy-open-cluster-management.io-proxy-agent-signer-client-cert\"" Jul 16 00:41:04.648106 ip-10-0-143-200 kubenswrapper[2572]: I0716 00:41:04.647917 2572 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"open-cluster-management-agent-addon\"/\"cluster-proxy-service-proxy-server-certificates\"" Jul 16 00:41:04.653570 ip-10-0-143-200 kubenswrapper[2572]: I0716 00:41:04.652941 2572 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"hub\" (UniqueName: \"kubernetes.io/secret/7f887d45-f998-499d-84da-8876b6800f46-hub\") pod \"cluster-proxy-proxy-agent-599f598f85-4q8hh\" (UID: \"7f887d45-f998-499d-84da-8876b6800f46\") " pod="open-cluster-management-agent-addon/cluster-proxy-proxy-agent-599f598f85-4q8hh" Jul 16 00:41:04.653570 ip-10-0-143-200 kubenswrapper[2572]: I0716 00:41:04.652987 2572 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"klusterlet-config\" (UniqueName: \"kubernetes.io/secret/6d8c8acf-58c5-4ff4-9e27-a96773c67b32-klusterlet-config\") pod \"klusterlet-addon-workmgr-f5fff84cd-z7htr\" (UID: \"6d8c8acf-58c5-4ff4-9e27-a96773c67b32\") " pod="open-cluster-management-agent-addon/klusterlet-addon-workmgr-f5fff84cd-z7htr" Jul 16 00:41:04.653570 ip-10-0-143-200 kubenswrapper[2572]: I0716 00:41:04.653042 2572 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"additional-service-ca\" (UniqueName: \"kubernetes.io/configmap/7f887d45-f998-499d-84da-8876b6800f46-additional-service-ca\") pod \"cluster-proxy-proxy-agent-599f598f85-4q8hh\" (UID: \"7f887d45-f998-499d-84da-8876b6800f46\") " pod="open-cluster-management-agent-addon/cluster-proxy-proxy-agent-599f598f85-4q8hh" Jul 16 00:41:04.653570 ip-10-0-143-200 kubenswrapper[2572]: I0716 00:41:04.653070 2572 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-6lffc\" (UniqueName: \"kubernetes.io/projected/7f887d45-f998-499d-84da-8876b6800f46-kube-api-access-6lffc\") pod \"cluster-proxy-proxy-agent-599f598f85-4q8hh\" (UID: \"7f887d45-f998-499d-84da-8876b6800f46\") " pod="open-cluster-management-agent-addon/cluster-proxy-proxy-agent-599f598f85-4q8hh" Jul 16 00:41:04.653570 ip-10-0-143-200 kubenswrapper[2572]: I0716 00:41:04.653129 2572 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-pm2zz\" (UniqueName: \"kubernetes.io/projected/6d8c8acf-58c5-4ff4-9e27-a96773c67b32-kube-api-access-pm2zz\") pod \"klusterlet-addon-workmgr-f5fff84cd-z7htr\" (UID: \"6d8c8acf-58c5-4ff4-9e27-a96773c67b32\") " pod="open-cluster-management-agent-addon/klusterlet-addon-workmgr-f5fff84cd-z7htr" Jul 16 00:41:04.653570 ip-10-0-143-200 kubenswrapper[2572]: I0716 00:41:04.653191 2572 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-g68cp\" (UniqueName: \"kubernetes.io/projected/636d7344-4105-43dd-9008-1910729748b4-kube-api-access-g68cp\") pod \"managed-serviceaccount-addon-agent-f9cd9dfd6-jh8np\" (UID: \"636d7344-4105-43dd-9008-1910729748b4\") " pod="open-cluster-management-agent-addon/managed-serviceaccount-addon-agent-f9cd9dfd6-jh8np" Jul 16 00:41:04.653570 ip-10-0-143-200 kubenswrapper[2572]: I0716 00:41:04.653252 2572 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"hub-kubeconfig\" (UniqueName: \"kubernetes.io/secret/7f887d45-f998-499d-84da-8876b6800f46-hub-kubeconfig\") pod \"cluster-proxy-proxy-agent-599f598f85-4q8hh\" (UID: \"7f887d45-f998-499d-84da-8876b6800f46\") " pod="open-cluster-management-agent-addon/cluster-proxy-proxy-agent-599f598f85-4q8hh" Jul 16 00:41:04.653570 ip-10-0-143-200 kubenswrapper[2572]: I0716 00:41:04.653287 2572 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"hub-kubeconfig\" (UniqueName: \"kubernetes.io/secret/636d7344-4105-43dd-9008-1910729748b4-hub-kubeconfig\") pod \"managed-serviceaccount-addon-agent-f9cd9dfd6-jh8np\" (UID: \"636d7344-4105-43dd-9008-1910729748b4\") " pod="open-cluster-management-agent-addon/managed-serviceaccount-addon-agent-f9cd9dfd6-jh8np" Jul 16 00:41:04.653570 ip-10-0-143-200 kubenswrapper[2572]: I0716 00:41:04.653328 2572 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"ca\" (UniqueName: \"kubernetes.io/secret/7f887d45-f998-499d-84da-8876b6800f46-ca\") pod \"cluster-proxy-proxy-agent-599f598f85-4q8hh\" (UID: \"7f887d45-f998-499d-84da-8876b6800f46\") " pod="open-cluster-management-agent-addon/cluster-proxy-proxy-agent-599f598f85-4q8hh" Jul 16 00:41:04.653570 ip-10-0-143-200 kubenswrapper[2572]: I0716 00:41:04.653360 2572 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"service-proxy-server-cert\" (UniqueName: \"kubernetes.io/secret/7f887d45-f998-499d-84da-8876b6800f46-service-proxy-server-cert\") pod \"cluster-proxy-proxy-agent-599f598f85-4q8hh\" (UID: \"7f887d45-f998-499d-84da-8876b6800f46\") " pod="open-cluster-management-agent-addon/cluster-proxy-proxy-agent-599f598f85-4q8hh" Jul 16 00:41:04.653570 ip-10-0-143-200 kubenswrapper[2572]: I0716 00:41:04.653404 2572 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tmp\" (UniqueName: \"kubernetes.io/empty-dir/6d8c8acf-58c5-4ff4-9e27-a96773c67b32-tmp\") pod \"klusterlet-addon-workmgr-f5fff84cd-z7htr\" (UID: \"6d8c8acf-58c5-4ff4-9e27-a96773c67b32\") " pod="open-cluster-management-agent-addon/klusterlet-addon-workmgr-f5fff84cd-z7htr" Jul 16 00:41:04.654405 ip-10-0-143-200 kubenswrapper[2572]: I0716 00:41:04.653970 2572 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9gnmq_3a7e030d-fad8-480d-9dc6-fe5057048cf9/konnectivity-agent/0.log" limit=1024 Jul 16 00:41:04.658486 ip-10-0-143-200 kubenswrapper[2572]: I0716 00:41:04.658461 2572 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["open-cluster-management-agent-addon/cluster-proxy-proxy-agent-599f598f85-4q8hh"] Jul 16 00:41:04.754708 ip-10-0-143-200 kubenswrapper[2572]: I0716 00:41:04.754672 2572 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"ca\" (UniqueName: \"kubernetes.io/secret/7f887d45-f998-499d-84da-8876b6800f46-ca\") pod \"cluster-proxy-proxy-agent-599f598f85-4q8hh\" (UID: \"7f887d45-f998-499d-84da-8876b6800f46\") " pod="open-cluster-management-agent-addon/cluster-proxy-proxy-agent-599f598f85-4q8hh" Jul 16 00:41:04.754708 ip-10-0-143-200 kubenswrapper[2572]: I0716 00:41:04.754727 2572 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"service-proxy-server-cert\" (UniqueName: \"kubernetes.io/secret/7f887d45-f998-499d-84da-8876b6800f46-service-proxy-server-cert\") pod \"cluster-proxy-proxy-agent-599f598f85-4q8hh\" (UID: \"7f887d45-f998-499d-84da-8876b6800f46\") " pod="open-cluster-management-agent-addon/cluster-proxy-proxy-agent-599f598f85-4q8hh" Jul 16 00:41:04.755183 ip-10-0-143-200 kubenswrapper[2572]: I0716 00:41:04.754759 2572 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tmp\" (UniqueName: \"kubernetes.io/empty-dir/6d8c8acf-58c5-4ff4-9e27-a96773c67b32-tmp\") pod \"klusterlet-addon-workmgr-f5fff84cd-z7htr\" (UID: \"6d8c8acf-58c5-4ff4-9e27-a96773c67b32\") " pod="open-cluster-management-agent-addon/klusterlet-addon-workmgr-f5fff84cd-z7htr" Jul 16 00:41:04.755183 ip-10-0-143-200 kubenswrapper[2572]: I0716 00:41:04.754812 2572 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"hub\" (UniqueName: \"kubernetes.io/secret/7f887d45-f998-499d-84da-8876b6800f46-hub\") pod \"cluster-proxy-proxy-agent-599f598f85-4q8hh\" (UID: \"7f887d45-f998-499d-84da-8876b6800f46\") " pod="open-cluster-management-agent-addon/cluster-proxy-proxy-agent-599f598f85-4q8hh" Jul 16 00:41:04.755183 ip-10-0-143-200 kubenswrapper[2572]: I0716 00:41:04.754846 2572 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"klusterlet-config\" (UniqueName: \"kubernetes.io/secret/6d8c8acf-58c5-4ff4-9e27-a96773c67b32-klusterlet-config\") pod \"klusterlet-addon-workmgr-f5fff84cd-z7htr\" (UID: \"6d8c8acf-58c5-4ff4-9e27-a96773c67b32\") " pod="open-cluster-management-agent-addon/klusterlet-addon-workmgr-f5fff84cd-z7htr" Jul 16 00:41:04.755183 ip-10-0-143-200 kubenswrapper[2572]: I0716 00:41:04.754885 2572 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"additional-service-ca\" (UniqueName: \"kubernetes.io/configmap/7f887d45-f998-499d-84da-8876b6800f46-additional-service-ca\") pod \"cluster-proxy-proxy-agent-599f598f85-4q8hh\" (UID: \"7f887d45-f998-499d-84da-8876b6800f46\") " pod="open-cluster-management-agent-addon/cluster-proxy-proxy-agent-599f598f85-4q8hh" Jul 16 00:41:04.755183 ip-10-0-143-200 kubenswrapper[2572]: I0716 00:41:04.754914 2572 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-6lffc\" (UniqueName: \"kubernetes.io/projected/7f887d45-f998-499d-84da-8876b6800f46-kube-api-access-6lffc\") pod \"cluster-proxy-proxy-agent-599f598f85-4q8hh\" (UID: \"7f887d45-f998-499d-84da-8876b6800f46\") " pod="open-cluster-management-agent-addon/cluster-proxy-proxy-agent-599f598f85-4q8hh" Jul 16 00:41:04.755183 ip-10-0-143-200 kubenswrapper[2572]: I0716 00:41:04.754949 2572 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-pm2zz\" (UniqueName: \"kubernetes.io/projected/6d8c8acf-58c5-4ff4-9e27-a96773c67b32-kube-api-access-pm2zz\") pod \"klusterlet-addon-workmgr-f5fff84cd-z7htr\" (UID: \"6d8c8acf-58c5-4ff4-9e27-a96773c67b32\") " pod="open-cluster-management-agent-addon/klusterlet-addon-workmgr-f5fff84cd-z7htr" Jul 16 00:41:04.755183 ip-10-0-143-200 kubenswrapper[2572]: I0716 00:41:04.754994 2572 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-g68cp\" (UniqueName: \"kubernetes.io/projected/636d7344-4105-43dd-9008-1910729748b4-kube-api-access-g68cp\") pod \"managed-serviceaccount-addon-agent-f9cd9dfd6-jh8np\" (UID: \"636d7344-4105-43dd-9008-1910729748b4\") " pod="open-cluster-management-agent-addon/managed-serviceaccount-addon-agent-f9cd9dfd6-jh8np" Jul 16 00:41:04.755183 ip-10-0-143-200 kubenswrapper[2572]: I0716 00:41:04.755113 2572 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"hub-kubeconfig\" (UniqueName: \"kubernetes.io/secret/7f887d45-f998-499d-84da-8876b6800f46-hub-kubeconfig\") pod \"cluster-proxy-proxy-agent-599f598f85-4q8hh\" (UID: \"7f887d45-f998-499d-84da-8876b6800f46\") " pod="open-cluster-management-agent-addon/cluster-proxy-proxy-agent-599f598f85-4q8hh" Jul 16 00:41:04.755183 ip-10-0-143-200 kubenswrapper[2572]: I0716 00:41:04.755147 2572 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"hub-kubeconfig\" (UniqueName: \"kubernetes.io/secret/636d7344-4105-43dd-9008-1910729748b4-hub-kubeconfig\") pod \"managed-serviceaccount-addon-agent-f9cd9dfd6-jh8np\" (UID: \"636d7344-4105-43dd-9008-1910729748b4\") " pod="open-cluster-management-agent-addon/managed-serviceaccount-addon-agent-f9cd9dfd6-jh8np" Jul 16 00:41:04.757384 ip-10-0-143-200 kubenswrapper[2572]: I0716 00:41:04.757237 2572 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"tmp\" (UniqueName: \"kubernetes.io/empty-dir/6d8c8acf-58c5-4ff4-9e27-a96773c67b32-tmp\") pod \"klusterlet-addon-workmgr-f5fff84cd-z7htr\" (UID: \"6d8c8acf-58c5-4ff4-9e27-a96773c67b32\") " pod="open-cluster-management-agent-addon/klusterlet-addon-workmgr-f5fff84cd-z7htr" Jul 16 00:41:04.757691 ip-10-0-143-200 kubenswrapper[2572]: I0716 00:41:04.757665 2572 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"additional-service-ca\" (UniqueName: \"kubernetes.io/configmap/7f887d45-f998-499d-84da-8876b6800f46-additional-service-ca\") pod \"cluster-proxy-proxy-agent-599f598f85-4q8hh\" (UID: \"7f887d45-f998-499d-84da-8876b6800f46\") " pod="open-cluster-management-agent-addon/cluster-proxy-proxy-agent-599f598f85-4q8hh" Jul 16 00:41:04.768115 ip-10-0-143-200 kubenswrapper[2572]: I0716 00:41:04.768071 2572 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"service-proxy-server-cert\" (UniqueName: \"kubernetes.io/secret/7f887d45-f998-499d-84da-8876b6800f46-service-proxy-server-cert\") pod \"cluster-proxy-proxy-agent-599f598f85-4q8hh\" (UID: \"7f887d45-f998-499d-84da-8876b6800f46\") " pod="open-cluster-management-agent-addon/cluster-proxy-proxy-agent-599f598f85-4q8hh" Jul 16 00:41:04.768218 ip-10-0-143-200 kubenswrapper[2572]: I0716 00:41:04.768150 2572 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"klusterlet-config\" (UniqueName: \"kubernetes.io/secret/6d8c8acf-58c5-4ff4-9e27-a96773c67b32-klusterlet-config\") pod \"klusterlet-addon-workmgr-f5fff84cd-z7htr\" (UID: \"6d8c8acf-58c5-4ff4-9e27-a96773c67b32\") " pod="open-cluster-management-agent-addon/klusterlet-addon-workmgr-f5fff84cd-z7htr" Jul 16 00:41:04.768765 ip-10-0-143-200 kubenswrapper[2572]: I0716 00:41:04.768740 2572 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"hub\" (UniqueName: \"kubernetes.io/secret/7f887d45-f998-499d-84da-8876b6800f46-hub\") pod \"cluster-proxy-proxy-agent-599f598f85-4q8hh\" (UID: \"7f887d45-f998-499d-84da-8876b6800f46\") " pod="open-cluster-management-agent-addon/cluster-proxy-proxy-agent-599f598f85-4q8hh" Jul 16 00:41:04.769184 ip-10-0-143-200 kubenswrapper[2572]: I0716 00:41:04.769142 2572 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"hub-kubeconfig\" (UniqueName: \"kubernetes.io/secret/636d7344-4105-43dd-9008-1910729748b4-hub-kubeconfig\") pod \"managed-serviceaccount-addon-agent-f9cd9dfd6-jh8np\" (UID: \"636d7344-4105-43dd-9008-1910729748b4\") " pod="open-cluster-management-agent-addon/managed-serviceaccount-addon-agent-f9cd9dfd6-jh8np" Jul 16 00:41:04.771204 ip-10-0-143-200 kubenswrapper[2572]: I0716 00:41:04.771161 2572 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"hub-kubeconfig\" (UniqueName: \"kubernetes.io/secret/7f887d45-f998-499d-84da-8876b6800f46-hub-kubeconfig\") pod \"cluster-proxy-proxy-agent-599f598f85-4q8hh\" (UID: \"7f887d45-f998-499d-84da-8876b6800f46\") " pod="open-cluster-management-agent-addon/cluster-proxy-proxy-agent-599f598f85-4q8hh" Jul 16 00:41:04.773957 ip-10-0-143-200 kubenswrapper[2572]: I0716 00:41:04.773906 2572 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"ca\" (UniqueName: \"kubernetes.io/secret/7f887d45-f998-499d-84da-8876b6800f46-ca\") pod \"cluster-proxy-proxy-agent-599f598f85-4q8hh\" (UID: \"7f887d45-f998-499d-84da-8876b6800f46\") " pod="open-cluster-management-agent-addon/cluster-proxy-proxy-agent-599f598f85-4q8hh" Jul 16 00:41:04.781593 ip-10-0-143-200 kubenswrapper[2572]: I0716 00:41:04.781547 2572 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-6lffc\" (UniqueName: \"kubernetes.io/projected/7f887d45-f998-499d-84da-8876b6800f46-kube-api-access-6lffc\") pod \"cluster-proxy-proxy-agent-599f598f85-4q8hh\" (UID: \"7f887d45-f998-499d-84da-8876b6800f46\") " pod="open-cluster-management-agent-addon/cluster-proxy-proxy-agent-599f598f85-4q8hh" Jul 16 00:41:04.783166 ip-10-0-143-200 kubenswrapper[2572]: I0716 00:41:04.783114 2572 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-g68cp\" (UniqueName: \"kubernetes.io/projected/636d7344-4105-43dd-9008-1910729748b4-kube-api-access-g68cp\") pod \"managed-serviceaccount-addon-agent-f9cd9dfd6-jh8np\" (UID: \"636d7344-4105-43dd-9008-1910729748b4\") " pod="open-cluster-management-agent-addon/managed-serviceaccount-addon-agent-f9cd9dfd6-jh8np" Jul 16 00:41:04.785789 ip-10-0-143-200 kubenswrapper[2572]: I0716 00:41:04.785749 2572 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-pm2zz\" (UniqueName: \"kubernetes.io/projected/6d8c8acf-58c5-4ff4-9e27-a96773c67b32-kube-api-access-pm2zz\") pod \"klusterlet-addon-workmgr-f5fff84cd-z7htr\" (UID: \"6d8c8acf-58c5-4ff4-9e27-a96773c67b32\") " pod="open-cluster-management-agent-addon/klusterlet-addon-workmgr-f5fff84cd-z7htr" Jul 16 00:41:04.816157 ip-10-0-143-200 kubenswrapper[2572]: I0716 00:41:04.816102 2572 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9gnmq_3a7e030d-fad8-480d-9dc6-fe5057048cf9/konnectivity-agent/0.log" limit=1024 Jul 16 00:41:04.872602 ip-10-0-143-200 kubenswrapper[2572]: I0716 00:41:04.871525 2572 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-multus/network-metrics-daemon-5tphg" event={"ID":"4a08cb1b-3bfa-49cf-8ff0-edef0ff944b9","Type":"ContainerStarted","Data":"648cd42a8d463194dbc4308f961123f9a2716dddcdfba7bf8439215259a9270d"} Jul 16 00:41:04.878121 ip-10-0-143-200 kubenswrapper[2572]: I0716 00:41:04.878067 2572 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="openshift-ingress/router-default-78f9d48cbb-xwpwf" Jul 16 00:41:04.938280 ip-10-0-143-200 kubenswrapper[2572]: I0716 00:41:04.938245 2572 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="open-cluster-management-agent-addon/klusterlet-addon-workmgr-f5fff84cd-z7htr" Jul 16 00:41:04.964068 ip-10-0-143-200 kubenswrapper[2572]: I0716 00:41:04.964040 2572 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="open-cluster-management-agent-addon/managed-serviceaccount-addon-agent-f9cd9dfd6-jh8np" Jul 16 00:41:04.972994 ip-10-0-143-200 kubenswrapper[2572]: I0716 00:41:04.972966 2572 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="open-cluster-management-agent-addon/cluster-proxy-proxy-agent-599f598f85-4q8hh" Jul 16 00:41:04.993185 ip-10-0-143-200 kubenswrapper[2572]: I0716 00:41:04.991444 2572 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-multus/network-metrics-daemon-5tphg" podStartSLOduration=68.528032248 podStartE2EDuration="1m9.991420633s" totalImagesPullingTime="1.463388385s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-16 00:39:55 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-16 00:41:04.92746919 +0000 UTC m=+70.036223866" watchObservedRunningTime="2026-07-16 00:41:04.991420633 +0000 UTC m=+70.100175310" Jul 16 00:41:05.124944 ip-10-0-143-200 kubenswrapper[2572]: I0716 00:41:05.124761 2572 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9gnmq_3a7e030d-fad8-480d-9dc6-fe5057048cf9/konnectivity-agent/0.log" limit=1024 Jul 16 00:41:05.350078 ip-10-0-143-200 kubenswrapper[2572]: I0716 00:41:05.349977 2572 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9gnmq_3a7e030d-fad8-480d-9dc6-fe5057048cf9/konnectivity-agent/0.log" limit=1024 Jul 16 00:41:05.590702 ip-10-0-143-200 kubenswrapper[2572]: I0716 00:41:05.590604 2572 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9gnmq_3a7e030d-fad8-480d-9dc6-fe5057048cf9/konnectivity-agent/0.log" limit=1024 Jul 16 00:41:05.736805 ip-10-0-143-200 kubenswrapper[2572]: I0716 00:41:05.736767 2572 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9gnmq_3a7e030d-fad8-480d-9dc6-fe5057048cf9/konnectivity-agent/0.log" limit=1024 Jul 16 00:41:05.876312 ip-10-0-143-200 kubenswrapper[2572]: I0716 00:41:05.876206 2572 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/router-default-78f9d48cbb-xwpwf" Jul 16 00:41:05.877511 ip-10-0-143-200 kubenswrapper[2572]: I0716 00:41:05.877416 2572 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/router-default-78f9d48cbb-xwpwf" Jul 16 00:41:06.043439 ip-10-0-143-200 kubenswrapper[2572]: I0716 00:41:06.043392 2572 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9gnmq_3a7e030d-fad8-480d-9dc6-fe5057048cf9/konnectivity-agent/0.log" limit=1024 Jul 16 00:41:06.139622 ip-10-0-143-200 kubenswrapper[2572]: I0716 00:41:06.139469 2572 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9gnmq_3a7e030d-fad8-480d-9dc6-fe5057048cf9/konnectivity-agent/0.log" limit=1024 Jul 16 00:41:06.859597 ip-10-0-143-200 kubenswrapper[2572]: I0716 00:41:06.859559 2572 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9gnmq_3a7e030d-fad8-480d-9dc6-fe5057048cf9/konnectivity-agent/0.log" limit=1024 Jul 16 00:41:06.963476 ip-10-0-143-200 kubenswrapper[2572]: I0716 00:41:06.963441 2572 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9gnmq_3a7e030d-fad8-480d-9dc6-fe5057048cf9/konnectivity-agent/0.log" limit=1024 Jul 16 00:41:07.222287 ip-10-0-143-200 kubenswrapper[2572]: I0716 00:41:07.222179 2572 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9gnmq_3a7e030d-fad8-480d-9dc6-fe5057048cf9/konnectivity-agent/0.log" limit=1024 Jul 16 00:41:07.364296 ip-10-0-143-200 kubenswrapper[2572]: I0716 00:41:07.364262 2572 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9gnmq_3a7e030d-fad8-480d-9dc6-fe5057048cf9/konnectivity-agent/0.log" limit=1024 Jul 16 00:41:10.023163 ip-10-0-143-200 kubenswrapper[2572]: I0716 00:41:10.023125 2572 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["open-cluster-management-agent-addon/cluster-proxy-proxy-agent-599f598f85-4q8hh"] Jul 16 00:41:10.740512 ip-10-0-143-200 kubenswrapper[2572]: W0716 00:41:10.740473 2572 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-besteffort.slice/kubepods-besteffort-pod7f887d45_f998_499d_84da_8876b6800f46.slice/crio-924424c92977a7444d5b3bbff0425fe72686dbde5f15958d3947e5de31319f61 WatchSource:0}: Error finding container 924424c92977a7444d5b3bbff0425fe72686dbde5f15958d3947e5de31319f61: Status 404 returned error can't find the container with id 924424c92977a7444d5b3bbff0425fe72686dbde5f15958d3947e5de31319f61 Jul 16 00:41:10.744354 ip-10-0-143-200 kubenswrapper[2572]: I0716 00:41:10.744321 2572 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9gnmq_3a7e030d-fad8-480d-9dc6-fe5057048cf9/konnectivity-agent/0.log" limit=1024 Jul 16 00:41:10.896874 ip-10-0-143-200 kubenswrapper[2572]: I0716 00:41:10.896852 2572 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["open-cluster-management-agent-addon/klusterlet-addon-workmgr-f5fff84cd-z7htr"] Jul 16 00:41:10.899165 ip-10-0-143-200 kubenswrapper[2572]: I0716 00:41:10.899138 2572 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="open-cluster-management-agent-addon/cluster-proxy-proxy-agent-599f598f85-4q8hh" event={"ID":"7f887d45-f998-499d-84da-8876b6800f46","Type":"ContainerStarted","Data":"924424c92977a7444d5b3bbff0425fe72686dbde5f15958d3947e5de31319f61"} Jul 16 00:41:10.917643 ip-10-0-143-200 kubenswrapper[2572]: I0716 00:41:10.917601 2572 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9gnmq_3a7e030d-fad8-480d-9dc6-fe5057048cf9/konnectivity-agent/0.log" limit=1024 Jul 16 00:41:10.935555 ip-10-0-143-200 kubenswrapper[2572]: W0716 00:41:10.935508 2572 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod6d8c8acf_58c5_4ff4_9e27_a96773c67b32.slice/crio-3b950fe7a73085568a7d6afefc9e1dacaa9086cf985409e98dbc1000c9dc1868 WatchSource:0}: Error finding container 3b950fe7a73085568a7d6afefc9e1dacaa9086cf985409e98dbc1000c9dc1868: Status 404 returned error can't find the container with id 3b950fe7a73085568a7d6afefc9e1dacaa9086cf985409e98dbc1000c9dc1868 Jul 16 00:41:10.992449 ip-10-0-143-200 kubenswrapper[2572]: I0716 00:41:10.992419 2572 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9gnmq_3a7e030d-fad8-480d-9dc6-fe5057048cf9/konnectivity-agent/0.log" limit=1024 Jul 16 00:41:11.079857 ip-10-0-143-200 kubenswrapper[2572]: I0716 00:41:11.079829 2572 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["open-cluster-management-agent-addon/managed-serviceaccount-addon-agent-f9cd9dfd6-jh8np"] Jul 16 00:41:11.085146 ip-10-0-143-200 kubenswrapper[2572]: W0716 00:41:11.085112 2572 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-besteffort.slice/kubepods-besteffort-pod636d7344_4105_43dd_9008_1910729748b4.slice/crio-8a3476a00ef1b8f90a596873719ba8fb8b43fe9a0539aae16aa04ae9fc44b0f6 WatchSource:0}: Error finding container 8a3476a00ef1b8f90a596873719ba8fb8b43fe9a0539aae16aa04ae9fc44b0f6: Status 404 returned error can't find the container with id 8a3476a00ef1b8f90a596873719ba8fb8b43fe9a0539aae16aa04ae9fc44b0f6 Jul 16 00:41:11.419658 ip-10-0-143-200 kubenswrapper[2572]: I0716 00:41:11.419607 2572 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9gnmq_3a7e030d-fad8-480d-9dc6-fe5057048cf9/konnectivity-agent/0.log" limit=1024 Jul 16 00:41:11.611964 ip-10-0-143-200 kubenswrapper[2572]: I0716 00:41:11.611854 2572 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9gnmq_3a7e030d-fad8-480d-9dc6-fe5057048cf9/konnectivity-agent/0.log" limit=1024 Jul 16 00:41:11.719870 ip-10-0-143-200 kubenswrapper[2572]: I0716 00:41:11.719829 2572 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9gnmq_3a7e030d-fad8-480d-9dc6-fe5057048cf9/konnectivity-agent/0.log" limit=1024 Jul 16 00:41:11.908704 ip-10-0-143-200 kubenswrapper[2572]: I0716 00:41:11.908608 2572 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/cluster-monitoring-operator-5f7f494dd4-dxc26" event={"ID":"265fe067-cade-414b-be13-cb6dbab18057","Type":"ContainerStarted","Data":"e4227a5a0125f2e5d7e4467bbc9b6dc6edcb23865ee4a57f30824c2e939910d3"} Jul 16 00:41:11.910439 ip-10-0-143-200 kubenswrapper[2572]: I0716 00:41:11.910402 2572 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9gnmq_3a7e030d-fad8-480d-9dc6-fe5057048cf9/konnectivity-agent/0.log" limit=1024 Jul 16 00:41:11.913690 ip-10-0-143-200 kubenswrapper[2572]: I0716 00:41:11.913659 2572 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-network-console/networking-console-plugin-6cf856758c-vc87p" event={"ID":"605373ad-28e7-4a57-9143-a698a1042a75","Type":"ContainerStarted","Data":"4cb73d5f2a178a844a3c833f035c5f7c291181624b432ba8df3ac954c7bdcdd1"} Jul 16 00:41:11.919217 ip-10-0-143-200 kubenswrapper[2572]: I0716 00:41:11.919126 2572 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-cluster-samples-operator/cluster-samples-operator-65db858894-zf8lz" event={"ID":"5aabb5b2-54bc-4893-91d2-3c0215fabd67","Type":"ContainerStarted","Data":"83259e653b77c32d76b07395c2ffbba505643fe81d4c971545e49d0ae752cd48"} Jul 16 00:41:11.919217 ip-10-0-143-200 kubenswrapper[2572]: I0716 00:41:11.919189 2572 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-cluster-samples-operator/cluster-samples-operator-65db858894-zf8lz" event={"ID":"5aabb5b2-54bc-4893-91d2-3c0215fabd67","Type":"ContainerStarted","Data":"5f190f85d8bf7d9952ba0f3149c9b5eb78df5e9e6b95cd0914887a3926d86a80"} Jul 16 00:41:11.925035 ip-10-0-143-200 kubenswrapper[2572]: I0716 00:41:11.925003 2572 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-dns/dns-default-w6cgc" event={"ID":"645fc509-972b-4332-8562-2309cc0eab1b","Type":"ContainerStarted","Data":"0ff84462b603869f3eeb307a4c27fd17abd2ae770a7395fbcc9f216331af09fa"} Jul 16 00:41:11.925166 ip-10-0-143-200 kubenswrapper[2572]: I0716 00:41:11.925065 2572 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-dns/dns-default-w6cgc" event={"ID":"645fc509-972b-4332-8562-2309cc0eab1b","Type":"ContainerStarted","Data":"17d7d9b33fb8bc284415a22475f41c224a43158de00b9492a2082256c5efeab0"} Jul 16 00:41:11.925783 ip-10-0-143-200 kubenswrapper[2572]: I0716 00:41:11.925721 2572 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-dns/dns-default-w6cgc" Jul 16 00:41:11.929379 ip-10-0-143-200 kubenswrapper[2572]: I0716 00:41:11.929141 2572 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kube-system/global-pull-secret-syncer-qcbcd" event={"ID":"a9bdff50-c8ce-4ca4-a810-20be3909b23c","Type":"ContainerStarted","Data":"63065d33069fc7072ef2cdd08e816f7c6de1687eeb0d6071a3ed496b6065cc18"} Jul 16 00:41:11.940972 ip-10-0-143-200 kubenswrapper[2572]: I0716 00:41:11.939252 2572 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-monitoring/cluster-monitoring-operator-5f7f494dd4-dxc26" podStartSLOduration=40.68275507 podStartE2EDuration="46.939231095s" totalImagesPullingTime="6.256476025s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-16 00:40:25 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-16 00:41:11.936588818 +0000 UTC m=+77.045343486" watchObservedRunningTime="2026-07-16 00:41:11.939231095 +0000 UTC m=+77.047985770" Jul 16 00:41:11.940972 ip-10-0-143-200 kubenswrapper[2572]: I0716 00:41:11.939673 2572 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="open-cluster-management-agent-addon/klusterlet-addon-workmgr-f5fff84cd-z7htr" event={"ID":"6d8c8acf-58c5-4ff4-9e27-a96773c67b32","Type":"ContainerStarted","Data":"3b950fe7a73085568a7d6afefc9e1dacaa9086cf985409e98dbc1000c9dc1868"} Jul 16 00:41:11.944588 ip-10-0-143-200 kubenswrapper[2572]: I0716 00:41:11.944526 2572 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-insights/insights-runtime-extractor-q8gql" event={"ID":"5f736cca-f28c-49b2-9dc9-24f0982e2d24","Type":"ContainerStarted","Data":"a965f948d5290710e951c7b07723a763fb09f370c981dab01afa8febd1816793"} Jul 16 00:41:11.947026 ip-10-0-143-200 kubenswrapper[2572]: I0716 00:41:11.946964 2572 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="open-cluster-management-agent-addon/managed-serviceaccount-addon-agent-f9cd9dfd6-jh8np" event={"ID":"636d7344-4105-43dd-9008-1910729748b4","Type":"ContainerStarted","Data":"8a3476a00ef1b8f90a596873719ba8fb8b43fe9a0539aae16aa04ae9fc44b0f6"} Jul 16 00:41:11.956874 ip-10-0-143-200 kubenswrapper[2572]: I0716 00:41:11.955869 2572 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="kube-system/global-pull-secret-syncer-qcbcd" podStartSLOduration=68.40825438 podStartE2EDuration="1m16.955847876s" totalImagesPullingTime="8.547593496s" totalInitContainerRuntime="0s" isStatefulPod=true podCreationTimestamp="2026-07-16 00:39:55 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-16 00:41:11.955780995 +0000 UTC m=+77.064535672" watchObservedRunningTime="2026-07-16 00:41:11.955847876 +0000 UTC m=+77.064602544" Jul 16 00:41:11.982558 ip-10-0-143-200 kubenswrapper[2572]: I0716 00:41:11.980858 2572 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-dns/dns-default-w6cgc" podStartSLOduration=34.669821445 podStartE2EDuration="41.980836352s" totalImagesPullingTime="7.311014907s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-16 00:40:30 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-16 00:41:11.979505077 +0000 UTC m=+77.088259775" watchObservedRunningTime="2026-07-16 00:41:11.980836352 +0000 UTC m=+77.089591030" Jul 16 00:41:12.013090 ip-10-0-143-200 kubenswrapper[2572]: I0716 00:41:12.012102 2572 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-network-console/networking-console-plugin-6cf856758c-vc87p" podStartSLOduration=35.774048253 podStartE2EDuration="42.012078471s" totalImagesPullingTime="6.238030218s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-16 00:40:30 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-16 00:41:12.011598507 +0000 UTC m=+77.120353183" watchObservedRunningTime="2026-07-16 00:41:12.012078471 +0000 UTC m=+77.120833128" Jul 16 00:41:12.042141 ip-10-0-143-200 kubenswrapper[2572]: I0716 00:41:12.040941 2572 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-cluster-samples-operator/cluster-samples-operator-65db858894-zf8lz" podStartSLOduration=40.149907702 podStartE2EDuration="47.040916467s" totalImagesPullingTime="6.891008765s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-16 00:40:25 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-16 00:41:12.038422481 +0000 UTC m=+77.147177168" watchObservedRunningTime="2026-07-16 00:41:12.040916467 +0000 UTC m=+77.149671142" Jul 16 00:41:12.147889 ip-10-0-143-200 kubenswrapper[2572]: I0716 00:41:12.147799 2572 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9gnmq_3a7e030d-fad8-480d-9dc6-fe5057048cf9/konnectivity-agent/0.log" limit=1024 Jul 16 00:41:12.248062 ip-10-0-143-200 kubenswrapper[2572]: I0716 00:41:12.247918 2572 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9gnmq_3a7e030d-fad8-480d-9dc6-fe5057048cf9/konnectivity-agent/0.log" limit=1024 Jul 16 00:41:13.446614 ip-10-0-143-200 kubenswrapper[2572]: I0716 00:41:13.446000 2572 scope.go:117] "RemoveContainer" containerID="032af45c0cea94209fd5ba66ef9e541cc34af375b80d46c5feea4a54ccac0a0a" Jul 16 00:41:13.446614 ip-10-0-143-200 kubenswrapper[2572]: E0716 00:41:13.446216 2572 pod_workers.go:1324] "Error syncing pod, skipping" err="failed to \"StartContainer\" for \"console-operator\" with CrashLoopBackOff: \"back-off 20s restarting failed container=console-operator pod=console-operator-7866497494-pjc6j_openshift-console-operator(a64bcd05-8596-4c12-b88e-e974d676e25f)\"" pod="openshift-console-operator/console-operator-7866497494-pjc6j" podUID="a64bcd05-8596-4c12-b88e-e974d676e25f" Jul 16 00:41:17.375982 ip-10-0-143-200 kubenswrapper[2572]: I0716 00:41:17.375840 2572 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9gnmq_3a7e030d-fad8-480d-9dc6-fe5057048cf9/konnectivity-agent/0.log" limit=1024 Jul 16 00:41:17.481726 ip-10-0-143-200 kubenswrapper[2572]: I0716 00:41:17.481687 2572 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9gnmq_3a7e030d-fad8-480d-9dc6-fe5057048cf9/konnectivity-agent/0.log" limit=1024 Jul 16 00:41:18.991786 ip-10-0-143-200 kubenswrapper[2572]: I0716 00:41:18.991743 2572 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="open-cluster-management-agent-addon/cluster-proxy-proxy-agent-599f598f85-4q8hh" event={"ID":"7f887d45-f998-499d-84da-8876b6800f46","Type":"ContainerStarted","Data":"067a0d3d3d588e7e59f706cb3fb91e64d73399cf79e1f285f39792a182f5b373"} Jul 16 00:41:18.993551 ip-10-0-143-200 kubenswrapper[2572]: I0716 00:41:18.993459 2572 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="open-cluster-management-agent-addon/klusterlet-addon-workmgr-f5fff84cd-z7htr" event={"ID":"6d8c8acf-58c5-4ff4-9e27-a96773c67b32","Type":"ContainerStarted","Data":"73450493522f60a88b7a39865f798a67d1685800eff49848d6bbe43dd7ac320d"} Jul 16 00:41:18.993824 ip-10-0-143-200 kubenswrapper[2572]: I0716 00:41:18.993805 2572 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="open-cluster-management-agent-addon/klusterlet-addon-workmgr-f5fff84cd-z7htr" Jul 16 00:41:18.995325 ip-10-0-143-200 kubenswrapper[2572]: I0716 00:41:18.995284 2572 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="open-cluster-management-agent-addon/klusterlet-addon-workmgr-f5fff84cd-z7htr" Jul 16 00:41:18.996131 ip-10-0-143-200 kubenswrapper[2572]: I0716 00:41:18.996106 2572 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-insights/insights-runtime-extractor-q8gql" event={"ID":"5f736cca-f28c-49b2-9dc9-24f0982e2d24","Type":"ContainerStarted","Data":"6504718120bbca8a0d74fd0a8e9d7d784672b952fbfd0a547a5a55a78da14532"} Jul 16 00:41:18.998009 ip-10-0-143-200 kubenswrapper[2572]: I0716 00:41:18.997987 2572 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="open-cluster-management-agent-addon/managed-serviceaccount-addon-agent-f9cd9dfd6-jh8np" event={"ID":"636d7344-4105-43dd-9008-1910729748b4","Type":"ContainerStarted","Data":"7896392f10d214683c3b938f31e632fd52548d6cd3cceb4fc3afacb525a168c0"} Jul 16 00:41:19.015399 ip-10-0-143-200 kubenswrapper[2572]: I0716 00:41:19.015323 2572 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="open-cluster-management-agent-addon/klusterlet-addon-workmgr-f5fff84cd-z7htr" podStartSLOduration=7.085579266 podStartE2EDuration="15.015302343s" totalImagesPullingTime="7.929723077s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-16 00:41:04 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-16 00:41:19.01420689 +0000 UTC m=+84.122961619" watchObservedRunningTime="2026-07-16 00:41:19.015302343 +0000 UTC m=+84.124057019" Jul 16 00:41:19.052007 ip-10-0-143-200 kubenswrapper[2572]: I0716 00:41:19.051948 2572 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="open-cluster-management-agent-addon/managed-serviceaccount-addon-agent-f9cd9dfd6-jh8np" podStartSLOduration=7.29500675 podStartE2EDuration="15.051925847s" totalImagesPullingTime="7.756919097s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-16 00:41:04 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-16 00:41:19.049576996 +0000 UTC m=+84.158331671" watchObservedRunningTime="2026-07-16 00:41:19.051925847 +0000 UTC m=+84.160680523" Jul 16 00:41:19.072325 ip-10-0-143-200 kubenswrapper[2572]: I0716 00:41:19.072271 2572 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-insights/insights-runtime-extractor-q8gql" podStartSLOduration=17.569388926 podStartE2EDuration="39.07225191s" totalImagesPullingTime="21.502862984s" totalInitContainerRuntime="0s" isStatefulPod=true podCreationTimestamp="2026-07-16 00:40:40 +0000 UTC" imagePullSessionsCount=3 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-16 00:41:19.071188217 +0000 UTC m=+84.179942892" watchObservedRunningTime="2026-07-16 00:41:19.07225191 +0000 UTC m=+84.181006585" Jul 16 00:41:19.635318 ip-10-0-143-200 kubenswrapper[2572]: I0716 00:41:19.635285 2572 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9gnmq_3a7e030d-fad8-480d-9dc6-fe5057048cf9/konnectivity-agent/0.log" limit=1024 Jul 16 00:41:19.745713 ip-10-0-143-200 kubenswrapper[2572]: I0716 00:41:19.745675 2572 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9gnmq_3a7e030d-fad8-480d-9dc6-fe5057048cf9/konnectivity-agent/0.log" limit=1024 Jul 16 00:41:20.004861 ip-10-0-143-200 kubenswrapper[2572]: I0716 00:41:20.004809 2572 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="open-cluster-management-agent-addon/cluster-proxy-proxy-agent-599f598f85-4q8hh" event={"ID":"7f887d45-f998-499d-84da-8876b6800f46","Type":"ContainerStarted","Data":"bd09a9742b4f5b44ee4f4bf95da395ee90f1872164fc6f4da5e9e8213ccb3b1b"} Jul 16 00:41:20.005338 ip-10-0-143-200 kubenswrapper[2572]: I0716 00:41:20.004877 2572 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="open-cluster-management-agent-addon/cluster-proxy-proxy-agent-599f598f85-4q8hh" event={"ID":"7f887d45-f998-499d-84da-8876b6800f46","Type":"ContainerStarted","Data":"a3e1e9bdaa023a1029ff62ed1f2b1b7b6340b4dda6e4f7b44709e6c025371f47"} Jul 16 00:41:20.030578 ip-10-0-143-200 kubenswrapper[2572]: I0716 00:41:20.030513 2572 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="open-cluster-management-agent-addon/cluster-proxy-proxy-agent-599f598f85-4q8hh" podStartSLOduration=7.940307512 podStartE2EDuration="16.030492798s" totalImagesPullingTime="8.090185286s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-16 00:41:04 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-16 00:41:20.029024888 +0000 UTC m=+85.137779555" watchObservedRunningTime="2026-07-16 00:41:20.030492798 +0000 UTC m=+85.139247478" Jul 16 00:41:20.120923 ip-10-0-143-200 kubenswrapper[2572]: I0716 00:41:20.120886 2572 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9gnmq_3a7e030d-fad8-480d-9dc6-fe5057048cf9/konnectivity-agent/0.log" limit=1024 Jul 16 00:41:22.810554 ip-10-0-143-200 kubenswrapper[2572]: I0716 00:41:22.810516 2572 patch_prober.go:28] interesting pod/image-registry-dd7c989d9-5w25c container/registry namespace/openshift-image-registry: Liveness probe status=failure output="HTTP probe failed with statuscode: 503" start-of-body={"errors":[{"code":"UNAVAILABLE","message":"service unavailable","detail":"health check failed: please see /debug/health"}]} Jul 16 00:41:22.810948 ip-10-0-143-200 kubenswrapper[2572]: I0716 00:41:22.810584 2572 prober.go:120] "Probe failed" probeType="Liveness" pod="openshift-image-registry/image-registry-dd7c989d9-5w25c" podUID="f103d7a8-2560-409b-801e-701bcc6835c1" containerName="registry" probeResult="failure" output="HTTP probe failed with statuscode: 503" Jul 16 00:41:22.968106 ip-10-0-143-200 kubenswrapper[2572]: I0716 00:41:22.968077 2572 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-dns/dns-default-w6cgc" Jul 16 00:41:23.021259 ip-10-0-143-200 kubenswrapper[2572]: I0716 00:41:23.021221 2572 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9gnmq_3a7e030d-fad8-480d-9dc6-fe5057048cf9/konnectivity-agent/0.log" limit=1024 Jul 16 00:41:23.118445 ip-10-0-143-200 kubenswrapper[2572]: I0716 00:41:23.118349 2572 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9gnmq_3a7e030d-fad8-480d-9dc6-fe5057048cf9/konnectivity-agent/0.log" limit=1024 Jul 16 00:41:23.472614 ip-10-0-143-200 kubenswrapper[2572]: I0716 00:41:23.472579 2572 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9gnmq_3a7e030d-fad8-480d-9dc6-fe5057048cf9/konnectivity-agent/0.log" limit=1024 Jul 16 00:41:23.906257 ip-10-0-143-200 kubenswrapper[2572]: I0716 00:41:23.906163 2572 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9gnmq_3a7e030d-fad8-480d-9dc6-fe5057048cf9/konnectivity-agent/0.log" limit=1024 Jul 16 00:41:24.025116 ip-10-0-143-200 kubenswrapper[2572]: I0716 00:41:24.025082 2572 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9gnmq_3a7e030d-fad8-480d-9dc6-fe5057048cf9/konnectivity-agent/0.log" limit=1024 Jul 16 00:41:24.876533 ip-10-0-143-200 kubenswrapper[2572]: I0716 00:41:24.876500 2572 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-image-registry/image-registry-dd7c989d9-5w25c" Jul 16 00:41:25.248371 ip-10-0-143-200 kubenswrapper[2572]: I0716 00:41:25.248333 2572 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9gnmq_3a7e030d-fad8-480d-9dc6-fe5057048cf9/konnectivity-agent/0.log" limit=1024 Jul 16 00:41:25.347427 ip-10-0-143-200 kubenswrapper[2572]: I0716 00:41:25.347389 2572 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9gnmq_3a7e030d-fad8-480d-9dc6-fe5057048cf9/konnectivity-agent/0.log" limit=1024 Jul 16 00:41:25.443774 ip-10-0-143-200 kubenswrapper[2572]: I0716 00:41:25.443750 2572 scope.go:117] "RemoveContainer" containerID="032af45c0cea94209fd5ba66ef9e541cc34af375b80d46c5feea4a54ccac0a0a" Jul 16 00:41:25.783323 ip-10-0-143-200 kubenswrapper[2572]: I0716 00:41:25.783289 2572 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9gnmq_3a7e030d-fad8-480d-9dc6-fe5057048cf9/konnectivity-agent/0.log" limit=1024 Jul 16 00:41:25.994669 ip-10-0-143-200 kubenswrapper[2572]: I0716 00:41:25.994621 2572 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9gnmq_3a7e030d-fad8-480d-9dc6-fe5057048cf9/konnectivity-agent/0.log" limit=1024 Jul 16 00:41:26.035778 ip-10-0-143-200 kubenswrapper[2572]: I0716 00:41:26.035686 2572 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-console-operator_console-operator-7866497494-pjc6j_a64bcd05-8596-4c12-b88e-e974d676e25f/console-operator/2.log" Jul 16 00:41:26.035951 ip-10-0-143-200 kubenswrapper[2572]: I0716 00:41:26.035829 2572 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-console-operator/console-operator-7866497494-pjc6j" event={"ID":"a64bcd05-8596-4c12-b88e-e974d676e25f","Type":"ContainerStarted","Data":"a5e4db05c6bea6645495dea5c0206d432a14c018bd408236107165687f60caec"} Jul 16 00:41:26.036421 ip-10-0-143-200 kubenswrapper[2572]: I0716 00:41:26.036398 2572 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-console-operator/console-operator-7866497494-pjc6j" Jul 16 00:41:26.065414 ip-10-0-143-200 kubenswrapper[2572]: I0716 00:41:26.065380 2572 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-monitoring/node-exporter-kwdjw"] Jul 16 00:41:26.068711 ip-10-0-143-200 kubenswrapper[2572]: I0716 00:41:26.068686 2572 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-monitoring/node-exporter-kwdjw" Jul 16 00:41:26.068966 ip-10-0-143-200 kubenswrapper[2572]: I0716 00:41:26.068913 2572 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-console-operator/console-operator-7866497494-pjc6j" podStartSLOduration=53.249956367 podStartE2EDuration="1m1.068895616s" totalImagesPullingTime="7.818939249s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-16 00:40:25 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-16 00:41:26.067881388 +0000 UTC m=+91.176636065" watchObservedRunningTime="2026-07-16 00:41:26.068895616 +0000 UTC m=+91.177650294" Jul 16 00:41:26.071973 ip-10-0-143-200 kubenswrapper[2572]: I0716 00:41:26.071950 2572 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"node-exporter-kube-rbac-proxy-config\"" Jul 16 00:41:26.072188 ip-10-0-143-200 kubenswrapper[2572]: I0716 00:41:26.072166 2572 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-monitoring\"/\"node-exporter-accelerators-collector-config\"" Jul 16 00:41:26.072371 ip-10-0-143-200 kubenswrapper[2572]: I0716 00:41:26.071959 2572 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"node-exporter-dockercfg-7s2vh\"" Jul 16 00:41:26.072436 ip-10-0-143-200 kubenswrapper[2572]: I0716 00:41:26.072380 2572 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-monitoring\"/\"metrics-client-ca\"" Jul 16 00:41:26.072487 ip-10-0-143-200 kubenswrapper[2572]: I0716 00:41:26.072006 2572 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"node-exporter-tls\"" Jul 16 00:41:26.109122 ip-10-0-143-200 kubenswrapper[2572]: I0716 00:41:26.109090 2572 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9gnmq_3a7e030d-fad8-480d-9dc6-fe5057048cf9/konnectivity-agent/0.log" limit=1024 Jul 16 00:41:26.157090 ip-10-0-143-200 kubenswrapper[2572]: I0716 00:41:26.157050 2572 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"node-exporter-tls\" (UniqueName: \"kubernetes.io/secret/49f06593-a9c5-408c-81e8-387df5345438-node-exporter-tls\") pod \"node-exporter-kwdjw\" (UID: \"49f06593-a9c5-408c-81e8-387df5345438\") " pod="openshift-monitoring/node-exporter-kwdjw" Jul 16 00:41:26.157261 ip-10-0-143-200 kubenswrapper[2572]: I0716 00:41:26.157115 2572 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"node-exporter-wtmp\" (UniqueName: \"kubernetes.io/host-path/49f06593-a9c5-408c-81e8-387df5345438-node-exporter-wtmp\") pod \"node-exporter-kwdjw\" (UID: \"49f06593-a9c5-408c-81e8-387df5345438\") " pod="openshift-monitoring/node-exporter-kwdjw" Jul 16 00:41:26.157261 ip-10-0-143-200 kubenswrapper[2572]: I0716 00:41:26.157158 2572 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"node-exporter-accelerators-collector-config\" (UniqueName: \"kubernetes.io/configmap/49f06593-a9c5-408c-81e8-387df5345438-node-exporter-accelerators-collector-config\") pod \"node-exporter-kwdjw\" (UID: \"49f06593-a9c5-408c-81e8-387df5345438\") " pod="openshift-monitoring/node-exporter-kwdjw" Jul 16 00:41:26.157261 ip-10-0-143-200 kubenswrapper[2572]: I0716 00:41:26.157201 2572 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"sys\" (UniqueName: \"kubernetes.io/host-path/49f06593-a9c5-408c-81e8-387df5345438-sys\") pod \"node-exporter-kwdjw\" (UID: \"49f06593-a9c5-408c-81e8-387df5345438\") " pod="openshift-monitoring/node-exporter-kwdjw" Jul 16 00:41:26.157261 ip-10-0-143-200 kubenswrapper[2572]: I0716 00:41:26.157242 2572 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"root\" (UniqueName: \"kubernetes.io/host-path/49f06593-a9c5-408c-81e8-387df5345438-root\") pod \"node-exporter-kwdjw\" (UID: \"49f06593-a9c5-408c-81e8-387df5345438\") " pod="openshift-monitoring/node-exporter-kwdjw" Jul 16 00:41:26.157496 ip-10-0-143-200 kubenswrapper[2572]: I0716 00:41:26.157266 2572 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"node-exporter-textfile\" (UniqueName: \"kubernetes.io/empty-dir/49f06593-a9c5-408c-81e8-387df5345438-node-exporter-textfile\") pod \"node-exporter-kwdjw\" (UID: \"49f06593-a9c5-408c-81e8-387df5345438\") " pod="openshift-monitoring/node-exporter-kwdjw" Jul 16 00:41:26.157496 ip-10-0-143-200 kubenswrapper[2572]: I0716 00:41:26.157286 2572 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"metrics-client-ca\" (UniqueName: \"kubernetes.io/configmap/49f06593-a9c5-408c-81e8-387df5345438-metrics-client-ca\") pod \"node-exporter-kwdjw\" (UID: \"49f06593-a9c5-408c-81e8-387df5345438\") " pod="openshift-monitoring/node-exporter-kwdjw" Jul 16 00:41:26.157496 ip-10-0-143-200 kubenswrapper[2572]: I0716 00:41:26.157348 2572 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"node-exporter-kube-rbac-proxy-config\" (UniqueName: \"kubernetes.io/secret/49f06593-a9c5-408c-81e8-387df5345438-node-exporter-kube-rbac-proxy-config\") pod \"node-exporter-kwdjw\" (UID: \"49f06593-a9c5-408c-81e8-387df5345438\") " pod="openshift-monitoring/node-exporter-kwdjw" Jul 16 00:41:26.157496 ip-10-0-143-200 kubenswrapper[2572]: I0716 00:41:26.157404 2572 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-56v8t\" (UniqueName: \"kubernetes.io/projected/49f06593-a9c5-408c-81e8-387df5345438-kube-api-access-56v8t\") pod \"node-exporter-kwdjw\" (UID: \"49f06593-a9c5-408c-81e8-387df5345438\") " pod="openshift-monitoring/node-exporter-kwdjw" Jul 16 00:41:26.209099 ip-10-0-143-200 kubenswrapper[2572]: I0716 00:41:26.209057 2572 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9gnmq_3a7e030d-fad8-480d-9dc6-fe5057048cf9/konnectivity-agent/0.log" limit=1024 Jul 16 00:41:26.258718 ip-10-0-143-200 kubenswrapper[2572]: I0716 00:41:26.258670 2572 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"node-exporter-kube-rbac-proxy-config\" (UniqueName: \"kubernetes.io/secret/49f06593-a9c5-408c-81e8-387df5345438-node-exporter-kube-rbac-proxy-config\") pod \"node-exporter-kwdjw\" (UID: \"49f06593-a9c5-408c-81e8-387df5345438\") " pod="openshift-monitoring/node-exporter-kwdjw" Jul 16 00:41:26.259188 ip-10-0-143-200 kubenswrapper[2572]: I0716 00:41:26.258738 2572 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-56v8t\" (UniqueName: \"kubernetes.io/projected/49f06593-a9c5-408c-81e8-387df5345438-kube-api-access-56v8t\") pod \"node-exporter-kwdjw\" (UID: \"49f06593-a9c5-408c-81e8-387df5345438\") " pod="openshift-monitoring/node-exporter-kwdjw" Jul 16 00:41:26.259188 ip-10-0-143-200 kubenswrapper[2572]: I0716 00:41:26.258789 2572 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"node-exporter-tls\" (UniqueName: \"kubernetes.io/secret/49f06593-a9c5-408c-81e8-387df5345438-node-exporter-tls\") pod \"node-exporter-kwdjw\" (UID: \"49f06593-a9c5-408c-81e8-387df5345438\") " pod="openshift-monitoring/node-exporter-kwdjw" Jul 16 00:41:26.259188 ip-10-0-143-200 kubenswrapper[2572]: I0716 00:41:26.258817 2572 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"node-exporter-wtmp\" (UniqueName: \"kubernetes.io/host-path/49f06593-a9c5-408c-81e8-387df5345438-node-exporter-wtmp\") pod \"node-exporter-kwdjw\" (UID: \"49f06593-a9c5-408c-81e8-387df5345438\") " pod="openshift-monitoring/node-exporter-kwdjw" Jul 16 00:41:26.259188 ip-10-0-143-200 kubenswrapper[2572]: I0716 00:41:26.258857 2572 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"node-exporter-accelerators-collector-config\" (UniqueName: \"kubernetes.io/configmap/49f06593-a9c5-408c-81e8-387df5345438-node-exporter-accelerators-collector-config\") pod \"node-exporter-kwdjw\" (UID: \"49f06593-a9c5-408c-81e8-387df5345438\") " pod="openshift-monitoring/node-exporter-kwdjw" Jul 16 00:41:26.259188 ip-10-0-143-200 kubenswrapper[2572]: I0716 00:41:26.258887 2572 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"sys\" (UniqueName: \"kubernetes.io/host-path/49f06593-a9c5-408c-81e8-387df5345438-sys\") pod \"node-exporter-kwdjw\" (UID: \"49f06593-a9c5-408c-81e8-387df5345438\") " pod="openshift-monitoring/node-exporter-kwdjw" Jul 16 00:41:26.259188 ip-10-0-143-200 kubenswrapper[2572]: E0716 00:41:26.258914 2572 secret.go:189] Couldn't get secret openshift-monitoring/node-exporter-tls: secret "node-exporter-tls" not found Jul 16 00:41:26.259188 ip-10-0-143-200 kubenswrapper[2572]: I0716 00:41:26.258958 2572 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"root\" (UniqueName: \"kubernetes.io/host-path/49f06593-a9c5-408c-81e8-387df5345438-root\") pod \"node-exporter-kwdjw\" (UID: \"49f06593-a9c5-408c-81e8-387df5345438\") " pod="openshift-monitoring/node-exporter-kwdjw" Jul 16 00:41:26.259188 ip-10-0-143-200 kubenswrapper[2572]: E0716 00:41:26.258984 2572 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/49f06593-a9c5-408c-81e8-387df5345438-node-exporter-tls podName:49f06593-a9c5-408c-81e8-387df5345438 nodeName:}" failed. No retries permitted until 2026-07-16 00:41:26.758963542 +0000 UTC m=+91.867718208 (durationBeforeRetry 500ms). Error: MountVolume.SetUp failed for volume "node-exporter-tls" (UniqueName: "kubernetes.io/secret/49f06593-a9c5-408c-81e8-387df5345438-node-exporter-tls") pod "node-exporter-kwdjw" (UID: "49f06593-a9c5-408c-81e8-387df5345438") : secret "node-exporter-tls" not found Jul 16 00:41:26.259188 ip-10-0-143-200 kubenswrapper[2572]: I0716 00:41:26.258916 2572 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"root\" (UniqueName: \"kubernetes.io/host-path/49f06593-a9c5-408c-81e8-387df5345438-root\") pod \"node-exporter-kwdjw\" (UID: \"49f06593-a9c5-408c-81e8-387df5345438\") " pod="openshift-monitoring/node-exporter-kwdjw" Jul 16 00:41:26.259188 ip-10-0-143-200 kubenswrapper[2572]: I0716 00:41:26.259026 2572 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"node-exporter-textfile\" (UniqueName: \"kubernetes.io/empty-dir/49f06593-a9c5-408c-81e8-387df5345438-node-exporter-textfile\") pod \"node-exporter-kwdjw\" (UID: \"49f06593-a9c5-408c-81e8-387df5345438\") " pod="openshift-monitoring/node-exporter-kwdjw" Jul 16 00:41:26.259188 ip-10-0-143-200 kubenswrapper[2572]: I0716 00:41:26.259054 2572 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-client-ca\" (UniqueName: \"kubernetes.io/configmap/49f06593-a9c5-408c-81e8-387df5345438-metrics-client-ca\") pod \"node-exporter-kwdjw\" (UID: \"49f06593-a9c5-408c-81e8-387df5345438\") " pod="openshift-monitoring/node-exporter-kwdjw" Jul 16 00:41:26.259188 ip-10-0-143-200 kubenswrapper[2572]: I0716 00:41:26.259079 2572 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"node-exporter-wtmp\" (UniqueName: \"kubernetes.io/host-path/49f06593-a9c5-408c-81e8-387df5345438-node-exporter-wtmp\") pod \"node-exporter-kwdjw\" (UID: \"49f06593-a9c5-408c-81e8-387df5345438\") " pod="openshift-monitoring/node-exporter-kwdjw" Jul 16 00:41:26.259188 ip-10-0-143-200 kubenswrapper[2572]: I0716 00:41:26.259130 2572 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"sys\" (UniqueName: \"kubernetes.io/host-path/49f06593-a9c5-408c-81e8-387df5345438-sys\") pod \"node-exporter-kwdjw\" (UID: \"49f06593-a9c5-408c-81e8-387df5345438\") " pod="openshift-monitoring/node-exporter-kwdjw" Jul 16 00:41:26.259878 ip-10-0-143-200 kubenswrapper[2572]: I0716 00:41:26.259437 2572 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"node-exporter-textfile\" (UniqueName: \"kubernetes.io/empty-dir/49f06593-a9c5-408c-81e8-387df5345438-node-exporter-textfile\") pod \"node-exporter-kwdjw\" (UID: \"49f06593-a9c5-408c-81e8-387df5345438\") " pod="openshift-monitoring/node-exporter-kwdjw" Jul 16 00:41:26.259878 ip-10-0-143-200 kubenswrapper[2572]: I0716 00:41:26.259664 2572 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"metrics-client-ca\" (UniqueName: \"kubernetes.io/configmap/49f06593-a9c5-408c-81e8-387df5345438-metrics-client-ca\") pod \"node-exporter-kwdjw\" (UID: \"49f06593-a9c5-408c-81e8-387df5345438\") " pod="openshift-monitoring/node-exporter-kwdjw" Jul 16 00:41:26.259981 ip-10-0-143-200 kubenswrapper[2572]: I0716 00:41:26.259967 2572 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"node-exporter-accelerators-collector-config\" (UniqueName: \"kubernetes.io/configmap/49f06593-a9c5-408c-81e8-387df5345438-node-exporter-accelerators-collector-config\") pod \"node-exporter-kwdjw\" (UID: \"49f06593-a9c5-408c-81e8-387df5345438\") " pod="openshift-monitoring/node-exporter-kwdjw" Jul 16 00:41:26.261709 ip-10-0-143-200 kubenswrapper[2572]: I0716 00:41:26.261667 2572 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"node-exporter-kube-rbac-proxy-config\" (UniqueName: \"kubernetes.io/secret/49f06593-a9c5-408c-81e8-387df5345438-node-exporter-kube-rbac-proxy-config\") pod \"node-exporter-kwdjw\" (UID: \"49f06593-a9c5-408c-81e8-387df5345438\") " pod="openshift-monitoring/node-exporter-kwdjw" Jul 16 00:41:26.281876 ip-10-0-143-200 kubenswrapper[2572]: I0716 00:41:26.281789 2572 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-56v8t\" (UniqueName: \"kubernetes.io/projected/49f06593-a9c5-408c-81e8-387df5345438-kube-api-access-56v8t\") pod \"node-exporter-kwdjw\" (UID: \"49f06593-a9c5-408c-81e8-387df5345438\") " pod="openshift-monitoring/node-exporter-kwdjw" Jul 16 00:41:26.298891 ip-10-0-143-200 kubenswrapper[2572]: I0716 00:41:26.298797 2572 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9gnmq_3a7e030d-fad8-480d-9dc6-fe5057048cf9/konnectivity-agent/0.log" limit=1024 Jul 16 00:41:26.369019 ip-10-0-143-200 kubenswrapper[2572]: I0716 00:41:26.368980 2572 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9gnmq_3a7e030d-fad8-480d-9dc6-fe5057048cf9/konnectivity-agent/0.log" limit=1024 Jul 16 00:41:26.468692 ip-10-0-143-200 kubenswrapper[2572]: I0716 00:41:26.468653 2572 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9gnmq_3a7e030d-fad8-480d-9dc6-fe5057048cf9/konnectivity-agent/0.log" limit=1024 Jul 16 00:41:26.531531 ip-10-0-143-200 kubenswrapper[2572]: I0716 00:41:26.531500 2572 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-console-operator/console-operator-7866497494-pjc6j" Jul 16 00:41:26.563685 ip-10-0-143-200 kubenswrapper[2572]: I0716 00:41:26.563569 2572 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9gnmq_3a7e030d-fad8-480d-9dc6-fe5057048cf9/konnectivity-agent/0.log" limit=1024 Jul 16 00:41:26.647662 ip-10-0-143-200 kubenswrapper[2572]: I0716 00:41:26.647002 2572 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9gnmq_3a7e030d-fad8-480d-9dc6-fe5057048cf9/konnectivity-agent/0.log" limit=1024 Jul 16 00:41:26.742920 ip-10-0-143-200 kubenswrapper[2572]: I0716 00:41:26.742881 2572 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9gnmq_3a7e030d-fad8-480d-9dc6-fe5057048cf9/konnectivity-agent/0.log" limit=1024 Jul 16 00:41:26.764969 ip-10-0-143-200 kubenswrapper[2572]: I0716 00:41:26.764922 2572 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"node-exporter-tls\" (UniqueName: \"kubernetes.io/secret/49f06593-a9c5-408c-81e8-387df5345438-node-exporter-tls\") pod \"node-exporter-kwdjw\" (UID: \"49f06593-a9c5-408c-81e8-387df5345438\") " pod="openshift-monitoring/node-exporter-kwdjw" Jul 16 00:41:26.767354 ip-10-0-143-200 kubenswrapper[2572]: I0716 00:41:26.767326 2572 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"node-exporter-tls\" (UniqueName: \"kubernetes.io/secret/49f06593-a9c5-408c-81e8-387df5345438-node-exporter-tls\") pod \"node-exporter-kwdjw\" (UID: \"49f06593-a9c5-408c-81e8-387df5345438\") " pod="openshift-monitoring/node-exporter-kwdjw" Jul 16 00:41:26.840041 ip-10-0-143-200 kubenswrapper[2572]: I0716 00:41:26.839927 2572 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9gnmq_3a7e030d-fad8-480d-9dc6-fe5057048cf9/konnectivity-agent/0.log" limit=1024 Jul 16 00:41:26.983142 ip-10-0-143-200 kubenswrapper[2572]: I0716 00:41:26.983111 2572 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-monitoring/node-exporter-kwdjw" Jul 16 00:41:26.999944 ip-10-0-143-200 kubenswrapper[2572]: W0716 00:41:26.999906 2572 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod49f06593_a9c5_408c_81e8_387df5345438.slice/crio-031ee19ea4ef3d7aeb6888782d84c53a792431418ca2da4287be842c069f7cbb WatchSource:0}: Error finding container 031ee19ea4ef3d7aeb6888782d84c53a792431418ca2da4287be842c069f7cbb: Status 404 returned error can't find the container with id 031ee19ea4ef3d7aeb6888782d84c53a792431418ca2da4287be842c069f7cbb Jul 16 00:41:27.037734 ip-10-0-143-200 kubenswrapper[2572]: I0716 00:41:27.037699 2572 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9gnmq_3a7e030d-fad8-480d-9dc6-fe5057048cf9/konnectivity-agent/0.log" limit=1024 Jul 16 00:41:27.041696 ip-10-0-143-200 kubenswrapper[2572]: I0716 00:41:27.041666 2572 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/node-exporter-kwdjw" event={"ID":"49f06593-a9c5-408c-81e8-387df5345438","Type":"ContainerStarted","Data":"031ee19ea4ef3d7aeb6888782d84c53a792431418ca2da4287be842c069f7cbb"} Jul 16 00:41:27.125450 ip-10-0-143-200 kubenswrapper[2572]: I0716 00:41:27.125344 2572 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9gnmq_3a7e030d-fad8-480d-9dc6-fe5057048cf9/konnectivity-agent/0.log" limit=1024 Jul 16 00:41:27.230592 ip-10-0-143-200 kubenswrapper[2572]: I0716 00:41:27.230552 2572 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9gnmq_3a7e030d-fad8-480d-9dc6-fe5057048cf9/konnectivity-agent/0.log" limit=1024 Jul 16 00:41:27.304335 ip-10-0-143-200 kubenswrapper[2572]: I0716 00:41:27.304304 2572 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9gnmq_3a7e030d-fad8-480d-9dc6-fe5057048cf9/konnectivity-agent/0.log" limit=1024 Jul 16 00:41:27.426644 ip-10-0-143-200 kubenswrapper[2572]: I0716 00:41:27.426530 2572 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9gnmq_3a7e030d-fad8-480d-9dc6-fe5057048cf9/konnectivity-agent/0.log" limit=1024 Jul 16 00:41:27.511085 ip-10-0-143-200 kubenswrapper[2572]: I0716 00:41:27.511039 2572 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9gnmq_3a7e030d-fad8-480d-9dc6-fe5057048cf9/konnectivity-agent/0.log" limit=1024 Jul 16 00:41:27.858442 ip-10-0-143-200 kubenswrapper[2572]: I0716 00:41:27.858405 2572 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9gnmq_3a7e030d-fad8-480d-9dc6-fe5057048cf9/konnectivity-agent/0.log" limit=1024 Jul 16 00:41:28.048257 ip-10-0-143-200 kubenswrapper[2572]: I0716 00:41:28.048221 2572 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/node-exporter-kwdjw" event={"ID":"49f06593-a9c5-408c-81e8-387df5345438","Type":"ContainerStarted","Data":"7050994e7c94c0cb7c3197cc57b00600a63a43f5bbd5ddd4c4c4b5c10729f989"} Jul 16 00:41:28.052097 ip-10-0-143-200 kubenswrapper[2572]: I0716 00:41:28.052057 2572 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9gnmq_3a7e030d-fad8-480d-9dc6-fe5057048cf9/konnectivity-agent/0.log" limit=1024 Jul 16 00:41:28.256055 ip-10-0-143-200 kubenswrapper[2572]: I0716 00:41:28.256021 2572 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9gnmq_3a7e030d-fad8-480d-9dc6-fe5057048cf9/konnectivity-agent/0.log" limit=1024 Jul 16 00:41:28.421187 ip-10-0-143-200 kubenswrapper[2572]: I0716 00:41:28.421136 2572 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9gnmq_3a7e030d-fad8-480d-9dc6-fe5057048cf9/konnectivity-agent/0.log" limit=1024 Jul 16 00:41:28.596745 ip-10-0-143-200 kubenswrapper[2572]: I0716 00:41:28.596647 2572 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9gnmq_3a7e030d-fad8-480d-9dc6-fe5057048cf9/konnectivity-agent/0.log" limit=1024 Jul 16 00:41:28.817052 ip-10-0-143-200 kubenswrapper[2572]: I0716 00:41:28.817018 2572 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9gnmq_3a7e030d-fad8-480d-9dc6-fe5057048cf9/konnectivity-agent/0.log" limit=1024 Jul 16 00:41:29.054439 ip-10-0-143-200 kubenswrapper[2572]: I0716 00:41:29.054402 2572 generic.go:356] "Generic (PLEG): container finished" podID="49f06593-a9c5-408c-81e8-387df5345438" containerID="7050994e7c94c0cb7c3197cc57b00600a63a43f5bbd5ddd4c4c4b5c10729f989" exitCode=0 Jul 16 00:41:29.054649 ip-10-0-143-200 kubenswrapper[2572]: I0716 00:41:29.054488 2572 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/node-exporter-kwdjw" event={"ID":"49f06593-a9c5-408c-81e8-387df5345438","Type":"ContainerDied","Data":"7050994e7c94c0cb7c3197cc57b00600a63a43f5bbd5ddd4c4c4b5c10729f989"} Jul 16 00:41:29.065802 ip-10-0-143-200 kubenswrapper[2572]: I0716 00:41:29.065769 2572 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9gnmq_3a7e030d-fad8-480d-9dc6-fe5057048cf9/konnectivity-agent/0.log" limit=1024 Jul 16 00:41:29.132747 ip-10-0-143-200 kubenswrapper[2572]: I0716 00:41:29.132720 2572 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9gnmq_3a7e030d-fad8-480d-9dc6-fe5057048cf9/konnectivity-agent/0.log" limit=1024 Jul 16 00:41:29.271693 ip-10-0-143-200 kubenswrapper[2572]: I0716 00:41:29.271661 2572 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9gnmq_3a7e030d-fad8-480d-9dc6-fe5057048cf9/konnectivity-agent/0.log" limit=1024 Jul 16 00:41:29.369239 ip-10-0-143-200 kubenswrapper[2572]: I0716 00:41:29.369157 2572 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9gnmq_3a7e030d-fad8-480d-9dc6-fe5057048cf9/konnectivity-agent/0.log" limit=1024 Jul 16 00:41:29.468295 ip-10-0-143-200 kubenswrapper[2572]: I0716 00:41:29.468259 2572 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9gnmq_3a7e030d-fad8-480d-9dc6-fe5057048cf9/konnectivity-agent/0.log" limit=1024 Jul 16 00:41:30.060207 ip-10-0-143-200 kubenswrapper[2572]: I0716 00:41:30.060167 2572 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/node-exporter-kwdjw" event={"ID":"49f06593-a9c5-408c-81e8-387df5345438","Type":"ContainerStarted","Data":"c5b19d677b033f3d8315c4907f0e08eea838a3d0f7b517e2f4fd2cf393892e34"} Jul 16 00:41:30.060375 ip-10-0-143-200 kubenswrapper[2572]: I0716 00:41:30.060229 2572 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/node-exporter-kwdjw" event={"ID":"49f06593-a9c5-408c-81e8-387df5345438","Type":"ContainerStarted","Data":"bad47216e5ff88da0b737aeaa6e80d076f61dfa0cb57b57f91ce3b2f4bf4f0e7"} Jul 16 00:41:30.081485 ip-10-0-143-200 kubenswrapper[2572]: I0716 00:41:30.081405 2572 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-monitoring/node-exporter-kwdjw" podStartSLOduration=3.099706035 podStartE2EDuration="4.081389967s" totalImagesPullingTime="914.924821ms" totalInitContainerRuntime="66.759111ms" isStatefulPod=true podCreationTimestamp="2026-07-16 00:41:26 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-16 00:41:30.079155718 +0000 UTC m=+95.187910406" watchObservedRunningTime="2026-07-16 00:41:30.081389967 +0000 UTC m=+95.190144679" Jul 16 00:41:30.133365 ip-10-0-143-200 kubenswrapper[2572]: I0716 00:41:30.133334 2572 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9gnmq_3a7e030d-fad8-480d-9dc6-fe5057048cf9/konnectivity-agent/0.log" limit=1024 Jul 16 00:41:30.230418 ip-10-0-143-200 kubenswrapper[2572]: I0716 00:41:30.230378 2572 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9gnmq_3a7e030d-fad8-480d-9dc6-fe5057048cf9/konnectivity-agent/0.log" limit=1024 Jul 16 00:41:30.485882 ip-10-0-143-200 kubenswrapper[2572]: I0716 00:41:30.485848 2572 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9gnmq_3a7e030d-fad8-480d-9dc6-fe5057048cf9/konnectivity-agent/0.log" limit=1024 Jul 16 00:41:30.559176 ip-10-0-143-200 kubenswrapper[2572]: I0716 00:41:30.559140 2572 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9gnmq_3a7e030d-fad8-480d-9dc6-fe5057048cf9/konnectivity-agent/0.log" limit=1024 Jul 16 00:41:30.800455 ip-10-0-143-200 kubenswrapper[2572]: I0716 00:41:30.800366 2572 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9gnmq_3a7e030d-fad8-480d-9dc6-fe5057048cf9/konnectivity-agent/0.log" limit=1024 Jul 16 00:41:30.879559 ip-10-0-143-200 kubenswrapper[2572]: I0716 00:41:30.879523 2572 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9gnmq_3a7e030d-fad8-480d-9dc6-fe5057048cf9/konnectivity-agent/0.log" limit=1024 Jul 16 00:41:31.015579 ip-10-0-143-200 kubenswrapper[2572]: I0716 00:41:31.015545 2572 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9gnmq_3a7e030d-fad8-480d-9dc6-fe5057048cf9/konnectivity-agent/0.log" limit=1024 Jul 16 00:41:32.210345 ip-10-0-143-200 kubenswrapper[2572]: I0716 00:41:32.210308 2572 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9gnmq_3a7e030d-fad8-480d-9dc6-fe5057048cf9/konnectivity-agent/0.log" limit=1024 Jul 16 00:41:32.320212 ip-10-0-143-200 kubenswrapper[2572]: I0716 00:41:32.320173 2572 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9gnmq_3a7e030d-fad8-480d-9dc6-fe5057048cf9/konnectivity-agent/0.log" limit=1024 Jul 16 00:41:32.372077 ip-10-0-143-200 kubenswrapper[2572]: I0716 00:41:32.372036 2572 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-image-registry/image-registry-dd7c989d9-5w25c"] Jul 16 00:41:32.422326 ip-10-0-143-200 kubenswrapper[2572]: I0716 00:41:32.422291 2572 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9gnmq_3a7e030d-fad8-480d-9dc6-fe5057048cf9/konnectivity-agent/0.log" limit=1024 Jul 16 00:41:32.516831 ip-10-0-143-200 kubenswrapper[2572]: I0716 00:41:32.516741 2572 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9gnmq_3a7e030d-fad8-480d-9dc6-fe5057048cf9/konnectivity-agent/0.log" limit=1024 Jul 16 00:41:32.617178 ip-10-0-143-200 kubenswrapper[2572]: I0716 00:41:32.617143 2572 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9gnmq_3a7e030d-fad8-480d-9dc6-fe5057048cf9/konnectivity-agent/0.log" limit=1024 Jul 16 00:41:33.860309 ip-10-0-143-200 kubenswrapper[2572]: I0716 00:41:33.860280 2572 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-network-diagnostics/network-check-target-52n54" Jul 16 00:41:34.468090 ip-10-0-143-200 kubenswrapper[2572]: I0716 00:41:34.468051 2572 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9gnmq_3a7e030d-fad8-480d-9dc6-fe5057048cf9/konnectivity-agent/0.log" limit=1024 Jul 16 00:41:34.569166 ip-10-0-143-200 kubenswrapper[2572]: I0716 00:41:34.569127 2572 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9gnmq_3a7e030d-fad8-480d-9dc6-fe5057048cf9/konnectivity-agent/0.log" limit=1024 Jul 16 00:41:35.149258 ip-10-0-143-200 kubenswrapper[2572]: I0716 00:41:35.149225 2572 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9gnmq_3a7e030d-fad8-480d-9dc6-fe5057048cf9/konnectivity-agent/0.log" limit=1024 Jul 16 00:41:35.211576 ip-10-0-143-200 kubenswrapper[2572]: I0716 00:41:35.211540 2572 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9gnmq_3a7e030d-fad8-480d-9dc6-fe5057048cf9/konnectivity-agent/0.log" limit=1024 Jul 16 00:41:36.562244 ip-10-0-143-200 kubenswrapper[2572]: I0716 00:41:36.562208 2572 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9gnmq_3a7e030d-fad8-480d-9dc6-fe5057048cf9/konnectivity-agent/0.log" limit=1024 Jul 16 00:41:36.660773 ip-10-0-143-200 kubenswrapper[2572]: I0716 00:41:36.660736 2572 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9gnmq_3a7e030d-fad8-480d-9dc6-fe5057048cf9/konnectivity-agent/0.log" limit=1024 Jul 16 00:41:37.707098 ip-10-0-143-200 kubenswrapper[2572]: I0716 00:41:37.707062 2572 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9gnmq_3a7e030d-fad8-480d-9dc6-fe5057048cf9/konnectivity-agent/0.log" limit=1024 Jul 16 00:41:39.290943 ip-10-0-143-200 kubenswrapper[2572]: I0716 00:41:39.290904 2572 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9gnmq_3a7e030d-fad8-480d-9dc6-fe5057048cf9/konnectivity-agent/0.log" limit=1024 Jul 16 00:41:39.391467 ip-10-0-143-200 kubenswrapper[2572]: I0716 00:41:39.391424 2572 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9gnmq_3a7e030d-fad8-480d-9dc6-fe5057048cf9/konnectivity-agent/0.log" limit=1024 Jul 16 00:41:39.496473 ip-10-0-143-200 kubenswrapper[2572]: I0716 00:41:39.496436 2572 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9gnmq_3a7e030d-fad8-480d-9dc6-fe5057048cf9/konnectivity-agent/0.log" limit=1024 Jul 16 00:41:40.835744 ip-10-0-143-200 kubenswrapper[2572]: I0716 00:41:40.835708 2572 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9gnmq_3a7e030d-fad8-480d-9dc6-fe5057048cf9/konnectivity-agent/0.log" limit=1024 Jul 16 00:41:40.931485 ip-10-0-143-200 kubenswrapper[2572]: I0716 00:41:40.931449 2572 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9gnmq_3a7e030d-fad8-480d-9dc6-fe5057048cf9/konnectivity-agent/0.log" limit=1024 Jul 16 00:41:41.150896 ip-10-0-143-200 kubenswrapper[2572]: I0716 00:41:41.150823 2572 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9gnmq_3a7e030d-fad8-480d-9dc6-fe5057048cf9/konnectivity-agent/0.log" limit=1024 Jul 16 00:41:41.251535 ip-10-0-143-200 kubenswrapper[2572]: I0716 00:41:41.251502 2572 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9gnmq_3a7e030d-fad8-480d-9dc6-fe5057048cf9/konnectivity-agent/0.log" limit=1024 Jul 16 00:41:43.589748 ip-10-0-143-200 kubenswrapper[2572]: I0716 00:41:43.589705 2572 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9gnmq_3a7e030d-fad8-480d-9dc6-fe5057048cf9/konnectivity-agent/0.log" limit=1024 Jul 16 00:41:43.685295 ip-10-0-143-200 kubenswrapper[2572]: I0716 00:41:43.685262 2572 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9gnmq_3a7e030d-fad8-480d-9dc6-fe5057048cf9/konnectivity-agent/0.log" limit=1024 Jul 16 00:41:44.250814 ip-10-0-143-200 kubenswrapper[2572]: I0716 00:41:44.250779 2572 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9gnmq_3a7e030d-fad8-480d-9dc6-fe5057048cf9/konnectivity-agent/0.log" limit=1024 Jul 16 00:41:44.337616 ip-10-0-143-200 kubenswrapper[2572]: I0716 00:41:44.337566 2572 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9gnmq_3a7e030d-fad8-480d-9dc6-fe5057048cf9/konnectivity-agent/0.log" limit=1024 Jul 16 00:41:46.124148 ip-10-0-143-200 kubenswrapper[2572]: I0716 00:41:46.124064 2572 generic.go:356] "Generic (PLEG): container finished" podID="1c3985e6-ff56-469e-bdd6-809bf98870ab" containerID="c6b18d98eda073910a1c9ed20d30a469180128d82ef566e74c0fe377242dd123" exitCode=0 Jul 16 00:41:46.124148 ip-10-0-143-200 kubenswrapper[2572]: I0716 00:41:46.124114 2572 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-service-ca-operator/service-ca-operator-7695fc6c8c-7v8cr" event={"ID":"1c3985e6-ff56-469e-bdd6-809bf98870ab","Type":"ContainerDied","Data":"c6b18d98eda073910a1c9ed20d30a469180128d82ef566e74c0fe377242dd123"} Jul 16 00:41:46.124726 ip-10-0-143-200 kubenswrapper[2572]: I0716 00:41:46.124415 2572 scope.go:117] "RemoveContainer" containerID="c6b18d98eda073910a1c9ed20d30a469180128d82ef566e74c0fe377242dd123" Jul 16 00:41:47.129588 ip-10-0-143-200 kubenswrapper[2572]: I0716 00:41:47.129553 2572 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-service-ca-operator/service-ca-operator-7695fc6c8c-7v8cr" event={"ID":"1c3985e6-ff56-469e-bdd6-809bf98870ab","Type":"ContainerStarted","Data":"22c85f62a0777fc6e9ca2c086655631a1dd3848301e644b68436a4bb9e26fdbf"} Jul 16 00:41:49.286531 ip-10-0-143-200 kubenswrapper[2572]: I0716 00:41:49.286494 2572 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9gnmq_3a7e030d-fad8-480d-9dc6-fe5057048cf9/konnectivity-agent/0.log" limit=1024 Jul 16 00:41:49.394170 ip-10-0-143-200 kubenswrapper[2572]: I0716 00:41:49.394132 2572 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9gnmq_3a7e030d-fad8-480d-9dc6-fe5057048cf9/konnectivity-agent/0.log" limit=1024 Jul 16 00:41:49.584803 ip-10-0-143-200 kubenswrapper[2572]: I0716 00:41:49.584687 2572 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9gnmq_3a7e030d-fad8-480d-9dc6-fe5057048cf9/konnectivity-agent/0.log" limit=1024 Jul 16 00:41:49.683216 ip-10-0-143-200 kubenswrapper[2572]: I0716 00:41:49.683181 2572 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9gnmq_3a7e030d-fad8-480d-9dc6-fe5057048cf9/konnectivity-agent/0.log" limit=1024 Jul 16 00:41:53.320851 ip-10-0-143-200 kubenswrapper[2572]: I0716 00:41:53.320777 2572 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9gnmq_3a7e030d-fad8-480d-9dc6-fe5057048cf9/konnectivity-agent/0.log" limit=1024 Jul 16 00:41:53.448718 ip-10-0-143-200 kubenswrapper[2572]: I0716 00:41:53.448685 2572 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9gnmq_3a7e030d-fad8-480d-9dc6-fe5057048cf9/konnectivity-agent/0.log" limit=1024 Jul 16 00:41:55.339277 ip-10-0-143-200 kubenswrapper[2572]: I0716 00:41:55.339247 2572 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9gnmq_3a7e030d-fad8-480d-9dc6-fe5057048cf9/konnectivity-agent/0.log" limit=1024 Jul 16 00:41:55.440194 ip-10-0-143-200 kubenswrapper[2572]: I0716 00:41:55.440160 2572 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9gnmq_3a7e030d-fad8-480d-9dc6-fe5057048cf9/konnectivity-agent/0.log" limit=1024 Jul 16 00:41:55.540446 ip-10-0-143-200 kubenswrapper[2572]: I0716 00:41:55.540414 2572 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9gnmq_3a7e030d-fad8-480d-9dc6-fe5057048cf9/konnectivity-agent/0.log" limit=1024 Jul 16 00:41:55.605189 ip-10-0-143-200 kubenswrapper[2572]: I0716 00:41:55.605093 2572 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9gnmq_3a7e030d-fad8-480d-9dc6-fe5057048cf9/konnectivity-agent/0.log" limit=1024 Jul 16 00:41:55.864445 ip-10-0-143-200 kubenswrapper[2572]: I0716 00:41:55.864346 2572 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9gnmq_3a7e030d-fad8-480d-9dc6-fe5057048cf9/konnectivity-agent/0.log" limit=1024 Jul 16 00:41:55.932208 ip-10-0-143-200 kubenswrapper[2572]: I0716 00:41:55.932171 2572 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9gnmq_3a7e030d-fad8-480d-9dc6-fe5057048cf9/konnectivity-agent/0.log" limit=1024 Jul 16 00:41:57.398759 ip-10-0-143-200 kubenswrapper[2572]: I0716 00:41:57.398711 2572 kuberuntime_container.go:873] "Killing container with a grace period" pod="openshift-image-registry/image-registry-dd7c989d9-5w25c" podUID="f103d7a8-2560-409b-801e-701bcc6835c1" containerName="registry" containerID="cri-o://7b74de616c853f60ff635e09a7ab38a0bbeb4e38be86d83574484fe3fe2ae387" gracePeriod=30 Jul 16 00:41:57.642096 ip-10-0-143-200 kubenswrapper[2572]: I0716 00:41:57.642070 2572 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-image-registry/image-registry-dd7c989d9-5w25c" Jul 16 00:41:57.740860 ip-10-0-143-200 kubenswrapper[2572]: I0716 00:41:57.740821 2572 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"registry-certificates\" (UniqueName: \"kubernetes.io/configmap/f103d7a8-2560-409b-801e-701bcc6835c1-registry-certificates\") pod \"f103d7a8-2560-409b-801e-701bcc6835c1\" (UID: \"f103d7a8-2560-409b-801e-701bcc6835c1\") " Jul 16 00:41:57.741036 ip-10-0-143-200 kubenswrapper[2572]: I0716 00:41:57.740880 2572 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"trusted-ca\" (UniqueName: \"kubernetes.io/configmap/f103d7a8-2560-409b-801e-701bcc6835c1-trusted-ca\") pod \"f103d7a8-2560-409b-801e-701bcc6835c1\" (UID: \"f103d7a8-2560-409b-801e-701bcc6835c1\") " Jul 16 00:41:57.741036 ip-10-0-143-200 kubenswrapper[2572]: I0716 00:41:57.740911 2572 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-kmrt8\" (UniqueName: \"kubernetes.io/projected/f103d7a8-2560-409b-801e-701bcc6835c1-kube-api-access-kmrt8\") pod \"f103d7a8-2560-409b-801e-701bcc6835c1\" (UID: \"f103d7a8-2560-409b-801e-701bcc6835c1\") " Jul 16 00:41:57.741036 ip-10-0-143-200 kubenswrapper[2572]: I0716 00:41:57.741005 2572 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"installation-pull-secrets\" (UniqueName: \"kubernetes.io/secret/f103d7a8-2560-409b-801e-701bcc6835c1-installation-pull-secrets\") pod \"f103d7a8-2560-409b-801e-701bcc6835c1\" (UID: \"f103d7a8-2560-409b-801e-701bcc6835c1\") " Jul 16 00:41:57.741154 ip-10-0-143-200 kubenswrapper[2572]: I0716 00:41:57.741066 2572 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"registry-tls\" (UniqueName: \"kubernetes.io/projected/f103d7a8-2560-409b-801e-701bcc6835c1-registry-tls\") pod \"f103d7a8-2560-409b-801e-701bcc6835c1\" (UID: \"f103d7a8-2560-409b-801e-701bcc6835c1\") " Jul 16 00:41:57.741154 ip-10-0-143-200 kubenswrapper[2572]: I0716 00:41:57.741088 2572 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"ca-trust-extracted\" (UniqueName: \"kubernetes.io/empty-dir/f103d7a8-2560-409b-801e-701bcc6835c1-ca-trust-extracted\") pod \"f103d7a8-2560-409b-801e-701bcc6835c1\" (UID: \"f103d7a8-2560-409b-801e-701bcc6835c1\") " Jul 16 00:41:57.741154 ip-10-0-143-200 kubenswrapper[2572]: I0716 00:41:57.741116 2572 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"image-registry-private-configuration\" (UniqueName: \"kubernetes.io/secret/f103d7a8-2560-409b-801e-701bcc6835c1-image-registry-private-configuration\") pod \"f103d7a8-2560-409b-801e-701bcc6835c1\" (UID: \"f103d7a8-2560-409b-801e-701bcc6835c1\") " Jul 16 00:41:57.741304 ip-10-0-143-200 kubenswrapper[2572]: I0716 00:41:57.741157 2572 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"bound-sa-token\" (UniqueName: \"kubernetes.io/projected/f103d7a8-2560-409b-801e-701bcc6835c1-bound-sa-token\") pod \"f103d7a8-2560-409b-801e-701bcc6835c1\" (UID: \"f103d7a8-2560-409b-801e-701bcc6835c1\") " Jul 16 00:41:57.741469 ip-10-0-143-200 kubenswrapper[2572]: I0716 00:41:57.741439 2572 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/f103d7a8-2560-409b-801e-701bcc6835c1-trusted-ca" (OuterVolumeSpecName: "trusted-ca") pod "f103d7a8-2560-409b-801e-701bcc6835c1" (UID: "f103d7a8-2560-409b-801e-701bcc6835c1"). InnerVolumeSpecName "trusted-ca". PluginName "kubernetes.io/configmap", VolumeGIDValue "" Jul 16 00:41:57.741653 ip-10-0-143-200 kubenswrapper[2572]: I0716 00:41:57.741609 2572 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/f103d7a8-2560-409b-801e-701bcc6835c1-registry-certificates" (OuterVolumeSpecName: "registry-certificates") pod "f103d7a8-2560-409b-801e-701bcc6835c1" (UID: "f103d7a8-2560-409b-801e-701bcc6835c1"). InnerVolumeSpecName "registry-certificates". PluginName "kubernetes.io/configmap", VolumeGIDValue "" Jul 16 00:41:57.743580 ip-10-0-143-200 kubenswrapper[2572]: I0716 00:41:57.743549 2572 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/f103d7a8-2560-409b-801e-701bcc6835c1-image-registry-private-configuration" (OuterVolumeSpecName: "image-registry-private-configuration") pod "f103d7a8-2560-409b-801e-701bcc6835c1" (UID: "f103d7a8-2560-409b-801e-701bcc6835c1"). InnerVolumeSpecName "image-registry-private-configuration". PluginName "kubernetes.io/secret", VolumeGIDValue "" Jul 16 00:41:57.743684 ip-10-0-143-200 kubenswrapper[2572]: I0716 00:41:57.743558 2572 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/f103d7a8-2560-409b-801e-701bcc6835c1-kube-api-access-kmrt8" (OuterVolumeSpecName: "kube-api-access-kmrt8") pod "f103d7a8-2560-409b-801e-701bcc6835c1" (UID: "f103d7a8-2560-409b-801e-701bcc6835c1"). InnerVolumeSpecName "kube-api-access-kmrt8". PluginName "kubernetes.io/projected", VolumeGIDValue "" Jul 16 00:41:57.743775 ip-10-0-143-200 kubenswrapper[2572]: I0716 00:41:57.743751 2572 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/f103d7a8-2560-409b-801e-701bcc6835c1-registry-tls" (OuterVolumeSpecName: "registry-tls") pod "f103d7a8-2560-409b-801e-701bcc6835c1" (UID: "f103d7a8-2560-409b-801e-701bcc6835c1"). InnerVolumeSpecName "registry-tls". PluginName "kubernetes.io/projected", VolumeGIDValue "" Jul 16 00:41:57.743873 ip-10-0-143-200 kubenswrapper[2572]: I0716 00:41:57.743857 2572 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/f103d7a8-2560-409b-801e-701bcc6835c1-installation-pull-secrets" (OuterVolumeSpecName: "installation-pull-secrets") pod "f103d7a8-2560-409b-801e-701bcc6835c1" (UID: "f103d7a8-2560-409b-801e-701bcc6835c1"). InnerVolumeSpecName "installation-pull-secrets". PluginName "kubernetes.io/secret", VolumeGIDValue "" Jul 16 00:41:57.743917 ip-10-0-143-200 kubenswrapper[2572]: I0716 00:41:57.743863 2572 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/f103d7a8-2560-409b-801e-701bcc6835c1-bound-sa-token" (OuterVolumeSpecName: "bound-sa-token") pod "f103d7a8-2560-409b-801e-701bcc6835c1" (UID: "f103d7a8-2560-409b-801e-701bcc6835c1"). InnerVolumeSpecName "bound-sa-token". PluginName "kubernetes.io/projected", VolumeGIDValue "" Jul 16 00:41:57.749596 ip-10-0-143-200 kubenswrapper[2572]: I0716 00:41:57.749565 2572 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/f103d7a8-2560-409b-801e-701bcc6835c1-ca-trust-extracted" (OuterVolumeSpecName: "ca-trust-extracted") pod "f103d7a8-2560-409b-801e-701bcc6835c1" (UID: "f103d7a8-2560-409b-801e-701bcc6835c1"). InnerVolumeSpecName "ca-trust-extracted". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Jul 16 00:41:57.841946 ip-10-0-143-200 kubenswrapper[2572]: I0716 00:41:57.841894 2572 reconciler_common.go:299] "Volume detached for volume \"image-registry-private-configuration\" (UniqueName: \"kubernetes.io/secret/f103d7a8-2560-409b-801e-701bcc6835c1-image-registry-private-configuration\") on node \"ip-10-0-143-200.ec2.internal\" DevicePath \"\"" Jul 16 00:41:57.841946 ip-10-0-143-200 kubenswrapper[2572]: I0716 00:41:57.841940 2572 reconciler_common.go:299] "Volume detached for volume \"bound-sa-token\" (UniqueName: \"kubernetes.io/projected/f103d7a8-2560-409b-801e-701bcc6835c1-bound-sa-token\") on node \"ip-10-0-143-200.ec2.internal\" DevicePath \"\"" Jul 16 00:41:57.841946 ip-10-0-143-200 kubenswrapper[2572]: I0716 00:41:57.841952 2572 reconciler_common.go:299] "Volume detached for volume \"registry-certificates\" (UniqueName: \"kubernetes.io/configmap/f103d7a8-2560-409b-801e-701bcc6835c1-registry-certificates\") on node \"ip-10-0-143-200.ec2.internal\" DevicePath \"\"" Jul 16 00:41:57.842202 ip-10-0-143-200 kubenswrapper[2572]: I0716 00:41:57.841993 2572 reconciler_common.go:299] "Volume detached for volume \"trusted-ca\" (UniqueName: \"kubernetes.io/configmap/f103d7a8-2560-409b-801e-701bcc6835c1-trusted-ca\") on node \"ip-10-0-143-200.ec2.internal\" DevicePath \"\"" Jul 16 00:41:57.842202 ip-10-0-143-200 kubenswrapper[2572]: I0716 00:41:57.842002 2572 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-kmrt8\" (UniqueName: \"kubernetes.io/projected/f103d7a8-2560-409b-801e-701bcc6835c1-kube-api-access-kmrt8\") on node \"ip-10-0-143-200.ec2.internal\" DevicePath \"\"" Jul 16 00:41:57.842202 ip-10-0-143-200 kubenswrapper[2572]: I0716 00:41:57.842011 2572 reconciler_common.go:299] "Volume detached for volume \"installation-pull-secrets\" (UniqueName: \"kubernetes.io/secret/f103d7a8-2560-409b-801e-701bcc6835c1-installation-pull-secrets\") on node \"ip-10-0-143-200.ec2.internal\" DevicePath \"\"" Jul 16 00:41:57.842202 ip-10-0-143-200 kubenswrapper[2572]: I0716 00:41:57.842020 2572 reconciler_common.go:299] "Volume detached for volume \"registry-tls\" (UniqueName: \"kubernetes.io/projected/f103d7a8-2560-409b-801e-701bcc6835c1-registry-tls\") on node \"ip-10-0-143-200.ec2.internal\" DevicePath \"\"" Jul 16 00:41:57.842202 ip-10-0-143-200 kubenswrapper[2572]: I0716 00:41:57.842028 2572 reconciler_common.go:299] "Volume detached for volume \"ca-trust-extracted\" (UniqueName: \"kubernetes.io/empty-dir/f103d7a8-2560-409b-801e-701bcc6835c1-ca-trust-extracted\") on node \"ip-10-0-143-200.ec2.internal\" DevicePath \"\"" Jul 16 00:41:58.174437 ip-10-0-143-200 kubenswrapper[2572]: I0716 00:41:58.174347 2572 generic.go:356] "Generic (PLEG): container finished" podID="f103d7a8-2560-409b-801e-701bcc6835c1" containerID="7b74de616c853f60ff635e09a7ab38a0bbeb4e38be86d83574484fe3fe2ae387" exitCode=0 Jul 16 00:41:58.174437 ip-10-0-143-200 kubenswrapper[2572]: I0716 00:41:58.174388 2572 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-image-registry/image-registry-dd7c989d9-5w25c" event={"ID":"f103d7a8-2560-409b-801e-701bcc6835c1","Type":"ContainerDied","Data":"7b74de616c853f60ff635e09a7ab38a0bbeb4e38be86d83574484fe3fe2ae387"} Jul 16 00:41:58.174437 ip-10-0-143-200 kubenswrapper[2572]: I0716 00:41:58.174394 2572 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-image-registry/image-registry-dd7c989d9-5w25c" Jul 16 00:41:58.174437 ip-10-0-143-200 kubenswrapper[2572]: I0716 00:41:58.174429 2572 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-image-registry/image-registry-dd7c989d9-5w25c" event={"ID":"f103d7a8-2560-409b-801e-701bcc6835c1","Type":"ContainerDied","Data":"8930e9612ebbcaeb1380f49c8578c122926b656baaeb1217fba519c550b0cb5a"} Jul 16 00:41:58.174753 ip-10-0-143-200 kubenswrapper[2572]: I0716 00:41:58.174449 2572 scope.go:117] "RemoveContainer" containerID="7b74de616c853f60ff635e09a7ab38a0bbeb4e38be86d83574484fe3fe2ae387" Jul 16 00:41:58.183083 ip-10-0-143-200 kubenswrapper[2572]: I0716 00:41:58.183065 2572 scope.go:117] "RemoveContainer" containerID="7b74de616c853f60ff635e09a7ab38a0bbeb4e38be86d83574484fe3fe2ae387" Jul 16 00:41:58.183361 ip-10-0-143-200 kubenswrapper[2572]: E0716 00:41:58.183342 2572 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"7b74de616c853f60ff635e09a7ab38a0bbeb4e38be86d83574484fe3fe2ae387\": container with ID starting with 7b74de616c853f60ff635e09a7ab38a0bbeb4e38be86d83574484fe3fe2ae387 not found: ID does not exist" containerID="7b74de616c853f60ff635e09a7ab38a0bbeb4e38be86d83574484fe3fe2ae387" Jul 16 00:41:58.183407 ip-10-0-143-200 kubenswrapper[2572]: I0716 00:41:58.183369 2572 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"7b74de616c853f60ff635e09a7ab38a0bbeb4e38be86d83574484fe3fe2ae387"} err="failed to get container status \"7b74de616c853f60ff635e09a7ab38a0bbeb4e38be86d83574484fe3fe2ae387\": rpc error: code = NotFound desc = could not find container \"7b74de616c853f60ff635e09a7ab38a0bbeb4e38be86d83574484fe3fe2ae387\": container with ID starting with 7b74de616c853f60ff635e09a7ab38a0bbeb4e38be86d83574484fe3fe2ae387 not found: ID does not exist" Jul 16 00:41:58.198332 ip-10-0-143-200 kubenswrapper[2572]: I0716 00:41:58.198299 2572 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-image-registry/image-registry-dd7c989d9-5w25c"] Jul 16 00:41:58.202551 ip-10-0-143-200 kubenswrapper[2572]: I0716 00:41:58.202522 2572 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["openshift-image-registry/image-registry-dd7c989d9-5w25c"] Jul 16 00:41:59.446887 ip-10-0-143-200 kubenswrapper[2572]: I0716 00:41:59.446851 2572 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="f103d7a8-2560-409b-801e-701bcc6835c1" path="/var/lib/kubelet/pods/f103d7a8-2560-409b-801e-701bcc6835c1/volumes" Jul 16 00:42:04.541795 ip-10-0-143-200 kubenswrapper[2572]: I0716 00:42:04.541748 2572 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9gnmq_3a7e030d-fad8-480d-9dc6-fe5057048cf9/konnectivity-agent/0.log" limit=1024 Jul 16 00:42:04.643586 ip-10-0-143-200 kubenswrapper[2572]: I0716 00:42:04.643550 2572 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9gnmq_3a7e030d-fad8-480d-9dc6-fe5057048cf9/konnectivity-agent/0.log" limit=1024 Jul 16 00:42:06.206425 ip-10-0-143-200 kubenswrapper[2572]: I0716 00:42:06.206394 2572 generic.go:356] "Generic (PLEG): container finished" podID="0563669f-9ac5-43d0-a3c0-47c36275ee46" containerID="7b90d55e500c9c647f94e7dcee88d09ccfdeb84311ec13c717021fe67c88bb96" exitCode=0 Jul 16 00:42:06.206821 ip-10-0-143-200 kubenswrapper[2572]: I0716 00:42:06.206461 2572 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-insights/insights-operator-68d9dd94cf-d4nrt" event={"ID":"0563669f-9ac5-43d0-a3c0-47c36275ee46","Type":"ContainerDied","Data":"7b90d55e500c9c647f94e7dcee88d09ccfdeb84311ec13c717021fe67c88bb96"} Jul 16 00:42:06.206884 ip-10-0-143-200 kubenswrapper[2572]: I0716 00:42:06.206839 2572 scope.go:117] "RemoveContainer" containerID="7b90d55e500c9c647f94e7dcee88d09ccfdeb84311ec13c717021fe67c88bb96" Jul 16 00:42:07.219608 ip-10-0-143-200 kubenswrapper[2572]: I0716 00:42:07.218207 2572 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-insights/insights-operator-68d9dd94cf-d4nrt" event={"ID":"0563669f-9ac5-43d0-a3c0-47c36275ee46","Type":"ContainerStarted","Data":"a24b5c85b7814ef6bc8beacd33b69b4d7f882d7ea0ac1047f9d14fa7e29a4313"} Jul 16 00:42:10.231583 ip-10-0-143-200 kubenswrapper[2572]: I0716 00:42:10.231495 2572 generic.go:356] "Generic (PLEG): container finished" podID="07e0ab9b-1bfe-4335-bac1-8d9d158b0085" containerID="5c7cb475bb36e6926df0a29350ca8fd94ac35b5568f1410195dff810ee0e5dcc" exitCode=0 Jul 16 00:42:10.231583 ip-10-0-143-200 kubenswrapper[2572]: I0716 00:42:10.231545 2572 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-kube-storage-version-migrator-operator/kube-storage-version-migrator-operator-799cb7f958-tksqh" event={"ID":"07e0ab9b-1bfe-4335-bac1-8d9d158b0085","Type":"ContainerDied","Data":"5c7cb475bb36e6926df0a29350ca8fd94ac35b5568f1410195dff810ee0e5dcc"} Jul 16 00:42:10.232156 ip-10-0-143-200 kubenswrapper[2572]: I0716 00:42:10.231851 2572 scope.go:117] "RemoveContainer" containerID="5c7cb475bb36e6926df0a29350ca8fd94ac35b5568f1410195dff810ee0e5dcc" Jul 16 00:42:11.237197 ip-10-0-143-200 kubenswrapper[2572]: I0716 00:42:11.237161 2572 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-kube-storage-version-migrator-operator/kube-storage-version-migrator-operator-799cb7f958-tksqh" event={"ID":"07e0ab9b-1bfe-4335-bac1-8d9d158b0085","Type":"ContainerStarted","Data":"5dc6b81581bf6fecce04338809b05e314e82a47fff667fcb71ebe95eddc682c1"} Jul 16 00:42:15.629178 ip-10-0-143-200 kubenswrapper[2572]: I0716 00:42:15.629140 2572 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9gnmq_3a7e030d-fad8-480d-9dc6-fe5057048cf9/konnectivity-agent/0.log" limit=1024 Jul 16 00:42:19.540573 ip-10-0-143-200 kubenswrapper[2572]: I0716 00:42:19.540531 2572 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9gnmq_3a7e030d-fad8-480d-9dc6-fe5057048cf9/konnectivity-agent/0.log" limit=1024 Jul 16 00:42:19.613825 ip-10-0-143-200 kubenswrapper[2572]: I0716 00:42:19.613790 2572 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9gnmq_3a7e030d-fad8-480d-9dc6-fe5057048cf9/konnectivity-agent/0.log" limit=1024 Jul 16 00:42:45.384441 ip-10-0-143-200 kubenswrapper[2572]: I0716 00:42:45.384405 2572 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9gnmq_3a7e030d-fad8-480d-9dc6-fe5057048cf9/konnectivity-agent/0.log" limit=1024 Jul 16 00:42:45.493463 ip-10-0-143-200 kubenswrapper[2572]: I0716 00:42:45.493427 2572 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9gnmq_3a7e030d-fad8-480d-9dc6-fe5057048cf9/konnectivity-agent/0.log" limit=1024 Jul 16 00:42:55.095082 ip-10-0-143-200 kubenswrapper[2572]: I0716 00:42:55.094994 2572 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9gnmq_3a7e030d-fad8-480d-9dc6-fe5057048cf9/konnectivity-agent/0.log" limit=1024 Jul 16 00:42:55.197573 ip-10-0-143-200 kubenswrapper[2572]: I0716 00:42:55.197541 2572 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9gnmq_3a7e030d-fad8-480d-9dc6-fe5057048cf9/konnectivity-agent/0.log" limit=1024 Jul 16 00:42:55.264787 ip-10-0-143-200 kubenswrapper[2572]: I0716 00:42:55.264751 2572 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9gnmq_3a7e030d-fad8-480d-9dc6-fe5057048cf9/konnectivity-agent/0.log" limit=1024 Jul 16 00:42:55.827673 ip-10-0-143-200 kubenswrapper[2572]: I0716 00:42:55.827606 2572 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9gnmq_3a7e030d-fad8-480d-9dc6-fe5057048cf9/konnectivity-agent/0.log" limit=1024 Jul 16 00:42:55.890016 ip-10-0-143-200 kubenswrapper[2572]: I0716 00:42:55.889980 2572 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9gnmq_3a7e030d-fad8-480d-9dc6-fe5057048cf9/konnectivity-agent/0.log" limit=1024 Jul 16 00:43:05.577942 ip-10-0-143-200 kubenswrapper[2572]: I0716 00:43:05.577904 2572 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9gnmq_3a7e030d-fad8-480d-9dc6-fe5057048cf9/konnectivity-agent/0.log" limit=1024 Jul 16 00:43:05.678698 ip-10-0-143-200 kubenswrapper[2572]: I0716 00:43:05.678669 2572 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9gnmq_3a7e030d-fad8-480d-9dc6-fe5057048cf9/konnectivity-agent/0.log" limit=1024 Jul 16 00:43:05.739330 ip-10-0-143-200 kubenswrapper[2572]: I0716 00:43:05.739295 2572 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9gnmq_3a7e030d-fad8-480d-9dc6-fe5057048cf9/konnectivity-agent/0.log" limit=1024 Jul 16 00:43:37.340160 ip-10-0-143-200 kubenswrapper[2572]: I0716 00:43:37.340114 2572 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9gnmq_3a7e030d-fad8-480d-9dc6-fe5057048cf9/konnectivity-agent/0.log" limit=1024 Jul 16 00:44:00.631670 ip-10-0-143-200 kubenswrapper[2572]: I0716 00:44:00.631620 2572 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9gnmq_3a7e030d-fad8-480d-9dc6-fe5057048cf9/konnectivity-agent/0.log" limit=1024 Jul 16 00:44:00.727604 ip-10-0-143-200 kubenswrapper[2572]: I0716 00:44:00.727568 2572 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9gnmq_3a7e030d-fad8-480d-9dc6-fe5057048cf9/konnectivity-agent/0.log" limit=1024 Jul 16 00:44:01.220235 ip-10-0-143-200 kubenswrapper[2572]: I0716 00:44:01.220197 2572 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9gnmq_3a7e030d-fad8-480d-9dc6-fe5057048cf9/konnectivity-agent/0.log" limit=1024 Jul 16 00:44:01.292116 ip-10-0-143-200 kubenswrapper[2572]: I0716 00:44:01.292084 2572 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9gnmq_3a7e030d-fad8-480d-9dc6-fe5057048cf9/konnectivity-agent/0.log" limit=1024 Jul 16 00:44:01.363484 ip-10-0-143-200 kubenswrapper[2572]: I0716 00:44:01.363445 2572 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9gnmq_3a7e030d-fad8-480d-9dc6-fe5057048cf9/konnectivity-agent/0.log" limit=1024 Jul 16 00:44:01.934131 ip-10-0-143-200 kubenswrapper[2572]: I0716 00:44:01.934091 2572 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9gnmq_3a7e030d-fad8-480d-9dc6-fe5057048cf9/konnectivity-agent/0.log" limit=1024 Jul 16 00:44:02.035287 ip-10-0-143-200 kubenswrapper[2572]: I0716 00:44:02.035243 2572 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9gnmq_3a7e030d-fad8-480d-9dc6-fe5057048cf9/konnectivity-agent/0.log" limit=1024 Jul 16 00:44:11.797050 ip-10-0-143-200 kubenswrapper[2572]: I0716 00:44:11.797007 2572 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9gnmq_3a7e030d-fad8-480d-9dc6-fe5057048cf9/konnectivity-agent/0.log" limit=1024 Jul 16 00:44:11.899048 ip-10-0-143-200 kubenswrapper[2572]: I0716 00:44:11.899010 2572 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9gnmq_3a7e030d-fad8-480d-9dc6-fe5057048cf9/konnectivity-agent/0.log" limit=1024 Jul 16 00:44:11.963362 ip-10-0-143-200 kubenswrapper[2572]: I0716 00:44:11.963324 2572 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9gnmq_3a7e030d-fad8-480d-9dc6-fe5057048cf9/konnectivity-agent/0.log" limit=1024 Jul 16 00:44:12.546000 ip-10-0-143-200 kubenswrapper[2572]: I0716 00:44:12.545961 2572 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9gnmq_3a7e030d-fad8-480d-9dc6-fe5057048cf9/konnectivity-agent/0.log" limit=1024 Jul 16 00:44:12.647792 ip-10-0-143-200 kubenswrapper[2572]: I0716 00:44:12.647761 2572 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9gnmq_3a7e030d-fad8-480d-9dc6-fe5057048cf9/konnectivity-agent/0.log" limit=1024 Jul 16 00:44:19.557807 ip-10-0-143-200 kubenswrapper[2572]: I0716 00:44:19.557723 2572 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9gnmq_3a7e030d-fad8-480d-9dc6-fe5057048cf9/konnectivity-agent/0.log" limit=1024 Jul 16 00:44:19.658326 ip-10-0-143-200 kubenswrapper[2572]: I0716 00:44:19.658284 2572 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9gnmq_3a7e030d-fad8-480d-9dc6-fe5057048cf9/konnectivity-agent/0.log" limit=1024 Jul 16 00:44:55.317713 ip-10-0-143-200 kubenswrapper[2572]: I0716 00:44:55.317679 2572 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-console-operator_console-operator-7866497494-pjc6j_a64bcd05-8596-4c12-b88e-e974d676e25f/console-operator/2.log" Jul 16 00:44:55.322707 ip-10-0-143-200 kubenswrapper[2572]: I0716 00:44:55.322680 2572 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ovn-kubernetes_ovnkube-node-fp78g_aa66f2b6-0ad8-45d0-8ff5-40fc21966b5a/ovn-acl-logging/0.log" Jul 16 00:44:55.323113 ip-10-0-143-200 kubenswrapper[2572]: I0716 00:44:55.323092 2572 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-console-operator_console-operator-7866497494-pjc6j_a64bcd05-8596-4c12-b88e-e974d676e25f/console-operator/2.log" Jul 16 00:44:55.327579 ip-10-0-143-200 kubenswrapper[2572]: I0716 00:44:55.327554 2572 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ovn-kubernetes_ovnkube-node-fp78g_aa66f2b6-0ad8-45d0-8ff5-40fc21966b5a/ovn-acl-logging/0.log" Jul 16 00:44:55.329576 ip-10-0-143-200 kubenswrapper[2572]: I0716 00:44:55.329554 2572 kubelet.go:1648] "Image garbage collection succeeded" Jul 16 00:45:02.672878 ip-10-0-143-200 kubenswrapper[2572]: I0716 00:45:02.672841 2572 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9gnmq_3a7e030d-fad8-480d-9dc6-fe5057048cf9/konnectivity-agent/0.log" limit=1024 Jul 16 00:45:03.908151 ip-10-0-143-200 kubenswrapper[2572]: I0716 00:45:03.908114 2572 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9gnmq_3a7e030d-fad8-480d-9dc6-fe5057048cf9/konnectivity-agent/0.log" limit=1024 Jul 16 00:45:24.404974 ip-10-0-143-200 kubenswrapper[2572]: I0716 00:45:24.404932 2572 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9gnmq_3a7e030d-fad8-480d-9dc6-fe5057048cf9/konnectivity-agent/0.log" limit=1024 Jul 16 00:45:25.222322 ip-10-0-143-200 kubenswrapper[2572]: I0716 00:45:25.222275 2572 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9gnmq_3a7e030d-fad8-480d-9dc6-fe5057048cf9/konnectivity-agent/0.log" limit=1024 Jul 16 00:45:28.019269 ip-10-0-143-200 kubenswrapper[2572]: I0716 00:45:28.019232 2572 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9gnmq_3a7e030d-fad8-480d-9dc6-fe5057048cf9/konnectivity-agent/0.log" limit=1024 Jul 16 00:45:28.123820 ip-10-0-143-200 kubenswrapper[2572]: I0716 00:45:28.123784 2572 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9gnmq_3a7e030d-fad8-480d-9dc6-fe5057048cf9/konnectivity-agent/0.log" limit=1024 Jul 16 00:45:29.389465 ip-10-0-143-200 kubenswrapper[2572]: I0716 00:45:29.389429 2572 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9gnmq_3a7e030d-fad8-480d-9dc6-fe5057048cf9/konnectivity-agent/0.log" limit=1024 Jul 16 00:45:29.514401 ip-10-0-143-200 kubenswrapper[2572]: I0716 00:45:29.514366 2572 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9gnmq_3a7e030d-fad8-480d-9dc6-fe5057048cf9/konnectivity-agent/0.log" limit=1024 Jul 16 00:45:30.381251 ip-10-0-143-200 kubenswrapper[2572]: I0716 00:45:30.381208 2572 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9gnmq_3a7e030d-fad8-480d-9dc6-fe5057048cf9/konnectivity-agent/0.log" limit=1024 Jul 16 00:45:30.493946 ip-10-0-143-200 kubenswrapper[2572]: I0716 00:45:30.493913 2572 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9gnmq_3a7e030d-fad8-480d-9dc6-fe5057048cf9/konnectivity-agent/0.log" limit=1024 Jul 16 00:45:31.419569 ip-10-0-143-200 kubenswrapper[2572]: I0716 00:45:31.419522 2572 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9gnmq_3a7e030d-fad8-480d-9dc6-fe5057048cf9/konnectivity-agent/0.log" limit=1024 Jul 16 00:45:31.792979 ip-10-0-143-200 kubenswrapper[2572]: I0716 00:45:31.792937 2572 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9gnmq_3a7e030d-fad8-480d-9dc6-fe5057048cf9/konnectivity-agent/0.log" limit=1024 Jul 16 00:45:35.006173 ip-10-0-143-200 kubenswrapper[2572]: I0716 00:45:35.006133 2572 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9gnmq_3a7e030d-fad8-480d-9dc6-fe5057048cf9/konnectivity-agent/0.log" limit=1024 Jul 16 00:45:35.107434 ip-10-0-143-200 kubenswrapper[2572]: I0716 00:45:35.107400 2572 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9gnmq_3a7e030d-fad8-480d-9dc6-fe5057048cf9/konnectivity-agent/0.log" limit=1024 Jul 16 00:45:37.883759 ip-10-0-143-200 kubenswrapper[2572]: I0716 00:45:37.883702 2572 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9gnmq_3a7e030d-fad8-480d-9dc6-fe5057048cf9/konnectivity-agent/0.log" limit=1024 Jul 16 00:45:37.981794 ip-10-0-143-200 kubenswrapper[2572]: I0716 00:45:37.981762 2572 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9gnmq_3a7e030d-fad8-480d-9dc6-fe5057048cf9/konnectivity-agent/0.log" limit=1024 Jul 16 00:45:38.081770 ip-10-0-143-200 kubenswrapper[2572]: I0716 00:45:38.081736 2572 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9gnmq_3a7e030d-fad8-480d-9dc6-fe5057048cf9/konnectivity-agent/0.log" limit=1024 Jul 16 00:45:40.008579 ip-10-0-143-200 kubenswrapper[2572]: I0716 00:45:40.008549 2572 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9gnmq_3a7e030d-fad8-480d-9dc6-fe5057048cf9/konnectivity-agent/0.log" limit=1024 Jul 16 00:45:45.023952 ip-10-0-143-200 kubenswrapper[2572]: I0716 00:45:45.023915 2572 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9gnmq_3a7e030d-fad8-480d-9dc6-fe5057048cf9/konnectivity-agent/0.log" limit=1024 Jul 16 00:45:45.132444 ip-10-0-143-200 kubenswrapper[2572]: I0716 00:45:45.132411 2572 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9gnmq_3a7e030d-fad8-480d-9dc6-fe5057048cf9/konnectivity-agent/0.log" limit=1024 Jul 16 00:45:51.124920 ip-10-0-143-200 kubenswrapper[2572]: I0716 00:45:51.124834 2572 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9gnmq_3a7e030d-fad8-480d-9dc6-fe5057048cf9/konnectivity-agent/0.log" limit=1024 Jul 16 00:45:51.225850 ip-10-0-143-200 kubenswrapper[2572]: I0716 00:45:51.225816 2572 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9gnmq_3a7e030d-fad8-480d-9dc6-fe5057048cf9/konnectivity-agent/0.log" limit=1024 Jul 16 00:45:52.965391 ip-10-0-143-200 kubenswrapper[2572]: I0716 00:45:52.965344 2572 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9gnmq_3a7e030d-fad8-480d-9dc6-fe5057048cf9/konnectivity-agent/0.log" limit=1024 Jul 16 00:46:01.281184 ip-10-0-143-200 kubenswrapper[2572]: I0716 00:46:01.281149 2572 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9gnmq_3a7e030d-fad8-480d-9dc6-fe5057048cf9/konnectivity-agent/0.log" limit=1024 Jul 16 00:46:01.399641 ip-10-0-143-200 kubenswrapper[2572]: I0716 00:46:01.399585 2572 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9gnmq_3a7e030d-fad8-480d-9dc6-fe5057048cf9/konnectivity-agent/0.log" limit=1024 Jul 16 00:46:08.626288 ip-10-0-143-200 kubenswrapper[2572]: I0716 00:46:08.626235 2572 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9gnmq_3a7e030d-fad8-480d-9dc6-fe5057048cf9/konnectivity-agent/0.log" limit=1024 Jul 16 00:46:08.792385 ip-10-0-143-200 kubenswrapper[2572]: I0716 00:46:08.792343 2572 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9gnmq_3a7e030d-fad8-480d-9dc6-fe5057048cf9/konnectivity-agent/0.log" limit=1024 Jul 16 00:46:09.231921 ip-10-0-143-200 kubenswrapper[2572]: I0716 00:46:09.231886 2572 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9gnmq_3a7e030d-fad8-480d-9dc6-fe5057048cf9/konnectivity-agent/0.log" limit=1024 Jul 16 00:46:09.449870 ip-10-0-143-200 kubenswrapper[2572]: I0716 00:46:09.449834 2572 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9gnmq_3a7e030d-fad8-480d-9dc6-fe5057048cf9/konnectivity-agent/0.log" limit=1024 Jul 16 00:46:09.550193 ip-10-0-143-200 kubenswrapper[2572]: I0716 00:46:09.550097 2572 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9gnmq_3a7e030d-fad8-480d-9dc6-fe5057048cf9/konnectivity-agent/0.log" limit=1024 Jul 16 00:46:09.570206 ip-10-0-143-200 kubenswrapper[2572]: I0716 00:46:09.570168 2572 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["opendatahub/opendatahub-operator-controller-manager-5cf8c55d45-bfxb8"] Jul 16 00:46:09.570495 ip-10-0-143-200 kubenswrapper[2572]: I0716 00:46:09.570482 2572 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="f103d7a8-2560-409b-801e-701bcc6835c1" containerName="registry" Jul 16 00:46:09.570495 ip-10-0-143-200 kubenswrapper[2572]: I0716 00:46:09.570495 2572 state_mem.go:107] "Deleted CPUSet assignment" podUID="f103d7a8-2560-409b-801e-701bcc6835c1" containerName="registry" Jul 16 00:46:09.570570 ip-10-0-143-200 kubenswrapper[2572]: I0716 00:46:09.570552 2572 memory_manager.go:368] "RemoveStaleState removing state" podUID="f103d7a8-2560-409b-801e-701bcc6835c1" containerName="registry" Jul 16 00:46:09.572619 ip-10-0-143-200 kubenswrapper[2572]: I0716 00:46:09.572603 2572 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="opendatahub/opendatahub-operator-controller-manager-5cf8c55d45-bfxb8" Jul 16 00:46:09.576214 ip-10-0-143-200 kubenswrapper[2572]: I0716 00:46:09.576191 2572 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"opendatahub\"/\"openshift-service-ca.crt\"" Jul 16 00:46:09.576214 ip-10-0-143-200 kubenswrapper[2572]: I0716 00:46:09.576193 2572 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"opendatahub\"/\"opendatahub-operator-controller-webhook-cert\"" Jul 16 00:46:09.576512 ip-10-0-143-200 kubenswrapper[2572]: I0716 00:46:09.576247 2572 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"opendatahub\"/\"opendatahub-operator-controller-manager-service-cert\"" Jul 16 00:46:09.576610 ip-10-0-143-200 kubenswrapper[2572]: I0716 00:46:09.576594 2572 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"opendatahub\"/\"opendatahub-operator-controller-manager-dockercfg-9kszx\"" Jul 16 00:46:09.577268 ip-10-0-143-200 kubenswrapper[2572]: I0716 00:46:09.577255 2572 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"opendatahub\"/\"kube-root-ca.crt\"" Jul 16 00:46:09.594047 ip-10-0-143-200 kubenswrapper[2572]: I0716 00:46:09.594014 2572 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["opendatahub/opendatahub-operator-controller-manager-5cf8c55d45-bfxb8"] Jul 16 00:46:09.609946 ip-10-0-143-200 kubenswrapper[2572]: I0716 00:46:09.609913 2572 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tmp\" (UniqueName: \"kubernetes.io/empty-dir/e58cb8b4-4bdb-4a40-bdd9-aa6f34eb09a4-tmp\") pod \"opendatahub-operator-controller-manager-5cf8c55d45-bfxb8\" (UID: \"e58cb8b4-4bdb-4a40-bdd9-aa6f34eb09a4\") " pod="opendatahub/opendatahub-operator-controller-manager-5cf8c55d45-bfxb8" Jul 16 00:46:09.609946 ip-10-0-143-200 kubenswrapper[2572]: I0716 00:46:09.609950 2572 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"webhook-cert\" (UniqueName: \"kubernetes.io/secret/e58cb8b4-4bdb-4a40-bdd9-aa6f34eb09a4-webhook-cert\") pod \"opendatahub-operator-controller-manager-5cf8c55d45-bfxb8\" (UID: \"e58cb8b4-4bdb-4a40-bdd9-aa6f34eb09a4\") " pod="opendatahub/opendatahub-operator-controller-manager-5cf8c55d45-bfxb8" Jul 16 00:46:09.610173 ip-10-0-143-200 kubenswrapper[2572]: I0716 00:46:09.610051 2572 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"apiservice-cert\" (UniqueName: \"kubernetes.io/secret/e58cb8b4-4bdb-4a40-bdd9-aa6f34eb09a4-apiservice-cert\") pod \"opendatahub-operator-controller-manager-5cf8c55d45-bfxb8\" (UID: \"e58cb8b4-4bdb-4a40-bdd9-aa6f34eb09a4\") " pod="opendatahub/opendatahub-operator-controller-manager-5cf8c55d45-bfxb8" Jul 16 00:46:09.610173 ip-10-0-143-200 kubenswrapper[2572]: I0716 00:46:09.610108 2572 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"manifests\" (UniqueName: \"kubernetes.io/empty-dir/e58cb8b4-4bdb-4a40-bdd9-aa6f34eb09a4-manifests\") pod \"opendatahub-operator-controller-manager-5cf8c55d45-bfxb8\" (UID: \"e58cb8b4-4bdb-4a40-bdd9-aa6f34eb09a4\") " pod="opendatahub/opendatahub-operator-controller-manager-5cf8c55d45-bfxb8" Jul 16 00:46:09.610260 ip-10-0-143-200 kubenswrapper[2572]: I0716 00:46:09.610166 2572 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-n8c6q\" (UniqueName: \"kubernetes.io/projected/e58cb8b4-4bdb-4a40-bdd9-aa6f34eb09a4-kube-api-access-n8c6q\") pod \"opendatahub-operator-controller-manager-5cf8c55d45-bfxb8\" (UID: \"e58cb8b4-4bdb-4a40-bdd9-aa6f34eb09a4\") " pod="opendatahub/opendatahub-operator-controller-manager-5cf8c55d45-bfxb8" Jul 16 00:46:09.711430 ip-10-0-143-200 kubenswrapper[2572]: I0716 00:46:09.711390 2572 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"apiservice-cert\" (UniqueName: \"kubernetes.io/secret/e58cb8b4-4bdb-4a40-bdd9-aa6f34eb09a4-apiservice-cert\") pod \"opendatahub-operator-controller-manager-5cf8c55d45-bfxb8\" (UID: \"e58cb8b4-4bdb-4a40-bdd9-aa6f34eb09a4\") " pod="opendatahub/opendatahub-operator-controller-manager-5cf8c55d45-bfxb8" Jul 16 00:46:09.711876 ip-10-0-143-200 kubenswrapper[2572]: I0716 00:46:09.711441 2572 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"manifests\" (UniqueName: \"kubernetes.io/empty-dir/e58cb8b4-4bdb-4a40-bdd9-aa6f34eb09a4-manifests\") pod \"opendatahub-operator-controller-manager-5cf8c55d45-bfxb8\" (UID: \"e58cb8b4-4bdb-4a40-bdd9-aa6f34eb09a4\") " pod="opendatahub/opendatahub-operator-controller-manager-5cf8c55d45-bfxb8" Jul 16 00:46:09.711876 ip-10-0-143-200 kubenswrapper[2572]: I0716 00:46:09.711461 2572 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-n8c6q\" (UniqueName: \"kubernetes.io/projected/e58cb8b4-4bdb-4a40-bdd9-aa6f34eb09a4-kube-api-access-n8c6q\") pod \"opendatahub-operator-controller-manager-5cf8c55d45-bfxb8\" (UID: \"e58cb8b4-4bdb-4a40-bdd9-aa6f34eb09a4\") " pod="opendatahub/opendatahub-operator-controller-manager-5cf8c55d45-bfxb8" Jul 16 00:46:09.711876 ip-10-0-143-200 kubenswrapper[2572]: I0716 00:46:09.711497 2572 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tmp\" (UniqueName: \"kubernetes.io/empty-dir/e58cb8b4-4bdb-4a40-bdd9-aa6f34eb09a4-tmp\") pod \"opendatahub-operator-controller-manager-5cf8c55d45-bfxb8\" (UID: \"e58cb8b4-4bdb-4a40-bdd9-aa6f34eb09a4\") " pod="opendatahub/opendatahub-operator-controller-manager-5cf8c55d45-bfxb8" Jul 16 00:46:09.711876 ip-10-0-143-200 kubenswrapper[2572]: I0716 00:46:09.711517 2572 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"webhook-cert\" (UniqueName: \"kubernetes.io/secret/e58cb8b4-4bdb-4a40-bdd9-aa6f34eb09a4-webhook-cert\") pod \"opendatahub-operator-controller-manager-5cf8c55d45-bfxb8\" (UID: \"e58cb8b4-4bdb-4a40-bdd9-aa6f34eb09a4\") " pod="opendatahub/opendatahub-operator-controller-manager-5cf8c55d45-bfxb8" Jul 16 00:46:09.712092 ip-10-0-143-200 kubenswrapper[2572]: I0716 00:46:09.711975 2572 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"manifests\" (UniqueName: \"kubernetes.io/empty-dir/e58cb8b4-4bdb-4a40-bdd9-aa6f34eb09a4-manifests\") pod \"opendatahub-operator-controller-manager-5cf8c55d45-bfxb8\" (UID: \"e58cb8b4-4bdb-4a40-bdd9-aa6f34eb09a4\") " pod="opendatahub/opendatahub-operator-controller-manager-5cf8c55d45-bfxb8" Jul 16 00:46:09.713908 ip-10-0-143-200 kubenswrapper[2572]: I0716 00:46:09.713882 2572 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"tmp\" (UniqueName: \"kubernetes.io/empty-dir/e58cb8b4-4bdb-4a40-bdd9-aa6f34eb09a4-tmp\") pod \"opendatahub-operator-controller-manager-5cf8c55d45-bfxb8\" (UID: \"e58cb8b4-4bdb-4a40-bdd9-aa6f34eb09a4\") " pod="opendatahub/opendatahub-operator-controller-manager-5cf8c55d45-bfxb8" Jul 16 00:46:09.714045 ip-10-0-143-200 kubenswrapper[2572]: I0716 00:46:09.713980 2572 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"webhook-cert\" (UniqueName: \"kubernetes.io/secret/e58cb8b4-4bdb-4a40-bdd9-aa6f34eb09a4-webhook-cert\") pod \"opendatahub-operator-controller-manager-5cf8c55d45-bfxb8\" (UID: \"e58cb8b4-4bdb-4a40-bdd9-aa6f34eb09a4\") " pod="opendatahub/opendatahub-operator-controller-manager-5cf8c55d45-bfxb8" Jul 16 00:46:09.714045 ip-10-0-143-200 kubenswrapper[2572]: I0716 00:46:09.714014 2572 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"apiservice-cert\" (UniqueName: \"kubernetes.io/secret/e58cb8b4-4bdb-4a40-bdd9-aa6f34eb09a4-apiservice-cert\") pod \"opendatahub-operator-controller-manager-5cf8c55d45-bfxb8\" (UID: \"e58cb8b4-4bdb-4a40-bdd9-aa6f34eb09a4\") " pod="opendatahub/opendatahub-operator-controller-manager-5cf8c55d45-bfxb8" Jul 16 00:46:09.728059 ip-10-0-143-200 kubenswrapper[2572]: I0716 00:46:09.728024 2572 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-n8c6q\" (UniqueName: \"kubernetes.io/projected/e58cb8b4-4bdb-4a40-bdd9-aa6f34eb09a4-kube-api-access-n8c6q\") pod \"opendatahub-operator-controller-manager-5cf8c55d45-bfxb8\" (UID: \"e58cb8b4-4bdb-4a40-bdd9-aa6f34eb09a4\") " pod="opendatahub/opendatahub-operator-controller-manager-5cf8c55d45-bfxb8" Jul 16 00:46:09.752386 ip-10-0-143-200 kubenswrapper[2572]: I0716 00:46:09.752347 2572 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9gnmq_3a7e030d-fad8-480d-9dc6-fe5057048cf9/konnectivity-agent/0.log" limit=1024 Jul 16 00:46:09.852599 ip-10-0-143-200 kubenswrapper[2572]: I0716 00:46:09.852505 2572 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9gnmq_3a7e030d-fad8-480d-9dc6-fe5057048cf9/konnectivity-agent/0.log" limit=1024 Jul 16 00:46:09.883020 ip-10-0-143-200 kubenswrapper[2572]: I0716 00:46:09.882981 2572 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="opendatahub/opendatahub-operator-controller-manager-5cf8c55d45-bfxb8" Jul 16 00:46:09.953111 ip-10-0-143-200 kubenswrapper[2572]: I0716 00:46:09.953070 2572 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9gnmq_3a7e030d-fad8-480d-9dc6-fe5057048cf9/konnectivity-agent/0.log" limit=1024 Jul 16 00:46:10.023347 ip-10-0-143-200 kubenswrapper[2572]: I0716 00:46:10.023314 2572 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["opendatahub/opendatahub-operator-controller-manager-5cf8c55d45-bfxb8"] Jul 16 00:46:10.025726 ip-10-0-143-200 kubenswrapper[2572]: I0716 00:46:10.025695 2572 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9gnmq_3a7e030d-fad8-480d-9dc6-fe5057048cf9/konnectivity-agent/0.log" limit=1024 Jul 16 00:46:10.027132 ip-10-0-143-200 kubenswrapper[2572]: W0716 00:46:10.027098 2572 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pode58cb8b4_4bdb_4a40_bdd9_aa6f34eb09a4.slice/crio-89fa9ec757058a77760e5189a5e3d9dd0e21c963b17ebd3a19d884e2c75ab17d WatchSource:0}: Error finding container 89fa9ec757058a77760e5189a5e3d9dd0e21c963b17ebd3a19d884e2c75ab17d: Status 404 returned error can't find the container with id 89fa9ec757058a77760e5189a5e3d9dd0e21c963b17ebd3a19d884e2c75ab17d Jul 16 00:46:10.028791 ip-10-0-143-200 kubenswrapper[2572]: I0716 00:46:10.028774 2572 provider.go:93] Refreshing cache for provider: *credentialprovider.defaultDockerConfigProvider Jul 16 00:46:10.089467 ip-10-0-143-200 kubenswrapper[2572]: I0716 00:46:10.089434 2572 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9gnmq_3a7e030d-fad8-480d-9dc6-fe5057048cf9/konnectivity-agent/0.log" limit=1024 Jul 16 00:46:10.169248 ip-10-0-143-200 kubenswrapper[2572]: I0716 00:46:10.169163 2572 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="opendatahub/opendatahub-operator-controller-manager-5cf8c55d45-bfxb8" event={"ID":"e58cb8b4-4bdb-4a40-bdd9-aa6f34eb09a4","Type":"ContainerStarted","Data":"89fa9ec757058a77760e5189a5e3d9dd0e21c963b17ebd3a19d884e2c75ab17d"} Jul 16 00:46:10.196815 ip-10-0-143-200 kubenswrapper[2572]: I0716 00:46:10.196777 2572 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9gnmq_3a7e030d-fad8-480d-9dc6-fe5057048cf9/konnectivity-agent/0.log" limit=1024 Jul 16 00:46:10.290000 ip-10-0-143-200 kubenswrapper[2572]: I0716 00:46:10.289956 2572 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9gnmq_3a7e030d-fad8-480d-9dc6-fe5057048cf9/konnectivity-agent/0.log" limit=1024 Jul 16 00:46:14.190854 ip-10-0-143-200 kubenswrapper[2572]: I0716 00:46:14.190812 2572 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="opendatahub/opendatahub-operator-controller-manager-5cf8c55d45-bfxb8" event={"ID":"e58cb8b4-4bdb-4a40-bdd9-aa6f34eb09a4","Type":"ContainerStarted","Data":"d8de4c963239cb53f16804f156c2e8c00eb00e3f20d68019d5d45506d443d069"} Jul 16 00:46:15.196353 ip-10-0-143-200 kubenswrapper[2572]: I0716 00:46:15.196314 2572 generic.go:356] "Generic (PLEG): container finished" podID="e58cb8b4-4bdb-4a40-bdd9-aa6f34eb09a4" containerID="d8de4c963239cb53f16804f156c2e8c00eb00e3f20d68019d5d45506d443d069" exitCode=0 Jul 16 00:46:15.196860 ip-10-0-143-200 kubenswrapper[2572]: I0716 00:46:15.196401 2572 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="opendatahub/opendatahub-operator-controller-manager-5cf8c55d45-bfxb8" event={"ID":"e58cb8b4-4bdb-4a40-bdd9-aa6f34eb09a4","Type":"ContainerDied","Data":"d8de4c963239cb53f16804f156c2e8c00eb00e3f20d68019d5d45506d443d069"} Jul 16 00:46:16.202824 ip-10-0-143-200 kubenswrapper[2572]: I0716 00:46:16.202788 2572 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="opendatahub/opendatahub-operator-controller-manager-5cf8c55d45-bfxb8" event={"ID":"e58cb8b4-4bdb-4a40-bdd9-aa6f34eb09a4","Type":"ContainerStarted","Data":"ffdb80ce44ae54b787658c771aa356fae98fe92439b129641e48e6e677797e8e"} Jul 16 00:46:16.203304 ip-10-0-143-200 kubenswrapper[2572]: I0716 00:46:16.202887 2572 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="opendatahub/opendatahub-operator-controller-manager-5cf8c55d45-bfxb8" Jul 16 00:46:16.239213 ip-10-0-143-200 kubenswrapper[2572]: I0716 00:46:16.239154 2572 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="opendatahub/opendatahub-operator-controller-manager-5cf8c55d45-bfxb8" podStartSLOduration=2.609888806 podStartE2EDuration="7.239135583s" totalImagesPullingTime="3.510176826s" totalInitContainerRuntime="1.119069951s" isStatefulPod=false podCreationTimestamp="2026-07-16 00:46:09 +0000 UTC" imagePullSessionsCount=2 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-16 00:46:16.236610268 +0000 UTC m=+381.345364965" watchObservedRunningTime="2026-07-16 00:46:16.239135583 +0000 UTC m=+381.347890258" Jul 16 00:46:24.298148 ip-10-0-143-200 kubenswrapper[2572]: I0716 00:46:24.298105 2572 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9gnmq_3a7e030d-fad8-480d-9dc6-fe5057048cf9/konnectivity-agent/0.log" limit=1024 Jul 16 00:46:24.357075 ip-10-0-143-200 kubenswrapper[2572]: I0716 00:46:24.357042 2572 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9gnmq_3a7e030d-fad8-480d-9dc6-fe5057048cf9/konnectivity-agent/0.log" limit=1024 Jul 16 00:46:24.743876 ip-10-0-143-200 kubenswrapper[2572]: I0716 00:46:24.743833 2572 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9gnmq_3a7e030d-fad8-480d-9dc6-fe5057048cf9/konnectivity-agent/0.log" limit=1024 Jul 16 00:46:24.830075 ip-10-0-143-200 kubenswrapper[2572]: I0716 00:46:24.830038 2572 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9gnmq_3a7e030d-fad8-480d-9dc6-fe5057048cf9/konnectivity-agent/0.log" limit=1024 Jul 16 00:46:27.208600 ip-10-0-143-200 kubenswrapper[2572]: I0716 00:46:27.208569 2572 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="opendatahub/opendatahub-operator-controller-manager-5cf8c55d45-bfxb8" Jul 16 00:46:29.162870 ip-10-0-143-200 kubenswrapper[2572]: I0716 00:46:29.162833 2572 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9gnmq_3a7e030d-fad8-480d-9dc6-fe5057048cf9/konnectivity-agent/0.log" limit=1024 Jul 16 00:46:29.306704 ip-10-0-143-200 kubenswrapper[2572]: I0716 00:46:29.306658 2572 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9gnmq_3a7e030d-fad8-480d-9dc6-fe5057048cf9/konnectivity-agent/0.log" limit=1024 Jul 16 00:46:29.796958 ip-10-0-143-200 kubenswrapper[2572]: I0716 00:46:29.796911 2572 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9gnmq_3a7e030d-fad8-480d-9dc6-fe5057048cf9/konnectivity-agent/0.log" limit=1024 Jul 16 00:46:29.831567 ip-10-0-143-200 kubenswrapper[2572]: I0716 00:46:29.831531 2572 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-ingress/kube-auth-proxy-7ffc9f7c5d-k6lk9"] Jul 16 00:46:29.840373 ip-10-0-143-200 kubenswrapper[2572]: I0716 00:46:29.840347 2572 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/kube-auth-proxy-7ffc9f7c5d-k6lk9" Jul 16 00:46:29.843564 ip-10-0-143-200 kubenswrapper[2572]: I0716 00:46:29.843539 2572 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress\"/\"kube-auth-proxy-tls\"" Jul 16 00:46:29.843564 ip-10-0-143-200 kubenswrapper[2572]: I0716 00:46:29.843561 2572 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress\"/\"kube-auth-proxy-creds\"" Jul 16 00:46:29.861440 ip-10-0-143-200 kubenswrapper[2572]: I0716 00:46:29.861403 2572 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/kube-auth-proxy-7ffc9f7c5d-k6lk9"] Jul 16 00:46:29.983198 ip-10-0-143-200 kubenswrapper[2572]: I0716 00:46:29.983151 2572 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/48d896b0-4c11-4a5f-813e-84b1195c94a7-tls-certs\") pod \"kube-auth-proxy-7ffc9f7c5d-k6lk9\" (UID: \"48d896b0-4c11-4a5f-813e-84b1195c94a7\") " pod="openshift-ingress/kube-auth-proxy-7ffc9f7c5d-k6lk9" Jul 16 00:46:29.983393 ip-10-0-143-200 kubenswrapper[2572]: I0716 00:46:29.983239 2572 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tmp\" (UniqueName: \"kubernetes.io/empty-dir/48d896b0-4c11-4a5f-813e-84b1195c94a7-tmp\") pod \"kube-auth-proxy-7ffc9f7c5d-k6lk9\" (UID: \"48d896b0-4c11-4a5f-813e-84b1195c94a7\") " pod="openshift-ingress/kube-auth-proxy-7ffc9f7c5d-k6lk9" Jul 16 00:46:29.983393 ip-10-0-143-200 kubenswrapper[2572]: I0716 00:46:29.983285 2572 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-qhxcw\" (UniqueName: \"kubernetes.io/projected/48d896b0-4c11-4a5f-813e-84b1195c94a7-kube-api-access-qhxcw\") pod \"kube-auth-proxy-7ffc9f7c5d-k6lk9\" (UID: \"48d896b0-4c11-4a5f-813e-84b1195c94a7\") " pod="openshift-ingress/kube-auth-proxy-7ffc9f7c5d-k6lk9" Jul 16 00:46:30.046590 ip-10-0-143-200 kubenswrapper[2572]: I0716 00:46:30.046552 2572 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["opendatahub/odh-model-controller-7998c67dc4-xdsc2"] Jul 16 00:46:30.048667 ip-10-0-143-200 kubenswrapper[2572]: I0716 00:46:30.048605 2572 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="opendatahub/odh-model-controller-7998c67dc4-xdsc2" Jul 16 00:46:30.052586 ip-10-0-143-200 kubenswrapper[2572]: I0716 00:46:30.052565 2572 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"opendatahub\"/\"odh-model-controller-dockercfg-sznp8\"" Jul 16 00:46:30.072049 ip-10-0-143-200 kubenswrapper[2572]: I0716 00:46:30.072018 2572 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"opendatahub\"/\"odh-model-controller-webhook-cert\"" Jul 16 00:46:30.083839 ip-10-0-143-200 kubenswrapper[2572]: I0716 00:46:30.083801 2572 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/48d896b0-4c11-4a5f-813e-84b1195c94a7-tls-certs\") pod \"kube-auth-proxy-7ffc9f7c5d-k6lk9\" (UID: \"48d896b0-4c11-4a5f-813e-84b1195c94a7\") " pod="openshift-ingress/kube-auth-proxy-7ffc9f7c5d-k6lk9" Jul 16 00:46:30.084016 ip-10-0-143-200 kubenswrapper[2572]: I0716 00:46:30.083878 2572 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tmp\" (UniqueName: \"kubernetes.io/empty-dir/48d896b0-4c11-4a5f-813e-84b1195c94a7-tmp\") pod \"kube-auth-proxy-7ffc9f7c5d-k6lk9\" (UID: \"48d896b0-4c11-4a5f-813e-84b1195c94a7\") " pod="openshift-ingress/kube-auth-proxy-7ffc9f7c5d-k6lk9" Jul 16 00:46:30.084016 ip-10-0-143-200 kubenswrapper[2572]: I0716 00:46:30.083922 2572 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-qhxcw\" (UniqueName: \"kubernetes.io/projected/48d896b0-4c11-4a5f-813e-84b1195c94a7-kube-api-access-qhxcw\") pod \"kube-auth-proxy-7ffc9f7c5d-k6lk9\" (UID: \"48d896b0-4c11-4a5f-813e-84b1195c94a7\") " pod="openshift-ingress/kube-auth-proxy-7ffc9f7c5d-k6lk9" Jul 16 00:46:30.084016 ip-10-0-143-200 kubenswrapper[2572]: E0716 00:46:30.083955 2572 secret.go:189] Couldn't get secret openshift-ingress/kube-auth-proxy-tls: secret "kube-auth-proxy-tls" not found Jul 16 00:46:30.084179 ip-10-0-143-200 kubenswrapper[2572]: E0716 00:46:30.084039 2572 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/48d896b0-4c11-4a5f-813e-84b1195c94a7-tls-certs podName:48d896b0-4c11-4a5f-813e-84b1195c94a7 nodeName:}" failed. No retries permitted until 2026-07-16 00:46:30.584018676 +0000 UTC m=+395.692773344 (durationBeforeRetry 500ms). Error: MountVolume.SetUp failed for volume "tls-certs" (UniqueName: "kubernetes.io/secret/48d896b0-4c11-4a5f-813e-84b1195c94a7-tls-certs") pod "kube-auth-proxy-7ffc9f7c5d-k6lk9" (UID: "48d896b0-4c11-4a5f-813e-84b1195c94a7") : secret "kube-auth-proxy-tls" not found Jul 16 00:46:30.086166 ip-10-0-143-200 kubenswrapper[2572]: I0716 00:46:30.086144 2572 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"tmp\" (UniqueName: \"kubernetes.io/empty-dir/48d896b0-4c11-4a5f-813e-84b1195c94a7-tmp\") pod \"kube-auth-proxy-7ffc9f7c5d-k6lk9\" (UID: \"48d896b0-4c11-4a5f-813e-84b1195c94a7\") " pod="openshift-ingress/kube-auth-proxy-7ffc9f7c5d-k6lk9" Jul 16 00:46:30.103641 ip-10-0-143-200 kubenswrapper[2572]: I0716 00:46:30.103586 2572 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["opendatahub/odh-model-controller-7998c67dc4-xdsc2"] Jul 16 00:46:30.116362 ip-10-0-143-200 kubenswrapper[2572]: I0716 00:46:30.116329 2572 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-qhxcw\" (UniqueName: \"kubernetes.io/projected/48d896b0-4c11-4a5f-813e-84b1195c94a7-kube-api-access-qhxcw\") pod \"kube-auth-proxy-7ffc9f7c5d-k6lk9\" (UID: \"48d896b0-4c11-4a5f-813e-84b1195c94a7\") " pod="openshift-ingress/kube-auth-proxy-7ffc9f7c5d-k6lk9" Jul 16 00:46:30.184565 ip-10-0-143-200 kubenswrapper[2572]: I0716 00:46:30.184514 2572 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-ds48h\" (UniqueName: \"kubernetes.io/projected/b145063f-5d40-458a-b461-836198d4a9dc-kube-api-access-ds48h\") pod \"odh-model-controller-7998c67dc4-xdsc2\" (UID: \"b145063f-5d40-458a-b461-836198d4a9dc\") " pod="opendatahub/odh-model-controller-7998c67dc4-xdsc2" Jul 16 00:46:30.185183 ip-10-0-143-200 kubenswrapper[2572]: I0716 00:46:30.185149 2572 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"cert\" (UniqueName: \"kubernetes.io/secret/b145063f-5d40-458a-b461-836198d4a9dc-cert\") pod \"odh-model-controller-7998c67dc4-xdsc2\" (UID: \"b145063f-5d40-458a-b461-836198d4a9dc\") " pod="opendatahub/odh-model-controller-7998c67dc4-xdsc2" Jul 16 00:46:30.189118 ip-10-0-143-200 kubenswrapper[2572]: I0716 00:46:30.189086 2572 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9gnmq_3a7e030d-fad8-480d-9dc6-fe5057048cf9/konnectivity-agent/0.log" limit=1024 Jul 16 00:46:30.286684 ip-10-0-143-200 kubenswrapper[2572]: I0716 00:46:30.286615 2572 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-ds48h\" (UniqueName: \"kubernetes.io/projected/b145063f-5d40-458a-b461-836198d4a9dc-kube-api-access-ds48h\") pod \"odh-model-controller-7998c67dc4-xdsc2\" (UID: \"b145063f-5d40-458a-b461-836198d4a9dc\") " pod="opendatahub/odh-model-controller-7998c67dc4-xdsc2" Jul 16 00:46:30.286903 ip-10-0-143-200 kubenswrapper[2572]: I0716 00:46:30.286701 2572 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"cert\" (UniqueName: \"kubernetes.io/secret/b145063f-5d40-458a-b461-836198d4a9dc-cert\") pod \"odh-model-controller-7998c67dc4-xdsc2\" (UID: \"b145063f-5d40-458a-b461-836198d4a9dc\") " pod="opendatahub/odh-model-controller-7998c67dc4-xdsc2" Jul 16 00:46:30.286903 ip-10-0-143-200 kubenswrapper[2572]: E0716 00:46:30.286835 2572 secret.go:189] Couldn't get secret opendatahub/odh-model-controller-webhook-cert: secret "odh-model-controller-webhook-cert" not found Jul 16 00:46:30.286903 ip-10-0-143-200 kubenswrapper[2572]: E0716 00:46:30.286904 2572 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/b145063f-5d40-458a-b461-836198d4a9dc-cert podName:b145063f-5d40-458a-b461-836198d4a9dc nodeName:}" failed. No retries permitted until 2026-07-16 00:46:30.786887985 +0000 UTC m=+395.895642650 (durationBeforeRetry 500ms). Error: MountVolume.SetUp failed for volume "cert" (UniqueName: "kubernetes.io/secret/b145063f-5d40-458a-b461-836198d4a9dc-cert") pod "odh-model-controller-7998c67dc4-xdsc2" (UID: "b145063f-5d40-458a-b461-836198d4a9dc") : secret "odh-model-controller-webhook-cert" not found Jul 16 00:46:30.299239 ip-10-0-143-200 kubenswrapper[2572]: I0716 00:46:30.299160 2572 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-ds48h\" (UniqueName: \"kubernetes.io/projected/b145063f-5d40-458a-b461-836198d4a9dc-kube-api-access-ds48h\") pod \"odh-model-controller-7998c67dc4-xdsc2\" (UID: \"b145063f-5d40-458a-b461-836198d4a9dc\") " pod="opendatahub/odh-model-controller-7998c67dc4-xdsc2" Jul 16 00:46:30.312975 ip-10-0-143-200 kubenswrapper[2572]: I0716 00:46:30.312937 2572 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9gnmq_3a7e030d-fad8-480d-9dc6-fe5057048cf9/konnectivity-agent/0.log" limit=1024 Jul 16 00:46:30.464065 ip-10-0-143-200 kubenswrapper[2572]: I0716 00:46:30.464027 2572 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9gnmq_3a7e030d-fad8-480d-9dc6-fe5057048cf9/konnectivity-agent/0.log" limit=1024 Jul 16 00:46:30.589434 ip-10-0-143-200 kubenswrapper[2572]: I0716 00:46:30.589332 2572 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/48d896b0-4c11-4a5f-813e-84b1195c94a7-tls-certs\") pod \"kube-auth-proxy-7ffc9f7c5d-k6lk9\" (UID: \"48d896b0-4c11-4a5f-813e-84b1195c94a7\") " pod="openshift-ingress/kube-auth-proxy-7ffc9f7c5d-k6lk9" Jul 16 00:46:30.591806 ip-10-0-143-200 kubenswrapper[2572]: I0716 00:46:30.591785 2572 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/48d896b0-4c11-4a5f-813e-84b1195c94a7-tls-certs\") pod \"kube-auth-proxy-7ffc9f7c5d-k6lk9\" (UID: \"48d896b0-4c11-4a5f-813e-84b1195c94a7\") " pod="openshift-ingress/kube-auth-proxy-7ffc9f7c5d-k6lk9" Jul 16 00:46:30.614221 ip-10-0-143-200 kubenswrapper[2572]: I0716 00:46:30.614180 2572 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9gnmq_3a7e030d-fad8-480d-9dc6-fe5057048cf9/konnectivity-agent/0.log" limit=1024 Jul 16 00:46:30.750283 ip-10-0-143-200 kubenswrapper[2572]: I0716 00:46:30.750245 2572 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/kube-auth-proxy-7ffc9f7c5d-k6lk9" Jul 16 00:46:30.791332 ip-10-0-143-200 kubenswrapper[2572]: I0716 00:46:30.791296 2572 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"cert\" (UniqueName: \"kubernetes.io/secret/b145063f-5d40-458a-b461-836198d4a9dc-cert\") pod \"odh-model-controller-7998c67dc4-xdsc2\" (UID: \"b145063f-5d40-458a-b461-836198d4a9dc\") " pod="opendatahub/odh-model-controller-7998c67dc4-xdsc2" Jul 16 00:46:30.793737 ip-10-0-143-200 kubenswrapper[2572]: I0716 00:46:30.793713 2572 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"cert\" (UniqueName: \"kubernetes.io/secret/b145063f-5d40-458a-b461-836198d4a9dc-cert\") pod \"odh-model-controller-7998c67dc4-xdsc2\" (UID: \"b145063f-5d40-458a-b461-836198d4a9dc\") " pod="opendatahub/odh-model-controller-7998c67dc4-xdsc2" Jul 16 00:46:30.925339 ip-10-0-143-200 kubenswrapper[2572]: I0716 00:46:30.925191 2572 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/kube-auth-proxy-7ffc9f7c5d-k6lk9"] Jul 16 00:46:30.927976 ip-10-0-143-200 kubenswrapper[2572]: W0716 00:46:30.927949 2572 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod48d896b0_4c11_4a5f_813e_84b1195c94a7.slice/crio-3559a4159803ab940edb9d09e2e4b960564368b027963dc6596313bd4a2f80ba WatchSource:0}: Error finding container 3559a4159803ab940edb9d09e2e4b960564368b027963dc6596313bd4a2f80ba: Status 404 returned error can't find the container with id 3559a4159803ab940edb9d09e2e4b960564368b027963dc6596313bd4a2f80ba Jul 16 00:46:30.958516 ip-10-0-143-200 kubenswrapper[2572]: I0716 00:46:30.958474 2572 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="opendatahub/odh-model-controller-7998c67dc4-xdsc2" Jul 16 00:46:31.237350 ip-10-0-143-200 kubenswrapper[2572]: I0716 00:46:31.237318 2572 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["opendatahub/odh-model-controller-7998c67dc4-xdsc2"] Jul 16 00:46:31.243719 ip-10-0-143-200 kubenswrapper[2572]: W0716 00:46:31.243687 2572 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podb145063f_5d40_458a_b461_836198d4a9dc.slice/crio-80c09119f650eb6d9ad355fe983a418f0631d71e85fb4cc47651940d22954a07 WatchSource:0}: Error finding container 80c09119f650eb6d9ad355fe983a418f0631d71e85fb4cc47651940d22954a07: Status 404 returned error can't find the container with id 80c09119f650eb6d9ad355fe983a418f0631d71e85fb4cc47651940d22954a07 Jul 16 00:46:31.265068 ip-10-0-143-200 kubenswrapper[2572]: I0716 00:46:31.265038 2572 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="opendatahub/odh-model-controller-7998c67dc4-xdsc2" event={"ID":"b145063f-5d40-458a-b461-836198d4a9dc","Type":"ContainerStarted","Data":"80c09119f650eb6d9ad355fe983a418f0631d71e85fb4cc47651940d22954a07"} Jul 16 00:46:31.266159 ip-10-0-143-200 kubenswrapper[2572]: I0716 00:46:31.266125 2572 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/kube-auth-proxy-7ffc9f7c5d-k6lk9" event={"ID":"48d896b0-4c11-4a5f-813e-84b1195c94a7","Type":"ContainerStarted","Data":"3559a4159803ab940edb9d09e2e4b960564368b027963dc6596313bd4a2f80ba"} Jul 16 00:46:31.690488 ip-10-0-143-200 kubenswrapper[2572]: I0716 00:46:31.690379 2572 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9gnmq_3a7e030d-fad8-480d-9dc6-fe5057048cf9/konnectivity-agent/0.log" limit=1024 Jul 16 00:46:31.697697 ip-10-0-143-200 kubenswrapper[2572]: I0716 00:46:31.697666 2572 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["opendatahub/model-serving-api-54d89bd5fc-xhll8"] Jul 16 00:46:31.700291 ip-10-0-143-200 kubenswrapper[2572]: I0716 00:46:31.700265 2572 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="opendatahub/model-serving-api-54d89bd5fc-xhll8" Jul 16 00:46:31.704762 ip-10-0-143-200 kubenswrapper[2572]: I0716 00:46:31.704737 2572 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"opendatahub\"/\"model-serving-api-tls\"" Jul 16 00:46:31.730615 ip-10-0-143-200 kubenswrapper[2572]: I0716 00:46:31.730579 2572 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["opendatahub/model-serving-api-54d89bd5fc-xhll8"] Jul 16 00:46:31.800436 ip-10-0-143-200 kubenswrapper[2572]: I0716 00:46:31.800398 2572 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-84fk9\" (UniqueName: \"kubernetes.io/projected/d0512aa0-6a35-4682-8ada-3de714156d10-kube-api-access-84fk9\") pod \"model-serving-api-54d89bd5fc-xhll8\" (UID: \"d0512aa0-6a35-4682-8ada-3de714156d10\") " pod="opendatahub/model-serving-api-54d89bd5fc-xhll8" Jul 16 00:46:31.800436 ip-10-0-143-200 kubenswrapper[2572]: I0716 00:46:31.800440 2572 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/d0512aa0-6a35-4682-8ada-3de714156d10-tls-certs\") pod \"model-serving-api-54d89bd5fc-xhll8\" (UID: \"d0512aa0-6a35-4682-8ada-3de714156d10\") " pod="opendatahub/model-serving-api-54d89bd5fc-xhll8" Jul 16 00:46:31.901417 ip-10-0-143-200 kubenswrapper[2572]: I0716 00:46:31.901376 2572 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-84fk9\" (UniqueName: \"kubernetes.io/projected/d0512aa0-6a35-4682-8ada-3de714156d10-kube-api-access-84fk9\") pod \"model-serving-api-54d89bd5fc-xhll8\" (UID: \"d0512aa0-6a35-4682-8ada-3de714156d10\") " pod="opendatahub/model-serving-api-54d89bd5fc-xhll8" Jul 16 00:46:31.901615 ip-10-0-143-200 kubenswrapper[2572]: I0716 00:46:31.901444 2572 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/d0512aa0-6a35-4682-8ada-3de714156d10-tls-certs\") pod \"model-serving-api-54d89bd5fc-xhll8\" (UID: \"d0512aa0-6a35-4682-8ada-3de714156d10\") " pod="opendatahub/model-serving-api-54d89bd5fc-xhll8" Jul 16 00:46:31.905348 ip-10-0-143-200 kubenswrapper[2572]: I0716 00:46:31.905316 2572 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/d0512aa0-6a35-4682-8ada-3de714156d10-tls-certs\") pod \"model-serving-api-54d89bd5fc-xhll8\" (UID: \"d0512aa0-6a35-4682-8ada-3de714156d10\") " pod="opendatahub/model-serving-api-54d89bd5fc-xhll8" Jul 16 00:46:31.920560 ip-10-0-143-200 kubenswrapper[2572]: I0716 00:46:31.920525 2572 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-84fk9\" (UniqueName: \"kubernetes.io/projected/d0512aa0-6a35-4682-8ada-3de714156d10-kube-api-access-84fk9\") pod \"model-serving-api-54d89bd5fc-xhll8\" (UID: \"d0512aa0-6a35-4682-8ada-3de714156d10\") " pod="opendatahub/model-serving-api-54d89bd5fc-xhll8" Jul 16 00:46:31.981753 ip-10-0-143-200 kubenswrapper[2572]: I0716 00:46:31.981718 2572 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9gnmq_3a7e030d-fad8-480d-9dc6-fe5057048cf9/konnectivity-agent/0.log" limit=1024 Jul 16 00:46:32.015073 ip-10-0-143-200 kubenswrapper[2572]: I0716 00:46:32.015029 2572 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="opendatahub/model-serving-api-54d89bd5fc-xhll8" Jul 16 00:46:32.197386 ip-10-0-143-200 kubenswrapper[2572]: I0716 00:46:32.197296 2572 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["opendatahub/model-serving-api-54d89bd5fc-xhll8"] Jul 16 00:46:32.202722 ip-10-0-143-200 kubenswrapper[2572]: W0716 00:46:32.202683 2572 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podd0512aa0_6a35_4682_8ada_3de714156d10.slice/crio-4b3bb62f57e01fa7897e5266a2e7787d19a7097ffcb9d8b7ac67e61ead8d66fe WatchSource:0}: Error finding container 4b3bb62f57e01fa7897e5266a2e7787d19a7097ffcb9d8b7ac67e61ead8d66fe: Status 404 returned error can't find the container with id 4b3bb62f57e01fa7897e5266a2e7787d19a7097ffcb9d8b7ac67e61ead8d66fe Jul 16 00:46:32.272652 ip-10-0-143-200 kubenswrapper[2572]: I0716 00:46:32.272536 2572 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="opendatahub/model-serving-api-54d89bd5fc-xhll8" event={"ID":"d0512aa0-6a35-4682-8ada-3de714156d10","Type":"ContainerStarted","Data":"4b3bb62f57e01fa7897e5266a2e7787d19a7097ffcb9d8b7ac67e61ead8d66fe"} Jul 16 00:46:32.498599 ip-10-0-143-200 kubenswrapper[2572]: I0716 00:46:32.498191 2572 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9gnmq_3a7e030d-fad8-480d-9dc6-fe5057048cf9/konnectivity-agent/0.log" limit=1024 Jul 16 00:46:32.516376 ip-10-0-143-200 kubenswrapper[2572]: I0716 00:46:32.516337 2572 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-ingress/istiod-openshift-gateway-7574459656-flnhz"] Jul 16 00:46:32.520526 ip-10-0-143-200 kubenswrapper[2572]: I0716 00:46:32.520497 2572 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/istiod-openshift-gateway-7574459656-flnhz" Jul 16 00:46:32.523823 ip-10-0-143-200 kubenswrapper[2572]: I0716 00:46:32.523743 2572 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress\"/\"cacerts\"" Jul 16 00:46:32.523994 ip-10-0-143-200 kubenswrapper[2572]: I0716 00:46:32.523978 2572 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress\"/\"istio-kubeconfig\"" Jul 16 00:46:32.524285 ip-10-0-143-200 kubenswrapper[2572]: I0716 00:46:32.524267 2572 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress\"/\"istiod-tls\"" Jul 16 00:46:32.524501 ip-10-0-143-200 kubenswrapper[2572]: I0716 00:46:32.524485 2572 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress\"/\"istiod-openshift-gateway-dockercfg-lggwx\"" Jul 16 00:46:32.525111 ip-10-0-143-200 kubenswrapper[2572]: I0716 00:46:32.525070 2572 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ingress\"/\"openshift-gw-ca-root-cert\"" Jul 16 00:46:32.536486 ip-10-0-143-200 kubenswrapper[2572]: I0716 00:46:32.536460 2572 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/istiod-openshift-gateway-7574459656-flnhz"] Jul 16 00:46:32.594864 ip-10-0-143-200 kubenswrapper[2572]: I0716 00:46:32.594820 2572 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9gnmq_3a7e030d-fad8-480d-9dc6-fe5057048cf9/konnectivity-agent/0.log" limit=1024 Jul 16 00:46:32.608350 ip-10-0-143-200 kubenswrapper[2572]: I0716 00:46:32.608310 2572 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"local-certs\" (UniqueName: \"kubernetes.io/empty-dir/f16361c6-509c-4261-8777-872298146540-local-certs\") pod \"istiod-openshift-gateway-7574459656-flnhz\" (UID: \"f16361c6-509c-4261-8777-872298146540\") " pod="openshift-ingress/istiod-openshift-gateway-7574459656-flnhz" Jul 16 00:46:32.608553 ip-10-0-143-200 kubenswrapper[2572]: I0716 00:46:32.608371 2572 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-csr-ca-configmap\" (UniqueName: \"kubernetes.io/configmap/f16361c6-509c-4261-8777-872298146540-istio-csr-ca-configmap\") pod \"istiod-openshift-gateway-7574459656-flnhz\" (UID: \"f16361c6-509c-4261-8777-872298146540\") " pod="openshift-ingress/istiod-openshift-gateway-7574459656-flnhz" Jul 16 00:46:32.608553 ip-10-0-143-200 kubenswrapper[2572]: I0716 00:46:32.608407 2572 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-kubeconfig\" (UniqueName: \"kubernetes.io/secret/f16361c6-509c-4261-8777-872298146540-istio-kubeconfig\") pod \"istiod-openshift-gateway-7574459656-flnhz\" (UID: \"f16361c6-509c-4261-8777-872298146540\") " pod="openshift-ingress/istiod-openshift-gateway-7574459656-flnhz" Jul 16 00:46:32.608553 ip-10-0-143-200 kubenswrapper[2572]: I0716 00:46:32.608430 2572 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-kmf7g\" (UniqueName: \"kubernetes.io/projected/f16361c6-509c-4261-8777-872298146540-kube-api-access-kmf7g\") pod \"istiod-openshift-gateway-7574459656-flnhz\" (UID: \"f16361c6-509c-4261-8777-872298146540\") " pod="openshift-ingress/istiod-openshift-gateway-7574459656-flnhz" Jul 16 00:46:32.608763 ip-10-0-143-200 kubenswrapper[2572]: I0716 00:46:32.608554 2572 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/f16361c6-509c-4261-8777-872298146540-istio-token\") pod \"istiod-openshift-gateway-7574459656-flnhz\" (UID: \"f16361c6-509c-4261-8777-872298146540\") " pod="openshift-ingress/istiod-openshift-gateway-7574459656-flnhz" Jul 16 00:46:32.608763 ip-10-0-143-200 kubenswrapper[2572]: I0716 00:46:32.608650 2572 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"cacerts\" (UniqueName: \"kubernetes.io/secret/f16361c6-509c-4261-8777-872298146540-cacerts\") pod \"istiod-openshift-gateway-7574459656-flnhz\" (UID: \"f16361c6-509c-4261-8777-872298146540\") " pod="openshift-ingress/istiod-openshift-gateway-7574459656-flnhz" Jul 16 00:46:32.608763 ip-10-0-143-200 kubenswrapper[2572]: I0716 00:46:32.608678 2572 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-csr-dns-cert\" (UniqueName: \"kubernetes.io/secret/f16361c6-509c-4261-8777-872298146540-istio-csr-dns-cert\") pod \"istiod-openshift-gateway-7574459656-flnhz\" (UID: \"f16361c6-509c-4261-8777-872298146540\") " pod="openshift-ingress/istiod-openshift-gateway-7574459656-flnhz" Jul 16 00:46:32.709224 ip-10-0-143-200 kubenswrapper[2572]: I0716 00:46:32.709077 2572 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-csr-ca-configmap\" (UniqueName: \"kubernetes.io/configmap/f16361c6-509c-4261-8777-872298146540-istio-csr-ca-configmap\") pod \"istiod-openshift-gateway-7574459656-flnhz\" (UID: \"f16361c6-509c-4261-8777-872298146540\") " pod="openshift-ingress/istiod-openshift-gateway-7574459656-flnhz" Jul 16 00:46:32.709224 ip-10-0-143-200 kubenswrapper[2572]: I0716 00:46:32.709142 2572 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-kubeconfig\" (UniqueName: \"kubernetes.io/secret/f16361c6-509c-4261-8777-872298146540-istio-kubeconfig\") pod \"istiod-openshift-gateway-7574459656-flnhz\" (UID: \"f16361c6-509c-4261-8777-872298146540\") " pod="openshift-ingress/istiod-openshift-gateway-7574459656-flnhz" Jul 16 00:46:32.709224 ip-10-0-143-200 kubenswrapper[2572]: I0716 00:46:32.709174 2572 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-kmf7g\" (UniqueName: \"kubernetes.io/projected/f16361c6-509c-4261-8777-872298146540-kube-api-access-kmf7g\") pod \"istiod-openshift-gateway-7574459656-flnhz\" (UID: \"f16361c6-509c-4261-8777-872298146540\") " pod="openshift-ingress/istiod-openshift-gateway-7574459656-flnhz" Jul 16 00:46:32.709224 ip-10-0-143-200 kubenswrapper[2572]: I0716 00:46:32.709218 2572 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/f16361c6-509c-4261-8777-872298146540-istio-token\") pod \"istiod-openshift-gateway-7574459656-flnhz\" (UID: \"f16361c6-509c-4261-8777-872298146540\") " pod="openshift-ingress/istiod-openshift-gateway-7574459656-flnhz" Jul 16 00:46:32.709566 ip-10-0-143-200 kubenswrapper[2572]: I0716 00:46:32.709269 2572 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"cacerts\" (UniqueName: \"kubernetes.io/secret/f16361c6-509c-4261-8777-872298146540-cacerts\") pod \"istiod-openshift-gateway-7574459656-flnhz\" (UID: \"f16361c6-509c-4261-8777-872298146540\") " pod="openshift-ingress/istiod-openshift-gateway-7574459656-flnhz" Jul 16 00:46:32.709566 ip-10-0-143-200 kubenswrapper[2572]: I0716 00:46:32.709297 2572 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-csr-dns-cert\" (UniqueName: \"kubernetes.io/secret/f16361c6-509c-4261-8777-872298146540-istio-csr-dns-cert\") pod \"istiod-openshift-gateway-7574459656-flnhz\" (UID: \"f16361c6-509c-4261-8777-872298146540\") " pod="openshift-ingress/istiod-openshift-gateway-7574459656-flnhz" Jul 16 00:46:32.709566 ip-10-0-143-200 kubenswrapper[2572]: I0716 00:46:32.709377 2572 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"local-certs\" (UniqueName: \"kubernetes.io/empty-dir/f16361c6-509c-4261-8777-872298146540-local-certs\") pod \"istiod-openshift-gateway-7574459656-flnhz\" (UID: \"f16361c6-509c-4261-8777-872298146540\") " pod="openshift-ingress/istiod-openshift-gateway-7574459656-flnhz" Jul 16 00:46:32.711154 ip-10-0-143-200 kubenswrapper[2572]: I0716 00:46:32.711120 2572 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istio-csr-ca-configmap\" (UniqueName: \"kubernetes.io/configmap/f16361c6-509c-4261-8777-872298146540-istio-csr-ca-configmap\") pod \"istiod-openshift-gateway-7574459656-flnhz\" (UID: \"f16361c6-509c-4261-8777-872298146540\") " pod="openshift-ingress/istiod-openshift-gateway-7574459656-flnhz" Jul 16 00:46:32.713164 ip-10-0-143-200 kubenswrapper[2572]: I0716 00:46:32.713136 2572 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istio-csr-dns-cert\" (UniqueName: \"kubernetes.io/secret/f16361c6-509c-4261-8777-872298146540-istio-csr-dns-cert\") pod \"istiod-openshift-gateway-7574459656-flnhz\" (UID: \"f16361c6-509c-4261-8777-872298146540\") " pod="openshift-ingress/istiod-openshift-gateway-7574459656-flnhz" Jul 16 00:46:32.713406 ip-10-0-143-200 kubenswrapper[2572]: I0716 00:46:32.713379 2572 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"local-certs\" (UniqueName: \"kubernetes.io/empty-dir/f16361c6-509c-4261-8777-872298146540-local-certs\") pod \"istiod-openshift-gateway-7574459656-flnhz\" (UID: \"f16361c6-509c-4261-8777-872298146540\") " pod="openshift-ingress/istiod-openshift-gateway-7574459656-flnhz" Jul 16 00:46:32.714901 ip-10-0-143-200 kubenswrapper[2572]: I0716 00:46:32.714873 2572 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"cacerts\" (UniqueName: \"kubernetes.io/secret/f16361c6-509c-4261-8777-872298146540-cacerts\") pod \"istiod-openshift-gateway-7574459656-flnhz\" (UID: \"f16361c6-509c-4261-8777-872298146540\") " pod="openshift-ingress/istiod-openshift-gateway-7574459656-flnhz" Jul 16 00:46:32.715040 ip-10-0-143-200 kubenswrapper[2572]: I0716 00:46:32.715023 2572 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istio-kubeconfig\" (UniqueName: \"kubernetes.io/secret/f16361c6-509c-4261-8777-872298146540-istio-kubeconfig\") pod \"istiod-openshift-gateway-7574459656-flnhz\" (UID: \"f16361c6-509c-4261-8777-872298146540\") " pod="openshift-ingress/istiod-openshift-gateway-7574459656-flnhz" Jul 16 00:46:32.720156 ip-10-0-143-200 kubenswrapper[2572]: I0716 00:46:32.720131 2572 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-kmf7g\" (UniqueName: \"kubernetes.io/projected/f16361c6-509c-4261-8777-872298146540-kube-api-access-kmf7g\") pod \"istiod-openshift-gateway-7574459656-flnhz\" (UID: \"f16361c6-509c-4261-8777-872298146540\") " pod="openshift-ingress/istiod-openshift-gateway-7574459656-flnhz" Jul 16 00:46:32.722034 ip-10-0-143-200 kubenswrapper[2572]: I0716 00:46:32.722006 2572 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/f16361c6-509c-4261-8777-872298146540-istio-token\") pod \"istiod-openshift-gateway-7574459656-flnhz\" (UID: \"f16361c6-509c-4261-8777-872298146540\") " pod="openshift-ingress/istiod-openshift-gateway-7574459656-flnhz" Jul 16 00:46:32.835027 ip-10-0-143-200 kubenswrapper[2572]: I0716 00:46:32.834934 2572 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/istiod-openshift-gateway-7574459656-flnhz" Jul 16 00:46:34.199395 ip-10-0-143-200 kubenswrapper[2572]: I0716 00:46:34.199347 2572 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9gnmq_3a7e030d-fad8-480d-9dc6-fe5057048cf9/konnectivity-agent/0.log" limit=1024 Jul 16 00:46:35.293065 ip-10-0-143-200 kubenswrapper[2572]: I0716 00:46:35.292981 2572 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/kube-auth-proxy-7ffc9f7c5d-k6lk9" event={"ID":"48d896b0-4c11-4a5f-813e-84b1195c94a7","Type":"ContainerStarted","Data":"deebc6cb9aa2b7ae9e5d63377eefe0947e712980e7787dc25484dbb091f0d677"} Jul 16 00:46:35.296169 ip-10-0-143-200 kubenswrapper[2572]: I0716 00:46:35.295824 2572 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="opendatahub/model-serving-api-54d89bd5fc-xhll8" event={"ID":"d0512aa0-6a35-4682-8ada-3de714156d10","Type":"ContainerStarted","Data":"871cca634631ad727b4da21e91e2719f00cfc73f529ad611e9d8538e3c7b0a81"} Jul 16 00:46:35.296169 ip-10-0-143-200 kubenswrapper[2572]: I0716 00:46:35.295942 2572 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="opendatahub/model-serving-api-54d89bd5fc-xhll8" Jul 16 00:46:35.297493 ip-10-0-143-200 kubenswrapper[2572]: I0716 00:46:35.297471 2572 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="opendatahub/odh-model-controller-7998c67dc4-xdsc2" event={"ID":"b145063f-5d40-458a-b461-836198d4a9dc","Type":"ContainerStarted","Data":"8b83899720fe547a4a9ef0adb604c81afcd90b91407aec3195dd5b37733d85d0"} Jul 16 00:46:35.297648 ip-10-0-143-200 kubenswrapper[2572]: I0716 00:46:35.297522 2572 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="opendatahub/odh-model-controller-7998c67dc4-xdsc2" Jul 16 00:46:35.323816 ip-10-0-143-200 kubenswrapper[2572]: I0716 00:46:35.323763 2572 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-ingress/kube-auth-proxy-7ffc9f7c5d-k6lk9" podStartSLOduration=2.088776625 podStartE2EDuration="6.323743328s" totalImagesPullingTime="4.234966703s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-16 00:46:29 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-16 00:46:35.321196443 +0000 UTC m=+400.429951117" watchObservedRunningTime="2026-07-16 00:46:35.323743328 +0000 UTC m=+400.432498003" Jul 16 00:46:35.340963 ip-10-0-143-200 kubenswrapper[2572]: I0716 00:46:35.340928 2572 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/istiod-openshift-gateway-7574459656-flnhz"] Jul 16 00:46:35.344003 ip-10-0-143-200 kubenswrapper[2572]: W0716 00:46:35.343958 2572 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podf16361c6_509c_4261_8777_872298146540.slice/crio-63eacae4f6e5b00582b51fedb5f1948c6b1209fe6eefa7885241914fd86501c6 WatchSource:0}: Error finding container 63eacae4f6e5b00582b51fedb5f1948c6b1209fe6eefa7885241914fd86501c6: Status 404 returned error can't find the container with id 63eacae4f6e5b00582b51fedb5f1948c6b1209fe6eefa7885241914fd86501c6 Jul 16 00:46:35.347219 ip-10-0-143-200 kubenswrapper[2572]: I0716 00:46:35.347178 2572 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="opendatahub/odh-model-controller-7998c67dc4-xdsc2" podStartSLOduration=2.429165015 podStartE2EDuration="6.347161793s" totalImagesPullingTime="3.917996778s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-16 00:46:29 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-16 00:46:35.345132074 +0000 UTC m=+400.453886750" watchObservedRunningTime="2026-07-16 00:46:35.347161793 +0000 UTC m=+400.455916468" Jul 16 00:46:35.375306 ip-10-0-143-200 kubenswrapper[2572]: I0716 00:46:35.375240 2572 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="opendatahub/model-serving-api-54d89bd5fc-xhll8" podStartSLOduration=1.418269706 podStartE2EDuration="4.375217845s" totalImagesPullingTime="2.956948139s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-16 00:46:31 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-16 00:46:35.372041542 +0000 UTC m=+400.480796218" watchObservedRunningTime="2026-07-16 00:46:35.375217845 +0000 UTC m=+400.483972522" Jul 16 00:46:36.305060 ip-10-0-143-200 kubenswrapper[2572]: I0716 00:46:36.305020 2572 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/istiod-openshift-gateway-7574459656-flnhz" event={"ID":"f16361c6-509c-4261-8777-872298146540","Type":"ContainerStarted","Data":"63eacae4f6e5b00582b51fedb5f1948c6b1209fe6eefa7885241914fd86501c6"} Jul 16 00:46:39.304268 ip-10-0-143-200 kubenswrapper[2572]: I0716 00:46:39.304213 2572 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236228Ki","pods":"250"} Jul 16 00:46:39.304545 ip-10-0-143-200 kubenswrapper[2572]: I0716 00:46:39.304312 2572 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236228Ki","pods":"250"} Jul 16 00:46:39.949321 ip-10-0-143-200 kubenswrapper[2572]: I0716 00:46:39.949268 2572 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9gnmq_3a7e030d-fad8-480d-9dc6-fe5057048cf9/konnectivity-agent/0.log" limit=1024 Jul 16 00:46:40.084915 ip-10-0-143-200 kubenswrapper[2572]: I0716 00:46:40.084864 2572 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9gnmq_3a7e030d-fad8-480d-9dc6-fe5057048cf9/konnectivity-agent/0.log" limit=1024 Jul 16 00:46:40.325439 ip-10-0-143-200 kubenswrapper[2572]: I0716 00:46:40.325401 2572 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/istiod-openshift-gateway-7574459656-flnhz" event={"ID":"f16361c6-509c-4261-8777-872298146540","Type":"ContainerStarted","Data":"bf1949f36aecc7b8c7d7dec845b7f4b73276c6ca8508a8e36309b6419ae8f800"} Jul 16 00:46:40.325943 ip-10-0-143-200 kubenswrapper[2572]: I0716 00:46:40.325611 2572 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/istiod-openshift-gateway-7574459656-flnhz" Jul 16 00:46:40.327444 ip-10-0-143-200 kubenswrapper[2572]: I0716 00:46:40.327420 2572 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/istiod-openshift-gateway-7574459656-flnhz" Jul 16 00:46:40.519934 ip-10-0-143-200 kubenswrapper[2572]: I0716 00:46:40.519876 2572 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-ingress/istiod-openshift-gateway-7574459656-flnhz" podStartSLOduration=4.562339762 podStartE2EDuration="8.51985527s" totalImagesPullingTime="3.957515508s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-16 00:46:32 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-16 00:46:40.477833206 +0000 UTC m=+405.586587863" watchObservedRunningTime="2026-07-16 00:46:40.51985527 +0000 UTC m=+405.628609945" Jul 16 00:46:40.520806 ip-10-0-143-200 kubenswrapper[2572]: I0716 00:46:40.520784 2572 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-ingress/data-science-gateway-data-science-gateway-class-6576859cf4tzwx2"] Jul 16 00:46:40.523738 ip-10-0-143-200 kubenswrapper[2572]: I0716 00:46:40.523722 2572 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/data-science-gateway-data-science-gateway-class-6576859cf4tzwx2" Jul 16 00:46:40.539057 ip-10-0-143-200 kubenswrapper[2572]: I0716 00:46:40.539022 2572 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress\"/\"data-science-gateway-data-science-gateway-class-dockercfg-fbmz6\"" Jul 16 00:46:40.579012 ip-10-0-143-200 kubenswrapper[2572]: I0716 00:46:40.578923 2572 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/data-science-gateway-data-science-gateway-class-6576859cf4tzwx2"] Jul 16 00:46:40.663682 ip-10-0-143-200 kubenswrapper[2572]: I0716 00:46:40.663645 2572 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9gnmq_3a7e030d-fad8-480d-9dc6-fe5057048cf9/konnectivity-agent/0.log" limit=1024 Jul 16 00:46:40.685077 ip-10-0-143-200 kubenswrapper[2572]: I0716 00:46:40.685035 2572 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/485db377-3d7c-46e8-9d8c-547ab86d96c0-istio-token\") pod \"data-science-gateway-data-science-gateway-class-6576859cf4tzwx2\" (UID: \"485db377-3d7c-46e8-9d8c-547ab86d96c0\") " pod="openshift-ingress/data-science-gateway-data-science-gateway-class-6576859cf4tzwx2" Jul 16 00:46:40.685224 ip-10-0-143-200 kubenswrapper[2572]: I0716 00:46:40.685094 2572 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/485db377-3d7c-46e8-9d8c-547ab86d96c0-istio-data\") pod \"data-science-gateway-data-science-gateway-class-6576859cf4tzwx2\" (UID: \"485db377-3d7c-46e8-9d8c-547ab86d96c0\") " pod="openshift-ingress/data-science-gateway-data-science-gateway-class-6576859cf4tzwx2" Jul 16 00:46:40.685224 ip-10-0-143-200 kubenswrapper[2572]: I0716 00:46:40.685128 2572 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/485db377-3d7c-46e8-9d8c-547ab86d96c0-istiod-ca-cert\") pod \"data-science-gateway-data-science-gateway-class-6576859cf4tzwx2\" (UID: \"485db377-3d7c-46e8-9d8c-547ab86d96c0\") " pod="openshift-ingress/data-science-gateway-data-science-gateway-class-6576859cf4tzwx2" Jul 16 00:46:40.685224 ip-10-0-143-200 kubenswrapper[2572]: I0716 00:46:40.685155 2572 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/485db377-3d7c-46e8-9d8c-547ab86d96c0-istio-envoy\") pod \"data-science-gateway-data-science-gateway-class-6576859cf4tzwx2\" (UID: \"485db377-3d7c-46e8-9d8c-547ab86d96c0\") " pod="openshift-ingress/data-science-gateway-data-science-gateway-class-6576859cf4tzwx2" Jul 16 00:46:40.685224 ip-10-0-143-200 kubenswrapper[2572]: I0716 00:46:40.685208 2572 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/485db377-3d7c-46e8-9d8c-547ab86d96c0-workload-socket\") pod \"data-science-gateway-data-science-gateway-class-6576859cf4tzwx2\" (UID: \"485db377-3d7c-46e8-9d8c-547ab86d96c0\") " pod="openshift-ingress/data-science-gateway-data-science-gateway-class-6576859cf4tzwx2" Jul 16 00:46:40.685368 ip-10-0-143-200 kubenswrapper[2572]: I0716 00:46:40.685234 2572 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/485db377-3d7c-46e8-9d8c-547ab86d96c0-istio-podinfo\") pod \"data-science-gateway-data-science-gateway-class-6576859cf4tzwx2\" (UID: \"485db377-3d7c-46e8-9d8c-547ab86d96c0\") " pod="openshift-ingress/data-science-gateway-data-science-gateway-class-6576859cf4tzwx2" Jul 16 00:46:40.685368 ip-10-0-143-200 kubenswrapper[2572]: I0716 00:46:40.685280 2572 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/485db377-3d7c-46e8-9d8c-547ab86d96c0-workload-certs\") pod \"data-science-gateway-data-science-gateway-class-6576859cf4tzwx2\" (UID: \"485db377-3d7c-46e8-9d8c-547ab86d96c0\") " pod="openshift-ingress/data-science-gateway-data-science-gateway-class-6576859cf4tzwx2" Jul 16 00:46:40.685368 ip-10-0-143-200 kubenswrapper[2572]: I0716 00:46:40.685347 2572 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/485db377-3d7c-46e8-9d8c-547ab86d96c0-credential-socket\") pod \"data-science-gateway-data-science-gateway-class-6576859cf4tzwx2\" (UID: \"485db377-3d7c-46e8-9d8c-547ab86d96c0\") " pod="openshift-ingress/data-science-gateway-data-science-gateway-class-6576859cf4tzwx2" Jul 16 00:46:40.685460 ip-10-0-143-200 kubenswrapper[2572]: I0716 00:46:40.685375 2572 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-kdwf2\" (UniqueName: \"kubernetes.io/projected/485db377-3d7c-46e8-9d8c-547ab86d96c0-kube-api-access-kdwf2\") pod \"data-science-gateway-data-science-gateway-class-6576859cf4tzwx2\" (UID: \"485db377-3d7c-46e8-9d8c-547ab86d96c0\") " pod="openshift-ingress/data-science-gateway-data-science-gateway-class-6576859cf4tzwx2" Jul 16 00:46:40.786288 ip-10-0-143-200 kubenswrapper[2572]: I0716 00:46:40.786242 2572 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/485db377-3d7c-46e8-9d8c-547ab86d96c0-workload-socket\") pod \"data-science-gateway-data-science-gateway-class-6576859cf4tzwx2\" (UID: \"485db377-3d7c-46e8-9d8c-547ab86d96c0\") " pod="openshift-ingress/data-science-gateway-data-science-gateway-class-6576859cf4tzwx2" Jul 16 00:46:40.786288 ip-10-0-143-200 kubenswrapper[2572]: I0716 00:46:40.786290 2572 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/485db377-3d7c-46e8-9d8c-547ab86d96c0-istio-podinfo\") pod \"data-science-gateway-data-science-gateway-class-6576859cf4tzwx2\" (UID: \"485db377-3d7c-46e8-9d8c-547ab86d96c0\") " pod="openshift-ingress/data-science-gateway-data-science-gateway-class-6576859cf4tzwx2" Jul 16 00:46:40.786553 ip-10-0-143-200 kubenswrapper[2572]: I0716 00:46:40.786342 2572 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/485db377-3d7c-46e8-9d8c-547ab86d96c0-workload-certs\") pod \"data-science-gateway-data-science-gateway-class-6576859cf4tzwx2\" (UID: \"485db377-3d7c-46e8-9d8c-547ab86d96c0\") " pod="openshift-ingress/data-science-gateway-data-science-gateway-class-6576859cf4tzwx2" Jul 16 00:46:40.786553 ip-10-0-143-200 kubenswrapper[2572]: I0716 00:46:40.786384 2572 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/485db377-3d7c-46e8-9d8c-547ab86d96c0-credential-socket\") pod \"data-science-gateway-data-science-gateway-class-6576859cf4tzwx2\" (UID: \"485db377-3d7c-46e8-9d8c-547ab86d96c0\") " pod="openshift-ingress/data-science-gateway-data-science-gateway-class-6576859cf4tzwx2" Jul 16 00:46:40.786553 ip-10-0-143-200 kubenswrapper[2572]: I0716 00:46:40.786409 2572 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-kdwf2\" (UniqueName: \"kubernetes.io/projected/485db377-3d7c-46e8-9d8c-547ab86d96c0-kube-api-access-kdwf2\") pod \"data-science-gateway-data-science-gateway-class-6576859cf4tzwx2\" (UID: \"485db377-3d7c-46e8-9d8c-547ab86d96c0\") " pod="openshift-ingress/data-science-gateway-data-science-gateway-class-6576859cf4tzwx2" Jul 16 00:46:40.786553 ip-10-0-143-200 kubenswrapper[2572]: I0716 00:46:40.786467 2572 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/485db377-3d7c-46e8-9d8c-547ab86d96c0-istio-token\") pod \"data-science-gateway-data-science-gateway-class-6576859cf4tzwx2\" (UID: \"485db377-3d7c-46e8-9d8c-547ab86d96c0\") " pod="openshift-ingress/data-science-gateway-data-science-gateway-class-6576859cf4tzwx2" Jul 16 00:46:40.786553 ip-10-0-143-200 kubenswrapper[2572]: I0716 00:46:40.786513 2572 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/485db377-3d7c-46e8-9d8c-547ab86d96c0-istio-data\") pod \"data-science-gateway-data-science-gateway-class-6576859cf4tzwx2\" (UID: \"485db377-3d7c-46e8-9d8c-547ab86d96c0\") " pod="openshift-ingress/data-science-gateway-data-science-gateway-class-6576859cf4tzwx2" Jul 16 00:46:40.786553 ip-10-0-143-200 kubenswrapper[2572]: I0716 00:46:40.786545 2572 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/485db377-3d7c-46e8-9d8c-547ab86d96c0-istiod-ca-cert\") pod \"data-science-gateway-data-science-gateway-class-6576859cf4tzwx2\" (UID: \"485db377-3d7c-46e8-9d8c-547ab86d96c0\") " pod="openshift-ingress/data-science-gateway-data-science-gateway-class-6576859cf4tzwx2" Jul 16 00:46:40.786891 ip-10-0-143-200 kubenswrapper[2572]: I0716 00:46:40.786572 2572 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/485db377-3d7c-46e8-9d8c-547ab86d96c0-istio-envoy\") pod \"data-science-gateway-data-science-gateway-class-6576859cf4tzwx2\" (UID: \"485db377-3d7c-46e8-9d8c-547ab86d96c0\") " pod="openshift-ingress/data-science-gateway-data-science-gateway-class-6576859cf4tzwx2" Jul 16 00:46:40.786891 ip-10-0-143-200 kubenswrapper[2572]: I0716 00:46:40.786795 2572 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/485db377-3d7c-46e8-9d8c-547ab86d96c0-workload-socket\") pod \"data-science-gateway-data-science-gateway-class-6576859cf4tzwx2\" (UID: \"485db377-3d7c-46e8-9d8c-547ab86d96c0\") " pod="openshift-ingress/data-science-gateway-data-science-gateway-class-6576859cf4tzwx2" Jul 16 00:46:40.786891 ip-10-0-143-200 kubenswrapper[2572]: I0716 00:46:40.786831 2572 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/485db377-3d7c-46e8-9d8c-547ab86d96c0-workload-certs\") pod \"data-science-gateway-data-science-gateway-class-6576859cf4tzwx2\" (UID: \"485db377-3d7c-46e8-9d8c-547ab86d96c0\") " pod="openshift-ingress/data-science-gateway-data-science-gateway-class-6576859cf4tzwx2" Jul 16 00:46:40.787048 ip-10-0-143-200 kubenswrapper[2572]: I0716 00:46:40.786985 2572 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/485db377-3d7c-46e8-9d8c-547ab86d96c0-credential-socket\") pod \"data-science-gateway-data-science-gateway-class-6576859cf4tzwx2\" (UID: \"485db377-3d7c-46e8-9d8c-547ab86d96c0\") " pod="openshift-ingress/data-science-gateway-data-science-gateway-class-6576859cf4tzwx2" Jul 16 00:46:40.787124 ip-10-0-143-200 kubenswrapper[2572]: I0716 00:46:40.787100 2572 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/485db377-3d7c-46e8-9d8c-547ab86d96c0-istio-data\") pod \"data-science-gateway-data-science-gateway-class-6576859cf4tzwx2\" (UID: \"485db377-3d7c-46e8-9d8c-547ab86d96c0\") " pod="openshift-ingress/data-science-gateway-data-science-gateway-class-6576859cf4tzwx2" Jul 16 00:46:40.787370 ip-10-0-143-200 kubenswrapper[2572]: I0716 00:46:40.787348 2572 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/485db377-3d7c-46e8-9d8c-547ab86d96c0-istiod-ca-cert\") pod \"data-science-gateway-data-science-gateway-class-6576859cf4tzwx2\" (UID: \"485db377-3d7c-46e8-9d8c-547ab86d96c0\") " pod="openshift-ingress/data-science-gateway-data-science-gateway-class-6576859cf4tzwx2" Jul 16 00:46:40.788882 ip-10-0-143-200 kubenswrapper[2572]: I0716 00:46:40.788864 2572 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/485db377-3d7c-46e8-9d8c-547ab86d96c0-istio-envoy\") pod \"data-science-gateway-data-science-gateway-class-6576859cf4tzwx2\" (UID: \"485db377-3d7c-46e8-9d8c-547ab86d96c0\") " pod="openshift-ingress/data-science-gateway-data-science-gateway-class-6576859cf4tzwx2" Jul 16 00:46:40.789035 ip-10-0-143-200 kubenswrapper[2572]: I0716 00:46:40.789019 2572 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/485db377-3d7c-46e8-9d8c-547ab86d96c0-istio-podinfo\") pod \"data-science-gateway-data-science-gateway-class-6576859cf4tzwx2\" (UID: \"485db377-3d7c-46e8-9d8c-547ab86d96c0\") " pod="openshift-ingress/data-science-gateway-data-science-gateway-class-6576859cf4tzwx2" Jul 16 00:46:40.810482 ip-10-0-143-200 kubenswrapper[2572]: I0716 00:46:40.810452 2572 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-kdwf2\" (UniqueName: \"kubernetes.io/projected/485db377-3d7c-46e8-9d8c-547ab86d96c0-kube-api-access-kdwf2\") pod \"data-science-gateway-data-science-gateway-class-6576859cf4tzwx2\" (UID: \"485db377-3d7c-46e8-9d8c-547ab86d96c0\") " pod="openshift-ingress/data-science-gateway-data-science-gateway-class-6576859cf4tzwx2" Jul 16 00:46:40.813991 ip-10-0-143-200 kubenswrapper[2572]: I0716 00:46:40.813967 2572 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/485db377-3d7c-46e8-9d8c-547ab86d96c0-istio-token\") pod \"data-science-gateway-data-science-gateway-class-6576859cf4tzwx2\" (UID: \"485db377-3d7c-46e8-9d8c-547ab86d96c0\") " pod="openshift-ingress/data-science-gateway-data-science-gateway-class-6576859cf4tzwx2" Jul 16 00:46:40.834032 ip-10-0-143-200 kubenswrapper[2572]: I0716 00:46:40.833950 2572 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/data-science-gateway-data-science-gateway-class-6576859cf4tzwx2" Jul 16 00:46:41.001371 ip-10-0-143-200 kubenswrapper[2572]: I0716 00:46:41.001339 2572 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/data-science-gateway-data-science-gateway-class-6576859cf4tzwx2"] Jul 16 00:46:41.004290 ip-10-0-143-200 kubenswrapper[2572]: W0716 00:46:41.004260 2572 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod485db377_3d7c_46e8_9d8c_547ab86d96c0.slice/crio-e834f979cf2763805dd2da5f357e9c0037096e97fc7da9e93bb13fcc86e25a99 WatchSource:0}: Error finding container e834f979cf2763805dd2da5f357e9c0037096e97fc7da9e93bb13fcc86e25a99: Status 404 returned error can't find the container with id e834f979cf2763805dd2da5f357e9c0037096e97fc7da9e93bb13fcc86e25a99 Jul 16 00:46:41.057249 ip-10-0-143-200 kubenswrapper[2572]: I0716 00:46:41.057195 2572 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9gnmq_3a7e030d-fad8-480d-9dc6-fe5057048cf9/konnectivity-agent/0.log" limit=1024 Jul 16 00:46:41.213791 ip-10-0-143-200 kubenswrapper[2572]: I0716 00:46:41.213754 2572 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9gnmq_3a7e030d-fad8-480d-9dc6-fe5057048cf9/konnectivity-agent/0.log" limit=1024 Jul 16 00:46:41.331143 ip-10-0-143-200 kubenswrapper[2572]: I0716 00:46:41.331100 2572 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/data-science-gateway-data-science-gateway-class-6576859cf4tzwx2" event={"ID":"485db377-3d7c-46e8-9d8c-547ab86d96c0","Type":"ContainerStarted","Data":"e834f979cf2763805dd2da5f357e9c0037096e97fc7da9e93bb13fcc86e25a99"} Jul 16 00:46:43.074469 ip-10-0-143-200 kubenswrapper[2572]: I0716 00:46:43.074331 2572 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9gnmq_3a7e030d-fad8-480d-9dc6-fe5057048cf9/konnectivity-agent/0.log" limit=1024 Jul 16 00:46:43.301647 ip-10-0-143-200 kubenswrapper[2572]: I0716 00:46:43.301602 2572 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["opendatahub/llmisvc-controller-manager-777c6bd7d5-b8vdj"] Jul 16 00:46:43.309129 ip-10-0-143-200 kubenswrapper[2572]: I0716 00:46:43.309099 2572 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="opendatahub/llmisvc-controller-manager-777c6bd7d5-b8vdj" Jul 16 00:46:43.320173 ip-10-0-143-200 kubenswrapper[2572]: I0716 00:46:43.320149 2572 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"opendatahub\"/\"llmisvc-webhook-server-cert\"" Jul 16 00:46:43.320309 ip-10-0-143-200 kubenswrapper[2572]: I0716 00:46:43.320202 2572 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"opendatahub\"/\"llmisvc-controller-manager-dockercfg-4q799\"" Jul 16 00:46:43.329220 ip-10-0-143-200 kubenswrapper[2572]: I0716 00:46:43.329143 2572 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9gnmq_3a7e030d-fad8-480d-9dc6-fe5057048cf9/konnectivity-agent/0.log" limit=1024 Jul 16 00:46:43.333297 ip-10-0-143-200 kubenswrapper[2572]: I0716 00:46:43.333269 2572 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["opendatahub/llmisvc-controller-manager-777c6bd7d5-b8vdj"] Jul 16 00:46:43.411671 ip-10-0-143-200 kubenswrapper[2572]: I0716 00:46:43.411603 2572 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-kn8lj\" (UniqueName: \"kubernetes.io/projected/170820d7-8bbf-48db-8dfa-f6181d9ebae7-kube-api-access-kn8lj\") pod \"llmisvc-controller-manager-777c6bd7d5-b8vdj\" (UID: \"170820d7-8bbf-48db-8dfa-f6181d9ebae7\") " pod="opendatahub/llmisvc-controller-manager-777c6bd7d5-b8vdj" Jul 16 00:46:43.411671 ip-10-0-143-200 kubenswrapper[2572]: I0716 00:46:43.411673 2572 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"cert\" (UniqueName: \"kubernetes.io/secret/170820d7-8bbf-48db-8dfa-f6181d9ebae7-cert\") pod \"llmisvc-controller-manager-777c6bd7d5-b8vdj\" (UID: \"170820d7-8bbf-48db-8dfa-f6181d9ebae7\") " pod="opendatahub/llmisvc-controller-manager-777c6bd7d5-b8vdj" Jul 16 00:46:43.513060 ip-10-0-143-200 kubenswrapper[2572]: I0716 00:46:43.513006 2572 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-kn8lj\" (UniqueName: \"kubernetes.io/projected/170820d7-8bbf-48db-8dfa-f6181d9ebae7-kube-api-access-kn8lj\") pod \"llmisvc-controller-manager-777c6bd7d5-b8vdj\" (UID: \"170820d7-8bbf-48db-8dfa-f6181d9ebae7\") " pod="opendatahub/llmisvc-controller-manager-777c6bd7d5-b8vdj" Jul 16 00:46:43.513244 ip-10-0-143-200 kubenswrapper[2572]: I0716 00:46:43.513081 2572 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"cert\" (UniqueName: \"kubernetes.io/secret/170820d7-8bbf-48db-8dfa-f6181d9ebae7-cert\") pod \"llmisvc-controller-manager-777c6bd7d5-b8vdj\" (UID: \"170820d7-8bbf-48db-8dfa-f6181d9ebae7\") " pod="opendatahub/llmisvc-controller-manager-777c6bd7d5-b8vdj" Jul 16 00:46:43.513244 ip-10-0-143-200 kubenswrapper[2572]: E0716 00:46:43.513219 2572 secret.go:189] Couldn't get secret opendatahub/llmisvc-webhook-server-cert: secret "llmisvc-webhook-server-cert" not found Jul 16 00:46:43.513355 ip-10-0-143-200 kubenswrapper[2572]: E0716 00:46:43.513299 2572 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/170820d7-8bbf-48db-8dfa-f6181d9ebae7-cert podName:170820d7-8bbf-48db-8dfa-f6181d9ebae7 nodeName:}" failed. No retries permitted until 2026-07-16 00:46:44.013280455 +0000 UTC m=+409.122035117 (durationBeforeRetry 500ms). Error: MountVolume.SetUp failed for volume "cert" (UniqueName: "kubernetes.io/secret/170820d7-8bbf-48db-8dfa-f6181d9ebae7-cert") pod "llmisvc-controller-manager-777c6bd7d5-b8vdj" (UID: "170820d7-8bbf-48db-8dfa-f6181d9ebae7") : secret "llmisvc-webhook-server-cert" not found Jul 16 00:46:43.537928 ip-10-0-143-200 kubenswrapper[2572]: I0716 00:46:43.537896 2572 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-kn8lj\" (UniqueName: \"kubernetes.io/projected/170820d7-8bbf-48db-8dfa-f6181d9ebae7-kube-api-access-kn8lj\") pod \"llmisvc-controller-manager-777c6bd7d5-b8vdj\" (UID: \"170820d7-8bbf-48db-8dfa-f6181d9ebae7\") " pod="opendatahub/llmisvc-controller-manager-777c6bd7d5-b8vdj" Jul 16 00:46:43.664400 ip-10-0-143-200 kubenswrapper[2572]: I0716 00:46:43.664313 2572 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9gnmq_3a7e030d-fad8-480d-9dc6-fe5057048cf9/konnectivity-agent/0.log" limit=1024 Jul 16 00:46:43.719240 ip-10-0-143-200 kubenswrapper[2572]: I0716 00:46:43.719211 2572 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236228Ki","pods":"250"} Jul 16 00:46:43.719325 ip-10-0-143-200 kubenswrapper[2572]: I0716 00:46:43.719279 2572 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236228Ki","pods":"250"} Jul 16 00:46:43.719325 ip-10-0-143-200 kubenswrapper[2572]: I0716 00:46:43.719306 2572 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236228Ki","pods":"250"} Jul 16 00:46:43.807579 ip-10-0-143-200 kubenswrapper[2572]: I0716 00:46:43.807547 2572 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9gnmq_3a7e030d-fad8-480d-9dc6-fe5057048cf9/konnectivity-agent/0.log" limit=1024 Jul 16 00:46:44.017109 ip-10-0-143-200 kubenswrapper[2572]: I0716 00:46:44.017060 2572 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"cert\" (UniqueName: \"kubernetes.io/secret/170820d7-8bbf-48db-8dfa-f6181d9ebae7-cert\") pod \"llmisvc-controller-manager-777c6bd7d5-b8vdj\" (UID: \"170820d7-8bbf-48db-8dfa-f6181d9ebae7\") " pod="opendatahub/llmisvc-controller-manager-777c6bd7d5-b8vdj" Jul 16 00:46:44.019643 ip-10-0-143-200 kubenswrapper[2572]: I0716 00:46:44.019601 2572 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"cert\" (UniqueName: \"kubernetes.io/secret/170820d7-8bbf-48db-8dfa-f6181d9ebae7-cert\") pod \"llmisvc-controller-manager-777c6bd7d5-b8vdj\" (UID: \"170820d7-8bbf-48db-8dfa-f6181d9ebae7\") " pod="opendatahub/llmisvc-controller-manager-777c6bd7d5-b8vdj" Jul 16 00:46:44.221293 ip-10-0-143-200 kubenswrapper[2572]: I0716 00:46:44.221248 2572 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="opendatahub/llmisvc-controller-manager-777c6bd7d5-b8vdj" Jul 16 00:46:44.335070 ip-10-0-143-200 kubenswrapper[2572]: I0716 00:46:44.335031 2572 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9gnmq_3a7e030d-fad8-480d-9dc6-fe5057048cf9/konnectivity-agent/0.log" limit=1024 Jul 16 00:46:44.366973 ip-10-0-143-200 kubenswrapper[2572]: I0716 00:46:44.366941 2572 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/data-science-gateway-data-science-gateway-class-6576859cf4tzwx2" event={"ID":"485db377-3d7c-46e8-9d8c-547ab86d96c0","Type":"ContainerStarted","Data":"2ce97aba402764fc2424f2c2b945af2d586194ef4952b9ceb9efbb32607e4007"} Jul 16 00:46:44.388695 ip-10-0-143-200 kubenswrapper[2572]: I0716 00:46:44.388662 2572 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["opendatahub/llmisvc-controller-manager-777c6bd7d5-b8vdj"] Jul 16 00:46:44.391469 ip-10-0-143-200 kubenswrapper[2572]: W0716 00:46:44.391441 2572 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-pod170820d7_8bbf_48db_8dfa_f6181d9ebae7.slice/crio-a557f38f6d8e54ac5bde6d7e3232afdb0d3f6879e6ee7916a7940250de4d265d WatchSource:0}: Error finding container a557f38f6d8e54ac5bde6d7e3232afdb0d3f6879e6ee7916a7940250de4d265d: Status 404 returned error can't find the container with id a557f38f6d8e54ac5bde6d7e3232afdb0d3f6879e6ee7916a7940250de4d265d Jul 16 00:46:44.415025 ip-10-0-143-200 kubenswrapper[2572]: I0716 00:46:44.414976 2572 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-ingress/data-science-gateway-data-science-gateway-class-6576859cf4tzwx2" podStartSLOduration=1.702458954 podStartE2EDuration="4.414957436s" totalImagesPullingTime="2.712498482s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-16 00:46:40 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-16 00:46:44.413904397 +0000 UTC m=+409.522659072" watchObservedRunningTime="2026-07-16 00:46:44.414957436 +0000 UTC m=+409.523712174" Jul 16 00:46:44.834968 ip-10-0-143-200 kubenswrapper[2572]: I0716 00:46:44.834929 2572 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="openshift-ingress/data-science-gateway-data-science-gateway-class-6576859cf4tzwx2" Jul 16 00:46:44.839736 ip-10-0-143-200 kubenswrapper[2572]: I0716 00:46:44.839706 2572 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="openshift-ingress/data-science-gateway-data-science-gateway-class-6576859cf4tzwx2" Jul 16 00:46:45.372489 ip-10-0-143-200 kubenswrapper[2572]: I0716 00:46:45.372452 2572 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="opendatahub/llmisvc-controller-manager-777c6bd7d5-b8vdj" event={"ID":"170820d7-8bbf-48db-8dfa-f6181d9ebae7","Type":"ContainerStarted","Data":"a557f38f6d8e54ac5bde6d7e3232afdb0d3f6879e6ee7916a7940250de4d265d"} Jul 16 00:46:45.372979 ip-10-0-143-200 kubenswrapper[2572]: I0716 00:46:45.372803 2572 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/data-science-gateway-data-science-gateway-class-6576859cf4tzwx2" Jul 16 00:46:45.373932 ip-10-0-143-200 kubenswrapper[2572]: I0716 00:46:45.373911 2572 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/data-science-gateway-data-science-gateway-class-6576859cf4tzwx2" Jul 16 00:46:46.307516 ip-10-0-143-200 kubenswrapper[2572]: I0716 00:46:46.307481 2572 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="opendatahub/odh-model-controller-7998c67dc4-xdsc2" Jul 16 00:46:46.309149 ip-10-0-143-200 kubenswrapper[2572]: I0716 00:46:46.309128 2572 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="opendatahub/model-serving-api-54d89bd5fc-xhll8" Jul 16 00:46:46.466971 ip-10-0-143-200 kubenswrapper[2572]: I0716 00:46:46.466936 2572 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-ingress/istiod-openshift-gateway-7574459656-flnhz"] Jul 16 00:46:46.467429 ip-10-0-143-200 kubenswrapper[2572]: I0716 00:46:46.467152 2572 kuberuntime_container.go:873] "Killing container with a grace period" pod="openshift-ingress/istiod-openshift-gateway-7574459656-flnhz" podUID="f16361c6-509c-4261-8777-872298146540" containerName="discovery" containerID="cri-o://bf1949f36aecc7b8c7d7dec845b7f4b73276c6ca8508a8e36309b6419ae8f800" gracePeriod=30 Jul 16 00:46:46.534338 ip-10-0-143-200 kubenswrapper[2572]: E0716 00:46:46.534309 2572 cadvisor_stats_provider.go:567] "Partial failure issuing cadvisor.ContainerInfoV2" err="partial failures: [\"/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podf16361c6_509c_4261_8777_872298146540.slice/crio-bf1949f36aecc7b8c7d7dec845b7f4b73276c6ca8508a8e36309b6419ae8f800.scope\": RecentStats: unable to find data in memory cache]" Jul 16 00:46:46.713808 ip-10-0-143-200 kubenswrapper[2572]: I0716 00:46:46.713592 2572 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9gnmq_3a7e030d-fad8-480d-9dc6-fe5057048cf9/konnectivity-agent/0.log" limit=1024 Jul 16 00:46:46.728874 ip-10-0-143-200 kubenswrapper[2572]: I0716 00:46:46.728850 2572 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/istiod-openshift-gateway-7574459656-flnhz" Jul 16 00:46:46.740796 ip-10-0-143-200 kubenswrapper[2572]: I0716 00:46:46.740770 2572 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istio-csr-ca-configmap\" (UniqueName: \"kubernetes.io/configmap/f16361c6-509c-4261-8777-872298146540-istio-csr-ca-configmap\") pod \"f16361c6-509c-4261-8777-872298146540\" (UID: \"f16361c6-509c-4261-8777-872298146540\") " Jul 16 00:46:46.740927 ip-10-0-143-200 kubenswrapper[2572]: I0716 00:46:46.740831 2572 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"cacerts\" (UniqueName: \"kubernetes.io/secret/f16361c6-509c-4261-8777-872298146540-cacerts\") pod \"f16361c6-509c-4261-8777-872298146540\" (UID: \"f16361c6-509c-4261-8777-872298146540\") " Jul 16 00:46:46.740927 ip-10-0-143-200 kubenswrapper[2572]: I0716 00:46:46.740856 2572 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"local-certs\" (UniqueName: \"kubernetes.io/empty-dir/f16361c6-509c-4261-8777-872298146540-local-certs\") pod \"f16361c6-509c-4261-8777-872298146540\" (UID: \"f16361c6-509c-4261-8777-872298146540\") " Jul 16 00:46:46.741050 ip-10-0-143-200 kubenswrapper[2572]: I0716 00:46:46.740960 2572 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/f16361c6-509c-4261-8777-872298146540-istio-token\") pod \"f16361c6-509c-4261-8777-872298146540\" (UID: \"f16361c6-509c-4261-8777-872298146540\") " Jul 16 00:46:46.741110 ip-10-0-143-200 kubenswrapper[2572]: I0716 00:46:46.741076 2572 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istio-csr-dns-cert\" (UniqueName: \"kubernetes.io/secret/f16361c6-509c-4261-8777-872298146540-istio-csr-dns-cert\") pod \"f16361c6-509c-4261-8777-872298146540\" (UID: \"f16361c6-509c-4261-8777-872298146540\") " Jul 16 00:46:46.741110 ip-10-0-143-200 kubenswrapper[2572]: I0716 00:46:46.741103 2572 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-kmf7g\" (UniqueName: \"kubernetes.io/projected/f16361c6-509c-4261-8777-872298146540-kube-api-access-kmf7g\") pod \"f16361c6-509c-4261-8777-872298146540\" (UID: \"f16361c6-509c-4261-8777-872298146540\") " Jul 16 00:46:46.741206 ip-10-0-143-200 kubenswrapper[2572]: I0716 00:46:46.741131 2572 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istio-kubeconfig\" (UniqueName: \"kubernetes.io/secret/f16361c6-509c-4261-8777-872298146540-istio-kubeconfig\") pod \"f16361c6-509c-4261-8777-872298146540\" (UID: \"f16361c6-509c-4261-8777-872298146540\") " Jul 16 00:46:46.741462 ip-10-0-143-200 kubenswrapper[2572]: I0716 00:46:46.741264 2572 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/f16361c6-509c-4261-8777-872298146540-istio-csr-ca-configmap" (OuterVolumeSpecName: "istio-csr-ca-configmap") pod "f16361c6-509c-4261-8777-872298146540" (UID: "f16361c6-509c-4261-8777-872298146540"). InnerVolumeSpecName "istio-csr-ca-configmap". PluginName "kubernetes.io/configmap", VolumeGIDValue "" Jul 16 00:46:46.741462 ip-10-0-143-200 kubenswrapper[2572]: I0716 00:46:46.741418 2572 reconciler_common.go:299] "Volume detached for volume \"istio-csr-ca-configmap\" (UniqueName: \"kubernetes.io/configmap/f16361c6-509c-4261-8777-872298146540-istio-csr-ca-configmap\") on node \"ip-10-0-143-200.ec2.internal\" DevicePath \"\"" Jul 16 00:46:46.743480 ip-10-0-143-200 kubenswrapper[2572]: I0716 00:46:46.743447 2572 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/f16361c6-509c-4261-8777-872298146540-kube-api-access-kmf7g" (OuterVolumeSpecName: "kube-api-access-kmf7g") pod "f16361c6-509c-4261-8777-872298146540" (UID: "f16361c6-509c-4261-8777-872298146540"). InnerVolumeSpecName "kube-api-access-kmf7g". PluginName "kubernetes.io/projected", VolumeGIDValue "" Jul 16 00:46:46.743673 ip-10-0-143-200 kubenswrapper[2572]: I0716 00:46:46.743618 2572 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/f16361c6-509c-4261-8777-872298146540-local-certs" (OuterVolumeSpecName: "local-certs") pod "f16361c6-509c-4261-8777-872298146540" (UID: "f16361c6-509c-4261-8777-872298146540"). InnerVolumeSpecName "local-certs". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Jul 16 00:46:46.743673 ip-10-0-143-200 kubenswrapper[2572]: I0716 00:46:46.743620 2572 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/f16361c6-509c-4261-8777-872298146540-istio-token" (OuterVolumeSpecName: "istio-token") pod "f16361c6-509c-4261-8777-872298146540" (UID: "f16361c6-509c-4261-8777-872298146540"). InnerVolumeSpecName "istio-token". PluginName "kubernetes.io/projected", VolumeGIDValue "" Jul 16 00:46:46.743673 ip-10-0-143-200 kubenswrapper[2572]: I0716 00:46:46.743655 2572 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/f16361c6-509c-4261-8777-872298146540-cacerts" (OuterVolumeSpecName: "cacerts") pod "f16361c6-509c-4261-8777-872298146540" (UID: "f16361c6-509c-4261-8777-872298146540"). InnerVolumeSpecName "cacerts". PluginName "kubernetes.io/secret", VolumeGIDValue "" Jul 16 00:46:46.743851 ip-10-0-143-200 kubenswrapper[2572]: I0716 00:46:46.743710 2572 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/f16361c6-509c-4261-8777-872298146540-istio-kubeconfig" (OuterVolumeSpecName: "istio-kubeconfig") pod "f16361c6-509c-4261-8777-872298146540" (UID: "f16361c6-509c-4261-8777-872298146540"). InnerVolumeSpecName "istio-kubeconfig". PluginName "kubernetes.io/secret", VolumeGIDValue "" Jul 16 00:46:46.743903 ip-10-0-143-200 kubenswrapper[2572]: I0716 00:46:46.743889 2572 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/f16361c6-509c-4261-8777-872298146540-istio-csr-dns-cert" (OuterVolumeSpecName: "istio-csr-dns-cert") pod "f16361c6-509c-4261-8777-872298146540" (UID: "f16361c6-509c-4261-8777-872298146540"). InnerVolumeSpecName "istio-csr-dns-cert". PluginName "kubernetes.io/secret", VolumeGIDValue "" Jul 16 00:46:46.842640 ip-10-0-143-200 kubenswrapper[2572]: I0716 00:46:46.842513 2572 reconciler_common.go:299] "Volume detached for volume \"cacerts\" (UniqueName: \"kubernetes.io/secret/f16361c6-509c-4261-8777-872298146540-cacerts\") on node \"ip-10-0-143-200.ec2.internal\" DevicePath \"\"" Jul 16 00:46:46.842640 ip-10-0-143-200 kubenswrapper[2572]: I0716 00:46:46.842542 2572 reconciler_common.go:299] "Volume detached for volume \"local-certs\" (UniqueName: \"kubernetes.io/empty-dir/f16361c6-509c-4261-8777-872298146540-local-certs\") on node \"ip-10-0-143-200.ec2.internal\" DevicePath \"\"" Jul 16 00:46:46.842640 ip-10-0-143-200 kubenswrapper[2572]: I0716 00:46:46.842558 2572 reconciler_common.go:299] "Volume detached for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/f16361c6-509c-4261-8777-872298146540-istio-token\") on node \"ip-10-0-143-200.ec2.internal\" DevicePath \"\"" Jul 16 00:46:46.842640 ip-10-0-143-200 kubenswrapper[2572]: I0716 00:46:46.842571 2572 reconciler_common.go:299] "Volume detached for volume \"istio-csr-dns-cert\" (UniqueName: \"kubernetes.io/secret/f16361c6-509c-4261-8777-872298146540-istio-csr-dns-cert\") on node \"ip-10-0-143-200.ec2.internal\" DevicePath \"\"" Jul 16 00:46:46.842640 ip-10-0-143-200 kubenswrapper[2572]: I0716 00:46:46.842584 2572 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-kmf7g\" (UniqueName: \"kubernetes.io/projected/f16361c6-509c-4261-8777-872298146540-kube-api-access-kmf7g\") on node \"ip-10-0-143-200.ec2.internal\" DevicePath \"\"" Jul 16 00:46:46.842640 ip-10-0-143-200 kubenswrapper[2572]: I0716 00:46:46.842594 2572 reconciler_common.go:299] "Volume detached for volume \"istio-kubeconfig\" (UniqueName: \"kubernetes.io/secret/f16361c6-509c-4261-8777-872298146540-istio-kubeconfig\") on node \"ip-10-0-143-200.ec2.internal\" DevicePath \"\"" Jul 16 00:46:46.955561 ip-10-0-143-200 kubenswrapper[2572]: I0716 00:46:46.955521 2572 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9gnmq_3a7e030d-fad8-480d-9dc6-fe5057048cf9/konnectivity-agent/0.log" limit=1024 Jul 16 00:46:47.383589 ip-10-0-143-200 kubenswrapper[2572]: I0716 00:46:47.383556 2572 generic.go:356] "Generic (PLEG): container finished" podID="f16361c6-509c-4261-8777-872298146540" containerID="bf1949f36aecc7b8c7d7dec845b7f4b73276c6ca8508a8e36309b6419ae8f800" exitCode=0 Jul 16 00:46:47.383770 ip-10-0-143-200 kubenswrapper[2572]: I0716 00:46:47.383605 2572 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/istiod-openshift-gateway-7574459656-flnhz" Jul 16 00:46:47.383770 ip-10-0-143-200 kubenswrapper[2572]: I0716 00:46:47.383653 2572 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/istiod-openshift-gateway-7574459656-flnhz" event={"ID":"f16361c6-509c-4261-8777-872298146540","Type":"ContainerDied","Data":"bf1949f36aecc7b8c7d7dec845b7f4b73276c6ca8508a8e36309b6419ae8f800"} Jul 16 00:46:47.383770 ip-10-0-143-200 kubenswrapper[2572]: I0716 00:46:47.383705 2572 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/istiod-openshift-gateway-7574459656-flnhz" event={"ID":"f16361c6-509c-4261-8777-872298146540","Type":"ContainerDied","Data":"63eacae4f6e5b00582b51fedb5f1948c6b1209fe6eefa7885241914fd86501c6"} Jul 16 00:46:47.383770 ip-10-0-143-200 kubenswrapper[2572]: I0716 00:46:47.383735 2572 scope.go:117] "RemoveContainer" containerID="bf1949f36aecc7b8c7d7dec845b7f4b73276c6ca8508a8e36309b6419ae8f800" Jul 16 00:46:47.409160 ip-10-0-143-200 kubenswrapper[2572]: I0716 00:46:47.409136 2572 scope.go:117] "RemoveContainer" containerID="bf1949f36aecc7b8c7d7dec845b7f4b73276c6ca8508a8e36309b6419ae8f800" Jul 16 00:46:47.409560 ip-10-0-143-200 kubenswrapper[2572]: E0716 00:46:47.409531 2572 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"bf1949f36aecc7b8c7d7dec845b7f4b73276c6ca8508a8e36309b6419ae8f800\": container with ID starting with bf1949f36aecc7b8c7d7dec845b7f4b73276c6ca8508a8e36309b6419ae8f800 not found: ID does not exist" containerID="bf1949f36aecc7b8c7d7dec845b7f4b73276c6ca8508a8e36309b6419ae8f800" Jul 16 00:46:47.409602 ip-10-0-143-200 kubenswrapper[2572]: I0716 00:46:47.409575 2572 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"bf1949f36aecc7b8c7d7dec845b7f4b73276c6ca8508a8e36309b6419ae8f800"} err="failed to get container status \"bf1949f36aecc7b8c7d7dec845b7f4b73276c6ca8508a8e36309b6419ae8f800\": rpc error: code = NotFound desc = could not find container \"bf1949f36aecc7b8c7d7dec845b7f4b73276c6ca8508a8e36309b6419ae8f800\": container with ID starting with bf1949f36aecc7b8c7d7dec845b7f4b73276c6ca8508a8e36309b6419ae8f800 not found: ID does not exist" Jul 16 00:46:47.413835 ip-10-0-143-200 kubenswrapper[2572]: I0716 00:46:47.413797 2572 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-ingress/istiod-openshift-gateway-7574459656-flnhz"] Jul 16 00:46:47.418813 ip-10-0-143-200 kubenswrapper[2572]: I0716 00:46:47.418791 2572 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["openshift-ingress/istiod-openshift-gateway-7574459656-flnhz"] Jul 16 00:46:47.448013 ip-10-0-143-200 kubenswrapper[2572]: I0716 00:46:47.447985 2572 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="f16361c6-509c-4261-8777-872298146540" path="/var/lib/kubelet/pods/f16361c6-509c-4261-8777-872298146540/volumes" Jul 16 00:46:48.389690 ip-10-0-143-200 kubenswrapper[2572]: I0716 00:46:48.389654 2572 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="opendatahub/llmisvc-controller-manager-777c6bd7d5-b8vdj" event={"ID":"170820d7-8bbf-48db-8dfa-f6181d9ebae7","Type":"ContainerStarted","Data":"eab653074180d57f21c4a65e6ab85452cc116cb713fec6160d800dc53970224a"} Jul 16 00:46:48.390178 ip-10-0-143-200 kubenswrapper[2572]: I0716 00:46:48.389723 2572 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="opendatahub/llmisvc-controller-manager-777c6bd7d5-b8vdj" Jul 16 00:46:48.422009 ip-10-0-143-200 kubenswrapper[2572]: I0716 00:46:48.421957 2572 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="opendatahub/llmisvc-controller-manager-777c6bd7d5-b8vdj" podStartSLOduration=2.399239868 podStartE2EDuration="5.421939861s" totalImagesPullingTime="3.022699993s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-16 00:46:43 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-16 00:46:48.420974403 +0000 UTC m=+413.529729108" watchObservedRunningTime="2026-07-16 00:46:48.421939861 +0000 UTC m=+413.530694541" Jul 16 00:46:54.937380 ip-10-0-143-200 kubenswrapper[2572]: I0716 00:46:54.937343 2572 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9gnmq_3a7e030d-fad8-480d-9dc6-fe5057048cf9/konnectivity-agent/0.log" limit=1024 Jul 16 00:46:55.049415 ip-10-0-143-200 kubenswrapper[2572]: I0716 00:46:55.049354 2572 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9gnmq_3a7e030d-fad8-480d-9dc6-fe5057048cf9/konnectivity-agent/0.log" limit=1024 Jul 16 00:47:04.545059 ip-10-0-143-200 kubenswrapper[2572]: I0716 00:47:04.545024 2572 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9gnmq_3a7e030d-fad8-480d-9dc6-fe5057048cf9/konnectivity-agent/0.log" limit=1024 Jul 16 00:47:04.644110 ip-10-0-143-200 kubenswrapper[2572]: I0716 00:47:04.644071 2572 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9gnmq_3a7e030d-fad8-480d-9dc6-fe5057048cf9/konnectivity-agent/0.log" limit=1024 Jul 16 00:47:19.402459 ip-10-0-143-200 kubenswrapper[2572]: I0716 00:47:19.402381 2572 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="opendatahub/llmisvc-controller-manager-777c6bd7d5-b8vdj" Jul 16 00:47:30.612022 ip-10-0-143-200 kubenswrapper[2572]: I0716 00:47:30.611981 2572 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9gnmq_3a7e030d-fad8-480d-9dc6-fe5057048cf9/konnectivity-agent/0.log" limit=1024 Jul 16 00:47:30.711849 ip-10-0-143-200 kubenswrapper[2572]: I0716 00:47:30.711809 2572 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9gnmq_3a7e030d-fad8-480d-9dc6-fe5057048cf9/konnectivity-agent/0.log" limit=1024 Jul 16 00:47:30.743202 ip-10-0-143-200 kubenswrapper[2572]: I0716 00:47:30.743166 2572 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["kuadrant-system/kuadrant-operator-catalog-ldbbn"] Jul 16 00:47:30.743524 ip-10-0-143-200 kubenswrapper[2572]: I0716 00:47:30.743512 2572 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="f16361c6-509c-4261-8777-872298146540" containerName="discovery" Jul 16 00:47:30.743575 ip-10-0-143-200 kubenswrapper[2572]: I0716 00:47:30.743527 2572 state_mem.go:107] "Deleted CPUSet assignment" podUID="f16361c6-509c-4261-8777-872298146540" containerName="discovery" Jul 16 00:47:30.743611 ip-10-0-143-200 kubenswrapper[2572]: I0716 00:47:30.743597 2572 memory_manager.go:368] "RemoveStaleState removing state" podUID="f16361c6-509c-4261-8777-872298146540" containerName="discovery" Jul 16 00:47:30.745466 ip-10-0-143-200 kubenswrapper[2572]: I0716 00:47:30.745451 2572 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/kuadrant-operator-catalog-ldbbn" Jul 16 00:47:30.749502 ip-10-0-143-200 kubenswrapper[2572]: I0716 00:47:30.749466 2572 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"kuadrant-system\"/\"openshift-service-ca.crt\"" Jul 16 00:47:30.750736 ip-10-0-143-200 kubenswrapper[2572]: I0716 00:47:30.750715 2572 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"kuadrant-system\"/\"kube-root-ca.crt\"" Jul 16 00:47:30.750863 ip-10-0-143-200 kubenswrapper[2572]: I0716 00:47:30.750717 2572 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"kuadrant-system\"/\"kuadrant-operator-catalog-dockercfg-tjvbm\"" Jul 16 00:47:30.756694 ip-10-0-143-200 kubenswrapper[2572]: I0716 00:47:30.756667 2572 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/kuadrant-operator-catalog-ldbbn"] Jul 16 00:47:30.809462 ip-10-0-143-200 kubenswrapper[2572]: I0716 00:47:30.809421 2572 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9gnmq_3a7e030d-fad8-480d-9dc6-fe5057048cf9/konnectivity-agent/0.log" limit=1024 Jul 16 00:47:30.928356 ip-10-0-143-200 kubenswrapper[2572]: I0716 00:47:30.928240 2572 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-z5jr2\" (UniqueName: \"kubernetes.io/projected/03d6e4e7-b930-4813-800c-ce5a7be1b857-kube-api-access-z5jr2\") pod \"kuadrant-operator-catalog-ldbbn\" (UID: \"03d6e4e7-b930-4813-800c-ce5a7be1b857\") " pod="kuadrant-system/kuadrant-operator-catalog-ldbbn" Jul 16 00:47:31.029316 ip-10-0-143-200 kubenswrapper[2572]: I0716 00:47:31.029275 2572 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-z5jr2\" (UniqueName: \"kubernetes.io/projected/03d6e4e7-b930-4813-800c-ce5a7be1b857-kube-api-access-z5jr2\") pod \"kuadrant-operator-catalog-ldbbn\" (UID: \"03d6e4e7-b930-4813-800c-ce5a7be1b857\") " pod="kuadrant-system/kuadrant-operator-catalog-ldbbn" Jul 16 00:47:31.039769 ip-10-0-143-200 kubenswrapper[2572]: I0716 00:47:31.039737 2572 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-z5jr2\" (UniqueName: \"kubernetes.io/projected/03d6e4e7-b930-4813-800c-ce5a7be1b857-kube-api-access-z5jr2\") pod \"kuadrant-operator-catalog-ldbbn\" (UID: \"03d6e4e7-b930-4813-800c-ce5a7be1b857\") " pod="kuadrant-system/kuadrant-operator-catalog-ldbbn" Jul 16 00:47:31.057623 ip-10-0-143-200 kubenswrapper[2572]: I0716 00:47:31.057586 2572 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/kuadrant-operator-catalog-ldbbn" Jul 16 00:47:31.095044 ip-10-0-143-200 kubenswrapper[2572]: I0716 00:47:31.095010 2572 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["kuadrant-system/kuadrant-operator-catalog-ldbbn"] Jul 16 00:47:31.189988 ip-10-0-143-200 kubenswrapper[2572]: I0716 00:47:31.189966 2572 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["kuadrant-system/kuadrant-operator-catalog-ldbbn"] Jul 16 00:47:31.192235 ip-10-0-143-200 kubenswrapper[2572]: W0716 00:47:31.192206 2572 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod03d6e4e7_b930_4813_800c_ce5a7be1b857.slice/crio-42aeee0df7ecd2df4fb6e019c3cbd6a47dc33c9cee09d9f89214a87834916b6b WatchSource:0}: Error finding container 42aeee0df7ecd2df4fb6e019c3cbd6a47dc33c9cee09d9f89214a87834916b6b: Status 404 returned error can't find the container with id 42aeee0df7ecd2df4fb6e019c3cbd6a47dc33c9cee09d9f89214a87834916b6b Jul 16 00:47:31.307910 ip-10-0-143-200 kubenswrapper[2572]: I0716 00:47:31.307874 2572 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["kuadrant-system/kuadrant-operator-catalog-vsp77"] Jul 16 00:47:31.311107 ip-10-0-143-200 kubenswrapper[2572]: I0716 00:47:31.311087 2572 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/kuadrant-operator-catalog-vsp77" Jul 16 00:47:31.325203 ip-10-0-143-200 kubenswrapper[2572]: I0716 00:47:31.325175 2572 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/kuadrant-operator-catalog-vsp77"] Jul 16 00:47:31.432303 ip-10-0-143-200 kubenswrapper[2572]: I0716 00:47:31.432263 2572 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-xt9mv\" (UniqueName: \"kubernetes.io/projected/a31ac65b-1f88-41aa-b1d1-a141190eaae4-kube-api-access-xt9mv\") pod \"kuadrant-operator-catalog-vsp77\" (UID: \"a31ac65b-1f88-41aa-b1d1-a141190eaae4\") " pod="kuadrant-system/kuadrant-operator-catalog-vsp77" Jul 16 00:47:31.529731 ip-10-0-143-200 kubenswrapper[2572]: I0716 00:47:31.529703 2572 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9gnmq_3a7e030d-fad8-480d-9dc6-fe5057048cf9/konnectivity-agent/0.log" limit=1024 Jul 16 00:47:31.532972 ip-10-0-143-200 kubenswrapper[2572]: I0716 00:47:31.532946 2572 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-xt9mv\" (UniqueName: \"kubernetes.io/projected/a31ac65b-1f88-41aa-b1d1-a141190eaae4-kube-api-access-xt9mv\") pod \"kuadrant-operator-catalog-vsp77\" (UID: \"a31ac65b-1f88-41aa-b1d1-a141190eaae4\") " pod="kuadrant-system/kuadrant-operator-catalog-vsp77" Jul 16 00:47:31.542423 ip-10-0-143-200 kubenswrapper[2572]: I0716 00:47:31.542402 2572 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-xt9mv\" (UniqueName: \"kubernetes.io/projected/a31ac65b-1f88-41aa-b1d1-a141190eaae4-kube-api-access-xt9mv\") pod \"kuadrant-operator-catalog-vsp77\" (UID: \"a31ac65b-1f88-41aa-b1d1-a141190eaae4\") " pod="kuadrant-system/kuadrant-operator-catalog-vsp77" Jul 16 00:47:31.581201 ip-10-0-143-200 kubenswrapper[2572]: I0716 00:47:31.581165 2572 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/kuadrant-operator-catalog-ldbbn" event={"ID":"03d6e4e7-b930-4813-800c-ce5a7be1b857","Type":"ContainerStarted","Data":"42aeee0df7ecd2df4fb6e019c3cbd6a47dc33c9cee09d9f89214a87834916b6b"} Jul 16 00:47:31.620136 ip-10-0-143-200 kubenswrapper[2572]: I0716 00:47:31.620101 2572 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/kuadrant-operator-catalog-vsp77" Jul 16 00:47:31.738388 ip-10-0-143-200 kubenswrapper[2572]: I0716 00:47:31.738356 2572 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9gnmq_3a7e030d-fad8-480d-9dc6-fe5057048cf9/konnectivity-agent/0.log" limit=1024 Jul 16 00:47:31.765541 ip-10-0-143-200 kubenswrapper[2572]: I0716 00:47:31.765516 2572 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/kuadrant-operator-catalog-vsp77"] Jul 16 00:47:31.782817 ip-10-0-143-200 kubenswrapper[2572]: W0716 00:47:31.782751 2572 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-poda31ac65b_1f88_41aa_b1d1_a141190eaae4.slice/crio-b5541b1600f5d89951c428babbf6691499cff153a2a2be4af607365ab5c8f35b WatchSource:0}: Error finding container b5541b1600f5d89951c428babbf6691499cff153a2a2be4af607365ab5c8f35b: Status 404 returned error can't find the container with id b5541b1600f5d89951c428babbf6691499cff153a2a2be4af607365ab5c8f35b Jul 16 00:47:32.587759 ip-10-0-143-200 kubenswrapper[2572]: I0716 00:47:32.587721 2572 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/kuadrant-operator-catalog-vsp77" event={"ID":"a31ac65b-1f88-41aa-b1d1-a141190eaae4","Type":"ContainerStarted","Data":"b5541b1600f5d89951c428babbf6691499cff153a2a2be4af607365ab5c8f35b"} Jul 16 00:47:33.593359 ip-10-0-143-200 kubenswrapper[2572]: I0716 00:47:33.593256 2572 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/kuadrant-operator-catalog-ldbbn" event={"ID":"03d6e4e7-b930-4813-800c-ce5a7be1b857","Type":"ContainerStarted","Data":"e1a87f589ab38dc5093fef3e66e9f24aa02da19f84c01f85c85ba362b6b34020"} Jul 16 00:47:33.593359 ip-10-0-143-200 kubenswrapper[2572]: I0716 00:47:33.593300 2572 kuberuntime_container.go:873] "Killing container with a grace period" pod="kuadrant-system/kuadrant-operator-catalog-ldbbn" podUID="03d6e4e7-b930-4813-800c-ce5a7be1b857" containerName="registry-server" containerID="cri-o://e1a87f589ab38dc5093fef3e66e9f24aa02da19f84c01f85c85ba362b6b34020" gracePeriod=2 Jul 16 00:47:33.594890 ip-10-0-143-200 kubenswrapper[2572]: I0716 00:47:33.594863 2572 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/kuadrant-operator-catalog-vsp77" event={"ID":"a31ac65b-1f88-41aa-b1d1-a141190eaae4","Type":"ContainerStarted","Data":"f3d433d9eae884f3ea3ed01b491eab29beba5abf1acdb5f3b485618749d39938"} Jul 16 00:47:33.610586 ip-10-0-143-200 kubenswrapper[2572]: I0716 00:47:33.610544 2572 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="kuadrant-system/kuadrant-operator-catalog-ldbbn" podStartSLOduration=1.5487691 podStartE2EDuration="3.610528489s" totalImagesPullingTime="2.061759389s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-16 00:47:30 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-16 00:47:33.608431728 +0000 UTC m=+458.717186408" watchObservedRunningTime="2026-07-16 00:47:33.610528489 +0000 UTC m=+458.719283164" Jul 16 00:47:33.623712 ip-10-0-143-200 kubenswrapper[2572]: I0716 00:47:33.623653 2572 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="kuadrant-system/kuadrant-operator-catalog-vsp77" podStartSLOduration=1.155925289 podStartE2EDuration="2.623611673s" totalImagesPullingTime="1.467686384s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-16 00:47:31 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-16 00:47:33.622990478 +0000 UTC m=+458.731745152" watchObservedRunningTime="2026-07-16 00:47:33.623611673 +0000 UTC m=+458.732366348" Jul 16 00:47:33.838639 ip-10-0-143-200 kubenswrapper[2572]: I0716 00:47:33.838601 2572 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/kuadrant-operator-catalog-ldbbn" Jul 16 00:47:33.849797 ip-10-0-143-200 kubenswrapper[2572]: I0716 00:47:33.849721 2572 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-z5jr2\" (UniqueName: \"kubernetes.io/projected/03d6e4e7-b930-4813-800c-ce5a7be1b857-kube-api-access-z5jr2\") pod \"03d6e4e7-b930-4813-800c-ce5a7be1b857\" (UID: \"03d6e4e7-b930-4813-800c-ce5a7be1b857\") " Jul 16 00:47:33.851759 ip-10-0-143-200 kubenswrapper[2572]: I0716 00:47:33.851736 2572 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/03d6e4e7-b930-4813-800c-ce5a7be1b857-kube-api-access-z5jr2" (OuterVolumeSpecName: "kube-api-access-z5jr2") pod "03d6e4e7-b930-4813-800c-ce5a7be1b857" (UID: "03d6e4e7-b930-4813-800c-ce5a7be1b857"). InnerVolumeSpecName "kube-api-access-z5jr2". PluginName "kubernetes.io/projected", VolumeGIDValue "" Jul 16 00:47:33.950882 ip-10-0-143-200 kubenswrapper[2572]: I0716 00:47:33.950838 2572 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-z5jr2\" (UniqueName: \"kubernetes.io/projected/03d6e4e7-b930-4813-800c-ce5a7be1b857-kube-api-access-z5jr2\") on node \"ip-10-0-143-200.ec2.internal\" DevicePath \"\"" Jul 16 00:47:34.600891 ip-10-0-143-200 kubenswrapper[2572]: I0716 00:47:34.600850 2572 generic.go:356] "Generic (PLEG): container finished" podID="03d6e4e7-b930-4813-800c-ce5a7be1b857" containerID="e1a87f589ab38dc5093fef3e66e9f24aa02da19f84c01f85c85ba362b6b34020" exitCode=0 Jul 16 00:47:34.601326 ip-10-0-143-200 kubenswrapper[2572]: I0716 00:47:34.600914 2572 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/kuadrant-operator-catalog-ldbbn" Jul 16 00:47:34.601326 ip-10-0-143-200 kubenswrapper[2572]: I0716 00:47:34.600938 2572 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/kuadrant-operator-catalog-ldbbn" event={"ID":"03d6e4e7-b930-4813-800c-ce5a7be1b857","Type":"ContainerDied","Data":"e1a87f589ab38dc5093fef3e66e9f24aa02da19f84c01f85c85ba362b6b34020"} Jul 16 00:47:34.601326 ip-10-0-143-200 kubenswrapper[2572]: I0716 00:47:34.600995 2572 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/kuadrant-operator-catalog-ldbbn" event={"ID":"03d6e4e7-b930-4813-800c-ce5a7be1b857","Type":"ContainerDied","Data":"42aeee0df7ecd2df4fb6e019c3cbd6a47dc33c9cee09d9f89214a87834916b6b"} Jul 16 00:47:34.601326 ip-10-0-143-200 kubenswrapper[2572]: I0716 00:47:34.601018 2572 scope.go:117] "RemoveContainer" containerID="e1a87f589ab38dc5093fef3e66e9f24aa02da19f84c01f85c85ba362b6b34020" Jul 16 00:47:34.609986 ip-10-0-143-200 kubenswrapper[2572]: I0716 00:47:34.609968 2572 scope.go:117] "RemoveContainer" containerID="e1a87f589ab38dc5093fef3e66e9f24aa02da19f84c01f85c85ba362b6b34020" Jul 16 00:47:34.610242 ip-10-0-143-200 kubenswrapper[2572]: E0716 00:47:34.610221 2572 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"e1a87f589ab38dc5093fef3e66e9f24aa02da19f84c01f85c85ba362b6b34020\": container with ID starting with e1a87f589ab38dc5093fef3e66e9f24aa02da19f84c01f85c85ba362b6b34020 not found: ID does not exist" containerID="e1a87f589ab38dc5093fef3e66e9f24aa02da19f84c01f85c85ba362b6b34020" Jul 16 00:47:34.610332 ip-10-0-143-200 kubenswrapper[2572]: I0716 00:47:34.610247 2572 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"e1a87f589ab38dc5093fef3e66e9f24aa02da19f84c01f85c85ba362b6b34020"} err="failed to get container status \"e1a87f589ab38dc5093fef3e66e9f24aa02da19f84c01f85c85ba362b6b34020\": rpc error: code = NotFound desc = could not find container \"e1a87f589ab38dc5093fef3e66e9f24aa02da19f84c01f85c85ba362b6b34020\": container with ID starting with e1a87f589ab38dc5093fef3e66e9f24aa02da19f84c01f85c85ba362b6b34020 not found: ID does not exist" Jul 16 00:47:34.625343 ip-10-0-143-200 kubenswrapper[2572]: I0716 00:47:34.625320 2572 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["kuadrant-system/kuadrant-operator-catalog-ldbbn"] Jul 16 00:47:34.627311 ip-10-0-143-200 kubenswrapper[2572]: I0716 00:47:34.627290 2572 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["kuadrant-system/kuadrant-operator-catalog-ldbbn"] Jul 16 00:47:35.447966 ip-10-0-143-200 kubenswrapper[2572]: I0716 00:47:35.447932 2572 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="03d6e4e7-b930-4813-800c-ce5a7be1b857" path="/var/lib/kubelet/pods/03d6e4e7-b930-4813-800c-ce5a7be1b857/volumes" Jul 16 00:47:36.950814 ip-10-0-143-200 kubenswrapper[2572]: I0716 00:47:36.950780 2572 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9gnmq_3a7e030d-fad8-480d-9dc6-fe5057048cf9/konnectivity-agent/0.log" limit=1024 Jul 16 00:47:41.621206 ip-10-0-143-200 kubenswrapper[2572]: I0716 00:47:41.621155 2572 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="kuadrant-system/kuadrant-operator-catalog-vsp77" Jul 16 00:47:41.621681 ip-10-0-143-200 kubenswrapper[2572]: I0716 00:47:41.621226 2572 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="kuadrant-system/kuadrant-operator-catalog-vsp77" Jul 16 00:47:41.644431 ip-10-0-143-200 kubenswrapper[2572]: I0716 00:47:41.644402 2572 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="kuadrant-system/kuadrant-operator-catalog-vsp77" Jul 16 00:47:41.667682 ip-10-0-143-200 kubenswrapper[2572]: I0716 00:47:41.667651 2572 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="kuadrant-system/kuadrant-operator-catalog-vsp77" Jul 16 00:47:56.862453 ip-10-0-143-200 kubenswrapper[2572]: I0716 00:47:56.862405 2572 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9gnmq_3a7e030d-fad8-480d-9dc6-fe5057048cf9/konnectivity-agent/0.log" limit=1024 Jul 16 00:47:57.300253 ip-10-0-143-200 kubenswrapper[2572]: I0716 00:47:57.300216 2572 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9gnmq_3a7e030d-fad8-480d-9dc6-fe5057048cf9/konnectivity-agent/0.log" limit=1024 Jul 16 00:47:57.403711 ip-10-0-143-200 kubenswrapper[2572]: I0716 00:47:57.403667 2572 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9gnmq_3a7e030d-fad8-480d-9dc6-fe5057048cf9/konnectivity-agent/0.log" limit=1024 Jul 16 00:47:59.557760 ip-10-0-143-200 kubenswrapper[2572]: I0716 00:47:59.557727 2572 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["kuadrant-system/dns-operator-controller-manager-864dd78c4c-clxrd"] Jul 16 00:47:59.558149 ip-10-0-143-200 kubenswrapper[2572]: I0716 00:47:59.558062 2572 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="03d6e4e7-b930-4813-800c-ce5a7be1b857" containerName="registry-server" Jul 16 00:47:59.558149 ip-10-0-143-200 kubenswrapper[2572]: I0716 00:47:59.558073 2572 state_mem.go:107] "Deleted CPUSet assignment" podUID="03d6e4e7-b930-4813-800c-ce5a7be1b857" containerName="registry-server" Jul 16 00:47:59.558149 ip-10-0-143-200 kubenswrapper[2572]: I0716 00:47:59.558148 2572 memory_manager.go:368] "RemoveStaleState removing state" podUID="03d6e4e7-b930-4813-800c-ce5a7be1b857" containerName="registry-server" Jul 16 00:47:59.560676 ip-10-0-143-200 kubenswrapper[2572]: I0716 00:47:59.560654 2572 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/dns-operator-controller-manager-864dd78c4c-clxrd" Jul 16 00:47:59.567609 ip-10-0-143-200 kubenswrapper[2572]: I0716 00:47:59.567589 2572 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"kuadrant-system\"/\"dns-operator-controller-manager-dockercfg-g5lsf\"" Jul 16 00:47:59.567910 ip-10-0-143-200 kubenswrapper[2572]: I0716 00:47:59.567896 2572 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"kuadrant-system\"/\"dns-operator-controller-env\"" Jul 16 00:47:59.581079 ip-10-0-143-200 kubenswrapper[2572]: I0716 00:47:59.581051 2572 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/dns-operator-controller-manager-864dd78c4c-clxrd"] Jul 16 00:47:59.667078 ip-10-0-143-200 kubenswrapper[2572]: I0716 00:47:59.667041 2572 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-mcq46\" (UniqueName: \"kubernetes.io/projected/3ca593f9-fb02-418b-b4d4-934ac40611bd-kube-api-access-mcq46\") pod \"dns-operator-controller-manager-864dd78c4c-clxrd\" (UID: \"3ca593f9-fb02-418b-b4d4-934ac40611bd\") " pod="kuadrant-system/dns-operator-controller-manager-864dd78c4c-clxrd" Jul 16 00:47:59.767955 ip-10-0-143-200 kubenswrapper[2572]: I0716 00:47:59.767914 2572 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-mcq46\" (UniqueName: \"kubernetes.io/projected/3ca593f9-fb02-418b-b4d4-934ac40611bd-kube-api-access-mcq46\") pod \"dns-operator-controller-manager-864dd78c4c-clxrd\" (UID: \"3ca593f9-fb02-418b-b4d4-934ac40611bd\") " pod="kuadrant-system/dns-operator-controller-manager-864dd78c4c-clxrd" Jul 16 00:47:59.786043 ip-10-0-143-200 kubenswrapper[2572]: I0716 00:47:59.786015 2572 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-mcq46\" (UniqueName: \"kubernetes.io/projected/3ca593f9-fb02-418b-b4d4-934ac40611bd-kube-api-access-mcq46\") pod \"dns-operator-controller-manager-864dd78c4c-clxrd\" (UID: \"3ca593f9-fb02-418b-b4d4-934ac40611bd\") " pod="kuadrant-system/dns-operator-controller-manager-864dd78c4c-clxrd" Jul 16 00:47:59.870261 ip-10-0-143-200 kubenswrapper[2572]: I0716 00:47:59.870158 2572 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/dns-operator-controller-manager-864dd78c4c-clxrd" Jul 16 00:48:00.017345 ip-10-0-143-200 kubenswrapper[2572]: I0716 00:48:00.015861 2572 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/dns-operator-controller-manager-864dd78c4c-clxrd"] Jul 16 00:48:00.719003 ip-10-0-143-200 kubenswrapper[2572]: I0716 00:48:00.718964 2572 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/dns-operator-controller-manager-864dd78c4c-clxrd" event={"ID":"3ca593f9-fb02-418b-b4d4-934ac40611bd","Type":"ContainerStarted","Data":"6ea94bd8998a6ae0fe6c5ed14dae656f11e804b3097a15241489e26f742c667f"} Jul 16 00:48:01.090212 ip-10-0-143-200 kubenswrapper[2572]: I0716 00:48:01.090116 2572 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9gnmq_3a7e030d-fad8-480d-9dc6-fe5057048cf9/konnectivity-agent/0.log" limit=1024 Jul 16 00:48:02.731229 ip-10-0-143-200 kubenswrapper[2572]: I0716 00:48:02.731186 2572 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/dns-operator-controller-manager-864dd78c4c-clxrd" event={"ID":"3ca593f9-fb02-418b-b4d4-934ac40611bd","Type":"ContainerStarted","Data":"d11537d72772eaf73b96aa8b4188d68af0c773108dd710bad7518b0f0538af61"} Jul 16 00:48:02.731670 ip-10-0-143-200 kubenswrapper[2572]: I0716 00:48:02.731335 2572 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="kuadrant-system/dns-operator-controller-manager-864dd78c4c-clxrd" Jul 16 00:48:02.750161 ip-10-0-143-200 kubenswrapper[2572]: I0716 00:48:02.750092 2572 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="kuadrant-system/dns-operator-controller-manager-864dd78c4c-clxrd" podStartSLOduration=1.5237361310000002 podStartE2EDuration="3.75007466s" totalImagesPullingTime="2.226338529s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-16 00:47:59 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-16 00:48:02.748979435 +0000 UTC m=+487.857734111" watchObservedRunningTime="2026-07-16 00:48:02.75007466 +0000 UTC m=+487.858829335" Jul 16 00:48:02.912527 ip-10-0-143-200 kubenswrapper[2572]: I0716 00:48:02.912488 2572 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["kuadrant-system/limitador-operator-controller-manager-859d7b78bd-bx6hr"] Jul 16 00:48:02.915134 ip-10-0-143-200 kubenswrapper[2572]: I0716 00:48:02.915112 2572 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/limitador-operator-controller-manager-859d7b78bd-bx6hr" Jul 16 00:48:02.918089 ip-10-0-143-200 kubenswrapper[2572]: I0716 00:48:02.918064 2572 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"kuadrant-system\"/\"limitador-operator-controller-manager-dockercfg-vrjvw\"" Jul 16 00:48:02.926569 ip-10-0-143-200 kubenswrapper[2572]: I0716 00:48:02.926537 2572 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-operator-controller-manager-859d7b78bd-bx6hr"] Jul 16 00:48:02.998937 ip-10-0-143-200 kubenswrapper[2572]: I0716 00:48:02.998833 2572 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-jjc6v\" (UniqueName: \"kubernetes.io/projected/e2858360-c03d-4d0f-aa28-f9b47ecc34cf-kube-api-access-jjc6v\") pod \"limitador-operator-controller-manager-859d7b78bd-bx6hr\" (UID: \"e2858360-c03d-4d0f-aa28-f9b47ecc34cf\") " pod="kuadrant-system/limitador-operator-controller-manager-859d7b78bd-bx6hr" Jul 16 00:48:03.100187 ip-10-0-143-200 kubenswrapper[2572]: I0716 00:48:03.100145 2572 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-jjc6v\" (UniqueName: \"kubernetes.io/projected/e2858360-c03d-4d0f-aa28-f9b47ecc34cf-kube-api-access-jjc6v\") pod \"limitador-operator-controller-manager-859d7b78bd-bx6hr\" (UID: \"e2858360-c03d-4d0f-aa28-f9b47ecc34cf\") " pod="kuadrant-system/limitador-operator-controller-manager-859d7b78bd-bx6hr" Jul 16 00:48:03.117974 ip-10-0-143-200 kubenswrapper[2572]: I0716 00:48:03.117939 2572 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-jjc6v\" (UniqueName: \"kubernetes.io/projected/e2858360-c03d-4d0f-aa28-f9b47ecc34cf-kube-api-access-jjc6v\") pod \"limitador-operator-controller-manager-859d7b78bd-bx6hr\" (UID: \"e2858360-c03d-4d0f-aa28-f9b47ecc34cf\") " pod="kuadrant-system/limitador-operator-controller-manager-859d7b78bd-bx6hr" Jul 16 00:48:03.225534 ip-10-0-143-200 kubenswrapper[2572]: I0716 00:48:03.225485 2572 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/limitador-operator-controller-manager-859d7b78bd-bx6hr" Jul 16 00:48:03.380924 ip-10-0-143-200 kubenswrapper[2572]: I0716 00:48:03.380894 2572 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-operator-controller-manager-859d7b78bd-bx6hr"] Jul 16 00:48:03.383085 ip-10-0-143-200 kubenswrapper[2572]: W0716 00:48:03.383053 2572 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pode2858360_c03d_4d0f_aa28_f9b47ecc34cf.slice/crio-7aae340005e8160b64b2840ebb0a83d5601ed5e7bf4fa48e5b31e434cd12f574 WatchSource:0}: Error finding container 7aae340005e8160b64b2840ebb0a83d5601ed5e7bf4fa48e5b31e434cd12f574: Status 404 returned error can't find the container with id 7aae340005e8160b64b2840ebb0a83d5601ed5e7bf4fa48e5b31e434cd12f574 Jul 16 00:48:03.531373 ip-10-0-143-200 kubenswrapper[2572]: I0716 00:48:03.531284 2572 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9gnmq_3a7e030d-fad8-480d-9dc6-fe5057048cf9/konnectivity-agent/0.log" limit=1024 Jul 16 00:48:03.659370 ip-10-0-143-200 kubenswrapper[2572]: I0716 00:48:03.659332 2572 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9gnmq_3a7e030d-fad8-480d-9dc6-fe5057048cf9/konnectivity-agent/0.log" limit=1024 Jul 16 00:48:03.736799 ip-10-0-143-200 kubenswrapper[2572]: I0716 00:48:03.736755 2572 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/limitador-operator-controller-manager-859d7b78bd-bx6hr" event={"ID":"e2858360-c03d-4d0f-aa28-f9b47ecc34cf","Type":"ContainerStarted","Data":"7aae340005e8160b64b2840ebb0a83d5601ed5e7bf4fa48e5b31e434cd12f574"} Jul 16 00:48:04.931511 ip-10-0-143-200 kubenswrapper[2572]: I0716 00:48:04.931471 2572 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9gnmq_3a7e030d-fad8-480d-9dc6-fe5057048cf9/konnectivity-agent/0.log" limit=1024 Jul 16 00:48:05.747541 ip-10-0-143-200 kubenswrapper[2572]: I0716 00:48:05.747504 2572 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/limitador-operator-controller-manager-859d7b78bd-bx6hr" event={"ID":"e2858360-c03d-4d0f-aa28-f9b47ecc34cf","Type":"ContainerStarted","Data":"0f28bbe2e87e415ff5fe8d1b28fca1c83c85f3b80bb99c49917477aa62c1b00f"} Jul 16 00:48:05.747741 ip-10-0-143-200 kubenswrapper[2572]: I0716 00:48:05.747582 2572 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="kuadrant-system/limitador-operator-controller-manager-859d7b78bd-bx6hr" Jul 16 00:48:05.770023 ip-10-0-143-200 kubenswrapper[2572]: I0716 00:48:05.769972 2572 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="kuadrant-system/limitador-operator-controller-manager-859d7b78bd-bx6hr" podStartSLOduration=1.9942719370000002 podStartE2EDuration="3.769954903s" totalImagesPullingTime="1.775682966s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-16 00:48:02 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-16 00:48:05.769015596 +0000 UTC m=+490.877770303" watchObservedRunningTime="2026-07-16 00:48:05.769954903 +0000 UTC m=+490.878709578" Jul 16 00:48:07.973045 ip-10-0-143-200 kubenswrapper[2572]: I0716 00:48:07.973009 2572 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9gnmq_3a7e030d-fad8-480d-9dc6-fe5057048cf9/konnectivity-agent/0.log" limit=1024 Jul 16 00:48:07.995123 ip-10-0-143-200 kubenswrapper[2572]: I0716 00:48:07.995086 2572 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["kuadrant-system/kuadrant-operator-controller-manager-8684c9ff49-hmjv9"] Jul 16 00:48:08.002743 ip-10-0-143-200 kubenswrapper[2572]: I0716 00:48:08.002716 2572 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/kuadrant-operator-controller-manager-8684c9ff49-hmjv9" Jul 16 00:48:08.005947 ip-10-0-143-200 kubenswrapper[2572]: I0716 00:48:08.005926 2572 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"kuadrant-system\"/\"kuadrant-operator-controller-manager-dockercfg-mf4mr\"" Jul 16 00:48:08.016251 ip-10-0-143-200 kubenswrapper[2572]: I0716 00:48:08.016227 2572 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/kuadrant-operator-controller-manager-8684c9ff49-hmjv9"] Jul 16 00:48:08.145101 ip-10-0-143-200 kubenswrapper[2572]: I0716 00:48:08.145062 2572 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"extensions-socket-volume\" (UniqueName: \"kubernetes.io/empty-dir/e7fe893d-bc9e-4e89-ad11-3c5f63d85587-extensions-socket-volume\") pod \"kuadrant-operator-controller-manager-8684c9ff49-hmjv9\" (UID: \"e7fe893d-bc9e-4e89-ad11-3c5f63d85587\") " pod="kuadrant-system/kuadrant-operator-controller-manager-8684c9ff49-hmjv9" Jul 16 00:48:08.145101 ip-10-0-143-200 kubenswrapper[2572]: I0716 00:48:08.145102 2572 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-z8d67\" (UniqueName: \"kubernetes.io/projected/e7fe893d-bc9e-4e89-ad11-3c5f63d85587-kube-api-access-z8d67\") pod \"kuadrant-operator-controller-manager-8684c9ff49-hmjv9\" (UID: \"e7fe893d-bc9e-4e89-ad11-3c5f63d85587\") " pod="kuadrant-system/kuadrant-operator-controller-manager-8684c9ff49-hmjv9" Jul 16 00:48:08.246097 ip-10-0-143-200 kubenswrapper[2572]: I0716 00:48:08.245996 2572 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"extensions-socket-volume\" (UniqueName: \"kubernetes.io/empty-dir/e7fe893d-bc9e-4e89-ad11-3c5f63d85587-extensions-socket-volume\") pod \"kuadrant-operator-controller-manager-8684c9ff49-hmjv9\" (UID: \"e7fe893d-bc9e-4e89-ad11-3c5f63d85587\") " pod="kuadrant-system/kuadrant-operator-controller-manager-8684c9ff49-hmjv9" Jul 16 00:48:08.246097 ip-10-0-143-200 kubenswrapper[2572]: I0716 00:48:08.246037 2572 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-z8d67\" (UniqueName: \"kubernetes.io/projected/e7fe893d-bc9e-4e89-ad11-3c5f63d85587-kube-api-access-z8d67\") pod \"kuadrant-operator-controller-manager-8684c9ff49-hmjv9\" (UID: \"e7fe893d-bc9e-4e89-ad11-3c5f63d85587\") " pod="kuadrant-system/kuadrant-operator-controller-manager-8684c9ff49-hmjv9" Jul 16 00:48:08.246444 ip-10-0-143-200 kubenswrapper[2572]: I0716 00:48:08.246424 2572 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"extensions-socket-volume\" (UniqueName: \"kubernetes.io/empty-dir/e7fe893d-bc9e-4e89-ad11-3c5f63d85587-extensions-socket-volume\") pod \"kuadrant-operator-controller-manager-8684c9ff49-hmjv9\" (UID: \"e7fe893d-bc9e-4e89-ad11-3c5f63d85587\") " pod="kuadrant-system/kuadrant-operator-controller-manager-8684c9ff49-hmjv9" Jul 16 00:48:08.258084 ip-10-0-143-200 kubenswrapper[2572]: I0716 00:48:08.258052 2572 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-z8d67\" (UniqueName: \"kubernetes.io/projected/e7fe893d-bc9e-4e89-ad11-3c5f63d85587-kube-api-access-z8d67\") pod \"kuadrant-operator-controller-manager-8684c9ff49-hmjv9\" (UID: \"e7fe893d-bc9e-4e89-ad11-3c5f63d85587\") " pod="kuadrant-system/kuadrant-operator-controller-manager-8684c9ff49-hmjv9" Jul 16 00:48:08.313169 ip-10-0-143-200 kubenswrapper[2572]: I0716 00:48:08.313134 2572 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/kuadrant-operator-controller-manager-8684c9ff49-hmjv9" Jul 16 00:48:08.433876 ip-10-0-143-200 kubenswrapper[2572]: I0716 00:48:08.433845 2572 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9gnmq_3a7e030d-fad8-480d-9dc6-fe5057048cf9/konnectivity-agent/0.log" limit=1024 Jul 16 00:48:08.454761 ip-10-0-143-200 kubenswrapper[2572]: I0716 00:48:08.454734 2572 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/kuadrant-operator-controller-manager-8684c9ff49-hmjv9"] Jul 16 00:48:08.456558 ip-10-0-143-200 kubenswrapper[2572]: W0716 00:48:08.456527 2572 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pode7fe893d_bc9e_4e89_ad11_3c5f63d85587.slice/crio-6b7aa9ebf00eb7a5fcfa49e2b9c8c03111766edc8d7fe5fdf2405a6b208a4452 WatchSource:0}: Error finding container 6b7aa9ebf00eb7a5fcfa49e2b9c8c03111766edc8d7fe5fdf2405a6b208a4452: Status 404 returned error can't find the container with id 6b7aa9ebf00eb7a5fcfa49e2b9c8c03111766edc8d7fe5fdf2405a6b208a4452 Jul 16 00:48:08.537560 ip-10-0-143-200 kubenswrapper[2572]: I0716 00:48:08.537463 2572 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9gnmq_3a7e030d-fad8-480d-9dc6-fe5057048cf9/konnectivity-agent/0.log" limit=1024 Jul 16 00:48:08.762689 ip-10-0-143-200 kubenswrapper[2572]: I0716 00:48:08.762648 2572 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/kuadrant-operator-controller-manager-8684c9ff49-hmjv9" event={"ID":"e7fe893d-bc9e-4e89-ad11-3c5f63d85587","Type":"ContainerStarted","Data":"6b7aa9ebf00eb7a5fcfa49e2b9c8c03111766edc8d7fe5fdf2405a6b208a4452"} Jul 16 00:48:08.888023 ip-10-0-143-200 kubenswrapper[2572]: I0716 00:48:08.887928 2572 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9gnmq_3a7e030d-fad8-480d-9dc6-fe5057048cf9/konnectivity-agent/0.log" limit=1024 Jul 16 00:48:09.275437 ip-10-0-143-200 kubenswrapper[2572]: I0716 00:48:09.275399 2572 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9gnmq_3a7e030d-fad8-480d-9dc6-fe5057048cf9/konnectivity-agent/0.log" limit=1024 Jul 16 00:48:12.312122 ip-10-0-143-200 kubenswrapper[2572]: I0716 00:48:12.312082 2572 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["kuadrant-system/authorino-operator-75cc8d5cb7-9xhgm"] Jul 16 00:48:12.315746 ip-10-0-143-200 kubenswrapper[2572]: I0716 00:48:12.315727 2572 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/authorino-operator-75cc8d5cb7-9xhgm" Jul 16 00:48:12.320338 ip-10-0-143-200 kubenswrapper[2572]: I0716 00:48:12.320315 2572 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"kuadrant-system\"/\"authorino-operator-dockercfg-nhmkp\"" Jul 16 00:48:12.327976 ip-10-0-143-200 kubenswrapper[2572]: I0716 00:48:12.327951 2572 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/authorino-operator-75cc8d5cb7-9xhgm"] Jul 16 00:48:12.484574 ip-10-0-143-200 kubenswrapper[2572]: I0716 00:48:12.484527 2572 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-9hdkd\" (UniqueName: \"kubernetes.io/projected/5afe06aa-986b-47a9-a12a-6376957e65df-kube-api-access-9hdkd\") pod \"authorino-operator-75cc8d5cb7-9xhgm\" (UID: \"5afe06aa-986b-47a9-a12a-6376957e65df\") " pod="kuadrant-system/authorino-operator-75cc8d5cb7-9xhgm" Jul 16 00:48:12.586156 ip-10-0-143-200 kubenswrapper[2572]: I0716 00:48:12.586048 2572 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-9hdkd\" (UniqueName: \"kubernetes.io/projected/5afe06aa-986b-47a9-a12a-6376957e65df-kube-api-access-9hdkd\") pod \"authorino-operator-75cc8d5cb7-9xhgm\" (UID: \"5afe06aa-986b-47a9-a12a-6376957e65df\") " pod="kuadrant-system/authorino-operator-75cc8d5cb7-9xhgm" Jul 16 00:48:12.603202 ip-10-0-143-200 kubenswrapper[2572]: I0716 00:48:12.603169 2572 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-9hdkd\" (UniqueName: \"kubernetes.io/projected/5afe06aa-986b-47a9-a12a-6376957e65df-kube-api-access-9hdkd\") pod \"authorino-operator-75cc8d5cb7-9xhgm\" (UID: \"5afe06aa-986b-47a9-a12a-6376957e65df\") " pod="kuadrant-system/authorino-operator-75cc8d5cb7-9xhgm" Jul 16 00:48:12.626592 ip-10-0-143-200 kubenswrapper[2572]: I0716 00:48:12.626554 2572 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/authorino-operator-75cc8d5cb7-9xhgm" Jul 16 00:48:12.776795 ip-10-0-143-200 kubenswrapper[2572]: I0716 00:48:12.776769 2572 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/authorino-operator-75cc8d5cb7-9xhgm"] Jul 16 00:48:12.779004 ip-10-0-143-200 kubenswrapper[2572]: W0716 00:48:12.778958 2572 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod5afe06aa_986b_47a9_a12a_6376957e65df.slice/crio-91e1887170be4bc554baa0956ab3251dbecdb4ee347fc08124639193164ab033 WatchSource:0}: Error finding container 91e1887170be4bc554baa0956ab3251dbecdb4ee347fc08124639193164ab033: Status 404 returned error can't find the container with id 91e1887170be4bc554baa0956ab3251dbecdb4ee347fc08124639193164ab033 Jul 16 00:48:12.901901 ip-10-0-143-200 kubenswrapper[2572]: I0716 00:48:12.901810 2572 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9gnmq_3a7e030d-fad8-480d-9dc6-fe5057048cf9/konnectivity-agent/0.log" limit=1024 Jul 16 00:48:13.739602 ip-10-0-143-200 kubenswrapper[2572]: I0716 00:48:13.739568 2572 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="kuadrant-system/dns-operator-controller-manager-864dd78c4c-clxrd" Jul 16 00:48:13.792638 ip-10-0-143-200 kubenswrapper[2572]: I0716 00:48:13.792600 2572 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/authorino-operator-75cc8d5cb7-9xhgm" event={"ID":"5afe06aa-986b-47a9-a12a-6376957e65df","Type":"ContainerStarted","Data":"91e1887170be4bc554baa0956ab3251dbecdb4ee347fc08124639193164ab033"} Jul 16 00:48:14.801144 ip-10-0-143-200 kubenswrapper[2572]: I0716 00:48:14.801107 2572 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/kuadrant-operator-controller-manager-8684c9ff49-hmjv9" event={"ID":"e7fe893d-bc9e-4e89-ad11-3c5f63d85587","Type":"ContainerStarted","Data":"320114588dd0e8a325a1085854a0a0009e764a43b032972189253a43f3b28036"} Jul 16 00:48:14.801601 ip-10-0-143-200 kubenswrapper[2572]: I0716 00:48:14.801198 2572 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="kuadrant-system/kuadrant-operator-controller-manager-8684c9ff49-hmjv9" Jul 16 00:48:14.875261 ip-10-0-143-200 kubenswrapper[2572]: I0716 00:48:14.875219 2572 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9gnmq_3a7e030d-fad8-480d-9dc6-fe5057048cf9/konnectivity-agent/0.log" limit=1024 Jul 16 00:48:14.977225 ip-10-0-143-200 kubenswrapper[2572]: I0716 00:48:14.977190 2572 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9gnmq_3a7e030d-fad8-480d-9dc6-fe5057048cf9/konnectivity-agent/0.log" limit=1024 Jul 16 00:48:16.755190 ip-10-0-143-200 kubenswrapper[2572]: I0716 00:48:16.755149 2572 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="kuadrant-system/limitador-operator-controller-manager-859d7b78bd-bx6hr" Jul 16 00:48:16.775236 ip-10-0-143-200 kubenswrapper[2572]: I0716 00:48:16.775181 2572 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="kuadrant-system/kuadrant-operator-controller-manager-8684c9ff49-hmjv9" podStartSLOduration=4.026267871 podStartE2EDuration="9.775163957s" totalImagesPullingTime="5.748896086s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-16 00:48:07 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-16 00:48:14.8339769 +0000 UTC m=+499.942731576" watchObservedRunningTime="2026-07-16 00:48:16.775163957 +0000 UTC m=+501.883918632" Jul 16 00:48:16.813164 ip-10-0-143-200 kubenswrapper[2572]: I0716 00:48:16.813130 2572 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/authorino-operator-75cc8d5cb7-9xhgm" event={"ID":"5afe06aa-986b-47a9-a12a-6376957e65df","Type":"ContainerStarted","Data":"081bf9311e1ce2423eb1f60150117ebb9fb343574c53d9d83ccfa89714c48f7c"} Jul 16 00:48:16.813349 ip-10-0-143-200 kubenswrapper[2572]: I0716 00:48:16.813269 2572 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="kuadrant-system/authorino-operator-75cc8d5cb7-9xhgm" Jul 16 00:48:16.842942 ip-10-0-143-200 kubenswrapper[2572]: I0716 00:48:16.842886 2572 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="kuadrant-system/authorino-operator-75cc8d5cb7-9xhgm" podStartSLOduration=1.5511323049999999 podStartE2EDuration="4.842871031s" totalImagesPullingTime="3.291738726s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-16 00:48:12 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-16 00:48:16.84129271 +0000 UTC m=+501.950047384" watchObservedRunningTime="2026-07-16 00:48:16.842871031 +0000 UTC m=+501.951625706" Jul 16 00:48:17.444946 ip-10-0-143-200 kubenswrapper[2572]: I0716 00:48:17.444907 2572 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9gnmq_3a7e030d-fad8-480d-9dc6-fe5057048cf9/konnectivity-agent/0.log" limit=1024 Jul 16 00:48:17.639899 ip-10-0-143-200 kubenswrapper[2572]: I0716 00:48:17.639866 2572 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9gnmq_3a7e030d-fad8-480d-9dc6-fe5057048cf9/konnectivity-agent/0.log" limit=1024 Jul 16 00:48:18.266266 ip-10-0-143-200 kubenswrapper[2572]: I0716 00:48:18.266232 2572 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9gnmq_3a7e030d-fad8-480d-9dc6-fe5057048cf9/konnectivity-agent/0.log" limit=1024 Jul 16 00:48:18.365503 ip-10-0-143-200 kubenswrapper[2572]: I0716 00:48:18.365467 2572 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9gnmq_3a7e030d-fad8-480d-9dc6-fe5057048cf9/konnectivity-agent/0.log" limit=1024 Jul 16 00:48:21.868116 ip-10-0-143-200 kubenswrapper[2572]: I0716 00:48:21.868075 2572 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9gnmq_3a7e030d-fad8-480d-9dc6-fe5057048cf9/konnectivity-agent/0.log" limit=1024 Jul 16 00:48:21.937754 ip-10-0-143-200 kubenswrapper[2572]: I0716 00:48:21.937704 2572 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9gnmq_3a7e030d-fad8-480d-9dc6-fe5057048cf9/konnectivity-agent/0.log" limit=1024 Jul 16 00:48:25.809723 ip-10-0-143-200 kubenswrapper[2572]: I0716 00:48:25.809686 2572 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="kuadrant-system/kuadrant-operator-controller-manager-8684c9ff49-hmjv9" Jul 16 00:48:26.753658 ip-10-0-143-200 kubenswrapper[2572]: I0716 00:48:26.753593 2572 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9gnmq_3a7e030d-fad8-480d-9dc6-fe5057048cf9/konnectivity-agent/0.log" limit=1024 Jul 16 00:48:26.857247 ip-10-0-143-200 kubenswrapper[2572]: I0716 00:48:26.857210 2572 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9gnmq_3a7e030d-fad8-480d-9dc6-fe5057048cf9/konnectivity-agent/0.log" limit=1024 Jul 16 00:48:27.551263 ip-10-0-143-200 kubenswrapper[2572]: I0716 00:48:27.551228 2572 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["kuadrant-system/kuadrant-operator-controller-manager-8684c9ff49-hmjv9"] Jul 16 00:48:27.551484 ip-10-0-143-200 kubenswrapper[2572]: I0716 00:48:27.551436 2572 kuberuntime_container.go:873] "Killing container with a grace period" pod="kuadrant-system/kuadrant-operator-controller-manager-8684c9ff49-hmjv9" podUID="e7fe893d-bc9e-4e89-ad11-3c5f63d85587" containerName="manager" containerID="cri-o://320114588dd0e8a325a1085854a0a0009e764a43b032972189253a43f3b28036" gracePeriod=2 Jul 16 00:48:27.558264 ip-10-0-143-200 kubenswrapper[2572]: I0716 00:48:27.558232 2572 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["kuadrant-system/kuadrant-operator-controller-manager-8684c9ff49-hmjv9"] Jul 16 00:48:27.575019 ip-10-0-143-200 kubenswrapper[2572]: I0716 00:48:27.574992 2572 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["kuadrant-system/limitador-operator-controller-manager-859d7b78bd-bx6hr"] Jul 16 00:48:27.575209 ip-10-0-143-200 kubenswrapper[2572]: I0716 00:48:27.575183 2572 kuberuntime_container.go:873] "Killing container with a grace period" pod="kuadrant-system/limitador-operator-controller-manager-859d7b78bd-bx6hr" podUID="e2858360-c03d-4d0f-aa28-f9b47ecc34cf" containerName="manager" containerID="cri-o://0f28bbe2e87e415ff5fe8d1b28fca1c83c85f3b80bb99c49917477aa62c1b00f" gracePeriod=2 Jul 16 00:48:27.603954 ip-10-0-143-200 kubenswrapper[2572]: I0716 00:48:27.603921 2572 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["kuadrant-system/kuadrant-operator-controller-manager-8684c9ff49-szcrm"] Jul 16 00:48:27.604334 ip-10-0-143-200 kubenswrapper[2572]: I0716 00:48:27.604298 2572 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="e7fe893d-bc9e-4e89-ad11-3c5f63d85587" containerName="manager" Jul 16 00:48:27.604334 ip-10-0-143-200 kubenswrapper[2572]: I0716 00:48:27.604314 2572 state_mem.go:107] "Deleted CPUSet assignment" podUID="e7fe893d-bc9e-4e89-ad11-3c5f63d85587" containerName="manager" Jul 16 00:48:27.604463 ip-10-0-143-200 kubenswrapper[2572]: I0716 00:48:27.604425 2572 memory_manager.go:368] "RemoveStaleState removing state" podUID="e7fe893d-bc9e-4e89-ad11-3c5f63d85587" containerName="manager" Jul 16 00:48:27.607477 ip-10-0-143-200 kubenswrapper[2572]: I0716 00:48:27.607454 2572 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/kuadrant-operator-controller-manager-8684c9ff49-szcrm" Jul 16 00:48:27.609009 ip-10-0-143-200 kubenswrapper[2572]: I0716 00:48:27.608991 2572 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["kuadrant-system/limitador-operator-controller-manager-859d7b78bd-bx6hr"] Jul 16 00:48:27.614436 ip-10-0-143-200 kubenswrapper[2572]: E0716 00:48:27.614398 2572 status_manager.go:1041] "Failed to update status for pod" err="failed to patch status \"{\\\"metadata\\\":{\\\"uid\\\":\\\"e2858360-c03d-4d0f-aa28-f9b47ecc34cf\\\"},\\\"status\\\":{\\\"$setElementOrder/conditions\\\":[{\\\"type\\\":\\\"PodReadyToStartContainers\\\"},{\\\"type\\\":\\\"Initialized\\\"},{\\\"type\\\":\\\"Ready\\\"},{\\\"type\\\":\\\"ContainersReady\\\"},{\\\"type\\\":\\\"PodScheduled\\\"}],\\\"conditions\\\":[{\\\"observedGeneration\\\":2,\\\"type\\\":\\\"PodReadyToStartContainers\\\"},{\\\"observedGeneration\\\":2,\\\"type\\\":\\\"Initialized\\\"},{\\\"observedGeneration\\\":2,\\\"type\\\":\\\"Ready\\\"},{\\\"observedGeneration\\\":2,\\\"type\\\":\\\"ContainersReady\\\"},{\\\"observedGeneration\\\":2,\\\"type\\\":\\\"PodScheduled\\\"}],\\\"observedGeneration\\\":2}}\" for pod \"kuadrant-system\"/\"limitador-operator-controller-manager-859d7b78bd-bx6hr\": pods \"limitador-operator-controller-manager-859d7b78bd-bx6hr\" not found" pod="kuadrant-system/limitador-operator-controller-manager-859d7b78bd-bx6hr" Jul 16 00:48:27.617181 ip-10-0-143-200 kubenswrapper[2572]: E0716 00:48:27.617155 2572 status_manager.go:1018] "Failed to get status for pod" err="pods \"kuadrant-operator-controller-manager-8684c9ff49-hmjv9\" is forbidden: User \"system:node:ip-10-0-143-200.ec2.internal\" cannot get resource \"pods\" in API group \"\" in the namespace \"kuadrant-system\": no relationship found between node 'ip-10-0-143-200.ec2.internal' and this object" podUID="e7fe893d-bc9e-4e89-ad11-3c5f63d85587" pod="kuadrant-system/kuadrant-operator-controller-manager-8684c9ff49-hmjv9" Jul 16 00:48:27.620851 ip-10-0-143-200 kubenswrapper[2572]: E0716 00:48:27.620819 2572 status_manager.go:1018] "Failed to get status for pod" err="pods \"limitador-operator-controller-manager-859d7b78bd-bx6hr\" is forbidden: User \"system:node:ip-10-0-143-200.ec2.internal\" cannot get resource \"pods\" in API group \"\" in the namespace \"kuadrant-system\": no relationship found between node 'ip-10-0-143-200.ec2.internal' and this object" podUID="e2858360-c03d-4d0f-aa28-f9b47ecc34cf" pod="kuadrant-system/limitador-operator-controller-manager-859d7b78bd-bx6hr" Jul 16 00:48:27.623516 ip-10-0-143-200 kubenswrapper[2572]: I0716 00:48:27.623492 2572 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"extensions-socket-volume\" (UniqueName: \"kubernetes.io/empty-dir/b03e9597-45db-4044-bd65-560a261a2d80-extensions-socket-volume\") pod \"kuadrant-operator-controller-manager-8684c9ff49-szcrm\" (UID: \"b03e9597-45db-4044-bd65-560a261a2d80\") " pod="kuadrant-system/kuadrant-operator-controller-manager-8684c9ff49-szcrm" Jul 16 00:48:27.623603 ip-10-0-143-200 kubenswrapper[2572]: I0716 00:48:27.623576 2572 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-7m8s7\" (UniqueName: \"kubernetes.io/projected/b03e9597-45db-4044-bd65-560a261a2d80-kube-api-access-7m8s7\") pod \"kuadrant-operator-controller-manager-8684c9ff49-szcrm\" (UID: \"b03e9597-45db-4044-bd65-560a261a2d80\") " pod="kuadrant-system/kuadrant-operator-controller-manager-8684c9ff49-szcrm" Jul 16 00:48:27.623770 ip-10-0-143-200 kubenswrapper[2572]: E0716 00:48:27.623740 2572 status_manager.go:1018] "Failed to get status for pod" err="pods \"kuadrant-operator-controller-manager-8684c9ff49-hmjv9\" is forbidden: User \"system:node:ip-10-0-143-200.ec2.internal\" cannot get resource \"pods\" in API group \"\" in the namespace \"kuadrant-system\": no relationship found between node 'ip-10-0-143-200.ec2.internal' and this object" podUID="e7fe893d-bc9e-4e89-ad11-3c5f63d85587" pod="kuadrant-system/kuadrant-operator-controller-manager-8684c9ff49-hmjv9" Jul 16 00:48:27.628070 ip-10-0-143-200 kubenswrapper[2572]: I0716 00:48:27.628045 2572 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/kuadrant-operator-controller-manager-8684c9ff49-szcrm"] Jul 16 00:48:27.724984 ip-10-0-143-200 kubenswrapper[2572]: I0716 00:48:27.724949 2572 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-7m8s7\" (UniqueName: \"kubernetes.io/projected/b03e9597-45db-4044-bd65-560a261a2d80-kube-api-access-7m8s7\") pod \"kuadrant-operator-controller-manager-8684c9ff49-szcrm\" (UID: \"b03e9597-45db-4044-bd65-560a261a2d80\") " pod="kuadrant-system/kuadrant-operator-controller-manager-8684c9ff49-szcrm" Jul 16 00:48:27.725174 ip-10-0-143-200 kubenswrapper[2572]: I0716 00:48:27.725030 2572 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"extensions-socket-volume\" (UniqueName: \"kubernetes.io/empty-dir/b03e9597-45db-4044-bd65-560a261a2d80-extensions-socket-volume\") pod \"kuadrant-operator-controller-manager-8684c9ff49-szcrm\" (UID: \"b03e9597-45db-4044-bd65-560a261a2d80\") " pod="kuadrant-system/kuadrant-operator-controller-manager-8684c9ff49-szcrm" Jul 16 00:48:27.725450 ip-10-0-143-200 kubenswrapper[2572]: I0716 00:48:27.725422 2572 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"extensions-socket-volume\" (UniqueName: \"kubernetes.io/empty-dir/b03e9597-45db-4044-bd65-560a261a2d80-extensions-socket-volume\") pod \"kuadrant-operator-controller-manager-8684c9ff49-szcrm\" (UID: \"b03e9597-45db-4044-bd65-560a261a2d80\") " pod="kuadrant-system/kuadrant-operator-controller-manager-8684c9ff49-szcrm" Jul 16 00:48:27.738440 ip-10-0-143-200 kubenswrapper[2572]: I0716 00:48:27.738406 2572 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-7m8s7\" (UniqueName: \"kubernetes.io/projected/b03e9597-45db-4044-bd65-560a261a2d80-kube-api-access-7m8s7\") pod \"kuadrant-operator-controller-manager-8684c9ff49-szcrm\" (UID: \"b03e9597-45db-4044-bd65-560a261a2d80\") " pod="kuadrant-system/kuadrant-operator-controller-manager-8684c9ff49-szcrm" Jul 16 00:48:27.820202 ip-10-0-143-200 kubenswrapper[2572]: I0716 00:48:27.820174 2572 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="kuadrant-system/authorino-operator-75cc8d5cb7-9xhgm" Jul 16 00:48:27.822883 ip-10-0-143-200 kubenswrapper[2572]: E0716 00:48:27.822857 2572 status_manager.go:1018] "Failed to get status for pod" err="pods \"limitador-operator-controller-manager-859d7b78bd-bx6hr\" is forbidden: User \"system:node:ip-10-0-143-200.ec2.internal\" cannot get resource \"pods\" in API group \"\" in the namespace \"kuadrant-system\": no relationship found between node 'ip-10-0-143-200.ec2.internal' and this object" podUID="e2858360-c03d-4d0f-aa28-f9b47ecc34cf" pod="kuadrant-system/limitador-operator-controller-manager-859d7b78bd-bx6hr" Jul 16 00:48:27.826160 ip-10-0-143-200 kubenswrapper[2572]: I0716 00:48:27.826131 2572 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/kuadrant-operator-controller-manager-8684c9ff49-hmjv9" Jul 16 00:48:27.828866 ip-10-0-143-200 kubenswrapper[2572]: I0716 00:48:27.828844 2572 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/limitador-operator-controller-manager-859d7b78bd-bx6hr" Jul 16 00:48:27.852712 ip-10-0-143-200 kubenswrapper[2572]: E0716 00:48:27.852674 2572 status_manager.go:1018] "Failed to get status for pod" err="pods \"kuadrant-operator-controller-manager-8684c9ff49-hmjv9\" is forbidden: User \"system:node:ip-10-0-143-200.ec2.internal\" cannot get resource \"pods\" in API group \"\" in the namespace \"kuadrant-system\": no relationship found between node 'ip-10-0-143-200.ec2.internal' and this object" podUID="e7fe893d-bc9e-4e89-ad11-3c5f63d85587" pod="kuadrant-system/kuadrant-operator-controller-manager-8684c9ff49-hmjv9" Jul 16 00:48:27.855737 ip-10-0-143-200 kubenswrapper[2572]: E0716 00:48:27.855712 2572 status_manager.go:1018] "Failed to get status for pod" err="pods \"kuadrant-operator-controller-manager-8684c9ff49-hmjv9\" is forbidden: User \"system:node:ip-10-0-143-200.ec2.internal\" cannot get resource \"pods\" in API group \"\" in the namespace \"kuadrant-system\": no relationship found between node 'ip-10-0-143-200.ec2.internal' and this object" podUID="e7fe893d-bc9e-4e89-ad11-3c5f63d85587" pod="kuadrant-system/kuadrant-operator-controller-manager-8684c9ff49-hmjv9" Jul 16 00:48:27.857746 ip-10-0-143-200 kubenswrapper[2572]: E0716 00:48:27.857722 2572 status_manager.go:1018] "Failed to get status for pod" err="pods \"limitador-operator-controller-manager-859d7b78bd-bx6hr\" is forbidden: User \"system:node:ip-10-0-143-200.ec2.internal\" cannot get resource \"pods\" in API group \"\" in the namespace \"kuadrant-system\": no relationship found between node 'ip-10-0-143-200.ec2.internal' and this object" podUID="e2858360-c03d-4d0f-aa28-f9b47ecc34cf" pod="kuadrant-system/limitador-operator-controller-manager-859d7b78bd-bx6hr" Jul 16 00:48:27.876236 ip-10-0-143-200 kubenswrapper[2572]: I0716 00:48:27.876200 2572 generic.go:356] "Generic (PLEG): container finished" podID="e2858360-c03d-4d0f-aa28-f9b47ecc34cf" containerID="0f28bbe2e87e415ff5fe8d1b28fca1c83c85f3b80bb99c49917477aa62c1b00f" exitCode=0 Jul 16 00:48:27.876410 ip-10-0-143-200 kubenswrapper[2572]: I0716 00:48:27.876248 2572 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/limitador-operator-controller-manager-859d7b78bd-bx6hr" Jul 16 00:48:27.876410 ip-10-0-143-200 kubenswrapper[2572]: I0716 00:48:27.876297 2572 scope.go:117] "RemoveContainer" containerID="0f28bbe2e87e415ff5fe8d1b28fca1c83c85f3b80bb99c49917477aa62c1b00f" Jul 16 00:48:27.877881 ip-10-0-143-200 kubenswrapper[2572]: I0716 00:48:27.877846 2572 generic.go:356] "Generic (PLEG): container finished" podID="e7fe893d-bc9e-4e89-ad11-3c5f63d85587" containerID="320114588dd0e8a325a1085854a0a0009e764a43b032972189253a43f3b28036" exitCode=0 Jul 16 00:48:27.877980 ip-10-0-143-200 kubenswrapper[2572]: I0716 00:48:27.877892 2572 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/kuadrant-operator-controller-manager-8684c9ff49-hmjv9" Jul 16 00:48:27.879642 ip-10-0-143-200 kubenswrapper[2572]: E0716 00:48:27.879606 2572 status_manager.go:1018] "Failed to get status for pod" err="pods \"kuadrant-operator-controller-manager-8684c9ff49-hmjv9\" is forbidden: User \"system:node:ip-10-0-143-200.ec2.internal\" cannot get resource \"pods\" in API group \"\" in the namespace \"kuadrant-system\": no relationship found between node 'ip-10-0-143-200.ec2.internal' and this object" podUID="e7fe893d-bc9e-4e89-ad11-3c5f63d85587" pod="kuadrant-system/kuadrant-operator-controller-manager-8684c9ff49-hmjv9" Jul 16 00:48:27.882969 ip-10-0-143-200 kubenswrapper[2572]: E0716 00:48:27.882944 2572 status_manager.go:1018] "Failed to get status for pod" err="pods \"limitador-operator-controller-manager-859d7b78bd-bx6hr\" is forbidden: User \"system:node:ip-10-0-143-200.ec2.internal\" cannot get resource \"pods\" in API group \"\" in the namespace \"kuadrant-system\": no relationship found between node 'ip-10-0-143-200.ec2.internal' and this object" podUID="e2858360-c03d-4d0f-aa28-f9b47ecc34cf" pod="kuadrant-system/limitador-operator-controller-manager-859d7b78bd-bx6hr" Jul 16 00:48:27.885286 ip-10-0-143-200 kubenswrapper[2572]: E0716 00:48:27.885260 2572 status_manager.go:1018] "Failed to get status for pod" err="pods \"kuadrant-operator-controller-manager-8684c9ff49-hmjv9\" is forbidden: User \"system:node:ip-10-0-143-200.ec2.internal\" cannot get resource \"pods\" in API group \"\" in the namespace \"kuadrant-system\": no relationship found between node 'ip-10-0-143-200.ec2.internal' and this object" podUID="e7fe893d-bc9e-4e89-ad11-3c5f63d85587" pod="kuadrant-system/kuadrant-operator-controller-manager-8684c9ff49-hmjv9" Jul 16 00:48:27.885389 ip-10-0-143-200 kubenswrapper[2572]: I0716 00:48:27.885266 2572 scope.go:117] "RemoveContainer" containerID="0f28bbe2e87e415ff5fe8d1b28fca1c83c85f3b80bb99c49917477aa62c1b00f" Jul 16 00:48:27.885672 ip-10-0-143-200 kubenswrapper[2572]: E0716 00:48:27.885643 2572 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"0f28bbe2e87e415ff5fe8d1b28fca1c83c85f3b80bb99c49917477aa62c1b00f\": container with ID starting with 0f28bbe2e87e415ff5fe8d1b28fca1c83c85f3b80bb99c49917477aa62c1b00f not found: ID does not exist" containerID="0f28bbe2e87e415ff5fe8d1b28fca1c83c85f3b80bb99c49917477aa62c1b00f" Jul 16 00:48:27.885780 ip-10-0-143-200 kubenswrapper[2572]: I0716 00:48:27.885684 2572 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"0f28bbe2e87e415ff5fe8d1b28fca1c83c85f3b80bb99c49917477aa62c1b00f"} err="failed to get container status \"0f28bbe2e87e415ff5fe8d1b28fca1c83c85f3b80bb99c49917477aa62c1b00f\": rpc error: code = NotFound desc = could not find container \"0f28bbe2e87e415ff5fe8d1b28fca1c83c85f3b80bb99c49917477aa62c1b00f\": container with ID starting with 0f28bbe2e87e415ff5fe8d1b28fca1c83c85f3b80bb99c49917477aa62c1b00f not found: ID does not exist" Jul 16 00:48:27.885780 ip-10-0-143-200 kubenswrapper[2572]: I0716 00:48:27.885710 2572 scope.go:117] "RemoveContainer" containerID="320114588dd0e8a325a1085854a0a0009e764a43b032972189253a43f3b28036" Jul 16 00:48:27.888277 ip-10-0-143-200 kubenswrapper[2572]: E0716 00:48:27.888249 2572 status_manager.go:1018] "Failed to get status for pod" err="pods \"limitador-operator-controller-manager-859d7b78bd-bx6hr\" is forbidden: User \"system:node:ip-10-0-143-200.ec2.internal\" cannot get resource \"pods\" in API group \"\" in the namespace \"kuadrant-system\": no relationship found between node 'ip-10-0-143-200.ec2.internal' and this object" podUID="e2858360-c03d-4d0f-aa28-f9b47ecc34cf" pod="kuadrant-system/limitador-operator-controller-manager-859d7b78bd-bx6hr" Jul 16 00:48:27.894310 ip-10-0-143-200 kubenswrapper[2572]: I0716 00:48:27.894291 2572 scope.go:117] "RemoveContainer" containerID="320114588dd0e8a325a1085854a0a0009e764a43b032972189253a43f3b28036" Jul 16 00:48:27.894603 ip-10-0-143-200 kubenswrapper[2572]: E0716 00:48:27.894582 2572 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"320114588dd0e8a325a1085854a0a0009e764a43b032972189253a43f3b28036\": container with ID starting with 320114588dd0e8a325a1085854a0a0009e764a43b032972189253a43f3b28036 not found: ID does not exist" containerID="320114588dd0e8a325a1085854a0a0009e764a43b032972189253a43f3b28036" Jul 16 00:48:27.894705 ip-10-0-143-200 kubenswrapper[2572]: I0716 00:48:27.894610 2572 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"320114588dd0e8a325a1085854a0a0009e764a43b032972189253a43f3b28036"} err="failed to get container status \"320114588dd0e8a325a1085854a0a0009e764a43b032972189253a43f3b28036\": rpc error: code = NotFound desc = could not find container \"320114588dd0e8a325a1085854a0a0009e764a43b032972189253a43f3b28036\": container with ID starting with 320114588dd0e8a325a1085854a0a0009e764a43b032972189253a43f3b28036 not found: ID does not exist" Jul 16 00:48:27.926087 ip-10-0-143-200 kubenswrapper[2572]: I0716 00:48:27.926043 2572 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"extensions-socket-volume\" (UniqueName: \"kubernetes.io/empty-dir/e7fe893d-bc9e-4e89-ad11-3c5f63d85587-extensions-socket-volume\") pod \"e7fe893d-bc9e-4e89-ad11-3c5f63d85587\" (UID: \"e7fe893d-bc9e-4e89-ad11-3c5f63d85587\") " Jul 16 00:48:27.926257 ip-10-0-143-200 kubenswrapper[2572]: I0716 00:48:27.926098 2572 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-jjc6v\" (UniqueName: \"kubernetes.io/projected/e2858360-c03d-4d0f-aa28-f9b47ecc34cf-kube-api-access-jjc6v\") pod \"e2858360-c03d-4d0f-aa28-f9b47ecc34cf\" (UID: \"e2858360-c03d-4d0f-aa28-f9b47ecc34cf\") " Jul 16 00:48:27.926257 ip-10-0-143-200 kubenswrapper[2572]: I0716 00:48:27.926150 2572 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-z8d67\" (UniqueName: \"kubernetes.io/projected/e7fe893d-bc9e-4e89-ad11-3c5f63d85587-kube-api-access-z8d67\") pod \"e7fe893d-bc9e-4e89-ad11-3c5f63d85587\" (UID: \"e7fe893d-bc9e-4e89-ad11-3c5f63d85587\") " Jul 16 00:48:27.926644 ip-10-0-143-200 kubenswrapper[2572]: I0716 00:48:27.926595 2572 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/e7fe893d-bc9e-4e89-ad11-3c5f63d85587-extensions-socket-volume" (OuterVolumeSpecName: "extensions-socket-volume") pod "e7fe893d-bc9e-4e89-ad11-3c5f63d85587" (UID: "e7fe893d-bc9e-4e89-ad11-3c5f63d85587"). InnerVolumeSpecName "extensions-socket-volume". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Jul 16 00:48:27.928166 ip-10-0-143-200 kubenswrapper[2572]: I0716 00:48:27.928145 2572 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/e2858360-c03d-4d0f-aa28-f9b47ecc34cf-kube-api-access-jjc6v" (OuterVolumeSpecName: "kube-api-access-jjc6v") pod "e2858360-c03d-4d0f-aa28-f9b47ecc34cf" (UID: "e2858360-c03d-4d0f-aa28-f9b47ecc34cf"). InnerVolumeSpecName "kube-api-access-jjc6v". PluginName "kubernetes.io/projected", VolumeGIDValue "" Jul 16 00:48:27.928233 ip-10-0-143-200 kubenswrapper[2572]: I0716 00:48:27.928150 2572 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/e7fe893d-bc9e-4e89-ad11-3c5f63d85587-kube-api-access-z8d67" (OuterVolumeSpecName: "kube-api-access-z8d67") pod "e7fe893d-bc9e-4e89-ad11-3c5f63d85587" (UID: "e7fe893d-bc9e-4e89-ad11-3c5f63d85587"). InnerVolumeSpecName "kube-api-access-z8d67". PluginName "kubernetes.io/projected", VolumeGIDValue "" Jul 16 00:48:27.967460 ip-10-0-143-200 kubenswrapper[2572]: I0716 00:48:27.967413 2572 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9gnmq_3a7e030d-fad8-480d-9dc6-fe5057048cf9/konnectivity-agent/0.log" limit=1024 Jul 16 00:48:27.982263 ip-10-0-143-200 kubenswrapper[2572]: I0716 00:48:27.982228 2572 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/kuadrant-operator-controller-manager-8684c9ff49-szcrm" Jul 16 00:48:28.027199 ip-10-0-143-200 kubenswrapper[2572]: I0716 00:48:28.027160 2572 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-z8d67\" (UniqueName: \"kubernetes.io/projected/e7fe893d-bc9e-4e89-ad11-3c5f63d85587-kube-api-access-z8d67\") on node \"ip-10-0-143-200.ec2.internal\" DevicePath \"\"" Jul 16 00:48:28.027199 ip-10-0-143-200 kubenswrapper[2572]: I0716 00:48:28.027193 2572 reconciler_common.go:299] "Volume detached for volume \"extensions-socket-volume\" (UniqueName: \"kubernetes.io/empty-dir/e7fe893d-bc9e-4e89-ad11-3c5f63d85587-extensions-socket-volume\") on node \"ip-10-0-143-200.ec2.internal\" DevicePath \"\"" Jul 16 00:48:28.027199 ip-10-0-143-200 kubenswrapper[2572]: I0716 00:48:28.027206 2572 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-jjc6v\" (UniqueName: \"kubernetes.io/projected/e2858360-c03d-4d0f-aa28-f9b47ecc34cf-kube-api-access-jjc6v\") on node \"ip-10-0-143-200.ec2.internal\" DevicePath \"\"" Jul 16 00:48:28.120731 ip-10-0-143-200 kubenswrapper[2572]: I0716 00:48:28.120692 2572 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9gnmq_3a7e030d-fad8-480d-9dc6-fe5057048cf9/konnectivity-agent/0.log" limit=1024 Jul 16 00:48:28.159300 ip-10-0-143-200 kubenswrapper[2572]: I0716 00:48:28.159265 2572 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/kuadrant-operator-controller-manager-8684c9ff49-szcrm"] Jul 16 00:48:28.162865 ip-10-0-143-200 kubenswrapper[2572]: W0716 00:48:28.162837 2572 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podb03e9597_45db_4044_bd65_560a261a2d80.slice/crio-8c74707f86e331c1422805085c212dbdd0b553d713afede7d1dbd73c5599a842 WatchSource:0}: Error finding container 8c74707f86e331c1422805085c212dbdd0b553d713afede7d1dbd73c5599a842: Status 404 returned error can't find the container with id 8c74707f86e331c1422805085c212dbdd0b553d713afede7d1dbd73c5599a842 Jul 16 00:48:28.205573 ip-10-0-143-200 kubenswrapper[2572]: E0716 00:48:28.205543 2572 status_manager.go:1018] "Failed to get status for pod" err="pods \"limitador-operator-controller-manager-859d7b78bd-bx6hr\" is forbidden: User \"system:node:ip-10-0-143-200.ec2.internal\" cannot get resource \"pods\" in API group \"\" in the namespace \"kuadrant-system\": no relationship found between node 'ip-10-0-143-200.ec2.internal' and this object" podUID="e2858360-c03d-4d0f-aa28-f9b47ecc34cf" pod="kuadrant-system/limitador-operator-controller-manager-859d7b78bd-bx6hr" Jul 16 00:48:28.208067 ip-10-0-143-200 kubenswrapper[2572]: E0716 00:48:28.208033 2572 status_manager.go:1018] "Failed to get status for pod" err="pods \"kuadrant-operator-controller-manager-8684c9ff49-hmjv9\" is forbidden: User \"system:node:ip-10-0-143-200.ec2.internal\" cannot get resource \"pods\" in API group \"\" in the namespace \"kuadrant-system\": no relationship found between node 'ip-10-0-143-200.ec2.internal' and this object" podUID="e7fe893d-bc9e-4e89-ad11-3c5f63d85587" pod="kuadrant-system/kuadrant-operator-controller-manager-8684c9ff49-hmjv9" Jul 16 00:48:28.210436 ip-10-0-143-200 kubenswrapper[2572]: E0716 00:48:28.210407 2572 status_manager.go:1018] "Failed to get status for pod" err="pods \"limitador-operator-controller-manager-859d7b78bd-bx6hr\" is forbidden: User \"system:node:ip-10-0-143-200.ec2.internal\" cannot get resource \"pods\" in API group \"\" in the namespace \"kuadrant-system\": no relationship found between node 'ip-10-0-143-200.ec2.internal' and this object" podUID="e2858360-c03d-4d0f-aa28-f9b47ecc34cf" pod="kuadrant-system/limitador-operator-controller-manager-859d7b78bd-bx6hr" Jul 16 00:48:28.212836 ip-10-0-143-200 kubenswrapper[2572]: E0716 00:48:28.212810 2572 status_manager.go:1018] "Failed to get status for pod" err="pods \"kuadrant-operator-controller-manager-8684c9ff49-hmjv9\" is forbidden: User \"system:node:ip-10-0-143-200.ec2.internal\" cannot get resource \"pods\" in API group \"\" in the namespace \"kuadrant-system\": no relationship found between node 'ip-10-0-143-200.ec2.internal' and this object" podUID="e7fe893d-bc9e-4e89-ad11-3c5f63d85587" pod="kuadrant-system/kuadrant-operator-controller-manager-8684c9ff49-hmjv9" Jul 16 00:48:28.865541 ip-10-0-143-200 kubenswrapper[2572]: I0716 00:48:28.865455 2572 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["kuadrant-system/kuadrant-operator-controller-manager-86878c769f-w87nk"] Jul 16 00:48:28.865950 ip-10-0-143-200 kubenswrapper[2572]: I0716 00:48:28.865864 2572 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="e2858360-c03d-4d0f-aa28-f9b47ecc34cf" containerName="manager" Jul 16 00:48:28.865950 ip-10-0-143-200 kubenswrapper[2572]: I0716 00:48:28.865877 2572 state_mem.go:107] "Deleted CPUSet assignment" podUID="e2858360-c03d-4d0f-aa28-f9b47ecc34cf" containerName="manager" Jul 16 00:48:28.865950 ip-10-0-143-200 kubenswrapper[2572]: I0716 00:48:28.865942 2572 memory_manager.go:368] "RemoveStaleState removing state" podUID="e2858360-c03d-4d0f-aa28-f9b47ecc34cf" containerName="manager" Jul 16 00:48:28.869156 ip-10-0-143-200 kubenswrapper[2572]: I0716 00:48:28.869138 2572 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/kuadrant-operator-controller-manager-86878c769f-w87nk" Jul 16 00:48:28.871920 ip-10-0-143-200 kubenswrapper[2572]: E0716 00:48:28.871884 2572 status_manager.go:1018] "Failed to get status for pod" err="pods \"limitador-operator-controller-manager-859d7b78bd-bx6hr\" is forbidden: User \"system:node:ip-10-0-143-200.ec2.internal\" cannot get resource \"pods\" in API group \"\" in the namespace \"kuadrant-system\": no relationship found between node 'ip-10-0-143-200.ec2.internal' and this object" podUID="e2858360-c03d-4d0f-aa28-f9b47ecc34cf" pod="kuadrant-system/limitador-operator-controller-manager-859d7b78bd-bx6hr" Jul 16 00:48:28.884860 ip-10-0-143-200 kubenswrapper[2572]: I0716 00:48:28.884828 2572 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/kuadrant-operator-controller-manager-8684c9ff49-szcrm" event={"ID":"b03e9597-45db-4044-bd65-560a261a2d80","Type":"ContainerStarted","Data":"c14b27dcc07b66035e6ddb1e051637768d584be3f0dede7f0c2ef62474dd1ce8"} Jul 16 00:48:28.885028 ip-10-0-143-200 kubenswrapper[2572]: I0716 00:48:28.884889 2572 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/kuadrant-operator-controller-manager-8684c9ff49-szcrm" event={"ID":"b03e9597-45db-4044-bd65-560a261a2d80","Type":"ContainerStarted","Data":"8c74707f86e331c1422805085c212dbdd0b553d713afede7d1dbd73c5599a842"} Jul 16 00:48:28.885028 ip-10-0-143-200 kubenswrapper[2572]: I0716 00:48:28.884943 2572 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="kuadrant-system/kuadrant-operator-controller-manager-8684c9ff49-szcrm" Jul 16 00:48:28.899927 ip-10-0-143-200 kubenswrapper[2572]: I0716 00:48:28.899895 2572 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/kuadrant-operator-controller-manager-86878c769f-w87nk"] Jul 16 00:48:28.919385 ip-10-0-143-200 kubenswrapper[2572]: E0716 00:48:28.919343 2572 status_manager.go:1018] "Failed to get status for pod" err="pods \"kuadrant-operator-controller-manager-8684c9ff49-hmjv9\" is forbidden: User \"system:node:ip-10-0-143-200.ec2.internal\" cannot get resource \"pods\" in API group \"\" in the namespace \"kuadrant-system\": no relationship found between node 'ip-10-0-143-200.ec2.internal' and this object" podUID="e7fe893d-bc9e-4e89-ad11-3c5f63d85587" pod="kuadrant-system/kuadrant-operator-controller-manager-8684c9ff49-hmjv9" Jul 16 00:48:28.922087 ip-10-0-143-200 kubenswrapper[2572]: E0716 00:48:28.922059 2572 status_manager.go:1018] "Failed to get status for pod" err="pods \"limitador-operator-controller-manager-859d7b78bd-bx6hr\" is forbidden: User \"system:node:ip-10-0-143-200.ec2.internal\" cannot get resource \"pods\" in API group \"\" in the namespace \"kuadrant-system\": no relationship found between node 'ip-10-0-143-200.ec2.internal' and this object" podUID="e2858360-c03d-4d0f-aa28-f9b47ecc34cf" pod="kuadrant-system/limitador-operator-controller-manager-859d7b78bd-bx6hr" Jul 16 00:48:28.924134 ip-10-0-143-200 kubenswrapper[2572]: E0716 00:48:28.924117 2572 status_manager.go:1018] "Failed to get status for pod" err="pods \"kuadrant-operator-controller-manager-8684c9ff49-hmjv9\" is forbidden: User \"system:node:ip-10-0-143-200.ec2.internal\" cannot get resource \"pods\" in API group \"\" in the namespace \"kuadrant-system\": no relationship found between node 'ip-10-0-143-200.ec2.internal' and this object" podUID="e7fe893d-bc9e-4e89-ad11-3c5f63d85587" pod="kuadrant-system/kuadrant-operator-controller-manager-8684c9ff49-hmjv9" Jul 16 00:48:28.934402 ip-10-0-143-200 kubenswrapper[2572]: I0716 00:48:28.934367 2572 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"extensions-socket-volume\" (UniqueName: \"kubernetes.io/empty-dir/cc2779f5-93cd-4ea3-a618-c8a51ed440b4-extensions-socket-volume\") pod \"kuadrant-operator-controller-manager-86878c769f-w87nk\" (UID: \"cc2779f5-93cd-4ea3-a618-c8a51ed440b4\") " pod="kuadrant-system/kuadrant-operator-controller-manager-86878c769f-w87nk" Jul 16 00:48:28.934504 ip-10-0-143-200 kubenswrapper[2572]: I0716 00:48:28.934448 2572 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-z5qwh\" (UniqueName: \"kubernetes.io/projected/cc2779f5-93cd-4ea3-a618-c8a51ed440b4-kube-api-access-z5qwh\") pod \"kuadrant-operator-controller-manager-86878c769f-w87nk\" (UID: \"cc2779f5-93cd-4ea3-a618-c8a51ed440b4\") " pod="kuadrant-system/kuadrant-operator-controller-manager-86878c769f-w87nk" Jul 16 00:48:28.951157 ip-10-0-143-200 kubenswrapper[2572]: I0716 00:48:28.951106 2572 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="kuadrant-system/kuadrant-operator-controller-manager-8684c9ff49-szcrm" podStartSLOduration=1.951090614 podStartE2EDuration="1.951090614s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-16 00:48:27 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-16 00:48:28.949387645 +0000 UTC m=+514.058142321" watchObservedRunningTime="2026-07-16 00:48:28.951090614 +0000 UTC m=+514.059845287" Jul 16 00:48:29.035198 ip-10-0-143-200 kubenswrapper[2572]: I0716 00:48:29.035153 2572 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-z5qwh\" (UniqueName: \"kubernetes.io/projected/cc2779f5-93cd-4ea3-a618-c8a51ed440b4-kube-api-access-z5qwh\") pod \"kuadrant-operator-controller-manager-86878c769f-w87nk\" (UID: \"cc2779f5-93cd-4ea3-a618-c8a51ed440b4\") " pod="kuadrant-system/kuadrant-operator-controller-manager-86878c769f-w87nk" Jul 16 00:48:29.035392 ip-10-0-143-200 kubenswrapper[2572]: I0716 00:48:29.035244 2572 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"extensions-socket-volume\" (UniqueName: \"kubernetes.io/empty-dir/cc2779f5-93cd-4ea3-a618-c8a51ed440b4-extensions-socket-volume\") pod \"kuadrant-operator-controller-manager-86878c769f-w87nk\" (UID: \"cc2779f5-93cd-4ea3-a618-c8a51ed440b4\") " pod="kuadrant-system/kuadrant-operator-controller-manager-86878c769f-w87nk" Jul 16 00:48:29.035666 ip-10-0-143-200 kubenswrapper[2572]: I0716 00:48:29.035615 2572 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"extensions-socket-volume\" (UniqueName: \"kubernetes.io/empty-dir/cc2779f5-93cd-4ea3-a618-c8a51ed440b4-extensions-socket-volume\") pod \"kuadrant-operator-controller-manager-86878c769f-w87nk\" (UID: \"cc2779f5-93cd-4ea3-a618-c8a51ed440b4\") " pod="kuadrant-system/kuadrant-operator-controller-manager-86878c769f-w87nk" Jul 16 00:48:29.060084 ip-10-0-143-200 kubenswrapper[2572]: I0716 00:48:29.060049 2572 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-z5qwh\" (UniqueName: \"kubernetes.io/projected/cc2779f5-93cd-4ea3-a618-c8a51ed440b4-kube-api-access-z5qwh\") pod \"kuadrant-operator-controller-manager-86878c769f-w87nk\" (UID: \"cc2779f5-93cd-4ea3-a618-c8a51ed440b4\") " pod="kuadrant-system/kuadrant-operator-controller-manager-86878c769f-w87nk" Jul 16 00:48:29.179948 ip-10-0-143-200 kubenswrapper[2572]: I0716 00:48:29.179853 2572 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/kuadrant-operator-controller-manager-86878c769f-w87nk" Jul 16 00:48:29.324740 ip-10-0-143-200 kubenswrapper[2572]: I0716 00:48:29.324716 2572 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/kuadrant-operator-controller-manager-86878c769f-w87nk"] Jul 16 00:48:29.326994 ip-10-0-143-200 kubenswrapper[2572]: W0716 00:48:29.326966 2572 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podcc2779f5_93cd_4ea3_a618_c8a51ed440b4.slice/crio-45cc4754cc69e325232a0c5f44fa5bc52cb74ae7a010ce8da982fb121bf2fad7 WatchSource:0}: Error finding container 45cc4754cc69e325232a0c5f44fa5bc52cb74ae7a010ce8da982fb121bf2fad7: Status 404 returned error can't find the container with id 45cc4754cc69e325232a0c5f44fa5bc52cb74ae7a010ce8da982fb121bf2fad7 Jul 16 00:48:29.449046 ip-10-0-143-200 kubenswrapper[2572]: I0716 00:48:29.448958 2572 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="e2858360-c03d-4d0f-aa28-f9b47ecc34cf" path="/var/lib/kubelet/pods/e2858360-c03d-4d0f-aa28-f9b47ecc34cf/volumes" Jul 16 00:48:29.449327 ip-10-0-143-200 kubenswrapper[2572]: I0716 00:48:29.449314 2572 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="e7fe893d-bc9e-4e89-ad11-3c5f63d85587" path="/var/lib/kubelet/pods/e7fe893d-bc9e-4e89-ad11-3c5f63d85587/volumes" Jul 16 00:48:29.893302 ip-10-0-143-200 kubenswrapper[2572]: I0716 00:48:29.893264 2572 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/kuadrant-operator-controller-manager-86878c769f-w87nk" event={"ID":"cc2779f5-93cd-4ea3-a618-c8a51ed440b4","Type":"ContainerStarted","Data":"61ab879bfde8f1f8c87e65d286100317a66d9d7685eb11f3921f7a279db505f0"} Jul 16 00:48:29.893684 ip-10-0-143-200 kubenswrapper[2572]: I0716 00:48:29.893324 2572 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/kuadrant-operator-controller-manager-86878c769f-w87nk" event={"ID":"cc2779f5-93cd-4ea3-a618-c8a51ed440b4","Type":"ContainerStarted","Data":"45cc4754cc69e325232a0c5f44fa5bc52cb74ae7a010ce8da982fb121bf2fad7"} Jul 16 00:48:29.893684 ip-10-0-143-200 kubenswrapper[2572]: I0716 00:48:29.893591 2572 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="kuadrant-system/kuadrant-operator-controller-manager-86878c769f-w87nk" Jul 16 00:48:29.918995 ip-10-0-143-200 kubenswrapper[2572]: I0716 00:48:29.918933 2572 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="kuadrant-system/kuadrant-operator-controller-manager-86878c769f-w87nk" podStartSLOduration=1.918910975 podStartE2EDuration="1.918910975s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-16 00:48:28 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-16 00:48:29.916831358 +0000 UTC m=+515.025586032" watchObservedRunningTime="2026-07-16 00:48:29.918910975 +0000 UTC m=+515.027665651" Jul 16 00:48:31.851691 ip-10-0-143-200 kubenswrapper[2572]: I0716 00:48:31.851656 2572 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9gnmq_3a7e030d-fad8-480d-9dc6-fe5057048cf9/konnectivity-agent/0.log" limit=1024 Jul 16 00:48:31.951971 ip-10-0-143-200 kubenswrapper[2572]: I0716 00:48:31.951935 2572 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9gnmq_3a7e030d-fad8-480d-9dc6-fe5057048cf9/konnectivity-agent/0.log" limit=1024 Jul 16 00:48:32.258767 ip-10-0-143-200 kubenswrapper[2572]: I0716 00:48:32.258728 2572 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9gnmq_3a7e030d-fad8-480d-9dc6-fe5057048cf9/konnectivity-agent/0.log" limit=1024 Jul 16 00:48:32.359540 ip-10-0-143-200 kubenswrapper[2572]: I0716 00:48:32.359502 2572 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9gnmq_3a7e030d-fad8-480d-9dc6-fe5057048cf9/konnectivity-agent/0.log" limit=1024 Jul 16 00:48:36.671717 ip-10-0-143-200 kubenswrapper[2572]: I0716 00:48:36.671682 2572 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9gnmq_3a7e030d-fad8-480d-9dc6-fe5057048cf9/konnectivity-agent/0.log" limit=1024 Jul 16 00:48:36.772164 ip-10-0-143-200 kubenswrapper[2572]: I0716 00:48:36.772129 2572 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9gnmq_3a7e030d-fad8-480d-9dc6-fe5057048cf9/konnectivity-agent/0.log" limit=1024 Jul 16 00:48:39.895449 ip-10-0-143-200 kubenswrapper[2572]: I0716 00:48:39.895412 2572 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="kuadrant-system/kuadrant-operator-controller-manager-8684c9ff49-szcrm" Jul 16 00:48:40.045685 ip-10-0-143-200 kubenswrapper[2572]: I0716 00:48:40.045613 2572 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9gnmq_3a7e030d-fad8-480d-9dc6-fe5057048cf9/konnectivity-agent/0.log" limit=1024 Jul 16 00:48:40.145711 ip-10-0-143-200 kubenswrapper[2572]: I0716 00:48:40.145588 2572 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9gnmq_3a7e030d-fad8-480d-9dc6-fe5057048cf9/konnectivity-agent/0.log" limit=1024 Jul 16 00:48:40.901138 ip-10-0-143-200 kubenswrapper[2572]: I0716 00:48:40.901109 2572 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="kuadrant-system/kuadrant-operator-controller-manager-86878c769f-w87nk" Jul 16 00:48:40.993084 ip-10-0-143-200 kubenswrapper[2572]: I0716 00:48:40.993053 2572 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["kuadrant-system/kuadrant-operator-controller-manager-8684c9ff49-szcrm"] Jul 16 00:48:40.993273 ip-10-0-143-200 kubenswrapper[2572]: I0716 00:48:40.993249 2572 kuberuntime_container.go:873] "Killing container with a grace period" pod="kuadrant-system/kuadrant-operator-controller-manager-8684c9ff49-szcrm" podUID="b03e9597-45db-4044-bd65-560a261a2d80" containerName="manager" containerID="cri-o://c14b27dcc07b66035e6ddb1e051637768d584be3f0dede7f0c2ef62474dd1ce8" gracePeriod=10 Jul 16 00:48:41.234142 ip-10-0-143-200 kubenswrapper[2572]: I0716 00:48:41.234116 2572 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/kuadrant-operator-controller-manager-8684c9ff49-szcrm" Jul 16 00:48:41.346474 ip-10-0-143-200 kubenswrapper[2572]: I0716 00:48:41.346439 2572 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"extensions-socket-volume\" (UniqueName: \"kubernetes.io/empty-dir/b03e9597-45db-4044-bd65-560a261a2d80-extensions-socket-volume\") pod \"b03e9597-45db-4044-bd65-560a261a2d80\" (UID: \"b03e9597-45db-4044-bd65-560a261a2d80\") " Jul 16 00:48:41.346661 ip-10-0-143-200 kubenswrapper[2572]: I0716 00:48:41.346552 2572 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-7m8s7\" (UniqueName: \"kubernetes.io/projected/b03e9597-45db-4044-bd65-560a261a2d80-kube-api-access-7m8s7\") pod \"b03e9597-45db-4044-bd65-560a261a2d80\" (UID: \"b03e9597-45db-4044-bd65-560a261a2d80\") " Jul 16 00:48:41.346955 ip-10-0-143-200 kubenswrapper[2572]: I0716 00:48:41.346925 2572 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/b03e9597-45db-4044-bd65-560a261a2d80-extensions-socket-volume" (OuterVolumeSpecName: "extensions-socket-volume") pod "b03e9597-45db-4044-bd65-560a261a2d80" (UID: "b03e9597-45db-4044-bd65-560a261a2d80"). InnerVolumeSpecName "extensions-socket-volume". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Jul 16 00:48:41.348674 ip-10-0-143-200 kubenswrapper[2572]: I0716 00:48:41.348646 2572 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/b03e9597-45db-4044-bd65-560a261a2d80-kube-api-access-7m8s7" (OuterVolumeSpecName: "kube-api-access-7m8s7") pod "b03e9597-45db-4044-bd65-560a261a2d80" (UID: "b03e9597-45db-4044-bd65-560a261a2d80"). InnerVolumeSpecName "kube-api-access-7m8s7". PluginName "kubernetes.io/projected", VolumeGIDValue "" Jul 16 00:48:41.447606 ip-10-0-143-200 kubenswrapper[2572]: I0716 00:48:41.447528 2572 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-7m8s7\" (UniqueName: \"kubernetes.io/projected/b03e9597-45db-4044-bd65-560a261a2d80-kube-api-access-7m8s7\") on node \"ip-10-0-143-200.ec2.internal\" DevicePath \"\"" Jul 16 00:48:41.447606 ip-10-0-143-200 kubenswrapper[2572]: I0716 00:48:41.447553 2572 reconciler_common.go:299] "Volume detached for volume \"extensions-socket-volume\" (UniqueName: \"kubernetes.io/empty-dir/b03e9597-45db-4044-bd65-560a261a2d80-extensions-socket-volume\") on node \"ip-10-0-143-200.ec2.internal\" DevicePath \"\"" Jul 16 00:48:41.953306 ip-10-0-143-200 kubenswrapper[2572]: I0716 00:48:41.953267 2572 generic.go:356] "Generic (PLEG): container finished" podID="b03e9597-45db-4044-bd65-560a261a2d80" containerID="c14b27dcc07b66035e6ddb1e051637768d584be3f0dede7f0c2ef62474dd1ce8" exitCode=0 Jul 16 00:48:41.953776 ip-10-0-143-200 kubenswrapper[2572]: I0716 00:48:41.953309 2572 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/kuadrant-operator-controller-manager-8684c9ff49-szcrm" Jul 16 00:48:41.953776 ip-10-0-143-200 kubenswrapper[2572]: I0716 00:48:41.953347 2572 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/kuadrant-operator-controller-manager-8684c9ff49-szcrm" event={"ID":"b03e9597-45db-4044-bd65-560a261a2d80","Type":"ContainerDied","Data":"c14b27dcc07b66035e6ddb1e051637768d584be3f0dede7f0c2ef62474dd1ce8"} Jul 16 00:48:41.953776 ip-10-0-143-200 kubenswrapper[2572]: I0716 00:48:41.953397 2572 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/kuadrant-operator-controller-manager-8684c9ff49-szcrm" event={"ID":"b03e9597-45db-4044-bd65-560a261a2d80","Type":"ContainerDied","Data":"8c74707f86e331c1422805085c212dbdd0b553d713afede7d1dbd73c5599a842"} Jul 16 00:48:41.953776 ip-10-0-143-200 kubenswrapper[2572]: I0716 00:48:41.953420 2572 scope.go:117] "RemoveContainer" containerID="c14b27dcc07b66035e6ddb1e051637768d584be3f0dede7f0c2ef62474dd1ce8" Jul 16 00:48:41.962770 ip-10-0-143-200 kubenswrapper[2572]: I0716 00:48:41.962748 2572 scope.go:117] "RemoveContainer" containerID="c14b27dcc07b66035e6ddb1e051637768d584be3f0dede7f0c2ef62474dd1ce8" Jul 16 00:48:41.963032 ip-10-0-143-200 kubenswrapper[2572]: E0716 00:48:41.963014 2572 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"c14b27dcc07b66035e6ddb1e051637768d584be3f0dede7f0c2ef62474dd1ce8\": container with ID starting with c14b27dcc07b66035e6ddb1e051637768d584be3f0dede7f0c2ef62474dd1ce8 not found: ID does not exist" containerID="c14b27dcc07b66035e6ddb1e051637768d584be3f0dede7f0c2ef62474dd1ce8" Jul 16 00:48:41.963079 ip-10-0-143-200 kubenswrapper[2572]: I0716 00:48:41.963043 2572 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"c14b27dcc07b66035e6ddb1e051637768d584be3f0dede7f0c2ef62474dd1ce8"} err="failed to get container status \"c14b27dcc07b66035e6ddb1e051637768d584be3f0dede7f0c2ef62474dd1ce8\": rpc error: code = NotFound desc = could not find container \"c14b27dcc07b66035e6ddb1e051637768d584be3f0dede7f0c2ef62474dd1ce8\": container with ID starting with c14b27dcc07b66035e6ddb1e051637768d584be3f0dede7f0c2ef62474dd1ce8 not found: ID does not exist" Jul 16 00:48:41.978240 ip-10-0-143-200 kubenswrapper[2572]: I0716 00:48:41.978210 2572 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["kuadrant-system/kuadrant-operator-controller-manager-8684c9ff49-szcrm"] Jul 16 00:48:41.985494 ip-10-0-143-200 kubenswrapper[2572]: I0716 00:48:41.985467 2572 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["kuadrant-system/kuadrant-operator-controller-manager-8684c9ff49-szcrm"] Jul 16 00:48:43.070741 ip-10-0-143-200 kubenswrapper[2572]: I0716 00:48:43.070706 2572 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9gnmq_3a7e030d-fad8-480d-9dc6-fe5057048cf9/konnectivity-agent/0.log" limit=1024 Jul 16 00:48:43.176139 ip-10-0-143-200 kubenswrapper[2572]: I0716 00:48:43.176100 2572 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9gnmq_3a7e030d-fad8-480d-9dc6-fe5057048cf9/konnectivity-agent/0.log" limit=1024 Jul 16 00:48:43.450450 ip-10-0-143-200 kubenswrapper[2572]: I0716 00:48:43.450422 2572 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="b03e9597-45db-4044-bd65-560a261a2d80" path="/var/lib/kubelet/pods/b03e9597-45db-4044-bd65-560a261a2d80/volumes" Jul 16 00:48:51.894787 ip-10-0-143-200 kubenswrapper[2572]: I0716 00:48:51.894696 2572 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9gnmq_3a7e030d-fad8-480d-9dc6-fe5057048cf9/konnectivity-agent/0.log" limit=1024 Jul 16 00:48:51.995920 ip-10-0-143-200 kubenswrapper[2572]: I0716 00:48:51.995876 2572 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9gnmq_3a7e030d-fad8-480d-9dc6-fe5057048cf9/konnectivity-agent/0.log" limit=1024 Jul 16 00:48:54.500038 ip-10-0-143-200 kubenswrapper[2572]: I0716 00:48:54.500003 2572 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9gnmq_3a7e030d-fad8-480d-9dc6-fe5057048cf9/konnectivity-agent/0.log" limit=1024 Jul 16 00:48:54.597523 ip-10-0-143-200 kubenswrapper[2572]: I0716 00:48:54.597486 2572 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9gnmq_3a7e030d-fad8-480d-9dc6-fe5057048cf9/konnectivity-agent/0.log" limit=1024 Jul 16 00:48:57.489924 ip-10-0-143-200 kubenswrapper[2572]: I0716 00:48:57.489890 2572 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-ingress/maas-default-gateway-openshift-default-56654dcf7-gr57g"] Jul 16 00:48:57.490321 ip-10-0-143-200 kubenswrapper[2572]: I0716 00:48:57.490271 2572 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="b03e9597-45db-4044-bd65-560a261a2d80" containerName="manager" Jul 16 00:48:57.490321 ip-10-0-143-200 kubenswrapper[2572]: I0716 00:48:57.490284 2572 state_mem.go:107] "Deleted CPUSet assignment" podUID="b03e9597-45db-4044-bd65-560a261a2d80" containerName="manager" Jul 16 00:48:57.490398 ip-10-0-143-200 kubenswrapper[2572]: I0716 00:48:57.490343 2572 memory_manager.go:368] "RemoveStaleState removing state" podUID="b03e9597-45db-4044-bd65-560a261a2d80" containerName="manager" Jul 16 00:48:57.492925 ip-10-0-143-200 kubenswrapper[2572]: I0716 00:48:57.492907 2572 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/maas-default-gateway-openshift-default-56654dcf7-gr57g" Jul 16 00:48:57.496127 ip-10-0-143-200 kubenswrapper[2572]: I0716 00:48:57.496108 2572 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress\"/\"maas-default-gateway-openshift-default-dockercfg-zhsn6\"" Jul 16 00:48:57.505019 ip-10-0-143-200 kubenswrapper[2572]: I0716 00:48:57.504997 2572 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/maas-default-gateway-openshift-default-56654dcf7-gr57g"] Jul 16 00:48:57.521149 ip-10-0-143-200 kubenswrapper[2572]: I0716 00:48:57.521119 2572 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9gnmq_3a7e030d-fad8-480d-9dc6-fe5057048cf9/konnectivity-agent/0.log" limit=1024 Jul 16 00:48:57.588361 ip-10-0-143-200 kubenswrapper[2572]: I0716 00:48:57.588322 2572 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/3fc0bdda-3400-495e-b694-4e085c84fd49-istio-data\") pod \"maas-default-gateway-openshift-default-56654dcf7-gr57g\" (UID: \"3fc0bdda-3400-495e-b694-4e085c84fd49\") " pod="openshift-ingress/maas-default-gateway-openshift-default-56654dcf7-gr57g" Jul 16 00:48:57.588554 ip-10-0-143-200 kubenswrapper[2572]: I0716 00:48:57.588395 2572 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/3fc0bdda-3400-495e-b694-4e085c84fd49-istio-podinfo\") pod \"maas-default-gateway-openshift-default-56654dcf7-gr57g\" (UID: \"3fc0bdda-3400-495e-b694-4e085c84fd49\") " pod="openshift-ingress/maas-default-gateway-openshift-default-56654dcf7-gr57g" Jul 16 00:48:57.588554 ip-10-0-143-200 kubenswrapper[2572]: I0716 00:48:57.588425 2572 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/3fc0bdda-3400-495e-b694-4e085c84fd49-istio-token\") pod \"maas-default-gateway-openshift-default-56654dcf7-gr57g\" (UID: \"3fc0bdda-3400-495e-b694-4e085c84fd49\") " pod="openshift-ingress/maas-default-gateway-openshift-default-56654dcf7-gr57g" Jul 16 00:48:57.588554 ip-10-0-143-200 kubenswrapper[2572]: I0716 00:48:57.588445 2572 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/3fc0bdda-3400-495e-b694-4e085c84fd49-workload-socket\") pod \"maas-default-gateway-openshift-default-56654dcf7-gr57g\" (UID: \"3fc0bdda-3400-495e-b694-4e085c84fd49\") " pod="openshift-ingress/maas-default-gateway-openshift-default-56654dcf7-gr57g" Jul 16 00:48:57.588554 ip-10-0-143-200 kubenswrapper[2572]: I0716 00:48:57.588499 2572 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/3fc0bdda-3400-495e-b694-4e085c84fd49-istio-envoy\") pod \"maas-default-gateway-openshift-default-56654dcf7-gr57g\" (UID: \"3fc0bdda-3400-495e-b694-4e085c84fd49\") " pod="openshift-ingress/maas-default-gateway-openshift-default-56654dcf7-gr57g" Jul 16 00:48:57.588554 ip-10-0-143-200 kubenswrapper[2572]: I0716 00:48:57.588521 2572 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/3fc0bdda-3400-495e-b694-4e085c84fd49-istiod-ca-cert\") pod \"maas-default-gateway-openshift-default-56654dcf7-gr57g\" (UID: \"3fc0bdda-3400-495e-b694-4e085c84fd49\") " pod="openshift-ingress/maas-default-gateway-openshift-default-56654dcf7-gr57g" Jul 16 00:48:57.588862 ip-10-0-143-200 kubenswrapper[2572]: I0716 00:48:57.588575 2572 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-tv8v4\" (UniqueName: \"kubernetes.io/projected/3fc0bdda-3400-495e-b694-4e085c84fd49-kube-api-access-tv8v4\") pod \"maas-default-gateway-openshift-default-56654dcf7-gr57g\" (UID: \"3fc0bdda-3400-495e-b694-4e085c84fd49\") " pod="openshift-ingress/maas-default-gateway-openshift-default-56654dcf7-gr57g" Jul 16 00:48:57.588862 ip-10-0-143-200 kubenswrapper[2572]: I0716 00:48:57.588664 2572 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/3fc0bdda-3400-495e-b694-4e085c84fd49-credential-socket\") pod \"maas-default-gateway-openshift-default-56654dcf7-gr57g\" (UID: \"3fc0bdda-3400-495e-b694-4e085c84fd49\") " pod="openshift-ingress/maas-default-gateway-openshift-default-56654dcf7-gr57g" Jul 16 00:48:57.588862 ip-10-0-143-200 kubenswrapper[2572]: I0716 00:48:57.588693 2572 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/3fc0bdda-3400-495e-b694-4e085c84fd49-workload-certs\") pod \"maas-default-gateway-openshift-default-56654dcf7-gr57g\" (UID: \"3fc0bdda-3400-495e-b694-4e085c84fd49\") " pod="openshift-ingress/maas-default-gateway-openshift-default-56654dcf7-gr57g" Jul 16 00:48:57.677964 ip-10-0-143-200 kubenswrapper[2572]: I0716 00:48:57.677927 2572 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9gnmq_3a7e030d-fad8-480d-9dc6-fe5057048cf9/konnectivity-agent/0.log" limit=1024 Jul 16 00:48:57.689995 ip-10-0-143-200 kubenswrapper[2572]: I0716 00:48:57.689968 2572 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-tv8v4\" (UniqueName: \"kubernetes.io/projected/3fc0bdda-3400-495e-b694-4e085c84fd49-kube-api-access-tv8v4\") pod \"maas-default-gateway-openshift-default-56654dcf7-gr57g\" (UID: \"3fc0bdda-3400-495e-b694-4e085c84fd49\") " pod="openshift-ingress/maas-default-gateway-openshift-default-56654dcf7-gr57g" Jul 16 00:48:57.690146 ip-10-0-143-200 kubenswrapper[2572]: I0716 00:48:57.690003 2572 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/3fc0bdda-3400-495e-b694-4e085c84fd49-credential-socket\") pod \"maas-default-gateway-openshift-default-56654dcf7-gr57g\" (UID: \"3fc0bdda-3400-495e-b694-4e085c84fd49\") " pod="openshift-ingress/maas-default-gateway-openshift-default-56654dcf7-gr57g" Jul 16 00:48:57.690146 ip-10-0-143-200 kubenswrapper[2572]: I0716 00:48:57.690022 2572 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/3fc0bdda-3400-495e-b694-4e085c84fd49-workload-certs\") pod \"maas-default-gateway-openshift-default-56654dcf7-gr57g\" (UID: \"3fc0bdda-3400-495e-b694-4e085c84fd49\") " pod="openshift-ingress/maas-default-gateway-openshift-default-56654dcf7-gr57g" Jul 16 00:48:57.690146 ip-10-0-143-200 kubenswrapper[2572]: I0716 00:48:57.690058 2572 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/3fc0bdda-3400-495e-b694-4e085c84fd49-istio-data\") pod \"maas-default-gateway-openshift-default-56654dcf7-gr57g\" (UID: \"3fc0bdda-3400-495e-b694-4e085c84fd49\") " pod="openshift-ingress/maas-default-gateway-openshift-default-56654dcf7-gr57g" Jul 16 00:48:57.690146 ip-10-0-143-200 kubenswrapper[2572]: I0716 00:48:57.690095 2572 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/3fc0bdda-3400-495e-b694-4e085c84fd49-istio-podinfo\") pod \"maas-default-gateway-openshift-default-56654dcf7-gr57g\" (UID: \"3fc0bdda-3400-495e-b694-4e085c84fd49\") " pod="openshift-ingress/maas-default-gateway-openshift-default-56654dcf7-gr57g" Jul 16 00:48:57.690146 ip-10-0-143-200 kubenswrapper[2572]: I0716 00:48:57.690114 2572 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/3fc0bdda-3400-495e-b694-4e085c84fd49-istio-token\") pod \"maas-default-gateway-openshift-default-56654dcf7-gr57g\" (UID: \"3fc0bdda-3400-495e-b694-4e085c84fd49\") " pod="openshift-ingress/maas-default-gateway-openshift-default-56654dcf7-gr57g" Jul 16 00:48:57.690146 ip-10-0-143-200 kubenswrapper[2572]: I0716 00:48:57.690133 2572 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/3fc0bdda-3400-495e-b694-4e085c84fd49-workload-socket\") pod \"maas-default-gateway-openshift-default-56654dcf7-gr57g\" (UID: \"3fc0bdda-3400-495e-b694-4e085c84fd49\") " pod="openshift-ingress/maas-default-gateway-openshift-default-56654dcf7-gr57g" Jul 16 00:48:57.690455 ip-10-0-143-200 kubenswrapper[2572]: I0716 00:48:57.690171 2572 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/3fc0bdda-3400-495e-b694-4e085c84fd49-istio-envoy\") pod \"maas-default-gateway-openshift-default-56654dcf7-gr57g\" (UID: \"3fc0bdda-3400-495e-b694-4e085c84fd49\") " pod="openshift-ingress/maas-default-gateway-openshift-default-56654dcf7-gr57g" Jul 16 00:48:57.690455 ip-10-0-143-200 kubenswrapper[2572]: I0716 00:48:57.690195 2572 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/3fc0bdda-3400-495e-b694-4e085c84fd49-istiod-ca-cert\") pod \"maas-default-gateway-openshift-default-56654dcf7-gr57g\" (UID: \"3fc0bdda-3400-495e-b694-4e085c84fd49\") " pod="openshift-ingress/maas-default-gateway-openshift-default-56654dcf7-gr57g" Jul 16 00:48:57.690455 ip-10-0-143-200 kubenswrapper[2572]: I0716 00:48:57.690443 2572 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/3fc0bdda-3400-495e-b694-4e085c84fd49-credential-socket\") pod \"maas-default-gateway-openshift-default-56654dcf7-gr57g\" (UID: \"3fc0bdda-3400-495e-b694-4e085c84fd49\") " pod="openshift-ingress/maas-default-gateway-openshift-default-56654dcf7-gr57g" Jul 16 00:48:57.690655 ip-10-0-143-200 kubenswrapper[2572]: I0716 00:48:57.690483 2572 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/3fc0bdda-3400-495e-b694-4e085c84fd49-workload-certs\") pod \"maas-default-gateway-openshift-default-56654dcf7-gr57g\" (UID: \"3fc0bdda-3400-495e-b694-4e085c84fd49\") " pod="openshift-ingress/maas-default-gateway-openshift-default-56654dcf7-gr57g" Jul 16 00:48:57.690655 ip-10-0-143-200 kubenswrapper[2572]: I0716 00:48:57.690529 2572 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/3fc0bdda-3400-495e-b694-4e085c84fd49-istio-data\") pod \"maas-default-gateway-openshift-default-56654dcf7-gr57g\" (UID: \"3fc0bdda-3400-495e-b694-4e085c84fd49\") " pod="openshift-ingress/maas-default-gateway-openshift-default-56654dcf7-gr57g" Jul 16 00:48:57.690655 ip-10-0-143-200 kubenswrapper[2572]: I0716 00:48:57.690558 2572 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/3fc0bdda-3400-495e-b694-4e085c84fd49-workload-socket\") pod \"maas-default-gateway-openshift-default-56654dcf7-gr57g\" (UID: \"3fc0bdda-3400-495e-b694-4e085c84fd49\") " pod="openshift-ingress/maas-default-gateway-openshift-default-56654dcf7-gr57g" Jul 16 00:48:57.690937 ip-10-0-143-200 kubenswrapper[2572]: I0716 00:48:57.690917 2572 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/3fc0bdda-3400-495e-b694-4e085c84fd49-istiod-ca-cert\") pod \"maas-default-gateway-openshift-default-56654dcf7-gr57g\" (UID: \"3fc0bdda-3400-495e-b694-4e085c84fd49\") " pod="openshift-ingress/maas-default-gateway-openshift-default-56654dcf7-gr57g" Jul 16 00:48:57.692459 ip-10-0-143-200 kubenswrapper[2572]: I0716 00:48:57.692439 2572 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/3fc0bdda-3400-495e-b694-4e085c84fd49-istio-envoy\") pod \"maas-default-gateway-openshift-default-56654dcf7-gr57g\" (UID: \"3fc0bdda-3400-495e-b694-4e085c84fd49\") " pod="openshift-ingress/maas-default-gateway-openshift-default-56654dcf7-gr57g" Jul 16 00:48:57.692763 ip-10-0-143-200 kubenswrapper[2572]: I0716 00:48:57.692744 2572 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/3fc0bdda-3400-495e-b694-4e085c84fd49-istio-podinfo\") pod \"maas-default-gateway-openshift-default-56654dcf7-gr57g\" (UID: \"3fc0bdda-3400-495e-b694-4e085c84fd49\") " pod="openshift-ingress/maas-default-gateway-openshift-default-56654dcf7-gr57g" Jul 16 00:48:57.702080 ip-10-0-143-200 kubenswrapper[2572]: I0716 00:48:57.702047 2572 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/3fc0bdda-3400-495e-b694-4e085c84fd49-istio-token\") pod \"maas-default-gateway-openshift-default-56654dcf7-gr57g\" (UID: \"3fc0bdda-3400-495e-b694-4e085c84fd49\") " pod="openshift-ingress/maas-default-gateway-openshift-default-56654dcf7-gr57g" Jul 16 00:48:57.702234 ip-10-0-143-200 kubenswrapper[2572]: I0716 00:48:57.702216 2572 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-tv8v4\" (UniqueName: \"kubernetes.io/projected/3fc0bdda-3400-495e-b694-4e085c84fd49-kube-api-access-tv8v4\") pod \"maas-default-gateway-openshift-default-56654dcf7-gr57g\" (UID: \"3fc0bdda-3400-495e-b694-4e085c84fd49\") " pod="openshift-ingress/maas-default-gateway-openshift-default-56654dcf7-gr57g" Jul 16 00:48:57.780583 ip-10-0-143-200 kubenswrapper[2572]: I0716 00:48:57.780502 2572 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9gnmq_3a7e030d-fad8-480d-9dc6-fe5057048cf9/konnectivity-agent/0.log" limit=1024 Jul 16 00:48:57.805308 ip-10-0-143-200 kubenswrapper[2572]: I0716 00:48:57.805282 2572 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/maas-default-gateway-openshift-default-56654dcf7-gr57g" Jul 16 00:48:57.948392 ip-10-0-143-200 kubenswrapper[2572]: I0716 00:48:57.948367 2572 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/maas-default-gateway-openshift-default-56654dcf7-gr57g"] Jul 16 00:48:57.949892 ip-10-0-143-200 kubenswrapper[2572]: W0716 00:48:57.949865 2572 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod3fc0bdda_3400_495e_b694_4e085c84fd49.slice/crio-4ba0e9fcb22ee4b82ccf8df08db27d2a2d281a8c4fa9b5f3812a18e7b7fd9086 WatchSource:0}: Error finding container 4ba0e9fcb22ee4b82ccf8df08db27d2a2d281a8c4fa9b5f3812a18e7b7fd9086: Status 404 returned error can't find the container with id 4ba0e9fcb22ee4b82ccf8df08db27d2a2d281a8c4fa9b5f3812a18e7b7fd9086 Jul 16 00:48:57.952034 ip-10-0-143-200 kubenswrapper[2572]: I0716 00:48:57.952006 2572 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236228Ki","pods":"250"} Jul 16 00:48:57.952097 ip-10-0-143-200 kubenswrapper[2572]: I0716 00:48:57.952068 2572 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236228Ki","pods":"250"} Jul 16 00:48:57.952140 ip-10-0-143-200 kubenswrapper[2572]: I0716 00:48:57.952097 2572 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236228Ki","pods":"250"} Jul 16 00:48:58.029548 ip-10-0-143-200 kubenswrapper[2572]: I0716 00:48:58.029500 2572 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/maas-default-gateway-openshift-default-56654dcf7-gr57g" event={"ID":"3fc0bdda-3400-495e-b694-4e085c84fd49","Type":"ContainerStarted","Data":"a8d213e2798005b4d5a4a6fc7b66c3dab9558ccdf9536ae18674c1a1f81a1158"} Jul 16 00:48:58.029700 ip-10-0-143-200 kubenswrapper[2572]: I0716 00:48:58.029585 2572 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/maas-default-gateway-openshift-default-56654dcf7-gr57g" event={"ID":"3fc0bdda-3400-495e-b694-4e085c84fd49","Type":"ContainerStarted","Data":"4ba0e9fcb22ee4b82ccf8df08db27d2a2d281a8c4fa9b5f3812a18e7b7fd9086"} Jul 16 00:48:58.060964 ip-10-0-143-200 kubenswrapper[2572]: I0716 00:48:58.060856 2572 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-ingress/maas-default-gateway-openshift-default-56654dcf7-gr57g" podStartSLOduration=1.060840101 podStartE2EDuration="1.060840101s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-16 00:48:57 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-16 00:48:58.056828341 +0000 UTC m=+543.165583029" watchObservedRunningTime="2026-07-16 00:48:58.060840101 +0000 UTC m=+543.169594776" Jul 16 00:48:58.806458 ip-10-0-143-200 kubenswrapper[2572]: I0716 00:48:58.806416 2572 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="openshift-ingress/maas-default-gateway-openshift-default-56654dcf7-gr57g" Jul 16 00:48:59.122399 ip-10-0-143-200 kubenswrapper[2572]: I0716 00:48:59.122317 2572 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9gnmq_3a7e030d-fad8-480d-9dc6-fe5057048cf9/konnectivity-agent/0.log" limit=1024 Jul 16 00:48:59.146474 ip-10-0-143-200 kubenswrapper[2572]: I0716 00:48:59.146449 2572 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["kuadrant-system/limitador-limitador-64c94fd64f-x87mv"] Jul 16 00:48:59.148933 ip-10-0-143-200 kubenswrapper[2572]: I0716 00:48:59.148910 2572 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/limitador-limitador-64c94fd64f-x87mv" Jul 16 00:48:59.153127 ip-10-0-143-200 kubenswrapper[2572]: I0716 00:48:59.152921 2572 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"kuadrant-system\"/\"default-dockercfg-tcss8\"" Jul 16 00:48:59.153127 ip-10-0-143-200 kubenswrapper[2572]: I0716 00:48:59.152930 2572 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"kuadrant-system\"/\"limitador-limits-config-limitador\"" Jul 16 00:48:59.168208 ip-10-0-143-200 kubenswrapper[2572]: I0716 00:48:59.168184 2572 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-64c94fd64f-x87mv"] Jul 16 00:48:59.210650 ip-10-0-143-200 kubenswrapper[2572]: I0716 00:48:59.210578 2572 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9gnmq_3a7e030d-fad8-480d-9dc6-fe5057048cf9/konnectivity-agent/0.log" limit=1024 Jul 16 00:48:59.220281 ip-10-0-143-200 kubenswrapper[2572]: I0716 00:48:59.220255 2572 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-64c94fd64f-x87mv"] Jul 16 00:48:59.305399 ip-10-0-143-200 kubenswrapper[2572]: I0716 00:48:59.305363 2572 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"config-file\" (UniqueName: \"kubernetes.io/configmap/965c9589-5d41-4543-a625-9bc94f822ad7-config-file\") pod \"limitador-limitador-64c94fd64f-x87mv\" (UID: \"965c9589-5d41-4543-a625-9bc94f822ad7\") " pod="kuadrant-system/limitador-limitador-64c94fd64f-x87mv" Jul 16 00:48:59.305593 ip-10-0-143-200 kubenswrapper[2572]: I0716 00:48:59.305433 2572 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-k8pgm\" (UniqueName: \"kubernetes.io/projected/965c9589-5d41-4543-a625-9bc94f822ad7-kube-api-access-k8pgm\") pod \"limitador-limitador-64c94fd64f-x87mv\" (UID: \"965c9589-5d41-4543-a625-9bc94f822ad7\") " pod="kuadrant-system/limitador-limitador-64c94fd64f-x87mv" Jul 16 00:48:59.311527 ip-10-0-143-200 kubenswrapper[2572]: I0716 00:48:59.311496 2572 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9gnmq_3a7e030d-fad8-480d-9dc6-fe5057048cf9/konnectivity-agent/0.log" limit=1024 Jul 16 00:48:59.406804 ip-10-0-143-200 kubenswrapper[2572]: I0716 00:48:59.406712 2572 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"config-file\" (UniqueName: \"kubernetes.io/configmap/965c9589-5d41-4543-a625-9bc94f822ad7-config-file\") pod \"limitador-limitador-64c94fd64f-x87mv\" (UID: \"965c9589-5d41-4543-a625-9bc94f822ad7\") " pod="kuadrant-system/limitador-limitador-64c94fd64f-x87mv" Jul 16 00:48:59.406804 ip-10-0-143-200 kubenswrapper[2572]: I0716 00:48:59.406762 2572 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-k8pgm\" (UniqueName: \"kubernetes.io/projected/965c9589-5d41-4543-a625-9bc94f822ad7-kube-api-access-k8pgm\") pod \"limitador-limitador-64c94fd64f-x87mv\" (UID: \"965c9589-5d41-4543-a625-9bc94f822ad7\") " pod="kuadrant-system/limitador-limitador-64c94fd64f-x87mv" Jul 16 00:48:59.407352 ip-10-0-143-200 kubenswrapper[2572]: I0716 00:48:59.407329 2572 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"config-file\" (UniqueName: \"kubernetes.io/configmap/965c9589-5d41-4543-a625-9bc94f822ad7-config-file\") pod \"limitador-limitador-64c94fd64f-x87mv\" (UID: \"965c9589-5d41-4543-a625-9bc94f822ad7\") " pod="kuadrant-system/limitador-limitador-64c94fd64f-x87mv" Jul 16 00:48:59.416157 ip-10-0-143-200 kubenswrapper[2572]: I0716 00:48:59.416124 2572 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-k8pgm\" (UniqueName: \"kubernetes.io/projected/965c9589-5d41-4543-a625-9bc94f822ad7-kube-api-access-k8pgm\") pod \"limitador-limitador-64c94fd64f-x87mv\" (UID: \"965c9589-5d41-4543-a625-9bc94f822ad7\") " pod="kuadrant-system/limitador-limitador-64c94fd64f-x87mv" Jul 16 00:48:59.463564 ip-10-0-143-200 kubenswrapper[2572]: I0716 00:48:59.463531 2572 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/limitador-limitador-64c94fd64f-x87mv" Jul 16 00:48:59.598978 ip-10-0-143-200 kubenswrapper[2572]: I0716 00:48:59.598952 2572 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-64c94fd64f-x87mv"] Jul 16 00:48:59.600932 ip-10-0-143-200 kubenswrapper[2572]: W0716 00:48:59.600904 2572 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod965c9589_5d41_4543_a625_9bc94f822ad7.slice/crio-dc3c96a89f90a1e050937380820fd25f1442c12d8f85e660a161b9ef8a6ea647 WatchSource:0}: Error finding container dc3c96a89f90a1e050937380820fd25f1442c12d8f85e660a161b9ef8a6ea647: Status 404 returned error can't find the container with id dc3c96a89f90a1e050937380820fd25f1442c12d8f85e660a161b9ef8a6ea647 Jul 16 00:48:59.811400 ip-10-0-143-200 kubenswrapper[2572]: I0716 00:48:59.811369 2572 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="openshift-ingress/maas-default-gateway-openshift-default-56654dcf7-gr57g" Jul 16 00:49:00.041403 ip-10-0-143-200 kubenswrapper[2572]: I0716 00:49:00.041368 2572 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/limitador-limitador-64c94fd64f-x87mv" event={"ID":"965c9589-5d41-4543-a625-9bc94f822ad7","Type":"ContainerStarted","Data":"dc3c96a89f90a1e050937380820fd25f1442c12d8f85e660a161b9ef8a6ea647"} Jul 16 00:49:00.041754 ip-10-0-143-200 kubenswrapper[2572]: I0716 00:49:00.041732 2572 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/maas-default-gateway-openshift-default-56654dcf7-gr57g" Jul 16 00:49:00.042766 ip-10-0-143-200 kubenswrapper[2572]: I0716 00:49:00.042746 2572 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/maas-default-gateway-openshift-default-56654dcf7-gr57g" Jul 16 00:49:00.090280 ip-10-0-143-200 kubenswrapper[2572]: I0716 00:49:00.090181 2572 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9gnmq_3a7e030d-fad8-480d-9dc6-fe5057048cf9/konnectivity-agent/0.log" limit=1024 Jul 16 00:49:00.181053 ip-10-0-143-200 kubenswrapper[2572]: I0716 00:49:00.181019 2572 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9gnmq_3a7e030d-fad8-480d-9dc6-fe5057048cf9/konnectivity-agent/0.log" limit=1024 Jul 16 00:49:01.099853 ip-10-0-143-200 kubenswrapper[2572]: I0716 00:49:01.099813 2572 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9gnmq_3a7e030d-fad8-480d-9dc6-fe5057048cf9/konnectivity-agent/0.log" limit=1024 Jul 16 00:49:01.199811 ip-10-0-143-200 kubenswrapper[2572]: I0716 00:49:01.199758 2572 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9gnmq_3a7e030d-fad8-480d-9dc6-fe5057048cf9/konnectivity-agent/0.log" limit=1024 Jul 16 00:49:03.057706 ip-10-0-143-200 kubenswrapper[2572]: I0716 00:49:03.057665 2572 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/limitador-limitador-64c94fd64f-x87mv" event={"ID":"965c9589-5d41-4543-a625-9bc94f822ad7","Type":"ContainerStarted","Data":"c24af316572e532b729e8c46bfd7ebdd44893a553dd01837dbb6a9ed7d1895df"} Jul 16 00:49:03.058110 ip-10-0-143-200 kubenswrapper[2572]: I0716 00:49:03.057844 2572 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="kuadrant-system/limitador-limitador-64c94fd64f-x87mv" Jul 16 00:49:03.075984 ip-10-0-143-200 kubenswrapper[2572]: I0716 00:49:03.075933 2572 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="kuadrant-system/limitador-limitador-64c94fd64f-x87mv" podStartSLOduration=1.370683398 podStartE2EDuration="4.075917249s" totalImagesPullingTime="2.705233851s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-16 00:48:59 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-16 00:49:03.074624635 +0000 UTC m=+548.183379310" watchObservedRunningTime="2026-07-16 00:49:03.075917249 +0000 UTC m=+548.184671924" Jul 16 00:49:09.730428 ip-10-0-143-200 kubenswrapper[2572]: I0716 00:49:09.730385 2572 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9gnmq_3a7e030d-fad8-480d-9dc6-fe5057048cf9/konnectivity-agent/0.log" limit=1024 Jul 16 00:49:09.827915 ip-10-0-143-200 kubenswrapper[2572]: I0716 00:49:09.827881 2572 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9gnmq_3a7e030d-fad8-480d-9dc6-fe5057048cf9/konnectivity-agent/0.log" limit=1024 Jul 16 00:49:14.063374 ip-10-0-143-200 kubenswrapper[2572]: I0716 00:49:14.063344 2572 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="kuadrant-system/limitador-limitador-64c94fd64f-x87mv" Jul 16 00:49:16.333412 ip-10-0-143-200 kubenswrapper[2572]: I0716 00:49:16.333377 2572 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["kuadrant-system/limitador-limitador-64c94fd64f-x87mv"] Jul 16 00:49:16.333898 ip-10-0-143-200 kubenswrapper[2572]: I0716 00:49:16.333550 2572 kuberuntime_container.go:873] "Killing container with a grace period" pod="kuadrant-system/limitador-limitador-64c94fd64f-x87mv" podUID="965c9589-5d41-4543-a625-9bc94f822ad7" containerName="limitador" containerID="cri-o://c24af316572e532b729e8c46bfd7ebdd44893a553dd01837dbb6a9ed7d1895df" gracePeriod=30 Jul 16 00:49:17.124806 ip-10-0-143-200 kubenswrapper[2572]: I0716 00:49:17.124772 2572 generic.go:356] "Generic (PLEG): container finished" podID="965c9589-5d41-4543-a625-9bc94f822ad7" containerID="c24af316572e532b729e8c46bfd7ebdd44893a553dd01837dbb6a9ed7d1895df" exitCode=0 Jul 16 00:49:17.124995 ip-10-0-143-200 kubenswrapper[2572]: I0716 00:49:17.124847 2572 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/limitador-limitador-64c94fd64f-x87mv" event={"ID":"965c9589-5d41-4543-a625-9bc94f822ad7","Type":"ContainerDied","Data":"c24af316572e532b729e8c46bfd7ebdd44893a553dd01837dbb6a9ed7d1895df"} Jul 16 00:49:17.282963 ip-10-0-143-200 kubenswrapper[2572]: I0716 00:49:17.282937 2572 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/limitador-limitador-64c94fd64f-x87mv" Jul 16 00:49:17.363364 ip-10-0-143-200 kubenswrapper[2572]: I0716 00:49:17.363330 2572 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-k8pgm\" (UniqueName: \"kubernetes.io/projected/965c9589-5d41-4543-a625-9bc94f822ad7-kube-api-access-k8pgm\") pod \"965c9589-5d41-4543-a625-9bc94f822ad7\" (UID: \"965c9589-5d41-4543-a625-9bc94f822ad7\") " Jul 16 00:49:17.363867 ip-10-0-143-200 kubenswrapper[2572]: I0716 00:49:17.363373 2572 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"config-file\" (UniqueName: \"kubernetes.io/configmap/965c9589-5d41-4543-a625-9bc94f822ad7-config-file\") pod \"965c9589-5d41-4543-a625-9bc94f822ad7\" (UID: \"965c9589-5d41-4543-a625-9bc94f822ad7\") " Jul 16 00:49:17.363867 ip-10-0-143-200 kubenswrapper[2572]: I0716 00:49:17.363839 2572 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/965c9589-5d41-4543-a625-9bc94f822ad7-config-file" (OuterVolumeSpecName: "config-file") pod "965c9589-5d41-4543-a625-9bc94f822ad7" (UID: "965c9589-5d41-4543-a625-9bc94f822ad7"). InnerVolumeSpecName "config-file". PluginName "kubernetes.io/configmap", VolumeGIDValue "" Jul 16 00:49:17.365321 ip-10-0-143-200 kubenswrapper[2572]: I0716 00:49:17.365295 2572 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/965c9589-5d41-4543-a625-9bc94f822ad7-kube-api-access-k8pgm" (OuterVolumeSpecName: "kube-api-access-k8pgm") pod "965c9589-5d41-4543-a625-9bc94f822ad7" (UID: "965c9589-5d41-4543-a625-9bc94f822ad7"). InnerVolumeSpecName "kube-api-access-k8pgm". PluginName "kubernetes.io/projected", VolumeGIDValue "" Jul 16 00:49:17.464609 ip-10-0-143-200 kubenswrapper[2572]: I0716 00:49:17.464576 2572 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-k8pgm\" (UniqueName: \"kubernetes.io/projected/965c9589-5d41-4543-a625-9bc94f822ad7-kube-api-access-k8pgm\") on node \"ip-10-0-143-200.ec2.internal\" DevicePath \"\"" Jul 16 00:49:17.464609 ip-10-0-143-200 kubenswrapper[2572]: I0716 00:49:17.464602 2572 reconciler_common.go:299] "Volume detached for volume \"config-file\" (UniqueName: \"kubernetes.io/configmap/965c9589-5d41-4543-a625-9bc94f822ad7-config-file\") on node \"ip-10-0-143-200.ec2.internal\" DevicePath \"\"" Jul 16 00:49:18.130946 ip-10-0-143-200 kubenswrapper[2572]: I0716 00:49:18.130918 2572 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/limitador-limitador-64c94fd64f-x87mv" Jul 16 00:49:18.130946 ip-10-0-143-200 kubenswrapper[2572]: I0716 00:49:18.130943 2572 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/limitador-limitador-64c94fd64f-x87mv" event={"ID":"965c9589-5d41-4543-a625-9bc94f822ad7","Type":"ContainerDied","Data":"dc3c96a89f90a1e050937380820fd25f1442c12d8f85e660a161b9ef8a6ea647"} Jul 16 00:49:18.131192 ip-10-0-143-200 kubenswrapper[2572]: I0716 00:49:18.130999 2572 scope.go:117] "RemoveContainer" containerID="c24af316572e532b729e8c46bfd7ebdd44893a553dd01837dbb6a9ed7d1895df" Jul 16 00:49:18.151116 ip-10-0-143-200 kubenswrapper[2572]: I0716 00:49:18.151084 2572 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["kuadrant-system/limitador-limitador-64c94fd64f-x87mv"] Jul 16 00:49:18.154413 ip-10-0-143-200 kubenswrapper[2572]: I0716 00:49:18.154389 2572 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["kuadrant-system/limitador-limitador-64c94fd64f-x87mv"] Jul 16 00:49:19.447681 ip-10-0-143-200 kubenswrapper[2572]: I0716 00:49:19.447646 2572 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="965c9589-5d41-4543-a625-9bc94f822ad7" path="/var/lib/kubelet/pods/965c9589-5d41-4543-a625-9bc94f822ad7/volumes" Jul 16 00:49:20.371783 ip-10-0-143-200 kubenswrapper[2572]: I0716 00:49:20.371749 2572 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9gnmq_3a7e030d-fad8-480d-9dc6-fe5057048cf9/konnectivity-agent/0.log" limit=1024 Jul 16 00:49:20.479501 ip-10-0-143-200 kubenswrapper[2572]: I0716 00:49:20.479470 2572 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9gnmq_3a7e030d-fad8-480d-9dc6-fe5057048cf9/konnectivity-agent/0.log" limit=1024 Jul 16 00:49:20.855553 ip-10-0-143-200 kubenswrapper[2572]: I0716 00:49:20.855507 2572 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["odh-ai-gateway-infra/postgres-69fb4594cf-4ldvf"] Jul 16 00:49:20.856013 ip-10-0-143-200 kubenswrapper[2572]: I0716 00:49:20.855992 2572 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="965c9589-5d41-4543-a625-9bc94f822ad7" containerName="limitador" Jul 16 00:49:20.856111 ip-10-0-143-200 kubenswrapper[2572]: I0716 00:49:20.856015 2572 state_mem.go:107] "Deleted CPUSet assignment" podUID="965c9589-5d41-4543-a625-9bc94f822ad7" containerName="limitador" Jul 16 00:49:20.856111 ip-10-0-143-200 kubenswrapper[2572]: I0716 00:49:20.856099 2572 memory_manager.go:368] "RemoveStaleState removing state" podUID="965c9589-5d41-4543-a625-9bc94f822ad7" containerName="limitador" Jul 16 00:49:20.859307 ip-10-0-143-200 kubenswrapper[2572]: I0716 00:49:20.859283 2572 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/postgres-69fb4594cf-4ldvf" Jul 16 00:49:20.862902 ip-10-0-143-200 kubenswrapper[2572]: I0716 00:49:20.862882 2572 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"odh-ai-gateway-infra\"/\"kube-root-ca.crt\"" Jul 16 00:49:20.864096 ip-10-0-143-200 kubenswrapper[2572]: I0716 00:49:20.864066 2572 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"odh-ai-gateway-infra\"/\"postgres-creds\"" Jul 16 00:49:20.864359 ip-10-0-143-200 kubenswrapper[2572]: I0716 00:49:20.864340 2572 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"odh-ai-gateway-infra\"/\"default-dockercfg-zhrh9\"" Jul 16 00:49:20.864679 ip-10-0-143-200 kubenswrapper[2572]: I0716 00:49:20.864659 2572 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"odh-ai-gateway-infra\"/\"openshift-service-ca.crt\"" Jul 16 00:49:20.884813 ip-10-0-143-200 kubenswrapper[2572]: I0716 00:49:20.884784 2572 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/postgres-69fb4594cf-4ldvf"] Jul 16 00:49:20.896258 ip-10-0-143-200 kubenswrapper[2572]: I0716 00:49:20.896232 2572 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-x5z9r\" (UniqueName: \"kubernetes.io/projected/371615ce-ddf1-442f-a3b8-9b00b1d269d5-kube-api-access-x5z9r\") pod \"postgres-69fb4594cf-4ldvf\" (UID: \"371615ce-ddf1-442f-a3b8-9b00b1d269d5\") " pod="odh-ai-gateway-infra/postgres-69fb4594cf-4ldvf" Jul 16 00:49:20.896383 ip-10-0-143-200 kubenswrapper[2572]: I0716 00:49:20.896335 2572 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"data\" (UniqueName: \"kubernetes.io/empty-dir/371615ce-ddf1-442f-a3b8-9b00b1d269d5-data\") pod \"postgres-69fb4594cf-4ldvf\" (UID: \"371615ce-ddf1-442f-a3b8-9b00b1d269d5\") " pod="odh-ai-gateway-infra/postgres-69fb4594cf-4ldvf" Jul 16 00:49:20.945514 ip-10-0-143-200 kubenswrapper[2572]: I0716 00:49:20.945478 2572 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9gnmq_3a7e030d-fad8-480d-9dc6-fe5057048cf9/konnectivity-agent/0.log" limit=1024 Jul 16 00:49:20.997823 ip-10-0-143-200 kubenswrapper[2572]: I0716 00:49:20.997774 2572 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"data\" (UniqueName: \"kubernetes.io/empty-dir/371615ce-ddf1-442f-a3b8-9b00b1d269d5-data\") pod \"postgres-69fb4594cf-4ldvf\" (UID: \"371615ce-ddf1-442f-a3b8-9b00b1d269d5\") " pod="odh-ai-gateway-infra/postgres-69fb4594cf-4ldvf" Jul 16 00:49:20.998008 ip-10-0-143-200 kubenswrapper[2572]: I0716 00:49:20.997866 2572 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-x5z9r\" (UniqueName: \"kubernetes.io/projected/371615ce-ddf1-442f-a3b8-9b00b1d269d5-kube-api-access-x5z9r\") pod \"postgres-69fb4594cf-4ldvf\" (UID: \"371615ce-ddf1-442f-a3b8-9b00b1d269d5\") " pod="odh-ai-gateway-infra/postgres-69fb4594cf-4ldvf" Jul 16 00:49:20.998252 ip-10-0-143-200 kubenswrapper[2572]: I0716 00:49:20.998232 2572 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"data\" (UniqueName: \"kubernetes.io/empty-dir/371615ce-ddf1-442f-a3b8-9b00b1d269d5-data\") pod \"postgres-69fb4594cf-4ldvf\" (UID: \"371615ce-ddf1-442f-a3b8-9b00b1d269d5\") " pod="odh-ai-gateway-infra/postgres-69fb4594cf-4ldvf" Jul 16 00:49:21.009267 ip-10-0-143-200 kubenswrapper[2572]: I0716 00:49:21.009235 2572 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-x5z9r\" (UniqueName: \"kubernetes.io/projected/371615ce-ddf1-442f-a3b8-9b00b1d269d5-kube-api-access-x5z9r\") pod \"postgres-69fb4594cf-4ldvf\" (UID: \"371615ce-ddf1-442f-a3b8-9b00b1d269d5\") " pod="odh-ai-gateway-infra/postgres-69fb4594cf-4ldvf" Jul 16 00:49:21.174311 ip-10-0-143-200 kubenswrapper[2572]: I0716 00:49:21.174224 2572 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/postgres-69fb4594cf-4ldvf" Jul 16 00:49:21.313786 ip-10-0-143-200 kubenswrapper[2572]: I0716 00:49:21.313762 2572 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/postgres-69fb4594cf-4ldvf"] Jul 16 00:49:21.316185 ip-10-0-143-200 kubenswrapper[2572]: W0716 00:49:21.316159 2572 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod371615ce_ddf1_442f_a3b8_9b00b1d269d5.slice/crio-aef9f0a9d0af3c1233dbb7f7a270ac4a0160afa7a23c34a0e725932fb156b9ba WatchSource:0}: Error finding container aef9f0a9d0af3c1233dbb7f7a270ac4a0160afa7a23c34a0e725932fb156b9ba: Status 404 returned error can't find the container with id aef9f0a9d0af3c1233dbb7f7a270ac4a0160afa7a23c34a0e725932fb156b9ba Jul 16 00:49:22.155667 ip-10-0-143-200 kubenswrapper[2572]: I0716 00:49:22.155609 2572 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/postgres-69fb4594cf-4ldvf" event={"ID":"371615ce-ddf1-442f-a3b8-9b00b1d269d5","Type":"ContainerStarted","Data":"aef9f0a9d0af3c1233dbb7f7a270ac4a0160afa7a23c34a0e725932fb156b9ba"} Jul 16 00:49:27.186281 ip-10-0-143-200 kubenswrapper[2572]: I0716 00:49:27.186242 2572 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/postgres-69fb4594cf-4ldvf" event={"ID":"371615ce-ddf1-442f-a3b8-9b00b1d269d5","Type":"ContainerStarted","Data":"8eb0ff5578396eb0b4f9f7822a44f42d9aa0504d35133caa45363f74fefbe39a"} Jul 16 00:49:27.186680 ip-10-0-143-200 kubenswrapper[2572]: I0716 00:49:27.186376 2572 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="odh-ai-gateway-infra/postgres-69fb4594cf-4ldvf" Jul 16 00:49:27.203846 ip-10-0-143-200 kubenswrapper[2572]: I0716 00:49:27.203797 2572 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="odh-ai-gateway-infra/postgres-69fb4594cf-4ldvf" podStartSLOduration=2.291662942 podStartE2EDuration="7.203778192s" totalImagesPullingTime="4.91211525s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-16 00:49:20 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-16 00:49:27.203018368 +0000 UTC m=+572.311773043" watchObservedRunningTime="2026-07-16 00:49:27.203778192 +0000 UTC m=+572.312532867" Jul 16 00:49:33.221815 ip-10-0-143-200 kubenswrapper[2572]: I0716 00:49:33.221779 2572 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="odh-ai-gateway-infra/postgres-69fb4594cf-4ldvf" Jul 16 00:49:33.600417 ip-10-0-143-200 kubenswrapper[2572]: I0716 00:49:33.600319 2572 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9gnmq_3a7e030d-fad8-480d-9dc6-fe5057048cf9/konnectivity-agent/0.log" limit=1024 Jul 16 00:49:33.965096 ip-10-0-143-200 kubenswrapper[2572]: I0716 00:49:33.965045 2572 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9gnmq_3a7e030d-fad8-480d-9dc6-fe5057048cf9/konnectivity-agent/0.log" limit=1024 Jul 16 00:49:41.061291 ip-10-0-143-200 kubenswrapper[2572]: I0716 00:49:41.061246 2572 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9gnmq_3a7e030d-fad8-480d-9dc6-fe5057048cf9/konnectivity-agent/0.log" limit=1024 Jul 16 00:49:41.078535 ip-10-0-143-200 kubenswrapper[2572]: I0716 00:49:41.078498 2572 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["opendatahub/maas-controller-5dcf9b48bf-5w48w"] Jul 16 00:49:41.087551 ip-10-0-143-200 kubenswrapper[2572]: I0716 00:49:41.087523 2572 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="opendatahub/maas-controller-5dcf9b48bf-5w48w" Jul 16 00:49:41.090765 ip-10-0-143-200 kubenswrapper[2572]: I0716 00:49:41.090738 2572 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"opendatahub\"/\"maas-controller-webhook-cert\"" Jul 16 00:49:41.090765 ip-10-0-143-200 kubenswrapper[2572]: I0716 00:49:41.090756 2572 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"opendatahub\"/\"maas-parameters\"" Jul 16 00:49:41.090964 ip-10-0-143-200 kubenswrapper[2572]: I0716 00:49:41.090776 2572 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"opendatahub\"/\"maas-controller-dockercfg-nvvzq\"" Jul 16 00:49:41.094452 ip-10-0-143-200 kubenswrapper[2572]: I0716 00:49:41.094431 2572 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["opendatahub/maas-controller-5dcf9b48bf-5w48w"] Jul 16 00:49:41.162248 ip-10-0-143-200 kubenswrapper[2572]: I0716 00:49:41.162211 2572 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9gnmq_3a7e030d-fad8-480d-9dc6-fe5057048cf9/konnectivity-agent/0.log" limit=1024 Jul 16 00:49:41.183340 ip-10-0-143-200 kubenswrapper[2572]: I0716 00:49:41.183311 2572 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-ph29h\" (UniqueName: \"kubernetes.io/projected/771a1bf6-5a02-47d3-b3da-7582d6bbd7c1-kube-api-access-ph29h\") pod \"maas-controller-5dcf9b48bf-5w48w\" (UID: \"771a1bf6-5a02-47d3-b3da-7582d6bbd7c1\") " pod="opendatahub/maas-controller-5dcf9b48bf-5w48w" Jul 16 00:49:41.183491 ip-10-0-143-200 kubenswrapper[2572]: I0716 00:49:41.183385 2572 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"webhook-cert\" (UniqueName: \"kubernetes.io/secret/771a1bf6-5a02-47d3-b3da-7582d6bbd7c1-webhook-cert\") pod \"maas-controller-5dcf9b48bf-5w48w\" (UID: \"771a1bf6-5a02-47d3-b3da-7582d6bbd7c1\") " pod="opendatahub/maas-controller-5dcf9b48bf-5w48w" Jul 16 00:49:41.264430 ip-10-0-143-200 kubenswrapper[2572]: I0716 00:49:41.264393 2572 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9gnmq_3a7e030d-fad8-480d-9dc6-fe5057048cf9/konnectivity-agent/0.log" limit=1024 Jul 16 00:49:41.275585 ip-10-0-143-200 kubenswrapper[2572]: I0716 00:49:41.275550 2572 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["opendatahub/maas-controller-79cf8b95fb-9jrsf"] Jul 16 00:49:41.279597 ip-10-0-143-200 kubenswrapper[2572]: I0716 00:49:41.279577 2572 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="opendatahub/maas-controller-79cf8b95fb-9jrsf" Jul 16 00:49:41.283883 ip-10-0-143-200 kubenswrapper[2572]: I0716 00:49:41.283863 2572 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"webhook-cert\" (UniqueName: \"kubernetes.io/secret/771a1bf6-5a02-47d3-b3da-7582d6bbd7c1-webhook-cert\") pod \"maas-controller-5dcf9b48bf-5w48w\" (UID: \"771a1bf6-5a02-47d3-b3da-7582d6bbd7c1\") " pod="opendatahub/maas-controller-5dcf9b48bf-5w48w" Jul 16 00:49:41.284007 ip-10-0-143-200 kubenswrapper[2572]: I0716 00:49:41.283922 2572 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-ph29h\" (UniqueName: \"kubernetes.io/projected/771a1bf6-5a02-47d3-b3da-7582d6bbd7c1-kube-api-access-ph29h\") pod \"maas-controller-5dcf9b48bf-5w48w\" (UID: \"771a1bf6-5a02-47d3-b3da-7582d6bbd7c1\") " pod="opendatahub/maas-controller-5dcf9b48bf-5w48w" Jul 16 00:49:41.286509 ip-10-0-143-200 kubenswrapper[2572]: I0716 00:49:41.286374 2572 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"webhook-cert\" (UniqueName: \"kubernetes.io/secret/771a1bf6-5a02-47d3-b3da-7582d6bbd7c1-webhook-cert\") pod \"maas-controller-5dcf9b48bf-5w48w\" (UID: \"771a1bf6-5a02-47d3-b3da-7582d6bbd7c1\") " pod="opendatahub/maas-controller-5dcf9b48bf-5w48w" Jul 16 00:49:41.290588 ip-10-0-143-200 kubenswrapper[2572]: I0716 00:49:41.290562 2572 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["opendatahub/maas-controller-79cf8b95fb-9jrsf"] Jul 16 00:49:41.295759 ip-10-0-143-200 kubenswrapper[2572]: I0716 00:49:41.295739 2572 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-ph29h\" (UniqueName: \"kubernetes.io/projected/771a1bf6-5a02-47d3-b3da-7582d6bbd7c1-kube-api-access-ph29h\") pod \"maas-controller-5dcf9b48bf-5w48w\" (UID: \"771a1bf6-5a02-47d3-b3da-7582d6bbd7c1\") " pod="opendatahub/maas-controller-5dcf9b48bf-5w48w" Jul 16 00:49:41.328738 ip-10-0-143-200 kubenswrapper[2572]: I0716 00:49:41.328651 2572 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9gnmq_3a7e030d-fad8-480d-9dc6-fe5057048cf9/konnectivity-agent/0.log" limit=1024 Jul 16 00:49:41.384884 ip-10-0-143-200 kubenswrapper[2572]: I0716 00:49:41.384850 2572 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-lt58f\" (UniqueName: \"kubernetes.io/projected/0039286c-075d-49a3-93c5-94b605749489-kube-api-access-lt58f\") pod \"maas-controller-79cf8b95fb-9jrsf\" (UID: \"0039286c-075d-49a3-93c5-94b605749489\") " pod="opendatahub/maas-controller-79cf8b95fb-9jrsf" Jul 16 00:49:41.385045 ip-10-0-143-200 kubenswrapper[2572]: I0716 00:49:41.384888 2572 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"webhook-cert\" (UniqueName: \"kubernetes.io/secret/0039286c-075d-49a3-93c5-94b605749489-webhook-cert\") pod \"maas-controller-79cf8b95fb-9jrsf\" (UID: \"0039286c-075d-49a3-93c5-94b605749489\") " pod="opendatahub/maas-controller-79cf8b95fb-9jrsf" Jul 16 00:49:41.400740 ip-10-0-143-200 kubenswrapper[2572]: I0716 00:49:41.400713 2572 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="opendatahub/maas-controller-5dcf9b48bf-5w48w" Jul 16 00:49:41.486339 ip-10-0-143-200 kubenswrapper[2572]: I0716 00:49:41.486297 2572 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-lt58f\" (UniqueName: \"kubernetes.io/projected/0039286c-075d-49a3-93c5-94b605749489-kube-api-access-lt58f\") pod \"maas-controller-79cf8b95fb-9jrsf\" (UID: \"0039286c-075d-49a3-93c5-94b605749489\") " pod="opendatahub/maas-controller-79cf8b95fb-9jrsf" Jul 16 00:49:41.486339 ip-10-0-143-200 kubenswrapper[2572]: I0716 00:49:41.486343 2572 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"webhook-cert\" (UniqueName: \"kubernetes.io/secret/0039286c-075d-49a3-93c5-94b605749489-webhook-cert\") pod \"maas-controller-79cf8b95fb-9jrsf\" (UID: \"0039286c-075d-49a3-93c5-94b605749489\") " pod="opendatahub/maas-controller-79cf8b95fb-9jrsf" Jul 16 00:49:41.490092 ip-10-0-143-200 kubenswrapper[2572]: I0716 00:49:41.490054 2572 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"webhook-cert\" (UniqueName: \"kubernetes.io/secret/0039286c-075d-49a3-93c5-94b605749489-webhook-cert\") pod \"maas-controller-79cf8b95fb-9jrsf\" (UID: \"0039286c-075d-49a3-93c5-94b605749489\") " pod="opendatahub/maas-controller-79cf8b95fb-9jrsf" Jul 16 00:49:41.496490 ip-10-0-143-200 kubenswrapper[2572]: I0716 00:49:41.496461 2572 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-lt58f\" (UniqueName: \"kubernetes.io/projected/0039286c-075d-49a3-93c5-94b605749489-kube-api-access-lt58f\") pod \"maas-controller-79cf8b95fb-9jrsf\" (UID: \"0039286c-075d-49a3-93c5-94b605749489\") " pod="opendatahub/maas-controller-79cf8b95fb-9jrsf" Jul 16 00:49:41.536703 ip-10-0-143-200 kubenswrapper[2572]: I0716 00:49:41.536678 2572 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["opendatahub/maas-controller-5dcf9b48bf-5w48w"] Jul 16 00:49:41.538700 ip-10-0-143-200 kubenswrapper[2572]: W0716 00:49:41.538678 2572 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod771a1bf6_5a02_47d3_b3da_7582d6bbd7c1.slice/crio-d9dd1c8ba2d599412ccc32657784ae383672f9a104cd1904795a8905ffcaadf3 WatchSource:0}: Error finding container d9dd1c8ba2d599412ccc32657784ae383672f9a104cd1904795a8905ffcaadf3: Status 404 returned error can't find the container with id d9dd1c8ba2d599412ccc32657784ae383672f9a104cd1904795a8905ffcaadf3 Jul 16 00:49:41.613464 ip-10-0-143-200 kubenswrapper[2572]: I0716 00:49:41.613371 2572 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="opendatahub/maas-controller-79cf8b95fb-9jrsf" Jul 16 00:49:41.773720 ip-10-0-143-200 kubenswrapper[2572]: I0716 00:49:41.773694 2572 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["opendatahub/maas-controller-79cf8b95fb-9jrsf"] Jul 16 00:49:41.776156 ip-10-0-143-200 kubenswrapper[2572]: W0716 00:49:41.776121 2572 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod0039286c_075d_49a3_93c5_94b605749489.slice/crio-5ce1390ea9a335ee19dac3f472a7877807a6581e7a6c2e3a5c5e1e03562c7411 WatchSource:0}: Error finding container 5ce1390ea9a335ee19dac3f472a7877807a6581e7a6c2e3a5c5e1e03562c7411: Status 404 returned error can't find the container with id 5ce1390ea9a335ee19dac3f472a7877807a6581e7a6c2e3a5c5e1e03562c7411 Jul 16 00:49:42.270322 ip-10-0-143-200 kubenswrapper[2572]: I0716 00:49:42.270267 2572 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="opendatahub/maas-controller-79cf8b95fb-9jrsf" event={"ID":"0039286c-075d-49a3-93c5-94b605749489","Type":"ContainerStarted","Data":"5ce1390ea9a335ee19dac3f472a7877807a6581e7a6c2e3a5c5e1e03562c7411"} Jul 16 00:49:42.272343 ip-10-0-143-200 kubenswrapper[2572]: I0716 00:49:42.272299 2572 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="opendatahub/maas-controller-5dcf9b48bf-5w48w" event={"ID":"771a1bf6-5a02-47d3-b3da-7582d6bbd7c1","Type":"ContainerStarted","Data":"d9dd1c8ba2d599412ccc32657784ae383672f9a104cd1904795a8905ffcaadf3"} Jul 16 00:49:44.286701 ip-10-0-143-200 kubenswrapper[2572]: I0716 00:49:44.286619 2572 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="opendatahub/maas-controller-5dcf9b48bf-5w48w" event={"ID":"771a1bf6-5a02-47d3-b3da-7582d6bbd7c1","Type":"ContainerStarted","Data":"8e7dc9268ec6e1d9195cdfdaf192161aa79463ff3a246e2dd15857dd915d1b5e"} Jul 16 00:49:44.287188 ip-10-0-143-200 kubenswrapper[2572]: I0716 00:49:44.286790 2572 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="opendatahub/maas-controller-5dcf9b48bf-5w48w" Jul 16 00:49:44.288235 ip-10-0-143-200 kubenswrapper[2572]: I0716 00:49:44.288210 2572 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="opendatahub/maas-controller-79cf8b95fb-9jrsf" event={"ID":"0039286c-075d-49a3-93c5-94b605749489","Type":"ContainerStarted","Data":"554971dd12abb0dd2b4f1d5a171714fed8b2558820a9f49e0b2feb7e87bfc957"} Jul 16 00:49:44.288363 ip-10-0-143-200 kubenswrapper[2572]: I0716 00:49:44.288302 2572 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="opendatahub/maas-controller-79cf8b95fb-9jrsf" Jul 16 00:49:44.304882 ip-10-0-143-200 kubenswrapper[2572]: I0716 00:49:44.304829 2572 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="opendatahub/maas-controller-5dcf9b48bf-5w48w" podStartSLOduration=0.694462235 podStartE2EDuration="3.30481432s" totalImagesPullingTime="2.610352085s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-16 00:49:41 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-16 00:49:44.303255901 +0000 UTC m=+589.412010577" watchObservedRunningTime="2026-07-16 00:49:44.30481432 +0000 UTC m=+589.413568994" Jul 16 00:49:44.328283 ip-10-0-143-200 kubenswrapper[2572]: I0716 00:49:44.328222 2572 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="opendatahub/maas-controller-79cf8b95fb-9jrsf" podStartSLOduration=0.942083602 podStartE2EDuration="3.328201708s" totalImagesPullingTime="2.386118106s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-16 00:49:41 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-16 00:49:44.32491666 +0000 UTC m=+589.433671335" watchObservedRunningTime="2026-07-16 00:49:44.328201708 +0000 UTC m=+589.436956384" Jul 16 00:49:44.371534 ip-10-0-143-200 kubenswrapper[2572]: I0716 00:49:44.371499 2572 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9gnmq_3a7e030d-fad8-480d-9dc6-fe5057048cf9/konnectivity-agent/0.log" limit=1024 Jul 16 00:49:44.470664 ip-10-0-143-200 kubenswrapper[2572]: I0716 00:49:44.470590 2572 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9gnmq_3a7e030d-fad8-480d-9dc6-fe5057048cf9/konnectivity-agent/0.log" limit=1024 Jul 16 00:49:46.748560 ip-10-0-143-200 kubenswrapper[2572]: I0716 00:49:46.748522 2572 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9gnmq_3a7e030d-fad8-480d-9dc6-fe5057048cf9/konnectivity-agent/0.log" limit=1024 Jul 16 00:49:46.849666 ip-10-0-143-200 kubenswrapper[2572]: I0716 00:49:46.849617 2572 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9gnmq_3a7e030d-fad8-480d-9dc6-fe5057048cf9/konnectivity-agent/0.log" limit=1024 Jul 16 00:49:49.546679 ip-10-0-143-200 kubenswrapper[2572]: I0716 00:49:49.546624 2572 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9gnmq_3a7e030d-fad8-480d-9dc6-fe5057048cf9/konnectivity-agent/0.log" limit=1024 Jul 16 00:49:49.647720 ip-10-0-143-200 kubenswrapper[2572]: I0716 00:49:49.647683 2572 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9gnmq_3a7e030d-fad8-480d-9dc6-fe5057048cf9/konnectivity-agent/0.log" limit=1024 Jul 16 00:49:55.299689 ip-10-0-143-200 kubenswrapper[2572]: I0716 00:49:55.299657 2572 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="opendatahub/maas-controller-79cf8b95fb-9jrsf" Jul 16 00:49:55.300083 ip-10-0-143-200 kubenswrapper[2572]: I0716 00:49:55.299732 2572 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="opendatahub/maas-controller-5dcf9b48bf-5w48w" Jul 16 00:49:55.366746 ip-10-0-143-200 kubenswrapper[2572]: I0716 00:49:55.366715 2572 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-console-operator_console-operator-7866497494-pjc6j_a64bcd05-8596-4c12-b88e-e974d676e25f/console-operator/2.log" Jul 16 00:49:55.368854 ip-10-0-143-200 kubenswrapper[2572]: I0716 00:49:55.368832 2572 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-console-operator_console-operator-7866497494-pjc6j_a64bcd05-8596-4c12-b88e-e974d676e25f/console-operator/2.log" Jul 16 00:49:55.372773 ip-10-0-143-200 kubenswrapper[2572]: I0716 00:49:55.372750 2572 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ovn-kubernetes_ovnkube-node-fp78g_aa66f2b6-0ad8-45d0-8ff5-40fc21966b5a/ovn-acl-logging/0.log" Jul 16 00:49:55.374759 ip-10-0-143-200 kubenswrapper[2572]: I0716 00:49:55.374736 2572 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["opendatahub/maas-controller-5dcf9b48bf-5w48w"] Jul 16 00:49:55.374870 ip-10-0-143-200 kubenswrapper[2572]: I0716 00:49:55.374824 2572 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ovn-kubernetes_ovnkube-node-fp78g_aa66f2b6-0ad8-45d0-8ff5-40fc21966b5a/ovn-acl-logging/0.log" Jul 16 00:49:55.374989 ip-10-0-143-200 kubenswrapper[2572]: I0716 00:49:55.374960 2572 kuberuntime_container.go:873] "Killing container with a grace period" pod="opendatahub/maas-controller-5dcf9b48bf-5w48w" podUID="771a1bf6-5a02-47d3-b3da-7582d6bbd7c1" containerName="manager" containerID="cri-o://8e7dc9268ec6e1d9195cdfdaf192161aa79463ff3a246e2dd15857dd915d1b5e" gracePeriod=30 Jul 16 00:49:55.622021 ip-10-0-143-200 kubenswrapper[2572]: I0716 00:49:55.621992 2572 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="opendatahub/maas-controller-5dcf9b48bf-5w48w" Jul 16 00:49:55.719058 ip-10-0-143-200 kubenswrapper[2572]: I0716 00:49:55.719022 2572 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-ph29h\" (UniqueName: \"kubernetes.io/projected/771a1bf6-5a02-47d3-b3da-7582d6bbd7c1-kube-api-access-ph29h\") pod \"771a1bf6-5a02-47d3-b3da-7582d6bbd7c1\" (UID: \"771a1bf6-5a02-47d3-b3da-7582d6bbd7c1\") " Jul 16 00:49:55.719058 ip-10-0-143-200 kubenswrapper[2572]: I0716 00:49:55.719063 2572 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"webhook-cert\" (UniqueName: \"kubernetes.io/secret/771a1bf6-5a02-47d3-b3da-7582d6bbd7c1-webhook-cert\") pod \"771a1bf6-5a02-47d3-b3da-7582d6bbd7c1\" (UID: \"771a1bf6-5a02-47d3-b3da-7582d6bbd7c1\") " Jul 16 00:49:55.721299 ip-10-0-143-200 kubenswrapper[2572]: I0716 00:49:55.721267 2572 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/771a1bf6-5a02-47d3-b3da-7582d6bbd7c1-webhook-cert" (OuterVolumeSpecName: "webhook-cert") pod "771a1bf6-5a02-47d3-b3da-7582d6bbd7c1" (UID: "771a1bf6-5a02-47d3-b3da-7582d6bbd7c1"). InnerVolumeSpecName "webhook-cert". PluginName "kubernetes.io/secret", VolumeGIDValue "" Jul 16 00:49:55.721408 ip-10-0-143-200 kubenswrapper[2572]: I0716 00:49:55.721326 2572 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/771a1bf6-5a02-47d3-b3da-7582d6bbd7c1-kube-api-access-ph29h" (OuterVolumeSpecName: "kube-api-access-ph29h") pod "771a1bf6-5a02-47d3-b3da-7582d6bbd7c1" (UID: "771a1bf6-5a02-47d3-b3da-7582d6bbd7c1"). InnerVolumeSpecName "kube-api-access-ph29h". PluginName "kubernetes.io/projected", VolumeGIDValue "" Jul 16 00:49:55.820240 ip-10-0-143-200 kubenswrapper[2572]: I0716 00:49:55.820202 2572 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-ph29h\" (UniqueName: \"kubernetes.io/projected/771a1bf6-5a02-47d3-b3da-7582d6bbd7c1-kube-api-access-ph29h\") on node \"ip-10-0-143-200.ec2.internal\" DevicePath \"\"" Jul 16 00:49:55.820240 ip-10-0-143-200 kubenswrapper[2572]: I0716 00:49:55.820235 2572 reconciler_common.go:299] "Volume detached for volume \"webhook-cert\" (UniqueName: \"kubernetes.io/secret/771a1bf6-5a02-47d3-b3da-7582d6bbd7c1-webhook-cert\") on node \"ip-10-0-143-200.ec2.internal\" DevicePath \"\"" Jul 16 00:49:56.355855 ip-10-0-143-200 kubenswrapper[2572]: I0716 00:49:56.355818 2572 generic.go:356] "Generic (PLEG): container finished" podID="771a1bf6-5a02-47d3-b3da-7582d6bbd7c1" containerID="8e7dc9268ec6e1d9195cdfdaf192161aa79463ff3a246e2dd15857dd915d1b5e" exitCode=0 Jul 16 00:49:56.355855 ip-10-0-143-200 kubenswrapper[2572]: I0716 00:49:56.355860 2572 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="opendatahub/maas-controller-5dcf9b48bf-5w48w" Jul 16 00:49:56.356369 ip-10-0-143-200 kubenswrapper[2572]: I0716 00:49:56.355868 2572 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="opendatahub/maas-controller-5dcf9b48bf-5w48w" event={"ID":"771a1bf6-5a02-47d3-b3da-7582d6bbd7c1","Type":"ContainerDied","Data":"8e7dc9268ec6e1d9195cdfdaf192161aa79463ff3a246e2dd15857dd915d1b5e"} Jul 16 00:49:56.356369 ip-10-0-143-200 kubenswrapper[2572]: I0716 00:49:56.355919 2572 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="opendatahub/maas-controller-5dcf9b48bf-5w48w" event={"ID":"771a1bf6-5a02-47d3-b3da-7582d6bbd7c1","Type":"ContainerDied","Data":"d9dd1c8ba2d599412ccc32657784ae383672f9a104cd1904795a8905ffcaadf3"} Jul 16 00:49:56.356369 ip-10-0-143-200 kubenswrapper[2572]: I0716 00:49:56.355950 2572 scope.go:117] "RemoveContainer" containerID="8e7dc9268ec6e1d9195cdfdaf192161aa79463ff3a246e2dd15857dd915d1b5e" Jul 16 00:49:56.365772 ip-10-0-143-200 kubenswrapper[2572]: I0716 00:49:56.365689 2572 scope.go:117] "RemoveContainer" containerID="8e7dc9268ec6e1d9195cdfdaf192161aa79463ff3a246e2dd15857dd915d1b5e" Jul 16 00:49:56.365977 ip-10-0-143-200 kubenswrapper[2572]: E0716 00:49:56.365958 2572 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"8e7dc9268ec6e1d9195cdfdaf192161aa79463ff3a246e2dd15857dd915d1b5e\": container with ID starting with 8e7dc9268ec6e1d9195cdfdaf192161aa79463ff3a246e2dd15857dd915d1b5e not found: ID does not exist" containerID="8e7dc9268ec6e1d9195cdfdaf192161aa79463ff3a246e2dd15857dd915d1b5e" Jul 16 00:49:56.366026 ip-10-0-143-200 kubenswrapper[2572]: I0716 00:49:56.365989 2572 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"8e7dc9268ec6e1d9195cdfdaf192161aa79463ff3a246e2dd15857dd915d1b5e"} err="failed to get container status \"8e7dc9268ec6e1d9195cdfdaf192161aa79463ff3a246e2dd15857dd915d1b5e\": rpc error: code = NotFound desc = could not find container \"8e7dc9268ec6e1d9195cdfdaf192161aa79463ff3a246e2dd15857dd915d1b5e\": container with ID starting with 8e7dc9268ec6e1d9195cdfdaf192161aa79463ff3a246e2dd15857dd915d1b5e not found: ID does not exist" Jul 16 00:49:56.378901 ip-10-0-143-200 kubenswrapper[2572]: I0716 00:49:56.378878 2572 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["opendatahub/maas-controller-5dcf9b48bf-5w48w"] Jul 16 00:49:56.382465 ip-10-0-143-200 kubenswrapper[2572]: I0716 00:49:56.382443 2572 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["opendatahub/maas-controller-5dcf9b48bf-5w48w"] Jul 16 00:49:57.447821 ip-10-0-143-200 kubenswrapper[2572]: I0716 00:49:57.447784 2572 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="771a1bf6-5a02-47d3-b3da-7582d6bbd7c1" path="/var/lib/kubelet/pods/771a1bf6-5a02-47d3-b3da-7582d6bbd7c1/volumes" Jul 16 00:50:07.378307 ip-10-0-143-200 kubenswrapper[2572]: I0716 00:50:07.378262 2572 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9gnmq_3a7e030d-fad8-480d-9dc6-fe5057048cf9/konnectivity-agent/0.log" limit=1024 Jul 16 00:50:08.907374 ip-10-0-143-200 kubenswrapper[2572]: I0716 00:50:08.907337 2572 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9gnmq_3a7e030d-fad8-480d-9dc6-fe5057048cf9/konnectivity-agent/0.log" limit=1024 Jul 16 00:50:14.009404 ip-10-0-143-200 kubenswrapper[2572]: I0716 00:50:14.009363 2572 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9gnmq_3a7e030d-fad8-480d-9dc6-fe5057048cf9/konnectivity-agent/0.log" limit=1024 Jul 16 00:50:14.101769 ip-10-0-143-200 kubenswrapper[2572]: I0716 00:50:14.101734 2572 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9gnmq_3a7e030d-fad8-480d-9dc6-fe5057048cf9/konnectivity-agent/0.log" limit=1024 Jul 16 00:50:14.489772 ip-10-0-143-200 kubenswrapper[2572]: I0716 00:50:14.489739 2572 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9gnmq_3a7e030d-fad8-480d-9dc6-fe5057048cf9/konnectivity-agent/0.log" limit=1024 Jul 16 00:50:14.540041 ip-10-0-143-200 kubenswrapper[2572]: I0716 00:50:14.540002 2572 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["odh-ai-gateway-infra/maas-api-7464b7965d-lm97p"] Jul 16 00:50:14.540511 ip-10-0-143-200 kubenswrapper[2572]: I0716 00:50:14.540490 2572 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="771a1bf6-5a02-47d3-b3da-7582d6bbd7c1" containerName="manager" Jul 16 00:50:14.540511 ip-10-0-143-200 kubenswrapper[2572]: I0716 00:50:14.540513 2572 state_mem.go:107] "Deleted CPUSet assignment" podUID="771a1bf6-5a02-47d3-b3da-7582d6bbd7c1" containerName="manager" Jul 16 00:50:14.540724 ip-10-0-143-200 kubenswrapper[2572]: I0716 00:50:14.540607 2572 memory_manager.go:368] "RemoveStaleState removing state" podUID="771a1bf6-5a02-47d3-b3da-7582d6bbd7c1" containerName="manager" Jul 16 00:50:14.543913 ip-10-0-143-200 kubenswrapper[2572]: I0716 00:50:14.543892 2572 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-7464b7965d-lm97p" Jul 16 00:50:14.548379 ip-10-0-143-200 kubenswrapper[2572]: I0716 00:50:14.548360 2572 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"odh-ai-gateway-infra\"/\"maas-api-serving-cert\"" Jul 16 00:50:14.564453 ip-10-0-143-200 kubenswrapper[2572]: I0716 00:50:14.564429 2572 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-7464b7965d-lm97p"] Jul 16 00:50:14.586116 ip-10-0-143-200 kubenswrapper[2572]: I0716 00:50:14.586083 2572 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-s45xk\" (UniqueName: \"kubernetes.io/projected/602f254c-3d38-4fa4-87d9-005fcfd1e70e-kube-api-access-s45xk\") pod \"maas-api-7464b7965d-lm97p\" (UID: \"602f254c-3d38-4fa4-87d9-005fcfd1e70e\") " pod="odh-ai-gateway-infra/maas-api-7464b7965d-lm97p" Jul 16 00:50:14.586288 ip-10-0-143-200 kubenswrapper[2572]: I0716 00:50:14.586121 2572 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/602f254c-3d38-4fa4-87d9-005fcfd1e70e-openshift-service-ca\") pod \"maas-api-7464b7965d-lm97p\" (UID: \"602f254c-3d38-4fa4-87d9-005fcfd1e70e\") " pod="odh-ai-gateway-infra/maas-api-7464b7965d-lm97p" Jul 16 00:50:14.586288 ip-10-0-143-200 kubenswrapper[2572]: I0716 00:50:14.586205 2572 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/602f254c-3d38-4fa4-87d9-005fcfd1e70e-maas-api-tls\") pod \"maas-api-7464b7965d-lm97p\" (UID: \"602f254c-3d38-4fa4-87d9-005fcfd1e70e\") " pod="odh-ai-gateway-infra/maas-api-7464b7965d-lm97p" Jul 16 00:50:14.640257 ip-10-0-143-200 kubenswrapper[2572]: I0716 00:50:14.640221 2572 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9gnmq_3a7e030d-fad8-480d-9dc6-fe5057048cf9/konnectivity-agent/0.log" limit=1024 Jul 16 00:50:14.687655 ip-10-0-143-200 kubenswrapper[2572]: I0716 00:50:14.687582 2572 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-s45xk\" (UniqueName: \"kubernetes.io/projected/602f254c-3d38-4fa4-87d9-005fcfd1e70e-kube-api-access-s45xk\") pod \"maas-api-7464b7965d-lm97p\" (UID: \"602f254c-3d38-4fa4-87d9-005fcfd1e70e\") " pod="odh-ai-gateway-infra/maas-api-7464b7965d-lm97p" Jul 16 00:50:14.687843 ip-10-0-143-200 kubenswrapper[2572]: I0716 00:50:14.687664 2572 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/602f254c-3d38-4fa4-87d9-005fcfd1e70e-openshift-service-ca\") pod \"maas-api-7464b7965d-lm97p\" (UID: \"602f254c-3d38-4fa4-87d9-005fcfd1e70e\") " pod="odh-ai-gateway-infra/maas-api-7464b7965d-lm97p" Jul 16 00:50:14.687843 ip-10-0-143-200 kubenswrapper[2572]: I0716 00:50:14.687763 2572 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/602f254c-3d38-4fa4-87d9-005fcfd1e70e-maas-api-tls\") pod \"maas-api-7464b7965d-lm97p\" (UID: \"602f254c-3d38-4fa4-87d9-005fcfd1e70e\") " pod="odh-ai-gateway-infra/maas-api-7464b7965d-lm97p" Jul 16 00:50:14.688562 ip-10-0-143-200 kubenswrapper[2572]: I0716 00:50:14.688536 2572 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/602f254c-3d38-4fa4-87d9-005fcfd1e70e-openshift-service-ca\") pod \"maas-api-7464b7965d-lm97p\" (UID: \"602f254c-3d38-4fa4-87d9-005fcfd1e70e\") " pod="odh-ai-gateway-infra/maas-api-7464b7965d-lm97p" Jul 16 00:50:14.690899 ip-10-0-143-200 kubenswrapper[2572]: I0716 00:50:14.690875 2572 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/602f254c-3d38-4fa4-87d9-005fcfd1e70e-maas-api-tls\") pod \"maas-api-7464b7965d-lm97p\" (UID: \"602f254c-3d38-4fa4-87d9-005fcfd1e70e\") " pod="odh-ai-gateway-infra/maas-api-7464b7965d-lm97p" Jul 16 00:50:14.700516 ip-10-0-143-200 kubenswrapper[2572]: I0716 00:50:14.700491 2572 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-s45xk\" (UniqueName: \"kubernetes.io/projected/602f254c-3d38-4fa4-87d9-005fcfd1e70e-kube-api-access-s45xk\") pod \"maas-api-7464b7965d-lm97p\" (UID: \"602f254c-3d38-4fa4-87d9-005fcfd1e70e\") " pod="odh-ai-gateway-infra/maas-api-7464b7965d-lm97p" Jul 16 00:50:14.853710 ip-10-0-143-200 kubenswrapper[2572]: I0716 00:50:14.853609 2572 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-7464b7965d-lm97p" Jul 16 00:50:14.995508 ip-10-0-143-200 kubenswrapper[2572]: I0716 00:50:14.995483 2572 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-7464b7965d-lm97p"] Jul 16 00:50:14.997328 ip-10-0-143-200 kubenswrapper[2572]: W0716 00:50:14.997295 2572 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod602f254c_3d38_4fa4_87d9_005fcfd1e70e.slice/crio-4844695aa18d171766bf61cd0fefd8e65602a0c2d2d9e06d17c82608981b608d WatchSource:0}: Error finding container 4844695aa18d171766bf61cd0fefd8e65602a0c2d2d9e06d17c82608981b608d: Status 404 returned error can't find the container with id 4844695aa18d171766bf61cd0fefd8e65602a0c2d2d9e06d17c82608981b608d Jul 16 00:50:14.998423 ip-10-0-143-200 kubenswrapper[2572]: I0716 00:50:14.998362 2572 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9gnmq_3a7e030d-fad8-480d-9dc6-fe5057048cf9/konnectivity-agent/0.log" limit=1024 Jul 16 00:50:15.094893 ip-10-0-143-200 kubenswrapper[2572]: I0716 00:50:15.094855 2572 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9gnmq_3a7e030d-fad8-480d-9dc6-fe5057048cf9/konnectivity-agent/0.log" limit=1024 Jul 16 00:50:15.125686 ip-10-0-143-200 kubenswrapper[2572]: I0716 00:50:15.125590 2572 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-ingress/payload-processing-f9b8d9b9b-fj2lp"] Jul 16 00:50:15.130258 ip-10-0-143-200 kubenswrapper[2572]: I0716 00:50:15.130225 2572 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-processing-f9b8d9b9b-fj2lp" Jul 16 00:50:15.133119 ip-10-0-143-200 kubenswrapper[2572]: I0716 00:50:15.133094 2572 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ingress\"/\"payload-processing-plugins\"" Jul 16 00:50:15.133119 ip-10-0-143-200 kubenswrapper[2572]: I0716 00:50:15.133118 2572 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress\"/\"payload-processing-dockercfg-kvcs9\"" Jul 16 00:50:15.148595 ip-10-0-143-200 kubenswrapper[2572]: I0716 00:50:15.148564 2572 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/payload-processing-f9b8d9b9b-fj2lp"] Jul 16 00:50:15.192281 ip-10-0-143-200 kubenswrapper[2572]: I0716 00:50:15.192251 2572 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/c9557a6b-396b-4ab0-a3cb-8ddcf3c37f19-plugins-config-volume\") pod \"payload-processing-f9b8d9b9b-fj2lp\" (UID: \"c9557a6b-396b-4ab0-a3cb-8ddcf3c37f19\") " pod="openshift-ingress/payload-processing-f9b8d9b9b-fj2lp" Jul 16 00:50:15.192443 ip-10-0-143-200 kubenswrapper[2572]: I0716 00:50:15.192345 2572 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-hdz5x\" (UniqueName: \"kubernetes.io/projected/c9557a6b-396b-4ab0-a3cb-8ddcf3c37f19-kube-api-access-hdz5x\") pod \"payload-processing-f9b8d9b9b-fj2lp\" (UID: \"c9557a6b-396b-4ab0-a3cb-8ddcf3c37f19\") " pod="openshift-ingress/payload-processing-f9b8d9b9b-fj2lp" Jul 16 00:50:15.293830 ip-10-0-143-200 kubenswrapper[2572]: I0716 00:50:15.293791 2572 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-hdz5x\" (UniqueName: \"kubernetes.io/projected/c9557a6b-396b-4ab0-a3cb-8ddcf3c37f19-kube-api-access-hdz5x\") pod \"payload-processing-f9b8d9b9b-fj2lp\" (UID: \"c9557a6b-396b-4ab0-a3cb-8ddcf3c37f19\") " pod="openshift-ingress/payload-processing-f9b8d9b9b-fj2lp" Jul 16 00:50:15.294040 ip-10-0-143-200 kubenswrapper[2572]: I0716 00:50:15.293867 2572 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/c9557a6b-396b-4ab0-a3cb-8ddcf3c37f19-plugins-config-volume\") pod \"payload-processing-f9b8d9b9b-fj2lp\" (UID: \"c9557a6b-396b-4ab0-a3cb-8ddcf3c37f19\") " pod="openshift-ingress/payload-processing-f9b8d9b9b-fj2lp" Jul 16 00:50:15.294696 ip-10-0-143-200 kubenswrapper[2572]: I0716 00:50:15.294677 2572 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/c9557a6b-396b-4ab0-a3cb-8ddcf3c37f19-plugins-config-volume\") pod \"payload-processing-f9b8d9b9b-fj2lp\" (UID: \"c9557a6b-396b-4ab0-a3cb-8ddcf3c37f19\") " pod="openshift-ingress/payload-processing-f9b8d9b9b-fj2lp" Jul 16 00:50:15.304245 ip-10-0-143-200 kubenswrapper[2572]: I0716 00:50:15.304219 2572 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-hdz5x\" (UniqueName: \"kubernetes.io/projected/c9557a6b-396b-4ab0-a3cb-8ddcf3c37f19-kube-api-access-hdz5x\") pod \"payload-processing-f9b8d9b9b-fj2lp\" (UID: \"c9557a6b-396b-4ab0-a3cb-8ddcf3c37f19\") " pod="openshift-ingress/payload-processing-f9b8d9b9b-fj2lp" Jul 16 00:50:15.441351 ip-10-0-143-200 kubenswrapper[2572]: I0716 00:50:15.441247 2572 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-processing-f9b8d9b9b-fj2lp" Jul 16 00:50:15.461979 ip-10-0-143-200 kubenswrapper[2572]: I0716 00:50:15.461948 2572 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-7464b7965d-lm97p" event={"ID":"602f254c-3d38-4fa4-87d9-005fcfd1e70e","Type":"ContainerStarted","Data":"4844695aa18d171766bf61cd0fefd8e65602a0c2d2d9e06d17c82608981b608d"} Jul 16 00:50:15.589471 ip-10-0-143-200 kubenswrapper[2572]: I0716 00:50:15.589436 2572 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/payload-processing-f9b8d9b9b-fj2lp"] Jul 16 00:50:15.590481 ip-10-0-143-200 kubenswrapper[2572]: W0716 00:50:15.590452 2572 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podc9557a6b_396b_4ab0_a3cb_8ddcf3c37f19.slice/crio-f8f967bd36a7c4796933e89406925560497bc6708a6c9d92251f9e318673d6e3 WatchSource:0}: Error finding container f8f967bd36a7c4796933e89406925560497bc6708a6c9d92251f9e318673d6e3: Status 404 returned error can't find the container with id f8f967bd36a7c4796933e89406925560497bc6708a6c9d92251f9e318673d6e3 Jul 16 00:50:16.469685 ip-10-0-143-200 kubenswrapper[2572]: I0716 00:50:16.469646 2572 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-processing-f9b8d9b9b-fj2lp" event={"ID":"c9557a6b-396b-4ab0-a3cb-8ddcf3c37f19","Type":"ContainerStarted","Data":"f8f967bd36a7c4796933e89406925560497bc6708a6c9d92251f9e318673d6e3"} Jul 16 00:50:18.481153 ip-10-0-143-200 kubenswrapper[2572]: I0716 00:50:18.481112 2572 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-processing-f9b8d9b9b-fj2lp" event={"ID":"c9557a6b-396b-4ab0-a3cb-8ddcf3c37f19","Type":"ContainerStarted","Data":"b1ba8b690884c5ecdaabe33a70077691645d594508863cad87d849d2800f4d53"} Jul 16 00:50:18.481701 ip-10-0-143-200 kubenswrapper[2572]: I0716 00:50:18.481224 2572 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/payload-processing-f9b8d9b9b-fj2lp" Jul 16 00:50:18.483194 ip-10-0-143-200 kubenswrapper[2572]: I0716 00:50:18.483154 2572 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-7464b7965d-lm97p" event={"ID":"602f254c-3d38-4fa4-87d9-005fcfd1e70e","Type":"ContainerStarted","Data":"45cb0bb19d7cf9b2560e65186cec0b2eb0b41dc9aef2d706267fedf4430afd2c"} Jul 16 00:50:18.483814 ip-10-0-143-200 kubenswrapper[2572]: I0716 00:50:18.483787 2572 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="odh-ai-gateway-infra/maas-api-7464b7965d-lm97p" Jul 16 00:50:18.499794 ip-10-0-143-200 kubenswrapper[2572]: I0716 00:50:18.499707 2572 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-ingress/payload-processing-f9b8d9b9b-fj2lp" podStartSLOduration=1.575826492 podStartE2EDuration="3.499681849s" totalImagesPullingTime="1.923855357s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-16 00:50:15 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-16 00:50:18.497969995 +0000 UTC m=+623.606724668" watchObservedRunningTime="2026-07-16 00:50:18.499681849 +0000 UTC m=+623.608436525" Jul 16 00:50:23.522990 ip-10-0-143-200 kubenswrapper[2572]: I0716 00:50:23.522956 2572 generic.go:356] "Generic (PLEG): container finished" podID="3fc0bdda-3400-495e-b694-4e085c84fd49" containerID="a8d213e2798005b4d5a4a6fc7b66c3dab9558ccdf9536ae18674c1a1f81a1158" exitCode=0 Jul 16 00:50:23.523530 ip-10-0-143-200 kubenswrapper[2572]: I0716 00:50:23.523007 2572 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/maas-default-gateway-openshift-default-56654dcf7-gr57g" event={"ID":"3fc0bdda-3400-495e-b694-4e085c84fd49","Type":"ContainerDied","Data":"a8d213e2798005b4d5a4a6fc7b66c3dab9558ccdf9536ae18674c1a1f81a1158"} Jul 16 00:50:23.523530 ip-10-0-143-200 kubenswrapper[2572]: I0716 00:50:23.523519 2572 scope.go:117] "RemoveContainer" containerID="a8d213e2798005b4d5a4a6fc7b66c3dab9558ccdf9536ae18674c1a1f81a1158" Jul 16 00:50:23.524657 ip-10-0-143-200 kubenswrapper[2572]: I0716 00:50:23.524605 2572 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236228Ki","pods":"250"} Jul 16 00:50:23.524768 ip-10-0-143-200 kubenswrapper[2572]: I0716 00:50:23.524714 2572 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236228Ki","pods":"250"} Jul 16 00:50:23.524768 ip-10-0-143-200 kubenswrapper[2572]: I0716 00:50:23.524762 2572 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236228Ki","pods":"250"} Jul 16 00:50:23.545692 ip-10-0-143-200 kubenswrapper[2572]: I0716 00:50:23.545641 2572 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="odh-ai-gateway-infra/maas-api-7464b7965d-lm97p" podStartSLOduration=7.029737317 podStartE2EDuration="9.54560749s" totalImagesPullingTime="2.515870173s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-16 00:50:14 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-16 00:50:18.515683467 +0000 UTC m=+623.624438144" watchObservedRunningTime="2026-07-16 00:50:23.54560749 +0000 UTC m=+628.654362166" Jul 16 00:50:24.530255 ip-10-0-143-200 kubenswrapper[2572]: I0716 00:50:24.530218 2572 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/maas-default-gateway-openshift-default-56654dcf7-gr57g" event={"ID":"3fc0bdda-3400-495e-b694-4e085c84fd49","Type":"ContainerStarted","Data":"15dfd74ed2e617fa648fc3e81fcb5ad59fcb4a43aea56b1c5f48054fd7320bba"} Jul 16 00:50:24.806822 ip-10-0-143-200 kubenswrapper[2572]: I0716 00:50:24.806717 2572 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="openshift-ingress/maas-default-gateway-openshift-default-56654dcf7-gr57g" Jul 16 00:50:24.807030 ip-10-0-143-200 kubenswrapper[2572]: I0716 00:50:24.806850 2572 patch_prober.go:28] interesting pod/maas-default-gateway-openshift-default-56654dcf7-gr57g container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.132.0.40:15021/healthz/ready\": dial tcp 10.132.0.40:15021: connect: connection refused" start-of-body= Jul 16 00:50:24.807030 ip-10-0-143-200 kubenswrapper[2572]: I0716 00:50:24.806905 2572 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/maas-default-gateway-openshift-default-56654dcf7-gr57g" podUID="3fc0bdda-3400-495e-b694-4e085c84fd49" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.132.0.40:15021/healthz/ready\": dial tcp 10.132.0.40:15021: connect: connection refused" Jul 16 00:50:25.512644 ip-10-0-143-200 kubenswrapper[2572]: I0716 00:50:25.512600 2572 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="odh-ai-gateway-infra/maas-api-7464b7965d-lm97p" Jul 16 00:50:25.806317 ip-10-0-143-200 kubenswrapper[2572]: I0716 00:50:25.806205 2572 patch_prober.go:28] interesting pod/maas-default-gateway-openshift-default-56654dcf7-gr57g container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.132.0.40:15021/healthz/ready\": dial tcp 10.132.0.40:15021: connect: connection refused" start-of-body= Jul 16 00:50:25.806317 ip-10-0-143-200 kubenswrapper[2572]: I0716 00:50:25.806277 2572 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/maas-default-gateway-openshift-default-56654dcf7-gr57g" podUID="3fc0bdda-3400-495e-b694-4e085c84fd49" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.132.0.40:15021/healthz/ready\": dial tcp 10.132.0.40:15021: connect: connection refused" Jul 16 00:50:26.503861 ip-10-0-143-200 kubenswrapper[2572]: I0716 00:50:26.503772 2572 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9gnmq_3a7e030d-fad8-480d-9dc6-fe5057048cf9/konnectivity-agent/0.log" limit=1024 Jul 16 00:50:26.603174 ip-10-0-143-200 kubenswrapper[2572]: I0716 00:50:26.603138 2572 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9gnmq_3a7e030d-fad8-480d-9dc6-fe5057048cf9/konnectivity-agent/0.log" limit=1024 Jul 16 00:50:26.806844 ip-10-0-143-200 kubenswrapper[2572]: I0716 00:50:26.806746 2572 patch_prober.go:28] interesting pod/maas-default-gateway-openshift-default-56654dcf7-gr57g container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.132.0.40:15021/healthz/ready\": dial tcp 10.132.0.40:15021: connect: connection refused" start-of-body= Jul 16 00:50:26.806844 ip-10-0-143-200 kubenswrapper[2572]: I0716 00:50:26.806807 2572 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/maas-default-gateway-openshift-default-56654dcf7-gr57g" podUID="3fc0bdda-3400-495e-b694-4e085c84fd49" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.132.0.40:15021/healthz/ready\": dial tcp 10.132.0.40:15021: connect: connection refused" Jul 16 00:50:27.814882 ip-10-0-143-200 kubenswrapper[2572]: I0716 00:50:27.814842 2572 patch_prober.go:28] interesting pod/maas-default-gateway-openshift-default-56654dcf7-gr57g container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.132.0.40:15021/healthz/ready\": dial tcp 10.132.0.40:15021: connect: connection refused" start-of-body= Jul 16 00:50:27.815447 ip-10-0-143-200 kubenswrapper[2572]: I0716 00:50:27.815412 2572 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/maas-default-gateway-openshift-default-56654dcf7-gr57g" podUID="3fc0bdda-3400-495e-b694-4e085c84fd49" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.132.0.40:15021/healthz/ready\": dial tcp 10.132.0.40:15021: connect: connection refused" Jul 16 00:50:28.810400 ip-10-0-143-200 kubenswrapper[2572]: I0716 00:50:28.810371 2572 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="openshift-ingress/maas-default-gateway-openshift-default-56654dcf7-gr57g" Jul 16 00:50:28.810673 ip-10-0-143-200 kubenswrapper[2572]: I0716 00:50:28.810651 2572 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/maas-default-gateway-openshift-default-56654dcf7-gr57g" Jul 16 00:50:28.811297 ip-10-0-143-200 kubenswrapper[2572]: I0716 00:50:28.811277 2572 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/maas-default-gateway-openshift-default-56654dcf7-gr57g" Jul 16 00:50:29.501314 ip-10-0-143-200 kubenswrapper[2572]: I0716 00:50:29.501283 2572 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/payload-processing-f9b8d9b9b-fj2lp" Jul 16 00:50:42.274469 ip-10-0-143-200 kubenswrapper[2572]: I0716 00:50:42.274426 2572 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9gnmq_3a7e030d-fad8-480d-9dc6-fe5057048cf9/konnectivity-agent/0.log" limit=1024 Jul 16 00:50:51.522153 ip-10-0-143-200 kubenswrapper[2572]: I0716 00:50:51.522108 2572 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["llm/e2e-distinct-simulated-kserve-59975d69f9-cfqcq"] Jul 16 00:50:51.526192 ip-10-0-143-200 kubenswrapper[2572]: I0716 00:50:51.526168 2572 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="llm/e2e-distinct-simulated-kserve-59975d69f9-cfqcq" Jul 16 00:50:51.528924 ip-10-0-143-200 kubenswrapper[2572]: I0716 00:50:51.528897 2572 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"llm\"/\"openshift-service-ca.crt\"" Jul 16 00:50:51.529029 ip-10-0-143-200 kubenswrapper[2572]: I0716 00:50:51.528930 2572 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"llm\"/\"kube-root-ca.crt\"" Jul 16 00:50:51.530246 ip-10-0-143-200 kubenswrapper[2572]: I0716 00:50:51.530223 2572 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"llm\"/\"default-dockercfg-scpnw\"" Jul 16 00:50:51.530246 ip-10-0-143-200 kubenswrapper[2572]: I0716 00:50:51.530226 2572 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"llm\"/\"e2e-distinct-simulated-kserve-self-signed-certs\"" Jul 16 00:50:51.535943 ip-10-0-143-200 kubenswrapper[2572]: I0716 00:50:51.535923 2572 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["llm/e2e-distinct-simulated-kserve-59975d69f9-cfqcq"] Jul 16 00:50:51.540952 ip-10-0-143-200 kubenswrapper[2572]: I0716 00:50:51.540930 2572 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/a9bd8f1c-1abe-4f9b-975e-84b0964c2d3e-model-cache\") pod \"e2e-distinct-simulated-kserve-59975d69f9-cfqcq\" (UID: \"a9bd8f1c-1abe-4f9b-975e-84b0964c2d3e\") " pod="llm/e2e-distinct-simulated-kserve-59975d69f9-cfqcq" Jul 16 00:50:51.541038 ip-10-0-143-200 kubenswrapper[2572]: I0716 00:50:51.540972 2572 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-th258\" (UniqueName: \"kubernetes.io/projected/a9bd8f1c-1abe-4f9b-975e-84b0964c2d3e-kube-api-access-th258\") pod \"e2e-distinct-simulated-kserve-59975d69f9-cfqcq\" (UID: \"a9bd8f1c-1abe-4f9b-975e-84b0964c2d3e\") " pod="llm/e2e-distinct-simulated-kserve-59975d69f9-cfqcq" Jul 16 00:50:51.541038 ip-10-0-143-200 kubenswrapper[2572]: I0716 00:50:51.540994 2572 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/a9bd8f1c-1abe-4f9b-975e-84b0964c2d3e-home\") pod \"e2e-distinct-simulated-kserve-59975d69f9-cfqcq\" (UID: \"a9bd8f1c-1abe-4f9b-975e-84b0964c2d3e\") " pod="llm/e2e-distinct-simulated-kserve-59975d69f9-cfqcq" Jul 16 00:50:51.541038 ip-10-0-143-200 kubenswrapper[2572]: I0716 00:50:51.541009 2572 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/a9bd8f1c-1abe-4f9b-975e-84b0964c2d3e-tls-certs\") pod \"e2e-distinct-simulated-kserve-59975d69f9-cfqcq\" (UID: \"a9bd8f1c-1abe-4f9b-975e-84b0964c2d3e\") " pod="llm/e2e-distinct-simulated-kserve-59975d69f9-cfqcq" Jul 16 00:50:51.541140 ip-10-0-143-200 kubenswrapper[2572]: I0716 00:50:51.541058 2572 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/a9bd8f1c-1abe-4f9b-975e-84b0964c2d3e-tmp-dir\") pod \"e2e-distinct-simulated-kserve-59975d69f9-cfqcq\" (UID: \"a9bd8f1c-1abe-4f9b-975e-84b0964c2d3e\") " pod="llm/e2e-distinct-simulated-kserve-59975d69f9-cfqcq" Jul 16 00:50:51.541140 ip-10-0-143-200 kubenswrapper[2572]: I0716 00:50:51.541117 2572 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/a9bd8f1c-1abe-4f9b-975e-84b0964c2d3e-dshm\") pod \"e2e-distinct-simulated-kserve-59975d69f9-cfqcq\" (UID: \"a9bd8f1c-1abe-4f9b-975e-84b0964c2d3e\") " pod="llm/e2e-distinct-simulated-kserve-59975d69f9-cfqcq" Jul 16 00:50:51.642015 ip-10-0-143-200 kubenswrapper[2572]: I0716 00:50:51.641984 2572 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-th258\" (UniqueName: \"kubernetes.io/projected/a9bd8f1c-1abe-4f9b-975e-84b0964c2d3e-kube-api-access-th258\") pod \"e2e-distinct-simulated-kserve-59975d69f9-cfqcq\" (UID: \"a9bd8f1c-1abe-4f9b-975e-84b0964c2d3e\") " pod="llm/e2e-distinct-simulated-kserve-59975d69f9-cfqcq" Jul 16 00:50:51.642190 ip-10-0-143-200 kubenswrapper[2572]: I0716 00:50:51.642026 2572 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/a9bd8f1c-1abe-4f9b-975e-84b0964c2d3e-home\") pod \"e2e-distinct-simulated-kserve-59975d69f9-cfqcq\" (UID: \"a9bd8f1c-1abe-4f9b-975e-84b0964c2d3e\") " pod="llm/e2e-distinct-simulated-kserve-59975d69f9-cfqcq" Jul 16 00:50:51.642190 ip-10-0-143-200 kubenswrapper[2572]: I0716 00:50:51.642056 2572 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/a9bd8f1c-1abe-4f9b-975e-84b0964c2d3e-tls-certs\") pod \"e2e-distinct-simulated-kserve-59975d69f9-cfqcq\" (UID: \"a9bd8f1c-1abe-4f9b-975e-84b0964c2d3e\") " pod="llm/e2e-distinct-simulated-kserve-59975d69f9-cfqcq" Jul 16 00:50:51.642190 ip-10-0-143-200 kubenswrapper[2572]: I0716 00:50:51.642096 2572 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/a9bd8f1c-1abe-4f9b-975e-84b0964c2d3e-tmp-dir\") pod \"e2e-distinct-simulated-kserve-59975d69f9-cfqcq\" (UID: \"a9bd8f1c-1abe-4f9b-975e-84b0964c2d3e\") " pod="llm/e2e-distinct-simulated-kserve-59975d69f9-cfqcq" Jul 16 00:50:51.642190 ip-10-0-143-200 kubenswrapper[2572]: I0716 00:50:51.642125 2572 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/a9bd8f1c-1abe-4f9b-975e-84b0964c2d3e-dshm\") pod \"e2e-distinct-simulated-kserve-59975d69f9-cfqcq\" (UID: \"a9bd8f1c-1abe-4f9b-975e-84b0964c2d3e\") " pod="llm/e2e-distinct-simulated-kserve-59975d69f9-cfqcq" Jul 16 00:50:51.642365 ip-10-0-143-200 kubenswrapper[2572]: I0716 00:50:51.642224 2572 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/a9bd8f1c-1abe-4f9b-975e-84b0964c2d3e-model-cache\") pod \"e2e-distinct-simulated-kserve-59975d69f9-cfqcq\" (UID: \"a9bd8f1c-1abe-4f9b-975e-84b0964c2d3e\") " pod="llm/e2e-distinct-simulated-kserve-59975d69f9-cfqcq" Jul 16 00:50:51.642484 ip-10-0-143-200 kubenswrapper[2572]: I0716 00:50:51.642458 2572 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/a9bd8f1c-1abe-4f9b-975e-84b0964c2d3e-home\") pod \"e2e-distinct-simulated-kserve-59975d69f9-cfqcq\" (UID: \"a9bd8f1c-1abe-4f9b-975e-84b0964c2d3e\") " pod="llm/e2e-distinct-simulated-kserve-59975d69f9-cfqcq" Jul 16 00:50:51.642546 ip-10-0-143-200 kubenswrapper[2572]: I0716 00:50:51.642496 2572 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/a9bd8f1c-1abe-4f9b-975e-84b0964c2d3e-tmp-dir\") pod \"e2e-distinct-simulated-kserve-59975d69f9-cfqcq\" (UID: \"a9bd8f1c-1abe-4f9b-975e-84b0964c2d3e\") " pod="llm/e2e-distinct-simulated-kserve-59975d69f9-cfqcq" Jul 16 00:50:51.642604 ip-10-0-143-200 kubenswrapper[2572]: I0716 00:50:51.642576 2572 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/a9bd8f1c-1abe-4f9b-975e-84b0964c2d3e-model-cache\") pod \"e2e-distinct-simulated-kserve-59975d69f9-cfqcq\" (UID: \"a9bd8f1c-1abe-4f9b-975e-84b0964c2d3e\") " pod="llm/e2e-distinct-simulated-kserve-59975d69f9-cfqcq" Jul 16 00:50:51.644570 ip-10-0-143-200 kubenswrapper[2572]: I0716 00:50:51.644534 2572 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/a9bd8f1c-1abe-4f9b-975e-84b0964c2d3e-dshm\") pod \"e2e-distinct-simulated-kserve-59975d69f9-cfqcq\" (UID: \"a9bd8f1c-1abe-4f9b-975e-84b0964c2d3e\") " pod="llm/e2e-distinct-simulated-kserve-59975d69f9-cfqcq" Jul 16 00:50:51.644721 ip-10-0-143-200 kubenswrapper[2572]: I0716 00:50:51.644663 2572 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9gnmq_3a7e030d-fad8-480d-9dc6-fe5057048cf9/konnectivity-agent/0.log" limit=1024 Jul 16 00:50:51.644947 ip-10-0-143-200 kubenswrapper[2572]: I0716 00:50:51.644928 2572 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/a9bd8f1c-1abe-4f9b-975e-84b0964c2d3e-tls-certs\") pod \"e2e-distinct-simulated-kserve-59975d69f9-cfqcq\" (UID: \"a9bd8f1c-1abe-4f9b-975e-84b0964c2d3e\") " pod="llm/e2e-distinct-simulated-kserve-59975d69f9-cfqcq" Jul 16 00:50:51.654795 ip-10-0-143-200 kubenswrapper[2572]: I0716 00:50:51.654772 2572 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-th258\" (UniqueName: \"kubernetes.io/projected/a9bd8f1c-1abe-4f9b-975e-84b0964c2d3e-kube-api-access-th258\") pod \"e2e-distinct-simulated-kserve-59975d69f9-cfqcq\" (UID: \"a9bd8f1c-1abe-4f9b-975e-84b0964c2d3e\") " pod="llm/e2e-distinct-simulated-kserve-59975d69f9-cfqcq" Jul 16 00:50:51.837207 ip-10-0-143-200 kubenswrapper[2572]: I0716 00:50:51.837113 2572 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="llm/e2e-distinct-simulated-kserve-59975d69f9-cfqcq" Jul 16 00:50:51.977321 ip-10-0-143-200 kubenswrapper[2572]: I0716 00:50:51.977281 2572 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["llm/e2e-distinct-simulated-kserve-59975d69f9-cfqcq"] Jul 16 00:50:52.671262 ip-10-0-143-200 kubenswrapper[2572]: I0716 00:50:52.671190 2572 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="llm/e2e-distinct-simulated-kserve-59975d69f9-cfqcq" event={"ID":"a9bd8f1c-1abe-4f9b-975e-84b0964c2d3e","Type":"ContainerStarted","Data":"f35485757300e4061162d4f1583e122d329d095963ce5b5df4df5d4c73437f8e"} Jul 16 00:50:53.677890 ip-10-0-143-200 kubenswrapper[2572]: I0716 00:50:53.677853 2572 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="llm/e2e-distinct-simulated-kserve-59975d69f9-cfqcq" event={"ID":"a9bd8f1c-1abe-4f9b-975e-84b0964c2d3e","Type":"ContainerStarted","Data":"9b827f7916b6eba65bd3ba5b972b0ffc7f9ccc0b7a65317ae2b69856ac0001eb"} Jul 16 00:50:53.696371 ip-10-0-143-200 kubenswrapper[2572]: I0716 00:50:53.696320 2572 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="llm/e2e-distinct-simulated-kserve-59975d69f9-cfqcq" podStartSLOduration=1.6478773549999999 podStartE2EDuration="2.69630339s" totalImagesPullingTime="1.048426035s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-16 00:50:51 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-16 00:50:53.694944114 +0000 UTC m=+658.803698790" watchObservedRunningTime="2026-07-16 00:50:53.69630339 +0000 UTC m=+658.805058065" Jul 16 00:51:01.837760 ip-10-0-143-200 kubenswrapper[2572]: I0716 00:51:01.837709 2572 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="llm/e2e-distinct-simulated-kserve-59975d69f9-cfqcq" Jul 16 00:51:01.838212 ip-10-0-143-200 kubenswrapper[2572]: I0716 00:51:01.837785 2572 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="llm/e2e-distinct-simulated-kserve-59975d69f9-cfqcq" Jul 16 00:51:01.850532 ip-10-0-143-200 kubenswrapper[2572]: I0716 00:51:01.850499 2572 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="llm/e2e-distinct-simulated-kserve-59975d69f9-cfqcq" Jul 16 00:51:02.737970 ip-10-0-143-200 kubenswrapper[2572]: I0716 00:51:02.737937 2572 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="llm/e2e-distinct-simulated-kserve-59975d69f9-cfqcq" Jul 16 00:51:09.938651 ip-10-0-143-200 kubenswrapper[2572]: I0716 00:51:09.938602 2572 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["llm/e2e-trlp-test-simulated-kserve-598466548d-hvsmm"] Jul 16 00:51:09.943672 ip-10-0-143-200 kubenswrapper[2572]: I0716 00:51:09.943645 2572 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="llm/e2e-trlp-test-simulated-kserve-598466548d-hvsmm" Jul 16 00:51:09.946567 ip-10-0-143-200 kubenswrapper[2572]: I0716 00:51:09.946539 2572 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"llm\"/\"e2e-trlp-test-simulated-kserve-self-signed-certs\"" Jul 16 00:51:09.952908 ip-10-0-143-200 kubenswrapper[2572]: I0716 00:51:09.952880 2572 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["llm/e2e-trlp-test-simulated-kserve-598466548d-hvsmm"] Jul 16 00:51:09.979862 ip-10-0-143-200 kubenswrapper[2572]: I0716 00:51:09.979830 2572 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9gnmq_3a7e030d-fad8-480d-9dc6-fe5057048cf9/konnectivity-agent/0.log" limit=1024 Jul 16 00:51:10.008961 ip-10-0-143-200 kubenswrapper[2572]: I0716 00:51:10.008928 2572 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/47e15b8d-37df-4292-9c1f-f1ebfed0fd1c-model-cache\") pod \"e2e-trlp-test-simulated-kserve-598466548d-hvsmm\" (UID: \"47e15b8d-37df-4292-9c1f-f1ebfed0fd1c\") " pod="llm/e2e-trlp-test-simulated-kserve-598466548d-hvsmm" Jul 16 00:51:10.009159 ip-10-0-143-200 kubenswrapper[2572]: I0716 00:51:10.008971 2572 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/47e15b8d-37df-4292-9c1f-f1ebfed0fd1c-home\") pod \"e2e-trlp-test-simulated-kserve-598466548d-hvsmm\" (UID: \"47e15b8d-37df-4292-9c1f-f1ebfed0fd1c\") " pod="llm/e2e-trlp-test-simulated-kserve-598466548d-hvsmm" Jul 16 00:51:10.009159 ip-10-0-143-200 kubenswrapper[2572]: I0716 00:51:10.008997 2572 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/47e15b8d-37df-4292-9c1f-f1ebfed0fd1c-tmp-dir\") pod \"e2e-trlp-test-simulated-kserve-598466548d-hvsmm\" (UID: \"47e15b8d-37df-4292-9c1f-f1ebfed0fd1c\") " pod="llm/e2e-trlp-test-simulated-kserve-598466548d-hvsmm" Jul 16 00:51:10.009159 ip-10-0-143-200 kubenswrapper[2572]: I0716 00:51:10.009057 2572 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-jv7cd\" (UniqueName: \"kubernetes.io/projected/47e15b8d-37df-4292-9c1f-f1ebfed0fd1c-kube-api-access-jv7cd\") pod \"e2e-trlp-test-simulated-kserve-598466548d-hvsmm\" (UID: \"47e15b8d-37df-4292-9c1f-f1ebfed0fd1c\") " pod="llm/e2e-trlp-test-simulated-kserve-598466548d-hvsmm" Jul 16 00:51:10.009159 ip-10-0-143-200 kubenswrapper[2572]: I0716 00:51:10.009106 2572 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/47e15b8d-37df-4292-9c1f-f1ebfed0fd1c-dshm\") pod \"e2e-trlp-test-simulated-kserve-598466548d-hvsmm\" (UID: \"47e15b8d-37df-4292-9c1f-f1ebfed0fd1c\") " pod="llm/e2e-trlp-test-simulated-kserve-598466548d-hvsmm" Jul 16 00:51:10.009159 ip-10-0-143-200 kubenswrapper[2572]: I0716 00:51:10.009129 2572 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/47e15b8d-37df-4292-9c1f-f1ebfed0fd1c-tls-certs\") pod \"e2e-trlp-test-simulated-kserve-598466548d-hvsmm\" (UID: \"47e15b8d-37df-4292-9c1f-f1ebfed0fd1c\") " pod="llm/e2e-trlp-test-simulated-kserve-598466548d-hvsmm" Jul 16 00:51:10.110525 ip-10-0-143-200 kubenswrapper[2572]: I0716 00:51:10.110489 2572 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-jv7cd\" (UniqueName: \"kubernetes.io/projected/47e15b8d-37df-4292-9c1f-f1ebfed0fd1c-kube-api-access-jv7cd\") pod \"e2e-trlp-test-simulated-kserve-598466548d-hvsmm\" (UID: \"47e15b8d-37df-4292-9c1f-f1ebfed0fd1c\") " pod="llm/e2e-trlp-test-simulated-kserve-598466548d-hvsmm" Jul 16 00:51:10.110748 ip-10-0-143-200 kubenswrapper[2572]: I0716 00:51:10.110565 2572 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/47e15b8d-37df-4292-9c1f-f1ebfed0fd1c-dshm\") pod \"e2e-trlp-test-simulated-kserve-598466548d-hvsmm\" (UID: \"47e15b8d-37df-4292-9c1f-f1ebfed0fd1c\") " pod="llm/e2e-trlp-test-simulated-kserve-598466548d-hvsmm" Jul 16 00:51:10.110748 ip-10-0-143-200 kubenswrapper[2572]: I0716 00:51:10.110599 2572 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/47e15b8d-37df-4292-9c1f-f1ebfed0fd1c-tls-certs\") pod \"e2e-trlp-test-simulated-kserve-598466548d-hvsmm\" (UID: \"47e15b8d-37df-4292-9c1f-f1ebfed0fd1c\") " pod="llm/e2e-trlp-test-simulated-kserve-598466548d-hvsmm" Jul 16 00:51:10.110748 ip-10-0-143-200 kubenswrapper[2572]: I0716 00:51:10.110685 2572 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/47e15b8d-37df-4292-9c1f-f1ebfed0fd1c-model-cache\") pod \"e2e-trlp-test-simulated-kserve-598466548d-hvsmm\" (UID: \"47e15b8d-37df-4292-9c1f-f1ebfed0fd1c\") " pod="llm/e2e-trlp-test-simulated-kserve-598466548d-hvsmm" Jul 16 00:51:10.110748 ip-10-0-143-200 kubenswrapper[2572]: I0716 00:51:10.110707 2572 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/47e15b8d-37df-4292-9c1f-f1ebfed0fd1c-home\") pod \"e2e-trlp-test-simulated-kserve-598466548d-hvsmm\" (UID: \"47e15b8d-37df-4292-9c1f-f1ebfed0fd1c\") " pod="llm/e2e-trlp-test-simulated-kserve-598466548d-hvsmm" Jul 16 00:51:10.110748 ip-10-0-143-200 kubenswrapper[2572]: I0716 00:51:10.110738 2572 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/47e15b8d-37df-4292-9c1f-f1ebfed0fd1c-tmp-dir\") pod \"e2e-trlp-test-simulated-kserve-598466548d-hvsmm\" (UID: \"47e15b8d-37df-4292-9c1f-f1ebfed0fd1c\") " pod="llm/e2e-trlp-test-simulated-kserve-598466548d-hvsmm" Jul 16 00:51:10.111117 ip-10-0-143-200 kubenswrapper[2572]: I0716 00:51:10.111092 2572 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/47e15b8d-37df-4292-9c1f-f1ebfed0fd1c-model-cache\") pod \"e2e-trlp-test-simulated-kserve-598466548d-hvsmm\" (UID: \"47e15b8d-37df-4292-9c1f-f1ebfed0fd1c\") " pod="llm/e2e-trlp-test-simulated-kserve-598466548d-hvsmm" Jul 16 00:51:10.111187 ip-10-0-143-200 kubenswrapper[2572]: I0716 00:51:10.111122 2572 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/47e15b8d-37df-4292-9c1f-f1ebfed0fd1c-home\") pod \"e2e-trlp-test-simulated-kserve-598466548d-hvsmm\" (UID: \"47e15b8d-37df-4292-9c1f-f1ebfed0fd1c\") " pod="llm/e2e-trlp-test-simulated-kserve-598466548d-hvsmm" Jul 16 00:51:10.111187 ip-10-0-143-200 kubenswrapper[2572]: I0716 00:51:10.111148 2572 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/47e15b8d-37df-4292-9c1f-f1ebfed0fd1c-tmp-dir\") pod \"e2e-trlp-test-simulated-kserve-598466548d-hvsmm\" (UID: \"47e15b8d-37df-4292-9c1f-f1ebfed0fd1c\") " pod="llm/e2e-trlp-test-simulated-kserve-598466548d-hvsmm" Jul 16 00:51:10.113539 ip-10-0-143-200 kubenswrapper[2572]: I0716 00:51:10.113511 2572 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/47e15b8d-37df-4292-9c1f-f1ebfed0fd1c-dshm\") pod \"e2e-trlp-test-simulated-kserve-598466548d-hvsmm\" (UID: \"47e15b8d-37df-4292-9c1f-f1ebfed0fd1c\") " pod="llm/e2e-trlp-test-simulated-kserve-598466548d-hvsmm" Jul 16 00:51:10.113728 ip-10-0-143-200 kubenswrapper[2572]: I0716 00:51:10.113585 2572 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/47e15b8d-37df-4292-9c1f-f1ebfed0fd1c-tls-certs\") pod \"e2e-trlp-test-simulated-kserve-598466548d-hvsmm\" (UID: \"47e15b8d-37df-4292-9c1f-f1ebfed0fd1c\") " pod="llm/e2e-trlp-test-simulated-kserve-598466548d-hvsmm" Jul 16 00:51:10.118985 ip-10-0-143-200 kubenswrapper[2572]: I0716 00:51:10.118948 2572 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-jv7cd\" (UniqueName: \"kubernetes.io/projected/47e15b8d-37df-4292-9c1f-f1ebfed0fd1c-kube-api-access-jv7cd\") pod \"e2e-trlp-test-simulated-kserve-598466548d-hvsmm\" (UID: \"47e15b8d-37df-4292-9c1f-f1ebfed0fd1c\") " pod="llm/e2e-trlp-test-simulated-kserve-598466548d-hvsmm" Jul 16 00:51:10.256815 ip-10-0-143-200 kubenswrapper[2572]: I0716 00:51:10.256777 2572 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="llm/e2e-trlp-test-simulated-kserve-598466548d-hvsmm" Jul 16 00:51:10.390263 ip-10-0-143-200 kubenswrapper[2572]: I0716 00:51:10.390236 2572 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["llm/e2e-trlp-test-simulated-kserve-598466548d-hvsmm"] Jul 16 00:51:10.392272 ip-10-0-143-200 kubenswrapper[2572]: W0716 00:51:10.392245 2572 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod47e15b8d_37df_4292_9c1f_f1ebfed0fd1c.slice/crio-0e431a78f3e395c156b681b87ad9afc0634bfef4bedbcdd8e9bb7c4b5c776551 WatchSource:0}: Error finding container 0e431a78f3e395c156b681b87ad9afc0634bfef4bedbcdd8e9bb7c4b5c776551: Status 404 returned error can't find the container with id 0e431a78f3e395c156b681b87ad9afc0634bfef4bedbcdd8e9bb7c4b5c776551 Jul 16 00:51:10.393689 ip-10-0-143-200 kubenswrapper[2572]: I0716 00:51:10.393671 2572 provider.go:93] Refreshing cache for provider: *credentialprovider.defaultDockerConfigProvider Jul 16 00:51:10.769406 ip-10-0-143-200 kubenswrapper[2572]: I0716 00:51:10.769368 2572 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="llm/e2e-trlp-test-simulated-kserve-598466548d-hvsmm" event={"ID":"47e15b8d-37df-4292-9c1f-f1ebfed0fd1c","Type":"ContainerStarted","Data":"b9f390904bb01dd5f508d612c68a21d2a251cbf6bd4645c2afe67146131124de"} Jul 16 00:51:10.769585 ip-10-0-143-200 kubenswrapper[2572]: I0716 00:51:10.769435 2572 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="llm/e2e-trlp-test-simulated-kserve-598466548d-hvsmm" event={"ID":"47e15b8d-37df-4292-9c1f-f1ebfed0fd1c","Type":"ContainerStarted","Data":"0e431a78f3e395c156b681b87ad9afc0634bfef4bedbcdd8e9bb7c4b5c776551"} Jul 16 00:51:10.789505 ip-10-0-143-200 kubenswrapper[2572]: I0716 00:51:10.789452 2572 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="llm/e2e-trlp-test-simulated-kserve-598466548d-hvsmm" podStartSLOduration=1.614765548 podStartE2EDuration="1.789436725s" totalImagesPullingTime="174.671177ms" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-16 00:51:09 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-16 00:51:10.786219335 +0000 UTC m=+675.894974009" watchObservedRunningTime="2026-07-16 00:51:10.789436725 +0000 UTC m=+675.898191400" Jul 16 00:51:14.668833 ip-10-0-143-200 kubenswrapper[2572]: I0716 00:51:14.668801 2572 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9gnmq_3a7e030d-fad8-480d-9dc6-fe5057048cf9/konnectivity-agent/0.log" limit=1024 Jul 16 00:51:20.257475 ip-10-0-143-200 kubenswrapper[2572]: I0716 00:51:20.257429 2572 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="llm/e2e-trlp-test-simulated-kserve-598466548d-hvsmm" Jul 16 00:51:20.257999 ip-10-0-143-200 kubenswrapper[2572]: I0716 00:51:20.257529 2572 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="llm/e2e-trlp-test-simulated-kserve-598466548d-hvsmm" Jul 16 00:51:20.269963 ip-10-0-143-200 kubenswrapper[2572]: I0716 00:51:20.269935 2572 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="llm/e2e-trlp-test-simulated-kserve-598466548d-hvsmm" Jul 16 00:51:20.834277 ip-10-0-143-200 kubenswrapper[2572]: I0716 00:51:20.834248 2572 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="llm/e2e-trlp-test-simulated-kserve-598466548d-hvsmm" Jul 16 00:51:22.589760 ip-10-0-143-200 kubenswrapper[2572]: I0716 00:51:22.589711 2572 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9gnmq_3a7e030d-fad8-480d-9dc6-fe5057048cf9/konnectivity-agent/0.log" limit=1024 Jul 16 00:51:33.084103 ip-10-0-143-200 kubenswrapper[2572]: I0716 00:51:33.084067 2572 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9gnmq_3a7e030d-fad8-480d-9dc6-fe5057048cf9/konnectivity-agent/0.log" limit=1024 Jul 16 00:52:11.704242 ip-10-0-143-200 kubenswrapper[2572]: I0716 00:52:11.704154 2572 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9gnmq_3a7e030d-fad8-480d-9dc6-fe5057048cf9/konnectivity-agent/0.log" limit=1024 Jul 16 00:52:11.771570 ip-10-0-143-200 kubenswrapper[2572]: I0716 00:52:11.771533 2572 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9gnmq_3a7e030d-fad8-480d-9dc6-fe5057048cf9/konnectivity-agent/0.log" limit=1024 Jul 16 00:52:41.879939 ip-10-0-143-200 kubenswrapper[2572]: I0716 00:52:41.879894 2572 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9gnmq_3a7e030d-fad8-480d-9dc6-fe5057048cf9/konnectivity-agent/0.log" limit=1024 Jul 16 00:52:41.973581 ip-10-0-143-200 kubenswrapper[2572]: I0716 00:52:41.973531 2572 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9gnmq_3a7e030d-fad8-480d-9dc6-fe5057048cf9/konnectivity-agent/0.log" limit=1024 Jul 16 00:52:43.009484 ip-10-0-143-200 kubenswrapper[2572]: I0716 00:52:43.009449 2572 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9gnmq_3a7e030d-fad8-480d-9dc6-fe5057048cf9/konnectivity-agent/0.log" limit=1024 Jul 16 00:52:43.108113 ip-10-0-143-200 kubenswrapper[2572]: I0716 00:52:43.108076 2572 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9gnmq_3a7e030d-fad8-480d-9dc6-fe5057048cf9/konnectivity-agent/0.log" limit=1024 Jul 16 00:53:28.440257 ip-10-0-143-200 kubenswrapper[2572]: I0716 00:53:28.440166 2572 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9gnmq_3a7e030d-fad8-480d-9dc6-fe5057048cf9/konnectivity-agent/0.log" limit=1024 Jul 16 00:53:43.464073 ip-10-0-143-200 kubenswrapper[2572]: I0716 00:53:43.464038 2572 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9gnmq_3a7e030d-fad8-480d-9dc6-fe5057048cf9/konnectivity-agent/0.log" limit=1024 Jul 16 00:53:53.815251 ip-10-0-143-200 kubenswrapper[2572]: I0716 00:53:53.815213 2572 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9gnmq_3a7e030d-fad8-480d-9dc6-fe5057048cf9/konnectivity-agent/0.log" limit=1024 Jul 16 00:53:53.895617 ip-10-0-143-200 kubenswrapper[2572]: I0716 00:53:53.895587 2572 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["opendatahub/maas-controller-79cf8b95fb-9jrsf"] Jul 16 00:53:53.895874 ip-10-0-143-200 kubenswrapper[2572]: I0716 00:53:53.895849 2572 kuberuntime_container.go:873] "Killing container with a grace period" pod="opendatahub/maas-controller-79cf8b95fb-9jrsf" podUID="0039286c-075d-49a3-93c5-94b605749489" containerName="manager" containerID="cri-o://554971dd12abb0dd2b4f1d5a171714fed8b2558820a9f49e0b2feb7e87bfc957" gracePeriod=30 Jul 16 00:53:54.152967 ip-10-0-143-200 kubenswrapper[2572]: I0716 00:53:54.152941 2572 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="opendatahub/maas-controller-79cf8b95fb-9jrsf" Jul 16 00:53:54.252435 ip-10-0-143-200 kubenswrapper[2572]: I0716 00:53:54.252404 2572 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-lt58f\" (UniqueName: \"kubernetes.io/projected/0039286c-075d-49a3-93c5-94b605749489-kube-api-access-lt58f\") pod \"0039286c-075d-49a3-93c5-94b605749489\" (UID: \"0039286c-075d-49a3-93c5-94b605749489\") " Jul 16 00:53:54.252599 ip-10-0-143-200 kubenswrapper[2572]: I0716 00:53:54.252517 2572 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"webhook-cert\" (UniqueName: \"kubernetes.io/secret/0039286c-075d-49a3-93c5-94b605749489-webhook-cert\") pod \"0039286c-075d-49a3-93c5-94b605749489\" (UID: \"0039286c-075d-49a3-93c5-94b605749489\") " Jul 16 00:53:54.254360 ip-10-0-143-200 kubenswrapper[2572]: I0716 00:53:54.254328 2572 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/0039286c-075d-49a3-93c5-94b605749489-kube-api-access-lt58f" (OuterVolumeSpecName: "kube-api-access-lt58f") pod "0039286c-075d-49a3-93c5-94b605749489" (UID: "0039286c-075d-49a3-93c5-94b605749489"). InnerVolumeSpecName "kube-api-access-lt58f". PluginName "kubernetes.io/projected", VolumeGIDValue "" Jul 16 00:53:54.254465 ip-10-0-143-200 kubenswrapper[2572]: I0716 00:53:54.254401 2572 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/0039286c-075d-49a3-93c5-94b605749489-webhook-cert" (OuterVolumeSpecName: "webhook-cert") pod "0039286c-075d-49a3-93c5-94b605749489" (UID: "0039286c-075d-49a3-93c5-94b605749489"). InnerVolumeSpecName "webhook-cert". PluginName "kubernetes.io/secret", VolumeGIDValue "" Jul 16 00:53:54.353522 ip-10-0-143-200 kubenswrapper[2572]: I0716 00:53:54.353442 2572 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-lt58f\" (UniqueName: \"kubernetes.io/projected/0039286c-075d-49a3-93c5-94b605749489-kube-api-access-lt58f\") on node \"ip-10-0-143-200.ec2.internal\" DevicePath \"\"" Jul 16 00:53:54.353522 ip-10-0-143-200 kubenswrapper[2572]: I0716 00:53:54.353471 2572 reconciler_common.go:299] "Volume detached for volume \"webhook-cert\" (UniqueName: \"kubernetes.io/secret/0039286c-075d-49a3-93c5-94b605749489-webhook-cert\") on node \"ip-10-0-143-200.ec2.internal\" DevicePath \"\"" Jul 16 00:53:54.604062 ip-10-0-143-200 kubenswrapper[2572]: I0716 00:53:54.603969 2572 generic.go:356] "Generic (PLEG): container finished" podID="0039286c-075d-49a3-93c5-94b605749489" containerID="554971dd12abb0dd2b4f1d5a171714fed8b2558820a9f49e0b2feb7e87bfc957" exitCode=0 Jul 16 00:53:54.604062 ip-10-0-143-200 kubenswrapper[2572]: I0716 00:53:54.604021 2572 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="opendatahub/maas-controller-79cf8b95fb-9jrsf" Jul 16 00:53:54.604243 ip-10-0-143-200 kubenswrapper[2572]: I0716 00:53:54.604060 2572 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="opendatahub/maas-controller-79cf8b95fb-9jrsf" event={"ID":"0039286c-075d-49a3-93c5-94b605749489","Type":"ContainerDied","Data":"554971dd12abb0dd2b4f1d5a171714fed8b2558820a9f49e0b2feb7e87bfc957"} Jul 16 00:53:54.604243 ip-10-0-143-200 kubenswrapper[2572]: I0716 00:53:54.604116 2572 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="opendatahub/maas-controller-79cf8b95fb-9jrsf" event={"ID":"0039286c-075d-49a3-93c5-94b605749489","Type":"ContainerDied","Data":"5ce1390ea9a335ee19dac3f472a7877807a6581e7a6c2e3a5c5e1e03562c7411"} Jul 16 00:53:54.604243 ip-10-0-143-200 kubenswrapper[2572]: I0716 00:53:54.604138 2572 scope.go:117] "RemoveContainer" containerID="554971dd12abb0dd2b4f1d5a171714fed8b2558820a9f49e0b2feb7e87bfc957" Jul 16 00:53:54.613873 ip-10-0-143-200 kubenswrapper[2572]: I0716 00:53:54.613854 2572 scope.go:117] "RemoveContainer" containerID="554971dd12abb0dd2b4f1d5a171714fed8b2558820a9f49e0b2feb7e87bfc957" Jul 16 00:53:54.614164 ip-10-0-143-200 kubenswrapper[2572]: E0716 00:53:54.614145 2572 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"554971dd12abb0dd2b4f1d5a171714fed8b2558820a9f49e0b2feb7e87bfc957\": container with ID starting with 554971dd12abb0dd2b4f1d5a171714fed8b2558820a9f49e0b2feb7e87bfc957 not found: ID does not exist" containerID="554971dd12abb0dd2b4f1d5a171714fed8b2558820a9f49e0b2feb7e87bfc957" Jul 16 00:53:54.614231 ip-10-0-143-200 kubenswrapper[2572]: I0716 00:53:54.614171 2572 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"554971dd12abb0dd2b4f1d5a171714fed8b2558820a9f49e0b2feb7e87bfc957"} err="failed to get container status \"554971dd12abb0dd2b4f1d5a171714fed8b2558820a9f49e0b2feb7e87bfc957\": rpc error: code = NotFound desc = could not find container \"554971dd12abb0dd2b4f1d5a171714fed8b2558820a9f49e0b2feb7e87bfc957\": container with ID starting with 554971dd12abb0dd2b4f1d5a171714fed8b2558820a9f49e0b2feb7e87bfc957 not found: ID does not exist" Jul 16 00:53:54.633530 ip-10-0-143-200 kubenswrapper[2572]: I0716 00:53:54.633498 2572 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["opendatahub/maas-controller-79cf8b95fb-9jrsf"] Jul 16 00:53:54.635993 ip-10-0-143-200 kubenswrapper[2572]: I0716 00:53:54.635965 2572 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["opendatahub/maas-controller-79cf8b95fb-9jrsf"] Jul 16 00:53:55.418158 ip-10-0-143-200 kubenswrapper[2572]: I0716 00:53:55.418115 2572 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9gnmq_3a7e030d-fad8-480d-9dc6-fe5057048cf9/konnectivity-agent/0.log" limit=1024 Jul 16 00:53:55.450475 ip-10-0-143-200 kubenswrapper[2572]: I0716 00:53:55.450445 2572 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="0039286c-075d-49a3-93c5-94b605749489" path="/var/lib/kubelet/pods/0039286c-075d-49a3-93c5-94b605749489/volumes" Jul 16 00:53:55.519793 ip-10-0-143-200 kubenswrapper[2572]: I0716 00:53:55.519757 2572 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["opendatahub/maas-controller-79cf8b95fb-7nqz7"] Jul 16 00:53:55.520244 ip-10-0-143-200 kubenswrapper[2572]: I0716 00:53:55.520227 2572 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="0039286c-075d-49a3-93c5-94b605749489" containerName="manager" Jul 16 00:53:55.520244 ip-10-0-143-200 kubenswrapper[2572]: I0716 00:53:55.520246 2572 state_mem.go:107] "Deleted CPUSet assignment" podUID="0039286c-075d-49a3-93c5-94b605749489" containerName="manager" Jul 16 00:53:55.520347 ip-10-0-143-200 kubenswrapper[2572]: I0716 00:53:55.520321 2572 memory_manager.go:368] "RemoveStaleState removing state" podUID="0039286c-075d-49a3-93c5-94b605749489" containerName="manager" Jul 16 00:53:55.523092 ip-10-0-143-200 kubenswrapper[2572]: I0716 00:53:55.523068 2572 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="opendatahub/maas-controller-79cf8b95fb-7nqz7" Jul 16 00:53:55.529693 ip-10-0-143-200 kubenswrapper[2572]: I0716 00:53:55.529672 2572 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"opendatahub\"/\"maas-controller-dockercfg-nvvzq\"" Jul 16 00:53:55.529848 ip-10-0-143-200 kubenswrapper[2572]: I0716 00:53:55.529707 2572 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"opendatahub\"/\"maas-parameters\"" Jul 16 00:53:55.529848 ip-10-0-143-200 kubenswrapper[2572]: I0716 00:53:55.529707 2572 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"opendatahub\"/\"maas-controller-webhook-cert\"" Jul 16 00:53:55.555071 ip-10-0-143-200 kubenswrapper[2572]: I0716 00:53:55.555043 2572 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["opendatahub/maas-controller-79cf8b95fb-7nqz7"] Jul 16 00:53:55.565051 ip-10-0-143-200 kubenswrapper[2572]: I0716 00:53:55.565020 2572 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-dxvb5\" (UniqueName: \"kubernetes.io/projected/25d23071-268d-46ed-a5b5-9ddc749fd269-kube-api-access-dxvb5\") pod \"maas-controller-79cf8b95fb-7nqz7\" (UID: \"25d23071-268d-46ed-a5b5-9ddc749fd269\") " pod="opendatahub/maas-controller-79cf8b95fb-7nqz7" Jul 16 00:53:55.565176 ip-10-0-143-200 kubenswrapper[2572]: I0716 00:53:55.565087 2572 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"webhook-cert\" (UniqueName: \"kubernetes.io/secret/25d23071-268d-46ed-a5b5-9ddc749fd269-webhook-cert\") pod \"maas-controller-79cf8b95fb-7nqz7\" (UID: \"25d23071-268d-46ed-a5b5-9ddc749fd269\") " pod="opendatahub/maas-controller-79cf8b95fb-7nqz7" Jul 16 00:53:55.666387 ip-10-0-143-200 kubenswrapper[2572]: I0716 00:53:55.666350 2572 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-dxvb5\" (UniqueName: \"kubernetes.io/projected/25d23071-268d-46ed-a5b5-9ddc749fd269-kube-api-access-dxvb5\") pod \"maas-controller-79cf8b95fb-7nqz7\" (UID: \"25d23071-268d-46ed-a5b5-9ddc749fd269\") " pod="opendatahub/maas-controller-79cf8b95fb-7nqz7" Jul 16 00:53:55.666606 ip-10-0-143-200 kubenswrapper[2572]: I0716 00:53:55.666412 2572 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"webhook-cert\" (UniqueName: \"kubernetes.io/secret/25d23071-268d-46ed-a5b5-9ddc749fd269-webhook-cert\") pod \"maas-controller-79cf8b95fb-7nqz7\" (UID: \"25d23071-268d-46ed-a5b5-9ddc749fd269\") " pod="opendatahub/maas-controller-79cf8b95fb-7nqz7" Jul 16 00:53:55.669014 ip-10-0-143-200 kubenswrapper[2572]: I0716 00:53:55.668953 2572 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"webhook-cert\" (UniqueName: \"kubernetes.io/secret/25d23071-268d-46ed-a5b5-9ddc749fd269-webhook-cert\") pod \"maas-controller-79cf8b95fb-7nqz7\" (UID: \"25d23071-268d-46ed-a5b5-9ddc749fd269\") " pod="opendatahub/maas-controller-79cf8b95fb-7nqz7" Jul 16 00:53:55.680412 ip-10-0-143-200 kubenswrapper[2572]: I0716 00:53:55.680386 2572 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-dxvb5\" (UniqueName: \"kubernetes.io/projected/25d23071-268d-46ed-a5b5-9ddc749fd269-kube-api-access-dxvb5\") pod \"maas-controller-79cf8b95fb-7nqz7\" (UID: \"25d23071-268d-46ed-a5b5-9ddc749fd269\") " pod="opendatahub/maas-controller-79cf8b95fb-7nqz7" Jul 16 00:53:55.834640 ip-10-0-143-200 kubenswrapper[2572]: I0716 00:53:55.834596 2572 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="opendatahub/maas-controller-79cf8b95fb-7nqz7" Jul 16 00:53:55.981447 ip-10-0-143-200 kubenswrapper[2572]: I0716 00:53:55.981382 2572 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["opendatahub/maas-controller-79cf8b95fb-7nqz7"] Jul 16 00:53:55.983949 ip-10-0-143-200 kubenswrapper[2572]: W0716 00:53:55.983915 2572 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod25d23071_268d_46ed_a5b5_9ddc749fd269.slice/crio-3d2db973e7f96a6e610eca82ef07c6c737f4ff504a37cb197c3b7658bbeb0f9a WatchSource:0}: Error finding container 3d2db973e7f96a6e610eca82ef07c6c737f4ff504a37cb197c3b7658bbeb0f9a: Status 404 returned error can't find the container with id 3d2db973e7f96a6e610eca82ef07c6c737f4ff504a37cb197c3b7658bbeb0f9a Jul 16 00:53:56.615689 ip-10-0-143-200 kubenswrapper[2572]: I0716 00:53:56.615661 2572 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="opendatahub/maas-controller-79cf8b95fb-7nqz7" event={"ID":"25d23071-268d-46ed-a5b5-9ddc749fd269","Type":"ContainerStarted","Data":"3d2db973e7f96a6e610eca82ef07c6c737f4ff504a37cb197c3b7658bbeb0f9a"} Jul 16 00:53:57.622752 ip-10-0-143-200 kubenswrapper[2572]: I0716 00:53:57.622714 2572 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="opendatahub/maas-controller-79cf8b95fb-7nqz7" event={"ID":"25d23071-268d-46ed-a5b5-9ddc749fd269","Type":"ContainerStarted","Data":"dccdb3cf2ec953a054180cd22c8b340df3d132dad7f20b584fab6be0fbf8b623"} Jul 16 00:53:57.623185 ip-10-0-143-200 kubenswrapper[2572]: I0716 00:53:57.622786 2572 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="opendatahub/maas-controller-79cf8b95fb-7nqz7" Jul 16 00:53:57.650134 ip-10-0-143-200 kubenswrapper[2572]: I0716 00:53:57.650072 2572 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="opendatahub/maas-controller-79cf8b95fb-7nqz7" podStartSLOduration=2.040213868 podStartE2EDuration="2.65005726s" totalImagesPullingTime="609.843392ms" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-16 00:53:55 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-16 00:53:57.649108931 +0000 UTC m=+842.757863607" watchObservedRunningTime="2026-07-16 00:53:57.65005726 +0000 UTC m=+842.758811936" Jul 16 00:54:08.631485 ip-10-0-143-200 kubenswrapper[2572]: I0716 00:54:08.631451 2572 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="opendatahub/maas-controller-79cf8b95fb-7nqz7" Jul 16 00:54:11.238073 ip-10-0-143-200 kubenswrapper[2572]: I0716 00:54:11.238036 2572 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9gnmq_3a7e030d-fad8-480d-9dc6-fe5057048cf9/konnectivity-agent/0.log" limit=1024 Jul 16 00:54:40.173443 ip-10-0-143-200 kubenswrapper[2572]: I0716 00:54:40.173403 2572 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9gnmq_3a7e030d-fad8-480d-9dc6-fe5057048cf9/konnectivity-agent/0.log" limit=1024 Jul 16 00:54:40.269949 ip-10-0-143-200 kubenswrapper[2572]: I0716 00:54:40.269910 2572 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9gnmq_3a7e030d-fad8-480d-9dc6-fe5057048cf9/konnectivity-agent/0.log" limit=1024 Jul 16 00:54:40.396995 ip-10-0-143-200 kubenswrapper[2572]: I0716 00:54:40.396956 2572 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9gnmq_3a7e030d-fad8-480d-9dc6-fe5057048cf9/konnectivity-agent/0.log" limit=1024 Jul 16 00:54:46.306805 ip-10-0-143-200 kubenswrapper[2572]: I0716 00:54:46.306763 2572 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9gnmq_3a7e030d-fad8-480d-9dc6-fe5057048cf9/konnectivity-agent/0.log" limit=1024 Jul 16 00:54:46.408041 ip-10-0-143-200 kubenswrapper[2572]: I0716 00:54:46.408003 2572 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9gnmq_3a7e030d-fad8-480d-9dc6-fe5057048cf9/konnectivity-agent/0.log" limit=1024 Jul 16 00:54:46.504504 ip-10-0-143-200 kubenswrapper[2572]: I0716 00:54:46.504465 2572 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9gnmq_3a7e030d-fad8-480d-9dc6-fe5057048cf9/konnectivity-agent/0.log" limit=1024 Jul 16 00:54:46.659832 ip-10-0-143-200 kubenswrapper[2572]: I0716 00:54:46.659728 2572 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9gnmq_3a7e030d-fad8-480d-9dc6-fe5057048cf9/konnectivity-agent/0.log" limit=1024 Jul 16 00:54:52.776399 ip-10-0-143-200 kubenswrapper[2572]: I0716 00:54:52.776315 2572 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9gnmq_3a7e030d-fad8-480d-9dc6-fe5057048cf9/konnectivity-agent/0.log" limit=1024 Jul 16 00:54:52.869087 ip-10-0-143-200 kubenswrapper[2572]: I0716 00:54:52.869042 2572 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9gnmq_3a7e030d-fad8-480d-9dc6-fe5057048cf9/konnectivity-agent/0.log" limit=1024 Jul 16 00:54:55.402984 ip-10-0-143-200 kubenswrapper[2572]: I0716 00:54:55.402951 2572 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-console-operator_console-operator-7866497494-pjc6j_a64bcd05-8596-4c12-b88e-e974d676e25f/console-operator/2.log" Jul 16 00:54:55.404656 ip-10-0-143-200 kubenswrapper[2572]: I0716 00:54:55.404611 2572 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-console-operator_console-operator-7866497494-pjc6j_a64bcd05-8596-4c12-b88e-e974d676e25f/console-operator/2.log" Jul 16 00:54:55.407744 ip-10-0-143-200 kubenswrapper[2572]: I0716 00:54:55.407724 2572 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ovn-kubernetes_ovnkube-node-fp78g_aa66f2b6-0ad8-45d0-8ff5-40fc21966b5a/ovn-acl-logging/0.log" Jul 16 00:54:55.409382 ip-10-0-143-200 kubenswrapper[2572]: I0716 00:54:55.409363 2572 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ovn-kubernetes_ovnkube-node-fp78g_aa66f2b6-0ad8-45d0-8ff5-40fc21966b5a/ovn-acl-logging/0.log" Jul 16 00:55:45.899004 ip-10-0-143-200 kubenswrapper[2572]: I0716 00:55:45.898967 2572 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9gnmq_3a7e030d-fad8-480d-9dc6-fe5057048cf9/konnectivity-agent/0.log" limit=1024 Jul 16 00:57:33.438783 ip-10-0-143-200 kubenswrapper[2572]: I0716 00:57:33.438732 2572 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9gnmq_3a7e030d-fad8-480d-9dc6-fe5057048cf9/konnectivity-agent/0.log" limit=1024 Jul 16 00:57:33.501205 ip-10-0-143-200 kubenswrapper[2572]: I0716 00:57:33.501172 2572 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9gnmq_3a7e030d-fad8-480d-9dc6-fe5057048cf9/konnectivity-agent/0.log" limit=1024 Jul 16 00:58:46.992422 ip-10-0-143-200 kubenswrapper[2572]: I0716 00:58:46.992385 2572 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9gnmq_3a7e030d-fad8-480d-9dc6-fe5057048cf9/konnectivity-agent/0.log" limit=1024 Jul 16 00:58:48.978429 ip-10-0-143-200 kubenswrapper[2572]: I0716 00:58:48.978389 2572 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["kuadrant-system/kuadrant-operator-controller-manager-86878c769f-w87nk"] Jul 16 00:58:48.978896 ip-10-0-143-200 kubenswrapper[2572]: I0716 00:58:48.978588 2572 kuberuntime_container.go:873] "Killing container with a grace period" pod="kuadrant-system/kuadrant-operator-controller-manager-86878c769f-w87nk" podUID="cc2779f5-93cd-4ea3-a618-c8a51ed440b4" containerName="manager" containerID="cri-o://61ab879bfde8f1f8c87e65d286100317a66d9d7685eb11f3921f7a279db505f0" gracePeriod=10 Jul 16 00:58:51.549721 ip-10-0-143-200 kubenswrapper[2572]: I0716 00:58:51.549617 2572 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/kuadrant-operator-controller-manager-86878c769f-w87nk" Jul 16 00:58:51.633062 ip-10-0-143-200 kubenswrapper[2572]: I0716 00:58:51.633026 2572 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-z5qwh\" (UniqueName: \"kubernetes.io/projected/cc2779f5-93cd-4ea3-a618-c8a51ed440b4-kube-api-access-z5qwh\") pod \"cc2779f5-93cd-4ea3-a618-c8a51ed440b4\" (UID: \"cc2779f5-93cd-4ea3-a618-c8a51ed440b4\") " Jul 16 00:58:51.633231 ip-10-0-143-200 kubenswrapper[2572]: I0716 00:58:51.633163 2572 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"extensions-socket-volume\" (UniqueName: \"kubernetes.io/empty-dir/cc2779f5-93cd-4ea3-a618-c8a51ed440b4-extensions-socket-volume\") pod \"cc2779f5-93cd-4ea3-a618-c8a51ed440b4\" (UID: \"cc2779f5-93cd-4ea3-a618-c8a51ed440b4\") " Jul 16 00:58:51.633528 ip-10-0-143-200 kubenswrapper[2572]: I0716 00:58:51.633504 2572 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/cc2779f5-93cd-4ea3-a618-c8a51ed440b4-extensions-socket-volume" (OuterVolumeSpecName: "extensions-socket-volume") pod "cc2779f5-93cd-4ea3-a618-c8a51ed440b4" (UID: "cc2779f5-93cd-4ea3-a618-c8a51ed440b4"). InnerVolumeSpecName "extensions-socket-volume". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Jul 16 00:58:51.635155 ip-10-0-143-200 kubenswrapper[2572]: I0716 00:58:51.635135 2572 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/cc2779f5-93cd-4ea3-a618-c8a51ed440b4-kube-api-access-z5qwh" (OuterVolumeSpecName: "kube-api-access-z5qwh") pod "cc2779f5-93cd-4ea3-a618-c8a51ed440b4" (UID: "cc2779f5-93cd-4ea3-a618-c8a51ed440b4"). InnerVolumeSpecName "kube-api-access-z5qwh". PluginName "kubernetes.io/projected", VolumeGIDValue "" Jul 16 00:58:51.734562 ip-10-0-143-200 kubenswrapper[2572]: I0716 00:58:51.734508 2572 reconciler_common.go:299] "Volume detached for volume \"extensions-socket-volume\" (UniqueName: \"kubernetes.io/empty-dir/cc2779f5-93cd-4ea3-a618-c8a51ed440b4-extensions-socket-volume\") on node \"ip-10-0-143-200.ec2.internal\" DevicePath \"\"" Jul 16 00:58:51.734562 ip-10-0-143-200 kubenswrapper[2572]: I0716 00:58:51.734556 2572 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-z5qwh\" (UniqueName: \"kubernetes.io/projected/cc2779f5-93cd-4ea3-a618-c8a51ed440b4-kube-api-access-z5qwh\") on node \"ip-10-0-143-200.ec2.internal\" DevicePath \"\"" Jul 16 00:58:52.146232 ip-10-0-143-200 kubenswrapper[2572]: I0716 00:58:52.146200 2572 generic.go:356] "Generic (PLEG): container finished" podID="cc2779f5-93cd-4ea3-a618-c8a51ed440b4" containerID="61ab879bfde8f1f8c87e65d286100317a66d9d7685eb11f3921f7a279db505f0" exitCode=0 Jul 16 00:58:52.146792 ip-10-0-143-200 kubenswrapper[2572]: I0716 00:58:52.146253 2572 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/kuadrant-operator-controller-manager-86878c769f-w87nk" Jul 16 00:58:52.146792 ip-10-0-143-200 kubenswrapper[2572]: I0716 00:58:52.146272 2572 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/kuadrant-operator-controller-manager-86878c769f-w87nk" event={"ID":"cc2779f5-93cd-4ea3-a618-c8a51ed440b4","Type":"ContainerDied","Data":"61ab879bfde8f1f8c87e65d286100317a66d9d7685eb11f3921f7a279db505f0"} Jul 16 00:58:52.146792 ip-10-0-143-200 kubenswrapper[2572]: I0716 00:58:52.146314 2572 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/kuadrant-operator-controller-manager-86878c769f-w87nk" event={"ID":"cc2779f5-93cd-4ea3-a618-c8a51ed440b4","Type":"ContainerDied","Data":"45cc4754cc69e325232a0c5f44fa5bc52cb74ae7a010ce8da982fb121bf2fad7"} Jul 16 00:58:52.146792 ip-10-0-143-200 kubenswrapper[2572]: I0716 00:58:52.146336 2572 scope.go:117] "RemoveContainer" containerID="61ab879bfde8f1f8c87e65d286100317a66d9d7685eb11f3921f7a279db505f0" Jul 16 00:58:52.156741 ip-10-0-143-200 kubenswrapper[2572]: I0716 00:58:52.156722 2572 scope.go:117] "RemoveContainer" containerID="61ab879bfde8f1f8c87e65d286100317a66d9d7685eb11f3921f7a279db505f0" Jul 16 00:58:52.156992 ip-10-0-143-200 kubenswrapper[2572]: E0716 00:58:52.156973 2572 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"61ab879bfde8f1f8c87e65d286100317a66d9d7685eb11f3921f7a279db505f0\": container with ID starting with 61ab879bfde8f1f8c87e65d286100317a66d9d7685eb11f3921f7a279db505f0 not found: ID does not exist" containerID="61ab879bfde8f1f8c87e65d286100317a66d9d7685eb11f3921f7a279db505f0" Jul 16 00:58:52.157037 ip-10-0-143-200 kubenswrapper[2572]: I0716 00:58:52.156999 2572 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"61ab879bfde8f1f8c87e65d286100317a66d9d7685eb11f3921f7a279db505f0"} err="failed to get container status \"61ab879bfde8f1f8c87e65d286100317a66d9d7685eb11f3921f7a279db505f0\": rpc error: code = NotFound desc = could not find container \"61ab879bfde8f1f8c87e65d286100317a66d9d7685eb11f3921f7a279db505f0\": container with ID starting with 61ab879bfde8f1f8c87e65d286100317a66d9d7685eb11f3921f7a279db505f0 not found: ID does not exist" Jul 16 00:58:52.169495 ip-10-0-143-200 kubenswrapper[2572]: I0716 00:58:52.169468 2572 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["kuadrant-system/kuadrant-operator-controller-manager-86878c769f-w87nk"] Jul 16 00:58:52.175566 ip-10-0-143-200 kubenswrapper[2572]: I0716 00:58:52.175540 2572 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["kuadrant-system/kuadrant-operator-controller-manager-86878c769f-w87nk"] Jul 16 00:58:53.451817 ip-10-0-143-200 kubenswrapper[2572]: I0716 00:58:53.450115 2572 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="cc2779f5-93cd-4ea3-a618-c8a51ed440b4" path="/var/lib/kubelet/pods/cc2779f5-93cd-4ea3-a618-c8a51ed440b4/volumes" Jul 16 00:59:42.973857 ip-10-0-143-200 kubenswrapper[2572]: I0716 00:59:42.973759 2572 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9gnmq_3a7e030d-fad8-480d-9dc6-fe5057048cf9/konnectivity-agent/0.log" limit=1024 Jul 16 00:59:43.073893 ip-10-0-143-200 kubenswrapper[2572]: I0716 00:59:43.073853 2572 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9gnmq_3a7e030d-fad8-480d-9dc6-fe5057048cf9/konnectivity-agent/0.log" limit=1024 Jul 16 00:59:50.154866 ip-10-0-143-200 kubenswrapper[2572]: I0716 00:59:50.154830 2572 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["kuadrant-system/kuadrant-operator-controller-manager-86878c769f-r4zb2"] Jul 16 00:59:50.155317 ip-10-0-143-200 kubenswrapper[2572]: I0716 00:59:50.155294 2572 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="cc2779f5-93cd-4ea3-a618-c8a51ed440b4" containerName="manager" Jul 16 00:59:50.155317 ip-10-0-143-200 kubenswrapper[2572]: I0716 00:59:50.155307 2572 state_mem.go:107] "Deleted CPUSet assignment" podUID="cc2779f5-93cd-4ea3-a618-c8a51ed440b4" containerName="manager" Jul 16 00:59:50.155404 ip-10-0-143-200 kubenswrapper[2572]: I0716 00:59:50.155393 2572 memory_manager.go:368] "RemoveStaleState removing state" podUID="cc2779f5-93cd-4ea3-a618-c8a51ed440b4" containerName="manager" Jul 16 00:59:50.158531 ip-10-0-143-200 kubenswrapper[2572]: I0716 00:59:50.158512 2572 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/kuadrant-operator-controller-manager-86878c769f-r4zb2" Jul 16 00:59:50.163284 ip-10-0-143-200 kubenswrapper[2572]: I0716 00:59:50.163267 2572 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"kuadrant-system\"/\"kuadrant-operator-controller-manager-dockercfg-mf4mr\"" Jul 16 00:59:50.175461 ip-10-0-143-200 kubenswrapper[2572]: I0716 00:59:50.175422 2572 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/kuadrant-operator-controller-manager-86878c769f-r4zb2"] Jul 16 00:59:50.295778 ip-10-0-143-200 kubenswrapper[2572]: I0716 00:59:50.295731 2572 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"extensions-socket-volume\" (UniqueName: \"kubernetes.io/empty-dir/d605272b-dbc7-41b7-8272-a3f710fd6296-extensions-socket-volume\") pod \"kuadrant-operator-controller-manager-86878c769f-r4zb2\" (UID: \"d605272b-dbc7-41b7-8272-a3f710fd6296\") " pod="kuadrant-system/kuadrant-operator-controller-manager-86878c769f-r4zb2" Jul 16 00:59:50.295778 ip-10-0-143-200 kubenswrapper[2572]: I0716 00:59:50.295776 2572 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-xhmn5\" (UniqueName: \"kubernetes.io/projected/d605272b-dbc7-41b7-8272-a3f710fd6296-kube-api-access-xhmn5\") pod \"kuadrant-operator-controller-manager-86878c769f-r4zb2\" (UID: \"d605272b-dbc7-41b7-8272-a3f710fd6296\") " pod="kuadrant-system/kuadrant-operator-controller-manager-86878c769f-r4zb2" Jul 16 00:59:50.396644 ip-10-0-143-200 kubenswrapper[2572]: I0716 00:59:50.396596 2572 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"extensions-socket-volume\" (UniqueName: \"kubernetes.io/empty-dir/d605272b-dbc7-41b7-8272-a3f710fd6296-extensions-socket-volume\") pod \"kuadrant-operator-controller-manager-86878c769f-r4zb2\" (UID: \"d605272b-dbc7-41b7-8272-a3f710fd6296\") " pod="kuadrant-system/kuadrant-operator-controller-manager-86878c769f-r4zb2" Jul 16 00:59:50.396847 ip-10-0-143-200 kubenswrapper[2572]: I0716 00:59:50.396678 2572 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-xhmn5\" (UniqueName: \"kubernetes.io/projected/d605272b-dbc7-41b7-8272-a3f710fd6296-kube-api-access-xhmn5\") pod \"kuadrant-operator-controller-manager-86878c769f-r4zb2\" (UID: \"d605272b-dbc7-41b7-8272-a3f710fd6296\") " pod="kuadrant-system/kuadrant-operator-controller-manager-86878c769f-r4zb2" Jul 16 00:59:50.397014 ip-10-0-143-200 kubenswrapper[2572]: I0716 00:59:50.396994 2572 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"extensions-socket-volume\" (UniqueName: \"kubernetes.io/empty-dir/d605272b-dbc7-41b7-8272-a3f710fd6296-extensions-socket-volume\") pod \"kuadrant-operator-controller-manager-86878c769f-r4zb2\" (UID: \"d605272b-dbc7-41b7-8272-a3f710fd6296\") " pod="kuadrant-system/kuadrant-operator-controller-manager-86878c769f-r4zb2" Jul 16 00:59:50.407802 ip-10-0-143-200 kubenswrapper[2572]: I0716 00:59:50.407731 2572 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-xhmn5\" (UniqueName: \"kubernetes.io/projected/d605272b-dbc7-41b7-8272-a3f710fd6296-kube-api-access-xhmn5\") pod \"kuadrant-operator-controller-manager-86878c769f-r4zb2\" (UID: \"d605272b-dbc7-41b7-8272-a3f710fd6296\") " pod="kuadrant-system/kuadrant-operator-controller-manager-86878c769f-r4zb2" Jul 16 00:59:50.468964 ip-10-0-143-200 kubenswrapper[2572]: I0716 00:59:50.468924 2572 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/kuadrant-operator-controller-manager-86878c769f-r4zb2" Jul 16 00:59:50.631026 ip-10-0-143-200 kubenswrapper[2572]: I0716 00:59:50.630985 2572 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/kuadrant-operator-controller-manager-86878c769f-r4zb2"] Jul 16 00:59:50.635139 ip-10-0-143-200 kubenswrapper[2572]: W0716 00:59:50.635111 2572 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podd605272b_dbc7_41b7_8272_a3f710fd6296.slice/crio-a10c9b9d0c8af82de559575d7a2629f1fba770a3d198be2e627ee4348470e6b9 WatchSource:0}: Error finding container a10c9b9d0c8af82de559575d7a2629f1fba770a3d198be2e627ee4348470e6b9: Status 404 returned error can't find the container with id a10c9b9d0c8af82de559575d7a2629f1fba770a3d198be2e627ee4348470e6b9 Jul 16 00:59:51.451262 ip-10-0-143-200 kubenswrapper[2572]: I0716 00:59:51.451226 2572 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/kuadrant-operator-controller-manager-86878c769f-r4zb2" event={"ID":"d605272b-dbc7-41b7-8272-a3f710fd6296","Type":"ContainerStarted","Data":"1a81e89be8b33e75d09070c78a036cbf7589d6c44a0ece79b7f10d854866c577"} Jul 16 00:59:51.451711 ip-10-0-143-200 kubenswrapper[2572]: I0716 00:59:51.451286 2572 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/kuadrant-operator-controller-manager-86878c769f-r4zb2" event={"ID":"d605272b-dbc7-41b7-8272-a3f710fd6296","Type":"ContainerStarted","Data":"a10c9b9d0c8af82de559575d7a2629f1fba770a3d198be2e627ee4348470e6b9"} Jul 16 00:59:51.451711 ip-10-0-143-200 kubenswrapper[2572]: I0716 00:59:51.451325 2572 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="kuadrant-system/kuadrant-operator-controller-manager-86878c769f-r4zb2" Jul 16 00:59:51.476896 ip-10-0-143-200 kubenswrapper[2572]: I0716 00:59:51.476833 2572 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="kuadrant-system/kuadrant-operator-controller-manager-86878c769f-r4zb2" podStartSLOduration=1.476810688 podStartE2EDuration="1.476810688s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-16 00:59:50 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-16 00:59:51.475015819 +0000 UTC m=+1196.583770494" watchObservedRunningTime="2026-07-16 00:59:51.476810688 +0000 UTC m=+1196.585565364" Jul 16 00:59:55.434329 ip-10-0-143-200 kubenswrapper[2572]: I0716 00:59:55.434293 2572 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-console-operator_console-operator-7866497494-pjc6j_a64bcd05-8596-4c12-b88e-e974d676e25f/console-operator/2.log" Jul 16 00:59:55.436756 ip-10-0-143-200 kubenswrapper[2572]: I0716 00:59:55.436733 2572 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-console-operator_console-operator-7866497494-pjc6j_a64bcd05-8596-4c12-b88e-e974d676e25f/console-operator/2.log" Jul 16 00:59:55.439287 ip-10-0-143-200 kubenswrapper[2572]: I0716 00:59:55.439265 2572 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ovn-kubernetes_ovnkube-node-fp78g_aa66f2b6-0ad8-45d0-8ff5-40fc21966b5a/ovn-acl-logging/0.log" Jul 16 00:59:55.441976 ip-10-0-143-200 kubenswrapper[2572]: I0716 00:59:55.441954 2572 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ovn-kubernetes_ovnkube-node-fp78g_aa66f2b6-0ad8-45d0-8ff5-40fc21966b5a/ovn-acl-logging/0.log" Jul 16 01:00:02.460883 ip-10-0-143-200 kubenswrapper[2572]: I0716 01:00:02.460849 2572 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="kuadrant-system/kuadrant-operator-controller-manager-86878c769f-r4zb2" Jul 16 01:04:55.466166 ip-10-0-143-200 kubenswrapper[2572]: I0716 01:04:55.466131 2572 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-console-operator_console-operator-7866497494-pjc6j_a64bcd05-8596-4c12-b88e-e974d676e25f/console-operator/2.log" Jul 16 01:04:55.470807 ip-10-0-143-200 kubenswrapper[2572]: I0716 01:04:55.470783 2572 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-console-operator_console-operator-7866497494-pjc6j_a64bcd05-8596-4c12-b88e-e974d676e25f/console-operator/2.log" Jul 16 01:04:55.470940 ip-10-0-143-200 kubenswrapper[2572]: I0716 01:04:55.470909 2572 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ovn-kubernetes_ovnkube-node-fp78g_aa66f2b6-0ad8-45d0-8ff5-40fc21966b5a/ovn-acl-logging/0.log" Jul 16 01:04:55.475485 ip-10-0-143-200 kubenswrapper[2572]: I0716 01:04:55.475466 2572 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ovn-kubernetes_ovnkube-node-fp78g_aa66f2b6-0ad8-45d0-8ff5-40fc21966b5a/ovn-acl-logging/0.log" Jul 16 01:05:20.601075 ip-10-0-143-200 kubenswrapper[2572]: I0716 01:05:20.600986 2572 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9gnmq_3a7e030d-fad8-480d-9dc6-fe5057048cf9/konnectivity-agent/0.log" limit=1024 Jul 16 01:05:25.993952 ip-10-0-143-200 kubenswrapper[2572]: I0716 01:05:25.993913 2572 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9gnmq_3a7e030d-fad8-480d-9dc6-fe5057048cf9/konnectivity-agent/0.log" limit=1024 Jul 16 01:05:26.376719 ip-10-0-143-200 kubenswrapper[2572]: I0716 01:05:26.376610 2572 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9gnmq_3a7e030d-fad8-480d-9dc6-fe5057048cf9/konnectivity-agent/0.log" limit=1024 Jul 16 01:05:26.471935 ip-10-0-143-200 kubenswrapper[2572]: I0716 01:05:26.471900 2572 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9gnmq_3a7e030d-fad8-480d-9dc6-fe5057048cf9/konnectivity-agent/0.log" limit=1024 Jul 16 01:05:39.703502 ip-10-0-143-200 kubenswrapper[2572]: I0716 01:05:39.703458 2572 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9gnmq_3a7e030d-fad8-480d-9dc6-fe5057048cf9/konnectivity-agent/0.log" limit=1024 Jul 16 01:05:47.176251 ip-10-0-143-200 kubenswrapper[2572]: I0716 01:05:47.176216 2572 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9gnmq_3a7e030d-fad8-480d-9dc6-fe5057048cf9/konnectivity-agent/0.log" limit=1024 Jul 16 01:05:55.460376 ip-10-0-143-200 kubenswrapper[2572]: I0716 01:05:55.460342 2572 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9gnmq_3a7e030d-fad8-480d-9dc6-fe5057048cf9/konnectivity-agent/0.log" limit=1024 Jul 16 01:06:00.528173 ip-10-0-143-200 kubenswrapper[2572]: I0716 01:06:00.528132 2572 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9gnmq_3a7e030d-fad8-480d-9dc6-fe5057048cf9/konnectivity-agent/0.log" limit=1024 Jul 16 01:06:03.599472 ip-10-0-143-200 kubenswrapper[2572]: I0716 01:06:03.599436 2572 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9gnmq_3a7e030d-fad8-480d-9dc6-fe5057048cf9/konnectivity-agent/0.log" limit=1024 Jul 16 01:06:03.698824 ip-10-0-143-200 kubenswrapper[2572]: I0716 01:06:03.698788 2572 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9gnmq_3a7e030d-fad8-480d-9dc6-fe5057048cf9/konnectivity-agent/0.log" limit=1024 Jul 16 01:06:04.058511 ip-10-0-143-200 kubenswrapper[2572]: I0716 01:06:04.058481 2572 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9gnmq_3a7e030d-fad8-480d-9dc6-fe5057048cf9/konnectivity-agent/0.log" limit=1024 Jul 16 01:06:10.374739 ip-10-0-143-200 kubenswrapper[2572]: I0716 01:06:10.374703 2572 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9gnmq_3a7e030d-fad8-480d-9dc6-fe5057048cf9/konnectivity-agent/0.log" limit=1024 Jul 16 01:06:10.460756 ip-10-0-143-200 kubenswrapper[2572]: I0716 01:06:10.460717 2572 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9gnmq_3a7e030d-fad8-480d-9dc6-fe5057048cf9/konnectivity-agent/0.log" limit=1024 Jul 16 01:06:10.637703 ip-10-0-143-200 kubenswrapper[2572]: I0716 01:06:10.637664 2572 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-ingress/e2e-ait-241c63c6-openshift-default-85dccf9599-4pvbp"] Jul 16 01:06:10.640905 ip-10-0-143-200 kubenswrapper[2572]: I0716 01:06:10.640886 2572 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-ait-241c63c6-openshift-default-85dccf9599-4pvbp" Jul 16 01:06:10.643760 ip-10-0-143-200 kubenswrapper[2572]: I0716 01:06:10.643744 2572 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress\"/\"e2e-ait-241c63c6-openshift-default-dockercfg-r5z5x\"" Jul 16 01:06:10.674108 ip-10-0-143-200 kubenswrapper[2572]: I0716 01:06:10.674067 2572 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/e2e-ait-241c63c6-openshift-default-85dccf9599-4pvbp"] Jul 16 01:06:10.676752 ip-10-0-143-200 kubenswrapper[2572]: I0716 01:06:10.676720 2572 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/6bd32f4f-ed9b-412a-8431-2f42130c4722-istio-data\") pod \"e2e-ait-241c63c6-openshift-default-85dccf9599-4pvbp\" (UID: \"6bd32f4f-ed9b-412a-8431-2f42130c4722\") " pod="openshift-ingress/e2e-ait-241c63c6-openshift-default-85dccf9599-4pvbp" Jul 16 01:06:10.676909 ip-10-0-143-200 kubenswrapper[2572]: I0716 01:06:10.676777 2572 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-sdzhl\" (UniqueName: \"kubernetes.io/projected/6bd32f4f-ed9b-412a-8431-2f42130c4722-kube-api-access-sdzhl\") pod \"e2e-ait-241c63c6-openshift-default-85dccf9599-4pvbp\" (UID: \"6bd32f4f-ed9b-412a-8431-2f42130c4722\") " pod="openshift-ingress/e2e-ait-241c63c6-openshift-default-85dccf9599-4pvbp" Jul 16 01:06:10.676972 ip-10-0-143-200 kubenswrapper[2572]: I0716 01:06:10.676854 2572 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/6bd32f4f-ed9b-412a-8431-2f42130c4722-workload-certs\") pod \"e2e-ait-241c63c6-openshift-default-85dccf9599-4pvbp\" (UID: \"6bd32f4f-ed9b-412a-8431-2f42130c4722\") " pod="openshift-ingress/e2e-ait-241c63c6-openshift-default-85dccf9599-4pvbp" Jul 16 01:06:10.677021 ip-10-0-143-200 kubenswrapper[2572]: I0716 01:06:10.676973 2572 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/6bd32f4f-ed9b-412a-8431-2f42130c4722-istio-envoy\") pod \"e2e-ait-241c63c6-openshift-default-85dccf9599-4pvbp\" (UID: \"6bd32f4f-ed9b-412a-8431-2f42130c4722\") " pod="openshift-ingress/e2e-ait-241c63c6-openshift-default-85dccf9599-4pvbp" Jul 16 01:06:10.677076 ip-10-0-143-200 kubenswrapper[2572]: I0716 01:06:10.677018 2572 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/6bd32f4f-ed9b-412a-8431-2f42130c4722-istio-token\") pod \"e2e-ait-241c63c6-openshift-default-85dccf9599-4pvbp\" (UID: \"6bd32f4f-ed9b-412a-8431-2f42130c4722\") " pod="openshift-ingress/e2e-ait-241c63c6-openshift-default-85dccf9599-4pvbp" Jul 16 01:06:10.677126 ip-10-0-143-200 kubenswrapper[2572]: I0716 01:06:10.677077 2572 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/6bd32f4f-ed9b-412a-8431-2f42130c4722-credential-socket\") pod \"e2e-ait-241c63c6-openshift-default-85dccf9599-4pvbp\" (UID: \"6bd32f4f-ed9b-412a-8431-2f42130c4722\") " pod="openshift-ingress/e2e-ait-241c63c6-openshift-default-85dccf9599-4pvbp" Jul 16 01:06:10.677126 ip-10-0-143-200 kubenswrapper[2572]: I0716 01:06:10.677114 2572 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/6bd32f4f-ed9b-412a-8431-2f42130c4722-workload-socket\") pod \"e2e-ait-241c63c6-openshift-default-85dccf9599-4pvbp\" (UID: \"6bd32f4f-ed9b-412a-8431-2f42130c4722\") " pod="openshift-ingress/e2e-ait-241c63c6-openshift-default-85dccf9599-4pvbp" Jul 16 01:06:10.677227 ip-10-0-143-200 kubenswrapper[2572]: I0716 01:06:10.677139 2572 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/6bd32f4f-ed9b-412a-8431-2f42130c4722-istio-podinfo\") pod \"e2e-ait-241c63c6-openshift-default-85dccf9599-4pvbp\" (UID: \"6bd32f4f-ed9b-412a-8431-2f42130c4722\") " pod="openshift-ingress/e2e-ait-241c63c6-openshift-default-85dccf9599-4pvbp" Jul 16 01:06:10.677227 ip-10-0-143-200 kubenswrapper[2572]: I0716 01:06:10.677174 2572 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/6bd32f4f-ed9b-412a-8431-2f42130c4722-istiod-ca-cert\") pod \"e2e-ait-241c63c6-openshift-default-85dccf9599-4pvbp\" (UID: \"6bd32f4f-ed9b-412a-8431-2f42130c4722\") " pod="openshift-ingress/e2e-ait-241c63c6-openshift-default-85dccf9599-4pvbp" Jul 16 01:06:10.679123 ip-10-0-143-200 kubenswrapper[2572]: I0716 01:06:10.679096 2572 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9gnmq_3a7e030d-fad8-480d-9dc6-fe5057048cf9/konnectivity-agent/0.log" limit=1024 Jul 16 01:06:10.778027 ip-10-0-143-200 kubenswrapper[2572]: I0716 01:06:10.777987 2572 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/6bd32f4f-ed9b-412a-8431-2f42130c4722-credential-socket\") pod \"e2e-ait-241c63c6-openshift-default-85dccf9599-4pvbp\" (UID: \"6bd32f4f-ed9b-412a-8431-2f42130c4722\") " pod="openshift-ingress/e2e-ait-241c63c6-openshift-default-85dccf9599-4pvbp" Jul 16 01:06:10.778227 ip-10-0-143-200 kubenswrapper[2572]: I0716 01:06:10.778040 2572 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/6bd32f4f-ed9b-412a-8431-2f42130c4722-workload-socket\") pod \"e2e-ait-241c63c6-openshift-default-85dccf9599-4pvbp\" (UID: \"6bd32f4f-ed9b-412a-8431-2f42130c4722\") " pod="openshift-ingress/e2e-ait-241c63c6-openshift-default-85dccf9599-4pvbp" Jul 16 01:06:10.778306 ip-10-0-143-200 kubenswrapper[2572]: I0716 01:06:10.778252 2572 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/6bd32f4f-ed9b-412a-8431-2f42130c4722-istio-podinfo\") pod \"e2e-ait-241c63c6-openshift-default-85dccf9599-4pvbp\" (UID: \"6bd32f4f-ed9b-412a-8431-2f42130c4722\") " pod="openshift-ingress/e2e-ait-241c63c6-openshift-default-85dccf9599-4pvbp" Jul 16 01:06:10.778363 ip-10-0-143-200 kubenswrapper[2572]: I0716 01:06:10.778317 2572 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/6bd32f4f-ed9b-412a-8431-2f42130c4722-istiod-ca-cert\") pod \"e2e-ait-241c63c6-openshift-default-85dccf9599-4pvbp\" (UID: \"6bd32f4f-ed9b-412a-8431-2f42130c4722\") " pod="openshift-ingress/e2e-ait-241c63c6-openshift-default-85dccf9599-4pvbp" Jul 16 01:06:10.778416 ip-10-0-143-200 kubenswrapper[2572]: I0716 01:06:10.778387 2572 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/6bd32f4f-ed9b-412a-8431-2f42130c4722-istio-data\") pod \"e2e-ait-241c63c6-openshift-default-85dccf9599-4pvbp\" (UID: \"6bd32f4f-ed9b-412a-8431-2f42130c4722\") " pod="openshift-ingress/e2e-ait-241c63c6-openshift-default-85dccf9599-4pvbp" Jul 16 01:06:10.778469 ip-10-0-143-200 kubenswrapper[2572]: I0716 01:06:10.778433 2572 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-sdzhl\" (UniqueName: \"kubernetes.io/projected/6bd32f4f-ed9b-412a-8431-2f42130c4722-kube-api-access-sdzhl\") pod \"e2e-ait-241c63c6-openshift-default-85dccf9599-4pvbp\" (UID: \"6bd32f4f-ed9b-412a-8431-2f42130c4722\") " pod="openshift-ingress/e2e-ait-241c63c6-openshift-default-85dccf9599-4pvbp" Jul 16 01:06:10.778469 ip-10-0-143-200 kubenswrapper[2572]: I0716 01:06:10.778441 2572 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/6bd32f4f-ed9b-412a-8431-2f42130c4722-credential-socket\") pod \"e2e-ait-241c63c6-openshift-default-85dccf9599-4pvbp\" (UID: \"6bd32f4f-ed9b-412a-8431-2f42130c4722\") " pod="openshift-ingress/e2e-ait-241c63c6-openshift-default-85dccf9599-4pvbp" Jul 16 01:06:10.778469 ip-10-0-143-200 kubenswrapper[2572]: I0716 01:06:10.778451 2572 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/6bd32f4f-ed9b-412a-8431-2f42130c4722-workload-socket\") pod \"e2e-ait-241c63c6-openshift-default-85dccf9599-4pvbp\" (UID: \"6bd32f4f-ed9b-412a-8431-2f42130c4722\") " pod="openshift-ingress/e2e-ait-241c63c6-openshift-default-85dccf9599-4pvbp" Jul 16 01:06:10.778622 ip-10-0-143-200 kubenswrapper[2572]: I0716 01:06:10.778518 2572 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/6bd32f4f-ed9b-412a-8431-2f42130c4722-workload-certs\") pod \"e2e-ait-241c63c6-openshift-default-85dccf9599-4pvbp\" (UID: \"6bd32f4f-ed9b-412a-8431-2f42130c4722\") " pod="openshift-ingress/e2e-ait-241c63c6-openshift-default-85dccf9599-4pvbp" Jul 16 01:06:10.778622 ip-10-0-143-200 kubenswrapper[2572]: I0716 01:06:10.778543 2572 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/6bd32f4f-ed9b-412a-8431-2f42130c4722-istio-envoy\") pod \"e2e-ait-241c63c6-openshift-default-85dccf9599-4pvbp\" (UID: \"6bd32f4f-ed9b-412a-8431-2f42130c4722\") " pod="openshift-ingress/e2e-ait-241c63c6-openshift-default-85dccf9599-4pvbp" Jul 16 01:06:10.778622 ip-10-0-143-200 kubenswrapper[2572]: I0716 01:06:10.778583 2572 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/6bd32f4f-ed9b-412a-8431-2f42130c4722-istio-token\") pod \"e2e-ait-241c63c6-openshift-default-85dccf9599-4pvbp\" (UID: \"6bd32f4f-ed9b-412a-8431-2f42130c4722\") " pod="openshift-ingress/e2e-ait-241c63c6-openshift-default-85dccf9599-4pvbp" Jul 16 01:06:10.778816 ip-10-0-143-200 kubenswrapper[2572]: I0716 01:06:10.778777 2572 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/6bd32f4f-ed9b-412a-8431-2f42130c4722-istio-data\") pod \"e2e-ait-241c63c6-openshift-default-85dccf9599-4pvbp\" (UID: \"6bd32f4f-ed9b-412a-8431-2f42130c4722\") " pod="openshift-ingress/e2e-ait-241c63c6-openshift-default-85dccf9599-4pvbp" Jul 16 01:06:10.778872 ip-10-0-143-200 kubenswrapper[2572]: I0716 01:06:10.778853 2572 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/6bd32f4f-ed9b-412a-8431-2f42130c4722-workload-certs\") pod \"e2e-ait-241c63c6-openshift-default-85dccf9599-4pvbp\" (UID: \"6bd32f4f-ed9b-412a-8431-2f42130c4722\") " pod="openshift-ingress/e2e-ait-241c63c6-openshift-default-85dccf9599-4pvbp" Jul 16 01:06:10.779169 ip-10-0-143-200 kubenswrapper[2572]: I0716 01:06:10.779143 2572 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/6bd32f4f-ed9b-412a-8431-2f42130c4722-istiod-ca-cert\") pod \"e2e-ait-241c63c6-openshift-default-85dccf9599-4pvbp\" (UID: \"6bd32f4f-ed9b-412a-8431-2f42130c4722\") " pod="openshift-ingress/e2e-ait-241c63c6-openshift-default-85dccf9599-4pvbp" Jul 16 01:06:10.780713 ip-10-0-143-200 kubenswrapper[2572]: I0716 01:06:10.780692 2572 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/6bd32f4f-ed9b-412a-8431-2f42130c4722-istio-envoy\") pod \"e2e-ait-241c63c6-openshift-default-85dccf9599-4pvbp\" (UID: \"6bd32f4f-ed9b-412a-8431-2f42130c4722\") " pod="openshift-ingress/e2e-ait-241c63c6-openshift-default-85dccf9599-4pvbp" Jul 16 01:06:10.780814 ip-10-0-143-200 kubenswrapper[2572]: I0716 01:06:10.780799 2572 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/6bd32f4f-ed9b-412a-8431-2f42130c4722-istio-podinfo\") pod \"e2e-ait-241c63c6-openshift-default-85dccf9599-4pvbp\" (UID: \"6bd32f4f-ed9b-412a-8431-2f42130c4722\") " pod="openshift-ingress/e2e-ait-241c63c6-openshift-default-85dccf9599-4pvbp" Jul 16 01:06:10.794171 ip-10-0-143-200 kubenswrapper[2572]: I0716 01:06:10.794139 2572 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/6bd32f4f-ed9b-412a-8431-2f42130c4722-istio-token\") pod \"e2e-ait-241c63c6-openshift-default-85dccf9599-4pvbp\" (UID: \"6bd32f4f-ed9b-412a-8431-2f42130c4722\") " pod="openshift-ingress/e2e-ait-241c63c6-openshift-default-85dccf9599-4pvbp" Jul 16 01:06:10.794320 ip-10-0-143-200 kubenswrapper[2572]: I0716 01:06:10.794193 2572 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-sdzhl\" (UniqueName: \"kubernetes.io/projected/6bd32f4f-ed9b-412a-8431-2f42130c4722-kube-api-access-sdzhl\") pod \"e2e-ait-241c63c6-openshift-default-85dccf9599-4pvbp\" (UID: \"6bd32f4f-ed9b-412a-8431-2f42130c4722\") " pod="openshift-ingress/e2e-ait-241c63c6-openshift-default-85dccf9599-4pvbp" Jul 16 01:06:10.844467 ip-10-0-143-200 kubenswrapper[2572]: I0716 01:06:10.844434 2572 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-ait-241c63c6-559cdc5ccc-wmpjg"] Jul 16 01:06:10.849463 ip-10-0-143-200 kubenswrapper[2572]: I0716 01:06:10.849438 2572 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-e2e-ait-241c63c6-559cdc5ccc-wmpjg" Jul 16 01:06:10.853930 ip-10-0-143-200 kubenswrapper[2572]: I0716 01:06:10.853908 2572 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"odh-ai-gateway-infra\"/\"maas-api-e2e-ait-241c63c6-serving-cert\"" Jul 16 01:06:10.854252 ip-10-0-143-200 kubenswrapper[2572]: I0716 01:06:10.854232 2572 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"odh-ai-gateway-infra\"/\"maas-api-dockercfg-ms2nn\"" Jul 16 01:06:10.861655 ip-10-0-143-200 kubenswrapper[2572]: I0716 01:06:10.861610 2572 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-ait-241c63c6-559cdc5ccc-wmpjg"] Jul 16 01:06:10.952322 ip-10-0-143-200 kubenswrapper[2572]: I0716 01:06:10.952293 2572 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-ait-241c63c6-openshift-default-85dccf9599-4pvbp" Jul 16 01:06:10.980422 ip-10-0-143-200 kubenswrapper[2572]: I0716 01:06:10.980376 2572 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-4xrvr\" (UniqueName: \"kubernetes.io/projected/5173bdc1-9d45-4ee4-8436-08828ef8a0a1-kube-api-access-4xrvr\") pod \"maas-api-e2e-ait-241c63c6-559cdc5ccc-wmpjg\" (UID: \"5173bdc1-9d45-4ee4-8436-08828ef8a0a1\") " pod="odh-ai-gateway-infra/maas-api-e2e-ait-241c63c6-559cdc5ccc-wmpjg" Jul 16 01:06:10.980590 ip-10-0-143-200 kubenswrapper[2572]: I0716 01:06:10.980424 2572 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/5173bdc1-9d45-4ee4-8436-08828ef8a0a1-maas-api-tls\") pod \"maas-api-e2e-ait-241c63c6-559cdc5ccc-wmpjg\" (UID: \"5173bdc1-9d45-4ee4-8436-08828ef8a0a1\") " pod="odh-ai-gateway-infra/maas-api-e2e-ait-241c63c6-559cdc5ccc-wmpjg" Jul 16 01:06:10.980590 ip-10-0-143-200 kubenswrapper[2572]: I0716 01:06:10.980454 2572 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/5173bdc1-9d45-4ee4-8436-08828ef8a0a1-openshift-service-ca\") pod \"maas-api-e2e-ait-241c63c6-559cdc5ccc-wmpjg\" (UID: \"5173bdc1-9d45-4ee4-8436-08828ef8a0a1\") " pod="odh-ai-gateway-infra/maas-api-e2e-ait-241c63c6-559cdc5ccc-wmpjg" Jul 16 01:06:11.081689 ip-10-0-143-200 kubenswrapper[2572]: I0716 01:06:11.081604 2572 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-4xrvr\" (UniqueName: \"kubernetes.io/projected/5173bdc1-9d45-4ee4-8436-08828ef8a0a1-kube-api-access-4xrvr\") pod \"maas-api-e2e-ait-241c63c6-559cdc5ccc-wmpjg\" (UID: \"5173bdc1-9d45-4ee4-8436-08828ef8a0a1\") " pod="odh-ai-gateway-infra/maas-api-e2e-ait-241c63c6-559cdc5ccc-wmpjg" Jul 16 01:06:11.081689 ip-10-0-143-200 kubenswrapper[2572]: I0716 01:06:11.081668 2572 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/5173bdc1-9d45-4ee4-8436-08828ef8a0a1-maas-api-tls\") pod \"maas-api-e2e-ait-241c63c6-559cdc5ccc-wmpjg\" (UID: \"5173bdc1-9d45-4ee4-8436-08828ef8a0a1\") " pod="odh-ai-gateway-infra/maas-api-e2e-ait-241c63c6-559cdc5ccc-wmpjg" Jul 16 01:06:11.081689 ip-10-0-143-200 kubenswrapper[2572]: I0716 01:06:11.081694 2572 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/5173bdc1-9d45-4ee4-8436-08828ef8a0a1-openshift-service-ca\") pod \"maas-api-e2e-ait-241c63c6-559cdc5ccc-wmpjg\" (UID: \"5173bdc1-9d45-4ee4-8436-08828ef8a0a1\") " pod="odh-ai-gateway-infra/maas-api-e2e-ait-241c63c6-559cdc5ccc-wmpjg" Jul 16 01:06:11.082735 ip-10-0-143-200 kubenswrapper[2572]: I0716 01:06:11.082559 2572 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/5173bdc1-9d45-4ee4-8436-08828ef8a0a1-openshift-service-ca\") pod \"maas-api-e2e-ait-241c63c6-559cdc5ccc-wmpjg\" (UID: \"5173bdc1-9d45-4ee4-8436-08828ef8a0a1\") " pod="odh-ai-gateway-infra/maas-api-e2e-ait-241c63c6-559cdc5ccc-wmpjg" Jul 16 01:06:11.085126 ip-10-0-143-200 kubenswrapper[2572]: I0716 01:06:11.085101 2572 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/5173bdc1-9d45-4ee4-8436-08828ef8a0a1-maas-api-tls\") pod \"maas-api-e2e-ait-241c63c6-559cdc5ccc-wmpjg\" (UID: \"5173bdc1-9d45-4ee4-8436-08828ef8a0a1\") " pod="odh-ai-gateway-infra/maas-api-e2e-ait-241c63c6-559cdc5ccc-wmpjg" Jul 16 01:06:11.109268 ip-10-0-143-200 kubenswrapper[2572]: I0716 01:06:11.109239 2572 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-4xrvr\" (UniqueName: \"kubernetes.io/projected/5173bdc1-9d45-4ee4-8436-08828ef8a0a1-kube-api-access-4xrvr\") pod \"maas-api-e2e-ait-241c63c6-559cdc5ccc-wmpjg\" (UID: \"5173bdc1-9d45-4ee4-8436-08828ef8a0a1\") " pod="odh-ai-gateway-infra/maas-api-e2e-ait-241c63c6-559cdc5ccc-wmpjg" Jul 16 01:06:11.128275 ip-10-0-143-200 kubenswrapper[2572]: I0716 01:06:11.128249 2572 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/e2e-ait-241c63c6-openshift-default-85dccf9599-4pvbp"] Jul 16 01:06:11.130583 ip-10-0-143-200 kubenswrapper[2572]: W0716 01:06:11.130552 2572 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod6bd32f4f_ed9b_412a_8431_2f42130c4722.slice/crio-51152b35c81bb876d1d759c68549bda1579f2ffde8cc2e1d8c9d6f23667920f7 WatchSource:0}: Error finding container 51152b35c81bb876d1d759c68549bda1579f2ffde8cc2e1d8c9d6f23667920f7: Status 404 returned error can't find the container with id 51152b35c81bb876d1d759c68549bda1579f2ffde8cc2e1d8c9d6f23667920f7 Jul 16 01:06:11.132738 ip-10-0-143-200 kubenswrapper[2572]: I0716 01:06:11.132707 2572 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236228Ki","pods":"250"} Jul 16 01:06:11.132831 ip-10-0-143-200 kubenswrapper[2572]: I0716 01:06:11.132771 2572 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236228Ki","pods":"250"} Jul 16 01:06:11.132831 ip-10-0-143-200 kubenswrapper[2572]: I0716 01:06:11.132800 2572 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236228Ki","pods":"250"} Jul 16 01:06:11.150546 ip-10-0-143-200 kubenswrapper[2572]: I0716 01:06:11.150519 2572 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9gnmq_3a7e030d-fad8-480d-9dc6-fe5057048cf9/konnectivity-agent/0.log" limit=1024 Jul 16 01:06:11.164454 ip-10-0-143-200 kubenswrapper[2572]: I0716 01:06:11.164434 2572 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-e2e-ait-241c63c6-559cdc5ccc-wmpjg" Jul 16 01:06:11.242424 ip-10-0-143-200 kubenswrapper[2572]: I0716 01:06:11.242355 2572 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9gnmq_3a7e030d-fad8-480d-9dc6-fe5057048cf9/konnectivity-agent/0.log" limit=1024 Jul 16 01:06:11.332238 ip-10-0-143-200 kubenswrapper[2572]: I0716 01:06:11.332215 2572 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-ait-241c63c6-559cdc5ccc-wmpjg"] Jul 16 01:06:11.334861 ip-10-0-143-200 kubenswrapper[2572]: W0716 01:06:11.334835 2572 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod5173bdc1_9d45_4ee4_8436_08828ef8a0a1.slice/crio-f0a58ed1c009c4d8dc1543a45ef45d56e12169bf1c7bb79e3660d32e765d182a WatchSource:0}: Error finding container f0a58ed1c009c4d8dc1543a45ef45d56e12169bf1c7bb79e3660d32e765d182a: Status 404 returned error can't find the container with id f0a58ed1c009c4d8dc1543a45ef45d56e12169bf1c7bb79e3660d32e765d182a Jul 16 01:06:11.336460 ip-10-0-143-200 kubenswrapper[2572]: I0716 01:06:11.336444 2572 provider.go:93] Refreshing cache for provider: *credentialprovider.defaultDockerConfigProvider Jul 16 01:06:11.418702 ip-10-0-143-200 kubenswrapper[2572]: I0716 01:06:11.418670 2572 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-e2e-ait-241c63c6-559cdc5ccc-wmpjg" event={"ID":"5173bdc1-9d45-4ee4-8436-08828ef8a0a1","Type":"ContainerStarted","Data":"f0a58ed1c009c4d8dc1543a45ef45d56e12169bf1c7bb79e3660d32e765d182a"} Jul 16 01:06:11.420466 ip-10-0-143-200 kubenswrapper[2572]: I0716 01:06:11.420434 2572 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-ait-241c63c6-openshift-default-85dccf9599-4pvbp" event={"ID":"6bd32f4f-ed9b-412a-8431-2f42130c4722","Type":"ContainerStarted","Data":"c2befe6f026de5a2c7b3922db4cd2bece34317ac042b978c804ec5e570d9701b"} Jul 16 01:06:11.420601 ip-10-0-143-200 kubenswrapper[2572]: I0716 01:06:11.420490 2572 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-ait-241c63c6-openshift-default-85dccf9599-4pvbp" event={"ID":"6bd32f4f-ed9b-412a-8431-2f42130c4722","Type":"ContainerStarted","Data":"51152b35c81bb876d1d759c68549bda1579f2ffde8cc2e1d8c9d6f23667920f7"} Jul 16 01:06:11.455184 ip-10-0-143-200 kubenswrapper[2572]: I0716 01:06:11.455131 2572 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-ingress/e2e-ait-241c63c6-openshift-default-85dccf9599-4pvbp" podStartSLOduration=1.455115797 podStartE2EDuration="1.455115797s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-16 01:06:10 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-16 01:06:11.452087089 +0000 UTC m=+1576.560841764" watchObservedRunningTime="2026-07-16 01:06:11.455115797 +0000 UTC m=+1576.563870471" Jul 16 01:06:11.591874 ip-10-0-143-200 kubenswrapper[2572]: I0716 01:06:11.591792 2572 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9gnmq_3a7e030d-fad8-480d-9dc6-fe5057048cf9/konnectivity-agent/0.log" limit=1024 Jul 16 01:06:11.692940 ip-10-0-143-200 kubenswrapper[2572]: I0716 01:06:11.692898 2572 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9gnmq_3a7e030d-fad8-480d-9dc6-fe5057048cf9/konnectivity-agent/0.log" limit=1024 Jul 16 01:06:11.953399 ip-10-0-143-200 kubenswrapper[2572]: I0716 01:06:11.953366 2572 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="openshift-ingress/e2e-ait-241c63c6-openshift-default-85dccf9599-4pvbp" Jul 16 01:06:12.427175 ip-10-0-143-200 kubenswrapper[2572]: I0716 01:06:12.427141 2572 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-e2e-ait-241c63c6-559cdc5ccc-wmpjg" event={"ID":"5173bdc1-9d45-4ee4-8436-08828ef8a0a1","Type":"ContainerStarted","Data":"60478fcdedf04cb746df1bd3dc68c7b5095d162e186b1aa95ec0f74e3dfb5b1e"} Jul 16 01:06:12.427594 ip-10-0-143-200 kubenswrapper[2572]: I0716 01:06:12.427220 2572 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="odh-ai-gateway-infra/maas-api-e2e-ait-241c63c6-559cdc5ccc-wmpjg" Jul 16 01:06:12.452516 ip-10-0-143-200 kubenswrapper[2572]: I0716 01:06:12.452454 2572 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="odh-ai-gateway-infra/maas-api-e2e-ait-241c63c6-559cdc5ccc-wmpjg" podStartSLOduration=1.9466469339999999 podStartE2EDuration="2.452435092s" totalImagesPullingTime="505.788158ms" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-16 01:06:10 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-16 01:06:12.451184472 +0000 UTC m=+1577.559939158" watchObservedRunningTime="2026-07-16 01:06:12.452435092 +0000 UTC m=+1577.561189768" Jul 16 01:06:12.957707 ip-10-0-143-200 kubenswrapper[2572]: I0716 01:06:12.957672 2572 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="openshift-ingress/e2e-ait-241c63c6-openshift-default-85dccf9599-4pvbp" Jul 16 01:06:13.431813 ip-10-0-143-200 kubenswrapper[2572]: I0716 01:06:13.431725 2572 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/e2e-ait-241c63c6-openshift-default-85dccf9599-4pvbp" Jul 16 01:06:13.432867 ip-10-0-143-200 kubenswrapper[2572]: I0716 01:06:13.432846 2572 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/e2e-ait-241c63c6-openshift-default-85dccf9599-4pvbp" Jul 16 01:06:14.646234 ip-10-0-143-200 kubenswrapper[2572]: I0716 01:06:14.646203 2572 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9gnmq_3a7e030d-fad8-480d-9dc6-fe5057048cf9/konnectivity-agent/0.log" limit=1024 Jul 16 01:06:16.932433 ip-10-0-143-200 kubenswrapper[2572]: I0716 01:06:16.932399 2572 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-ingress/e2e-ait-241c63c6-openshift-default-85dccf9599-4pvbp"] Jul 16 01:06:16.932887 ip-10-0-143-200 kubenswrapper[2572]: I0716 01:06:16.932585 2572 kuberuntime_container.go:873] "Killing container with a grace period" pod="openshift-ingress/e2e-ait-241c63c6-openshift-default-85dccf9599-4pvbp" podUID="6bd32f4f-ed9b-412a-8431-2f42130c4722" containerName="istio-proxy" containerID="cri-o://c2befe6f026de5a2c7b3922db4cd2bece34317ac042b978c804ec5e570d9701b" gracePeriod=30 Jul 16 01:06:16.958563 ip-10-0-143-200 kubenswrapper[2572]: I0716 01:06:16.958524 2572 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9gnmq_3a7e030d-fad8-480d-9dc6-fe5057048cf9/konnectivity-agent/0.log" limit=1024 Jul 16 01:06:17.102301 ip-10-0-143-200 kubenswrapper[2572]: I0716 01:06:17.102238 2572 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9gnmq_3a7e030d-fad8-480d-9dc6-fe5057048cf9/konnectivity-agent/0.log" limit=1024 Jul 16 01:06:18.438099 ip-10-0-143-200 kubenswrapper[2572]: I0716 01:06:18.438068 2572 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="odh-ai-gateway-infra/maas-api-e2e-ait-241c63c6-559cdc5ccc-wmpjg" Jul 16 01:06:18.715885 ip-10-0-143-200 kubenswrapper[2572]: I0716 01:06:18.715829 2572 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9gnmq_3a7e030d-fad8-480d-9dc6-fe5057048cf9/konnectivity-agent/0.log" limit=1024 Jul 16 01:06:23.693666 ip-10-0-143-200 kubenswrapper[2572]: I0716 01:06:23.693640 2572 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-ait-241c63c6-openshift-default-85dccf9599-4pvbp" Jul 16 01:06:23.797690 ip-10-0-143-200 kubenswrapper[2572]: I0716 01:06:23.797582 2572 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9gnmq_3a7e030d-fad8-480d-9dc6-fe5057048cf9/konnectivity-agent/0.log" limit=1024 Jul 16 01:06:23.811453 ip-10-0-143-200 kubenswrapper[2572]: I0716 01:06:23.811422 2572 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/6bd32f4f-ed9b-412a-8431-2f42130c4722-istio-token\") pod \"6bd32f4f-ed9b-412a-8431-2f42130c4722\" (UID: \"6bd32f4f-ed9b-412a-8431-2f42130c4722\") " Jul 16 01:06:23.811595 ip-10-0-143-200 kubenswrapper[2572]: I0716 01:06:23.811463 2572 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/6bd32f4f-ed9b-412a-8431-2f42130c4722-credential-socket\") pod \"6bd32f4f-ed9b-412a-8431-2f42130c4722\" (UID: \"6bd32f4f-ed9b-412a-8431-2f42130c4722\") " Jul 16 01:06:23.811595 ip-10-0-143-200 kubenswrapper[2572]: I0716 01:06:23.811504 2572 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/6bd32f4f-ed9b-412a-8431-2f42130c4722-istiod-ca-cert\") pod \"6bd32f4f-ed9b-412a-8431-2f42130c4722\" (UID: \"6bd32f4f-ed9b-412a-8431-2f42130c4722\") " Jul 16 01:06:23.811595 ip-10-0-143-200 kubenswrapper[2572]: I0716 01:06:23.811524 2572 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/6bd32f4f-ed9b-412a-8431-2f42130c4722-workload-socket\") pod \"6bd32f4f-ed9b-412a-8431-2f42130c4722\" (UID: \"6bd32f4f-ed9b-412a-8431-2f42130c4722\") " Jul 16 01:06:23.811595 ip-10-0-143-200 kubenswrapper[2572]: I0716 01:06:23.811549 2572 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/6bd32f4f-ed9b-412a-8431-2f42130c4722-istio-envoy\") pod \"6bd32f4f-ed9b-412a-8431-2f42130c4722\" (UID: \"6bd32f4f-ed9b-412a-8431-2f42130c4722\") " Jul 16 01:06:23.811842 ip-10-0-143-200 kubenswrapper[2572]: I0716 01:06:23.811623 2572 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/6bd32f4f-ed9b-412a-8431-2f42130c4722-istio-data\") pod \"6bd32f4f-ed9b-412a-8431-2f42130c4722\" (UID: \"6bd32f4f-ed9b-412a-8431-2f42130c4722\") " Jul 16 01:06:23.811842 ip-10-0-143-200 kubenswrapper[2572]: I0716 01:06:23.811682 2572 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/6bd32f4f-ed9b-412a-8431-2f42130c4722-workload-certs\") pod \"6bd32f4f-ed9b-412a-8431-2f42130c4722\" (UID: \"6bd32f4f-ed9b-412a-8431-2f42130c4722\") " Jul 16 01:06:23.811842 ip-10-0-143-200 kubenswrapper[2572]: I0716 01:06:23.811711 2572 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/6bd32f4f-ed9b-412a-8431-2f42130c4722-istio-podinfo\") pod \"6bd32f4f-ed9b-412a-8431-2f42130c4722\" (UID: \"6bd32f4f-ed9b-412a-8431-2f42130c4722\") " Jul 16 01:06:23.811842 ip-10-0-143-200 kubenswrapper[2572]: I0716 01:06:23.811733 2572 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-sdzhl\" (UniqueName: \"kubernetes.io/projected/6bd32f4f-ed9b-412a-8431-2f42130c4722-kube-api-access-sdzhl\") pod \"6bd32f4f-ed9b-412a-8431-2f42130c4722\" (UID: \"6bd32f4f-ed9b-412a-8431-2f42130c4722\") " Jul 16 01:06:23.812051 ip-10-0-143-200 kubenswrapper[2572]: I0716 01:06:23.811891 2572 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/6bd32f4f-ed9b-412a-8431-2f42130c4722-workload-socket" (OuterVolumeSpecName: "workload-socket") pod "6bd32f4f-ed9b-412a-8431-2f42130c4722" (UID: "6bd32f4f-ed9b-412a-8431-2f42130c4722"). InnerVolumeSpecName "workload-socket". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Jul 16 01:06:23.812051 ip-10-0-143-200 kubenswrapper[2572]: I0716 01:06:23.812030 2572 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/6bd32f4f-ed9b-412a-8431-2f42130c4722-workload-certs" (OuterVolumeSpecName: "workload-certs") pod "6bd32f4f-ed9b-412a-8431-2f42130c4722" (UID: "6bd32f4f-ed9b-412a-8431-2f42130c4722"). InnerVolumeSpecName "workload-certs". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Jul 16 01:06:23.812158 ip-10-0-143-200 kubenswrapper[2572]: I0716 01:06:23.812054 2572 reconciler_common.go:299] "Volume detached for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/6bd32f4f-ed9b-412a-8431-2f42130c4722-workload-socket\") on node \"ip-10-0-143-200.ec2.internal\" DevicePath \"\"" Jul 16 01:06:23.812158 ip-10-0-143-200 kubenswrapper[2572]: I0716 01:06:23.812099 2572 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/6bd32f4f-ed9b-412a-8431-2f42130c4722-istiod-ca-cert" (OuterVolumeSpecName: "istiod-ca-cert") pod "6bd32f4f-ed9b-412a-8431-2f42130c4722" (UID: "6bd32f4f-ed9b-412a-8431-2f42130c4722"). InnerVolumeSpecName "istiod-ca-cert". PluginName "kubernetes.io/configmap", VolumeGIDValue "" Jul 16 01:06:23.812281 ip-10-0-143-200 kubenswrapper[2572]: I0716 01:06:23.812256 2572 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/6bd32f4f-ed9b-412a-8431-2f42130c4722-credential-socket" (OuterVolumeSpecName: "credential-socket") pod "6bd32f4f-ed9b-412a-8431-2f42130c4722" (UID: "6bd32f4f-ed9b-412a-8431-2f42130c4722"). InnerVolumeSpecName "credential-socket". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Jul 16 01:06:23.812773 ip-10-0-143-200 kubenswrapper[2572]: I0716 01:06:23.812751 2572 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/6bd32f4f-ed9b-412a-8431-2f42130c4722-istio-data" (OuterVolumeSpecName: "istio-data") pod "6bd32f4f-ed9b-412a-8431-2f42130c4722" (UID: "6bd32f4f-ed9b-412a-8431-2f42130c4722"). InnerVolumeSpecName "istio-data". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Jul 16 01:06:23.814023 ip-10-0-143-200 kubenswrapper[2572]: I0716 01:06:23.813990 2572 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/downward-api/6bd32f4f-ed9b-412a-8431-2f42130c4722-istio-podinfo" (OuterVolumeSpecName: "istio-podinfo") pod "6bd32f4f-ed9b-412a-8431-2f42130c4722" (UID: "6bd32f4f-ed9b-412a-8431-2f42130c4722"). InnerVolumeSpecName "istio-podinfo". PluginName "kubernetes.io/downward-api", VolumeGIDValue "" Jul 16 01:06:23.814125 ip-10-0-143-200 kubenswrapper[2572]: I0716 01:06:23.813995 2572 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/6bd32f4f-ed9b-412a-8431-2f42130c4722-kube-api-access-sdzhl" (OuterVolumeSpecName: "kube-api-access-sdzhl") pod "6bd32f4f-ed9b-412a-8431-2f42130c4722" (UID: "6bd32f4f-ed9b-412a-8431-2f42130c4722"). InnerVolumeSpecName "kube-api-access-sdzhl". PluginName "kubernetes.io/projected", VolumeGIDValue "" Jul 16 01:06:23.814125 ip-10-0-143-200 kubenswrapper[2572]: I0716 01:06:23.814104 2572 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/6bd32f4f-ed9b-412a-8431-2f42130c4722-istio-envoy" (OuterVolumeSpecName: "istio-envoy") pod "6bd32f4f-ed9b-412a-8431-2f42130c4722" (UID: "6bd32f4f-ed9b-412a-8431-2f42130c4722"). InnerVolumeSpecName "istio-envoy". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Jul 16 01:06:23.814208 ip-10-0-143-200 kubenswrapper[2572]: I0716 01:06:23.814119 2572 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/6bd32f4f-ed9b-412a-8431-2f42130c4722-istio-token" (OuterVolumeSpecName: "istio-token") pod "6bd32f4f-ed9b-412a-8431-2f42130c4722" (UID: "6bd32f4f-ed9b-412a-8431-2f42130c4722"). InnerVolumeSpecName "istio-token". PluginName "kubernetes.io/projected", VolumeGIDValue "" Jul 16 01:06:23.906837 ip-10-0-143-200 kubenswrapper[2572]: I0716 01:06:23.906799 2572 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9gnmq_3a7e030d-fad8-480d-9dc6-fe5057048cf9/konnectivity-agent/0.log" limit=1024 Jul 16 01:06:23.912994 ip-10-0-143-200 kubenswrapper[2572]: I0716 01:06:23.912972 2572 reconciler_common.go:299] "Volume detached for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/6bd32f4f-ed9b-412a-8431-2f42130c4722-istio-token\") on node \"ip-10-0-143-200.ec2.internal\" DevicePath \"\"" Jul 16 01:06:23.913071 ip-10-0-143-200 kubenswrapper[2572]: I0716 01:06:23.912999 2572 reconciler_common.go:299] "Volume detached for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/6bd32f4f-ed9b-412a-8431-2f42130c4722-credential-socket\") on node \"ip-10-0-143-200.ec2.internal\" DevicePath \"\"" Jul 16 01:06:23.913071 ip-10-0-143-200 kubenswrapper[2572]: I0716 01:06:23.913009 2572 reconciler_common.go:299] "Volume detached for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/6bd32f4f-ed9b-412a-8431-2f42130c4722-istiod-ca-cert\") on node \"ip-10-0-143-200.ec2.internal\" DevicePath \"\"" Jul 16 01:06:23.913071 ip-10-0-143-200 kubenswrapper[2572]: I0716 01:06:23.913018 2572 reconciler_common.go:299] "Volume detached for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/6bd32f4f-ed9b-412a-8431-2f42130c4722-istio-envoy\") on node \"ip-10-0-143-200.ec2.internal\" DevicePath \"\"" Jul 16 01:06:23.913071 ip-10-0-143-200 kubenswrapper[2572]: I0716 01:06:23.913027 2572 reconciler_common.go:299] "Volume detached for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/6bd32f4f-ed9b-412a-8431-2f42130c4722-istio-data\") on node \"ip-10-0-143-200.ec2.internal\" DevicePath \"\"" Jul 16 01:06:23.913071 ip-10-0-143-200 kubenswrapper[2572]: I0716 01:06:23.913034 2572 reconciler_common.go:299] "Volume detached for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/6bd32f4f-ed9b-412a-8431-2f42130c4722-workload-certs\") on node \"ip-10-0-143-200.ec2.internal\" DevicePath \"\"" Jul 16 01:06:23.913071 ip-10-0-143-200 kubenswrapper[2572]: I0716 01:06:23.913042 2572 reconciler_common.go:299] "Volume detached for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/6bd32f4f-ed9b-412a-8431-2f42130c4722-istio-podinfo\") on node \"ip-10-0-143-200.ec2.internal\" DevicePath \"\"" Jul 16 01:06:23.913071 ip-10-0-143-200 kubenswrapper[2572]: I0716 01:06:23.913050 2572 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-sdzhl\" (UniqueName: \"kubernetes.io/projected/6bd32f4f-ed9b-412a-8431-2f42130c4722-kube-api-access-sdzhl\") on node \"ip-10-0-143-200.ec2.internal\" DevicePath \"\"" Jul 16 01:06:24.182118 ip-10-0-143-200 kubenswrapper[2572]: I0716 01:06:24.182016 2572 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9gnmq_3a7e030d-fad8-480d-9dc6-fe5057048cf9/konnectivity-agent/0.log" limit=1024 Jul 16 01:06:24.379039 ip-10-0-143-200 kubenswrapper[2572]: I0716 01:06:24.379004 2572 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9gnmq_3a7e030d-fad8-480d-9dc6-fe5057048cf9/konnectivity-agent/0.log" limit=1024 Jul 16 01:06:24.430454 ip-10-0-143-200 kubenswrapper[2572]: I0716 01:06:24.430423 2572 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-ingress/e2e-migrate-cefec0b9-gw-openshift-default-85fc975755-sqmz7"] Jul 16 01:06:24.430946 ip-10-0-143-200 kubenswrapper[2572]: I0716 01:06:24.430916 2572 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="6bd32f4f-ed9b-412a-8431-2f42130c4722" containerName="istio-proxy" Jul 16 01:06:24.430946 ip-10-0-143-200 kubenswrapper[2572]: I0716 01:06:24.430937 2572 state_mem.go:107] "Deleted CPUSet assignment" podUID="6bd32f4f-ed9b-412a-8431-2f42130c4722" containerName="istio-proxy" Jul 16 01:06:24.431108 ip-10-0-143-200 kubenswrapper[2572]: I0716 01:06:24.431040 2572 memory_manager.go:368] "RemoveStaleState removing state" podUID="6bd32f4f-ed9b-412a-8431-2f42130c4722" containerName="istio-proxy" Jul 16 01:06:24.433386 ip-10-0-143-200 kubenswrapper[2572]: I0716 01:06:24.433342 2572 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-migrate-cefec0b9-gw-openshift-default-85fc975755-sqmz7" Jul 16 01:06:24.444008 ip-10-0-143-200 kubenswrapper[2572]: I0716 01:06:24.443982 2572 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress\"/\"e2e-migrate-cefec0b9-gw-openshift-default-dockercfg-nl8v8\"" Jul 16 01:06:24.467313 ip-10-0-143-200 kubenswrapper[2572]: I0716 01:06:24.467277 2572 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/e2e-migrate-cefec0b9-gw-openshift-default-85fc975755-sqmz7"] Jul 16 01:06:24.495963 ip-10-0-143-200 kubenswrapper[2572]: I0716 01:06:24.495929 2572 generic.go:356] "Generic (PLEG): container finished" podID="6bd32f4f-ed9b-412a-8431-2f42130c4722" containerID="c2befe6f026de5a2c7b3922db4cd2bece34317ac042b978c804ec5e570d9701b" exitCode=0 Jul 16 01:06:24.496144 ip-10-0-143-200 kubenswrapper[2572]: I0716 01:06:24.495979 2572 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-ait-241c63c6-openshift-default-85dccf9599-4pvbp" Jul 16 01:06:24.496144 ip-10-0-143-200 kubenswrapper[2572]: I0716 01:06:24.495988 2572 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-ait-241c63c6-openshift-default-85dccf9599-4pvbp" event={"ID":"6bd32f4f-ed9b-412a-8431-2f42130c4722","Type":"ContainerDied","Data":"c2befe6f026de5a2c7b3922db4cd2bece34317ac042b978c804ec5e570d9701b"} Jul 16 01:06:24.496144 ip-10-0-143-200 kubenswrapper[2572]: I0716 01:06:24.496032 2572 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-ait-241c63c6-openshift-default-85dccf9599-4pvbp" event={"ID":"6bd32f4f-ed9b-412a-8431-2f42130c4722","Type":"ContainerDied","Data":"51152b35c81bb876d1d759c68549bda1579f2ffde8cc2e1d8c9d6f23667920f7"} Jul 16 01:06:24.496144 ip-10-0-143-200 kubenswrapper[2572]: I0716 01:06:24.496056 2572 scope.go:117] "RemoveContainer" containerID="c2befe6f026de5a2c7b3922db4cd2bece34317ac042b978c804ec5e570d9701b" Jul 16 01:06:24.505976 ip-10-0-143-200 kubenswrapper[2572]: I0716 01:06:24.505955 2572 scope.go:117] "RemoveContainer" containerID="c2befe6f026de5a2c7b3922db4cd2bece34317ac042b978c804ec5e570d9701b" Jul 16 01:06:24.506270 ip-10-0-143-200 kubenswrapper[2572]: E0716 01:06:24.506245 2572 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"c2befe6f026de5a2c7b3922db4cd2bece34317ac042b978c804ec5e570d9701b\": container with ID starting with c2befe6f026de5a2c7b3922db4cd2bece34317ac042b978c804ec5e570d9701b not found: ID does not exist" containerID="c2befe6f026de5a2c7b3922db4cd2bece34317ac042b978c804ec5e570d9701b" Jul 16 01:06:24.506339 ip-10-0-143-200 kubenswrapper[2572]: I0716 01:06:24.506283 2572 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"c2befe6f026de5a2c7b3922db4cd2bece34317ac042b978c804ec5e570d9701b"} err="failed to get container status \"c2befe6f026de5a2c7b3922db4cd2bece34317ac042b978c804ec5e570d9701b\": rpc error: code = NotFound desc = could not find container \"c2befe6f026de5a2c7b3922db4cd2bece34317ac042b978c804ec5e570d9701b\": container with ID starting with c2befe6f026de5a2c7b3922db4cd2bece34317ac042b978c804ec5e570d9701b not found: ID does not exist" Jul 16 01:06:24.519833 ip-10-0-143-200 kubenswrapper[2572]: I0716 01:06:24.519797 2572 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/c2465b13-bf6b-4f0c-b215-a718fd9a34f1-istio-token\") pod \"e2e-migrate-cefec0b9-gw-openshift-default-85fc975755-sqmz7\" (UID: \"c2465b13-bf6b-4f0c-b215-a718fd9a34f1\") " pod="openshift-ingress/e2e-migrate-cefec0b9-gw-openshift-default-85fc975755-sqmz7" Jul 16 01:06:24.519989 ip-10-0-143-200 kubenswrapper[2572]: I0716 01:06:24.519861 2572 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/c2465b13-bf6b-4f0c-b215-a718fd9a34f1-credential-socket\") pod \"e2e-migrate-cefec0b9-gw-openshift-default-85fc975755-sqmz7\" (UID: \"c2465b13-bf6b-4f0c-b215-a718fd9a34f1\") " pod="openshift-ingress/e2e-migrate-cefec0b9-gw-openshift-default-85fc975755-sqmz7" Jul 16 01:06:24.519989 ip-10-0-143-200 kubenswrapper[2572]: I0716 01:06:24.519880 2572 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/c2465b13-bf6b-4f0c-b215-a718fd9a34f1-istio-envoy\") pod \"e2e-migrate-cefec0b9-gw-openshift-default-85fc975755-sqmz7\" (UID: \"c2465b13-bf6b-4f0c-b215-a718fd9a34f1\") " pod="openshift-ingress/e2e-migrate-cefec0b9-gw-openshift-default-85fc975755-sqmz7" Jul 16 01:06:24.520204 ip-10-0-143-200 kubenswrapper[2572]: I0716 01:06:24.520184 2572 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/c2465b13-bf6b-4f0c-b215-a718fd9a34f1-workload-certs\") pod \"e2e-migrate-cefec0b9-gw-openshift-default-85fc975755-sqmz7\" (UID: \"c2465b13-bf6b-4f0c-b215-a718fd9a34f1\") " pod="openshift-ingress/e2e-migrate-cefec0b9-gw-openshift-default-85fc975755-sqmz7" Jul 16 01:06:24.520258 ip-10-0-143-200 kubenswrapper[2572]: I0716 01:06:24.520226 2572 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/c2465b13-bf6b-4f0c-b215-a718fd9a34f1-istio-podinfo\") pod \"e2e-migrate-cefec0b9-gw-openshift-default-85fc975755-sqmz7\" (UID: \"c2465b13-bf6b-4f0c-b215-a718fd9a34f1\") " pod="openshift-ingress/e2e-migrate-cefec0b9-gw-openshift-default-85fc975755-sqmz7" Jul 16 01:06:24.520318 ip-10-0-143-200 kubenswrapper[2572]: I0716 01:06:24.520292 2572 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/c2465b13-bf6b-4f0c-b215-a718fd9a34f1-workload-socket\") pod \"e2e-migrate-cefec0b9-gw-openshift-default-85fc975755-sqmz7\" (UID: \"c2465b13-bf6b-4f0c-b215-a718fd9a34f1\") " pod="openshift-ingress/e2e-migrate-cefec0b9-gw-openshift-default-85fc975755-sqmz7" Jul 16 01:06:24.520362 ip-10-0-143-200 kubenswrapper[2572]: I0716 01:06:24.520333 2572 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/c2465b13-bf6b-4f0c-b215-a718fd9a34f1-istio-data\") pod \"e2e-migrate-cefec0b9-gw-openshift-default-85fc975755-sqmz7\" (UID: \"c2465b13-bf6b-4f0c-b215-a718fd9a34f1\") " pod="openshift-ingress/e2e-migrate-cefec0b9-gw-openshift-default-85fc975755-sqmz7" Jul 16 01:06:24.520412 ip-10-0-143-200 kubenswrapper[2572]: I0716 01:06:24.520385 2572 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/c2465b13-bf6b-4f0c-b215-a718fd9a34f1-istiod-ca-cert\") pod \"e2e-migrate-cefec0b9-gw-openshift-default-85fc975755-sqmz7\" (UID: \"c2465b13-bf6b-4f0c-b215-a718fd9a34f1\") " pod="openshift-ingress/e2e-migrate-cefec0b9-gw-openshift-default-85fc975755-sqmz7" Jul 16 01:06:24.520505 ip-10-0-143-200 kubenswrapper[2572]: I0716 01:06:24.520486 2572 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-hqp7f\" (UniqueName: \"kubernetes.io/projected/c2465b13-bf6b-4f0c-b215-a718fd9a34f1-kube-api-access-hqp7f\") pod \"e2e-migrate-cefec0b9-gw-openshift-default-85fc975755-sqmz7\" (UID: \"c2465b13-bf6b-4f0c-b215-a718fd9a34f1\") " pod="openshift-ingress/e2e-migrate-cefec0b9-gw-openshift-default-85fc975755-sqmz7" Jul 16 01:06:24.554260 ip-10-0-143-200 kubenswrapper[2572]: I0716 01:06:24.554221 2572 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-ingress/e2e-ait-241c63c6-openshift-default-85dccf9599-4pvbp"] Jul 16 01:06:24.564072 ip-10-0-143-200 kubenswrapper[2572]: I0716 01:06:24.564045 2572 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9gnmq_3a7e030d-fad8-480d-9dc6-fe5057048cf9/konnectivity-agent/0.log" limit=1024 Jul 16 01:06:24.582613 ip-10-0-143-200 kubenswrapper[2572]: I0716 01:06:24.582569 2572 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["openshift-ingress/e2e-ait-241c63c6-openshift-default-85dccf9599-4pvbp"] Jul 16 01:06:24.621421 ip-10-0-143-200 kubenswrapper[2572]: I0716 01:06:24.621387 2572 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/c2465b13-bf6b-4f0c-b215-a718fd9a34f1-workload-socket\") pod \"e2e-migrate-cefec0b9-gw-openshift-default-85fc975755-sqmz7\" (UID: \"c2465b13-bf6b-4f0c-b215-a718fd9a34f1\") " pod="openshift-ingress/e2e-migrate-cefec0b9-gw-openshift-default-85fc975755-sqmz7" Jul 16 01:06:24.621609 ip-10-0-143-200 kubenswrapper[2572]: I0716 01:06:24.621425 2572 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/c2465b13-bf6b-4f0c-b215-a718fd9a34f1-istio-data\") pod \"e2e-migrate-cefec0b9-gw-openshift-default-85fc975755-sqmz7\" (UID: \"c2465b13-bf6b-4f0c-b215-a718fd9a34f1\") " pod="openshift-ingress/e2e-migrate-cefec0b9-gw-openshift-default-85fc975755-sqmz7" Jul 16 01:06:24.621609 ip-10-0-143-200 kubenswrapper[2572]: I0716 01:06:24.621466 2572 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/c2465b13-bf6b-4f0c-b215-a718fd9a34f1-istiod-ca-cert\") pod \"e2e-migrate-cefec0b9-gw-openshift-default-85fc975755-sqmz7\" (UID: \"c2465b13-bf6b-4f0c-b215-a718fd9a34f1\") " pod="openshift-ingress/e2e-migrate-cefec0b9-gw-openshift-default-85fc975755-sqmz7" Jul 16 01:06:24.621609 ip-10-0-143-200 kubenswrapper[2572]: I0716 01:06:24.621512 2572 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-hqp7f\" (UniqueName: \"kubernetes.io/projected/c2465b13-bf6b-4f0c-b215-a718fd9a34f1-kube-api-access-hqp7f\") pod \"e2e-migrate-cefec0b9-gw-openshift-default-85fc975755-sqmz7\" (UID: \"c2465b13-bf6b-4f0c-b215-a718fd9a34f1\") " pod="openshift-ingress/e2e-migrate-cefec0b9-gw-openshift-default-85fc975755-sqmz7" Jul 16 01:06:24.621609 ip-10-0-143-200 kubenswrapper[2572]: I0716 01:06:24.621584 2572 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/c2465b13-bf6b-4f0c-b215-a718fd9a34f1-istio-token\") pod \"e2e-migrate-cefec0b9-gw-openshift-default-85fc975755-sqmz7\" (UID: \"c2465b13-bf6b-4f0c-b215-a718fd9a34f1\") " pod="openshift-ingress/e2e-migrate-cefec0b9-gw-openshift-default-85fc975755-sqmz7" Jul 16 01:06:24.621835 ip-10-0-143-200 kubenswrapper[2572]: I0716 01:06:24.621678 2572 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/c2465b13-bf6b-4f0c-b215-a718fd9a34f1-credential-socket\") pod \"e2e-migrate-cefec0b9-gw-openshift-default-85fc975755-sqmz7\" (UID: \"c2465b13-bf6b-4f0c-b215-a718fd9a34f1\") " pod="openshift-ingress/e2e-migrate-cefec0b9-gw-openshift-default-85fc975755-sqmz7" Jul 16 01:06:24.621835 ip-10-0-143-200 kubenswrapper[2572]: I0716 01:06:24.621706 2572 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/c2465b13-bf6b-4f0c-b215-a718fd9a34f1-istio-envoy\") pod \"e2e-migrate-cefec0b9-gw-openshift-default-85fc975755-sqmz7\" (UID: \"c2465b13-bf6b-4f0c-b215-a718fd9a34f1\") " pod="openshift-ingress/e2e-migrate-cefec0b9-gw-openshift-default-85fc975755-sqmz7" Jul 16 01:06:24.621835 ip-10-0-143-200 kubenswrapper[2572]: I0716 01:06:24.621772 2572 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/c2465b13-bf6b-4f0c-b215-a718fd9a34f1-workload-certs\") pod \"e2e-migrate-cefec0b9-gw-openshift-default-85fc975755-sqmz7\" (UID: \"c2465b13-bf6b-4f0c-b215-a718fd9a34f1\") " pod="openshift-ingress/e2e-migrate-cefec0b9-gw-openshift-default-85fc975755-sqmz7" Jul 16 01:06:24.621970 ip-10-0-143-200 kubenswrapper[2572]: I0716 01:06:24.621812 2572 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/c2465b13-bf6b-4f0c-b215-a718fd9a34f1-istio-podinfo\") pod \"e2e-migrate-cefec0b9-gw-openshift-default-85fc975755-sqmz7\" (UID: \"c2465b13-bf6b-4f0c-b215-a718fd9a34f1\") " pod="openshift-ingress/e2e-migrate-cefec0b9-gw-openshift-default-85fc975755-sqmz7" Jul 16 01:06:24.621970 ip-10-0-143-200 kubenswrapper[2572]: I0716 01:06:24.621908 2572 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/c2465b13-bf6b-4f0c-b215-a718fd9a34f1-workload-socket\") pod \"e2e-migrate-cefec0b9-gw-openshift-default-85fc975755-sqmz7\" (UID: \"c2465b13-bf6b-4f0c-b215-a718fd9a34f1\") " pod="openshift-ingress/e2e-migrate-cefec0b9-gw-openshift-default-85fc975755-sqmz7" Jul 16 01:06:24.622117 ip-10-0-143-200 kubenswrapper[2572]: I0716 01:06:24.622095 2572 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/c2465b13-bf6b-4f0c-b215-a718fd9a34f1-credential-socket\") pod \"e2e-migrate-cefec0b9-gw-openshift-default-85fc975755-sqmz7\" (UID: \"c2465b13-bf6b-4f0c-b215-a718fd9a34f1\") " pod="openshift-ingress/e2e-migrate-cefec0b9-gw-openshift-default-85fc975755-sqmz7" Jul 16 01:06:24.622212 ip-10-0-143-200 kubenswrapper[2572]: I0716 01:06:24.622183 2572 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/c2465b13-bf6b-4f0c-b215-a718fd9a34f1-workload-certs\") pod \"e2e-migrate-cefec0b9-gw-openshift-default-85fc975755-sqmz7\" (UID: \"c2465b13-bf6b-4f0c-b215-a718fd9a34f1\") " pod="openshift-ingress/e2e-migrate-cefec0b9-gw-openshift-default-85fc975755-sqmz7" Jul 16 01:06:24.622278 ip-10-0-143-200 kubenswrapper[2572]: I0716 01:06:24.622221 2572 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/c2465b13-bf6b-4f0c-b215-a718fd9a34f1-istio-data\") pod \"e2e-migrate-cefec0b9-gw-openshift-default-85fc975755-sqmz7\" (UID: \"c2465b13-bf6b-4f0c-b215-a718fd9a34f1\") " pod="openshift-ingress/e2e-migrate-cefec0b9-gw-openshift-default-85fc975755-sqmz7" Jul 16 01:06:24.622545 ip-10-0-143-200 kubenswrapper[2572]: I0716 01:06:24.622523 2572 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/c2465b13-bf6b-4f0c-b215-a718fd9a34f1-istiod-ca-cert\") pod \"e2e-migrate-cefec0b9-gw-openshift-default-85fc975755-sqmz7\" (UID: \"c2465b13-bf6b-4f0c-b215-a718fd9a34f1\") " pod="openshift-ingress/e2e-migrate-cefec0b9-gw-openshift-default-85fc975755-sqmz7" Jul 16 01:06:24.624004 ip-10-0-143-200 kubenswrapper[2572]: I0716 01:06:24.623984 2572 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/c2465b13-bf6b-4f0c-b215-a718fd9a34f1-istio-envoy\") pod \"e2e-migrate-cefec0b9-gw-openshift-default-85fc975755-sqmz7\" (UID: \"c2465b13-bf6b-4f0c-b215-a718fd9a34f1\") " pod="openshift-ingress/e2e-migrate-cefec0b9-gw-openshift-default-85fc975755-sqmz7" Jul 16 01:06:24.624265 ip-10-0-143-200 kubenswrapper[2572]: I0716 01:06:24.624247 2572 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/c2465b13-bf6b-4f0c-b215-a718fd9a34f1-istio-podinfo\") pod \"e2e-migrate-cefec0b9-gw-openshift-default-85fc975755-sqmz7\" (UID: \"c2465b13-bf6b-4f0c-b215-a718fd9a34f1\") " pod="openshift-ingress/e2e-migrate-cefec0b9-gw-openshift-default-85fc975755-sqmz7" Jul 16 01:06:24.637874 ip-10-0-143-200 kubenswrapper[2572]: I0716 01:06:24.637839 2572 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-hqp7f\" (UniqueName: \"kubernetes.io/projected/c2465b13-bf6b-4f0c-b215-a718fd9a34f1-kube-api-access-hqp7f\") pod \"e2e-migrate-cefec0b9-gw-openshift-default-85fc975755-sqmz7\" (UID: \"c2465b13-bf6b-4f0c-b215-a718fd9a34f1\") " pod="openshift-ingress/e2e-migrate-cefec0b9-gw-openshift-default-85fc975755-sqmz7" Jul 16 01:06:24.639736 ip-10-0-143-200 kubenswrapper[2572]: I0716 01:06:24.639680 2572 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/c2465b13-bf6b-4f0c-b215-a718fd9a34f1-istio-token\") pod \"e2e-migrate-cefec0b9-gw-openshift-default-85fc975755-sqmz7\" (UID: \"c2465b13-bf6b-4f0c-b215-a718fd9a34f1\") " pod="openshift-ingress/e2e-migrate-cefec0b9-gw-openshift-default-85fc975755-sqmz7" Jul 16 01:06:24.729763 ip-10-0-143-200 kubenswrapper[2572]: I0716 01:06:24.729727 2572 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9gnmq_3a7e030d-fad8-480d-9dc6-fe5057048cf9/konnectivity-agent/0.log" limit=1024 Jul 16 01:06:24.745279 ip-10-0-143-200 kubenswrapper[2572]: I0716 01:06:24.745254 2572 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-migrate-cefec0b9-gw-openshift-default-85fc975755-sqmz7" Jul 16 01:06:24.892369 ip-10-0-143-200 kubenswrapper[2572]: I0716 01:06:24.892311 2572 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9gnmq_3a7e030d-fad8-480d-9dc6-fe5057048cf9/konnectivity-agent/0.log" limit=1024 Jul 16 01:06:24.918639 ip-10-0-143-200 kubenswrapper[2572]: I0716 01:06:24.918595 2572 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/e2e-migrate-cefec0b9-gw-openshift-default-85fc975755-sqmz7"] Jul 16 01:06:24.920394 ip-10-0-143-200 kubenswrapper[2572]: W0716 01:06:24.920363 2572 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podc2465b13_bf6b_4f0c_b215_a718fd9a34f1.slice/crio-ebbbaa5a52da2ebed4ae24a33c6d86c5c71401b8c3aa1c612a09898315bb80d4 WatchSource:0}: Error finding container ebbbaa5a52da2ebed4ae24a33c6d86c5c71401b8c3aa1c612a09898315bb80d4: Status 404 returned error can't find the container with id ebbbaa5a52da2ebed4ae24a33c6d86c5c71401b8c3aa1c612a09898315bb80d4 Jul 16 01:06:24.922546 ip-10-0-143-200 kubenswrapper[2572]: I0716 01:06:24.922518 2572 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236228Ki","pods":"250"} Jul 16 01:06:24.922619 ip-10-0-143-200 kubenswrapper[2572]: I0716 01:06:24.922578 2572 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236228Ki","pods":"250"} Jul 16 01:06:24.922677 ip-10-0-143-200 kubenswrapper[2572]: I0716 01:06:24.922618 2572 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236228Ki","pods":"250"} Jul 16 01:06:25.069580 ip-10-0-143-200 kubenswrapper[2572]: I0716 01:06:25.069514 2572 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9gnmq_3a7e030d-fad8-480d-9dc6-fe5057048cf9/konnectivity-agent/0.log" limit=1024 Jul 16 01:06:25.270159 ip-10-0-143-200 kubenswrapper[2572]: I0716 01:06:25.270063 2572 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9gnmq_3a7e030d-fad8-480d-9dc6-fe5057048cf9/konnectivity-agent/0.log" limit=1024 Jul 16 01:06:25.450139 ip-10-0-143-200 kubenswrapper[2572]: I0716 01:06:25.450103 2572 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="6bd32f4f-ed9b-412a-8431-2f42130c4722" path="/var/lib/kubelet/pods/6bd32f4f-ed9b-412a-8431-2f42130c4722/volumes" Jul 16 01:06:25.503275 ip-10-0-143-200 kubenswrapper[2572]: I0716 01:06:25.503230 2572 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-migrate-cefec0b9-gw-openshift-default-85fc975755-sqmz7" event={"ID":"c2465b13-bf6b-4f0c-b215-a718fd9a34f1","Type":"ContainerStarted","Data":"136fe22e48e0f1170c94928c1d4aa885d76e38487c9e8174922fb3dada02f639"} Jul 16 01:06:25.503492 ip-10-0-143-200 kubenswrapper[2572]: I0716 01:06:25.503290 2572 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-migrate-cefec0b9-gw-openshift-default-85fc975755-sqmz7" event={"ID":"c2465b13-bf6b-4f0c-b215-a718fd9a34f1","Type":"ContainerStarted","Data":"ebbbaa5a52da2ebed4ae24a33c6d86c5c71401b8c3aa1c612a09898315bb80d4"} Jul 16 01:06:25.537114 ip-10-0-143-200 kubenswrapper[2572]: I0716 01:06:25.537010 2572 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-ingress/e2e-migrate-cefec0b9-gw-openshift-default-85fc975755-sqmz7" podStartSLOduration=1.536995001 podStartE2EDuration="1.536995001s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-16 01:06:24 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-16 01:06:25.535316557 +0000 UTC m=+1590.644071231" watchObservedRunningTime="2026-07-16 01:06:25.536995001 +0000 UTC m=+1590.645749743" Jul 16 01:06:25.746030 ip-10-0-143-200 kubenswrapper[2572]: I0716 01:06:25.745990 2572 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="openshift-ingress/e2e-migrate-cefec0b9-gw-openshift-default-85fc975755-sqmz7" Jul 16 01:06:25.749513 ip-10-0-143-200 kubenswrapper[2572]: I0716 01:06:25.749477 2572 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9gnmq_3a7e030d-fad8-480d-9dc6-fe5057048cf9/konnectivity-agent/0.log" limit=1024 Jul 16 01:06:25.751892 ip-10-0-143-200 kubenswrapper[2572]: I0716 01:06:25.751871 2572 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="openshift-ingress/e2e-migrate-cefec0b9-gw-openshift-default-85fc975755-sqmz7" Jul 16 01:06:25.908140 ip-10-0-143-200 kubenswrapper[2572]: I0716 01:06:25.907984 2572 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9gnmq_3a7e030d-fad8-480d-9dc6-fe5057048cf9/konnectivity-agent/0.log" limit=1024 Jul 16 01:06:26.510683 ip-10-0-143-200 kubenswrapper[2572]: I0716 01:06:26.510649 2572 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/e2e-migrate-cefec0b9-gw-openshift-default-85fc975755-sqmz7" Jul 16 01:06:26.511623 ip-10-0-143-200 kubenswrapper[2572]: I0716 01:06:26.511597 2572 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/e2e-migrate-cefec0b9-gw-openshift-default-85fc975755-sqmz7" Jul 16 01:06:27.697330 ip-10-0-143-200 kubenswrapper[2572]: I0716 01:06:27.697298 2572 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9gnmq_3a7e030d-fad8-480d-9dc6-fe5057048cf9/konnectivity-agent/0.log" limit=1024 Jul 16 01:06:30.252123 ip-10-0-143-200 kubenswrapper[2572]: I0716 01:06:30.252089 2572 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-ingress/e2e-migrate-cefec0b9-gw-openshift-default-85fc975755-sqmz7"] Jul 16 01:06:30.252528 ip-10-0-143-200 kubenswrapper[2572]: I0716 01:06:30.252271 2572 kuberuntime_container.go:873] "Killing container with a grace period" pod="openshift-ingress/e2e-migrate-cefec0b9-gw-openshift-default-85fc975755-sqmz7" podUID="c2465b13-bf6b-4f0c-b215-a718fd9a34f1" containerName="istio-proxy" containerID="cri-o://136fe22e48e0f1170c94928c1d4aa885d76e38487c9e8174922fb3dada02f639" gracePeriod=30 Jul 16 01:06:30.281575 ip-10-0-143-200 kubenswrapper[2572]: I0716 01:06:30.281546 2572 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9gnmq_3a7e030d-fad8-480d-9dc6-fe5057048cf9/konnectivity-agent/0.log" limit=1024 Jul 16 01:06:30.386267 ip-10-0-143-200 kubenswrapper[2572]: I0716 01:06:30.386228 2572 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9gnmq_3a7e030d-fad8-480d-9dc6-fe5057048cf9/konnectivity-agent/0.log" limit=1024 Jul 16 01:06:31.613407 ip-10-0-143-200 kubenswrapper[2572]: I0716 01:06:31.613307 2572 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9gnmq_3a7e030d-fad8-480d-9dc6-fe5057048cf9/konnectivity-agent/0.log" limit=1024 Jul 16 01:06:36.684081 ip-10-0-143-200 kubenswrapper[2572]: I0716 01:06:36.684045 2572 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9gnmq_3a7e030d-fad8-480d-9dc6-fe5057048cf9/konnectivity-agent/0.log" limit=1024 Jul 16 01:06:36.788770 ip-10-0-143-200 kubenswrapper[2572]: I0716 01:06:36.788667 2572 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9gnmq_3a7e030d-fad8-480d-9dc6-fe5057048cf9/konnectivity-agent/0.log" limit=1024 Jul 16 01:06:36.826926 ip-10-0-143-200 kubenswrapper[2572]: I0716 01:06:36.826850 2572 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-migrate-cefec0b9-gw-openshift-default-85fc975755-sqmz7" Jul 16 01:06:36.848491 ip-10-0-143-200 kubenswrapper[2572]: I0716 01:06:36.848459 2572 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/c2465b13-bf6b-4f0c-b215-a718fd9a34f1-istio-envoy\") pod \"c2465b13-bf6b-4f0c-b215-a718fd9a34f1\" (UID: \"c2465b13-bf6b-4f0c-b215-a718fd9a34f1\") " Jul 16 01:06:36.848491 ip-10-0-143-200 kubenswrapper[2572]: I0716 01:06:36.848491 2572 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/c2465b13-bf6b-4f0c-b215-a718fd9a34f1-istio-podinfo\") pod \"c2465b13-bf6b-4f0c-b215-a718fd9a34f1\" (UID: \"c2465b13-bf6b-4f0c-b215-a718fd9a34f1\") " Jul 16 01:06:36.848743 ip-10-0-143-200 kubenswrapper[2572]: I0716 01:06:36.848512 2572 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/c2465b13-bf6b-4f0c-b215-a718fd9a34f1-workload-socket\") pod \"c2465b13-bf6b-4f0c-b215-a718fd9a34f1\" (UID: \"c2465b13-bf6b-4f0c-b215-a718fd9a34f1\") " Jul 16 01:06:36.848743 ip-10-0-143-200 kubenswrapper[2572]: I0716 01:06:36.848549 2572 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/c2465b13-bf6b-4f0c-b215-a718fd9a34f1-istio-data\") pod \"c2465b13-bf6b-4f0c-b215-a718fd9a34f1\" (UID: \"c2465b13-bf6b-4f0c-b215-a718fd9a34f1\") " Jul 16 01:06:36.848743 ip-10-0-143-200 kubenswrapper[2572]: I0716 01:06:36.848566 2572 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/c2465b13-bf6b-4f0c-b215-a718fd9a34f1-istiod-ca-cert\") pod \"c2465b13-bf6b-4f0c-b215-a718fd9a34f1\" (UID: \"c2465b13-bf6b-4f0c-b215-a718fd9a34f1\") " Jul 16 01:06:36.848743 ip-10-0-143-200 kubenswrapper[2572]: I0716 01:06:36.848585 2572 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/c2465b13-bf6b-4f0c-b215-a718fd9a34f1-istio-token\") pod \"c2465b13-bf6b-4f0c-b215-a718fd9a34f1\" (UID: \"c2465b13-bf6b-4f0c-b215-a718fd9a34f1\") " Jul 16 01:06:36.848743 ip-10-0-143-200 kubenswrapper[2572]: I0716 01:06:36.848604 2572 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-hqp7f\" (UniqueName: \"kubernetes.io/projected/c2465b13-bf6b-4f0c-b215-a718fd9a34f1-kube-api-access-hqp7f\") pod \"c2465b13-bf6b-4f0c-b215-a718fd9a34f1\" (UID: \"c2465b13-bf6b-4f0c-b215-a718fd9a34f1\") " Jul 16 01:06:36.848743 ip-10-0-143-200 kubenswrapper[2572]: I0716 01:06:36.848698 2572 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/c2465b13-bf6b-4f0c-b215-a718fd9a34f1-credential-socket\") pod \"c2465b13-bf6b-4f0c-b215-a718fd9a34f1\" (UID: \"c2465b13-bf6b-4f0c-b215-a718fd9a34f1\") " Jul 16 01:06:36.848743 ip-10-0-143-200 kubenswrapper[2572]: I0716 01:06:36.848728 2572 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/c2465b13-bf6b-4f0c-b215-a718fd9a34f1-workload-certs\") pod \"c2465b13-bf6b-4f0c-b215-a718fd9a34f1\" (UID: \"c2465b13-bf6b-4f0c-b215-a718fd9a34f1\") " Jul 16 01:06:36.849106 ip-10-0-143-200 kubenswrapper[2572]: I0716 01:06:36.848886 2572 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/c2465b13-bf6b-4f0c-b215-a718fd9a34f1-workload-socket" (OuterVolumeSpecName: "workload-socket") pod "c2465b13-bf6b-4f0c-b215-a718fd9a34f1" (UID: "c2465b13-bf6b-4f0c-b215-a718fd9a34f1"). InnerVolumeSpecName "workload-socket". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Jul 16 01:06:36.849106 ip-10-0-143-200 kubenswrapper[2572]: I0716 01:06:36.849054 2572 reconciler_common.go:299] "Volume detached for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/c2465b13-bf6b-4f0c-b215-a718fd9a34f1-workload-socket\") on node \"ip-10-0-143-200.ec2.internal\" DevicePath \"\"" Jul 16 01:06:36.849220 ip-10-0-143-200 kubenswrapper[2572]: I0716 01:06:36.849104 2572 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/c2465b13-bf6b-4f0c-b215-a718fd9a34f1-workload-certs" (OuterVolumeSpecName: "workload-certs") pod "c2465b13-bf6b-4f0c-b215-a718fd9a34f1" (UID: "c2465b13-bf6b-4f0c-b215-a718fd9a34f1"). InnerVolumeSpecName "workload-certs". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Jul 16 01:06:36.849276 ip-10-0-143-200 kubenswrapper[2572]: I0716 01:06:36.849229 2572 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/c2465b13-bf6b-4f0c-b215-a718fd9a34f1-credential-socket" (OuterVolumeSpecName: "credential-socket") pod "c2465b13-bf6b-4f0c-b215-a718fd9a34f1" (UID: "c2465b13-bf6b-4f0c-b215-a718fd9a34f1"). InnerVolumeSpecName "credential-socket". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Jul 16 01:06:36.849472 ip-10-0-143-200 kubenswrapper[2572]: I0716 01:06:36.849450 2572 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/c2465b13-bf6b-4f0c-b215-a718fd9a34f1-istiod-ca-cert" (OuterVolumeSpecName: "istiod-ca-cert") pod "c2465b13-bf6b-4f0c-b215-a718fd9a34f1" (UID: "c2465b13-bf6b-4f0c-b215-a718fd9a34f1"). InnerVolumeSpecName "istiod-ca-cert". PluginName "kubernetes.io/configmap", VolumeGIDValue "" Jul 16 01:06:36.849572 ip-10-0-143-200 kubenswrapper[2572]: I0716 01:06:36.849555 2572 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/c2465b13-bf6b-4f0c-b215-a718fd9a34f1-istio-data" (OuterVolumeSpecName: "istio-data") pod "c2465b13-bf6b-4f0c-b215-a718fd9a34f1" (UID: "c2465b13-bf6b-4f0c-b215-a718fd9a34f1"). InnerVolumeSpecName "istio-data". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Jul 16 01:06:36.851229 ip-10-0-143-200 kubenswrapper[2572]: I0716 01:06:36.851209 2572 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/c2465b13-bf6b-4f0c-b215-a718fd9a34f1-istio-envoy" (OuterVolumeSpecName: "istio-envoy") pod "c2465b13-bf6b-4f0c-b215-a718fd9a34f1" (UID: "c2465b13-bf6b-4f0c-b215-a718fd9a34f1"). InnerVolumeSpecName "istio-envoy". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Jul 16 01:06:36.851326 ip-10-0-143-200 kubenswrapper[2572]: I0716 01:06:36.851225 2572 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/downward-api/c2465b13-bf6b-4f0c-b215-a718fd9a34f1-istio-podinfo" (OuterVolumeSpecName: "istio-podinfo") pod "c2465b13-bf6b-4f0c-b215-a718fd9a34f1" (UID: "c2465b13-bf6b-4f0c-b215-a718fd9a34f1"). InnerVolumeSpecName "istio-podinfo". PluginName "kubernetes.io/downward-api", VolumeGIDValue "" Jul 16 01:06:36.851395 ip-10-0-143-200 kubenswrapper[2572]: I0716 01:06:36.851321 2572 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/c2465b13-bf6b-4f0c-b215-a718fd9a34f1-istio-token" (OuterVolumeSpecName: "istio-token") pod "c2465b13-bf6b-4f0c-b215-a718fd9a34f1" (UID: "c2465b13-bf6b-4f0c-b215-a718fd9a34f1"). InnerVolumeSpecName "istio-token". PluginName "kubernetes.io/projected", VolumeGIDValue "" Jul 16 01:06:36.851395 ip-10-0-143-200 kubenswrapper[2572]: I0716 01:06:36.851326 2572 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/c2465b13-bf6b-4f0c-b215-a718fd9a34f1-kube-api-access-hqp7f" (OuterVolumeSpecName: "kube-api-access-hqp7f") pod "c2465b13-bf6b-4f0c-b215-a718fd9a34f1" (UID: "c2465b13-bf6b-4f0c-b215-a718fd9a34f1"). InnerVolumeSpecName "kube-api-access-hqp7f". PluginName "kubernetes.io/projected", VolumeGIDValue "" Jul 16 01:06:36.879887 ip-10-0-143-200 kubenswrapper[2572]: I0716 01:06:36.879852 2572 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-ingress/e2e-derive-ccc0a86e-openshift-default-79d76bd567-qgd42"] Jul 16 01:06:36.880470 ip-10-0-143-200 kubenswrapper[2572]: I0716 01:06:36.880442 2572 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="c2465b13-bf6b-4f0c-b215-a718fd9a34f1" containerName="istio-proxy" Jul 16 01:06:36.880470 ip-10-0-143-200 kubenswrapper[2572]: I0716 01:06:36.880469 2572 state_mem.go:107] "Deleted CPUSet assignment" podUID="c2465b13-bf6b-4f0c-b215-a718fd9a34f1" containerName="istio-proxy" Jul 16 01:06:36.880669 ip-10-0-143-200 kubenswrapper[2572]: I0716 01:06:36.880568 2572 memory_manager.go:368] "RemoveStaleState removing state" podUID="c2465b13-bf6b-4f0c-b215-a718fd9a34f1" containerName="istio-proxy" Jul 16 01:06:36.883196 ip-10-0-143-200 kubenswrapper[2572]: I0716 01:06:36.883158 2572 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-derive-ccc0a86e-openshift-default-79d76bd567-qgd42" Jul 16 01:06:36.886292 ip-10-0-143-200 kubenswrapper[2572]: I0716 01:06:36.886269 2572 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress\"/\"e2e-derive-ccc0a86e-openshift-default-dockercfg-xsn4p\"" Jul 16 01:06:36.903669 ip-10-0-143-200 kubenswrapper[2572]: I0716 01:06:36.903619 2572 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/e2e-derive-ccc0a86e-openshift-default-79d76bd567-qgd42"] Jul 16 01:06:36.944736 ip-10-0-143-200 kubenswrapper[2572]: I0716 01:06:36.944683 2572 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9gnmq_3a7e030d-fad8-480d-9dc6-fe5057048cf9/konnectivity-agent/0.log" limit=1024 Jul 16 01:06:36.949937 ip-10-0-143-200 kubenswrapper[2572]: I0716 01:06:36.949905 2572 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/46c5f745-1157-4244-96c9-72496814e558-workload-socket\") pod \"e2e-derive-ccc0a86e-openshift-default-79d76bd567-qgd42\" (UID: \"46c5f745-1157-4244-96c9-72496814e558\") " pod="openshift-ingress/e2e-derive-ccc0a86e-openshift-default-79d76bd567-qgd42" Jul 16 01:06:36.950121 ip-10-0-143-200 kubenswrapper[2572]: I0716 01:06:36.949943 2572 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/46c5f745-1157-4244-96c9-72496814e558-istio-podinfo\") pod \"e2e-derive-ccc0a86e-openshift-default-79d76bd567-qgd42\" (UID: \"46c5f745-1157-4244-96c9-72496814e558\") " pod="openshift-ingress/e2e-derive-ccc0a86e-openshift-default-79d76bd567-qgd42" Jul 16 01:06:36.950121 ip-10-0-143-200 kubenswrapper[2572]: I0716 01:06:36.950033 2572 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/46c5f745-1157-4244-96c9-72496814e558-istio-data\") pod \"e2e-derive-ccc0a86e-openshift-default-79d76bd567-qgd42\" (UID: \"46c5f745-1157-4244-96c9-72496814e558\") " pod="openshift-ingress/e2e-derive-ccc0a86e-openshift-default-79d76bd567-qgd42" Jul 16 01:06:36.950121 ip-10-0-143-200 kubenswrapper[2572]: I0716 01:06:36.950082 2572 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/46c5f745-1157-4244-96c9-72496814e558-istiod-ca-cert\") pod \"e2e-derive-ccc0a86e-openshift-default-79d76bd567-qgd42\" (UID: \"46c5f745-1157-4244-96c9-72496814e558\") " pod="openshift-ingress/e2e-derive-ccc0a86e-openshift-default-79d76bd567-qgd42" Jul 16 01:06:36.950310 ip-10-0-143-200 kubenswrapper[2572]: I0716 01:06:36.950132 2572 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/46c5f745-1157-4244-96c9-72496814e558-workload-certs\") pod \"e2e-derive-ccc0a86e-openshift-default-79d76bd567-qgd42\" (UID: \"46c5f745-1157-4244-96c9-72496814e558\") " pod="openshift-ingress/e2e-derive-ccc0a86e-openshift-default-79d76bd567-qgd42" Jul 16 01:06:36.950310 ip-10-0-143-200 kubenswrapper[2572]: I0716 01:06:36.950267 2572 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/46c5f745-1157-4244-96c9-72496814e558-istio-token\") pod \"e2e-derive-ccc0a86e-openshift-default-79d76bd567-qgd42\" (UID: \"46c5f745-1157-4244-96c9-72496814e558\") " pod="openshift-ingress/e2e-derive-ccc0a86e-openshift-default-79d76bd567-qgd42" Jul 16 01:06:36.950310 ip-10-0-143-200 kubenswrapper[2572]: I0716 01:06:36.950301 2572 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/46c5f745-1157-4244-96c9-72496814e558-istio-envoy\") pod \"e2e-derive-ccc0a86e-openshift-default-79d76bd567-qgd42\" (UID: \"46c5f745-1157-4244-96c9-72496814e558\") " pod="openshift-ingress/e2e-derive-ccc0a86e-openshift-default-79d76bd567-qgd42" Jul 16 01:06:36.950440 ip-10-0-143-200 kubenswrapper[2572]: I0716 01:06:36.950337 2572 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/46c5f745-1157-4244-96c9-72496814e558-credential-socket\") pod \"e2e-derive-ccc0a86e-openshift-default-79d76bd567-qgd42\" (UID: \"46c5f745-1157-4244-96c9-72496814e558\") " pod="openshift-ingress/e2e-derive-ccc0a86e-openshift-default-79d76bd567-qgd42" Jul 16 01:06:36.950440 ip-10-0-143-200 kubenswrapper[2572]: I0716 01:06:36.950404 2572 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-2bflw\" (UniqueName: \"kubernetes.io/projected/46c5f745-1157-4244-96c9-72496814e558-kube-api-access-2bflw\") pod \"e2e-derive-ccc0a86e-openshift-default-79d76bd567-qgd42\" (UID: \"46c5f745-1157-4244-96c9-72496814e558\") " pod="openshift-ingress/e2e-derive-ccc0a86e-openshift-default-79d76bd567-qgd42" Jul 16 01:06:36.950519 ip-10-0-143-200 kubenswrapper[2572]: I0716 01:06:36.950461 2572 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-hqp7f\" (UniqueName: \"kubernetes.io/projected/c2465b13-bf6b-4f0c-b215-a718fd9a34f1-kube-api-access-hqp7f\") on node \"ip-10-0-143-200.ec2.internal\" DevicePath \"\"" Jul 16 01:06:36.950519 ip-10-0-143-200 kubenswrapper[2572]: I0716 01:06:36.950472 2572 reconciler_common.go:299] "Volume detached for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/c2465b13-bf6b-4f0c-b215-a718fd9a34f1-credential-socket\") on node \"ip-10-0-143-200.ec2.internal\" DevicePath \"\"" Jul 16 01:06:36.950519 ip-10-0-143-200 kubenswrapper[2572]: I0716 01:06:36.950481 2572 reconciler_common.go:299] "Volume detached for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/c2465b13-bf6b-4f0c-b215-a718fd9a34f1-workload-certs\") on node \"ip-10-0-143-200.ec2.internal\" DevicePath \"\"" Jul 16 01:06:36.950519 ip-10-0-143-200 kubenswrapper[2572]: I0716 01:06:36.950495 2572 reconciler_common.go:299] "Volume detached for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/c2465b13-bf6b-4f0c-b215-a718fd9a34f1-istio-data\") on node \"ip-10-0-143-200.ec2.internal\" DevicePath \"\"" Jul 16 01:06:36.950519 ip-10-0-143-200 kubenswrapper[2572]: I0716 01:06:36.950507 2572 reconciler_common.go:299] "Volume detached for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/c2465b13-bf6b-4f0c-b215-a718fd9a34f1-istiod-ca-cert\") on node \"ip-10-0-143-200.ec2.internal\" DevicePath \"\"" Jul 16 01:06:36.950519 ip-10-0-143-200 kubenswrapper[2572]: I0716 01:06:36.950515 2572 reconciler_common.go:299] "Volume detached for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/c2465b13-bf6b-4f0c-b215-a718fd9a34f1-istio-token\") on node \"ip-10-0-143-200.ec2.internal\" DevicePath \"\"" Jul 16 01:06:36.950733 ip-10-0-143-200 kubenswrapper[2572]: I0716 01:06:36.950523 2572 reconciler_common.go:299] "Volume detached for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/c2465b13-bf6b-4f0c-b215-a718fd9a34f1-istio-podinfo\") on node \"ip-10-0-143-200.ec2.internal\" DevicePath \"\"" Jul 16 01:06:36.950733 ip-10-0-143-200 kubenswrapper[2572]: I0716 01:06:36.950530 2572 reconciler_common.go:299] "Volume detached for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/c2465b13-bf6b-4f0c-b215-a718fd9a34f1-istio-envoy\") on node \"ip-10-0-143-200.ec2.internal\" DevicePath \"\"" Jul 16 01:06:37.050032 ip-10-0-143-200 kubenswrapper[2572]: I0716 01:06:37.049999 2572 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9gnmq_3a7e030d-fad8-480d-9dc6-fe5057048cf9/konnectivity-agent/0.log" limit=1024 Jul 16 01:06:37.051230 ip-10-0-143-200 kubenswrapper[2572]: I0716 01:06:37.051195 2572 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/46c5f745-1157-4244-96c9-72496814e558-workload-certs\") pod \"e2e-derive-ccc0a86e-openshift-default-79d76bd567-qgd42\" (UID: \"46c5f745-1157-4244-96c9-72496814e558\") " pod="openshift-ingress/e2e-derive-ccc0a86e-openshift-default-79d76bd567-qgd42" Jul 16 01:06:37.051369 ip-10-0-143-200 kubenswrapper[2572]: I0716 01:06:37.051245 2572 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/46c5f745-1157-4244-96c9-72496814e558-istio-token\") pod \"e2e-derive-ccc0a86e-openshift-default-79d76bd567-qgd42\" (UID: \"46c5f745-1157-4244-96c9-72496814e558\") " pod="openshift-ingress/e2e-derive-ccc0a86e-openshift-default-79d76bd567-qgd42" Jul 16 01:06:37.051369 ip-10-0-143-200 kubenswrapper[2572]: I0716 01:06:37.051264 2572 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/46c5f745-1157-4244-96c9-72496814e558-istio-envoy\") pod \"e2e-derive-ccc0a86e-openshift-default-79d76bd567-qgd42\" (UID: \"46c5f745-1157-4244-96c9-72496814e558\") " pod="openshift-ingress/e2e-derive-ccc0a86e-openshift-default-79d76bd567-qgd42" Jul 16 01:06:37.051484 ip-10-0-143-200 kubenswrapper[2572]: I0716 01:06:37.051398 2572 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/46c5f745-1157-4244-96c9-72496814e558-credential-socket\") pod \"e2e-derive-ccc0a86e-openshift-default-79d76bd567-qgd42\" (UID: \"46c5f745-1157-4244-96c9-72496814e558\") " pod="openshift-ingress/e2e-derive-ccc0a86e-openshift-default-79d76bd567-qgd42" Jul 16 01:06:37.051484 ip-10-0-143-200 kubenswrapper[2572]: I0716 01:06:37.051454 2572 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-2bflw\" (UniqueName: \"kubernetes.io/projected/46c5f745-1157-4244-96c9-72496814e558-kube-api-access-2bflw\") pod \"e2e-derive-ccc0a86e-openshift-default-79d76bd567-qgd42\" (UID: \"46c5f745-1157-4244-96c9-72496814e558\") " pod="openshift-ingress/e2e-derive-ccc0a86e-openshift-default-79d76bd567-qgd42" Jul 16 01:06:37.051582 ip-10-0-143-200 kubenswrapper[2572]: I0716 01:06:37.051505 2572 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/46c5f745-1157-4244-96c9-72496814e558-workload-socket\") pod \"e2e-derive-ccc0a86e-openshift-default-79d76bd567-qgd42\" (UID: \"46c5f745-1157-4244-96c9-72496814e558\") " pod="openshift-ingress/e2e-derive-ccc0a86e-openshift-default-79d76bd567-qgd42" Jul 16 01:06:37.051582 ip-10-0-143-200 kubenswrapper[2572]: I0716 01:06:37.051521 2572 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/46c5f745-1157-4244-96c9-72496814e558-istio-podinfo\") pod \"e2e-derive-ccc0a86e-openshift-default-79d76bd567-qgd42\" (UID: \"46c5f745-1157-4244-96c9-72496814e558\") " pod="openshift-ingress/e2e-derive-ccc0a86e-openshift-default-79d76bd567-qgd42" Jul 16 01:06:37.051707 ip-10-0-143-200 kubenswrapper[2572]: I0716 01:06:37.051592 2572 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/46c5f745-1157-4244-96c9-72496814e558-istio-data\") pod \"e2e-derive-ccc0a86e-openshift-default-79d76bd567-qgd42\" (UID: \"46c5f745-1157-4244-96c9-72496814e558\") " pod="openshift-ingress/e2e-derive-ccc0a86e-openshift-default-79d76bd567-qgd42" Jul 16 01:06:37.051707 ip-10-0-143-200 kubenswrapper[2572]: I0716 01:06:37.051649 2572 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/46c5f745-1157-4244-96c9-72496814e558-workload-certs\") pod \"e2e-derive-ccc0a86e-openshift-default-79d76bd567-qgd42\" (UID: \"46c5f745-1157-4244-96c9-72496814e558\") " pod="openshift-ingress/e2e-derive-ccc0a86e-openshift-default-79d76bd567-qgd42" Jul 16 01:06:37.051707 ip-10-0-143-200 kubenswrapper[2572]: I0716 01:06:37.051664 2572 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/46c5f745-1157-4244-96c9-72496814e558-istiod-ca-cert\") pod \"e2e-derive-ccc0a86e-openshift-default-79d76bd567-qgd42\" (UID: \"46c5f745-1157-4244-96c9-72496814e558\") " pod="openshift-ingress/e2e-derive-ccc0a86e-openshift-default-79d76bd567-qgd42" Jul 16 01:06:37.051861 ip-10-0-143-200 kubenswrapper[2572]: I0716 01:06:37.051816 2572 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/46c5f745-1157-4244-96c9-72496814e558-credential-socket\") pod \"e2e-derive-ccc0a86e-openshift-default-79d76bd567-qgd42\" (UID: \"46c5f745-1157-4244-96c9-72496814e558\") " pod="openshift-ingress/e2e-derive-ccc0a86e-openshift-default-79d76bd567-qgd42" Jul 16 01:06:37.052014 ip-10-0-143-200 kubenswrapper[2572]: I0716 01:06:37.051994 2572 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/46c5f745-1157-4244-96c9-72496814e558-istio-data\") pod \"e2e-derive-ccc0a86e-openshift-default-79d76bd567-qgd42\" (UID: \"46c5f745-1157-4244-96c9-72496814e558\") " pod="openshift-ingress/e2e-derive-ccc0a86e-openshift-default-79d76bd567-qgd42" Jul 16 01:06:37.052060 ip-10-0-143-200 kubenswrapper[2572]: I0716 01:06:37.052031 2572 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/46c5f745-1157-4244-96c9-72496814e558-workload-socket\") pod \"e2e-derive-ccc0a86e-openshift-default-79d76bd567-qgd42\" (UID: \"46c5f745-1157-4244-96c9-72496814e558\") " pod="openshift-ingress/e2e-derive-ccc0a86e-openshift-default-79d76bd567-qgd42" Jul 16 01:06:37.052417 ip-10-0-143-200 kubenswrapper[2572]: I0716 01:06:37.052397 2572 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/46c5f745-1157-4244-96c9-72496814e558-istiod-ca-cert\") pod \"e2e-derive-ccc0a86e-openshift-default-79d76bd567-qgd42\" (UID: \"46c5f745-1157-4244-96c9-72496814e558\") " pod="openshift-ingress/e2e-derive-ccc0a86e-openshift-default-79d76bd567-qgd42" Jul 16 01:06:37.053833 ip-10-0-143-200 kubenswrapper[2572]: I0716 01:06:37.053801 2572 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/46c5f745-1157-4244-96c9-72496814e558-istio-envoy\") pod \"e2e-derive-ccc0a86e-openshift-default-79d76bd567-qgd42\" (UID: \"46c5f745-1157-4244-96c9-72496814e558\") " pod="openshift-ingress/e2e-derive-ccc0a86e-openshift-default-79d76bd567-qgd42" Jul 16 01:06:37.054140 ip-10-0-143-200 kubenswrapper[2572]: I0716 01:06:37.054121 2572 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/46c5f745-1157-4244-96c9-72496814e558-istio-podinfo\") pod \"e2e-derive-ccc0a86e-openshift-default-79d76bd567-qgd42\" (UID: \"46c5f745-1157-4244-96c9-72496814e558\") " pod="openshift-ingress/e2e-derive-ccc0a86e-openshift-default-79d76bd567-qgd42" Jul 16 01:06:37.068842 ip-10-0-143-200 kubenswrapper[2572]: I0716 01:06:37.068810 2572 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/46c5f745-1157-4244-96c9-72496814e558-istio-token\") pod \"e2e-derive-ccc0a86e-openshift-default-79d76bd567-qgd42\" (UID: \"46c5f745-1157-4244-96c9-72496814e558\") " pod="openshift-ingress/e2e-derive-ccc0a86e-openshift-default-79d76bd567-qgd42" Jul 16 01:06:37.068965 ip-10-0-143-200 kubenswrapper[2572]: I0716 01:06:37.068905 2572 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-2bflw\" (UniqueName: \"kubernetes.io/projected/46c5f745-1157-4244-96c9-72496814e558-kube-api-access-2bflw\") pod \"e2e-derive-ccc0a86e-openshift-default-79d76bd567-qgd42\" (UID: \"46c5f745-1157-4244-96c9-72496814e558\") " pod="openshift-ingress/e2e-derive-ccc0a86e-openshift-default-79d76bd567-qgd42" Jul 16 01:06:37.112058 ip-10-0-143-200 kubenswrapper[2572]: I0716 01:06:37.111974 2572 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-derive-ccc0a86e-bc67c5bbc-lmsg2"] Jul 16 01:06:37.115963 ip-10-0-143-200 kubenswrapper[2572]: I0716 01:06:37.115934 2572 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-e2e-derive-ccc0a86e-bc67c5bbc-lmsg2" Jul 16 01:06:37.119009 ip-10-0-143-200 kubenswrapper[2572]: I0716 01:06:37.118978 2572 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"odh-ai-gateway-infra\"/\"maas-api-e2e-derive-ccc0a86e-serving-cert\"" Jul 16 01:06:37.130420 ip-10-0-143-200 kubenswrapper[2572]: I0716 01:06:37.130394 2572 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-derive-ccc0a86e-bc67c5bbc-lmsg2"] Jul 16 01:06:37.152365 ip-10-0-143-200 kubenswrapper[2572]: I0716 01:06:37.152328 2572 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/d17a27cf-6819-47e6-a8ca-f0c24a740469-openshift-service-ca\") pod \"maas-api-e2e-derive-ccc0a86e-bc67c5bbc-lmsg2\" (UID: \"d17a27cf-6819-47e6-a8ca-f0c24a740469\") " pod="odh-ai-gateway-infra/maas-api-e2e-derive-ccc0a86e-bc67c5bbc-lmsg2" Jul 16 01:06:37.152545 ip-10-0-143-200 kubenswrapper[2572]: I0716 01:06:37.152400 2572 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/d17a27cf-6819-47e6-a8ca-f0c24a740469-maas-api-tls\") pod \"maas-api-e2e-derive-ccc0a86e-bc67c5bbc-lmsg2\" (UID: \"d17a27cf-6819-47e6-a8ca-f0c24a740469\") " pod="odh-ai-gateway-infra/maas-api-e2e-derive-ccc0a86e-bc67c5bbc-lmsg2" Jul 16 01:06:37.152545 ip-10-0-143-200 kubenswrapper[2572]: I0716 01:06:37.152500 2572 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-rpgwg\" (UniqueName: \"kubernetes.io/projected/d17a27cf-6819-47e6-a8ca-f0c24a740469-kube-api-access-rpgwg\") pod \"maas-api-e2e-derive-ccc0a86e-bc67c5bbc-lmsg2\" (UID: \"d17a27cf-6819-47e6-a8ca-f0c24a740469\") " pod="odh-ai-gateway-infra/maas-api-e2e-derive-ccc0a86e-bc67c5bbc-lmsg2" Jul 16 01:06:37.182437 ip-10-0-143-200 kubenswrapper[2572]: I0716 01:06:37.182401 2572 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9gnmq_3a7e030d-fad8-480d-9dc6-fe5057048cf9/konnectivity-agent/0.log" limit=1024 Jul 16 01:06:37.197372 ip-10-0-143-200 kubenswrapper[2572]: I0716 01:06:37.197339 2572 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-derive-ccc0a86e-openshift-default-79d76bd567-qgd42" Jul 16 01:06:37.253801 ip-10-0-143-200 kubenswrapper[2572]: I0716 01:06:37.253480 2572 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/d17a27cf-6819-47e6-a8ca-f0c24a740469-openshift-service-ca\") pod \"maas-api-e2e-derive-ccc0a86e-bc67c5bbc-lmsg2\" (UID: \"d17a27cf-6819-47e6-a8ca-f0c24a740469\") " pod="odh-ai-gateway-infra/maas-api-e2e-derive-ccc0a86e-bc67c5bbc-lmsg2" Jul 16 01:06:37.253801 ip-10-0-143-200 kubenswrapper[2572]: I0716 01:06:37.253570 2572 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/d17a27cf-6819-47e6-a8ca-f0c24a740469-maas-api-tls\") pod \"maas-api-e2e-derive-ccc0a86e-bc67c5bbc-lmsg2\" (UID: \"d17a27cf-6819-47e6-a8ca-f0c24a740469\") " pod="odh-ai-gateway-infra/maas-api-e2e-derive-ccc0a86e-bc67c5bbc-lmsg2" Jul 16 01:06:37.253801 ip-10-0-143-200 kubenswrapper[2572]: I0716 01:06:37.253647 2572 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-rpgwg\" (UniqueName: \"kubernetes.io/projected/d17a27cf-6819-47e6-a8ca-f0c24a740469-kube-api-access-rpgwg\") pod \"maas-api-e2e-derive-ccc0a86e-bc67c5bbc-lmsg2\" (UID: \"d17a27cf-6819-47e6-a8ca-f0c24a740469\") " pod="odh-ai-gateway-infra/maas-api-e2e-derive-ccc0a86e-bc67c5bbc-lmsg2" Jul 16 01:06:37.256793 ip-10-0-143-200 kubenswrapper[2572]: I0716 01:06:37.254955 2572 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/d17a27cf-6819-47e6-a8ca-f0c24a740469-openshift-service-ca\") pod \"maas-api-e2e-derive-ccc0a86e-bc67c5bbc-lmsg2\" (UID: \"d17a27cf-6819-47e6-a8ca-f0c24a740469\") " pod="odh-ai-gateway-infra/maas-api-e2e-derive-ccc0a86e-bc67c5bbc-lmsg2" Jul 16 01:06:37.256793 ip-10-0-143-200 kubenswrapper[2572]: E0716 01:06:37.255101 2572 secret.go:189] Couldn't get secret odh-ai-gateway-infra/maas-api-e2e-derive-ccc0a86e-serving-cert: secret "maas-api-e2e-derive-ccc0a86e-serving-cert" not found Jul 16 01:06:37.256793 ip-10-0-143-200 kubenswrapper[2572]: E0716 01:06:37.255202 2572 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/d17a27cf-6819-47e6-a8ca-f0c24a740469-maas-api-tls podName:d17a27cf-6819-47e6-a8ca-f0c24a740469 nodeName:}" failed. No retries permitted until 2026-07-16 01:06:37.75518202 +0000 UTC m=+1602.863936691 (durationBeforeRetry 500ms). Error: MountVolume.SetUp failed for volume "maas-api-tls" (UniqueName: "kubernetes.io/secret/d17a27cf-6819-47e6-a8ca-f0c24a740469-maas-api-tls") pod "maas-api-e2e-derive-ccc0a86e-bc67c5bbc-lmsg2" (UID: "d17a27cf-6819-47e6-a8ca-f0c24a740469") : secret "maas-api-e2e-derive-ccc0a86e-serving-cert" not found Jul 16 01:06:37.264967 ip-10-0-143-200 kubenswrapper[2572]: I0716 01:06:37.264934 2572 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-rpgwg\" (UniqueName: \"kubernetes.io/projected/d17a27cf-6819-47e6-a8ca-f0c24a740469-kube-api-access-rpgwg\") pod \"maas-api-e2e-derive-ccc0a86e-bc67c5bbc-lmsg2\" (UID: \"d17a27cf-6819-47e6-a8ca-f0c24a740469\") " pod="odh-ai-gateway-infra/maas-api-e2e-derive-ccc0a86e-bc67c5bbc-lmsg2" Jul 16 01:06:37.278759 ip-10-0-143-200 kubenswrapper[2572]: I0716 01:06:37.278726 2572 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9gnmq_3a7e030d-fad8-480d-9dc6-fe5057048cf9/konnectivity-agent/0.log" limit=1024 Jul 16 01:06:37.363761 ip-10-0-143-200 kubenswrapper[2572]: I0716 01:06:37.363735 2572 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/e2e-derive-ccc0a86e-openshift-default-79d76bd567-qgd42"] Jul 16 01:06:37.365449 ip-10-0-143-200 kubenswrapper[2572]: W0716 01:06:37.365415 2572 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod46c5f745_1157_4244_96c9_72496814e558.slice/crio-b243c33db3f6a95aace781f579569aedba00d94dfb2dbb5a1df3a0e3c54ecec6 WatchSource:0}: Error finding container b243c33db3f6a95aace781f579569aedba00d94dfb2dbb5a1df3a0e3c54ecec6: Status 404 returned error can't find the container with id b243c33db3f6a95aace781f579569aedba00d94dfb2dbb5a1df3a0e3c54ecec6 Jul 16 01:06:37.367577 ip-10-0-143-200 kubenswrapper[2572]: I0716 01:06:37.367549 2572 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236228Ki","pods":"250"} Jul 16 01:06:37.367669 ip-10-0-143-200 kubenswrapper[2572]: I0716 01:06:37.367613 2572 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236228Ki","pods":"250"} Jul 16 01:06:37.367669 ip-10-0-143-200 kubenswrapper[2572]: I0716 01:06:37.367661 2572 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236228Ki","pods":"250"} Jul 16 01:06:37.571678 ip-10-0-143-200 kubenswrapper[2572]: I0716 01:06:37.571621 2572 generic.go:356] "Generic (PLEG): container finished" podID="c2465b13-bf6b-4f0c-b215-a718fd9a34f1" containerID="136fe22e48e0f1170c94928c1d4aa885d76e38487c9e8174922fb3dada02f639" exitCode=0 Jul 16 01:06:37.571872 ip-10-0-143-200 kubenswrapper[2572]: I0716 01:06:37.571670 2572 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-migrate-cefec0b9-gw-openshift-default-85fc975755-sqmz7" event={"ID":"c2465b13-bf6b-4f0c-b215-a718fd9a34f1","Type":"ContainerDied","Data":"136fe22e48e0f1170c94928c1d4aa885d76e38487c9e8174922fb3dada02f639"} Jul 16 01:06:37.571872 ip-10-0-143-200 kubenswrapper[2572]: I0716 01:06:37.571698 2572 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-migrate-cefec0b9-gw-openshift-default-85fc975755-sqmz7" Jul 16 01:06:37.571872 ip-10-0-143-200 kubenswrapper[2572]: I0716 01:06:37.571747 2572 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-migrate-cefec0b9-gw-openshift-default-85fc975755-sqmz7" event={"ID":"c2465b13-bf6b-4f0c-b215-a718fd9a34f1","Type":"ContainerDied","Data":"ebbbaa5a52da2ebed4ae24a33c6d86c5c71401b8c3aa1c612a09898315bb80d4"} Jul 16 01:06:37.571872 ip-10-0-143-200 kubenswrapper[2572]: I0716 01:06:37.571782 2572 scope.go:117] "RemoveContainer" containerID="136fe22e48e0f1170c94928c1d4aa885d76e38487c9e8174922fb3dada02f639" Jul 16 01:06:37.574157 ip-10-0-143-200 kubenswrapper[2572]: I0716 01:06:37.574129 2572 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-derive-ccc0a86e-openshift-default-79d76bd567-qgd42" event={"ID":"46c5f745-1157-4244-96c9-72496814e558","Type":"ContainerStarted","Data":"e44a298a87148055fa32bb285b8bb6f3d936c78df4bfc9ec4a667f1392a76b13"} Jul 16 01:06:37.574291 ip-10-0-143-200 kubenswrapper[2572]: I0716 01:06:37.574182 2572 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-derive-ccc0a86e-openshift-default-79d76bd567-qgd42" event={"ID":"46c5f745-1157-4244-96c9-72496814e558","Type":"ContainerStarted","Data":"b243c33db3f6a95aace781f579569aedba00d94dfb2dbb5a1df3a0e3c54ecec6"} Jul 16 01:06:37.594826 ip-10-0-143-200 kubenswrapper[2572]: I0716 01:06:37.594806 2572 scope.go:117] "RemoveContainer" containerID="136fe22e48e0f1170c94928c1d4aa885d76e38487c9e8174922fb3dada02f639" Jul 16 01:06:37.595191 ip-10-0-143-200 kubenswrapper[2572]: E0716 01:06:37.595170 2572 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"136fe22e48e0f1170c94928c1d4aa885d76e38487c9e8174922fb3dada02f639\": container with ID starting with 136fe22e48e0f1170c94928c1d4aa885d76e38487c9e8174922fb3dada02f639 not found: ID does not exist" containerID="136fe22e48e0f1170c94928c1d4aa885d76e38487c9e8174922fb3dada02f639" Jul 16 01:06:37.595249 ip-10-0-143-200 kubenswrapper[2572]: I0716 01:06:37.595200 2572 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"136fe22e48e0f1170c94928c1d4aa885d76e38487c9e8174922fb3dada02f639"} err="failed to get container status \"136fe22e48e0f1170c94928c1d4aa885d76e38487c9e8174922fb3dada02f639\": rpc error: code = NotFound desc = could not find container \"136fe22e48e0f1170c94928c1d4aa885d76e38487c9e8174922fb3dada02f639\": container with ID starting with 136fe22e48e0f1170c94928c1d4aa885d76e38487c9e8174922fb3dada02f639 not found: ID does not exist" Jul 16 01:06:37.604824 ip-10-0-143-200 kubenswrapper[2572]: I0716 01:06:37.604766 2572 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-ingress/e2e-derive-ccc0a86e-openshift-default-79d76bd567-qgd42" podStartSLOduration=1.604752812 podStartE2EDuration="1.604752812s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-16 01:06:36 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-16 01:06:37.603362313 +0000 UTC m=+1602.712116991" watchObservedRunningTime="2026-07-16 01:06:37.604752812 +0000 UTC m=+1602.713507557" Jul 16 01:06:37.611583 ip-10-0-143-200 kubenswrapper[2572]: I0716 01:06:37.611562 2572 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9gnmq_3a7e030d-fad8-480d-9dc6-fe5057048cf9/konnectivity-agent/0.log" limit=1024 Jul 16 01:06:37.628260 ip-10-0-143-200 kubenswrapper[2572]: I0716 01:06:37.628177 2572 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-ingress/e2e-migrate-cefec0b9-gw-openshift-default-85fc975755-sqmz7"] Jul 16 01:06:37.647493 ip-10-0-143-200 kubenswrapper[2572]: I0716 01:06:37.647464 2572 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["openshift-ingress/e2e-migrate-cefec0b9-gw-openshift-default-85fc975755-sqmz7"] Jul 16 01:06:37.759414 ip-10-0-143-200 kubenswrapper[2572]: I0716 01:06:37.759366 2572 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/d17a27cf-6819-47e6-a8ca-f0c24a740469-maas-api-tls\") pod \"maas-api-e2e-derive-ccc0a86e-bc67c5bbc-lmsg2\" (UID: \"d17a27cf-6819-47e6-a8ca-f0c24a740469\") " pod="odh-ai-gateway-infra/maas-api-e2e-derive-ccc0a86e-bc67c5bbc-lmsg2" Jul 16 01:06:37.762004 ip-10-0-143-200 kubenswrapper[2572]: I0716 01:06:37.761982 2572 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/d17a27cf-6819-47e6-a8ca-f0c24a740469-maas-api-tls\") pod \"maas-api-e2e-derive-ccc0a86e-bc67c5bbc-lmsg2\" (UID: \"d17a27cf-6819-47e6-a8ca-f0c24a740469\") " pod="odh-ai-gateway-infra/maas-api-e2e-derive-ccc0a86e-bc67c5bbc-lmsg2" Jul 16 01:06:37.949338 ip-10-0-143-200 kubenswrapper[2572]: I0716 01:06:37.949199 2572 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9gnmq_3a7e030d-fad8-480d-9dc6-fe5057048cf9/konnectivity-agent/0.log" limit=1024 Jul 16 01:06:38.028659 ip-10-0-143-200 kubenswrapper[2572]: I0716 01:06:38.028601 2572 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-e2e-derive-ccc0a86e-bc67c5bbc-lmsg2" Jul 16 01:06:38.054309 ip-10-0-143-200 kubenswrapper[2572]: I0716 01:06:38.054274 2572 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9gnmq_3a7e030d-fad8-480d-9dc6-fe5057048cf9/konnectivity-agent/0.log" limit=1024 Jul 16 01:06:38.191029 ip-10-0-143-200 kubenswrapper[2572]: I0716 01:06:38.191004 2572 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-derive-ccc0a86e-bc67c5bbc-lmsg2"] Jul 16 01:06:38.193575 ip-10-0-143-200 kubenswrapper[2572]: W0716 01:06:38.193546 2572 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podd17a27cf_6819_47e6_a8ca_f0c24a740469.slice/crio-d81efc4e6674f3fb84c48088f635bee33fbdee935c694dc29d8af7fc6e287c1c WatchSource:0}: Error finding container d81efc4e6674f3fb84c48088f635bee33fbdee935c694dc29d8af7fc6e287c1c: Status 404 returned error can't find the container with id d81efc4e6674f3fb84c48088f635bee33fbdee935c694dc29d8af7fc6e287c1c Jul 16 01:06:38.197864 ip-10-0-143-200 kubenswrapper[2572]: I0716 01:06:38.197836 2572 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="openshift-ingress/e2e-derive-ccc0a86e-openshift-default-79d76bd567-qgd42" Jul 16 01:06:38.199575 ip-10-0-143-200 kubenswrapper[2572]: I0716 01:06:38.199518 2572 patch_prober.go:28] interesting pod/e2e-derive-ccc0a86e-openshift-default-79d76bd567-qgd42 container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.132.0.54:15021/healthz/ready\": dial tcp 10.132.0.54:15021: connect: connection refused" start-of-body= Jul 16 01:06:38.199688 ip-10-0-143-200 kubenswrapper[2572]: I0716 01:06:38.199572 2572 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-derive-ccc0a86e-openshift-default-79d76bd567-qgd42" podUID="46c5f745-1157-4244-96c9-72496814e558" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.132.0.54:15021/healthz/ready\": dial tcp 10.132.0.54:15021: connect: connection refused" Jul 16 01:06:38.581148 ip-10-0-143-200 kubenswrapper[2572]: I0716 01:06:38.581071 2572 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-e2e-derive-ccc0a86e-bc67c5bbc-lmsg2" event={"ID":"d17a27cf-6819-47e6-a8ca-f0c24a740469","Type":"ContainerStarted","Data":"d81efc4e6674f3fb84c48088f635bee33fbdee935c694dc29d8af7fc6e287c1c"} Jul 16 01:06:38.583198 ip-10-0-143-200 kubenswrapper[2572]: I0716 01:06:38.583167 2572 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9gnmq_3a7e030d-fad8-480d-9dc6-fe5057048cf9/konnectivity-agent/0.log" limit=1024 Jul 16 01:06:39.201424 ip-10-0-143-200 kubenswrapper[2572]: I0716 01:06:39.201388 2572 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="openshift-ingress/e2e-derive-ccc0a86e-openshift-default-79d76bd567-qgd42" Jul 16 01:06:39.449013 ip-10-0-143-200 kubenswrapper[2572]: I0716 01:06:39.448970 2572 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="c2465b13-bf6b-4f0c-b215-a718fd9a34f1" path="/var/lib/kubelet/pods/c2465b13-bf6b-4f0c-b215-a718fd9a34f1/volumes" Jul 16 01:06:39.590910 ip-10-0-143-200 kubenswrapper[2572]: I0716 01:06:39.590813 2572 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-e2e-derive-ccc0a86e-bc67c5bbc-lmsg2" event={"ID":"d17a27cf-6819-47e6-a8ca-f0c24a740469","Type":"ContainerStarted","Data":"62a92b295d7816ad40fab8f2ae90cfa1a3363c606d05a222826337bbfd67f40f"} Jul 16 01:06:39.591058 ip-10-0-143-200 kubenswrapper[2572]: I0716 01:06:39.591027 2572 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/e2e-derive-ccc0a86e-openshift-default-79d76bd567-qgd42" Jul 16 01:06:39.591956 ip-10-0-143-200 kubenswrapper[2572]: I0716 01:06:39.591935 2572 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/e2e-derive-ccc0a86e-openshift-default-79d76bd567-qgd42" Jul 16 01:06:39.614074 ip-10-0-143-200 kubenswrapper[2572]: I0716 01:06:39.614021 2572 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="odh-ai-gateway-infra/maas-api-e2e-derive-ccc0a86e-bc67c5bbc-lmsg2" podStartSLOduration=2.224336806 podStartE2EDuration="2.61400896s" totalImagesPullingTime="389.672154ms" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-16 01:06:37 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-16 01:06:39.612351401 +0000 UTC m=+1604.721106098" watchObservedRunningTime="2026-07-16 01:06:39.61400896 +0000 UTC m=+1604.722763635" Jul 16 01:06:40.528008 ip-10-0-143-200 kubenswrapper[2572]: I0716 01:06:40.527971 2572 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9gnmq_3a7e030d-fad8-480d-9dc6-fe5057048cf9/konnectivity-agent/0.log" limit=1024 Jul 16 01:06:40.596697 ip-10-0-143-200 kubenswrapper[2572]: I0716 01:06:40.596658 2572 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="odh-ai-gateway-infra/maas-api-e2e-derive-ccc0a86e-bc67c5bbc-lmsg2" Jul 16 01:06:42.571002 ip-10-0-143-200 kubenswrapper[2572]: I0716 01:06:42.570950 2572 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-ingress/e2e-derive-ccc0a86e-openshift-default-79d76bd567-qgd42"] Jul 16 01:06:42.594949 ip-10-0-143-200 kubenswrapper[2572]: I0716 01:06:42.594880 2572 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9gnmq_3a7e030d-fad8-480d-9dc6-fe5057048cf9/konnectivity-agent/0.log" limit=1024 Jul 16 01:06:42.611339 ip-10-0-143-200 kubenswrapper[2572]: I0716 01:06:42.611292 2572 kuberuntime_container.go:873] "Killing container with a grace period" pod="openshift-ingress/e2e-derive-ccc0a86e-openshift-default-79d76bd567-qgd42" podUID="46c5f745-1157-4244-96c9-72496814e558" containerName="istio-proxy" containerID="cri-o://e44a298a87148055fa32bb285b8bb6f3d936c78df4bfc9ec4a667f1392a76b13" gracePeriod=30 Jul 16 01:06:42.694021 ip-10-0-143-200 kubenswrapper[2572]: I0716 01:06:42.693985 2572 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9gnmq_3a7e030d-fad8-480d-9dc6-fe5057048cf9/konnectivity-agent/0.log" limit=1024 Jul 16 01:06:44.278008 ip-10-0-143-200 kubenswrapper[2572]: I0716 01:06:44.277976 2572 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9gnmq_3a7e030d-fad8-480d-9dc6-fe5057048cf9/konnectivity-agent/0.log" limit=1024 Jul 16 01:06:46.609749 ip-10-0-143-200 kubenswrapper[2572]: I0716 01:06:46.609706 2572 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="odh-ai-gateway-infra/maas-api-e2e-derive-ccc0a86e-bc67c5bbc-lmsg2" Jul 16 01:06:49.879245 ip-10-0-143-200 kubenswrapper[2572]: I0716 01:06:49.879176 2572 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-derive-ccc0a86e-openshift-default-79d76bd567-qgd42" Jul 16 01:06:49.980178 ip-10-0-143-200 kubenswrapper[2572]: I0716 01:06:49.980141 2572 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/46c5f745-1157-4244-96c9-72496814e558-workload-certs\") pod \"46c5f745-1157-4244-96c9-72496814e558\" (UID: \"46c5f745-1157-4244-96c9-72496814e558\") " Jul 16 01:06:49.980349 ip-10-0-143-200 kubenswrapper[2572]: I0716 01:06:49.980190 2572 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/46c5f745-1157-4244-96c9-72496814e558-istio-token\") pod \"46c5f745-1157-4244-96c9-72496814e558\" (UID: \"46c5f745-1157-4244-96c9-72496814e558\") " Jul 16 01:06:49.980349 ip-10-0-143-200 kubenswrapper[2572]: I0716 01:06:49.980246 2572 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/46c5f745-1157-4244-96c9-72496814e558-istio-envoy\") pod \"46c5f745-1157-4244-96c9-72496814e558\" (UID: \"46c5f745-1157-4244-96c9-72496814e558\") " Jul 16 01:06:49.980349 ip-10-0-143-200 kubenswrapper[2572]: I0716 01:06:49.980276 2572 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/46c5f745-1157-4244-96c9-72496814e558-istiod-ca-cert\") pod \"46c5f745-1157-4244-96c9-72496814e558\" (UID: \"46c5f745-1157-4244-96c9-72496814e558\") " Jul 16 01:06:49.980349 ip-10-0-143-200 kubenswrapper[2572]: I0716 01:06:49.980307 2572 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-2bflw\" (UniqueName: \"kubernetes.io/projected/46c5f745-1157-4244-96c9-72496814e558-kube-api-access-2bflw\") pod \"46c5f745-1157-4244-96c9-72496814e558\" (UID: \"46c5f745-1157-4244-96c9-72496814e558\") " Jul 16 01:06:49.980543 ip-10-0-143-200 kubenswrapper[2572]: I0716 01:06:49.980350 2572 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/46c5f745-1157-4244-96c9-72496814e558-istio-podinfo\") pod \"46c5f745-1157-4244-96c9-72496814e558\" (UID: \"46c5f745-1157-4244-96c9-72496814e558\") " Jul 16 01:06:49.980543 ip-10-0-143-200 kubenswrapper[2572]: I0716 01:06:49.980397 2572 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/46c5f745-1157-4244-96c9-72496814e558-credential-socket\") pod \"46c5f745-1157-4244-96c9-72496814e558\" (UID: \"46c5f745-1157-4244-96c9-72496814e558\") " Jul 16 01:06:49.980543 ip-10-0-143-200 kubenswrapper[2572]: I0716 01:06:49.980429 2572 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/46c5f745-1157-4244-96c9-72496814e558-istio-data\") pod \"46c5f745-1157-4244-96c9-72496814e558\" (UID: \"46c5f745-1157-4244-96c9-72496814e558\") " Jul 16 01:06:49.980543 ip-10-0-143-200 kubenswrapper[2572]: I0716 01:06:49.980446 2572 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/46c5f745-1157-4244-96c9-72496814e558-workload-certs" (OuterVolumeSpecName: "workload-certs") pod "46c5f745-1157-4244-96c9-72496814e558" (UID: "46c5f745-1157-4244-96c9-72496814e558"). InnerVolumeSpecName "workload-certs". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Jul 16 01:06:49.980543 ip-10-0-143-200 kubenswrapper[2572]: I0716 01:06:49.980527 2572 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/46c5f745-1157-4244-96c9-72496814e558-workload-socket\") pod \"46c5f745-1157-4244-96c9-72496814e558\" (UID: \"46c5f745-1157-4244-96c9-72496814e558\") " Jul 16 01:06:49.980850 ip-10-0-143-200 kubenswrapper[2572]: I0716 01:06:49.980673 2572 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/46c5f745-1157-4244-96c9-72496814e558-credential-socket" (OuterVolumeSpecName: "credential-socket") pod "46c5f745-1157-4244-96c9-72496814e558" (UID: "46c5f745-1157-4244-96c9-72496814e558"). InnerVolumeSpecName "credential-socket". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Jul 16 01:06:49.980850 ip-10-0-143-200 kubenswrapper[2572]: I0716 01:06:49.980715 2572 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/46c5f745-1157-4244-96c9-72496814e558-istiod-ca-cert" (OuterVolumeSpecName: "istiod-ca-cert") pod "46c5f745-1157-4244-96c9-72496814e558" (UID: "46c5f745-1157-4244-96c9-72496814e558"). InnerVolumeSpecName "istiod-ca-cert". PluginName "kubernetes.io/configmap", VolumeGIDValue "" Jul 16 01:06:49.980959 ip-10-0-143-200 kubenswrapper[2572]: I0716 01:06:49.980862 2572 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/46c5f745-1157-4244-96c9-72496814e558-workload-socket" (OuterVolumeSpecName: "workload-socket") pod "46c5f745-1157-4244-96c9-72496814e558" (UID: "46c5f745-1157-4244-96c9-72496814e558"). InnerVolumeSpecName "workload-socket". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Jul 16 01:06:49.981099 ip-10-0-143-200 kubenswrapper[2572]: I0716 01:06:49.981037 2572 reconciler_common.go:299] "Volume detached for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/46c5f745-1157-4244-96c9-72496814e558-credential-socket\") on node \"ip-10-0-143-200.ec2.internal\" DevicePath \"\"" Jul 16 01:06:49.981099 ip-10-0-143-200 kubenswrapper[2572]: I0716 01:06:49.981053 2572 reconciler_common.go:299] "Volume detached for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/46c5f745-1157-4244-96c9-72496814e558-workload-socket\") on node \"ip-10-0-143-200.ec2.internal\" DevicePath \"\"" Jul 16 01:06:49.981099 ip-10-0-143-200 kubenswrapper[2572]: I0716 01:06:49.981068 2572 reconciler_common.go:299] "Volume detached for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/46c5f745-1157-4244-96c9-72496814e558-workload-certs\") on node \"ip-10-0-143-200.ec2.internal\" DevicePath \"\"" Jul 16 01:06:49.981099 ip-10-0-143-200 kubenswrapper[2572]: I0716 01:06:49.981081 2572 reconciler_common.go:299] "Volume detached for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/46c5f745-1157-4244-96c9-72496814e558-istiod-ca-cert\") on node \"ip-10-0-143-200.ec2.internal\" DevicePath \"\"" Jul 16 01:06:49.981370 ip-10-0-143-200 kubenswrapper[2572]: I0716 01:06:49.981347 2572 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/46c5f745-1157-4244-96c9-72496814e558-istio-data" (OuterVolumeSpecName: "istio-data") pod "46c5f745-1157-4244-96c9-72496814e558" (UID: "46c5f745-1157-4244-96c9-72496814e558"). InnerVolumeSpecName "istio-data". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Jul 16 01:06:49.983029 ip-10-0-143-200 kubenswrapper[2572]: I0716 01:06:49.982970 2572 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/downward-api/46c5f745-1157-4244-96c9-72496814e558-istio-podinfo" (OuterVolumeSpecName: "istio-podinfo") pod "46c5f745-1157-4244-96c9-72496814e558" (UID: "46c5f745-1157-4244-96c9-72496814e558"). InnerVolumeSpecName "istio-podinfo". PluginName "kubernetes.io/downward-api", VolumeGIDValue "" Jul 16 01:06:49.983157 ip-10-0-143-200 kubenswrapper[2572]: I0716 01:06:49.983083 2572 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/46c5f745-1157-4244-96c9-72496814e558-istio-token" (OuterVolumeSpecName: "istio-token") pod "46c5f745-1157-4244-96c9-72496814e558" (UID: "46c5f745-1157-4244-96c9-72496814e558"). InnerVolumeSpecName "istio-token". PluginName "kubernetes.io/projected", VolumeGIDValue "" Jul 16 01:06:49.983157 ip-10-0-143-200 kubenswrapper[2572]: I0716 01:06:49.983095 2572 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/46c5f745-1157-4244-96c9-72496814e558-istio-envoy" (OuterVolumeSpecName: "istio-envoy") pod "46c5f745-1157-4244-96c9-72496814e558" (UID: "46c5f745-1157-4244-96c9-72496814e558"). InnerVolumeSpecName "istio-envoy". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Jul 16 01:06:49.983157 ip-10-0-143-200 kubenswrapper[2572]: I0716 01:06:49.983095 2572 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/46c5f745-1157-4244-96c9-72496814e558-kube-api-access-2bflw" (OuterVolumeSpecName: "kube-api-access-2bflw") pod "46c5f745-1157-4244-96c9-72496814e558" (UID: "46c5f745-1157-4244-96c9-72496814e558"). InnerVolumeSpecName "kube-api-access-2bflw". PluginName "kubernetes.io/projected", VolumeGIDValue "" Jul 16 01:06:50.081823 ip-10-0-143-200 kubenswrapper[2572]: I0716 01:06:50.081781 2572 reconciler_common.go:299] "Volume detached for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/46c5f745-1157-4244-96c9-72496814e558-istio-token\") on node \"ip-10-0-143-200.ec2.internal\" DevicePath \"\"" Jul 16 01:06:50.081823 ip-10-0-143-200 kubenswrapper[2572]: I0716 01:06:50.081816 2572 reconciler_common.go:299] "Volume detached for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/46c5f745-1157-4244-96c9-72496814e558-istio-envoy\") on node \"ip-10-0-143-200.ec2.internal\" DevicePath \"\"" Jul 16 01:06:50.081823 ip-10-0-143-200 kubenswrapper[2572]: I0716 01:06:50.081826 2572 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-2bflw\" (UniqueName: \"kubernetes.io/projected/46c5f745-1157-4244-96c9-72496814e558-kube-api-access-2bflw\") on node \"ip-10-0-143-200.ec2.internal\" DevicePath \"\"" Jul 16 01:06:50.082077 ip-10-0-143-200 kubenswrapper[2572]: I0716 01:06:50.081837 2572 reconciler_common.go:299] "Volume detached for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/46c5f745-1157-4244-96c9-72496814e558-istio-podinfo\") on node \"ip-10-0-143-200.ec2.internal\" DevicePath \"\"" Jul 16 01:06:50.082077 ip-10-0-143-200 kubenswrapper[2572]: I0716 01:06:50.081845 2572 reconciler_common.go:299] "Volume detached for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/46c5f745-1157-4244-96c9-72496814e558-istio-data\") on node \"ip-10-0-143-200.ec2.internal\" DevicePath \"\"" Jul 16 01:06:50.323919 ip-10-0-143-200 kubenswrapper[2572]: I0716 01:06:50.323885 2572 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9gnmq_3a7e030d-fad8-480d-9dc6-fe5057048cf9/konnectivity-agent/0.log" limit=1024 Jul 16 01:06:50.458925 ip-10-0-143-200 kubenswrapper[2572]: I0716 01:06:50.458883 2572 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9gnmq_3a7e030d-fad8-480d-9dc6-fe5057048cf9/konnectivity-agent/0.log" limit=1024 Jul 16 01:06:50.657749 ip-10-0-143-200 kubenswrapper[2572]: I0716 01:06:50.657649 2572 generic.go:356] "Generic (PLEG): container finished" podID="46c5f745-1157-4244-96c9-72496814e558" containerID="e44a298a87148055fa32bb285b8bb6f3d936c78df4bfc9ec4a667f1392a76b13" exitCode=0 Jul 16 01:06:50.657749 ip-10-0-143-200 kubenswrapper[2572]: I0716 01:06:50.657684 2572 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-derive-ccc0a86e-openshift-default-79d76bd567-qgd42" Jul 16 01:06:50.657749 ip-10-0-143-200 kubenswrapper[2572]: I0716 01:06:50.657730 2572 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-derive-ccc0a86e-openshift-default-79d76bd567-qgd42" event={"ID":"46c5f745-1157-4244-96c9-72496814e558","Type":"ContainerDied","Data":"e44a298a87148055fa32bb285b8bb6f3d936c78df4bfc9ec4a667f1392a76b13"} Jul 16 01:06:50.658046 ip-10-0-143-200 kubenswrapper[2572]: I0716 01:06:50.657793 2572 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-derive-ccc0a86e-openshift-default-79d76bd567-qgd42" event={"ID":"46c5f745-1157-4244-96c9-72496814e558","Type":"ContainerDied","Data":"b243c33db3f6a95aace781f579569aedba00d94dfb2dbb5a1df3a0e3c54ecec6"} Jul 16 01:06:50.658046 ip-10-0-143-200 kubenswrapper[2572]: I0716 01:06:50.657824 2572 scope.go:117] "RemoveContainer" containerID="e44a298a87148055fa32bb285b8bb6f3d936c78df4bfc9ec4a667f1392a76b13" Jul 16 01:06:50.672192 ip-10-0-143-200 kubenswrapper[2572]: I0716 01:06:50.669496 2572 scope.go:117] "RemoveContainer" containerID="e44a298a87148055fa32bb285b8bb6f3d936c78df4bfc9ec4a667f1392a76b13" Jul 16 01:06:50.672192 ip-10-0-143-200 kubenswrapper[2572]: E0716 01:06:50.669850 2572 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"e44a298a87148055fa32bb285b8bb6f3d936c78df4bfc9ec4a667f1392a76b13\": container with ID starting with e44a298a87148055fa32bb285b8bb6f3d936c78df4bfc9ec4a667f1392a76b13 not found: ID does not exist" containerID="e44a298a87148055fa32bb285b8bb6f3d936c78df4bfc9ec4a667f1392a76b13" Jul 16 01:06:50.672192 ip-10-0-143-200 kubenswrapper[2572]: I0716 01:06:50.669888 2572 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"e44a298a87148055fa32bb285b8bb6f3d936c78df4bfc9ec4a667f1392a76b13"} err="failed to get container status \"e44a298a87148055fa32bb285b8bb6f3d936c78df4bfc9ec4a667f1392a76b13\": rpc error: code = NotFound desc = could not find container \"e44a298a87148055fa32bb285b8bb6f3d936c78df4bfc9ec4a667f1392a76b13\": container with ID starting with e44a298a87148055fa32bb285b8bb6f3d936c78df4bfc9ec4a667f1392a76b13 not found: ID does not exist" Jul 16 01:06:50.696482 ip-10-0-143-200 kubenswrapper[2572]: I0716 01:06:50.696451 2572 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-ingress/e2e-derive-ccc0a86e-openshift-default-79d76bd567-qgd42"] Jul 16 01:06:50.714860 ip-10-0-143-200 kubenswrapper[2572]: I0716 01:06:50.714833 2572 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["openshift-ingress/e2e-derive-ccc0a86e-openshift-default-79d76bd567-qgd42"] Jul 16 01:06:50.978750 ip-10-0-143-200 kubenswrapper[2572]: I0716 01:06:50.978712 2572 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9gnmq_3a7e030d-fad8-480d-9dc6-fe5057048cf9/konnectivity-agent/0.log" limit=1024 Jul 16 01:06:51.127186 ip-10-0-143-200 kubenswrapper[2572]: I0716 01:06:51.127152 2572 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9gnmq_3a7e030d-fad8-480d-9dc6-fe5057048cf9/konnectivity-agent/0.log" limit=1024 Jul 16 01:06:51.448852 ip-10-0-143-200 kubenswrapper[2572]: I0716 01:06:51.448761 2572 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="46c5f745-1157-4244-96c9-72496814e558" path="/var/lib/kubelet/pods/46c5f745-1157-4244-96c9-72496814e558/volumes" Jul 16 01:06:52.661901 ip-10-0-143-200 kubenswrapper[2572]: I0716 01:06:52.661870 2572 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9gnmq_3a7e030d-fad8-480d-9dc6-fe5057048cf9/konnectivity-agent/0.log" limit=1024 Jul 16 01:07:01.625702 ip-10-0-143-200 kubenswrapper[2572]: I0716 01:07:01.625585 2572 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9gnmq_3a7e030d-fad8-480d-9dc6-fe5057048cf9/konnectivity-agent/0.log" limit=1024 Jul 16 01:07:01.840838 ip-10-0-143-200 kubenswrapper[2572]: I0716 01:07:01.840796 2572 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9gnmq_3a7e030d-fad8-480d-9dc6-fe5057048cf9/konnectivity-agent/0.log" limit=1024 Jul 16 01:07:02.016119 ip-10-0-143-200 kubenswrapper[2572]: I0716 01:07:02.016080 2572 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9gnmq_3a7e030d-fad8-480d-9dc6-fe5057048cf9/konnectivity-agent/0.log" limit=1024 Jul 16 01:07:02.549394 ip-10-0-143-200 kubenswrapper[2572]: I0716 01:07:02.549359 2572 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9gnmq_3a7e030d-fad8-480d-9dc6-fe5057048cf9/konnectivity-agent/0.log" limit=1024 Jul 16 01:07:28.597025 ip-10-0-143-200 kubenswrapper[2572]: I0716 01:07:28.596985 2572 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9gnmq_3a7e030d-fad8-480d-9dc6-fe5057048cf9/konnectivity-agent/0.log" limit=1024 Jul 16 01:07:35.018086 ip-10-0-143-200 kubenswrapper[2572]: I0716 01:07:35.018047 2572 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9gnmq_3a7e030d-fad8-480d-9dc6-fe5057048cf9/konnectivity-agent/0.log" limit=1024 Jul 16 01:07:35.099904 ip-10-0-143-200 kubenswrapper[2572]: I0716 01:07:35.099865 2572 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9gnmq_3a7e030d-fad8-480d-9dc6-fe5057048cf9/konnectivity-agent/0.log" limit=1024 Jul 16 01:07:35.220939 ip-10-0-143-200 kubenswrapper[2572]: I0716 01:07:35.220907 2572 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9gnmq_3a7e030d-fad8-480d-9dc6-fe5057048cf9/konnectivity-agent/0.log" limit=1024 Jul 16 01:07:51.191074 ip-10-0-143-200 kubenswrapper[2572]: I0716 01:07:51.191039 2572 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9gnmq_3a7e030d-fad8-480d-9dc6-fe5057048cf9/konnectivity-agent/0.log" limit=1024 Jul 16 01:07:57.192607 ip-10-0-143-200 kubenswrapper[2572]: I0716 01:07:57.192569 2572 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9gnmq_3a7e030d-fad8-480d-9dc6-fe5057048cf9/konnectivity-agent/0.log" limit=1024 Jul 16 01:07:57.301548 ip-10-0-143-200 kubenswrapper[2572]: I0716 01:07:57.301512 2572 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9gnmq_3a7e030d-fad8-480d-9dc6-fe5057048cf9/konnectivity-agent/0.log" limit=1024 Jul 16 01:07:57.397664 ip-10-0-143-200 kubenswrapper[2572]: I0716 01:07:57.397609 2572 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9gnmq_3a7e030d-fad8-480d-9dc6-fe5057048cf9/konnectivity-agent/0.log" limit=1024 Jul 16 01:08:07.961147 ip-10-0-143-200 kubenswrapper[2572]: I0716 01:08:07.961109 2572 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9gnmq_3a7e030d-fad8-480d-9dc6-fe5057048cf9/konnectivity-agent/0.log" limit=1024 Jul 16 01:08:08.755709 ip-10-0-143-200 kubenswrapper[2572]: I0716 01:08:08.755671 2572 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9gnmq_3a7e030d-fad8-480d-9dc6-fe5057048cf9/konnectivity-agent/0.log" limit=1024 Jul 16 01:08:14.647818 ip-10-0-143-200 kubenswrapper[2572]: I0716 01:08:14.647773 2572 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9gnmq_3a7e030d-fad8-480d-9dc6-fe5057048cf9/konnectivity-agent/0.log" limit=1024 Jul 16 01:08:14.861159 ip-10-0-143-200 kubenswrapper[2572]: I0716 01:08:14.861119 2572 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9gnmq_3a7e030d-fad8-480d-9dc6-fe5057048cf9/konnectivity-agent/0.log" limit=1024 Jul 16 01:08:15.041999 ip-10-0-143-200 kubenswrapper[2572]: I0716 01:08:15.041969 2572 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9gnmq_3a7e030d-fad8-480d-9dc6-fe5057048cf9/konnectivity-agent/0.log" limit=1024 Jul 16 01:08:15.819079 ip-10-0-143-200 kubenswrapper[2572]: I0716 01:08:15.819040 2572 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9gnmq_3a7e030d-fad8-480d-9dc6-fe5057048cf9/konnectivity-agent/0.log" limit=1024 Jul 16 01:08:15.938905 ip-10-0-143-200 kubenswrapper[2572]: I0716 01:08:15.938868 2572 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9gnmq_3a7e030d-fad8-480d-9dc6-fe5057048cf9/konnectivity-agent/0.log" limit=1024 Jul 16 01:08:17.744871 ip-10-0-143-200 kubenswrapper[2572]: I0716 01:08:17.744783 2572 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9gnmq_3a7e030d-fad8-480d-9dc6-fe5057048cf9/konnectivity-agent/0.log" limit=1024 Jul 16 01:08:23.929821 ip-10-0-143-200 kubenswrapper[2572]: I0716 01:08:23.929780 2572 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9gnmq_3a7e030d-fad8-480d-9dc6-fe5057048cf9/konnectivity-agent/0.log" limit=1024 Jul 16 01:08:24.021399 ip-10-0-143-200 kubenswrapper[2572]: I0716 01:08:24.021362 2572 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9gnmq_3a7e030d-fad8-480d-9dc6-fe5057048cf9/konnectivity-agent/0.log" limit=1024 Jul 16 01:08:24.627108 ip-10-0-143-200 kubenswrapper[2572]: I0716 01:08:24.627074 2572 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9gnmq_3a7e030d-fad8-480d-9dc6-fe5057048cf9/konnectivity-agent/0.log" limit=1024 Jul 16 01:08:30.794904 ip-10-0-143-200 kubenswrapper[2572]: I0716 01:08:30.794867 2572 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9gnmq_3a7e030d-fad8-480d-9dc6-fe5057048cf9/konnectivity-agent/0.log" limit=1024 Jul 16 01:08:30.898220 ip-10-0-143-200 kubenswrapper[2572]: I0716 01:08:30.898186 2572 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9gnmq_3a7e030d-fad8-480d-9dc6-fe5057048cf9/konnectivity-agent/0.log" limit=1024 Jul 16 01:08:31.039954 ip-10-0-143-200 kubenswrapper[2572]: I0716 01:08:31.039922 2572 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9gnmq_3a7e030d-fad8-480d-9dc6-fe5057048cf9/konnectivity-agent/0.log" limit=1024 Jul 16 01:08:31.357999 ip-10-0-143-200 kubenswrapper[2572]: I0716 01:08:31.357961 2572 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-ingress/e2e-mt-cdf52386-openshift-default-55d96d8955-qj6zk"] Jul 16 01:08:31.358423 ip-10-0-143-200 kubenswrapper[2572]: I0716 01:08:31.358400 2572 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="46c5f745-1157-4244-96c9-72496814e558" containerName="istio-proxy" Jul 16 01:08:31.358423 ip-10-0-143-200 kubenswrapper[2572]: I0716 01:08:31.358415 2572 state_mem.go:107] "Deleted CPUSet assignment" podUID="46c5f745-1157-4244-96c9-72496814e558" containerName="istio-proxy" Jul 16 01:08:31.358546 ip-10-0-143-200 kubenswrapper[2572]: I0716 01:08:31.358496 2572 memory_manager.go:368] "RemoveStaleState removing state" podUID="46c5f745-1157-4244-96c9-72496814e558" containerName="istio-proxy" Jul 16 01:08:31.361355 ip-10-0-143-200 kubenswrapper[2572]: I0716 01:08:31.361332 2572 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-mt-cdf52386-openshift-default-55d96d8955-qj6zk" Jul 16 01:08:31.364166 ip-10-0-143-200 kubenswrapper[2572]: I0716 01:08:31.364148 2572 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress\"/\"e2e-mt-cdf52386-openshift-default-dockercfg-rlgkg\"" Jul 16 01:08:31.389556 ip-10-0-143-200 kubenswrapper[2572]: I0716 01:08:31.389517 2572 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/e2e-mt-cdf52386-openshift-default-55d96d8955-qj6zk"] Jul 16 01:08:31.403719 ip-10-0-143-200 kubenswrapper[2572]: I0716 01:08:31.403675 2572 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9gnmq_3a7e030d-fad8-480d-9dc6-fe5057048cf9/konnectivity-agent/0.log" limit=1024 Jul 16 01:08:31.436773 ip-10-0-143-200 kubenswrapper[2572]: I0716 01:08:31.436727 2572 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/e0d9d180-cdd0-442f-a220-f365b8b74b24-istio-envoy\") pod \"e2e-mt-cdf52386-openshift-default-55d96d8955-qj6zk\" (UID: \"e0d9d180-cdd0-442f-a220-f365b8b74b24\") " pod="openshift-ingress/e2e-mt-cdf52386-openshift-default-55d96d8955-qj6zk" Jul 16 01:08:31.436937 ip-10-0-143-200 kubenswrapper[2572]: I0716 01:08:31.436810 2572 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/e0d9d180-cdd0-442f-a220-f365b8b74b24-istiod-ca-cert\") pod \"e2e-mt-cdf52386-openshift-default-55d96d8955-qj6zk\" (UID: \"e0d9d180-cdd0-442f-a220-f365b8b74b24\") " pod="openshift-ingress/e2e-mt-cdf52386-openshift-default-55d96d8955-qj6zk" Jul 16 01:08:31.436937 ip-10-0-143-200 kubenswrapper[2572]: I0716 01:08:31.436865 2572 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/e0d9d180-cdd0-442f-a220-f365b8b74b24-credential-socket\") pod \"e2e-mt-cdf52386-openshift-default-55d96d8955-qj6zk\" (UID: \"e0d9d180-cdd0-442f-a220-f365b8b74b24\") " pod="openshift-ingress/e2e-mt-cdf52386-openshift-default-55d96d8955-qj6zk" Jul 16 01:08:31.436937 ip-10-0-143-200 kubenswrapper[2572]: I0716 01:08:31.436909 2572 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/e0d9d180-cdd0-442f-a220-f365b8b74b24-istio-token\") pod \"e2e-mt-cdf52386-openshift-default-55d96d8955-qj6zk\" (UID: \"e0d9d180-cdd0-442f-a220-f365b8b74b24\") " pod="openshift-ingress/e2e-mt-cdf52386-openshift-default-55d96d8955-qj6zk" Jul 16 01:08:31.437078 ip-10-0-143-200 kubenswrapper[2572]: I0716 01:08:31.436960 2572 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-6hq65\" (UniqueName: \"kubernetes.io/projected/e0d9d180-cdd0-442f-a220-f365b8b74b24-kube-api-access-6hq65\") pod \"e2e-mt-cdf52386-openshift-default-55d96d8955-qj6zk\" (UID: \"e0d9d180-cdd0-442f-a220-f365b8b74b24\") " pod="openshift-ingress/e2e-mt-cdf52386-openshift-default-55d96d8955-qj6zk" Jul 16 01:08:31.437078 ip-10-0-143-200 kubenswrapper[2572]: I0716 01:08:31.436997 2572 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/e0d9d180-cdd0-442f-a220-f365b8b74b24-workload-certs\") pod \"e2e-mt-cdf52386-openshift-default-55d96d8955-qj6zk\" (UID: \"e0d9d180-cdd0-442f-a220-f365b8b74b24\") " pod="openshift-ingress/e2e-mt-cdf52386-openshift-default-55d96d8955-qj6zk" Jul 16 01:08:31.437156 ip-10-0-143-200 kubenswrapper[2572]: I0716 01:08:31.437095 2572 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/e0d9d180-cdd0-442f-a220-f365b8b74b24-istio-data\") pod \"e2e-mt-cdf52386-openshift-default-55d96d8955-qj6zk\" (UID: \"e0d9d180-cdd0-442f-a220-f365b8b74b24\") " pod="openshift-ingress/e2e-mt-cdf52386-openshift-default-55d96d8955-qj6zk" Jul 16 01:08:31.437156 ip-10-0-143-200 kubenswrapper[2572]: I0716 01:08:31.437130 2572 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/e0d9d180-cdd0-442f-a220-f365b8b74b24-istio-podinfo\") pod \"e2e-mt-cdf52386-openshift-default-55d96d8955-qj6zk\" (UID: \"e0d9d180-cdd0-442f-a220-f365b8b74b24\") " pod="openshift-ingress/e2e-mt-cdf52386-openshift-default-55d96d8955-qj6zk" Jul 16 01:08:31.437242 ip-10-0-143-200 kubenswrapper[2572]: I0716 01:08:31.437184 2572 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/e0d9d180-cdd0-442f-a220-f365b8b74b24-workload-socket\") pod \"e2e-mt-cdf52386-openshift-default-55d96d8955-qj6zk\" (UID: \"e0d9d180-cdd0-442f-a220-f365b8b74b24\") " pod="openshift-ingress/e2e-mt-cdf52386-openshift-default-55d96d8955-qj6zk" Jul 16 01:08:31.538751 ip-10-0-143-200 kubenswrapper[2572]: I0716 01:08:31.538692 2572 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/e0d9d180-cdd0-442f-a220-f365b8b74b24-istiod-ca-cert\") pod \"e2e-mt-cdf52386-openshift-default-55d96d8955-qj6zk\" (UID: \"e0d9d180-cdd0-442f-a220-f365b8b74b24\") " pod="openshift-ingress/e2e-mt-cdf52386-openshift-default-55d96d8955-qj6zk" Jul 16 01:08:31.538751 ip-10-0-143-200 kubenswrapper[2572]: I0716 01:08:31.538758 2572 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/e0d9d180-cdd0-442f-a220-f365b8b74b24-credential-socket\") pod \"e2e-mt-cdf52386-openshift-default-55d96d8955-qj6zk\" (UID: \"e0d9d180-cdd0-442f-a220-f365b8b74b24\") " pod="openshift-ingress/e2e-mt-cdf52386-openshift-default-55d96d8955-qj6zk" Jul 16 01:08:31.539036 ip-10-0-143-200 kubenswrapper[2572]: I0716 01:08:31.538804 2572 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/e0d9d180-cdd0-442f-a220-f365b8b74b24-istio-token\") pod \"e2e-mt-cdf52386-openshift-default-55d96d8955-qj6zk\" (UID: \"e0d9d180-cdd0-442f-a220-f365b8b74b24\") " pod="openshift-ingress/e2e-mt-cdf52386-openshift-default-55d96d8955-qj6zk" Jul 16 01:08:31.539036 ip-10-0-143-200 kubenswrapper[2572]: I0716 01:08:31.538861 2572 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-6hq65\" (UniqueName: \"kubernetes.io/projected/e0d9d180-cdd0-442f-a220-f365b8b74b24-kube-api-access-6hq65\") pod \"e2e-mt-cdf52386-openshift-default-55d96d8955-qj6zk\" (UID: \"e0d9d180-cdd0-442f-a220-f365b8b74b24\") " pod="openshift-ingress/e2e-mt-cdf52386-openshift-default-55d96d8955-qj6zk" Jul 16 01:08:31.539036 ip-10-0-143-200 kubenswrapper[2572]: I0716 01:08:31.538901 2572 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/e0d9d180-cdd0-442f-a220-f365b8b74b24-workload-certs\") pod \"e2e-mt-cdf52386-openshift-default-55d96d8955-qj6zk\" (UID: \"e0d9d180-cdd0-442f-a220-f365b8b74b24\") " pod="openshift-ingress/e2e-mt-cdf52386-openshift-default-55d96d8955-qj6zk" Jul 16 01:08:31.539036 ip-10-0-143-200 kubenswrapper[2572]: I0716 01:08:31.538961 2572 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/e0d9d180-cdd0-442f-a220-f365b8b74b24-istio-data\") pod \"e2e-mt-cdf52386-openshift-default-55d96d8955-qj6zk\" (UID: \"e0d9d180-cdd0-442f-a220-f365b8b74b24\") " pod="openshift-ingress/e2e-mt-cdf52386-openshift-default-55d96d8955-qj6zk" Jul 16 01:08:31.539036 ip-10-0-143-200 kubenswrapper[2572]: I0716 01:08:31.538999 2572 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/e0d9d180-cdd0-442f-a220-f365b8b74b24-istio-podinfo\") pod \"e2e-mt-cdf52386-openshift-default-55d96d8955-qj6zk\" (UID: \"e0d9d180-cdd0-442f-a220-f365b8b74b24\") " pod="openshift-ingress/e2e-mt-cdf52386-openshift-default-55d96d8955-qj6zk" Jul 16 01:08:31.539299 ip-10-0-143-200 kubenswrapper[2572]: I0716 01:08:31.539038 2572 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/e0d9d180-cdd0-442f-a220-f365b8b74b24-workload-socket\") pod \"e2e-mt-cdf52386-openshift-default-55d96d8955-qj6zk\" (UID: \"e0d9d180-cdd0-442f-a220-f365b8b74b24\") " pod="openshift-ingress/e2e-mt-cdf52386-openshift-default-55d96d8955-qj6zk" Jul 16 01:08:31.539299 ip-10-0-143-200 kubenswrapper[2572]: I0716 01:08:31.539214 2572 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/e0d9d180-cdd0-442f-a220-f365b8b74b24-credential-socket\") pod \"e2e-mt-cdf52386-openshift-default-55d96d8955-qj6zk\" (UID: \"e0d9d180-cdd0-442f-a220-f365b8b74b24\") " pod="openshift-ingress/e2e-mt-cdf52386-openshift-default-55d96d8955-qj6zk" Jul 16 01:08:31.539415 ip-10-0-143-200 kubenswrapper[2572]: I0716 01:08:31.539297 2572 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/e0d9d180-cdd0-442f-a220-f365b8b74b24-workload-certs\") pod \"e2e-mt-cdf52386-openshift-default-55d96d8955-qj6zk\" (UID: \"e0d9d180-cdd0-442f-a220-f365b8b74b24\") " pod="openshift-ingress/e2e-mt-cdf52386-openshift-default-55d96d8955-qj6zk" Jul 16 01:08:31.539415 ip-10-0-143-200 kubenswrapper[2572]: I0716 01:08:31.539405 2572 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/e0d9d180-cdd0-442f-a220-f365b8b74b24-istio-data\") pod \"e2e-mt-cdf52386-openshift-default-55d96d8955-qj6zk\" (UID: \"e0d9d180-cdd0-442f-a220-f365b8b74b24\") " pod="openshift-ingress/e2e-mt-cdf52386-openshift-default-55d96d8955-qj6zk" Jul 16 01:08:31.539555 ip-10-0-143-200 kubenswrapper[2572]: I0716 01:08:31.539530 2572 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/e0d9d180-cdd0-442f-a220-f365b8b74b24-istio-envoy\") pod \"e2e-mt-cdf52386-openshift-default-55d96d8955-qj6zk\" (UID: \"e0d9d180-cdd0-442f-a220-f365b8b74b24\") " pod="openshift-ingress/e2e-mt-cdf52386-openshift-default-55d96d8955-qj6zk" Jul 16 01:08:31.539613 ip-10-0-143-200 kubenswrapper[2572]: I0716 01:08:31.539560 2572 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/e0d9d180-cdd0-442f-a220-f365b8b74b24-workload-socket\") pod \"e2e-mt-cdf52386-openshift-default-55d96d8955-qj6zk\" (UID: \"e0d9d180-cdd0-442f-a220-f365b8b74b24\") " pod="openshift-ingress/e2e-mt-cdf52386-openshift-default-55d96d8955-qj6zk" Jul 16 01:08:31.539922 ip-10-0-143-200 kubenswrapper[2572]: I0716 01:08:31.539900 2572 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/e0d9d180-cdd0-442f-a220-f365b8b74b24-istiod-ca-cert\") pod \"e2e-mt-cdf52386-openshift-default-55d96d8955-qj6zk\" (UID: \"e0d9d180-cdd0-442f-a220-f365b8b74b24\") " pod="openshift-ingress/e2e-mt-cdf52386-openshift-default-55d96d8955-qj6zk" Jul 16 01:08:31.541599 ip-10-0-143-200 kubenswrapper[2572]: I0716 01:08:31.541567 2572 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/e0d9d180-cdd0-442f-a220-f365b8b74b24-istio-envoy\") pod \"e2e-mt-cdf52386-openshift-default-55d96d8955-qj6zk\" (UID: \"e0d9d180-cdd0-442f-a220-f365b8b74b24\") " pod="openshift-ingress/e2e-mt-cdf52386-openshift-default-55d96d8955-qj6zk" Jul 16 01:08:31.541731 ip-10-0-143-200 kubenswrapper[2572]: I0716 01:08:31.541701 2572 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/e0d9d180-cdd0-442f-a220-f365b8b74b24-istio-podinfo\") pod \"e2e-mt-cdf52386-openshift-default-55d96d8955-qj6zk\" (UID: \"e0d9d180-cdd0-442f-a220-f365b8b74b24\") " pod="openshift-ingress/e2e-mt-cdf52386-openshift-default-55d96d8955-qj6zk" Jul 16 01:08:31.566711 ip-10-0-143-200 kubenswrapper[2572]: I0716 01:08:31.566673 2572 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/e0d9d180-cdd0-442f-a220-f365b8b74b24-istio-token\") pod \"e2e-mt-cdf52386-openshift-default-55d96d8955-qj6zk\" (UID: \"e0d9d180-cdd0-442f-a220-f365b8b74b24\") " pod="openshift-ingress/e2e-mt-cdf52386-openshift-default-55d96d8955-qj6zk" Jul 16 01:08:31.566884 ip-10-0-143-200 kubenswrapper[2572]: I0716 01:08:31.566757 2572 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-6hq65\" (UniqueName: \"kubernetes.io/projected/e0d9d180-cdd0-442f-a220-f365b8b74b24-kube-api-access-6hq65\") pod \"e2e-mt-cdf52386-openshift-default-55d96d8955-qj6zk\" (UID: \"e0d9d180-cdd0-442f-a220-f365b8b74b24\") " pod="openshift-ingress/e2e-mt-cdf52386-openshift-default-55d96d8955-qj6zk" Jul 16 01:08:31.672337 ip-10-0-143-200 kubenswrapper[2572]: I0716 01:08:31.672255 2572 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-mt-cdf52386-openshift-default-55d96d8955-qj6zk" Jul 16 01:08:31.829399 ip-10-0-143-200 kubenswrapper[2572]: I0716 01:08:31.829371 2572 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/e2e-mt-cdf52386-openshift-default-55d96d8955-qj6zk"] Jul 16 01:08:31.831650 ip-10-0-143-200 kubenswrapper[2572]: W0716 01:08:31.831603 2572 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pode0d9d180_cdd0_442f_a220_f365b8b74b24.slice/crio-6f8b1b51926272917a158eb5571695cb56a4d2a1d5ab4d59fc95de4adbf23171 WatchSource:0}: Error finding container 6f8b1b51926272917a158eb5571695cb56a4d2a1d5ab4d59fc95de4adbf23171: Status 404 returned error can't find the container with id 6f8b1b51926272917a158eb5571695cb56a4d2a1d5ab4d59fc95de4adbf23171 Jul 16 01:08:31.833742 ip-10-0-143-200 kubenswrapper[2572]: I0716 01:08:31.833706 2572 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236228Ki","pods":"250"} Jul 16 01:08:31.833821 ip-10-0-143-200 kubenswrapper[2572]: I0716 01:08:31.833778 2572 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236228Ki","pods":"250"} Jul 16 01:08:31.833821 ip-10-0-143-200 kubenswrapper[2572]: I0716 01:08:31.833810 2572 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236228Ki","pods":"250"} Jul 16 01:08:32.037127 ip-10-0-143-200 kubenswrapper[2572]: I0716 01:08:32.037080 2572 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9gnmq_3a7e030d-fad8-480d-9dc6-fe5057048cf9/konnectivity-agent/0.log" limit=1024 Jul 16 01:08:32.185671 ip-10-0-143-200 kubenswrapper[2572]: I0716 01:08:32.185613 2572 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-mt-cdf52386-openshift-default-55d96d8955-qj6zk" event={"ID":"e0d9d180-cdd0-442f-a220-f365b8b74b24","Type":"ContainerStarted","Data":"dcc56033cab10b1c27d472906f94e782a94143e3b557fe0c97ffaf9a66c52059"} Jul 16 01:08:32.185877 ip-10-0-143-200 kubenswrapper[2572]: I0716 01:08:32.185743 2572 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-mt-cdf52386-openshift-default-55d96d8955-qj6zk" event={"ID":"e0d9d180-cdd0-442f-a220-f365b8b74b24","Type":"ContainerStarted","Data":"6f8b1b51926272917a158eb5571695cb56a4d2a1d5ab4d59fc95de4adbf23171"} Jul 16 01:08:32.212405 ip-10-0-143-200 kubenswrapper[2572]: I0716 01:08:32.212341 2572 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-ingress/e2e-mt-cdf52386-openshift-default-55d96d8955-qj6zk" podStartSLOduration=1.2123201749999999 podStartE2EDuration="1.212320175s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-16 01:08:31 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-16 01:08:32.209167474 +0000 UTC m=+1717.317922153" watchObservedRunningTime="2026-07-16 01:08:32.212320175 +0000 UTC m=+1717.321074851" Jul 16 01:08:32.673514 ip-10-0-143-200 kubenswrapper[2572]: I0716 01:08:32.673474 2572 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="openshift-ingress/e2e-mt-cdf52386-openshift-default-55d96d8955-qj6zk" Jul 16 01:08:32.679037 ip-10-0-143-200 kubenswrapper[2572]: I0716 01:08:32.679009 2572 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="openshift-ingress/e2e-mt-cdf52386-openshift-default-55d96d8955-qj6zk" Jul 16 01:08:33.193206 ip-10-0-143-200 kubenswrapper[2572]: I0716 01:08:33.193167 2572 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/e2e-mt-cdf52386-openshift-default-55d96d8955-qj6zk" Jul 16 01:08:33.194285 ip-10-0-143-200 kubenswrapper[2572]: I0716 01:08:33.194257 2572 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/e2e-mt-cdf52386-openshift-default-55d96d8955-qj6zk" Jul 16 01:08:33.253955 ip-10-0-143-200 kubenswrapper[2572]: I0716 01:08:33.253920 2572 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9gnmq_3a7e030d-fad8-480d-9dc6-fe5057048cf9/konnectivity-agent/0.log" limit=1024 Jul 16 01:08:33.387933 ip-10-0-143-200 kubenswrapper[2572]: I0716 01:08:33.387897 2572 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9gnmq_3a7e030d-fad8-480d-9dc6-fe5057048cf9/konnectivity-agent/0.log" limit=1024 Jul 16 01:08:33.504055 ip-10-0-143-200 kubenswrapper[2572]: I0716 01:08:33.504018 2572 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9gnmq_3a7e030d-fad8-480d-9dc6-fe5057048cf9/konnectivity-agent/0.log" limit=1024 Jul 16 01:08:34.069808 ip-10-0-143-200 kubenswrapper[2572]: I0716 01:08:34.069773 2572 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9gnmq_3a7e030d-fad8-480d-9dc6-fe5057048cf9/konnectivity-agent/0.log" limit=1024 Jul 16 01:08:34.191988 ip-10-0-143-200 kubenswrapper[2572]: I0716 01:08:34.191951 2572 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9gnmq_3a7e030d-fad8-480d-9dc6-fe5057048cf9/konnectivity-agent/0.log" limit=1024 Jul 16 01:08:38.695290 ip-10-0-143-200 kubenswrapper[2572]: I0716 01:08:38.695251 2572 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9gnmq_3a7e030d-fad8-480d-9dc6-fe5057048cf9/konnectivity-agent/0.log" limit=1024 Jul 16 01:08:39.388212 ip-10-0-143-200 kubenswrapper[2572]: I0716 01:08:39.388125 2572 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9gnmq_3a7e030d-fad8-480d-9dc6-fe5057048cf9/konnectivity-agent/0.log" limit=1024 Jul 16 01:08:39.480446 ip-10-0-143-200 kubenswrapper[2572]: I0716 01:08:39.480412 2572 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9gnmq_3a7e030d-fad8-480d-9dc6-fe5057048cf9/konnectivity-agent/0.log" limit=1024 Jul 16 01:08:39.822444 ip-10-0-143-200 kubenswrapper[2572]: I0716 01:08:39.822407 2572 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9gnmq_3a7e030d-fad8-480d-9dc6-fe5057048cf9/konnectivity-agent/0.log" limit=1024 Jul 16 01:08:45.656646 ip-10-0-143-200 kubenswrapper[2572]: I0716 01:08:45.656595 2572 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9gnmq_3a7e030d-fad8-480d-9dc6-fe5057048cf9/konnectivity-agent/0.log" limit=1024 Jul 16 01:08:45.757817 ip-10-0-143-200 kubenswrapper[2572]: I0716 01:08:45.757775 2572 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9gnmq_3a7e030d-fad8-480d-9dc6-fe5057048cf9/konnectivity-agent/0.log" limit=1024 Jul 16 01:08:46.266729 ip-10-0-143-200 kubenswrapper[2572]: I0716 01:08:46.266693 2572 generic.go:356] "Generic (PLEG): container finished" podID="e0d9d180-cdd0-442f-a220-f365b8b74b24" containerID="dcc56033cab10b1c27d472906f94e782a94143e3b557fe0c97ffaf9a66c52059" exitCode=0 Jul 16 01:08:46.266943 ip-10-0-143-200 kubenswrapper[2572]: I0716 01:08:46.266743 2572 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-mt-cdf52386-openshift-default-55d96d8955-qj6zk" event={"ID":"e0d9d180-cdd0-442f-a220-f365b8b74b24","Type":"ContainerDied","Data":"dcc56033cab10b1c27d472906f94e782a94143e3b557fe0c97ffaf9a66c52059"} Jul 16 01:08:46.267243 ip-10-0-143-200 kubenswrapper[2572]: I0716 01:08:46.267226 2572 scope.go:117] "RemoveContainer" containerID="dcc56033cab10b1c27d472906f94e782a94143e3b557fe0c97ffaf9a66c52059" Jul 16 01:08:46.268340 ip-10-0-143-200 kubenswrapper[2572]: I0716 01:08:46.268310 2572 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236228Ki","pods":"250"} Jul 16 01:08:46.268415 ip-10-0-143-200 kubenswrapper[2572]: I0716 01:08:46.268398 2572 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236228Ki","pods":"250"} Jul 16 01:08:46.268465 ip-10-0-143-200 kubenswrapper[2572]: I0716 01:08:46.268451 2572 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236228Ki","pods":"250"} Jul 16 01:08:46.351939 ip-10-0-143-200 kubenswrapper[2572]: I0716 01:08:46.351909 2572 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9gnmq_3a7e030d-fad8-480d-9dc6-fe5057048cf9/konnectivity-agent/0.log" limit=1024 Jul 16 01:08:47.273759 ip-10-0-143-200 kubenswrapper[2572]: I0716 01:08:47.273725 2572 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-mt-cdf52386-openshift-default-55d96d8955-qj6zk" event={"ID":"e0d9d180-cdd0-442f-a220-f365b8b74b24","Type":"ContainerStarted","Data":"6ebc1c8d21c7f79bb7da6ea513a194523d118a97763fea60f64ac3a0b32285f7"} Jul 16 01:08:47.673054 ip-10-0-143-200 kubenswrapper[2572]: I0716 01:08:47.672959 2572 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="openshift-ingress/e2e-mt-cdf52386-openshift-default-55d96d8955-qj6zk" Jul 16 01:08:47.674261 ip-10-0-143-200 kubenswrapper[2572]: I0716 01:08:47.674236 2572 patch_prober.go:28] interesting pod/e2e-mt-cdf52386-openshift-default-55d96d8955-qj6zk container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.132.0.56:15021/healthz/ready\": dial tcp 10.132.0.56:15021: connect: connection refused" start-of-body= Jul 16 01:08:47.674320 ip-10-0-143-200 kubenswrapper[2572]: I0716 01:08:47.674299 2572 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-mt-cdf52386-openshift-default-55d96d8955-qj6zk" podUID="e0d9d180-cdd0-442f-a220-f365b8b74b24" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.132.0.56:15021/healthz/ready\": dial tcp 10.132.0.56:15021: connect: connection refused" Jul 16 01:08:48.193930 ip-10-0-143-200 kubenswrapper[2572]: I0716 01:08:48.193897 2572 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/e2e-mt-cdf52386-openshift-default-55d96d8955-qj6zk" Jul 16 01:08:48.673733 ip-10-0-143-200 kubenswrapper[2572]: I0716 01:08:48.673695 2572 patch_prober.go:28] interesting pod/e2e-mt-cdf52386-openshift-default-55d96d8955-qj6zk container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.132.0.56:15021/healthz/ready\": dial tcp 10.132.0.56:15021: connect: connection refused" start-of-body= Jul 16 01:08:48.674150 ip-10-0-143-200 kubenswrapper[2572]: I0716 01:08:48.673758 2572 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-mt-cdf52386-openshift-default-55d96d8955-qj6zk" podUID="e0d9d180-cdd0-442f-a220-f365b8b74b24" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.132.0.56:15021/healthz/ready\": dial tcp 10.132.0.56:15021: connect: connection refused" Jul 16 01:08:49.673521 ip-10-0-143-200 kubenswrapper[2572]: I0716 01:08:49.673484 2572 patch_prober.go:28] interesting pod/e2e-mt-cdf52386-openshift-default-55d96d8955-qj6zk container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.132.0.56:15021/healthz/ready\": dial tcp 10.132.0.56:15021: connect: connection refused" start-of-body= Jul 16 01:08:49.673712 ip-10-0-143-200 kubenswrapper[2572]: I0716 01:08:49.673550 2572 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-mt-cdf52386-openshift-default-55d96d8955-qj6zk" podUID="e0d9d180-cdd0-442f-a220-f365b8b74b24" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.132.0.56:15021/healthz/ready\": dial tcp 10.132.0.56:15021: connect: connection refused" Jul 16 01:08:50.673399 ip-10-0-143-200 kubenswrapper[2572]: I0716 01:08:50.673356 2572 patch_prober.go:28] interesting pod/e2e-mt-cdf52386-openshift-default-55d96d8955-qj6zk container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.132.0.56:15021/healthz/ready\": dial tcp 10.132.0.56:15021: connect: connection refused" start-of-body= Jul 16 01:08:50.673919 ip-10-0-143-200 kubenswrapper[2572]: I0716 01:08:50.673440 2572 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-mt-cdf52386-openshift-default-55d96d8955-qj6zk" podUID="e0d9d180-cdd0-442f-a220-f365b8b74b24" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.132.0.56:15021/healthz/ready\": dial tcp 10.132.0.56:15021: connect: connection refused" Jul 16 01:08:51.676713 ip-10-0-143-200 kubenswrapper[2572]: I0716 01:08:51.676684 2572 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="openshift-ingress/e2e-mt-cdf52386-openshift-default-55d96d8955-qj6zk" Jul 16 01:08:51.677689 ip-10-0-143-200 kubenswrapper[2572]: I0716 01:08:51.677672 2572 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/e2e-mt-cdf52386-openshift-default-55d96d8955-qj6zk" Jul 16 01:08:52.457081 ip-10-0-143-200 kubenswrapper[2572]: I0716 01:08:52.457023 2572 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9gnmq_3a7e030d-fad8-480d-9dc6-fe5057048cf9/konnectivity-agent/0.log" limit=1024 Jul 16 01:08:52.558684 ip-10-0-143-200 kubenswrapper[2572]: I0716 01:08:52.558620 2572 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9gnmq_3a7e030d-fad8-480d-9dc6-fe5057048cf9/konnectivity-agent/0.log" limit=1024 Jul 16 01:08:52.712291 ip-10-0-143-200 kubenswrapper[2572]: I0716 01:08:52.712189 2572 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-ingress/e2e-mt-cdf52386-openshift-default-55d96d8955-qj6zk"] Jul 16 01:08:52.735756 ip-10-0-143-200 kubenswrapper[2572]: I0716 01:08:52.735697 2572 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9gnmq_3a7e030d-fad8-480d-9dc6-fe5057048cf9/konnectivity-agent/0.log" limit=1024 Jul 16 01:08:52.968584 ip-10-0-143-200 kubenswrapper[2572]: I0716 01:08:52.968476 2572 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9gnmq_3a7e030d-fad8-480d-9dc6-fe5057048cf9/konnectivity-agent/0.log" limit=1024 Jul 16 01:08:53.102677 ip-10-0-143-200 kubenswrapper[2572]: I0716 01:08:53.102605 2572 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9gnmq_3a7e030d-fad8-480d-9dc6-fe5057048cf9/konnectivity-agent/0.log" limit=1024 Jul 16 01:08:53.201801 ip-10-0-143-200 kubenswrapper[2572]: I0716 01:08:53.201771 2572 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9gnmq_3a7e030d-fad8-480d-9dc6-fe5057048cf9/konnectivity-agent/0.log" limit=1024 Jul 16 01:08:53.309229 ip-10-0-143-200 kubenswrapper[2572]: I0716 01:08:53.309107 2572 kuberuntime_container.go:873] "Killing container with a grace period" pod="openshift-ingress/e2e-mt-cdf52386-openshift-default-55d96d8955-qj6zk" podUID="e0d9d180-cdd0-442f-a220-f365b8b74b24" containerName="istio-proxy" containerID="cri-o://6ebc1c8d21c7f79bb7da6ea513a194523d118a97763fea60f64ac3a0b32285f7" gracePeriod=30 Jul 16 01:08:53.319069 ip-10-0-143-200 kubenswrapper[2572]: I0716 01:08:53.319043 2572 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9gnmq_3a7e030d-fad8-480d-9dc6-fe5057048cf9/konnectivity-agent/0.log" limit=1024 Jul 16 01:08:58.571303 ip-10-0-143-200 kubenswrapper[2572]: I0716 01:08:58.571242 2572 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-mt-cdf52386-openshift-default-55d96d8955-qj6zk" Jul 16 01:08:58.601578 ip-10-0-143-200 kubenswrapper[2572]: I0716 01:08:58.601553 2572 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/e0d9d180-cdd0-442f-a220-f365b8b74b24-istiod-ca-cert\") pod \"e0d9d180-cdd0-442f-a220-f365b8b74b24\" (UID: \"e0d9d180-cdd0-442f-a220-f365b8b74b24\") " Jul 16 01:08:58.601765 ip-10-0-143-200 kubenswrapper[2572]: I0716 01:08:58.601593 2572 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/e0d9d180-cdd0-442f-a220-f365b8b74b24-credential-socket\") pod \"e0d9d180-cdd0-442f-a220-f365b8b74b24\" (UID: \"e0d9d180-cdd0-442f-a220-f365b8b74b24\") " Jul 16 01:08:58.601765 ip-10-0-143-200 kubenswrapper[2572]: I0716 01:08:58.601617 2572 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/e0d9d180-cdd0-442f-a220-f365b8b74b24-workload-socket\") pod \"e0d9d180-cdd0-442f-a220-f365b8b74b24\" (UID: \"e0d9d180-cdd0-442f-a220-f365b8b74b24\") " Jul 16 01:08:58.601765 ip-10-0-143-200 kubenswrapper[2572]: I0716 01:08:58.601700 2572 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/e0d9d180-cdd0-442f-a220-f365b8b74b24-istio-data\") pod \"e0d9d180-cdd0-442f-a220-f365b8b74b24\" (UID: \"e0d9d180-cdd0-442f-a220-f365b8b74b24\") " Jul 16 01:08:58.601765 ip-10-0-143-200 kubenswrapper[2572]: I0716 01:08:58.601719 2572 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/e0d9d180-cdd0-442f-a220-f365b8b74b24-istio-podinfo\") pod \"e0d9d180-cdd0-442f-a220-f365b8b74b24\" (UID: \"e0d9d180-cdd0-442f-a220-f365b8b74b24\") " Jul 16 01:08:58.601765 ip-10-0-143-200 kubenswrapper[2572]: I0716 01:08:58.601747 2572 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/e0d9d180-cdd0-442f-a220-f365b8b74b24-workload-certs\") pod \"e0d9d180-cdd0-442f-a220-f365b8b74b24\" (UID: \"e0d9d180-cdd0-442f-a220-f365b8b74b24\") " Jul 16 01:08:58.602040 ip-10-0-143-200 kubenswrapper[2572]: I0716 01:08:58.601780 2572 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/e0d9d180-cdd0-442f-a220-f365b8b74b24-istio-envoy\") pod \"e0d9d180-cdd0-442f-a220-f365b8b74b24\" (UID: \"e0d9d180-cdd0-442f-a220-f365b8b74b24\") " Jul 16 01:08:58.602040 ip-10-0-143-200 kubenswrapper[2572]: I0716 01:08:58.601802 2572 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/e0d9d180-cdd0-442f-a220-f365b8b74b24-istio-token\") pod \"e0d9d180-cdd0-442f-a220-f365b8b74b24\" (UID: \"e0d9d180-cdd0-442f-a220-f365b8b74b24\") " Jul 16 01:08:58.602040 ip-10-0-143-200 kubenswrapper[2572]: I0716 01:08:58.601823 2572 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-6hq65\" (UniqueName: \"kubernetes.io/projected/e0d9d180-cdd0-442f-a220-f365b8b74b24-kube-api-access-6hq65\") pod \"e0d9d180-cdd0-442f-a220-f365b8b74b24\" (UID: \"e0d9d180-cdd0-442f-a220-f365b8b74b24\") " Jul 16 01:08:58.602040 ip-10-0-143-200 kubenswrapper[2572]: I0716 01:08:58.601925 2572 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/e0d9d180-cdd0-442f-a220-f365b8b74b24-credential-socket" (OuterVolumeSpecName: "credential-socket") pod "e0d9d180-cdd0-442f-a220-f365b8b74b24" (UID: "e0d9d180-cdd0-442f-a220-f365b8b74b24"). InnerVolumeSpecName "credential-socket". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Jul 16 01:08:58.602040 ip-10-0-143-200 kubenswrapper[2572]: I0716 01:08:58.601938 2572 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/e0d9d180-cdd0-442f-a220-f365b8b74b24-istiod-ca-cert" (OuterVolumeSpecName: "istiod-ca-cert") pod "e0d9d180-cdd0-442f-a220-f365b8b74b24" (UID: "e0d9d180-cdd0-442f-a220-f365b8b74b24"). InnerVolumeSpecName "istiod-ca-cert". PluginName "kubernetes.io/configmap", VolumeGIDValue "" Jul 16 01:08:58.602346 ip-10-0-143-200 kubenswrapper[2572]: I0716 01:08:58.602193 2572 reconciler_common.go:299] "Volume detached for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/e0d9d180-cdd0-442f-a220-f365b8b74b24-istiod-ca-cert\") on node \"ip-10-0-143-200.ec2.internal\" DevicePath \"\"" Jul 16 01:08:58.602346 ip-10-0-143-200 kubenswrapper[2572]: I0716 01:08:58.602205 2572 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/e0d9d180-cdd0-442f-a220-f365b8b74b24-workload-certs" (OuterVolumeSpecName: "workload-certs") pod "e0d9d180-cdd0-442f-a220-f365b8b74b24" (UID: "e0d9d180-cdd0-442f-a220-f365b8b74b24"). InnerVolumeSpecName "workload-certs". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Jul 16 01:08:58.602346 ip-10-0-143-200 kubenswrapper[2572]: I0716 01:08:58.602215 2572 reconciler_common.go:299] "Volume detached for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/e0d9d180-cdd0-442f-a220-f365b8b74b24-credential-socket\") on node \"ip-10-0-143-200.ec2.internal\" DevicePath \"\"" Jul 16 01:08:58.602346 ip-10-0-143-200 kubenswrapper[2572]: I0716 01:08:58.602256 2572 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/e0d9d180-cdd0-442f-a220-f365b8b74b24-workload-socket" (OuterVolumeSpecName: "workload-socket") pod "e0d9d180-cdd0-442f-a220-f365b8b74b24" (UID: "e0d9d180-cdd0-442f-a220-f365b8b74b24"). InnerVolumeSpecName "workload-socket". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Jul 16 01:08:58.603030 ip-10-0-143-200 kubenswrapper[2572]: I0716 01:08:58.602999 2572 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/e0d9d180-cdd0-442f-a220-f365b8b74b24-istio-data" (OuterVolumeSpecName: "istio-data") pod "e0d9d180-cdd0-442f-a220-f365b8b74b24" (UID: "e0d9d180-cdd0-442f-a220-f365b8b74b24"). InnerVolumeSpecName "istio-data". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Jul 16 01:08:58.604318 ip-10-0-143-200 kubenswrapper[2572]: I0716 01:08:58.604287 2572 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/e0d9d180-cdd0-442f-a220-f365b8b74b24-istio-token" (OuterVolumeSpecName: "istio-token") pod "e0d9d180-cdd0-442f-a220-f365b8b74b24" (UID: "e0d9d180-cdd0-442f-a220-f365b8b74b24"). InnerVolumeSpecName "istio-token". PluginName "kubernetes.io/projected", VolumeGIDValue "" Jul 16 01:08:58.604430 ip-10-0-143-200 kubenswrapper[2572]: I0716 01:08:58.604332 2572 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/e0d9d180-cdd0-442f-a220-f365b8b74b24-istio-envoy" (OuterVolumeSpecName: "istio-envoy") pod "e0d9d180-cdd0-442f-a220-f365b8b74b24" (UID: "e0d9d180-cdd0-442f-a220-f365b8b74b24"). InnerVolumeSpecName "istio-envoy". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Jul 16 01:08:58.604577 ip-10-0-143-200 kubenswrapper[2572]: I0716 01:08:58.604554 2572 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/e0d9d180-cdd0-442f-a220-f365b8b74b24-kube-api-access-6hq65" (OuterVolumeSpecName: "kube-api-access-6hq65") pod "e0d9d180-cdd0-442f-a220-f365b8b74b24" (UID: "e0d9d180-cdd0-442f-a220-f365b8b74b24"). InnerVolumeSpecName "kube-api-access-6hq65". PluginName "kubernetes.io/projected", VolumeGIDValue "" Jul 16 01:08:58.604577 ip-10-0-143-200 kubenswrapper[2572]: I0716 01:08:58.604565 2572 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/downward-api/e0d9d180-cdd0-442f-a220-f365b8b74b24-istio-podinfo" (OuterVolumeSpecName: "istio-podinfo") pod "e0d9d180-cdd0-442f-a220-f365b8b74b24" (UID: "e0d9d180-cdd0-442f-a220-f365b8b74b24"). InnerVolumeSpecName "istio-podinfo". PluginName "kubernetes.io/downward-api", VolumeGIDValue "" Jul 16 01:08:58.702748 ip-10-0-143-200 kubenswrapper[2572]: I0716 01:08:58.702715 2572 reconciler_common.go:299] "Volume detached for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/e0d9d180-cdd0-442f-a220-f365b8b74b24-istio-data\") on node \"ip-10-0-143-200.ec2.internal\" DevicePath \"\"" Jul 16 01:08:58.702748 ip-10-0-143-200 kubenswrapper[2572]: I0716 01:08:58.702746 2572 reconciler_common.go:299] "Volume detached for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/e0d9d180-cdd0-442f-a220-f365b8b74b24-istio-podinfo\") on node \"ip-10-0-143-200.ec2.internal\" DevicePath \"\"" Jul 16 01:08:58.702748 ip-10-0-143-200 kubenswrapper[2572]: I0716 01:08:58.702755 2572 reconciler_common.go:299] "Volume detached for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/e0d9d180-cdd0-442f-a220-f365b8b74b24-workload-certs\") on node \"ip-10-0-143-200.ec2.internal\" DevicePath \"\"" Jul 16 01:08:58.702993 ip-10-0-143-200 kubenswrapper[2572]: I0716 01:08:58.702765 2572 reconciler_common.go:299] "Volume detached for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/e0d9d180-cdd0-442f-a220-f365b8b74b24-istio-envoy\") on node \"ip-10-0-143-200.ec2.internal\" DevicePath \"\"" Jul 16 01:08:58.702993 ip-10-0-143-200 kubenswrapper[2572]: I0716 01:08:58.702773 2572 reconciler_common.go:299] "Volume detached for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/e0d9d180-cdd0-442f-a220-f365b8b74b24-istio-token\") on node \"ip-10-0-143-200.ec2.internal\" DevicePath \"\"" Jul 16 01:08:58.702993 ip-10-0-143-200 kubenswrapper[2572]: I0716 01:08:58.702780 2572 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-6hq65\" (UniqueName: \"kubernetes.io/projected/e0d9d180-cdd0-442f-a220-f365b8b74b24-kube-api-access-6hq65\") on node \"ip-10-0-143-200.ec2.internal\" DevicePath \"\"" Jul 16 01:08:58.702993 ip-10-0-143-200 kubenswrapper[2572]: I0716 01:08:58.702788 2572 reconciler_common.go:299] "Volume detached for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/e0d9d180-cdd0-442f-a220-f365b8b74b24-workload-socket\") on node \"ip-10-0-143-200.ec2.internal\" DevicePath \"\"" Jul 16 01:08:59.333102 ip-10-0-143-200 kubenswrapper[2572]: I0716 01:08:59.333065 2572 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9gnmq_3a7e030d-fad8-480d-9dc6-fe5057048cf9/konnectivity-agent/0.log" limit=1024 Jul 16 01:08:59.342465 ip-10-0-143-200 kubenswrapper[2572]: I0716 01:08:59.342439 2572 generic.go:356] "Generic (PLEG): container finished" podID="e0d9d180-cdd0-442f-a220-f365b8b74b24" containerID="6ebc1c8d21c7f79bb7da6ea513a194523d118a97763fea60f64ac3a0b32285f7" exitCode=0 Jul 16 01:08:59.342623 ip-10-0-143-200 kubenswrapper[2572]: I0716 01:08:59.342494 2572 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-mt-cdf52386-openshift-default-55d96d8955-qj6zk" Jul 16 01:08:59.342623 ip-10-0-143-200 kubenswrapper[2572]: I0716 01:08:59.342504 2572 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-mt-cdf52386-openshift-default-55d96d8955-qj6zk" event={"ID":"e0d9d180-cdd0-442f-a220-f365b8b74b24","Type":"ContainerDied","Data":"6ebc1c8d21c7f79bb7da6ea513a194523d118a97763fea60f64ac3a0b32285f7"} Jul 16 01:08:59.342623 ip-10-0-143-200 kubenswrapper[2572]: I0716 01:08:59.342547 2572 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-mt-cdf52386-openshift-default-55d96d8955-qj6zk" event={"ID":"e0d9d180-cdd0-442f-a220-f365b8b74b24","Type":"ContainerDied","Data":"6f8b1b51926272917a158eb5571695cb56a4d2a1d5ab4d59fc95de4adbf23171"} Jul 16 01:08:59.342623 ip-10-0-143-200 kubenswrapper[2572]: I0716 01:08:59.342572 2572 scope.go:117] "RemoveContainer" containerID="6ebc1c8d21c7f79bb7da6ea513a194523d118a97763fea60f64ac3a0b32285f7" Jul 16 01:08:59.352306 ip-10-0-143-200 kubenswrapper[2572]: I0716 01:08:59.352287 2572 scope.go:117] "RemoveContainer" containerID="dcc56033cab10b1c27d472906f94e782a94143e3b557fe0c97ffaf9a66c52059" Jul 16 01:08:59.360735 ip-10-0-143-200 kubenswrapper[2572]: I0716 01:08:59.360719 2572 scope.go:117] "RemoveContainer" containerID="6ebc1c8d21c7f79bb7da6ea513a194523d118a97763fea60f64ac3a0b32285f7" Jul 16 01:08:59.360982 ip-10-0-143-200 kubenswrapper[2572]: E0716 01:08:59.360964 2572 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"6ebc1c8d21c7f79bb7da6ea513a194523d118a97763fea60f64ac3a0b32285f7\": container with ID starting with 6ebc1c8d21c7f79bb7da6ea513a194523d118a97763fea60f64ac3a0b32285f7 not found: ID does not exist" containerID="6ebc1c8d21c7f79bb7da6ea513a194523d118a97763fea60f64ac3a0b32285f7" Jul 16 01:08:59.361027 ip-10-0-143-200 kubenswrapper[2572]: I0716 01:08:59.360991 2572 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"6ebc1c8d21c7f79bb7da6ea513a194523d118a97763fea60f64ac3a0b32285f7"} err="failed to get container status \"6ebc1c8d21c7f79bb7da6ea513a194523d118a97763fea60f64ac3a0b32285f7\": rpc error: code = NotFound desc = could not find container \"6ebc1c8d21c7f79bb7da6ea513a194523d118a97763fea60f64ac3a0b32285f7\": container with ID starting with 6ebc1c8d21c7f79bb7da6ea513a194523d118a97763fea60f64ac3a0b32285f7 not found: ID does not exist" Jul 16 01:08:59.361027 ip-10-0-143-200 kubenswrapper[2572]: I0716 01:08:59.361008 2572 scope.go:117] "RemoveContainer" containerID="dcc56033cab10b1c27d472906f94e782a94143e3b557fe0c97ffaf9a66c52059" Jul 16 01:08:59.361246 ip-10-0-143-200 kubenswrapper[2572]: E0716 01:08:59.361227 2572 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"dcc56033cab10b1c27d472906f94e782a94143e3b557fe0c97ffaf9a66c52059\": container with ID starting with dcc56033cab10b1c27d472906f94e782a94143e3b557fe0c97ffaf9a66c52059 not found: ID does not exist" containerID="dcc56033cab10b1c27d472906f94e782a94143e3b557fe0c97ffaf9a66c52059" Jul 16 01:08:59.361287 ip-10-0-143-200 kubenswrapper[2572]: I0716 01:08:59.361251 2572 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"dcc56033cab10b1c27d472906f94e782a94143e3b557fe0c97ffaf9a66c52059"} err="failed to get container status \"dcc56033cab10b1c27d472906f94e782a94143e3b557fe0c97ffaf9a66c52059\": rpc error: code = NotFound desc = could not find container \"dcc56033cab10b1c27d472906f94e782a94143e3b557fe0c97ffaf9a66c52059\": container with ID starting with dcc56033cab10b1c27d472906f94e782a94143e3b557fe0c97ffaf9a66c52059 not found: ID does not exist" Jul 16 01:08:59.370554 ip-10-0-143-200 kubenswrapper[2572]: I0716 01:08:59.370533 2572 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-ingress/e2e-mt-cdf52386-openshift-default-55d96d8955-qj6zk"] Jul 16 01:08:59.374079 ip-10-0-143-200 kubenswrapper[2572]: I0716 01:08:59.374056 2572 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["openshift-ingress/e2e-mt-cdf52386-openshift-default-55d96d8955-qj6zk"] Jul 16 01:08:59.439555 ip-10-0-143-200 kubenswrapper[2572]: I0716 01:08:59.439516 2572 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9gnmq_3a7e030d-fad8-480d-9dc6-fe5057048cf9/konnectivity-agent/0.log" limit=1024 Jul 16 01:08:59.449363 ip-10-0-143-200 kubenswrapper[2572]: I0716 01:08:59.449336 2572 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="e0d9d180-cdd0-442f-a220-f365b8b74b24" path="/var/lib/kubelet/pods/e0d9d180-cdd0-442f-a220-f365b8b74b24/volumes" Jul 16 01:08:59.540903 ip-10-0-143-200 kubenswrapper[2572]: I0716 01:08:59.540869 2572 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9gnmq_3a7e030d-fad8-480d-9dc6-fe5057048cf9/konnectivity-agent/0.log" limit=1024 Jul 16 01:08:59.695051 ip-10-0-143-200 kubenswrapper[2572]: I0716 01:08:59.694960 2572 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9gnmq_3a7e030d-fad8-480d-9dc6-fe5057048cf9/konnectivity-agent/0.log" limit=1024 Jul 16 01:09:05.772804 ip-10-0-143-200 kubenswrapper[2572]: I0716 01:09:05.772766 2572 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9gnmq_3a7e030d-fad8-480d-9dc6-fe5057048cf9/konnectivity-agent/0.log" limit=1024 Jul 16 01:09:05.866146 ip-10-0-143-200 kubenswrapper[2572]: I0716 01:09:05.866113 2572 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9gnmq_3a7e030d-fad8-480d-9dc6-fe5057048cf9/konnectivity-agent/0.log" limit=1024 Jul 16 01:09:13.376663 ip-10-0-143-200 kubenswrapper[2572]: I0716 01:09:13.376613 2572 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9gnmq_3a7e030d-fad8-480d-9dc6-fe5057048cf9/konnectivity-agent/0.log" limit=1024 Jul 16 01:09:13.477231 ip-10-0-143-200 kubenswrapper[2572]: I0716 01:09:13.477194 2572 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9gnmq_3a7e030d-fad8-480d-9dc6-fe5057048cf9/konnectivity-agent/0.log" limit=1024 Jul 16 01:09:13.578190 ip-10-0-143-200 kubenswrapper[2572]: I0716 01:09:13.578162 2572 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9gnmq_3a7e030d-fad8-480d-9dc6-fe5057048cf9/konnectivity-agent/0.log" limit=1024 Jul 16 01:09:14.373254 ip-10-0-143-200 kubenswrapper[2572]: I0716 01:09:14.373221 2572 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9gnmq_3a7e030d-fad8-480d-9dc6-fe5057048cf9/konnectivity-agent/0.log" limit=1024 Jul 16 01:09:14.479437 ip-10-0-143-200 kubenswrapper[2572]: I0716 01:09:14.479401 2572 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9gnmq_3a7e030d-fad8-480d-9dc6-fe5057048cf9/konnectivity-agent/0.log" limit=1024 Jul 16 01:09:14.571406 ip-10-0-143-200 kubenswrapper[2572]: I0716 01:09:14.571370 2572 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9gnmq_3a7e030d-fad8-480d-9dc6-fe5057048cf9/konnectivity-agent/0.log" limit=1024 Jul 16 01:09:15.927562 ip-10-0-143-200 kubenswrapper[2572]: I0716 01:09:15.927524 2572 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9gnmq_3a7e030d-fad8-480d-9dc6-fe5057048cf9/konnectivity-agent/0.log" limit=1024 Jul 16 01:09:17.571063 ip-10-0-143-200 kubenswrapper[2572]: I0716 01:09:17.571030 2572 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9gnmq_3a7e030d-fad8-480d-9dc6-fe5057048cf9/konnectivity-agent/0.log" limit=1024 Jul 16 01:09:17.666908 ip-10-0-143-200 kubenswrapper[2572]: I0716 01:09:17.666876 2572 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9gnmq_3a7e030d-fad8-480d-9dc6-fe5057048cf9/konnectivity-agent/0.log" limit=1024 Jul 16 01:09:21.974449 ip-10-0-143-200 kubenswrapper[2572]: I0716 01:09:21.974419 2572 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9gnmq_3a7e030d-fad8-480d-9dc6-fe5057048cf9/konnectivity-agent/0.log" limit=1024 Jul 16 01:09:22.044446 ip-10-0-143-200 kubenswrapper[2572]: I0716 01:09:22.044409 2572 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9gnmq_3a7e030d-fad8-480d-9dc6-fe5057048cf9/konnectivity-agent/0.log" limit=1024 Jul 16 01:09:22.664255 ip-10-0-143-200 kubenswrapper[2572]: I0716 01:09:22.664207 2572 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9gnmq_3a7e030d-fad8-480d-9dc6-fe5057048cf9/konnectivity-agent/0.log" limit=1024 Jul 16 01:09:28.791319 ip-10-0-143-200 kubenswrapper[2572]: I0716 01:09:28.791266 2572 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9gnmq_3a7e030d-fad8-480d-9dc6-fe5057048cf9/konnectivity-agent/0.log" limit=1024 Jul 16 01:09:28.859303 ip-10-0-143-200 kubenswrapper[2572]: I0716 01:09:28.859256 2572 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9gnmq_3a7e030d-fad8-480d-9dc6-fe5057048cf9/konnectivity-agent/0.log" limit=1024 Jul 16 01:09:28.973020 ip-10-0-143-200 kubenswrapper[2572]: I0716 01:09:28.972992 2572 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9gnmq_3a7e030d-fad8-480d-9dc6-fe5057048cf9/konnectivity-agent/0.log" limit=1024 Jul 16 01:09:39.511006 ip-10-0-143-200 kubenswrapper[2572]: I0716 01:09:39.510972 2572 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9gnmq_3a7e030d-fad8-480d-9dc6-fe5057048cf9/konnectivity-agent/0.log" limit=1024 Jul 16 01:09:39.842769 ip-10-0-143-200 kubenswrapper[2572]: I0716 01:09:39.842664 2572 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9gnmq_3a7e030d-fad8-480d-9dc6-fe5057048cf9/konnectivity-agent/0.log" limit=1024 Jul 16 01:09:55.496383 ip-10-0-143-200 kubenswrapper[2572]: I0716 01:09:55.496351 2572 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-console-operator_console-operator-7866497494-pjc6j_a64bcd05-8596-4c12-b88e-e974d676e25f/console-operator/2.log" Jul 16 01:09:55.501260 ip-10-0-143-200 kubenswrapper[2572]: I0716 01:09:55.501236 2572 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ovn-kubernetes_ovnkube-node-fp78g_aa66f2b6-0ad8-45d0-8ff5-40fc21966b5a/ovn-acl-logging/0.log" Jul 16 01:09:55.503662 ip-10-0-143-200 kubenswrapper[2572]: I0716 01:09:55.503644 2572 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-console-operator_console-operator-7866497494-pjc6j_a64bcd05-8596-4c12-b88e-e974d676e25f/console-operator/2.log" Jul 16 01:09:55.508396 ip-10-0-143-200 kubenswrapper[2572]: I0716 01:09:55.508376 2572 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ovn-kubernetes_ovnkube-node-fp78g_aa66f2b6-0ad8-45d0-8ff5-40fc21966b5a/ovn-acl-logging/0.log" Jul 16 01:10:00.824298 ip-10-0-143-200 kubenswrapper[2572]: I0716 01:10:00.824259 2572 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9gnmq_3a7e030d-fad8-480d-9dc6-fe5057048cf9/konnectivity-agent/0.log" limit=1024 Jul 16 01:10:07.176458 ip-10-0-143-200 kubenswrapper[2572]: I0716 01:10:07.176423 2572 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9gnmq_3a7e030d-fad8-480d-9dc6-fe5057048cf9/konnectivity-agent/0.log" limit=1024 Jul 16 01:10:07.305957 ip-10-0-143-200 kubenswrapper[2572]: I0716 01:10:07.305927 2572 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9gnmq_3a7e030d-fad8-480d-9dc6-fe5057048cf9/konnectivity-agent/0.log" limit=1024 Jul 16 01:10:07.542973 ip-10-0-143-200 kubenswrapper[2572]: I0716 01:10:07.542941 2572 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9gnmq_3a7e030d-fad8-480d-9dc6-fe5057048cf9/konnectivity-agent/0.log" limit=1024 Jul 16 01:10:07.879510 ip-10-0-143-200 kubenswrapper[2572]: I0716 01:10:07.879418 2572 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-ingress/e2e-inf-a-7d5e85-openshift-default-6b7d58d6b5-tk5sq"] Jul 16 01:10:07.879988 ip-10-0-143-200 kubenswrapper[2572]: I0716 01:10:07.879966 2572 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="e0d9d180-cdd0-442f-a220-f365b8b74b24" containerName="istio-proxy" Jul 16 01:10:07.880085 ip-10-0-143-200 kubenswrapper[2572]: I0716 01:10:07.879991 2572 state_mem.go:107] "Deleted CPUSet assignment" podUID="e0d9d180-cdd0-442f-a220-f365b8b74b24" containerName="istio-proxy" Jul 16 01:10:07.880085 ip-10-0-143-200 kubenswrapper[2572]: I0716 01:10:07.880012 2572 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="e0d9d180-cdd0-442f-a220-f365b8b74b24" containerName="istio-proxy" Jul 16 01:10:07.880085 ip-10-0-143-200 kubenswrapper[2572]: I0716 01:10:07.880020 2572 state_mem.go:107] "Deleted CPUSet assignment" podUID="e0d9d180-cdd0-442f-a220-f365b8b74b24" containerName="istio-proxy" Jul 16 01:10:07.880232 ip-10-0-143-200 kubenswrapper[2572]: I0716 01:10:07.880153 2572 memory_manager.go:368] "RemoveStaleState removing state" podUID="e0d9d180-cdd0-442f-a220-f365b8b74b24" containerName="istio-proxy" Jul 16 01:10:07.880232 ip-10-0-143-200 kubenswrapper[2572]: I0716 01:10:07.880171 2572 memory_manager.go:368] "RemoveStaleState removing state" podUID="e0d9d180-cdd0-442f-a220-f365b8b74b24" containerName="istio-proxy" Jul 16 01:10:07.883471 ip-10-0-143-200 kubenswrapper[2572]: I0716 01:10:07.883448 2572 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-inf-a-7d5e85-openshift-default-6b7d58d6b5-tk5sq" Jul 16 01:10:07.890696 ip-10-0-143-200 kubenswrapper[2572]: I0716 01:10:07.890261 2572 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress\"/\"e2e-inf-a-7d5e85-openshift-default-dockercfg-pkzr9\"" Jul 16 01:10:07.901442 ip-10-0-143-200 kubenswrapper[2572]: I0716 01:10:07.901417 2572 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/e2e-inf-a-7d5e85-openshift-default-6b7d58d6b5-tk5sq"] Jul 16 01:10:07.907554 ip-10-0-143-200 kubenswrapper[2572]: I0716 01:10:07.907530 2572 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9gnmq_3a7e030d-fad8-480d-9dc6-fe5057048cf9/konnectivity-agent/0.log" limit=1024 Jul 16 01:10:07.961892 ip-10-0-143-200 kubenswrapper[2572]: I0716 01:10:07.961816 2572 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/a505deae-c477-4c11-8f18-ac8f1dc02b57-istiod-ca-cert\") pod \"e2e-inf-a-7d5e85-openshift-default-6b7d58d6b5-tk5sq\" (UID: \"a505deae-c477-4c11-8f18-ac8f1dc02b57\") " pod="openshift-ingress/e2e-inf-a-7d5e85-openshift-default-6b7d58d6b5-tk5sq" Jul 16 01:10:07.962106 ip-10-0-143-200 kubenswrapper[2572]: I0716 01:10:07.961946 2572 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/a505deae-c477-4c11-8f18-ac8f1dc02b57-workload-certs\") pod \"e2e-inf-a-7d5e85-openshift-default-6b7d58d6b5-tk5sq\" (UID: \"a505deae-c477-4c11-8f18-ac8f1dc02b57\") " pod="openshift-ingress/e2e-inf-a-7d5e85-openshift-default-6b7d58d6b5-tk5sq" Jul 16 01:10:07.962106 ip-10-0-143-200 kubenswrapper[2572]: I0716 01:10:07.962057 2572 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/a505deae-c477-4c11-8f18-ac8f1dc02b57-credential-socket\") pod \"e2e-inf-a-7d5e85-openshift-default-6b7d58d6b5-tk5sq\" (UID: \"a505deae-c477-4c11-8f18-ac8f1dc02b57\") " pod="openshift-ingress/e2e-inf-a-7d5e85-openshift-default-6b7d58d6b5-tk5sq" Jul 16 01:10:07.962216 ip-10-0-143-200 kubenswrapper[2572]: I0716 01:10:07.962114 2572 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/a505deae-c477-4c11-8f18-ac8f1dc02b57-istio-token\") pod \"e2e-inf-a-7d5e85-openshift-default-6b7d58d6b5-tk5sq\" (UID: \"a505deae-c477-4c11-8f18-ac8f1dc02b57\") " pod="openshift-ingress/e2e-inf-a-7d5e85-openshift-default-6b7d58d6b5-tk5sq" Jul 16 01:10:07.962216 ip-10-0-143-200 kubenswrapper[2572]: I0716 01:10:07.962146 2572 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/a505deae-c477-4c11-8f18-ac8f1dc02b57-workload-socket\") pod \"e2e-inf-a-7d5e85-openshift-default-6b7d58d6b5-tk5sq\" (UID: \"a505deae-c477-4c11-8f18-ac8f1dc02b57\") " pod="openshift-ingress/e2e-inf-a-7d5e85-openshift-default-6b7d58d6b5-tk5sq" Jul 16 01:10:07.962216 ip-10-0-143-200 kubenswrapper[2572]: I0716 01:10:07.962205 2572 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/a505deae-c477-4c11-8f18-ac8f1dc02b57-istio-data\") pod \"e2e-inf-a-7d5e85-openshift-default-6b7d58d6b5-tk5sq\" (UID: \"a505deae-c477-4c11-8f18-ac8f1dc02b57\") " pod="openshift-ingress/e2e-inf-a-7d5e85-openshift-default-6b7d58d6b5-tk5sq" Jul 16 01:10:07.962451 ip-10-0-143-200 kubenswrapper[2572]: I0716 01:10:07.962262 2572 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/a505deae-c477-4c11-8f18-ac8f1dc02b57-istio-envoy\") pod \"e2e-inf-a-7d5e85-openshift-default-6b7d58d6b5-tk5sq\" (UID: \"a505deae-c477-4c11-8f18-ac8f1dc02b57\") " pod="openshift-ingress/e2e-inf-a-7d5e85-openshift-default-6b7d58d6b5-tk5sq" Jul 16 01:10:07.962451 ip-10-0-143-200 kubenswrapper[2572]: I0716 01:10:07.962295 2572 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/a505deae-c477-4c11-8f18-ac8f1dc02b57-istio-podinfo\") pod \"e2e-inf-a-7d5e85-openshift-default-6b7d58d6b5-tk5sq\" (UID: \"a505deae-c477-4c11-8f18-ac8f1dc02b57\") " pod="openshift-ingress/e2e-inf-a-7d5e85-openshift-default-6b7d58d6b5-tk5sq" Jul 16 01:10:07.962451 ip-10-0-143-200 kubenswrapper[2572]: I0716 01:10:07.962353 2572 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-nttkz\" (UniqueName: \"kubernetes.io/projected/a505deae-c477-4c11-8f18-ac8f1dc02b57-kube-api-access-nttkz\") pod \"e2e-inf-a-7d5e85-openshift-default-6b7d58d6b5-tk5sq\" (UID: \"a505deae-c477-4c11-8f18-ac8f1dc02b57\") " pod="openshift-ingress/e2e-inf-a-7d5e85-openshift-default-6b7d58d6b5-tk5sq" Jul 16 01:10:08.063929 ip-10-0-143-200 kubenswrapper[2572]: I0716 01:10:08.063890 2572 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/a505deae-c477-4c11-8f18-ac8f1dc02b57-istiod-ca-cert\") pod \"e2e-inf-a-7d5e85-openshift-default-6b7d58d6b5-tk5sq\" (UID: \"a505deae-c477-4c11-8f18-ac8f1dc02b57\") " pod="openshift-ingress/e2e-inf-a-7d5e85-openshift-default-6b7d58d6b5-tk5sq" Jul 16 01:10:08.064133 ip-10-0-143-200 kubenswrapper[2572]: I0716 01:10:08.063950 2572 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/a505deae-c477-4c11-8f18-ac8f1dc02b57-workload-certs\") pod \"e2e-inf-a-7d5e85-openshift-default-6b7d58d6b5-tk5sq\" (UID: \"a505deae-c477-4c11-8f18-ac8f1dc02b57\") " pod="openshift-ingress/e2e-inf-a-7d5e85-openshift-default-6b7d58d6b5-tk5sq" Jul 16 01:10:08.064133 ip-10-0-143-200 kubenswrapper[2572]: I0716 01:10:08.063978 2572 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/a505deae-c477-4c11-8f18-ac8f1dc02b57-credential-socket\") pod \"e2e-inf-a-7d5e85-openshift-default-6b7d58d6b5-tk5sq\" (UID: \"a505deae-c477-4c11-8f18-ac8f1dc02b57\") " pod="openshift-ingress/e2e-inf-a-7d5e85-openshift-default-6b7d58d6b5-tk5sq" Jul 16 01:10:08.064133 ip-10-0-143-200 kubenswrapper[2572]: I0716 01:10:08.064003 2572 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/a505deae-c477-4c11-8f18-ac8f1dc02b57-istio-token\") pod \"e2e-inf-a-7d5e85-openshift-default-6b7d58d6b5-tk5sq\" (UID: \"a505deae-c477-4c11-8f18-ac8f1dc02b57\") " pod="openshift-ingress/e2e-inf-a-7d5e85-openshift-default-6b7d58d6b5-tk5sq" Jul 16 01:10:08.064133 ip-10-0-143-200 kubenswrapper[2572]: I0716 01:10:08.064026 2572 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/a505deae-c477-4c11-8f18-ac8f1dc02b57-workload-socket\") pod \"e2e-inf-a-7d5e85-openshift-default-6b7d58d6b5-tk5sq\" (UID: \"a505deae-c477-4c11-8f18-ac8f1dc02b57\") " pod="openshift-ingress/e2e-inf-a-7d5e85-openshift-default-6b7d58d6b5-tk5sq" Jul 16 01:10:08.064133 ip-10-0-143-200 kubenswrapper[2572]: I0716 01:10:08.064063 2572 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/a505deae-c477-4c11-8f18-ac8f1dc02b57-istio-data\") pod \"e2e-inf-a-7d5e85-openshift-default-6b7d58d6b5-tk5sq\" (UID: \"a505deae-c477-4c11-8f18-ac8f1dc02b57\") " pod="openshift-ingress/e2e-inf-a-7d5e85-openshift-default-6b7d58d6b5-tk5sq" Jul 16 01:10:08.064133 ip-10-0-143-200 kubenswrapper[2572]: I0716 01:10:08.064110 2572 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/a505deae-c477-4c11-8f18-ac8f1dc02b57-istio-envoy\") pod \"e2e-inf-a-7d5e85-openshift-default-6b7d58d6b5-tk5sq\" (UID: \"a505deae-c477-4c11-8f18-ac8f1dc02b57\") " pod="openshift-ingress/e2e-inf-a-7d5e85-openshift-default-6b7d58d6b5-tk5sq" Jul 16 01:10:08.064438 ip-10-0-143-200 kubenswrapper[2572]: I0716 01:10:08.064141 2572 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/a505deae-c477-4c11-8f18-ac8f1dc02b57-istio-podinfo\") pod \"e2e-inf-a-7d5e85-openshift-default-6b7d58d6b5-tk5sq\" (UID: \"a505deae-c477-4c11-8f18-ac8f1dc02b57\") " pod="openshift-ingress/e2e-inf-a-7d5e85-openshift-default-6b7d58d6b5-tk5sq" Jul 16 01:10:08.064561 ip-10-0-143-200 kubenswrapper[2572]: I0716 01:10:08.064530 2572 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-nttkz\" (UniqueName: \"kubernetes.io/projected/a505deae-c477-4c11-8f18-ac8f1dc02b57-kube-api-access-nttkz\") pod \"e2e-inf-a-7d5e85-openshift-default-6b7d58d6b5-tk5sq\" (UID: \"a505deae-c477-4c11-8f18-ac8f1dc02b57\") " pod="openshift-ingress/e2e-inf-a-7d5e85-openshift-default-6b7d58d6b5-tk5sq" Jul 16 01:10:08.064681 ip-10-0-143-200 kubenswrapper[2572]: I0716 01:10:08.064608 2572 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/a505deae-c477-4c11-8f18-ac8f1dc02b57-credential-socket\") pod \"e2e-inf-a-7d5e85-openshift-default-6b7d58d6b5-tk5sq\" (UID: \"a505deae-c477-4c11-8f18-ac8f1dc02b57\") " pod="openshift-ingress/e2e-inf-a-7d5e85-openshift-default-6b7d58d6b5-tk5sq" Jul 16 01:10:08.064681 ip-10-0-143-200 kubenswrapper[2572]: I0716 01:10:08.064543 2572 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/a505deae-c477-4c11-8f18-ac8f1dc02b57-workload-certs\") pod \"e2e-inf-a-7d5e85-openshift-default-6b7d58d6b5-tk5sq\" (UID: \"a505deae-c477-4c11-8f18-ac8f1dc02b57\") " pod="openshift-ingress/e2e-inf-a-7d5e85-openshift-default-6b7d58d6b5-tk5sq" Jul 16 01:10:08.064681 ip-10-0-143-200 kubenswrapper[2572]: I0716 01:10:08.064446 2572 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/a505deae-c477-4c11-8f18-ac8f1dc02b57-workload-socket\") pod \"e2e-inf-a-7d5e85-openshift-default-6b7d58d6b5-tk5sq\" (UID: \"a505deae-c477-4c11-8f18-ac8f1dc02b57\") " pod="openshift-ingress/e2e-inf-a-7d5e85-openshift-default-6b7d58d6b5-tk5sq" Jul 16 01:10:08.064862 ip-10-0-143-200 kubenswrapper[2572]: I0716 01:10:08.064697 2572 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/a505deae-c477-4c11-8f18-ac8f1dc02b57-istio-data\") pod \"e2e-inf-a-7d5e85-openshift-default-6b7d58d6b5-tk5sq\" (UID: \"a505deae-c477-4c11-8f18-ac8f1dc02b57\") " pod="openshift-ingress/e2e-inf-a-7d5e85-openshift-default-6b7d58d6b5-tk5sq" Jul 16 01:10:08.064862 ip-10-0-143-200 kubenswrapper[2572]: I0716 01:10:08.064770 2572 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/a505deae-c477-4c11-8f18-ac8f1dc02b57-istiod-ca-cert\") pod \"e2e-inf-a-7d5e85-openshift-default-6b7d58d6b5-tk5sq\" (UID: \"a505deae-c477-4c11-8f18-ac8f1dc02b57\") " pod="openshift-ingress/e2e-inf-a-7d5e85-openshift-default-6b7d58d6b5-tk5sq" Jul 16 01:10:08.066567 ip-10-0-143-200 kubenswrapper[2572]: I0716 01:10:08.066547 2572 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/a505deae-c477-4c11-8f18-ac8f1dc02b57-istio-envoy\") pod \"e2e-inf-a-7d5e85-openshift-default-6b7d58d6b5-tk5sq\" (UID: \"a505deae-c477-4c11-8f18-ac8f1dc02b57\") " pod="openshift-ingress/e2e-inf-a-7d5e85-openshift-default-6b7d58d6b5-tk5sq" Jul 16 01:10:08.066928 ip-10-0-143-200 kubenswrapper[2572]: I0716 01:10:08.066910 2572 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/a505deae-c477-4c11-8f18-ac8f1dc02b57-istio-podinfo\") pod \"e2e-inf-a-7d5e85-openshift-default-6b7d58d6b5-tk5sq\" (UID: \"a505deae-c477-4c11-8f18-ac8f1dc02b57\") " pod="openshift-ingress/e2e-inf-a-7d5e85-openshift-default-6b7d58d6b5-tk5sq" Jul 16 01:10:08.074351 ip-10-0-143-200 kubenswrapper[2572]: I0716 01:10:08.074327 2572 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/a505deae-c477-4c11-8f18-ac8f1dc02b57-istio-token\") pod \"e2e-inf-a-7d5e85-openshift-default-6b7d58d6b5-tk5sq\" (UID: \"a505deae-c477-4c11-8f18-ac8f1dc02b57\") " pod="openshift-ingress/e2e-inf-a-7d5e85-openshift-default-6b7d58d6b5-tk5sq" Jul 16 01:10:08.074684 ip-10-0-143-200 kubenswrapper[2572]: I0716 01:10:08.074665 2572 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-nttkz\" (UniqueName: \"kubernetes.io/projected/a505deae-c477-4c11-8f18-ac8f1dc02b57-kube-api-access-nttkz\") pod \"e2e-inf-a-7d5e85-openshift-default-6b7d58d6b5-tk5sq\" (UID: \"a505deae-c477-4c11-8f18-ac8f1dc02b57\") " pod="openshift-ingress/e2e-inf-a-7d5e85-openshift-default-6b7d58d6b5-tk5sq" Jul 16 01:10:08.191036 ip-10-0-143-200 kubenswrapper[2572]: I0716 01:10:08.190953 2572 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9gnmq_3a7e030d-fad8-480d-9dc6-fe5057048cf9/konnectivity-agent/0.log" limit=1024 Jul 16 01:10:08.194241 ip-10-0-143-200 kubenswrapper[2572]: I0716 01:10:08.194221 2572 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-inf-a-7d5e85-openshift-default-6b7d58d6b5-tk5sq" Jul 16 01:10:08.344995 ip-10-0-143-200 kubenswrapper[2572]: I0716 01:10:08.344857 2572 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/e2e-inf-a-7d5e85-openshift-default-6b7d58d6b5-tk5sq"] Jul 16 01:10:08.347484 ip-10-0-143-200 kubenswrapper[2572]: W0716 01:10:08.347436 2572 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-poda505deae_c477_4c11_8f18_ac8f1dc02b57.slice/crio-4059089df3fdef5d3769c0eedcaa83e523516e286fb7b6f49debf364459e4682 WatchSource:0}: Error finding container 4059089df3fdef5d3769c0eedcaa83e523516e286fb7b6f49debf364459e4682: Status 404 returned error can't find the container with id 4059089df3fdef5d3769c0eedcaa83e523516e286fb7b6f49debf364459e4682 Jul 16 01:10:08.349663 ip-10-0-143-200 kubenswrapper[2572]: I0716 01:10:08.349608 2572 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236228Ki","pods":"250"} Jul 16 01:10:08.349746 ip-10-0-143-200 kubenswrapper[2572]: I0716 01:10:08.349684 2572 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236228Ki","pods":"250"} Jul 16 01:10:08.349746 ip-10-0-143-200 kubenswrapper[2572]: I0716 01:10:08.349718 2572 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236228Ki","pods":"250"} Jul 16 01:10:08.708840 ip-10-0-143-200 kubenswrapper[2572]: I0716 01:10:08.708803 2572 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-inf-a-7d5e85-openshift-default-6b7d58d6b5-tk5sq" event={"ID":"a505deae-c477-4c11-8f18-ac8f1dc02b57","Type":"ContainerStarted","Data":"b5f7d803a646e97f3c3f3dc8eac6af365582374ab2dc873fa322c1b25912e34b"} Jul 16 01:10:08.709017 ip-10-0-143-200 kubenswrapper[2572]: I0716 01:10:08.708859 2572 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-inf-a-7d5e85-openshift-default-6b7d58d6b5-tk5sq" event={"ID":"a505deae-c477-4c11-8f18-ac8f1dc02b57","Type":"ContainerStarted","Data":"4059089df3fdef5d3769c0eedcaa83e523516e286fb7b6f49debf364459e4682"} Jul 16 01:10:08.732315 ip-10-0-143-200 kubenswrapper[2572]: I0716 01:10:08.732262 2572 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-ingress/e2e-inf-a-7d5e85-openshift-default-6b7d58d6b5-tk5sq" podStartSLOduration=1.7322452099999999 podStartE2EDuration="1.73224521s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-16 01:10:07 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-16 01:10:08.730307765 +0000 UTC m=+1813.839062433" watchObservedRunningTime="2026-07-16 01:10:08.73224521 +0000 UTC m=+1813.840999875" Jul 16 01:10:09.195082 ip-10-0-143-200 kubenswrapper[2572]: I0716 01:10:09.194997 2572 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="openshift-ingress/e2e-inf-a-7d5e85-openshift-default-6b7d58d6b5-tk5sq" Jul 16 01:10:09.200083 ip-10-0-143-200 kubenswrapper[2572]: I0716 01:10:09.200056 2572 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="openshift-ingress/e2e-inf-a-7d5e85-openshift-default-6b7d58d6b5-tk5sq" Jul 16 01:10:09.649970 ip-10-0-143-200 kubenswrapper[2572]: I0716 01:10:09.649932 2572 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9gnmq_3a7e030d-fad8-480d-9dc6-fe5057048cf9/konnectivity-agent/0.log" limit=1024 Jul 16 01:10:09.715271 ip-10-0-143-200 kubenswrapper[2572]: I0716 01:10:09.715229 2572 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/e2e-inf-a-7d5e85-openshift-default-6b7d58d6b5-tk5sq" Jul 16 01:10:09.716229 ip-10-0-143-200 kubenswrapper[2572]: I0716 01:10:09.716205 2572 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/e2e-inf-a-7d5e85-openshift-default-6b7d58d6b5-tk5sq" Jul 16 01:10:09.750256 ip-10-0-143-200 kubenswrapper[2572]: I0716 01:10:09.750221 2572 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9gnmq_3a7e030d-fad8-480d-9dc6-fe5057048cf9/konnectivity-agent/0.log" limit=1024 Jul 16 01:10:09.832298 ip-10-0-143-200 kubenswrapper[2572]: I0716 01:10:09.832261 2572 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9gnmq_3a7e030d-fad8-480d-9dc6-fe5057048cf9/konnectivity-agent/0.log" limit=1024 Jul 16 01:10:09.953336 ip-10-0-143-200 kubenswrapper[2572]: I0716 01:10:09.953304 2572 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9gnmq_3a7e030d-fad8-480d-9dc6-fe5057048cf9/konnectivity-agent/0.log" limit=1024 Jul 16 01:10:10.301489 ip-10-0-143-200 kubenswrapper[2572]: I0716 01:10:10.301400 2572 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9gnmq_3a7e030d-fad8-480d-9dc6-fe5057048cf9/konnectivity-agent/0.log" limit=1024 Jul 16 01:10:10.401104 ip-10-0-143-200 kubenswrapper[2572]: I0716 01:10:10.401071 2572 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9gnmq_3a7e030d-fad8-480d-9dc6-fe5057048cf9/konnectivity-agent/0.log" limit=1024 Jul 16 01:10:15.172257 ip-10-0-143-200 kubenswrapper[2572]: I0716 01:10:15.172223 2572 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9gnmq_3a7e030d-fad8-480d-9dc6-fe5057048cf9/konnectivity-agent/0.log" limit=1024 Jul 16 01:10:15.514774 ip-10-0-143-200 kubenswrapper[2572]: I0716 01:10:15.514719 2572 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9gnmq_3a7e030d-fad8-480d-9dc6-fe5057048cf9/konnectivity-agent/0.log" limit=1024 Jul 16 01:10:15.823403 ip-10-0-143-200 kubenswrapper[2572]: I0716 01:10:15.823309 2572 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-ingress/e2e-inf-b-e5854b-openshift-default-77bc9577d9-8qdwp"] Jul 16 01:10:15.827837 ip-10-0-143-200 kubenswrapper[2572]: I0716 01:10:15.827809 2572 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/e2e-inf-b-e5854b-openshift-default-77bc9577d9-8qdwp"] Jul 16 01:10:15.827993 ip-10-0-143-200 kubenswrapper[2572]: I0716 01:10:15.827930 2572 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-inf-b-e5854b-openshift-default-77bc9577d9-8qdwp" Jul 16 01:10:15.829201 ip-10-0-143-200 kubenswrapper[2572]: I0716 01:10:15.829141 2572 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9gnmq_3a7e030d-fad8-480d-9dc6-fe5057048cf9/konnectivity-agent/0.log" limit=1024 Jul 16 01:10:15.830575 ip-10-0-143-200 kubenswrapper[2572]: I0716 01:10:15.830553 2572 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress\"/\"e2e-inf-b-e5854b-openshift-default-dockercfg-2wl8x\"" Jul 16 01:10:15.844426 ip-10-0-143-200 kubenswrapper[2572]: I0716 01:10:15.844406 2572 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/301fffd1-42b5-4cfd-a97d-53a754a45110-istio-token\") pod \"e2e-inf-b-e5854b-openshift-default-77bc9577d9-8qdwp\" (UID: \"301fffd1-42b5-4cfd-a97d-53a754a45110\") " pod="openshift-ingress/e2e-inf-b-e5854b-openshift-default-77bc9577d9-8qdwp" Jul 16 01:10:15.844531 ip-10-0-143-200 kubenswrapper[2572]: I0716 01:10:15.844438 2572 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/301fffd1-42b5-4cfd-a97d-53a754a45110-workload-socket\") pod \"e2e-inf-b-e5854b-openshift-default-77bc9577d9-8qdwp\" (UID: \"301fffd1-42b5-4cfd-a97d-53a754a45110\") " pod="openshift-ingress/e2e-inf-b-e5854b-openshift-default-77bc9577d9-8qdwp" Jul 16 01:10:15.844531 ip-10-0-143-200 kubenswrapper[2572]: I0716 01:10:15.844457 2572 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-c7jpq\" (UniqueName: \"kubernetes.io/projected/301fffd1-42b5-4cfd-a97d-53a754a45110-kube-api-access-c7jpq\") pod \"e2e-inf-b-e5854b-openshift-default-77bc9577d9-8qdwp\" (UID: \"301fffd1-42b5-4cfd-a97d-53a754a45110\") " pod="openshift-ingress/e2e-inf-b-e5854b-openshift-default-77bc9577d9-8qdwp" Jul 16 01:10:15.844531 ip-10-0-143-200 kubenswrapper[2572]: I0716 01:10:15.844491 2572 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/301fffd1-42b5-4cfd-a97d-53a754a45110-istio-data\") pod \"e2e-inf-b-e5854b-openshift-default-77bc9577d9-8qdwp\" (UID: \"301fffd1-42b5-4cfd-a97d-53a754a45110\") " pod="openshift-ingress/e2e-inf-b-e5854b-openshift-default-77bc9577d9-8qdwp" Jul 16 01:10:15.844682 ip-10-0-143-200 kubenswrapper[2572]: I0716 01:10:15.844564 2572 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/301fffd1-42b5-4cfd-a97d-53a754a45110-istio-podinfo\") pod \"e2e-inf-b-e5854b-openshift-default-77bc9577d9-8qdwp\" (UID: \"301fffd1-42b5-4cfd-a97d-53a754a45110\") " pod="openshift-ingress/e2e-inf-b-e5854b-openshift-default-77bc9577d9-8qdwp" Jul 16 01:10:15.844682 ip-10-0-143-200 kubenswrapper[2572]: I0716 01:10:15.844610 2572 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/301fffd1-42b5-4cfd-a97d-53a754a45110-credential-socket\") pod \"e2e-inf-b-e5854b-openshift-default-77bc9577d9-8qdwp\" (UID: \"301fffd1-42b5-4cfd-a97d-53a754a45110\") " pod="openshift-ingress/e2e-inf-b-e5854b-openshift-default-77bc9577d9-8qdwp" Jul 16 01:10:15.844682 ip-10-0-143-200 kubenswrapper[2572]: I0716 01:10:15.844654 2572 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/301fffd1-42b5-4cfd-a97d-53a754a45110-istio-envoy\") pod \"e2e-inf-b-e5854b-openshift-default-77bc9577d9-8qdwp\" (UID: \"301fffd1-42b5-4cfd-a97d-53a754a45110\") " pod="openshift-ingress/e2e-inf-b-e5854b-openshift-default-77bc9577d9-8qdwp" Jul 16 01:10:15.844789 ip-10-0-143-200 kubenswrapper[2572]: I0716 01:10:15.844679 2572 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/301fffd1-42b5-4cfd-a97d-53a754a45110-istiod-ca-cert\") pod \"e2e-inf-b-e5854b-openshift-default-77bc9577d9-8qdwp\" (UID: \"301fffd1-42b5-4cfd-a97d-53a754a45110\") " pod="openshift-ingress/e2e-inf-b-e5854b-openshift-default-77bc9577d9-8qdwp" Jul 16 01:10:15.844789 ip-10-0-143-200 kubenswrapper[2572]: I0716 01:10:15.844734 2572 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/301fffd1-42b5-4cfd-a97d-53a754a45110-workload-certs\") pod \"e2e-inf-b-e5854b-openshift-default-77bc9577d9-8qdwp\" (UID: \"301fffd1-42b5-4cfd-a97d-53a754a45110\") " pod="openshift-ingress/e2e-inf-b-e5854b-openshift-default-77bc9577d9-8qdwp" Jul 16 01:10:15.945260 ip-10-0-143-200 kubenswrapper[2572]: I0716 01:10:15.945225 2572 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/301fffd1-42b5-4cfd-a97d-53a754a45110-istio-token\") pod \"e2e-inf-b-e5854b-openshift-default-77bc9577d9-8qdwp\" (UID: \"301fffd1-42b5-4cfd-a97d-53a754a45110\") " pod="openshift-ingress/e2e-inf-b-e5854b-openshift-default-77bc9577d9-8qdwp" Jul 16 01:10:15.945481 ip-10-0-143-200 kubenswrapper[2572]: I0716 01:10:15.945279 2572 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/301fffd1-42b5-4cfd-a97d-53a754a45110-workload-socket\") pod \"e2e-inf-b-e5854b-openshift-default-77bc9577d9-8qdwp\" (UID: \"301fffd1-42b5-4cfd-a97d-53a754a45110\") " pod="openshift-ingress/e2e-inf-b-e5854b-openshift-default-77bc9577d9-8qdwp" Jul 16 01:10:15.945481 ip-10-0-143-200 kubenswrapper[2572]: I0716 01:10:15.945306 2572 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-c7jpq\" (UniqueName: \"kubernetes.io/projected/301fffd1-42b5-4cfd-a97d-53a754a45110-kube-api-access-c7jpq\") pod \"e2e-inf-b-e5854b-openshift-default-77bc9577d9-8qdwp\" (UID: \"301fffd1-42b5-4cfd-a97d-53a754a45110\") " pod="openshift-ingress/e2e-inf-b-e5854b-openshift-default-77bc9577d9-8qdwp" Jul 16 01:10:15.945481 ip-10-0-143-200 kubenswrapper[2572]: I0716 01:10:15.945343 2572 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/301fffd1-42b5-4cfd-a97d-53a754a45110-istio-data\") pod \"e2e-inf-b-e5854b-openshift-default-77bc9577d9-8qdwp\" (UID: \"301fffd1-42b5-4cfd-a97d-53a754a45110\") " pod="openshift-ingress/e2e-inf-b-e5854b-openshift-default-77bc9577d9-8qdwp" Jul 16 01:10:15.945481 ip-10-0-143-200 kubenswrapper[2572]: I0716 01:10:15.945384 2572 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/301fffd1-42b5-4cfd-a97d-53a754a45110-istio-podinfo\") pod \"e2e-inf-b-e5854b-openshift-default-77bc9577d9-8qdwp\" (UID: \"301fffd1-42b5-4cfd-a97d-53a754a45110\") " pod="openshift-ingress/e2e-inf-b-e5854b-openshift-default-77bc9577d9-8qdwp" Jul 16 01:10:15.945481 ip-10-0-143-200 kubenswrapper[2572]: I0716 01:10:15.945416 2572 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/301fffd1-42b5-4cfd-a97d-53a754a45110-credential-socket\") pod \"e2e-inf-b-e5854b-openshift-default-77bc9577d9-8qdwp\" (UID: \"301fffd1-42b5-4cfd-a97d-53a754a45110\") " pod="openshift-ingress/e2e-inf-b-e5854b-openshift-default-77bc9577d9-8qdwp" Jul 16 01:10:15.945481 ip-10-0-143-200 kubenswrapper[2572]: I0716 01:10:15.945439 2572 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/301fffd1-42b5-4cfd-a97d-53a754a45110-istio-envoy\") pod \"e2e-inf-b-e5854b-openshift-default-77bc9577d9-8qdwp\" (UID: \"301fffd1-42b5-4cfd-a97d-53a754a45110\") " pod="openshift-ingress/e2e-inf-b-e5854b-openshift-default-77bc9577d9-8qdwp" Jul 16 01:10:15.945481 ip-10-0-143-200 kubenswrapper[2572]: I0716 01:10:15.945463 2572 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/301fffd1-42b5-4cfd-a97d-53a754a45110-istiod-ca-cert\") pod \"e2e-inf-b-e5854b-openshift-default-77bc9577d9-8qdwp\" (UID: \"301fffd1-42b5-4cfd-a97d-53a754a45110\") " pod="openshift-ingress/e2e-inf-b-e5854b-openshift-default-77bc9577d9-8qdwp" Jul 16 01:10:15.945833 ip-10-0-143-200 kubenswrapper[2572]: I0716 01:10:15.945515 2572 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/301fffd1-42b5-4cfd-a97d-53a754a45110-workload-certs\") pod \"e2e-inf-b-e5854b-openshift-default-77bc9577d9-8qdwp\" (UID: \"301fffd1-42b5-4cfd-a97d-53a754a45110\") " pod="openshift-ingress/e2e-inf-b-e5854b-openshift-default-77bc9577d9-8qdwp" Jul 16 01:10:15.945833 ip-10-0-143-200 kubenswrapper[2572]: I0716 01:10:15.945694 2572 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/301fffd1-42b5-4cfd-a97d-53a754a45110-workload-socket\") pod \"e2e-inf-b-e5854b-openshift-default-77bc9577d9-8qdwp\" (UID: \"301fffd1-42b5-4cfd-a97d-53a754a45110\") " pod="openshift-ingress/e2e-inf-b-e5854b-openshift-default-77bc9577d9-8qdwp" Jul 16 01:10:15.945960 ip-10-0-143-200 kubenswrapper[2572]: I0716 01:10:15.945933 2572 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/301fffd1-42b5-4cfd-a97d-53a754a45110-credential-socket\") pod \"e2e-inf-b-e5854b-openshift-default-77bc9577d9-8qdwp\" (UID: \"301fffd1-42b5-4cfd-a97d-53a754a45110\") " pod="openshift-ingress/e2e-inf-b-e5854b-openshift-default-77bc9577d9-8qdwp" Jul 16 01:10:15.946022 ip-10-0-143-200 kubenswrapper[2572]: I0716 01:10:15.945959 2572 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/301fffd1-42b5-4cfd-a97d-53a754a45110-istio-data\") pod \"e2e-inf-b-e5854b-openshift-default-77bc9577d9-8qdwp\" (UID: \"301fffd1-42b5-4cfd-a97d-53a754a45110\") " pod="openshift-ingress/e2e-inf-b-e5854b-openshift-default-77bc9577d9-8qdwp" Jul 16 01:10:15.946022 ip-10-0-143-200 kubenswrapper[2572]: I0716 01:10:15.945987 2572 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/301fffd1-42b5-4cfd-a97d-53a754a45110-workload-certs\") pod \"e2e-inf-b-e5854b-openshift-default-77bc9577d9-8qdwp\" (UID: \"301fffd1-42b5-4cfd-a97d-53a754a45110\") " pod="openshift-ingress/e2e-inf-b-e5854b-openshift-default-77bc9577d9-8qdwp" Jul 16 01:10:15.946130 ip-10-0-143-200 kubenswrapper[2572]: I0716 01:10:15.946030 2572 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/301fffd1-42b5-4cfd-a97d-53a754a45110-istiod-ca-cert\") pod \"e2e-inf-b-e5854b-openshift-default-77bc9577d9-8qdwp\" (UID: \"301fffd1-42b5-4cfd-a97d-53a754a45110\") " pod="openshift-ingress/e2e-inf-b-e5854b-openshift-default-77bc9577d9-8qdwp" Jul 16 01:10:15.948074 ip-10-0-143-200 kubenswrapper[2572]: I0716 01:10:15.948045 2572 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/301fffd1-42b5-4cfd-a97d-53a754a45110-istio-envoy\") pod \"e2e-inf-b-e5854b-openshift-default-77bc9577d9-8qdwp\" (UID: \"301fffd1-42b5-4cfd-a97d-53a754a45110\") " pod="openshift-ingress/e2e-inf-b-e5854b-openshift-default-77bc9577d9-8qdwp" Jul 16 01:10:15.948184 ip-10-0-143-200 kubenswrapper[2572]: I0716 01:10:15.948160 2572 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/301fffd1-42b5-4cfd-a97d-53a754a45110-istio-podinfo\") pod \"e2e-inf-b-e5854b-openshift-default-77bc9577d9-8qdwp\" (UID: \"301fffd1-42b5-4cfd-a97d-53a754a45110\") " pod="openshift-ingress/e2e-inf-b-e5854b-openshift-default-77bc9577d9-8qdwp" Jul 16 01:10:15.953439 ip-10-0-143-200 kubenswrapper[2572]: I0716 01:10:15.953413 2572 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/301fffd1-42b5-4cfd-a97d-53a754a45110-istio-token\") pod \"e2e-inf-b-e5854b-openshift-default-77bc9577d9-8qdwp\" (UID: \"301fffd1-42b5-4cfd-a97d-53a754a45110\") " pod="openshift-ingress/e2e-inf-b-e5854b-openshift-default-77bc9577d9-8qdwp" Jul 16 01:10:15.953545 ip-10-0-143-200 kubenswrapper[2572]: I0716 01:10:15.953515 2572 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-c7jpq\" (UniqueName: \"kubernetes.io/projected/301fffd1-42b5-4cfd-a97d-53a754a45110-kube-api-access-c7jpq\") pod \"e2e-inf-b-e5854b-openshift-default-77bc9577d9-8qdwp\" (UID: \"301fffd1-42b5-4cfd-a97d-53a754a45110\") " pod="openshift-ingress/e2e-inf-b-e5854b-openshift-default-77bc9577d9-8qdwp" Jul 16 01:10:16.119576 ip-10-0-143-200 kubenswrapper[2572]: I0716 01:10:16.119484 2572 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9gnmq_3a7e030d-fad8-480d-9dc6-fe5057048cf9/konnectivity-agent/0.log" limit=1024 Jul 16 01:10:16.140608 ip-10-0-143-200 kubenswrapper[2572]: I0716 01:10:16.140581 2572 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-inf-b-e5854b-openshift-default-77bc9577d9-8qdwp" Jul 16 01:10:16.347756 ip-10-0-143-200 kubenswrapper[2572]: I0716 01:10:16.347732 2572 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/e2e-inf-b-e5854b-openshift-default-77bc9577d9-8qdwp"] Jul 16 01:10:16.349748 ip-10-0-143-200 kubenswrapper[2572]: W0716 01:10:16.349716 2572 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod301fffd1_42b5_4cfd_a97d_53a754a45110.slice/crio-68e00eefc46a5c043efa876084f3455d646d8c325a276017e0abe0a093706c54 WatchSource:0}: Error finding container 68e00eefc46a5c043efa876084f3455d646d8c325a276017e0abe0a093706c54: Status 404 returned error can't find the container with id 68e00eefc46a5c043efa876084f3455d646d8c325a276017e0abe0a093706c54 Jul 16 01:10:16.351945 ip-10-0-143-200 kubenswrapper[2572]: I0716 01:10:16.351912 2572 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236228Ki","pods":"250"} Jul 16 01:10:16.352044 ip-10-0-143-200 kubenswrapper[2572]: I0716 01:10:16.351976 2572 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236228Ki","pods":"250"} Jul 16 01:10:16.352044 ip-10-0-143-200 kubenswrapper[2572]: I0716 01:10:16.352004 2572 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236228Ki","pods":"250"} Jul 16 01:10:16.753815 ip-10-0-143-200 kubenswrapper[2572]: I0716 01:10:16.753779 2572 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-inf-b-e5854b-openshift-default-77bc9577d9-8qdwp" event={"ID":"301fffd1-42b5-4cfd-a97d-53a754a45110","Type":"ContainerStarted","Data":"09ee4e1110f8f4521d6a38d1ac8b9e163a433ce54997cc7a139ba67c0247d056"} Jul 16 01:10:16.753992 ip-10-0-143-200 kubenswrapper[2572]: I0716 01:10:16.753836 2572 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-inf-b-e5854b-openshift-default-77bc9577d9-8qdwp" event={"ID":"301fffd1-42b5-4cfd-a97d-53a754a45110","Type":"ContainerStarted","Data":"68e00eefc46a5c043efa876084f3455d646d8c325a276017e0abe0a093706c54"} Jul 16 01:10:16.780158 ip-10-0-143-200 kubenswrapper[2572]: I0716 01:10:16.780107 2572 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-ingress/e2e-inf-b-e5854b-openshift-default-77bc9577d9-8qdwp" podStartSLOduration=1.780092424 podStartE2EDuration="1.780092424s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-16 01:10:15 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-16 01:10:16.777452238 +0000 UTC m=+1821.886206914" watchObservedRunningTime="2026-07-16 01:10:16.780092424 +0000 UTC m=+1821.888847155" Jul 16 01:10:17.141356 ip-10-0-143-200 kubenswrapper[2572]: I0716 01:10:17.141251 2572 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="openshift-ingress/e2e-inf-b-e5854b-openshift-default-77bc9577d9-8qdwp" Jul 16 01:10:17.143202 ip-10-0-143-200 kubenswrapper[2572]: I0716 01:10:17.143173 2572 patch_prober.go:28] interesting pod/e2e-inf-b-e5854b-openshift-default-77bc9577d9-8qdwp container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.132.0.58:15021/healthz/ready\": dial tcp 10.132.0.58:15021: connect: connection refused" start-of-body= Jul 16 01:10:17.143364 ip-10-0-143-200 kubenswrapper[2572]: I0716 01:10:17.143231 2572 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-inf-b-e5854b-openshift-default-77bc9577d9-8qdwp" podUID="301fffd1-42b5-4cfd-a97d-53a754a45110" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.132.0.58:15021/healthz/ready\": dial tcp 10.132.0.58:15021: connect: connection refused" Jul 16 01:10:17.816106 ip-10-0-143-200 kubenswrapper[2572]: I0716 01:10:17.816062 2572 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9gnmq_3a7e030d-fad8-480d-9dc6-fe5057048cf9/konnectivity-agent/0.log" limit=1024 Jul 16 01:10:17.937033 ip-10-0-143-200 kubenswrapper[2572]: I0716 01:10:17.936999 2572 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9gnmq_3a7e030d-fad8-480d-9dc6-fe5057048cf9/konnectivity-agent/0.log" limit=1024 Jul 16 01:10:18.008195 ip-10-0-143-200 kubenswrapper[2572]: I0716 01:10:18.008151 2572 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9gnmq_3a7e030d-fad8-480d-9dc6-fe5057048cf9/konnectivity-agent/0.log" limit=1024 Jul 16 01:10:18.145848 ip-10-0-143-200 kubenswrapper[2572]: I0716 01:10:18.145752 2572 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="openshift-ingress/e2e-inf-b-e5854b-openshift-default-77bc9577d9-8qdwp" Jul 16 01:10:18.364705 ip-10-0-143-200 kubenswrapper[2572]: I0716 01:10:18.364663 2572 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9gnmq_3a7e030d-fad8-480d-9dc6-fe5057048cf9/konnectivity-agent/0.log" limit=1024 Jul 16 01:10:18.508587 ip-10-0-143-200 kubenswrapper[2572]: I0716 01:10:18.508550 2572 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9gnmq_3a7e030d-fad8-480d-9dc6-fe5057048cf9/konnectivity-agent/0.log" limit=1024 Jul 16 01:10:18.607374 ip-10-0-143-200 kubenswrapper[2572]: I0716 01:10:18.607329 2572 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9gnmq_3a7e030d-fad8-480d-9dc6-fe5057048cf9/konnectivity-agent/0.log" limit=1024 Jul 16 01:10:18.768213 ip-10-0-143-200 kubenswrapper[2572]: I0716 01:10:18.768122 2572 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/e2e-inf-b-e5854b-openshift-default-77bc9577d9-8qdwp" Jul 16 01:10:18.769121 ip-10-0-143-200 kubenswrapper[2572]: I0716 01:10:18.769099 2572 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/e2e-inf-b-e5854b-openshift-default-77bc9577d9-8qdwp" Jul 16 01:10:20.780963 ip-10-0-143-200 kubenswrapper[2572]: I0716 01:10:20.780928 2572 generic.go:356] "Generic (PLEG): container finished" podID="a505deae-c477-4c11-8f18-ac8f1dc02b57" containerID="b5f7d803a646e97f3c3f3dc8eac6af365582374ab2dc873fa322c1b25912e34b" exitCode=0 Jul 16 01:10:20.781388 ip-10-0-143-200 kubenswrapper[2572]: I0716 01:10:20.781000 2572 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-inf-a-7d5e85-openshift-default-6b7d58d6b5-tk5sq" event={"ID":"a505deae-c477-4c11-8f18-ac8f1dc02b57","Type":"ContainerDied","Data":"b5f7d803a646e97f3c3f3dc8eac6af365582374ab2dc873fa322c1b25912e34b"} Jul 16 01:10:20.781560 ip-10-0-143-200 kubenswrapper[2572]: I0716 01:10:20.781543 2572 scope.go:117] "RemoveContainer" containerID="b5f7d803a646e97f3c3f3dc8eac6af365582374ab2dc873fa322c1b25912e34b" Jul 16 01:10:20.782566 ip-10-0-143-200 kubenswrapper[2572]: I0716 01:10:20.782541 2572 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236228Ki","pods":"250"} Jul 16 01:10:20.782673 ip-10-0-143-200 kubenswrapper[2572]: I0716 01:10:20.782601 2572 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236228Ki","pods":"250"} Jul 16 01:10:20.782673 ip-10-0-143-200 kubenswrapper[2572]: I0716 01:10:20.782649 2572 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236228Ki","pods":"250"} Jul 16 01:10:21.792729 ip-10-0-143-200 kubenswrapper[2572]: I0716 01:10:21.792686 2572 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-inf-a-7d5e85-openshift-default-6b7d58d6b5-tk5sq" event={"ID":"a505deae-c477-4c11-8f18-ac8f1dc02b57","Type":"ContainerStarted","Data":"09d9b0080331a48a122988dbbcfd9c97e6e43008934bfe595f743129cff2a9e5"} Jul 16 01:10:22.195175 ip-10-0-143-200 kubenswrapper[2572]: I0716 01:10:22.195061 2572 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="openshift-ingress/e2e-inf-a-7d5e85-openshift-default-6b7d58d6b5-tk5sq" Jul 16 01:10:22.196515 ip-10-0-143-200 kubenswrapper[2572]: I0716 01:10:22.196471 2572 patch_prober.go:28] interesting pod/e2e-inf-a-7d5e85-openshift-default-6b7d58d6b5-tk5sq container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.132.0.57:15021/healthz/ready\": dial tcp 10.132.0.57:15021: connect: connection refused" start-of-body= Jul 16 01:10:22.196795 ip-10-0-143-200 kubenswrapper[2572]: I0716 01:10:22.196751 2572 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-inf-a-7d5e85-openshift-default-6b7d58d6b5-tk5sq" podUID="a505deae-c477-4c11-8f18-ac8f1dc02b57" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.132.0.57:15021/healthz/ready\": dial tcp 10.132.0.57:15021: connect: connection refused" Jul 16 01:10:23.195367 ip-10-0-143-200 kubenswrapper[2572]: I0716 01:10:23.195302 2572 patch_prober.go:28] interesting pod/e2e-inf-a-7d5e85-openshift-default-6b7d58d6b5-tk5sq container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.132.0.57:15021/healthz/ready\": dial tcp 10.132.0.57:15021: connect: connection refused" start-of-body= Jul 16 01:10:23.195894 ip-10-0-143-200 kubenswrapper[2572]: I0716 01:10:23.195409 2572 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-inf-a-7d5e85-openshift-default-6b7d58d6b5-tk5sq" podUID="a505deae-c477-4c11-8f18-ac8f1dc02b57" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.132.0.57:15021/healthz/ready\": dial tcp 10.132.0.57:15021: connect: connection refused" Jul 16 01:10:23.300246 ip-10-0-143-200 kubenswrapper[2572]: I0716 01:10:23.300059 2572 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9gnmq_3a7e030d-fad8-480d-9dc6-fe5057048cf9/konnectivity-agent/0.log" limit=1024 Jul 16 01:10:24.195022 ip-10-0-143-200 kubenswrapper[2572]: I0716 01:10:24.194982 2572 patch_prober.go:28] interesting pod/e2e-inf-a-7d5e85-openshift-default-6b7d58d6b5-tk5sq container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.132.0.57:15021/healthz/ready\": dial tcp 10.132.0.57:15021: connect: connection refused" start-of-body= Jul 16 01:10:24.195352 ip-10-0-143-200 kubenswrapper[2572]: I0716 01:10:24.195317 2572 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-inf-a-7d5e85-openshift-default-6b7d58d6b5-tk5sq" podUID="a505deae-c477-4c11-8f18-ac8f1dc02b57" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.132.0.57:15021/healthz/ready\": dial tcp 10.132.0.57:15021: connect: connection refused" Jul 16 01:10:24.715746 ip-10-0-143-200 kubenswrapper[2572]: I0716 01:10:24.715695 2572 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/e2e-inf-a-7d5e85-openshift-default-6b7d58d6b5-tk5sq" Jul 16 01:10:25.195301 ip-10-0-143-200 kubenswrapper[2572]: I0716 01:10:25.195191 2572 patch_prober.go:28] interesting pod/e2e-inf-a-7d5e85-openshift-default-6b7d58d6b5-tk5sq container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.132.0.57:15021/healthz/ready\": dial tcp 10.132.0.57:15021: connect: connection refused" start-of-body= Jul 16 01:10:25.195301 ip-10-0-143-200 kubenswrapper[2572]: I0716 01:10:25.195266 2572 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-inf-a-7d5e85-openshift-default-6b7d58d6b5-tk5sq" podUID="a505deae-c477-4c11-8f18-ac8f1dc02b57" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.132.0.57:15021/healthz/ready\": dial tcp 10.132.0.57:15021: connect: connection refused" Jul 16 01:10:26.206884 ip-10-0-143-200 kubenswrapper[2572]: I0716 01:10:26.206844 2572 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="openshift-ingress/e2e-inf-a-7d5e85-openshift-default-6b7d58d6b5-tk5sq" Jul 16 01:10:26.210166 ip-10-0-143-200 kubenswrapper[2572]: I0716 01:10:26.210146 2572 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/e2e-inf-a-7d5e85-openshift-default-6b7d58d6b5-tk5sq" Jul 16 01:10:31.134980 ip-10-0-143-200 kubenswrapper[2572]: I0716 01:10:31.134877 2572 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9gnmq_3a7e030d-fad8-480d-9dc6-fe5057048cf9/konnectivity-agent/0.log" limit=1024 Jul 16 01:10:31.227058 ip-10-0-143-200 kubenswrapper[2572]: I0716 01:10:31.227022 2572 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9gnmq_3a7e030d-fad8-480d-9dc6-fe5057048cf9/konnectivity-agent/0.log" limit=1024 Jul 16 01:10:31.861847 ip-10-0-143-200 kubenswrapper[2572]: I0716 01:10:31.861813 2572 generic.go:356] "Generic (PLEG): container finished" podID="301fffd1-42b5-4cfd-a97d-53a754a45110" containerID="09ee4e1110f8f4521d6a38d1ac8b9e163a433ce54997cc7a139ba67c0247d056" exitCode=0 Jul 16 01:10:31.862082 ip-10-0-143-200 kubenswrapper[2572]: I0716 01:10:31.861894 2572 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-inf-b-e5854b-openshift-default-77bc9577d9-8qdwp" event={"ID":"301fffd1-42b5-4cfd-a97d-53a754a45110","Type":"ContainerDied","Data":"09ee4e1110f8f4521d6a38d1ac8b9e163a433ce54997cc7a139ba67c0247d056"} Jul 16 01:10:31.862379 ip-10-0-143-200 kubenswrapper[2572]: I0716 01:10:31.862363 2572 scope.go:117] "RemoveContainer" containerID="09ee4e1110f8f4521d6a38d1ac8b9e163a433ce54997cc7a139ba67c0247d056" Jul 16 01:10:31.863458 ip-10-0-143-200 kubenswrapper[2572]: I0716 01:10:31.863431 2572 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236228Ki","pods":"250"} Jul 16 01:10:31.863549 ip-10-0-143-200 kubenswrapper[2572]: I0716 01:10:31.863511 2572 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236228Ki","pods":"250"} Jul 16 01:10:31.863610 ip-10-0-143-200 kubenswrapper[2572]: I0716 01:10:31.863558 2572 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236228Ki","pods":"250"} Jul 16 01:10:32.869415 ip-10-0-143-200 kubenswrapper[2572]: I0716 01:10:32.869377 2572 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-inf-b-e5854b-openshift-default-77bc9577d9-8qdwp" event={"ID":"301fffd1-42b5-4cfd-a97d-53a754a45110","Type":"ContainerStarted","Data":"c89849f625eec5d523a6cf4d43bb2c2c0c838701e78dcb102026ab8521ab29ad"} Jul 16 01:10:33.141325 ip-10-0-143-200 kubenswrapper[2572]: I0716 01:10:33.141229 2572 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="openshift-ingress/e2e-inf-b-e5854b-openshift-default-77bc9577d9-8qdwp" Jul 16 01:10:33.142618 ip-10-0-143-200 kubenswrapper[2572]: I0716 01:10:33.142582 2572 patch_prober.go:28] interesting pod/e2e-inf-b-e5854b-openshift-default-77bc9577d9-8qdwp container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.132.0.58:15021/healthz/ready\": dial tcp 10.132.0.58:15021: connect: connection refused" start-of-body= Jul 16 01:10:33.142783 ip-10-0-143-200 kubenswrapper[2572]: I0716 01:10:33.142647 2572 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-inf-b-e5854b-openshift-default-77bc9577d9-8qdwp" podUID="301fffd1-42b5-4cfd-a97d-53a754a45110" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.132.0.58:15021/healthz/ready\": dial tcp 10.132.0.58:15021: connect: connection refused" Jul 16 01:10:33.768603 ip-10-0-143-200 kubenswrapper[2572]: I0716 01:10:33.768568 2572 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/e2e-inf-b-e5854b-openshift-default-77bc9577d9-8qdwp" Jul 16 01:10:34.141127 ip-10-0-143-200 kubenswrapper[2572]: I0716 01:10:34.141030 2572 patch_prober.go:28] interesting pod/e2e-inf-b-e5854b-openshift-default-77bc9577d9-8qdwp container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.132.0.58:15021/healthz/ready\": dial tcp 10.132.0.58:15021: connect: connection refused" start-of-body= Jul 16 01:10:34.141127 ip-10-0-143-200 kubenswrapper[2572]: I0716 01:10:34.141096 2572 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-inf-b-e5854b-openshift-default-77bc9577d9-8qdwp" podUID="301fffd1-42b5-4cfd-a97d-53a754a45110" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.132.0.58:15021/healthz/ready\": dial tcp 10.132.0.58:15021: connect: connection refused" Jul 16 01:10:35.141807 ip-10-0-143-200 kubenswrapper[2572]: I0716 01:10:35.141768 2572 patch_prober.go:28] interesting pod/e2e-inf-b-e5854b-openshift-default-77bc9577d9-8qdwp container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.132.0.58:15021/healthz/ready\": dial tcp 10.132.0.58:15021: connect: connection refused" start-of-body= Jul 16 01:10:35.142214 ip-10-0-143-200 kubenswrapper[2572]: I0716 01:10:35.141831 2572 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-inf-b-e5854b-openshift-default-77bc9577d9-8qdwp" podUID="301fffd1-42b5-4cfd-a97d-53a754a45110" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.132.0.58:15021/healthz/ready\": dial tcp 10.132.0.58:15021: connect: connection refused" Jul 16 01:10:36.114119 ip-10-0-143-200 kubenswrapper[2572]: I0716 01:10:36.114066 2572 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9gnmq_3a7e030d-fad8-480d-9dc6-fe5057048cf9/konnectivity-agent/0.log" limit=1024 Jul 16 01:10:36.141891 ip-10-0-143-200 kubenswrapper[2572]: I0716 01:10:36.141855 2572 patch_prober.go:28] interesting pod/e2e-inf-b-e5854b-openshift-default-77bc9577d9-8qdwp container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.132.0.58:15021/healthz/ready\": dial tcp 10.132.0.58:15021: connect: connection refused" start-of-body= Jul 16 01:10:36.142388 ip-10-0-143-200 kubenswrapper[2572]: I0716 01:10:36.141953 2572 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-inf-b-e5854b-openshift-default-77bc9577d9-8qdwp" podUID="301fffd1-42b5-4cfd-a97d-53a754a45110" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.132.0.58:15021/healthz/ready\": dial tcp 10.132.0.58:15021: connect: connection refused" Jul 16 01:10:37.145110 ip-10-0-143-200 kubenswrapper[2572]: I0716 01:10:37.145078 2572 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="openshift-ingress/e2e-inf-b-e5854b-openshift-default-77bc9577d9-8qdwp" Jul 16 01:10:37.146139 ip-10-0-143-200 kubenswrapper[2572]: I0716 01:10:37.146119 2572 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/e2e-inf-b-e5854b-openshift-default-77bc9577d9-8qdwp" Jul 16 01:10:39.440296 ip-10-0-143-200 kubenswrapper[2572]: I0716 01:10:39.440259 2572 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9gnmq_3a7e030d-fad8-480d-9dc6-fe5057048cf9/konnectivity-agent/0.log" limit=1024 Jul 16 01:10:39.539757 ip-10-0-143-200 kubenswrapper[2572]: I0716 01:10:39.539721 2572 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9gnmq_3a7e030d-fad8-480d-9dc6-fe5057048cf9/konnectivity-agent/0.log" limit=1024 Jul 16 01:11:17.453661 ip-10-0-143-200 kubenswrapper[2572]: I0716 01:11:17.453567 2572 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9gnmq_3a7e030d-fad8-480d-9dc6-fe5057048cf9/konnectivity-agent/0.log" limit=1024 Jul 16 01:11:42.435771 ip-10-0-143-200 kubenswrapper[2572]: I0716 01:11:42.435732 2572 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9gnmq_3a7e030d-fad8-480d-9dc6-fe5057048cf9/konnectivity-agent/0.log" limit=1024 Jul 16 01:11:43.400970 ip-10-0-143-200 kubenswrapper[2572]: I0716 01:11:43.400917 2572 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9gnmq_3a7e030d-fad8-480d-9dc6-fe5057048cf9/konnectivity-agent/0.log" limit=1024 Jul 16 01:11:43.873103 ip-10-0-143-200 kubenswrapper[2572]: I0716 01:11:43.873069 2572 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9gnmq_3a7e030d-fad8-480d-9dc6-fe5057048cf9/konnectivity-agent/0.log" limit=1024 Jul 16 01:11:48.953027 ip-10-0-143-200 kubenswrapper[2572]: I0716 01:11:48.952991 2572 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9gnmq_3a7e030d-fad8-480d-9dc6-fe5057048cf9/konnectivity-agent/0.log" limit=1024 Jul 16 01:12:03.080213 ip-10-0-143-200 kubenswrapper[2572]: I0716 01:12:03.080179 2572 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9gnmq_3a7e030d-fad8-480d-9dc6-fe5057048cf9/konnectivity-agent/0.log" limit=1024 Jul 16 01:12:25.744139 ip-10-0-143-200 kubenswrapper[2572]: I0716 01:12:25.744102 2572 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9gnmq_3a7e030d-fad8-480d-9dc6-fe5057048cf9/konnectivity-agent/0.log" limit=1024 Jul 16 01:12:25.988419 ip-10-0-143-200 kubenswrapper[2572]: I0716 01:12:25.988383 2572 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-ingress/e2e-inf-a-7d5e85-openshift-default-6b7d58d6b5-tk5sq"] Jul 16 01:12:25.988686 ip-10-0-143-200 kubenswrapper[2572]: I0716 01:12:25.988659 2572 kuberuntime_container.go:873] "Killing container with a grace period" pod="openshift-ingress/e2e-inf-a-7d5e85-openshift-default-6b7d58d6b5-tk5sq" podUID="a505deae-c477-4c11-8f18-ac8f1dc02b57" containerName="istio-proxy" containerID="cri-o://09d9b0080331a48a122988dbbcfd9c97e6e43008934bfe595f743129cff2a9e5" gracePeriod=30 Jul 16 01:12:26.000931 ip-10-0-143-200 kubenswrapper[2572]: I0716 01:12:26.000855 2572 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9gnmq_3a7e030d-fad8-480d-9dc6-fe5057048cf9/konnectivity-agent/0.log" limit=1024 Jul 16 01:12:26.210960 ip-10-0-143-200 kubenswrapper[2572]: I0716 01:12:26.210924 2572 patch_prober.go:28] interesting pod/e2e-inf-a-7d5e85-openshift-default-6b7d58d6b5-tk5sq container/istio-proxy namespace/openshift-ingress: Readiness probe status=failure output="HTTP probe failed with statuscode: 503" start-of-body= Jul 16 01:12:26.211218 ip-10-0-143-200 kubenswrapper[2572]: I0716 01:12:26.210996 2572 prober.go:120] "Probe failed" probeType="Readiness" pod="openshift-ingress/e2e-inf-a-7d5e85-openshift-default-6b7d58d6b5-tk5sq" podUID="a505deae-c477-4c11-8f18-ac8f1dc02b57" containerName="istio-proxy" probeResult="failure" output="HTTP probe failed with statuscode: 503" Jul 16 01:12:26.261971 ip-10-0-143-200 kubenswrapper[2572]: I0716 01:12:26.261886 2572 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9gnmq_3a7e030d-fad8-480d-9dc6-fe5057048cf9/konnectivity-agent/0.log" limit=1024 Jul 16 01:12:26.713085 ip-10-0-143-200 kubenswrapper[2572]: I0716 01:12:26.713052 2572 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9gnmq_3a7e030d-fad8-480d-9dc6-fe5057048cf9/konnectivity-agent/0.log" limit=1024 Jul 16 01:12:31.254483 ip-10-0-143-200 kubenswrapper[2572]: I0716 01:12:31.254409 2572 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-inf-a-7d5e85-openshift-default-6b7d58d6b5-tk5sq" Jul 16 01:12:31.416094 ip-10-0-143-200 kubenswrapper[2572]: I0716 01:12:31.416051 2572 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/a505deae-c477-4c11-8f18-ac8f1dc02b57-credential-socket\") pod \"a505deae-c477-4c11-8f18-ac8f1dc02b57\" (UID: \"a505deae-c477-4c11-8f18-ac8f1dc02b57\") " Jul 16 01:12:31.416306 ip-10-0-143-200 kubenswrapper[2572]: I0716 01:12:31.416131 2572 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-nttkz\" (UniqueName: \"kubernetes.io/projected/a505deae-c477-4c11-8f18-ac8f1dc02b57-kube-api-access-nttkz\") pod \"a505deae-c477-4c11-8f18-ac8f1dc02b57\" (UID: \"a505deae-c477-4c11-8f18-ac8f1dc02b57\") " Jul 16 01:12:31.416306 ip-10-0-143-200 kubenswrapper[2572]: I0716 01:12:31.416166 2572 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/a505deae-c477-4c11-8f18-ac8f1dc02b57-istio-envoy\") pod \"a505deae-c477-4c11-8f18-ac8f1dc02b57\" (UID: \"a505deae-c477-4c11-8f18-ac8f1dc02b57\") " Jul 16 01:12:31.416306 ip-10-0-143-200 kubenswrapper[2572]: I0716 01:12:31.416187 2572 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/a505deae-c477-4c11-8f18-ac8f1dc02b57-istio-podinfo\") pod \"a505deae-c477-4c11-8f18-ac8f1dc02b57\" (UID: \"a505deae-c477-4c11-8f18-ac8f1dc02b57\") " Jul 16 01:12:31.416306 ip-10-0-143-200 kubenswrapper[2572]: I0716 01:12:31.416206 2572 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/a505deae-c477-4c11-8f18-ac8f1dc02b57-workload-certs\") pod \"a505deae-c477-4c11-8f18-ac8f1dc02b57\" (UID: \"a505deae-c477-4c11-8f18-ac8f1dc02b57\") " Jul 16 01:12:31.416306 ip-10-0-143-200 kubenswrapper[2572]: I0716 01:12:31.416230 2572 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/a505deae-c477-4c11-8f18-ac8f1dc02b57-workload-socket\") pod \"a505deae-c477-4c11-8f18-ac8f1dc02b57\" (UID: \"a505deae-c477-4c11-8f18-ac8f1dc02b57\") " Jul 16 01:12:31.416306 ip-10-0-143-200 kubenswrapper[2572]: I0716 01:12:31.416256 2572 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/a505deae-c477-4c11-8f18-ac8f1dc02b57-istio-token\") pod \"a505deae-c477-4c11-8f18-ac8f1dc02b57\" (UID: \"a505deae-c477-4c11-8f18-ac8f1dc02b57\") " Jul 16 01:12:31.416661 ip-10-0-143-200 kubenswrapper[2572]: I0716 01:12:31.416322 2572 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/a505deae-c477-4c11-8f18-ac8f1dc02b57-istiod-ca-cert\") pod \"a505deae-c477-4c11-8f18-ac8f1dc02b57\" (UID: \"a505deae-c477-4c11-8f18-ac8f1dc02b57\") " Jul 16 01:12:31.416661 ip-10-0-143-200 kubenswrapper[2572]: I0716 01:12:31.416358 2572 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/a505deae-c477-4c11-8f18-ac8f1dc02b57-istio-data\") pod \"a505deae-c477-4c11-8f18-ac8f1dc02b57\" (UID: \"a505deae-c477-4c11-8f18-ac8f1dc02b57\") " Jul 16 01:12:31.416661 ip-10-0-143-200 kubenswrapper[2572]: I0716 01:12:31.416421 2572 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/a505deae-c477-4c11-8f18-ac8f1dc02b57-credential-socket" (OuterVolumeSpecName: "credential-socket") pod "a505deae-c477-4c11-8f18-ac8f1dc02b57" (UID: "a505deae-c477-4c11-8f18-ac8f1dc02b57"). InnerVolumeSpecName "credential-socket". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Jul 16 01:12:31.416830 ip-10-0-143-200 kubenswrapper[2572]: I0716 01:12:31.416621 2572 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/a505deae-c477-4c11-8f18-ac8f1dc02b57-workload-socket" (OuterVolumeSpecName: "workload-socket") pod "a505deae-c477-4c11-8f18-ac8f1dc02b57" (UID: "a505deae-c477-4c11-8f18-ac8f1dc02b57"). InnerVolumeSpecName "workload-socket". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Jul 16 01:12:31.416830 ip-10-0-143-200 kubenswrapper[2572]: I0716 01:12:31.416760 2572 reconciler_common.go:299] "Volume detached for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/a505deae-c477-4c11-8f18-ac8f1dc02b57-credential-socket\") on node \"ip-10-0-143-200.ec2.internal\" DevicePath \"\"" Jul 16 01:12:31.416830 ip-10-0-143-200 kubenswrapper[2572]: I0716 01:12:31.416780 2572 reconciler_common.go:299] "Volume detached for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/a505deae-c477-4c11-8f18-ac8f1dc02b57-workload-socket\") on node \"ip-10-0-143-200.ec2.internal\" DevicePath \"\"" Jul 16 01:12:31.416976 ip-10-0-143-200 kubenswrapper[2572]: I0716 01:12:31.416857 2572 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/a505deae-c477-4c11-8f18-ac8f1dc02b57-workload-certs" (OuterVolumeSpecName: "workload-certs") pod "a505deae-c477-4c11-8f18-ac8f1dc02b57" (UID: "a505deae-c477-4c11-8f18-ac8f1dc02b57"). InnerVolumeSpecName "workload-certs". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Jul 16 01:12:31.416976 ip-10-0-143-200 kubenswrapper[2572]: I0716 01:12:31.416893 2572 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/a505deae-c477-4c11-8f18-ac8f1dc02b57-istiod-ca-cert" (OuterVolumeSpecName: "istiod-ca-cert") pod "a505deae-c477-4c11-8f18-ac8f1dc02b57" (UID: "a505deae-c477-4c11-8f18-ac8f1dc02b57"). InnerVolumeSpecName "istiod-ca-cert". PluginName "kubernetes.io/configmap", VolumeGIDValue "" Jul 16 01:12:31.417596 ip-10-0-143-200 kubenswrapper[2572]: I0716 01:12:31.417572 2572 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/a505deae-c477-4c11-8f18-ac8f1dc02b57-istio-data" (OuterVolumeSpecName: "istio-data") pod "a505deae-c477-4c11-8f18-ac8f1dc02b57" (UID: "a505deae-c477-4c11-8f18-ac8f1dc02b57"). InnerVolumeSpecName "istio-data". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Jul 16 01:12:31.418613 ip-10-0-143-200 kubenswrapper[2572]: I0716 01:12:31.418576 2572 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/downward-api/a505deae-c477-4c11-8f18-ac8f1dc02b57-istio-podinfo" (OuterVolumeSpecName: "istio-podinfo") pod "a505deae-c477-4c11-8f18-ac8f1dc02b57" (UID: "a505deae-c477-4c11-8f18-ac8f1dc02b57"). InnerVolumeSpecName "istio-podinfo". PluginName "kubernetes.io/downward-api", VolumeGIDValue "" Jul 16 01:12:31.418778 ip-10-0-143-200 kubenswrapper[2572]: I0716 01:12:31.418610 2572 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/a505deae-c477-4c11-8f18-ac8f1dc02b57-istio-token" (OuterVolumeSpecName: "istio-token") pod "a505deae-c477-4c11-8f18-ac8f1dc02b57" (UID: "a505deae-c477-4c11-8f18-ac8f1dc02b57"). InnerVolumeSpecName "istio-token". PluginName "kubernetes.io/projected", VolumeGIDValue "" Jul 16 01:12:31.418929 ip-10-0-143-200 kubenswrapper[2572]: I0716 01:12:31.418903 2572 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/a505deae-c477-4c11-8f18-ac8f1dc02b57-istio-envoy" (OuterVolumeSpecName: "istio-envoy") pod "a505deae-c477-4c11-8f18-ac8f1dc02b57" (UID: "a505deae-c477-4c11-8f18-ac8f1dc02b57"). InnerVolumeSpecName "istio-envoy". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Jul 16 01:12:31.418929 ip-10-0-143-200 kubenswrapper[2572]: I0716 01:12:31.418915 2572 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/a505deae-c477-4c11-8f18-ac8f1dc02b57-kube-api-access-nttkz" (OuterVolumeSpecName: "kube-api-access-nttkz") pod "a505deae-c477-4c11-8f18-ac8f1dc02b57" (UID: "a505deae-c477-4c11-8f18-ac8f1dc02b57"). InnerVolumeSpecName "kube-api-access-nttkz". PluginName "kubernetes.io/projected", VolumeGIDValue "" Jul 16 01:12:31.511020 ip-10-0-143-200 kubenswrapper[2572]: I0716 01:12:31.510940 2572 generic.go:356] "Generic (PLEG): container finished" podID="a505deae-c477-4c11-8f18-ac8f1dc02b57" containerID="09d9b0080331a48a122988dbbcfd9c97e6e43008934bfe595f743129cff2a9e5" exitCode=0 Jul 16 01:12:31.511020 ip-10-0-143-200 kubenswrapper[2572]: I0716 01:12:31.510987 2572 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-inf-a-7d5e85-openshift-default-6b7d58d6b5-tk5sq" Jul 16 01:12:31.511193 ip-10-0-143-200 kubenswrapper[2572]: I0716 01:12:31.511015 2572 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-inf-a-7d5e85-openshift-default-6b7d58d6b5-tk5sq" event={"ID":"a505deae-c477-4c11-8f18-ac8f1dc02b57","Type":"ContainerDied","Data":"09d9b0080331a48a122988dbbcfd9c97e6e43008934bfe595f743129cff2a9e5"} Jul 16 01:12:31.511193 ip-10-0-143-200 kubenswrapper[2572]: I0716 01:12:31.511069 2572 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-inf-a-7d5e85-openshift-default-6b7d58d6b5-tk5sq" event={"ID":"a505deae-c477-4c11-8f18-ac8f1dc02b57","Type":"ContainerDied","Data":"4059089df3fdef5d3769c0eedcaa83e523516e286fb7b6f49debf364459e4682"} Jul 16 01:12:31.511193 ip-10-0-143-200 kubenswrapper[2572]: I0716 01:12:31.511090 2572 scope.go:117] "RemoveContainer" containerID="09d9b0080331a48a122988dbbcfd9c97e6e43008934bfe595f743129cff2a9e5" Jul 16 01:12:31.518083 ip-10-0-143-200 kubenswrapper[2572]: I0716 01:12:31.518054 2572 reconciler_common.go:299] "Volume detached for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/a505deae-c477-4c11-8f18-ac8f1dc02b57-istio-envoy\") on node \"ip-10-0-143-200.ec2.internal\" DevicePath \"\"" Jul 16 01:12:31.518083 ip-10-0-143-200 kubenswrapper[2572]: I0716 01:12:31.518078 2572 reconciler_common.go:299] "Volume detached for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/a505deae-c477-4c11-8f18-ac8f1dc02b57-istio-podinfo\") on node \"ip-10-0-143-200.ec2.internal\" DevicePath \"\"" Jul 16 01:12:31.518307 ip-10-0-143-200 kubenswrapper[2572]: I0716 01:12:31.518092 2572 reconciler_common.go:299] "Volume detached for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/a505deae-c477-4c11-8f18-ac8f1dc02b57-workload-certs\") on node \"ip-10-0-143-200.ec2.internal\" DevicePath \"\"" Jul 16 01:12:31.518307 ip-10-0-143-200 kubenswrapper[2572]: I0716 01:12:31.518104 2572 reconciler_common.go:299] "Volume detached for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/a505deae-c477-4c11-8f18-ac8f1dc02b57-istio-token\") on node \"ip-10-0-143-200.ec2.internal\" DevicePath \"\"" Jul 16 01:12:31.518307 ip-10-0-143-200 kubenswrapper[2572]: I0716 01:12:31.518114 2572 reconciler_common.go:299] "Volume detached for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/a505deae-c477-4c11-8f18-ac8f1dc02b57-istiod-ca-cert\") on node \"ip-10-0-143-200.ec2.internal\" DevicePath \"\"" Jul 16 01:12:31.518307 ip-10-0-143-200 kubenswrapper[2572]: I0716 01:12:31.518124 2572 reconciler_common.go:299] "Volume detached for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/a505deae-c477-4c11-8f18-ac8f1dc02b57-istio-data\") on node \"ip-10-0-143-200.ec2.internal\" DevicePath \"\"" Jul 16 01:12:31.518307 ip-10-0-143-200 kubenswrapper[2572]: I0716 01:12:31.518166 2572 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-nttkz\" (UniqueName: \"kubernetes.io/projected/a505deae-c477-4c11-8f18-ac8f1dc02b57-kube-api-access-nttkz\") on node \"ip-10-0-143-200.ec2.internal\" DevicePath \"\"" Jul 16 01:12:31.521133 ip-10-0-143-200 kubenswrapper[2572]: I0716 01:12:31.521113 2572 scope.go:117] "RemoveContainer" containerID="b5f7d803a646e97f3c3f3dc8eac6af365582374ab2dc873fa322c1b25912e34b" Jul 16 01:12:31.530068 ip-10-0-143-200 kubenswrapper[2572]: I0716 01:12:31.530048 2572 scope.go:117] "RemoveContainer" containerID="09d9b0080331a48a122988dbbcfd9c97e6e43008934bfe595f743129cff2a9e5" Jul 16 01:12:31.530354 ip-10-0-143-200 kubenswrapper[2572]: E0716 01:12:31.530335 2572 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"09d9b0080331a48a122988dbbcfd9c97e6e43008934bfe595f743129cff2a9e5\": container with ID starting with 09d9b0080331a48a122988dbbcfd9c97e6e43008934bfe595f743129cff2a9e5 not found: ID does not exist" containerID="09d9b0080331a48a122988dbbcfd9c97e6e43008934bfe595f743129cff2a9e5" Jul 16 01:12:31.530421 ip-10-0-143-200 kubenswrapper[2572]: I0716 01:12:31.530360 2572 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"09d9b0080331a48a122988dbbcfd9c97e6e43008934bfe595f743129cff2a9e5"} err="failed to get container status \"09d9b0080331a48a122988dbbcfd9c97e6e43008934bfe595f743129cff2a9e5\": rpc error: code = NotFound desc = could not find container \"09d9b0080331a48a122988dbbcfd9c97e6e43008934bfe595f743129cff2a9e5\": container with ID starting with 09d9b0080331a48a122988dbbcfd9c97e6e43008934bfe595f743129cff2a9e5 not found: ID does not exist" Jul 16 01:12:31.530421 ip-10-0-143-200 kubenswrapper[2572]: I0716 01:12:31.530377 2572 scope.go:117] "RemoveContainer" containerID="b5f7d803a646e97f3c3f3dc8eac6af365582374ab2dc873fa322c1b25912e34b" Jul 16 01:12:31.530667 ip-10-0-143-200 kubenswrapper[2572]: E0716 01:12:31.530649 2572 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"b5f7d803a646e97f3c3f3dc8eac6af365582374ab2dc873fa322c1b25912e34b\": container with ID starting with b5f7d803a646e97f3c3f3dc8eac6af365582374ab2dc873fa322c1b25912e34b not found: ID does not exist" containerID="b5f7d803a646e97f3c3f3dc8eac6af365582374ab2dc873fa322c1b25912e34b" Jul 16 01:12:31.530737 ip-10-0-143-200 kubenswrapper[2572]: I0716 01:12:31.530676 2572 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"b5f7d803a646e97f3c3f3dc8eac6af365582374ab2dc873fa322c1b25912e34b"} err="failed to get container status \"b5f7d803a646e97f3c3f3dc8eac6af365582374ab2dc873fa322c1b25912e34b\": rpc error: code = NotFound desc = could not find container \"b5f7d803a646e97f3c3f3dc8eac6af365582374ab2dc873fa322c1b25912e34b\": container with ID starting with b5f7d803a646e97f3c3f3dc8eac6af365582374ab2dc873fa322c1b25912e34b not found: ID does not exist" Jul 16 01:12:31.530778 ip-10-0-143-200 kubenswrapper[2572]: I0716 01:12:31.530738 2572 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-ingress/e2e-inf-a-7d5e85-openshift-default-6b7d58d6b5-tk5sq"] Jul 16 01:12:31.534929 ip-10-0-143-200 kubenswrapper[2572]: I0716 01:12:31.534908 2572 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["openshift-ingress/e2e-inf-a-7d5e85-openshift-default-6b7d58d6b5-tk5sq"] Jul 16 01:12:32.762845 ip-10-0-143-200 kubenswrapper[2572]: I0716 01:12:32.762813 2572 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9gnmq_3a7e030d-fad8-480d-9dc6-fe5057048cf9/konnectivity-agent/0.log" limit=1024 Jul 16 01:12:32.862224 ip-10-0-143-200 kubenswrapper[2572]: I0716 01:12:32.862189 2572 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9gnmq_3a7e030d-fad8-480d-9dc6-fe5057048cf9/konnectivity-agent/0.log" limit=1024 Jul 16 01:12:33.040304 ip-10-0-143-200 kubenswrapper[2572]: I0716 01:12:33.040214 2572 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-ingress/e2e-inf-b-e5854b-openshift-default-77bc9577d9-8qdwp"] Jul 16 01:12:33.040453 ip-10-0-143-200 kubenswrapper[2572]: I0716 01:12:33.040420 2572 kuberuntime_container.go:873] "Killing container with a grace period" pod="openshift-ingress/e2e-inf-b-e5854b-openshift-default-77bc9577d9-8qdwp" podUID="301fffd1-42b5-4cfd-a97d-53a754a45110" containerName="istio-proxy" containerID="cri-o://c89849f625eec5d523a6cf4d43bb2c2c0c838701e78dcb102026ab8521ab29ad" gracePeriod=30 Jul 16 01:12:33.079591 ip-10-0-143-200 kubenswrapper[2572]: I0716 01:12:33.079560 2572 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9gnmq_3a7e030d-fad8-480d-9dc6-fe5057048cf9/konnectivity-agent/0.log" limit=1024 Jul 16 01:12:33.302106 ip-10-0-143-200 kubenswrapper[2572]: I0716 01:12:33.302011 2572 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9gnmq_3a7e030d-fad8-480d-9dc6-fe5057048cf9/konnectivity-agent/0.log" limit=1024 Jul 16 01:12:33.448346 ip-10-0-143-200 kubenswrapper[2572]: I0716 01:12:33.448311 2572 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="a505deae-c477-4c11-8f18-ac8f1dc02b57" path="/var/lib/kubelet/pods/a505deae-c477-4c11-8f18-ac8f1dc02b57/volumes" Jul 16 01:12:33.604787 ip-10-0-143-200 kubenswrapper[2572]: I0716 01:12:33.604706 2572 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9gnmq_3a7e030d-fad8-480d-9dc6-fe5057048cf9/konnectivity-agent/0.log" limit=1024 Jul 16 01:12:33.894232 ip-10-0-143-200 kubenswrapper[2572]: I0716 01:12:33.894144 2572 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-ingress/e2e-api-a-f1f83d-openshift-default-7f95675bf9-2bb7g"] Jul 16 01:12:33.894905 ip-10-0-143-200 kubenswrapper[2572]: I0716 01:12:33.894883 2572 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="a505deae-c477-4c11-8f18-ac8f1dc02b57" containerName="istio-proxy" Jul 16 01:12:33.894996 ip-10-0-143-200 kubenswrapper[2572]: I0716 01:12:33.894908 2572 state_mem.go:107] "Deleted CPUSet assignment" podUID="a505deae-c477-4c11-8f18-ac8f1dc02b57" containerName="istio-proxy" Jul 16 01:12:33.894996 ip-10-0-143-200 kubenswrapper[2572]: I0716 01:12:33.894934 2572 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="a505deae-c477-4c11-8f18-ac8f1dc02b57" containerName="istio-proxy" Jul 16 01:12:33.894996 ip-10-0-143-200 kubenswrapper[2572]: I0716 01:12:33.894940 2572 state_mem.go:107] "Deleted CPUSet assignment" podUID="a505deae-c477-4c11-8f18-ac8f1dc02b57" containerName="istio-proxy" Jul 16 01:12:33.895119 ip-10-0-143-200 kubenswrapper[2572]: I0716 01:12:33.895008 2572 memory_manager.go:368] "RemoveStaleState removing state" podUID="a505deae-c477-4c11-8f18-ac8f1dc02b57" containerName="istio-proxy" Jul 16 01:12:33.895119 ip-10-0-143-200 kubenswrapper[2572]: I0716 01:12:33.895025 2572 memory_manager.go:368] "RemoveStaleState removing state" podUID="a505deae-c477-4c11-8f18-ac8f1dc02b57" containerName="istio-proxy" Jul 16 01:12:33.899771 ip-10-0-143-200 kubenswrapper[2572]: I0716 01:12:33.899751 2572 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-api-a-f1f83d-openshift-default-7f95675bf9-2bb7g" Jul 16 01:12:33.903063 ip-10-0-143-200 kubenswrapper[2572]: I0716 01:12:33.903037 2572 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress\"/\"e2e-api-a-f1f83d-openshift-default-dockercfg-5l6qg\"" Jul 16 01:12:33.910868 ip-10-0-143-200 kubenswrapper[2572]: I0716 01:12:33.910844 2572 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/e2e-api-a-f1f83d-openshift-default-7f95675bf9-2bb7g"] Jul 16 01:12:33.964197 ip-10-0-143-200 kubenswrapper[2572]: I0716 01:12:33.964160 2572 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9gnmq_3a7e030d-fad8-480d-9dc6-fe5057048cf9/konnectivity-agent/0.log" limit=1024 Jul 16 01:12:34.039968 ip-10-0-143-200 kubenswrapper[2572]: I0716 01:12:34.039930 2572 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/b6672fdb-54f4-4adc-844d-4072c3bf9876-workload-socket\") pod \"e2e-api-a-f1f83d-openshift-default-7f95675bf9-2bb7g\" (UID: \"b6672fdb-54f4-4adc-844d-4072c3bf9876\") " pod="openshift-ingress/e2e-api-a-f1f83d-openshift-default-7f95675bf9-2bb7g" Jul 16 01:12:34.039968 ip-10-0-143-200 kubenswrapper[2572]: I0716 01:12:34.039973 2572 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/b6672fdb-54f4-4adc-844d-4072c3bf9876-workload-certs\") pod \"e2e-api-a-f1f83d-openshift-default-7f95675bf9-2bb7g\" (UID: \"b6672fdb-54f4-4adc-844d-4072c3bf9876\") " pod="openshift-ingress/e2e-api-a-f1f83d-openshift-default-7f95675bf9-2bb7g" Jul 16 01:12:34.040188 ip-10-0-143-200 kubenswrapper[2572]: I0716 01:12:34.040041 2572 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/b6672fdb-54f4-4adc-844d-4072c3bf9876-istio-podinfo\") pod \"e2e-api-a-f1f83d-openshift-default-7f95675bf9-2bb7g\" (UID: \"b6672fdb-54f4-4adc-844d-4072c3bf9876\") " pod="openshift-ingress/e2e-api-a-f1f83d-openshift-default-7f95675bf9-2bb7g" Jul 16 01:12:34.040188 ip-10-0-143-200 kubenswrapper[2572]: I0716 01:12:34.040092 2572 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/b6672fdb-54f4-4adc-844d-4072c3bf9876-istio-data\") pod \"e2e-api-a-f1f83d-openshift-default-7f95675bf9-2bb7g\" (UID: \"b6672fdb-54f4-4adc-844d-4072c3bf9876\") " pod="openshift-ingress/e2e-api-a-f1f83d-openshift-default-7f95675bf9-2bb7g" Jul 16 01:12:34.040188 ip-10-0-143-200 kubenswrapper[2572]: I0716 01:12:34.040114 2572 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/b6672fdb-54f4-4adc-844d-4072c3bf9876-istiod-ca-cert\") pod \"e2e-api-a-f1f83d-openshift-default-7f95675bf9-2bb7g\" (UID: \"b6672fdb-54f4-4adc-844d-4072c3bf9876\") " pod="openshift-ingress/e2e-api-a-f1f83d-openshift-default-7f95675bf9-2bb7g" Jul 16 01:12:34.040188 ip-10-0-143-200 kubenswrapper[2572]: I0716 01:12:34.040135 2572 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/b6672fdb-54f4-4adc-844d-4072c3bf9876-istio-token\") pod \"e2e-api-a-f1f83d-openshift-default-7f95675bf9-2bb7g\" (UID: \"b6672fdb-54f4-4adc-844d-4072c3bf9876\") " pod="openshift-ingress/e2e-api-a-f1f83d-openshift-default-7f95675bf9-2bb7g" Jul 16 01:12:34.040358 ip-10-0-143-200 kubenswrapper[2572]: I0716 01:12:34.040198 2572 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/b6672fdb-54f4-4adc-844d-4072c3bf9876-credential-socket\") pod \"e2e-api-a-f1f83d-openshift-default-7f95675bf9-2bb7g\" (UID: \"b6672fdb-54f4-4adc-844d-4072c3bf9876\") " pod="openshift-ingress/e2e-api-a-f1f83d-openshift-default-7f95675bf9-2bb7g" Jul 16 01:12:34.040358 ip-10-0-143-200 kubenswrapper[2572]: I0716 01:12:34.040280 2572 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-cdxb2\" (UniqueName: \"kubernetes.io/projected/b6672fdb-54f4-4adc-844d-4072c3bf9876-kube-api-access-cdxb2\") pod \"e2e-api-a-f1f83d-openshift-default-7f95675bf9-2bb7g\" (UID: \"b6672fdb-54f4-4adc-844d-4072c3bf9876\") " pod="openshift-ingress/e2e-api-a-f1f83d-openshift-default-7f95675bf9-2bb7g" Jul 16 01:12:34.040358 ip-10-0-143-200 kubenswrapper[2572]: I0716 01:12:34.040334 2572 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/b6672fdb-54f4-4adc-844d-4072c3bf9876-istio-envoy\") pod \"e2e-api-a-f1f83d-openshift-default-7f95675bf9-2bb7g\" (UID: \"b6672fdb-54f4-4adc-844d-4072c3bf9876\") " pod="openshift-ingress/e2e-api-a-f1f83d-openshift-default-7f95675bf9-2bb7g" Jul 16 01:12:34.141089 ip-10-0-143-200 kubenswrapper[2572]: I0716 01:12:34.141048 2572 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/b6672fdb-54f4-4adc-844d-4072c3bf9876-credential-socket\") pod \"e2e-api-a-f1f83d-openshift-default-7f95675bf9-2bb7g\" (UID: \"b6672fdb-54f4-4adc-844d-4072c3bf9876\") " pod="openshift-ingress/e2e-api-a-f1f83d-openshift-default-7f95675bf9-2bb7g" Jul 16 01:12:34.141296 ip-10-0-143-200 kubenswrapper[2572]: I0716 01:12:34.141159 2572 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-cdxb2\" (UniqueName: \"kubernetes.io/projected/b6672fdb-54f4-4adc-844d-4072c3bf9876-kube-api-access-cdxb2\") pod \"e2e-api-a-f1f83d-openshift-default-7f95675bf9-2bb7g\" (UID: \"b6672fdb-54f4-4adc-844d-4072c3bf9876\") " pod="openshift-ingress/e2e-api-a-f1f83d-openshift-default-7f95675bf9-2bb7g" Jul 16 01:12:34.141296 ip-10-0-143-200 kubenswrapper[2572]: I0716 01:12:34.141189 2572 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/b6672fdb-54f4-4adc-844d-4072c3bf9876-istio-envoy\") pod \"e2e-api-a-f1f83d-openshift-default-7f95675bf9-2bb7g\" (UID: \"b6672fdb-54f4-4adc-844d-4072c3bf9876\") " pod="openshift-ingress/e2e-api-a-f1f83d-openshift-default-7f95675bf9-2bb7g" Jul 16 01:12:34.141296 ip-10-0-143-200 kubenswrapper[2572]: I0716 01:12:34.141223 2572 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/b6672fdb-54f4-4adc-844d-4072c3bf9876-workload-socket\") pod \"e2e-api-a-f1f83d-openshift-default-7f95675bf9-2bb7g\" (UID: \"b6672fdb-54f4-4adc-844d-4072c3bf9876\") " pod="openshift-ingress/e2e-api-a-f1f83d-openshift-default-7f95675bf9-2bb7g" Jul 16 01:12:34.141296 ip-10-0-143-200 kubenswrapper[2572]: I0716 01:12:34.141252 2572 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/b6672fdb-54f4-4adc-844d-4072c3bf9876-workload-certs\") pod \"e2e-api-a-f1f83d-openshift-default-7f95675bf9-2bb7g\" (UID: \"b6672fdb-54f4-4adc-844d-4072c3bf9876\") " pod="openshift-ingress/e2e-api-a-f1f83d-openshift-default-7f95675bf9-2bb7g" Jul 16 01:12:34.141296 ip-10-0-143-200 kubenswrapper[2572]: I0716 01:12:34.141286 2572 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/b6672fdb-54f4-4adc-844d-4072c3bf9876-istio-podinfo\") pod \"e2e-api-a-f1f83d-openshift-default-7f95675bf9-2bb7g\" (UID: \"b6672fdb-54f4-4adc-844d-4072c3bf9876\") " pod="openshift-ingress/e2e-api-a-f1f83d-openshift-default-7f95675bf9-2bb7g" Jul 16 01:12:34.141563 ip-10-0-143-200 kubenswrapper[2572]: I0716 01:12:34.141311 2572 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/b6672fdb-54f4-4adc-844d-4072c3bf9876-istio-data\") pod \"e2e-api-a-f1f83d-openshift-default-7f95675bf9-2bb7g\" (UID: \"b6672fdb-54f4-4adc-844d-4072c3bf9876\") " pod="openshift-ingress/e2e-api-a-f1f83d-openshift-default-7f95675bf9-2bb7g" Jul 16 01:12:34.141563 ip-10-0-143-200 kubenswrapper[2572]: I0716 01:12:34.141344 2572 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/b6672fdb-54f4-4adc-844d-4072c3bf9876-istiod-ca-cert\") pod \"e2e-api-a-f1f83d-openshift-default-7f95675bf9-2bb7g\" (UID: \"b6672fdb-54f4-4adc-844d-4072c3bf9876\") " pod="openshift-ingress/e2e-api-a-f1f83d-openshift-default-7f95675bf9-2bb7g" Jul 16 01:12:34.141563 ip-10-0-143-200 kubenswrapper[2572]: I0716 01:12:34.141377 2572 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/b6672fdb-54f4-4adc-844d-4072c3bf9876-istio-token\") pod \"e2e-api-a-f1f83d-openshift-default-7f95675bf9-2bb7g\" (UID: \"b6672fdb-54f4-4adc-844d-4072c3bf9876\") " pod="openshift-ingress/e2e-api-a-f1f83d-openshift-default-7f95675bf9-2bb7g" Jul 16 01:12:34.141563 ip-10-0-143-200 kubenswrapper[2572]: I0716 01:12:34.141551 2572 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/b6672fdb-54f4-4adc-844d-4072c3bf9876-credential-socket\") pod \"e2e-api-a-f1f83d-openshift-default-7f95675bf9-2bb7g\" (UID: \"b6672fdb-54f4-4adc-844d-4072c3bf9876\") " pod="openshift-ingress/e2e-api-a-f1f83d-openshift-default-7f95675bf9-2bb7g" Jul 16 01:12:34.141805 ip-10-0-143-200 kubenswrapper[2572]: I0716 01:12:34.141735 2572 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/b6672fdb-54f4-4adc-844d-4072c3bf9876-istio-data\") pod \"e2e-api-a-f1f83d-openshift-default-7f95675bf9-2bb7g\" (UID: \"b6672fdb-54f4-4adc-844d-4072c3bf9876\") " pod="openshift-ingress/e2e-api-a-f1f83d-openshift-default-7f95675bf9-2bb7g" Jul 16 01:12:34.141899 ip-10-0-143-200 kubenswrapper[2572]: I0716 01:12:34.141876 2572 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/b6672fdb-54f4-4adc-844d-4072c3bf9876-workload-socket\") pod \"e2e-api-a-f1f83d-openshift-default-7f95675bf9-2bb7g\" (UID: \"b6672fdb-54f4-4adc-844d-4072c3bf9876\") " pod="openshift-ingress/e2e-api-a-f1f83d-openshift-default-7f95675bf9-2bb7g" Jul 16 01:12:34.141990 ip-10-0-143-200 kubenswrapper[2572]: I0716 01:12:34.141972 2572 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/b6672fdb-54f4-4adc-844d-4072c3bf9876-workload-certs\") pod \"e2e-api-a-f1f83d-openshift-default-7f95675bf9-2bb7g\" (UID: \"b6672fdb-54f4-4adc-844d-4072c3bf9876\") " pod="openshift-ingress/e2e-api-a-f1f83d-openshift-default-7f95675bf9-2bb7g" Jul 16 01:12:34.142238 ip-10-0-143-200 kubenswrapper[2572]: I0716 01:12:34.142215 2572 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/b6672fdb-54f4-4adc-844d-4072c3bf9876-istiod-ca-cert\") pod \"e2e-api-a-f1f83d-openshift-default-7f95675bf9-2bb7g\" (UID: \"b6672fdb-54f4-4adc-844d-4072c3bf9876\") " pod="openshift-ingress/e2e-api-a-f1f83d-openshift-default-7f95675bf9-2bb7g" Jul 16 01:12:34.143481 ip-10-0-143-200 kubenswrapper[2572]: I0716 01:12:34.143456 2572 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/b6672fdb-54f4-4adc-844d-4072c3bf9876-istio-envoy\") pod \"e2e-api-a-f1f83d-openshift-default-7f95675bf9-2bb7g\" (UID: \"b6672fdb-54f4-4adc-844d-4072c3bf9876\") " pod="openshift-ingress/e2e-api-a-f1f83d-openshift-default-7f95675bf9-2bb7g" Jul 16 01:12:34.143813 ip-10-0-143-200 kubenswrapper[2572]: I0716 01:12:34.143793 2572 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/b6672fdb-54f4-4adc-844d-4072c3bf9876-istio-podinfo\") pod \"e2e-api-a-f1f83d-openshift-default-7f95675bf9-2bb7g\" (UID: \"b6672fdb-54f4-4adc-844d-4072c3bf9876\") " pod="openshift-ingress/e2e-api-a-f1f83d-openshift-default-7f95675bf9-2bb7g" Jul 16 01:12:34.156406 ip-10-0-143-200 kubenswrapper[2572]: I0716 01:12:34.156341 2572 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-cdxb2\" (UniqueName: \"kubernetes.io/projected/b6672fdb-54f4-4adc-844d-4072c3bf9876-kube-api-access-cdxb2\") pod \"e2e-api-a-f1f83d-openshift-default-7f95675bf9-2bb7g\" (UID: \"b6672fdb-54f4-4adc-844d-4072c3bf9876\") " pod="openshift-ingress/e2e-api-a-f1f83d-openshift-default-7f95675bf9-2bb7g" Jul 16 01:12:34.156511 ip-10-0-143-200 kubenswrapper[2572]: I0716 01:12:34.156409 2572 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/b6672fdb-54f4-4adc-844d-4072c3bf9876-istio-token\") pod \"e2e-api-a-f1f83d-openshift-default-7f95675bf9-2bb7g\" (UID: \"b6672fdb-54f4-4adc-844d-4072c3bf9876\") " pod="openshift-ingress/e2e-api-a-f1f83d-openshift-default-7f95675bf9-2bb7g" Jul 16 01:12:34.211683 ip-10-0-143-200 kubenswrapper[2572]: I0716 01:12:34.211622 2572 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-api-a-f1f83d-openshift-default-7f95675bf9-2bb7g" Jul 16 01:12:34.383586 ip-10-0-143-200 kubenswrapper[2572]: I0716 01:12:34.383557 2572 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/e2e-api-a-f1f83d-openshift-default-7f95675bf9-2bb7g"] Jul 16 01:12:34.385572 ip-10-0-143-200 kubenswrapper[2572]: W0716 01:12:34.385545 2572 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podb6672fdb_54f4_4adc_844d_4072c3bf9876.slice/crio-5f643a7695ba4b5c726656167c6efa90bdec6406fbc182a6d9a5269e492e2b7d WatchSource:0}: Error finding container 5f643a7695ba4b5c726656167c6efa90bdec6406fbc182a6d9a5269e492e2b7d: Status 404 returned error can't find the container with id 5f643a7695ba4b5c726656167c6efa90bdec6406fbc182a6d9a5269e492e2b7d Jul 16 01:12:34.388269 ip-10-0-143-200 kubenswrapper[2572]: I0716 01:12:34.388238 2572 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236228Ki","pods":"250"} Jul 16 01:12:34.388347 ip-10-0-143-200 kubenswrapper[2572]: I0716 01:12:34.388308 2572 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236228Ki","pods":"250"} Jul 16 01:12:34.388347 ip-10-0-143-200 kubenswrapper[2572]: I0716 01:12:34.388339 2572 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236228Ki","pods":"250"} Jul 16 01:12:34.458935 ip-10-0-143-200 kubenswrapper[2572]: I0716 01:12:34.458908 2572 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9gnmq_3a7e030d-fad8-480d-9dc6-fe5057048cf9/konnectivity-agent/0.log" limit=1024 Jul 16 01:12:34.530044 ip-10-0-143-200 kubenswrapper[2572]: I0716 01:12:34.529996 2572 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-api-a-f1f83d-openshift-default-7f95675bf9-2bb7g" event={"ID":"b6672fdb-54f4-4adc-844d-4072c3bf9876","Type":"ContainerStarted","Data":"3141b387235d12c5cfb3c8687aa7f55d3f00858bfda0c06e0f08118844b5508e"} Jul 16 01:12:34.530249 ip-10-0-143-200 kubenswrapper[2572]: I0716 01:12:34.530070 2572 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-api-a-f1f83d-openshift-default-7f95675bf9-2bb7g" event={"ID":"b6672fdb-54f4-4adc-844d-4072c3bf9876","Type":"ContainerStarted","Data":"5f643a7695ba4b5c726656167c6efa90bdec6406fbc182a6d9a5269e492e2b7d"} Jul 16 01:12:34.553360 ip-10-0-143-200 kubenswrapper[2572]: I0716 01:12:34.553299 2572 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-ingress/e2e-api-a-f1f83d-openshift-default-7f95675bf9-2bb7g" podStartSLOduration=1.553276189 podStartE2EDuration="1.553276189s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-16 01:12:33 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-16 01:12:34.552318597 +0000 UTC m=+1959.661073275" watchObservedRunningTime="2026-07-16 01:12:34.553276189 +0000 UTC m=+1959.662030865" Jul 16 01:12:35.212052 ip-10-0-143-200 kubenswrapper[2572]: I0716 01:12:35.212016 2572 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="openshift-ingress/e2e-api-a-f1f83d-openshift-default-7f95675bf9-2bb7g" Jul 16 01:12:35.217717 ip-10-0-143-200 kubenswrapper[2572]: I0716 01:12:35.217688 2572 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="openshift-ingress/e2e-api-a-f1f83d-openshift-default-7f95675bf9-2bb7g" Jul 16 01:12:35.487132 ip-10-0-143-200 kubenswrapper[2572]: I0716 01:12:35.487047 2572 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9gnmq_3a7e030d-fad8-480d-9dc6-fe5057048cf9/konnectivity-agent/0.log" limit=1024 Jul 16 01:12:35.535694 ip-10-0-143-200 kubenswrapper[2572]: I0716 01:12:35.535663 2572 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/e2e-api-a-f1f83d-openshift-default-7f95675bf9-2bb7g" Jul 16 01:12:35.536793 ip-10-0-143-200 kubenswrapper[2572]: I0716 01:12:35.536771 2572 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/e2e-api-a-f1f83d-openshift-default-7f95675bf9-2bb7g" Jul 16 01:12:35.605892 ip-10-0-143-200 kubenswrapper[2572]: I0716 01:12:35.605857 2572 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9gnmq_3a7e030d-fad8-480d-9dc6-fe5057048cf9/konnectivity-agent/0.log" limit=1024 Jul 16 01:12:35.684054 ip-10-0-143-200 kubenswrapper[2572]: I0716 01:12:35.684014 2572 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9gnmq_3a7e030d-fad8-480d-9dc6-fe5057048cf9/konnectivity-agent/0.log" limit=1024 Jul 16 01:12:35.922712 ip-10-0-143-200 kubenswrapper[2572]: I0716 01:12:35.922601 2572 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9gnmq_3a7e030d-fad8-480d-9dc6-fe5057048cf9/konnectivity-agent/0.log" limit=1024 Jul 16 01:12:36.242660 ip-10-0-143-200 kubenswrapper[2572]: I0716 01:12:36.242615 2572 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9gnmq_3a7e030d-fad8-480d-9dc6-fe5057048cf9/konnectivity-agent/0.log" limit=1024 Jul 16 01:12:36.346039 ip-10-0-143-200 kubenswrapper[2572]: I0716 01:12:36.345996 2572 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9gnmq_3a7e030d-fad8-480d-9dc6-fe5057048cf9/konnectivity-agent/0.log" limit=1024 Jul 16 01:12:37.147586 ip-10-0-143-200 kubenswrapper[2572]: I0716 01:12:37.147552 2572 patch_prober.go:28] interesting pod/e2e-inf-b-e5854b-openshift-default-77bc9577d9-8qdwp container/istio-proxy namespace/openshift-ingress: Readiness probe status=failure output="HTTP probe failed with statuscode: 503" start-of-body= Jul 16 01:12:37.147772 ip-10-0-143-200 kubenswrapper[2572]: I0716 01:12:37.147604 2572 prober.go:120] "Probe failed" probeType="Readiness" pod="openshift-ingress/e2e-inf-b-e5854b-openshift-default-77bc9577d9-8qdwp" podUID="301fffd1-42b5-4cfd-a97d-53a754a45110" containerName="istio-proxy" probeResult="failure" output="HTTP probe failed with statuscode: 503" Jul 16 01:12:38.314250 ip-10-0-143-200 kubenswrapper[2572]: I0716 01:12:38.314189 2572 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-inf-b-e5854b-openshift-default-77bc9577d9-8qdwp" Jul 16 01:12:38.483393 ip-10-0-143-200 kubenswrapper[2572]: I0716 01:12:38.483357 2572 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-c7jpq\" (UniqueName: \"kubernetes.io/projected/301fffd1-42b5-4cfd-a97d-53a754a45110-kube-api-access-c7jpq\") pod \"301fffd1-42b5-4cfd-a97d-53a754a45110\" (UID: \"301fffd1-42b5-4cfd-a97d-53a754a45110\") " Jul 16 01:12:38.483393 ip-10-0-143-200 kubenswrapper[2572]: I0716 01:12:38.483399 2572 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/301fffd1-42b5-4cfd-a97d-53a754a45110-credential-socket\") pod \"301fffd1-42b5-4cfd-a97d-53a754a45110\" (UID: \"301fffd1-42b5-4cfd-a97d-53a754a45110\") " Jul 16 01:12:38.483623 ip-10-0-143-200 kubenswrapper[2572]: I0716 01:12:38.483458 2572 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/301fffd1-42b5-4cfd-a97d-53a754a45110-workload-socket\") pod \"301fffd1-42b5-4cfd-a97d-53a754a45110\" (UID: \"301fffd1-42b5-4cfd-a97d-53a754a45110\") " Jul 16 01:12:38.483623 ip-10-0-143-200 kubenswrapper[2572]: I0716 01:12:38.483505 2572 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/301fffd1-42b5-4cfd-a97d-53a754a45110-istio-token\") pod \"301fffd1-42b5-4cfd-a97d-53a754a45110\" (UID: \"301fffd1-42b5-4cfd-a97d-53a754a45110\") " Jul 16 01:12:38.483623 ip-10-0-143-200 kubenswrapper[2572]: I0716 01:12:38.483529 2572 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/301fffd1-42b5-4cfd-a97d-53a754a45110-istiod-ca-cert\") pod \"301fffd1-42b5-4cfd-a97d-53a754a45110\" (UID: \"301fffd1-42b5-4cfd-a97d-53a754a45110\") " Jul 16 01:12:38.483623 ip-10-0-143-200 kubenswrapper[2572]: I0716 01:12:38.483576 2572 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/301fffd1-42b5-4cfd-a97d-53a754a45110-istio-podinfo\") pod \"301fffd1-42b5-4cfd-a97d-53a754a45110\" (UID: \"301fffd1-42b5-4cfd-a97d-53a754a45110\") " Jul 16 01:12:38.483623 ip-10-0-143-200 kubenswrapper[2572]: I0716 01:12:38.483600 2572 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/301fffd1-42b5-4cfd-a97d-53a754a45110-workload-certs\") pod \"301fffd1-42b5-4cfd-a97d-53a754a45110\" (UID: \"301fffd1-42b5-4cfd-a97d-53a754a45110\") " Jul 16 01:12:38.483623 ip-10-0-143-200 kubenswrapper[2572]: I0716 01:12:38.483656 2572 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/301fffd1-42b5-4cfd-a97d-53a754a45110-istio-data\") pod \"301fffd1-42b5-4cfd-a97d-53a754a45110\" (UID: \"301fffd1-42b5-4cfd-a97d-53a754a45110\") " Jul 16 01:12:38.483990 ip-10-0-143-200 kubenswrapper[2572]: I0716 01:12:38.483688 2572 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/301fffd1-42b5-4cfd-a97d-53a754a45110-istio-envoy\") pod \"301fffd1-42b5-4cfd-a97d-53a754a45110\" (UID: \"301fffd1-42b5-4cfd-a97d-53a754a45110\") " Jul 16 01:12:38.483990 ip-10-0-143-200 kubenswrapper[2572]: I0716 01:12:38.483769 2572 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/301fffd1-42b5-4cfd-a97d-53a754a45110-credential-socket" (OuterVolumeSpecName: "credential-socket") pod "301fffd1-42b5-4cfd-a97d-53a754a45110" (UID: "301fffd1-42b5-4cfd-a97d-53a754a45110"). InnerVolumeSpecName "credential-socket". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Jul 16 01:12:38.483990 ip-10-0-143-200 kubenswrapper[2572]: I0716 01:12:38.483826 2572 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/301fffd1-42b5-4cfd-a97d-53a754a45110-workload-socket" (OuterVolumeSpecName: "workload-socket") pod "301fffd1-42b5-4cfd-a97d-53a754a45110" (UID: "301fffd1-42b5-4cfd-a97d-53a754a45110"). InnerVolumeSpecName "workload-socket". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Jul 16 01:12:38.484148 ip-10-0-143-200 kubenswrapper[2572]: I0716 01:12:38.484021 2572 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/301fffd1-42b5-4cfd-a97d-53a754a45110-workload-certs" (OuterVolumeSpecName: "workload-certs") pod "301fffd1-42b5-4cfd-a97d-53a754a45110" (UID: "301fffd1-42b5-4cfd-a97d-53a754a45110"). InnerVolumeSpecName "workload-certs". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Jul 16 01:12:38.484148 ip-10-0-143-200 kubenswrapper[2572]: I0716 01:12:38.484075 2572 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/301fffd1-42b5-4cfd-a97d-53a754a45110-istiod-ca-cert" (OuterVolumeSpecName: "istiod-ca-cert") pod "301fffd1-42b5-4cfd-a97d-53a754a45110" (UID: "301fffd1-42b5-4cfd-a97d-53a754a45110"). InnerVolumeSpecName "istiod-ca-cert". PluginName "kubernetes.io/configmap", VolumeGIDValue "" Jul 16 01:12:38.484931 ip-10-0-143-200 kubenswrapper[2572]: I0716 01:12:38.484901 2572 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/301fffd1-42b5-4cfd-a97d-53a754a45110-istio-data" (OuterVolumeSpecName: "istio-data") pod "301fffd1-42b5-4cfd-a97d-53a754a45110" (UID: "301fffd1-42b5-4cfd-a97d-53a754a45110"). InnerVolumeSpecName "istio-data". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Jul 16 01:12:38.485089 ip-10-0-143-200 kubenswrapper[2572]: I0716 01:12:38.485049 2572 reconciler_common.go:299] "Volume detached for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/301fffd1-42b5-4cfd-a97d-53a754a45110-istiod-ca-cert\") on node \"ip-10-0-143-200.ec2.internal\" DevicePath \"\"" Jul 16 01:12:38.485089 ip-10-0-143-200 kubenswrapper[2572]: I0716 01:12:38.485072 2572 reconciler_common.go:299] "Volume detached for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/301fffd1-42b5-4cfd-a97d-53a754a45110-workload-certs\") on node \"ip-10-0-143-200.ec2.internal\" DevicePath \"\"" Jul 16 01:12:38.485089 ip-10-0-143-200 kubenswrapper[2572]: I0716 01:12:38.485086 2572 reconciler_common.go:299] "Volume detached for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/301fffd1-42b5-4cfd-a97d-53a754a45110-istio-data\") on node \"ip-10-0-143-200.ec2.internal\" DevicePath \"\"" Jul 16 01:12:38.485256 ip-10-0-143-200 kubenswrapper[2572]: I0716 01:12:38.485098 2572 reconciler_common.go:299] "Volume detached for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/301fffd1-42b5-4cfd-a97d-53a754a45110-credential-socket\") on node \"ip-10-0-143-200.ec2.internal\" DevicePath \"\"" Jul 16 01:12:38.485256 ip-10-0-143-200 kubenswrapper[2572]: I0716 01:12:38.485114 2572 reconciler_common.go:299] "Volume detached for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/301fffd1-42b5-4cfd-a97d-53a754a45110-workload-socket\") on node \"ip-10-0-143-200.ec2.internal\" DevicePath \"\"" Jul 16 01:12:38.485992 ip-10-0-143-200 kubenswrapper[2572]: I0716 01:12:38.485961 2572 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/301fffd1-42b5-4cfd-a97d-53a754a45110-istio-token" (OuterVolumeSpecName: "istio-token") pod "301fffd1-42b5-4cfd-a97d-53a754a45110" (UID: "301fffd1-42b5-4cfd-a97d-53a754a45110"). InnerVolumeSpecName "istio-token". PluginName "kubernetes.io/projected", VolumeGIDValue "" Jul 16 01:12:38.486126 ip-10-0-143-200 kubenswrapper[2572]: I0716 01:12:38.486047 2572 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/downward-api/301fffd1-42b5-4cfd-a97d-53a754a45110-istio-podinfo" (OuterVolumeSpecName: "istio-podinfo") pod "301fffd1-42b5-4cfd-a97d-53a754a45110" (UID: "301fffd1-42b5-4cfd-a97d-53a754a45110"). InnerVolumeSpecName "istio-podinfo". PluginName "kubernetes.io/downward-api", VolumeGIDValue "" Jul 16 01:12:38.486192 ip-10-0-143-200 kubenswrapper[2572]: I0716 01:12:38.486150 2572 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/301fffd1-42b5-4cfd-a97d-53a754a45110-kube-api-access-c7jpq" (OuterVolumeSpecName: "kube-api-access-c7jpq") pod "301fffd1-42b5-4cfd-a97d-53a754a45110" (UID: "301fffd1-42b5-4cfd-a97d-53a754a45110"). InnerVolumeSpecName "kube-api-access-c7jpq". PluginName "kubernetes.io/projected", VolumeGIDValue "" Jul 16 01:12:38.486333 ip-10-0-143-200 kubenswrapper[2572]: I0716 01:12:38.486317 2572 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/301fffd1-42b5-4cfd-a97d-53a754a45110-istio-envoy" (OuterVolumeSpecName: "istio-envoy") pod "301fffd1-42b5-4cfd-a97d-53a754a45110" (UID: "301fffd1-42b5-4cfd-a97d-53a754a45110"). InnerVolumeSpecName "istio-envoy". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Jul 16 01:12:38.553250 ip-10-0-143-200 kubenswrapper[2572]: I0716 01:12:38.553211 2572 generic.go:356] "Generic (PLEG): container finished" podID="301fffd1-42b5-4cfd-a97d-53a754a45110" containerID="c89849f625eec5d523a6cf4d43bb2c2c0c838701e78dcb102026ab8521ab29ad" exitCode=0 Jul 16 01:12:38.553451 ip-10-0-143-200 kubenswrapper[2572]: I0716 01:12:38.553261 2572 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-inf-b-e5854b-openshift-default-77bc9577d9-8qdwp" Jul 16 01:12:38.553451 ip-10-0-143-200 kubenswrapper[2572]: I0716 01:12:38.553304 2572 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-inf-b-e5854b-openshift-default-77bc9577d9-8qdwp" event={"ID":"301fffd1-42b5-4cfd-a97d-53a754a45110","Type":"ContainerDied","Data":"c89849f625eec5d523a6cf4d43bb2c2c0c838701e78dcb102026ab8521ab29ad"} Jul 16 01:12:38.553451 ip-10-0-143-200 kubenswrapper[2572]: I0716 01:12:38.553368 2572 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-inf-b-e5854b-openshift-default-77bc9577d9-8qdwp" event={"ID":"301fffd1-42b5-4cfd-a97d-53a754a45110","Type":"ContainerDied","Data":"68e00eefc46a5c043efa876084f3455d646d8c325a276017e0abe0a093706c54"} Jul 16 01:12:38.553451 ip-10-0-143-200 kubenswrapper[2572]: I0716 01:12:38.553400 2572 scope.go:117] "RemoveContainer" containerID="c89849f625eec5d523a6cf4d43bb2c2c0c838701e78dcb102026ab8521ab29ad" Jul 16 01:12:38.563948 ip-10-0-143-200 kubenswrapper[2572]: I0716 01:12:38.563931 2572 scope.go:117] "RemoveContainer" containerID="09ee4e1110f8f4521d6a38d1ac8b9e163a433ce54997cc7a139ba67c0247d056" Jul 16 01:12:38.572336 ip-10-0-143-200 kubenswrapper[2572]: I0716 01:12:38.572320 2572 scope.go:117] "RemoveContainer" containerID="c89849f625eec5d523a6cf4d43bb2c2c0c838701e78dcb102026ab8521ab29ad" Jul 16 01:12:38.572585 ip-10-0-143-200 kubenswrapper[2572]: E0716 01:12:38.572568 2572 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"c89849f625eec5d523a6cf4d43bb2c2c0c838701e78dcb102026ab8521ab29ad\": container with ID starting with c89849f625eec5d523a6cf4d43bb2c2c0c838701e78dcb102026ab8521ab29ad not found: ID does not exist" containerID="c89849f625eec5d523a6cf4d43bb2c2c0c838701e78dcb102026ab8521ab29ad" Jul 16 01:12:38.572742 ip-10-0-143-200 kubenswrapper[2572]: I0716 01:12:38.572593 2572 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"c89849f625eec5d523a6cf4d43bb2c2c0c838701e78dcb102026ab8521ab29ad"} err="failed to get container status \"c89849f625eec5d523a6cf4d43bb2c2c0c838701e78dcb102026ab8521ab29ad\": rpc error: code = NotFound desc = could not find container \"c89849f625eec5d523a6cf4d43bb2c2c0c838701e78dcb102026ab8521ab29ad\": container with ID starting with c89849f625eec5d523a6cf4d43bb2c2c0c838701e78dcb102026ab8521ab29ad not found: ID does not exist" Jul 16 01:12:38.572742 ip-10-0-143-200 kubenswrapper[2572]: I0716 01:12:38.572608 2572 scope.go:117] "RemoveContainer" containerID="09ee4e1110f8f4521d6a38d1ac8b9e163a433ce54997cc7a139ba67c0247d056" Jul 16 01:12:38.572919 ip-10-0-143-200 kubenswrapper[2572]: E0716 01:12:38.572903 2572 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"09ee4e1110f8f4521d6a38d1ac8b9e163a433ce54997cc7a139ba67c0247d056\": container with ID starting with 09ee4e1110f8f4521d6a38d1ac8b9e163a433ce54997cc7a139ba67c0247d056 not found: ID does not exist" containerID="09ee4e1110f8f4521d6a38d1ac8b9e163a433ce54997cc7a139ba67c0247d056" Jul 16 01:12:38.572964 ip-10-0-143-200 kubenswrapper[2572]: I0716 01:12:38.572924 2572 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"09ee4e1110f8f4521d6a38d1ac8b9e163a433ce54997cc7a139ba67c0247d056"} err="failed to get container status \"09ee4e1110f8f4521d6a38d1ac8b9e163a433ce54997cc7a139ba67c0247d056\": rpc error: code = NotFound desc = could not find container \"09ee4e1110f8f4521d6a38d1ac8b9e163a433ce54997cc7a139ba67c0247d056\": container with ID starting with 09ee4e1110f8f4521d6a38d1ac8b9e163a433ce54997cc7a139ba67c0247d056 not found: ID does not exist" Jul 16 01:12:38.581152 ip-10-0-143-200 kubenswrapper[2572]: I0716 01:12:38.581129 2572 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-ingress/e2e-inf-b-e5854b-openshift-default-77bc9577d9-8qdwp"] Jul 16 01:12:38.584945 ip-10-0-143-200 kubenswrapper[2572]: I0716 01:12:38.584924 2572 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["openshift-ingress/e2e-inf-b-e5854b-openshift-default-77bc9577d9-8qdwp"] Jul 16 01:12:38.586164 ip-10-0-143-200 kubenswrapper[2572]: I0716 01:12:38.586145 2572 reconciler_common.go:299] "Volume detached for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/301fffd1-42b5-4cfd-a97d-53a754a45110-istio-token\") on node \"ip-10-0-143-200.ec2.internal\" DevicePath \"\"" Jul 16 01:12:38.586228 ip-10-0-143-200 kubenswrapper[2572]: I0716 01:12:38.586168 2572 reconciler_common.go:299] "Volume detached for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/301fffd1-42b5-4cfd-a97d-53a754a45110-istio-podinfo\") on node \"ip-10-0-143-200.ec2.internal\" DevicePath \"\"" Jul 16 01:12:38.586228 ip-10-0-143-200 kubenswrapper[2572]: I0716 01:12:38.586177 2572 reconciler_common.go:299] "Volume detached for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/301fffd1-42b5-4cfd-a97d-53a754a45110-istio-envoy\") on node \"ip-10-0-143-200.ec2.internal\" DevicePath \"\"" Jul 16 01:12:38.586228 ip-10-0-143-200 kubenswrapper[2572]: I0716 01:12:38.586187 2572 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-c7jpq\" (UniqueName: \"kubernetes.io/projected/301fffd1-42b5-4cfd-a97d-53a754a45110-kube-api-access-c7jpq\") on node \"ip-10-0-143-200.ec2.internal\" DevicePath \"\"" Jul 16 01:12:39.448979 ip-10-0-143-200 kubenswrapper[2572]: I0716 01:12:39.448945 2572 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="301fffd1-42b5-4cfd-a97d-53a754a45110" path="/var/lib/kubelet/pods/301fffd1-42b5-4cfd-a97d-53a754a45110/volumes" Jul 16 01:12:41.009362 ip-10-0-143-200 kubenswrapper[2572]: I0716 01:12:41.009321 2572 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9gnmq_3a7e030d-fad8-480d-9dc6-fe5057048cf9/konnectivity-agent/0.log" limit=1024 Jul 16 01:12:41.509087 ip-10-0-143-200 kubenswrapper[2572]: I0716 01:12:41.509049 2572 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9gnmq_3a7e030d-fad8-480d-9dc6-fe5057048cf9/konnectivity-agent/0.log" limit=1024 Jul 16 01:12:41.804683 ip-10-0-143-200 kubenswrapper[2572]: I0716 01:12:41.804568 2572 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-ingress/e2e-api-b-2814a2-openshift-default-9ccd5f9d5-rh5vs"] Jul 16 01:12:41.805924 ip-10-0-143-200 kubenswrapper[2572]: I0716 01:12:41.805901 2572 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="301fffd1-42b5-4cfd-a97d-53a754a45110" containerName="istio-proxy" Jul 16 01:12:41.806180 ip-10-0-143-200 kubenswrapper[2572]: I0716 01:12:41.806151 2572 state_mem.go:107] "Deleted CPUSet assignment" podUID="301fffd1-42b5-4cfd-a97d-53a754a45110" containerName="istio-proxy" Jul 16 01:12:41.806486 ip-10-0-143-200 kubenswrapper[2572]: I0716 01:12:41.806473 2572 memory_manager.go:368] "RemoveStaleState removing state" podUID="301fffd1-42b5-4cfd-a97d-53a754a45110" containerName="istio-proxy" Jul 16 01:12:41.828670 ip-10-0-143-200 kubenswrapper[2572]: I0716 01:12:41.828620 2572 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/e2e-api-b-2814a2-openshift-default-9ccd5f9d5-rh5vs"] Jul 16 01:12:41.828906 ip-10-0-143-200 kubenswrapper[2572]: I0716 01:12:41.828740 2572 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-api-b-2814a2-openshift-default-9ccd5f9d5-rh5vs" Jul 16 01:12:41.831711 ip-10-0-143-200 kubenswrapper[2572]: I0716 01:12:41.831681 2572 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress\"/\"e2e-api-b-2814a2-openshift-default-dockercfg-x4mr5\"" Jul 16 01:12:41.844561 ip-10-0-143-200 kubenswrapper[2572]: I0716 01:12:41.844532 2572 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9gnmq_3a7e030d-fad8-480d-9dc6-fe5057048cf9/konnectivity-agent/0.log" limit=1024 Jul 16 01:12:41.924046 ip-10-0-143-200 kubenswrapper[2572]: I0716 01:12:41.924008 2572 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/57868c2b-17b7-49f4-a452-1b1db8a9e706-istiod-ca-cert\") pod \"e2e-api-b-2814a2-openshift-default-9ccd5f9d5-rh5vs\" (UID: \"57868c2b-17b7-49f4-a452-1b1db8a9e706\") " pod="openshift-ingress/e2e-api-b-2814a2-openshift-default-9ccd5f9d5-rh5vs" Jul 16 01:12:41.924214 ip-10-0-143-200 kubenswrapper[2572]: I0716 01:12:41.924151 2572 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/57868c2b-17b7-49f4-a452-1b1db8a9e706-istio-data\") pod \"e2e-api-b-2814a2-openshift-default-9ccd5f9d5-rh5vs\" (UID: \"57868c2b-17b7-49f4-a452-1b1db8a9e706\") " pod="openshift-ingress/e2e-api-b-2814a2-openshift-default-9ccd5f9d5-rh5vs" Jul 16 01:12:41.924305 ip-10-0-143-200 kubenswrapper[2572]: I0716 01:12:41.924228 2572 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/57868c2b-17b7-49f4-a452-1b1db8a9e706-credential-socket\") pod \"e2e-api-b-2814a2-openshift-default-9ccd5f9d5-rh5vs\" (UID: \"57868c2b-17b7-49f4-a452-1b1db8a9e706\") " pod="openshift-ingress/e2e-api-b-2814a2-openshift-default-9ccd5f9d5-rh5vs" Jul 16 01:12:41.924376 ip-10-0-143-200 kubenswrapper[2572]: I0716 01:12:41.924360 2572 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/57868c2b-17b7-49f4-a452-1b1db8a9e706-istio-podinfo\") pod \"e2e-api-b-2814a2-openshift-default-9ccd5f9d5-rh5vs\" (UID: \"57868c2b-17b7-49f4-a452-1b1db8a9e706\") " pod="openshift-ingress/e2e-api-b-2814a2-openshift-default-9ccd5f9d5-rh5vs" Jul 16 01:12:41.924438 ip-10-0-143-200 kubenswrapper[2572]: I0716 01:12:41.924391 2572 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/57868c2b-17b7-49f4-a452-1b1db8a9e706-workload-certs\") pod \"e2e-api-b-2814a2-openshift-default-9ccd5f9d5-rh5vs\" (UID: \"57868c2b-17b7-49f4-a452-1b1db8a9e706\") " pod="openshift-ingress/e2e-api-b-2814a2-openshift-default-9ccd5f9d5-rh5vs" Jul 16 01:12:41.924438 ip-10-0-143-200 kubenswrapper[2572]: I0716 01:12:41.924426 2572 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-2h85f\" (UniqueName: \"kubernetes.io/projected/57868c2b-17b7-49f4-a452-1b1db8a9e706-kube-api-access-2h85f\") pod \"e2e-api-b-2814a2-openshift-default-9ccd5f9d5-rh5vs\" (UID: \"57868c2b-17b7-49f4-a452-1b1db8a9e706\") " pod="openshift-ingress/e2e-api-b-2814a2-openshift-default-9ccd5f9d5-rh5vs" Jul 16 01:12:41.924573 ip-10-0-143-200 kubenswrapper[2572]: I0716 01:12:41.924555 2572 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/57868c2b-17b7-49f4-a452-1b1db8a9e706-istio-envoy\") pod \"e2e-api-b-2814a2-openshift-default-9ccd5f9d5-rh5vs\" (UID: \"57868c2b-17b7-49f4-a452-1b1db8a9e706\") " pod="openshift-ingress/e2e-api-b-2814a2-openshift-default-9ccd5f9d5-rh5vs" Jul 16 01:12:41.924661 ip-10-0-143-200 kubenswrapper[2572]: I0716 01:12:41.924587 2572 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/57868c2b-17b7-49f4-a452-1b1db8a9e706-workload-socket\") pod \"e2e-api-b-2814a2-openshift-default-9ccd5f9d5-rh5vs\" (UID: \"57868c2b-17b7-49f4-a452-1b1db8a9e706\") " pod="openshift-ingress/e2e-api-b-2814a2-openshift-default-9ccd5f9d5-rh5vs" Jul 16 01:12:41.924725 ip-10-0-143-200 kubenswrapper[2572]: I0716 01:12:41.924663 2572 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/57868c2b-17b7-49f4-a452-1b1db8a9e706-istio-token\") pod \"e2e-api-b-2814a2-openshift-default-9ccd5f9d5-rh5vs\" (UID: \"57868c2b-17b7-49f4-a452-1b1db8a9e706\") " pod="openshift-ingress/e2e-api-b-2814a2-openshift-default-9ccd5f9d5-rh5vs" Jul 16 01:12:42.025263 ip-10-0-143-200 kubenswrapper[2572]: I0716 01:12:42.025229 2572 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/57868c2b-17b7-49f4-a452-1b1db8a9e706-credential-socket\") pod \"e2e-api-b-2814a2-openshift-default-9ccd5f9d5-rh5vs\" (UID: \"57868c2b-17b7-49f4-a452-1b1db8a9e706\") " pod="openshift-ingress/e2e-api-b-2814a2-openshift-default-9ccd5f9d5-rh5vs" Jul 16 01:12:42.026243 ip-10-0-143-200 kubenswrapper[2572]: I0716 01:12:42.025554 2572 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/57868c2b-17b7-49f4-a452-1b1db8a9e706-credential-socket\") pod \"e2e-api-b-2814a2-openshift-default-9ccd5f9d5-rh5vs\" (UID: \"57868c2b-17b7-49f4-a452-1b1db8a9e706\") " pod="openshift-ingress/e2e-api-b-2814a2-openshift-default-9ccd5f9d5-rh5vs" Jul 16 01:12:42.026243 ip-10-0-143-200 kubenswrapper[2572]: I0716 01:12:42.025865 2572 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/57868c2b-17b7-49f4-a452-1b1db8a9e706-istio-podinfo\") pod \"e2e-api-b-2814a2-openshift-default-9ccd5f9d5-rh5vs\" (UID: \"57868c2b-17b7-49f4-a452-1b1db8a9e706\") " pod="openshift-ingress/e2e-api-b-2814a2-openshift-default-9ccd5f9d5-rh5vs" Jul 16 01:12:42.026243 ip-10-0-143-200 kubenswrapper[2572]: I0716 01:12:42.025912 2572 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/57868c2b-17b7-49f4-a452-1b1db8a9e706-workload-certs\") pod \"e2e-api-b-2814a2-openshift-default-9ccd5f9d5-rh5vs\" (UID: \"57868c2b-17b7-49f4-a452-1b1db8a9e706\") " pod="openshift-ingress/e2e-api-b-2814a2-openshift-default-9ccd5f9d5-rh5vs" Jul 16 01:12:42.026243 ip-10-0-143-200 kubenswrapper[2572]: I0716 01:12:42.025954 2572 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-2h85f\" (UniqueName: \"kubernetes.io/projected/57868c2b-17b7-49f4-a452-1b1db8a9e706-kube-api-access-2h85f\") pod \"e2e-api-b-2814a2-openshift-default-9ccd5f9d5-rh5vs\" (UID: \"57868c2b-17b7-49f4-a452-1b1db8a9e706\") " pod="openshift-ingress/e2e-api-b-2814a2-openshift-default-9ccd5f9d5-rh5vs" Jul 16 01:12:42.026243 ip-10-0-143-200 kubenswrapper[2572]: I0716 01:12:42.026023 2572 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/57868c2b-17b7-49f4-a452-1b1db8a9e706-istio-envoy\") pod \"e2e-api-b-2814a2-openshift-default-9ccd5f9d5-rh5vs\" (UID: \"57868c2b-17b7-49f4-a452-1b1db8a9e706\") " pod="openshift-ingress/e2e-api-b-2814a2-openshift-default-9ccd5f9d5-rh5vs" Jul 16 01:12:42.026243 ip-10-0-143-200 kubenswrapper[2572]: I0716 01:12:42.026062 2572 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/57868c2b-17b7-49f4-a452-1b1db8a9e706-workload-socket\") pod \"e2e-api-b-2814a2-openshift-default-9ccd5f9d5-rh5vs\" (UID: \"57868c2b-17b7-49f4-a452-1b1db8a9e706\") " pod="openshift-ingress/e2e-api-b-2814a2-openshift-default-9ccd5f9d5-rh5vs" Jul 16 01:12:42.026243 ip-10-0-143-200 kubenswrapper[2572]: I0716 01:12:42.026129 2572 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/57868c2b-17b7-49f4-a452-1b1db8a9e706-istio-token\") pod \"e2e-api-b-2814a2-openshift-default-9ccd5f9d5-rh5vs\" (UID: \"57868c2b-17b7-49f4-a452-1b1db8a9e706\") " pod="openshift-ingress/e2e-api-b-2814a2-openshift-default-9ccd5f9d5-rh5vs" Jul 16 01:12:42.026243 ip-10-0-143-200 kubenswrapper[2572]: I0716 01:12:42.026171 2572 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/57868c2b-17b7-49f4-a452-1b1db8a9e706-istiod-ca-cert\") pod \"e2e-api-b-2814a2-openshift-default-9ccd5f9d5-rh5vs\" (UID: \"57868c2b-17b7-49f4-a452-1b1db8a9e706\") " pod="openshift-ingress/e2e-api-b-2814a2-openshift-default-9ccd5f9d5-rh5vs" Jul 16 01:12:42.026243 ip-10-0-143-200 kubenswrapper[2572]: I0716 01:12:42.026219 2572 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/57868c2b-17b7-49f4-a452-1b1db8a9e706-istio-data\") pod \"e2e-api-b-2814a2-openshift-default-9ccd5f9d5-rh5vs\" (UID: \"57868c2b-17b7-49f4-a452-1b1db8a9e706\") " pod="openshift-ingress/e2e-api-b-2814a2-openshift-default-9ccd5f9d5-rh5vs" Jul 16 01:12:42.026885 ip-10-0-143-200 kubenswrapper[2572]: I0716 01:12:42.026567 2572 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/57868c2b-17b7-49f4-a452-1b1db8a9e706-workload-socket\") pod \"e2e-api-b-2814a2-openshift-default-9ccd5f9d5-rh5vs\" (UID: \"57868c2b-17b7-49f4-a452-1b1db8a9e706\") " pod="openshift-ingress/e2e-api-b-2814a2-openshift-default-9ccd5f9d5-rh5vs" Jul 16 01:12:42.026885 ip-10-0-143-200 kubenswrapper[2572]: I0716 01:12:42.026623 2572 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/57868c2b-17b7-49f4-a452-1b1db8a9e706-istio-data\") pod \"e2e-api-b-2814a2-openshift-default-9ccd5f9d5-rh5vs\" (UID: \"57868c2b-17b7-49f4-a452-1b1db8a9e706\") " pod="openshift-ingress/e2e-api-b-2814a2-openshift-default-9ccd5f9d5-rh5vs" Jul 16 01:12:42.027283 ip-10-0-143-200 kubenswrapper[2572]: I0716 01:12:42.027256 2572 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/57868c2b-17b7-49f4-a452-1b1db8a9e706-workload-certs\") pod \"e2e-api-b-2814a2-openshift-default-9ccd5f9d5-rh5vs\" (UID: \"57868c2b-17b7-49f4-a452-1b1db8a9e706\") " pod="openshift-ingress/e2e-api-b-2814a2-openshift-default-9ccd5f9d5-rh5vs" Jul 16 01:12:42.027551 ip-10-0-143-200 kubenswrapper[2572]: I0716 01:12:42.027530 2572 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/57868c2b-17b7-49f4-a452-1b1db8a9e706-istiod-ca-cert\") pod \"e2e-api-b-2814a2-openshift-default-9ccd5f9d5-rh5vs\" (UID: \"57868c2b-17b7-49f4-a452-1b1db8a9e706\") " pod="openshift-ingress/e2e-api-b-2814a2-openshift-default-9ccd5f9d5-rh5vs" Jul 16 01:12:42.029051 ip-10-0-143-200 kubenswrapper[2572]: I0716 01:12:42.029028 2572 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/57868c2b-17b7-49f4-a452-1b1db8a9e706-istio-envoy\") pod \"e2e-api-b-2814a2-openshift-default-9ccd5f9d5-rh5vs\" (UID: \"57868c2b-17b7-49f4-a452-1b1db8a9e706\") " pod="openshift-ingress/e2e-api-b-2814a2-openshift-default-9ccd5f9d5-rh5vs" Jul 16 01:12:42.029159 ip-10-0-143-200 kubenswrapper[2572]: I0716 01:12:42.029139 2572 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/57868c2b-17b7-49f4-a452-1b1db8a9e706-istio-podinfo\") pod \"e2e-api-b-2814a2-openshift-default-9ccd5f9d5-rh5vs\" (UID: \"57868c2b-17b7-49f4-a452-1b1db8a9e706\") " pod="openshift-ingress/e2e-api-b-2814a2-openshift-default-9ccd5f9d5-rh5vs" Jul 16 01:12:42.035582 ip-10-0-143-200 kubenswrapper[2572]: I0716 01:12:42.035553 2572 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-2h85f\" (UniqueName: \"kubernetes.io/projected/57868c2b-17b7-49f4-a452-1b1db8a9e706-kube-api-access-2h85f\") pod \"e2e-api-b-2814a2-openshift-default-9ccd5f9d5-rh5vs\" (UID: \"57868c2b-17b7-49f4-a452-1b1db8a9e706\") " pod="openshift-ingress/e2e-api-b-2814a2-openshift-default-9ccd5f9d5-rh5vs" Jul 16 01:12:42.036037 ip-10-0-143-200 kubenswrapper[2572]: I0716 01:12:42.036017 2572 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/57868c2b-17b7-49f4-a452-1b1db8a9e706-istio-token\") pod \"e2e-api-b-2814a2-openshift-default-9ccd5f9d5-rh5vs\" (UID: \"57868c2b-17b7-49f4-a452-1b1db8a9e706\") " pod="openshift-ingress/e2e-api-b-2814a2-openshift-default-9ccd5f9d5-rh5vs" Jul 16 01:12:42.142960 ip-10-0-143-200 kubenswrapper[2572]: I0716 01:12:42.142858 2572 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-api-b-2814a2-openshift-default-9ccd5f9d5-rh5vs" Jul 16 01:12:42.236204 ip-10-0-143-200 kubenswrapper[2572]: I0716 01:12:42.236173 2572 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9gnmq_3a7e030d-fad8-480d-9dc6-fe5057048cf9/konnectivity-agent/0.log" limit=1024 Jul 16 01:12:42.353111 ip-10-0-143-200 kubenswrapper[2572]: I0716 01:12:42.353079 2572 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/e2e-api-b-2814a2-openshift-default-9ccd5f9d5-rh5vs"] Jul 16 01:12:42.354763 ip-10-0-143-200 kubenswrapper[2572]: W0716 01:12:42.354729 2572 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod57868c2b_17b7_49f4_a452_1b1db8a9e706.slice/crio-57a19b6ca02b5166dfdb76685eae602dcf8fa821d936df8f1bd839d188545064 WatchSource:0}: Error finding container 57a19b6ca02b5166dfdb76685eae602dcf8fa821d936df8f1bd839d188545064: Status 404 returned error can't find the container with id 57a19b6ca02b5166dfdb76685eae602dcf8fa821d936df8f1bd839d188545064 Jul 16 01:12:42.357077 ip-10-0-143-200 kubenswrapper[2572]: I0716 01:12:42.357036 2572 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236228Ki","pods":"250"} Jul 16 01:12:42.357161 ip-10-0-143-200 kubenswrapper[2572]: I0716 01:12:42.357125 2572 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236228Ki","pods":"250"} Jul 16 01:12:42.357201 ip-10-0-143-200 kubenswrapper[2572]: I0716 01:12:42.357173 2572 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236228Ki","pods":"250"} Jul 16 01:12:42.578677 ip-10-0-143-200 kubenswrapper[2572]: I0716 01:12:42.578642 2572 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-api-b-2814a2-openshift-default-9ccd5f9d5-rh5vs" event={"ID":"57868c2b-17b7-49f4-a452-1b1db8a9e706","Type":"ContainerStarted","Data":"740ab1b2a1f513b74b278701db9f35b52ea7b72b90df928d45d794c4159f34e7"} Jul 16 01:12:42.578848 ip-10-0-143-200 kubenswrapper[2572]: I0716 01:12:42.578696 2572 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-api-b-2814a2-openshift-default-9ccd5f9d5-rh5vs" event={"ID":"57868c2b-17b7-49f4-a452-1b1db8a9e706","Type":"ContainerStarted","Data":"57a19b6ca02b5166dfdb76685eae602dcf8fa821d936df8f1bd839d188545064"} Jul 16 01:12:42.602605 ip-10-0-143-200 kubenswrapper[2572]: I0716 01:12:42.602537 2572 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-ingress/e2e-api-b-2814a2-openshift-default-9ccd5f9d5-rh5vs" podStartSLOduration=1.602513597 podStartE2EDuration="1.602513597s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-16 01:12:41 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-16 01:12:42.601072031 +0000 UTC m=+1967.709826707" watchObservedRunningTime="2026-07-16 01:12:42.602513597 +0000 UTC m=+1967.711268272" Jul 16 01:12:43.143566 ip-10-0-143-200 kubenswrapper[2572]: I0716 01:12:43.143530 2572 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="openshift-ingress/e2e-api-b-2814a2-openshift-default-9ccd5f9d5-rh5vs" Jul 16 01:12:43.148656 ip-10-0-143-200 kubenswrapper[2572]: I0716 01:12:43.148612 2572 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="openshift-ingress/e2e-api-b-2814a2-openshift-default-9ccd5f9d5-rh5vs" Jul 16 01:12:43.577806 ip-10-0-143-200 kubenswrapper[2572]: I0716 01:12:43.577770 2572 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9gnmq_3a7e030d-fad8-480d-9dc6-fe5057048cf9/konnectivity-agent/0.log" limit=1024 Jul 16 01:12:43.585970 ip-10-0-143-200 kubenswrapper[2572]: I0716 01:12:43.585944 2572 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/e2e-api-b-2814a2-openshift-default-9ccd5f9d5-rh5vs" Jul 16 01:12:43.587204 ip-10-0-143-200 kubenswrapper[2572]: I0716 01:12:43.587157 2572 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/e2e-api-b-2814a2-openshift-default-9ccd5f9d5-rh5vs" Jul 16 01:12:43.701325 ip-10-0-143-200 kubenswrapper[2572]: I0716 01:12:43.701289 2572 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9gnmq_3a7e030d-fad8-480d-9dc6-fe5057048cf9/konnectivity-agent/0.log" limit=1024 Jul 16 01:12:43.787919 ip-10-0-143-200 kubenswrapper[2572]: I0716 01:12:43.787876 2572 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9gnmq_3a7e030d-fad8-480d-9dc6-fe5057048cf9/konnectivity-agent/0.log" limit=1024 Jul 16 01:12:44.357623 ip-10-0-143-200 kubenswrapper[2572]: I0716 01:12:44.357577 2572 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9gnmq_3a7e030d-fad8-480d-9dc6-fe5057048cf9/konnectivity-agent/0.log" limit=1024 Jul 16 01:12:44.496433 ip-10-0-143-200 kubenswrapper[2572]: I0716 01:12:44.496387 2572 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9gnmq_3a7e030d-fad8-480d-9dc6-fe5057048cf9/konnectivity-agent/0.log" limit=1024 Jul 16 01:12:44.580376 ip-10-0-143-200 kubenswrapper[2572]: I0716 01:12:44.580341 2572 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9gnmq_3a7e030d-fad8-480d-9dc6-fe5057048cf9/konnectivity-agent/0.log" limit=1024 Jul 16 01:12:46.605961 ip-10-0-143-200 kubenswrapper[2572]: I0716 01:12:46.605923 2572 generic.go:356] "Generic (PLEG): container finished" podID="b6672fdb-54f4-4adc-844d-4072c3bf9876" containerID="3141b387235d12c5cfb3c8687aa7f55d3f00858bfda0c06e0f08118844b5508e" exitCode=0 Jul 16 01:12:46.606469 ip-10-0-143-200 kubenswrapper[2572]: I0716 01:12:46.605993 2572 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-api-a-f1f83d-openshift-default-7f95675bf9-2bb7g" event={"ID":"b6672fdb-54f4-4adc-844d-4072c3bf9876","Type":"ContainerDied","Data":"3141b387235d12c5cfb3c8687aa7f55d3f00858bfda0c06e0f08118844b5508e"} Jul 16 01:12:46.606533 ip-10-0-143-200 kubenswrapper[2572]: I0716 01:12:46.606495 2572 scope.go:117] "RemoveContainer" containerID="3141b387235d12c5cfb3c8687aa7f55d3f00858bfda0c06e0f08118844b5508e" Jul 16 01:12:46.607547 ip-10-0-143-200 kubenswrapper[2572]: I0716 01:12:46.607521 2572 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236228Ki","pods":"250"} Jul 16 01:12:46.607602 ip-10-0-143-200 kubenswrapper[2572]: I0716 01:12:46.607589 2572 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236228Ki","pods":"250"} Jul 16 01:12:46.607655 ip-10-0-143-200 kubenswrapper[2572]: I0716 01:12:46.607640 2572 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236228Ki","pods":"250"} Jul 16 01:12:47.614896 ip-10-0-143-200 kubenswrapper[2572]: I0716 01:12:47.614804 2572 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-api-a-f1f83d-openshift-default-7f95675bf9-2bb7g" event={"ID":"b6672fdb-54f4-4adc-844d-4072c3bf9876","Type":"ContainerStarted","Data":"e431baa23c143f8982dc4998f4241e7cd3be3365ca0e297961635bd56c744f7c"} Jul 16 01:12:48.212842 ip-10-0-143-200 kubenswrapper[2572]: I0716 01:12:48.212800 2572 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="openshift-ingress/e2e-api-a-f1f83d-openshift-default-7f95675bf9-2bb7g" Jul 16 01:12:48.213919 ip-10-0-143-200 kubenswrapper[2572]: I0716 01:12:48.213889 2572 patch_prober.go:28] interesting pod/e2e-api-a-f1f83d-openshift-default-7f95675bf9-2bb7g container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.132.0.59:15021/healthz/ready\": dial tcp 10.132.0.59:15021: connect: connection refused" start-of-body= Jul 16 01:12:48.214065 ip-10-0-143-200 kubenswrapper[2572]: I0716 01:12:48.213939 2572 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-api-a-f1f83d-openshift-default-7f95675bf9-2bb7g" podUID="b6672fdb-54f4-4adc-844d-4072c3bf9876" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.132.0.59:15021/healthz/ready\": dial tcp 10.132.0.59:15021: connect: connection refused" Jul 16 01:12:49.053957 ip-10-0-143-200 kubenswrapper[2572]: I0716 01:12:49.053921 2572 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9gnmq_3a7e030d-fad8-480d-9dc6-fe5057048cf9/konnectivity-agent/0.log" limit=1024 Jul 16 01:12:49.213083 ip-10-0-143-200 kubenswrapper[2572]: I0716 01:12:49.213040 2572 patch_prober.go:28] interesting pod/e2e-api-a-f1f83d-openshift-default-7f95675bf9-2bb7g container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.132.0.59:15021/healthz/ready\": dial tcp 10.132.0.59:15021: connect: connection refused" start-of-body= Jul 16 01:12:49.213269 ip-10-0-143-200 kubenswrapper[2572]: I0716 01:12:49.213120 2572 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-api-a-f1f83d-openshift-default-7f95675bf9-2bb7g" podUID="b6672fdb-54f4-4adc-844d-4072c3bf9876" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.132.0.59:15021/healthz/ready\": dial tcp 10.132.0.59:15021: connect: connection refused" Jul 16 01:12:50.212583 ip-10-0-143-200 kubenswrapper[2572]: I0716 01:12:50.212547 2572 patch_prober.go:28] interesting pod/e2e-api-a-f1f83d-openshift-default-7f95675bf9-2bb7g container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.132.0.59:15021/healthz/ready\": dial tcp 10.132.0.59:15021: connect: connection refused" start-of-body= Jul 16 01:12:50.213092 ip-10-0-143-200 kubenswrapper[2572]: I0716 01:12:50.213054 2572 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-api-a-f1f83d-openshift-default-7f95675bf9-2bb7g" podUID="b6672fdb-54f4-4adc-844d-4072c3bf9876" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.132.0.59:15021/healthz/ready\": dial tcp 10.132.0.59:15021: connect: connection refused" Jul 16 01:12:50.536089 ip-10-0-143-200 kubenswrapper[2572]: I0716 01:12:50.536006 2572 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/e2e-api-a-f1f83d-openshift-default-7f95675bf9-2bb7g" Jul 16 01:12:51.212502 ip-10-0-143-200 kubenswrapper[2572]: I0716 01:12:51.212459 2572 patch_prober.go:28] interesting pod/e2e-api-a-f1f83d-openshift-default-7f95675bf9-2bb7g container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.132.0.59:15021/healthz/ready\": dial tcp 10.132.0.59:15021: connect: connection refused" start-of-body= Jul 16 01:12:51.212945 ip-10-0-143-200 kubenswrapper[2572]: I0716 01:12:51.212907 2572 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-api-a-f1f83d-openshift-default-7f95675bf9-2bb7g" podUID="b6672fdb-54f4-4adc-844d-4072c3bf9876" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.132.0.59:15021/healthz/ready\": dial tcp 10.132.0.59:15021: connect: connection refused" Jul 16 01:12:52.216878 ip-10-0-143-200 kubenswrapper[2572]: I0716 01:12:52.216846 2572 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="openshift-ingress/e2e-api-a-f1f83d-openshift-default-7f95675bf9-2bb7g" Jul 16 01:12:52.218016 ip-10-0-143-200 kubenswrapper[2572]: I0716 01:12:52.217993 2572 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/e2e-api-a-f1f83d-openshift-default-7f95675bf9-2bb7g" Jul 16 01:12:57.068307 ip-10-0-143-200 kubenswrapper[2572]: I0716 01:12:57.068270 2572 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9gnmq_3a7e030d-fad8-480d-9dc6-fe5057048cf9/konnectivity-agent/0.log" limit=1024 Jul 16 01:12:57.168709 ip-10-0-143-200 kubenswrapper[2572]: I0716 01:12:57.168673 2572 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9gnmq_3a7e030d-fad8-480d-9dc6-fe5057048cf9/konnectivity-agent/0.log" limit=1024 Jul 16 01:12:57.471812 ip-10-0-143-200 kubenswrapper[2572]: I0716 01:12:57.471782 2572 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9gnmq_3a7e030d-fad8-480d-9dc6-fe5057048cf9/konnectivity-agent/0.log" limit=1024 Jul 16 01:12:57.675576 ip-10-0-143-200 kubenswrapper[2572]: I0716 01:12:57.675480 2572 generic.go:356] "Generic (PLEG): container finished" podID="57868c2b-17b7-49f4-a452-1b1db8a9e706" containerID="740ab1b2a1f513b74b278701db9f35b52ea7b72b90df928d45d794c4159f34e7" exitCode=0 Jul 16 01:12:57.675576 ip-10-0-143-200 kubenswrapper[2572]: I0716 01:12:57.675553 2572 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-api-b-2814a2-openshift-default-9ccd5f9d5-rh5vs" event={"ID":"57868c2b-17b7-49f4-a452-1b1db8a9e706","Type":"ContainerDied","Data":"740ab1b2a1f513b74b278701db9f35b52ea7b72b90df928d45d794c4159f34e7"} Jul 16 01:12:57.675978 ip-10-0-143-200 kubenswrapper[2572]: I0716 01:12:57.675965 2572 scope.go:117] "RemoveContainer" containerID="740ab1b2a1f513b74b278701db9f35b52ea7b72b90df928d45d794c4159f34e7" Jul 16 01:12:57.676932 ip-10-0-143-200 kubenswrapper[2572]: I0716 01:12:57.676907 2572 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236228Ki","pods":"250"} Jul 16 01:12:57.676988 ip-10-0-143-200 kubenswrapper[2572]: I0716 01:12:57.676971 2572 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236228Ki","pods":"250"} Jul 16 01:12:57.677031 ip-10-0-143-200 kubenswrapper[2572]: I0716 01:12:57.677002 2572 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236228Ki","pods":"250"} Jul 16 01:12:58.586840 ip-10-0-143-200 kubenswrapper[2572]: I0716 01:12:58.586798 2572 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/e2e-api-b-2814a2-openshift-default-9ccd5f9d5-rh5vs" Jul 16 01:12:58.685137 ip-10-0-143-200 kubenswrapper[2572]: I0716 01:12:58.684131 2572 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-api-b-2814a2-openshift-default-9ccd5f9d5-rh5vs" event={"ID":"57868c2b-17b7-49f4-a452-1b1db8a9e706","Type":"ContainerStarted","Data":"012575f81483af091cbe2e9e9127b8a5a7e261486744ffa5f62b83e7374f45b2"} Jul 16 01:12:59.144225 ip-10-0-143-200 kubenswrapper[2572]: I0716 01:12:59.144189 2572 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="openshift-ingress/e2e-api-b-2814a2-openshift-default-9ccd5f9d5-rh5vs" Jul 16 01:12:59.145475 ip-10-0-143-200 kubenswrapper[2572]: I0716 01:12:59.145449 2572 patch_prober.go:28] interesting pod/e2e-api-b-2814a2-openshift-default-9ccd5f9d5-rh5vs container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.132.0.60:15021/healthz/ready\": dial tcp 10.132.0.60:15021: connect: connection refused" start-of-body= Jul 16 01:12:59.145603 ip-10-0-143-200 kubenswrapper[2572]: I0716 01:12:59.145501 2572 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-api-b-2814a2-openshift-default-9ccd5f9d5-rh5vs" podUID="57868c2b-17b7-49f4-a452-1b1db8a9e706" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.132.0.60:15021/healthz/ready\": dial tcp 10.132.0.60:15021: connect: connection refused" Jul 16 01:13:00.143599 ip-10-0-143-200 kubenswrapper[2572]: I0716 01:13:00.143560 2572 patch_prober.go:28] interesting pod/e2e-api-b-2814a2-openshift-default-9ccd5f9d5-rh5vs container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.132.0.60:15021/healthz/ready\": dial tcp 10.132.0.60:15021: connect: connection refused" start-of-body= Jul 16 01:13:00.144022 ip-10-0-143-200 kubenswrapper[2572]: I0716 01:13:00.143640 2572 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-api-b-2814a2-openshift-default-9ccd5f9d5-rh5vs" podUID="57868c2b-17b7-49f4-a452-1b1db8a9e706" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.132.0.60:15021/healthz/ready\": dial tcp 10.132.0.60:15021: connect: connection refused" Jul 16 01:13:01.144219 ip-10-0-143-200 kubenswrapper[2572]: I0716 01:13:01.144182 2572 patch_prober.go:28] interesting pod/e2e-api-b-2814a2-openshift-default-9ccd5f9d5-rh5vs container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.132.0.60:15021/healthz/ready\": dial tcp 10.132.0.60:15021: connect: connection refused" start-of-body= Jul 16 01:13:01.144621 ip-10-0-143-200 kubenswrapper[2572]: I0716 01:13:01.144242 2572 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-api-b-2814a2-openshift-default-9ccd5f9d5-rh5vs" podUID="57868c2b-17b7-49f4-a452-1b1db8a9e706" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.132.0.60:15021/healthz/ready\": dial tcp 10.132.0.60:15021: connect: connection refused" Jul 16 01:13:02.143499 ip-10-0-143-200 kubenswrapper[2572]: I0716 01:13:02.143462 2572 patch_prober.go:28] interesting pod/e2e-api-b-2814a2-openshift-default-9ccd5f9d5-rh5vs container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.132.0.60:15021/healthz/ready\": dial tcp 10.132.0.60:15021: connect: connection refused" start-of-body= Jul 16 01:13:02.143702 ip-10-0-143-200 kubenswrapper[2572]: I0716 01:13:02.143519 2572 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-api-b-2814a2-openshift-default-9ccd5f9d5-rh5vs" podUID="57868c2b-17b7-49f4-a452-1b1db8a9e706" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.132.0.60:15021/healthz/ready\": dial tcp 10.132.0.60:15021: connect: connection refused" Jul 16 01:13:03.148325 ip-10-0-143-200 kubenswrapper[2572]: I0716 01:13:03.148291 2572 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="openshift-ingress/e2e-api-b-2814a2-openshift-default-9ccd5f9d5-rh5vs" Jul 16 01:13:03.148836 ip-10-0-143-200 kubenswrapper[2572]: I0716 01:13:03.148663 2572 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/e2e-api-b-2814a2-openshift-default-9ccd5f9d5-rh5vs" Jul 16 01:13:03.149550 ip-10-0-143-200 kubenswrapper[2572]: I0716 01:13:03.149532 2572 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/e2e-api-b-2814a2-openshift-default-9ccd5f9d5-rh5vs" Jul 16 01:13:03.515255 ip-10-0-143-200 kubenswrapper[2572]: I0716 01:13:03.515217 2572 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9gnmq_3a7e030d-fad8-480d-9dc6-fe5057048cf9/konnectivity-agent/0.log" limit=1024 Jul 16 01:13:05.415424 ip-10-0-143-200 kubenswrapper[2572]: I0716 01:13:05.415387 2572 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9gnmq_3a7e030d-fad8-480d-9dc6-fe5057048cf9/konnectivity-agent/0.log" limit=1024 Jul 16 01:13:05.516346 ip-10-0-143-200 kubenswrapper[2572]: I0716 01:13:05.516311 2572 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9gnmq_3a7e030d-fad8-480d-9dc6-fe5057048cf9/konnectivity-agent/0.log" limit=1024 Jul 16 01:13:09.661692 ip-10-0-143-200 kubenswrapper[2572]: I0716 01:13:09.661657 2572 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9gnmq_3a7e030d-fad8-480d-9dc6-fe5057048cf9/konnectivity-agent/0.log" limit=1024 Jul 16 01:13:09.770878 ip-10-0-143-200 kubenswrapper[2572]: I0716 01:13:09.770846 2572 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9gnmq_3a7e030d-fad8-480d-9dc6-fe5057048cf9/konnectivity-agent/0.log" limit=1024 Jul 16 01:13:09.996289 ip-10-0-143-200 kubenswrapper[2572]: I0716 01:13:09.996241 2572 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-ingress/e2e-api-a-f1f83d-openshift-default-7f95675bf9-2bb7g"] Jul 16 01:13:09.996507 ip-10-0-143-200 kubenswrapper[2572]: I0716 01:13:09.996483 2572 kuberuntime_container.go:873] "Killing container with a grace period" pod="openshift-ingress/e2e-api-a-f1f83d-openshift-default-7f95675bf9-2bb7g" podUID="b6672fdb-54f4-4adc-844d-4072c3bf9876" containerName="istio-proxy" containerID="cri-o://e431baa23c143f8982dc4998f4241e7cd3be3365ca0e297961635bd56c744f7c" gracePeriod=30 Jul 16 01:13:10.012873 ip-10-0-143-200 kubenswrapper[2572]: I0716 01:13:10.012838 2572 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9gnmq_3a7e030d-fad8-480d-9dc6-fe5057048cf9/konnectivity-agent/0.log" limit=1024 Jul 16 01:13:10.224309 ip-10-0-143-200 kubenswrapper[2572]: I0716 01:13:10.224275 2572 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9gnmq_3a7e030d-fad8-480d-9dc6-fe5057048cf9/konnectivity-agent/0.log" limit=1024 Jul 16 01:13:10.727894 ip-10-0-143-200 kubenswrapper[2572]: I0716 01:13:10.727861 2572 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9gnmq_3a7e030d-fad8-480d-9dc6-fe5057048cf9/konnectivity-agent/0.log" limit=1024 Jul 16 01:13:15.249462 ip-10-0-143-200 kubenswrapper[2572]: I0716 01:13:15.249438 2572 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-api-a-f1f83d-openshift-default-7f95675bf9-2bb7g" Jul 16 01:13:15.374782 ip-10-0-143-200 kubenswrapper[2572]: I0716 01:13:15.374697 2572 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/b6672fdb-54f4-4adc-844d-4072c3bf9876-istio-podinfo\") pod \"b6672fdb-54f4-4adc-844d-4072c3bf9876\" (UID: \"b6672fdb-54f4-4adc-844d-4072c3bf9876\") " Jul 16 01:13:15.374782 ip-10-0-143-200 kubenswrapper[2572]: I0716 01:13:15.374736 2572 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/b6672fdb-54f4-4adc-844d-4072c3bf9876-workload-socket\") pod \"b6672fdb-54f4-4adc-844d-4072c3bf9876\" (UID: \"b6672fdb-54f4-4adc-844d-4072c3bf9876\") " Jul 16 01:13:15.374782 ip-10-0-143-200 kubenswrapper[2572]: I0716 01:13:15.374764 2572 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/b6672fdb-54f4-4adc-844d-4072c3bf9876-workload-certs\") pod \"b6672fdb-54f4-4adc-844d-4072c3bf9876\" (UID: \"b6672fdb-54f4-4adc-844d-4072c3bf9876\") " Jul 16 01:13:15.375067 ip-10-0-143-200 kubenswrapper[2572]: I0716 01:13:15.374792 2572 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/b6672fdb-54f4-4adc-844d-4072c3bf9876-istio-token\") pod \"b6672fdb-54f4-4adc-844d-4072c3bf9876\" (UID: \"b6672fdb-54f4-4adc-844d-4072c3bf9876\") " Jul 16 01:13:15.375067 ip-10-0-143-200 kubenswrapper[2572]: I0716 01:13:15.374822 2572 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/b6672fdb-54f4-4adc-844d-4072c3bf9876-istio-envoy\") pod \"b6672fdb-54f4-4adc-844d-4072c3bf9876\" (UID: \"b6672fdb-54f4-4adc-844d-4072c3bf9876\") " Jul 16 01:13:15.375067 ip-10-0-143-200 kubenswrapper[2572]: I0716 01:13:15.374928 2572 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/b6672fdb-54f4-4adc-844d-4072c3bf9876-istio-data\") pod \"b6672fdb-54f4-4adc-844d-4072c3bf9876\" (UID: \"b6672fdb-54f4-4adc-844d-4072c3bf9876\") " Jul 16 01:13:15.375067 ip-10-0-143-200 kubenswrapper[2572]: I0716 01:13:15.374978 2572 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/b6672fdb-54f4-4adc-844d-4072c3bf9876-credential-socket\") pod \"b6672fdb-54f4-4adc-844d-4072c3bf9876\" (UID: \"b6672fdb-54f4-4adc-844d-4072c3bf9876\") " Jul 16 01:13:15.375067 ip-10-0-143-200 kubenswrapper[2572]: I0716 01:13:15.375005 2572 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-cdxb2\" (UniqueName: \"kubernetes.io/projected/b6672fdb-54f4-4adc-844d-4072c3bf9876-kube-api-access-cdxb2\") pod \"b6672fdb-54f4-4adc-844d-4072c3bf9876\" (UID: \"b6672fdb-54f4-4adc-844d-4072c3bf9876\") " Jul 16 01:13:15.375067 ip-10-0-143-200 kubenswrapper[2572]: I0716 01:13:15.375026 2572 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/b6672fdb-54f4-4adc-844d-4072c3bf9876-workload-socket" (OuterVolumeSpecName: "workload-socket") pod "b6672fdb-54f4-4adc-844d-4072c3bf9876" (UID: "b6672fdb-54f4-4adc-844d-4072c3bf9876"). InnerVolumeSpecName "workload-socket". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Jul 16 01:13:15.375349 ip-10-0-143-200 kubenswrapper[2572]: I0716 01:13:15.375046 2572 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/b6672fdb-54f4-4adc-844d-4072c3bf9876-istiod-ca-cert\") pod \"b6672fdb-54f4-4adc-844d-4072c3bf9876\" (UID: \"b6672fdb-54f4-4adc-844d-4072c3bf9876\") " Jul 16 01:13:15.375349 ip-10-0-143-200 kubenswrapper[2572]: I0716 01:13:15.375102 2572 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/b6672fdb-54f4-4adc-844d-4072c3bf9876-workload-certs" (OuterVolumeSpecName: "workload-certs") pod "b6672fdb-54f4-4adc-844d-4072c3bf9876" (UID: "b6672fdb-54f4-4adc-844d-4072c3bf9876"). InnerVolumeSpecName "workload-certs". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Jul 16 01:13:15.375349 ip-10-0-143-200 kubenswrapper[2572]: I0716 01:13:15.375316 2572 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/b6672fdb-54f4-4adc-844d-4072c3bf9876-credential-socket" (OuterVolumeSpecName: "credential-socket") pod "b6672fdb-54f4-4adc-844d-4072c3bf9876" (UID: "b6672fdb-54f4-4adc-844d-4072c3bf9876"). InnerVolumeSpecName "credential-socket". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Jul 16 01:13:15.375507 ip-10-0-143-200 kubenswrapper[2572]: I0716 01:13:15.375350 2572 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/b6672fdb-54f4-4adc-844d-4072c3bf9876-istiod-ca-cert" (OuterVolumeSpecName: "istiod-ca-cert") pod "b6672fdb-54f4-4adc-844d-4072c3bf9876" (UID: "b6672fdb-54f4-4adc-844d-4072c3bf9876"). InnerVolumeSpecName "istiod-ca-cert". PluginName "kubernetes.io/configmap", VolumeGIDValue "" Jul 16 01:13:15.375567 ip-10-0-143-200 kubenswrapper[2572]: I0716 01:13:15.375506 2572 reconciler_common.go:299] "Volume detached for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/b6672fdb-54f4-4adc-844d-4072c3bf9876-credential-socket\") on node \"ip-10-0-143-200.ec2.internal\" DevicePath \"\"" Jul 16 01:13:15.375567 ip-10-0-143-200 kubenswrapper[2572]: I0716 01:13:15.375523 2572 reconciler_common.go:299] "Volume detached for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/b6672fdb-54f4-4adc-844d-4072c3bf9876-istiod-ca-cert\") on node \"ip-10-0-143-200.ec2.internal\" DevicePath \"\"" Jul 16 01:13:15.375567 ip-10-0-143-200 kubenswrapper[2572]: I0716 01:13:15.375537 2572 reconciler_common.go:299] "Volume detached for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/b6672fdb-54f4-4adc-844d-4072c3bf9876-workload-socket\") on node \"ip-10-0-143-200.ec2.internal\" DevicePath \"\"" Jul 16 01:13:15.375567 ip-10-0-143-200 kubenswrapper[2572]: I0716 01:13:15.375550 2572 reconciler_common.go:299] "Volume detached for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/b6672fdb-54f4-4adc-844d-4072c3bf9876-workload-certs\") on node \"ip-10-0-143-200.ec2.internal\" DevicePath \"\"" Jul 16 01:13:15.376258 ip-10-0-143-200 kubenswrapper[2572]: I0716 01:13:15.376224 2572 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/b6672fdb-54f4-4adc-844d-4072c3bf9876-istio-data" (OuterVolumeSpecName: "istio-data") pod "b6672fdb-54f4-4adc-844d-4072c3bf9876" (UID: "b6672fdb-54f4-4adc-844d-4072c3bf9876"). InnerVolumeSpecName "istio-data". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Jul 16 01:13:15.377087 ip-10-0-143-200 kubenswrapper[2572]: I0716 01:13:15.377058 2572 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/b6672fdb-54f4-4adc-844d-4072c3bf9876-istio-envoy" (OuterVolumeSpecName: "istio-envoy") pod "b6672fdb-54f4-4adc-844d-4072c3bf9876" (UID: "b6672fdb-54f4-4adc-844d-4072c3bf9876"). InnerVolumeSpecName "istio-envoy". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Jul 16 01:13:15.377179 ip-10-0-143-200 kubenswrapper[2572]: I0716 01:13:15.377103 2572 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/downward-api/b6672fdb-54f4-4adc-844d-4072c3bf9876-istio-podinfo" (OuterVolumeSpecName: "istio-podinfo") pod "b6672fdb-54f4-4adc-844d-4072c3bf9876" (UID: "b6672fdb-54f4-4adc-844d-4072c3bf9876"). InnerVolumeSpecName "istio-podinfo". PluginName "kubernetes.io/downward-api", VolumeGIDValue "" Jul 16 01:13:15.377244 ip-10-0-143-200 kubenswrapper[2572]: I0716 01:13:15.377226 2572 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/b6672fdb-54f4-4adc-844d-4072c3bf9876-kube-api-access-cdxb2" (OuterVolumeSpecName: "kube-api-access-cdxb2") pod "b6672fdb-54f4-4adc-844d-4072c3bf9876" (UID: "b6672fdb-54f4-4adc-844d-4072c3bf9876"). InnerVolumeSpecName "kube-api-access-cdxb2". PluginName "kubernetes.io/projected", VolumeGIDValue "" Jul 16 01:13:15.377282 ip-10-0-143-200 kubenswrapper[2572]: I0716 01:13:15.377238 2572 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/b6672fdb-54f4-4adc-844d-4072c3bf9876-istio-token" (OuterVolumeSpecName: "istio-token") pod "b6672fdb-54f4-4adc-844d-4072c3bf9876" (UID: "b6672fdb-54f4-4adc-844d-4072c3bf9876"). InnerVolumeSpecName "istio-token". PluginName "kubernetes.io/projected", VolumeGIDValue "" Jul 16 01:13:15.476393 ip-10-0-143-200 kubenswrapper[2572]: I0716 01:13:15.476365 2572 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-cdxb2\" (UniqueName: \"kubernetes.io/projected/b6672fdb-54f4-4adc-844d-4072c3bf9876-kube-api-access-cdxb2\") on node \"ip-10-0-143-200.ec2.internal\" DevicePath \"\"" Jul 16 01:13:15.476393 ip-10-0-143-200 kubenswrapper[2572]: I0716 01:13:15.476393 2572 reconciler_common.go:299] "Volume detached for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/b6672fdb-54f4-4adc-844d-4072c3bf9876-istio-podinfo\") on node \"ip-10-0-143-200.ec2.internal\" DevicePath \"\"" Jul 16 01:13:15.476537 ip-10-0-143-200 kubenswrapper[2572]: I0716 01:13:15.476403 2572 reconciler_common.go:299] "Volume detached for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/b6672fdb-54f4-4adc-844d-4072c3bf9876-istio-token\") on node \"ip-10-0-143-200.ec2.internal\" DevicePath \"\"" Jul 16 01:13:15.476537 ip-10-0-143-200 kubenswrapper[2572]: I0716 01:13:15.476411 2572 reconciler_common.go:299] "Volume detached for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/b6672fdb-54f4-4adc-844d-4072c3bf9876-istio-envoy\") on node \"ip-10-0-143-200.ec2.internal\" DevicePath \"\"" Jul 16 01:13:15.476537 ip-10-0-143-200 kubenswrapper[2572]: I0716 01:13:15.476419 2572 reconciler_common.go:299] "Volume detached for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/b6672fdb-54f4-4adc-844d-4072c3bf9876-istio-data\") on node \"ip-10-0-143-200.ec2.internal\" DevicePath \"\"" Jul 16 01:13:15.784707 ip-10-0-143-200 kubenswrapper[2572]: I0716 01:13:15.784668 2572 generic.go:356] "Generic (PLEG): container finished" podID="b6672fdb-54f4-4adc-844d-4072c3bf9876" containerID="e431baa23c143f8982dc4998f4241e7cd3be3365ca0e297961635bd56c744f7c" exitCode=0 Jul 16 01:13:15.784893 ip-10-0-143-200 kubenswrapper[2572]: I0716 01:13:15.784729 2572 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-api-a-f1f83d-openshift-default-7f95675bf9-2bb7g" Jul 16 01:13:15.784893 ip-10-0-143-200 kubenswrapper[2572]: I0716 01:13:15.784734 2572 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-api-a-f1f83d-openshift-default-7f95675bf9-2bb7g" event={"ID":"b6672fdb-54f4-4adc-844d-4072c3bf9876","Type":"ContainerDied","Data":"e431baa23c143f8982dc4998f4241e7cd3be3365ca0e297961635bd56c744f7c"} Jul 16 01:13:15.784893 ip-10-0-143-200 kubenswrapper[2572]: I0716 01:13:15.784865 2572 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-api-a-f1f83d-openshift-default-7f95675bf9-2bb7g" event={"ID":"b6672fdb-54f4-4adc-844d-4072c3bf9876","Type":"ContainerDied","Data":"5f643a7695ba4b5c726656167c6efa90bdec6406fbc182a6d9a5269e492e2b7d"} Jul 16 01:13:15.784893 ip-10-0-143-200 kubenswrapper[2572]: I0716 01:13:15.784887 2572 scope.go:117] "RemoveContainer" containerID="e431baa23c143f8982dc4998f4241e7cd3be3365ca0e297961635bd56c744f7c" Jul 16 01:13:15.794766 ip-10-0-143-200 kubenswrapper[2572]: I0716 01:13:15.794745 2572 scope.go:117] "RemoveContainer" containerID="3141b387235d12c5cfb3c8687aa7f55d3f00858bfda0c06e0f08118844b5508e" Jul 16 01:13:15.803131 ip-10-0-143-200 kubenswrapper[2572]: I0716 01:13:15.803114 2572 scope.go:117] "RemoveContainer" containerID="e431baa23c143f8982dc4998f4241e7cd3be3365ca0e297961635bd56c744f7c" Jul 16 01:13:15.803408 ip-10-0-143-200 kubenswrapper[2572]: E0716 01:13:15.803388 2572 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"e431baa23c143f8982dc4998f4241e7cd3be3365ca0e297961635bd56c744f7c\": container with ID starting with e431baa23c143f8982dc4998f4241e7cd3be3365ca0e297961635bd56c744f7c not found: ID does not exist" containerID="e431baa23c143f8982dc4998f4241e7cd3be3365ca0e297961635bd56c744f7c" Jul 16 01:13:15.803477 ip-10-0-143-200 kubenswrapper[2572]: I0716 01:13:15.803417 2572 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"e431baa23c143f8982dc4998f4241e7cd3be3365ca0e297961635bd56c744f7c"} err="failed to get container status \"e431baa23c143f8982dc4998f4241e7cd3be3365ca0e297961635bd56c744f7c\": rpc error: code = NotFound desc = could not find container \"e431baa23c143f8982dc4998f4241e7cd3be3365ca0e297961635bd56c744f7c\": container with ID starting with e431baa23c143f8982dc4998f4241e7cd3be3365ca0e297961635bd56c744f7c not found: ID does not exist" Jul 16 01:13:15.803477 ip-10-0-143-200 kubenswrapper[2572]: I0716 01:13:15.803435 2572 scope.go:117] "RemoveContainer" containerID="3141b387235d12c5cfb3c8687aa7f55d3f00858bfda0c06e0f08118844b5508e" Jul 16 01:13:15.803698 ip-10-0-143-200 kubenswrapper[2572]: E0716 01:13:15.803676 2572 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"3141b387235d12c5cfb3c8687aa7f55d3f00858bfda0c06e0f08118844b5508e\": container with ID starting with 3141b387235d12c5cfb3c8687aa7f55d3f00858bfda0c06e0f08118844b5508e not found: ID does not exist" containerID="3141b387235d12c5cfb3c8687aa7f55d3f00858bfda0c06e0f08118844b5508e" Jul 16 01:13:15.803811 ip-10-0-143-200 kubenswrapper[2572]: I0716 01:13:15.803708 2572 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"3141b387235d12c5cfb3c8687aa7f55d3f00858bfda0c06e0f08118844b5508e"} err="failed to get container status \"3141b387235d12c5cfb3c8687aa7f55d3f00858bfda0c06e0f08118844b5508e\": rpc error: code = NotFound desc = could not find container \"3141b387235d12c5cfb3c8687aa7f55d3f00858bfda0c06e0f08118844b5508e\": container with ID starting with 3141b387235d12c5cfb3c8687aa7f55d3f00858bfda0c06e0f08118844b5508e not found: ID does not exist" Jul 16 01:13:15.805226 ip-10-0-143-200 kubenswrapper[2572]: I0716 01:13:15.805204 2572 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-ingress/e2e-api-a-f1f83d-openshift-default-7f95675bf9-2bb7g"] Jul 16 01:13:15.812495 ip-10-0-143-200 kubenswrapper[2572]: I0716 01:13:15.812474 2572 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["openshift-ingress/e2e-api-a-f1f83d-openshift-default-7f95675bf9-2bb7g"] Jul 16 01:13:16.565823 ip-10-0-143-200 kubenswrapper[2572]: I0716 01:13:16.565788 2572 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9gnmq_3a7e030d-fad8-480d-9dc6-fe5057048cf9/konnectivity-agent/0.log" limit=1024 Jul 16 01:13:17.449247 ip-10-0-143-200 kubenswrapper[2572]: I0716 01:13:17.449212 2572 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="b6672fdb-54f4-4adc-844d-4072c3bf9876" path="/var/lib/kubelet/pods/b6672fdb-54f4-4adc-844d-4072c3bf9876/volumes" Jul 16 01:13:22.207623 ip-10-0-143-200 kubenswrapper[2572]: I0716 01:13:22.207592 2572 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9gnmq_3a7e030d-fad8-480d-9dc6-fe5057048cf9/konnectivity-agent/0.log" limit=1024 Jul 16 01:13:22.281552 ip-10-0-143-200 kubenswrapper[2572]: I0716 01:13:22.281521 2572 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9gnmq_3a7e030d-fad8-480d-9dc6-fe5057048cf9/konnectivity-agent/0.log" limit=1024 Jul 16 01:13:22.473790 ip-10-0-143-200 kubenswrapper[2572]: I0716 01:13:22.473709 2572 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-ingress/e2e-api-b-2814a2-openshift-default-9ccd5f9d5-rh5vs"] Jul 16 01:13:22.473969 ip-10-0-143-200 kubenswrapper[2572]: I0716 01:13:22.473933 2572 kuberuntime_container.go:873] "Killing container with a grace period" pod="openshift-ingress/e2e-api-b-2814a2-openshift-default-9ccd5f9d5-rh5vs" podUID="57868c2b-17b7-49f4-a452-1b1db8a9e706" containerName="istio-proxy" containerID="cri-o://012575f81483af091cbe2e9e9127b8a5a7e261486744ffa5f62b83e7374f45b2" gracePeriod=30 Jul 16 01:13:22.510555 ip-10-0-143-200 kubenswrapper[2572]: I0716 01:13:22.510519 2572 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9gnmq_3a7e030d-fad8-480d-9dc6-fe5057048cf9/konnectivity-agent/0.log" limit=1024 Jul 16 01:13:22.723435 ip-10-0-143-200 kubenswrapper[2572]: I0716 01:13:22.723405 2572 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9gnmq_3a7e030d-fad8-480d-9dc6-fe5057048cf9/konnectivity-agent/0.log" limit=1024 Jul 16 01:13:23.072016 ip-10-0-143-200 kubenswrapper[2572]: I0716 01:13:23.071987 2572 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9gnmq_3a7e030d-fad8-480d-9dc6-fe5057048cf9/konnectivity-agent/0.log" limit=1024 Jul 16 01:13:23.402263 ip-10-0-143-200 kubenswrapper[2572]: I0716 01:13:23.402179 2572 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-ingress/e2e-shared-a-f9f50d-openshift-default-5d5b68dcd7-xl2x6"] Jul 16 01:13:23.402615 ip-10-0-143-200 kubenswrapper[2572]: I0716 01:13:23.402595 2572 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="b6672fdb-54f4-4adc-844d-4072c3bf9876" containerName="istio-proxy" Jul 16 01:13:23.402615 ip-10-0-143-200 kubenswrapper[2572]: I0716 01:13:23.402606 2572 state_mem.go:107] "Deleted CPUSet assignment" podUID="b6672fdb-54f4-4adc-844d-4072c3bf9876" containerName="istio-proxy" Jul 16 01:13:23.402720 ip-10-0-143-200 kubenswrapper[2572]: I0716 01:13:23.402622 2572 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="301fffd1-42b5-4cfd-a97d-53a754a45110" containerName="istio-proxy" Jul 16 01:13:23.402720 ip-10-0-143-200 kubenswrapper[2572]: I0716 01:13:23.402653 2572 state_mem.go:107] "Deleted CPUSet assignment" podUID="301fffd1-42b5-4cfd-a97d-53a754a45110" containerName="istio-proxy" Jul 16 01:13:23.402720 ip-10-0-143-200 kubenswrapper[2572]: I0716 01:13:23.402665 2572 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="b6672fdb-54f4-4adc-844d-4072c3bf9876" containerName="istio-proxy" Jul 16 01:13:23.402720 ip-10-0-143-200 kubenswrapper[2572]: I0716 01:13:23.402671 2572 state_mem.go:107] "Deleted CPUSet assignment" podUID="b6672fdb-54f4-4adc-844d-4072c3bf9876" containerName="istio-proxy" Jul 16 01:13:23.402841 ip-10-0-143-200 kubenswrapper[2572]: I0716 01:13:23.402746 2572 memory_manager.go:368] "RemoveStaleState removing state" podUID="301fffd1-42b5-4cfd-a97d-53a754a45110" containerName="istio-proxy" Jul 16 01:13:23.402841 ip-10-0-143-200 kubenswrapper[2572]: I0716 01:13:23.402755 2572 memory_manager.go:368] "RemoveStaleState removing state" podUID="b6672fdb-54f4-4adc-844d-4072c3bf9876" containerName="istio-proxy" Jul 16 01:13:23.407408 ip-10-0-143-200 kubenswrapper[2572]: I0716 01:13:23.407388 2572 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-shared-a-f9f50d-openshift-default-5d5b68dcd7-xl2x6" Jul 16 01:13:23.411280 ip-10-0-143-200 kubenswrapper[2572]: I0716 01:13:23.411262 2572 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress\"/\"e2e-shared-a-f9f50d-openshift-default-dockercfg-cxlb9\"" Jul 16 01:13:23.422452 ip-10-0-143-200 kubenswrapper[2572]: I0716 01:13:23.422430 2572 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/e2e-shared-a-f9f50d-openshift-default-5d5b68dcd7-xl2x6"] Jul 16 01:13:23.448200 ip-10-0-143-200 kubenswrapper[2572]: I0716 01:13:23.448126 2572 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9gnmq_3a7e030d-fad8-480d-9dc6-fe5057048cf9/konnectivity-agent/0.log" limit=1024 Jul 16 01:13:23.450332 ip-10-0-143-200 kubenswrapper[2572]: I0716 01:13:23.450306 2572 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/c3a9a894-3f1a-4746-a760-831441b8ac8b-workload-socket\") pod \"e2e-shared-a-f9f50d-openshift-default-5d5b68dcd7-xl2x6\" (UID: \"c3a9a894-3f1a-4746-a760-831441b8ac8b\") " pod="openshift-ingress/e2e-shared-a-f9f50d-openshift-default-5d5b68dcd7-xl2x6" Jul 16 01:13:23.450442 ip-10-0-143-200 kubenswrapper[2572]: I0716 01:13:23.450350 2572 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/c3a9a894-3f1a-4746-a760-831441b8ac8b-istiod-ca-cert\") pod \"e2e-shared-a-f9f50d-openshift-default-5d5b68dcd7-xl2x6\" (UID: \"c3a9a894-3f1a-4746-a760-831441b8ac8b\") " pod="openshift-ingress/e2e-shared-a-f9f50d-openshift-default-5d5b68dcd7-xl2x6" Jul 16 01:13:23.450499 ip-10-0-143-200 kubenswrapper[2572]: I0716 01:13:23.450439 2572 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/c3a9a894-3f1a-4746-a760-831441b8ac8b-istio-podinfo\") pod \"e2e-shared-a-f9f50d-openshift-default-5d5b68dcd7-xl2x6\" (UID: \"c3a9a894-3f1a-4746-a760-831441b8ac8b\") " pod="openshift-ingress/e2e-shared-a-f9f50d-openshift-default-5d5b68dcd7-xl2x6" Jul 16 01:13:23.450499 ip-10-0-143-200 kubenswrapper[2572]: I0716 01:13:23.450478 2572 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/c3a9a894-3f1a-4746-a760-831441b8ac8b-istio-data\") pod \"e2e-shared-a-f9f50d-openshift-default-5d5b68dcd7-xl2x6\" (UID: \"c3a9a894-3f1a-4746-a760-831441b8ac8b\") " pod="openshift-ingress/e2e-shared-a-f9f50d-openshift-default-5d5b68dcd7-xl2x6" Jul 16 01:13:23.450616 ip-10-0-143-200 kubenswrapper[2572]: I0716 01:13:23.450598 2572 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-6hpxr\" (UniqueName: \"kubernetes.io/projected/c3a9a894-3f1a-4746-a760-831441b8ac8b-kube-api-access-6hpxr\") pod \"e2e-shared-a-f9f50d-openshift-default-5d5b68dcd7-xl2x6\" (UID: \"c3a9a894-3f1a-4746-a760-831441b8ac8b\") " pod="openshift-ingress/e2e-shared-a-f9f50d-openshift-default-5d5b68dcd7-xl2x6" Jul 16 01:13:23.450697 ip-10-0-143-200 kubenswrapper[2572]: I0716 01:13:23.450652 2572 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/c3a9a894-3f1a-4746-a760-831441b8ac8b-istio-token\") pod \"e2e-shared-a-f9f50d-openshift-default-5d5b68dcd7-xl2x6\" (UID: \"c3a9a894-3f1a-4746-a760-831441b8ac8b\") " pod="openshift-ingress/e2e-shared-a-f9f50d-openshift-default-5d5b68dcd7-xl2x6" Jul 16 01:13:23.450697 ip-10-0-143-200 kubenswrapper[2572]: I0716 01:13:23.450689 2572 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/c3a9a894-3f1a-4746-a760-831441b8ac8b-workload-certs\") pod \"e2e-shared-a-f9f50d-openshift-default-5d5b68dcd7-xl2x6\" (UID: \"c3a9a894-3f1a-4746-a760-831441b8ac8b\") " pod="openshift-ingress/e2e-shared-a-f9f50d-openshift-default-5d5b68dcd7-xl2x6" Jul 16 01:13:23.450809 ip-10-0-143-200 kubenswrapper[2572]: I0716 01:13:23.450735 2572 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/c3a9a894-3f1a-4746-a760-831441b8ac8b-credential-socket\") pod \"e2e-shared-a-f9f50d-openshift-default-5d5b68dcd7-xl2x6\" (UID: \"c3a9a894-3f1a-4746-a760-831441b8ac8b\") " pod="openshift-ingress/e2e-shared-a-f9f50d-openshift-default-5d5b68dcd7-xl2x6" Jul 16 01:13:23.450809 ip-10-0-143-200 kubenswrapper[2572]: I0716 01:13:23.450769 2572 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/c3a9a894-3f1a-4746-a760-831441b8ac8b-istio-envoy\") pod \"e2e-shared-a-f9f50d-openshift-default-5d5b68dcd7-xl2x6\" (UID: \"c3a9a894-3f1a-4746-a760-831441b8ac8b\") " pod="openshift-ingress/e2e-shared-a-f9f50d-openshift-default-5d5b68dcd7-xl2x6" Jul 16 01:13:23.551915 ip-10-0-143-200 kubenswrapper[2572]: I0716 01:13:23.551879 2572 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-6hpxr\" (UniqueName: \"kubernetes.io/projected/c3a9a894-3f1a-4746-a760-831441b8ac8b-kube-api-access-6hpxr\") pod \"e2e-shared-a-f9f50d-openshift-default-5d5b68dcd7-xl2x6\" (UID: \"c3a9a894-3f1a-4746-a760-831441b8ac8b\") " pod="openshift-ingress/e2e-shared-a-f9f50d-openshift-default-5d5b68dcd7-xl2x6" Jul 16 01:13:23.551915 ip-10-0-143-200 kubenswrapper[2572]: I0716 01:13:23.551919 2572 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/c3a9a894-3f1a-4746-a760-831441b8ac8b-istio-token\") pod \"e2e-shared-a-f9f50d-openshift-default-5d5b68dcd7-xl2x6\" (UID: \"c3a9a894-3f1a-4746-a760-831441b8ac8b\") " pod="openshift-ingress/e2e-shared-a-f9f50d-openshift-default-5d5b68dcd7-xl2x6" Jul 16 01:13:23.552181 ip-10-0-143-200 kubenswrapper[2572]: I0716 01:13:23.551948 2572 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/c3a9a894-3f1a-4746-a760-831441b8ac8b-workload-certs\") pod \"e2e-shared-a-f9f50d-openshift-default-5d5b68dcd7-xl2x6\" (UID: \"c3a9a894-3f1a-4746-a760-831441b8ac8b\") " pod="openshift-ingress/e2e-shared-a-f9f50d-openshift-default-5d5b68dcd7-xl2x6" Jul 16 01:13:23.552181 ip-10-0-143-200 kubenswrapper[2572]: I0716 01:13:23.551977 2572 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/c3a9a894-3f1a-4746-a760-831441b8ac8b-credential-socket\") pod \"e2e-shared-a-f9f50d-openshift-default-5d5b68dcd7-xl2x6\" (UID: \"c3a9a894-3f1a-4746-a760-831441b8ac8b\") " pod="openshift-ingress/e2e-shared-a-f9f50d-openshift-default-5d5b68dcd7-xl2x6" Jul 16 01:13:23.552181 ip-10-0-143-200 kubenswrapper[2572]: I0716 01:13:23.551999 2572 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/c3a9a894-3f1a-4746-a760-831441b8ac8b-istio-envoy\") pod \"e2e-shared-a-f9f50d-openshift-default-5d5b68dcd7-xl2x6\" (UID: \"c3a9a894-3f1a-4746-a760-831441b8ac8b\") " pod="openshift-ingress/e2e-shared-a-f9f50d-openshift-default-5d5b68dcd7-xl2x6" Jul 16 01:13:23.552181 ip-10-0-143-200 kubenswrapper[2572]: I0716 01:13:23.552028 2572 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/c3a9a894-3f1a-4746-a760-831441b8ac8b-workload-socket\") pod \"e2e-shared-a-f9f50d-openshift-default-5d5b68dcd7-xl2x6\" (UID: \"c3a9a894-3f1a-4746-a760-831441b8ac8b\") " pod="openshift-ingress/e2e-shared-a-f9f50d-openshift-default-5d5b68dcd7-xl2x6" Jul 16 01:13:23.552181 ip-10-0-143-200 kubenswrapper[2572]: I0716 01:13:23.552058 2572 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/c3a9a894-3f1a-4746-a760-831441b8ac8b-istiod-ca-cert\") pod \"e2e-shared-a-f9f50d-openshift-default-5d5b68dcd7-xl2x6\" (UID: \"c3a9a894-3f1a-4746-a760-831441b8ac8b\") " pod="openshift-ingress/e2e-shared-a-f9f50d-openshift-default-5d5b68dcd7-xl2x6" Jul 16 01:13:23.552447 ip-10-0-143-200 kubenswrapper[2572]: I0716 01:13:23.552182 2572 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/c3a9a894-3f1a-4746-a760-831441b8ac8b-istio-podinfo\") pod \"e2e-shared-a-f9f50d-openshift-default-5d5b68dcd7-xl2x6\" (UID: \"c3a9a894-3f1a-4746-a760-831441b8ac8b\") " pod="openshift-ingress/e2e-shared-a-f9f50d-openshift-default-5d5b68dcd7-xl2x6" Jul 16 01:13:23.552447 ip-10-0-143-200 kubenswrapper[2572]: I0716 01:13:23.552226 2572 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/c3a9a894-3f1a-4746-a760-831441b8ac8b-istio-data\") pod \"e2e-shared-a-f9f50d-openshift-default-5d5b68dcd7-xl2x6\" (UID: \"c3a9a894-3f1a-4746-a760-831441b8ac8b\") " pod="openshift-ingress/e2e-shared-a-f9f50d-openshift-default-5d5b68dcd7-xl2x6" Jul 16 01:13:23.552447 ip-10-0-143-200 kubenswrapper[2572]: I0716 01:13:23.552414 2572 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/c3a9a894-3f1a-4746-a760-831441b8ac8b-workload-certs\") pod \"e2e-shared-a-f9f50d-openshift-default-5d5b68dcd7-xl2x6\" (UID: \"c3a9a894-3f1a-4746-a760-831441b8ac8b\") " pod="openshift-ingress/e2e-shared-a-f9f50d-openshift-default-5d5b68dcd7-xl2x6" Jul 16 01:13:23.552611 ip-10-0-143-200 kubenswrapper[2572]: I0716 01:13:23.552468 2572 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/c3a9a894-3f1a-4746-a760-831441b8ac8b-credential-socket\") pod \"e2e-shared-a-f9f50d-openshift-default-5d5b68dcd7-xl2x6\" (UID: \"c3a9a894-3f1a-4746-a760-831441b8ac8b\") " pod="openshift-ingress/e2e-shared-a-f9f50d-openshift-default-5d5b68dcd7-xl2x6" Jul 16 01:13:23.552693 ip-10-0-143-200 kubenswrapper[2572]: I0716 01:13:23.552613 2572 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/c3a9a894-3f1a-4746-a760-831441b8ac8b-workload-socket\") pod \"e2e-shared-a-f9f50d-openshift-default-5d5b68dcd7-xl2x6\" (UID: \"c3a9a894-3f1a-4746-a760-831441b8ac8b\") " pod="openshift-ingress/e2e-shared-a-f9f50d-openshift-default-5d5b68dcd7-xl2x6" Jul 16 01:13:23.552693 ip-10-0-143-200 kubenswrapper[2572]: I0716 01:13:23.552680 2572 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/c3a9a894-3f1a-4746-a760-831441b8ac8b-istio-data\") pod \"e2e-shared-a-f9f50d-openshift-default-5d5b68dcd7-xl2x6\" (UID: \"c3a9a894-3f1a-4746-a760-831441b8ac8b\") " pod="openshift-ingress/e2e-shared-a-f9f50d-openshift-default-5d5b68dcd7-xl2x6" Jul 16 01:13:23.553129 ip-10-0-143-200 kubenswrapper[2572]: I0716 01:13:23.553109 2572 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/c3a9a894-3f1a-4746-a760-831441b8ac8b-istiod-ca-cert\") pod \"e2e-shared-a-f9f50d-openshift-default-5d5b68dcd7-xl2x6\" (UID: \"c3a9a894-3f1a-4746-a760-831441b8ac8b\") " pod="openshift-ingress/e2e-shared-a-f9f50d-openshift-default-5d5b68dcd7-xl2x6" Jul 16 01:13:23.554587 ip-10-0-143-200 kubenswrapper[2572]: I0716 01:13:23.554566 2572 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/c3a9a894-3f1a-4746-a760-831441b8ac8b-istio-envoy\") pod \"e2e-shared-a-f9f50d-openshift-default-5d5b68dcd7-xl2x6\" (UID: \"c3a9a894-3f1a-4746-a760-831441b8ac8b\") " pod="openshift-ingress/e2e-shared-a-f9f50d-openshift-default-5d5b68dcd7-xl2x6" Jul 16 01:13:23.554709 ip-10-0-143-200 kubenswrapper[2572]: I0716 01:13:23.554690 2572 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/c3a9a894-3f1a-4746-a760-831441b8ac8b-istio-podinfo\") pod \"e2e-shared-a-f9f50d-openshift-default-5d5b68dcd7-xl2x6\" (UID: \"c3a9a894-3f1a-4746-a760-831441b8ac8b\") " pod="openshift-ingress/e2e-shared-a-f9f50d-openshift-default-5d5b68dcd7-xl2x6" Jul 16 01:13:23.562278 ip-10-0-143-200 kubenswrapper[2572]: I0716 01:13:23.562246 2572 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/c3a9a894-3f1a-4746-a760-831441b8ac8b-istio-token\") pod \"e2e-shared-a-f9f50d-openshift-default-5d5b68dcd7-xl2x6\" (UID: \"c3a9a894-3f1a-4746-a760-831441b8ac8b\") " pod="openshift-ingress/e2e-shared-a-f9f50d-openshift-default-5d5b68dcd7-xl2x6" Jul 16 01:13:23.563056 ip-10-0-143-200 kubenswrapper[2572]: I0716 01:13:23.563034 2572 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-6hpxr\" (UniqueName: \"kubernetes.io/projected/c3a9a894-3f1a-4746-a760-831441b8ac8b-kube-api-access-6hpxr\") pod \"e2e-shared-a-f9f50d-openshift-default-5d5b68dcd7-xl2x6\" (UID: \"c3a9a894-3f1a-4746-a760-831441b8ac8b\") " pod="openshift-ingress/e2e-shared-a-f9f50d-openshift-default-5d5b68dcd7-xl2x6" Jul 16 01:13:23.700077 ip-10-0-143-200 kubenswrapper[2572]: I0716 01:13:23.700048 2572 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9gnmq_3a7e030d-fad8-480d-9dc6-fe5057048cf9/konnectivity-agent/0.log" limit=1024 Jul 16 01:13:23.720477 ip-10-0-143-200 kubenswrapper[2572]: I0716 01:13:23.720451 2572 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-shared-a-f9f50d-openshift-default-5d5b68dcd7-xl2x6" Jul 16 01:13:23.869055 ip-10-0-143-200 kubenswrapper[2572]: I0716 01:13:23.869024 2572 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/e2e-shared-a-f9f50d-openshift-default-5d5b68dcd7-xl2x6"] Jul 16 01:13:23.870215 ip-10-0-143-200 kubenswrapper[2572]: W0716 01:13:23.870190 2572 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podc3a9a894_3f1a_4746_a760_831441b8ac8b.slice/crio-6b7150a8fefa5f338aa474b24424770e5656a4202edfa289a1bb565845963b79 WatchSource:0}: Error finding container 6b7150a8fefa5f338aa474b24424770e5656a4202edfa289a1bb565845963b79: Status 404 returned error can't find the container with id 6b7150a8fefa5f338aa474b24424770e5656a4202edfa289a1bb565845963b79 Jul 16 01:13:23.872332 ip-10-0-143-200 kubenswrapper[2572]: I0716 01:13:23.872303 2572 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236228Ki","pods":"250"} Jul 16 01:13:23.872391 ip-10-0-143-200 kubenswrapper[2572]: I0716 01:13:23.872361 2572 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236228Ki","pods":"250"} Jul 16 01:13:23.872437 ip-10-0-143-200 kubenswrapper[2572]: I0716 01:13:23.872388 2572 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236228Ki","pods":"250"} Jul 16 01:13:24.840344 ip-10-0-143-200 kubenswrapper[2572]: I0716 01:13:24.840311 2572 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-shared-a-f9f50d-openshift-default-5d5b68dcd7-xl2x6" event={"ID":"c3a9a894-3f1a-4746-a760-831441b8ac8b","Type":"ContainerStarted","Data":"eaac463958ff9bae0954276dddf7bb360f38fde51746934c5d22c8b9f804ffc5"} Jul 16 01:13:24.840769 ip-10-0-143-200 kubenswrapper[2572]: I0716 01:13:24.840367 2572 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-shared-a-f9f50d-openshift-default-5d5b68dcd7-xl2x6" event={"ID":"c3a9a894-3f1a-4746-a760-831441b8ac8b","Type":"ContainerStarted","Data":"6b7150a8fefa5f338aa474b24424770e5656a4202edfa289a1bb565845963b79"} Jul 16 01:13:24.862860 ip-10-0-143-200 kubenswrapper[2572]: I0716 01:13:24.862804 2572 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-ingress/e2e-shared-a-f9f50d-openshift-default-5d5b68dcd7-xl2x6" podStartSLOduration=1.862785384 podStartE2EDuration="1.862785384s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-16 01:13:23 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-16 01:13:24.86191904 +0000 UTC m=+2009.970673714" watchObservedRunningTime="2026-07-16 01:13:24.862785384 +0000 UTC m=+2009.971540060" Jul 16 01:13:25.720841 ip-10-0-143-200 kubenswrapper[2572]: I0716 01:13:25.720802 2572 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="openshift-ingress/e2e-shared-a-f9f50d-openshift-default-5d5b68dcd7-xl2x6" Jul 16 01:13:25.726004 ip-10-0-143-200 kubenswrapper[2572]: I0716 01:13:25.725977 2572 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="openshift-ingress/e2e-shared-a-f9f50d-openshift-default-5d5b68dcd7-xl2x6" Jul 16 01:13:25.772590 ip-10-0-143-200 kubenswrapper[2572]: I0716 01:13:25.772560 2572 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9gnmq_3a7e030d-fad8-480d-9dc6-fe5057048cf9/konnectivity-agent/0.log" limit=1024 Jul 16 01:13:25.848685 ip-10-0-143-200 kubenswrapper[2572]: I0716 01:13:25.848658 2572 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/e2e-shared-a-f9f50d-openshift-default-5d5b68dcd7-xl2x6" Jul 16 01:13:25.849527 ip-10-0-143-200 kubenswrapper[2572]: I0716 01:13:25.849502 2572 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/e2e-shared-a-f9f50d-openshift-default-5d5b68dcd7-xl2x6" Jul 16 01:13:25.872698 ip-10-0-143-200 kubenswrapper[2572]: I0716 01:13:25.872668 2572 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9gnmq_3a7e030d-fad8-480d-9dc6-fe5057048cf9/konnectivity-agent/0.log" limit=1024 Jul 16 01:13:25.986000 ip-10-0-143-200 kubenswrapper[2572]: I0716 01:13:25.985869 2572 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9gnmq_3a7e030d-fad8-480d-9dc6-fe5057048cf9/konnectivity-agent/0.log" limit=1024 Jul 16 01:13:26.178751 ip-10-0-143-200 kubenswrapper[2572]: I0716 01:13:26.178717 2572 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9gnmq_3a7e030d-fad8-480d-9dc6-fe5057048cf9/konnectivity-agent/0.log" limit=1024 Jul 16 01:13:26.666818 ip-10-0-143-200 kubenswrapper[2572]: I0716 01:13:26.666787 2572 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9gnmq_3a7e030d-fad8-480d-9dc6-fe5057048cf9/konnectivity-agent/0.log" limit=1024 Jul 16 01:13:26.767136 ip-10-0-143-200 kubenswrapper[2572]: I0716 01:13:26.767104 2572 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9gnmq_3a7e030d-fad8-480d-9dc6-fe5057048cf9/konnectivity-agent/0.log" limit=1024 Jul 16 01:13:27.750339 ip-10-0-143-200 kubenswrapper[2572]: I0716 01:13:27.750275 2572 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-api-b-2814a2-openshift-default-9ccd5f9d5-rh5vs" Jul 16 01:13:27.792147 ip-10-0-143-200 kubenswrapper[2572]: I0716 01:13:27.792117 2572 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/57868c2b-17b7-49f4-a452-1b1db8a9e706-credential-socket\") pod \"57868c2b-17b7-49f4-a452-1b1db8a9e706\" (UID: \"57868c2b-17b7-49f4-a452-1b1db8a9e706\") " Jul 16 01:13:27.792306 ip-10-0-143-200 kubenswrapper[2572]: I0716 01:13:27.792206 2572 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/57868c2b-17b7-49f4-a452-1b1db8a9e706-istio-envoy\") pod \"57868c2b-17b7-49f4-a452-1b1db8a9e706\" (UID: \"57868c2b-17b7-49f4-a452-1b1db8a9e706\") " Jul 16 01:13:27.792306 ip-10-0-143-200 kubenswrapper[2572]: I0716 01:13:27.792242 2572 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/57868c2b-17b7-49f4-a452-1b1db8a9e706-istio-token\") pod \"57868c2b-17b7-49f4-a452-1b1db8a9e706\" (UID: \"57868c2b-17b7-49f4-a452-1b1db8a9e706\") " Jul 16 01:13:27.792306 ip-10-0-143-200 kubenswrapper[2572]: I0716 01:13:27.792271 2572 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/57868c2b-17b7-49f4-a452-1b1db8a9e706-istio-data\") pod \"57868c2b-17b7-49f4-a452-1b1db8a9e706\" (UID: \"57868c2b-17b7-49f4-a452-1b1db8a9e706\") " Jul 16 01:13:27.792462 ip-10-0-143-200 kubenswrapper[2572]: I0716 01:13:27.792308 2572 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/57868c2b-17b7-49f4-a452-1b1db8a9e706-istio-podinfo\") pod \"57868c2b-17b7-49f4-a452-1b1db8a9e706\" (UID: \"57868c2b-17b7-49f4-a452-1b1db8a9e706\") " Jul 16 01:13:27.792462 ip-10-0-143-200 kubenswrapper[2572]: I0716 01:13:27.792359 2572 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/57868c2b-17b7-49f4-a452-1b1db8a9e706-workload-certs\") pod \"57868c2b-17b7-49f4-a452-1b1db8a9e706\" (UID: \"57868c2b-17b7-49f4-a452-1b1db8a9e706\") " Jul 16 01:13:27.792462 ip-10-0-143-200 kubenswrapper[2572]: I0716 01:13:27.792388 2572 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/57868c2b-17b7-49f4-a452-1b1db8a9e706-workload-socket\") pod \"57868c2b-17b7-49f4-a452-1b1db8a9e706\" (UID: \"57868c2b-17b7-49f4-a452-1b1db8a9e706\") " Jul 16 01:13:27.792462 ip-10-0-143-200 kubenswrapper[2572]: I0716 01:13:27.792422 2572 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-2h85f\" (UniqueName: \"kubernetes.io/projected/57868c2b-17b7-49f4-a452-1b1db8a9e706-kube-api-access-2h85f\") pod \"57868c2b-17b7-49f4-a452-1b1db8a9e706\" (UID: \"57868c2b-17b7-49f4-a452-1b1db8a9e706\") " Jul 16 01:13:27.792462 ip-10-0-143-200 kubenswrapper[2572]: I0716 01:13:27.792421 2572 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/57868c2b-17b7-49f4-a452-1b1db8a9e706-credential-socket" (OuterVolumeSpecName: "credential-socket") pod "57868c2b-17b7-49f4-a452-1b1db8a9e706" (UID: "57868c2b-17b7-49f4-a452-1b1db8a9e706"). InnerVolumeSpecName "credential-socket". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Jul 16 01:13:27.792462 ip-10-0-143-200 kubenswrapper[2572]: I0716 01:13:27.792455 2572 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/57868c2b-17b7-49f4-a452-1b1db8a9e706-istiod-ca-cert\") pod \"57868c2b-17b7-49f4-a452-1b1db8a9e706\" (UID: \"57868c2b-17b7-49f4-a452-1b1db8a9e706\") " Jul 16 01:13:27.793141 ip-10-0-143-200 kubenswrapper[2572]: I0716 01:13:27.793066 2572 reconciler_common.go:299] "Volume detached for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/57868c2b-17b7-49f4-a452-1b1db8a9e706-credential-socket\") on node \"ip-10-0-143-200.ec2.internal\" DevicePath \"\"" Jul 16 01:13:27.793141 ip-10-0-143-200 kubenswrapper[2572]: I0716 01:13:27.793131 2572 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/57868c2b-17b7-49f4-a452-1b1db8a9e706-workload-socket" (OuterVolumeSpecName: "workload-socket") pod "57868c2b-17b7-49f4-a452-1b1db8a9e706" (UID: "57868c2b-17b7-49f4-a452-1b1db8a9e706"). InnerVolumeSpecName "workload-socket". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Jul 16 01:13:27.793850 ip-10-0-143-200 kubenswrapper[2572]: I0716 01:13:27.793503 2572 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/57868c2b-17b7-49f4-a452-1b1db8a9e706-istio-data" (OuterVolumeSpecName: "istio-data") pod "57868c2b-17b7-49f4-a452-1b1db8a9e706" (UID: "57868c2b-17b7-49f4-a452-1b1db8a9e706"). InnerVolumeSpecName "istio-data". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Jul 16 01:13:27.793850 ip-10-0-143-200 kubenswrapper[2572]: I0716 01:13:27.793797 2572 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/57868c2b-17b7-49f4-a452-1b1db8a9e706-workload-certs" (OuterVolumeSpecName: "workload-certs") pod "57868c2b-17b7-49f4-a452-1b1db8a9e706" (UID: "57868c2b-17b7-49f4-a452-1b1db8a9e706"). InnerVolumeSpecName "workload-certs". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Jul 16 01:13:27.794020 ip-10-0-143-200 kubenswrapper[2572]: I0716 01:13:27.793881 2572 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/57868c2b-17b7-49f4-a452-1b1db8a9e706-istiod-ca-cert" (OuterVolumeSpecName: "istiod-ca-cert") pod "57868c2b-17b7-49f4-a452-1b1db8a9e706" (UID: "57868c2b-17b7-49f4-a452-1b1db8a9e706"). InnerVolumeSpecName "istiod-ca-cert". PluginName "kubernetes.io/configmap", VolumeGIDValue "" Jul 16 01:13:27.794937 ip-10-0-143-200 kubenswrapper[2572]: I0716 01:13:27.794913 2572 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/downward-api/57868c2b-17b7-49f4-a452-1b1db8a9e706-istio-podinfo" (OuterVolumeSpecName: "istio-podinfo") pod "57868c2b-17b7-49f4-a452-1b1db8a9e706" (UID: "57868c2b-17b7-49f4-a452-1b1db8a9e706"). InnerVolumeSpecName "istio-podinfo". PluginName "kubernetes.io/downward-api", VolumeGIDValue "" Jul 16 01:13:27.795178 ip-10-0-143-200 kubenswrapper[2572]: I0716 01:13:27.795160 2572 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/57868c2b-17b7-49f4-a452-1b1db8a9e706-istio-token" (OuterVolumeSpecName: "istio-token") pod "57868c2b-17b7-49f4-a452-1b1db8a9e706" (UID: "57868c2b-17b7-49f4-a452-1b1db8a9e706"). InnerVolumeSpecName "istio-token". PluginName "kubernetes.io/projected", VolumeGIDValue "" Jul 16 01:13:27.795456 ip-10-0-143-200 kubenswrapper[2572]: I0716 01:13:27.795427 2572 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/57868c2b-17b7-49f4-a452-1b1db8a9e706-istio-envoy" (OuterVolumeSpecName: "istio-envoy") pod "57868c2b-17b7-49f4-a452-1b1db8a9e706" (UID: "57868c2b-17b7-49f4-a452-1b1db8a9e706"). InnerVolumeSpecName "istio-envoy". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Jul 16 01:13:27.795456 ip-10-0-143-200 kubenswrapper[2572]: I0716 01:13:27.795432 2572 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/57868c2b-17b7-49f4-a452-1b1db8a9e706-kube-api-access-2h85f" (OuterVolumeSpecName: "kube-api-access-2h85f") pod "57868c2b-17b7-49f4-a452-1b1db8a9e706" (UID: "57868c2b-17b7-49f4-a452-1b1db8a9e706"). InnerVolumeSpecName "kube-api-access-2h85f". PluginName "kubernetes.io/projected", VolumeGIDValue "" Jul 16 01:13:27.863606 ip-10-0-143-200 kubenswrapper[2572]: I0716 01:13:27.863570 2572 generic.go:356] "Generic (PLEG): container finished" podID="57868c2b-17b7-49f4-a452-1b1db8a9e706" containerID="012575f81483af091cbe2e9e9127b8a5a7e261486744ffa5f62b83e7374f45b2" exitCode=0 Jul 16 01:13:27.863825 ip-10-0-143-200 kubenswrapper[2572]: I0716 01:13:27.863654 2572 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-api-b-2814a2-openshift-default-9ccd5f9d5-rh5vs" Jul 16 01:13:27.863825 ip-10-0-143-200 kubenswrapper[2572]: I0716 01:13:27.863663 2572 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-api-b-2814a2-openshift-default-9ccd5f9d5-rh5vs" event={"ID":"57868c2b-17b7-49f4-a452-1b1db8a9e706","Type":"ContainerDied","Data":"012575f81483af091cbe2e9e9127b8a5a7e261486744ffa5f62b83e7374f45b2"} Jul 16 01:13:27.863825 ip-10-0-143-200 kubenswrapper[2572]: I0716 01:13:27.863734 2572 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-api-b-2814a2-openshift-default-9ccd5f9d5-rh5vs" event={"ID":"57868c2b-17b7-49f4-a452-1b1db8a9e706","Type":"ContainerDied","Data":"57a19b6ca02b5166dfdb76685eae602dcf8fa821d936df8f1bd839d188545064"} Jul 16 01:13:27.863825 ip-10-0-143-200 kubenswrapper[2572]: I0716 01:13:27.863764 2572 scope.go:117] "RemoveContainer" containerID="012575f81483af091cbe2e9e9127b8a5a7e261486744ffa5f62b83e7374f45b2" Jul 16 01:13:27.873413 ip-10-0-143-200 kubenswrapper[2572]: I0716 01:13:27.873392 2572 scope.go:117] "RemoveContainer" containerID="740ab1b2a1f513b74b278701db9f35b52ea7b72b90df928d45d794c4159f34e7" Jul 16 01:13:27.881513 ip-10-0-143-200 kubenswrapper[2572]: I0716 01:13:27.881497 2572 scope.go:117] "RemoveContainer" containerID="012575f81483af091cbe2e9e9127b8a5a7e261486744ffa5f62b83e7374f45b2" Jul 16 01:13:27.881819 ip-10-0-143-200 kubenswrapper[2572]: E0716 01:13:27.881784 2572 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"012575f81483af091cbe2e9e9127b8a5a7e261486744ffa5f62b83e7374f45b2\": container with ID starting with 012575f81483af091cbe2e9e9127b8a5a7e261486744ffa5f62b83e7374f45b2 not found: ID does not exist" containerID="012575f81483af091cbe2e9e9127b8a5a7e261486744ffa5f62b83e7374f45b2" Jul 16 01:13:27.881889 ip-10-0-143-200 kubenswrapper[2572]: I0716 01:13:27.881827 2572 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"012575f81483af091cbe2e9e9127b8a5a7e261486744ffa5f62b83e7374f45b2"} err="failed to get container status \"012575f81483af091cbe2e9e9127b8a5a7e261486744ffa5f62b83e7374f45b2\": rpc error: code = NotFound desc = could not find container \"012575f81483af091cbe2e9e9127b8a5a7e261486744ffa5f62b83e7374f45b2\": container with ID starting with 012575f81483af091cbe2e9e9127b8a5a7e261486744ffa5f62b83e7374f45b2 not found: ID does not exist" Jul 16 01:13:27.881889 ip-10-0-143-200 kubenswrapper[2572]: I0716 01:13:27.881842 2572 scope.go:117] "RemoveContainer" containerID="740ab1b2a1f513b74b278701db9f35b52ea7b72b90df928d45d794c4159f34e7" Jul 16 01:13:27.882081 ip-10-0-143-200 kubenswrapper[2572]: E0716 01:13:27.882064 2572 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"740ab1b2a1f513b74b278701db9f35b52ea7b72b90df928d45d794c4159f34e7\": container with ID starting with 740ab1b2a1f513b74b278701db9f35b52ea7b72b90df928d45d794c4159f34e7 not found: ID does not exist" containerID="740ab1b2a1f513b74b278701db9f35b52ea7b72b90df928d45d794c4159f34e7" Jul 16 01:13:27.882144 ip-10-0-143-200 kubenswrapper[2572]: I0716 01:13:27.882084 2572 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"740ab1b2a1f513b74b278701db9f35b52ea7b72b90df928d45d794c4159f34e7"} err="failed to get container status \"740ab1b2a1f513b74b278701db9f35b52ea7b72b90df928d45d794c4159f34e7\": rpc error: code = NotFound desc = could not find container \"740ab1b2a1f513b74b278701db9f35b52ea7b72b90df928d45d794c4159f34e7\": container with ID starting with 740ab1b2a1f513b74b278701db9f35b52ea7b72b90df928d45d794c4159f34e7 not found: ID does not exist" Jul 16 01:13:27.894102 ip-10-0-143-200 kubenswrapper[2572]: I0716 01:13:27.894065 2572 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-ingress/e2e-api-b-2814a2-openshift-default-9ccd5f9d5-rh5vs"] Jul 16 01:13:27.894256 ip-10-0-143-200 kubenswrapper[2572]: I0716 01:13:27.894186 2572 reconciler_common.go:299] "Volume detached for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/57868c2b-17b7-49f4-a452-1b1db8a9e706-istio-token\") on node \"ip-10-0-143-200.ec2.internal\" DevicePath \"\"" Jul 16 01:13:27.894256 ip-10-0-143-200 kubenswrapper[2572]: I0716 01:13:27.894209 2572 reconciler_common.go:299] "Volume detached for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/57868c2b-17b7-49f4-a452-1b1db8a9e706-istio-data\") on node \"ip-10-0-143-200.ec2.internal\" DevicePath \"\"" Jul 16 01:13:27.894256 ip-10-0-143-200 kubenswrapper[2572]: I0716 01:13:27.894221 2572 reconciler_common.go:299] "Volume detached for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/57868c2b-17b7-49f4-a452-1b1db8a9e706-istio-podinfo\") on node \"ip-10-0-143-200.ec2.internal\" DevicePath \"\"" Jul 16 01:13:27.894256 ip-10-0-143-200 kubenswrapper[2572]: I0716 01:13:27.894234 2572 reconciler_common.go:299] "Volume detached for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/57868c2b-17b7-49f4-a452-1b1db8a9e706-workload-certs\") on node \"ip-10-0-143-200.ec2.internal\" DevicePath \"\"" Jul 16 01:13:27.894256 ip-10-0-143-200 kubenswrapper[2572]: I0716 01:13:27.894248 2572 reconciler_common.go:299] "Volume detached for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/57868c2b-17b7-49f4-a452-1b1db8a9e706-workload-socket\") on node \"ip-10-0-143-200.ec2.internal\" DevicePath \"\"" Jul 16 01:13:27.894497 ip-10-0-143-200 kubenswrapper[2572]: I0716 01:13:27.894261 2572 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-2h85f\" (UniqueName: \"kubernetes.io/projected/57868c2b-17b7-49f4-a452-1b1db8a9e706-kube-api-access-2h85f\") on node \"ip-10-0-143-200.ec2.internal\" DevicePath \"\"" Jul 16 01:13:27.894497 ip-10-0-143-200 kubenswrapper[2572]: I0716 01:13:27.894274 2572 reconciler_common.go:299] "Volume detached for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/57868c2b-17b7-49f4-a452-1b1db8a9e706-istiod-ca-cert\") on node \"ip-10-0-143-200.ec2.internal\" DevicePath \"\"" Jul 16 01:13:27.894497 ip-10-0-143-200 kubenswrapper[2572]: I0716 01:13:27.894288 2572 reconciler_common.go:299] "Volume detached for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/57868c2b-17b7-49f4-a452-1b1db8a9e706-istio-envoy\") on node \"ip-10-0-143-200.ec2.internal\" DevicePath \"\"" Jul 16 01:13:27.897340 ip-10-0-143-200 kubenswrapper[2572]: I0716 01:13:27.897315 2572 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["openshift-ingress/e2e-api-b-2814a2-openshift-default-9ccd5f9d5-rh5vs"] Jul 16 01:13:29.448023 ip-10-0-143-200 kubenswrapper[2572]: I0716 01:13:29.447988 2572 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="57868c2b-17b7-49f4-a452-1b1db8a9e706" path="/var/lib/kubelet/pods/57868c2b-17b7-49f4-a452-1b1db8a9e706/volumes" Jul 16 01:13:31.310512 ip-10-0-143-200 kubenswrapper[2572]: I0716 01:13:31.310475 2572 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9gnmq_3a7e030d-fad8-480d-9dc6-fe5057048cf9/konnectivity-agent/0.log" limit=1024 Jul 16 01:13:31.676739 ip-10-0-143-200 kubenswrapper[2572]: I0716 01:13:31.676623 2572 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9gnmq_3a7e030d-fad8-480d-9dc6-fe5057048cf9/konnectivity-agent/0.log" limit=1024 Jul 16 01:13:32.029518 ip-10-0-143-200 kubenswrapper[2572]: I0716 01:13:32.029485 2572 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-ingress/e2e-shared-b-628c71-openshift-default-5d446f7cff-sp9c9"] Jul 16 01:13:32.030204 ip-10-0-143-200 kubenswrapper[2572]: I0716 01:13:32.030176 2572 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="57868c2b-17b7-49f4-a452-1b1db8a9e706" containerName="istio-proxy" Jul 16 01:13:32.030372 ip-10-0-143-200 kubenswrapper[2572]: I0716 01:13:32.030352 2572 state_mem.go:107] "Deleted CPUSet assignment" podUID="57868c2b-17b7-49f4-a452-1b1db8a9e706" containerName="istio-proxy" Jul 16 01:13:32.030464 ip-10-0-143-200 kubenswrapper[2572]: I0716 01:13:32.030398 2572 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="57868c2b-17b7-49f4-a452-1b1db8a9e706" containerName="istio-proxy" Jul 16 01:13:32.030464 ip-10-0-143-200 kubenswrapper[2572]: I0716 01:13:32.030408 2572 state_mem.go:107] "Deleted CPUSet assignment" podUID="57868c2b-17b7-49f4-a452-1b1db8a9e706" containerName="istio-proxy" Jul 16 01:13:32.030571 ip-10-0-143-200 kubenswrapper[2572]: I0716 01:13:32.030558 2572 memory_manager.go:368] "RemoveStaleState removing state" podUID="b6672fdb-54f4-4adc-844d-4072c3bf9876" containerName="istio-proxy" Jul 16 01:13:32.030606 ip-10-0-143-200 kubenswrapper[2572]: I0716 01:13:32.030575 2572 memory_manager.go:368] "RemoveStaleState removing state" podUID="57868c2b-17b7-49f4-a452-1b1db8a9e706" containerName="istio-proxy" Jul 16 01:13:32.030606 ip-10-0-143-200 kubenswrapper[2572]: I0716 01:13:32.030590 2572 memory_manager.go:368] "RemoveStaleState removing state" podUID="57868c2b-17b7-49f4-a452-1b1db8a9e706" containerName="istio-proxy" Jul 16 01:13:32.034509 ip-10-0-143-200 kubenswrapper[2572]: I0716 01:13:32.034488 2572 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-shared-b-628c71-openshift-default-5d446f7cff-sp9c9" Jul 16 01:13:32.037587 ip-10-0-143-200 kubenswrapper[2572]: I0716 01:13:32.037568 2572 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress\"/\"e2e-shared-b-628c71-openshift-default-dockercfg-r9qqr\"" Jul 16 01:13:32.054491 ip-10-0-143-200 kubenswrapper[2572]: I0716 01:13:32.054465 2572 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/e2e-shared-b-628c71-openshift-default-5d446f7cff-sp9c9"] Jul 16 01:13:32.065943 ip-10-0-143-200 kubenswrapper[2572]: I0716 01:13:32.065919 2572 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9gnmq_3a7e030d-fad8-480d-9dc6-fe5057048cf9/konnectivity-agent/0.log" limit=1024 Jul 16 01:13:32.136472 ip-10-0-143-200 kubenswrapper[2572]: I0716 01:13:32.136434 2572 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/4a730754-3193-49df-a072-f019e5b92dc2-istio-data\") pod \"e2e-shared-b-628c71-openshift-default-5d446f7cff-sp9c9\" (UID: \"4a730754-3193-49df-a072-f019e5b92dc2\") " pod="openshift-ingress/e2e-shared-b-628c71-openshift-default-5d446f7cff-sp9c9" Jul 16 01:13:32.136691 ip-10-0-143-200 kubenswrapper[2572]: I0716 01:13:32.136500 2572 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/4a730754-3193-49df-a072-f019e5b92dc2-istio-envoy\") pod \"e2e-shared-b-628c71-openshift-default-5d446f7cff-sp9c9\" (UID: \"4a730754-3193-49df-a072-f019e5b92dc2\") " pod="openshift-ingress/e2e-shared-b-628c71-openshift-default-5d446f7cff-sp9c9" Jul 16 01:13:32.136691 ip-10-0-143-200 kubenswrapper[2572]: I0716 01:13:32.136546 2572 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/4a730754-3193-49df-a072-f019e5b92dc2-workload-socket\") pod \"e2e-shared-b-628c71-openshift-default-5d446f7cff-sp9c9\" (UID: \"4a730754-3193-49df-a072-f019e5b92dc2\") " pod="openshift-ingress/e2e-shared-b-628c71-openshift-default-5d446f7cff-sp9c9" Jul 16 01:13:32.136691 ip-10-0-143-200 kubenswrapper[2572]: I0716 01:13:32.136591 2572 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/4a730754-3193-49df-a072-f019e5b92dc2-istio-podinfo\") pod \"e2e-shared-b-628c71-openshift-default-5d446f7cff-sp9c9\" (UID: \"4a730754-3193-49df-a072-f019e5b92dc2\") " pod="openshift-ingress/e2e-shared-b-628c71-openshift-default-5d446f7cff-sp9c9" Jul 16 01:13:32.136899 ip-10-0-143-200 kubenswrapper[2572]: I0716 01:13:32.136722 2572 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/4a730754-3193-49df-a072-f019e5b92dc2-workload-certs\") pod \"e2e-shared-b-628c71-openshift-default-5d446f7cff-sp9c9\" (UID: \"4a730754-3193-49df-a072-f019e5b92dc2\") " pod="openshift-ingress/e2e-shared-b-628c71-openshift-default-5d446f7cff-sp9c9" Jul 16 01:13:32.136899 ip-10-0-143-200 kubenswrapper[2572]: I0716 01:13:32.136770 2572 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/4a730754-3193-49df-a072-f019e5b92dc2-istio-token\") pod \"e2e-shared-b-628c71-openshift-default-5d446f7cff-sp9c9\" (UID: \"4a730754-3193-49df-a072-f019e5b92dc2\") " pod="openshift-ingress/e2e-shared-b-628c71-openshift-default-5d446f7cff-sp9c9" Jul 16 01:13:32.136899 ip-10-0-143-200 kubenswrapper[2572]: I0716 01:13:32.136846 2572 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/4a730754-3193-49df-a072-f019e5b92dc2-credential-socket\") pod \"e2e-shared-b-628c71-openshift-default-5d446f7cff-sp9c9\" (UID: \"4a730754-3193-49df-a072-f019e5b92dc2\") " pod="openshift-ingress/e2e-shared-b-628c71-openshift-default-5d446f7cff-sp9c9" Jul 16 01:13:32.137036 ip-10-0-143-200 kubenswrapper[2572]: I0716 01:13:32.136880 2572 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-z9g8w\" (UniqueName: \"kubernetes.io/projected/4a730754-3193-49df-a072-f019e5b92dc2-kube-api-access-z9g8w\") pod \"e2e-shared-b-628c71-openshift-default-5d446f7cff-sp9c9\" (UID: \"4a730754-3193-49df-a072-f019e5b92dc2\") " pod="openshift-ingress/e2e-shared-b-628c71-openshift-default-5d446f7cff-sp9c9" Jul 16 01:13:32.137036 ip-10-0-143-200 kubenswrapper[2572]: I0716 01:13:32.136944 2572 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/4a730754-3193-49df-a072-f019e5b92dc2-istiod-ca-cert\") pod \"e2e-shared-b-628c71-openshift-default-5d446f7cff-sp9c9\" (UID: \"4a730754-3193-49df-a072-f019e5b92dc2\") " pod="openshift-ingress/e2e-shared-b-628c71-openshift-default-5d446f7cff-sp9c9" Jul 16 01:13:32.237422 ip-10-0-143-200 kubenswrapper[2572]: I0716 01:13:32.237383 2572 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/4a730754-3193-49df-a072-f019e5b92dc2-workload-socket\") pod \"e2e-shared-b-628c71-openshift-default-5d446f7cff-sp9c9\" (UID: \"4a730754-3193-49df-a072-f019e5b92dc2\") " pod="openshift-ingress/e2e-shared-b-628c71-openshift-default-5d446f7cff-sp9c9" Jul 16 01:13:32.237618 ip-10-0-143-200 kubenswrapper[2572]: I0716 01:13:32.237436 2572 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/4a730754-3193-49df-a072-f019e5b92dc2-istio-podinfo\") pod \"e2e-shared-b-628c71-openshift-default-5d446f7cff-sp9c9\" (UID: \"4a730754-3193-49df-a072-f019e5b92dc2\") " pod="openshift-ingress/e2e-shared-b-628c71-openshift-default-5d446f7cff-sp9c9" Jul 16 01:13:32.237618 ip-10-0-143-200 kubenswrapper[2572]: I0716 01:13:32.237463 2572 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/4a730754-3193-49df-a072-f019e5b92dc2-workload-certs\") pod \"e2e-shared-b-628c71-openshift-default-5d446f7cff-sp9c9\" (UID: \"4a730754-3193-49df-a072-f019e5b92dc2\") " pod="openshift-ingress/e2e-shared-b-628c71-openshift-default-5d446f7cff-sp9c9" Jul 16 01:13:32.237618 ip-10-0-143-200 kubenswrapper[2572]: I0716 01:13:32.237484 2572 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/4a730754-3193-49df-a072-f019e5b92dc2-istio-token\") pod \"e2e-shared-b-628c71-openshift-default-5d446f7cff-sp9c9\" (UID: \"4a730754-3193-49df-a072-f019e5b92dc2\") " pod="openshift-ingress/e2e-shared-b-628c71-openshift-default-5d446f7cff-sp9c9" Jul 16 01:13:32.237618 ip-10-0-143-200 kubenswrapper[2572]: I0716 01:13:32.237507 2572 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/4a730754-3193-49df-a072-f019e5b92dc2-credential-socket\") pod \"e2e-shared-b-628c71-openshift-default-5d446f7cff-sp9c9\" (UID: \"4a730754-3193-49df-a072-f019e5b92dc2\") " pod="openshift-ingress/e2e-shared-b-628c71-openshift-default-5d446f7cff-sp9c9" Jul 16 01:13:32.237618 ip-10-0-143-200 kubenswrapper[2572]: I0716 01:13:32.237533 2572 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-z9g8w\" (UniqueName: \"kubernetes.io/projected/4a730754-3193-49df-a072-f019e5b92dc2-kube-api-access-z9g8w\") pod \"e2e-shared-b-628c71-openshift-default-5d446f7cff-sp9c9\" (UID: \"4a730754-3193-49df-a072-f019e5b92dc2\") " pod="openshift-ingress/e2e-shared-b-628c71-openshift-default-5d446f7cff-sp9c9" Jul 16 01:13:32.237618 ip-10-0-143-200 kubenswrapper[2572]: I0716 01:13:32.237577 2572 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/4a730754-3193-49df-a072-f019e5b92dc2-istiod-ca-cert\") pod \"e2e-shared-b-628c71-openshift-default-5d446f7cff-sp9c9\" (UID: \"4a730754-3193-49df-a072-f019e5b92dc2\") " pod="openshift-ingress/e2e-shared-b-628c71-openshift-default-5d446f7cff-sp9c9" Jul 16 01:13:32.237618 ip-10-0-143-200 kubenswrapper[2572]: I0716 01:13:32.237615 2572 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/4a730754-3193-49df-a072-f019e5b92dc2-istio-data\") pod \"e2e-shared-b-628c71-openshift-default-5d446f7cff-sp9c9\" (UID: \"4a730754-3193-49df-a072-f019e5b92dc2\") " pod="openshift-ingress/e2e-shared-b-628c71-openshift-default-5d446f7cff-sp9c9" Jul 16 01:13:32.238084 ip-10-0-143-200 kubenswrapper[2572]: I0716 01:13:32.237695 2572 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/4a730754-3193-49df-a072-f019e5b92dc2-istio-envoy\") pod \"e2e-shared-b-628c71-openshift-default-5d446f7cff-sp9c9\" (UID: \"4a730754-3193-49df-a072-f019e5b92dc2\") " pod="openshift-ingress/e2e-shared-b-628c71-openshift-default-5d446f7cff-sp9c9" Jul 16 01:13:32.238084 ip-10-0-143-200 kubenswrapper[2572]: I0716 01:13:32.237845 2572 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/4a730754-3193-49df-a072-f019e5b92dc2-workload-socket\") pod \"e2e-shared-b-628c71-openshift-default-5d446f7cff-sp9c9\" (UID: \"4a730754-3193-49df-a072-f019e5b92dc2\") " pod="openshift-ingress/e2e-shared-b-628c71-openshift-default-5d446f7cff-sp9c9" Jul 16 01:13:32.238084 ip-10-0-143-200 kubenswrapper[2572]: I0716 01:13:32.237884 2572 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/4a730754-3193-49df-a072-f019e5b92dc2-workload-certs\") pod \"e2e-shared-b-628c71-openshift-default-5d446f7cff-sp9c9\" (UID: \"4a730754-3193-49df-a072-f019e5b92dc2\") " pod="openshift-ingress/e2e-shared-b-628c71-openshift-default-5d446f7cff-sp9c9" Jul 16 01:13:32.238246 ip-10-0-143-200 kubenswrapper[2572]: I0716 01:13:32.238179 2572 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/4a730754-3193-49df-a072-f019e5b92dc2-credential-socket\") pod \"e2e-shared-b-628c71-openshift-default-5d446f7cff-sp9c9\" (UID: \"4a730754-3193-49df-a072-f019e5b92dc2\") " pod="openshift-ingress/e2e-shared-b-628c71-openshift-default-5d446f7cff-sp9c9" Jul 16 01:13:32.238458 ip-10-0-143-200 kubenswrapper[2572]: I0716 01:13:32.238437 2572 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/4a730754-3193-49df-a072-f019e5b92dc2-istio-data\") pod \"e2e-shared-b-628c71-openshift-default-5d446f7cff-sp9c9\" (UID: \"4a730754-3193-49df-a072-f019e5b92dc2\") " pod="openshift-ingress/e2e-shared-b-628c71-openshift-default-5d446f7cff-sp9c9" Jul 16 01:13:32.238835 ip-10-0-143-200 kubenswrapper[2572]: I0716 01:13:32.238808 2572 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/4a730754-3193-49df-a072-f019e5b92dc2-istiod-ca-cert\") pod \"e2e-shared-b-628c71-openshift-default-5d446f7cff-sp9c9\" (UID: \"4a730754-3193-49df-a072-f019e5b92dc2\") " pod="openshift-ingress/e2e-shared-b-628c71-openshift-default-5d446f7cff-sp9c9" Jul 16 01:13:32.240549 ip-10-0-143-200 kubenswrapper[2572]: I0716 01:13:32.240522 2572 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/4a730754-3193-49df-a072-f019e5b92dc2-istio-podinfo\") pod \"e2e-shared-b-628c71-openshift-default-5d446f7cff-sp9c9\" (UID: \"4a730754-3193-49df-a072-f019e5b92dc2\") " pod="openshift-ingress/e2e-shared-b-628c71-openshift-default-5d446f7cff-sp9c9" Jul 16 01:13:32.240622 ip-10-0-143-200 kubenswrapper[2572]: I0716 01:13:32.240603 2572 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/4a730754-3193-49df-a072-f019e5b92dc2-istio-envoy\") pod \"e2e-shared-b-628c71-openshift-default-5d446f7cff-sp9c9\" (UID: \"4a730754-3193-49df-a072-f019e5b92dc2\") " pod="openshift-ingress/e2e-shared-b-628c71-openshift-default-5d446f7cff-sp9c9" Jul 16 01:13:32.247030 ip-10-0-143-200 kubenswrapper[2572]: I0716 01:13:32.247004 2572 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/4a730754-3193-49df-a072-f019e5b92dc2-istio-token\") pod \"e2e-shared-b-628c71-openshift-default-5d446f7cff-sp9c9\" (UID: \"4a730754-3193-49df-a072-f019e5b92dc2\") " pod="openshift-ingress/e2e-shared-b-628c71-openshift-default-5d446f7cff-sp9c9" Jul 16 01:13:32.247281 ip-10-0-143-200 kubenswrapper[2572]: I0716 01:13:32.247257 2572 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-z9g8w\" (UniqueName: \"kubernetes.io/projected/4a730754-3193-49df-a072-f019e5b92dc2-kube-api-access-z9g8w\") pod \"e2e-shared-b-628c71-openshift-default-5d446f7cff-sp9c9\" (UID: \"4a730754-3193-49df-a072-f019e5b92dc2\") " pod="openshift-ingress/e2e-shared-b-628c71-openshift-default-5d446f7cff-sp9c9" Jul 16 01:13:32.345390 ip-10-0-143-200 kubenswrapper[2572]: I0716 01:13:32.345294 2572 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-shared-b-628c71-openshift-default-5d446f7cff-sp9c9" Jul 16 01:13:32.556038 ip-10-0-143-200 kubenswrapper[2572]: I0716 01:13:32.556013 2572 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/e2e-shared-b-628c71-openshift-default-5d446f7cff-sp9c9"] Jul 16 01:13:32.558292 ip-10-0-143-200 kubenswrapper[2572]: W0716 01:13:32.558266 2572 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod4a730754_3193_49df_a072_f019e5b92dc2.slice/crio-be267733117155f4ae07c4f8beef2e9c0d93b08bbaf6ecf63d2f2bff470295e4 WatchSource:0}: Error finding container be267733117155f4ae07c4f8beef2e9c0d93b08bbaf6ecf63d2f2bff470295e4: Status 404 returned error can't find the container with id be267733117155f4ae07c4f8beef2e9c0d93b08bbaf6ecf63d2f2bff470295e4 Jul 16 01:13:32.560794 ip-10-0-143-200 kubenswrapper[2572]: I0716 01:13:32.560759 2572 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236228Ki","pods":"250"} Jul 16 01:13:32.560877 ip-10-0-143-200 kubenswrapper[2572]: I0716 01:13:32.560842 2572 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236228Ki","pods":"250"} Jul 16 01:13:32.560914 ip-10-0-143-200 kubenswrapper[2572]: I0716 01:13:32.560892 2572 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236228Ki","pods":"250"} Jul 16 01:13:32.604654 ip-10-0-143-200 kubenswrapper[2572]: I0716 01:13:32.601134 2572 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9gnmq_3a7e030d-fad8-480d-9dc6-fe5057048cf9/konnectivity-agent/0.log" limit=1024 Jul 16 01:13:32.898318 ip-10-0-143-200 kubenswrapper[2572]: I0716 01:13:32.898212 2572 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-shared-b-628c71-openshift-default-5d446f7cff-sp9c9" event={"ID":"4a730754-3193-49df-a072-f019e5b92dc2","Type":"ContainerStarted","Data":"cd76511ee253b7f7162dbcb9786e80feb4de1619614c4dfcb7dc9fa5206a46c6"} Jul 16 01:13:32.898318 ip-10-0-143-200 kubenswrapper[2572]: I0716 01:13:32.898291 2572 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-shared-b-628c71-openshift-default-5d446f7cff-sp9c9" event={"ID":"4a730754-3193-49df-a072-f019e5b92dc2","Type":"ContainerStarted","Data":"be267733117155f4ae07c4f8beef2e9c0d93b08bbaf6ecf63d2f2bff470295e4"} Jul 16 01:13:32.929755 ip-10-0-143-200 kubenswrapper[2572]: I0716 01:13:32.929704 2572 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-ingress/e2e-shared-b-628c71-openshift-default-5d446f7cff-sp9c9" podStartSLOduration=1.929685216 podStartE2EDuration="1.929685216s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-16 01:13:31 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-16 01:13:32.926956957 +0000 UTC m=+2018.035711632" watchObservedRunningTime="2026-07-16 01:13:32.929685216 +0000 UTC m=+2018.038439896" Jul 16 01:13:33.345638 ip-10-0-143-200 kubenswrapper[2572]: I0716 01:13:33.345599 2572 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="openshift-ingress/e2e-shared-b-628c71-openshift-default-5d446f7cff-sp9c9" Jul 16 01:13:33.351099 ip-10-0-143-200 kubenswrapper[2572]: I0716 01:13:33.351073 2572 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="openshift-ingress/e2e-shared-b-628c71-openshift-default-5d446f7cff-sp9c9" Jul 16 01:13:33.898390 ip-10-0-143-200 kubenswrapper[2572]: I0716 01:13:33.898347 2572 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9gnmq_3a7e030d-fad8-480d-9dc6-fe5057048cf9/konnectivity-agent/0.log" limit=1024 Jul 16 01:13:33.907075 ip-10-0-143-200 kubenswrapper[2572]: I0716 01:13:33.907038 2572 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/e2e-shared-b-628c71-openshift-default-5d446f7cff-sp9c9" Jul 16 01:13:33.912653 ip-10-0-143-200 kubenswrapper[2572]: I0716 01:13:33.911874 2572 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/e2e-shared-b-628c71-openshift-default-5d446f7cff-sp9c9" Jul 16 01:13:34.050876 ip-10-0-143-200 kubenswrapper[2572]: I0716 01:13:34.050835 2572 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9gnmq_3a7e030d-fad8-480d-9dc6-fe5057048cf9/konnectivity-agent/0.log" limit=1024 Jul 16 01:13:34.130025 ip-10-0-143-200 kubenswrapper[2572]: I0716 01:13:34.129970 2572 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9gnmq_3a7e030d-fad8-480d-9dc6-fe5057048cf9/konnectivity-agent/0.log" limit=1024 Jul 16 01:13:34.613937 ip-10-0-143-200 kubenswrapper[2572]: I0716 01:13:34.613902 2572 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9gnmq_3a7e030d-fad8-480d-9dc6-fe5057048cf9/konnectivity-agent/0.log" limit=1024 Jul 16 01:13:34.710760 ip-10-0-143-200 kubenswrapper[2572]: I0716 01:13:34.710725 2572 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9gnmq_3a7e030d-fad8-480d-9dc6-fe5057048cf9/konnectivity-agent/0.log" limit=1024 Jul 16 01:13:37.932760 ip-10-0-143-200 kubenswrapper[2572]: I0716 01:13:37.932665 2572 generic.go:356] "Generic (PLEG): container finished" podID="c3a9a894-3f1a-4746-a760-831441b8ac8b" containerID="eaac463958ff9bae0954276dddf7bb360f38fde51746934c5d22c8b9f804ffc5" exitCode=0 Jul 16 01:13:37.932760 ip-10-0-143-200 kubenswrapper[2572]: I0716 01:13:37.932706 2572 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-shared-a-f9f50d-openshift-default-5d5b68dcd7-xl2x6" event={"ID":"c3a9a894-3f1a-4746-a760-831441b8ac8b","Type":"ContainerDied","Data":"eaac463958ff9bae0954276dddf7bb360f38fde51746934c5d22c8b9f804ffc5"} Jul 16 01:13:37.933235 ip-10-0-143-200 kubenswrapper[2572]: I0716 01:13:37.933142 2572 scope.go:117] "RemoveContainer" containerID="eaac463958ff9bae0954276dddf7bb360f38fde51746934c5d22c8b9f804ffc5" Jul 16 01:13:37.934600 ip-10-0-143-200 kubenswrapper[2572]: I0716 01:13:37.934559 2572 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236228Ki","pods":"250"} Jul 16 01:13:37.934716 ip-10-0-143-200 kubenswrapper[2572]: I0716 01:13:37.934659 2572 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236228Ki","pods":"250"} Jul 16 01:13:37.934716 ip-10-0-143-200 kubenswrapper[2572]: I0716 01:13:37.934692 2572 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236228Ki","pods":"250"} Jul 16 01:13:38.940530 ip-10-0-143-200 kubenswrapper[2572]: I0716 01:13:38.940497 2572 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-shared-a-f9f50d-openshift-default-5d5b68dcd7-xl2x6" event={"ID":"c3a9a894-3f1a-4746-a760-831441b8ac8b","Type":"ContainerStarted","Data":"67ec4749753860e169712b2bdacf62af10fdc37c593739009128208995f2e186"} Jul 16 01:13:39.376862 ip-10-0-143-200 kubenswrapper[2572]: I0716 01:13:39.376828 2572 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9gnmq_3a7e030d-fad8-480d-9dc6-fe5057048cf9/konnectivity-agent/0.log" limit=1024 Jul 16 01:13:39.720898 ip-10-0-143-200 kubenswrapper[2572]: I0716 01:13:39.720856 2572 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="openshift-ingress/e2e-shared-a-f9f50d-openshift-default-5d5b68dcd7-xl2x6" Jul 16 01:13:39.722276 ip-10-0-143-200 kubenswrapper[2572]: I0716 01:13:39.722247 2572 patch_prober.go:28] interesting pod/e2e-shared-a-f9f50d-openshift-default-5d5b68dcd7-xl2x6 container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.132.0.61:15021/healthz/ready\": dial tcp 10.132.0.61:15021: connect: connection refused" start-of-body= Jul 16 01:13:39.722397 ip-10-0-143-200 kubenswrapper[2572]: I0716 01:13:39.722300 2572 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-shared-a-f9f50d-openshift-default-5d5b68dcd7-xl2x6" podUID="c3a9a894-3f1a-4746-a760-831441b8ac8b" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.132.0.61:15021/healthz/ready\": dial tcp 10.132.0.61:15021: connect: connection refused" Jul 16 01:13:40.721620 ip-10-0-143-200 kubenswrapper[2572]: I0716 01:13:40.721577 2572 patch_prober.go:28] interesting pod/e2e-shared-a-f9f50d-openshift-default-5d5b68dcd7-xl2x6 container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.132.0.61:15021/healthz/ready\": dial tcp 10.132.0.61:15021: connect: connection refused" start-of-body= Jul 16 01:13:40.722060 ip-10-0-143-200 kubenswrapper[2572]: I0716 01:13:40.721671 2572 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-shared-a-f9f50d-openshift-default-5d5b68dcd7-xl2x6" podUID="c3a9a894-3f1a-4746-a760-831441b8ac8b" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.132.0.61:15021/healthz/ready\": dial tcp 10.132.0.61:15021: connect: connection refused" Jul 16 01:13:40.849577 ip-10-0-143-200 kubenswrapper[2572]: I0716 01:13:40.849520 2572 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/e2e-shared-a-f9f50d-openshift-default-5d5b68dcd7-xl2x6" Jul 16 01:13:41.721649 ip-10-0-143-200 kubenswrapper[2572]: I0716 01:13:41.721071 2572 patch_prober.go:28] interesting pod/e2e-shared-a-f9f50d-openshift-default-5d5b68dcd7-xl2x6 container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.132.0.61:15021/healthz/ready\": dial tcp 10.132.0.61:15021: connect: connection refused" start-of-body= Jul 16 01:13:41.721649 ip-10-0-143-200 kubenswrapper[2572]: I0716 01:13:41.721140 2572 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-shared-a-f9f50d-openshift-default-5d5b68dcd7-xl2x6" podUID="c3a9a894-3f1a-4746-a760-831441b8ac8b" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.132.0.61:15021/healthz/ready\": dial tcp 10.132.0.61:15021: connect: connection refused" Jul 16 01:13:42.722012 ip-10-0-143-200 kubenswrapper[2572]: I0716 01:13:42.721965 2572 patch_prober.go:28] interesting pod/e2e-shared-a-f9f50d-openshift-default-5d5b68dcd7-xl2x6 container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.132.0.61:15021/healthz/ready\": dial tcp 10.132.0.61:15021: connect: connection refused" start-of-body= Jul 16 01:13:42.722515 ip-10-0-143-200 kubenswrapper[2572]: I0716 01:13:42.722048 2572 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-shared-a-f9f50d-openshift-default-5d5b68dcd7-xl2x6" podUID="c3a9a894-3f1a-4746-a760-831441b8ac8b" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.132.0.61:15021/healthz/ready\": dial tcp 10.132.0.61:15021: connect: connection refused" Jul 16 01:13:43.724718 ip-10-0-143-200 kubenswrapper[2572]: I0716 01:13:43.724685 2572 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="openshift-ingress/e2e-shared-a-f9f50d-openshift-default-5d5b68dcd7-xl2x6" Jul 16 01:13:43.725824 ip-10-0-143-200 kubenswrapper[2572]: I0716 01:13:43.725803 2572 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/e2e-shared-a-f9f50d-openshift-default-5d5b68dcd7-xl2x6" Jul 16 01:13:47.564075 ip-10-0-143-200 kubenswrapper[2572]: I0716 01:13:47.564027 2572 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9gnmq_3a7e030d-fad8-480d-9dc6-fe5057048cf9/konnectivity-agent/0.log" limit=1024 Jul 16 01:13:47.669874 ip-10-0-143-200 kubenswrapper[2572]: I0716 01:13:47.669832 2572 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9gnmq_3a7e030d-fad8-480d-9dc6-fe5057048cf9/konnectivity-agent/0.log" limit=1024 Jul 16 01:13:48.909018 ip-10-0-143-200 kubenswrapper[2572]: I0716 01:13:48.908980 2572 patch_prober.go:28] interesting pod/e2e-shared-b-628c71-openshift-default-5d446f7cff-sp9c9 container/istio-proxy namespace/openshift-ingress: Readiness probe status=failure output="Get \"http://10.132.0.62:15021/healthz/ready\": dial tcp 10.132.0.62:15021: connect: connection refused" start-of-body= Jul 16 01:13:48.909406 ip-10-0-143-200 kubenswrapper[2572]: I0716 01:13:48.909046 2572 prober.go:120] "Probe failed" probeType="Readiness" pod="openshift-ingress/e2e-shared-b-628c71-openshift-default-5d446f7cff-sp9c9" podUID="4a730754-3193-49df-a072-f019e5b92dc2" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.132.0.62:15021/healthz/ready\": dial tcp 10.132.0.62:15021: connect: connection refused" Jul 16 01:13:48.999322 ip-10-0-143-200 kubenswrapper[2572]: I0716 01:13:48.999291 2572 generic.go:356] "Generic (PLEG): container finished" podID="4a730754-3193-49df-a072-f019e5b92dc2" containerID="cd76511ee253b7f7162dbcb9786e80feb4de1619614c4dfcb7dc9fa5206a46c6" exitCode=0 Jul 16 01:13:48.999482 ip-10-0-143-200 kubenswrapper[2572]: I0716 01:13:48.999364 2572 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-shared-b-628c71-openshift-default-5d446f7cff-sp9c9" event={"ID":"4a730754-3193-49df-a072-f019e5b92dc2","Type":"ContainerDied","Data":"cd76511ee253b7f7162dbcb9786e80feb4de1619614c4dfcb7dc9fa5206a46c6"} Jul 16 01:13:48.999833 ip-10-0-143-200 kubenswrapper[2572]: I0716 01:13:48.999819 2572 scope.go:117] "RemoveContainer" containerID="cd76511ee253b7f7162dbcb9786e80feb4de1619614c4dfcb7dc9fa5206a46c6" Jul 16 01:13:49.000976 ip-10-0-143-200 kubenswrapper[2572]: I0716 01:13:49.000949 2572 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236228Ki","pods":"250"} Jul 16 01:13:49.001044 ip-10-0-143-200 kubenswrapper[2572]: I0716 01:13:49.001017 2572 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236228Ki","pods":"250"} Jul 16 01:13:49.001081 ip-10-0-143-200 kubenswrapper[2572]: I0716 01:13:49.001054 2572 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236228Ki","pods":"250"} Jul 16 01:13:50.007170 ip-10-0-143-200 kubenswrapper[2572]: I0716 01:13:50.007121 2572 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-shared-b-628c71-openshift-default-5d446f7cff-sp9c9" event={"ID":"4a730754-3193-49df-a072-f019e5b92dc2","Type":"ContainerStarted","Data":"ee46d87c7e2514b7e86b33dd2a999f6a9292010ad113a5174124c7df1bf0c8c4"} Jul 16 01:13:50.346370 ip-10-0-143-200 kubenswrapper[2572]: I0716 01:13:50.346279 2572 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="openshift-ingress/e2e-shared-b-628c71-openshift-default-5d446f7cff-sp9c9" Jul 16 01:13:50.346650 ip-10-0-143-200 kubenswrapper[2572]: I0716 01:13:50.346602 2572 patch_prober.go:28] interesting pod/e2e-shared-b-628c71-openshift-default-5d446f7cff-sp9c9 container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.132.0.62:15021/healthz/ready\": dial tcp 10.132.0.62:15021: connect: connection refused" start-of-body= Jul 16 01:13:50.346721 ip-10-0-143-200 kubenswrapper[2572]: I0716 01:13:50.346675 2572 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-shared-b-628c71-openshift-default-5d446f7cff-sp9c9" podUID="4a730754-3193-49df-a072-f019e5b92dc2" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.132.0.62:15021/healthz/ready\": dial tcp 10.132.0.62:15021: connect: connection refused" Jul 16 01:13:51.345886 ip-10-0-143-200 kubenswrapper[2572]: I0716 01:13:51.345850 2572 patch_prober.go:28] interesting pod/e2e-shared-b-628c71-openshift-default-5d446f7cff-sp9c9 container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.132.0.62:15021/healthz/ready\": dial tcp 10.132.0.62:15021: connect: connection refused" start-of-body= Jul 16 01:13:51.346292 ip-10-0-143-200 kubenswrapper[2572]: I0716 01:13:51.345909 2572 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-shared-b-628c71-openshift-default-5d446f7cff-sp9c9" podUID="4a730754-3193-49df-a072-f019e5b92dc2" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.132.0.62:15021/healthz/ready\": dial tcp 10.132.0.62:15021: connect: connection refused" Jul 16 01:13:51.506247 ip-10-0-143-200 kubenswrapper[2572]: I0716 01:13:51.506196 2572 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9gnmq_3a7e030d-fad8-480d-9dc6-fe5057048cf9/konnectivity-agent/0.log" limit=1024 Jul 16 01:13:52.346175 ip-10-0-143-200 kubenswrapper[2572]: I0716 01:13:52.346143 2572 patch_prober.go:28] interesting pod/e2e-shared-b-628c71-openshift-default-5d446f7cff-sp9c9 container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.132.0.62:15021/healthz/ready\": dial tcp 10.132.0.62:15021: connect: connection refused" start-of-body= Jul 16 01:13:52.346715 ip-10-0-143-200 kubenswrapper[2572]: I0716 01:13:52.346686 2572 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-shared-b-628c71-openshift-default-5d446f7cff-sp9c9" podUID="4a730754-3193-49df-a072-f019e5b92dc2" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.132.0.62:15021/healthz/ready\": dial tcp 10.132.0.62:15021: connect: connection refused" Jul 16 01:13:53.346547 ip-10-0-143-200 kubenswrapper[2572]: I0716 01:13:53.346510 2572 patch_prober.go:28] interesting pod/e2e-shared-b-628c71-openshift-default-5d446f7cff-sp9c9 container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.132.0.62:15021/healthz/ready\": dial tcp 10.132.0.62:15021: connect: connection refused" start-of-body= Jul 16 01:13:53.347142 ip-10-0-143-200 kubenswrapper[2572]: I0716 01:13:53.347092 2572 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-shared-b-628c71-openshift-default-5d446f7cff-sp9c9" podUID="4a730754-3193-49df-a072-f019e5b92dc2" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.132.0.62:15021/healthz/ready\": dial tcp 10.132.0.62:15021: connect: connection refused" Jul 16 01:13:54.349602 ip-10-0-143-200 kubenswrapper[2572]: I0716 01:13:54.349567 2572 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="openshift-ingress/e2e-shared-b-628c71-openshift-default-5d446f7cff-sp9c9" Jul 16 01:13:54.350118 ip-10-0-143-200 kubenswrapper[2572]: I0716 01:13:54.349983 2572 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/e2e-shared-b-628c71-openshift-default-5d446f7cff-sp9c9" Jul 16 01:13:54.350729 ip-10-0-143-200 kubenswrapper[2572]: I0716 01:13:54.350706 2572 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/e2e-shared-b-628c71-openshift-default-5d446f7cff-sp9c9" Jul 16 01:13:55.412300 ip-10-0-143-200 kubenswrapper[2572]: I0716 01:13:55.412256 2572 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9gnmq_3a7e030d-fad8-480d-9dc6-fe5057048cf9/konnectivity-agent/0.log" limit=1024 Jul 16 01:13:55.512781 ip-10-0-143-200 kubenswrapper[2572]: I0716 01:13:55.512745 2572 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9gnmq_3a7e030d-fad8-480d-9dc6-fe5057048cf9/konnectivity-agent/0.log" limit=1024 Jul 16 01:14:03.038308 ip-10-0-143-200 kubenswrapper[2572]: I0716 01:14:03.038266 2572 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9gnmq_3a7e030d-fad8-480d-9dc6-fe5057048cf9/konnectivity-agent/0.log" limit=1024 Jul 16 01:14:09.537446 ip-10-0-143-200 kubenswrapper[2572]: I0716 01:14:09.537417 2572 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9gnmq_3a7e030d-fad8-480d-9dc6-fe5057048cf9/konnectivity-agent/0.log" limit=1024 Jul 16 01:14:18.498821 ip-10-0-143-200 kubenswrapper[2572]: I0716 01:14:18.498725 2572 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9gnmq_3a7e030d-fad8-480d-9dc6-fe5057048cf9/konnectivity-agent/0.log" limit=1024 Jul 16 01:14:18.603387 ip-10-0-143-200 kubenswrapper[2572]: I0716 01:14:18.603353 2572 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9gnmq_3a7e030d-fad8-480d-9dc6-fe5057048cf9/konnectivity-agent/0.log" limit=1024 Jul 16 01:14:45.368285 ip-10-0-143-200 kubenswrapper[2572]: I0716 01:14:45.368247 2572 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9gnmq_3a7e030d-fad8-480d-9dc6-fe5057048cf9/konnectivity-agent/0.log" limit=1024 Jul 16 01:14:45.470738 ip-10-0-143-200 kubenswrapper[2572]: I0716 01:14:45.470704 2572 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9gnmq_3a7e030d-fad8-480d-9dc6-fe5057048cf9/konnectivity-agent/0.log" limit=1024 Jul 16 01:14:55.529850 ip-10-0-143-200 kubenswrapper[2572]: I0716 01:14:55.529821 2572 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-console-operator_console-operator-7866497494-pjc6j_a64bcd05-8596-4c12-b88e-e974d676e25f/console-operator/2.log" Jul 16 01:14:55.534765 ip-10-0-143-200 kubenswrapper[2572]: I0716 01:14:55.534740 2572 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ovn-kubernetes_ovnkube-node-fp78g_aa66f2b6-0ad8-45d0-8ff5-40fc21966b5a/ovn-acl-logging/0.log" Jul 16 01:14:55.538713 ip-10-0-143-200 kubenswrapper[2572]: I0716 01:14:55.538684 2572 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-console-operator_console-operator-7866497494-pjc6j_a64bcd05-8596-4c12-b88e-e974d676e25f/console-operator/2.log" Jul 16 01:14:55.543502 ip-10-0-143-200 kubenswrapper[2572]: I0716 01:14:55.543485 2572 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ovn-kubernetes_ovnkube-node-fp78g_aa66f2b6-0ad8-45d0-8ff5-40fc21966b5a/ovn-acl-logging/0.log" Jul 16 01:15:00.154329 ip-10-0-143-200 kubenswrapper[2572]: I0716 01:15:00.154287 2572 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["odh-ai-gateway-infra/maas-api-key-cleanup-e2e-inf-b-e5854b-29736075-2wdsn"] Jul 16 01:15:00.158180 ip-10-0-143-200 kubenswrapper[2572]: I0716 01:15:00.158157 2572 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-key-cleanup-e2e-inf-b-e5854b-29736075-2wdsn" Jul 16 01:15:00.165428 ip-10-0-143-200 kubenswrapper[2572]: I0716 01:15:00.165368 2572 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"odh-ai-gateway-infra\"/\"maas-api-cleanup-dockercfg-trkf6\"" Jul 16 01:15:00.169522 ip-10-0-143-200 kubenswrapper[2572]: I0716 01:15:00.169496 2572 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-key-cleanup-e2e-inf-b-e5854b-29736075-2wdsn" Jul 16 01:15:00.172339 ip-10-0-143-200 kubenswrapper[2572]: I0716 01:15:00.172309 2572 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-key-cleanup-e2e-inf-b-e5854b-29736075-2wdsn"] Jul 16 01:15:00.346108 ip-10-0-143-200 kubenswrapper[2572]: I0716 01:15:00.346077 2572 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-key-cleanup-e2e-inf-b-e5854b-29736075-2wdsn"] Jul 16 01:15:00.347855 ip-10-0-143-200 kubenswrapper[2572]: W0716 01:15:00.347829 2572 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podbf10ff5f_2652_4637_92c0_65f7672204c3.slice/crio-77d6e4518bb303de695762128e018e956c431b6f9512af546a7dade9484aedb7 WatchSource:0}: Error finding container 77d6e4518bb303de695762128e018e956c431b6f9512af546a7dade9484aedb7: Status 404 returned error can't find the container with id 77d6e4518bb303de695762128e018e956c431b6f9512af546a7dade9484aedb7 Jul 16 01:15:00.349798 ip-10-0-143-200 kubenswrapper[2572]: I0716 01:15:00.349782 2572 provider.go:93] Refreshing cache for provider: *credentialprovider.defaultDockerConfigProvider Jul 16 01:15:00.403157 ip-10-0-143-200 kubenswrapper[2572]: I0716 01:15:00.403119 2572 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-key-cleanup-e2e-inf-b-e5854b-29736075-2wdsn" event={"ID":"bf10ff5f-2652-4637-92c0-65f7672204c3","Type":"ContainerStarted","Data":"77d6e4518bb303de695762128e018e956c431b6f9512af546a7dade9484aedb7"} Jul 16 01:15:03.422609 ip-10-0-143-200 kubenswrapper[2572]: I0716 01:15:03.422494 2572 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-key-cleanup-e2e-inf-b-e5854b-29736075-2wdsn" event={"ID":"bf10ff5f-2652-4637-92c0-65f7672204c3","Type":"ContainerStarted","Data":"885081f46439b275ef96eddf3363a900c274b3c83603cc2d76a61b789e8a6c18"} Jul 16 01:15:03.458399 ip-10-0-143-200 kubenswrapper[2572]: I0716 01:15:03.458345 2572 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="odh-ai-gateway-infra/maas-api-key-cleanup-e2e-inf-b-e5854b-29736075-2wdsn" podStartSLOduration=1.098153741 podStartE2EDuration="3.458328717s" totalImagesPullingTime="2.360174976s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-16 01:15:00 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-16 01:15:03.457599352 +0000 UTC m=+2108.566354027" watchObservedRunningTime="2026-07-16 01:15:03.458328717 +0000 UTC m=+2108.567083392" Jul 16 01:15:04.444129 ip-10-0-143-200 kubenswrapper[2572]: I0716 01:15:04.444090 2572 generic.go:356] "Generic (PLEG): container finished" podID="bf10ff5f-2652-4637-92c0-65f7672204c3" containerID="885081f46439b275ef96eddf3363a900c274b3c83603cc2d76a61b789e8a6c18" exitCode=0 Jul 16 01:15:04.444650 ip-10-0-143-200 kubenswrapper[2572]: I0716 01:15:04.444176 2572 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-key-cleanup-e2e-inf-b-e5854b-29736075-2wdsn" event={"ID":"bf10ff5f-2652-4637-92c0-65f7672204c3","Type":"ContainerDied","Data":"885081f46439b275ef96eddf3363a900c274b3c83603cc2d76a61b789e8a6c18"} Jul 16 01:15:05.604143 ip-10-0-143-200 kubenswrapper[2572]: I0716 01:15:05.604098 2572 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-key-cleanup-e2e-inf-b-e5854b-29736075-2wdsn" Jul 16 01:15:06.457810 ip-10-0-143-200 kubenswrapper[2572]: I0716 01:15:06.457777 2572 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-key-cleanup-e2e-inf-b-e5854b-29736075-2wdsn" Jul 16 01:15:06.457810 ip-10-0-143-200 kubenswrapper[2572]: I0716 01:15:06.457791 2572 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-key-cleanup-e2e-inf-b-e5854b-29736075-2wdsn" event={"ID":"bf10ff5f-2652-4637-92c0-65f7672204c3","Type":"ContainerDied","Data":"77d6e4518bb303de695762128e018e956c431b6f9512af546a7dade9484aedb7"} Jul 16 01:15:06.458032 ip-10-0-143-200 kubenswrapper[2572]: I0716 01:15:06.457837 2572 pod_container_deletor.go:80] "Container not found in pod's containers" containerID="77d6e4518bb303de695762128e018e956c431b6f9512af546a7dade9484aedb7" Jul 16 01:16:44.111990 ip-10-0-143-200 kubenswrapper[2572]: I0716 01:16:44.111945 2572 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9gnmq_3a7e030d-fad8-480d-9dc6-fe5057048cf9/konnectivity-agent/0.log" limit=1024 Jul 16 01:16:45.425874 ip-10-0-143-200 kubenswrapper[2572]: I0716 01:16:45.425836 2572 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9gnmq_3a7e030d-fad8-480d-9dc6-fe5057048cf9/konnectivity-agent/0.log" limit=1024 Jul 16 01:17:09.480696 ip-10-0-143-200 kubenswrapper[2572]: I0716 01:17:09.480648 2572 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9gnmq_3a7e030d-fad8-480d-9dc6-fe5057048cf9/konnectivity-agent/0.log" limit=1024 Jul 16 01:17:09.580335 ip-10-0-143-200 kubenswrapper[2572]: I0716 01:17:09.580298 2572 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9gnmq_3a7e030d-fad8-480d-9dc6-fe5057048cf9/konnectivity-agent/0.log" limit=1024 Jul 16 01:17:20.677927 ip-10-0-143-200 kubenswrapper[2572]: I0716 01:17:20.677835 2572 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9gnmq_3a7e030d-fad8-480d-9dc6-fe5057048cf9/konnectivity-agent/0.log" limit=1024 Jul 16 01:17:55.703831 ip-10-0-143-200 kubenswrapper[2572]: I0716 01:17:55.703786 2572 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9gnmq_3a7e030d-fad8-480d-9dc6-fe5057048cf9/konnectivity-agent/0.log" limit=1024 Jul 16 01:18:18.449208 ip-10-0-143-200 kubenswrapper[2572]: I0716 01:18:18.449158 2572 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9gnmq_3a7e030d-fad8-480d-9dc6-fe5057048cf9/konnectivity-agent/0.log" limit=1024 Jul 16 01:18:35.837135 ip-10-0-143-200 kubenswrapper[2572]: I0716 01:18:35.837099 2572 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9gnmq_3a7e030d-fad8-480d-9dc6-fe5057048cf9/konnectivity-agent/0.log" limit=1024 Jul 16 01:18:36.746849 ip-10-0-143-200 kubenswrapper[2572]: I0716 01:18:36.746810 2572 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9gnmq_3a7e030d-fad8-480d-9dc6-fe5057048cf9/konnectivity-agent/0.log" limit=1024 Jul 16 01:18:41.834526 ip-10-0-143-200 kubenswrapper[2572]: I0716 01:18:41.834480 2572 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9gnmq_3a7e030d-fad8-480d-9dc6-fe5057048cf9/konnectivity-agent/0.log" limit=1024 Jul 16 01:18:48.477357 ip-10-0-143-200 kubenswrapper[2572]: I0716 01:18:48.477267 2572 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9gnmq_3a7e030d-fad8-480d-9dc6-fe5057048cf9/konnectivity-agent/0.log" limit=1024 Jul 16 01:18:48.586100 ip-10-0-143-200 kubenswrapper[2572]: I0716 01:18:48.586060 2572 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9gnmq_3a7e030d-fad8-480d-9dc6-fe5057048cf9/konnectivity-agent/0.log" limit=1024 Jul 16 01:18:48.682342 ip-10-0-143-200 kubenswrapper[2572]: I0716 01:18:48.682294 2572 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9gnmq_3a7e030d-fad8-480d-9dc6-fe5057048cf9/konnectivity-agent/0.log" limit=1024 Jul 16 01:18:48.743398 ip-10-0-143-200 kubenswrapper[2572]: I0716 01:18:48.743292 2572 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9gnmq_3a7e030d-fad8-480d-9dc6-fe5057048cf9/konnectivity-agent/0.log" limit=1024 Jul 16 01:18:56.267269 ip-10-0-143-200 kubenswrapper[2572]: I0716 01:18:56.267233 2572 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9gnmq_3a7e030d-fad8-480d-9dc6-fe5057048cf9/konnectivity-agent/0.log" limit=1024 Jul 16 01:18:56.364288 ip-10-0-143-200 kubenswrapper[2572]: I0716 01:18:56.364249 2572 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9gnmq_3a7e030d-fad8-480d-9dc6-fe5057048cf9/konnectivity-agent/0.log" limit=1024 Jul 16 01:18:58.383160 ip-10-0-143-200 kubenswrapper[2572]: I0716 01:18:58.383119 2572 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9gnmq_3a7e030d-fad8-480d-9dc6-fe5057048cf9/konnectivity-agent/0.log" limit=1024 Jul 16 01:19:19.284491 ip-10-0-143-200 kubenswrapper[2572]: I0716 01:19:19.284449 2572 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9gnmq_3a7e030d-fad8-480d-9dc6-fe5057048cf9/konnectivity-agent/0.log" limit=1024 Jul 16 01:19:19.384148 ip-10-0-143-200 kubenswrapper[2572]: I0716 01:19:19.384109 2572 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9gnmq_3a7e030d-fad8-480d-9dc6-fe5057048cf9/konnectivity-agent/0.log" limit=1024 Jul 16 01:19:42.994324 ip-10-0-143-200 kubenswrapper[2572]: I0716 01:19:42.994287 2572 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9gnmq_3a7e030d-fad8-480d-9dc6-fe5057048cf9/konnectivity-agent/0.log" limit=1024 Jul 16 01:19:55.563149 ip-10-0-143-200 kubenswrapper[2572]: I0716 01:19:55.563117 2572 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-console-operator_console-operator-7866497494-pjc6j_a64bcd05-8596-4c12-b88e-e974d676e25f/console-operator/2.log" Jul 16 01:19:55.567891 ip-10-0-143-200 kubenswrapper[2572]: I0716 01:19:55.567858 2572 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ovn-kubernetes_ovnkube-node-fp78g_aa66f2b6-0ad8-45d0-8ff5-40fc21966b5a/ovn-acl-logging/0.log" Jul 16 01:19:55.572482 ip-10-0-143-200 kubenswrapper[2572]: I0716 01:19:55.572458 2572 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-console-operator_console-operator-7866497494-pjc6j_a64bcd05-8596-4c12-b88e-e974d676e25f/console-operator/2.log" Jul 16 01:19:55.576900 ip-10-0-143-200 kubenswrapper[2572]: I0716 01:19:55.576878 2572 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ovn-kubernetes_ovnkube-node-fp78g_aa66f2b6-0ad8-45d0-8ff5-40fc21966b5a/ovn-acl-logging/0.log" Jul 16 01:19:57.304340 ip-10-0-143-200 kubenswrapper[2572]: I0716 01:19:57.304299 2572 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9gnmq_3a7e030d-fad8-480d-9dc6-fe5057048cf9/konnectivity-agent/0.log" limit=1024 Jul 16 01:19:58.162291 ip-10-0-143-200 kubenswrapper[2572]: I0716 01:19:58.162262 2572 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9gnmq_3a7e030d-fad8-480d-9dc6-fe5057048cf9/konnectivity-agent/0.log" limit=1024 Jul 16 01:19:59.120711 ip-10-0-143-200 kubenswrapper[2572]: I0716 01:19:59.120675 2572 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["opendatahub/test-curl-7507-139912869367600"] Jul 16 01:19:59.121111 ip-10-0-143-200 kubenswrapper[2572]: I0716 01:19:59.121097 2572 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="bf10ff5f-2652-4637-92c0-65f7672204c3" containerName="cleanup" Jul 16 01:19:59.121164 ip-10-0-143-200 kubenswrapper[2572]: I0716 01:19:59.121112 2572 state_mem.go:107] "Deleted CPUSet assignment" podUID="bf10ff5f-2652-4637-92c0-65f7672204c3" containerName="cleanup" Jul 16 01:19:59.121210 ip-10-0-143-200 kubenswrapper[2572]: I0716 01:19:59.121202 2572 memory_manager.go:368] "RemoveStaleState removing state" podUID="bf10ff5f-2652-4637-92c0-65f7672204c3" containerName="cleanup" Jul 16 01:19:59.124488 ip-10-0-143-200 kubenswrapper[2572]: I0716 01:19:59.124468 2572 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="opendatahub/test-curl-7507-139912869367600" Jul 16 01:19:59.127264 ip-10-0-143-200 kubenswrapper[2572]: I0716 01:19:59.127242 2572 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"opendatahub\"/\"default-dockercfg-d9hgd\"" Jul 16 01:19:59.134448 ip-10-0-143-200 kubenswrapper[2572]: I0716 01:19:59.134419 2572 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["opendatahub/test-curl-7507-139912869367600"] Jul 16 01:19:59.151227 ip-10-0-143-200 kubenswrapper[2572]: I0716 01:19:59.151204 2572 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-gkvfm\" (UniqueName: \"kubernetes.io/projected/0ea41688-c9b4-412e-97e7-3eca5747534d-kube-api-access-gkvfm\") pod \"test-curl-7507-139912869367600\" (UID: \"0ea41688-c9b4-412e-97e7-3eca5747534d\") " pod="opendatahub/test-curl-7507-139912869367600" Jul 16 01:19:59.252065 ip-10-0-143-200 kubenswrapper[2572]: I0716 01:19:59.252027 2572 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-gkvfm\" (UniqueName: \"kubernetes.io/projected/0ea41688-c9b4-412e-97e7-3eca5747534d-kube-api-access-gkvfm\") pod \"test-curl-7507-139912869367600\" (UID: \"0ea41688-c9b4-412e-97e7-3eca5747534d\") " pod="opendatahub/test-curl-7507-139912869367600" Jul 16 01:19:59.261357 ip-10-0-143-200 kubenswrapper[2572]: I0716 01:19:59.261325 2572 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-gkvfm\" (UniqueName: \"kubernetes.io/projected/0ea41688-c9b4-412e-97e7-3eca5747534d-kube-api-access-gkvfm\") pod \"test-curl-7507-139912869367600\" (UID: \"0ea41688-c9b4-412e-97e7-3eca5747534d\") " pod="opendatahub/test-curl-7507-139912869367600" Jul 16 01:19:59.434479 ip-10-0-143-200 kubenswrapper[2572]: I0716 01:19:59.434390 2572 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="opendatahub/test-curl-7507-139912869367600" Jul 16 01:19:59.572243 ip-10-0-143-200 kubenswrapper[2572]: I0716 01:19:59.572218 2572 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["opendatahub/test-curl-7507-139912869367600"] Jul 16 01:19:59.575102 ip-10-0-143-200 kubenswrapper[2572]: W0716 01:19:59.575072 2572 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-besteffort.slice/kubepods-besteffort-pod0ea41688_c9b4_412e_97e7_3eca5747534d.slice/crio-2b7a2926ee98a2e35758bbc35570165cf445c069bbde7744a3489f6b35948e25 WatchSource:0}: Error finding container 2b7a2926ee98a2e35758bbc35570165cf445c069bbde7744a3489f6b35948e25: Status 404 returned error can't find the container with id 2b7a2926ee98a2e35758bbc35570165cf445c069bbde7744a3489f6b35948e25 Jul 16 01:20:00.082084 ip-10-0-143-200 kubenswrapper[2572]: I0716 01:20:00.082038 2572 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="opendatahub/test-curl-7507-139912869367600" event={"ID":"0ea41688-c9b4-412e-97e7-3eca5747534d","Type":"ContainerStarted","Data":"2b7a2926ee98a2e35758bbc35570165cf445c069bbde7744a3489f6b35948e25"} Jul 16 01:20:01.091742 ip-10-0-143-200 kubenswrapper[2572]: I0716 01:20:01.091698 2572 generic.go:356] "Generic (PLEG): container finished" podID="0ea41688-c9b4-412e-97e7-3eca5747534d" containerID="3cdf64d328b91540a478ba426f27b5a95fef1aaf72ca823c4f5368951fa10a33" exitCode=0 Jul 16 01:20:01.092126 ip-10-0-143-200 kubenswrapper[2572]: I0716 01:20:01.091748 2572 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="opendatahub/test-curl-7507-139912869367600" event={"ID":"0ea41688-c9b4-412e-97e7-3eca5747534d","Type":"ContainerDied","Data":"3cdf64d328b91540a478ba426f27b5a95fef1aaf72ca823c4f5368951fa10a33"} Jul 16 01:20:02.246095 ip-10-0-143-200 kubenswrapper[2572]: I0716 01:20:02.246064 2572 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="opendatahub/test-curl-7507-139912869367600" Jul 16 01:20:02.267284 ip-10-0-143-200 kubenswrapper[2572]: I0716 01:20:02.267241 2572 log.go:25] "Finished parsing log file" path="/var/log/pods/opendatahub_test-curl-7507-139912869367600_0ea41688-c9b4-412e-97e7-3eca5747534d/test-curl-7507-139912869367600/0.log" Jul 16 01:20:02.283496 ip-10-0-143-200 kubenswrapper[2572]: I0716 01:20:02.283465 2572 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-gkvfm\" (UniqueName: \"kubernetes.io/projected/0ea41688-c9b4-412e-97e7-3eca5747534d-kube-api-access-gkvfm\") pod \"0ea41688-c9b4-412e-97e7-3eca5747534d\" (UID: \"0ea41688-c9b4-412e-97e7-3eca5747534d\") " Jul 16 01:20:02.285642 ip-10-0-143-200 kubenswrapper[2572]: I0716 01:20:02.285599 2572 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/0ea41688-c9b4-412e-97e7-3eca5747534d-kube-api-access-gkvfm" (OuterVolumeSpecName: "kube-api-access-gkvfm") pod "0ea41688-c9b4-412e-97e7-3eca5747534d" (UID: "0ea41688-c9b4-412e-97e7-3eca5747534d"). InnerVolumeSpecName "kube-api-access-gkvfm". PluginName "kubernetes.io/projected", VolumeGIDValue "" Jul 16 01:20:02.293847 ip-10-0-143-200 kubenswrapper[2572]: I0716 01:20:02.293817 2572 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["opendatahub/test-curl-7507-139912869367600"] Jul 16 01:20:02.298442 ip-10-0-143-200 kubenswrapper[2572]: I0716 01:20:02.298418 2572 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["opendatahub/test-curl-7507-139912869367600"] Jul 16 01:20:02.384964 ip-10-0-143-200 kubenswrapper[2572]: I0716 01:20:02.384866 2572 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-gkvfm\" (UniqueName: \"kubernetes.io/projected/0ea41688-c9b4-412e-97e7-3eca5747534d-kube-api-access-gkvfm\") on node \"ip-10-0-143-200.ec2.internal\" DevicePath \"\"" Jul 16 01:20:03.104650 ip-10-0-143-200 kubenswrapper[2572]: I0716 01:20:03.104593 2572 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="opendatahub/test-curl-7507-139912869367600" Jul 16 01:20:03.104838 ip-10-0-143-200 kubenswrapper[2572]: I0716 01:20:03.104599 2572 pod_container_deletor.go:80] "Container not found in pod's containers" containerID="2b7a2926ee98a2e35758bbc35570165cf445c069bbde7744a3489f6b35948e25" Jul 16 01:20:03.115384 ip-10-0-143-200 kubenswrapper[2572]: E0716 01:20:03.115354 2572 status_manager.go:1018] "Failed to get status for pod" err="pods \"test-curl-7507-139912869367600\" is forbidden: User \"system:node:ip-10-0-143-200.ec2.internal\" cannot get resource \"pods\" in API group \"\" in the namespace \"opendatahub\": no relationship found between node 'ip-10-0-143-200.ec2.internal' and this object" podUID="0ea41688-c9b4-412e-97e7-3eca5747534d" pod="opendatahub/test-curl-7507-139912869367600" Jul 16 01:20:03.449320 ip-10-0-143-200 kubenswrapper[2572]: I0716 01:20:03.449230 2572 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="0ea41688-c9b4-412e-97e7-3eca5747534d" path="/var/lib/kubelet/pods/0ea41688-c9b4-412e-97e7-3eca5747534d/volumes" Jul 16 01:20:17.704605 ip-10-0-143-200 kubenswrapper[2572]: I0716 01:20:17.704508 2572 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9gnmq_3a7e030d-fad8-480d-9dc6-fe5057048cf9/konnectivity-agent/0.log" limit=1024 Jul 16 01:20:23.988243 ip-10-0-143-200 kubenswrapper[2572]: I0716 01:20:23.988201 2572 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9gnmq_3a7e030d-fad8-480d-9dc6-fe5057048cf9/konnectivity-agent/0.log" limit=1024 Jul 16 01:20:24.086314 ip-10-0-143-200 kubenswrapper[2572]: I0716 01:20:24.086277 2572 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9gnmq_3a7e030d-fad8-480d-9dc6-fe5057048cf9/konnectivity-agent/0.log" limit=1024 Jul 16 01:20:24.341184 ip-10-0-143-200 kubenswrapper[2572]: I0716 01:20:24.341092 2572 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9gnmq_3a7e030d-fad8-480d-9dc6-fe5057048cf9/konnectivity-agent/0.log" limit=1024 Jul 16 01:20:24.390028 ip-10-0-143-200 kubenswrapper[2572]: I0716 01:20:24.389994 2572 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-ingress/e2e-shared-a-f9f50d-openshift-default-5d5b68dcd7-xl2x6"] Jul 16 01:20:24.390223 ip-10-0-143-200 kubenswrapper[2572]: I0716 01:20:24.390198 2572 kuberuntime_container.go:873] "Killing container with a grace period" pod="openshift-ingress/e2e-shared-a-f9f50d-openshift-default-5d5b68dcd7-xl2x6" podUID="c3a9a894-3f1a-4746-a760-831441b8ac8b" containerName="istio-proxy" containerID="cri-o://67ec4749753860e169712b2bdacf62af10fdc37c593739009128208995f2e186" gracePeriod=30 Jul 16 01:20:24.595306 ip-10-0-143-200 kubenswrapper[2572]: I0716 01:20:24.595209 2572 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9gnmq_3a7e030d-fad8-480d-9dc6-fe5057048cf9/konnectivity-agent/0.log" limit=1024 Jul 16 01:20:25.002465 ip-10-0-143-200 kubenswrapper[2572]: I0716 01:20:25.002428 2572 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9gnmq_3a7e030d-fad8-480d-9dc6-fe5057048cf9/konnectivity-agent/0.log" limit=1024 Jul 16 01:20:28.727496 ip-10-0-143-200 kubenswrapper[2572]: I0716 01:20:28.727456 2572 patch_prober.go:28] interesting pod/e2e-shared-a-f9f50d-openshift-default-5d5b68dcd7-xl2x6 container/istio-proxy namespace/openshift-ingress: Readiness probe status=failure output="HTTP probe failed with statuscode: 503" start-of-body= Jul 16 01:20:28.727905 ip-10-0-143-200 kubenswrapper[2572]: I0716 01:20:28.727515 2572 prober.go:120] "Probe failed" probeType="Readiness" pod="openshift-ingress/e2e-shared-a-f9f50d-openshift-default-5d5b68dcd7-xl2x6" podUID="c3a9a894-3f1a-4746-a760-831441b8ac8b" containerName="istio-proxy" probeResult="failure" output="HTTP probe failed with statuscode: 503" Jul 16 01:20:29.658971 ip-10-0-143-200 kubenswrapper[2572]: I0716 01:20:29.658903 2572 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-shared-a-f9f50d-openshift-default-5d5b68dcd7-xl2x6" Jul 16 01:20:29.745239 ip-10-0-143-200 kubenswrapper[2572]: I0716 01:20:29.745205 2572 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/c3a9a894-3f1a-4746-a760-831441b8ac8b-istio-data\") pod \"c3a9a894-3f1a-4746-a760-831441b8ac8b\" (UID: \"c3a9a894-3f1a-4746-a760-831441b8ac8b\") " Jul 16 01:20:29.745705 ip-10-0-143-200 kubenswrapper[2572]: I0716 01:20:29.745250 2572 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/c3a9a894-3f1a-4746-a760-831441b8ac8b-istio-podinfo\") pod \"c3a9a894-3f1a-4746-a760-831441b8ac8b\" (UID: \"c3a9a894-3f1a-4746-a760-831441b8ac8b\") " Jul 16 01:20:29.745705 ip-10-0-143-200 kubenswrapper[2572]: I0716 01:20:29.745294 2572 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/c3a9a894-3f1a-4746-a760-831441b8ac8b-workload-certs\") pod \"c3a9a894-3f1a-4746-a760-831441b8ac8b\" (UID: \"c3a9a894-3f1a-4746-a760-831441b8ac8b\") " Jul 16 01:20:29.745705 ip-10-0-143-200 kubenswrapper[2572]: I0716 01:20:29.745324 2572 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/c3a9a894-3f1a-4746-a760-831441b8ac8b-credential-socket\") pod \"c3a9a894-3f1a-4746-a760-831441b8ac8b\" (UID: \"c3a9a894-3f1a-4746-a760-831441b8ac8b\") " Jul 16 01:20:29.745705 ip-10-0-143-200 kubenswrapper[2572]: I0716 01:20:29.745355 2572 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/c3a9a894-3f1a-4746-a760-831441b8ac8b-istiod-ca-cert\") pod \"c3a9a894-3f1a-4746-a760-831441b8ac8b\" (UID: \"c3a9a894-3f1a-4746-a760-831441b8ac8b\") " Jul 16 01:20:29.745705 ip-10-0-143-200 kubenswrapper[2572]: I0716 01:20:29.745413 2572 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/c3a9a894-3f1a-4746-a760-831441b8ac8b-workload-socket\") pod \"c3a9a894-3f1a-4746-a760-831441b8ac8b\" (UID: \"c3a9a894-3f1a-4746-a760-831441b8ac8b\") " Jul 16 01:20:29.745705 ip-10-0-143-200 kubenswrapper[2572]: I0716 01:20:29.745469 2572 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-6hpxr\" (UniqueName: \"kubernetes.io/projected/c3a9a894-3f1a-4746-a760-831441b8ac8b-kube-api-access-6hpxr\") pod \"c3a9a894-3f1a-4746-a760-831441b8ac8b\" (UID: \"c3a9a894-3f1a-4746-a760-831441b8ac8b\") " Jul 16 01:20:29.745705 ip-10-0-143-200 kubenswrapper[2572]: I0716 01:20:29.745569 2572 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/c3a9a894-3f1a-4746-a760-831441b8ac8b-istio-token\") pod \"c3a9a894-3f1a-4746-a760-831441b8ac8b\" (UID: \"c3a9a894-3f1a-4746-a760-831441b8ac8b\") " Jul 16 01:20:29.745705 ip-10-0-143-200 kubenswrapper[2572]: I0716 01:20:29.745598 2572 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/c3a9a894-3f1a-4746-a760-831441b8ac8b-istio-envoy\") pod \"c3a9a894-3f1a-4746-a760-831441b8ac8b\" (UID: \"c3a9a894-3f1a-4746-a760-831441b8ac8b\") " Jul 16 01:20:29.746108 ip-10-0-143-200 kubenswrapper[2572]: I0716 01:20:29.745719 2572 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/c3a9a894-3f1a-4746-a760-831441b8ac8b-workload-certs" (OuterVolumeSpecName: "workload-certs") pod "c3a9a894-3f1a-4746-a760-831441b8ac8b" (UID: "c3a9a894-3f1a-4746-a760-831441b8ac8b"). InnerVolumeSpecName "workload-certs". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Jul 16 01:20:29.746108 ip-10-0-143-200 kubenswrapper[2572]: I0716 01:20:29.745796 2572 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/c3a9a894-3f1a-4746-a760-831441b8ac8b-workload-socket" (OuterVolumeSpecName: "workload-socket") pod "c3a9a894-3f1a-4746-a760-831441b8ac8b" (UID: "c3a9a894-3f1a-4746-a760-831441b8ac8b"). InnerVolumeSpecName "workload-socket". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Jul 16 01:20:29.746108 ip-10-0-143-200 kubenswrapper[2572]: I0716 01:20:29.745982 2572 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/c3a9a894-3f1a-4746-a760-831441b8ac8b-istiod-ca-cert" (OuterVolumeSpecName: "istiod-ca-cert") pod "c3a9a894-3f1a-4746-a760-831441b8ac8b" (UID: "c3a9a894-3f1a-4746-a760-831441b8ac8b"). InnerVolumeSpecName "istiod-ca-cert". PluginName "kubernetes.io/configmap", VolumeGIDValue "" Jul 16 01:20:29.746318 ip-10-0-143-200 kubenswrapper[2572]: I0716 01:20:29.745806 2572 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/c3a9a894-3f1a-4746-a760-831441b8ac8b-credential-socket" (OuterVolumeSpecName: "credential-socket") pod "c3a9a894-3f1a-4746-a760-831441b8ac8b" (UID: "c3a9a894-3f1a-4746-a760-831441b8ac8b"). InnerVolumeSpecName "credential-socket". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Jul 16 01:20:29.746318 ip-10-0-143-200 kubenswrapper[2572]: I0716 01:20:29.746265 2572 reconciler_common.go:299] "Volume detached for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/c3a9a894-3f1a-4746-a760-831441b8ac8b-workload-certs\") on node \"ip-10-0-143-200.ec2.internal\" DevicePath \"\"" Jul 16 01:20:29.746318 ip-10-0-143-200 kubenswrapper[2572]: I0716 01:20:29.746271 2572 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/c3a9a894-3f1a-4746-a760-831441b8ac8b-istio-data" (OuterVolumeSpecName: "istio-data") pod "c3a9a894-3f1a-4746-a760-831441b8ac8b" (UID: "c3a9a894-3f1a-4746-a760-831441b8ac8b"). InnerVolumeSpecName "istio-data". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Jul 16 01:20:29.746318 ip-10-0-143-200 kubenswrapper[2572]: I0716 01:20:29.746286 2572 reconciler_common.go:299] "Volume detached for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/c3a9a894-3f1a-4746-a760-831441b8ac8b-credential-socket\") on node \"ip-10-0-143-200.ec2.internal\" DevicePath \"\"" Jul 16 01:20:29.746318 ip-10-0-143-200 kubenswrapper[2572]: I0716 01:20:29.746301 2572 reconciler_common.go:299] "Volume detached for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/c3a9a894-3f1a-4746-a760-831441b8ac8b-istiod-ca-cert\") on node \"ip-10-0-143-200.ec2.internal\" DevicePath \"\"" Jul 16 01:20:29.746318 ip-10-0-143-200 kubenswrapper[2572]: I0716 01:20:29.746315 2572 reconciler_common.go:299] "Volume detached for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/c3a9a894-3f1a-4746-a760-831441b8ac8b-workload-socket\") on node \"ip-10-0-143-200.ec2.internal\" DevicePath \"\"" Jul 16 01:20:29.747673 ip-10-0-143-200 kubenswrapper[2572]: I0716 01:20:29.747640 2572 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/downward-api/c3a9a894-3f1a-4746-a760-831441b8ac8b-istio-podinfo" (OuterVolumeSpecName: "istio-podinfo") pod "c3a9a894-3f1a-4746-a760-831441b8ac8b" (UID: "c3a9a894-3f1a-4746-a760-831441b8ac8b"). InnerVolumeSpecName "istio-podinfo". PluginName "kubernetes.io/downward-api", VolumeGIDValue "" Jul 16 01:20:29.747795 ip-10-0-143-200 kubenswrapper[2572]: I0716 01:20:29.747753 2572 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/c3a9a894-3f1a-4746-a760-831441b8ac8b-kube-api-access-6hpxr" (OuterVolumeSpecName: "kube-api-access-6hpxr") pod "c3a9a894-3f1a-4746-a760-831441b8ac8b" (UID: "c3a9a894-3f1a-4746-a760-831441b8ac8b"). InnerVolumeSpecName "kube-api-access-6hpxr". PluginName "kubernetes.io/projected", VolumeGIDValue "" Jul 16 01:20:29.747846 ip-10-0-143-200 kubenswrapper[2572]: I0716 01:20:29.747807 2572 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/c3a9a894-3f1a-4746-a760-831441b8ac8b-istio-envoy" (OuterVolumeSpecName: "istio-envoy") pod "c3a9a894-3f1a-4746-a760-831441b8ac8b" (UID: "c3a9a894-3f1a-4746-a760-831441b8ac8b"). InnerVolumeSpecName "istio-envoy". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Jul 16 01:20:29.747945 ip-10-0-143-200 kubenswrapper[2572]: I0716 01:20:29.747928 2572 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/c3a9a894-3f1a-4746-a760-831441b8ac8b-istio-token" (OuterVolumeSpecName: "istio-token") pod "c3a9a894-3f1a-4746-a760-831441b8ac8b" (UID: "c3a9a894-3f1a-4746-a760-831441b8ac8b"). InnerVolumeSpecName "istio-token". PluginName "kubernetes.io/projected", VolumeGIDValue "" Jul 16 01:20:29.847106 ip-10-0-143-200 kubenswrapper[2572]: I0716 01:20:29.847068 2572 reconciler_common.go:299] "Volume detached for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/c3a9a894-3f1a-4746-a760-831441b8ac8b-istio-envoy\") on node \"ip-10-0-143-200.ec2.internal\" DevicePath \"\"" Jul 16 01:20:29.847106 ip-10-0-143-200 kubenswrapper[2572]: I0716 01:20:29.847101 2572 reconciler_common.go:299] "Volume detached for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/c3a9a894-3f1a-4746-a760-831441b8ac8b-istio-data\") on node \"ip-10-0-143-200.ec2.internal\" DevicePath \"\"" Jul 16 01:20:29.847106 ip-10-0-143-200 kubenswrapper[2572]: I0716 01:20:29.847111 2572 reconciler_common.go:299] "Volume detached for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/c3a9a894-3f1a-4746-a760-831441b8ac8b-istio-podinfo\") on node \"ip-10-0-143-200.ec2.internal\" DevicePath \"\"" Jul 16 01:20:29.847393 ip-10-0-143-200 kubenswrapper[2572]: I0716 01:20:29.847120 2572 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-6hpxr\" (UniqueName: \"kubernetes.io/projected/c3a9a894-3f1a-4746-a760-831441b8ac8b-kube-api-access-6hpxr\") on node \"ip-10-0-143-200.ec2.internal\" DevicePath \"\"" Jul 16 01:20:29.847393 ip-10-0-143-200 kubenswrapper[2572]: I0716 01:20:29.847130 2572 reconciler_common.go:299] "Volume detached for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/c3a9a894-3f1a-4746-a760-831441b8ac8b-istio-token\") on node \"ip-10-0-143-200.ec2.internal\" DevicePath \"\"" Jul 16 01:20:30.254465 ip-10-0-143-200 kubenswrapper[2572]: I0716 01:20:30.254429 2572 generic.go:356] "Generic (PLEG): container finished" podID="c3a9a894-3f1a-4746-a760-831441b8ac8b" containerID="67ec4749753860e169712b2bdacf62af10fdc37c593739009128208995f2e186" exitCode=0 Jul 16 01:20:30.254689 ip-10-0-143-200 kubenswrapper[2572]: I0716 01:20:30.254472 2572 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-shared-a-f9f50d-openshift-default-5d5b68dcd7-xl2x6" event={"ID":"c3a9a894-3f1a-4746-a760-831441b8ac8b","Type":"ContainerDied","Data":"67ec4749753860e169712b2bdacf62af10fdc37c593739009128208995f2e186"} Jul 16 01:20:30.254689 ip-10-0-143-200 kubenswrapper[2572]: I0716 01:20:30.254536 2572 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-shared-a-f9f50d-openshift-default-5d5b68dcd7-xl2x6" event={"ID":"c3a9a894-3f1a-4746-a760-831441b8ac8b","Type":"ContainerDied","Data":"6b7150a8fefa5f338aa474b24424770e5656a4202edfa289a1bb565845963b79"} Jul 16 01:20:30.254689 ip-10-0-143-200 kubenswrapper[2572]: I0716 01:20:30.254484 2572 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-shared-a-f9f50d-openshift-default-5d5b68dcd7-xl2x6" Jul 16 01:20:30.254689 ip-10-0-143-200 kubenswrapper[2572]: I0716 01:20:30.254568 2572 scope.go:117] "RemoveContainer" containerID="67ec4749753860e169712b2bdacf62af10fdc37c593739009128208995f2e186" Jul 16 01:20:30.264323 ip-10-0-143-200 kubenswrapper[2572]: I0716 01:20:30.264291 2572 scope.go:117] "RemoveContainer" containerID="eaac463958ff9bae0954276dddf7bb360f38fde51746934c5d22c8b9f804ffc5" Jul 16 01:20:30.273276 ip-10-0-143-200 kubenswrapper[2572]: I0716 01:20:30.273260 2572 scope.go:117] "RemoveContainer" containerID="67ec4749753860e169712b2bdacf62af10fdc37c593739009128208995f2e186" Jul 16 01:20:30.273565 ip-10-0-143-200 kubenswrapper[2572]: E0716 01:20:30.273539 2572 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"67ec4749753860e169712b2bdacf62af10fdc37c593739009128208995f2e186\": container with ID starting with 67ec4749753860e169712b2bdacf62af10fdc37c593739009128208995f2e186 not found: ID does not exist" containerID="67ec4749753860e169712b2bdacf62af10fdc37c593739009128208995f2e186" Jul 16 01:20:30.273690 ip-10-0-143-200 kubenswrapper[2572]: I0716 01:20:30.273570 2572 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"67ec4749753860e169712b2bdacf62af10fdc37c593739009128208995f2e186"} err="failed to get container status \"67ec4749753860e169712b2bdacf62af10fdc37c593739009128208995f2e186\": rpc error: code = NotFound desc = could not find container \"67ec4749753860e169712b2bdacf62af10fdc37c593739009128208995f2e186\": container with ID starting with 67ec4749753860e169712b2bdacf62af10fdc37c593739009128208995f2e186 not found: ID does not exist" Jul 16 01:20:30.273690 ip-10-0-143-200 kubenswrapper[2572]: I0716 01:20:30.273586 2572 scope.go:117] "RemoveContainer" containerID="eaac463958ff9bae0954276dddf7bb360f38fde51746934c5d22c8b9f804ffc5" Jul 16 01:20:30.273857 ip-10-0-143-200 kubenswrapper[2572]: E0716 01:20:30.273839 2572 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"eaac463958ff9bae0954276dddf7bb360f38fde51746934c5d22c8b9f804ffc5\": container with ID starting with eaac463958ff9bae0954276dddf7bb360f38fde51746934c5d22c8b9f804ffc5 not found: ID does not exist" containerID="eaac463958ff9bae0954276dddf7bb360f38fde51746934c5d22c8b9f804ffc5" Jul 16 01:20:30.273905 ip-10-0-143-200 kubenswrapper[2572]: I0716 01:20:30.273864 2572 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"eaac463958ff9bae0954276dddf7bb360f38fde51746934c5d22c8b9f804ffc5"} err="failed to get container status \"eaac463958ff9bae0954276dddf7bb360f38fde51746934c5d22c8b9f804ffc5\": rpc error: code = NotFound desc = could not find container \"eaac463958ff9bae0954276dddf7bb360f38fde51746934c5d22c8b9f804ffc5\": container with ID starting with eaac463958ff9bae0954276dddf7bb360f38fde51746934c5d22c8b9f804ffc5 not found: ID does not exist" Jul 16 01:20:30.280375 ip-10-0-143-200 kubenswrapper[2572]: I0716 01:20:30.280351 2572 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-ingress/e2e-shared-a-f9f50d-openshift-default-5d5b68dcd7-xl2x6"] Jul 16 01:20:30.285403 ip-10-0-143-200 kubenswrapper[2572]: I0716 01:20:30.285381 2572 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["openshift-ingress/e2e-shared-a-f9f50d-openshift-default-5d5b68dcd7-xl2x6"] Jul 16 01:20:31.294237 ip-10-0-143-200 kubenswrapper[2572]: I0716 01:20:31.294199 2572 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9gnmq_3a7e030d-fad8-480d-9dc6-fe5057048cf9/konnectivity-agent/0.log" limit=1024 Jul 16 01:20:31.399231 ip-10-0-143-200 kubenswrapper[2572]: I0716 01:20:31.399194 2572 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9gnmq_3a7e030d-fad8-480d-9dc6-fe5057048cf9/konnectivity-agent/0.log" limit=1024 Jul 16 01:20:31.448360 ip-10-0-143-200 kubenswrapper[2572]: I0716 01:20:31.448318 2572 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="c3a9a894-3f1a-4746-a760-831441b8ac8b" path="/var/lib/kubelet/pods/c3a9a894-3f1a-4746-a760-831441b8ac8b/volumes" Jul 16 01:20:31.540961 ip-10-0-143-200 kubenswrapper[2572]: I0716 01:20:31.540927 2572 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-ingress/e2e-shared-b-628c71-openshift-default-5d446f7cff-sp9c9"] Jul 16 01:20:31.541201 ip-10-0-143-200 kubenswrapper[2572]: I0716 01:20:31.541172 2572 kuberuntime_container.go:873] "Killing container with a grace period" pod="openshift-ingress/e2e-shared-b-628c71-openshift-default-5d446f7cff-sp9c9" podUID="4a730754-3193-49df-a072-f019e5b92dc2" containerName="istio-proxy" containerID="cri-o://ee46d87c7e2514b7e86b33dd2a999f6a9292010ad113a5174124c7df1bf0c8c4" gracePeriod=30 Jul 16 01:20:31.553045 ip-10-0-143-200 kubenswrapper[2572]: I0716 01:20:31.552976 2572 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9gnmq_3a7e030d-fad8-480d-9dc6-fe5057048cf9/konnectivity-agent/0.log" limit=1024 Jul 16 01:20:31.814514 ip-10-0-143-200 kubenswrapper[2572]: I0716 01:20:31.814425 2572 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9gnmq_3a7e030d-fad8-480d-9dc6-fe5057048cf9/konnectivity-agent/0.log" limit=1024 Jul 16 01:20:36.795697 ip-10-0-143-200 kubenswrapper[2572]: I0716 01:20:36.795671 2572 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-shared-b-628c71-openshift-default-5d446f7cff-sp9c9" Jul 16 01:20:36.818949 ip-10-0-143-200 kubenswrapper[2572]: I0716 01:20:36.818858 2572 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/4a730754-3193-49df-a072-f019e5b92dc2-istio-token\") pod \"4a730754-3193-49df-a072-f019e5b92dc2\" (UID: \"4a730754-3193-49df-a072-f019e5b92dc2\") " Jul 16 01:20:36.818949 ip-10-0-143-200 kubenswrapper[2572]: I0716 01:20:36.818907 2572 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/4a730754-3193-49df-a072-f019e5b92dc2-credential-socket\") pod \"4a730754-3193-49df-a072-f019e5b92dc2\" (UID: \"4a730754-3193-49df-a072-f019e5b92dc2\") " Jul 16 01:20:36.818949 ip-10-0-143-200 kubenswrapper[2572]: I0716 01:20:36.818945 2572 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-z9g8w\" (UniqueName: \"kubernetes.io/projected/4a730754-3193-49df-a072-f019e5b92dc2-kube-api-access-z9g8w\") pod \"4a730754-3193-49df-a072-f019e5b92dc2\" (UID: \"4a730754-3193-49df-a072-f019e5b92dc2\") " Jul 16 01:20:36.819218 ip-10-0-143-200 kubenswrapper[2572]: I0716 01:20:36.818980 2572 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/4a730754-3193-49df-a072-f019e5b92dc2-istio-envoy\") pod \"4a730754-3193-49df-a072-f019e5b92dc2\" (UID: \"4a730754-3193-49df-a072-f019e5b92dc2\") " Jul 16 01:20:36.819218 ip-10-0-143-200 kubenswrapper[2572]: I0716 01:20:36.819028 2572 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/4a730754-3193-49df-a072-f019e5b92dc2-workload-certs\") pod \"4a730754-3193-49df-a072-f019e5b92dc2\" (UID: \"4a730754-3193-49df-a072-f019e5b92dc2\") " Jul 16 01:20:36.819218 ip-10-0-143-200 kubenswrapper[2572]: I0716 01:20:36.819063 2572 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/4a730754-3193-49df-a072-f019e5b92dc2-istio-data\") pod \"4a730754-3193-49df-a072-f019e5b92dc2\" (UID: \"4a730754-3193-49df-a072-f019e5b92dc2\") " Jul 16 01:20:36.819218 ip-10-0-143-200 kubenswrapper[2572]: I0716 01:20:36.819124 2572 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/4a730754-3193-49df-a072-f019e5b92dc2-istio-podinfo\") pod \"4a730754-3193-49df-a072-f019e5b92dc2\" (UID: \"4a730754-3193-49df-a072-f019e5b92dc2\") " Jul 16 01:20:36.819218 ip-10-0-143-200 kubenswrapper[2572]: I0716 01:20:36.819160 2572 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/4a730754-3193-49df-a072-f019e5b92dc2-workload-socket\") pod \"4a730754-3193-49df-a072-f019e5b92dc2\" (UID: \"4a730754-3193-49df-a072-f019e5b92dc2\") " Jul 16 01:20:36.819459 ip-10-0-143-200 kubenswrapper[2572]: I0716 01:20:36.819226 2572 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/4a730754-3193-49df-a072-f019e5b92dc2-istiod-ca-cert\") pod \"4a730754-3193-49df-a072-f019e5b92dc2\" (UID: \"4a730754-3193-49df-a072-f019e5b92dc2\") " Jul 16 01:20:36.819893 ip-10-0-143-200 kubenswrapper[2572]: I0716 01:20:36.819861 2572 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/4a730754-3193-49df-a072-f019e5b92dc2-credential-socket" (OuterVolumeSpecName: "credential-socket") pod "4a730754-3193-49df-a072-f019e5b92dc2" (UID: "4a730754-3193-49df-a072-f019e5b92dc2"). InnerVolumeSpecName "credential-socket". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Jul 16 01:20:36.820094 ip-10-0-143-200 kubenswrapper[2572]: I0716 01:20:36.819880 2572 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/4a730754-3193-49df-a072-f019e5b92dc2-istiod-ca-cert" (OuterVolumeSpecName: "istiod-ca-cert") pod "4a730754-3193-49df-a072-f019e5b92dc2" (UID: "4a730754-3193-49df-a072-f019e5b92dc2"). InnerVolumeSpecName "istiod-ca-cert". PluginName "kubernetes.io/configmap", VolumeGIDValue "" Jul 16 01:20:36.820226 ip-10-0-143-200 kubenswrapper[2572]: I0716 01:20:36.820055 2572 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/4a730754-3193-49df-a072-f019e5b92dc2-workload-certs" (OuterVolumeSpecName: "workload-certs") pod "4a730754-3193-49df-a072-f019e5b92dc2" (UID: "4a730754-3193-49df-a072-f019e5b92dc2"). InnerVolumeSpecName "workload-certs". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Jul 16 01:20:36.820364 ip-10-0-143-200 kubenswrapper[2572]: I0716 01:20:36.820130 2572 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/4a730754-3193-49df-a072-f019e5b92dc2-workload-socket" (OuterVolumeSpecName: "workload-socket") pod "4a730754-3193-49df-a072-f019e5b92dc2" (UID: "4a730754-3193-49df-a072-f019e5b92dc2"). InnerVolumeSpecName "workload-socket". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Jul 16 01:20:36.820465 ip-10-0-143-200 kubenswrapper[2572]: I0716 01:20:36.820242 2572 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/4a730754-3193-49df-a072-f019e5b92dc2-istio-data" (OuterVolumeSpecName: "istio-data") pod "4a730754-3193-49df-a072-f019e5b92dc2" (UID: "4a730754-3193-49df-a072-f019e5b92dc2"). InnerVolumeSpecName "istio-data". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Jul 16 01:20:36.822387 ip-10-0-143-200 kubenswrapper[2572]: I0716 01:20:36.822357 2572 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/4a730754-3193-49df-a072-f019e5b92dc2-istio-token" (OuterVolumeSpecName: "istio-token") pod "4a730754-3193-49df-a072-f019e5b92dc2" (UID: "4a730754-3193-49df-a072-f019e5b92dc2"). InnerVolumeSpecName "istio-token". PluginName "kubernetes.io/projected", VolumeGIDValue "" Jul 16 01:20:36.822518 ip-10-0-143-200 kubenswrapper[2572]: I0716 01:20:36.822441 2572 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/downward-api/4a730754-3193-49df-a072-f019e5b92dc2-istio-podinfo" (OuterVolumeSpecName: "istio-podinfo") pod "4a730754-3193-49df-a072-f019e5b92dc2" (UID: "4a730754-3193-49df-a072-f019e5b92dc2"). InnerVolumeSpecName "istio-podinfo". PluginName "kubernetes.io/downward-api", VolumeGIDValue "" Jul 16 01:20:36.822518 ip-10-0-143-200 kubenswrapper[2572]: I0716 01:20:36.822508 2572 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/4a730754-3193-49df-a072-f019e5b92dc2-kube-api-access-z9g8w" (OuterVolumeSpecName: "kube-api-access-z9g8w") pod "4a730754-3193-49df-a072-f019e5b92dc2" (UID: "4a730754-3193-49df-a072-f019e5b92dc2"). InnerVolumeSpecName "kube-api-access-z9g8w". PluginName "kubernetes.io/projected", VolumeGIDValue "" Jul 16 01:20:36.822872 ip-10-0-143-200 kubenswrapper[2572]: I0716 01:20:36.822849 2572 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/4a730754-3193-49df-a072-f019e5b92dc2-istio-envoy" (OuterVolumeSpecName: "istio-envoy") pod "4a730754-3193-49df-a072-f019e5b92dc2" (UID: "4a730754-3193-49df-a072-f019e5b92dc2"). InnerVolumeSpecName "istio-envoy". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Jul 16 01:20:36.920821 ip-10-0-143-200 kubenswrapper[2572]: I0716 01:20:36.920786 2572 reconciler_common.go:299] "Volume detached for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/4a730754-3193-49df-a072-f019e5b92dc2-istiod-ca-cert\") on node \"ip-10-0-143-200.ec2.internal\" DevicePath \"\"" Jul 16 01:20:36.920821 ip-10-0-143-200 kubenswrapper[2572]: I0716 01:20:36.920817 2572 reconciler_common.go:299] "Volume detached for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/4a730754-3193-49df-a072-f019e5b92dc2-istio-token\") on node \"ip-10-0-143-200.ec2.internal\" DevicePath \"\"" Jul 16 01:20:36.920821 ip-10-0-143-200 kubenswrapper[2572]: I0716 01:20:36.920826 2572 reconciler_common.go:299] "Volume detached for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/4a730754-3193-49df-a072-f019e5b92dc2-credential-socket\") on node \"ip-10-0-143-200.ec2.internal\" DevicePath \"\"" Jul 16 01:20:36.920821 ip-10-0-143-200 kubenswrapper[2572]: I0716 01:20:36.920836 2572 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-z9g8w\" (UniqueName: \"kubernetes.io/projected/4a730754-3193-49df-a072-f019e5b92dc2-kube-api-access-z9g8w\") on node \"ip-10-0-143-200.ec2.internal\" DevicePath \"\"" Jul 16 01:20:36.921103 ip-10-0-143-200 kubenswrapper[2572]: I0716 01:20:36.920845 2572 reconciler_common.go:299] "Volume detached for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/4a730754-3193-49df-a072-f019e5b92dc2-istio-envoy\") on node \"ip-10-0-143-200.ec2.internal\" DevicePath \"\"" Jul 16 01:20:36.921103 ip-10-0-143-200 kubenswrapper[2572]: I0716 01:20:36.920854 2572 reconciler_common.go:299] "Volume detached for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/4a730754-3193-49df-a072-f019e5b92dc2-workload-certs\") on node \"ip-10-0-143-200.ec2.internal\" DevicePath \"\"" Jul 16 01:20:36.921103 ip-10-0-143-200 kubenswrapper[2572]: I0716 01:20:36.920861 2572 reconciler_common.go:299] "Volume detached for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/4a730754-3193-49df-a072-f019e5b92dc2-istio-data\") on node \"ip-10-0-143-200.ec2.internal\" DevicePath \"\"" Jul 16 01:20:36.921103 ip-10-0-143-200 kubenswrapper[2572]: I0716 01:20:36.920868 2572 reconciler_common.go:299] "Volume detached for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/4a730754-3193-49df-a072-f019e5b92dc2-istio-podinfo\") on node \"ip-10-0-143-200.ec2.internal\" DevicePath \"\"" Jul 16 01:20:36.921103 ip-10-0-143-200 kubenswrapper[2572]: I0716 01:20:36.920875 2572 reconciler_common.go:299] "Volume detached for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/4a730754-3193-49df-a072-f019e5b92dc2-workload-socket\") on node \"ip-10-0-143-200.ec2.internal\" DevicePath \"\"" Jul 16 01:20:37.300697 ip-10-0-143-200 kubenswrapper[2572]: I0716 01:20:37.300665 2572 generic.go:356] "Generic (PLEG): container finished" podID="4a730754-3193-49df-a072-f019e5b92dc2" containerID="ee46d87c7e2514b7e86b33dd2a999f6a9292010ad113a5174124c7df1bf0c8c4" exitCode=0 Jul 16 01:20:37.300888 ip-10-0-143-200 kubenswrapper[2572]: I0716 01:20:37.300721 2572 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-shared-b-628c71-openshift-default-5d446f7cff-sp9c9" Jul 16 01:20:37.300888 ip-10-0-143-200 kubenswrapper[2572]: I0716 01:20:37.300720 2572 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-shared-b-628c71-openshift-default-5d446f7cff-sp9c9" event={"ID":"4a730754-3193-49df-a072-f019e5b92dc2","Type":"ContainerDied","Data":"ee46d87c7e2514b7e86b33dd2a999f6a9292010ad113a5174124c7df1bf0c8c4"} Jul 16 01:20:37.300888 ip-10-0-143-200 kubenswrapper[2572]: I0716 01:20:37.300808 2572 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-shared-b-628c71-openshift-default-5d446f7cff-sp9c9" event={"ID":"4a730754-3193-49df-a072-f019e5b92dc2","Type":"ContainerDied","Data":"be267733117155f4ae07c4f8beef2e9c0d93b08bbaf6ecf63d2f2bff470295e4"} Jul 16 01:20:37.300888 ip-10-0-143-200 kubenswrapper[2572]: I0716 01:20:37.300831 2572 scope.go:117] "RemoveContainer" containerID="ee46d87c7e2514b7e86b33dd2a999f6a9292010ad113a5174124c7df1bf0c8c4" Jul 16 01:20:37.311582 ip-10-0-143-200 kubenswrapper[2572]: I0716 01:20:37.311564 2572 scope.go:117] "RemoveContainer" containerID="cd76511ee253b7f7162dbcb9786e80feb4de1619614c4dfcb7dc9fa5206a46c6" Jul 16 01:20:37.319878 ip-10-0-143-200 kubenswrapper[2572]: I0716 01:20:37.319862 2572 scope.go:117] "RemoveContainer" containerID="ee46d87c7e2514b7e86b33dd2a999f6a9292010ad113a5174124c7df1bf0c8c4" Jul 16 01:20:37.320118 ip-10-0-143-200 kubenswrapper[2572]: E0716 01:20:37.320102 2572 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"ee46d87c7e2514b7e86b33dd2a999f6a9292010ad113a5174124c7df1bf0c8c4\": container with ID starting with ee46d87c7e2514b7e86b33dd2a999f6a9292010ad113a5174124c7df1bf0c8c4 not found: ID does not exist" containerID="ee46d87c7e2514b7e86b33dd2a999f6a9292010ad113a5174124c7df1bf0c8c4" Jul 16 01:20:37.320165 ip-10-0-143-200 kubenswrapper[2572]: I0716 01:20:37.320125 2572 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"ee46d87c7e2514b7e86b33dd2a999f6a9292010ad113a5174124c7df1bf0c8c4"} err="failed to get container status \"ee46d87c7e2514b7e86b33dd2a999f6a9292010ad113a5174124c7df1bf0c8c4\": rpc error: code = NotFound desc = could not find container \"ee46d87c7e2514b7e86b33dd2a999f6a9292010ad113a5174124c7df1bf0c8c4\": container with ID starting with ee46d87c7e2514b7e86b33dd2a999f6a9292010ad113a5174124c7df1bf0c8c4 not found: ID does not exist" Jul 16 01:20:37.320165 ip-10-0-143-200 kubenswrapper[2572]: I0716 01:20:37.320139 2572 scope.go:117] "RemoveContainer" containerID="cd76511ee253b7f7162dbcb9786e80feb4de1619614c4dfcb7dc9fa5206a46c6" Jul 16 01:20:37.320372 ip-10-0-143-200 kubenswrapper[2572]: E0716 01:20:37.320354 2572 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"cd76511ee253b7f7162dbcb9786e80feb4de1619614c4dfcb7dc9fa5206a46c6\": container with ID starting with cd76511ee253b7f7162dbcb9786e80feb4de1619614c4dfcb7dc9fa5206a46c6 not found: ID does not exist" containerID="cd76511ee253b7f7162dbcb9786e80feb4de1619614c4dfcb7dc9fa5206a46c6" Jul 16 01:20:37.320411 ip-10-0-143-200 kubenswrapper[2572]: I0716 01:20:37.320378 2572 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"cd76511ee253b7f7162dbcb9786e80feb4de1619614c4dfcb7dc9fa5206a46c6"} err="failed to get container status \"cd76511ee253b7f7162dbcb9786e80feb4de1619614c4dfcb7dc9fa5206a46c6\": rpc error: code = NotFound desc = could not find container \"cd76511ee253b7f7162dbcb9786e80feb4de1619614c4dfcb7dc9fa5206a46c6\": container with ID starting with cd76511ee253b7f7162dbcb9786e80feb4de1619614c4dfcb7dc9fa5206a46c6 not found: ID does not exist" Jul 16 01:20:37.325457 ip-10-0-143-200 kubenswrapper[2572]: I0716 01:20:37.325435 2572 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-ingress/e2e-shared-b-628c71-openshift-default-5d446f7cff-sp9c9"] Jul 16 01:20:37.328938 ip-10-0-143-200 kubenswrapper[2572]: I0716 01:20:37.328918 2572 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["openshift-ingress/e2e-shared-b-628c71-openshift-default-5d446f7cff-sp9c9"] Jul 16 01:20:37.448798 ip-10-0-143-200 kubenswrapper[2572]: I0716 01:20:37.448764 2572 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="4a730754-3193-49df-a072-f019e5b92dc2" path="/var/lib/kubelet/pods/4a730754-3193-49df-a072-f019e5b92dc2/volumes" Jul 16 01:20:37.878643 ip-10-0-143-200 kubenswrapper[2572]: I0716 01:20:37.878610 2572 log.go:25] "Finished parsing log file" path="/var/log/pods/opendatahub_llmisvc-controller-manager-777c6bd7d5-b8vdj_170820d7-8bbf-48db-8dfa-f6181d9ebae7/manager/0.log" Jul 16 01:20:38.011677 ip-10-0-143-200 kubenswrapper[2572]: I0716 01:20:38.011642 2572 log.go:25] "Finished parsing log file" path="/var/log/pods/opendatahub_maas-controller-79cf8b95fb-7nqz7_25d23071-268d-46ed-a5b5-9ddc749fd269/manager/0.log" Jul 16 01:20:38.135745 ip-10-0-143-200 kubenswrapper[2572]: I0716 01:20:38.135662 2572 log.go:25] "Finished parsing log file" path="/var/log/pods/opendatahub_model-serving-api-54d89bd5fc-xhll8_d0512aa0-6a35-4682-8ada-3de714156d10/server/0.log" Jul 16 01:20:38.262490 ip-10-0-143-200 kubenswrapper[2572]: I0716 01:20:38.262414 2572 log.go:25] "Finished parsing log file" path="/var/log/pods/opendatahub_odh-model-controller-7998c67dc4-xdsc2_b145063f-5d40-458a-b461-836198d4a9dc/manager/0.log" Jul 16 01:20:38.659338 ip-10-0-143-200 kubenswrapper[2572]: I0716 01:20:38.659306 2572 log.go:25] "Finished parsing log file" path="/var/log/pods/opendatahub_opendatahub-operator-controller-manager-5cf8c55d45-bfxb8_e58cb8b4-4bdb-4a40-bdd9-aa6f34eb09a4/copy-manifests/0.log" Jul 16 01:20:38.667504 ip-10-0-143-200 kubenswrapper[2572]: I0716 01:20:38.667473 2572 log.go:25] "Finished parsing log file" path="/var/log/pods/opendatahub_opendatahub-operator-controller-manager-5cf8c55d45-bfxb8_e58cb8b4-4bdb-4a40-bdd9-aa6f34eb09a4/manager/0.log" Jul 16 01:20:39.056128 ip-10-0-143-200 kubenswrapper[2572]: I0716 01:20:39.056073 2572 log.go:25] "Finished parsing log file" path="/var/log/pods/odh-ai-gateway-infra_maas-api-7464b7965d-lm97p_602f254c-3d38-4fa4-87d9-005fcfd1e70e/maas-api/0.log" Jul 16 01:20:39.173478 ip-10-0-143-200 kubenswrapper[2572]: I0716 01:20:39.173439 2572 log.go:25] "Finished parsing log file" path="/var/log/pods/odh-ai-gateway-infra_maas-api-e2e-ait-241c63c6-559cdc5ccc-wmpjg_5173bdc1-9d45-4ee4-8436-08828ef8a0a1/maas-api/0.log" Jul 16 01:20:39.558134 ip-10-0-143-200 kubenswrapper[2572]: I0716 01:20:39.558104 2572 log.go:25] "Finished parsing log file" path="/var/log/pods/odh-ai-gateway-infra_maas-api-e2e-derive-ccc0a86e-bc67c5bbc-lmsg2_d17a27cf-6819-47e6-a8ca-f0c24a740469/maas-api/0.log" Jul 16 01:20:41.174986 ip-10-0-143-200 kubenswrapper[2572]: I0716 01:20:41.174952 2572 log.go:25] "Finished parsing log file" path="/var/log/pods/odh-ai-gateway-infra_maas-api-key-cleanup-e2e-inf-b-e5854b-29736075-2wdsn_bf10ff5f-2652-4637-92c0-65f7672204c3/cleanup/0.log" Jul 16 01:20:41.742295 ip-10-0-143-200 kubenswrapper[2572]: I0716 01:20:41.742263 2572 log.go:25] "Finished parsing log file" path="/var/log/pods/odh-ai-gateway-infra_postgres-69fb4594cf-4ldvf_371615ce-ddf1-442f-a3b8-9b00b1d269d5/postgres/0.log" Jul 16 01:20:43.213021 ip-10-0-143-200 kubenswrapper[2572]: I0716 01:20:43.212991 2572 log.go:25] "Finished parsing log file" path="/var/log/pods/kuadrant-system_authorino-operator-75cc8d5cb7-9xhgm_5afe06aa-986b-47a9-a12a-6376957e65df/manager/0.log" Jul 16 01:20:43.330952 ip-10-0-143-200 kubenswrapper[2572]: I0716 01:20:43.330917 2572 log.go:25] "Finished parsing log file" path="/var/log/pods/kuadrant-system_dns-operator-controller-manager-864dd78c4c-clxrd_3ca593f9-fb02-418b-b4d4-934ac40611bd/manager/0.log" Jul 16 01:20:43.573326 ip-10-0-143-200 kubenswrapper[2572]: I0716 01:20:43.573239 2572 log.go:25] "Finished parsing log file" path="/var/log/pods/kuadrant-system_kuadrant-operator-catalog-vsp77_a31ac65b-1f88-41aa-b1d1-a141190eaae4/registry-server/0.log" Jul 16 01:20:43.699619 ip-10-0-143-200 kubenswrapper[2572]: I0716 01:20:43.699575 2572 log.go:25] "Finished parsing log file" path="/var/log/pods/kuadrant-system_kuadrant-operator-controller-manager-86878c769f-r4zb2_d605272b-dbc7-41b7-8272-a3f710fd6296/manager/0.log" Jul 16 01:20:44.311816 ip-10-0-143-200 kubenswrapper[2572]: I0716 01:20:44.311767 2572 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ingress_data-science-gateway-data-science-gateway-class-6576859cf4tzwx2_485db377-3d7c-46e8-9d8c-547ab86d96c0/istio-proxy/0.log" Jul 16 01:20:44.548494 ip-10-0-143-200 kubenswrapper[2572]: I0716 01:20:44.548465 2572 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ingress_kube-auth-proxy-7ffc9f7c5d-k6lk9_48d896b0-4c11-4a5f-813e-84b1195c94a7/kube-auth-proxy/0.log" Jul 16 01:20:44.791286 ip-10-0-143-200 kubenswrapper[2572]: I0716 01:20:44.791234 2572 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ingress_maas-default-gateway-openshift-default-56654dcf7-gr57g_3fc0bdda-3400-495e-b694-4e085c84fd49/istio-proxy/1.log" Jul 16 01:20:45.035243 ip-10-0-143-200 kubenswrapper[2572]: I0716 01:20:45.035200 2572 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ingress_payload-processing-f9b8d9b9b-fj2lp_c9557a6b-396b-4ab0-a3cb-8ddcf3c37f19/payload-processing/0.log" Jul 16 01:20:45.155067 ip-10-0-143-200 kubenswrapper[2572]: I0716 01:20:45.154986 2572 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ingress_router-default-78f9d48cbb-xwpwf_b82b08fd-bf48-4ee3-9d58-b648d6cba972/router/0.log" Jul 16 01:20:45.613071 ip-10-0-143-200 kubenswrapper[2572]: I0716 01:20:45.613034 2572 log.go:25] "Finished parsing log file" path="/var/log/pods/llm_e2e-distinct-simulated-kserve-59975d69f9-cfqcq_a9bd8f1c-1abe-4f9b-975e-84b0964c2d3e/main/0.log" Jul 16 01:20:45.726444 ip-10-0-143-200 kubenswrapper[2572]: I0716 01:20:45.726417 2572 log.go:25] "Finished parsing log file" path="/var/log/pods/llm_e2e-trlp-test-simulated-kserve-598466548d-hvsmm_47e15b8d-37df-4292-9c1f-f1ebfed0fd1c/main/0.log" Jul 16 01:20:51.785611 ip-10-0-143-200 kubenswrapper[2572]: I0716 01:20:51.785577 2572 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9gnmq_3a7e030d-fad8-480d-9dc6-fe5057048cf9/konnectivity-agent/0.log" limit=1024 Jul 16 01:20:51.900070 ip-10-0-143-200 kubenswrapper[2572]: I0716 01:20:51.900024 2572 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9gnmq_3a7e030d-fad8-480d-9dc6-fe5057048cf9/konnectivity-agent/0.log" limit=1024 Jul 16 01:20:54.743294 ip-10-0-143-200 kubenswrapper[2572]: I0716 01:20:54.743257 2572 log.go:25] "Finished parsing log file" path="/var/log/pods/kube-system_global-pull-secret-syncer-qcbcd_a9bdff50-c8ce-4ca4-a810-20be3909b23c/global-pull-secret-syncer/0.log" Jul 16 01:20:54.847451 ip-10-0-143-200 kubenswrapper[2572]: I0716 01:20:54.847426 2572 log.go:25] "Finished parsing log file" path="/var/log/pods/kube-system_konnectivity-agent-9gnmq_3a7e030d-fad8-480d-9dc6-fe5057048cf9/konnectivity-agent/0.log" Jul 16 01:20:54.958517 ip-10-0-143-200 kubenswrapper[2572]: I0716 01:20:54.958489 2572 log.go:25] "Finished parsing log file" path="/var/log/pods/kube-system_kube-apiserver-proxy-ip-10-0-143-200.ec2.internal_f0dfd958ab883b381e2b09c1edf12ef3/haproxy/0.log" Jul 16 01:21:00.512406 ip-10-0-143-200 kubenswrapper[2572]: I0716 01:21:00.512371 2572 log.go:25] "Finished parsing log file" path="/var/log/pods/kuadrant-system_authorino-operator-75cc8d5cb7-9xhgm_5afe06aa-986b-47a9-a12a-6376957e65df/manager/0.log" Jul 16 01:21:00.549975 ip-10-0-143-200 kubenswrapper[2572]: I0716 01:21:00.549927 2572 log.go:25] "Finished parsing log file" path="/var/log/pods/kuadrant-system_dns-operator-controller-manager-864dd78c4c-clxrd_3ca593f9-fb02-418b-b4d4-934ac40611bd/manager/0.log" Jul 16 01:21:00.635760 ip-10-0-143-200 kubenswrapper[2572]: I0716 01:21:00.635722 2572 log.go:25] "Finished parsing log file" path="/var/log/pods/kuadrant-system_kuadrant-operator-catalog-vsp77_a31ac65b-1f88-41aa-b1d1-a141190eaae4/registry-server/0.log" Jul 16 01:21:00.835752 ip-10-0-143-200 kubenswrapper[2572]: I0716 01:21:00.835670 2572 log.go:25] "Finished parsing log file" path="/var/log/pods/kuadrant-system_kuadrant-operator-controller-manager-86878c769f-r4zb2_d605272b-dbc7-41b7-8272-a3f710fd6296/manager/0.log" Jul 16 01:21:02.878189 ip-10-0-143-200 kubenswrapper[2572]: I0716 01:21:02.878154 2572 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_cluster-monitoring-operator-5f7f494dd4-dxc26_265fe067-cade-414b-be13-cb6dbab18057/cluster-monitoring-operator/0.log" Jul 16 01:21:03.218016 ip-10-0-143-200 kubenswrapper[2572]: I0716 01:21:03.217984 2572 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_node-exporter-kwdjw_49f06593-a9c5-408c-81e8-387df5345438/node-exporter/0.log" Jul 16 01:21:03.241670 ip-10-0-143-200 kubenswrapper[2572]: I0716 01:21:03.241643 2572 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_node-exporter-kwdjw_49f06593-a9c5-408c-81e8-387df5345438/kube-rbac-proxy/0.log" Jul 16 01:21:03.272244 ip-10-0-143-200 kubenswrapper[2572]: I0716 01:21:03.272219 2572 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_node-exporter-kwdjw_49f06593-a9c5-408c-81e8-387df5345438/init-textfile/0.log" Jul 16 01:21:04.999518 ip-10-0-143-200 kubenswrapper[2572]: I0716 01:21:04.999477 2572 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-must-gather-5b5g8/perf-node-gather-daemonset-6bh5j"] Jul 16 01:21:05.000113 ip-10-0-143-200 kubenswrapper[2572]: I0716 01:21:05.000092 2572 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="c3a9a894-3f1a-4746-a760-831441b8ac8b" containerName="istio-proxy" Jul 16 01:21:05.000198 ip-10-0-143-200 kubenswrapper[2572]: I0716 01:21:05.000115 2572 state_mem.go:107] "Deleted CPUSet assignment" podUID="c3a9a894-3f1a-4746-a760-831441b8ac8b" containerName="istio-proxy" Jul 16 01:21:05.000198 ip-10-0-143-200 kubenswrapper[2572]: I0716 01:21:05.000131 2572 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="4a730754-3193-49df-a072-f019e5b92dc2" containerName="istio-proxy" Jul 16 01:21:05.000198 ip-10-0-143-200 kubenswrapper[2572]: I0716 01:21:05.000140 2572 state_mem.go:107] "Deleted CPUSet assignment" podUID="4a730754-3193-49df-a072-f019e5b92dc2" containerName="istio-proxy" Jul 16 01:21:05.000198 ip-10-0-143-200 kubenswrapper[2572]: I0716 01:21:05.000164 2572 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="4a730754-3193-49df-a072-f019e5b92dc2" containerName="istio-proxy" Jul 16 01:21:05.000198 ip-10-0-143-200 kubenswrapper[2572]: I0716 01:21:05.000174 2572 state_mem.go:107] "Deleted CPUSet assignment" podUID="4a730754-3193-49df-a072-f019e5b92dc2" containerName="istio-proxy" Jul 16 01:21:05.000198 ip-10-0-143-200 kubenswrapper[2572]: I0716 01:21:05.000197 2572 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="0ea41688-c9b4-412e-97e7-3eca5747534d" containerName="test-curl-7507-139912869367600" Jul 16 01:21:05.000492 ip-10-0-143-200 kubenswrapper[2572]: I0716 01:21:05.000207 2572 state_mem.go:107] "Deleted CPUSet assignment" podUID="0ea41688-c9b4-412e-97e7-3eca5747534d" containerName="test-curl-7507-139912869367600" Jul 16 01:21:05.000492 ip-10-0-143-200 kubenswrapper[2572]: I0716 01:21:05.000328 2572 memory_manager.go:368] "RemoveStaleState removing state" podUID="4a730754-3193-49df-a072-f019e5b92dc2" containerName="istio-proxy" Jul 16 01:21:05.000492 ip-10-0-143-200 kubenswrapper[2572]: I0716 01:21:05.000344 2572 memory_manager.go:368] "RemoveStaleState removing state" podUID="0ea41688-c9b4-412e-97e7-3eca5747534d" containerName="test-curl-7507-139912869367600" Jul 16 01:21:05.000492 ip-10-0-143-200 kubenswrapper[2572]: I0716 01:21:05.000358 2572 memory_manager.go:368] "RemoveStaleState removing state" podUID="c3a9a894-3f1a-4746-a760-831441b8ac8b" containerName="istio-proxy" Jul 16 01:21:05.000492 ip-10-0-143-200 kubenswrapper[2572]: I0716 01:21:05.000369 2572 memory_manager.go:368] "RemoveStaleState removing state" podUID="c3a9a894-3f1a-4746-a760-831441b8ac8b" containerName="istio-proxy" Jul 16 01:21:05.003548 ip-10-0-143-200 kubenswrapper[2572]: I0716 01:21:05.003525 2572 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-must-gather-5b5g8/perf-node-gather-daemonset-6bh5j" Jul 16 01:21:05.006437 ip-10-0-143-200 kubenswrapper[2572]: I0716 01:21:05.006416 2572 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-must-gather-5b5g8\"/\"openshift-service-ca.crt\"" Jul 16 01:21:05.006547 ip-10-0-143-200 kubenswrapper[2572]: I0716 01:21:05.006436 2572 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-must-gather-5b5g8\"/\"kube-root-ca.crt\"" Jul 16 01:21:05.007597 ip-10-0-143-200 kubenswrapper[2572]: I0716 01:21:05.007576 2572 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-must-gather-5b5g8\"/\"default-dockercfg-h42dq\"" Jul 16 01:21:05.014354 ip-10-0-143-200 kubenswrapper[2572]: I0716 01:21:05.014331 2572 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-must-gather-5b5g8/perf-node-gather-daemonset-6bh5j"] Jul 16 01:21:05.095780 ip-10-0-143-200 kubenswrapper[2572]: I0716 01:21:05.095747 2572 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"podres\" (UniqueName: \"kubernetes.io/host-path/1b7e3d0b-795b-42e8-ae34-5221d37e8bf0-podres\") pod \"perf-node-gather-daemonset-6bh5j\" (UID: \"1b7e3d0b-795b-42e8-ae34-5221d37e8bf0\") " pod="openshift-must-gather-5b5g8/perf-node-gather-daemonset-6bh5j" Jul 16 01:21:05.095992 ip-10-0-143-200 kubenswrapper[2572]: I0716 01:21:05.095810 2572 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"proc\" (UniqueName: \"kubernetes.io/host-path/1b7e3d0b-795b-42e8-ae34-5221d37e8bf0-proc\") pod \"perf-node-gather-daemonset-6bh5j\" (UID: \"1b7e3d0b-795b-42e8-ae34-5221d37e8bf0\") " pod="openshift-must-gather-5b5g8/perf-node-gather-daemonset-6bh5j" Jul 16 01:21:05.095992 ip-10-0-143-200 kubenswrapper[2572]: I0716 01:21:05.095899 2572 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-dgqkd\" (UniqueName: \"kubernetes.io/projected/1b7e3d0b-795b-42e8-ae34-5221d37e8bf0-kube-api-access-dgqkd\") pod \"perf-node-gather-daemonset-6bh5j\" (UID: \"1b7e3d0b-795b-42e8-ae34-5221d37e8bf0\") " pod="openshift-must-gather-5b5g8/perf-node-gather-daemonset-6bh5j" Jul 16 01:21:05.095992 ip-10-0-143-200 kubenswrapper[2572]: I0716 01:21:05.095974 2572 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"lib-modules\" (UniqueName: \"kubernetes.io/host-path/1b7e3d0b-795b-42e8-ae34-5221d37e8bf0-lib-modules\") pod \"perf-node-gather-daemonset-6bh5j\" (UID: \"1b7e3d0b-795b-42e8-ae34-5221d37e8bf0\") " pod="openshift-must-gather-5b5g8/perf-node-gather-daemonset-6bh5j" Jul 16 01:21:05.096154 ip-10-0-143-200 kubenswrapper[2572]: I0716 01:21:05.096044 2572 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"sys\" (UniqueName: \"kubernetes.io/host-path/1b7e3d0b-795b-42e8-ae34-5221d37e8bf0-sys\") pod \"perf-node-gather-daemonset-6bh5j\" (UID: \"1b7e3d0b-795b-42e8-ae34-5221d37e8bf0\") " pod="openshift-must-gather-5b5g8/perf-node-gather-daemonset-6bh5j" Jul 16 01:21:05.109685 ip-10-0-143-200 kubenswrapper[2572]: I0716 01:21:05.109660 2572 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-network-console_networking-console-plugin-6cf856758c-vc87p_605373ad-28e7-4a57-9143-a698a1042a75/networking-console-plugin/0.log" Jul 16 01:21:05.197566 ip-10-0-143-200 kubenswrapper[2572]: I0716 01:21:05.197529 2572 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"sys\" (UniqueName: \"kubernetes.io/host-path/1b7e3d0b-795b-42e8-ae34-5221d37e8bf0-sys\") pod \"perf-node-gather-daemonset-6bh5j\" (UID: \"1b7e3d0b-795b-42e8-ae34-5221d37e8bf0\") " pod="openshift-must-gather-5b5g8/perf-node-gather-daemonset-6bh5j" Jul 16 01:21:05.197566 ip-10-0-143-200 kubenswrapper[2572]: I0716 01:21:05.197574 2572 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"podres\" (UniqueName: \"kubernetes.io/host-path/1b7e3d0b-795b-42e8-ae34-5221d37e8bf0-podres\") pod \"perf-node-gather-daemonset-6bh5j\" (UID: \"1b7e3d0b-795b-42e8-ae34-5221d37e8bf0\") " pod="openshift-must-gather-5b5g8/perf-node-gather-daemonset-6bh5j" Jul 16 01:21:05.197845 ip-10-0-143-200 kubenswrapper[2572]: I0716 01:21:05.197708 2572 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"sys\" (UniqueName: \"kubernetes.io/host-path/1b7e3d0b-795b-42e8-ae34-5221d37e8bf0-sys\") pod \"perf-node-gather-daemonset-6bh5j\" (UID: \"1b7e3d0b-795b-42e8-ae34-5221d37e8bf0\") " pod="openshift-must-gather-5b5g8/perf-node-gather-daemonset-6bh5j" Jul 16 01:21:05.197845 ip-10-0-143-200 kubenswrapper[2572]: I0716 01:21:05.197755 2572 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"podres\" (UniqueName: \"kubernetes.io/host-path/1b7e3d0b-795b-42e8-ae34-5221d37e8bf0-podres\") pod \"perf-node-gather-daemonset-6bh5j\" (UID: \"1b7e3d0b-795b-42e8-ae34-5221d37e8bf0\") " pod="openshift-must-gather-5b5g8/perf-node-gather-daemonset-6bh5j" Jul 16 01:21:05.197845 ip-10-0-143-200 kubenswrapper[2572]: I0716 01:21:05.197782 2572 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"proc\" (UniqueName: \"kubernetes.io/host-path/1b7e3d0b-795b-42e8-ae34-5221d37e8bf0-proc\") pod \"perf-node-gather-daemonset-6bh5j\" (UID: \"1b7e3d0b-795b-42e8-ae34-5221d37e8bf0\") " pod="openshift-must-gather-5b5g8/perf-node-gather-daemonset-6bh5j" Jul 16 01:21:05.197845 ip-10-0-143-200 kubenswrapper[2572]: I0716 01:21:05.197809 2572 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-dgqkd\" (UniqueName: \"kubernetes.io/projected/1b7e3d0b-795b-42e8-ae34-5221d37e8bf0-kube-api-access-dgqkd\") pod \"perf-node-gather-daemonset-6bh5j\" (UID: \"1b7e3d0b-795b-42e8-ae34-5221d37e8bf0\") " pod="openshift-must-gather-5b5g8/perf-node-gather-daemonset-6bh5j" Jul 16 01:21:05.197845 ip-10-0-143-200 kubenswrapper[2572]: I0716 01:21:05.197847 2572 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"lib-modules\" (UniqueName: \"kubernetes.io/host-path/1b7e3d0b-795b-42e8-ae34-5221d37e8bf0-lib-modules\") pod \"perf-node-gather-daemonset-6bh5j\" (UID: \"1b7e3d0b-795b-42e8-ae34-5221d37e8bf0\") " pod="openshift-must-gather-5b5g8/perf-node-gather-daemonset-6bh5j" Jul 16 01:21:05.198054 ip-10-0-143-200 kubenswrapper[2572]: I0716 01:21:05.197855 2572 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"proc\" (UniqueName: \"kubernetes.io/host-path/1b7e3d0b-795b-42e8-ae34-5221d37e8bf0-proc\") pod \"perf-node-gather-daemonset-6bh5j\" (UID: \"1b7e3d0b-795b-42e8-ae34-5221d37e8bf0\") " pod="openshift-must-gather-5b5g8/perf-node-gather-daemonset-6bh5j" Jul 16 01:21:05.198054 ip-10-0-143-200 kubenswrapper[2572]: I0716 01:21:05.198022 2572 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"lib-modules\" (UniqueName: \"kubernetes.io/host-path/1b7e3d0b-795b-42e8-ae34-5221d37e8bf0-lib-modules\") pod \"perf-node-gather-daemonset-6bh5j\" (UID: \"1b7e3d0b-795b-42e8-ae34-5221d37e8bf0\") " pod="openshift-must-gather-5b5g8/perf-node-gather-daemonset-6bh5j" Jul 16 01:21:05.207011 ip-10-0-143-200 kubenswrapper[2572]: I0716 01:21:05.206980 2572 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-dgqkd\" (UniqueName: \"kubernetes.io/projected/1b7e3d0b-795b-42e8-ae34-5221d37e8bf0-kube-api-access-dgqkd\") pod \"perf-node-gather-daemonset-6bh5j\" (UID: \"1b7e3d0b-795b-42e8-ae34-5221d37e8bf0\") " pod="openshift-must-gather-5b5g8/perf-node-gather-daemonset-6bh5j" Jul 16 01:21:05.314854 ip-10-0-143-200 kubenswrapper[2572]: I0716 01:21:05.314749 2572 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-must-gather-5b5g8/perf-node-gather-daemonset-6bh5j" Jul 16 01:21:05.457321 ip-10-0-143-200 kubenswrapper[2572]: I0716 01:21:05.457294 2572 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-must-gather-5b5g8/perf-node-gather-daemonset-6bh5j"] Jul 16 01:21:05.458583 ip-10-0-143-200 kubenswrapper[2572]: W0716 01:21:05.458554 2572 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-pod1b7e3d0b_795b_42e8_ae34_5221d37e8bf0.slice/crio-19c3237fea906138c340747af7d88c5445b0bde4733c72cab6a14176188c1edb WatchSource:0}: Error finding container 19c3237fea906138c340747af7d88c5445b0bde4733c72cab6a14176188c1edb: Status 404 returned error can't find the container with id 19c3237fea906138c340747af7d88c5445b0bde4733c72cab6a14176188c1edb Jul 16 01:21:05.693304 ip-10-0-143-200 kubenswrapper[2572]: I0716 01:21:05.693277 2572 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-console-operator_console-operator-7866497494-pjc6j_a64bcd05-8596-4c12-b88e-e974d676e25f/console-operator/2.log" Jul 16 01:21:05.701701 ip-10-0-143-200 kubenswrapper[2572]: I0716 01:21:05.701677 2572 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-console-operator_console-operator-7866497494-pjc6j_a64bcd05-8596-4c12-b88e-e974d676e25f/console-operator/3.log" Jul 16 01:21:06.459465 ip-10-0-143-200 kubenswrapper[2572]: I0716 01:21:06.459430 2572 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-must-gather-5b5g8/perf-node-gather-daemonset-6bh5j" event={"ID":"1b7e3d0b-795b-42e8-ae34-5221d37e8bf0","Type":"ContainerStarted","Data":"ac3e4464a7e376ff987d7cec596e85fa02751dac1e48ec3aa65e44aeb4516ec9"} Jul 16 01:21:06.459893 ip-10-0-143-200 kubenswrapper[2572]: I0716 01:21:06.459492 2572 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-must-gather-5b5g8/perf-node-gather-daemonset-6bh5j" event={"ID":"1b7e3d0b-795b-42e8-ae34-5221d37e8bf0","Type":"ContainerStarted","Data":"19c3237fea906138c340747af7d88c5445b0bde4733c72cab6a14176188c1edb"} Jul 16 01:21:06.459893 ip-10-0-143-200 kubenswrapper[2572]: I0716 01:21:06.459524 2572 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-must-gather-5b5g8/perf-node-gather-daemonset-6bh5j" Jul 16 01:21:06.485652 ip-10-0-143-200 kubenswrapper[2572]: I0716 01:21:06.485573 2572 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-must-gather-5b5g8/perf-node-gather-daemonset-6bh5j" podStartSLOduration=2.485556309 podStartE2EDuration="2.485556309s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=true podCreationTimestamp="2026-07-16 01:21:04 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-16 01:21:06.48344597 +0000 UTC m=+2471.592200648" watchObservedRunningTime="2026-07-16 01:21:06.485556309 +0000 UTC m=+2471.594310984" Jul 16 01:21:06.785331 ip-10-0-143-200 kubenswrapper[2572]: I0716 01:21:06.785243 2572 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-cluster-storage-operator_volume-data-source-validator-78495597f6-wffwm_3cb174c0-4582-44f3-879a-23697944bc47/volume-data-source-validator/0.log" Jul 16 01:21:08.064399 ip-10-0-143-200 kubenswrapper[2572]: I0716 01:21:08.064360 2572 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-dns_dns-default-w6cgc_645fc509-972b-4332-8562-2309cc0eab1b/dns/0.log" Jul 16 01:21:08.108886 ip-10-0-143-200 kubenswrapper[2572]: I0716 01:21:08.108859 2572 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-dns_dns-default-w6cgc_645fc509-972b-4332-8562-2309cc0eab1b/kube-rbac-proxy/0.log" Jul 16 01:21:08.139550 ip-10-0-143-200 kubenswrapper[2572]: I0716 01:21:08.139524 2572 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-dns_node-resolver-6qlgz_24fd12c1-ddcb-427f-ac7e-4e2877450cb8/dns-node-resolver/0.log" Jul 16 01:21:08.806607 ip-10-0-143-200 kubenswrapper[2572]: I0716 01:21:08.806574 2572 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-image-registry_node-ca-6msnd_ff468ef8-66ba-48f2-8531-da662a5b19d3/node-ca/0.log" Jul 16 01:21:09.814580 ip-10-0-143-200 kubenswrapper[2572]: I0716 01:21:09.814549 2572 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ingress_data-science-gateway-data-science-gateway-class-6576859cf4tzwx2_485db377-3d7c-46e8-9d8c-547ab86d96c0/istio-proxy/0.log" Jul 16 01:21:09.930728 ip-10-0-143-200 kubenswrapper[2572]: I0716 01:21:09.930689 2572 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ingress_kube-auth-proxy-7ffc9f7c5d-k6lk9_48d896b0-4c11-4a5f-813e-84b1195c94a7/kube-auth-proxy/0.log" Jul 16 01:21:09.991398 ip-10-0-143-200 kubenswrapper[2572]: I0716 01:21:09.991365 2572 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ingress_maas-default-gateway-openshift-default-56654dcf7-gr57g_3fc0bdda-3400-495e-b694-4e085c84fd49/istio-proxy/0.log" Jul 16 01:21:10.049359 ip-10-0-143-200 kubenswrapper[2572]: I0716 01:21:10.049325 2572 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ingress_maas-default-gateway-openshift-default-56654dcf7-gr57g_3fc0bdda-3400-495e-b694-4e085c84fd49/istio-proxy/1.log" Jul 16 01:21:10.118385 ip-10-0-143-200 kubenswrapper[2572]: I0716 01:21:10.118297 2572 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ingress_payload-processing-f9b8d9b9b-fj2lp_c9557a6b-396b-4ab0-a3cb-8ddcf3c37f19/payload-processing/0.log" Jul 16 01:21:10.143806 ip-10-0-143-200 kubenswrapper[2572]: I0716 01:21:10.143781 2572 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ingress_router-default-78f9d48cbb-xwpwf_b82b08fd-bf48-4ee3-9d58-b648d6cba972/router/0.log" Jul 16 01:21:10.857869 ip-10-0-143-200 kubenswrapper[2572]: I0716 01:21:10.857840 2572 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ingress-canary_ingress-canary-hhtd6_6f7eb059-755b-47de-82ab-bfaa72b8f899/serve-healthcheck-canary/0.log" Jul 16 01:21:11.368929 ip-10-0-143-200 kubenswrapper[2572]: I0716 01:21:11.368897 2572 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-insights_insights-operator-68d9dd94cf-d4nrt_0563669f-9ac5-43d0-a3c0-47c36275ee46/insights-operator/0.log" Jul 16 01:21:11.370027 ip-10-0-143-200 kubenswrapper[2572]: I0716 01:21:11.369979 2572 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-insights_insights-operator-68d9dd94cf-d4nrt_0563669f-9ac5-43d0-a3c0-47c36275ee46/insights-operator/1.log" Jul 16 01:21:11.540716 ip-10-0-143-200 kubenswrapper[2572]: I0716 01:21:11.540686 2572 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-insights_insights-runtime-extractor-q8gql_5f736cca-f28c-49b2-9dc9-24f0982e2d24/kube-rbac-proxy/0.log" Jul 16 01:21:11.561715 ip-10-0-143-200 kubenswrapper[2572]: I0716 01:21:11.561692 2572 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-insights_insights-runtime-extractor-q8gql_5f736cca-f28c-49b2-9dc9-24f0982e2d24/exporter/0.log" Jul 16 01:21:11.585012 ip-10-0-143-200 kubenswrapper[2572]: I0716 01:21:11.584985 2572 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-insights_insights-runtime-extractor-q8gql_5f736cca-f28c-49b2-9dc9-24f0982e2d24/extractor/0.log" Jul 16 01:21:12.474363 ip-10-0-143-200 kubenswrapper[2572]: I0716 01:21:12.474330 2572 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-must-gather-5b5g8/perf-node-gather-daemonset-6bh5j" Jul 16 01:21:13.892563 ip-10-0-143-200 kubenswrapper[2572]: I0716 01:21:13.892527 2572 log.go:25] "Finished parsing log file" path="/var/log/pods/opendatahub_llmisvc-controller-manager-777c6bd7d5-b8vdj_170820d7-8bbf-48db-8dfa-f6181d9ebae7/manager/0.log" Jul 16 01:21:14.046282 ip-10-0-143-200 kubenswrapper[2572]: I0716 01:21:14.046238 2572 log.go:25] "Finished parsing log file" path="/var/log/pods/opendatahub_maas-controller-79cf8b95fb-7nqz7_25d23071-268d-46ed-a5b5-9ddc749fd269/manager/0.log" Jul 16 01:21:14.066104 ip-10-0-143-200 kubenswrapper[2572]: I0716 01:21:14.066078 2572 log.go:25] "Finished parsing log file" path="/var/log/pods/opendatahub_model-serving-api-54d89bd5fc-xhll8_d0512aa0-6a35-4682-8ada-3de714156d10/server/0.log" Jul 16 01:21:14.167199 ip-10-0-143-200 kubenswrapper[2572]: I0716 01:21:14.167110 2572 log.go:25] "Finished parsing log file" path="/var/log/pods/opendatahub_odh-model-controller-7998c67dc4-xdsc2_b145063f-5d40-458a-b461-836198d4a9dc/manager/0.log" Jul 16 01:21:14.363449 ip-10-0-143-200 kubenswrapper[2572]: I0716 01:21:14.363414 2572 log.go:25] "Finished parsing log file" path="/var/log/pods/opendatahub_opendatahub-operator-controller-manager-5cf8c55d45-bfxb8_e58cb8b4-4bdb-4a40-bdd9-aa6f34eb09a4/manager/0.log" Jul 16 01:21:14.385566 ip-10-0-143-200 kubenswrapper[2572]: I0716 01:21:14.385539 2572 log.go:25] "Finished parsing log file" path="/var/log/pods/opendatahub_opendatahub-operator-controller-manager-5cf8c55d45-bfxb8_e58cb8b4-4bdb-4a40-bdd9-aa6f34eb09a4/copy-manifests/0.log" Jul 16 01:21:16.025306 ip-10-0-143-200 kubenswrapper[2572]: I0716 01:21:16.025276 2572 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ingress_data-science-gateway-data-science-gateway-class-6576859cf4tzwx2_485db377-3d7c-46e8-9d8c-547ab86d96c0/istio-proxy/0.log" Jul 16 01:21:16.139794 ip-10-0-143-200 kubenswrapper[2572]: I0716 01:21:16.139768 2572 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ingress_kube-auth-proxy-7ffc9f7c5d-k6lk9_48d896b0-4c11-4a5f-813e-84b1195c94a7/kube-auth-proxy/0.log" Jul 16 01:21:16.187134 ip-10-0-143-200 kubenswrapper[2572]: I0716 01:21:16.187096 2572 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ingress_maas-default-gateway-openshift-default-56654dcf7-gr57g_3fc0bdda-3400-495e-b694-4e085c84fd49/istio-proxy/0.log" Jul 16 01:21:16.244666 ip-10-0-143-200 kubenswrapper[2572]: I0716 01:21:16.244623 2572 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ingress_maas-default-gateway-openshift-default-56654dcf7-gr57g_3fc0bdda-3400-495e-b694-4e085c84fd49/istio-proxy/1.log" Jul 16 01:21:16.300085 ip-10-0-143-200 kubenswrapper[2572]: I0716 01:21:16.300054 2572 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ingress_payload-processing-f9b8d9b9b-fj2lp_c9557a6b-396b-4ab0-a3cb-8ddcf3c37f19/payload-processing/0.log" Jul 16 01:21:16.321398 ip-10-0-143-200 kubenswrapper[2572]: I0716 01:21:16.321310 2572 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ingress_router-default-78f9d48cbb-xwpwf_b82b08fd-bf48-4ee3-9d58-b648d6cba972/router/0.log" Jul 16 01:21:17.494560 ip-10-0-143-200 kubenswrapper[2572]: I0716 01:21:17.494528 2572 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ingress_data-science-gateway-data-science-gateway-class-6576859cf4tzwx2_485db377-3d7c-46e8-9d8c-547ab86d96c0/istio-proxy/0.log" Jul 16 01:21:17.598200 ip-10-0-143-200 kubenswrapper[2572]: I0716 01:21:17.598164 2572 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ingress_kube-auth-proxy-7ffc9f7c5d-k6lk9_48d896b0-4c11-4a5f-813e-84b1195c94a7/kube-auth-proxy/0.log" Jul 16 01:21:17.645591 ip-10-0-143-200 kubenswrapper[2572]: I0716 01:21:17.645488 2572 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ingress_maas-default-gateway-openshift-default-56654dcf7-gr57g_3fc0bdda-3400-495e-b694-4e085c84fd49/istio-proxy/0.log" Jul 16 01:21:17.708669 ip-10-0-143-200 kubenswrapper[2572]: I0716 01:21:17.708623 2572 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ingress_maas-default-gateway-openshift-default-56654dcf7-gr57g_3fc0bdda-3400-495e-b694-4e085c84fd49/istio-proxy/1.log" Jul 16 01:21:17.768874 ip-10-0-143-200 kubenswrapper[2572]: I0716 01:21:17.768842 2572 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ingress_payload-processing-f9b8d9b9b-fj2lp_c9557a6b-396b-4ab0-a3cb-8ddcf3c37f19/payload-processing/0.log" Jul 16 01:21:17.795733 ip-10-0-143-200 kubenswrapper[2572]: I0716 01:21:17.795705 2572 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ingress_router-default-78f9d48cbb-xwpwf_b82b08fd-bf48-4ee3-9d58-b648d6cba972/router/0.log" Jul 16 01:21:21.003236 ip-10-0-143-200 kubenswrapper[2572]: I0716 01:21:21.003203 2572 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-kube-storage-version-migrator_migrator-7f59b54f6c-m7dk5_dab79fd4-2058-46eb-8555-d178887d1e28/migrator/0.log" Jul 16 01:21:21.024790 ip-10-0-143-200 kubenswrapper[2572]: I0716 01:21:21.024759 2572 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-kube-storage-version-migrator_migrator-7f59b54f6c-m7dk5_dab79fd4-2058-46eb-8555-d178887d1e28/graceful-termination/0.log" Jul 16 01:21:21.388675 ip-10-0-143-200 kubenswrapper[2572]: I0716 01:21:21.388559 2572 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-kube-storage-version-migrator-operator_kube-storage-version-migrator-operator-799cb7f958-tksqh_07e0ab9b-1bfe-4335-bac1-8d9d158b0085/kube-storage-version-migrator-operator/1.log" Jul 16 01:21:21.390842 ip-10-0-143-200 kubenswrapper[2572]: I0716 01:21:21.390818 2572 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-kube-storage-version-migrator-operator_kube-storage-version-migrator-operator-799cb7f958-tksqh_07e0ab9b-1bfe-4335-bac1-8d9d158b0085/kube-storage-version-migrator-operator/0.log" Jul 16 01:21:22.768110 ip-10-0-143-200 kubenswrapper[2572]: I0716 01:21:22.768078 2572 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-multus_multus-additional-cni-plugins-vfntt_a51f2d32-4840-4f0c-804d-c8aa6015ae5a/kube-multus-additional-cni-plugins/0.log" Jul 16 01:21:22.791094 ip-10-0-143-200 kubenswrapper[2572]: I0716 01:21:22.791067 2572 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-multus_multus-additional-cni-plugins-vfntt_a51f2d32-4840-4f0c-804d-c8aa6015ae5a/egress-router-binary-copy/0.log" Jul 16 01:21:22.814265 ip-10-0-143-200 kubenswrapper[2572]: I0716 01:21:22.814240 2572 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-multus_multus-additional-cni-plugins-vfntt_a51f2d32-4840-4f0c-804d-c8aa6015ae5a/cni-plugins/0.log" Jul 16 01:21:22.837859 ip-10-0-143-200 kubenswrapper[2572]: I0716 01:21:22.837828 2572 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-multus_multus-additional-cni-plugins-vfntt_a51f2d32-4840-4f0c-804d-c8aa6015ae5a/bond-cni-plugin/0.log" Jul 16 01:21:22.865648 ip-10-0-143-200 kubenswrapper[2572]: I0716 01:21:22.865603 2572 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-multus_multus-additional-cni-plugins-vfntt_a51f2d32-4840-4f0c-804d-c8aa6015ae5a/routeoverride-cni/0.log" Jul 16 01:21:22.888149 ip-10-0-143-200 kubenswrapper[2572]: I0716 01:21:22.888122 2572 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-multus_multus-additional-cni-plugins-vfntt_a51f2d32-4840-4f0c-804d-c8aa6015ae5a/whereabouts-cni-bincopy/0.log" Jul 16 01:21:22.910927 ip-10-0-143-200 kubenswrapper[2572]: I0716 01:21:22.910895 2572 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-multus_multus-additional-cni-plugins-vfntt_a51f2d32-4840-4f0c-804d-c8aa6015ae5a/whereabouts-cni/0.log" Jul 16 01:21:22.992890 ip-10-0-143-200 kubenswrapper[2572]: I0716 01:21:22.992857 2572 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-multus_multus-szqbq_2b89891b-83eb-48c2-abe4-747d1ca26caf/kube-multus/0.log" Jul 16 01:21:23.049681 ip-10-0-143-200 kubenswrapper[2572]: I0716 01:21:23.049571 2572 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-multus_network-metrics-daemon-5tphg_4a08cb1b-3bfa-49cf-8ff0-edef0ff944b9/network-metrics-daemon/0.log" Jul 16 01:21:23.068538 ip-10-0-143-200 kubenswrapper[2572]: I0716 01:21:23.068476 2572 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-multus_network-metrics-daemon-5tphg_4a08cb1b-3bfa-49cf-8ff0-edef0ff944b9/kube-rbac-proxy/0.log"