Jul 29 19:14:58.258239 ip-10-0-131-93 systemd[1]: kubelet.service: Failed to load environment files: No such file or directory Jul 29 19:14:58.258256 ip-10-0-131-93 systemd[1]: kubelet.service: Failed to run 'start-pre' task: No such file or directory Jul 29 19:14:58.258264 ip-10-0-131-93 systemd[1]: kubelet.service: Failed with result 'resources'. Jul 29 19:14:58.258549 ip-10-0-131-93 systemd[1]: Failed to start Kubernetes Kubelet. Jul 29 19:15:08.327088 ip-10-0-131-93 systemd[1]: kubelet.service: Failed to schedule restart job: Unit crio.service not found. Jul 29 19:15:08.327105 ip-10-0-131-93 systemd[1]: kubelet.service: Failed with result 'resources'. -- Boot e33fc32dd4e54213bd8fd6f08f838937 -- Jul 29 19:17:02.322513 ip-10-0-131-93 systemd[1]: Starting Kubernetes Kubelet... Jul 29 19:17:02.685772 ip-10-0-131-93 kubenswrapper[2564]: Flag --container-runtime-endpoint has been deprecated, This parameter should be set via the config file specified by the Kubelet's --config flag. See https://kubernetes.io/docs/tasks/administer-cluster/kubelet-config-file/ for more information. Jul 29 19:17:02.685772 ip-10-0-131-93 kubenswrapper[2564]: Flag --minimum-container-ttl-duration has been deprecated, Use --eviction-hard or --eviction-soft instead. Will be removed in a future version. Jul 29 19:17:02.685772 ip-10-0-131-93 kubenswrapper[2564]: Flag --volume-plugin-dir has been deprecated, This parameter should be set via the config file specified by the Kubelet's --config flag. See https://kubernetes.io/docs/tasks/administer-cluster/kubelet-config-file/ for more information. Jul 29 19:17:02.685772 ip-10-0-131-93 kubenswrapper[2564]: Flag --pod-infra-container-image has been deprecated, will be removed in 1.35. Image garbage collector will get sandbox image information from CRI. Jul 29 19:17:02.685772 ip-10-0-131-93 kubenswrapper[2564]: Flag --system-reserved has been deprecated, This parameter should be set via the config file specified by the Kubelet's --config flag. See https://kubernetes.io/docs/tasks/administer-cluster/kubelet-config-file/ for more information. Jul 29 19:17:02.687390 ip-10-0-131-93 kubenswrapper[2564]: I0729 19:17:02.687303 2564 server.go:213] "--pod-infra-container-image will not be pruned by the image garbage collector in kubelet and should also be set in the remote runtime" Jul 29 19:17:02.692374 ip-10-0-131-93 kubenswrapper[2564]: W0729 19:17:02.692359 2564 feature_gate.go:328] unrecognized feature gate: VSphereConfigurableMaxAllowedBlockVolumesPerNode Jul 29 19:17:02.692374 ip-10-0-131-93 kubenswrapper[2564]: W0729 19:17:02.692373 2564 feature_gate.go:328] unrecognized feature gate: GatewayAPIWithoutOLM Jul 29 19:17:02.692437 ip-10-0-131-93 kubenswrapper[2564]: W0729 19:17:02.692377 2564 feature_gate.go:328] unrecognized feature gate: GCPDualStackInstall Jul 29 19:17:02.692437 ip-10-0-131-93 kubenswrapper[2564]: W0729 19:17:02.692381 2564 feature_gate.go:328] unrecognized feature gate: GCPCustomAPIEndpointsInstall Jul 29 19:17:02.692437 ip-10-0-131-93 kubenswrapper[2564]: W0729 19:17:02.692384 2564 feature_gate.go:328] unrecognized feature gate: AlibabaPlatform Jul 29 19:17:02.692437 ip-10-0-131-93 kubenswrapper[2564]: W0729 19:17:02.692392 2564 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesAzure Jul 29 19:17:02.692437 ip-10-0-131-93 kubenswrapper[2564]: W0729 19:17:02.692395 2564 feature_gate.go:328] unrecognized feature gate: MetricsCollectionProfiles Jul 29 19:17:02.692437 ip-10-0-131-93 kubenswrapper[2564]: W0729 19:17:02.692398 2564 feature_gate.go:328] unrecognized feature gate: ExternalOIDC Jul 29 19:17:02.692437 ip-10-0-131-93 kubenswrapper[2564]: W0729 19:17:02.692401 2564 feature_gate.go:328] unrecognized feature gate: OVNObservability Jul 29 19:17:02.692437 ip-10-0-131-93 kubenswrapper[2564]: W0729 19:17:02.692404 2564 feature_gate.go:328] unrecognized feature gate: AWSDualStackInstall Jul 29 19:17:02.692437 ip-10-0-131-93 kubenswrapper[2564]: W0729 19:17:02.692407 2564 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesAWS Jul 29 19:17:02.692437 ip-10-0-131-93 kubenswrapper[2564]: W0729 19:17:02.692409 2564 feature_gate.go:328] unrecognized feature gate: ShortCertRotation Jul 29 19:17:02.692437 ip-10-0-131-93 kubenswrapper[2564]: W0729 19:17:02.692412 2564 feature_gate.go:328] unrecognized feature gate: AutomatedEtcdBackup Jul 29 19:17:02.692437 ip-10-0-131-93 kubenswrapper[2564]: W0729 19:17:02.692414 2564 feature_gate.go:328] unrecognized feature gate: MixedCPUsAllocation Jul 29 19:17:02.692437 ip-10-0-131-93 kubenswrapper[2564]: W0729 19:17:02.692417 2564 feature_gate.go:328] unrecognized feature gate: SignatureStores Jul 29 19:17:02.692437 ip-10-0-131-93 kubenswrapper[2564]: W0729 19:17:02.692419 2564 feature_gate.go:328] unrecognized feature gate: EventTTL Jul 29 19:17:02.692437 ip-10-0-131-93 kubenswrapper[2564]: W0729 19:17:02.692422 2564 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesCPMS Jul 29 19:17:02.692437 ip-10-0-131-93 kubenswrapper[2564]: W0729 19:17:02.692424 2564 feature_gate.go:328] unrecognized feature gate: MachineAPIOperatorDisableMachineHealthCheckController Jul 29 19:17:02.692437 ip-10-0-131-93 kubenswrapper[2564]: W0729 19:17:02.692428 2564 feature_gate.go:328] unrecognized feature gate: HyperShiftOnlyDynamicResourceAllocation Jul 29 19:17:02.692437 ip-10-0-131-93 kubenswrapper[2564]: W0729 19:17:02.692430 2564 feature_gate.go:328] unrecognized feature gate: ClusterAPIMachineManagement Jul 29 19:17:02.692437 ip-10-0-131-93 kubenswrapper[2564]: W0729 19:17:02.692433 2564 feature_gate.go:328] unrecognized feature gate: NoRegistryClusterInstall Jul 29 19:17:02.692437 ip-10-0-131-93 kubenswrapper[2564]: W0729 19:17:02.692435 2564 feature_gate.go:328] unrecognized feature gate: BootImageSkewEnforcement Jul 29 19:17:02.692911 ip-10-0-131-93 kubenswrapper[2564]: W0729 19:17:02.692438 2564 feature_gate.go:328] unrecognized feature gate: AWSDedicatedHosts Jul 29 19:17:02.692911 ip-10-0-131-93 kubenswrapper[2564]: W0729 19:17:02.692441 2564 feature_gate.go:328] unrecognized feature gate: NetworkSegmentation Jul 29 19:17:02.692911 ip-10-0-131-93 kubenswrapper[2564]: W0729 19:17:02.692444 2564 feature_gate.go:328] unrecognized feature gate: RouteAdvertisements Jul 29 19:17:02.692911 ip-10-0-131-93 kubenswrapper[2564]: W0729 19:17:02.692446 2564 feature_gate.go:328] unrecognized feature gate: NewOLMPreflightPermissionChecks Jul 29 19:17:02.692911 ip-10-0-131-93 kubenswrapper[2564]: W0729 19:17:02.692449 2564 feature_gate.go:328] unrecognized feature gate: OnPremDNSRecords Jul 29 19:17:02.692911 ip-10-0-131-93 kubenswrapper[2564]: W0729 19:17:02.692452 2564 feature_gate.go:328] unrecognized feature gate: ClusterVersionOperatorConfiguration Jul 29 19:17:02.692911 ip-10-0-131-93 kubenswrapper[2564]: W0729 19:17:02.692454 2564 feature_gate.go:328] unrecognized feature gate: AzureDedicatedHosts Jul 29 19:17:02.692911 ip-10-0-131-93 kubenswrapper[2564]: W0729 19:17:02.692457 2564 feature_gate.go:328] unrecognized feature gate: DyanmicServiceEndpointIBMCloud Jul 29 19:17:02.692911 ip-10-0-131-93 kubenswrapper[2564]: W0729 19:17:02.692459 2564 feature_gate.go:328] unrecognized feature gate: ManagedBootImages Jul 29 19:17:02.692911 ip-10-0-131-93 kubenswrapper[2564]: W0729 19:17:02.692462 2564 feature_gate.go:328] unrecognized feature gate: AWSServiceLBNetworkSecurityGroup Jul 29 19:17:02.692911 ip-10-0-131-93 kubenswrapper[2564]: W0729 19:17:02.692466 2564 feature_gate.go:351] Setting GA feature gate ServiceAccountTokenNodeBinding=true. It will be removed in a future release. Jul 29 19:17:02.692911 ip-10-0-131-93 kubenswrapper[2564]: W0729 19:17:02.692470 2564 feature_gate.go:328] unrecognized feature gate: NetworkDiagnosticsConfig Jul 29 19:17:02.692911 ip-10-0-131-93 kubenswrapper[2564]: W0729 19:17:02.692473 2564 feature_gate.go:328] unrecognized feature gate: AdminNetworkPolicy Jul 29 19:17:02.692911 ip-10-0-131-93 kubenswrapper[2564]: W0729 19:17:02.692475 2564 feature_gate.go:328] unrecognized feature gate: ClusterAPIInstallIBMCloud Jul 29 19:17:02.692911 ip-10-0-131-93 kubenswrapper[2564]: W0729 19:17:02.692477 2564 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesvSphere Jul 29 19:17:02.692911 ip-10-0-131-93 kubenswrapper[2564]: W0729 19:17:02.692480 2564 feature_gate.go:328] unrecognized feature gate: PreconfiguredUDNAddresses Jul 29 19:17:02.692911 ip-10-0-131-93 kubenswrapper[2564]: W0729 19:17:02.692483 2564 feature_gate.go:328] unrecognized feature gate: ImageModeStatusReporting Jul 29 19:17:02.692911 ip-10-0-131-93 kubenswrapper[2564]: W0729 19:17:02.692485 2564 feature_gate.go:328] unrecognized feature gate: DualReplica Jul 29 19:17:02.692911 ip-10-0-131-93 kubenswrapper[2564]: W0729 19:17:02.692488 2564 feature_gate.go:328] unrecognized feature gate: AdditionalRoutingCapabilities Jul 29 19:17:02.693532 ip-10-0-131-93 kubenswrapper[2564]: W0729 19:17:02.692490 2564 feature_gate.go:328] unrecognized feature gate: ConsolePluginContentSecurityPolicy Jul 29 19:17:02.693532 ip-10-0-131-93 kubenswrapper[2564]: W0729 19:17:02.692493 2564 feature_gate.go:328] unrecognized feature gate: UpgradeStatus Jul 29 19:17:02.693532 ip-10-0-131-93 kubenswrapper[2564]: W0729 19:17:02.692495 2564 feature_gate.go:328] unrecognized feature gate: GCPCustomAPIEndpoints Jul 29 19:17:02.693532 ip-10-0-131-93 kubenswrapper[2564]: W0729 19:17:02.692498 2564 feature_gate.go:328] unrecognized feature gate: ExternalOIDCWithUIDAndExtraClaimMappings Jul 29 19:17:02.693532 ip-10-0-131-93 kubenswrapper[2564]: W0729 19:17:02.692501 2564 feature_gate.go:328] unrecognized feature gate: NewOLMBoxCutterRuntime Jul 29 19:17:02.693532 ip-10-0-131-93 kubenswrapper[2564]: W0729 19:17:02.692504 2564 feature_gate.go:328] unrecognized feature gate: CRDCompatibilityRequirementOperator Jul 29 19:17:02.693532 ip-10-0-131-93 kubenswrapper[2564]: W0729 19:17:02.692506 2564 feature_gate.go:328] unrecognized feature gate: InsightsConfig Jul 29 19:17:02.693532 ip-10-0-131-93 kubenswrapper[2564]: W0729 19:17:02.692509 2564 feature_gate.go:328] unrecognized feature gate: IrreconcilableMachineConfig Jul 29 19:17:02.693532 ip-10-0-131-93 kubenswrapper[2564]: W0729 19:17:02.692511 2564 feature_gate.go:328] unrecognized feature gate: VSphereMultiNetworks Jul 29 19:17:02.693532 ip-10-0-131-93 kubenswrapper[2564]: W0729 19:17:02.692513 2564 feature_gate.go:328] unrecognized feature gate: CPMSMachineNamePrefix Jul 29 19:17:02.693532 ip-10-0-131-93 kubenswrapper[2564]: W0729 19:17:02.692516 2564 feature_gate.go:328] unrecognized feature gate: ClusterAPIInstall Jul 29 19:17:02.693532 ip-10-0-131-93 kubenswrapper[2564]: W0729 19:17:02.692518 2564 feature_gate.go:328] unrecognized feature gate: NewOLM Jul 29 19:17:02.693532 ip-10-0-131-93 kubenswrapper[2564]: W0729 19:17:02.692521 2564 feature_gate.go:328] unrecognized feature gate: KMSEncryption Jul 29 19:17:02.693532 ip-10-0-131-93 kubenswrapper[2564]: W0729 19:17:02.692523 2564 feature_gate.go:328] unrecognized feature gate: VolumeGroupSnapshot Jul 29 19:17:02.693532 ip-10-0-131-93 kubenswrapper[2564]: W0729 19:17:02.692528 2564 feature_gate.go:349] Setting deprecated feature gate KMSv1=true. It will be removed in a future release. Jul 29 19:17:02.693532 ip-10-0-131-93 kubenswrapper[2564]: W0729 19:17:02.692532 2564 feature_gate.go:328] unrecognized feature gate: NewOLMWebhookProviderOpenshiftServiceCA Jul 29 19:17:02.693532 ip-10-0-131-93 kubenswrapper[2564]: W0729 19:17:02.692535 2564 feature_gate.go:328] unrecognized feature gate: BootcNodeManagement Jul 29 19:17:02.693532 ip-10-0-131-93 kubenswrapper[2564]: W0729 19:17:02.692538 2564 feature_gate.go:328] unrecognized feature gate: VSphereHostVMGroupZonal Jul 29 19:17:02.693532 ip-10-0-131-93 kubenswrapper[2564]: W0729 19:17:02.692541 2564 feature_gate.go:328] unrecognized feature gate: InsightsOnDemandDataGather Jul 29 19:17:02.694223 ip-10-0-131-93 kubenswrapper[2564]: W0729 19:17:02.692544 2564 feature_gate.go:328] unrecognized feature gate: NetworkLiveMigration Jul 29 19:17:02.694223 ip-10-0-131-93 kubenswrapper[2564]: W0729 19:17:02.692546 2564 feature_gate.go:328] unrecognized feature gate: ImageStreamImportMode Jul 29 19:17:02.694223 ip-10-0-131-93 kubenswrapper[2564]: W0729 19:17:02.692549 2564 feature_gate.go:328] unrecognized feature gate: MultiArchInstallAzure Jul 29 19:17:02.694223 ip-10-0-131-93 kubenswrapper[2564]: W0729 19:17:02.692552 2564 feature_gate.go:328] unrecognized feature gate: AzureMultiDisk Jul 29 19:17:02.694223 ip-10-0-131-93 kubenswrapper[2564]: W0729 19:17:02.692554 2564 feature_gate.go:328] unrecognized feature gate: VSphereMixedNodeEnv Jul 29 19:17:02.694223 ip-10-0-131-93 kubenswrapper[2564]: W0729 19:17:02.692557 2564 feature_gate.go:328] unrecognized feature gate: AWSClusterHostedDNS Jul 29 19:17:02.694223 ip-10-0-131-93 kubenswrapper[2564]: W0729 19:17:02.692559 2564 feature_gate.go:328] unrecognized feature gate: Example2 Jul 29 19:17:02.694223 ip-10-0-131-93 kubenswrapper[2564]: W0729 19:17:02.692562 2564 feature_gate.go:328] unrecognized feature gate: NewOLMOwnSingleNamespace Jul 29 19:17:02.694223 ip-10-0-131-93 kubenswrapper[2564]: W0729 19:17:02.692564 2564 feature_gate.go:328] unrecognized feature gate: KMSEncryptionProvider Jul 29 19:17:02.694223 ip-10-0-131-93 kubenswrapper[2564]: W0729 19:17:02.692568 2564 feature_gate.go:328] unrecognized feature gate: MachineConfigNodes Jul 29 19:17:02.694223 ip-10-0-131-93 kubenswrapper[2564]: W0729 19:17:02.692570 2564 feature_gate.go:328] unrecognized feature gate: IngressControllerDynamicConfigurationManager Jul 29 19:17:02.694223 ip-10-0-131-93 kubenswrapper[2564]: W0729 19:17:02.692573 2564 feature_gate.go:328] unrecognized feature gate: OSStreams Jul 29 19:17:02.694223 ip-10-0-131-93 kubenswrapper[2564]: W0729 19:17:02.692576 2564 feature_gate.go:328] unrecognized feature gate: AzureDualStackInstall Jul 29 19:17:02.694223 ip-10-0-131-93 kubenswrapper[2564]: W0729 19:17:02.692579 2564 feature_gate.go:328] unrecognized feature gate: AWSClusterHostedDNSInstall Jul 29 19:17:02.694223 ip-10-0-131-93 kubenswrapper[2564]: W0729 19:17:02.692581 2564 feature_gate.go:328] unrecognized feature gate: MultiDiskSetup Jul 29 19:17:02.694223 ip-10-0-131-93 kubenswrapper[2564]: W0729 19:17:02.692584 2564 feature_gate.go:328] unrecognized feature gate: SigstoreImageVerification Jul 29 19:17:02.694223 ip-10-0-131-93 kubenswrapper[2564]: W0729 19:17:02.692587 2564 feature_gate.go:328] unrecognized feature gate: ClusterAPIMachineManagementVSphere Jul 29 19:17:02.694223 ip-10-0-131-93 kubenswrapper[2564]: W0729 19:17:02.692589 2564 feature_gate.go:328] unrecognized feature gate: GatewayAPI Jul 29 19:17:02.694223 ip-10-0-131-93 kubenswrapper[2564]: W0729 19:17:02.692592 2564 feature_gate.go:328] unrecognized feature gate: GCPClusterHostedDNSInstall Jul 29 19:17:02.694223 ip-10-0-131-93 kubenswrapper[2564]: W0729 19:17:02.692594 2564 feature_gate.go:328] unrecognized feature gate: AzureWorkloadIdentity Jul 29 19:17:02.694759 ip-10-0-131-93 kubenswrapper[2564]: W0729 19:17:02.692597 2564 feature_gate.go:328] unrecognized feature gate: ProvisioningRequestAvailable Jul 29 19:17:02.694759 ip-10-0-131-93 kubenswrapper[2564]: W0729 19:17:02.692599 2564 feature_gate.go:328] unrecognized feature gate: BuildCSIVolumes Jul 29 19:17:02.694759 ip-10-0-131-93 kubenswrapper[2564]: W0729 19:17:02.692602 2564 feature_gate.go:328] unrecognized feature gate: MachineAPIMigration Jul 29 19:17:02.694759 ip-10-0-131-93 kubenswrapper[2564]: W0729 19:17:02.692604 2564 feature_gate.go:328] unrecognized feature gate: GCPClusterHostedDNS Jul 29 19:17:02.694759 ip-10-0-131-93 kubenswrapper[2564]: W0729 19:17:02.692607 2564 feature_gate.go:328] unrecognized feature gate: ClusterMonitoringConfig Jul 29 19:17:02.694759 ip-10-0-131-93 kubenswrapper[2564]: W0729 19:17:02.692609 2564 feature_gate.go:328] unrecognized feature gate: ExternalSnapshotMetadata Jul 29 19:17:02.694759 ip-10-0-131-93 kubenswrapper[2564]: W0729 19:17:02.692612 2564 feature_gate.go:328] unrecognized feature gate: NewOLMCatalogdAPIV1Metas Jul 29 19:17:02.694759 ip-10-0-131-93 kubenswrapper[2564]: W0729 19:17:02.692614 2564 feature_gate.go:328] unrecognized feature gate: OpenShiftPodSecurityAdmission Jul 29 19:17:02.694759 ip-10-0-131-93 kubenswrapper[2564]: W0729 19:17:02.692618 2564 feature_gate.go:328] unrecognized feature gate: EtcdBackendQuota Jul 29 19:17:02.694759 ip-10-0-131-93 kubenswrapper[2564]: W0729 19:17:02.692621 2564 feature_gate.go:328] unrecognized feature gate: PinnedImages Jul 29 19:17:02.694759 ip-10-0-131-93 kubenswrapper[2564]: W0729 19:17:02.692623 2564 feature_gate.go:328] unrecognized feature gate: VSphereMultiDisk Jul 29 19:17:02.694759 ip-10-0-131-93 kubenswrapper[2564]: W0729 19:17:02.692626 2564 feature_gate.go:328] unrecognized feature gate: NutanixMultiSubnets Jul 29 19:17:02.694759 ip-10-0-131-93 kubenswrapper[2564]: W0729 19:17:02.692628 2564 feature_gate.go:328] unrecognized feature gate: AzureClusterHostedDNSInstall Jul 29 19:17:02.694759 ip-10-0-131-93 kubenswrapper[2564]: W0729 19:17:02.692631 2564 feature_gate.go:328] unrecognized feature gate: GatewayAPIController Jul 29 19:17:02.694759 ip-10-0-131-93 kubenswrapper[2564]: W0729 19:17:02.692635 2564 feature_gate.go:351] Setting GA feature gate VolumeAttributesClass=true. It will be removed in a future release. Jul 29 19:17:02.694759 ip-10-0-131-93 kubenswrapper[2564]: W0729 19:17:02.692638 2564 feature_gate.go:328] unrecognized feature gate: HighlyAvailableArbiter Jul 29 19:17:02.694759 ip-10-0-131-93 kubenswrapper[2564]: W0729 19:17:02.692641 2564 feature_gate.go:328] unrecognized feature gate: DNSNameResolver Jul 29 19:17:02.694759 ip-10-0-131-93 kubenswrapper[2564]: W0729 19:17:02.692644 2564 feature_gate.go:328] unrecognized feature gate: Example Jul 29 19:17:02.694759 ip-10-0-131-93 kubenswrapper[2564]: W0729 19:17:02.692647 2564 feature_gate.go:328] unrecognized feature gate: SigstoreImageVerificationPKI Jul 29 19:17:02.695572 ip-10-0-131-93 kubenswrapper[2564]: W0729 19:17:02.695554 2564 feature_gate.go:328] unrecognized feature gate: ClusterMonitoringConfig Jul 29 19:17:02.695572 ip-10-0-131-93 kubenswrapper[2564]: W0729 19:17:02.695571 2564 feature_gate.go:328] unrecognized feature gate: NetworkLiveMigration Jul 29 19:17:02.695627 ip-10-0-131-93 kubenswrapper[2564]: W0729 19:17:02.695576 2564 feature_gate.go:328] unrecognized feature gate: ConsolePluginContentSecurityPolicy Jul 29 19:17:02.695627 ip-10-0-131-93 kubenswrapper[2564]: W0729 19:17:02.695579 2564 feature_gate.go:328] unrecognized feature gate: Example Jul 29 19:17:02.695627 ip-10-0-131-93 kubenswrapper[2564]: W0729 19:17:02.695582 2564 feature_gate.go:328] unrecognized feature gate: MachineAPIOperatorDisableMachineHealthCheckController Jul 29 19:17:02.695627 ip-10-0-131-93 kubenswrapper[2564]: W0729 19:17:02.695585 2564 feature_gate.go:328] unrecognized feature gate: HyperShiftOnlyDynamicResourceAllocation Jul 29 19:17:02.695627 ip-10-0-131-93 kubenswrapper[2564]: W0729 19:17:02.695588 2564 feature_gate.go:328] unrecognized feature gate: VSphereMultiDisk Jul 29 19:17:02.695627 ip-10-0-131-93 kubenswrapper[2564]: W0729 19:17:02.695591 2564 feature_gate.go:328] unrecognized feature gate: ClusterVersionOperatorConfiguration Jul 29 19:17:02.695627 ip-10-0-131-93 kubenswrapper[2564]: W0729 19:17:02.695593 2564 feature_gate.go:328] unrecognized feature gate: AzureClusterHostedDNSInstall Jul 29 19:17:02.695627 ip-10-0-131-93 kubenswrapper[2564]: W0729 19:17:02.695597 2564 feature_gate.go:328] unrecognized feature gate: GCPClusterHostedDNSInstall Jul 29 19:17:02.695627 ip-10-0-131-93 kubenswrapper[2564]: W0729 19:17:02.695601 2564 feature_gate.go:328] unrecognized feature gate: NewOLMWebhookProviderOpenshiftServiceCA Jul 29 19:17:02.695627 ip-10-0-131-93 kubenswrapper[2564]: W0729 19:17:02.695603 2564 feature_gate.go:328] unrecognized feature gate: NewOLM Jul 29 19:17:02.695627 ip-10-0-131-93 kubenswrapper[2564]: W0729 19:17:02.695606 2564 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesvSphere Jul 29 19:17:02.695627 ip-10-0-131-93 kubenswrapper[2564]: W0729 19:17:02.695610 2564 feature_gate.go:349] Setting deprecated feature gate KMSv1=true. It will be removed in a future release. Jul 29 19:17:02.695627 ip-10-0-131-93 kubenswrapper[2564]: W0729 19:17:02.695614 2564 feature_gate.go:328] unrecognized feature gate: AdminNetworkPolicy Jul 29 19:17:02.695627 ip-10-0-131-93 kubenswrapper[2564]: W0729 19:17:02.695617 2564 feature_gate.go:328] unrecognized feature gate: NewOLMOwnSingleNamespace Jul 29 19:17:02.695627 ip-10-0-131-93 kubenswrapper[2564]: W0729 19:17:02.695620 2564 feature_gate.go:328] unrecognized feature gate: GatewayAPI Jul 29 19:17:02.695627 ip-10-0-131-93 kubenswrapper[2564]: W0729 19:17:02.695622 2564 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesAWS Jul 29 19:17:02.695627 ip-10-0-131-93 kubenswrapper[2564]: W0729 19:17:02.695625 2564 feature_gate.go:328] unrecognized feature gate: OnPremDNSRecords Jul 29 19:17:02.695627 ip-10-0-131-93 kubenswrapper[2564]: W0729 19:17:02.695628 2564 feature_gate.go:328] unrecognized feature gate: AlibabaPlatform Jul 29 19:17:02.695627 ip-10-0-131-93 kubenswrapper[2564]: W0729 19:17:02.695631 2564 feature_gate.go:328] unrecognized feature gate: VSphereMixedNodeEnv Jul 29 19:17:02.696095 ip-10-0-131-93 kubenswrapper[2564]: W0729 19:17:02.695633 2564 feature_gate.go:328] unrecognized feature gate: ProvisioningRequestAvailable Jul 29 19:17:02.696095 ip-10-0-131-93 kubenswrapper[2564]: W0729 19:17:02.695637 2564 feature_gate.go:328] unrecognized feature gate: GCPClusterHostedDNS Jul 29 19:17:02.696095 ip-10-0-131-93 kubenswrapper[2564]: W0729 19:17:02.695640 2564 feature_gate.go:328] unrecognized feature gate: AdditionalRoutingCapabilities Jul 29 19:17:02.696095 ip-10-0-131-93 kubenswrapper[2564]: W0729 19:17:02.695643 2564 feature_gate.go:328] unrecognized feature gate: NewOLMCatalogdAPIV1Metas Jul 29 19:17:02.696095 ip-10-0-131-93 kubenswrapper[2564]: W0729 19:17:02.695645 2564 feature_gate.go:328] unrecognized feature gate: VSphereHostVMGroupZonal Jul 29 19:17:02.696095 ip-10-0-131-93 kubenswrapper[2564]: W0729 19:17:02.695648 2564 feature_gate.go:328] unrecognized feature gate: AWSClusterHostedDNSInstall Jul 29 19:17:02.696095 ip-10-0-131-93 kubenswrapper[2564]: W0729 19:17:02.695651 2564 feature_gate.go:328] unrecognized feature gate: BuildCSIVolumes Jul 29 19:17:02.696095 ip-10-0-131-93 kubenswrapper[2564]: W0729 19:17:02.695653 2564 feature_gate.go:328] unrecognized feature gate: ShortCertRotation Jul 29 19:17:02.696095 ip-10-0-131-93 kubenswrapper[2564]: W0729 19:17:02.695656 2564 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesCPMS Jul 29 19:17:02.696095 ip-10-0-131-93 kubenswrapper[2564]: W0729 19:17:02.695658 2564 feature_gate.go:328] unrecognized feature gate: SigstoreImageVerificationPKI Jul 29 19:17:02.696095 ip-10-0-131-93 kubenswrapper[2564]: W0729 19:17:02.695668 2564 feature_gate.go:351] Setting GA feature gate VolumeAttributesClass=true. It will be removed in a future release. Jul 29 19:17:02.696095 ip-10-0-131-93 kubenswrapper[2564]: W0729 19:17:02.695672 2564 feature_gate.go:328] unrecognized feature gate: SigstoreImageVerification Jul 29 19:17:02.696095 ip-10-0-131-93 kubenswrapper[2564]: W0729 19:17:02.695675 2564 feature_gate.go:328] unrecognized feature gate: KMSEncryption Jul 29 19:17:02.696095 ip-10-0-131-93 kubenswrapper[2564]: W0729 19:17:02.695678 2564 feature_gate.go:328] unrecognized feature gate: Example2 Jul 29 19:17:02.696095 ip-10-0-131-93 kubenswrapper[2564]: W0729 19:17:02.695682 2564 feature_gate.go:351] Setting GA feature gate ServiceAccountTokenNodeBinding=true. It will be removed in a future release. Jul 29 19:17:02.696095 ip-10-0-131-93 kubenswrapper[2564]: W0729 19:17:02.695685 2564 feature_gate.go:328] unrecognized feature gate: SignatureStores Jul 29 19:17:02.696095 ip-10-0-131-93 kubenswrapper[2564]: W0729 19:17:02.695688 2564 feature_gate.go:328] unrecognized feature gate: MultiDiskSetup Jul 29 19:17:02.696095 ip-10-0-131-93 kubenswrapper[2564]: W0729 19:17:02.695691 2564 feature_gate.go:328] unrecognized feature gate: NetworkDiagnosticsConfig Jul 29 19:17:02.696095 ip-10-0-131-93 kubenswrapper[2564]: W0729 19:17:02.695693 2564 feature_gate.go:328] unrecognized feature gate: InsightsOnDemandDataGather Jul 29 19:17:02.696577 ip-10-0-131-93 kubenswrapper[2564]: W0729 19:17:02.695696 2564 feature_gate.go:328] unrecognized feature gate: VSphereConfigurableMaxAllowedBlockVolumesPerNode Jul 29 19:17:02.696577 ip-10-0-131-93 kubenswrapper[2564]: W0729 19:17:02.695699 2564 feature_gate.go:328] unrecognized feature gate: AutomatedEtcdBackup Jul 29 19:17:02.696577 ip-10-0-131-93 kubenswrapper[2564]: W0729 19:17:02.695701 2564 feature_gate.go:328] unrecognized feature gate: ClusterAPIInstall Jul 29 19:17:02.696577 ip-10-0-131-93 kubenswrapper[2564]: W0729 19:17:02.695704 2564 feature_gate.go:328] unrecognized feature gate: AWSDedicatedHosts Jul 29 19:17:02.696577 ip-10-0-131-93 kubenswrapper[2564]: W0729 19:17:02.695706 2564 feature_gate.go:328] unrecognized feature gate: NutanixMultiSubnets Jul 29 19:17:02.696577 ip-10-0-131-93 kubenswrapper[2564]: W0729 19:17:02.695709 2564 feature_gate.go:328] unrecognized feature gate: GCPCustomAPIEndpointsInstall Jul 29 19:17:02.696577 ip-10-0-131-93 kubenswrapper[2564]: W0729 19:17:02.695711 2564 feature_gate.go:328] unrecognized feature gate: AWSClusterHostedDNS Jul 29 19:17:02.696577 ip-10-0-131-93 kubenswrapper[2564]: W0729 19:17:02.695714 2564 feature_gate.go:328] unrecognized feature gate: NetworkSegmentation Jul 29 19:17:02.696577 ip-10-0-131-93 kubenswrapper[2564]: W0729 19:17:02.695716 2564 feature_gate.go:328] unrecognized feature gate: MachineAPIMigration Jul 29 19:17:02.696577 ip-10-0-131-93 kubenswrapper[2564]: W0729 19:17:02.695719 2564 feature_gate.go:328] unrecognized feature gate: EtcdBackendQuota Jul 29 19:17:02.696577 ip-10-0-131-93 kubenswrapper[2564]: W0729 19:17:02.695722 2564 feature_gate.go:328] unrecognized feature gate: PinnedImages Jul 29 19:17:02.696577 ip-10-0-131-93 kubenswrapper[2564]: W0729 19:17:02.695725 2564 feature_gate.go:328] unrecognized feature gate: DyanmicServiceEndpointIBMCloud Jul 29 19:17:02.696577 ip-10-0-131-93 kubenswrapper[2564]: W0729 19:17:02.695727 2564 feature_gate.go:328] unrecognized feature gate: BootcNodeManagement Jul 29 19:17:02.696577 ip-10-0-131-93 kubenswrapper[2564]: W0729 19:17:02.695729 2564 feature_gate.go:328] unrecognized feature gate: InsightsConfig Jul 29 19:17:02.696577 ip-10-0-131-93 kubenswrapper[2564]: W0729 19:17:02.695733 2564 feature_gate.go:328] unrecognized feature gate: AzureDualStackInstall Jul 29 19:17:02.696577 ip-10-0-131-93 kubenswrapper[2564]: W0729 19:17:02.695735 2564 feature_gate.go:328] unrecognized feature gate: VSphereMultiNetworks Jul 29 19:17:02.696577 ip-10-0-131-93 kubenswrapper[2564]: W0729 19:17:02.695738 2564 feature_gate.go:328] unrecognized feature gate: MachineConfigNodes Jul 29 19:17:02.696577 ip-10-0-131-93 kubenswrapper[2564]: W0729 19:17:02.695740 2564 feature_gate.go:328] unrecognized feature gate: IngressControllerDynamicConfigurationManager Jul 29 19:17:02.696577 ip-10-0-131-93 kubenswrapper[2564]: W0729 19:17:02.695743 2564 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesAzure Jul 29 19:17:02.696577 ip-10-0-131-93 kubenswrapper[2564]: W0729 19:17:02.695745 2564 feature_gate.go:328] unrecognized feature gate: GCPDualStackInstall Jul 29 19:17:02.697065 ip-10-0-131-93 kubenswrapper[2564]: W0729 19:17:02.695748 2564 feature_gate.go:328] unrecognized feature gate: DNSNameResolver Jul 29 19:17:02.697065 ip-10-0-131-93 kubenswrapper[2564]: W0729 19:17:02.695751 2564 feature_gate.go:328] unrecognized feature gate: ImageModeStatusReporting Jul 29 19:17:02.697065 ip-10-0-131-93 kubenswrapper[2564]: W0729 19:17:02.695753 2564 feature_gate.go:328] unrecognized feature gate: OpenShiftPodSecurityAdmission Jul 29 19:17:02.697065 ip-10-0-131-93 kubenswrapper[2564]: W0729 19:17:02.695755 2564 feature_gate.go:328] unrecognized feature gate: GCPCustomAPIEndpoints Jul 29 19:17:02.697065 ip-10-0-131-93 kubenswrapper[2564]: W0729 19:17:02.695758 2564 feature_gate.go:328] unrecognized feature gate: NewOLMBoxCutterRuntime Jul 29 19:17:02.697065 ip-10-0-131-93 kubenswrapper[2564]: W0729 19:17:02.695760 2564 feature_gate.go:328] unrecognized feature gate: MultiArchInstallAzure Jul 29 19:17:02.697065 ip-10-0-131-93 kubenswrapper[2564]: W0729 19:17:02.695763 2564 feature_gate.go:328] unrecognized feature gate: OVNObservability Jul 29 19:17:02.697065 ip-10-0-131-93 kubenswrapper[2564]: W0729 19:17:02.695766 2564 feature_gate.go:328] unrecognized feature gate: MixedCPUsAllocation Jul 29 19:17:02.697065 ip-10-0-131-93 kubenswrapper[2564]: W0729 19:17:02.695769 2564 feature_gate.go:328] unrecognized feature gate: AzureDedicatedHosts Jul 29 19:17:02.697065 ip-10-0-131-93 kubenswrapper[2564]: W0729 19:17:02.695771 2564 feature_gate.go:328] unrecognized feature gate: NewOLMPreflightPermissionChecks Jul 29 19:17:02.697065 ip-10-0-131-93 kubenswrapper[2564]: W0729 19:17:02.695773 2564 feature_gate.go:328] unrecognized feature gate: HighlyAvailableArbiter Jul 29 19:17:02.697065 ip-10-0-131-93 kubenswrapper[2564]: W0729 19:17:02.695776 2564 feature_gate.go:328] unrecognized feature gate: GatewayAPIWithoutOLM Jul 29 19:17:02.697065 ip-10-0-131-93 kubenswrapper[2564]: W0729 19:17:02.695779 2564 feature_gate.go:328] unrecognized feature gate: KMSEncryptionProvider Jul 29 19:17:02.697065 ip-10-0-131-93 kubenswrapper[2564]: W0729 19:17:02.695781 2564 feature_gate.go:328] unrecognized feature gate: EventTTL Jul 29 19:17:02.697065 ip-10-0-131-93 kubenswrapper[2564]: W0729 19:17:02.695783 2564 feature_gate.go:328] unrecognized feature gate: RouteAdvertisements Jul 29 19:17:02.697065 ip-10-0-131-93 kubenswrapper[2564]: W0729 19:17:02.695786 2564 feature_gate.go:328] unrecognized feature gate: ExternalSnapshotMetadata Jul 29 19:17:02.697065 ip-10-0-131-93 kubenswrapper[2564]: W0729 19:17:02.695789 2564 feature_gate.go:328] unrecognized feature gate: IrreconcilableMachineConfig Jul 29 19:17:02.697065 ip-10-0-131-93 kubenswrapper[2564]: W0729 19:17:02.695792 2564 feature_gate.go:328] unrecognized feature gate: AWSServiceLBNetworkSecurityGroup Jul 29 19:17:02.697065 ip-10-0-131-93 kubenswrapper[2564]: W0729 19:17:02.695794 2564 feature_gate.go:328] unrecognized feature gate: OSStreams Jul 29 19:17:02.697065 ip-10-0-131-93 kubenswrapper[2564]: W0729 19:17:02.695797 2564 feature_gate.go:328] unrecognized feature gate: VolumeGroupSnapshot Jul 29 19:17:02.697639 ip-10-0-131-93 kubenswrapper[2564]: W0729 19:17:02.695799 2564 feature_gate.go:328] unrecognized feature gate: PreconfiguredUDNAddresses Jul 29 19:17:02.697639 ip-10-0-131-93 kubenswrapper[2564]: W0729 19:17:02.695802 2564 feature_gate.go:328] unrecognized feature gate: ExternalOIDC Jul 29 19:17:02.697639 ip-10-0-131-93 kubenswrapper[2564]: W0729 19:17:02.695804 2564 feature_gate.go:328] unrecognized feature gate: BootImageSkewEnforcement Jul 29 19:17:02.697639 ip-10-0-131-93 kubenswrapper[2564]: W0729 19:17:02.695807 2564 feature_gate.go:328] unrecognized feature gate: GatewayAPIController Jul 29 19:17:02.697639 ip-10-0-131-93 kubenswrapper[2564]: W0729 19:17:02.695809 2564 feature_gate.go:328] unrecognized feature gate: UpgradeStatus Jul 29 19:17:02.697639 ip-10-0-131-93 kubenswrapper[2564]: W0729 19:17:02.695812 2564 feature_gate.go:328] unrecognized feature gate: AzureWorkloadIdentity Jul 29 19:17:02.697639 ip-10-0-131-93 kubenswrapper[2564]: W0729 19:17:02.695815 2564 feature_gate.go:328] unrecognized feature gate: ClusterAPIMachineManagement Jul 29 19:17:02.697639 ip-10-0-131-93 kubenswrapper[2564]: W0729 19:17:02.695818 2564 feature_gate.go:328] unrecognized feature gate: ClusterAPIMachineManagementVSphere Jul 29 19:17:02.697639 ip-10-0-131-93 kubenswrapper[2564]: W0729 19:17:02.695820 2564 feature_gate.go:328] unrecognized feature gate: ManagedBootImages Jul 29 19:17:02.697639 ip-10-0-131-93 kubenswrapper[2564]: W0729 19:17:02.695823 2564 feature_gate.go:328] unrecognized feature gate: ExternalOIDCWithUIDAndExtraClaimMappings Jul 29 19:17:02.697639 ip-10-0-131-93 kubenswrapper[2564]: W0729 19:17:02.695826 2564 feature_gate.go:328] unrecognized feature gate: ImageStreamImportMode Jul 29 19:17:02.697639 ip-10-0-131-93 kubenswrapper[2564]: W0729 19:17:02.695828 2564 feature_gate.go:328] unrecognized feature gate: AWSDualStackInstall Jul 29 19:17:02.697639 ip-10-0-131-93 kubenswrapper[2564]: W0729 19:17:02.695831 2564 feature_gate.go:328] unrecognized feature gate: CPMSMachineNamePrefix Jul 29 19:17:02.697639 ip-10-0-131-93 kubenswrapper[2564]: W0729 19:17:02.695833 2564 feature_gate.go:328] unrecognized feature gate: DualReplica Jul 29 19:17:02.697639 ip-10-0-131-93 kubenswrapper[2564]: W0729 19:17:02.695836 2564 feature_gate.go:328] unrecognized feature gate: ClusterAPIInstallIBMCloud Jul 29 19:17:02.697639 ip-10-0-131-93 kubenswrapper[2564]: W0729 19:17:02.695843 2564 feature_gate.go:328] unrecognized feature gate: CRDCompatibilityRequirementOperator Jul 29 19:17:02.697639 ip-10-0-131-93 kubenswrapper[2564]: W0729 19:17:02.695845 2564 feature_gate.go:328] unrecognized feature gate: AzureMultiDisk Jul 29 19:17:02.697639 ip-10-0-131-93 kubenswrapper[2564]: W0729 19:17:02.695848 2564 feature_gate.go:328] unrecognized feature gate: MetricsCollectionProfiles Jul 29 19:17:02.697639 ip-10-0-131-93 kubenswrapper[2564]: W0729 19:17:02.695850 2564 feature_gate.go:328] unrecognized feature gate: NoRegistryClusterInstall Jul 29 19:17:02.697639 ip-10-0-131-93 kubenswrapper[2564]: W0729 19:17:02.696244 2564 feature_gate.go:328] unrecognized feature gate: GCPClusterHostedDNS Jul 29 19:17:02.698149 ip-10-0-131-93 kubenswrapper[2564]: W0729 19:17:02.696250 2564 feature_gate.go:328] unrecognized feature gate: NutanixMultiSubnets Jul 29 19:17:02.698149 ip-10-0-131-93 kubenswrapper[2564]: W0729 19:17:02.696255 2564 feature_gate.go:349] Setting deprecated feature gate KMSv1=true. It will be removed in a future release. Jul 29 19:17:02.698149 ip-10-0-131-93 kubenswrapper[2564]: W0729 19:17:02.696259 2564 feature_gate.go:328] unrecognized feature gate: AutomatedEtcdBackup Jul 29 19:17:02.698149 ip-10-0-131-93 kubenswrapper[2564]: W0729 19:17:02.696262 2564 feature_gate.go:328] unrecognized feature gate: AdditionalRoutingCapabilities Jul 29 19:17:02.698149 ip-10-0-131-93 kubenswrapper[2564]: W0729 19:17:02.696264 2564 feature_gate.go:328] unrecognized feature gate: AWSServiceLBNetworkSecurityGroup Jul 29 19:17:02.698149 ip-10-0-131-93 kubenswrapper[2564]: W0729 19:17:02.696267 2564 feature_gate.go:328] unrecognized feature gate: NewOLMPreflightPermissionChecks Jul 29 19:17:02.698149 ip-10-0-131-93 kubenswrapper[2564]: W0729 19:17:02.696270 2564 feature_gate.go:328] unrecognized feature gate: MachineAPIOperatorDisableMachineHealthCheckController Jul 29 19:17:02.698149 ip-10-0-131-93 kubenswrapper[2564]: W0729 19:17:02.696272 2564 feature_gate.go:328] unrecognized feature gate: KMSEncryption Jul 29 19:17:02.698149 ip-10-0-131-93 kubenswrapper[2564]: W0729 19:17:02.696275 2564 feature_gate.go:328] unrecognized feature gate: AWSClusterHostedDNSInstall Jul 29 19:17:02.698149 ip-10-0-131-93 kubenswrapper[2564]: W0729 19:17:02.696277 2564 feature_gate.go:328] unrecognized feature gate: MetricsCollectionProfiles Jul 29 19:17:02.698149 ip-10-0-131-93 kubenswrapper[2564]: W0729 19:17:02.696280 2564 feature_gate.go:328] unrecognized feature gate: AzureDedicatedHosts Jul 29 19:17:02.698149 ip-10-0-131-93 kubenswrapper[2564]: W0729 19:17:02.696282 2564 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesCPMS Jul 29 19:17:02.698149 ip-10-0-131-93 kubenswrapper[2564]: W0729 19:17:02.696284 2564 feature_gate.go:328] unrecognized feature gate: NetworkSegmentation Jul 29 19:17:02.698149 ip-10-0-131-93 kubenswrapper[2564]: W0729 19:17:02.696287 2564 feature_gate.go:328] unrecognized feature gate: MultiDiskSetup Jul 29 19:17:02.698149 ip-10-0-131-93 kubenswrapper[2564]: W0729 19:17:02.696289 2564 feature_gate.go:328] unrecognized feature gate: NetworkDiagnosticsConfig Jul 29 19:17:02.698149 ip-10-0-131-93 kubenswrapper[2564]: W0729 19:17:02.696292 2564 feature_gate.go:328] unrecognized feature gate: CRDCompatibilityRequirementOperator Jul 29 19:17:02.698149 ip-10-0-131-93 kubenswrapper[2564]: W0729 19:17:02.696295 2564 feature_gate.go:328] unrecognized feature gate: AdminNetworkPolicy Jul 29 19:17:02.698149 ip-10-0-131-93 kubenswrapper[2564]: W0729 19:17:02.696298 2564 feature_gate.go:328] unrecognized feature gate: NewOLMOwnSingleNamespace Jul 29 19:17:02.698149 ip-10-0-131-93 kubenswrapper[2564]: W0729 19:17:02.696301 2564 feature_gate.go:328] unrecognized feature gate: AWSDualStackInstall Jul 29 19:17:02.698624 ip-10-0-131-93 kubenswrapper[2564]: W0729 19:17:02.696304 2564 feature_gate.go:328] unrecognized feature gate: OSStreams Jul 29 19:17:02.698624 ip-10-0-131-93 kubenswrapper[2564]: W0729 19:17:02.696307 2564 feature_gate.go:328] unrecognized feature gate: MixedCPUsAllocation Jul 29 19:17:02.698624 ip-10-0-131-93 kubenswrapper[2564]: W0729 19:17:02.696310 2564 feature_gate.go:328] unrecognized feature gate: BuildCSIVolumes Jul 29 19:17:02.698624 ip-10-0-131-93 kubenswrapper[2564]: W0729 19:17:02.696312 2564 feature_gate.go:328] unrecognized feature gate: ShortCertRotation Jul 29 19:17:02.698624 ip-10-0-131-93 kubenswrapper[2564]: W0729 19:17:02.696315 2564 feature_gate.go:328] unrecognized feature gate: UpgradeStatus Jul 29 19:17:02.698624 ip-10-0-131-93 kubenswrapper[2564]: W0729 19:17:02.696317 2564 feature_gate.go:328] unrecognized feature gate: AzureMultiDisk Jul 29 19:17:02.698624 ip-10-0-131-93 kubenswrapper[2564]: W0729 19:17:02.696320 2564 feature_gate.go:328] unrecognized feature gate: ExternalOIDCWithUIDAndExtraClaimMappings Jul 29 19:17:02.698624 ip-10-0-131-93 kubenswrapper[2564]: W0729 19:17:02.696322 2564 feature_gate.go:328] unrecognized feature gate: BootcNodeManagement Jul 29 19:17:02.698624 ip-10-0-131-93 kubenswrapper[2564]: W0729 19:17:02.696334 2564 feature_gate.go:328] unrecognized feature gate: ClusterMonitoringConfig Jul 29 19:17:02.698624 ip-10-0-131-93 kubenswrapper[2564]: W0729 19:17:02.696338 2564 feature_gate.go:328] unrecognized feature gate: Example2 Jul 29 19:17:02.698624 ip-10-0-131-93 kubenswrapper[2564]: W0729 19:17:02.696342 2564 feature_gate.go:328] unrecognized feature gate: HyperShiftOnlyDynamicResourceAllocation Jul 29 19:17:02.698624 ip-10-0-131-93 kubenswrapper[2564]: W0729 19:17:02.696347 2564 feature_gate.go:328] unrecognized feature gate: DyanmicServiceEndpointIBMCloud Jul 29 19:17:02.698624 ip-10-0-131-93 kubenswrapper[2564]: W0729 19:17:02.696351 2564 feature_gate.go:328] unrecognized feature gate: NewOLMCatalogdAPIV1Metas Jul 29 19:17:02.698624 ip-10-0-131-93 kubenswrapper[2564]: W0729 19:17:02.696354 2564 feature_gate.go:328] unrecognized feature gate: SignatureStores Jul 29 19:17:02.698624 ip-10-0-131-93 kubenswrapper[2564]: W0729 19:17:02.696358 2564 feature_gate.go:328] unrecognized feature gate: GatewayAPI Jul 29 19:17:02.698624 ip-10-0-131-93 kubenswrapper[2564]: W0729 19:17:02.696362 2564 feature_gate.go:328] unrecognized feature gate: ConsolePluginContentSecurityPolicy Jul 29 19:17:02.698624 ip-10-0-131-93 kubenswrapper[2564]: W0729 19:17:02.696365 2564 feature_gate.go:328] unrecognized feature gate: CPMSMachineNamePrefix Jul 29 19:17:02.698624 ip-10-0-131-93 kubenswrapper[2564]: W0729 19:17:02.696368 2564 feature_gate.go:328] unrecognized feature gate: VSphereConfigurableMaxAllowedBlockVolumesPerNode Jul 29 19:17:02.698624 ip-10-0-131-93 kubenswrapper[2564]: W0729 19:17:02.696370 2564 feature_gate.go:328] unrecognized feature gate: MachineConfigNodes Jul 29 19:17:02.698624 ip-10-0-131-93 kubenswrapper[2564]: W0729 19:17:02.696373 2564 feature_gate.go:328] unrecognized feature gate: NewOLM Jul 29 19:17:02.699154 ip-10-0-131-93 kubenswrapper[2564]: W0729 19:17:02.696375 2564 feature_gate.go:328] unrecognized feature gate: BootImageSkewEnforcement Jul 29 19:17:02.699154 ip-10-0-131-93 kubenswrapper[2564]: W0729 19:17:02.696378 2564 feature_gate.go:328] unrecognized feature gate: GCPCustomAPIEndpoints Jul 29 19:17:02.699154 ip-10-0-131-93 kubenswrapper[2564]: W0729 19:17:02.696380 2564 feature_gate.go:328] unrecognized feature gate: RouteAdvertisements Jul 29 19:17:02.699154 ip-10-0-131-93 kubenswrapper[2564]: W0729 19:17:02.696383 2564 feature_gate.go:328] unrecognized feature gate: HighlyAvailableArbiter Jul 29 19:17:02.699154 ip-10-0-131-93 kubenswrapper[2564]: W0729 19:17:02.696386 2564 feature_gate.go:328] unrecognized feature gate: ImageModeStatusReporting Jul 29 19:17:02.699154 ip-10-0-131-93 kubenswrapper[2564]: W0729 19:17:02.696388 2564 feature_gate.go:328] unrecognized feature gate: GCPClusterHostedDNSInstall Jul 29 19:17:02.699154 ip-10-0-131-93 kubenswrapper[2564]: W0729 19:17:02.696391 2564 feature_gate.go:328] unrecognized feature gate: EtcdBackendQuota Jul 29 19:17:02.699154 ip-10-0-131-93 kubenswrapper[2564]: W0729 19:17:02.696394 2564 feature_gate.go:328] unrecognized feature gate: SigstoreImageVerificationPKI Jul 29 19:17:02.699154 ip-10-0-131-93 kubenswrapper[2564]: W0729 19:17:02.696396 2564 feature_gate.go:328] unrecognized feature gate: VSphereMultiDisk Jul 29 19:17:02.699154 ip-10-0-131-93 kubenswrapper[2564]: W0729 19:17:02.696400 2564 feature_gate.go:328] unrecognized feature gate: InsightsOnDemandDataGather Jul 29 19:17:02.699154 ip-10-0-131-93 kubenswrapper[2564]: W0729 19:17:02.696402 2564 feature_gate.go:328] unrecognized feature gate: ClusterAPIInstall Jul 29 19:17:02.699154 ip-10-0-131-93 kubenswrapper[2564]: W0729 19:17:02.696406 2564 feature_gate.go:328] unrecognized feature gate: ClusterAPIMachineManagementVSphere Jul 29 19:17:02.699154 ip-10-0-131-93 kubenswrapper[2564]: W0729 19:17:02.696409 2564 feature_gate.go:328] unrecognized feature gate: VSphereMultiNetworks Jul 29 19:17:02.699154 ip-10-0-131-93 kubenswrapper[2564]: W0729 19:17:02.696411 2564 feature_gate.go:328] unrecognized feature gate: DualReplica Jul 29 19:17:02.699154 ip-10-0-131-93 kubenswrapper[2564]: W0729 19:17:02.696414 2564 feature_gate.go:328] unrecognized feature gate: NewOLMWebhookProviderOpenshiftServiceCA Jul 29 19:17:02.699154 ip-10-0-131-93 kubenswrapper[2564]: W0729 19:17:02.696417 2564 feature_gate.go:328] unrecognized feature gate: ClusterAPIMachineManagement Jul 29 19:17:02.699154 ip-10-0-131-93 kubenswrapper[2564]: W0729 19:17:02.696419 2564 feature_gate.go:328] unrecognized feature gate: DNSNameResolver Jul 29 19:17:02.699154 ip-10-0-131-93 kubenswrapper[2564]: W0729 19:17:02.696422 2564 feature_gate.go:328] unrecognized feature gate: VSphereHostVMGroupZonal Jul 29 19:17:02.699154 ip-10-0-131-93 kubenswrapper[2564]: W0729 19:17:02.696425 2564 feature_gate.go:328] unrecognized feature gate: ImageStreamImportMode Jul 29 19:17:02.699154 ip-10-0-131-93 kubenswrapper[2564]: W0729 19:17:02.696428 2564 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesAWS Jul 29 19:17:02.699638 ip-10-0-131-93 kubenswrapper[2564]: W0729 19:17:02.696430 2564 feature_gate.go:328] unrecognized feature gate: ClusterAPIInstallIBMCloud Jul 29 19:17:02.699638 ip-10-0-131-93 kubenswrapper[2564]: W0729 19:17:02.696433 2564 feature_gate.go:328] unrecognized feature gate: ExternalOIDC Jul 29 19:17:02.699638 ip-10-0-131-93 kubenswrapper[2564]: W0729 19:17:02.696435 2564 feature_gate.go:328] unrecognized feature gate: GatewayAPIWithoutOLM Jul 29 19:17:02.699638 ip-10-0-131-93 kubenswrapper[2564]: W0729 19:17:02.696438 2564 feature_gate.go:328] unrecognized feature gate: KMSEncryptionProvider Jul 29 19:17:02.699638 ip-10-0-131-93 kubenswrapper[2564]: W0729 19:17:02.696440 2564 feature_gate.go:328] unrecognized feature gate: VSphereMixedNodeEnv Jul 29 19:17:02.699638 ip-10-0-131-93 kubenswrapper[2564]: W0729 19:17:02.696442 2564 feature_gate.go:328] unrecognized feature gate: Example Jul 29 19:17:02.699638 ip-10-0-131-93 kubenswrapper[2564]: W0729 19:17:02.696445 2564 feature_gate.go:328] unrecognized feature gate: AzureDualStackInstall Jul 29 19:17:02.699638 ip-10-0-131-93 kubenswrapper[2564]: W0729 19:17:02.696448 2564 feature_gate.go:328] unrecognized feature gate: IrreconcilableMachineConfig Jul 29 19:17:02.699638 ip-10-0-131-93 kubenswrapper[2564]: W0729 19:17:02.696451 2564 feature_gate.go:328] unrecognized feature gate: ManagedBootImages Jul 29 19:17:02.699638 ip-10-0-131-93 kubenswrapper[2564]: W0729 19:17:02.696453 2564 feature_gate.go:328] unrecognized feature gate: ExternalSnapshotMetadata Jul 29 19:17:02.699638 ip-10-0-131-93 kubenswrapper[2564]: W0729 19:17:02.696456 2564 feature_gate.go:328] unrecognized feature gate: VolumeGroupSnapshot Jul 29 19:17:02.699638 ip-10-0-131-93 kubenswrapper[2564]: W0729 19:17:02.696458 2564 feature_gate.go:328] unrecognized feature gate: MultiArchInstallAzure Jul 29 19:17:02.699638 ip-10-0-131-93 kubenswrapper[2564]: W0729 19:17:02.696461 2564 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesAzure Jul 29 19:17:02.699638 ip-10-0-131-93 kubenswrapper[2564]: W0729 19:17:02.696463 2564 feature_gate.go:328] unrecognized feature gate: AlibabaPlatform Jul 29 19:17:02.699638 ip-10-0-131-93 kubenswrapper[2564]: W0729 19:17:02.696466 2564 feature_gate.go:328] unrecognized feature gate: EventTTL Jul 29 19:17:02.699638 ip-10-0-131-93 kubenswrapper[2564]: W0729 19:17:02.696468 2564 feature_gate.go:328] unrecognized feature gate: GCPCustomAPIEndpointsInstall Jul 29 19:17:02.699638 ip-10-0-131-93 kubenswrapper[2564]: W0729 19:17:02.696471 2564 feature_gate.go:328] unrecognized feature gate: OVNObservability Jul 29 19:17:02.699638 ip-10-0-131-93 kubenswrapper[2564]: W0729 19:17:02.696474 2564 feature_gate.go:328] unrecognized feature gate: AzureClusterHostedDNSInstall Jul 29 19:17:02.699638 ip-10-0-131-93 kubenswrapper[2564]: W0729 19:17:02.696476 2564 feature_gate.go:328] unrecognized feature gate: OnPremDNSRecords Jul 29 19:17:02.699638 ip-10-0-131-93 kubenswrapper[2564]: W0729 19:17:02.696479 2564 feature_gate.go:328] unrecognized feature gate: MachineAPIMigration Jul 29 19:17:02.700154 ip-10-0-131-93 kubenswrapper[2564]: W0729 19:17:02.696481 2564 feature_gate.go:328] unrecognized feature gate: AzureWorkloadIdentity Jul 29 19:17:02.700154 ip-10-0-131-93 kubenswrapper[2564]: W0729 19:17:02.696483 2564 feature_gate.go:328] unrecognized feature gate: PreconfiguredUDNAddresses Jul 29 19:17:02.700154 ip-10-0-131-93 kubenswrapper[2564]: W0729 19:17:02.696487 2564 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesvSphere Jul 29 19:17:02.700154 ip-10-0-131-93 kubenswrapper[2564]: W0729 19:17:02.696490 2564 feature_gate.go:328] unrecognized feature gate: OpenShiftPodSecurityAdmission Jul 29 19:17:02.700154 ip-10-0-131-93 kubenswrapper[2564]: W0729 19:17:02.696494 2564 feature_gate.go:328] unrecognized feature gate: PinnedImages Jul 29 19:17:02.700154 ip-10-0-131-93 kubenswrapper[2564]: W0729 19:17:02.696498 2564 feature_gate.go:351] Setting GA feature gate ServiceAccountTokenNodeBinding=true. It will be removed in a future release. Jul 29 19:17:02.700154 ip-10-0-131-93 kubenswrapper[2564]: W0729 19:17:02.696502 2564 feature_gate.go:328] unrecognized feature gate: GatewayAPIController Jul 29 19:17:02.700154 ip-10-0-131-93 kubenswrapper[2564]: W0729 19:17:02.696505 2564 feature_gate.go:328] unrecognized feature gate: InsightsConfig Jul 29 19:17:02.700154 ip-10-0-131-93 kubenswrapper[2564]: W0729 19:17:02.696507 2564 feature_gate.go:328] unrecognized feature gate: IngressControllerDynamicConfigurationManager Jul 29 19:17:02.700154 ip-10-0-131-93 kubenswrapper[2564]: W0729 19:17:02.696511 2564 feature_gate.go:328] unrecognized feature gate: AWSClusterHostedDNS Jul 29 19:17:02.700154 ip-10-0-131-93 kubenswrapper[2564]: W0729 19:17:02.696513 2564 feature_gate.go:328] unrecognized feature gate: GCPDualStackInstall Jul 29 19:17:02.700154 ip-10-0-131-93 kubenswrapper[2564]: W0729 19:17:02.696522 2564 feature_gate.go:328] unrecognized feature gate: SigstoreImageVerification Jul 29 19:17:02.700154 ip-10-0-131-93 kubenswrapper[2564]: W0729 19:17:02.696525 2564 feature_gate.go:328] unrecognized feature gate: ClusterVersionOperatorConfiguration Jul 29 19:17:02.700154 ip-10-0-131-93 kubenswrapper[2564]: W0729 19:17:02.696528 2564 feature_gate.go:328] unrecognized feature gate: AWSDedicatedHosts Jul 29 19:17:02.700154 ip-10-0-131-93 kubenswrapper[2564]: W0729 19:17:02.696531 2564 feature_gate.go:328] unrecognized feature gate: NoRegistryClusterInstall Jul 29 19:17:02.700154 ip-10-0-131-93 kubenswrapper[2564]: W0729 19:17:02.696534 2564 feature_gate.go:328] unrecognized feature gate: ProvisioningRequestAvailable Jul 29 19:17:02.700154 ip-10-0-131-93 kubenswrapper[2564]: W0729 19:17:02.696537 2564 feature_gate.go:328] unrecognized feature gate: NetworkLiveMigration Jul 29 19:17:02.700154 ip-10-0-131-93 kubenswrapper[2564]: W0729 19:17:02.696539 2564 feature_gate.go:328] unrecognized feature gate: NewOLMBoxCutterRuntime Jul 29 19:17:02.700154 ip-10-0-131-93 kubenswrapper[2564]: W0729 19:17:02.696544 2564 feature_gate.go:351] Setting GA feature gate VolumeAttributesClass=true. It will be removed in a future release. Jul 29 19:17:02.700611 ip-10-0-131-93 kubenswrapper[2564]: I0729 19:17:02.696610 2564 flags.go:64] FLAG: --address="0.0.0.0" Jul 29 19:17:02.700611 ip-10-0-131-93 kubenswrapper[2564]: I0729 19:17:02.696617 2564 flags.go:64] FLAG: --allowed-unsafe-sysctls="[]" Jul 29 19:17:02.700611 ip-10-0-131-93 kubenswrapper[2564]: I0729 19:17:02.696624 2564 flags.go:64] FLAG: --anonymous-auth="true" Jul 29 19:17:02.700611 ip-10-0-131-93 kubenswrapper[2564]: I0729 19:17:02.696628 2564 flags.go:64] FLAG: --application-metrics-count-limit="100" Jul 29 19:17:02.700611 ip-10-0-131-93 kubenswrapper[2564]: I0729 19:17:02.696632 2564 flags.go:64] FLAG: --authentication-token-webhook="false" Jul 29 19:17:02.700611 ip-10-0-131-93 kubenswrapper[2564]: I0729 19:17:02.696635 2564 flags.go:64] FLAG: --authentication-token-webhook-cache-ttl="2m0s" Jul 29 19:17:02.700611 ip-10-0-131-93 kubenswrapper[2564]: I0729 19:17:02.696639 2564 flags.go:64] FLAG: --authorization-mode="AlwaysAllow" Jul 29 19:17:02.700611 ip-10-0-131-93 kubenswrapper[2564]: I0729 19:17:02.696643 2564 flags.go:64] FLAG: --authorization-webhook-cache-authorized-ttl="5m0s" Jul 29 19:17:02.700611 ip-10-0-131-93 kubenswrapper[2564]: I0729 19:17:02.696646 2564 flags.go:64] FLAG: --authorization-webhook-cache-unauthorized-ttl="30s" Jul 29 19:17:02.700611 ip-10-0-131-93 kubenswrapper[2564]: I0729 19:17:02.696649 2564 flags.go:64] FLAG: --boot-id-file="/proc/sys/kernel/random/boot_id" Jul 29 19:17:02.700611 ip-10-0-131-93 kubenswrapper[2564]: I0729 19:17:02.696653 2564 flags.go:64] FLAG: --bootstrap-kubeconfig="/etc/kubernetes/kubeconfig" Jul 29 19:17:02.700611 ip-10-0-131-93 kubenswrapper[2564]: I0729 19:17:02.696656 2564 flags.go:64] FLAG: --cert-dir="/var/lib/kubelet/pki" Jul 29 19:17:02.700611 ip-10-0-131-93 kubenswrapper[2564]: I0729 19:17:02.696659 2564 flags.go:64] FLAG: --cgroup-driver="cgroupfs" Jul 29 19:17:02.700611 ip-10-0-131-93 kubenswrapper[2564]: I0729 19:17:02.696662 2564 flags.go:64] FLAG: --cgroup-root="" Jul 29 19:17:02.700611 ip-10-0-131-93 kubenswrapper[2564]: I0729 19:17:02.696678 2564 flags.go:64] FLAG: --cgroups-per-qos="true" Jul 29 19:17:02.700611 ip-10-0-131-93 kubenswrapper[2564]: I0729 19:17:02.696681 2564 flags.go:64] FLAG: --client-ca-file="" Jul 29 19:17:02.700611 ip-10-0-131-93 kubenswrapper[2564]: I0729 19:17:02.696685 2564 flags.go:64] FLAG: --cloud-provider="external" Jul 29 19:17:02.700611 ip-10-0-131-93 kubenswrapper[2564]: I0729 19:17:02.696688 2564 flags.go:64] FLAG: --cluster-dns="[]" Jul 29 19:17:02.700611 ip-10-0-131-93 kubenswrapper[2564]: I0729 19:17:02.696693 2564 flags.go:64] FLAG: --cluster-domain="" Jul 29 19:17:02.700611 ip-10-0-131-93 kubenswrapper[2564]: I0729 19:17:02.696695 2564 flags.go:64] FLAG: --config="/etc/kubernetes/kubelet.conf" Jul 29 19:17:02.700611 ip-10-0-131-93 kubenswrapper[2564]: I0729 19:17:02.696698 2564 flags.go:64] FLAG: --config-dir="/etc/openshift/kubelet.conf.d" Jul 29 19:17:02.700611 ip-10-0-131-93 kubenswrapper[2564]: I0729 19:17:02.696701 2564 flags.go:64] FLAG: --container-hints="/etc/cadvisor/container_hints.json" Jul 29 19:17:02.700611 ip-10-0-131-93 kubenswrapper[2564]: I0729 19:17:02.696705 2564 flags.go:64] FLAG: --container-log-max-files="5" Jul 29 19:17:02.700611 ip-10-0-131-93 kubenswrapper[2564]: I0729 19:17:02.696712 2564 flags.go:64] FLAG: --container-log-max-size="10Mi" Jul 29 19:17:02.701202 ip-10-0-131-93 kubenswrapper[2564]: I0729 19:17:02.696715 2564 flags.go:64] FLAG: --container-runtime-endpoint="/var/run/crio/crio.sock" Jul 29 19:17:02.701202 ip-10-0-131-93 kubenswrapper[2564]: I0729 19:17:02.696718 2564 flags.go:64] FLAG: --containerd="/run/containerd/containerd.sock" Jul 29 19:17:02.701202 ip-10-0-131-93 kubenswrapper[2564]: I0729 19:17:02.696722 2564 flags.go:64] FLAG: --containerd-namespace="k8s.io" Jul 29 19:17:02.701202 ip-10-0-131-93 kubenswrapper[2564]: I0729 19:17:02.696725 2564 flags.go:64] FLAG: --contention-profiling="false" Jul 29 19:17:02.701202 ip-10-0-131-93 kubenswrapper[2564]: I0729 19:17:02.696728 2564 flags.go:64] FLAG: --cpu-cfs-quota="true" Jul 29 19:17:02.701202 ip-10-0-131-93 kubenswrapper[2564]: I0729 19:17:02.696731 2564 flags.go:64] FLAG: --cpu-cfs-quota-period="100ms" Jul 29 19:17:02.701202 ip-10-0-131-93 kubenswrapper[2564]: I0729 19:17:02.696734 2564 flags.go:64] FLAG: --cpu-manager-policy="none" Jul 29 19:17:02.701202 ip-10-0-131-93 kubenswrapper[2564]: I0729 19:17:02.696737 2564 flags.go:64] FLAG: --cpu-manager-policy-options="" Jul 29 19:17:02.701202 ip-10-0-131-93 kubenswrapper[2564]: I0729 19:17:02.696741 2564 flags.go:64] FLAG: --cpu-manager-reconcile-period="10s" Jul 29 19:17:02.701202 ip-10-0-131-93 kubenswrapper[2564]: I0729 19:17:02.696744 2564 flags.go:64] FLAG: --enable-controller-attach-detach="true" Jul 29 19:17:02.701202 ip-10-0-131-93 kubenswrapper[2564]: I0729 19:17:02.696747 2564 flags.go:64] FLAG: --enable-debugging-handlers="true" Jul 29 19:17:02.701202 ip-10-0-131-93 kubenswrapper[2564]: I0729 19:17:02.696750 2564 flags.go:64] FLAG: --enable-load-reader="false" Jul 29 19:17:02.701202 ip-10-0-131-93 kubenswrapper[2564]: I0729 19:17:02.696752 2564 flags.go:64] FLAG: --enable-server="true" Jul 29 19:17:02.701202 ip-10-0-131-93 kubenswrapper[2564]: I0729 19:17:02.696755 2564 flags.go:64] FLAG: --enforce-node-allocatable="[pods]" Jul 29 19:17:02.701202 ip-10-0-131-93 kubenswrapper[2564]: I0729 19:17:02.696759 2564 flags.go:64] FLAG: --event-burst="100" Jul 29 19:17:02.701202 ip-10-0-131-93 kubenswrapper[2564]: I0729 19:17:02.696763 2564 flags.go:64] FLAG: --event-qps="50" Jul 29 19:17:02.701202 ip-10-0-131-93 kubenswrapper[2564]: I0729 19:17:02.696766 2564 flags.go:64] FLAG: --event-storage-age-limit="default=0" Jul 29 19:17:02.701202 ip-10-0-131-93 kubenswrapper[2564]: I0729 19:17:02.696769 2564 flags.go:64] FLAG: --event-storage-event-limit="default=0" Jul 29 19:17:02.701202 ip-10-0-131-93 kubenswrapper[2564]: I0729 19:17:02.696772 2564 flags.go:64] FLAG: --eviction-hard="" Jul 29 19:17:02.701202 ip-10-0-131-93 kubenswrapper[2564]: I0729 19:17:02.696775 2564 flags.go:64] FLAG: --eviction-max-pod-grace-period="0" Jul 29 19:17:02.701202 ip-10-0-131-93 kubenswrapper[2564]: I0729 19:17:02.696778 2564 flags.go:64] FLAG: --eviction-minimum-reclaim="" Jul 29 19:17:02.701202 ip-10-0-131-93 kubenswrapper[2564]: I0729 19:17:02.696781 2564 flags.go:64] FLAG: --eviction-pressure-transition-period="5m0s" Jul 29 19:17:02.701202 ip-10-0-131-93 kubenswrapper[2564]: I0729 19:17:02.696784 2564 flags.go:64] FLAG: --eviction-soft="" Jul 29 19:17:02.701202 ip-10-0-131-93 kubenswrapper[2564]: I0729 19:17:02.696786 2564 flags.go:64] FLAG: --eviction-soft-grace-period="" Jul 29 19:17:02.701202 ip-10-0-131-93 kubenswrapper[2564]: I0729 19:17:02.696789 2564 flags.go:64] FLAG: --exit-on-lock-contention="false" Jul 29 19:17:02.701848 ip-10-0-131-93 kubenswrapper[2564]: I0729 19:17:02.696792 2564 flags.go:64] FLAG: --experimental-allocatable-ignore-eviction="false" Jul 29 19:17:02.701848 ip-10-0-131-93 kubenswrapper[2564]: I0729 19:17:02.696795 2564 flags.go:64] FLAG: --experimental-mounter-path="" Jul 29 19:17:02.701848 ip-10-0-131-93 kubenswrapper[2564]: I0729 19:17:02.696798 2564 flags.go:64] FLAG: --fail-cgroupv1="false" Jul 29 19:17:02.701848 ip-10-0-131-93 kubenswrapper[2564]: I0729 19:17:02.696801 2564 flags.go:64] FLAG: --fail-swap-on="true" Jul 29 19:17:02.701848 ip-10-0-131-93 kubenswrapper[2564]: I0729 19:17:02.696803 2564 flags.go:64] FLAG: --feature-gates="" Jul 29 19:17:02.701848 ip-10-0-131-93 kubenswrapper[2564]: I0729 19:17:02.696807 2564 flags.go:64] FLAG: --file-check-frequency="20s" Jul 29 19:17:02.701848 ip-10-0-131-93 kubenswrapper[2564]: I0729 19:17:02.696809 2564 flags.go:64] FLAG: --global-housekeeping-interval="1m0s" Jul 29 19:17:02.701848 ip-10-0-131-93 kubenswrapper[2564]: I0729 19:17:02.696812 2564 flags.go:64] FLAG: --hairpin-mode="promiscuous-bridge" Jul 29 19:17:02.701848 ip-10-0-131-93 kubenswrapper[2564]: I0729 19:17:02.696815 2564 flags.go:64] FLAG: --healthz-bind-address="127.0.0.1" Jul 29 19:17:02.701848 ip-10-0-131-93 kubenswrapper[2564]: I0729 19:17:02.696818 2564 flags.go:64] FLAG: --healthz-port="10248" Jul 29 19:17:02.701848 ip-10-0-131-93 kubenswrapper[2564]: I0729 19:17:02.696821 2564 flags.go:64] FLAG: --help="false" Jul 29 19:17:02.701848 ip-10-0-131-93 kubenswrapper[2564]: I0729 19:17:02.696823 2564 flags.go:64] FLAG: --hostname-override="ip-10-0-131-93.ec2.internal" Jul 29 19:17:02.701848 ip-10-0-131-93 kubenswrapper[2564]: I0729 19:17:02.696826 2564 flags.go:64] FLAG: --housekeeping-interval="10s" Jul 29 19:17:02.701848 ip-10-0-131-93 kubenswrapper[2564]: I0729 19:17:02.696831 2564 flags.go:64] FLAG: --http-check-frequency="20s" Jul 29 19:17:02.701848 ip-10-0-131-93 kubenswrapper[2564]: I0729 19:17:02.696834 2564 flags.go:64] FLAG: --image-credential-provider-bin-dir="/usr/libexec/kubelet-image-credential-provider-plugins" Jul 29 19:17:02.701848 ip-10-0-131-93 kubenswrapper[2564]: I0729 19:17:02.696838 2564 flags.go:64] FLAG: --image-credential-provider-config="/etc/kubernetes/credential-providers/ecr-credential-provider.yaml" Jul 29 19:17:02.701848 ip-10-0-131-93 kubenswrapper[2564]: I0729 19:17:02.696841 2564 flags.go:64] FLAG: --image-gc-high-threshold="85" Jul 29 19:17:02.701848 ip-10-0-131-93 kubenswrapper[2564]: I0729 19:17:02.696844 2564 flags.go:64] FLAG: --image-gc-low-threshold="80" Jul 29 19:17:02.701848 ip-10-0-131-93 kubenswrapper[2564]: I0729 19:17:02.696847 2564 flags.go:64] FLAG: --image-service-endpoint="" Jul 29 19:17:02.701848 ip-10-0-131-93 kubenswrapper[2564]: I0729 19:17:02.696850 2564 flags.go:64] FLAG: --kernel-memcg-notification="false" Jul 29 19:17:02.701848 ip-10-0-131-93 kubenswrapper[2564]: I0729 19:17:02.696852 2564 flags.go:64] FLAG: --kube-api-burst="100" Jul 29 19:17:02.701848 ip-10-0-131-93 kubenswrapper[2564]: I0729 19:17:02.696855 2564 flags.go:64] FLAG: --kube-api-content-type="application/vnd.kubernetes.protobuf" Jul 29 19:17:02.701848 ip-10-0-131-93 kubenswrapper[2564]: I0729 19:17:02.696859 2564 flags.go:64] FLAG: --kube-api-qps="50" Jul 29 19:17:02.701848 ip-10-0-131-93 kubenswrapper[2564]: I0729 19:17:02.696862 2564 flags.go:64] FLAG: --kube-reserved="" Jul 29 19:17:02.702452 ip-10-0-131-93 kubenswrapper[2564]: I0729 19:17:02.696865 2564 flags.go:64] FLAG: --kube-reserved-cgroup="" Jul 29 19:17:02.702452 ip-10-0-131-93 kubenswrapper[2564]: I0729 19:17:02.696868 2564 flags.go:64] FLAG: --kubeconfig="/var/lib/kubelet/kubeconfig" Jul 29 19:17:02.702452 ip-10-0-131-93 kubenswrapper[2564]: I0729 19:17:02.696871 2564 flags.go:64] FLAG: --kubelet-cgroups="" Jul 29 19:17:02.702452 ip-10-0-131-93 kubenswrapper[2564]: I0729 19:17:02.696873 2564 flags.go:64] FLAG: --local-storage-capacity-isolation="true" Jul 29 19:17:02.702452 ip-10-0-131-93 kubenswrapper[2564]: I0729 19:17:02.696876 2564 flags.go:64] FLAG: --lock-file="" Jul 29 19:17:02.702452 ip-10-0-131-93 kubenswrapper[2564]: I0729 19:17:02.696879 2564 flags.go:64] FLAG: --log-cadvisor-usage="false" Jul 29 19:17:02.702452 ip-10-0-131-93 kubenswrapper[2564]: I0729 19:17:02.696882 2564 flags.go:64] FLAG: --log-flush-frequency="5s" Jul 29 19:17:02.702452 ip-10-0-131-93 kubenswrapper[2564]: I0729 19:17:02.696884 2564 flags.go:64] FLAG: --log-json-info-buffer-size="0" Jul 29 19:17:02.702452 ip-10-0-131-93 kubenswrapper[2564]: I0729 19:17:02.696890 2564 flags.go:64] FLAG: --log-json-split-stream="false" Jul 29 19:17:02.702452 ip-10-0-131-93 kubenswrapper[2564]: I0729 19:17:02.696893 2564 flags.go:64] FLAG: --log-text-info-buffer-size="0" Jul 29 19:17:02.702452 ip-10-0-131-93 kubenswrapper[2564]: I0729 19:17:02.696896 2564 flags.go:64] FLAG: --log-text-split-stream="false" Jul 29 19:17:02.702452 ip-10-0-131-93 kubenswrapper[2564]: I0729 19:17:02.696898 2564 flags.go:64] FLAG: --logging-format="text" Jul 29 19:17:02.702452 ip-10-0-131-93 kubenswrapper[2564]: I0729 19:17:02.696901 2564 flags.go:64] FLAG: --machine-id-file="/etc/machine-id,/var/lib/dbus/machine-id" Jul 29 19:17:02.702452 ip-10-0-131-93 kubenswrapper[2564]: I0729 19:17:02.696904 2564 flags.go:64] FLAG: --make-iptables-util-chains="true" Jul 29 19:17:02.702452 ip-10-0-131-93 kubenswrapper[2564]: I0729 19:17:02.696907 2564 flags.go:64] FLAG: --manifest-url="" Jul 29 19:17:02.702452 ip-10-0-131-93 kubenswrapper[2564]: I0729 19:17:02.696910 2564 flags.go:64] FLAG: --manifest-url-header="" Jul 29 19:17:02.702452 ip-10-0-131-93 kubenswrapper[2564]: I0729 19:17:02.696914 2564 flags.go:64] FLAG: --max-housekeeping-interval="15s" Jul 29 19:17:02.702452 ip-10-0-131-93 kubenswrapper[2564]: I0729 19:17:02.696917 2564 flags.go:64] FLAG: --max-open-files="1000000" Jul 29 19:17:02.702452 ip-10-0-131-93 kubenswrapper[2564]: I0729 19:17:02.696921 2564 flags.go:64] FLAG: --max-pods="110" Jul 29 19:17:02.702452 ip-10-0-131-93 kubenswrapper[2564]: I0729 19:17:02.696924 2564 flags.go:64] FLAG: --maximum-dead-containers="-1" Jul 29 19:17:02.702452 ip-10-0-131-93 kubenswrapper[2564]: I0729 19:17:02.696927 2564 flags.go:64] FLAG: --maximum-dead-containers-per-container="1" Jul 29 19:17:02.702452 ip-10-0-131-93 kubenswrapper[2564]: I0729 19:17:02.696931 2564 flags.go:64] FLAG: --memory-manager-policy="None" Jul 29 19:17:02.702452 ip-10-0-131-93 kubenswrapper[2564]: I0729 19:17:02.696933 2564 flags.go:64] FLAG: --minimum-container-ttl-duration="6m0s" Jul 29 19:17:02.702452 ip-10-0-131-93 kubenswrapper[2564]: I0729 19:17:02.696936 2564 flags.go:64] FLAG: --minimum-image-ttl-duration="2m0s" Jul 29 19:17:02.702452 ip-10-0-131-93 kubenswrapper[2564]: I0729 19:17:02.696939 2564 flags.go:64] FLAG: --node-ip="0.0.0.0" Jul 29 19:17:02.703113 ip-10-0-131-93 kubenswrapper[2564]: I0729 19:17:02.696942 2564 flags.go:64] FLAG: --node-labels="node-role.kubernetes.io/worker=,node.openshift.io/os_id=rhel" Jul 29 19:17:02.703113 ip-10-0-131-93 kubenswrapper[2564]: I0729 19:17:02.696949 2564 flags.go:64] FLAG: --node-status-max-images="50" Jul 29 19:17:02.703113 ip-10-0-131-93 kubenswrapper[2564]: I0729 19:17:02.696952 2564 flags.go:64] FLAG: --node-status-update-frequency="10s" Jul 29 19:17:02.703113 ip-10-0-131-93 kubenswrapper[2564]: I0729 19:17:02.696955 2564 flags.go:64] FLAG: --oom-score-adj="-999" Jul 29 19:17:02.703113 ip-10-0-131-93 kubenswrapper[2564]: I0729 19:17:02.696958 2564 flags.go:64] FLAG: --pod-cidr="" Jul 29 19:17:02.703113 ip-10-0-131-93 kubenswrapper[2564]: I0729 19:17:02.696961 2564 flags.go:64] FLAG: --pod-infra-container-image="quay.io/openshift-release-dev/ocp-v4.0-art-dev@sha256:7505336a406394b8c1c77d14d890a9f628aa4cfff172312179cb9af8c28c30bb" Jul 29 19:17:02.703113 ip-10-0-131-93 kubenswrapper[2564]: I0729 19:17:02.696967 2564 flags.go:64] FLAG: --pod-manifest-path="" Jul 29 19:17:02.703113 ip-10-0-131-93 kubenswrapper[2564]: I0729 19:17:02.696970 2564 flags.go:64] FLAG: --pod-max-pids="-1" Jul 29 19:17:02.703113 ip-10-0-131-93 kubenswrapper[2564]: I0729 19:17:02.696973 2564 flags.go:64] FLAG: --pods-per-core="0" Jul 29 19:17:02.703113 ip-10-0-131-93 kubenswrapper[2564]: I0729 19:17:02.696976 2564 flags.go:64] FLAG: --port="10250" Jul 29 19:17:02.703113 ip-10-0-131-93 kubenswrapper[2564]: I0729 19:17:02.696979 2564 flags.go:64] FLAG: --protect-kernel-defaults="false" Jul 29 19:17:02.703113 ip-10-0-131-93 kubenswrapper[2564]: I0729 19:17:02.696982 2564 flags.go:64] FLAG: --provider-id="aws:///us-east-1a/i-04d7e0976f4204558" Jul 29 19:17:02.703113 ip-10-0-131-93 kubenswrapper[2564]: I0729 19:17:02.696985 2564 flags.go:64] FLAG: --qos-reserved="" Jul 29 19:17:02.703113 ip-10-0-131-93 kubenswrapper[2564]: I0729 19:17:02.696988 2564 flags.go:64] FLAG: --read-only-port="10255" Jul 29 19:17:02.703113 ip-10-0-131-93 kubenswrapper[2564]: I0729 19:17:02.696991 2564 flags.go:64] FLAG: --register-node="true" Jul 29 19:17:02.703113 ip-10-0-131-93 kubenswrapper[2564]: I0729 19:17:02.696994 2564 flags.go:64] FLAG: --register-with-taints="" Jul 29 19:17:02.703113 ip-10-0-131-93 kubenswrapper[2564]: I0729 19:17:02.696997 2564 flags.go:64] FLAG: --registry-burst="10" Jul 29 19:17:02.703113 ip-10-0-131-93 kubenswrapper[2564]: I0729 19:17:02.697000 2564 flags.go:64] FLAG: --registry-qps="5" Jul 29 19:17:02.703113 ip-10-0-131-93 kubenswrapper[2564]: I0729 19:17:02.697004 2564 flags.go:64] FLAG: --reserved-cpus="" Jul 29 19:17:02.703113 ip-10-0-131-93 kubenswrapper[2564]: I0729 19:17:02.697006 2564 flags.go:64] FLAG: --reserved-memory="" Jul 29 19:17:02.703113 ip-10-0-131-93 kubenswrapper[2564]: I0729 19:17:02.697010 2564 flags.go:64] FLAG: --resolv-conf="/etc/resolv.conf" Jul 29 19:17:02.703113 ip-10-0-131-93 kubenswrapper[2564]: I0729 19:17:02.697013 2564 flags.go:64] FLAG: --root-dir="/var/lib/kubelet" Jul 29 19:17:02.703113 ip-10-0-131-93 kubenswrapper[2564]: I0729 19:17:02.697016 2564 flags.go:64] FLAG: --rotate-certificates="false" Jul 29 19:17:02.703113 ip-10-0-131-93 kubenswrapper[2564]: I0729 19:17:02.697019 2564 flags.go:64] FLAG: --rotate-server-certificates="false" Jul 29 19:17:02.703113 ip-10-0-131-93 kubenswrapper[2564]: I0729 19:17:02.697021 2564 flags.go:64] FLAG: --runonce="false" Jul 29 19:17:02.704213 ip-10-0-131-93 kubenswrapper[2564]: I0729 19:17:02.697024 2564 flags.go:64] FLAG: --runtime-cgroups="/system.slice/crio.service" Jul 29 19:17:02.704213 ip-10-0-131-93 kubenswrapper[2564]: I0729 19:17:02.697027 2564 flags.go:64] FLAG: --runtime-request-timeout="2m0s" Jul 29 19:17:02.704213 ip-10-0-131-93 kubenswrapper[2564]: I0729 19:17:02.697030 2564 flags.go:64] FLAG: --seccomp-default="false" Jul 29 19:17:02.704213 ip-10-0-131-93 kubenswrapper[2564]: I0729 19:17:02.697033 2564 flags.go:64] FLAG: --serialize-image-pulls="true" Jul 29 19:17:02.704213 ip-10-0-131-93 kubenswrapper[2564]: I0729 19:17:02.697038 2564 flags.go:64] FLAG: --storage-driver-buffer-duration="1m0s" Jul 29 19:17:02.704213 ip-10-0-131-93 kubenswrapper[2564]: I0729 19:17:02.697040 2564 flags.go:64] FLAG: --storage-driver-db="cadvisor" Jul 29 19:17:02.704213 ip-10-0-131-93 kubenswrapper[2564]: I0729 19:17:02.697043 2564 flags.go:64] FLAG: --storage-driver-host="localhost:8086" Jul 29 19:17:02.704213 ip-10-0-131-93 kubenswrapper[2564]: I0729 19:17:02.697046 2564 flags.go:64] FLAG: --storage-driver-password="root" Jul 29 19:17:02.704213 ip-10-0-131-93 kubenswrapper[2564]: I0729 19:17:02.697049 2564 flags.go:64] FLAG: --storage-driver-secure="false" Jul 29 19:17:02.704213 ip-10-0-131-93 kubenswrapper[2564]: I0729 19:17:02.697052 2564 flags.go:64] FLAG: --storage-driver-table="stats" Jul 29 19:17:02.704213 ip-10-0-131-93 kubenswrapper[2564]: I0729 19:17:02.697054 2564 flags.go:64] FLAG: --storage-driver-user="root" Jul 29 19:17:02.704213 ip-10-0-131-93 kubenswrapper[2564]: I0729 19:17:02.697057 2564 flags.go:64] FLAG: --streaming-connection-idle-timeout="4h0m0s" Jul 29 19:17:02.704213 ip-10-0-131-93 kubenswrapper[2564]: I0729 19:17:02.697060 2564 flags.go:64] FLAG: --sync-frequency="1m0s" Jul 29 19:17:02.704213 ip-10-0-131-93 kubenswrapper[2564]: I0729 19:17:02.697063 2564 flags.go:64] FLAG: --system-cgroups="" Jul 29 19:17:02.704213 ip-10-0-131-93 kubenswrapper[2564]: I0729 19:17:02.697066 2564 flags.go:64] FLAG: --system-reserved="cpu=500m,ephemeral-storage=1Gi,memory=1Gi" Jul 29 19:17:02.704213 ip-10-0-131-93 kubenswrapper[2564]: I0729 19:17:02.697071 2564 flags.go:64] FLAG: --system-reserved-cgroup="" Jul 29 19:17:02.704213 ip-10-0-131-93 kubenswrapper[2564]: I0729 19:17:02.697074 2564 flags.go:64] FLAG: --tls-cert-file="" Jul 29 19:17:02.704213 ip-10-0-131-93 kubenswrapper[2564]: I0729 19:17:02.697077 2564 flags.go:64] FLAG: --tls-cipher-suites="[]" Jul 29 19:17:02.704213 ip-10-0-131-93 kubenswrapper[2564]: I0729 19:17:02.697081 2564 flags.go:64] FLAG: --tls-min-version="" Jul 29 19:17:02.704213 ip-10-0-131-93 kubenswrapper[2564]: I0729 19:17:02.697084 2564 flags.go:64] FLAG: --tls-private-key-file="" Jul 29 19:17:02.704213 ip-10-0-131-93 kubenswrapper[2564]: I0729 19:17:02.697086 2564 flags.go:64] FLAG: --topology-manager-policy="none" Jul 29 19:17:02.704213 ip-10-0-131-93 kubenswrapper[2564]: I0729 19:17:02.697089 2564 flags.go:64] FLAG: --topology-manager-policy-options="" Jul 29 19:17:02.704213 ip-10-0-131-93 kubenswrapper[2564]: I0729 19:17:02.697092 2564 flags.go:64] FLAG: --topology-manager-scope="container" Jul 29 19:17:02.704213 ip-10-0-131-93 kubenswrapper[2564]: I0729 19:17:02.697095 2564 flags.go:64] FLAG: --v="2" Jul 29 19:17:02.704213 ip-10-0-131-93 kubenswrapper[2564]: I0729 19:17:02.697098 2564 flags.go:64] FLAG: --version="false" Jul 29 19:17:02.705269 ip-10-0-131-93 kubenswrapper[2564]: I0729 19:17:02.697104 2564 flags.go:64] FLAG: --vmodule="" Jul 29 19:17:02.705269 ip-10-0-131-93 kubenswrapper[2564]: I0729 19:17:02.697107 2564 flags.go:64] FLAG: --volume-plugin-dir="/etc/kubernetes/kubelet-plugins/volume/exec" Jul 29 19:17:02.705269 ip-10-0-131-93 kubenswrapper[2564]: I0729 19:17:02.697110 2564 flags.go:64] FLAG: --volume-stats-agg-period="1m0s" Jul 29 19:17:02.705269 ip-10-0-131-93 kubenswrapper[2564]: W0729 19:17:02.697243 2564 feature_gate.go:328] unrecognized feature gate: Example Jul 29 19:17:02.705269 ip-10-0-131-93 kubenswrapper[2564]: W0729 19:17:02.697248 2564 feature_gate.go:328] unrecognized feature gate: AzureDualStackInstall Jul 29 19:17:02.705269 ip-10-0-131-93 kubenswrapper[2564]: W0729 19:17:02.697251 2564 feature_gate.go:328] unrecognized feature gate: NetworkSegmentation Jul 29 19:17:02.705269 ip-10-0-131-93 kubenswrapper[2564]: W0729 19:17:02.697254 2564 feature_gate.go:328] unrecognized feature gate: OVNObservability Jul 29 19:17:02.705269 ip-10-0-131-93 kubenswrapper[2564]: W0729 19:17:02.697257 2564 feature_gate.go:328] unrecognized feature gate: HyperShiftOnlyDynamicResourceAllocation Jul 29 19:17:02.705269 ip-10-0-131-93 kubenswrapper[2564]: W0729 19:17:02.697260 2564 feature_gate.go:328] unrecognized feature gate: BuildCSIVolumes Jul 29 19:17:02.705269 ip-10-0-131-93 kubenswrapper[2564]: W0729 19:17:02.697262 2564 feature_gate.go:328] unrecognized feature gate: AlibabaPlatform Jul 29 19:17:02.705269 ip-10-0-131-93 kubenswrapper[2564]: W0729 19:17:02.697265 2564 feature_gate.go:328] unrecognized feature gate: OSStreams Jul 29 19:17:02.705269 ip-10-0-131-93 kubenswrapper[2564]: W0729 19:17:02.697269 2564 feature_gate.go:328] unrecognized feature gate: ClusterVersionOperatorConfiguration Jul 29 19:17:02.705269 ip-10-0-131-93 kubenswrapper[2564]: W0729 19:17:02.697272 2564 feature_gate.go:328] unrecognized feature gate: IngressControllerDynamicConfigurationManager Jul 29 19:17:02.705269 ip-10-0-131-93 kubenswrapper[2564]: W0729 19:17:02.697274 2564 feature_gate.go:328] unrecognized feature gate: MixedCPUsAllocation Jul 29 19:17:02.705269 ip-10-0-131-93 kubenswrapper[2564]: W0729 19:17:02.697277 2564 feature_gate.go:328] unrecognized feature gate: VSphereHostVMGroupZonal Jul 29 19:17:02.705269 ip-10-0-131-93 kubenswrapper[2564]: W0729 19:17:02.697279 2564 feature_gate.go:328] unrecognized feature gate: AutomatedEtcdBackup Jul 29 19:17:02.705269 ip-10-0-131-93 kubenswrapper[2564]: W0729 19:17:02.697282 2564 feature_gate.go:328] unrecognized feature gate: NewOLMWebhookProviderOpenshiftServiceCA Jul 29 19:17:02.705269 ip-10-0-131-93 kubenswrapper[2564]: W0729 19:17:02.697285 2564 feature_gate.go:328] unrecognized feature gate: NewOLMOwnSingleNamespace Jul 29 19:17:02.705269 ip-10-0-131-93 kubenswrapper[2564]: W0729 19:17:02.697287 2564 feature_gate.go:328] unrecognized feature gate: OpenShiftPodSecurityAdmission Jul 29 19:17:02.705269 ip-10-0-131-93 kubenswrapper[2564]: W0729 19:17:02.697292 2564 feature_gate.go:328] unrecognized feature gate: AWSDedicatedHosts Jul 29 19:17:02.705810 ip-10-0-131-93 kubenswrapper[2564]: W0729 19:17:02.697296 2564 feature_gate.go:349] Setting deprecated feature gate KMSv1=true. It will be removed in a future release. Jul 29 19:17:02.705810 ip-10-0-131-93 kubenswrapper[2564]: W0729 19:17:02.697300 2564 feature_gate.go:328] unrecognized feature gate: ProvisioningRequestAvailable Jul 29 19:17:02.705810 ip-10-0-131-93 kubenswrapper[2564]: W0729 19:17:02.697302 2564 feature_gate.go:328] unrecognized feature gate: Example2 Jul 29 19:17:02.705810 ip-10-0-131-93 kubenswrapper[2564]: W0729 19:17:02.697305 2564 feature_gate.go:328] unrecognized feature gate: MultiArchInstallAzure Jul 29 19:17:02.705810 ip-10-0-131-93 kubenswrapper[2564]: W0729 19:17:02.697308 2564 feature_gate.go:328] unrecognized feature gate: NetworkDiagnosticsConfig Jul 29 19:17:02.705810 ip-10-0-131-93 kubenswrapper[2564]: W0729 19:17:02.697310 2564 feature_gate.go:328] unrecognized feature gate: AWSClusterHostedDNS Jul 29 19:17:02.705810 ip-10-0-131-93 kubenswrapper[2564]: W0729 19:17:02.697313 2564 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesAzure Jul 29 19:17:02.705810 ip-10-0-131-93 kubenswrapper[2564]: W0729 19:17:02.697316 2564 feature_gate.go:328] unrecognized feature gate: AzureWorkloadIdentity Jul 29 19:17:02.705810 ip-10-0-131-93 kubenswrapper[2564]: W0729 19:17:02.697318 2564 feature_gate.go:328] unrecognized feature gate: UpgradeStatus Jul 29 19:17:02.705810 ip-10-0-131-93 kubenswrapper[2564]: W0729 19:17:02.697321 2564 feature_gate.go:328] unrecognized feature gate: ClusterAPIInstallIBMCloud Jul 29 19:17:02.705810 ip-10-0-131-93 kubenswrapper[2564]: W0729 19:17:02.697323 2564 feature_gate.go:328] unrecognized feature gate: NewOLMPreflightPermissionChecks Jul 29 19:17:02.705810 ip-10-0-131-93 kubenswrapper[2564]: W0729 19:17:02.697326 2564 feature_gate.go:328] unrecognized feature gate: ClusterAPIMachineManagement Jul 29 19:17:02.705810 ip-10-0-131-93 kubenswrapper[2564]: W0729 19:17:02.697329 2564 feature_gate.go:328] unrecognized feature gate: MultiDiskSetup Jul 29 19:17:02.705810 ip-10-0-131-93 kubenswrapper[2564]: W0729 19:17:02.697332 2564 feature_gate.go:328] unrecognized feature gate: RouteAdvertisements Jul 29 19:17:02.705810 ip-10-0-131-93 kubenswrapper[2564]: W0729 19:17:02.697334 2564 feature_gate.go:328] unrecognized feature gate: BootcNodeManagement Jul 29 19:17:02.705810 ip-10-0-131-93 kubenswrapper[2564]: W0729 19:17:02.697337 2564 feature_gate.go:328] unrecognized feature gate: GatewayAPI Jul 29 19:17:02.705810 ip-10-0-131-93 kubenswrapper[2564]: W0729 19:17:02.697339 2564 feature_gate.go:328] unrecognized feature gate: EventTTL Jul 29 19:17:02.705810 ip-10-0-131-93 kubenswrapper[2564]: W0729 19:17:02.697342 2564 feature_gate.go:328] unrecognized feature gate: DNSNameResolver Jul 29 19:17:02.705810 ip-10-0-131-93 kubenswrapper[2564]: W0729 19:17:02.697344 2564 feature_gate.go:328] unrecognized feature gate: GCPCustomAPIEndpoints Jul 29 19:17:02.705810 ip-10-0-131-93 kubenswrapper[2564]: W0729 19:17:02.697347 2564 feature_gate.go:328] unrecognized feature gate: AzureDedicatedHosts Jul 29 19:17:02.706612 ip-10-0-131-93 kubenswrapper[2564]: W0729 19:17:02.697349 2564 feature_gate.go:328] unrecognized feature gate: OnPremDNSRecords Jul 29 19:17:02.706612 ip-10-0-131-93 kubenswrapper[2564]: W0729 19:17:02.697353 2564 feature_gate.go:351] Setting GA feature gate ServiceAccountTokenNodeBinding=true. It will be removed in a future release. Jul 29 19:17:02.706612 ip-10-0-131-93 kubenswrapper[2564]: W0729 19:17:02.697357 2564 feature_gate.go:328] unrecognized feature gate: VSphereConfigurableMaxAllowedBlockVolumesPerNode Jul 29 19:17:02.706612 ip-10-0-131-93 kubenswrapper[2564]: W0729 19:17:02.697361 2564 feature_gate.go:328] unrecognized feature gate: NewOLM Jul 29 19:17:02.706612 ip-10-0-131-93 kubenswrapper[2564]: W0729 19:17:02.697364 2564 feature_gate.go:328] unrecognized feature gate: ManagedBootImages Jul 29 19:17:02.706612 ip-10-0-131-93 kubenswrapper[2564]: W0729 19:17:02.697368 2564 feature_gate.go:351] Setting GA feature gate VolumeAttributesClass=true. It will be removed in a future release. Jul 29 19:17:02.706612 ip-10-0-131-93 kubenswrapper[2564]: W0729 19:17:02.697371 2564 feature_gate.go:328] unrecognized feature gate: GatewayAPIController Jul 29 19:17:02.706612 ip-10-0-131-93 kubenswrapper[2564]: W0729 19:17:02.697375 2564 feature_gate.go:328] unrecognized feature gate: AdminNetworkPolicy Jul 29 19:17:02.706612 ip-10-0-131-93 kubenswrapper[2564]: W0729 19:17:02.697377 2564 feature_gate.go:328] unrecognized feature gate: InsightsOnDemandDataGather Jul 29 19:17:02.706612 ip-10-0-131-93 kubenswrapper[2564]: W0729 19:17:02.697381 2564 feature_gate.go:328] unrecognized feature gate: NewOLMCatalogdAPIV1Metas Jul 29 19:17:02.706612 ip-10-0-131-93 kubenswrapper[2564]: W0729 19:17:02.697383 2564 feature_gate.go:328] unrecognized feature gate: ExternalOIDC Jul 29 19:17:02.706612 ip-10-0-131-93 kubenswrapper[2564]: W0729 19:17:02.697388 2564 feature_gate.go:328] unrecognized feature gate: GCPCustomAPIEndpointsInstall Jul 29 19:17:02.706612 ip-10-0-131-93 kubenswrapper[2564]: W0729 19:17:02.697391 2564 feature_gate.go:328] unrecognized feature gate: VSphereMultiDisk Jul 29 19:17:02.706612 ip-10-0-131-93 kubenswrapper[2564]: W0729 19:17:02.697394 2564 feature_gate.go:328] unrecognized feature gate: GCPDualStackInstall Jul 29 19:17:02.706612 ip-10-0-131-93 kubenswrapper[2564]: W0729 19:17:02.697396 2564 feature_gate.go:328] unrecognized feature gate: AzureClusterHostedDNSInstall Jul 29 19:17:02.706612 ip-10-0-131-93 kubenswrapper[2564]: W0729 19:17:02.697399 2564 feature_gate.go:328] unrecognized feature gate: CRDCompatibilityRequirementOperator Jul 29 19:17:02.706612 ip-10-0-131-93 kubenswrapper[2564]: W0729 19:17:02.697402 2564 feature_gate.go:328] unrecognized feature gate: MachineAPIMigration Jul 29 19:17:02.706612 ip-10-0-131-93 kubenswrapper[2564]: W0729 19:17:02.697404 2564 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesAWS Jul 29 19:17:02.706612 ip-10-0-131-93 kubenswrapper[2564]: W0729 19:17:02.697407 2564 feature_gate.go:328] unrecognized feature gate: ShortCertRotation Jul 29 19:17:02.707443 ip-10-0-131-93 kubenswrapper[2564]: W0729 19:17:02.697409 2564 feature_gate.go:328] unrecognized feature gate: CPMSMachineNamePrefix Jul 29 19:17:02.707443 ip-10-0-131-93 kubenswrapper[2564]: W0729 19:17:02.697412 2564 feature_gate.go:328] unrecognized feature gate: EtcdBackendQuota Jul 29 19:17:02.707443 ip-10-0-131-93 kubenswrapper[2564]: W0729 19:17:02.697415 2564 feature_gate.go:328] unrecognized feature gate: SigstoreImageVerification Jul 29 19:17:02.707443 ip-10-0-131-93 kubenswrapper[2564]: W0729 19:17:02.697417 2564 feature_gate.go:328] unrecognized feature gate: NewOLMBoxCutterRuntime Jul 29 19:17:02.707443 ip-10-0-131-93 kubenswrapper[2564]: W0729 19:17:02.697420 2564 feature_gate.go:328] unrecognized feature gate: ClusterAPIMachineManagementVSphere Jul 29 19:17:02.707443 ip-10-0-131-93 kubenswrapper[2564]: W0729 19:17:02.697422 2564 feature_gate.go:328] unrecognized feature gate: VSphereMixedNodeEnv Jul 29 19:17:02.707443 ip-10-0-131-93 kubenswrapper[2564]: W0729 19:17:02.697425 2564 feature_gate.go:328] unrecognized feature gate: NoRegistryClusterInstall Jul 29 19:17:02.707443 ip-10-0-131-93 kubenswrapper[2564]: W0729 19:17:02.697428 2564 feature_gate.go:328] unrecognized feature gate: PreconfiguredUDNAddresses Jul 29 19:17:02.707443 ip-10-0-131-93 kubenswrapper[2564]: W0729 19:17:02.697430 2564 feature_gate.go:328] unrecognized feature gate: KMSEncryptionProvider Jul 29 19:17:02.707443 ip-10-0-131-93 kubenswrapper[2564]: W0729 19:17:02.697433 2564 feature_gate.go:328] unrecognized feature gate: ImageStreamImportMode Jul 29 19:17:02.707443 ip-10-0-131-93 kubenswrapper[2564]: W0729 19:17:02.697436 2564 feature_gate.go:328] unrecognized feature gate: ExternalOIDCWithUIDAndExtraClaimMappings Jul 29 19:17:02.707443 ip-10-0-131-93 kubenswrapper[2564]: W0729 19:17:02.697438 2564 feature_gate.go:328] unrecognized feature gate: GCPClusterHostedDNS Jul 29 19:17:02.707443 ip-10-0-131-93 kubenswrapper[2564]: W0729 19:17:02.697441 2564 feature_gate.go:328] unrecognized feature gate: SigstoreImageVerificationPKI Jul 29 19:17:02.707443 ip-10-0-131-93 kubenswrapper[2564]: W0729 19:17:02.697443 2564 feature_gate.go:328] unrecognized feature gate: ClusterMonitoringConfig Jul 29 19:17:02.707443 ip-10-0-131-93 kubenswrapper[2564]: W0729 19:17:02.697446 2564 feature_gate.go:328] unrecognized feature gate: MetricsCollectionProfiles Jul 29 19:17:02.707443 ip-10-0-131-93 kubenswrapper[2564]: W0729 19:17:02.697448 2564 feature_gate.go:328] unrecognized feature gate: HighlyAvailableArbiter Jul 29 19:17:02.707443 ip-10-0-131-93 kubenswrapper[2564]: W0729 19:17:02.697452 2564 feature_gate.go:328] unrecognized feature gate: NutanixMultiSubnets Jul 29 19:17:02.707443 ip-10-0-131-93 kubenswrapper[2564]: W0729 19:17:02.697455 2564 feature_gate.go:328] unrecognized feature gate: AdditionalRoutingCapabilities Jul 29 19:17:02.707443 ip-10-0-131-93 kubenswrapper[2564]: W0729 19:17:02.697457 2564 feature_gate.go:328] unrecognized feature gate: IrreconcilableMachineConfig Jul 29 19:17:02.707972 ip-10-0-131-93 kubenswrapper[2564]: W0729 19:17:02.697460 2564 feature_gate.go:328] unrecognized feature gate: ImageModeStatusReporting Jul 29 19:17:02.707972 ip-10-0-131-93 kubenswrapper[2564]: W0729 19:17:02.697462 2564 feature_gate.go:328] unrecognized feature gate: MachineAPIOperatorDisableMachineHealthCheckController Jul 29 19:17:02.707972 ip-10-0-131-93 kubenswrapper[2564]: W0729 19:17:02.697465 2564 feature_gate.go:328] unrecognized feature gate: ClusterAPIInstall Jul 29 19:17:02.707972 ip-10-0-131-93 kubenswrapper[2564]: W0729 19:17:02.697467 2564 feature_gate.go:328] unrecognized feature gate: AzureMultiDisk Jul 29 19:17:02.707972 ip-10-0-131-93 kubenswrapper[2564]: W0729 19:17:02.697470 2564 feature_gate.go:328] unrecognized feature gate: VSphereMultiNetworks Jul 29 19:17:02.707972 ip-10-0-131-93 kubenswrapper[2564]: W0729 19:17:02.697474 2564 feature_gate.go:328] unrecognized feature gate: DyanmicServiceEndpointIBMCloud Jul 29 19:17:02.707972 ip-10-0-131-93 kubenswrapper[2564]: W0729 19:17:02.697477 2564 feature_gate.go:328] unrecognized feature gate: GCPClusterHostedDNSInstall Jul 29 19:17:02.707972 ip-10-0-131-93 kubenswrapper[2564]: W0729 19:17:02.697479 2564 feature_gate.go:328] unrecognized feature gate: AWSServiceLBNetworkSecurityGroup Jul 29 19:17:02.707972 ip-10-0-131-93 kubenswrapper[2564]: W0729 19:17:02.697482 2564 feature_gate.go:328] unrecognized feature gate: KMSEncryption Jul 29 19:17:02.707972 ip-10-0-131-93 kubenswrapper[2564]: W0729 19:17:02.697484 2564 feature_gate.go:328] unrecognized feature gate: MachineConfigNodes Jul 29 19:17:02.707972 ip-10-0-131-93 kubenswrapper[2564]: W0729 19:17:02.697487 2564 feature_gate.go:328] unrecognized feature gate: SignatureStores Jul 29 19:17:02.707972 ip-10-0-131-93 kubenswrapper[2564]: W0729 19:17:02.697489 2564 feature_gate.go:328] unrecognized feature gate: BootImageSkewEnforcement Jul 29 19:17:02.707972 ip-10-0-131-93 kubenswrapper[2564]: W0729 19:17:02.697492 2564 feature_gate.go:328] unrecognized feature gate: VolumeGroupSnapshot Jul 29 19:17:02.707972 ip-10-0-131-93 kubenswrapper[2564]: W0729 19:17:02.697495 2564 feature_gate.go:328] unrecognized feature gate: ExternalSnapshotMetadata Jul 29 19:17:02.707972 ip-10-0-131-93 kubenswrapper[2564]: W0729 19:17:02.697497 2564 feature_gate.go:328] unrecognized feature gate: InsightsConfig Jul 29 19:17:02.707972 ip-10-0-131-93 kubenswrapper[2564]: W0729 19:17:02.697500 2564 feature_gate.go:328] unrecognized feature gate: AWSDualStackInstall Jul 29 19:17:02.707972 ip-10-0-131-93 kubenswrapper[2564]: W0729 19:17:02.697503 2564 feature_gate.go:328] unrecognized feature gate: GatewayAPIWithoutOLM Jul 29 19:17:02.707972 ip-10-0-131-93 kubenswrapper[2564]: W0729 19:17:02.697505 2564 feature_gate.go:328] unrecognized feature gate: ConsolePluginContentSecurityPolicy Jul 29 19:17:02.707972 ip-10-0-131-93 kubenswrapper[2564]: W0729 19:17:02.697508 2564 feature_gate.go:328] unrecognized feature gate: NetworkLiveMigration Jul 29 19:17:02.707972 ip-10-0-131-93 kubenswrapper[2564]: W0729 19:17:02.697510 2564 feature_gate.go:328] unrecognized feature gate: PinnedImages Jul 29 19:17:02.708830 ip-10-0-131-93 kubenswrapper[2564]: W0729 19:17:02.697513 2564 feature_gate.go:328] unrecognized feature gate: DualReplica Jul 29 19:17:02.708830 ip-10-0-131-93 kubenswrapper[2564]: W0729 19:17:02.697516 2564 feature_gate.go:328] unrecognized feature gate: AWSClusterHostedDNSInstall Jul 29 19:17:02.708830 ip-10-0-131-93 kubenswrapper[2564]: W0729 19:17:02.697518 2564 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesCPMS Jul 29 19:17:02.708830 ip-10-0-131-93 kubenswrapper[2564]: W0729 19:17:02.697521 2564 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesvSphere Jul 29 19:17:02.708830 ip-10-0-131-93 kubenswrapper[2564]: I0729 19:17:02.698039 2564 feature_gate.go:384] feature gates: {map[CBORServingAndStorage:false ClientsAllowCBOR:false ClientsPreferCBOR:false EventedPLEG:false ImageVolume:true KMSv1:true MaxUnavailableStatefulSet:false MinimumKubeletVersion:false MutableCSINodeAllocatableCount:false MutatingAdmissionPolicy:false ProcMountType:true RouteExternalCertificate:true SELinuxMount:false ServiceAccountTokenNodeBinding:true StoragePerformantSecurityPolicy:true TranslateStreamCloseWebsocketRequests:false UserNamespacesPodSecurityStandards:true UserNamespacesSupport:true VolumeAttributesClass:true]} Jul 29 19:17:02.708830 ip-10-0-131-93 kubenswrapper[2564]: I0729 19:17:02.705329 2564 server.go:529] "Kubelet version" kubeletVersion="v1.34.9" Jul 29 19:17:02.708830 ip-10-0-131-93 kubenswrapper[2564]: I0729 19:17:02.705348 2564 server.go:531] "Golang settings" GOGC="" GOMAXPROCS="" GOTRACEBACK="" Jul 29 19:17:02.708830 ip-10-0-131-93 kubenswrapper[2564]: I0729 19:17:02.705987 2564 watchdog_linux.go:95] "Systemd watchdog is not enabled" Jul 29 19:17:02.708830 ip-10-0-131-93 kubenswrapper[2564]: I0729 19:17:02.705998 2564 watchdog_linux.go:137] "Systemd watchdog is not enabled or the interval is invalid, so health checking will not be started." Jul 29 19:17:02.708830 ip-10-0-131-93 kubenswrapper[2564]: W0729 19:17:02.706087 2564 feature_gate.go:328] unrecognized feature gate: AutomatedEtcdBackup Jul 29 19:17:02.708830 ip-10-0-131-93 kubenswrapper[2564]: W0729 19:17:02.706097 2564 feature_gate.go:328] unrecognized feature gate: InsightsOnDemandDataGather Jul 29 19:17:02.708830 ip-10-0-131-93 kubenswrapper[2564]: W0729 19:17:02.706103 2564 feature_gate.go:328] unrecognized feature gate: AWSDedicatedHosts Jul 29 19:17:02.708830 ip-10-0-131-93 kubenswrapper[2564]: W0729 19:17:02.706108 2564 feature_gate.go:328] unrecognized feature gate: KMSEncryptionProvider Jul 29 19:17:02.708830 ip-10-0-131-93 kubenswrapper[2564]: W0729 19:17:02.706114 2564 feature_gate.go:328] unrecognized feature gate: VSphereMultiDisk Jul 29 19:17:02.708830 ip-10-0-131-93 kubenswrapper[2564]: W0729 19:17:02.706118 2564 feature_gate.go:328] unrecognized feature gate: ClusterAPIMachineManagement Jul 29 19:17:02.709488 ip-10-0-131-93 kubenswrapper[2564]: W0729 19:17:02.706123 2564 feature_gate.go:328] unrecognized feature gate: ExternalSnapshotMetadata Jul 29 19:17:02.709488 ip-10-0-131-93 kubenswrapper[2564]: W0729 19:17:02.706128 2564 feature_gate.go:328] unrecognized feature gate: AWSClusterHostedDNSInstall Jul 29 19:17:02.709488 ip-10-0-131-93 kubenswrapper[2564]: W0729 19:17:02.706151 2564 feature_gate.go:328] unrecognized feature gate: ClusterAPIInstall Jul 29 19:17:02.709488 ip-10-0-131-93 kubenswrapper[2564]: W0729 19:17:02.706157 2564 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesAWS Jul 29 19:17:02.709488 ip-10-0-131-93 kubenswrapper[2564]: W0729 19:17:02.706161 2564 feature_gate.go:328] unrecognized feature gate: MetricsCollectionProfiles Jul 29 19:17:02.709488 ip-10-0-131-93 kubenswrapper[2564]: W0729 19:17:02.706165 2564 feature_gate.go:328] unrecognized feature gate: ManagedBootImages Jul 29 19:17:02.709488 ip-10-0-131-93 kubenswrapper[2564]: W0729 19:17:02.706170 2564 feature_gate.go:328] unrecognized feature gate: GCPDualStackInstall Jul 29 19:17:02.709488 ip-10-0-131-93 kubenswrapper[2564]: W0729 19:17:02.706174 2564 feature_gate.go:328] unrecognized feature gate: SigstoreImageVerification Jul 29 19:17:02.709488 ip-10-0-131-93 kubenswrapper[2564]: W0729 19:17:02.706179 2564 feature_gate.go:328] unrecognized feature gate: ClusterVersionOperatorConfiguration Jul 29 19:17:02.709488 ip-10-0-131-93 kubenswrapper[2564]: W0729 19:17:02.706183 2564 feature_gate.go:328] unrecognized feature gate: ClusterMonitoringConfig Jul 29 19:17:02.709488 ip-10-0-131-93 kubenswrapper[2564]: W0729 19:17:02.706187 2564 feature_gate.go:328] unrecognized feature gate: MultiDiskSetup Jul 29 19:17:02.709488 ip-10-0-131-93 kubenswrapper[2564]: W0729 19:17:02.706192 2564 feature_gate.go:328] unrecognized feature gate: OSStreams Jul 29 19:17:02.709488 ip-10-0-131-93 kubenswrapper[2564]: W0729 19:17:02.706196 2564 feature_gate.go:328] unrecognized feature gate: VSphereConfigurableMaxAllowedBlockVolumesPerNode Jul 29 19:17:02.709488 ip-10-0-131-93 kubenswrapper[2564]: W0729 19:17:02.706200 2564 feature_gate.go:328] unrecognized feature gate: RouteAdvertisements Jul 29 19:17:02.709488 ip-10-0-131-93 kubenswrapper[2564]: W0729 19:17:02.706204 2564 feature_gate.go:328] unrecognized feature gate: AzureWorkloadIdentity Jul 29 19:17:02.709488 ip-10-0-131-93 kubenswrapper[2564]: W0729 19:17:02.706208 2564 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesvSphere Jul 29 19:17:02.709488 ip-10-0-131-93 kubenswrapper[2564]: W0729 19:17:02.706212 2564 feature_gate.go:328] unrecognized feature gate: NetworkDiagnosticsConfig Jul 29 19:17:02.709488 ip-10-0-131-93 kubenswrapper[2564]: W0729 19:17:02.706216 2564 feature_gate.go:328] unrecognized feature gate: AdminNetworkPolicy Jul 29 19:17:02.709488 ip-10-0-131-93 kubenswrapper[2564]: W0729 19:17:02.706221 2564 feature_gate.go:328] unrecognized feature gate: GCPCustomAPIEndpoints Jul 29 19:17:02.709488 ip-10-0-131-93 kubenswrapper[2564]: W0729 19:17:02.706225 2564 feature_gate.go:328] unrecognized feature gate: ClusterAPIMachineManagementVSphere Jul 29 19:17:02.710001 ip-10-0-131-93 kubenswrapper[2564]: W0729 19:17:02.706229 2564 feature_gate.go:328] unrecognized feature gate: VSphereHostVMGroupZonal Jul 29 19:17:02.710001 ip-10-0-131-93 kubenswrapper[2564]: W0729 19:17:02.706234 2564 feature_gate.go:328] unrecognized feature gate: OpenShiftPodSecurityAdmission Jul 29 19:17:02.710001 ip-10-0-131-93 kubenswrapper[2564]: W0729 19:17:02.706238 2564 feature_gate.go:328] unrecognized feature gate: InsightsConfig Jul 29 19:17:02.710001 ip-10-0-131-93 kubenswrapper[2564]: W0729 19:17:02.706244 2564 feature_gate.go:328] unrecognized feature gate: AzureDedicatedHosts Jul 29 19:17:02.710001 ip-10-0-131-93 kubenswrapper[2564]: W0729 19:17:02.706249 2564 feature_gate.go:328] unrecognized feature gate: BuildCSIVolumes Jul 29 19:17:02.710001 ip-10-0-131-93 kubenswrapper[2564]: W0729 19:17:02.706254 2564 feature_gate.go:328] unrecognized feature gate: NewOLMCatalogdAPIV1Metas Jul 29 19:17:02.710001 ip-10-0-131-93 kubenswrapper[2564]: W0729 19:17:02.706258 2564 feature_gate.go:328] unrecognized feature gate: MachineConfigNodes Jul 29 19:17:02.710001 ip-10-0-131-93 kubenswrapper[2564]: W0729 19:17:02.706262 2564 feature_gate.go:328] unrecognized feature gate: ProvisioningRequestAvailable Jul 29 19:17:02.710001 ip-10-0-131-93 kubenswrapper[2564]: W0729 19:17:02.706267 2564 feature_gate.go:328] unrecognized feature gate: NewOLMWebhookProviderOpenshiftServiceCA Jul 29 19:17:02.710001 ip-10-0-131-93 kubenswrapper[2564]: W0729 19:17:02.706271 2564 feature_gate.go:328] unrecognized feature gate: AzureClusterHostedDNSInstall Jul 29 19:17:02.710001 ip-10-0-131-93 kubenswrapper[2564]: W0729 19:17:02.706275 2564 feature_gate.go:328] unrecognized feature gate: AdditionalRoutingCapabilities Jul 29 19:17:02.710001 ip-10-0-131-93 kubenswrapper[2564]: W0729 19:17:02.706280 2564 feature_gate.go:328] unrecognized feature gate: MachineAPIOperatorDisableMachineHealthCheckController Jul 29 19:17:02.710001 ip-10-0-131-93 kubenswrapper[2564]: W0729 19:17:02.706285 2564 feature_gate.go:328] unrecognized feature gate: Example2 Jul 29 19:17:02.710001 ip-10-0-131-93 kubenswrapper[2564]: W0729 19:17:02.706289 2564 feature_gate.go:328] unrecognized feature gate: ExternalOIDC Jul 29 19:17:02.710001 ip-10-0-131-93 kubenswrapper[2564]: W0729 19:17:02.706293 2564 feature_gate.go:328] unrecognized feature gate: VolumeGroupSnapshot Jul 29 19:17:02.710001 ip-10-0-131-93 kubenswrapper[2564]: W0729 19:17:02.706297 2564 feature_gate.go:328] unrecognized feature gate: NewOLMPreflightPermissionChecks Jul 29 19:17:02.710001 ip-10-0-131-93 kubenswrapper[2564]: W0729 19:17:02.706302 2564 feature_gate.go:328] unrecognized feature gate: GatewayAPIWithoutOLM Jul 29 19:17:02.710001 ip-10-0-131-93 kubenswrapper[2564]: W0729 19:17:02.706306 2564 feature_gate.go:328] unrecognized feature gate: IrreconcilableMachineConfig Jul 29 19:17:02.710001 ip-10-0-131-93 kubenswrapper[2564]: W0729 19:17:02.706310 2564 feature_gate.go:328] unrecognized feature gate: OVNObservability Jul 29 19:17:02.710001 ip-10-0-131-93 kubenswrapper[2564]: W0729 19:17:02.706314 2564 feature_gate.go:328] unrecognized feature gate: NutanixMultiSubnets Jul 29 19:17:02.710625 ip-10-0-131-93 kubenswrapper[2564]: W0729 19:17:02.706319 2564 feature_gate.go:328] unrecognized feature gate: DNSNameResolver Jul 29 19:17:02.710625 ip-10-0-131-93 kubenswrapper[2564]: W0729 19:17:02.706323 2564 feature_gate.go:328] unrecognized feature gate: ShortCertRotation Jul 29 19:17:02.710625 ip-10-0-131-93 kubenswrapper[2564]: W0729 19:17:02.706328 2564 feature_gate.go:328] unrecognized feature gate: BootcNodeManagement Jul 29 19:17:02.710625 ip-10-0-131-93 kubenswrapper[2564]: W0729 19:17:02.706332 2564 feature_gate.go:328] unrecognized feature gate: EtcdBackendQuota Jul 29 19:17:02.710625 ip-10-0-131-93 kubenswrapper[2564]: W0729 19:17:02.706337 2564 feature_gate.go:328] unrecognized feature gate: SigstoreImageVerificationPKI Jul 29 19:17:02.710625 ip-10-0-131-93 kubenswrapper[2564]: W0729 19:17:02.706341 2564 feature_gate.go:328] unrecognized feature gate: KMSEncryption Jul 29 19:17:02.710625 ip-10-0-131-93 kubenswrapper[2564]: W0729 19:17:02.706345 2564 feature_gate.go:328] unrecognized feature gate: SignatureStores Jul 29 19:17:02.710625 ip-10-0-131-93 kubenswrapper[2564]: W0729 19:17:02.706350 2564 feature_gate.go:328] unrecognized feature gate: MixedCPUsAllocation Jul 29 19:17:02.710625 ip-10-0-131-93 kubenswrapper[2564]: W0729 19:17:02.706354 2564 feature_gate.go:328] unrecognized feature gate: GCPClusterHostedDNSInstall Jul 29 19:17:02.710625 ip-10-0-131-93 kubenswrapper[2564]: W0729 19:17:02.706358 2564 feature_gate.go:328] unrecognized feature gate: IngressControllerDynamicConfigurationManager Jul 29 19:17:02.710625 ip-10-0-131-93 kubenswrapper[2564]: W0729 19:17:02.706362 2564 feature_gate.go:328] unrecognized feature gate: AWSServiceLBNetworkSecurityGroup Jul 29 19:17:02.710625 ip-10-0-131-93 kubenswrapper[2564]: W0729 19:17:02.706367 2564 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesAzure Jul 29 19:17:02.710625 ip-10-0-131-93 kubenswrapper[2564]: W0729 19:17:02.706371 2564 feature_gate.go:328] unrecognized feature gate: EventTTL Jul 29 19:17:02.710625 ip-10-0-131-93 kubenswrapper[2564]: W0729 19:17:02.706375 2564 feature_gate.go:328] unrecognized feature gate: AWSDualStackInstall Jul 29 19:17:02.710625 ip-10-0-131-93 kubenswrapper[2564]: W0729 19:17:02.706379 2564 feature_gate.go:328] unrecognized feature gate: ExternalOIDCWithUIDAndExtraClaimMappings Jul 29 19:17:02.710625 ip-10-0-131-93 kubenswrapper[2564]: W0729 19:17:02.706385 2564 feature_gate.go:328] unrecognized feature gate: HyperShiftOnlyDynamicResourceAllocation Jul 29 19:17:02.710625 ip-10-0-131-93 kubenswrapper[2564]: W0729 19:17:02.706391 2564 feature_gate.go:328] unrecognized feature gate: AWSClusterHostedDNS Jul 29 19:17:02.710625 ip-10-0-131-93 kubenswrapper[2564]: W0729 19:17:02.706396 2564 feature_gate.go:328] unrecognized feature gate: NoRegistryClusterInstall Jul 29 19:17:02.710625 ip-10-0-131-93 kubenswrapper[2564]: W0729 19:17:02.706400 2564 feature_gate.go:328] unrecognized feature gate: NetworkSegmentation Jul 29 19:17:02.710625 ip-10-0-131-93 kubenswrapper[2564]: W0729 19:17:02.706405 2564 feature_gate.go:328] unrecognized feature gate: CRDCompatibilityRequirementOperator Jul 29 19:17:02.711169 ip-10-0-131-93 kubenswrapper[2564]: W0729 19:17:02.706409 2564 feature_gate.go:328] unrecognized feature gate: BootImageSkewEnforcement Jul 29 19:17:02.711169 ip-10-0-131-93 kubenswrapper[2564]: W0729 19:17:02.706413 2564 feature_gate.go:328] unrecognized feature gate: ImageStreamImportMode Jul 29 19:17:02.711169 ip-10-0-131-93 kubenswrapper[2564]: W0729 19:17:02.706418 2564 feature_gate.go:328] unrecognized feature gate: ClusterAPIInstallIBMCloud Jul 29 19:17:02.711169 ip-10-0-131-93 kubenswrapper[2564]: W0729 19:17:02.706422 2564 feature_gate.go:328] unrecognized feature gate: MachineAPIMigration Jul 29 19:17:02.711169 ip-10-0-131-93 kubenswrapper[2564]: W0729 19:17:02.706427 2564 feature_gate.go:328] unrecognized feature gate: PinnedImages Jul 29 19:17:02.711169 ip-10-0-131-93 kubenswrapper[2564]: W0729 19:17:02.706431 2564 feature_gate.go:328] unrecognized feature gate: NewOLMOwnSingleNamespace Jul 29 19:17:02.711169 ip-10-0-131-93 kubenswrapper[2564]: W0729 19:17:02.706438 2564 feature_gate.go:351] Setting GA feature gate VolumeAttributesClass=true. It will be removed in a future release. Jul 29 19:17:02.711169 ip-10-0-131-93 kubenswrapper[2564]: W0729 19:17:02.706445 2564 feature_gate.go:349] Setting deprecated feature gate KMSv1=true. It will be removed in a future release. Jul 29 19:17:02.711169 ip-10-0-131-93 kubenswrapper[2564]: W0729 19:17:02.706451 2564 feature_gate.go:328] unrecognized feature gate: NewOLM Jul 29 19:17:02.711169 ip-10-0-131-93 kubenswrapper[2564]: W0729 19:17:02.706456 2564 feature_gate.go:328] unrecognized feature gate: NetworkLiveMigration Jul 29 19:17:02.711169 ip-10-0-131-93 kubenswrapper[2564]: W0729 19:17:02.706461 2564 feature_gate.go:328] unrecognized feature gate: AzureMultiDisk Jul 29 19:17:02.711169 ip-10-0-131-93 kubenswrapper[2564]: W0729 19:17:02.706466 2564 feature_gate.go:328] unrecognized feature gate: ConsolePluginContentSecurityPolicy Jul 29 19:17:02.711169 ip-10-0-131-93 kubenswrapper[2564]: W0729 19:17:02.706470 2564 feature_gate.go:328] unrecognized feature gate: CPMSMachineNamePrefix Jul 29 19:17:02.711169 ip-10-0-131-93 kubenswrapper[2564]: W0729 19:17:02.706475 2564 feature_gate.go:328] unrecognized feature gate: Example Jul 29 19:17:02.711169 ip-10-0-131-93 kubenswrapper[2564]: W0729 19:17:02.706479 2564 feature_gate.go:328] unrecognized feature gate: MultiArchInstallAzure Jul 29 19:17:02.711169 ip-10-0-131-93 kubenswrapper[2564]: W0729 19:17:02.706484 2564 feature_gate.go:328] unrecognized feature gate: HighlyAvailableArbiter Jul 29 19:17:02.711169 ip-10-0-131-93 kubenswrapper[2564]: W0729 19:17:02.706488 2564 feature_gate.go:328] unrecognized feature gate: DualReplica Jul 29 19:17:02.711169 ip-10-0-131-93 kubenswrapper[2564]: W0729 19:17:02.706492 2564 feature_gate.go:328] unrecognized feature gate: AzureDualStackInstall Jul 29 19:17:02.711169 ip-10-0-131-93 kubenswrapper[2564]: W0729 19:17:02.706496 2564 feature_gate.go:328] unrecognized feature gate: GCPCustomAPIEndpointsInstall Jul 29 19:17:02.711668 ip-10-0-131-93 kubenswrapper[2564]: W0729 19:17:02.706503 2564 feature_gate.go:351] Setting GA feature gate ServiceAccountTokenNodeBinding=true. It will be removed in a future release. Jul 29 19:17:02.711668 ip-10-0-131-93 kubenswrapper[2564]: W0729 19:17:02.706508 2564 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesCPMS Jul 29 19:17:02.711668 ip-10-0-131-93 kubenswrapper[2564]: W0729 19:17:02.706513 2564 feature_gate.go:328] unrecognized feature gate: ImageModeStatusReporting Jul 29 19:17:02.711668 ip-10-0-131-93 kubenswrapper[2564]: W0729 19:17:02.706518 2564 feature_gate.go:328] unrecognized feature gate: VSphereMultiNetworks Jul 29 19:17:02.711668 ip-10-0-131-93 kubenswrapper[2564]: W0729 19:17:02.706522 2564 feature_gate.go:328] unrecognized feature gate: PreconfiguredUDNAddresses Jul 29 19:17:02.711668 ip-10-0-131-93 kubenswrapper[2564]: W0729 19:17:02.706527 2564 feature_gate.go:328] unrecognized feature gate: UpgradeStatus Jul 29 19:17:02.711668 ip-10-0-131-93 kubenswrapper[2564]: W0729 19:17:02.706531 2564 feature_gate.go:328] unrecognized feature gate: VSphereMixedNodeEnv Jul 29 19:17:02.711668 ip-10-0-131-93 kubenswrapper[2564]: W0729 19:17:02.706535 2564 feature_gate.go:328] unrecognized feature gate: OnPremDNSRecords Jul 29 19:17:02.711668 ip-10-0-131-93 kubenswrapper[2564]: W0729 19:17:02.706540 2564 feature_gate.go:328] unrecognized feature gate: DyanmicServiceEndpointIBMCloud Jul 29 19:17:02.711668 ip-10-0-131-93 kubenswrapper[2564]: W0729 19:17:02.706545 2564 feature_gate.go:328] unrecognized feature gate: AlibabaPlatform Jul 29 19:17:02.711668 ip-10-0-131-93 kubenswrapper[2564]: W0729 19:17:02.706550 2564 feature_gate.go:328] unrecognized feature gate: GCPClusterHostedDNS Jul 29 19:17:02.711668 ip-10-0-131-93 kubenswrapper[2564]: W0729 19:17:02.706554 2564 feature_gate.go:328] unrecognized feature gate: GatewayAPI Jul 29 19:17:02.711668 ip-10-0-131-93 kubenswrapper[2564]: W0729 19:17:02.706558 2564 feature_gate.go:328] unrecognized feature gate: GatewayAPIController Jul 29 19:17:02.711668 ip-10-0-131-93 kubenswrapper[2564]: W0729 19:17:02.706563 2564 feature_gate.go:328] unrecognized feature gate: NewOLMBoxCutterRuntime Jul 29 19:17:02.711668 ip-10-0-131-93 kubenswrapper[2564]: I0729 19:17:02.706571 2564 feature_gate.go:384] feature gates: {map[CBORServingAndStorage:false ClientsAllowCBOR:false ClientsPreferCBOR:false EventedPLEG:false ImageVolume:true KMSv1:true MaxUnavailableStatefulSet:false MinimumKubeletVersion:false MutableCSINodeAllocatableCount:false MutatingAdmissionPolicy:false ProcMountType:true RouteExternalCertificate:true SELinuxMount:false ServiceAccountTokenNodeBinding:true StoragePerformantSecurityPolicy:true TranslateStreamCloseWebsocketRequests:false UserNamespacesPodSecurityStandards:true UserNamespacesSupport:true VolumeAttributesClass:true]} Jul 29 19:17:02.712039 ip-10-0-131-93 kubenswrapper[2564]: W0729 19:17:02.706755 2564 feature_gate.go:328] unrecognized feature gate: IngressControllerDynamicConfigurationManager Jul 29 19:17:02.712039 ip-10-0-131-93 kubenswrapper[2564]: W0729 19:17:02.706763 2564 feature_gate.go:328] unrecognized feature gate: NewOLMWebhookProviderOpenshiftServiceCA Jul 29 19:17:02.712039 ip-10-0-131-93 kubenswrapper[2564]: W0729 19:17:02.706769 2564 feature_gate.go:328] unrecognized feature gate: HighlyAvailableArbiter Jul 29 19:17:02.712039 ip-10-0-131-93 kubenswrapper[2564]: W0729 19:17:02.706773 2564 feature_gate.go:328] unrecognized feature gate: ShortCertRotation Jul 29 19:17:02.712039 ip-10-0-131-93 kubenswrapper[2564]: W0729 19:17:02.706778 2564 feature_gate.go:328] unrecognized feature gate: AzureMultiDisk Jul 29 19:17:02.712039 ip-10-0-131-93 kubenswrapper[2564]: W0729 19:17:02.706783 2564 feature_gate.go:328] unrecognized feature gate: NewOLMOwnSingleNamespace Jul 29 19:17:02.712039 ip-10-0-131-93 kubenswrapper[2564]: W0729 19:17:02.706788 2564 feature_gate.go:328] unrecognized feature gate: MultiArchInstallAzure Jul 29 19:17:02.712039 ip-10-0-131-93 kubenswrapper[2564]: W0729 19:17:02.706792 2564 feature_gate.go:328] unrecognized feature gate: ImageModeStatusReporting Jul 29 19:17:02.712039 ip-10-0-131-93 kubenswrapper[2564]: W0729 19:17:02.706797 2564 feature_gate.go:328] unrecognized feature gate: OpenShiftPodSecurityAdmission Jul 29 19:17:02.712039 ip-10-0-131-93 kubenswrapper[2564]: W0729 19:17:02.706801 2564 feature_gate.go:328] unrecognized feature gate: ClusterVersionOperatorConfiguration Jul 29 19:17:02.712039 ip-10-0-131-93 kubenswrapper[2564]: W0729 19:17:02.706808 2564 feature_gate.go:351] Setting GA feature gate ServiceAccountTokenNodeBinding=true. It will be removed in a future release. Jul 29 19:17:02.712039 ip-10-0-131-93 kubenswrapper[2564]: W0729 19:17:02.706814 2564 feature_gate.go:328] unrecognized feature gate: OVNObservability Jul 29 19:17:02.712039 ip-10-0-131-93 kubenswrapper[2564]: W0729 19:17:02.706819 2564 feature_gate.go:328] unrecognized feature gate: MachineAPIMigration Jul 29 19:17:02.712039 ip-10-0-131-93 kubenswrapper[2564]: W0729 19:17:02.706824 2564 feature_gate.go:328] unrecognized feature gate: VSphereMultiDisk Jul 29 19:17:02.712039 ip-10-0-131-93 kubenswrapper[2564]: W0729 19:17:02.706828 2564 feature_gate.go:328] unrecognized feature gate: SigstoreImageVerificationPKI Jul 29 19:17:02.712039 ip-10-0-131-93 kubenswrapper[2564]: W0729 19:17:02.706833 2564 feature_gate.go:328] unrecognized feature gate: AutomatedEtcdBackup Jul 29 19:17:02.712039 ip-10-0-131-93 kubenswrapper[2564]: W0729 19:17:02.706838 2564 feature_gate.go:328] unrecognized feature gate: InsightsOnDemandDataGather Jul 29 19:17:02.712039 ip-10-0-131-93 kubenswrapper[2564]: W0729 19:17:02.706842 2564 feature_gate.go:328] unrecognized feature gate: HyperShiftOnlyDynamicResourceAllocation Jul 29 19:17:02.712039 ip-10-0-131-93 kubenswrapper[2564]: W0729 19:17:02.706847 2564 feature_gate.go:328] unrecognized feature gate: KMSEncryptionProvider Jul 29 19:17:02.712554 ip-10-0-131-93 kubenswrapper[2564]: W0729 19:17:02.706852 2564 feature_gate.go:328] unrecognized feature gate: DualReplica Jul 29 19:17:02.712554 ip-10-0-131-93 kubenswrapper[2564]: W0729 19:17:02.706857 2564 feature_gate.go:328] unrecognized feature gate: ClusterMonitoringConfig Jul 29 19:17:02.712554 ip-10-0-131-93 kubenswrapper[2564]: W0729 19:17:02.706861 2564 feature_gate.go:328] unrecognized feature gate: GCPClusterHostedDNSInstall Jul 29 19:17:02.712554 ip-10-0-131-93 kubenswrapper[2564]: W0729 19:17:02.706866 2564 feature_gate.go:328] unrecognized feature gate: ManagedBootImages Jul 29 19:17:02.712554 ip-10-0-131-93 kubenswrapper[2564]: W0729 19:17:02.706870 2564 feature_gate.go:328] unrecognized feature gate: GCPDualStackInstall Jul 29 19:17:02.712554 ip-10-0-131-93 kubenswrapper[2564]: W0729 19:17:02.706874 2564 feature_gate.go:328] unrecognized feature gate: VSphereConfigurableMaxAllowedBlockVolumesPerNode Jul 29 19:17:02.712554 ip-10-0-131-93 kubenswrapper[2564]: W0729 19:17:02.706878 2564 feature_gate.go:328] unrecognized feature gate: IrreconcilableMachineConfig Jul 29 19:17:02.712554 ip-10-0-131-93 kubenswrapper[2564]: W0729 19:17:02.706883 2564 feature_gate.go:328] unrecognized feature gate: UpgradeStatus Jul 29 19:17:02.712554 ip-10-0-131-93 kubenswrapper[2564]: W0729 19:17:02.706887 2564 feature_gate.go:328] unrecognized feature gate: MultiDiskSetup Jul 29 19:17:02.712554 ip-10-0-131-93 kubenswrapper[2564]: W0729 19:17:02.706893 2564 feature_gate.go:328] unrecognized feature gate: OSStreams Jul 29 19:17:02.712554 ip-10-0-131-93 kubenswrapper[2564]: W0729 19:17:02.706897 2564 feature_gate.go:328] unrecognized feature gate: NewOLMCatalogdAPIV1Metas Jul 29 19:17:02.712554 ip-10-0-131-93 kubenswrapper[2564]: W0729 19:17:02.706901 2564 feature_gate.go:328] unrecognized feature gate: NetworkLiveMigration Jul 29 19:17:02.712554 ip-10-0-131-93 kubenswrapper[2564]: W0729 19:17:02.706905 2564 feature_gate.go:328] unrecognized feature gate: AlibabaPlatform Jul 29 19:17:02.712554 ip-10-0-131-93 kubenswrapper[2564]: W0729 19:17:02.706910 2564 feature_gate.go:328] unrecognized feature gate: Example2 Jul 29 19:17:02.712554 ip-10-0-131-93 kubenswrapper[2564]: W0729 19:17:02.706914 2564 feature_gate.go:328] unrecognized feature gate: AdminNetworkPolicy Jul 29 19:17:02.712554 ip-10-0-131-93 kubenswrapper[2564]: W0729 19:17:02.706918 2564 feature_gate.go:328] unrecognized feature gate: NewOLMBoxCutterRuntime Jul 29 19:17:02.712554 ip-10-0-131-93 kubenswrapper[2564]: W0729 19:17:02.706922 2564 feature_gate.go:328] unrecognized feature gate: Example Jul 29 19:17:02.712554 ip-10-0-131-93 kubenswrapper[2564]: W0729 19:17:02.706953 2564 feature_gate.go:328] unrecognized feature gate: NutanixMultiSubnets Jul 29 19:17:02.712554 ip-10-0-131-93 kubenswrapper[2564]: W0729 19:17:02.706961 2564 feature_gate.go:328] unrecognized feature gate: PinnedImages Jul 29 19:17:02.712554 ip-10-0-131-93 kubenswrapper[2564]: W0729 19:17:02.706967 2564 feature_gate.go:328] unrecognized feature gate: OnPremDNSRecords Jul 29 19:17:02.712554 ip-10-0-131-93 kubenswrapper[2564]: W0729 19:17:02.706972 2564 feature_gate.go:328] unrecognized feature gate: NoRegistryClusterInstall Jul 29 19:17:02.713092 ip-10-0-131-93 kubenswrapper[2564]: W0729 19:17:02.706976 2564 feature_gate.go:328] unrecognized feature gate: AzureWorkloadIdentity Jul 29 19:17:02.713092 ip-10-0-131-93 kubenswrapper[2564]: W0729 19:17:02.706981 2564 feature_gate.go:328] unrecognized feature gate: SigstoreImageVerification Jul 29 19:17:02.713092 ip-10-0-131-93 kubenswrapper[2564]: W0729 19:17:02.706986 2564 feature_gate.go:328] unrecognized feature gate: MachineConfigNodes Jul 29 19:17:02.713092 ip-10-0-131-93 kubenswrapper[2564]: W0729 19:17:02.706991 2564 feature_gate.go:328] unrecognized feature gate: AzureDedicatedHosts Jul 29 19:17:02.713092 ip-10-0-131-93 kubenswrapper[2564]: W0729 19:17:02.706996 2564 feature_gate.go:328] unrecognized feature gate: ClusterAPIMachineManagementVSphere Jul 29 19:17:02.713092 ip-10-0-131-93 kubenswrapper[2564]: W0729 19:17:02.707000 2564 feature_gate.go:328] unrecognized feature gate: DNSNameResolver Jul 29 19:17:02.713092 ip-10-0-131-93 kubenswrapper[2564]: W0729 19:17:02.707005 2564 feature_gate.go:328] unrecognized feature gate: BootcNodeManagement Jul 29 19:17:02.713092 ip-10-0-131-93 kubenswrapper[2564]: W0729 19:17:02.707009 2564 feature_gate.go:328] unrecognized feature gate: MixedCPUsAllocation Jul 29 19:17:02.713092 ip-10-0-131-93 kubenswrapper[2564]: W0729 19:17:02.707013 2564 feature_gate.go:328] unrecognized feature gate: AzureDualStackInstall Jul 29 19:17:02.713092 ip-10-0-131-93 kubenswrapper[2564]: W0729 19:17:02.707017 2564 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesAzure Jul 29 19:17:02.713092 ip-10-0-131-93 kubenswrapper[2564]: W0729 19:17:02.707021 2564 feature_gate.go:328] unrecognized feature gate: CPMSMachineNamePrefix Jul 29 19:17:02.713092 ip-10-0-131-93 kubenswrapper[2564]: W0729 19:17:02.707025 2564 feature_gate.go:328] unrecognized feature gate: BootImageSkewEnforcement Jul 29 19:17:02.713092 ip-10-0-131-93 kubenswrapper[2564]: W0729 19:17:02.707030 2564 feature_gate.go:328] unrecognized feature gate: AWSDedicatedHosts Jul 29 19:17:02.713092 ip-10-0-131-93 kubenswrapper[2564]: W0729 19:17:02.707034 2564 feature_gate.go:328] unrecognized feature gate: AWSServiceLBNetworkSecurityGroup Jul 29 19:17:02.713092 ip-10-0-131-93 kubenswrapper[2564]: W0729 19:17:02.707038 2564 feature_gate.go:328] unrecognized feature gate: ExternalOIDC Jul 29 19:17:02.713092 ip-10-0-131-93 kubenswrapper[2564]: W0729 19:17:02.707042 2564 feature_gate.go:328] unrecognized feature gate: VSphereHostVMGroupZonal Jul 29 19:17:02.713092 ip-10-0-131-93 kubenswrapper[2564]: W0729 19:17:02.707046 2564 feature_gate.go:328] unrecognized feature gate: AWSDualStackInstall Jul 29 19:17:02.713092 ip-10-0-131-93 kubenswrapper[2564]: W0729 19:17:02.707053 2564 feature_gate.go:349] Setting deprecated feature gate KMSv1=true. It will be removed in a future release. Jul 29 19:17:02.713092 ip-10-0-131-93 kubenswrapper[2564]: W0729 19:17:02.707060 2564 feature_gate.go:328] unrecognized feature gate: ClusterAPIInstall Jul 29 19:17:02.713092 ip-10-0-131-93 kubenswrapper[2564]: W0729 19:17:02.707064 2564 feature_gate.go:328] unrecognized feature gate: AWSClusterHostedDNS Jul 29 19:17:02.713831 ip-10-0-131-93 kubenswrapper[2564]: W0729 19:17:02.707068 2564 feature_gate.go:328] unrecognized feature gate: ClusterAPIInstallIBMCloud Jul 29 19:17:02.713831 ip-10-0-131-93 kubenswrapper[2564]: W0729 19:17:02.707073 2564 feature_gate.go:328] unrecognized feature gate: EventTTL Jul 29 19:17:02.713831 ip-10-0-131-93 kubenswrapper[2564]: W0729 19:17:02.707078 2564 feature_gate.go:328] unrecognized feature gate: CRDCompatibilityRequirementOperator Jul 29 19:17:02.713831 ip-10-0-131-93 kubenswrapper[2564]: W0729 19:17:02.707082 2564 feature_gate.go:328] unrecognized feature gate: DyanmicServiceEndpointIBMCloud Jul 29 19:17:02.713831 ip-10-0-131-93 kubenswrapper[2564]: W0729 19:17:02.707086 2564 feature_gate.go:328] unrecognized feature gate: PreconfiguredUDNAddresses Jul 29 19:17:02.713831 ip-10-0-131-93 kubenswrapper[2564]: W0729 19:17:02.707090 2564 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesAWS Jul 29 19:17:02.713831 ip-10-0-131-93 kubenswrapper[2564]: W0729 19:17:02.707094 2564 feature_gate.go:328] unrecognized feature gate: ExternalSnapshotMetadata Jul 29 19:17:02.713831 ip-10-0-131-93 kubenswrapper[2564]: W0729 19:17:02.707099 2564 feature_gate.go:328] unrecognized feature gate: MachineAPIOperatorDisableMachineHealthCheckController Jul 29 19:17:02.713831 ip-10-0-131-93 kubenswrapper[2564]: W0729 19:17:02.707103 2564 feature_gate.go:328] unrecognized feature gate: GCPClusterHostedDNS Jul 29 19:17:02.713831 ip-10-0-131-93 kubenswrapper[2564]: W0729 19:17:02.707107 2564 feature_gate.go:328] unrecognized feature gate: InsightsConfig Jul 29 19:17:02.713831 ip-10-0-131-93 kubenswrapper[2564]: W0729 19:17:02.707111 2564 feature_gate.go:328] unrecognized feature gate: ImageStreamImportMode Jul 29 19:17:02.713831 ip-10-0-131-93 kubenswrapper[2564]: W0729 19:17:02.707115 2564 feature_gate.go:328] unrecognized feature gate: GatewayAPI Jul 29 19:17:02.713831 ip-10-0-131-93 kubenswrapper[2564]: W0729 19:17:02.707120 2564 feature_gate.go:328] unrecognized feature gate: AWSClusterHostedDNSInstall Jul 29 19:17:02.713831 ip-10-0-131-93 kubenswrapper[2564]: W0729 19:17:02.707124 2564 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesvSphere Jul 29 19:17:02.713831 ip-10-0-131-93 kubenswrapper[2564]: W0729 19:17:02.707128 2564 feature_gate.go:328] unrecognized feature gate: GCPCustomAPIEndpoints Jul 29 19:17:02.713831 ip-10-0-131-93 kubenswrapper[2564]: W0729 19:17:02.707147 2564 feature_gate.go:328] unrecognized feature gate: KMSEncryption Jul 29 19:17:02.713831 ip-10-0-131-93 kubenswrapper[2564]: W0729 19:17:02.707152 2564 feature_gate.go:328] unrecognized feature gate: ExternalOIDCWithUIDAndExtraClaimMappings Jul 29 19:17:02.713831 ip-10-0-131-93 kubenswrapper[2564]: W0729 19:17:02.707156 2564 feature_gate.go:328] unrecognized feature gate: SignatureStores Jul 29 19:17:02.713831 ip-10-0-131-93 kubenswrapper[2564]: W0729 19:17:02.707161 2564 feature_gate.go:328] unrecognized feature gate: NetworkSegmentation Jul 29 19:17:02.713831 ip-10-0-131-93 kubenswrapper[2564]: W0729 19:17:02.707165 2564 feature_gate.go:328] unrecognized feature gate: VolumeGroupSnapshot Jul 29 19:17:02.714341 ip-10-0-131-93 kubenswrapper[2564]: W0729 19:17:02.707171 2564 feature_gate.go:351] Setting GA feature gate VolumeAttributesClass=true. It will be removed in a future release. Jul 29 19:17:02.714341 ip-10-0-131-93 kubenswrapper[2564]: W0729 19:17:02.707176 2564 feature_gate.go:328] unrecognized feature gate: EtcdBackendQuota Jul 29 19:17:02.714341 ip-10-0-131-93 kubenswrapper[2564]: W0729 19:17:02.707181 2564 feature_gate.go:328] unrecognized feature gate: AdditionalRoutingCapabilities Jul 29 19:17:02.714341 ip-10-0-131-93 kubenswrapper[2564]: W0729 19:17:02.707187 2564 feature_gate.go:328] unrecognized feature gate: GatewayAPIController Jul 29 19:17:02.714341 ip-10-0-131-93 kubenswrapper[2564]: W0729 19:17:02.707192 2564 feature_gate.go:328] unrecognized feature gate: NewOLMPreflightPermissionChecks Jul 29 19:17:02.714341 ip-10-0-131-93 kubenswrapper[2564]: W0729 19:17:02.707196 2564 feature_gate.go:328] unrecognized feature gate: GCPCustomAPIEndpointsInstall Jul 29 19:17:02.714341 ip-10-0-131-93 kubenswrapper[2564]: W0729 19:17:02.707200 2564 feature_gate.go:328] unrecognized feature gate: BuildCSIVolumes Jul 29 19:17:02.714341 ip-10-0-131-93 kubenswrapper[2564]: W0729 19:17:02.707204 2564 feature_gate.go:328] unrecognized feature gate: VSphereMixedNodeEnv Jul 29 19:17:02.714341 ip-10-0-131-93 kubenswrapper[2564]: W0729 19:17:02.707208 2564 feature_gate.go:328] unrecognized feature gate: ProvisioningRequestAvailable Jul 29 19:17:02.714341 ip-10-0-131-93 kubenswrapper[2564]: W0729 19:17:02.707213 2564 feature_gate.go:328] unrecognized feature gate: VSphereMultiNetworks Jul 29 19:17:02.714341 ip-10-0-131-93 kubenswrapper[2564]: W0729 19:17:02.707218 2564 feature_gate.go:328] unrecognized feature gate: RouteAdvertisements Jul 29 19:17:02.714341 ip-10-0-131-93 kubenswrapper[2564]: W0729 19:17:02.707222 2564 feature_gate.go:328] unrecognized feature gate: ConsolePluginContentSecurityPolicy Jul 29 19:17:02.714341 ip-10-0-131-93 kubenswrapper[2564]: W0729 19:17:02.707226 2564 feature_gate.go:328] unrecognized feature gate: ClusterAPIMachineManagement Jul 29 19:17:02.714341 ip-10-0-131-93 kubenswrapper[2564]: W0729 19:17:02.707231 2564 feature_gate.go:328] unrecognized feature gate: MetricsCollectionProfiles Jul 29 19:17:02.714341 ip-10-0-131-93 kubenswrapper[2564]: W0729 19:17:02.707236 2564 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesCPMS Jul 29 19:17:02.714341 ip-10-0-131-93 kubenswrapper[2564]: W0729 19:17:02.707240 2564 feature_gate.go:328] unrecognized feature gate: GatewayAPIWithoutOLM Jul 29 19:17:02.714341 ip-10-0-131-93 kubenswrapper[2564]: W0729 19:17:02.707244 2564 feature_gate.go:328] unrecognized feature gate: AzureClusterHostedDNSInstall Jul 29 19:17:02.714341 ip-10-0-131-93 kubenswrapper[2564]: W0729 19:17:02.707248 2564 feature_gate.go:328] unrecognized feature gate: NewOLM Jul 29 19:17:02.714341 ip-10-0-131-93 kubenswrapper[2564]: W0729 19:17:02.707253 2564 feature_gate.go:328] unrecognized feature gate: NetworkDiagnosticsConfig Jul 29 19:17:02.714819 ip-10-0-131-93 kubenswrapper[2564]: I0729 19:17:02.707260 2564 feature_gate.go:384] feature gates: {map[CBORServingAndStorage:false ClientsAllowCBOR:false ClientsPreferCBOR:false EventedPLEG:false ImageVolume:true KMSv1:true MaxUnavailableStatefulSet:false MinimumKubeletVersion:false MutableCSINodeAllocatableCount:false MutatingAdmissionPolicy:false ProcMountType:true RouteExternalCertificate:true SELinuxMount:false ServiceAccountTokenNodeBinding:true StoragePerformantSecurityPolicy:true TranslateStreamCloseWebsocketRequests:false UserNamespacesPodSecurityStandards:true UserNamespacesSupport:true VolumeAttributesClass:true]} Jul 29 19:17:02.714819 ip-10-0-131-93 kubenswrapper[2564]: I0729 19:17:02.707433 2564 server.go:962] "Client rotation is on, will bootstrap in background" Jul 29 19:17:02.714819 ip-10-0-131-93 kubenswrapper[2564]: I0729 19:17:02.710064 2564 bootstrap.go:101] "Use the bootstrap credentials to request a cert, and set kubeconfig to point to the certificate dir" Jul 29 19:17:02.714819 ip-10-0-131-93 kubenswrapper[2564]: I0729 19:17:02.713049 2564 server.go:1019] "Starting client certificate rotation" Jul 29 19:17:02.714819 ip-10-0-131-93 kubenswrapper[2564]: I0729 19:17:02.713167 2564 certificate_manager.go:422] "Certificate rotation is enabled" logger="kubernetes.io/kube-apiserver-client-kubelet" Jul 29 19:17:02.714819 ip-10-0-131-93 kubenswrapper[2564]: I0729 19:17:02.713205 2564 certificate_manager.go:566] "Rotating certificates" logger="kubernetes.io/kube-apiserver-client-kubelet" Jul 29 19:17:02.715550 ip-10-0-131-93 kubenswrapper[2564]: I0729 19:17:02.715533 2564 dynamic_cafile_content.go:123] "Loaded a new CA Bundle and Verifier" name="client-ca-bundle::/etc/kubernetes/kubelet-ca.crt" Jul 29 19:17:02.720803 ip-10-0-131-93 kubenswrapper[2564]: I0729 19:17:02.720786 2564 dynamic_cafile_content.go:161] "Starting controller" name="client-ca-bundle::/etc/kubernetes/kubelet-ca.crt" Jul 29 19:17:02.734970 ip-10-0-131-93 kubenswrapper[2564]: I0729 19:17:02.734955 2564 log.go:25] "Validated CRI v1 runtime API" Jul 29 19:17:02.738375 ip-10-0-131-93 kubenswrapper[2564]: I0729 19:17:02.738358 2564 log.go:25] "Validated CRI v1 image API" Jul 29 19:17:02.739110 ip-10-0-131-93 kubenswrapper[2564]: I0729 19:17:02.739094 2564 server.go:1429] "Using cgroup driver setting received from the CRI runtime" cgroupDriver="systemd" Jul 29 19:17:02.739700 ip-10-0-131-93 kubenswrapper[2564]: I0729 19:17:02.739684 2564 reflector.go:439] "Caches populated" logger="kubernetes.io/kube-apiserver-client-kubelet" type="*v1.CertificateSigningRequest" reflector="k8s.io/client-go/tools/watch/informerwatcher.go:162" Jul 29 19:17:02.741357 ip-10-0-131-93 kubenswrapper[2564]: I0729 19:17:02.741340 2564 fs.go:135] Filesystem UUIDs: map[7B77-95E7:/dev/nvme0n1p2 ce783cee-146f-4816-a18d-cd18b8cec85f:/dev/nvme0n1p3 f8677080-6c1e-4006-a8e8-5d4516cf3f51:/dev/nvme0n1p4] Jul 29 19:17:02.741418 ip-10-0-131-93 kubenswrapper[2564]: I0729 19:17:02.741358 2564 fs.go:136] Filesystem partitions: map[/dev/nvme0n1p3:{mountpoint:/boot major:259 minor:3 fsType:ext4 blockSize:0} /dev/nvme0n1p4:{mountpoint:/var major:259 minor:4 fsType:xfs blockSize:0} /dev/shm:{mountpoint:/dev/shm major:0 minor:22 fsType:tmpfs blockSize:0} /run:{mountpoint:/run major:0 minor:24 fsType:tmpfs blockSize:0} /tmp:{mountpoint:/tmp major:0 minor:31 fsType:tmpfs blockSize:0} composefs_0-33:{mountpoint:/ major:0 minor:33 fsType:overlay blockSize:0}] Jul 29 19:17:02.747071 ip-10-0-131-93 kubenswrapper[2564]: I0729 19:17:02.746966 2564 manager.go:216] Machine: {Timestamp:2026-07-29 19:17:02.744871709 +0000 UTC m=+0.310168876 CPUVendorID:GenuineIntel NumCores:8 NumPhysicalCores:4 NumSockets:1 CpuFrequency:3097046 MemoryCapacity:33164492800 SwapCapacity:0 MemoryByType:map[] NVMInfo:{MemoryModeCapacity:0 AppDirectModeCapacity:0 AvgPowerBudget:0} HugePages:[{PageSize:1048576 NumPages:0} {PageSize:2048 NumPages:0}] MachineID:ec268e8634f869a1a5f17594805c5e97 SystemUUID:ec268e86-34f8-69a1-a5f1-7594805c5e97 BootID:e33fc32d-d4e5-4213-bd8f-d6f08f838937 Filesystems:[{Device:/dev/nvme0n1p3 DeviceMajor:259 DeviceMinor:3 Capacity:366869504 Type:vfs Inodes:98304 HasInodes:true} {Device:/dev/shm DeviceMajor:0 DeviceMinor:22 Capacity:16582246400 Type:vfs Inodes:4048400 HasInodes:true} {Device:/run DeviceMajor:0 DeviceMinor:24 Capacity:6632898560 Type:vfs Inodes:819200 HasInodes:true} {Device:/dev/nvme0n1p4 DeviceMajor:259 DeviceMinor:4 Capacity:128243970048 Type:vfs Inodes:62651840 HasInodes:true} {Device:composefs_0-33 DeviceMajor:0 DeviceMinor:33 Capacity:6111232 Type:vfs Inodes:18446744073709551615 HasInodes:true} {Device:/tmp DeviceMajor:0 DeviceMinor:31 Capacity:16582246400 Type:vfs Inodes:1048576 HasInodes:true}] DiskMap:map[259:0:{Name:nvme0n1 Major:259 Minor:0 Size:128849018880 Scheduler:none}] NetworkDevices:[{Name:br-ex MacAddress:02:42:11:36:5d:91 Speed:0 Mtu:9001} {Name:ens5 MacAddress:02:42:11:36:5d:91 Speed:0 Mtu:9001} {Name:ovs-system MacAddress:8a:ab:d7:5c:c1:44 Speed:0 Mtu:1500}] Topology:[{Id:0 Memory:33164492800 HugePages:[{PageSize:1048576 NumPages:0} {PageSize:2048 NumPages:0}] Cores:[{Id:0 Threads:[0 4] Caches:[{Id:0 Size:32768 Type:Data Level:1} {Id:0 Size:32768 Type:Instruction Level:1} {Id:0 Size:1048576 Type:Unified Level:2}] UncoreCaches:[] SocketID:0 BookID: DrawerID:} {Id:1 Threads:[1 5] Caches:[{Id:1 Size:32768 Type:Data Level:1} {Id:1 Size:32768 Type:Instruction Level:1} {Id:1 Size:1048576 Type:Unified Level:2}] UncoreCaches:[] SocketID:0 BookID: DrawerID:} {Id:2 Threads:[2 6] Caches:[{Id:2 Size:32768 Type:Data Level:1} {Id:2 Size:32768 Type:Instruction Level:1} {Id:2 Size:1048576 Type:Unified Level:2}] UncoreCaches:[] SocketID:0 BookID: DrawerID:} {Id:3 Threads:[3 7] Caches:[{Id:3 Size:32768 Type:Data Level:1} {Id:3 Size:32768 Type:Instruction Level:1} {Id:3 Size:1048576 Type:Unified Level:2}] UncoreCaches:[] SocketID:0 BookID: DrawerID:}] Caches:[{Id:0 Size:37486592 Type:Unified Level:3}] Distances:[10]}] CloudProvider:Unknown InstanceType:Unknown InstanceID:None} Jul 29 19:17:02.747071 ip-10-0-131-93 kubenswrapper[2564]: I0729 19:17:02.747065 2564 manager_no_libpfm.go:29] cAdvisor is build without cgo and/or libpfm support. Perf event counters are not available. Jul 29 19:17:02.747190 ip-10-0-131-93 kubenswrapper[2564]: I0729 19:17:02.747154 2564 manager.go:232] Version: {KernelVersion:5.14.0-570.129.1.el9_6.x86_64 ContainerOsVersion:Red Hat Enterprise Linux CoreOS 9.6.20260717-0 (Plow) DockerVersion: DockerAPIVersion: CadvisorVersion: CadvisorRevision:} Jul 29 19:17:02.750357 ip-10-0-131-93 kubenswrapper[2564]: I0729 19:17:02.750334 2564 container_manager_linux.go:273] "Container manager verified user specified cgroup-root exists" cgroupRoot=[] Jul 29 19:17:02.750488 ip-10-0-131-93 kubenswrapper[2564]: I0729 19:17:02.750361 2564 container_manager_linux.go:278] "Creating Container Manager object based on Node Config" nodeConfig={"NodeName":"ip-10-0-131-93.ec2.internal","RuntimeCgroupsName":"/system.slice/crio.service","SystemCgroupsName":"/system.slice","KubeletCgroupsName":"","KubeletOOMScoreAdj":-999,"ContainerRuntime":"","CgroupsPerQOS":true,"CgroupRoot":"/","CgroupDriver":"systemd","KubeletRootDir":"/var/lib/kubelet","ProtectKernelDefaults":true,"KubeReservedCgroupName":"","SystemReservedCgroupName":"","ReservedSystemCPUs":{},"EnforceNodeAllocatable":{"pods":{}},"KubeReserved":null,"SystemReserved":{"cpu":"500m","ephemeral-storage":"1Gi","memory":"1Gi"},"HardEvictionThresholds":[{"Signal":"imagefs.available","Operator":"LessThan","Value":{"Quantity":null,"Percentage":0.15},"GracePeriod":0,"MinReclaim":null},{"Signal":"imagefs.inodesFree","Operator":"LessThan","Value":{"Quantity":null,"Percentage":0.05},"GracePeriod":0,"MinReclaim":null},{"Signal":"memory.available","Operator":"LessThan","Value":{"Quantity":"100Mi","Percentage":0},"GracePeriod":0,"MinReclaim":null},{"Signal":"nodefs.available","Operator":"LessThan","Value":{"Quantity":null,"Percentage":0.1},"GracePeriod":0,"MinReclaim":null},{"Signal":"nodefs.inodesFree","Operator":"LessThan","Value":{"Quantity":null,"Percentage":0.05},"GracePeriod":0,"MinReclaim":null}],"QOSReserved":{},"CPUManagerPolicy":"none","CPUManagerPolicyOptions":null,"TopologyManagerScope":"container","CPUManagerReconcilePeriod":10000000000,"MemoryManagerPolicy":"None","MemoryManagerReservedMemory":null,"PodPidsLimit":4096,"EnforceCPULimits":true,"CPUCFSQuotaPeriod":100000000,"TopologyManagerPolicy":"none","TopologyManagerPolicyOptions":null,"CgroupVersion":2} Jul 29 19:17:02.751720 ip-10-0-131-93 kubenswrapper[2564]: I0729 19:17:02.751710 2564 topology_manager.go:138] "Creating topology manager with none policy" Jul 29 19:17:02.751754 ip-10-0-131-93 kubenswrapper[2564]: I0729 19:17:02.751723 2564 container_manager_linux.go:309] "Creating device plugin manager" Jul 29 19:17:02.751754 ip-10-0-131-93 kubenswrapper[2564]: I0729 19:17:02.751730 2564 manager.go:141] "Creating Device Plugin manager" path="/var/lib/kubelet/device-plugins/kubelet.sock" Jul 29 19:17:02.752459 ip-10-0-131-93 kubenswrapper[2564]: I0729 19:17:02.752449 2564 server.go:74] "Creating device plugin registration server" version="v1beta1" socket="/var/lib/kubelet/device-plugins/kubelet.sock" Jul 29 19:17:02.753255 ip-10-0-131-93 kubenswrapper[2564]: I0729 19:17:02.753245 2564 container_manager_linux.go:318] "Creating Dynamic Resource Allocation (DRA) manager" Jul 29 19:17:02.755346 ip-10-0-131-93 kubenswrapper[2564]: I0729 19:17:02.755335 2564 state_mem.go:36] "Initialized new in-memory state store" Jul 29 19:17:02.755462 ip-10-0-131-93 kubenswrapper[2564]: I0729 19:17:02.755453 2564 server.go:1240] "Using root directory" path="/var/lib/kubelet" Jul 29 19:17:02.755491 ip-10-0-131-93 kubenswrapper[2564]: I0729 19:17:02.755483 2564 kubelet.go:486] "Attempting to sync node with API server" Jul 29 19:17:02.755526 ip-10-0-131-93 kubenswrapper[2564]: I0729 19:17:02.755495 2564 kubelet.go:387] "Adding static pod path" path="/etc/kubernetes/manifests" Jul 29 19:17:02.755526 ip-10-0-131-93 kubenswrapper[2564]: I0729 19:17:02.755509 2564 file.go:69] "Watching path" path="/etc/kubernetes/manifests" Jul 29 19:17:02.755526 ip-10-0-131-93 kubenswrapper[2564]: I0729 19:17:02.755518 2564 kubelet.go:398] "Adding apiserver pod source" Jul 29 19:17:02.755526 ip-10-0-131-93 kubenswrapper[2564]: I0729 19:17:02.755527 2564 apiserver.go:42] "Waiting for node sync before watching apiserver pods" Jul 29 19:17:02.755864 ip-10-0-131-93 kubenswrapper[2564]: I0729 19:17:02.755853 2564 state_mem.go:40] "Initialized new in-memory state store for pod resource information tracking" Jul 29 19:17:02.755903 ip-10-0-131-93 kubenswrapper[2564]: I0729 19:17:02.755870 2564 state_mem.go:40] "Initialized new in-memory state store for pod resource information tracking" Jul 29 19:17:02.758019 ip-10-0-131-93 kubenswrapper[2564]: I0729 19:17:02.758002 2564 kuberuntime_manager.go:301] "Container runtime initialized" containerRuntime="cri-o" version="1.34.10-2.rhaos4.21.git83ee25b.el9" apiVersion="v1" Jul 29 19:17:02.759812 ip-10-0-131-93 kubenswrapper[2564]: I0729 19:17:02.759799 2564 kubelet.go:960] "Not starting ClusterTrustBundle informer because we are in static kubelet mode or the ClusterTrustBundleProjection featuregate is disabled" Jul 29 19:17:02.759854 ip-10-0-131-93 kubenswrapper[2564]: I0729 19:17:02.759830 2564 kubelet.go:984] "Not starting PodCertificateRequest manager because we are in static kubelet mode or the PodCertificateProjection feature gate is disabled" Jul 29 19:17:02.761414 ip-10-0-131-93 kubenswrapper[2564]: I0729 19:17:02.761403 2564 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/portworx-volume" Jul 29 19:17:02.761450 ip-10-0-131-93 kubenswrapper[2564]: I0729 19:17:02.761419 2564 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/empty-dir" Jul 29 19:17:02.761450 ip-10-0-131-93 kubenswrapper[2564]: I0729 19:17:02.761426 2564 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/git-repo" Jul 29 19:17:02.761450 ip-10-0-131-93 kubenswrapper[2564]: I0729 19:17:02.761432 2564 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/host-path" Jul 29 19:17:02.761450 ip-10-0-131-93 kubenswrapper[2564]: I0729 19:17:02.761445 2564 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/nfs" Jul 29 19:17:02.761545 ip-10-0-131-93 kubenswrapper[2564]: I0729 19:17:02.761455 2564 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/secret" Jul 29 19:17:02.761545 ip-10-0-131-93 kubenswrapper[2564]: I0729 19:17:02.761465 2564 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/iscsi" Jul 29 19:17:02.761545 ip-10-0-131-93 kubenswrapper[2564]: I0729 19:17:02.761470 2564 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/downward-api" Jul 29 19:17:02.761545 ip-10-0-131-93 kubenswrapper[2564]: I0729 19:17:02.761477 2564 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/fc" Jul 29 19:17:02.761545 ip-10-0-131-93 kubenswrapper[2564]: I0729 19:17:02.761483 2564 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/configmap" Jul 29 19:17:02.761545 ip-10-0-131-93 kubenswrapper[2564]: I0729 19:17:02.761491 2564 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/projected" Jul 29 19:17:02.761545 ip-10-0-131-93 kubenswrapper[2564]: I0729 19:17:02.761497 2564 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/local-volume" Jul 29 19:17:02.762914 ip-10-0-131-93 kubenswrapper[2564]: I0729 19:17:02.762902 2564 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/csi" Jul 29 19:17:02.762953 ip-10-0-131-93 kubenswrapper[2564]: I0729 19:17:02.762916 2564 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/image" Jul 29 19:17:02.765880 ip-10-0-131-93 kubenswrapper[2564]: I0729 19:17:02.765860 2564 server.go:1268] "Started kubelet" Jul 29 19:17:02.766059 ip-10-0-131-93 kubenswrapper[2564]: I0729 19:17:02.766005 2564 server.go:180] "Starting to listen" address="0.0.0.0" port=10250 Jul 29 19:17:02.766222 ip-10-0-131-93 kubenswrapper[2564]: I0729 19:17:02.766179 2564 ratelimit.go:56] "Setting rate limiting for endpoint" service="podresources" qps=100 burstTokens=10 Jul 29 19:17:02.766276 ip-10-0-131-93 kubenswrapper[2564]: I0729 19:17:02.766244 2564 server_v1.go:49] "podresources" method="list" useActivePods=true Jul 29 19:17:02.767373 ip-10-0-131-93 systemd[1]: Started Kubernetes Kubelet. Jul 29 19:17:02.767574 ip-10-0-131-93 kubenswrapper[2564]: I0729 19:17:02.767558 2564 server.go:310] "Adding debug handlers to kubelet server" Jul 29 19:17:02.769868 ip-10-0-131-93 kubenswrapper[2564]: I0729 19:17:02.769850 2564 csi_plugin.go:990] Failed to contact API server when waiting for CSINode publishing: csinodes.storage.k8s.io "ip-10-0-131-93.ec2.internal" is forbidden: User "system:anonymous" cannot get resource "csinodes" in API group "storage.k8s.io" at the cluster scope Jul 29 19:17:02.769868 ip-10-0-131-93 kubenswrapper[2564]: E0729 19:17:02.769859 2564 reflector.go:205] "Failed to watch" err="failed to list *v1.Service: services is forbidden: User \"system:anonymous\" cannot list resource \"services\" in API group \"\" at the cluster scope" logger="UnhandledError" reflector="k8s.io/client-go/informers/factory.go:160" type="*v1.Service" Jul 29 19:17:02.769995 ip-10-0-131-93 kubenswrapper[2564]: E0729 19:17:02.769886 2564 reflector.go:205] "Failed to watch" err="failed to list *v1.Node: nodes \"ip-10-0-131-93.ec2.internal\" is forbidden: User \"system:anonymous\" cannot list resource \"nodes\" in API group \"\" at the cluster scope" logger="UnhandledError" reflector="k8s.io/client-go/informers/factory.go:160" type="*v1.Node" Jul 29 19:17:02.770231 ip-10-0-131-93 kubenswrapper[2564]: I0729 19:17:02.770216 2564 server.go:249] "Starting to serve the podresources API" endpoint="unix:/var/lib/kubelet/pod-resources/kubelet.sock" Jul 29 19:17:02.771194 ip-10-0-131-93 kubenswrapper[2564]: E0729 19:17:02.771177 2564 kubelet.go:1638] "Image garbage collection failed once. Stats initialization may not have completed yet" err="failed to get imageFs info: unable to find data in memory cache" Jul 29 19:17:02.772210 ip-10-0-131-93 kubenswrapper[2564]: E0729 19:17:02.771409 2564 event.go:359] "Server rejected event (will not retry!)" err="events is forbidden: User \"system:anonymous\" cannot create resource \"events\" in API group \"\" in the namespace \"default\"" event="&Event{ObjectMeta:{ip-10-0-131-93.ec2.internal.18c6d8c04ec4fde1 default 0 0001-01-01 00:00:00 +0000 UTC map[] map[] [] [] []},InvolvedObject:ObjectReference{Kind:Node,Namespace:,Name:ip-10-0-131-93.ec2.internal,UID:ip-10-0-131-93.ec2.internal,APIVersion:,ResourceVersion:,FieldPath:,},Reason:Starting,Message:Starting kubelet.,Source:EventSource{Component:kubelet,Host:ip-10-0-131-93.ec2.internal,},FirstTimestamp:2026-07-29 19:17:02.765833697 +0000 UTC m=+0.331130866,LastTimestamp:2026-07-29 19:17:02.765833697 +0000 UTC m=+0.331130866,Count:1,Type:Normal,EventTime:0001-01-01 00:00:00 +0000 UTC,Series:nil,Action:,Related:nil,ReportingController:kubelet,ReportingInstance:ip-10-0-131-93.ec2.internal,}" Jul 29 19:17:02.776959 ip-10-0-131-93 kubenswrapper[2564]: I0729 19:17:02.776943 2564 certificate_manager.go:422] "Certificate rotation is enabled" logger="kubernetes.io/kubelet-serving" Jul 29 19:17:02.777040 ip-10-0-131-93 kubenswrapper[2564]: I0729 19:17:02.777015 2564 fs_resource_analyzer.go:67] "Starting FS ResourceAnalyzer" Jul 29 19:17:02.777113 ip-10-0-131-93 kubenswrapper[2564]: E0729 19:17:02.777097 2564 kubelet_node_status.go:454] "Error getting the current node from lister" err="node \"ip-10-0-131-93.ec2.internal\" not found" Jul 29 19:17:02.778321 ip-10-0-131-93 kubenswrapper[2564]: I0729 19:17:02.778296 2564 factory.go:55] Registering systemd factory Jul 29 19:17:02.778393 ip-10-0-131-93 kubenswrapper[2564]: I0729 19:17:02.778324 2564 factory.go:223] Registration of the systemd container factory successfully Jul 29 19:17:02.778737 ip-10-0-131-93 kubenswrapper[2564]: I0729 19:17:02.778723 2564 volume_manager.go:311] "The desired_state_of_world populator starts" Jul 29 19:17:02.778790 ip-10-0-131-93 kubenswrapper[2564]: I0729 19:17:02.778742 2564 volume_manager.go:313] "Starting Kubelet Volume Manager" Jul 29 19:17:02.778897 ip-10-0-131-93 kubenswrapper[2564]: I0729 19:17:02.778881 2564 reconstruct.go:97] "Volume reconstruction finished" Jul 29 19:17:02.778897 ip-10-0-131-93 kubenswrapper[2564]: I0729 19:17:02.778896 2564 reconciler.go:29] "Reconciler: start to sync state" Jul 29 19:17:02.779029 ip-10-0-131-93 kubenswrapper[2564]: I0729 19:17:02.778912 2564 desired_state_of_world_populator.go:146] "Desired state populator starts to run" Jul 29 19:17:02.779853 ip-10-0-131-93 kubenswrapper[2564]: I0729 19:17:02.779833 2564 factory.go:153] Registering CRI-O factory Jul 29 19:17:02.779940 ip-10-0-131-93 kubenswrapper[2564]: I0729 19:17:02.779856 2564 factory.go:223] Registration of the crio container factory successfully Jul 29 19:17:02.779940 ip-10-0-131-93 kubenswrapper[2564]: I0729 19:17:02.779909 2564 factory.go:221] Registration of the containerd container factory failed: unable to create containerd client: containerd: cannot unix dial containerd api service: dial unix /run/containerd/containerd.sock: connect: no such file or directory Jul 29 19:17:02.780047 ip-10-0-131-93 kubenswrapper[2564]: I0729 19:17:02.779942 2564 factory.go:103] Registering Raw factory Jul 29 19:17:02.780047 ip-10-0-131-93 kubenswrapper[2564]: I0729 19:17:02.779961 2564 manager.go:1196] Started watching for new ooms in manager Jul 29 19:17:02.780618 ip-10-0-131-93 kubenswrapper[2564]: I0729 19:17:02.780603 2564 manager.go:349] Starting recovery of all containers Jul 29 19:17:02.782360 ip-10-0-131-93 kubenswrapper[2564]: E0729 19:17:02.782336 2564 reflector.go:205] "Failed to watch" err="failed to list *v1.CSIDriver: csidrivers.storage.k8s.io is forbidden: User \"system:anonymous\" cannot list resource \"csidrivers\" in API group \"storage.k8s.io\" at the cluster scope" logger="UnhandledError" reflector="k8s.io/client-go/informers/factory.go:160" type="*v1.CSIDriver" Jul 29 19:17:02.782446 ip-10-0-131-93 kubenswrapper[2564]: E0729 19:17:02.782373 2564 controller.go:145] "Failed to ensure lease exists, will retry" err="leases.coordination.k8s.io \"ip-10-0-131-93.ec2.internal\" is forbidden: User \"system:anonymous\" cannot get resource \"leases\" in API group \"coordination.k8s.io\" in the namespace \"kube-node-lease\"" interval="200ms" Jul 29 19:17:02.789490 ip-10-0-131-93 kubenswrapper[2564]: I0729 19:17:02.789322 2564 csr.go:274] "Certificate signing request is approved, waiting to be issued" logger="kubernetes.io/kube-apiserver-client-kubelet" csr="csr-52pfr" Jul 29 19:17:02.792361 ip-10-0-131-93 kubenswrapper[2564]: I0729 19:17:02.792346 2564 manager.go:354] Recovery completed Jul 29 19:17:02.797255 ip-10-0-131-93 kubenswrapper[2564]: I0729 19:17:02.797241 2564 kubelet_node_status.go:400] "Setting node annotation to enable volume controller attach/detach" Jul 29 19:17:02.797642 ip-10-0-131-93 kubenswrapper[2564]: I0729 19:17:02.797625 2564 csr.go:270] "Certificate signing request is issued" logger="kubernetes.io/kube-apiserver-client-kubelet" csr="csr-52pfr" Jul 29 19:17:02.799743 ip-10-0-131-93 kubenswrapper[2564]: I0729 19:17:02.799728 2564 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-131-93.ec2.internal" event="NodeHasSufficientMemory" Jul 29 19:17:02.799799 ip-10-0-131-93 kubenswrapper[2564]: I0729 19:17:02.799758 2564 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-131-93.ec2.internal" event="NodeHasNoDiskPressure" Jul 29 19:17:02.799799 ip-10-0-131-93 kubenswrapper[2564]: I0729 19:17:02.799771 2564 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-131-93.ec2.internal" event="NodeHasSufficientPID" Jul 29 19:17:02.800247 ip-10-0-131-93 kubenswrapper[2564]: I0729 19:17:02.800234 2564 cpu_manager.go:222] "Starting CPU manager" policy="none" Jul 29 19:17:02.800247 ip-10-0-131-93 kubenswrapper[2564]: I0729 19:17:02.800244 2564 cpu_manager.go:223] "Reconciling" reconcilePeriod="10s" Jul 29 19:17:02.800339 ip-10-0-131-93 kubenswrapper[2564]: I0729 19:17:02.800249 2564 kubelet_node_status.go:400] "Setting node annotation to enable volume controller attach/detach" Jul 29 19:17:02.800339 ip-10-0-131-93 kubenswrapper[2564]: I0729 19:17:02.800259 2564 state_mem.go:36] "Initialized new in-memory state store" Jul 29 19:17:02.800983 ip-10-0-131-93 kubenswrapper[2564]: I0729 19:17:02.800968 2564 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-131-93.ec2.internal" event="NodeHasSufficientMemory" Jul 29 19:17:02.801046 ip-10-0-131-93 kubenswrapper[2564]: I0729 19:17:02.800995 2564 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-131-93.ec2.internal" event="NodeHasNoDiskPressure" Jul 29 19:17:02.801046 ip-10-0-131-93 kubenswrapper[2564]: I0729 19:17:02.801010 2564 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-131-93.ec2.internal" event="NodeHasSufficientPID" Jul 29 19:17:02.801899 ip-10-0-131-93 kubenswrapper[2564]: E0729 19:17:02.801834 2564 event.go:359] "Server rejected event (will not retry!)" err="events is forbidden: User \"system:anonymous\" cannot create resource \"events\" in API group \"\" in the namespace \"default\"" event="&Event{ObjectMeta:{ip-10-0-131-93.ec2.internal.18c6d8c050ca6a7b default 0 0001-01-01 00:00:00 +0000 UTC map[] map[] [] [] []},InvolvedObject:ObjectReference{Kind:Node,Namespace:,Name:ip-10-0-131-93.ec2.internal,UID:ip-10-0-131-93.ec2.internal,APIVersion:,ResourceVersion:,FieldPath:,},Reason:NodeHasSufficientMemory,Message:Node ip-10-0-131-93.ec2.internal status is now: NodeHasSufficientMemory,Source:EventSource{Component:kubelet,Host:ip-10-0-131-93.ec2.internal,},FirstTimestamp:2026-07-29 19:17:02.799743611 +0000 UTC m=+0.365040778,LastTimestamp:2026-07-29 19:17:02.799743611 +0000 UTC m=+0.365040778,Count:1,Type:Normal,EventTime:0001-01-01 00:00:00 +0000 UTC,Series:nil,Action:,Related:nil,ReportingController:kubelet,ReportingInstance:ip-10-0-131-93.ec2.internal,}" Jul 29 19:17:02.801963 ip-10-0-131-93 kubenswrapper[2564]: I0729 19:17:02.801952 2564 policy_none.go:49] "None policy: Start" Jul 29 19:17:02.801998 ip-10-0-131-93 kubenswrapper[2564]: I0729 19:17:02.801964 2564 memory_manager.go:187] "Starting memorymanager" policy="None" Jul 29 19:17:02.801998 ip-10-0-131-93 kubenswrapper[2564]: I0729 19:17:02.801974 2564 state_mem.go:36] "Initializing new in-memory state store" logger="Memory Manager state checkpoint" Jul 29 19:17:02.803286 ip-10-0-131-93 kubenswrapper[2564]: I0729 19:17:02.803273 2564 policy_none.go:47] "Start" Jul 29 19:17:02.836705 ip-10-0-131-93 kubenswrapper[2564]: I0729 19:17:02.836693 2564 manager.go:337] "Starting Device Plugin manager" Jul 29 19:17:02.836768 ip-10-0-131-93 kubenswrapper[2564]: E0729 19:17:02.836726 2564 manager.go:513] "Failed to read data from checkpoint" err="checkpoint is not found" checkpoint="kubelet_internal_checkpoint" Jul 29 19:17:02.836768 ip-10-0-131-93 kubenswrapper[2564]: I0729 19:17:02.836736 2564 server.go:87] "Starting device plugin registration server" Jul 29 19:17:02.837020 ip-10-0-131-93 kubenswrapper[2564]: I0729 19:17:02.837002 2564 eviction_manager.go:189] "Eviction manager: starting control loop" Jul 29 19:17:02.837073 ip-10-0-131-93 kubenswrapper[2564]: I0729 19:17:02.837021 2564 container_log_manager.go:146] "Initializing container log rotate workers" workers=1 monitorPeriod="10s" Jul 29 19:17:02.837167 ip-10-0-131-93 kubenswrapper[2564]: I0729 19:17:02.837149 2564 plugin_watcher.go:51] "Plugin Watcher Start" path="/var/lib/kubelet/plugins_registry" Jul 29 19:17:02.837254 ip-10-0-131-93 kubenswrapper[2564]: I0729 19:17:02.837245 2564 plugin_manager.go:116] "The desired_state_of_world populator (plugin watcher) starts" Jul 29 19:17:02.837337 ip-10-0-131-93 kubenswrapper[2564]: I0729 19:17:02.837256 2564 plugin_manager.go:118] "Starting Kubelet Plugin Manager" Jul 29 19:17:02.837783 ip-10-0-131-93 kubenswrapper[2564]: E0729 19:17:02.837759 2564 eviction_manager.go:267] "eviction manager: failed to check if we have separate container filesystem. Ignoring." err="non-existent label \"crio-containers\"" Jul 29 19:17:02.837849 ip-10-0-131-93 kubenswrapper[2564]: E0729 19:17:02.837818 2564 eviction_manager.go:292] "Eviction manager: failed to get summary stats" err="failed to get node info: node \"ip-10-0-131-93.ec2.internal\" not found" Jul 29 19:17:04.775412 ip-10-0-131-93 kubenswrapper[2564]: I0729 19:17:04.775361 2564 kubelet_network_linux.go:54] "Initialized iptables rules." protocol="IPv4" Jul 29 19:17:04.776607 ip-10-0-131-93 kubenswrapper[2564]: I0729 19:17:04.776513 2564 kubelet_network_linux.go:54] "Initialized iptables rules." protocol="IPv6" Jul 29 19:17:04.776607 ip-10-0-131-93 kubenswrapper[2564]: I0729 19:17:04.776546 2564 status_manager.go:244] "Starting to sync pod status with apiserver" Jul 29 19:17:04.776607 ip-10-0-131-93 kubenswrapper[2564]: I0729 19:17:04.776567 2564 kubelet.go:2448] "Starting kubelet main sync loop" Jul 29 19:17:04.776607 ip-10-0-131-93 kubenswrapper[2564]: E0729 19:17:04.776599 2564 kubelet.go:2472] "Skipping pod synchronization" err="PLEG is not healthy: pleg has yet to be successful" Jul 29 19:17:04.778361 ip-10-0-131-93 kubenswrapper[2564]: I0729 19:17:04.778321 2564 kubelet_node_status.go:400] "Setting node annotation to enable volume controller attach/detach" Jul 29 19:17:04.778762 ip-10-0-131-93 kubenswrapper[2564]: I0729 19:17:04.778746 2564 reflector.go:439] "Caches populated" type="*v1.RuntimeClass" reflector="k8s.io/client-go/informers/factory.go:160" Jul 29 19:17:04.779540 ip-10-0-131-93 kubenswrapper[2564]: I0729 19:17:04.779522 2564 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-131-93.ec2.internal" event="NodeHasSufficientMemory" Jul 29 19:17:04.779646 ip-10-0-131-93 kubenswrapper[2564]: I0729 19:17:04.779554 2564 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-131-93.ec2.internal" event="NodeHasNoDiskPressure" Jul 29 19:17:04.779646 ip-10-0-131-93 kubenswrapper[2564]: I0729 19:17:04.779578 2564 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-131-93.ec2.internal" event="NodeHasSufficientPID" Jul 29 19:17:04.779646 ip-10-0-131-93 kubenswrapper[2564]: I0729 19:17:04.779599 2564 kubelet_node_status.go:78] "Attempting to register node" node="ip-10-0-131-93.ec2.internal" Jul 29 19:17:04.788550 ip-10-0-131-93 kubenswrapper[2564]: I0729 19:17:04.788534 2564 kubelet_node_status.go:81] "Successfully registered node" node="ip-10-0-131-93.ec2.internal" Jul 29 19:17:04.788629 ip-10-0-131-93 kubenswrapper[2564]: E0729 19:17:04.788557 2564 kubelet_node_status.go:536] "Error updating node status, will retry" err="error getting node \"ip-10-0-131-93.ec2.internal\": node \"ip-10-0-131-93.ec2.internal\" not found" Jul 29 19:17:04.815667 ip-10-0-131-93 kubenswrapper[2564]: E0729 19:17:04.815645 2564 kubelet_node_status.go:454] "Error getting the current node from lister" err="node \"ip-10-0-131-93.ec2.internal\" not found" Jul 29 19:17:04.877278 ip-10-0-131-93 kubenswrapper[2564]: I0729 19:17:04.877244 2564 kubelet.go:2535] "SyncLoop ADD" source="file" pods=["kube-system/kube-apiserver-proxy-ip-10-0-131-93.ec2.internal","openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-131-93.ec2.internal"] Jul 29 19:17:04.877436 ip-10-0-131-93 kubenswrapper[2564]: I0729 19:17:04.877323 2564 kubelet_node_status.go:400] "Setting node annotation to enable volume controller attach/detach" Jul 29 19:17:04.878698 ip-10-0-131-93 kubenswrapper[2564]: I0729 19:17:04.878683 2564 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-131-93.ec2.internal" event="NodeHasSufficientMemory" Jul 29 19:17:04.878772 ip-10-0-131-93 kubenswrapper[2564]: I0729 19:17:04.878710 2564 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-131-93.ec2.internal" event="NodeHasNoDiskPressure" Jul 29 19:17:04.878772 ip-10-0-131-93 kubenswrapper[2564]: I0729 19:17:04.878719 2564 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-131-93.ec2.internal" event="NodeHasSufficientPID" Jul 29 19:17:04.880190 ip-10-0-131-93 kubenswrapper[2564]: I0729 19:17:04.880175 2564 kubelet_node_status.go:400] "Setting node annotation to enable volume controller attach/detach" Jul 29 19:17:04.880344 ip-10-0-131-93 kubenswrapper[2564]: I0729 19:17:04.880330 2564 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/kube-apiserver-proxy-ip-10-0-131-93.ec2.internal" Jul 29 19:17:04.880393 ip-10-0-131-93 kubenswrapper[2564]: I0729 19:17:04.880359 2564 kubelet_node_status.go:400] "Setting node annotation to enable volume controller attach/detach" Jul 29 19:17:04.880956 ip-10-0-131-93 kubenswrapper[2564]: I0729 19:17:04.880941 2564 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-131-93.ec2.internal" event="NodeHasSufficientMemory" Jul 29 19:17:04.880956 ip-10-0-131-93 kubenswrapper[2564]: I0729 19:17:04.880950 2564 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-131-93.ec2.internal" event="NodeHasSufficientMemory" Jul 29 19:17:04.881058 ip-10-0-131-93 kubenswrapper[2564]: I0729 19:17:04.880966 2564 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-131-93.ec2.internal" event="NodeHasNoDiskPressure" Jul 29 19:17:04.881058 ip-10-0-131-93 kubenswrapper[2564]: I0729 19:17:04.880970 2564 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-131-93.ec2.internal" event="NodeHasNoDiskPressure" Jul 29 19:17:04.881058 ip-10-0-131-93 kubenswrapper[2564]: I0729 19:17:04.880976 2564 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-131-93.ec2.internal" event="NodeHasSufficientPID" Jul 29 19:17:04.881058 ip-10-0-131-93 kubenswrapper[2564]: I0729 19:17:04.880981 2564 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-131-93.ec2.internal" event="NodeHasSufficientPID" Jul 29 19:17:04.882287 ip-10-0-131-93 kubenswrapper[2564]: I0729 19:17:04.882274 2564 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-131-93.ec2.internal" Jul 29 19:17:04.882386 ip-10-0-131-93 kubenswrapper[2564]: I0729 19:17:04.882299 2564 kubelet_node_status.go:400] "Setting node annotation to enable volume controller attach/detach" Jul 29 19:17:04.883071 ip-10-0-131-93 kubenswrapper[2564]: I0729 19:17:04.883056 2564 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-131-93.ec2.internal" event="NodeHasSufficientMemory" Jul 29 19:17:04.883158 ip-10-0-131-93 kubenswrapper[2564]: I0729 19:17:04.883084 2564 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-131-93.ec2.internal" event="NodeHasNoDiskPressure" Jul 29 19:17:04.883158 ip-10-0-131-93 kubenswrapper[2564]: I0729 19:17:04.883098 2564 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-131-93.ec2.internal" event="NodeHasSufficientPID" Jul 29 19:17:04.903983 ip-10-0-131-93 kubenswrapper[2564]: E0729 19:17:04.903963 2564 kubelet.go:3244] "No need to create a mirror pod, since failed to get node info from the cluster" err="node \"ip-10-0-131-93.ec2.internal\" not found" node="ip-10-0-131-93.ec2.internal" Jul 29 19:17:04.908226 ip-10-0-131-93 kubenswrapper[2564]: E0729 19:17:04.908210 2564 kubelet.go:3244] "No need to create a mirror pod, since failed to get node info from the cluster" err="node \"ip-10-0-131-93.ec2.internal\" not found" node="ip-10-0-131-93.ec2.internal" Jul 29 19:17:04.916258 ip-10-0-131-93 kubenswrapper[2564]: E0729 19:17:04.916244 2564 kubelet_node_status.go:454] "Error getting the current node from lister" err="node \"ip-10-0-131-93.ec2.internal\" not found" Jul 29 19:17:04.921519 ip-10-0-131-93 kubenswrapper[2564]: I0729 19:17:04.921501 2564 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"config\" (UniqueName: \"kubernetes.io/host-path/a1b5cf65d676393f097a7d0896c452c5-config\") pod \"kube-apiserver-proxy-ip-10-0-131-93.ec2.internal\" (UID: \"a1b5cf65d676393f097a7d0896c452c5\") " pod="kube-system/kube-apiserver-proxy-ip-10-0-131-93.ec2.internal" Jul 29 19:17:04.921600 ip-10-0-131-93 kubenswrapper[2564]: I0729 19:17:04.921525 2564 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"etc-kube\" (UniqueName: \"kubernetes.io/host-path/f5406ede63492de294830916375c7f86-etc-kube\") pod \"kube-rbac-proxy-crio-ip-10-0-131-93.ec2.internal\" (UID: \"f5406ede63492de294830916375c7f86\") " pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-131-93.ec2.internal" Jul 29 19:17:04.921600 ip-10-0-131-93 kubenswrapper[2564]: I0729 19:17:04.921543 2564 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"var-lib-kubelet\" (UniqueName: \"kubernetes.io/host-path/f5406ede63492de294830916375c7f86-var-lib-kubelet\") pod \"kube-rbac-proxy-crio-ip-10-0-131-93.ec2.internal\" (UID: \"f5406ede63492de294830916375c7f86\") " pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-131-93.ec2.internal" Jul 29 19:17:05.017273 ip-10-0-131-93 kubenswrapper[2564]: E0729 19:17:05.017239 2564 kubelet_node_status.go:454] "Error getting the current node from lister" err="node \"ip-10-0-131-93.ec2.internal\" not found" Jul 29 19:17:05.022646 ip-10-0-131-93 kubenswrapper[2564]: I0729 19:17:05.022629 2564 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"config\" (UniqueName: \"kubernetes.io/host-path/a1b5cf65d676393f097a7d0896c452c5-config\") pod \"kube-apiserver-proxy-ip-10-0-131-93.ec2.internal\" (UID: \"a1b5cf65d676393f097a7d0896c452c5\") " pod="kube-system/kube-apiserver-proxy-ip-10-0-131-93.ec2.internal" Jul 29 19:17:05.022711 ip-10-0-131-93 kubenswrapper[2564]: I0729 19:17:05.022655 2564 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"etc-kube\" (UniqueName: \"kubernetes.io/host-path/f5406ede63492de294830916375c7f86-etc-kube\") pod \"kube-rbac-proxy-crio-ip-10-0-131-93.ec2.internal\" (UID: \"f5406ede63492de294830916375c7f86\") " pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-131-93.ec2.internal" Jul 29 19:17:05.022711 ip-10-0-131-93 kubenswrapper[2564]: I0729 19:17:05.022672 2564 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"var-lib-kubelet\" (UniqueName: \"kubernetes.io/host-path/f5406ede63492de294830916375c7f86-var-lib-kubelet\") pod \"kube-rbac-proxy-crio-ip-10-0-131-93.ec2.internal\" (UID: \"f5406ede63492de294830916375c7f86\") " pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-131-93.ec2.internal" Jul 29 19:17:05.022781 ip-10-0-131-93 kubenswrapper[2564]: I0729 19:17:05.022717 2564 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"var-lib-kubelet\" (UniqueName: \"kubernetes.io/host-path/f5406ede63492de294830916375c7f86-var-lib-kubelet\") pod \"kube-rbac-proxy-crio-ip-10-0-131-93.ec2.internal\" (UID: \"f5406ede63492de294830916375c7f86\") " pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-131-93.ec2.internal" Jul 29 19:17:05.022781 ip-10-0-131-93 kubenswrapper[2564]: I0729 19:17:05.022727 2564 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"config\" (UniqueName: \"kubernetes.io/host-path/a1b5cf65d676393f097a7d0896c452c5-config\") pod \"kube-apiserver-proxy-ip-10-0-131-93.ec2.internal\" (UID: \"a1b5cf65d676393f097a7d0896c452c5\") " pod="kube-system/kube-apiserver-proxy-ip-10-0-131-93.ec2.internal" Jul 29 19:17:05.022781 ip-10-0-131-93 kubenswrapper[2564]: I0729 19:17:05.022738 2564 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"etc-kube\" (UniqueName: \"kubernetes.io/host-path/f5406ede63492de294830916375c7f86-etc-kube\") pod \"kube-rbac-proxy-crio-ip-10-0-131-93.ec2.internal\" (UID: \"f5406ede63492de294830916375c7f86\") " pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-131-93.ec2.internal" Jul 29 19:17:05.118113 ip-10-0-131-93 kubenswrapper[2564]: E0729 19:17:05.118039 2564 kubelet_node_status.go:454] "Error getting the current node from lister" err="node \"ip-10-0-131-93.ec2.internal\" not found" Jul 29 19:17:05.206668 ip-10-0-131-93 kubenswrapper[2564]: I0729 19:17:05.206626 2564 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/kube-apiserver-proxy-ip-10-0-131-93.ec2.internal" Jul 29 19:17:05.211659 ip-10-0-131-93 kubenswrapper[2564]: I0729 19:17:05.211639 2564 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-131-93.ec2.internal" Jul 29 19:17:05.218582 ip-10-0-131-93 kubenswrapper[2564]: E0729 19:17:05.218565 2564 kubelet_node_status.go:454] "Error getting the current node from lister" err="node \"ip-10-0-131-93.ec2.internal\" not found" Jul 29 19:17:05.319162 ip-10-0-131-93 kubenswrapper[2564]: E0729 19:17:05.319130 2564 kubelet_node_status.go:454] "Error getting the current node from lister" err="node \"ip-10-0-131-93.ec2.internal\" not found" Jul 29 19:17:05.419642 ip-10-0-131-93 kubenswrapper[2564]: E0729 19:17:05.419567 2564 kubelet_node_status.go:454] "Error getting the current node from lister" err="node \"ip-10-0-131-93.ec2.internal\" not found" Jul 29 19:17:05.520087 ip-10-0-131-93 kubenswrapper[2564]: E0729 19:17:05.520056 2564 kubelet_node_status.go:454] "Error getting the current node from lister" err="node \"ip-10-0-131-93.ec2.internal\" not found" Jul 29 19:17:05.553432 ip-10-0-131-93 kubenswrapper[2564]: I0729 19:17:05.553413 2564 transport.go:147] "Certificate rotation detected, shutting down client connections to start using new credentials" Jul 29 19:17:05.553979 ip-10-0-131-93 kubenswrapper[2564]: I0729 19:17:05.553543 2564 reflector.go:571] "Warning: watch ended with error" reflector="k8s.io/client-go/informers/factory.go:160" type="*v1.RuntimeClass" err="very short watch: k8s.io/client-go/informers/factory.go:160: Unexpected watch close - watch lasted less than a second and no items received" Jul 29 19:17:05.621069 ip-10-0-131-93 kubenswrapper[2564]: E0729 19:17:05.621038 2564 kubelet_node_status.go:454] "Error getting the current node from lister" err="node \"ip-10-0-131-93.ec2.internal\" not found" Jul 29 19:17:05.643191 ip-10-0-131-93 kubenswrapper[2564]: I0729 19:17:05.643156 2564 certificate_manager.go:715] "Certificate rotation deadline determined" logger="kubernetes.io/kube-apiserver-client-kubelet" expiration="2028-07-28 19:12:04 +0000 UTC" deadline="2028-02-20 12:22:56.938664226 +0000 UTC" Jul 29 19:17:05.643191 ip-10-0-131-93 kubenswrapper[2564]: I0729 19:17:05.643188 2564 certificate_manager.go:431] "Waiting for next certificate rotation" logger="kubernetes.io/kube-apiserver-client-kubelet" sleep="13697h5m51.295480579s" Jul 29 19:17:05.698078 ip-10-0-131-93 kubenswrapper[2564]: I0729 19:17:05.698060 2564 provider.go:93] Refreshing cache for provider: *credentialprovider.defaultDockerConfigProvider Jul 29 19:17:05.721250 ip-10-0-131-93 kubenswrapper[2564]: E0729 19:17:05.721222 2564 kubelet_node_status.go:454] "Error getting the current node from lister" err="node \"ip-10-0-131-93.ec2.internal\" not found" Jul 29 19:17:05.725531 ip-10-0-131-93 kubenswrapper[2564]: W0729 19:17:05.725508 2564 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-poda1b5cf65d676393f097a7d0896c452c5.slice/crio-d21bafcced894cf7578a9a82462558eeabf140312009c2b4a9272b85c316c9d3 WatchSource:0}: Error finding container d21bafcced894cf7578a9a82462558eeabf140312009c2b4a9272b85c316c9d3: Status 404 returned error can't find the container with id d21bafcced894cf7578a9a82462558eeabf140312009c2b4a9272b85c316c9d3 Jul 29 19:17:05.758959 ip-10-0-131-93 kubenswrapper[2564]: I0729 19:17:05.758932 2564 reflector.go:439] "Caches populated" type="*v1.Service" reflector="k8s.io/client-go/informers/factory.go:160" Jul 29 19:17:05.780251 ip-10-0-131-93 kubenswrapper[2564]: I0729 19:17:05.780202 2564 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kube-system/kube-apiserver-proxy-ip-10-0-131-93.ec2.internal" event={"ID":"a1b5cf65d676393f097a7d0896c452c5","Type":"ContainerStarted","Data":"d21bafcced894cf7578a9a82462558eeabf140312009c2b4a9272b85c316c9d3"} Jul 29 19:17:05.780604 ip-10-0-131-93 kubenswrapper[2564]: I0729 19:17:05.780589 2564 reflector.go:439] "Caches populated" type="*v1.Node" reflector="k8s.io/client-go/informers/factory.go:160" Jul 29 19:17:05.781185 ip-10-0-131-93 kubenswrapper[2564]: I0729 19:17:05.781159 2564 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-131-93.ec2.internal" event={"ID":"f5406ede63492de294830916375c7f86","Type":"ContainerStarted","Data":"2b2effd1b469bb5d1ded82f3c46d5c1890bfb7e0c05eb86267fe70167117f2cd"} Jul 29 19:17:05.818626 ip-10-0-131-93 kubenswrapper[2564]: I0729 19:17:05.818591 2564 kubelet.go:3248] "Creating a mirror pod for static pod" pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-131-93.ec2.internal" Jul 29 19:17:05.830875 ip-10-0-131-93 kubenswrapper[2564]: I0729 19:17:05.830856 2564 warnings.go:110] "Warning: metadata.name: this is used in the Pod's hostname, which can result in surprising behavior; a DNS label is recommended: [must not contain dots]" Jul 29 19:17:05.831676 ip-10-0-131-93 kubenswrapper[2564]: I0729 19:17:05.831663 2564 kubelet.go:3248] "Creating a mirror pod for static pod" pod="kube-system/kube-apiserver-proxy-ip-10-0-131-93.ec2.internal" Jul 29 19:17:05.837956 ip-10-0-131-93 kubenswrapper[2564]: I0729 19:17:05.837938 2564 warnings.go:110] "Warning: metadata.name: this is used in the Pod's hostname, which can result in surprising behavior; a DNS label is recommended: [must not contain dots]" Jul 29 19:17:05.945982 ip-10-0-131-93 kubenswrapper[2564]: I0729 19:17:05.945912 2564 reflector.go:439] "Caches populated" type="*v1.CSIDriver" reflector="k8s.io/client-go/informers/factory.go:160" Jul 29 19:17:06.470827 ip-10-0-131-93 kubenswrapper[2564]: I0729 19:17:06.470791 2564 reflector.go:439] "Caches populated" type="*v1.RuntimeClass" reflector="k8s.io/client-go/informers/factory.go:160" Jul 29 19:17:06.598454 ip-10-0-131-93 kubenswrapper[2564]: I0729 19:17:06.598426 2564 apiserver.go:52] "Watching apiserver" Jul 29 19:17:06.605818 ip-10-0-131-93 kubenswrapper[2564]: I0729 19:17:06.605634 2564 reflector.go:439] "Caches populated" type="*v1.Pod" reflector="pkg/kubelet/config/apiserver.go:66" Jul 29 19:17:06.606988 ip-10-0-131-93 kubenswrapper[2564]: I0729 19:17:06.606962 2564 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-network-diagnostics/network-check-target-464tm","openshift-ovn-kubernetes/ovnkube-node-kl2xl","kube-system/global-pull-secret-syncer-zcgr9","kube-system/kube-apiserver-proxy-ip-10-0-131-93.ec2.internal","openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-z7ptq","openshift-dns/node-resolver-lzsmp","openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-131-93.ec2.internal","openshift-multus/multus-additional-cni-plugins-nmm7d","openshift-multus/multus-ccv7v","openshift-network-operator/iptables-alerter-5n889","kube-system/konnectivity-agent-444jx","openshift-cluster-node-tuning-operator/tuned-s26vt","openshift-image-registry/node-ca-2lv4n","openshift-multus/network-metrics-daemon-hkxvx"] Jul 29 19:17:06.608488 ip-10-0-131-93 kubenswrapper[2564]: I0729 19:17:06.608466 2564 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/multus-ccv7v" Jul 29 19:17:06.610020 ip-10-0-131-93 kubenswrapper[2564]: I0729 19:17:06.609994 2564 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ovn-kubernetes/ovnkube-node-kl2xl" Jul 29 19:17:06.611267 ip-10-0-131-93 kubenswrapper[2564]: I0729 19:17:06.611136 2564 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-multus\"/\"kube-root-ca.crt\"" Jul 29 19:17:06.611267 ip-10-0-131-93 kubenswrapper[2564]: I0729 19:17:06.611215 2564 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-multus\"/\"openshift-service-ca.crt\"" Jul 29 19:17:06.611267 ip-10-0-131-93 kubenswrapper[2564]: I0729 19:17:06.611261 2564 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-multus\"/\"cni-copy-resources\"" Jul 29 19:17:06.611461 ip-10-0-131-93 kubenswrapper[2564]: I0729 19:17:06.611445 2564 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-multus\"/\"default-dockercfg-7zpns\"" Jul 29 19:17:06.611560 ip-10-0-131-93 kubenswrapper[2564]: I0729 19:17:06.611545 2564 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-multus\"/\"multus-daemon-config\"" Jul 29 19:17:06.611657 ip-10-0-131-93 kubenswrapper[2564]: I0729 19:17:06.611641 2564 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-hkxvx" Jul 29 19:17:06.611746 ip-10-0-131-93 kubenswrapper[2564]: E0729 19:17:06.611712 2564 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-hkxvx" podUID="ae220db3-299c-481a-9591-01a8f3873d9a" Jul 29 19:17:06.612552 ip-10-0-131-93 kubenswrapper[2564]: I0729 19:17:06.612131 2564 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ovn-kubernetes\"/\"openshift-service-ca.crt\"" Jul 29 19:17:06.612552 ip-10-0-131-93 kubenswrapper[2564]: I0729 19:17:06.612229 2564 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ovn-kubernetes\"/\"ovn-node-metrics-cert\"" Jul 29 19:17:06.612552 ip-10-0-131-93 kubenswrapper[2564]: I0729 19:17:06.612365 2564 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ovn-kubernetes\"/\"ovn-kubernetes-node-dockercfg-zt8hq\"" Jul 29 19:17:06.612552 ip-10-0-131-93 kubenswrapper[2564]: I0729 19:17:06.612448 2564 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ovn-kubernetes\"/\"ovnkube-config\"" Jul 29 19:17:06.612904 ip-10-0-131-93 kubenswrapper[2564]: I0729 19:17:06.612890 2564 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-464tm" Jul 29 19:17:06.612972 ip-10-0-131-93 kubenswrapper[2564]: E0729 19:17:06.612946 2564 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-464tm" podUID="02612aaa-a23a-4e6b-b224-123af1bb8230" Jul 29 19:17:06.613151 ip-10-0-131-93 kubenswrapper[2564]: I0729 19:17:06.613131 2564 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ovn-kubernetes\"/\"env-overrides\"" Jul 29 19:17:06.613979 ip-10-0-131-93 kubenswrapper[2564]: I0729 19:17:06.613957 2564 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ovn-kubernetes\"/\"kube-root-ca.crt\"" Jul 29 19:17:06.614095 ip-10-0-131-93 kubenswrapper[2564]: I0729 19:17:06.614020 2564 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ovn-kubernetes\"/\"ovnkube-script-lib\"" Jul 29 19:17:06.616433 ip-10-0-131-93 kubenswrapper[2564]: I0729 19:17:06.615090 2564 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-z7ptq" Jul 29 19:17:06.617651 ip-10-0-131-93 kubenswrapper[2564]: I0729 19:17:06.617632 2564 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-cluster-csi-drivers\"/\"aws-ebs-csi-driver-node-sa-dockercfg-l8llc\"" Jul 29 19:17:06.617885 ip-10-0-131-93 kubenswrapper[2564]: I0729 19:17:06.617859 2564 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/konnectivity-agent-444jx" Jul 29 19:17:06.618390 ip-10-0-131-93 kubenswrapper[2564]: I0729 19:17:06.618355 2564 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-cluster-csi-drivers\"/\"openshift-service-ca.crt\"" Jul 29 19:17:06.619048 ip-10-0-131-93 kubenswrapper[2564]: I0729 19:17:06.618622 2564 certificate_manager.go:566] "Rotating certificates" logger="kubernetes.io/kubelet-serving" Jul 29 19:17:06.619048 ip-10-0-131-93 kubenswrapper[2564]: I0729 19:17:06.618782 2564 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-cluster-csi-drivers\"/\"kube-root-ca.crt\"" Jul 29 19:17:06.619048 ip-10-0-131-93 kubenswrapper[2564]: I0729 19:17:06.619027 2564 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-cluster-csi-drivers\"/\"aws-ebs-csi-driver-node-metrics-serving-cert\"" Jul 29 19:17:06.619624 ip-10-0-131-93 kubenswrapper[2564]: I0729 19:17:06.619607 2564 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-operator/iptables-alerter-5n889" Jul 29 19:17:06.621293 ip-10-0-131-93 kubenswrapper[2564]: I0729 19:17:06.621123 2564 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"kube-system\"/\"konnectivity-ca-bundle\"" Jul 29 19:17:06.621976 ip-10-0-131-93 kubenswrapper[2564]: I0729 19:17:06.621601 2564 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"kube-system\"/\"konnectivity-agent\"" Jul 29 19:17:06.621976 ip-10-0-131-93 kubenswrapper[2564]: I0729 19:17:06.621705 2564 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"kube-system\"/\"default-dockercfg-dg5pc\"" Jul 29 19:17:06.621976 ip-10-0-131-93 kubenswrapper[2564]: I0729 19:17:06.621747 2564 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-network-operator\"/\"openshift-service-ca.crt\"" Jul 29 19:17:06.621976 ip-10-0-131-93 kubenswrapper[2564]: I0729 19:17:06.621794 2564 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/multus-additional-cni-plugins-nmm7d" Jul 29 19:17:06.622262 ip-10-0-131-93 kubenswrapper[2564]: I0729 19:17:06.622137 2564 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-network-operator\"/\"kube-root-ca.crt\"" Jul 29 19:17:06.622262 ip-10-0-131-93 kubenswrapper[2564]: I0729 19:17:06.622151 2564 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-network-operator\"/\"iptables-alerter-script\"" Jul 29 19:17:06.622262 ip-10-0-131-93 kubenswrapper[2564]: I0729 19:17:06.622160 2564 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-network-operator\"/\"iptables-alerter-dockercfg-bs74d\"" Jul 29 19:17:06.623082 ip-10-0-131-93 kubenswrapper[2564]: I0729 19:17:06.623061 2564 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-dns/node-resolver-lzsmp" Jul 29 19:17:06.624238 ip-10-0-131-93 kubenswrapper[2564]: I0729 19:17:06.624118 2564 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-multus\"/\"multus-ancillary-tools-dockercfg-6rqgc\"" Jul 29 19:17:06.624341 ip-10-0-131-93 kubenswrapper[2564]: I0729 19:17:06.624238 2564 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-multus\"/\"whereabouts-flatfile-config\"" Jul 29 19:17:06.624418 ip-10-0-131-93 kubenswrapper[2564]: I0729 19:17:06.624368 2564 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-multus\"/\"default-cni-sysctl-allowlist\"" Jul 29 19:17:06.624959 ip-10-0-131-93 kubenswrapper[2564]: I0729 19:17:06.624943 2564 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-cluster-node-tuning-operator/tuned-s26vt" Jul 29 19:17:06.625240 ip-10-0-131-93 kubenswrapper[2564]: I0729 19:17:06.625223 2564 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-dns\"/\"openshift-service-ca.crt\"" Jul 29 19:17:06.625744 ip-10-0-131-93 kubenswrapper[2564]: I0729 19:17:06.625725 2564 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-dns\"/\"node-resolver-dockercfg-njst7\"" Jul 29 19:17:06.626563 ip-10-0-131-93 kubenswrapper[2564]: I0729 19:17:06.626387 2564 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-dns\"/\"kube-root-ca.crt\"" Jul 29 19:17:06.626563 ip-10-0-131-93 kubenswrapper[2564]: I0729 19:17:06.626405 2564 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-image-registry/node-ca-2lv4n" Jul 29 19:17:06.627242 ip-10-0-131-93 kubenswrapper[2564]: I0729 19:17:06.627225 2564 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-cluster-node-tuning-operator\"/\"openshift-service-ca.crt\"" Jul 29 19:17:06.627468 ip-10-0-131-93 kubenswrapper[2564]: I0729 19:17:06.627254 2564 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-cluster-node-tuning-operator\"/\"tuned-dockercfg-p8jqc\"" Jul 29 19:17:06.627685 ip-10-0-131-93 kubenswrapper[2564]: I0729 19:17:06.627666 2564 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-cluster-node-tuning-operator\"/\"kube-root-ca.crt\"" Jul 29 19:17:06.628275 ip-10-0-131-93 kubenswrapper[2564]: I0729 19:17:06.628253 2564 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-zcgr9" Jul 29 19:17:06.628361 ip-10-0-131-93 kubenswrapper[2564]: E0729 19:17:06.628324 2564 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-zcgr9" podUID="5c0eec03-fbbc-40db-9c78-43c8f98534c7" Jul 29 19:17:06.629003 ip-10-0-131-93 kubenswrapper[2564]: I0729 19:17:06.628754 2564 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-image-registry\"/\"kube-root-ca.crt\"" Jul 29 19:17:06.629003 ip-10-0-131-93 kubenswrapper[2564]: I0729 19:17:06.628835 2564 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-image-registry\"/\"node-ca-dockercfg-hbdfk\"" Jul 29 19:17:06.629003 ip-10-0-131-93 kubenswrapper[2564]: I0729 19:17:06.628855 2564 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-image-registry\"/\"image-registry-certificates\"" Jul 29 19:17:06.629428 ip-10-0-131-93 kubenswrapper[2564]: I0729 19:17:06.629406 2564 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-image-registry\"/\"openshift-service-ca.crt\"" Jul 29 19:17:06.630148 ip-10-0-131-93 kubenswrapper[2564]: I0729 19:17:06.630126 2564 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-run-k8s-cni-cncf-io\" (UniqueName: \"kubernetes.io/host-path/6f081e41-049f-44dc-9aa4-47c2c509d0ff-host-run-k8s-cni-cncf-io\") pod \"multus-ccv7v\" (UID: \"6f081e41-049f-44dc-9aa4-47c2c509d0ff\") " pod="openshift-multus/multus-ccv7v" Jul 29 19:17:06.630233 ip-10-0-131-93 kubenswrapper[2564]: I0729 19:17:06.630163 2564 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-run-multus-certs\" (UniqueName: \"kubernetes.io/host-path/6f081e41-049f-44dc-9aa4-47c2c509d0ff-host-run-multus-certs\") pod \"multus-ccv7v\" (UID: \"6f081e41-049f-44dc-9aa4-47c2c509d0ff\") " pod="openshift-multus/multus-ccv7v" Jul 29 19:17:06.630233 ip-10-0-131-93 kubenswrapper[2564]: I0729 19:17:06.630187 2564 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-kubelet\" (UniqueName: \"kubernetes.io/host-path/9568e377-3546-4fbc-bae8-e0ae7766424a-host-kubelet\") pod \"ovnkube-node-kl2xl\" (UID: \"9568e377-3546-4fbc-bae8-e0ae7766424a\") " pod="openshift-ovn-kubernetes/ovnkube-node-kl2xl" Jul 29 19:17:06.630233 ip-10-0-131-93 kubenswrapper[2564]: I0729 19:17:06.630209 2564 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"env-overrides\" (UniqueName: \"kubernetes.io/configmap/9568e377-3546-4fbc-bae8-e0ae7766424a-env-overrides\") pod \"ovnkube-node-kl2xl\" (UID: \"9568e377-3546-4fbc-bae8-e0ae7766424a\") " pod="openshift-ovn-kubernetes/ovnkube-node-kl2xl" Jul 29 19:17:06.630391 ip-10-0-131-93 kubenswrapper[2564]: I0729 19:17:06.630233 2564 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"etc-selinux\" (UniqueName: \"kubernetes.io/host-path/9a4df9c4-1d87-473b-8d61-abdfb4860574-etc-selinux\") pod \"aws-ebs-csi-driver-node-z7ptq\" (UID: \"9a4df9c4-1d87-473b-8d61-abdfb4860574\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-z7ptq" Jul 29 19:17:06.630391 ip-10-0-131-93 kubenswrapper[2564]: I0729 19:17:06.630256 2564 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tuning-conf-dir\" (UniqueName: \"kubernetes.io/host-path/01073023-f0f3-4517-affc-d5c86021c64d-tuning-conf-dir\") pod \"multus-additional-cni-plugins-nmm7d\" (UID: \"01073023-f0f3-4517-affc-d5c86021c64d\") " pod="openshift-multus/multus-additional-cni-plugins-nmm7d" Jul 29 19:17:06.630391 ip-10-0-131-93 kubenswrapper[2564]: I0729 19:17:06.630280 2564 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-qgvzl\" (UniqueName: \"kubernetes.io/projected/02612aaa-a23a-4e6b-b224-123af1bb8230-kube-api-access-qgvzl\") pod \"network-check-target-464tm\" (UID: \"02612aaa-a23a-4e6b-b224-123af1bb8230\") " pod="openshift-network-diagnostics/network-check-target-464tm" Jul 29 19:17:06.630391 ip-10-0-131-93 kubenswrapper[2564]: I0729 19:17:06.630332 2564 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-run-netns\" (UniqueName: \"kubernetes.io/host-path/9568e377-3546-4fbc-bae8-e0ae7766424a-host-run-netns\") pod \"ovnkube-node-kl2xl\" (UID: \"9568e377-3546-4fbc-bae8-e0ae7766424a\") " pod="openshift-ovn-kubernetes/ovnkube-node-kl2xl" Jul 29 19:17:06.630391 ip-10-0-131-93 kubenswrapper[2564]: I0729 19:17:06.630362 2564 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"multus-daemon-config\" (UniqueName: \"kubernetes.io/configmap/6f081e41-049f-44dc-9aa4-47c2c509d0ff-multus-daemon-config\") pod \"multus-ccv7v\" (UID: \"6f081e41-049f-44dc-9aa4-47c2c509d0ff\") " pod="openshift-multus/multus-ccv7v" Jul 29 19:17:06.630573 ip-10-0-131-93 kubenswrapper[2564]: I0729 19:17:06.630409 2564 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-slash\" (UniqueName: \"kubernetes.io/host-path/9568e377-3546-4fbc-bae8-e0ae7766424a-host-slash\") pod \"ovnkube-node-kl2xl\" (UID: \"9568e377-3546-4fbc-bae8-e0ae7766424a\") " pod="openshift-ovn-kubernetes/ovnkube-node-kl2xl" Jul 29 19:17:06.630573 ip-10-0-131-93 kubenswrapper[2564]: I0729 19:17:06.630434 2564 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-run-ovn-kubernetes\" (UniqueName: \"kubernetes.io/host-path/9568e377-3546-4fbc-bae8-e0ae7766424a-host-run-ovn-kubernetes\") pod \"ovnkube-node-kl2xl\" (UID: \"9568e377-3546-4fbc-bae8-e0ae7766424a\") " pod="openshift-ovn-kubernetes/ovnkube-node-kl2xl" Jul 29 19:17:06.630573 ip-10-0-131-93 kubenswrapper[2564]: I0729 19:17:06.630456 2564 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"ovn-node-metrics-cert\" (UniqueName: \"kubernetes.io/secret/9568e377-3546-4fbc-bae8-e0ae7766424a-ovn-node-metrics-cert\") pod \"ovnkube-node-kl2xl\" (UID: \"9568e377-3546-4fbc-bae8-e0ae7766424a\") " pod="openshift-ovn-kubernetes/ovnkube-node-kl2xl" Jul 29 19:17:06.630573 ip-10-0-131-93 kubenswrapper[2564]: I0729 19:17:06.630480 2564 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"ovnkube-config\" (UniqueName: \"kubernetes.io/configmap/9568e377-3546-4fbc-bae8-e0ae7766424a-ovnkube-config\") pod \"ovnkube-node-kl2xl\" (UID: \"9568e377-3546-4fbc-bae8-e0ae7766424a\") " pod="openshift-ovn-kubernetes/ovnkube-node-kl2xl" Jul 29 19:17:06.630573 ip-10-0-131-93 kubenswrapper[2564]: I0729 19:17:06.630504 2564 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-6fg74\" (UniqueName: \"kubernetes.io/projected/01073023-f0f3-4517-affc-d5c86021c64d-kube-api-access-6fg74\") pod \"multus-additional-cni-plugins-nmm7d\" (UID: \"01073023-f0f3-4517-affc-d5c86021c64d\") " pod="openshift-multus/multus-additional-cni-plugins-nmm7d" Jul 29 19:17:06.630573 ip-10-0-131-93 kubenswrapper[2564]: I0729 19:17:06.630531 2564 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"iptables-alerter-script\" (UniqueName: \"kubernetes.io/configmap/723728d7-dd35-40ea-bb53-b597ec0e8ef1-iptables-alerter-script\") pod \"iptables-alerter-5n889\" (UID: \"723728d7-dd35-40ea-bb53-b597ec0e8ef1\") " pod="openshift-network-operator/iptables-alerter-5n889" Jul 29 19:17:06.630573 ip-10-0-131-93 kubenswrapper[2564]: I0729 19:17:06.630553 2564 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"multus-conf-dir\" (UniqueName: \"kubernetes.io/host-path/6f081e41-049f-44dc-9aa4-47c2c509d0ff-multus-conf-dir\") pod \"multus-ccv7v\" (UID: \"6f081e41-049f-44dc-9aa4-47c2c509d0ff\") " pod="openshift-multus/multus-ccv7v" Jul 29 19:17:06.630766 ip-10-0-131-93 kubenswrapper[2564]: I0729 19:17:06.630573 2564 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"node-log\" (UniqueName: \"kubernetes.io/host-path/9568e377-3546-4fbc-bae8-e0ae7766424a-node-log\") pod \"ovnkube-node-kl2xl\" (UID: \"9568e377-3546-4fbc-bae8-e0ae7766424a\") " pod="openshift-ovn-kubernetes/ovnkube-node-kl2xl" Jul 29 19:17:06.630766 ip-10-0-131-93 kubenswrapper[2564]: I0729 19:17:06.630597 2564 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-var-lib-cni-networks-ovn-kubernetes\" (UniqueName: \"kubernetes.io/host-path/9568e377-3546-4fbc-bae8-e0ae7766424a-host-var-lib-cni-networks-ovn-kubernetes\") pod \"ovnkube-node-kl2xl\" (UID: \"9568e377-3546-4fbc-bae8-e0ae7766424a\") " pod="openshift-ovn-kubernetes/ovnkube-node-kl2xl" Jul 29 19:17:06.630766 ip-10-0-131-93 kubenswrapper[2564]: I0729 19:17:06.630621 2564 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"socket-dir\" (UniqueName: \"kubernetes.io/host-path/9a4df9c4-1d87-473b-8d61-abdfb4860574-socket-dir\") pod \"aws-ebs-csi-driver-node-z7ptq\" (UID: \"9a4df9c4-1d87-473b-8d61-abdfb4860574\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-z7ptq" Jul 29 19:17:06.630766 ip-10-0-131-93 kubenswrapper[2564]: I0729 19:17:06.630657 2564 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"agent-certs\" (UniqueName: \"kubernetes.io/secret/faa8c546-c75b-4e10-93b4-661aa2b5f53e-agent-certs\") pod \"konnectivity-agent-444jx\" (UID: \"faa8c546-c75b-4e10-93b4-661aa2b5f53e\") " pod="kube-system/konnectivity-agent-444jx" Jul 29 19:17:06.630766 ip-10-0-131-93 kubenswrapper[2564]: I0729 19:17:06.630679 2564 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-var-lib-kubelet\" (UniqueName: \"kubernetes.io/host-path/6f081e41-049f-44dc-9aa4-47c2c509d0ff-host-var-lib-kubelet\") pod \"multus-ccv7v\" (UID: \"6f081e41-049f-44dc-9aa4-47c2c509d0ff\") " pod="openshift-multus/multus-ccv7v" Jul 29 19:17:06.630766 ip-10-0-131-93 kubenswrapper[2564]: I0729 19:17:06.630701 2564 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-jzc6t\" (UniqueName: \"kubernetes.io/projected/6f081e41-049f-44dc-9aa4-47c2c509d0ff-kube-api-access-jzc6t\") pod \"multus-ccv7v\" (UID: \"6f081e41-049f-44dc-9aa4-47c2c509d0ff\") " pod="openshift-multus/multus-ccv7v" Jul 29 19:17:06.630766 ip-10-0-131-93 kubenswrapper[2564]: I0729 19:17:06.630738 2564 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"var-lib-openvswitch\" (UniqueName: \"kubernetes.io/host-path/9568e377-3546-4fbc-bae8-e0ae7766424a-var-lib-openvswitch\") pod \"ovnkube-node-kl2xl\" (UID: \"9568e377-3546-4fbc-bae8-e0ae7766424a\") " pod="openshift-ovn-kubernetes/ovnkube-node-kl2xl" Jul 29 19:17:06.630966 ip-10-0-131-93 kubenswrapper[2564]: I0729 19:17:06.630760 2564 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"run-openvswitch\" (UniqueName: \"kubernetes.io/host-path/9568e377-3546-4fbc-bae8-e0ae7766424a-run-openvswitch\") pod \"ovnkube-node-kl2xl\" (UID: \"9568e377-3546-4fbc-bae8-e0ae7766424a\") " pod="openshift-ovn-kubernetes/ovnkube-node-kl2xl" Jul 29 19:17:06.630966 ip-10-0-131-93 kubenswrapper[2564]: I0729 19:17:06.630790 2564 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"run-ovn\" (UniqueName: \"kubernetes.io/host-path/9568e377-3546-4fbc-bae8-e0ae7766424a-run-ovn\") pod \"ovnkube-node-kl2xl\" (UID: \"9568e377-3546-4fbc-bae8-e0ae7766424a\") " pod="openshift-ovn-kubernetes/ovnkube-node-kl2xl" Jul 29 19:17:06.630966 ip-10-0-131-93 kubenswrapper[2564]: I0729 19:17:06.630816 2564 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"run-systemd\" (UniqueName: \"kubernetes.io/host-path/9568e377-3546-4fbc-bae8-e0ae7766424a-run-systemd\") pod \"ovnkube-node-kl2xl\" (UID: \"9568e377-3546-4fbc-bae8-e0ae7766424a\") " pod="openshift-ovn-kubernetes/ovnkube-node-kl2xl" Jul 29 19:17:06.630966 ip-10-0-131-93 kubenswrapper[2564]: I0729 19:17:06.630853 2564 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-var-lib-cni-bin\" (UniqueName: \"kubernetes.io/host-path/6f081e41-049f-44dc-9aa4-47c2c509d0ff-host-var-lib-cni-bin\") pod \"multus-ccv7v\" (UID: \"6f081e41-049f-44dc-9aa4-47c2c509d0ff\") " pod="openshift-multus/multus-ccv7v" Jul 29 19:17:06.630966 ip-10-0-131-93 kubenswrapper[2564]: I0729 19:17:06.630881 2564 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"log-socket\" (UniqueName: \"kubernetes.io/host-path/9568e377-3546-4fbc-bae8-e0ae7766424a-log-socket\") pod \"ovnkube-node-kl2xl\" (UID: \"9568e377-3546-4fbc-bae8-e0ae7766424a\") " pod="openshift-ovn-kubernetes/ovnkube-node-kl2xl" Jul 29 19:17:06.630966 ip-10-0-131-93 kubenswrapper[2564]: I0729 19:17:06.630904 2564 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kubelet-dir\" (UniqueName: \"kubernetes.io/host-path/9a4df9c4-1d87-473b-8d61-abdfb4860574-kubelet-dir\") pod \"aws-ebs-csi-driver-node-z7ptq\" (UID: \"9a4df9c4-1d87-473b-8d61-abdfb4860574\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-z7ptq" Jul 29 19:17:06.630966 ip-10-0-131-93 kubenswrapper[2564]: I0729 19:17:06.630928 2564 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"konnectivity-ca\" (UniqueName: \"kubernetes.io/configmap/faa8c546-c75b-4e10-93b4-661aa2b5f53e-konnectivity-ca\") pod \"konnectivity-agent-444jx\" (UID: \"faa8c546-c75b-4e10-93b4-661aa2b5f53e\") " pod="kube-system/konnectivity-agent-444jx" Jul 29 19:17:06.631284 ip-10-0-131-93 kubenswrapper[2564]: I0729 19:17:06.630992 2564 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-zlbv2\" (UniqueName: \"kubernetes.io/projected/723728d7-dd35-40ea-bb53-b597ec0e8ef1-kube-api-access-zlbv2\") pod \"iptables-alerter-5n889\" (UID: \"723728d7-dd35-40ea-bb53-b597ec0e8ef1\") " pod="openshift-network-operator/iptables-alerter-5n889" Jul 29 19:17:06.631284 ip-10-0-131-93 kubenswrapper[2564]: I0729 19:17:06.631123 2564 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"cni-binary-copy\" (UniqueName: \"kubernetes.io/configmap/01073023-f0f3-4517-affc-d5c86021c64d-cni-binary-copy\") pod \"multus-additional-cni-plugins-nmm7d\" (UID: \"01073023-f0f3-4517-affc-d5c86021c64d\") " pod="openshift-multus/multus-additional-cni-plugins-nmm7d" Jul 29 19:17:06.631284 ip-10-0-131-93 kubenswrapper[2564]: I0729 19:17:06.631190 2564 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"multus-cni-dir\" (UniqueName: \"kubernetes.io/host-path/6f081e41-049f-44dc-9aa4-47c2c509d0ff-multus-cni-dir\") pod \"multus-ccv7v\" (UID: \"6f081e41-049f-44dc-9aa4-47c2c509d0ff\") " pod="openshift-multus/multus-ccv7v" Jul 29 19:17:06.631284 ip-10-0-131-93 kubenswrapper[2564]: I0729 19:17:06.631219 2564 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-7drmm\" (UniqueName: \"kubernetes.io/projected/9568e377-3546-4fbc-bae8-e0ae7766424a-kube-api-access-7drmm\") pod \"ovnkube-node-kl2xl\" (UID: \"9568e377-3546-4fbc-bae8-e0ae7766424a\") " pod="openshift-ovn-kubernetes/ovnkube-node-kl2xl" Jul 29 19:17:06.631284 ip-10-0-131-93 kubenswrapper[2564]: I0729 19:17:06.631258 2564 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"cnibin\" (UniqueName: \"kubernetes.io/host-path/01073023-f0f3-4517-affc-d5c86021c64d-cnibin\") pod \"multus-additional-cni-plugins-nmm7d\" (UID: \"01073023-f0f3-4517-affc-d5c86021c64d\") " pod="openshift-multus/multus-additional-cni-plugins-nmm7d" Jul 29 19:17:06.631544 ip-10-0-131-93 kubenswrapper[2564]: I0729 19:17:06.631291 2564 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-cni-netd\" (UniqueName: \"kubernetes.io/host-path/9568e377-3546-4fbc-bae8-e0ae7766424a-host-cni-netd\") pod \"ovnkube-node-kl2xl\" (UID: \"9568e377-3546-4fbc-bae8-e0ae7766424a\") " pod="openshift-ovn-kubernetes/ovnkube-node-kl2xl" Jul 29 19:17:06.631544 ip-10-0-131-93 kubenswrapper[2564]: I0729 19:17:06.631325 2564 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-65p97\" (UniqueName: \"kubernetes.io/projected/9a4df9c4-1d87-473b-8d61-abdfb4860574-kube-api-access-65p97\") pod \"aws-ebs-csi-driver-node-z7ptq\" (UID: \"9a4df9c4-1d87-473b-8d61-abdfb4860574\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-z7ptq" Jul 29 19:17:06.631544 ip-10-0-131-93 kubenswrapper[2564]: I0729 19:17:06.631351 2564 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"os-release\" (UniqueName: \"kubernetes.io/host-path/01073023-f0f3-4517-affc-d5c86021c64d-os-release\") pod \"multus-additional-cni-plugins-nmm7d\" (UID: \"01073023-f0f3-4517-affc-d5c86021c64d\") " pod="openshift-multus/multus-additional-cni-plugins-nmm7d" Jul 29 19:17:06.631544 ip-10-0-131-93 kubenswrapper[2564]: I0729 19:17:06.631400 2564 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"whereabouts-flatfile-configmap\" (UniqueName: \"kubernetes.io/configmap/01073023-f0f3-4517-affc-d5c86021c64d-whereabouts-flatfile-configmap\") pod \"multus-additional-cni-plugins-nmm7d\" (UID: \"01073023-f0f3-4517-affc-d5c86021c64d\") " pod="openshift-multus/multus-additional-cni-plugins-nmm7d" Jul 29 19:17:06.631544 ip-10-0-131-93 kubenswrapper[2564]: I0729 19:17:06.631434 2564 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"systemd-units\" (UniqueName: \"kubernetes.io/host-path/9568e377-3546-4fbc-bae8-e0ae7766424a-systemd-units\") pod \"ovnkube-node-kl2xl\" (UID: \"9568e377-3546-4fbc-bae8-e0ae7766424a\") " pod="openshift-ovn-kubernetes/ovnkube-node-kl2xl" Jul 29 19:17:06.631544 ip-10-0-131-93 kubenswrapper[2564]: I0729 19:17:06.631449 2564 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-cni-bin\" (UniqueName: \"kubernetes.io/host-path/9568e377-3546-4fbc-bae8-e0ae7766424a-host-cni-bin\") pod \"ovnkube-node-kl2xl\" (UID: \"9568e377-3546-4fbc-bae8-e0ae7766424a\") " pod="openshift-ovn-kubernetes/ovnkube-node-kl2xl" Jul 29 19:17:06.631544 ip-10-0-131-93 kubenswrapper[2564]: I0729 19:17:06.631488 2564 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"ovnkube-script-lib\" (UniqueName: \"kubernetes.io/configmap/9568e377-3546-4fbc-bae8-e0ae7766424a-ovnkube-script-lib\") pod \"ovnkube-node-kl2xl\" (UID: \"9568e377-3546-4fbc-bae8-e0ae7766424a\") " pod="openshift-ovn-kubernetes/ovnkube-node-kl2xl" Jul 29 19:17:06.631544 ip-10-0-131-93 kubenswrapper[2564]: I0729 19:17:06.631542 2564 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"registration-dir\" (UniqueName: \"kubernetes.io/host-path/9a4df9c4-1d87-473b-8d61-abdfb4860574-registration-dir\") pod \"aws-ebs-csi-driver-node-z7ptq\" (UID: \"9a4df9c4-1d87-473b-8d61-abdfb4860574\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-z7ptq" Jul 29 19:17:06.631943 ip-10-0-131-93 kubenswrapper[2564]: I0729 19:17:06.631583 2564 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-slash\" (UniqueName: \"kubernetes.io/host-path/723728d7-dd35-40ea-bb53-b597ec0e8ef1-host-slash\") pod \"iptables-alerter-5n889\" (UID: \"723728d7-dd35-40ea-bb53-b597ec0e8ef1\") " pod="openshift-network-operator/iptables-alerter-5n889" Jul 29 19:17:06.631943 ip-10-0-131-93 kubenswrapper[2564]: I0729 19:17:06.631625 2564 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"cni-sysctl-allowlist\" (UniqueName: \"kubernetes.io/configmap/01073023-f0f3-4517-affc-d5c86021c64d-cni-sysctl-allowlist\") pod \"multus-additional-cni-plugins-nmm7d\" (UID: \"01073023-f0f3-4517-affc-d5c86021c64d\") " pod="openshift-multus/multus-additional-cni-plugins-nmm7d" Jul 29 19:17:06.631943 ip-10-0-131-93 kubenswrapper[2564]: I0729 19:17:06.631681 2564 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"system-cni-dir\" (UniqueName: \"kubernetes.io/host-path/6f081e41-049f-44dc-9aa4-47c2c509d0ff-system-cni-dir\") pod \"multus-ccv7v\" (UID: \"6f081e41-049f-44dc-9aa4-47c2c509d0ff\") " pod="openshift-multus/multus-ccv7v" Jul 29 19:17:06.631943 ip-10-0-131-93 kubenswrapper[2564]: I0729 19:17:06.631719 2564 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"cnibin\" (UniqueName: \"kubernetes.io/host-path/6f081e41-049f-44dc-9aa4-47c2c509d0ff-cnibin\") pod \"multus-ccv7v\" (UID: \"6f081e41-049f-44dc-9aa4-47c2c509d0ff\") " pod="openshift-multus/multus-ccv7v" Jul 29 19:17:06.631943 ip-10-0-131-93 kubenswrapper[2564]: I0729 19:17:06.631755 2564 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"cni-binary-copy\" (UniqueName: \"kubernetes.io/configmap/6f081e41-049f-44dc-9aa4-47c2c509d0ff-cni-binary-copy\") pod \"multus-ccv7v\" (UID: \"6f081e41-049f-44dc-9aa4-47c2c509d0ff\") " pod="openshift-multus/multus-ccv7v" Jul 29 19:17:06.631943 ip-10-0-131-93 kubenswrapper[2564]: I0729 19:17:06.631806 2564 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-run-netns\" (UniqueName: \"kubernetes.io/host-path/6f081e41-049f-44dc-9aa4-47c2c509d0ff-host-run-netns\") pod \"multus-ccv7v\" (UID: \"6f081e41-049f-44dc-9aa4-47c2c509d0ff\") " pod="openshift-multus/multus-ccv7v" Jul 29 19:17:06.631943 ip-10-0-131-93 kubenswrapper[2564]: I0729 19:17:06.631842 2564 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-var-lib-cni-multus\" (UniqueName: \"kubernetes.io/host-path/6f081e41-049f-44dc-9aa4-47c2c509d0ff-host-var-lib-cni-multus\") pod \"multus-ccv7v\" (UID: \"6f081e41-049f-44dc-9aa4-47c2c509d0ff\") " pod="openshift-multus/multus-ccv7v" Jul 29 19:17:06.631943 ip-10-0-131-93 kubenswrapper[2564]: I0729 19:17:06.631873 2564 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"hostroot\" (UniqueName: \"kubernetes.io/host-path/6f081e41-049f-44dc-9aa4-47c2c509d0ff-hostroot\") pod \"multus-ccv7v\" (UID: \"6f081e41-049f-44dc-9aa4-47c2c509d0ff\") " pod="openshift-multus/multus-ccv7v" Jul 29 19:17:06.631943 ip-10-0-131-93 kubenswrapper[2564]: I0729 19:17:06.631897 2564 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-xfrc2\" (UniqueName: \"kubernetes.io/projected/ae220db3-299c-481a-9591-01a8f3873d9a-kube-api-access-xfrc2\") pod \"network-metrics-daemon-hkxvx\" (UID: \"ae220db3-299c-481a-9591-01a8f3873d9a\") " pod="openshift-multus/network-metrics-daemon-hkxvx" Jul 29 19:17:06.631943 ip-10-0-131-93 kubenswrapper[2564]: I0729 19:17:06.631925 2564 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"system-cni-dir\" (UniqueName: \"kubernetes.io/host-path/01073023-f0f3-4517-affc-d5c86021c64d-system-cni-dir\") pod \"multus-additional-cni-plugins-nmm7d\" (UID: \"01073023-f0f3-4517-affc-d5c86021c64d\") " pod="openshift-multus/multus-additional-cni-plugins-nmm7d" Jul 29 19:17:06.632340 ip-10-0-131-93 kubenswrapper[2564]: I0729 19:17:06.631950 2564 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"etc-openvswitch\" (UniqueName: \"kubernetes.io/host-path/9568e377-3546-4fbc-bae8-e0ae7766424a-etc-openvswitch\") pod \"ovnkube-node-kl2xl\" (UID: \"9568e377-3546-4fbc-bae8-e0ae7766424a\") " pod="openshift-ovn-kubernetes/ovnkube-node-kl2xl" Jul 29 19:17:06.632340 ip-10-0-131-93 kubenswrapper[2564]: I0729 19:17:06.631982 2564 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"etc-kubernetes\" (UniqueName: \"kubernetes.io/host-path/6f081e41-049f-44dc-9aa4-47c2c509d0ff-etc-kubernetes\") pod \"multus-ccv7v\" (UID: \"6f081e41-049f-44dc-9aa4-47c2c509d0ff\") " pod="openshift-multus/multus-ccv7v" Jul 29 19:17:06.632340 ip-10-0-131-93 kubenswrapper[2564]: I0729 19:17:06.632009 2564 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"device-dir\" (UniqueName: \"kubernetes.io/host-path/9a4df9c4-1d87-473b-8d61-abdfb4860574-device-dir\") pod \"aws-ebs-csi-driver-node-z7ptq\" (UID: \"9a4df9c4-1d87-473b-8d61-abdfb4860574\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-z7ptq" Jul 29 19:17:06.632340 ip-10-0-131-93 kubenswrapper[2564]: I0729 19:17:06.632053 2564 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/ae220db3-299c-481a-9591-01a8f3873d9a-metrics-certs\") pod \"network-metrics-daemon-hkxvx\" (UID: \"ae220db3-299c-481a-9591-01a8f3873d9a\") " pod="openshift-multus/network-metrics-daemon-hkxvx" Jul 29 19:17:06.632340 ip-10-0-131-93 kubenswrapper[2564]: I0729 19:17:06.632076 2564 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"os-release\" (UniqueName: \"kubernetes.io/host-path/6f081e41-049f-44dc-9aa4-47c2c509d0ff-os-release\") pod \"multus-ccv7v\" (UID: \"6f081e41-049f-44dc-9aa4-47c2c509d0ff\") " pod="openshift-multus/multus-ccv7v" Jul 29 19:17:06.632340 ip-10-0-131-93 kubenswrapper[2564]: I0729 19:17:06.632096 2564 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"multus-socket-dir-parent\" (UniqueName: \"kubernetes.io/host-path/6f081e41-049f-44dc-9aa4-47c2c509d0ff-multus-socket-dir-parent\") pod \"multus-ccv7v\" (UID: \"6f081e41-049f-44dc-9aa4-47c2c509d0ff\") " pod="openshift-multus/multus-ccv7v" Jul 29 19:17:06.632340 ip-10-0-131-93 kubenswrapper[2564]: I0729 19:17:06.632117 2564 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"sys-fs\" (UniqueName: \"kubernetes.io/host-path/9a4df9c4-1d87-473b-8d61-abdfb4860574-sys-fs\") pod \"aws-ebs-csi-driver-node-z7ptq\" (UID: \"9a4df9c4-1d87-473b-8d61-abdfb4860574\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-z7ptq" Jul 29 19:17:06.632867 ip-10-0-131-93 kubenswrapper[2564]: I0729 19:17:06.632845 2564 reflector.go:439] "Caches populated" logger="kubernetes.io/kubelet-serving" type="*v1.CertificateSigningRequest" reflector="k8s.io/client-go/tools/watch/informerwatcher.go:162" Jul 29 19:17:06.654293 ip-10-0-131-93 kubenswrapper[2564]: I0729 19:17:06.654257 2564 csr.go:274] "Certificate signing request is approved, waiting to be issued" logger="kubernetes.io/kubelet-serving" csr="csr-r6cxn" Jul 29 19:17:06.661992 ip-10-0-131-93 kubenswrapper[2564]: I0729 19:17:06.661960 2564 csr.go:270] "Certificate signing request is issued" logger="kubernetes.io/kubelet-serving" csr="csr-r6cxn" Jul 29 19:17:06.721543 ip-10-0-131-93 kubenswrapper[2564]: I0729 19:17:06.721472 2564 desired_state_of_world_populator.go:154] "Finished populating initial desired state of world" Jul 29 19:17:06.732886 ip-10-0-131-93 kubenswrapper[2564]: I0729 19:17:06.732851 2564 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"var-lib-openvswitch\" (UniqueName: \"kubernetes.io/host-path/9568e377-3546-4fbc-bae8-e0ae7766424a-var-lib-openvswitch\") pod \"ovnkube-node-kl2xl\" (UID: \"9568e377-3546-4fbc-bae8-e0ae7766424a\") " pod="openshift-ovn-kubernetes/ovnkube-node-kl2xl" Jul 29 19:17:06.733021 ip-10-0-131-93 kubenswrapper[2564]: I0729 19:17:06.732905 2564 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"run-openvswitch\" (UniqueName: \"kubernetes.io/host-path/9568e377-3546-4fbc-bae8-e0ae7766424a-run-openvswitch\") pod \"ovnkube-node-kl2xl\" (UID: \"9568e377-3546-4fbc-bae8-e0ae7766424a\") " pod="openshift-ovn-kubernetes/ovnkube-node-kl2xl" Jul 29 19:17:06.733021 ip-10-0-131-93 kubenswrapper[2564]: I0729 19:17:06.732930 2564 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"run-ovn\" (UniqueName: \"kubernetes.io/host-path/9568e377-3546-4fbc-bae8-e0ae7766424a-run-ovn\") pod \"ovnkube-node-kl2xl\" (UID: \"9568e377-3546-4fbc-bae8-e0ae7766424a\") " pod="openshift-ovn-kubernetes/ovnkube-node-kl2xl" Jul 29 19:17:06.733021 ip-10-0-131-93 kubenswrapper[2564]: I0729 19:17:06.732960 2564 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kubelet-config\" (UniqueName: \"kubernetes.io/host-path/5c0eec03-fbbc-40db-9c78-43c8f98534c7-kubelet-config\") pod \"global-pull-secret-syncer-zcgr9\" (UID: \"5c0eec03-fbbc-40db-9c78-43c8f98534c7\") " pod="kube-system/global-pull-secret-syncer-zcgr9" Jul 29 19:17:06.733021 ip-10-0-131-93 kubenswrapper[2564]: I0729 19:17:06.732939 2564 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"var-lib-openvswitch\" (UniqueName: \"kubernetes.io/host-path/9568e377-3546-4fbc-bae8-e0ae7766424a-var-lib-openvswitch\") pod \"ovnkube-node-kl2xl\" (UID: \"9568e377-3546-4fbc-bae8-e0ae7766424a\") " pod="openshift-ovn-kubernetes/ovnkube-node-kl2xl" Jul 29 19:17:06.733021 ip-10-0-131-93 kubenswrapper[2564]: I0729 19:17:06.732986 2564 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"run-systemd\" (UniqueName: \"kubernetes.io/host-path/9568e377-3546-4fbc-bae8-e0ae7766424a-run-systemd\") pod \"ovnkube-node-kl2xl\" (UID: \"9568e377-3546-4fbc-bae8-e0ae7766424a\") " pod="openshift-ovn-kubernetes/ovnkube-node-kl2xl" Jul 29 19:17:06.733021 ip-10-0-131-93 kubenswrapper[2564]: I0729 19:17:06.733002 2564 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"run-openvswitch\" (UniqueName: \"kubernetes.io/host-path/9568e377-3546-4fbc-bae8-e0ae7766424a-run-openvswitch\") pod \"ovnkube-node-kl2xl\" (UID: \"9568e377-3546-4fbc-bae8-e0ae7766424a\") " pod="openshift-ovn-kubernetes/ovnkube-node-kl2xl" Jul 29 19:17:06.733289 ip-10-0-131-93 kubenswrapper[2564]: I0729 19:17:06.733032 2564 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"run-systemd\" (UniqueName: \"kubernetes.io/host-path/9568e377-3546-4fbc-bae8-e0ae7766424a-run-systemd\") pod \"ovnkube-node-kl2xl\" (UID: \"9568e377-3546-4fbc-bae8-e0ae7766424a\") " pod="openshift-ovn-kubernetes/ovnkube-node-kl2xl" Jul 29 19:17:06.733289 ip-10-0-131-93 kubenswrapper[2564]: I0729 19:17:06.733054 2564 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-var-lib-cni-bin\" (UniqueName: \"kubernetes.io/host-path/6f081e41-049f-44dc-9aa4-47c2c509d0ff-host-var-lib-cni-bin\") pod \"multus-ccv7v\" (UID: \"6f081e41-049f-44dc-9aa4-47c2c509d0ff\") " pod="openshift-multus/multus-ccv7v" Jul 29 19:17:06.733289 ip-10-0-131-93 kubenswrapper[2564]: I0729 19:17:06.733053 2564 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"run-ovn\" (UniqueName: \"kubernetes.io/host-path/9568e377-3546-4fbc-bae8-e0ae7766424a-run-ovn\") pod \"ovnkube-node-kl2xl\" (UID: \"9568e377-3546-4fbc-bae8-e0ae7766424a\") " pod="openshift-ovn-kubernetes/ovnkube-node-kl2xl" Jul 29 19:17:06.733289 ip-10-0-131-93 kubenswrapper[2564]: I0729 19:17:06.733068 2564 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"log-socket\" (UniqueName: \"kubernetes.io/host-path/9568e377-3546-4fbc-bae8-e0ae7766424a-log-socket\") pod \"ovnkube-node-kl2xl\" (UID: \"9568e377-3546-4fbc-bae8-e0ae7766424a\") " pod="openshift-ovn-kubernetes/ovnkube-node-kl2xl" Jul 29 19:17:06.733289 ip-10-0-131-93 kubenswrapper[2564]: I0729 19:17:06.733087 2564 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kubelet-dir\" (UniqueName: \"kubernetes.io/host-path/9a4df9c4-1d87-473b-8d61-abdfb4860574-kubelet-dir\") pod \"aws-ebs-csi-driver-node-z7ptq\" (UID: \"9a4df9c4-1d87-473b-8d61-abdfb4860574\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-z7ptq" Jul 29 19:17:06.733289 ip-10-0-131-93 kubenswrapper[2564]: I0729 19:17:06.733110 2564 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"konnectivity-ca\" (UniqueName: \"kubernetes.io/configmap/faa8c546-c75b-4e10-93b4-661aa2b5f53e-konnectivity-ca\") pod \"konnectivity-agent-444jx\" (UID: \"faa8c546-c75b-4e10-93b4-661aa2b5f53e\") " pod="kube-system/konnectivity-agent-444jx" Jul 29 19:17:06.733289 ip-10-0-131-93 kubenswrapper[2564]: I0729 19:17:06.733117 2564 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"host-var-lib-cni-bin\" (UniqueName: \"kubernetes.io/host-path/6f081e41-049f-44dc-9aa4-47c2c509d0ff-host-var-lib-cni-bin\") pod \"multus-ccv7v\" (UID: \"6f081e41-049f-44dc-9aa4-47c2c509d0ff\") " pod="openshift-multus/multus-ccv7v" Jul 29 19:17:06.733289 ip-10-0-131-93 kubenswrapper[2564]: I0729 19:17:06.733134 2564 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-zlbv2\" (UniqueName: \"kubernetes.io/projected/723728d7-dd35-40ea-bb53-b597ec0e8ef1-kube-api-access-zlbv2\") pod \"iptables-alerter-5n889\" (UID: \"723728d7-dd35-40ea-bb53-b597ec0e8ef1\") " pod="openshift-network-operator/iptables-alerter-5n889" Jul 29 19:17:06.733289 ip-10-0-131-93 kubenswrapper[2564]: I0729 19:17:06.733139 2564 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"log-socket\" (UniqueName: \"kubernetes.io/host-path/9568e377-3546-4fbc-bae8-e0ae7766424a-log-socket\") pod \"ovnkube-node-kl2xl\" (UID: \"9568e377-3546-4fbc-bae8-e0ae7766424a\") " pod="openshift-ovn-kubernetes/ovnkube-node-kl2xl" Jul 29 19:17:06.733289 ip-10-0-131-93 kubenswrapper[2564]: I0729 19:17:06.733160 2564 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"cni-binary-copy\" (UniqueName: \"kubernetes.io/configmap/01073023-f0f3-4517-affc-d5c86021c64d-cni-binary-copy\") pod \"multus-additional-cni-plugins-nmm7d\" (UID: \"01073023-f0f3-4517-affc-d5c86021c64d\") " pod="openshift-multus/multus-additional-cni-plugins-nmm7d" Jul 29 19:17:06.733289 ip-10-0-131-93 kubenswrapper[2564]: I0729 19:17:06.733179 2564 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kubelet-dir\" (UniqueName: \"kubernetes.io/host-path/9a4df9c4-1d87-473b-8d61-abdfb4860574-kubelet-dir\") pod \"aws-ebs-csi-driver-node-z7ptq\" (UID: \"9a4df9c4-1d87-473b-8d61-abdfb4860574\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-z7ptq" Jul 29 19:17:06.733289 ip-10-0-131-93 kubenswrapper[2564]: I0729 19:17:06.733279 2564 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"multus-cni-dir\" (UniqueName: \"kubernetes.io/host-path/6f081e41-049f-44dc-9aa4-47c2c509d0ff-multus-cni-dir\") pod \"multus-ccv7v\" (UID: \"6f081e41-049f-44dc-9aa4-47c2c509d0ff\") " pod="openshift-multus/multus-ccv7v" Jul 29 19:17:06.733815 ip-10-0-131-93 kubenswrapper[2564]: I0729 19:17:06.733312 2564 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-7drmm\" (UniqueName: \"kubernetes.io/projected/9568e377-3546-4fbc-bae8-e0ae7766424a-kube-api-access-7drmm\") pod \"ovnkube-node-kl2xl\" (UID: \"9568e377-3546-4fbc-bae8-e0ae7766424a\") " pod="openshift-ovn-kubernetes/ovnkube-node-kl2xl" Jul 29 19:17:06.733815 ip-10-0-131-93 kubenswrapper[2564]: I0729 19:17:06.733336 2564 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"cnibin\" (UniqueName: \"kubernetes.io/host-path/01073023-f0f3-4517-affc-d5c86021c64d-cnibin\") pod \"multus-additional-cni-plugins-nmm7d\" (UID: \"01073023-f0f3-4517-affc-d5c86021c64d\") " pod="openshift-multus/multus-additional-cni-plugins-nmm7d" Jul 29 19:17:06.733815 ip-10-0-131-93 kubenswrapper[2564]: I0729 19:17:06.733364 2564 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tmp\" (UniqueName: \"kubernetes.io/empty-dir/afd27443-e808-4fe1-b68f-140ec2d0c81f-tmp\") pod \"tuned-s26vt\" (UID: \"afd27443-e808-4fe1-b68f-140ec2d0c81f\") " pod="openshift-cluster-node-tuning-operator/tuned-s26vt" Jul 29 19:17:06.733815 ip-10-0-131-93 kubenswrapper[2564]: I0729 19:17:06.733407 2564 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-cni-netd\" (UniqueName: \"kubernetes.io/host-path/9568e377-3546-4fbc-bae8-e0ae7766424a-host-cni-netd\") pod \"ovnkube-node-kl2xl\" (UID: \"9568e377-3546-4fbc-bae8-e0ae7766424a\") " pod="openshift-ovn-kubernetes/ovnkube-node-kl2xl" Jul 29 19:17:06.733815 ip-10-0-131-93 kubenswrapper[2564]: I0729 19:17:06.733431 2564 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-65p97\" (UniqueName: \"kubernetes.io/projected/9a4df9c4-1d87-473b-8d61-abdfb4860574-kube-api-access-65p97\") pod \"aws-ebs-csi-driver-node-z7ptq\" (UID: \"9a4df9c4-1d87-473b-8d61-abdfb4860574\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-z7ptq" Jul 29 19:17:06.733815 ip-10-0-131-93 kubenswrapper[2564]: I0729 19:17:06.733455 2564 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"os-release\" (UniqueName: \"kubernetes.io/host-path/01073023-f0f3-4517-affc-d5c86021c64d-os-release\") pod \"multus-additional-cni-plugins-nmm7d\" (UID: \"01073023-f0f3-4517-affc-d5c86021c64d\") " pod="openshift-multus/multus-additional-cni-plugins-nmm7d" Jul 29 19:17:06.733815 ip-10-0-131-93 kubenswrapper[2564]: I0729 19:17:06.733479 2564 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"whereabouts-flatfile-configmap\" (UniqueName: \"kubernetes.io/configmap/01073023-f0f3-4517-affc-d5c86021c64d-whereabouts-flatfile-configmap\") pod \"multus-additional-cni-plugins-nmm7d\" (UID: \"01073023-f0f3-4517-affc-d5c86021c64d\") " pod="openshift-multus/multus-additional-cni-plugins-nmm7d" Jul 29 19:17:06.733815 ip-10-0-131-93 kubenswrapper[2564]: I0729 19:17:06.733505 2564 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"etc-modprobe-d\" (UniqueName: \"kubernetes.io/host-path/afd27443-e808-4fe1-b68f-140ec2d0c81f-etc-modprobe-d\") pod \"tuned-s26vt\" (UID: \"afd27443-e808-4fe1-b68f-140ec2d0c81f\") " pod="openshift-cluster-node-tuning-operator/tuned-s26vt" Jul 29 19:17:06.733815 ip-10-0-131-93 kubenswrapper[2564]: I0729 19:17:06.733527 2564 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"etc-sysconfig\" (UniqueName: \"kubernetes.io/host-path/afd27443-e808-4fe1-b68f-140ec2d0c81f-etc-sysconfig\") pod \"tuned-s26vt\" (UID: \"afd27443-e808-4fe1-b68f-140ec2d0c81f\") " pod="openshift-cluster-node-tuning-operator/tuned-s26vt" Jul 29 19:17:06.733815 ip-10-0-131-93 kubenswrapper[2564]: I0729 19:17:06.733549 2564 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"sys\" (UniqueName: \"kubernetes.io/host-path/afd27443-e808-4fe1-b68f-140ec2d0c81f-sys\") pod \"tuned-s26vt\" (UID: \"afd27443-e808-4fe1-b68f-140ec2d0c81f\") " pod="openshift-cluster-node-tuning-operator/tuned-s26vt" Jul 29 19:17:06.733815 ip-10-0-131-93 kubenswrapper[2564]: I0729 19:17:06.733575 2564 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"systemd-units\" (UniqueName: \"kubernetes.io/host-path/9568e377-3546-4fbc-bae8-e0ae7766424a-systemd-units\") pod \"ovnkube-node-kl2xl\" (UID: \"9568e377-3546-4fbc-bae8-e0ae7766424a\") " pod="openshift-ovn-kubernetes/ovnkube-node-kl2xl" Jul 29 19:17:06.733815 ip-10-0-131-93 kubenswrapper[2564]: I0729 19:17:06.733598 2564 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-cni-bin\" (UniqueName: \"kubernetes.io/host-path/9568e377-3546-4fbc-bae8-e0ae7766424a-host-cni-bin\") pod \"ovnkube-node-kl2xl\" (UID: \"9568e377-3546-4fbc-bae8-e0ae7766424a\") " pod="openshift-ovn-kubernetes/ovnkube-node-kl2xl" Jul 29 19:17:06.733815 ip-10-0-131-93 kubenswrapper[2564]: I0729 19:17:06.733621 2564 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"ovnkube-script-lib\" (UniqueName: \"kubernetes.io/configmap/9568e377-3546-4fbc-bae8-e0ae7766424a-ovnkube-script-lib\") pod \"ovnkube-node-kl2xl\" (UID: \"9568e377-3546-4fbc-bae8-e0ae7766424a\") " pod="openshift-ovn-kubernetes/ovnkube-node-kl2xl" Jul 29 19:17:06.733815 ip-10-0-131-93 kubenswrapper[2564]: I0729 19:17:06.733645 2564 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"registration-dir\" (UniqueName: \"kubernetes.io/host-path/9a4df9c4-1d87-473b-8d61-abdfb4860574-registration-dir\") pod \"aws-ebs-csi-driver-node-z7ptq\" (UID: \"9a4df9c4-1d87-473b-8d61-abdfb4860574\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-z7ptq" Jul 29 19:17:06.733815 ip-10-0-131-93 kubenswrapper[2564]: I0729 19:17:06.733668 2564 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-slash\" (UniqueName: \"kubernetes.io/host-path/723728d7-dd35-40ea-bb53-b597ec0e8ef1-host-slash\") pod \"iptables-alerter-5n889\" (UID: \"723728d7-dd35-40ea-bb53-b597ec0e8ef1\") " pod="openshift-network-operator/iptables-alerter-5n889" Jul 29 19:17:06.733815 ip-10-0-131-93 kubenswrapper[2564]: I0729 19:17:06.733691 2564 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"cni-sysctl-allowlist\" (UniqueName: \"kubernetes.io/configmap/01073023-f0f3-4517-affc-d5c86021c64d-cni-sysctl-allowlist\") pod \"multus-additional-cni-plugins-nmm7d\" (UID: \"01073023-f0f3-4517-affc-d5c86021c64d\") " pod="openshift-multus/multus-additional-cni-plugins-nmm7d" Jul 29 19:17:06.733815 ip-10-0-131-93 kubenswrapper[2564]: I0729 19:17:06.733715 2564 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"system-cni-dir\" (UniqueName: \"kubernetes.io/host-path/6f081e41-049f-44dc-9aa4-47c2c509d0ff-system-cni-dir\") pod \"multus-ccv7v\" (UID: \"6f081e41-049f-44dc-9aa4-47c2c509d0ff\") " pod="openshift-multus/multus-ccv7v" Jul 29 19:17:06.734522 ip-10-0-131-93 kubenswrapper[2564]: I0729 19:17:06.733721 2564 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"cni-binary-copy\" (UniqueName: \"kubernetes.io/configmap/01073023-f0f3-4517-affc-d5c86021c64d-cni-binary-copy\") pod \"multus-additional-cni-plugins-nmm7d\" (UID: \"01073023-f0f3-4517-affc-d5c86021c64d\") " pod="openshift-multus/multus-additional-cni-plugins-nmm7d" Jul 29 19:17:06.734522 ip-10-0-131-93 kubenswrapper[2564]: I0729 19:17:06.733737 2564 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"cnibin\" (UniqueName: \"kubernetes.io/host-path/6f081e41-049f-44dc-9aa4-47c2c509d0ff-cnibin\") pod \"multus-ccv7v\" (UID: \"6f081e41-049f-44dc-9aa4-47c2c509d0ff\") " pod="openshift-multus/multus-ccv7v" Jul 29 19:17:06.734522 ip-10-0-131-93 kubenswrapper[2564]: I0729 19:17:06.733768 2564 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"cni-binary-copy\" (UniqueName: \"kubernetes.io/configmap/6f081e41-049f-44dc-9aa4-47c2c509d0ff-cni-binary-copy\") pod \"multus-ccv7v\" (UID: \"6f081e41-049f-44dc-9aa4-47c2c509d0ff\") " pod="openshift-multus/multus-ccv7v" Jul 29 19:17:06.734522 ip-10-0-131-93 kubenswrapper[2564]: I0729 19:17:06.733762 2564 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"konnectivity-ca\" (UniqueName: \"kubernetes.io/configmap/faa8c546-c75b-4e10-93b4-661aa2b5f53e-konnectivity-ca\") pod \"konnectivity-agent-444jx\" (UID: \"faa8c546-c75b-4e10-93b4-661aa2b5f53e\") " pod="kube-system/konnectivity-agent-444jx" Jul 29 19:17:06.734522 ip-10-0-131-93 kubenswrapper[2564]: I0729 19:17:06.733791 2564 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-run-netns\" (UniqueName: \"kubernetes.io/host-path/6f081e41-049f-44dc-9aa4-47c2c509d0ff-host-run-netns\") pod \"multus-ccv7v\" (UID: \"6f081e41-049f-44dc-9aa4-47c2c509d0ff\") " pod="openshift-multus/multus-ccv7v" Jul 29 19:17:06.734522 ip-10-0-131-93 kubenswrapper[2564]: I0729 19:17:06.733806 2564 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"multus-cni-dir\" (UniqueName: \"kubernetes.io/host-path/6f081e41-049f-44dc-9aa4-47c2c509d0ff-multus-cni-dir\") pod \"multus-ccv7v\" (UID: \"6f081e41-049f-44dc-9aa4-47c2c509d0ff\") " pod="openshift-multus/multus-ccv7v" Jul 29 19:17:06.734522 ip-10-0-131-93 kubenswrapper[2564]: I0729 19:17:06.733815 2564 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-var-lib-cni-multus\" (UniqueName: \"kubernetes.io/host-path/6f081e41-049f-44dc-9aa4-47c2c509d0ff-host-var-lib-cni-multus\") pod \"multus-ccv7v\" (UID: \"6f081e41-049f-44dc-9aa4-47c2c509d0ff\") " pod="openshift-multus/multus-ccv7v" Jul 29 19:17:06.734522 ip-10-0-131-93 kubenswrapper[2564]: I0729 19:17:06.733837 2564 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"host-cni-netd\" (UniqueName: \"kubernetes.io/host-path/9568e377-3546-4fbc-bae8-e0ae7766424a-host-cni-netd\") pod \"ovnkube-node-kl2xl\" (UID: \"9568e377-3546-4fbc-bae8-e0ae7766424a\") " pod="openshift-ovn-kubernetes/ovnkube-node-kl2xl" Jul 29 19:17:06.734522 ip-10-0-131-93 kubenswrapper[2564]: I0729 19:17:06.733841 2564 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"hostroot\" (UniqueName: \"kubernetes.io/host-path/6f081e41-049f-44dc-9aa4-47c2c509d0ff-hostroot\") pod \"multus-ccv7v\" (UID: \"6f081e41-049f-44dc-9aa4-47c2c509d0ff\") " pod="openshift-multus/multus-ccv7v" Jul 29 19:17:06.734522 ip-10-0-131-93 kubenswrapper[2564]: I0729 19:17:06.733883 2564 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-xfrc2\" (UniqueName: \"kubernetes.io/projected/ae220db3-299c-481a-9591-01a8f3873d9a-kube-api-access-xfrc2\") pod \"network-metrics-daemon-hkxvx\" (UID: \"ae220db3-299c-481a-9591-01a8f3873d9a\") " pod="openshift-multus/network-metrics-daemon-hkxvx" Jul 29 19:17:06.734522 ip-10-0-131-93 kubenswrapper[2564]: I0729 19:17:06.733892 2564 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"cnibin\" (UniqueName: \"kubernetes.io/host-path/01073023-f0f3-4517-affc-d5c86021c64d-cnibin\") pod \"multus-additional-cni-plugins-nmm7d\" (UID: \"01073023-f0f3-4517-affc-d5c86021c64d\") " pod="openshift-multus/multus-additional-cni-plugins-nmm7d" Jul 29 19:17:06.734522 ip-10-0-131-93 kubenswrapper[2564]: I0729 19:17:06.733932 2564 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"host-cni-bin\" (UniqueName: \"kubernetes.io/host-path/9568e377-3546-4fbc-bae8-e0ae7766424a-host-cni-bin\") pod \"ovnkube-node-kl2xl\" (UID: \"9568e377-3546-4fbc-bae8-e0ae7766424a\") " pod="openshift-ovn-kubernetes/ovnkube-node-kl2xl" Jul 29 19:17:06.734522 ip-10-0-131-93 kubenswrapper[2564]: I0729 19:17:06.733938 2564 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"hostroot\" (UniqueName: \"kubernetes.io/host-path/6f081e41-049f-44dc-9aa4-47c2c509d0ff-hostroot\") pod \"multus-ccv7v\" (UID: \"6f081e41-049f-44dc-9aa4-47c2c509d0ff\") " pod="openshift-multus/multus-ccv7v" Jul 29 19:17:06.734522 ip-10-0-131-93 kubenswrapper[2564]: I0729 19:17:06.733964 2564 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"etc-sysctl-conf\" (UniqueName: \"kubernetes.io/host-path/afd27443-e808-4fe1-b68f-140ec2d0c81f-etc-sysctl-conf\") pod \"tuned-s26vt\" (UID: \"afd27443-e808-4fe1-b68f-140ec2d0c81f\") " pod="openshift-cluster-node-tuning-operator/tuned-s26vt" Jul 29 19:17:06.734522 ip-10-0-131-93 kubenswrapper[2564]: I0729 19:17:06.733992 2564 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"system-cni-dir\" (UniqueName: \"kubernetes.io/host-path/01073023-f0f3-4517-affc-d5c86021c64d-system-cni-dir\") pod \"multus-additional-cni-plugins-nmm7d\" (UID: \"01073023-f0f3-4517-affc-d5c86021c64d\") " pod="openshift-multus/multus-additional-cni-plugins-nmm7d" Jul 29 19:17:06.734522 ip-10-0-131-93 kubenswrapper[2564]: I0729 19:17:06.733993 2564 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"registration-dir\" (UniqueName: \"kubernetes.io/host-path/9a4df9c4-1d87-473b-8d61-abdfb4860574-registration-dir\") pod \"aws-ebs-csi-driver-node-z7ptq\" (UID: \"9a4df9c4-1d87-473b-8d61-abdfb4860574\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-z7ptq" Jul 29 19:17:06.734522 ip-10-0-131-93 kubenswrapper[2564]: I0729 19:17:06.734015 2564 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"etc-openvswitch\" (UniqueName: \"kubernetes.io/host-path/9568e377-3546-4fbc-bae8-e0ae7766424a-etc-openvswitch\") pod \"ovnkube-node-kl2xl\" (UID: \"9568e377-3546-4fbc-bae8-e0ae7766424a\") " pod="openshift-ovn-kubernetes/ovnkube-node-kl2xl" Jul 29 19:17:06.734522 ip-10-0-131-93 kubenswrapper[2564]: I0729 19:17:06.734041 2564 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"etc-kubernetes\" (UniqueName: \"kubernetes.io/host-path/6f081e41-049f-44dc-9aa4-47c2c509d0ff-etc-kubernetes\") pod \"multus-ccv7v\" (UID: \"6f081e41-049f-44dc-9aa4-47c2c509d0ff\") " pod="openshift-multus/multus-ccv7v" Jul 29 19:17:06.735240 ip-10-0-131-93 kubenswrapper[2564]: I0729 19:17:06.734064 2564 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"device-dir\" (UniqueName: \"kubernetes.io/host-path/9a4df9c4-1d87-473b-8d61-abdfb4860574-device-dir\") pod \"aws-ebs-csi-driver-node-z7ptq\" (UID: \"9a4df9c4-1d87-473b-8d61-abdfb4860574\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-z7ptq" Jul 29 19:17:06.735240 ip-10-0-131-93 kubenswrapper[2564]: I0729 19:17:06.734076 2564 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"os-release\" (UniqueName: \"kubernetes.io/host-path/01073023-f0f3-4517-affc-d5c86021c64d-os-release\") pod \"multus-additional-cni-plugins-nmm7d\" (UID: \"01073023-f0f3-4517-affc-d5c86021c64d\") " pod="openshift-multus/multus-additional-cni-plugins-nmm7d" Jul 29 19:17:06.735240 ip-10-0-131-93 kubenswrapper[2564]: I0729 19:17:06.734085 2564 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/ae220db3-299c-481a-9591-01a8f3873d9a-metrics-certs\") pod \"network-metrics-daemon-hkxvx\" (UID: \"ae220db3-299c-481a-9591-01a8f3873d9a\") " pod="openshift-multus/network-metrics-daemon-hkxvx" Jul 29 19:17:06.735240 ip-10-0-131-93 kubenswrapper[2564]: I0729 19:17:06.734093 2564 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"system-cni-dir\" (UniqueName: \"kubernetes.io/host-path/6f081e41-049f-44dc-9aa4-47c2c509d0ff-system-cni-dir\") pod \"multus-ccv7v\" (UID: \"6f081e41-049f-44dc-9aa4-47c2c509d0ff\") " pod="openshift-multus/multus-ccv7v" Jul 29 19:17:06.735240 ip-10-0-131-93 kubenswrapper[2564]: I0729 19:17:06.734141 2564 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"systemd-units\" (UniqueName: \"kubernetes.io/host-path/9568e377-3546-4fbc-bae8-e0ae7766424a-systemd-units\") pod \"ovnkube-node-kl2xl\" (UID: \"9568e377-3546-4fbc-bae8-e0ae7766424a\") " pod="openshift-ovn-kubernetes/ovnkube-node-kl2xl" Jul 29 19:17:06.735240 ip-10-0-131-93 kubenswrapper[2564]: E0729 19:17:06.734192 2564 secret.go:189] Couldn't get secret openshift-multus/metrics-daemon-secret: object "openshift-multus"/"metrics-daemon-secret" not registered Jul 29 19:17:06.735240 ip-10-0-131-93 kubenswrapper[2564]: I0729 19:17:06.734200 2564 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"device-dir\" (UniqueName: \"kubernetes.io/host-path/9a4df9c4-1d87-473b-8d61-abdfb4860574-device-dir\") pod \"aws-ebs-csi-driver-node-z7ptq\" (UID: \"9a4df9c4-1d87-473b-8d61-abdfb4860574\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-z7ptq" Jul 29 19:17:06.735240 ip-10-0-131-93 kubenswrapper[2564]: I0729 19:17:06.734243 2564 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"host-var-lib-cni-multus\" (UniqueName: \"kubernetes.io/host-path/6f081e41-049f-44dc-9aa4-47c2c509d0ff-host-var-lib-cni-multus\") pod \"multus-ccv7v\" (UID: \"6f081e41-049f-44dc-9aa4-47c2c509d0ff\") " pod="openshift-multus/multus-ccv7v" Jul 29 19:17:06.735240 ip-10-0-131-93 kubenswrapper[2564]: E0729 19:17:06.734253 2564 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/ae220db3-299c-481a-9591-01a8f3873d9a-metrics-certs podName:ae220db3-299c-481a-9591-01a8f3873d9a nodeName:}" failed. No retries permitted until 2026-07-29 19:17:07.2342346 +0000 UTC m=+2.958294285 (durationBeforeRetry 500ms). Error: MountVolume.SetUp failed for volume "metrics-certs" (UniqueName: "kubernetes.io/secret/ae220db3-299c-481a-9591-01a8f3873d9a-metrics-certs") pod "network-metrics-daemon-hkxvx" (UID: "ae220db3-299c-481a-9591-01a8f3873d9a") : object "openshift-multus"/"metrics-daemon-secret" not registered Jul 29 19:17:06.735240 ip-10-0-131-93 kubenswrapper[2564]: I0729 19:17:06.734296 2564 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"cnibin\" (UniqueName: \"kubernetes.io/host-path/6f081e41-049f-44dc-9aa4-47c2c509d0ff-cnibin\") pod \"multus-ccv7v\" (UID: \"6f081e41-049f-44dc-9aa4-47c2c509d0ff\") " pod="openshift-multus/multus-ccv7v" Jul 29 19:17:06.735240 ip-10-0-131-93 kubenswrapper[2564]: I0729 19:17:06.734427 2564 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"dbus\" (UniqueName: \"kubernetes.io/host-path/5c0eec03-fbbc-40db-9c78-43c8f98534c7-dbus\") pod \"global-pull-secret-syncer-zcgr9\" (UID: \"5c0eec03-fbbc-40db-9c78-43c8f98534c7\") " pod="kube-system/global-pull-secret-syncer-zcgr9" Jul 29 19:17:06.735240 ip-10-0-131-93 kubenswrapper[2564]: I0729 19:17:06.734463 2564 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"os-release\" (UniqueName: \"kubernetes.io/host-path/6f081e41-049f-44dc-9aa4-47c2c509d0ff-os-release\") pod \"multus-ccv7v\" (UID: \"6f081e41-049f-44dc-9aa4-47c2c509d0ff\") " pod="openshift-multus/multus-ccv7v" Jul 29 19:17:06.735240 ip-10-0-131-93 kubenswrapper[2564]: I0729 19:17:06.734493 2564 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"multus-socket-dir-parent\" (UniqueName: \"kubernetes.io/host-path/6f081e41-049f-44dc-9aa4-47c2c509d0ff-multus-socket-dir-parent\") pod \"multus-ccv7v\" (UID: \"6f081e41-049f-44dc-9aa4-47c2c509d0ff\") " pod="openshift-multus/multus-ccv7v" Jul 29 19:17:06.735240 ip-10-0-131-93 kubenswrapper[2564]: I0729 19:17:06.734536 2564 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"sys-fs\" (UniqueName: \"kubernetes.io/host-path/9a4df9c4-1d87-473b-8d61-abdfb4860574-sys-fs\") pod \"aws-ebs-csi-driver-node-z7ptq\" (UID: \"9a4df9c4-1d87-473b-8d61-abdfb4860574\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-z7ptq" Jul 29 19:17:06.735240 ip-10-0-131-93 kubenswrapper[2564]: I0729 19:17:06.734561 2564 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"etc-kubernetes\" (UniqueName: \"kubernetes.io/host-path/afd27443-e808-4fe1-b68f-140ec2d0c81f-etc-kubernetes\") pod \"tuned-s26vt\" (UID: \"afd27443-e808-4fe1-b68f-140ec2d0c81f\") " pod="openshift-cluster-node-tuning-operator/tuned-s26vt" Jul 29 19:17:06.735240 ip-10-0-131-93 kubenswrapper[2564]: I0729 19:17:06.734588 2564 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host\" (UniqueName: \"kubernetes.io/host-path/afd27443-e808-4fe1-b68f-140ec2d0c81f-host\") pod \"tuned-s26vt\" (UID: \"afd27443-e808-4fe1-b68f-140ec2d0c81f\") " pod="openshift-cluster-node-tuning-operator/tuned-s26vt" Jul 29 19:17:06.735240 ip-10-0-131-93 kubenswrapper[2564]: I0729 19:17:06.734612 2564 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-hz5lq\" (UniqueName: \"kubernetes.io/projected/afd27443-e808-4fe1-b68f-140ec2d0c81f-kube-api-access-hz5lq\") pod \"tuned-s26vt\" (UID: \"afd27443-e808-4fe1-b68f-140ec2d0c81f\") " pod="openshift-cluster-node-tuning-operator/tuned-s26vt" Jul 29 19:17:06.736038 ip-10-0-131-93 kubenswrapper[2564]: I0729 19:17:06.734623 2564 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"whereabouts-flatfile-configmap\" (UniqueName: \"kubernetes.io/configmap/01073023-f0f3-4517-affc-d5c86021c64d-whereabouts-flatfile-configmap\") pod \"multus-additional-cni-plugins-nmm7d\" (UID: \"01073023-f0f3-4517-affc-d5c86021c64d\") " pod="openshift-multus/multus-additional-cni-plugins-nmm7d" Jul 29 19:17:06.736038 ip-10-0-131-93 kubenswrapper[2564]: I0729 19:17:06.734637 2564 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-mvp9q\" (UniqueName: \"kubernetes.io/projected/ffe05666-ed11-4573-9442-a9646b974e61-kube-api-access-mvp9q\") pod \"node-resolver-lzsmp\" (UID: \"ffe05666-ed11-4573-9442-a9646b974e61\") " pod="openshift-dns/node-resolver-lzsmp" Jul 29 19:17:06.736038 ip-10-0-131-93 kubenswrapper[2564]: I0729 19:17:06.734666 2564 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-run-k8s-cni-cncf-io\" (UniqueName: \"kubernetes.io/host-path/6f081e41-049f-44dc-9aa4-47c2c509d0ff-host-run-k8s-cni-cncf-io\") pod \"multus-ccv7v\" (UID: \"6f081e41-049f-44dc-9aa4-47c2c509d0ff\") " pod="openshift-multus/multus-ccv7v" Jul 29 19:17:06.736038 ip-10-0-131-93 kubenswrapper[2564]: I0729 19:17:06.734671 2564 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"host-run-netns\" (UniqueName: \"kubernetes.io/host-path/6f081e41-049f-44dc-9aa4-47c2c509d0ff-host-run-netns\") pod \"multus-ccv7v\" (UID: \"6f081e41-049f-44dc-9aa4-47c2c509d0ff\") " pod="openshift-multus/multus-ccv7v" Jul 29 19:17:06.736038 ip-10-0-131-93 kubenswrapper[2564]: I0729 19:17:06.734679 2564 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"os-release\" (UniqueName: \"kubernetes.io/host-path/6f081e41-049f-44dc-9aa4-47c2c509d0ff-os-release\") pod \"multus-ccv7v\" (UID: \"6f081e41-049f-44dc-9aa4-47c2c509d0ff\") " pod="openshift-multus/multus-ccv7v" Jul 29 19:17:06.736038 ip-10-0-131-93 kubenswrapper[2564]: I0729 19:17:06.734729 2564 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"system-cni-dir\" (UniqueName: \"kubernetes.io/host-path/01073023-f0f3-4517-affc-d5c86021c64d-system-cni-dir\") pod \"multus-additional-cni-plugins-nmm7d\" (UID: \"01073023-f0f3-4517-affc-d5c86021c64d\") " pod="openshift-multus/multus-additional-cni-plugins-nmm7d" Jul 29 19:17:06.736038 ip-10-0-131-93 kubenswrapper[2564]: I0729 19:17:06.734729 2564 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"ovnkube-script-lib\" (UniqueName: \"kubernetes.io/configmap/9568e377-3546-4fbc-bae8-e0ae7766424a-ovnkube-script-lib\") pod \"ovnkube-node-kl2xl\" (UID: \"9568e377-3546-4fbc-bae8-e0ae7766424a\") " pod="openshift-ovn-kubernetes/ovnkube-node-kl2xl" Jul 29 19:17:06.736038 ip-10-0-131-93 kubenswrapper[2564]: I0729 19:17:06.734738 2564 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"multus-socket-dir-parent\" (UniqueName: \"kubernetes.io/host-path/6f081e41-049f-44dc-9aa4-47c2c509d0ff-multus-socket-dir-parent\") pod \"multus-ccv7v\" (UID: \"6f081e41-049f-44dc-9aa4-47c2c509d0ff\") " pod="openshift-multus/multus-ccv7v" Jul 29 19:17:06.736038 ip-10-0-131-93 kubenswrapper[2564]: I0729 19:17:06.734763 2564 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-run-multus-certs\" (UniqueName: \"kubernetes.io/host-path/6f081e41-049f-44dc-9aa4-47c2c509d0ff-host-run-multus-certs\") pod \"multus-ccv7v\" (UID: \"6f081e41-049f-44dc-9aa4-47c2c509d0ff\") " pod="openshift-multus/multus-ccv7v" Jul 29 19:17:06.736038 ip-10-0-131-93 kubenswrapper[2564]: I0729 19:17:06.734776 2564 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"etc-kubernetes\" (UniqueName: \"kubernetes.io/host-path/6f081e41-049f-44dc-9aa4-47c2c509d0ff-etc-kubernetes\") pod \"multus-ccv7v\" (UID: \"6f081e41-049f-44dc-9aa4-47c2c509d0ff\") " pod="openshift-multus/multus-ccv7v" Jul 29 19:17:06.736038 ip-10-0-131-93 kubenswrapper[2564]: I0729 19:17:06.734738 2564 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"host-run-k8s-cni-cncf-io\" (UniqueName: \"kubernetes.io/host-path/6f081e41-049f-44dc-9aa4-47c2c509d0ff-host-run-k8s-cni-cncf-io\") pod \"multus-ccv7v\" (UID: \"6f081e41-049f-44dc-9aa4-47c2c509d0ff\") " pod="openshift-multus/multus-ccv7v" Jul 29 19:17:06.736038 ip-10-0-131-93 kubenswrapper[2564]: I0729 19:17:06.734768 2564 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"etc-openvswitch\" (UniqueName: \"kubernetes.io/host-path/9568e377-3546-4fbc-bae8-e0ae7766424a-etc-openvswitch\") pod \"ovnkube-node-kl2xl\" (UID: \"9568e377-3546-4fbc-bae8-e0ae7766424a\") " pod="openshift-ovn-kubernetes/ovnkube-node-kl2xl" Jul 29 19:17:06.736038 ip-10-0-131-93 kubenswrapper[2564]: I0729 19:17:06.734789 2564 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"cni-sysctl-allowlist\" (UniqueName: \"kubernetes.io/configmap/01073023-f0f3-4517-affc-d5c86021c64d-cni-sysctl-allowlist\") pod \"multus-additional-cni-plugins-nmm7d\" (UID: \"01073023-f0f3-4517-affc-d5c86021c64d\") " pod="openshift-multus/multus-additional-cni-plugins-nmm7d" Jul 29 19:17:06.736038 ip-10-0-131-93 kubenswrapper[2564]: I0729 19:17:06.734802 2564 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-kubelet\" (UniqueName: \"kubernetes.io/host-path/9568e377-3546-4fbc-bae8-e0ae7766424a-host-kubelet\") pod \"ovnkube-node-kl2xl\" (UID: \"9568e377-3546-4fbc-bae8-e0ae7766424a\") " pod="openshift-ovn-kubernetes/ovnkube-node-kl2xl" Jul 29 19:17:06.736038 ip-10-0-131-93 kubenswrapper[2564]: I0729 19:17:06.734794 2564 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"cni-binary-copy\" (UniqueName: \"kubernetes.io/configmap/6f081e41-049f-44dc-9aa4-47c2c509d0ff-cni-binary-copy\") pod \"multus-ccv7v\" (UID: \"6f081e41-049f-44dc-9aa4-47c2c509d0ff\") " pod="openshift-multus/multus-ccv7v" Jul 29 19:17:06.736038 ip-10-0-131-93 kubenswrapper[2564]: I0729 19:17:06.734813 2564 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"sys-fs\" (UniqueName: \"kubernetes.io/host-path/9a4df9c4-1d87-473b-8d61-abdfb4860574-sys-fs\") pod \"aws-ebs-csi-driver-node-z7ptq\" (UID: \"9a4df9c4-1d87-473b-8d61-abdfb4860574\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-z7ptq" Jul 29 19:17:06.736038 ip-10-0-131-93 kubenswrapper[2564]: I0729 19:17:06.734833 2564 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"host-run-multus-certs\" (UniqueName: \"kubernetes.io/host-path/6f081e41-049f-44dc-9aa4-47c2c509d0ff-host-run-multus-certs\") pod \"multus-ccv7v\" (UID: \"6f081e41-049f-44dc-9aa4-47c2c509d0ff\") " pod="openshift-multus/multus-ccv7v" Jul 29 19:17:06.736663 ip-10-0-131-93 kubenswrapper[2564]: I0729 19:17:06.734841 2564 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"host-slash\" (UniqueName: \"kubernetes.io/host-path/723728d7-dd35-40ea-bb53-b597ec0e8ef1-host-slash\") pod \"iptables-alerter-5n889\" (UID: \"723728d7-dd35-40ea-bb53-b597ec0e8ef1\") " pod="openshift-network-operator/iptables-alerter-5n889" Jul 29 19:17:06.736663 ip-10-0-131-93 kubenswrapper[2564]: I0729 19:17:06.734870 2564 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"env-overrides\" (UniqueName: \"kubernetes.io/configmap/9568e377-3546-4fbc-bae8-e0ae7766424a-env-overrides\") pod \"ovnkube-node-kl2xl\" (UID: \"9568e377-3546-4fbc-bae8-e0ae7766424a\") " pod="openshift-ovn-kubernetes/ovnkube-node-kl2xl" Jul 29 19:17:06.736663 ip-10-0-131-93 kubenswrapper[2564]: I0729 19:17:06.734880 2564 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"host-kubelet\" (UniqueName: \"kubernetes.io/host-path/9568e377-3546-4fbc-bae8-e0ae7766424a-host-kubelet\") pod \"ovnkube-node-kl2xl\" (UID: \"9568e377-3546-4fbc-bae8-e0ae7766424a\") " pod="openshift-ovn-kubernetes/ovnkube-node-kl2xl" Jul 29 19:17:06.736663 ip-10-0-131-93 kubenswrapper[2564]: I0729 19:17:06.734912 2564 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"etc-selinux\" (UniqueName: \"kubernetes.io/host-path/9a4df9c4-1d87-473b-8d61-abdfb4860574-etc-selinux\") pod \"aws-ebs-csi-driver-node-z7ptq\" (UID: \"9a4df9c4-1d87-473b-8d61-abdfb4860574\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-z7ptq" Jul 29 19:17:06.736663 ip-10-0-131-93 kubenswrapper[2564]: I0729 19:17:06.734937 2564 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tuning-conf-dir\" (UniqueName: \"kubernetes.io/host-path/01073023-f0f3-4517-affc-d5c86021c64d-tuning-conf-dir\") pod \"multus-additional-cni-plugins-nmm7d\" (UID: \"01073023-f0f3-4517-affc-d5c86021c64d\") " pod="openshift-multus/multus-additional-cni-plugins-nmm7d" Jul 29 19:17:06.736663 ip-10-0-131-93 kubenswrapper[2564]: I0729 19:17:06.735025 2564 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-qgvzl\" (UniqueName: \"kubernetes.io/projected/02612aaa-a23a-4e6b-b224-123af1bb8230-kube-api-access-qgvzl\") pod \"network-check-target-464tm\" (UID: \"02612aaa-a23a-4e6b-b224-123af1bb8230\") " pod="openshift-network-diagnostics/network-check-target-464tm" Jul 29 19:17:06.736663 ip-10-0-131-93 kubenswrapper[2564]: I0729 19:17:06.735063 2564 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"etc-selinux\" (UniqueName: \"kubernetes.io/host-path/9a4df9c4-1d87-473b-8d61-abdfb4860574-etc-selinux\") pod \"aws-ebs-csi-driver-node-z7ptq\" (UID: \"9a4df9c4-1d87-473b-8d61-abdfb4860574\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-z7ptq" Jul 29 19:17:06.736663 ip-10-0-131-93 kubenswrapper[2564]: I0729 19:17:06.735086 2564 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"lib-modules\" (UniqueName: \"kubernetes.io/host-path/afd27443-e808-4fe1-b68f-140ec2d0c81f-lib-modules\") pod \"tuned-s26vt\" (UID: \"afd27443-e808-4fe1-b68f-140ec2d0c81f\") " pod="openshift-cluster-node-tuning-operator/tuned-s26vt" Jul 29 19:17:06.736663 ip-10-0-131-93 kubenswrapper[2564]: I0729 19:17:06.735114 2564 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-run-netns\" (UniqueName: \"kubernetes.io/host-path/9568e377-3546-4fbc-bae8-e0ae7766424a-host-run-netns\") pod \"ovnkube-node-kl2xl\" (UID: \"9568e377-3546-4fbc-bae8-e0ae7766424a\") " pod="openshift-ovn-kubernetes/ovnkube-node-kl2xl" Jul 29 19:17:06.736663 ip-10-0-131-93 kubenswrapper[2564]: I0729 19:17:06.735116 2564 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"tuning-conf-dir\" (UniqueName: \"kubernetes.io/host-path/01073023-f0f3-4517-affc-d5c86021c64d-tuning-conf-dir\") pod \"multus-additional-cni-plugins-nmm7d\" (UID: \"01073023-f0f3-4517-affc-d5c86021c64d\") " pod="openshift-multus/multus-additional-cni-plugins-nmm7d" Jul 29 19:17:06.736663 ip-10-0-131-93 kubenswrapper[2564]: I0729 19:17:06.735148 2564 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"host-run-netns\" (UniqueName: \"kubernetes.io/host-path/9568e377-3546-4fbc-bae8-e0ae7766424a-host-run-netns\") pod \"ovnkube-node-kl2xl\" (UID: \"9568e377-3546-4fbc-bae8-e0ae7766424a\") " pod="openshift-ovn-kubernetes/ovnkube-node-kl2xl" Jul 29 19:17:06.736663 ip-10-0-131-93 kubenswrapper[2564]: I0729 19:17:06.735182 2564 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"multus-daemon-config\" (UniqueName: \"kubernetes.io/configmap/6f081e41-049f-44dc-9aa4-47c2c509d0ff-multus-daemon-config\") pod \"multus-ccv7v\" (UID: \"6f081e41-049f-44dc-9aa4-47c2c509d0ff\") " pod="openshift-multus/multus-ccv7v" Jul 29 19:17:06.736663 ip-10-0-131-93 kubenswrapper[2564]: I0729 19:17:06.735218 2564 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-slash\" (UniqueName: \"kubernetes.io/host-path/9568e377-3546-4fbc-bae8-e0ae7766424a-host-slash\") pod \"ovnkube-node-kl2xl\" (UID: \"9568e377-3546-4fbc-bae8-e0ae7766424a\") " pod="openshift-ovn-kubernetes/ovnkube-node-kl2xl" Jul 29 19:17:06.736663 ip-10-0-131-93 kubenswrapper[2564]: I0729 19:17:06.735243 2564 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-run-ovn-kubernetes\" (UniqueName: \"kubernetes.io/host-path/9568e377-3546-4fbc-bae8-e0ae7766424a-host-run-ovn-kubernetes\") pod \"ovnkube-node-kl2xl\" (UID: \"9568e377-3546-4fbc-bae8-e0ae7766424a\") " pod="openshift-ovn-kubernetes/ovnkube-node-kl2xl" Jul 29 19:17:06.736663 ip-10-0-131-93 kubenswrapper[2564]: I0729 19:17:06.735269 2564 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"ovn-node-metrics-cert\" (UniqueName: \"kubernetes.io/secret/9568e377-3546-4fbc-bae8-e0ae7766424a-ovn-node-metrics-cert\") pod \"ovnkube-node-kl2xl\" (UID: \"9568e377-3546-4fbc-bae8-e0ae7766424a\") " pod="openshift-ovn-kubernetes/ovnkube-node-kl2xl" Jul 29 19:17:06.736663 ip-10-0-131-93 kubenswrapper[2564]: I0729 19:17:06.735281 2564 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"env-overrides\" (UniqueName: \"kubernetes.io/configmap/9568e377-3546-4fbc-bae8-e0ae7766424a-env-overrides\") pod \"ovnkube-node-kl2xl\" (UID: \"9568e377-3546-4fbc-bae8-e0ae7766424a\") " pod="openshift-ovn-kubernetes/ovnkube-node-kl2xl" Jul 29 19:17:06.736663 ip-10-0-131-93 kubenswrapper[2564]: I0729 19:17:06.735284 2564 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"host-run-ovn-kubernetes\" (UniqueName: \"kubernetes.io/host-path/9568e377-3546-4fbc-bae8-e0ae7766424a-host-run-ovn-kubernetes\") pod \"ovnkube-node-kl2xl\" (UID: \"9568e377-3546-4fbc-bae8-e0ae7766424a\") " pod="openshift-ovn-kubernetes/ovnkube-node-kl2xl" Jul 29 19:17:06.737275 ip-10-0-131-93 kubenswrapper[2564]: I0729 19:17:06.735269 2564 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"host-slash\" (UniqueName: \"kubernetes.io/host-path/9568e377-3546-4fbc-bae8-e0ae7766424a-host-slash\") pod \"ovnkube-node-kl2xl\" (UID: \"9568e377-3546-4fbc-bae8-e0ae7766424a\") " pod="openshift-ovn-kubernetes/ovnkube-node-kl2xl" Jul 29 19:17:06.737275 ip-10-0-131-93 kubenswrapper[2564]: I0729 19:17:06.735291 2564 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"etc-sysctl-d\" (UniqueName: \"kubernetes.io/host-path/afd27443-e808-4fe1-b68f-140ec2d0c81f-etc-sysctl-d\") pod \"tuned-s26vt\" (UID: \"afd27443-e808-4fe1-b68f-140ec2d0c81f\") " pod="openshift-cluster-node-tuning-operator/tuned-s26vt" Jul 29 19:17:06.737275 ip-10-0-131-93 kubenswrapper[2564]: I0729 19:17:06.735335 2564 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"etc-systemd\" (UniqueName: \"kubernetes.io/host-path/afd27443-e808-4fe1-b68f-140ec2d0c81f-etc-systemd\") pod \"tuned-s26vt\" (UID: \"afd27443-e808-4fe1-b68f-140ec2d0c81f\") " pod="openshift-cluster-node-tuning-operator/tuned-s26vt" Jul 29 19:17:06.737275 ip-10-0-131-93 kubenswrapper[2564]: I0729 19:17:06.735364 2564 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"ovnkube-config\" (UniqueName: \"kubernetes.io/configmap/9568e377-3546-4fbc-bae8-e0ae7766424a-ovnkube-config\") pod \"ovnkube-node-kl2xl\" (UID: \"9568e377-3546-4fbc-bae8-e0ae7766424a\") " pod="openshift-ovn-kubernetes/ovnkube-node-kl2xl" Jul 29 19:17:06.737275 ip-10-0-131-93 kubenswrapper[2564]: I0729 19:17:06.735408 2564 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/ffe05666-ed11-4573-9442-a9646b974e61-tmp-dir\") pod \"node-resolver-lzsmp\" (UID: \"ffe05666-ed11-4573-9442-a9646b974e61\") " pod="openshift-dns/node-resolver-lzsmp" Jul 29 19:17:06.737275 ip-10-0-131-93 kubenswrapper[2564]: I0729 19:17:06.735429 2564 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"original-pull-secret\" (UniqueName: \"kubernetes.io/secret/5c0eec03-fbbc-40db-9c78-43c8f98534c7-original-pull-secret\") pod \"global-pull-secret-syncer-zcgr9\" (UID: \"5c0eec03-fbbc-40db-9c78-43c8f98534c7\") " pod="kube-system/global-pull-secret-syncer-zcgr9" Jul 29 19:17:06.737275 ip-10-0-131-93 kubenswrapper[2564]: I0729 19:17:06.735446 2564 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-6fg74\" (UniqueName: \"kubernetes.io/projected/01073023-f0f3-4517-affc-d5c86021c64d-kube-api-access-6fg74\") pod \"multus-additional-cni-plugins-nmm7d\" (UID: \"01073023-f0f3-4517-affc-d5c86021c64d\") " pod="openshift-multus/multus-additional-cni-plugins-nmm7d" Jul 29 19:17:06.737275 ip-10-0-131-93 kubenswrapper[2564]: I0729 19:17:06.735472 2564 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"var-lib-kubelet\" (UniqueName: \"kubernetes.io/host-path/afd27443-e808-4fe1-b68f-140ec2d0c81f-var-lib-kubelet\") pod \"tuned-s26vt\" (UID: \"afd27443-e808-4fe1-b68f-140ec2d0c81f\") " pod="openshift-cluster-node-tuning-operator/tuned-s26vt" Jul 29 19:17:06.737275 ip-10-0-131-93 kubenswrapper[2564]: I0729 19:17:06.735491 2564 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"hosts-file\" (UniqueName: \"kubernetes.io/host-path/ffe05666-ed11-4573-9442-a9646b974e61-hosts-file\") pod \"node-resolver-lzsmp\" (UID: \"ffe05666-ed11-4573-9442-a9646b974e61\") " pod="openshift-dns/node-resolver-lzsmp" Jul 29 19:17:06.737275 ip-10-0-131-93 kubenswrapper[2564]: I0729 19:17:06.735505 2564 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"serviceca\" (UniqueName: \"kubernetes.io/configmap/edfe2676-f3b8-4eaa-ad17-ecf737b9f786-serviceca\") pod \"node-ca-2lv4n\" (UID: \"edfe2676-f3b8-4eaa-ad17-ecf737b9f786\") " pod="openshift-image-registry/node-ca-2lv4n" Jul 29 19:17:06.737275 ip-10-0-131-93 kubenswrapper[2564]: I0729 19:17:06.735525 2564 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"iptables-alerter-script\" (UniqueName: \"kubernetes.io/configmap/723728d7-dd35-40ea-bb53-b597ec0e8ef1-iptables-alerter-script\") pod \"iptables-alerter-5n889\" (UID: \"723728d7-dd35-40ea-bb53-b597ec0e8ef1\") " pod="openshift-network-operator/iptables-alerter-5n889" Jul 29 19:17:06.737275 ip-10-0-131-93 kubenswrapper[2564]: I0729 19:17:06.735554 2564 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host\" (UniqueName: \"kubernetes.io/host-path/edfe2676-f3b8-4eaa-ad17-ecf737b9f786-host\") pod \"node-ca-2lv4n\" (UID: \"edfe2676-f3b8-4eaa-ad17-ecf737b9f786\") " pod="openshift-image-registry/node-ca-2lv4n" Jul 29 19:17:06.737275 ip-10-0-131-93 kubenswrapper[2564]: I0729 19:17:06.735586 2564 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"multus-conf-dir\" (UniqueName: \"kubernetes.io/host-path/6f081e41-049f-44dc-9aa4-47c2c509d0ff-multus-conf-dir\") pod \"multus-ccv7v\" (UID: \"6f081e41-049f-44dc-9aa4-47c2c509d0ff\") " pod="openshift-multus/multus-ccv7v" Jul 29 19:17:06.737275 ip-10-0-131-93 kubenswrapper[2564]: I0729 19:17:06.735568 2564 swap_util.go:74] "error creating dir to test if tmpfs noswap is enabled. Assuming not supported" mount path="" error="stat /var/lib/kubelet/plugins/kubernetes.io/empty-dir: no such file or directory" Jul 29 19:17:06.737275 ip-10-0-131-93 kubenswrapper[2564]: I0729 19:17:06.735610 2564 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"node-log\" (UniqueName: \"kubernetes.io/host-path/9568e377-3546-4fbc-bae8-e0ae7766424a-node-log\") pod \"ovnkube-node-kl2xl\" (UID: \"9568e377-3546-4fbc-bae8-e0ae7766424a\") " pod="openshift-ovn-kubernetes/ovnkube-node-kl2xl" Jul 29 19:17:06.737275 ip-10-0-131-93 kubenswrapper[2564]: I0729 19:17:06.735648 2564 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"multus-daemon-config\" (UniqueName: \"kubernetes.io/configmap/6f081e41-049f-44dc-9aa4-47c2c509d0ff-multus-daemon-config\") pod \"multus-ccv7v\" (UID: \"6f081e41-049f-44dc-9aa4-47c2c509d0ff\") " pod="openshift-multus/multus-ccv7v" Jul 29 19:17:06.737275 ip-10-0-131-93 kubenswrapper[2564]: I0729 19:17:06.735674 2564 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-var-lib-cni-networks-ovn-kubernetes\" (UniqueName: \"kubernetes.io/host-path/9568e377-3546-4fbc-bae8-e0ae7766424a-host-var-lib-cni-networks-ovn-kubernetes\") pod \"ovnkube-node-kl2xl\" (UID: \"9568e377-3546-4fbc-bae8-e0ae7766424a\") " pod="openshift-ovn-kubernetes/ovnkube-node-kl2xl" Jul 29 19:17:06.737928 ip-10-0-131-93 kubenswrapper[2564]: I0729 19:17:06.735703 2564 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"multus-conf-dir\" (UniqueName: \"kubernetes.io/host-path/6f081e41-049f-44dc-9aa4-47c2c509d0ff-multus-conf-dir\") pod \"multus-ccv7v\" (UID: \"6f081e41-049f-44dc-9aa4-47c2c509d0ff\") " pod="openshift-multus/multus-ccv7v" Jul 29 19:17:06.737928 ip-10-0-131-93 kubenswrapper[2564]: I0729 19:17:06.735715 2564 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"socket-dir\" (UniqueName: \"kubernetes.io/host-path/9a4df9c4-1d87-473b-8d61-abdfb4860574-socket-dir\") pod \"aws-ebs-csi-driver-node-z7ptq\" (UID: \"9a4df9c4-1d87-473b-8d61-abdfb4860574\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-z7ptq" Jul 29 19:17:06.737928 ip-10-0-131-93 kubenswrapper[2564]: I0729 19:17:06.735726 2564 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"host-var-lib-cni-networks-ovn-kubernetes\" (UniqueName: \"kubernetes.io/host-path/9568e377-3546-4fbc-bae8-e0ae7766424a-host-var-lib-cni-networks-ovn-kubernetes\") pod \"ovnkube-node-kl2xl\" (UID: \"9568e377-3546-4fbc-bae8-e0ae7766424a\") " pod="openshift-ovn-kubernetes/ovnkube-node-kl2xl" Jul 29 19:17:06.737928 ip-10-0-131-93 kubenswrapper[2564]: I0729 19:17:06.735731 2564 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"node-log\" (UniqueName: \"kubernetes.io/host-path/9568e377-3546-4fbc-bae8-e0ae7766424a-node-log\") pod \"ovnkube-node-kl2xl\" (UID: \"9568e377-3546-4fbc-bae8-e0ae7766424a\") " pod="openshift-ovn-kubernetes/ovnkube-node-kl2xl" Jul 29 19:17:06.737928 ip-10-0-131-93 kubenswrapper[2564]: I0729 19:17:06.735763 2564 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"agent-certs\" (UniqueName: \"kubernetes.io/secret/faa8c546-c75b-4e10-93b4-661aa2b5f53e-agent-certs\") pod \"konnectivity-agent-444jx\" (UID: \"faa8c546-c75b-4e10-93b4-661aa2b5f53e\") " pod="kube-system/konnectivity-agent-444jx" Jul 29 19:17:06.737928 ip-10-0-131-93 kubenswrapper[2564]: I0729 19:17:06.735803 2564 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"run\" (UniqueName: \"kubernetes.io/host-path/afd27443-e808-4fe1-b68f-140ec2d0c81f-run\") pod \"tuned-s26vt\" (UID: \"afd27443-e808-4fe1-b68f-140ec2d0c81f\") " pod="openshift-cluster-node-tuning-operator/tuned-s26vt" Jul 29 19:17:06.737928 ip-10-0-131-93 kubenswrapper[2564]: I0729 19:17:06.735845 2564 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"socket-dir\" (UniqueName: \"kubernetes.io/host-path/9a4df9c4-1d87-473b-8d61-abdfb4860574-socket-dir\") pod \"aws-ebs-csi-driver-node-z7ptq\" (UID: \"9a4df9c4-1d87-473b-8d61-abdfb4860574\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-z7ptq" Jul 29 19:17:06.737928 ip-10-0-131-93 kubenswrapper[2564]: I0729 19:17:06.735871 2564 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"ovnkube-config\" (UniqueName: \"kubernetes.io/configmap/9568e377-3546-4fbc-bae8-e0ae7766424a-ovnkube-config\") pod \"ovnkube-node-kl2xl\" (UID: \"9568e377-3546-4fbc-bae8-e0ae7766424a\") " pod="openshift-ovn-kubernetes/ovnkube-node-kl2xl" Jul 29 19:17:06.737928 ip-10-0-131-93 kubenswrapper[2564]: I0729 19:17:06.735903 2564 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-var-lib-kubelet\" (UniqueName: \"kubernetes.io/host-path/6f081e41-049f-44dc-9aa4-47c2c509d0ff-host-var-lib-kubelet\") pod \"multus-ccv7v\" (UID: \"6f081e41-049f-44dc-9aa4-47c2c509d0ff\") " pod="openshift-multus/multus-ccv7v" Jul 29 19:17:06.737928 ip-10-0-131-93 kubenswrapper[2564]: I0729 19:17:06.735936 2564 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"etc-tuned\" (UniqueName: \"kubernetes.io/empty-dir/afd27443-e808-4fe1-b68f-140ec2d0c81f-etc-tuned\") pod \"tuned-s26vt\" (UID: \"afd27443-e808-4fe1-b68f-140ec2d0c81f\") " pod="openshift-cluster-node-tuning-operator/tuned-s26vt" Jul 29 19:17:06.737928 ip-10-0-131-93 kubenswrapper[2564]: I0729 19:17:06.735951 2564 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"host-var-lib-kubelet\" (UniqueName: \"kubernetes.io/host-path/6f081e41-049f-44dc-9aa4-47c2c509d0ff-host-var-lib-kubelet\") pod \"multus-ccv7v\" (UID: \"6f081e41-049f-44dc-9aa4-47c2c509d0ff\") " pod="openshift-multus/multus-ccv7v" Jul 29 19:17:06.737928 ip-10-0-131-93 kubenswrapper[2564]: I0729 19:17:06.735974 2564 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-sdfb2\" (UniqueName: \"kubernetes.io/projected/edfe2676-f3b8-4eaa-ad17-ecf737b9f786-kube-api-access-sdfb2\") pod \"node-ca-2lv4n\" (UID: \"edfe2676-f3b8-4eaa-ad17-ecf737b9f786\") " pod="openshift-image-registry/node-ca-2lv4n" Jul 29 19:17:06.737928 ip-10-0-131-93 kubenswrapper[2564]: I0729 19:17:06.735998 2564 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-jzc6t\" (UniqueName: \"kubernetes.io/projected/6f081e41-049f-44dc-9aa4-47c2c509d0ff-kube-api-access-jzc6t\") pod \"multus-ccv7v\" (UID: \"6f081e41-049f-44dc-9aa4-47c2c509d0ff\") " pod="openshift-multus/multus-ccv7v" Jul 29 19:17:06.737928 ip-10-0-131-93 kubenswrapper[2564]: I0729 19:17:06.736116 2564 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"iptables-alerter-script\" (UniqueName: \"kubernetes.io/configmap/723728d7-dd35-40ea-bb53-b597ec0e8ef1-iptables-alerter-script\") pod \"iptables-alerter-5n889\" (UID: \"723728d7-dd35-40ea-bb53-b597ec0e8ef1\") " pod="openshift-network-operator/iptables-alerter-5n889" Jul 29 19:17:06.738877 ip-10-0-131-93 kubenswrapper[2564]: I0729 19:17:06.738854 2564 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"ovn-node-metrics-cert\" (UniqueName: \"kubernetes.io/secret/9568e377-3546-4fbc-bae8-e0ae7766424a-ovn-node-metrics-cert\") pod \"ovnkube-node-kl2xl\" (UID: \"9568e377-3546-4fbc-bae8-e0ae7766424a\") " pod="openshift-ovn-kubernetes/ovnkube-node-kl2xl" Jul 29 19:17:06.739036 ip-10-0-131-93 kubenswrapper[2564]: I0729 19:17:06.739015 2564 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"agent-certs\" (UniqueName: \"kubernetes.io/secret/faa8c546-c75b-4e10-93b4-661aa2b5f53e-agent-certs\") pod \"konnectivity-agent-444jx\" (UID: \"faa8c546-c75b-4e10-93b4-661aa2b5f53e\") " pod="kube-system/konnectivity-agent-444jx" Jul 29 19:17:06.742994 ip-10-0-131-93 kubenswrapper[2564]: E0729 19:17:06.742968 2564 projected.go:291] Couldn't get configMap openshift-network-diagnostics/kube-root-ca.crt: object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered Jul 29 19:17:06.742994 ip-10-0-131-93 kubenswrapper[2564]: E0729 19:17:06.742991 2564 projected.go:291] Couldn't get configMap openshift-network-diagnostics/openshift-service-ca.crt: object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered Jul 29 19:17:06.743245 ip-10-0-131-93 kubenswrapper[2564]: E0729 19:17:06.743005 2564 projected.go:196] Error preparing data for projected volume kube-api-access-qgvzl for pod openshift-network-diagnostics/network-check-target-464tm: [object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered, object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered] Jul 29 19:17:06.743245 ip-10-0-131-93 kubenswrapper[2564]: E0729 19:17:06.743081 2564 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/projected/02612aaa-a23a-4e6b-b224-123af1bb8230-kube-api-access-qgvzl podName:02612aaa-a23a-4e6b-b224-123af1bb8230 nodeName:}" failed. No retries permitted until 2026-07-29 19:17:07.243063392 +0000 UTC m=+2.967123083 (durationBeforeRetry 500ms). Error: MountVolume.SetUp failed for volume "kube-api-access-qgvzl" (UniqueName: "kubernetes.io/projected/02612aaa-a23a-4e6b-b224-123af1bb8230-kube-api-access-qgvzl") pod "network-check-target-464tm" (UID: "02612aaa-a23a-4e6b-b224-123af1bb8230") : [object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered, object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered] Jul 29 19:17:06.744922 ip-10-0-131-93 kubenswrapper[2564]: I0729 19:17:06.744897 2564 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-xfrc2\" (UniqueName: \"kubernetes.io/projected/ae220db3-299c-481a-9591-01a8f3873d9a-kube-api-access-xfrc2\") pod \"network-metrics-daemon-hkxvx\" (UID: \"ae220db3-299c-481a-9591-01a8f3873d9a\") " pod="openshift-multus/network-metrics-daemon-hkxvx" Jul 29 19:17:06.745448 ip-10-0-131-93 kubenswrapper[2564]: I0729 19:17:06.745113 2564 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-65p97\" (UniqueName: \"kubernetes.io/projected/9a4df9c4-1d87-473b-8d61-abdfb4860574-kube-api-access-65p97\") pod \"aws-ebs-csi-driver-node-z7ptq\" (UID: \"9a4df9c4-1d87-473b-8d61-abdfb4860574\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-z7ptq" Jul 29 19:17:06.745448 ip-10-0-131-93 kubenswrapper[2564]: I0729 19:17:06.745409 2564 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-zlbv2\" (UniqueName: \"kubernetes.io/projected/723728d7-dd35-40ea-bb53-b597ec0e8ef1-kube-api-access-zlbv2\") pod \"iptables-alerter-5n889\" (UID: \"723728d7-dd35-40ea-bb53-b597ec0e8ef1\") " pod="openshift-network-operator/iptables-alerter-5n889" Jul 29 19:17:06.745886 ip-10-0-131-93 kubenswrapper[2564]: I0729 19:17:06.745860 2564 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-6fg74\" (UniqueName: \"kubernetes.io/projected/01073023-f0f3-4517-affc-d5c86021c64d-kube-api-access-6fg74\") pod \"multus-additional-cni-plugins-nmm7d\" (UID: \"01073023-f0f3-4517-affc-d5c86021c64d\") " pod="openshift-multus/multus-additional-cni-plugins-nmm7d" Jul 29 19:17:06.746540 ip-10-0-131-93 kubenswrapper[2564]: I0729 19:17:06.746519 2564 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-7drmm\" (UniqueName: \"kubernetes.io/projected/9568e377-3546-4fbc-bae8-e0ae7766424a-kube-api-access-7drmm\") pod \"ovnkube-node-kl2xl\" (UID: \"9568e377-3546-4fbc-bae8-e0ae7766424a\") " pod="openshift-ovn-kubernetes/ovnkube-node-kl2xl" Jul 29 19:17:06.746620 ip-10-0-131-93 kubenswrapper[2564]: I0729 19:17:06.746543 2564 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-jzc6t\" (UniqueName: \"kubernetes.io/projected/6f081e41-049f-44dc-9aa4-47c2c509d0ff-kube-api-access-jzc6t\") pod \"multus-ccv7v\" (UID: \"6f081e41-049f-44dc-9aa4-47c2c509d0ff\") " pod="openshift-multus/multus-ccv7v" Jul 29 19:17:06.836433 ip-10-0-131-93 kubenswrapper[2564]: I0729 19:17:06.836392 2564 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"var-lib-kubelet\" (UniqueName: \"kubernetes.io/host-path/afd27443-e808-4fe1-b68f-140ec2d0c81f-var-lib-kubelet\") pod \"tuned-s26vt\" (UID: \"afd27443-e808-4fe1-b68f-140ec2d0c81f\") " pod="openshift-cluster-node-tuning-operator/tuned-s26vt" Jul 29 19:17:06.836433 ip-10-0-131-93 kubenswrapper[2564]: I0729 19:17:06.836435 2564 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"hosts-file\" (UniqueName: \"kubernetes.io/host-path/ffe05666-ed11-4573-9442-a9646b974e61-hosts-file\") pod \"node-resolver-lzsmp\" (UID: \"ffe05666-ed11-4573-9442-a9646b974e61\") " pod="openshift-dns/node-resolver-lzsmp" Jul 29 19:17:06.836722 ip-10-0-131-93 kubenswrapper[2564]: I0729 19:17:06.836459 2564 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"serviceca\" (UniqueName: \"kubernetes.io/configmap/edfe2676-f3b8-4eaa-ad17-ecf737b9f786-serviceca\") pod \"node-ca-2lv4n\" (UID: \"edfe2676-f3b8-4eaa-ad17-ecf737b9f786\") " pod="openshift-image-registry/node-ca-2lv4n" Jul 29 19:17:06.836722 ip-10-0-131-93 kubenswrapper[2564]: I0729 19:17:06.836475 2564 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"var-lib-kubelet\" (UniqueName: \"kubernetes.io/host-path/afd27443-e808-4fe1-b68f-140ec2d0c81f-var-lib-kubelet\") pod \"tuned-s26vt\" (UID: \"afd27443-e808-4fe1-b68f-140ec2d0c81f\") " pod="openshift-cluster-node-tuning-operator/tuned-s26vt" Jul 29 19:17:06.836722 ip-10-0-131-93 kubenswrapper[2564]: I0729 19:17:06.836502 2564 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"hosts-file\" (UniqueName: \"kubernetes.io/host-path/ffe05666-ed11-4573-9442-a9646b974e61-hosts-file\") pod \"node-resolver-lzsmp\" (UID: \"ffe05666-ed11-4573-9442-a9646b974e61\") " pod="openshift-dns/node-resolver-lzsmp" Jul 29 19:17:06.836722 ip-10-0-131-93 kubenswrapper[2564]: I0729 19:17:06.836483 2564 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host\" (UniqueName: \"kubernetes.io/host-path/edfe2676-f3b8-4eaa-ad17-ecf737b9f786-host\") pod \"node-ca-2lv4n\" (UID: \"edfe2676-f3b8-4eaa-ad17-ecf737b9f786\") " pod="openshift-image-registry/node-ca-2lv4n" Jul 29 19:17:06.836722 ip-10-0-131-93 kubenswrapper[2564]: I0729 19:17:06.836530 2564 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"host\" (UniqueName: \"kubernetes.io/host-path/edfe2676-f3b8-4eaa-ad17-ecf737b9f786-host\") pod \"node-ca-2lv4n\" (UID: \"edfe2676-f3b8-4eaa-ad17-ecf737b9f786\") " pod="openshift-image-registry/node-ca-2lv4n" Jul 29 19:17:06.836722 ip-10-0-131-93 kubenswrapper[2564]: I0729 19:17:06.836577 2564 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"run\" (UniqueName: \"kubernetes.io/host-path/afd27443-e808-4fe1-b68f-140ec2d0c81f-run\") pod \"tuned-s26vt\" (UID: \"afd27443-e808-4fe1-b68f-140ec2d0c81f\") " pod="openshift-cluster-node-tuning-operator/tuned-s26vt" Jul 29 19:17:06.836722 ip-10-0-131-93 kubenswrapper[2564]: I0729 19:17:06.836609 2564 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"etc-tuned\" (UniqueName: \"kubernetes.io/empty-dir/afd27443-e808-4fe1-b68f-140ec2d0c81f-etc-tuned\") pod \"tuned-s26vt\" (UID: \"afd27443-e808-4fe1-b68f-140ec2d0c81f\") " pod="openshift-cluster-node-tuning-operator/tuned-s26vt" Jul 29 19:17:06.836722 ip-10-0-131-93 kubenswrapper[2564]: I0729 19:17:06.836627 2564 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-sdfb2\" (UniqueName: \"kubernetes.io/projected/edfe2676-f3b8-4eaa-ad17-ecf737b9f786-kube-api-access-sdfb2\") pod \"node-ca-2lv4n\" (UID: \"edfe2676-f3b8-4eaa-ad17-ecf737b9f786\") " pod="openshift-image-registry/node-ca-2lv4n" Jul 29 19:17:06.836722 ip-10-0-131-93 kubenswrapper[2564]: I0729 19:17:06.836648 2564 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kubelet-config\" (UniqueName: \"kubernetes.io/host-path/5c0eec03-fbbc-40db-9c78-43c8f98534c7-kubelet-config\") pod \"global-pull-secret-syncer-zcgr9\" (UID: \"5c0eec03-fbbc-40db-9c78-43c8f98534c7\") " pod="kube-system/global-pull-secret-syncer-zcgr9" Jul 29 19:17:06.836722 ip-10-0-131-93 kubenswrapper[2564]: I0729 19:17:06.836680 2564 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tmp\" (UniqueName: \"kubernetes.io/empty-dir/afd27443-e808-4fe1-b68f-140ec2d0c81f-tmp\") pod \"tuned-s26vt\" (UID: \"afd27443-e808-4fe1-b68f-140ec2d0c81f\") " pod="openshift-cluster-node-tuning-operator/tuned-s26vt" Jul 29 19:17:06.836722 ip-10-0-131-93 kubenswrapper[2564]: I0729 19:17:06.836706 2564 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"etc-modprobe-d\" (UniqueName: \"kubernetes.io/host-path/afd27443-e808-4fe1-b68f-140ec2d0c81f-etc-modprobe-d\") pod \"tuned-s26vt\" (UID: \"afd27443-e808-4fe1-b68f-140ec2d0c81f\") " pod="openshift-cluster-node-tuning-operator/tuned-s26vt" Jul 29 19:17:06.836722 ip-10-0-131-93 kubenswrapper[2564]: I0729 19:17:06.836716 2564 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"run\" (UniqueName: \"kubernetes.io/host-path/afd27443-e808-4fe1-b68f-140ec2d0c81f-run\") pod \"tuned-s26vt\" (UID: \"afd27443-e808-4fe1-b68f-140ec2d0c81f\") " pod="openshift-cluster-node-tuning-operator/tuned-s26vt" Jul 29 19:17:06.836722 ip-10-0-131-93 kubenswrapper[2564]: I0729 19:17:06.836729 2564 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"etc-sysconfig\" (UniqueName: \"kubernetes.io/host-path/afd27443-e808-4fe1-b68f-140ec2d0c81f-etc-sysconfig\") pod \"tuned-s26vt\" (UID: \"afd27443-e808-4fe1-b68f-140ec2d0c81f\") " pod="openshift-cluster-node-tuning-operator/tuned-s26vt" Jul 29 19:17:06.837292 ip-10-0-131-93 kubenswrapper[2564]: I0729 19:17:06.836741 2564 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kubelet-config\" (UniqueName: \"kubernetes.io/host-path/5c0eec03-fbbc-40db-9c78-43c8f98534c7-kubelet-config\") pod \"global-pull-secret-syncer-zcgr9\" (UID: \"5c0eec03-fbbc-40db-9c78-43c8f98534c7\") " pod="kube-system/global-pull-secret-syncer-zcgr9" Jul 29 19:17:06.837292 ip-10-0-131-93 kubenswrapper[2564]: I0729 19:17:06.836752 2564 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"sys\" (UniqueName: \"kubernetes.io/host-path/afd27443-e808-4fe1-b68f-140ec2d0c81f-sys\") pod \"tuned-s26vt\" (UID: \"afd27443-e808-4fe1-b68f-140ec2d0c81f\") " pod="openshift-cluster-node-tuning-operator/tuned-s26vt" Jul 29 19:17:06.837292 ip-10-0-131-93 kubenswrapper[2564]: I0729 19:17:06.836786 2564 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"etc-sysctl-conf\" (UniqueName: \"kubernetes.io/host-path/afd27443-e808-4fe1-b68f-140ec2d0c81f-etc-sysctl-conf\") pod \"tuned-s26vt\" (UID: \"afd27443-e808-4fe1-b68f-140ec2d0c81f\") " pod="openshift-cluster-node-tuning-operator/tuned-s26vt" Jul 29 19:17:06.837292 ip-10-0-131-93 kubenswrapper[2564]: I0729 19:17:06.836822 2564 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"dbus\" (UniqueName: \"kubernetes.io/host-path/5c0eec03-fbbc-40db-9c78-43c8f98534c7-dbus\") pod \"global-pull-secret-syncer-zcgr9\" (UID: \"5c0eec03-fbbc-40db-9c78-43c8f98534c7\") " pod="kube-system/global-pull-secret-syncer-zcgr9" Jul 29 19:17:06.837292 ip-10-0-131-93 kubenswrapper[2564]: I0729 19:17:06.836846 2564 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"etc-kubernetes\" (UniqueName: \"kubernetes.io/host-path/afd27443-e808-4fe1-b68f-140ec2d0c81f-etc-kubernetes\") pod \"tuned-s26vt\" (UID: \"afd27443-e808-4fe1-b68f-140ec2d0c81f\") " pod="openshift-cluster-node-tuning-operator/tuned-s26vt" Jul 29 19:17:06.837292 ip-10-0-131-93 kubenswrapper[2564]: I0729 19:17:06.836858 2564 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"etc-modprobe-d\" (UniqueName: \"kubernetes.io/host-path/afd27443-e808-4fe1-b68f-140ec2d0c81f-etc-modprobe-d\") pod \"tuned-s26vt\" (UID: \"afd27443-e808-4fe1-b68f-140ec2d0c81f\") " pod="openshift-cluster-node-tuning-operator/tuned-s26vt" Jul 29 19:17:06.837292 ip-10-0-131-93 kubenswrapper[2564]: I0729 19:17:06.836867 2564 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host\" (UniqueName: \"kubernetes.io/host-path/afd27443-e808-4fe1-b68f-140ec2d0c81f-host\") pod \"tuned-s26vt\" (UID: \"afd27443-e808-4fe1-b68f-140ec2d0c81f\") " pod="openshift-cluster-node-tuning-operator/tuned-s26vt" Jul 29 19:17:06.837292 ip-10-0-131-93 kubenswrapper[2564]: I0729 19:17:06.836890 2564 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-hz5lq\" (UniqueName: \"kubernetes.io/projected/afd27443-e808-4fe1-b68f-140ec2d0c81f-kube-api-access-hz5lq\") pod \"tuned-s26vt\" (UID: \"afd27443-e808-4fe1-b68f-140ec2d0c81f\") " pod="openshift-cluster-node-tuning-operator/tuned-s26vt" Jul 29 19:17:06.837292 ip-10-0-131-93 kubenswrapper[2564]: I0729 19:17:06.836913 2564 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-mvp9q\" (UniqueName: \"kubernetes.io/projected/ffe05666-ed11-4573-9442-a9646b974e61-kube-api-access-mvp9q\") pod \"node-resolver-lzsmp\" (UID: \"ffe05666-ed11-4573-9442-a9646b974e61\") " pod="openshift-dns/node-resolver-lzsmp" Jul 29 19:17:06.837292 ip-10-0-131-93 kubenswrapper[2564]: I0729 19:17:06.836952 2564 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"host\" (UniqueName: \"kubernetes.io/host-path/afd27443-e808-4fe1-b68f-140ec2d0c81f-host\") pod \"tuned-s26vt\" (UID: \"afd27443-e808-4fe1-b68f-140ec2d0c81f\") " pod="openshift-cluster-node-tuning-operator/tuned-s26vt" Jul 29 19:17:06.837292 ip-10-0-131-93 kubenswrapper[2564]: I0729 19:17:06.836954 2564 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"lib-modules\" (UniqueName: \"kubernetes.io/host-path/afd27443-e808-4fe1-b68f-140ec2d0c81f-lib-modules\") pod \"tuned-s26vt\" (UID: \"afd27443-e808-4fe1-b68f-140ec2d0c81f\") " pod="openshift-cluster-node-tuning-operator/tuned-s26vt" Jul 29 19:17:06.837292 ip-10-0-131-93 kubenswrapper[2564]: I0729 19:17:06.836997 2564 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"etc-sysctl-d\" (UniqueName: \"kubernetes.io/host-path/afd27443-e808-4fe1-b68f-140ec2d0c81f-etc-sysctl-d\") pod \"tuned-s26vt\" (UID: \"afd27443-e808-4fe1-b68f-140ec2d0c81f\") " pod="openshift-cluster-node-tuning-operator/tuned-s26vt" Jul 29 19:17:06.837292 ip-10-0-131-93 kubenswrapper[2564]: I0729 19:17:06.837019 2564 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"dbus\" (UniqueName: \"kubernetes.io/host-path/5c0eec03-fbbc-40db-9c78-43c8f98534c7-dbus\") pod \"global-pull-secret-syncer-zcgr9\" (UID: \"5c0eec03-fbbc-40db-9c78-43c8f98534c7\") " pod="kube-system/global-pull-secret-syncer-zcgr9" Jul 29 19:17:06.837292 ip-10-0-131-93 kubenswrapper[2564]: I0729 19:17:06.837021 2564 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"etc-systemd\" (UniqueName: \"kubernetes.io/host-path/afd27443-e808-4fe1-b68f-140ec2d0c81f-etc-systemd\") pod \"tuned-s26vt\" (UID: \"afd27443-e808-4fe1-b68f-140ec2d0c81f\") " pod="openshift-cluster-node-tuning-operator/tuned-s26vt" Jul 29 19:17:06.837292 ip-10-0-131-93 kubenswrapper[2564]: I0729 19:17:06.837030 2564 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"etc-kubernetes\" (UniqueName: \"kubernetes.io/host-path/afd27443-e808-4fe1-b68f-140ec2d0c81f-etc-kubernetes\") pod \"tuned-s26vt\" (UID: \"afd27443-e808-4fe1-b68f-140ec2d0c81f\") " pod="openshift-cluster-node-tuning-operator/tuned-s26vt" Jul 29 19:17:06.837292 ip-10-0-131-93 kubenswrapper[2564]: I0729 19:17:06.837051 2564 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"lib-modules\" (UniqueName: \"kubernetes.io/host-path/afd27443-e808-4fe1-b68f-140ec2d0c81f-lib-modules\") pod \"tuned-s26vt\" (UID: \"afd27443-e808-4fe1-b68f-140ec2d0c81f\") " pod="openshift-cluster-node-tuning-operator/tuned-s26vt" Jul 29 19:17:06.837292 ip-10-0-131-93 kubenswrapper[2564]: I0729 19:17:06.837062 2564 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/ffe05666-ed11-4573-9442-a9646b974e61-tmp-dir\") pod \"node-resolver-lzsmp\" (UID: \"ffe05666-ed11-4573-9442-a9646b974e61\") " pod="openshift-dns/node-resolver-lzsmp" Jul 29 19:17:06.837292 ip-10-0-131-93 kubenswrapper[2564]: I0729 19:17:06.837090 2564 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"original-pull-secret\" (UniqueName: \"kubernetes.io/secret/5c0eec03-fbbc-40db-9c78-43c8f98534c7-original-pull-secret\") pod \"global-pull-secret-syncer-zcgr9\" (UID: \"5c0eec03-fbbc-40db-9c78-43c8f98534c7\") " pod="kube-system/global-pull-secret-syncer-zcgr9" Jul 29 19:17:06.838153 ip-10-0-131-93 kubenswrapper[2564]: I0729 19:17:06.837103 2564 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"sys\" (UniqueName: \"kubernetes.io/host-path/afd27443-e808-4fe1-b68f-140ec2d0c81f-sys\") pod \"tuned-s26vt\" (UID: \"afd27443-e808-4fe1-b68f-140ec2d0c81f\") " pod="openshift-cluster-node-tuning-operator/tuned-s26vt" Jul 29 19:17:06.838153 ip-10-0-131-93 kubenswrapper[2564]: I0729 19:17:06.837108 2564 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"etc-sysctl-d\" (UniqueName: \"kubernetes.io/host-path/afd27443-e808-4fe1-b68f-140ec2d0c81f-etc-sysctl-d\") pod \"tuned-s26vt\" (UID: \"afd27443-e808-4fe1-b68f-140ec2d0c81f\") " pod="openshift-cluster-node-tuning-operator/tuned-s26vt" Jul 29 19:17:06.838153 ip-10-0-131-93 kubenswrapper[2564]: I0729 19:17:06.837150 2564 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"etc-systemd\" (UniqueName: \"kubernetes.io/host-path/afd27443-e808-4fe1-b68f-140ec2d0c81f-etc-systemd\") pod \"tuned-s26vt\" (UID: \"afd27443-e808-4fe1-b68f-140ec2d0c81f\") " pod="openshift-cluster-node-tuning-operator/tuned-s26vt" Jul 29 19:17:06.838153 ip-10-0-131-93 kubenswrapper[2564]: I0729 19:17:06.837151 2564 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"etc-sysconfig\" (UniqueName: \"kubernetes.io/host-path/afd27443-e808-4fe1-b68f-140ec2d0c81f-etc-sysconfig\") pod \"tuned-s26vt\" (UID: \"afd27443-e808-4fe1-b68f-140ec2d0c81f\") " pod="openshift-cluster-node-tuning-operator/tuned-s26vt" Jul 29 19:17:06.838153 ip-10-0-131-93 kubenswrapper[2564]: E0729 19:17:06.837185 2564 secret.go:189] Couldn't get secret kube-system/original-pull-secret: object "kube-system"/"original-pull-secret" not registered Jul 29 19:17:06.838153 ip-10-0-131-93 kubenswrapper[2564]: E0729 19:17:06.837244 2564 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/5c0eec03-fbbc-40db-9c78-43c8f98534c7-original-pull-secret podName:5c0eec03-fbbc-40db-9c78-43c8f98534c7 nodeName:}" failed. No retries permitted until 2026-07-29 19:17:07.337228263 +0000 UTC m=+3.061287940 (durationBeforeRetry 500ms). Error: MountVolume.SetUp failed for volume "original-pull-secret" (UniqueName: "kubernetes.io/secret/5c0eec03-fbbc-40db-9c78-43c8f98534c7-original-pull-secret") pod "global-pull-secret-syncer-zcgr9" (UID: "5c0eec03-fbbc-40db-9c78-43c8f98534c7") : object "kube-system"/"original-pull-secret" not registered Jul 29 19:17:06.838153 ip-10-0-131-93 kubenswrapper[2564]: I0729 19:17:06.836902 2564 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"serviceca\" (UniqueName: \"kubernetes.io/configmap/edfe2676-f3b8-4eaa-ad17-ecf737b9f786-serviceca\") pod \"node-ca-2lv4n\" (UID: \"edfe2676-f3b8-4eaa-ad17-ecf737b9f786\") " pod="openshift-image-registry/node-ca-2lv4n" Jul 29 19:17:06.838153 ip-10-0-131-93 kubenswrapper[2564]: I0729 19:17:06.837279 2564 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"etc-sysctl-conf\" (UniqueName: \"kubernetes.io/host-path/afd27443-e808-4fe1-b68f-140ec2d0c81f-etc-sysctl-conf\") pod \"tuned-s26vt\" (UID: \"afd27443-e808-4fe1-b68f-140ec2d0c81f\") " pod="openshift-cluster-node-tuning-operator/tuned-s26vt" Jul 29 19:17:06.838153 ip-10-0-131-93 kubenswrapper[2564]: I0729 19:17:06.837554 2564 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/ffe05666-ed11-4573-9442-a9646b974e61-tmp-dir\") pod \"node-resolver-lzsmp\" (UID: \"ffe05666-ed11-4573-9442-a9646b974e61\") " pod="openshift-dns/node-resolver-lzsmp" Jul 29 19:17:06.839151 ip-10-0-131-93 kubenswrapper[2564]: I0729 19:17:06.839132 2564 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"etc-tuned\" (UniqueName: \"kubernetes.io/empty-dir/afd27443-e808-4fe1-b68f-140ec2d0c81f-etc-tuned\") pod \"tuned-s26vt\" (UID: \"afd27443-e808-4fe1-b68f-140ec2d0c81f\") " pod="openshift-cluster-node-tuning-operator/tuned-s26vt" Jul 29 19:17:06.839231 ip-10-0-131-93 kubenswrapper[2564]: I0729 19:17:06.839187 2564 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"tmp\" (UniqueName: \"kubernetes.io/empty-dir/afd27443-e808-4fe1-b68f-140ec2d0c81f-tmp\") pod \"tuned-s26vt\" (UID: \"afd27443-e808-4fe1-b68f-140ec2d0c81f\") " pod="openshift-cluster-node-tuning-operator/tuned-s26vt" Jul 29 19:17:06.845578 ip-10-0-131-93 kubenswrapper[2564]: I0729 19:17:06.845547 2564 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-hz5lq\" (UniqueName: \"kubernetes.io/projected/afd27443-e808-4fe1-b68f-140ec2d0c81f-kube-api-access-hz5lq\") pod \"tuned-s26vt\" (UID: \"afd27443-e808-4fe1-b68f-140ec2d0c81f\") " pod="openshift-cluster-node-tuning-operator/tuned-s26vt" Jul 29 19:17:06.846272 ip-10-0-131-93 kubenswrapper[2564]: I0729 19:17:06.846252 2564 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-mvp9q\" (UniqueName: \"kubernetes.io/projected/ffe05666-ed11-4573-9442-a9646b974e61-kube-api-access-mvp9q\") pod \"node-resolver-lzsmp\" (UID: \"ffe05666-ed11-4573-9442-a9646b974e61\") " pod="openshift-dns/node-resolver-lzsmp" Jul 29 19:17:06.846514 ip-10-0-131-93 kubenswrapper[2564]: I0729 19:17:06.846498 2564 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-sdfb2\" (UniqueName: \"kubernetes.io/projected/edfe2676-f3b8-4eaa-ad17-ecf737b9f786-kube-api-access-sdfb2\") pod \"node-ca-2lv4n\" (UID: \"edfe2676-f3b8-4eaa-ad17-ecf737b9f786\") " pod="openshift-image-registry/node-ca-2lv4n" Jul 29 19:17:06.925601 ip-10-0-131-93 kubenswrapper[2564]: I0729 19:17:06.925562 2564 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/multus-ccv7v" Jul 29 19:17:06.931169 ip-10-0-131-93 kubenswrapper[2564]: I0729 19:17:06.931143 2564 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ovn-kubernetes/ovnkube-node-kl2xl" Jul 29 19:17:06.941134 ip-10-0-131-93 kubenswrapper[2564]: I0729 19:17:06.941110 2564 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-z7ptq" Jul 29 19:17:06.945967 ip-10-0-131-93 kubenswrapper[2564]: I0729 19:17:06.945944 2564 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/konnectivity-agent-444jx" Jul 29 19:17:06.955656 ip-10-0-131-93 kubenswrapper[2564]: I0729 19:17:06.955635 2564 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-operator/iptables-alerter-5n889" Jul 29 19:17:06.961250 ip-10-0-131-93 kubenswrapper[2564]: I0729 19:17:06.961228 2564 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/multus-additional-cni-plugins-nmm7d" Jul 29 19:17:06.968085 ip-10-0-131-93 kubenswrapper[2564]: I0729 19:17:06.968069 2564 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-dns/node-resolver-lzsmp" Jul 29 19:17:06.979891 ip-10-0-131-93 kubenswrapper[2564]: I0729 19:17:06.979841 2564 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-cluster-node-tuning-operator/tuned-s26vt" Jul 29 19:17:06.984605 ip-10-0-131-93 kubenswrapper[2564]: I0729 19:17:06.984587 2564 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-image-registry/node-ca-2lv4n" Jul 29 19:17:07.240285 ip-10-0-131-93 kubenswrapper[2564]: I0729 19:17:07.240204 2564 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/ae220db3-299c-481a-9591-01a8f3873d9a-metrics-certs\") pod \"network-metrics-daemon-hkxvx\" (UID: \"ae220db3-299c-481a-9591-01a8f3873d9a\") " pod="openshift-multus/network-metrics-daemon-hkxvx" Jul 29 19:17:07.240446 ip-10-0-131-93 kubenswrapper[2564]: E0729 19:17:07.240341 2564 secret.go:189] Couldn't get secret openshift-multus/metrics-daemon-secret: object "openshift-multus"/"metrics-daemon-secret" not registered Jul 29 19:17:07.240446 ip-10-0-131-93 kubenswrapper[2564]: E0729 19:17:07.240426 2564 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/ae220db3-299c-481a-9591-01a8f3873d9a-metrics-certs podName:ae220db3-299c-481a-9591-01a8f3873d9a nodeName:}" failed. No retries permitted until 2026-07-29 19:17:08.240406841 +0000 UTC m=+3.964466514 (durationBeforeRetry 1s). Error: MountVolume.SetUp failed for volume "metrics-certs" (UniqueName: "kubernetes.io/secret/ae220db3-299c-481a-9591-01a8f3873d9a-metrics-certs") pod "network-metrics-daemon-hkxvx" (UID: "ae220db3-299c-481a-9591-01a8f3873d9a") : object "openshift-multus"/"metrics-daemon-secret" not registered Jul 29 19:17:07.341496 ip-10-0-131-93 kubenswrapper[2564]: I0729 19:17:07.341456 2564 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-qgvzl\" (UniqueName: \"kubernetes.io/projected/02612aaa-a23a-4e6b-b224-123af1bb8230-kube-api-access-qgvzl\") pod \"network-check-target-464tm\" (UID: \"02612aaa-a23a-4e6b-b224-123af1bb8230\") " pod="openshift-network-diagnostics/network-check-target-464tm" Jul 29 19:17:07.341496 ip-10-0-131-93 kubenswrapper[2564]: I0729 19:17:07.341505 2564 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"original-pull-secret\" (UniqueName: \"kubernetes.io/secret/5c0eec03-fbbc-40db-9c78-43c8f98534c7-original-pull-secret\") pod \"global-pull-secret-syncer-zcgr9\" (UID: \"5c0eec03-fbbc-40db-9c78-43c8f98534c7\") " pod="kube-system/global-pull-secret-syncer-zcgr9" Jul 29 19:17:07.341734 ip-10-0-131-93 kubenswrapper[2564]: E0729 19:17:07.341641 2564 secret.go:189] Couldn't get secret kube-system/original-pull-secret: object "kube-system"/"original-pull-secret" not registered Jul 29 19:17:07.341734 ip-10-0-131-93 kubenswrapper[2564]: E0729 19:17:07.341650 2564 projected.go:291] Couldn't get configMap openshift-network-diagnostics/kube-root-ca.crt: object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered Jul 29 19:17:07.341734 ip-10-0-131-93 kubenswrapper[2564]: E0729 19:17:07.341679 2564 projected.go:291] Couldn't get configMap openshift-network-diagnostics/openshift-service-ca.crt: object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered Jul 29 19:17:07.341734 ip-10-0-131-93 kubenswrapper[2564]: E0729 19:17:07.341695 2564 projected.go:196] Error preparing data for projected volume kube-api-access-qgvzl for pod openshift-network-diagnostics/network-check-target-464tm: [object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered, object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered] Jul 29 19:17:07.341734 ip-10-0-131-93 kubenswrapper[2564]: E0729 19:17:07.341698 2564 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/5c0eec03-fbbc-40db-9c78-43c8f98534c7-original-pull-secret podName:5c0eec03-fbbc-40db-9c78-43c8f98534c7 nodeName:}" failed. No retries permitted until 2026-07-29 19:17:08.341681946 +0000 UTC m=+4.065741619 (durationBeforeRetry 1s). Error: MountVolume.SetUp failed for volume "original-pull-secret" (UniqueName: "kubernetes.io/secret/5c0eec03-fbbc-40db-9c78-43c8f98534c7-original-pull-secret") pod "global-pull-secret-syncer-zcgr9" (UID: "5c0eec03-fbbc-40db-9c78-43c8f98534c7") : object "kube-system"/"original-pull-secret" not registered Jul 29 19:17:07.341930 ip-10-0-131-93 kubenswrapper[2564]: E0729 19:17:07.341750 2564 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/projected/02612aaa-a23a-4e6b-b224-123af1bb8230-kube-api-access-qgvzl podName:02612aaa-a23a-4e6b-b224-123af1bb8230 nodeName:}" failed. No retries permitted until 2026-07-29 19:17:08.341733103 +0000 UTC m=+4.065792795 (durationBeforeRetry 1s). Error: MountVolume.SetUp failed for volume "kube-api-access-qgvzl" (UniqueName: "kubernetes.io/projected/02612aaa-a23a-4e6b-b224-123af1bb8230-kube-api-access-qgvzl") pod "network-check-target-464tm" (UID: "02612aaa-a23a-4e6b-b224-123af1bb8230") : [object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered, object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered] Jul 29 19:17:07.614333 ip-10-0-131-93 kubenswrapper[2564]: W0729 19:17:07.614303 2564 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod9a4df9c4_1d87_473b_8d61_abdfb4860574.slice/crio-0e3d46b10bf256eb467db504229547f219e20e6b09aa90c912cd14d4e1086bb1 WatchSource:0}: Error finding container 0e3d46b10bf256eb467db504229547f219e20e6b09aa90c912cd14d4e1086bb1: Status 404 returned error can't find the container with id 0e3d46b10bf256eb467db504229547f219e20e6b09aa90c912cd14d4e1086bb1 Jul 29 19:17:07.619213 ip-10-0-131-93 kubenswrapper[2564]: W0729 19:17:07.619191 2564 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod6f081e41_049f_44dc_9aa4_47c2c509d0ff.slice/crio-b07333e564b11e39caa6b51f3c285a026354df9a4ee979d8bbfb1decfed14586 WatchSource:0}: Error finding container b07333e564b11e39caa6b51f3c285a026354df9a4ee979d8bbfb1decfed14586: Status 404 returned error can't find the container with id b07333e564b11e39caa6b51f3c285a026354df9a4ee979d8bbfb1decfed14586 Jul 29 19:17:07.620396 ip-10-0-131-93 kubenswrapper[2564]: W0729 19:17:07.620363 2564 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podfaa8c546_c75b_4e10_93b4_661aa2b5f53e.slice/crio-edd4609bb6cc1db4a87b6d37ea4600cc6387c56c7b06c2f1e48537f27bb83263 WatchSource:0}: Error finding container edd4609bb6cc1db4a87b6d37ea4600cc6387c56c7b06c2f1e48537f27bb83263: Status 404 returned error can't find the container with id edd4609bb6cc1db4a87b6d37ea4600cc6387c56c7b06c2f1e48537f27bb83263 Jul 29 19:17:07.621704 ip-10-0-131-93 kubenswrapper[2564]: W0729 19:17:07.621650 2564 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod723728d7_dd35_40ea_bb53_b597ec0e8ef1.slice/crio-4f7d00ef27dbf5d4677091b047b5daa633eb38ad674ea658b11b523a0ae5dd27 WatchSource:0}: Error finding container 4f7d00ef27dbf5d4677091b047b5daa633eb38ad674ea658b11b523a0ae5dd27: Status 404 returned error can't find the container with id 4f7d00ef27dbf5d4677091b047b5daa633eb38ad674ea658b11b523a0ae5dd27 Jul 29 19:17:07.623130 ip-10-0-131-93 kubenswrapper[2564]: W0729 19:17:07.623109 2564 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod9568e377_3546_4fbc_bae8_e0ae7766424a.slice/crio-ebcf497244bb4b86a6cbef51b4321cf91ab0282c5c9bf27886e0022898a3d284 WatchSource:0}: Error finding container ebcf497244bb4b86a6cbef51b4321cf91ab0282c5c9bf27886e0022898a3d284: Status 404 returned error can't find the container with id ebcf497244bb4b86a6cbef51b4321cf91ab0282c5c9bf27886e0022898a3d284 Jul 29 19:17:07.624838 ip-10-0-131-93 kubenswrapper[2564]: W0729 19:17:07.624785 2564 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podffe05666_ed11_4573_9442_a9646b974e61.slice/crio-6d5e31a940dc5306840c8f0a985d355f1ac7b3d4c0561613da971637aa7e16ba WatchSource:0}: Error finding container 6d5e31a940dc5306840c8f0a985d355f1ac7b3d4c0561613da971637aa7e16ba: Status 404 returned error can't find the container with id 6d5e31a940dc5306840c8f0a985d355f1ac7b3d4c0561613da971637aa7e16ba Jul 29 19:17:07.627010 ip-10-0-131-93 kubenswrapper[2564]: W0729 19:17:07.626727 2564 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podedfe2676_f3b8_4eaa_ad17_ecf737b9f786.slice/crio-ec77afcff2931cb78f364549b33c2b9607f64b637da0cf49d0bb2d8bf7dfead4 WatchSource:0}: Error finding container ec77afcff2931cb78f364549b33c2b9607f64b637da0cf49d0bb2d8bf7dfead4: Status 404 returned error can't find the container with id ec77afcff2931cb78f364549b33c2b9607f64b637da0cf49d0bb2d8bf7dfead4 Jul 29 19:17:07.629786 ip-10-0-131-93 kubenswrapper[2564]: W0729 19:17:07.629750 2564 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod01073023_f0f3_4517_affc_d5c86021c64d.slice/crio-3776cf109f8261d4bbf84a3819d57c68bbe4c779b928b04077307aab190d758b WatchSource:0}: Error finding container 3776cf109f8261d4bbf84a3819d57c68bbe4c779b928b04077307aab190d758b: Status 404 returned error can't find the container with id 3776cf109f8261d4bbf84a3819d57c68bbe4c779b928b04077307aab190d758b Jul 29 19:17:07.631059 ip-10-0-131-93 kubenswrapper[2564]: W0729 19:17:07.631035 2564 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podafd27443_e808_4fe1_b68f_140ec2d0c81f.slice/crio-f9a82fede3c7554d124250e7e10cb8b2878985e39a84ba9292f5de58ae73e203 WatchSource:0}: Error finding container f9a82fede3c7554d124250e7e10cb8b2878985e39a84ba9292f5de58ae73e203: Status 404 returned error can't find the container with id f9a82fede3c7554d124250e7e10cb8b2878985e39a84ba9292f5de58ae73e203 Jul 29 19:17:07.664129 ip-10-0-131-93 kubenswrapper[2564]: I0729 19:17:07.663984 2564 certificate_manager.go:715] "Certificate rotation deadline determined" logger="kubernetes.io/kubelet-serving" expiration="2028-07-28 19:12:06 +0000 UTC" deadline="2028-05-07 06:19:37.00538249 +0000 UTC" Jul 29 19:17:07.664129 ip-10-0-131-93 kubenswrapper[2564]: I0729 19:17:07.664129 2564 certificate_manager.go:431] "Waiting for next certificate rotation" logger="kubernetes.io/kubelet-serving" sleep="15539h2m29.341257028s" Jul 29 19:17:07.776813 ip-10-0-131-93 kubenswrapper[2564]: I0729 19:17:07.776781 2564 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-zcgr9" Jul 29 19:17:07.776986 ip-10-0-131-93 kubenswrapper[2564]: E0729 19:17:07.776915 2564 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-zcgr9" podUID="5c0eec03-fbbc-40db-9c78-43c8f98534c7" Jul 29 19:17:07.786201 ip-10-0-131-93 kubenswrapper[2564]: I0729 19:17:07.786174 2564 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ovn-kubernetes/ovnkube-node-kl2xl" event={"ID":"9568e377-3546-4fbc-bae8-e0ae7766424a","Type":"ContainerStarted","Data":"ebcf497244bb4b86a6cbef51b4321cf91ab0282c5c9bf27886e0022898a3d284"} Jul 29 19:17:07.787432 ip-10-0-131-93 kubenswrapper[2564]: I0729 19:17:07.787402 2564 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-network-operator/iptables-alerter-5n889" event={"ID":"723728d7-dd35-40ea-bb53-b597ec0e8ef1","Type":"ContainerStarted","Data":"4f7d00ef27dbf5d4677091b047b5daa633eb38ad674ea658b11b523a0ae5dd27"} Jul 29 19:17:07.788512 ip-10-0-131-93 kubenswrapper[2564]: I0729 19:17:07.788488 2564 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-multus/multus-ccv7v" event={"ID":"6f081e41-049f-44dc-9aa4-47c2c509d0ff","Type":"ContainerStarted","Data":"b07333e564b11e39caa6b51f3c285a026354df9a4ee979d8bbfb1decfed14586"} Jul 29 19:17:07.789709 ip-10-0-131-93 kubenswrapper[2564]: I0729 19:17:07.789681 2564 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-z7ptq" event={"ID":"9a4df9c4-1d87-473b-8d61-abdfb4860574","Type":"ContainerStarted","Data":"0e3d46b10bf256eb467db504229547f219e20e6b09aa90c912cd14d4e1086bb1"} Jul 29 19:17:07.791690 ip-10-0-131-93 kubenswrapper[2564]: I0729 19:17:07.791670 2564 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kube-system/kube-apiserver-proxy-ip-10-0-131-93.ec2.internal" event={"ID":"a1b5cf65d676393f097a7d0896c452c5","Type":"ContainerStarted","Data":"125cc2d5deadd5ddcdc06c17ba1e7551d5123548f126879a2b9dff592d74bb7c"} Jul 29 19:17:07.794556 ip-10-0-131-93 kubenswrapper[2564]: I0729 19:17:07.794534 2564 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-cluster-node-tuning-operator/tuned-s26vt" event={"ID":"afd27443-e808-4fe1-b68f-140ec2d0c81f","Type":"ContainerStarted","Data":"f9a82fede3c7554d124250e7e10cb8b2878985e39a84ba9292f5de58ae73e203"} Jul 29 19:17:07.795947 ip-10-0-131-93 kubenswrapper[2564]: I0729 19:17:07.795929 2564 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-dns/node-resolver-lzsmp" event={"ID":"ffe05666-ed11-4573-9442-a9646b974e61","Type":"ContainerStarted","Data":"6d5e31a940dc5306840c8f0a985d355f1ac7b3d4c0561613da971637aa7e16ba"} Jul 29 19:17:07.796929 ip-10-0-131-93 kubenswrapper[2564]: I0729 19:17:07.796912 2564 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-multus/multus-additional-cni-plugins-nmm7d" event={"ID":"01073023-f0f3-4517-affc-d5c86021c64d","Type":"ContainerStarted","Data":"3776cf109f8261d4bbf84a3819d57c68bbe4c779b928b04077307aab190d758b"} Jul 29 19:17:07.797868 ip-10-0-131-93 kubenswrapper[2564]: I0729 19:17:07.797849 2564 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-image-registry/node-ca-2lv4n" event={"ID":"edfe2676-f3b8-4eaa-ad17-ecf737b9f786","Type":"ContainerStarted","Data":"ec77afcff2931cb78f364549b33c2b9607f64b637da0cf49d0bb2d8bf7dfead4"} Jul 29 19:17:07.798903 ip-10-0-131-93 kubenswrapper[2564]: I0729 19:17:07.798877 2564 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kube-system/konnectivity-agent-444jx" event={"ID":"faa8c546-c75b-4e10-93b4-661aa2b5f53e","Type":"ContainerStarted","Data":"edd4609bb6cc1db4a87b6d37ea4600cc6387c56c7b06c2f1e48537f27bb83263"} Jul 29 19:17:07.810031 ip-10-0-131-93 kubenswrapper[2564]: I0729 19:17:07.809973 2564 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="kube-system/kube-apiserver-proxy-ip-10-0-131-93.ec2.internal" podStartSLOduration=2.809953724 podStartE2EDuration="2.809953724s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=true podCreationTimestamp="2026-07-29 19:17:05 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-29 19:17:07.809409741 +0000 UTC m=+3.533469432" watchObservedRunningTime="2026-07-29 19:17:07.809953724 +0000 UTC m=+3.534013417" Jul 29 19:17:08.250269 ip-10-0-131-93 kubenswrapper[2564]: I0729 19:17:08.250228 2564 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/ae220db3-299c-481a-9591-01a8f3873d9a-metrics-certs\") pod \"network-metrics-daemon-hkxvx\" (UID: \"ae220db3-299c-481a-9591-01a8f3873d9a\") " pod="openshift-multus/network-metrics-daemon-hkxvx" Jul 29 19:17:08.250487 ip-10-0-131-93 kubenswrapper[2564]: E0729 19:17:08.250454 2564 secret.go:189] Couldn't get secret openshift-multus/metrics-daemon-secret: object "openshift-multus"/"metrics-daemon-secret" not registered Jul 29 19:17:08.250577 ip-10-0-131-93 kubenswrapper[2564]: E0729 19:17:08.250523 2564 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/ae220db3-299c-481a-9591-01a8f3873d9a-metrics-certs podName:ae220db3-299c-481a-9591-01a8f3873d9a nodeName:}" failed. No retries permitted until 2026-07-29 19:17:10.250502306 +0000 UTC m=+5.974561992 (durationBeforeRetry 2s). Error: MountVolume.SetUp failed for volume "metrics-certs" (UniqueName: "kubernetes.io/secret/ae220db3-299c-481a-9591-01a8f3873d9a-metrics-certs") pod "network-metrics-daemon-hkxvx" (UID: "ae220db3-299c-481a-9591-01a8f3873d9a") : object "openshift-multus"/"metrics-daemon-secret" not registered Jul 29 19:17:08.350880 ip-10-0-131-93 kubenswrapper[2564]: I0729 19:17:08.350820 2564 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-qgvzl\" (UniqueName: \"kubernetes.io/projected/02612aaa-a23a-4e6b-b224-123af1bb8230-kube-api-access-qgvzl\") pod \"network-check-target-464tm\" (UID: \"02612aaa-a23a-4e6b-b224-123af1bb8230\") " pod="openshift-network-diagnostics/network-check-target-464tm" Jul 29 19:17:08.351079 ip-10-0-131-93 kubenswrapper[2564]: I0729 19:17:08.350913 2564 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"original-pull-secret\" (UniqueName: \"kubernetes.io/secret/5c0eec03-fbbc-40db-9c78-43c8f98534c7-original-pull-secret\") pod \"global-pull-secret-syncer-zcgr9\" (UID: \"5c0eec03-fbbc-40db-9c78-43c8f98534c7\") " pod="kube-system/global-pull-secret-syncer-zcgr9" Jul 29 19:17:08.351142 ip-10-0-131-93 kubenswrapper[2564]: E0729 19:17:08.351087 2564 secret.go:189] Couldn't get secret kube-system/original-pull-secret: object "kube-system"/"original-pull-secret" not registered Jul 29 19:17:08.351196 ip-10-0-131-93 kubenswrapper[2564]: E0729 19:17:08.351150 2564 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/5c0eec03-fbbc-40db-9c78-43c8f98534c7-original-pull-secret podName:5c0eec03-fbbc-40db-9c78-43c8f98534c7 nodeName:}" failed. No retries permitted until 2026-07-29 19:17:10.351133028 +0000 UTC m=+6.075192700 (durationBeforeRetry 2s). Error: MountVolume.SetUp failed for volume "original-pull-secret" (UniqueName: "kubernetes.io/secret/5c0eec03-fbbc-40db-9c78-43c8f98534c7-original-pull-secret") pod "global-pull-secret-syncer-zcgr9" (UID: "5c0eec03-fbbc-40db-9c78-43c8f98534c7") : object "kube-system"/"original-pull-secret" not registered Jul 29 19:17:08.351265 ip-10-0-131-93 kubenswrapper[2564]: E0729 19:17:08.351240 2564 projected.go:291] Couldn't get configMap openshift-network-diagnostics/kube-root-ca.crt: object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered Jul 29 19:17:08.351265 ip-10-0-131-93 kubenswrapper[2564]: E0729 19:17:08.351257 2564 projected.go:291] Couldn't get configMap openshift-network-diagnostics/openshift-service-ca.crt: object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered Jul 29 19:17:08.351365 ip-10-0-131-93 kubenswrapper[2564]: E0729 19:17:08.351270 2564 projected.go:196] Error preparing data for projected volume kube-api-access-qgvzl for pod openshift-network-diagnostics/network-check-target-464tm: [object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered, object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered] Jul 29 19:17:08.351365 ip-10-0-131-93 kubenswrapper[2564]: E0729 19:17:08.351306 2564 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/projected/02612aaa-a23a-4e6b-b224-123af1bb8230-kube-api-access-qgvzl podName:02612aaa-a23a-4e6b-b224-123af1bb8230 nodeName:}" failed. No retries permitted until 2026-07-29 19:17:10.35129455 +0000 UTC m=+6.075354225 (durationBeforeRetry 2s). Error: MountVolume.SetUp failed for volume "kube-api-access-qgvzl" (UniqueName: "kubernetes.io/projected/02612aaa-a23a-4e6b-b224-123af1bb8230-kube-api-access-qgvzl") pod "network-check-target-464tm" (UID: "02612aaa-a23a-4e6b-b224-123af1bb8230") : [object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered, object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered] Jul 29 19:17:08.664906 ip-10-0-131-93 kubenswrapper[2564]: I0729 19:17:08.664791 2564 certificate_manager.go:715] "Certificate rotation deadline determined" logger="kubernetes.io/kubelet-serving" expiration="2028-07-28 19:12:06 +0000 UTC" deadline="2028-02-26 10:29:21.303808796 +0000 UTC" Jul 29 19:17:08.664906 ip-10-0-131-93 kubenswrapper[2564]: I0729 19:17:08.664855 2564 certificate_manager.go:431] "Waiting for next certificate rotation" logger="kubernetes.io/kubelet-serving" sleep="13839h12m12.638958089s" Jul 29 19:17:08.778430 ip-10-0-131-93 kubenswrapper[2564]: I0729 19:17:08.778395 2564 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-464tm" Jul 29 19:17:08.778601 ip-10-0-131-93 kubenswrapper[2564]: E0729 19:17:08.778535 2564 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-464tm" podUID="02612aaa-a23a-4e6b-b224-123af1bb8230" Jul 29 19:17:08.778669 ip-10-0-131-93 kubenswrapper[2564]: I0729 19:17:08.778624 2564 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-hkxvx" Jul 29 19:17:08.778720 ip-10-0-131-93 kubenswrapper[2564]: E0729 19:17:08.778707 2564 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-hkxvx" podUID="ae220db3-299c-481a-9591-01a8f3873d9a" Jul 29 19:17:08.813465 ip-10-0-131-93 kubenswrapper[2564]: I0729 19:17:08.813432 2564 generic.go:356] "Generic (PLEG): container finished" podID="f5406ede63492de294830916375c7f86" containerID="0cd642dbb0d34a451b56a6c266da637127ae2c5889a360c14afd0fee60a9ff35" exitCode=0 Jul 29 19:17:08.814317 ip-10-0-131-93 kubenswrapper[2564]: I0729 19:17:08.814293 2564 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-131-93.ec2.internal" event={"ID":"f5406ede63492de294830916375c7f86","Type":"ContainerDied","Data":"0cd642dbb0d34a451b56a6c266da637127ae2c5889a360c14afd0fee60a9ff35"} Jul 29 19:17:09.777242 ip-10-0-131-93 kubenswrapper[2564]: I0729 19:17:09.777207 2564 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-zcgr9" Jul 29 19:17:09.777756 ip-10-0-131-93 kubenswrapper[2564]: E0729 19:17:09.777346 2564 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-zcgr9" podUID="5c0eec03-fbbc-40db-9c78-43c8f98534c7" Jul 29 19:17:09.823474 ip-10-0-131-93 kubenswrapper[2564]: I0729 19:17:09.823433 2564 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-131-93.ec2.internal" event={"ID":"f5406ede63492de294830916375c7f86","Type":"ContainerStarted","Data":"2a3578c7ee1bfeddc674c35733c968ea0c73ec98e4d575bfbdcb733067fd8141"} Jul 29 19:17:10.266448 ip-10-0-131-93 kubenswrapper[2564]: I0729 19:17:10.265772 2564 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/ae220db3-299c-481a-9591-01a8f3873d9a-metrics-certs\") pod \"network-metrics-daemon-hkxvx\" (UID: \"ae220db3-299c-481a-9591-01a8f3873d9a\") " pod="openshift-multus/network-metrics-daemon-hkxvx" Jul 29 19:17:10.266448 ip-10-0-131-93 kubenswrapper[2564]: E0729 19:17:10.265955 2564 secret.go:189] Couldn't get secret openshift-multus/metrics-daemon-secret: object "openshift-multus"/"metrics-daemon-secret" not registered Jul 29 19:17:10.266448 ip-10-0-131-93 kubenswrapper[2564]: E0729 19:17:10.266023 2564 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/ae220db3-299c-481a-9591-01a8f3873d9a-metrics-certs podName:ae220db3-299c-481a-9591-01a8f3873d9a nodeName:}" failed. No retries permitted until 2026-07-29 19:17:14.266005401 +0000 UTC m=+9.990065071 (durationBeforeRetry 4s). Error: MountVolume.SetUp failed for volume "metrics-certs" (UniqueName: "kubernetes.io/secret/ae220db3-299c-481a-9591-01a8f3873d9a-metrics-certs") pod "network-metrics-daemon-hkxvx" (UID: "ae220db3-299c-481a-9591-01a8f3873d9a") : object "openshift-multus"/"metrics-daemon-secret" not registered Jul 29 19:17:10.367525 ip-10-0-131-93 kubenswrapper[2564]: I0729 19:17:10.366719 2564 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-qgvzl\" (UniqueName: \"kubernetes.io/projected/02612aaa-a23a-4e6b-b224-123af1bb8230-kube-api-access-qgvzl\") pod \"network-check-target-464tm\" (UID: \"02612aaa-a23a-4e6b-b224-123af1bb8230\") " pod="openshift-network-diagnostics/network-check-target-464tm" Jul 29 19:17:10.367525 ip-10-0-131-93 kubenswrapper[2564]: I0729 19:17:10.366772 2564 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"original-pull-secret\" (UniqueName: \"kubernetes.io/secret/5c0eec03-fbbc-40db-9c78-43c8f98534c7-original-pull-secret\") pod \"global-pull-secret-syncer-zcgr9\" (UID: \"5c0eec03-fbbc-40db-9c78-43c8f98534c7\") " pod="kube-system/global-pull-secret-syncer-zcgr9" Jul 29 19:17:10.367525 ip-10-0-131-93 kubenswrapper[2564]: E0729 19:17:10.366897 2564 secret.go:189] Couldn't get secret kube-system/original-pull-secret: object "kube-system"/"original-pull-secret" not registered Jul 29 19:17:10.367525 ip-10-0-131-93 kubenswrapper[2564]: E0729 19:17:10.366956 2564 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/5c0eec03-fbbc-40db-9c78-43c8f98534c7-original-pull-secret podName:5c0eec03-fbbc-40db-9c78-43c8f98534c7 nodeName:}" failed. No retries permitted until 2026-07-29 19:17:14.366939256 +0000 UTC m=+10.090998932 (durationBeforeRetry 4s). Error: MountVolume.SetUp failed for volume "original-pull-secret" (UniqueName: "kubernetes.io/secret/5c0eec03-fbbc-40db-9c78-43c8f98534c7-original-pull-secret") pod "global-pull-secret-syncer-zcgr9" (UID: "5c0eec03-fbbc-40db-9c78-43c8f98534c7") : object "kube-system"/"original-pull-secret" not registered Jul 29 19:17:10.367525 ip-10-0-131-93 kubenswrapper[2564]: E0729 19:17:10.367350 2564 projected.go:291] Couldn't get configMap openshift-network-diagnostics/kube-root-ca.crt: object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered Jul 29 19:17:10.367525 ip-10-0-131-93 kubenswrapper[2564]: E0729 19:17:10.367386 2564 projected.go:291] Couldn't get configMap openshift-network-diagnostics/openshift-service-ca.crt: object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered Jul 29 19:17:10.367525 ip-10-0-131-93 kubenswrapper[2564]: E0729 19:17:10.367400 2564 projected.go:196] Error preparing data for projected volume kube-api-access-qgvzl for pod openshift-network-diagnostics/network-check-target-464tm: [object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered, object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered] Jul 29 19:17:10.367525 ip-10-0-131-93 kubenswrapper[2564]: E0729 19:17:10.367444 2564 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/projected/02612aaa-a23a-4e6b-b224-123af1bb8230-kube-api-access-qgvzl podName:02612aaa-a23a-4e6b-b224-123af1bb8230 nodeName:}" failed. No retries permitted until 2026-07-29 19:17:14.367429595 +0000 UTC m=+10.091489267 (durationBeforeRetry 4s). Error: MountVolume.SetUp failed for volume "kube-api-access-qgvzl" (UniqueName: "kubernetes.io/projected/02612aaa-a23a-4e6b-b224-123af1bb8230-kube-api-access-qgvzl") pod "network-check-target-464tm" (UID: "02612aaa-a23a-4e6b-b224-123af1bb8230") : [object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered, object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered] Jul 29 19:17:10.778036 ip-10-0-131-93 kubenswrapper[2564]: I0729 19:17:10.777586 2564 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-hkxvx" Jul 29 19:17:10.778036 ip-10-0-131-93 kubenswrapper[2564]: I0729 19:17:10.777587 2564 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-464tm" Jul 29 19:17:10.778036 ip-10-0-131-93 kubenswrapper[2564]: E0729 19:17:10.777704 2564 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-hkxvx" podUID="ae220db3-299c-481a-9591-01a8f3873d9a" Jul 29 19:17:10.778036 ip-10-0-131-93 kubenswrapper[2564]: E0729 19:17:10.777787 2564 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-464tm" podUID="02612aaa-a23a-4e6b-b224-123af1bb8230" Jul 29 19:17:11.777339 ip-10-0-131-93 kubenswrapper[2564]: I0729 19:17:11.777304 2564 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-zcgr9" Jul 29 19:17:11.777536 ip-10-0-131-93 kubenswrapper[2564]: E0729 19:17:11.777456 2564 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-zcgr9" podUID="5c0eec03-fbbc-40db-9c78-43c8f98534c7" Jul 29 19:17:12.777411 ip-10-0-131-93 kubenswrapper[2564]: I0729 19:17:12.777365 2564 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-hkxvx" Jul 29 19:17:12.777829 ip-10-0-131-93 kubenswrapper[2564]: I0729 19:17:12.777365 2564 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-464tm" Jul 29 19:17:12.777829 ip-10-0-131-93 kubenswrapper[2564]: E0729 19:17:12.777523 2564 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-hkxvx" podUID="ae220db3-299c-481a-9591-01a8f3873d9a" Jul 29 19:17:12.777829 ip-10-0-131-93 kubenswrapper[2564]: E0729 19:17:12.777587 2564 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-464tm" podUID="02612aaa-a23a-4e6b-b224-123af1bb8230" Jul 29 19:17:13.777238 ip-10-0-131-93 kubenswrapper[2564]: I0729 19:17:13.777029 2564 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-zcgr9" Jul 29 19:17:13.777238 ip-10-0-131-93 kubenswrapper[2564]: E0729 19:17:13.777169 2564 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-zcgr9" podUID="5c0eec03-fbbc-40db-9c78-43c8f98534c7" Jul 29 19:17:14.297839 ip-10-0-131-93 kubenswrapper[2564]: I0729 19:17:14.297216 2564 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/ae220db3-299c-481a-9591-01a8f3873d9a-metrics-certs\") pod \"network-metrics-daemon-hkxvx\" (UID: \"ae220db3-299c-481a-9591-01a8f3873d9a\") " pod="openshift-multus/network-metrics-daemon-hkxvx" Jul 29 19:17:14.297839 ip-10-0-131-93 kubenswrapper[2564]: E0729 19:17:14.297429 2564 secret.go:189] Couldn't get secret openshift-multus/metrics-daemon-secret: object "openshift-multus"/"metrics-daemon-secret" not registered Jul 29 19:17:14.297839 ip-10-0-131-93 kubenswrapper[2564]: E0729 19:17:14.297495 2564 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/ae220db3-299c-481a-9591-01a8f3873d9a-metrics-certs podName:ae220db3-299c-481a-9591-01a8f3873d9a nodeName:}" failed. No retries permitted until 2026-07-29 19:17:22.297474321 +0000 UTC m=+18.021534006 (durationBeforeRetry 8s). Error: MountVolume.SetUp failed for volume "metrics-certs" (UniqueName: "kubernetes.io/secret/ae220db3-299c-481a-9591-01a8f3873d9a-metrics-certs") pod "network-metrics-daemon-hkxvx" (UID: "ae220db3-299c-481a-9591-01a8f3873d9a") : object "openshift-multus"/"metrics-daemon-secret" not registered Jul 29 19:17:14.398383 ip-10-0-131-93 kubenswrapper[2564]: I0729 19:17:14.398330 2564 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-qgvzl\" (UniqueName: \"kubernetes.io/projected/02612aaa-a23a-4e6b-b224-123af1bb8230-kube-api-access-qgvzl\") pod \"network-check-target-464tm\" (UID: \"02612aaa-a23a-4e6b-b224-123af1bb8230\") " pod="openshift-network-diagnostics/network-check-target-464tm" Jul 29 19:17:14.398604 ip-10-0-131-93 kubenswrapper[2564]: I0729 19:17:14.398396 2564 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"original-pull-secret\" (UniqueName: \"kubernetes.io/secret/5c0eec03-fbbc-40db-9c78-43c8f98534c7-original-pull-secret\") pod \"global-pull-secret-syncer-zcgr9\" (UID: \"5c0eec03-fbbc-40db-9c78-43c8f98534c7\") " pod="kube-system/global-pull-secret-syncer-zcgr9" Jul 29 19:17:14.398604 ip-10-0-131-93 kubenswrapper[2564]: E0729 19:17:14.398488 2564 secret.go:189] Couldn't get secret kube-system/original-pull-secret: object "kube-system"/"original-pull-secret" not registered Jul 29 19:17:14.398604 ip-10-0-131-93 kubenswrapper[2564]: E0729 19:17:14.398532 2564 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/5c0eec03-fbbc-40db-9c78-43c8f98534c7-original-pull-secret podName:5c0eec03-fbbc-40db-9c78-43c8f98534c7 nodeName:}" failed. No retries permitted until 2026-07-29 19:17:22.398519665 +0000 UTC m=+18.122579337 (durationBeforeRetry 8s). Error: MountVolume.SetUp failed for volume "original-pull-secret" (UniqueName: "kubernetes.io/secret/5c0eec03-fbbc-40db-9c78-43c8f98534c7-original-pull-secret") pod "global-pull-secret-syncer-zcgr9" (UID: "5c0eec03-fbbc-40db-9c78-43c8f98534c7") : object "kube-system"/"original-pull-secret" not registered Jul 29 19:17:14.398919 ip-10-0-131-93 kubenswrapper[2564]: E0729 19:17:14.398885 2564 projected.go:291] Couldn't get configMap openshift-network-diagnostics/kube-root-ca.crt: object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered Jul 29 19:17:14.398919 ip-10-0-131-93 kubenswrapper[2564]: E0729 19:17:14.398914 2564 projected.go:291] Couldn't get configMap openshift-network-diagnostics/openshift-service-ca.crt: object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered Jul 29 19:17:14.399021 ip-10-0-131-93 kubenswrapper[2564]: E0729 19:17:14.398930 2564 projected.go:196] Error preparing data for projected volume kube-api-access-qgvzl for pod openshift-network-diagnostics/network-check-target-464tm: [object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered, object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered] Jul 29 19:17:14.399021 ip-10-0-131-93 kubenswrapper[2564]: E0729 19:17:14.398991 2564 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/projected/02612aaa-a23a-4e6b-b224-123af1bb8230-kube-api-access-qgvzl podName:02612aaa-a23a-4e6b-b224-123af1bb8230 nodeName:}" failed. No retries permitted until 2026-07-29 19:17:22.398975109 +0000 UTC m=+18.123034780 (durationBeforeRetry 8s). Error: MountVolume.SetUp failed for volume "kube-api-access-qgvzl" (UniqueName: "kubernetes.io/projected/02612aaa-a23a-4e6b-b224-123af1bb8230-kube-api-access-qgvzl") pod "network-check-target-464tm" (UID: "02612aaa-a23a-4e6b-b224-123af1bb8230") : [object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered, object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered] Jul 29 19:17:14.778470 ip-10-0-131-93 kubenswrapper[2564]: I0729 19:17:14.778397 2564 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-hkxvx" Jul 29 19:17:14.778914 ip-10-0-131-93 kubenswrapper[2564]: E0729 19:17:14.778525 2564 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-hkxvx" podUID="ae220db3-299c-481a-9591-01a8f3873d9a" Jul 29 19:17:14.778914 ip-10-0-131-93 kubenswrapper[2564]: I0729 19:17:14.778642 2564 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-464tm" Jul 29 19:17:14.778914 ip-10-0-131-93 kubenswrapper[2564]: E0729 19:17:14.778756 2564 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-464tm" podUID="02612aaa-a23a-4e6b-b224-123af1bb8230" Jul 29 19:17:15.777183 ip-10-0-131-93 kubenswrapper[2564]: I0729 19:17:15.777149 2564 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-zcgr9" Jul 29 19:17:15.777439 ip-10-0-131-93 kubenswrapper[2564]: E0729 19:17:15.777287 2564 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-zcgr9" podUID="5c0eec03-fbbc-40db-9c78-43c8f98534c7" Jul 29 19:17:16.777534 ip-10-0-131-93 kubenswrapper[2564]: I0729 19:17:16.777468 2564 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-464tm" Jul 29 19:17:16.777534 ip-10-0-131-93 kubenswrapper[2564]: I0729 19:17:16.777485 2564 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-hkxvx" Jul 29 19:17:16.778036 ip-10-0-131-93 kubenswrapper[2564]: E0729 19:17:16.777598 2564 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-464tm" podUID="02612aaa-a23a-4e6b-b224-123af1bb8230" Jul 29 19:17:16.778036 ip-10-0-131-93 kubenswrapper[2564]: E0729 19:17:16.777979 2564 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-hkxvx" podUID="ae220db3-299c-481a-9591-01a8f3873d9a" Jul 29 19:17:17.777251 ip-10-0-131-93 kubenswrapper[2564]: I0729 19:17:17.777219 2564 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-zcgr9" Jul 29 19:17:17.777436 ip-10-0-131-93 kubenswrapper[2564]: E0729 19:17:17.777342 2564 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-zcgr9" podUID="5c0eec03-fbbc-40db-9c78-43c8f98534c7" Jul 29 19:17:18.776786 ip-10-0-131-93 kubenswrapper[2564]: I0729 19:17:18.776748 2564 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-hkxvx" Jul 29 19:17:18.776786 ip-10-0-131-93 kubenswrapper[2564]: I0729 19:17:18.776773 2564 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-464tm" Jul 29 19:17:18.777303 ip-10-0-131-93 kubenswrapper[2564]: E0729 19:17:18.776894 2564 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-hkxvx" podUID="ae220db3-299c-481a-9591-01a8f3873d9a" Jul 29 19:17:18.777303 ip-10-0-131-93 kubenswrapper[2564]: E0729 19:17:18.777054 2564 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-464tm" podUID="02612aaa-a23a-4e6b-b224-123af1bb8230" Jul 29 19:17:19.777464 ip-10-0-131-93 kubenswrapper[2564]: I0729 19:17:19.777431 2564 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-zcgr9" Jul 29 19:17:19.777871 ip-10-0-131-93 kubenswrapper[2564]: E0729 19:17:19.777559 2564 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-zcgr9" podUID="5c0eec03-fbbc-40db-9c78-43c8f98534c7" Jul 29 19:17:20.777226 ip-10-0-131-93 kubenswrapper[2564]: I0729 19:17:20.777187 2564 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-hkxvx" Jul 29 19:17:20.777409 ip-10-0-131-93 kubenswrapper[2564]: E0729 19:17:20.777325 2564 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-hkxvx" podUID="ae220db3-299c-481a-9591-01a8f3873d9a" Jul 29 19:17:20.777409 ip-10-0-131-93 kubenswrapper[2564]: I0729 19:17:20.777389 2564 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-464tm" Jul 29 19:17:20.777536 ip-10-0-131-93 kubenswrapper[2564]: E0729 19:17:20.777499 2564 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-464tm" podUID="02612aaa-a23a-4e6b-b224-123af1bb8230" Jul 29 19:17:21.777445 ip-10-0-131-93 kubenswrapper[2564]: I0729 19:17:21.777404 2564 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-zcgr9" Jul 29 19:17:21.777642 ip-10-0-131-93 kubenswrapper[2564]: E0729 19:17:21.777517 2564 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-zcgr9" podUID="5c0eec03-fbbc-40db-9c78-43c8f98534c7" Jul 29 19:17:22.363944 ip-10-0-131-93 kubenswrapper[2564]: I0729 19:17:22.363898 2564 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/ae220db3-299c-481a-9591-01a8f3873d9a-metrics-certs\") pod \"network-metrics-daemon-hkxvx\" (UID: \"ae220db3-299c-481a-9591-01a8f3873d9a\") " pod="openshift-multus/network-metrics-daemon-hkxvx" Jul 29 19:17:22.364118 ip-10-0-131-93 kubenswrapper[2564]: E0729 19:17:22.364062 2564 secret.go:189] Couldn't get secret openshift-multus/metrics-daemon-secret: object "openshift-multus"/"metrics-daemon-secret" not registered Jul 29 19:17:22.364173 ip-10-0-131-93 kubenswrapper[2564]: E0729 19:17:22.364125 2564 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/ae220db3-299c-481a-9591-01a8f3873d9a-metrics-certs podName:ae220db3-299c-481a-9591-01a8f3873d9a nodeName:}" failed. No retries permitted until 2026-07-29 19:17:38.364108959 +0000 UTC m=+34.088168628 (durationBeforeRetry 16s). Error: MountVolume.SetUp failed for volume "metrics-certs" (UniqueName: "kubernetes.io/secret/ae220db3-299c-481a-9591-01a8f3873d9a-metrics-certs") pod "network-metrics-daemon-hkxvx" (UID: "ae220db3-299c-481a-9591-01a8f3873d9a") : object "openshift-multus"/"metrics-daemon-secret" not registered Jul 29 19:17:22.464884 ip-10-0-131-93 kubenswrapper[2564]: I0729 19:17:22.464835 2564 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-qgvzl\" (UniqueName: \"kubernetes.io/projected/02612aaa-a23a-4e6b-b224-123af1bb8230-kube-api-access-qgvzl\") pod \"network-check-target-464tm\" (UID: \"02612aaa-a23a-4e6b-b224-123af1bb8230\") " pod="openshift-network-diagnostics/network-check-target-464tm" Jul 29 19:17:22.464884 ip-10-0-131-93 kubenswrapper[2564]: I0729 19:17:22.464884 2564 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"original-pull-secret\" (UniqueName: \"kubernetes.io/secret/5c0eec03-fbbc-40db-9c78-43c8f98534c7-original-pull-secret\") pod \"global-pull-secret-syncer-zcgr9\" (UID: \"5c0eec03-fbbc-40db-9c78-43c8f98534c7\") " pod="kube-system/global-pull-secret-syncer-zcgr9" Jul 29 19:17:22.465094 ip-10-0-131-93 kubenswrapper[2564]: E0729 19:17:22.465016 2564 secret.go:189] Couldn't get secret kube-system/original-pull-secret: object "kube-system"/"original-pull-secret" not registered Jul 29 19:17:22.465094 ip-10-0-131-93 kubenswrapper[2564]: E0729 19:17:22.465021 2564 projected.go:291] Couldn't get configMap openshift-network-diagnostics/kube-root-ca.crt: object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered Jul 29 19:17:22.465094 ip-10-0-131-93 kubenswrapper[2564]: E0729 19:17:22.465046 2564 projected.go:291] Couldn't get configMap openshift-network-diagnostics/openshift-service-ca.crt: object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered Jul 29 19:17:22.465094 ip-10-0-131-93 kubenswrapper[2564]: E0729 19:17:22.465059 2564 projected.go:196] Error preparing data for projected volume kube-api-access-qgvzl for pod openshift-network-diagnostics/network-check-target-464tm: [object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered, object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered] Jul 29 19:17:22.465094 ip-10-0-131-93 kubenswrapper[2564]: E0729 19:17:22.465074 2564 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/5c0eec03-fbbc-40db-9c78-43c8f98534c7-original-pull-secret podName:5c0eec03-fbbc-40db-9c78-43c8f98534c7 nodeName:}" failed. No retries permitted until 2026-07-29 19:17:38.465056943 +0000 UTC m=+34.189116618 (durationBeforeRetry 16s). Error: MountVolume.SetUp failed for volume "original-pull-secret" (UniqueName: "kubernetes.io/secret/5c0eec03-fbbc-40db-9c78-43c8f98534c7-original-pull-secret") pod "global-pull-secret-syncer-zcgr9" (UID: "5c0eec03-fbbc-40db-9c78-43c8f98534c7") : object "kube-system"/"original-pull-secret" not registered Jul 29 19:17:22.465299 ip-10-0-131-93 kubenswrapper[2564]: E0729 19:17:22.465111 2564 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/projected/02612aaa-a23a-4e6b-b224-123af1bb8230-kube-api-access-qgvzl podName:02612aaa-a23a-4e6b-b224-123af1bb8230 nodeName:}" failed. No retries permitted until 2026-07-29 19:17:38.465094731 +0000 UTC m=+34.189154407 (durationBeforeRetry 16s). Error: MountVolume.SetUp failed for volume "kube-api-access-qgvzl" (UniqueName: "kubernetes.io/projected/02612aaa-a23a-4e6b-b224-123af1bb8230-kube-api-access-qgvzl") pod "network-check-target-464tm" (UID: "02612aaa-a23a-4e6b-b224-123af1bb8230") : [object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered, object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered] Jul 29 19:17:22.777280 ip-10-0-131-93 kubenswrapper[2564]: I0729 19:17:22.777194 2564 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-hkxvx" Jul 29 19:17:22.777467 ip-10-0-131-93 kubenswrapper[2564]: I0729 19:17:22.777194 2564 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-464tm" Jul 29 19:17:22.777467 ip-10-0-131-93 kubenswrapper[2564]: E0729 19:17:22.777345 2564 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-hkxvx" podUID="ae220db3-299c-481a-9591-01a8f3873d9a" Jul 29 19:17:22.777467 ip-10-0-131-93 kubenswrapper[2564]: E0729 19:17:22.777365 2564 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-464tm" podUID="02612aaa-a23a-4e6b-b224-123af1bb8230" Jul 29 19:17:23.777739 ip-10-0-131-93 kubenswrapper[2564]: I0729 19:17:23.777702 2564 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-zcgr9" Jul 29 19:17:23.778153 ip-10-0-131-93 kubenswrapper[2564]: E0729 19:17:23.777811 2564 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-zcgr9" podUID="5c0eec03-fbbc-40db-9c78-43c8f98534c7" Jul 29 19:17:24.777835 ip-10-0-131-93 kubenswrapper[2564]: I0729 19:17:24.777798 2564 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-hkxvx" Jul 29 19:17:24.778321 ip-10-0-131-93 kubenswrapper[2564]: E0729 19:17:24.777911 2564 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-hkxvx" podUID="ae220db3-299c-481a-9591-01a8f3873d9a" Jul 29 19:17:24.778321 ip-10-0-131-93 kubenswrapper[2564]: I0729 19:17:24.777973 2564 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-464tm" Jul 29 19:17:24.778321 ip-10-0-131-93 kubenswrapper[2564]: E0729 19:17:24.778091 2564 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-464tm" podUID="02612aaa-a23a-4e6b-b224-123af1bb8230" Jul 29 19:17:25.777716 ip-10-0-131-93 kubenswrapper[2564]: I0729 19:17:25.777676 2564 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-zcgr9" Jul 29 19:17:25.777896 ip-10-0-131-93 kubenswrapper[2564]: E0729 19:17:25.777814 2564 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-zcgr9" podUID="5c0eec03-fbbc-40db-9c78-43c8f98534c7" Jul 29 19:17:26.777895 ip-10-0-131-93 kubenswrapper[2564]: I0729 19:17:26.777500 2564 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-hkxvx" Jul 29 19:17:26.778772 ip-10-0-131-93 kubenswrapper[2564]: I0729 19:17:26.777555 2564 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-464tm" Jul 29 19:17:26.778772 ip-10-0-131-93 kubenswrapper[2564]: E0729 19:17:26.778017 2564 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-hkxvx" podUID="ae220db3-299c-481a-9591-01a8f3873d9a" Jul 29 19:17:26.778772 ip-10-0-131-93 kubenswrapper[2564]: E0729 19:17:26.778050 2564 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-464tm" podUID="02612aaa-a23a-4e6b-b224-123af1bb8230" Jul 29 19:17:26.864433 ip-10-0-131-93 kubenswrapper[2564]: I0729 19:17:26.864395 2564 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-multus/multus-ccv7v" event={"ID":"6f081e41-049f-44dc-9aa4-47c2c509d0ff","Type":"ContainerStarted","Data":"99e627610b9d0dadcec0574ee811f1e28c8dfe809033c13a1f593675ab373f7f"} Jul 29 19:17:26.865850 ip-10-0-131-93 kubenswrapper[2564]: I0729 19:17:26.865827 2564 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-z7ptq" event={"ID":"9a4df9c4-1d87-473b-8d61-abdfb4860574","Type":"ContainerStarted","Data":"4cc0d45b441b72285e7337e718a8ebab5d420c2d4ed289ef5c1793b21bce1c36"} Jul 29 19:17:26.867132 ip-10-0-131-93 kubenswrapper[2564]: I0729 19:17:26.867110 2564 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-cluster-node-tuning-operator/tuned-s26vt" event={"ID":"afd27443-e808-4fe1-b68f-140ec2d0c81f","Type":"ContainerStarted","Data":"119c6bae8ccface87780feb307feb4eba48903f4ed7494f8f5868333068a0e1d"} Jul 29 19:17:26.868590 ip-10-0-131-93 kubenswrapper[2564]: I0729 19:17:26.868570 2564 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-dns/node-resolver-lzsmp" event={"ID":"ffe05666-ed11-4573-9442-a9646b974e61","Type":"ContainerStarted","Data":"14ba8cab720d1c2002c7dfa957045af98fc07f88b8a616a47d4db4980a3b9698"} Jul 29 19:17:26.869928 ip-10-0-131-93 kubenswrapper[2564]: I0729 19:17:26.869910 2564 generic.go:356] "Generic (PLEG): container finished" podID="01073023-f0f3-4517-affc-d5c86021c64d" containerID="4ce93ee404f0caedcb2666109caf34687fff291ca8ac1a3677a5f09491198d64" exitCode=0 Jul 29 19:17:26.870020 ip-10-0-131-93 kubenswrapper[2564]: I0729 19:17:26.869953 2564 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-multus/multus-additional-cni-plugins-nmm7d" event={"ID":"01073023-f0f3-4517-affc-d5c86021c64d","Type":"ContainerDied","Data":"4ce93ee404f0caedcb2666109caf34687fff291ca8ac1a3677a5f09491198d64"} Jul 29 19:17:26.871469 ip-10-0-131-93 kubenswrapper[2564]: I0729 19:17:26.871444 2564 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-image-registry/node-ca-2lv4n" event={"ID":"edfe2676-f3b8-4eaa-ad17-ecf737b9f786","Type":"ContainerStarted","Data":"cfe2ecb62054f169f2033a2ec50525793fb85475a6bc98c5829149813e152c72"} Jul 29 19:17:26.872781 ip-10-0-131-93 kubenswrapper[2564]: I0729 19:17:26.872758 2564 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kube-system/konnectivity-agent-444jx" event={"ID":"faa8c546-c75b-4e10-93b4-661aa2b5f53e","Type":"ContainerStarted","Data":"2ea5a3a90a3745f63b4ca52802f1e89ee345c707c93b5c636b1b644d790c4830"} Jul 29 19:17:26.875697 ip-10-0-131-93 kubenswrapper[2564]: I0729 19:17:26.875667 2564 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ovn-kubernetes/ovnkube-node-kl2xl" event={"ID":"9568e377-3546-4fbc-bae8-e0ae7766424a","Type":"ContainerStarted","Data":"98a935170c919b35837b5d9f95aa2feee5fdfaf648ec0ed853a37cec9dc3e393"} Jul 29 19:17:26.875777 ip-10-0-131-93 kubenswrapper[2564]: I0729 19:17:26.875730 2564 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ovn-kubernetes/ovnkube-node-kl2xl" event={"ID":"9568e377-3546-4fbc-bae8-e0ae7766424a","Type":"ContainerStarted","Data":"64f321df195818b381782af26115be7e020eb003212ffec2fef8f0aa6c182f30"} Jul 29 19:17:26.875777 ip-10-0-131-93 kubenswrapper[2564]: I0729 19:17:26.875761 2564 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ovn-kubernetes/ovnkube-node-kl2xl" event={"ID":"9568e377-3546-4fbc-bae8-e0ae7766424a","Type":"ContainerStarted","Data":"518d2eac14a6068d671d50492abd2c9d14d3487bdf0eaaee25aea676cbc855ad"} Jul 29 19:17:26.875851 ip-10-0-131-93 kubenswrapper[2564]: I0729 19:17:26.875796 2564 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ovn-kubernetes/ovnkube-node-kl2xl" event={"ID":"9568e377-3546-4fbc-bae8-e0ae7766424a","Type":"ContainerStarted","Data":"1e6b0e7b1a1cd6ca2588f303633e23bc8ba982bfbf11005c268534f7eb483d75"} Jul 29 19:17:26.875851 ip-10-0-131-93 kubenswrapper[2564]: I0729 19:17:26.875822 2564 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ovn-kubernetes/ovnkube-node-kl2xl" event={"ID":"9568e377-3546-4fbc-bae8-e0ae7766424a","Type":"ContainerStarted","Data":"7c7ebcda2d5bd85291afe47ba045171c7dd7251c8747e6c56f7cb6d6d9734cfd"} Jul 29 19:17:26.875851 ip-10-0-131-93 kubenswrapper[2564]: I0729 19:17:26.875848 2564 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ovn-kubernetes/ovnkube-node-kl2xl" event={"ID":"9568e377-3546-4fbc-bae8-e0ae7766424a","Type":"ContainerStarted","Data":"be7a8f7cd1612631c391cfa5e8da780c353f2fed0fa7d25ca2b2409cd8e5665b"} Jul 29 19:17:26.881897 ip-10-0-131-93 kubenswrapper[2564]: I0729 19:17:26.881861 2564 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-131-93.ec2.internal" podStartSLOduration=21.881850708 podStartE2EDuration="21.881850708s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=true podCreationTimestamp="2026-07-29 19:17:05 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-29 19:17:09.842970994 +0000 UTC m=+5.567030686" watchObservedRunningTime="2026-07-29 19:17:26.881850708 +0000 UTC m=+22.605910399" Jul 29 19:17:26.882004 ip-10-0-131-93 kubenswrapper[2564]: I0729 19:17:26.881986 2564 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-multus/multus-ccv7v" podStartSLOduration=4.555332256 podStartE2EDuration="22.881980687s" totalImagesPullingTime="18.326648431s" totalInitContainerRuntime="0s" isStatefulPod=true podCreationTimestamp="2026-07-29 19:17:04 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-29 19:17:26.881470846 +0000 UTC m=+22.605530535" watchObservedRunningTime="2026-07-29 19:17:26.881980687 +0000 UTC m=+22.606040433" Jul 29 19:17:26.895835 ip-10-0-131-93 kubenswrapper[2564]: I0729 19:17:26.895785 2564 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-image-registry/node-ca-2lv4n" podStartSLOduration=3.655928546 podStartE2EDuration="21.895765518s" totalImagesPullingTime="18.239836972s" totalInitContainerRuntime="0s" isStatefulPod=true podCreationTimestamp="2026-07-29 19:17:05 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-29 19:17:26.894757407 +0000 UTC m=+22.618817099" watchObservedRunningTime="2026-07-29 19:17:26.895765518 +0000 UTC m=+22.619825258" Jul 29 19:17:26.929620 ip-10-0-131-93 kubenswrapper[2564]: I0729 19:17:26.929573 2564 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-dns/node-resolver-lzsmp" podStartSLOduration=3.68672948 podStartE2EDuration="21.929557691s" totalImagesPullingTime="18.242828211s" totalInitContainerRuntime="0s" isStatefulPod=true podCreationTimestamp="2026-07-29 19:17:05 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-29 19:17:26.929092572 +0000 UTC m=+22.653152270" watchObservedRunningTime="2026-07-29 19:17:26.929557691 +0000 UTC m=+22.653617383" Jul 29 19:17:26.948617 ip-10-0-131-93 kubenswrapper[2564]: I0729 19:17:26.948564 2564 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-cluster-node-tuning-operator/tuned-s26vt" podStartSLOduration=3.710332549 podStartE2EDuration="21.948548683s" totalImagesPullingTime="18.238216134s" totalInitContainerRuntime="0s" isStatefulPod=true podCreationTimestamp="2026-07-29 19:17:05 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-29 19:17:26.948425441 +0000 UTC m=+22.672485137" watchObservedRunningTime="2026-07-29 19:17:26.948548683 +0000 UTC m=+22.672608374" Jul 29 19:17:26.966176 ip-10-0-131-93 kubenswrapper[2564]: I0729 19:17:26.966129 2564 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="kube-system/konnectivity-agent-444jx" podStartSLOduration=4.719120541 podStartE2EDuration="22.966118443s" totalImagesPullingTime="18.246997902s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-29 19:17:04 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-29 19:17:26.96609963 +0000 UTC m=+22.690159325" watchObservedRunningTime="2026-07-29 19:17:26.966118443 +0000 UTC m=+22.690178134" Jul 29 19:17:27.444885 ip-10-0-131-93 kubenswrapper[2564]: I0729 19:17:27.444860 2564 plugin_watcher.go:194] "Adding socket path or updating timestamp to desired state cache" path="/var/lib/kubelet/plugins_registry/ebs.csi.aws.com-reg.sock" Jul 29 19:17:27.691441 ip-10-0-131-93 kubenswrapper[2564]: I0729 19:17:27.691339 2564 reconciler.go:161] "OperationExecutor.RegisterPlugin started" plugin={"SocketPath":"/var/lib/kubelet/plugins_registry/ebs.csi.aws.com-reg.sock","Timestamp":"2026-07-29T19:17:27.444879594Z","UUID":"91d479ee-7228-491e-9a5b-4cf662edd5b5","Handler":null,"Name":"","Endpoint":""} Jul 29 19:17:27.693675 ip-10-0-131-93 kubenswrapper[2564]: I0729 19:17:27.693651 2564 csi_plugin.go:106] kubernetes.io/csi: Trying to validate a new CSI Driver with name: ebs.csi.aws.com endpoint: /var/lib/kubelet/plugins/ebs.csi.aws.com/csi.sock versions: 1.0.0 Jul 29 19:17:27.693856 ip-10-0-131-93 kubenswrapper[2564]: I0729 19:17:27.693698 2564 csi_plugin.go:119] kubernetes.io/csi: Register new plugin with name: ebs.csi.aws.com at endpoint: /var/lib/kubelet/plugins/ebs.csi.aws.com/csi.sock Jul 29 19:17:27.777614 ip-10-0-131-93 kubenswrapper[2564]: I0729 19:17:27.777577 2564 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-zcgr9" Jul 29 19:17:27.777835 ip-10-0-131-93 kubenswrapper[2564]: E0729 19:17:27.777747 2564 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-zcgr9" podUID="5c0eec03-fbbc-40db-9c78-43c8f98534c7" Jul 29 19:17:27.879862 ip-10-0-131-93 kubenswrapper[2564]: I0729 19:17:27.879646 2564 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-network-operator/iptables-alerter-5n889" event={"ID":"723728d7-dd35-40ea-bb53-b597ec0e8ef1","Type":"ContainerStarted","Data":"aca061d6124c0849b114ab7f9fd9ddae74e3e85e0857bcf881fb00a744cd8236"} Jul 29 19:17:27.882143 ip-10-0-131-93 kubenswrapper[2564]: I0729 19:17:27.882113 2564 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-z7ptq" event={"ID":"9a4df9c4-1d87-473b-8d61-abdfb4860574","Type":"ContainerStarted","Data":"f20961f1962eccaa519c043226708c512a06d071a42102c5538ad5176f98c8f5"} Jul 29 19:17:27.895055 ip-10-0-131-93 kubenswrapper[2564]: I0729 19:17:27.895004 2564 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-network-operator/iptables-alerter-5n889" podStartSLOduration=5.640181223 podStartE2EDuration="23.894989785s" totalImagesPullingTime="18.254808562s" totalInitContainerRuntime="0s" isStatefulPod=true podCreationTimestamp="2026-07-29 19:17:04 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-29 19:17:27.894728999 +0000 UTC m=+23.618788693" watchObservedRunningTime="2026-07-29 19:17:27.894989785 +0000 UTC m=+23.619049535" Jul 29 19:17:28.747563 ip-10-0-131-93 kubenswrapper[2564]: I0729 19:17:28.747326 2564 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="kube-system/konnectivity-agent-444jx" Jul 29 19:17:28.748073 ip-10-0-131-93 kubenswrapper[2564]: I0729 19:17:28.748050 2564 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="kube-system/konnectivity-agent-444jx" Jul 29 19:17:28.780506 ip-10-0-131-93 kubenswrapper[2564]: I0729 19:17:28.780484 2564 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-464tm" Jul 29 19:17:28.780633 ip-10-0-131-93 kubenswrapper[2564]: E0729 19:17:28.780597 2564 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-464tm" podUID="02612aaa-a23a-4e6b-b224-123af1bb8230" Jul 29 19:17:28.780695 ip-10-0-131-93 kubenswrapper[2564]: I0729 19:17:28.780652 2564 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-hkxvx" Jul 29 19:17:28.780775 ip-10-0-131-93 kubenswrapper[2564]: E0729 19:17:28.780748 2564 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-hkxvx" podUID="ae220db3-299c-481a-9591-01a8f3873d9a" Jul 29 19:17:28.889265 ip-10-0-131-93 kubenswrapper[2564]: I0729 19:17:28.889235 2564 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ovn-kubernetes/ovnkube-node-kl2xl" event={"ID":"9568e377-3546-4fbc-bae8-e0ae7766424a","Type":"ContainerStarted","Data":"b0990127c8557cfde1f29b848f3f90042070b0062a22c2d8a15d43db3bb79794"} Jul 29 19:17:29.777641 ip-10-0-131-93 kubenswrapper[2564]: I0729 19:17:29.777613 2564 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-zcgr9" Jul 29 19:17:29.777832 ip-10-0-131-93 kubenswrapper[2564]: E0729 19:17:29.777727 2564 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-zcgr9" podUID="5c0eec03-fbbc-40db-9c78-43c8f98534c7" Jul 29 19:17:29.899908 ip-10-0-131-93 kubenswrapper[2564]: I0729 19:17:29.899865 2564 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-z7ptq" event={"ID":"9a4df9c4-1d87-473b-8d61-abdfb4860574","Type":"ContainerStarted","Data":"32150822825de542ca639ecd1f4fa68a6e54baff2a8ba885883140333d6f099b"} Jul 29 19:17:29.899908 ip-10-0-131-93 kubenswrapper[2564]: I0729 19:17:29.899903 2564 prober_manager.go:312] "Failed to trigger a manual run" probe="Readiness" Jul 29 19:17:29.919433 ip-10-0-131-93 kubenswrapper[2564]: I0729 19:17:29.919393 2564 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-z7ptq" podStartSLOduration=5.114449464 podStartE2EDuration="25.919360326s" totalImagesPullingTime="20.804910862s" totalInitContainerRuntime="0s" isStatefulPod=true podCreationTimestamp="2026-07-29 19:17:04 +0000 UTC" imagePullSessionsCount=3 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-29 19:17:29.919006707 +0000 UTC m=+25.643066399" watchObservedRunningTime="2026-07-29 19:17:29.919360326 +0000 UTC m=+25.643420018" Jul 29 19:17:30.776957 ip-10-0-131-93 kubenswrapper[2564]: I0729 19:17:30.776929 2564 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-hkxvx" Jul 29 19:17:30.777159 ip-10-0-131-93 kubenswrapper[2564]: I0729 19:17:30.776993 2564 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-464tm" Jul 29 19:17:30.777159 ip-10-0-131-93 kubenswrapper[2564]: E0729 19:17:30.777081 2564 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-hkxvx" podUID="ae220db3-299c-481a-9591-01a8f3873d9a" Jul 29 19:17:30.777272 ip-10-0-131-93 kubenswrapper[2564]: E0729 19:17:30.777235 2564 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-464tm" podUID="02612aaa-a23a-4e6b-b224-123af1bb8230" Jul 29 19:17:31.776888 ip-10-0-131-93 kubenswrapper[2564]: I0729 19:17:31.776710 2564 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-zcgr9" Jul 29 19:17:31.777453 ip-10-0-131-93 kubenswrapper[2564]: E0729 19:17:31.776965 2564 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-zcgr9" podUID="5c0eec03-fbbc-40db-9c78-43c8f98534c7" Jul 29 19:17:31.907537 ip-10-0-131-93 kubenswrapper[2564]: I0729 19:17:31.907508 2564 generic.go:356] "Generic (PLEG): container finished" podID="01073023-f0f3-4517-affc-d5c86021c64d" containerID="a8e3950798a9cee0eef664e4d6487a60385366f453f2b3a9ddf52f2ca3b93874" exitCode=0 Jul 29 19:17:31.907721 ip-10-0-131-93 kubenswrapper[2564]: I0729 19:17:31.907591 2564 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-multus/multus-additional-cni-plugins-nmm7d" event={"ID":"01073023-f0f3-4517-affc-d5c86021c64d","Type":"ContainerDied","Data":"a8e3950798a9cee0eef664e4d6487a60385366f453f2b3a9ddf52f2ca3b93874"} Jul 29 19:17:31.911350 ip-10-0-131-93 kubenswrapper[2564]: I0729 19:17:31.911325 2564 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ovn-kubernetes/ovnkube-node-kl2xl" event={"ID":"9568e377-3546-4fbc-bae8-e0ae7766424a","Type":"ContainerStarted","Data":"8b4bdc9cc21284c5f993225a7c3bfdb93b3db2fb7106a32a444f7f25c6036c95"} Jul 29 19:17:31.911626 ip-10-0-131-93 kubenswrapper[2564]: I0729 19:17:31.911607 2564 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ovn-kubernetes/ovnkube-node-kl2xl" Jul 29 19:17:31.911742 ip-10-0-131-93 kubenswrapper[2564]: I0729 19:17:31.911675 2564 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ovn-kubernetes/ovnkube-node-kl2xl" Jul 29 19:17:31.928066 ip-10-0-131-93 kubenswrapper[2564]: I0729 19:17:31.928043 2564 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ovn-kubernetes/ovnkube-node-kl2xl" Jul 29 19:17:31.928200 ip-10-0-131-93 kubenswrapper[2564]: I0729 19:17:31.928181 2564 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ovn-kubernetes/ovnkube-node-kl2xl" Jul 29 19:17:31.976810 ip-10-0-131-93 kubenswrapper[2564]: I0729 19:17:31.976768 2564 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-ovn-kubernetes/ovnkube-node-kl2xl" podStartSLOduration=9.65717981 podStartE2EDuration="27.976749428s" totalImagesPullingTime="18.319569618s" totalInitContainerRuntime="0s" isStatefulPod=true podCreationTimestamp="2026-07-29 19:17:04 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-29 19:17:31.976283025 +0000 UTC m=+27.700342716" watchObservedRunningTime="2026-07-29 19:17:31.976749428 +0000 UTC m=+27.700809119" Jul 29 19:17:32.777416 ip-10-0-131-93 kubenswrapper[2564]: I0729 19:17:32.777383 2564 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-hkxvx" Jul 29 19:17:32.777827 ip-10-0-131-93 kubenswrapper[2564]: I0729 19:17:32.777398 2564 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-464tm" Jul 29 19:17:32.777827 ip-10-0-131-93 kubenswrapper[2564]: E0729 19:17:32.777532 2564 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-hkxvx" podUID="ae220db3-299c-481a-9591-01a8f3873d9a" Jul 29 19:17:32.777827 ip-10-0-131-93 kubenswrapper[2564]: E0729 19:17:32.777576 2564 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-464tm" podUID="02612aaa-a23a-4e6b-b224-123af1bb8230" Jul 29 19:17:32.913141 ip-10-0-131-93 kubenswrapper[2564]: I0729 19:17:32.913118 2564 prober_manager.go:312] "Failed to trigger a manual run" probe="Readiness" Jul 29 19:17:33.077158 ip-10-0-131-93 kubenswrapper[2564]: I0729 19:17:33.077121 2564 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-multus/network-metrics-daemon-hkxvx"] Jul 29 19:17:33.077304 ip-10-0-131-93 kubenswrapper[2564]: I0729 19:17:33.077273 2564 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-hkxvx" Jul 29 19:17:33.077437 ip-10-0-131-93 kubenswrapper[2564]: E0729 19:17:33.077414 2564 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-hkxvx" podUID="ae220db3-299c-481a-9591-01a8f3873d9a" Jul 29 19:17:33.077910 ip-10-0-131-93 kubenswrapper[2564]: I0729 19:17:33.077889 2564 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kube-system/global-pull-secret-syncer-zcgr9"] Jul 29 19:17:33.078023 ip-10-0-131-93 kubenswrapper[2564]: I0729 19:17:33.077992 2564 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-zcgr9" Jul 29 19:17:33.078108 ip-10-0-131-93 kubenswrapper[2564]: E0729 19:17:33.078087 2564 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-zcgr9" podUID="5c0eec03-fbbc-40db-9c78-43c8f98534c7" Jul 29 19:17:33.090310 ip-10-0-131-93 kubenswrapper[2564]: I0729 19:17:33.090283 2564 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-network-diagnostics/network-check-target-464tm"] Jul 29 19:17:33.090445 ip-10-0-131-93 kubenswrapper[2564]: I0729 19:17:33.090408 2564 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-464tm" Jul 29 19:17:33.090506 ip-10-0-131-93 kubenswrapper[2564]: E0729 19:17:33.090490 2564 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-464tm" podUID="02612aaa-a23a-4e6b-b224-123af1bb8230" Jul 29 19:17:33.428432 ip-10-0-131-93 kubenswrapper[2564]: I0729 19:17:33.428404 2564 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="kube-system/konnectivity-agent-444jx" Jul 29 19:17:33.428589 ip-10-0-131-93 kubenswrapper[2564]: I0729 19:17:33.428509 2564 prober_manager.go:312] "Failed to trigger a manual run" probe="Readiness" Jul 29 19:17:33.429121 ip-10-0-131-93 kubenswrapper[2564]: I0729 19:17:33.429100 2564 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="kube-system/konnectivity-agent-444jx" Jul 29 19:17:33.916614 ip-10-0-131-93 kubenswrapper[2564]: I0729 19:17:33.916580 2564 generic.go:356] "Generic (PLEG): container finished" podID="01073023-f0f3-4517-affc-d5c86021c64d" containerID="5c047e3a79fae1a120561afe35393748e53e503d2bee8086aeb52a591f84d468" exitCode=0 Jul 29 19:17:33.917074 ip-10-0-131-93 kubenswrapper[2564]: I0729 19:17:33.916671 2564 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-multus/multus-additional-cni-plugins-nmm7d" event={"ID":"01073023-f0f3-4517-affc-d5c86021c64d","Type":"ContainerDied","Data":"5c047e3a79fae1a120561afe35393748e53e503d2bee8086aeb52a591f84d468"} Jul 29 19:17:33.917074 ip-10-0-131-93 kubenswrapper[2564]: I0729 19:17:33.916926 2564 prober_manager.go:312] "Failed to trigger a manual run" probe="Readiness" Jul 29 19:17:34.781567 ip-10-0-131-93 kubenswrapper[2564]: I0729 19:17:34.778885 2564 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-hkxvx" Jul 29 19:17:34.781567 ip-10-0-131-93 kubenswrapper[2564]: I0729 19:17:34.778962 2564 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-464tm" Jul 29 19:17:34.781567 ip-10-0-131-93 kubenswrapper[2564]: I0729 19:17:34.779101 2564 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-zcgr9" Jul 29 19:17:34.781567 ip-10-0-131-93 kubenswrapper[2564]: E0729 19:17:34.779232 2564 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-zcgr9" podUID="5c0eec03-fbbc-40db-9c78-43c8f98534c7" Jul 29 19:17:34.781567 ip-10-0-131-93 kubenswrapper[2564]: E0729 19:17:34.779788 2564 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-hkxvx" podUID="ae220db3-299c-481a-9591-01a8f3873d9a" Jul 29 19:17:34.781567 ip-10-0-131-93 kubenswrapper[2564]: E0729 19:17:34.779902 2564 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-464tm" podUID="02612aaa-a23a-4e6b-b224-123af1bb8230" Jul 29 19:17:35.923682 ip-10-0-131-93 kubenswrapper[2564]: I0729 19:17:35.923504 2564 generic.go:356] "Generic (PLEG): container finished" podID="01073023-f0f3-4517-affc-d5c86021c64d" containerID="98306244c013eab904146f4676d56f45d1d477ec613bd0dd979ec4693dda4267" exitCode=0 Jul 29 19:17:35.924217 ip-10-0-131-93 kubenswrapper[2564]: I0729 19:17:35.923588 2564 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-multus/multus-additional-cni-plugins-nmm7d" event={"ID":"01073023-f0f3-4517-affc-d5c86021c64d","Type":"ContainerDied","Data":"98306244c013eab904146f4676d56f45d1d477ec613bd0dd979ec4693dda4267"} Jul 29 19:17:36.569671 ip-10-0-131-93 kubenswrapper[2564]: I0729 19:17:36.569641 2564 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ovn-kubernetes/ovnkube-node-kl2xl" Jul 29 19:17:36.569901 ip-10-0-131-93 kubenswrapper[2564]: I0729 19:17:36.569879 2564 prober_manager.go:312] "Failed to trigger a manual run" probe="Readiness" Jul 29 19:17:36.580904 ip-10-0-131-93 kubenswrapper[2564]: I0729 19:17:36.580868 2564 prober.go:120] "Probe failed" probeType="Readiness" pod="openshift-ovn-kubernetes/ovnkube-node-kl2xl" podUID="9568e377-3546-4fbc-bae8-e0ae7766424a" containerName="ovnkube-controller" probeResult="failure" output="" Jul 29 19:17:36.590449 ip-10-0-131-93 kubenswrapper[2564]: I0729 19:17:36.590417 2564 prober.go:120] "Probe failed" probeType="Readiness" pod="openshift-ovn-kubernetes/ovnkube-node-kl2xl" podUID="9568e377-3546-4fbc-bae8-e0ae7766424a" containerName="ovnkube-controller" probeResult="failure" output="" Jul 29 19:17:36.780717 ip-10-0-131-93 kubenswrapper[2564]: I0729 19:17:36.780689 2564 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-zcgr9" Jul 29 19:17:36.780887 ip-10-0-131-93 kubenswrapper[2564]: I0729 19:17:36.780689 2564 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-464tm" Jul 29 19:17:36.780887 ip-10-0-131-93 kubenswrapper[2564]: E0729 19:17:36.780807 2564 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-zcgr9" podUID="5c0eec03-fbbc-40db-9c78-43c8f98534c7" Jul 29 19:17:36.780887 ip-10-0-131-93 kubenswrapper[2564]: I0729 19:17:36.780689 2564 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-hkxvx" Jul 29 19:17:36.780887 ip-10-0-131-93 kubenswrapper[2564]: E0729 19:17:36.780880 2564 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-464tm" podUID="02612aaa-a23a-4e6b-b224-123af1bb8230" Jul 29 19:17:36.781044 ip-10-0-131-93 kubenswrapper[2564]: E0729 19:17:36.780974 2564 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-hkxvx" podUID="ae220db3-299c-481a-9591-01a8f3873d9a" Jul 29 19:17:38.381149 ip-10-0-131-93 kubenswrapper[2564]: I0729 19:17:38.381110 2564 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/ae220db3-299c-481a-9591-01a8f3873d9a-metrics-certs\") pod \"network-metrics-daemon-hkxvx\" (UID: \"ae220db3-299c-481a-9591-01a8f3873d9a\") " pod="openshift-multus/network-metrics-daemon-hkxvx" Jul 29 19:17:38.381780 ip-10-0-131-93 kubenswrapper[2564]: E0729 19:17:38.381230 2564 secret.go:189] Couldn't get secret openshift-multus/metrics-daemon-secret: object "openshift-multus"/"metrics-daemon-secret" not registered Jul 29 19:17:38.381780 ip-10-0-131-93 kubenswrapper[2564]: E0729 19:17:38.381302 2564 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/ae220db3-299c-481a-9591-01a8f3873d9a-metrics-certs podName:ae220db3-299c-481a-9591-01a8f3873d9a nodeName:}" failed. No retries permitted until 2026-07-29 19:18:10.381283962 +0000 UTC m=+66.105343636 (durationBeforeRetry 32s). Error: MountVolume.SetUp failed for volume "metrics-certs" (UniqueName: "kubernetes.io/secret/ae220db3-299c-481a-9591-01a8f3873d9a-metrics-certs") pod "network-metrics-daemon-hkxvx" (UID: "ae220db3-299c-481a-9591-01a8f3873d9a") : object "openshift-multus"/"metrics-daemon-secret" not registered Jul 29 19:17:38.482459 ip-10-0-131-93 kubenswrapper[2564]: I0729 19:17:38.482427 2564 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-qgvzl\" (UniqueName: \"kubernetes.io/projected/02612aaa-a23a-4e6b-b224-123af1bb8230-kube-api-access-qgvzl\") pod \"network-check-target-464tm\" (UID: \"02612aaa-a23a-4e6b-b224-123af1bb8230\") " pod="openshift-network-diagnostics/network-check-target-464tm" Jul 29 19:17:38.482633 ip-10-0-131-93 kubenswrapper[2564]: I0729 19:17:38.482480 2564 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"original-pull-secret\" (UniqueName: \"kubernetes.io/secret/5c0eec03-fbbc-40db-9c78-43c8f98534c7-original-pull-secret\") pod \"global-pull-secret-syncer-zcgr9\" (UID: \"5c0eec03-fbbc-40db-9c78-43c8f98534c7\") " pod="kube-system/global-pull-secret-syncer-zcgr9" Jul 29 19:17:38.482633 ip-10-0-131-93 kubenswrapper[2564]: E0729 19:17:38.482587 2564 secret.go:189] Couldn't get secret kube-system/original-pull-secret: object "kube-system"/"original-pull-secret" not registered Jul 29 19:17:38.482633 ip-10-0-131-93 kubenswrapper[2564]: E0729 19:17:38.482603 2564 projected.go:291] Couldn't get configMap openshift-network-diagnostics/kube-root-ca.crt: object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered Jul 29 19:17:38.482633 ip-10-0-131-93 kubenswrapper[2564]: E0729 19:17:38.482626 2564 projected.go:291] Couldn't get configMap openshift-network-diagnostics/openshift-service-ca.crt: object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered Jul 29 19:17:38.482633 ip-10-0-131-93 kubenswrapper[2564]: E0729 19:17:38.482637 2564 projected.go:196] Error preparing data for projected volume kube-api-access-qgvzl for pod openshift-network-diagnostics/network-check-target-464tm: [object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered, object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered] Jul 29 19:17:38.482831 ip-10-0-131-93 kubenswrapper[2564]: E0729 19:17:38.482654 2564 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/5c0eec03-fbbc-40db-9c78-43c8f98534c7-original-pull-secret podName:5c0eec03-fbbc-40db-9c78-43c8f98534c7 nodeName:}" failed. No retries permitted until 2026-07-29 19:18:10.482634208 +0000 UTC m=+66.206693893 (durationBeforeRetry 32s). Error: MountVolume.SetUp failed for volume "original-pull-secret" (UniqueName: "kubernetes.io/secret/5c0eec03-fbbc-40db-9c78-43c8f98534c7-original-pull-secret") pod "global-pull-secret-syncer-zcgr9" (UID: "5c0eec03-fbbc-40db-9c78-43c8f98534c7") : object "kube-system"/"original-pull-secret" not registered Jul 29 19:17:38.482831 ip-10-0-131-93 kubenswrapper[2564]: E0729 19:17:38.482674 2564 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/projected/02612aaa-a23a-4e6b-b224-123af1bb8230-kube-api-access-qgvzl podName:02612aaa-a23a-4e6b-b224-123af1bb8230 nodeName:}" failed. No retries permitted until 2026-07-29 19:18:10.482662349 +0000 UTC m=+66.206722020 (durationBeforeRetry 32s). Error: MountVolume.SetUp failed for volume "kube-api-access-qgvzl" (UniqueName: "kubernetes.io/projected/02612aaa-a23a-4e6b-b224-123af1bb8230-kube-api-access-qgvzl") pod "network-check-target-464tm" (UID: "02612aaa-a23a-4e6b-b224-123af1bb8230") : [object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered, object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered] Jul 29 19:17:38.579403 ip-10-0-131-93 kubenswrapper[2564]: I0729 19:17:38.579314 2564 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-131-93.ec2.internal" event="NodeReady" Jul 29 19:17:38.579564 ip-10-0-131-93 kubenswrapper[2564]: I0729 19:17:38.579509 2564 kubelet_node_status.go:489] "Fast updating node status as it just became ready" Jul 29 19:17:38.617713 ip-10-0-131-93 kubenswrapper[2564]: I0729 19:17:38.617686 2564 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-image-registry/image-registry-7f95c97455-tkxb8"] Jul 29 19:17:38.620452 ip-10-0-131-93 kubenswrapper[2564]: I0729 19:17:38.620434 2564 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-image-registry/image-registry-7f95c97455-tkxb8" Jul 29 19:17:38.623157 ip-10-0-131-93 kubenswrapper[2564]: I0729 19:17:38.623134 2564 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-image-registry\"/\"installation-pull-secrets\"" Jul 29 19:17:38.623451 ip-10-0-131-93 kubenswrapper[2564]: I0729 19:17:38.623426 2564 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-image-registry\"/\"image-registry-private-configuration\"" Jul 29 19:17:38.624054 ip-10-0-131-93 kubenswrapper[2564]: I0729 19:17:38.624034 2564 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-image-registry\"/\"registry-dockercfg-k7njp\"" Jul 29 19:17:38.628254 ip-10-0-131-93 kubenswrapper[2564]: I0729 19:17:38.628233 2564 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-image-registry\"/\"image-registry-tls\"" Jul 29 19:17:38.638934 ip-10-0-131-93 kubenswrapper[2564]: I0729 19:17:38.638911 2564 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-image-registry\"/\"trusted-ca\"" Jul 29 19:17:38.639963 ip-10-0-131-93 kubenswrapper[2564]: I0729 19:17:38.639942 2564 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-ingress-canary/ingress-canary-jbktw"] Jul 29 19:17:38.641994 ip-10-0-131-93 kubenswrapper[2564]: I0729 19:17:38.641974 2564 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress-canary/ingress-canary-jbktw" Jul 29 19:17:38.645561 ip-10-0-131-93 kubenswrapper[2564]: I0729 19:17:38.645543 2564 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress-canary\"/\"canary-serving-cert\"" Jul 29 19:17:38.645658 ip-10-0-131-93 kubenswrapper[2564]: I0729 19:17:38.645622 2564 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ingress-canary\"/\"kube-root-ca.crt\"" Jul 29 19:17:38.645717 ip-10-0-131-93 kubenswrapper[2564]: I0729 19:17:38.645543 2564 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ingress-canary\"/\"openshift-service-ca.crt\"" Jul 29 19:17:38.646089 ip-10-0-131-93 kubenswrapper[2564]: I0729 19:17:38.646060 2564 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress-canary\"/\"default-dockercfg-2gz5l\"" Jul 29 19:17:38.659814 ip-10-0-131-93 kubenswrapper[2564]: I0729 19:17:38.659789 2564 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-image-registry/image-registry-7f95c97455-tkxb8"] Jul 29 19:17:38.665157 ip-10-0-131-93 kubenswrapper[2564]: I0729 19:17:38.665132 2564 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress-canary/ingress-canary-jbktw"] Jul 29 19:17:38.676340 ip-10-0-131-93 kubenswrapper[2564]: I0729 19:17:38.676315 2564 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-dns/dns-default-kkmn5"] Jul 29 19:17:38.678844 ip-10-0-131-93 kubenswrapper[2564]: I0729 19:17:38.678826 2564 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-dns/dns-default-kkmn5" Jul 29 19:17:38.681573 ip-10-0-131-93 kubenswrapper[2564]: I0729 19:17:38.681555 2564 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-dns\"/\"dns-default\"" Jul 29 19:17:38.681732 ip-10-0-131-93 kubenswrapper[2564]: I0729 19:17:38.681646 2564 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-dns\"/\"dns-default-metrics-tls\"" Jul 29 19:17:38.681800 ip-10-0-131-93 kubenswrapper[2564]: I0729 19:17:38.681750 2564 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-dns\"/\"dns-dockercfg-74g8d\"" Jul 29 19:17:38.694292 ip-10-0-131-93 kubenswrapper[2564]: I0729 19:17:38.694248 2564 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-dns/dns-default-kkmn5"] Jul 29 19:17:38.777454 ip-10-0-131-93 kubenswrapper[2564]: I0729 19:17:38.777424 2564 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-464tm" Jul 29 19:17:38.777619 ip-10-0-131-93 kubenswrapper[2564]: I0729 19:17:38.777424 2564 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-hkxvx" Jul 29 19:17:38.777619 ip-10-0-131-93 kubenswrapper[2564]: I0729 19:17:38.777435 2564 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-zcgr9" Jul 29 19:17:38.780002 ip-10-0-131-93 kubenswrapper[2564]: I0729 19:17:38.779879 2564 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-network-diagnostics\"/\"openshift-service-ca.crt\"" Jul 29 19:17:38.780002 ip-10-0-131-93 kubenswrapper[2564]: I0729 19:17:38.779936 2564 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-multus\"/\"metrics-daemon-secret\"" Jul 29 19:17:38.780002 ip-10-0-131-93 kubenswrapper[2564]: I0729 19:17:38.779952 2564 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-network-diagnostics\"/\"default-dockercfg-bfbzc\"" Jul 29 19:17:38.780556 ip-10-0-131-93 kubenswrapper[2564]: I0729 19:17:38.780524 2564 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-multus\"/\"metrics-daemon-sa-dockercfg-tgbwc\"" Jul 29 19:17:38.780670 ip-10-0-131-93 kubenswrapper[2564]: I0729 19:17:38.780586 2564 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"kube-system\"/\"original-pull-secret\"" Jul 29 19:17:38.780670 ip-10-0-131-93 kubenswrapper[2564]: I0729 19:17:38.780586 2564 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-network-diagnostics\"/\"kube-root-ca.crt\"" Jul 29 19:17:38.784156 ip-10-0-131-93 kubenswrapper[2564]: I0729 19:17:38.784133 2564 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"trusted-ca\" (UniqueName: \"kubernetes.io/configmap/5ff17af5-17c3-45e6-b47f-408ba15ed5ca-trusted-ca\") pod \"image-registry-7f95c97455-tkxb8\" (UID: \"5ff17af5-17c3-45e6-b47f-408ba15ed5ca\") " pod="openshift-image-registry/image-registry-7f95c97455-tkxb8" Jul 29 19:17:38.784264 ip-10-0-131-93 kubenswrapper[2564]: I0729 19:17:38.784171 2564 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"registry-certificates\" (UniqueName: \"kubernetes.io/configmap/5ff17af5-17c3-45e6-b47f-408ba15ed5ca-registry-certificates\") pod \"image-registry-7f95c97455-tkxb8\" (UID: \"5ff17af5-17c3-45e6-b47f-408ba15ed5ca\") " pod="openshift-image-registry/image-registry-7f95c97455-tkxb8" Jul 29 19:17:38.784264 ip-10-0-131-93 kubenswrapper[2564]: I0729 19:17:38.784200 2564 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-zwdkc\" (UniqueName: \"kubernetes.io/projected/d5e5a7da-26f5-4d8d-8e9e-4e8991392e8e-kube-api-access-zwdkc\") pod \"dns-default-kkmn5\" (UID: \"d5e5a7da-26f5-4d8d-8e9e-4e8991392e8e\") " pod="openshift-dns/dns-default-kkmn5" Jul 29 19:17:38.784264 ip-10-0-131-93 kubenswrapper[2564]: I0729 19:17:38.784254 2564 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"installation-pull-secrets\" (UniqueName: \"kubernetes.io/secret/5ff17af5-17c3-45e6-b47f-408ba15ed5ca-installation-pull-secrets\") pod \"image-registry-7f95c97455-tkxb8\" (UID: \"5ff17af5-17c3-45e6-b47f-408ba15ed5ca\") " pod="openshift-image-registry/image-registry-7f95c97455-tkxb8" Jul 29 19:17:38.784442 ip-10-0-131-93 kubenswrapper[2564]: I0729 19:17:38.784280 2564 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"metrics-tls\" (UniqueName: \"kubernetes.io/secret/d5e5a7da-26f5-4d8d-8e9e-4e8991392e8e-metrics-tls\") pod \"dns-default-kkmn5\" (UID: \"d5e5a7da-26f5-4d8d-8e9e-4e8991392e8e\") " pod="openshift-dns/dns-default-kkmn5" Jul 29 19:17:38.784442 ip-10-0-131-93 kubenswrapper[2564]: I0729 19:17:38.784337 2564 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/d5e5a7da-26f5-4d8d-8e9e-4e8991392e8e-tmp-dir\") pod \"dns-default-kkmn5\" (UID: \"d5e5a7da-26f5-4d8d-8e9e-4e8991392e8e\") " pod="openshift-dns/dns-default-kkmn5" Jul 29 19:17:38.784442 ip-10-0-131-93 kubenswrapper[2564]: I0729 19:17:38.784410 2564 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"bound-sa-token\" (UniqueName: \"kubernetes.io/projected/5ff17af5-17c3-45e6-b47f-408ba15ed5ca-bound-sa-token\") pod \"image-registry-7f95c97455-tkxb8\" (UID: \"5ff17af5-17c3-45e6-b47f-408ba15ed5ca\") " pod="openshift-image-registry/image-registry-7f95c97455-tkxb8" Jul 29 19:17:38.784442 ip-10-0-131-93 kubenswrapper[2564]: I0729 19:17:38.784432 2564 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"cert\" (UniqueName: \"kubernetes.io/secret/3832137d-c339-44b6-8ddc-af8c959c659e-cert\") pod \"ingress-canary-jbktw\" (UID: \"3832137d-c339-44b6-8ddc-af8c959c659e\") " pod="openshift-ingress-canary/ingress-canary-jbktw" Jul 29 19:17:38.784605 ip-10-0-131-93 kubenswrapper[2564]: I0729 19:17:38.784478 2564 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-2z8d6\" (UniqueName: \"kubernetes.io/projected/3832137d-c339-44b6-8ddc-af8c959c659e-kube-api-access-2z8d6\") pod \"ingress-canary-jbktw\" (UID: \"3832137d-c339-44b6-8ddc-af8c959c659e\") " pod="openshift-ingress-canary/ingress-canary-jbktw" Jul 29 19:17:38.784605 ip-10-0-131-93 kubenswrapper[2564]: I0729 19:17:38.784527 2564 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"ca-trust-extracted\" (UniqueName: \"kubernetes.io/empty-dir/5ff17af5-17c3-45e6-b47f-408ba15ed5ca-ca-trust-extracted\") pod \"image-registry-7f95c97455-tkxb8\" (UID: \"5ff17af5-17c3-45e6-b47f-408ba15ed5ca\") " pod="openshift-image-registry/image-registry-7f95c97455-tkxb8" Jul 29 19:17:38.784605 ip-10-0-131-93 kubenswrapper[2564]: I0729 19:17:38.784576 2564 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"image-registry-private-configuration\" (UniqueName: \"kubernetes.io/secret/5ff17af5-17c3-45e6-b47f-408ba15ed5ca-image-registry-private-configuration\") pod \"image-registry-7f95c97455-tkxb8\" (UID: \"5ff17af5-17c3-45e6-b47f-408ba15ed5ca\") " pod="openshift-image-registry/image-registry-7f95c97455-tkxb8" Jul 29 19:17:38.784605 ip-10-0-131-93 kubenswrapper[2564]: I0729 19:17:38.784595 2564 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"registry-tls\" (UniqueName: \"kubernetes.io/projected/5ff17af5-17c3-45e6-b47f-408ba15ed5ca-registry-tls\") pod \"image-registry-7f95c97455-tkxb8\" (UID: \"5ff17af5-17c3-45e6-b47f-408ba15ed5ca\") " pod="openshift-image-registry/image-registry-7f95c97455-tkxb8" Jul 29 19:17:38.784793 ip-10-0-131-93 kubenswrapper[2564]: I0729 19:17:38.784614 2564 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-wcwpv\" (UniqueName: \"kubernetes.io/projected/5ff17af5-17c3-45e6-b47f-408ba15ed5ca-kube-api-access-wcwpv\") pod \"image-registry-7f95c97455-tkxb8\" (UID: \"5ff17af5-17c3-45e6-b47f-408ba15ed5ca\") " pod="openshift-image-registry/image-registry-7f95c97455-tkxb8" Jul 29 19:17:38.784793 ip-10-0-131-93 kubenswrapper[2564]: I0729 19:17:38.784633 2564 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"config-volume\" (UniqueName: \"kubernetes.io/configmap/d5e5a7da-26f5-4d8d-8e9e-4e8991392e8e-config-volume\") pod \"dns-default-kkmn5\" (UID: \"d5e5a7da-26f5-4d8d-8e9e-4e8991392e8e\") " pod="openshift-dns/dns-default-kkmn5" Jul 29 19:17:38.885041 ip-10-0-131-93 kubenswrapper[2564]: I0729 19:17:38.884955 2564 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"cert\" (UniqueName: \"kubernetes.io/secret/3832137d-c339-44b6-8ddc-af8c959c659e-cert\") pod \"ingress-canary-jbktw\" (UID: \"3832137d-c339-44b6-8ddc-af8c959c659e\") " pod="openshift-ingress-canary/ingress-canary-jbktw" Jul 29 19:17:38.885041 ip-10-0-131-93 kubenswrapper[2564]: I0729 19:17:38.885001 2564 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-2z8d6\" (UniqueName: \"kubernetes.io/projected/3832137d-c339-44b6-8ddc-af8c959c659e-kube-api-access-2z8d6\") pod \"ingress-canary-jbktw\" (UID: \"3832137d-c339-44b6-8ddc-af8c959c659e\") " pod="openshift-ingress-canary/ingress-canary-jbktw" Jul 29 19:17:38.885041 ip-10-0-131-93 kubenswrapper[2564]: I0729 19:17:38.885028 2564 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"ca-trust-extracted\" (UniqueName: \"kubernetes.io/empty-dir/5ff17af5-17c3-45e6-b47f-408ba15ed5ca-ca-trust-extracted\") pod \"image-registry-7f95c97455-tkxb8\" (UID: \"5ff17af5-17c3-45e6-b47f-408ba15ed5ca\") " pod="openshift-image-registry/image-registry-7f95c97455-tkxb8" Jul 29 19:17:38.885314 ip-10-0-131-93 kubenswrapper[2564]: I0729 19:17:38.885059 2564 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"image-registry-private-configuration\" (UniqueName: \"kubernetes.io/secret/5ff17af5-17c3-45e6-b47f-408ba15ed5ca-image-registry-private-configuration\") pod \"image-registry-7f95c97455-tkxb8\" (UID: \"5ff17af5-17c3-45e6-b47f-408ba15ed5ca\") " pod="openshift-image-registry/image-registry-7f95c97455-tkxb8" Jul 29 19:17:38.885314 ip-10-0-131-93 kubenswrapper[2564]: I0729 19:17:38.885085 2564 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"registry-tls\" (UniqueName: \"kubernetes.io/projected/5ff17af5-17c3-45e6-b47f-408ba15ed5ca-registry-tls\") pod \"image-registry-7f95c97455-tkxb8\" (UID: \"5ff17af5-17c3-45e6-b47f-408ba15ed5ca\") " pod="openshift-image-registry/image-registry-7f95c97455-tkxb8" Jul 29 19:17:38.885314 ip-10-0-131-93 kubenswrapper[2564]: I0729 19:17:38.885110 2564 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-wcwpv\" (UniqueName: \"kubernetes.io/projected/5ff17af5-17c3-45e6-b47f-408ba15ed5ca-kube-api-access-wcwpv\") pod \"image-registry-7f95c97455-tkxb8\" (UID: \"5ff17af5-17c3-45e6-b47f-408ba15ed5ca\") " pod="openshift-image-registry/image-registry-7f95c97455-tkxb8" Jul 29 19:17:38.885314 ip-10-0-131-93 kubenswrapper[2564]: I0729 19:17:38.885153 2564 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"config-volume\" (UniqueName: \"kubernetes.io/configmap/d5e5a7da-26f5-4d8d-8e9e-4e8991392e8e-config-volume\") pod \"dns-default-kkmn5\" (UID: \"d5e5a7da-26f5-4d8d-8e9e-4e8991392e8e\") " pod="openshift-dns/dns-default-kkmn5" Jul 29 19:17:38.885314 ip-10-0-131-93 kubenswrapper[2564]: E0729 19:17:38.885215 2564 projected.go:266] Couldn't get secret openshift-image-registry/image-registry-tls: secret "image-registry-tls" not found Jul 29 19:17:38.885314 ip-10-0-131-93 kubenswrapper[2564]: E0729 19:17:38.885236 2564 projected.go:196] Error preparing data for projected volume registry-tls for pod openshift-image-registry/image-registry-7f95c97455-tkxb8: secret "image-registry-tls" not found Jul 29 19:17:38.885314 ip-10-0-131-93 kubenswrapper[2564]: I0729 19:17:38.885219 2564 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"trusted-ca\" (UniqueName: \"kubernetes.io/configmap/5ff17af5-17c3-45e6-b47f-408ba15ed5ca-trusted-ca\") pod \"image-registry-7f95c97455-tkxb8\" (UID: \"5ff17af5-17c3-45e6-b47f-408ba15ed5ca\") " pod="openshift-image-registry/image-registry-7f95c97455-tkxb8" Jul 29 19:17:38.885314 ip-10-0-131-93 kubenswrapper[2564]: E0729 19:17:38.885298 2564 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/projected/5ff17af5-17c3-45e6-b47f-408ba15ed5ca-registry-tls podName:5ff17af5-17c3-45e6-b47f-408ba15ed5ca nodeName:}" failed. No retries permitted until 2026-07-29 19:17:39.385276658 +0000 UTC m=+35.109336339 (durationBeforeRetry 500ms). Error: MountVolume.SetUp failed for volume "registry-tls" (UniqueName: "kubernetes.io/projected/5ff17af5-17c3-45e6-b47f-408ba15ed5ca-registry-tls") pod "image-registry-7f95c97455-tkxb8" (UID: "5ff17af5-17c3-45e6-b47f-408ba15ed5ca") : secret "image-registry-tls" not found Jul 29 19:17:38.885692 ip-10-0-131-93 kubenswrapper[2564]: I0729 19:17:38.885348 2564 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"registry-certificates\" (UniqueName: \"kubernetes.io/configmap/5ff17af5-17c3-45e6-b47f-408ba15ed5ca-registry-certificates\") pod \"image-registry-7f95c97455-tkxb8\" (UID: \"5ff17af5-17c3-45e6-b47f-408ba15ed5ca\") " pod="openshift-image-registry/image-registry-7f95c97455-tkxb8" Jul 29 19:17:38.885692 ip-10-0-131-93 kubenswrapper[2564]: I0729 19:17:38.885399 2564 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-zwdkc\" (UniqueName: \"kubernetes.io/projected/d5e5a7da-26f5-4d8d-8e9e-4e8991392e8e-kube-api-access-zwdkc\") pod \"dns-default-kkmn5\" (UID: \"d5e5a7da-26f5-4d8d-8e9e-4e8991392e8e\") " pod="openshift-dns/dns-default-kkmn5" Jul 29 19:17:38.885920 ip-10-0-131-93 kubenswrapper[2564]: I0729 19:17:38.885897 2564 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"installation-pull-secrets\" (UniqueName: \"kubernetes.io/secret/5ff17af5-17c3-45e6-b47f-408ba15ed5ca-installation-pull-secrets\") pod \"image-registry-7f95c97455-tkxb8\" (UID: \"5ff17af5-17c3-45e6-b47f-408ba15ed5ca\") " pod="openshift-image-registry/image-registry-7f95c97455-tkxb8" Jul 29 19:17:38.885962 ip-10-0-131-93 kubenswrapper[2564]: I0729 19:17:38.885952 2564 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-tls\" (UniqueName: \"kubernetes.io/secret/d5e5a7da-26f5-4d8d-8e9e-4e8991392e8e-metrics-tls\") pod \"dns-default-kkmn5\" (UID: \"d5e5a7da-26f5-4d8d-8e9e-4e8991392e8e\") " pod="openshift-dns/dns-default-kkmn5" Jul 29 19:17:38.886012 ip-10-0-131-93 kubenswrapper[2564]: I0729 19:17:38.885993 2564 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"ca-trust-extracted\" (UniqueName: \"kubernetes.io/empty-dir/5ff17af5-17c3-45e6-b47f-408ba15ed5ca-ca-trust-extracted\") pod \"image-registry-7f95c97455-tkxb8\" (UID: \"5ff17af5-17c3-45e6-b47f-408ba15ed5ca\") " pod="openshift-image-registry/image-registry-7f95c97455-tkxb8" Jul 29 19:17:38.886078 ip-10-0-131-93 kubenswrapper[2564]: I0729 19:17:38.886064 2564 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/d5e5a7da-26f5-4d8d-8e9e-4e8991392e8e-tmp-dir\") pod \"dns-default-kkmn5\" (UID: \"d5e5a7da-26f5-4d8d-8e9e-4e8991392e8e\") " pod="openshift-dns/dns-default-kkmn5" Jul 29 19:17:38.886167 ip-10-0-131-93 kubenswrapper[2564]: I0729 19:17:38.886150 2564 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"bound-sa-token\" (UniqueName: \"kubernetes.io/projected/5ff17af5-17c3-45e6-b47f-408ba15ed5ca-bound-sa-token\") pod \"image-registry-7f95c97455-tkxb8\" (UID: \"5ff17af5-17c3-45e6-b47f-408ba15ed5ca\") " pod="openshift-image-registry/image-registry-7f95c97455-tkxb8" Jul 29 19:17:38.887891 ip-10-0-131-93 kubenswrapper[2564]: I0729 19:17:38.886980 2564 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"config-volume\" (UniqueName: \"kubernetes.io/configmap/d5e5a7da-26f5-4d8d-8e9e-4e8991392e8e-config-volume\") pod \"dns-default-kkmn5\" (UID: \"d5e5a7da-26f5-4d8d-8e9e-4e8991392e8e\") " pod="openshift-dns/dns-default-kkmn5" Jul 29 19:17:38.887891 ip-10-0-131-93 kubenswrapper[2564]: I0729 19:17:38.887290 2564 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/d5e5a7da-26f5-4d8d-8e9e-4e8991392e8e-tmp-dir\") pod \"dns-default-kkmn5\" (UID: \"d5e5a7da-26f5-4d8d-8e9e-4e8991392e8e\") " pod="openshift-dns/dns-default-kkmn5" Jul 29 19:17:38.887891 ip-10-0-131-93 kubenswrapper[2564]: E0729 19:17:38.887432 2564 secret.go:189] Couldn't get secret openshift-dns/dns-default-metrics-tls: secret "dns-default-metrics-tls" not found Jul 29 19:17:38.887891 ip-10-0-131-93 kubenswrapper[2564]: E0729 19:17:38.887858 2564 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/d5e5a7da-26f5-4d8d-8e9e-4e8991392e8e-metrics-tls podName:d5e5a7da-26f5-4d8d-8e9e-4e8991392e8e nodeName:}" failed. No retries permitted until 2026-07-29 19:17:39.387828987 +0000 UTC m=+35.111888660 (durationBeforeRetry 500ms). Error: MountVolume.SetUp failed for volume "metrics-tls" (UniqueName: "kubernetes.io/secret/d5e5a7da-26f5-4d8d-8e9e-4e8991392e8e-metrics-tls") pod "dns-default-kkmn5" (UID: "d5e5a7da-26f5-4d8d-8e9e-4e8991392e8e") : secret "dns-default-metrics-tls" not found Jul 29 19:17:38.887891 ip-10-0-131-93 kubenswrapper[2564]: I0729 19:17:38.887868 2564 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"trusted-ca\" (UniqueName: \"kubernetes.io/configmap/5ff17af5-17c3-45e6-b47f-408ba15ed5ca-trusted-ca\") pod \"image-registry-7f95c97455-tkxb8\" (UID: \"5ff17af5-17c3-45e6-b47f-408ba15ed5ca\") " pod="openshift-image-registry/image-registry-7f95c97455-tkxb8" Jul 29 19:17:38.888191 ip-10-0-131-93 kubenswrapper[2564]: I0729 19:17:38.888057 2564 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"registry-certificates\" (UniqueName: \"kubernetes.io/configmap/5ff17af5-17c3-45e6-b47f-408ba15ed5ca-registry-certificates\") pod \"image-registry-7f95c97455-tkxb8\" (UID: \"5ff17af5-17c3-45e6-b47f-408ba15ed5ca\") " pod="openshift-image-registry/image-registry-7f95c97455-tkxb8" Jul 29 19:17:38.891463 ip-10-0-131-93 kubenswrapper[2564]: I0729 19:17:38.891440 2564 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"image-registry-private-configuration\" (UniqueName: \"kubernetes.io/secret/5ff17af5-17c3-45e6-b47f-408ba15ed5ca-image-registry-private-configuration\") pod \"image-registry-7f95c97455-tkxb8\" (UID: \"5ff17af5-17c3-45e6-b47f-408ba15ed5ca\") " pod="openshift-image-registry/image-registry-7f95c97455-tkxb8" Jul 29 19:17:38.891572 ip-10-0-131-93 kubenswrapper[2564]: I0729 19:17:38.891450 2564 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"cert\" (UniqueName: \"kubernetes.io/secret/3832137d-c339-44b6-8ddc-af8c959c659e-cert\") pod \"ingress-canary-jbktw\" (UID: \"3832137d-c339-44b6-8ddc-af8c959c659e\") " pod="openshift-ingress-canary/ingress-canary-jbktw" Jul 29 19:17:38.892025 ip-10-0-131-93 kubenswrapper[2564]: I0729 19:17:38.891998 2564 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"installation-pull-secrets\" (UniqueName: \"kubernetes.io/secret/5ff17af5-17c3-45e6-b47f-408ba15ed5ca-installation-pull-secrets\") pod \"image-registry-7f95c97455-tkxb8\" (UID: \"5ff17af5-17c3-45e6-b47f-408ba15ed5ca\") " pod="openshift-image-registry/image-registry-7f95c97455-tkxb8" Jul 29 19:17:38.894865 ip-10-0-131-93 kubenswrapper[2564]: I0729 19:17:38.894843 2564 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-zwdkc\" (UniqueName: \"kubernetes.io/projected/d5e5a7da-26f5-4d8d-8e9e-4e8991392e8e-kube-api-access-zwdkc\") pod \"dns-default-kkmn5\" (UID: \"d5e5a7da-26f5-4d8d-8e9e-4e8991392e8e\") " pod="openshift-dns/dns-default-kkmn5" Jul 29 19:17:38.894960 ip-10-0-131-93 kubenswrapper[2564]: I0729 19:17:38.894944 2564 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-2z8d6\" (UniqueName: \"kubernetes.io/projected/3832137d-c339-44b6-8ddc-af8c959c659e-kube-api-access-2z8d6\") pod \"ingress-canary-jbktw\" (UID: \"3832137d-c339-44b6-8ddc-af8c959c659e\") " pod="openshift-ingress-canary/ingress-canary-jbktw" Jul 29 19:17:38.895023 ip-10-0-131-93 kubenswrapper[2564]: I0729 19:17:38.895002 2564 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"bound-sa-token\" (UniqueName: \"kubernetes.io/projected/5ff17af5-17c3-45e6-b47f-408ba15ed5ca-bound-sa-token\") pod \"image-registry-7f95c97455-tkxb8\" (UID: \"5ff17af5-17c3-45e6-b47f-408ba15ed5ca\") " pod="openshift-image-registry/image-registry-7f95c97455-tkxb8" Jul 29 19:17:38.895388 ip-10-0-131-93 kubenswrapper[2564]: I0729 19:17:38.895349 2564 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-wcwpv\" (UniqueName: \"kubernetes.io/projected/5ff17af5-17c3-45e6-b47f-408ba15ed5ca-kube-api-access-wcwpv\") pod \"image-registry-7f95c97455-tkxb8\" (UID: \"5ff17af5-17c3-45e6-b47f-408ba15ed5ca\") " pod="openshift-image-registry/image-registry-7f95c97455-tkxb8" Jul 29 19:17:38.951734 ip-10-0-131-93 kubenswrapper[2564]: I0729 19:17:38.951700 2564 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress-canary/ingress-canary-jbktw" Jul 29 19:17:39.391681 ip-10-0-131-93 kubenswrapper[2564]: I0729 19:17:39.391644 2564 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"registry-tls\" (UniqueName: \"kubernetes.io/projected/5ff17af5-17c3-45e6-b47f-408ba15ed5ca-registry-tls\") pod \"image-registry-7f95c97455-tkxb8\" (UID: \"5ff17af5-17c3-45e6-b47f-408ba15ed5ca\") " pod="openshift-image-registry/image-registry-7f95c97455-tkxb8" Jul 29 19:17:39.392120 ip-10-0-131-93 kubenswrapper[2564]: I0729 19:17:39.391702 2564 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-tls\" (UniqueName: \"kubernetes.io/secret/d5e5a7da-26f5-4d8d-8e9e-4e8991392e8e-metrics-tls\") pod \"dns-default-kkmn5\" (UID: \"d5e5a7da-26f5-4d8d-8e9e-4e8991392e8e\") " pod="openshift-dns/dns-default-kkmn5" Jul 29 19:17:39.392120 ip-10-0-131-93 kubenswrapper[2564]: E0729 19:17:39.391803 2564 projected.go:266] Couldn't get secret openshift-image-registry/image-registry-tls: secret "image-registry-tls" not found Jul 29 19:17:39.392120 ip-10-0-131-93 kubenswrapper[2564]: E0729 19:17:39.391813 2564 secret.go:189] Couldn't get secret openshift-dns/dns-default-metrics-tls: secret "dns-default-metrics-tls" not found Jul 29 19:17:39.392120 ip-10-0-131-93 kubenswrapper[2564]: E0729 19:17:39.391822 2564 projected.go:196] Error preparing data for projected volume registry-tls for pod openshift-image-registry/image-registry-7f95c97455-tkxb8: secret "image-registry-tls" not found Jul 29 19:17:39.392120 ip-10-0-131-93 kubenswrapper[2564]: E0729 19:17:39.391875 2564 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/d5e5a7da-26f5-4d8d-8e9e-4e8991392e8e-metrics-tls podName:d5e5a7da-26f5-4d8d-8e9e-4e8991392e8e nodeName:}" failed. No retries permitted until 2026-07-29 19:17:40.391857454 +0000 UTC m=+36.115917125 (durationBeforeRetry 1s). Error: MountVolume.SetUp failed for volume "metrics-tls" (UniqueName: "kubernetes.io/secret/d5e5a7da-26f5-4d8d-8e9e-4e8991392e8e-metrics-tls") pod "dns-default-kkmn5" (UID: "d5e5a7da-26f5-4d8d-8e9e-4e8991392e8e") : secret "dns-default-metrics-tls" not found Jul 29 19:17:39.392120 ip-10-0-131-93 kubenswrapper[2564]: E0729 19:17:39.391894 2564 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/projected/5ff17af5-17c3-45e6-b47f-408ba15ed5ca-registry-tls podName:5ff17af5-17c3-45e6-b47f-408ba15ed5ca nodeName:}" failed. No retries permitted until 2026-07-29 19:17:40.391884083 +0000 UTC m=+36.115943757 (durationBeforeRetry 1s). Error: MountVolume.SetUp failed for volume "registry-tls" (UniqueName: "kubernetes.io/projected/5ff17af5-17c3-45e6-b47f-408ba15ed5ca-registry-tls") pod "image-registry-7f95c97455-tkxb8" (UID: "5ff17af5-17c3-45e6-b47f-408ba15ed5ca") : secret "image-registry-tls" not found Jul 29 19:17:39.951852 ip-10-0-131-93 kubenswrapper[2564]: I0729 19:17:39.951612 2564 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress-canary/ingress-canary-jbktw"] Jul 29 19:17:39.979197 ip-10-0-131-93 kubenswrapper[2564]: W0729 19:17:39.979166 2564 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod3832137d_c339_44b6_8ddc_af8c959c659e.slice/crio-b80771caad9a7e2271b43a9b758c1ecbea9c9dd8723cfb6ee97b5c477fca0375 WatchSource:0}: Error finding container b80771caad9a7e2271b43a9b758c1ecbea9c9dd8723cfb6ee97b5c477fca0375: Status 404 returned error can't find the container with id b80771caad9a7e2271b43a9b758c1ecbea9c9dd8723cfb6ee97b5c477fca0375 Jul 29 19:17:40.398084 ip-10-0-131-93 kubenswrapper[2564]: I0729 19:17:40.397999 2564 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"registry-tls\" (UniqueName: \"kubernetes.io/projected/5ff17af5-17c3-45e6-b47f-408ba15ed5ca-registry-tls\") pod \"image-registry-7f95c97455-tkxb8\" (UID: \"5ff17af5-17c3-45e6-b47f-408ba15ed5ca\") " pod="openshift-image-registry/image-registry-7f95c97455-tkxb8" Jul 29 19:17:40.398084 ip-10-0-131-93 kubenswrapper[2564]: I0729 19:17:40.398048 2564 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-tls\" (UniqueName: \"kubernetes.io/secret/d5e5a7da-26f5-4d8d-8e9e-4e8991392e8e-metrics-tls\") pod \"dns-default-kkmn5\" (UID: \"d5e5a7da-26f5-4d8d-8e9e-4e8991392e8e\") " pod="openshift-dns/dns-default-kkmn5" Jul 29 19:17:40.398525 ip-10-0-131-93 kubenswrapper[2564]: E0729 19:17:40.398151 2564 projected.go:266] Couldn't get secret openshift-image-registry/image-registry-tls: secret "image-registry-tls" not found Jul 29 19:17:40.398525 ip-10-0-131-93 kubenswrapper[2564]: E0729 19:17:40.398175 2564 projected.go:196] Error preparing data for projected volume registry-tls for pod openshift-image-registry/image-registry-7f95c97455-tkxb8: secret "image-registry-tls" not found Jul 29 19:17:40.398525 ip-10-0-131-93 kubenswrapper[2564]: E0729 19:17:40.398153 2564 secret.go:189] Couldn't get secret openshift-dns/dns-default-metrics-tls: secret "dns-default-metrics-tls" not found Jul 29 19:17:40.398525 ip-10-0-131-93 kubenswrapper[2564]: E0729 19:17:40.398228 2564 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/d5e5a7da-26f5-4d8d-8e9e-4e8991392e8e-metrics-tls podName:d5e5a7da-26f5-4d8d-8e9e-4e8991392e8e nodeName:}" failed. No retries permitted until 2026-07-29 19:17:42.398214781 +0000 UTC m=+38.122274451 (durationBeforeRetry 2s). Error: MountVolume.SetUp failed for volume "metrics-tls" (UniqueName: "kubernetes.io/secret/d5e5a7da-26f5-4d8d-8e9e-4e8991392e8e-metrics-tls") pod "dns-default-kkmn5" (UID: "d5e5a7da-26f5-4d8d-8e9e-4e8991392e8e") : secret "dns-default-metrics-tls" not found Jul 29 19:17:40.398525 ip-10-0-131-93 kubenswrapper[2564]: E0729 19:17:40.398243 2564 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/projected/5ff17af5-17c3-45e6-b47f-408ba15ed5ca-registry-tls podName:5ff17af5-17c3-45e6-b47f-408ba15ed5ca nodeName:}" failed. No retries permitted until 2026-07-29 19:17:42.398237808 +0000 UTC m=+38.122297478 (durationBeforeRetry 2s). Error: MountVolume.SetUp failed for volume "registry-tls" (UniqueName: "kubernetes.io/projected/5ff17af5-17c3-45e6-b47f-408ba15ed5ca-registry-tls") pod "image-registry-7f95c97455-tkxb8" (UID: "5ff17af5-17c3-45e6-b47f-408ba15ed5ca") : secret "image-registry-tls" not found Jul 29 19:17:40.936404 ip-10-0-131-93 kubenswrapper[2564]: I0729 19:17:40.936348 2564 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress-canary/ingress-canary-jbktw" event={"ID":"3832137d-c339-44b6-8ddc-af8c959c659e","Type":"ContainerStarted","Data":"b80771caad9a7e2271b43a9b758c1ecbea9c9dd8723cfb6ee97b5c477fca0375"} Jul 29 19:17:40.939540 ip-10-0-131-93 kubenswrapper[2564]: I0729 19:17:40.939511 2564 generic.go:356] "Generic (PLEG): container finished" podID="01073023-f0f3-4517-affc-d5c86021c64d" containerID="255f5f35ff4bf8e3bbb736e1c22b4174677179e38fe90ed721623e8035421ad8" exitCode=0 Jul 29 19:17:40.939686 ip-10-0-131-93 kubenswrapper[2564]: I0729 19:17:40.939553 2564 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-multus/multus-additional-cni-plugins-nmm7d" event={"ID":"01073023-f0f3-4517-affc-d5c86021c64d","Type":"ContainerDied","Data":"255f5f35ff4bf8e3bbb736e1c22b4174677179e38fe90ed721623e8035421ad8"} Jul 29 19:17:41.945781 ip-10-0-131-93 kubenswrapper[2564]: I0729 19:17:41.945745 2564 generic.go:356] "Generic (PLEG): container finished" podID="01073023-f0f3-4517-affc-d5c86021c64d" containerID="3fb2dab48ed8235fdd158088a4979b0dfbacde62278c0affe83ab938ab7387aa" exitCode=0 Jul 29 19:17:41.946203 ip-10-0-131-93 kubenswrapper[2564]: I0729 19:17:41.945828 2564 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-multus/multus-additional-cni-plugins-nmm7d" event={"ID":"01073023-f0f3-4517-affc-d5c86021c64d","Type":"ContainerDied","Data":"3fb2dab48ed8235fdd158088a4979b0dfbacde62278c0affe83ab938ab7387aa"} Jul 29 19:17:42.419054 ip-10-0-131-93 kubenswrapper[2564]: I0729 19:17:42.418973 2564 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"registry-tls\" (UniqueName: \"kubernetes.io/projected/5ff17af5-17c3-45e6-b47f-408ba15ed5ca-registry-tls\") pod \"image-registry-7f95c97455-tkxb8\" (UID: \"5ff17af5-17c3-45e6-b47f-408ba15ed5ca\") " pod="openshift-image-registry/image-registry-7f95c97455-tkxb8" Jul 29 19:17:42.419054 ip-10-0-131-93 kubenswrapper[2564]: I0729 19:17:42.419037 2564 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-tls\" (UniqueName: \"kubernetes.io/secret/d5e5a7da-26f5-4d8d-8e9e-4e8991392e8e-metrics-tls\") pod \"dns-default-kkmn5\" (UID: \"d5e5a7da-26f5-4d8d-8e9e-4e8991392e8e\") " pod="openshift-dns/dns-default-kkmn5" Jul 29 19:17:42.419253 ip-10-0-131-93 kubenswrapper[2564]: E0729 19:17:42.419123 2564 projected.go:266] Couldn't get secret openshift-image-registry/image-registry-tls: secret "image-registry-tls" not found Jul 29 19:17:42.419253 ip-10-0-131-93 kubenswrapper[2564]: E0729 19:17:42.419148 2564 projected.go:196] Error preparing data for projected volume registry-tls for pod openshift-image-registry/image-registry-7f95c97455-tkxb8: secret "image-registry-tls" not found Jul 29 19:17:42.419253 ip-10-0-131-93 kubenswrapper[2564]: E0729 19:17:42.419184 2564 secret.go:189] Couldn't get secret openshift-dns/dns-default-metrics-tls: secret "dns-default-metrics-tls" not found Jul 29 19:17:42.419253 ip-10-0-131-93 kubenswrapper[2564]: E0729 19:17:42.419216 2564 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/projected/5ff17af5-17c3-45e6-b47f-408ba15ed5ca-registry-tls podName:5ff17af5-17c3-45e6-b47f-408ba15ed5ca nodeName:}" failed. No retries permitted until 2026-07-29 19:17:46.419195888 +0000 UTC m=+42.143255580 (durationBeforeRetry 4s). Error: MountVolume.SetUp failed for volume "registry-tls" (UniqueName: "kubernetes.io/projected/5ff17af5-17c3-45e6-b47f-408ba15ed5ca-registry-tls") pod "image-registry-7f95c97455-tkxb8" (UID: "5ff17af5-17c3-45e6-b47f-408ba15ed5ca") : secret "image-registry-tls" not found Jul 29 19:17:42.419253 ip-10-0-131-93 kubenswrapper[2564]: E0729 19:17:42.419243 2564 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/d5e5a7da-26f5-4d8d-8e9e-4e8991392e8e-metrics-tls podName:d5e5a7da-26f5-4d8d-8e9e-4e8991392e8e nodeName:}" failed. No retries permitted until 2026-07-29 19:17:46.419225426 +0000 UTC m=+42.143285111 (durationBeforeRetry 4s). Error: MountVolume.SetUp failed for volume "metrics-tls" (UniqueName: "kubernetes.io/secret/d5e5a7da-26f5-4d8d-8e9e-4e8991392e8e-metrics-tls") pod "dns-default-kkmn5" (UID: "d5e5a7da-26f5-4d8d-8e9e-4e8991392e8e") : secret "dns-default-metrics-tls" not found Jul 29 19:17:42.950128 ip-10-0-131-93 kubenswrapper[2564]: I0729 19:17:42.949878 2564 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress-canary/ingress-canary-jbktw" event={"ID":"3832137d-c339-44b6-8ddc-af8c959c659e","Type":"ContainerStarted","Data":"6670f21416bd9bd1217d45976480115b1975ed2f5bacd0e11a07a7668e920f71"} Jul 29 19:17:42.953423 ip-10-0-131-93 kubenswrapper[2564]: I0729 19:17:42.953396 2564 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-multus/multus-additional-cni-plugins-nmm7d" event={"ID":"01073023-f0f3-4517-affc-d5c86021c64d","Type":"ContainerStarted","Data":"d52045535f587a00d9eb7fe4fcbb186153c395303f95d177732badf0e2c68d24"} Jul 29 19:17:42.967762 ip-10-0-131-93 kubenswrapper[2564]: I0729 19:17:42.966754 2564 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-ingress-canary/ingress-canary-jbktw" podStartSLOduration=2.205194772 podStartE2EDuration="4.966735465s" totalImagesPullingTime="2.761540693s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-29 19:17:38 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-29 19:17:42.965647447 +0000 UTC m=+38.689707140" watchObservedRunningTime="2026-07-29 19:17:42.966735465 +0000 UTC m=+38.690795157" Jul 29 19:17:42.990324 ip-10-0-131-93 kubenswrapper[2564]: I0729 19:17:42.990233 2564 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-multus/multus-additional-cni-plugins-nmm7d" podStartSLOduration=10.298916424 podStartE2EDuration="38.990216482s" totalImagesPullingTime="28.691300058s" totalInitContainerRuntime="0s" isStatefulPod=true podCreationTimestamp="2026-07-29 19:17:04 +0000 UTC" imagePullSessionsCount=5 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-29 19:17:42.988211336 +0000 UTC m=+38.712271029" watchObservedRunningTime="2026-07-29 19:17:42.990216482 +0000 UTC m=+38.714276174" Jul 29 19:17:43.150173 ip-10-0-131-93 kubenswrapper[2564]: I0729 19:17:43.150104 2564 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-444jx_faa8c546-c75b-4e10-93b4-661aa2b5f53e/konnectivity-agent/0.log" limit=1024 Jul 29 19:17:43.327952 ip-10-0-131-93 kubenswrapper[2564]: I0729 19:17:43.327924 2564 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-444jx_faa8c546-c75b-4e10-93b4-661aa2b5f53e/konnectivity-agent/0.log" limit=1024 Jul 29 19:17:44.010527 ip-10-0-131-93 kubenswrapper[2564]: I0729 19:17:44.010497 2564 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-444jx_faa8c546-c75b-4e10-93b4-661aa2b5f53e/konnectivity-agent/0.log" limit=1024 Jul 29 19:17:44.866629 ip-10-0-131-93 kubenswrapper[2564]: I0729 19:17:44.866591 2564 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-444jx_faa8c546-c75b-4e10-93b4-661aa2b5f53e/konnectivity-agent/0.log" limit=1024 Jul 29 19:17:44.961920 ip-10-0-131-93 kubenswrapper[2564]: I0729 19:17:44.961894 2564 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-444jx_faa8c546-c75b-4e10-93b4-661aa2b5f53e/konnectivity-agent/0.log" limit=1024 Jul 29 19:17:46.449761 ip-10-0-131-93 kubenswrapper[2564]: I0729 19:17:46.449718 2564 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-tls\" (UniqueName: \"kubernetes.io/secret/d5e5a7da-26f5-4d8d-8e9e-4e8991392e8e-metrics-tls\") pod \"dns-default-kkmn5\" (UID: \"d5e5a7da-26f5-4d8d-8e9e-4e8991392e8e\") " pod="openshift-dns/dns-default-kkmn5" Jul 29 19:17:46.450130 ip-10-0-131-93 kubenswrapper[2564]: I0729 19:17:46.449796 2564 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"registry-tls\" (UniqueName: \"kubernetes.io/projected/5ff17af5-17c3-45e6-b47f-408ba15ed5ca-registry-tls\") pod \"image-registry-7f95c97455-tkxb8\" (UID: \"5ff17af5-17c3-45e6-b47f-408ba15ed5ca\") " pod="openshift-image-registry/image-registry-7f95c97455-tkxb8" Jul 29 19:17:46.450130 ip-10-0-131-93 kubenswrapper[2564]: E0729 19:17:46.449918 2564 projected.go:266] Couldn't get secret openshift-image-registry/image-registry-tls: secret "image-registry-tls" not found Jul 29 19:17:46.450130 ip-10-0-131-93 kubenswrapper[2564]: E0729 19:17:46.449948 2564 projected.go:196] Error preparing data for projected volume registry-tls for pod openshift-image-registry/image-registry-7f95c97455-tkxb8: secret "image-registry-tls" not found Jul 29 19:17:46.450130 ip-10-0-131-93 kubenswrapper[2564]: E0729 19:17:46.450004 2564 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/projected/5ff17af5-17c3-45e6-b47f-408ba15ed5ca-registry-tls podName:5ff17af5-17c3-45e6-b47f-408ba15ed5ca nodeName:}" failed. No retries permitted until 2026-07-29 19:17:54.449988306 +0000 UTC m=+50.174047975 (durationBeforeRetry 8s). Error: MountVolume.SetUp failed for volume "registry-tls" (UniqueName: "kubernetes.io/projected/5ff17af5-17c3-45e6-b47f-408ba15ed5ca-registry-tls") pod "image-registry-7f95c97455-tkxb8" (UID: "5ff17af5-17c3-45e6-b47f-408ba15ed5ca") : secret "image-registry-tls" not found Jul 29 19:17:46.450130 ip-10-0-131-93 kubenswrapper[2564]: E0729 19:17:46.449915 2564 secret.go:189] Couldn't get secret openshift-dns/dns-default-metrics-tls: secret "dns-default-metrics-tls" not found Jul 29 19:17:46.450130 ip-10-0-131-93 kubenswrapper[2564]: E0729 19:17:46.450061 2564 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/d5e5a7da-26f5-4d8d-8e9e-4e8991392e8e-metrics-tls podName:d5e5a7da-26f5-4d8d-8e9e-4e8991392e8e nodeName:}" failed. No retries permitted until 2026-07-29 19:17:54.450049804 +0000 UTC m=+50.174109482 (durationBeforeRetry 8s). Error: MountVolume.SetUp failed for volume "metrics-tls" (UniqueName: "kubernetes.io/secret/d5e5a7da-26f5-4d8d-8e9e-4e8991392e8e-metrics-tls") pod "dns-default-kkmn5" (UID: "d5e5a7da-26f5-4d8d-8e9e-4e8991392e8e") : secret "dns-default-metrics-tls" not found Jul 29 19:17:48.032268 ip-10-0-131-93 kubenswrapper[2564]: I0729 19:17:48.032236 2564 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-444jx_faa8c546-c75b-4e10-93b4-661aa2b5f53e/konnectivity-agent/0.log" limit=1024 Jul 29 19:17:48.134152 ip-10-0-131-93 kubenswrapper[2564]: I0729 19:17:48.134114 2564 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-444jx_faa8c546-c75b-4e10-93b4-661aa2b5f53e/konnectivity-agent/0.log" limit=1024 Jul 29 19:17:48.230317 ip-10-0-131-93 kubenswrapper[2564]: I0729 19:17:48.230289 2564 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-444jx_faa8c546-c75b-4e10-93b4-661aa2b5f53e/konnectivity-agent/0.log" limit=1024 Jul 29 19:17:48.305413 ip-10-0-131-93 kubenswrapper[2564]: I0729 19:17:48.305334 2564 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-444jx_faa8c546-c75b-4e10-93b4-661aa2b5f53e/konnectivity-agent/0.log" limit=1024 Jul 29 19:17:48.903840 ip-10-0-131-93 kubenswrapper[2564]: I0729 19:17:48.903810 2564 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-444jx_faa8c546-c75b-4e10-93b4-661aa2b5f53e/konnectivity-agent/0.log" limit=1024 Jul 29 19:17:49.001759 ip-10-0-131-93 kubenswrapper[2564]: I0729 19:17:49.001726 2564 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-444jx_faa8c546-c75b-4e10-93b4-661aa2b5f53e/konnectivity-agent/0.log" limit=1024 Jul 29 19:17:49.547634 ip-10-0-131-93 kubenswrapper[2564]: I0729 19:17:49.547604 2564 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-444jx_faa8c546-c75b-4e10-93b4-661aa2b5f53e/konnectivity-agent/0.log" limit=1024 Jul 29 19:17:49.605953 ip-10-0-131-93 kubenswrapper[2564]: I0729 19:17:49.605922 2564 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-444jx_faa8c546-c75b-4e10-93b4-661aa2b5f53e/konnectivity-agent/0.log" limit=1024 Jul 29 19:17:54.508330 ip-10-0-131-93 kubenswrapper[2564]: I0729 19:17:54.508282 2564 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-tls\" (UniqueName: \"kubernetes.io/secret/d5e5a7da-26f5-4d8d-8e9e-4e8991392e8e-metrics-tls\") pod \"dns-default-kkmn5\" (UID: \"d5e5a7da-26f5-4d8d-8e9e-4e8991392e8e\") " pod="openshift-dns/dns-default-kkmn5" Jul 29 19:17:54.508812 ip-10-0-131-93 kubenswrapper[2564]: I0729 19:17:54.508368 2564 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"registry-tls\" (UniqueName: \"kubernetes.io/projected/5ff17af5-17c3-45e6-b47f-408ba15ed5ca-registry-tls\") pod \"image-registry-7f95c97455-tkxb8\" (UID: \"5ff17af5-17c3-45e6-b47f-408ba15ed5ca\") " pod="openshift-image-registry/image-registry-7f95c97455-tkxb8" Jul 29 19:17:54.508812 ip-10-0-131-93 kubenswrapper[2564]: E0729 19:17:54.508456 2564 secret.go:189] Couldn't get secret openshift-dns/dns-default-metrics-tls: secret "dns-default-metrics-tls" not found Jul 29 19:17:54.508812 ip-10-0-131-93 kubenswrapper[2564]: E0729 19:17:54.508507 2564 projected.go:266] Couldn't get secret openshift-image-registry/image-registry-tls: secret "image-registry-tls" not found Jul 29 19:17:54.508812 ip-10-0-131-93 kubenswrapper[2564]: E0729 19:17:54.508524 2564 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/d5e5a7da-26f5-4d8d-8e9e-4e8991392e8e-metrics-tls podName:d5e5a7da-26f5-4d8d-8e9e-4e8991392e8e nodeName:}" failed. No retries permitted until 2026-07-29 19:18:10.508506502 +0000 UTC m=+66.232566172 (durationBeforeRetry 16s). Error: MountVolume.SetUp failed for volume "metrics-tls" (UniqueName: "kubernetes.io/secret/d5e5a7da-26f5-4d8d-8e9e-4e8991392e8e-metrics-tls") pod "dns-default-kkmn5" (UID: "d5e5a7da-26f5-4d8d-8e9e-4e8991392e8e") : secret "dns-default-metrics-tls" not found Jul 29 19:17:54.508812 ip-10-0-131-93 kubenswrapper[2564]: E0729 19:17:54.508529 2564 projected.go:196] Error preparing data for projected volume registry-tls for pod openshift-image-registry/image-registry-7f95c97455-tkxb8: secret "image-registry-tls" not found Jul 29 19:17:54.508812 ip-10-0-131-93 kubenswrapper[2564]: E0729 19:17:54.508610 2564 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/projected/5ff17af5-17c3-45e6-b47f-408ba15ed5ca-registry-tls podName:5ff17af5-17c3-45e6-b47f-408ba15ed5ca nodeName:}" failed. No retries permitted until 2026-07-29 19:18:10.508591398 +0000 UTC m=+66.232651070 (durationBeforeRetry 16s). Error: MountVolume.SetUp failed for volume "registry-tls" (UniqueName: "kubernetes.io/projected/5ff17af5-17c3-45e6-b47f-408ba15ed5ca-registry-tls") pod "image-registry-7f95c97455-tkxb8" (UID: "5ff17af5-17c3-45e6-b47f-408ba15ed5ca") : secret "image-registry-tls" not found Jul 29 19:18:06.590258 ip-10-0-131-93 kubenswrapper[2564]: I0729 19:18:06.590229 2564 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ovn-kubernetes/ovnkube-node-kl2xl" Jul 29 19:18:06.802960 ip-10-0-131-93 kubenswrapper[2564]: I0729 19:18:06.802931 2564 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-444jx_faa8c546-c75b-4e10-93b4-661aa2b5f53e/konnectivity-agent/0.log" limit=1024 Jul 29 19:18:06.908318 ip-10-0-131-93 kubenswrapper[2564]: I0729 19:18:06.908254 2564 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-444jx_faa8c546-c75b-4e10-93b4-661aa2b5f53e/konnectivity-agent/0.log" limit=1024 Jul 29 19:18:09.348463 ip-10-0-131-93 kubenswrapper[2564]: I0729 19:18:09.348426 2564 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-network-diagnostics/network-check-source-7d88cb55d8-vvlr8"] Jul 29 19:18:09.360329 ip-10-0-131-93 kubenswrapper[2564]: I0729 19:18:09.360302 2564 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-source-7d88cb55d8-vvlr8" Jul 29 19:18:09.363099 ip-10-0-131-93 kubenswrapper[2564]: I0729 19:18:09.363073 2564 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-network-diagnostics\"/\"network-diagnostics-dockercfg-grwrj\"" Jul 29 19:18:09.367002 ip-10-0-131-93 kubenswrapper[2564]: I0729 19:18:09.366923 2564 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-network-diagnostics/network-check-source-7d88cb55d8-vvlr8"] Jul 29 19:18:09.412501 ip-10-0-131-93 kubenswrapper[2564]: I0729 19:18:09.412455 2564 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-ws9dv\" (UniqueName: \"kubernetes.io/projected/91d2a9e6-9c64-406b-afd1-c6e0ee4c92f6-kube-api-access-ws9dv\") pod \"network-check-source-7d88cb55d8-vvlr8\" (UID: \"91d2a9e6-9c64-406b-afd1-c6e0ee4c92f6\") " pod="openshift-network-diagnostics/network-check-source-7d88cb55d8-vvlr8" Jul 29 19:18:09.513801 ip-10-0-131-93 kubenswrapper[2564]: I0729 19:18:09.513763 2564 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-ws9dv\" (UniqueName: \"kubernetes.io/projected/91d2a9e6-9c64-406b-afd1-c6e0ee4c92f6-kube-api-access-ws9dv\") pod \"network-check-source-7d88cb55d8-vvlr8\" (UID: \"91d2a9e6-9c64-406b-afd1-c6e0ee4c92f6\") " pod="openshift-network-diagnostics/network-check-source-7d88cb55d8-vvlr8" Jul 29 19:18:09.528490 ip-10-0-131-93 kubenswrapper[2564]: I0729 19:18:09.528468 2564 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-network-diagnostics\"/\"kube-root-ca.crt\"" Jul 29 19:18:09.536308 ip-10-0-131-93 kubenswrapper[2564]: I0729 19:18:09.536288 2564 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-network-diagnostics\"/\"openshift-service-ca.crt\"" Jul 29 19:18:09.548770 ip-10-0-131-93 kubenswrapper[2564]: I0729 19:18:09.548745 2564 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-ws9dv\" (UniqueName: \"kubernetes.io/projected/91d2a9e6-9c64-406b-afd1-c6e0ee4c92f6-kube-api-access-ws9dv\") pod \"network-check-source-7d88cb55d8-vvlr8\" (UID: \"91d2a9e6-9c64-406b-afd1-c6e0ee4c92f6\") " pod="openshift-network-diagnostics/network-check-source-7d88cb55d8-vvlr8" Jul 29 19:18:09.565426 ip-10-0-131-93 kubenswrapper[2564]: I0729 19:18:09.565387 2564 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-444jx_faa8c546-c75b-4e10-93b4-661aa2b5f53e/konnectivity-agent/0.log" limit=1024 Jul 29 19:18:09.667043 ip-10-0-131-93 kubenswrapper[2564]: I0729 19:18:09.666957 2564 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-444jx_faa8c546-c75b-4e10-93b4-661aa2b5f53e/konnectivity-agent/0.log" limit=1024 Jul 29 19:18:09.669905 ip-10-0-131-93 kubenswrapper[2564]: I0729 19:18:09.669883 2564 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-source-7d88cb55d8-vvlr8" Jul 29 19:18:09.790189 ip-10-0-131-93 kubenswrapper[2564]: I0729 19:18:09.790167 2564 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-network-diagnostics/network-check-source-7d88cb55d8-vvlr8"] Jul 29 19:18:09.792176 ip-10-0-131-93 kubenswrapper[2564]: W0729 19:18:09.792148 2564 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod91d2a9e6_9c64_406b_afd1_c6e0ee4c92f6.slice/crio-7bafc008becf4ae47d6f686dbfb915e90cd0435df1f97c3cadf752069c757bf7 WatchSource:0}: Error finding container 7bafc008becf4ae47d6f686dbfb915e90cd0435df1f97c3cadf752069c757bf7: Status 404 returned error can't find the container with id 7bafc008becf4ae47d6f686dbfb915e90cd0435df1f97c3cadf752069c757bf7 Jul 29 19:18:10.015655 ip-10-0-131-93 kubenswrapper[2564]: I0729 19:18:10.015626 2564 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-network-diagnostics/network-check-source-7d88cb55d8-vvlr8" event={"ID":"91d2a9e6-9c64-406b-afd1-c6e0ee4c92f6","Type":"ContainerStarted","Data":"7bafc008becf4ae47d6f686dbfb915e90cd0435df1f97c3cadf752069c757bf7"} Jul 29 19:18:10.159556 ip-10-0-131-93 kubenswrapper[2564]: I0729 19:18:10.159522 2564 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-444jx_faa8c546-c75b-4e10-93b4-661aa2b5f53e/konnectivity-agent/0.log" limit=1024 Jul 29 19:18:10.260319 ip-10-0-131-93 kubenswrapper[2564]: I0729 19:18:10.260289 2564 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-444jx_faa8c546-c75b-4e10-93b4-661aa2b5f53e/konnectivity-agent/0.log" limit=1024 Jul 29 19:18:10.413734 ip-10-0-131-93 kubenswrapper[2564]: I0729 19:18:10.413651 2564 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-444jx_faa8c546-c75b-4e10-93b4-661aa2b5f53e/konnectivity-agent/0.log" limit=1024 Jul 29 19:18:10.422260 ip-10-0-131-93 kubenswrapper[2564]: I0729 19:18:10.422230 2564 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/ae220db3-299c-481a-9591-01a8f3873d9a-metrics-certs\") pod \"network-metrics-daemon-hkxvx\" (UID: \"ae220db3-299c-481a-9591-01a8f3873d9a\") " pod="openshift-multus/network-metrics-daemon-hkxvx" Jul 29 19:18:10.425520 ip-10-0-131-93 kubenswrapper[2564]: I0729 19:18:10.425488 2564 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-multus\"/\"metrics-daemon-secret\"" Jul 29 19:18:10.433155 ip-10-0-131-93 kubenswrapper[2564]: E0729 19:18:10.433134 2564 secret.go:189] Couldn't get secret openshift-multus/metrics-daemon-secret: secret "metrics-daemon-secret" not found Jul 29 19:18:10.433281 ip-10-0-131-93 kubenswrapper[2564]: E0729 19:18:10.433208 2564 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/ae220db3-299c-481a-9591-01a8f3873d9a-metrics-certs podName:ae220db3-299c-481a-9591-01a8f3873d9a nodeName:}" failed. No retries permitted until 2026-07-29 19:19:14.433186012 +0000 UTC m=+130.157245698 (durationBeforeRetry 1m4s). Error: MountVolume.SetUp failed for volume "metrics-certs" (UniqueName: "kubernetes.io/secret/ae220db3-299c-481a-9591-01a8f3873d9a-metrics-certs") pod "network-metrics-daemon-hkxvx" (UID: "ae220db3-299c-481a-9591-01a8f3873d9a") : secret "metrics-daemon-secret" not found Jul 29 19:18:10.515303 ip-10-0-131-93 kubenswrapper[2564]: I0729 19:18:10.515264 2564 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-444jx_faa8c546-c75b-4e10-93b4-661aa2b5f53e/konnectivity-agent/0.log" limit=1024 Jul 29 19:18:10.523433 ip-10-0-131-93 kubenswrapper[2564]: I0729 19:18:10.523405 2564 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-tls\" (UniqueName: \"kubernetes.io/secret/d5e5a7da-26f5-4d8d-8e9e-4e8991392e8e-metrics-tls\") pod \"dns-default-kkmn5\" (UID: \"d5e5a7da-26f5-4d8d-8e9e-4e8991392e8e\") " pod="openshift-dns/dns-default-kkmn5" Jul 29 19:18:10.523581 ip-10-0-131-93 kubenswrapper[2564]: I0729 19:18:10.523462 2564 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-qgvzl\" (UniqueName: \"kubernetes.io/projected/02612aaa-a23a-4e6b-b224-123af1bb8230-kube-api-access-qgvzl\") pod \"network-check-target-464tm\" (UID: \"02612aaa-a23a-4e6b-b224-123af1bb8230\") " pod="openshift-network-diagnostics/network-check-target-464tm" Jul 29 19:18:10.523635 ip-10-0-131-93 kubenswrapper[2564]: E0729 19:18:10.523577 2564 secret.go:189] Couldn't get secret openshift-dns/dns-default-metrics-tls: secret "dns-default-metrics-tls" not found Jul 29 19:18:10.523686 ip-10-0-131-93 kubenswrapper[2564]: I0729 19:18:10.523669 2564 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"original-pull-secret\" (UniqueName: \"kubernetes.io/secret/5c0eec03-fbbc-40db-9c78-43c8f98534c7-original-pull-secret\") pod \"global-pull-secret-syncer-zcgr9\" (UID: \"5c0eec03-fbbc-40db-9c78-43c8f98534c7\") " pod="kube-system/global-pull-secret-syncer-zcgr9" Jul 29 19:18:10.523734 ip-10-0-131-93 kubenswrapper[2564]: E0729 19:18:10.523689 2564 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/d5e5a7da-26f5-4d8d-8e9e-4e8991392e8e-metrics-tls podName:d5e5a7da-26f5-4d8d-8e9e-4e8991392e8e nodeName:}" failed. No retries permitted until 2026-07-29 19:18:42.523670595 +0000 UTC m=+98.247730269 (durationBeforeRetry 32s). Error: MountVolume.SetUp failed for volume "metrics-tls" (UniqueName: "kubernetes.io/secret/d5e5a7da-26f5-4d8d-8e9e-4e8991392e8e-metrics-tls") pod "dns-default-kkmn5" (UID: "d5e5a7da-26f5-4d8d-8e9e-4e8991392e8e") : secret "dns-default-metrics-tls" not found Jul 29 19:18:10.523792 ip-10-0-131-93 kubenswrapper[2564]: I0729 19:18:10.523740 2564 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"registry-tls\" (UniqueName: \"kubernetes.io/projected/5ff17af5-17c3-45e6-b47f-408ba15ed5ca-registry-tls\") pod \"image-registry-7f95c97455-tkxb8\" (UID: \"5ff17af5-17c3-45e6-b47f-408ba15ed5ca\") " pod="openshift-image-registry/image-registry-7f95c97455-tkxb8" Jul 29 19:18:10.523862 ip-10-0-131-93 kubenswrapper[2564]: E0729 19:18:10.523840 2564 projected.go:266] Couldn't get secret openshift-image-registry/image-registry-tls: secret "image-registry-tls" not found Jul 29 19:18:10.523862 ip-10-0-131-93 kubenswrapper[2564]: E0729 19:18:10.523856 2564 projected.go:196] Error preparing data for projected volume registry-tls for pod openshift-image-registry/image-registry-7f95c97455-tkxb8: secret "image-registry-tls" not found Jul 29 19:18:10.523978 ip-10-0-131-93 kubenswrapper[2564]: E0729 19:18:10.523898 2564 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/projected/5ff17af5-17c3-45e6-b47f-408ba15ed5ca-registry-tls podName:5ff17af5-17c3-45e6-b47f-408ba15ed5ca nodeName:}" failed. No retries permitted until 2026-07-29 19:18:42.52388612 +0000 UTC m=+98.247945796 (durationBeforeRetry 32s). Error: MountVolume.SetUp failed for volume "registry-tls" (UniqueName: "kubernetes.io/projected/5ff17af5-17c3-45e6-b47f-408ba15ed5ca-registry-tls") pod "image-registry-7f95c97455-tkxb8" (UID: "5ff17af5-17c3-45e6-b47f-408ba15ed5ca") : secret "image-registry-tls" not found Jul 29 19:18:10.526149 ip-10-0-131-93 kubenswrapper[2564]: I0729 19:18:10.526131 2564 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-qgvzl\" (UniqueName: \"kubernetes.io/projected/02612aaa-a23a-4e6b-b224-123af1bb8230-kube-api-access-qgvzl\") pod \"network-check-target-464tm\" (UID: \"02612aaa-a23a-4e6b-b224-123af1bb8230\") " pod="openshift-network-diagnostics/network-check-target-464tm" Jul 29 19:18:10.537933 ip-10-0-131-93 kubenswrapper[2564]: I0729 19:18:10.537909 2564 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"kube-system\"/\"original-pull-secret\"" Jul 29 19:18:10.547053 ip-10-0-131-93 kubenswrapper[2564]: I0729 19:18:10.547021 2564 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"original-pull-secret\" (UniqueName: \"kubernetes.io/secret/5c0eec03-fbbc-40db-9c78-43c8f98534c7-original-pull-secret\") pod \"global-pull-secret-syncer-zcgr9\" (UID: \"5c0eec03-fbbc-40db-9c78-43c8f98534c7\") " pod="kube-system/global-pull-secret-syncer-zcgr9" Jul 29 19:18:10.592224 ip-10-0-131-93 kubenswrapper[2564]: I0729 19:18:10.592190 2564 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-network-diagnostics\"/\"default-dockercfg-bfbzc\"" Jul 29 19:18:10.599805 ip-10-0-131-93 kubenswrapper[2564]: I0729 19:18:10.599781 2564 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-464tm" Jul 29 19:18:10.601082 ip-10-0-131-93 kubenswrapper[2564]: I0729 19:18:10.601060 2564 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-zcgr9" Jul 29 19:18:10.771251 ip-10-0-131-93 kubenswrapper[2564]: I0729 19:18:10.771222 2564 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-network-diagnostics/network-check-target-464tm"] Jul 29 19:18:10.775438 ip-10-0-131-93 kubenswrapper[2564]: W0729 19:18:10.775412 2564 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod02612aaa_a23a_4e6b_b224_123af1bb8230.slice/crio-52890d50b082dc1717b48bd56f70ac9e749161c67109c3bcce1194548dfe3cee WatchSource:0}: Error finding container 52890d50b082dc1717b48bd56f70ac9e749161c67109c3bcce1194548dfe3cee: Status 404 returned error can't find the container with id 52890d50b082dc1717b48bd56f70ac9e749161c67109c3bcce1194548dfe3cee Jul 29 19:18:10.799525 ip-10-0-131-93 kubenswrapper[2564]: I0729 19:18:10.799501 2564 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kube-system/global-pull-secret-syncer-zcgr9"] Jul 29 19:18:10.801333 ip-10-0-131-93 kubenswrapper[2564]: W0729 19:18:10.801307 2564 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod5c0eec03_fbbc_40db_9c78_43c8f98534c7.slice/crio-2f3befbd5dbdcd50b50f4d9629a3c2a139f6aec59ceb03b0e1241d867beffe23 WatchSource:0}: Error finding container 2f3befbd5dbdcd50b50f4d9629a3c2a139f6aec59ceb03b0e1241d867beffe23: Status 404 returned error can't find the container with id 2f3befbd5dbdcd50b50f4d9629a3c2a139f6aec59ceb03b0e1241d867beffe23 Jul 29 19:18:11.019964 ip-10-0-131-93 kubenswrapper[2564]: I0729 19:18:11.019928 2564 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kube-system/global-pull-secret-syncer-zcgr9" event={"ID":"5c0eec03-fbbc-40db-9c78-43c8f98534c7","Type":"ContainerStarted","Data":"2f3befbd5dbdcd50b50f4d9629a3c2a139f6aec59ceb03b0e1241d867beffe23"} Jul 29 19:18:11.021194 ip-10-0-131-93 kubenswrapper[2564]: I0729 19:18:11.021163 2564 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-network-diagnostics/network-check-target-464tm" event={"ID":"02612aaa-a23a-4e6b-b224-123af1bb8230","Type":"ContainerStarted","Data":"52890d50b082dc1717b48bd56f70ac9e749161c67109c3bcce1194548dfe3cee"} Jul 29 19:18:11.214708 ip-10-0-131-93 kubenswrapper[2564]: I0729 19:18:11.214662 2564 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-444jx_faa8c546-c75b-4e10-93b4-661aa2b5f53e/konnectivity-agent/0.log" limit=1024 Jul 29 19:18:11.314251 ip-10-0-131-93 kubenswrapper[2564]: I0729 19:18:11.314171 2564 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-444jx_faa8c546-c75b-4e10-93b4-661aa2b5f53e/konnectivity-agent/0.log" limit=1024 Jul 29 19:18:11.604273 ip-10-0-131-93 kubenswrapper[2564]: I0729 19:18:11.604190 2564 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-444jx_faa8c546-c75b-4e10-93b4-661aa2b5f53e/konnectivity-agent/0.log" limit=1024 Jul 29 19:18:11.705223 ip-10-0-131-93 kubenswrapper[2564]: I0729 19:18:11.705122 2564 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-444jx_faa8c546-c75b-4e10-93b4-661aa2b5f53e/konnectivity-agent/0.log" limit=1024 Jul 29 19:18:12.089472 ip-10-0-131-93 kubenswrapper[2564]: I0729 19:18:12.089433 2564 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-444jx_faa8c546-c75b-4e10-93b4-661aa2b5f53e/konnectivity-agent/0.log" limit=1024 Jul 29 19:18:12.190711 ip-10-0-131-93 kubenswrapper[2564]: I0729 19:18:12.190618 2564 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-444jx_faa8c546-c75b-4e10-93b4-661aa2b5f53e/konnectivity-agent/0.log" limit=1024 Jul 29 19:18:13.001903 ip-10-0-131-93 kubenswrapper[2564]: I0729 19:18:13.001866 2564 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-insights/insights-runtime-extractor-q4bqd"] Jul 29 19:18:13.004176 ip-10-0-131-93 kubenswrapper[2564]: I0729 19:18:13.004155 2564 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-insights/insights-runtime-extractor-q4bqd" Jul 29 19:18:13.018162 ip-10-0-131-93 kubenswrapper[2564]: I0729 19:18:13.018138 2564 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-insights\"/\"insights-runtime-extractor-tls\"" Jul 29 19:18:13.018958 ip-10-0-131-93 kubenswrapper[2564]: I0729 19:18:13.018937 2564 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-insights\"/\"kube-rbac-proxy\"" Jul 29 19:18:13.033899 ip-10-0-131-93 kubenswrapper[2564]: I0729 19:18:13.033681 2564 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-insights\"/\"kube-root-ca.crt\"" Jul 29 19:18:13.038337 ip-10-0-131-93 kubenswrapper[2564]: I0729 19:18:13.038303 2564 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-insights/insights-runtime-extractor-q4bqd"] Jul 29 19:18:13.038768 ip-10-0-131-93 kubenswrapper[2564]: I0729 19:18:13.038743 2564 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-insights\"/\"openshift-service-ca.crt\"" Jul 29 19:18:13.041690 ip-10-0-131-93 kubenswrapper[2564]: I0729 19:18:13.041651 2564 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-insights\"/\"insights-runtime-extractor-sa-dockercfg-knhpr\"" Jul 29 19:18:13.046887 ip-10-0-131-93 kubenswrapper[2564]: I0729 19:18:13.046865 2564 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"data-volume\" (UniqueName: \"kubernetes.io/empty-dir/9887221a-35c3-4cb7-8f93-d03a8e1c0e8e-data-volume\") pod \"insights-runtime-extractor-q4bqd\" (UID: \"9887221a-35c3-4cb7-8f93-d03a8e1c0e8e\") " pod="openshift-insights/insights-runtime-extractor-q4bqd" Jul 29 19:18:13.046974 ip-10-0-131-93 kubenswrapper[2564]: I0729 19:18:13.046896 2564 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-rbac-proxy-cm\" (UniqueName: \"kubernetes.io/configmap/9887221a-35c3-4cb7-8f93-d03a8e1c0e8e-kube-rbac-proxy-cm\") pod \"insights-runtime-extractor-q4bqd\" (UID: \"9887221a-35c3-4cb7-8f93-d03a8e1c0e8e\") " pod="openshift-insights/insights-runtime-extractor-q4bqd" Jul 29 19:18:13.046974 ip-10-0-131-93 kubenswrapper[2564]: I0729 19:18:13.046917 2564 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"crio-socket\" (UniqueName: \"kubernetes.io/host-path/9887221a-35c3-4cb7-8f93-d03a8e1c0e8e-crio-socket\") pod \"insights-runtime-extractor-q4bqd\" (UID: \"9887221a-35c3-4cb7-8f93-d03a8e1c0e8e\") " pod="openshift-insights/insights-runtime-extractor-q4bqd" Jul 29 19:18:13.046974 ip-10-0-131-93 kubenswrapper[2564]: I0729 19:18:13.046949 2564 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-ttztt\" (UniqueName: \"kubernetes.io/projected/9887221a-35c3-4cb7-8f93-d03a8e1c0e8e-kube-api-access-ttztt\") pod \"insights-runtime-extractor-q4bqd\" (UID: \"9887221a-35c3-4cb7-8f93-d03a8e1c0e8e\") " pod="openshift-insights/insights-runtime-extractor-q4bqd" Jul 29 19:18:13.047117 ip-10-0-131-93 kubenswrapper[2564]: I0729 19:18:13.046999 2564 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"insights-runtime-extractor-tls\" (UniqueName: \"kubernetes.io/secret/9887221a-35c3-4cb7-8f93-d03a8e1c0e8e-insights-runtime-extractor-tls\") pod \"insights-runtime-extractor-q4bqd\" (UID: \"9887221a-35c3-4cb7-8f93-d03a8e1c0e8e\") " pod="openshift-insights/insights-runtime-extractor-q4bqd" Jul 29 19:18:13.128591 ip-10-0-131-93 kubenswrapper[2564]: I0729 19:18:13.128552 2564 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-444jx_faa8c546-c75b-4e10-93b4-661aa2b5f53e/konnectivity-agent/0.log" limit=1024 Jul 29 19:18:13.147983 ip-10-0-131-93 kubenswrapper[2564]: I0729 19:18:13.147952 2564 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-ttztt\" (UniqueName: \"kubernetes.io/projected/9887221a-35c3-4cb7-8f93-d03a8e1c0e8e-kube-api-access-ttztt\") pod \"insights-runtime-extractor-q4bqd\" (UID: \"9887221a-35c3-4cb7-8f93-d03a8e1c0e8e\") " pod="openshift-insights/insights-runtime-extractor-q4bqd" Jul 29 19:18:13.148190 ip-10-0-131-93 kubenswrapper[2564]: I0729 19:18:13.148155 2564 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"insights-runtime-extractor-tls\" (UniqueName: \"kubernetes.io/secret/9887221a-35c3-4cb7-8f93-d03a8e1c0e8e-insights-runtime-extractor-tls\") pod \"insights-runtime-extractor-q4bqd\" (UID: \"9887221a-35c3-4cb7-8f93-d03a8e1c0e8e\") " pod="openshift-insights/insights-runtime-extractor-q4bqd" Jul 29 19:18:13.148309 ip-10-0-131-93 kubenswrapper[2564]: I0729 19:18:13.148208 2564 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"data-volume\" (UniqueName: \"kubernetes.io/empty-dir/9887221a-35c3-4cb7-8f93-d03a8e1c0e8e-data-volume\") pod \"insights-runtime-extractor-q4bqd\" (UID: \"9887221a-35c3-4cb7-8f93-d03a8e1c0e8e\") " pod="openshift-insights/insights-runtime-extractor-q4bqd" Jul 29 19:18:13.148309 ip-10-0-131-93 kubenswrapper[2564]: I0729 19:18:13.148237 2564 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-rbac-proxy-cm\" (UniqueName: \"kubernetes.io/configmap/9887221a-35c3-4cb7-8f93-d03a8e1c0e8e-kube-rbac-proxy-cm\") pod \"insights-runtime-extractor-q4bqd\" (UID: \"9887221a-35c3-4cb7-8f93-d03a8e1c0e8e\") " pod="openshift-insights/insights-runtime-extractor-q4bqd" Jul 29 19:18:13.148454 ip-10-0-131-93 kubenswrapper[2564]: E0729 19:18:13.148320 2564 configmap.go:193] Couldn't get configMap openshift-insights/kube-rbac-proxy: configmap "kube-rbac-proxy" not found Jul 29 19:18:13.148454 ip-10-0-131-93 kubenswrapper[2564]: E0729 19:18:13.148340 2564 secret.go:189] Couldn't get secret openshift-insights/insights-runtime-extractor-tls: secret "insights-runtime-extractor-tls" not found Jul 29 19:18:13.148454 ip-10-0-131-93 kubenswrapper[2564]: E0729 19:18:13.148395 2564 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/configmap/9887221a-35c3-4cb7-8f93-d03a8e1c0e8e-kube-rbac-proxy-cm podName:9887221a-35c3-4cb7-8f93-d03a8e1c0e8e nodeName:}" failed. No retries permitted until 2026-07-29 19:18:13.648363195 +0000 UTC m=+69.372422881 (durationBeforeRetry 500ms). Error: MountVolume.SetUp failed for volume "kube-rbac-proxy-cm" (UniqueName: "kubernetes.io/configmap/9887221a-35c3-4cb7-8f93-d03a8e1c0e8e-kube-rbac-proxy-cm") pod "insights-runtime-extractor-q4bqd" (UID: "9887221a-35c3-4cb7-8f93-d03a8e1c0e8e") : configmap "kube-rbac-proxy" not found Jul 29 19:18:13.148454 ip-10-0-131-93 kubenswrapper[2564]: I0729 19:18:13.148416 2564 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"crio-socket\" (UniqueName: \"kubernetes.io/host-path/9887221a-35c3-4cb7-8f93-d03a8e1c0e8e-crio-socket\") pod \"insights-runtime-extractor-q4bqd\" (UID: \"9887221a-35c3-4cb7-8f93-d03a8e1c0e8e\") " pod="openshift-insights/insights-runtime-extractor-q4bqd" Jul 29 19:18:13.148647 ip-10-0-131-93 kubenswrapper[2564]: E0729 19:18:13.148483 2564 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/9887221a-35c3-4cb7-8f93-d03a8e1c0e8e-insights-runtime-extractor-tls podName:9887221a-35c3-4cb7-8f93-d03a8e1c0e8e nodeName:}" failed. No retries permitted until 2026-07-29 19:18:13.648465985 +0000 UTC m=+69.372525655 (durationBeforeRetry 500ms). Error: MountVolume.SetUp failed for volume "insights-runtime-extractor-tls" (UniqueName: "kubernetes.io/secret/9887221a-35c3-4cb7-8f93-d03a8e1c0e8e-insights-runtime-extractor-tls") pod "insights-runtime-extractor-q4bqd" (UID: "9887221a-35c3-4cb7-8f93-d03a8e1c0e8e") : secret "insights-runtime-extractor-tls" not found Jul 29 19:18:13.148647 ip-10-0-131-93 kubenswrapper[2564]: I0729 19:18:13.148542 2564 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"crio-socket\" (UniqueName: \"kubernetes.io/host-path/9887221a-35c3-4cb7-8f93-d03a8e1c0e8e-crio-socket\") pod \"insights-runtime-extractor-q4bqd\" (UID: \"9887221a-35c3-4cb7-8f93-d03a8e1c0e8e\") " pod="openshift-insights/insights-runtime-extractor-q4bqd" Jul 29 19:18:13.148751 ip-10-0-131-93 kubenswrapper[2564]: I0729 19:18:13.148671 2564 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"data-volume\" (UniqueName: \"kubernetes.io/empty-dir/9887221a-35c3-4cb7-8f93-d03a8e1c0e8e-data-volume\") pod \"insights-runtime-extractor-q4bqd\" (UID: \"9887221a-35c3-4cb7-8f93-d03a8e1c0e8e\") " pod="openshift-insights/insights-runtime-extractor-q4bqd" Jul 29 19:18:13.157272 ip-10-0-131-93 kubenswrapper[2564]: I0729 19:18:13.157235 2564 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-ttztt\" (UniqueName: \"kubernetes.io/projected/9887221a-35c3-4cb7-8f93-d03a8e1c0e8e-kube-api-access-ttztt\") pod \"insights-runtime-extractor-q4bqd\" (UID: \"9887221a-35c3-4cb7-8f93-d03a8e1c0e8e\") " pod="openshift-insights/insights-runtime-extractor-q4bqd" Jul 29 19:18:13.225397 ip-10-0-131-93 kubenswrapper[2564]: I0729 19:18:13.225353 2564 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-444jx_faa8c546-c75b-4e10-93b4-661aa2b5f53e/konnectivity-agent/0.log" limit=1024 Jul 29 19:18:13.652214 ip-10-0-131-93 kubenswrapper[2564]: I0729 19:18:13.652178 2564 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"insights-runtime-extractor-tls\" (UniqueName: \"kubernetes.io/secret/9887221a-35c3-4cb7-8f93-d03a8e1c0e8e-insights-runtime-extractor-tls\") pod \"insights-runtime-extractor-q4bqd\" (UID: \"9887221a-35c3-4cb7-8f93-d03a8e1c0e8e\") " pod="openshift-insights/insights-runtime-extractor-q4bqd" Jul 29 19:18:13.652419 ip-10-0-131-93 kubenswrapper[2564]: I0729 19:18:13.652236 2564 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-rbac-proxy-cm\" (UniqueName: \"kubernetes.io/configmap/9887221a-35c3-4cb7-8f93-d03a8e1c0e8e-kube-rbac-proxy-cm\") pod \"insights-runtime-extractor-q4bqd\" (UID: \"9887221a-35c3-4cb7-8f93-d03a8e1c0e8e\") " pod="openshift-insights/insights-runtime-extractor-q4bqd" Jul 29 19:18:13.652419 ip-10-0-131-93 kubenswrapper[2564]: E0729 19:18:13.652325 2564 configmap.go:193] Couldn't get configMap openshift-insights/kube-rbac-proxy: configmap "kube-rbac-proxy" not found Jul 29 19:18:13.652419 ip-10-0-131-93 kubenswrapper[2564]: E0729 19:18:13.652410 2564 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/configmap/9887221a-35c3-4cb7-8f93-d03a8e1c0e8e-kube-rbac-proxy-cm podName:9887221a-35c3-4cb7-8f93-d03a8e1c0e8e nodeName:}" failed. No retries permitted until 2026-07-29 19:18:14.652367696 +0000 UTC m=+70.376427383 (durationBeforeRetry 1s). Error: MountVolume.SetUp failed for volume "kube-rbac-proxy-cm" (UniqueName: "kubernetes.io/configmap/9887221a-35c3-4cb7-8f93-d03a8e1c0e8e-kube-rbac-proxy-cm") pod "insights-runtime-extractor-q4bqd" (UID: "9887221a-35c3-4cb7-8f93-d03a8e1c0e8e") : configmap "kube-rbac-proxy" not found Jul 29 19:18:13.652595 ip-10-0-131-93 kubenswrapper[2564]: E0729 19:18:13.652433 2564 secret.go:189] Couldn't get secret openshift-insights/insights-runtime-extractor-tls: secret "insights-runtime-extractor-tls" not found Jul 29 19:18:13.652595 ip-10-0-131-93 kubenswrapper[2564]: E0729 19:18:13.652496 2564 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/9887221a-35c3-4cb7-8f93-d03a8e1c0e8e-insights-runtime-extractor-tls podName:9887221a-35c3-4cb7-8f93-d03a8e1c0e8e nodeName:}" failed. No retries permitted until 2026-07-29 19:18:14.652479 +0000 UTC m=+70.376538677 (durationBeforeRetry 1s). Error: MountVolume.SetUp failed for volume "insights-runtime-extractor-tls" (UniqueName: "kubernetes.io/secret/9887221a-35c3-4cb7-8f93-d03a8e1c0e8e-insights-runtime-extractor-tls") pod "insights-runtime-extractor-q4bqd" (UID: "9887221a-35c3-4cb7-8f93-d03a8e1c0e8e") : secret "insights-runtime-extractor-tls" not found Jul 29 19:18:14.030870 ip-10-0-131-93 kubenswrapper[2564]: I0729 19:18:14.030819 2564 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-network-diagnostics/network-check-source-7d88cb55d8-vvlr8" event={"ID":"91d2a9e6-9c64-406b-afd1-c6e0ee4c92f6","Type":"ContainerStarted","Data":"882a10ae53a46fb3c61154801e504621a44a106963da259d490b85fed70e8cfb"} Jul 29 19:18:14.032923 ip-10-0-131-93 kubenswrapper[2564]: I0729 19:18:14.032890 2564 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-network-diagnostics/network-check-target-464tm" event={"ID":"02612aaa-a23a-4e6b-b224-123af1bb8230","Type":"ContainerStarted","Data":"6159b72f51a3204865e2e8bbb298eadde3cfbac0ae8200b2991aa62d7baf4b46"} Jul 29 19:18:14.033068 ip-10-0-131-93 kubenswrapper[2564]: I0729 19:18:14.033019 2564 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-network-diagnostics/network-check-target-464tm" Jul 29 19:18:14.046109 ip-10-0-131-93 kubenswrapper[2564]: I0729 19:18:14.046058 2564 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-network-diagnostics/network-check-source-7d88cb55d8-vvlr8" podStartSLOduration=1.786163484 podStartE2EDuration="5.046039895s" totalImagesPullingTime="3.259876411s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-29 19:18:09 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-29 19:18:14.045402995 +0000 UTC m=+69.769462684" watchObservedRunningTime="2026-07-29 19:18:14.046039895 +0000 UTC m=+69.770099587" Jul 29 19:18:14.060049 ip-10-0-131-93 kubenswrapper[2564]: I0729 19:18:14.060003 2564 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-network-diagnostics/network-check-target-464tm" podStartSLOduration=67.769064816 podStartE2EDuration="1m10.059988656s" totalImagesPullingTime="2.29092384s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-29 19:17:04 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-29 19:18:14.058943409 +0000 UTC m=+69.783003101" watchObservedRunningTime="2026-07-29 19:18:14.059988656 +0000 UTC m=+69.784048332" Jul 29 19:18:14.144493 ip-10-0-131-93 kubenswrapper[2564]: I0729 19:18:14.144458 2564 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-444jx_faa8c546-c75b-4e10-93b4-661aa2b5f53e/konnectivity-agent/0.log" limit=1024 Jul 29 19:18:14.153955 ip-10-0-131-93 kubenswrapper[2564]: I0729 19:18:14.153928 2564 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-kube-storage-version-migrator/migrator-6865c9967-g6r5w"] Jul 29 19:18:14.156416 ip-10-0-131-93 kubenswrapper[2564]: I0729 19:18:14.156400 2564 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-kube-storage-version-migrator/migrator-6865c9967-g6r5w" Jul 29 19:18:14.158830 ip-10-0-131-93 kubenswrapper[2564]: I0729 19:18:14.158804 2564 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-kube-storage-version-migrator\"/\"kube-storage-version-migrator-sa-dockercfg-75zzw\"" Jul 29 19:18:14.159053 ip-10-0-131-93 kubenswrapper[2564]: I0729 19:18:14.159040 2564 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-kube-storage-version-migrator\"/\"kube-root-ca.crt\"" Jul 29 19:18:14.159309 ip-10-0-131-93 kubenswrapper[2564]: I0729 19:18:14.159292 2564 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-kube-storage-version-migrator\"/\"openshift-service-ca.crt\"" Jul 29 19:18:14.167140 ip-10-0-131-93 kubenswrapper[2564]: I0729 19:18:14.167117 2564 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-kube-storage-version-migrator/migrator-6865c9967-g6r5w"] Jul 29 19:18:14.238045 ip-10-0-131-93 kubenswrapper[2564]: I0729 19:18:14.237976 2564 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-444jx_faa8c546-c75b-4e10-93b4-661aa2b5f53e/konnectivity-agent/0.log" limit=1024 Jul 29 19:18:14.257199 ip-10-0-131-93 kubenswrapper[2564]: I0729 19:18:14.257168 2564 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-rrzqm\" (UniqueName: \"kubernetes.io/projected/969f8591-6cf6-4c21-ac1e-4d8b6d827810-kube-api-access-rrzqm\") pod \"migrator-6865c9967-g6r5w\" (UID: \"969f8591-6cf6-4c21-ac1e-4d8b6d827810\") " pod="openshift-kube-storage-version-migrator/migrator-6865c9967-g6r5w" Jul 29 19:18:14.358702 ip-10-0-131-93 kubenswrapper[2564]: I0729 19:18:14.358626 2564 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-rrzqm\" (UniqueName: \"kubernetes.io/projected/969f8591-6cf6-4c21-ac1e-4d8b6d827810-kube-api-access-rrzqm\") pod \"migrator-6865c9967-g6r5w\" (UID: \"969f8591-6cf6-4c21-ac1e-4d8b6d827810\") " pod="openshift-kube-storage-version-migrator/migrator-6865c9967-g6r5w" Jul 29 19:18:14.369660 ip-10-0-131-93 kubenswrapper[2564]: I0729 19:18:14.369528 2564 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-444jx_faa8c546-c75b-4e10-93b4-661aa2b5f53e/konnectivity-agent/0.log" limit=1024 Jul 29 19:18:14.369924 ip-10-0-131-93 kubenswrapper[2564]: I0729 19:18:14.369867 2564 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-rrzqm\" (UniqueName: \"kubernetes.io/projected/969f8591-6cf6-4c21-ac1e-4d8b6d827810-kube-api-access-rrzqm\") pod \"migrator-6865c9967-g6r5w\" (UID: \"969f8591-6cf6-4c21-ac1e-4d8b6d827810\") " pod="openshift-kube-storage-version-migrator/migrator-6865c9967-g6r5w" Jul 29 19:18:14.464791 ip-10-0-131-93 kubenswrapper[2564]: I0729 19:18:14.464758 2564 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-kube-storage-version-migrator/migrator-6865c9967-g6r5w" Jul 29 19:18:14.465319 ip-10-0-131-93 kubenswrapper[2564]: I0729 19:18:14.465288 2564 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-444jx_faa8c546-c75b-4e10-93b4-661aa2b5f53e/konnectivity-agent/0.log" limit=1024 Jul 29 19:18:14.593427 ip-10-0-131-93 kubenswrapper[2564]: I0729 19:18:14.593354 2564 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-kube-storage-version-migrator/migrator-6865c9967-g6r5w"] Jul 29 19:18:14.597921 ip-10-0-131-93 kubenswrapper[2564]: W0729 19:18:14.597880 2564 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod969f8591_6cf6_4c21_ac1e_4d8b6d827810.slice/crio-3ba0c84ed5127e8de23ed6ecbbb1432d0ab5f5df21229eeae7f31f0f7da426b7 WatchSource:0}: Error finding container 3ba0c84ed5127e8de23ed6ecbbb1432d0ab5f5df21229eeae7f31f0f7da426b7: Status 404 returned error can't find the container with id 3ba0c84ed5127e8de23ed6ecbbb1432d0ab5f5df21229eeae7f31f0f7da426b7 Jul 29 19:18:14.661746 ip-10-0-131-93 kubenswrapper[2564]: I0729 19:18:14.661663 2564 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"insights-runtime-extractor-tls\" (UniqueName: \"kubernetes.io/secret/9887221a-35c3-4cb7-8f93-d03a8e1c0e8e-insights-runtime-extractor-tls\") pod \"insights-runtime-extractor-q4bqd\" (UID: \"9887221a-35c3-4cb7-8f93-d03a8e1c0e8e\") " pod="openshift-insights/insights-runtime-extractor-q4bqd" Jul 29 19:18:14.661746 ip-10-0-131-93 kubenswrapper[2564]: I0729 19:18:14.661730 2564 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-rbac-proxy-cm\" (UniqueName: \"kubernetes.io/configmap/9887221a-35c3-4cb7-8f93-d03a8e1c0e8e-kube-rbac-proxy-cm\") pod \"insights-runtime-extractor-q4bqd\" (UID: \"9887221a-35c3-4cb7-8f93-d03a8e1c0e8e\") " pod="openshift-insights/insights-runtime-extractor-q4bqd" Jul 29 19:18:14.661929 ip-10-0-131-93 kubenswrapper[2564]: E0729 19:18:14.661843 2564 configmap.go:193] Couldn't get configMap openshift-insights/kube-rbac-proxy: configmap "kube-rbac-proxy" not found Jul 29 19:18:14.661929 ip-10-0-131-93 kubenswrapper[2564]: E0729 19:18:14.661844 2564 secret.go:189] Couldn't get secret openshift-insights/insights-runtime-extractor-tls: secret "insights-runtime-extractor-tls" not found Jul 29 19:18:14.661929 ip-10-0-131-93 kubenswrapper[2564]: E0729 19:18:14.661897 2564 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/configmap/9887221a-35c3-4cb7-8f93-d03a8e1c0e8e-kube-rbac-proxy-cm podName:9887221a-35c3-4cb7-8f93-d03a8e1c0e8e nodeName:}" failed. No retries permitted until 2026-07-29 19:18:16.661882436 +0000 UTC m=+72.385942106 (durationBeforeRetry 2s). Error: MountVolume.SetUp failed for volume "kube-rbac-proxy-cm" (UniqueName: "kubernetes.io/configmap/9887221a-35c3-4cb7-8f93-d03a8e1c0e8e-kube-rbac-proxy-cm") pod "insights-runtime-extractor-q4bqd" (UID: "9887221a-35c3-4cb7-8f93-d03a8e1c0e8e") : configmap "kube-rbac-proxy" not found Jul 29 19:18:14.661929 ip-10-0-131-93 kubenswrapper[2564]: E0729 19:18:14.661913 2564 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/9887221a-35c3-4cb7-8f93-d03a8e1c0e8e-insights-runtime-extractor-tls podName:9887221a-35c3-4cb7-8f93-d03a8e1c0e8e nodeName:}" failed. No retries permitted until 2026-07-29 19:18:16.661904073 +0000 UTC m=+72.385963747 (durationBeforeRetry 2s). Error: MountVolume.SetUp failed for volume "insights-runtime-extractor-tls" (UniqueName: "kubernetes.io/secret/9887221a-35c3-4cb7-8f93-d03a8e1c0e8e-insights-runtime-extractor-tls") pod "insights-runtime-extractor-q4bqd" (UID: "9887221a-35c3-4cb7-8f93-d03a8e1c0e8e") : secret "insights-runtime-extractor-tls" not found Jul 29 19:18:14.677492 ip-10-0-131-93 kubenswrapper[2564]: I0729 19:18:14.677460 2564 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-444jx_faa8c546-c75b-4e10-93b4-661aa2b5f53e/konnectivity-agent/0.log" limit=1024 Jul 29 19:18:14.732736 ip-10-0-131-93 kubenswrapper[2564]: I0729 19:18:14.732698 2564 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-dns_node-resolver-lzsmp_ffe05666-ed11-4573-9442-a9646b974e61/dns-node-resolver/0.log" Jul 29 19:18:14.778419 ip-10-0-131-93 kubenswrapper[2564]: I0729 19:18:14.778363 2564 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-444jx_faa8c546-c75b-4e10-93b4-661aa2b5f53e/konnectivity-agent/0.log" limit=1024 Jul 29 19:18:15.036794 ip-10-0-131-93 kubenswrapper[2564]: I0729 19:18:15.036755 2564 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-kube-storage-version-migrator/migrator-6865c9967-g6r5w" event={"ID":"969f8591-6cf6-4c21-ac1e-4d8b6d827810","Type":"ContainerStarted","Data":"3ba0c84ed5127e8de23ed6ecbbb1432d0ab5f5df21229eeae7f31f0f7da426b7"} Jul 29 19:18:15.146835 ip-10-0-131-93 kubenswrapper[2564]: I0729 19:18:15.146791 2564 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-444jx_faa8c546-c75b-4e10-93b4-661aa2b5f53e/konnectivity-agent/0.log" limit=1024 Jul 29 19:18:15.252757 ip-10-0-131-93 kubenswrapper[2564]: I0729 19:18:15.252725 2564 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-444jx_faa8c546-c75b-4e10-93b4-661aa2b5f53e/konnectivity-agent/0.log" limit=1024 Jul 29 19:18:15.735607 ip-10-0-131-93 kubenswrapper[2564]: I0729 19:18:15.735572 2564 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-image-registry_node-ca-2lv4n_edfe2676-f3b8-4eaa-ad17-ecf737b9f786/node-ca/0.log" Jul 29 19:18:16.676190 ip-10-0-131-93 kubenswrapper[2564]: I0729 19:18:16.676155 2564 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"insights-runtime-extractor-tls\" (UniqueName: \"kubernetes.io/secret/9887221a-35c3-4cb7-8f93-d03a8e1c0e8e-insights-runtime-extractor-tls\") pod \"insights-runtime-extractor-q4bqd\" (UID: \"9887221a-35c3-4cb7-8f93-d03a8e1c0e8e\") " pod="openshift-insights/insights-runtime-extractor-q4bqd" Jul 29 19:18:16.676190 ip-10-0-131-93 kubenswrapper[2564]: I0729 19:18:16.676198 2564 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-rbac-proxy-cm\" (UniqueName: \"kubernetes.io/configmap/9887221a-35c3-4cb7-8f93-d03a8e1c0e8e-kube-rbac-proxy-cm\") pod \"insights-runtime-extractor-q4bqd\" (UID: \"9887221a-35c3-4cb7-8f93-d03a8e1c0e8e\") " pod="openshift-insights/insights-runtime-extractor-q4bqd" Jul 29 19:18:16.676618 ip-10-0-131-93 kubenswrapper[2564]: E0729 19:18:16.676265 2564 configmap.go:193] Couldn't get configMap openshift-insights/kube-rbac-proxy: configmap "kube-rbac-proxy" not found Jul 29 19:18:16.676618 ip-10-0-131-93 kubenswrapper[2564]: E0729 19:18:16.676317 2564 secret.go:189] Couldn't get secret openshift-insights/insights-runtime-extractor-tls: secret "insights-runtime-extractor-tls" not found Jul 29 19:18:16.676618 ip-10-0-131-93 kubenswrapper[2564]: E0729 19:18:16.676321 2564 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/configmap/9887221a-35c3-4cb7-8f93-d03a8e1c0e8e-kube-rbac-proxy-cm podName:9887221a-35c3-4cb7-8f93-d03a8e1c0e8e nodeName:}" failed. No retries permitted until 2026-07-29 19:18:20.67630583 +0000 UTC m=+76.400365507 (durationBeforeRetry 4s). Error: MountVolume.SetUp failed for volume "kube-rbac-proxy-cm" (UniqueName: "kubernetes.io/configmap/9887221a-35c3-4cb7-8f93-d03a8e1c0e8e-kube-rbac-proxy-cm") pod "insights-runtime-extractor-q4bqd" (UID: "9887221a-35c3-4cb7-8f93-d03a8e1c0e8e") : configmap "kube-rbac-proxy" not found Jul 29 19:18:16.676618 ip-10-0-131-93 kubenswrapper[2564]: E0729 19:18:16.676403 2564 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/9887221a-35c3-4cb7-8f93-d03a8e1c0e8e-insights-runtime-extractor-tls podName:9887221a-35c3-4cb7-8f93-d03a8e1c0e8e nodeName:}" failed. No retries permitted until 2026-07-29 19:18:20.676368188 +0000 UTC m=+76.400427861 (durationBeforeRetry 4s). Error: MountVolume.SetUp failed for volume "insights-runtime-extractor-tls" (UniqueName: "kubernetes.io/secret/9887221a-35c3-4cb7-8f93-d03a8e1c0e8e-insights-runtime-extractor-tls") pod "insights-runtime-extractor-q4bqd" (UID: "9887221a-35c3-4cb7-8f93-d03a8e1c0e8e") : secret "insights-runtime-extractor-tls" not found Jul 29 19:18:16.741097 ip-10-0-131-93 kubenswrapper[2564]: I0729 19:18:16.741048 2564 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ingress-canary_ingress-canary-jbktw_3832137d-c339-44b6-8ddc-af8c959c659e/serve-healthcheck-canary/0.log" Jul 29 19:18:16.798791 ip-10-0-131-93 kubenswrapper[2564]: I0729 19:18:16.798757 2564 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-444jx_faa8c546-c75b-4e10-93b4-661aa2b5f53e/konnectivity-agent/0.log" limit=1024 Jul 29 19:18:17.043402 ip-10-0-131-93 kubenswrapper[2564]: I0729 19:18:17.043348 2564 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kube-system/global-pull-secret-syncer-zcgr9" event={"ID":"5c0eec03-fbbc-40db-9c78-43c8f98534c7","Type":"ContainerStarted","Data":"5701ee50d7ab1dd6550e33b3ffcaac79850159caf5e85a397216297cb072920c"} Jul 29 19:18:17.066229 ip-10-0-131-93 kubenswrapper[2564]: I0729 19:18:17.066177 2564 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="kube-system/global-pull-secret-syncer-zcgr9" podStartSLOduration=66.878142865 podStartE2EDuration="1m12.066161999s" totalImagesPullingTime="5.188019134s" totalInitContainerRuntime="0s" isStatefulPod=true podCreationTimestamp="2026-07-29 19:17:05 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-29 19:18:17.064725741 +0000 UTC m=+72.788785432" watchObservedRunningTime="2026-07-29 19:18:17.066161999 +0000 UTC m=+72.790221690" Jul 29 19:18:17.403515 ip-10-0-131-93 kubenswrapper[2564]: I0729 19:18:17.403489 2564 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-444jx_faa8c546-c75b-4e10-93b4-661aa2b5f53e/konnectivity-agent/0.log" limit=1024 Jul 29 19:18:18.049998 ip-10-0-131-93 kubenswrapper[2564]: I0729 19:18:18.049969 2564 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-kube-storage-version-migrator/migrator-6865c9967-g6r5w" event={"ID":"969f8591-6cf6-4c21-ac1e-4d8b6d827810","Type":"ContainerStarted","Data":"afc0927f412fc821c7c7bb5beb67b2160f68e33405efe798d2854a3533efcced"} Jul 29 19:18:18.050397 ip-10-0-131-93 kubenswrapper[2564]: I0729 19:18:18.050015 2564 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-kube-storage-version-migrator/migrator-6865c9967-g6r5w" event={"ID":"969f8591-6cf6-4c21-ac1e-4d8b6d827810","Type":"ContainerStarted","Data":"4b3357d997866ce1ef750cc4f2ffa6f619bb9d7502116dc0bee69800cd7d52ee"} Jul 29 19:18:18.073521 ip-10-0-131-93 kubenswrapper[2564]: I0729 19:18:18.073479 2564 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-kube-storage-version-migrator/migrator-6865c9967-g6r5w" podStartSLOduration=1.450764165 podStartE2EDuration="4.073462104s" totalImagesPullingTime="2.622697939s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-29 19:18:14 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-29 19:18:18.072912202 +0000 UTC m=+73.796971895" watchObservedRunningTime="2026-07-29 19:18:18.073462104 +0000 UTC m=+73.797521795" Jul 29 19:18:18.387295 ip-10-0-131-93 kubenswrapper[2564]: I0729 19:18:18.387220 2564 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-444jx_faa8c546-c75b-4e10-93b4-661aa2b5f53e/konnectivity-agent/0.log" limit=1024 Jul 29 19:18:18.880696 ip-10-0-131-93 kubenswrapper[2564]: I0729 19:18:18.880664 2564 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-444jx_faa8c546-c75b-4e10-93b4-661aa2b5f53e/konnectivity-agent/0.log" limit=1024 Jul 29 19:18:18.980346 ip-10-0-131-93 kubenswrapper[2564]: I0729 19:18:18.980313 2564 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-444jx_faa8c546-c75b-4e10-93b4-661aa2b5f53e/konnectivity-agent/0.log" limit=1024 Jul 29 19:18:20.707179 ip-10-0-131-93 kubenswrapper[2564]: I0729 19:18:20.707121 2564 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"insights-runtime-extractor-tls\" (UniqueName: \"kubernetes.io/secret/9887221a-35c3-4cb7-8f93-d03a8e1c0e8e-insights-runtime-extractor-tls\") pod \"insights-runtime-extractor-q4bqd\" (UID: \"9887221a-35c3-4cb7-8f93-d03a8e1c0e8e\") " pod="openshift-insights/insights-runtime-extractor-q4bqd" Jul 29 19:18:20.707179 ip-10-0-131-93 kubenswrapper[2564]: I0729 19:18:20.707183 2564 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-rbac-proxy-cm\" (UniqueName: \"kubernetes.io/configmap/9887221a-35c3-4cb7-8f93-d03a8e1c0e8e-kube-rbac-proxy-cm\") pod \"insights-runtime-extractor-q4bqd\" (UID: \"9887221a-35c3-4cb7-8f93-d03a8e1c0e8e\") " pod="openshift-insights/insights-runtime-extractor-q4bqd" Jul 29 19:18:20.707707 ip-10-0-131-93 kubenswrapper[2564]: E0729 19:18:20.707262 2564 configmap.go:193] Couldn't get configMap openshift-insights/kube-rbac-proxy: configmap "kube-rbac-proxy" not found Jul 29 19:18:20.707707 ip-10-0-131-93 kubenswrapper[2564]: E0729 19:18:20.707310 2564 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/configmap/9887221a-35c3-4cb7-8f93-d03a8e1c0e8e-kube-rbac-proxy-cm podName:9887221a-35c3-4cb7-8f93-d03a8e1c0e8e nodeName:}" failed. No retries permitted until 2026-07-29 19:18:28.707297385 +0000 UTC m=+84.431357059 (durationBeforeRetry 8s). Error: MountVolume.SetUp failed for volume "kube-rbac-proxy-cm" (UniqueName: "kubernetes.io/configmap/9887221a-35c3-4cb7-8f93-d03a8e1c0e8e-kube-rbac-proxy-cm") pod "insights-runtime-extractor-q4bqd" (UID: "9887221a-35c3-4cb7-8f93-d03a8e1c0e8e") : configmap "kube-rbac-proxy" not found Jul 29 19:18:20.707707 ip-10-0-131-93 kubenswrapper[2564]: E0729 19:18:20.707262 2564 secret.go:189] Couldn't get secret openshift-insights/insights-runtime-extractor-tls: secret "insights-runtime-extractor-tls" not found Jul 29 19:18:20.707707 ip-10-0-131-93 kubenswrapper[2564]: E0729 19:18:20.707414 2564 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/9887221a-35c3-4cb7-8f93-d03a8e1c0e8e-insights-runtime-extractor-tls podName:9887221a-35c3-4cb7-8f93-d03a8e1c0e8e nodeName:}" failed. No retries permitted until 2026-07-29 19:18:28.707396269 +0000 UTC m=+84.431455939 (durationBeforeRetry 8s). Error: MountVolume.SetUp failed for volume "insights-runtime-extractor-tls" (UniqueName: "kubernetes.io/secret/9887221a-35c3-4cb7-8f93-d03a8e1c0e8e-insights-runtime-extractor-tls") pod "insights-runtime-extractor-q4bqd" (UID: "9887221a-35c3-4cb7-8f93-d03a8e1c0e8e") : secret "insights-runtime-extractor-tls" not found Jul 29 19:18:20.818305 ip-10-0-131-93 kubenswrapper[2564]: I0729 19:18:20.818277 2564 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-444jx_faa8c546-c75b-4e10-93b4-661aa2b5f53e/konnectivity-agent/0.log" limit=1024 Jul 29 19:18:22.619030 ip-10-0-131-93 kubenswrapper[2564]: I0729 19:18:22.619000 2564 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-444jx_faa8c546-c75b-4e10-93b4-661aa2b5f53e/konnectivity-agent/0.log" limit=1024 Jul 29 19:18:23.279311 ip-10-0-131-93 kubenswrapper[2564]: I0729 19:18:23.279277 2564 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-444jx_faa8c546-c75b-4e10-93b4-661aa2b5f53e/konnectivity-agent/0.log" limit=1024 Jul 29 19:18:23.381919 ip-10-0-131-93 kubenswrapper[2564]: I0729 19:18:23.381889 2564 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-444jx_faa8c546-c75b-4e10-93b4-661aa2b5f53e/konnectivity-agent/0.log" limit=1024 Jul 29 19:18:23.479866 ip-10-0-131-93 kubenswrapper[2564]: I0729 19:18:23.479838 2564 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-444jx_faa8c546-c75b-4e10-93b4-661aa2b5f53e/konnectivity-agent/0.log" limit=1024 Jul 29 19:18:23.547899 ip-10-0-131-93 kubenswrapper[2564]: I0729 19:18:23.547831 2564 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-444jx_faa8c546-c75b-4e10-93b4-661aa2b5f53e/konnectivity-agent/0.log" limit=1024 Jul 29 19:18:24.310666 ip-10-0-131-93 kubenswrapper[2564]: I0729 19:18:24.310632 2564 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-444jx_faa8c546-c75b-4e10-93b4-661aa2b5f53e/konnectivity-agent/0.log" limit=1024 Jul 29 19:18:24.908096 ip-10-0-131-93 kubenswrapper[2564]: I0729 19:18:24.908064 2564 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-444jx_faa8c546-c75b-4e10-93b4-661aa2b5f53e/konnectivity-agent/0.log" limit=1024 Jul 29 19:18:25.106227 ip-10-0-131-93 kubenswrapper[2564]: I0729 19:18:25.106199 2564 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-444jx_faa8c546-c75b-4e10-93b4-661aa2b5f53e/konnectivity-agent/0.log" limit=1024 Jul 29 19:18:25.314388 ip-10-0-131-93 kubenswrapper[2564]: I0729 19:18:25.314346 2564 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-444jx_faa8c546-c75b-4e10-93b4-661aa2b5f53e/konnectivity-agent/0.log" limit=1024 Jul 29 19:18:25.718564 ip-10-0-131-93 kubenswrapper[2564]: I0729 19:18:25.718541 2564 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-444jx_faa8c546-c75b-4e10-93b4-661aa2b5f53e/konnectivity-agent/0.log" limit=1024 Jul 29 19:18:26.190536 ip-10-0-131-93 kubenswrapper[2564]: I0729 19:18:26.190509 2564 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-444jx_faa8c546-c75b-4e10-93b4-661aa2b5f53e/konnectivity-agent/0.log" limit=1024 Jul 29 19:18:26.358425 ip-10-0-131-93 kubenswrapper[2564]: I0729 19:18:26.358395 2564 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-444jx_faa8c546-c75b-4e10-93b4-661aa2b5f53e/konnectivity-agent/0.log" limit=1024 Jul 29 19:18:26.432275 ip-10-0-131-93 kubenswrapper[2564]: I0729 19:18:26.432244 2564 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-444jx_faa8c546-c75b-4e10-93b4-661aa2b5f53e/konnectivity-agent/0.log" limit=1024 Jul 29 19:18:26.907460 ip-10-0-131-93 kubenswrapper[2564]: I0729 19:18:26.907429 2564 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-444jx_faa8c546-c75b-4e10-93b4-661aa2b5f53e/konnectivity-agent/0.log" limit=1024 Jul 29 19:18:27.150909 ip-10-0-131-93 kubenswrapper[2564]: I0729 19:18:27.150880 2564 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-444jx_faa8c546-c75b-4e10-93b4-661aa2b5f53e/konnectivity-agent/0.log" limit=1024 Jul 29 19:18:27.244265 ip-10-0-131-93 kubenswrapper[2564]: I0729 19:18:27.244239 2564 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-444jx_faa8c546-c75b-4e10-93b4-661aa2b5f53e/konnectivity-agent/0.log" limit=1024 Jul 29 19:18:27.694095 ip-10-0-131-93 kubenswrapper[2564]: I0729 19:18:27.694070 2564 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-444jx_faa8c546-c75b-4e10-93b4-661aa2b5f53e/konnectivity-agent/0.log" limit=1024 Jul 29 19:18:27.795333 ip-10-0-131-93 kubenswrapper[2564]: I0729 19:18:27.795306 2564 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-444jx_faa8c546-c75b-4e10-93b4-661aa2b5f53e/konnectivity-agent/0.log" limit=1024 Jul 29 19:18:28.326462 ip-10-0-131-93 kubenswrapper[2564]: I0729 19:18:28.326421 2564 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-444jx_faa8c546-c75b-4e10-93b4-661aa2b5f53e/konnectivity-agent/0.log" limit=1024 Jul 29 19:18:28.713005 ip-10-0-131-93 kubenswrapper[2564]: I0729 19:18:28.712978 2564 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-444jx_faa8c546-c75b-4e10-93b4-661aa2b5f53e/konnectivity-agent/0.log" limit=1024 Jul 29 19:18:28.773478 ip-10-0-131-93 kubenswrapper[2564]: I0729 19:18:28.773452 2564 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-rbac-proxy-cm\" (UniqueName: \"kubernetes.io/configmap/9887221a-35c3-4cb7-8f93-d03a8e1c0e8e-kube-rbac-proxy-cm\") pod \"insights-runtime-extractor-q4bqd\" (UID: \"9887221a-35c3-4cb7-8f93-d03a8e1c0e8e\") " pod="openshift-insights/insights-runtime-extractor-q4bqd" Jul 29 19:18:28.773556 ip-10-0-131-93 kubenswrapper[2564]: I0729 19:18:28.773531 2564 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"insights-runtime-extractor-tls\" (UniqueName: \"kubernetes.io/secret/9887221a-35c3-4cb7-8f93-d03a8e1c0e8e-insights-runtime-extractor-tls\") pod \"insights-runtime-extractor-q4bqd\" (UID: \"9887221a-35c3-4cb7-8f93-d03a8e1c0e8e\") " pod="openshift-insights/insights-runtime-extractor-q4bqd" Jul 29 19:18:28.773631 ip-10-0-131-93 kubenswrapper[2564]: E0729 19:18:28.773619 2564 secret.go:189] Couldn't get secret openshift-insights/insights-runtime-extractor-tls: secret "insights-runtime-extractor-tls" not found Jul 29 19:18:28.773689 ip-10-0-131-93 kubenswrapper[2564]: E0729 19:18:28.773680 2564 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/9887221a-35c3-4cb7-8f93-d03a8e1c0e8e-insights-runtime-extractor-tls podName:9887221a-35c3-4cb7-8f93-d03a8e1c0e8e nodeName:}" failed. No retries permitted until 2026-07-29 19:18:44.773665865 +0000 UTC m=+100.497725535 (durationBeforeRetry 16s). Error: MountVolume.SetUp failed for volume "insights-runtime-extractor-tls" (UniqueName: "kubernetes.io/secret/9887221a-35c3-4cb7-8f93-d03a8e1c0e8e-insights-runtime-extractor-tls") pod "insights-runtime-extractor-q4bqd" (UID: "9887221a-35c3-4cb7-8f93-d03a8e1c0e8e") : secret "insights-runtime-extractor-tls" not found Jul 29 19:18:28.773947 ip-10-0-131-93 kubenswrapper[2564]: I0729 19:18:28.773927 2564 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-rbac-proxy-cm\" (UniqueName: \"kubernetes.io/configmap/9887221a-35c3-4cb7-8f93-d03a8e1c0e8e-kube-rbac-proxy-cm\") pod \"insights-runtime-extractor-q4bqd\" (UID: \"9887221a-35c3-4cb7-8f93-d03a8e1c0e8e\") " pod="openshift-insights/insights-runtime-extractor-q4bqd" Jul 29 19:18:28.918329 ip-10-0-131-93 kubenswrapper[2564]: I0729 19:18:28.918296 2564 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-444jx_faa8c546-c75b-4e10-93b4-661aa2b5f53e/konnectivity-agent/0.log" limit=1024 Jul 29 19:18:29.120260 ip-10-0-131-93 kubenswrapper[2564]: I0729 19:18:29.120177 2564 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-444jx_faa8c546-c75b-4e10-93b4-661aa2b5f53e/konnectivity-agent/0.log" limit=1024 Jul 29 19:18:29.515081 ip-10-0-131-93 kubenswrapper[2564]: I0729 19:18:29.515048 2564 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-444jx_faa8c546-c75b-4e10-93b4-661aa2b5f53e/konnectivity-agent/0.log" limit=1024 Jul 29 19:18:29.589412 ip-10-0-131-93 kubenswrapper[2564]: I0729 19:18:29.589384 2564 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-444jx_faa8c546-c75b-4e10-93b4-661aa2b5f53e/konnectivity-agent/0.log" limit=1024 Jul 29 19:18:29.709695 ip-10-0-131-93 kubenswrapper[2564]: I0729 19:18:29.709665 2564 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-444jx_faa8c546-c75b-4e10-93b4-661aa2b5f53e/konnectivity-agent/0.log" limit=1024 Jul 29 19:18:29.859309 ip-10-0-131-93 kubenswrapper[2564]: I0729 19:18:29.859230 2564 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-444jx_faa8c546-c75b-4e10-93b4-661aa2b5f53e/konnectivity-agent/0.log" limit=1024 Jul 29 19:18:29.962912 ip-10-0-131-93 kubenswrapper[2564]: I0729 19:18:29.962880 2564 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-444jx_faa8c546-c75b-4e10-93b4-661aa2b5f53e/konnectivity-agent/0.log" limit=1024 Jul 29 19:18:30.113599 ip-10-0-131-93 kubenswrapper[2564]: I0729 19:18:30.113519 2564 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-444jx_faa8c546-c75b-4e10-93b4-661aa2b5f53e/konnectivity-agent/0.log" limit=1024 Jul 29 19:18:30.706830 ip-10-0-131-93 kubenswrapper[2564]: I0729 19:18:30.706800 2564 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-444jx_faa8c546-c75b-4e10-93b4-661aa2b5f53e/konnectivity-agent/0.log" limit=1024 Jul 29 19:18:31.107801 ip-10-0-131-93 kubenswrapper[2564]: I0729 19:18:31.107725 2564 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-444jx_faa8c546-c75b-4e10-93b4-661aa2b5f53e/konnectivity-agent/0.log" limit=1024 Jul 29 19:18:31.535282 ip-10-0-131-93 kubenswrapper[2564]: I0729 19:18:31.535248 2564 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-444jx_faa8c546-c75b-4e10-93b4-661aa2b5f53e/konnectivity-agent/0.log" limit=1024 Jul 29 19:18:31.916344 ip-10-0-131-93 kubenswrapper[2564]: I0729 19:18:31.916265 2564 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-444jx_faa8c546-c75b-4e10-93b4-661aa2b5f53e/konnectivity-agent/0.log" limit=1024 Jul 29 19:18:32.305868 ip-10-0-131-93 kubenswrapper[2564]: I0729 19:18:32.305833 2564 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-444jx_faa8c546-c75b-4e10-93b4-661aa2b5f53e/konnectivity-agent/0.log" limit=1024 Jul 29 19:18:32.717179 ip-10-0-131-93 kubenswrapper[2564]: I0729 19:18:32.717147 2564 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-444jx_faa8c546-c75b-4e10-93b4-661aa2b5f53e/konnectivity-agent/0.log" limit=1024 Jul 29 19:18:32.913591 ip-10-0-131-93 kubenswrapper[2564]: I0729 19:18:32.913549 2564 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-444jx_faa8c546-c75b-4e10-93b4-661aa2b5f53e/konnectivity-agent/0.log" limit=1024 Jul 29 19:18:33.294137 ip-10-0-131-93 kubenswrapper[2564]: I0729 19:18:33.294109 2564 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-444jx_faa8c546-c75b-4e10-93b4-661aa2b5f53e/konnectivity-agent/0.log" limit=1024 Jul 29 19:18:33.395219 ip-10-0-131-93 kubenswrapper[2564]: I0729 19:18:33.395193 2564 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-444jx_faa8c546-c75b-4e10-93b4-661aa2b5f53e/konnectivity-agent/0.log" limit=1024 Jul 29 19:18:33.496945 ip-10-0-131-93 kubenswrapper[2564]: I0729 19:18:33.496916 2564 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-444jx_faa8c546-c75b-4e10-93b4-661aa2b5f53e/konnectivity-agent/0.log" limit=1024 Jul 29 19:18:33.595637 ip-10-0-131-93 kubenswrapper[2564]: I0729 19:18:33.595560 2564 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-444jx_faa8c546-c75b-4e10-93b4-661aa2b5f53e/konnectivity-agent/0.log" limit=1024 Jul 29 19:18:33.663234 ip-10-0-131-93 kubenswrapper[2564]: I0729 19:18:33.663205 2564 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-444jx_faa8c546-c75b-4e10-93b4-661aa2b5f53e/konnectivity-agent/0.log" limit=1024 Jul 29 19:18:33.779339 ip-10-0-131-93 kubenswrapper[2564]: I0729 19:18:33.779310 2564 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-444jx_faa8c546-c75b-4e10-93b4-661aa2b5f53e/konnectivity-agent/0.log" limit=1024 Jul 29 19:18:33.854839 ip-10-0-131-93 kubenswrapper[2564]: I0729 19:18:33.854769 2564 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-444jx_faa8c546-c75b-4e10-93b4-661aa2b5f53e/konnectivity-agent/0.log" limit=1024 Jul 29 19:18:34.177051 ip-10-0-131-93 kubenswrapper[2564]: I0729 19:18:34.176967 2564 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-444jx_faa8c546-c75b-4e10-93b4-661aa2b5f53e/konnectivity-agent/0.log" limit=1024 Jul 29 19:18:34.239299 ip-10-0-131-93 kubenswrapper[2564]: I0729 19:18:34.239271 2564 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-444jx_faa8c546-c75b-4e10-93b4-661aa2b5f53e/konnectivity-agent/0.log" limit=1024 Jul 29 19:18:34.375680 ip-10-0-131-93 kubenswrapper[2564]: I0729 19:18:34.375649 2564 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-444jx_faa8c546-c75b-4e10-93b4-661aa2b5f53e/konnectivity-agent/0.log" limit=1024 Jul 29 19:18:34.445471 ip-10-0-131-93 kubenswrapper[2564]: I0729 19:18:34.445441 2564 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-444jx_faa8c546-c75b-4e10-93b4-661aa2b5f53e/konnectivity-agent/0.log" limit=1024 Jul 29 19:18:34.781708 ip-10-0-131-93 kubenswrapper[2564]: I0729 19:18:34.781627 2564 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-444jx_faa8c546-c75b-4e10-93b4-661aa2b5f53e/konnectivity-agent/0.log" limit=1024 Jul 29 19:18:34.884308 ip-10-0-131-93 kubenswrapper[2564]: I0729 19:18:34.884282 2564 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-444jx_faa8c546-c75b-4e10-93b4-661aa2b5f53e/konnectivity-agent/0.log" limit=1024 Jul 29 19:18:34.984480 ip-10-0-131-93 kubenswrapper[2564]: I0729 19:18:34.984447 2564 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-444jx_faa8c546-c75b-4e10-93b4-661aa2b5f53e/konnectivity-agent/0.log" limit=1024 Jul 29 19:18:35.208876 ip-10-0-131-93 kubenswrapper[2564]: I0729 19:18:35.208853 2564 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-444jx_faa8c546-c75b-4e10-93b4-661aa2b5f53e/konnectivity-agent/0.log" limit=1024 Jul 29 19:18:36.881503 ip-10-0-131-93 kubenswrapper[2564]: I0729 19:18:36.881473 2564 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-444jx_faa8c546-c75b-4e10-93b4-661aa2b5f53e/konnectivity-agent/0.log" limit=1024 Jul 29 19:18:36.981717 ip-10-0-131-93 kubenswrapper[2564]: I0729 19:18:36.981685 2564 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-444jx_faa8c546-c75b-4e10-93b4-661aa2b5f53e/konnectivity-agent/0.log" limit=1024 Jul 29 19:18:39.621461 ip-10-0-131-93 kubenswrapper[2564]: I0729 19:18:39.621429 2564 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["open-cluster-management-agent-addon/cluster-proxy-proxy-agent-6dccbbc475-kzvq7"] Jul 29 19:18:39.624280 ip-10-0-131-93 kubenswrapper[2564]: I0729 19:18:39.624260 2564 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="open-cluster-management-agent-addon/cluster-proxy-proxy-agent-6dccbbc475-kzvq7" Jul 29 19:18:39.630587 ip-10-0-131-93 kubenswrapper[2564]: I0729 19:18:39.630567 2564 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"open-cluster-management-agent-addon\"/\"openshift-service-ca.crt\"" Jul 29 19:18:39.630696 ip-10-0-131-93 kubenswrapper[2564]: I0729 19:18:39.630605 2564 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"open-cluster-management-agent-addon\"/\"cluster-proxy-open-cluster-management.io-proxy-agent-signer-client-cert\"" Jul 29 19:18:39.631266 ip-10-0-131-93 kubenswrapper[2564]: I0729 19:18:39.631244 2564 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"open-cluster-management-agent-addon\"/\"cluster-proxy-ca\"" Jul 29 19:18:39.631363 ip-10-0-131-93 kubenswrapper[2564]: I0729 19:18:39.631320 2564 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"open-cluster-management-agent-addon\"/\"kube-root-ca.crt\"" Jul 29 19:18:39.631363 ip-10-0-131-93 kubenswrapper[2564]: I0729 19:18:39.631335 2564 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"open-cluster-management-agent-addon\"/\"open-cluster-management-image-pull-credentials\"" Jul 29 19:18:39.631363 ip-10-0-131-93 kubenswrapper[2564]: I0729 19:18:39.631347 2564 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"open-cluster-management-agent-addon\"/\"cluster-proxy-service-proxy-server-certificates\"" Jul 29 19:18:39.631552 ip-10-0-131-93 kubenswrapper[2564]: I0729 19:18:39.631335 2564 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"open-cluster-management-agent-addon\"/\"cluster-proxy-hub-kubeconfig\"" Jul 29 19:18:39.640164 ip-10-0-131-93 kubenswrapper[2564]: I0729 19:18:39.640142 2564 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["open-cluster-management-agent-addon/cluster-proxy-proxy-agent-6dccbbc475-kzvq7"] Jul 29 19:18:39.726434 ip-10-0-131-93 kubenswrapper[2564]: I0729 19:18:39.726399 2564 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-444jx_faa8c546-c75b-4e10-93b4-661aa2b5f53e/konnectivity-agent/0.log" limit=1024 Jul 29 19:18:39.761580 ip-10-0-131-93 kubenswrapper[2564]: I0729 19:18:39.761545 2564 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"ca\" (UniqueName: \"kubernetes.io/secret/f6de805a-eafb-45e7-bcc8-f55d134227b5-ca\") pod \"cluster-proxy-proxy-agent-6dccbbc475-kzvq7\" (UID: \"f6de805a-eafb-45e7-bcc8-f55d134227b5\") " pod="open-cluster-management-agent-addon/cluster-proxy-proxy-agent-6dccbbc475-kzvq7" Jul 29 19:18:39.761580 ip-10-0-131-93 kubenswrapper[2564]: I0729 19:18:39.761580 2564 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"additional-service-ca\" (UniqueName: \"kubernetes.io/configmap/f6de805a-eafb-45e7-bcc8-f55d134227b5-additional-service-ca\") pod \"cluster-proxy-proxy-agent-6dccbbc475-kzvq7\" (UID: \"f6de805a-eafb-45e7-bcc8-f55d134227b5\") " pod="open-cluster-management-agent-addon/cluster-proxy-proxy-agent-6dccbbc475-kzvq7" Jul 29 19:18:39.761793 ip-10-0-131-93 kubenswrapper[2564]: I0729 19:18:39.761649 2564 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"hub-kubeconfig\" (UniqueName: \"kubernetes.io/secret/f6de805a-eafb-45e7-bcc8-f55d134227b5-hub-kubeconfig\") pod \"cluster-proxy-proxy-agent-6dccbbc475-kzvq7\" (UID: \"f6de805a-eafb-45e7-bcc8-f55d134227b5\") " pod="open-cluster-management-agent-addon/cluster-proxy-proxy-agent-6dccbbc475-kzvq7" Jul 29 19:18:39.761793 ip-10-0-131-93 kubenswrapper[2564]: I0729 19:18:39.761726 2564 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-4h25g\" (UniqueName: \"kubernetes.io/projected/f6de805a-eafb-45e7-bcc8-f55d134227b5-kube-api-access-4h25g\") pod \"cluster-proxy-proxy-agent-6dccbbc475-kzvq7\" (UID: \"f6de805a-eafb-45e7-bcc8-f55d134227b5\") " pod="open-cluster-management-agent-addon/cluster-proxy-proxy-agent-6dccbbc475-kzvq7" Jul 29 19:18:39.761793 ip-10-0-131-93 kubenswrapper[2564]: I0729 19:18:39.761747 2564 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"hub\" (UniqueName: \"kubernetes.io/secret/f6de805a-eafb-45e7-bcc8-f55d134227b5-hub\") pod \"cluster-proxy-proxy-agent-6dccbbc475-kzvq7\" (UID: \"f6de805a-eafb-45e7-bcc8-f55d134227b5\") " pod="open-cluster-management-agent-addon/cluster-proxy-proxy-agent-6dccbbc475-kzvq7" Jul 29 19:18:39.761793 ip-10-0-131-93 kubenswrapper[2564]: I0729 19:18:39.761770 2564 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"service-proxy-server-cert\" (UniqueName: \"kubernetes.io/secret/f6de805a-eafb-45e7-bcc8-f55d134227b5-service-proxy-server-cert\") pod \"cluster-proxy-proxy-agent-6dccbbc475-kzvq7\" (UID: \"f6de805a-eafb-45e7-bcc8-f55d134227b5\") " pod="open-cluster-management-agent-addon/cluster-proxy-proxy-agent-6dccbbc475-kzvq7" Jul 29 19:18:39.831744 ip-10-0-131-93 kubenswrapper[2564]: I0729 19:18:39.831716 2564 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-444jx_faa8c546-c75b-4e10-93b4-661aa2b5f53e/konnectivity-agent/0.log" limit=1024 Jul 29 19:18:39.862850 ip-10-0-131-93 kubenswrapper[2564]: I0729 19:18:39.862817 2564 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-4h25g\" (UniqueName: \"kubernetes.io/projected/f6de805a-eafb-45e7-bcc8-f55d134227b5-kube-api-access-4h25g\") pod \"cluster-proxy-proxy-agent-6dccbbc475-kzvq7\" (UID: \"f6de805a-eafb-45e7-bcc8-f55d134227b5\") " pod="open-cluster-management-agent-addon/cluster-proxy-proxy-agent-6dccbbc475-kzvq7" Jul 29 19:18:39.862961 ip-10-0-131-93 kubenswrapper[2564]: I0729 19:18:39.862858 2564 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"hub\" (UniqueName: \"kubernetes.io/secret/f6de805a-eafb-45e7-bcc8-f55d134227b5-hub\") pod \"cluster-proxy-proxy-agent-6dccbbc475-kzvq7\" (UID: \"f6de805a-eafb-45e7-bcc8-f55d134227b5\") " pod="open-cluster-management-agent-addon/cluster-proxy-proxy-agent-6dccbbc475-kzvq7" Jul 29 19:18:39.862961 ip-10-0-131-93 kubenswrapper[2564]: I0729 19:18:39.862875 2564 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"service-proxy-server-cert\" (UniqueName: \"kubernetes.io/secret/f6de805a-eafb-45e7-bcc8-f55d134227b5-service-proxy-server-cert\") pod \"cluster-proxy-proxy-agent-6dccbbc475-kzvq7\" (UID: \"f6de805a-eafb-45e7-bcc8-f55d134227b5\") " pod="open-cluster-management-agent-addon/cluster-proxy-proxy-agent-6dccbbc475-kzvq7" Jul 29 19:18:39.862961 ip-10-0-131-93 kubenswrapper[2564]: I0729 19:18:39.862924 2564 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"ca\" (UniqueName: \"kubernetes.io/secret/f6de805a-eafb-45e7-bcc8-f55d134227b5-ca\") pod \"cluster-proxy-proxy-agent-6dccbbc475-kzvq7\" (UID: \"f6de805a-eafb-45e7-bcc8-f55d134227b5\") " pod="open-cluster-management-agent-addon/cluster-proxy-proxy-agent-6dccbbc475-kzvq7" Jul 29 19:18:39.863089 ip-10-0-131-93 kubenswrapper[2564]: I0729 19:18:39.862951 2564 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"additional-service-ca\" (UniqueName: \"kubernetes.io/configmap/f6de805a-eafb-45e7-bcc8-f55d134227b5-additional-service-ca\") pod \"cluster-proxy-proxy-agent-6dccbbc475-kzvq7\" (UID: \"f6de805a-eafb-45e7-bcc8-f55d134227b5\") " pod="open-cluster-management-agent-addon/cluster-proxy-proxy-agent-6dccbbc475-kzvq7" Jul 29 19:18:39.863089 ip-10-0-131-93 kubenswrapper[2564]: I0729 19:18:39.863019 2564 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"hub-kubeconfig\" (UniqueName: \"kubernetes.io/secret/f6de805a-eafb-45e7-bcc8-f55d134227b5-hub-kubeconfig\") pod \"cluster-proxy-proxy-agent-6dccbbc475-kzvq7\" (UID: \"f6de805a-eafb-45e7-bcc8-f55d134227b5\") " pod="open-cluster-management-agent-addon/cluster-proxy-proxy-agent-6dccbbc475-kzvq7" Jul 29 19:18:39.863808 ip-10-0-131-93 kubenswrapper[2564]: I0729 19:18:39.863780 2564 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"additional-service-ca\" (UniqueName: \"kubernetes.io/configmap/f6de805a-eafb-45e7-bcc8-f55d134227b5-additional-service-ca\") pod \"cluster-proxy-proxy-agent-6dccbbc475-kzvq7\" (UID: \"f6de805a-eafb-45e7-bcc8-f55d134227b5\") " pod="open-cluster-management-agent-addon/cluster-proxy-proxy-agent-6dccbbc475-kzvq7" Jul 29 19:18:39.865549 ip-10-0-131-93 kubenswrapper[2564]: I0729 19:18:39.865519 2564 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"ca\" (UniqueName: \"kubernetes.io/secret/f6de805a-eafb-45e7-bcc8-f55d134227b5-ca\") pod \"cluster-proxy-proxy-agent-6dccbbc475-kzvq7\" (UID: \"f6de805a-eafb-45e7-bcc8-f55d134227b5\") " pod="open-cluster-management-agent-addon/cluster-proxy-proxy-agent-6dccbbc475-kzvq7" Jul 29 19:18:39.865549 ip-10-0-131-93 kubenswrapper[2564]: I0729 19:18:39.865539 2564 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"hub\" (UniqueName: \"kubernetes.io/secret/f6de805a-eafb-45e7-bcc8-f55d134227b5-hub\") pod \"cluster-proxy-proxy-agent-6dccbbc475-kzvq7\" (UID: \"f6de805a-eafb-45e7-bcc8-f55d134227b5\") " pod="open-cluster-management-agent-addon/cluster-proxy-proxy-agent-6dccbbc475-kzvq7" Jul 29 19:18:39.865725 ip-10-0-131-93 kubenswrapper[2564]: I0729 19:18:39.865706 2564 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"service-proxy-server-cert\" (UniqueName: \"kubernetes.io/secret/f6de805a-eafb-45e7-bcc8-f55d134227b5-service-proxy-server-cert\") pod \"cluster-proxy-proxy-agent-6dccbbc475-kzvq7\" (UID: \"f6de805a-eafb-45e7-bcc8-f55d134227b5\") " pod="open-cluster-management-agent-addon/cluster-proxy-proxy-agent-6dccbbc475-kzvq7" Jul 29 19:18:39.865768 ip-10-0-131-93 kubenswrapper[2564]: I0729 19:18:39.865738 2564 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"hub-kubeconfig\" (UniqueName: \"kubernetes.io/secret/f6de805a-eafb-45e7-bcc8-f55d134227b5-hub-kubeconfig\") pod \"cluster-proxy-proxy-agent-6dccbbc475-kzvq7\" (UID: \"f6de805a-eafb-45e7-bcc8-f55d134227b5\") " pod="open-cluster-management-agent-addon/cluster-proxy-proxy-agent-6dccbbc475-kzvq7" Jul 29 19:18:39.872258 ip-10-0-131-93 kubenswrapper[2564]: I0729 19:18:39.872207 2564 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-4h25g\" (UniqueName: \"kubernetes.io/projected/f6de805a-eafb-45e7-bcc8-f55d134227b5-kube-api-access-4h25g\") pod \"cluster-proxy-proxy-agent-6dccbbc475-kzvq7\" (UID: \"f6de805a-eafb-45e7-bcc8-f55d134227b5\") " pod="open-cluster-management-agent-addon/cluster-proxy-proxy-agent-6dccbbc475-kzvq7" Jul 29 19:18:39.946261 ip-10-0-131-93 kubenswrapper[2564]: I0729 19:18:39.946229 2564 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="open-cluster-management-agent-addon/cluster-proxy-proxy-agent-6dccbbc475-kzvq7" Jul 29 19:18:40.071395 ip-10-0-131-93 kubenswrapper[2564]: I0729 19:18:40.071353 2564 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["open-cluster-management-agent-addon/cluster-proxy-proxy-agent-6dccbbc475-kzvq7"] Jul 29 19:18:40.074847 ip-10-0-131-93 kubenswrapper[2564]: W0729 19:18:40.074813 2564 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-besteffort.slice/kubepods-besteffort-podf6de805a_eafb_45e7_bcc8_f55d134227b5.slice/crio-4aa47ba65e29378c1e8570e1438b9b66b849d064e201a7e0613383009da65091 WatchSource:0}: Error finding container 4aa47ba65e29378c1e8570e1438b9b66b849d064e201a7e0613383009da65091: Status 404 returned error can't find the container with id 4aa47ba65e29378c1e8570e1438b9b66b849d064e201a7e0613383009da65091 Jul 29 19:18:40.109865 ip-10-0-131-93 kubenswrapper[2564]: I0729 19:18:40.109829 2564 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="open-cluster-management-agent-addon/cluster-proxy-proxy-agent-6dccbbc475-kzvq7" event={"ID":"f6de805a-eafb-45e7-bcc8-f55d134227b5","Type":"ContainerStarted","Data":"4aa47ba65e29378c1e8570e1438b9b66b849d064e201a7e0613383009da65091"} Jul 29 19:18:42.585614 ip-10-0-131-93 kubenswrapper[2564]: I0729 19:18:42.585576 2564 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"registry-tls\" (UniqueName: \"kubernetes.io/projected/5ff17af5-17c3-45e6-b47f-408ba15ed5ca-registry-tls\") pod \"image-registry-7f95c97455-tkxb8\" (UID: \"5ff17af5-17c3-45e6-b47f-408ba15ed5ca\") " pod="openshift-image-registry/image-registry-7f95c97455-tkxb8" Jul 29 19:18:42.586087 ip-10-0-131-93 kubenswrapper[2564]: I0729 19:18:42.585657 2564 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-tls\" (UniqueName: \"kubernetes.io/secret/d5e5a7da-26f5-4d8d-8e9e-4e8991392e8e-metrics-tls\") pod \"dns-default-kkmn5\" (UID: \"d5e5a7da-26f5-4d8d-8e9e-4e8991392e8e\") " pod="openshift-dns/dns-default-kkmn5" Jul 29 19:18:42.588546 ip-10-0-131-93 kubenswrapper[2564]: I0729 19:18:42.588522 2564 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"registry-tls\" (UniqueName: \"kubernetes.io/projected/5ff17af5-17c3-45e6-b47f-408ba15ed5ca-registry-tls\") pod \"image-registry-7f95c97455-tkxb8\" (UID: \"5ff17af5-17c3-45e6-b47f-408ba15ed5ca\") " pod="openshift-image-registry/image-registry-7f95c97455-tkxb8" Jul 29 19:18:42.588546 ip-10-0-131-93 kubenswrapper[2564]: I0729 19:18:42.588533 2564 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"metrics-tls\" (UniqueName: \"kubernetes.io/secret/d5e5a7da-26f5-4d8d-8e9e-4e8991392e8e-metrics-tls\") pod \"dns-default-kkmn5\" (UID: \"d5e5a7da-26f5-4d8d-8e9e-4e8991392e8e\") " pod="openshift-dns/dns-default-kkmn5" Jul 29 19:18:42.833981 ip-10-0-131-93 kubenswrapper[2564]: I0729 19:18:42.833943 2564 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-image-registry\"/\"registry-dockercfg-k7njp\"" Jul 29 19:18:42.842295 ip-10-0-131-93 kubenswrapper[2564]: I0729 19:18:42.842242 2564 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-image-registry/image-registry-7f95c97455-tkxb8" Jul 29 19:18:42.891433 ip-10-0-131-93 kubenswrapper[2564]: I0729 19:18:42.891398 2564 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-dns\"/\"dns-dockercfg-74g8d\"" Jul 29 19:18:42.899280 ip-10-0-131-93 kubenswrapper[2564]: I0729 19:18:42.899243 2564 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-dns/dns-default-kkmn5" Jul 29 19:18:42.990784 ip-10-0-131-93 kubenswrapper[2564]: I0729 19:18:42.990754 2564 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-image-registry/image-registry-7f95c97455-tkxb8"] Jul 29 19:18:43.282091 ip-10-0-131-93 kubenswrapper[2564]: I0729 19:18:43.282052 2564 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-444jx_faa8c546-c75b-4e10-93b4-661aa2b5f53e/konnectivity-agent/0.log" limit=1024 Jul 29 19:18:43.383136 ip-10-0-131-93 kubenswrapper[2564]: I0729 19:18:43.383090 2564 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-444jx_faa8c546-c75b-4e10-93b4-661aa2b5f53e/konnectivity-agent/0.log" limit=1024 Jul 29 19:18:44.008368 ip-10-0-131-93 kubenswrapper[2564]: W0729 19:18:44.008322 2564 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod5ff17af5_17c3_45e6_b47f_408ba15ed5ca.slice/crio-ed8d5f94988d01ddbf2469045e30a230a3425dfef797c7e67b3bf4af4a9e3293 WatchSource:0}: Error finding container ed8d5f94988d01ddbf2469045e30a230a3425dfef797c7e67b3bf4af4a9e3293: Status 404 returned error can't find the container with id ed8d5f94988d01ddbf2469045e30a230a3425dfef797c7e67b3bf4af4a9e3293 Jul 29 19:18:44.064406 ip-10-0-131-93 kubenswrapper[2564]: I0729 19:18:44.064360 2564 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-444jx_faa8c546-c75b-4e10-93b4-661aa2b5f53e/konnectivity-agent/0.log" limit=1024 Jul 29 19:18:44.125344 ip-10-0-131-93 kubenswrapper[2564]: I0729 19:18:44.125311 2564 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-image-registry/image-registry-7f95c97455-tkxb8" event={"ID":"5ff17af5-17c3-45e6-b47f-408ba15ed5ca","Type":"ContainerStarted","Data":"7537ab80628a911b100f9f02e182cec7147c9e87cf1063b4425e9d06a18ba234"} Jul 29 19:18:44.125476 ip-10-0-131-93 kubenswrapper[2564]: I0729 19:18:44.125368 2564 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-image-registry/image-registry-7f95c97455-tkxb8" event={"ID":"5ff17af5-17c3-45e6-b47f-408ba15ed5ca","Type":"ContainerStarted","Data":"ed8d5f94988d01ddbf2469045e30a230a3425dfef797c7e67b3bf4af4a9e3293"} Jul 29 19:18:44.125476 ip-10-0-131-93 kubenswrapper[2564]: I0729 19:18:44.125426 2564 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-image-registry/image-registry-7f95c97455-tkxb8" Jul 29 19:18:44.137675 ip-10-0-131-93 kubenswrapper[2564]: I0729 19:18:44.137654 2564 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-dns/dns-default-kkmn5"] Jul 29 19:18:44.141554 ip-10-0-131-93 kubenswrapper[2564]: I0729 19:18:44.141498 2564 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-444jx_faa8c546-c75b-4e10-93b4-661aa2b5f53e/konnectivity-agent/0.log" limit=1024 Jul 29 19:18:44.141781 ip-10-0-131-93 kubenswrapper[2564]: W0729 19:18:44.141752 2564 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podd5e5a7da_26f5_4d8d_8e9e_4e8991392e8e.slice/crio-e8d2e20f0d646dbfff021405817b339903033d1596a8cc04359f316b895b99a7 WatchSource:0}: Error finding container e8d2e20f0d646dbfff021405817b339903033d1596a8cc04359f316b895b99a7: Status 404 returned error can't find the container with id e8d2e20f0d646dbfff021405817b339903033d1596a8cc04359f316b895b99a7 Jul 29 19:18:44.148205 ip-10-0-131-93 kubenswrapper[2564]: I0729 19:18:44.148154 2564 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-image-registry/image-registry-7f95c97455-tkxb8" podStartSLOduration=99.148133313 podStartE2EDuration="1m39.148133313s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-29 19:17:05 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-29 19:18:44.146142924 +0000 UTC m=+99.870202615" watchObservedRunningTime="2026-07-29 19:18:44.148133313 +0000 UTC m=+99.872193007" Jul 29 19:18:44.334267 ip-10-0-131-93 kubenswrapper[2564]: I0729 19:18:44.334197 2564 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-444jx_faa8c546-c75b-4e10-93b4-661aa2b5f53e/konnectivity-agent/0.log" limit=1024 Jul 29 19:18:44.437719 ip-10-0-131-93 kubenswrapper[2564]: I0729 19:18:44.437683 2564 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-444jx_faa8c546-c75b-4e10-93b4-661aa2b5f53e/konnectivity-agent/0.log" limit=1024 Jul 29 19:18:44.537027 ip-10-0-131-93 kubenswrapper[2564]: I0729 19:18:44.536992 2564 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-444jx_faa8c546-c75b-4e10-93b4-661aa2b5f53e/konnectivity-agent/0.log" limit=1024 Jul 29 19:18:44.619267 ip-10-0-131-93 kubenswrapper[2564]: I0729 19:18:44.619184 2564 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-444jx_faa8c546-c75b-4e10-93b4-661aa2b5f53e/konnectivity-agent/0.log" limit=1024 Jul 29 19:18:44.802060 ip-10-0-131-93 kubenswrapper[2564]: I0729 19:18:44.802029 2564 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"insights-runtime-extractor-tls\" (UniqueName: \"kubernetes.io/secret/9887221a-35c3-4cb7-8f93-d03a8e1c0e8e-insights-runtime-extractor-tls\") pod \"insights-runtime-extractor-q4bqd\" (UID: \"9887221a-35c3-4cb7-8f93-d03a8e1c0e8e\") " pod="openshift-insights/insights-runtime-extractor-q4bqd" Jul 29 19:18:44.804901 ip-10-0-131-93 kubenswrapper[2564]: I0729 19:18:44.804873 2564 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"insights-runtime-extractor-tls\" (UniqueName: \"kubernetes.io/secret/9887221a-35c3-4cb7-8f93-d03a8e1c0e8e-insights-runtime-extractor-tls\") pod \"insights-runtime-extractor-q4bqd\" (UID: \"9887221a-35c3-4cb7-8f93-d03a8e1c0e8e\") " pod="openshift-insights/insights-runtime-extractor-q4bqd" Jul 29 19:18:44.815819 ip-10-0-131-93 kubenswrapper[2564]: I0729 19:18:44.815799 2564 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-insights/insights-runtime-extractor-q4bqd" Jul 29 19:18:44.961610 ip-10-0-131-93 kubenswrapper[2564]: I0729 19:18:44.961576 2564 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-insights/insights-runtime-extractor-q4bqd"] Jul 29 19:18:44.964831 ip-10-0-131-93 kubenswrapper[2564]: W0729 19:18:44.964802 2564 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod9887221a_35c3_4cb7_8f93_d03a8e1c0e8e.slice/crio-1da2e6dc3f38d44633ec25f44c8d826d8f868886bf3cb8bff1b78eaa20f454b0 WatchSource:0}: Error finding container 1da2e6dc3f38d44633ec25f44c8d826d8f868886bf3cb8bff1b78eaa20f454b0: Status 404 returned error can't find the container with id 1da2e6dc3f38d44633ec25f44c8d826d8f868886bf3cb8bff1b78eaa20f454b0 Jul 29 19:18:44.980097 ip-10-0-131-93 kubenswrapper[2564]: I0729 19:18:44.980073 2564 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-444jx_faa8c546-c75b-4e10-93b4-661aa2b5f53e/konnectivity-agent/0.log" limit=1024 Jul 29 19:18:45.039570 ip-10-0-131-93 kubenswrapper[2564]: I0729 19:18:45.039542 2564 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-network-diagnostics/network-check-target-464tm" Jul 29 19:18:45.129804 ip-10-0-131-93 kubenswrapper[2564]: I0729 19:18:45.129707 2564 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-insights/insights-runtime-extractor-q4bqd" event={"ID":"9887221a-35c3-4cb7-8f93-d03a8e1c0e8e","Type":"ContainerStarted","Data":"1da2e6dc3f38d44633ec25f44c8d826d8f868886bf3cb8bff1b78eaa20f454b0"} Jul 29 19:18:45.131930 ip-10-0-131-93 kubenswrapper[2564]: I0729 19:18:45.131901 2564 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="open-cluster-management-agent-addon/cluster-proxy-proxy-agent-6dccbbc475-kzvq7" event={"ID":"f6de805a-eafb-45e7-bcc8-f55d134227b5","Type":"ContainerStarted","Data":"44b217a4210cf579961d07933c8c233200c9f98e4fab8b63af73127e0f0dcd8b"} Jul 29 19:18:45.132066 ip-10-0-131-93 kubenswrapper[2564]: I0729 19:18:45.131944 2564 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="open-cluster-management-agent-addon/cluster-proxy-proxy-agent-6dccbbc475-kzvq7" event={"ID":"f6de805a-eafb-45e7-bcc8-f55d134227b5","Type":"ContainerStarted","Data":"a9864f5257914508d7bfbc4dff615134b9ad0921c10a7af2b63bb26a182b8aaf"} Jul 29 19:18:45.132066 ip-10-0-131-93 kubenswrapper[2564]: I0729 19:18:45.131965 2564 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="open-cluster-management-agent-addon/cluster-proxy-proxy-agent-6dccbbc475-kzvq7" event={"ID":"f6de805a-eafb-45e7-bcc8-f55d134227b5","Type":"ContainerStarted","Data":"f323a76a2c7808c390ae1fb870342d1459b9994c75ef66fb4323337a3d9fc97f"} Jul 29 19:18:45.133094 ip-10-0-131-93 kubenswrapper[2564]: I0729 19:18:45.133068 2564 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-dns/dns-default-kkmn5" event={"ID":"d5e5a7da-26f5-4d8d-8e9e-4e8991392e8e","Type":"ContainerStarted","Data":"e8d2e20f0d646dbfff021405817b339903033d1596a8cc04359f316b895b99a7"} Jul 29 19:18:45.153014 ip-10-0-131-93 kubenswrapper[2564]: I0729 19:18:45.152953 2564 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="open-cluster-management-agent-addon/cluster-proxy-proxy-agent-6dccbbc475-kzvq7" podStartSLOduration=2.163142611 podStartE2EDuration="6.152936311s" totalImagesPullingTime="3.9897937s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-29 19:18:39 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-29 19:18:45.151987136 +0000 UTC m=+100.876046828" watchObservedRunningTime="2026-07-29 19:18:45.152936311 +0000 UTC m=+100.876995980" Jul 29 19:18:46.081461 ip-10-0-131-93 kubenswrapper[2564]: I0729 19:18:46.081433 2564 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-444jx_faa8c546-c75b-4e10-93b4-661aa2b5f53e/konnectivity-agent/0.log" limit=1024 Jul 29 19:18:46.138295 ip-10-0-131-93 kubenswrapper[2564]: I0729 19:18:46.138257 2564 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-dns/dns-default-kkmn5" event={"ID":"d5e5a7da-26f5-4d8d-8e9e-4e8991392e8e","Type":"ContainerStarted","Data":"279ecf4fa2cd042eb70fae16cc822e8a1133ce8675047059c45ced5eeda77fba"} Jul 29 19:18:46.182196 ip-10-0-131-93 kubenswrapper[2564]: I0729 19:18:46.182139 2564 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-444jx_faa8c546-c75b-4e10-93b4-661aa2b5f53e/konnectivity-agent/0.log" limit=1024 Jul 29 19:18:47.144208 ip-10-0-131-93 kubenswrapper[2564]: I0729 19:18:47.144171 2564 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-dns/dns-default-kkmn5" event={"ID":"d5e5a7da-26f5-4d8d-8e9e-4e8991392e8e","Type":"ContainerStarted","Data":"f8d8f5329afdea9233753b883a4132ce6dc233c41d33963cfc10022a4ec726cf"} Jul 29 19:18:47.144676 ip-10-0-131-93 kubenswrapper[2564]: I0729 19:18:47.144286 2564 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-dns/dns-default-kkmn5" Jul 29 19:18:47.164129 ip-10-0-131-93 kubenswrapper[2564]: I0729 19:18:47.164078 2564 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-dns/dns-default-kkmn5" podStartSLOduration=67.286849953 podStartE2EDuration="1m9.164061722s" totalImagesPullingTime="1.877211769s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-29 19:17:38 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-29 19:18:47.163019442 +0000 UTC m=+102.887079134" watchObservedRunningTime="2026-07-29 19:18:47.164061722 +0000 UTC m=+102.888121413" Jul 29 19:18:50.645887 ip-10-0-131-93 kubenswrapper[2564]: I0729 19:18:50.645853 2564 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-444jx_faa8c546-c75b-4e10-93b4-661aa2b5f53e/konnectivity-agent/0.log" limit=1024 Jul 29 19:18:50.734030 ip-10-0-131-93 kubenswrapper[2564]: I0729 19:18:50.733993 2564 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-444jx_faa8c546-c75b-4e10-93b4-661aa2b5f53e/konnectivity-agent/0.log" limit=1024 Jul 29 19:18:50.937285 ip-10-0-131-93 kubenswrapper[2564]: I0729 19:18:50.937250 2564 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-444jx_faa8c546-c75b-4e10-93b4-661aa2b5f53e/konnectivity-agent/0.log" limit=1024 Jul 29 19:18:52.168317 ip-10-0-131-93 kubenswrapper[2564]: I0729 19:18:52.168272 2564 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-insights/insights-runtime-extractor-q4bqd" event={"ID":"9887221a-35c3-4cb7-8f93-d03a8e1c0e8e","Type":"ContainerStarted","Data":"fcb05ac190851ae4397a15868c359a1a55eb8d4f143e2ff776773dce37e25041"} Jul 29 19:18:54.030987 ip-10-0-131-93 kubenswrapper[2564]: I0729 19:18:54.030925 2564 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-444jx_faa8c546-c75b-4e10-93b4-661aa2b5f53e/konnectivity-agent/0.log" limit=1024 Jul 29 19:18:54.131353 ip-10-0-131-93 kubenswrapper[2564]: I0729 19:18:54.131310 2564 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-444jx_faa8c546-c75b-4e10-93b4-661aa2b5f53e/konnectivity-agent/0.log" limit=1024 Jul 29 19:18:55.181762 ip-10-0-131-93 kubenswrapper[2564]: I0729 19:18:55.181689 2564 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-insights/insights-runtime-extractor-q4bqd" event={"ID":"9887221a-35c3-4cb7-8f93-d03a8e1c0e8e","Type":"ContainerStarted","Data":"e2c6b080dc73f83a20a6d64ec8d2b8b06446ca0e3e6bc2615a98a90175b96e30"} Jul 29 19:18:55.205927 ip-10-0-131-93 kubenswrapper[2564]: I0729 19:18:55.205901 2564 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-444jx_faa8c546-c75b-4e10-93b4-661aa2b5f53e/konnectivity-agent/0.log" limit=1024 Jul 29 19:18:55.271055 ip-10-0-131-93 kubenswrapper[2564]: I0729 19:18:55.271023 2564 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-444jx_faa8c546-c75b-4e10-93b4-661aa2b5f53e/konnectivity-agent/0.log" limit=1024 Jul 29 19:18:57.150866 ip-10-0-131-93 kubenswrapper[2564]: I0729 19:18:57.150839 2564 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-dns/dns-default-kkmn5" Jul 29 19:18:59.197424 ip-10-0-131-93 kubenswrapper[2564]: I0729 19:18:59.197384 2564 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-insights/insights-runtime-extractor-q4bqd" event={"ID":"9887221a-35c3-4cb7-8f93-d03a8e1c0e8e","Type":"ContainerStarted","Data":"264dfb29690daa30ab1436e27e2ae6872a3a5c5b9ce2ec7c81ec7881c08ffe6f"} Jul 29 19:18:59.217295 ip-10-0-131-93 kubenswrapper[2564]: I0729 19:18:59.217252 2564 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-insights/insights-runtime-extractor-q4bqd" podStartSLOduration=34.041547895 podStartE2EDuration="47.217236703s" totalImagesPullingTime="13.175688808s" totalInitContainerRuntime="0s" isStatefulPod=true podCreationTimestamp="2026-07-29 19:18:12 +0000 UTC" imagePullSessionsCount=3 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-29 19:18:59.216173106 +0000 UTC m=+114.940232798" watchObservedRunningTime="2026-07-29 19:18:59.217236703 +0000 UTC m=+114.941296419" Jul 29 19:18:59.603581 ip-10-0-131-93 kubenswrapper[2564]: I0729 19:18:59.603502 2564 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-444jx_faa8c546-c75b-4e10-93b4-661aa2b5f53e/konnectivity-agent/0.log" limit=1024 Jul 29 19:18:59.657078 ip-10-0-131-93 kubenswrapper[2564]: I0729 19:18:59.657044 2564 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-444jx_faa8c546-c75b-4e10-93b4-661aa2b5f53e/konnectivity-agent/0.log" limit=1024 Jul 29 19:19:00.088366 ip-10-0-131-93 kubenswrapper[2564]: I0729 19:19:00.088330 2564 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-444jx_faa8c546-c75b-4e10-93b4-661aa2b5f53e/konnectivity-agent/0.log" limit=1024 Jul 29 19:19:00.188953 ip-10-0-131-93 kubenswrapper[2564]: I0729 19:19:00.188923 2564 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-444jx_faa8c546-c75b-4e10-93b4-661aa2b5f53e/konnectivity-agent/0.log" limit=1024 Jul 29 19:19:02.846971 ip-10-0-131-93 kubenswrapper[2564]: I0729 19:19:02.846937 2564 patch_prober.go:28] interesting pod/image-registry-7f95c97455-tkxb8 container/registry namespace/openshift-image-registry: Liveness probe status=failure output="HTTP probe failed with statuscode: 503" start-of-body={"errors":[{"code":"UNAVAILABLE","message":"service unavailable","detail":"health check failed: please see /debug/health"}]} Jul 29 19:19:02.847332 ip-10-0-131-93 kubenswrapper[2564]: I0729 19:19:02.846988 2564 prober.go:120] "Probe failed" probeType="Liveness" pod="openshift-image-registry/image-registry-7f95c97455-tkxb8" podUID="5ff17af5-17c3-45e6-b47f-408ba15ed5ca" containerName="registry" probeResult="failure" output="HTTP probe failed with statuscode: 503" Jul 29 19:19:03.127556 ip-10-0-131-93 kubenswrapper[2564]: I0729 19:19:03.127480 2564 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-image-registry/image-registry-7f95c97455-tkxb8"] Jul 29 19:19:03.131836 ip-10-0-131-93 kubenswrapper[2564]: I0729 19:19:03.131808 2564 patch_prober.go:28] interesting pod/image-registry-7f95c97455-tkxb8 container/registry namespace/openshift-image-registry: Readiness probe status=failure output="HTTP probe failed with statuscode: 503" start-of-body={"errors":[{"code":"UNAVAILABLE","message":"service unavailable","detail":"health check failed: please see /debug/health"}]} Jul 29 19:19:03.132020 ip-10-0-131-93 kubenswrapper[2564]: I0729 19:19:03.131857 2564 prober.go:120] "Probe failed" probeType="Readiness" pod="openshift-image-registry/image-registry-7f95c97455-tkxb8" podUID="5ff17af5-17c3-45e6-b47f-408ba15ed5ca" containerName="registry" probeResult="failure" output="HTTP probe failed with statuscode: 503" Jul 29 19:19:03.211195 ip-10-0-131-93 kubenswrapper[2564]: I0729 19:19:03.211163 2564 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-444jx_faa8c546-c75b-4e10-93b4-661aa2b5f53e/konnectivity-agent/0.log" limit=1024 Jul 29 19:19:03.310870 ip-10-0-131-93 kubenswrapper[2564]: I0729 19:19:03.310838 2564 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-444jx_faa8c546-c75b-4e10-93b4-661aa2b5f53e/konnectivity-agent/0.log" limit=1024 Jul 29 19:19:05.402084 ip-10-0-131-93 kubenswrapper[2564]: I0729 19:19:05.402055 2564 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-444jx_faa8c546-c75b-4e10-93b4-661aa2b5f53e/konnectivity-agent/0.log" limit=1024 Jul 29 19:19:05.470999 ip-10-0-131-93 kubenswrapper[2564]: I0729 19:19:05.470965 2564 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-444jx_faa8c546-c75b-4e10-93b4-661aa2b5f53e/konnectivity-agent/0.log" limit=1024 Jul 29 19:19:06.698834 ip-10-0-131-93 kubenswrapper[2564]: I0729 19:19:06.698795 2564 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-444jx_faa8c546-c75b-4e10-93b4-661aa2b5f53e/konnectivity-agent/0.log" limit=1024 Jul 29 19:19:06.753719 ip-10-0-131-93 kubenswrapper[2564]: I0729 19:19:06.753691 2564 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-444jx_faa8c546-c75b-4e10-93b4-661aa2b5f53e/konnectivity-agent/0.log" limit=1024 Jul 29 19:19:07.129446 ip-10-0-131-93 kubenswrapper[2564]: I0729 19:19:07.129345 2564 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-444jx_faa8c546-c75b-4e10-93b4-661aa2b5f53e/konnectivity-agent/0.log" limit=1024 Jul 29 19:19:07.229086 ip-10-0-131-93 kubenswrapper[2564]: I0729 19:19:07.229056 2564 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-444jx_faa8c546-c75b-4e10-93b4-661aa2b5f53e/konnectivity-agent/0.log" limit=1024 Jul 29 19:19:07.329665 ip-10-0-131-93 kubenswrapper[2564]: I0729 19:19:07.329632 2564 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-444jx_faa8c546-c75b-4e10-93b4-661aa2b5f53e/konnectivity-agent/0.log" limit=1024 Jul 29 19:19:08.795743 ip-10-0-131-93 kubenswrapper[2564]: I0729 19:19:08.795711 2564 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-444jx_faa8c546-c75b-4e10-93b4-661aa2b5f53e/konnectivity-agent/0.log" limit=1024 Jul 29 19:19:09.077397 ip-10-0-131-93 kubenswrapper[2564]: I0729 19:19:09.077298 2564 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-444jx_faa8c546-c75b-4e10-93b4-661aa2b5f53e/konnectivity-agent/0.log" limit=1024 Jul 29 19:19:09.120235 ip-10-0-131-93 kubenswrapper[2564]: I0729 19:19:09.120206 2564 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-monitoring/node-exporter-95bmx"] Jul 29 19:19:09.122617 ip-10-0-131-93 kubenswrapper[2564]: I0729 19:19:09.122598 2564 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-monitoring/node-exporter-95bmx" Jul 29 19:19:09.131225 ip-10-0-131-93 kubenswrapper[2564]: I0729 19:19:09.131200 2564 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"node-exporter-kube-rbac-proxy-config\"" Jul 29 19:19:09.131225 ip-10-0-131-93 kubenswrapper[2564]: I0729 19:19:09.131213 2564 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"node-exporter-tls\"" Jul 29 19:19:09.132215 ip-10-0-131-93 kubenswrapper[2564]: I0729 19:19:09.132201 2564 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-monitoring\"/\"node-exporter-accelerators-collector-config\"" Jul 29 19:19:09.132257 ip-10-0-131-93 kubenswrapper[2564]: I0729 19:19:09.132216 2564 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"node-exporter-dockercfg-2pdcd\"" Jul 29 19:19:09.132257 ip-10-0-131-93 kubenswrapper[2564]: I0729 19:19:09.132242 2564 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-monitoring/kube-state-metrics-56d9f944b4-mb46f"] Jul 29 19:19:09.132332 ip-10-0-131-93 kubenswrapper[2564]: I0729 19:19:09.132207 2564 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-monitoring\"/\"openshift-service-ca.crt\"" Jul 29 19:19:09.132332 ip-10-0-131-93 kubenswrapper[2564]: I0729 19:19:09.132208 2564 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-monitoring\"/\"metrics-client-ca\"" Jul 29 19:19:09.132720 ip-10-0-131-93 kubenswrapper[2564]: I0729 19:19:09.132707 2564 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-monitoring\"/\"kube-root-ca.crt\"" Jul 29 19:19:09.134271 ip-10-0-131-93 kubenswrapper[2564]: I0729 19:19:09.134258 2564 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-monitoring/kube-state-metrics-56d9f944b4-mb46f" Jul 29 19:19:09.136387 ip-10-0-131-93 kubenswrapper[2564]: I0729 19:19:09.136359 2564 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"kube-state-metrics-tls\"" Jul 29 19:19:09.136623 ip-10-0-131-93 kubenswrapper[2564]: I0729 19:19:09.136588 2564 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"kube-state-metrics-dockercfg-qmn25\"" Jul 29 19:19:09.136623 ip-10-0-131-93 kubenswrapper[2564]: I0729 19:19:09.136618 2564 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"kube-state-metrics-kube-rbac-proxy-config\"" Jul 29 19:19:09.136905 ip-10-0-131-93 kubenswrapper[2564]: I0729 19:19:09.136887 2564 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-monitoring\"/\"kube-state-metrics-custom-resource-state-configmap\"" Jul 29 19:19:09.149237 ip-10-0-131-93 kubenswrapper[2564]: I0729 19:19:09.149211 2564 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-monitoring/kube-state-metrics-56d9f944b4-mb46f"] Jul 29 19:19:09.162704 ip-10-0-131-93 kubenswrapper[2564]: I0729 19:19:09.162654 2564 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-444jx_faa8c546-c75b-4e10-93b4-661aa2b5f53e/konnectivity-agent/0.log" limit=1024 Jul 29 19:19:09.186365 ip-10-0-131-93 kubenswrapper[2564]: I0729 19:19:09.186340 2564 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-rrrvc\" (UniqueName: \"kubernetes.io/projected/09be503d-5e55-4ef7-9339-7f1a9b8c2029-kube-api-access-rrrvc\") pod \"node-exporter-95bmx\" (UID: \"09be503d-5e55-4ef7-9339-7f1a9b8c2029\") " pod="openshift-monitoring/node-exporter-95bmx" Jul 29 19:19:09.186491 ip-10-0-131-93 kubenswrapper[2564]: I0729 19:19:09.186386 2564 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"node-exporter-accelerators-collector-config\" (UniqueName: \"kubernetes.io/configmap/09be503d-5e55-4ef7-9339-7f1a9b8c2029-node-exporter-accelerators-collector-config\") pod \"node-exporter-95bmx\" (UID: \"09be503d-5e55-4ef7-9339-7f1a9b8c2029\") " pod="openshift-monitoring/node-exporter-95bmx" Jul 29 19:19:09.186491 ip-10-0-131-93 kubenswrapper[2564]: I0729 19:19:09.186420 2564 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"node-exporter-tls\" (UniqueName: \"kubernetes.io/secret/09be503d-5e55-4ef7-9339-7f1a9b8c2029-node-exporter-tls\") pod \"node-exporter-95bmx\" (UID: \"09be503d-5e55-4ef7-9339-7f1a9b8c2029\") " pod="openshift-monitoring/node-exporter-95bmx" Jul 29 19:19:09.186491 ip-10-0-131-93 kubenswrapper[2564]: I0729 19:19:09.186443 2564 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"metrics-client-ca\" (UniqueName: \"kubernetes.io/configmap/09be503d-5e55-4ef7-9339-7f1a9b8c2029-metrics-client-ca\") pod \"node-exporter-95bmx\" (UID: \"09be503d-5e55-4ef7-9339-7f1a9b8c2029\") " pod="openshift-monitoring/node-exporter-95bmx" Jul 29 19:19:09.186491 ip-10-0-131-93 kubenswrapper[2564]: I0729 19:19:09.186463 2564 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"metrics-client-ca\" (UniqueName: \"kubernetes.io/configmap/b36c4189-c565-4c30-97c8-8b8811a881b2-metrics-client-ca\") pod \"kube-state-metrics-56d9f944b4-mb46f\" (UID: \"b36c4189-c565-4c30-97c8-8b8811a881b2\") " pod="openshift-monitoring/kube-state-metrics-56d9f944b4-mb46f" Jul 29 19:19:09.186627 ip-10-0-131-93 kubenswrapper[2564]: I0729 19:19:09.186500 2564 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-state-metrics-tls\" (UniqueName: \"kubernetes.io/secret/b36c4189-c565-4c30-97c8-8b8811a881b2-kube-state-metrics-tls\") pod \"kube-state-metrics-56d9f944b4-mb46f\" (UID: \"b36c4189-c565-4c30-97c8-8b8811a881b2\") " pod="openshift-monitoring/kube-state-metrics-56d9f944b4-mb46f" Jul 29 19:19:09.186627 ip-10-0-131-93 kubenswrapper[2564]: I0729 19:19:09.186526 2564 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-65qzw\" (UniqueName: \"kubernetes.io/projected/b36c4189-c565-4c30-97c8-8b8811a881b2-kube-api-access-65qzw\") pod \"kube-state-metrics-56d9f944b4-mb46f\" (UID: \"b36c4189-c565-4c30-97c8-8b8811a881b2\") " pod="openshift-monitoring/kube-state-metrics-56d9f944b4-mb46f" Jul 29 19:19:09.186627 ip-10-0-131-93 kubenswrapper[2564]: I0729 19:19:09.186553 2564 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"node-exporter-kube-rbac-proxy-config\" (UniqueName: \"kubernetes.io/secret/09be503d-5e55-4ef7-9339-7f1a9b8c2029-node-exporter-kube-rbac-proxy-config\") pod \"node-exporter-95bmx\" (UID: \"09be503d-5e55-4ef7-9339-7f1a9b8c2029\") " pod="openshift-monitoring/node-exporter-95bmx" Jul 29 19:19:09.186627 ip-10-0-131-93 kubenswrapper[2564]: I0729 19:19:09.186574 2564 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-state-metrics-custom-resource-state-configmap\" (UniqueName: \"kubernetes.io/configmap/b36c4189-c565-4c30-97c8-8b8811a881b2-kube-state-metrics-custom-resource-state-configmap\") pod \"kube-state-metrics-56d9f944b4-mb46f\" (UID: \"b36c4189-c565-4c30-97c8-8b8811a881b2\") " pod="openshift-monitoring/kube-state-metrics-56d9f944b4-mb46f" Jul 29 19:19:09.186627 ip-10-0-131-93 kubenswrapper[2564]: I0729 19:19:09.186604 2564 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-state-metrics-kube-rbac-proxy-config\" (UniqueName: \"kubernetes.io/secret/b36c4189-c565-4c30-97c8-8b8811a881b2-kube-state-metrics-kube-rbac-proxy-config\") pod \"kube-state-metrics-56d9f944b4-mb46f\" (UID: \"b36c4189-c565-4c30-97c8-8b8811a881b2\") " pod="openshift-monitoring/kube-state-metrics-56d9f944b4-mb46f" Jul 29 19:19:09.186627 ip-10-0-131-93 kubenswrapper[2564]: I0729 19:19:09.186621 2564 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"root\" (UniqueName: \"kubernetes.io/host-path/09be503d-5e55-4ef7-9339-7f1a9b8c2029-root\") pod \"node-exporter-95bmx\" (UID: \"09be503d-5e55-4ef7-9339-7f1a9b8c2029\") " pod="openshift-monitoring/node-exporter-95bmx" Jul 29 19:19:09.186800 ip-10-0-131-93 kubenswrapper[2564]: I0729 19:19:09.186645 2564 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"node-exporter-wtmp\" (UniqueName: \"kubernetes.io/host-path/09be503d-5e55-4ef7-9339-7f1a9b8c2029-node-exporter-wtmp\") pod \"node-exporter-95bmx\" (UID: \"09be503d-5e55-4ef7-9339-7f1a9b8c2029\") " pod="openshift-monitoring/node-exporter-95bmx" Jul 29 19:19:09.186800 ip-10-0-131-93 kubenswrapper[2564]: I0729 19:19:09.186665 2564 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"volume-directive-shadow\" (UniqueName: \"kubernetes.io/empty-dir/b36c4189-c565-4c30-97c8-8b8811a881b2-volume-directive-shadow\") pod \"kube-state-metrics-56d9f944b4-mb46f\" (UID: \"b36c4189-c565-4c30-97c8-8b8811a881b2\") " pod="openshift-monitoring/kube-state-metrics-56d9f944b4-mb46f" Jul 29 19:19:09.186800 ip-10-0-131-93 kubenswrapper[2564]: I0729 19:19:09.186718 2564 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"sys\" (UniqueName: \"kubernetes.io/host-path/09be503d-5e55-4ef7-9339-7f1a9b8c2029-sys\") pod \"node-exporter-95bmx\" (UID: \"09be503d-5e55-4ef7-9339-7f1a9b8c2029\") " pod="openshift-monitoring/node-exporter-95bmx" Jul 29 19:19:09.186800 ip-10-0-131-93 kubenswrapper[2564]: I0729 19:19:09.186735 2564 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"node-exporter-textfile\" (UniqueName: \"kubernetes.io/empty-dir/09be503d-5e55-4ef7-9339-7f1a9b8c2029-node-exporter-textfile\") pod \"node-exporter-95bmx\" (UID: \"09be503d-5e55-4ef7-9339-7f1a9b8c2029\") " pod="openshift-monitoring/node-exporter-95bmx" Jul 29 19:19:09.229278 ip-10-0-131-93 kubenswrapper[2564]: I0729 19:19:09.229254 2564 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-444jx_faa8c546-c75b-4e10-93b4-661aa2b5f53e/konnectivity-agent/0.log" limit=1024 Jul 29 19:19:09.287217 ip-10-0-131-93 kubenswrapper[2564]: I0729 19:19:09.287183 2564 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-client-ca\" (UniqueName: \"kubernetes.io/configmap/b36c4189-c565-4c30-97c8-8b8811a881b2-metrics-client-ca\") pod \"kube-state-metrics-56d9f944b4-mb46f\" (UID: \"b36c4189-c565-4c30-97c8-8b8811a881b2\") " pod="openshift-monitoring/kube-state-metrics-56d9f944b4-mb46f" Jul 29 19:19:09.287217 ip-10-0-131-93 kubenswrapper[2564]: I0729 19:19:09.287221 2564 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-state-metrics-tls\" (UniqueName: \"kubernetes.io/secret/b36c4189-c565-4c30-97c8-8b8811a881b2-kube-state-metrics-tls\") pod \"kube-state-metrics-56d9f944b4-mb46f\" (UID: \"b36c4189-c565-4c30-97c8-8b8811a881b2\") " pod="openshift-monitoring/kube-state-metrics-56d9f944b4-mb46f" Jul 29 19:19:09.287499 ip-10-0-131-93 kubenswrapper[2564]: I0729 19:19:09.287238 2564 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-65qzw\" (UniqueName: \"kubernetes.io/projected/b36c4189-c565-4c30-97c8-8b8811a881b2-kube-api-access-65qzw\") pod \"kube-state-metrics-56d9f944b4-mb46f\" (UID: \"b36c4189-c565-4c30-97c8-8b8811a881b2\") " pod="openshift-monitoring/kube-state-metrics-56d9f944b4-mb46f" Jul 29 19:19:09.287499 ip-10-0-131-93 kubenswrapper[2564]: I0729 19:19:09.287271 2564 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"node-exporter-kube-rbac-proxy-config\" (UniqueName: \"kubernetes.io/secret/09be503d-5e55-4ef7-9339-7f1a9b8c2029-node-exporter-kube-rbac-proxy-config\") pod \"node-exporter-95bmx\" (UID: \"09be503d-5e55-4ef7-9339-7f1a9b8c2029\") " pod="openshift-monitoring/node-exporter-95bmx" Jul 29 19:19:09.287499 ip-10-0-131-93 kubenswrapper[2564]: I0729 19:19:09.287291 2564 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-state-metrics-custom-resource-state-configmap\" (UniqueName: \"kubernetes.io/configmap/b36c4189-c565-4c30-97c8-8b8811a881b2-kube-state-metrics-custom-resource-state-configmap\") pod \"kube-state-metrics-56d9f944b4-mb46f\" (UID: \"b36c4189-c565-4c30-97c8-8b8811a881b2\") " pod="openshift-monitoring/kube-state-metrics-56d9f944b4-mb46f" Jul 29 19:19:09.287499 ip-10-0-131-93 kubenswrapper[2564]: I0729 19:19:09.287310 2564 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-state-metrics-kube-rbac-proxy-config\" (UniqueName: \"kubernetes.io/secret/b36c4189-c565-4c30-97c8-8b8811a881b2-kube-state-metrics-kube-rbac-proxy-config\") pod \"kube-state-metrics-56d9f944b4-mb46f\" (UID: \"b36c4189-c565-4c30-97c8-8b8811a881b2\") " pod="openshift-monitoring/kube-state-metrics-56d9f944b4-mb46f" Jul 29 19:19:09.287499 ip-10-0-131-93 kubenswrapper[2564]: I0729 19:19:09.287335 2564 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"root\" (UniqueName: \"kubernetes.io/host-path/09be503d-5e55-4ef7-9339-7f1a9b8c2029-root\") pod \"node-exporter-95bmx\" (UID: \"09be503d-5e55-4ef7-9339-7f1a9b8c2029\") " pod="openshift-monitoring/node-exporter-95bmx" Jul 29 19:19:09.287499 ip-10-0-131-93 kubenswrapper[2564]: I0729 19:19:09.287386 2564 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"node-exporter-wtmp\" (UniqueName: \"kubernetes.io/host-path/09be503d-5e55-4ef7-9339-7f1a9b8c2029-node-exporter-wtmp\") pod \"node-exporter-95bmx\" (UID: \"09be503d-5e55-4ef7-9339-7f1a9b8c2029\") " pod="openshift-monitoring/node-exporter-95bmx" Jul 29 19:19:09.287804 ip-10-0-131-93 kubenswrapper[2564]: I0729 19:19:09.287496 2564 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"root\" (UniqueName: \"kubernetes.io/host-path/09be503d-5e55-4ef7-9339-7f1a9b8c2029-root\") pod \"node-exporter-95bmx\" (UID: \"09be503d-5e55-4ef7-9339-7f1a9b8c2029\") " pod="openshift-monitoring/node-exporter-95bmx" Jul 29 19:19:09.287804 ip-10-0-131-93 kubenswrapper[2564]: I0729 19:19:09.287515 2564 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"node-exporter-wtmp\" (UniqueName: \"kubernetes.io/host-path/09be503d-5e55-4ef7-9339-7f1a9b8c2029-node-exporter-wtmp\") pod \"node-exporter-95bmx\" (UID: \"09be503d-5e55-4ef7-9339-7f1a9b8c2029\") " pod="openshift-monitoring/node-exporter-95bmx" Jul 29 19:19:09.287804 ip-10-0-131-93 kubenswrapper[2564]: I0729 19:19:09.287559 2564 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"volume-directive-shadow\" (UniqueName: \"kubernetes.io/empty-dir/b36c4189-c565-4c30-97c8-8b8811a881b2-volume-directive-shadow\") pod \"kube-state-metrics-56d9f944b4-mb46f\" (UID: \"b36c4189-c565-4c30-97c8-8b8811a881b2\") " pod="openshift-monitoring/kube-state-metrics-56d9f944b4-mb46f" Jul 29 19:19:09.287804 ip-10-0-131-93 kubenswrapper[2564]: I0729 19:19:09.287604 2564 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"sys\" (UniqueName: \"kubernetes.io/host-path/09be503d-5e55-4ef7-9339-7f1a9b8c2029-sys\") pod \"node-exporter-95bmx\" (UID: \"09be503d-5e55-4ef7-9339-7f1a9b8c2029\") " pod="openshift-monitoring/node-exporter-95bmx" Jul 29 19:19:09.287804 ip-10-0-131-93 kubenswrapper[2564]: I0729 19:19:09.287634 2564 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"node-exporter-textfile\" (UniqueName: \"kubernetes.io/empty-dir/09be503d-5e55-4ef7-9339-7f1a9b8c2029-node-exporter-textfile\") pod \"node-exporter-95bmx\" (UID: \"09be503d-5e55-4ef7-9339-7f1a9b8c2029\") " pod="openshift-monitoring/node-exporter-95bmx" Jul 29 19:19:09.287804 ip-10-0-131-93 kubenswrapper[2564]: I0729 19:19:09.287667 2564 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-rrrvc\" (UniqueName: \"kubernetes.io/projected/09be503d-5e55-4ef7-9339-7f1a9b8c2029-kube-api-access-rrrvc\") pod \"node-exporter-95bmx\" (UID: \"09be503d-5e55-4ef7-9339-7f1a9b8c2029\") " pod="openshift-monitoring/node-exporter-95bmx" Jul 29 19:19:09.287804 ip-10-0-131-93 kubenswrapper[2564]: I0729 19:19:09.287670 2564 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"sys\" (UniqueName: \"kubernetes.io/host-path/09be503d-5e55-4ef7-9339-7f1a9b8c2029-sys\") pod \"node-exporter-95bmx\" (UID: \"09be503d-5e55-4ef7-9339-7f1a9b8c2029\") " pod="openshift-monitoring/node-exporter-95bmx" Jul 29 19:19:09.287804 ip-10-0-131-93 kubenswrapper[2564]: I0729 19:19:09.287694 2564 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"node-exporter-accelerators-collector-config\" (UniqueName: \"kubernetes.io/configmap/09be503d-5e55-4ef7-9339-7f1a9b8c2029-node-exporter-accelerators-collector-config\") pod \"node-exporter-95bmx\" (UID: \"09be503d-5e55-4ef7-9339-7f1a9b8c2029\") " pod="openshift-monitoring/node-exporter-95bmx" Jul 29 19:19:09.287804 ip-10-0-131-93 kubenswrapper[2564]: I0729 19:19:09.287731 2564 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"node-exporter-tls\" (UniqueName: \"kubernetes.io/secret/09be503d-5e55-4ef7-9339-7f1a9b8c2029-node-exporter-tls\") pod \"node-exporter-95bmx\" (UID: \"09be503d-5e55-4ef7-9339-7f1a9b8c2029\") " pod="openshift-monitoring/node-exporter-95bmx" Jul 29 19:19:09.287804 ip-10-0-131-93 kubenswrapper[2564]: I0729 19:19:09.287760 2564 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-client-ca\" (UniqueName: \"kubernetes.io/configmap/09be503d-5e55-4ef7-9339-7f1a9b8c2029-metrics-client-ca\") pod \"node-exporter-95bmx\" (UID: \"09be503d-5e55-4ef7-9339-7f1a9b8c2029\") " pod="openshift-monitoring/node-exporter-95bmx" Jul 29 19:19:09.288290 ip-10-0-131-93 kubenswrapper[2564]: I0729 19:19:09.288040 2564 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"metrics-client-ca\" (UniqueName: \"kubernetes.io/configmap/b36c4189-c565-4c30-97c8-8b8811a881b2-metrics-client-ca\") pod \"kube-state-metrics-56d9f944b4-mb46f\" (UID: \"b36c4189-c565-4c30-97c8-8b8811a881b2\") " pod="openshift-monitoring/kube-state-metrics-56d9f944b4-mb46f" Jul 29 19:19:09.288290 ip-10-0-131-93 kubenswrapper[2564]: I0729 19:19:09.288182 2564 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"volume-directive-shadow\" (UniqueName: \"kubernetes.io/empty-dir/b36c4189-c565-4c30-97c8-8b8811a881b2-volume-directive-shadow\") pod \"kube-state-metrics-56d9f944b4-mb46f\" (UID: \"b36c4189-c565-4c30-97c8-8b8811a881b2\") " pod="openshift-monitoring/kube-state-metrics-56d9f944b4-mb46f" Jul 29 19:19:09.288415 ip-10-0-131-93 kubenswrapper[2564]: I0729 19:19:09.288319 2564 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"node-exporter-textfile\" (UniqueName: \"kubernetes.io/empty-dir/09be503d-5e55-4ef7-9339-7f1a9b8c2029-node-exporter-textfile\") pod \"node-exporter-95bmx\" (UID: \"09be503d-5e55-4ef7-9339-7f1a9b8c2029\") " pod="openshift-monitoring/node-exporter-95bmx" Jul 29 19:19:09.288484 ip-10-0-131-93 kubenswrapper[2564]: I0729 19:19:09.288441 2564 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-state-metrics-custom-resource-state-configmap\" (UniqueName: \"kubernetes.io/configmap/b36c4189-c565-4c30-97c8-8b8811a881b2-kube-state-metrics-custom-resource-state-configmap\") pod \"kube-state-metrics-56d9f944b4-mb46f\" (UID: \"b36c4189-c565-4c30-97c8-8b8811a881b2\") " pod="openshift-monitoring/kube-state-metrics-56d9f944b4-mb46f" Jul 29 19:19:09.288959 ip-10-0-131-93 kubenswrapper[2564]: I0729 19:19:09.288934 2564 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"node-exporter-accelerators-collector-config\" (UniqueName: \"kubernetes.io/configmap/09be503d-5e55-4ef7-9339-7f1a9b8c2029-node-exporter-accelerators-collector-config\") pod \"node-exporter-95bmx\" (UID: \"09be503d-5e55-4ef7-9339-7f1a9b8c2029\") " pod="openshift-monitoring/node-exporter-95bmx" Jul 29 19:19:09.289082 ip-10-0-131-93 kubenswrapper[2564]: I0729 19:19:09.288996 2564 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"metrics-client-ca\" (UniqueName: \"kubernetes.io/configmap/09be503d-5e55-4ef7-9339-7f1a9b8c2029-metrics-client-ca\") pod \"node-exporter-95bmx\" (UID: \"09be503d-5e55-4ef7-9339-7f1a9b8c2029\") " pod="openshift-monitoring/node-exporter-95bmx" Jul 29 19:19:09.290007 ip-10-0-131-93 kubenswrapper[2564]: I0729 19:19:09.289982 2564 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-state-metrics-tls\" (UniqueName: \"kubernetes.io/secret/b36c4189-c565-4c30-97c8-8b8811a881b2-kube-state-metrics-tls\") pod \"kube-state-metrics-56d9f944b4-mb46f\" (UID: \"b36c4189-c565-4c30-97c8-8b8811a881b2\") " pod="openshift-monitoring/kube-state-metrics-56d9f944b4-mb46f" Jul 29 19:19:09.290007 ip-10-0-131-93 kubenswrapper[2564]: I0729 19:19:09.289988 2564 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"node-exporter-kube-rbac-proxy-config\" (UniqueName: \"kubernetes.io/secret/09be503d-5e55-4ef7-9339-7f1a9b8c2029-node-exporter-kube-rbac-proxy-config\") pod \"node-exporter-95bmx\" (UID: \"09be503d-5e55-4ef7-9339-7f1a9b8c2029\") " pod="openshift-monitoring/node-exporter-95bmx" Jul 29 19:19:09.290157 ip-10-0-131-93 kubenswrapper[2564]: I0729 19:19:09.290074 2564 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-state-metrics-kube-rbac-proxy-config\" (UniqueName: \"kubernetes.io/secret/b36c4189-c565-4c30-97c8-8b8811a881b2-kube-state-metrics-kube-rbac-proxy-config\") pod \"kube-state-metrics-56d9f944b4-mb46f\" (UID: \"b36c4189-c565-4c30-97c8-8b8811a881b2\") " pod="openshift-monitoring/kube-state-metrics-56d9f944b4-mb46f" Jul 29 19:19:09.290465 ip-10-0-131-93 kubenswrapper[2564]: I0729 19:19:09.290448 2564 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"node-exporter-tls\" (UniqueName: \"kubernetes.io/secret/09be503d-5e55-4ef7-9339-7f1a9b8c2029-node-exporter-tls\") pod \"node-exporter-95bmx\" (UID: \"09be503d-5e55-4ef7-9339-7f1a9b8c2029\") " pod="openshift-monitoring/node-exporter-95bmx" Jul 29 19:19:09.295418 ip-10-0-131-93 kubenswrapper[2564]: I0729 19:19:09.295398 2564 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-rrrvc\" (UniqueName: \"kubernetes.io/projected/09be503d-5e55-4ef7-9339-7f1a9b8c2029-kube-api-access-rrrvc\") pod \"node-exporter-95bmx\" (UID: \"09be503d-5e55-4ef7-9339-7f1a9b8c2029\") " pod="openshift-monitoring/node-exporter-95bmx" Jul 29 19:19:09.295554 ip-10-0-131-93 kubenswrapper[2564]: I0729 19:19:09.295533 2564 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-65qzw\" (UniqueName: \"kubernetes.io/projected/b36c4189-c565-4c30-97c8-8b8811a881b2-kube-api-access-65qzw\") pod \"kube-state-metrics-56d9f944b4-mb46f\" (UID: \"b36c4189-c565-4c30-97c8-8b8811a881b2\") " pod="openshift-monitoring/kube-state-metrics-56d9f944b4-mb46f" Jul 29 19:19:09.327460 ip-10-0-131-93 kubenswrapper[2564]: I0729 19:19:09.327367 2564 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-444jx_faa8c546-c75b-4e10-93b4-661aa2b5f53e/konnectivity-agent/0.log" limit=1024 Jul 29 19:19:09.427659 ip-10-0-131-93 kubenswrapper[2564]: I0729 19:19:09.427632 2564 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-444jx_faa8c546-c75b-4e10-93b4-661aa2b5f53e/konnectivity-agent/0.log" limit=1024 Jul 29 19:19:09.431544 ip-10-0-131-93 kubenswrapper[2564]: I0729 19:19:09.431528 2564 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-monitoring/node-exporter-95bmx" Jul 29 19:19:09.440991 ip-10-0-131-93 kubenswrapper[2564]: W0729 19:19:09.440967 2564 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod09be503d_5e55_4ef7_9339_7f1a9b8c2029.slice/crio-ed085c549c277a3526b63f1842495cad088a084c09eb1f83f2dda32c6ed77260 WatchSource:0}: Error finding container ed085c549c277a3526b63f1842495cad088a084c09eb1f83f2dda32c6ed77260: Status 404 returned error can't find the container with id ed085c549c277a3526b63f1842495cad088a084c09eb1f83f2dda32c6ed77260 Jul 29 19:19:09.442037 ip-10-0-131-93 kubenswrapper[2564]: I0729 19:19:09.442015 2564 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-monitoring/kube-state-metrics-56d9f944b4-mb46f" Jul 29 19:19:09.563398 ip-10-0-131-93 kubenswrapper[2564]: I0729 19:19:09.563362 2564 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-monitoring/kube-state-metrics-56d9f944b4-mb46f"] Jul 29 19:19:09.565662 ip-10-0-131-93 kubenswrapper[2564]: W0729 19:19:09.565636 2564 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podb36c4189_c565_4c30_97c8_8b8811a881b2.slice/crio-0b8460b55326b935c7e60e7bc1fafe1058fcc980011be89effd7b760f580b542 WatchSource:0}: Error finding container 0b8460b55326b935c7e60e7bc1fafe1058fcc980011be89effd7b760f580b542: Status 404 returned error can't find the container with id 0b8460b55326b935c7e60e7bc1fafe1058fcc980011be89effd7b760f580b542 Jul 29 19:19:09.747906 ip-10-0-131-93 kubenswrapper[2564]: I0729 19:19:09.747874 2564 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-444jx_faa8c546-c75b-4e10-93b4-661aa2b5f53e/konnectivity-agent/0.log" limit=1024 Jul 29 19:19:10.018346 ip-10-0-131-93 kubenswrapper[2564]: I0729 19:19:10.018269 2564 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-444jx_faa8c546-c75b-4e10-93b4-661aa2b5f53e/konnectivity-agent/0.log" limit=1024 Jul 29 19:19:10.112113 ip-10-0-131-93 kubenswrapper[2564]: I0729 19:19:10.112077 2564 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-444jx_faa8c546-c75b-4e10-93b4-661aa2b5f53e/konnectivity-agent/0.log" limit=1024 Jul 29 19:19:10.201441 ip-10-0-131-93 kubenswrapper[2564]: I0729 19:19:10.201408 2564 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-444jx_faa8c546-c75b-4e10-93b4-661aa2b5f53e/konnectivity-agent/0.log" limit=1024 Jul 29 19:19:10.231691 ip-10-0-131-93 kubenswrapper[2564]: I0729 19:19:10.231619 2564 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/kube-state-metrics-56d9f944b4-mb46f" event={"ID":"b36c4189-c565-4c30-97c8-8b8811a881b2","Type":"ContainerStarted","Data":"0b8460b55326b935c7e60e7bc1fafe1058fcc980011be89effd7b760f580b542"} Jul 29 19:19:10.232915 ip-10-0-131-93 kubenswrapper[2564]: I0729 19:19:10.232892 2564 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/node-exporter-95bmx" event={"ID":"09be503d-5e55-4ef7-9339-7f1a9b8c2029","Type":"ContainerStarted","Data":"ed085c549c277a3526b63f1842495cad088a084c09eb1f83f2dda32c6ed77260"} Jul 29 19:19:11.099423 ip-10-0-131-93 kubenswrapper[2564]: I0729 19:19:11.099321 2564 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-444jx_faa8c546-c75b-4e10-93b4-661aa2b5f53e/konnectivity-agent/0.log" limit=1024 Jul 29 19:19:11.187042 ip-10-0-131-93 kubenswrapper[2564]: I0729 19:19:11.186996 2564 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-444jx_faa8c546-c75b-4e10-93b4-661aa2b5f53e/konnectivity-agent/0.log" limit=1024 Jul 29 19:19:11.238476 ip-10-0-131-93 kubenswrapper[2564]: I0729 19:19:11.238443 2564 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/kube-state-metrics-56d9f944b4-mb46f" event={"ID":"b36c4189-c565-4c30-97c8-8b8811a881b2","Type":"ContainerStarted","Data":"da4384877a30debe901a611f9423e7c8a0bd9493057029d08aed3bbca6f16a90"} Jul 29 19:19:11.238653 ip-10-0-131-93 kubenswrapper[2564]: I0729 19:19:11.238503 2564 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/kube-state-metrics-56d9f944b4-mb46f" event={"ID":"b36c4189-c565-4c30-97c8-8b8811a881b2","Type":"ContainerStarted","Data":"c40317c8dc08623b61b81d94afabc5b9d82b2a969f0f6f49a4650653b18b4cbf"} Jul 29 19:19:11.238653 ip-10-0-131-93 kubenswrapper[2564]: I0729 19:19:11.238524 2564 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/kube-state-metrics-56d9f944b4-mb46f" event={"ID":"b36c4189-c565-4c30-97c8-8b8811a881b2","Type":"ContainerStarted","Data":"41e311b4c500fda0501d2392ae8acd338a03e23f63266f0e9d6c72fe410f11b8"} Jul 29 19:19:11.264434 ip-10-0-131-93 kubenswrapper[2564]: I0729 19:19:11.264364 2564 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-monitoring/kube-state-metrics-56d9f944b4-mb46f" podStartSLOduration=1.121045867 podStartE2EDuration="2.264344353s" totalImagesPullingTime="1.143298486s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-29 19:19:09 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-29 19:19:11.262248591 +0000 UTC m=+126.986308284" watchObservedRunningTime="2026-07-29 19:19:11.264344353 +0000 UTC m=+126.988404051" Jul 29 19:19:11.974166 ip-10-0-131-93 kubenswrapper[2564]: I0729 19:19:11.974130 2564 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-444jx_faa8c546-c75b-4e10-93b4-661aa2b5f53e/konnectivity-agent/0.log" limit=1024 Jul 29 19:19:12.053751 ip-10-0-131-93 kubenswrapper[2564]: I0729 19:19:12.053719 2564 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-444jx_faa8c546-c75b-4e10-93b4-661aa2b5f53e/konnectivity-agent/0.log" limit=1024 Jul 29 19:19:12.169180 ip-10-0-131-93 kubenswrapper[2564]: I0729 19:19:12.169148 2564 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-444jx_faa8c546-c75b-4e10-93b4-661aa2b5f53e/konnectivity-agent/0.log" limit=1024 Jul 29 19:19:12.243551 ip-10-0-131-93 kubenswrapper[2564]: I0729 19:19:12.243464 2564 generic.go:356] "Generic (PLEG): container finished" podID="09be503d-5e55-4ef7-9339-7f1a9b8c2029" containerID="b9fbb6333ea48aa4b80b3024ce87153040b03b2b7961ae6a7fcefb8313bb5253" exitCode=0 Jul 29 19:19:12.243693 ip-10-0-131-93 kubenswrapper[2564]: I0729 19:19:12.243559 2564 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/node-exporter-95bmx" event={"ID":"09be503d-5e55-4ef7-9339-7f1a9b8c2029","Type":"ContainerDied","Data":"b9fbb6333ea48aa4b80b3024ce87153040b03b2b7961ae6a7fcefb8313bb5253"} Jul 29 19:19:13.132706 ip-10-0-131-93 kubenswrapper[2564]: I0729 19:19:13.132683 2564 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-image-registry/image-registry-7f95c97455-tkxb8" Jul 29 19:19:13.249247 ip-10-0-131-93 kubenswrapper[2564]: I0729 19:19:13.249215 2564 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/node-exporter-95bmx" event={"ID":"09be503d-5e55-4ef7-9339-7f1a9b8c2029","Type":"ContainerStarted","Data":"7301aa695950372e8362a55719dc7f1196bcdf8d2ab4d700afe457ab888c3d79"} Jul 29 19:19:13.249635 ip-10-0-131-93 kubenswrapper[2564]: I0729 19:19:13.249269 2564 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/node-exporter-95bmx" event={"ID":"09be503d-5e55-4ef7-9339-7f1a9b8c2029","Type":"ContainerStarted","Data":"f81f7c6a06bd723a32ec1b9a4e3ebf842c5fe86a9c3431bccafafbcecaa54af8"} Jul 29 19:19:13.268573 ip-10-0-131-93 kubenswrapper[2564]: I0729 19:19:13.268527 2564 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-monitoring/node-exporter-95bmx" podStartSLOduration=2.390211121 podStartE2EDuration="4.268511909s" totalImagesPullingTime="1.81502847s" totalInitContainerRuntime="63.272318ms" isStatefulPod=true podCreationTimestamp="2026-07-29 19:19:09 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-29 19:19:13.267801358 +0000 UTC m=+128.991861050" watchObservedRunningTime="2026-07-29 19:19:13.268511909 +0000 UTC m=+128.992571601" Jul 29 19:19:13.274711 ip-10-0-131-93 kubenswrapper[2564]: I0729 19:19:13.274688 2564 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-444jx_faa8c546-c75b-4e10-93b4-661aa2b5f53e/konnectivity-agent/0.log" limit=1024 Jul 29 19:19:14.067153 ip-10-0-131-93 kubenswrapper[2564]: I0729 19:19:14.067121 2564 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-444jx_faa8c546-c75b-4e10-93b4-661aa2b5f53e/konnectivity-agent/0.log" limit=1024 Jul 29 19:19:14.167995 ip-10-0-131-93 kubenswrapper[2564]: I0729 19:19:14.167963 2564 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-444jx_faa8c546-c75b-4e10-93b4-661aa2b5f53e/konnectivity-agent/0.log" limit=1024 Jul 29 19:19:14.528398 ip-10-0-131-93 kubenswrapper[2564]: I0729 19:19:14.528323 2564 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/ae220db3-299c-481a-9591-01a8f3873d9a-metrics-certs\") pod \"network-metrics-daemon-hkxvx\" (UID: \"ae220db3-299c-481a-9591-01a8f3873d9a\") " pod="openshift-multus/network-metrics-daemon-hkxvx" Jul 29 19:19:14.530802 ip-10-0-131-93 kubenswrapper[2564]: I0729 19:19:14.530778 2564 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/ae220db3-299c-481a-9591-01a8f3873d9a-metrics-certs\") pod \"network-metrics-daemon-hkxvx\" (UID: \"ae220db3-299c-481a-9591-01a8f3873d9a\") " pod="openshift-multus/network-metrics-daemon-hkxvx" Jul 29 19:19:14.798547 ip-10-0-131-93 kubenswrapper[2564]: I0729 19:19:14.798470 2564 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-multus\"/\"metrics-daemon-sa-dockercfg-tgbwc\"" Jul 29 19:19:14.806103 ip-10-0-131-93 kubenswrapper[2564]: I0729 19:19:14.806086 2564 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-hkxvx" Jul 29 19:19:14.874718 ip-10-0-131-93 kubenswrapper[2564]: I0729 19:19:14.874645 2564 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-444jx_faa8c546-c75b-4e10-93b4-661aa2b5f53e/konnectivity-agent/0.log" limit=1024 Jul 29 19:19:14.934148 ip-10-0-131-93 kubenswrapper[2564]: I0729 19:19:14.934118 2564 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-multus/network-metrics-daemon-hkxvx"] Jul 29 19:19:14.937054 ip-10-0-131-93 kubenswrapper[2564]: W0729 19:19:14.937027 2564 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podae220db3_299c_481a_9591_01a8f3873d9a.slice/crio-87623d5baeec6fdc2c6f43f4714732d817fbdb491967dc558ccbf78d8fc4abe2 WatchSource:0}: Error finding container 87623d5baeec6fdc2c6f43f4714732d817fbdb491967dc558ccbf78d8fc4abe2: Status 404 returned error can't find the container with id 87623d5baeec6fdc2c6f43f4714732d817fbdb491967dc558ccbf78d8fc4abe2 Jul 29 19:19:14.974059 ip-10-0-131-93 kubenswrapper[2564]: I0729 19:19:14.974032 2564 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-444jx_faa8c546-c75b-4e10-93b4-661aa2b5f53e/konnectivity-agent/0.log" limit=1024 Jul 29 19:19:15.256677 ip-10-0-131-93 kubenswrapper[2564]: I0729 19:19:15.256647 2564 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-multus/network-metrics-daemon-hkxvx" event={"ID":"ae220db3-299c-481a-9591-01a8f3873d9a","Type":"ContainerStarted","Data":"87623d5baeec6fdc2c6f43f4714732d817fbdb491967dc558ccbf78d8fc4abe2"} Jul 29 19:19:15.292327 ip-10-0-131-93 kubenswrapper[2564]: I0729 19:19:15.292293 2564 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-444jx_faa8c546-c75b-4e10-93b4-661aa2b5f53e/konnectivity-agent/0.log" limit=1024 Jul 29 19:19:15.387725 ip-10-0-131-93 kubenswrapper[2564]: I0729 19:19:15.387686 2564 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-444jx_faa8c546-c75b-4e10-93b4-661aa2b5f53e/konnectivity-agent/0.log" limit=1024 Jul 29 19:19:16.262394 ip-10-0-131-93 kubenswrapper[2564]: I0729 19:19:16.262350 2564 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-multus/network-metrics-daemon-hkxvx" event={"ID":"ae220db3-299c-481a-9591-01a8f3873d9a","Type":"ContainerStarted","Data":"efc9dece8013de5cde0fba84197a0cf5210496ab72e994bbf9626dbd7706af8c"} Jul 29 19:19:16.262735 ip-10-0-131-93 kubenswrapper[2564]: I0729 19:19:16.262427 2564 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-multus/network-metrics-daemon-hkxvx" event={"ID":"ae220db3-299c-481a-9591-01a8f3873d9a","Type":"ContainerStarted","Data":"f73e55099a24b37c7c8a6edf6ef7944f92cf9d3a500638a06015bda36fda5d45"} Jul 29 19:19:16.289515 ip-10-0-131-93 kubenswrapper[2564]: I0729 19:19:16.289470 2564 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-multus/network-metrics-daemon-hkxvx" podStartSLOduration=131.390094939 podStartE2EDuration="2m12.289454675s" totalImagesPullingTime="899.359736ms" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-29 19:17:04 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-29 19:19:16.28756595 +0000 UTC m=+132.011625643" watchObservedRunningTime="2026-07-29 19:19:16.289454675 +0000 UTC m=+132.013514367" Jul 29 19:19:16.483694 ip-10-0-131-93 kubenswrapper[2564]: I0729 19:19:16.483652 2564 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-444jx_faa8c546-c75b-4e10-93b4-661aa2b5f53e/konnectivity-agent/0.log" limit=1024 Jul 29 19:19:16.584073 ip-10-0-131-93 kubenswrapper[2564]: I0729 19:19:16.583994 2564 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-444jx_faa8c546-c75b-4e10-93b4-661aa2b5f53e/konnectivity-agent/0.log" limit=1024 Jul 29 19:19:19.526016 ip-10-0-131-93 kubenswrapper[2564]: I0729 19:19:19.525980 2564 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-444jx_faa8c546-c75b-4e10-93b4-661aa2b5f53e/konnectivity-agent/0.log" limit=1024 Jul 29 19:19:19.627328 ip-10-0-131-93 kubenswrapper[2564]: I0729 19:19:19.627297 2564 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-444jx_faa8c546-c75b-4e10-93b4-661aa2b5f53e/konnectivity-agent/0.log" limit=1024 Jul 29 19:19:20.822097 ip-10-0-131-93 kubenswrapper[2564]: I0729 19:19:20.822062 2564 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-444jx_faa8c546-c75b-4e10-93b4-661aa2b5f53e/konnectivity-agent/0.log" limit=1024 Jul 29 19:19:21.456543 ip-10-0-131-93 kubenswrapper[2564]: I0729 19:19:21.456509 2564 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-444jx_faa8c546-c75b-4e10-93b4-661aa2b5f53e/konnectivity-agent/0.log" limit=1024 Jul 29 19:19:21.556837 ip-10-0-131-93 kubenswrapper[2564]: I0729 19:19:21.556803 2564 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-444jx_faa8c546-c75b-4e10-93b4-661aa2b5f53e/konnectivity-agent/0.log" limit=1024 Jul 29 19:19:23.606972 ip-10-0-131-93 kubenswrapper[2564]: I0729 19:19:23.606935 2564 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-444jx_faa8c546-c75b-4e10-93b4-661aa2b5f53e/konnectivity-agent/0.log" limit=1024 Jul 29 19:19:23.707667 ip-10-0-131-93 kubenswrapper[2564]: I0729 19:19:23.707638 2564 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-444jx_faa8c546-c75b-4e10-93b4-661aa2b5f53e/konnectivity-agent/0.log" limit=1024 Jul 29 19:19:24.520860 ip-10-0-131-93 kubenswrapper[2564]: I0729 19:19:24.520830 2564 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-444jx_faa8c546-c75b-4e10-93b4-661aa2b5f53e/konnectivity-agent/0.log" limit=1024 Jul 29 19:19:24.621256 ip-10-0-131-93 kubenswrapper[2564]: I0729 19:19:24.621228 2564 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-444jx_faa8c546-c75b-4e10-93b4-661aa2b5f53e/konnectivity-agent/0.log" limit=1024 Jul 29 19:19:24.721284 ip-10-0-131-93 kubenswrapper[2564]: I0729 19:19:24.721252 2564 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-444jx_faa8c546-c75b-4e10-93b4-661aa2b5f53e/konnectivity-agent/0.log" limit=1024 Jul 29 19:19:26.532848 ip-10-0-131-93 kubenswrapper[2564]: I0729 19:19:26.532819 2564 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-444jx_faa8c546-c75b-4e10-93b4-661aa2b5f53e/konnectivity-agent/0.log" limit=1024 Jul 29 19:19:26.589303 ip-10-0-131-93 kubenswrapper[2564]: I0729 19:19:26.589275 2564 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-444jx_faa8c546-c75b-4e10-93b4-661aa2b5f53e/konnectivity-agent/0.log" limit=1024 Jul 29 19:19:28.146075 ip-10-0-131-93 kubenswrapper[2564]: I0729 19:19:28.146032 2564 kuberuntime_container.go:873] "Killing container with a grace period" pod="openshift-image-registry/image-registry-7f95c97455-tkxb8" podUID="5ff17af5-17c3-45e6-b47f-408ba15ed5ca" containerName="registry" containerID="cri-o://7537ab80628a911b100f9f02e182cec7147c9e87cf1063b4425e9d06a18ba234" gracePeriod=30 Jul 29 19:19:28.308321 ip-10-0-131-93 kubenswrapper[2564]: I0729 19:19:28.308292 2564 generic.go:356] "Generic (PLEG): container finished" podID="5ff17af5-17c3-45e6-b47f-408ba15ed5ca" containerID="7537ab80628a911b100f9f02e182cec7147c9e87cf1063b4425e9d06a18ba234" exitCode=0 Jul 29 19:19:28.308449 ip-10-0-131-93 kubenswrapper[2564]: I0729 19:19:28.308334 2564 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-image-registry/image-registry-7f95c97455-tkxb8" event={"ID":"5ff17af5-17c3-45e6-b47f-408ba15ed5ca","Type":"ContainerDied","Data":"7537ab80628a911b100f9f02e182cec7147c9e87cf1063b4425e9d06a18ba234"} Jul 29 19:19:28.503881 ip-10-0-131-93 kubenswrapper[2564]: I0729 19:19:28.503856 2564 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-image-registry/image-registry-7f95c97455-tkxb8" Jul 29 19:19:28.633786 ip-10-0-131-93 kubenswrapper[2564]: I0729 19:19:28.633756 2564 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"bound-sa-token\" (UniqueName: \"kubernetes.io/projected/5ff17af5-17c3-45e6-b47f-408ba15ed5ca-bound-sa-token\") pod \"5ff17af5-17c3-45e6-b47f-408ba15ed5ca\" (UID: \"5ff17af5-17c3-45e6-b47f-408ba15ed5ca\") " Jul 29 19:19:28.633786 ip-10-0-131-93 kubenswrapper[2564]: I0729 19:19:28.633793 2564 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"trusted-ca\" (UniqueName: \"kubernetes.io/configmap/5ff17af5-17c3-45e6-b47f-408ba15ed5ca-trusted-ca\") pod \"5ff17af5-17c3-45e6-b47f-408ba15ed5ca\" (UID: \"5ff17af5-17c3-45e6-b47f-408ba15ed5ca\") " Jul 29 19:19:28.633985 ip-10-0-131-93 kubenswrapper[2564]: I0729 19:19:28.633813 2564 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-wcwpv\" (UniqueName: \"kubernetes.io/projected/5ff17af5-17c3-45e6-b47f-408ba15ed5ca-kube-api-access-wcwpv\") pod \"5ff17af5-17c3-45e6-b47f-408ba15ed5ca\" (UID: \"5ff17af5-17c3-45e6-b47f-408ba15ed5ca\") " Jul 29 19:19:28.633985 ip-10-0-131-93 kubenswrapper[2564]: I0729 19:19:28.633837 2564 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"ca-trust-extracted\" (UniqueName: \"kubernetes.io/empty-dir/5ff17af5-17c3-45e6-b47f-408ba15ed5ca-ca-trust-extracted\") pod \"5ff17af5-17c3-45e6-b47f-408ba15ed5ca\" (UID: \"5ff17af5-17c3-45e6-b47f-408ba15ed5ca\") " Jul 29 19:19:28.633985 ip-10-0-131-93 kubenswrapper[2564]: I0729 19:19:28.633969 2564 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"registry-tls\" (UniqueName: \"kubernetes.io/projected/5ff17af5-17c3-45e6-b47f-408ba15ed5ca-registry-tls\") pod \"5ff17af5-17c3-45e6-b47f-408ba15ed5ca\" (UID: \"5ff17af5-17c3-45e6-b47f-408ba15ed5ca\") " Jul 29 19:19:28.634107 ip-10-0-131-93 kubenswrapper[2564]: I0729 19:19:28.634016 2564 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"installation-pull-secrets\" (UniqueName: \"kubernetes.io/secret/5ff17af5-17c3-45e6-b47f-408ba15ed5ca-installation-pull-secrets\") pod \"5ff17af5-17c3-45e6-b47f-408ba15ed5ca\" (UID: \"5ff17af5-17c3-45e6-b47f-408ba15ed5ca\") " Jul 29 19:19:28.634107 ip-10-0-131-93 kubenswrapper[2564]: I0729 19:19:28.634064 2564 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"image-registry-private-configuration\" (UniqueName: \"kubernetes.io/secret/5ff17af5-17c3-45e6-b47f-408ba15ed5ca-image-registry-private-configuration\") pod \"5ff17af5-17c3-45e6-b47f-408ba15ed5ca\" (UID: \"5ff17af5-17c3-45e6-b47f-408ba15ed5ca\") " Jul 29 19:19:28.634198 ip-10-0-131-93 kubenswrapper[2564]: I0729 19:19:28.634118 2564 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"registry-certificates\" (UniqueName: \"kubernetes.io/configmap/5ff17af5-17c3-45e6-b47f-408ba15ed5ca-registry-certificates\") pod \"5ff17af5-17c3-45e6-b47f-408ba15ed5ca\" (UID: \"5ff17af5-17c3-45e6-b47f-408ba15ed5ca\") " Jul 29 19:19:28.634260 ip-10-0-131-93 kubenswrapper[2564]: I0729 19:19:28.634182 2564 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/5ff17af5-17c3-45e6-b47f-408ba15ed5ca-trusted-ca" (OuterVolumeSpecName: "trusted-ca") pod "5ff17af5-17c3-45e6-b47f-408ba15ed5ca" (UID: "5ff17af5-17c3-45e6-b47f-408ba15ed5ca"). InnerVolumeSpecName "trusted-ca". PluginName "kubernetes.io/configmap", VolumeGIDValue "" Jul 29 19:19:28.634656 ip-10-0-131-93 kubenswrapper[2564]: I0729 19:19:28.634357 2564 reconciler_common.go:299] "Volume detached for volume \"trusted-ca\" (UniqueName: \"kubernetes.io/configmap/5ff17af5-17c3-45e6-b47f-408ba15ed5ca-trusted-ca\") on node \"ip-10-0-131-93.ec2.internal\" DevicePath \"\"" Jul 29 19:19:28.634656 ip-10-0-131-93 kubenswrapper[2564]: I0729 19:19:28.634574 2564 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/5ff17af5-17c3-45e6-b47f-408ba15ed5ca-registry-certificates" (OuterVolumeSpecName: "registry-certificates") pod "5ff17af5-17c3-45e6-b47f-408ba15ed5ca" (UID: "5ff17af5-17c3-45e6-b47f-408ba15ed5ca"). InnerVolumeSpecName "registry-certificates". PluginName "kubernetes.io/configmap", VolumeGIDValue "" Jul 29 19:19:28.636673 ip-10-0-131-93 kubenswrapper[2564]: I0729 19:19:28.636627 2564 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/5ff17af5-17c3-45e6-b47f-408ba15ed5ca-installation-pull-secrets" (OuterVolumeSpecName: "installation-pull-secrets") pod "5ff17af5-17c3-45e6-b47f-408ba15ed5ca" (UID: "5ff17af5-17c3-45e6-b47f-408ba15ed5ca"). InnerVolumeSpecName "installation-pull-secrets". PluginName "kubernetes.io/secret", VolumeGIDValue "" Jul 29 19:19:28.636775 ip-10-0-131-93 kubenswrapper[2564]: I0729 19:19:28.636707 2564 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/5ff17af5-17c3-45e6-b47f-408ba15ed5ca-bound-sa-token" (OuterVolumeSpecName: "bound-sa-token") pod "5ff17af5-17c3-45e6-b47f-408ba15ed5ca" (UID: "5ff17af5-17c3-45e6-b47f-408ba15ed5ca"). InnerVolumeSpecName "bound-sa-token". PluginName "kubernetes.io/projected", VolumeGIDValue "" Jul 29 19:19:28.636775 ip-10-0-131-93 kubenswrapper[2564]: I0729 19:19:28.636715 2564 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/5ff17af5-17c3-45e6-b47f-408ba15ed5ca-registry-tls" (OuterVolumeSpecName: "registry-tls") pod "5ff17af5-17c3-45e6-b47f-408ba15ed5ca" (UID: "5ff17af5-17c3-45e6-b47f-408ba15ed5ca"). InnerVolumeSpecName "registry-tls". PluginName "kubernetes.io/projected", VolumeGIDValue "" Jul 29 19:19:28.636895 ip-10-0-131-93 kubenswrapper[2564]: I0729 19:19:28.636873 2564 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/5ff17af5-17c3-45e6-b47f-408ba15ed5ca-kube-api-access-wcwpv" (OuterVolumeSpecName: "kube-api-access-wcwpv") pod "5ff17af5-17c3-45e6-b47f-408ba15ed5ca" (UID: "5ff17af5-17c3-45e6-b47f-408ba15ed5ca"). InnerVolumeSpecName "kube-api-access-wcwpv". PluginName "kubernetes.io/projected", VolumeGIDValue "" Jul 29 19:19:28.636972 ip-10-0-131-93 kubenswrapper[2564]: I0729 19:19:28.636947 2564 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/5ff17af5-17c3-45e6-b47f-408ba15ed5ca-image-registry-private-configuration" (OuterVolumeSpecName: "image-registry-private-configuration") pod "5ff17af5-17c3-45e6-b47f-408ba15ed5ca" (UID: "5ff17af5-17c3-45e6-b47f-408ba15ed5ca"). InnerVolumeSpecName "image-registry-private-configuration". PluginName "kubernetes.io/secret", VolumeGIDValue "" Jul 29 19:19:28.642283 ip-10-0-131-93 kubenswrapper[2564]: I0729 19:19:28.642259 2564 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/5ff17af5-17c3-45e6-b47f-408ba15ed5ca-ca-trust-extracted" (OuterVolumeSpecName: "ca-trust-extracted") pod "5ff17af5-17c3-45e6-b47f-408ba15ed5ca" (UID: "5ff17af5-17c3-45e6-b47f-408ba15ed5ca"). InnerVolumeSpecName "ca-trust-extracted". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Jul 29 19:19:28.735416 ip-10-0-131-93 kubenswrapper[2564]: I0729 19:19:28.735325 2564 reconciler_common.go:299] "Volume detached for volume \"registry-certificates\" (UniqueName: \"kubernetes.io/configmap/5ff17af5-17c3-45e6-b47f-408ba15ed5ca-registry-certificates\") on node \"ip-10-0-131-93.ec2.internal\" DevicePath \"\"" Jul 29 19:19:28.735416 ip-10-0-131-93 kubenswrapper[2564]: I0729 19:19:28.735350 2564 reconciler_common.go:299] "Volume detached for volume \"bound-sa-token\" (UniqueName: \"kubernetes.io/projected/5ff17af5-17c3-45e6-b47f-408ba15ed5ca-bound-sa-token\") on node \"ip-10-0-131-93.ec2.internal\" DevicePath \"\"" Jul 29 19:19:28.735416 ip-10-0-131-93 kubenswrapper[2564]: I0729 19:19:28.735363 2564 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-wcwpv\" (UniqueName: \"kubernetes.io/projected/5ff17af5-17c3-45e6-b47f-408ba15ed5ca-kube-api-access-wcwpv\") on node \"ip-10-0-131-93.ec2.internal\" DevicePath \"\"" Jul 29 19:19:28.735416 ip-10-0-131-93 kubenswrapper[2564]: I0729 19:19:28.735383 2564 reconciler_common.go:299] "Volume detached for volume \"ca-trust-extracted\" (UniqueName: \"kubernetes.io/empty-dir/5ff17af5-17c3-45e6-b47f-408ba15ed5ca-ca-trust-extracted\") on node \"ip-10-0-131-93.ec2.internal\" DevicePath \"\"" Jul 29 19:19:28.735416 ip-10-0-131-93 kubenswrapper[2564]: I0729 19:19:28.735394 2564 reconciler_common.go:299] "Volume detached for volume \"registry-tls\" (UniqueName: \"kubernetes.io/projected/5ff17af5-17c3-45e6-b47f-408ba15ed5ca-registry-tls\") on node \"ip-10-0-131-93.ec2.internal\" DevicePath \"\"" Jul 29 19:19:28.735416 ip-10-0-131-93 kubenswrapper[2564]: I0729 19:19:28.735402 2564 reconciler_common.go:299] "Volume detached for volume \"installation-pull-secrets\" (UniqueName: \"kubernetes.io/secret/5ff17af5-17c3-45e6-b47f-408ba15ed5ca-installation-pull-secrets\") on node \"ip-10-0-131-93.ec2.internal\" DevicePath \"\"" Jul 29 19:19:28.735416 ip-10-0-131-93 kubenswrapper[2564]: I0729 19:19:28.735410 2564 reconciler_common.go:299] "Volume detached for volume \"image-registry-private-configuration\" (UniqueName: \"kubernetes.io/secret/5ff17af5-17c3-45e6-b47f-408ba15ed5ca-image-registry-private-configuration\") on node \"ip-10-0-131-93.ec2.internal\" DevicePath \"\"" Jul 29 19:19:29.312817 ip-10-0-131-93 kubenswrapper[2564]: I0729 19:19:29.312787 2564 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-image-registry/image-registry-7f95c97455-tkxb8" Jul 29 19:19:29.312817 ip-10-0-131-93 kubenswrapper[2564]: I0729 19:19:29.312800 2564 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-image-registry/image-registry-7f95c97455-tkxb8" event={"ID":"5ff17af5-17c3-45e6-b47f-408ba15ed5ca","Type":"ContainerDied","Data":"ed8d5f94988d01ddbf2469045e30a230a3425dfef797c7e67b3bf4af4a9e3293"} Jul 29 19:19:29.313300 ip-10-0-131-93 kubenswrapper[2564]: I0729 19:19:29.312849 2564 scope.go:117] "RemoveContainer" containerID="7537ab80628a911b100f9f02e182cec7147c9e87cf1063b4425e9d06a18ba234" Jul 29 19:19:29.331623 ip-10-0-131-93 kubenswrapper[2564]: I0729 19:19:29.331595 2564 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-image-registry/image-registry-7f95c97455-tkxb8"] Jul 29 19:19:29.336365 ip-10-0-131-93 kubenswrapper[2564]: I0729 19:19:29.336340 2564 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["openshift-image-registry/image-registry-7f95c97455-tkxb8"] Jul 29 19:19:30.781216 ip-10-0-131-93 kubenswrapper[2564]: I0729 19:19:30.781179 2564 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="5ff17af5-17c3-45e6-b47f-408ba15ed5ca" path="/var/lib/kubelet/pods/5ff17af5-17c3-45e6-b47f-408ba15ed5ca/volumes" Jul 29 19:19:38.554100 ip-10-0-131-93 kubenswrapper[2564]: I0729 19:19:38.554065 2564 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-444jx_faa8c546-c75b-4e10-93b4-661aa2b5f53e/konnectivity-agent/0.log" limit=1024 Jul 29 19:19:39.955282 ip-10-0-131-93 kubenswrapper[2564]: I0729 19:19:39.955244 2564 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-444jx_faa8c546-c75b-4e10-93b4-661aa2b5f53e/konnectivity-agent/0.log" limit=1024 Jul 29 19:19:40.017435 ip-10-0-131-93 kubenswrapper[2564]: I0729 19:19:40.017407 2564 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-444jx_faa8c546-c75b-4e10-93b4-661aa2b5f53e/konnectivity-agent/0.log" limit=1024 Jul 29 19:19:44.852633 ip-10-0-131-93 kubenswrapper[2564]: I0729 19:19:44.852603 2564 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-444jx_faa8c546-c75b-4e10-93b4-661aa2b5f53e/konnectivity-agent/0.log" limit=1024 Jul 29 19:19:44.953805 ip-10-0-131-93 kubenswrapper[2564]: I0729 19:19:44.953775 2564 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-444jx_faa8c546-c75b-4e10-93b4-661aa2b5f53e/konnectivity-agent/0.log" limit=1024 Jul 29 19:20:28.272748 ip-10-0-131-93 kubenswrapper[2564]: I0729 19:20:28.272714 2564 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-444jx_faa8c546-c75b-4e10-93b4-661aa2b5f53e/konnectivity-agent/0.log" limit=1024 Jul 29 19:20:28.549410 ip-10-0-131-93 kubenswrapper[2564]: I0729 19:20:28.549308 2564 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-444jx_faa8c546-c75b-4e10-93b4-661aa2b5f53e/konnectivity-agent/0.log" limit=1024 Jul 29 19:20:37.066255 ip-10-0-131-93 kubenswrapper[2564]: I0729 19:20:37.066194 2564 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-444jx_faa8c546-c75b-4e10-93b4-661aa2b5f53e/konnectivity-agent/0.log" limit=1024 Jul 29 19:20:38.127040 ip-10-0-131-93 kubenswrapper[2564]: I0729 19:20:38.127008 2564 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-444jx_faa8c546-c75b-4e10-93b4-661aa2b5f53e/konnectivity-agent/0.log" limit=1024 Jul 29 19:20:38.187085 ip-10-0-131-93 kubenswrapper[2564]: I0729 19:20:38.187051 2564 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-444jx_faa8c546-c75b-4e10-93b4-661aa2b5f53e/konnectivity-agent/0.log" limit=1024 Jul 29 19:20:43.549433 ip-10-0-131-93 kubenswrapper[2564]: I0729 19:20:43.549401 2564 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-444jx_faa8c546-c75b-4e10-93b4-661aa2b5f53e/konnectivity-agent/0.log" limit=1024 Jul 29 19:20:43.652207 ip-10-0-131-93 kubenswrapper[2564]: I0729 19:20:43.652171 2564 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-444jx_faa8c546-c75b-4e10-93b4-661aa2b5f53e/konnectivity-agent/0.log" limit=1024 Jul 29 19:20:55.481106 ip-10-0-131-93 kubenswrapper[2564]: I0729 19:20:55.481044 2564 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-444jx_faa8c546-c75b-4e10-93b4-661aa2b5f53e/konnectivity-agent/0.log" limit=1024 Jul 29 19:20:55.539140 ip-10-0-131-93 kubenswrapper[2564]: I0729 19:20:55.539092 2564 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-444jx_faa8c546-c75b-4e10-93b4-661aa2b5f53e/konnectivity-agent/0.log" limit=1024 Jul 29 19:20:55.876867 ip-10-0-131-93 kubenswrapper[2564]: I0729 19:20:55.876777 2564 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-444jx_faa8c546-c75b-4e10-93b4-661aa2b5f53e/konnectivity-agent/0.log" limit=1024 Jul 29 19:20:55.977276 ip-10-0-131-93 kubenswrapper[2564]: I0729 19:20:55.977226 2564 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-444jx_faa8c546-c75b-4e10-93b4-661aa2b5f53e/konnectivity-agent/0.log" limit=1024 Jul 29 19:20:56.036452 ip-10-0-131-93 kubenswrapper[2564]: I0729 19:20:56.036422 2564 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-444jx_faa8c546-c75b-4e10-93b4-661aa2b5f53e/konnectivity-agent/0.log" limit=1024 Jul 29 19:20:59.861569 ip-10-0-131-93 kubenswrapper[2564]: I0729 19:20:59.861528 2564 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-444jx_faa8c546-c75b-4e10-93b4-661aa2b5f53e/konnectivity-agent/0.log" limit=1024 Jul 29 19:20:59.962580 ip-10-0-131-93 kubenswrapper[2564]: I0729 19:20:59.962529 2564 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-444jx_faa8c546-c75b-4e10-93b4-661aa2b5f53e/konnectivity-agent/0.log" limit=1024 Jul 29 19:21:49.253361 ip-10-0-131-93 kubenswrapper[2564]: I0729 19:21:49.253326 2564 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-444jx_faa8c546-c75b-4e10-93b4-661aa2b5f53e/konnectivity-agent/0.log" limit=1024 Jul 29 19:21:49.351892 ip-10-0-131-93 kubenswrapper[2564]: I0729 19:21:49.351865 2564 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-444jx_faa8c546-c75b-4e10-93b4-661aa2b5f53e/konnectivity-agent/0.log" limit=1024 Jul 29 19:21:49.893602 ip-10-0-131-93 kubenswrapper[2564]: I0729 19:21:49.893568 2564 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-marketplace/685f1ed643dcce69727e2826bbf9be585096b4cf774051b5daa5e67540zdn26"] Jul 29 19:21:49.893804 ip-10-0-131-93 kubenswrapper[2564]: I0729 19:21:49.893792 2564 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="5ff17af5-17c3-45e6-b47f-408ba15ed5ca" containerName="registry" Jul 29 19:21:49.893848 ip-10-0-131-93 kubenswrapper[2564]: I0729 19:21:49.893805 2564 state_mem.go:107] "Deleted CPUSet assignment" podUID="5ff17af5-17c3-45e6-b47f-408ba15ed5ca" containerName="registry" Jul 29 19:21:49.893881 ip-10-0-131-93 kubenswrapper[2564]: I0729 19:21:49.893866 2564 memory_manager.go:368] "RemoveStaleState removing state" podUID="5ff17af5-17c3-45e6-b47f-408ba15ed5ca" containerName="registry" Jul 29 19:21:49.895728 ip-10-0-131-93 kubenswrapper[2564]: I0729 19:21:49.895714 2564 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-marketplace/685f1ed643dcce69727e2826bbf9be585096b4cf774051b5daa5e67540zdn26" Jul 29 19:21:49.902769 ip-10-0-131-93 kubenswrapper[2564]: I0729 19:21:49.902743 2564 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-marketplace\"/\"default-dockercfg-7n999\"" Jul 29 19:21:49.903054 ip-10-0-131-93 kubenswrapper[2564]: I0729 19:21:49.903034 2564 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-marketplace\"/\"openshift-service-ca.crt\"" Jul 29 19:21:49.903520 ip-10-0-131-93 kubenswrapper[2564]: I0729 19:21:49.903154 2564 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-marketplace\"/\"kube-root-ca.crt\"" Jul 29 19:21:49.909852 ip-10-0-131-93 kubenswrapper[2564]: I0729 19:21:49.909821 2564 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-marketplace/685f1ed643dcce69727e2826bbf9be585096b4cf774051b5daa5e67540zdn26"] Jul 29 19:21:49.981185 ip-10-0-131-93 kubenswrapper[2564]: I0729 19:21:49.981143 2564 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"bundle\" (UniqueName: \"kubernetes.io/empty-dir/ddd90981-94ee-4b7c-b34e-f3904327d84c-bundle\") pod \"685f1ed643dcce69727e2826bbf9be585096b4cf774051b5daa5e67540zdn26\" (UID: \"ddd90981-94ee-4b7c-b34e-f3904327d84c\") " pod="openshift-marketplace/685f1ed643dcce69727e2826bbf9be585096b4cf774051b5daa5e67540zdn26" Jul 29 19:21:49.981399 ip-10-0-131-93 kubenswrapper[2564]: I0729 19:21:49.981199 2564 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"util\" (UniqueName: \"kubernetes.io/empty-dir/ddd90981-94ee-4b7c-b34e-f3904327d84c-util\") pod \"685f1ed643dcce69727e2826bbf9be585096b4cf774051b5daa5e67540zdn26\" (UID: \"ddd90981-94ee-4b7c-b34e-f3904327d84c\") " pod="openshift-marketplace/685f1ed643dcce69727e2826bbf9be585096b4cf774051b5daa5e67540zdn26" Jul 29 19:21:49.981399 ip-10-0-131-93 kubenswrapper[2564]: I0729 19:21:49.981279 2564 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-j9gjc\" (UniqueName: \"kubernetes.io/projected/ddd90981-94ee-4b7c-b34e-f3904327d84c-kube-api-access-j9gjc\") pod \"685f1ed643dcce69727e2826bbf9be585096b4cf774051b5daa5e67540zdn26\" (UID: \"ddd90981-94ee-4b7c-b34e-f3904327d84c\") " pod="openshift-marketplace/685f1ed643dcce69727e2826bbf9be585096b4cf774051b5daa5e67540zdn26" Jul 29 19:21:50.082408 ip-10-0-131-93 kubenswrapper[2564]: I0729 19:21:50.082362 2564 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"util\" (UniqueName: \"kubernetes.io/empty-dir/ddd90981-94ee-4b7c-b34e-f3904327d84c-util\") pod \"685f1ed643dcce69727e2826bbf9be585096b4cf774051b5daa5e67540zdn26\" (UID: \"ddd90981-94ee-4b7c-b34e-f3904327d84c\") " pod="openshift-marketplace/685f1ed643dcce69727e2826bbf9be585096b4cf774051b5daa5e67540zdn26" Jul 29 19:21:50.082522 ip-10-0-131-93 kubenswrapper[2564]: I0729 19:21:50.082420 2564 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-j9gjc\" (UniqueName: \"kubernetes.io/projected/ddd90981-94ee-4b7c-b34e-f3904327d84c-kube-api-access-j9gjc\") pod \"685f1ed643dcce69727e2826bbf9be585096b4cf774051b5daa5e67540zdn26\" (UID: \"ddd90981-94ee-4b7c-b34e-f3904327d84c\") " pod="openshift-marketplace/685f1ed643dcce69727e2826bbf9be585096b4cf774051b5daa5e67540zdn26" Jul 29 19:21:50.082522 ip-10-0-131-93 kubenswrapper[2564]: I0729 19:21:50.082459 2564 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"bundle\" (UniqueName: \"kubernetes.io/empty-dir/ddd90981-94ee-4b7c-b34e-f3904327d84c-bundle\") pod \"685f1ed643dcce69727e2826bbf9be585096b4cf774051b5daa5e67540zdn26\" (UID: \"ddd90981-94ee-4b7c-b34e-f3904327d84c\") " pod="openshift-marketplace/685f1ed643dcce69727e2826bbf9be585096b4cf774051b5daa5e67540zdn26" Jul 29 19:21:50.082762 ip-10-0-131-93 kubenswrapper[2564]: I0729 19:21:50.082741 2564 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"util\" (UniqueName: \"kubernetes.io/empty-dir/ddd90981-94ee-4b7c-b34e-f3904327d84c-util\") pod \"685f1ed643dcce69727e2826bbf9be585096b4cf774051b5daa5e67540zdn26\" (UID: \"ddd90981-94ee-4b7c-b34e-f3904327d84c\") " pod="openshift-marketplace/685f1ed643dcce69727e2826bbf9be585096b4cf774051b5daa5e67540zdn26" Jul 29 19:21:50.082820 ip-10-0-131-93 kubenswrapper[2564]: I0729 19:21:50.082764 2564 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"bundle\" (UniqueName: \"kubernetes.io/empty-dir/ddd90981-94ee-4b7c-b34e-f3904327d84c-bundle\") pod \"685f1ed643dcce69727e2826bbf9be585096b4cf774051b5daa5e67540zdn26\" (UID: \"ddd90981-94ee-4b7c-b34e-f3904327d84c\") " pod="openshift-marketplace/685f1ed643dcce69727e2826bbf9be585096b4cf774051b5daa5e67540zdn26" Jul 29 19:21:50.090969 ip-10-0-131-93 kubenswrapper[2564]: I0729 19:21:50.090949 2564 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-j9gjc\" (UniqueName: \"kubernetes.io/projected/ddd90981-94ee-4b7c-b34e-f3904327d84c-kube-api-access-j9gjc\") pod \"685f1ed643dcce69727e2826bbf9be585096b4cf774051b5daa5e67540zdn26\" (UID: \"ddd90981-94ee-4b7c-b34e-f3904327d84c\") " pod="openshift-marketplace/685f1ed643dcce69727e2826bbf9be585096b4cf774051b5daa5e67540zdn26" Jul 29 19:21:50.206213 ip-10-0-131-93 kubenswrapper[2564]: I0729 19:21:50.206179 2564 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-marketplace/685f1ed643dcce69727e2826bbf9be585096b4cf774051b5daa5e67540zdn26" Jul 29 19:21:50.328834 ip-10-0-131-93 kubenswrapper[2564]: I0729 19:21:50.328811 2564 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-marketplace/685f1ed643dcce69727e2826bbf9be585096b4cf774051b5daa5e67540zdn26"] Jul 29 19:21:50.331228 ip-10-0-131-93 kubenswrapper[2564]: W0729 19:21:50.331196 2564 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podddd90981_94ee_4b7c_b34e_f3904327d84c.slice/crio-04e0d0392c69c0b26a901faf01af2bd237553fda1dcc52ee7f77062672a8f26f WatchSource:0}: Error finding container 04e0d0392c69c0b26a901faf01af2bd237553fda1dcc52ee7f77062672a8f26f: Status 404 returned error can't find the container with id 04e0d0392c69c0b26a901faf01af2bd237553fda1dcc52ee7f77062672a8f26f Jul 29 19:21:50.759387 ip-10-0-131-93 kubenswrapper[2564]: I0729 19:21:50.759340 2564 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-marketplace/685f1ed643dcce69727e2826bbf9be585096b4cf774051b5daa5e67540zdn26" event={"ID":"ddd90981-94ee-4b7c-b34e-f3904327d84c","Type":"ContainerStarted","Data":"04e0d0392c69c0b26a901faf01af2bd237553fda1dcc52ee7f77062672a8f26f"} Jul 29 19:21:57.785555 ip-10-0-131-93 kubenswrapper[2564]: I0729 19:21:57.785517 2564 generic.go:356] "Generic (PLEG): container finished" podID="ddd90981-94ee-4b7c-b34e-f3904327d84c" containerID="d4e7becb5ca8a015ec32246097411eaf4d5914e819b2bb5df9de11ed64715f19" exitCode=0 Jul 29 19:21:57.785910 ip-10-0-131-93 kubenswrapper[2564]: I0729 19:21:57.785573 2564 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-marketplace/685f1ed643dcce69727e2826bbf9be585096b4cf774051b5daa5e67540zdn26" event={"ID":"ddd90981-94ee-4b7c-b34e-f3904327d84c","Type":"ContainerDied","Data":"d4e7becb5ca8a015ec32246097411eaf4d5914e819b2bb5df9de11ed64715f19"} Jul 29 19:21:59.794276 ip-10-0-131-93 kubenswrapper[2564]: I0729 19:21:59.794241 2564 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-marketplace/685f1ed643dcce69727e2826bbf9be585096b4cf774051b5daa5e67540zdn26" event={"ID":"ddd90981-94ee-4b7c-b34e-f3904327d84c","Type":"ContainerStarted","Data":"6f24c252eef3a6a0acc688399104a9ac2fac51616909d903a05416738c040b50"} Jul 29 19:22:00.798684 ip-10-0-131-93 kubenswrapper[2564]: I0729 19:22:00.798650 2564 generic.go:356] "Generic (PLEG): container finished" podID="ddd90981-94ee-4b7c-b34e-f3904327d84c" containerID="6f24c252eef3a6a0acc688399104a9ac2fac51616909d903a05416738c040b50" exitCode=0 Jul 29 19:22:00.799051 ip-10-0-131-93 kubenswrapper[2564]: I0729 19:22:00.798723 2564 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-marketplace/685f1ed643dcce69727e2826bbf9be585096b4cf774051b5daa5e67540zdn26" event={"ID":"ddd90981-94ee-4b7c-b34e-f3904327d84c","Type":"ContainerDied","Data":"6f24c252eef3a6a0acc688399104a9ac2fac51616909d903a05416738c040b50"} Jul 29 19:22:04.637536 ip-10-0-131-93 kubenswrapper[2564]: I0729 19:22:04.637508 2564 kubelet.go:1648] "Image garbage collection succeeded" Jul 29 19:22:07.827320 ip-10-0-131-93 kubenswrapper[2564]: I0729 19:22:07.827285 2564 generic.go:356] "Generic (PLEG): container finished" podID="ddd90981-94ee-4b7c-b34e-f3904327d84c" containerID="a36c70a3c5a6a3f944a2e95a535f408469b8675843b1b9a8d6cdf75d391617a4" exitCode=0 Jul 29 19:22:07.827700 ip-10-0-131-93 kubenswrapper[2564]: I0729 19:22:07.827340 2564 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-marketplace/685f1ed643dcce69727e2826bbf9be585096b4cf774051b5daa5e67540zdn26" event={"ID":"ddd90981-94ee-4b7c-b34e-f3904327d84c","Type":"ContainerDied","Data":"a36c70a3c5a6a3f944a2e95a535f408469b8675843b1b9a8d6cdf75d391617a4"} Jul 29 19:22:08.949604 ip-10-0-131-93 kubenswrapper[2564]: I0729 19:22:08.949580 2564 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-marketplace/685f1ed643dcce69727e2826bbf9be585096b4cf774051b5daa5e67540zdn26" Jul 29 19:22:09.028356 ip-10-0-131-93 kubenswrapper[2564]: I0729 19:22:09.028319 2564 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"bundle\" (UniqueName: \"kubernetes.io/empty-dir/ddd90981-94ee-4b7c-b34e-f3904327d84c-bundle\") pod \"ddd90981-94ee-4b7c-b34e-f3904327d84c\" (UID: \"ddd90981-94ee-4b7c-b34e-f3904327d84c\") " Jul 29 19:22:09.028356 ip-10-0-131-93 kubenswrapper[2564]: I0729 19:22:09.028360 2564 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"util\" (UniqueName: \"kubernetes.io/empty-dir/ddd90981-94ee-4b7c-b34e-f3904327d84c-util\") pod \"ddd90981-94ee-4b7c-b34e-f3904327d84c\" (UID: \"ddd90981-94ee-4b7c-b34e-f3904327d84c\") " Jul 29 19:22:09.028596 ip-10-0-131-93 kubenswrapper[2564]: I0729 19:22:09.028444 2564 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-j9gjc\" (UniqueName: \"kubernetes.io/projected/ddd90981-94ee-4b7c-b34e-f3904327d84c-kube-api-access-j9gjc\") pod \"ddd90981-94ee-4b7c-b34e-f3904327d84c\" (UID: \"ddd90981-94ee-4b7c-b34e-f3904327d84c\") " Jul 29 19:22:09.029061 ip-10-0-131-93 kubenswrapper[2564]: I0729 19:22:09.029035 2564 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/ddd90981-94ee-4b7c-b34e-f3904327d84c-bundle" (OuterVolumeSpecName: "bundle") pod "ddd90981-94ee-4b7c-b34e-f3904327d84c" (UID: "ddd90981-94ee-4b7c-b34e-f3904327d84c"). InnerVolumeSpecName "bundle". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Jul 29 19:22:09.030626 ip-10-0-131-93 kubenswrapper[2564]: I0729 19:22:09.030599 2564 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/ddd90981-94ee-4b7c-b34e-f3904327d84c-kube-api-access-j9gjc" (OuterVolumeSpecName: "kube-api-access-j9gjc") pod "ddd90981-94ee-4b7c-b34e-f3904327d84c" (UID: "ddd90981-94ee-4b7c-b34e-f3904327d84c"). InnerVolumeSpecName "kube-api-access-j9gjc". PluginName "kubernetes.io/projected", VolumeGIDValue "" Jul 29 19:22:09.032715 ip-10-0-131-93 kubenswrapper[2564]: I0729 19:22:09.032684 2564 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/ddd90981-94ee-4b7c-b34e-f3904327d84c-util" (OuterVolumeSpecName: "util") pod "ddd90981-94ee-4b7c-b34e-f3904327d84c" (UID: "ddd90981-94ee-4b7c-b34e-f3904327d84c"). InnerVolumeSpecName "util". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Jul 29 19:22:09.129759 ip-10-0-131-93 kubenswrapper[2564]: I0729 19:22:09.129683 2564 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-j9gjc\" (UniqueName: \"kubernetes.io/projected/ddd90981-94ee-4b7c-b34e-f3904327d84c-kube-api-access-j9gjc\") on node \"ip-10-0-131-93.ec2.internal\" DevicePath \"\"" Jul 29 19:22:09.129759 ip-10-0-131-93 kubenswrapper[2564]: I0729 19:22:09.129710 2564 reconciler_common.go:299] "Volume detached for volume \"bundle\" (UniqueName: \"kubernetes.io/empty-dir/ddd90981-94ee-4b7c-b34e-f3904327d84c-bundle\") on node \"ip-10-0-131-93.ec2.internal\" DevicePath \"\"" Jul 29 19:22:09.129759 ip-10-0-131-93 kubenswrapper[2564]: I0729 19:22:09.129733 2564 reconciler_common.go:299] "Volume detached for volume \"util\" (UniqueName: \"kubernetes.io/empty-dir/ddd90981-94ee-4b7c-b34e-f3904327d84c-util\") on node \"ip-10-0-131-93.ec2.internal\" DevicePath \"\"" Jul 29 19:22:09.835916 ip-10-0-131-93 kubenswrapper[2564]: I0729 19:22:09.835882 2564 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-marketplace/685f1ed643dcce69727e2826bbf9be585096b4cf774051b5daa5e67540zdn26" event={"ID":"ddd90981-94ee-4b7c-b34e-f3904327d84c","Type":"ContainerDied","Data":"04e0d0392c69c0b26a901faf01af2bd237553fda1dcc52ee7f77062672a8f26f"} Jul 29 19:22:09.835916 ip-10-0-131-93 kubenswrapper[2564]: I0729 19:22:09.835917 2564 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-marketplace/685f1ed643dcce69727e2826bbf9be585096b4cf774051b5daa5e67540zdn26" Jul 29 19:22:09.835916 ip-10-0-131-93 kubenswrapper[2564]: I0729 19:22:09.835926 2564 pod_container_deletor.go:80] "Container not found in pod's containers" containerID="04e0d0392c69c0b26a901faf01af2bd237553fda1dcc52ee7f77062672a8f26f" Jul 29 19:22:11.102480 ip-10-0-131-93 kubenswrapper[2564]: I0729 19:22:11.102437 2564 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-444jx_faa8c546-c75b-4e10-93b4-661aa2b5f53e/konnectivity-agent/0.log" limit=1024 Jul 29 19:22:12.559522 ip-10-0-131-93 kubenswrapper[2564]: I0729 19:22:12.559478 2564 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-444jx_faa8c546-c75b-4e10-93b4-661aa2b5f53e/konnectivity-agent/0.log" limit=1024 Jul 29 19:22:12.810395 ip-10-0-131-93 kubenswrapper[2564]: I0729 19:22:12.810299 2564 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-444jx_faa8c546-c75b-4e10-93b4-661aa2b5f53e/konnectivity-agent/0.log" limit=1024 Jul 29 19:22:12.951046 ip-10-0-131-93 kubenswrapper[2564]: I0729 19:22:12.951018 2564 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-444jx_faa8c546-c75b-4e10-93b4-661aa2b5f53e/konnectivity-agent/0.log" limit=1024 Jul 29 19:22:13.122839 ip-10-0-131-93 kubenswrapper[2564]: I0729 19:22:13.122761 2564 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["cert-manager-operator/cert-manager-operator-controller-manager-55c444bf86-27dlp"] Jul 29 19:22:13.123087 ip-10-0-131-93 kubenswrapper[2564]: I0729 19:22:13.123071 2564 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="ddd90981-94ee-4b7c-b34e-f3904327d84c" containerName="pull" Jul 29 19:22:13.123164 ip-10-0-131-93 kubenswrapper[2564]: I0729 19:22:13.123090 2564 state_mem.go:107] "Deleted CPUSet assignment" podUID="ddd90981-94ee-4b7c-b34e-f3904327d84c" containerName="pull" Jul 29 19:22:13.123164 ip-10-0-131-93 kubenswrapper[2564]: I0729 19:22:13.123110 2564 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="ddd90981-94ee-4b7c-b34e-f3904327d84c" containerName="util" Jul 29 19:22:13.123164 ip-10-0-131-93 kubenswrapper[2564]: I0729 19:22:13.123119 2564 state_mem.go:107] "Deleted CPUSet assignment" podUID="ddd90981-94ee-4b7c-b34e-f3904327d84c" containerName="util" Jul 29 19:22:13.123164 ip-10-0-131-93 kubenswrapper[2564]: I0729 19:22:13.123133 2564 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="ddd90981-94ee-4b7c-b34e-f3904327d84c" containerName="extract" Jul 29 19:22:13.123164 ip-10-0-131-93 kubenswrapper[2564]: I0729 19:22:13.123142 2564 state_mem.go:107] "Deleted CPUSet assignment" podUID="ddd90981-94ee-4b7c-b34e-f3904327d84c" containerName="extract" Jul 29 19:22:13.123425 ip-10-0-131-93 kubenswrapper[2564]: I0729 19:22:13.123241 2564 memory_manager.go:368] "RemoveStaleState removing state" podUID="ddd90981-94ee-4b7c-b34e-f3904327d84c" containerName="extract" Jul 29 19:22:13.155365 ip-10-0-131-93 kubenswrapper[2564]: I0729 19:22:13.155337 2564 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="cert-manager-operator/cert-manager-operator-controller-manager-55c444bf86-27dlp" Jul 29 19:22:13.160747 ip-10-0-131-93 kubenswrapper[2564]: I0729 19:22:13.160707 2564 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"cert-manager-operator\"/\"kube-root-ca.crt\"" Jul 29 19:22:13.160886 ip-10-0-131-93 kubenswrapper[2564]: I0729 19:22:13.160743 2564 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"cert-manager-operator\"/\"cert-manager-operator-controller-manager-dockercfg-nd9m9\"" Jul 29 19:22:13.160886 ip-10-0-131-93 kubenswrapper[2564]: I0729 19:22:13.160829 2564 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"cert-manager-operator\"/\"openshift-service-ca.crt\"" Jul 29 19:22:13.163161 ip-10-0-131-93 kubenswrapper[2564]: I0729 19:22:13.163139 2564 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["cert-manager-operator/cert-manager-operator-controller-manager-55c444bf86-27dlp"] Jul 29 19:22:13.260513 ip-10-0-131-93 kubenswrapper[2564]: I0729 19:22:13.260477 2564 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-282dm\" (UniqueName: \"kubernetes.io/projected/0d579451-3b41-45e8-a57e-07cb5d704e99-kube-api-access-282dm\") pod \"cert-manager-operator-controller-manager-55c444bf86-27dlp\" (UID: \"0d579451-3b41-45e8-a57e-07cb5d704e99\") " pod="cert-manager-operator/cert-manager-operator-controller-manager-55c444bf86-27dlp" Jul 29 19:22:13.260676 ip-10-0-131-93 kubenswrapper[2564]: I0729 19:22:13.260531 2564 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tmp\" (UniqueName: \"kubernetes.io/empty-dir/0d579451-3b41-45e8-a57e-07cb5d704e99-tmp\") pod \"cert-manager-operator-controller-manager-55c444bf86-27dlp\" (UID: \"0d579451-3b41-45e8-a57e-07cb5d704e99\") " pod="cert-manager-operator/cert-manager-operator-controller-manager-55c444bf86-27dlp" Jul 29 19:22:13.361786 ip-10-0-131-93 kubenswrapper[2564]: I0729 19:22:13.361751 2564 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tmp\" (UniqueName: \"kubernetes.io/empty-dir/0d579451-3b41-45e8-a57e-07cb5d704e99-tmp\") pod \"cert-manager-operator-controller-manager-55c444bf86-27dlp\" (UID: \"0d579451-3b41-45e8-a57e-07cb5d704e99\") " pod="cert-manager-operator/cert-manager-operator-controller-manager-55c444bf86-27dlp" Jul 29 19:22:13.361933 ip-10-0-131-93 kubenswrapper[2564]: I0729 19:22:13.361815 2564 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-282dm\" (UniqueName: \"kubernetes.io/projected/0d579451-3b41-45e8-a57e-07cb5d704e99-kube-api-access-282dm\") pod \"cert-manager-operator-controller-manager-55c444bf86-27dlp\" (UID: \"0d579451-3b41-45e8-a57e-07cb5d704e99\") " pod="cert-manager-operator/cert-manager-operator-controller-manager-55c444bf86-27dlp" Jul 29 19:22:13.362128 ip-10-0-131-93 kubenswrapper[2564]: I0729 19:22:13.362108 2564 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"tmp\" (UniqueName: \"kubernetes.io/empty-dir/0d579451-3b41-45e8-a57e-07cb5d704e99-tmp\") pod \"cert-manager-operator-controller-manager-55c444bf86-27dlp\" (UID: \"0d579451-3b41-45e8-a57e-07cb5d704e99\") " pod="cert-manager-operator/cert-manager-operator-controller-manager-55c444bf86-27dlp" Jul 29 19:22:13.371792 ip-10-0-131-93 kubenswrapper[2564]: I0729 19:22:13.371766 2564 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-282dm\" (UniqueName: \"kubernetes.io/projected/0d579451-3b41-45e8-a57e-07cb5d704e99-kube-api-access-282dm\") pod \"cert-manager-operator-controller-manager-55c444bf86-27dlp\" (UID: \"0d579451-3b41-45e8-a57e-07cb5d704e99\") " pod="cert-manager-operator/cert-manager-operator-controller-manager-55c444bf86-27dlp" Jul 29 19:22:13.466306 ip-10-0-131-93 kubenswrapper[2564]: I0729 19:22:13.466270 2564 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="cert-manager-operator/cert-manager-operator-controller-manager-55c444bf86-27dlp" Jul 29 19:22:13.618925 ip-10-0-131-93 kubenswrapper[2564]: I0729 19:22:13.618901 2564 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["cert-manager-operator/cert-manager-operator-controller-manager-55c444bf86-27dlp"] Jul 29 19:22:13.620938 ip-10-0-131-93 kubenswrapper[2564]: W0729 19:22:13.620911 2564 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod0d579451_3b41_45e8_a57e_07cb5d704e99.slice/crio-04a362a47ed118bd1672ea505182a149b076f8664f95a572799731ab1bde1f34 WatchSource:0}: Error finding container 04a362a47ed118bd1672ea505182a149b076f8664f95a572799731ab1bde1f34: Status 404 returned error can't find the container with id 04a362a47ed118bd1672ea505182a149b076f8664f95a572799731ab1bde1f34 Jul 29 19:22:13.623577 ip-10-0-131-93 kubenswrapper[2564]: I0729 19:22:13.623562 2564 provider.go:93] Refreshing cache for provider: *credentialprovider.defaultDockerConfigProvider Jul 29 19:22:13.856183 ip-10-0-131-93 kubenswrapper[2564]: I0729 19:22:13.856079 2564 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="cert-manager-operator/cert-manager-operator-controller-manager-55c444bf86-27dlp" event={"ID":"0d579451-3b41-45e8-a57e-07cb5d704e99","Type":"ContainerStarted","Data":"04a362a47ed118bd1672ea505182a149b076f8664f95a572799731ab1bde1f34"} Jul 29 19:22:14.932578 ip-10-0-131-93 kubenswrapper[2564]: I0729 19:22:14.932535 2564 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-444jx_faa8c546-c75b-4e10-93b4-661aa2b5f53e/konnectivity-agent/0.log" limit=1024 Jul 29 19:22:15.034338 ip-10-0-131-93 kubenswrapper[2564]: I0729 19:22:15.034312 2564 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-444jx_faa8c546-c75b-4e10-93b4-661aa2b5f53e/konnectivity-agent/0.log" limit=1024 Jul 29 19:22:15.865589 ip-10-0-131-93 kubenswrapper[2564]: I0729 19:22:15.865513 2564 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="cert-manager-operator/cert-manager-operator-controller-manager-55c444bf86-27dlp" event={"ID":"0d579451-3b41-45e8-a57e-07cb5d704e99","Type":"ContainerStarted","Data":"e2dea02d6531622b1747e31a1f949a4db8111742f7243abaa60399739d8e7486"} Jul 29 19:22:15.906128 ip-10-0-131-93 kubenswrapper[2564]: I0729 19:22:15.906082 2564 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="cert-manager-operator/cert-manager-operator-controller-manager-55c444bf86-27dlp" podStartSLOduration=1.034203717 podStartE2EDuration="2.906066832s" totalImagesPullingTime="1.871863115s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-29 19:22:13 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-29 19:22:15.905435581 +0000 UTC m=+311.629495273" watchObservedRunningTime="2026-07-29 19:22:15.906066832 +0000 UTC m=+311.630126523" Jul 29 19:22:16.444561 ip-10-0-131-93 kubenswrapper[2564]: I0729 19:22:16.444525 2564 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-444jx_faa8c546-c75b-4e10-93b4-661aa2b5f53e/konnectivity-agent/0.log" limit=1024 Jul 29 19:22:16.638242 ip-10-0-131-93 kubenswrapper[2564]: I0729 19:22:16.638211 2564 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-444jx_faa8c546-c75b-4e10-93b4-661aa2b5f53e/konnectivity-agent/0.log" limit=1024 Jul 29 19:22:17.639191 ip-10-0-131-93 kubenswrapper[2564]: I0729 19:22:17.639164 2564 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-marketplace/77defbb6647882b321b46d16bdefe62633c3425354d3d93571a1d6a87fsnmgn"] Jul 29 19:22:17.641647 ip-10-0-131-93 kubenswrapper[2564]: I0729 19:22:17.641631 2564 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-marketplace/77defbb6647882b321b46d16bdefe62633c3425354d3d93571a1d6a87fsnmgn" Jul 29 19:22:17.644682 ip-10-0-131-93 kubenswrapper[2564]: I0729 19:22:17.644663 2564 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-marketplace\"/\"kube-root-ca.crt\"" Jul 29 19:22:17.649180 ip-10-0-131-93 kubenswrapper[2564]: I0729 19:22:17.649164 2564 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-marketplace\"/\"openshift-service-ca.crt\"" Jul 29 19:22:17.654864 ip-10-0-131-93 kubenswrapper[2564]: I0729 19:22:17.654849 2564 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-marketplace\"/\"default-dockercfg-7n999\"" Jul 29 19:22:17.682680 ip-10-0-131-93 kubenswrapper[2564]: I0729 19:22:17.682658 2564 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-marketplace/77defbb6647882b321b46d16bdefe62633c3425354d3d93571a1d6a87fsnmgn"] Jul 29 19:22:17.792215 ip-10-0-131-93 kubenswrapper[2564]: I0729 19:22:17.792187 2564 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-t5zhv\" (UniqueName: \"kubernetes.io/projected/8afb765f-b59a-49cd-afe6-5d2ab5152ba9-kube-api-access-t5zhv\") pod \"77defbb6647882b321b46d16bdefe62633c3425354d3d93571a1d6a87fsnmgn\" (UID: \"8afb765f-b59a-49cd-afe6-5d2ab5152ba9\") " pod="openshift-marketplace/77defbb6647882b321b46d16bdefe62633c3425354d3d93571a1d6a87fsnmgn" Jul 29 19:22:17.792397 ip-10-0-131-93 kubenswrapper[2564]: I0729 19:22:17.792224 2564 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"bundle\" (UniqueName: \"kubernetes.io/empty-dir/8afb765f-b59a-49cd-afe6-5d2ab5152ba9-bundle\") pod \"77defbb6647882b321b46d16bdefe62633c3425354d3d93571a1d6a87fsnmgn\" (UID: \"8afb765f-b59a-49cd-afe6-5d2ab5152ba9\") " pod="openshift-marketplace/77defbb6647882b321b46d16bdefe62633c3425354d3d93571a1d6a87fsnmgn" Jul 29 19:22:17.792397 ip-10-0-131-93 kubenswrapper[2564]: I0729 19:22:17.792267 2564 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"util\" (UniqueName: \"kubernetes.io/empty-dir/8afb765f-b59a-49cd-afe6-5d2ab5152ba9-util\") pod \"77defbb6647882b321b46d16bdefe62633c3425354d3d93571a1d6a87fsnmgn\" (UID: \"8afb765f-b59a-49cd-afe6-5d2ab5152ba9\") " pod="openshift-marketplace/77defbb6647882b321b46d16bdefe62633c3425354d3d93571a1d6a87fsnmgn" Jul 29 19:22:17.892881 ip-10-0-131-93 kubenswrapper[2564]: I0729 19:22:17.892780 2564 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"util\" (UniqueName: \"kubernetes.io/empty-dir/8afb765f-b59a-49cd-afe6-5d2ab5152ba9-util\") pod \"77defbb6647882b321b46d16bdefe62633c3425354d3d93571a1d6a87fsnmgn\" (UID: \"8afb765f-b59a-49cd-afe6-5d2ab5152ba9\") " pod="openshift-marketplace/77defbb6647882b321b46d16bdefe62633c3425354d3d93571a1d6a87fsnmgn" Jul 29 19:22:17.893052 ip-10-0-131-93 kubenswrapper[2564]: I0729 19:22:17.892898 2564 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-t5zhv\" (UniqueName: \"kubernetes.io/projected/8afb765f-b59a-49cd-afe6-5d2ab5152ba9-kube-api-access-t5zhv\") pod \"77defbb6647882b321b46d16bdefe62633c3425354d3d93571a1d6a87fsnmgn\" (UID: \"8afb765f-b59a-49cd-afe6-5d2ab5152ba9\") " pod="openshift-marketplace/77defbb6647882b321b46d16bdefe62633c3425354d3d93571a1d6a87fsnmgn" Jul 29 19:22:17.893052 ip-10-0-131-93 kubenswrapper[2564]: I0729 19:22:17.892958 2564 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"bundle\" (UniqueName: \"kubernetes.io/empty-dir/8afb765f-b59a-49cd-afe6-5d2ab5152ba9-bundle\") pod \"77defbb6647882b321b46d16bdefe62633c3425354d3d93571a1d6a87fsnmgn\" (UID: \"8afb765f-b59a-49cd-afe6-5d2ab5152ba9\") " pod="openshift-marketplace/77defbb6647882b321b46d16bdefe62633c3425354d3d93571a1d6a87fsnmgn" Jul 29 19:22:17.893446 ip-10-0-131-93 kubenswrapper[2564]: I0729 19:22:17.893410 2564 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"util\" (UniqueName: \"kubernetes.io/empty-dir/8afb765f-b59a-49cd-afe6-5d2ab5152ba9-util\") pod \"77defbb6647882b321b46d16bdefe62633c3425354d3d93571a1d6a87fsnmgn\" (UID: \"8afb765f-b59a-49cd-afe6-5d2ab5152ba9\") " pod="openshift-marketplace/77defbb6647882b321b46d16bdefe62633c3425354d3d93571a1d6a87fsnmgn" Jul 29 19:22:17.893515 ip-10-0-131-93 kubenswrapper[2564]: I0729 19:22:17.893455 2564 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"bundle\" (UniqueName: \"kubernetes.io/empty-dir/8afb765f-b59a-49cd-afe6-5d2ab5152ba9-bundle\") pod \"77defbb6647882b321b46d16bdefe62633c3425354d3d93571a1d6a87fsnmgn\" (UID: \"8afb765f-b59a-49cd-afe6-5d2ab5152ba9\") " pod="openshift-marketplace/77defbb6647882b321b46d16bdefe62633c3425354d3d93571a1d6a87fsnmgn" Jul 29 19:22:17.903237 ip-10-0-131-93 kubenswrapper[2564]: I0729 19:22:17.903215 2564 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-t5zhv\" (UniqueName: \"kubernetes.io/projected/8afb765f-b59a-49cd-afe6-5d2ab5152ba9-kube-api-access-t5zhv\") pod \"77defbb6647882b321b46d16bdefe62633c3425354d3d93571a1d6a87fsnmgn\" (UID: \"8afb765f-b59a-49cd-afe6-5d2ab5152ba9\") " pod="openshift-marketplace/77defbb6647882b321b46d16bdefe62633c3425354d3d93571a1d6a87fsnmgn" Jul 29 19:22:17.950234 ip-10-0-131-93 kubenswrapper[2564]: I0729 19:22:17.950210 2564 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-marketplace/77defbb6647882b321b46d16bdefe62633c3425354d3d93571a1d6a87fsnmgn" Jul 29 19:22:18.071703 ip-10-0-131-93 kubenswrapper[2564]: I0729 19:22:18.071600 2564 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-marketplace/77defbb6647882b321b46d16bdefe62633c3425354d3d93571a1d6a87fsnmgn"] Jul 29 19:22:18.073676 ip-10-0-131-93 kubenswrapper[2564]: W0729 19:22:18.073654 2564 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod8afb765f_b59a_49cd_afe6_5d2ab5152ba9.slice/crio-746ea4d537041343108dae9a843a6b30ea54b49c0a4559ebc390d4ca0d09ac99 WatchSource:0}: Error finding container 746ea4d537041343108dae9a843a6b30ea54b49c0a4559ebc390d4ca0d09ac99: Status 404 returned error can't find the container with id 746ea4d537041343108dae9a843a6b30ea54b49c0a4559ebc390d4ca0d09ac99 Jul 29 19:22:18.878414 ip-10-0-131-93 kubenswrapper[2564]: I0729 19:22:18.878382 2564 generic.go:356] "Generic (PLEG): container finished" podID="8afb765f-b59a-49cd-afe6-5d2ab5152ba9" containerID="4d0dd9e5e261c51568cd90d32e1cfb6e5be5fca18c96fd4b1018313b8b2c325c" exitCode=0 Jul 29 19:22:18.878877 ip-10-0-131-93 kubenswrapper[2564]: I0729 19:22:18.878482 2564 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-marketplace/77defbb6647882b321b46d16bdefe62633c3425354d3d93571a1d6a87fsnmgn" event={"ID":"8afb765f-b59a-49cd-afe6-5d2ab5152ba9","Type":"ContainerDied","Data":"4d0dd9e5e261c51568cd90d32e1cfb6e5be5fca18c96fd4b1018313b8b2c325c"} Jul 29 19:22:18.878877 ip-10-0-131-93 kubenswrapper[2564]: I0729 19:22:18.878543 2564 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-marketplace/77defbb6647882b321b46d16bdefe62633c3425354d3d93571a1d6a87fsnmgn" event={"ID":"8afb765f-b59a-49cd-afe6-5d2ab5152ba9","Type":"ContainerStarted","Data":"746ea4d537041343108dae9a843a6b30ea54b49c0a4559ebc390d4ca0d09ac99"} Jul 29 19:22:19.061833 ip-10-0-131-93 kubenswrapper[2564]: I0729 19:22:19.061791 2564 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-444jx_faa8c546-c75b-4e10-93b4-661aa2b5f53e/konnectivity-agent/0.log" limit=1024 Jul 29 19:22:19.465135 ip-10-0-131-93 kubenswrapper[2564]: I0729 19:22:19.465106 2564 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-444jx_faa8c546-c75b-4e10-93b4-661aa2b5f53e/konnectivity-agent/0.log" limit=1024 Jul 29 19:22:20.888889 ip-10-0-131-93 kubenswrapper[2564]: I0729 19:22:20.888853 2564 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-marketplace/77defbb6647882b321b46d16bdefe62633c3425354d3d93571a1d6a87fsnmgn" event={"ID":"8afb765f-b59a-49cd-afe6-5d2ab5152ba9","Type":"ContainerStarted","Data":"1447f36d06372b69c0e327367572eb66ee84674079dd417867f723b7c2e9142b"} Jul 29 19:22:21.758584 ip-10-0-131-93 kubenswrapper[2564]: I0729 19:22:21.758549 2564 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-444jx_faa8c546-c75b-4e10-93b4-661aa2b5f53e/konnectivity-agent/0.log" limit=1024 Jul 29 19:22:21.859056 ip-10-0-131-93 kubenswrapper[2564]: I0729 19:22:21.859024 2564 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-444jx_faa8c546-c75b-4e10-93b4-661aa2b5f53e/konnectivity-agent/0.log" limit=1024 Jul 29 19:22:21.893999 ip-10-0-131-93 kubenswrapper[2564]: I0729 19:22:21.893969 2564 generic.go:356] "Generic (PLEG): container finished" podID="8afb765f-b59a-49cd-afe6-5d2ab5152ba9" containerID="1447f36d06372b69c0e327367572eb66ee84674079dd417867f723b7c2e9142b" exitCode=0 Jul 29 19:22:21.894404 ip-10-0-131-93 kubenswrapper[2564]: I0729 19:22:21.894007 2564 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-marketplace/77defbb6647882b321b46d16bdefe62633c3425354d3d93571a1d6a87fsnmgn" event={"ID":"8afb765f-b59a-49cd-afe6-5d2ab5152ba9","Type":"ContainerDied","Data":"1447f36d06372b69c0e327367572eb66ee84674079dd417867f723b7c2e9142b"} Jul 29 19:22:22.898981 ip-10-0-131-93 kubenswrapper[2564]: I0729 19:22:22.898945 2564 generic.go:356] "Generic (PLEG): container finished" podID="8afb765f-b59a-49cd-afe6-5d2ab5152ba9" containerID="9267cabac6e1ecba023080b33a15adcb80a07eef4735cf6c8ffe291221aaae27" exitCode=0 Jul 29 19:22:22.899353 ip-10-0-131-93 kubenswrapper[2564]: I0729 19:22:22.899005 2564 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-marketplace/77defbb6647882b321b46d16bdefe62633c3425354d3d93571a1d6a87fsnmgn" event={"ID":"8afb765f-b59a-49cd-afe6-5d2ab5152ba9","Type":"ContainerDied","Data":"9267cabac6e1ecba023080b33a15adcb80a07eef4735cf6c8ffe291221aaae27"} Jul 29 19:22:24.020311 ip-10-0-131-93 kubenswrapper[2564]: I0729 19:22:24.020288 2564 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-marketplace/77defbb6647882b321b46d16bdefe62633c3425354d3d93571a1d6a87fsnmgn" Jul 29 19:22:24.138910 ip-10-0-131-93 kubenswrapper[2564]: I0729 19:22:24.138882 2564 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"bundle\" (UniqueName: \"kubernetes.io/empty-dir/8afb765f-b59a-49cd-afe6-5d2ab5152ba9-bundle\") pod \"8afb765f-b59a-49cd-afe6-5d2ab5152ba9\" (UID: \"8afb765f-b59a-49cd-afe6-5d2ab5152ba9\") " Jul 29 19:22:24.139061 ip-10-0-131-93 kubenswrapper[2564]: I0729 19:22:24.138925 2564 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-t5zhv\" (UniqueName: \"kubernetes.io/projected/8afb765f-b59a-49cd-afe6-5d2ab5152ba9-kube-api-access-t5zhv\") pod \"8afb765f-b59a-49cd-afe6-5d2ab5152ba9\" (UID: \"8afb765f-b59a-49cd-afe6-5d2ab5152ba9\") " Jul 29 19:22:24.139061 ip-10-0-131-93 kubenswrapper[2564]: I0729 19:22:24.138954 2564 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"util\" (UniqueName: \"kubernetes.io/empty-dir/8afb765f-b59a-49cd-afe6-5d2ab5152ba9-util\") pod \"8afb765f-b59a-49cd-afe6-5d2ab5152ba9\" (UID: \"8afb765f-b59a-49cd-afe6-5d2ab5152ba9\") " Jul 29 19:22:24.139362 ip-10-0-131-93 kubenswrapper[2564]: I0729 19:22:24.139331 2564 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/8afb765f-b59a-49cd-afe6-5d2ab5152ba9-bundle" (OuterVolumeSpecName: "bundle") pod "8afb765f-b59a-49cd-afe6-5d2ab5152ba9" (UID: "8afb765f-b59a-49cd-afe6-5d2ab5152ba9"). InnerVolumeSpecName "bundle". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Jul 29 19:22:24.141122 ip-10-0-131-93 kubenswrapper[2564]: I0729 19:22:24.141096 2564 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/8afb765f-b59a-49cd-afe6-5d2ab5152ba9-kube-api-access-t5zhv" (OuterVolumeSpecName: "kube-api-access-t5zhv") pod "8afb765f-b59a-49cd-afe6-5d2ab5152ba9" (UID: "8afb765f-b59a-49cd-afe6-5d2ab5152ba9"). InnerVolumeSpecName "kube-api-access-t5zhv". PluginName "kubernetes.io/projected", VolumeGIDValue "" Jul 29 19:22:24.143603 ip-10-0-131-93 kubenswrapper[2564]: I0729 19:22:24.143568 2564 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/8afb765f-b59a-49cd-afe6-5d2ab5152ba9-util" (OuterVolumeSpecName: "util") pod "8afb765f-b59a-49cd-afe6-5d2ab5152ba9" (UID: "8afb765f-b59a-49cd-afe6-5d2ab5152ba9"). InnerVolumeSpecName "util". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Jul 29 19:22:24.239578 ip-10-0-131-93 kubenswrapper[2564]: I0729 19:22:24.239553 2564 reconciler_common.go:299] "Volume detached for volume \"bundle\" (UniqueName: \"kubernetes.io/empty-dir/8afb765f-b59a-49cd-afe6-5d2ab5152ba9-bundle\") on node \"ip-10-0-131-93.ec2.internal\" DevicePath \"\"" Jul 29 19:22:24.239578 ip-10-0-131-93 kubenswrapper[2564]: I0729 19:22:24.239579 2564 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-t5zhv\" (UniqueName: \"kubernetes.io/projected/8afb765f-b59a-49cd-afe6-5d2ab5152ba9-kube-api-access-t5zhv\") on node \"ip-10-0-131-93.ec2.internal\" DevicePath \"\"" Jul 29 19:22:24.239772 ip-10-0-131-93 kubenswrapper[2564]: I0729 19:22:24.239588 2564 reconciler_common.go:299] "Volume detached for volume \"util\" (UniqueName: \"kubernetes.io/empty-dir/8afb765f-b59a-49cd-afe6-5d2ab5152ba9-util\") on node \"ip-10-0-131-93.ec2.internal\" DevicePath \"\"" Jul 29 19:22:24.621235 ip-10-0-131-93 kubenswrapper[2564]: I0729 19:22:24.621152 2564 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["cert-manager/cert-manager-webhook-6486ffd997-55mfm"] Jul 29 19:22:24.621442 ip-10-0-131-93 kubenswrapper[2564]: I0729 19:22:24.621429 2564 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="8afb765f-b59a-49cd-afe6-5d2ab5152ba9" containerName="util" Jul 29 19:22:24.621442 ip-10-0-131-93 kubenswrapper[2564]: I0729 19:22:24.621443 2564 state_mem.go:107] "Deleted CPUSet assignment" podUID="8afb765f-b59a-49cd-afe6-5d2ab5152ba9" containerName="util" Jul 29 19:22:24.621540 ip-10-0-131-93 kubenswrapper[2564]: I0729 19:22:24.621461 2564 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="8afb765f-b59a-49cd-afe6-5d2ab5152ba9" containerName="extract" Jul 29 19:22:24.621540 ip-10-0-131-93 kubenswrapper[2564]: I0729 19:22:24.621466 2564 state_mem.go:107] "Deleted CPUSet assignment" podUID="8afb765f-b59a-49cd-afe6-5d2ab5152ba9" containerName="extract" Jul 29 19:22:24.621540 ip-10-0-131-93 kubenswrapper[2564]: I0729 19:22:24.621476 2564 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="8afb765f-b59a-49cd-afe6-5d2ab5152ba9" containerName="pull" Jul 29 19:22:24.621540 ip-10-0-131-93 kubenswrapper[2564]: I0729 19:22:24.621481 2564 state_mem.go:107] "Deleted CPUSet assignment" podUID="8afb765f-b59a-49cd-afe6-5d2ab5152ba9" containerName="pull" Jul 29 19:22:24.621540 ip-10-0-131-93 kubenswrapper[2564]: I0729 19:22:24.621517 2564 memory_manager.go:368] "RemoveStaleState removing state" podUID="8afb765f-b59a-49cd-afe6-5d2ab5152ba9" containerName="extract" Jul 29 19:22:24.623434 ip-10-0-131-93 kubenswrapper[2564]: I0729 19:22:24.623418 2564 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="cert-manager/cert-manager-webhook-6486ffd997-55mfm" Jul 29 19:22:24.626318 ip-10-0-131-93 kubenswrapper[2564]: I0729 19:22:24.626301 2564 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"cert-manager\"/\"openshift-service-ca.crt\"" Jul 29 19:22:24.626672 ip-10-0-131-93 kubenswrapper[2564]: I0729 19:22:24.626653 2564 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"cert-manager\"/\"cert-manager-webhook-dockercfg-fhgl6\"" Jul 29 19:22:24.627340 ip-10-0-131-93 kubenswrapper[2564]: I0729 19:22:24.627327 2564 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"cert-manager\"/\"kube-root-ca.crt\"" Jul 29 19:22:24.634877 ip-10-0-131-93 kubenswrapper[2564]: I0729 19:22:24.634858 2564 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["cert-manager/cert-manager-webhook-6486ffd997-55mfm"] Jul 29 19:22:24.742104 ip-10-0-131-93 kubenswrapper[2564]: I0729 19:22:24.742068 2564 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-z7sj8\" (UniqueName: \"kubernetes.io/projected/9b3de621-c041-4b7a-96d9-41edea12c9a9-kube-api-access-z7sj8\") pod \"cert-manager-webhook-6486ffd997-55mfm\" (UID: \"9b3de621-c041-4b7a-96d9-41edea12c9a9\") " pod="cert-manager/cert-manager-webhook-6486ffd997-55mfm" Jul 29 19:22:24.742289 ip-10-0-131-93 kubenswrapper[2564]: I0729 19:22:24.742130 2564 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"bound-sa-token\" (UniqueName: \"kubernetes.io/projected/9b3de621-c041-4b7a-96d9-41edea12c9a9-bound-sa-token\") pod \"cert-manager-webhook-6486ffd997-55mfm\" (UID: \"9b3de621-c041-4b7a-96d9-41edea12c9a9\") " pod="cert-manager/cert-manager-webhook-6486ffd997-55mfm" Jul 29 19:22:24.842498 ip-10-0-131-93 kubenswrapper[2564]: I0729 19:22:24.842466 2564 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"bound-sa-token\" (UniqueName: \"kubernetes.io/projected/9b3de621-c041-4b7a-96d9-41edea12c9a9-bound-sa-token\") pod \"cert-manager-webhook-6486ffd997-55mfm\" (UID: \"9b3de621-c041-4b7a-96d9-41edea12c9a9\") " pod="cert-manager/cert-manager-webhook-6486ffd997-55mfm" Jul 29 19:22:24.842654 ip-10-0-131-93 kubenswrapper[2564]: I0729 19:22:24.842548 2564 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-z7sj8\" (UniqueName: \"kubernetes.io/projected/9b3de621-c041-4b7a-96d9-41edea12c9a9-kube-api-access-z7sj8\") pod \"cert-manager-webhook-6486ffd997-55mfm\" (UID: \"9b3de621-c041-4b7a-96d9-41edea12c9a9\") " pod="cert-manager/cert-manager-webhook-6486ffd997-55mfm" Jul 29 19:22:24.850898 ip-10-0-131-93 kubenswrapper[2564]: I0729 19:22:24.850867 2564 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"bound-sa-token\" (UniqueName: \"kubernetes.io/projected/9b3de621-c041-4b7a-96d9-41edea12c9a9-bound-sa-token\") pod \"cert-manager-webhook-6486ffd997-55mfm\" (UID: \"9b3de621-c041-4b7a-96d9-41edea12c9a9\") " pod="cert-manager/cert-manager-webhook-6486ffd997-55mfm" Jul 29 19:22:24.851052 ip-10-0-131-93 kubenswrapper[2564]: I0729 19:22:24.851034 2564 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-z7sj8\" (UniqueName: \"kubernetes.io/projected/9b3de621-c041-4b7a-96d9-41edea12c9a9-kube-api-access-z7sj8\") pod \"cert-manager-webhook-6486ffd997-55mfm\" (UID: \"9b3de621-c041-4b7a-96d9-41edea12c9a9\") " pod="cert-manager/cert-manager-webhook-6486ffd997-55mfm" Jul 29 19:22:24.908545 ip-10-0-131-93 kubenswrapper[2564]: I0729 19:22:24.908463 2564 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-marketplace/77defbb6647882b321b46d16bdefe62633c3425354d3d93571a1d6a87fsnmgn" event={"ID":"8afb765f-b59a-49cd-afe6-5d2ab5152ba9","Type":"ContainerDied","Data":"746ea4d537041343108dae9a843a6b30ea54b49c0a4559ebc390d4ca0d09ac99"} Jul 29 19:22:24.908545 ip-10-0-131-93 kubenswrapper[2564]: I0729 19:22:24.908500 2564 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-marketplace/77defbb6647882b321b46d16bdefe62633c3425354d3d93571a1d6a87fsnmgn" Jul 29 19:22:24.908545 ip-10-0-131-93 kubenswrapper[2564]: I0729 19:22:24.908508 2564 pod_container_deletor.go:80] "Container not found in pod's containers" containerID="746ea4d537041343108dae9a843a6b30ea54b49c0a4559ebc390d4ca0d09ac99" Jul 29 19:22:24.932676 ip-10-0-131-93 kubenswrapper[2564]: I0729 19:22:24.932652 2564 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="cert-manager/cert-manager-webhook-6486ffd997-55mfm" Jul 29 19:22:25.057566 ip-10-0-131-93 kubenswrapper[2564]: I0729 19:22:25.057540 2564 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["cert-manager/cert-manager-webhook-6486ffd997-55mfm"] Jul 29 19:22:25.059795 ip-10-0-131-93 kubenswrapper[2564]: W0729 19:22:25.059768 2564 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-besteffort.slice/kubepods-besteffort-pod9b3de621_c041_4b7a_96d9_41edea12c9a9.slice/crio-17054f649d2bcef6c5173a3e746e473c083a16acd15e8c12637cefde388e2330 WatchSource:0}: Error finding container 17054f649d2bcef6c5173a3e746e473c083a16acd15e8c12637cefde388e2330: Status 404 returned error can't find the container with id 17054f649d2bcef6c5173a3e746e473c083a16acd15e8c12637cefde388e2330 Jul 29 19:22:25.913760 ip-10-0-131-93 kubenswrapper[2564]: I0729 19:22:25.913675 2564 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="cert-manager/cert-manager-webhook-6486ffd997-55mfm" event={"ID":"9b3de621-c041-4b7a-96d9-41edea12c9a9","Type":"ContainerStarted","Data":"17054f649d2bcef6c5173a3e746e473c083a16acd15e8c12637cefde388e2330"} Jul 29 19:22:26.289350 ip-10-0-131-93 kubenswrapper[2564]: I0729 19:22:26.289318 2564 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-444jx_faa8c546-c75b-4e10-93b4-661aa2b5f53e/konnectivity-agent/0.log" limit=1024 Jul 29 19:22:27.041368 ip-10-0-131-93 kubenswrapper[2564]: I0729 19:22:27.041332 2564 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["cert-manager/cert-manager-55b4d954d5-nwbvk"] Jul 29 19:22:27.043714 ip-10-0-131-93 kubenswrapper[2564]: I0729 19:22:27.043696 2564 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="cert-manager/cert-manager-55b4d954d5-nwbvk" Jul 29 19:22:27.046415 ip-10-0-131-93 kubenswrapper[2564]: I0729 19:22:27.046391 2564 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"cert-manager\"/\"cert-manager-dockercfg-c2xts\"" Jul 29 19:22:27.055408 ip-10-0-131-93 kubenswrapper[2564]: I0729 19:22:27.055367 2564 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["cert-manager/cert-manager-55b4d954d5-nwbvk"] Jul 29 19:22:27.162096 ip-10-0-131-93 kubenswrapper[2564]: I0729 19:22:27.162062 2564 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"bound-sa-token\" (UniqueName: \"kubernetes.io/projected/afb8c055-366b-4658-87b3-9ac9db127c69-bound-sa-token\") pod \"cert-manager-55b4d954d5-nwbvk\" (UID: \"afb8c055-366b-4658-87b3-9ac9db127c69\") " pod="cert-manager/cert-manager-55b4d954d5-nwbvk" Jul 29 19:22:27.162283 ip-10-0-131-93 kubenswrapper[2564]: I0729 19:22:27.162124 2564 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-mbgjv\" (UniqueName: \"kubernetes.io/projected/afb8c055-366b-4658-87b3-9ac9db127c69-kube-api-access-mbgjv\") pod \"cert-manager-55b4d954d5-nwbvk\" (UID: \"afb8c055-366b-4658-87b3-9ac9db127c69\") " pod="cert-manager/cert-manager-55b4d954d5-nwbvk" Jul 29 19:22:27.263425 ip-10-0-131-93 kubenswrapper[2564]: I0729 19:22:27.263387 2564 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"bound-sa-token\" (UniqueName: \"kubernetes.io/projected/afb8c055-366b-4658-87b3-9ac9db127c69-bound-sa-token\") pod \"cert-manager-55b4d954d5-nwbvk\" (UID: \"afb8c055-366b-4658-87b3-9ac9db127c69\") " pod="cert-manager/cert-manager-55b4d954d5-nwbvk" Jul 29 19:22:27.263425 ip-10-0-131-93 kubenswrapper[2564]: I0729 19:22:27.263432 2564 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-mbgjv\" (UniqueName: \"kubernetes.io/projected/afb8c055-366b-4658-87b3-9ac9db127c69-kube-api-access-mbgjv\") pod \"cert-manager-55b4d954d5-nwbvk\" (UID: \"afb8c055-366b-4658-87b3-9ac9db127c69\") " pod="cert-manager/cert-manager-55b4d954d5-nwbvk" Jul 29 19:22:27.273367 ip-10-0-131-93 kubenswrapper[2564]: I0729 19:22:27.273334 2564 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"bound-sa-token\" (UniqueName: \"kubernetes.io/projected/afb8c055-366b-4658-87b3-9ac9db127c69-bound-sa-token\") pod \"cert-manager-55b4d954d5-nwbvk\" (UID: \"afb8c055-366b-4658-87b3-9ac9db127c69\") " pod="cert-manager/cert-manager-55b4d954d5-nwbvk" Jul 29 19:22:27.273531 ip-10-0-131-93 kubenswrapper[2564]: I0729 19:22:27.273466 2564 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-mbgjv\" (UniqueName: \"kubernetes.io/projected/afb8c055-366b-4658-87b3-9ac9db127c69-kube-api-access-mbgjv\") pod \"cert-manager-55b4d954d5-nwbvk\" (UID: \"afb8c055-366b-4658-87b3-9ac9db127c69\") " pod="cert-manager/cert-manager-55b4d954d5-nwbvk" Jul 29 19:22:27.355284 ip-10-0-131-93 kubenswrapper[2564]: I0729 19:22:27.355204 2564 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="cert-manager/cert-manager-55b4d954d5-nwbvk" Jul 29 19:22:27.606182 ip-10-0-131-93 kubenswrapper[2564]: I0729 19:22:27.606154 2564 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["cert-manager/cert-manager-55b4d954d5-nwbvk"] Jul 29 19:22:27.607899 ip-10-0-131-93 kubenswrapper[2564]: W0729 19:22:27.607867 2564 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-besteffort.slice/kubepods-besteffort-podafb8c055_366b_4658_87b3_9ac9db127c69.slice/crio-ddb703b207b16affc6de959a76cad3ea14a03387d2cd07231bb0850b2afec7ac WatchSource:0}: Error finding container ddb703b207b16affc6de959a76cad3ea14a03387d2cd07231bb0850b2afec7ac: Status 404 returned error can't find the container with id ddb703b207b16affc6de959a76cad3ea14a03387d2cd07231bb0850b2afec7ac Jul 29 19:22:27.771410 ip-10-0-131-93 kubenswrapper[2564]: I0729 19:22:27.771356 2564 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-444jx_faa8c546-c75b-4e10-93b4-661aa2b5f53e/konnectivity-agent/0.log" limit=1024 Jul 29 19:22:27.848192 ip-10-0-131-93 kubenswrapper[2564]: I0729 19:22:27.848167 2564 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-444jx_faa8c546-c75b-4e10-93b4-661aa2b5f53e/konnectivity-agent/0.log" limit=1024 Jul 29 19:22:27.928368 ip-10-0-131-93 kubenswrapper[2564]: I0729 19:22:27.928270 2564 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="cert-manager/cert-manager-webhook-6486ffd997-55mfm" event={"ID":"9b3de621-c041-4b7a-96d9-41edea12c9a9","Type":"ContainerStarted","Data":"cc90f2d76b439bdc79ddf1245318ea9d3e1b6fb77ebffe1d2ad45cacd2088041"} Jul 29 19:22:27.928536 ip-10-0-131-93 kubenswrapper[2564]: I0729 19:22:27.928401 2564 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="cert-manager/cert-manager-webhook-6486ffd997-55mfm" Jul 29 19:22:27.929640 ip-10-0-131-93 kubenswrapper[2564]: I0729 19:22:27.929615 2564 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="cert-manager/cert-manager-55b4d954d5-nwbvk" event={"ID":"afb8c055-366b-4658-87b3-9ac9db127c69","Type":"ContainerStarted","Data":"2fded27050993fe6797fc2c9a4ef7860eab18b00c5c23bcc3dfbc74bd7510168"} Jul 29 19:22:27.929737 ip-10-0-131-93 kubenswrapper[2564]: I0729 19:22:27.929648 2564 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="cert-manager/cert-manager-55b4d954d5-nwbvk" event={"ID":"afb8c055-366b-4658-87b3-9ac9db127c69","Type":"ContainerStarted","Data":"ddb703b207b16affc6de959a76cad3ea14a03387d2cd07231bb0850b2afec7ac"} Jul 29 19:22:27.947106 ip-10-0-131-93 kubenswrapper[2564]: I0729 19:22:27.947055 2564 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="cert-manager/cert-manager-webhook-6486ffd997-55mfm" podStartSLOduration=1.494009681 podStartE2EDuration="3.947041247s" totalImagesPullingTime="2.453031566s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-29 19:22:24 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-29 19:22:27.946151071 +0000 UTC m=+323.670210763" watchObservedRunningTime="2026-07-29 19:22:27.947041247 +0000 UTC m=+323.671100938" Jul 29 19:22:27.965205 ip-10-0-131-93 kubenswrapper[2564]: I0729 19:22:27.965164 2564 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="cert-manager/cert-manager-55b4d954d5-nwbvk" podStartSLOduration=0.965150301 podStartE2EDuration="965.150301ms" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-29 19:22:27 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-29 19:22:27.963809425 +0000 UTC m=+323.687869118" watchObservedRunningTime="2026-07-29 19:22:27.965150301 +0000 UTC m=+323.689209993" Jul 29 19:22:31.131649 ip-10-0-131-93 kubenswrapper[2564]: I0729 19:22:31.131621 2564 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-444jx_faa8c546-c75b-4e10-93b4-661aa2b5f53e/konnectivity-agent/0.log" limit=1024 Jul 29 19:22:33.935409 ip-10-0-131-93 kubenswrapper[2564]: I0729 19:22:33.935358 2564 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="cert-manager/cert-manager-webhook-6486ffd997-55mfm" Jul 29 19:22:34.007400 ip-10-0-131-93 kubenswrapper[2564]: I0729 19:22:34.007340 2564 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-444jx_faa8c546-c75b-4e10-93b4-661aa2b5f53e/konnectivity-agent/0.log" limit=1024 Jul 29 19:22:36.865173 ip-10-0-131-93 kubenswrapper[2564]: I0729 19:22:36.865141 2564 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-444jx_faa8c546-c75b-4e10-93b4-661aa2b5f53e/konnectivity-agent/0.log" limit=1024 Jul 29 19:22:36.966533 ip-10-0-131-93 kubenswrapper[2564]: I0729 19:22:36.966502 2564 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-444jx_faa8c546-c75b-4e10-93b4-661aa2b5f53e/konnectivity-agent/0.log" limit=1024 Jul 29 19:22:38.408286 ip-10-0-131-93 kubenswrapper[2564]: I0729 19:22:38.408257 2564 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-marketplace/629fd74b02459b8d9ee224b8bdbb513172290d80b07a5bf327840796afhgkvd"] Jul 29 19:22:38.410760 ip-10-0-131-93 kubenswrapper[2564]: I0729 19:22:38.410740 2564 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-marketplace/629fd74b02459b8d9ee224b8bdbb513172290d80b07a5bf327840796afhgkvd" Jul 29 19:22:38.413247 ip-10-0-131-93 kubenswrapper[2564]: I0729 19:22:38.413222 2564 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-marketplace\"/\"default-dockercfg-7n999\"" Jul 29 19:22:38.413433 ip-10-0-131-93 kubenswrapper[2564]: I0729 19:22:38.413417 2564 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-marketplace\"/\"openshift-service-ca.crt\"" Jul 29 19:22:38.414260 ip-10-0-131-93 kubenswrapper[2564]: I0729 19:22:38.414245 2564 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-marketplace\"/\"kube-root-ca.crt\"" Jul 29 19:22:38.421853 ip-10-0-131-93 kubenswrapper[2564]: I0729 19:22:38.421833 2564 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-marketplace/629fd74b02459b8d9ee224b8bdbb513172290d80b07a5bf327840796afhgkvd"] Jul 29 19:22:38.442202 ip-10-0-131-93 kubenswrapper[2564]: I0729 19:22:38.442173 2564 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"bundle\" (UniqueName: \"kubernetes.io/empty-dir/892b44d6-0b4e-46a4-9824-68bc08225045-bundle\") pod \"629fd74b02459b8d9ee224b8bdbb513172290d80b07a5bf327840796afhgkvd\" (UID: \"892b44d6-0b4e-46a4-9824-68bc08225045\") " pod="openshift-marketplace/629fd74b02459b8d9ee224b8bdbb513172290d80b07a5bf327840796afhgkvd" Jul 29 19:22:38.442202 ip-10-0-131-93 kubenswrapper[2564]: I0729 19:22:38.442206 2564 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"util\" (UniqueName: \"kubernetes.io/empty-dir/892b44d6-0b4e-46a4-9824-68bc08225045-util\") pod \"629fd74b02459b8d9ee224b8bdbb513172290d80b07a5bf327840796afhgkvd\" (UID: \"892b44d6-0b4e-46a4-9824-68bc08225045\") " pod="openshift-marketplace/629fd74b02459b8d9ee224b8bdbb513172290d80b07a5bf327840796afhgkvd" Jul 29 19:22:38.442420 ip-10-0-131-93 kubenswrapper[2564]: I0729 19:22:38.442238 2564 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-7vffg\" (UniqueName: \"kubernetes.io/projected/892b44d6-0b4e-46a4-9824-68bc08225045-kube-api-access-7vffg\") pod \"629fd74b02459b8d9ee224b8bdbb513172290d80b07a5bf327840796afhgkvd\" (UID: \"892b44d6-0b4e-46a4-9824-68bc08225045\") " pod="openshift-marketplace/629fd74b02459b8d9ee224b8bdbb513172290d80b07a5bf327840796afhgkvd" Jul 29 19:22:38.542620 ip-10-0-131-93 kubenswrapper[2564]: I0729 19:22:38.542586 2564 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"bundle\" (UniqueName: \"kubernetes.io/empty-dir/892b44d6-0b4e-46a4-9824-68bc08225045-bundle\") pod \"629fd74b02459b8d9ee224b8bdbb513172290d80b07a5bf327840796afhgkvd\" (UID: \"892b44d6-0b4e-46a4-9824-68bc08225045\") " pod="openshift-marketplace/629fd74b02459b8d9ee224b8bdbb513172290d80b07a5bf327840796afhgkvd" Jul 29 19:22:38.542620 ip-10-0-131-93 kubenswrapper[2564]: I0729 19:22:38.542622 2564 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"util\" (UniqueName: \"kubernetes.io/empty-dir/892b44d6-0b4e-46a4-9824-68bc08225045-util\") pod \"629fd74b02459b8d9ee224b8bdbb513172290d80b07a5bf327840796afhgkvd\" (UID: \"892b44d6-0b4e-46a4-9824-68bc08225045\") " pod="openshift-marketplace/629fd74b02459b8d9ee224b8bdbb513172290d80b07a5bf327840796afhgkvd" Jul 29 19:22:38.542801 ip-10-0-131-93 kubenswrapper[2564]: I0729 19:22:38.542751 2564 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-7vffg\" (UniqueName: \"kubernetes.io/projected/892b44d6-0b4e-46a4-9824-68bc08225045-kube-api-access-7vffg\") pod \"629fd74b02459b8d9ee224b8bdbb513172290d80b07a5bf327840796afhgkvd\" (UID: \"892b44d6-0b4e-46a4-9824-68bc08225045\") " pod="openshift-marketplace/629fd74b02459b8d9ee224b8bdbb513172290d80b07a5bf327840796afhgkvd" Jul 29 19:22:38.542953 ip-10-0-131-93 kubenswrapper[2564]: I0729 19:22:38.542940 2564 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"util\" (UniqueName: \"kubernetes.io/empty-dir/892b44d6-0b4e-46a4-9824-68bc08225045-util\") pod \"629fd74b02459b8d9ee224b8bdbb513172290d80b07a5bf327840796afhgkvd\" (UID: \"892b44d6-0b4e-46a4-9824-68bc08225045\") " pod="openshift-marketplace/629fd74b02459b8d9ee224b8bdbb513172290d80b07a5bf327840796afhgkvd" Jul 29 19:22:38.542995 ip-10-0-131-93 kubenswrapper[2564]: I0729 19:22:38.542978 2564 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"bundle\" (UniqueName: \"kubernetes.io/empty-dir/892b44d6-0b4e-46a4-9824-68bc08225045-bundle\") pod \"629fd74b02459b8d9ee224b8bdbb513172290d80b07a5bf327840796afhgkvd\" (UID: \"892b44d6-0b4e-46a4-9824-68bc08225045\") " pod="openshift-marketplace/629fd74b02459b8d9ee224b8bdbb513172290d80b07a5bf327840796afhgkvd" Jul 29 19:22:38.551959 ip-10-0-131-93 kubenswrapper[2564]: I0729 19:22:38.551929 2564 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-7vffg\" (UniqueName: \"kubernetes.io/projected/892b44d6-0b4e-46a4-9824-68bc08225045-kube-api-access-7vffg\") pod \"629fd74b02459b8d9ee224b8bdbb513172290d80b07a5bf327840796afhgkvd\" (UID: \"892b44d6-0b4e-46a4-9824-68bc08225045\") " pod="openshift-marketplace/629fd74b02459b8d9ee224b8bdbb513172290d80b07a5bf327840796afhgkvd" Jul 29 19:22:38.719976 ip-10-0-131-93 kubenswrapper[2564]: I0729 19:22:38.719945 2564 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-marketplace/629fd74b02459b8d9ee224b8bdbb513172290d80b07a5bf327840796afhgkvd" Jul 29 19:22:38.846687 ip-10-0-131-93 kubenswrapper[2564]: I0729 19:22:38.846658 2564 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-marketplace/629fd74b02459b8d9ee224b8bdbb513172290d80b07a5bf327840796afhgkvd"] Jul 29 19:22:38.850025 ip-10-0-131-93 kubenswrapper[2564]: W0729 19:22:38.849999 2564 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod892b44d6_0b4e_46a4_9824_68bc08225045.slice/crio-b9c062fc9b3a56cd3d0d216cc2502468bb80c951318cf9ab13c6e950b7fc51cc WatchSource:0}: Error finding container b9c062fc9b3a56cd3d0d216cc2502468bb80c951318cf9ab13c6e950b7fc51cc: Status 404 returned error can't find the container with id b9c062fc9b3a56cd3d0d216cc2502468bb80c951318cf9ab13c6e950b7fc51cc Jul 29 19:22:38.972695 ip-10-0-131-93 kubenswrapper[2564]: I0729 19:22:38.972616 2564 generic.go:356] "Generic (PLEG): container finished" podID="892b44d6-0b4e-46a4-9824-68bc08225045" containerID="bd25712a5145af6f1f1d984f4f204a99dbc59dfa0238315b4519d87ad9b5220d" exitCode=0 Jul 29 19:22:38.972819 ip-10-0-131-93 kubenswrapper[2564]: I0729 19:22:38.972706 2564 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-marketplace/629fd74b02459b8d9ee224b8bdbb513172290d80b07a5bf327840796afhgkvd" event={"ID":"892b44d6-0b4e-46a4-9824-68bc08225045","Type":"ContainerDied","Data":"bd25712a5145af6f1f1d984f4f204a99dbc59dfa0238315b4519d87ad9b5220d"} Jul 29 19:22:38.972819 ip-10-0-131-93 kubenswrapper[2564]: I0729 19:22:38.972762 2564 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-marketplace/629fd74b02459b8d9ee224b8bdbb513172290d80b07a5bf327840796afhgkvd" event={"ID":"892b44d6-0b4e-46a4-9824-68bc08225045","Type":"ContainerStarted","Data":"b9c062fc9b3a56cd3d0d216cc2502468bb80c951318cf9ab13c6e950b7fc51cc"} Jul 29 19:22:39.027589 ip-10-0-131-93 kubenswrapper[2564]: I0729 19:22:39.027537 2564 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-444jx_faa8c546-c75b-4e10-93b4-661aa2b5f53e/konnectivity-agent/0.log" limit=1024 Jul 29 19:22:41.985249 ip-10-0-131-93 kubenswrapper[2564]: I0729 19:22:41.985218 2564 generic.go:356] "Generic (PLEG): container finished" podID="892b44d6-0b4e-46a4-9824-68bc08225045" containerID="8153072ee511bd24554104cb4a549b75b011e80a16bbc0385ff77574bc1bf8e8" exitCode=0 Jul 29 19:22:41.985730 ip-10-0-131-93 kubenswrapper[2564]: I0729 19:22:41.985280 2564 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-marketplace/629fd74b02459b8d9ee224b8bdbb513172290d80b07a5bf327840796afhgkvd" event={"ID":"892b44d6-0b4e-46a4-9824-68bc08225045","Type":"ContainerDied","Data":"8153072ee511bd24554104cb4a549b75b011e80a16bbc0385ff77574bc1bf8e8"} Jul 29 19:22:42.991565 ip-10-0-131-93 kubenswrapper[2564]: I0729 19:22:42.991529 2564 generic.go:356] "Generic (PLEG): container finished" podID="892b44d6-0b4e-46a4-9824-68bc08225045" containerID="d9cb4a25025319d15775929cc6eabf6aacf6e6047d1a55086681fa710bb10932" exitCode=0 Jul 29 19:22:42.991962 ip-10-0-131-93 kubenswrapper[2564]: I0729 19:22:42.991616 2564 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-marketplace/629fd74b02459b8d9ee224b8bdbb513172290d80b07a5bf327840796afhgkvd" event={"ID":"892b44d6-0b4e-46a4-9824-68bc08225045","Type":"ContainerDied","Data":"d9cb4a25025319d15775929cc6eabf6aacf6e6047d1a55086681fa710bb10932"} Jul 29 19:22:44.110446 ip-10-0-131-93 kubenswrapper[2564]: I0729 19:22:44.110418 2564 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-marketplace/629fd74b02459b8d9ee224b8bdbb513172290d80b07a5bf327840796afhgkvd" Jul 29 19:22:44.185606 ip-10-0-131-93 kubenswrapper[2564]: I0729 19:22:44.185573 2564 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"util\" (UniqueName: \"kubernetes.io/empty-dir/892b44d6-0b4e-46a4-9824-68bc08225045-util\") pod \"892b44d6-0b4e-46a4-9824-68bc08225045\" (UID: \"892b44d6-0b4e-46a4-9824-68bc08225045\") " Jul 29 19:22:44.185756 ip-10-0-131-93 kubenswrapper[2564]: I0729 19:22:44.185625 2564 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"bundle\" (UniqueName: \"kubernetes.io/empty-dir/892b44d6-0b4e-46a4-9824-68bc08225045-bundle\") pod \"892b44d6-0b4e-46a4-9824-68bc08225045\" (UID: \"892b44d6-0b4e-46a4-9824-68bc08225045\") " Jul 29 19:22:44.185756 ip-10-0-131-93 kubenswrapper[2564]: I0729 19:22:44.185656 2564 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-7vffg\" (UniqueName: \"kubernetes.io/projected/892b44d6-0b4e-46a4-9824-68bc08225045-kube-api-access-7vffg\") pod \"892b44d6-0b4e-46a4-9824-68bc08225045\" (UID: \"892b44d6-0b4e-46a4-9824-68bc08225045\") " Jul 29 19:22:44.186305 ip-10-0-131-93 kubenswrapper[2564]: I0729 19:22:44.186275 2564 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/892b44d6-0b4e-46a4-9824-68bc08225045-bundle" (OuterVolumeSpecName: "bundle") pod "892b44d6-0b4e-46a4-9824-68bc08225045" (UID: "892b44d6-0b4e-46a4-9824-68bc08225045"). InnerVolumeSpecName "bundle". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Jul 29 19:22:44.187638 ip-10-0-131-93 kubenswrapper[2564]: I0729 19:22:44.187614 2564 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/892b44d6-0b4e-46a4-9824-68bc08225045-kube-api-access-7vffg" (OuterVolumeSpecName: "kube-api-access-7vffg") pod "892b44d6-0b4e-46a4-9824-68bc08225045" (UID: "892b44d6-0b4e-46a4-9824-68bc08225045"). InnerVolumeSpecName "kube-api-access-7vffg". PluginName "kubernetes.io/projected", VolumeGIDValue "" Jul 29 19:22:44.204411 ip-10-0-131-93 kubenswrapper[2564]: I0729 19:22:44.204389 2564 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/892b44d6-0b4e-46a4-9824-68bc08225045-util" (OuterVolumeSpecName: "util") pod "892b44d6-0b4e-46a4-9824-68bc08225045" (UID: "892b44d6-0b4e-46a4-9824-68bc08225045"). InnerVolumeSpecName "util". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Jul 29 19:22:44.286702 ip-10-0-131-93 kubenswrapper[2564]: I0729 19:22:44.286635 2564 reconciler_common.go:299] "Volume detached for volume \"util\" (UniqueName: \"kubernetes.io/empty-dir/892b44d6-0b4e-46a4-9824-68bc08225045-util\") on node \"ip-10-0-131-93.ec2.internal\" DevicePath \"\"" Jul 29 19:22:44.286702 ip-10-0-131-93 kubenswrapper[2564]: I0729 19:22:44.286657 2564 reconciler_common.go:299] "Volume detached for volume \"bundle\" (UniqueName: \"kubernetes.io/empty-dir/892b44d6-0b4e-46a4-9824-68bc08225045-bundle\") on node \"ip-10-0-131-93.ec2.internal\" DevicePath \"\"" Jul 29 19:22:44.286702 ip-10-0-131-93 kubenswrapper[2564]: I0729 19:22:44.286666 2564 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-7vffg\" (UniqueName: \"kubernetes.io/projected/892b44d6-0b4e-46a4-9824-68bc08225045-kube-api-access-7vffg\") on node \"ip-10-0-131-93.ec2.internal\" DevicePath \"\"" Jul 29 19:22:45.001160 ip-10-0-131-93 kubenswrapper[2564]: I0729 19:22:45.001119 2564 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-marketplace/629fd74b02459b8d9ee224b8bdbb513172290d80b07a5bf327840796afhgkvd" event={"ID":"892b44d6-0b4e-46a4-9824-68bc08225045","Type":"ContainerDied","Data":"b9c062fc9b3a56cd3d0d216cc2502468bb80c951318cf9ab13c6e950b7fc51cc"} Jul 29 19:22:45.001321 ip-10-0-131-93 kubenswrapper[2564]: I0729 19:22:45.001180 2564 pod_container_deletor.go:80] "Container not found in pod's containers" containerID="b9c062fc9b3a56cd3d0d216cc2502468bb80c951318cf9ab13c6e950b7fc51cc" Jul 29 19:22:45.001321 ip-10-0-131-93 kubenswrapper[2564]: I0729 19:22:45.001141 2564 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-marketplace/629fd74b02459b8d9ee224b8bdbb513172290d80b07a5bf327840796afhgkvd" Jul 29 19:22:51.456033 ip-10-0-131-93 kubenswrapper[2564]: I0729 19:22:51.456000 2564 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-lws-operator/lws-controller-manager-ffdbfbbc8-d65tt"] Jul 29 19:22:51.456421 ip-10-0-131-93 kubenswrapper[2564]: I0729 19:22:51.456250 2564 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="892b44d6-0b4e-46a4-9824-68bc08225045" containerName="util" Jul 29 19:22:51.456421 ip-10-0-131-93 kubenswrapper[2564]: I0729 19:22:51.456260 2564 state_mem.go:107] "Deleted CPUSet assignment" podUID="892b44d6-0b4e-46a4-9824-68bc08225045" containerName="util" Jul 29 19:22:51.456421 ip-10-0-131-93 kubenswrapper[2564]: I0729 19:22:51.456267 2564 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="892b44d6-0b4e-46a4-9824-68bc08225045" containerName="pull" Jul 29 19:22:51.456421 ip-10-0-131-93 kubenswrapper[2564]: I0729 19:22:51.456272 2564 state_mem.go:107] "Deleted CPUSet assignment" podUID="892b44d6-0b4e-46a4-9824-68bc08225045" containerName="pull" Jul 29 19:22:51.456421 ip-10-0-131-93 kubenswrapper[2564]: I0729 19:22:51.456279 2564 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="892b44d6-0b4e-46a4-9824-68bc08225045" containerName="extract" Jul 29 19:22:51.456421 ip-10-0-131-93 kubenswrapper[2564]: I0729 19:22:51.456285 2564 state_mem.go:107] "Deleted CPUSet assignment" podUID="892b44d6-0b4e-46a4-9824-68bc08225045" containerName="extract" Jul 29 19:22:51.456421 ip-10-0-131-93 kubenswrapper[2564]: I0729 19:22:51.456325 2564 memory_manager.go:368] "RemoveStaleState removing state" podUID="892b44d6-0b4e-46a4-9824-68bc08225045" containerName="extract" Jul 29 19:22:51.459171 ip-10-0-131-93 kubenswrapper[2564]: I0729 19:22:51.459155 2564 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-lws-operator/lws-controller-manager-ffdbfbbc8-d65tt" Jul 29 19:22:51.462527 ip-10-0-131-93 kubenswrapper[2564]: I0729 19:22:51.462507 2564 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-lws-operator\"/\"openshift-service-ca.crt\"" Jul 29 19:22:51.462527 ip-10-0-131-93 kubenswrapper[2564]: I0729 19:22:51.462511 2564 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-lws-operator\"/\"metrics-server-cert\"" Jul 29 19:22:51.463264 ip-10-0-131-93 kubenswrapper[2564]: I0729 19:22:51.463249 2564 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-lws-operator\"/\"lws-manager-config\"" Jul 29 19:22:51.463329 ip-10-0-131-93 kubenswrapper[2564]: I0729 19:22:51.463279 2564 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-lws-operator\"/\"kube-root-ca.crt\"" Jul 29 19:22:51.463605 ip-10-0-131-93 kubenswrapper[2564]: I0729 19:22:51.463590 2564 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-lws-operator\"/\"webhook-server-cert\"" Jul 29 19:22:51.463651 ip-10-0-131-93 kubenswrapper[2564]: I0729 19:22:51.463643 2564 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-lws-operator\"/\"lws-controller-manager-dockercfg-pj965\"" Jul 29 19:22:51.482339 ip-10-0-131-93 kubenswrapper[2564]: I0729 19:22:51.482313 2564 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-lws-operator/lws-controller-manager-ffdbfbbc8-d65tt"] Jul 29 19:22:51.537359 ip-10-0-131-93 kubenswrapper[2564]: I0729 19:22:51.537325 2564 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"manager-config\" (UniqueName: \"kubernetes.io/configmap/d369ea92-58ba-436e-891e-b85d304eaf09-manager-config\") pod \"lws-controller-manager-ffdbfbbc8-d65tt\" (UID: \"d369ea92-58ba-436e-891e-b85d304eaf09\") " pod="openshift-lws-operator/lws-controller-manager-ffdbfbbc8-d65tt" Jul 29 19:22:51.537527 ip-10-0-131-93 kubenswrapper[2564]: I0729 19:22:51.537368 2564 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"cert\" (UniqueName: \"kubernetes.io/secret/d369ea92-58ba-436e-891e-b85d304eaf09-cert\") pod \"lws-controller-manager-ffdbfbbc8-d65tt\" (UID: \"d369ea92-58ba-436e-891e-b85d304eaf09\") " pod="openshift-lws-operator/lws-controller-manager-ffdbfbbc8-d65tt" Jul 29 19:22:51.537527 ip-10-0-131-93 kubenswrapper[2564]: I0729 19:22:51.537410 2564 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-xmwz5\" (UniqueName: \"kubernetes.io/projected/d369ea92-58ba-436e-891e-b85d304eaf09-kube-api-access-xmwz5\") pod \"lws-controller-manager-ffdbfbbc8-d65tt\" (UID: \"d369ea92-58ba-436e-891e-b85d304eaf09\") " pod="openshift-lws-operator/lws-controller-manager-ffdbfbbc8-d65tt" Jul 29 19:22:51.537527 ip-10-0-131-93 kubenswrapper[2564]: I0729 19:22:51.537482 2564 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"metrics-cert\" (UniqueName: \"kubernetes.io/secret/d369ea92-58ba-436e-891e-b85d304eaf09-metrics-cert\") pod \"lws-controller-manager-ffdbfbbc8-d65tt\" (UID: \"d369ea92-58ba-436e-891e-b85d304eaf09\") " pod="openshift-lws-operator/lws-controller-manager-ffdbfbbc8-d65tt" Jul 29 19:22:51.638744 ip-10-0-131-93 kubenswrapper[2564]: I0729 19:22:51.638704 2564 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"cert\" (UniqueName: \"kubernetes.io/secret/d369ea92-58ba-436e-891e-b85d304eaf09-cert\") pod \"lws-controller-manager-ffdbfbbc8-d65tt\" (UID: \"d369ea92-58ba-436e-891e-b85d304eaf09\") " pod="openshift-lws-operator/lws-controller-manager-ffdbfbbc8-d65tt" Jul 29 19:22:51.638890 ip-10-0-131-93 kubenswrapper[2564]: I0729 19:22:51.638752 2564 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-xmwz5\" (UniqueName: \"kubernetes.io/projected/d369ea92-58ba-436e-891e-b85d304eaf09-kube-api-access-xmwz5\") pod \"lws-controller-manager-ffdbfbbc8-d65tt\" (UID: \"d369ea92-58ba-436e-891e-b85d304eaf09\") " pod="openshift-lws-operator/lws-controller-manager-ffdbfbbc8-d65tt" Jul 29 19:22:51.638890 ip-10-0-131-93 kubenswrapper[2564]: I0729 19:22:51.638782 2564 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-cert\" (UniqueName: \"kubernetes.io/secret/d369ea92-58ba-436e-891e-b85d304eaf09-metrics-cert\") pod \"lws-controller-manager-ffdbfbbc8-d65tt\" (UID: \"d369ea92-58ba-436e-891e-b85d304eaf09\") " pod="openshift-lws-operator/lws-controller-manager-ffdbfbbc8-d65tt" Jul 29 19:22:51.638890 ip-10-0-131-93 kubenswrapper[2564]: I0729 19:22:51.638811 2564 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"manager-config\" (UniqueName: \"kubernetes.io/configmap/d369ea92-58ba-436e-891e-b85d304eaf09-manager-config\") pod \"lws-controller-manager-ffdbfbbc8-d65tt\" (UID: \"d369ea92-58ba-436e-891e-b85d304eaf09\") " pod="openshift-lws-operator/lws-controller-manager-ffdbfbbc8-d65tt" Jul 29 19:22:51.639520 ip-10-0-131-93 kubenswrapper[2564]: I0729 19:22:51.639497 2564 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"manager-config\" (UniqueName: \"kubernetes.io/configmap/d369ea92-58ba-436e-891e-b85d304eaf09-manager-config\") pod \"lws-controller-manager-ffdbfbbc8-d65tt\" (UID: \"d369ea92-58ba-436e-891e-b85d304eaf09\") " pod="openshift-lws-operator/lws-controller-manager-ffdbfbbc8-d65tt" Jul 29 19:22:51.641420 ip-10-0-131-93 kubenswrapper[2564]: I0729 19:22:51.641390 2564 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"metrics-cert\" (UniqueName: \"kubernetes.io/secret/d369ea92-58ba-436e-891e-b85d304eaf09-metrics-cert\") pod \"lws-controller-manager-ffdbfbbc8-d65tt\" (UID: \"d369ea92-58ba-436e-891e-b85d304eaf09\") " pod="openshift-lws-operator/lws-controller-manager-ffdbfbbc8-d65tt" Jul 29 19:22:51.641511 ip-10-0-131-93 kubenswrapper[2564]: I0729 19:22:51.641366 2564 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"cert\" (UniqueName: \"kubernetes.io/secret/d369ea92-58ba-436e-891e-b85d304eaf09-cert\") pod \"lws-controller-manager-ffdbfbbc8-d65tt\" (UID: \"d369ea92-58ba-436e-891e-b85d304eaf09\") " pod="openshift-lws-operator/lws-controller-manager-ffdbfbbc8-d65tt" Jul 29 19:22:51.649899 ip-10-0-131-93 kubenswrapper[2564]: I0729 19:22:51.649879 2564 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-xmwz5\" (UniqueName: \"kubernetes.io/projected/d369ea92-58ba-436e-891e-b85d304eaf09-kube-api-access-xmwz5\") pod \"lws-controller-manager-ffdbfbbc8-d65tt\" (UID: \"d369ea92-58ba-436e-891e-b85d304eaf09\") " pod="openshift-lws-operator/lws-controller-manager-ffdbfbbc8-d65tt" Jul 29 19:22:51.768546 ip-10-0-131-93 kubenswrapper[2564]: I0729 19:22:51.768465 2564 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-lws-operator/lws-controller-manager-ffdbfbbc8-d65tt" Jul 29 19:22:51.911344 ip-10-0-131-93 kubenswrapper[2564]: W0729 19:22:51.911319 2564 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podd369ea92_58ba_436e_891e_b85d304eaf09.slice/crio-2238c6f12d90637d25a5b3cc5f618aff17ad857cd873be9d051faf5bc11cb092 WatchSource:0}: Error finding container 2238c6f12d90637d25a5b3cc5f618aff17ad857cd873be9d051faf5bc11cb092: Status 404 returned error can't find the container with id 2238c6f12d90637d25a5b3cc5f618aff17ad857cd873be9d051faf5bc11cb092 Jul 29 19:22:51.917905 ip-10-0-131-93 kubenswrapper[2564]: I0729 19:22:51.917882 2564 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-lws-operator/lws-controller-manager-ffdbfbbc8-d65tt"] Jul 29 19:22:52.029598 ip-10-0-131-93 kubenswrapper[2564]: I0729 19:22:52.029509 2564 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-lws-operator/lws-controller-manager-ffdbfbbc8-d65tt" event={"ID":"d369ea92-58ba-436e-891e-b85d304eaf09","Type":"ContainerStarted","Data":"2238c6f12d90637d25a5b3cc5f618aff17ad857cd873be9d051faf5bc11cb092"} Jul 29 19:22:52.878282 ip-10-0-131-93 kubenswrapper[2564]: I0729 19:22:52.878244 2564 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-444jx_faa8c546-c75b-4e10-93b4-661aa2b5f53e/konnectivity-agent/0.log" limit=1024 Jul 29 19:22:52.980539 ip-10-0-131-93 kubenswrapper[2564]: I0729 19:22:52.980506 2564 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-444jx_faa8c546-c75b-4e10-93b4-661aa2b5f53e/konnectivity-agent/0.log" limit=1024 Jul 29 19:22:54.039034 ip-10-0-131-93 kubenswrapper[2564]: I0729 19:22:54.039002 2564 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-lws-operator/lws-controller-manager-ffdbfbbc8-d65tt" event={"ID":"d369ea92-58ba-436e-891e-b85d304eaf09","Type":"ContainerStarted","Data":"1309d723dc7c7ff23880e16425aa5e860ce717c4b6f33acc79e1544025568950"} Jul 29 19:22:54.039444 ip-10-0-131-93 kubenswrapper[2564]: I0729 19:22:54.039135 2564 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-lws-operator/lws-controller-manager-ffdbfbbc8-d65tt" Jul 29 19:22:54.065998 ip-10-0-131-93 kubenswrapper[2564]: I0729 19:22:54.065916 2564 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-lws-operator/lws-controller-manager-ffdbfbbc8-d65tt" podStartSLOduration=1.960468664 podStartE2EDuration="3.065897462s" totalImagesPullingTime="1.105428798s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-29 19:22:51 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-29 19:22:54.064339153 +0000 UTC m=+349.788398849" watchObservedRunningTime="2026-07-29 19:22:54.065897462 +0000 UTC m=+349.789957155" Jul 29 19:22:54.333230 ip-10-0-131-93 kubenswrapper[2564]: I0729 19:22:54.333144 2564 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-444jx_faa8c546-c75b-4e10-93b4-661aa2b5f53e/konnectivity-agent/0.log" limit=1024 Jul 29 19:22:54.519788 ip-10-0-131-93 kubenswrapper[2564]: I0729 19:22:54.519756 2564 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-444jx_faa8c546-c75b-4e10-93b4-661aa2b5f53e/konnectivity-agent/0.log" limit=1024 Jul 29 19:22:55.548197 ip-10-0-131-93 kubenswrapper[2564]: I0729 19:22:55.548163 2564 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["opendatahub/opendatahub-operator-controller-manager-d84699fcf-k4tz9"] Jul 29 19:22:55.554168 ip-10-0-131-93 kubenswrapper[2564]: I0729 19:22:55.554149 2564 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="opendatahub/opendatahub-operator-controller-manager-d84699fcf-k4tz9" Jul 29 19:22:55.556826 ip-10-0-131-93 kubenswrapper[2564]: I0729 19:22:55.556804 2564 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"opendatahub\"/\"opendatahub-operator-controller-webhook-cert\"" Jul 29 19:22:55.556959 ip-10-0-131-93 kubenswrapper[2564]: I0729 19:22:55.556842 2564 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"opendatahub\"/\"opendatahub-operator-controller-manager-service-cert\"" Jul 29 19:22:55.557148 ip-10-0-131-93 kubenswrapper[2564]: I0729 19:22:55.557132 2564 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"opendatahub\"/\"opendatahub-operator-controller-manager-dockercfg-72wzr\"" Jul 29 19:22:55.557220 ip-10-0-131-93 kubenswrapper[2564]: I0729 19:22:55.557172 2564 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"opendatahub\"/\"openshift-service-ca.crt\"" Jul 29 19:22:55.557356 ip-10-0-131-93 kubenswrapper[2564]: I0729 19:22:55.557344 2564 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"opendatahub\"/\"kube-root-ca.crt\"" Jul 29 19:22:55.573815 ip-10-0-131-93 kubenswrapper[2564]: I0729 19:22:55.573781 2564 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["opendatahub/opendatahub-operator-controller-manager-d84699fcf-k4tz9"] Jul 29 19:22:55.671180 ip-10-0-131-93 kubenswrapper[2564]: I0729 19:22:55.671140 2564 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tmp\" (UniqueName: \"kubernetes.io/empty-dir/ed963675-70ff-4f05-84ae-1e2b2702100c-tmp\") pod \"opendatahub-operator-controller-manager-d84699fcf-k4tz9\" (UID: \"ed963675-70ff-4f05-84ae-1e2b2702100c\") " pod="opendatahub/opendatahub-operator-controller-manager-d84699fcf-k4tz9" Jul 29 19:22:55.671335 ip-10-0-131-93 kubenswrapper[2564]: I0729 19:22:55.671184 2564 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"webhook-cert\" (UniqueName: \"kubernetes.io/secret/ed963675-70ff-4f05-84ae-1e2b2702100c-webhook-cert\") pod \"opendatahub-operator-controller-manager-d84699fcf-k4tz9\" (UID: \"ed963675-70ff-4f05-84ae-1e2b2702100c\") " pod="opendatahub/opendatahub-operator-controller-manager-d84699fcf-k4tz9" Jul 29 19:22:55.671335 ip-10-0-131-93 kubenswrapper[2564]: I0729 19:22:55.671226 2564 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-fn4q5\" (UniqueName: \"kubernetes.io/projected/ed963675-70ff-4f05-84ae-1e2b2702100c-kube-api-access-fn4q5\") pod \"opendatahub-operator-controller-manager-d84699fcf-k4tz9\" (UID: \"ed963675-70ff-4f05-84ae-1e2b2702100c\") " pod="opendatahub/opendatahub-operator-controller-manager-d84699fcf-k4tz9" Jul 29 19:22:55.671335 ip-10-0-131-93 kubenswrapper[2564]: I0729 19:22:55.671255 2564 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"apiservice-cert\" (UniqueName: \"kubernetes.io/secret/ed963675-70ff-4f05-84ae-1e2b2702100c-apiservice-cert\") pod \"opendatahub-operator-controller-manager-d84699fcf-k4tz9\" (UID: \"ed963675-70ff-4f05-84ae-1e2b2702100c\") " pod="opendatahub/opendatahub-operator-controller-manager-d84699fcf-k4tz9" Jul 29 19:22:55.671335 ip-10-0-131-93 kubenswrapper[2564]: I0729 19:22:55.671277 2564 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"manifests\" (UniqueName: \"kubernetes.io/empty-dir/ed963675-70ff-4f05-84ae-1e2b2702100c-manifests\") pod \"opendatahub-operator-controller-manager-d84699fcf-k4tz9\" (UID: \"ed963675-70ff-4f05-84ae-1e2b2702100c\") " pod="opendatahub/opendatahub-operator-controller-manager-d84699fcf-k4tz9" Jul 29 19:22:55.772199 ip-10-0-131-93 kubenswrapper[2564]: I0729 19:22:55.772157 2564 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tmp\" (UniqueName: \"kubernetes.io/empty-dir/ed963675-70ff-4f05-84ae-1e2b2702100c-tmp\") pod \"opendatahub-operator-controller-manager-d84699fcf-k4tz9\" (UID: \"ed963675-70ff-4f05-84ae-1e2b2702100c\") " pod="opendatahub/opendatahub-operator-controller-manager-d84699fcf-k4tz9" Jul 29 19:22:55.772199 ip-10-0-131-93 kubenswrapper[2564]: I0729 19:22:55.772203 2564 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"webhook-cert\" (UniqueName: \"kubernetes.io/secret/ed963675-70ff-4f05-84ae-1e2b2702100c-webhook-cert\") pod \"opendatahub-operator-controller-manager-d84699fcf-k4tz9\" (UID: \"ed963675-70ff-4f05-84ae-1e2b2702100c\") " pod="opendatahub/opendatahub-operator-controller-manager-d84699fcf-k4tz9" Jul 29 19:22:55.772495 ip-10-0-131-93 kubenswrapper[2564]: I0729 19:22:55.772227 2564 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-fn4q5\" (UniqueName: \"kubernetes.io/projected/ed963675-70ff-4f05-84ae-1e2b2702100c-kube-api-access-fn4q5\") pod \"opendatahub-operator-controller-manager-d84699fcf-k4tz9\" (UID: \"ed963675-70ff-4f05-84ae-1e2b2702100c\") " pod="opendatahub/opendatahub-operator-controller-manager-d84699fcf-k4tz9" Jul 29 19:22:55.772495 ip-10-0-131-93 kubenswrapper[2564]: I0729 19:22:55.772259 2564 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"apiservice-cert\" (UniqueName: \"kubernetes.io/secret/ed963675-70ff-4f05-84ae-1e2b2702100c-apiservice-cert\") pod \"opendatahub-operator-controller-manager-d84699fcf-k4tz9\" (UID: \"ed963675-70ff-4f05-84ae-1e2b2702100c\") " pod="opendatahub/opendatahub-operator-controller-manager-d84699fcf-k4tz9" Jul 29 19:22:55.772495 ip-10-0-131-93 kubenswrapper[2564]: I0729 19:22:55.772283 2564 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"manifests\" (UniqueName: \"kubernetes.io/empty-dir/ed963675-70ff-4f05-84ae-1e2b2702100c-manifests\") pod \"opendatahub-operator-controller-manager-d84699fcf-k4tz9\" (UID: \"ed963675-70ff-4f05-84ae-1e2b2702100c\") " pod="opendatahub/opendatahub-operator-controller-manager-d84699fcf-k4tz9" Jul 29 19:22:55.772722 ip-10-0-131-93 kubenswrapper[2564]: I0729 19:22:55.772701 2564 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"manifests\" (UniqueName: \"kubernetes.io/empty-dir/ed963675-70ff-4f05-84ae-1e2b2702100c-manifests\") pod \"opendatahub-operator-controller-manager-d84699fcf-k4tz9\" (UID: \"ed963675-70ff-4f05-84ae-1e2b2702100c\") " pod="opendatahub/opendatahub-operator-controller-manager-d84699fcf-k4tz9" Jul 29 19:22:55.774490 ip-10-0-131-93 kubenswrapper[2564]: I0729 19:22:55.774451 2564 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"tmp\" (UniqueName: \"kubernetes.io/empty-dir/ed963675-70ff-4f05-84ae-1e2b2702100c-tmp\") pod \"opendatahub-operator-controller-manager-d84699fcf-k4tz9\" (UID: \"ed963675-70ff-4f05-84ae-1e2b2702100c\") " pod="opendatahub/opendatahub-operator-controller-manager-d84699fcf-k4tz9" Jul 29 19:22:55.774737 ip-10-0-131-93 kubenswrapper[2564]: I0729 19:22:55.774719 2564 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"apiservice-cert\" (UniqueName: \"kubernetes.io/secret/ed963675-70ff-4f05-84ae-1e2b2702100c-apiservice-cert\") pod \"opendatahub-operator-controller-manager-d84699fcf-k4tz9\" (UID: \"ed963675-70ff-4f05-84ae-1e2b2702100c\") " pod="opendatahub/opendatahub-operator-controller-manager-d84699fcf-k4tz9" Jul 29 19:22:55.774803 ip-10-0-131-93 kubenswrapper[2564]: I0729 19:22:55.774720 2564 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"webhook-cert\" (UniqueName: \"kubernetes.io/secret/ed963675-70ff-4f05-84ae-1e2b2702100c-webhook-cert\") pod \"opendatahub-operator-controller-manager-d84699fcf-k4tz9\" (UID: \"ed963675-70ff-4f05-84ae-1e2b2702100c\") " pod="opendatahub/opendatahub-operator-controller-manager-d84699fcf-k4tz9" Jul 29 19:22:55.780970 ip-10-0-131-93 kubenswrapper[2564]: I0729 19:22:55.780948 2564 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-fn4q5\" (UniqueName: \"kubernetes.io/projected/ed963675-70ff-4f05-84ae-1e2b2702100c-kube-api-access-fn4q5\") pod \"opendatahub-operator-controller-manager-d84699fcf-k4tz9\" (UID: \"ed963675-70ff-4f05-84ae-1e2b2702100c\") " pod="opendatahub/opendatahub-operator-controller-manager-d84699fcf-k4tz9" Jul 29 19:22:55.864594 ip-10-0-131-93 kubenswrapper[2564]: I0729 19:22:55.864197 2564 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="opendatahub/opendatahub-operator-controller-manager-d84699fcf-k4tz9" Jul 29 19:22:56.015438 ip-10-0-131-93 kubenswrapper[2564]: I0729 19:22:56.015413 2564 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["opendatahub/opendatahub-operator-controller-manager-d84699fcf-k4tz9"] Jul 29 19:22:56.017457 ip-10-0-131-93 kubenswrapper[2564]: W0729 19:22:56.017430 2564 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-poded963675_70ff_4f05_84ae_1e2b2702100c.slice/crio-b014c1730338d77ed67f729f49f8052db5c1e720e49f9c17b480124a43fff35b WatchSource:0}: Error finding container b014c1730338d77ed67f729f49f8052db5c1e720e49f9c17b480124a43fff35b: Status 404 returned error can't find the container with id b014c1730338d77ed67f729f49f8052db5c1e720e49f9c17b480124a43fff35b Jul 29 19:22:56.049583 ip-10-0-131-93 kubenswrapper[2564]: I0729 19:22:56.049530 2564 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="opendatahub/opendatahub-operator-controller-manager-d84699fcf-k4tz9" event={"ID":"ed963675-70ff-4f05-84ae-1e2b2702100c","Type":"ContainerStarted","Data":"b014c1730338d77ed67f729f49f8052db5c1e720e49f9c17b480124a43fff35b"} Jul 29 19:22:56.340628 ip-10-0-131-93 kubenswrapper[2564]: I0729 19:22:56.340597 2564 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-444jx_faa8c546-c75b-4e10-93b4-661aa2b5f53e/konnectivity-agent/0.log" limit=1024 Jul 29 19:22:56.442646 ip-10-0-131-93 kubenswrapper[2564]: I0729 19:22:56.442605 2564 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-444jx_faa8c546-c75b-4e10-93b4-661aa2b5f53e/konnectivity-agent/0.log" limit=1024 Jul 29 19:23:01.074573 ip-10-0-131-93 kubenswrapper[2564]: I0729 19:23:01.074534 2564 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="opendatahub/opendatahub-operator-controller-manager-d84699fcf-k4tz9" event={"ID":"ed963675-70ff-4f05-84ae-1e2b2702100c","Type":"ContainerStarted","Data":"2f42d13412e340c0f7f3ccc20d210a32bb29616282f9706061f9737f9d6f181b"} Jul 29 19:23:02.080694 ip-10-0-131-93 kubenswrapper[2564]: I0729 19:23:02.080662 2564 generic.go:356] "Generic (PLEG): container finished" podID="ed963675-70ff-4f05-84ae-1e2b2702100c" containerID="2f42d13412e340c0f7f3ccc20d210a32bb29616282f9706061f9737f9d6f181b" exitCode=0 Jul 29 19:23:02.081062 ip-10-0-131-93 kubenswrapper[2564]: I0729 19:23:02.080712 2564 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="opendatahub/opendatahub-operator-controller-manager-d84699fcf-k4tz9" event={"ID":"ed963675-70ff-4f05-84ae-1e2b2702100c","Type":"ContainerDied","Data":"2f42d13412e340c0f7f3ccc20d210a32bb29616282f9706061f9737f9d6f181b"} Jul 29 19:23:03.086513 ip-10-0-131-93 kubenswrapper[2564]: I0729 19:23:03.086466 2564 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="opendatahub/opendatahub-operator-controller-manager-d84699fcf-k4tz9" event={"ID":"ed963675-70ff-4f05-84ae-1e2b2702100c","Type":"ContainerStarted","Data":"843b4eba3d1459dd41c45cf38a6cf5b0fa94e5ccb18270e96f976c1ea0ddfde7"} Jul 29 19:23:03.086861 ip-10-0-131-93 kubenswrapper[2564]: I0729 19:23:03.086592 2564 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="opendatahub/opendatahub-operator-controller-manager-d84699fcf-k4tz9" Jul 29 19:23:03.110321 ip-10-0-131-93 kubenswrapper[2564]: I0729 19:23:03.110269 2564 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="opendatahub/opendatahub-operator-controller-manager-d84699fcf-k4tz9" podStartSLOduration=2.691314041 podStartE2EDuration="8.110253145s" totalImagesPullingTime="4.414099421s" totalInitContainerRuntime="1.004839683s" isStatefulPod=false podCreationTimestamp="2026-07-29 19:22:55 +0000 UTC" imagePullSessionsCount=2 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-29 19:23:03.109245076 +0000 UTC m=+358.833304768" watchObservedRunningTime="2026-07-29 19:23:03.110253145 +0000 UTC m=+358.834312838" Jul 29 19:23:05.046023 ip-10-0-131-93 kubenswrapper[2564]: I0729 19:23:05.045997 2564 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-lws-operator/lws-controller-manager-ffdbfbbc8-d65tt" Jul 29 19:23:14.093100 ip-10-0-131-93 kubenswrapper[2564]: I0729 19:23:14.093069 2564 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="opendatahub/opendatahub-operator-controller-manager-d84699fcf-k4tz9" Jul 29 19:23:15.597556 ip-10-0-131-93 kubenswrapper[2564]: I0729 19:23:15.597501 2564 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-444jx_faa8c546-c75b-4e10-93b4-661aa2b5f53e/konnectivity-agent/0.log" limit=1024 Jul 29 19:23:15.733071 ip-10-0-131-93 kubenswrapper[2564]: I0729 19:23:15.733037 2564 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-444jx_faa8c546-c75b-4e10-93b4-661aa2b5f53e/konnectivity-agent/0.log" limit=1024 Jul 29 19:23:21.585171 ip-10-0-131-93 kubenswrapper[2564]: I0729 19:23:21.585134 2564 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-444jx_faa8c546-c75b-4e10-93b4-661aa2b5f53e/konnectivity-agent/0.log" limit=1024 Jul 29 19:23:21.937157 ip-10-0-131-93 kubenswrapper[2564]: I0729 19:23:21.937124 2564 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-444jx_faa8c546-c75b-4e10-93b4-661aa2b5f53e/konnectivity-agent/0.log" limit=1024 Jul 29 19:23:22.116186 ip-10-0-131-93 kubenswrapper[2564]: I0729 19:23:22.116156 2564 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-444jx_faa8c546-c75b-4e10-93b4-661aa2b5f53e/konnectivity-agent/0.log" limit=1024 Jul 29 19:23:22.295549 ip-10-0-131-93 kubenswrapper[2564]: I0729 19:23:22.295468 2564 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-444jx_faa8c546-c75b-4e10-93b4-661aa2b5f53e/konnectivity-agent/0.log" limit=1024 Jul 29 19:23:22.498269 ip-10-0-131-93 kubenswrapper[2564]: I0729 19:23:22.498238 2564 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-444jx_faa8c546-c75b-4e10-93b4-661aa2b5f53e/konnectivity-agent/0.log" limit=1024 Jul 29 19:23:22.698620 ip-10-0-131-93 kubenswrapper[2564]: I0729 19:23:22.698591 2564 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-444jx_faa8c546-c75b-4e10-93b4-661aa2b5f53e/konnectivity-agent/0.log" limit=1024 Jul 29 19:23:23.464007 ip-10-0-131-93 kubenswrapper[2564]: I0729 19:23:23.463908 2564 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-444jx_faa8c546-c75b-4e10-93b4-661aa2b5f53e/konnectivity-agent/0.log" limit=1024 Jul 29 19:23:23.819615 ip-10-0-131-93 kubenswrapper[2564]: I0729 19:23:23.819523 2564 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-444jx_faa8c546-c75b-4e10-93b4-661aa2b5f53e/konnectivity-agent/0.log" limit=1024 Jul 29 19:23:23.937066 ip-10-0-131-93 kubenswrapper[2564]: I0729 19:23:23.937016 2564 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-444jx_faa8c546-c75b-4e10-93b4-661aa2b5f53e/konnectivity-agent/0.log" limit=1024 Jul 29 19:23:24.051482 ip-10-0-131-93 kubenswrapper[2564]: I0729 19:23:24.051436 2564 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-444jx_faa8c546-c75b-4e10-93b4-661aa2b5f53e/konnectivity-agent/0.log" limit=1024 Jul 29 19:23:25.568555 ip-10-0-131-93 kubenswrapper[2564]: I0729 19:23:25.568523 2564 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-444jx_faa8c546-c75b-4e10-93b4-661aa2b5f53e/konnectivity-agent/0.log" limit=1024 Jul 29 19:23:29.422457 ip-10-0-131-93 kubenswrapper[2564]: I0729 19:23:29.422424 2564 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-444jx_faa8c546-c75b-4e10-93b4-661aa2b5f53e/konnectivity-agent/0.log" limit=1024 Jul 29 19:23:29.604191 ip-10-0-131-93 kubenswrapper[2564]: I0729 19:23:29.604156 2564 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-444jx_faa8c546-c75b-4e10-93b4-661aa2b5f53e/konnectivity-agent/0.log" limit=1024 Jul 29 19:23:29.784578 ip-10-0-131-93 kubenswrapper[2564]: I0729 19:23:29.784538 2564 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-ingress/data-science-gateway-data-science-gateway-class-d66cd8c7c-jzkd8"] Jul 29 19:23:29.787461 ip-10-0-131-93 kubenswrapper[2564]: I0729 19:23:29.787439 2564 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/data-science-gateway-data-science-gateway-class-d66cd8c7c-jzkd8" Jul 29 19:23:29.790121 ip-10-0-131-93 kubenswrapper[2564]: I0729 19:23:29.790101 2564 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ingress\"/\"openshift-gw-ca-root-cert\"" Jul 29 19:23:29.790663 ip-10-0-131-93 kubenswrapper[2564]: I0729 19:23:29.790645 2564 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ingress\"/\"kube-root-ca.crt\"" Jul 29 19:23:29.790756 ip-10-0-131-93 kubenswrapper[2564]: I0729 19:23:29.790662 2564 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ingress\"/\"openshift-service-ca.crt\"" Jul 29 19:23:29.791183 ip-10-0-131-93 kubenswrapper[2564]: I0729 19:23:29.791165 2564 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress\"/\"data-science-gateway-data-science-gateway-class-dockercfg-tn8sv\"" Jul 29 19:23:29.806981 ip-10-0-131-93 kubenswrapper[2564]: I0729 19:23:29.806959 2564 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/data-science-gateway-data-science-gateway-class-d66cd8c7c-jzkd8"] Jul 29 19:23:29.864256 ip-10-0-131-93 kubenswrapper[2564]: I0729 19:23:29.864224 2564 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-444jx_faa8c546-c75b-4e10-93b4-661aa2b5f53e/konnectivity-agent/0.log" limit=1024 Jul 29 19:23:29.944146 ip-10-0-131-93 kubenswrapper[2564]: I0729 19:23:29.944111 2564 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/ed425122-f06f-4c32-aaeb-dbe3595cfd27-workload-certs\") pod \"data-science-gateway-data-science-gateway-class-d66cd8c7c-jzkd8\" (UID: \"ed425122-f06f-4c32-aaeb-dbe3595cfd27\") " pod="openshift-ingress/data-science-gateway-data-science-gateway-class-d66cd8c7c-jzkd8" Jul 29 19:23:29.944146 ip-10-0-131-93 kubenswrapper[2564]: I0729 19:23:29.944145 2564 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/ed425122-f06f-4c32-aaeb-dbe3595cfd27-workload-socket\") pod \"data-science-gateway-data-science-gateway-class-d66cd8c7c-jzkd8\" (UID: \"ed425122-f06f-4c32-aaeb-dbe3595cfd27\") " pod="openshift-ingress/data-science-gateway-data-science-gateway-class-d66cd8c7c-jzkd8" Jul 29 19:23:29.944430 ip-10-0-131-93 kubenswrapper[2564]: I0729 19:23:29.944171 2564 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/ed425122-f06f-4c32-aaeb-dbe3595cfd27-istiod-ca-cert\") pod \"data-science-gateway-data-science-gateway-class-d66cd8c7c-jzkd8\" (UID: \"ed425122-f06f-4c32-aaeb-dbe3595cfd27\") " pod="openshift-ingress/data-science-gateway-data-science-gateway-class-d66cd8c7c-jzkd8" Jul 29 19:23:29.944430 ip-10-0-131-93 kubenswrapper[2564]: I0729 19:23:29.944237 2564 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/ed425122-f06f-4c32-aaeb-dbe3595cfd27-istio-token\") pod \"data-science-gateway-data-science-gateway-class-d66cd8c7c-jzkd8\" (UID: \"ed425122-f06f-4c32-aaeb-dbe3595cfd27\") " pod="openshift-ingress/data-science-gateway-data-science-gateway-class-d66cd8c7c-jzkd8" Jul 29 19:23:29.944430 ip-10-0-131-93 kubenswrapper[2564]: I0729 19:23:29.944285 2564 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/ed425122-f06f-4c32-aaeb-dbe3595cfd27-credential-socket\") pod \"data-science-gateway-data-science-gateway-class-d66cd8c7c-jzkd8\" (UID: \"ed425122-f06f-4c32-aaeb-dbe3595cfd27\") " pod="openshift-ingress/data-science-gateway-data-science-gateway-class-d66cd8c7c-jzkd8" Jul 29 19:23:29.944430 ip-10-0-131-93 kubenswrapper[2564]: I0729 19:23:29.944305 2564 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/ed425122-f06f-4c32-aaeb-dbe3595cfd27-istio-envoy\") pod \"data-science-gateway-data-science-gateway-class-d66cd8c7c-jzkd8\" (UID: \"ed425122-f06f-4c32-aaeb-dbe3595cfd27\") " pod="openshift-ingress/data-science-gateway-data-science-gateway-class-d66cd8c7c-jzkd8" Jul 29 19:23:29.944430 ip-10-0-131-93 kubenswrapper[2564]: I0729 19:23:29.944345 2564 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/ed425122-f06f-4c32-aaeb-dbe3595cfd27-istio-podinfo\") pod \"data-science-gateway-data-science-gateway-class-d66cd8c7c-jzkd8\" (UID: \"ed425122-f06f-4c32-aaeb-dbe3595cfd27\") " pod="openshift-ingress/data-science-gateway-data-science-gateway-class-d66cd8c7c-jzkd8" Jul 29 19:23:29.944430 ip-10-0-131-93 kubenswrapper[2564]: I0729 19:23:29.944428 2564 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/ed425122-f06f-4c32-aaeb-dbe3595cfd27-istio-data\") pod \"data-science-gateway-data-science-gateway-class-d66cd8c7c-jzkd8\" (UID: \"ed425122-f06f-4c32-aaeb-dbe3595cfd27\") " pod="openshift-ingress/data-science-gateway-data-science-gateway-class-d66cd8c7c-jzkd8" Jul 29 19:23:29.944709 ip-10-0-131-93 kubenswrapper[2564]: I0729 19:23:29.944447 2564 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-cf4pf\" (UniqueName: \"kubernetes.io/projected/ed425122-f06f-4c32-aaeb-dbe3595cfd27-kube-api-access-cf4pf\") pod \"data-science-gateway-data-science-gateway-class-d66cd8c7c-jzkd8\" (UID: \"ed425122-f06f-4c32-aaeb-dbe3595cfd27\") " pod="openshift-ingress/data-science-gateway-data-science-gateway-class-d66cd8c7c-jzkd8" Jul 29 19:23:29.966858 ip-10-0-131-93 kubenswrapper[2564]: I0729 19:23:29.966826 2564 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-ingress/istiod-openshift-gateway-5677ccb49b-xdczl"] Jul 29 19:23:29.969573 ip-10-0-131-93 kubenswrapper[2564]: I0729 19:23:29.969556 2564 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/istiod-openshift-gateway-5677ccb49b-xdczl" Jul 29 19:23:29.972498 ip-10-0-131-93 kubenswrapper[2564]: I0729 19:23:29.972470 2564 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress\"/\"istio-kubeconfig\"" Jul 29 19:23:29.972498 ip-10-0-131-93 kubenswrapper[2564]: I0729 19:23:29.972485 2564 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress\"/\"istiod-openshift-gateway-dockercfg-flws2\"" Jul 29 19:23:29.980543 ip-10-0-131-93 kubenswrapper[2564]: I0729 19:23:29.980524 2564 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress\"/\"istiod-tls\"" Jul 29 19:23:29.980638 ip-10-0-131-93 kubenswrapper[2564]: I0729 19:23:29.980525 2564 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress\"/\"cacerts\"" Jul 29 19:23:30.005948 ip-10-0-131-93 kubenswrapper[2564]: I0729 19:23:30.005917 2564 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/istiod-openshift-gateway-5677ccb49b-xdczl"] Jul 29 19:23:30.045390 ip-10-0-131-93 kubenswrapper[2564]: I0729 19:23:30.045302 2564 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/ed425122-f06f-4c32-aaeb-dbe3595cfd27-istio-data\") pod \"data-science-gateway-data-science-gateway-class-d66cd8c7c-jzkd8\" (UID: \"ed425122-f06f-4c32-aaeb-dbe3595cfd27\") " pod="openshift-ingress/data-science-gateway-data-science-gateway-class-d66cd8c7c-jzkd8" Jul 29 19:23:30.045390 ip-10-0-131-93 kubenswrapper[2564]: I0729 19:23:30.045332 2564 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-cf4pf\" (UniqueName: \"kubernetes.io/projected/ed425122-f06f-4c32-aaeb-dbe3595cfd27-kube-api-access-cf4pf\") pod \"data-science-gateway-data-science-gateway-class-d66cd8c7c-jzkd8\" (UID: \"ed425122-f06f-4c32-aaeb-dbe3595cfd27\") " pod="openshift-ingress/data-science-gateway-data-science-gateway-class-d66cd8c7c-jzkd8" Jul 29 19:23:30.045390 ip-10-0-131-93 kubenswrapper[2564]: I0729 19:23:30.045355 2564 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/ed425122-f06f-4c32-aaeb-dbe3595cfd27-workload-certs\") pod \"data-science-gateway-data-science-gateway-class-d66cd8c7c-jzkd8\" (UID: \"ed425122-f06f-4c32-aaeb-dbe3595cfd27\") " pod="openshift-ingress/data-science-gateway-data-science-gateway-class-d66cd8c7c-jzkd8" Jul 29 19:23:30.045641 ip-10-0-131-93 kubenswrapper[2564]: I0729 19:23:30.045392 2564 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/ed425122-f06f-4c32-aaeb-dbe3595cfd27-workload-socket\") pod \"data-science-gateway-data-science-gateway-class-d66cd8c7c-jzkd8\" (UID: \"ed425122-f06f-4c32-aaeb-dbe3595cfd27\") " pod="openshift-ingress/data-science-gateway-data-science-gateway-class-d66cd8c7c-jzkd8" Jul 29 19:23:30.045641 ip-10-0-131-93 kubenswrapper[2564]: I0729 19:23:30.045441 2564 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/ed425122-f06f-4c32-aaeb-dbe3595cfd27-istiod-ca-cert\") pod \"data-science-gateway-data-science-gateway-class-d66cd8c7c-jzkd8\" (UID: \"ed425122-f06f-4c32-aaeb-dbe3595cfd27\") " pod="openshift-ingress/data-science-gateway-data-science-gateway-class-d66cd8c7c-jzkd8" Jul 29 19:23:30.045641 ip-10-0-131-93 kubenswrapper[2564]: I0729 19:23:30.045501 2564 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/ed425122-f06f-4c32-aaeb-dbe3595cfd27-istio-token\") pod \"data-science-gateway-data-science-gateway-class-d66cd8c7c-jzkd8\" (UID: \"ed425122-f06f-4c32-aaeb-dbe3595cfd27\") " pod="openshift-ingress/data-science-gateway-data-science-gateway-class-d66cd8c7c-jzkd8" Jul 29 19:23:30.045641 ip-10-0-131-93 kubenswrapper[2564]: I0729 19:23:30.045522 2564 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/ed425122-f06f-4c32-aaeb-dbe3595cfd27-credential-socket\") pod \"data-science-gateway-data-science-gateway-class-d66cd8c7c-jzkd8\" (UID: \"ed425122-f06f-4c32-aaeb-dbe3595cfd27\") " pod="openshift-ingress/data-science-gateway-data-science-gateway-class-d66cd8c7c-jzkd8" Jul 29 19:23:30.045641 ip-10-0-131-93 kubenswrapper[2564]: I0729 19:23:30.045543 2564 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/ed425122-f06f-4c32-aaeb-dbe3595cfd27-istio-envoy\") pod \"data-science-gateway-data-science-gateway-class-d66cd8c7c-jzkd8\" (UID: \"ed425122-f06f-4c32-aaeb-dbe3595cfd27\") " pod="openshift-ingress/data-science-gateway-data-science-gateway-class-d66cd8c7c-jzkd8" Jul 29 19:23:30.045641 ip-10-0-131-93 kubenswrapper[2564]: I0729 19:23:30.045585 2564 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/ed425122-f06f-4c32-aaeb-dbe3595cfd27-istio-podinfo\") pod \"data-science-gateway-data-science-gateway-class-d66cd8c7c-jzkd8\" (UID: \"ed425122-f06f-4c32-aaeb-dbe3595cfd27\") " pod="openshift-ingress/data-science-gateway-data-science-gateway-class-d66cd8c7c-jzkd8" Jul 29 19:23:30.045938 ip-10-0-131-93 kubenswrapper[2564]: I0729 19:23:30.045773 2564 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/ed425122-f06f-4c32-aaeb-dbe3595cfd27-istio-data\") pod \"data-science-gateway-data-science-gateway-class-d66cd8c7c-jzkd8\" (UID: \"ed425122-f06f-4c32-aaeb-dbe3595cfd27\") " pod="openshift-ingress/data-science-gateway-data-science-gateway-class-d66cd8c7c-jzkd8" Jul 29 19:23:30.045938 ip-10-0-131-93 kubenswrapper[2564]: I0729 19:23:30.045805 2564 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/ed425122-f06f-4c32-aaeb-dbe3595cfd27-workload-certs\") pod \"data-science-gateway-data-science-gateway-class-d66cd8c7c-jzkd8\" (UID: \"ed425122-f06f-4c32-aaeb-dbe3595cfd27\") " pod="openshift-ingress/data-science-gateway-data-science-gateway-class-d66cd8c7c-jzkd8" Jul 29 19:23:30.046051 ip-10-0-131-93 kubenswrapper[2564]: I0729 19:23:30.045935 2564 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/ed425122-f06f-4c32-aaeb-dbe3595cfd27-workload-socket\") pod \"data-science-gateway-data-science-gateway-class-d66cd8c7c-jzkd8\" (UID: \"ed425122-f06f-4c32-aaeb-dbe3595cfd27\") " pod="openshift-ingress/data-science-gateway-data-science-gateway-class-d66cd8c7c-jzkd8" Jul 29 19:23:30.046051 ip-10-0-131-93 kubenswrapper[2564]: I0729 19:23:30.045984 2564 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/ed425122-f06f-4c32-aaeb-dbe3595cfd27-credential-socket\") pod \"data-science-gateway-data-science-gateway-class-d66cd8c7c-jzkd8\" (UID: \"ed425122-f06f-4c32-aaeb-dbe3595cfd27\") " pod="openshift-ingress/data-science-gateway-data-science-gateway-class-d66cd8c7c-jzkd8" Jul 29 19:23:30.046215 ip-10-0-131-93 kubenswrapper[2564]: I0729 19:23:30.046194 2564 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/ed425122-f06f-4c32-aaeb-dbe3595cfd27-istiod-ca-cert\") pod \"data-science-gateway-data-science-gateway-class-d66cd8c7c-jzkd8\" (UID: \"ed425122-f06f-4c32-aaeb-dbe3595cfd27\") " pod="openshift-ingress/data-science-gateway-data-science-gateway-class-d66cd8c7c-jzkd8" Jul 29 19:23:30.047934 ip-10-0-131-93 kubenswrapper[2564]: I0729 19:23:30.047912 2564 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/ed425122-f06f-4c32-aaeb-dbe3595cfd27-istio-envoy\") pod \"data-science-gateway-data-science-gateway-class-d66cd8c7c-jzkd8\" (UID: \"ed425122-f06f-4c32-aaeb-dbe3595cfd27\") " pod="openshift-ingress/data-science-gateway-data-science-gateway-class-d66cd8c7c-jzkd8" Jul 29 19:23:30.048006 ip-10-0-131-93 kubenswrapper[2564]: I0729 19:23:30.047987 2564 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/ed425122-f06f-4c32-aaeb-dbe3595cfd27-istio-podinfo\") pod \"data-science-gateway-data-science-gateway-class-d66cd8c7c-jzkd8\" (UID: \"ed425122-f06f-4c32-aaeb-dbe3595cfd27\") " pod="openshift-ingress/data-science-gateway-data-science-gateway-class-d66cd8c7c-jzkd8" Jul 29 19:23:30.083955 ip-10-0-131-93 kubenswrapper[2564]: I0729 19:23:30.083921 2564 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-cf4pf\" (UniqueName: \"kubernetes.io/projected/ed425122-f06f-4c32-aaeb-dbe3595cfd27-kube-api-access-cf4pf\") pod \"data-science-gateway-data-science-gateway-class-d66cd8c7c-jzkd8\" (UID: \"ed425122-f06f-4c32-aaeb-dbe3595cfd27\") " pod="openshift-ingress/data-science-gateway-data-science-gateway-class-d66cd8c7c-jzkd8" Jul 29 19:23:30.097145 ip-10-0-131-93 kubenswrapper[2564]: I0729 19:23:30.097122 2564 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/ed425122-f06f-4c32-aaeb-dbe3595cfd27-istio-token\") pod \"data-science-gateway-data-science-gateway-class-d66cd8c7c-jzkd8\" (UID: \"ed425122-f06f-4c32-aaeb-dbe3595cfd27\") " pod="openshift-ingress/data-science-gateway-data-science-gateway-class-d66cd8c7c-jzkd8" Jul 29 19:23:30.146512 ip-10-0-131-93 kubenswrapper[2564]: I0729 19:23:30.146463 2564 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"local-certs\" (UniqueName: \"kubernetes.io/empty-dir/6178ee97-b1fe-4de9-9fc6-2101b33978fe-local-certs\") pod \"istiod-openshift-gateway-5677ccb49b-xdczl\" (UID: \"6178ee97-b1fe-4de9-9fc6-2101b33978fe\") " pod="openshift-ingress/istiod-openshift-gateway-5677ccb49b-xdczl" Jul 29 19:23:30.146512 ip-10-0-131-93 kubenswrapper[2564]: I0729 19:23:30.146514 2564 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-kubeconfig\" (UniqueName: \"kubernetes.io/secret/6178ee97-b1fe-4de9-9fc6-2101b33978fe-istio-kubeconfig\") pod \"istiod-openshift-gateway-5677ccb49b-xdczl\" (UID: \"6178ee97-b1fe-4de9-9fc6-2101b33978fe\") " pod="openshift-ingress/istiod-openshift-gateway-5677ccb49b-xdczl" Jul 29 19:23:30.146769 ip-10-0-131-93 kubenswrapper[2564]: I0729 19:23:30.146578 2564 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"cacerts\" (UniqueName: \"kubernetes.io/secret/6178ee97-b1fe-4de9-9fc6-2101b33978fe-cacerts\") pod \"istiod-openshift-gateway-5677ccb49b-xdczl\" (UID: \"6178ee97-b1fe-4de9-9fc6-2101b33978fe\") " pod="openshift-ingress/istiod-openshift-gateway-5677ccb49b-xdczl" Jul 29 19:23:30.146769 ip-10-0-131-93 kubenswrapper[2564]: I0729 19:23:30.146611 2564 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-csr-dns-cert\" (UniqueName: \"kubernetes.io/secret/6178ee97-b1fe-4de9-9fc6-2101b33978fe-istio-csr-dns-cert\") pod \"istiod-openshift-gateway-5677ccb49b-xdczl\" (UID: \"6178ee97-b1fe-4de9-9fc6-2101b33978fe\") " pod="openshift-ingress/istiod-openshift-gateway-5677ccb49b-xdczl" Jul 29 19:23:30.146769 ip-10-0-131-93 kubenswrapper[2564]: I0729 19:23:30.146635 2564 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-lxtc5\" (UniqueName: \"kubernetes.io/projected/6178ee97-b1fe-4de9-9fc6-2101b33978fe-kube-api-access-lxtc5\") pod \"istiod-openshift-gateway-5677ccb49b-xdczl\" (UID: \"6178ee97-b1fe-4de9-9fc6-2101b33978fe\") " pod="openshift-ingress/istiod-openshift-gateway-5677ccb49b-xdczl" Jul 29 19:23:30.146769 ip-10-0-131-93 kubenswrapper[2564]: I0729 19:23:30.146659 2564 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/6178ee97-b1fe-4de9-9fc6-2101b33978fe-istio-token\") pod \"istiod-openshift-gateway-5677ccb49b-xdczl\" (UID: \"6178ee97-b1fe-4de9-9fc6-2101b33978fe\") " pod="openshift-ingress/istiod-openshift-gateway-5677ccb49b-xdczl" Jul 29 19:23:30.146769 ip-10-0-131-93 kubenswrapper[2564]: I0729 19:23:30.146708 2564 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-csr-ca-configmap\" (UniqueName: \"kubernetes.io/configmap/6178ee97-b1fe-4de9-9fc6-2101b33978fe-istio-csr-ca-configmap\") pod \"istiod-openshift-gateway-5677ccb49b-xdczl\" (UID: \"6178ee97-b1fe-4de9-9fc6-2101b33978fe\") " pod="openshift-ingress/istiod-openshift-gateway-5677ccb49b-xdczl" Jul 29 19:23:30.247209 ip-10-0-131-93 kubenswrapper[2564]: I0729 19:23:30.247171 2564 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"cacerts\" (UniqueName: \"kubernetes.io/secret/6178ee97-b1fe-4de9-9fc6-2101b33978fe-cacerts\") pod \"istiod-openshift-gateway-5677ccb49b-xdczl\" (UID: \"6178ee97-b1fe-4de9-9fc6-2101b33978fe\") " pod="openshift-ingress/istiod-openshift-gateway-5677ccb49b-xdczl" Jul 29 19:23:30.247442 ip-10-0-131-93 kubenswrapper[2564]: I0729 19:23:30.247217 2564 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-csr-dns-cert\" (UniqueName: \"kubernetes.io/secret/6178ee97-b1fe-4de9-9fc6-2101b33978fe-istio-csr-dns-cert\") pod \"istiod-openshift-gateway-5677ccb49b-xdczl\" (UID: \"6178ee97-b1fe-4de9-9fc6-2101b33978fe\") " pod="openshift-ingress/istiod-openshift-gateway-5677ccb49b-xdczl" Jul 29 19:23:30.247442 ip-10-0-131-93 kubenswrapper[2564]: I0729 19:23:30.247279 2564 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-lxtc5\" (UniqueName: \"kubernetes.io/projected/6178ee97-b1fe-4de9-9fc6-2101b33978fe-kube-api-access-lxtc5\") pod \"istiod-openshift-gateway-5677ccb49b-xdczl\" (UID: \"6178ee97-b1fe-4de9-9fc6-2101b33978fe\") " pod="openshift-ingress/istiod-openshift-gateway-5677ccb49b-xdczl" Jul 29 19:23:30.247442 ip-10-0-131-93 kubenswrapper[2564]: I0729 19:23:30.247305 2564 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/6178ee97-b1fe-4de9-9fc6-2101b33978fe-istio-token\") pod \"istiod-openshift-gateway-5677ccb49b-xdczl\" (UID: \"6178ee97-b1fe-4de9-9fc6-2101b33978fe\") " pod="openshift-ingress/istiod-openshift-gateway-5677ccb49b-xdczl" Jul 29 19:23:30.247442 ip-10-0-131-93 kubenswrapper[2564]: I0729 19:23:30.247354 2564 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-csr-ca-configmap\" (UniqueName: \"kubernetes.io/configmap/6178ee97-b1fe-4de9-9fc6-2101b33978fe-istio-csr-ca-configmap\") pod \"istiod-openshift-gateway-5677ccb49b-xdczl\" (UID: \"6178ee97-b1fe-4de9-9fc6-2101b33978fe\") " pod="openshift-ingress/istiod-openshift-gateway-5677ccb49b-xdczl" Jul 29 19:23:30.247442 ip-10-0-131-93 kubenswrapper[2564]: I0729 19:23:30.247439 2564 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"local-certs\" (UniqueName: \"kubernetes.io/empty-dir/6178ee97-b1fe-4de9-9fc6-2101b33978fe-local-certs\") pod \"istiod-openshift-gateway-5677ccb49b-xdczl\" (UID: \"6178ee97-b1fe-4de9-9fc6-2101b33978fe\") " pod="openshift-ingress/istiod-openshift-gateway-5677ccb49b-xdczl" Jul 29 19:23:30.247716 ip-10-0-131-93 kubenswrapper[2564]: I0729 19:23:30.247462 2564 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-kubeconfig\" (UniqueName: \"kubernetes.io/secret/6178ee97-b1fe-4de9-9fc6-2101b33978fe-istio-kubeconfig\") pod \"istiod-openshift-gateway-5677ccb49b-xdczl\" (UID: \"6178ee97-b1fe-4de9-9fc6-2101b33978fe\") " pod="openshift-ingress/istiod-openshift-gateway-5677ccb49b-xdczl" Jul 29 19:23:30.248199 ip-10-0-131-93 kubenswrapper[2564]: I0729 19:23:30.248168 2564 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istio-csr-ca-configmap\" (UniqueName: \"kubernetes.io/configmap/6178ee97-b1fe-4de9-9fc6-2101b33978fe-istio-csr-ca-configmap\") pod \"istiod-openshift-gateway-5677ccb49b-xdczl\" (UID: \"6178ee97-b1fe-4de9-9fc6-2101b33978fe\") " pod="openshift-ingress/istiod-openshift-gateway-5677ccb49b-xdczl" Jul 29 19:23:30.249699 ip-10-0-131-93 kubenswrapper[2564]: I0729 19:23:30.249665 2564 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"local-certs\" (UniqueName: \"kubernetes.io/empty-dir/6178ee97-b1fe-4de9-9fc6-2101b33978fe-local-certs\") pod \"istiod-openshift-gateway-5677ccb49b-xdczl\" (UID: \"6178ee97-b1fe-4de9-9fc6-2101b33978fe\") " pod="openshift-ingress/istiod-openshift-gateway-5677ccb49b-xdczl" Jul 29 19:23:30.249870 ip-10-0-131-93 kubenswrapper[2564]: I0729 19:23:30.249835 2564 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"cacerts\" (UniqueName: \"kubernetes.io/secret/6178ee97-b1fe-4de9-9fc6-2101b33978fe-cacerts\") pod \"istiod-openshift-gateway-5677ccb49b-xdczl\" (UID: \"6178ee97-b1fe-4de9-9fc6-2101b33978fe\") " pod="openshift-ingress/istiod-openshift-gateway-5677ccb49b-xdczl" Jul 29 19:23:30.249870 ip-10-0-131-93 kubenswrapper[2564]: I0729 19:23:30.249852 2564 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istio-csr-dns-cert\" (UniqueName: \"kubernetes.io/secret/6178ee97-b1fe-4de9-9fc6-2101b33978fe-istio-csr-dns-cert\") pod \"istiod-openshift-gateway-5677ccb49b-xdczl\" (UID: \"6178ee97-b1fe-4de9-9fc6-2101b33978fe\") " pod="openshift-ingress/istiod-openshift-gateway-5677ccb49b-xdczl" Jul 29 19:23:30.249987 ip-10-0-131-93 kubenswrapper[2564]: I0729 19:23:30.249700 2564 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istio-kubeconfig\" (UniqueName: \"kubernetes.io/secret/6178ee97-b1fe-4de9-9fc6-2101b33978fe-istio-kubeconfig\") pod \"istiod-openshift-gateway-5677ccb49b-xdczl\" (UID: \"6178ee97-b1fe-4de9-9fc6-2101b33978fe\") " pod="openshift-ingress/istiod-openshift-gateway-5677ccb49b-xdczl" Jul 29 19:23:30.273056 ip-10-0-131-93 kubenswrapper[2564]: I0729 19:23:30.273031 2564 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/6178ee97-b1fe-4de9-9fc6-2101b33978fe-istio-token\") pod \"istiod-openshift-gateway-5677ccb49b-xdczl\" (UID: \"6178ee97-b1fe-4de9-9fc6-2101b33978fe\") " pod="openshift-ingress/istiod-openshift-gateway-5677ccb49b-xdczl" Jul 29 19:23:30.273179 ip-10-0-131-93 kubenswrapper[2564]: I0729 19:23:30.273135 2564 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-lxtc5\" (UniqueName: \"kubernetes.io/projected/6178ee97-b1fe-4de9-9fc6-2101b33978fe-kube-api-access-lxtc5\") pod \"istiod-openshift-gateway-5677ccb49b-xdczl\" (UID: \"6178ee97-b1fe-4de9-9fc6-2101b33978fe\") " pod="openshift-ingress/istiod-openshift-gateway-5677ccb49b-xdczl" Jul 29 19:23:30.278962 ip-10-0-131-93 kubenswrapper[2564]: I0729 19:23:30.278942 2564 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/istiod-openshift-gateway-5677ccb49b-xdczl" Jul 29 19:23:30.396948 ip-10-0-131-93 kubenswrapper[2564]: I0729 19:23:30.396916 2564 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/data-science-gateway-data-science-gateway-class-d66cd8c7c-jzkd8" Jul 29 19:23:30.424134 ip-10-0-131-93 kubenswrapper[2564]: I0729 19:23:30.424109 2564 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/istiod-openshift-gateway-5677ccb49b-xdczl"] Jul 29 19:23:30.426083 ip-10-0-131-93 kubenswrapper[2564]: W0729 19:23:30.426050 2564 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod6178ee97_b1fe_4de9_9fc6_2101b33978fe.slice/crio-5cf07c2ea6276ba1c0f974329d892de41897548a0cdb5aff200874c4829c658f WatchSource:0}: Error finding container 5cf07c2ea6276ba1c0f974329d892de41897548a0cdb5aff200874c4829c658f: Status 404 returned error can't find the container with id 5cf07c2ea6276ba1c0f974329d892de41897548a0cdb5aff200874c4829c658f Jul 29 19:23:30.532033 ip-10-0-131-93 kubenswrapper[2564]: I0729 19:23:30.532011 2564 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/data-science-gateway-data-science-gateway-class-d66cd8c7c-jzkd8"] Jul 29 19:23:30.534803 ip-10-0-131-93 kubenswrapper[2564]: W0729 19:23:30.534774 2564 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-poded425122_f06f_4c32_aaeb_dbe3595cfd27.slice/crio-41205ea12ba0bf85548121aa22cbe309e672feb430496fdc25e62ef4210ca761 WatchSource:0}: Error finding container 41205ea12ba0bf85548121aa22cbe309e672feb430496fdc25e62ef4210ca761: Status 404 returned error can't find the container with id 41205ea12ba0bf85548121aa22cbe309e672feb430496fdc25e62ef4210ca761 Jul 29 19:23:31.202124 ip-10-0-131-93 kubenswrapper[2564]: I0729 19:23:31.202087 2564 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/data-science-gateway-data-science-gateway-class-d66cd8c7c-jzkd8" event={"ID":"ed425122-f06f-4c32-aaeb-dbe3595cfd27","Type":"ContainerStarted","Data":"41205ea12ba0bf85548121aa22cbe309e672feb430496fdc25e62ef4210ca761"} Jul 29 19:23:31.203860 ip-10-0-131-93 kubenswrapper[2564]: I0729 19:23:31.203826 2564 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/istiod-openshift-gateway-5677ccb49b-xdczl" event={"ID":"6178ee97-b1fe-4de9-9fc6-2101b33978fe","Type":"ContainerStarted","Data":"5cf07c2ea6276ba1c0f974329d892de41897548a0cdb5aff200874c4829c658f"} Jul 29 19:23:31.514118 ip-10-0-131-93 kubenswrapper[2564]: I0729 19:23:31.514034 2564 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-444jx_faa8c546-c75b-4e10-93b4-661aa2b5f53e/konnectivity-agent/0.log" limit=1024 Jul 29 19:23:31.578858 ip-10-0-131-93 kubenswrapper[2564]: I0729 19:23:31.578814 2564 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-444jx_faa8c546-c75b-4e10-93b4-661aa2b5f53e/konnectivity-agent/0.log" limit=1024 Jul 29 19:23:34.016562 ip-10-0-131-93 kubenswrapper[2564]: I0729 19:23:34.016523 2564 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236224Ki","pods":"250"} Jul 29 19:23:34.016866 ip-10-0-131-93 kubenswrapper[2564]: I0729 19:23:34.016594 2564 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236224Ki","pods":"250"} Jul 29 19:23:34.019601 ip-10-0-131-93 kubenswrapper[2564]: I0729 19:23:34.019568 2564 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236224Ki","pods":"250"} Jul 29 19:23:34.019706 ip-10-0-131-93 kubenswrapper[2564]: I0729 19:23:34.019645 2564 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236224Ki","pods":"250"} Jul 29 19:23:34.019706 ip-10-0-131-93 kubenswrapper[2564]: I0729 19:23:34.019690 2564 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236224Ki","pods":"250"} Jul 29 19:23:34.033631 ip-10-0-131-93 kubenswrapper[2564]: I0729 19:23:34.033605 2564 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-444jx_faa8c546-c75b-4e10-93b4-661aa2b5f53e/konnectivity-agent/0.log" limit=1024 Jul 29 19:23:34.195794 ip-10-0-131-93 kubenswrapper[2564]: I0729 19:23:34.195763 2564 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-444jx_faa8c546-c75b-4e10-93b4-661aa2b5f53e/konnectivity-agent/0.log" limit=1024 Jul 29 19:23:34.223167 ip-10-0-131-93 kubenswrapper[2564]: I0729 19:23:34.223131 2564 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/data-science-gateway-data-science-gateway-class-d66cd8c7c-jzkd8" event={"ID":"ed425122-f06f-4c32-aaeb-dbe3595cfd27","Type":"ContainerStarted","Data":"1c5b19697f7106fef4fbbe7642ccae0b634d97e524beaee0e4216e065e339223"} Jul 29 19:23:34.224986 ip-10-0-131-93 kubenswrapper[2564]: I0729 19:23:34.224952 2564 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/istiod-openshift-gateway-5677ccb49b-xdczl" event={"ID":"6178ee97-b1fe-4de9-9fc6-2101b33978fe","Type":"ContainerStarted","Data":"56c9222dfb10bc46e6f26ecaa5ad8f6d980e47d3a862f0feb51172aaeb76c6a8"} Jul 29 19:23:34.225106 ip-10-0-131-93 kubenswrapper[2564]: I0729 19:23:34.225020 2564 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/istiod-openshift-gateway-5677ccb49b-xdczl" Jul 29 19:23:34.280590 ip-10-0-131-93 kubenswrapper[2564]: I0729 19:23:34.280476 2564 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-ingress/data-science-gateway-data-science-gateway-class-d66cd8c7c-jzkd8" podStartSLOduration=1.798013134 podStartE2EDuration="5.2804594s" totalImagesPullingTime="3.482446266s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-29 19:23:29 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-29 19:23:34.279181944 +0000 UTC m=+390.003241637" watchObservedRunningTime="2026-07-29 19:23:34.2804594 +0000 UTC m=+390.004519091" Jul 29 19:23:34.351261 ip-10-0-131-93 kubenswrapper[2564]: I0729 19:23:34.351201 2564 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-ingress/istiod-openshift-gateway-5677ccb49b-xdczl" podStartSLOduration=1.762964598 podStartE2EDuration="5.351185889s" totalImagesPullingTime="3.588221291s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-29 19:23:29 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-29 19:23:34.34804111 +0000 UTC m=+390.072100802" watchObservedRunningTime="2026-07-29 19:23:34.351185889 +0000 UTC m=+390.075245581" Jul 29 19:23:34.384452 ip-10-0-131-93 kubenswrapper[2564]: I0729 19:23:34.384406 2564 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-444jx_faa8c546-c75b-4e10-93b4-661aa2b5f53e/konnectivity-agent/0.log" limit=1024 Jul 29 19:23:34.397419 ip-10-0-131-93 kubenswrapper[2564]: I0729 19:23:34.397393 2564 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="openshift-ingress/data-science-gateway-data-science-gateway-class-d66cd8c7c-jzkd8" Jul 29 19:23:34.565119 ip-10-0-131-93 kubenswrapper[2564]: I0729 19:23:34.565034 2564 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-444jx_faa8c546-c75b-4e10-93b4-661aa2b5f53e/konnectivity-agent/0.log" limit=1024 Jul 29 19:23:35.231504 ip-10-0-131-93 kubenswrapper[2564]: I0729 19:23:35.231474 2564 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/istiod-openshift-gateway-5677ccb49b-xdczl" Jul 29 19:23:35.341777 ip-10-0-131-93 kubenswrapper[2564]: I0729 19:23:35.341746 2564 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-444jx_faa8c546-c75b-4e10-93b4-661aa2b5f53e/konnectivity-agent/0.log" limit=1024 Jul 29 19:23:35.403426 ip-10-0-131-93 kubenswrapper[2564]: I0729 19:23:35.403399 2564 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="openshift-ingress/data-science-gateway-data-science-gateway-class-d66cd8c7c-jzkd8" Jul 29 19:23:35.437391 ip-10-0-131-93 kubenswrapper[2564]: I0729 19:23:35.437341 2564 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-444jx_faa8c546-c75b-4e10-93b4-661aa2b5f53e/konnectivity-agent/0.log" limit=1024 Jul 29 19:23:35.537233 ip-10-0-131-93 kubenswrapper[2564]: I0729 19:23:35.537155 2564 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-444jx_faa8c546-c75b-4e10-93b4-661aa2b5f53e/konnectivity-agent/0.log" limit=1024 Jul 29 19:23:36.234109 ip-10-0-131-93 kubenswrapper[2564]: I0729 19:23:36.234076 2564 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/data-science-gateway-data-science-gateway-class-d66cd8c7c-jzkd8" Jul 29 19:23:36.235116 ip-10-0-131-93 kubenswrapper[2564]: I0729 19:23:36.235099 2564 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/data-science-gateway-data-science-gateway-class-d66cd8c7c-jzkd8" Jul 29 19:24:17.058394 ip-10-0-131-93 kubenswrapper[2564]: I0729 19:24:17.058341 2564 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-444jx_faa8c546-c75b-4e10-93b4-661aa2b5f53e/konnectivity-agent/0.log" limit=1024 Jul 29 19:24:17.156206 ip-10-0-131-93 kubenswrapper[2564]: I0729 19:24:17.156175 2564 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-444jx_faa8c546-c75b-4e10-93b4-661aa2b5f53e/konnectivity-agent/0.log" limit=1024 Jul 29 19:24:17.310675 ip-10-0-131-93 kubenswrapper[2564]: I0729 19:24:17.310594 2564 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-444jx_faa8c546-c75b-4e10-93b4-661aa2b5f53e/konnectivity-agent/0.log" limit=1024 Jul 29 19:24:17.942117 ip-10-0-131-93 kubenswrapper[2564]: I0729 19:24:17.942083 2564 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-444jx_faa8c546-c75b-4e10-93b4-661aa2b5f53e/konnectivity-agent/0.log" limit=1024 Jul 29 19:24:18.155127 ip-10-0-131-93 kubenswrapper[2564]: I0729 19:24:18.155093 2564 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-444jx_faa8c546-c75b-4e10-93b4-661aa2b5f53e/konnectivity-agent/0.log" limit=1024 Jul 29 19:24:32.756698 ip-10-0-131-93 kubenswrapper[2564]: I0729 19:24:32.756619 2564 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["kuadrant-system/19cb86e64775c5699d5aacf881a09c2d51e7f55f9e1ff096f2a667c5e0cwwkj"] Jul 29 19:24:32.758990 ip-10-0-131-93 kubenswrapper[2564]: I0729 19:24:32.758972 2564 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/19cb86e64775c5699d5aacf881a09c2d51e7f55f9e1ff096f2a667c5e0cwwkj" Jul 29 19:24:32.761355 ip-10-0-131-93 kubenswrapper[2564]: I0729 19:24:32.761335 2564 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"kuadrant-system\"/\"default-dockercfg-w9774\"" Jul 29 19:24:32.761355 ip-10-0-131-93 kubenswrapper[2564]: I0729 19:24:32.761353 2564 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"kuadrant-system\"/\"kube-root-ca.crt\"" Jul 29 19:24:32.761548 ip-10-0-131-93 kubenswrapper[2564]: I0729 19:24:32.761452 2564 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"kuadrant-system\"/\"openshift-service-ca.crt\"" Jul 29 19:24:32.768150 ip-10-0-131-93 kubenswrapper[2564]: I0729 19:24:32.768128 2564 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/19cb86e64775c5699d5aacf881a09c2d51e7f55f9e1ff096f2a667c5e0cwwkj"] Jul 29 19:24:32.823998 ip-10-0-131-93 kubenswrapper[2564]: I0729 19:24:32.823958 2564 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-spfbx\" (UniqueName: \"kubernetes.io/projected/9e24b906-6ade-47ef-818c-28a1c450e0be-kube-api-access-spfbx\") pod \"19cb86e64775c5699d5aacf881a09c2d51e7f55f9e1ff096f2a667c5e0cwwkj\" (UID: \"9e24b906-6ade-47ef-818c-28a1c450e0be\") " pod="kuadrant-system/19cb86e64775c5699d5aacf881a09c2d51e7f55f9e1ff096f2a667c5e0cwwkj" Jul 29 19:24:32.823998 ip-10-0-131-93 kubenswrapper[2564]: I0729 19:24:32.824003 2564 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"util\" (UniqueName: \"kubernetes.io/empty-dir/9e24b906-6ade-47ef-818c-28a1c450e0be-util\") pod \"19cb86e64775c5699d5aacf881a09c2d51e7f55f9e1ff096f2a667c5e0cwwkj\" (UID: \"9e24b906-6ade-47ef-818c-28a1c450e0be\") " pod="kuadrant-system/19cb86e64775c5699d5aacf881a09c2d51e7f55f9e1ff096f2a667c5e0cwwkj" Jul 29 19:24:32.824198 ip-10-0-131-93 kubenswrapper[2564]: I0729 19:24:32.824088 2564 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"bundle\" (UniqueName: \"kubernetes.io/empty-dir/9e24b906-6ade-47ef-818c-28a1c450e0be-bundle\") pod \"19cb86e64775c5699d5aacf881a09c2d51e7f55f9e1ff096f2a667c5e0cwwkj\" (UID: \"9e24b906-6ade-47ef-818c-28a1c450e0be\") " pod="kuadrant-system/19cb86e64775c5699d5aacf881a09c2d51e7f55f9e1ff096f2a667c5e0cwwkj" Jul 29 19:24:32.925340 ip-10-0-131-93 kubenswrapper[2564]: I0729 19:24:32.925297 2564 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-spfbx\" (UniqueName: \"kubernetes.io/projected/9e24b906-6ade-47ef-818c-28a1c450e0be-kube-api-access-spfbx\") pod \"19cb86e64775c5699d5aacf881a09c2d51e7f55f9e1ff096f2a667c5e0cwwkj\" (UID: \"9e24b906-6ade-47ef-818c-28a1c450e0be\") " pod="kuadrant-system/19cb86e64775c5699d5aacf881a09c2d51e7f55f9e1ff096f2a667c5e0cwwkj" Jul 29 19:24:32.925340 ip-10-0-131-93 kubenswrapper[2564]: I0729 19:24:32.925344 2564 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"util\" (UniqueName: \"kubernetes.io/empty-dir/9e24b906-6ade-47ef-818c-28a1c450e0be-util\") pod \"19cb86e64775c5699d5aacf881a09c2d51e7f55f9e1ff096f2a667c5e0cwwkj\" (UID: \"9e24b906-6ade-47ef-818c-28a1c450e0be\") " pod="kuadrant-system/19cb86e64775c5699d5aacf881a09c2d51e7f55f9e1ff096f2a667c5e0cwwkj" Jul 29 19:24:32.925614 ip-10-0-131-93 kubenswrapper[2564]: I0729 19:24:32.925411 2564 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"bundle\" (UniqueName: \"kubernetes.io/empty-dir/9e24b906-6ade-47ef-818c-28a1c450e0be-bundle\") pod \"19cb86e64775c5699d5aacf881a09c2d51e7f55f9e1ff096f2a667c5e0cwwkj\" (UID: \"9e24b906-6ade-47ef-818c-28a1c450e0be\") " pod="kuadrant-system/19cb86e64775c5699d5aacf881a09c2d51e7f55f9e1ff096f2a667c5e0cwwkj" Jul 29 19:24:32.925786 ip-10-0-131-93 kubenswrapper[2564]: I0729 19:24:32.925764 2564 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"util\" (UniqueName: \"kubernetes.io/empty-dir/9e24b906-6ade-47ef-818c-28a1c450e0be-util\") pod \"19cb86e64775c5699d5aacf881a09c2d51e7f55f9e1ff096f2a667c5e0cwwkj\" (UID: \"9e24b906-6ade-47ef-818c-28a1c450e0be\") " pod="kuadrant-system/19cb86e64775c5699d5aacf881a09c2d51e7f55f9e1ff096f2a667c5e0cwwkj" Jul 29 19:24:32.925847 ip-10-0-131-93 kubenswrapper[2564]: I0729 19:24:32.925791 2564 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"bundle\" (UniqueName: \"kubernetes.io/empty-dir/9e24b906-6ade-47ef-818c-28a1c450e0be-bundle\") pod \"19cb86e64775c5699d5aacf881a09c2d51e7f55f9e1ff096f2a667c5e0cwwkj\" (UID: \"9e24b906-6ade-47ef-818c-28a1c450e0be\") " pod="kuadrant-system/19cb86e64775c5699d5aacf881a09c2d51e7f55f9e1ff096f2a667c5e0cwwkj" Jul 29 19:24:32.935302 ip-10-0-131-93 kubenswrapper[2564]: I0729 19:24:32.935276 2564 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-spfbx\" (UniqueName: \"kubernetes.io/projected/9e24b906-6ade-47ef-818c-28a1c450e0be-kube-api-access-spfbx\") pod \"19cb86e64775c5699d5aacf881a09c2d51e7f55f9e1ff096f2a667c5e0cwwkj\" (UID: \"9e24b906-6ade-47ef-818c-28a1c450e0be\") " pod="kuadrant-system/19cb86e64775c5699d5aacf881a09c2d51e7f55f9e1ff096f2a667c5e0cwwkj" Jul 29 19:24:33.068720 ip-10-0-131-93 kubenswrapper[2564]: I0729 19:24:33.068622 2564 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/19cb86e64775c5699d5aacf881a09c2d51e7f55f9e1ff096f2a667c5e0cwwkj" Jul 29 19:24:33.163282 ip-10-0-131-93 kubenswrapper[2564]: I0729 19:24:33.163254 2564 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["kuadrant-system/9438e18bbad664a016701a3153a6b421e2d977be7eb0117f80bcf45ef1g46qj"] Jul 29 19:24:33.166251 ip-10-0-131-93 kubenswrapper[2564]: I0729 19:24:33.166233 2564 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/9438e18bbad664a016701a3153a6b421e2d977be7eb0117f80bcf45ef1g46qj" Jul 29 19:24:33.177829 ip-10-0-131-93 kubenswrapper[2564]: I0729 19:24:33.177794 2564 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/9438e18bbad664a016701a3153a6b421e2d977be7eb0117f80bcf45ef1g46qj"] Jul 29 19:24:33.203777 ip-10-0-131-93 kubenswrapper[2564]: I0729 19:24:33.203754 2564 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/19cb86e64775c5699d5aacf881a09c2d51e7f55f9e1ff096f2a667c5e0cwwkj"] Jul 29 19:24:33.206505 ip-10-0-131-93 kubenswrapper[2564]: W0729 19:24:33.206477 2564 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod9e24b906_6ade_47ef_818c_28a1c450e0be.slice/crio-b614d899e49759119cb512023c9afb78f05b9243b04dc60d8696d1dcaa17208b WatchSource:0}: Error finding container b614d899e49759119cb512023c9afb78f05b9243b04dc60d8696d1dcaa17208b: Status 404 returned error can't find the container with id b614d899e49759119cb512023c9afb78f05b9243b04dc60d8696d1dcaa17208b Jul 29 19:24:33.228632 ip-10-0-131-93 kubenswrapper[2564]: I0729 19:24:33.228609 2564 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"bundle\" (UniqueName: \"kubernetes.io/empty-dir/dcd5230c-d041-4831-8280-c00a332f038d-bundle\") pod \"9438e18bbad664a016701a3153a6b421e2d977be7eb0117f80bcf45ef1g46qj\" (UID: \"dcd5230c-d041-4831-8280-c00a332f038d\") " pod="kuadrant-system/9438e18bbad664a016701a3153a6b421e2d977be7eb0117f80bcf45ef1g46qj" Jul 29 19:24:33.228711 ip-10-0-131-93 kubenswrapper[2564]: I0729 19:24:33.228676 2564 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"util\" (UniqueName: \"kubernetes.io/empty-dir/dcd5230c-d041-4831-8280-c00a332f038d-util\") pod \"9438e18bbad664a016701a3153a6b421e2d977be7eb0117f80bcf45ef1g46qj\" (UID: \"dcd5230c-d041-4831-8280-c00a332f038d\") " pod="kuadrant-system/9438e18bbad664a016701a3153a6b421e2d977be7eb0117f80bcf45ef1g46qj" Jul 29 19:24:33.228747 ip-10-0-131-93 kubenswrapper[2564]: I0729 19:24:33.228716 2564 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-kc5ns\" (UniqueName: \"kubernetes.io/projected/dcd5230c-d041-4831-8280-c00a332f038d-kube-api-access-kc5ns\") pod \"9438e18bbad664a016701a3153a6b421e2d977be7eb0117f80bcf45ef1g46qj\" (UID: \"dcd5230c-d041-4831-8280-c00a332f038d\") " pod="kuadrant-system/9438e18bbad664a016701a3153a6b421e2d977be7eb0117f80bcf45ef1g46qj" Jul 29 19:24:33.329338 ip-10-0-131-93 kubenswrapper[2564]: I0729 19:24:33.329257 2564 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"util\" (UniqueName: \"kubernetes.io/empty-dir/dcd5230c-d041-4831-8280-c00a332f038d-util\") pod \"9438e18bbad664a016701a3153a6b421e2d977be7eb0117f80bcf45ef1g46qj\" (UID: \"dcd5230c-d041-4831-8280-c00a332f038d\") " pod="kuadrant-system/9438e18bbad664a016701a3153a6b421e2d977be7eb0117f80bcf45ef1g46qj" Jul 29 19:24:33.329338 ip-10-0-131-93 kubenswrapper[2564]: I0729 19:24:33.329312 2564 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-kc5ns\" (UniqueName: \"kubernetes.io/projected/dcd5230c-d041-4831-8280-c00a332f038d-kube-api-access-kc5ns\") pod \"9438e18bbad664a016701a3153a6b421e2d977be7eb0117f80bcf45ef1g46qj\" (UID: \"dcd5230c-d041-4831-8280-c00a332f038d\") " pod="kuadrant-system/9438e18bbad664a016701a3153a6b421e2d977be7eb0117f80bcf45ef1g46qj" Jul 29 19:24:33.329577 ip-10-0-131-93 kubenswrapper[2564]: I0729 19:24:33.329348 2564 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"bundle\" (UniqueName: \"kubernetes.io/empty-dir/dcd5230c-d041-4831-8280-c00a332f038d-bundle\") pod \"9438e18bbad664a016701a3153a6b421e2d977be7eb0117f80bcf45ef1g46qj\" (UID: \"dcd5230c-d041-4831-8280-c00a332f038d\") " pod="kuadrant-system/9438e18bbad664a016701a3153a6b421e2d977be7eb0117f80bcf45ef1g46qj" Jul 29 19:24:33.329711 ip-10-0-131-93 kubenswrapper[2564]: I0729 19:24:33.329690 2564 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"util\" (UniqueName: \"kubernetes.io/empty-dir/dcd5230c-d041-4831-8280-c00a332f038d-util\") pod \"9438e18bbad664a016701a3153a6b421e2d977be7eb0117f80bcf45ef1g46qj\" (UID: \"dcd5230c-d041-4831-8280-c00a332f038d\") " pod="kuadrant-system/9438e18bbad664a016701a3153a6b421e2d977be7eb0117f80bcf45ef1g46qj" Jul 29 19:24:33.329770 ip-10-0-131-93 kubenswrapper[2564]: I0729 19:24:33.329721 2564 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"bundle\" (UniqueName: \"kubernetes.io/empty-dir/dcd5230c-d041-4831-8280-c00a332f038d-bundle\") pod \"9438e18bbad664a016701a3153a6b421e2d977be7eb0117f80bcf45ef1g46qj\" (UID: \"dcd5230c-d041-4831-8280-c00a332f038d\") " pod="kuadrant-system/9438e18bbad664a016701a3153a6b421e2d977be7eb0117f80bcf45ef1g46qj" Jul 29 19:24:33.337264 ip-10-0-131-93 kubenswrapper[2564]: I0729 19:24:33.337243 2564 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-kc5ns\" (UniqueName: \"kubernetes.io/projected/dcd5230c-d041-4831-8280-c00a332f038d-kube-api-access-kc5ns\") pod \"9438e18bbad664a016701a3153a6b421e2d977be7eb0117f80bcf45ef1g46qj\" (UID: \"dcd5230c-d041-4831-8280-c00a332f038d\") " pod="kuadrant-system/9438e18bbad664a016701a3153a6b421e2d977be7eb0117f80bcf45ef1g46qj" Jul 29 19:24:33.476112 ip-10-0-131-93 kubenswrapper[2564]: I0729 19:24:33.476070 2564 generic.go:356] "Generic (PLEG): container finished" podID="9e24b906-6ade-47ef-818c-28a1c450e0be" containerID="4943e1cf8047daeb9e1846771d781eaa1af6a8910ac6c7ec49271bea16e1722a" exitCode=0 Jul 29 19:24:33.476284 ip-10-0-131-93 kubenswrapper[2564]: I0729 19:24:33.476119 2564 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/19cb86e64775c5699d5aacf881a09c2d51e7f55f9e1ff096f2a667c5e0cwwkj" event={"ID":"9e24b906-6ade-47ef-818c-28a1c450e0be","Type":"ContainerDied","Data":"4943e1cf8047daeb9e1846771d781eaa1af6a8910ac6c7ec49271bea16e1722a"} Jul 29 19:24:33.476284 ip-10-0-131-93 kubenswrapper[2564]: I0729 19:24:33.476169 2564 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/19cb86e64775c5699d5aacf881a09c2d51e7f55f9e1ff096f2a667c5e0cwwkj" event={"ID":"9e24b906-6ade-47ef-818c-28a1c450e0be","Type":"ContainerStarted","Data":"b614d899e49759119cb512023c9afb78f05b9243b04dc60d8696d1dcaa17208b"} Jul 29 19:24:33.479080 ip-10-0-131-93 kubenswrapper[2564]: I0729 19:24:33.479062 2564 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/9438e18bbad664a016701a3153a6b421e2d977be7eb0117f80bcf45ef1g46qj" Jul 29 19:24:33.558523 ip-10-0-131-93 kubenswrapper[2564]: I0729 19:24:33.558495 2564 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["kuadrant-system/5fc36eb1065777914bfe0ff85f9a202e2a3bafefb563d9e2994474ed736jqbs"] Jul 29 19:24:33.561243 ip-10-0-131-93 kubenswrapper[2564]: I0729 19:24:33.561220 2564 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/5fc36eb1065777914bfe0ff85f9a202e2a3bafefb563d9e2994474ed736jqbs" Jul 29 19:24:33.571935 ip-10-0-131-93 kubenswrapper[2564]: I0729 19:24:33.571910 2564 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/5fc36eb1065777914bfe0ff85f9a202e2a3bafefb563d9e2994474ed736jqbs"] Jul 29 19:24:33.607005 ip-10-0-131-93 kubenswrapper[2564]: I0729 19:24:33.606924 2564 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/9438e18bbad664a016701a3153a6b421e2d977be7eb0117f80bcf45ef1g46qj"] Jul 29 19:24:33.610706 ip-10-0-131-93 kubenswrapper[2564]: W0729 19:24:33.610681 2564 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-poddcd5230c_d041_4831_8280_c00a332f038d.slice/crio-45cd2fb09fbff6e73de365e880af9006e67342b3f9ce279fc17fc3d5b269dfd5 WatchSource:0}: Error finding container 45cd2fb09fbff6e73de365e880af9006e67342b3f9ce279fc17fc3d5b269dfd5: Status 404 returned error can't find the container with id 45cd2fb09fbff6e73de365e880af9006e67342b3f9ce279fc17fc3d5b269dfd5 Jul 29 19:24:33.633033 ip-10-0-131-93 kubenswrapper[2564]: I0729 19:24:33.633011 2564 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-4k2ng\" (UniqueName: \"kubernetes.io/projected/83b6dd40-61a5-45e0-b860-ef5f84491141-kube-api-access-4k2ng\") pod \"5fc36eb1065777914bfe0ff85f9a202e2a3bafefb563d9e2994474ed736jqbs\" (UID: \"83b6dd40-61a5-45e0-b860-ef5f84491141\") " pod="kuadrant-system/5fc36eb1065777914bfe0ff85f9a202e2a3bafefb563d9e2994474ed736jqbs" Jul 29 19:24:33.633118 ip-10-0-131-93 kubenswrapper[2564]: I0729 19:24:33.633071 2564 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"bundle\" (UniqueName: \"kubernetes.io/empty-dir/83b6dd40-61a5-45e0-b860-ef5f84491141-bundle\") pod \"5fc36eb1065777914bfe0ff85f9a202e2a3bafefb563d9e2994474ed736jqbs\" (UID: \"83b6dd40-61a5-45e0-b860-ef5f84491141\") " pod="kuadrant-system/5fc36eb1065777914bfe0ff85f9a202e2a3bafefb563d9e2994474ed736jqbs" Jul 29 19:24:33.633156 ip-10-0-131-93 kubenswrapper[2564]: I0729 19:24:33.633127 2564 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"util\" (UniqueName: \"kubernetes.io/empty-dir/83b6dd40-61a5-45e0-b860-ef5f84491141-util\") pod \"5fc36eb1065777914bfe0ff85f9a202e2a3bafefb563d9e2994474ed736jqbs\" (UID: \"83b6dd40-61a5-45e0-b860-ef5f84491141\") " pod="kuadrant-system/5fc36eb1065777914bfe0ff85f9a202e2a3bafefb563d9e2994474ed736jqbs" Jul 29 19:24:33.733815 ip-10-0-131-93 kubenswrapper[2564]: I0729 19:24:33.733789 2564 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-4k2ng\" (UniqueName: \"kubernetes.io/projected/83b6dd40-61a5-45e0-b860-ef5f84491141-kube-api-access-4k2ng\") pod \"5fc36eb1065777914bfe0ff85f9a202e2a3bafefb563d9e2994474ed736jqbs\" (UID: \"83b6dd40-61a5-45e0-b860-ef5f84491141\") " pod="kuadrant-system/5fc36eb1065777914bfe0ff85f9a202e2a3bafefb563d9e2994474ed736jqbs" Jul 29 19:24:33.733937 ip-10-0-131-93 kubenswrapper[2564]: I0729 19:24:33.733853 2564 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"bundle\" (UniqueName: \"kubernetes.io/empty-dir/83b6dd40-61a5-45e0-b860-ef5f84491141-bundle\") pod \"5fc36eb1065777914bfe0ff85f9a202e2a3bafefb563d9e2994474ed736jqbs\" (UID: \"83b6dd40-61a5-45e0-b860-ef5f84491141\") " pod="kuadrant-system/5fc36eb1065777914bfe0ff85f9a202e2a3bafefb563d9e2994474ed736jqbs" Jul 29 19:24:33.733937 ip-10-0-131-93 kubenswrapper[2564]: I0729 19:24:33.733875 2564 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"util\" (UniqueName: \"kubernetes.io/empty-dir/83b6dd40-61a5-45e0-b860-ef5f84491141-util\") pod \"5fc36eb1065777914bfe0ff85f9a202e2a3bafefb563d9e2994474ed736jqbs\" (UID: \"83b6dd40-61a5-45e0-b860-ef5f84491141\") " pod="kuadrant-system/5fc36eb1065777914bfe0ff85f9a202e2a3bafefb563d9e2994474ed736jqbs" Jul 29 19:24:33.734157 ip-10-0-131-93 kubenswrapper[2564]: I0729 19:24:33.734140 2564 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"util\" (UniqueName: \"kubernetes.io/empty-dir/83b6dd40-61a5-45e0-b860-ef5f84491141-util\") pod \"5fc36eb1065777914bfe0ff85f9a202e2a3bafefb563d9e2994474ed736jqbs\" (UID: \"83b6dd40-61a5-45e0-b860-ef5f84491141\") " pod="kuadrant-system/5fc36eb1065777914bfe0ff85f9a202e2a3bafefb563d9e2994474ed736jqbs" Jul 29 19:24:33.734217 ip-10-0-131-93 kubenswrapper[2564]: I0729 19:24:33.734182 2564 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"bundle\" (UniqueName: \"kubernetes.io/empty-dir/83b6dd40-61a5-45e0-b860-ef5f84491141-bundle\") pod \"5fc36eb1065777914bfe0ff85f9a202e2a3bafefb563d9e2994474ed736jqbs\" (UID: \"83b6dd40-61a5-45e0-b860-ef5f84491141\") " pod="kuadrant-system/5fc36eb1065777914bfe0ff85f9a202e2a3bafefb563d9e2994474ed736jqbs" Jul 29 19:24:33.742752 ip-10-0-131-93 kubenswrapper[2564]: I0729 19:24:33.742734 2564 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-4k2ng\" (UniqueName: \"kubernetes.io/projected/83b6dd40-61a5-45e0-b860-ef5f84491141-kube-api-access-4k2ng\") pod \"5fc36eb1065777914bfe0ff85f9a202e2a3bafefb563d9e2994474ed736jqbs\" (UID: \"83b6dd40-61a5-45e0-b860-ef5f84491141\") " pod="kuadrant-system/5fc36eb1065777914bfe0ff85f9a202e2a3bafefb563d9e2994474ed736jqbs" Jul 29 19:24:33.872529 ip-10-0-131-93 kubenswrapper[2564]: I0729 19:24:33.872444 2564 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/5fc36eb1065777914bfe0ff85f9a202e2a3bafefb563d9e2994474ed736jqbs" Jul 29 19:24:33.997004 ip-10-0-131-93 kubenswrapper[2564]: I0729 19:24:33.996975 2564 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/5fc36eb1065777914bfe0ff85f9a202e2a3bafefb563d9e2994474ed736jqbs"] Jul 29 19:24:34.031111 ip-10-0-131-93 kubenswrapper[2564]: W0729 19:24:34.031072 2564 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod83b6dd40_61a5_45e0_b860_ef5f84491141.slice/crio-00bc517c864e4d71e4a8ff3c6a8b9269f0c3ae3a0ae9e4b44b55cc963a18a055 WatchSource:0}: Error finding container 00bc517c864e4d71e4a8ff3c6a8b9269f0c3ae3a0ae9e4b44b55cc963a18a055: Status 404 returned error can't find the container with id 00bc517c864e4d71e4a8ff3c6a8b9269f0c3ae3a0ae9e4b44b55cc963a18a055 Jul 29 19:24:34.156734 ip-10-0-131-93 kubenswrapper[2564]: I0729 19:24:34.156704 2564 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["kuadrant-system/0acee64185f523d1d1272e9af2e4d9333e0dcde792ba30e1fa9605b7596mj7k"] Jul 29 19:24:34.158957 ip-10-0-131-93 kubenswrapper[2564]: I0729 19:24:34.158942 2564 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/0acee64185f523d1d1272e9af2e4d9333e0dcde792ba30e1fa9605b7596mj7k" Jul 29 19:24:34.169223 ip-10-0-131-93 kubenswrapper[2564]: I0729 19:24:34.169200 2564 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/0acee64185f523d1d1272e9af2e4d9333e0dcde792ba30e1fa9605b7596mj7k"] Jul 29 19:24:34.237099 ip-10-0-131-93 kubenswrapper[2564]: I0729 19:24:34.237068 2564 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"bundle\" (UniqueName: \"kubernetes.io/empty-dir/32a5fe7d-49f0-44cb-af5a-e4a06e29b452-bundle\") pod \"0acee64185f523d1d1272e9af2e4d9333e0dcde792ba30e1fa9605b7596mj7k\" (UID: \"32a5fe7d-49f0-44cb-af5a-e4a06e29b452\") " pod="kuadrant-system/0acee64185f523d1d1272e9af2e4d9333e0dcde792ba30e1fa9605b7596mj7k" Jul 29 19:24:34.237268 ip-10-0-131-93 kubenswrapper[2564]: I0729 19:24:34.237109 2564 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-6kt5c\" (UniqueName: \"kubernetes.io/projected/32a5fe7d-49f0-44cb-af5a-e4a06e29b452-kube-api-access-6kt5c\") pod \"0acee64185f523d1d1272e9af2e4d9333e0dcde792ba30e1fa9605b7596mj7k\" (UID: \"32a5fe7d-49f0-44cb-af5a-e4a06e29b452\") " pod="kuadrant-system/0acee64185f523d1d1272e9af2e4d9333e0dcde792ba30e1fa9605b7596mj7k" Jul 29 19:24:34.237268 ip-10-0-131-93 kubenswrapper[2564]: I0729 19:24:34.237182 2564 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"util\" (UniqueName: \"kubernetes.io/empty-dir/32a5fe7d-49f0-44cb-af5a-e4a06e29b452-util\") pod \"0acee64185f523d1d1272e9af2e4d9333e0dcde792ba30e1fa9605b7596mj7k\" (UID: \"32a5fe7d-49f0-44cb-af5a-e4a06e29b452\") " pod="kuadrant-system/0acee64185f523d1d1272e9af2e4d9333e0dcde792ba30e1fa9605b7596mj7k" Jul 29 19:24:34.337943 ip-10-0-131-93 kubenswrapper[2564]: I0729 19:24:34.337913 2564 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"bundle\" (UniqueName: \"kubernetes.io/empty-dir/32a5fe7d-49f0-44cb-af5a-e4a06e29b452-bundle\") pod \"0acee64185f523d1d1272e9af2e4d9333e0dcde792ba30e1fa9605b7596mj7k\" (UID: \"32a5fe7d-49f0-44cb-af5a-e4a06e29b452\") " pod="kuadrant-system/0acee64185f523d1d1272e9af2e4d9333e0dcde792ba30e1fa9605b7596mj7k" Jul 29 19:24:34.338118 ip-10-0-131-93 kubenswrapper[2564]: I0729 19:24:34.337952 2564 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-6kt5c\" (UniqueName: \"kubernetes.io/projected/32a5fe7d-49f0-44cb-af5a-e4a06e29b452-kube-api-access-6kt5c\") pod \"0acee64185f523d1d1272e9af2e4d9333e0dcde792ba30e1fa9605b7596mj7k\" (UID: \"32a5fe7d-49f0-44cb-af5a-e4a06e29b452\") " pod="kuadrant-system/0acee64185f523d1d1272e9af2e4d9333e0dcde792ba30e1fa9605b7596mj7k" Jul 29 19:24:34.338118 ip-10-0-131-93 kubenswrapper[2564]: I0729 19:24:34.338075 2564 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"util\" (UniqueName: \"kubernetes.io/empty-dir/32a5fe7d-49f0-44cb-af5a-e4a06e29b452-util\") pod \"0acee64185f523d1d1272e9af2e4d9333e0dcde792ba30e1fa9605b7596mj7k\" (UID: \"32a5fe7d-49f0-44cb-af5a-e4a06e29b452\") " pod="kuadrant-system/0acee64185f523d1d1272e9af2e4d9333e0dcde792ba30e1fa9605b7596mj7k" Jul 29 19:24:34.338287 ip-10-0-131-93 kubenswrapper[2564]: I0729 19:24:34.338271 2564 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"bundle\" (UniqueName: \"kubernetes.io/empty-dir/32a5fe7d-49f0-44cb-af5a-e4a06e29b452-bundle\") pod \"0acee64185f523d1d1272e9af2e4d9333e0dcde792ba30e1fa9605b7596mj7k\" (UID: \"32a5fe7d-49f0-44cb-af5a-e4a06e29b452\") " pod="kuadrant-system/0acee64185f523d1d1272e9af2e4d9333e0dcde792ba30e1fa9605b7596mj7k" Jul 29 19:24:34.338407 ip-10-0-131-93 kubenswrapper[2564]: I0729 19:24:34.338365 2564 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"util\" (UniqueName: \"kubernetes.io/empty-dir/32a5fe7d-49f0-44cb-af5a-e4a06e29b452-util\") pod \"0acee64185f523d1d1272e9af2e4d9333e0dcde792ba30e1fa9605b7596mj7k\" (UID: \"32a5fe7d-49f0-44cb-af5a-e4a06e29b452\") " pod="kuadrant-system/0acee64185f523d1d1272e9af2e4d9333e0dcde792ba30e1fa9605b7596mj7k" Jul 29 19:24:34.347506 ip-10-0-131-93 kubenswrapper[2564]: I0729 19:24:34.347486 2564 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-6kt5c\" (UniqueName: \"kubernetes.io/projected/32a5fe7d-49f0-44cb-af5a-e4a06e29b452-kube-api-access-6kt5c\") pod \"0acee64185f523d1d1272e9af2e4d9333e0dcde792ba30e1fa9605b7596mj7k\" (UID: \"32a5fe7d-49f0-44cb-af5a-e4a06e29b452\") " pod="kuadrant-system/0acee64185f523d1d1272e9af2e4d9333e0dcde792ba30e1fa9605b7596mj7k" Jul 29 19:24:34.481807 ip-10-0-131-93 kubenswrapper[2564]: I0729 19:24:34.481771 2564 generic.go:356] "Generic (PLEG): container finished" podID="83b6dd40-61a5-45e0-b860-ef5f84491141" containerID="007d5f332f081d722c763dab4df62612b68a4ed563e07858b9220fde807ecb77" exitCode=0 Jul 29 19:24:34.482010 ip-10-0-131-93 kubenswrapper[2564]: I0729 19:24:34.481857 2564 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/5fc36eb1065777914bfe0ff85f9a202e2a3bafefb563d9e2994474ed736jqbs" event={"ID":"83b6dd40-61a5-45e0-b860-ef5f84491141","Type":"ContainerDied","Data":"007d5f332f081d722c763dab4df62612b68a4ed563e07858b9220fde807ecb77"} Jul 29 19:24:34.482010 ip-10-0-131-93 kubenswrapper[2564]: I0729 19:24:34.481909 2564 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/5fc36eb1065777914bfe0ff85f9a202e2a3bafefb563d9e2994474ed736jqbs" event={"ID":"83b6dd40-61a5-45e0-b860-ef5f84491141","Type":"ContainerStarted","Data":"00bc517c864e4d71e4a8ff3c6a8b9269f0c3ae3a0ae9e4b44b55cc963a18a055"} Jul 29 19:24:34.483801 ip-10-0-131-93 kubenswrapper[2564]: I0729 19:24:34.483778 2564 generic.go:356] "Generic (PLEG): container finished" podID="9e24b906-6ade-47ef-818c-28a1c450e0be" containerID="d3ef5f80c741fa3f30fb96024f045c2524940051bbeb7f5954b54b2945c1bf58" exitCode=0 Jul 29 19:24:34.483917 ip-10-0-131-93 kubenswrapper[2564]: I0729 19:24:34.483863 2564 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/19cb86e64775c5699d5aacf881a09c2d51e7f55f9e1ff096f2a667c5e0cwwkj" event={"ID":"9e24b906-6ade-47ef-818c-28a1c450e0be","Type":"ContainerDied","Data":"d3ef5f80c741fa3f30fb96024f045c2524940051bbeb7f5954b54b2945c1bf58"} Jul 29 19:24:34.485476 ip-10-0-131-93 kubenswrapper[2564]: I0729 19:24:34.485453 2564 generic.go:356] "Generic (PLEG): container finished" podID="dcd5230c-d041-4831-8280-c00a332f038d" containerID="12f0fd93367587cd4b75d72dbe269300ca9576cf0eb4bde9424691daf75e9f85" exitCode=0 Jul 29 19:24:34.485593 ip-10-0-131-93 kubenswrapper[2564]: I0729 19:24:34.485485 2564 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/9438e18bbad664a016701a3153a6b421e2d977be7eb0117f80bcf45ef1g46qj" event={"ID":"dcd5230c-d041-4831-8280-c00a332f038d","Type":"ContainerDied","Data":"12f0fd93367587cd4b75d72dbe269300ca9576cf0eb4bde9424691daf75e9f85"} Jul 29 19:24:34.485593 ip-10-0-131-93 kubenswrapper[2564]: I0729 19:24:34.485514 2564 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/9438e18bbad664a016701a3153a6b421e2d977be7eb0117f80bcf45ef1g46qj" event={"ID":"dcd5230c-d041-4831-8280-c00a332f038d","Type":"ContainerStarted","Data":"45cd2fb09fbff6e73de365e880af9006e67342b3f9ce279fc17fc3d5b269dfd5"} Jul 29 19:24:34.554618 ip-10-0-131-93 kubenswrapper[2564]: I0729 19:24:34.554597 2564 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/0acee64185f523d1d1272e9af2e4d9333e0dcde792ba30e1fa9605b7596mj7k" Jul 29 19:24:34.692668 ip-10-0-131-93 kubenswrapper[2564]: I0729 19:24:34.692648 2564 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/0acee64185f523d1d1272e9af2e4d9333e0dcde792ba30e1fa9605b7596mj7k"] Jul 29 19:24:34.694889 ip-10-0-131-93 kubenswrapper[2564]: W0729 19:24:34.694863 2564 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod32a5fe7d_49f0_44cb_af5a_e4a06e29b452.slice/crio-d864c20b7c6825c44d0dbd833f9cbf56d28438a6cd3ff98ba5a0135ebacc80d9 WatchSource:0}: Error finding container d864c20b7c6825c44d0dbd833f9cbf56d28438a6cd3ff98ba5a0135ebacc80d9: Status 404 returned error can't find the container with id d864c20b7c6825c44d0dbd833f9cbf56d28438a6cd3ff98ba5a0135ebacc80d9 Jul 29 19:24:35.491946 ip-10-0-131-93 kubenswrapper[2564]: I0729 19:24:35.491914 2564 generic.go:356] "Generic (PLEG): container finished" podID="9e24b906-6ade-47ef-818c-28a1c450e0be" containerID="fa580a330fd0c2643b5bc14f4a7a805933f1fa69bdfab33de714faac10af2c2c" exitCode=0 Jul 29 19:24:35.492331 ip-10-0-131-93 kubenswrapper[2564]: I0729 19:24:35.491997 2564 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/19cb86e64775c5699d5aacf881a09c2d51e7f55f9e1ff096f2a667c5e0cwwkj" event={"ID":"9e24b906-6ade-47ef-818c-28a1c450e0be","Type":"ContainerDied","Data":"fa580a330fd0c2643b5bc14f4a7a805933f1fa69bdfab33de714faac10af2c2c"} Jul 29 19:24:35.493700 ip-10-0-131-93 kubenswrapper[2564]: I0729 19:24:35.493679 2564 generic.go:356] "Generic (PLEG): container finished" podID="dcd5230c-d041-4831-8280-c00a332f038d" containerID="81b192b30df26df8db28062aeae4fe8fa7e4bddd05c206131624926f4dcb800b" exitCode=0 Jul 29 19:24:35.493813 ip-10-0-131-93 kubenswrapper[2564]: I0729 19:24:35.493758 2564 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/9438e18bbad664a016701a3153a6b421e2d977be7eb0117f80bcf45ef1g46qj" event={"ID":"dcd5230c-d041-4831-8280-c00a332f038d","Type":"ContainerDied","Data":"81b192b30df26df8db28062aeae4fe8fa7e4bddd05c206131624926f4dcb800b"} Jul 29 19:24:35.500900 ip-10-0-131-93 kubenswrapper[2564]: I0729 19:24:35.500875 2564 generic.go:356] "Generic (PLEG): container finished" podID="83b6dd40-61a5-45e0-b860-ef5f84491141" containerID="8cd98cfae3de8e5494e57a6789287154a67d832b3765cc4fcfa316236d95cf23" exitCode=0 Jul 29 19:24:35.501009 ip-10-0-131-93 kubenswrapper[2564]: I0729 19:24:35.500963 2564 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/5fc36eb1065777914bfe0ff85f9a202e2a3bafefb563d9e2994474ed736jqbs" event={"ID":"83b6dd40-61a5-45e0-b860-ef5f84491141","Type":"ContainerDied","Data":"8cd98cfae3de8e5494e57a6789287154a67d832b3765cc4fcfa316236d95cf23"} Jul 29 19:24:35.502410 ip-10-0-131-93 kubenswrapper[2564]: I0729 19:24:35.502391 2564 generic.go:356] "Generic (PLEG): container finished" podID="32a5fe7d-49f0-44cb-af5a-e4a06e29b452" containerID="c960047d24601fd1f8666d55fa03d0c7c6b4741f26bf79f5534e3eac1901bec2" exitCode=0 Jul 29 19:24:35.502515 ip-10-0-131-93 kubenswrapper[2564]: I0729 19:24:35.502457 2564 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/0acee64185f523d1d1272e9af2e4d9333e0dcde792ba30e1fa9605b7596mj7k" event={"ID":"32a5fe7d-49f0-44cb-af5a-e4a06e29b452","Type":"ContainerDied","Data":"c960047d24601fd1f8666d55fa03d0c7c6b4741f26bf79f5534e3eac1901bec2"} Jul 29 19:24:35.502515 ip-10-0-131-93 kubenswrapper[2564]: I0729 19:24:35.502506 2564 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/0acee64185f523d1d1272e9af2e4d9333e0dcde792ba30e1fa9605b7596mj7k" event={"ID":"32a5fe7d-49f0-44cb-af5a-e4a06e29b452","Type":"ContainerStarted","Data":"d864c20b7c6825c44d0dbd833f9cbf56d28438a6cd3ff98ba5a0135ebacc80d9"} Jul 29 19:24:36.509115 ip-10-0-131-93 kubenswrapper[2564]: I0729 19:24:36.509080 2564 generic.go:356] "Generic (PLEG): container finished" podID="dcd5230c-d041-4831-8280-c00a332f038d" containerID="12b4305d92ad8d750f56f22c366ec195e1d51b2af2fcd10eda8c0bec32025ff7" exitCode=0 Jul 29 19:24:36.509575 ip-10-0-131-93 kubenswrapper[2564]: I0729 19:24:36.509155 2564 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/9438e18bbad664a016701a3153a6b421e2d977be7eb0117f80bcf45ef1g46qj" event={"ID":"dcd5230c-d041-4831-8280-c00a332f038d","Type":"ContainerDied","Data":"12b4305d92ad8d750f56f22c366ec195e1d51b2af2fcd10eda8c0bec32025ff7"} Jul 29 19:24:36.511166 ip-10-0-131-93 kubenswrapper[2564]: I0729 19:24:36.511148 2564 generic.go:356] "Generic (PLEG): container finished" podID="83b6dd40-61a5-45e0-b860-ef5f84491141" containerID="59e8f6105fc4e9df2423d25da3ec1b80490779c037730076c3a312893d1bf5d8" exitCode=0 Jul 29 19:24:36.511277 ip-10-0-131-93 kubenswrapper[2564]: I0729 19:24:36.511179 2564 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/5fc36eb1065777914bfe0ff85f9a202e2a3bafefb563d9e2994474ed736jqbs" event={"ID":"83b6dd40-61a5-45e0-b860-ef5f84491141","Type":"ContainerDied","Data":"59e8f6105fc4e9df2423d25da3ec1b80490779c037730076c3a312893d1bf5d8"} Jul 29 19:24:36.512781 ip-10-0-131-93 kubenswrapper[2564]: I0729 19:24:36.512760 2564 generic.go:356] "Generic (PLEG): container finished" podID="32a5fe7d-49f0-44cb-af5a-e4a06e29b452" containerID="afb79969c3601e4e6f6717b09772856977eaa066e5ebec6966ccd44b87d99557" exitCode=0 Jul 29 19:24:36.512856 ip-10-0-131-93 kubenswrapper[2564]: I0729 19:24:36.512841 2564 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/0acee64185f523d1d1272e9af2e4d9333e0dcde792ba30e1fa9605b7596mj7k" event={"ID":"32a5fe7d-49f0-44cb-af5a-e4a06e29b452","Type":"ContainerDied","Data":"afb79969c3601e4e6f6717b09772856977eaa066e5ebec6966ccd44b87d99557"} Jul 29 19:24:36.641112 ip-10-0-131-93 kubenswrapper[2564]: I0729 19:24:36.641084 2564 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/19cb86e64775c5699d5aacf881a09c2d51e7f55f9e1ff096f2a667c5e0cwwkj" Jul 29 19:24:36.757963 ip-10-0-131-93 kubenswrapper[2564]: I0729 19:24:36.757934 2564 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"util\" (UniqueName: \"kubernetes.io/empty-dir/9e24b906-6ade-47ef-818c-28a1c450e0be-util\") pod \"9e24b906-6ade-47ef-818c-28a1c450e0be\" (UID: \"9e24b906-6ade-47ef-818c-28a1c450e0be\") " Jul 29 19:24:36.758113 ip-10-0-131-93 kubenswrapper[2564]: I0729 19:24:36.757990 2564 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"bundle\" (UniqueName: \"kubernetes.io/empty-dir/9e24b906-6ade-47ef-818c-28a1c450e0be-bundle\") pod \"9e24b906-6ade-47ef-818c-28a1c450e0be\" (UID: \"9e24b906-6ade-47ef-818c-28a1c450e0be\") " Jul 29 19:24:36.758113 ip-10-0-131-93 kubenswrapper[2564]: I0729 19:24:36.758022 2564 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-spfbx\" (UniqueName: \"kubernetes.io/projected/9e24b906-6ade-47ef-818c-28a1c450e0be-kube-api-access-spfbx\") pod \"9e24b906-6ade-47ef-818c-28a1c450e0be\" (UID: \"9e24b906-6ade-47ef-818c-28a1c450e0be\") " Jul 29 19:24:36.758522 ip-10-0-131-93 kubenswrapper[2564]: I0729 19:24:36.758489 2564 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/9e24b906-6ade-47ef-818c-28a1c450e0be-bundle" (OuterVolumeSpecName: "bundle") pod "9e24b906-6ade-47ef-818c-28a1c450e0be" (UID: "9e24b906-6ade-47ef-818c-28a1c450e0be"). InnerVolumeSpecName "bundle". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Jul 29 19:24:36.760223 ip-10-0-131-93 kubenswrapper[2564]: I0729 19:24:36.760170 2564 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/9e24b906-6ade-47ef-818c-28a1c450e0be-kube-api-access-spfbx" (OuterVolumeSpecName: "kube-api-access-spfbx") pod "9e24b906-6ade-47ef-818c-28a1c450e0be" (UID: "9e24b906-6ade-47ef-818c-28a1c450e0be"). InnerVolumeSpecName "kube-api-access-spfbx". PluginName "kubernetes.io/projected", VolumeGIDValue "" Jul 29 19:24:36.762860 ip-10-0-131-93 kubenswrapper[2564]: I0729 19:24:36.762836 2564 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/9e24b906-6ade-47ef-818c-28a1c450e0be-util" (OuterVolumeSpecName: "util") pod "9e24b906-6ade-47ef-818c-28a1c450e0be" (UID: "9e24b906-6ade-47ef-818c-28a1c450e0be"). InnerVolumeSpecName "util". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Jul 29 19:24:36.858661 ip-10-0-131-93 kubenswrapper[2564]: I0729 19:24:36.858610 2564 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-spfbx\" (UniqueName: \"kubernetes.io/projected/9e24b906-6ade-47ef-818c-28a1c450e0be-kube-api-access-spfbx\") on node \"ip-10-0-131-93.ec2.internal\" DevicePath \"\"" Jul 29 19:24:36.858661 ip-10-0-131-93 kubenswrapper[2564]: I0729 19:24:36.858646 2564 reconciler_common.go:299] "Volume detached for volume \"util\" (UniqueName: \"kubernetes.io/empty-dir/9e24b906-6ade-47ef-818c-28a1c450e0be-util\") on node \"ip-10-0-131-93.ec2.internal\" DevicePath \"\"" Jul 29 19:24:36.858661 ip-10-0-131-93 kubenswrapper[2564]: I0729 19:24:36.858662 2564 reconciler_common.go:299] "Volume detached for volume \"bundle\" (UniqueName: \"kubernetes.io/empty-dir/9e24b906-6ade-47ef-818c-28a1c450e0be-bundle\") on node \"ip-10-0-131-93.ec2.internal\" DevicePath \"\"" Jul 29 19:24:37.518893 ip-10-0-131-93 kubenswrapper[2564]: I0729 19:24:37.518862 2564 generic.go:356] "Generic (PLEG): container finished" podID="32a5fe7d-49f0-44cb-af5a-e4a06e29b452" containerID="716dda33d57d531bc10f952297571350ef72729f89a561dae8b4a923d3626666" exitCode=0 Jul 29 19:24:37.519304 ip-10-0-131-93 kubenswrapper[2564]: I0729 19:24:37.518958 2564 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/0acee64185f523d1d1272e9af2e4d9333e0dcde792ba30e1fa9605b7596mj7k" event={"ID":"32a5fe7d-49f0-44cb-af5a-e4a06e29b452","Type":"ContainerDied","Data":"716dda33d57d531bc10f952297571350ef72729f89a561dae8b4a923d3626666"} Jul 29 19:24:37.520650 ip-10-0-131-93 kubenswrapper[2564]: I0729 19:24:37.520635 2564 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/19cb86e64775c5699d5aacf881a09c2d51e7f55f9e1ff096f2a667c5e0cwwkj" Jul 29 19:24:37.520742 ip-10-0-131-93 kubenswrapper[2564]: I0729 19:24:37.520670 2564 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/19cb86e64775c5699d5aacf881a09c2d51e7f55f9e1ff096f2a667c5e0cwwkj" event={"ID":"9e24b906-6ade-47ef-818c-28a1c450e0be","Type":"ContainerDied","Data":"b614d899e49759119cb512023c9afb78f05b9243b04dc60d8696d1dcaa17208b"} Jul 29 19:24:37.520742 ip-10-0-131-93 kubenswrapper[2564]: I0729 19:24:37.520703 2564 pod_container_deletor.go:80] "Container not found in pod's containers" containerID="b614d899e49759119cb512023c9afb78f05b9243b04dc60d8696d1dcaa17208b" Jul 29 19:24:37.670634 ip-10-0-131-93 kubenswrapper[2564]: I0729 19:24:37.670614 2564 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/5fc36eb1065777914bfe0ff85f9a202e2a3bafefb563d9e2994474ed736jqbs" Jul 29 19:24:37.672889 ip-10-0-131-93 kubenswrapper[2564]: I0729 19:24:37.672871 2564 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/9438e18bbad664a016701a3153a6b421e2d977be7eb0117f80bcf45ef1g46qj" Jul 29 19:24:37.766598 ip-10-0-131-93 kubenswrapper[2564]: I0729 19:24:37.766559 2564 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-kc5ns\" (UniqueName: \"kubernetes.io/projected/dcd5230c-d041-4831-8280-c00a332f038d-kube-api-access-kc5ns\") pod \"dcd5230c-d041-4831-8280-c00a332f038d\" (UID: \"dcd5230c-d041-4831-8280-c00a332f038d\") " Jul 29 19:24:37.766777 ip-10-0-131-93 kubenswrapper[2564]: I0729 19:24:37.766632 2564 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"util\" (UniqueName: \"kubernetes.io/empty-dir/dcd5230c-d041-4831-8280-c00a332f038d-util\") pod \"dcd5230c-d041-4831-8280-c00a332f038d\" (UID: \"dcd5230c-d041-4831-8280-c00a332f038d\") " Jul 29 19:24:37.766777 ip-10-0-131-93 kubenswrapper[2564]: I0729 19:24:37.766655 2564 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-4k2ng\" (UniqueName: \"kubernetes.io/projected/83b6dd40-61a5-45e0-b860-ef5f84491141-kube-api-access-4k2ng\") pod \"83b6dd40-61a5-45e0-b860-ef5f84491141\" (UID: \"83b6dd40-61a5-45e0-b860-ef5f84491141\") " Jul 29 19:24:37.766777 ip-10-0-131-93 kubenswrapper[2564]: I0729 19:24:37.766703 2564 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"bundle\" (UniqueName: \"kubernetes.io/empty-dir/83b6dd40-61a5-45e0-b860-ef5f84491141-bundle\") pod \"83b6dd40-61a5-45e0-b860-ef5f84491141\" (UID: \"83b6dd40-61a5-45e0-b860-ef5f84491141\") " Jul 29 19:24:37.766777 ip-10-0-131-93 kubenswrapper[2564]: I0729 19:24:37.766763 2564 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"bundle\" (UniqueName: \"kubernetes.io/empty-dir/dcd5230c-d041-4831-8280-c00a332f038d-bundle\") pod \"dcd5230c-d041-4831-8280-c00a332f038d\" (UID: \"dcd5230c-d041-4831-8280-c00a332f038d\") " Jul 29 19:24:37.766996 ip-10-0-131-93 kubenswrapper[2564]: I0729 19:24:37.766802 2564 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"util\" (UniqueName: \"kubernetes.io/empty-dir/83b6dd40-61a5-45e0-b860-ef5f84491141-util\") pod \"83b6dd40-61a5-45e0-b860-ef5f84491141\" (UID: \"83b6dd40-61a5-45e0-b860-ef5f84491141\") " Jul 29 19:24:37.767422 ip-10-0-131-93 kubenswrapper[2564]: I0729 19:24:37.767360 2564 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/dcd5230c-d041-4831-8280-c00a332f038d-bundle" (OuterVolumeSpecName: "bundle") pod "dcd5230c-d041-4831-8280-c00a332f038d" (UID: "dcd5230c-d041-4831-8280-c00a332f038d"). InnerVolumeSpecName "bundle". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Jul 29 19:24:37.767565 ip-10-0-131-93 kubenswrapper[2564]: I0729 19:24:37.767480 2564 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/83b6dd40-61a5-45e0-b860-ef5f84491141-bundle" (OuterVolumeSpecName: "bundle") pod "83b6dd40-61a5-45e0-b860-ef5f84491141" (UID: "83b6dd40-61a5-45e0-b860-ef5f84491141"). InnerVolumeSpecName "bundle". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Jul 29 19:24:37.768815 ip-10-0-131-93 kubenswrapper[2564]: I0729 19:24:37.768787 2564 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/dcd5230c-d041-4831-8280-c00a332f038d-kube-api-access-kc5ns" (OuterVolumeSpecName: "kube-api-access-kc5ns") pod "dcd5230c-d041-4831-8280-c00a332f038d" (UID: "dcd5230c-d041-4831-8280-c00a332f038d"). InnerVolumeSpecName "kube-api-access-kc5ns". PluginName "kubernetes.io/projected", VolumeGIDValue "" Jul 29 19:24:37.768955 ip-10-0-131-93 kubenswrapper[2564]: I0729 19:24:37.768914 2564 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/83b6dd40-61a5-45e0-b860-ef5f84491141-kube-api-access-4k2ng" (OuterVolumeSpecName: "kube-api-access-4k2ng") pod "83b6dd40-61a5-45e0-b860-ef5f84491141" (UID: "83b6dd40-61a5-45e0-b860-ef5f84491141"). InnerVolumeSpecName "kube-api-access-4k2ng". PluginName "kubernetes.io/projected", VolumeGIDValue "" Jul 29 19:24:37.772476 ip-10-0-131-93 kubenswrapper[2564]: I0729 19:24:37.772439 2564 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/dcd5230c-d041-4831-8280-c00a332f038d-util" (OuterVolumeSpecName: "util") pod "dcd5230c-d041-4831-8280-c00a332f038d" (UID: "dcd5230c-d041-4831-8280-c00a332f038d"). InnerVolumeSpecName "util". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Jul 29 19:24:37.775660 ip-10-0-131-93 kubenswrapper[2564]: I0729 19:24:37.775636 2564 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/83b6dd40-61a5-45e0-b860-ef5f84491141-util" (OuterVolumeSpecName: "util") pod "83b6dd40-61a5-45e0-b860-ef5f84491141" (UID: "83b6dd40-61a5-45e0-b860-ef5f84491141"). InnerVolumeSpecName "util". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Jul 29 19:24:37.868128 ip-10-0-131-93 kubenswrapper[2564]: I0729 19:24:37.868094 2564 reconciler_common.go:299] "Volume detached for volume \"bundle\" (UniqueName: \"kubernetes.io/empty-dir/dcd5230c-d041-4831-8280-c00a332f038d-bundle\") on node \"ip-10-0-131-93.ec2.internal\" DevicePath \"\"" Jul 29 19:24:37.868128 ip-10-0-131-93 kubenswrapper[2564]: I0729 19:24:37.868122 2564 reconciler_common.go:299] "Volume detached for volume \"util\" (UniqueName: \"kubernetes.io/empty-dir/83b6dd40-61a5-45e0-b860-ef5f84491141-util\") on node \"ip-10-0-131-93.ec2.internal\" DevicePath \"\"" Jul 29 19:24:37.868128 ip-10-0-131-93 kubenswrapper[2564]: I0729 19:24:37.868131 2564 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-kc5ns\" (UniqueName: \"kubernetes.io/projected/dcd5230c-d041-4831-8280-c00a332f038d-kube-api-access-kc5ns\") on node \"ip-10-0-131-93.ec2.internal\" DevicePath \"\"" Jul 29 19:24:37.868349 ip-10-0-131-93 kubenswrapper[2564]: I0729 19:24:37.868141 2564 reconciler_common.go:299] "Volume detached for volume \"util\" (UniqueName: \"kubernetes.io/empty-dir/dcd5230c-d041-4831-8280-c00a332f038d-util\") on node \"ip-10-0-131-93.ec2.internal\" DevicePath \"\"" Jul 29 19:24:37.868349 ip-10-0-131-93 kubenswrapper[2564]: I0729 19:24:37.868150 2564 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-4k2ng\" (UniqueName: \"kubernetes.io/projected/83b6dd40-61a5-45e0-b860-ef5f84491141-kube-api-access-4k2ng\") on node \"ip-10-0-131-93.ec2.internal\" DevicePath \"\"" Jul 29 19:24:37.868349 ip-10-0-131-93 kubenswrapper[2564]: I0729 19:24:37.868158 2564 reconciler_common.go:299] "Volume detached for volume \"bundle\" (UniqueName: \"kubernetes.io/empty-dir/83b6dd40-61a5-45e0-b860-ef5f84491141-bundle\") on node \"ip-10-0-131-93.ec2.internal\" DevicePath \"\"" Jul 29 19:24:38.527086 ip-10-0-131-93 kubenswrapper[2564]: I0729 19:24:38.527044 2564 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/9438e18bbad664a016701a3153a6b421e2d977be7eb0117f80bcf45ef1g46qj" event={"ID":"dcd5230c-d041-4831-8280-c00a332f038d","Type":"ContainerDied","Data":"45cd2fb09fbff6e73de365e880af9006e67342b3f9ce279fc17fc3d5b269dfd5"} Jul 29 19:24:38.527567 ip-10-0-131-93 kubenswrapper[2564]: I0729 19:24:38.527104 2564 pod_container_deletor.go:80] "Container not found in pod's containers" containerID="45cd2fb09fbff6e73de365e880af9006e67342b3f9ce279fc17fc3d5b269dfd5" Jul 29 19:24:38.527567 ip-10-0-131-93 kubenswrapper[2564]: I0729 19:24:38.527057 2564 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/9438e18bbad664a016701a3153a6b421e2d977be7eb0117f80bcf45ef1g46qj" Jul 29 19:24:38.528797 ip-10-0-131-93 kubenswrapper[2564]: I0729 19:24:38.528770 2564 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/5fc36eb1065777914bfe0ff85f9a202e2a3bafefb563d9e2994474ed736jqbs" event={"ID":"83b6dd40-61a5-45e0-b860-ef5f84491141","Type":"ContainerDied","Data":"00bc517c864e4d71e4a8ff3c6a8b9269f0c3ae3a0ae9e4b44b55cc963a18a055"} Jul 29 19:24:38.528921 ip-10-0-131-93 kubenswrapper[2564]: I0729 19:24:38.528810 2564 pod_container_deletor.go:80] "Container not found in pod's containers" containerID="00bc517c864e4d71e4a8ff3c6a8b9269f0c3ae3a0ae9e4b44b55cc963a18a055" Jul 29 19:24:38.528982 ip-10-0-131-93 kubenswrapper[2564]: I0729 19:24:38.528925 2564 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/5fc36eb1065777914bfe0ff85f9a202e2a3bafefb563d9e2994474ed736jqbs" Jul 29 19:24:38.655862 ip-10-0-131-93 kubenswrapper[2564]: I0729 19:24:38.655842 2564 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/0acee64185f523d1d1272e9af2e4d9333e0dcde792ba30e1fa9605b7596mj7k" Jul 29 19:24:38.773477 ip-10-0-131-93 kubenswrapper[2564]: I0729 19:24:38.773447 2564 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"util\" (UniqueName: \"kubernetes.io/empty-dir/32a5fe7d-49f0-44cb-af5a-e4a06e29b452-util\") pod \"32a5fe7d-49f0-44cb-af5a-e4a06e29b452\" (UID: \"32a5fe7d-49f0-44cb-af5a-e4a06e29b452\") " Jul 29 19:24:38.773667 ip-10-0-131-93 kubenswrapper[2564]: I0729 19:24:38.773513 2564 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"bundle\" (UniqueName: \"kubernetes.io/empty-dir/32a5fe7d-49f0-44cb-af5a-e4a06e29b452-bundle\") pod \"32a5fe7d-49f0-44cb-af5a-e4a06e29b452\" (UID: \"32a5fe7d-49f0-44cb-af5a-e4a06e29b452\") " Jul 29 19:24:38.773667 ip-10-0-131-93 kubenswrapper[2564]: I0729 19:24:38.773558 2564 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-6kt5c\" (UniqueName: \"kubernetes.io/projected/32a5fe7d-49f0-44cb-af5a-e4a06e29b452-kube-api-access-6kt5c\") pod \"32a5fe7d-49f0-44cb-af5a-e4a06e29b452\" (UID: \"32a5fe7d-49f0-44cb-af5a-e4a06e29b452\") " Jul 29 19:24:38.773954 ip-10-0-131-93 kubenswrapper[2564]: I0729 19:24:38.773929 2564 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/32a5fe7d-49f0-44cb-af5a-e4a06e29b452-bundle" (OuterVolumeSpecName: "bundle") pod "32a5fe7d-49f0-44cb-af5a-e4a06e29b452" (UID: "32a5fe7d-49f0-44cb-af5a-e4a06e29b452"). InnerVolumeSpecName "bundle". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Jul 29 19:24:38.775644 ip-10-0-131-93 kubenswrapper[2564]: I0729 19:24:38.775623 2564 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/32a5fe7d-49f0-44cb-af5a-e4a06e29b452-kube-api-access-6kt5c" (OuterVolumeSpecName: "kube-api-access-6kt5c") pod "32a5fe7d-49f0-44cb-af5a-e4a06e29b452" (UID: "32a5fe7d-49f0-44cb-af5a-e4a06e29b452"). InnerVolumeSpecName "kube-api-access-6kt5c". PluginName "kubernetes.io/projected", VolumeGIDValue "" Jul 29 19:24:38.781627 ip-10-0-131-93 kubenswrapper[2564]: I0729 19:24:38.781567 2564 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/32a5fe7d-49f0-44cb-af5a-e4a06e29b452-util" (OuterVolumeSpecName: "util") pod "32a5fe7d-49f0-44cb-af5a-e4a06e29b452" (UID: "32a5fe7d-49f0-44cb-af5a-e4a06e29b452"). InnerVolumeSpecName "util". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Jul 29 19:24:38.874521 ip-10-0-131-93 kubenswrapper[2564]: I0729 19:24:38.874483 2564 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-6kt5c\" (UniqueName: \"kubernetes.io/projected/32a5fe7d-49f0-44cb-af5a-e4a06e29b452-kube-api-access-6kt5c\") on node \"ip-10-0-131-93.ec2.internal\" DevicePath \"\"" Jul 29 19:24:38.874521 ip-10-0-131-93 kubenswrapper[2564]: I0729 19:24:38.874514 2564 reconciler_common.go:299] "Volume detached for volume \"util\" (UniqueName: \"kubernetes.io/empty-dir/32a5fe7d-49f0-44cb-af5a-e4a06e29b452-util\") on node \"ip-10-0-131-93.ec2.internal\" DevicePath \"\"" Jul 29 19:24:38.874521 ip-10-0-131-93 kubenswrapper[2564]: I0729 19:24:38.874524 2564 reconciler_common.go:299] "Volume detached for volume \"bundle\" (UniqueName: \"kubernetes.io/empty-dir/32a5fe7d-49f0-44cb-af5a-e4a06e29b452-bundle\") on node \"ip-10-0-131-93.ec2.internal\" DevicePath \"\"" Jul 29 19:24:39.534737 ip-10-0-131-93 kubenswrapper[2564]: I0729 19:24:39.534696 2564 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/0acee64185f523d1d1272e9af2e4d9333e0dcde792ba30e1fa9605b7596mj7k" event={"ID":"32a5fe7d-49f0-44cb-af5a-e4a06e29b452","Type":"ContainerDied","Data":"d864c20b7c6825c44d0dbd833f9cbf56d28438a6cd3ff98ba5a0135ebacc80d9"} Jul 29 19:24:39.534737 ip-10-0-131-93 kubenswrapper[2564]: I0729 19:24:39.534739 2564 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/0acee64185f523d1d1272e9af2e4d9333e0dcde792ba30e1fa9605b7596mj7k" Jul 29 19:24:39.535163 ip-10-0-131-93 kubenswrapper[2564]: I0729 19:24:39.534754 2564 pod_container_deletor.go:80] "Container not found in pod's containers" containerID="d864c20b7c6825c44d0dbd833f9cbf56d28438a6cd3ff98ba5a0135ebacc80d9" Jul 29 19:24:41.848175 ip-10-0-131-93 kubenswrapper[2564]: I0729 19:24:41.848141 2564 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-444jx_faa8c546-c75b-4e10-93b4-661aa2b5f53e/konnectivity-agent/0.log" limit=1024 Jul 29 19:24:41.952619 ip-10-0-131-93 kubenswrapper[2564]: I0729 19:24:41.952587 2564 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-444jx_faa8c546-c75b-4e10-93b4-661aa2b5f53e/konnectivity-agent/0.log" limit=1024 Jul 29 19:24:45.249637 ip-10-0-131-93 kubenswrapper[2564]: I0729 19:24:45.249590 2564 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-444jx_faa8c546-c75b-4e10-93b4-661aa2b5f53e/konnectivity-agent/0.log" limit=1024 Jul 29 19:24:49.616791 ip-10-0-131-93 kubenswrapper[2564]: I0729 19:24:49.616754 2564 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-444jx_faa8c546-c75b-4e10-93b4-661aa2b5f53e/konnectivity-agent/0.log" limit=1024 Jul 29 19:24:50.729169 ip-10-0-131-93 kubenswrapper[2564]: I0729 19:24:50.729133 2564 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-444jx_faa8c546-c75b-4e10-93b4-661aa2b5f53e/konnectivity-agent/0.log" limit=1024 Jul 29 19:24:51.569100 ip-10-0-131-93 kubenswrapper[2564]: I0729 19:24:51.569057 2564 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-444jx_faa8c546-c75b-4e10-93b4-661aa2b5f53e/konnectivity-agent/0.log" limit=1024 Jul 29 19:24:52.805570 ip-10-0-131-93 kubenswrapper[2564]: I0729 19:24:52.805528 2564 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-444jx_faa8c546-c75b-4e10-93b4-661aa2b5f53e/konnectivity-agent/0.log" limit=1024 Jul 29 19:24:53.431418 ip-10-0-131-93 kubenswrapper[2564]: I0729 19:24:53.431363 2564 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-444jx_faa8c546-c75b-4e10-93b4-661aa2b5f53e/konnectivity-agent/0.log" limit=1024 Jul 29 19:24:53.611690 ip-10-0-131-93 kubenswrapper[2564]: I0729 19:24:53.611641 2564 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-444jx_faa8c546-c75b-4e10-93b4-661aa2b5f53e/konnectivity-agent/0.log" limit=1024 Jul 29 19:24:53.884795 ip-10-0-131-93 kubenswrapper[2564]: I0729 19:24:53.884764 2564 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-444jx_faa8c546-c75b-4e10-93b4-661aa2b5f53e/konnectivity-agent/0.log" limit=1024 Jul 29 19:24:54.724006 ip-10-0-131-93 kubenswrapper[2564]: I0729 19:24:54.723974 2564 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-444jx_faa8c546-c75b-4e10-93b4-661aa2b5f53e/konnectivity-agent/0.log" limit=1024 Jul 29 19:24:55.495932 ip-10-0-131-93 kubenswrapper[2564]: I0729 19:24:55.495897 2564 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-444jx_faa8c546-c75b-4e10-93b4-661aa2b5f53e/konnectivity-agent/0.log" limit=1024 Jul 29 19:24:55.672994 ip-10-0-131-93 kubenswrapper[2564]: I0729 19:24:55.672964 2564 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-444jx_faa8c546-c75b-4e10-93b4-661aa2b5f53e/konnectivity-agent/0.log" limit=1024 Jul 29 19:24:57.260771 ip-10-0-131-93 kubenswrapper[2564]: I0729 19:24:57.260738 2564 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-444jx_faa8c546-c75b-4e10-93b4-661aa2b5f53e/konnectivity-agent/0.log" limit=1024 Jul 29 19:24:57.352227 ip-10-0-131-93 kubenswrapper[2564]: I0729 19:24:57.352193 2564 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-444jx_faa8c546-c75b-4e10-93b4-661aa2b5f53e/konnectivity-agent/0.log" limit=1024 Jul 29 19:24:57.452886 ip-10-0-131-93 kubenswrapper[2564]: I0729 19:24:57.452858 2564 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-444jx_faa8c546-c75b-4e10-93b4-661aa2b5f53e/konnectivity-agent/0.log" limit=1024 Jul 29 19:24:57.554929 ip-10-0-131-93 kubenswrapper[2564]: I0729 19:24:57.554843 2564 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-444jx_faa8c546-c75b-4e10-93b4-661aa2b5f53e/konnectivity-agent/0.log" limit=1024 Jul 29 19:24:58.844735 ip-10-0-131-93 kubenswrapper[2564]: I0729 19:24:58.844703 2564 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-444jx_faa8c546-c75b-4e10-93b4-661aa2b5f53e/konnectivity-agent/0.log" limit=1024 Jul 29 19:24:59.122424 ip-10-0-131-93 kubenswrapper[2564]: I0729 19:24:59.122311 2564 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-444jx_faa8c546-c75b-4e10-93b4-661aa2b5f53e/konnectivity-agent/0.log" limit=1024 Jul 29 19:25:01.496276 ip-10-0-131-93 kubenswrapper[2564]: I0729 19:25:01.496240 2564 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-444jx_faa8c546-c75b-4e10-93b4-661aa2b5f53e/konnectivity-agent/0.log" limit=1024 Jul 29 19:25:01.568503 ip-10-0-131-93 kubenswrapper[2564]: I0729 19:25:01.568467 2564 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-444jx_faa8c546-c75b-4e10-93b4-661aa2b5f53e/konnectivity-agent/0.log" limit=1024 Jul 29 19:25:02.659654 ip-10-0-131-93 kubenswrapper[2564]: I0729 19:25:02.659625 2564 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-444jx_faa8c546-c75b-4e10-93b4-661aa2b5f53e/konnectivity-agent/0.log" limit=1024 Jul 29 19:25:02.758493 ip-10-0-131-93 kubenswrapper[2564]: I0729 19:25:02.758455 2564 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-444jx_faa8c546-c75b-4e10-93b4-661aa2b5f53e/konnectivity-agent/0.log" limit=1024 Jul 29 19:25:09.436504 ip-10-0-131-93 kubenswrapper[2564]: I0729 19:25:09.436467 2564 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-444jx_faa8c546-c75b-4e10-93b4-661aa2b5f53e/konnectivity-agent/0.log" limit=1024 Jul 29 19:25:09.627088 ip-10-0-131-93 kubenswrapper[2564]: I0729 19:25:09.627054 2564 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-444jx_faa8c546-c75b-4e10-93b4-661aa2b5f53e/konnectivity-agent/0.log" limit=1024 Jul 29 19:25:14.003598 ip-10-0-131-93 kubenswrapper[2564]: I0729 19:25:14.003565 2564 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-444jx_faa8c546-c75b-4e10-93b4-661aa2b5f53e/konnectivity-agent/0.log" limit=1024 Jul 29 19:25:14.119417 ip-10-0-131-93 kubenswrapper[2564]: I0729 19:25:14.119364 2564 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-444jx_faa8c546-c75b-4e10-93b4-661aa2b5f53e/konnectivity-agent/0.log" limit=1024 Jul 29 19:25:17.840585 ip-10-0-131-93 kubenswrapper[2564]: I0729 19:25:17.840548 2564 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-444jx_faa8c546-c75b-4e10-93b4-661aa2b5f53e/konnectivity-agent/0.log" limit=1024 Jul 29 19:25:17.917771 ip-10-0-131-93 kubenswrapper[2564]: I0729 19:25:17.917735 2564 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-444jx_faa8c546-c75b-4e10-93b4-661aa2b5f53e/konnectivity-agent/0.log" limit=1024 Jul 29 19:25:18.511820 ip-10-0-131-93 kubenswrapper[2564]: I0729 19:25:18.511786 2564 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-444jx_faa8c546-c75b-4e10-93b4-661aa2b5f53e/konnectivity-agent/0.log" limit=1024 Jul 29 19:25:18.610550 ip-10-0-131-93 kubenswrapper[2564]: I0729 19:25:18.610513 2564 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-444jx_faa8c546-c75b-4e10-93b4-661aa2b5f53e/konnectivity-agent/0.log" limit=1024 Jul 29 19:25:21.950973 ip-10-0-131-93 kubenswrapper[2564]: I0729 19:25:21.950944 2564 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-444jx_faa8c546-c75b-4e10-93b4-661aa2b5f53e/konnectivity-agent/0.log" limit=1024 Jul 29 19:25:23.300255 ip-10-0-131-93 kubenswrapper[2564]: I0729 19:25:23.300224 2564 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-444jx_faa8c546-c75b-4e10-93b4-661aa2b5f53e/konnectivity-agent/0.log" limit=1024 Jul 29 19:25:23.396063 ip-10-0-131-93 kubenswrapper[2564]: I0729 19:25:23.396032 2564 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-444jx_faa8c546-c75b-4e10-93b4-661aa2b5f53e/konnectivity-agent/0.log" limit=1024 Jul 29 19:25:28.108232 ip-10-0-131-93 kubenswrapper[2564]: I0729 19:25:28.108196 2564 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-444jx_faa8c546-c75b-4e10-93b4-661aa2b5f53e/konnectivity-agent/0.log" limit=1024 Jul 29 19:25:28.208747 ip-10-0-131-93 kubenswrapper[2564]: I0729 19:25:28.208713 2564 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-444jx_faa8c546-c75b-4e10-93b4-661aa2b5f53e/konnectivity-agent/0.log" limit=1024 Jul 29 19:25:32.880447 ip-10-0-131-93 kubenswrapper[2564]: I0729 19:25:32.880416 2564 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-444jx_faa8c546-c75b-4e10-93b4-661aa2b5f53e/konnectivity-agent/0.log" limit=1024 Jul 29 19:25:32.983521 ip-10-0-131-93 kubenswrapper[2564]: I0729 19:25:32.983482 2564 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-444jx_faa8c546-c75b-4e10-93b4-661aa2b5f53e/konnectivity-agent/0.log" limit=1024 Jul 29 19:25:33.304970 ip-10-0-131-93 kubenswrapper[2564]: I0729 19:25:33.304937 2564 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-444jx_faa8c546-c75b-4e10-93b4-661aa2b5f53e/konnectivity-agent/0.log" limit=1024 Jul 29 19:25:33.407460 ip-10-0-131-93 kubenswrapper[2564]: I0729 19:25:33.407427 2564 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-444jx_faa8c546-c75b-4e10-93b4-661aa2b5f53e/konnectivity-agent/0.log" limit=1024 Jul 29 19:25:40.506098 ip-10-0-131-93 kubenswrapper[2564]: I0729 19:25:40.506066 2564 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-444jx_faa8c546-c75b-4e10-93b4-661aa2b5f53e/konnectivity-agent/0.log" limit=1024 Jul 29 19:25:40.855911 ip-10-0-131-93 kubenswrapper[2564]: I0729 19:25:40.855826 2564 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-444jx_faa8c546-c75b-4e10-93b4-661aa2b5f53e/konnectivity-agent/0.log" limit=1024 Jul 29 19:25:41.848887 ip-10-0-131-93 kubenswrapper[2564]: I0729 19:25:41.848852 2564 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-444jx_faa8c546-c75b-4e10-93b4-661aa2b5f53e/konnectivity-agent/0.log" limit=1024 Jul 29 19:25:41.929035 ip-10-0-131-93 kubenswrapper[2564]: I0729 19:25:41.929007 2564 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-444jx_faa8c546-c75b-4e10-93b4-661aa2b5f53e/konnectivity-agent/0.log" limit=1024 Jul 29 19:25:45.238717 ip-10-0-131-93 kubenswrapper[2564]: I0729 19:25:45.238680 2564 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-444jx_faa8c546-c75b-4e10-93b4-661aa2b5f53e/konnectivity-agent/0.log" limit=1024 Jul 29 19:25:45.343348 ip-10-0-131-93 kubenswrapper[2564]: I0729 19:25:45.343314 2564 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-444jx_faa8c546-c75b-4e10-93b4-661aa2b5f53e/konnectivity-agent/0.log" limit=1024 Jul 29 19:25:45.444415 ip-10-0-131-93 kubenswrapper[2564]: I0729 19:25:45.444363 2564 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-444jx_faa8c546-c75b-4e10-93b4-661aa2b5f53e/konnectivity-agent/0.log" limit=1024 Jul 29 19:25:46.065599 ip-10-0-131-93 kubenswrapper[2564]: I0729 19:25:46.065569 2564 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-444jx_faa8c546-c75b-4e10-93b4-661aa2b5f53e/konnectivity-agent/0.log" limit=1024 Jul 29 19:25:46.156819 ip-10-0-131-93 kubenswrapper[2564]: I0729 19:25:46.156788 2564 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-444jx_faa8c546-c75b-4e10-93b4-661aa2b5f53e/konnectivity-agent/0.log" limit=1024 Jul 29 19:25:53.151500 ip-10-0-131-93 kubenswrapper[2564]: I0729 19:25:53.151466 2564 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-444jx_faa8c546-c75b-4e10-93b4-661aa2b5f53e/konnectivity-agent/0.log" limit=1024 Jul 29 19:25:53.250980 ip-10-0-131-93 kubenswrapper[2564]: I0729 19:25:53.250944 2564 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-444jx_faa8c546-c75b-4e10-93b4-661aa2b5f53e/konnectivity-agent/0.log" limit=1024 Jul 29 19:25:53.349581 ip-10-0-131-93 kubenswrapper[2564]: I0729 19:25:53.349549 2564 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-444jx_faa8c546-c75b-4e10-93b4-661aa2b5f53e/konnectivity-agent/0.log" limit=1024 Jul 29 19:25:53.454913 ip-10-0-131-93 kubenswrapper[2564]: I0729 19:25:53.454874 2564 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-444jx_faa8c546-c75b-4e10-93b4-661aa2b5f53e/konnectivity-agent/0.log" limit=1024 Jul 29 19:26:06.495047 ip-10-0-131-93 kubenswrapper[2564]: I0729 19:26:06.494962 2564 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-444jx_faa8c546-c75b-4e10-93b4-661aa2b5f53e/konnectivity-agent/0.log" limit=1024 Jul 29 19:26:06.594940 ip-10-0-131-93 kubenswrapper[2564]: I0729 19:26:06.594888 2564 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-444jx_faa8c546-c75b-4e10-93b4-661aa2b5f53e/konnectivity-agent/0.log" limit=1024 Jul 29 19:26:06.982864 ip-10-0-131-93 kubenswrapper[2564]: I0729 19:26:06.982817 2564 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-444jx_faa8c546-c75b-4e10-93b4-661aa2b5f53e/konnectivity-agent/0.log" limit=1024 Jul 29 19:26:19.452083 ip-10-0-131-93 kubenswrapper[2564]: I0729 19:26:19.452034 2564 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-444jx_faa8c546-c75b-4e10-93b4-661aa2b5f53e/konnectivity-agent/0.log" limit=1024 Jul 29 19:26:19.715394 ip-10-0-131-93 kubenswrapper[2564]: I0729 19:26:19.715288 2564 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-444jx_faa8c546-c75b-4e10-93b4-661aa2b5f53e/konnectivity-agent/0.log" limit=1024 Jul 29 19:26:26.761095 ip-10-0-131-93 kubenswrapper[2564]: I0729 19:26:26.761060 2564 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-444jx_faa8c546-c75b-4e10-93b4-661aa2b5f53e/konnectivity-agent/0.log" limit=1024 Jul 29 19:26:26.791278 ip-10-0-131-93 kubenswrapper[2564]: I0729 19:26:26.791250 2564 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["opendatahub/maas-controller-84d5849b5d-fl8d4"] Jul 29 19:26:26.791605 ip-10-0-131-93 kubenswrapper[2564]: I0729 19:26:26.791589 2564 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="9e24b906-6ade-47ef-818c-28a1c450e0be" containerName="pull" Jul 29 19:26:26.791693 ip-10-0-131-93 kubenswrapper[2564]: I0729 19:26:26.791608 2564 state_mem.go:107] "Deleted CPUSet assignment" podUID="9e24b906-6ade-47ef-818c-28a1c450e0be" containerName="pull" Jul 29 19:26:26.791693 ip-10-0-131-93 kubenswrapper[2564]: I0729 19:26:26.791621 2564 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="83b6dd40-61a5-45e0-b860-ef5f84491141" containerName="extract" Jul 29 19:26:26.791693 ip-10-0-131-93 kubenswrapper[2564]: I0729 19:26:26.791628 2564 state_mem.go:107] "Deleted CPUSet assignment" podUID="83b6dd40-61a5-45e0-b860-ef5f84491141" containerName="extract" Jul 29 19:26:26.791693 ip-10-0-131-93 kubenswrapper[2564]: I0729 19:26:26.791641 2564 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="83b6dd40-61a5-45e0-b860-ef5f84491141" containerName="util" Jul 29 19:26:26.791693 ip-10-0-131-93 kubenswrapper[2564]: I0729 19:26:26.791649 2564 state_mem.go:107] "Deleted CPUSet assignment" podUID="83b6dd40-61a5-45e0-b860-ef5f84491141" containerName="util" Jul 29 19:26:26.791693 ip-10-0-131-93 kubenswrapper[2564]: I0729 19:26:26.791661 2564 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="9e24b906-6ade-47ef-818c-28a1c450e0be" containerName="extract" Jul 29 19:26:26.791693 ip-10-0-131-93 kubenswrapper[2564]: I0729 19:26:26.791669 2564 state_mem.go:107] "Deleted CPUSet assignment" podUID="9e24b906-6ade-47ef-818c-28a1c450e0be" containerName="extract" Jul 29 19:26:26.791693 ip-10-0-131-93 kubenswrapper[2564]: I0729 19:26:26.791679 2564 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="32a5fe7d-49f0-44cb-af5a-e4a06e29b452" containerName="pull" Jul 29 19:26:26.791693 ip-10-0-131-93 kubenswrapper[2564]: I0729 19:26:26.791687 2564 state_mem.go:107] "Deleted CPUSet assignment" podUID="32a5fe7d-49f0-44cb-af5a-e4a06e29b452" containerName="pull" Jul 29 19:26:26.792107 ip-10-0-131-93 kubenswrapper[2564]: I0729 19:26:26.791707 2564 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="dcd5230c-d041-4831-8280-c00a332f038d" containerName="util" Jul 29 19:26:26.792107 ip-10-0-131-93 kubenswrapper[2564]: I0729 19:26:26.791715 2564 state_mem.go:107] "Deleted CPUSet assignment" podUID="dcd5230c-d041-4831-8280-c00a332f038d" containerName="util" Jul 29 19:26:26.792107 ip-10-0-131-93 kubenswrapper[2564]: I0729 19:26:26.791725 2564 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="32a5fe7d-49f0-44cb-af5a-e4a06e29b452" containerName="extract" Jul 29 19:26:26.792107 ip-10-0-131-93 kubenswrapper[2564]: I0729 19:26:26.791733 2564 state_mem.go:107] "Deleted CPUSet assignment" podUID="32a5fe7d-49f0-44cb-af5a-e4a06e29b452" containerName="extract" Jul 29 19:26:26.792107 ip-10-0-131-93 kubenswrapper[2564]: I0729 19:26:26.791745 2564 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="83b6dd40-61a5-45e0-b860-ef5f84491141" containerName="pull" Jul 29 19:26:26.792107 ip-10-0-131-93 kubenswrapper[2564]: I0729 19:26:26.791752 2564 state_mem.go:107] "Deleted CPUSet assignment" podUID="83b6dd40-61a5-45e0-b860-ef5f84491141" containerName="pull" Jul 29 19:26:26.792107 ip-10-0-131-93 kubenswrapper[2564]: I0729 19:26:26.791762 2564 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="dcd5230c-d041-4831-8280-c00a332f038d" containerName="extract" Jul 29 19:26:26.792107 ip-10-0-131-93 kubenswrapper[2564]: I0729 19:26:26.791771 2564 state_mem.go:107] "Deleted CPUSet assignment" podUID="dcd5230c-d041-4831-8280-c00a332f038d" containerName="extract" Jul 29 19:26:26.792107 ip-10-0-131-93 kubenswrapper[2564]: I0729 19:26:26.791782 2564 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="dcd5230c-d041-4831-8280-c00a332f038d" containerName="pull" Jul 29 19:26:26.792107 ip-10-0-131-93 kubenswrapper[2564]: I0729 19:26:26.791790 2564 state_mem.go:107] "Deleted CPUSet assignment" podUID="dcd5230c-d041-4831-8280-c00a332f038d" containerName="pull" Jul 29 19:26:26.792107 ip-10-0-131-93 kubenswrapper[2564]: I0729 19:26:26.791801 2564 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="32a5fe7d-49f0-44cb-af5a-e4a06e29b452" containerName="util" Jul 29 19:26:26.792107 ip-10-0-131-93 kubenswrapper[2564]: I0729 19:26:26.791809 2564 state_mem.go:107] "Deleted CPUSet assignment" podUID="32a5fe7d-49f0-44cb-af5a-e4a06e29b452" containerName="util" Jul 29 19:26:26.792107 ip-10-0-131-93 kubenswrapper[2564]: I0729 19:26:26.791819 2564 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="9e24b906-6ade-47ef-818c-28a1c450e0be" containerName="util" Jul 29 19:26:26.792107 ip-10-0-131-93 kubenswrapper[2564]: I0729 19:26:26.791827 2564 state_mem.go:107] "Deleted CPUSet assignment" podUID="9e24b906-6ade-47ef-818c-28a1c450e0be" containerName="util" Jul 29 19:26:26.792107 ip-10-0-131-93 kubenswrapper[2564]: I0729 19:26:26.791902 2564 memory_manager.go:368] "RemoveStaleState removing state" podUID="9e24b906-6ade-47ef-818c-28a1c450e0be" containerName="extract" Jul 29 19:26:26.792107 ip-10-0-131-93 kubenswrapper[2564]: I0729 19:26:26.791915 2564 memory_manager.go:368] "RemoveStaleState removing state" podUID="32a5fe7d-49f0-44cb-af5a-e4a06e29b452" containerName="extract" Jul 29 19:26:26.792107 ip-10-0-131-93 kubenswrapper[2564]: I0729 19:26:26.791927 2564 memory_manager.go:368] "RemoveStaleState removing state" podUID="dcd5230c-d041-4831-8280-c00a332f038d" containerName="extract" Jul 29 19:26:26.792107 ip-10-0-131-93 kubenswrapper[2564]: I0729 19:26:26.791936 2564 memory_manager.go:368] "RemoveStaleState removing state" podUID="83b6dd40-61a5-45e0-b860-ef5f84491141" containerName="extract" Jul 29 19:26:26.794365 ip-10-0-131-93 kubenswrapper[2564]: I0729 19:26:26.794348 2564 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="opendatahub/maas-controller-84d5849b5d-fl8d4" Jul 29 19:26:26.797823 ip-10-0-131-93 kubenswrapper[2564]: I0729 19:26:26.797800 2564 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"opendatahub\"/\"maas-controller-dockercfg-d6zg6\"" Jul 29 19:26:26.797823 ip-10-0-131-93 kubenswrapper[2564]: I0729 19:26:26.797810 2564 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"opendatahub\"/\"maas-parameters\"" Jul 29 19:26:26.797983 ip-10-0-131-93 kubenswrapper[2564]: I0729 19:26:26.797812 2564 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"opendatahub\"/\"maas-controller-webhook-cert\"" Jul 29 19:26:26.820001 ip-10-0-131-93 kubenswrapper[2564]: I0729 19:26:26.819976 2564 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["opendatahub/maas-controller-84d5849b5d-fl8d4"] Jul 29 19:26:26.822205 ip-10-0-131-93 kubenswrapper[2564]: I0729 19:26:26.822185 2564 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-sbplr\" (UniqueName: \"kubernetes.io/projected/dde5f313-e028-44b2-8adb-4689b8675327-kube-api-access-sbplr\") pod \"maas-controller-84d5849b5d-fl8d4\" (UID: \"dde5f313-e028-44b2-8adb-4689b8675327\") " pod="opendatahub/maas-controller-84d5849b5d-fl8d4" Jul 29 19:26:26.822297 ip-10-0-131-93 kubenswrapper[2564]: I0729 19:26:26.822216 2564 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"webhook-cert\" (UniqueName: \"kubernetes.io/secret/dde5f313-e028-44b2-8adb-4689b8675327-webhook-cert\") pod \"maas-controller-84d5849b5d-fl8d4\" (UID: \"dde5f313-e028-44b2-8adb-4689b8675327\") " pod="opendatahub/maas-controller-84d5849b5d-fl8d4" Jul 29 19:26:26.857115 ip-10-0-131-93 kubenswrapper[2564]: I0729 19:26:26.857076 2564 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-444jx_faa8c546-c75b-4e10-93b4-661aa2b5f53e/konnectivity-agent/0.log" limit=1024 Jul 29 19:26:26.919487 ip-10-0-131-93 kubenswrapper[2564]: I0729 19:26:26.919444 2564 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-444jx_faa8c546-c75b-4e10-93b4-661aa2b5f53e/konnectivity-agent/0.log" limit=1024 Jul 29 19:26:26.923570 ip-10-0-131-93 kubenswrapper[2564]: I0729 19:26:26.923549 2564 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-sbplr\" (UniqueName: \"kubernetes.io/projected/dde5f313-e028-44b2-8adb-4689b8675327-kube-api-access-sbplr\") pod \"maas-controller-84d5849b5d-fl8d4\" (UID: \"dde5f313-e028-44b2-8adb-4689b8675327\") " pod="opendatahub/maas-controller-84d5849b5d-fl8d4" Jul 29 19:26:26.923640 ip-10-0-131-93 kubenswrapper[2564]: I0729 19:26:26.923588 2564 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"webhook-cert\" (UniqueName: \"kubernetes.io/secret/dde5f313-e028-44b2-8adb-4689b8675327-webhook-cert\") pod \"maas-controller-84d5849b5d-fl8d4\" (UID: \"dde5f313-e028-44b2-8adb-4689b8675327\") " pod="opendatahub/maas-controller-84d5849b5d-fl8d4" Jul 29 19:26:26.926035 ip-10-0-131-93 kubenswrapper[2564]: I0729 19:26:26.926012 2564 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"webhook-cert\" (UniqueName: \"kubernetes.io/secret/dde5f313-e028-44b2-8adb-4689b8675327-webhook-cert\") pod \"maas-controller-84d5849b5d-fl8d4\" (UID: \"dde5f313-e028-44b2-8adb-4689b8675327\") " pod="opendatahub/maas-controller-84d5849b5d-fl8d4" Jul 29 19:26:26.931167 ip-10-0-131-93 kubenswrapper[2564]: I0729 19:26:26.931143 2564 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-sbplr\" (UniqueName: \"kubernetes.io/projected/dde5f313-e028-44b2-8adb-4689b8675327-kube-api-access-sbplr\") pod \"maas-controller-84d5849b5d-fl8d4\" (UID: \"dde5f313-e028-44b2-8adb-4689b8675327\") " pod="opendatahub/maas-controller-84d5849b5d-fl8d4" Jul 29 19:26:26.982781 ip-10-0-131-93 kubenswrapper[2564]: I0729 19:26:26.982750 2564 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["opendatahub/maas-controller-865c54c777-z6f87"] Jul 29 19:26:26.985212 ip-10-0-131-93 kubenswrapper[2564]: I0729 19:26:26.985197 2564 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="opendatahub/maas-controller-865c54c777-z6f87" Jul 29 19:26:26.998029 ip-10-0-131-93 kubenswrapper[2564]: I0729 19:26:26.998004 2564 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["opendatahub/maas-controller-865c54c777-z6f87"] Jul 29 19:26:27.024993 ip-10-0-131-93 kubenswrapper[2564]: I0729 19:26:27.024909 2564 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-mqrnt\" (UniqueName: \"kubernetes.io/projected/96579673-c9a2-42bd-8159-7329efcf6772-kube-api-access-mqrnt\") pod \"maas-controller-865c54c777-z6f87\" (UID: \"96579673-c9a2-42bd-8159-7329efcf6772\") " pod="opendatahub/maas-controller-865c54c777-z6f87" Jul 29 19:26:27.024993 ip-10-0-131-93 kubenswrapper[2564]: I0729 19:26:27.024956 2564 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"webhook-cert\" (UniqueName: \"kubernetes.io/secret/96579673-c9a2-42bd-8159-7329efcf6772-webhook-cert\") pod \"maas-controller-865c54c777-z6f87\" (UID: \"96579673-c9a2-42bd-8159-7329efcf6772\") " pod="opendatahub/maas-controller-865c54c777-z6f87" Jul 29 19:26:27.104813 ip-10-0-131-93 kubenswrapper[2564]: I0729 19:26:27.104783 2564 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="opendatahub/maas-controller-84d5849b5d-fl8d4" Jul 29 19:26:27.126435 ip-10-0-131-93 kubenswrapper[2564]: I0729 19:26:27.126407 2564 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"webhook-cert\" (UniqueName: \"kubernetes.io/secret/96579673-c9a2-42bd-8159-7329efcf6772-webhook-cert\") pod \"maas-controller-865c54c777-z6f87\" (UID: \"96579673-c9a2-42bd-8159-7329efcf6772\") " pod="opendatahub/maas-controller-865c54c777-z6f87" Jul 29 19:26:27.126576 ip-10-0-131-93 kubenswrapper[2564]: I0729 19:26:27.126511 2564 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-mqrnt\" (UniqueName: \"kubernetes.io/projected/96579673-c9a2-42bd-8159-7329efcf6772-kube-api-access-mqrnt\") pod \"maas-controller-865c54c777-z6f87\" (UID: \"96579673-c9a2-42bd-8159-7329efcf6772\") " pod="opendatahub/maas-controller-865c54c777-z6f87" Jul 29 19:26:27.128976 ip-10-0-131-93 kubenswrapper[2564]: I0729 19:26:27.128948 2564 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"webhook-cert\" (UniqueName: \"kubernetes.io/secret/96579673-c9a2-42bd-8159-7329efcf6772-webhook-cert\") pod \"maas-controller-865c54c777-z6f87\" (UID: \"96579673-c9a2-42bd-8159-7329efcf6772\") " pod="opendatahub/maas-controller-865c54c777-z6f87" Jul 29 19:26:27.135558 ip-10-0-131-93 kubenswrapper[2564]: I0729 19:26:27.135537 2564 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-mqrnt\" (UniqueName: \"kubernetes.io/projected/96579673-c9a2-42bd-8159-7329efcf6772-kube-api-access-mqrnt\") pod \"maas-controller-865c54c777-z6f87\" (UID: \"96579673-c9a2-42bd-8159-7329efcf6772\") " pod="opendatahub/maas-controller-865c54c777-z6f87" Jul 29 19:26:27.232935 ip-10-0-131-93 kubenswrapper[2564]: I0729 19:26:27.232914 2564 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["opendatahub/maas-controller-84d5849b5d-fl8d4"] Jul 29 19:26:27.235523 ip-10-0-131-93 kubenswrapper[2564]: W0729 19:26:27.235494 2564 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-poddde5f313_e028_44b2_8adb_4689b8675327.slice/crio-7676886692fdd2472b3aab5c67b426cd8c701bf544c157e213b58fc61dd1f409 WatchSource:0}: Error finding container 7676886692fdd2472b3aab5c67b426cd8c701bf544c157e213b58fc61dd1f409: Status 404 returned error can't find the container with id 7676886692fdd2472b3aab5c67b426cd8c701bf544c157e213b58fc61dd1f409 Jul 29 19:26:27.295745 ip-10-0-131-93 kubenswrapper[2564]: I0729 19:26:27.295652 2564 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="opendatahub/maas-controller-865c54c777-z6f87" Jul 29 19:26:27.423622 ip-10-0-131-93 kubenswrapper[2564]: I0729 19:26:27.423598 2564 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["opendatahub/maas-controller-865c54c777-z6f87"] Jul 29 19:26:27.425200 ip-10-0-131-93 kubenswrapper[2564]: W0729 19:26:27.425179 2564 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod96579673_c9a2_42bd_8159_7329efcf6772.slice/crio-ed87458591c61397556bbda73186e280fd985a7760dc00c05cdd0f6f447186b4 WatchSource:0}: Error finding container ed87458591c61397556bbda73186e280fd985a7760dc00c05cdd0f6f447186b4: Status 404 returned error can't find the container with id ed87458591c61397556bbda73186e280fd985a7760dc00c05cdd0f6f447186b4 Jul 29 19:26:28.028429 ip-10-0-131-93 kubenswrapper[2564]: I0729 19:26:28.028389 2564 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="opendatahub/maas-controller-865c54c777-z6f87" event={"ID":"96579673-c9a2-42bd-8159-7329efcf6772","Type":"ContainerStarted","Data":"ed87458591c61397556bbda73186e280fd985a7760dc00c05cdd0f6f447186b4"} Jul 29 19:26:28.029405 ip-10-0-131-93 kubenswrapper[2564]: I0729 19:26:28.029367 2564 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="opendatahub/maas-controller-84d5849b5d-fl8d4" event={"ID":"dde5f313-e028-44b2-8adb-4689b8675327","Type":"ContainerStarted","Data":"7676886692fdd2472b3aab5c67b426cd8c701bf544c157e213b58fc61dd1f409"} Jul 29 19:26:31.489649 ip-10-0-131-93 kubenswrapper[2564]: I0729 19:26:31.489617 2564 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-444jx_faa8c546-c75b-4e10-93b4-661aa2b5f53e/konnectivity-agent/0.log" limit=1024 Jul 29 19:26:31.588765 ip-10-0-131-93 kubenswrapper[2564]: I0729 19:26:31.588668 2564 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-444jx_faa8c546-c75b-4e10-93b4-661aa2b5f53e/konnectivity-agent/0.log" limit=1024 Jul 29 19:26:32.052013 ip-10-0-131-93 kubenswrapper[2564]: I0729 19:26:32.051977 2564 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="opendatahub/maas-controller-865c54c777-z6f87" event={"ID":"96579673-c9a2-42bd-8159-7329efcf6772","Type":"ContainerStarted","Data":"2960ac3c6a20a036f8f72de1b9849955427d08c8bed825b754d60c7b9f19aec8"} Jul 29 19:26:32.052202 ip-10-0-131-93 kubenswrapper[2564]: I0729 19:26:32.052102 2564 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="opendatahub/maas-controller-865c54c777-z6f87" Jul 29 19:26:32.053495 ip-10-0-131-93 kubenswrapper[2564]: I0729 19:26:32.053471 2564 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="opendatahub/maas-controller-84d5849b5d-fl8d4" event={"ID":"dde5f313-e028-44b2-8adb-4689b8675327","Type":"ContainerStarted","Data":"af28e819ebf6037bec40c5a990b4ff3c470abd2a5a6c255529ed700052ab4c31"} Jul 29 19:26:32.053613 ip-10-0-131-93 kubenswrapper[2564]: I0729 19:26:32.053603 2564 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="opendatahub/maas-controller-84d5849b5d-fl8d4" Jul 29 19:26:32.071151 ip-10-0-131-93 kubenswrapper[2564]: I0729 19:26:32.071107 2564 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="opendatahub/maas-controller-865c54c777-z6f87" podStartSLOduration=2.177056597 podStartE2EDuration="6.071090485s" totalImagesPullingTime="3.894033888s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-29 19:26:26 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-29 19:26:32.070162205 +0000 UTC m=+567.794221896" watchObservedRunningTime="2026-07-29 19:26:32.071090485 +0000 UTC m=+567.795150178" Jul 29 19:26:32.093193 ip-10-0-131-93 kubenswrapper[2564]: I0729 19:26:32.093130 2564 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="opendatahub/maas-controller-84d5849b5d-fl8d4" podStartSLOduration=2.016053708 podStartE2EDuration="6.093108747s" totalImagesPullingTime="4.077055039s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-29 19:26:26 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-29 19:26:32.090345484 +0000 UTC m=+567.814405167" watchObservedRunningTime="2026-07-29 19:26:32.093108747 +0000 UTC m=+567.817168443" Jul 29 19:26:43.060667 ip-10-0-131-93 kubenswrapper[2564]: I0729 19:26:43.060635 2564 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="opendatahub/maas-controller-84d5849b5d-fl8d4" Jul 29 19:26:43.061025 ip-10-0-131-93 kubenswrapper[2564]: I0729 19:26:43.060791 2564 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="opendatahub/maas-controller-865c54c777-z6f87" Jul 29 19:26:43.125779 ip-10-0-131-93 kubenswrapper[2564]: I0729 19:26:43.125748 2564 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["opendatahub/maas-controller-84d5849b5d-fl8d4"] Jul 29 19:26:43.126010 ip-10-0-131-93 kubenswrapper[2564]: I0729 19:26:43.125976 2564 kuberuntime_container.go:873] "Killing container with a grace period" pod="opendatahub/maas-controller-84d5849b5d-fl8d4" podUID="dde5f313-e028-44b2-8adb-4689b8675327" containerName="manager" containerID="cri-o://af28e819ebf6037bec40c5a990b4ff3c470abd2a5a6c255529ed700052ab4c31" gracePeriod=30 Jul 29 19:26:43.363545 ip-10-0-131-93 kubenswrapper[2564]: I0729 19:26:43.363519 2564 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="opendatahub/maas-controller-84d5849b5d-fl8d4" Jul 29 19:26:43.468303 ip-10-0-131-93 kubenswrapper[2564]: I0729 19:26:43.468274 2564 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-sbplr\" (UniqueName: \"kubernetes.io/projected/dde5f313-e028-44b2-8adb-4689b8675327-kube-api-access-sbplr\") pod \"dde5f313-e028-44b2-8adb-4689b8675327\" (UID: \"dde5f313-e028-44b2-8adb-4689b8675327\") " Jul 29 19:26:43.468476 ip-10-0-131-93 kubenswrapper[2564]: I0729 19:26:43.468312 2564 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"webhook-cert\" (UniqueName: \"kubernetes.io/secret/dde5f313-e028-44b2-8adb-4689b8675327-webhook-cert\") pod \"dde5f313-e028-44b2-8adb-4689b8675327\" (UID: \"dde5f313-e028-44b2-8adb-4689b8675327\") " Jul 29 19:26:43.470523 ip-10-0-131-93 kubenswrapper[2564]: I0729 19:26:43.470488 2564 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/dde5f313-e028-44b2-8adb-4689b8675327-webhook-cert" (OuterVolumeSpecName: "webhook-cert") pod "dde5f313-e028-44b2-8adb-4689b8675327" (UID: "dde5f313-e028-44b2-8adb-4689b8675327"). InnerVolumeSpecName "webhook-cert". PluginName "kubernetes.io/secret", VolumeGIDValue "" Jul 29 19:26:43.470621 ip-10-0-131-93 kubenswrapper[2564]: I0729 19:26:43.470539 2564 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/dde5f313-e028-44b2-8adb-4689b8675327-kube-api-access-sbplr" (OuterVolumeSpecName: "kube-api-access-sbplr") pod "dde5f313-e028-44b2-8adb-4689b8675327" (UID: "dde5f313-e028-44b2-8adb-4689b8675327"). InnerVolumeSpecName "kube-api-access-sbplr". PluginName "kubernetes.io/projected", VolumeGIDValue "" Jul 29 19:26:43.568807 ip-10-0-131-93 kubenswrapper[2564]: I0729 19:26:43.568773 2564 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-sbplr\" (UniqueName: \"kubernetes.io/projected/dde5f313-e028-44b2-8adb-4689b8675327-kube-api-access-sbplr\") on node \"ip-10-0-131-93.ec2.internal\" DevicePath \"\"" Jul 29 19:26:43.568807 ip-10-0-131-93 kubenswrapper[2564]: I0729 19:26:43.568800 2564 reconciler_common.go:299] "Volume detached for volume \"webhook-cert\" (UniqueName: \"kubernetes.io/secret/dde5f313-e028-44b2-8adb-4689b8675327-webhook-cert\") on node \"ip-10-0-131-93.ec2.internal\" DevicePath \"\"" Jul 29 19:26:44.115102 ip-10-0-131-93 kubenswrapper[2564]: I0729 19:26:44.115063 2564 generic.go:356] "Generic (PLEG): container finished" podID="dde5f313-e028-44b2-8adb-4689b8675327" containerID="af28e819ebf6037bec40c5a990b4ff3c470abd2a5a6c255529ed700052ab4c31" exitCode=0 Jul 29 19:26:44.115102 ip-10-0-131-93 kubenswrapper[2564]: I0729 19:26:44.115106 2564 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="opendatahub/maas-controller-84d5849b5d-fl8d4" Jul 29 19:26:44.115576 ip-10-0-131-93 kubenswrapper[2564]: I0729 19:26:44.115156 2564 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="opendatahub/maas-controller-84d5849b5d-fl8d4" event={"ID":"dde5f313-e028-44b2-8adb-4689b8675327","Type":"ContainerDied","Data":"af28e819ebf6037bec40c5a990b4ff3c470abd2a5a6c255529ed700052ab4c31"} Jul 29 19:26:44.115576 ip-10-0-131-93 kubenswrapper[2564]: I0729 19:26:44.115200 2564 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="opendatahub/maas-controller-84d5849b5d-fl8d4" event={"ID":"dde5f313-e028-44b2-8adb-4689b8675327","Type":"ContainerDied","Data":"7676886692fdd2472b3aab5c67b426cd8c701bf544c157e213b58fc61dd1f409"} Jul 29 19:26:44.115576 ip-10-0-131-93 kubenswrapper[2564]: I0729 19:26:44.115220 2564 scope.go:117] "RemoveContainer" containerID="af28e819ebf6037bec40c5a990b4ff3c470abd2a5a6c255529ed700052ab4c31" Jul 29 19:26:44.124047 ip-10-0-131-93 kubenswrapper[2564]: I0729 19:26:44.124029 2564 scope.go:117] "RemoveContainer" containerID="af28e819ebf6037bec40c5a990b4ff3c470abd2a5a6c255529ed700052ab4c31" Jul 29 19:26:44.124314 ip-10-0-131-93 kubenswrapper[2564]: E0729 19:26:44.124296 2564 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"af28e819ebf6037bec40c5a990b4ff3c470abd2a5a6c255529ed700052ab4c31\": container with ID starting with af28e819ebf6037bec40c5a990b4ff3c470abd2a5a6c255529ed700052ab4c31 not found: ID does not exist" containerID="af28e819ebf6037bec40c5a990b4ff3c470abd2a5a6c255529ed700052ab4c31" Jul 29 19:26:44.124360 ip-10-0-131-93 kubenswrapper[2564]: I0729 19:26:44.124322 2564 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"af28e819ebf6037bec40c5a990b4ff3c470abd2a5a6c255529ed700052ab4c31"} err="failed to get container status \"af28e819ebf6037bec40c5a990b4ff3c470abd2a5a6c255529ed700052ab4c31\": rpc error: code = NotFound desc = could not find container \"af28e819ebf6037bec40c5a990b4ff3c470abd2a5a6c255529ed700052ab4c31\": container with ID starting with af28e819ebf6037bec40c5a990b4ff3c470abd2a5a6c255529ed700052ab4c31 not found: ID does not exist" Jul 29 19:26:44.135790 ip-10-0-131-93 kubenswrapper[2564]: I0729 19:26:44.135763 2564 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["opendatahub/maas-controller-84d5849b5d-fl8d4"] Jul 29 19:26:44.140673 ip-10-0-131-93 kubenswrapper[2564]: I0729 19:26:44.140651 2564 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["opendatahub/maas-controller-84d5849b5d-fl8d4"] Jul 29 19:26:44.782329 ip-10-0-131-93 kubenswrapper[2564]: I0729 19:26:44.782289 2564 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="dde5f313-e028-44b2-8adb-4689b8675327" path="/var/lib/kubelet/pods/dde5f313-e028-44b2-8adb-4689b8675327/volumes" Jul 29 19:26:52.109267 ip-10-0-131-93 kubenswrapper[2564]: I0729 19:26:52.109229 2564 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-444jx_faa8c546-c75b-4e10-93b4-661aa2b5f53e/konnectivity-agent/0.log" limit=1024 Jul 29 19:27:00.106113 ip-10-0-131-93 kubenswrapper[2564]: I0729 19:27:00.106064 2564 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-444jx_faa8c546-c75b-4e10-93b4-661aa2b5f53e/konnectivity-agent/0.log" limit=1024 Jul 29 19:27:00.203899 ip-10-0-131-93 kubenswrapper[2564]: I0729 19:27:00.203869 2564 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-444jx_faa8c546-c75b-4e10-93b4-661aa2b5f53e/konnectivity-agent/0.log" limit=1024 Jul 29 19:27:01.932119 ip-10-0-131-93 kubenswrapper[2564]: I0729 19:27:01.932091 2564 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["odh-ai-gateway-infra/maas-api-7899cfcd6f-cmvs7"] Jul 29 19:27:01.932506 ip-10-0-131-93 kubenswrapper[2564]: I0729 19:27:01.932419 2564 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="dde5f313-e028-44b2-8adb-4689b8675327" containerName="manager" Jul 29 19:27:01.932506 ip-10-0-131-93 kubenswrapper[2564]: I0729 19:27:01.932432 2564 state_mem.go:107] "Deleted CPUSet assignment" podUID="dde5f313-e028-44b2-8adb-4689b8675327" containerName="manager" Jul 29 19:27:01.932506 ip-10-0-131-93 kubenswrapper[2564]: I0729 19:27:01.932481 2564 memory_manager.go:368] "RemoveStaleState removing state" podUID="dde5f313-e028-44b2-8adb-4689b8675327" containerName="manager" Jul 29 19:27:01.938030 ip-10-0-131-93 kubenswrapper[2564]: I0729 19:27:01.938010 2564 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-7899cfcd6f-cmvs7" Jul 29 19:27:01.940615 ip-10-0-131-93 kubenswrapper[2564]: I0729 19:27:01.940593 2564 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"odh-ai-gateway-infra\"/\"maas-api-serving-cert\"" Jul 29 19:27:01.941565 ip-10-0-131-93 kubenswrapper[2564]: I0729 19:27:01.941548 2564 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"odh-ai-gateway-infra\"/\"kube-root-ca.crt\"" Jul 29 19:27:01.941657 ip-10-0-131-93 kubenswrapper[2564]: I0729 19:27:01.941551 2564 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"odh-ai-gateway-infra\"/\"openshift-service-ca.crt\"" Jul 29 19:27:01.949860 ip-10-0-131-93 kubenswrapper[2564]: I0729 19:27:01.949830 2564 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-7899cfcd6f-cmvs7"] Jul 29 19:27:02.017825 ip-10-0-131-93 kubenswrapper[2564]: I0729 19:27:02.017786 2564 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/af405bb4-058a-46df-84b0-1b5154a7f124-maas-api-tls\") pod \"maas-api-7899cfcd6f-cmvs7\" (UID: \"af405bb4-058a-46df-84b0-1b5154a7f124\") " pod="odh-ai-gateway-infra/maas-api-7899cfcd6f-cmvs7" Jul 29 19:27:02.017825 ip-10-0-131-93 kubenswrapper[2564]: I0729 19:27:02.017827 2564 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/af405bb4-058a-46df-84b0-1b5154a7f124-openshift-service-ca\") pod \"maas-api-7899cfcd6f-cmvs7\" (UID: \"af405bb4-058a-46df-84b0-1b5154a7f124\") " pod="odh-ai-gateway-infra/maas-api-7899cfcd6f-cmvs7" Jul 29 19:27:02.018030 ip-10-0-131-93 kubenswrapper[2564]: I0729 19:27:02.017859 2564 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-vxkz5\" (UniqueName: \"kubernetes.io/projected/af405bb4-058a-46df-84b0-1b5154a7f124-kube-api-access-vxkz5\") pod \"maas-api-7899cfcd6f-cmvs7\" (UID: \"af405bb4-058a-46df-84b0-1b5154a7f124\") " pod="odh-ai-gateway-infra/maas-api-7899cfcd6f-cmvs7" Jul 29 19:27:02.118465 ip-10-0-131-93 kubenswrapper[2564]: I0729 19:27:02.118412 2564 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-vxkz5\" (UniqueName: \"kubernetes.io/projected/af405bb4-058a-46df-84b0-1b5154a7f124-kube-api-access-vxkz5\") pod \"maas-api-7899cfcd6f-cmvs7\" (UID: \"af405bb4-058a-46df-84b0-1b5154a7f124\") " pod="odh-ai-gateway-infra/maas-api-7899cfcd6f-cmvs7" Jul 29 19:27:02.118655 ip-10-0-131-93 kubenswrapper[2564]: I0729 19:27:02.118508 2564 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/af405bb4-058a-46df-84b0-1b5154a7f124-maas-api-tls\") pod \"maas-api-7899cfcd6f-cmvs7\" (UID: \"af405bb4-058a-46df-84b0-1b5154a7f124\") " pod="odh-ai-gateway-infra/maas-api-7899cfcd6f-cmvs7" Jul 29 19:27:02.118655 ip-10-0-131-93 kubenswrapper[2564]: I0729 19:27:02.118531 2564 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/af405bb4-058a-46df-84b0-1b5154a7f124-openshift-service-ca\") pod \"maas-api-7899cfcd6f-cmvs7\" (UID: \"af405bb4-058a-46df-84b0-1b5154a7f124\") " pod="odh-ai-gateway-infra/maas-api-7899cfcd6f-cmvs7" Jul 29 19:27:02.119110 ip-10-0-131-93 kubenswrapper[2564]: I0729 19:27:02.119082 2564 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/af405bb4-058a-46df-84b0-1b5154a7f124-openshift-service-ca\") pod \"maas-api-7899cfcd6f-cmvs7\" (UID: \"af405bb4-058a-46df-84b0-1b5154a7f124\") " pod="odh-ai-gateway-infra/maas-api-7899cfcd6f-cmvs7" Jul 29 19:27:02.120929 ip-10-0-131-93 kubenswrapper[2564]: I0729 19:27:02.120900 2564 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/af405bb4-058a-46df-84b0-1b5154a7f124-maas-api-tls\") pod \"maas-api-7899cfcd6f-cmvs7\" (UID: \"af405bb4-058a-46df-84b0-1b5154a7f124\") " pod="odh-ai-gateway-infra/maas-api-7899cfcd6f-cmvs7" Jul 29 19:27:02.127171 ip-10-0-131-93 kubenswrapper[2564]: I0729 19:27:02.127151 2564 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-vxkz5\" (UniqueName: \"kubernetes.io/projected/af405bb4-058a-46df-84b0-1b5154a7f124-kube-api-access-vxkz5\") pod \"maas-api-7899cfcd6f-cmvs7\" (UID: \"af405bb4-058a-46df-84b0-1b5154a7f124\") " pod="odh-ai-gateway-infra/maas-api-7899cfcd6f-cmvs7" Jul 29 19:27:02.249448 ip-10-0-131-93 kubenswrapper[2564]: I0729 19:27:02.249410 2564 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-7899cfcd6f-cmvs7" Jul 29 19:27:02.393990 ip-10-0-131-93 kubenswrapper[2564]: I0729 19:27:02.393961 2564 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-7899cfcd6f-cmvs7"] Jul 29 19:27:02.395608 ip-10-0-131-93 kubenswrapper[2564]: W0729 19:27:02.395564 2564 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podaf405bb4_058a_46df_84b0_1b5154a7f124.slice/crio-e68a0071ae00b189a5e121e99191aab6f893c0409d158b3905823f31552fc9f1 WatchSource:0}: Error finding container e68a0071ae00b189a5e121e99191aab6f893c0409d158b3905823f31552fc9f1: Status 404 returned error can't find the container with id e68a0071ae00b189a5e121e99191aab6f893c0409d158b3905823f31552fc9f1 Jul 29 19:27:02.517636 ip-10-0-131-93 kubenswrapper[2564]: I0729 19:27:02.517520 2564 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-ingress/payload-processing-5cc6cd444b-zf8l2"] Jul 29 19:27:02.523247 ip-10-0-131-93 kubenswrapper[2564]: I0729 19:27:02.523228 2564 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-processing-5cc6cd444b-zf8l2" Jul 29 19:27:02.525963 ip-10-0-131-93 kubenswrapper[2564]: I0729 19:27:02.525941 2564 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress\"/\"payload-processing-dockercfg-ddj85\"" Jul 29 19:27:02.526166 ip-10-0-131-93 kubenswrapper[2564]: I0729 19:27:02.526152 2564 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ingress\"/\"payload-processing-plugins\"" Jul 29 19:27:02.534109 ip-10-0-131-93 kubenswrapper[2564]: I0729 19:27:02.534087 2564 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/payload-processing-5cc6cd444b-zf8l2"] Jul 29 19:27:02.623229 ip-10-0-131-93 kubenswrapper[2564]: I0729 19:27:02.623191 2564 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-mx5fz\" (UniqueName: \"kubernetes.io/projected/21cbaa33-9f8b-47c3-8cc1-5a368b477eb2-kube-api-access-mx5fz\") pod \"payload-processing-5cc6cd444b-zf8l2\" (UID: \"21cbaa33-9f8b-47c3-8cc1-5a368b477eb2\") " pod="openshift-ingress/payload-processing-5cc6cd444b-zf8l2" Jul 29 19:27:02.623426 ip-10-0-131-93 kubenswrapper[2564]: I0729 19:27:02.623248 2564 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/21cbaa33-9f8b-47c3-8cc1-5a368b477eb2-plugins-config-volume\") pod \"payload-processing-5cc6cd444b-zf8l2\" (UID: \"21cbaa33-9f8b-47c3-8cc1-5a368b477eb2\") " pod="openshift-ingress/payload-processing-5cc6cd444b-zf8l2" Jul 29 19:27:02.724332 ip-10-0-131-93 kubenswrapper[2564]: I0729 19:27:02.724294 2564 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/21cbaa33-9f8b-47c3-8cc1-5a368b477eb2-plugins-config-volume\") pod \"payload-processing-5cc6cd444b-zf8l2\" (UID: \"21cbaa33-9f8b-47c3-8cc1-5a368b477eb2\") " pod="openshift-ingress/payload-processing-5cc6cd444b-zf8l2" Jul 29 19:27:02.724526 ip-10-0-131-93 kubenswrapper[2564]: I0729 19:27:02.724455 2564 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-mx5fz\" (UniqueName: \"kubernetes.io/projected/21cbaa33-9f8b-47c3-8cc1-5a368b477eb2-kube-api-access-mx5fz\") pod \"payload-processing-5cc6cd444b-zf8l2\" (UID: \"21cbaa33-9f8b-47c3-8cc1-5a368b477eb2\") " pod="openshift-ingress/payload-processing-5cc6cd444b-zf8l2" Jul 29 19:27:02.725044 ip-10-0-131-93 kubenswrapper[2564]: I0729 19:27:02.725024 2564 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/21cbaa33-9f8b-47c3-8cc1-5a368b477eb2-plugins-config-volume\") pod \"payload-processing-5cc6cd444b-zf8l2\" (UID: \"21cbaa33-9f8b-47c3-8cc1-5a368b477eb2\") " pod="openshift-ingress/payload-processing-5cc6cd444b-zf8l2" Jul 29 19:27:02.732432 ip-10-0-131-93 kubenswrapper[2564]: I0729 19:27:02.732408 2564 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-mx5fz\" (UniqueName: \"kubernetes.io/projected/21cbaa33-9f8b-47c3-8cc1-5a368b477eb2-kube-api-access-mx5fz\") pod \"payload-processing-5cc6cd444b-zf8l2\" (UID: \"21cbaa33-9f8b-47c3-8cc1-5a368b477eb2\") " pod="openshift-ingress/payload-processing-5cc6cd444b-zf8l2" Jul 29 19:27:02.834201 ip-10-0-131-93 kubenswrapper[2564]: I0729 19:27:02.834114 2564 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-processing-5cc6cd444b-zf8l2" Jul 29 19:27:02.962235 ip-10-0-131-93 kubenswrapper[2564]: I0729 19:27:02.962210 2564 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/payload-processing-5cc6cd444b-zf8l2"] Jul 29 19:27:02.964722 ip-10-0-131-93 kubenswrapper[2564]: W0729 19:27:02.964692 2564 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod21cbaa33_9f8b_47c3_8cc1_5a368b477eb2.slice/crio-5fb677b7923df9e4de776679b14d1e94cda8500c73b04df37d9cc7bdf106129a WatchSource:0}: Error finding container 5fb677b7923df9e4de776679b14d1e94cda8500c73b04df37d9cc7bdf106129a: Status 404 returned error can't find the container with id 5fb677b7923df9e4de776679b14d1e94cda8500c73b04df37d9cc7bdf106129a Jul 29 19:27:03.204243 ip-10-0-131-93 kubenswrapper[2564]: I0729 19:27:03.204205 2564 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-7899cfcd6f-cmvs7" event={"ID":"af405bb4-058a-46df-84b0-1b5154a7f124","Type":"ContainerStarted","Data":"e68a0071ae00b189a5e121e99191aab6f893c0409d158b3905823f31552fc9f1"} Jul 29 19:27:03.205610 ip-10-0-131-93 kubenswrapper[2564]: I0729 19:27:03.205575 2564 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-processing-5cc6cd444b-zf8l2" event={"ID":"21cbaa33-9f8b-47c3-8cc1-5a368b477eb2","Type":"ContainerStarted","Data":"5fb677b7923df9e4de776679b14d1e94cda8500c73b04df37d9cc7bdf106129a"} Jul 29 19:27:03.464625 ip-10-0-131-93 kubenswrapper[2564]: I0729 19:27:03.464540 2564 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-444jx_faa8c546-c75b-4e10-93b4-661aa2b5f53e/konnectivity-agent/0.log" limit=1024 Jul 29 19:27:03.565584 ip-10-0-131-93 kubenswrapper[2564]: I0729 19:27:03.565544 2564 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-444jx_faa8c546-c75b-4e10-93b4-661aa2b5f53e/konnectivity-agent/0.log" limit=1024 Jul 29 19:27:05.218994 ip-10-0-131-93 kubenswrapper[2564]: I0729 19:27:05.218954 2564 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-processing-5cc6cd444b-zf8l2" event={"ID":"21cbaa33-9f8b-47c3-8cc1-5a368b477eb2","Type":"ContainerStarted","Data":"d65fae2fba0a0ac8032880f3a36142808856c60eb0f066b810f6a1cd5796e694"} Jul 29 19:27:05.219496 ip-10-0-131-93 kubenswrapper[2564]: I0729 19:27:05.219130 2564 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/payload-processing-5cc6cd444b-zf8l2" Jul 29 19:27:05.220535 ip-10-0-131-93 kubenswrapper[2564]: I0729 19:27:05.220513 2564 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-7899cfcd6f-cmvs7" event={"ID":"af405bb4-058a-46df-84b0-1b5154a7f124","Type":"ContainerStarted","Data":"b1d51e59c25f52dd8a56fa4b9ae0763fc91b05e70fb39ba0e0507698da258f18"} Jul 29 19:27:05.220675 ip-10-0-131-93 kubenswrapper[2564]: I0729 19:27:05.220565 2564 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="odh-ai-gateway-infra/maas-api-7899cfcd6f-cmvs7" Jul 29 19:27:05.237038 ip-10-0-131-93 kubenswrapper[2564]: I0729 19:27:05.236990 2564 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-ingress/payload-processing-5cc6cd444b-zf8l2" podStartSLOduration=1.600221097 podStartE2EDuration="3.236978641s" totalImagesPullingTime="1.636757544s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-29 19:27:02 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-29 19:27:05.235475479 +0000 UTC m=+600.959535165" watchObservedRunningTime="2026-07-29 19:27:05.236978641 +0000 UTC m=+600.961038336" Jul 29 19:27:05.257671 ip-10-0-131-93 kubenswrapper[2564]: I0729 19:27:05.257629 2564 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="odh-ai-gateway-infra/maas-api-7899cfcd6f-cmvs7" podStartSLOduration=2.429516443 podStartE2EDuration="4.257616086s" totalImagesPullingTime="1.828099643s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-29 19:27:01 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-29 19:27:05.255593468 +0000 UTC m=+600.979653153" watchObservedRunningTime="2026-07-29 19:27:05.257616086 +0000 UTC m=+600.981675779" Jul 29 19:27:11.231169 ip-10-0-131-93 kubenswrapper[2564]: I0729 19:27:11.231139 2564 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="odh-ai-gateway-infra/maas-api-7899cfcd6f-cmvs7" Jul 29 19:27:12.259315 ip-10-0-131-93 kubenswrapper[2564]: I0729 19:27:12.259269 2564 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-444jx_faa8c546-c75b-4e10-93b4-661aa2b5f53e/konnectivity-agent/0.log" limit=1024 Jul 29 19:27:12.355393 ip-10-0-131-93 kubenswrapper[2564]: I0729 19:27:12.355341 2564 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-444jx_faa8c546-c75b-4e10-93b4-661aa2b5f53e/konnectivity-agent/0.log" limit=1024 Jul 29 19:27:16.228550 ip-10-0-131-93 kubenswrapper[2564]: I0729 19:27:16.228519 2564 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/payload-processing-5cc6cd444b-zf8l2" Jul 29 19:27:19.099635 ip-10-0-131-93 kubenswrapper[2564]: I0729 19:27:19.099600 2564 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-444jx_faa8c546-c75b-4e10-93b4-661aa2b5f53e/konnectivity-agent/0.log" limit=1024 Jul 29 19:27:20.425153 ip-10-0-131-93 kubenswrapper[2564]: I0729 19:27:20.425113 2564 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-444jx_faa8c546-c75b-4e10-93b4-661aa2b5f53e/konnectivity-agent/0.log" limit=1024 Jul 29 19:27:28.670032 ip-10-0-131-93 kubenswrapper[2564]: I0729 19:27:28.669949 2564 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-444jx_faa8c546-c75b-4e10-93b4-661aa2b5f53e/konnectivity-agent/0.log" limit=1024 Jul 29 19:27:44.980431 ip-10-0-131-93 kubenswrapper[2564]: I0729 19:27:44.980398 2564 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-444jx_faa8c546-c75b-4e10-93b4-661aa2b5f53e/konnectivity-agent/0.log" limit=1024 Jul 29 19:27:45.338464 ip-10-0-131-93 kubenswrapper[2564]: I0729 19:27:45.338395 2564 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["llm/e2e-trlp-test-simulated-kserve-598466548d-m8bd5"] Jul 29 19:27:45.341887 ip-10-0-131-93 kubenswrapper[2564]: I0729 19:27:45.341865 2564 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="llm/e2e-trlp-test-simulated-kserve-598466548d-m8bd5" Jul 29 19:27:45.345326 ip-10-0-131-93 kubenswrapper[2564]: I0729 19:27:45.345302 2564 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"llm\"/\"default-dockercfg-qm9st\"" Jul 29 19:27:45.345476 ip-10-0-131-93 kubenswrapper[2564]: I0729 19:27:45.345307 2564 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"llm\"/\"openshift-service-ca.crt\"" Jul 29 19:27:45.345476 ip-10-0-131-93 kubenswrapper[2564]: I0729 19:27:45.345337 2564 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"llm\"/\"kube-root-ca.crt\"" Jul 29 19:27:45.345476 ip-10-0-131-93 kubenswrapper[2564]: I0729 19:27:45.345366 2564 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"llm\"/\"e2e-trlp-test-simulated-kserve-self-signed-certs\"" Jul 29 19:27:45.352050 ip-10-0-131-93 kubenswrapper[2564]: I0729 19:27:45.352030 2564 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["llm/e2e-trlp-test-simulated-kserve-598466548d-m8bd5"] Jul 29 19:27:45.450808 ip-10-0-131-93 kubenswrapper[2564]: I0729 19:27:45.450778 2564 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/9c2ff71a-f846-43de-92b0-5b8975ff575d-home\") pod \"e2e-trlp-test-simulated-kserve-598466548d-m8bd5\" (UID: \"9c2ff71a-f846-43de-92b0-5b8975ff575d\") " pod="llm/e2e-trlp-test-simulated-kserve-598466548d-m8bd5" Jul 29 19:27:45.450968 ip-10-0-131-93 kubenswrapper[2564]: I0729 19:27:45.450818 2564 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/9c2ff71a-f846-43de-92b0-5b8975ff575d-tmp-dir\") pod \"e2e-trlp-test-simulated-kserve-598466548d-m8bd5\" (UID: \"9c2ff71a-f846-43de-92b0-5b8975ff575d\") " pod="llm/e2e-trlp-test-simulated-kserve-598466548d-m8bd5" Jul 29 19:27:45.450968 ip-10-0-131-93 kubenswrapper[2564]: I0729 19:27:45.450841 2564 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/9c2ff71a-f846-43de-92b0-5b8975ff575d-tls-certs\") pod \"e2e-trlp-test-simulated-kserve-598466548d-m8bd5\" (UID: \"9c2ff71a-f846-43de-92b0-5b8975ff575d\") " pod="llm/e2e-trlp-test-simulated-kserve-598466548d-m8bd5" Jul 29 19:27:45.450968 ip-10-0-131-93 kubenswrapper[2564]: I0729 19:27:45.450871 2564 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-rq4nq\" (UniqueName: \"kubernetes.io/projected/9c2ff71a-f846-43de-92b0-5b8975ff575d-kube-api-access-rq4nq\") pod \"e2e-trlp-test-simulated-kserve-598466548d-m8bd5\" (UID: \"9c2ff71a-f846-43de-92b0-5b8975ff575d\") " pod="llm/e2e-trlp-test-simulated-kserve-598466548d-m8bd5" Jul 29 19:27:45.450968 ip-10-0-131-93 kubenswrapper[2564]: I0729 19:27:45.450912 2564 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/9c2ff71a-f846-43de-92b0-5b8975ff575d-model-cache\") pod \"e2e-trlp-test-simulated-kserve-598466548d-m8bd5\" (UID: \"9c2ff71a-f846-43de-92b0-5b8975ff575d\") " pod="llm/e2e-trlp-test-simulated-kserve-598466548d-m8bd5" Jul 29 19:27:45.450968 ip-10-0-131-93 kubenswrapper[2564]: I0729 19:27:45.450929 2564 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/9c2ff71a-f846-43de-92b0-5b8975ff575d-dshm\") pod \"e2e-trlp-test-simulated-kserve-598466548d-m8bd5\" (UID: \"9c2ff71a-f846-43de-92b0-5b8975ff575d\") " pod="llm/e2e-trlp-test-simulated-kserve-598466548d-m8bd5" Jul 29 19:27:45.551835 ip-10-0-131-93 kubenswrapper[2564]: I0729 19:27:45.551794 2564 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/9c2ff71a-f846-43de-92b0-5b8975ff575d-tmp-dir\") pod \"e2e-trlp-test-simulated-kserve-598466548d-m8bd5\" (UID: \"9c2ff71a-f846-43de-92b0-5b8975ff575d\") " pod="llm/e2e-trlp-test-simulated-kserve-598466548d-m8bd5" Jul 29 19:27:45.552028 ip-10-0-131-93 kubenswrapper[2564]: I0729 19:27:45.551844 2564 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/9c2ff71a-f846-43de-92b0-5b8975ff575d-tls-certs\") pod \"e2e-trlp-test-simulated-kserve-598466548d-m8bd5\" (UID: \"9c2ff71a-f846-43de-92b0-5b8975ff575d\") " pod="llm/e2e-trlp-test-simulated-kserve-598466548d-m8bd5" Jul 29 19:27:45.552028 ip-10-0-131-93 kubenswrapper[2564]: I0729 19:27:45.551891 2564 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-rq4nq\" (UniqueName: \"kubernetes.io/projected/9c2ff71a-f846-43de-92b0-5b8975ff575d-kube-api-access-rq4nq\") pod \"e2e-trlp-test-simulated-kserve-598466548d-m8bd5\" (UID: \"9c2ff71a-f846-43de-92b0-5b8975ff575d\") " pod="llm/e2e-trlp-test-simulated-kserve-598466548d-m8bd5" Jul 29 19:27:45.552028 ip-10-0-131-93 kubenswrapper[2564]: I0729 19:27:45.551919 2564 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/9c2ff71a-f846-43de-92b0-5b8975ff575d-model-cache\") pod \"e2e-trlp-test-simulated-kserve-598466548d-m8bd5\" (UID: \"9c2ff71a-f846-43de-92b0-5b8975ff575d\") " pod="llm/e2e-trlp-test-simulated-kserve-598466548d-m8bd5" Jul 29 19:27:45.552028 ip-10-0-131-93 kubenswrapper[2564]: I0729 19:27:45.551945 2564 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/9c2ff71a-f846-43de-92b0-5b8975ff575d-dshm\") pod \"e2e-trlp-test-simulated-kserve-598466548d-m8bd5\" (UID: \"9c2ff71a-f846-43de-92b0-5b8975ff575d\") " pod="llm/e2e-trlp-test-simulated-kserve-598466548d-m8bd5" Jul 29 19:27:45.552028 ip-10-0-131-93 kubenswrapper[2564]: I0729 19:27:45.551998 2564 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/9c2ff71a-f846-43de-92b0-5b8975ff575d-home\") pod \"e2e-trlp-test-simulated-kserve-598466548d-m8bd5\" (UID: \"9c2ff71a-f846-43de-92b0-5b8975ff575d\") " pod="llm/e2e-trlp-test-simulated-kserve-598466548d-m8bd5" Jul 29 19:27:45.552288 ip-10-0-131-93 kubenswrapper[2564]: I0729 19:27:45.552200 2564 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/9c2ff71a-f846-43de-92b0-5b8975ff575d-tmp-dir\") pod \"e2e-trlp-test-simulated-kserve-598466548d-m8bd5\" (UID: \"9c2ff71a-f846-43de-92b0-5b8975ff575d\") " pod="llm/e2e-trlp-test-simulated-kserve-598466548d-m8bd5" Jul 29 19:27:45.552347 ip-10-0-131-93 kubenswrapper[2564]: I0729 19:27:45.552295 2564 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/9c2ff71a-f846-43de-92b0-5b8975ff575d-model-cache\") pod \"e2e-trlp-test-simulated-kserve-598466548d-m8bd5\" (UID: \"9c2ff71a-f846-43de-92b0-5b8975ff575d\") " pod="llm/e2e-trlp-test-simulated-kserve-598466548d-m8bd5" Jul 29 19:27:45.552438 ip-10-0-131-93 kubenswrapper[2564]: I0729 19:27:45.552404 2564 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/9c2ff71a-f846-43de-92b0-5b8975ff575d-home\") pod \"e2e-trlp-test-simulated-kserve-598466548d-m8bd5\" (UID: \"9c2ff71a-f846-43de-92b0-5b8975ff575d\") " pod="llm/e2e-trlp-test-simulated-kserve-598466548d-m8bd5" Jul 29 19:27:45.554278 ip-10-0-131-93 kubenswrapper[2564]: I0729 19:27:45.554253 2564 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/9c2ff71a-f846-43de-92b0-5b8975ff575d-dshm\") pod \"e2e-trlp-test-simulated-kserve-598466548d-m8bd5\" (UID: \"9c2ff71a-f846-43de-92b0-5b8975ff575d\") " pod="llm/e2e-trlp-test-simulated-kserve-598466548d-m8bd5" Jul 29 19:27:45.554580 ip-10-0-131-93 kubenswrapper[2564]: I0729 19:27:45.554560 2564 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/9c2ff71a-f846-43de-92b0-5b8975ff575d-tls-certs\") pod \"e2e-trlp-test-simulated-kserve-598466548d-m8bd5\" (UID: \"9c2ff71a-f846-43de-92b0-5b8975ff575d\") " pod="llm/e2e-trlp-test-simulated-kserve-598466548d-m8bd5" Jul 29 19:27:45.560967 ip-10-0-131-93 kubenswrapper[2564]: I0729 19:27:45.560944 2564 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-rq4nq\" (UniqueName: \"kubernetes.io/projected/9c2ff71a-f846-43de-92b0-5b8975ff575d-kube-api-access-rq4nq\") pod \"e2e-trlp-test-simulated-kserve-598466548d-m8bd5\" (UID: \"9c2ff71a-f846-43de-92b0-5b8975ff575d\") " pod="llm/e2e-trlp-test-simulated-kserve-598466548d-m8bd5" Jul 29 19:27:45.652705 ip-10-0-131-93 kubenswrapper[2564]: I0729 19:27:45.652615 2564 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="llm/e2e-trlp-test-simulated-kserve-598466548d-m8bd5" Jul 29 19:27:45.779599 ip-10-0-131-93 kubenswrapper[2564]: I0729 19:27:45.779577 2564 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["llm/e2e-trlp-test-simulated-kserve-598466548d-m8bd5"] Jul 29 19:27:45.784393 ip-10-0-131-93 kubenswrapper[2564]: W0729 19:27:45.782019 2564 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod9c2ff71a_f846_43de_92b0_5b8975ff575d.slice/crio-0c8bb137f6f5f32108650c214f3a63113bb613405a2545b46872cb0ac3d0b275 WatchSource:0}: Error finding container 0c8bb137f6f5f32108650c214f3a63113bb613405a2545b46872cb0ac3d0b275: Status 404 returned error can't find the container with id 0c8bb137f6f5f32108650c214f3a63113bb613405a2545b46872cb0ac3d0b275 Jul 29 19:27:45.785347 ip-10-0-131-93 kubenswrapper[2564]: I0729 19:27:45.785330 2564 provider.go:93] Refreshing cache for provider: *credentialprovider.defaultDockerConfigProvider Jul 29 19:27:46.413329 ip-10-0-131-93 kubenswrapper[2564]: I0729 19:27:46.413282 2564 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="llm/e2e-trlp-test-simulated-kserve-598466548d-m8bd5" event={"ID":"9c2ff71a-f846-43de-92b0-5b8975ff575d","Type":"ContainerStarted","Data":"0c8bb137f6f5f32108650c214f3a63113bb613405a2545b46872cb0ac3d0b275"} Jul 29 19:27:47.418839 ip-10-0-131-93 kubenswrapper[2564]: I0729 19:27:47.418798 2564 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="llm/e2e-trlp-test-simulated-kserve-598466548d-m8bd5" event={"ID":"9c2ff71a-f846-43de-92b0-5b8975ff575d","Type":"ContainerStarted","Data":"36b9e559ed8bac05a1e4179ef9c8bf4dbd933379c0ee9ede64e825d55c07d7d8"} Jul 29 19:27:47.439643 ip-10-0-131-93 kubenswrapper[2564]: I0729 19:27:47.439594 2564 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="llm/e2e-trlp-test-simulated-kserve-598466548d-m8bd5" podStartSLOduration=1.5140236040000001 podStartE2EDuration="2.439578855s" totalImagesPullingTime="925.555251ms" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-29 19:27:45 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-29 19:27:47.43786732 +0000 UTC m=+643.161927024" watchObservedRunningTime="2026-07-29 19:27:47.439578855 +0000 UTC m=+643.163638546" Jul 29 19:27:48.488233 ip-10-0-131-93 kubenswrapper[2564]: I0729 19:27:48.488201 2564 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-444jx_faa8c546-c75b-4e10-93b4-661aa2b5f53e/konnectivity-agent/0.log" limit=1024 Jul 29 19:27:54.655723 ip-10-0-131-93 kubenswrapper[2564]: I0729 19:27:54.655690 2564 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["llm/e2e-unconfigured-facebook-opt-125m-simulated-kserve-85b68f6cd4t"] Jul 29 19:27:54.658257 ip-10-0-131-93 kubenswrapper[2564]: I0729 19:27:54.658239 2564 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="llm/e2e-unconfigured-facebook-opt-125m-simulated-kserve-85b68f6cd4t" Jul 29 19:27:54.660822 ip-10-0-131-93 kubenswrapper[2564]: I0729 19:27:54.660803 2564 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"llm\"/\"e2e-unab60ef4d3a239b5143b412cab04acac3-kserve-self-signed-certs\"" Jul 29 19:27:54.670008 ip-10-0-131-93 kubenswrapper[2564]: I0729 19:27:54.669985 2564 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["llm/e2e-unconfigured-facebook-opt-125m-simulated-kserve-85b68f6cd4t"] Jul 29 19:27:54.681682 ip-10-0-131-93 kubenswrapper[2564]: I0729 19:27:54.681659 2564 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-444jx_faa8c546-c75b-4e10-93b4-661aa2b5f53e/konnectivity-agent/0.log" limit=1024 Jul 29 19:27:54.730745 ip-10-0-131-93 kubenswrapper[2564]: I0729 19:27:54.730711 2564 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/9c140dd1-b52c-4615-9145-b0c0e4a53e43-model-cache\") pod \"e2e-unconfigured-facebook-opt-125m-simulated-kserve-85b68f6cd4t\" (UID: \"9c140dd1-b52c-4615-9145-b0c0e4a53e43\") " pod="llm/e2e-unconfigured-facebook-opt-125m-simulated-kserve-85b68f6cd4t" Jul 29 19:27:54.730745 ip-10-0-131-93 kubenswrapper[2564]: I0729 19:27:54.730749 2564 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/9c140dd1-b52c-4615-9145-b0c0e4a53e43-tmp-dir\") pod \"e2e-unconfigured-facebook-opt-125m-simulated-kserve-85b68f6cd4t\" (UID: \"9c140dd1-b52c-4615-9145-b0c0e4a53e43\") " pod="llm/e2e-unconfigured-facebook-opt-125m-simulated-kserve-85b68f6cd4t" Jul 29 19:27:54.730943 ip-10-0-131-93 kubenswrapper[2564]: I0729 19:27:54.730784 2564 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/9c140dd1-b52c-4615-9145-b0c0e4a53e43-home\") pod \"e2e-unconfigured-facebook-opt-125m-simulated-kserve-85b68f6cd4t\" (UID: \"9c140dd1-b52c-4615-9145-b0c0e4a53e43\") " pod="llm/e2e-unconfigured-facebook-opt-125m-simulated-kserve-85b68f6cd4t" Jul 29 19:27:54.730943 ip-10-0-131-93 kubenswrapper[2564]: I0729 19:27:54.730807 2564 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/9c140dd1-b52c-4615-9145-b0c0e4a53e43-tls-certs\") pod \"e2e-unconfigured-facebook-opt-125m-simulated-kserve-85b68f6cd4t\" (UID: \"9c140dd1-b52c-4615-9145-b0c0e4a53e43\") " pod="llm/e2e-unconfigured-facebook-opt-125m-simulated-kserve-85b68f6cd4t" Jul 29 19:27:54.730943 ip-10-0-131-93 kubenswrapper[2564]: I0729 19:27:54.730833 2564 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/9c140dd1-b52c-4615-9145-b0c0e4a53e43-dshm\") pod \"e2e-unconfigured-facebook-opt-125m-simulated-kserve-85b68f6cd4t\" (UID: \"9c140dd1-b52c-4615-9145-b0c0e4a53e43\") " pod="llm/e2e-unconfigured-facebook-opt-125m-simulated-kserve-85b68f6cd4t" Jul 29 19:27:54.730943 ip-10-0-131-93 kubenswrapper[2564]: I0729 19:27:54.730850 2564 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-b2vqx\" (UniqueName: \"kubernetes.io/projected/9c140dd1-b52c-4615-9145-b0c0e4a53e43-kube-api-access-b2vqx\") pod \"e2e-unconfigured-facebook-opt-125m-simulated-kserve-85b68f6cd4t\" (UID: \"9c140dd1-b52c-4615-9145-b0c0e4a53e43\") " pod="llm/e2e-unconfigured-facebook-opt-125m-simulated-kserve-85b68f6cd4t" Jul 29 19:27:54.831413 ip-10-0-131-93 kubenswrapper[2564]: I0729 19:27:54.831360 2564 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/9c140dd1-b52c-4615-9145-b0c0e4a53e43-model-cache\") pod \"e2e-unconfigured-facebook-opt-125m-simulated-kserve-85b68f6cd4t\" (UID: \"9c140dd1-b52c-4615-9145-b0c0e4a53e43\") " pod="llm/e2e-unconfigured-facebook-opt-125m-simulated-kserve-85b68f6cd4t" Jul 29 19:27:54.831586 ip-10-0-131-93 kubenswrapper[2564]: I0729 19:27:54.831429 2564 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/9c140dd1-b52c-4615-9145-b0c0e4a53e43-tmp-dir\") pod \"e2e-unconfigured-facebook-opt-125m-simulated-kserve-85b68f6cd4t\" (UID: \"9c140dd1-b52c-4615-9145-b0c0e4a53e43\") " pod="llm/e2e-unconfigured-facebook-opt-125m-simulated-kserve-85b68f6cd4t" Jul 29 19:27:54.831586 ip-10-0-131-93 kubenswrapper[2564]: I0729 19:27:54.831464 2564 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/9c140dd1-b52c-4615-9145-b0c0e4a53e43-home\") pod \"e2e-unconfigured-facebook-opt-125m-simulated-kserve-85b68f6cd4t\" (UID: \"9c140dd1-b52c-4615-9145-b0c0e4a53e43\") " pod="llm/e2e-unconfigured-facebook-opt-125m-simulated-kserve-85b68f6cd4t" Jul 29 19:27:54.831586 ip-10-0-131-93 kubenswrapper[2564]: I0729 19:27:54.831486 2564 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/9c140dd1-b52c-4615-9145-b0c0e4a53e43-tls-certs\") pod \"e2e-unconfigured-facebook-opt-125m-simulated-kserve-85b68f6cd4t\" (UID: \"9c140dd1-b52c-4615-9145-b0c0e4a53e43\") " pod="llm/e2e-unconfigured-facebook-opt-125m-simulated-kserve-85b68f6cd4t" Jul 29 19:27:54.831586 ip-10-0-131-93 kubenswrapper[2564]: I0729 19:27:54.831520 2564 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/9c140dd1-b52c-4615-9145-b0c0e4a53e43-dshm\") pod \"e2e-unconfigured-facebook-opt-125m-simulated-kserve-85b68f6cd4t\" (UID: \"9c140dd1-b52c-4615-9145-b0c0e4a53e43\") " pod="llm/e2e-unconfigured-facebook-opt-125m-simulated-kserve-85b68f6cd4t" Jul 29 19:27:54.831814 ip-10-0-131-93 kubenswrapper[2564]: I0729 19:27:54.831631 2564 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-b2vqx\" (UniqueName: \"kubernetes.io/projected/9c140dd1-b52c-4615-9145-b0c0e4a53e43-kube-api-access-b2vqx\") pod \"e2e-unconfigured-facebook-opt-125m-simulated-kserve-85b68f6cd4t\" (UID: \"9c140dd1-b52c-4615-9145-b0c0e4a53e43\") " pod="llm/e2e-unconfigured-facebook-opt-125m-simulated-kserve-85b68f6cd4t" Jul 29 19:27:54.831869 ip-10-0-131-93 kubenswrapper[2564]: I0729 19:27:54.831809 2564 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/9c140dd1-b52c-4615-9145-b0c0e4a53e43-model-cache\") pod \"e2e-unconfigured-facebook-opt-125m-simulated-kserve-85b68f6cd4t\" (UID: \"9c140dd1-b52c-4615-9145-b0c0e4a53e43\") " pod="llm/e2e-unconfigured-facebook-opt-125m-simulated-kserve-85b68f6cd4t" Jul 29 19:27:54.831869 ip-10-0-131-93 kubenswrapper[2564]: I0729 19:27:54.831836 2564 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/9c140dd1-b52c-4615-9145-b0c0e4a53e43-tmp-dir\") pod \"e2e-unconfigured-facebook-opt-125m-simulated-kserve-85b68f6cd4t\" (UID: \"9c140dd1-b52c-4615-9145-b0c0e4a53e43\") " pod="llm/e2e-unconfigured-facebook-opt-125m-simulated-kserve-85b68f6cd4t" Jul 29 19:27:54.832170 ip-10-0-131-93 kubenswrapper[2564]: I0729 19:27:54.832141 2564 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/9c140dd1-b52c-4615-9145-b0c0e4a53e43-home\") pod \"e2e-unconfigured-facebook-opt-125m-simulated-kserve-85b68f6cd4t\" (UID: \"9c140dd1-b52c-4615-9145-b0c0e4a53e43\") " pod="llm/e2e-unconfigured-facebook-opt-125m-simulated-kserve-85b68f6cd4t" Jul 29 19:27:54.834038 ip-10-0-131-93 kubenswrapper[2564]: I0729 19:27:54.834004 2564 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/9c140dd1-b52c-4615-9145-b0c0e4a53e43-dshm\") pod \"e2e-unconfigured-facebook-opt-125m-simulated-kserve-85b68f6cd4t\" (UID: \"9c140dd1-b52c-4615-9145-b0c0e4a53e43\") " pod="llm/e2e-unconfigured-facebook-opt-125m-simulated-kserve-85b68f6cd4t" Jul 29 19:27:54.834432 ip-10-0-131-93 kubenswrapper[2564]: I0729 19:27:54.834406 2564 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/9c140dd1-b52c-4615-9145-b0c0e4a53e43-tls-certs\") pod \"e2e-unconfigured-facebook-opt-125m-simulated-kserve-85b68f6cd4t\" (UID: \"9c140dd1-b52c-4615-9145-b0c0e4a53e43\") " pod="llm/e2e-unconfigured-facebook-opt-125m-simulated-kserve-85b68f6cd4t" Jul 29 19:27:54.844525 ip-10-0-131-93 kubenswrapper[2564]: I0729 19:27:54.844474 2564 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-b2vqx\" (UniqueName: \"kubernetes.io/projected/9c140dd1-b52c-4615-9145-b0c0e4a53e43-kube-api-access-b2vqx\") pod \"e2e-unconfigured-facebook-opt-125m-simulated-kserve-85b68f6cd4t\" (UID: \"9c140dd1-b52c-4615-9145-b0c0e4a53e43\") " pod="llm/e2e-unconfigured-facebook-opt-125m-simulated-kserve-85b68f6cd4t" Jul 29 19:27:54.968510 ip-10-0-131-93 kubenswrapper[2564]: I0729 19:27:54.968472 2564 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="llm/e2e-unconfigured-facebook-opt-125m-simulated-kserve-85b68f6cd4t" Jul 29 19:27:55.094753 ip-10-0-131-93 kubenswrapper[2564]: I0729 19:27:55.094725 2564 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["llm/e2e-unconfigured-facebook-opt-125m-simulated-kserve-85b68f6cd4t"] Jul 29 19:27:55.096292 ip-10-0-131-93 kubenswrapper[2564]: W0729 19:27:55.096268 2564 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod9c140dd1_b52c_4615_9145_b0c0e4a53e43.slice/crio-faa53b2e19100aa35c0a99caf48e7d8f5a509755a19970547aa0f1cbd7e29ccc WatchSource:0}: Error finding container faa53b2e19100aa35c0a99caf48e7d8f5a509755a19970547aa0f1cbd7e29ccc: Status 404 returned error can't find the container with id faa53b2e19100aa35c0a99caf48e7d8f5a509755a19970547aa0f1cbd7e29ccc Jul 29 19:27:55.456251 ip-10-0-131-93 kubenswrapper[2564]: I0729 19:27:55.456195 2564 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="llm/e2e-unconfigured-facebook-opt-125m-simulated-kserve-85b68f6cd4t" event={"ID":"9c140dd1-b52c-4615-9145-b0c0e4a53e43","Type":"ContainerStarted","Data":"61e9c6cf5a4586c11499f9cdb402969a6dbc787734d1010de89b5b3cc8769062"} Jul 29 19:27:55.456446 ip-10-0-131-93 kubenswrapper[2564]: I0729 19:27:55.456271 2564 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="llm/e2e-unconfigured-facebook-opt-125m-simulated-kserve-85b68f6cd4t" event={"ID":"9c140dd1-b52c-4615-9145-b0c0e4a53e43","Type":"ContainerStarted","Data":"faa53b2e19100aa35c0a99caf48e7d8f5a509755a19970547aa0f1cbd7e29ccc"} Jul 29 19:27:55.475186 ip-10-0-131-93 kubenswrapper[2564]: I0729 19:27:55.475136 2564 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="llm/e2e-unconfigured-facebook-opt-125m-simulated-kserve-85b68f6cd4t" podStartSLOduration=1.296914143 podStartE2EDuration="1.475119662s" totalImagesPullingTime="178.205519ms" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-29 19:27:54 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-29 19:27:55.473796216 +0000 UTC m=+651.197855907" watchObservedRunningTime="2026-07-29 19:27:55.475119662 +0000 UTC m=+651.199179353" Jul 29 19:27:55.652912 ip-10-0-131-93 kubenswrapper[2564]: I0729 19:27:55.652829 2564 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="llm/e2e-trlp-test-simulated-kserve-598466548d-m8bd5" Jul 29 19:27:55.652912 ip-10-0-131-93 kubenswrapper[2564]: I0729 19:27:55.652899 2564 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="llm/e2e-trlp-test-simulated-kserve-598466548d-m8bd5" Jul 29 19:27:55.665017 ip-10-0-131-93 kubenswrapper[2564]: I0729 19:27:55.664992 2564 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="llm/e2e-trlp-test-simulated-kserve-598466548d-m8bd5" Jul 29 19:27:56.472428 ip-10-0-131-93 kubenswrapper[2564]: I0729 19:27:56.472394 2564 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="llm/e2e-trlp-test-simulated-kserve-598466548d-m8bd5" Jul 29 19:27:57.945516 ip-10-0-131-93 kubenswrapper[2564]: I0729 19:27:57.945484 2564 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-444jx_faa8c546-c75b-4e10-93b4-661aa2b5f53e/konnectivity-agent/0.log" limit=1024 Jul 29 19:28:04.969424 ip-10-0-131-93 kubenswrapper[2564]: I0729 19:28:04.969358 2564 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="llm/e2e-unconfigured-facebook-opt-125m-simulated-kserve-85b68f6cd4t" Jul 29 19:28:04.969795 ip-10-0-131-93 kubenswrapper[2564]: I0729 19:28:04.969458 2564 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="llm/e2e-unconfigured-facebook-opt-125m-simulated-kserve-85b68f6cd4t" Jul 29 19:28:04.981499 ip-10-0-131-93 kubenswrapper[2564]: I0729 19:28:04.981478 2564 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="llm/e2e-unconfigured-facebook-opt-125m-simulated-kserve-85b68f6cd4t" Jul 29 19:28:05.512750 ip-10-0-131-93 kubenswrapper[2564]: I0729 19:28:05.512725 2564 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="llm/e2e-unconfigured-facebook-opt-125m-simulated-kserve-85b68f6cd4t" Jul 29 19:28:11.984583 ip-10-0-131-93 kubenswrapper[2564]: I0729 19:28:11.984539 2564 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-444jx_faa8c546-c75b-4e10-93b4-661aa2b5f53e/konnectivity-agent/0.log" limit=1024 Jul 29 19:28:59.824142 ip-10-0-131-93 kubenswrapper[2564]: I0729 19:28:59.824030 2564 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-444jx_faa8c546-c75b-4e10-93b4-661aa2b5f53e/konnectivity-agent/0.log" limit=1024 Jul 29 19:28:59.881786 ip-10-0-131-93 kubenswrapper[2564]: I0729 19:28:59.881737 2564 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-444jx_faa8c546-c75b-4e10-93b4-661aa2b5f53e/konnectivity-agent/0.log" limit=1024 Jul 29 19:29:00.337121 ip-10-0-131-93 kubenswrapper[2564]: I0729 19:29:00.337076 2564 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-444jx_faa8c546-c75b-4e10-93b4-661aa2b5f53e/konnectivity-agent/0.log" limit=1024 Jul 29 19:29:00.393811 ip-10-0-131-93 kubenswrapper[2564]: I0729 19:29:00.393779 2564 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-444jx_faa8c546-c75b-4e10-93b4-661aa2b5f53e/konnectivity-agent/0.log" limit=1024 Jul 29 19:29:14.797278 ip-10-0-131-93 kubenswrapper[2564]: I0729 19:29:14.797246 2564 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-444jx_faa8c546-c75b-4e10-93b4-661aa2b5f53e/konnectivity-agent/0.log" limit=1024 Jul 29 19:29:14.896053 ip-10-0-131-93 kubenswrapper[2564]: I0729 19:29:14.896022 2564 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-444jx_faa8c546-c75b-4e10-93b4-661aa2b5f53e/konnectivity-agent/0.log" limit=1024 Jul 29 19:29:15.907278 ip-10-0-131-93 kubenswrapper[2564]: I0729 19:29:15.907242 2564 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-444jx_faa8c546-c75b-4e10-93b4-661aa2b5f53e/konnectivity-agent/0.log" limit=1024 Jul 29 19:29:15.993706 ip-10-0-131-93 kubenswrapper[2564]: I0729 19:29:15.993671 2564 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-444jx_faa8c546-c75b-4e10-93b4-661aa2b5f53e/konnectivity-agent/0.log" limit=1024 Jul 29 19:29:47.900997 ip-10-0-131-93 kubenswrapper[2564]: I0729 19:29:47.900960 2564 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-444jx_faa8c546-c75b-4e10-93b4-661aa2b5f53e/konnectivity-agent/0.log" limit=1024 Jul 29 19:29:47.966024 ip-10-0-131-93 kubenswrapper[2564]: I0729 19:29:47.965989 2564 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-444jx_faa8c546-c75b-4e10-93b4-661aa2b5f53e/konnectivity-agent/0.log" limit=1024 Jul 29 19:30:01.508210 ip-10-0-131-93 kubenswrapper[2564]: I0729 19:30:01.508170 2564 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["opendatahub/maas-controller-865c54c777-z6f87"] Jul 29 19:30:01.508725 ip-10-0-131-93 kubenswrapper[2564]: I0729 19:30:01.508409 2564 kuberuntime_container.go:873] "Killing container with a grace period" pod="opendatahub/maas-controller-865c54c777-z6f87" podUID="96579673-c9a2-42bd-8159-7329efcf6772" containerName="manager" containerID="cri-o://2960ac3c6a20a036f8f72de1b9849955427d08c8bed825b754d60c7b9f19aec8" gracePeriod=30 Jul 29 19:30:01.764743 ip-10-0-131-93 kubenswrapper[2564]: I0729 19:30:01.764684 2564 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="opendatahub/maas-controller-865c54c777-z6f87" Jul 29 19:30:01.898475 ip-10-0-131-93 kubenswrapper[2564]: I0729 19:30:01.898425 2564 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"webhook-cert\" (UniqueName: \"kubernetes.io/secret/96579673-c9a2-42bd-8159-7329efcf6772-webhook-cert\") pod \"96579673-c9a2-42bd-8159-7329efcf6772\" (UID: \"96579673-c9a2-42bd-8159-7329efcf6772\") " Jul 29 19:30:01.898660 ip-10-0-131-93 kubenswrapper[2564]: I0729 19:30:01.898489 2564 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-mqrnt\" (UniqueName: \"kubernetes.io/projected/96579673-c9a2-42bd-8159-7329efcf6772-kube-api-access-mqrnt\") pod \"96579673-c9a2-42bd-8159-7329efcf6772\" (UID: \"96579673-c9a2-42bd-8159-7329efcf6772\") " Jul 29 19:30:01.900640 ip-10-0-131-93 kubenswrapper[2564]: I0729 19:30:01.900605 2564 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/96579673-c9a2-42bd-8159-7329efcf6772-kube-api-access-mqrnt" (OuterVolumeSpecName: "kube-api-access-mqrnt") pod "96579673-c9a2-42bd-8159-7329efcf6772" (UID: "96579673-c9a2-42bd-8159-7329efcf6772"). InnerVolumeSpecName "kube-api-access-mqrnt". PluginName "kubernetes.io/projected", VolumeGIDValue "" Jul 29 19:30:01.900640 ip-10-0-131-93 kubenswrapper[2564]: I0729 19:30:01.900628 2564 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/96579673-c9a2-42bd-8159-7329efcf6772-webhook-cert" (OuterVolumeSpecName: "webhook-cert") pod "96579673-c9a2-42bd-8159-7329efcf6772" (UID: "96579673-c9a2-42bd-8159-7329efcf6772"). InnerVolumeSpecName "webhook-cert". PluginName "kubernetes.io/secret", VolumeGIDValue "" Jul 29 19:30:01.999959 ip-10-0-131-93 kubenswrapper[2564]: I0729 19:30:01.999918 2564 reconciler_common.go:299] "Volume detached for volume \"webhook-cert\" (UniqueName: \"kubernetes.io/secret/96579673-c9a2-42bd-8159-7329efcf6772-webhook-cert\") on node \"ip-10-0-131-93.ec2.internal\" DevicePath \"\"" Jul 29 19:30:01.999959 ip-10-0-131-93 kubenswrapper[2564]: I0729 19:30:01.999952 2564 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-mqrnt\" (UniqueName: \"kubernetes.io/projected/96579673-c9a2-42bd-8159-7329efcf6772-kube-api-access-mqrnt\") on node \"ip-10-0-131-93.ec2.internal\" DevicePath \"\"" Jul 29 19:30:02.042232 ip-10-0-131-93 kubenswrapper[2564]: I0729 19:30:02.042142 2564 generic.go:356] "Generic (PLEG): container finished" podID="96579673-c9a2-42bd-8159-7329efcf6772" containerID="2960ac3c6a20a036f8f72de1b9849955427d08c8bed825b754d60c7b9f19aec8" exitCode=0 Jul 29 19:30:02.042232 ip-10-0-131-93 kubenswrapper[2564]: I0729 19:30:02.042200 2564 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="opendatahub/maas-controller-865c54c777-z6f87" Jul 29 19:30:02.042232 ip-10-0-131-93 kubenswrapper[2564]: I0729 19:30:02.042216 2564 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="opendatahub/maas-controller-865c54c777-z6f87" event={"ID":"96579673-c9a2-42bd-8159-7329efcf6772","Type":"ContainerDied","Data":"2960ac3c6a20a036f8f72de1b9849955427d08c8bed825b754d60c7b9f19aec8"} Jul 29 19:30:02.042475 ip-10-0-131-93 kubenswrapper[2564]: I0729 19:30:02.042285 2564 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="opendatahub/maas-controller-865c54c777-z6f87" event={"ID":"96579673-c9a2-42bd-8159-7329efcf6772","Type":"ContainerDied","Data":"ed87458591c61397556bbda73186e280fd985a7760dc00c05cdd0f6f447186b4"} Jul 29 19:30:02.042475 ip-10-0-131-93 kubenswrapper[2564]: I0729 19:30:02.042308 2564 scope.go:117] "RemoveContainer" containerID="2960ac3c6a20a036f8f72de1b9849955427d08c8bed825b754d60c7b9f19aec8" Jul 29 19:30:02.051478 ip-10-0-131-93 kubenswrapper[2564]: I0729 19:30:02.051457 2564 scope.go:117] "RemoveContainer" containerID="2960ac3c6a20a036f8f72de1b9849955427d08c8bed825b754d60c7b9f19aec8" Jul 29 19:30:02.051766 ip-10-0-131-93 kubenswrapper[2564]: E0729 19:30:02.051745 2564 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"2960ac3c6a20a036f8f72de1b9849955427d08c8bed825b754d60c7b9f19aec8\": container with ID starting with 2960ac3c6a20a036f8f72de1b9849955427d08c8bed825b754d60c7b9f19aec8 not found: ID does not exist" containerID="2960ac3c6a20a036f8f72de1b9849955427d08c8bed825b754d60c7b9f19aec8" Jul 29 19:30:02.051837 ip-10-0-131-93 kubenswrapper[2564]: I0729 19:30:02.051778 2564 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"2960ac3c6a20a036f8f72de1b9849955427d08c8bed825b754d60c7b9f19aec8"} err="failed to get container status \"2960ac3c6a20a036f8f72de1b9849955427d08c8bed825b754d60c7b9f19aec8\": rpc error: code = NotFound desc = could not find container \"2960ac3c6a20a036f8f72de1b9849955427d08c8bed825b754d60c7b9f19aec8\": container with ID starting with 2960ac3c6a20a036f8f72de1b9849955427d08c8bed825b754d60c7b9f19aec8 not found: ID does not exist" Jul 29 19:30:02.064824 ip-10-0-131-93 kubenswrapper[2564]: I0729 19:30:02.064789 2564 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["opendatahub/maas-controller-865c54c777-z6f87"] Jul 29 19:30:02.068577 ip-10-0-131-93 kubenswrapper[2564]: I0729 19:30:02.068550 2564 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["opendatahub/maas-controller-865c54c777-z6f87"] Jul 29 19:30:02.782983 ip-10-0-131-93 kubenswrapper[2564]: I0729 19:30:02.782950 2564 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="96579673-c9a2-42bd-8159-7329efcf6772" path="/var/lib/kubelet/pods/96579673-c9a2-42bd-8159-7329efcf6772/volumes" Jul 29 19:30:31.944107 ip-10-0-131-93 kubenswrapper[2564]: I0729 19:30:31.944024 2564 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-444jx_faa8c546-c75b-4e10-93b4-661aa2b5f53e/konnectivity-agent/0.log" limit=1024 Jul 29 19:30:42.541282 ip-10-0-131-93 kubenswrapper[2564]: I0729 19:30:42.541247 2564 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-444jx_faa8c546-c75b-4e10-93b4-661aa2b5f53e/konnectivity-agent/0.log" limit=1024 Jul 29 19:31:02.942548 ip-10-0-131-93 kubenswrapper[2564]: I0729 19:31:02.942516 2564 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["opendatahub/maas-controller-865c54c777-5m7vp"] Jul 29 19:31:02.942933 ip-10-0-131-93 kubenswrapper[2564]: I0729 19:31:02.942833 2564 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="96579673-c9a2-42bd-8159-7329efcf6772" containerName="manager" Jul 29 19:31:02.942933 ip-10-0-131-93 kubenswrapper[2564]: I0729 19:31:02.942844 2564 state_mem.go:107] "Deleted CPUSet assignment" podUID="96579673-c9a2-42bd-8159-7329efcf6772" containerName="manager" Jul 29 19:31:02.942933 ip-10-0-131-93 kubenswrapper[2564]: I0729 19:31:02.942905 2564 memory_manager.go:368] "RemoveStaleState removing state" podUID="96579673-c9a2-42bd-8159-7329efcf6772" containerName="manager" Jul 29 19:31:02.944813 ip-10-0-131-93 kubenswrapper[2564]: I0729 19:31:02.944797 2564 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="opendatahub/maas-controller-865c54c777-5m7vp" Jul 29 19:31:02.947796 ip-10-0-131-93 kubenswrapper[2564]: I0729 19:31:02.947775 2564 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"opendatahub\"/\"maas-parameters\"" Jul 29 19:31:02.948636 ip-10-0-131-93 kubenswrapper[2564]: I0729 19:31:02.948618 2564 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"opendatahub\"/\"maas-controller-dockercfg-d6zg6\"" Jul 29 19:31:02.948715 ip-10-0-131-93 kubenswrapper[2564]: I0729 19:31:02.948655 2564 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"opendatahub\"/\"maas-controller-webhook-cert\"" Jul 29 19:31:02.958026 ip-10-0-131-93 kubenswrapper[2564]: I0729 19:31:02.958004 2564 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["opendatahub/maas-controller-865c54c777-5m7vp"] Jul 29 19:31:03.020639 ip-10-0-131-93 kubenswrapper[2564]: I0729 19:31:03.020603 2564 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"webhook-cert\" (UniqueName: \"kubernetes.io/secret/13774296-611c-41f9-86d7-71f894b44465-webhook-cert\") pod \"maas-controller-865c54c777-5m7vp\" (UID: \"13774296-611c-41f9-86d7-71f894b44465\") " pod="opendatahub/maas-controller-865c54c777-5m7vp" Jul 29 19:31:03.020798 ip-10-0-131-93 kubenswrapper[2564]: I0729 19:31:03.020678 2564 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-bmzsf\" (UniqueName: \"kubernetes.io/projected/13774296-611c-41f9-86d7-71f894b44465-kube-api-access-bmzsf\") pod \"maas-controller-865c54c777-5m7vp\" (UID: \"13774296-611c-41f9-86d7-71f894b44465\") " pod="opendatahub/maas-controller-865c54c777-5m7vp" Jul 29 19:31:03.121092 ip-10-0-131-93 kubenswrapper[2564]: I0729 19:31:03.121060 2564 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"webhook-cert\" (UniqueName: \"kubernetes.io/secret/13774296-611c-41f9-86d7-71f894b44465-webhook-cert\") pod \"maas-controller-865c54c777-5m7vp\" (UID: \"13774296-611c-41f9-86d7-71f894b44465\") " pod="opendatahub/maas-controller-865c54c777-5m7vp" Jul 29 19:31:03.121264 ip-10-0-131-93 kubenswrapper[2564]: I0729 19:31:03.121120 2564 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-bmzsf\" (UniqueName: \"kubernetes.io/projected/13774296-611c-41f9-86d7-71f894b44465-kube-api-access-bmzsf\") pod \"maas-controller-865c54c777-5m7vp\" (UID: \"13774296-611c-41f9-86d7-71f894b44465\") " pod="opendatahub/maas-controller-865c54c777-5m7vp" Jul 29 19:31:03.123610 ip-10-0-131-93 kubenswrapper[2564]: I0729 19:31:03.123585 2564 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"webhook-cert\" (UniqueName: \"kubernetes.io/secret/13774296-611c-41f9-86d7-71f894b44465-webhook-cert\") pod \"maas-controller-865c54c777-5m7vp\" (UID: \"13774296-611c-41f9-86d7-71f894b44465\") " pod="opendatahub/maas-controller-865c54c777-5m7vp" Jul 29 19:31:03.130185 ip-10-0-131-93 kubenswrapper[2564]: I0729 19:31:03.130161 2564 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-bmzsf\" (UniqueName: \"kubernetes.io/projected/13774296-611c-41f9-86d7-71f894b44465-kube-api-access-bmzsf\") pod \"maas-controller-865c54c777-5m7vp\" (UID: \"13774296-611c-41f9-86d7-71f894b44465\") " pod="opendatahub/maas-controller-865c54c777-5m7vp" Jul 29 19:31:03.255335 ip-10-0-131-93 kubenswrapper[2564]: I0729 19:31:03.255244 2564 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="opendatahub/maas-controller-865c54c777-5m7vp" Jul 29 19:31:03.383945 ip-10-0-131-93 kubenswrapper[2564]: I0729 19:31:03.383920 2564 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["opendatahub/maas-controller-865c54c777-5m7vp"] Jul 29 19:31:03.385437 ip-10-0-131-93 kubenswrapper[2564]: W0729 19:31:03.385409 2564 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod13774296_611c_41f9_86d7_71f894b44465.slice/crio-141915d4696995448e1885cdc4500f2a88d9aea6ea6570fe0d9aac9f5a1179ad WatchSource:0}: Error finding container 141915d4696995448e1885cdc4500f2a88d9aea6ea6570fe0d9aac9f5a1179ad: Status 404 returned error can't find the container with id 141915d4696995448e1885cdc4500f2a88d9aea6ea6570fe0d9aac9f5a1179ad Jul 29 19:31:04.323875 ip-10-0-131-93 kubenswrapper[2564]: I0729 19:31:04.323839 2564 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="opendatahub/maas-controller-865c54c777-5m7vp" event={"ID":"13774296-611c-41f9-86d7-71f894b44465","Type":"ContainerStarted","Data":"dad217ef9c79ddce1f8f272fc62840ecacfe0916e4bbf8c672058658bbc1435f"} Jul 29 19:31:04.324421 ip-10-0-131-93 kubenswrapper[2564]: I0729 19:31:04.323891 2564 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="opendatahub/maas-controller-865c54c777-5m7vp" event={"ID":"13774296-611c-41f9-86d7-71f894b44465","Type":"ContainerStarted","Data":"141915d4696995448e1885cdc4500f2a88d9aea6ea6570fe0d9aac9f5a1179ad"} Jul 29 19:31:04.324421 ip-10-0-131-93 kubenswrapper[2564]: I0729 19:31:04.323988 2564 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="opendatahub/maas-controller-865c54c777-5m7vp" Jul 29 19:31:04.343521 ip-10-0-131-93 kubenswrapper[2564]: I0729 19:31:04.343479 2564 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="opendatahub/maas-controller-865c54c777-5m7vp" podStartSLOduration=1.9368278399999999 podStartE2EDuration="2.343464886s" totalImagesPullingTime="406.637046ms" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-29 19:31:02 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-29 19:31:04.341444106 +0000 UTC m=+840.065503825" watchObservedRunningTime="2026-07-29 19:31:04.343464886 +0000 UTC m=+840.067524577" Jul 29 19:31:15.330952 ip-10-0-131-93 kubenswrapper[2564]: I0729 19:31:15.330922 2564 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="opendatahub/maas-controller-865c54c777-5m7vp" Jul 29 19:31:19.859213 ip-10-0-131-93 kubenswrapper[2564]: I0729 19:31:19.859179 2564 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-444jx_faa8c546-c75b-4e10-93b4-661aa2b5f53e/konnectivity-agent/0.log" limit=1024 Jul 29 19:31:19.953330 ip-10-0-131-93 kubenswrapper[2564]: I0729 19:31:19.953298 2564 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["opendatahub/maas-controller-865c54c777-5m7vp"] Jul 29 19:31:19.953528 ip-10-0-131-93 kubenswrapper[2564]: I0729 19:31:19.953504 2564 kuberuntime_container.go:873] "Killing container with a grace period" pod="opendatahub/maas-controller-865c54c777-5m7vp" podUID="13774296-611c-41f9-86d7-71f894b44465" containerName="manager" containerID="cri-o://dad217ef9c79ddce1f8f272fc62840ecacfe0916e4bbf8c672058658bbc1435f" gracePeriod=30 Jul 29 19:31:20.200175 ip-10-0-131-93 kubenswrapper[2564]: I0729 19:31:20.200150 2564 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="opendatahub/maas-controller-865c54c777-5m7vp" Jul 29 19:31:20.368656 ip-10-0-131-93 kubenswrapper[2564]: I0729 19:31:20.368576 2564 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"webhook-cert\" (UniqueName: \"kubernetes.io/secret/13774296-611c-41f9-86d7-71f894b44465-webhook-cert\") pod \"13774296-611c-41f9-86d7-71f894b44465\" (UID: \"13774296-611c-41f9-86d7-71f894b44465\") " Jul 29 19:31:20.368656 ip-10-0-131-93 kubenswrapper[2564]: I0729 19:31:20.368609 2564 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-bmzsf\" (UniqueName: \"kubernetes.io/projected/13774296-611c-41f9-86d7-71f894b44465-kube-api-access-bmzsf\") pod \"13774296-611c-41f9-86d7-71f894b44465\" (UID: \"13774296-611c-41f9-86d7-71f894b44465\") " Jul 29 19:31:20.370746 ip-10-0-131-93 kubenswrapper[2564]: I0729 19:31:20.370717 2564 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/13774296-611c-41f9-86d7-71f894b44465-webhook-cert" (OuterVolumeSpecName: "webhook-cert") pod "13774296-611c-41f9-86d7-71f894b44465" (UID: "13774296-611c-41f9-86d7-71f894b44465"). InnerVolumeSpecName "webhook-cert". PluginName "kubernetes.io/secret", VolumeGIDValue "" Jul 29 19:31:20.370870 ip-10-0-131-93 kubenswrapper[2564]: I0729 19:31:20.370782 2564 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/13774296-611c-41f9-86d7-71f894b44465-kube-api-access-bmzsf" (OuterVolumeSpecName: "kube-api-access-bmzsf") pod "13774296-611c-41f9-86d7-71f894b44465" (UID: "13774296-611c-41f9-86d7-71f894b44465"). InnerVolumeSpecName "kube-api-access-bmzsf". PluginName "kubernetes.io/projected", VolumeGIDValue "" Jul 29 19:31:20.400981 ip-10-0-131-93 kubenswrapper[2564]: I0729 19:31:20.400954 2564 generic.go:356] "Generic (PLEG): container finished" podID="13774296-611c-41f9-86d7-71f894b44465" containerID="dad217ef9c79ddce1f8f272fc62840ecacfe0916e4bbf8c672058658bbc1435f" exitCode=0 Jul 29 19:31:20.401108 ip-10-0-131-93 kubenswrapper[2564]: I0729 19:31:20.400998 2564 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="opendatahub/maas-controller-865c54c777-5m7vp" Jul 29 19:31:20.401108 ip-10-0-131-93 kubenswrapper[2564]: I0729 19:31:20.401033 2564 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="opendatahub/maas-controller-865c54c777-5m7vp" event={"ID":"13774296-611c-41f9-86d7-71f894b44465","Type":"ContainerDied","Data":"dad217ef9c79ddce1f8f272fc62840ecacfe0916e4bbf8c672058658bbc1435f"} Jul 29 19:31:20.401108 ip-10-0-131-93 kubenswrapper[2564]: I0729 19:31:20.401086 2564 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="opendatahub/maas-controller-865c54c777-5m7vp" event={"ID":"13774296-611c-41f9-86d7-71f894b44465","Type":"ContainerDied","Data":"141915d4696995448e1885cdc4500f2a88d9aea6ea6570fe0d9aac9f5a1179ad"} Jul 29 19:31:20.401108 ip-10-0-131-93 kubenswrapper[2564]: I0729 19:31:20.401109 2564 scope.go:117] "RemoveContainer" containerID="dad217ef9c79ddce1f8f272fc62840ecacfe0916e4bbf8c672058658bbc1435f" Jul 29 19:31:20.409470 ip-10-0-131-93 kubenswrapper[2564]: I0729 19:31:20.409425 2564 scope.go:117] "RemoveContainer" containerID="dad217ef9c79ddce1f8f272fc62840ecacfe0916e4bbf8c672058658bbc1435f" Jul 29 19:31:20.409690 ip-10-0-131-93 kubenswrapper[2564]: E0729 19:31:20.409669 2564 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"dad217ef9c79ddce1f8f272fc62840ecacfe0916e4bbf8c672058658bbc1435f\": container with ID starting with dad217ef9c79ddce1f8f272fc62840ecacfe0916e4bbf8c672058658bbc1435f not found: ID does not exist" containerID="dad217ef9c79ddce1f8f272fc62840ecacfe0916e4bbf8c672058658bbc1435f" Jul 29 19:31:20.409732 ip-10-0-131-93 kubenswrapper[2564]: I0729 19:31:20.409704 2564 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"dad217ef9c79ddce1f8f272fc62840ecacfe0916e4bbf8c672058658bbc1435f"} err="failed to get container status \"dad217ef9c79ddce1f8f272fc62840ecacfe0916e4bbf8c672058658bbc1435f\": rpc error: code = NotFound desc = could not find container \"dad217ef9c79ddce1f8f272fc62840ecacfe0916e4bbf8c672058658bbc1435f\": container with ID starting with dad217ef9c79ddce1f8f272fc62840ecacfe0916e4bbf8c672058658bbc1435f not found: ID does not exist" Jul 29 19:31:20.422269 ip-10-0-131-93 kubenswrapper[2564]: I0729 19:31:20.422246 2564 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["opendatahub/maas-controller-865c54c777-5m7vp"] Jul 29 19:31:20.428927 ip-10-0-131-93 kubenswrapper[2564]: I0729 19:31:20.428906 2564 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["opendatahub/maas-controller-865c54c777-5m7vp"] Jul 29 19:31:20.469314 ip-10-0-131-93 kubenswrapper[2564]: I0729 19:31:20.469292 2564 reconciler_common.go:299] "Volume detached for volume \"webhook-cert\" (UniqueName: \"kubernetes.io/secret/13774296-611c-41f9-86d7-71f894b44465-webhook-cert\") on node \"ip-10-0-131-93.ec2.internal\" DevicePath \"\"" Jul 29 19:31:20.469314 ip-10-0-131-93 kubenswrapper[2564]: I0729 19:31:20.469312 2564 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-bmzsf\" (UniqueName: \"kubernetes.io/projected/13774296-611c-41f9-86d7-71f894b44465-kube-api-access-bmzsf\") on node \"ip-10-0-131-93.ec2.internal\" DevicePath \"\"" Jul 29 19:31:20.787268 ip-10-0-131-93 kubenswrapper[2564]: I0729 19:31:20.787226 2564 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="13774296-611c-41f9-86d7-71f894b44465" path="/var/lib/kubelet/pods/13774296-611c-41f9-86d7-71f894b44465/volumes" Jul 29 19:31:21.208012 ip-10-0-131-93 kubenswrapper[2564]: I0729 19:31:21.207976 2564 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["opendatahub/maas-controller-865c54c777-mdwcz"] Jul 29 19:31:21.208403 ip-10-0-131-93 kubenswrapper[2564]: I0729 19:31:21.208296 2564 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="13774296-611c-41f9-86d7-71f894b44465" containerName="manager" Jul 29 19:31:21.208403 ip-10-0-131-93 kubenswrapper[2564]: I0729 19:31:21.208311 2564 state_mem.go:107] "Deleted CPUSet assignment" podUID="13774296-611c-41f9-86d7-71f894b44465" containerName="manager" Jul 29 19:31:21.208496 ip-10-0-131-93 kubenswrapper[2564]: I0729 19:31:21.208410 2564 memory_manager.go:368] "RemoveStaleState removing state" podUID="13774296-611c-41f9-86d7-71f894b44465" containerName="manager" Jul 29 19:31:21.211566 ip-10-0-131-93 kubenswrapper[2564]: I0729 19:31:21.211547 2564 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="opendatahub/maas-controller-865c54c777-mdwcz" Jul 29 19:31:21.215368 ip-10-0-131-93 kubenswrapper[2564]: I0729 19:31:21.215347 2564 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"opendatahub\"/\"maas-controller-dockercfg-d6zg6\"" Jul 29 19:31:21.215479 ip-10-0-131-93 kubenswrapper[2564]: I0729 19:31:21.215391 2564 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"opendatahub\"/\"maas-parameters\"" Jul 29 19:31:21.215479 ip-10-0-131-93 kubenswrapper[2564]: I0729 19:31:21.215413 2564 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"opendatahub\"/\"maas-controller-webhook-cert\"" Jul 29 19:31:21.221992 ip-10-0-131-93 kubenswrapper[2564]: I0729 19:31:21.221969 2564 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["opendatahub/maas-controller-865c54c777-mdwcz"] Jul 29 19:31:21.267218 ip-10-0-131-93 kubenswrapper[2564]: I0729 19:31:21.267184 2564 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-444jx_faa8c546-c75b-4e10-93b4-661aa2b5f53e/konnectivity-agent/0.log" limit=1024 Jul 29 19:31:21.276464 ip-10-0-131-93 kubenswrapper[2564]: I0729 19:31:21.276436 2564 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-p7mtp\" (UniqueName: \"kubernetes.io/projected/a624f438-2b48-4e07-ae3f-aa7b3cd3d6a8-kube-api-access-p7mtp\") pod \"maas-controller-865c54c777-mdwcz\" (UID: \"a624f438-2b48-4e07-ae3f-aa7b3cd3d6a8\") " pod="opendatahub/maas-controller-865c54c777-mdwcz" Jul 29 19:31:21.276611 ip-10-0-131-93 kubenswrapper[2564]: I0729 19:31:21.276546 2564 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"webhook-cert\" (UniqueName: \"kubernetes.io/secret/a624f438-2b48-4e07-ae3f-aa7b3cd3d6a8-webhook-cert\") pod \"maas-controller-865c54c777-mdwcz\" (UID: \"a624f438-2b48-4e07-ae3f-aa7b3cd3d6a8\") " pod="opendatahub/maas-controller-865c54c777-mdwcz" Jul 29 19:31:21.377260 ip-10-0-131-93 kubenswrapper[2564]: I0729 19:31:21.377220 2564 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"webhook-cert\" (UniqueName: \"kubernetes.io/secret/a624f438-2b48-4e07-ae3f-aa7b3cd3d6a8-webhook-cert\") pod \"maas-controller-865c54c777-mdwcz\" (UID: \"a624f438-2b48-4e07-ae3f-aa7b3cd3d6a8\") " pod="opendatahub/maas-controller-865c54c777-mdwcz" Jul 29 19:31:21.377260 ip-10-0-131-93 kubenswrapper[2564]: I0729 19:31:21.377265 2564 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-p7mtp\" (UniqueName: \"kubernetes.io/projected/a624f438-2b48-4e07-ae3f-aa7b3cd3d6a8-kube-api-access-p7mtp\") pod \"maas-controller-865c54c777-mdwcz\" (UID: \"a624f438-2b48-4e07-ae3f-aa7b3cd3d6a8\") " pod="opendatahub/maas-controller-865c54c777-mdwcz" Jul 29 19:31:21.379655 ip-10-0-131-93 kubenswrapper[2564]: I0729 19:31:21.379624 2564 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"webhook-cert\" (UniqueName: \"kubernetes.io/secret/a624f438-2b48-4e07-ae3f-aa7b3cd3d6a8-webhook-cert\") pod \"maas-controller-865c54c777-mdwcz\" (UID: \"a624f438-2b48-4e07-ae3f-aa7b3cd3d6a8\") " pod="opendatahub/maas-controller-865c54c777-mdwcz" Jul 29 19:31:21.385769 ip-10-0-131-93 kubenswrapper[2564]: I0729 19:31:21.385743 2564 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-p7mtp\" (UniqueName: \"kubernetes.io/projected/a624f438-2b48-4e07-ae3f-aa7b3cd3d6a8-kube-api-access-p7mtp\") pod \"maas-controller-865c54c777-mdwcz\" (UID: \"a624f438-2b48-4e07-ae3f-aa7b3cd3d6a8\") " pod="opendatahub/maas-controller-865c54c777-mdwcz" Jul 29 19:31:21.522351 ip-10-0-131-93 kubenswrapper[2564]: I0729 19:31:21.522264 2564 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="opendatahub/maas-controller-865c54c777-mdwcz" Jul 29 19:31:21.648330 ip-10-0-131-93 kubenswrapper[2564]: I0729 19:31:21.648165 2564 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["opendatahub/maas-controller-865c54c777-mdwcz"] Jul 29 19:31:21.650951 ip-10-0-131-93 kubenswrapper[2564]: W0729 19:31:21.650922 2564 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-poda624f438_2b48_4e07_ae3f_aa7b3cd3d6a8.slice/crio-c28c2d43bc98af903cfa547fc26657e680507c17853ed7c25bed4778d718e0c1 WatchSource:0}: Error finding container c28c2d43bc98af903cfa547fc26657e680507c17853ed7c25bed4778d718e0c1: Status 404 returned error can't find the container with id c28c2d43bc98af903cfa547fc26657e680507c17853ed7c25bed4778d718e0c1 Jul 29 19:31:22.413548 ip-10-0-131-93 kubenswrapper[2564]: I0729 19:31:22.413475 2564 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="opendatahub/maas-controller-865c54c777-mdwcz" event={"ID":"a624f438-2b48-4e07-ae3f-aa7b3cd3d6a8","Type":"ContainerStarted","Data":"d1e17883d74a32d45c8d2e461a1150b22a5743b499d433f699cd2ecf7f286a00"} Jul 29 19:31:22.413548 ip-10-0-131-93 kubenswrapper[2564]: I0729 19:31:22.413525 2564 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="opendatahub/maas-controller-865c54c777-mdwcz" event={"ID":"a624f438-2b48-4e07-ae3f-aa7b3cd3d6a8","Type":"ContainerStarted","Data":"c28c2d43bc98af903cfa547fc26657e680507c17853ed7c25bed4778d718e0c1"} Jul 29 19:31:22.413928 ip-10-0-131-93 kubenswrapper[2564]: I0729 19:31:22.413582 2564 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="opendatahub/maas-controller-865c54c777-mdwcz" Jul 29 19:31:22.433174 ip-10-0-131-93 kubenswrapper[2564]: I0729 19:31:22.433116 2564 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="opendatahub/maas-controller-865c54c777-mdwcz" podStartSLOduration=1.012482757 podStartE2EDuration="1.433097337s" totalImagesPullingTime="420.61458ms" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-29 19:31:21 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-29 19:31:22.430987328 +0000 UTC m=+858.155047022" watchObservedRunningTime="2026-07-29 19:31:22.433097337 +0000 UTC m=+858.157157031" Jul 29 19:31:33.421165 ip-10-0-131-93 kubenswrapper[2564]: I0729 19:31:33.421135 2564 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="opendatahub/maas-controller-865c54c777-mdwcz" Jul 29 19:32:00.608139 ip-10-0-131-93 kubenswrapper[2564]: I0729 19:32:00.608064 2564 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-444jx_faa8c546-c75b-4e10-93b4-661aa2b5f53e/konnectivity-agent/0.log" limit=1024 Jul 29 19:32:00.706402 ip-10-0-131-93 kubenswrapper[2564]: I0729 19:32:00.706356 2564 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-444jx_faa8c546-c75b-4e10-93b4-661aa2b5f53e/konnectivity-agent/0.log" limit=1024 Jul 29 19:32:00.809992 ip-10-0-131-93 kubenswrapper[2564]: I0729 19:32:00.809958 2564 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-444jx_faa8c546-c75b-4e10-93b4-661aa2b5f53e/konnectivity-agent/0.log" limit=1024 Jul 29 19:32:06.564644 ip-10-0-131-93 kubenswrapper[2564]: I0729 19:32:06.564610 2564 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-444jx_faa8c546-c75b-4e10-93b4-661aa2b5f53e/konnectivity-agent/0.log" limit=1024 Jul 29 19:32:06.665392 ip-10-0-131-93 kubenswrapper[2564]: I0729 19:32:06.665346 2564 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-444jx_faa8c546-c75b-4e10-93b4-661aa2b5f53e/konnectivity-agent/0.log" limit=1024 Jul 29 19:32:06.767222 ip-10-0-131-93 kubenswrapper[2564]: I0729 19:32:06.767187 2564 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-444jx_faa8c546-c75b-4e10-93b4-661aa2b5f53e/konnectivity-agent/0.log" limit=1024 Jul 29 19:32:06.916201 ip-10-0-131-93 kubenswrapper[2564]: I0729 19:32:06.916097 2564 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-444jx_faa8c546-c75b-4e10-93b4-661aa2b5f53e/konnectivity-agent/0.log" limit=1024 Jul 29 19:32:12.850851 ip-10-0-131-93 kubenswrapper[2564]: I0729 19:32:12.850819 2564 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-444jx_faa8c546-c75b-4e10-93b4-661aa2b5f53e/konnectivity-agent/0.log" limit=1024 Jul 29 19:32:12.921461 ip-10-0-131-93 kubenswrapper[2564]: I0729 19:32:12.921430 2564 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-444jx_faa8c546-c75b-4e10-93b4-661aa2b5f53e/konnectivity-agent/0.log" limit=1024 Jul 29 19:35:29.362324 ip-10-0-131-93 kubenswrapper[2564]: I0729 19:35:29.362289 2564 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-444jx_faa8c546-c75b-4e10-93b4-661aa2b5f53e/konnectivity-agent/0.log" limit=1024 Jul 29 19:36:36.397346 ip-10-0-131-93 kubenswrapper[2564]: I0729 19:36:36.397307 2564 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-444jx_faa8c546-c75b-4e10-93b4-661aa2b5f53e/konnectivity-agent/0.log" limit=1024 Jul 29 19:36:36.496321 ip-10-0-131-93 kubenswrapper[2564]: I0729 19:36:36.496290 2564 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-444jx_faa8c546-c75b-4e10-93b4-661aa2b5f53e/konnectivity-agent/0.log" limit=1024 Jul 29 19:37:02.408072 ip-10-0-131-93 kubenswrapper[2564]: I0729 19:37:02.408017 2564 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-444jx_faa8c546-c75b-4e10-93b4-661aa2b5f53e/konnectivity-agent/0.log" limit=1024 Jul 29 19:37:32.612206 ip-10-0-131-93 kubenswrapper[2564]: I0729 19:37:32.612130 2564 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-444jx_faa8c546-c75b-4e10-93b4-661aa2b5f53e/konnectivity-agent/0.log" limit=1024 Jul 29 19:40:17.896546 ip-10-0-131-93 kubenswrapper[2564]: I0729 19:40:17.896516 2564 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-444jx_faa8c546-c75b-4e10-93b4-661aa2b5f53e/konnectivity-agent/0.log" limit=1024 Jul 29 19:40:17.961001 ip-10-0-131-93 kubenswrapper[2564]: I0729 19:40:17.960961 2564 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-444jx_faa8c546-c75b-4e10-93b4-661aa2b5f53e/konnectivity-agent/0.log" limit=1024 Jul 29 19:40:46.996190 ip-10-0-131-93 kubenswrapper[2564]: I0729 19:40:46.996154 2564 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-444jx_faa8c546-c75b-4e10-93b4-661aa2b5f53e/konnectivity-agent/0.log" limit=1024 Jul 29 19:40:56.679204 ip-10-0-131-93 kubenswrapper[2564]: I0729 19:40:56.679125 2564 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-444jx_faa8c546-c75b-4e10-93b4-661aa2b5f53e/konnectivity-agent/0.log" limit=1024 Jul 29 19:41:27.324771 ip-10-0-131-93 kubenswrapper[2564]: I0729 19:41:27.324734 2564 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-444jx_faa8c546-c75b-4e10-93b4-661aa2b5f53e/konnectivity-agent/0.log" limit=1024 Jul 29 19:41:28.424646 ip-10-0-131-93 kubenswrapper[2564]: I0729 19:41:28.424606 2564 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-444jx_faa8c546-c75b-4e10-93b4-661aa2b5f53e/konnectivity-agent/0.log" limit=1024 Jul 29 19:44:29.741760 ip-10-0-131-93 kubenswrapper[2564]: I0729 19:44:29.741728 2564 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-444jx_faa8c546-c75b-4e10-93b4-661aa2b5f53e/konnectivity-agent/0.log" limit=1024 Jul 29 19:44:29.826042 ip-10-0-131-93 kubenswrapper[2564]: I0729 19:44:29.826008 2564 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-444jx_faa8c546-c75b-4e10-93b4-661aa2b5f53e/konnectivity-agent/0.log" limit=1024 Jul 29 19:44:40.943223 ip-10-0-131-93 kubenswrapper[2564]: I0729 19:44:40.943178 2564 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-444jx_faa8c546-c75b-4e10-93b4-661aa2b5f53e/konnectivity-agent/0.log" limit=1024 Jul 29 19:45:02.642189 ip-10-0-131-93 kubenswrapper[2564]: I0729 19:45:02.642158 2564 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-444jx_faa8c546-c75b-4e10-93b4-661aa2b5f53e/konnectivity-agent/0.log" limit=1024 Jul 29 19:45:05.566855 ip-10-0-131-93 kubenswrapper[2564]: I0729 19:45:05.566769 2564 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-444jx_faa8c546-c75b-4e10-93b4-661aa2b5f53e/konnectivity-agent/0.log" limit=1024 Jul 29 19:45:05.658132 ip-10-0-131-93 kubenswrapper[2564]: I0729 19:45:05.658101 2564 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-444jx_faa8c546-c75b-4e10-93b4-661aa2b5f53e/konnectivity-agent/0.log" limit=1024 Jul 29 19:45:05.988218 ip-10-0-131-93 kubenswrapper[2564]: I0729 19:45:05.988169 2564 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-444jx_faa8c546-c75b-4e10-93b4-661aa2b5f53e/konnectivity-agent/0.log" limit=1024 Jul 29 19:45:12.699574 ip-10-0-131-93 kubenswrapper[2564]: I0729 19:45:12.699539 2564 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-444jx_faa8c546-c75b-4e10-93b4-661aa2b5f53e/konnectivity-agent/0.log" limit=1024 Jul 29 19:45:12.797929 ip-10-0-131-93 kubenswrapper[2564]: I0729 19:45:12.797892 2564 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-444jx_faa8c546-c75b-4e10-93b4-661aa2b5f53e/konnectivity-agent/0.log" limit=1024 Jul 29 19:45:13.099661 ip-10-0-131-93 kubenswrapper[2564]: I0729 19:45:13.099569 2564 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-444jx_faa8c546-c75b-4e10-93b4-661aa2b5f53e/konnectivity-agent/0.log" limit=1024 Jul 29 19:45:13.277263 ip-10-0-131-93 kubenswrapper[2564]: I0729 19:45:13.277228 2564 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-ingress/e2e-ait-d0aaa6d0-openshift-default-7cb489fc6b-s5n62"] Jul 29 19:45:13.280356 ip-10-0-131-93 kubenswrapper[2564]: I0729 19:45:13.280341 2564 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-ait-d0aaa6d0-openshift-default-7cb489fc6b-s5n62" Jul 29 19:45:13.283129 ip-10-0-131-93 kubenswrapper[2564]: I0729 19:45:13.283111 2564 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress\"/\"e2e-ait-d0aaa6d0-openshift-default-dockercfg-8lqz4\"" Jul 29 19:45:13.296074 ip-10-0-131-93 kubenswrapper[2564]: I0729 19:45:13.296051 2564 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/e2e-ait-d0aaa6d0-openshift-default-7cb489fc6b-s5n62"] Jul 29 19:45:13.304707 ip-10-0-131-93 kubenswrapper[2564]: I0729 19:45:13.304685 2564 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-444jx_faa8c546-c75b-4e10-93b4-661aa2b5f53e/konnectivity-agent/0.log" limit=1024 Jul 29 19:45:13.410784 ip-10-0-131-93 kubenswrapper[2564]: I0729 19:45:13.410700 2564 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/c44b5d07-ffb1-4f5e-a949-38d459fecd10-credential-socket\") pod \"e2e-ait-d0aaa6d0-openshift-default-7cb489fc6b-s5n62\" (UID: \"c44b5d07-ffb1-4f5e-a949-38d459fecd10\") " pod="openshift-ingress/e2e-ait-d0aaa6d0-openshift-default-7cb489fc6b-s5n62" Jul 29 19:45:13.410784 ip-10-0-131-93 kubenswrapper[2564]: I0729 19:45:13.410742 2564 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-48z98\" (UniqueName: \"kubernetes.io/projected/c44b5d07-ffb1-4f5e-a949-38d459fecd10-kube-api-access-48z98\") pod \"e2e-ait-d0aaa6d0-openshift-default-7cb489fc6b-s5n62\" (UID: \"c44b5d07-ffb1-4f5e-a949-38d459fecd10\") " pod="openshift-ingress/e2e-ait-d0aaa6d0-openshift-default-7cb489fc6b-s5n62" Jul 29 19:45:13.410784 ip-10-0-131-93 kubenswrapper[2564]: I0729 19:45:13.410766 2564 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/c44b5d07-ffb1-4f5e-a949-38d459fecd10-istio-data\") pod \"e2e-ait-d0aaa6d0-openshift-default-7cb489fc6b-s5n62\" (UID: \"c44b5d07-ffb1-4f5e-a949-38d459fecd10\") " pod="openshift-ingress/e2e-ait-d0aaa6d0-openshift-default-7cb489fc6b-s5n62" Jul 29 19:45:13.411013 ip-10-0-131-93 kubenswrapper[2564]: I0729 19:45:13.410821 2564 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/c44b5d07-ffb1-4f5e-a949-38d459fecd10-istio-podinfo\") pod \"e2e-ait-d0aaa6d0-openshift-default-7cb489fc6b-s5n62\" (UID: \"c44b5d07-ffb1-4f5e-a949-38d459fecd10\") " pod="openshift-ingress/e2e-ait-d0aaa6d0-openshift-default-7cb489fc6b-s5n62" Jul 29 19:45:13.411013 ip-10-0-131-93 kubenswrapper[2564]: I0729 19:45:13.410862 2564 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/c44b5d07-ffb1-4f5e-a949-38d459fecd10-workload-socket\") pod \"e2e-ait-d0aaa6d0-openshift-default-7cb489fc6b-s5n62\" (UID: \"c44b5d07-ffb1-4f5e-a949-38d459fecd10\") " pod="openshift-ingress/e2e-ait-d0aaa6d0-openshift-default-7cb489fc6b-s5n62" Jul 29 19:45:13.411013 ip-10-0-131-93 kubenswrapper[2564]: I0729 19:45:13.410915 2564 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/c44b5d07-ffb1-4f5e-a949-38d459fecd10-istio-envoy\") pod \"e2e-ait-d0aaa6d0-openshift-default-7cb489fc6b-s5n62\" (UID: \"c44b5d07-ffb1-4f5e-a949-38d459fecd10\") " pod="openshift-ingress/e2e-ait-d0aaa6d0-openshift-default-7cb489fc6b-s5n62" Jul 29 19:45:13.411013 ip-10-0-131-93 kubenswrapper[2564]: I0729 19:45:13.410932 2564 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/c44b5d07-ffb1-4f5e-a949-38d459fecd10-istiod-ca-cert\") pod \"e2e-ait-d0aaa6d0-openshift-default-7cb489fc6b-s5n62\" (UID: \"c44b5d07-ffb1-4f5e-a949-38d459fecd10\") " pod="openshift-ingress/e2e-ait-d0aaa6d0-openshift-default-7cb489fc6b-s5n62" Jul 29 19:45:13.411013 ip-10-0-131-93 kubenswrapper[2564]: I0729 19:45:13.410954 2564 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/c44b5d07-ffb1-4f5e-a949-38d459fecd10-workload-certs\") pod \"e2e-ait-d0aaa6d0-openshift-default-7cb489fc6b-s5n62\" (UID: \"c44b5d07-ffb1-4f5e-a949-38d459fecd10\") " pod="openshift-ingress/e2e-ait-d0aaa6d0-openshift-default-7cb489fc6b-s5n62" Jul 29 19:45:13.411013 ip-10-0-131-93 kubenswrapper[2564]: I0729 19:45:13.410968 2564 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/c44b5d07-ffb1-4f5e-a949-38d459fecd10-istio-token\") pod \"e2e-ait-d0aaa6d0-openshift-default-7cb489fc6b-s5n62\" (UID: \"c44b5d07-ffb1-4f5e-a949-38d459fecd10\") " pod="openshift-ingress/e2e-ait-d0aaa6d0-openshift-default-7cb489fc6b-s5n62" Jul 29 19:45:13.512188 ip-10-0-131-93 kubenswrapper[2564]: I0729 19:45:13.512152 2564 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/c44b5d07-ffb1-4f5e-a949-38d459fecd10-credential-socket\") pod \"e2e-ait-d0aaa6d0-openshift-default-7cb489fc6b-s5n62\" (UID: \"c44b5d07-ffb1-4f5e-a949-38d459fecd10\") " pod="openshift-ingress/e2e-ait-d0aaa6d0-openshift-default-7cb489fc6b-s5n62" Jul 29 19:45:13.512188 ip-10-0-131-93 kubenswrapper[2564]: I0729 19:45:13.512191 2564 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-48z98\" (UniqueName: \"kubernetes.io/projected/c44b5d07-ffb1-4f5e-a949-38d459fecd10-kube-api-access-48z98\") pod \"e2e-ait-d0aaa6d0-openshift-default-7cb489fc6b-s5n62\" (UID: \"c44b5d07-ffb1-4f5e-a949-38d459fecd10\") " pod="openshift-ingress/e2e-ait-d0aaa6d0-openshift-default-7cb489fc6b-s5n62" Jul 29 19:45:13.512454 ip-10-0-131-93 kubenswrapper[2564]: I0729 19:45:13.512221 2564 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/c44b5d07-ffb1-4f5e-a949-38d459fecd10-istio-data\") pod \"e2e-ait-d0aaa6d0-openshift-default-7cb489fc6b-s5n62\" (UID: \"c44b5d07-ffb1-4f5e-a949-38d459fecd10\") " pod="openshift-ingress/e2e-ait-d0aaa6d0-openshift-default-7cb489fc6b-s5n62" Jul 29 19:45:13.512454 ip-10-0-131-93 kubenswrapper[2564]: I0729 19:45:13.512242 2564 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/c44b5d07-ffb1-4f5e-a949-38d459fecd10-istio-podinfo\") pod \"e2e-ait-d0aaa6d0-openshift-default-7cb489fc6b-s5n62\" (UID: \"c44b5d07-ffb1-4f5e-a949-38d459fecd10\") " pod="openshift-ingress/e2e-ait-d0aaa6d0-openshift-default-7cb489fc6b-s5n62" Jul 29 19:45:13.512454 ip-10-0-131-93 kubenswrapper[2564]: I0729 19:45:13.512265 2564 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/c44b5d07-ffb1-4f5e-a949-38d459fecd10-workload-socket\") pod \"e2e-ait-d0aaa6d0-openshift-default-7cb489fc6b-s5n62\" (UID: \"c44b5d07-ffb1-4f5e-a949-38d459fecd10\") " pod="openshift-ingress/e2e-ait-d0aaa6d0-openshift-default-7cb489fc6b-s5n62" Jul 29 19:45:13.512454 ip-10-0-131-93 kubenswrapper[2564]: I0729 19:45:13.512300 2564 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/c44b5d07-ffb1-4f5e-a949-38d459fecd10-istio-envoy\") pod \"e2e-ait-d0aaa6d0-openshift-default-7cb489fc6b-s5n62\" (UID: \"c44b5d07-ffb1-4f5e-a949-38d459fecd10\") " pod="openshift-ingress/e2e-ait-d0aaa6d0-openshift-default-7cb489fc6b-s5n62" Jul 29 19:45:13.512454 ip-10-0-131-93 kubenswrapper[2564]: I0729 19:45:13.512319 2564 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/c44b5d07-ffb1-4f5e-a949-38d459fecd10-istiod-ca-cert\") pod \"e2e-ait-d0aaa6d0-openshift-default-7cb489fc6b-s5n62\" (UID: \"c44b5d07-ffb1-4f5e-a949-38d459fecd10\") " pod="openshift-ingress/e2e-ait-d0aaa6d0-openshift-default-7cb489fc6b-s5n62" Jul 29 19:45:13.512454 ip-10-0-131-93 kubenswrapper[2564]: I0729 19:45:13.512342 2564 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/c44b5d07-ffb1-4f5e-a949-38d459fecd10-workload-certs\") pod \"e2e-ait-d0aaa6d0-openshift-default-7cb489fc6b-s5n62\" (UID: \"c44b5d07-ffb1-4f5e-a949-38d459fecd10\") " pod="openshift-ingress/e2e-ait-d0aaa6d0-openshift-default-7cb489fc6b-s5n62" Jul 29 19:45:13.512454 ip-10-0-131-93 kubenswrapper[2564]: I0729 19:45:13.512365 2564 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/c44b5d07-ffb1-4f5e-a949-38d459fecd10-istio-token\") pod \"e2e-ait-d0aaa6d0-openshift-default-7cb489fc6b-s5n62\" (UID: \"c44b5d07-ffb1-4f5e-a949-38d459fecd10\") " pod="openshift-ingress/e2e-ait-d0aaa6d0-openshift-default-7cb489fc6b-s5n62" Jul 29 19:45:13.512805 ip-10-0-131-93 kubenswrapper[2564]: I0729 19:45:13.512686 2564 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/c44b5d07-ffb1-4f5e-a949-38d459fecd10-istio-data\") pod \"e2e-ait-d0aaa6d0-openshift-default-7cb489fc6b-s5n62\" (UID: \"c44b5d07-ffb1-4f5e-a949-38d459fecd10\") " pod="openshift-ingress/e2e-ait-d0aaa6d0-openshift-default-7cb489fc6b-s5n62" Jul 29 19:45:13.512859 ip-10-0-131-93 kubenswrapper[2564]: I0729 19:45:13.512821 2564 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/c44b5d07-ffb1-4f5e-a949-38d459fecd10-credential-socket\") pod \"e2e-ait-d0aaa6d0-openshift-default-7cb489fc6b-s5n62\" (UID: \"c44b5d07-ffb1-4f5e-a949-38d459fecd10\") " pod="openshift-ingress/e2e-ait-d0aaa6d0-openshift-default-7cb489fc6b-s5n62" Jul 29 19:45:13.512926 ip-10-0-131-93 kubenswrapper[2564]: I0729 19:45:13.512900 2564 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/c44b5d07-ffb1-4f5e-a949-38d459fecd10-workload-socket\") pod \"e2e-ait-d0aaa6d0-openshift-default-7cb489fc6b-s5n62\" (UID: \"c44b5d07-ffb1-4f5e-a949-38d459fecd10\") " pod="openshift-ingress/e2e-ait-d0aaa6d0-openshift-default-7cb489fc6b-s5n62" Jul 29 19:45:13.512988 ip-10-0-131-93 kubenswrapper[2564]: I0729 19:45:13.512920 2564 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/c44b5d07-ffb1-4f5e-a949-38d459fecd10-workload-certs\") pod \"e2e-ait-d0aaa6d0-openshift-default-7cb489fc6b-s5n62\" (UID: \"c44b5d07-ffb1-4f5e-a949-38d459fecd10\") " pod="openshift-ingress/e2e-ait-d0aaa6d0-openshift-default-7cb489fc6b-s5n62" Jul 29 19:45:13.513140 ip-10-0-131-93 kubenswrapper[2564]: I0729 19:45:13.513120 2564 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/c44b5d07-ffb1-4f5e-a949-38d459fecd10-istiod-ca-cert\") pod \"e2e-ait-d0aaa6d0-openshift-default-7cb489fc6b-s5n62\" (UID: \"c44b5d07-ffb1-4f5e-a949-38d459fecd10\") " pod="openshift-ingress/e2e-ait-d0aaa6d0-openshift-default-7cb489fc6b-s5n62" Jul 29 19:45:13.514566 ip-10-0-131-93 kubenswrapper[2564]: I0729 19:45:13.514544 2564 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/c44b5d07-ffb1-4f5e-a949-38d459fecd10-istio-envoy\") pod \"e2e-ait-d0aaa6d0-openshift-default-7cb489fc6b-s5n62\" (UID: \"c44b5d07-ffb1-4f5e-a949-38d459fecd10\") " pod="openshift-ingress/e2e-ait-d0aaa6d0-openshift-default-7cb489fc6b-s5n62" Jul 29 19:45:13.514793 ip-10-0-131-93 kubenswrapper[2564]: I0729 19:45:13.514776 2564 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/c44b5d07-ffb1-4f5e-a949-38d459fecd10-istio-podinfo\") pod \"e2e-ait-d0aaa6d0-openshift-default-7cb489fc6b-s5n62\" (UID: \"c44b5d07-ffb1-4f5e-a949-38d459fecd10\") " pod="openshift-ingress/e2e-ait-d0aaa6d0-openshift-default-7cb489fc6b-s5n62" Jul 29 19:45:13.523673 ip-10-0-131-93 kubenswrapper[2564]: I0729 19:45:13.523653 2564 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-48z98\" (UniqueName: \"kubernetes.io/projected/c44b5d07-ffb1-4f5e-a949-38d459fecd10-kube-api-access-48z98\") pod \"e2e-ait-d0aaa6d0-openshift-default-7cb489fc6b-s5n62\" (UID: \"c44b5d07-ffb1-4f5e-a949-38d459fecd10\") " pod="openshift-ingress/e2e-ait-d0aaa6d0-openshift-default-7cb489fc6b-s5n62" Jul 29 19:45:13.528365 ip-10-0-131-93 kubenswrapper[2564]: I0729 19:45:13.528342 2564 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/c44b5d07-ffb1-4f5e-a949-38d459fecd10-istio-token\") pod \"e2e-ait-d0aaa6d0-openshift-default-7cb489fc6b-s5n62\" (UID: \"c44b5d07-ffb1-4f5e-a949-38d459fecd10\") " pod="openshift-ingress/e2e-ait-d0aaa6d0-openshift-default-7cb489fc6b-s5n62" Jul 29 19:45:13.541879 ip-10-0-131-93 kubenswrapper[2564]: I0729 19:45:13.541836 2564 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-444jx_faa8c546-c75b-4e10-93b4-661aa2b5f53e/konnectivity-agent/0.log" limit=1024 Jul 29 19:45:13.591029 ip-10-0-131-93 kubenswrapper[2564]: I0729 19:45:13.590995 2564 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-ait-d0aaa6d0-openshift-default-7cb489fc6b-s5n62" Jul 29 19:45:13.733302 ip-10-0-131-93 kubenswrapper[2564]: I0729 19:45:13.733279 2564 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/e2e-ait-d0aaa6d0-openshift-default-7cb489fc6b-s5n62"] Jul 29 19:45:13.735329 ip-10-0-131-93 kubenswrapper[2564]: W0729 19:45:13.735301 2564 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podc44b5d07_ffb1_4f5e_a949_38d459fecd10.slice/crio-bfb8a5691e80d2120e833953755c49c69f8630040e429155bc9f5e47567d383f WatchSource:0}: Error finding container bfb8a5691e80d2120e833953755c49c69f8630040e429155bc9f5e47567d383f: Status 404 returned error can't find the container with id bfb8a5691e80d2120e833953755c49c69f8630040e429155bc9f5e47567d383f Jul 29 19:45:13.737322 ip-10-0-131-93 kubenswrapper[2564]: I0729 19:45:13.737294 2564 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236224Ki","pods":"250"} Jul 29 19:45:13.737397 ip-10-0-131-93 kubenswrapper[2564]: I0729 19:45:13.737356 2564 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236224Ki","pods":"250"} Jul 29 19:45:13.737440 ip-10-0-131-93 kubenswrapper[2564]: I0729 19:45:13.737396 2564 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236224Ki","pods":"250"} Jul 29 19:45:14.344057 ip-10-0-131-93 kubenswrapper[2564]: I0729 19:45:14.344021 2564 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-ait-d0aaa6d0-openshift-default-7cb489fc6b-s5n62" event={"ID":"c44b5d07-ffb1-4f5e-a949-38d459fecd10","Type":"ContainerStarted","Data":"a25c0bf9433b7908b5ed235d8f28e1e4f8c5ca652569b20b90931b3b2a880ea2"} Jul 29 19:45:14.344234 ip-10-0-131-93 kubenswrapper[2564]: I0729 19:45:14.344107 2564 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-ait-d0aaa6d0-openshift-default-7cb489fc6b-s5n62" event={"ID":"c44b5d07-ffb1-4f5e-a949-38d459fecd10","Type":"ContainerStarted","Data":"bfb8a5691e80d2120e833953755c49c69f8630040e429155bc9f5e47567d383f"} Jul 29 19:45:14.591344 ip-10-0-131-93 kubenswrapper[2564]: I0729 19:45:14.591299 2564 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="openshift-ingress/e2e-ait-d0aaa6d0-openshift-default-7cb489fc6b-s5n62" Jul 29 19:45:14.596216 ip-10-0-131-93 kubenswrapper[2564]: I0729 19:45:14.596154 2564 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="openshift-ingress/e2e-ait-d0aaa6d0-openshift-default-7cb489fc6b-s5n62" Jul 29 19:45:14.619900 ip-10-0-131-93 kubenswrapper[2564]: I0729 19:45:14.619849 2564 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-ingress/e2e-ait-d0aaa6d0-openshift-default-7cb489fc6b-s5n62" podStartSLOduration=1.619832056 podStartE2EDuration="1.619832056s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-29 19:45:13 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-29 19:45:14.376495916 +0000 UTC m=+1690.100555607" watchObservedRunningTime="2026-07-29 19:45:14.619832056 +0000 UTC m=+1690.343891747" Jul 29 19:45:15.349834 ip-10-0-131-93 kubenswrapper[2564]: I0729 19:45:15.349803 2564 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/e2e-ait-d0aaa6d0-openshift-default-7cb489fc6b-s5n62" Jul 29 19:45:15.350960 ip-10-0-131-93 kubenswrapper[2564]: I0729 19:45:15.350939 2564 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/e2e-ait-d0aaa6d0-openshift-default-7cb489fc6b-s5n62" Jul 29 19:45:15.792233 ip-10-0-131-93 kubenswrapper[2564]: I0729 19:45:15.792203 2564 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-444jx_faa8c546-c75b-4e10-93b4-661aa2b5f53e/konnectivity-agent/0.log" limit=1024 Jul 29 19:45:15.892594 ip-10-0-131-93 kubenswrapper[2564]: I0729 19:45:15.892560 2564 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-444jx_faa8c546-c75b-4e10-93b4-661aa2b5f53e/konnectivity-agent/0.log" limit=1024 Jul 29 19:45:16.903393 ip-10-0-131-93 kubenswrapper[2564]: I0729 19:45:16.903349 2564 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-ait-d0aaa6d0-59bd7fc879-m22bn"] Jul 29 19:45:16.906747 ip-10-0-131-93 kubenswrapper[2564]: I0729 19:45:16.906726 2564 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-e2e-ait-d0aaa6d0-59bd7fc879-m22bn" Jul 29 19:45:16.909554 ip-10-0-131-93 kubenswrapper[2564]: I0729 19:45:16.909533 2564 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"odh-ai-gateway-infra\"/\"maas-api-e2e-ait-d0aaa6d0-serving-cert\"" Jul 29 19:45:16.909693 ip-10-0-131-93 kubenswrapper[2564]: I0729 19:45:16.909676 2564 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"odh-ai-gateway-infra\"/\"maas-api-dockercfg-4dw6s\"" Jul 29 19:45:16.923820 ip-10-0-131-93 kubenswrapper[2564]: I0729 19:45:16.923795 2564 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-ait-d0aaa6d0-59bd7fc879-m22bn"] Jul 29 19:45:16.942535 ip-10-0-131-93 kubenswrapper[2564]: I0729 19:45:16.942503 2564 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-8xsq5\" (UniqueName: \"kubernetes.io/projected/0eeb075c-42cc-489e-b07b-ff873cedd1da-kube-api-access-8xsq5\") pod \"maas-api-e2e-ait-d0aaa6d0-59bd7fc879-m22bn\" (UID: \"0eeb075c-42cc-489e-b07b-ff873cedd1da\") " pod="odh-ai-gateway-infra/maas-api-e2e-ait-d0aaa6d0-59bd7fc879-m22bn" Jul 29 19:45:16.942677 ip-10-0-131-93 kubenswrapper[2564]: I0729 19:45:16.942552 2564 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/0eeb075c-42cc-489e-b07b-ff873cedd1da-openshift-service-ca\") pod \"maas-api-e2e-ait-d0aaa6d0-59bd7fc879-m22bn\" (UID: \"0eeb075c-42cc-489e-b07b-ff873cedd1da\") " pod="odh-ai-gateway-infra/maas-api-e2e-ait-d0aaa6d0-59bd7fc879-m22bn" Jul 29 19:45:16.942677 ip-10-0-131-93 kubenswrapper[2564]: I0729 19:45:16.942639 2564 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/0eeb075c-42cc-489e-b07b-ff873cedd1da-maas-api-tls\") pod \"maas-api-e2e-ait-d0aaa6d0-59bd7fc879-m22bn\" (UID: \"0eeb075c-42cc-489e-b07b-ff873cedd1da\") " pod="odh-ai-gateway-infra/maas-api-e2e-ait-d0aaa6d0-59bd7fc879-m22bn" Jul 29 19:45:16.989811 ip-10-0-131-93 kubenswrapper[2564]: I0729 19:45:16.989774 2564 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-444jx_faa8c546-c75b-4e10-93b4-661aa2b5f53e/konnectivity-agent/0.log" limit=1024 Jul 29 19:45:17.044082 ip-10-0-131-93 kubenswrapper[2564]: I0729 19:45:17.044045 2564 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-8xsq5\" (UniqueName: \"kubernetes.io/projected/0eeb075c-42cc-489e-b07b-ff873cedd1da-kube-api-access-8xsq5\") pod \"maas-api-e2e-ait-d0aaa6d0-59bd7fc879-m22bn\" (UID: \"0eeb075c-42cc-489e-b07b-ff873cedd1da\") " pod="odh-ai-gateway-infra/maas-api-e2e-ait-d0aaa6d0-59bd7fc879-m22bn" Jul 29 19:45:17.044266 ip-10-0-131-93 kubenswrapper[2564]: I0729 19:45:17.044095 2564 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/0eeb075c-42cc-489e-b07b-ff873cedd1da-openshift-service-ca\") pod \"maas-api-e2e-ait-d0aaa6d0-59bd7fc879-m22bn\" (UID: \"0eeb075c-42cc-489e-b07b-ff873cedd1da\") " pod="odh-ai-gateway-infra/maas-api-e2e-ait-d0aaa6d0-59bd7fc879-m22bn" Jul 29 19:45:17.044266 ip-10-0-131-93 kubenswrapper[2564]: I0729 19:45:17.044134 2564 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/0eeb075c-42cc-489e-b07b-ff873cedd1da-maas-api-tls\") pod \"maas-api-e2e-ait-d0aaa6d0-59bd7fc879-m22bn\" (UID: \"0eeb075c-42cc-489e-b07b-ff873cedd1da\") " pod="odh-ai-gateway-infra/maas-api-e2e-ait-d0aaa6d0-59bd7fc879-m22bn" Jul 29 19:45:17.044266 ip-10-0-131-93 kubenswrapper[2564]: E0729 19:45:17.044220 2564 secret.go:189] Couldn't get secret odh-ai-gateway-infra/maas-api-e2e-ait-d0aaa6d0-serving-cert: secret "maas-api-e2e-ait-d0aaa6d0-serving-cert" not found Jul 29 19:45:17.044429 ip-10-0-131-93 kubenswrapper[2564]: E0729 19:45:17.044277 2564 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/0eeb075c-42cc-489e-b07b-ff873cedd1da-maas-api-tls podName:0eeb075c-42cc-489e-b07b-ff873cedd1da nodeName:}" failed. No retries permitted until 2026-07-29 19:45:17.544259372 +0000 UTC m=+1693.268319045 (durationBeforeRetry 500ms). Error: MountVolume.SetUp failed for volume "maas-api-tls" (UniqueName: "kubernetes.io/secret/0eeb075c-42cc-489e-b07b-ff873cedd1da-maas-api-tls") pod "maas-api-e2e-ait-d0aaa6d0-59bd7fc879-m22bn" (UID: "0eeb075c-42cc-489e-b07b-ff873cedd1da") : secret "maas-api-e2e-ait-d0aaa6d0-serving-cert" not found Jul 29 19:45:17.044873 ip-10-0-131-93 kubenswrapper[2564]: I0729 19:45:17.044855 2564 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/0eeb075c-42cc-489e-b07b-ff873cedd1da-openshift-service-ca\") pod \"maas-api-e2e-ait-d0aaa6d0-59bd7fc879-m22bn\" (UID: \"0eeb075c-42cc-489e-b07b-ff873cedd1da\") " pod="odh-ai-gateway-infra/maas-api-e2e-ait-d0aaa6d0-59bd7fc879-m22bn" Jul 29 19:45:17.055944 ip-10-0-131-93 kubenswrapper[2564]: I0729 19:45:17.055920 2564 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-8xsq5\" (UniqueName: \"kubernetes.io/projected/0eeb075c-42cc-489e-b07b-ff873cedd1da-kube-api-access-8xsq5\") pod \"maas-api-e2e-ait-d0aaa6d0-59bd7fc879-m22bn\" (UID: \"0eeb075c-42cc-489e-b07b-ff873cedd1da\") " pod="odh-ai-gateway-infra/maas-api-e2e-ait-d0aaa6d0-59bd7fc879-m22bn" Jul 29 19:45:17.260754 ip-10-0-131-93 kubenswrapper[2564]: I0729 19:45:17.260704 2564 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-444jx_faa8c546-c75b-4e10-93b4-661aa2b5f53e/konnectivity-agent/0.log" limit=1024 Jul 29 19:45:17.549105 ip-10-0-131-93 kubenswrapper[2564]: I0729 19:45:17.549005 2564 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/0eeb075c-42cc-489e-b07b-ff873cedd1da-maas-api-tls\") pod \"maas-api-e2e-ait-d0aaa6d0-59bd7fc879-m22bn\" (UID: \"0eeb075c-42cc-489e-b07b-ff873cedd1da\") " pod="odh-ai-gateway-infra/maas-api-e2e-ait-d0aaa6d0-59bd7fc879-m22bn" Jul 29 19:45:17.551421 ip-10-0-131-93 kubenswrapper[2564]: I0729 19:45:17.551396 2564 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/0eeb075c-42cc-489e-b07b-ff873cedd1da-maas-api-tls\") pod \"maas-api-e2e-ait-d0aaa6d0-59bd7fc879-m22bn\" (UID: \"0eeb075c-42cc-489e-b07b-ff873cedd1da\") " pod="odh-ai-gateway-infra/maas-api-e2e-ait-d0aaa6d0-59bd7fc879-m22bn" Jul 29 19:45:17.818693 ip-10-0-131-93 kubenswrapper[2564]: I0729 19:45:17.818612 2564 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-e2e-ait-d0aaa6d0-59bd7fc879-m22bn" Jul 29 19:45:17.918701 ip-10-0-131-93 kubenswrapper[2564]: I0729 19:45:17.918656 2564 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-444jx_faa8c546-c75b-4e10-93b4-661aa2b5f53e/konnectivity-agent/0.log" limit=1024 Jul 29 19:45:17.979577 ip-10-0-131-93 kubenswrapper[2564]: I0729 19:45:17.979547 2564 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-ait-d0aaa6d0-59bd7fc879-m22bn"] Jul 29 19:45:17.987426 ip-10-0-131-93 kubenswrapper[2564]: W0729 19:45:17.987345 2564 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod0eeb075c_42cc_489e_b07b_ff873cedd1da.slice/crio-53773fa68d697df91aeb8fb3e4d7fe48dd4eb08c966efb2de116d6b6310ecb5c WatchSource:0}: Error finding container 53773fa68d697df91aeb8fb3e4d7fe48dd4eb08c966efb2de116d6b6310ecb5c: Status 404 returned error can't find the container with id 53773fa68d697df91aeb8fb3e4d7fe48dd4eb08c966efb2de116d6b6310ecb5c Jul 29 19:45:17.989123 ip-10-0-131-93 kubenswrapper[2564]: I0729 19:45:17.989105 2564 provider.go:93] Refreshing cache for provider: *credentialprovider.defaultDockerConfigProvider Jul 29 19:45:18.102031 ip-10-0-131-93 kubenswrapper[2564]: I0729 19:45:18.101953 2564 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-444jx_faa8c546-c75b-4e10-93b4-661aa2b5f53e/konnectivity-agent/0.log" limit=1024 Jul 29 19:45:18.366166 ip-10-0-131-93 kubenswrapper[2564]: I0729 19:45:18.366092 2564 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-e2e-ait-d0aaa6d0-59bd7fc879-m22bn" event={"ID":"0eeb075c-42cc-489e-b07b-ff873cedd1da","Type":"ContainerStarted","Data":"53773fa68d697df91aeb8fb3e4d7fe48dd4eb08c966efb2de116d6b6310ecb5c"} Jul 29 19:45:19.372855 ip-10-0-131-93 kubenswrapper[2564]: I0729 19:45:19.372821 2564 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-e2e-ait-d0aaa6d0-59bd7fc879-m22bn" event={"ID":"0eeb075c-42cc-489e-b07b-ff873cedd1da","Type":"ContainerStarted","Data":"804b8bc8dc02e9cd223be379d0dbac85f88575f0efd6220f04deb79d21f507a8"} Jul 29 19:45:19.373247 ip-10-0-131-93 kubenswrapper[2564]: I0729 19:45:19.372922 2564 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="odh-ai-gateway-infra/maas-api-e2e-ait-d0aaa6d0-59bd7fc879-m22bn" Jul 29 19:45:19.394118 ip-10-0-131-93 kubenswrapper[2564]: I0729 19:45:19.394072 2564 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="odh-ai-gateway-infra/maas-api-e2e-ait-d0aaa6d0-59bd7fc879-m22bn" podStartSLOduration=2.868850031 podStartE2EDuration="3.394057324s" totalImagesPullingTime="525.207293ms" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-29 19:45:16 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-29 19:45:19.39157475 +0000 UTC m=+1695.115634441" watchObservedRunningTime="2026-07-29 19:45:19.394057324 +0000 UTC m=+1695.118117017" Jul 29 19:45:25.390267 ip-10-0-131-93 kubenswrapper[2564]: I0729 19:45:25.390234 2564 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="odh-ai-gateway-infra/maas-api-e2e-ait-d0aaa6d0-59bd7fc879-m22bn" Jul 29 19:45:30.777920 ip-10-0-131-93 kubenswrapper[2564]: I0729 19:45:30.777155 2564 patch_prober.go:28] interesting pod/e2e-ait-d0aaa6d0-openshift-default-7cb489fc6b-s5n62 container/istio-proxy namespace/openshift-ingress: Readiness probe status=failure output="Get \"http://10.132.0.36:15021/healthz/ready\": read tcp 10.132.0.2:59034->10.132.0.36:15021: read: connection reset by peer" start-of-body= Jul 29 19:45:30.777920 ip-10-0-131-93 kubenswrapper[2564]: I0729 19:45:30.777224 2564 prober.go:120] "Probe failed" probeType="Readiness" pod="openshift-ingress/e2e-ait-d0aaa6d0-openshift-default-7cb489fc6b-s5n62" podUID="c44b5d07-ffb1-4f5e-a949-38d459fecd10" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.132.0.36:15021/healthz/ready\": read tcp 10.132.0.2:59034->10.132.0.36:15021: read: connection reset by peer" Jul 29 19:45:31.435811 ip-10-0-131-93 kubenswrapper[2564]: I0729 19:45:31.435772 2564 generic.go:356] "Generic (PLEG): container finished" podID="c44b5d07-ffb1-4f5e-a949-38d459fecd10" containerID="a25c0bf9433b7908b5ed235d8f28e1e4f8c5ca652569b20b90931b3b2a880ea2" exitCode=0 Jul 29 19:45:31.435992 ip-10-0-131-93 kubenswrapper[2564]: I0729 19:45:31.435847 2564 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-ait-d0aaa6d0-openshift-default-7cb489fc6b-s5n62" event={"ID":"c44b5d07-ffb1-4f5e-a949-38d459fecd10","Type":"ContainerDied","Data":"a25c0bf9433b7908b5ed235d8f28e1e4f8c5ca652569b20b90931b3b2a880ea2"} Jul 29 19:45:31.436262 ip-10-0-131-93 kubenswrapper[2564]: I0729 19:45:31.436217 2564 scope.go:117] "RemoveContainer" containerID="a25c0bf9433b7908b5ed235d8f28e1e4f8c5ca652569b20b90931b3b2a880ea2" Jul 29 19:45:31.437185 ip-10-0-131-93 kubenswrapper[2564]: I0729 19:45:31.437160 2564 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236224Ki","pods":"250"} Jul 29 19:45:31.437241 ip-10-0-131-93 kubenswrapper[2564]: I0729 19:45:31.437224 2564 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236224Ki","pods":"250"} Jul 29 19:45:31.437279 ip-10-0-131-93 kubenswrapper[2564]: I0729 19:45:31.437253 2564 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236224Ki","pods":"250"} Jul 29 19:45:31.561210 ip-10-0-131-93 kubenswrapper[2564]: I0729 19:45:31.561174 2564 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-444jx_faa8c546-c75b-4e10-93b4-661aa2b5f53e/konnectivity-agent/0.log" limit=1024 Jul 29 19:45:32.442360 ip-10-0-131-93 kubenswrapper[2564]: I0729 19:45:32.442331 2564 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-ait-d0aaa6d0-openshift-default-7cb489fc6b-s5n62" event={"ID":"c44b5d07-ffb1-4f5e-a949-38d459fecd10","Type":"ContainerStarted","Data":"2da5bd91403cdaa55591391fec8d3991c94847d15ef7f63eb34ce4d2ebfe8f42"} Jul 29 19:45:32.591231 ip-10-0-131-93 kubenswrapper[2564]: I0729 19:45:32.591181 2564 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="openshift-ingress/e2e-ait-d0aaa6d0-openshift-default-7cb489fc6b-s5n62" Jul 29 19:45:32.591614 ip-10-0-131-93 kubenswrapper[2564]: I0729 19:45:32.591580 2564 patch_prober.go:28] interesting pod/e2e-ait-d0aaa6d0-openshift-default-7cb489fc6b-s5n62 container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.132.0.36:15021/healthz/ready\": dial tcp 10.132.0.36:15021: connect: connection refused" start-of-body= Jul 29 19:45:32.591725 ip-10-0-131-93 kubenswrapper[2564]: I0729 19:45:32.591664 2564 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-ait-d0aaa6d0-openshift-default-7cb489fc6b-s5n62" podUID="c44b5d07-ffb1-4f5e-a949-38d459fecd10" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.132.0.36:15021/healthz/ready\": dial tcp 10.132.0.36:15021: connect: connection refused" Jul 29 19:45:33.592433 ip-10-0-131-93 kubenswrapper[2564]: I0729 19:45:33.592387 2564 patch_prober.go:28] interesting pod/e2e-ait-d0aaa6d0-openshift-default-7cb489fc6b-s5n62 container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.132.0.36:15021/healthz/ready\": dial tcp 10.132.0.36:15021: connect: connection refused" start-of-body= Jul 29 19:45:33.592978 ip-10-0-131-93 kubenswrapper[2564]: I0729 19:45:33.592459 2564 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-ait-d0aaa6d0-openshift-default-7cb489fc6b-s5n62" podUID="c44b5d07-ffb1-4f5e-a949-38d459fecd10" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.132.0.36:15021/healthz/ready\": dial tcp 10.132.0.36:15021: connect: connection refused" Jul 29 19:45:34.592184 ip-10-0-131-93 kubenswrapper[2564]: I0729 19:45:34.592143 2564 patch_prober.go:28] interesting pod/e2e-ait-d0aaa6d0-openshift-default-7cb489fc6b-s5n62 container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.132.0.36:15021/healthz/ready\": dial tcp 10.132.0.36:15021: connect: connection refused" start-of-body= Jul 29 19:45:34.592628 ip-10-0-131-93 kubenswrapper[2564]: I0729 19:45:34.592597 2564 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-ait-d0aaa6d0-openshift-default-7cb489fc6b-s5n62" podUID="c44b5d07-ffb1-4f5e-a949-38d459fecd10" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.132.0.36:15021/healthz/ready\": dial tcp 10.132.0.36:15021: connect: connection refused" Jul 29 19:45:35.595855 ip-10-0-131-93 kubenswrapper[2564]: I0729 19:45:35.595815 2564 patch_prober.go:28] interesting pod/e2e-ait-d0aaa6d0-openshift-default-7cb489fc6b-s5n62 container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.132.0.36:15021/healthz/ready\": dial tcp 10.132.0.36:15021: connect: connection refused" start-of-body= Jul 29 19:45:35.596364 ip-10-0-131-93 kubenswrapper[2564]: I0729 19:45:35.595898 2564 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-ait-d0aaa6d0-openshift-default-7cb489fc6b-s5n62" podUID="c44b5d07-ffb1-4f5e-a949-38d459fecd10" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.132.0.36:15021/healthz/ready\": dial tcp 10.132.0.36:15021: connect: connection refused" Jul 29 19:45:36.594869 ip-10-0-131-93 kubenswrapper[2564]: I0729 19:45:36.594839 2564 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="openshift-ingress/e2e-ait-d0aaa6d0-openshift-default-7cb489fc6b-s5n62" Jul 29 19:45:36.595121 ip-10-0-131-93 kubenswrapper[2564]: I0729 19:45:36.595099 2564 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/e2e-ait-d0aaa6d0-openshift-default-7cb489fc6b-s5n62" Jul 29 19:45:36.595809 ip-10-0-131-93 kubenswrapper[2564]: I0729 19:45:36.595794 2564 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/e2e-ait-d0aaa6d0-openshift-default-7cb489fc6b-s5n62" Jul 29 19:45:47.670138 ip-10-0-131-93 kubenswrapper[2564]: I0729 19:45:47.670105 2564 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-ait-d0aaa6d0-59bd7fc879-m22bn"] Jul 29 19:45:47.670601 ip-10-0-131-93 kubenswrapper[2564]: I0729 19:45:47.670329 2564 kuberuntime_container.go:873] "Killing container with a grace period" pod="odh-ai-gateway-infra/maas-api-e2e-ait-d0aaa6d0-59bd7fc879-m22bn" podUID="0eeb075c-42cc-489e-b07b-ff873cedd1da" containerName="maas-api" containerID="cri-o://804b8bc8dc02e9cd223be379d0dbac85f88575f0efd6220f04deb79d21f507a8" gracePeriod=30 Jul 29 19:45:47.712824 ip-10-0-131-93 kubenswrapper[2564]: I0729 19:45:47.712795 2564 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-444jx_faa8c546-c75b-4e10-93b4-661aa2b5f53e/konnectivity-agent/0.log" limit=1024 Jul 29 19:45:47.799195 ip-10-0-131-93 kubenswrapper[2564]: I0729 19:45:47.799149 2564 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-444jx_faa8c546-c75b-4e10-93b4-661aa2b5f53e/konnectivity-agent/0.log" limit=1024 Jul 29 19:45:47.882091 ip-10-0-131-93 kubenswrapper[2564]: I0729 19:45:47.882062 2564 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-444jx_faa8c546-c75b-4e10-93b4-661aa2b5f53e/konnectivity-agent/0.log" limit=1024 Jul 29 19:45:47.939864 ip-10-0-131-93 kubenswrapper[2564]: I0729 19:45:47.939838 2564 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-e2e-ait-d0aaa6d0-59bd7fc879-m22bn" Jul 29 19:45:47.951269 ip-10-0-131-93 kubenswrapper[2564]: I0729 19:45:47.951219 2564 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-444jx_faa8c546-c75b-4e10-93b4-661aa2b5f53e/konnectivity-agent/0.log" limit=1024 Jul 29 19:45:48.070347 ip-10-0-131-93 kubenswrapper[2564]: I0729 19:45:48.070309 2564 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-ingress/e2e-ait-d0aaa6d0-openshift-default-7cb489fc6b-s5n62"] Jul 29 19:45:48.070695 ip-10-0-131-93 kubenswrapper[2564]: I0729 19:45:48.070641 2564 kuberuntime_container.go:873] "Killing container with a grace period" pod="openshift-ingress/e2e-ait-d0aaa6d0-openshift-default-7cb489fc6b-s5n62" podUID="c44b5d07-ffb1-4f5e-a949-38d459fecd10" containerName="istio-proxy" containerID="cri-o://2da5bd91403cdaa55591391fec8d3991c94847d15ef7f63eb34ce4d2ebfe8f42" gracePeriod=30 Jul 29 19:45:48.090597 ip-10-0-131-93 kubenswrapper[2564]: I0729 19:45:48.090569 2564 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-444jx_faa8c546-c75b-4e10-93b4-661aa2b5f53e/konnectivity-agent/0.log" limit=1024 Jul 29 19:45:48.117291 ip-10-0-131-93 kubenswrapper[2564]: I0729 19:45:48.117259 2564 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/0eeb075c-42cc-489e-b07b-ff873cedd1da-maas-api-tls\") pod \"0eeb075c-42cc-489e-b07b-ff873cedd1da\" (UID: \"0eeb075c-42cc-489e-b07b-ff873cedd1da\") " Jul 29 19:45:48.117456 ip-10-0-131-93 kubenswrapper[2564]: I0729 19:45:48.117340 2564 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/0eeb075c-42cc-489e-b07b-ff873cedd1da-openshift-service-ca\") pod \"0eeb075c-42cc-489e-b07b-ff873cedd1da\" (UID: \"0eeb075c-42cc-489e-b07b-ff873cedd1da\") " Jul 29 19:45:48.117456 ip-10-0-131-93 kubenswrapper[2564]: I0729 19:45:48.117425 2564 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-8xsq5\" (UniqueName: \"kubernetes.io/projected/0eeb075c-42cc-489e-b07b-ff873cedd1da-kube-api-access-8xsq5\") pod \"0eeb075c-42cc-489e-b07b-ff873cedd1da\" (UID: \"0eeb075c-42cc-489e-b07b-ff873cedd1da\") " Jul 29 19:45:48.117709 ip-10-0-131-93 kubenswrapper[2564]: I0729 19:45:48.117680 2564 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/0eeb075c-42cc-489e-b07b-ff873cedd1da-openshift-service-ca" (OuterVolumeSpecName: "openshift-service-ca") pod "0eeb075c-42cc-489e-b07b-ff873cedd1da" (UID: "0eeb075c-42cc-489e-b07b-ff873cedd1da"). InnerVolumeSpecName "openshift-service-ca". PluginName "kubernetes.io/configmap", VolumeGIDValue "" Jul 29 19:45:48.119490 ip-10-0-131-93 kubenswrapper[2564]: I0729 19:45:48.119467 2564 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/0eeb075c-42cc-489e-b07b-ff873cedd1da-maas-api-tls" (OuterVolumeSpecName: "maas-api-tls") pod "0eeb075c-42cc-489e-b07b-ff873cedd1da" (UID: "0eeb075c-42cc-489e-b07b-ff873cedd1da"). InnerVolumeSpecName "maas-api-tls". PluginName "kubernetes.io/secret", VolumeGIDValue "" Jul 29 19:45:48.119568 ip-10-0-131-93 kubenswrapper[2564]: I0729 19:45:48.119515 2564 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/0eeb075c-42cc-489e-b07b-ff873cedd1da-kube-api-access-8xsq5" (OuterVolumeSpecName: "kube-api-access-8xsq5") pod "0eeb075c-42cc-489e-b07b-ff873cedd1da" (UID: "0eeb075c-42cc-489e-b07b-ff873cedd1da"). InnerVolumeSpecName "kube-api-access-8xsq5". PluginName "kubernetes.io/projected", VolumeGIDValue "" Jul 29 19:45:48.218704 ip-10-0-131-93 kubenswrapper[2564]: I0729 19:45:48.218613 2564 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-8xsq5\" (UniqueName: \"kubernetes.io/projected/0eeb075c-42cc-489e-b07b-ff873cedd1da-kube-api-access-8xsq5\") on node \"ip-10-0-131-93.ec2.internal\" DevicePath \"\"" Jul 29 19:45:48.218704 ip-10-0-131-93 kubenswrapper[2564]: I0729 19:45:48.218644 2564 reconciler_common.go:299] "Volume detached for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/0eeb075c-42cc-489e-b07b-ff873cedd1da-maas-api-tls\") on node \"ip-10-0-131-93.ec2.internal\" DevicePath \"\"" Jul 29 19:45:48.218704 ip-10-0-131-93 kubenswrapper[2564]: I0729 19:45:48.218654 2564 reconciler_common.go:299] "Volume detached for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/0eeb075c-42cc-489e-b07b-ff873cedd1da-openshift-service-ca\") on node \"ip-10-0-131-93.ec2.internal\" DevicePath \"\"" Jul 29 19:45:48.320951 ip-10-0-131-93 kubenswrapper[2564]: I0729 19:45:48.320918 2564 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-444jx_faa8c546-c75b-4e10-93b4-661aa2b5f53e/konnectivity-agent/0.log" limit=1024 Jul 29 19:45:48.520647 ip-10-0-131-93 kubenswrapper[2564]: I0729 19:45:48.520565 2564 generic.go:356] "Generic (PLEG): container finished" podID="0eeb075c-42cc-489e-b07b-ff873cedd1da" containerID="804b8bc8dc02e9cd223be379d0dbac85f88575f0efd6220f04deb79d21f507a8" exitCode=0 Jul 29 19:45:48.520647 ip-10-0-131-93 kubenswrapper[2564]: I0729 19:45:48.520614 2564 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-e2e-ait-d0aaa6d0-59bd7fc879-m22bn" Jul 29 19:45:48.520834 ip-10-0-131-93 kubenswrapper[2564]: I0729 19:45:48.520658 2564 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-e2e-ait-d0aaa6d0-59bd7fc879-m22bn" event={"ID":"0eeb075c-42cc-489e-b07b-ff873cedd1da","Type":"ContainerDied","Data":"804b8bc8dc02e9cd223be379d0dbac85f88575f0efd6220f04deb79d21f507a8"} Jul 29 19:45:48.520834 ip-10-0-131-93 kubenswrapper[2564]: I0729 19:45:48.520709 2564 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-e2e-ait-d0aaa6d0-59bd7fc879-m22bn" event={"ID":"0eeb075c-42cc-489e-b07b-ff873cedd1da","Type":"ContainerDied","Data":"53773fa68d697df91aeb8fb3e4d7fe48dd4eb08c966efb2de116d6b6310ecb5c"} Jul 29 19:45:48.520834 ip-10-0-131-93 kubenswrapper[2564]: I0729 19:45:48.520733 2564 scope.go:117] "RemoveContainer" containerID="804b8bc8dc02e9cd223be379d0dbac85f88575f0efd6220f04deb79d21f507a8" Jul 29 19:45:48.529994 ip-10-0-131-93 kubenswrapper[2564]: I0729 19:45:48.529928 2564 scope.go:117] "RemoveContainer" containerID="804b8bc8dc02e9cd223be379d0dbac85f88575f0efd6220f04deb79d21f507a8" Jul 29 19:45:48.530292 ip-10-0-131-93 kubenswrapper[2564]: E0729 19:45:48.530273 2564 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"804b8bc8dc02e9cd223be379d0dbac85f88575f0efd6220f04deb79d21f507a8\": container with ID starting with 804b8bc8dc02e9cd223be379d0dbac85f88575f0efd6220f04deb79d21f507a8 not found: ID does not exist" containerID="804b8bc8dc02e9cd223be379d0dbac85f88575f0efd6220f04deb79d21f507a8" Jul 29 19:45:48.530358 ip-10-0-131-93 kubenswrapper[2564]: I0729 19:45:48.530300 2564 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"804b8bc8dc02e9cd223be379d0dbac85f88575f0efd6220f04deb79d21f507a8"} err="failed to get container status \"804b8bc8dc02e9cd223be379d0dbac85f88575f0efd6220f04deb79d21f507a8\": rpc error: code = NotFound desc = could not find container \"804b8bc8dc02e9cd223be379d0dbac85f88575f0efd6220f04deb79d21f507a8\": container with ID starting with 804b8bc8dc02e9cd223be379d0dbac85f88575f0efd6220f04deb79d21f507a8 not found: ID does not exist" Jul 29 19:45:48.553043 ip-10-0-131-93 kubenswrapper[2564]: I0729 19:45:48.553022 2564 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-ait-d0aaa6d0-59bd7fc879-m22bn"] Jul 29 19:45:48.575825 ip-10-0-131-93 kubenswrapper[2564]: I0729 19:45:48.575801 2564 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-ait-d0aaa6d0-59bd7fc879-m22bn"] Jul 29 19:45:48.783232 ip-10-0-131-93 kubenswrapper[2564]: I0729 19:45:48.783149 2564 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="0eeb075c-42cc-489e-b07b-ff873cedd1da" path="/var/lib/kubelet/pods/0eeb075c-42cc-489e-b07b-ff873cedd1da/volumes" Jul 29 19:45:49.087333 ip-10-0-131-93 kubenswrapper[2564]: I0729 19:45:49.087258 2564 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-ingress/e2e-migrate-13baf70f-gw-openshift-default-647c89464-7bskp"] Jul 29 19:45:49.087621 ip-10-0-131-93 kubenswrapper[2564]: I0729 19:45:49.087606 2564 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="0eeb075c-42cc-489e-b07b-ff873cedd1da" containerName="maas-api" Jul 29 19:45:49.087621 ip-10-0-131-93 kubenswrapper[2564]: I0729 19:45:49.087621 2564 state_mem.go:107] "Deleted CPUSet assignment" podUID="0eeb075c-42cc-489e-b07b-ff873cedd1da" containerName="maas-api" Jul 29 19:45:49.087726 ip-10-0-131-93 kubenswrapper[2564]: I0729 19:45:49.087678 2564 memory_manager.go:368] "RemoveStaleState removing state" podUID="0eeb075c-42cc-489e-b07b-ff873cedd1da" containerName="maas-api" Jul 29 19:45:49.090697 ip-10-0-131-93 kubenswrapper[2564]: I0729 19:45:49.090678 2564 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-migrate-13baf70f-gw-openshift-default-647c89464-7bskp" Jul 29 19:45:49.094926 ip-10-0-131-93 kubenswrapper[2564]: I0729 19:45:49.094906 2564 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress\"/\"e2e-migrate-13baf70f-gw-openshift-default-dockercfg-s59rf\"" Jul 29 19:45:49.105561 ip-10-0-131-93 kubenswrapper[2564]: I0729 19:45:49.105539 2564 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/e2e-migrate-13baf70f-gw-openshift-default-647c89464-7bskp"] Jul 29 19:45:49.228949 ip-10-0-131-93 kubenswrapper[2564]: I0729 19:45:49.228914 2564 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/8926f238-14d4-4e56-8635-95d47c45c78f-istio-envoy\") pod \"e2e-migrate-13baf70f-gw-openshift-default-647c89464-7bskp\" (UID: \"8926f238-14d4-4e56-8635-95d47c45c78f\") " pod="openshift-ingress/e2e-migrate-13baf70f-gw-openshift-default-647c89464-7bskp" Jul 29 19:45:49.228949 ip-10-0-131-93 kubenswrapper[2564]: I0729 19:45:49.228948 2564 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/8926f238-14d4-4e56-8635-95d47c45c78f-istio-podinfo\") pod \"e2e-migrate-13baf70f-gw-openshift-default-647c89464-7bskp\" (UID: \"8926f238-14d4-4e56-8635-95d47c45c78f\") " pod="openshift-ingress/e2e-migrate-13baf70f-gw-openshift-default-647c89464-7bskp" Jul 29 19:45:49.229177 ip-10-0-131-93 kubenswrapper[2564]: I0729 19:45:49.228971 2564 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/8926f238-14d4-4e56-8635-95d47c45c78f-istio-data\") pod \"e2e-migrate-13baf70f-gw-openshift-default-647c89464-7bskp\" (UID: \"8926f238-14d4-4e56-8635-95d47c45c78f\") " pod="openshift-ingress/e2e-migrate-13baf70f-gw-openshift-default-647c89464-7bskp" Jul 29 19:45:49.229177 ip-10-0-131-93 kubenswrapper[2564]: I0729 19:45:49.229022 2564 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/8926f238-14d4-4e56-8635-95d47c45c78f-istio-token\") pod \"e2e-migrate-13baf70f-gw-openshift-default-647c89464-7bskp\" (UID: \"8926f238-14d4-4e56-8635-95d47c45c78f\") " pod="openshift-ingress/e2e-migrate-13baf70f-gw-openshift-default-647c89464-7bskp" Jul 29 19:45:49.229177 ip-10-0-131-93 kubenswrapper[2564]: I0729 19:45:49.229059 2564 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/8926f238-14d4-4e56-8635-95d47c45c78f-credential-socket\") pod \"e2e-migrate-13baf70f-gw-openshift-default-647c89464-7bskp\" (UID: \"8926f238-14d4-4e56-8635-95d47c45c78f\") " pod="openshift-ingress/e2e-migrate-13baf70f-gw-openshift-default-647c89464-7bskp" Jul 29 19:45:49.229177 ip-10-0-131-93 kubenswrapper[2564]: I0729 19:45:49.229089 2564 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/8926f238-14d4-4e56-8635-95d47c45c78f-istiod-ca-cert\") pod \"e2e-migrate-13baf70f-gw-openshift-default-647c89464-7bskp\" (UID: \"8926f238-14d4-4e56-8635-95d47c45c78f\") " pod="openshift-ingress/e2e-migrate-13baf70f-gw-openshift-default-647c89464-7bskp" Jul 29 19:45:49.229177 ip-10-0-131-93 kubenswrapper[2564]: I0729 19:45:49.229140 2564 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/8926f238-14d4-4e56-8635-95d47c45c78f-workload-socket\") pod \"e2e-migrate-13baf70f-gw-openshift-default-647c89464-7bskp\" (UID: \"8926f238-14d4-4e56-8635-95d47c45c78f\") " pod="openshift-ingress/e2e-migrate-13baf70f-gw-openshift-default-647c89464-7bskp" Jul 29 19:45:49.229348 ip-10-0-131-93 kubenswrapper[2564]: I0729 19:45:49.229188 2564 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-64mxv\" (UniqueName: \"kubernetes.io/projected/8926f238-14d4-4e56-8635-95d47c45c78f-kube-api-access-64mxv\") pod \"e2e-migrate-13baf70f-gw-openshift-default-647c89464-7bskp\" (UID: \"8926f238-14d4-4e56-8635-95d47c45c78f\") " pod="openshift-ingress/e2e-migrate-13baf70f-gw-openshift-default-647c89464-7bskp" Jul 29 19:45:49.229348 ip-10-0-131-93 kubenswrapper[2564]: I0729 19:45:49.229240 2564 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/8926f238-14d4-4e56-8635-95d47c45c78f-workload-certs\") pod \"e2e-migrate-13baf70f-gw-openshift-default-647c89464-7bskp\" (UID: \"8926f238-14d4-4e56-8635-95d47c45c78f\") " pod="openshift-ingress/e2e-migrate-13baf70f-gw-openshift-default-647c89464-7bskp" Jul 29 19:45:49.330532 ip-10-0-131-93 kubenswrapper[2564]: I0729 19:45:49.330500 2564 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/8926f238-14d4-4e56-8635-95d47c45c78f-workload-socket\") pod \"e2e-migrate-13baf70f-gw-openshift-default-647c89464-7bskp\" (UID: \"8926f238-14d4-4e56-8635-95d47c45c78f\") " pod="openshift-ingress/e2e-migrate-13baf70f-gw-openshift-default-647c89464-7bskp" Jul 29 19:45:49.330739 ip-10-0-131-93 kubenswrapper[2564]: I0729 19:45:49.330551 2564 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-64mxv\" (UniqueName: \"kubernetes.io/projected/8926f238-14d4-4e56-8635-95d47c45c78f-kube-api-access-64mxv\") pod \"e2e-migrate-13baf70f-gw-openshift-default-647c89464-7bskp\" (UID: \"8926f238-14d4-4e56-8635-95d47c45c78f\") " pod="openshift-ingress/e2e-migrate-13baf70f-gw-openshift-default-647c89464-7bskp" Jul 29 19:45:49.330739 ip-10-0-131-93 kubenswrapper[2564]: I0729 19:45:49.330598 2564 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/8926f238-14d4-4e56-8635-95d47c45c78f-workload-certs\") pod \"e2e-migrate-13baf70f-gw-openshift-default-647c89464-7bskp\" (UID: \"8926f238-14d4-4e56-8635-95d47c45c78f\") " pod="openshift-ingress/e2e-migrate-13baf70f-gw-openshift-default-647c89464-7bskp" Jul 29 19:45:49.330739 ip-10-0-131-93 kubenswrapper[2564]: I0729 19:45:49.330623 2564 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/8926f238-14d4-4e56-8635-95d47c45c78f-istio-envoy\") pod \"e2e-migrate-13baf70f-gw-openshift-default-647c89464-7bskp\" (UID: \"8926f238-14d4-4e56-8635-95d47c45c78f\") " pod="openshift-ingress/e2e-migrate-13baf70f-gw-openshift-default-647c89464-7bskp" Jul 29 19:45:49.330739 ip-10-0-131-93 kubenswrapper[2564]: I0729 19:45:49.330646 2564 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/8926f238-14d4-4e56-8635-95d47c45c78f-istio-podinfo\") pod \"e2e-migrate-13baf70f-gw-openshift-default-647c89464-7bskp\" (UID: \"8926f238-14d4-4e56-8635-95d47c45c78f\") " pod="openshift-ingress/e2e-migrate-13baf70f-gw-openshift-default-647c89464-7bskp" Jul 29 19:45:49.330739 ip-10-0-131-93 kubenswrapper[2564]: I0729 19:45:49.330677 2564 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/8926f238-14d4-4e56-8635-95d47c45c78f-istio-data\") pod \"e2e-migrate-13baf70f-gw-openshift-default-647c89464-7bskp\" (UID: \"8926f238-14d4-4e56-8635-95d47c45c78f\") " pod="openshift-ingress/e2e-migrate-13baf70f-gw-openshift-default-647c89464-7bskp" Jul 29 19:45:49.330739 ip-10-0-131-93 kubenswrapper[2564]: I0729 19:45:49.330704 2564 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/8926f238-14d4-4e56-8635-95d47c45c78f-istio-token\") pod \"e2e-migrate-13baf70f-gw-openshift-default-647c89464-7bskp\" (UID: \"8926f238-14d4-4e56-8635-95d47c45c78f\") " pod="openshift-ingress/e2e-migrate-13baf70f-gw-openshift-default-647c89464-7bskp" Jul 29 19:45:49.331023 ip-10-0-131-93 kubenswrapper[2564]: I0729 19:45:49.330741 2564 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/8926f238-14d4-4e56-8635-95d47c45c78f-credential-socket\") pod \"e2e-migrate-13baf70f-gw-openshift-default-647c89464-7bskp\" (UID: \"8926f238-14d4-4e56-8635-95d47c45c78f\") " pod="openshift-ingress/e2e-migrate-13baf70f-gw-openshift-default-647c89464-7bskp" Jul 29 19:45:49.331023 ip-10-0-131-93 kubenswrapper[2564]: I0729 19:45:49.330775 2564 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/8926f238-14d4-4e56-8635-95d47c45c78f-istiod-ca-cert\") pod \"e2e-migrate-13baf70f-gw-openshift-default-647c89464-7bskp\" (UID: \"8926f238-14d4-4e56-8635-95d47c45c78f\") " pod="openshift-ingress/e2e-migrate-13baf70f-gw-openshift-default-647c89464-7bskp" Jul 29 19:45:49.331023 ip-10-0-131-93 kubenswrapper[2564]: I0729 19:45:49.330997 2564 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/8926f238-14d4-4e56-8635-95d47c45c78f-workload-socket\") pod \"e2e-migrate-13baf70f-gw-openshift-default-647c89464-7bskp\" (UID: \"8926f238-14d4-4e56-8635-95d47c45c78f\") " pod="openshift-ingress/e2e-migrate-13baf70f-gw-openshift-default-647c89464-7bskp" Jul 29 19:45:49.331198 ip-10-0-131-93 kubenswrapper[2564]: I0729 19:45:49.331076 2564 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/8926f238-14d4-4e56-8635-95d47c45c78f-workload-certs\") pod \"e2e-migrate-13baf70f-gw-openshift-default-647c89464-7bskp\" (UID: \"8926f238-14d4-4e56-8635-95d47c45c78f\") " pod="openshift-ingress/e2e-migrate-13baf70f-gw-openshift-default-647c89464-7bskp" Jul 29 19:45:49.331198 ip-10-0-131-93 kubenswrapper[2564]: I0729 19:45:49.331125 2564 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/8926f238-14d4-4e56-8635-95d47c45c78f-istio-data\") pod \"e2e-migrate-13baf70f-gw-openshift-default-647c89464-7bskp\" (UID: \"8926f238-14d4-4e56-8635-95d47c45c78f\") " pod="openshift-ingress/e2e-migrate-13baf70f-gw-openshift-default-647c89464-7bskp" Jul 29 19:45:49.331302 ip-10-0-131-93 kubenswrapper[2564]: I0729 19:45:49.331244 2564 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/8926f238-14d4-4e56-8635-95d47c45c78f-credential-socket\") pod \"e2e-migrate-13baf70f-gw-openshift-default-647c89464-7bskp\" (UID: \"8926f238-14d4-4e56-8635-95d47c45c78f\") " pod="openshift-ingress/e2e-migrate-13baf70f-gw-openshift-default-647c89464-7bskp" Jul 29 19:45:49.331607 ip-10-0-131-93 kubenswrapper[2564]: I0729 19:45:49.331590 2564 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/8926f238-14d4-4e56-8635-95d47c45c78f-istiod-ca-cert\") pod \"e2e-migrate-13baf70f-gw-openshift-default-647c89464-7bskp\" (UID: \"8926f238-14d4-4e56-8635-95d47c45c78f\") " pod="openshift-ingress/e2e-migrate-13baf70f-gw-openshift-default-647c89464-7bskp" Jul 29 19:45:49.332924 ip-10-0-131-93 kubenswrapper[2564]: I0729 19:45:49.332908 2564 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/8926f238-14d4-4e56-8635-95d47c45c78f-istio-envoy\") pod \"e2e-migrate-13baf70f-gw-openshift-default-647c89464-7bskp\" (UID: \"8926f238-14d4-4e56-8635-95d47c45c78f\") " pod="openshift-ingress/e2e-migrate-13baf70f-gw-openshift-default-647c89464-7bskp" Jul 29 19:45:49.333171 ip-10-0-131-93 kubenswrapper[2564]: I0729 19:45:49.333154 2564 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/8926f238-14d4-4e56-8635-95d47c45c78f-istio-podinfo\") pod \"e2e-migrate-13baf70f-gw-openshift-default-647c89464-7bskp\" (UID: \"8926f238-14d4-4e56-8635-95d47c45c78f\") " pod="openshift-ingress/e2e-migrate-13baf70f-gw-openshift-default-647c89464-7bskp" Jul 29 19:45:49.343435 ip-10-0-131-93 kubenswrapper[2564]: I0729 19:45:49.343359 2564 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/8926f238-14d4-4e56-8635-95d47c45c78f-istio-token\") pod \"e2e-migrate-13baf70f-gw-openshift-default-647c89464-7bskp\" (UID: \"8926f238-14d4-4e56-8635-95d47c45c78f\") " pod="openshift-ingress/e2e-migrate-13baf70f-gw-openshift-default-647c89464-7bskp" Jul 29 19:45:49.343529 ip-10-0-131-93 kubenswrapper[2564]: I0729 19:45:49.343463 2564 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-64mxv\" (UniqueName: \"kubernetes.io/projected/8926f238-14d4-4e56-8635-95d47c45c78f-kube-api-access-64mxv\") pod \"e2e-migrate-13baf70f-gw-openshift-default-647c89464-7bskp\" (UID: \"8926f238-14d4-4e56-8635-95d47c45c78f\") " pod="openshift-ingress/e2e-migrate-13baf70f-gw-openshift-default-647c89464-7bskp" Jul 29 19:45:49.400559 ip-10-0-131-93 kubenswrapper[2564]: I0729 19:45:49.400526 2564 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-migrate-13baf70f-gw-openshift-default-647c89464-7bskp" Jul 29 19:45:49.556060 ip-10-0-131-93 kubenswrapper[2564]: I0729 19:45:49.556039 2564 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/e2e-migrate-13baf70f-gw-openshift-default-647c89464-7bskp"] Jul 29 19:45:49.557806 ip-10-0-131-93 kubenswrapper[2564]: W0729 19:45:49.557759 2564 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod8926f238_14d4_4e56_8635_95d47c45c78f.slice/crio-d84a31768ec9ebd0126d32c656625ade352177f075b490ff32200f7bdea1fca5 WatchSource:0}: Error finding container d84a31768ec9ebd0126d32c656625ade352177f075b490ff32200f7bdea1fca5: Status 404 returned error can't find the container with id d84a31768ec9ebd0126d32c656625ade352177f075b490ff32200f7bdea1fca5 Jul 29 19:45:49.559777 ip-10-0-131-93 kubenswrapper[2564]: I0729 19:45:49.559747 2564 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236224Ki","pods":"250"} Jul 29 19:45:49.559858 ip-10-0-131-93 kubenswrapper[2564]: I0729 19:45:49.559810 2564 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236224Ki","pods":"250"} Jul 29 19:45:49.559858 ip-10-0-131-93 kubenswrapper[2564]: I0729 19:45:49.559839 2564 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236224Ki","pods":"250"} Jul 29 19:45:49.895648 ip-10-0-131-93 kubenswrapper[2564]: I0729 19:45:49.895571 2564 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-444jx_faa8c546-c75b-4e10-93b4-661aa2b5f53e/konnectivity-agent/0.log" limit=1024 Jul 29 19:45:50.536524 ip-10-0-131-93 kubenswrapper[2564]: I0729 19:45:50.536488 2564 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-migrate-13baf70f-gw-openshift-default-647c89464-7bskp" event={"ID":"8926f238-14d4-4e56-8635-95d47c45c78f","Type":"ContainerStarted","Data":"118be6285f9d54c535c91b6ea1cc08e3c442d616f1f28d607b5a046b18035917"} Jul 29 19:45:50.536728 ip-10-0-131-93 kubenswrapper[2564]: I0729 19:45:50.536538 2564 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-migrate-13baf70f-gw-openshift-default-647c89464-7bskp" event={"ID":"8926f238-14d4-4e56-8635-95d47c45c78f","Type":"ContainerStarted","Data":"d84a31768ec9ebd0126d32c656625ade352177f075b490ff32200f7bdea1fca5"} Jul 29 19:45:50.562168 ip-10-0-131-93 kubenswrapper[2564]: I0729 19:45:50.562117 2564 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-ingress/e2e-migrate-13baf70f-gw-openshift-default-647c89464-7bskp" podStartSLOduration=1.562101321 podStartE2EDuration="1.562101321s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-29 19:45:49 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-29 19:45:50.560622487 +0000 UTC m=+1726.284682183" watchObservedRunningTime="2026-07-29 19:45:50.562101321 +0000 UTC m=+1726.286161089" Jul 29 19:45:51.401450 ip-10-0-131-93 kubenswrapper[2564]: I0729 19:45:51.401417 2564 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="openshift-ingress/e2e-migrate-13baf70f-gw-openshift-default-647c89464-7bskp" Jul 29 19:45:51.406223 ip-10-0-131-93 kubenswrapper[2564]: I0729 19:45:51.406197 2564 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="openshift-ingress/e2e-migrate-13baf70f-gw-openshift-default-647c89464-7bskp" Jul 29 19:45:51.541439 ip-10-0-131-93 kubenswrapper[2564]: I0729 19:45:51.541409 2564 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/e2e-migrate-13baf70f-gw-openshift-default-647c89464-7bskp" Jul 29 19:45:51.542447 ip-10-0-131-93 kubenswrapper[2564]: I0729 19:45:51.542427 2564 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/e2e-migrate-13baf70f-gw-openshift-default-647c89464-7bskp" Jul 29 19:45:51.597343 ip-10-0-131-93 kubenswrapper[2564]: I0729 19:45:51.597310 2564 patch_prober.go:28] interesting pod/e2e-ait-d0aaa6d0-openshift-default-7cb489fc6b-s5n62 container/istio-proxy namespace/openshift-ingress: Readiness probe status=failure output="HTTP probe failed with statuscode: 503" start-of-body= Jul 29 19:45:51.597525 ip-10-0-131-93 kubenswrapper[2564]: I0729 19:45:51.597421 2564 prober.go:120] "Probe failed" probeType="Readiness" pod="openshift-ingress/e2e-ait-d0aaa6d0-openshift-default-7cb489fc6b-s5n62" podUID="c44b5d07-ffb1-4f5e-a949-38d459fecd10" containerName="istio-proxy" probeResult="failure" output="HTTP probe failed with statuscode: 503" Jul 29 19:45:52.033715 ip-10-0-131-93 kubenswrapper[2564]: I0729 19:45:52.033680 2564 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-444jx_faa8c546-c75b-4e10-93b4-661aa2b5f53e/konnectivity-agent/0.log" limit=1024 Jul 29 19:45:52.163055 ip-10-0-131-93 kubenswrapper[2564]: I0729 19:45:52.163026 2564 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-444jx_faa8c546-c75b-4e10-93b4-661aa2b5f53e/konnectivity-agent/0.log" limit=1024 Jul 29 19:45:52.516941 ip-10-0-131-93 kubenswrapper[2564]: I0729 19:45:52.516907 2564 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-444jx_faa8c546-c75b-4e10-93b4-661aa2b5f53e/konnectivity-agent/0.log" limit=1024 Jul 29 19:45:52.864069 ip-10-0-131-93 kubenswrapper[2564]: I0729 19:45:52.863973 2564 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-444jx_faa8c546-c75b-4e10-93b4-661aa2b5f53e/konnectivity-agent/0.log" limit=1024 Jul 29 19:45:52.960109 ip-10-0-131-93 kubenswrapper[2564]: I0729 19:45:52.960074 2564 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-444jx_faa8c546-c75b-4e10-93b4-661aa2b5f53e/konnectivity-agent/0.log" limit=1024 Jul 29 19:45:53.327504 ip-10-0-131-93 kubenswrapper[2564]: I0729 19:45:53.327483 2564 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-ait-d0aaa6d0-openshift-default-7cb489fc6b-s5n62" Jul 29 19:45:53.359632 ip-10-0-131-93 kubenswrapper[2564]: I0729 19:45:53.359606 2564 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/c44b5d07-ffb1-4f5e-a949-38d459fecd10-istio-data\") pod \"c44b5d07-ffb1-4f5e-a949-38d459fecd10\" (UID: \"c44b5d07-ffb1-4f5e-a949-38d459fecd10\") " Jul 29 19:45:53.359783 ip-10-0-131-93 kubenswrapper[2564]: I0729 19:45:53.359658 2564 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/c44b5d07-ffb1-4f5e-a949-38d459fecd10-workload-socket\") pod \"c44b5d07-ffb1-4f5e-a949-38d459fecd10\" (UID: \"c44b5d07-ffb1-4f5e-a949-38d459fecd10\") " Jul 29 19:45:53.359783 ip-10-0-131-93 kubenswrapper[2564]: I0729 19:45:53.359685 2564 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/c44b5d07-ffb1-4f5e-a949-38d459fecd10-istiod-ca-cert\") pod \"c44b5d07-ffb1-4f5e-a949-38d459fecd10\" (UID: \"c44b5d07-ffb1-4f5e-a949-38d459fecd10\") " Jul 29 19:45:53.359783 ip-10-0-131-93 kubenswrapper[2564]: I0729 19:45:53.359712 2564 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/c44b5d07-ffb1-4f5e-a949-38d459fecd10-istio-token\") pod \"c44b5d07-ffb1-4f5e-a949-38d459fecd10\" (UID: \"c44b5d07-ffb1-4f5e-a949-38d459fecd10\") " Jul 29 19:45:53.359783 ip-10-0-131-93 kubenswrapper[2564]: I0729 19:45:53.359736 2564 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/c44b5d07-ffb1-4f5e-a949-38d459fecd10-istio-envoy\") pod \"c44b5d07-ffb1-4f5e-a949-38d459fecd10\" (UID: \"c44b5d07-ffb1-4f5e-a949-38d459fecd10\") " Jul 29 19:45:53.359783 ip-10-0-131-93 kubenswrapper[2564]: I0729 19:45:53.359774 2564 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/c44b5d07-ffb1-4f5e-a949-38d459fecd10-credential-socket\") pod \"c44b5d07-ffb1-4f5e-a949-38d459fecd10\" (UID: \"c44b5d07-ffb1-4f5e-a949-38d459fecd10\") " Jul 29 19:45:53.360048 ip-10-0-131-93 kubenswrapper[2564]: I0729 19:45:53.359803 2564 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/c44b5d07-ffb1-4f5e-a949-38d459fecd10-workload-certs\") pod \"c44b5d07-ffb1-4f5e-a949-38d459fecd10\" (UID: \"c44b5d07-ffb1-4f5e-a949-38d459fecd10\") " Jul 29 19:45:53.360048 ip-10-0-131-93 kubenswrapper[2564]: I0729 19:45:53.359872 2564 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-48z98\" (UniqueName: \"kubernetes.io/projected/c44b5d07-ffb1-4f5e-a949-38d459fecd10-kube-api-access-48z98\") pod \"c44b5d07-ffb1-4f5e-a949-38d459fecd10\" (UID: \"c44b5d07-ffb1-4f5e-a949-38d459fecd10\") " Jul 29 19:45:53.360048 ip-10-0-131-93 kubenswrapper[2564]: I0729 19:45:53.359898 2564 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/c44b5d07-ffb1-4f5e-a949-38d459fecd10-istio-podinfo\") pod \"c44b5d07-ffb1-4f5e-a949-38d459fecd10\" (UID: \"c44b5d07-ffb1-4f5e-a949-38d459fecd10\") " Jul 29 19:45:53.360229 ip-10-0-131-93 kubenswrapper[2564]: I0729 19:45:53.360203 2564 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/c44b5d07-ffb1-4f5e-a949-38d459fecd10-istiod-ca-cert" (OuterVolumeSpecName: "istiod-ca-cert") pod "c44b5d07-ffb1-4f5e-a949-38d459fecd10" (UID: "c44b5d07-ffb1-4f5e-a949-38d459fecd10"). InnerVolumeSpecName "istiod-ca-cert". PluginName "kubernetes.io/configmap", VolumeGIDValue "" Jul 29 19:45:53.360488 ip-10-0-131-93 kubenswrapper[2564]: I0729 19:45:53.360463 2564 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/c44b5d07-ffb1-4f5e-a949-38d459fecd10-credential-socket" (OuterVolumeSpecName: "credential-socket") pod "c44b5d07-ffb1-4f5e-a949-38d459fecd10" (UID: "c44b5d07-ffb1-4f5e-a949-38d459fecd10"). InnerVolumeSpecName "credential-socket". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Jul 29 19:45:53.360584 ip-10-0-131-93 kubenswrapper[2564]: I0729 19:45:53.360483 2564 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/c44b5d07-ffb1-4f5e-a949-38d459fecd10-workload-certs" (OuterVolumeSpecName: "workload-certs") pod "c44b5d07-ffb1-4f5e-a949-38d459fecd10" (UID: "c44b5d07-ffb1-4f5e-a949-38d459fecd10"). InnerVolumeSpecName "workload-certs". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Jul 29 19:45:53.360584 ip-10-0-131-93 kubenswrapper[2564]: I0729 19:45:53.360546 2564 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/c44b5d07-ffb1-4f5e-a949-38d459fecd10-workload-socket" (OuterVolumeSpecName: "workload-socket") pod "c44b5d07-ffb1-4f5e-a949-38d459fecd10" (UID: "c44b5d07-ffb1-4f5e-a949-38d459fecd10"). InnerVolumeSpecName "workload-socket". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Jul 29 19:45:53.360703 ip-10-0-131-93 kubenswrapper[2564]: I0729 19:45:53.360624 2564 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/c44b5d07-ffb1-4f5e-a949-38d459fecd10-istio-data" (OuterVolumeSpecName: "istio-data") pod "c44b5d07-ffb1-4f5e-a949-38d459fecd10" (UID: "c44b5d07-ffb1-4f5e-a949-38d459fecd10"). InnerVolumeSpecName "istio-data". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Jul 29 19:45:53.362417 ip-10-0-131-93 kubenswrapper[2564]: I0729 19:45:53.362352 2564 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/downward-api/c44b5d07-ffb1-4f5e-a949-38d459fecd10-istio-podinfo" (OuterVolumeSpecName: "istio-podinfo") pod "c44b5d07-ffb1-4f5e-a949-38d459fecd10" (UID: "c44b5d07-ffb1-4f5e-a949-38d459fecd10"). InnerVolumeSpecName "istio-podinfo". PluginName "kubernetes.io/downward-api", VolumeGIDValue "" Jul 29 19:45:53.362515 ip-10-0-131-93 kubenswrapper[2564]: I0729 19:45:53.362492 2564 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/c44b5d07-ffb1-4f5e-a949-38d459fecd10-istio-token" (OuterVolumeSpecName: "istio-token") pod "c44b5d07-ffb1-4f5e-a949-38d459fecd10" (UID: "c44b5d07-ffb1-4f5e-a949-38d459fecd10"). InnerVolumeSpecName "istio-token". PluginName "kubernetes.io/projected", VolumeGIDValue "" Jul 29 19:45:53.362596 ip-10-0-131-93 kubenswrapper[2564]: I0729 19:45:53.362575 2564 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/c44b5d07-ffb1-4f5e-a949-38d459fecd10-istio-envoy" (OuterVolumeSpecName: "istio-envoy") pod "c44b5d07-ffb1-4f5e-a949-38d459fecd10" (UID: "c44b5d07-ffb1-4f5e-a949-38d459fecd10"). InnerVolumeSpecName "istio-envoy". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Jul 29 19:45:53.362658 ip-10-0-131-93 kubenswrapper[2564]: I0729 19:45:53.362618 2564 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/c44b5d07-ffb1-4f5e-a949-38d459fecd10-kube-api-access-48z98" (OuterVolumeSpecName: "kube-api-access-48z98") pod "c44b5d07-ffb1-4f5e-a949-38d459fecd10" (UID: "c44b5d07-ffb1-4f5e-a949-38d459fecd10"). InnerVolumeSpecName "kube-api-access-48z98". PluginName "kubernetes.io/projected", VolumeGIDValue "" Jul 29 19:45:53.460598 ip-10-0-131-93 kubenswrapper[2564]: I0729 19:45:53.460515 2564 reconciler_common.go:299] "Volume detached for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/c44b5d07-ffb1-4f5e-a949-38d459fecd10-istio-data\") on node \"ip-10-0-131-93.ec2.internal\" DevicePath \"\"" Jul 29 19:45:53.460598 ip-10-0-131-93 kubenswrapper[2564]: I0729 19:45:53.460542 2564 reconciler_common.go:299] "Volume detached for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/c44b5d07-ffb1-4f5e-a949-38d459fecd10-workload-socket\") on node \"ip-10-0-131-93.ec2.internal\" DevicePath \"\"" Jul 29 19:45:53.460598 ip-10-0-131-93 kubenswrapper[2564]: I0729 19:45:53.460551 2564 reconciler_common.go:299] "Volume detached for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/c44b5d07-ffb1-4f5e-a949-38d459fecd10-istiod-ca-cert\") on node \"ip-10-0-131-93.ec2.internal\" DevicePath \"\"" Jul 29 19:45:53.460598 ip-10-0-131-93 kubenswrapper[2564]: I0729 19:45:53.460559 2564 reconciler_common.go:299] "Volume detached for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/c44b5d07-ffb1-4f5e-a949-38d459fecd10-istio-token\") on node \"ip-10-0-131-93.ec2.internal\" DevicePath \"\"" Jul 29 19:45:53.460598 ip-10-0-131-93 kubenswrapper[2564]: I0729 19:45:53.460567 2564 reconciler_common.go:299] "Volume detached for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/c44b5d07-ffb1-4f5e-a949-38d459fecd10-istio-envoy\") on node \"ip-10-0-131-93.ec2.internal\" DevicePath \"\"" Jul 29 19:45:53.460598 ip-10-0-131-93 kubenswrapper[2564]: I0729 19:45:53.460575 2564 reconciler_common.go:299] "Volume detached for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/c44b5d07-ffb1-4f5e-a949-38d459fecd10-credential-socket\") on node \"ip-10-0-131-93.ec2.internal\" DevicePath \"\"" Jul 29 19:45:53.460598 ip-10-0-131-93 kubenswrapper[2564]: I0729 19:45:53.460584 2564 reconciler_common.go:299] "Volume detached for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/c44b5d07-ffb1-4f5e-a949-38d459fecd10-workload-certs\") on node \"ip-10-0-131-93.ec2.internal\" DevicePath \"\"" Jul 29 19:45:53.460598 ip-10-0-131-93 kubenswrapper[2564]: I0729 19:45:53.460594 2564 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-48z98\" (UniqueName: \"kubernetes.io/projected/c44b5d07-ffb1-4f5e-a949-38d459fecd10-kube-api-access-48z98\") on node \"ip-10-0-131-93.ec2.internal\" DevicePath \"\"" Jul 29 19:45:53.460598 ip-10-0-131-93 kubenswrapper[2564]: I0729 19:45:53.460602 2564 reconciler_common.go:299] "Volume detached for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/c44b5d07-ffb1-4f5e-a949-38d459fecd10-istio-podinfo\") on node \"ip-10-0-131-93.ec2.internal\" DevicePath \"\"" Jul 29 19:45:53.552057 ip-10-0-131-93 kubenswrapper[2564]: I0729 19:45:53.552024 2564 generic.go:356] "Generic (PLEG): container finished" podID="c44b5d07-ffb1-4f5e-a949-38d459fecd10" containerID="2da5bd91403cdaa55591391fec8d3991c94847d15ef7f63eb34ce4d2ebfe8f42" exitCode=0 Jul 29 19:45:53.552542 ip-10-0-131-93 kubenswrapper[2564]: I0729 19:45:53.552079 2564 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-ait-d0aaa6d0-openshift-default-7cb489fc6b-s5n62" Jul 29 19:45:53.552542 ip-10-0-131-93 kubenswrapper[2564]: I0729 19:45:53.552121 2564 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-ait-d0aaa6d0-openshift-default-7cb489fc6b-s5n62" event={"ID":"c44b5d07-ffb1-4f5e-a949-38d459fecd10","Type":"ContainerDied","Data":"2da5bd91403cdaa55591391fec8d3991c94847d15ef7f63eb34ce4d2ebfe8f42"} Jul 29 19:45:53.552542 ip-10-0-131-93 kubenswrapper[2564]: I0729 19:45:53.552188 2564 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-ait-d0aaa6d0-openshift-default-7cb489fc6b-s5n62" event={"ID":"c44b5d07-ffb1-4f5e-a949-38d459fecd10","Type":"ContainerDied","Data":"bfb8a5691e80d2120e833953755c49c69f8630040e429155bc9f5e47567d383f"} Jul 29 19:45:53.552542 ip-10-0-131-93 kubenswrapper[2564]: I0729 19:45:53.552225 2564 scope.go:117] "RemoveContainer" containerID="2da5bd91403cdaa55591391fec8d3991c94847d15ef7f63eb34ce4d2ebfe8f42" Jul 29 19:45:53.561944 ip-10-0-131-93 kubenswrapper[2564]: I0729 19:45:53.561923 2564 scope.go:117] "RemoveContainer" containerID="a25c0bf9433b7908b5ed235d8f28e1e4f8c5ca652569b20b90931b3b2a880ea2" Jul 29 19:45:53.570493 ip-10-0-131-93 kubenswrapper[2564]: I0729 19:45:53.570463 2564 scope.go:117] "RemoveContainer" containerID="2da5bd91403cdaa55591391fec8d3991c94847d15ef7f63eb34ce4d2ebfe8f42" Jul 29 19:45:53.570750 ip-10-0-131-93 kubenswrapper[2564]: E0729 19:45:53.570728 2564 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"2da5bd91403cdaa55591391fec8d3991c94847d15ef7f63eb34ce4d2ebfe8f42\": container with ID starting with 2da5bd91403cdaa55591391fec8d3991c94847d15ef7f63eb34ce4d2ebfe8f42 not found: ID does not exist" containerID="2da5bd91403cdaa55591391fec8d3991c94847d15ef7f63eb34ce4d2ebfe8f42" Jul 29 19:45:53.570793 ip-10-0-131-93 kubenswrapper[2564]: I0729 19:45:53.570760 2564 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"2da5bd91403cdaa55591391fec8d3991c94847d15ef7f63eb34ce4d2ebfe8f42"} err="failed to get container status \"2da5bd91403cdaa55591391fec8d3991c94847d15ef7f63eb34ce4d2ebfe8f42\": rpc error: code = NotFound desc = could not find container \"2da5bd91403cdaa55591391fec8d3991c94847d15ef7f63eb34ce4d2ebfe8f42\": container with ID starting with 2da5bd91403cdaa55591391fec8d3991c94847d15ef7f63eb34ce4d2ebfe8f42 not found: ID does not exist" Jul 29 19:45:53.570793 ip-10-0-131-93 kubenswrapper[2564]: I0729 19:45:53.570776 2564 scope.go:117] "RemoveContainer" containerID="a25c0bf9433b7908b5ed235d8f28e1e4f8c5ca652569b20b90931b3b2a880ea2" Jul 29 19:45:53.570980 ip-10-0-131-93 kubenswrapper[2564]: E0729 19:45:53.570964 2564 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"a25c0bf9433b7908b5ed235d8f28e1e4f8c5ca652569b20b90931b3b2a880ea2\": container with ID starting with a25c0bf9433b7908b5ed235d8f28e1e4f8c5ca652569b20b90931b3b2a880ea2 not found: ID does not exist" containerID="a25c0bf9433b7908b5ed235d8f28e1e4f8c5ca652569b20b90931b3b2a880ea2" Jul 29 19:45:53.571024 ip-10-0-131-93 kubenswrapper[2564]: I0729 19:45:53.570988 2564 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"a25c0bf9433b7908b5ed235d8f28e1e4f8c5ca652569b20b90931b3b2a880ea2"} err="failed to get container status \"a25c0bf9433b7908b5ed235d8f28e1e4f8c5ca652569b20b90931b3b2a880ea2\": rpc error: code = NotFound desc = could not find container \"a25c0bf9433b7908b5ed235d8f28e1e4f8c5ca652569b20b90931b3b2a880ea2\": container with ID starting with a25c0bf9433b7908b5ed235d8f28e1e4f8c5ca652569b20b90931b3b2a880ea2 not found: ID does not exist" Jul 29 19:45:53.575661 ip-10-0-131-93 kubenswrapper[2564]: I0729 19:45:53.575638 2564 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-ingress/e2e-ait-d0aaa6d0-openshift-default-7cb489fc6b-s5n62"] Jul 29 19:45:53.581531 ip-10-0-131-93 kubenswrapper[2564]: I0729 19:45:53.581509 2564 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["openshift-ingress/e2e-ait-d0aaa6d0-openshift-default-7cb489fc6b-s5n62"] Jul 29 19:45:54.784494 ip-10-0-131-93 kubenswrapper[2564]: I0729 19:45:54.784460 2564 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="c44b5d07-ffb1-4f5e-a949-38d459fecd10" path="/var/lib/kubelet/pods/c44b5d07-ffb1-4f5e-a949-38d459fecd10/volumes" Jul 29 19:46:04.996003 ip-10-0-131-93 kubenswrapper[2564]: I0729 19:46:04.995962 2564 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-444jx_faa8c546-c75b-4e10-93b4-661aa2b5f53e/konnectivity-agent/0.log" limit=1024 Jul 29 19:46:05.623991 ip-10-0-131-93 kubenswrapper[2564]: I0729 19:46:05.623892 2564 generic.go:356] "Generic (PLEG): container finished" podID="8926f238-14d4-4e56-8635-95d47c45c78f" containerID="118be6285f9d54c535c91b6ea1cc08e3c442d616f1f28d607b5a046b18035917" exitCode=0 Jul 29 19:46:05.623991 ip-10-0-131-93 kubenswrapper[2564]: I0729 19:46:05.623965 2564 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-migrate-13baf70f-gw-openshift-default-647c89464-7bskp" event={"ID":"8926f238-14d4-4e56-8635-95d47c45c78f","Type":"ContainerDied","Data":"118be6285f9d54c535c91b6ea1cc08e3c442d616f1f28d607b5a046b18035917"} Jul 29 19:46:05.624341 ip-10-0-131-93 kubenswrapper[2564]: I0729 19:46:05.624326 2564 scope.go:117] "RemoveContainer" containerID="118be6285f9d54c535c91b6ea1cc08e3c442d616f1f28d607b5a046b18035917" Jul 29 19:46:05.625294 ip-10-0-131-93 kubenswrapper[2564]: I0729 19:46:05.625266 2564 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236224Ki","pods":"250"} Jul 29 19:46:05.625421 ip-10-0-131-93 kubenswrapper[2564]: I0729 19:46:05.625330 2564 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236224Ki","pods":"250"} Jul 29 19:46:05.625475 ip-10-0-131-93 kubenswrapper[2564]: I0729 19:46:05.625434 2564 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236224Ki","pods":"250"} Jul 29 19:46:06.541587 ip-10-0-131-93 kubenswrapper[2564]: I0729 19:46:06.541552 2564 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/e2e-migrate-13baf70f-gw-openshift-default-647c89464-7bskp" Jul 29 19:46:06.630693 ip-10-0-131-93 kubenswrapper[2564]: I0729 19:46:06.630656 2564 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-migrate-13baf70f-gw-openshift-default-647c89464-7bskp" event={"ID":"8926f238-14d4-4e56-8635-95d47c45c78f","Type":"ContainerStarted","Data":"a9c940a8426569768e015edc75b0a5d1ad9f680fe4da55a2178ddc1a5e1d4322"} Jul 29 19:46:07.401222 ip-10-0-131-93 kubenswrapper[2564]: I0729 19:46:07.401175 2564 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="openshift-ingress/e2e-migrate-13baf70f-gw-openshift-default-647c89464-7bskp" Jul 29 19:46:07.402508 ip-10-0-131-93 kubenswrapper[2564]: I0729 19:46:07.402473 2564 patch_prober.go:28] interesting pod/e2e-migrate-13baf70f-gw-openshift-default-647c89464-7bskp container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.132.0.38:15021/healthz/ready\": dial tcp 10.132.0.38:15021: connect: connection refused" start-of-body= Jul 29 19:46:07.402657 ip-10-0-131-93 kubenswrapper[2564]: I0729 19:46:07.402528 2564 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-migrate-13baf70f-gw-openshift-default-647c89464-7bskp" podUID="8926f238-14d4-4e56-8635-95d47c45c78f" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.132.0.38:15021/healthz/ready\": dial tcp 10.132.0.38:15021: connect: connection refused" Jul 29 19:46:08.401690 ip-10-0-131-93 kubenswrapper[2564]: I0729 19:46:08.401655 2564 patch_prober.go:28] interesting pod/e2e-migrate-13baf70f-gw-openshift-default-647c89464-7bskp container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.132.0.38:15021/healthz/ready\": dial tcp 10.132.0.38:15021: connect: connection refused" start-of-body= Jul 29 19:46:08.402106 ip-10-0-131-93 kubenswrapper[2564]: I0729 19:46:08.401718 2564 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-migrate-13baf70f-gw-openshift-default-647c89464-7bskp" podUID="8926f238-14d4-4e56-8635-95d47c45c78f" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.132.0.38:15021/healthz/ready\": dial tcp 10.132.0.38:15021: connect: connection refused" Jul 29 19:46:09.401642 ip-10-0-131-93 kubenswrapper[2564]: I0729 19:46:09.401602 2564 patch_prober.go:28] interesting pod/e2e-migrate-13baf70f-gw-openshift-default-647c89464-7bskp container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.132.0.38:15021/healthz/ready\": dial tcp 10.132.0.38:15021: connect: connection refused" start-of-body= Jul 29 19:46:09.401842 ip-10-0-131-93 kubenswrapper[2564]: I0729 19:46:09.401659 2564 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-migrate-13baf70f-gw-openshift-default-647c89464-7bskp" podUID="8926f238-14d4-4e56-8635-95d47c45c78f" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.132.0.38:15021/healthz/ready\": dial tcp 10.132.0.38:15021: connect: connection refused" Jul 29 19:46:10.401359 ip-10-0-131-93 kubenswrapper[2564]: I0729 19:46:10.401322 2564 patch_prober.go:28] interesting pod/e2e-migrate-13baf70f-gw-openshift-default-647c89464-7bskp container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.132.0.38:15021/healthz/ready\": dial tcp 10.132.0.38:15021: connect: connection refused" start-of-body= Jul 29 19:46:10.401593 ip-10-0-131-93 kubenswrapper[2564]: I0729 19:46:10.401399 2564 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-migrate-13baf70f-gw-openshift-default-647c89464-7bskp" podUID="8926f238-14d4-4e56-8635-95d47c45c78f" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.132.0.38:15021/healthz/ready\": dial tcp 10.132.0.38:15021: connect: connection refused" Jul 29 19:46:11.405117 ip-10-0-131-93 kubenswrapper[2564]: I0729 19:46:11.405086 2564 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="openshift-ingress/e2e-migrate-13baf70f-gw-openshift-default-647c89464-7bskp" Jul 29 19:46:11.405601 ip-10-0-131-93 kubenswrapper[2564]: I0729 19:46:11.405487 2564 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/e2e-migrate-13baf70f-gw-openshift-default-647c89464-7bskp" Jul 29 19:46:11.406204 ip-10-0-131-93 kubenswrapper[2564]: I0729 19:46:11.406187 2564 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/e2e-migrate-13baf70f-gw-openshift-default-647c89464-7bskp" Jul 29 19:46:23.485338 ip-10-0-131-93 kubenswrapper[2564]: I0729 19:46:23.485298 2564 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-444jx_faa8c546-c75b-4e10-93b4-661aa2b5f53e/konnectivity-agent/0.log" limit=1024 Jul 29 19:46:23.559513 ip-10-0-131-93 kubenswrapper[2564]: I0729 19:46:23.559481 2564 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-444jx_faa8c546-c75b-4e10-93b4-661aa2b5f53e/konnectivity-agent/0.log" limit=1024 Jul 29 19:46:23.617917 ip-10-0-131-93 kubenswrapper[2564]: I0729 19:46:23.617876 2564 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-444jx_faa8c546-c75b-4e10-93b4-661aa2b5f53e/konnectivity-agent/0.log" limit=1024 Jul 29 19:46:23.676281 ip-10-0-131-93 kubenswrapper[2564]: I0729 19:46:23.676249 2564 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-444jx_faa8c546-c75b-4e10-93b4-661aa2b5f53e/konnectivity-agent/0.log" limit=1024 Jul 29 19:46:23.830461 ip-10-0-131-93 kubenswrapper[2564]: I0729 19:46:23.830363 2564 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-ingress/e2e-migrate-13baf70f-gw-openshift-default-647c89464-7bskp"] Jul 29 19:46:23.830607 ip-10-0-131-93 kubenswrapper[2564]: I0729 19:46:23.830584 2564 kuberuntime_container.go:873] "Killing container with a grace period" pod="openshift-ingress/e2e-migrate-13baf70f-gw-openshift-default-647c89464-7bskp" podUID="8926f238-14d4-4e56-8635-95d47c45c78f" containerName="istio-proxy" containerID="cri-o://a9c940a8426569768e015edc75b0a5d1ad9f680fe4da55a2178ddc1a5e1d4322" gracePeriod=30 Jul 29 19:46:23.846525 ip-10-0-131-93 kubenswrapper[2564]: I0729 19:46:23.846498 2564 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-444jx_faa8c546-c75b-4e10-93b4-661aa2b5f53e/konnectivity-agent/0.log" limit=1024 Jul 29 19:46:24.051256 ip-10-0-131-93 kubenswrapper[2564]: I0729 19:46:24.051216 2564 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-444jx_faa8c546-c75b-4e10-93b4-661aa2b5f53e/konnectivity-agent/0.log" limit=1024 Jul 29 19:46:24.277315 ip-10-0-131-93 kubenswrapper[2564]: I0729 19:46:24.277279 2564 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-444jx_faa8c546-c75b-4e10-93b4-661aa2b5f53e/konnectivity-agent/0.log" limit=1024 Jul 29 19:46:24.569911 ip-10-0-131-93 kubenswrapper[2564]: I0729 19:46:24.569834 2564 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-ingress/e2e-ait-a4a71d55-openshift-default-c9c5dc8bd-rl5p8"] Jul 29 19:46:24.570255 ip-10-0-131-93 kubenswrapper[2564]: I0729 19:46:24.570213 2564 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="c44b5d07-ffb1-4f5e-a949-38d459fecd10" containerName="istio-proxy" Jul 29 19:46:24.570255 ip-10-0-131-93 kubenswrapper[2564]: I0729 19:46:24.570226 2564 state_mem.go:107] "Deleted CPUSet assignment" podUID="c44b5d07-ffb1-4f5e-a949-38d459fecd10" containerName="istio-proxy" Jul 29 19:46:24.570255 ip-10-0-131-93 kubenswrapper[2564]: I0729 19:46:24.570239 2564 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="c44b5d07-ffb1-4f5e-a949-38d459fecd10" containerName="istio-proxy" Jul 29 19:46:24.570255 ip-10-0-131-93 kubenswrapper[2564]: I0729 19:46:24.570245 2564 state_mem.go:107] "Deleted CPUSet assignment" podUID="c44b5d07-ffb1-4f5e-a949-38d459fecd10" containerName="istio-proxy" Jul 29 19:46:24.570420 ip-10-0-131-93 kubenswrapper[2564]: I0729 19:46:24.570307 2564 memory_manager.go:368] "RemoveStaleState removing state" podUID="c44b5d07-ffb1-4f5e-a949-38d459fecd10" containerName="istio-proxy" Jul 29 19:46:24.570420 ip-10-0-131-93 kubenswrapper[2564]: I0729 19:46:24.570318 2564 memory_manager.go:368] "RemoveStaleState removing state" podUID="c44b5d07-ffb1-4f5e-a949-38d459fecd10" containerName="istio-proxy" Jul 29 19:46:24.572891 ip-10-0-131-93 kubenswrapper[2564]: I0729 19:46:24.572865 2564 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-ait-a4a71d55-openshift-default-c9c5dc8bd-rl5p8" Jul 29 19:46:24.577559 ip-10-0-131-93 kubenswrapper[2564]: I0729 19:46:24.577536 2564 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress\"/\"e2e-ait-a4a71d55-openshift-default-dockercfg-rpm6k\"" Jul 29 19:46:24.587740 ip-10-0-131-93 kubenswrapper[2564]: I0729 19:46:24.587565 2564 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/e2e-ait-a4a71d55-openshift-default-c9c5dc8bd-rl5p8"] Jul 29 19:46:24.591120 ip-10-0-131-93 kubenswrapper[2564]: I0729 19:46:24.591092 2564 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-444jx_faa8c546-c75b-4e10-93b4-661aa2b5f53e/konnectivity-agent/0.log" limit=1024 Jul 29 19:46:24.641136 ip-10-0-131-93 kubenswrapper[2564]: I0729 19:46:24.641100 2564 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-d56p5\" (UniqueName: \"kubernetes.io/projected/4e980d2a-9b00-4791-a563-78cb03cf6697-kube-api-access-d56p5\") pod \"e2e-ait-a4a71d55-openshift-default-c9c5dc8bd-rl5p8\" (UID: \"4e980d2a-9b00-4791-a563-78cb03cf6697\") " pod="openshift-ingress/e2e-ait-a4a71d55-openshift-default-c9c5dc8bd-rl5p8" Jul 29 19:46:24.641136 ip-10-0-131-93 kubenswrapper[2564]: I0729 19:46:24.641136 2564 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/4e980d2a-9b00-4791-a563-78cb03cf6697-istio-podinfo\") pod \"e2e-ait-a4a71d55-openshift-default-c9c5dc8bd-rl5p8\" (UID: \"4e980d2a-9b00-4791-a563-78cb03cf6697\") " pod="openshift-ingress/e2e-ait-a4a71d55-openshift-default-c9c5dc8bd-rl5p8" Jul 29 19:46:24.641405 ip-10-0-131-93 kubenswrapper[2564]: I0729 19:46:24.641154 2564 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/4e980d2a-9b00-4791-a563-78cb03cf6697-istiod-ca-cert\") pod \"e2e-ait-a4a71d55-openshift-default-c9c5dc8bd-rl5p8\" (UID: \"4e980d2a-9b00-4791-a563-78cb03cf6697\") " pod="openshift-ingress/e2e-ait-a4a71d55-openshift-default-c9c5dc8bd-rl5p8" Jul 29 19:46:24.641405 ip-10-0-131-93 kubenswrapper[2564]: I0729 19:46:24.641183 2564 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/4e980d2a-9b00-4791-a563-78cb03cf6697-workload-certs\") pod \"e2e-ait-a4a71d55-openshift-default-c9c5dc8bd-rl5p8\" (UID: \"4e980d2a-9b00-4791-a563-78cb03cf6697\") " pod="openshift-ingress/e2e-ait-a4a71d55-openshift-default-c9c5dc8bd-rl5p8" Jul 29 19:46:24.641405 ip-10-0-131-93 kubenswrapper[2564]: I0729 19:46:24.641279 2564 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/4e980d2a-9b00-4791-a563-78cb03cf6697-credential-socket\") pod \"e2e-ait-a4a71d55-openshift-default-c9c5dc8bd-rl5p8\" (UID: \"4e980d2a-9b00-4791-a563-78cb03cf6697\") " pod="openshift-ingress/e2e-ait-a4a71d55-openshift-default-c9c5dc8bd-rl5p8" Jul 29 19:46:24.641405 ip-10-0-131-93 kubenswrapper[2564]: I0729 19:46:24.641329 2564 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/4e980d2a-9b00-4791-a563-78cb03cf6697-istio-token\") pod \"e2e-ait-a4a71d55-openshift-default-c9c5dc8bd-rl5p8\" (UID: \"4e980d2a-9b00-4791-a563-78cb03cf6697\") " pod="openshift-ingress/e2e-ait-a4a71d55-openshift-default-c9c5dc8bd-rl5p8" Jul 29 19:46:24.641645 ip-10-0-131-93 kubenswrapper[2564]: I0729 19:46:24.641433 2564 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/4e980d2a-9b00-4791-a563-78cb03cf6697-workload-socket\") pod \"e2e-ait-a4a71d55-openshift-default-c9c5dc8bd-rl5p8\" (UID: \"4e980d2a-9b00-4791-a563-78cb03cf6697\") " pod="openshift-ingress/e2e-ait-a4a71d55-openshift-default-c9c5dc8bd-rl5p8" Jul 29 19:46:24.641645 ip-10-0-131-93 kubenswrapper[2564]: I0729 19:46:24.641486 2564 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/4e980d2a-9b00-4791-a563-78cb03cf6697-istio-data\") pod \"e2e-ait-a4a71d55-openshift-default-c9c5dc8bd-rl5p8\" (UID: \"4e980d2a-9b00-4791-a563-78cb03cf6697\") " pod="openshift-ingress/e2e-ait-a4a71d55-openshift-default-c9c5dc8bd-rl5p8" Jul 29 19:46:24.641645 ip-10-0-131-93 kubenswrapper[2564]: I0729 19:46:24.641517 2564 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/4e980d2a-9b00-4791-a563-78cb03cf6697-istio-envoy\") pod \"e2e-ait-a4a71d55-openshift-default-c9c5dc8bd-rl5p8\" (UID: \"4e980d2a-9b00-4791-a563-78cb03cf6697\") " pod="openshift-ingress/e2e-ait-a4a71d55-openshift-default-c9c5dc8bd-rl5p8" Jul 29 19:46:24.742352 ip-10-0-131-93 kubenswrapper[2564]: I0729 19:46:24.742319 2564 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/4e980d2a-9b00-4791-a563-78cb03cf6697-workload-socket\") pod \"e2e-ait-a4a71d55-openshift-default-c9c5dc8bd-rl5p8\" (UID: \"4e980d2a-9b00-4791-a563-78cb03cf6697\") " pod="openshift-ingress/e2e-ait-a4a71d55-openshift-default-c9c5dc8bd-rl5p8" Jul 29 19:46:24.742551 ip-10-0-131-93 kubenswrapper[2564]: I0729 19:46:24.742366 2564 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/4e980d2a-9b00-4791-a563-78cb03cf6697-istio-data\") pod \"e2e-ait-a4a71d55-openshift-default-c9c5dc8bd-rl5p8\" (UID: \"4e980d2a-9b00-4791-a563-78cb03cf6697\") " pod="openshift-ingress/e2e-ait-a4a71d55-openshift-default-c9c5dc8bd-rl5p8" Jul 29 19:46:24.742551 ip-10-0-131-93 kubenswrapper[2564]: I0729 19:46:24.742421 2564 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/4e980d2a-9b00-4791-a563-78cb03cf6697-istio-envoy\") pod \"e2e-ait-a4a71d55-openshift-default-c9c5dc8bd-rl5p8\" (UID: \"4e980d2a-9b00-4791-a563-78cb03cf6697\") " pod="openshift-ingress/e2e-ait-a4a71d55-openshift-default-c9c5dc8bd-rl5p8" Jul 29 19:46:24.742551 ip-10-0-131-93 kubenswrapper[2564]: I0729 19:46:24.742448 2564 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-d56p5\" (UniqueName: \"kubernetes.io/projected/4e980d2a-9b00-4791-a563-78cb03cf6697-kube-api-access-d56p5\") pod \"e2e-ait-a4a71d55-openshift-default-c9c5dc8bd-rl5p8\" (UID: \"4e980d2a-9b00-4791-a563-78cb03cf6697\") " pod="openshift-ingress/e2e-ait-a4a71d55-openshift-default-c9c5dc8bd-rl5p8" Jul 29 19:46:24.742551 ip-10-0-131-93 kubenswrapper[2564]: I0729 19:46:24.742469 2564 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/4e980d2a-9b00-4791-a563-78cb03cf6697-istio-podinfo\") pod \"e2e-ait-a4a71d55-openshift-default-c9c5dc8bd-rl5p8\" (UID: \"4e980d2a-9b00-4791-a563-78cb03cf6697\") " pod="openshift-ingress/e2e-ait-a4a71d55-openshift-default-c9c5dc8bd-rl5p8" Jul 29 19:46:24.742551 ip-10-0-131-93 kubenswrapper[2564]: I0729 19:46:24.742494 2564 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/4e980d2a-9b00-4791-a563-78cb03cf6697-istiod-ca-cert\") pod \"e2e-ait-a4a71d55-openshift-default-c9c5dc8bd-rl5p8\" (UID: \"4e980d2a-9b00-4791-a563-78cb03cf6697\") " pod="openshift-ingress/e2e-ait-a4a71d55-openshift-default-c9c5dc8bd-rl5p8" Jul 29 19:46:24.742551 ip-10-0-131-93 kubenswrapper[2564]: I0729 19:46:24.742519 2564 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/4e980d2a-9b00-4791-a563-78cb03cf6697-workload-certs\") pod \"e2e-ait-a4a71d55-openshift-default-c9c5dc8bd-rl5p8\" (UID: \"4e980d2a-9b00-4791-a563-78cb03cf6697\") " pod="openshift-ingress/e2e-ait-a4a71d55-openshift-default-c9c5dc8bd-rl5p8" Jul 29 19:46:24.742894 ip-10-0-131-93 kubenswrapper[2564]: I0729 19:46:24.742565 2564 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/4e980d2a-9b00-4791-a563-78cb03cf6697-credential-socket\") pod \"e2e-ait-a4a71d55-openshift-default-c9c5dc8bd-rl5p8\" (UID: \"4e980d2a-9b00-4791-a563-78cb03cf6697\") " pod="openshift-ingress/e2e-ait-a4a71d55-openshift-default-c9c5dc8bd-rl5p8" Jul 29 19:46:24.742894 ip-10-0-131-93 kubenswrapper[2564]: I0729 19:46:24.742605 2564 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/4e980d2a-9b00-4791-a563-78cb03cf6697-istio-token\") pod \"e2e-ait-a4a71d55-openshift-default-c9c5dc8bd-rl5p8\" (UID: \"4e980d2a-9b00-4791-a563-78cb03cf6697\") " pod="openshift-ingress/e2e-ait-a4a71d55-openshift-default-c9c5dc8bd-rl5p8" Jul 29 19:46:24.742894 ip-10-0-131-93 kubenswrapper[2564]: I0729 19:46:24.742795 2564 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/4e980d2a-9b00-4791-a563-78cb03cf6697-workload-socket\") pod \"e2e-ait-a4a71d55-openshift-default-c9c5dc8bd-rl5p8\" (UID: \"4e980d2a-9b00-4791-a563-78cb03cf6697\") " pod="openshift-ingress/e2e-ait-a4a71d55-openshift-default-c9c5dc8bd-rl5p8" Jul 29 19:46:24.742894 ip-10-0-131-93 kubenswrapper[2564]: I0729 19:46:24.742807 2564 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/4e980d2a-9b00-4791-a563-78cb03cf6697-istio-data\") pod \"e2e-ait-a4a71d55-openshift-default-c9c5dc8bd-rl5p8\" (UID: \"4e980d2a-9b00-4791-a563-78cb03cf6697\") " pod="openshift-ingress/e2e-ait-a4a71d55-openshift-default-c9c5dc8bd-rl5p8" Jul 29 19:46:24.742894 ip-10-0-131-93 kubenswrapper[2564]: I0729 19:46:24.742875 2564 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/4e980d2a-9b00-4791-a563-78cb03cf6697-workload-certs\") pod \"e2e-ait-a4a71d55-openshift-default-c9c5dc8bd-rl5p8\" (UID: \"4e980d2a-9b00-4791-a563-78cb03cf6697\") " pod="openshift-ingress/e2e-ait-a4a71d55-openshift-default-c9c5dc8bd-rl5p8" Jul 29 19:46:24.743165 ip-10-0-131-93 kubenswrapper[2564]: I0729 19:46:24.742934 2564 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/4e980d2a-9b00-4791-a563-78cb03cf6697-credential-socket\") pod \"e2e-ait-a4a71d55-openshift-default-c9c5dc8bd-rl5p8\" (UID: \"4e980d2a-9b00-4791-a563-78cb03cf6697\") " pod="openshift-ingress/e2e-ait-a4a71d55-openshift-default-c9c5dc8bd-rl5p8" Jul 29 19:46:24.743273 ip-10-0-131-93 kubenswrapper[2564]: I0729 19:46:24.743250 2564 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/4e980d2a-9b00-4791-a563-78cb03cf6697-istiod-ca-cert\") pod \"e2e-ait-a4a71d55-openshift-default-c9c5dc8bd-rl5p8\" (UID: \"4e980d2a-9b00-4791-a563-78cb03cf6697\") " pod="openshift-ingress/e2e-ait-a4a71d55-openshift-default-c9c5dc8bd-rl5p8" Jul 29 19:46:24.745135 ip-10-0-131-93 kubenswrapper[2564]: I0729 19:46:24.745113 2564 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/4e980d2a-9b00-4791-a563-78cb03cf6697-istio-envoy\") pod \"e2e-ait-a4a71d55-openshift-default-c9c5dc8bd-rl5p8\" (UID: \"4e980d2a-9b00-4791-a563-78cb03cf6697\") " pod="openshift-ingress/e2e-ait-a4a71d55-openshift-default-c9c5dc8bd-rl5p8" Jul 29 19:46:24.745309 ip-10-0-131-93 kubenswrapper[2564]: I0729 19:46:24.745290 2564 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/4e980d2a-9b00-4791-a563-78cb03cf6697-istio-podinfo\") pod \"e2e-ait-a4a71d55-openshift-default-c9c5dc8bd-rl5p8\" (UID: \"4e980d2a-9b00-4791-a563-78cb03cf6697\") " pod="openshift-ingress/e2e-ait-a4a71d55-openshift-default-c9c5dc8bd-rl5p8" Jul 29 19:46:24.751490 ip-10-0-131-93 kubenswrapper[2564]: I0729 19:46:24.751463 2564 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/4e980d2a-9b00-4791-a563-78cb03cf6697-istio-token\") pod \"e2e-ait-a4a71d55-openshift-default-c9c5dc8bd-rl5p8\" (UID: \"4e980d2a-9b00-4791-a563-78cb03cf6697\") " pod="openshift-ingress/e2e-ait-a4a71d55-openshift-default-c9c5dc8bd-rl5p8" Jul 29 19:46:24.751711 ip-10-0-131-93 kubenswrapper[2564]: I0729 19:46:24.751687 2564 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-d56p5\" (UniqueName: \"kubernetes.io/projected/4e980d2a-9b00-4791-a563-78cb03cf6697-kube-api-access-d56p5\") pod \"e2e-ait-a4a71d55-openshift-default-c9c5dc8bd-rl5p8\" (UID: \"4e980d2a-9b00-4791-a563-78cb03cf6697\") " pod="openshift-ingress/e2e-ait-a4a71d55-openshift-default-c9c5dc8bd-rl5p8" Jul 29 19:46:24.883354 ip-10-0-131-93 kubenswrapper[2564]: I0729 19:46:24.883265 2564 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-ait-a4a71d55-openshift-default-c9c5dc8bd-rl5p8" Jul 29 19:46:24.921806 ip-10-0-131-93 kubenswrapper[2564]: I0729 19:46:24.921711 2564 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-444jx_faa8c546-c75b-4e10-93b4-661aa2b5f53e/konnectivity-agent/0.log" limit=1024 Jul 29 19:46:25.020145 ip-10-0-131-93 kubenswrapper[2564]: I0729 19:46:25.020119 2564 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/e2e-ait-a4a71d55-openshift-default-c9c5dc8bd-rl5p8"] Jul 29 19:46:25.022771 ip-10-0-131-93 kubenswrapper[2564]: W0729 19:46:25.022734 2564 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod4e980d2a_9b00_4791_a563_78cb03cf6697.slice/crio-d1d6b96f610dada81438abdf0e0cc7299610193f922251a1cb4edefb87782832 WatchSource:0}: Error finding container d1d6b96f610dada81438abdf0e0cc7299610193f922251a1cb4edefb87782832: Status 404 returned error can't find the container with id d1d6b96f610dada81438abdf0e0cc7299610193f922251a1cb4edefb87782832 Jul 29 19:46:25.024922 ip-10-0-131-93 kubenswrapper[2564]: I0729 19:46:25.024893 2564 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236224Ki","pods":"250"} Jul 29 19:46:25.024994 ip-10-0-131-93 kubenswrapper[2564]: I0729 19:46:25.024975 2564 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236224Ki","pods":"250"} Jul 29 19:46:25.025041 ip-10-0-131-93 kubenswrapper[2564]: I0729 19:46:25.025003 2564 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236224Ki","pods":"250"} Jul 29 19:46:25.727059 ip-10-0-131-93 kubenswrapper[2564]: I0729 19:46:25.727021 2564 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-ait-a4a71d55-openshift-default-c9c5dc8bd-rl5p8" event={"ID":"4e980d2a-9b00-4791-a563-78cb03cf6697","Type":"ContainerStarted","Data":"a635dcf92daebdde2feb1db9cb53d9dbaebbb570749eb7a8567bc236a157a115"} Jul 29 19:46:25.727515 ip-10-0-131-93 kubenswrapper[2564]: I0729 19:46:25.727073 2564 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-ait-a4a71d55-openshift-default-c9c5dc8bd-rl5p8" event={"ID":"4e980d2a-9b00-4791-a563-78cb03cf6697","Type":"ContainerStarted","Data":"d1d6b96f610dada81438abdf0e0cc7299610193f922251a1cb4edefb87782832"} Jul 29 19:46:25.747268 ip-10-0-131-93 kubenswrapper[2564]: I0729 19:46:25.747212 2564 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-ingress/e2e-ait-a4a71d55-openshift-default-c9c5dc8bd-rl5p8" podStartSLOduration=1.74718982 podStartE2EDuration="1.74718982s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-29 19:46:24 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-29 19:46:25.746505929 +0000 UTC m=+1761.470565622" watchObservedRunningTime="2026-07-29 19:46:25.74718982 +0000 UTC m=+1761.471249517" Jul 29 19:46:25.883945 ip-10-0-131-93 kubenswrapper[2564]: I0729 19:46:25.883905 2564 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="openshift-ingress/e2e-ait-a4a71d55-openshift-default-c9c5dc8bd-rl5p8" Jul 29 19:46:26.407452 ip-10-0-131-93 kubenswrapper[2564]: I0729 19:46:26.407416 2564 patch_prober.go:28] interesting pod/e2e-migrate-13baf70f-gw-openshift-default-647c89464-7bskp container/istio-proxy namespace/openshift-ingress: Readiness probe status=failure output="HTTP probe failed with statuscode: 503" start-of-body= Jul 29 19:46:26.407601 ip-10-0-131-93 kubenswrapper[2564]: I0729 19:46:26.407471 2564 prober.go:120] "Probe failed" probeType="Readiness" pod="openshift-ingress/e2e-migrate-13baf70f-gw-openshift-default-647c89464-7bskp" podUID="8926f238-14d4-4e56-8635-95d47c45c78f" containerName="istio-proxy" probeResult="failure" output="HTTP probe failed with statuscode: 503" Jul 29 19:46:26.889807 ip-10-0-131-93 kubenswrapper[2564]: I0729 19:46:26.889776 2564 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="openshift-ingress/e2e-ait-a4a71d55-openshift-default-c9c5dc8bd-rl5p8" Jul 29 19:46:27.091787 ip-10-0-131-93 kubenswrapper[2564]: I0729 19:46:27.091756 2564 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-444jx_faa8c546-c75b-4e10-93b4-661aa2b5f53e/konnectivity-agent/0.log" limit=1024 Jul 29 19:46:27.232003 ip-10-0-131-93 kubenswrapper[2564]: I0729 19:46:27.231964 2564 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-444jx_faa8c546-c75b-4e10-93b4-661aa2b5f53e/konnectivity-agent/0.log" limit=1024 Jul 29 19:46:27.736795 ip-10-0-131-93 kubenswrapper[2564]: I0729 19:46:27.736770 2564 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/e2e-ait-a4a71d55-openshift-default-c9c5dc8bd-rl5p8" Jul 29 19:46:27.737655 ip-10-0-131-93 kubenswrapper[2564]: I0729 19:46:27.737631 2564 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/e2e-ait-a4a71d55-openshift-default-c9c5dc8bd-rl5p8" Jul 29 19:46:28.201621 ip-10-0-131-93 kubenswrapper[2564]: I0729 19:46:28.201586 2564 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-444jx_faa8c546-c75b-4e10-93b4-661aa2b5f53e/konnectivity-agent/0.log" limit=1024 Jul 29 19:46:28.839135 ip-10-0-131-93 kubenswrapper[2564]: I0729 19:46:28.839097 2564 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-444jx_faa8c546-c75b-4e10-93b4-661aa2b5f53e/konnectivity-agent/0.log" limit=1024 Jul 29 19:46:28.933321 ip-10-0-131-93 kubenswrapper[2564]: I0729 19:46:28.933283 2564 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-444jx_faa8c546-c75b-4e10-93b4-661aa2b5f53e/konnectivity-agent/0.log" limit=1024 Jul 29 19:46:29.001366 ip-10-0-131-93 kubenswrapper[2564]: I0729 19:46:29.001335 2564 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-444jx_faa8c546-c75b-4e10-93b4-661aa2b5f53e/konnectivity-agent/0.log" limit=1024 Jul 29 19:46:29.095524 ip-10-0-131-93 kubenswrapper[2564]: I0729 19:46:29.095453 2564 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-migrate-13baf70f-gw-openshift-default-647c89464-7bskp" Jul 29 19:46:29.182823 ip-10-0-131-93 kubenswrapper[2564]: I0729 19:46:29.182791 2564 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/8926f238-14d4-4e56-8635-95d47c45c78f-istio-data\") pod \"8926f238-14d4-4e56-8635-95d47c45c78f\" (UID: \"8926f238-14d4-4e56-8635-95d47c45c78f\") " Jul 29 19:46:29.183000 ip-10-0-131-93 kubenswrapper[2564]: I0729 19:46:29.182834 2564 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/8926f238-14d4-4e56-8635-95d47c45c78f-istio-envoy\") pod \"8926f238-14d4-4e56-8635-95d47c45c78f\" (UID: \"8926f238-14d4-4e56-8635-95d47c45c78f\") " Jul 29 19:46:29.183000 ip-10-0-131-93 kubenswrapper[2564]: I0729 19:46:29.182851 2564 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/8926f238-14d4-4e56-8635-95d47c45c78f-workload-socket\") pod \"8926f238-14d4-4e56-8635-95d47c45c78f\" (UID: \"8926f238-14d4-4e56-8635-95d47c45c78f\") " Jul 29 19:46:29.183000 ip-10-0-131-93 kubenswrapper[2564]: I0729 19:46:29.182872 2564 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/8926f238-14d4-4e56-8635-95d47c45c78f-credential-socket\") pod \"8926f238-14d4-4e56-8635-95d47c45c78f\" (UID: \"8926f238-14d4-4e56-8635-95d47c45c78f\") " Jul 29 19:46:29.183000 ip-10-0-131-93 kubenswrapper[2564]: I0729 19:46:29.182898 2564 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/8926f238-14d4-4e56-8635-95d47c45c78f-workload-certs\") pod \"8926f238-14d4-4e56-8635-95d47c45c78f\" (UID: \"8926f238-14d4-4e56-8635-95d47c45c78f\") " Jul 29 19:46:29.183000 ip-10-0-131-93 kubenswrapper[2564]: I0729 19:46:29.182935 2564 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/8926f238-14d4-4e56-8635-95d47c45c78f-istiod-ca-cert\") pod \"8926f238-14d4-4e56-8635-95d47c45c78f\" (UID: \"8926f238-14d4-4e56-8635-95d47c45c78f\") " Jul 29 19:46:29.183000 ip-10-0-131-93 kubenswrapper[2564]: I0729 19:46:29.182964 2564 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/8926f238-14d4-4e56-8635-95d47c45c78f-istio-token\") pod \"8926f238-14d4-4e56-8635-95d47c45c78f\" (UID: \"8926f238-14d4-4e56-8635-95d47c45c78f\") " Jul 29 19:46:29.183000 ip-10-0-131-93 kubenswrapper[2564]: I0729 19:46:29.182999 2564 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/8926f238-14d4-4e56-8635-95d47c45c78f-istio-podinfo\") pod \"8926f238-14d4-4e56-8635-95d47c45c78f\" (UID: \"8926f238-14d4-4e56-8635-95d47c45c78f\") " Jul 29 19:46:29.183415 ip-10-0-131-93 kubenswrapper[2564]: I0729 19:46:29.183037 2564 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-64mxv\" (UniqueName: \"kubernetes.io/projected/8926f238-14d4-4e56-8635-95d47c45c78f-kube-api-access-64mxv\") pod \"8926f238-14d4-4e56-8635-95d47c45c78f\" (UID: \"8926f238-14d4-4e56-8635-95d47c45c78f\") " Jul 29 19:46:29.183415 ip-10-0-131-93 kubenswrapper[2564]: I0729 19:46:29.183116 2564 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/8926f238-14d4-4e56-8635-95d47c45c78f-workload-socket" (OuterVolumeSpecName: "workload-socket") pod "8926f238-14d4-4e56-8635-95d47c45c78f" (UID: "8926f238-14d4-4e56-8635-95d47c45c78f"). InnerVolumeSpecName "workload-socket". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Jul 29 19:46:29.183415 ip-10-0-131-93 kubenswrapper[2564]: I0729 19:46:29.183156 2564 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/8926f238-14d4-4e56-8635-95d47c45c78f-credential-socket" (OuterVolumeSpecName: "credential-socket") pod "8926f238-14d4-4e56-8635-95d47c45c78f" (UID: "8926f238-14d4-4e56-8635-95d47c45c78f"). InnerVolumeSpecName "credential-socket". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Jul 29 19:46:29.183415 ip-10-0-131-93 kubenswrapper[2564]: I0729 19:46:29.183199 2564 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/8926f238-14d4-4e56-8635-95d47c45c78f-workload-certs" (OuterVolumeSpecName: "workload-certs") pod "8926f238-14d4-4e56-8635-95d47c45c78f" (UID: "8926f238-14d4-4e56-8635-95d47c45c78f"). InnerVolumeSpecName "workload-certs". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Jul 29 19:46:29.183415 ip-10-0-131-93 kubenswrapper[2564]: I0729 19:46:29.183356 2564 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/8926f238-14d4-4e56-8635-95d47c45c78f-istiod-ca-cert" (OuterVolumeSpecName: "istiod-ca-cert") pod "8926f238-14d4-4e56-8635-95d47c45c78f" (UID: "8926f238-14d4-4e56-8635-95d47c45c78f"). InnerVolumeSpecName "istiod-ca-cert". PluginName "kubernetes.io/configmap", VolumeGIDValue "" Jul 29 19:46:29.183693 ip-10-0-131-93 kubenswrapper[2564]: I0729 19:46:29.183447 2564 reconciler_common.go:299] "Volume detached for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/8926f238-14d4-4e56-8635-95d47c45c78f-workload-socket\") on node \"ip-10-0-131-93.ec2.internal\" DevicePath \"\"" Jul 29 19:46:29.183693 ip-10-0-131-93 kubenswrapper[2564]: I0729 19:46:29.183467 2564 reconciler_common.go:299] "Volume detached for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/8926f238-14d4-4e56-8635-95d47c45c78f-credential-socket\") on node \"ip-10-0-131-93.ec2.internal\" DevicePath \"\"" Jul 29 19:46:29.183693 ip-10-0-131-93 kubenswrapper[2564]: I0729 19:46:29.183482 2564 reconciler_common.go:299] "Volume detached for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/8926f238-14d4-4e56-8635-95d47c45c78f-workload-certs\") on node \"ip-10-0-131-93.ec2.internal\" DevicePath \"\"" Jul 29 19:46:29.183693 ip-10-0-131-93 kubenswrapper[2564]: I0729 19:46:29.183672 2564 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/8926f238-14d4-4e56-8635-95d47c45c78f-istio-data" (OuterVolumeSpecName: "istio-data") pod "8926f238-14d4-4e56-8635-95d47c45c78f" (UID: "8926f238-14d4-4e56-8635-95d47c45c78f"). InnerVolumeSpecName "istio-data". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Jul 29 19:46:29.185392 ip-10-0-131-93 kubenswrapper[2564]: I0729 19:46:29.185299 2564 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/downward-api/8926f238-14d4-4e56-8635-95d47c45c78f-istio-podinfo" (OuterVolumeSpecName: "istio-podinfo") pod "8926f238-14d4-4e56-8635-95d47c45c78f" (UID: "8926f238-14d4-4e56-8635-95d47c45c78f"). InnerVolumeSpecName "istio-podinfo". PluginName "kubernetes.io/downward-api", VolumeGIDValue "" Jul 29 19:46:29.185519 ip-10-0-131-93 kubenswrapper[2564]: I0729 19:46:29.185447 2564 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/8926f238-14d4-4e56-8635-95d47c45c78f-istio-token" (OuterVolumeSpecName: "istio-token") pod "8926f238-14d4-4e56-8635-95d47c45c78f" (UID: "8926f238-14d4-4e56-8635-95d47c45c78f"). InnerVolumeSpecName "istio-token". PluginName "kubernetes.io/projected", VolumeGIDValue "" Jul 29 19:46:29.185584 ip-10-0-131-93 kubenswrapper[2564]: I0729 19:46:29.185553 2564 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/8926f238-14d4-4e56-8635-95d47c45c78f-istio-envoy" (OuterVolumeSpecName: "istio-envoy") pod "8926f238-14d4-4e56-8635-95d47c45c78f" (UID: "8926f238-14d4-4e56-8635-95d47c45c78f"). InnerVolumeSpecName "istio-envoy". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Jul 29 19:46:29.185640 ip-10-0-131-93 kubenswrapper[2564]: I0729 19:46:29.185624 2564 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/8926f238-14d4-4e56-8635-95d47c45c78f-kube-api-access-64mxv" (OuterVolumeSpecName: "kube-api-access-64mxv") pod "8926f238-14d4-4e56-8635-95d47c45c78f" (UID: "8926f238-14d4-4e56-8635-95d47c45c78f"). InnerVolumeSpecName "kube-api-access-64mxv". PluginName "kubernetes.io/projected", VolumeGIDValue "" Jul 29 19:46:29.284725 ip-10-0-131-93 kubenswrapper[2564]: I0729 19:46:29.284691 2564 reconciler_common.go:299] "Volume detached for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/8926f238-14d4-4e56-8635-95d47c45c78f-istio-data\") on node \"ip-10-0-131-93.ec2.internal\" DevicePath \"\"" Jul 29 19:46:29.284725 ip-10-0-131-93 kubenswrapper[2564]: I0729 19:46:29.284720 2564 reconciler_common.go:299] "Volume detached for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/8926f238-14d4-4e56-8635-95d47c45c78f-istio-envoy\") on node \"ip-10-0-131-93.ec2.internal\" DevicePath \"\"" Jul 29 19:46:29.284725 ip-10-0-131-93 kubenswrapper[2564]: I0729 19:46:29.284729 2564 reconciler_common.go:299] "Volume detached for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/8926f238-14d4-4e56-8635-95d47c45c78f-istiod-ca-cert\") on node \"ip-10-0-131-93.ec2.internal\" DevicePath \"\"" Jul 29 19:46:29.285168 ip-10-0-131-93 kubenswrapper[2564]: I0729 19:46:29.284737 2564 reconciler_common.go:299] "Volume detached for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/8926f238-14d4-4e56-8635-95d47c45c78f-istio-token\") on node \"ip-10-0-131-93.ec2.internal\" DevicePath \"\"" Jul 29 19:46:29.285168 ip-10-0-131-93 kubenswrapper[2564]: I0729 19:46:29.284745 2564 reconciler_common.go:299] "Volume detached for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/8926f238-14d4-4e56-8635-95d47c45c78f-istio-podinfo\") on node \"ip-10-0-131-93.ec2.internal\" DevicePath \"\"" Jul 29 19:46:29.285168 ip-10-0-131-93 kubenswrapper[2564]: I0729 19:46:29.284753 2564 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-64mxv\" (UniqueName: \"kubernetes.io/projected/8926f238-14d4-4e56-8635-95d47c45c78f-kube-api-access-64mxv\") on node \"ip-10-0-131-93.ec2.internal\" DevicePath \"\"" Jul 29 19:46:29.747573 ip-10-0-131-93 kubenswrapper[2564]: I0729 19:46:29.747542 2564 generic.go:356] "Generic (PLEG): container finished" podID="8926f238-14d4-4e56-8635-95d47c45c78f" containerID="a9c940a8426569768e015edc75b0a5d1ad9f680fe4da55a2178ddc1a5e1d4322" exitCode=0 Jul 29 19:46:29.747765 ip-10-0-131-93 kubenswrapper[2564]: I0729 19:46:29.747591 2564 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-migrate-13baf70f-gw-openshift-default-647c89464-7bskp" Jul 29 19:46:29.747765 ip-10-0-131-93 kubenswrapper[2564]: I0729 19:46:29.747614 2564 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-migrate-13baf70f-gw-openshift-default-647c89464-7bskp" event={"ID":"8926f238-14d4-4e56-8635-95d47c45c78f","Type":"ContainerDied","Data":"a9c940a8426569768e015edc75b0a5d1ad9f680fe4da55a2178ddc1a5e1d4322"} Jul 29 19:46:29.747765 ip-10-0-131-93 kubenswrapper[2564]: I0729 19:46:29.747670 2564 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-migrate-13baf70f-gw-openshift-default-647c89464-7bskp" event={"ID":"8926f238-14d4-4e56-8635-95d47c45c78f","Type":"ContainerDied","Data":"d84a31768ec9ebd0126d32c656625ade352177f075b490ff32200f7bdea1fca5"} Jul 29 19:46:29.747765 ip-10-0-131-93 kubenswrapper[2564]: I0729 19:46:29.747702 2564 scope.go:117] "RemoveContainer" containerID="a9c940a8426569768e015edc75b0a5d1ad9f680fe4da55a2178ddc1a5e1d4322" Jul 29 19:46:29.757499 ip-10-0-131-93 kubenswrapper[2564]: I0729 19:46:29.757484 2564 scope.go:117] "RemoveContainer" containerID="118be6285f9d54c535c91b6ea1cc08e3c442d616f1f28d607b5a046b18035917" Jul 29 19:46:29.765553 ip-10-0-131-93 kubenswrapper[2564]: I0729 19:46:29.765534 2564 scope.go:117] "RemoveContainer" containerID="a9c940a8426569768e015edc75b0a5d1ad9f680fe4da55a2178ddc1a5e1d4322" Jul 29 19:46:29.765823 ip-10-0-131-93 kubenswrapper[2564]: E0729 19:46:29.765803 2564 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"a9c940a8426569768e015edc75b0a5d1ad9f680fe4da55a2178ddc1a5e1d4322\": container with ID starting with a9c940a8426569768e015edc75b0a5d1ad9f680fe4da55a2178ddc1a5e1d4322 not found: ID does not exist" containerID="a9c940a8426569768e015edc75b0a5d1ad9f680fe4da55a2178ddc1a5e1d4322" Jul 29 19:46:29.765892 ip-10-0-131-93 kubenswrapper[2564]: I0729 19:46:29.765832 2564 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"a9c940a8426569768e015edc75b0a5d1ad9f680fe4da55a2178ddc1a5e1d4322"} err="failed to get container status \"a9c940a8426569768e015edc75b0a5d1ad9f680fe4da55a2178ddc1a5e1d4322\": rpc error: code = NotFound desc = could not find container \"a9c940a8426569768e015edc75b0a5d1ad9f680fe4da55a2178ddc1a5e1d4322\": container with ID starting with a9c940a8426569768e015edc75b0a5d1ad9f680fe4da55a2178ddc1a5e1d4322 not found: ID does not exist" Jul 29 19:46:29.765892 ip-10-0-131-93 kubenswrapper[2564]: I0729 19:46:29.765849 2564 scope.go:117] "RemoveContainer" containerID="118be6285f9d54c535c91b6ea1cc08e3c442d616f1f28d607b5a046b18035917" Jul 29 19:46:29.766106 ip-10-0-131-93 kubenswrapper[2564]: E0729 19:46:29.766087 2564 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"118be6285f9d54c535c91b6ea1cc08e3c442d616f1f28d607b5a046b18035917\": container with ID starting with 118be6285f9d54c535c91b6ea1cc08e3c442d616f1f28d607b5a046b18035917 not found: ID does not exist" containerID="118be6285f9d54c535c91b6ea1cc08e3c442d616f1f28d607b5a046b18035917" Jul 29 19:46:29.766143 ip-10-0-131-93 kubenswrapper[2564]: I0729 19:46:29.766113 2564 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"118be6285f9d54c535c91b6ea1cc08e3c442d616f1f28d607b5a046b18035917"} err="failed to get container status \"118be6285f9d54c535c91b6ea1cc08e3c442d616f1f28d607b5a046b18035917\": rpc error: code = NotFound desc = could not find container \"118be6285f9d54c535c91b6ea1cc08e3c442d616f1f28d607b5a046b18035917\": container with ID starting with 118be6285f9d54c535c91b6ea1cc08e3c442d616f1f28d607b5a046b18035917 not found: ID does not exist" Jul 29 19:46:29.772236 ip-10-0-131-93 kubenswrapper[2564]: I0729 19:46:29.772215 2564 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-ingress/e2e-migrate-13baf70f-gw-openshift-default-647c89464-7bskp"] Jul 29 19:46:29.776424 ip-10-0-131-93 kubenswrapper[2564]: I0729 19:46:29.776402 2564 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["openshift-ingress/e2e-migrate-13baf70f-gw-openshift-default-647c89464-7bskp"] Jul 29 19:46:30.783616 ip-10-0-131-93 kubenswrapper[2564]: I0729 19:46:30.783581 2564 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="8926f238-14d4-4e56-8635-95d47c45c78f" path="/var/lib/kubelet/pods/8926f238-14d4-4e56-8635-95d47c45c78f/volumes" Jul 29 19:46:32.948823 ip-10-0-131-93 kubenswrapper[2564]: I0729 19:46:32.948786 2564 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-444jx_faa8c546-c75b-4e10-93b4-661aa2b5f53e/konnectivity-agent/0.log" limit=1024 Jul 29 19:46:33.049307 ip-10-0-131-93 kubenswrapper[2564]: I0729 19:46:33.049269 2564 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-444jx_faa8c546-c75b-4e10-93b4-661aa2b5f53e/konnectivity-agent/0.log" limit=1024 Jul 29 19:46:41.820302 ip-10-0-131-93 kubenswrapper[2564]: I0729 19:46:41.820266 2564 generic.go:356] "Generic (PLEG): container finished" podID="4e980d2a-9b00-4791-a563-78cb03cf6697" containerID="a635dcf92daebdde2feb1db9cb53d9dbaebbb570749eb7a8567bc236a157a115" exitCode=0 Jul 29 19:46:41.820770 ip-10-0-131-93 kubenswrapper[2564]: I0729 19:46:41.820316 2564 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-ait-a4a71d55-openshift-default-c9c5dc8bd-rl5p8" event={"ID":"4e980d2a-9b00-4791-a563-78cb03cf6697","Type":"ContainerDied","Data":"a635dcf92daebdde2feb1db9cb53d9dbaebbb570749eb7a8567bc236a157a115"} Jul 29 19:46:41.820770 ip-10-0-131-93 kubenswrapper[2564]: I0729 19:46:41.820719 2564 scope.go:117] "RemoveContainer" containerID="a635dcf92daebdde2feb1db9cb53d9dbaebbb570749eb7a8567bc236a157a115" Jul 29 19:46:41.821743 ip-10-0-131-93 kubenswrapper[2564]: I0729 19:46:41.821712 2564 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236224Ki","pods":"250"} Jul 29 19:46:41.821845 ip-10-0-131-93 kubenswrapper[2564]: I0729 19:46:41.821798 2564 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236224Ki","pods":"250"} Jul 29 19:46:41.821911 ip-10-0-131-93 kubenswrapper[2564]: I0729 19:46:41.821848 2564 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236224Ki","pods":"250"} Jul 29 19:46:42.737036 ip-10-0-131-93 kubenswrapper[2564]: I0729 19:46:42.736986 2564 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/e2e-ait-a4a71d55-openshift-default-c9c5dc8bd-rl5p8" Jul 29 19:46:42.826597 ip-10-0-131-93 kubenswrapper[2564]: I0729 19:46:42.826560 2564 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-ait-a4a71d55-openshift-default-c9c5dc8bd-rl5p8" event={"ID":"4e980d2a-9b00-4791-a563-78cb03cf6697","Type":"ContainerStarted","Data":"4424582b8861a44d2b7ab5df7e5d63e5d3072104610dd3015b133aade62e231a"} Jul 29 19:46:42.884246 ip-10-0-131-93 kubenswrapper[2564]: I0729 19:46:42.884204 2564 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="openshift-ingress/e2e-ait-a4a71d55-openshift-default-c9c5dc8bd-rl5p8" Jul 29 19:46:42.885356 ip-10-0-131-93 kubenswrapper[2564]: I0729 19:46:42.885323 2564 patch_prober.go:28] interesting pod/e2e-ait-a4a71d55-openshift-default-c9c5dc8bd-rl5p8 container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.132.0.39:15021/healthz/ready\": dial tcp 10.132.0.39:15021: connect: connection refused" start-of-body= Jul 29 19:46:42.885525 ip-10-0-131-93 kubenswrapper[2564]: I0729 19:46:42.885440 2564 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-ait-a4a71d55-openshift-default-c9c5dc8bd-rl5p8" podUID="4e980d2a-9b00-4791-a563-78cb03cf6697" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.132.0.39:15021/healthz/ready\": dial tcp 10.132.0.39:15021: connect: connection refused" Jul 29 19:46:43.884320 ip-10-0-131-93 kubenswrapper[2564]: I0729 19:46:43.884279 2564 patch_prober.go:28] interesting pod/e2e-ait-a4a71d55-openshift-default-c9c5dc8bd-rl5p8 container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.132.0.39:15021/healthz/ready\": dial tcp 10.132.0.39:15021: connect: connection refused" start-of-body= Jul 29 19:46:43.884950 ip-10-0-131-93 kubenswrapper[2564]: I0729 19:46:43.884916 2564 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-ait-a4a71d55-openshift-default-c9c5dc8bd-rl5p8" podUID="4e980d2a-9b00-4791-a563-78cb03cf6697" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.132.0.39:15021/healthz/ready\": dial tcp 10.132.0.39:15021: connect: connection refused" Jul 29 19:46:44.884565 ip-10-0-131-93 kubenswrapper[2564]: I0729 19:46:44.884525 2564 patch_prober.go:28] interesting pod/e2e-ait-a4a71d55-openshift-default-c9c5dc8bd-rl5p8 container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.132.0.39:15021/healthz/ready\": dial tcp 10.132.0.39:15021: connect: connection refused" start-of-body= Jul 29 19:46:44.885065 ip-10-0-131-93 kubenswrapper[2564]: I0729 19:46:44.884610 2564 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-ait-a4a71d55-openshift-default-c9c5dc8bd-rl5p8" podUID="4e980d2a-9b00-4791-a563-78cb03cf6697" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.132.0.39:15021/healthz/ready\": dial tcp 10.132.0.39:15021: connect: connection refused" Jul 29 19:46:45.883883 ip-10-0-131-93 kubenswrapper[2564]: I0729 19:46:45.883836 2564 patch_prober.go:28] interesting pod/e2e-ait-a4a71d55-openshift-default-c9c5dc8bd-rl5p8 container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.132.0.39:15021/healthz/ready\": dial tcp 10.132.0.39:15021: connect: connection refused" start-of-body= Jul 29 19:46:45.884093 ip-10-0-131-93 kubenswrapper[2564]: I0729 19:46:45.883918 2564 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-ait-a4a71d55-openshift-default-c9c5dc8bd-rl5p8" podUID="4e980d2a-9b00-4791-a563-78cb03cf6697" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.132.0.39:15021/healthz/ready\": dial tcp 10.132.0.39:15021: connect: connection refused" Jul 29 19:46:46.888012 ip-10-0-131-93 kubenswrapper[2564]: I0729 19:46:46.887981 2564 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="openshift-ingress/e2e-ait-a4a71d55-openshift-default-c9c5dc8bd-rl5p8" Jul 29 19:46:46.888461 ip-10-0-131-93 kubenswrapper[2564]: I0729 19:46:46.888350 2564 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/e2e-ait-a4a71d55-openshift-default-c9c5dc8bd-rl5p8" Jul 29 19:46:46.889111 ip-10-0-131-93 kubenswrapper[2564]: I0729 19:46:46.889093 2564 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/e2e-ait-a4a71d55-openshift-default-c9c5dc8bd-rl5p8" Jul 29 19:47:09.036345 ip-10-0-131-93 kubenswrapper[2564]: I0729 19:47:09.036308 2564 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-444jx_faa8c546-c75b-4e10-93b4-661aa2b5f53e/konnectivity-agent/0.log" limit=1024 Jul 29 19:47:09.121388 ip-10-0-131-93 kubenswrapper[2564]: I0729 19:47:09.121335 2564 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-444jx_faa8c546-c75b-4e10-93b4-661aa2b5f53e/konnectivity-agent/0.log" limit=1024 Jul 29 19:47:09.200293 ip-10-0-131-93 kubenswrapper[2564]: I0729 19:47:09.200255 2564 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-444jx_faa8c546-c75b-4e10-93b4-661aa2b5f53e/konnectivity-agent/0.log" limit=1024 Jul 29 19:47:09.265803 ip-10-0-131-93 kubenswrapper[2564]: I0729 19:47:09.265772 2564 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-444jx_faa8c546-c75b-4e10-93b4-661aa2b5f53e/konnectivity-agent/0.log" limit=1024 Jul 29 19:47:10.397975 ip-10-0-131-93 kubenswrapper[2564]: I0729 19:47:10.397941 2564 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-ingress/e2e-ait-a4a71d55-openshift-default-c9c5dc8bd-rl5p8"] Jul 29 19:47:10.398460 ip-10-0-131-93 kubenswrapper[2564]: I0729 19:47:10.398148 2564 kuberuntime_container.go:873] "Killing container with a grace period" pod="openshift-ingress/e2e-ait-a4a71d55-openshift-default-c9c5dc8bd-rl5p8" podUID="4e980d2a-9b00-4791-a563-78cb03cf6697" containerName="istio-proxy" containerID="cri-o://4424582b8861a44d2b7ab5df7e5d63e5d3072104610dd3015b133aade62e231a" gracePeriod=30 Jul 29 19:47:10.406426 ip-10-0-131-93 kubenswrapper[2564]: I0729 19:47:10.406395 2564 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-444jx_faa8c546-c75b-4e10-93b4-661aa2b5f53e/konnectivity-agent/0.log" limit=1024 Jul 29 19:47:10.614533 ip-10-0-131-93 kubenswrapper[2564]: I0729 19:47:10.614430 2564 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-444jx_faa8c546-c75b-4e10-93b4-661aa2b5f53e/konnectivity-agent/0.log" limit=1024 Jul 29 19:47:10.827315 ip-10-0-131-93 kubenswrapper[2564]: I0729 19:47:10.827280 2564 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-444jx_faa8c546-c75b-4e10-93b4-661aa2b5f53e/konnectivity-agent/0.log" limit=1024 Jul 29 19:47:11.092071 ip-10-0-131-93 kubenswrapper[2564]: I0729 19:47:11.092042 2564 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-ingress/e2e-derive-23697b7f-openshift-default-554bf767cb-t866z"] Jul 29 19:47:11.092392 ip-10-0-131-93 kubenswrapper[2564]: I0729 19:47:11.092360 2564 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="8926f238-14d4-4e56-8635-95d47c45c78f" containerName="istio-proxy" Jul 29 19:47:11.092445 ip-10-0-131-93 kubenswrapper[2564]: I0729 19:47:11.092396 2564 state_mem.go:107] "Deleted CPUSet assignment" podUID="8926f238-14d4-4e56-8635-95d47c45c78f" containerName="istio-proxy" Jul 29 19:47:11.092445 ip-10-0-131-93 kubenswrapper[2564]: I0729 19:47:11.092410 2564 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="8926f238-14d4-4e56-8635-95d47c45c78f" containerName="istio-proxy" Jul 29 19:47:11.092445 ip-10-0-131-93 kubenswrapper[2564]: I0729 19:47:11.092416 2564 state_mem.go:107] "Deleted CPUSet assignment" podUID="8926f238-14d4-4e56-8635-95d47c45c78f" containerName="istio-proxy" Jul 29 19:47:11.092554 ip-10-0-131-93 kubenswrapper[2564]: I0729 19:47:11.092471 2564 memory_manager.go:368] "RemoveStaleState removing state" podUID="8926f238-14d4-4e56-8635-95d47c45c78f" containerName="istio-proxy" Jul 29 19:47:11.092554 ip-10-0-131-93 kubenswrapper[2564]: I0729 19:47:11.092481 2564 memory_manager.go:368] "RemoveStaleState removing state" podUID="8926f238-14d4-4e56-8635-95d47c45c78f" containerName="istio-proxy" Jul 29 19:47:11.094856 ip-10-0-131-93 kubenswrapper[2564]: I0729 19:47:11.094834 2564 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-derive-23697b7f-openshift-default-554bf767cb-t866z" Jul 29 19:47:11.100418 ip-10-0-131-93 kubenswrapper[2564]: I0729 19:47:11.100396 2564 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress\"/\"e2e-derive-23697b7f-openshift-default-dockercfg-2dfn7\"" Jul 29 19:47:11.115602 ip-10-0-131-93 kubenswrapper[2564]: I0729 19:47:11.115577 2564 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/e2e-derive-23697b7f-openshift-default-554bf767cb-t866z"] Jul 29 19:47:11.144984 ip-10-0-131-93 kubenswrapper[2564]: I0729 19:47:11.144951 2564 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-444jx_faa8c546-c75b-4e10-93b4-661aa2b5f53e/konnectivity-agent/0.log" limit=1024 Jul 29 19:47:11.264085 ip-10-0-131-93 kubenswrapper[2564]: I0729 19:47:11.264046 2564 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/ba30c537-a242-4d8c-a15a-a21d57983742-istio-envoy\") pod \"e2e-derive-23697b7f-openshift-default-554bf767cb-t866z\" (UID: \"ba30c537-a242-4d8c-a15a-a21d57983742\") " pod="openshift-ingress/e2e-derive-23697b7f-openshift-default-554bf767cb-t866z" Jul 29 19:47:11.264085 ip-10-0-131-93 kubenswrapper[2564]: I0729 19:47:11.264088 2564 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-hhp4s\" (UniqueName: \"kubernetes.io/projected/ba30c537-a242-4d8c-a15a-a21d57983742-kube-api-access-hhp4s\") pod \"e2e-derive-23697b7f-openshift-default-554bf767cb-t866z\" (UID: \"ba30c537-a242-4d8c-a15a-a21d57983742\") " pod="openshift-ingress/e2e-derive-23697b7f-openshift-default-554bf767cb-t866z" Jul 29 19:47:11.264289 ip-10-0-131-93 kubenswrapper[2564]: I0729 19:47:11.264113 2564 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/ba30c537-a242-4d8c-a15a-a21d57983742-istiod-ca-cert\") pod \"e2e-derive-23697b7f-openshift-default-554bf767cb-t866z\" (UID: \"ba30c537-a242-4d8c-a15a-a21d57983742\") " pod="openshift-ingress/e2e-derive-23697b7f-openshift-default-554bf767cb-t866z" Jul 29 19:47:11.264289 ip-10-0-131-93 kubenswrapper[2564]: I0729 19:47:11.264191 2564 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/ba30c537-a242-4d8c-a15a-a21d57983742-workload-socket\") pod \"e2e-derive-23697b7f-openshift-default-554bf767cb-t866z\" (UID: \"ba30c537-a242-4d8c-a15a-a21d57983742\") " pod="openshift-ingress/e2e-derive-23697b7f-openshift-default-554bf767cb-t866z" Jul 29 19:47:11.264289 ip-10-0-131-93 kubenswrapper[2564]: I0729 19:47:11.264245 2564 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/ba30c537-a242-4d8c-a15a-a21d57983742-istio-podinfo\") pod \"e2e-derive-23697b7f-openshift-default-554bf767cb-t866z\" (UID: \"ba30c537-a242-4d8c-a15a-a21d57983742\") " pod="openshift-ingress/e2e-derive-23697b7f-openshift-default-554bf767cb-t866z" Jul 29 19:47:11.264430 ip-10-0-131-93 kubenswrapper[2564]: I0729 19:47:11.264289 2564 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/ba30c537-a242-4d8c-a15a-a21d57983742-istio-data\") pod \"e2e-derive-23697b7f-openshift-default-554bf767cb-t866z\" (UID: \"ba30c537-a242-4d8c-a15a-a21d57983742\") " pod="openshift-ingress/e2e-derive-23697b7f-openshift-default-554bf767cb-t866z" Jul 29 19:47:11.264430 ip-10-0-131-93 kubenswrapper[2564]: I0729 19:47:11.264319 2564 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/ba30c537-a242-4d8c-a15a-a21d57983742-credential-socket\") pod \"e2e-derive-23697b7f-openshift-default-554bf767cb-t866z\" (UID: \"ba30c537-a242-4d8c-a15a-a21d57983742\") " pod="openshift-ingress/e2e-derive-23697b7f-openshift-default-554bf767cb-t866z" Jul 29 19:47:11.264430 ip-10-0-131-93 kubenswrapper[2564]: I0729 19:47:11.264366 2564 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/ba30c537-a242-4d8c-a15a-a21d57983742-workload-certs\") pod \"e2e-derive-23697b7f-openshift-default-554bf767cb-t866z\" (UID: \"ba30c537-a242-4d8c-a15a-a21d57983742\") " pod="openshift-ingress/e2e-derive-23697b7f-openshift-default-554bf767cb-t866z" Jul 29 19:47:11.264532 ip-10-0-131-93 kubenswrapper[2564]: I0729 19:47:11.264439 2564 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/ba30c537-a242-4d8c-a15a-a21d57983742-istio-token\") pod \"e2e-derive-23697b7f-openshift-default-554bf767cb-t866z\" (UID: \"ba30c537-a242-4d8c-a15a-a21d57983742\") " pod="openshift-ingress/e2e-derive-23697b7f-openshift-default-554bf767cb-t866z" Jul 29 19:47:11.365650 ip-10-0-131-93 kubenswrapper[2564]: I0729 19:47:11.365561 2564 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/ba30c537-a242-4d8c-a15a-a21d57983742-credential-socket\") pod \"e2e-derive-23697b7f-openshift-default-554bf767cb-t866z\" (UID: \"ba30c537-a242-4d8c-a15a-a21d57983742\") " pod="openshift-ingress/e2e-derive-23697b7f-openshift-default-554bf767cb-t866z" Jul 29 19:47:11.365650 ip-10-0-131-93 kubenswrapper[2564]: I0729 19:47:11.365604 2564 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/ba30c537-a242-4d8c-a15a-a21d57983742-workload-certs\") pod \"e2e-derive-23697b7f-openshift-default-554bf767cb-t866z\" (UID: \"ba30c537-a242-4d8c-a15a-a21d57983742\") " pod="openshift-ingress/e2e-derive-23697b7f-openshift-default-554bf767cb-t866z" Jul 29 19:47:11.365650 ip-10-0-131-93 kubenswrapper[2564]: I0729 19:47:11.365630 2564 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/ba30c537-a242-4d8c-a15a-a21d57983742-istio-token\") pod \"e2e-derive-23697b7f-openshift-default-554bf767cb-t866z\" (UID: \"ba30c537-a242-4d8c-a15a-a21d57983742\") " pod="openshift-ingress/e2e-derive-23697b7f-openshift-default-554bf767cb-t866z" Jul 29 19:47:11.365935 ip-10-0-131-93 kubenswrapper[2564]: I0729 19:47:11.365660 2564 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/ba30c537-a242-4d8c-a15a-a21d57983742-istio-envoy\") pod \"e2e-derive-23697b7f-openshift-default-554bf767cb-t866z\" (UID: \"ba30c537-a242-4d8c-a15a-a21d57983742\") " pod="openshift-ingress/e2e-derive-23697b7f-openshift-default-554bf767cb-t866z" Jul 29 19:47:11.365935 ip-10-0-131-93 kubenswrapper[2564]: I0729 19:47:11.365678 2564 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-hhp4s\" (UniqueName: \"kubernetes.io/projected/ba30c537-a242-4d8c-a15a-a21d57983742-kube-api-access-hhp4s\") pod \"e2e-derive-23697b7f-openshift-default-554bf767cb-t866z\" (UID: \"ba30c537-a242-4d8c-a15a-a21d57983742\") " pod="openshift-ingress/e2e-derive-23697b7f-openshift-default-554bf767cb-t866z" Jul 29 19:47:11.365935 ip-10-0-131-93 kubenswrapper[2564]: I0729 19:47:11.365703 2564 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/ba30c537-a242-4d8c-a15a-a21d57983742-istiod-ca-cert\") pod \"e2e-derive-23697b7f-openshift-default-554bf767cb-t866z\" (UID: \"ba30c537-a242-4d8c-a15a-a21d57983742\") " pod="openshift-ingress/e2e-derive-23697b7f-openshift-default-554bf767cb-t866z" Jul 29 19:47:11.365935 ip-10-0-131-93 kubenswrapper[2564]: I0729 19:47:11.365724 2564 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/ba30c537-a242-4d8c-a15a-a21d57983742-workload-socket\") pod \"e2e-derive-23697b7f-openshift-default-554bf767cb-t866z\" (UID: \"ba30c537-a242-4d8c-a15a-a21d57983742\") " pod="openshift-ingress/e2e-derive-23697b7f-openshift-default-554bf767cb-t866z" Jul 29 19:47:11.365935 ip-10-0-131-93 kubenswrapper[2564]: I0729 19:47:11.365877 2564 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/ba30c537-a242-4d8c-a15a-a21d57983742-istio-podinfo\") pod \"e2e-derive-23697b7f-openshift-default-554bf767cb-t866z\" (UID: \"ba30c537-a242-4d8c-a15a-a21d57983742\") " pod="openshift-ingress/e2e-derive-23697b7f-openshift-default-554bf767cb-t866z" Jul 29 19:47:11.366193 ip-10-0-131-93 kubenswrapper[2564]: I0729 19:47:11.365937 2564 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/ba30c537-a242-4d8c-a15a-a21d57983742-istio-data\") pod \"e2e-derive-23697b7f-openshift-default-554bf767cb-t866z\" (UID: \"ba30c537-a242-4d8c-a15a-a21d57983742\") " pod="openshift-ingress/e2e-derive-23697b7f-openshift-default-554bf767cb-t866z" Jul 29 19:47:11.366193 ip-10-0-131-93 kubenswrapper[2564]: I0729 19:47:11.366043 2564 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/ba30c537-a242-4d8c-a15a-a21d57983742-workload-certs\") pod \"e2e-derive-23697b7f-openshift-default-554bf767cb-t866z\" (UID: \"ba30c537-a242-4d8c-a15a-a21d57983742\") " pod="openshift-ingress/e2e-derive-23697b7f-openshift-default-554bf767cb-t866z" Jul 29 19:47:11.366193 ip-10-0-131-93 kubenswrapper[2564]: I0729 19:47:11.366085 2564 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/ba30c537-a242-4d8c-a15a-a21d57983742-credential-socket\") pod \"e2e-derive-23697b7f-openshift-default-554bf767cb-t866z\" (UID: \"ba30c537-a242-4d8c-a15a-a21d57983742\") " pod="openshift-ingress/e2e-derive-23697b7f-openshift-default-554bf767cb-t866z" Jul 29 19:47:11.366346 ip-10-0-131-93 kubenswrapper[2564]: I0729 19:47:11.366191 2564 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/ba30c537-a242-4d8c-a15a-a21d57983742-workload-socket\") pod \"e2e-derive-23697b7f-openshift-default-554bf767cb-t866z\" (UID: \"ba30c537-a242-4d8c-a15a-a21d57983742\") " pod="openshift-ingress/e2e-derive-23697b7f-openshift-default-554bf767cb-t866z" Jul 29 19:47:11.366346 ip-10-0-131-93 kubenswrapper[2564]: I0729 19:47:11.366236 2564 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/ba30c537-a242-4d8c-a15a-a21d57983742-istio-data\") pod \"e2e-derive-23697b7f-openshift-default-554bf767cb-t866z\" (UID: \"ba30c537-a242-4d8c-a15a-a21d57983742\") " pod="openshift-ingress/e2e-derive-23697b7f-openshift-default-554bf767cb-t866z" Jul 29 19:47:11.366463 ip-10-0-131-93 kubenswrapper[2564]: I0729 19:47:11.366396 2564 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/ba30c537-a242-4d8c-a15a-a21d57983742-istiod-ca-cert\") pod \"e2e-derive-23697b7f-openshift-default-554bf767cb-t866z\" (UID: \"ba30c537-a242-4d8c-a15a-a21d57983742\") " pod="openshift-ingress/e2e-derive-23697b7f-openshift-default-554bf767cb-t866z" Jul 29 19:47:11.368179 ip-10-0-131-93 kubenswrapper[2564]: I0729 19:47:11.368158 2564 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/ba30c537-a242-4d8c-a15a-a21d57983742-istio-envoy\") pod \"e2e-derive-23697b7f-openshift-default-554bf767cb-t866z\" (UID: \"ba30c537-a242-4d8c-a15a-a21d57983742\") " pod="openshift-ingress/e2e-derive-23697b7f-openshift-default-554bf767cb-t866z" Jul 29 19:47:11.368414 ip-10-0-131-93 kubenswrapper[2564]: I0729 19:47:11.368391 2564 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/ba30c537-a242-4d8c-a15a-a21d57983742-istio-podinfo\") pod \"e2e-derive-23697b7f-openshift-default-554bf767cb-t866z\" (UID: \"ba30c537-a242-4d8c-a15a-a21d57983742\") " pod="openshift-ingress/e2e-derive-23697b7f-openshift-default-554bf767cb-t866z" Jul 29 19:47:11.378799 ip-10-0-131-93 kubenswrapper[2564]: I0729 19:47:11.378766 2564 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/ba30c537-a242-4d8c-a15a-a21d57983742-istio-token\") pod \"e2e-derive-23697b7f-openshift-default-554bf767cb-t866z\" (UID: \"ba30c537-a242-4d8c-a15a-a21d57983742\") " pod="openshift-ingress/e2e-derive-23697b7f-openshift-default-554bf767cb-t866z" Jul 29 19:47:11.392527 ip-10-0-131-93 kubenswrapper[2564]: I0729 19:47:11.392470 2564 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-hhp4s\" (UniqueName: \"kubernetes.io/projected/ba30c537-a242-4d8c-a15a-a21d57983742-kube-api-access-hhp4s\") pod \"e2e-derive-23697b7f-openshift-default-554bf767cb-t866z\" (UID: \"ba30c537-a242-4d8c-a15a-a21d57983742\") " pod="openshift-ingress/e2e-derive-23697b7f-openshift-default-554bf767cb-t866z" Jul 29 19:47:11.405335 ip-10-0-131-93 kubenswrapper[2564]: I0729 19:47:11.405302 2564 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-derive-23697b7f-openshift-default-554bf767cb-t866z" Jul 29 19:47:11.489650 ip-10-0-131-93 kubenswrapper[2564]: I0729 19:47:11.489620 2564 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-444jx_faa8c546-c75b-4e10-93b4-661aa2b5f53e/konnectivity-agent/0.log" limit=1024 Jul 29 19:47:11.550010 ip-10-0-131-93 kubenswrapper[2564]: I0729 19:47:11.549980 2564 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/e2e-derive-23697b7f-openshift-default-554bf767cb-t866z"] Jul 29 19:47:11.551145 ip-10-0-131-93 kubenswrapper[2564]: W0729 19:47:11.551115 2564 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podba30c537_a242_4d8c_a15a_a21d57983742.slice/crio-43f5401287b62a34899cf8a52ce82f2481b9d0c9269d2ffdd7ed46a4d0a684a6 WatchSource:0}: Error finding container 43f5401287b62a34899cf8a52ce82f2481b9d0c9269d2ffdd7ed46a4d0a684a6: Status 404 returned error can't find the container with id 43f5401287b62a34899cf8a52ce82f2481b9d0c9269d2ffdd7ed46a4d0a684a6 Jul 29 19:47:11.553233 ip-10-0-131-93 kubenswrapper[2564]: I0729 19:47:11.553205 2564 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236224Ki","pods":"250"} Jul 29 19:47:11.553294 ip-10-0-131-93 kubenswrapper[2564]: I0729 19:47:11.553268 2564 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236224Ki","pods":"250"} Jul 29 19:47:11.553335 ip-10-0-131-93 kubenswrapper[2564]: I0729 19:47:11.553313 2564 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236224Ki","pods":"250"} Jul 29 19:47:11.965314 ip-10-0-131-93 kubenswrapper[2564]: I0729 19:47:11.965279 2564 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-derive-23697b7f-openshift-default-554bf767cb-t866z" event={"ID":"ba30c537-a242-4d8c-a15a-a21d57983742","Type":"ContainerStarted","Data":"ee9b85502b1e75c76f2e936390bdd5a4608d00405d239dd7d97ae3f6988e6eaa"} Jul 29 19:47:11.965507 ip-10-0-131-93 kubenswrapper[2564]: I0729 19:47:11.965332 2564 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-derive-23697b7f-openshift-default-554bf767cb-t866z" event={"ID":"ba30c537-a242-4d8c-a15a-a21d57983742","Type":"ContainerStarted","Data":"43f5401287b62a34899cf8a52ce82f2481b9d0c9269d2ffdd7ed46a4d0a684a6"} Jul 29 19:47:11.992360 ip-10-0-131-93 kubenswrapper[2564]: I0729 19:47:11.992309 2564 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-ingress/e2e-derive-23697b7f-openshift-default-554bf767cb-t866z" podStartSLOduration=0.992293882 podStartE2EDuration="992.293882ms" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-29 19:47:11 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-29 19:47:11.990737895 +0000 UTC m=+1807.714797596" watchObservedRunningTime="2026-07-29 19:47:11.992293882 +0000 UTC m=+1807.716353574" Jul 29 19:47:12.406101 ip-10-0-131-93 kubenswrapper[2564]: I0729 19:47:12.406014 2564 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="openshift-ingress/e2e-derive-23697b7f-openshift-default-554bf767cb-t866z" Jul 29 19:47:12.410860 ip-10-0-131-93 kubenswrapper[2564]: I0729 19:47:12.410835 2564 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="openshift-ingress/e2e-derive-23697b7f-openshift-default-554bf767cb-t866z" Jul 29 19:47:12.970855 ip-10-0-131-93 kubenswrapper[2564]: I0729 19:47:12.970818 2564 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/e2e-derive-23697b7f-openshift-default-554bf767cb-t866z" Jul 29 19:47:12.971773 ip-10-0-131-93 kubenswrapper[2564]: I0729 19:47:12.971752 2564 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/e2e-derive-23697b7f-openshift-default-554bf767cb-t866z" Jul 29 19:47:13.328309 ip-10-0-131-93 kubenswrapper[2564]: I0729 19:47:13.328228 2564 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-444jx_faa8c546-c75b-4e10-93b4-661aa2b5f53e/konnectivity-agent/0.log" limit=1024 Jul 29 19:47:13.422525 ip-10-0-131-93 kubenswrapper[2564]: I0729 19:47:13.422490 2564 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-444jx_faa8c546-c75b-4e10-93b4-661aa2b5f53e/konnectivity-agent/0.log" limit=1024 Jul 29 19:47:14.453098 ip-10-0-131-93 kubenswrapper[2564]: I0729 19:47:14.453064 2564 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-444jx_faa8c546-c75b-4e10-93b4-661aa2b5f53e/konnectivity-agent/0.log" limit=1024 Jul 29 19:47:14.959526 ip-10-0-131-93 kubenswrapper[2564]: I0729 19:47:14.959482 2564 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-444jx_faa8c546-c75b-4e10-93b4-661aa2b5f53e/konnectivity-agent/0.log" limit=1024 Jul 29 19:47:15.057204 ip-10-0-131-93 kubenswrapper[2564]: I0729 19:47:15.057165 2564 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-444jx_faa8c546-c75b-4e10-93b4-661aa2b5f53e/konnectivity-agent/0.log" limit=1024 Jul 29 19:47:15.137311 ip-10-0-131-93 kubenswrapper[2564]: I0729 19:47:15.137275 2564 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-444jx_faa8c546-c75b-4e10-93b4-661aa2b5f53e/konnectivity-agent/0.log" limit=1024 Jul 29 19:47:15.655337 ip-10-0-131-93 kubenswrapper[2564]: I0729 19:47:15.655311 2564 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-ait-a4a71d55-openshift-default-c9c5dc8bd-rl5p8" Jul 29 19:47:15.810404 ip-10-0-131-93 kubenswrapper[2564]: I0729 19:47:15.810353 2564 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/4e980d2a-9b00-4791-a563-78cb03cf6697-istio-podinfo\") pod \"4e980d2a-9b00-4791-a563-78cb03cf6697\" (UID: \"4e980d2a-9b00-4791-a563-78cb03cf6697\") " Jul 29 19:47:15.810595 ip-10-0-131-93 kubenswrapper[2564]: I0729 19:47:15.810436 2564 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-d56p5\" (UniqueName: \"kubernetes.io/projected/4e980d2a-9b00-4791-a563-78cb03cf6697-kube-api-access-d56p5\") pod \"4e980d2a-9b00-4791-a563-78cb03cf6697\" (UID: \"4e980d2a-9b00-4791-a563-78cb03cf6697\") " Jul 29 19:47:15.810595 ip-10-0-131-93 kubenswrapper[2564]: I0729 19:47:15.810469 2564 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/4e980d2a-9b00-4791-a563-78cb03cf6697-istio-data\") pod \"4e980d2a-9b00-4791-a563-78cb03cf6697\" (UID: \"4e980d2a-9b00-4791-a563-78cb03cf6697\") " Jul 29 19:47:15.810595 ip-10-0-131-93 kubenswrapper[2564]: I0729 19:47:15.810504 2564 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/4e980d2a-9b00-4791-a563-78cb03cf6697-credential-socket\") pod \"4e980d2a-9b00-4791-a563-78cb03cf6697\" (UID: \"4e980d2a-9b00-4791-a563-78cb03cf6697\") " Jul 29 19:47:15.810595 ip-10-0-131-93 kubenswrapper[2564]: I0729 19:47:15.810566 2564 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/4e980d2a-9b00-4791-a563-78cb03cf6697-istio-envoy\") pod \"4e980d2a-9b00-4791-a563-78cb03cf6697\" (UID: \"4e980d2a-9b00-4791-a563-78cb03cf6697\") " Jul 29 19:47:15.810834 ip-10-0-131-93 kubenswrapper[2564]: I0729 19:47:15.810622 2564 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/4e980d2a-9b00-4791-a563-78cb03cf6697-istiod-ca-cert\") pod \"4e980d2a-9b00-4791-a563-78cb03cf6697\" (UID: \"4e980d2a-9b00-4791-a563-78cb03cf6697\") " Jul 29 19:47:15.810834 ip-10-0-131-93 kubenswrapper[2564]: I0729 19:47:15.810677 2564 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/4e980d2a-9b00-4791-a563-78cb03cf6697-workload-certs\") pod \"4e980d2a-9b00-4791-a563-78cb03cf6697\" (UID: \"4e980d2a-9b00-4791-a563-78cb03cf6697\") " Jul 29 19:47:15.811868 ip-10-0-131-93 kubenswrapper[2564]: I0729 19:47:15.811010 2564 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/4e980d2a-9b00-4791-a563-78cb03cf6697-workload-certs" (OuterVolumeSpecName: "workload-certs") pod "4e980d2a-9b00-4791-a563-78cb03cf6697" (UID: "4e980d2a-9b00-4791-a563-78cb03cf6697"). InnerVolumeSpecName "workload-certs". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Jul 29 19:47:15.811868 ip-10-0-131-93 kubenswrapper[2564]: I0729 19:47:15.811021 2564 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/4e980d2a-9b00-4791-a563-78cb03cf6697-istio-token\") pod \"4e980d2a-9b00-4791-a563-78cb03cf6697\" (UID: \"4e980d2a-9b00-4791-a563-78cb03cf6697\") " Jul 29 19:47:15.811868 ip-10-0-131-93 kubenswrapper[2564]: I0729 19:47:15.811079 2564 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/4e980d2a-9b00-4791-a563-78cb03cf6697-workload-socket\") pod \"4e980d2a-9b00-4791-a563-78cb03cf6697\" (UID: \"4e980d2a-9b00-4791-a563-78cb03cf6697\") " Jul 29 19:47:15.811868 ip-10-0-131-93 kubenswrapper[2564]: I0729 19:47:15.811362 2564 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/4e980d2a-9b00-4791-a563-78cb03cf6697-credential-socket" (OuterVolumeSpecName: "credential-socket") pod "4e980d2a-9b00-4791-a563-78cb03cf6697" (UID: "4e980d2a-9b00-4791-a563-78cb03cf6697"). InnerVolumeSpecName "credential-socket". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Jul 29 19:47:15.811868 ip-10-0-131-93 kubenswrapper[2564]: I0729 19:47:15.811492 2564 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/4e980d2a-9b00-4791-a563-78cb03cf6697-istiod-ca-cert" (OuterVolumeSpecName: "istiod-ca-cert") pod "4e980d2a-9b00-4791-a563-78cb03cf6697" (UID: "4e980d2a-9b00-4791-a563-78cb03cf6697"). InnerVolumeSpecName "istiod-ca-cert". PluginName "kubernetes.io/configmap", VolumeGIDValue "" Jul 29 19:47:15.811868 ip-10-0-131-93 kubenswrapper[2564]: I0729 19:47:15.811573 2564 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/4e980d2a-9b00-4791-a563-78cb03cf6697-istio-data" (OuterVolumeSpecName: "istio-data") pod "4e980d2a-9b00-4791-a563-78cb03cf6697" (UID: "4e980d2a-9b00-4791-a563-78cb03cf6697"). InnerVolumeSpecName "istio-data". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Jul 29 19:47:15.811868 ip-10-0-131-93 kubenswrapper[2564]: I0729 19:47:15.811836 2564 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/4e980d2a-9b00-4791-a563-78cb03cf6697-workload-socket" (OuterVolumeSpecName: "workload-socket") pod "4e980d2a-9b00-4791-a563-78cb03cf6697" (UID: "4e980d2a-9b00-4791-a563-78cb03cf6697"). InnerVolumeSpecName "workload-socket". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Jul 29 19:47:15.812323 ip-10-0-131-93 kubenswrapper[2564]: I0729 19:47:15.812307 2564 reconciler_common.go:299] "Volume detached for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/4e980d2a-9b00-4791-a563-78cb03cf6697-istio-data\") on node \"ip-10-0-131-93.ec2.internal\" DevicePath \"\"" Jul 29 19:47:15.812400 ip-10-0-131-93 kubenswrapper[2564]: I0729 19:47:15.812331 2564 reconciler_common.go:299] "Volume detached for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/4e980d2a-9b00-4791-a563-78cb03cf6697-credential-socket\") on node \"ip-10-0-131-93.ec2.internal\" DevicePath \"\"" Jul 29 19:47:15.812400 ip-10-0-131-93 kubenswrapper[2564]: I0729 19:47:15.812347 2564 reconciler_common.go:299] "Volume detached for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/4e980d2a-9b00-4791-a563-78cb03cf6697-istiod-ca-cert\") on node \"ip-10-0-131-93.ec2.internal\" DevicePath \"\"" Jul 29 19:47:15.812400 ip-10-0-131-93 kubenswrapper[2564]: I0729 19:47:15.812362 2564 reconciler_common.go:299] "Volume detached for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/4e980d2a-9b00-4791-a563-78cb03cf6697-workload-certs\") on node \"ip-10-0-131-93.ec2.internal\" DevicePath \"\"" Jul 29 19:47:15.812400 ip-10-0-131-93 kubenswrapper[2564]: I0729 19:47:15.812390 2564 reconciler_common.go:299] "Volume detached for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/4e980d2a-9b00-4791-a563-78cb03cf6697-workload-socket\") on node \"ip-10-0-131-93.ec2.internal\" DevicePath \"\"" Jul 29 19:47:15.812928 ip-10-0-131-93 kubenswrapper[2564]: I0729 19:47:15.812895 2564 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/downward-api/4e980d2a-9b00-4791-a563-78cb03cf6697-istio-podinfo" (OuterVolumeSpecName: "istio-podinfo") pod "4e980d2a-9b00-4791-a563-78cb03cf6697" (UID: "4e980d2a-9b00-4791-a563-78cb03cf6697"). InnerVolumeSpecName "istio-podinfo". PluginName "kubernetes.io/downward-api", VolumeGIDValue "" Jul 29 19:47:15.813114 ip-10-0-131-93 kubenswrapper[2564]: I0729 19:47:15.813095 2564 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/4e980d2a-9b00-4791-a563-78cb03cf6697-kube-api-access-d56p5" (OuterVolumeSpecName: "kube-api-access-d56p5") pod "4e980d2a-9b00-4791-a563-78cb03cf6697" (UID: "4e980d2a-9b00-4791-a563-78cb03cf6697"). InnerVolumeSpecName "kube-api-access-d56p5". PluginName "kubernetes.io/projected", VolumeGIDValue "" Jul 29 19:47:15.813163 ip-10-0-131-93 kubenswrapper[2564]: I0729 19:47:15.813132 2564 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/4e980d2a-9b00-4791-a563-78cb03cf6697-istio-envoy" (OuterVolumeSpecName: "istio-envoy") pod "4e980d2a-9b00-4791-a563-78cb03cf6697" (UID: "4e980d2a-9b00-4791-a563-78cb03cf6697"). InnerVolumeSpecName "istio-envoy". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Jul 29 19:47:15.813309 ip-10-0-131-93 kubenswrapper[2564]: I0729 19:47:15.813286 2564 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/4e980d2a-9b00-4791-a563-78cb03cf6697-istio-token" (OuterVolumeSpecName: "istio-token") pod "4e980d2a-9b00-4791-a563-78cb03cf6697" (UID: "4e980d2a-9b00-4791-a563-78cb03cf6697"). InnerVolumeSpecName "istio-token". PluginName "kubernetes.io/projected", VolumeGIDValue "" Jul 29 19:47:15.913485 ip-10-0-131-93 kubenswrapper[2564]: I0729 19:47:15.913447 2564 reconciler_common.go:299] "Volume detached for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/4e980d2a-9b00-4791-a563-78cb03cf6697-istio-token\") on node \"ip-10-0-131-93.ec2.internal\" DevicePath \"\"" Jul 29 19:47:15.913704 ip-10-0-131-93 kubenswrapper[2564]: I0729 19:47:15.913507 2564 reconciler_common.go:299] "Volume detached for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/4e980d2a-9b00-4791-a563-78cb03cf6697-istio-podinfo\") on node \"ip-10-0-131-93.ec2.internal\" DevicePath \"\"" Jul 29 19:47:15.913704 ip-10-0-131-93 kubenswrapper[2564]: I0729 19:47:15.913523 2564 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-d56p5\" (UniqueName: \"kubernetes.io/projected/4e980d2a-9b00-4791-a563-78cb03cf6697-kube-api-access-d56p5\") on node \"ip-10-0-131-93.ec2.internal\" DevicePath \"\"" Jul 29 19:47:15.913704 ip-10-0-131-93 kubenswrapper[2564]: I0729 19:47:15.913538 2564 reconciler_common.go:299] "Volume detached for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/4e980d2a-9b00-4791-a563-78cb03cf6697-istio-envoy\") on node \"ip-10-0-131-93.ec2.internal\" DevicePath \"\"" Jul 29 19:47:15.989194 ip-10-0-131-93 kubenswrapper[2564]: I0729 19:47:15.989158 2564 generic.go:356] "Generic (PLEG): container finished" podID="4e980d2a-9b00-4791-a563-78cb03cf6697" containerID="4424582b8861a44d2b7ab5df7e5d63e5d3072104610dd3015b133aade62e231a" exitCode=0 Jul 29 19:47:15.989407 ip-10-0-131-93 kubenswrapper[2564]: I0729 19:47:15.989216 2564 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-ait-a4a71d55-openshift-default-c9c5dc8bd-rl5p8" Jul 29 19:47:15.989407 ip-10-0-131-93 kubenswrapper[2564]: I0729 19:47:15.989245 2564 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-ait-a4a71d55-openshift-default-c9c5dc8bd-rl5p8" event={"ID":"4e980d2a-9b00-4791-a563-78cb03cf6697","Type":"ContainerDied","Data":"4424582b8861a44d2b7ab5df7e5d63e5d3072104610dd3015b133aade62e231a"} Jul 29 19:47:15.989407 ip-10-0-131-93 kubenswrapper[2564]: I0729 19:47:15.989296 2564 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-ait-a4a71d55-openshift-default-c9c5dc8bd-rl5p8" event={"ID":"4e980d2a-9b00-4791-a563-78cb03cf6697","Type":"ContainerDied","Data":"d1d6b96f610dada81438abdf0e0cc7299610193f922251a1cb4edefb87782832"} Jul 29 19:47:15.989407 ip-10-0-131-93 kubenswrapper[2564]: I0729 19:47:15.989321 2564 scope.go:117] "RemoveContainer" containerID="4424582b8861a44d2b7ab5df7e5d63e5d3072104610dd3015b133aade62e231a" Jul 29 19:47:15.999312 ip-10-0-131-93 kubenswrapper[2564]: I0729 19:47:15.999296 2564 scope.go:117] "RemoveContainer" containerID="a635dcf92daebdde2feb1db9cb53d9dbaebbb570749eb7a8567bc236a157a115" Jul 29 19:47:16.007112 ip-10-0-131-93 kubenswrapper[2564]: I0729 19:47:16.007095 2564 scope.go:117] "RemoveContainer" containerID="4424582b8861a44d2b7ab5df7e5d63e5d3072104610dd3015b133aade62e231a" Jul 29 19:47:16.007359 ip-10-0-131-93 kubenswrapper[2564]: E0729 19:47:16.007343 2564 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"4424582b8861a44d2b7ab5df7e5d63e5d3072104610dd3015b133aade62e231a\": container with ID starting with 4424582b8861a44d2b7ab5df7e5d63e5d3072104610dd3015b133aade62e231a not found: ID does not exist" containerID="4424582b8861a44d2b7ab5df7e5d63e5d3072104610dd3015b133aade62e231a" Jul 29 19:47:16.007412 ip-10-0-131-93 kubenswrapper[2564]: I0729 19:47:16.007367 2564 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"4424582b8861a44d2b7ab5df7e5d63e5d3072104610dd3015b133aade62e231a"} err="failed to get container status \"4424582b8861a44d2b7ab5df7e5d63e5d3072104610dd3015b133aade62e231a\": rpc error: code = NotFound desc = could not find container \"4424582b8861a44d2b7ab5df7e5d63e5d3072104610dd3015b133aade62e231a\": container with ID starting with 4424582b8861a44d2b7ab5df7e5d63e5d3072104610dd3015b133aade62e231a not found: ID does not exist" Jul 29 19:47:16.007412 ip-10-0-131-93 kubenswrapper[2564]: I0729 19:47:16.007398 2564 scope.go:117] "RemoveContainer" containerID="a635dcf92daebdde2feb1db9cb53d9dbaebbb570749eb7a8567bc236a157a115" Jul 29 19:47:16.007651 ip-10-0-131-93 kubenswrapper[2564]: E0729 19:47:16.007631 2564 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"a635dcf92daebdde2feb1db9cb53d9dbaebbb570749eb7a8567bc236a157a115\": container with ID starting with a635dcf92daebdde2feb1db9cb53d9dbaebbb570749eb7a8567bc236a157a115 not found: ID does not exist" containerID="a635dcf92daebdde2feb1db9cb53d9dbaebbb570749eb7a8567bc236a157a115" Jul 29 19:47:16.007699 ip-10-0-131-93 kubenswrapper[2564]: I0729 19:47:16.007659 2564 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"a635dcf92daebdde2feb1db9cb53d9dbaebbb570749eb7a8567bc236a157a115"} err="failed to get container status \"a635dcf92daebdde2feb1db9cb53d9dbaebbb570749eb7a8567bc236a157a115\": rpc error: code = NotFound desc = could not find container \"a635dcf92daebdde2feb1db9cb53d9dbaebbb570749eb7a8567bc236a157a115\": container with ID starting with a635dcf92daebdde2feb1db9cb53d9dbaebbb570749eb7a8567bc236a157a115 not found: ID does not exist" Jul 29 19:47:16.012596 ip-10-0-131-93 kubenswrapper[2564]: I0729 19:47:16.012574 2564 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-ingress/e2e-ait-a4a71d55-openshift-default-c9c5dc8bd-rl5p8"] Jul 29 19:47:16.016485 ip-10-0-131-93 kubenswrapper[2564]: I0729 19:47:16.016463 2564 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["openshift-ingress/e2e-ait-a4a71d55-openshift-default-c9c5dc8bd-rl5p8"] Jul 29 19:47:16.782702 ip-10-0-131-93 kubenswrapper[2564]: I0729 19:47:16.782673 2564 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="4e980d2a-9b00-4791-a563-78cb03cf6697" path="/var/lib/kubelet/pods/4e980d2a-9b00-4791-a563-78cb03cf6697/volumes" Jul 29 19:47:27.047944 ip-10-0-131-93 kubenswrapper[2564]: I0729 19:47:27.047848 2564 generic.go:356] "Generic (PLEG): container finished" podID="ba30c537-a242-4d8c-a15a-a21d57983742" containerID="ee9b85502b1e75c76f2e936390bdd5a4608d00405d239dd7d97ae3f6988e6eaa" exitCode=0 Jul 29 19:47:27.047944 ip-10-0-131-93 kubenswrapper[2564]: I0729 19:47:27.047923 2564 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-derive-23697b7f-openshift-default-554bf767cb-t866z" event={"ID":"ba30c537-a242-4d8c-a15a-a21d57983742","Type":"ContainerDied","Data":"ee9b85502b1e75c76f2e936390bdd5a4608d00405d239dd7d97ae3f6988e6eaa"} Jul 29 19:47:27.048430 ip-10-0-131-93 kubenswrapper[2564]: I0729 19:47:27.048297 2564 scope.go:117] "RemoveContainer" containerID="ee9b85502b1e75c76f2e936390bdd5a4608d00405d239dd7d97ae3f6988e6eaa" Jul 29 19:47:27.049345 ip-10-0-131-93 kubenswrapper[2564]: I0729 19:47:27.049321 2564 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236224Ki","pods":"250"} Jul 29 19:47:27.049418 ip-10-0-131-93 kubenswrapper[2564]: I0729 19:47:27.049401 2564 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236224Ki","pods":"250"} Jul 29 19:47:27.049456 ip-10-0-131-93 kubenswrapper[2564]: I0729 19:47:27.049435 2564 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236224Ki","pods":"250"} Jul 29 19:47:27.971445 ip-10-0-131-93 kubenswrapper[2564]: I0729 19:47:27.971406 2564 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/e2e-derive-23697b7f-openshift-default-554bf767cb-t866z" Jul 29 19:47:28.054718 ip-10-0-131-93 kubenswrapper[2564]: I0729 19:47:28.054683 2564 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-derive-23697b7f-openshift-default-554bf767cb-t866z" event={"ID":"ba30c537-a242-4d8c-a15a-a21d57983742","Type":"ContainerStarted","Data":"c4555e8d7383068c6a5c6e97442ab02e4082bc4b6aac539b666d4235e9f39afd"} Jul 29 19:47:28.406013 ip-10-0-131-93 kubenswrapper[2564]: I0729 19:47:28.405918 2564 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="openshift-ingress/e2e-derive-23697b7f-openshift-default-554bf767cb-t866z" Jul 29 19:47:28.407002 ip-10-0-131-93 kubenswrapper[2564]: I0729 19:47:28.406973 2564 patch_prober.go:28] interesting pod/e2e-derive-23697b7f-openshift-default-554bf767cb-t866z container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.132.0.40:15021/healthz/ready\": dial tcp 10.132.0.40:15021: connect: connection refused" start-of-body= Jul 29 19:47:28.407153 ip-10-0-131-93 kubenswrapper[2564]: I0729 19:47:28.407030 2564 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-derive-23697b7f-openshift-default-554bf767cb-t866z" podUID="ba30c537-a242-4d8c-a15a-a21d57983742" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.132.0.40:15021/healthz/ready\": dial tcp 10.132.0.40:15021: connect: connection refused" Jul 29 19:47:29.405847 ip-10-0-131-93 kubenswrapper[2564]: I0729 19:47:29.405802 2564 patch_prober.go:28] interesting pod/e2e-derive-23697b7f-openshift-default-554bf767cb-t866z container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.132.0.40:15021/healthz/ready\": dial tcp 10.132.0.40:15021: connect: connection refused" start-of-body= Jul 29 19:47:29.406248 ip-10-0-131-93 kubenswrapper[2564]: I0729 19:47:29.405861 2564 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-derive-23697b7f-openshift-default-554bf767cb-t866z" podUID="ba30c537-a242-4d8c-a15a-a21d57983742" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.132.0.40:15021/healthz/ready\": dial tcp 10.132.0.40:15021: connect: connection refused" Jul 29 19:47:30.406554 ip-10-0-131-93 kubenswrapper[2564]: I0729 19:47:30.406511 2564 patch_prober.go:28] interesting pod/e2e-derive-23697b7f-openshift-default-554bf767cb-t866z container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.132.0.40:15021/healthz/ready\": dial tcp 10.132.0.40:15021: connect: connection refused" start-of-body= Jul 29 19:47:30.406962 ip-10-0-131-93 kubenswrapper[2564]: I0729 19:47:30.406590 2564 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-derive-23697b7f-openshift-default-554bf767cb-t866z" podUID="ba30c537-a242-4d8c-a15a-a21d57983742" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.132.0.40:15021/healthz/ready\": dial tcp 10.132.0.40:15021: connect: connection refused" Jul 29 19:47:31.406198 ip-10-0-131-93 kubenswrapper[2564]: I0729 19:47:31.406154 2564 patch_prober.go:28] interesting pod/e2e-derive-23697b7f-openshift-default-554bf767cb-t866z container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.132.0.40:15021/healthz/ready\": dial tcp 10.132.0.40:15021: connect: connection refused" start-of-body= Jul 29 19:47:31.406440 ip-10-0-131-93 kubenswrapper[2564]: I0729 19:47:31.406223 2564 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-derive-23697b7f-openshift-default-554bf767cb-t866z" podUID="ba30c537-a242-4d8c-a15a-a21d57983742" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.132.0.40:15021/healthz/ready\": dial tcp 10.132.0.40:15021: connect: connection refused" Jul 29 19:47:32.409956 ip-10-0-131-93 kubenswrapper[2564]: I0729 19:47:32.409920 2564 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="openshift-ingress/e2e-derive-23697b7f-openshift-default-554bf767cb-t866z" Jul 29 19:47:32.410442 ip-10-0-131-93 kubenswrapper[2564]: I0729 19:47:32.410231 2564 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/e2e-derive-23697b7f-openshift-default-554bf767cb-t866z" Jul 29 19:47:32.410893 ip-10-0-131-93 kubenswrapper[2564]: I0729 19:47:32.410875 2564 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/e2e-derive-23697b7f-openshift-default-554bf767cb-t866z" Jul 29 19:47:45.007679 ip-10-0-131-93 kubenswrapper[2564]: I0729 19:47:45.007643 2564 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-444jx_faa8c546-c75b-4e10-93b4-661aa2b5f53e/konnectivity-agent/0.log" limit=1024 Jul 29 19:47:45.107437 ip-10-0-131-93 kubenswrapper[2564]: I0729 19:47:45.107402 2564 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-444jx_faa8c546-c75b-4e10-93b4-661aa2b5f53e/konnectivity-agent/0.log" limit=1024 Jul 29 19:47:45.190156 ip-10-0-131-93 kubenswrapper[2564]: I0729 19:47:45.190119 2564 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-444jx_faa8c546-c75b-4e10-93b4-661aa2b5f53e/konnectivity-agent/0.log" limit=1024 Jul 29 19:47:45.341275 ip-10-0-131-93 kubenswrapper[2564]: I0729 19:47:45.341204 2564 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-444jx_faa8c546-c75b-4e10-93b4-661aa2b5f53e/konnectivity-agent/0.log" limit=1024 Jul 29 19:47:45.388129 ip-10-0-131-93 kubenswrapper[2564]: I0729 19:47:45.388091 2564 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-ingress/e2e-derive-23697b7f-openshift-default-554bf767cb-t866z"] Jul 29 19:47:45.388421 ip-10-0-131-93 kubenswrapper[2564]: I0729 19:47:45.388391 2564 kuberuntime_container.go:873] "Killing container with a grace period" pod="openshift-ingress/e2e-derive-23697b7f-openshift-default-554bf767cb-t866z" podUID="ba30c537-a242-4d8c-a15a-a21d57983742" containerName="istio-proxy" containerID="cri-o://c4555e8d7383068c6a5c6e97442ab02e4082bc4b6aac539b666d4235e9f39afd" gracePeriod=30 Jul 29 19:47:45.422328 ip-10-0-131-93 kubenswrapper[2564]: I0729 19:47:45.422293 2564 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-444jx_faa8c546-c75b-4e10-93b4-661aa2b5f53e/konnectivity-agent/0.log" limit=1024 Jul 29 19:47:45.617150 ip-10-0-131-93 kubenswrapper[2564]: I0729 19:47:45.617058 2564 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-444jx_faa8c546-c75b-4e10-93b4-661aa2b5f53e/konnectivity-agent/0.log" limit=1024 Jul 29 19:47:46.117482 ip-10-0-131-93 kubenswrapper[2564]: I0729 19:47:46.117422 2564 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-444jx_faa8c546-c75b-4e10-93b4-661aa2b5f53e/konnectivity-agent/0.log" limit=1024 Jul 29 19:47:46.202483 ip-10-0-131-93 kubenswrapper[2564]: I0729 19:47:46.202440 2564 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-444jx_faa8c546-c75b-4e10-93b4-661aa2b5f53e/konnectivity-agent/0.log" limit=1024 Jul 29 19:47:47.413202 ip-10-0-131-93 kubenswrapper[2564]: I0729 19:47:47.413153 2564 patch_prober.go:28] interesting pod/e2e-derive-23697b7f-openshift-default-554bf767cb-t866z container/istio-proxy namespace/openshift-ingress: Readiness probe status=failure output="HTTP probe failed with statuscode: 503" start-of-body= Jul 29 19:47:47.413996 ip-10-0-131-93 kubenswrapper[2564]: I0729 19:47:47.413239 2564 prober.go:120] "Probe failed" probeType="Readiness" pod="openshift-ingress/e2e-derive-23697b7f-openshift-default-554bf767cb-t866z" podUID="ba30c537-a242-4d8c-a15a-a21d57983742" containerName="istio-proxy" probeResult="failure" output="HTTP probe failed with statuscode: 503" Jul 29 19:47:47.723252 ip-10-0-131-93 kubenswrapper[2564]: I0729 19:47:47.723222 2564 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-444jx_faa8c546-c75b-4e10-93b4-661aa2b5f53e/konnectivity-agent/0.log" limit=1024 Jul 29 19:47:50.631686 ip-10-0-131-93 kubenswrapper[2564]: I0729 19:47:50.631662 2564 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-derive-23697b7f-openshift-default-554bf767cb-t866z" Jul 29 19:47:50.723473 ip-10-0-131-93 kubenswrapper[2564]: I0729 19:47:50.723436 2564 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/ba30c537-a242-4d8c-a15a-a21d57983742-workload-certs\") pod \"ba30c537-a242-4d8c-a15a-a21d57983742\" (UID: \"ba30c537-a242-4d8c-a15a-a21d57983742\") " Jul 29 19:47:50.723677 ip-10-0-131-93 kubenswrapper[2564]: I0729 19:47:50.723486 2564 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/ba30c537-a242-4d8c-a15a-a21d57983742-istio-token\") pod \"ba30c537-a242-4d8c-a15a-a21d57983742\" (UID: \"ba30c537-a242-4d8c-a15a-a21d57983742\") " Jul 29 19:47:50.723677 ip-10-0-131-93 kubenswrapper[2564]: I0729 19:47:50.723510 2564 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/ba30c537-a242-4d8c-a15a-a21d57983742-workload-socket\") pod \"ba30c537-a242-4d8c-a15a-a21d57983742\" (UID: \"ba30c537-a242-4d8c-a15a-a21d57983742\") " Jul 29 19:47:50.723677 ip-10-0-131-93 kubenswrapper[2564]: I0729 19:47:50.723557 2564 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/ba30c537-a242-4d8c-a15a-a21d57983742-istio-envoy\") pod \"ba30c537-a242-4d8c-a15a-a21d57983742\" (UID: \"ba30c537-a242-4d8c-a15a-a21d57983742\") " Jul 29 19:47:50.723853 ip-10-0-131-93 kubenswrapper[2564]: I0729 19:47:50.723698 2564 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/ba30c537-a242-4d8c-a15a-a21d57983742-credential-socket\") pod \"ba30c537-a242-4d8c-a15a-a21d57983742\" (UID: \"ba30c537-a242-4d8c-a15a-a21d57983742\") " Jul 29 19:47:50.723853 ip-10-0-131-93 kubenswrapper[2564]: I0729 19:47:50.723747 2564 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/ba30c537-a242-4d8c-a15a-a21d57983742-istio-podinfo\") pod \"ba30c537-a242-4d8c-a15a-a21d57983742\" (UID: \"ba30c537-a242-4d8c-a15a-a21d57983742\") " Jul 29 19:47:50.723853 ip-10-0-131-93 kubenswrapper[2564]: I0729 19:47:50.723796 2564 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-hhp4s\" (UniqueName: \"kubernetes.io/projected/ba30c537-a242-4d8c-a15a-a21d57983742-kube-api-access-hhp4s\") pod \"ba30c537-a242-4d8c-a15a-a21d57983742\" (UID: \"ba30c537-a242-4d8c-a15a-a21d57983742\") " Jul 29 19:47:50.723853 ip-10-0-131-93 kubenswrapper[2564]: I0729 19:47:50.723802 2564 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/ba30c537-a242-4d8c-a15a-a21d57983742-workload-certs" (OuterVolumeSpecName: "workload-certs") pod "ba30c537-a242-4d8c-a15a-a21d57983742" (UID: "ba30c537-a242-4d8c-a15a-a21d57983742"). InnerVolumeSpecName "workload-certs". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Jul 29 19:47:50.724062 ip-10-0-131-93 kubenswrapper[2564]: I0729 19:47:50.723824 2564 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/ba30c537-a242-4d8c-a15a-a21d57983742-istio-data\") pod \"ba30c537-a242-4d8c-a15a-a21d57983742\" (UID: \"ba30c537-a242-4d8c-a15a-a21d57983742\") " Jul 29 19:47:50.724062 ip-10-0-131-93 kubenswrapper[2564]: I0729 19:47:50.723874 2564 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/ba30c537-a242-4d8c-a15a-a21d57983742-workload-socket" (OuterVolumeSpecName: "workload-socket") pod "ba30c537-a242-4d8c-a15a-a21d57983742" (UID: "ba30c537-a242-4d8c-a15a-a21d57983742"). InnerVolumeSpecName "workload-socket". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Jul 29 19:47:50.724062 ip-10-0-131-93 kubenswrapper[2564]: I0729 19:47:50.723953 2564 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/ba30c537-a242-4d8c-a15a-a21d57983742-credential-socket" (OuterVolumeSpecName: "credential-socket") pod "ba30c537-a242-4d8c-a15a-a21d57983742" (UID: "ba30c537-a242-4d8c-a15a-a21d57983742"). InnerVolumeSpecName "credential-socket". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Jul 29 19:47:50.724062 ip-10-0-131-93 kubenswrapper[2564]: I0729 19:47:50.723912 2564 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/ba30c537-a242-4d8c-a15a-a21d57983742-istiod-ca-cert\") pod \"ba30c537-a242-4d8c-a15a-a21d57983742\" (UID: \"ba30c537-a242-4d8c-a15a-a21d57983742\") " Jul 29 19:47:50.724272 ip-10-0-131-93 kubenswrapper[2564]: I0729 19:47:50.724213 2564 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/ba30c537-a242-4d8c-a15a-a21d57983742-istiod-ca-cert" (OuterVolumeSpecName: "istiod-ca-cert") pod "ba30c537-a242-4d8c-a15a-a21d57983742" (UID: "ba30c537-a242-4d8c-a15a-a21d57983742"). InnerVolumeSpecName "istiod-ca-cert". PluginName "kubernetes.io/configmap", VolumeGIDValue "" Jul 29 19:47:50.724325 ip-10-0-131-93 kubenswrapper[2564]: I0729 19:47:50.724308 2564 reconciler_common.go:299] "Volume detached for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/ba30c537-a242-4d8c-a15a-a21d57983742-workload-certs\") on node \"ip-10-0-131-93.ec2.internal\" DevicePath \"\"" Jul 29 19:47:50.724413 ip-10-0-131-93 kubenswrapper[2564]: I0729 19:47:50.724324 2564 reconciler_common.go:299] "Volume detached for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/ba30c537-a242-4d8c-a15a-a21d57983742-workload-socket\") on node \"ip-10-0-131-93.ec2.internal\" DevicePath \"\"" Jul 29 19:47:50.724413 ip-10-0-131-93 kubenswrapper[2564]: I0729 19:47:50.724338 2564 reconciler_common.go:299] "Volume detached for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/ba30c537-a242-4d8c-a15a-a21d57983742-credential-socket\") on node \"ip-10-0-131-93.ec2.internal\" DevicePath \"\"" Jul 29 19:47:50.724413 ip-10-0-131-93 kubenswrapper[2564]: I0729 19:47:50.724352 2564 reconciler_common.go:299] "Volume detached for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/ba30c537-a242-4d8c-a15a-a21d57983742-istiod-ca-cert\") on node \"ip-10-0-131-93.ec2.internal\" DevicePath \"\"" Jul 29 19:47:50.724757 ip-10-0-131-93 kubenswrapper[2564]: I0729 19:47:50.724734 2564 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/ba30c537-a242-4d8c-a15a-a21d57983742-istio-data" (OuterVolumeSpecName: "istio-data") pod "ba30c537-a242-4d8c-a15a-a21d57983742" (UID: "ba30c537-a242-4d8c-a15a-a21d57983742"). InnerVolumeSpecName "istio-data". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Jul 29 19:47:50.725932 ip-10-0-131-93 kubenswrapper[2564]: I0729 19:47:50.725908 2564 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/ba30c537-a242-4d8c-a15a-a21d57983742-istio-envoy" (OuterVolumeSpecName: "istio-envoy") pod "ba30c537-a242-4d8c-a15a-a21d57983742" (UID: "ba30c537-a242-4d8c-a15a-a21d57983742"). InnerVolumeSpecName "istio-envoy". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Jul 29 19:47:50.726044 ip-10-0-131-93 kubenswrapper[2564]: I0729 19:47:50.725946 2564 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/ba30c537-a242-4d8c-a15a-a21d57983742-kube-api-access-hhp4s" (OuterVolumeSpecName: "kube-api-access-hhp4s") pod "ba30c537-a242-4d8c-a15a-a21d57983742" (UID: "ba30c537-a242-4d8c-a15a-a21d57983742"). InnerVolumeSpecName "kube-api-access-hhp4s". PluginName "kubernetes.io/projected", VolumeGIDValue "" Jul 29 19:47:50.726181 ip-10-0-131-93 kubenswrapper[2564]: I0729 19:47:50.726164 2564 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/downward-api/ba30c537-a242-4d8c-a15a-a21d57983742-istio-podinfo" (OuterVolumeSpecName: "istio-podinfo") pod "ba30c537-a242-4d8c-a15a-a21d57983742" (UID: "ba30c537-a242-4d8c-a15a-a21d57983742"). InnerVolumeSpecName "istio-podinfo". PluginName "kubernetes.io/downward-api", VolumeGIDValue "" Jul 29 19:47:50.726244 ip-10-0-131-93 kubenswrapper[2564]: I0729 19:47:50.726166 2564 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/ba30c537-a242-4d8c-a15a-a21d57983742-istio-token" (OuterVolumeSpecName: "istio-token") pod "ba30c537-a242-4d8c-a15a-a21d57983742" (UID: "ba30c537-a242-4d8c-a15a-a21d57983742"). InnerVolumeSpecName "istio-token". PluginName "kubernetes.io/projected", VolumeGIDValue "" Jul 29 19:47:50.825420 ip-10-0-131-93 kubenswrapper[2564]: I0729 19:47:50.825367 2564 reconciler_common.go:299] "Volume detached for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/ba30c537-a242-4d8c-a15a-a21d57983742-istio-token\") on node \"ip-10-0-131-93.ec2.internal\" DevicePath \"\"" Jul 29 19:47:50.825420 ip-10-0-131-93 kubenswrapper[2564]: I0729 19:47:50.825418 2564 reconciler_common.go:299] "Volume detached for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/ba30c537-a242-4d8c-a15a-a21d57983742-istio-envoy\") on node \"ip-10-0-131-93.ec2.internal\" DevicePath \"\"" Jul 29 19:47:50.825599 ip-10-0-131-93 kubenswrapper[2564]: I0729 19:47:50.825428 2564 reconciler_common.go:299] "Volume detached for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/ba30c537-a242-4d8c-a15a-a21d57983742-istio-podinfo\") on node \"ip-10-0-131-93.ec2.internal\" DevicePath \"\"" Jul 29 19:47:50.825599 ip-10-0-131-93 kubenswrapper[2564]: I0729 19:47:50.825437 2564 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-hhp4s\" (UniqueName: \"kubernetes.io/projected/ba30c537-a242-4d8c-a15a-a21d57983742-kube-api-access-hhp4s\") on node \"ip-10-0-131-93.ec2.internal\" DevicePath \"\"" Jul 29 19:47:50.825599 ip-10-0-131-93 kubenswrapper[2564]: I0729 19:47:50.825446 2564 reconciler_common.go:299] "Volume detached for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/ba30c537-a242-4d8c-a15a-a21d57983742-istio-data\") on node \"ip-10-0-131-93.ec2.internal\" DevicePath \"\"" Jul 29 19:47:51.174860 ip-10-0-131-93 kubenswrapper[2564]: I0729 19:47:51.174769 2564 generic.go:356] "Generic (PLEG): container finished" podID="ba30c537-a242-4d8c-a15a-a21d57983742" containerID="c4555e8d7383068c6a5c6e97442ab02e4082bc4b6aac539b666d4235e9f39afd" exitCode=0 Jul 29 19:47:51.174860 ip-10-0-131-93 kubenswrapper[2564]: I0729 19:47:51.174824 2564 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-derive-23697b7f-openshift-default-554bf767cb-t866z" Jul 29 19:47:51.175046 ip-10-0-131-93 kubenswrapper[2564]: I0729 19:47:51.174851 2564 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-derive-23697b7f-openshift-default-554bf767cb-t866z" event={"ID":"ba30c537-a242-4d8c-a15a-a21d57983742","Type":"ContainerDied","Data":"c4555e8d7383068c6a5c6e97442ab02e4082bc4b6aac539b666d4235e9f39afd"} Jul 29 19:47:51.175046 ip-10-0-131-93 kubenswrapper[2564]: I0729 19:47:51.174910 2564 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-derive-23697b7f-openshift-default-554bf767cb-t866z" event={"ID":"ba30c537-a242-4d8c-a15a-a21d57983742","Type":"ContainerDied","Data":"43f5401287b62a34899cf8a52ce82f2481b9d0c9269d2ffdd7ed46a4d0a684a6"} Jul 29 19:47:51.175046 ip-10-0-131-93 kubenswrapper[2564]: I0729 19:47:51.174937 2564 scope.go:117] "RemoveContainer" containerID="c4555e8d7383068c6a5c6e97442ab02e4082bc4b6aac539b666d4235e9f39afd" Jul 29 19:47:51.186303 ip-10-0-131-93 kubenswrapper[2564]: I0729 19:47:51.186284 2564 scope.go:117] "RemoveContainer" containerID="ee9b85502b1e75c76f2e936390bdd5a4608d00405d239dd7d97ae3f6988e6eaa" Jul 29 19:47:51.193773 ip-10-0-131-93 kubenswrapper[2564]: I0729 19:47:51.193754 2564 scope.go:117] "RemoveContainer" containerID="c4555e8d7383068c6a5c6e97442ab02e4082bc4b6aac539b666d4235e9f39afd" Jul 29 19:47:51.194018 ip-10-0-131-93 kubenswrapper[2564]: E0729 19:47:51.193998 2564 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"c4555e8d7383068c6a5c6e97442ab02e4082bc4b6aac539b666d4235e9f39afd\": container with ID starting with c4555e8d7383068c6a5c6e97442ab02e4082bc4b6aac539b666d4235e9f39afd not found: ID does not exist" containerID="c4555e8d7383068c6a5c6e97442ab02e4082bc4b6aac539b666d4235e9f39afd" Jul 29 19:47:51.194068 ip-10-0-131-93 kubenswrapper[2564]: I0729 19:47:51.194027 2564 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"c4555e8d7383068c6a5c6e97442ab02e4082bc4b6aac539b666d4235e9f39afd"} err="failed to get container status \"c4555e8d7383068c6a5c6e97442ab02e4082bc4b6aac539b666d4235e9f39afd\": rpc error: code = NotFound desc = could not find container \"c4555e8d7383068c6a5c6e97442ab02e4082bc4b6aac539b666d4235e9f39afd\": container with ID starting with c4555e8d7383068c6a5c6e97442ab02e4082bc4b6aac539b666d4235e9f39afd not found: ID does not exist" Jul 29 19:47:51.194068 ip-10-0-131-93 kubenswrapper[2564]: I0729 19:47:51.194045 2564 scope.go:117] "RemoveContainer" containerID="ee9b85502b1e75c76f2e936390bdd5a4608d00405d239dd7d97ae3f6988e6eaa" Jul 29 19:47:51.194253 ip-10-0-131-93 kubenswrapper[2564]: E0729 19:47:51.194237 2564 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"ee9b85502b1e75c76f2e936390bdd5a4608d00405d239dd7d97ae3f6988e6eaa\": container with ID starting with ee9b85502b1e75c76f2e936390bdd5a4608d00405d239dd7d97ae3f6988e6eaa not found: ID does not exist" containerID="ee9b85502b1e75c76f2e936390bdd5a4608d00405d239dd7d97ae3f6988e6eaa" Jul 29 19:47:51.194297 ip-10-0-131-93 kubenswrapper[2564]: I0729 19:47:51.194256 2564 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"ee9b85502b1e75c76f2e936390bdd5a4608d00405d239dd7d97ae3f6988e6eaa"} err="failed to get container status \"ee9b85502b1e75c76f2e936390bdd5a4608d00405d239dd7d97ae3f6988e6eaa\": rpc error: code = NotFound desc = could not find container \"ee9b85502b1e75c76f2e936390bdd5a4608d00405d239dd7d97ae3f6988e6eaa\": container with ID starting with ee9b85502b1e75c76f2e936390bdd5a4608d00405d239dd7d97ae3f6988e6eaa not found: ID does not exist" Jul 29 19:47:51.198415 ip-10-0-131-93 kubenswrapper[2564]: I0729 19:47:51.198395 2564 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-ingress/e2e-derive-23697b7f-openshift-default-554bf767cb-t866z"] Jul 29 19:47:51.203191 ip-10-0-131-93 kubenswrapper[2564]: I0729 19:47:51.203168 2564 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["openshift-ingress/e2e-derive-23697b7f-openshift-default-554bf767cb-t866z"] Jul 29 19:47:52.782853 ip-10-0-131-93 kubenswrapper[2564]: I0729 19:47:52.782823 2564 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="ba30c537-a242-4d8c-a15a-a21d57983742" path="/var/lib/kubelet/pods/ba30c537-a242-4d8c-a15a-a21d57983742/volumes" Jul 29 19:47:53.979230 ip-10-0-131-93 kubenswrapper[2564]: I0729 19:47:53.979192 2564 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-444jx_faa8c546-c75b-4e10-93b4-661aa2b5f53e/konnectivity-agent/0.log" limit=1024 Jul 29 19:47:54.103743 ip-10-0-131-93 kubenswrapper[2564]: I0729 19:47:54.103712 2564 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-444jx_faa8c546-c75b-4e10-93b4-661aa2b5f53e/konnectivity-agent/0.log" limit=1024 Jul 29 19:47:54.253779 ip-10-0-131-93 kubenswrapper[2564]: I0729 19:47:54.253693 2564 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-444jx_faa8c546-c75b-4e10-93b4-661aa2b5f53e/konnectivity-agent/0.log" limit=1024 Jul 29 19:47:54.447852 ip-10-0-131-93 kubenswrapper[2564]: I0729 19:47:54.447820 2564 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-444jx_faa8c546-c75b-4e10-93b4-661aa2b5f53e/konnectivity-agent/0.log" limit=1024 Jul 29 19:47:54.706143 ip-10-0-131-93 kubenswrapper[2564]: I0729 19:47:54.706110 2564 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-444jx_faa8c546-c75b-4e10-93b4-661aa2b5f53e/konnectivity-agent/0.log" limit=1024 Jul 29 19:48:10.186889 ip-10-0-131-93 kubenswrapper[2564]: I0729 19:48:10.186852 2564 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-444jx_faa8c546-c75b-4e10-93b4-661aa2b5f53e/konnectivity-agent/0.log" limit=1024 Jul 29 19:48:20.640112 ip-10-0-131-93 kubenswrapper[2564]: I0729 19:48:20.640076 2564 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-444jx_faa8c546-c75b-4e10-93b4-661aa2b5f53e/konnectivity-agent/0.log" limit=1024 Jul 29 19:48:27.216053 ip-10-0-131-93 kubenswrapper[2564]: I0729 19:48:27.215976 2564 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-444jx_faa8c546-c75b-4e10-93b4-661aa2b5f53e/konnectivity-agent/0.log" limit=1024 Jul 29 19:48:27.328632 ip-10-0-131-93 kubenswrapper[2564]: I0729 19:48:27.328590 2564 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-444jx_faa8c546-c75b-4e10-93b4-661aa2b5f53e/konnectivity-agent/0.log" limit=1024 Jul 29 19:48:27.415968 ip-10-0-131-93 kubenswrapper[2564]: I0729 19:48:27.415933 2564 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-444jx_faa8c546-c75b-4e10-93b4-661aa2b5f53e/konnectivity-agent/0.log" limit=1024 Jul 29 19:48:43.318452 ip-10-0-131-93 kubenswrapper[2564]: I0729 19:48:43.318415 2564 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-444jx_faa8c546-c75b-4e10-93b4-661aa2b5f53e/konnectivity-agent/0.log" limit=1024 Jul 29 19:48:49.411500 ip-10-0-131-93 kubenswrapper[2564]: I0729 19:48:49.411461 2564 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-444jx_faa8c546-c75b-4e10-93b4-661aa2b5f53e/konnectivity-agent/0.log" limit=1024 Jul 29 19:48:49.520418 ip-10-0-131-93 kubenswrapper[2564]: I0729 19:48:49.520364 2564 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-444jx_faa8c546-c75b-4e10-93b4-661aa2b5f53e/konnectivity-agent/0.log" limit=1024 Jul 29 19:48:49.614492 ip-10-0-131-93 kubenswrapper[2564]: I0729 19:48:49.614464 2564 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-444jx_faa8c546-c75b-4e10-93b4-661aa2b5f53e/konnectivity-agent/0.log" limit=1024 Jul 29 19:49:00.136777 ip-10-0-131-93 kubenswrapper[2564]: I0729 19:49:00.136738 2564 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-444jx_faa8c546-c75b-4e10-93b4-661aa2b5f53e/konnectivity-agent/0.log" limit=1024 Jul 29 19:49:00.892575 ip-10-0-131-93 kubenswrapper[2564]: I0729 19:49:00.892546 2564 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-444jx_faa8c546-c75b-4e10-93b4-661aa2b5f53e/konnectivity-agent/0.log" limit=1024 Jul 29 19:49:08.634087 ip-10-0-131-93 kubenswrapper[2564]: I0729 19:49:08.634051 2564 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-444jx_faa8c546-c75b-4e10-93b4-661aa2b5f53e/konnectivity-agent/0.log" limit=1024 Jul 29 19:49:08.745064 ip-10-0-131-93 kubenswrapper[2564]: I0729 19:49:08.745020 2564 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-444jx_faa8c546-c75b-4e10-93b4-661aa2b5f53e/konnectivity-agent/0.log" limit=1024 Jul 29 19:49:08.864677 ip-10-0-131-93 kubenswrapper[2564]: I0729 19:49:08.864641 2564 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-444jx_faa8c546-c75b-4e10-93b4-661aa2b5f53e/konnectivity-agent/0.log" limit=1024 Jul 29 19:49:08.958303 ip-10-0-131-93 kubenswrapper[2564]: I0729 19:49:08.958275 2564 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-444jx_faa8c546-c75b-4e10-93b4-661aa2b5f53e/konnectivity-agent/0.log" limit=1024 Jul 29 19:49:20.344778 ip-10-0-131-93 kubenswrapper[2564]: I0729 19:49:20.344741 2564 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-444jx_faa8c546-c75b-4e10-93b4-661aa2b5f53e/konnectivity-agent/0.log" limit=1024 Jul 29 19:49:20.447012 ip-10-0-131-93 kubenswrapper[2564]: I0729 19:49:20.446977 2564 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-444jx_faa8c546-c75b-4e10-93b4-661aa2b5f53e/konnectivity-agent/0.log" limit=1024 Jul 29 19:49:20.553487 ip-10-0-131-93 kubenswrapper[2564]: I0729 19:49:20.553446 2564 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-444jx_faa8c546-c75b-4e10-93b4-661aa2b5f53e/konnectivity-agent/0.log" limit=1024 Jul 29 19:49:22.278335 ip-10-0-131-93 kubenswrapper[2564]: I0729 19:49:22.278300 2564 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-444jx_faa8c546-c75b-4e10-93b4-661aa2b5f53e/konnectivity-agent/0.log" limit=1024 Jul 29 19:49:28.007869 ip-10-0-131-93 kubenswrapper[2564]: I0729 19:49:28.007830 2564 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-444jx_faa8c546-c75b-4e10-93b4-661aa2b5f53e/konnectivity-agent/0.log" limit=1024 Jul 29 19:49:28.078019 ip-10-0-131-93 kubenswrapper[2564]: I0729 19:49:28.077971 2564 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-444jx_faa8c546-c75b-4e10-93b4-661aa2b5f53e/konnectivity-agent/0.log" limit=1024 Jul 29 19:49:28.733043 ip-10-0-131-93 kubenswrapper[2564]: I0729 19:49:28.733011 2564 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-444jx_faa8c546-c75b-4e10-93b4-661aa2b5f53e/konnectivity-agent/0.log" limit=1024 Jul 29 19:49:35.174866 ip-10-0-131-93 kubenswrapper[2564]: I0729 19:49:35.174828 2564 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-444jx_faa8c546-c75b-4e10-93b4-661aa2b5f53e/konnectivity-agent/0.log" limit=1024 Jul 29 19:49:35.368115 ip-10-0-131-93 kubenswrapper[2564]: I0729 19:49:35.368083 2564 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-444jx_faa8c546-c75b-4e10-93b4-661aa2b5f53e/konnectivity-agent/0.log" limit=1024 Jul 29 19:49:35.474023 ip-10-0-131-93 kubenswrapper[2564]: I0729 19:49:35.473994 2564 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-444jx_faa8c546-c75b-4e10-93b4-661aa2b5f53e/konnectivity-agent/0.log" limit=1024 Jul 29 19:49:35.719775 ip-10-0-131-93 kubenswrapper[2564]: I0729 19:49:35.719740 2564 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-ingress/e2e-mt-20262e43-openshift-default-5bdbbd7c84-dz9pt"] Jul 29 19:49:35.720051 ip-10-0-131-93 kubenswrapper[2564]: I0729 19:49:35.720039 2564 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="ba30c537-a242-4d8c-a15a-a21d57983742" containerName="istio-proxy" Jul 29 19:49:35.720051 ip-10-0-131-93 kubenswrapper[2564]: I0729 19:49:35.720052 2564 state_mem.go:107] "Deleted CPUSet assignment" podUID="ba30c537-a242-4d8c-a15a-a21d57983742" containerName="istio-proxy" Jul 29 19:49:35.720187 ip-10-0-131-93 kubenswrapper[2564]: I0729 19:49:35.720060 2564 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="4e980d2a-9b00-4791-a563-78cb03cf6697" containerName="istio-proxy" Jul 29 19:49:35.720187 ip-10-0-131-93 kubenswrapper[2564]: I0729 19:49:35.720065 2564 state_mem.go:107] "Deleted CPUSet assignment" podUID="4e980d2a-9b00-4791-a563-78cb03cf6697" containerName="istio-proxy" Jul 29 19:49:35.720187 ip-10-0-131-93 kubenswrapper[2564]: I0729 19:49:35.720083 2564 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="4e980d2a-9b00-4791-a563-78cb03cf6697" containerName="istio-proxy" Jul 29 19:49:35.720187 ip-10-0-131-93 kubenswrapper[2564]: I0729 19:49:35.720089 2564 state_mem.go:107] "Deleted CPUSet assignment" podUID="4e980d2a-9b00-4791-a563-78cb03cf6697" containerName="istio-proxy" Jul 29 19:49:35.720187 ip-10-0-131-93 kubenswrapper[2564]: I0729 19:49:35.720144 2564 memory_manager.go:368] "RemoveStaleState removing state" podUID="4e980d2a-9b00-4791-a563-78cb03cf6697" containerName="istio-proxy" Jul 29 19:49:35.720187 ip-10-0-131-93 kubenswrapper[2564]: I0729 19:49:35.720151 2564 memory_manager.go:368] "RemoveStaleState removing state" podUID="ba30c537-a242-4d8c-a15a-a21d57983742" containerName="istio-proxy" Jul 29 19:49:35.720187 ip-10-0-131-93 kubenswrapper[2564]: I0729 19:49:35.720157 2564 memory_manager.go:368] "RemoveStaleState removing state" podUID="ba30c537-a242-4d8c-a15a-a21d57983742" containerName="istio-proxy" Jul 29 19:49:35.722036 ip-10-0-131-93 kubenswrapper[2564]: I0729 19:49:35.722021 2564 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-mt-20262e43-openshift-default-5bdbbd7c84-dz9pt" Jul 29 19:49:35.724857 ip-10-0-131-93 kubenswrapper[2564]: I0729 19:49:35.724814 2564 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress\"/\"e2e-mt-20262e43-openshift-default-dockercfg-wlgh4\"" Jul 29 19:49:35.751518 ip-10-0-131-93 kubenswrapper[2564]: I0729 19:49:35.751492 2564 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-444jx_faa8c546-c75b-4e10-93b4-661aa2b5f53e/konnectivity-agent/0.log" limit=1024 Jul 29 19:49:35.752107 ip-10-0-131-93 kubenswrapper[2564]: I0729 19:49:35.752087 2564 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/e2e-mt-20262e43-openshift-default-5bdbbd7c84-dz9pt"] Jul 29 19:49:35.779891 ip-10-0-131-93 kubenswrapper[2564]: I0729 19:49:35.779854 2564 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-wng9g\" (UniqueName: \"kubernetes.io/projected/910195fa-e9a6-48a6-81c9-9633feb14b95-kube-api-access-wng9g\") pod \"e2e-mt-20262e43-openshift-default-5bdbbd7c84-dz9pt\" (UID: \"910195fa-e9a6-48a6-81c9-9633feb14b95\") " pod="openshift-ingress/e2e-mt-20262e43-openshift-default-5bdbbd7c84-dz9pt" Jul 29 19:49:35.779891 ip-10-0-131-93 kubenswrapper[2564]: I0729 19:49:35.779886 2564 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/910195fa-e9a6-48a6-81c9-9633feb14b95-istio-token\") pod \"e2e-mt-20262e43-openshift-default-5bdbbd7c84-dz9pt\" (UID: \"910195fa-e9a6-48a6-81c9-9633feb14b95\") " pod="openshift-ingress/e2e-mt-20262e43-openshift-default-5bdbbd7c84-dz9pt" Jul 29 19:49:35.780058 ip-10-0-131-93 kubenswrapper[2564]: I0729 19:49:35.779916 2564 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/910195fa-e9a6-48a6-81c9-9633feb14b95-workload-certs\") pod \"e2e-mt-20262e43-openshift-default-5bdbbd7c84-dz9pt\" (UID: \"910195fa-e9a6-48a6-81c9-9633feb14b95\") " pod="openshift-ingress/e2e-mt-20262e43-openshift-default-5bdbbd7c84-dz9pt" Jul 29 19:49:35.780058 ip-10-0-131-93 kubenswrapper[2564]: I0729 19:49:35.779931 2564 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/910195fa-e9a6-48a6-81c9-9633feb14b95-istiod-ca-cert\") pod \"e2e-mt-20262e43-openshift-default-5bdbbd7c84-dz9pt\" (UID: \"910195fa-e9a6-48a6-81c9-9633feb14b95\") " pod="openshift-ingress/e2e-mt-20262e43-openshift-default-5bdbbd7c84-dz9pt" Jul 29 19:49:35.780058 ip-10-0-131-93 kubenswrapper[2564]: I0729 19:49:35.780032 2564 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/910195fa-e9a6-48a6-81c9-9633feb14b95-istio-podinfo\") pod \"e2e-mt-20262e43-openshift-default-5bdbbd7c84-dz9pt\" (UID: \"910195fa-e9a6-48a6-81c9-9633feb14b95\") " pod="openshift-ingress/e2e-mt-20262e43-openshift-default-5bdbbd7c84-dz9pt" Jul 29 19:49:35.780167 ip-10-0-131-93 kubenswrapper[2564]: I0729 19:49:35.780082 2564 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/910195fa-e9a6-48a6-81c9-9633feb14b95-credential-socket\") pod \"e2e-mt-20262e43-openshift-default-5bdbbd7c84-dz9pt\" (UID: \"910195fa-e9a6-48a6-81c9-9633feb14b95\") " pod="openshift-ingress/e2e-mt-20262e43-openshift-default-5bdbbd7c84-dz9pt" Jul 29 19:49:35.780167 ip-10-0-131-93 kubenswrapper[2564]: I0729 19:49:35.780104 2564 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/910195fa-e9a6-48a6-81c9-9633feb14b95-istio-envoy\") pod \"e2e-mt-20262e43-openshift-default-5bdbbd7c84-dz9pt\" (UID: \"910195fa-e9a6-48a6-81c9-9633feb14b95\") " pod="openshift-ingress/e2e-mt-20262e43-openshift-default-5bdbbd7c84-dz9pt" Jul 29 19:49:35.780167 ip-10-0-131-93 kubenswrapper[2564]: I0729 19:49:35.780118 2564 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/910195fa-e9a6-48a6-81c9-9633feb14b95-istio-data\") pod \"e2e-mt-20262e43-openshift-default-5bdbbd7c84-dz9pt\" (UID: \"910195fa-e9a6-48a6-81c9-9633feb14b95\") " pod="openshift-ingress/e2e-mt-20262e43-openshift-default-5bdbbd7c84-dz9pt" Jul 29 19:49:35.780167 ip-10-0-131-93 kubenswrapper[2564]: I0729 19:49:35.780142 2564 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/910195fa-e9a6-48a6-81c9-9633feb14b95-workload-socket\") pod \"e2e-mt-20262e43-openshift-default-5bdbbd7c84-dz9pt\" (UID: \"910195fa-e9a6-48a6-81c9-9633feb14b95\") " pod="openshift-ingress/e2e-mt-20262e43-openshift-default-5bdbbd7c84-dz9pt" Jul 29 19:49:35.845801 ip-10-0-131-93 kubenswrapper[2564]: I0729 19:49:35.845766 2564 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-444jx_faa8c546-c75b-4e10-93b4-661aa2b5f53e/konnectivity-agent/0.log" limit=1024 Jul 29 19:49:35.881306 ip-10-0-131-93 kubenswrapper[2564]: I0729 19:49:35.881271 2564 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/910195fa-e9a6-48a6-81c9-9633feb14b95-workload-socket\") pod \"e2e-mt-20262e43-openshift-default-5bdbbd7c84-dz9pt\" (UID: \"910195fa-e9a6-48a6-81c9-9633feb14b95\") " pod="openshift-ingress/e2e-mt-20262e43-openshift-default-5bdbbd7c84-dz9pt" Jul 29 19:49:35.881485 ip-10-0-131-93 kubenswrapper[2564]: I0729 19:49:35.881339 2564 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-wng9g\" (UniqueName: \"kubernetes.io/projected/910195fa-e9a6-48a6-81c9-9633feb14b95-kube-api-access-wng9g\") pod \"e2e-mt-20262e43-openshift-default-5bdbbd7c84-dz9pt\" (UID: \"910195fa-e9a6-48a6-81c9-9633feb14b95\") " pod="openshift-ingress/e2e-mt-20262e43-openshift-default-5bdbbd7c84-dz9pt" Jul 29 19:49:35.881485 ip-10-0-131-93 kubenswrapper[2564]: I0729 19:49:35.881410 2564 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/910195fa-e9a6-48a6-81c9-9633feb14b95-istio-token\") pod \"e2e-mt-20262e43-openshift-default-5bdbbd7c84-dz9pt\" (UID: \"910195fa-e9a6-48a6-81c9-9633feb14b95\") " pod="openshift-ingress/e2e-mt-20262e43-openshift-default-5bdbbd7c84-dz9pt" Jul 29 19:49:35.881485 ip-10-0-131-93 kubenswrapper[2564]: I0729 19:49:35.881457 2564 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/910195fa-e9a6-48a6-81c9-9633feb14b95-workload-certs\") pod \"e2e-mt-20262e43-openshift-default-5bdbbd7c84-dz9pt\" (UID: \"910195fa-e9a6-48a6-81c9-9633feb14b95\") " pod="openshift-ingress/e2e-mt-20262e43-openshift-default-5bdbbd7c84-dz9pt" Jul 29 19:49:35.881669 ip-10-0-131-93 kubenswrapper[2564]: I0729 19:49:35.881645 2564 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/910195fa-e9a6-48a6-81c9-9633feb14b95-istiod-ca-cert\") pod \"e2e-mt-20262e43-openshift-default-5bdbbd7c84-dz9pt\" (UID: \"910195fa-e9a6-48a6-81c9-9633feb14b95\") " pod="openshift-ingress/e2e-mt-20262e43-openshift-default-5bdbbd7c84-dz9pt" Jul 29 19:49:35.881727 ip-10-0-131-93 kubenswrapper[2564]: I0729 19:49:35.881714 2564 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/910195fa-e9a6-48a6-81c9-9633feb14b95-istio-podinfo\") pod \"e2e-mt-20262e43-openshift-default-5bdbbd7c84-dz9pt\" (UID: \"910195fa-e9a6-48a6-81c9-9633feb14b95\") " pod="openshift-ingress/e2e-mt-20262e43-openshift-default-5bdbbd7c84-dz9pt" Jul 29 19:49:35.881812 ip-10-0-131-93 kubenswrapper[2564]: I0729 19:49:35.881795 2564 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/910195fa-e9a6-48a6-81c9-9633feb14b95-credential-socket\") pod \"e2e-mt-20262e43-openshift-default-5bdbbd7c84-dz9pt\" (UID: \"910195fa-e9a6-48a6-81c9-9633feb14b95\") " pod="openshift-ingress/e2e-mt-20262e43-openshift-default-5bdbbd7c84-dz9pt" Jul 29 19:49:35.881875 ip-10-0-131-93 kubenswrapper[2564]: I0729 19:49:35.881837 2564 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/910195fa-e9a6-48a6-81c9-9633feb14b95-istio-envoy\") pod \"e2e-mt-20262e43-openshift-default-5bdbbd7c84-dz9pt\" (UID: \"910195fa-e9a6-48a6-81c9-9633feb14b95\") " pod="openshift-ingress/e2e-mt-20262e43-openshift-default-5bdbbd7c84-dz9pt" Jul 29 19:49:35.881875 ip-10-0-131-93 kubenswrapper[2564]: I0729 19:49:35.881861 2564 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/910195fa-e9a6-48a6-81c9-9633feb14b95-istio-data\") pod \"e2e-mt-20262e43-openshift-default-5bdbbd7c84-dz9pt\" (UID: \"910195fa-e9a6-48a6-81c9-9633feb14b95\") " pod="openshift-ingress/e2e-mt-20262e43-openshift-default-5bdbbd7c84-dz9pt" Jul 29 19:49:35.882113 ip-10-0-131-93 kubenswrapper[2564]: I0729 19:49:35.882091 2564 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/910195fa-e9a6-48a6-81c9-9633feb14b95-workload-socket\") pod \"e2e-mt-20262e43-openshift-default-5bdbbd7c84-dz9pt\" (UID: \"910195fa-e9a6-48a6-81c9-9633feb14b95\") " pod="openshift-ingress/e2e-mt-20262e43-openshift-default-5bdbbd7c84-dz9pt" Jul 29 19:49:35.882197 ip-10-0-131-93 kubenswrapper[2564]: I0729 19:49:35.882142 2564 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/910195fa-e9a6-48a6-81c9-9633feb14b95-workload-certs\") pod \"e2e-mt-20262e43-openshift-default-5bdbbd7c84-dz9pt\" (UID: \"910195fa-e9a6-48a6-81c9-9633feb14b95\") " pod="openshift-ingress/e2e-mt-20262e43-openshift-default-5bdbbd7c84-dz9pt" Jul 29 19:49:35.882197 ip-10-0-131-93 kubenswrapper[2564]: I0729 19:49:35.882177 2564 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/910195fa-e9a6-48a6-81c9-9633feb14b95-credential-socket\") pod \"e2e-mt-20262e43-openshift-default-5bdbbd7c84-dz9pt\" (UID: \"910195fa-e9a6-48a6-81c9-9633feb14b95\") " pod="openshift-ingress/e2e-mt-20262e43-openshift-default-5bdbbd7c84-dz9pt" Jul 29 19:49:35.882312 ip-10-0-131-93 kubenswrapper[2564]: I0729 19:49:35.882288 2564 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/910195fa-e9a6-48a6-81c9-9633feb14b95-istio-data\") pod \"e2e-mt-20262e43-openshift-default-5bdbbd7c84-dz9pt\" (UID: \"910195fa-e9a6-48a6-81c9-9633feb14b95\") " pod="openshift-ingress/e2e-mt-20262e43-openshift-default-5bdbbd7c84-dz9pt" Jul 29 19:49:35.882805 ip-10-0-131-93 kubenswrapper[2564]: I0729 19:49:35.882784 2564 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/910195fa-e9a6-48a6-81c9-9633feb14b95-istiod-ca-cert\") pod \"e2e-mt-20262e43-openshift-default-5bdbbd7c84-dz9pt\" (UID: \"910195fa-e9a6-48a6-81c9-9633feb14b95\") " pod="openshift-ingress/e2e-mt-20262e43-openshift-default-5bdbbd7c84-dz9pt" Jul 29 19:49:35.884146 ip-10-0-131-93 kubenswrapper[2564]: I0729 19:49:35.884118 2564 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/910195fa-e9a6-48a6-81c9-9633feb14b95-istio-envoy\") pod \"e2e-mt-20262e43-openshift-default-5bdbbd7c84-dz9pt\" (UID: \"910195fa-e9a6-48a6-81c9-9633feb14b95\") " pod="openshift-ingress/e2e-mt-20262e43-openshift-default-5bdbbd7c84-dz9pt" Jul 29 19:49:35.884510 ip-10-0-131-93 kubenswrapper[2564]: I0729 19:49:35.884481 2564 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/910195fa-e9a6-48a6-81c9-9633feb14b95-istio-podinfo\") pod \"e2e-mt-20262e43-openshift-default-5bdbbd7c84-dz9pt\" (UID: \"910195fa-e9a6-48a6-81c9-9633feb14b95\") " pod="openshift-ingress/e2e-mt-20262e43-openshift-default-5bdbbd7c84-dz9pt" Jul 29 19:49:35.893843 ip-10-0-131-93 kubenswrapper[2564]: I0729 19:49:35.893818 2564 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/910195fa-e9a6-48a6-81c9-9633feb14b95-istio-token\") pod \"e2e-mt-20262e43-openshift-default-5bdbbd7c84-dz9pt\" (UID: \"910195fa-e9a6-48a6-81c9-9633feb14b95\") " pod="openshift-ingress/e2e-mt-20262e43-openshift-default-5bdbbd7c84-dz9pt" Jul 29 19:49:35.893938 ip-10-0-131-93 kubenswrapper[2564]: I0729 19:49:35.893891 2564 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-wng9g\" (UniqueName: \"kubernetes.io/projected/910195fa-e9a6-48a6-81c9-9633feb14b95-kube-api-access-wng9g\") pod \"e2e-mt-20262e43-openshift-default-5bdbbd7c84-dz9pt\" (UID: \"910195fa-e9a6-48a6-81c9-9633feb14b95\") " pod="openshift-ingress/e2e-mt-20262e43-openshift-default-5bdbbd7c84-dz9pt" Jul 29 19:49:36.032031 ip-10-0-131-93 kubenswrapper[2564]: I0729 19:49:36.031950 2564 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-mt-20262e43-openshift-default-5bdbbd7c84-dz9pt" Jul 29 19:49:36.162342 ip-10-0-131-93 kubenswrapper[2564]: I0729 19:49:36.162317 2564 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/e2e-mt-20262e43-openshift-default-5bdbbd7c84-dz9pt"] Jul 29 19:49:36.163900 ip-10-0-131-93 kubenswrapper[2564]: W0729 19:49:36.163868 2564 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod910195fa_e9a6_48a6_81c9_9633feb14b95.slice/crio-52a21c591a2ea6e7bf697277b3912d47d860e630d77a82dfdd4838b21be6f682 WatchSource:0}: Error finding container 52a21c591a2ea6e7bf697277b3912d47d860e630d77a82dfdd4838b21be6f682: Status 404 returned error can't find the container with id 52a21c591a2ea6e7bf697277b3912d47d860e630d77a82dfdd4838b21be6f682 Jul 29 19:49:36.165918 ip-10-0-131-93 kubenswrapper[2564]: I0729 19:49:36.165888 2564 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236224Ki","pods":"250"} Jul 29 19:49:36.166004 ip-10-0-131-93 kubenswrapper[2564]: I0729 19:49:36.165951 2564 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236224Ki","pods":"250"} Jul 29 19:49:36.166004 ip-10-0-131-93 kubenswrapper[2564]: I0729 19:49:36.165978 2564 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236224Ki","pods":"250"} Jul 29 19:49:36.680351 ip-10-0-131-93 kubenswrapper[2564]: I0729 19:49:36.680293 2564 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-mt-20262e43-openshift-default-5bdbbd7c84-dz9pt" event={"ID":"910195fa-e9a6-48a6-81c9-9633feb14b95","Type":"ContainerStarted","Data":"760a337783f9e06522f523e4b19955f9d79beee22ea519654f2a6f8ecfe3dbd6"} Jul 29 19:49:36.680766 ip-10-0-131-93 kubenswrapper[2564]: I0729 19:49:36.680388 2564 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-mt-20262e43-openshift-default-5bdbbd7c84-dz9pt" event={"ID":"910195fa-e9a6-48a6-81c9-9633feb14b95","Type":"ContainerStarted","Data":"52a21c591a2ea6e7bf697277b3912d47d860e630d77a82dfdd4838b21be6f682"} Jul 29 19:49:36.716463 ip-10-0-131-93 kubenswrapper[2564]: I0729 19:49:36.716411 2564 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-ingress/e2e-mt-20262e43-openshift-default-5bdbbd7c84-dz9pt" podStartSLOduration=1.716393582 podStartE2EDuration="1.716393582s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-29 19:49:35 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-29 19:49:36.71464528 +0000 UTC m=+1952.438704972" watchObservedRunningTime="2026-07-29 19:49:36.716393582 +0000 UTC m=+1952.440453265" Jul 29 19:49:37.032857 ip-10-0-131-93 kubenswrapper[2564]: I0729 19:49:37.032819 2564 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="openshift-ingress/e2e-mt-20262e43-openshift-default-5bdbbd7c84-dz9pt" Jul 29 19:49:37.037734 ip-10-0-131-93 kubenswrapper[2564]: I0729 19:49:37.037710 2564 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="openshift-ingress/e2e-mt-20262e43-openshift-default-5bdbbd7c84-dz9pt" Jul 29 19:49:37.684910 ip-10-0-131-93 kubenswrapper[2564]: I0729 19:49:37.684879 2564 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/e2e-mt-20262e43-openshift-default-5bdbbd7c84-dz9pt" Jul 29 19:49:37.685756 ip-10-0-131-93 kubenswrapper[2564]: I0729 19:49:37.685736 2564 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/e2e-mt-20262e43-openshift-default-5bdbbd7c84-dz9pt" Jul 29 19:49:38.699846 ip-10-0-131-93 kubenswrapper[2564]: I0729 19:49:38.699812 2564 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-444jx_faa8c546-c75b-4e10-93b4-661aa2b5f53e/konnectivity-agent/0.log" limit=1024 Jul 29 19:49:38.800799 ip-10-0-131-93 kubenswrapper[2564]: I0729 19:49:38.800767 2564 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-444jx_faa8c546-c75b-4e10-93b4-661aa2b5f53e/konnectivity-agent/0.log" limit=1024 Jul 29 19:49:39.875645 ip-10-0-131-93 kubenswrapper[2564]: I0729 19:49:39.875611 2564 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-444jx_faa8c546-c75b-4e10-93b4-661aa2b5f53e/konnectivity-agent/0.log" limit=1024 Jul 29 19:49:40.132631 ip-10-0-131-93 kubenswrapper[2564]: I0729 19:49:40.132551 2564 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-444jx_faa8c546-c75b-4e10-93b4-661aa2b5f53e/konnectivity-agent/0.log" limit=1024 Jul 29 19:49:40.579006 ip-10-0-131-93 kubenswrapper[2564]: I0729 19:49:40.578972 2564 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-444jx_faa8c546-c75b-4e10-93b4-661aa2b5f53e/konnectivity-agent/0.log" limit=1024 Jul 29 19:49:40.753479 ip-10-0-131-93 kubenswrapper[2564]: I0729 19:49:40.753447 2564 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-444jx_faa8c546-c75b-4e10-93b4-661aa2b5f53e/konnectivity-agent/0.log" limit=1024 Jul 29 19:49:53.126913 ip-10-0-131-93 kubenswrapper[2564]: I0729 19:49:53.126875 2564 patch_prober.go:28] interesting pod/e2e-mt-20262e43-openshift-default-5bdbbd7c84-dz9pt container/istio-proxy namespace/openshift-ingress: Readiness probe status=failure output="Get \"http://10.132.0.41:15021/healthz/ready\": read tcp 10.132.0.2:56384->10.132.0.41:15021: read: connection reset by peer" start-of-body= Jul 29 19:49:53.127364 ip-10-0-131-93 kubenswrapper[2564]: I0729 19:49:53.126965 2564 prober.go:120] "Probe failed" probeType="Readiness" pod="openshift-ingress/e2e-mt-20262e43-openshift-default-5bdbbd7c84-dz9pt" podUID="910195fa-e9a6-48a6-81c9-9633feb14b95" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.132.0.41:15021/healthz/ready\": read tcp 10.132.0.2:56384->10.132.0.41:15021: read: connection reset by peer" Jul 29 19:49:53.761025 ip-10-0-131-93 kubenswrapper[2564]: I0729 19:49:53.760990 2564 generic.go:356] "Generic (PLEG): container finished" podID="910195fa-e9a6-48a6-81c9-9633feb14b95" containerID="760a337783f9e06522f523e4b19955f9d79beee22ea519654f2a6f8ecfe3dbd6" exitCode=0 Jul 29 19:49:53.761195 ip-10-0-131-93 kubenswrapper[2564]: I0729 19:49:53.761061 2564 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-mt-20262e43-openshift-default-5bdbbd7c84-dz9pt" event={"ID":"910195fa-e9a6-48a6-81c9-9633feb14b95","Type":"ContainerDied","Data":"760a337783f9e06522f523e4b19955f9d79beee22ea519654f2a6f8ecfe3dbd6"} Jul 29 19:49:53.761463 ip-10-0-131-93 kubenswrapper[2564]: I0729 19:49:53.761449 2564 scope.go:117] "RemoveContainer" containerID="760a337783f9e06522f523e4b19955f9d79beee22ea519654f2a6f8ecfe3dbd6" Jul 29 19:49:53.762494 ip-10-0-131-93 kubenswrapper[2564]: I0729 19:49:53.762467 2564 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236224Ki","pods":"250"} Jul 29 19:49:53.762561 ip-10-0-131-93 kubenswrapper[2564]: I0729 19:49:53.762540 2564 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236224Ki","pods":"250"} Jul 29 19:49:53.762603 ip-10-0-131-93 kubenswrapper[2564]: I0729 19:49:53.762573 2564 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236224Ki","pods":"250"} Jul 29 19:49:54.767285 ip-10-0-131-93 kubenswrapper[2564]: I0729 19:49:54.767253 2564 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-mt-20262e43-openshift-default-5bdbbd7c84-dz9pt" event={"ID":"910195fa-e9a6-48a6-81c9-9633feb14b95","Type":"ContainerStarted","Data":"33d535ffaa4475513e9c8914df0c88abf0e2eab2656da032102cbe4f0d188585"} Jul 29 19:49:55.032604 ip-10-0-131-93 kubenswrapper[2564]: I0729 19:49:55.032508 2564 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="openshift-ingress/e2e-mt-20262e43-openshift-default-5bdbbd7c84-dz9pt" Jul 29 19:49:55.032765 ip-10-0-131-93 kubenswrapper[2564]: I0729 19:49:55.032707 2564 patch_prober.go:28] interesting pod/e2e-mt-20262e43-openshift-default-5bdbbd7c84-dz9pt container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.132.0.41:15021/healthz/ready\": dial tcp 10.132.0.41:15021: connect: connection refused" start-of-body= Jul 29 19:49:55.032765 ip-10-0-131-93 kubenswrapper[2564]: I0729 19:49:55.032756 2564 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-mt-20262e43-openshift-default-5bdbbd7c84-dz9pt" podUID="910195fa-e9a6-48a6-81c9-9633feb14b95" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.132.0.41:15021/healthz/ready\": dial tcp 10.132.0.41:15021: connect: connection refused" Jul 29 19:49:56.033329 ip-10-0-131-93 kubenswrapper[2564]: I0729 19:49:56.033296 2564 patch_prober.go:28] interesting pod/e2e-mt-20262e43-openshift-default-5bdbbd7c84-dz9pt container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.132.0.41:15021/healthz/ready\": dial tcp 10.132.0.41:15021: connect: connection refused" start-of-body= Jul 29 19:49:56.033747 ip-10-0-131-93 kubenswrapper[2564]: I0729 19:49:56.033365 2564 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-mt-20262e43-openshift-default-5bdbbd7c84-dz9pt" podUID="910195fa-e9a6-48a6-81c9-9633feb14b95" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.132.0.41:15021/healthz/ready\": dial tcp 10.132.0.41:15021: connect: connection refused" Jul 29 19:49:57.033240 ip-10-0-131-93 kubenswrapper[2564]: I0729 19:49:57.033199 2564 patch_prober.go:28] interesting pod/e2e-mt-20262e43-openshift-default-5bdbbd7c84-dz9pt container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.132.0.41:15021/healthz/ready\": dial tcp 10.132.0.41:15021: connect: connection refused" start-of-body= Jul 29 19:49:57.033443 ip-10-0-131-93 kubenswrapper[2564]: I0729 19:49:57.033267 2564 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-mt-20262e43-openshift-default-5bdbbd7c84-dz9pt" podUID="910195fa-e9a6-48a6-81c9-9633feb14b95" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.132.0.41:15021/healthz/ready\": dial tcp 10.132.0.41:15021: connect: connection refused" Jul 29 19:49:58.033002 ip-10-0-131-93 kubenswrapper[2564]: I0729 19:49:58.032964 2564 patch_prober.go:28] interesting pod/e2e-mt-20262e43-openshift-default-5bdbbd7c84-dz9pt container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.132.0.41:15021/healthz/ready\": dial tcp 10.132.0.41:15021: connect: connection refused" start-of-body= Jul 29 19:49:58.033193 ip-10-0-131-93 kubenswrapper[2564]: I0729 19:49:58.033021 2564 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-mt-20262e43-openshift-default-5bdbbd7c84-dz9pt" podUID="910195fa-e9a6-48a6-81c9-9633feb14b95" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.132.0.41:15021/healthz/ready\": dial tcp 10.132.0.41:15021: connect: connection refused" Jul 29 19:49:59.036368 ip-10-0-131-93 kubenswrapper[2564]: I0729 19:49:59.036335 2564 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="openshift-ingress/e2e-mt-20262e43-openshift-default-5bdbbd7c84-dz9pt" Jul 29 19:49:59.036788 ip-10-0-131-93 kubenswrapper[2564]: I0729 19:49:59.036621 2564 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/e2e-mt-20262e43-openshift-default-5bdbbd7c84-dz9pt" Jul 29 19:49:59.037184 ip-10-0-131-93 kubenswrapper[2564]: I0729 19:49:59.037164 2564 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/e2e-mt-20262e43-openshift-default-5bdbbd7c84-dz9pt" Jul 29 19:50:03.610171 ip-10-0-131-93 kubenswrapper[2564]: I0729 19:50:03.610133 2564 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-444jx_faa8c546-c75b-4e10-93b4-661aa2b5f53e/konnectivity-agent/0.log" limit=1024 Jul 29 19:50:04.281502 ip-10-0-131-93 kubenswrapper[2564]: I0729 19:50:04.281467 2564 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-444jx_faa8c546-c75b-4e10-93b4-661aa2b5f53e/konnectivity-agent/0.log" limit=1024 Jul 29 19:50:04.405300 ip-10-0-131-93 kubenswrapper[2564]: I0729 19:50:04.405264 2564 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-444jx_faa8c546-c75b-4e10-93b4-661aa2b5f53e/konnectivity-agent/0.log" limit=1024 Jul 29 19:50:04.763503 ip-10-0-131-93 kubenswrapper[2564]: I0729 19:50:04.763469 2564 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-444jx_faa8c546-c75b-4e10-93b4-661aa2b5f53e/konnectivity-agent/0.log" limit=1024 Jul 29 19:50:10.748547 ip-10-0-131-93 kubenswrapper[2564]: I0729 19:50:10.748507 2564 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-444jx_faa8c546-c75b-4e10-93b4-661aa2b5f53e/konnectivity-agent/0.log" limit=1024 Jul 29 19:50:10.847521 ip-10-0-131-93 kubenswrapper[2564]: I0729 19:50:10.847481 2564 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-444jx_faa8c546-c75b-4e10-93b4-661aa2b5f53e/konnectivity-agent/0.log" limit=1024 Jul 29 19:50:21.844169 ip-10-0-131-93 kubenswrapper[2564]: I0729 19:50:21.844131 2564 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-444jx_faa8c546-c75b-4e10-93b4-661aa2b5f53e/konnectivity-agent/0.log" limit=1024 Jul 29 19:50:21.938691 ip-10-0-131-93 kubenswrapper[2564]: I0729 19:50:21.938651 2564 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-444jx_faa8c546-c75b-4e10-93b4-661aa2b5f53e/konnectivity-agent/0.log" limit=1024 Jul 29 19:50:22.041115 ip-10-0-131-93 kubenswrapper[2564]: I0729 19:50:22.041083 2564 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-444jx_faa8c546-c75b-4e10-93b4-661aa2b5f53e/konnectivity-agent/0.log" limit=1024 Jul 29 19:50:22.445492 ip-10-0-131-93 kubenswrapper[2564]: I0729 19:50:22.445459 2564 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-444jx_faa8c546-c75b-4e10-93b4-661aa2b5f53e/konnectivity-agent/0.log" limit=1024 Jul 29 19:50:28.453426 ip-10-0-131-93 kubenswrapper[2564]: I0729 19:50:28.453369 2564 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-444jx_faa8c546-c75b-4e10-93b4-661aa2b5f53e/konnectivity-agent/0.log" limit=1024 Jul 29 19:50:28.514177 ip-10-0-131-93 kubenswrapper[2564]: I0729 19:50:28.514145 2564 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-444jx_faa8c546-c75b-4e10-93b4-661aa2b5f53e/konnectivity-agent/0.log" limit=1024 Jul 29 19:50:28.708520 ip-10-0-131-93 kubenswrapper[2564]: I0729 19:50:28.708445 2564 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-ingress/e2e-mt-20262e43-openshift-default-5bdbbd7c84-dz9pt"] Jul 29 19:50:28.708678 ip-10-0-131-93 kubenswrapper[2564]: I0729 19:50:28.708658 2564 kuberuntime_container.go:873] "Killing container with a grace period" pod="openshift-ingress/e2e-mt-20262e43-openshift-default-5bdbbd7c84-dz9pt" podUID="910195fa-e9a6-48a6-81c9-9633feb14b95" containerName="istio-proxy" containerID="cri-o://33d535ffaa4475513e9c8914df0c88abf0e2eab2656da032102cbe4f0d188585" gracePeriod=30 Jul 29 19:50:28.849658 ip-10-0-131-93 kubenswrapper[2564]: I0729 19:50:28.849626 2564 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-444jx_faa8c546-c75b-4e10-93b4-661aa2b5f53e/konnectivity-agent/0.log" limit=1024 Jul 29 19:50:28.967749 ip-10-0-131-93 kubenswrapper[2564]: I0729 19:50:28.967654 2564 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-444jx_faa8c546-c75b-4e10-93b4-661aa2b5f53e/konnectivity-agent/0.log" limit=1024 Jul 29 19:50:29.039422 ip-10-0-131-93 kubenswrapper[2564]: I0729 19:50:29.039364 2564 patch_prober.go:28] interesting pod/e2e-mt-20262e43-openshift-default-5bdbbd7c84-dz9pt container/istio-proxy namespace/openshift-ingress: Readiness probe status=failure output="HTTP probe failed with statuscode: 503" start-of-body= Jul 29 19:50:29.039572 ip-10-0-131-93 kubenswrapper[2564]: I0729 19:50:29.039441 2564 prober.go:120] "Probe failed" probeType="Readiness" pod="openshift-ingress/e2e-mt-20262e43-openshift-default-5bdbbd7c84-dz9pt" podUID="910195fa-e9a6-48a6-81c9-9633feb14b95" containerName="istio-proxy" probeResult="failure" output="HTTP probe failed with statuscode: 503" Jul 29 19:50:29.088630 ip-10-0-131-93 kubenswrapper[2564]: I0729 19:50:29.088585 2564 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-444jx_faa8c546-c75b-4e10-93b4-661aa2b5f53e/konnectivity-agent/0.log" limit=1024 Jul 29 19:50:29.175191 ip-10-0-131-93 kubenswrapper[2564]: I0729 19:50:29.175147 2564 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-444jx_faa8c546-c75b-4e10-93b4-661aa2b5f53e/konnectivity-agent/0.log" limit=1024 Jul 29 19:50:29.286176 ip-10-0-131-93 kubenswrapper[2564]: I0729 19:50:29.286078 2564 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-444jx_faa8c546-c75b-4e10-93b4-661aa2b5f53e/konnectivity-agent/0.log" limit=1024 Jul 29 19:50:33.950524 ip-10-0-131-93 kubenswrapper[2564]: I0729 19:50:33.950500 2564 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-mt-20262e43-openshift-default-5bdbbd7c84-dz9pt" Jul 29 19:50:33.958612 ip-10-0-131-93 kubenswrapper[2564]: I0729 19:50:33.958587 2564 generic.go:356] "Generic (PLEG): container finished" podID="910195fa-e9a6-48a6-81c9-9633feb14b95" containerID="33d535ffaa4475513e9c8914df0c88abf0e2eab2656da032102cbe4f0d188585" exitCode=0 Jul 29 19:50:33.958732 ip-10-0-131-93 kubenswrapper[2564]: I0729 19:50:33.958628 2564 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-mt-20262e43-openshift-default-5bdbbd7c84-dz9pt" Jul 29 19:50:33.958732 ip-10-0-131-93 kubenswrapper[2564]: I0729 19:50:33.958666 2564 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-mt-20262e43-openshift-default-5bdbbd7c84-dz9pt" event={"ID":"910195fa-e9a6-48a6-81c9-9633feb14b95","Type":"ContainerDied","Data":"33d535ffaa4475513e9c8914df0c88abf0e2eab2656da032102cbe4f0d188585"} Jul 29 19:50:33.958732 ip-10-0-131-93 kubenswrapper[2564]: I0729 19:50:33.958719 2564 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-mt-20262e43-openshift-default-5bdbbd7c84-dz9pt" event={"ID":"910195fa-e9a6-48a6-81c9-9633feb14b95","Type":"ContainerDied","Data":"52a21c591a2ea6e7bf697277b3912d47d860e630d77a82dfdd4838b21be6f682"} Jul 29 19:50:33.958846 ip-10-0-131-93 kubenswrapper[2564]: I0729 19:50:33.958744 2564 scope.go:117] "RemoveContainer" containerID="33d535ffaa4475513e9c8914df0c88abf0e2eab2656da032102cbe4f0d188585" Jul 29 19:50:33.969419 ip-10-0-131-93 kubenswrapper[2564]: I0729 19:50:33.969403 2564 scope.go:117] "RemoveContainer" containerID="760a337783f9e06522f523e4b19955f9d79beee22ea519654f2a6f8ecfe3dbd6" Jul 29 19:50:33.979019 ip-10-0-131-93 kubenswrapper[2564]: I0729 19:50:33.978995 2564 scope.go:117] "RemoveContainer" containerID="33d535ffaa4475513e9c8914df0c88abf0e2eab2656da032102cbe4f0d188585" Jul 29 19:50:33.979291 ip-10-0-131-93 kubenswrapper[2564]: E0729 19:50:33.979268 2564 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"33d535ffaa4475513e9c8914df0c88abf0e2eab2656da032102cbe4f0d188585\": container with ID starting with 33d535ffaa4475513e9c8914df0c88abf0e2eab2656da032102cbe4f0d188585 not found: ID does not exist" containerID="33d535ffaa4475513e9c8914df0c88abf0e2eab2656da032102cbe4f0d188585" Jul 29 19:50:33.979363 ip-10-0-131-93 kubenswrapper[2564]: I0729 19:50:33.979300 2564 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"33d535ffaa4475513e9c8914df0c88abf0e2eab2656da032102cbe4f0d188585"} err="failed to get container status \"33d535ffaa4475513e9c8914df0c88abf0e2eab2656da032102cbe4f0d188585\": rpc error: code = NotFound desc = could not find container \"33d535ffaa4475513e9c8914df0c88abf0e2eab2656da032102cbe4f0d188585\": container with ID starting with 33d535ffaa4475513e9c8914df0c88abf0e2eab2656da032102cbe4f0d188585 not found: ID does not exist" Jul 29 19:50:33.979363 ip-10-0-131-93 kubenswrapper[2564]: I0729 19:50:33.979319 2564 scope.go:117] "RemoveContainer" containerID="760a337783f9e06522f523e4b19955f9d79beee22ea519654f2a6f8ecfe3dbd6" Jul 29 19:50:33.979621 ip-10-0-131-93 kubenswrapper[2564]: E0729 19:50:33.979602 2564 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"760a337783f9e06522f523e4b19955f9d79beee22ea519654f2a6f8ecfe3dbd6\": container with ID starting with 760a337783f9e06522f523e4b19955f9d79beee22ea519654f2a6f8ecfe3dbd6 not found: ID does not exist" containerID="760a337783f9e06522f523e4b19955f9d79beee22ea519654f2a6f8ecfe3dbd6" Jul 29 19:50:33.979679 ip-10-0-131-93 kubenswrapper[2564]: I0729 19:50:33.979629 2564 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"760a337783f9e06522f523e4b19955f9d79beee22ea519654f2a6f8ecfe3dbd6"} err="failed to get container status \"760a337783f9e06522f523e4b19955f9d79beee22ea519654f2a6f8ecfe3dbd6\": rpc error: code = NotFound desc = could not find container \"760a337783f9e06522f523e4b19955f9d79beee22ea519654f2a6f8ecfe3dbd6\": container with ID starting with 760a337783f9e06522f523e4b19955f9d79beee22ea519654f2a6f8ecfe3dbd6 not found: ID does not exist" Jul 29 19:50:34.075242 ip-10-0-131-93 kubenswrapper[2564]: I0729 19:50:34.075208 2564 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/910195fa-e9a6-48a6-81c9-9633feb14b95-istio-data\") pod \"910195fa-e9a6-48a6-81c9-9633feb14b95\" (UID: \"910195fa-e9a6-48a6-81c9-9633feb14b95\") " Jul 29 19:50:34.075434 ip-10-0-131-93 kubenswrapper[2564]: I0729 19:50:34.075247 2564 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/910195fa-e9a6-48a6-81c9-9633feb14b95-istio-podinfo\") pod \"910195fa-e9a6-48a6-81c9-9633feb14b95\" (UID: \"910195fa-e9a6-48a6-81c9-9633feb14b95\") " Jul 29 19:50:34.075434 ip-10-0-131-93 kubenswrapper[2564]: I0729 19:50:34.075282 2564 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/910195fa-e9a6-48a6-81c9-9633feb14b95-istio-token\") pod \"910195fa-e9a6-48a6-81c9-9633feb14b95\" (UID: \"910195fa-e9a6-48a6-81c9-9633feb14b95\") " Jul 29 19:50:34.075434 ip-10-0-131-93 kubenswrapper[2564]: I0729 19:50:34.075303 2564 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/910195fa-e9a6-48a6-81c9-9633feb14b95-workload-socket\") pod \"910195fa-e9a6-48a6-81c9-9633feb14b95\" (UID: \"910195fa-e9a6-48a6-81c9-9633feb14b95\") " Jul 29 19:50:34.075434 ip-10-0-131-93 kubenswrapper[2564]: I0729 19:50:34.075322 2564 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/910195fa-e9a6-48a6-81c9-9633feb14b95-istiod-ca-cert\") pod \"910195fa-e9a6-48a6-81c9-9633feb14b95\" (UID: \"910195fa-e9a6-48a6-81c9-9633feb14b95\") " Jul 29 19:50:34.075434 ip-10-0-131-93 kubenswrapper[2564]: I0729 19:50:34.075357 2564 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/910195fa-e9a6-48a6-81c9-9633feb14b95-credential-socket\") pod \"910195fa-e9a6-48a6-81c9-9633feb14b95\" (UID: \"910195fa-e9a6-48a6-81c9-9633feb14b95\") " Jul 29 19:50:34.075434 ip-10-0-131-93 kubenswrapper[2564]: I0729 19:50:34.075413 2564 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-wng9g\" (UniqueName: \"kubernetes.io/projected/910195fa-e9a6-48a6-81c9-9633feb14b95-kube-api-access-wng9g\") pod \"910195fa-e9a6-48a6-81c9-9633feb14b95\" (UID: \"910195fa-e9a6-48a6-81c9-9633feb14b95\") " Jul 29 19:50:34.075747 ip-10-0-131-93 kubenswrapper[2564]: I0729 19:50:34.075566 2564 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/910195fa-e9a6-48a6-81c9-9633feb14b95-istio-envoy\") pod \"910195fa-e9a6-48a6-81c9-9633feb14b95\" (UID: \"910195fa-e9a6-48a6-81c9-9633feb14b95\") " Jul 29 19:50:34.075747 ip-10-0-131-93 kubenswrapper[2564]: I0729 19:50:34.075596 2564 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/910195fa-e9a6-48a6-81c9-9633feb14b95-workload-certs\") pod \"910195fa-e9a6-48a6-81c9-9633feb14b95\" (UID: \"910195fa-e9a6-48a6-81c9-9633feb14b95\") " Jul 29 19:50:34.075747 ip-10-0-131-93 kubenswrapper[2564]: I0729 19:50:34.075737 2564 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/910195fa-e9a6-48a6-81c9-9633feb14b95-workload-socket" (OuterVolumeSpecName: "workload-socket") pod "910195fa-e9a6-48a6-81c9-9633feb14b95" (UID: "910195fa-e9a6-48a6-81c9-9633feb14b95"). InnerVolumeSpecName "workload-socket". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Jul 29 19:50:34.075976 ip-10-0-131-93 kubenswrapper[2564]: I0729 19:50:34.075762 2564 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/910195fa-e9a6-48a6-81c9-9633feb14b95-credential-socket" (OuterVolumeSpecName: "credential-socket") pod "910195fa-e9a6-48a6-81c9-9633feb14b95" (UID: "910195fa-e9a6-48a6-81c9-9633feb14b95"). InnerVolumeSpecName "credential-socket". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Jul 29 19:50:34.075976 ip-10-0-131-93 kubenswrapper[2564]: I0729 19:50:34.075926 2564 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/910195fa-e9a6-48a6-81c9-9633feb14b95-workload-certs" (OuterVolumeSpecName: "workload-certs") pod "910195fa-e9a6-48a6-81c9-9633feb14b95" (UID: "910195fa-e9a6-48a6-81c9-9633feb14b95"). InnerVolumeSpecName "workload-certs". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Jul 29 19:50:34.075976 ip-10-0-131-93 kubenswrapper[2564]: I0729 19:50:34.075960 2564 reconciler_common.go:299] "Volume detached for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/910195fa-e9a6-48a6-81c9-9633feb14b95-workload-socket\") on node \"ip-10-0-131-93.ec2.internal\" DevicePath \"\"" Jul 29 19:50:34.076132 ip-10-0-131-93 kubenswrapper[2564]: I0729 19:50:34.075977 2564 reconciler_common.go:299] "Volume detached for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/910195fa-e9a6-48a6-81c9-9633feb14b95-credential-socket\") on node \"ip-10-0-131-93.ec2.internal\" DevicePath \"\"" Jul 29 19:50:34.076132 ip-10-0-131-93 kubenswrapper[2564]: I0729 19:50:34.076062 2564 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/910195fa-e9a6-48a6-81c9-9633feb14b95-istiod-ca-cert" (OuterVolumeSpecName: "istiod-ca-cert") pod "910195fa-e9a6-48a6-81c9-9633feb14b95" (UID: "910195fa-e9a6-48a6-81c9-9633feb14b95"). InnerVolumeSpecName "istiod-ca-cert". PluginName "kubernetes.io/configmap", VolumeGIDValue "" Jul 29 19:50:34.076400 ip-10-0-131-93 kubenswrapper[2564]: I0729 19:50:34.076353 2564 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/910195fa-e9a6-48a6-81c9-9633feb14b95-istio-data" (OuterVolumeSpecName: "istio-data") pod "910195fa-e9a6-48a6-81c9-9633feb14b95" (UID: "910195fa-e9a6-48a6-81c9-9633feb14b95"). InnerVolumeSpecName "istio-data". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Jul 29 19:50:34.077561 ip-10-0-131-93 kubenswrapper[2564]: I0729 19:50:34.077535 2564 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/downward-api/910195fa-e9a6-48a6-81c9-9633feb14b95-istio-podinfo" (OuterVolumeSpecName: "istio-podinfo") pod "910195fa-e9a6-48a6-81c9-9633feb14b95" (UID: "910195fa-e9a6-48a6-81c9-9633feb14b95"). InnerVolumeSpecName "istio-podinfo". PluginName "kubernetes.io/downward-api", VolumeGIDValue "" Jul 29 19:50:34.077664 ip-10-0-131-93 kubenswrapper[2564]: I0729 19:50:34.077586 2564 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/910195fa-e9a6-48a6-81c9-9633feb14b95-istio-token" (OuterVolumeSpecName: "istio-token") pod "910195fa-e9a6-48a6-81c9-9633feb14b95" (UID: "910195fa-e9a6-48a6-81c9-9633feb14b95"). InnerVolumeSpecName "istio-token". PluginName "kubernetes.io/projected", VolumeGIDValue "" Jul 29 19:50:34.077712 ip-10-0-131-93 kubenswrapper[2564]: I0729 19:50:34.077686 2564 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/910195fa-e9a6-48a6-81c9-9633feb14b95-kube-api-access-wng9g" (OuterVolumeSpecName: "kube-api-access-wng9g") pod "910195fa-e9a6-48a6-81c9-9633feb14b95" (UID: "910195fa-e9a6-48a6-81c9-9633feb14b95"). InnerVolumeSpecName "kube-api-access-wng9g". PluginName "kubernetes.io/projected", VolumeGIDValue "" Jul 29 19:50:34.077779 ip-10-0-131-93 kubenswrapper[2564]: I0729 19:50:34.077760 2564 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/910195fa-e9a6-48a6-81c9-9633feb14b95-istio-envoy" (OuterVolumeSpecName: "istio-envoy") pod "910195fa-e9a6-48a6-81c9-9633feb14b95" (UID: "910195fa-e9a6-48a6-81c9-9633feb14b95"). InnerVolumeSpecName "istio-envoy". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Jul 29 19:50:34.177332 ip-10-0-131-93 kubenswrapper[2564]: I0729 19:50:34.177299 2564 reconciler_common.go:299] "Volume detached for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/910195fa-e9a6-48a6-81c9-9633feb14b95-istiod-ca-cert\") on node \"ip-10-0-131-93.ec2.internal\" DevicePath \"\"" Jul 29 19:50:34.177332 ip-10-0-131-93 kubenswrapper[2564]: I0729 19:50:34.177324 2564 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-wng9g\" (UniqueName: \"kubernetes.io/projected/910195fa-e9a6-48a6-81c9-9633feb14b95-kube-api-access-wng9g\") on node \"ip-10-0-131-93.ec2.internal\" DevicePath \"\"" Jul 29 19:50:34.177332 ip-10-0-131-93 kubenswrapper[2564]: I0729 19:50:34.177333 2564 reconciler_common.go:299] "Volume detached for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/910195fa-e9a6-48a6-81c9-9633feb14b95-istio-envoy\") on node \"ip-10-0-131-93.ec2.internal\" DevicePath \"\"" Jul 29 19:50:34.177332 ip-10-0-131-93 kubenswrapper[2564]: I0729 19:50:34.177341 2564 reconciler_common.go:299] "Volume detached for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/910195fa-e9a6-48a6-81c9-9633feb14b95-workload-certs\") on node \"ip-10-0-131-93.ec2.internal\" DevicePath \"\"" Jul 29 19:50:34.177592 ip-10-0-131-93 kubenswrapper[2564]: I0729 19:50:34.177349 2564 reconciler_common.go:299] "Volume detached for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/910195fa-e9a6-48a6-81c9-9633feb14b95-istio-data\") on node \"ip-10-0-131-93.ec2.internal\" DevicePath \"\"" Jul 29 19:50:34.177592 ip-10-0-131-93 kubenswrapper[2564]: I0729 19:50:34.177357 2564 reconciler_common.go:299] "Volume detached for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/910195fa-e9a6-48a6-81c9-9633feb14b95-istio-podinfo\") on node \"ip-10-0-131-93.ec2.internal\" DevicePath \"\"" Jul 29 19:50:34.177592 ip-10-0-131-93 kubenswrapper[2564]: I0729 19:50:34.177364 2564 reconciler_common.go:299] "Volume detached for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/910195fa-e9a6-48a6-81c9-9633feb14b95-istio-token\") on node \"ip-10-0-131-93.ec2.internal\" DevicePath \"\"" Jul 29 19:50:34.281517 ip-10-0-131-93 kubenswrapper[2564]: I0729 19:50:34.281434 2564 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-ingress/e2e-mt-20262e43-openshift-default-5bdbbd7c84-dz9pt"] Jul 29 19:50:34.283610 ip-10-0-131-93 kubenswrapper[2564]: I0729 19:50:34.283586 2564 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["openshift-ingress/e2e-mt-20262e43-openshift-default-5bdbbd7c84-dz9pt"] Jul 29 19:50:34.782994 ip-10-0-131-93 kubenswrapper[2564]: I0729 19:50:34.782956 2564 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="910195fa-e9a6-48a6-81c9-9633feb14b95" path="/var/lib/kubelet/pods/910195fa-e9a6-48a6-81c9-9633feb14b95/volumes" Jul 29 19:50:35.042812 ip-10-0-131-93 kubenswrapper[2564]: I0729 19:50:35.042715 2564 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-444jx_faa8c546-c75b-4e10-93b4-661aa2b5f53e/konnectivity-agent/0.log" limit=1024 Jul 29 19:50:35.151925 ip-10-0-131-93 kubenswrapper[2564]: I0729 19:50:35.151894 2564 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-444jx_faa8c546-c75b-4e10-93b4-661aa2b5f53e/konnectivity-agent/0.log" limit=1024 Jul 29 19:50:35.246169 ip-10-0-131-93 kubenswrapper[2564]: I0729 19:50:35.246137 2564 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-444jx_faa8c546-c75b-4e10-93b4-661aa2b5f53e/konnectivity-agent/0.log" limit=1024 Jul 29 19:50:35.374507 ip-10-0-131-93 kubenswrapper[2564]: I0729 19:50:35.374414 2564 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-444jx_faa8c546-c75b-4e10-93b4-661aa2b5f53e/konnectivity-agent/0.log" limit=1024 Jul 29 19:50:41.651491 ip-10-0-131-93 kubenswrapper[2564]: I0729 19:50:41.651461 2564 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-444jx_faa8c546-c75b-4e10-93b4-661aa2b5f53e/konnectivity-agent/0.log" limit=1024 Jul 29 19:50:41.718784 ip-10-0-131-93 kubenswrapper[2564]: I0729 19:50:41.718751 2564 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-444jx_faa8c546-c75b-4e10-93b4-661aa2b5f53e/konnectivity-agent/0.log" limit=1024 Jul 29 19:50:48.447810 ip-10-0-131-93 kubenswrapper[2564]: I0729 19:50:48.447775 2564 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-444jx_faa8c546-c75b-4e10-93b4-661aa2b5f53e/konnectivity-agent/0.log" limit=1024 Jul 29 19:50:48.749179 ip-10-0-131-93 kubenswrapper[2564]: I0729 19:50:48.749101 2564 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-ingress/e2e-mt-2d31776a-openshift-default-68599858f6-hvln8"] Jul 29 19:50:48.749459 ip-10-0-131-93 kubenswrapper[2564]: I0729 19:50:48.749445 2564 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="910195fa-e9a6-48a6-81c9-9633feb14b95" containerName="istio-proxy" Jul 29 19:50:48.749525 ip-10-0-131-93 kubenswrapper[2564]: I0729 19:50:48.749460 2564 state_mem.go:107] "Deleted CPUSet assignment" podUID="910195fa-e9a6-48a6-81c9-9633feb14b95" containerName="istio-proxy" Jul 29 19:50:48.749525 ip-10-0-131-93 kubenswrapper[2564]: I0729 19:50:48.749471 2564 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="ba30c537-a242-4d8c-a15a-a21d57983742" containerName="istio-proxy" Jul 29 19:50:48.749525 ip-10-0-131-93 kubenswrapper[2564]: I0729 19:50:48.749476 2564 state_mem.go:107] "Deleted CPUSet assignment" podUID="ba30c537-a242-4d8c-a15a-a21d57983742" containerName="istio-proxy" Jul 29 19:50:48.749630 ip-10-0-131-93 kubenswrapper[2564]: I0729 19:50:48.749534 2564 memory_manager.go:368] "RemoveStaleState removing state" podUID="910195fa-e9a6-48a6-81c9-9633feb14b95" containerName="istio-proxy" Jul 29 19:50:48.749630 ip-10-0-131-93 kubenswrapper[2564]: I0729 19:50:48.749542 2564 memory_manager.go:368] "RemoveStaleState removing state" podUID="4e980d2a-9b00-4791-a563-78cb03cf6697" containerName="istio-proxy" Jul 29 19:50:48.749630 ip-10-0-131-93 kubenswrapper[2564]: I0729 19:50:48.749549 2564 memory_manager.go:368] "RemoveStaleState removing state" podUID="910195fa-e9a6-48a6-81c9-9633feb14b95" containerName="istio-proxy" Jul 29 19:50:48.751643 ip-10-0-131-93 kubenswrapper[2564]: I0729 19:50:48.751623 2564 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-mt-2d31776a-openshift-default-68599858f6-hvln8" Jul 29 19:50:48.754019 ip-10-0-131-93 kubenswrapper[2564]: I0729 19:50:48.753999 2564 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress\"/\"e2e-mt-2d31776a-openshift-default-dockercfg-zqp7x\"" Jul 29 19:50:48.768234 ip-10-0-131-93 kubenswrapper[2564]: I0729 19:50:48.768213 2564 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/e2e-mt-2d31776a-openshift-default-68599858f6-hvln8"] Jul 29 19:50:48.788532 ip-10-0-131-93 kubenswrapper[2564]: I0729 19:50:48.788505 2564 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-444jx_faa8c546-c75b-4e10-93b4-661aa2b5f53e/konnectivity-agent/0.log" limit=1024 Jul 29 19:50:48.895395 ip-10-0-131-93 kubenswrapper[2564]: I0729 19:50:48.895337 2564 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/54900853-16b3-4e52-b5a3-a6a054dc32ba-istio-envoy\") pod \"e2e-mt-2d31776a-openshift-default-68599858f6-hvln8\" (UID: \"54900853-16b3-4e52-b5a3-a6a054dc32ba\") " pod="openshift-ingress/e2e-mt-2d31776a-openshift-default-68599858f6-hvln8" Jul 29 19:50:48.895575 ip-10-0-131-93 kubenswrapper[2564]: I0729 19:50:48.895401 2564 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/54900853-16b3-4e52-b5a3-a6a054dc32ba-istiod-ca-cert\") pod \"e2e-mt-2d31776a-openshift-default-68599858f6-hvln8\" (UID: \"54900853-16b3-4e52-b5a3-a6a054dc32ba\") " pod="openshift-ingress/e2e-mt-2d31776a-openshift-default-68599858f6-hvln8" Jul 29 19:50:48.895575 ip-10-0-131-93 kubenswrapper[2564]: I0729 19:50:48.895479 2564 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/54900853-16b3-4e52-b5a3-a6a054dc32ba-istio-token\") pod \"e2e-mt-2d31776a-openshift-default-68599858f6-hvln8\" (UID: \"54900853-16b3-4e52-b5a3-a6a054dc32ba\") " pod="openshift-ingress/e2e-mt-2d31776a-openshift-default-68599858f6-hvln8" Jul 29 19:50:48.895575 ip-10-0-131-93 kubenswrapper[2564]: I0729 19:50:48.895527 2564 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/54900853-16b3-4e52-b5a3-a6a054dc32ba-istio-data\") pod \"e2e-mt-2d31776a-openshift-default-68599858f6-hvln8\" (UID: \"54900853-16b3-4e52-b5a3-a6a054dc32ba\") " pod="openshift-ingress/e2e-mt-2d31776a-openshift-default-68599858f6-hvln8" Jul 29 19:50:48.895575 ip-10-0-131-93 kubenswrapper[2564]: I0729 19:50:48.895551 2564 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-vxwkh\" (UniqueName: \"kubernetes.io/projected/54900853-16b3-4e52-b5a3-a6a054dc32ba-kube-api-access-vxwkh\") pod \"e2e-mt-2d31776a-openshift-default-68599858f6-hvln8\" (UID: \"54900853-16b3-4e52-b5a3-a6a054dc32ba\") " pod="openshift-ingress/e2e-mt-2d31776a-openshift-default-68599858f6-hvln8" Jul 29 19:50:48.895716 ip-10-0-131-93 kubenswrapper[2564]: I0729 19:50:48.895579 2564 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/54900853-16b3-4e52-b5a3-a6a054dc32ba-workload-certs\") pod \"e2e-mt-2d31776a-openshift-default-68599858f6-hvln8\" (UID: \"54900853-16b3-4e52-b5a3-a6a054dc32ba\") " pod="openshift-ingress/e2e-mt-2d31776a-openshift-default-68599858f6-hvln8" Jul 29 19:50:48.895716 ip-10-0-131-93 kubenswrapper[2564]: I0729 19:50:48.895602 2564 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/54900853-16b3-4e52-b5a3-a6a054dc32ba-credential-socket\") pod \"e2e-mt-2d31776a-openshift-default-68599858f6-hvln8\" (UID: \"54900853-16b3-4e52-b5a3-a6a054dc32ba\") " pod="openshift-ingress/e2e-mt-2d31776a-openshift-default-68599858f6-hvln8" Jul 29 19:50:48.895716 ip-10-0-131-93 kubenswrapper[2564]: I0729 19:50:48.895626 2564 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/54900853-16b3-4e52-b5a3-a6a054dc32ba-istio-podinfo\") pod \"e2e-mt-2d31776a-openshift-default-68599858f6-hvln8\" (UID: \"54900853-16b3-4e52-b5a3-a6a054dc32ba\") " pod="openshift-ingress/e2e-mt-2d31776a-openshift-default-68599858f6-hvln8" Jul 29 19:50:48.895716 ip-10-0-131-93 kubenswrapper[2564]: I0729 19:50:48.895644 2564 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/54900853-16b3-4e52-b5a3-a6a054dc32ba-workload-socket\") pod \"e2e-mt-2d31776a-openshift-default-68599858f6-hvln8\" (UID: \"54900853-16b3-4e52-b5a3-a6a054dc32ba\") " pod="openshift-ingress/e2e-mt-2d31776a-openshift-default-68599858f6-hvln8" Jul 29 19:50:48.996358 ip-10-0-131-93 kubenswrapper[2564]: I0729 19:50:48.996327 2564 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/54900853-16b3-4e52-b5a3-a6a054dc32ba-istio-data\") pod \"e2e-mt-2d31776a-openshift-default-68599858f6-hvln8\" (UID: \"54900853-16b3-4e52-b5a3-a6a054dc32ba\") " pod="openshift-ingress/e2e-mt-2d31776a-openshift-default-68599858f6-hvln8" Jul 29 19:50:48.996550 ip-10-0-131-93 kubenswrapper[2564]: I0729 19:50:48.996367 2564 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-vxwkh\" (UniqueName: \"kubernetes.io/projected/54900853-16b3-4e52-b5a3-a6a054dc32ba-kube-api-access-vxwkh\") pod \"e2e-mt-2d31776a-openshift-default-68599858f6-hvln8\" (UID: \"54900853-16b3-4e52-b5a3-a6a054dc32ba\") " pod="openshift-ingress/e2e-mt-2d31776a-openshift-default-68599858f6-hvln8" Jul 29 19:50:48.996550 ip-10-0-131-93 kubenswrapper[2564]: I0729 19:50:48.996429 2564 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/54900853-16b3-4e52-b5a3-a6a054dc32ba-workload-certs\") pod \"e2e-mt-2d31776a-openshift-default-68599858f6-hvln8\" (UID: \"54900853-16b3-4e52-b5a3-a6a054dc32ba\") " pod="openshift-ingress/e2e-mt-2d31776a-openshift-default-68599858f6-hvln8" Jul 29 19:50:48.996550 ip-10-0-131-93 kubenswrapper[2564]: I0729 19:50:48.996454 2564 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/54900853-16b3-4e52-b5a3-a6a054dc32ba-credential-socket\") pod \"e2e-mt-2d31776a-openshift-default-68599858f6-hvln8\" (UID: \"54900853-16b3-4e52-b5a3-a6a054dc32ba\") " pod="openshift-ingress/e2e-mt-2d31776a-openshift-default-68599858f6-hvln8" Jul 29 19:50:48.996718 ip-10-0-131-93 kubenswrapper[2564]: I0729 19:50:48.996588 2564 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/54900853-16b3-4e52-b5a3-a6a054dc32ba-istio-podinfo\") pod \"e2e-mt-2d31776a-openshift-default-68599858f6-hvln8\" (UID: \"54900853-16b3-4e52-b5a3-a6a054dc32ba\") " pod="openshift-ingress/e2e-mt-2d31776a-openshift-default-68599858f6-hvln8" Jul 29 19:50:48.996718 ip-10-0-131-93 kubenswrapper[2564]: I0729 19:50:48.996623 2564 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/54900853-16b3-4e52-b5a3-a6a054dc32ba-workload-socket\") pod \"e2e-mt-2d31776a-openshift-default-68599858f6-hvln8\" (UID: \"54900853-16b3-4e52-b5a3-a6a054dc32ba\") " pod="openshift-ingress/e2e-mt-2d31776a-openshift-default-68599858f6-hvln8" Jul 29 19:50:48.996718 ip-10-0-131-93 kubenswrapper[2564]: I0729 19:50:48.996674 2564 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/54900853-16b3-4e52-b5a3-a6a054dc32ba-istio-envoy\") pod \"e2e-mt-2d31776a-openshift-default-68599858f6-hvln8\" (UID: \"54900853-16b3-4e52-b5a3-a6a054dc32ba\") " pod="openshift-ingress/e2e-mt-2d31776a-openshift-default-68599858f6-hvln8" Jul 29 19:50:48.996718 ip-10-0-131-93 kubenswrapper[2564]: I0729 19:50:48.996699 2564 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/54900853-16b3-4e52-b5a3-a6a054dc32ba-istiod-ca-cert\") pod \"e2e-mt-2d31776a-openshift-default-68599858f6-hvln8\" (UID: \"54900853-16b3-4e52-b5a3-a6a054dc32ba\") " pod="openshift-ingress/e2e-mt-2d31776a-openshift-default-68599858f6-hvln8" Jul 29 19:50:48.996919 ip-10-0-131-93 kubenswrapper[2564]: I0729 19:50:48.996753 2564 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/54900853-16b3-4e52-b5a3-a6a054dc32ba-istio-token\") pod \"e2e-mt-2d31776a-openshift-default-68599858f6-hvln8\" (UID: \"54900853-16b3-4e52-b5a3-a6a054dc32ba\") " pod="openshift-ingress/e2e-mt-2d31776a-openshift-default-68599858f6-hvln8" Jul 29 19:50:48.996919 ip-10-0-131-93 kubenswrapper[2564]: I0729 19:50:48.996751 2564 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/54900853-16b3-4e52-b5a3-a6a054dc32ba-istio-data\") pod \"e2e-mt-2d31776a-openshift-default-68599858f6-hvln8\" (UID: \"54900853-16b3-4e52-b5a3-a6a054dc32ba\") " pod="openshift-ingress/e2e-mt-2d31776a-openshift-default-68599858f6-hvln8" Jul 29 19:50:48.996919 ip-10-0-131-93 kubenswrapper[2564]: I0729 19:50:48.996768 2564 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/54900853-16b3-4e52-b5a3-a6a054dc32ba-credential-socket\") pod \"e2e-mt-2d31776a-openshift-default-68599858f6-hvln8\" (UID: \"54900853-16b3-4e52-b5a3-a6a054dc32ba\") " pod="openshift-ingress/e2e-mt-2d31776a-openshift-default-68599858f6-hvln8" Jul 29 19:50:48.996919 ip-10-0-131-93 kubenswrapper[2564]: I0729 19:50:48.996816 2564 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/54900853-16b3-4e52-b5a3-a6a054dc32ba-workload-certs\") pod \"e2e-mt-2d31776a-openshift-default-68599858f6-hvln8\" (UID: \"54900853-16b3-4e52-b5a3-a6a054dc32ba\") " pod="openshift-ingress/e2e-mt-2d31776a-openshift-default-68599858f6-hvln8" Jul 29 19:50:48.997137 ip-10-0-131-93 kubenswrapper[2564]: I0729 19:50:48.997054 2564 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/54900853-16b3-4e52-b5a3-a6a054dc32ba-workload-socket\") pod \"e2e-mt-2d31776a-openshift-default-68599858f6-hvln8\" (UID: \"54900853-16b3-4e52-b5a3-a6a054dc32ba\") " pod="openshift-ingress/e2e-mt-2d31776a-openshift-default-68599858f6-hvln8" Jul 29 19:50:48.997430 ip-10-0-131-93 kubenswrapper[2564]: I0729 19:50:48.997408 2564 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/54900853-16b3-4e52-b5a3-a6a054dc32ba-istiod-ca-cert\") pod \"e2e-mt-2d31776a-openshift-default-68599858f6-hvln8\" (UID: \"54900853-16b3-4e52-b5a3-a6a054dc32ba\") " pod="openshift-ingress/e2e-mt-2d31776a-openshift-default-68599858f6-hvln8" Jul 29 19:50:48.998882 ip-10-0-131-93 kubenswrapper[2564]: I0729 19:50:48.998865 2564 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/54900853-16b3-4e52-b5a3-a6a054dc32ba-istio-envoy\") pod \"e2e-mt-2d31776a-openshift-default-68599858f6-hvln8\" (UID: \"54900853-16b3-4e52-b5a3-a6a054dc32ba\") " pod="openshift-ingress/e2e-mt-2d31776a-openshift-default-68599858f6-hvln8" Jul 29 19:50:48.999241 ip-10-0-131-93 kubenswrapper[2564]: I0729 19:50:48.999202 2564 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/54900853-16b3-4e52-b5a3-a6a054dc32ba-istio-podinfo\") pod \"e2e-mt-2d31776a-openshift-default-68599858f6-hvln8\" (UID: \"54900853-16b3-4e52-b5a3-a6a054dc32ba\") " pod="openshift-ingress/e2e-mt-2d31776a-openshift-default-68599858f6-hvln8" Jul 29 19:50:49.011136 ip-10-0-131-93 kubenswrapper[2564]: I0729 19:50:49.011111 2564 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/54900853-16b3-4e52-b5a3-a6a054dc32ba-istio-token\") pod \"e2e-mt-2d31776a-openshift-default-68599858f6-hvln8\" (UID: \"54900853-16b3-4e52-b5a3-a6a054dc32ba\") " pod="openshift-ingress/e2e-mt-2d31776a-openshift-default-68599858f6-hvln8" Jul 29 19:50:49.015618 ip-10-0-131-93 kubenswrapper[2564]: I0729 19:50:49.015598 2564 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-vxwkh\" (UniqueName: \"kubernetes.io/projected/54900853-16b3-4e52-b5a3-a6a054dc32ba-kube-api-access-vxwkh\") pod \"e2e-mt-2d31776a-openshift-default-68599858f6-hvln8\" (UID: \"54900853-16b3-4e52-b5a3-a6a054dc32ba\") " pod="openshift-ingress/e2e-mt-2d31776a-openshift-default-68599858f6-hvln8" Jul 29 19:50:49.061681 ip-10-0-131-93 kubenswrapper[2564]: I0729 19:50:49.061656 2564 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-mt-2d31776a-openshift-default-68599858f6-hvln8" Jul 29 19:50:49.103156 ip-10-0-131-93 kubenswrapper[2564]: I0729 19:50:49.103126 2564 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-444jx_faa8c546-c75b-4e10-93b4-661aa2b5f53e/konnectivity-agent/0.log" limit=1024 Jul 29 19:50:49.204206 ip-10-0-131-93 kubenswrapper[2564]: I0729 19:50:49.204182 2564 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/e2e-mt-2d31776a-openshift-default-68599858f6-hvln8"] Jul 29 19:50:49.206160 ip-10-0-131-93 kubenswrapper[2564]: W0729 19:50:49.206131 2564 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod54900853_16b3_4e52_b5a3_a6a054dc32ba.slice/crio-664df740a7e53f031fae6b972fcac7741e639dd505ac5125b04853aa5107a49d WatchSource:0}: Error finding container 664df740a7e53f031fae6b972fcac7741e639dd505ac5125b04853aa5107a49d: Status 404 returned error can't find the container with id 664df740a7e53f031fae6b972fcac7741e639dd505ac5125b04853aa5107a49d Jul 29 19:50:49.208310 ip-10-0-131-93 kubenswrapper[2564]: I0729 19:50:49.208268 2564 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236224Ki","pods":"250"} Jul 29 19:50:49.208432 ip-10-0-131-93 kubenswrapper[2564]: I0729 19:50:49.208365 2564 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236224Ki","pods":"250"} Jul 29 19:50:49.208489 ip-10-0-131-93 kubenswrapper[2564]: I0729 19:50:49.208453 2564 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236224Ki","pods":"250"} Jul 29 19:50:50.039072 ip-10-0-131-93 kubenswrapper[2564]: I0729 19:50:50.039036 2564 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-mt-2d31776a-openshift-default-68599858f6-hvln8" event={"ID":"54900853-16b3-4e52-b5a3-a6a054dc32ba","Type":"ContainerStarted","Data":"d38aa98c0f5e12aa1e85bd87edea8e0ca1f9914edcdfe53206384327e8874edf"} Jul 29 19:50:50.039469 ip-10-0-131-93 kubenswrapper[2564]: I0729 19:50:50.039089 2564 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-mt-2d31776a-openshift-default-68599858f6-hvln8" event={"ID":"54900853-16b3-4e52-b5a3-a6a054dc32ba","Type":"ContainerStarted","Data":"664df740a7e53f031fae6b972fcac7741e639dd505ac5125b04853aa5107a49d"} Jul 29 19:50:50.059525 ip-10-0-131-93 kubenswrapper[2564]: I0729 19:50:50.059482 2564 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-ingress/e2e-mt-2d31776a-openshift-default-68599858f6-hvln8" podStartSLOduration=2.059466091 podStartE2EDuration="2.059466091s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-29 19:50:48 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-29 19:50:50.058292258 +0000 UTC m=+2025.782351950" watchObservedRunningTime="2026-07-29 19:50:50.059466091 +0000 UTC m=+2025.783525783" Jul 29 19:50:50.062473 ip-10-0-131-93 kubenswrapper[2564]: I0729 19:50:50.062433 2564 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="openshift-ingress/e2e-mt-2d31776a-openshift-default-68599858f6-hvln8" Jul 29 19:50:50.067405 ip-10-0-131-93 kubenswrapper[2564]: I0729 19:50:50.067359 2564 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="openshift-ingress/e2e-mt-2d31776a-openshift-default-68599858f6-hvln8" Jul 29 19:50:51.044081 ip-10-0-131-93 kubenswrapper[2564]: I0729 19:50:51.044046 2564 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/e2e-mt-2d31776a-openshift-default-68599858f6-hvln8" Jul 29 19:50:51.045027 ip-10-0-131-93 kubenswrapper[2564]: I0729 19:50:51.045007 2564 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/e2e-mt-2d31776a-openshift-default-68599858f6-hvln8" Jul 29 19:50:52.039408 ip-10-0-131-93 kubenswrapper[2564]: I0729 19:50:52.039357 2564 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-444jx_faa8c546-c75b-4e10-93b4-661aa2b5f53e/konnectivity-agent/0.log" limit=1024 Jul 29 19:50:54.215351 ip-10-0-131-93 kubenswrapper[2564]: I0729 19:50:54.215316 2564 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-444jx_faa8c546-c75b-4e10-93b4-661aa2b5f53e/konnectivity-agent/0.log" limit=1024 Jul 29 19:51:05.127021 ip-10-0-131-93 kubenswrapper[2564]: I0729 19:51:05.126987 2564 generic.go:356] "Generic (PLEG): container finished" podID="54900853-16b3-4e52-b5a3-a6a054dc32ba" containerID="d38aa98c0f5e12aa1e85bd87edea8e0ca1f9914edcdfe53206384327e8874edf" exitCode=0 Jul 29 19:51:05.127466 ip-10-0-131-93 kubenswrapper[2564]: I0729 19:51:05.127066 2564 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-mt-2d31776a-openshift-default-68599858f6-hvln8" event={"ID":"54900853-16b3-4e52-b5a3-a6a054dc32ba","Type":"ContainerDied","Data":"d38aa98c0f5e12aa1e85bd87edea8e0ca1f9914edcdfe53206384327e8874edf"} Jul 29 19:51:05.127536 ip-10-0-131-93 kubenswrapper[2564]: I0729 19:51:05.127518 2564 scope.go:117] "RemoveContainer" containerID="d38aa98c0f5e12aa1e85bd87edea8e0ca1f9914edcdfe53206384327e8874edf" Jul 29 19:51:05.128658 ip-10-0-131-93 kubenswrapper[2564]: I0729 19:51:05.128628 2564 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236224Ki","pods":"250"} Jul 29 19:51:05.128773 ip-10-0-131-93 kubenswrapper[2564]: I0729 19:51:05.128695 2564 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236224Ki","pods":"250"} Jul 29 19:51:05.128773 ip-10-0-131-93 kubenswrapper[2564]: I0729 19:51:05.128728 2564 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236224Ki","pods":"250"} Jul 29 19:51:06.043989 ip-10-0-131-93 kubenswrapper[2564]: I0729 19:51:06.043960 2564 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/e2e-mt-2d31776a-openshift-default-68599858f6-hvln8" Jul 29 19:51:06.133849 ip-10-0-131-93 kubenswrapper[2564]: I0729 19:51:06.133810 2564 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-mt-2d31776a-openshift-default-68599858f6-hvln8" event={"ID":"54900853-16b3-4e52-b5a3-a6a054dc32ba","Type":"ContainerStarted","Data":"28cd92de9ea0d9a04c4d226cc7c89c085555bc57d321658dc9615a5109d7d62d"} Jul 29 19:51:07.062566 ip-10-0-131-93 kubenswrapper[2564]: I0729 19:51:07.062521 2564 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="openshift-ingress/e2e-mt-2d31776a-openshift-default-68599858f6-hvln8" Jul 29 19:51:07.063741 ip-10-0-131-93 kubenswrapper[2564]: I0729 19:51:07.063718 2564 patch_prober.go:28] interesting pod/e2e-mt-2d31776a-openshift-default-68599858f6-hvln8 container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.132.0.42:15021/healthz/ready\": dial tcp 10.132.0.42:15021: connect: connection refused" start-of-body= Jul 29 19:51:07.063838 ip-10-0-131-93 kubenswrapper[2564]: I0729 19:51:07.063767 2564 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-mt-2d31776a-openshift-default-68599858f6-hvln8" podUID="54900853-16b3-4e52-b5a3-a6a054dc32ba" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.132.0.42:15021/healthz/ready\": dial tcp 10.132.0.42:15021: connect: connection refused" Jul 29 19:51:08.062341 ip-10-0-131-93 kubenswrapper[2564]: I0729 19:51:08.062303 2564 patch_prober.go:28] interesting pod/e2e-mt-2d31776a-openshift-default-68599858f6-hvln8 container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.132.0.42:15021/healthz/ready\": dial tcp 10.132.0.42:15021: connect: connection refused" start-of-body= Jul 29 19:51:08.062742 ip-10-0-131-93 kubenswrapper[2564]: I0729 19:51:08.062398 2564 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-mt-2d31776a-openshift-default-68599858f6-hvln8" podUID="54900853-16b3-4e52-b5a3-a6a054dc32ba" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.132.0.42:15021/healthz/ready\": dial tcp 10.132.0.42:15021: connect: connection refused" Jul 29 19:51:09.062567 ip-10-0-131-93 kubenswrapper[2564]: I0729 19:51:09.062532 2564 patch_prober.go:28] interesting pod/e2e-mt-2d31776a-openshift-default-68599858f6-hvln8 container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.132.0.42:15021/healthz/ready\": dial tcp 10.132.0.42:15021: connect: connection refused" start-of-body= Jul 29 19:51:09.062945 ip-10-0-131-93 kubenswrapper[2564]: I0729 19:51:09.062594 2564 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-mt-2d31776a-openshift-default-68599858f6-hvln8" podUID="54900853-16b3-4e52-b5a3-a6a054dc32ba" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.132.0.42:15021/healthz/ready\": dial tcp 10.132.0.42:15021: connect: connection refused" Jul 29 19:51:10.062949 ip-10-0-131-93 kubenswrapper[2564]: I0729 19:51:10.062913 2564 patch_prober.go:28] interesting pod/e2e-mt-2d31776a-openshift-default-68599858f6-hvln8 container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.132.0.42:15021/healthz/ready\": dial tcp 10.132.0.42:15021: connect: connection refused" start-of-body= Jul 29 19:51:10.063338 ip-10-0-131-93 kubenswrapper[2564]: I0729 19:51:10.062979 2564 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-mt-2d31776a-openshift-default-68599858f6-hvln8" podUID="54900853-16b3-4e52-b5a3-a6a054dc32ba" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.132.0.42:15021/healthz/ready\": dial tcp 10.132.0.42:15021: connect: connection refused" Jul 29 19:51:11.066393 ip-10-0-131-93 kubenswrapper[2564]: I0729 19:51:11.066347 2564 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="openshift-ingress/e2e-mt-2d31776a-openshift-default-68599858f6-hvln8" Jul 29 19:51:11.066824 ip-10-0-131-93 kubenswrapper[2564]: I0729 19:51:11.066812 2564 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/e2e-mt-2d31776a-openshift-default-68599858f6-hvln8" Jul 29 19:51:11.067578 ip-10-0-131-93 kubenswrapper[2564]: I0729 19:51:11.067555 2564 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/e2e-mt-2d31776a-openshift-default-68599858f6-hvln8" Jul 29 19:51:16.888270 ip-10-0-131-93 kubenswrapper[2564]: I0729 19:51:16.888231 2564 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-444jx_faa8c546-c75b-4e10-93b4-661aa2b5f53e/konnectivity-agent/0.log" limit=1024 Jul 29 19:51:21.968806 ip-10-0-131-93 kubenswrapper[2564]: I0729 19:51:21.968771 2564 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-444jx_faa8c546-c75b-4e10-93b4-661aa2b5f53e/konnectivity-agent/0.log" limit=1024 Jul 29 19:51:46.867463 ip-10-0-131-93 kubenswrapper[2564]: I0729 19:51:46.867423 2564 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-444jx_faa8c546-c75b-4e10-93b4-661aa2b5f53e/konnectivity-agent/0.log" limit=1024 Jul 29 19:51:56.500942 ip-10-0-131-93 kubenswrapper[2564]: I0729 19:51:56.500905 2564 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-444jx_faa8c546-c75b-4e10-93b4-661aa2b5f53e/konnectivity-agent/0.log" limit=1024 Jul 29 19:51:56.572686 ip-10-0-131-93 kubenswrapper[2564]: I0729 19:51:56.572655 2564 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-444jx_faa8c546-c75b-4e10-93b4-661aa2b5f53e/konnectivity-agent/0.log" limit=1024 Jul 29 19:51:56.673362 ip-10-0-131-93 kubenswrapper[2564]: I0729 19:51:56.673332 2564 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-444jx_faa8c546-c75b-4e10-93b4-661aa2b5f53e/konnectivity-agent/0.log" limit=1024 Jul 29 19:51:56.738272 ip-10-0-131-93 kubenswrapper[2564]: I0729 19:51:56.738240 2564 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-444jx_faa8c546-c75b-4e10-93b4-661aa2b5f53e/konnectivity-agent/0.log" limit=1024 Jul 29 19:51:59.957390 ip-10-0-131-93 kubenswrapper[2564]: I0729 19:51:59.957335 2564 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-444jx_faa8c546-c75b-4e10-93b4-661aa2b5f53e/konnectivity-agent/0.log" limit=1024 Jul 29 19:52:04.983409 ip-10-0-131-93 kubenswrapper[2564]: I0729 19:52:04.983358 2564 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-444jx_faa8c546-c75b-4e10-93b4-661aa2b5f53e/konnectivity-agent/0.log" limit=1024 Jul 29 19:52:41.278876 ip-10-0-131-93 kubenswrapper[2564]: I0729 19:52:41.278841 2564 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-444jx_faa8c546-c75b-4e10-93b4-661aa2b5f53e/konnectivity-agent/0.log" limit=1024 Jul 29 19:52:42.102843 ip-10-0-131-93 kubenswrapper[2564]: I0729 19:52:42.102807 2564 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-444jx_faa8c546-c75b-4e10-93b4-661aa2b5f53e/konnectivity-agent/0.log" limit=1024 Jul 29 19:52:47.848800 ip-10-0-131-93 kubenswrapper[2564]: I0729 19:52:47.848769 2564 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-444jx_faa8c546-c75b-4e10-93b4-661aa2b5f53e/konnectivity-agent/0.log" limit=1024 Jul 29 19:52:52.929108 ip-10-0-131-93 kubenswrapper[2564]: I0729 19:52:52.929076 2564 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-444jx_faa8c546-c75b-4e10-93b4-661aa2b5f53e/konnectivity-agent/0.log" limit=1024 Jul 29 19:53:02.189511 ip-10-0-131-93 kubenswrapper[2564]: I0729 19:53:02.189431 2564 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-444jx_faa8c546-c75b-4e10-93b4-661aa2b5f53e/konnectivity-agent/0.log" limit=1024 Jul 29 19:53:02.269807 ip-10-0-131-93 kubenswrapper[2564]: I0729 19:53:02.269773 2564 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-444jx_faa8c546-c75b-4e10-93b4-661aa2b5f53e/konnectivity-agent/0.log" limit=1024 Jul 29 19:53:02.440402 ip-10-0-131-93 kubenswrapper[2564]: I0729 19:53:02.440288 2564 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-ingress/e2e-mt-2d31776a-openshift-default-68599858f6-hvln8"] Jul 29 19:53:02.440596 ip-10-0-131-93 kubenswrapper[2564]: I0729 19:53:02.440570 2564 kuberuntime_container.go:873] "Killing container with a grace period" pod="openshift-ingress/e2e-mt-2d31776a-openshift-default-68599858f6-hvln8" podUID="54900853-16b3-4e52-b5a3-a6a054dc32ba" containerName="istio-proxy" containerID="cri-o://28cd92de9ea0d9a04c4d226cc7c89c085555bc57d321658dc9615a5109d7d62d" gracePeriod=30 Jul 29 19:53:02.465993 ip-10-0-131-93 kubenswrapper[2564]: I0729 19:53:02.465967 2564 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-444jx_faa8c546-c75b-4e10-93b4-661aa2b5f53e/konnectivity-agent/0.log" limit=1024 Jul 29 19:53:02.685840 ip-10-0-131-93 kubenswrapper[2564]: I0729 19:53:02.685807 2564 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-444jx_faa8c546-c75b-4e10-93b4-661aa2b5f53e/konnectivity-agent/0.log" limit=1024 Jul 29 19:53:03.721859 ip-10-0-131-93 kubenswrapper[2564]: I0729 19:53:03.721823 2564 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-444jx_faa8c546-c75b-4e10-93b4-661aa2b5f53e/konnectivity-agent/0.log" limit=1024 Jul 29 19:53:03.829950 ip-10-0-131-93 kubenswrapper[2564]: I0729 19:53:03.829919 2564 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-444jx_faa8c546-c75b-4e10-93b4-661aa2b5f53e/konnectivity-agent/0.log" limit=1024 Jul 29 19:53:03.920959 ip-10-0-131-93 kubenswrapper[2564]: I0729 19:53:03.920921 2564 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-444jx_faa8c546-c75b-4e10-93b4-661aa2b5f53e/konnectivity-agent/0.log" limit=1024 Jul 29 19:53:04.657842 ip-10-0-131-93 kubenswrapper[2564]: I0729 19:53:04.657809 2564 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-444jx_faa8c546-c75b-4e10-93b4-661aa2b5f53e/konnectivity-agent/0.log" limit=1024 Jul 29 19:53:04.753239 ip-10-0-131-93 kubenswrapper[2564]: I0729 19:53:04.753179 2564 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-444jx_faa8c546-c75b-4e10-93b4-661aa2b5f53e/konnectivity-agent/0.log" limit=1024 Jul 29 19:53:04.855938 ip-10-0-131-93 kubenswrapper[2564]: I0729 19:53:04.855907 2564 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-444jx_faa8c546-c75b-4e10-93b4-661aa2b5f53e/konnectivity-agent/0.log" limit=1024 Jul 29 19:53:06.191696 ip-10-0-131-93 kubenswrapper[2564]: I0729 19:53:06.191663 2564 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-444jx_faa8c546-c75b-4e10-93b4-661aa2b5f53e/konnectivity-agent/0.log" limit=1024 Jul 29 19:53:07.696594 ip-10-0-131-93 kubenswrapper[2564]: I0729 19:53:07.696570 2564 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-mt-2d31776a-openshift-default-68599858f6-hvln8" Jul 29 19:53:07.714072 ip-10-0-131-93 kubenswrapper[2564]: I0729 19:53:07.713999 2564 generic.go:356] "Generic (PLEG): container finished" podID="54900853-16b3-4e52-b5a3-a6a054dc32ba" containerID="28cd92de9ea0d9a04c4d226cc7c89c085555bc57d321658dc9615a5109d7d62d" exitCode=0 Jul 29 19:53:07.714072 ip-10-0-131-93 kubenswrapper[2564]: I0729 19:53:07.714043 2564 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-mt-2d31776a-openshift-default-68599858f6-hvln8" Jul 29 19:53:07.714282 ip-10-0-131-93 kubenswrapper[2564]: I0729 19:53:07.714073 2564 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-mt-2d31776a-openshift-default-68599858f6-hvln8" event={"ID":"54900853-16b3-4e52-b5a3-a6a054dc32ba","Type":"ContainerDied","Data":"28cd92de9ea0d9a04c4d226cc7c89c085555bc57d321658dc9615a5109d7d62d"} Jul 29 19:53:07.714282 ip-10-0-131-93 kubenswrapper[2564]: I0729 19:53:07.714122 2564 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-mt-2d31776a-openshift-default-68599858f6-hvln8" event={"ID":"54900853-16b3-4e52-b5a3-a6a054dc32ba","Type":"ContainerDied","Data":"664df740a7e53f031fae6b972fcac7741e639dd505ac5125b04853aa5107a49d"} Jul 29 19:53:07.714282 ip-10-0-131-93 kubenswrapper[2564]: I0729 19:53:07.714145 2564 scope.go:117] "RemoveContainer" containerID="28cd92de9ea0d9a04c4d226cc7c89c085555bc57d321658dc9615a5109d7d62d" Jul 29 19:53:07.722516 ip-10-0-131-93 kubenswrapper[2564]: I0729 19:53:07.722497 2564 scope.go:117] "RemoveContainer" containerID="d38aa98c0f5e12aa1e85bd87edea8e0ca1f9914edcdfe53206384327e8874edf" Jul 29 19:53:07.730497 ip-10-0-131-93 kubenswrapper[2564]: I0729 19:53:07.730365 2564 scope.go:117] "RemoveContainer" containerID="28cd92de9ea0d9a04c4d226cc7c89c085555bc57d321658dc9615a5109d7d62d" Jul 29 19:53:07.730811 ip-10-0-131-93 kubenswrapper[2564]: E0729 19:53:07.730783 2564 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"28cd92de9ea0d9a04c4d226cc7c89c085555bc57d321658dc9615a5109d7d62d\": container with ID starting with 28cd92de9ea0d9a04c4d226cc7c89c085555bc57d321658dc9615a5109d7d62d not found: ID does not exist" containerID="28cd92de9ea0d9a04c4d226cc7c89c085555bc57d321658dc9615a5109d7d62d" Jul 29 19:53:07.730960 ip-10-0-131-93 kubenswrapper[2564]: I0729 19:53:07.730821 2564 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"28cd92de9ea0d9a04c4d226cc7c89c085555bc57d321658dc9615a5109d7d62d"} err="failed to get container status \"28cd92de9ea0d9a04c4d226cc7c89c085555bc57d321658dc9615a5109d7d62d\": rpc error: code = NotFound desc = could not find container \"28cd92de9ea0d9a04c4d226cc7c89c085555bc57d321658dc9615a5109d7d62d\": container with ID starting with 28cd92de9ea0d9a04c4d226cc7c89c085555bc57d321658dc9615a5109d7d62d not found: ID does not exist" Jul 29 19:53:07.730960 ip-10-0-131-93 kubenswrapper[2564]: I0729 19:53:07.730847 2564 scope.go:117] "RemoveContainer" containerID="d38aa98c0f5e12aa1e85bd87edea8e0ca1f9914edcdfe53206384327e8874edf" Jul 29 19:53:07.731223 ip-10-0-131-93 kubenswrapper[2564]: E0729 19:53:07.731197 2564 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"d38aa98c0f5e12aa1e85bd87edea8e0ca1f9914edcdfe53206384327e8874edf\": container with ID starting with d38aa98c0f5e12aa1e85bd87edea8e0ca1f9914edcdfe53206384327e8874edf not found: ID does not exist" containerID="d38aa98c0f5e12aa1e85bd87edea8e0ca1f9914edcdfe53206384327e8874edf" Jul 29 19:53:07.731308 ip-10-0-131-93 kubenswrapper[2564]: I0729 19:53:07.731231 2564 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"d38aa98c0f5e12aa1e85bd87edea8e0ca1f9914edcdfe53206384327e8874edf"} err="failed to get container status \"d38aa98c0f5e12aa1e85bd87edea8e0ca1f9914edcdfe53206384327e8874edf\": rpc error: code = NotFound desc = could not find container \"d38aa98c0f5e12aa1e85bd87edea8e0ca1f9914edcdfe53206384327e8874edf\": container with ID starting with d38aa98c0f5e12aa1e85bd87edea8e0ca1f9914edcdfe53206384327e8874edf not found: ID does not exist" Jul 29 19:53:07.757843 ip-10-0-131-93 kubenswrapper[2564]: I0729 19:53:07.757823 2564 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/54900853-16b3-4e52-b5a3-a6a054dc32ba-istiod-ca-cert\") pod \"54900853-16b3-4e52-b5a3-a6a054dc32ba\" (UID: \"54900853-16b3-4e52-b5a3-a6a054dc32ba\") " Jul 29 19:53:07.757952 ip-10-0-131-93 kubenswrapper[2564]: I0729 19:53:07.757856 2564 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/54900853-16b3-4e52-b5a3-a6a054dc32ba-credential-socket\") pod \"54900853-16b3-4e52-b5a3-a6a054dc32ba\" (UID: \"54900853-16b3-4e52-b5a3-a6a054dc32ba\") " Jul 29 19:53:07.757952 ip-10-0-131-93 kubenswrapper[2564]: I0729 19:53:07.757875 2564 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/54900853-16b3-4e52-b5a3-a6a054dc32ba-workload-socket\") pod \"54900853-16b3-4e52-b5a3-a6a054dc32ba\" (UID: \"54900853-16b3-4e52-b5a3-a6a054dc32ba\") " Jul 29 19:53:07.757952 ip-10-0-131-93 kubenswrapper[2564]: I0729 19:53:07.757902 2564 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/54900853-16b3-4e52-b5a3-a6a054dc32ba-istio-envoy\") pod \"54900853-16b3-4e52-b5a3-a6a054dc32ba\" (UID: \"54900853-16b3-4e52-b5a3-a6a054dc32ba\") " Jul 29 19:53:07.757952 ip-10-0-131-93 kubenswrapper[2564]: I0729 19:53:07.757935 2564 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/54900853-16b3-4e52-b5a3-a6a054dc32ba-istio-token\") pod \"54900853-16b3-4e52-b5a3-a6a054dc32ba\" (UID: \"54900853-16b3-4e52-b5a3-a6a054dc32ba\") " Jul 29 19:53:07.758103 ip-10-0-131-93 kubenswrapper[2564]: I0729 19:53:07.757998 2564 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/54900853-16b3-4e52-b5a3-a6a054dc32ba-istio-data\") pod \"54900853-16b3-4e52-b5a3-a6a054dc32ba\" (UID: \"54900853-16b3-4e52-b5a3-a6a054dc32ba\") " Jul 29 19:53:07.758103 ip-10-0-131-93 kubenswrapper[2564]: I0729 19:53:07.758035 2564 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/54900853-16b3-4e52-b5a3-a6a054dc32ba-workload-certs\") pod \"54900853-16b3-4e52-b5a3-a6a054dc32ba\" (UID: \"54900853-16b3-4e52-b5a3-a6a054dc32ba\") " Jul 29 19:53:07.758103 ip-10-0-131-93 kubenswrapper[2564]: I0729 19:53:07.758059 2564 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/54900853-16b3-4e52-b5a3-a6a054dc32ba-istio-podinfo\") pod \"54900853-16b3-4e52-b5a3-a6a054dc32ba\" (UID: \"54900853-16b3-4e52-b5a3-a6a054dc32ba\") " Jul 29 19:53:07.758103 ip-10-0-131-93 kubenswrapper[2564]: I0729 19:53:07.758085 2564 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-vxwkh\" (UniqueName: \"kubernetes.io/projected/54900853-16b3-4e52-b5a3-a6a054dc32ba-kube-api-access-vxwkh\") pod \"54900853-16b3-4e52-b5a3-a6a054dc32ba\" (UID: \"54900853-16b3-4e52-b5a3-a6a054dc32ba\") " Jul 29 19:53:07.758306 ip-10-0-131-93 kubenswrapper[2564]: I0729 19:53:07.758111 2564 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/54900853-16b3-4e52-b5a3-a6a054dc32ba-credential-socket" (OuterVolumeSpecName: "credential-socket") pod "54900853-16b3-4e52-b5a3-a6a054dc32ba" (UID: "54900853-16b3-4e52-b5a3-a6a054dc32ba"). InnerVolumeSpecName "credential-socket". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Jul 29 19:53:07.758306 ip-10-0-131-93 kubenswrapper[2564]: I0729 19:53:07.758183 2564 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/54900853-16b3-4e52-b5a3-a6a054dc32ba-workload-socket" (OuterVolumeSpecName: "workload-socket") pod "54900853-16b3-4e52-b5a3-a6a054dc32ba" (UID: "54900853-16b3-4e52-b5a3-a6a054dc32ba"). InnerVolumeSpecName "workload-socket". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Jul 29 19:53:07.758441 ip-10-0-131-93 kubenswrapper[2564]: I0729 19:53:07.758306 2564 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/54900853-16b3-4e52-b5a3-a6a054dc32ba-workload-certs" (OuterVolumeSpecName: "workload-certs") pod "54900853-16b3-4e52-b5a3-a6a054dc32ba" (UID: "54900853-16b3-4e52-b5a3-a6a054dc32ba"). InnerVolumeSpecName "workload-certs". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Jul 29 19:53:07.758441 ip-10-0-131-93 kubenswrapper[2564]: I0729 19:53:07.758256 2564 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/54900853-16b3-4e52-b5a3-a6a054dc32ba-istiod-ca-cert" (OuterVolumeSpecName: "istiod-ca-cert") pod "54900853-16b3-4e52-b5a3-a6a054dc32ba" (UID: "54900853-16b3-4e52-b5a3-a6a054dc32ba"). InnerVolumeSpecName "istiod-ca-cert". PluginName "kubernetes.io/configmap", VolumeGIDValue "" Jul 29 19:53:07.758441 ip-10-0-131-93 kubenswrapper[2564]: I0729 19:53:07.758429 2564 reconciler_common.go:299] "Volume detached for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/54900853-16b3-4e52-b5a3-a6a054dc32ba-credential-socket\") on node \"ip-10-0-131-93.ec2.internal\" DevicePath \"\"" Jul 29 19:53:07.758595 ip-10-0-131-93 kubenswrapper[2564]: I0729 19:53:07.758446 2564 reconciler_common.go:299] "Volume detached for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/54900853-16b3-4e52-b5a3-a6a054dc32ba-workload-socket\") on node \"ip-10-0-131-93.ec2.internal\" DevicePath \"\"" Jul 29 19:53:07.758595 ip-10-0-131-93 kubenswrapper[2564]: I0729 19:53:07.758468 2564 reconciler_common.go:299] "Volume detached for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/54900853-16b3-4e52-b5a3-a6a054dc32ba-workload-certs\") on node \"ip-10-0-131-93.ec2.internal\" DevicePath \"\"" Jul 29 19:53:07.758990 ip-10-0-131-93 kubenswrapper[2564]: I0729 19:53:07.758967 2564 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/54900853-16b3-4e52-b5a3-a6a054dc32ba-istio-data" (OuterVolumeSpecName: "istio-data") pod "54900853-16b3-4e52-b5a3-a6a054dc32ba" (UID: "54900853-16b3-4e52-b5a3-a6a054dc32ba"). InnerVolumeSpecName "istio-data". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Jul 29 19:53:07.760341 ip-10-0-131-93 kubenswrapper[2564]: I0729 19:53:07.760320 2564 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/54900853-16b3-4e52-b5a3-a6a054dc32ba-kube-api-access-vxwkh" (OuterVolumeSpecName: "kube-api-access-vxwkh") pod "54900853-16b3-4e52-b5a3-a6a054dc32ba" (UID: "54900853-16b3-4e52-b5a3-a6a054dc32ba"). InnerVolumeSpecName "kube-api-access-vxwkh". PluginName "kubernetes.io/projected", VolumeGIDValue "" Jul 29 19:53:07.760341 ip-10-0-131-93 kubenswrapper[2564]: I0729 19:53:07.760327 2564 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/downward-api/54900853-16b3-4e52-b5a3-a6a054dc32ba-istio-podinfo" (OuterVolumeSpecName: "istio-podinfo") pod "54900853-16b3-4e52-b5a3-a6a054dc32ba" (UID: "54900853-16b3-4e52-b5a3-a6a054dc32ba"). InnerVolumeSpecName "istio-podinfo". PluginName "kubernetes.io/downward-api", VolumeGIDValue "" Jul 29 19:53:07.760514 ip-10-0-131-93 kubenswrapper[2564]: I0729 19:53:07.760495 2564 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/54900853-16b3-4e52-b5a3-a6a054dc32ba-istio-token" (OuterVolumeSpecName: "istio-token") pod "54900853-16b3-4e52-b5a3-a6a054dc32ba" (UID: "54900853-16b3-4e52-b5a3-a6a054dc32ba"). InnerVolumeSpecName "istio-token". PluginName "kubernetes.io/projected", VolumeGIDValue "" Jul 29 19:53:07.760584 ip-10-0-131-93 kubenswrapper[2564]: I0729 19:53:07.760563 2564 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/54900853-16b3-4e52-b5a3-a6a054dc32ba-istio-envoy" (OuterVolumeSpecName: "istio-envoy") pod "54900853-16b3-4e52-b5a3-a6a054dc32ba" (UID: "54900853-16b3-4e52-b5a3-a6a054dc32ba"). InnerVolumeSpecName "istio-envoy". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Jul 29 19:53:07.859631 ip-10-0-131-93 kubenswrapper[2564]: I0729 19:53:07.859599 2564 reconciler_common.go:299] "Volume detached for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/54900853-16b3-4e52-b5a3-a6a054dc32ba-istio-data\") on node \"ip-10-0-131-93.ec2.internal\" DevicePath \"\"" Jul 29 19:53:07.859631 ip-10-0-131-93 kubenswrapper[2564]: I0729 19:53:07.859633 2564 reconciler_common.go:299] "Volume detached for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/54900853-16b3-4e52-b5a3-a6a054dc32ba-istio-podinfo\") on node \"ip-10-0-131-93.ec2.internal\" DevicePath \"\"" Jul 29 19:53:07.859805 ip-10-0-131-93 kubenswrapper[2564]: I0729 19:53:07.859643 2564 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-vxwkh\" (UniqueName: \"kubernetes.io/projected/54900853-16b3-4e52-b5a3-a6a054dc32ba-kube-api-access-vxwkh\") on node \"ip-10-0-131-93.ec2.internal\" DevicePath \"\"" Jul 29 19:53:07.859805 ip-10-0-131-93 kubenswrapper[2564]: I0729 19:53:07.859652 2564 reconciler_common.go:299] "Volume detached for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/54900853-16b3-4e52-b5a3-a6a054dc32ba-istiod-ca-cert\") on node \"ip-10-0-131-93.ec2.internal\" DevicePath \"\"" Jul 29 19:53:07.859805 ip-10-0-131-93 kubenswrapper[2564]: I0729 19:53:07.859660 2564 reconciler_common.go:299] "Volume detached for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/54900853-16b3-4e52-b5a3-a6a054dc32ba-istio-envoy\") on node \"ip-10-0-131-93.ec2.internal\" DevicePath \"\"" Jul 29 19:53:07.859805 ip-10-0-131-93 kubenswrapper[2564]: I0729 19:53:07.859667 2564 reconciler_common.go:299] "Volume detached for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/54900853-16b3-4e52-b5a3-a6a054dc32ba-istio-token\") on node \"ip-10-0-131-93.ec2.internal\" DevicePath \"\"" Jul 29 19:53:08.037986 ip-10-0-131-93 kubenswrapper[2564]: I0729 19:53:08.037952 2564 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-ingress/e2e-mt-2d31776a-openshift-default-68599858f6-hvln8"] Jul 29 19:53:08.042183 ip-10-0-131-93 kubenswrapper[2564]: I0729 19:53:08.042161 2564 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["openshift-ingress/e2e-mt-2d31776a-openshift-default-68599858f6-hvln8"] Jul 29 19:53:08.782180 ip-10-0-131-93 kubenswrapper[2564]: I0729 19:53:08.782134 2564 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="54900853-16b3-4e52-b5a3-a6a054dc32ba" path="/var/lib/kubelet/pods/54900853-16b3-4e52-b5a3-a6a054dc32ba/volumes" Jul 29 19:53:12.229696 ip-10-0-131-93 kubenswrapper[2564]: I0729 19:53:12.229660 2564 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-444jx_faa8c546-c75b-4e10-93b4-661aa2b5f53e/konnectivity-agent/0.log" limit=1024 Jul 29 19:53:12.330550 ip-10-0-131-93 kubenswrapper[2564]: I0729 19:53:12.330519 2564 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-444jx_faa8c546-c75b-4e10-93b4-661aa2b5f53e/konnectivity-agent/0.log" limit=1024 Jul 29 19:53:12.899628 ip-10-0-131-93 kubenswrapper[2564]: I0729 19:53:12.899586 2564 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-444jx_faa8c546-c75b-4e10-93b4-661aa2b5f53e/konnectivity-agent/0.log" limit=1024 Jul 29 19:53:19.532143 ip-10-0-131-93 kubenswrapper[2564]: I0729 19:53:19.532106 2564 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-444jx_faa8c546-c75b-4e10-93b4-661aa2b5f53e/konnectivity-agent/0.log" limit=1024 Jul 29 19:53:19.632397 ip-10-0-131-93 kubenswrapper[2564]: I0729 19:53:19.632344 2564 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-444jx_faa8c546-c75b-4e10-93b4-661aa2b5f53e/konnectivity-agent/0.log" limit=1024 Jul 29 19:53:30.015403 ip-10-0-131-93 kubenswrapper[2564]: I0729 19:53:30.015352 2564 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-444jx_faa8c546-c75b-4e10-93b4-661aa2b5f53e/konnectivity-agent/0.log" limit=1024 Jul 29 19:53:30.325628 ip-10-0-131-93 kubenswrapper[2564]: I0729 19:53:30.325520 2564 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-444jx_faa8c546-c75b-4e10-93b4-661aa2b5f53e/konnectivity-agent/0.log" limit=1024 Jul 29 19:53:51.252912 ip-10-0-131-93 kubenswrapper[2564]: I0729 19:53:51.252880 2564 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-444jx_faa8c546-c75b-4e10-93b4-661aa2b5f53e/konnectivity-agent/0.log" limit=1024 Jul 29 19:53:57.650220 ip-10-0-131-93 kubenswrapper[2564]: I0729 19:53:57.650187 2564 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-444jx_faa8c546-c75b-4e10-93b4-661aa2b5f53e/konnectivity-agent/0.log" limit=1024 Jul 29 19:53:57.748145 ip-10-0-131-93 kubenswrapper[2564]: I0729 19:53:57.748112 2564 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-444jx_faa8c546-c75b-4e10-93b4-661aa2b5f53e/konnectivity-agent/0.log" limit=1024 Jul 29 19:53:58.001826 ip-10-0-131-93 kubenswrapper[2564]: I0729 19:53:58.001796 2564 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-444jx_faa8c546-c75b-4e10-93b4-661aa2b5f53e/konnectivity-agent/0.log" limit=1024 Jul 29 19:53:58.293659 ip-10-0-131-93 kubenswrapper[2564]: I0729 19:53:58.293576 2564 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-ingress/e2e-inf-a-06a5f2-openshift-default-7446cc6b8b-8k7x4"] Jul 29 19:53:58.293976 ip-10-0-131-93 kubenswrapper[2564]: I0729 19:53:58.293957 2564 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="910195fa-e9a6-48a6-81c9-9633feb14b95" containerName="istio-proxy" Jul 29 19:53:58.293976 ip-10-0-131-93 kubenswrapper[2564]: I0729 19:53:58.293975 2564 state_mem.go:107] "Deleted CPUSet assignment" podUID="910195fa-e9a6-48a6-81c9-9633feb14b95" containerName="istio-proxy" Jul 29 19:53:58.294159 ip-10-0-131-93 kubenswrapper[2564]: I0729 19:53:58.293990 2564 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="54900853-16b3-4e52-b5a3-a6a054dc32ba" containerName="istio-proxy" Jul 29 19:53:58.294159 ip-10-0-131-93 kubenswrapper[2564]: I0729 19:53:58.293998 2564 state_mem.go:107] "Deleted CPUSet assignment" podUID="54900853-16b3-4e52-b5a3-a6a054dc32ba" containerName="istio-proxy" Jul 29 19:53:58.294159 ip-10-0-131-93 kubenswrapper[2564]: I0729 19:53:58.294026 2564 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="54900853-16b3-4e52-b5a3-a6a054dc32ba" containerName="istio-proxy" Jul 29 19:53:58.294159 ip-10-0-131-93 kubenswrapper[2564]: I0729 19:53:58.294034 2564 state_mem.go:107] "Deleted CPUSet assignment" podUID="54900853-16b3-4e52-b5a3-a6a054dc32ba" containerName="istio-proxy" Jul 29 19:53:58.294159 ip-10-0-131-93 kubenswrapper[2564]: I0729 19:53:58.294084 2564 memory_manager.go:368] "RemoveStaleState removing state" podUID="54900853-16b3-4e52-b5a3-a6a054dc32ba" containerName="istio-proxy" Jul 29 19:53:58.294159 ip-10-0-131-93 kubenswrapper[2564]: I0729 19:53:58.294091 2564 memory_manager.go:368] "RemoveStaleState removing state" podUID="54900853-16b3-4e52-b5a3-a6a054dc32ba" containerName="istio-proxy" Jul 29 19:53:58.297164 ip-10-0-131-93 kubenswrapper[2564]: I0729 19:53:58.297145 2564 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-inf-a-06a5f2-openshift-default-7446cc6b8b-8k7x4" Jul 29 19:53:58.299587 ip-10-0-131-93 kubenswrapper[2564]: I0729 19:53:58.299567 2564 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress\"/\"e2e-inf-a-06a5f2-openshift-default-dockercfg-42bpc\"" Jul 29 19:53:58.313055 ip-10-0-131-93 kubenswrapper[2564]: I0729 19:53:58.313028 2564 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/e2e-inf-a-06a5f2-openshift-default-7446cc6b8b-8k7x4"] Jul 29 19:53:58.323722 ip-10-0-131-93 kubenswrapper[2564]: I0729 19:53:58.323698 2564 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-444jx_faa8c546-c75b-4e10-93b4-661aa2b5f53e/konnectivity-agent/0.log" limit=1024 Jul 29 19:53:58.362697 ip-10-0-131-93 kubenswrapper[2564]: I0729 19:53:58.362667 2564 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/56703a5f-13f3-4b32-8249-e6e9009667d3-workload-certs\") pod \"e2e-inf-a-06a5f2-openshift-default-7446cc6b8b-8k7x4\" (UID: \"56703a5f-13f3-4b32-8249-e6e9009667d3\") " pod="openshift-ingress/e2e-inf-a-06a5f2-openshift-default-7446cc6b8b-8k7x4" Jul 29 19:53:58.362697 ip-10-0-131-93 kubenswrapper[2564]: I0729 19:53:58.362698 2564 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/56703a5f-13f3-4b32-8249-e6e9009667d3-istio-envoy\") pod \"e2e-inf-a-06a5f2-openshift-default-7446cc6b8b-8k7x4\" (UID: \"56703a5f-13f3-4b32-8249-e6e9009667d3\") " pod="openshift-ingress/e2e-inf-a-06a5f2-openshift-default-7446cc6b8b-8k7x4" Jul 29 19:53:58.362864 ip-10-0-131-93 kubenswrapper[2564]: I0729 19:53:58.362736 2564 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/56703a5f-13f3-4b32-8249-e6e9009667d3-credential-socket\") pod \"e2e-inf-a-06a5f2-openshift-default-7446cc6b8b-8k7x4\" (UID: \"56703a5f-13f3-4b32-8249-e6e9009667d3\") " pod="openshift-ingress/e2e-inf-a-06a5f2-openshift-default-7446cc6b8b-8k7x4" Jul 29 19:53:58.362864 ip-10-0-131-93 kubenswrapper[2564]: I0729 19:53:58.362786 2564 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/56703a5f-13f3-4b32-8249-e6e9009667d3-istio-data\") pod \"e2e-inf-a-06a5f2-openshift-default-7446cc6b8b-8k7x4\" (UID: \"56703a5f-13f3-4b32-8249-e6e9009667d3\") " pod="openshift-ingress/e2e-inf-a-06a5f2-openshift-default-7446cc6b8b-8k7x4" Jul 29 19:53:58.362864 ip-10-0-131-93 kubenswrapper[2564]: I0729 19:53:58.362814 2564 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-jbpj6\" (UniqueName: \"kubernetes.io/projected/56703a5f-13f3-4b32-8249-e6e9009667d3-kube-api-access-jbpj6\") pod \"e2e-inf-a-06a5f2-openshift-default-7446cc6b8b-8k7x4\" (UID: \"56703a5f-13f3-4b32-8249-e6e9009667d3\") " pod="openshift-ingress/e2e-inf-a-06a5f2-openshift-default-7446cc6b8b-8k7x4" Jul 29 19:53:58.362864 ip-10-0-131-93 kubenswrapper[2564]: I0729 19:53:58.362848 2564 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/56703a5f-13f3-4b32-8249-e6e9009667d3-istio-token\") pod \"e2e-inf-a-06a5f2-openshift-default-7446cc6b8b-8k7x4\" (UID: \"56703a5f-13f3-4b32-8249-e6e9009667d3\") " pod="openshift-ingress/e2e-inf-a-06a5f2-openshift-default-7446cc6b8b-8k7x4" Jul 29 19:53:58.362993 ip-10-0-131-93 kubenswrapper[2564]: I0729 19:53:58.362865 2564 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/56703a5f-13f3-4b32-8249-e6e9009667d3-istio-podinfo\") pod \"e2e-inf-a-06a5f2-openshift-default-7446cc6b8b-8k7x4\" (UID: \"56703a5f-13f3-4b32-8249-e6e9009667d3\") " pod="openshift-ingress/e2e-inf-a-06a5f2-openshift-default-7446cc6b8b-8k7x4" Jul 29 19:53:58.362993 ip-10-0-131-93 kubenswrapper[2564]: I0729 19:53:58.362939 2564 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/56703a5f-13f3-4b32-8249-e6e9009667d3-istiod-ca-cert\") pod \"e2e-inf-a-06a5f2-openshift-default-7446cc6b8b-8k7x4\" (UID: \"56703a5f-13f3-4b32-8249-e6e9009667d3\") " pod="openshift-ingress/e2e-inf-a-06a5f2-openshift-default-7446cc6b8b-8k7x4" Jul 29 19:53:58.362993 ip-10-0-131-93 kubenswrapper[2564]: I0729 19:53:58.362966 2564 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/56703a5f-13f3-4b32-8249-e6e9009667d3-workload-socket\") pod \"e2e-inf-a-06a5f2-openshift-default-7446cc6b8b-8k7x4\" (UID: \"56703a5f-13f3-4b32-8249-e6e9009667d3\") " pod="openshift-ingress/e2e-inf-a-06a5f2-openshift-default-7446cc6b8b-8k7x4" Jul 29 19:53:58.464256 ip-10-0-131-93 kubenswrapper[2564]: I0729 19:53:58.464224 2564 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/56703a5f-13f3-4b32-8249-e6e9009667d3-workload-certs\") pod \"e2e-inf-a-06a5f2-openshift-default-7446cc6b8b-8k7x4\" (UID: \"56703a5f-13f3-4b32-8249-e6e9009667d3\") " pod="openshift-ingress/e2e-inf-a-06a5f2-openshift-default-7446cc6b8b-8k7x4" Jul 29 19:53:58.464256 ip-10-0-131-93 kubenswrapper[2564]: I0729 19:53:58.464254 2564 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/56703a5f-13f3-4b32-8249-e6e9009667d3-istio-envoy\") pod \"e2e-inf-a-06a5f2-openshift-default-7446cc6b8b-8k7x4\" (UID: \"56703a5f-13f3-4b32-8249-e6e9009667d3\") " pod="openshift-ingress/e2e-inf-a-06a5f2-openshift-default-7446cc6b8b-8k7x4" Jul 29 19:53:58.464508 ip-10-0-131-93 kubenswrapper[2564]: I0729 19:53:58.464292 2564 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/56703a5f-13f3-4b32-8249-e6e9009667d3-credential-socket\") pod \"e2e-inf-a-06a5f2-openshift-default-7446cc6b8b-8k7x4\" (UID: \"56703a5f-13f3-4b32-8249-e6e9009667d3\") " pod="openshift-ingress/e2e-inf-a-06a5f2-openshift-default-7446cc6b8b-8k7x4" Jul 29 19:53:58.464508 ip-10-0-131-93 kubenswrapper[2564]: I0729 19:53:58.464325 2564 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/56703a5f-13f3-4b32-8249-e6e9009667d3-istio-data\") pod \"e2e-inf-a-06a5f2-openshift-default-7446cc6b8b-8k7x4\" (UID: \"56703a5f-13f3-4b32-8249-e6e9009667d3\") " pod="openshift-ingress/e2e-inf-a-06a5f2-openshift-default-7446cc6b8b-8k7x4" Jul 29 19:53:58.464508 ip-10-0-131-93 kubenswrapper[2564]: I0729 19:53:58.464358 2564 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-jbpj6\" (UniqueName: \"kubernetes.io/projected/56703a5f-13f3-4b32-8249-e6e9009667d3-kube-api-access-jbpj6\") pod \"e2e-inf-a-06a5f2-openshift-default-7446cc6b8b-8k7x4\" (UID: \"56703a5f-13f3-4b32-8249-e6e9009667d3\") " pod="openshift-ingress/e2e-inf-a-06a5f2-openshift-default-7446cc6b8b-8k7x4" Jul 29 19:53:58.464508 ip-10-0-131-93 kubenswrapper[2564]: I0729 19:53:58.464416 2564 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/56703a5f-13f3-4b32-8249-e6e9009667d3-istio-token\") pod \"e2e-inf-a-06a5f2-openshift-default-7446cc6b8b-8k7x4\" (UID: \"56703a5f-13f3-4b32-8249-e6e9009667d3\") " pod="openshift-ingress/e2e-inf-a-06a5f2-openshift-default-7446cc6b8b-8k7x4" Jul 29 19:53:58.464508 ip-10-0-131-93 kubenswrapper[2564]: I0729 19:53:58.464441 2564 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/56703a5f-13f3-4b32-8249-e6e9009667d3-istio-podinfo\") pod \"e2e-inf-a-06a5f2-openshift-default-7446cc6b8b-8k7x4\" (UID: \"56703a5f-13f3-4b32-8249-e6e9009667d3\") " pod="openshift-ingress/e2e-inf-a-06a5f2-openshift-default-7446cc6b8b-8k7x4" Jul 29 19:53:58.464508 ip-10-0-131-93 kubenswrapper[2564]: I0729 19:53:58.464500 2564 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/56703a5f-13f3-4b32-8249-e6e9009667d3-istiod-ca-cert\") pod \"e2e-inf-a-06a5f2-openshift-default-7446cc6b8b-8k7x4\" (UID: \"56703a5f-13f3-4b32-8249-e6e9009667d3\") " pod="openshift-ingress/e2e-inf-a-06a5f2-openshift-default-7446cc6b8b-8k7x4" Jul 29 19:53:58.464817 ip-10-0-131-93 kubenswrapper[2564]: I0729 19:53:58.464528 2564 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/56703a5f-13f3-4b32-8249-e6e9009667d3-workload-socket\") pod \"e2e-inf-a-06a5f2-openshift-default-7446cc6b8b-8k7x4\" (UID: \"56703a5f-13f3-4b32-8249-e6e9009667d3\") " pod="openshift-ingress/e2e-inf-a-06a5f2-openshift-default-7446cc6b8b-8k7x4" Jul 29 19:53:58.464817 ip-10-0-131-93 kubenswrapper[2564]: I0729 19:53:58.464653 2564 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/56703a5f-13f3-4b32-8249-e6e9009667d3-workload-certs\") pod \"e2e-inf-a-06a5f2-openshift-default-7446cc6b8b-8k7x4\" (UID: \"56703a5f-13f3-4b32-8249-e6e9009667d3\") " pod="openshift-ingress/e2e-inf-a-06a5f2-openshift-default-7446cc6b8b-8k7x4" Jul 29 19:53:58.464817 ip-10-0-131-93 kubenswrapper[2564]: I0729 19:53:58.464711 2564 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/56703a5f-13f3-4b32-8249-e6e9009667d3-credential-socket\") pod \"e2e-inf-a-06a5f2-openshift-default-7446cc6b8b-8k7x4\" (UID: \"56703a5f-13f3-4b32-8249-e6e9009667d3\") " pod="openshift-ingress/e2e-inf-a-06a5f2-openshift-default-7446cc6b8b-8k7x4" Jul 29 19:53:58.464978 ip-10-0-131-93 kubenswrapper[2564]: I0729 19:53:58.464834 2564 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/56703a5f-13f3-4b32-8249-e6e9009667d3-workload-socket\") pod \"e2e-inf-a-06a5f2-openshift-default-7446cc6b8b-8k7x4\" (UID: \"56703a5f-13f3-4b32-8249-e6e9009667d3\") " pod="openshift-ingress/e2e-inf-a-06a5f2-openshift-default-7446cc6b8b-8k7x4" Jul 29 19:53:58.464978 ip-10-0-131-93 kubenswrapper[2564]: I0729 19:53:58.464868 2564 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/56703a5f-13f3-4b32-8249-e6e9009667d3-istio-data\") pod \"e2e-inf-a-06a5f2-openshift-default-7446cc6b8b-8k7x4\" (UID: \"56703a5f-13f3-4b32-8249-e6e9009667d3\") " pod="openshift-ingress/e2e-inf-a-06a5f2-openshift-default-7446cc6b8b-8k7x4" Jul 29 19:53:58.465272 ip-10-0-131-93 kubenswrapper[2564]: I0729 19:53:58.465257 2564 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/56703a5f-13f3-4b32-8249-e6e9009667d3-istiod-ca-cert\") pod \"e2e-inf-a-06a5f2-openshift-default-7446cc6b8b-8k7x4\" (UID: \"56703a5f-13f3-4b32-8249-e6e9009667d3\") " pod="openshift-ingress/e2e-inf-a-06a5f2-openshift-default-7446cc6b8b-8k7x4" Jul 29 19:53:58.466675 ip-10-0-131-93 kubenswrapper[2564]: I0729 19:53:58.466654 2564 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/56703a5f-13f3-4b32-8249-e6e9009667d3-istio-envoy\") pod \"e2e-inf-a-06a5f2-openshift-default-7446cc6b8b-8k7x4\" (UID: \"56703a5f-13f3-4b32-8249-e6e9009667d3\") " pod="openshift-ingress/e2e-inf-a-06a5f2-openshift-default-7446cc6b8b-8k7x4" Jul 29 19:53:58.466922 ip-10-0-131-93 kubenswrapper[2564]: I0729 19:53:58.466901 2564 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/56703a5f-13f3-4b32-8249-e6e9009667d3-istio-podinfo\") pod \"e2e-inf-a-06a5f2-openshift-default-7446cc6b8b-8k7x4\" (UID: \"56703a5f-13f3-4b32-8249-e6e9009667d3\") " pod="openshift-ingress/e2e-inf-a-06a5f2-openshift-default-7446cc6b8b-8k7x4" Jul 29 19:53:58.473173 ip-10-0-131-93 kubenswrapper[2564]: I0729 19:53:58.473148 2564 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/56703a5f-13f3-4b32-8249-e6e9009667d3-istio-token\") pod \"e2e-inf-a-06a5f2-openshift-default-7446cc6b8b-8k7x4\" (UID: \"56703a5f-13f3-4b32-8249-e6e9009667d3\") " pod="openshift-ingress/e2e-inf-a-06a5f2-openshift-default-7446cc6b8b-8k7x4" Jul 29 19:53:58.473613 ip-10-0-131-93 kubenswrapper[2564]: I0729 19:53:58.473592 2564 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-jbpj6\" (UniqueName: \"kubernetes.io/projected/56703a5f-13f3-4b32-8249-e6e9009667d3-kube-api-access-jbpj6\") pod \"e2e-inf-a-06a5f2-openshift-default-7446cc6b8b-8k7x4\" (UID: \"56703a5f-13f3-4b32-8249-e6e9009667d3\") " pod="openshift-ingress/e2e-inf-a-06a5f2-openshift-default-7446cc6b8b-8k7x4" Jul 29 19:53:58.608042 ip-10-0-131-93 kubenswrapper[2564]: I0729 19:53:58.607931 2564 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-inf-a-06a5f2-openshift-default-7446cc6b8b-8k7x4" Jul 29 19:53:58.723055 ip-10-0-131-93 kubenswrapper[2564]: I0729 19:53:58.723018 2564 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-444jx_faa8c546-c75b-4e10-93b4-661aa2b5f53e/konnectivity-agent/0.log" limit=1024 Jul 29 19:53:58.766902 ip-10-0-131-93 kubenswrapper[2564]: I0729 19:53:58.766873 2564 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/e2e-inf-a-06a5f2-openshift-default-7446cc6b8b-8k7x4"] Jul 29 19:53:58.767435 ip-10-0-131-93 kubenswrapper[2564]: W0729 19:53:58.767410 2564 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod56703a5f_13f3_4b32_8249_e6e9009667d3.slice/crio-08111a830e116046620cb84e79e00e72add56e4ce0134693e0ad0373b276623e WatchSource:0}: Error finding container 08111a830e116046620cb84e79e00e72add56e4ce0134693e0ad0373b276623e: Status 404 returned error can't find the container with id 08111a830e116046620cb84e79e00e72add56e4ce0134693e0ad0373b276623e Jul 29 19:53:58.769387 ip-10-0-131-93 kubenswrapper[2564]: I0729 19:53:58.769340 2564 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236224Ki","pods":"250"} Jul 29 19:53:58.769496 ip-10-0-131-93 kubenswrapper[2564]: I0729 19:53:58.769433 2564 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236224Ki","pods":"250"} Jul 29 19:53:58.769496 ip-10-0-131-93 kubenswrapper[2564]: I0729 19:53:58.769465 2564 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236224Ki","pods":"250"} Jul 29 19:53:58.960481 ip-10-0-131-93 kubenswrapper[2564]: I0729 19:53:58.960440 2564 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-inf-a-06a5f2-openshift-default-7446cc6b8b-8k7x4" event={"ID":"56703a5f-13f3-4b32-8249-e6e9009667d3","Type":"ContainerStarted","Data":"5f118590bbade0c9af83f430f31bfb5a07c6967f13df304e3db055f4cead30b0"} Jul 29 19:53:58.960694 ip-10-0-131-93 kubenswrapper[2564]: I0729 19:53:58.960501 2564 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-inf-a-06a5f2-openshift-default-7446cc6b8b-8k7x4" event={"ID":"56703a5f-13f3-4b32-8249-e6e9009667d3","Type":"ContainerStarted","Data":"08111a830e116046620cb84e79e00e72add56e4ce0134693e0ad0373b276623e"} Jul 29 19:53:58.980624 ip-10-0-131-93 kubenswrapper[2564]: I0729 19:53:58.980567 2564 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-ingress/e2e-inf-a-06a5f2-openshift-default-7446cc6b8b-8k7x4" podStartSLOduration=0.980551091 podStartE2EDuration="980.551091ms" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-29 19:53:58 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-29 19:53:58.978908688 +0000 UTC m=+2214.702968382" watchObservedRunningTime="2026-07-29 19:53:58.980551091 +0000 UTC m=+2214.704610782" Jul 29 19:53:59.608630 ip-10-0-131-93 kubenswrapper[2564]: I0729 19:53:59.608597 2564 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="openshift-ingress/e2e-inf-a-06a5f2-openshift-default-7446cc6b8b-8k7x4" Jul 29 19:53:59.613360 ip-10-0-131-93 kubenswrapper[2564]: I0729 19:53:59.613335 2564 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="openshift-ingress/e2e-inf-a-06a5f2-openshift-default-7446cc6b8b-8k7x4" Jul 29 19:53:59.965637 ip-10-0-131-93 kubenswrapper[2564]: I0729 19:53:59.965606 2564 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/e2e-inf-a-06a5f2-openshift-default-7446cc6b8b-8k7x4" Jul 29 19:53:59.966691 ip-10-0-131-93 kubenswrapper[2564]: I0729 19:53:59.966668 2564 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/e2e-inf-a-06a5f2-openshift-default-7446cc6b8b-8k7x4" Jul 29 19:54:00.895580 ip-10-0-131-93 kubenswrapper[2564]: I0729 19:54:00.895546 2564 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-444jx_faa8c546-c75b-4e10-93b4-661aa2b5f53e/konnectivity-agent/0.log" limit=1024 Jul 29 19:54:00.996321 ip-10-0-131-93 kubenswrapper[2564]: I0729 19:54:00.996290 2564 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-444jx_faa8c546-c75b-4e10-93b4-661aa2b5f53e/konnectivity-agent/0.log" limit=1024 Jul 29 19:54:02.178367 ip-10-0-131-93 kubenswrapper[2564]: I0729 19:54:02.178330 2564 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-444jx_faa8c546-c75b-4e10-93b4-661aa2b5f53e/konnectivity-agent/0.log" limit=1024 Jul 29 19:54:02.333520 ip-10-0-131-93 kubenswrapper[2564]: I0729 19:54:02.333485 2564 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-444jx_faa8c546-c75b-4e10-93b4-661aa2b5f53e/konnectivity-agent/0.log" limit=1024 Jul 29 19:54:02.668244 ip-10-0-131-93 kubenswrapper[2564]: I0729 19:54:02.668211 2564 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-444jx_faa8c546-c75b-4e10-93b4-661aa2b5f53e/konnectivity-agent/0.log" limit=1024 Jul 29 19:54:02.765648 ip-10-0-131-93 kubenswrapper[2564]: I0729 19:54:02.765620 2564 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-444jx_faa8c546-c75b-4e10-93b4-661aa2b5f53e/konnectivity-agent/0.log" limit=1024 Jul 29 19:54:12.199294 ip-10-0-131-93 kubenswrapper[2564]: I0729 19:54:12.199257 2564 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-444jx_faa8c546-c75b-4e10-93b4-661aa2b5f53e/konnectivity-agent/0.log" limit=1024 Jul 29 19:54:12.260910 ip-10-0-131-93 kubenswrapper[2564]: I0729 19:54:12.260882 2564 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-444jx_faa8c546-c75b-4e10-93b4-661aa2b5f53e/konnectivity-agent/0.log" limit=1024 Jul 29 19:54:14.966995 ip-10-0-131-93 kubenswrapper[2564]: I0729 19:54:14.966951 2564 patch_prober.go:28] interesting pod/e2e-inf-a-06a5f2-openshift-default-7446cc6b8b-8k7x4 container/istio-proxy namespace/openshift-ingress: Readiness probe status=failure output="Get \"http://10.132.0.43:15021/healthz/ready\": dial tcp 10.132.0.43:15021: connect: connection refused" start-of-body= Jul 29 19:54:14.967470 ip-10-0-131-93 kubenswrapper[2564]: I0729 19:54:14.967031 2564 prober.go:120] "Probe failed" probeType="Readiness" pod="openshift-ingress/e2e-inf-a-06a5f2-openshift-default-7446cc6b8b-8k7x4" podUID="56703a5f-13f3-4b32-8249-e6e9009667d3" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.132.0.43:15021/healthz/ready\": dial tcp 10.132.0.43:15021: connect: connection refused" Jul 29 19:54:15.036496 ip-10-0-131-93 kubenswrapper[2564]: I0729 19:54:15.036412 2564 generic.go:356] "Generic (PLEG): container finished" podID="56703a5f-13f3-4b32-8249-e6e9009667d3" containerID="5f118590bbade0c9af83f430f31bfb5a07c6967f13df304e3db055f4cead30b0" exitCode=0 Jul 29 19:54:15.036639 ip-10-0-131-93 kubenswrapper[2564]: I0729 19:54:15.036488 2564 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-inf-a-06a5f2-openshift-default-7446cc6b8b-8k7x4" event={"ID":"56703a5f-13f3-4b32-8249-e6e9009667d3","Type":"ContainerDied","Data":"5f118590bbade0c9af83f430f31bfb5a07c6967f13df304e3db055f4cead30b0"} Jul 29 19:54:15.036891 ip-10-0-131-93 kubenswrapper[2564]: I0729 19:54:15.036877 2564 scope.go:117] "RemoveContainer" containerID="5f118590bbade0c9af83f430f31bfb5a07c6967f13df304e3db055f4cead30b0" Jul 29 19:54:15.038001 ip-10-0-131-93 kubenswrapper[2564]: I0729 19:54:15.037972 2564 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236224Ki","pods":"250"} Jul 29 19:54:15.038080 ip-10-0-131-93 kubenswrapper[2564]: I0729 19:54:15.038062 2564 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236224Ki","pods":"250"} Jul 29 19:54:15.038117 ip-10-0-131-93 kubenswrapper[2564]: I0729 19:54:15.038103 2564 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236224Ki","pods":"250"} Jul 29 19:54:16.043467 ip-10-0-131-93 kubenswrapper[2564]: I0729 19:54:16.043434 2564 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-inf-a-06a5f2-openshift-default-7446cc6b8b-8k7x4" event={"ID":"56703a5f-13f3-4b32-8249-e6e9009667d3","Type":"ContainerStarted","Data":"6f391610d94b1d2a514d648e48c64fcee38327dc75a1ff66642bd9896c9f73d7"} Jul 29 19:54:16.608478 ip-10-0-131-93 kubenswrapper[2564]: I0729 19:54:16.608443 2564 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="openshift-ingress/e2e-inf-a-06a5f2-openshift-default-7446cc6b8b-8k7x4" Jul 29 19:54:16.608689 ip-10-0-131-93 kubenswrapper[2564]: I0729 19:54:16.608659 2564 patch_prober.go:28] interesting pod/e2e-inf-a-06a5f2-openshift-default-7446cc6b8b-8k7x4 container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.132.0.43:15021/healthz/ready\": dial tcp 10.132.0.43:15021: connect: connection refused" start-of-body= Jul 29 19:54:16.608772 ip-10-0-131-93 kubenswrapper[2564]: I0729 19:54:16.608723 2564 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-inf-a-06a5f2-openshift-default-7446cc6b8b-8k7x4" podUID="56703a5f-13f3-4b32-8249-e6e9009667d3" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.132.0.43:15021/healthz/ready\": dial tcp 10.132.0.43:15021: connect: connection refused" Jul 29 19:54:17.608946 ip-10-0-131-93 kubenswrapper[2564]: I0729 19:54:17.608913 2564 patch_prober.go:28] interesting pod/e2e-inf-a-06a5f2-openshift-default-7446cc6b8b-8k7x4 container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.132.0.43:15021/healthz/ready\": dial tcp 10.132.0.43:15021: connect: connection refused" start-of-body= Jul 29 19:54:17.609341 ip-10-0-131-93 kubenswrapper[2564]: I0729 19:54:17.608969 2564 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-inf-a-06a5f2-openshift-default-7446cc6b8b-8k7x4" podUID="56703a5f-13f3-4b32-8249-e6e9009667d3" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.132.0.43:15021/healthz/ready\": dial tcp 10.132.0.43:15021: connect: connection refused" Jul 29 19:54:18.608451 ip-10-0-131-93 kubenswrapper[2564]: I0729 19:54:18.608415 2564 patch_prober.go:28] interesting pod/e2e-inf-a-06a5f2-openshift-default-7446cc6b8b-8k7x4 container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.132.0.43:15021/healthz/ready\": dial tcp 10.132.0.43:15021: connect: connection refused" start-of-body= Jul 29 19:54:18.608639 ip-10-0-131-93 kubenswrapper[2564]: I0729 19:54:18.608468 2564 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-inf-a-06a5f2-openshift-default-7446cc6b8b-8k7x4" podUID="56703a5f-13f3-4b32-8249-e6e9009667d3" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.132.0.43:15021/healthz/ready\": dial tcp 10.132.0.43:15021: connect: connection refused" Jul 29 19:54:19.608821 ip-10-0-131-93 kubenswrapper[2564]: I0729 19:54:19.608780 2564 patch_prober.go:28] interesting pod/e2e-inf-a-06a5f2-openshift-default-7446cc6b8b-8k7x4 container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.132.0.43:15021/healthz/ready\": dial tcp 10.132.0.43:15021: connect: connection refused" start-of-body= Jul 29 19:54:19.609293 ip-10-0-131-93 kubenswrapper[2564]: I0729 19:54:19.608842 2564 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-inf-a-06a5f2-openshift-default-7446cc6b8b-8k7x4" podUID="56703a5f-13f3-4b32-8249-e6e9009667d3" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.132.0.43:15021/healthz/ready\": dial tcp 10.132.0.43:15021: connect: connection refused" Jul 29 19:54:20.612551 ip-10-0-131-93 kubenswrapper[2564]: I0729 19:54:20.612523 2564 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="openshift-ingress/e2e-inf-a-06a5f2-openshift-default-7446cc6b8b-8k7x4" Jul 29 19:54:20.612910 ip-10-0-131-93 kubenswrapper[2564]: I0729 19:54:20.612785 2564 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/e2e-inf-a-06a5f2-openshift-default-7446cc6b8b-8k7x4" Jul 29 19:54:20.613423 ip-10-0-131-93 kubenswrapper[2564]: I0729 19:54:20.613407 2564 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/e2e-inf-a-06a5f2-openshift-default-7446cc6b8b-8k7x4" Jul 29 19:54:25.871356 ip-10-0-131-93 kubenswrapper[2564]: I0729 19:54:25.871311 2564 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-444jx_faa8c546-c75b-4e10-93b4-661aa2b5f53e/konnectivity-agent/0.log" limit=1024 Jul 29 19:54:26.201475 ip-10-0-131-93 kubenswrapper[2564]: I0729 19:54:26.201437 2564 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-444jx_faa8c546-c75b-4e10-93b4-661aa2b5f53e/konnectivity-agent/0.log" limit=1024 Jul 29 19:54:26.483040 ip-10-0-131-93 kubenswrapper[2564]: I0729 19:54:26.482962 2564 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-ingress/e2e-inf-b-1d9a27-openshift-default-6f55cd5b96-2sdpw"] Jul 29 19:54:26.485853 ip-10-0-131-93 kubenswrapper[2564]: I0729 19:54:26.485835 2564 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-inf-b-1d9a27-openshift-default-6f55cd5b96-2sdpw" Jul 29 19:54:26.488563 ip-10-0-131-93 kubenswrapper[2564]: I0729 19:54:26.488544 2564 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress\"/\"e2e-inf-b-1d9a27-openshift-default-dockercfg-vxbwx\"" Jul 29 19:54:26.499080 ip-10-0-131-93 kubenswrapper[2564]: I0729 19:54:26.499056 2564 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/e2e-inf-b-1d9a27-openshift-default-6f55cd5b96-2sdpw"] Jul 29 19:54:26.510359 ip-10-0-131-93 kubenswrapper[2564]: I0729 19:54:26.510320 2564 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-444jx_faa8c546-c75b-4e10-93b4-661aa2b5f53e/konnectivity-agent/0.log" limit=1024 Jul 29 19:54:26.613049 ip-10-0-131-93 kubenswrapper[2564]: I0729 19:54:26.613009 2564 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/63de2e3f-49bc-4d59-87c0-1bf702c71b5c-istio-podinfo\") pod \"e2e-inf-b-1d9a27-openshift-default-6f55cd5b96-2sdpw\" (UID: \"63de2e3f-49bc-4d59-87c0-1bf702c71b5c\") " pod="openshift-ingress/e2e-inf-b-1d9a27-openshift-default-6f55cd5b96-2sdpw" Jul 29 19:54:26.613049 ip-10-0-131-93 kubenswrapper[2564]: I0729 19:54:26.613045 2564 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/63de2e3f-49bc-4d59-87c0-1bf702c71b5c-istio-token\") pod \"e2e-inf-b-1d9a27-openshift-default-6f55cd5b96-2sdpw\" (UID: \"63de2e3f-49bc-4d59-87c0-1bf702c71b5c\") " pod="openshift-ingress/e2e-inf-b-1d9a27-openshift-default-6f55cd5b96-2sdpw" Jul 29 19:54:26.613285 ip-10-0-131-93 kubenswrapper[2564]: I0729 19:54:26.613073 2564 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/63de2e3f-49bc-4d59-87c0-1bf702c71b5c-workload-certs\") pod \"e2e-inf-b-1d9a27-openshift-default-6f55cd5b96-2sdpw\" (UID: \"63de2e3f-49bc-4d59-87c0-1bf702c71b5c\") " pod="openshift-ingress/e2e-inf-b-1d9a27-openshift-default-6f55cd5b96-2sdpw" Jul 29 19:54:26.613285 ip-10-0-131-93 kubenswrapper[2564]: I0729 19:54:26.613097 2564 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/63de2e3f-49bc-4d59-87c0-1bf702c71b5c-istio-envoy\") pod \"e2e-inf-b-1d9a27-openshift-default-6f55cd5b96-2sdpw\" (UID: \"63de2e3f-49bc-4d59-87c0-1bf702c71b5c\") " pod="openshift-ingress/e2e-inf-b-1d9a27-openshift-default-6f55cd5b96-2sdpw" Jul 29 19:54:26.613285 ip-10-0-131-93 kubenswrapper[2564]: I0729 19:54:26.613204 2564 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-2qs9x\" (UniqueName: \"kubernetes.io/projected/63de2e3f-49bc-4d59-87c0-1bf702c71b5c-kube-api-access-2qs9x\") pod \"e2e-inf-b-1d9a27-openshift-default-6f55cd5b96-2sdpw\" (UID: \"63de2e3f-49bc-4d59-87c0-1bf702c71b5c\") " pod="openshift-ingress/e2e-inf-b-1d9a27-openshift-default-6f55cd5b96-2sdpw" Jul 29 19:54:26.613429 ip-10-0-131-93 kubenswrapper[2564]: I0729 19:54:26.613282 2564 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/63de2e3f-49bc-4d59-87c0-1bf702c71b5c-istiod-ca-cert\") pod \"e2e-inf-b-1d9a27-openshift-default-6f55cd5b96-2sdpw\" (UID: \"63de2e3f-49bc-4d59-87c0-1bf702c71b5c\") " pod="openshift-ingress/e2e-inf-b-1d9a27-openshift-default-6f55cd5b96-2sdpw" Jul 29 19:54:26.613429 ip-10-0-131-93 kubenswrapper[2564]: I0729 19:54:26.613335 2564 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/63de2e3f-49bc-4d59-87c0-1bf702c71b5c-istio-data\") pod \"e2e-inf-b-1d9a27-openshift-default-6f55cd5b96-2sdpw\" (UID: \"63de2e3f-49bc-4d59-87c0-1bf702c71b5c\") " pod="openshift-ingress/e2e-inf-b-1d9a27-openshift-default-6f55cd5b96-2sdpw" Jul 29 19:54:26.613508 ip-10-0-131-93 kubenswrapper[2564]: I0729 19:54:26.613442 2564 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/63de2e3f-49bc-4d59-87c0-1bf702c71b5c-credential-socket\") pod \"e2e-inf-b-1d9a27-openshift-default-6f55cd5b96-2sdpw\" (UID: \"63de2e3f-49bc-4d59-87c0-1bf702c71b5c\") " pod="openshift-ingress/e2e-inf-b-1d9a27-openshift-default-6f55cd5b96-2sdpw" Jul 29 19:54:26.613508 ip-10-0-131-93 kubenswrapper[2564]: I0729 19:54:26.613487 2564 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/63de2e3f-49bc-4d59-87c0-1bf702c71b5c-workload-socket\") pod \"e2e-inf-b-1d9a27-openshift-default-6f55cd5b96-2sdpw\" (UID: \"63de2e3f-49bc-4d59-87c0-1bf702c71b5c\") " pod="openshift-ingress/e2e-inf-b-1d9a27-openshift-default-6f55cd5b96-2sdpw" Jul 29 19:54:26.638671 ip-10-0-131-93 kubenswrapper[2564]: I0729 19:54:26.638645 2564 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-444jx_faa8c546-c75b-4e10-93b4-661aa2b5f53e/konnectivity-agent/0.log" limit=1024 Jul 29 19:54:26.714697 ip-10-0-131-93 kubenswrapper[2564]: I0729 19:54:26.714664 2564 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/63de2e3f-49bc-4d59-87c0-1bf702c71b5c-istio-data\") pod \"e2e-inf-b-1d9a27-openshift-default-6f55cd5b96-2sdpw\" (UID: \"63de2e3f-49bc-4d59-87c0-1bf702c71b5c\") " pod="openshift-ingress/e2e-inf-b-1d9a27-openshift-default-6f55cd5b96-2sdpw" Jul 29 19:54:26.714697 ip-10-0-131-93 kubenswrapper[2564]: I0729 19:54:26.714706 2564 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/63de2e3f-49bc-4d59-87c0-1bf702c71b5c-credential-socket\") pod \"e2e-inf-b-1d9a27-openshift-default-6f55cd5b96-2sdpw\" (UID: \"63de2e3f-49bc-4d59-87c0-1bf702c71b5c\") " pod="openshift-ingress/e2e-inf-b-1d9a27-openshift-default-6f55cd5b96-2sdpw" Jul 29 19:54:26.714950 ip-10-0-131-93 kubenswrapper[2564]: I0729 19:54:26.714744 2564 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/63de2e3f-49bc-4d59-87c0-1bf702c71b5c-workload-socket\") pod \"e2e-inf-b-1d9a27-openshift-default-6f55cd5b96-2sdpw\" (UID: \"63de2e3f-49bc-4d59-87c0-1bf702c71b5c\") " pod="openshift-ingress/e2e-inf-b-1d9a27-openshift-default-6f55cd5b96-2sdpw" Jul 29 19:54:26.714950 ip-10-0-131-93 kubenswrapper[2564]: I0729 19:54:26.714770 2564 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/63de2e3f-49bc-4d59-87c0-1bf702c71b5c-istio-podinfo\") pod \"e2e-inf-b-1d9a27-openshift-default-6f55cd5b96-2sdpw\" (UID: \"63de2e3f-49bc-4d59-87c0-1bf702c71b5c\") " pod="openshift-ingress/e2e-inf-b-1d9a27-openshift-default-6f55cd5b96-2sdpw" Jul 29 19:54:26.714950 ip-10-0-131-93 kubenswrapper[2564]: I0729 19:54:26.714787 2564 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/63de2e3f-49bc-4d59-87c0-1bf702c71b5c-istio-token\") pod \"e2e-inf-b-1d9a27-openshift-default-6f55cd5b96-2sdpw\" (UID: \"63de2e3f-49bc-4d59-87c0-1bf702c71b5c\") " pod="openshift-ingress/e2e-inf-b-1d9a27-openshift-default-6f55cd5b96-2sdpw" Jul 29 19:54:26.714950 ip-10-0-131-93 kubenswrapper[2564]: I0729 19:54:26.714815 2564 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/63de2e3f-49bc-4d59-87c0-1bf702c71b5c-workload-certs\") pod \"e2e-inf-b-1d9a27-openshift-default-6f55cd5b96-2sdpw\" (UID: \"63de2e3f-49bc-4d59-87c0-1bf702c71b5c\") " pod="openshift-ingress/e2e-inf-b-1d9a27-openshift-default-6f55cd5b96-2sdpw" Jul 29 19:54:26.714950 ip-10-0-131-93 kubenswrapper[2564]: I0729 19:54:26.714838 2564 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/63de2e3f-49bc-4d59-87c0-1bf702c71b5c-istio-envoy\") pod \"e2e-inf-b-1d9a27-openshift-default-6f55cd5b96-2sdpw\" (UID: \"63de2e3f-49bc-4d59-87c0-1bf702c71b5c\") " pod="openshift-ingress/e2e-inf-b-1d9a27-openshift-default-6f55cd5b96-2sdpw" Jul 29 19:54:26.714950 ip-10-0-131-93 kubenswrapper[2564]: I0729 19:54:26.714891 2564 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-2qs9x\" (UniqueName: \"kubernetes.io/projected/63de2e3f-49bc-4d59-87c0-1bf702c71b5c-kube-api-access-2qs9x\") pod \"e2e-inf-b-1d9a27-openshift-default-6f55cd5b96-2sdpw\" (UID: \"63de2e3f-49bc-4d59-87c0-1bf702c71b5c\") " pod="openshift-ingress/e2e-inf-b-1d9a27-openshift-default-6f55cd5b96-2sdpw" Jul 29 19:54:26.714950 ip-10-0-131-93 kubenswrapper[2564]: I0729 19:54:26.714930 2564 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/63de2e3f-49bc-4d59-87c0-1bf702c71b5c-istiod-ca-cert\") pod \"e2e-inf-b-1d9a27-openshift-default-6f55cd5b96-2sdpw\" (UID: \"63de2e3f-49bc-4d59-87c0-1bf702c71b5c\") " pod="openshift-ingress/e2e-inf-b-1d9a27-openshift-default-6f55cd5b96-2sdpw" Jul 29 19:54:26.715287 ip-10-0-131-93 kubenswrapper[2564]: I0729 19:54:26.715135 2564 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/63de2e3f-49bc-4d59-87c0-1bf702c71b5c-istio-data\") pod \"e2e-inf-b-1d9a27-openshift-default-6f55cd5b96-2sdpw\" (UID: \"63de2e3f-49bc-4d59-87c0-1bf702c71b5c\") " pod="openshift-ingress/e2e-inf-b-1d9a27-openshift-default-6f55cd5b96-2sdpw" Jul 29 19:54:26.715287 ip-10-0-131-93 kubenswrapper[2564]: I0729 19:54:26.715221 2564 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/63de2e3f-49bc-4d59-87c0-1bf702c71b5c-workload-socket\") pod \"e2e-inf-b-1d9a27-openshift-default-6f55cd5b96-2sdpw\" (UID: \"63de2e3f-49bc-4d59-87c0-1bf702c71b5c\") " pod="openshift-ingress/e2e-inf-b-1d9a27-openshift-default-6f55cd5b96-2sdpw" Jul 29 19:54:26.715287 ip-10-0-131-93 kubenswrapper[2564]: I0729 19:54:26.715269 2564 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/63de2e3f-49bc-4d59-87c0-1bf702c71b5c-credential-socket\") pod \"e2e-inf-b-1d9a27-openshift-default-6f55cd5b96-2sdpw\" (UID: \"63de2e3f-49bc-4d59-87c0-1bf702c71b5c\") " pod="openshift-ingress/e2e-inf-b-1d9a27-openshift-default-6f55cd5b96-2sdpw" Jul 29 19:54:26.715447 ip-10-0-131-93 kubenswrapper[2564]: I0729 19:54:26.715310 2564 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/63de2e3f-49bc-4d59-87c0-1bf702c71b5c-workload-certs\") pod \"e2e-inf-b-1d9a27-openshift-default-6f55cd5b96-2sdpw\" (UID: \"63de2e3f-49bc-4d59-87c0-1bf702c71b5c\") " pod="openshift-ingress/e2e-inf-b-1d9a27-openshift-default-6f55cd5b96-2sdpw" Jul 29 19:54:26.715818 ip-10-0-131-93 kubenswrapper[2564]: I0729 19:54:26.715801 2564 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/63de2e3f-49bc-4d59-87c0-1bf702c71b5c-istiod-ca-cert\") pod \"e2e-inf-b-1d9a27-openshift-default-6f55cd5b96-2sdpw\" (UID: \"63de2e3f-49bc-4d59-87c0-1bf702c71b5c\") " pod="openshift-ingress/e2e-inf-b-1d9a27-openshift-default-6f55cd5b96-2sdpw" Jul 29 19:54:26.717186 ip-10-0-131-93 kubenswrapper[2564]: I0729 19:54:26.717160 2564 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/63de2e3f-49bc-4d59-87c0-1bf702c71b5c-istio-envoy\") pod \"e2e-inf-b-1d9a27-openshift-default-6f55cd5b96-2sdpw\" (UID: \"63de2e3f-49bc-4d59-87c0-1bf702c71b5c\") " pod="openshift-ingress/e2e-inf-b-1d9a27-openshift-default-6f55cd5b96-2sdpw" Jul 29 19:54:26.717422 ip-10-0-131-93 kubenswrapper[2564]: I0729 19:54:26.717404 2564 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/63de2e3f-49bc-4d59-87c0-1bf702c71b5c-istio-podinfo\") pod \"e2e-inf-b-1d9a27-openshift-default-6f55cd5b96-2sdpw\" (UID: \"63de2e3f-49bc-4d59-87c0-1bf702c71b5c\") " pod="openshift-ingress/e2e-inf-b-1d9a27-openshift-default-6f55cd5b96-2sdpw" Jul 29 19:54:26.722815 ip-10-0-131-93 kubenswrapper[2564]: I0729 19:54:26.722792 2564 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/63de2e3f-49bc-4d59-87c0-1bf702c71b5c-istio-token\") pod \"e2e-inf-b-1d9a27-openshift-default-6f55cd5b96-2sdpw\" (UID: \"63de2e3f-49bc-4d59-87c0-1bf702c71b5c\") " pod="openshift-ingress/e2e-inf-b-1d9a27-openshift-default-6f55cd5b96-2sdpw" Jul 29 19:54:26.722973 ip-10-0-131-93 kubenswrapper[2564]: I0729 19:54:26.722956 2564 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-2qs9x\" (UniqueName: \"kubernetes.io/projected/63de2e3f-49bc-4d59-87c0-1bf702c71b5c-kube-api-access-2qs9x\") pod \"e2e-inf-b-1d9a27-openshift-default-6f55cd5b96-2sdpw\" (UID: \"63de2e3f-49bc-4d59-87c0-1bf702c71b5c\") " pod="openshift-ingress/e2e-inf-b-1d9a27-openshift-default-6f55cd5b96-2sdpw" Jul 29 19:54:26.796329 ip-10-0-131-93 kubenswrapper[2564]: I0729 19:54:26.796254 2564 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-inf-b-1d9a27-openshift-default-6f55cd5b96-2sdpw" Jul 29 19:54:26.965551 ip-10-0-131-93 kubenswrapper[2564]: I0729 19:54:26.965516 2564 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/e2e-inf-b-1d9a27-openshift-default-6f55cd5b96-2sdpw"] Jul 29 19:54:26.968768 ip-10-0-131-93 kubenswrapper[2564]: W0729 19:54:26.968738 2564 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod63de2e3f_49bc_4d59_87c0_1bf702c71b5c.slice/crio-73b53189f8bf2d73366cfd0c5d177f196d5217aaf52770c6db1daf2e148cad29 WatchSource:0}: Error finding container 73b53189f8bf2d73366cfd0c5d177f196d5217aaf52770c6db1daf2e148cad29: Status 404 returned error can't find the container with id 73b53189f8bf2d73366cfd0c5d177f196d5217aaf52770c6db1daf2e148cad29 Jul 29 19:54:26.971303 ip-10-0-131-93 kubenswrapper[2564]: I0729 19:54:26.971271 2564 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236224Ki","pods":"250"} Jul 29 19:54:26.971439 ip-10-0-131-93 kubenswrapper[2564]: I0729 19:54:26.971349 2564 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236224Ki","pods":"250"} Jul 29 19:54:26.971439 ip-10-0-131-93 kubenswrapper[2564]: I0729 19:54:26.971413 2564 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236224Ki","pods":"250"} Jul 29 19:54:27.102967 ip-10-0-131-93 kubenswrapper[2564]: I0729 19:54:27.102882 2564 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-inf-b-1d9a27-openshift-default-6f55cd5b96-2sdpw" event={"ID":"63de2e3f-49bc-4d59-87c0-1bf702c71b5c","Type":"ContainerStarted","Data":"fa4a25d2fabfaaa9b80e2819b00a9dfc08d05c8903e41e983cd7e42f3fe4a0db"} Jul 29 19:54:27.102967 ip-10-0-131-93 kubenswrapper[2564]: I0729 19:54:27.102947 2564 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-inf-b-1d9a27-openshift-default-6f55cd5b96-2sdpw" event={"ID":"63de2e3f-49bc-4d59-87c0-1bf702c71b5c","Type":"ContainerStarted","Data":"73b53189f8bf2d73366cfd0c5d177f196d5217aaf52770c6db1daf2e148cad29"} Jul 29 19:54:27.124571 ip-10-0-131-93 kubenswrapper[2564]: I0729 19:54:27.124518 2564 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-ingress/e2e-inf-b-1d9a27-openshift-default-6f55cd5b96-2sdpw" podStartSLOduration=1.124501771 podStartE2EDuration="1.124501771s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-29 19:54:26 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-29 19:54:27.12157982 +0000 UTC m=+2242.845639511" watchObservedRunningTime="2026-07-29 19:54:27.124501771 +0000 UTC m=+2242.848561462" Jul 29 19:54:27.797127 ip-10-0-131-93 kubenswrapper[2564]: I0729 19:54:27.797096 2564 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="openshift-ingress/e2e-inf-b-1d9a27-openshift-default-6f55cd5b96-2sdpw" Jul 29 19:54:28.802088 ip-10-0-131-93 kubenswrapper[2564]: I0729 19:54:28.802061 2564 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="openshift-ingress/e2e-inf-b-1d9a27-openshift-default-6f55cd5b96-2sdpw" Jul 29 19:54:29.112187 ip-10-0-131-93 kubenswrapper[2564]: I0729 19:54:29.112106 2564 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/e2e-inf-b-1d9a27-openshift-default-6f55cd5b96-2sdpw" Jul 29 19:54:29.113101 ip-10-0-131-93 kubenswrapper[2564]: I0729 19:54:29.113083 2564 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/e2e-inf-b-1d9a27-openshift-default-6f55cd5b96-2sdpw" Jul 29 19:54:29.128657 ip-10-0-131-93 kubenswrapper[2564]: I0729 19:54:29.128630 2564 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-444jx_faa8c546-c75b-4e10-93b4-661aa2b5f53e/konnectivity-agent/0.log" limit=1024 Jul 29 19:54:29.229527 ip-10-0-131-93 kubenswrapper[2564]: I0729 19:54:29.229484 2564 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-444jx_faa8c546-c75b-4e10-93b4-661aa2b5f53e/konnectivity-agent/0.log" limit=1024 Jul 29 19:54:30.237474 ip-10-0-131-93 kubenswrapper[2564]: I0729 19:54:30.237439 2564 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-444jx_faa8c546-c75b-4e10-93b4-661aa2b5f53e/konnectivity-agent/0.log" limit=1024 Jul 29 19:54:30.336029 ip-10-0-131-93 kubenswrapper[2564]: I0729 19:54:30.335989 2564 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-444jx_faa8c546-c75b-4e10-93b4-661aa2b5f53e/konnectivity-agent/0.log" limit=1024 Jul 29 19:54:30.872930 ip-10-0-131-93 kubenswrapper[2564]: I0729 19:54:30.872900 2564 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-444jx_faa8c546-c75b-4e10-93b4-661aa2b5f53e/konnectivity-agent/0.log" limit=1024 Jul 29 19:54:30.973531 ip-10-0-131-93 kubenswrapper[2564]: I0729 19:54:30.973503 2564 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-444jx_faa8c546-c75b-4e10-93b4-661aa2b5f53e/konnectivity-agent/0.log" limit=1024 Jul 29 19:54:44.113326 ip-10-0-131-93 kubenswrapper[2564]: I0729 19:54:44.113286 2564 patch_prober.go:28] interesting pod/e2e-inf-b-1d9a27-openshift-default-6f55cd5b96-2sdpw container/istio-proxy namespace/openshift-ingress: Readiness probe status=failure output="Get \"http://10.132.0.44:15021/healthz/ready\": dial tcp 10.132.0.44:15021: connect: connection refused" start-of-body= Jul 29 19:54:44.113751 ip-10-0-131-93 kubenswrapper[2564]: I0729 19:54:44.113342 2564 prober.go:120] "Probe failed" probeType="Readiness" pod="openshift-ingress/e2e-inf-b-1d9a27-openshift-default-6f55cd5b96-2sdpw" podUID="63de2e3f-49bc-4d59-87c0-1bf702c71b5c" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.132.0.44:15021/healthz/ready\": dial tcp 10.132.0.44:15021: connect: connection refused" Jul 29 19:54:44.195236 ip-10-0-131-93 kubenswrapper[2564]: I0729 19:54:44.195205 2564 generic.go:356] "Generic (PLEG): container finished" podID="63de2e3f-49bc-4d59-87c0-1bf702c71b5c" containerID="fa4a25d2fabfaaa9b80e2819b00a9dfc08d05c8903e41e983cd7e42f3fe4a0db" exitCode=0 Jul 29 19:54:44.195421 ip-10-0-131-93 kubenswrapper[2564]: I0729 19:54:44.195257 2564 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-inf-b-1d9a27-openshift-default-6f55cd5b96-2sdpw" event={"ID":"63de2e3f-49bc-4d59-87c0-1bf702c71b5c","Type":"ContainerDied","Data":"fa4a25d2fabfaaa9b80e2819b00a9dfc08d05c8903e41e983cd7e42f3fe4a0db"} Jul 29 19:54:44.195675 ip-10-0-131-93 kubenswrapper[2564]: I0729 19:54:44.195660 2564 scope.go:117] "RemoveContainer" containerID="fa4a25d2fabfaaa9b80e2819b00a9dfc08d05c8903e41e983cd7e42f3fe4a0db" Jul 29 19:54:44.196723 ip-10-0-131-93 kubenswrapper[2564]: I0729 19:54:44.196693 2564 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236224Ki","pods":"250"} Jul 29 19:54:44.196797 ip-10-0-131-93 kubenswrapper[2564]: I0729 19:54:44.196784 2564 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236224Ki","pods":"250"} Jul 29 19:54:44.196839 ip-10-0-131-93 kubenswrapper[2564]: I0729 19:54:44.196818 2564 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236224Ki","pods":"250"} Jul 29 19:54:45.203460 ip-10-0-131-93 kubenswrapper[2564]: I0729 19:54:45.203420 2564 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-inf-b-1d9a27-openshift-default-6f55cd5b96-2sdpw" event={"ID":"63de2e3f-49bc-4d59-87c0-1bf702c71b5c","Type":"ContainerStarted","Data":"f3ce9cacdd9f85898dc78a17f5751818ac209b10309e48c7b76a2d427eafbb23"} Jul 29 19:54:45.797608 ip-10-0-131-93 kubenswrapper[2564]: I0729 19:54:45.797570 2564 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="openshift-ingress/e2e-inf-b-1d9a27-openshift-default-6f55cd5b96-2sdpw" Jul 29 19:54:45.797808 ip-10-0-131-93 kubenswrapper[2564]: I0729 19:54:45.797697 2564 patch_prober.go:28] interesting pod/e2e-inf-b-1d9a27-openshift-default-6f55cd5b96-2sdpw container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.132.0.44:15021/healthz/ready\": dial tcp 10.132.0.44:15021: connect: connection refused" start-of-body= Jul 29 19:54:45.797808 ip-10-0-131-93 kubenswrapper[2564]: I0729 19:54:45.797749 2564 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-inf-b-1d9a27-openshift-default-6f55cd5b96-2sdpw" podUID="63de2e3f-49bc-4d59-87c0-1bf702c71b5c" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.132.0.44:15021/healthz/ready\": dial tcp 10.132.0.44:15021: connect: connection refused" Jul 29 19:54:46.797496 ip-10-0-131-93 kubenswrapper[2564]: I0729 19:54:46.797454 2564 patch_prober.go:28] interesting pod/e2e-inf-b-1d9a27-openshift-default-6f55cd5b96-2sdpw container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.132.0.44:15021/healthz/ready\": dial tcp 10.132.0.44:15021: connect: connection refused" start-of-body= Jul 29 19:54:46.798004 ip-10-0-131-93 kubenswrapper[2564]: I0729 19:54:46.797518 2564 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-inf-b-1d9a27-openshift-default-6f55cd5b96-2sdpw" podUID="63de2e3f-49bc-4d59-87c0-1bf702c71b5c" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.132.0.44:15021/healthz/ready\": dial tcp 10.132.0.44:15021: connect: connection refused" Jul 29 19:54:47.797307 ip-10-0-131-93 kubenswrapper[2564]: I0729 19:54:47.797264 2564 patch_prober.go:28] interesting pod/e2e-inf-b-1d9a27-openshift-default-6f55cd5b96-2sdpw container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.132.0.44:15021/healthz/ready\": dial tcp 10.132.0.44:15021: connect: connection refused" start-of-body= Jul 29 19:54:47.797612 ip-10-0-131-93 kubenswrapper[2564]: I0729 19:54:47.797578 2564 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-inf-b-1d9a27-openshift-default-6f55cd5b96-2sdpw" podUID="63de2e3f-49bc-4d59-87c0-1bf702c71b5c" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.132.0.44:15021/healthz/ready\": dial tcp 10.132.0.44:15021: connect: connection refused" Jul 29 19:54:48.797676 ip-10-0-131-93 kubenswrapper[2564]: I0729 19:54:48.797638 2564 patch_prober.go:28] interesting pod/e2e-inf-b-1d9a27-openshift-default-6f55cd5b96-2sdpw container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.132.0.44:15021/healthz/ready\": dial tcp 10.132.0.44:15021: connect: connection refused" start-of-body= Jul 29 19:54:48.798264 ip-10-0-131-93 kubenswrapper[2564]: I0729 19:54:48.798228 2564 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-inf-b-1d9a27-openshift-default-6f55cd5b96-2sdpw" podUID="63de2e3f-49bc-4d59-87c0-1bf702c71b5c" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.132.0.44:15021/healthz/ready\": dial tcp 10.132.0.44:15021: connect: connection refused" Jul 29 19:54:49.800916 ip-10-0-131-93 kubenswrapper[2564]: I0729 19:54:49.800885 2564 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="openshift-ingress/e2e-inf-b-1d9a27-openshift-default-6f55cd5b96-2sdpw" Jul 29 19:54:49.801319 ip-10-0-131-93 kubenswrapper[2564]: I0729 19:54:49.801247 2564 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/e2e-inf-b-1d9a27-openshift-default-6f55cd5b96-2sdpw" Jul 29 19:54:49.801985 ip-10-0-131-93 kubenswrapper[2564]: I0729 19:54:49.801966 2564 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/e2e-inf-b-1d9a27-openshift-default-6f55cd5b96-2sdpw" Jul 29 19:54:53.995102 ip-10-0-131-93 kubenswrapper[2564]: I0729 19:54:53.995068 2564 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-444jx_faa8c546-c75b-4e10-93b4-661aa2b5f53e/konnectivity-agent/0.log" limit=1024 Jul 29 19:55:00.177890 ip-10-0-131-93 kubenswrapper[2564]: I0729 19:55:00.177854 2564 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-444jx_faa8c546-c75b-4e10-93b4-661aa2b5f53e/konnectivity-agent/0.log" limit=1024 Jul 29 19:55:24.085646 ip-10-0-131-93 kubenswrapper[2564]: I0729 19:55:24.085612 2564 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-444jx_faa8c546-c75b-4e10-93b4-661aa2b5f53e/konnectivity-agent/0.log" limit=1024 Jul 29 19:55:27.604015 ip-10-0-131-93 kubenswrapper[2564]: I0729 19:55:27.603977 2564 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-444jx_faa8c546-c75b-4e10-93b4-661aa2b5f53e/konnectivity-agent/0.log" limit=1024 Jul 29 19:55:44.276103 ip-10-0-131-93 kubenswrapper[2564]: I0729 19:55:44.276055 2564 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-444jx_faa8c546-c75b-4e10-93b4-661aa2b5f53e/konnectivity-agent/0.log" limit=1024 Jul 29 19:56:34.133796 ip-10-0-131-93 kubenswrapper[2564]: I0729 19:56:34.133763 2564 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-444jx_faa8c546-c75b-4e10-93b4-661aa2b5f53e/konnectivity-agent/0.log" limit=1024 Jul 29 19:56:35.528620 ip-10-0-131-93 kubenswrapper[2564]: I0729 19:56:35.528588 2564 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-444jx_faa8c546-c75b-4e10-93b4-661aa2b5f53e/konnectivity-agent/0.log" limit=1024 Jul 29 19:56:45.574736 ip-10-0-131-93 kubenswrapper[2564]: I0729 19:56:45.574695 2564 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-444jx_faa8c546-c75b-4e10-93b4-661aa2b5f53e/konnectivity-agent/0.log" limit=1024 Jul 29 19:56:45.674917 ip-10-0-131-93 kubenswrapper[2564]: I0729 19:56:45.674881 2564 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-444jx_faa8c546-c75b-4e10-93b4-661aa2b5f53e/konnectivity-agent/0.log" limit=1024 Jul 29 19:56:45.772194 ip-10-0-131-93 kubenswrapper[2564]: I0729 19:56:45.772149 2564 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-444jx_faa8c546-c75b-4e10-93b4-661aa2b5f53e/konnectivity-agent/0.log" limit=1024 Jul 29 19:56:45.843291 ip-10-0-131-93 kubenswrapper[2564]: I0729 19:56:45.843209 2564 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-444jx_faa8c546-c75b-4e10-93b4-661aa2b5f53e/konnectivity-agent/0.log" limit=1024 Jul 29 19:56:46.176833 ip-10-0-131-93 kubenswrapper[2564]: I0729 19:56:46.176717 2564 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-444jx_faa8c546-c75b-4e10-93b4-661aa2b5f53e/konnectivity-agent/0.log" limit=1024 Jul 29 19:56:52.181419 ip-10-0-131-93 kubenswrapper[2564]: I0729 19:56:52.181385 2564 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-444jx_faa8c546-c75b-4e10-93b4-661aa2b5f53e/konnectivity-agent/0.log" limit=1024 Jul 29 19:56:52.282732 ip-10-0-131-93 kubenswrapper[2564]: I0729 19:56:52.282702 2564 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-444jx_faa8c546-c75b-4e10-93b4-661aa2b5f53e/konnectivity-agent/0.log" limit=1024 Jul 29 19:56:52.429909 ip-10-0-131-93 kubenswrapper[2564]: I0729 19:56:52.429874 2564 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-ingress/e2e-inf-a-06a5f2-openshift-default-7446cc6b8b-8k7x4"] Jul 29 19:56:52.430310 ip-10-0-131-93 kubenswrapper[2564]: I0729 19:56:52.430278 2564 kuberuntime_container.go:873] "Killing container with a grace period" pod="openshift-ingress/e2e-inf-a-06a5f2-openshift-default-7446cc6b8b-8k7x4" podUID="56703a5f-13f3-4b32-8249-e6e9009667d3" containerName="istio-proxy" containerID="cri-o://6f391610d94b1d2a514d648e48c64fcee38327dc75a1ff66642bd9896c9f73d7" gracePeriod=30 Jul 29 19:56:52.444580 ip-10-0-131-93 kubenswrapper[2564]: I0729 19:56:52.444543 2564 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-444jx_faa8c546-c75b-4e10-93b4-661aa2b5f53e/konnectivity-agent/0.log" limit=1024 Jul 29 19:56:52.680337 ip-10-0-131-93 kubenswrapper[2564]: I0729 19:56:52.680302 2564 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-444jx_faa8c546-c75b-4e10-93b4-661aa2b5f53e/konnectivity-agent/0.log" limit=1024 Jul 29 19:56:57.685033 ip-10-0-131-93 kubenswrapper[2564]: I0729 19:56:57.685011 2564 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-inf-a-06a5f2-openshift-default-7446cc6b8b-8k7x4" Jul 29 19:56:57.847145 ip-10-0-131-93 kubenswrapper[2564]: I0729 19:56:57.847064 2564 generic.go:356] "Generic (PLEG): container finished" podID="56703a5f-13f3-4b32-8249-e6e9009667d3" containerID="6f391610d94b1d2a514d648e48c64fcee38327dc75a1ff66642bd9896c9f73d7" exitCode=0 Jul 29 19:56:57.847145 ip-10-0-131-93 kubenswrapper[2564]: I0729 19:56:57.847109 2564 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-inf-a-06a5f2-openshift-default-7446cc6b8b-8k7x4" Jul 29 19:56:57.847328 ip-10-0-131-93 kubenswrapper[2564]: I0729 19:56:57.847149 2564 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-inf-a-06a5f2-openshift-default-7446cc6b8b-8k7x4" event={"ID":"56703a5f-13f3-4b32-8249-e6e9009667d3","Type":"ContainerDied","Data":"6f391610d94b1d2a514d648e48c64fcee38327dc75a1ff66642bd9896c9f73d7"} Jul 29 19:56:57.847328 ip-10-0-131-93 kubenswrapper[2564]: I0729 19:56:57.847203 2564 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-inf-a-06a5f2-openshift-default-7446cc6b8b-8k7x4" event={"ID":"56703a5f-13f3-4b32-8249-e6e9009667d3","Type":"ContainerDied","Data":"08111a830e116046620cb84e79e00e72add56e4ce0134693e0ad0373b276623e"} Jul 29 19:56:57.847328 ip-10-0-131-93 kubenswrapper[2564]: I0729 19:56:57.847228 2564 scope.go:117] "RemoveContainer" containerID="6f391610d94b1d2a514d648e48c64fcee38327dc75a1ff66642bd9896c9f73d7" Jul 29 19:56:57.855779 ip-10-0-131-93 kubenswrapper[2564]: I0729 19:56:57.855764 2564 scope.go:117] "RemoveContainer" containerID="5f118590bbade0c9af83f430f31bfb5a07c6967f13df304e3db055f4cead30b0" Jul 29 19:56:57.859903 ip-10-0-131-93 kubenswrapper[2564]: I0729 19:56:57.859882 2564 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/56703a5f-13f3-4b32-8249-e6e9009667d3-istio-envoy\") pod \"56703a5f-13f3-4b32-8249-e6e9009667d3\" (UID: \"56703a5f-13f3-4b32-8249-e6e9009667d3\") " Jul 29 19:56:57.860015 ip-10-0-131-93 kubenswrapper[2564]: I0729 19:56:57.859920 2564 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/56703a5f-13f3-4b32-8249-e6e9009667d3-istiod-ca-cert\") pod \"56703a5f-13f3-4b32-8249-e6e9009667d3\" (UID: \"56703a5f-13f3-4b32-8249-e6e9009667d3\") " Jul 29 19:56:57.860015 ip-10-0-131-93 kubenswrapper[2564]: I0729 19:56:57.859941 2564 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/56703a5f-13f3-4b32-8249-e6e9009667d3-workload-socket\") pod \"56703a5f-13f3-4b32-8249-e6e9009667d3\" (UID: \"56703a5f-13f3-4b32-8249-e6e9009667d3\") " Jul 29 19:56:57.860015 ip-10-0-131-93 kubenswrapper[2564]: I0729 19:56:57.859981 2564 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/56703a5f-13f3-4b32-8249-e6e9009667d3-credential-socket\") pod \"56703a5f-13f3-4b32-8249-e6e9009667d3\" (UID: \"56703a5f-13f3-4b32-8249-e6e9009667d3\") " Jul 29 19:56:57.860015 ip-10-0-131-93 kubenswrapper[2564]: I0729 19:56:57.859998 2564 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-jbpj6\" (UniqueName: \"kubernetes.io/projected/56703a5f-13f3-4b32-8249-e6e9009667d3-kube-api-access-jbpj6\") pod \"56703a5f-13f3-4b32-8249-e6e9009667d3\" (UID: \"56703a5f-13f3-4b32-8249-e6e9009667d3\") " Jul 29 19:56:57.860220 ip-10-0-131-93 kubenswrapper[2564]: I0729 19:56:57.860083 2564 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/56703a5f-13f3-4b32-8249-e6e9009667d3-workload-certs\") pod \"56703a5f-13f3-4b32-8249-e6e9009667d3\" (UID: \"56703a5f-13f3-4b32-8249-e6e9009667d3\") " Jul 29 19:56:57.860220 ip-10-0-131-93 kubenswrapper[2564]: I0729 19:56:57.860122 2564 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/56703a5f-13f3-4b32-8249-e6e9009667d3-istio-podinfo\") pod \"56703a5f-13f3-4b32-8249-e6e9009667d3\" (UID: \"56703a5f-13f3-4b32-8249-e6e9009667d3\") " Jul 29 19:56:57.860220 ip-10-0-131-93 kubenswrapper[2564]: I0729 19:56:57.860172 2564 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/56703a5f-13f3-4b32-8249-e6e9009667d3-istio-token\") pod \"56703a5f-13f3-4b32-8249-e6e9009667d3\" (UID: \"56703a5f-13f3-4b32-8249-e6e9009667d3\") " Jul 29 19:56:57.860220 ip-10-0-131-93 kubenswrapper[2564]: I0729 19:56:57.860212 2564 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/56703a5f-13f3-4b32-8249-e6e9009667d3-istio-data\") pod \"56703a5f-13f3-4b32-8249-e6e9009667d3\" (UID: \"56703a5f-13f3-4b32-8249-e6e9009667d3\") " Jul 29 19:56:57.860470 ip-10-0-131-93 kubenswrapper[2564]: I0729 19:56:57.860272 2564 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/56703a5f-13f3-4b32-8249-e6e9009667d3-istiod-ca-cert" (OuterVolumeSpecName: "istiod-ca-cert") pod "56703a5f-13f3-4b32-8249-e6e9009667d3" (UID: "56703a5f-13f3-4b32-8249-e6e9009667d3"). InnerVolumeSpecName "istiod-ca-cert". PluginName "kubernetes.io/configmap", VolumeGIDValue "" Jul 29 19:56:57.860470 ip-10-0-131-93 kubenswrapper[2564]: I0729 19:56:57.860364 2564 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/56703a5f-13f3-4b32-8249-e6e9009667d3-workload-certs" (OuterVolumeSpecName: "workload-certs") pod "56703a5f-13f3-4b32-8249-e6e9009667d3" (UID: "56703a5f-13f3-4b32-8249-e6e9009667d3"). InnerVolumeSpecName "workload-certs". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Jul 29 19:56:57.860574 ip-10-0-131-93 kubenswrapper[2564]: I0729 19:56:57.860502 2564 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/56703a5f-13f3-4b32-8249-e6e9009667d3-workload-socket" (OuterVolumeSpecName: "workload-socket") pod "56703a5f-13f3-4b32-8249-e6e9009667d3" (UID: "56703a5f-13f3-4b32-8249-e6e9009667d3"). InnerVolumeSpecName "workload-socket". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Jul 29 19:56:57.860670 ip-10-0-131-93 kubenswrapper[2564]: I0729 19:56:57.860641 2564 reconciler_common.go:299] "Volume detached for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/56703a5f-13f3-4b32-8249-e6e9009667d3-workload-certs\") on node \"ip-10-0-131-93.ec2.internal\" DevicePath \"\"" Jul 29 19:56:57.860670 ip-10-0-131-93 kubenswrapper[2564]: I0729 19:56:57.860663 2564 reconciler_common.go:299] "Volume detached for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/56703a5f-13f3-4b32-8249-e6e9009667d3-istiod-ca-cert\") on node \"ip-10-0-131-93.ec2.internal\" DevicePath \"\"" Jul 29 19:56:57.860850 ip-10-0-131-93 kubenswrapper[2564]: I0729 19:56:57.860675 2564 reconciler_common.go:299] "Volume detached for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/56703a5f-13f3-4b32-8249-e6e9009667d3-workload-socket\") on node \"ip-10-0-131-93.ec2.internal\" DevicePath \"\"" Jul 29 19:56:57.860850 ip-10-0-131-93 kubenswrapper[2564]: I0729 19:56:57.860726 2564 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/56703a5f-13f3-4b32-8249-e6e9009667d3-credential-socket" (OuterVolumeSpecName: "credential-socket") pod "56703a5f-13f3-4b32-8249-e6e9009667d3" (UID: "56703a5f-13f3-4b32-8249-e6e9009667d3"). InnerVolumeSpecName "credential-socket". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Jul 29 19:56:57.861093 ip-10-0-131-93 kubenswrapper[2564]: I0729 19:56:57.861075 2564 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/56703a5f-13f3-4b32-8249-e6e9009667d3-istio-data" (OuterVolumeSpecName: "istio-data") pod "56703a5f-13f3-4b32-8249-e6e9009667d3" (UID: "56703a5f-13f3-4b32-8249-e6e9009667d3"). InnerVolumeSpecName "istio-data". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Jul 29 19:56:57.862292 ip-10-0-131-93 kubenswrapper[2564]: I0729 19:56:57.862264 2564 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/56703a5f-13f3-4b32-8249-e6e9009667d3-kube-api-access-jbpj6" (OuterVolumeSpecName: "kube-api-access-jbpj6") pod "56703a5f-13f3-4b32-8249-e6e9009667d3" (UID: "56703a5f-13f3-4b32-8249-e6e9009667d3"). InnerVolumeSpecName "kube-api-access-jbpj6". PluginName "kubernetes.io/projected", VolumeGIDValue "" Jul 29 19:56:57.862892 ip-10-0-131-93 kubenswrapper[2564]: I0729 19:56:57.862866 2564 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/downward-api/56703a5f-13f3-4b32-8249-e6e9009667d3-istio-podinfo" (OuterVolumeSpecName: "istio-podinfo") pod "56703a5f-13f3-4b32-8249-e6e9009667d3" (UID: "56703a5f-13f3-4b32-8249-e6e9009667d3"). InnerVolumeSpecName "istio-podinfo". PluginName "kubernetes.io/downward-api", VolumeGIDValue "" Jul 29 19:56:57.862892 ip-10-0-131-93 kubenswrapper[2564]: I0729 19:56:57.862877 2564 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/56703a5f-13f3-4b32-8249-e6e9009667d3-istio-envoy" (OuterVolumeSpecName: "istio-envoy") pod "56703a5f-13f3-4b32-8249-e6e9009667d3" (UID: "56703a5f-13f3-4b32-8249-e6e9009667d3"). InnerVolumeSpecName "istio-envoy". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Jul 29 19:56:57.862892 ip-10-0-131-93 kubenswrapper[2564]: I0729 19:56:57.862873 2564 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/56703a5f-13f3-4b32-8249-e6e9009667d3-istio-token" (OuterVolumeSpecName: "istio-token") pod "56703a5f-13f3-4b32-8249-e6e9009667d3" (UID: "56703a5f-13f3-4b32-8249-e6e9009667d3"). InnerVolumeSpecName "istio-token". PluginName "kubernetes.io/projected", VolumeGIDValue "" Jul 29 19:56:57.864269 ip-10-0-131-93 kubenswrapper[2564]: I0729 19:56:57.864256 2564 scope.go:117] "RemoveContainer" containerID="6f391610d94b1d2a514d648e48c64fcee38327dc75a1ff66642bd9896c9f73d7" Jul 29 19:56:57.864584 ip-10-0-131-93 kubenswrapper[2564]: E0729 19:56:57.864562 2564 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"6f391610d94b1d2a514d648e48c64fcee38327dc75a1ff66642bd9896c9f73d7\": container with ID starting with 6f391610d94b1d2a514d648e48c64fcee38327dc75a1ff66642bd9896c9f73d7 not found: ID does not exist" containerID="6f391610d94b1d2a514d648e48c64fcee38327dc75a1ff66642bd9896c9f73d7" Jul 29 19:56:57.864647 ip-10-0-131-93 kubenswrapper[2564]: I0729 19:56:57.864596 2564 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"6f391610d94b1d2a514d648e48c64fcee38327dc75a1ff66642bd9896c9f73d7"} err="failed to get container status \"6f391610d94b1d2a514d648e48c64fcee38327dc75a1ff66642bd9896c9f73d7\": rpc error: code = NotFound desc = could not find container \"6f391610d94b1d2a514d648e48c64fcee38327dc75a1ff66642bd9896c9f73d7\": container with ID starting with 6f391610d94b1d2a514d648e48c64fcee38327dc75a1ff66642bd9896c9f73d7 not found: ID does not exist" Jul 29 19:56:57.864647 ip-10-0-131-93 kubenswrapper[2564]: I0729 19:56:57.864623 2564 scope.go:117] "RemoveContainer" containerID="5f118590bbade0c9af83f430f31bfb5a07c6967f13df304e3db055f4cead30b0" Jul 29 19:56:57.864861 ip-10-0-131-93 kubenswrapper[2564]: E0729 19:56:57.864844 2564 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"5f118590bbade0c9af83f430f31bfb5a07c6967f13df304e3db055f4cead30b0\": container with ID starting with 5f118590bbade0c9af83f430f31bfb5a07c6967f13df304e3db055f4cead30b0 not found: ID does not exist" containerID="5f118590bbade0c9af83f430f31bfb5a07c6967f13df304e3db055f4cead30b0" Jul 29 19:56:57.864905 ip-10-0-131-93 kubenswrapper[2564]: I0729 19:56:57.864868 2564 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"5f118590bbade0c9af83f430f31bfb5a07c6967f13df304e3db055f4cead30b0"} err="failed to get container status \"5f118590bbade0c9af83f430f31bfb5a07c6967f13df304e3db055f4cead30b0\": rpc error: code = NotFound desc = could not find container \"5f118590bbade0c9af83f430f31bfb5a07c6967f13df304e3db055f4cead30b0\": container with ID starting with 5f118590bbade0c9af83f430f31bfb5a07c6967f13df304e3db055f4cead30b0 not found: ID does not exist" Jul 29 19:56:57.961713 ip-10-0-131-93 kubenswrapper[2564]: I0729 19:56:57.961673 2564 reconciler_common.go:299] "Volume detached for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/56703a5f-13f3-4b32-8249-e6e9009667d3-istio-data\") on node \"ip-10-0-131-93.ec2.internal\" DevicePath \"\"" Jul 29 19:56:57.961713 ip-10-0-131-93 kubenswrapper[2564]: I0729 19:56:57.961710 2564 reconciler_common.go:299] "Volume detached for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/56703a5f-13f3-4b32-8249-e6e9009667d3-istio-envoy\") on node \"ip-10-0-131-93.ec2.internal\" DevicePath \"\"" Jul 29 19:56:57.961968 ip-10-0-131-93 kubenswrapper[2564]: I0729 19:56:57.961725 2564 reconciler_common.go:299] "Volume detached for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/56703a5f-13f3-4b32-8249-e6e9009667d3-credential-socket\") on node \"ip-10-0-131-93.ec2.internal\" DevicePath \"\"" Jul 29 19:56:57.961968 ip-10-0-131-93 kubenswrapper[2564]: I0729 19:56:57.961741 2564 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-jbpj6\" (UniqueName: \"kubernetes.io/projected/56703a5f-13f3-4b32-8249-e6e9009667d3-kube-api-access-jbpj6\") on node \"ip-10-0-131-93.ec2.internal\" DevicePath \"\"" Jul 29 19:56:57.961968 ip-10-0-131-93 kubenswrapper[2564]: I0729 19:56:57.961756 2564 reconciler_common.go:299] "Volume detached for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/56703a5f-13f3-4b32-8249-e6e9009667d3-istio-podinfo\") on node \"ip-10-0-131-93.ec2.internal\" DevicePath \"\"" Jul 29 19:56:57.961968 ip-10-0-131-93 kubenswrapper[2564]: I0729 19:56:57.961768 2564 reconciler_common.go:299] "Volume detached for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/56703a5f-13f3-4b32-8249-e6e9009667d3-istio-token\") on node \"ip-10-0-131-93.ec2.internal\" DevicePath \"\"" Jul 29 19:56:58.170067 ip-10-0-131-93 kubenswrapper[2564]: I0729 19:56:58.170037 2564 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-ingress/e2e-inf-a-06a5f2-openshift-default-7446cc6b8b-8k7x4"] Jul 29 19:56:58.177257 ip-10-0-131-93 kubenswrapper[2564]: I0729 19:56:58.177231 2564 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["openshift-ingress/e2e-inf-a-06a5f2-openshift-default-7446cc6b8b-8k7x4"] Jul 29 19:56:58.784214 ip-10-0-131-93 kubenswrapper[2564]: I0729 19:56:58.784179 2564 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="56703a5f-13f3-4b32-8249-e6e9009667d3" path="/var/lib/kubelet/pods/56703a5f-13f3-4b32-8249-e6e9009667d3/volumes" Jul 29 19:57:03.344437 ip-10-0-131-93 kubenswrapper[2564]: I0729 19:57:03.344403 2564 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-444jx_faa8c546-c75b-4e10-93b4-661aa2b5f53e/konnectivity-agent/0.log" limit=1024 Jul 29 19:57:03.419246 ip-10-0-131-93 kubenswrapper[2564]: I0729 19:57:03.419211 2564 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-444jx_faa8c546-c75b-4e10-93b4-661aa2b5f53e/konnectivity-agent/0.log" limit=1024 Jul 29 19:57:03.495771 ip-10-0-131-93 kubenswrapper[2564]: I0729 19:57:03.495737 2564 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-444jx_faa8c546-c75b-4e10-93b4-661aa2b5f53e/konnectivity-agent/0.log" limit=1024 Jul 29 19:57:03.666415 ip-10-0-131-93 kubenswrapper[2564]: I0729 19:57:03.666317 2564 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-444jx_faa8c546-c75b-4e10-93b4-661aa2b5f53e/konnectivity-agent/0.log" limit=1024 Jul 29 19:57:03.933304 ip-10-0-131-93 kubenswrapper[2564]: I0729 19:57:03.933214 2564 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-444jx_faa8c546-c75b-4e10-93b4-661aa2b5f53e/konnectivity-agent/0.log" limit=1024 Jul 29 19:57:09.925141 ip-10-0-131-93 kubenswrapper[2564]: I0729 19:57:09.925110 2564 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-444jx_faa8c546-c75b-4e10-93b4-661aa2b5f53e/konnectivity-agent/0.log" limit=1024 Jul 29 19:57:09.995487 ip-10-0-131-93 kubenswrapper[2564]: I0729 19:57:09.995438 2564 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-444jx_faa8c546-c75b-4e10-93b4-661aa2b5f53e/konnectivity-agent/0.log" limit=1024 Jul 29 19:57:10.168318 ip-10-0-131-93 kubenswrapper[2564]: I0729 19:57:10.168286 2564 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-ingress/e2e-inf-b-1d9a27-openshift-default-6f55cd5b96-2sdpw"] Jul 29 19:57:10.168590 ip-10-0-131-93 kubenswrapper[2564]: I0729 19:57:10.168568 2564 kuberuntime_container.go:873] "Killing container with a grace period" pod="openshift-ingress/e2e-inf-b-1d9a27-openshift-default-6f55cd5b96-2sdpw" podUID="63de2e3f-49bc-4d59-87c0-1bf702c71b5c" containerName="istio-proxy" containerID="cri-o://f3ce9cacdd9f85898dc78a17f5751818ac209b10309e48c7b76a2d427eafbb23" gracePeriod=30 Jul 29 19:57:10.197976 ip-10-0-131-93 kubenswrapper[2564]: I0729 19:57:10.197946 2564 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-444jx_faa8c546-c75b-4e10-93b4-661aa2b5f53e/konnectivity-agent/0.log" limit=1024 Jul 29 19:57:10.388034 ip-10-0-131-93 kubenswrapper[2564]: I0729 19:57:10.388004 2564 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-444jx_faa8c546-c75b-4e10-93b4-661aa2b5f53e/konnectivity-agent/0.log" limit=1024 Jul 29 19:57:10.718562 ip-10-0-131-93 kubenswrapper[2564]: I0729 19:57:10.718534 2564 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-444jx_faa8c546-c75b-4e10-93b4-661aa2b5f53e/konnectivity-agent/0.log" limit=1024 Jul 29 19:57:11.016970 ip-10-0-131-93 kubenswrapper[2564]: I0729 19:57:11.016887 2564 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-ingress/e2e-api-a-466674-openshift-default-6ff787f978-2qrj8"] Jul 29 19:57:11.017435 ip-10-0-131-93 kubenswrapper[2564]: I0729 19:57:11.017359 2564 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="56703a5f-13f3-4b32-8249-e6e9009667d3" containerName="istio-proxy" Jul 29 19:57:11.017435 ip-10-0-131-93 kubenswrapper[2564]: I0729 19:57:11.017406 2564 state_mem.go:107] "Deleted CPUSet assignment" podUID="56703a5f-13f3-4b32-8249-e6e9009667d3" containerName="istio-proxy" Jul 29 19:57:11.017603 ip-10-0-131-93 kubenswrapper[2564]: I0729 19:57:11.017519 2564 memory_manager.go:368] "RemoveStaleState removing state" podUID="56703a5f-13f3-4b32-8249-e6e9009667d3" containerName="istio-proxy" Jul 29 19:57:11.020191 ip-10-0-131-93 kubenswrapper[2564]: I0729 19:57:11.020171 2564 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-api-a-466674-openshift-default-6ff787f978-2qrj8" Jul 29 19:57:11.022493 ip-10-0-131-93 kubenswrapper[2564]: I0729 19:57:11.022467 2564 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress\"/\"e2e-api-a-466674-openshift-default-dockercfg-vj6k5\"" Jul 29 19:57:11.035516 ip-10-0-131-93 kubenswrapper[2564]: I0729 19:57:11.035494 2564 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/e2e-api-a-466674-openshift-default-6ff787f978-2qrj8"] Jul 29 19:57:11.040335 ip-10-0-131-93 kubenswrapper[2564]: I0729 19:57:11.040305 2564 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-444jx_faa8c546-c75b-4e10-93b4-661aa2b5f53e/konnectivity-agent/0.log" limit=1024 Jul 29 19:57:11.069258 ip-10-0-131-93 kubenswrapper[2564]: I0729 19:57:11.069229 2564 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-p7k46\" (UniqueName: \"kubernetes.io/projected/fa1763e9-97b8-40c5-9d3e-457961d94202-kube-api-access-p7k46\") pod \"e2e-api-a-466674-openshift-default-6ff787f978-2qrj8\" (UID: \"fa1763e9-97b8-40c5-9d3e-457961d94202\") " pod="openshift-ingress/e2e-api-a-466674-openshift-default-6ff787f978-2qrj8" Jul 29 19:57:11.069414 ip-10-0-131-93 kubenswrapper[2564]: I0729 19:57:11.069279 2564 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/fa1763e9-97b8-40c5-9d3e-457961d94202-istio-envoy\") pod \"e2e-api-a-466674-openshift-default-6ff787f978-2qrj8\" (UID: \"fa1763e9-97b8-40c5-9d3e-457961d94202\") " pod="openshift-ingress/e2e-api-a-466674-openshift-default-6ff787f978-2qrj8" Jul 29 19:57:11.069414 ip-10-0-131-93 kubenswrapper[2564]: I0729 19:57:11.069332 2564 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/fa1763e9-97b8-40c5-9d3e-457961d94202-credential-socket\") pod \"e2e-api-a-466674-openshift-default-6ff787f978-2qrj8\" (UID: \"fa1763e9-97b8-40c5-9d3e-457961d94202\") " pod="openshift-ingress/e2e-api-a-466674-openshift-default-6ff787f978-2qrj8" Jul 29 19:57:11.069414 ip-10-0-131-93 kubenswrapper[2564]: I0729 19:57:11.069355 2564 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/fa1763e9-97b8-40c5-9d3e-457961d94202-workload-certs\") pod \"e2e-api-a-466674-openshift-default-6ff787f978-2qrj8\" (UID: \"fa1763e9-97b8-40c5-9d3e-457961d94202\") " pod="openshift-ingress/e2e-api-a-466674-openshift-default-6ff787f978-2qrj8" Jul 29 19:57:11.069519 ip-10-0-131-93 kubenswrapper[2564]: I0729 19:57:11.069428 2564 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/fa1763e9-97b8-40c5-9d3e-457961d94202-istiod-ca-cert\") pod \"e2e-api-a-466674-openshift-default-6ff787f978-2qrj8\" (UID: \"fa1763e9-97b8-40c5-9d3e-457961d94202\") " pod="openshift-ingress/e2e-api-a-466674-openshift-default-6ff787f978-2qrj8" Jul 29 19:57:11.069519 ip-10-0-131-93 kubenswrapper[2564]: I0729 19:57:11.069504 2564 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/fa1763e9-97b8-40c5-9d3e-457961d94202-workload-socket\") pod \"e2e-api-a-466674-openshift-default-6ff787f978-2qrj8\" (UID: \"fa1763e9-97b8-40c5-9d3e-457961d94202\") " pod="openshift-ingress/e2e-api-a-466674-openshift-default-6ff787f978-2qrj8" Jul 29 19:57:11.069587 ip-10-0-131-93 kubenswrapper[2564]: I0729 19:57:11.069530 2564 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/fa1763e9-97b8-40c5-9d3e-457961d94202-istio-token\") pod \"e2e-api-a-466674-openshift-default-6ff787f978-2qrj8\" (UID: \"fa1763e9-97b8-40c5-9d3e-457961d94202\") " pod="openshift-ingress/e2e-api-a-466674-openshift-default-6ff787f978-2qrj8" Jul 29 19:57:11.069587 ip-10-0-131-93 kubenswrapper[2564]: I0729 19:57:11.069551 2564 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/fa1763e9-97b8-40c5-9d3e-457961d94202-istio-podinfo\") pod \"e2e-api-a-466674-openshift-default-6ff787f978-2qrj8\" (UID: \"fa1763e9-97b8-40c5-9d3e-457961d94202\") " pod="openshift-ingress/e2e-api-a-466674-openshift-default-6ff787f978-2qrj8" Jul 29 19:57:11.069587 ip-10-0-131-93 kubenswrapper[2564]: I0729 19:57:11.069582 2564 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/fa1763e9-97b8-40c5-9d3e-457961d94202-istio-data\") pod \"e2e-api-a-466674-openshift-default-6ff787f978-2qrj8\" (UID: \"fa1763e9-97b8-40c5-9d3e-457961d94202\") " pod="openshift-ingress/e2e-api-a-466674-openshift-default-6ff787f978-2qrj8" Jul 29 19:57:11.170344 ip-10-0-131-93 kubenswrapper[2564]: I0729 19:57:11.170303 2564 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/fa1763e9-97b8-40c5-9d3e-457961d94202-istio-data\") pod \"e2e-api-a-466674-openshift-default-6ff787f978-2qrj8\" (UID: \"fa1763e9-97b8-40c5-9d3e-457961d94202\") " pod="openshift-ingress/e2e-api-a-466674-openshift-default-6ff787f978-2qrj8" Jul 29 19:57:11.170344 ip-10-0-131-93 kubenswrapper[2564]: I0729 19:57:11.170352 2564 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-p7k46\" (UniqueName: \"kubernetes.io/projected/fa1763e9-97b8-40c5-9d3e-457961d94202-kube-api-access-p7k46\") pod \"e2e-api-a-466674-openshift-default-6ff787f978-2qrj8\" (UID: \"fa1763e9-97b8-40c5-9d3e-457961d94202\") " pod="openshift-ingress/e2e-api-a-466674-openshift-default-6ff787f978-2qrj8" Jul 29 19:57:11.170650 ip-10-0-131-93 kubenswrapper[2564]: I0729 19:57:11.170394 2564 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/fa1763e9-97b8-40c5-9d3e-457961d94202-istio-envoy\") pod \"e2e-api-a-466674-openshift-default-6ff787f978-2qrj8\" (UID: \"fa1763e9-97b8-40c5-9d3e-457961d94202\") " pod="openshift-ingress/e2e-api-a-466674-openshift-default-6ff787f978-2qrj8" Jul 29 19:57:11.170650 ip-10-0-131-93 kubenswrapper[2564]: I0729 19:57:11.170412 2564 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/fa1763e9-97b8-40c5-9d3e-457961d94202-credential-socket\") pod \"e2e-api-a-466674-openshift-default-6ff787f978-2qrj8\" (UID: \"fa1763e9-97b8-40c5-9d3e-457961d94202\") " pod="openshift-ingress/e2e-api-a-466674-openshift-default-6ff787f978-2qrj8" Jul 29 19:57:11.170650 ip-10-0-131-93 kubenswrapper[2564]: I0729 19:57:11.170434 2564 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/fa1763e9-97b8-40c5-9d3e-457961d94202-workload-certs\") pod \"e2e-api-a-466674-openshift-default-6ff787f978-2qrj8\" (UID: \"fa1763e9-97b8-40c5-9d3e-457961d94202\") " pod="openshift-ingress/e2e-api-a-466674-openshift-default-6ff787f978-2qrj8" Jul 29 19:57:11.170650 ip-10-0-131-93 kubenswrapper[2564]: I0729 19:57:11.170460 2564 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/fa1763e9-97b8-40c5-9d3e-457961d94202-istiod-ca-cert\") pod \"e2e-api-a-466674-openshift-default-6ff787f978-2qrj8\" (UID: \"fa1763e9-97b8-40c5-9d3e-457961d94202\") " pod="openshift-ingress/e2e-api-a-466674-openshift-default-6ff787f978-2qrj8" Jul 29 19:57:11.170650 ip-10-0-131-93 kubenswrapper[2564]: I0729 19:57:11.170495 2564 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/fa1763e9-97b8-40c5-9d3e-457961d94202-workload-socket\") pod \"e2e-api-a-466674-openshift-default-6ff787f978-2qrj8\" (UID: \"fa1763e9-97b8-40c5-9d3e-457961d94202\") " pod="openshift-ingress/e2e-api-a-466674-openshift-default-6ff787f978-2qrj8" Jul 29 19:57:11.170650 ip-10-0-131-93 kubenswrapper[2564]: I0729 19:57:11.170529 2564 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/fa1763e9-97b8-40c5-9d3e-457961d94202-istio-token\") pod \"e2e-api-a-466674-openshift-default-6ff787f978-2qrj8\" (UID: \"fa1763e9-97b8-40c5-9d3e-457961d94202\") " pod="openshift-ingress/e2e-api-a-466674-openshift-default-6ff787f978-2qrj8" Jul 29 19:57:11.170650 ip-10-0-131-93 kubenswrapper[2564]: I0729 19:57:11.170555 2564 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/fa1763e9-97b8-40c5-9d3e-457961d94202-istio-podinfo\") pod \"e2e-api-a-466674-openshift-default-6ff787f978-2qrj8\" (UID: \"fa1763e9-97b8-40c5-9d3e-457961d94202\") " pod="openshift-ingress/e2e-api-a-466674-openshift-default-6ff787f978-2qrj8" Jul 29 19:57:11.171068 ip-10-0-131-93 kubenswrapper[2564]: I0729 19:57:11.170847 2564 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/fa1763e9-97b8-40c5-9d3e-457961d94202-credential-socket\") pod \"e2e-api-a-466674-openshift-default-6ff787f978-2qrj8\" (UID: \"fa1763e9-97b8-40c5-9d3e-457961d94202\") " pod="openshift-ingress/e2e-api-a-466674-openshift-default-6ff787f978-2qrj8" Jul 29 19:57:11.171068 ip-10-0-131-93 kubenswrapper[2564]: I0729 19:57:11.170847 2564 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/fa1763e9-97b8-40c5-9d3e-457961d94202-istio-data\") pod \"e2e-api-a-466674-openshift-default-6ff787f978-2qrj8\" (UID: \"fa1763e9-97b8-40c5-9d3e-457961d94202\") " pod="openshift-ingress/e2e-api-a-466674-openshift-default-6ff787f978-2qrj8" Jul 29 19:57:11.171068 ip-10-0-131-93 kubenswrapper[2564]: I0729 19:57:11.170947 2564 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/fa1763e9-97b8-40c5-9d3e-457961d94202-workload-certs\") pod \"e2e-api-a-466674-openshift-default-6ff787f978-2qrj8\" (UID: \"fa1763e9-97b8-40c5-9d3e-457961d94202\") " pod="openshift-ingress/e2e-api-a-466674-openshift-default-6ff787f978-2qrj8" Jul 29 19:57:11.171168 ip-10-0-131-93 kubenswrapper[2564]: I0729 19:57:11.171127 2564 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/fa1763e9-97b8-40c5-9d3e-457961d94202-workload-socket\") pod \"e2e-api-a-466674-openshift-default-6ff787f978-2qrj8\" (UID: \"fa1763e9-97b8-40c5-9d3e-457961d94202\") " pod="openshift-ingress/e2e-api-a-466674-openshift-default-6ff787f978-2qrj8" Jul 29 19:57:11.171263 ip-10-0-131-93 kubenswrapper[2564]: I0729 19:57:11.171239 2564 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/fa1763e9-97b8-40c5-9d3e-457961d94202-istiod-ca-cert\") pod \"e2e-api-a-466674-openshift-default-6ff787f978-2qrj8\" (UID: \"fa1763e9-97b8-40c5-9d3e-457961d94202\") " pod="openshift-ingress/e2e-api-a-466674-openshift-default-6ff787f978-2qrj8" Jul 29 19:57:11.172819 ip-10-0-131-93 kubenswrapper[2564]: I0729 19:57:11.172783 2564 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/fa1763e9-97b8-40c5-9d3e-457961d94202-istio-envoy\") pod \"e2e-api-a-466674-openshift-default-6ff787f978-2qrj8\" (UID: \"fa1763e9-97b8-40c5-9d3e-457961d94202\") " pod="openshift-ingress/e2e-api-a-466674-openshift-default-6ff787f978-2qrj8" Jul 29 19:57:11.173034 ip-10-0-131-93 kubenswrapper[2564]: I0729 19:57:11.173017 2564 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/fa1763e9-97b8-40c5-9d3e-457961d94202-istio-podinfo\") pod \"e2e-api-a-466674-openshift-default-6ff787f978-2qrj8\" (UID: \"fa1763e9-97b8-40c5-9d3e-457961d94202\") " pod="openshift-ingress/e2e-api-a-466674-openshift-default-6ff787f978-2qrj8" Jul 29 19:57:11.181430 ip-10-0-131-93 kubenswrapper[2564]: I0729 19:57:11.181354 2564 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/fa1763e9-97b8-40c5-9d3e-457961d94202-istio-token\") pod \"e2e-api-a-466674-openshift-default-6ff787f978-2qrj8\" (UID: \"fa1763e9-97b8-40c5-9d3e-457961d94202\") " pod="openshift-ingress/e2e-api-a-466674-openshift-default-6ff787f978-2qrj8" Jul 29 19:57:11.181802 ip-10-0-131-93 kubenswrapper[2564]: I0729 19:57:11.181786 2564 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-p7k46\" (UniqueName: \"kubernetes.io/projected/fa1763e9-97b8-40c5-9d3e-457961d94202-kube-api-access-p7k46\") pod \"e2e-api-a-466674-openshift-default-6ff787f978-2qrj8\" (UID: \"fa1763e9-97b8-40c5-9d3e-457961d94202\") " pod="openshift-ingress/e2e-api-a-466674-openshift-default-6ff787f978-2qrj8" Jul 29 19:57:11.330658 ip-10-0-131-93 kubenswrapper[2564]: I0729 19:57:11.330566 2564 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-api-a-466674-openshift-default-6ff787f978-2qrj8" Jul 29 19:57:11.342891 ip-10-0-131-93 kubenswrapper[2564]: I0729 19:57:11.342858 2564 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-444jx_faa8c546-c75b-4e10-93b4-661aa2b5f53e/konnectivity-agent/0.log" limit=1024 Jul 29 19:57:11.471593 ip-10-0-131-93 kubenswrapper[2564]: I0729 19:57:11.471567 2564 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/e2e-api-a-466674-openshift-default-6ff787f978-2qrj8"] Jul 29 19:57:11.472293 ip-10-0-131-93 kubenswrapper[2564]: W0729 19:57:11.472265 2564 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podfa1763e9_97b8_40c5_9d3e_457961d94202.slice/crio-75cb023e03ef9f45217f3b0fd0954dd616ea5ec7d269c355b9c63c08fced79e3 WatchSource:0}: Error finding container 75cb023e03ef9f45217f3b0fd0954dd616ea5ec7d269c355b9c63c08fced79e3: Status 404 returned error can't find the container with id 75cb023e03ef9f45217f3b0fd0954dd616ea5ec7d269c355b9c63c08fced79e3 Jul 29 19:57:11.474565 ip-10-0-131-93 kubenswrapper[2564]: I0729 19:57:11.474531 2564 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236224Ki","pods":"250"} Jul 29 19:57:11.474648 ip-10-0-131-93 kubenswrapper[2564]: I0729 19:57:11.474598 2564 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236224Ki","pods":"250"} Jul 29 19:57:11.474648 ip-10-0-131-93 kubenswrapper[2564]: I0729 19:57:11.474627 2564 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236224Ki","pods":"250"} Jul 29 19:57:11.919288 ip-10-0-131-93 kubenswrapper[2564]: I0729 19:57:11.919205 2564 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-api-a-466674-openshift-default-6ff787f978-2qrj8" event={"ID":"fa1763e9-97b8-40c5-9d3e-457961d94202","Type":"ContainerStarted","Data":"a0e35d719e24ca5945630169280aab7b60024de89df48b489b41b239097545d1"} Jul 29 19:57:11.919288 ip-10-0-131-93 kubenswrapper[2564]: I0729 19:57:11.919272 2564 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-api-a-466674-openshift-default-6ff787f978-2qrj8" event={"ID":"fa1763e9-97b8-40c5-9d3e-457961d94202","Type":"ContainerStarted","Data":"75cb023e03ef9f45217f3b0fd0954dd616ea5ec7d269c355b9c63c08fced79e3"} Jul 29 19:57:11.943071 ip-10-0-131-93 kubenswrapper[2564]: I0729 19:57:11.943022 2564 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-ingress/e2e-api-a-466674-openshift-default-6ff787f978-2qrj8" podStartSLOduration=1.94300513 podStartE2EDuration="1.94300513s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-29 19:57:10 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-29 19:57:11.941736681 +0000 UTC m=+2407.665796375" watchObservedRunningTime="2026-07-29 19:57:11.94300513 +0000 UTC m=+2407.667064822" Jul 29 19:57:12.331551 ip-10-0-131-93 kubenswrapper[2564]: I0729 19:57:12.331511 2564 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="openshift-ingress/e2e-api-a-466674-openshift-default-6ff787f978-2qrj8" Jul 29 19:57:12.336519 ip-10-0-131-93 kubenswrapper[2564]: I0729 19:57:12.336493 2564 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="openshift-ingress/e2e-api-a-466674-openshift-default-6ff787f978-2qrj8" Jul 29 19:57:12.925664 ip-10-0-131-93 kubenswrapper[2564]: I0729 19:57:12.925630 2564 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/e2e-api-a-466674-openshift-default-6ff787f978-2qrj8" Jul 29 19:57:12.926516 ip-10-0-131-93 kubenswrapper[2564]: I0729 19:57:12.926491 2564 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/e2e-api-a-466674-openshift-default-6ff787f978-2qrj8" Jul 29 19:57:13.795872 ip-10-0-131-93 kubenswrapper[2564]: I0729 19:57:13.795826 2564 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-444jx_faa8c546-c75b-4e10-93b4-661aa2b5f53e/konnectivity-agent/0.log" limit=1024 Jul 29 19:57:13.893186 ip-10-0-131-93 kubenswrapper[2564]: I0729 19:57:13.893148 2564 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-444jx_faa8c546-c75b-4e10-93b4-661aa2b5f53e/konnectivity-agent/0.log" limit=1024 Jul 29 19:57:14.923996 ip-10-0-131-93 kubenswrapper[2564]: I0729 19:57:14.923962 2564 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-444jx_faa8c546-c75b-4e10-93b4-661aa2b5f53e/konnectivity-agent/0.log" limit=1024 Jul 29 19:57:15.024840 ip-10-0-131-93 kubenswrapper[2564]: I0729 19:57:15.024790 2564 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-444jx_faa8c546-c75b-4e10-93b4-661aa2b5f53e/konnectivity-agent/0.log" limit=1024 Jul 29 19:57:15.429178 ip-10-0-131-93 kubenswrapper[2564]: I0729 19:57:15.429114 2564 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-inf-b-1d9a27-openshift-default-6f55cd5b96-2sdpw" Jul 29 19:57:15.507852 ip-10-0-131-93 kubenswrapper[2564]: I0729 19:57:15.507820 2564 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/63de2e3f-49bc-4d59-87c0-1bf702c71b5c-istio-podinfo\") pod \"63de2e3f-49bc-4d59-87c0-1bf702c71b5c\" (UID: \"63de2e3f-49bc-4d59-87c0-1bf702c71b5c\") " Jul 29 19:57:15.507852 ip-10-0-131-93 kubenswrapper[2564]: I0729 19:57:15.507854 2564 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/63de2e3f-49bc-4d59-87c0-1bf702c71b5c-workload-certs\") pod \"63de2e3f-49bc-4d59-87c0-1bf702c71b5c\" (UID: \"63de2e3f-49bc-4d59-87c0-1bf702c71b5c\") " Jul 29 19:57:15.508132 ip-10-0-131-93 kubenswrapper[2564]: I0729 19:57:15.507879 2564 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/63de2e3f-49bc-4d59-87c0-1bf702c71b5c-istio-token\") pod \"63de2e3f-49bc-4d59-87c0-1bf702c71b5c\" (UID: \"63de2e3f-49bc-4d59-87c0-1bf702c71b5c\") " Jul 29 19:57:15.508132 ip-10-0-131-93 kubenswrapper[2564]: I0729 19:57:15.507905 2564 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/63de2e3f-49bc-4d59-87c0-1bf702c71b5c-istio-data\") pod \"63de2e3f-49bc-4d59-87c0-1bf702c71b5c\" (UID: \"63de2e3f-49bc-4d59-87c0-1bf702c71b5c\") " Jul 29 19:57:15.508132 ip-10-0-131-93 kubenswrapper[2564]: I0729 19:57:15.507930 2564 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/63de2e3f-49bc-4d59-87c0-1bf702c71b5c-workload-socket\") pod \"63de2e3f-49bc-4d59-87c0-1bf702c71b5c\" (UID: \"63de2e3f-49bc-4d59-87c0-1bf702c71b5c\") " Jul 29 19:57:15.508132 ip-10-0-131-93 kubenswrapper[2564]: I0729 19:57:15.507990 2564 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/63de2e3f-49bc-4d59-87c0-1bf702c71b5c-credential-socket\") pod \"63de2e3f-49bc-4d59-87c0-1bf702c71b5c\" (UID: \"63de2e3f-49bc-4d59-87c0-1bf702c71b5c\") " Jul 29 19:57:15.508132 ip-10-0-131-93 kubenswrapper[2564]: I0729 19:57:15.508037 2564 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/63de2e3f-49bc-4d59-87c0-1bf702c71b5c-istio-envoy\") pod \"63de2e3f-49bc-4d59-87c0-1bf702c71b5c\" (UID: \"63de2e3f-49bc-4d59-87c0-1bf702c71b5c\") " Jul 29 19:57:15.508132 ip-10-0-131-93 kubenswrapper[2564]: I0729 19:57:15.508074 2564 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/63de2e3f-49bc-4d59-87c0-1bf702c71b5c-istiod-ca-cert\") pod \"63de2e3f-49bc-4d59-87c0-1bf702c71b5c\" (UID: \"63de2e3f-49bc-4d59-87c0-1bf702c71b5c\") " Jul 29 19:57:15.508132 ip-10-0-131-93 kubenswrapper[2564]: I0729 19:57:15.508115 2564 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-2qs9x\" (UniqueName: \"kubernetes.io/projected/63de2e3f-49bc-4d59-87c0-1bf702c71b5c-kube-api-access-2qs9x\") pod \"63de2e3f-49bc-4d59-87c0-1bf702c71b5c\" (UID: \"63de2e3f-49bc-4d59-87c0-1bf702c71b5c\") " Jul 29 19:57:15.508580 ip-10-0-131-93 kubenswrapper[2564]: I0729 19:57:15.508197 2564 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/63de2e3f-49bc-4d59-87c0-1bf702c71b5c-workload-certs" (OuterVolumeSpecName: "workload-certs") pod "63de2e3f-49bc-4d59-87c0-1bf702c71b5c" (UID: "63de2e3f-49bc-4d59-87c0-1bf702c71b5c"). InnerVolumeSpecName "workload-certs". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Jul 29 19:57:15.508580 ip-10-0-131-93 kubenswrapper[2564]: I0729 19:57:15.508509 2564 reconciler_common.go:299] "Volume detached for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/63de2e3f-49bc-4d59-87c0-1bf702c71b5c-workload-certs\") on node \"ip-10-0-131-93.ec2.internal\" DevicePath \"\"" Jul 29 19:57:15.508580 ip-10-0-131-93 kubenswrapper[2564]: I0729 19:57:15.508569 2564 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/63de2e3f-49bc-4d59-87c0-1bf702c71b5c-credential-socket" (OuterVolumeSpecName: "credential-socket") pod "63de2e3f-49bc-4d59-87c0-1bf702c71b5c" (UID: "63de2e3f-49bc-4d59-87c0-1bf702c71b5c"). InnerVolumeSpecName "credential-socket". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Jul 29 19:57:15.508732 ip-10-0-131-93 kubenswrapper[2564]: I0729 19:57:15.508616 2564 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/63de2e3f-49bc-4d59-87c0-1bf702c71b5c-istiod-ca-cert" (OuterVolumeSpecName: "istiod-ca-cert") pod "63de2e3f-49bc-4d59-87c0-1bf702c71b5c" (UID: "63de2e3f-49bc-4d59-87c0-1bf702c71b5c"). InnerVolumeSpecName "istiod-ca-cert". PluginName "kubernetes.io/configmap", VolumeGIDValue "" Jul 29 19:57:15.509860 ip-10-0-131-93 kubenswrapper[2564]: I0729 19:57:15.509330 2564 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/63de2e3f-49bc-4d59-87c0-1bf702c71b5c-workload-socket" (OuterVolumeSpecName: "workload-socket") pod "63de2e3f-49bc-4d59-87c0-1bf702c71b5c" (UID: "63de2e3f-49bc-4d59-87c0-1bf702c71b5c"). InnerVolumeSpecName "workload-socket". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Jul 29 19:57:15.509860 ip-10-0-131-93 kubenswrapper[2564]: I0729 19:57:15.509457 2564 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/63de2e3f-49bc-4d59-87c0-1bf702c71b5c-istio-data" (OuterVolumeSpecName: "istio-data") pod "63de2e3f-49bc-4d59-87c0-1bf702c71b5c" (UID: "63de2e3f-49bc-4d59-87c0-1bf702c71b5c"). InnerVolumeSpecName "istio-data". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Jul 29 19:57:15.510452 ip-10-0-131-93 kubenswrapper[2564]: I0729 19:57:15.510418 2564 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/downward-api/63de2e3f-49bc-4d59-87c0-1bf702c71b5c-istio-podinfo" (OuterVolumeSpecName: "istio-podinfo") pod "63de2e3f-49bc-4d59-87c0-1bf702c71b5c" (UID: "63de2e3f-49bc-4d59-87c0-1bf702c71b5c"). InnerVolumeSpecName "istio-podinfo". PluginName "kubernetes.io/downward-api", VolumeGIDValue "" Jul 29 19:57:15.510857 ip-10-0-131-93 kubenswrapper[2564]: I0729 19:57:15.510832 2564 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/63de2e3f-49bc-4d59-87c0-1bf702c71b5c-kube-api-access-2qs9x" (OuterVolumeSpecName: "kube-api-access-2qs9x") pod "63de2e3f-49bc-4d59-87c0-1bf702c71b5c" (UID: "63de2e3f-49bc-4d59-87c0-1bf702c71b5c"). InnerVolumeSpecName "kube-api-access-2qs9x". PluginName "kubernetes.io/projected", VolumeGIDValue "" Jul 29 19:57:15.511165 ip-10-0-131-93 kubenswrapper[2564]: I0729 19:57:15.511138 2564 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/63de2e3f-49bc-4d59-87c0-1bf702c71b5c-istio-token" (OuterVolumeSpecName: "istio-token") pod "63de2e3f-49bc-4d59-87c0-1bf702c71b5c" (UID: "63de2e3f-49bc-4d59-87c0-1bf702c71b5c"). InnerVolumeSpecName "istio-token". PluginName "kubernetes.io/projected", VolumeGIDValue "" Jul 29 19:57:15.511351 ip-10-0-131-93 kubenswrapper[2564]: I0729 19:57:15.511321 2564 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/63de2e3f-49bc-4d59-87c0-1bf702c71b5c-istio-envoy" (OuterVolumeSpecName: "istio-envoy") pod "63de2e3f-49bc-4d59-87c0-1bf702c71b5c" (UID: "63de2e3f-49bc-4d59-87c0-1bf702c71b5c"). InnerVolumeSpecName "istio-envoy". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Jul 29 19:57:15.535012 ip-10-0-131-93 kubenswrapper[2564]: I0729 19:57:15.534992 2564 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-444jx_faa8c546-c75b-4e10-93b4-661aa2b5f53e/konnectivity-agent/0.log" limit=1024 Jul 29 19:57:15.609065 ip-10-0-131-93 kubenswrapper[2564]: I0729 19:57:15.609028 2564 reconciler_common.go:299] "Volume detached for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/63de2e3f-49bc-4d59-87c0-1bf702c71b5c-istio-podinfo\") on node \"ip-10-0-131-93.ec2.internal\" DevicePath \"\"" Jul 29 19:57:15.609065 ip-10-0-131-93 kubenswrapper[2564]: I0729 19:57:15.609055 2564 reconciler_common.go:299] "Volume detached for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/63de2e3f-49bc-4d59-87c0-1bf702c71b5c-istio-token\") on node \"ip-10-0-131-93.ec2.internal\" DevicePath \"\"" Jul 29 19:57:15.609065 ip-10-0-131-93 kubenswrapper[2564]: I0729 19:57:15.609063 2564 reconciler_common.go:299] "Volume detached for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/63de2e3f-49bc-4d59-87c0-1bf702c71b5c-istio-data\") on node \"ip-10-0-131-93.ec2.internal\" DevicePath \"\"" Jul 29 19:57:15.609065 ip-10-0-131-93 kubenswrapper[2564]: I0729 19:57:15.609074 2564 reconciler_common.go:299] "Volume detached for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/63de2e3f-49bc-4d59-87c0-1bf702c71b5c-workload-socket\") on node \"ip-10-0-131-93.ec2.internal\" DevicePath \"\"" Jul 29 19:57:15.609323 ip-10-0-131-93 kubenswrapper[2564]: I0729 19:57:15.609082 2564 reconciler_common.go:299] "Volume detached for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/63de2e3f-49bc-4d59-87c0-1bf702c71b5c-credential-socket\") on node \"ip-10-0-131-93.ec2.internal\" DevicePath \"\"" Jul 29 19:57:15.609323 ip-10-0-131-93 kubenswrapper[2564]: I0729 19:57:15.609091 2564 reconciler_common.go:299] "Volume detached for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/63de2e3f-49bc-4d59-87c0-1bf702c71b5c-istio-envoy\") on node \"ip-10-0-131-93.ec2.internal\" DevicePath \"\"" Jul 29 19:57:15.609323 ip-10-0-131-93 kubenswrapper[2564]: I0729 19:57:15.609098 2564 reconciler_common.go:299] "Volume detached for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/63de2e3f-49bc-4d59-87c0-1bf702c71b5c-istiod-ca-cert\") on node \"ip-10-0-131-93.ec2.internal\" DevicePath \"\"" Jul 29 19:57:15.609323 ip-10-0-131-93 kubenswrapper[2564]: I0729 19:57:15.609107 2564 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-2qs9x\" (UniqueName: \"kubernetes.io/projected/63de2e3f-49bc-4d59-87c0-1bf702c71b5c-kube-api-access-2qs9x\") on node \"ip-10-0-131-93.ec2.internal\" DevicePath \"\"" Jul 29 19:57:15.635462 ip-10-0-131-93 kubenswrapper[2564]: I0729 19:57:15.635425 2564 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-444jx_faa8c546-c75b-4e10-93b4-661aa2b5f53e/konnectivity-agent/0.log" limit=1024 Jul 29 19:57:15.941279 ip-10-0-131-93 kubenswrapper[2564]: I0729 19:57:15.941246 2564 generic.go:356] "Generic (PLEG): container finished" podID="63de2e3f-49bc-4d59-87c0-1bf702c71b5c" containerID="f3ce9cacdd9f85898dc78a17f5751818ac209b10309e48c7b76a2d427eafbb23" exitCode=0 Jul 29 19:57:15.941791 ip-10-0-131-93 kubenswrapper[2564]: I0729 19:57:15.941305 2564 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-inf-b-1d9a27-openshift-default-6f55cd5b96-2sdpw" Jul 29 19:57:15.941791 ip-10-0-131-93 kubenswrapper[2564]: I0729 19:57:15.941305 2564 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-inf-b-1d9a27-openshift-default-6f55cd5b96-2sdpw" event={"ID":"63de2e3f-49bc-4d59-87c0-1bf702c71b5c","Type":"ContainerDied","Data":"f3ce9cacdd9f85898dc78a17f5751818ac209b10309e48c7b76a2d427eafbb23"} Jul 29 19:57:15.941791 ip-10-0-131-93 kubenswrapper[2564]: I0729 19:57:15.941434 2564 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-inf-b-1d9a27-openshift-default-6f55cd5b96-2sdpw" event={"ID":"63de2e3f-49bc-4d59-87c0-1bf702c71b5c","Type":"ContainerDied","Data":"73b53189f8bf2d73366cfd0c5d177f196d5217aaf52770c6db1daf2e148cad29"} Jul 29 19:57:15.941791 ip-10-0-131-93 kubenswrapper[2564]: I0729 19:57:15.941462 2564 scope.go:117] "RemoveContainer" containerID="f3ce9cacdd9f85898dc78a17f5751818ac209b10309e48c7b76a2d427eafbb23" Jul 29 19:57:15.951280 ip-10-0-131-93 kubenswrapper[2564]: I0729 19:57:15.951263 2564 scope.go:117] "RemoveContainer" containerID="fa4a25d2fabfaaa9b80e2819b00a9dfc08d05c8903e41e983cd7e42f3fe4a0db" Jul 29 19:57:15.959215 ip-10-0-131-93 kubenswrapper[2564]: I0729 19:57:15.959198 2564 scope.go:117] "RemoveContainer" containerID="f3ce9cacdd9f85898dc78a17f5751818ac209b10309e48c7b76a2d427eafbb23" Jul 29 19:57:15.959480 ip-10-0-131-93 kubenswrapper[2564]: E0729 19:57:15.959461 2564 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"f3ce9cacdd9f85898dc78a17f5751818ac209b10309e48c7b76a2d427eafbb23\": container with ID starting with f3ce9cacdd9f85898dc78a17f5751818ac209b10309e48c7b76a2d427eafbb23 not found: ID does not exist" containerID="f3ce9cacdd9f85898dc78a17f5751818ac209b10309e48c7b76a2d427eafbb23" Jul 29 19:57:15.959531 ip-10-0-131-93 kubenswrapper[2564]: I0729 19:57:15.959489 2564 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"f3ce9cacdd9f85898dc78a17f5751818ac209b10309e48c7b76a2d427eafbb23"} err="failed to get container status \"f3ce9cacdd9f85898dc78a17f5751818ac209b10309e48c7b76a2d427eafbb23\": rpc error: code = NotFound desc = could not find container \"f3ce9cacdd9f85898dc78a17f5751818ac209b10309e48c7b76a2d427eafbb23\": container with ID starting with f3ce9cacdd9f85898dc78a17f5751818ac209b10309e48c7b76a2d427eafbb23 not found: ID does not exist" Jul 29 19:57:15.959531 ip-10-0-131-93 kubenswrapper[2564]: I0729 19:57:15.959506 2564 scope.go:117] "RemoveContainer" containerID="fa4a25d2fabfaaa9b80e2819b00a9dfc08d05c8903e41e983cd7e42f3fe4a0db" Jul 29 19:57:15.959720 ip-10-0-131-93 kubenswrapper[2564]: E0729 19:57:15.959699 2564 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"fa4a25d2fabfaaa9b80e2819b00a9dfc08d05c8903e41e983cd7e42f3fe4a0db\": container with ID starting with fa4a25d2fabfaaa9b80e2819b00a9dfc08d05c8903e41e983cd7e42f3fe4a0db not found: ID does not exist" containerID="fa4a25d2fabfaaa9b80e2819b00a9dfc08d05c8903e41e983cd7e42f3fe4a0db" Jul 29 19:57:15.959775 ip-10-0-131-93 kubenswrapper[2564]: I0729 19:57:15.959732 2564 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"fa4a25d2fabfaaa9b80e2819b00a9dfc08d05c8903e41e983cd7e42f3fe4a0db"} err="failed to get container status \"fa4a25d2fabfaaa9b80e2819b00a9dfc08d05c8903e41e983cd7e42f3fe4a0db\": rpc error: code = NotFound desc = could not find container \"fa4a25d2fabfaaa9b80e2819b00a9dfc08d05c8903e41e983cd7e42f3fe4a0db\": container with ID starting with fa4a25d2fabfaaa9b80e2819b00a9dfc08d05c8903e41e983cd7e42f3fe4a0db not found: ID does not exist" Jul 29 19:57:15.963085 ip-10-0-131-93 kubenswrapper[2564]: I0729 19:57:15.963062 2564 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-ingress/e2e-inf-b-1d9a27-openshift-default-6f55cd5b96-2sdpw"] Jul 29 19:57:15.966362 ip-10-0-131-93 kubenswrapper[2564]: I0729 19:57:15.966342 2564 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["openshift-ingress/e2e-inf-b-1d9a27-openshift-default-6f55cd5b96-2sdpw"] Jul 29 19:57:16.783631 ip-10-0-131-93 kubenswrapper[2564]: I0729 19:57:16.783599 2564 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="63de2e3f-49bc-4d59-87c0-1bf702c71b5c" path="/var/lib/kubelet/pods/63de2e3f-49bc-4d59-87c0-1bf702c71b5c/volumes" Jul 29 19:57:28.721479 ip-10-0-131-93 kubenswrapper[2564]: I0729 19:57:28.721171 2564 patch_prober.go:28] interesting pod/e2e-api-a-466674-openshift-default-6ff787f978-2qrj8 container/istio-proxy namespace/openshift-ingress: Readiness probe status=failure output="Get \"http://10.132.0.45:15021/healthz/ready\": read tcp 10.132.0.2:58780->10.132.0.45:15021: read: connection reset by peer" start-of-body= Jul 29 19:57:28.721479 ip-10-0-131-93 kubenswrapper[2564]: I0729 19:57:28.721252 2564 prober.go:120] "Probe failed" probeType="Readiness" pod="openshift-ingress/e2e-api-a-466674-openshift-default-6ff787f978-2qrj8" podUID="fa1763e9-97b8-40c5-9d3e-457961d94202" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.132.0.45:15021/healthz/ready\": read tcp 10.132.0.2:58780->10.132.0.45:15021: read: connection reset by peer" Jul 29 19:57:29.012292 ip-10-0-131-93 kubenswrapper[2564]: I0729 19:57:29.012208 2564 generic.go:356] "Generic (PLEG): container finished" podID="fa1763e9-97b8-40c5-9d3e-457961d94202" containerID="a0e35d719e24ca5945630169280aab7b60024de89df48b489b41b239097545d1" exitCode=0 Jul 29 19:57:29.012292 ip-10-0-131-93 kubenswrapper[2564]: I0729 19:57:29.012256 2564 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-api-a-466674-openshift-default-6ff787f978-2qrj8" event={"ID":"fa1763e9-97b8-40c5-9d3e-457961d94202","Type":"ContainerDied","Data":"a0e35d719e24ca5945630169280aab7b60024de89df48b489b41b239097545d1"} Jul 29 19:57:29.012654 ip-10-0-131-93 kubenswrapper[2564]: I0729 19:57:29.012640 2564 scope.go:117] "RemoveContainer" containerID="a0e35d719e24ca5945630169280aab7b60024de89df48b489b41b239097545d1" Jul 29 19:57:29.013668 ip-10-0-131-93 kubenswrapper[2564]: I0729 19:57:29.013637 2564 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236224Ki","pods":"250"} Jul 29 19:57:29.013773 ip-10-0-131-93 kubenswrapper[2564]: I0729 19:57:29.013706 2564 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236224Ki","pods":"250"} Jul 29 19:57:29.013773 ip-10-0-131-93 kubenswrapper[2564]: I0729 19:57:29.013739 2564 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236224Ki","pods":"250"} Jul 29 19:57:30.018694 ip-10-0-131-93 kubenswrapper[2564]: I0729 19:57:30.018657 2564 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-api-a-466674-openshift-default-6ff787f978-2qrj8" event={"ID":"fa1763e9-97b8-40c5-9d3e-457961d94202","Type":"ContainerStarted","Data":"59c390d46a57b81ecd87f18e8da3fef25e98bc60e394f62d24f39d4bac57a1a5"} Jul 29 19:57:30.331273 ip-10-0-131-93 kubenswrapper[2564]: I0729 19:57:30.331186 2564 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="openshift-ingress/e2e-api-a-466674-openshift-default-6ff787f978-2qrj8" Jul 29 19:57:30.331465 ip-10-0-131-93 kubenswrapper[2564]: I0729 19:57:30.331435 2564 patch_prober.go:28] interesting pod/e2e-api-a-466674-openshift-default-6ff787f978-2qrj8 container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.132.0.45:15021/healthz/ready\": dial tcp 10.132.0.45:15021: connect: connection refused" start-of-body= Jul 29 19:57:30.331547 ip-10-0-131-93 kubenswrapper[2564]: I0729 19:57:30.331489 2564 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-api-a-466674-openshift-default-6ff787f978-2qrj8" podUID="fa1763e9-97b8-40c5-9d3e-457961d94202" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.132.0.45:15021/healthz/ready\": dial tcp 10.132.0.45:15021: connect: connection refused" Jul 29 19:57:31.331682 ip-10-0-131-93 kubenswrapper[2564]: I0729 19:57:31.331640 2564 patch_prober.go:28] interesting pod/e2e-api-a-466674-openshift-default-6ff787f978-2qrj8 container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.132.0.45:15021/healthz/ready\": dial tcp 10.132.0.45:15021: connect: connection refused" start-of-body= Jul 29 19:57:31.332086 ip-10-0-131-93 kubenswrapper[2564]: I0729 19:57:31.331699 2564 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-api-a-466674-openshift-default-6ff787f978-2qrj8" podUID="fa1763e9-97b8-40c5-9d3e-457961d94202" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.132.0.45:15021/healthz/ready\": dial tcp 10.132.0.45:15021: connect: connection refused" Jul 29 19:57:32.331272 ip-10-0-131-93 kubenswrapper[2564]: I0729 19:57:32.331236 2564 patch_prober.go:28] interesting pod/e2e-api-a-466674-openshift-default-6ff787f978-2qrj8 container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.132.0.45:15021/healthz/ready\": dial tcp 10.132.0.45:15021: connect: connection refused" start-of-body= Jul 29 19:57:32.331484 ip-10-0-131-93 kubenswrapper[2564]: I0729 19:57:32.331293 2564 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-api-a-466674-openshift-default-6ff787f978-2qrj8" podUID="fa1763e9-97b8-40c5-9d3e-457961d94202" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.132.0.45:15021/healthz/ready\": dial tcp 10.132.0.45:15021: connect: connection refused" Jul 29 19:57:33.331598 ip-10-0-131-93 kubenswrapper[2564]: I0729 19:57:33.331560 2564 patch_prober.go:28] interesting pod/e2e-api-a-466674-openshift-default-6ff787f978-2qrj8 container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.132.0.45:15021/healthz/ready\": dial tcp 10.132.0.45:15021: connect: connection refused" start-of-body= Jul 29 19:57:33.331986 ip-10-0-131-93 kubenswrapper[2564]: I0729 19:57:33.331616 2564 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-api-a-466674-openshift-default-6ff787f978-2qrj8" podUID="fa1763e9-97b8-40c5-9d3e-457961d94202" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.132.0.45:15021/healthz/ready\": dial tcp 10.132.0.45:15021: connect: connection refused" Jul 29 19:57:33.544342 ip-10-0-131-93 kubenswrapper[2564]: I0729 19:57:33.544308 2564 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-444jx_faa8c546-c75b-4e10-93b4-661aa2b5f53e/konnectivity-agent/0.log" limit=1024 Jul 29 19:57:34.335065 ip-10-0-131-93 kubenswrapper[2564]: I0729 19:57:34.335034 2564 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="openshift-ingress/e2e-api-a-466674-openshift-default-6ff787f978-2qrj8" Jul 29 19:57:34.335455 ip-10-0-131-93 kubenswrapper[2564]: I0729 19:57:34.335428 2564 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/e2e-api-a-466674-openshift-default-6ff787f978-2qrj8" Jul 29 19:57:34.336032 ip-10-0-131-93 kubenswrapper[2564]: I0729 19:57:34.336011 2564 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/e2e-api-a-466674-openshift-default-6ff787f978-2qrj8" Jul 29 19:57:38.739274 ip-10-0-131-93 kubenswrapper[2564]: I0729 19:57:38.739241 2564 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-444jx_faa8c546-c75b-4e10-93b4-661aa2b5f53e/konnectivity-agent/0.log" limit=1024 Jul 29 19:57:39.194476 ip-10-0-131-93 kubenswrapper[2564]: I0729 19:57:39.194449 2564 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-444jx_faa8c546-c75b-4e10-93b4-661aa2b5f53e/konnectivity-agent/0.log" limit=1024 Jul 29 19:57:39.463930 ip-10-0-131-93 kubenswrapper[2564]: I0729 19:57:39.463858 2564 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-ingress/e2e-api-b-ecc3b0-openshift-default-764746cf54-nrvc2"] Jul 29 19:57:39.464234 ip-10-0-131-93 kubenswrapper[2564]: I0729 19:57:39.464209 2564 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="56703a5f-13f3-4b32-8249-e6e9009667d3" containerName="istio-proxy" Jul 29 19:57:39.464234 ip-10-0-131-93 kubenswrapper[2564]: I0729 19:57:39.464232 2564 state_mem.go:107] "Deleted CPUSet assignment" podUID="56703a5f-13f3-4b32-8249-e6e9009667d3" containerName="istio-proxy" Jul 29 19:57:39.464435 ip-10-0-131-93 kubenswrapper[2564]: I0729 19:57:39.464249 2564 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="63de2e3f-49bc-4d59-87c0-1bf702c71b5c" containerName="istio-proxy" Jul 29 19:57:39.464435 ip-10-0-131-93 kubenswrapper[2564]: I0729 19:57:39.464254 2564 state_mem.go:107] "Deleted CPUSet assignment" podUID="63de2e3f-49bc-4d59-87c0-1bf702c71b5c" containerName="istio-proxy" Jul 29 19:57:39.464435 ip-10-0-131-93 kubenswrapper[2564]: I0729 19:57:39.464268 2564 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="63de2e3f-49bc-4d59-87c0-1bf702c71b5c" containerName="istio-proxy" Jul 29 19:57:39.464435 ip-10-0-131-93 kubenswrapper[2564]: I0729 19:57:39.464274 2564 state_mem.go:107] "Deleted CPUSet assignment" podUID="63de2e3f-49bc-4d59-87c0-1bf702c71b5c" containerName="istio-proxy" Jul 29 19:57:39.464435 ip-10-0-131-93 kubenswrapper[2564]: I0729 19:57:39.464335 2564 memory_manager.go:368] "RemoveStaleState removing state" podUID="56703a5f-13f3-4b32-8249-e6e9009667d3" containerName="istio-proxy" Jul 29 19:57:39.464435 ip-10-0-131-93 kubenswrapper[2564]: I0729 19:57:39.464344 2564 memory_manager.go:368] "RemoveStaleState removing state" podUID="63de2e3f-49bc-4d59-87c0-1bf702c71b5c" containerName="istio-proxy" Jul 29 19:57:39.464435 ip-10-0-131-93 kubenswrapper[2564]: I0729 19:57:39.464353 2564 memory_manager.go:368] "RemoveStaleState removing state" podUID="63de2e3f-49bc-4d59-87c0-1bf702c71b5c" containerName="istio-proxy" Jul 29 19:57:39.466712 ip-10-0-131-93 kubenswrapper[2564]: I0729 19:57:39.466691 2564 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-api-b-ecc3b0-openshift-default-764746cf54-nrvc2" Jul 29 19:57:39.469260 ip-10-0-131-93 kubenswrapper[2564]: I0729 19:57:39.469240 2564 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress\"/\"e2e-api-b-ecc3b0-openshift-default-dockercfg-chjsf\"" Jul 29 19:57:39.482492 ip-10-0-131-93 kubenswrapper[2564]: I0729 19:57:39.482472 2564 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/e2e-api-b-ecc3b0-openshift-default-764746cf54-nrvc2"] Jul 29 19:57:39.497996 ip-10-0-131-93 kubenswrapper[2564]: I0729 19:57:39.497946 2564 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-444jx_faa8c546-c75b-4e10-93b4-661aa2b5f53e/konnectivity-agent/0.log" limit=1024 Jul 29 19:57:39.616277 ip-10-0-131-93 kubenswrapper[2564]: I0729 19:57:39.616246 2564 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/f445712e-6292-49cc-a59b-59ee1015edea-istiod-ca-cert\") pod \"e2e-api-b-ecc3b0-openshift-default-764746cf54-nrvc2\" (UID: \"f445712e-6292-49cc-a59b-59ee1015edea\") " pod="openshift-ingress/e2e-api-b-ecc3b0-openshift-default-764746cf54-nrvc2" Jul 29 19:57:39.616495 ip-10-0-131-93 kubenswrapper[2564]: I0729 19:57:39.616308 2564 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/f445712e-6292-49cc-a59b-59ee1015edea-istio-data\") pod \"e2e-api-b-ecc3b0-openshift-default-764746cf54-nrvc2\" (UID: \"f445712e-6292-49cc-a59b-59ee1015edea\") " pod="openshift-ingress/e2e-api-b-ecc3b0-openshift-default-764746cf54-nrvc2" Jul 29 19:57:39.616495 ip-10-0-131-93 kubenswrapper[2564]: I0729 19:57:39.616339 2564 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-p8gtk\" (UniqueName: \"kubernetes.io/projected/f445712e-6292-49cc-a59b-59ee1015edea-kube-api-access-p8gtk\") pod \"e2e-api-b-ecc3b0-openshift-default-764746cf54-nrvc2\" (UID: \"f445712e-6292-49cc-a59b-59ee1015edea\") " pod="openshift-ingress/e2e-api-b-ecc3b0-openshift-default-764746cf54-nrvc2" Jul 29 19:57:39.616495 ip-10-0-131-93 kubenswrapper[2564]: I0729 19:57:39.616367 2564 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/f445712e-6292-49cc-a59b-59ee1015edea-istio-podinfo\") pod \"e2e-api-b-ecc3b0-openshift-default-764746cf54-nrvc2\" (UID: \"f445712e-6292-49cc-a59b-59ee1015edea\") " pod="openshift-ingress/e2e-api-b-ecc3b0-openshift-default-764746cf54-nrvc2" Jul 29 19:57:39.616495 ip-10-0-131-93 kubenswrapper[2564]: I0729 19:57:39.616463 2564 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/f445712e-6292-49cc-a59b-59ee1015edea-credential-socket\") pod \"e2e-api-b-ecc3b0-openshift-default-764746cf54-nrvc2\" (UID: \"f445712e-6292-49cc-a59b-59ee1015edea\") " pod="openshift-ingress/e2e-api-b-ecc3b0-openshift-default-764746cf54-nrvc2" Jul 29 19:57:39.616700 ip-10-0-131-93 kubenswrapper[2564]: I0729 19:57:39.616496 2564 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/f445712e-6292-49cc-a59b-59ee1015edea-istio-envoy\") pod \"e2e-api-b-ecc3b0-openshift-default-764746cf54-nrvc2\" (UID: \"f445712e-6292-49cc-a59b-59ee1015edea\") " pod="openshift-ingress/e2e-api-b-ecc3b0-openshift-default-764746cf54-nrvc2" Jul 29 19:57:39.616700 ip-10-0-131-93 kubenswrapper[2564]: I0729 19:57:39.616539 2564 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/f445712e-6292-49cc-a59b-59ee1015edea-workload-certs\") pod \"e2e-api-b-ecc3b0-openshift-default-764746cf54-nrvc2\" (UID: \"f445712e-6292-49cc-a59b-59ee1015edea\") " pod="openshift-ingress/e2e-api-b-ecc3b0-openshift-default-764746cf54-nrvc2" Jul 29 19:57:39.616700 ip-10-0-131-93 kubenswrapper[2564]: I0729 19:57:39.616571 2564 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/f445712e-6292-49cc-a59b-59ee1015edea-workload-socket\") pod \"e2e-api-b-ecc3b0-openshift-default-764746cf54-nrvc2\" (UID: \"f445712e-6292-49cc-a59b-59ee1015edea\") " pod="openshift-ingress/e2e-api-b-ecc3b0-openshift-default-764746cf54-nrvc2" Jul 29 19:57:39.616700 ip-10-0-131-93 kubenswrapper[2564]: I0729 19:57:39.616618 2564 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/f445712e-6292-49cc-a59b-59ee1015edea-istio-token\") pod \"e2e-api-b-ecc3b0-openshift-default-764746cf54-nrvc2\" (UID: \"f445712e-6292-49cc-a59b-59ee1015edea\") " pod="openshift-ingress/e2e-api-b-ecc3b0-openshift-default-764746cf54-nrvc2" Jul 29 19:57:39.717545 ip-10-0-131-93 kubenswrapper[2564]: I0729 19:57:39.717464 2564 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/f445712e-6292-49cc-a59b-59ee1015edea-istio-data\") pod \"e2e-api-b-ecc3b0-openshift-default-764746cf54-nrvc2\" (UID: \"f445712e-6292-49cc-a59b-59ee1015edea\") " pod="openshift-ingress/e2e-api-b-ecc3b0-openshift-default-764746cf54-nrvc2" Jul 29 19:57:39.717545 ip-10-0-131-93 kubenswrapper[2564]: I0729 19:57:39.717502 2564 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-p8gtk\" (UniqueName: \"kubernetes.io/projected/f445712e-6292-49cc-a59b-59ee1015edea-kube-api-access-p8gtk\") pod \"e2e-api-b-ecc3b0-openshift-default-764746cf54-nrvc2\" (UID: \"f445712e-6292-49cc-a59b-59ee1015edea\") " pod="openshift-ingress/e2e-api-b-ecc3b0-openshift-default-764746cf54-nrvc2" Jul 29 19:57:39.717545 ip-10-0-131-93 kubenswrapper[2564]: I0729 19:57:39.717525 2564 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/f445712e-6292-49cc-a59b-59ee1015edea-istio-podinfo\") pod \"e2e-api-b-ecc3b0-openshift-default-764746cf54-nrvc2\" (UID: \"f445712e-6292-49cc-a59b-59ee1015edea\") " pod="openshift-ingress/e2e-api-b-ecc3b0-openshift-default-764746cf54-nrvc2" Jul 29 19:57:39.717545 ip-10-0-131-93 kubenswrapper[2564]: I0729 19:57:39.717543 2564 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/f445712e-6292-49cc-a59b-59ee1015edea-credential-socket\") pod \"e2e-api-b-ecc3b0-openshift-default-764746cf54-nrvc2\" (UID: \"f445712e-6292-49cc-a59b-59ee1015edea\") " pod="openshift-ingress/e2e-api-b-ecc3b0-openshift-default-764746cf54-nrvc2" Jul 29 19:57:39.717842 ip-10-0-131-93 kubenswrapper[2564]: I0729 19:57:39.717558 2564 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/f445712e-6292-49cc-a59b-59ee1015edea-istio-envoy\") pod \"e2e-api-b-ecc3b0-openshift-default-764746cf54-nrvc2\" (UID: \"f445712e-6292-49cc-a59b-59ee1015edea\") " pod="openshift-ingress/e2e-api-b-ecc3b0-openshift-default-764746cf54-nrvc2" Jul 29 19:57:39.717842 ip-10-0-131-93 kubenswrapper[2564]: I0729 19:57:39.717685 2564 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/f445712e-6292-49cc-a59b-59ee1015edea-workload-certs\") pod \"e2e-api-b-ecc3b0-openshift-default-764746cf54-nrvc2\" (UID: \"f445712e-6292-49cc-a59b-59ee1015edea\") " pod="openshift-ingress/e2e-api-b-ecc3b0-openshift-default-764746cf54-nrvc2" Jul 29 19:57:39.717842 ip-10-0-131-93 kubenswrapper[2564]: I0729 19:57:39.717731 2564 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/f445712e-6292-49cc-a59b-59ee1015edea-workload-socket\") pod \"e2e-api-b-ecc3b0-openshift-default-764746cf54-nrvc2\" (UID: \"f445712e-6292-49cc-a59b-59ee1015edea\") " pod="openshift-ingress/e2e-api-b-ecc3b0-openshift-default-764746cf54-nrvc2" Jul 29 19:57:39.717842 ip-10-0-131-93 kubenswrapper[2564]: I0729 19:57:39.717765 2564 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/f445712e-6292-49cc-a59b-59ee1015edea-istio-token\") pod \"e2e-api-b-ecc3b0-openshift-default-764746cf54-nrvc2\" (UID: \"f445712e-6292-49cc-a59b-59ee1015edea\") " pod="openshift-ingress/e2e-api-b-ecc3b0-openshift-default-764746cf54-nrvc2" Jul 29 19:57:39.717842 ip-10-0-131-93 kubenswrapper[2564]: I0729 19:57:39.717833 2564 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/f445712e-6292-49cc-a59b-59ee1015edea-istiod-ca-cert\") pod \"e2e-api-b-ecc3b0-openshift-default-764746cf54-nrvc2\" (UID: \"f445712e-6292-49cc-a59b-59ee1015edea\") " pod="openshift-ingress/e2e-api-b-ecc3b0-openshift-default-764746cf54-nrvc2" Jul 29 19:57:39.718109 ip-10-0-131-93 kubenswrapper[2564]: I0729 19:57:39.717929 2564 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/f445712e-6292-49cc-a59b-59ee1015edea-istio-data\") pod \"e2e-api-b-ecc3b0-openshift-default-764746cf54-nrvc2\" (UID: \"f445712e-6292-49cc-a59b-59ee1015edea\") " pod="openshift-ingress/e2e-api-b-ecc3b0-openshift-default-764746cf54-nrvc2" Jul 29 19:57:39.718109 ip-10-0-131-93 kubenswrapper[2564]: I0729 19:57:39.717962 2564 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/f445712e-6292-49cc-a59b-59ee1015edea-credential-socket\") pod \"e2e-api-b-ecc3b0-openshift-default-764746cf54-nrvc2\" (UID: \"f445712e-6292-49cc-a59b-59ee1015edea\") " pod="openshift-ingress/e2e-api-b-ecc3b0-openshift-default-764746cf54-nrvc2" Jul 29 19:57:39.718109 ip-10-0-131-93 kubenswrapper[2564]: I0729 19:57:39.718013 2564 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/f445712e-6292-49cc-a59b-59ee1015edea-workload-certs\") pod \"e2e-api-b-ecc3b0-openshift-default-764746cf54-nrvc2\" (UID: \"f445712e-6292-49cc-a59b-59ee1015edea\") " pod="openshift-ingress/e2e-api-b-ecc3b0-openshift-default-764746cf54-nrvc2" Jul 29 19:57:39.718238 ip-10-0-131-93 kubenswrapper[2564]: I0729 19:57:39.718121 2564 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/f445712e-6292-49cc-a59b-59ee1015edea-workload-socket\") pod \"e2e-api-b-ecc3b0-openshift-default-764746cf54-nrvc2\" (UID: \"f445712e-6292-49cc-a59b-59ee1015edea\") " pod="openshift-ingress/e2e-api-b-ecc3b0-openshift-default-764746cf54-nrvc2" Jul 29 19:57:39.718605 ip-10-0-131-93 kubenswrapper[2564]: I0729 19:57:39.718583 2564 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/f445712e-6292-49cc-a59b-59ee1015edea-istiod-ca-cert\") pod \"e2e-api-b-ecc3b0-openshift-default-764746cf54-nrvc2\" (UID: \"f445712e-6292-49cc-a59b-59ee1015edea\") " pod="openshift-ingress/e2e-api-b-ecc3b0-openshift-default-764746cf54-nrvc2" Jul 29 19:57:39.719992 ip-10-0-131-93 kubenswrapper[2564]: I0729 19:57:39.719975 2564 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/f445712e-6292-49cc-a59b-59ee1015edea-istio-envoy\") pod \"e2e-api-b-ecc3b0-openshift-default-764746cf54-nrvc2\" (UID: \"f445712e-6292-49cc-a59b-59ee1015edea\") " pod="openshift-ingress/e2e-api-b-ecc3b0-openshift-default-764746cf54-nrvc2" Jul 29 19:57:39.720045 ip-10-0-131-93 kubenswrapper[2564]: I0729 19:57:39.720029 2564 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/f445712e-6292-49cc-a59b-59ee1015edea-istio-podinfo\") pod \"e2e-api-b-ecc3b0-openshift-default-764746cf54-nrvc2\" (UID: \"f445712e-6292-49cc-a59b-59ee1015edea\") " pod="openshift-ingress/e2e-api-b-ecc3b0-openshift-default-764746cf54-nrvc2" Jul 29 19:57:39.726417 ip-10-0-131-93 kubenswrapper[2564]: I0729 19:57:39.726392 2564 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-p8gtk\" (UniqueName: \"kubernetes.io/projected/f445712e-6292-49cc-a59b-59ee1015edea-kube-api-access-p8gtk\") pod \"e2e-api-b-ecc3b0-openshift-default-764746cf54-nrvc2\" (UID: \"f445712e-6292-49cc-a59b-59ee1015edea\") " pod="openshift-ingress/e2e-api-b-ecc3b0-openshift-default-764746cf54-nrvc2" Jul 29 19:57:39.727001 ip-10-0-131-93 kubenswrapper[2564]: I0729 19:57:39.726980 2564 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/f445712e-6292-49cc-a59b-59ee1015edea-istio-token\") pod \"e2e-api-b-ecc3b0-openshift-default-764746cf54-nrvc2\" (UID: \"f445712e-6292-49cc-a59b-59ee1015edea\") " pod="openshift-ingress/e2e-api-b-ecc3b0-openshift-default-764746cf54-nrvc2" Jul 29 19:57:39.777128 ip-10-0-131-93 kubenswrapper[2564]: I0729 19:57:39.777093 2564 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-api-b-ecc3b0-openshift-default-764746cf54-nrvc2" Jul 29 19:57:39.914238 ip-10-0-131-93 kubenswrapper[2564]: I0729 19:57:39.914211 2564 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/e2e-api-b-ecc3b0-openshift-default-764746cf54-nrvc2"] Jul 29 19:57:39.915327 ip-10-0-131-93 kubenswrapper[2564]: W0729 19:57:39.915298 2564 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podf445712e_6292_49cc_a59b_59ee1015edea.slice/crio-758c909f2fc137451b152724ed2670118c70d8090c41b2dea2bafebab6fd019d WatchSource:0}: Error finding container 758c909f2fc137451b152724ed2670118c70d8090c41b2dea2bafebab6fd019d: Status 404 returned error can't find the container with id 758c909f2fc137451b152724ed2670118c70d8090c41b2dea2bafebab6fd019d Jul 29 19:57:39.917483 ip-10-0-131-93 kubenswrapper[2564]: I0729 19:57:39.917453 2564 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236224Ki","pods":"250"} Jul 29 19:57:39.917541 ip-10-0-131-93 kubenswrapper[2564]: I0729 19:57:39.917515 2564 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236224Ki","pods":"250"} Jul 29 19:57:39.917591 ip-10-0-131-93 kubenswrapper[2564]: I0729 19:57:39.917556 2564 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236224Ki","pods":"250"} Jul 29 19:57:40.068173 ip-10-0-131-93 kubenswrapper[2564]: I0729 19:57:40.068093 2564 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-api-b-ecc3b0-openshift-default-764746cf54-nrvc2" event={"ID":"f445712e-6292-49cc-a59b-59ee1015edea","Type":"ContainerStarted","Data":"827206b5235940c6c74a447f55ee5f7490dc9dc72d0b019c19a05529105f2706"} Jul 29 19:57:40.068173 ip-10-0-131-93 kubenswrapper[2564]: I0729 19:57:40.068148 2564 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-api-b-ecc3b0-openshift-default-764746cf54-nrvc2" event={"ID":"f445712e-6292-49cc-a59b-59ee1015edea","Type":"ContainerStarted","Data":"758c909f2fc137451b152724ed2670118c70d8090c41b2dea2bafebab6fd019d"} Jul 29 19:57:40.081242 ip-10-0-131-93 kubenswrapper[2564]: I0729 19:57:40.081207 2564 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-444jx_faa8c546-c75b-4e10-93b4-661aa2b5f53e/konnectivity-agent/0.log" limit=1024 Jul 29 19:57:40.117813 ip-10-0-131-93 kubenswrapper[2564]: I0729 19:57:40.117762 2564 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-ingress/e2e-api-b-ecc3b0-openshift-default-764746cf54-nrvc2" podStartSLOduration=1.117741665 podStartE2EDuration="1.117741665s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-29 19:57:39 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-29 19:57:40.11511668 +0000 UTC m=+2435.839176373" watchObservedRunningTime="2026-07-29 19:57:40.117741665 +0000 UTC m=+2435.841801357" Jul 29 19:57:40.784591 ip-10-0-131-93 kubenswrapper[2564]: I0729 19:57:40.784562 2564 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="openshift-ingress/e2e-api-b-ecc3b0-openshift-default-764746cf54-nrvc2" Jul 29 19:57:40.785016 ip-10-0-131-93 kubenswrapper[2564]: I0729 19:57:40.784672 2564 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="openshift-ingress/e2e-api-b-ecc3b0-openshift-default-764746cf54-nrvc2" Jul 29 19:57:41.072854 ip-10-0-131-93 kubenswrapper[2564]: I0729 19:57:41.072772 2564 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/e2e-api-b-ecc3b0-openshift-default-764746cf54-nrvc2" Jul 29 19:57:41.073877 ip-10-0-131-93 kubenswrapper[2564]: I0729 19:57:41.073853 2564 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/e2e-api-b-ecc3b0-openshift-default-764746cf54-nrvc2" Jul 29 19:57:42.159550 ip-10-0-131-93 kubenswrapper[2564]: I0729 19:57:42.159523 2564 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-444jx_faa8c546-c75b-4e10-93b4-661aa2b5f53e/konnectivity-agent/0.log" limit=1024 Jul 29 19:57:42.340691 ip-10-0-131-93 kubenswrapper[2564]: I0729 19:57:42.340662 2564 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-444jx_faa8c546-c75b-4e10-93b4-661aa2b5f53e/konnectivity-agent/0.log" limit=1024 Jul 29 19:57:43.209242 ip-10-0-131-93 kubenswrapper[2564]: I0729 19:57:43.209212 2564 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-444jx_faa8c546-c75b-4e10-93b4-661aa2b5f53e/konnectivity-agent/0.log" limit=1024 Jul 29 19:57:43.450548 ip-10-0-131-93 kubenswrapper[2564]: I0729 19:57:43.450516 2564 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-444jx_faa8c546-c75b-4e10-93b4-661aa2b5f53e/konnectivity-agent/0.log" limit=1024 Jul 29 19:57:43.696873 ip-10-0-131-93 kubenswrapper[2564]: I0729 19:57:43.696848 2564 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-444jx_faa8c546-c75b-4e10-93b4-661aa2b5f53e/konnectivity-agent/0.log" limit=1024 Jul 29 19:57:43.795668 ip-10-0-131-93 kubenswrapper[2564]: I0729 19:57:43.795637 2564 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-444jx_faa8c546-c75b-4e10-93b4-661aa2b5f53e/konnectivity-agent/0.log" limit=1024 Jul 29 19:57:57.114551 ip-10-0-131-93 kubenswrapper[2564]: I0729 19:57:57.114509 2564 patch_prober.go:28] interesting pod/e2e-api-b-ecc3b0-openshift-default-764746cf54-nrvc2 container/istio-proxy namespace/openshift-ingress: Readiness probe status=failure output="Get \"http://10.132.0.46:15021/healthz/ready\": context deadline exceeded (Client.Timeout exceeded while awaiting headers)" start-of-body= Jul 29 19:57:57.114997 ip-10-0-131-93 kubenswrapper[2564]: I0729 19:57:57.114572 2564 prober.go:120] "Probe failed" probeType="Readiness" pod="openshift-ingress/e2e-api-b-ecc3b0-openshift-default-764746cf54-nrvc2" podUID="f445712e-6292-49cc-a59b-59ee1015edea" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.132.0.46:15021/healthz/ready\": context deadline exceeded (Client.Timeout exceeded while awaiting headers)" Jul 29 19:57:58.160844 ip-10-0-131-93 kubenswrapper[2564]: I0729 19:57:58.160810 2564 generic.go:356] "Generic (PLEG): container finished" podID="f445712e-6292-49cc-a59b-59ee1015edea" containerID="827206b5235940c6c74a447f55ee5f7490dc9dc72d0b019c19a05529105f2706" exitCode=0 Jul 29 19:57:58.161273 ip-10-0-131-93 kubenswrapper[2564]: I0729 19:57:58.160853 2564 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-api-b-ecc3b0-openshift-default-764746cf54-nrvc2" event={"ID":"f445712e-6292-49cc-a59b-59ee1015edea","Type":"ContainerDied","Data":"827206b5235940c6c74a447f55ee5f7490dc9dc72d0b019c19a05529105f2706"} Jul 29 19:57:58.161273 ip-10-0-131-93 kubenswrapper[2564]: I0729 19:57:58.161186 2564 scope.go:117] "RemoveContainer" containerID="827206b5235940c6c74a447f55ee5f7490dc9dc72d0b019c19a05529105f2706" Jul 29 19:57:58.162255 ip-10-0-131-93 kubenswrapper[2564]: I0729 19:57:58.162223 2564 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236224Ki","pods":"250"} Jul 29 19:57:58.162328 ip-10-0-131-93 kubenswrapper[2564]: I0729 19:57:58.162310 2564 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236224Ki","pods":"250"} Jul 29 19:57:58.162369 ip-10-0-131-93 kubenswrapper[2564]: I0729 19:57:58.162354 2564 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236224Ki","pods":"250"} Jul 29 19:57:59.167224 ip-10-0-131-93 kubenswrapper[2564]: I0729 19:57:59.167182 2564 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-api-b-ecc3b0-openshift-default-764746cf54-nrvc2" event={"ID":"f445712e-6292-49cc-a59b-59ee1015edea","Type":"ContainerStarted","Data":"4843fec23bcd168ca7bbe873b3cd222ff73123bc705e6dab7552fb16c57314bd"} Jul 29 19:57:59.777848 ip-10-0-131-93 kubenswrapper[2564]: I0729 19:57:59.777809 2564 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="openshift-ingress/e2e-api-b-ecc3b0-openshift-default-764746cf54-nrvc2" Jul 29 19:57:59.778132 ip-10-0-131-93 kubenswrapper[2564]: I0729 19:57:59.778103 2564 patch_prober.go:28] interesting pod/e2e-api-b-ecc3b0-openshift-default-764746cf54-nrvc2 container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.132.0.46:15021/healthz/ready\": dial tcp 10.132.0.46:15021: connect: connection refused" start-of-body= Jul 29 19:57:59.778223 ip-10-0-131-93 kubenswrapper[2564]: I0729 19:57:59.778163 2564 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-api-b-ecc3b0-openshift-default-764746cf54-nrvc2" podUID="f445712e-6292-49cc-a59b-59ee1015edea" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.132.0.46:15021/healthz/ready\": dial tcp 10.132.0.46:15021: connect: connection refused" Jul 29 19:58:00.780850 ip-10-0-131-93 kubenswrapper[2564]: I0729 19:58:00.780805 2564 patch_prober.go:28] interesting pod/e2e-api-b-ecc3b0-openshift-default-764746cf54-nrvc2 container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.132.0.46:15021/healthz/ready\": dial tcp 10.132.0.46:15021: connect: connection refused" start-of-body= Jul 29 19:58:00.781358 ip-10-0-131-93 kubenswrapper[2564]: I0729 19:58:00.780877 2564 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-api-b-ecc3b0-openshift-default-764746cf54-nrvc2" podUID="f445712e-6292-49cc-a59b-59ee1015edea" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.132.0.46:15021/healthz/ready\": dial tcp 10.132.0.46:15021: connect: connection refused" Jul 29 19:58:01.778072 ip-10-0-131-93 kubenswrapper[2564]: I0729 19:58:01.778033 2564 patch_prober.go:28] interesting pod/e2e-api-b-ecc3b0-openshift-default-764746cf54-nrvc2 container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.132.0.46:15021/healthz/ready\": dial tcp 10.132.0.46:15021: connect: connection refused" start-of-body= Jul 29 19:58:01.778356 ip-10-0-131-93 kubenswrapper[2564]: I0729 19:58:01.778329 2564 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-api-b-ecc3b0-openshift-default-764746cf54-nrvc2" podUID="f445712e-6292-49cc-a59b-59ee1015edea" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.132.0.46:15021/healthz/ready\": dial tcp 10.132.0.46:15021: connect: connection refused" Jul 29 19:58:02.778568 ip-10-0-131-93 kubenswrapper[2564]: I0729 19:58:02.778523 2564 patch_prober.go:28] interesting pod/e2e-api-b-ecc3b0-openshift-default-764746cf54-nrvc2 container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.132.0.46:15021/healthz/ready\": dial tcp 10.132.0.46:15021: connect: connection refused" start-of-body= Jul 29 19:58:02.779075 ip-10-0-131-93 kubenswrapper[2564]: I0729 19:58:02.778610 2564 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-api-b-ecc3b0-openshift-default-764746cf54-nrvc2" podUID="f445712e-6292-49cc-a59b-59ee1015edea" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.132.0.46:15021/healthz/ready\": dial tcp 10.132.0.46:15021: connect: connection refused" Jul 29 19:58:03.780837 ip-10-0-131-93 kubenswrapper[2564]: I0729 19:58:03.780802 2564 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="openshift-ingress/e2e-api-b-ecc3b0-openshift-default-764746cf54-nrvc2" Jul 29 19:58:03.781311 ip-10-0-131-93 kubenswrapper[2564]: I0729 19:58:03.781205 2564 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/e2e-api-b-ecc3b0-openshift-default-764746cf54-nrvc2" Jul 29 19:58:03.781929 ip-10-0-131-93 kubenswrapper[2564]: I0729 19:58:03.781908 2564 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/e2e-api-b-ecc3b0-openshift-default-764746cf54-nrvc2" Jul 29 19:58:07.075055 ip-10-0-131-93 kubenswrapper[2564]: I0729 19:58:07.075017 2564 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-444jx_faa8c546-c75b-4e10-93b4-661aa2b5f53e/konnectivity-agent/0.log" limit=1024 Jul 29 19:58:25.724983 ip-10-0-131-93 kubenswrapper[2564]: I0729 19:58:25.724946 2564 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-444jx_faa8c546-c75b-4e10-93b4-661aa2b5f53e/konnectivity-agent/0.log" limit=1024 Jul 29 19:58:25.822847 ip-10-0-131-93 kubenswrapper[2564]: I0729 19:58:25.822803 2564 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-444jx_faa8c546-c75b-4e10-93b4-661aa2b5f53e/konnectivity-agent/0.log" limit=1024 Jul 29 19:58:25.895324 ip-10-0-131-93 kubenswrapper[2564]: I0729 19:58:25.895289 2564 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-444jx_faa8c546-c75b-4e10-93b4-661aa2b5f53e/konnectivity-agent/0.log" limit=1024 Jul 29 19:58:26.243454 ip-10-0-131-93 kubenswrapper[2564]: I0729 19:58:26.243421 2564 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-444jx_faa8c546-c75b-4e10-93b4-661aa2b5f53e/konnectivity-agent/0.log" limit=1024 Jul 29 19:58:31.974041 ip-10-0-131-93 kubenswrapper[2564]: I0729 19:58:31.974009 2564 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-444jx_faa8c546-c75b-4e10-93b4-661aa2b5f53e/konnectivity-agent/0.log" limit=1024 Jul 29 19:58:32.075189 ip-10-0-131-93 kubenswrapper[2564]: I0729 19:58:32.075158 2564 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-444jx_faa8c546-c75b-4e10-93b4-661aa2b5f53e/konnectivity-agent/0.log" limit=1024 Jul 29 19:58:32.210709 ip-10-0-131-93 kubenswrapper[2564]: I0729 19:58:32.210677 2564 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-ingress/e2e-api-a-466674-openshift-default-6ff787f978-2qrj8"] Jul 29 19:58:32.210912 ip-10-0-131-93 kubenswrapper[2564]: I0729 19:58:32.210891 2564 kuberuntime_container.go:873] "Killing container with a grace period" pod="openshift-ingress/e2e-api-a-466674-openshift-default-6ff787f978-2qrj8" podUID="fa1763e9-97b8-40c5-9d3e-457961d94202" containerName="istio-proxy" containerID="cri-o://59c390d46a57b81ecd87f18e8da3fef25e98bc60e394f62d24f39d4bac57a1a5" gracePeriod=30 Jul 29 19:58:32.227260 ip-10-0-131-93 kubenswrapper[2564]: I0729 19:58:32.227193 2564 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-444jx_faa8c546-c75b-4e10-93b4-661aa2b5f53e/konnectivity-agent/0.log" limit=1024 Jul 29 19:58:32.438265 ip-10-0-131-93 kubenswrapper[2564]: I0729 19:58:32.438232 2564 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-444jx_faa8c546-c75b-4e10-93b4-661aa2b5f53e/konnectivity-agent/0.log" limit=1024 Jul 29 19:58:34.338019 ip-10-0-131-93 kubenswrapper[2564]: I0729 19:58:34.337978 2564 patch_prober.go:28] interesting pod/e2e-api-a-466674-openshift-default-6ff787f978-2qrj8 container/istio-proxy namespace/openshift-ingress: Readiness probe status=failure output="HTTP probe failed with statuscode: 503" start-of-body= Jul 29 19:58:34.338413 ip-10-0-131-93 kubenswrapper[2564]: I0729 19:58:34.338033 2564 prober.go:120] "Probe failed" probeType="Readiness" pod="openshift-ingress/e2e-api-a-466674-openshift-default-6ff787f978-2qrj8" podUID="fa1763e9-97b8-40c5-9d3e-457961d94202" containerName="istio-proxy" probeResult="failure" output="HTTP probe failed with statuscode: 503" Jul 29 19:58:37.366841 ip-10-0-131-93 kubenswrapper[2564]: I0729 19:58:37.366803 2564 generic.go:356] "Generic (PLEG): container finished" podID="fa1763e9-97b8-40c5-9d3e-457961d94202" containerID="59c390d46a57b81ecd87f18e8da3fef25e98bc60e394f62d24f39d4bac57a1a5" exitCode=0 Jul 29 19:58:37.367233 ip-10-0-131-93 kubenswrapper[2564]: I0729 19:58:37.366875 2564 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-api-a-466674-openshift-default-6ff787f978-2qrj8" event={"ID":"fa1763e9-97b8-40c5-9d3e-457961d94202","Type":"ContainerDied","Data":"59c390d46a57b81ecd87f18e8da3fef25e98bc60e394f62d24f39d4bac57a1a5"} Jul 29 19:58:37.367233 ip-10-0-131-93 kubenswrapper[2564]: I0729 19:58:37.366960 2564 scope.go:117] "RemoveContainer" containerID="a0e35d719e24ca5945630169280aab7b60024de89df48b489b41b239097545d1" Jul 29 19:58:37.452451 ip-10-0-131-93 kubenswrapper[2564]: I0729 19:58:37.452430 2564 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-api-a-466674-openshift-default-6ff787f978-2qrj8" Jul 29 19:58:37.622036 ip-10-0-131-93 kubenswrapper[2564]: I0729 19:58:37.621960 2564 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/fa1763e9-97b8-40c5-9d3e-457961d94202-credential-socket\") pod \"fa1763e9-97b8-40c5-9d3e-457961d94202\" (UID: \"fa1763e9-97b8-40c5-9d3e-457961d94202\") " Jul 29 19:58:37.622036 ip-10-0-131-93 kubenswrapper[2564]: I0729 19:58:37.622027 2564 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-p7k46\" (UniqueName: \"kubernetes.io/projected/fa1763e9-97b8-40c5-9d3e-457961d94202-kube-api-access-p7k46\") pod \"fa1763e9-97b8-40c5-9d3e-457961d94202\" (UID: \"fa1763e9-97b8-40c5-9d3e-457961d94202\") " Jul 29 19:58:37.622257 ip-10-0-131-93 kubenswrapper[2564]: I0729 19:58:37.622049 2564 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/fa1763e9-97b8-40c5-9d3e-457961d94202-istiod-ca-cert\") pod \"fa1763e9-97b8-40c5-9d3e-457961d94202\" (UID: \"fa1763e9-97b8-40c5-9d3e-457961d94202\") " Jul 29 19:58:37.622257 ip-10-0-131-93 kubenswrapper[2564]: I0729 19:58:37.622080 2564 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/fa1763e9-97b8-40c5-9d3e-457961d94202-workload-socket\") pod \"fa1763e9-97b8-40c5-9d3e-457961d94202\" (UID: \"fa1763e9-97b8-40c5-9d3e-457961d94202\") " Jul 29 19:58:37.622257 ip-10-0-131-93 kubenswrapper[2564]: I0729 19:58:37.622107 2564 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/fa1763e9-97b8-40c5-9d3e-457961d94202-istio-data\") pod \"fa1763e9-97b8-40c5-9d3e-457961d94202\" (UID: \"fa1763e9-97b8-40c5-9d3e-457961d94202\") " Jul 29 19:58:37.622412 ip-10-0-131-93 kubenswrapper[2564]: I0729 19:58:37.622257 2564 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/fa1763e9-97b8-40c5-9d3e-457961d94202-credential-socket" (OuterVolumeSpecName: "credential-socket") pod "fa1763e9-97b8-40c5-9d3e-457961d94202" (UID: "fa1763e9-97b8-40c5-9d3e-457961d94202"). InnerVolumeSpecName "credential-socket". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Jul 29 19:58:37.622412 ip-10-0-131-93 kubenswrapper[2564]: I0729 19:58:37.622301 2564 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/fa1763e9-97b8-40c5-9d3e-457961d94202-istio-podinfo\") pod \"fa1763e9-97b8-40c5-9d3e-457961d94202\" (UID: \"fa1763e9-97b8-40c5-9d3e-457961d94202\") " Jul 29 19:58:37.622412 ip-10-0-131-93 kubenswrapper[2564]: I0729 19:58:37.622396 2564 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/fa1763e9-97b8-40c5-9d3e-457961d94202-istio-envoy\") pod \"fa1763e9-97b8-40c5-9d3e-457961d94202\" (UID: \"fa1763e9-97b8-40c5-9d3e-457961d94202\") " Jul 29 19:58:37.622598 ip-10-0-131-93 kubenswrapper[2564]: I0729 19:58:37.622432 2564 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/fa1763e9-97b8-40c5-9d3e-457961d94202-workload-certs\") pod \"fa1763e9-97b8-40c5-9d3e-457961d94202\" (UID: \"fa1763e9-97b8-40c5-9d3e-457961d94202\") " Jul 29 19:58:37.622598 ip-10-0-131-93 kubenswrapper[2564]: I0729 19:58:37.622452 2564 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/fa1763e9-97b8-40c5-9d3e-457961d94202-workload-socket" (OuterVolumeSpecName: "workload-socket") pod "fa1763e9-97b8-40c5-9d3e-457961d94202" (UID: "fa1763e9-97b8-40c5-9d3e-457961d94202"). InnerVolumeSpecName "workload-socket". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Jul 29 19:58:37.622598 ip-10-0-131-93 kubenswrapper[2564]: I0729 19:58:37.622480 2564 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/fa1763e9-97b8-40c5-9d3e-457961d94202-istio-token\") pod \"fa1763e9-97b8-40c5-9d3e-457961d94202\" (UID: \"fa1763e9-97b8-40c5-9d3e-457961d94202\") " Jul 29 19:58:37.622598 ip-10-0-131-93 kubenswrapper[2564]: I0729 19:58:37.622482 2564 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/fa1763e9-97b8-40c5-9d3e-457961d94202-istiod-ca-cert" (OuterVolumeSpecName: "istiod-ca-cert") pod "fa1763e9-97b8-40c5-9d3e-457961d94202" (UID: "fa1763e9-97b8-40c5-9d3e-457961d94202"). InnerVolumeSpecName "istiod-ca-cert". PluginName "kubernetes.io/configmap", VolumeGIDValue "" Jul 29 19:58:37.622776 ip-10-0-131-93 kubenswrapper[2564]: I0729 19:58:37.622658 2564 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/fa1763e9-97b8-40c5-9d3e-457961d94202-workload-certs" (OuterVolumeSpecName: "workload-certs") pod "fa1763e9-97b8-40c5-9d3e-457961d94202" (UID: "fa1763e9-97b8-40c5-9d3e-457961d94202"). InnerVolumeSpecName "workload-certs". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Jul 29 19:58:37.622776 ip-10-0-131-93 kubenswrapper[2564]: I0729 19:58:37.622745 2564 reconciler_common.go:299] "Volume detached for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/fa1763e9-97b8-40c5-9d3e-457961d94202-workload-certs\") on node \"ip-10-0-131-93.ec2.internal\" DevicePath \"\"" Jul 29 19:58:37.622776 ip-10-0-131-93 kubenswrapper[2564]: I0729 19:58:37.622760 2564 reconciler_common.go:299] "Volume detached for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/fa1763e9-97b8-40c5-9d3e-457961d94202-credential-socket\") on node \"ip-10-0-131-93.ec2.internal\" DevicePath \"\"" Jul 29 19:58:37.622776 ip-10-0-131-93 kubenswrapper[2564]: I0729 19:58:37.622774 2564 reconciler_common.go:299] "Volume detached for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/fa1763e9-97b8-40c5-9d3e-457961d94202-istiod-ca-cert\") on node \"ip-10-0-131-93.ec2.internal\" DevicePath \"\"" Jul 29 19:58:37.622932 ip-10-0-131-93 kubenswrapper[2564]: I0729 19:58:37.622787 2564 reconciler_common.go:299] "Volume detached for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/fa1763e9-97b8-40c5-9d3e-457961d94202-workload-socket\") on node \"ip-10-0-131-93.ec2.internal\" DevicePath \"\"" Jul 29 19:58:37.623086 ip-10-0-131-93 kubenswrapper[2564]: I0729 19:58:37.623068 2564 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/fa1763e9-97b8-40c5-9d3e-457961d94202-istio-data" (OuterVolumeSpecName: "istio-data") pod "fa1763e9-97b8-40c5-9d3e-457961d94202" (UID: "fa1763e9-97b8-40c5-9d3e-457961d94202"). InnerVolumeSpecName "istio-data". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Jul 29 19:58:37.624533 ip-10-0-131-93 kubenswrapper[2564]: I0729 19:58:37.624503 2564 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/fa1763e9-97b8-40c5-9d3e-457961d94202-kube-api-access-p7k46" (OuterVolumeSpecName: "kube-api-access-p7k46") pod "fa1763e9-97b8-40c5-9d3e-457961d94202" (UID: "fa1763e9-97b8-40c5-9d3e-457961d94202"). InnerVolumeSpecName "kube-api-access-p7k46". PluginName "kubernetes.io/projected", VolumeGIDValue "" Jul 29 19:58:37.624649 ip-10-0-131-93 kubenswrapper[2564]: I0729 19:58:37.624588 2564 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/fa1763e9-97b8-40c5-9d3e-457961d94202-istio-token" (OuterVolumeSpecName: "istio-token") pod "fa1763e9-97b8-40c5-9d3e-457961d94202" (UID: "fa1763e9-97b8-40c5-9d3e-457961d94202"). InnerVolumeSpecName "istio-token". PluginName "kubernetes.io/projected", VolumeGIDValue "" Jul 29 19:58:37.624900 ip-10-0-131-93 kubenswrapper[2564]: I0729 19:58:37.624882 2564 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/fa1763e9-97b8-40c5-9d3e-457961d94202-istio-envoy" (OuterVolumeSpecName: "istio-envoy") pod "fa1763e9-97b8-40c5-9d3e-457961d94202" (UID: "fa1763e9-97b8-40c5-9d3e-457961d94202"). InnerVolumeSpecName "istio-envoy". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Jul 29 19:58:37.624900 ip-10-0-131-93 kubenswrapper[2564]: I0729 19:58:37.624886 2564 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/downward-api/fa1763e9-97b8-40c5-9d3e-457961d94202-istio-podinfo" (OuterVolumeSpecName: "istio-podinfo") pod "fa1763e9-97b8-40c5-9d3e-457961d94202" (UID: "fa1763e9-97b8-40c5-9d3e-457961d94202"). InnerVolumeSpecName "istio-podinfo". PluginName "kubernetes.io/downward-api", VolumeGIDValue "" Jul 29 19:58:37.723704 ip-10-0-131-93 kubenswrapper[2564]: I0729 19:58:37.723674 2564 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-p7k46\" (UniqueName: \"kubernetes.io/projected/fa1763e9-97b8-40c5-9d3e-457961d94202-kube-api-access-p7k46\") on node \"ip-10-0-131-93.ec2.internal\" DevicePath \"\"" Jul 29 19:58:37.723704 ip-10-0-131-93 kubenswrapper[2564]: I0729 19:58:37.723698 2564 reconciler_common.go:299] "Volume detached for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/fa1763e9-97b8-40c5-9d3e-457961d94202-istio-data\") on node \"ip-10-0-131-93.ec2.internal\" DevicePath \"\"" Jul 29 19:58:37.723704 ip-10-0-131-93 kubenswrapper[2564]: I0729 19:58:37.723708 2564 reconciler_common.go:299] "Volume detached for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/fa1763e9-97b8-40c5-9d3e-457961d94202-istio-podinfo\") on node \"ip-10-0-131-93.ec2.internal\" DevicePath \"\"" Jul 29 19:58:37.723917 ip-10-0-131-93 kubenswrapper[2564]: I0729 19:58:37.723718 2564 reconciler_common.go:299] "Volume detached for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/fa1763e9-97b8-40c5-9d3e-457961d94202-istio-envoy\") on node \"ip-10-0-131-93.ec2.internal\" DevicePath \"\"" Jul 29 19:58:37.723917 ip-10-0-131-93 kubenswrapper[2564]: I0729 19:58:37.723726 2564 reconciler_common.go:299] "Volume detached for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/fa1763e9-97b8-40c5-9d3e-457961d94202-istio-token\") on node \"ip-10-0-131-93.ec2.internal\" DevicePath \"\"" Jul 29 19:58:38.372941 ip-10-0-131-93 kubenswrapper[2564]: I0729 19:58:38.372911 2564 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-api-a-466674-openshift-default-6ff787f978-2qrj8" Jul 29 19:58:38.373364 ip-10-0-131-93 kubenswrapper[2564]: I0729 19:58:38.372954 2564 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-api-a-466674-openshift-default-6ff787f978-2qrj8" event={"ID":"fa1763e9-97b8-40c5-9d3e-457961d94202","Type":"ContainerDied","Data":"75cb023e03ef9f45217f3b0fd0954dd616ea5ec7d269c355b9c63c08fced79e3"} Jul 29 19:58:38.373364 ip-10-0-131-93 kubenswrapper[2564]: I0729 19:58:38.373015 2564 scope.go:117] "RemoveContainer" containerID="59c390d46a57b81ecd87f18e8da3fef25e98bc60e394f62d24f39d4bac57a1a5" Jul 29 19:58:38.394806 ip-10-0-131-93 kubenswrapper[2564]: I0729 19:58:38.394780 2564 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-ingress/e2e-api-a-466674-openshift-default-6ff787f978-2qrj8"] Jul 29 19:58:38.398768 ip-10-0-131-93 kubenswrapper[2564]: I0729 19:58:38.398745 2564 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["openshift-ingress/e2e-api-a-466674-openshift-default-6ff787f978-2qrj8"] Jul 29 19:58:38.782386 ip-10-0-131-93 kubenswrapper[2564]: I0729 19:58:38.782347 2564 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="fa1763e9-97b8-40c5-9d3e-457961d94202" path="/var/lib/kubelet/pods/fa1763e9-97b8-40c5-9d3e-457961d94202/volumes" Jul 29 19:58:42.078242 ip-10-0-131-93 kubenswrapper[2564]: I0729 19:58:42.078211 2564 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-444jx_faa8c546-c75b-4e10-93b4-661aa2b5f53e/konnectivity-agent/0.log" limit=1024 Jul 29 19:58:42.179266 ip-10-0-131-93 kubenswrapper[2564]: I0729 19:58:42.179236 2564 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-444jx_faa8c546-c75b-4e10-93b4-661aa2b5f53e/konnectivity-agent/0.log" limit=1024 Jul 29 19:58:48.095885 ip-10-0-131-93 kubenswrapper[2564]: I0729 19:58:48.095852 2564 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-444jx_faa8c546-c75b-4e10-93b4-661aa2b5f53e/konnectivity-agent/0.log" limit=1024 Jul 29 19:58:48.171878 ip-10-0-131-93 kubenswrapper[2564]: I0729 19:58:48.171844 2564 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-444jx_faa8c546-c75b-4e10-93b4-661aa2b5f53e/konnectivity-agent/0.log" limit=1024 Jul 29 19:58:48.241080 ip-10-0-131-93 kubenswrapper[2564]: I0729 19:58:48.241050 2564 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-444jx_faa8c546-c75b-4e10-93b4-661aa2b5f53e/konnectivity-agent/0.log" limit=1024 Jul 29 19:58:48.315580 ip-10-0-131-93 kubenswrapper[2564]: I0729 19:58:48.315540 2564 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-444jx_faa8c546-c75b-4e10-93b4-661aa2b5f53e/konnectivity-agent/0.log" limit=1024 Jul 29 19:58:48.669998 ip-10-0-131-93 kubenswrapper[2564]: I0729 19:58:48.669963 2564 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-444jx_faa8c546-c75b-4e10-93b4-661aa2b5f53e/konnectivity-agent/0.log" limit=1024 Jul 29 19:58:54.336791 ip-10-0-131-93 kubenswrapper[2564]: I0729 19:58:54.336752 2564 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-444jx_faa8c546-c75b-4e10-93b4-661aa2b5f53e/konnectivity-agent/0.log" limit=1024 Jul 29 19:58:54.437163 ip-10-0-131-93 kubenswrapper[2564]: I0729 19:58:54.437127 2564 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-444jx_faa8c546-c75b-4e10-93b4-661aa2b5f53e/konnectivity-agent/0.log" limit=1024 Jul 29 19:58:54.571169 ip-10-0-131-93 kubenswrapper[2564]: I0729 19:58:54.571122 2564 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-ingress/e2e-api-b-ecc3b0-openshift-default-764746cf54-nrvc2"] Jul 29 19:58:54.571332 ip-10-0-131-93 kubenswrapper[2564]: I0729 19:58:54.571311 2564 kuberuntime_container.go:873] "Killing container with a grace period" pod="openshift-ingress/e2e-api-b-ecc3b0-openshift-default-764746cf54-nrvc2" podUID="f445712e-6292-49cc-a59b-59ee1015edea" containerName="istio-proxy" containerID="cri-o://4843fec23bcd168ca7bbe873b3cd222ff73123bc705e6dab7552fb16c57314bd" gracePeriod=30 Jul 29 19:58:54.613667 ip-10-0-131-93 kubenswrapper[2564]: I0729 19:58:54.613578 2564 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-444jx_faa8c546-c75b-4e10-93b4-661aa2b5f53e/konnectivity-agent/0.log" limit=1024 Jul 29 19:58:54.825950 ip-10-0-131-93 kubenswrapper[2564]: I0729 19:58:54.825923 2564 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-444jx_faa8c546-c75b-4e10-93b4-661aa2b5f53e/konnectivity-agent/0.log" limit=1024 Jul 29 19:58:55.149959 ip-10-0-131-93 kubenswrapper[2564]: I0729 19:58:55.149928 2564 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-444jx_faa8c546-c75b-4e10-93b4-661aa2b5f53e/konnectivity-agent/0.log" limit=1024 Jul 29 19:58:55.435327 ip-10-0-131-93 kubenswrapper[2564]: I0729 19:58:55.435248 2564 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-ingress/e2e-shared-a-641e39-openshift-default-7898445948-2z4ct"] Jul 29 19:58:55.435711 ip-10-0-131-93 kubenswrapper[2564]: I0729 19:58:55.435590 2564 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="fa1763e9-97b8-40c5-9d3e-457961d94202" containerName="istio-proxy" Jul 29 19:58:55.435711 ip-10-0-131-93 kubenswrapper[2564]: I0729 19:58:55.435602 2564 state_mem.go:107] "Deleted CPUSet assignment" podUID="fa1763e9-97b8-40c5-9d3e-457961d94202" containerName="istio-proxy" Jul 29 19:58:55.435711 ip-10-0-131-93 kubenswrapper[2564]: I0729 19:58:55.435611 2564 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="fa1763e9-97b8-40c5-9d3e-457961d94202" containerName="istio-proxy" Jul 29 19:58:55.435711 ip-10-0-131-93 kubenswrapper[2564]: I0729 19:58:55.435616 2564 state_mem.go:107] "Deleted CPUSet assignment" podUID="fa1763e9-97b8-40c5-9d3e-457961d94202" containerName="istio-proxy" Jul 29 19:58:55.435711 ip-10-0-131-93 kubenswrapper[2564]: I0729 19:58:55.435674 2564 memory_manager.go:368] "RemoveStaleState removing state" podUID="fa1763e9-97b8-40c5-9d3e-457961d94202" containerName="istio-proxy" Jul 29 19:58:55.435711 ip-10-0-131-93 kubenswrapper[2564]: I0729 19:58:55.435682 2564 memory_manager.go:368] "RemoveStaleState removing state" podUID="fa1763e9-97b8-40c5-9d3e-457961d94202" containerName="istio-proxy" Jul 29 19:58:55.437783 ip-10-0-131-93 kubenswrapper[2564]: I0729 19:58:55.437767 2564 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-shared-a-641e39-openshift-default-7898445948-2z4ct" Jul 29 19:58:55.440567 ip-10-0-131-93 kubenswrapper[2564]: I0729 19:58:55.440546 2564 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress\"/\"e2e-shared-a-641e39-openshift-default-dockercfg-t2fp6\"" Jul 29 19:58:55.450132 ip-10-0-131-93 kubenswrapper[2564]: I0729 19:58:55.450104 2564 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/e2e-shared-a-641e39-openshift-default-7898445948-2z4ct"] Jul 29 19:58:55.458745 ip-10-0-131-93 kubenswrapper[2564]: I0729 19:58:55.458723 2564 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-444jx_faa8c546-c75b-4e10-93b4-661aa2b5f53e/konnectivity-agent/0.log" limit=1024 Jul 29 19:58:55.470953 ip-10-0-131-93 kubenswrapper[2564]: I0729 19:58:55.470920 2564 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-zl4jb\" (UniqueName: \"kubernetes.io/projected/c9bc01d1-d52b-4476-942e-68c5a3641d5f-kube-api-access-zl4jb\") pod \"e2e-shared-a-641e39-openshift-default-7898445948-2z4ct\" (UID: \"c9bc01d1-d52b-4476-942e-68c5a3641d5f\") " pod="openshift-ingress/e2e-shared-a-641e39-openshift-default-7898445948-2z4ct" Jul 29 19:58:55.471086 ip-10-0-131-93 kubenswrapper[2564]: I0729 19:58:55.470969 2564 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/c9bc01d1-d52b-4476-942e-68c5a3641d5f-credential-socket\") pod \"e2e-shared-a-641e39-openshift-default-7898445948-2z4ct\" (UID: \"c9bc01d1-d52b-4476-942e-68c5a3641d5f\") " pod="openshift-ingress/e2e-shared-a-641e39-openshift-default-7898445948-2z4ct" Jul 29 19:58:55.471086 ip-10-0-131-93 kubenswrapper[2564]: I0729 19:58:55.471045 2564 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/c9bc01d1-d52b-4476-942e-68c5a3641d5f-istio-podinfo\") pod \"e2e-shared-a-641e39-openshift-default-7898445948-2z4ct\" (UID: \"c9bc01d1-d52b-4476-942e-68c5a3641d5f\") " pod="openshift-ingress/e2e-shared-a-641e39-openshift-default-7898445948-2z4ct" Jul 29 19:58:55.471197 ip-10-0-131-93 kubenswrapper[2564]: I0729 19:58:55.471087 2564 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/c9bc01d1-d52b-4476-942e-68c5a3641d5f-workload-certs\") pod \"e2e-shared-a-641e39-openshift-default-7898445948-2z4ct\" (UID: \"c9bc01d1-d52b-4476-942e-68c5a3641d5f\") " pod="openshift-ingress/e2e-shared-a-641e39-openshift-default-7898445948-2z4ct" Jul 29 19:58:55.471197 ip-10-0-131-93 kubenswrapper[2564]: I0729 19:58:55.471131 2564 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/c9bc01d1-d52b-4476-942e-68c5a3641d5f-workload-socket\") pod \"e2e-shared-a-641e39-openshift-default-7898445948-2z4ct\" (UID: \"c9bc01d1-d52b-4476-942e-68c5a3641d5f\") " pod="openshift-ingress/e2e-shared-a-641e39-openshift-default-7898445948-2z4ct" Jul 29 19:58:55.471279 ip-10-0-131-93 kubenswrapper[2564]: I0729 19:58:55.471254 2564 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/c9bc01d1-d52b-4476-942e-68c5a3641d5f-istio-token\") pod \"e2e-shared-a-641e39-openshift-default-7898445948-2z4ct\" (UID: \"c9bc01d1-d52b-4476-942e-68c5a3641d5f\") " pod="openshift-ingress/e2e-shared-a-641e39-openshift-default-7898445948-2z4ct" Jul 29 19:58:55.471329 ip-10-0-131-93 kubenswrapper[2564]: I0729 19:58:55.471312 2564 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/c9bc01d1-d52b-4476-942e-68c5a3641d5f-istio-data\") pod \"e2e-shared-a-641e39-openshift-default-7898445948-2z4ct\" (UID: \"c9bc01d1-d52b-4476-942e-68c5a3641d5f\") " pod="openshift-ingress/e2e-shared-a-641e39-openshift-default-7898445948-2z4ct" Jul 29 19:58:55.471391 ip-10-0-131-93 kubenswrapper[2564]: I0729 19:58:55.471346 2564 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/c9bc01d1-d52b-4476-942e-68c5a3641d5f-istio-envoy\") pod \"e2e-shared-a-641e39-openshift-default-7898445948-2z4ct\" (UID: \"c9bc01d1-d52b-4476-942e-68c5a3641d5f\") " pod="openshift-ingress/e2e-shared-a-641e39-openshift-default-7898445948-2z4ct" Jul 29 19:58:55.471452 ip-10-0-131-93 kubenswrapper[2564]: I0729 19:58:55.471400 2564 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/c9bc01d1-d52b-4476-942e-68c5a3641d5f-istiod-ca-cert\") pod \"e2e-shared-a-641e39-openshift-default-7898445948-2z4ct\" (UID: \"c9bc01d1-d52b-4476-942e-68c5a3641d5f\") " pod="openshift-ingress/e2e-shared-a-641e39-openshift-default-7898445948-2z4ct" Jul 29 19:58:55.566336 ip-10-0-131-93 kubenswrapper[2564]: I0729 19:58:55.566304 2564 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-444jx_faa8c546-c75b-4e10-93b4-661aa2b5f53e/konnectivity-agent/0.log" limit=1024 Jul 29 19:58:55.572531 ip-10-0-131-93 kubenswrapper[2564]: I0729 19:58:55.572506 2564 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/c9bc01d1-d52b-4476-942e-68c5a3641d5f-istio-token\") pod \"e2e-shared-a-641e39-openshift-default-7898445948-2z4ct\" (UID: \"c9bc01d1-d52b-4476-942e-68c5a3641d5f\") " pod="openshift-ingress/e2e-shared-a-641e39-openshift-default-7898445948-2z4ct" Jul 29 19:58:55.572633 ip-10-0-131-93 kubenswrapper[2564]: I0729 19:58:55.572559 2564 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/c9bc01d1-d52b-4476-942e-68c5a3641d5f-istio-data\") pod \"e2e-shared-a-641e39-openshift-default-7898445948-2z4ct\" (UID: \"c9bc01d1-d52b-4476-942e-68c5a3641d5f\") " pod="openshift-ingress/e2e-shared-a-641e39-openshift-default-7898445948-2z4ct" Jul 29 19:58:55.572633 ip-10-0-131-93 kubenswrapper[2564]: I0729 19:58:55.572584 2564 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/c9bc01d1-d52b-4476-942e-68c5a3641d5f-istio-envoy\") pod \"e2e-shared-a-641e39-openshift-default-7898445948-2z4ct\" (UID: \"c9bc01d1-d52b-4476-942e-68c5a3641d5f\") " pod="openshift-ingress/e2e-shared-a-641e39-openshift-default-7898445948-2z4ct" Jul 29 19:58:55.572633 ip-10-0-131-93 kubenswrapper[2564]: I0729 19:58:55.572602 2564 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/c9bc01d1-d52b-4476-942e-68c5a3641d5f-istiod-ca-cert\") pod \"e2e-shared-a-641e39-openshift-default-7898445948-2z4ct\" (UID: \"c9bc01d1-d52b-4476-942e-68c5a3641d5f\") " pod="openshift-ingress/e2e-shared-a-641e39-openshift-default-7898445948-2z4ct" Jul 29 19:58:55.572771 ip-10-0-131-93 kubenswrapper[2564]: I0729 19:58:55.572625 2564 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-zl4jb\" (UniqueName: \"kubernetes.io/projected/c9bc01d1-d52b-4476-942e-68c5a3641d5f-kube-api-access-zl4jb\") pod \"e2e-shared-a-641e39-openshift-default-7898445948-2z4ct\" (UID: \"c9bc01d1-d52b-4476-942e-68c5a3641d5f\") " pod="openshift-ingress/e2e-shared-a-641e39-openshift-default-7898445948-2z4ct" Jul 29 19:58:55.572771 ip-10-0-131-93 kubenswrapper[2564]: I0729 19:58:55.572695 2564 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/c9bc01d1-d52b-4476-942e-68c5a3641d5f-credential-socket\") pod \"e2e-shared-a-641e39-openshift-default-7898445948-2z4ct\" (UID: \"c9bc01d1-d52b-4476-942e-68c5a3641d5f\") " pod="openshift-ingress/e2e-shared-a-641e39-openshift-default-7898445948-2z4ct" Jul 29 19:58:55.572771 ip-10-0-131-93 kubenswrapper[2564]: I0729 19:58:55.572736 2564 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/c9bc01d1-d52b-4476-942e-68c5a3641d5f-istio-podinfo\") pod \"e2e-shared-a-641e39-openshift-default-7898445948-2z4ct\" (UID: \"c9bc01d1-d52b-4476-942e-68c5a3641d5f\") " pod="openshift-ingress/e2e-shared-a-641e39-openshift-default-7898445948-2z4ct" Jul 29 19:58:55.572771 ip-10-0-131-93 kubenswrapper[2564]: I0729 19:58:55.572758 2564 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/c9bc01d1-d52b-4476-942e-68c5a3641d5f-workload-certs\") pod \"e2e-shared-a-641e39-openshift-default-7898445948-2z4ct\" (UID: \"c9bc01d1-d52b-4476-942e-68c5a3641d5f\") " pod="openshift-ingress/e2e-shared-a-641e39-openshift-default-7898445948-2z4ct" Jul 29 19:58:55.572967 ip-10-0-131-93 kubenswrapper[2564]: I0729 19:58:55.572793 2564 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/c9bc01d1-d52b-4476-942e-68c5a3641d5f-workload-socket\") pod \"e2e-shared-a-641e39-openshift-default-7898445948-2z4ct\" (UID: \"c9bc01d1-d52b-4476-942e-68c5a3641d5f\") " pod="openshift-ingress/e2e-shared-a-641e39-openshift-default-7898445948-2z4ct" Jul 29 19:58:55.573141 ip-10-0-131-93 kubenswrapper[2564]: I0729 19:58:55.573117 2564 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/c9bc01d1-d52b-4476-942e-68c5a3641d5f-istio-data\") pod \"e2e-shared-a-641e39-openshift-default-7898445948-2z4ct\" (UID: \"c9bc01d1-d52b-4476-942e-68c5a3641d5f\") " pod="openshift-ingress/e2e-shared-a-641e39-openshift-default-7898445948-2z4ct" Jul 29 19:58:55.573299 ip-10-0-131-93 kubenswrapper[2564]: I0729 19:58:55.573281 2564 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/c9bc01d1-d52b-4476-942e-68c5a3641d5f-workload-socket\") pod \"e2e-shared-a-641e39-openshift-default-7898445948-2z4ct\" (UID: \"c9bc01d1-d52b-4476-942e-68c5a3641d5f\") " pod="openshift-ingress/e2e-shared-a-641e39-openshift-default-7898445948-2z4ct" Jul 29 19:58:55.573391 ip-10-0-131-93 kubenswrapper[2564]: I0729 19:58:55.573335 2564 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/c9bc01d1-d52b-4476-942e-68c5a3641d5f-workload-certs\") pod \"e2e-shared-a-641e39-openshift-default-7898445948-2z4ct\" (UID: \"c9bc01d1-d52b-4476-942e-68c5a3641d5f\") " pod="openshift-ingress/e2e-shared-a-641e39-openshift-default-7898445948-2z4ct" Jul 29 19:58:55.573441 ip-10-0-131-93 kubenswrapper[2564]: I0729 19:58:55.573426 2564 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/c9bc01d1-d52b-4476-942e-68c5a3641d5f-credential-socket\") pod \"e2e-shared-a-641e39-openshift-default-7898445948-2z4ct\" (UID: \"c9bc01d1-d52b-4476-942e-68c5a3641d5f\") " pod="openshift-ingress/e2e-shared-a-641e39-openshift-default-7898445948-2z4ct" Jul 29 19:58:55.573485 ip-10-0-131-93 kubenswrapper[2564]: I0729 19:58:55.573461 2564 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/c9bc01d1-d52b-4476-942e-68c5a3641d5f-istiod-ca-cert\") pod \"e2e-shared-a-641e39-openshift-default-7898445948-2z4ct\" (UID: \"c9bc01d1-d52b-4476-942e-68c5a3641d5f\") " pod="openshift-ingress/e2e-shared-a-641e39-openshift-default-7898445948-2z4ct" Jul 29 19:58:55.574892 ip-10-0-131-93 kubenswrapper[2564]: I0729 19:58:55.574874 2564 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/c9bc01d1-d52b-4476-942e-68c5a3641d5f-istio-envoy\") pod \"e2e-shared-a-641e39-openshift-default-7898445948-2z4ct\" (UID: \"c9bc01d1-d52b-4476-942e-68c5a3641d5f\") " pod="openshift-ingress/e2e-shared-a-641e39-openshift-default-7898445948-2z4ct" Jul 29 19:58:55.575144 ip-10-0-131-93 kubenswrapper[2564]: I0729 19:58:55.575123 2564 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/c9bc01d1-d52b-4476-942e-68c5a3641d5f-istio-podinfo\") pod \"e2e-shared-a-641e39-openshift-default-7898445948-2z4ct\" (UID: \"c9bc01d1-d52b-4476-942e-68c5a3641d5f\") " pod="openshift-ingress/e2e-shared-a-641e39-openshift-default-7898445948-2z4ct" Jul 29 19:58:55.580293 ip-10-0-131-93 kubenswrapper[2564]: I0729 19:58:55.580271 2564 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-zl4jb\" (UniqueName: \"kubernetes.io/projected/c9bc01d1-d52b-4476-942e-68c5a3641d5f-kube-api-access-zl4jb\") pod \"e2e-shared-a-641e39-openshift-default-7898445948-2z4ct\" (UID: \"c9bc01d1-d52b-4476-942e-68c5a3641d5f\") " pod="openshift-ingress/e2e-shared-a-641e39-openshift-default-7898445948-2z4ct" Jul 29 19:58:55.580666 ip-10-0-131-93 kubenswrapper[2564]: I0729 19:58:55.580647 2564 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/c9bc01d1-d52b-4476-942e-68c5a3641d5f-istio-token\") pod \"e2e-shared-a-641e39-openshift-default-7898445948-2z4ct\" (UID: \"c9bc01d1-d52b-4476-942e-68c5a3641d5f\") " pod="openshift-ingress/e2e-shared-a-641e39-openshift-default-7898445948-2z4ct" Jul 29 19:58:55.747891 ip-10-0-131-93 kubenswrapper[2564]: I0729 19:58:55.747858 2564 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-shared-a-641e39-openshift-default-7898445948-2z4ct" Jul 29 19:58:55.912331 ip-10-0-131-93 kubenswrapper[2564]: I0729 19:58:55.912301 2564 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/e2e-shared-a-641e39-openshift-default-7898445948-2z4ct"] Jul 29 19:58:55.914395 ip-10-0-131-93 kubenswrapper[2564]: W0729 19:58:55.914357 2564 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podc9bc01d1_d52b_4476_942e_68c5a3641d5f.slice/crio-36f793126db2b7b471f1c907435363a76c52ab2cc3b88b8fafbbb4d58f93acbe WatchSource:0}: Error finding container 36f793126db2b7b471f1c907435363a76c52ab2cc3b88b8fafbbb4d58f93acbe: Status 404 returned error can't find the container with id 36f793126db2b7b471f1c907435363a76c52ab2cc3b88b8fafbbb4d58f93acbe Jul 29 19:58:55.916756 ip-10-0-131-93 kubenswrapper[2564]: I0729 19:58:55.916718 2564 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236224Ki","pods":"250"} Jul 29 19:58:55.916833 ip-10-0-131-93 kubenswrapper[2564]: I0729 19:58:55.916802 2564 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236224Ki","pods":"250"} Jul 29 19:58:55.916881 ip-10-0-131-93 kubenswrapper[2564]: I0729 19:58:55.916845 2564 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236224Ki","pods":"250"} Jul 29 19:58:56.462150 ip-10-0-131-93 kubenswrapper[2564]: I0729 19:58:56.462110 2564 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-shared-a-641e39-openshift-default-7898445948-2z4ct" event={"ID":"c9bc01d1-d52b-4476-942e-68c5a3641d5f","Type":"ContainerStarted","Data":"902c38b67ac57d88a03146f71d38d5110d4192d6f459be20582131647463183b"} Jul 29 19:58:56.462762 ip-10-0-131-93 kubenswrapper[2564]: I0729 19:58:56.462728 2564 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-shared-a-641e39-openshift-default-7898445948-2z4ct" event={"ID":"c9bc01d1-d52b-4476-942e-68c5a3641d5f","Type":"ContainerStarted","Data":"36f793126db2b7b471f1c907435363a76c52ab2cc3b88b8fafbbb4d58f93acbe"} Jul 29 19:58:56.490757 ip-10-0-131-93 kubenswrapper[2564]: I0729 19:58:56.490699 2564 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-ingress/e2e-shared-a-641e39-openshift-default-7898445948-2z4ct" podStartSLOduration=1.490680197 podStartE2EDuration="1.490680197s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-29 19:58:55 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-29 19:58:56.488464825 +0000 UTC m=+2512.212524519" watchObservedRunningTime="2026-07-29 19:58:56.490680197 +0000 UTC m=+2512.214739890" Jul 29 19:58:56.748778 ip-10-0-131-93 kubenswrapper[2564]: I0729 19:58:56.748684 2564 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="openshift-ingress/e2e-shared-a-641e39-openshift-default-7898445948-2z4ct" Jul 29 19:58:56.754573 ip-10-0-131-93 kubenswrapper[2564]: I0729 19:58:56.754548 2564 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="openshift-ingress/e2e-shared-a-641e39-openshift-default-7898445948-2z4ct" Jul 29 19:58:57.468392 ip-10-0-131-93 kubenswrapper[2564]: I0729 19:58:57.468356 2564 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/e2e-shared-a-641e39-openshift-default-7898445948-2z4ct" Jul 29 19:58:57.469473 ip-10-0-131-93 kubenswrapper[2564]: I0729 19:58:57.469448 2564 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/e2e-shared-a-641e39-openshift-default-7898445948-2z4ct" Jul 29 19:58:58.379103 ip-10-0-131-93 kubenswrapper[2564]: I0729 19:58:58.379075 2564 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-444jx_faa8c546-c75b-4e10-93b4-661aa2b5f53e/konnectivity-agent/0.log" limit=1024 Jul 29 19:58:58.482916 ip-10-0-131-93 kubenswrapper[2564]: I0729 19:58:58.482888 2564 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-444jx_faa8c546-c75b-4e10-93b4-661aa2b5f53e/konnectivity-agent/0.log" limit=1024 Jul 29 19:58:59.664875 ip-10-0-131-93 kubenswrapper[2564]: I0729 19:58:59.664847 2564 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-444jx_faa8c546-c75b-4e10-93b4-661aa2b5f53e/konnectivity-agent/0.log" limit=1024 Jul 29 19:58:59.833345 ip-10-0-131-93 kubenswrapper[2564]: I0729 19:58:59.833292 2564 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-api-b-ecc3b0-openshift-default-764746cf54-nrvc2" Jul 29 19:58:59.908401 ip-10-0-131-93 kubenswrapper[2564]: I0729 19:58:59.908347 2564 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/f445712e-6292-49cc-a59b-59ee1015edea-workload-certs\") pod \"f445712e-6292-49cc-a59b-59ee1015edea\" (UID: \"f445712e-6292-49cc-a59b-59ee1015edea\") " Jul 29 19:58:59.908559 ip-10-0-131-93 kubenswrapper[2564]: I0729 19:58:59.908438 2564 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/f445712e-6292-49cc-a59b-59ee1015edea-credential-socket\") pod \"f445712e-6292-49cc-a59b-59ee1015edea\" (UID: \"f445712e-6292-49cc-a59b-59ee1015edea\") " Jul 29 19:58:59.908559 ip-10-0-131-93 kubenswrapper[2564]: I0729 19:58:59.908472 2564 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-p8gtk\" (UniqueName: \"kubernetes.io/projected/f445712e-6292-49cc-a59b-59ee1015edea-kube-api-access-p8gtk\") pod \"f445712e-6292-49cc-a59b-59ee1015edea\" (UID: \"f445712e-6292-49cc-a59b-59ee1015edea\") " Jul 29 19:58:59.908559 ip-10-0-131-93 kubenswrapper[2564]: I0729 19:58:59.908506 2564 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/f445712e-6292-49cc-a59b-59ee1015edea-istio-token\") pod \"f445712e-6292-49cc-a59b-59ee1015edea\" (UID: \"f445712e-6292-49cc-a59b-59ee1015edea\") " Jul 29 19:58:59.908738 ip-10-0-131-93 kubenswrapper[2564]: I0729 19:58:59.908568 2564 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/f445712e-6292-49cc-a59b-59ee1015edea-istio-data\") pod \"f445712e-6292-49cc-a59b-59ee1015edea\" (UID: \"f445712e-6292-49cc-a59b-59ee1015edea\") " Jul 29 19:58:59.908738 ip-10-0-131-93 kubenswrapper[2564]: I0729 19:58:59.908607 2564 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/f445712e-6292-49cc-a59b-59ee1015edea-istio-podinfo\") pod \"f445712e-6292-49cc-a59b-59ee1015edea\" (UID: \"f445712e-6292-49cc-a59b-59ee1015edea\") " Jul 29 19:58:59.908738 ip-10-0-131-93 kubenswrapper[2564]: I0729 19:58:59.908628 2564 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/f445712e-6292-49cc-a59b-59ee1015edea-workload-certs" (OuterVolumeSpecName: "workload-certs") pod "f445712e-6292-49cc-a59b-59ee1015edea" (UID: "f445712e-6292-49cc-a59b-59ee1015edea"). InnerVolumeSpecName "workload-certs". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Jul 29 19:58:59.908738 ip-10-0-131-93 kubenswrapper[2564]: I0729 19:58:59.908643 2564 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/f445712e-6292-49cc-a59b-59ee1015edea-workload-socket\") pod \"f445712e-6292-49cc-a59b-59ee1015edea\" (UID: \"f445712e-6292-49cc-a59b-59ee1015edea\") " Jul 29 19:58:59.908738 ip-10-0-131-93 kubenswrapper[2564]: I0729 19:58:59.908672 2564 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/f445712e-6292-49cc-a59b-59ee1015edea-istiod-ca-cert\") pod \"f445712e-6292-49cc-a59b-59ee1015edea\" (UID: \"f445712e-6292-49cc-a59b-59ee1015edea\") " Jul 29 19:58:59.908738 ip-10-0-131-93 kubenswrapper[2564]: I0729 19:58:59.908701 2564 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/f445712e-6292-49cc-a59b-59ee1015edea-istio-envoy\") pod \"f445712e-6292-49cc-a59b-59ee1015edea\" (UID: \"f445712e-6292-49cc-a59b-59ee1015edea\") " Jul 29 19:58:59.909028 ip-10-0-131-93 kubenswrapper[2564]: I0729 19:58:59.908761 2564 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/f445712e-6292-49cc-a59b-59ee1015edea-credential-socket" (OuterVolumeSpecName: "credential-socket") pod "f445712e-6292-49cc-a59b-59ee1015edea" (UID: "f445712e-6292-49cc-a59b-59ee1015edea"). InnerVolumeSpecName "credential-socket". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Jul 29 19:58:59.909028 ip-10-0-131-93 kubenswrapper[2564]: I0729 19:58:59.908932 2564 reconciler_common.go:299] "Volume detached for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/f445712e-6292-49cc-a59b-59ee1015edea-workload-certs\") on node \"ip-10-0-131-93.ec2.internal\" DevicePath \"\"" Jul 29 19:58:59.909028 ip-10-0-131-93 kubenswrapper[2564]: I0729 19:58:59.908948 2564 reconciler_common.go:299] "Volume detached for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/f445712e-6292-49cc-a59b-59ee1015edea-credential-socket\") on node \"ip-10-0-131-93.ec2.internal\" DevicePath \"\"" Jul 29 19:58:59.909028 ip-10-0-131-93 kubenswrapper[2564]: I0729 19:58:59.908923 2564 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/f445712e-6292-49cc-a59b-59ee1015edea-workload-socket" (OuterVolumeSpecName: "workload-socket") pod "f445712e-6292-49cc-a59b-59ee1015edea" (UID: "f445712e-6292-49cc-a59b-59ee1015edea"). InnerVolumeSpecName "workload-socket". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Jul 29 19:58:59.909229 ip-10-0-131-93 kubenswrapper[2564]: I0729 19:58:59.909081 2564 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/f445712e-6292-49cc-a59b-59ee1015edea-istiod-ca-cert" (OuterVolumeSpecName: "istiod-ca-cert") pod "f445712e-6292-49cc-a59b-59ee1015edea" (UID: "f445712e-6292-49cc-a59b-59ee1015edea"). InnerVolumeSpecName "istiod-ca-cert". PluginName "kubernetes.io/configmap", VolumeGIDValue "" Jul 29 19:58:59.909460 ip-10-0-131-93 kubenswrapper[2564]: I0729 19:58:59.909436 2564 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/f445712e-6292-49cc-a59b-59ee1015edea-istio-data" (OuterVolumeSpecName: "istio-data") pod "f445712e-6292-49cc-a59b-59ee1015edea" (UID: "f445712e-6292-49cc-a59b-59ee1015edea"). InnerVolumeSpecName "istio-data". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Jul 29 19:58:59.910763 ip-10-0-131-93 kubenswrapper[2564]: I0729 19:58:59.910729 2564 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/f445712e-6292-49cc-a59b-59ee1015edea-istio-token" (OuterVolumeSpecName: "istio-token") pod "f445712e-6292-49cc-a59b-59ee1015edea" (UID: "f445712e-6292-49cc-a59b-59ee1015edea"). InnerVolumeSpecName "istio-token". PluginName "kubernetes.io/projected", VolumeGIDValue "" Jul 29 19:58:59.910763 ip-10-0-131-93 kubenswrapper[2564]: I0729 19:58:59.910740 2564 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/f445712e-6292-49cc-a59b-59ee1015edea-istio-envoy" (OuterVolumeSpecName: "istio-envoy") pod "f445712e-6292-49cc-a59b-59ee1015edea" (UID: "f445712e-6292-49cc-a59b-59ee1015edea"). InnerVolumeSpecName "istio-envoy". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Jul 29 19:58:59.911055 ip-10-0-131-93 kubenswrapper[2564]: I0729 19:58:59.911036 2564 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/f445712e-6292-49cc-a59b-59ee1015edea-kube-api-access-p8gtk" (OuterVolumeSpecName: "kube-api-access-p8gtk") pod "f445712e-6292-49cc-a59b-59ee1015edea" (UID: "f445712e-6292-49cc-a59b-59ee1015edea"). InnerVolumeSpecName "kube-api-access-p8gtk". PluginName "kubernetes.io/projected", VolumeGIDValue "" Jul 29 19:58:59.911134 ip-10-0-131-93 kubenswrapper[2564]: I0729 19:58:59.911115 2564 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/downward-api/f445712e-6292-49cc-a59b-59ee1015edea-istio-podinfo" (OuterVolumeSpecName: "istio-podinfo") pod "f445712e-6292-49cc-a59b-59ee1015edea" (UID: "f445712e-6292-49cc-a59b-59ee1015edea"). InnerVolumeSpecName "istio-podinfo". PluginName "kubernetes.io/downward-api", VolumeGIDValue "" Jul 29 19:58:59.974048 ip-10-0-131-93 kubenswrapper[2564]: I0729 19:58:59.974021 2564 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-444jx_faa8c546-c75b-4e10-93b4-661aa2b5f53e/konnectivity-agent/0.log" limit=1024 Jul 29 19:59:00.010280 ip-10-0-131-93 kubenswrapper[2564]: I0729 19:59:00.010248 2564 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-p8gtk\" (UniqueName: \"kubernetes.io/projected/f445712e-6292-49cc-a59b-59ee1015edea-kube-api-access-p8gtk\") on node \"ip-10-0-131-93.ec2.internal\" DevicePath \"\"" Jul 29 19:59:00.010280 ip-10-0-131-93 kubenswrapper[2564]: I0729 19:59:00.010276 2564 reconciler_common.go:299] "Volume detached for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/f445712e-6292-49cc-a59b-59ee1015edea-istio-token\") on node \"ip-10-0-131-93.ec2.internal\" DevicePath \"\"" Jul 29 19:59:00.010280 ip-10-0-131-93 kubenswrapper[2564]: I0729 19:59:00.010285 2564 reconciler_common.go:299] "Volume detached for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/f445712e-6292-49cc-a59b-59ee1015edea-istio-data\") on node \"ip-10-0-131-93.ec2.internal\" DevicePath \"\"" Jul 29 19:59:00.010514 ip-10-0-131-93 kubenswrapper[2564]: I0729 19:59:00.010293 2564 reconciler_common.go:299] "Volume detached for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/f445712e-6292-49cc-a59b-59ee1015edea-istio-podinfo\") on node \"ip-10-0-131-93.ec2.internal\" DevicePath \"\"" Jul 29 19:59:00.010514 ip-10-0-131-93 kubenswrapper[2564]: I0729 19:59:00.010302 2564 reconciler_common.go:299] "Volume detached for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/f445712e-6292-49cc-a59b-59ee1015edea-workload-socket\") on node \"ip-10-0-131-93.ec2.internal\" DevicePath \"\"" Jul 29 19:59:00.010514 ip-10-0-131-93 kubenswrapper[2564]: I0729 19:59:00.010310 2564 reconciler_common.go:299] "Volume detached for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/f445712e-6292-49cc-a59b-59ee1015edea-istiod-ca-cert\") on node \"ip-10-0-131-93.ec2.internal\" DevicePath \"\"" Jul 29 19:59:00.010514 ip-10-0-131-93 kubenswrapper[2564]: I0729 19:59:00.010318 2564 reconciler_common.go:299] "Volume detached for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/f445712e-6292-49cc-a59b-59ee1015edea-istio-envoy\") on node \"ip-10-0-131-93.ec2.internal\" DevicePath \"\"" Jul 29 19:59:00.225114 ip-10-0-131-93 kubenswrapper[2564]: I0729 19:59:00.225082 2564 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-444jx_faa8c546-c75b-4e10-93b4-661aa2b5f53e/konnectivity-agent/0.log" limit=1024 Jul 29 19:59:00.320139 ip-10-0-131-93 kubenswrapper[2564]: I0729 19:59:00.320104 2564 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-444jx_faa8c546-c75b-4e10-93b4-661aa2b5f53e/konnectivity-agent/0.log" limit=1024 Jul 29 19:59:00.484929 ip-10-0-131-93 kubenswrapper[2564]: I0729 19:59:00.484841 2564 generic.go:356] "Generic (PLEG): container finished" podID="f445712e-6292-49cc-a59b-59ee1015edea" containerID="4843fec23bcd168ca7bbe873b3cd222ff73123bc705e6dab7552fb16c57314bd" exitCode=0 Jul 29 19:59:00.484929 ip-10-0-131-93 kubenswrapper[2564]: I0729 19:59:00.484894 2564 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-api-b-ecc3b0-openshift-default-764746cf54-nrvc2" Jul 29 19:59:00.485130 ip-10-0-131-93 kubenswrapper[2564]: I0729 19:59:00.484927 2564 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-api-b-ecc3b0-openshift-default-764746cf54-nrvc2" event={"ID":"f445712e-6292-49cc-a59b-59ee1015edea","Type":"ContainerDied","Data":"4843fec23bcd168ca7bbe873b3cd222ff73123bc705e6dab7552fb16c57314bd"} Jul 29 19:59:00.485130 ip-10-0-131-93 kubenswrapper[2564]: I0729 19:59:00.484982 2564 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-api-b-ecc3b0-openshift-default-764746cf54-nrvc2" event={"ID":"f445712e-6292-49cc-a59b-59ee1015edea","Type":"ContainerDied","Data":"758c909f2fc137451b152724ed2670118c70d8090c41b2dea2bafebab6fd019d"} Jul 29 19:59:00.485130 ip-10-0-131-93 kubenswrapper[2564]: I0729 19:59:00.485007 2564 scope.go:117] "RemoveContainer" containerID="4843fec23bcd168ca7bbe873b3cd222ff73123bc705e6dab7552fb16c57314bd" Jul 29 19:59:00.494667 ip-10-0-131-93 kubenswrapper[2564]: I0729 19:59:00.494656 2564 scope.go:117] "RemoveContainer" containerID="827206b5235940c6c74a447f55ee5f7490dc9dc72d0b019c19a05529105f2706" Jul 29 19:59:00.502728 ip-10-0-131-93 kubenswrapper[2564]: I0729 19:59:00.502704 2564 scope.go:117] "RemoveContainer" containerID="4843fec23bcd168ca7bbe873b3cd222ff73123bc705e6dab7552fb16c57314bd" Jul 29 19:59:00.502997 ip-10-0-131-93 kubenswrapper[2564]: E0729 19:59:00.502974 2564 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"4843fec23bcd168ca7bbe873b3cd222ff73123bc705e6dab7552fb16c57314bd\": container with ID starting with 4843fec23bcd168ca7bbe873b3cd222ff73123bc705e6dab7552fb16c57314bd not found: ID does not exist" containerID="4843fec23bcd168ca7bbe873b3cd222ff73123bc705e6dab7552fb16c57314bd" Jul 29 19:59:00.503072 ip-10-0-131-93 kubenswrapper[2564]: I0729 19:59:00.503006 2564 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"4843fec23bcd168ca7bbe873b3cd222ff73123bc705e6dab7552fb16c57314bd"} err="failed to get container status \"4843fec23bcd168ca7bbe873b3cd222ff73123bc705e6dab7552fb16c57314bd\": rpc error: code = NotFound desc = could not find container \"4843fec23bcd168ca7bbe873b3cd222ff73123bc705e6dab7552fb16c57314bd\": container with ID starting with 4843fec23bcd168ca7bbe873b3cd222ff73123bc705e6dab7552fb16c57314bd not found: ID does not exist" Jul 29 19:59:00.503072 ip-10-0-131-93 kubenswrapper[2564]: I0729 19:59:00.503025 2564 scope.go:117] "RemoveContainer" containerID="827206b5235940c6c74a447f55ee5f7490dc9dc72d0b019c19a05529105f2706" Jul 29 19:59:00.503253 ip-10-0-131-93 kubenswrapper[2564]: E0729 19:59:00.503234 2564 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"827206b5235940c6c74a447f55ee5f7490dc9dc72d0b019c19a05529105f2706\": container with ID starting with 827206b5235940c6c74a447f55ee5f7490dc9dc72d0b019c19a05529105f2706 not found: ID does not exist" containerID="827206b5235940c6c74a447f55ee5f7490dc9dc72d0b019c19a05529105f2706" Jul 29 19:59:00.503320 ip-10-0-131-93 kubenswrapper[2564]: I0729 19:59:00.503261 2564 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"827206b5235940c6c74a447f55ee5f7490dc9dc72d0b019c19a05529105f2706"} err="failed to get container status \"827206b5235940c6c74a447f55ee5f7490dc9dc72d0b019c19a05529105f2706\": rpc error: code = NotFound desc = could not find container \"827206b5235940c6c74a447f55ee5f7490dc9dc72d0b019c19a05529105f2706\": container with ID starting with 827206b5235940c6c74a447f55ee5f7490dc9dc72d0b019c19a05529105f2706 not found: ID does not exist" Jul 29 19:59:00.507518 ip-10-0-131-93 kubenswrapper[2564]: I0729 19:59:00.507492 2564 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-ingress/e2e-api-b-ecc3b0-openshift-default-764746cf54-nrvc2"] Jul 29 19:59:00.512020 ip-10-0-131-93 kubenswrapper[2564]: I0729 19:59:00.511997 2564 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["openshift-ingress/e2e-api-b-ecc3b0-openshift-default-764746cf54-nrvc2"] Jul 29 19:59:00.783791 ip-10-0-131-93 kubenswrapper[2564]: I0729 19:59:00.783715 2564 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="f445712e-6292-49cc-a59b-59ee1015edea" path="/var/lib/kubelet/pods/f445712e-6292-49cc-a59b-59ee1015edea/volumes" Jul 29 19:59:11.542761 ip-10-0-131-93 kubenswrapper[2564]: I0729 19:59:11.542670 2564 generic.go:356] "Generic (PLEG): container finished" podID="c9bc01d1-d52b-4476-942e-68c5a3641d5f" containerID="902c38b67ac57d88a03146f71d38d5110d4192d6f459be20582131647463183b" exitCode=0 Jul 29 19:59:11.542761 ip-10-0-131-93 kubenswrapper[2564]: I0729 19:59:11.542731 2564 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-shared-a-641e39-openshift-default-7898445948-2z4ct" event={"ID":"c9bc01d1-d52b-4476-942e-68c5a3641d5f","Type":"ContainerDied","Data":"902c38b67ac57d88a03146f71d38d5110d4192d6f459be20582131647463183b"} Jul 29 19:59:11.543318 ip-10-0-131-93 kubenswrapper[2564]: I0729 19:59:11.543192 2564 scope.go:117] "RemoveContainer" containerID="902c38b67ac57d88a03146f71d38d5110d4192d6f459be20582131647463183b" Jul 29 19:59:11.544210 ip-10-0-131-93 kubenswrapper[2564]: I0729 19:59:11.544179 2564 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236224Ki","pods":"250"} Jul 29 19:59:11.544281 ip-10-0-131-93 kubenswrapper[2564]: I0729 19:59:11.544252 2564 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236224Ki","pods":"250"} Jul 29 19:59:11.544320 ip-10-0-131-93 kubenswrapper[2564]: I0729 19:59:11.544286 2564 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236224Ki","pods":"250"} Jul 29 19:59:12.468812 ip-10-0-131-93 kubenswrapper[2564]: I0729 19:59:12.468767 2564 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/e2e-shared-a-641e39-openshift-default-7898445948-2z4ct" Jul 29 19:59:12.550153 ip-10-0-131-93 kubenswrapper[2564]: I0729 19:59:12.550113 2564 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-shared-a-641e39-openshift-default-7898445948-2z4ct" event={"ID":"c9bc01d1-d52b-4476-942e-68c5a3641d5f","Type":"ContainerStarted","Data":"56035649b95b5c07685ccb8f7e916ce8f650b2af416e214a04399a3350a20e78"} Jul 29 19:59:12.749023 ip-10-0-131-93 kubenswrapper[2564]: I0729 19:59:12.748938 2564 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="openshift-ingress/e2e-shared-a-641e39-openshift-default-7898445948-2z4ct" Jul 29 19:59:12.750238 ip-10-0-131-93 kubenswrapper[2564]: I0729 19:59:12.750208 2564 patch_prober.go:28] interesting pod/e2e-shared-a-641e39-openshift-default-7898445948-2z4ct container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.132.0.47:15021/healthz/ready\": dial tcp 10.132.0.47:15021: connect: connection refused" start-of-body= Jul 29 19:59:12.750387 ip-10-0-131-93 kubenswrapper[2564]: I0729 19:59:12.750258 2564 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-shared-a-641e39-openshift-default-7898445948-2z4ct" podUID="c9bc01d1-d52b-4476-942e-68c5a3641d5f" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.132.0.47:15021/healthz/ready\": dial tcp 10.132.0.47:15021: connect: connection refused" Jul 29 19:59:13.748993 ip-10-0-131-93 kubenswrapper[2564]: I0729 19:59:13.748959 2564 patch_prober.go:28] interesting pod/e2e-shared-a-641e39-openshift-default-7898445948-2z4ct container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.132.0.47:15021/healthz/ready\": dial tcp 10.132.0.47:15021: connect: connection refused" start-of-body= Jul 29 19:59:13.749419 ip-10-0-131-93 kubenswrapper[2564]: I0729 19:59:13.749021 2564 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-shared-a-641e39-openshift-default-7898445948-2z4ct" podUID="c9bc01d1-d52b-4476-942e-68c5a3641d5f" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.132.0.47:15021/healthz/ready\": dial tcp 10.132.0.47:15021: connect: connection refused" Jul 29 19:59:14.749168 ip-10-0-131-93 kubenswrapper[2564]: I0729 19:59:14.749128 2564 patch_prober.go:28] interesting pod/e2e-shared-a-641e39-openshift-default-7898445948-2z4ct container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.132.0.47:15021/healthz/ready\": dial tcp 10.132.0.47:15021: connect: connection refused" start-of-body= Jul 29 19:59:14.749691 ip-10-0-131-93 kubenswrapper[2564]: I0729 19:59:14.749205 2564 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-shared-a-641e39-openshift-default-7898445948-2z4ct" podUID="c9bc01d1-d52b-4476-942e-68c5a3641d5f" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.132.0.47:15021/healthz/ready\": dial tcp 10.132.0.47:15021: connect: connection refused" Jul 29 19:59:15.748234 ip-10-0-131-93 kubenswrapper[2564]: I0729 19:59:15.748198 2564 patch_prober.go:28] interesting pod/e2e-shared-a-641e39-openshift-default-7898445948-2z4ct container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.132.0.47:15021/healthz/ready\": dial tcp 10.132.0.47:15021: connect: connection refused" start-of-body= Jul 29 19:59:15.748438 ip-10-0-131-93 kubenswrapper[2564]: I0729 19:59:15.748255 2564 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-shared-a-641e39-openshift-default-7898445948-2z4ct" podUID="c9bc01d1-d52b-4476-942e-68c5a3641d5f" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.132.0.47:15021/healthz/ready\": dial tcp 10.132.0.47:15021: connect: connection refused" Jul 29 19:59:16.752027 ip-10-0-131-93 kubenswrapper[2564]: I0729 19:59:16.751997 2564 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="openshift-ingress/e2e-shared-a-641e39-openshift-default-7898445948-2z4ct" Jul 29 19:59:16.752547 ip-10-0-131-93 kubenswrapper[2564]: I0729 19:59:16.752259 2564 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/e2e-shared-a-641e39-openshift-default-7898445948-2z4ct" Jul 29 19:59:16.752885 ip-10-0-131-93 kubenswrapper[2564]: I0729 19:59:16.752868 2564 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/e2e-shared-a-641e39-openshift-default-7898445948-2z4ct" Jul 29 19:59:23.326293 ip-10-0-131-93 kubenswrapper[2564]: I0729 19:59:23.326259 2564 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-444jx_faa8c546-c75b-4e10-93b4-661aa2b5f53e/konnectivity-agent/0.log" limit=1024 Jul 29 19:59:23.652751 ip-10-0-131-93 kubenswrapper[2564]: I0729 19:59:23.652668 2564 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-444jx_faa8c546-c75b-4e10-93b4-661aa2b5f53e/konnectivity-agent/0.log" limit=1024 Jul 29 19:59:23.926072 ip-10-0-131-93 kubenswrapper[2564]: I0729 19:59:23.925992 2564 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-ingress/e2e-shared-b-4c787c-openshift-default-66676df75b-f49kx"] Jul 29 19:59:23.926467 ip-10-0-131-93 kubenswrapper[2564]: I0729 19:59:23.926438 2564 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="f445712e-6292-49cc-a59b-59ee1015edea" containerName="istio-proxy" Jul 29 19:59:23.926467 ip-10-0-131-93 kubenswrapper[2564]: I0729 19:59:23.926462 2564 state_mem.go:107] "Deleted CPUSet assignment" podUID="f445712e-6292-49cc-a59b-59ee1015edea" containerName="istio-proxy" Jul 29 19:59:23.926687 ip-10-0-131-93 kubenswrapper[2564]: I0729 19:59:23.926490 2564 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="f445712e-6292-49cc-a59b-59ee1015edea" containerName="istio-proxy" Jul 29 19:59:23.926687 ip-10-0-131-93 kubenswrapper[2564]: I0729 19:59:23.926499 2564 state_mem.go:107] "Deleted CPUSet assignment" podUID="f445712e-6292-49cc-a59b-59ee1015edea" containerName="istio-proxy" Jul 29 19:59:23.926687 ip-10-0-131-93 kubenswrapper[2564]: I0729 19:59:23.926588 2564 memory_manager.go:368] "RemoveStaleState removing state" podUID="f445712e-6292-49cc-a59b-59ee1015edea" containerName="istio-proxy" Jul 29 19:59:23.926687 ip-10-0-131-93 kubenswrapper[2564]: I0729 19:59:23.926602 2564 memory_manager.go:368] "RemoveStaleState removing state" podUID="f445712e-6292-49cc-a59b-59ee1015edea" containerName="istio-proxy" Jul 29 19:59:23.928789 ip-10-0-131-93 kubenswrapper[2564]: I0729 19:59:23.928770 2564 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-shared-b-4c787c-openshift-default-66676df75b-f49kx" Jul 29 19:59:23.931306 ip-10-0-131-93 kubenswrapper[2564]: I0729 19:59:23.931288 2564 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress\"/\"e2e-shared-b-4c787c-openshift-default-dockercfg-8wgd2\"" Jul 29 19:59:23.942609 ip-10-0-131-93 kubenswrapper[2564]: I0729 19:59:23.942588 2564 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/e2e-shared-b-4c787c-openshift-default-66676df75b-f49kx"] Jul 29 19:59:23.955303 ip-10-0-131-93 kubenswrapper[2564]: I0729 19:59:23.955275 2564 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-444jx_faa8c546-c75b-4e10-93b4-661aa2b5f53e/konnectivity-agent/0.log" limit=1024 Jul 29 19:59:24.022330 ip-10-0-131-93 kubenswrapper[2564]: I0729 19:59:24.022291 2564 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-tsrpp\" (UniqueName: \"kubernetes.io/projected/15f29128-6b6b-4023-b057-184a58fe96cc-kube-api-access-tsrpp\") pod \"e2e-shared-b-4c787c-openshift-default-66676df75b-f49kx\" (UID: \"15f29128-6b6b-4023-b057-184a58fe96cc\") " pod="openshift-ingress/e2e-shared-b-4c787c-openshift-default-66676df75b-f49kx" Jul 29 19:59:24.022330 ip-10-0-131-93 kubenswrapper[2564]: I0729 19:59:24.022332 2564 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/15f29128-6b6b-4023-b057-184a58fe96cc-istio-data\") pod \"e2e-shared-b-4c787c-openshift-default-66676df75b-f49kx\" (UID: \"15f29128-6b6b-4023-b057-184a58fe96cc\") " pod="openshift-ingress/e2e-shared-b-4c787c-openshift-default-66676df75b-f49kx" Jul 29 19:59:24.022553 ip-10-0-131-93 kubenswrapper[2564]: I0729 19:59:24.022357 2564 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/15f29128-6b6b-4023-b057-184a58fe96cc-workload-certs\") pod \"e2e-shared-b-4c787c-openshift-default-66676df75b-f49kx\" (UID: \"15f29128-6b6b-4023-b057-184a58fe96cc\") " pod="openshift-ingress/e2e-shared-b-4c787c-openshift-default-66676df75b-f49kx" Jul 29 19:59:24.022553 ip-10-0-131-93 kubenswrapper[2564]: I0729 19:59:24.022399 2564 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/15f29128-6b6b-4023-b057-184a58fe96cc-istio-podinfo\") pod \"e2e-shared-b-4c787c-openshift-default-66676df75b-f49kx\" (UID: \"15f29128-6b6b-4023-b057-184a58fe96cc\") " pod="openshift-ingress/e2e-shared-b-4c787c-openshift-default-66676df75b-f49kx" Jul 29 19:59:24.022553 ip-10-0-131-93 kubenswrapper[2564]: I0729 19:59:24.022477 2564 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/15f29128-6b6b-4023-b057-184a58fe96cc-istio-token\") pod \"e2e-shared-b-4c787c-openshift-default-66676df75b-f49kx\" (UID: \"15f29128-6b6b-4023-b057-184a58fe96cc\") " pod="openshift-ingress/e2e-shared-b-4c787c-openshift-default-66676df75b-f49kx" Jul 29 19:59:24.022553 ip-10-0-131-93 kubenswrapper[2564]: I0729 19:59:24.022529 2564 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/15f29128-6b6b-4023-b057-184a58fe96cc-workload-socket\") pod \"e2e-shared-b-4c787c-openshift-default-66676df75b-f49kx\" (UID: \"15f29128-6b6b-4023-b057-184a58fe96cc\") " pod="openshift-ingress/e2e-shared-b-4c787c-openshift-default-66676df75b-f49kx" Jul 29 19:59:24.022700 ip-10-0-131-93 kubenswrapper[2564]: I0729 19:59:24.022598 2564 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/15f29128-6b6b-4023-b057-184a58fe96cc-credential-socket\") pod \"e2e-shared-b-4c787c-openshift-default-66676df75b-f49kx\" (UID: \"15f29128-6b6b-4023-b057-184a58fe96cc\") " pod="openshift-ingress/e2e-shared-b-4c787c-openshift-default-66676df75b-f49kx" Jul 29 19:59:24.022700 ip-10-0-131-93 kubenswrapper[2564]: I0729 19:59:24.022684 2564 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/15f29128-6b6b-4023-b057-184a58fe96cc-istio-envoy\") pod \"e2e-shared-b-4c787c-openshift-default-66676df75b-f49kx\" (UID: \"15f29128-6b6b-4023-b057-184a58fe96cc\") " pod="openshift-ingress/e2e-shared-b-4c787c-openshift-default-66676df75b-f49kx" Jul 29 19:59:24.022780 ip-10-0-131-93 kubenswrapper[2564]: I0729 19:59:24.022711 2564 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/15f29128-6b6b-4023-b057-184a58fe96cc-istiod-ca-cert\") pod \"e2e-shared-b-4c787c-openshift-default-66676df75b-f49kx\" (UID: \"15f29128-6b6b-4023-b057-184a58fe96cc\") " pod="openshift-ingress/e2e-shared-b-4c787c-openshift-default-66676df75b-f49kx" Jul 29 19:59:24.123788 ip-10-0-131-93 kubenswrapper[2564]: I0729 19:59:24.123753 2564 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/15f29128-6b6b-4023-b057-184a58fe96cc-credential-socket\") pod \"e2e-shared-b-4c787c-openshift-default-66676df75b-f49kx\" (UID: \"15f29128-6b6b-4023-b057-184a58fe96cc\") " pod="openshift-ingress/e2e-shared-b-4c787c-openshift-default-66676df75b-f49kx" Jul 29 19:59:24.123963 ip-10-0-131-93 kubenswrapper[2564]: I0729 19:59:24.123818 2564 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/15f29128-6b6b-4023-b057-184a58fe96cc-istio-envoy\") pod \"e2e-shared-b-4c787c-openshift-default-66676df75b-f49kx\" (UID: \"15f29128-6b6b-4023-b057-184a58fe96cc\") " pod="openshift-ingress/e2e-shared-b-4c787c-openshift-default-66676df75b-f49kx" Jul 29 19:59:24.123963 ip-10-0-131-93 kubenswrapper[2564]: I0729 19:59:24.123842 2564 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/15f29128-6b6b-4023-b057-184a58fe96cc-istiod-ca-cert\") pod \"e2e-shared-b-4c787c-openshift-default-66676df75b-f49kx\" (UID: \"15f29128-6b6b-4023-b057-184a58fe96cc\") " pod="openshift-ingress/e2e-shared-b-4c787c-openshift-default-66676df75b-f49kx" Jul 29 19:59:24.123963 ip-10-0-131-93 kubenswrapper[2564]: I0729 19:59:24.123872 2564 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-tsrpp\" (UniqueName: \"kubernetes.io/projected/15f29128-6b6b-4023-b057-184a58fe96cc-kube-api-access-tsrpp\") pod \"e2e-shared-b-4c787c-openshift-default-66676df75b-f49kx\" (UID: \"15f29128-6b6b-4023-b057-184a58fe96cc\") " pod="openshift-ingress/e2e-shared-b-4c787c-openshift-default-66676df75b-f49kx" Jul 29 19:59:24.123963 ip-10-0-131-93 kubenswrapper[2564]: I0729 19:59:24.123893 2564 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/15f29128-6b6b-4023-b057-184a58fe96cc-istio-data\") pod \"e2e-shared-b-4c787c-openshift-default-66676df75b-f49kx\" (UID: \"15f29128-6b6b-4023-b057-184a58fe96cc\") " pod="openshift-ingress/e2e-shared-b-4c787c-openshift-default-66676df75b-f49kx" Jul 29 19:59:24.123963 ip-10-0-131-93 kubenswrapper[2564]: I0729 19:59:24.123918 2564 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/15f29128-6b6b-4023-b057-184a58fe96cc-workload-certs\") pod \"e2e-shared-b-4c787c-openshift-default-66676df75b-f49kx\" (UID: \"15f29128-6b6b-4023-b057-184a58fe96cc\") " pod="openshift-ingress/e2e-shared-b-4c787c-openshift-default-66676df75b-f49kx" Jul 29 19:59:24.123963 ip-10-0-131-93 kubenswrapper[2564]: I0729 19:59:24.123943 2564 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/15f29128-6b6b-4023-b057-184a58fe96cc-istio-podinfo\") pod \"e2e-shared-b-4c787c-openshift-default-66676df75b-f49kx\" (UID: \"15f29128-6b6b-4023-b057-184a58fe96cc\") " pod="openshift-ingress/e2e-shared-b-4c787c-openshift-default-66676df75b-f49kx" Jul 29 19:59:24.124299 ip-10-0-131-93 kubenswrapper[2564]: I0729 19:59:24.123970 2564 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/15f29128-6b6b-4023-b057-184a58fe96cc-istio-token\") pod \"e2e-shared-b-4c787c-openshift-default-66676df75b-f49kx\" (UID: \"15f29128-6b6b-4023-b057-184a58fe96cc\") " pod="openshift-ingress/e2e-shared-b-4c787c-openshift-default-66676df75b-f49kx" Jul 29 19:59:24.124299 ip-10-0-131-93 kubenswrapper[2564]: I0729 19:59:24.124006 2564 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/15f29128-6b6b-4023-b057-184a58fe96cc-workload-socket\") pod \"e2e-shared-b-4c787c-openshift-default-66676df75b-f49kx\" (UID: \"15f29128-6b6b-4023-b057-184a58fe96cc\") " pod="openshift-ingress/e2e-shared-b-4c787c-openshift-default-66676df75b-f49kx" Jul 29 19:59:24.124299 ip-10-0-131-93 kubenswrapper[2564]: I0729 19:59:24.124189 2564 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/15f29128-6b6b-4023-b057-184a58fe96cc-credential-socket\") pod \"e2e-shared-b-4c787c-openshift-default-66676df75b-f49kx\" (UID: \"15f29128-6b6b-4023-b057-184a58fe96cc\") " pod="openshift-ingress/e2e-shared-b-4c787c-openshift-default-66676df75b-f49kx" Jul 29 19:59:24.124299 ip-10-0-131-93 kubenswrapper[2564]: I0729 19:59:24.124269 2564 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/15f29128-6b6b-4023-b057-184a58fe96cc-istio-data\") pod \"e2e-shared-b-4c787c-openshift-default-66676df75b-f49kx\" (UID: \"15f29128-6b6b-4023-b057-184a58fe96cc\") " pod="openshift-ingress/e2e-shared-b-4c787c-openshift-default-66676df75b-f49kx" Jul 29 19:59:24.124591 ip-10-0-131-93 kubenswrapper[2564]: I0729 19:59:24.124351 2564 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/15f29128-6b6b-4023-b057-184a58fe96cc-workload-certs\") pod \"e2e-shared-b-4c787c-openshift-default-66676df75b-f49kx\" (UID: \"15f29128-6b6b-4023-b057-184a58fe96cc\") " pod="openshift-ingress/e2e-shared-b-4c787c-openshift-default-66676df75b-f49kx" Jul 29 19:59:24.124649 ip-10-0-131-93 kubenswrapper[2564]: I0729 19:59:24.124601 2564 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/15f29128-6b6b-4023-b057-184a58fe96cc-workload-socket\") pod \"e2e-shared-b-4c787c-openshift-default-66676df75b-f49kx\" (UID: \"15f29128-6b6b-4023-b057-184a58fe96cc\") " pod="openshift-ingress/e2e-shared-b-4c787c-openshift-default-66676df75b-f49kx" Jul 29 19:59:24.124698 ip-10-0-131-93 kubenswrapper[2564]: I0729 19:59:24.124648 2564 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/15f29128-6b6b-4023-b057-184a58fe96cc-istiod-ca-cert\") pod \"e2e-shared-b-4c787c-openshift-default-66676df75b-f49kx\" (UID: \"15f29128-6b6b-4023-b057-184a58fe96cc\") " pod="openshift-ingress/e2e-shared-b-4c787c-openshift-default-66676df75b-f49kx" Jul 29 19:59:24.126144 ip-10-0-131-93 kubenswrapper[2564]: I0729 19:59:24.126122 2564 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/15f29128-6b6b-4023-b057-184a58fe96cc-istio-envoy\") pod \"e2e-shared-b-4c787c-openshift-default-66676df75b-f49kx\" (UID: \"15f29128-6b6b-4023-b057-184a58fe96cc\") " pod="openshift-ingress/e2e-shared-b-4c787c-openshift-default-66676df75b-f49kx" Jul 29 19:59:24.126397 ip-10-0-131-93 kubenswrapper[2564]: I0729 19:59:24.126355 2564 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/15f29128-6b6b-4023-b057-184a58fe96cc-istio-podinfo\") pod \"e2e-shared-b-4c787c-openshift-default-66676df75b-f49kx\" (UID: \"15f29128-6b6b-4023-b057-184a58fe96cc\") " pod="openshift-ingress/e2e-shared-b-4c787c-openshift-default-66676df75b-f49kx" Jul 29 19:59:24.148325 ip-10-0-131-93 kubenswrapper[2564]: I0729 19:59:24.148304 2564 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/15f29128-6b6b-4023-b057-184a58fe96cc-istio-token\") pod \"e2e-shared-b-4c787c-openshift-default-66676df75b-f49kx\" (UID: \"15f29128-6b6b-4023-b057-184a58fe96cc\") " pod="openshift-ingress/e2e-shared-b-4c787c-openshift-default-66676df75b-f49kx" Jul 29 19:59:24.149813 ip-10-0-131-93 kubenswrapper[2564]: I0729 19:59:24.149795 2564 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-tsrpp\" (UniqueName: \"kubernetes.io/projected/15f29128-6b6b-4023-b057-184a58fe96cc-kube-api-access-tsrpp\") pod \"e2e-shared-b-4c787c-openshift-default-66676df75b-f49kx\" (UID: \"15f29128-6b6b-4023-b057-184a58fe96cc\") " pod="openshift-ingress/e2e-shared-b-4c787c-openshift-default-66676df75b-f49kx" Jul 29 19:59:24.238687 ip-10-0-131-93 kubenswrapper[2564]: I0729 19:59:24.238662 2564 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-shared-b-4c787c-openshift-default-66676df75b-f49kx" Jul 29 19:59:24.384475 ip-10-0-131-93 kubenswrapper[2564]: I0729 19:59:24.384449 2564 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/e2e-shared-b-4c787c-openshift-default-66676df75b-f49kx"] Jul 29 19:59:24.385654 ip-10-0-131-93 kubenswrapper[2564]: W0729 19:59:24.385633 2564 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod15f29128_6b6b_4023_b057_184a58fe96cc.slice/crio-413990922ed5586d8e2b3b366f5e1b45c857d6ba5e423a264023df3215f54f37 WatchSource:0}: Error finding container 413990922ed5586d8e2b3b366f5e1b45c857d6ba5e423a264023df3215f54f37: Status 404 returned error can't find the container with id 413990922ed5586d8e2b3b366f5e1b45c857d6ba5e423a264023df3215f54f37 Jul 29 19:59:24.387778 ip-10-0-131-93 kubenswrapper[2564]: I0729 19:59:24.387745 2564 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236224Ki","pods":"250"} Jul 29 19:59:24.387859 ip-10-0-131-93 kubenswrapper[2564]: I0729 19:59:24.387806 2564 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236224Ki","pods":"250"} Jul 29 19:59:24.387859 ip-10-0-131-93 kubenswrapper[2564]: I0729 19:59:24.387836 2564 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236224Ki","pods":"250"} Jul 29 19:59:24.400867 ip-10-0-131-93 kubenswrapper[2564]: I0729 19:59:24.400847 2564 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-444jx_faa8c546-c75b-4e10-93b4-661aa2b5f53e/konnectivity-agent/0.log" limit=1024 Jul 29 19:59:24.608519 ip-10-0-131-93 kubenswrapper[2564]: I0729 19:59:24.608421 2564 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-shared-b-4c787c-openshift-default-66676df75b-f49kx" event={"ID":"15f29128-6b6b-4023-b057-184a58fe96cc","Type":"ContainerStarted","Data":"61942868a82e4552e3c5485e29efe49c8886ea19ab46b357eb2af8f7de8ab63a"} Jul 29 19:59:24.608519 ip-10-0-131-93 kubenswrapper[2564]: I0729 19:59:24.608476 2564 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-shared-b-4c787c-openshift-default-66676df75b-f49kx" event={"ID":"15f29128-6b6b-4023-b057-184a58fe96cc","Type":"ContainerStarted","Data":"413990922ed5586d8e2b3b366f5e1b45c857d6ba5e423a264023df3215f54f37"} Jul 29 19:59:24.632322 ip-10-0-131-93 kubenswrapper[2564]: I0729 19:59:24.632253 2564 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-ingress/e2e-shared-b-4c787c-openshift-default-66676df75b-f49kx" podStartSLOduration=1.6322317480000001 podStartE2EDuration="1.632231748s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-29 19:59:23 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-29 19:59:24.629301171 +0000 UTC m=+2540.353360867" watchObservedRunningTime="2026-07-29 19:59:24.632231748 +0000 UTC m=+2540.356291440" Jul 29 19:59:25.239024 ip-10-0-131-93 kubenswrapper[2564]: I0729 19:59:25.238988 2564 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="openshift-ingress/e2e-shared-b-4c787c-openshift-default-66676df75b-f49kx" Jul 29 19:59:25.243529 ip-10-0-131-93 kubenswrapper[2564]: I0729 19:59:25.243502 2564 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="openshift-ingress/e2e-shared-b-4c787c-openshift-default-66676df75b-f49kx" Jul 29 19:59:25.613484 ip-10-0-131-93 kubenswrapper[2564]: I0729 19:59:25.613407 2564 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/e2e-shared-b-4c787c-openshift-default-66676df75b-f49kx" Jul 29 19:59:25.614410 ip-10-0-131-93 kubenswrapper[2564]: I0729 19:59:25.614392 2564 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/e2e-shared-b-4c787c-openshift-default-66676df75b-f49kx" Jul 29 19:59:26.500412 ip-10-0-131-93 kubenswrapper[2564]: I0729 19:59:26.500357 2564 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-444jx_faa8c546-c75b-4e10-93b4-661aa2b5f53e/konnectivity-agent/0.log" limit=1024 Jul 29 19:59:26.600661 ip-10-0-131-93 kubenswrapper[2564]: I0729 19:59:26.600626 2564 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-444jx_faa8c546-c75b-4e10-93b4-661aa2b5f53e/konnectivity-agent/0.log" limit=1024 Jul 29 19:59:27.632811 ip-10-0-131-93 kubenswrapper[2564]: I0729 19:59:27.632770 2564 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-444jx_faa8c546-c75b-4e10-93b4-661aa2b5f53e/konnectivity-agent/0.log" limit=1024 Jul 29 19:59:27.743549 ip-10-0-131-93 kubenswrapper[2564]: I0729 19:59:27.743513 2564 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-444jx_faa8c546-c75b-4e10-93b4-661aa2b5f53e/konnectivity-agent/0.log" limit=1024 Jul 29 19:59:28.179860 ip-10-0-131-93 kubenswrapper[2564]: I0729 19:59:28.179826 2564 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-444jx_faa8c546-c75b-4e10-93b4-661aa2b5f53e/konnectivity-agent/0.log" limit=1024 Jul 29 19:59:28.280446 ip-10-0-131-93 kubenswrapper[2564]: I0729 19:59:28.280403 2564 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-444jx_faa8c546-c75b-4e10-93b4-661aa2b5f53e/konnectivity-agent/0.log" limit=1024 Jul 29 19:59:41.176610 ip-10-0-131-93 kubenswrapper[2564]: I0729 19:59:41.176153 2564 patch_prober.go:28] interesting pod/e2e-shared-b-4c787c-openshift-default-66676df75b-f49kx container/istio-proxy namespace/openshift-ingress: Readiness probe status=failure output="Get \"http://10.132.0.48:15021/healthz/ready\": read tcp 10.132.0.2:56656->10.132.0.48:15021: read: connection reset by peer" start-of-body= Jul 29 19:59:41.176610 ip-10-0-131-93 kubenswrapper[2564]: I0729 19:59:41.176259 2564 prober.go:120] "Probe failed" probeType="Readiness" pod="openshift-ingress/e2e-shared-b-4c787c-openshift-default-66676df75b-f49kx" podUID="15f29128-6b6b-4023-b057-184a58fe96cc" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.132.0.48:15021/healthz/ready\": read tcp 10.132.0.2:56656->10.132.0.48:15021: read: connection reset by peer" Jul 29 19:59:41.694951 ip-10-0-131-93 kubenswrapper[2564]: I0729 19:59:41.694918 2564 generic.go:356] "Generic (PLEG): container finished" podID="15f29128-6b6b-4023-b057-184a58fe96cc" containerID="61942868a82e4552e3c5485e29efe49c8886ea19ab46b357eb2af8f7de8ab63a" exitCode=0 Jul 29 19:59:41.695135 ip-10-0-131-93 kubenswrapper[2564]: I0729 19:59:41.694993 2564 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-shared-b-4c787c-openshift-default-66676df75b-f49kx" event={"ID":"15f29128-6b6b-4023-b057-184a58fe96cc","Type":"ContainerDied","Data":"61942868a82e4552e3c5485e29efe49c8886ea19ab46b357eb2af8f7de8ab63a"} Jul 29 19:59:41.695412 ip-10-0-131-93 kubenswrapper[2564]: I0729 19:59:41.695384 2564 scope.go:117] "RemoveContainer" containerID="61942868a82e4552e3c5485e29efe49c8886ea19ab46b357eb2af8f7de8ab63a" Jul 29 19:59:41.696385 ip-10-0-131-93 kubenswrapper[2564]: I0729 19:59:41.696345 2564 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236224Ki","pods":"250"} Jul 29 19:59:41.696453 ip-10-0-131-93 kubenswrapper[2564]: I0729 19:59:41.696425 2564 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236224Ki","pods":"250"} Jul 29 19:59:41.696496 ip-10-0-131-93 kubenswrapper[2564]: I0729 19:59:41.696459 2564 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236224Ki","pods":"250"} Jul 29 19:59:42.703902 ip-10-0-131-93 kubenswrapper[2564]: I0729 19:59:42.703866 2564 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-shared-b-4c787c-openshift-default-66676df75b-f49kx" event={"ID":"15f29128-6b6b-4023-b057-184a58fe96cc","Type":"ContainerStarted","Data":"6777daae1396096745b9593744e33d0f5ceb2e44a014d3532e60a488ab259e97"} Jul 29 19:59:43.239207 ip-10-0-131-93 kubenswrapper[2564]: I0729 19:59:43.239169 2564 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="openshift-ingress/e2e-shared-b-4c787c-openshift-default-66676df75b-f49kx" Jul 29 19:59:43.239437 ip-10-0-131-93 kubenswrapper[2564]: I0729 19:59:43.239414 2564 patch_prober.go:28] interesting pod/e2e-shared-b-4c787c-openshift-default-66676df75b-f49kx container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.132.0.48:15021/healthz/ready\": dial tcp 10.132.0.48:15021: connect: connection refused" start-of-body= Jul 29 19:59:43.239493 ip-10-0-131-93 kubenswrapper[2564]: I0729 19:59:43.239465 2564 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-shared-b-4c787c-openshift-default-66676df75b-f49kx" podUID="15f29128-6b6b-4023-b057-184a58fe96cc" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.132.0.48:15021/healthz/ready\": dial tcp 10.132.0.48:15021: connect: connection refused" Jul 29 19:59:44.239882 ip-10-0-131-93 kubenswrapper[2564]: I0729 19:59:44.239843 2564 patch_prober.go:28] interesting pod/e2e-shared-b-4c787c-openshift-default-66676df75b-f49kx container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.132.0.48:15021/healthz/ready\": dial tcp 10.132.0.48:15021: connect: connection refused" start-of-body= Jul 29 19:59:44.240270 ip-10-0-131-93 kubenswrapper[2564]: I0729 19:59:44.239900 2564 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-shared-b-4c787c-openshift-default-66676df75b-f49kx" podUID="15f29128-6b6b-4023-b057-184a58fe96cc" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.132.0.48:15021/healthz/ready\": dial tcp 10.132.0.48:15021: connect: connection refused" Jul 29 19:59:45.239651 ip-10-0-131-93 kubenswrapper[2564]: I0729 19:59:45.239617 2564 patch_prober.go:28] interesting pod/e2e-shared-b-4c787c-openshift-default-66676df75b-f49kx container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.132.0.48:15021/healthz/ready\": dial tcp 10.132.0.48:15021: connect: connection refused" start-of-body= Jul 29 19:59:45.239836 ip-10-0-131-93 kubenswrapper[2564]: I0729 19:59:45.239683 2564 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-shared-b-4c787c-openshift-default-66676df75b-f49kx" podUID="15f29128-6b6b-4023-b057-184a58fe96cc" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.132.0.48:15021/healthz/ready\": dial tcp 10.132.0.48:15021: connect: connection refused" Jul 29 19:59:46.239645 ip-10-0-131-93 kubenswrapper[2564]: I0729 19:59:46.239608 2564 patch_prober.go:28] interesting pod/e2e-shared-b-4c787c-openshift-default-66676df75b-f49kx container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.132.0.48:15021/healthz/ready\": dial tcp 10.132.0.48:15021: connect: connection refused" start-of-body= Jul 29 19:59:46.240016 ip-10-0-131-93 kubenswrapper[2564]: I0729 19:59:46.239661 2564 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-shared-b-4c787c-openshift-default-66676df75b-f49kx" podUID="15f29128-6b6b-4023-b057-184a58fe96cc" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.132.0.48:15021/healthz/ready\": dial tcp 10.132.0.48:15021: connect: connection refused" Jul 29 19:59:47.242421 ip-10-0-131-93 kubenswrapper[2564]: I0729 19:59:47.242367 2564 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="openshift-ingress/e2e-shared-b-4c787c-openshift-default-66676df75b-f49kx" Jul 29 19:59:47.242962 ip-10-0-131-93 kubenswrapper[2564]: I0729 19:59:47.242678 2564 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/e2e-shared-b-4c787c-openshift-default-66676df75b-f49kx" Jul 29 19:59:47.243475 ip-10-0-131-93 kubenswrapper[2564]: I0729 19:59:47.243454 2564 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/e2e-shared-b-4c787c-openshift-default-66676df75b-f49kx" Jul 29 19:59:51.475209 ip-10-0-131-93 kubenswrapper[2564]: I0729 19:59:51.475178 2564 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-444jx_faa8c546-c75b-4e10-93b4-661aa2b5f53e/konnectivity-agent/0.log" limit=1024 Jul 29 20:00:00.174050 ip-10-0-131-93 kubenswrapper[2564]: I0729 20:00:00.174001 2564 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-444jx_faa8c546-c75b-4e10-93b4-661aa2b5f53e/konnectivity-agent/0.log" limit=1024 Jul 29 20:00:30.671019 ip-10-0-131-93 kubenswrapper[2564]: I0729 20:00:30.670948 2564 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-444jx_faa8c546-c75b-4e10-93b4-661aa2b5f53e/konnectivity-agent/0.log" limit=1024 Jul 29 20:00:38.454074 ip-10-0-131-93 kubenswrapper[2564]: I0729 20:00:38.454025 2564 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-444jx_faa8c546-c75b-4e10-93b4-661aa2b5f53e/konnectivity-agent/0.log" limit=1024 Jul 29 20:00:38.504280 ip-10-0-131-93 kubenswrapper[2564]: I0729 20:00:38.504247 2564 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-444jx_faa8c546-c75b-4e10-93b4-661aa2b5f53e/konnectivity-agent/0.log" limit=1024 Jul 29 20:02:22.423195 ip-10-0-131-93 kubenswrapper[2564]: I0729 20:02:22.423104 2564 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-444jx_faa8c546-c75b-4e10-93b4-661aa2b5f53e/konnectivity-agent/0.log" limit=1024 Jul 29 20:03:48.112466 ip-10-0-131-93 kubenswrapper[2564]: I0729 20:03:48.112391 2564 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-444jx_faa8c546-c75b-4e10-93b4-661aa2b5f53e/konnectivity-agent/0.log" limit=1024 Jul 29 20:03:55.388583 ip-10-0-131-93 kubenswrapper[2564]: I0729 20:03:55.388550 2564 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-444jx_faa8c546-c75b-4e10-93b4-661aa2b5f53e/konnectivity-agent/0.log" limit=1024 Jul 29 20:04:28.799018 ip-10-0-131-93 kubenswrapper[2564]: I0729 20:04:28.798984 2564 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-444jx_faa8c546-c75b-4e10-93b4-661aa2b5f53e/konnectivity-agent/0.log" limit=1024 Jul 29 20:04:47.437293 ip-10-0-131-93 kubenswrapper[2564]: I0729 20:04:47.437256 2564 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-444jx_faa8c546-c75b-4e10-93b4-661aa2b5f53e/konnectivity-agent/0.log" limit=1024 Jul 29 20:04:48.235152 ip-10-0-131-93 kubenswrapper[2564]: I0729 20:04:48.235114 2564 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-444jx_faa8c546-c75b-4e10-93b4-661aa2b5f53e/konnectivity-agent/0.log" limit=1024 Jul 29 20:04:59.531324 ip-10-0-131-93 kubenswrapper[2564]: I0729 20:04:59.531291 2564 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["ai-tenant-e2e-shared-a-641e39/sub-test-model-641e39-kserve-5dbc96c4f5-wm6n6"] Jul 29 20:04:59.533805 ip-10-0-131-93 kubenswrapper[2564]: I0729 20:04:59.533781 2564 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="ai-tenant-e2e-shared-a-641e39/sub-test-model-641e39-kserve-5dbc96c4f5-wm6n6" Jul 29 20:04:59.536564 ip-10-0-131-93 kubenswrapper[2564]: I0729 20:04:59.536543 2564 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"ai-tenant-e2e-shared-a-641e39\"/\"openshift-service-ca.crt\"" Jul 29 20:04:59.536749 ip-10-0-131-93 kubenswrapper[2564]: I0729 20:04:59.536733 2564 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"ai-tenant-e2e-shared-a-641e39\"/\"sub-test-model-641e39-kserve-self-signed-certs\"" Jul 29 20:04:59.537458 ip-10-0-131-93 kubenswrapper[2564]: I0729 20:04:59.537444 2564 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"ai-tenant-e2e-shared-a-641e39\"/\"kube-root-ca.crt\"" Jul 29 20:04:59.538067 ip-10-0-131-93 kubenswrapper[2564]: I0729 20:04:59.538046 2564 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"ai-tenant-e2e-shared-a-641e39\"/\"default-dockercfg-mgmzf\"" Jul 29 20:04:59.545986 ip-10-0-131-93 kubenswrapper[2564]: I0729 20:04:59.545965 2564 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["ai-tenant-e2e-shared-a-641e39/sub-test-model-641e39-kserve-5dbc96c4f5-wm6n6"] Jul 29 20:04:59.575295 ip-10-0-131-93 kubenswrapper[2564]: I0729 20:04:59.575267 2564 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-444jx_faa8c546-c75b-4e10-93b4-661aa2b5f53e/konnectivity-agent/0.log" limit=1024 Jul 29 20:04:59.648823 ip-10-0-131-93 kubenswrapper[2564]: I0729 20:04:59.648796 2564 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/8b638f30-2ab0-489d-a89a-5891322457cd-tls-certs\") pod \"sub-test-model-641e39-kserve-5dbc96c4f5-wm6n6\" (UID: \"8b638f30-2ab0-489d-a89a-5891322457cd\") " pod="ai-tenant-e2e-shared-a-641e39/sub-test-model-641e39-kserve-5dbc96c4f5-wm6n6" Jul 29 20:04:59.648967 ip-10-0-131-93 kubenswrapper[2564]: I0729 20:04:59.648835 2564 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/8b638f30-2ab0-489d-a89a-5891322457cd-home\") pod \"sub-test-model-641e39-kserve-5dbc96c4f5-wm6n6\" (UID: \"8b638f30-2ab0-489d-a89a-5891322457cd\") " pod="ai-tenant-e2e-shared-a-641e39/sub-test-model-641e39-kserve-5dbc96c4f5-wm6n6" Jul 29 20:04:59.648967 ip-10-0-131-93 kubenswrapper[2564]: I0729 20:04:59.648856 2564 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/8b638f30-2ab0-489d-a89a-5891322457cd-tmp-dir\") pod \"sub-test-model-641e39-kserve-5dbc96c4f5-wm6n6\" (UID: \"8b638f30-2ab0-489d-a89a-5891322457cd\") " pod="ai-tenant-e2e-shared-a-641e39/sub-test-model-641e39-kserve-5dbc96c4f5-wm6n6" Jul 29 20:04:59.649075 ip-10-0-131-93 kubenswrapper[2564]: I0729 20:04:59.648962 2564 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/8b638f30-2ab0-489d-a89a-5891322457cd-dshm\") pod \"sub-test-model-641e39-kserve-5dbc96c4f5-wm6n6\" (UID: \"8b638f30-2ab0-489d-a89a-5891322457cd\") " pod="ai-tenant-e2e-shared-a-641e39/sub-test-model-641e39-kserve-5dbc96c4f5-wm6n6" Jul 29 20:04:59.649075 ip-10-0-131-93 kubenswrapper[2564]: I0729 20:04:59.648984 2564 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/8b638f30-2ab0-489d-a89a-5891322457cd-model-cache\") pod \"sub-test-model-641e39-kserve-5dbc96c4f5-wm6n6\" (UID: \"8b638f30-2ab0-489d-a89a-5891322457cd\") " pod="ai-tenant-e2e-shared-a-641e39/sub-test-model-641e39-kserve-5dbc96c4f5-wm6n6" Jul 29 20:04:59.649075 ip-10-0-131-93 kubenswrapper[2564]: I0729 20:04:59.649026 2564 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-4gvs9\" (UniqueName: \"kubernetes.io/projected/8b638f30-2ab0-489d-a89a-5891322457cd-kube-api-access-4gvs9\") pod \"sub-test-model-641e39-kserve-5dbc96c4f5-wm6n6\" (UID: \"8b638f30-2ab0-489d-a89a-5891322457cd\") " pod="ai-tenant-e2e-shared-a-641e39/sub-test-model-641e39-kserve-5dbc96c4f5-wm6n6" Jul 29 20:04:59.749551 ip-10-0-131-93 kubenswrapper[2564]: I0729 20:04:59.749518 2564 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/8b638f30-2ab0-489d-a89a-5891322457cd-tls-certs\") pod \"sub-test-model-641e39-kserve-5dbc96c4f5-wm6n6\" (UID: \"8b638f30-2ab0-489d-a89a-5891322457cd\") " pod="ai-tenant-e2e-shared-a-641e39/sub-test-model-641e39-kserve-5dbc96c4f5-wm6n6" Jul 29 20:04:59.749745 ip-10-0-131-93 kubenswrapper[2564]: I0729 20:04:59.749559 2564 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/8b638f30-2ab0-489d-a89a-5891322457cd-home\") pod \"sub-test-model-641e39-kserve-5dbc96c4f5-wm6n6\" (UID: \"8b638f30-2ab0-489d-a89a-5891322457cd\") " pod="ai-tenant-e2e-shared-a-641e39/sub-test-model-641e39-kserve-5dbc96c4f5-wm6n6" Jul 29 20:04:59.749745 ip-10-0-131-93 kubenswrapper[2564]: I0729 20:04:59.749580 2564 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/8b638f30-2ab0-489d-a89a-5891322457cd-tmp-dir\") pod \"sub-test-model-641e39-kserve-5dbc96c4f5-wm6n6\" (UID: \"8b638f30-2ab0-489d-a89a-5891322457cd\") " pod="ai-tenant-e2e-shared-a-641e39/sub-test-model-641e39-kserve-5dbc96c4f5-wm6n6" Jul 29 20:04:59.749745 ip-10-0-131-93 kubenswrapper[2564]: I0729 20:04:59.749635 2564 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/8b638f30-2ab0-489d-a89a-5891322457cd-dshm\") pod \"sub-test-model-641e39-kserve-5dbc96c4f5-wm6n6\" (UID: \"8b638f30-2ab0-489d-a89a-5891322457cd\") " pod="ai-tenant-e2e-shared-a-641e39/sub-test-model-641e39-kserve-5dbc96c4f5-wm6n6" Jul 29 20:04:59.749745 ip-10-0-131-93 kubenswrapper[2564]: I0729 20:04:59.749656 2564 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/8b638f30-2ab0-489d-a89a-5891322457cd-model-cache\") pod \"sub-test-model-641e39-kserve-5dbc96c4f5-wm6n6\" (UID: \"8b638f30-2ab0-489d-a89a-5891322457cd\") " pod="ai-tenant-e2e-shared-a-641e39/sub-test-model-641e39-kserve-5dbc96c4f5-wm6n6" Jul 29 20:04:59.749965 ip-10-0-131-93 kubenswrapper[2564]: I0729 20:04:59.749803 2564 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-4gvs9\" (UniqueName: \"kubernetes.io/projected/8b638f30-2ab0-489d-a89a-5891322457cd-kube-api-access-4gvs9\") pod \"sub-test-model-641e39-kserve-5dbc96c4f5-wm6n6\" (UID: \"8b638f30-2ab0-489d-a89a-5891322457cd\") " pod="ai-tenant-e2e-shared-a-641e39/sub-test-model-641e39-kserve-5dbc96c4f5-wm6n6" Jul 29 20:04:59.750023 ip-10-0-131-93 kubenswrapper[2564]: I0729 20:04:59.749982 2564 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/8b638f30-2ab0-489d-a89a-5891322457cd-home\") pod \"sub-test-model-641e39-kserve-5dbc96c4f5-wm6n6\" (UID: \"8b638f30-2ab0-489d-a89a-5891322457cd\") " pod="ai-tenant-e2e-shared-a-641e39/sub-test-model-641e39-kserve-5dbc96c4f5-wm6n6" Jul 29 20:04:59.750023 ip-10-0-131-93 kubenswrapper[2564]: I0729 20:04:59.750006 2564 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/8b638f30-2ab0-489d-a89a-5891322457cd-model-cache\") pod \"sub-test-model-641e39-kserve-5dbc96c4f5-wm6n6\" (UID: \"8b638f30-2ab0-489d-a89a-5891322457cd\") " pod="ai-tenant-e2e-shared-a-641e39/sub-test-model-641e39-kserve-5dbc96c4f5-wm6n6" Jul 29 20:04:59.750123 ip-10-0-131-93 kubenswrapper[2564]: I0729 20:04:59.750040 2564 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/8b638f30-2ab0-489d-a89a-5891322457cd-tmp-dir\") pod \"sub-test-model-641e39-kserve-5dbc96c4f5-wm6n6\" (UID: \"8b638f30-2ab0-489d-a89a-5891322457cd\") " pod="ai-tenant-e2e-shared-a-641e39/sub-test-model-641e39-kserve-5dbc96c4f5-wm6n6" Jul 29 20:04:59.751992 ip-10-0-131-93 kubenswrapper[2564]: I0729 20:04:59.751962 2564 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/8b638f30-2ab0-489d-a89a-5891322457cd-dshm\") pod \"sub-test-model-641e39-kserve-5dbc96c4f5-wm6n6\" (UID: \"8b638f30-2ab0-489d-a89a-5891322457cd\") " pod="ai-tenant-e2e-shared-a-641e39/sub-test-model-641e39-kserve-5dbc96c4f5-wm6n6" Jul 29 20:04:59.752146 ip-10-0-131-93 kubenswrapper[2564]: I0729 20:04:59.752126 2564 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/8b638f30-2ab0-489d-a89a-5891322457cd-tls-certs\") pod \"sub-test-model-641e39-kserve-5dbc96c4f5-wm6n6\" (UID: \"8b638f30-2ab0-489d-a89a-5891322457cd\") " pod="ai-tenant-e2e-shared-a-641e39/sub-test-model-641e39-kserve-5dbc96c4f5-wm6n6" Jul 29 20:04:59.757786 ip-10-0-131-93 kubenswrapper[2564]: I0729 20:04:59.757769 2564 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-4gvs9\" (UniqueName: \"kubernetes.io/projected/8b638f30-2ab0-489d-a89a-5891322457cd-kube-api-access-4gvs9\") pod \"sub-test-model-641e39-kserve-5dbc96c4f5-wm6n6\" (UID: \"8b638f30-2ab0-489d-a89a-5891322457cd\") " pod="ai-tenant-e2e-shared-a-641e39/sub-test-model-641e39-kserve-5dbc96c4f5-wm6n6" Jul 29 20:04:59.844241 ip-10-0-131-93 kubenswrapper[2564]: I0729 20:04:59.844158 2564 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="ai-tenant-e2e-shared-a-641e39/sub-test-model-641e39-kserve-5dbc96c4f5-wm6n6" Jul 29 20:04:59.977151 ip-10-0-131-93 kubenswrapper[2564]: I0729 20:04:59.977124 2564 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["ai-tenant-e2e-shared-a-641e39/sub-test-model-641e39-kserve-5dbc96c4f5-wm6n6"] Jul 29 20:04:59.978886 ip-10-0-131-93 kubenswrapper[2564]: W0729 20:04:59.978862 2564 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-besteffort.slice/kubepods-besteffort-pod8b638f30_2ab0_489d_a89a_5891322457cd.slice/crio-dde9aaf91a23bc96e6c43444c916bd254c34233cc3fd205459e88d1511884e94 WatchSource:0}: Error finding container dde9aaf91a23bc96e6c43444c916bd254c34233cc3fd205459e88d1511884e94: Status 404 returned error can't find the container with id dde9aaf91a23bc96e6c43444c916bd254c34233cc3fd205459e88d1511884e94 Jul 29 20:05:00.244851 ip-10-0-131-93 kubenswrapper[2564]: I0729 20:05:00.244816 2564 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="ai-tenant-e2e-shared-a-641e39/sub-test-model-641e39-kserve-5dbc96c4f5-wm6n6" event={"ID":"8b638f30-2ab0-489d-a89a-5891322457cd","Type":"ContainerStarted","Data":"2d7100e9b0a66ba688fcfffe4d8ab0c9f7a57e6cfedd1e7324d017b3127dcf1b"} Jul 29 20:05:00.245027 ip-10-0-131-93 kubenswrapper[2564]: I0729 20:05:00.244871 2564 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="ai-tenant-e2e-shared-a-641e39/sub-test-model-641e39-kserve-5dbc96c4f5-wm6n6" event={"ID":"8b638f30-2ab0-489d-a89a-5891322457cd","Type":"ContainerStarted","Data":"dde9aaf91a23bc96e6c43444c916bd254c34233cc3fd205459e88d1511884e94"} Jul 29 20:05:00.263256 ip-10-0-131-93 kubenswrapper[2564]: I0729 20:05:00.263205 2564 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="ai-tenant-e2e-shared-a-641e39/sub-test-model-641e39-kserve-5dbc96c4f5-wm6n6" podStartSLOduration=1.263186376 podStartE2EDuration="1.263186376s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-29 20:04:59 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-29 20:05:00.26135902 +0000 UTC m=+2875.985418714" watchObservedRunningTime="2026-07-29 20:05:00.263186376 +0000 UTC m=+2875.987246069" Jul 29 20:05:09.845135 ip-10-0-131-93 kubenswrapper[2564]: I0729 20:05:09.845097 2564 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="ai-tenant-e2e-shared-a-641e39/sub-test-model-641e39-kserve-5dbc96c4f5-wm6n6" Jul 29 20:05:09.845135 ip-10-0-131-93 kubenswrapper[2564]: I0729 20:05:09.845147 2564 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="ai-tenant-e2e-shared-a-641e39/sub-test-model-641e39-kserve-5dbc96c4f5-wm6n6" Jul 29 20:05:09.857950 ip-10-0-131-93 kubenswrapper[2564]: I0729 20:05:09.857921 2564 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="ai-tenant-e2e-shared-a-641e39/sub-test-model-641e39-kserve-5dbc96c4f5-wm6n6" Jul 29 20:05:10.303404 ip-10-0-131-93 kubenswrapper[2564]: I0729 20:05:10.303360 2564 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="ai-tenant-e2e-shared-a-641e39/sub-test-model-641e39-kserve-5dbc96c4f5-wm6n6" Jul 29 20:05:19.767818 ip-10-0-131-93 kubenswrapper[2564]: I0729 20:05:19.767785 2564 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-444jx_faa8c546-c75b-4e10-93b4-661aa2b5f53e/konnectivity-agent/0.log" limit=1024 Jul 29 20:06:17.047443 ip-10-0-131-93 kubenswrapper[2564]: I0729 20:06:17.047408 2564 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["ai-tenant-e2e-shared-a-641e39/sub-test-model-641e39-kserve-5dbc96c4f5-wm6n6"] Jul 29 20:06:17.061122 ip-10-0-131-93 kubenswrapper[2564]: I0729 20:06:17.061051 2564 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-444jx_faa8c546-c75b-4e10-93b4-661aa2b5f53e/konnectivity-agent/0.log" limit=1024 Jul 29 20:06:17.867695 ip-10-0-131-93 kubenswrapper[2564]: I0729 20:06:17.867659 2564 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-444jx_faa8c546-c75b-4e10-93b4-661aa2b5f53e/konnectivity-agent/0.log" limit=1024 Jul 29 20:06:24.780936 ip-10-0-131-93 kubenswrapper[2564]: I0729 20:06:24.780872 2564 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-444jx_faa8c546-c75b-4e10-93b4-661aa2b5f53e/konnectivity-agent/0.log" limit=1024 Jul 29 20:06:32.065776 ip-10-0-131-93 kubenswrapper[2564]: I0729 20:06:32.065696 2564 kuberuntime_container.go:873] "Killing container with a grace period" pod="ai-tenant-e2e-shared-a-641e39/sub-test-model-641e39-kserve-5dbc96c4f5-wm6n6" podUID="8b638f30-2ab0-489d-a89a-5891322457cd" containerName="main" containerID="cri-o://2d7100e9b0a66ba688fcfffe4d8ab0c9f7a57e6cfedd1e7324d017b3127dcf1b" gracePeriod=45 Jul 29 20:06:32.301739 ip-10-0-131-93 kubenswrapper[2564]: I0729 20:06:32.301717 2564 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="ai-tenant-e2e-shared-a-641e39/sub-test-model-641e39-kserve-5dbc96c4f5-wm6n6" Jul 29 20:06:32.452434 ip-10-0-131-93 kubenswrapper[2564]: I0729 20:06:32.452393 2564 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/8b638f30-2ab0-489d-a89a-5891322457cd-dshm\") pod \"8b638f30-2ab0-489d-a89a-5891322457cd\" (UID: \"8b638f30-2ab0-489d-a89a-5891322457cd\") " Jul 29 20:06:32.452434 ip-10-0-131-93 kubenswrapper[2564]: I0729 20:06:32.452440 2564 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/8b638f30-2ab0-489d-a89a-5891322457cd-home\") pod \"8b638f30-2ab0-489d-a89a-5891322457cd\" (UID: \"8b638f30-2ab0-489d-a89a-5891322457cd\") " Jul 29 20:06:32.452692 ip-10-0-131-93 kubenswrapper[2564]: I0729 20:06:32.452460 2564 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/8b638f30-2ab0-489d-a89a-5891322457cd-tmp-dir\") pod \"8b638f30-2ab0-489d-a89a-5891322457cd\" (UID: \"8b638f30-2ab0-489d-a89a-5891322457cd\") " Jul 29 20:06:32.452692 ip-10-0-131-93 kubenswrapper[2564]: I0729 20:06:32.452499 2564 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/8b638f30-2ab0-489d-a89a-5891322457cd-tls-certs\") pod \"8b638f30-2ab0-489d-a89a-5891322457cd\" (UID: \"8b638f30-2ab0-489d-a89a-5891322457cd\") " Jul 29 20:06:32.452692 ip-10-0-131-93 kubenswrapper[2564]: I0729 20:06:32.452514 2564 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/8b638f30-2ab0-489d-a89a-5891322457cd-model-cache\") pod \"8b638f30-2ab0-489d-a89a-5891322457cd\" (UID: \"8b638f30-2ab0-489d-a89a-5891322457cd\") " Jul 29 20:06:32.452692 ip-10-0-131-93 kubenswrapper[2564]: I0729 20:06:32.452537 2564 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-4gvs9\" (UniqueName: \"kubernetes.io/projected/8b638f30-2ab0-489d-a89a-5891322457cd-kube-api-access-4gvs9\") pod \"8b638f30-2ab0-489d-a89a-5891322457cd\" (UID: \"8b638f30-2ab0-489d-a89a-5891322457cd\") " Jul 29 20:06:32.452894 ip-10-0-131-93 kubenswrapper[2564]: I0729 20:06:32.452737 2564 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/8b638f30-2ab0-489d-a89a-5891322457cd-home" (OuterVolumeSpecName: "home") pod "8b638f30-2ab0-489d-a89a-5891322457cd" (UID: "8b638f30-2ab0-489d-a89a-5891322457cd"). InnerVolumeSpecName "home". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Jul 29 20:06:32.452894 ip-10-0-131-93 kubenswrapper[2564]: I0729 20:06:32.452801 2564 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/8b638f30-2ab0-489d-a89a-5891322457cd-model-cache" (OuterVolumeSpecName: "model-cache") pod "8b638f30-2ab0-489d-a89a-5891322457cd" (UID: "8b638f30-2ab0-489d-a89a-5891322457cd"). InnerVolumeSpecName "model-cache". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Jul 29 20:06:32.452894 ip-10-0-131-93 kubenswrapper[2564]: I0729 20:06:32.452821 2564 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/8b638f30-2ab0-489d-a89a-5891322457cd-tmp-dir" (OuterVolumeSpecName: "tmp-dir") pod "8b638f30-2ab0-489d-a89a-5891322457cd" (UID: "8b638f30-2ab0-489d-a89a-5891322457cd"). InnerVolumeSpecName "tmp-dir". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Jul 29 20:06:32.453047 ip-10-0-131-93 kubenswrapper[2564]: I0729 20:06:32.452907 2564 reconciler_common.go:299] "Volume detached for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/8b638f30-2ab0-489d-a89a-5891322457cd-home\") on node \"ip-10-0-131-93.ec2.internal\" DevicePath \"\"" Jul 29 20:06:32.453047 ip-10-0-131-93 kubenswrapper[2564]: I0729 20:06:32.452944 2564 reconciler_common.go:299] "Volume detached for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/8b638f30-2ab0-489d-a89a-5891322457cd-tmp-dir\") on node \"ip-10-0-131-93.ec2.internal\" DevicePath \"\"" Jul 29 20:06:32.453047 ip-10-0-131-93 kubenswrapper[2564]: I0729 20:06:32.452955 2564 reconciler_common.go:299] "Volume detached for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/8b638f30-2ab0-489d-a89a-5891322457cd-model-cache\") on node \"ip-10-0-131-93.ec2.internal\" DevicePath \"\"" Jul 29 20:06:32.454594 ip-10-0-131-93 kubenswrapper[2564]: I0729 20:06:32.454572 2564 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/8b638f30-2ab0-489d-a89a-5891322457cd-dshm" (OuterVolumeSpecName: "dshm") pod "8b638f30-2ab0-489d-a89a-5891322457cd" (UID: "8b638f30-2ab0-489d-a89a-5891322457cd"). InnerVolumeSpecName "dshm". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Jul 29 20:06:32.454697 ip-10-0-131-93 kubenswrapper[2564]: I0729 20:06:32.454665 2564 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/8b638f30-2ab0-489d-a89a-5891322457cd-kube-api-access-4gvs9" (OuterVolumeSpecName: "kube-api-access-4gvs9") pod "8b638f30-2ab0-489d-a89a-5891322457cd" (UID: "8b638f30-2ab0-489d-a89a-5891322457cd"). InnerVolumeSpecName "kube-api-access-4gvs9". PluginName "kubernetes.io/projected", VolumeGIDValue "" Jul 29 20:06:32.454762 ip-10-0-131-93 kubenswrapper[2564]: I0729 20:06:32.454741 2564 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/8b638f30-2ab0-489d-a89a-5891322457cd-tls-certs" (OuterVolumeSpecName: "tls-certs") pod "8b638f30-2ab0-489d-a89a-5891322457cd" (UID: "8b638f30-2ab0-489d-a89a-5891322457cd"). InnerVolumeSpecName "tls-certs". PluginName "kubernetes.io/secret", VolumeGIDValue "" Jul 29 20:06:32.554073 ip-10-0-131-93 kubenswrapper[2564]: I0729 20:06:32.554038 2564 reconciler_common.go:299] "Volume detached for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/8b638f30-2ab0-489d-a89a-5891322457cd-dshm\") on node \"ip-10-0-131-93.ec2.internal\" DevicePath \"\"" Jul 29 20:06:32.554073 ip-10-0-131-93 kubenswrapper[2564]: I0729 20:06:32.554066 2564 reconciler_common.go:299] "Volume detached for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/8b638f30-2ab0-489d-a89a-5891322457cd-tls-certs\") on node \"ip-10-0-131-93.ec2.internal\" DevicePath \"\"" Jul 29 20:06:32.554073 ip-10-0-131-93 kubenswrapper[2564]: I0729 20:06:32.554077 2564 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-4gvs9\" (UniqueName: \"kubernetes.io/projected/8b638f30-2ab0-489d-a89a-5891322457cd-kube-api-access-4gvs9\") on node \"ip-10-0-131-93.ec2.internal\" DevicePath \"\"" Jul 29 20:06:32.697131 ip-10-0-131-93 kubenswrapper[2564]: I0729 20:06:32.697097 2564 generic.go:356] "Generic (PLEG): container finished" podID="8b638f30-2ab0-489d-a89a-5891322457cd" containerID="2d7100e9b0a66ba688fcfffe4d8ab0c9f7a57e6cfedd1e7324d017b3127dcf1b" exitCode=0 Jul 29 20:06:32.697301 ip-10-0-131-93 kubenswrapper[2564]: I0729 20:06:32.697143 2564 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="ai-tenant-e2e-shared-a-641e39/sub-test-model-641e39-kserve-5dbc96c4f5-wm6n6" Jul 29 20:06:32.697301 ip-10-0-131-93 kubenswrapper[2564]: I0729 20:06:32.697151 2564 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="ai-tenant-e2e-shared-a-641e39/sub-test-model-641e39-kserve-5dbc96c4f5-wm6n6" event={"ID":"8b638f30-2ab0-489d-a89a-5891322457cd","Type":"ContainerDied","Data":"2d7100e9b0a66ba688fcfffe4d8ab0c9f7a57e6cfedd1e7324d017b3127dcf1b"} Jul 29 20:06:32.697301 ip-10-0-131-93 kubenswrapper[2564]: I0729 20:06:32.697182 2564 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="ai-tenant-e2e-shared-a-641e39/sub-test-model-641e39-kserve-5dbc96c4f5-wm6n6" event={"ID":"8b638f30-2ab0-489d-a89a-5891322457cd","Type":"ContainerDied","Data":"dde9aaf91a23bc96e6c43444c916bd254c34233cc3fd205459e88d1511884e94"} Jul 29 20:06:32.697301 ip-10-0-131-93 kubenswrapper[2564]: I0729 20:06:32.697199 2564 scope.go:117] "RemoveContainer" containerID="2d7100e9b0a66ba688fcfffe4d8ab0c9f7a57e6cfedd1e7324d017b3127dcf1b" Jul 29 20:06:32.706560 ip-10-0-131-93 kubenswrapper[2564]: I0729 20:06:32.706542 2564 scope.go:117] "RemoveContainer" containerID="2d7100e9b0a66ba688fcfffe4d8ab0c9f7a57e6cfedd1e7324d017b3127dcf1b" Jul 29 20:06:32.706780 ip-10-0-131-93 kubenswrapper[2564]: E0729 20:06:32.706763 2564 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"2d7100e9b0a66ba688fcfffe4d8ab0c9f7a57e6cfedd1e7324d017b3127dcf1b\": container with ID starting with 2d7100e9b0a66ba688fcfffe4d8ab0c9f7a57e6cfedd1e7324d017b3127dcf1b not found: ID does not exist" containerID="2d7100e9b0a66ba688fcfffe4d8ab0c9f7a57e6cfedd1e7324d017b3127dcf1b" Jul 29 20:06:32.706826 ip-10-0-131-93 kubenswrapper[2564]: I0729 20:06:32.706786 2564 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"2d7100e9b0a66ba688fcfffe4d8ab0c9f7a57e6cfedd1e7324d017b3127dcf1b"} err="failed to get container status \"2d7100e9b0a66ba688fcfffe4d8ab0c9f7a57e6cfedd1e7324d017b3127dcf1b\": rpc error: code = NotFound desc = could not find container \"2d7100e9b0a66ba688fcfffe4d8ab0c9f7a57e6cfedd1e7324d017b3127dcf1b\": container with ID starting with 2d7100e9b0a66ba688fcfffe4d8ab0c9f7a57e6cfedd1e7324d017b3127dcf1b not found: ID does not exist" Jul 29 20:06:32.719582 ip-10-0-131-93 kubenswrapper[2564]: I0729 20:06:32.719562 2564 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["ai-tenant-e2e-shared-a-641e39/sub-test-model-641e39-kserve-5dbc96c4f5-wm6n6"] Jul 29 20:06:32.723819 ip-10-0-131-93 kubenswrapper[2564]: I0729 20:06:32.723799 2564 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["ai-tenant-e2e-shared-a-641e39/sub-test-model-641e39-kserve-5dbc96c4f5-wm6n6"] Jul 29 20:06:32.783932 ip-10-0-131-93 kubenswrapper[2564]: I0729 20:06:32.783894 2564 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="8b638f30-2ab0-489d-a89a-5891322457cd" path="/var/lib/kubelet/pods/8b638f30-2ab0-489d-a89a-5891322457cd/volumes" Jul 29 20:06:50.694034 ip-10-0-131-93 kubenswrapper[2564]: I0729 20:06:50.693995 2564 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-444jx_faa8c546-c75b-4e10-93b4-661aa2b5f53e/konnectivity-agent/0.log" limit=1024 Jul 29 20:07:55.056470 ip-10-0-131-93 kubenswrapper[2564]: I0729 20:07:55.056437 2564 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-444jx_faa8c546-c75b-4e10-93b4-661aa2b5f53e/konnectivity-agent/0.log" limit=1024 Jul 29 20:08:04.887368 ip-10-0-131-93 kubenswrapper[2564]: I0729 20:08:04.887333 2564 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-444jx_faa8c546-c75b-4e10-93b4-661aa2b5f53e/konnectivity-agent/0.log" limit=1024 Jul 29 20:08:05.928680 ip-10-0-131-93 kubenswrapper[2564]: I0729 20:08:05.928646 2564 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-444jx_faa8c546-c75b-4e10-93b4-661aa2b5f53e/konnectivity-agent/0.log" limit=1024 Jul 29 20:08:25.608341 ip-10-0-131-93 kubenswrapper[2564]: I0729 20:08:25.608306 2564 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-444jx_faa8c546-c75b-4e10-93b4-661aa2b5f53e/konnectivity-agent/0.log" limit=1024 Jul 29 20:08:25.895541 ip-10-0-131-93 kubenswrapper[2564]: I0729 20:08:25.893950 2564 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-ingress/e2e-ipp-a-f0b78f-openshift-default-6d5b4668cb-lmfd2"] Jul 29 20:08:25.895541 ip-10-0-131-93 kubenswrapper[2564]: I0729 20:08:25.894712 2564 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="8b638f30-2ab0-489d-a89a-5891322457cd" containerName="main" Jul 29 20:08:25.895541 ip-10-0-131-93 kubenswrapper[2564]: I0729 20:08:25.894741 2564 state_mem.go:107] "Deleted CPUSet assignment" podUID="8b638f30-2ab0-489d-a89a-5891322457cd" containerName="main" Jul 29 20:08:25.895541 ip-10-0-131-93 kubenswrapper[2564]: I0729 20:08:25.894923 2564 memory_manager.go:368] "RemoveStaleState removing state" podUID="8b638f30-2ab0-489d-a89a-5891322457cd" containerName="main" Jul 29 20:08:25.898811 ip-10-0-131-93 kubenswrapper[2564]: I0729 20:08:25.898785 2564 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-ipp-a-f0b78f-openshift-default-6d5b4668cb-lmfd2" Jul 29 20:08:25.903006 ip-10-0-131-93 kubenswrapper[2564]: I0729 20:08:25.902981 2564 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress\"/\"e2e-ipp-a-f0b78f-openshift-default-dockercfg-sv4pq\"" Jul 29 20:08:25.928269 ip-10-0-131-93 kubenswrapper[2564]: I0729 20:08:25.928240 2564 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/e2e-ipp-a-f0b78f-openshift-default-6d5b4668cb-lmfd2"] Jul 29 20:08:25.938420 ip-10-0-131-93 kubenswrapper[2564]: I0729 20:08:25.938391 2564 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-444jx_faa8c546-c75b-4e10-93b4-661aa2b5f53e/konnectivity-agent/0.log" limit=1024 Jul 29 20:08:25.981550 ip-10-0-131-93 kubenswrapper[2564]: I0729 20:08:25.981513 2564 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/94a75046-fecd-413c-ad52-daf45c5732cd-istio-podinfo\") pod \"e2e-ipp-a-f0b78f-openshift-default-6d5b4668cb-lmfd2\" (UID: \"94a75046-fecd-413c-ad52-daf45c5732cd\") " pod="openshift-ingress/e2e-ipp-a-f0b78f-openshift-default-6d5b4668cb-lmfd2" Jul 29 20:08:25.981894 ip-10-0-131-93 kubenswrapper[2564]: I0729 20:08:25.981860 2564 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/94a75046-fecd-413c-ad52-daf45c5732cd-istio-envoy\") pod \"e2e-ipp-a-f0b78f-openshift-default-6d5b4668cb-lmfd2\" (UID: \"94a75046-fecd-413c-ad52-daf45c5732cd\") " pod="openshift-ingress/e2e-ipp-a-f0b78f-openshift-default-6d5b4668cb-lmfd2" Jul 29 20:08:25.982010 ip-10-0-131-93 kubenswrapper[2564]: I0729 20:08:25.981898 2564 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/94a75046-fecd-413c-ad52-daf45c5732cd-istio-token\") pod \"e2e-ipp-a-f0b78f-openshift-default-6d5b4668cb-lmfd2\" (UID: \"94a75046-fecd-413c-ad52-daf45c5732cd\") " pod="openshift-ingress/e2e-ipp-a-f0b78f-openshift-default-6d5b4668cb-lmfd2" Jul 29 20:08:25.982010 ip-10-0-131-93 kubenswrapper[2564]: I0729 20:08:25.981967 2564 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/94a75046-fecd-413c-ad52-daf45c5732cd-workload-certs\") pod \"e2e-ipp-a-f0b78f-openshift-default-6d5b4668cb-lmfd2\" (UID: \"94a75046-fecd-413c-ad52-daf45c5732cd\") " pod="openshift-ingress/e2e-ipp-a-f0b78f-openshift-default-6d5b4668cb-lmfd2" Jul 29 20:08:25.982118 ip-10-0-131-93 kubenswrapper[2564]: I0729 20:08:25.982010 2564 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/94a75046-fecd-413c-ad52-daf45c5732cd-istiod-ca-cert\") pod \"e2e-ipp-a-f0b78f-openshift-default-6d5b4668cb-lmfd2\" (UID: \"94a75046-fecd-413c-ad52-daf45c5732cd\") " pod="openshift-ingress/e2e-ipp-a-f0b78f-openshift-default-6d5b4668cb-lmfd2" Jul 29 20:08:25.982118 ip-10-0-131-93 kubenswrapper[2564]: I0729 20:08:25.982040 2564 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/94a75046-fecd-413c-ad52-daf45c5732cd-workload-socket\") pod \"e2e-ipp-a-f0b78f-openshift-default-6d5b4668cb-lmfd2\" (UID: \"94a75046-fecd-413c-ad52-daf45c5732cd\") " pod="openshift-ingress/e2e-ipp-a-f0b78f-openshift-default-6d5b4668cb-lmfd2" Jul 29 20:08:25.982118 ip-10-0-131-93 kubenswrapper[2564]: I0729 20:08:25.982065 2564 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/94a75046-fecd-413c-ad52-daf45c5732cd-istio-data\") pod \"e2e-ipp-a-f0b78f-openshift-default-6d5b4668cb-lmfd2\" (UID: \"94a75046-fecd-413c-ad52-daf45c5732cd\") " pod="openshift-ingress/e2e-ipp-a-f0b78f-openshift-default-6d5b4668cb-lmfd2" Jul 29 20:08:25.982278 ip-10-0-131-93 kubenswrapper[2564]: I0729 20:08:25.982168 2564 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-b9sfg\" (UniqueName: \"kubernetes.io/projected/94a75046-fecd-413c-ad52-daf45c5732cd-kube-api-access-b9sfg\") pod \"e2e-ipp-a-f0b78f-openshift-default-6d5b4668cb-lmfd2\" (UID: \"94a75046-fecd-413c-ad52-daf45c5732cd\") " pod="openshift-ingress/e2e-ipp-a-f0b78f-openshift-default-6d5b4668cb-lmfd2" Jul 29 20:08:25.982278 ip-10-0-131-93 kubenswrapper[2564]: I0729 20:08:25.982223 2564 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/94a75046-fecd-413c-ad52-daf45c5732cd-credential-socket\") pod \"e2e-ipp-a-f0b78f-openshift-default-6d5b4668cb-lmfd2\" (UID: \"94a75046-fecd-413c-ad52-daf45c5732cd\") " pod="openshift-ingress/e2e-ipp-a-f0b78f-openshift-default-6d5b4668cb-lmfd2" Jul 29 20:08:26.083731 ip-10-0-131-93 kubenswrapper[2564]: I0729 20:08:26.083693 2564 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/94a75046-fecd-413c-ad52-daf45c5732cd-istio-envoy\") pod \"e2e-ipp-a-f0b78f-openshift-default-6d5b4668cb-lmfd2\" (UID: \"94a75046-fecd-413c-ad52-daf45c5732cd\") " pod="openshift-ingress/e2e-ipp-a-f0b78f-openshift-default-6d5b4668cb-lmfd2" Jul 29 20:08:26.083731 ip-10-0-131-93 kubenswrapper[2564]: I0729 20:08:26.083729 2564 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/94a75046-fecd-413c-ad52-daf45c5732cd-istio-token\") pod \"e2e-ipp-a-f0b78f-openshift-default-6d5b4668cb-lmfd2\" (UID: \"94a75046-fecd-413c-ad52-daf45c5732cd\") " pod="openshift-ingress/e2e-ipp-a-f0b78f-openshift-default-6d5b4668cb-lmfd2" Jul 29 20:08:26.083990 ip-10-0-131-93 kubenswrapper[2564]: I0729 20:08:26.083748 2564 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/94a75046-fecd-413c-ad52-daf45c5732cd-workload-certs\") pod \"e2e-ipp-a-f0b78f-openshift-default-6d5b4668cb-lmfd2\" (UID: \"94a75046-fecd-413c-ad52-daf45c5732cd\") " pod="openshift-ingress/e2e-ipp-a-f0b78f-openshift-default-6d5b4668cb-lmfd2" Jul 29 20:08:26.083990 ip-10-0-131-93 kubenswrapper[2564]: I0729 20:08:26.083763 2564 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/94a75046-fecd-413c-ad52-daf45c5732cd-istiod-ca-cert\") pod \"e2e-ipp-a-f0b78f-openshift-default-6d5b4668cb-lmfd2\" (UID: \"94a75046-fecd-413c-ad52-daf45c5732cd\") " pod="openshift-ingress/e2e-ipp-a-f0b78f-openshift-default-6d5b4668cb-lmfd2" Jul 29 20:08:26.083990 ip-10-0-131-93 kubenswrapper[2564]: I0729 20:08:26.083779 2564 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/94a75046-fecd-413c-ad52-daf45c5732cd-workload-socket\") pod \"e2e-ipp-a-f0b78f-openshift-default-6d5b4668cb-lmfd2\" (UID: \"94a75046-fecd-413c-ad52-daf45c5732cd\") " pod="openshift-ingress/e2e-ipp-a-f0b78f-openshift-default-6d5b4668cb-lmfd2" Jul 29 20:08:26.083990 ip-10-0-131-93 kubenswrapper[2564]: I0729 20:08:26.083802 2564 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/94a75046-fecd-413c-ad52-daf45c5732cd-istio-data\") pod \"e2e-ipp-a-f0b78f-openshift-default-6d5b4668cb-lmfd2\" (UID: \"94a75046-fecd-413c-ad52-daf45c5732cd\") " pod="openshift-ingress/e2e-ipp-a-f0b78f-openshift-default-6d5b4668cb-lmfd2" Jul 29 20:08:26.083990 ip-10-0-131-93 kubenswrapper[2564]: I0729 20:08:26.083831 2564 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-b9sfg\" (UniqueName: \"kubernetes.io/projected/94a75046-fecd-413c-ad52-daf45c5732cd-kube-api-access-b9sfg\") pod \"e2e-ipp-a-f0b78f-openshift-default-6d5b4668cb-lmfd2\" (UID: \"94a75046-fecd-413c-ad52-daf45c5732cd\") " pod="openshift-ingress/e2e-ipp-a-f0b78f-openshift-default-6d5b4668cb-lmfd2" Jul 29 20:08:26.083990 ip-10-0-131-93 kubenswrapper[2564]: I0729 20:08:26.083867 2564 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/94a75046-fecd-413c-ad52-daf45c5732cd-credential-socket\") pod \"e2e-ipp-a-f0b78f-openshift-default-6d5b4668cb-lmfd2\" (UID: \"94a75046-fecd-413c-ad52-daf45c5732cd\") " pod="openshift-ingress/e2e-ipp-a-f0b78f-openshift-default-6d5b4668cb-lmfd2" Jul 29 20:08:26.083990 ip-10-0-131-93 kubenswrapper[2564]: I0729 20:08:26.083903 2564 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/94a75046-fecd-413c-ad52-daf45c5732cd-istio-podinfo\") pod \"e2e-ipp-a-f0b78f-openshift-default-6d5b4668cb-lmfd2\" (UID: \"94a75046-fecd-413c-ad52-daf45c5732cd\") " pod="openshift-ingress/e2e-ipp-a-f0b78f-openshift-default-6d5b4668cb-lmfd2" Jul 29 20:08:26.084356 ip-10-0-131-93 kubenswrapper[2564]: I0729 20:08:26.084190 2564 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/94a75046-fecd-413c-ad52-daf45c5732cd-workload-certs\") pod \"e2e-ipp-a-f0b78f-openshift-default-6d5b4668cb-lmfd2\" (UID: \"94a75046-fecd-413c-ad52-daf45c5732cd\") " pod="openshift-ingress/e2e-ipp-a-f0b78f-openshift-default-6d5b4668cb-lmfd2" Jul 29 20:08:26.084356 ip-10-0-131-93 kubenswrapper[2564]: I0729 20:08:26.084247 2564 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/94a75046-fecd-413c-ad52-daf45c5732cd-istio-data\") pod \"e2e-ipp-a-f0b78f-openshift-default-6d5b4668cb-lmfd2\" (UID: \"94a75046-fecd-413c-ad52-daf45c5732cd\") " pod="openshift-ingress/e2e-ipp-a-f0b78f-openshift-default-6d5b4668cb-lmfd2" Jul 29 20:08:26.084356 ip-10-0-131-93 kubenswrapper[2564]: I0729 20:08:26.084291 2564 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/94a75046-fecd-413c-ad52-daf45c5732cd-workload-socket\") pod \"e2e-ipp-a-f0b78f-openshift-default-6d5b4668cb-lmfd2\" (UID: \"94a75046-fecd-413c-ad52-daf45c5732cd\") " pod="openshift-ingress/e2e-ipp-a-f0b78f-openshift-default-6d5b4668cb-lmfd2" Jul 29 20:08:26.084528 ip-10-0-131-93 kubenswrapper[2564]: I0729 20:08:26.084508 2564 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/94a75046-fecd-413c-ad52-daf45c5732cd-credential-socket\") pod \"e2e-ipp-a-f0b78f-openshift-default-6d5b4668cb-lmfd2\" (UID: \"94a75046-fecd-413c-ad52-daf45c5732cd\") " pod="openshift-ingress/e2e-ipp-a-f0b78f-openshift-default-6d5b4668cb-lmfd2" Jul 29 20:08:26.084600 ip-10-0-131-93 kubenswrapper[2564]: I0729 20:08:26.084584 2564 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/94a75046-fecd-413c-ad52-daf45c5732cd-istiod-ca-cert\") pod \"e2e-ipp-a-f0b78f-openshift-default-6d5b4668cb-lmfd2\" (UID: \"94a75046-fecd-413c-ad52-daf45c5732cd\") " pod="openshift-ingress/e2e-ipp-a-f0b78f-openshift-default-6d5b4668cb-lmfd2" Jul 29 20:08:26.086110 ip-10-0-131-93 kubenswrapper[2564]: I0729 20:08:26.086084 2564 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/94a75046-fecd-413c-ad52-daf45c5732cd-istio-envoy\") pod \"e2e-ipp-a-f0b78f-openshift-default-6d5b4668cb-lmfd2\" (UID: \"94a75046-fecd-413c-ad52-daf45c5732cd\") " pod="openshift-ingress/e2e-ipp-a-f0b78f-openshift-default-6d5b4668cb-lmfd2" Jul 29 20:08:26.086331 ip-10-0-131-93 kubenswrapper[2564]: I0729 20:08:26.086313 2564 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/94a75046-fecd-413c-ad52-daf45c5732cd-istio-podinfo\") pod \"e2e-ipp-a-f0b78f-openshift-default-6d5b4668cb-lmfd2\" (UID: \"94a75046-fecd-413c-ad52-daf45c5732cd\") " pod="openshift-ingress/e2e-ipp-a-f0b78f-openshift-default-6d5b4668cb-lmfd2" Jul 29 20:08:26.091359 ip-10-0-131-93 kubenswrapper[2564]: I0729 20:08:26.091327 2564 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/94a75046-fecd-413c-ad52-daf45c5732cd-istio-token\") pod \"e2e-ipp-a-f0b78f-openshift-default-6d5b4668cb-lmfd2\" (UID: \"94a75046-fecd-413c-ad52-daf45c5732cd\") " pod="openshift-ingress/e2e-ipp-a-f0b78f-openshift-default-6d5b4668cb-lmfd2" Jul 29 20:08:26.091481 ip-10-0-131-93 kubenswrapper[2564]: I0729 20:08:26.091403 2564 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-b9sfg\" (UniqueName: \"kubernetes.io/projected/94a75046-fecd-413c-ad52-daf45c5732cd-kube-api-access-b9sfg\") pod \"e2e-ipp-a-f0b78f-openshift-default-6d5b4668cb-lmfd2\" (UID: \"94a75046-fecd-413c-ad52-daf45c5732cd\") " pod="openshift-ingress/e2e-ipp-a-f0b78f-openshift-default-6d5b4668cb-lmfd2" Jul 29 20:08:26.208725 ip-10-0-131-93 kubenswrapper[2564]: I0729 20:08:26.208693 2564 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-ipp-a-f0b78f-openshift-default-6d5b4668cb-lmfd2" Jul 29 20:08:26.288501 ip-10-0-131-93 kubenswrapper[2564]: I0729 20:08:26.288457 2564 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-444jx_faa8c546-c75b-4e10-93b4-661aa2b5f53e/konnectivity-agent/0.log" limit=1024 Jul 29 20:08:26.357750 ip-10-0-131-93 kubenswrapper[2564]: I0729 20:08:26.357714 2564 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/e2e-ipp-a-f0b78f-openshift-default-6d5b4668cb-lmfd2"] Jul 29 20:08:26.358870 ip-10-0-131-93 kubenswrapper[2564]: W0729 20:08:26.358837 2564 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod94a75046_fecd_413c_ad52_daf45c5732cd.slice/crio-f875df0c6c0e6e7f0a1b12fae7a99e5fa710c280d6befae522bb5b0d1da375c2 WatchSource:0}: Error finding container f875df0c6c0e6e7f0a1b12fae7a99e5fa710c280d6befae522bb5b0d1da375c2: Status 404 returned error can't find the container with id f875df0c6c0e6e7f0a1b12fae7a99e5fa710c280d6befae522bb5b0d1da375c2 Jul 29 20:08:26.361115 ip-10-0-131-93 kubenswrapper[2564]: I0729 20:08:26.361082 2564 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236224Ki","pods":"250"} Jul 29 20:08:26.361196 ip-10-0-131-93 kubenswrapper[2564]: I0729 20:08:26.361147 2564 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236224Ki","pods":"250"} Jul 29 20:08:26.361196 ip-10-0-131-93 kubenswrapper[2564]: I0729 20:08:26.361174 2564 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236224Ki","pods":"250"} Jul 29 20:08:27.255612 ip-10-0-131-93 kubenswrapper[2564]: I0729 20:08:27.255574 2564 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-ipp-a-f0b78f-openshift-default-6d5b4668cb-lmfd2" event={"ID":"94a75046-fecd-413c-ad52-daf45c5732cd","Type":"ContainerStarted","Data":"b76bf1acb7a868229a2c8e7da52658015fc10f09c4fb9ff14e66fefaa78c4efd"} Jul 29 20:08:27.255989 ip-10-0-131-93 kubenswrapper[2564]: I0729 20:08:27.255631 2564 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-ipp-a-f0b78f-openshift-default-6d5b4668cb-lmfd2" event={"ID":"94a75046-fecd-413c-ad52-daf45c5732cd","Type":"ContainerStarted","Data":"f875df0c6c0e6e7f0a1b12fae7a99e5fa710c280d6befae522bb5b0d1da375c2"} Jul 29 20:08:27.276401 ip-10-0-131-93 kubenswrapper[2564]: I0729 20:08:27.276326 2564 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-ingress/e2e-ipp-a-f0b78f-openshift-default-6d5b4668cb-lmfd2" podStartSLOduration=2.276310131 podStartE2EDuration="2.276310131s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-29 20:08:25 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-29 20:08:27.273480114 +0000 UTC m=+3082.997539805" watchObservedRunningTime="2026-07-29 20:08:27.276310131 +0000 UTC m=+3083.000369822" Jul 29 20:08:28.209907 ip-10-0-131-93 kubenswrapper[2564]: I0729 20:08:28.209873 2564 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="openshift-ingress/e2e-ipp-a-f0b78f-openshift-default-6d5b4668cb-lmfd2" Jul 29 20:08:28.214692 ip-10-0-131-93 kubenswrapper[2564]: I0729 20:08:28.214667 2564 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="openshift-ingress/e2e-ipp-a-f0b78f-openshift-default-6d5b4668cb-lmfd2" Jul 29 20:08:28.260987 ip-10-0-131-93 kubenswrapper[2564]: I0729 20:08:28.260952 2564 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/e2e-ipp-a-f0b78f-openshift-default-6d5b4668cb-lmfd2" Jul 29 20:08:28.261809 ip-10-0-131-93 kubenswrapper[2564]: I0729 20:08:28.261792 2564 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/e2e-ipp-a-f0b78f-openshift-default-6d5b4668cb-lmfd2" Jul 29 20:08:29.186015 ip-10-0-131-93 kubenswrapper[2564]: I0729 20:08:29.185983 2564 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-444jx_faa8c546-c75b-4e10-93b4-661aa2b5f53e/konnectivity-agent/0.log" limit=1024 Jul 29 20:08:29.280423 ip-10-0-131-93 kubenswrapper[2564]: I0729 20:08:29.280392 2564 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-444jx_faa8c546-c75b-4e10-93b4-661aa2b5f53e/konnectivity-agent/0.log" limit=1024 Jul 29 20:08:30.417053 ip-10-0-131-93 kubenswrapper[2564]: I0729 20:08:30.417008 2564 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-444jx_faa8c546-c75b-4e10-93b4-661aa2b5f53e/konnectivity-agent/0.log" limit=1024 Jul 29 20:08:30.572227 ip-10-0-131-93 kubenswrapper[2564]: I0729 20:08:30.572193 2564 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-444jx_faa8c546-c75b-4e10-93b4-661aa2b5f53e/konnectivity-agent/0.log" limit=1024 Jul 29 20:08:31.084350 ip-10-0-131-93 kubenswrapper[2564]: I0729 20:08:31.084309 2564 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-444jx_faa8c546-c75b-4e10-93b4-661aa2b5f53e/konnectivity-agent/0.log" limit=1024 Jul 29 20:08:31.175745 ip-10-0-131-93 kubenswrapper[2564]: I0729 20:08:31.175714 2564 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-444jx_faa8c546-c75b-4e10-93b4-661aa2b5f53e/konnectivity-agent/0.log" limit=1024 Jul 29 20:08:44.302545 ip-10-0-131-93 kubenswrapper[2564]: I0729 20:08:44.302502 2564 patch_prober.go:28] interesting pod/e2e-ipp-a-f0b78f-openshift-default-6d5b4668cb-lmfd2 container/istio-proxy namespace/openshift-ingress: Readiness probe status=failure output="Get \"http://10.132.0.50:15021/healthz/ready\": context deadline exceeded (Client.Timeout exceeded while awaiting headers)" start-of-body= Jul 29 20:08:44.302990 ip-10-0-131-93 kubenswrapper[2564]: I0729 20:08:44.302604 2564 prober.go:120] "Probe failed" probeType="Readiness" pod="openshift-ingress/e2e-ipp-a-f0b78f-openshift-default-6d5b4668cb-lmfd2" podUID="94a75046-fecd-413c-ad52-daf45c5732cd" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.132.0.50:15021/healthz/ready\": context deadline exceeded (Client.Timeout exceeded while awaiting headers)" Jul 29 20:08:46.355123 ip-10-0-131-93 kubenswrapper[2564]: I0729 20:08:46.355088 2564 generic.go:356] "Generic (PLEG): container finished" podID="94a75046-fecd-413c-ad52-daf45c5732cd" containerID="b76bf1acb7a868229a2c8e7da52658015fc10f09c4fb9ff14e66fefaa78c4efd" exitCode=0 Jul 29 20:08:46.355573 ip-10-0-131-93 kubenswrapper[2564]: I0729 20:08:46.355173 2564 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-ipp-a-f0b78f-openshift-default-6d5b4668cb-lmfd2" event={"ID":"94a75046-fecd-413c-ad52-daf45c5732cd","Type":"ContainerDied","Data":"b76bf1acb7a868229a2c8e7da52658015fc10f09c4fb9ff14e66fefaa78c4efd"} Jul 29 20:08:46.355616 ip-10-0-131-93 kubenswrapper[2564]: I0729 20:08:46.355571 2564 scope.go:117] "RemoveContainer" containerID="b76bf1acb7a868229a2c8e7da52658015fc10f09c4fb9ff14e66fefaa78c4efd" Jul 29 20:08:46.356732 ip-10-0-131-93 kubenswrapper[2564]: I0729 20:08:46.356703 2564 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236224Ki","pods":"250"} Jul 29 20:08:46.356819 ip-10-0-131-93 kubenswrapper[2564]: I0729 20:08:46.356780 2564 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236224Ki","pods":"250"} Jul 29 20:08:46.356819 ip-10-0-131-93 kubenswrapper[2564]: I0729 20:08:46.356814 2564 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236224Ki","pods":"250"} Jul 29 20:08:47.362640 ip-10-0-131-93 kubenswrapper[2564]: I0729 20:08:47.362606 2564 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-ipp-a-f0b78f-openshift-default-6d5b4668cb-lmfd2" event={"ID":"94a75046-fecd-413c-ad52-daf45c5732cd","Type":"ContainerStarted","Data":"fbb6df48fa33ee33dbbcbeabe66f88dd8deddc549cb48be59f5b6178789705bf"} Jul 29 20:08:48.209263 ip-10-0-131-93 kubenswrapper[2564]: I0729 20:08:48.209228 2564 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="openshift-ingress/e2e-ipp-a-f0b78f-openshift-default-6d5b4668cb-lmfd2" Jul 29 20:08:48.209530 ip-10-0-131-93 kubenswrapper[2564]: I0729 20:08:48.209508 2564 patch_prober.go:28] interesting pod/e2e-ipp-a-f0b78f-openshift-default-6d5b4668cb-lmfd2 container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.132.0.50:15021/healthz/ready\": dial tcp 10.132.0.50:15021: connect: connection refused" start-of-body= Jul 29 20:08:48.209584 ip-10-0-131-93 kubenswrapper[2564]: I0729 20:08:48.209556 2564 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-ipp-a-f0b78f-openshift-default-6d5b4668cb-lmfd2" podUID="94a75046-fecd-413c-ad52-daf45c5732cd" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.132.0.50:15021/healthz/ready\": dial tcp 10.132.0.50:15021: connect: connection refused" Jul 29 20:08:49.209813 ip-10-0-131-93 kubenswrapper[2564]: I0729 20:08:49.209777 2564 patch_prober.go:28] interesting pod/e2e-ipp-a-f0b78f-openshift-default-6d5b4668cb-lmfd2 container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.132.0.50:15021/healthz/ready\": dial tcp 10.132.0.50:15021: connect: connection refused" start-of-body= Jul 29 20:08:49.210204 ip-10-0-131-93 kubenswrapper[2564]: I0729 20:08:49.209851 2564 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-ipp-a-f0b78f-openshift-default-6d5b4668cb-lmfd2" podUID="94a75046-fecd-413c-ad52-daf45c5732cd" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.132.0.50:15021/healthz/ready\": dial tcp 10.132.0.50:15021: connect: connection refused" Jul 29 20:08:50.209600 ip-10-0-131-93 kubenswrapper[2564]: I0729 20:08:50.209561 2564 patch_prober.go:28] interesting pod/e2e-ipp-a-f0b78f-openshift-default-6d5b4668cb-lmfd2 container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.132.0.50:15021/healthz/ready\": dial tcp 10.132.0.50:15021: connect: connection refused" start-of-body= Jul 29 20:08:50.209787 ip-10-0-131-93 kubenswrapper[2564]: I0729 20:08:50.209645 2564 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-ipp-a-f0b78f-openshift-default-6d5b4668cb-lmfd2" podUID="94a75046-fecd-413c-ad52-daf45c5732cd" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.132.0.50:15021/healthz/ready\": dial tcp 10.132.0.50:15021: connect: connection refused" Jul 29 20:08:51.212589 ip-10-0-131-93 kubenswrapper[2564]: I0729 20:08:51.212556 2564 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="openshift-ingress/e2e-ipp-a-f0b78f-openshift-default-6d5b4668cb-lmfd2" Jul 29 20:08:51.212971 ip-10-0-131-93 kubenswrapper[2564]: I0729 20:08:51.212817 2564 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/e2e-ipp-a-f0b78f-openshift-default-6d5b4668cb-lmfd2" Jul 29 20:08:51.213536 ip-10-0-131-93 kubenswrapper[2564]: I0729 20:08:51.213518 2564 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/e2e-ipp-a-f0b78f-openshift-default-6d5b4668cb-lmfd2" Jul 29 20:08:54.139727 ip-10-0-131-93 kubenswrapper[2564]: I0729 20:08:54.139694 2564 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-444jx_faa8c546-c75b-4e10-93b4-661aa2b5f53e/konnectivity-agent/0.log" limit=1024 Jul 29 20:08:54.812399 ip-10-0-131-93 kubenswrapper[2564]: I0729 20:08:54.812350 2564 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-444jx_faa8c546-c75b-4e10-93b4-661aa2b5f53e/konnectivity-agent/0.log" limit=1024 Jul 29 20:08:55.168195 ip-10-0-131-93 kubenswrapper[2564]: I0729 20:08:55.168106 2564 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-444jx_faa8c546-c75b-4e10-93b4-661aa2b5f53e/konnectivity-agent/0.log" limit=1024 Jul 29 20:08:55.385773 ip-10-0-131-93 kubenswrapper[2564]: I0729 20:08:55.385742 2564 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-444jx_faa8c546-c75b-4e10-93b4-661aa2b5f53e/konnectivity-agent/0.log" limit=1024 Jul 29 20:08:58.032820 ip-10-0-131-93 kubenswrapper[2564]: I0729 20:08:58.032791 2564 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-444jx_faa8c546-c75b-4e10-93b4-661aa2b5f53e/konnectivity-agent/0.log" limit=1024 Jul 29 20:08:58.137234 ip-10-0-131-93 kubenswrapper[2564]: I0729 20:08:58.137196 2564 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-444jx_faa8c546-c75b-4e10-93b4-661aa2b5f53e/konnectivity-agent/0.log" limit=1024 Jul 29 20:08:59.162557 ip-10-0-131-93 kubenswrapper[2564]: I0729 20:08:59.162519 2564 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-444jx_faa8c546-c75b-4e10-93b4-661aa2b5f53e/konnectivity-agent/0.log" limit=1024 Jul 29 20:08:59.442037 ip-10-0-131-93 kubenswrapper[2564]: I0729 20:08:59.442012 2564 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-444jx_faa8c546-c75b-4e10-93b4-661aa2b5f53e/konnectivity-agent/0.log" limit=1024 Jul 29 20:08:59.679143 ip-10-0-131-93 kubenswrapper[2564]: I0729 20:08:59.679114 2564 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-444jx_faa8c546-c75b-4e10-93b4-661aa2b5f53e/konnectivity-agent/0.log" limit=1024 Jul 29 20:08:59.775569 ip-10-0-131-93 kubenswrapper[2564]: I0729 20:08:59.775494 2564 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-444jx_faa8c546-c75b-4e10-93b4-661aa2b5f53e/konnectivity-agent/0.log" limit=1024 Jul 29 20:08:59.782459 ip-10-0-131-93 kubenswrapper[2564]: I0729 20:08:59.782433 2564 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-ingress/payload-pre-processing-e2e-ipp-b-2d9c39-99cc99448-78rp6"] Jul 29 20:08:59.784806 ip-10-0-131-93 kubenswrapper[2564]: I0729 20:08:59.784792 2564 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-pre-processing-e2e-ipp-b-2d9c39-99cc99448-78rp6" Jul 29 20:08:59.787345 ip-10-0-131-93 kubenswrapper[2564]: I0729 20:08:59.787328 2564 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress\"/\"payload-processing-e2e-ipp-b-2d9c39-dockercfg-qsnh5\"" Jul 29 20:08:59.787446 ip-10-0-131-93 kubenswrapper[2564]: I0729 20:08:59.787346 2564 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ingress\"/\"payload-processing-plugins-e2e-ipp-b-2d9c39\"" Jul 29 20:08:59.796593 ip-10-0-131-93 kubenswrapper[2564]: I0729 20:08:59.796573 2564 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/payload-pre-processing-e2e-ipp-b-2d9c39-99cc99448-78rp6"] Jul 29 20:08:59.899113 ip-10-0-131-93 kubenswrapper[2564]: I0729 20:08:59.899075 2564 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/252a57d2-9e17-4d68-83bb-fff34a75bcd9-plugins-config-volume\") pod \"payload-pre-processing-e2e-ipp-b-2d9c39-99cc99448-78rp6\" (UID: \"252a57d2-9e17-4d68-83bb-fff34a75bcd9\") " pod="openshift-ingress/payload-pre-processing-e2e-ipp-b-2d9c39-99cc99448-78rp6" Jul 29 20:08:59.899459 ip-10-0-131-93 kubenswrapper[2564]: I0729 20:08:59.899428 2564 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-drrcn\" (UniqueName: \"kubernetes.io/projected/252a57d2-9e17-4d68-83bb-fff34a75bcd9-kube-api-access-drrcn\") pod \"payload-pre-processing-e2e-ipp-b-2d9c39-99cc99448-78rp6\" (UID: \"252a57d2-9e17-4d68-83bb-fff34a75bcd9\") " pod="openshift-ingress/payload-pre-processing-e2e-ipp-b-2d9c39-99cc99448-78rp6" Jul 29 20:09:00.000114 ip-10-0-131-93 kubenswrapper[2564]: I0729 20:09:00.000080 2564 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/252a57d2-9e17-4d68-83bb-fff34a75bcd9-plugins-config-volume\") pod \"payload-pre-processing-e2e-ipp-b-2d9c39-99cc99448-78rp6\" (UID: \"252a57d2-9e17-4d68-83bb-fff34a75bcd9\") " pod="openshift-ingress/payload-pre-processing-e2e-ipp-b-2d9c39-99cc99448-78rp6" Jul 29 20:09:00.000312 ip-10-0-131-93 kubenswrapper[2564]: I0729 20:09:00.000121 2564 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-drrcn\" (UniqueName: \"kubernetes.io/projected/252a57d2-9e17-4d68-83bb-fff34a75bcd9-kube-api-access-drrcn\") pod \"payload-pre-processing-e2e-ipp-b-2d9c39-99cc99448-78rp6\" (UID: \"252a57d2-9e17-4d68-83bb-fff34a75bcd9\") " pod="openshift-ingress/payload-pre-processing-e2e-ipp-b-2d9c39-99cc99448-78rp6" Jul 29 20:09:00.000862 ip-10-0-131-93 kubenswrapper[2564]: I0729 20:09:00.000836 2564 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/252a57d2-9e17-4d68-83bb-fff34a75bcd9-plugins-config-volume\") pod \"payload-pre-processing-e2e-ipp-b-2d9c39-99cc99448-78rp6\" (UID: \"252a57d2-9e17-4d68-83bb-fff34a75bcd9\") " pod="openshift-ingress/payload-pre-processing-e2e-ipp-b-2d9c39-99cc99448-78rp6" Jul 29 20:09:00.008672 ip-10-0-131-93 kubenswrapper[2564]: I0729 20:09:00.008648 2564 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-drrcn\" (UniqueName: \"kubernetes.io/projected/252a57d2-9e17-4d68-83bb-fff34a75bcd9-kube-api-access-drrcn\") pod \"payload-pre-processing-e2e-ipp-b-2d9c39-99cc99448-78rp6\" (UID: \"252a57d2-9e17-4d68-83bb-fff34a75bcd9\") " pod="openshift-ingress/payload-pre-processing-e2e-ipp-b-2d9c39-99cc99448-78rp6" Jul 29 20:09:00.096554 ip-10-0-131-93 kubenswrapper[2564]: I0729 20:09:00.096486 2564 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-pre-processing-e2e-ipp-b-2d9c39-99cc99448-78rp6" Jul 29 20:09:00.225359 ip-10-0-131-93 kubenswrapper[2564]: I0729 20:09:00.225329 2564 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/payload-pre-processing-e2e-ipp-b-2d9c39-99cc99448-78rp6"] Jul 29 20:09:00.226612 ip-10-0-131-93 kubenswrapper[2564]: W0729 20:09:00.226585 2564 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod252a57d2_9e17_4d68_83bb_fff34a75bcd9.slice/crio-b8458dfd2b6f46c5a4fbc5b0162b1bd387d5d2189c078e333f1ad8c255091721 WatchSource:0}: Error finding container b8458dfd2b6f46c5a4fbc5b0162b1bd387d5d2189c078e333f1ad8c255091721: Status 404 returned error can't find the container with id b8458dfd2b6f46c5a4fbc5b0162b1bd387d5d2189c078e333f1ad8c255091721 Jul 29 20:09:00.430035 ip-10-0-131-93 kubenswrapper[2564]: I0729 20:09:00.429951 2564 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-pre-processing-e2e-ipp-b-2d9c39-99cc99448-78rp6" event={"ID":"252a57d2-9e17-4d68-83bb-fff34a75bcd9","Type":"ContainerStarted","Data":"3e4e696f42dc7fe126bc7cb502b57d5dbf6b5131286b94e4629e0e5f19da2c87"} Jul 29 20:09:00.430035 ip-10-0-131-93 kubenswrapper[2564]: I0729 20:09:00.430004 2564 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-pre-processing-e2e-ipp-b-2d9c39-99cc99448-78rp6" event={"ID":"252a57d2-9e17-4d68-83bb-fff34a75bcd9","Type":"ContainerStarted","Data":"b8458dfd2b6f46c5a4fbc5b0162b1bd387d5d2189c078e333f1ad8c255091721"} Jul 29 20:09:00.430242 ip-10-0-131-93 kubenswrapper[2564]: I0729 20:09:00.430065 2564 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/payload-pre-processing-e2e-ipp-b-2d9c39-99cc99448-78rp6" Jul 29 20:09:00.446785 ip-10-0-131-93 kubenswrapper[2564]: I0729 20:09:00.446739 2564 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-ingress/payload-pre-processing-e2e-ipp-b-2d9c39-99cc99448-78rp6" podStartSLOduration=1.446723467 podStartE2EDuration="1.446723467s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-29 20:08:59 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-29 20:09:00.446106576 +0000 UTC m=+3116.170166269" watchObservedRunningTime="2026-07-29 20:09:00.446723467 +0000 UTC m=+3116.170783159" Jul 29 20:09:11.437980 ip-10-0-131-93 kubenswrapper[2564]: I0729 20:09:11.437945 2564 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/payload-pre-processing-e2e-ipp-b-2d9c39-99cc99448-78rp6" Jul 29 20:09:28.140699 ip-10-0-131-93 kubenswrapper[2564]: I0729 20:09:28.140626 2564 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-444jx_faa8c546-c75b-4e10-93b4-661aa2b5f53e/konnectivity-agent/0.log" limit=1024 Jul 29 20:09:35.398425 ip-10-0-131-93 kubenswrapper[2564]: I0729 20:09:35.398353 2564 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ingress_payload-processing-5cc6cd444b-zf8l2_21cbaa33-9f8b-47c3-8cc1-5a368b477eb2/payload-processing/0.log" Jul 29 20:09:40.873626 ip-10-0-131-93 kubenswrapper[2564]: I0729 20:09:40.873588 2564 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-444jx_faa8c546-c75b-4e10-93b4-661aa2b5f53e/konnectivity-agent/0.log" limit=1024 Jul 29 20:09:40.975079 ip-10-0-131-93 kubenswrapper[2564]: I0729 20:09:40.975041 2564 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-444jx_faa8c546-c75b-4e10-93b4-661aa2b5f53e/konnectivity-agent/0.log" limit=1024 Jul 29 20:09:41.036797 ip-10-0-131-93 kubenswrapper[2564]: I0729 20:09:41.036755 2564 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-444jx_faa8c546-c75b-4e10-93b4-661aa2b5f53e/konnectivity-agent/0.log" limit=1024 Jul 29 20:09:48.871389 ip-10-0-131-93 kubenswrapper[2564]: I0729 20:09:48.871316 2564 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-444jx_faa8c546-c75b-4e10-93b4-661aa2b5f53e/konnectivity-agent/0.log" limit=1024 Jul 29 20:09:53.954545 ip-10-0-131-93 kubenswrapper[2564]: I0729 20:09:53.954507 2564 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-444jx_faa8c546-c75b-4e10-93b4-661aa2b5f53e/konnectivity-agent/0.log" limit=1024 Jul 29 20:10:40.047021 ip-10-0-131-93 kubenswrapper[2564]: I0729 20:10:40.046988 2564 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-444jx_faa8c546-c75b-4e10-93b4-661aa2b5f53e/konnectivity-agent/0.log" limit=1024 Jul 29 20:10:40.864360 ip-10-0-131-93 kubenswrapper[2564]: I0729 20:10:40.864332 2564 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ingress_payload-processing-5cc6cd444b-zf8l2_21cbaa33-9f8b-47c3-8cc1-5a368b477eb2/payload-processing/0.log" Jul 29 20:10:41.247963 ip-10-0-131-93 kubenswrapper[2564]: I0729 20:10:41.247933 2564 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-444jx_faa8c546-c75b-4e10-93b4-661aa2b5f53e/konnectivity-agent/0.log" limit=1024 Jul 29 20:10:41.531834 ip-10-0-131-93 kubenswrapper[2564]: I0729 20:10:41.531763 2564 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-ingress/e2e-ipp-cleanup-5d7141-openshift-default-6678d486d-gbdmq"] Jul 29 20:10:41.534726 ip-10-0-131-93 kubenswrapper[2564]: I0729 20:10:41.534705 2564 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-ipp-cleanup-5d7141-openshift-default-6678d486d-gbdmq" Jul 29 20:10:41.537327 ip-10-0-131-93 kubenswrapper[2564]: I0729 20:10:41.537297 2564 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress\"/\"e2e-ipp-cleanup-5d7141-openshift-default-dockercfg-gksm9\"" Jul 29 20:10:41.548873 ip-10-0-131-93 kubenswrapper[2564]: I0729 20:10:41.548850 2564 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/e2e-ipp-cleanup-5d7141-openshift-default-6678d486d-gbdmq"] Jul 29 20:10:41.564417 ip-10-0-131-93 kubenswrapper[2564]: I0729 20:10:41.564366 2564 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-444jx_faa8c546-c75b-4e10-93b4-661aa2b5f53e/konnectivity-agent/0.log" limit=1024 Jul 29 20:10:41.588537 ip-10-0-131-93 kubenswrapper[2564]: I0729 20:10:41.588508 2564 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/b107bd8a-8f6f-4db0-8e9e-44a29271b1d5-istio-podinfo\") pod \"e2e-ipp-cleanup-5d7141-openshift-default-6678d486d-gbdmq\" (UID: \"b107bd8a-8f6f-4db0-8e9e-44a29271b1d5\") " pod="openshift-ingress/e2e-ipp-cleanup-5d7141-openshift-default-6678d486d-gbdmq" Jul 29 20:10:41.588641 ip-10-0-131-93 kubenswrapper[2564]: I0729 20:10:41.588556 2564 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/b107bd8a-8f6f-4db0-8e9e-44a29271b1d5-workload-socket\") pod \"e2e-ipp-cleanup-5d7141-openshift-default-6678d486d-gbdmq\" (UID: \"b107bd8a-8f6f-4db0-8e9e-44a29271b1d5\") " pod="openshift-ingress/e2e-ipp-cleanup-5d7141-openshift-default-6678d486d-gbdmq" Jul 29 20:10:41.588641 ip-10-0-131-93 kubenswrapper[2564]: I0729 20:10:41.588592 2564 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/b107bd8a-8f6f-4db0-8e9e-44a29271b1d5-istio-envoy\") pod \"e2e-ipp-cleanup-5d7141-openshift-default-6678d486d-gbdmq\" (UID: \"b107bd8a-8f6f-4db0-8e9e-44a29271b1d5\") " pod="openshift-ingress/e2e-ipp-cleanup-5d7141-openshift-default-6678d486d-gbdmq" Jul 29 20:10:41.588641 ip-10-0-131-93 kubenswrapper[2564]: I0729 20:10:41.588621 2564 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/b107bd8a-8f6f-4db0-8e9e-44a29271b1d5-istio-data\") pod \"e2e-ipp-cleanup-5d7141-openshift-default-6678d486d-gbdmq\" (UID: \"b107bd8a-8f6f-4db0-8e9e-44a29271b1d5\") " pod="openshift-ingress/e2e-ipp-cleanup-5d7141-openshift-default-6678d486d-gbdmq" Jul 29 20:10:41.588745 ip-10-0-131-93 kubenswrapper[2564]: I0729 20:10:41.588644 2564 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/b107bd8a-8f6f-4db0-8e9e-44a29271b1d5-workload-certs\") pod \"e2e-ipp-cleanup-5d7141-openshift-default-6678d486d-gbdmq\" (UID: \"b107bd8a-8f6f-4db0-8e9e-44a29271b1d5\") " pod="openshift-ingress/e2e-ipp-cleanup-5d7141-openshift-default-6678d486d-gbdmq" Jul 29 20:10:41.588745 ip-10-0-131-93 kubenswrapper[2564]: I0729 20:10:41.588661 2564 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/b107bd8a-8f6f-4db0-8e9e-44a29271b1d5-istio-token\") pod \"e2e-ipp-cleanup-5d7141-openshift-default-6678d486d-gbdmq\" (UID: \"b107bd8a-8f6f-4db0-8e9e-44a29271b1d5\") " pod="openshift-ingress/e2e-ipp-cleanup-5d7141-openshift-default-6678d486d-gbdmq" Jul 29 20:10:41.588745 ip-10-0-131-93 kubenswrapper[2564]: I0729 20:10:41.588682 2564 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-2jk7r\" (UniqueName: \"kubernetes.io/projected/b107bd8a-8f6f-4db0-8e9e-44a29271b1d5-kube-api-access-2jk7r\") pod \"e2e-ipp-cleanup-5d7141-openshift-default-6678d486d-gbdmq\" (UID: \"b107bd8a-8f6f-4db0-8e9e-44a29271b1d5\") " pod="openshift-ingress/e2e-ipp-cleanup-5d7141-openshift-default-6678d486d-gbdmq" Jul 29 20:10:41.588745 ip-10-0-131-93 kubenswrapper[2564]: I0729 20:10:41.588699 2564 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/b107bd8a-8f6f-4db0-8e9e-44a29271b1d5-istiod-ca-cert\") pod \"e2e-ipp-cleanup-5d7141-openshift-default-6678d486d-gbdmq\" (UID: \"b107bd8a-8f6f-4db0-8e9e-44a29271b1d5\") " pod="openshift-ingress/e2e-ipp-cleanup-5d7141-openshift-default-6678d486d-gbdmq" Jul 29 20:10:41.588874 ip-10-0-131-93 kubenswrapper[2564]: I0729 20:10:41.588800 2564 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/b107bd8a-8f6f-4db0-8e9e-44a29271b1d5-credential-socket\") pod \"e2e-ipp-cleanup-5d7141-openshift-default-6678d486d-gbdmq\" (UID: \"b107bd8a-8f6f-4db0-8e9e-44a29271b1d5\") " pod="openshift-ingress/e2e-ipp-cleanup-5d7141-openshift-default-6678d486d-gbdmq" Jul 29 20:10:41.689295 ip-10-0-131-93 kubenswrapper[2564]: I0729 20:10:41.689256 2564 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/b107bd8a-8f6f-4db0-8e9e-44a29271b1d5-workload-socket\") pod \"e2e-ipp-cleanup-5d7141-openshift-default-6678d486d-gbdmq\" (UID: \"b107bd8a-8f6f-4db0-8e9e-44a29271b1d5\") " pod="openshift-ingress/e2e-ipp-cleanup-5d7141-openshift-default-6678d486d-gbdmq" Jul 29 20:10:41.689498 ip-10-0-131-93 kubenswrapper[2564]: I0729 20:10:41.689308 2564 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/b107bd8a-8f6f-4db0-8e9e-44a29271b1d5-istio-envoy\") pod \"e2e-ipp-cleanup-5d7141-openshift-default-6678d486d-gbdmq\" (UID: \"b107bd8a-8f6f-4db0-8e9e-44a29271b1d5\") " pod="openshift-ingress/e2e-ipp-cleanup-5d7141-openshift-default-6678d486d-gbdmq" Jul 29 20:10:41.689498 ip-10-0-131-93 kubenswrapper[2564]: I0729 20:10:41.689422 2564 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/b107bd8a-8f6f-4db0-8e9e-44a29271b1d5-istio-data\") pod \"e2e-ipp-cleanup-5d7141-openshift-default-6678d486d-gbdmq\" (UID: \"b107bd8a-8f6f-4db0-8e9e-44a29271b1d5\") " pod="openshift-ingress/e2e-ipp-cleanup-5d7141-openshift-default-6678d486d-gbdmq" Jul 29 20:10:41.689498 ip-10-0-131-93 kubenswrapper[2564]: I0729 20:10:41.689470 2564 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/b107bd8a-8f6f-4db0-8e9e-44a29271b1d5-workload-certs\") pod \"e2e-ipp-cleanup-5d7141-openshift-default-6678d486d-gbdmq\" (UID: \"b107bd8a-8f6f-4db0-8e9e-44a29271b1d5\") " pod="openshift-ingress/e2e-ipp-cleanup-5d7141-openshift-default-6678d486d-gbdmq" Jul 29 20:10:41.689498 ip-10-0-131-93 kubenswrapper[2564]: I0729 20:10:41.689499 2564 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/b107bd8a-8f6f-4db0-8e9e-44a29271b1d5-istio-token\") pod \"e2e-ipp-cleanup-5d7141-openshift-default-6678d486d-gbdmq\" (UID: \"b107bd8a-8f6f-4db0-8e9e-44a29271b1d5\") " pod="openshift-ingress/e2e-ipp-cleanup-5d7141-openshift-default-6678d486d-gbdmq" Jul 29 20:10:41.689737 ip-10-0-131-93 kubenswrapper[2564]: I0729 20:10:41.689524 2564 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-2jk7r\" (UniqueName: \"kubernetes.io/projected/b107bd8a-8f6f-4db0-8e9e-44a29271b1d5-kube-api-access-2jk7r\") pod \"e2e-ipp-cleanup-5d7141-openshift-default-6678d486d-gbdmq\" (UID: \"b107bd8a-8f6f-4db0-8e9e-44a29271b1d5\") " pod="openshift-ingress/e2e-ipp-cleanup-5d7141-openshift-default-6678d486d-gbdmq" Jul 29 20:10:41.689737 ip-10-0-131-93 kubenswrapper[2564]: I0729 20:10:41.689545 2564 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/b107bd8a-8f6f-4db0-8e9e-44a29271b1d5-istiod-ca-cert\") pod \"e2e-ipp-cleanup-5d7141-openshift-default-6678d486d-gbdmq\" (UID: \"b107bd8a-8f6f-4db0-8e9e-44a29271b1d5\") " pod="openshift-ingress/e2e-ipp-cleanup-5d7141-openshift-default-6678d486d-gbdmq" Jul 29 20:10:41.689737 ip-10-0-131-93 kubenswrapper[2564]: I0729 20:10:41.689677 2564 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/b107bd8a-8f6f-4db0-8e9e-44a29271b1d5-workload-socket\") pod \"e2e-ipp-cleanup-5d7141-openshift-default-6678d486d-gbdmq\" (UID: \"b107bd8a-8f6f-4db0-8e9e-44a29271b1d5\") " pod="openshift-ingress/e2e-ipp-cleanup-5d7141-openshift-default-6678d486d-gbdmq" Jul 29 20:10:41.689899 ip-10-0-131-93 kubenswrapper[2564]: I0729 20:10:41.689773 2564 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/b107bd8a-8f6f-4db0-8e9e-44a29271b1d5-credential-socket\") pod \"e2e-ipp-cleanup-5d7141-openshift-default-6678d486d-gbdmq\" (UID: \"b107bd8a-8f6f-4db0-8e9e-44a29271b1d5\") " pod="openshift-ingress/e2e-ipp-cleanup-5d7141-openshift-default-6678d486d-gbdmq" Jul 29 20:10:41.689899 ip-10-0-131-93 kubenswrapper[2564]: I0729 20:10:41.689785 2564 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/b107bd8a-8f6f-4db0-8e9e-44a29271b1d5-istio-data\") pod \"e2e-ipp-cleanup-5d7141-openshift-default-6678d486d-gbdmq\" (UID: \"b107bd8a-8f6f-4db0-8e9e-44a29271b1d5\") " pod="openshift-ingress/e2e-ipp-cleanup-5d7141-openshift-default-6678d486d-gbdmq" Jul 29 20:10:41.689899 ip-10-0-131-93 kubenswrapper[2564]: I0729 20:10:41.689820 2564 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/b107bd8a-8f6f-4db0-8e9e-44a29271b1d5-istio-podinfo\") pod \"e2e-ipp-cleanup-5d7141-openshift-default-6678d486d-gbdmq\" (UID: \"b107bd8a-8f6f-4db0-8e9e-44a29271b1d5\") " pod="openshift-ingress/e2e-ipp-cleanup-5d7141-openshift-default-6678d486d-gbdmq" Jul 29 20:10:41.689899 ip-10-0-131-93 kubenswrapper[2564]: I0729 20:10:41.689892 2564 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/b107bd8a-8f6f-4db0-8e9e-44a29271b1d5-workload-certs\") pod \"e2e-ipp-cleanup-5d7141-openshift-default-6678d486d-gbdmq\" (UID: \"b107bd8a-8f6f-4db0-8e9e-44a29271b1d5\") " pod="openshift-ingress/e2e-ipp-cleanup-5d7141-openshift-default-6678d486d-gbdmq" Jul 29 20:10:41.690104 ip-10-0-131-93 kubenswrapper[2564]: I0729 20:10:41.690047 2564 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/b107bd8a-8f6f-4db0-8e9e-44a29271b1d5-credential-socket\") pod \"e2e-ipp-cleanup-5d7141-openshift-default-6678d486d-gbdmq\" (UID: \"b107bd8a-8f6f-4db0-8e9e-44a29271b1d5\") " pod="openshift-ingress/e2e-ipp-cleanup-5d7141-openshift-default-6678d486d-gbdmq" Jul 29 20:10:41.690265 ip-10-0-131-93 kubenswrapper[2564]: I0729 20:10:41.690239 2564 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/b107bd8a-8f6f-4db0-8e9e-44a29271b1d5-istiod-ca-cert\") pod \"e2e-ipp-cleanup-5d7141-openshift-default-6678d486d-gbdmq\" (UID: \"b107bd8a-8f6f-4db0-8e9e-44a29271b1d5\") " pod="openshift-ingress/e2e-ipp-cleanup-5d7141-openshift-default-6678d486d-gbdmq" Jul 29 20:10:41.691681 ip-10-0-131-93 kubenswrapper[2564]: I0729 20:10:41.691651 2564 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/b107bd8a-8f6f-4db0-8e9e-44a29271b1d5-istio-envoy\") pod \"e2e-ipp-cleanup-5d7141-openshift-default-6678d486d-gbdmq\" (UID: \"b107bd8a-8f6f-4db0-8e9e-44a29271b1d5\") " pod="openshift-ingress/e2e-ipp-cleanup-5d7141-openshift-default-6678d486d-gbdmq" Jul 29 20:10:41.692227 ip-10-0-131-93 kubenswrapper[2564]: I0729 20:10:41.692207 2564 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/b107bd8a-8f6f-4db0-8e9e-44a29271b1d5-istio-podinfo\") pod \"e2e-ipp-cleanup-5d7141-openshift-default-6678d486d-gbdmq\" (UID: \"b107bd8a-8f6f-4db0-8e9e-44a29271b1d5\") " pod="openshift-ingress/e2e-ipp-cleanup-5d7141-openshift-default-6678d486d-gbdmq" Jul 29 20:10:41.699203 ip-10-0-131-93 kubenswrapper[2564]: I0729 20:10:41.699182 2564 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/b107bd8a-8f6f-4db0-8e9e-44a29271b1d5-istio-token\") pod \"e2e-ipp-cleanup-5d7141-openshift-default-6678d486d-gbdmq\" (UID: \"b107bd8a-8f6f-4db0-8e9e-44a29271b1d5\") " pod="openshift-ingress/e2e-ipp-cleanup-5d7141-openshift-default-6678d486d-gbdmq" Jul 29 20:10:41.699496 ip-10-0-131-93 kubenswrapper[2564]: I0729 20:10:41.699473 2564 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-2jk7r\" (UniqueName: \"kubernetes.io/projected/b107bd8a-8f6f-4db0-8e9e-44a29271b1d5-kube-api-access-2jk7r\") pod \"e2e-ipp-cleanup-5d7141-openshift-default-6678d486d-gbdmq\" (UID: \"b107bd8a-8f6f-4db0-8e9e-44a29271b1d5\") " pod="openshift-ingress/e2e-ipp-cleanup-5d7141-openshift-default-6678d486d-gbdmq" Jul 29 20:10:41.844847 ip-10-0-131-93 kubenswrapper[2564]: I0729 20:10:41.844769 2564 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-ipp-cleanup-5d7141-openshift-default-6678d486d-gbdmq" Jul 29 20:10:41.983402 ip-10-0-131-93 kubenswrapper[2564]: I0729 20:10:41.983345 2564 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/e2e-ipp-cleanup-5d7141-openshift-default-6678d486d-gbdmq"] Jul 29 20:10:41.984206 ip-10-0-131-93 kubenswrapper[2564]: W0729 20:10:41.984179 2564 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podb107bd8a_8f6f_4db0_8e9e_44a29271b1d5.slice/crio-e709e8829b737deb52494cf52a89bcaa208bf9903f396ed4b81f4037a97f3fe0 WatchSource:0}: Error finding container e709e8829b737deb52494cf52a89bcaa208bf9903f396ed4b81f4037a97f3fe0: Status 404 returned error can't find the container with id e709e8829b737deb52494cf52a89bcaa208bf9903f396ed4b81f4037a97f3fe0 Jul 29 20:10:41.986618 ip-10-0-131-93 kubenswrapper[2564]: I0729 20:10:41.986584 2564 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236224Ki","pods":"250"} Jul 29 20:10:41.986705 ip-10-0-131-93 kubenswrapper[2564]: I0729 20:10:41.986651 2564 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236224Ki","pods":"250"} Jul 29 20:10:41.986705 ip-10-0-131-93 kubenswrapper[2564]: I0729 20:10:41.986682 2564 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236224Ki","pods":"250"} Jul 29 20:10:42.030211 ip-10-0-131-93 kubenswrapper[2564]: I0729 20:10:42.030047 2564 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-444jx_faa8c546-c75b-4e10-93b4-661aa2b5f53e/konnectivity-agent/0.log" limit=1024 Jul 29 20:10:42.947534 ip-10-0-131-93 kubenswrapper[2564]: I0729 20:10:42.947497 2564 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-ipp-cleanup-5d7141-openshift-default-6678d486d-gbdmq" event={"ID":"b107bd8a-8f6f-4db0-8e9e-44a29271b1d5","Type":"ContainerStarted","Data":"a692e3648b395e8a1a302d155e4363b08265b55b20ec068de691aa8167bbd2c2"} Jul 29 20:10:42.947534 ip-10-0-131-93 kubenswrapper[2564]: I0729 20:10:42.947553 2564 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-ipp-cleanup-5d7141-openshift-default-6678d486d-gbdmq" event={"ID":"b107bd8a-8f6f-4db0-8e9e-44a29271b1d5","Type":"ContainerStarted","Data":"e709e8829b737deb52494cf52a89bcaa208bf9903f396ed4b81f4037a97f3fe0"} Jul 29 20:10:42.974364 ip-10-0-131-93 kubenswrapper[2564]: I0729 20:10:42.974305 2564 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-ingress/e2e-ipp-cleanup-5d7141-openshift-default-6678d486d-gbdmq" podStartSLOduration=1.9742887169999999 podStartE2EDuration="1.974288717s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-29 20:10:41 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-29 20:10:42.971394452 +0000 UTC m=+3218.695454136" watchObservedRunningTime="2026-07-29 20:10:42.974288717 +0000 UTC m=+3218.698348409" Jul 29 20:10:43.845552 ip-10-0-131-93 kubenswrapper[2564]: I0729 20:10:43.845513 2564 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="openshift-ingress/e2e-ipp-cleanup-5d7141-openshift-default-6678d486d-gbdmq" Jul 29 20:10:43.850497 ip-10-0-131-93 kubenswrapper[2564]: I0729 20:10:43.850472 2564 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="openshift-ingress/e2e-ipp-cleanup-5d7141-openshift-default-6678d486d-gbdmq" Jul 29 20:10:43.952436 ip-10-0-131-93 kubenswrapper[2564]: I0729 20:10:43.952404 2564 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/e2e-ipp-cleanup-5d7141-openshift-default-6678d486d-gbdmq" Jul 29 20:10:43.953316 ip-10-0-131-93 kubenswrapper[2564]: I0729 20:10:43.953297 2564 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/e2e-ipp-cleanup-5d7141-openshift-default-6678d486d-gbdmq" Jul 29 20:10:44.969104 ip-10-0-131-93 kubenswrapper[2564]: I0729 20:10:44.969075 2564 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-444jx_faa8c546-c75b-4e10-93b4-661aa2b5f53e/konnectivity-agent/0.log" limit=1024 Jul 29 20:10:45.119673 ip-10-0-131-93 kubenswrapper[2564]: I0729 20:10:45.119617 2564 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-444jx_faa8c546-c75b-4e10-93b4-661aa2b5f53e/konnectivity-agent/0.log" limit=1024 Jul 29 20:10:46.233270 ip-10-0-131-93 kubenswrapper[2564]: I0729 20:10:46.233240 2564 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-ipp-cleanup-5d7141-5965d58c76-vvpx4"] Jul 29 20:10:46.235694 ip-10-0-131-93 kubenswrapper[2564]: I0729 20:10:46.235671 2564 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-e2e-ipp-cleanup-5d7141-5965d58c76-vvpx4" Jul 29 20:10:46.238306 ip-10-0-131-93 kubenswrapper[2564]: I0729 20:10:46.238276 2564 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"odh-ai-gateway-infra\"/\"maas-api-e2e-ipp-cleanup-5d7141-serving-cert\"" Jul 29 20:10:46.238306 ip-10-0-131-93 kubenswrapper[2564]: I0729 20:10:46.238302 2564 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"odh-ai-gateway-infra\"/\"maas-api-dockercfg-4dw6s\"" Jul 29 20:10:46.248792 ip-10-0-131-93 kubenswrapper[2564]: I0729 20:10:46.248767 2564 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-ipp-cleanup-5d7141-5965d58c76-vvpx4"] Jul 29 20:10:46.263339 ip-10-0-131-93 kubenswrapper[2564]: I0729 20:10:46.263311 2564 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-444jx_faa8c546-c75b-4e10-93b4-661aa2b5f53e/konnectivity-agent/0.log" limit=1024 Jul 29 20:10:46.334704 ip-10-0-131-93 kubenswrapper[2564]: I0729 20:10:46.334668 2564 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-77crl\" (UniqueName: \"kubernetes.io/projected/00002c08-1197-4913-9b28-4442dcb6b959-kube-api-access-77crl\") pod \"maas-api-e2e-ipp-cleanup-5d7141-5965d58c76-vvpx4\" (UID: \"00002c08-1197-4913-9b28-4442dcb6b959\") " pod="odh-ai-gateway-infra/maas-api-e2e-ipp-cleanup-5d7141-5965d58c76-vvpx4" Jul 29 20:10:46.334902 ip-10-0-131-93 kubenswrapper[2564]: I0729 20:10:46.334716 2564 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/00002c08-1197-4913-9b28-4442dcb6b959-openshift-service-ca\") pod \"maas-api-e2e-ipp-cleanup-5d7141-5965d58c76-vvpx4\" (UID: \"00002c08-1197-4913-9b28-4442dcb6b959\") " pod="odh-ai-gateway-infra/maas-api-e2e-ipp-cleanup-5d7141-5965d58c76-vvpx4" Jul 29 20:10:46.334902 ip-10-0-131-93 kubenswrapper[2564]: I0729 20:10:46.334838 2564 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/00002c08-1197-4913-9b28-4442dcb6b959-maas-api-tls\") pod \"maas-api-e2e-ipp-cleanup-5d7141-5965d58c76-vvpx4\" (UID: \"00002c08-1197-4913-9b28-4442dcb6b959\") " pod="odh-ai-gateway-infra/maas-api-e2e-ipp-cleanup-5d7141-5965d58c76-vvpx4" Jul 29 20:10:46.367073 ip-10-0-131-93 kubenswrapper[2564]: I0729 20:10:46.367039 2564 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-444jx_faa8c546-c75b-4e10-93b4-661aa2b5f53e/konnectivity-agent/0.log" limit=1024 Jul 29 20:10:46.436326 ip-10-0-131-93 kubenswrapper[2564]: I0729 20:10:46.436291 2564 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-77crl\" (UniqueName: \"kubernetes.io/projected/00002c08-1197-4913-9b28-4442dcb6b959-kube-api-access-77crl\") pod \"maas-api-e2e-ipp-cleanup-5d7141-5965d58c76-vvpx4\" (UID: \"00002c08-1197-4913-9b28-4442dcb6b959\") " pod="odh-ai-gateway-infra/maas-api-e2e-ipp-cleanup-5d7141-5965d58c76-vvpx4" Jul 29 20:10:46.436526 ip-10-0-131-93 kubenswrapper[2564]: I0729 20:10:46.436347 2564 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/00002c08-1197-4913-9b28-4442dcb6b959-openshift-service-ca\") pod \"maas-api-e2e-ipp-cleanup-5d7141-5965d58c76-vvpx4\" (UID: \"00002c08-1197-4913-9b28-4442dcb6b959\") " pod="odh-ai-gateway-infra/maas-api-e2e-ipp-cleanup-5d7141-5965d58c76-vvpx4" Jul 29 20:10:46.436526 ip-10-0-131-93 kubenswrapper[2564]: I0729 20:10:46.436455 2564 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/00002c08-1197-4913-9b28-4442dcb6b959-maas-api-tls\") pod \"maas-api-e2e-ipp-cleanup-5d7141-5965d58c76-vvpx4\" (UID: \"00002c08-1197-4913-9b28-4442dcb6b959\") " pod="odh-ai-gateway-infra/maas-api-e2e-ipp-cleanup-5d7141-5965d58c76-vvpx4" Jul 29 20:10:46.437142 ip-10-0-131-93 kubenswrapper[2564]: I0729 20:10:46.437122 2564 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/00002c08-1197-4913-9b28-4442dcb6b959-openshift-service-ca\") pod \"maas-api-e2e-ipp-cleanup-5d7141-5965d58c76-vvpx4\" (UID: \"00002c08-1197-4913-9b28-4442dcb6b959\") " pod="odh-ai-gateway-infra/maas-api-e2e-ipp-cleanup-5d7141-5965d58c76-vvpx4" Jul 29 20:10:46.438886 ip-10-0-131-93 kubenswrapper[2564]: I0729 20:10:46.438861 2564 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/00002c08-1197-4913-9b28-4442dcb6b959-maas-api-tls\") pod \"maas-api-e2e-ipp-cleanup-5d7141-5965d58c76-vvpx4\" (UID: \"00002c08-1197-4913-9b28-4442dcb6b959\") " pod="odh-ai-gateway-infra/maas-api-e2e-ipp-cleanup-5d7141-5965d58c76-vvpx4" Jul 29 20:10:46.443920 ip-10-0-131-93 kubenswrapper[2564]: I0729 20:10:46.443898 2564 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-77crl\" (UniqueName: \"kubernetes.io/projected/00002c08-1197-4913-9b28-4442dcb6b959-kube-api-access-77crl\") pod \"maas-api-e2e-ipp-cleanup-5d7141-5965d58c76-vvpx4\" (UID: \"00002c08-1197-4913-9b28-4442dcb6b959\") " pod="odh-ai-gateway-infra/maas-api-e2e-ipp-cleanup-5d7141-5965d58c76-vvpx4" Jul 29 20:10:46.550447 ip-10-0-131-93 kubenswrapper[2564]: I0729 20:10:46.550342 2564 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-e2e-ipp-cleanup-5d7141-5965d58c76-vvpx4" Jul 29 20:10:46.683956 ip-10-0-131-93 kubenswrapper[2564]: I0729 20:10:46.683930 2564 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-ipp-cleanup-5d7141-5965d58c76-vvpx4"] Jul 29 20:10:46.685606 ip-10-0-131-93 kubenswrapper[2564]: W0729 20:10:46.685582 2564 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod00002c08_1197_4913_9b28_4442dcb6b959.slice/crio-0937ad90a0ccf2277a8318a5e095964ba9f7c47a1510dfa37b26eed5c37224a6 WatchSource:0}: Error finding container 0937ad90a0ccf2277a8318a5e095964ba9f7c47a1510dfa37b26eed5c37224a6: Status 404 returned error can't find the container with id 0937ad90a0ccf2277a8318a5e095964ba9f7c47a1510dfa37b26eed5c37224a6 Jul 29 20:10:46.688086 ip-10-0-131-93 kubenswrapper[2564]: I0729 20:10:46.688061 2564 provider.go:93] Refreshing cache for provider: *credentialprovider.defaultDockerConfigProvider Jul 29 20:10:46.736917 ip-10-0-131-93 kubenswrapper[2564]: I0729 20:10:46.736880 2564 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-444jx_faa8c546-c75b-4e10-93b4-661aa2b5f53e/konnectivity-agent/0.log" limit=1024 Jul 29 20:10:46.839473 ip-10-0-131-93 kubenswrapper[2564]: I0729 20:10:46.839365 2564 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-444jx_faa8c546-c75b-4e10-93b4-661aa2b5f53e/konnectivity-agent/0.log" limit=1024 Jul 29 20:10:46.840470 ip-10-0-131-93 kubenswrapper[2564]: I0729 20:10:46.840448 2564 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-ingress/payload-pre-processing-e2e-ipp-cleanup-5d7141-64fc798b8d-mdhkh"] Jul 29 20:10:46.843585 ip-10-0-131-93 kubenswrapper[2564]: I0729 20:10:46.843570 2564 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-pre-processing-e2e-ipp-cleanup-5d7141-64fc798b8d-mdhkh" Jul 29 20:10:46.846139 ip-10-0-131-93 kubenswrapper[2564]: I0729 20:10:46.846120 2564 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ingress\"/\"payload-processing-plugins-e2e-ipp-cleanup-5d7141\"" Jul 29 20:10:46.846139 ip-10-0-131-93 kubenswrapper[2564]: I0729 20:10:46.846128 2564 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress\"/\"payload-processing-e2e-ipp-cleanup-5d7141-dockercfg-d5slm\"" Jul 29 20:10:46.854506 ip-10-0-131-93 kubenswrapper[2564]: I0729 20:10:46.854484 2564 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/payload-pre-processing-e2e-ipp-cleanup-5d7141-64fc798b8d-mdhkh"] Jul 29 20:10:46.945791 ip-10-0-131-93 kubenswrapper[2564]: I0729 20:10:46.944013 2564 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/b7200237-7be7-4661-ba46-f13ec3fc68a8-plugins-config-volume\") pod \"payload-pre-processing-e2e-ipp-cleanup-5d7141-64fc798b8d-mdhkh\" (UID: \"b7200237-7be7-4661-ba46-f13ec3fc68a8\") " pod="openshift-ingress/payload-pre-processing-e2e-ipp-cleanup-5d7141-64fc798b8d-mdhkh" Jul 29 20:10:46.945791 ip-10-0-131-93 kubenswrapper[2564]: I0729 20:10:46.944075 2564 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-njkbq\" (UniqueName: \"kubernetes.io/projected/b7200237-7be7-4661-ba46-f13ec3fc68a8-kube-api-access-njkbq\") pod \"payload-pre-processing-e2e-ipp-cleanup-5d7141-64fc798b8d-mdhkh\" (UID: \"b7200237-7be7-4661-ba46-f13ec3fc68a8\") " pod="openshift-ingress/payload-pre-processing-e2e-ipp-cleanup-5d7141-64fc798b8d-mdhkh" Jul 29 20:10:46.976840 ip-10-0-131-93 kubenswrapper[2564]: I0729 20:10:46.974966 2564 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-e2e-ipp-cleanup-5d7141-5965d58c76-vvpx4" event={"ID":"00002c08-1197-4913-9b28-4442dcb6b959","Type":"ContainerStarted","Data":"0937ad90a0ccf2277a8318a5e095964ba9f7c47a1510dfa37b26eed5c37224a6"} Jul 29 20:10:47.044763 ip-10-0-131-93 kubenswrapper[2564]: I0729 20:10:47.044723 2564 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/b7200237-7be7-4661-ba46-f13ec3fc68a8-plugins-config-volume\") pod \"payload-pre-processing-e2e-ipp-cleanup-5d7141-64fc798b8d-mdhkh\" (UID: \"b7200237-7be7-4661-ba46-f13ec3fc68a8\") " pod="openshift-ingress/payload-pre-processing-e2e-ipp-cleanup-5d7141-64fc798b8d-mdhkh" Jul 29 20:10:47.044953 ip-10-0-131-93 kubenswrapper[2564]: I0729 20:10:47.044766 2564 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-njkbq\" (UniqueName: \"kubernetes.io/projected/b7200237-7be7-4661-ba46-f13ec3fc68a8-kube-api-access-njkbq\") pod \"payload-pre-processing-e2e-ipp-cleanup-5d7141-64fc798b8d-mdhkh\" (UID: \"b7200237-7be7-4661-ba46-f13ec3fc68a8\") " pod="openshift-ingress/payload-pre-processing-e2e-ipp-cleanup-5d7141-64fc798b8d-mdhkh" Jul 29 20:10:47.045459 ip-10-0-131-93 kubenswrapper[2564]: I0729 20:10:47.045435 2564 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/b7200237-7be7-4661-ba46-f13ec3fc68a8-plugins-config-volume\") pod \"payload-pre-processing-e2e-ipp-cleanup-5d7141-64fc798b8d-mdhkh\" (UID: \"b7200237-7be7-4661-ba46-f13ec3fc68a8\") " pod="openshift-ingress/payload-pre-processing-e2e-ipp-cleanup-5d7141-64fc798b8d-mdhkh" Jul 29 20:10:47.053818 ip-10-0-131-93 kubenswrapper[2564]: I0729 20:10:47.053798 2564 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-njkbq\" (UniqueName: \"kubernetes.io/projected/b7200237-7be7-4661-ba46-f13ec3fc68a8-kube-api-access-njkbq\") pod \"payload-pre-processing-e2e-ipp-cleanup-5d7141-64fc798b8d-mdhkh\" (UID: \"b7200237-7be7-4661-ba46-f13ec3fc68a8\") " pod="openshift-ingress/payload-pre-processing-e2e-ipp-cleanup-5d7141-64fc798b8d-mdhkh" Jul 29 20:10:47.156319 ip-10-0-131-93 kubenswrapper[2564]: I0729 20:10:47.156225 2564 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-pre-processing-e2e-ipp-cleanup-5d7141-64fc798b8d-mdhkh" Jul 29 20:10:47.492561 ip-10-0-131-93 kubenswrapper[2564]: I0729 20:10:47.492532 2564 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/payload-pre-processing-e2e-ipp-cleanup-5d7141-64fc798b8d-mdhkh"] Jul 29 20:10:47.494211 ip-10-0-131-93 kubenswrapper[2564]: W0729 20:10:47.494187 2564 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podb7200237_7be7_4661_ba46_f13ec3fc68a8.slice/crio-9000fe8127b05b9d335c71917ec90cbb6fff82710aba40a0f74cf0356c2a3a8d WatchSource:0}: Error finding container 9000fe8127b05b9d335c71917ec90cbb6fff82710aba40a0f74cf0356c2a3a8d: Status 404 returned error can't find the container with id 9000fe8127b05b9d335c71917ec90cbb6fff82710aba40a0f74cf0356c2a3a8d Jul 29 20:10:47.981918 ip-10-0-131-93 kubenswrapper[2564]: I0729 20:10:47.981875 2564 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-pre-processing-e2e-ipp-cleanup-5d7141-64fc798b8d-mdhkh" event={"ID":"b7200237-7be7-4661-ba46-f13ec3fc68a8","Type":"ContainerStarted","Data":"af95ed6396add23da3c0c87cf35223b7264266e04e3069aa31fa68a46105b1e4"} Jul 29 20:10:47.982118 ip-10-0-131-93 kubenswrapper[2564]: I0729 20:10:47.981943 2564 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-pre-processing-e2e-ipp-cleanup-5d7141-64fc798b8d-mdhkh" event={"ID":"b7200237-7be7-4661-ba46-f13ec3fc68a8","Type":"ContainerStarted","Data":"9000fe8127b05b9d335c71917ec90cbb6fff82710aba40a0f74cf0356c2a3a8d"} Jul 29 20:10:47.982118 ip-10-0-131-93 kubenswrapper[2564]: I0729 20:10:47.981987 2564 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/payload-pre-processing-e2e-ipp-cleanup-5d7141-64fc798b8d-mdhkh" Jul 29 20:10:47.983497 ip-10-0-131-93 kubenswrapper[2564]: I0729 20:10:47.983473 2564 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-e2e-ipp-cleanup-5d7141-5965d58c76-vvpx4" event={"ID":"00002c08-1197-4913-9b28-4442dcb6b959","Type":"ContainerStarted","Data":"dee0f7e0e0f94007470b5e802da1268aede187d4c7f6a591cc1507320ea55e74"} Jul 29 20:10:47.983588 ip-10-0-131-93 kubenswrapper[2564]: I0729 20:10:47.983527 2564 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="odh-ai-gateway-infra/maas-api-e2e-ipp-cleanup-5d7141-5965d58c76-vvpx4" Jul 29 20:10:47.997862 ip-10-0-131-93 kubenswrapper[2564]: I0729 20:10:47.997813 2564 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-ingress/payload-pre-processing-e2e-ipp-cleanup-5d7141-64fc798b8d-mdhkh" podStartSLOduration=1.997800592 podStartE2EDuration="1.997800592s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-29 20:10:46 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-29 20:10:47.99713712 +0000 UTC m=+3223.721196812" watchObservedRunningTime="2026-07-29 20:10:47.997800592 +0000 UTC m=+3223.721860320" Jul 29 20:10:48.014869 ip-10-0-131-93 kubenswrapper[2564]: I0729 20:10:48.014802 2564 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="odh-ai-gateway-infra/maas-api-e2e-ipp-cleanup-5d7141-5965d58c76-vvpx4" podStartSLOduration=1.418827981 podStartE2EDuration="2.014790138s" totalImagesPullingTime="595.962157ms" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-29 20:10:46 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-29 20:10:48.013553038 +0000 UTC m=+3223.737612730" watchObservedRunningTime="2026-07-29 20:10:48.014790138 +0000 UTC m=+3223.738849831" Jul 29 20:10:53.995340 ip-10-0-131-93 kubenswrapper[2564]: I0729 20:10:53.995309 2564 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="odh-ai-gateway-infra/maas-api-e2e-ipp-cleanup-5d7141-5965d58c76-vvpx4" Jul 29 20:10:58.993238 ip-10-0-131-93 kubenswrapper[2564]: I0729 20:10:58.993207 2564 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/payload-pre-processing-e2e-ipp-cleanup-5d7141-64fc798b8d-mdhkh" Jul 29 20:11:06.088828 ip-10-0-131-93 kubenswrapper[2564]: I0729 20:11:06.088741 2564 generic.go:356] "Generic (PLEG): container finished" podID="b107bd8a-8f6f-4db0-8e9e-44a29271b1d5" containerID="a692e3648b395e8a1a302d155e4363b08265b55b20ec068de691aa8167bbd2c2" exitCode=0 Jul 29 20:11:06.089230 ip-10-0-131-93 kubenswrapper[2564]: I0729 20:11:06.088823 2564 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-ipp-cleanup-5d7141-openshift-default-6678d486d-gbdmq" event={"ID":"b107bd8a-8f6f-4db0-8e9e-44a29271b1d5","Type":"ContainerDied","Data":"a692e3648b395e8a1a302d155e4363b08265b55b20ec068de691aa8167bbd2c2"} Jul 29 20:11:06.089230 ip-10-0-131-93 kubenswrapper[2564]: I0729 20:11:06.089226 2564 scope.go:117] "RemoveContainer" containerID="a692e3648b395e8a1a302d155e4363b08265b55b20ec068de691aa8167bbd2c2" Jul 29 20:11:06.090465 ip-10-0-131-93 kubenswrapper[2564]: I0729 20:11:06.090434 2564 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236224Ki","pods":"250"} Jul 29 20:11:06.090536 ip-10-0-131-93 kubenswrapper[2564]: I0729 20:11:06.090504 2564 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236224Ki","pods":"250"} Jul 29 20:11:06.090572 ip-10-0-131-93 kubenswrapper[2564]: I0729 20:11:06.090548 2564 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236224Ki","pods":"250"} Jul 29 20:11:07.096341 ip-10-0-131-93 kubenswrapper[2564]: I0729 20:11:07.096304 2564 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-ipp-cleanup-5d7141-openshift-default-6678d486d-gbdmq" event={"ID":"b107bd8a-8f6f-4db0-8e9e-44a29271b1d5","Type":"ContainerStarted","Data":"ce9be9961c6822d2072390ad2c872e1138f0e0476efa1e22a38b6e9d56018ff5"} Jul 29 20:11:07.845777 ip-10-0-131-93 kubenswrapper[2564]: I0729 20:11:07.845741 2564 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="openshift-ingress/e2e-ipp-cleanup-5d7141-openshift-default-6678d486d-gbdmq" Jul 29 20:11:07.845954 ip-10-0-131-93 kubenswrapper[2564]: I0729 20:11:07.845913 2564 patch_prober.go:28] interesting pod/e2e-ipp-cleanup-5d7141-openshift-default-6678d486d-gbdmq container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.132.0.52:15021/healthz/ready\": dial tcp 10.132.0.52:15021: connect: connection refused" start-of-body= Jul 29 20:11:07.846012 ip-10-0-131-93 kubenswrapper[2564]: I0729 20:11:07.845959 2564 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-ipp-cleanup-5d7141-openshift-default-6678d486d-gbdmq" podUID="b107bd8a-8f6f-4db0-8e9e-44a29271b1d5" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.132.0.52:15021/healthz/ready\": dial tcp 10.132.0.52:15021: connect: connection refused" Jul 29 20:11:08.845639 ip-10-0-131-93 kubenswrapper[2564]: I0729 20:11:08.845603 2564 patch_prober.go:28] interesting pod/e2e-ipp-cleanup-5d7141-openshift-default-6678d486d-gbdmq container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.132.0.52:15021/healthz/ready\": dial tcp 10.132.0.52:15021: connect: connection refused" start-of-body= Jul 29 20:11:08.846040 ip-10-0-131-93 kubenswrapper[2564]: I0729 20:11:08.845669 2564 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-ipp-cleanup-5d7141-openshift-default-6678d486d-gbdmq" podUID="b107bd8a-8f6f-4db0-8e9e-44a29271b1d5" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.132.0.52:15021/healthz/ready\": dial tcp 10.132.0.52:15021: connect: connection refused" Jul 29 20:11:09.845528 ip-10-0-131-93 kubenswrapper[2564]: I0729 20:11:09.845489 2564 patch_prober.go:28] interesting pod/e2e-ipp-cleanup-5d7141-openshift-default-6678d486d-gbdmq container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.132.0.52:15021/healthz/ready\": dial tcp 10.132.0.52:15021: connect: connection refused" start-of-body= Jul 29 20:11:09.845713 ip-10-0-131-93 kubenswrapper[2564]: I0729 20:11:09.845544 2564 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-ipp-cleanup-5d7141-openshift-default-6678d486d-gbdmq" podUID="b107bd8a-8f6f-4db0-8e9e-44a29271b1d5" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.132.0.52:15021/healthz/ready\": dial tcp 10.132.0.52:15021: connect: connection refused" Jul 29 20:11:09.916248 ip-10-0-131-93 kubenswrapper[2564]: I0729 20:11:09.916196 2564 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-444jx_faa8c546-c75b-4e10-93b4-661aa2b5f53e/konnectivity-agent/0.log" limit=1024 Jul 29 20:11:10.845624 ip-10-0-131-93 kubenswrapper[2564]: I0729 20:11:10.845590 2564 patch_prober.go:28] interesting pod/e2e-ipp-cleanup-5d7141-openshift-default-6678d486d-gbdmq container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.132.0.52:15021/healthz/ready\": dial tcp 10.132.0.52:15021: connect: connection refused" start-of-body= Jul 29 20:11:10.845899 ip-10-0-131-93 kubenswrapper[2564]: I0729 20:11:10.845660 2564 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-ipp-cleanup-5d7141-openshift-default-6678d486d-gbdmq" podUID="b107bd8a-8f6f-4db0-8e9e-44a29271b1d5" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.132.0.52:15021/healthz/ready\": dial tcp 10.132.0.52:15021: connect: connection refused" Jul 29 20:11:11.849660 ip-10-0-131-93 kubenswrapper[2564]: I0729 20:11:11.849629 2564 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="openshift-ingress/e2e-ipp-cleanup-5d7141-openshift-default-6678d486d-gbdmq" Jul 29 20:11:11.850053 ip-10-0-131-93 kubenswrapper[2564]: I0729 20:11:11.849912 2564 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/e2e-ipp-cleanup-5d7141-openshift-default-6678d486d-gbdmq" Jul 29 20:11:11.850672 ip-10-0-131-93 kubenswrapper[2564]: I0729 20:11:11.850654 2564 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/e2e-ipp-cleanup-5d7141-openshift-default-6678d486d-gbdmq" Jul 29 20:11:16.743903 ip-10-0-131-93 kubenswrapper[2564]: I0729 20:11:16.743870 2564 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-ipp-cleanup-5d7141-5965d58c76-vvpx4"] Jul 29 20:11:16.744358 ip-10-0-131-93 kubenswrapper[2564]: I0729 20:11:16.744155 2564 kuberuntime_container.go:873] "Killing container with a grace period" pod="odh-ai-gateway-infra/maas-api-e2e-ipp-cleanup-5d7141-5965d58c76-vvpx4" podUID="00002c08-1197-4913-9b28-4442dcb6b959" containerName="maas-api" containerID="cri-o://dee0f7e0e0f94007470b5e802da1268aede187d4c7f6a591cc1507320ea55e74" gracePeriod=30 Jul 29 20:11:16.775562 ip-10-0-131-93 kubenswrapper[2564]: I0729 20:11:16.775524 2564 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-444jx_faa8c546-c75b-4e10-93b4-661aa2b5f53e/konnectivity-agent/0.log" limit=1024 Jul 29 20:11:16.810126 ip-10-0-131-93 kubenswrapper[2564]: I0729 20:11:16.810094 2564 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-ingress/payload-pre-processing-e2e-ipp-cleanup-5d7141-64fc798b8d-mdhkh"] Jul 29 20:11:16.810361 ip-10-0-131-93 kubenswrapper[2564]: I0729 20:11:16.810336 2564 kuberuntime_container.go:873] "Killing container with a grace period" pod="openshift-ingress/payload-pre-processing-e2e-ipp-cleanup-5d7141-64fc798b8d-mdhkh" podUID="b7200237-7be7-4661-ba46-f13ec3fc68a8" containerName="payload-pre-processing" containerID="cri-o://af95ed6396add23da3c0c87cf35223b7264266e04e3069aa31fa68a46105b1e4" gracePeriod=30 Jul 29 20:11:16.875799 ip-10-0-131-93 kubenswrapper[2564]: I0729 20:11:16.875770 2564 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-444jx_faa8c546-c75b-4e10-93b4-661aa2b5f53e/konnectivity-agent/0.log" limit=1024 Jul 29 20:11:17.116065 ip-10-0-131-93 kubenswrapper[2564]: I0729 20:11:17.116031 2564 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-e2e-ipp-cleanup-5d7141-5965d58c76-vvpx4" Jul 29 20:11:17.119023 ip-10-0-131-93 kubenswrapper[2564]: I0729 20:11:17.119003 2564 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-pre-processing-e2e-ipp-cleanup-5d7141-64fc798b8d-mdhkh" Jul 29 20:11:17.122003 ip-10-0-131-93 kubenswrapper[2564]: I0729 20:11:17.121983 2564 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/00002c08-1197-4913-9b28-4442dcb6b959-maas-api-tls\") pod \"00002c08-1197-4913-9b28-4442dcb6b959\" (UID: \"00002c08-1197-4913-9b28-4442dcb6b959\") " Jul 29 20:11:17.122073 ip-10-0-131-93 kubenswrapper[2564]: I0729 20:11:17.122011 2564 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-njkbq\" (UniqueName: \"kubernetes.io/projected/b7200237-7be7-4661-ba46-f13ec3fc68a8-kube-api-access-njkbq\") pod \"b7200237-7be7-4661-ba46-f13ec3fc68a8\" (UID: \"b7200237-7be7-4661-ba46-f13ec3fc68a8\") " Jul 29 20:11:17.122073 ip-10-0-131-93 kubenswrapper[2564]: I0729 20:11:17.122037 2564 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/00002c08-1197-4913-9b28-4442dcb6b959-openshift-service-ca\") pod \"00002c08-1197-4913-9b28-4442dcb6b959\" (UID: \"00002c08-1197-4913-9b28-4442dcb6b959\") " Jul 29 20:11:17.122073 ip-10-0-131-93 kubenswrapper[2564]: I0729 20:11:17.122066 2564 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/b7200237-7be7-4661-ba46-f13ec3fc68a8-plugins-config-volume\") pod \"b7200237-7be7-4661-ba46-f13ec3fc68a8\" (UID: \"b7200237-7be7-4661-ba46-f13ec3fc68a8\") " Jul 29 20:11:17.122233 ip-10-0-131-93 kubenswrapper[2564]: I0729 20:11:17.122162 2564 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-77crl\" (UniqueName: \"kubernetes.io/projected/00002c08-1197-4913-9b28-4442dcb6b959-kube-api-access-77crl\") pod \"00002c08-1197-4913-9b28-4442dcb6b959\" (UID: \"00002c08-1197-4913-9b28-4442dcb6b959\") " Jul 29 20:11:17.122444 ip-10-0-131-93 kubenswrapper[2564]: I0729 20:11:17.122418 2564 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/00002c08-1197-4913-9b28-4442dcb6b959-openshift-service-ca" (OuterVolumeSpecName: "openshift-service-ca") pod "00002c08-1197-4913-9b28-4442dcb6b959" (UID: "00002c08-1197-4913-9b28-4442dcb6b959"). InnerVolumeSpecName "openshift-service-ca". PluginName "kubernetes.io/configmap", VolumeGIDValue "" Jul 29 20:11:17.122539 ip-10-0-131-93 kubenswrapper[2564]: I0729 20:11:17.122459 2564 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/b7200237-7be7-4661-ba46-f13ec3fc68a8-plugins-config-volume" (OuterVolumeSpecName: "plugins-config-volume") pod "b7200237-7be7-4661-ba46-f13ec3fc68a8" (UID: "b7200237-7be7-4661-ba46-f13ec3fc68a8"). InnerVolumeSpecName "plugins-config-volume". PluginName "kubernetes.io/configmap", VolumeGIDValue "" Jul 29 20:11:17.122601 ip-10-0-131-93 kubenswrapper[2564]: I0729 20:11:17.122564 2564 reconciler_common.go:299] "Volume detached for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/00002c08-1197-4913-9b28-4442dcb6b959-openshift-service-ca\") on node \"ip-10-0-131-93.ec2.internal\" DevicePath \"\"" Jul 29 20:11:17.122601 ip-10-0-131-93 kubenswrapper[2564]: I0729 20:11:17.122584 2564 reconciler_common.go:299] "Volume detached for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/b7200237-7be7-4661-ba46-f13ec3fc68a8-plugins-config-volume\") on node \"ip-10-0-131-93.ec2.internal\" DevicePath \"\"" Jul 29 20:11:17.124166 ip-10-0-131-93 kubenswrapper[2564]: I0729 20:11:17.124140 2564 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/b7200237-7be7-4661-ba46-f13ec3fc68a8-kube-api-access-njkbq" (OuterVolumeSpecName: "kube-api-access-njkbq") pod "b7200237-7be7-4661-ba46-f13ec3fc68a8" (UID: "b7200237-7be7-4661-ba46-f13ec3fc68a8"). InnerVolumeSpecName "kube-api-access-njkbq". PluginName "kubernetes.io/projected", VolumeGIDValue "" Jul 29 20:11:17.124253 ip-10-0-131-93 kubenswrapper[2564]: I0729 20:11:17.124185 2564 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/00002c08-1197-4913-9b28-4442dcb6b959-kube-api-access-77crl" (OuterVolumeSpecName: "kube-api-access-77crl") pod "00002c08-1197-4913-9b28-4442dcb6b959" (UID: "00002c08-1197-4913-9b28-4442dcb6b959"). InnerVolumeSpecName "kube-api-access-77crl". PluginName "kubernetes.io/projected", VolumeGIDValue "" Jul 29 20:11:17.124253 ip-10-0-131-93 kubenswrapper[2564]: I0729 20:11:17.124193 2564 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/00002c08-1197-4913-9b28-4442dcb6b959-maas-api-tls" (OuterVolumeSpecName: "maas-api-tls") pod "00002c08-1197-4913-9b28-4442dcb6b959" (UID: "00002c08-1197-4913-9b28-4442dcb6b959"). InnerVolumeSpecName "maas-api-tls". PluginName "kubernetes.io/secret", VolumeGIDValue "" Jul 29 20:11:17.152770 ip-10-0-131-93 kubenswrapper[2564]: I0729 20:11:17.152531 2564 generic.go:356] "Generic (PLEG): container finished" podID="b7200237-7be7-4661-ba46-f13ec3fc68a8" containerID="af95ed6396add23da3c0c87cf35223b7264266e04e3069aa31fa68a46105b1e4" exitCode=0 Jul 29 20:11:17.152770 ip-10-0-131-93 kubenswrapper[2564]: I0729 20:11:17.152746 2564 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-pre-processing-e2e-ipp-cleanup-5d7141-64fc798b8d-mdhkh" event={"ID":"b7200237-7be7-4661-ba46-f13ec3fc68a8","Type":"ContainerDied","Data":"af95ed6396add23da3c0c87cf35223b7264266e04e3069aa31fa68a46105b1e4"} Jul 29 20:11:17.153041 ip-10-0-131-93 kubenswrapper[2564]: I0729 20:11:17.152806 2564 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-pre-processing-e2e-ipp-cleanup-5d7141-64fc798b8d-mdhkh" event={"ID":"b7200237-7be7-4661-ba46-f13ec3fc68a8","Type":"ContainerDied","Data":"9000fe8127b05b9d335c71917ec90cbb6fff82710aba40a0f74cf0356c2a3a8d"} Jul 29 20:11:17.153041 ip-10-0-131-93 kubenswrapper[2564]: I0729 20:11:17.152838 2564 scope.go:117] "RemoveContainer" containerID="af95ed6396add23da3c0c87cf35223b7264266e04e3069aa31fa68a46105b1e4" Jul 29 20:11:17.153041 ip-10-0-131-93 kubenswrapper[2564]: I0729 20:11:17.153007 2564 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-pre-processing-e2e-ipp-cleanup-5d7141-64fc798b8d-mdhkh" Jul 29 20:11:17.156894 ip-10-0-131-93 kubenswrapper[2564]: I0729 20:11:17.156740 2564 generic.go:356] "Generic (PLEG): container finished" podID="00002c08-1197-4913-9b28-4442dcb6b959" containerID="dee0f7e0e0f94007470b5e802da1268aede187d4c7f6a591cc1507320ea55e74" exitCode=0 Jul 29 20:11:17.156894 ip-10-0-131-93 kubenswrapper[2564]: I0729 20:11:17.156793 2564 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-e2e-ipp-cleanup-5d7141-5965d58c76-vvpx4" Jul 29 20:11:17.156894 ip-10-0-131-93 kubenswrapper[2564]: I0729 20:11:17.156801 2564 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-e2e-ipp-cleanup-5d7141-5965d58c76-vvpx4" event={"ID":"00002c08-1197-4913-9b28-4442dcb6b959","Type":"ContainerDied","Data":"dee0f7e0e0f94007470b5e802da1268aede187d4c7f6a591cc1507320ea55e74"} Jul 29 20:11:17.156894 ip-10-0-131-93 kubenswrapper[2564]: I0729 20:11:17.156857 2564 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-e2e-ipp-cleanup-5d7141-5965d58c76-vvpx4" event={"ID":"00002c08-1197-4913-9b28-4442dcb6b959","Type":"ContainerDied","Data":"0937ad90a0ccf2277a8318a5e095964ba9f7c47a1510dfa37b26eed5c37224a6"} Jul 29 20:11:17.164677 ip-10-0-131-93 kubenswrapper[2564]: I0729 20:11:17.164466 2564 scope.go:117] "RemoveContainer" containerID="af95ed6396add23da3c0c87cf35223b7264266e04e3069aa31fa68a46105b1e4" Jul 29 20:11:17.164853 ip-10-0-131-93 kubenswrapper[2564]: E0729 20:11:17.164819 2564 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"af95ed6396add23da3c0c87cf35223b7264266e04e3069aa31fa68a46105b1e4\": container with ID starting with af95ed6396add23da3c0c87cf35223b7264266e04e3069aa31fa68a46105b1e4 not found: ID does not exist" containerID="af95ed6396add23da3c0c87cf35223b7264266e04e3069aa31fa68a46105b1e4" Jul 29 20:11:17.164975 ip-10-0-131-93 kubenswrapper[2564]: I0729 20:11:17.164858 2564 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"af95ed6396add23da3c0c87cf35223b7264266e04e3069aa31fa68a46105b1e4"} err="failed to get container status \"af95ed6396add23da3c0c87cf35223b7264266e04e3069aa31fa68a46105b1e4\": rpc error: code = NotFound desc = could not find container \"af95ed6396add23da3c0c87cf35223b7264266e04e3069aa31fa68a46105b1e4\": container with ID starting with af95ed6396add23da3c0c87cf35223b7264266e04e3069aa31fa68a46105b1e4 not found: ID does not exist" Jul 29 20:11:17.164975 ip-10-0-131-93 kubenswrapper[2564]: I0729 20:11:17.164882 2564 scope.go:117] "RemoveContainer" containerID="dee0f7e0e0f94007470b5e802da1268aede187d4c7f6a591cc1507320ea55e74" Jul 29 20:11:17.174076 ip-10-0-131-93 kubenswrapper[2564]: I0729 20:11:17.174058 2564 scope.go:117] "RemoveContainer" containerID="dee0f7e0e0f94007470b5e802da1268aede187d4c7f6a591cc1507320ea55e74" Jul 29 20:11:17.174342 ip-10-0-131-93 kubenswrapper[2564]: E0729 20:11:17.174322 2564 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"dee0f7e0e0f94007470b5e802da1268aede187d4c7f6a591cc1507320ea55e74\": container with ID starting with dee0f7e0e0f94007470b5e802da1268aede187d4c7f6a591cc1507320ea55e74 not found: ID does not exist" containerID="dee0f7e0e0f94007470b5e802da1268aede187d4c7f6a591cc1507320ea55e74" Jul 29 20:11:17.174468 ip-10-0-131-93 kubenswrapper[2564]: I0729 20:11:17.174350 2564 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"dee0f7e0e0f94007470b5e802da1268aede187d4c7f6a591cc1507320ea55e74"} err="failed to get container status \"dee0f7e0e0f94007470b5e802da1268aede187d4c7f6a591cc1507320ea55e74\": rpc error: code = NotFound desc = could not find container \"dee0f7e0e0f94007470b5e802da1268aede187d4c7f6a591cc1507320ea55e74\": container with ID starting with dee0f7e0e0f94007470b5e802da1268aede187d4c7f6a591cc1507320ea55e74 not found: ID does not exist" Jul 29 20:11:17.179119 ip-10-0-131-93 kubenswrapper[2564]: I0729 20:11:17.179096 2564 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-ingress/payload-pre-processing-e2e-ipp-cleanup-5d7141-64fc798b8d-mdhkh"] Jul 29 20:11:17.185213 ip-10-0-131-93 kubenswrapper[2564]: I0729 20:11:17.185192 2564 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["openshift-ingress/payload-pre-processing-e2e-ipp-cleanup-5d7141-64fc798b8d-mdhkh"] Jul 29 20:11:17.197324 ip-10-0-131-93 kubenswrapper[2564]: I0729 20:11:17.197302 2564 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-ipp-cleanup-5d7141-5965d58c76-vvpx4"] Jul 29 20:11:17.203061 ip-10-0-131-93 kubenswrapper[2564]: I0729 20:11:17.203039 2564 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-ipp-cleanup-5d7141-5965d58c76-vvpx4"] Jul 29 20:11:17.222963 ip-10-0-131-93 kubenswrapper[2564]: I0729 20:11:17.222941 2564 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-77crl\" (UniqueName: \"kubernetes.io/projected/00002c08-1197-4913-9b28-4442dcb6b959-kube-api-access-77crl\") on node \"ip-10-0-131-93.ec2.internal\" DevicePath \"\"" Jul 29 20:11:17.223056 ip-10-0-131-93 kubenswrapper[2564]: I0729 20:11:17.222964 2564 reconciler_common.go:299] "Volume detached for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/00002c08-1197-4913-9b28-4442dcb6b959-maas-api-tls\") on node \"ip-10-0-131-93.ec2.internal\" DevicePath \"\"" Jul 29 20:11:17.223056 ip-10-0-131-93 kubenswrapper[2564]: I0729 20:11:17.222980 2564 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-njkbq\" (UniqueName: \"kubernetes.io/projected/b7200237-7be7-4661-ba46-f13ec3fc68a8-kube-api-access-njkbq\") on node \"ip-10-0-131-93.ec2.internal\" DevicePath \"\"" Jul 29 20:11:17.412693 ip-10-0-131-93 kubenswrapper[2564]: I0729 20:11:17.412610 2564 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-444jx_faa8c546-c75b-4e10-93b4-661aa2b5f53e/konnectivity-agent/0.log" limit=1024 Jul 29 20:11:17.513564 ip-10-0-131-93 kubenswrapper[2564]: I0729 20:11:17.513529 2564 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-444jx_faa8c546-c75b-4e10-93b4-661aa2b5f53e/konnectivity-agent/0.log" limit=1024 Jul 29 20:11:18.783074 ip-10-0-131-93 kubenswrapper[2564]: I0729 20:11:18.783041 2564 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="00002c08-1197-4913-9b28-4442dcb6b959" path="/var/lib/kubelet/pods/00002c08-1197-4913-9b28-4442dcb6b959/volumes" Jul 29 20:11:18.783610 ip-10-0-131-93 kubenswrapper[2564]: I0729 20:11:18.783590 2564 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="b7200237-7be7-4661-ba46-f13ec3fc68a8" path="/var/lib/kubelet/pods/b7200237-7be7-4661-ba46-f13ec3fc68a8/volumes" Jul 29 20:11:23.176624 ip-10-0-131-93 kubenswrapper[2564]: I0729 20:11:23.176586 2564 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-444jx_faa8c546-c75b-4e10-93b4-661aa2b5f53e/konnectivity-agent/0.log" limit=1024 Jul 29 20:11:23.238933 ip-10-0-131-93 kubenswrapper[2564]: I0729 20:11:23.238899 2564 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-444jx_faa8c546-c75b-4e10-93b4-661aa2b5f53e/konnectivity-agent/0.log" limit=1024 Jul 29 20:11:23.414148 ip-10-0-131-93 kubenswrapper[2564]: I0729 20:11:23.414113 2564 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-ingress/e2e-ipp-cleanup-5d7141-openshift-default-6678d486d-gbdmq"] Jul 29 20:11:23.414442 ip-10-0-131-93 kubenswrapper[2564]: I0729 20:11:23.414410 2564 kuberuntime_container.go:873] "Killing container with a grace period" pod="openshift-ingress/e2e-ipp-cleanup-5d7141-openshift-default-6678d486d-gbdmq" podUID="b107bd8a-8f6f-4db0-8e9e-44a29271b1d5" containerName="istio-proxy" containerID="cri-o://ce9be9961c6822d2072390ad2c872e1138f0e0476efa1e22a38b6e9d56018ff5" gracePeriod=30 Jul 29 20:11:23.429409 ip-10-0-131-93 kubenswrapper[2564]: I0729 20:11:23.429338 2564 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-444jx_faa8c546-c75b-4e10-93b4-661aa2b5f53e/konnectivity-agent/0.log" limit=1024 Jul 29 20:11:24.185317 ip-10-0-131-93 kubenswrapper[2564]: I0729 20:11:24.185287 2564 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-444jx_faa8c546-c75b-4e10-93b4-661aa2b5f53e/konnectivity-agent/0.log" limit=1024 Jul 29 20:11:26.852247 ip-10-0-131-93 kubenswrapper[2564]: I0729 20:11:26.852208 2564 patch_prober.go:28] interesting pod/e2e-ipp-cleanup-5d7141-openshift-default-6678d486d-gbdmq container/istio-proxy namespace/openshift-ingress: Readiness probe status=failure output="HTTP probe failed with statuscode: 503" start-of-body= Jul 29 20:11:26.852649 ip-10-0-131-93 kubenswrapper[2564]: I0729 20:11:26.852277 2564 prober.go:120] "Probe failed" probeType="Readiness" pod="openshift-ingress/e2e-ipp-cleanup-5d7141-openshift-default-6678d486d-gbdmq" podUID="b107bd8a-8f6f-4db0-8e9e-44a29271b1d5" containerName="istio-proxy" probeResult="failure" output="HTTP probe failed with statuscode: 503" Jul 29 20:11:28.680280 ip-10-0-131-93 kubenswrapper[2564]: I0729 20:11:28.680218 2564 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-ipp-cleanup-5d7141-openshift-default-6678d486d-gbdmq" Jul 29 20:11:28.731883 ip-10-0-131-93 kubenswrapper[2564]: I0729 20:11:28.731854 2564 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/b107bd8a-8f6f-4db0-8e9e-44a29271b1d5-istio-podinfo\") pod \"b107bd8a-8f6f-4db0-8e9e-44a29271b1d5\" (UID: \"b107bd8a-8f6f-4db0-8e9e-44a29271b1d5\") " Jul 29 20:11:28.732048 ip-10-0-131-93 kubenswrapper[2564]: I0729 20:11:28.731914 2564 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/b107bd8a-8f6f-4db0-8e9e-44a29271b1d5-workload-socket\") pod \"b107bd8a-8f6f-4db0-8e9e-44a29271b1d5\" (UID: \"b107bd8a-8f6f-4db0-8e9e-44a29271b1d5\") " Jul 29 20:11:28.732048 ip-10-0-131-93 kubenswrapper[2564]: I0729 20:11:28.731934 2564 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/b107bd8a-8f6f-4db0-8e9e-44a29271b1d5-credential-socket\") pod \"b107bd8a-8f6f-4db0-8e9e-44a29271b1d5\" (UID: \"b107bd8a-8f6f-4db0-8e9e-44a29271b1d5\") " Jul 29 20:11:28.732153 ip-10-0-131-93 kubenswrapper[2564]: I0729 20:11:28.732052 2564 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/b107bd8a-8f6f-4db0-8e9e-44a29271b1d5-istio-data\") pod \"b107bd8a-8f6f-4db0-8e9e-44a29271b1d5\" (UID: \"b107bd8a-8f6f-4db0-8e9e-44a29271b1d5\") " Jul 29 20:11:28.732153 ip-10-0-131-93 kubenswrapper[2564]: I0729 20:11:28.732113 2564 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-2jk7r\" (UniqueName: \"kubernetes.io/projected/b107bd8a-8f6f-4db0-8e9e-44a29271b1d5-kube-api-access-2jk7r\") pod \"b107bd8a-8f6f-4db0-8e9e-44a29271b1d5\" (UID: \"b107bd8a-8f6f-4db0-8e9e-44a29271b1d5\") " Jul 29 20:11:28.732153 ip-10-0-131-93 kubenswrapper[2564]: I0729 20:11:28.732141 2564 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/b107bd8a-8f6f-4db0-8e9e-44a29271b1d5-istiod-ca-cert\") pod \"b107bd8a-8f6f-4db0-8e9e-44a29271b1d5\" (UID: \"b107bd8a-8f6f-4db0-8e9e-44a29271b1d5\") " Jul 29 20:11:28.732322 ip-10-0-131-93 kubenswrapper[2564]: I0729 20:11:28.732190 2564 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/b107bd8a-8f6f-4db0-8e9e-44a29271b1d5-workload-socket" (OuterVolumeSpecName: "workload-socket") pod "b107bd8a-8f6f-4db0-8e9e-44a29271b1d5" (UID: "b107bd8a-8f6f-4db0-8e9e-44a29271b1d5"). InnerVolumeSpecName "workload-socket". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Jul 29 20:11:28.732322 ip-10-0-131-93 kubenswrapper[2564]: I0729 20:11:28.732212 2564 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/b107bd8a-8f6f-4db0-8e9e-44a29271b1d5-credential-socket" (OuterVolumeSpecName: "credential-socket") pod "b107bd8a-8f6f-4db0-8e9e-44a29271b1d5" (UID: "b107bd8a-8f6f-4db0-8e9e-44a29271b1d5"). InnerVolumeSpecName "credential-socket". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Jul 29 20:11:28.732322 ip-10-0-131-93 kubenswrapper[2564]: I0729 20:11:28.732217 2564 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/b107bd8a-8f6f-4db0-8e9e-44a29271b1d5-workload-certs\") pod \"b107bd8a-8f6f-4db0-8e9e-44a29271b1d5\" (UID: \"b107bd8a-8f6f-4db0-8e9e-44a29271b1d5\") " Jul 29 20:11:28.732322 ip-10-0-131-93 kubenswrapper[2564]: I0729 20:11:28.732306 2564 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/b107bd8a-8f6f-4db0-8e9e-44a29271b1d5-istio-token\") pod \"b107bd8a-8f6f-4db0-8e9e-44a29271b1d5\" (UID: \"b107bd8a-8f6f-4db0-8e9e-44a29271b1d5\") " Jul 29 20:11:28.732548 ip-10-0-131-93 kubenswrapper[2564]: I0729 20:11:28.732336 2564 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/b107bd8a-8f6f-4db0-8e9e-44a29271b1d5-istio-envoy\") pod \"b107bd8a-8f6f-4db0-8e9e-44a29271b1d5\" (UID: \"b107bd8a-8f6f-4db0-8e9e-44a29271b1d5\") " Jul 29 20:11:28.732548 ip-10-0-131-93 kubenswrapper[2564]: I0729 20:11:28.732405 2564 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/b107bd8a-8f6f-4db0-8e9e-44a29271b1d5-workload-certs" (OuterVolumeSpecName: "workload-certs") pod "b107bd8a-8f6f-4db0-8e9e-44a29271b1d5" (UID: "b107bd8a-8f6f-4db0-8e9e-44a29271b1d5"). InnerVolumeSpecName "workload-certs". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Jul 29 20:11:28.732548 ip-10-0-131-93 kubenswrapper[2564]: I0729 20:11:28.732536 2564 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/b107bd8a-8f6f-4db0-8e9e-44a29271b1d5-istiod-ca-cert" (OuterVolumeSpecName: "istiod-ca-cert") pod "b107bd8a-8f6f-4db0-8e9e-44a29271b1d5" (UID: "b107bd8a-8f6f-4db0-8e9e-44a29271b1d5"). InnerVolumeSpecName "istiod-ca-cert". PluginName "kubernetes.io/configmap", VolumeGIDValue "" Jul 29 20:11:28.732726 ip-10-0-131-93 kubenswrapper[2564]: I0729 20:11:28.732704 2564 reconciler_common.go:299] "Volume detached for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/b107bd8a-8f6f-4db0-8e9e-44a29271b1d5-istiod-ca-cert\") on node \"ip-10-0-131-93.ec2.internal\" DevicePath \"\"" Jul 29 20:11:28.732779 ip-10-0-131-93 kubenswrapper[2564]: I0729 20:11:28.732731 2564 reconciler_common.go:299] "Volume detached for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/b107bd8a-8f6f-4db0-8e9e-44a29271b1d5-workload-certs\") on node \"ip-10-0-131-93.ec2.internal\" DevicePath \"\"" Jul 29 20:11:28.732779 ip-10-0-131-93 kubenswrapper[2564]: I0729 20:11:28.732744 2564 reconciler_common.go:299] "Volume detached for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/b107bd8a-8f6f-4db0-8e9e-44a29271b1d5-workload-socket\") on node \"ip-10-0-131-93.ec2.internal\" DevicePath \"\"" Jul 29 20:11:28.732779 ip-10-0-131-93 kubenswrapper[2564]: I0729 20:11:28.732757 2564 reconciler_common.go:299] "Volume detached for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/b107bd8a-8f6f-4db0-8e9e-44a29271b1d5-credential-socket\") on node \"ip-10-0-131-93.ec2.internal\" DevicePath \"\"" Jul 29 20:11:28.733085 ip-10-0-131-93 kubenswrapper[2564]: I0729 20:11:28.733058 2564 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/b107bd8a-8f6f-4db0-8e9e-44a29271b1d5-istio-data" (OuterVolumeSpecName: "istio-data") pod "b107bd8a-8f6f-4db0-8e9e-44a29271b1d5" (UID: "b107bd8a-8f6f-4db0-8e9e-44a29271b1d5"). InnerVolumeSpecName "istio-data". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Jul 29 20:11:28.734518 ip-10-0-131-93 kubenswrapper[2564]: I0729 20:11:28.734489 2564 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/downward-api/b107bd8a-8f6f-4db0-8e9e-44a29271b1d5-istio-podinfo" (OuterVolumeSpecName: "istio-podinfo") pod "b107bd8a-8f6f-4db0-8e9e-44a29271b1d5" (UID: "b107bd8a-8f6f-4db0-8e9e-44a29271b1d5"). InnerVolumeSpecName "istio-podinfo". PluginName "kubernetes.io/downward-api", VolumeGIDValue "" Jul 29 20:11:28.734747 ip-10-0-131-93 kubenswrapper[2564]: I0729 20:11:28.734716 2564 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/b107bd8a-8f6f-4db0-8e9e-44a29271b1d5-istio-token" (OuterVolumeSpecName: "istio-token") pod "b107bd8a-8f6f-4db0-8e9e-44a29271b1d5" (UID: "b107bd8a-8f6f-4db0-8e9e-44a29271b1d5"). InnerVolumeSpecName "istio-token". PluginName "kubernetes.io/projected", VolumeGIDValue "" Jul 29 20:11:28.734833 ip-10-0-131-93 kubenswrapper[2564]: I0729 20:11:28.734787 2564 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/b107bd8a-8f6f-4db0-8e9e-44a29271b1d5-istio-envoy" (OuterVolumeSpecName: "istio-envoy") pod "b107bd8a-8f6f-4db0-8e9e-44a29271b1d5" (UID: "b107bd8a-8f6f-4db0-8e9e-44a29271b1d5"). InnerVolumeSpecName "istio-envoy". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Jul 29 20:11:28.735049 ip-10-0-131-93 kubenswrapper[2564]: I0729 20:11:28.735023 2564 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/b107bd8a-8f6f-4db0-8e9e-44a29271b1d5-kube-api-access-2jk7r" (OuterVolumeSpecName: "kube-api-access-2jk7r") pod "b107bd8a-8f6f-4db0-8e9e-44a29271b1d5" (UID: "b107bd8a-8f6f-4db0-8e9e-44a29271b1d5"). InnerVolumeSpecName "kube-api-access-2jk7r". PluginName "kubernetes.io/projected", VolumeGIDValue "" Jul 29 20:11:28.833862 ip-10-0-131-93 kubenswrapper[2564]: I0729 20:11:28.833832 2564 reconciler_common.go:299] "Volume detached for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/b107bd8a-8f6f-4db0-8e9e-44a29271b1d5-istio-podinfo\") on node \"ip-10-0-131-93.ec2.internal\" DevicePath \"\"" Jul 29 20:11:28.833862 ip-10-0-131-93 kubenswrapper[2564]: I0729 20:11:28.833857 2564 reconciler_common.go:299] "Volume detached for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/b107bd8a-8f6f-4db0-8e9e-44a29271b1d5-istio-data\") on node \"ip-10-0-131-93.ec2.internal\" DevicePath \"\"" Jul 29 20:11:28.833862 ip-10-0-131-93 kubenswrapper[2564]: I0729 20:11:28.833866 2564 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-2jk7r\" (UniqueName: \"kubernetes.io/projected/b107bd8a-8f6f-4db0-8e9e-44a29271b1d5-kube-api-access-2jk7r\") on node \"ip-10-0-131-93.ec2.internal\" DevicePath \"\"" Jul 29 20:11:28.834070 ip-10-0-131-93 kubenswrapper[2564]: I0729 20:11:28.833876 2564 reconciler_common.go:299] "Volume detached for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/b107bd8a-8f6f-4db0-8e9e-44a29271b1d5-istio-token\") on node \"ip-10-0-131-93.ec2.internal\" DevicePath \"\"" Jul 29 20:11:28.834070 ip-10-0-131-93 kubenswrapper[2564]: I0729 20:11:28.833885 2564 reconciler_common.go:299] "Volume detached for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/b107bd8a-8f6f-4db0-8e9e-44a29271b1d5-istio-envoy\") on node \"ip-10-0-131-93.ec2.internal\" DevicePath \"\"" Jul 29 20:11:29.222472 ip-10-0-131-93 kubenswrapper[2564]: I0729 20:11:29.222436 2564 generic.go:356] "Generic (PLEG): container finished" podID="b107bd8a-8f6f-4db0-8e9e-44a29271b1d5" containerID="ce9be9961c6822d2072390ad2c872e1138f0e0476efa1e22a38b6e9d56018ff5" exitCode=0 Jul 29 20:11:29.222638 ip-10-0-131-93 kubenswrapper[2564]: I0729 20:11:29.222501 2564 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-ipp-cleanup-5d7141-openshift-default-6678d486d-gbdmq" Jul 29 20:11:29.222638 ip-10-0-131-93 kubenswrapper[2564]: I0729 20:11:29.222528 2564 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-ipp-cleanup-5d7141-openshift-default-6678d486d-gbdmq" event={"ID":"b107bd8a-8f6f-4db0-8e9e-44a29271b1d5","Type":"ContainerDied","Data":"ce9be9961c6822d2072390ad2c872e1138f0e0476efa1e22a38b6e9d56018ff5"} Jul 29 20:11:29.222638 ip-10-0-131-93 kubenswrapper[2564]: I0729 20:11:29.222578 2564 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-ipp-cleanup-5d7141-openshift-default-6678d486d-gbdmq" event={"ID":"b107bd8a-8f6f-4db0-8e9e-44a29271b1d5","Type":"ContainerDied","Data":"e709e8829b737deb52494cf52a89bcaa208bf9903f396ed4b81f4037a97f3fe0"} Jul 29 20:11:29.222638 ip-10-0-131-93 kubenswrapper[2564]: I0729 20:11:29.222601 2564 scope.go:117] "RemoveContainer" containerID="ce9be9961c6822d2072390ad2c872e1138f0e0476efa1e22a38b6e9d56018ff5" Jul 29 20:11:29.231927 ip-10-0-131-93 kubenswrapper[2564]: I0729 20:11:29.231912 2564 scope.go:117] "RemoveContainer" containerID="a692e3648b395e8a1a302d155e4363b08265b55b20ec068de691aa8167bbd2c2" Jul 29 20:11:29.240987 ip-10-0-131-93 kubenswrapper[2564]: I0729 20:11:29.240967 2564 scope.go:117] "RemoveContainer" containerID="ce9be9961c6822d2072390ad2c872e1138f0e0476efa1e22a38b6e9d56018ff5" Jul 29 20:11:29.241105 ip-10-0-131-93 kubenswrapper[2564]: I0729 20:11:29.241075 2564 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-ingress/e2e-ipp-cleanup-5d7141-openshift-default-6678d486d-gbdmq"] Jul 29 20:11:29.241235 ip-10-0-131-93 kubenswrapper[2564]: E0729 20:11:29.241217 2564 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"ce9be9961c6822d2072390ad2c872e1138f0e0476efa1e22a38b6e9d56018ff5\": container with ID starting with ce9be9961c6822d2072390ad2c872e1138f0e0476efa1e22a38b6e9d56018ff5 not found: ID does not exist" containerID="ce9be9961c6822d2072390ad2c872e1138f0e0476efa1e22a38b6e9d56018ff5" Jul 29 20:11:29.241300 ip-10-0-131-93 kubenswrapper[2564]: I0729 20:11:29.241248 2564 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"ce9be9961c6822d2072390ad2c872e1138f0e0476efa1e22a38b6e9d56018ff5"} err="failed to get container status \"ce9be9961c6822d2072390ad2c872e1138f0e0476efa1e22a38b6e9d56018ff5\": rpc error: code = NotFound desc = could not find container \"ce9be9961c6822d2072390ad2c872e1138f0e0476efa1e22a38b6e9d56018ff5\": container with ID starting with ce9be9961c6822d2072390ad2c872e1138f0e0476efa1e22a38b6e9d56018ff5 not found: ID does not exist" Jul 29 20:11:29.241300 ip-10-0-131-93 kubenswrapper[2564]: I0729 20:11:29.241275 2564 scope.go:117] "RemoveContainer" containerID="a692e3648b395e8a1a302d155e4363b08265b55b20ec068de691aa8167bbd2c2" Jul 29 20:11:29.241575 ip-10-0-131-93 kubenswrapper[2564]: E0729 20:11:29.241555 2564 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"a692e3648b395e8a1a302d155e4363b08265b55b20ec068de691aa8167bbd2c2\": container with ID starting with a692e3648b395e8a1a302d155e4363b08265b55b20ec068de691aa8167bbd2c2 not found: ID does not exist" containerID="a692e3648b395e8a1a302d155e4363b08265b55b20ec068de691aa8167bbd2c2" Jul 29 20:11:29.241638 ip-10-0-131-93 kubenswrapper[2564]: I0729 20:11:29.241582 2564 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"a692e3648b395e8a1a302d155e4363b08265b55b20ec068de691aa8167bbd2c2"} err="failed to get container status \"a692e3648b395e8a1a302d155e4363b08265b55b20ec068de691aa8167bbd2c2\": rpc error: code = NotFound desc = could not find container \"a692e3648b395e8a1a302d155e4363b08265b55b20ec068de691aa8167bbd2c2\": container with ID starting with a692e3648b395e8a1a302d155e4363b08265b55b20ec068de691aa8167bbd2c2 not found: ID does not exist" Jul 29 20:11:29.244843 ip-10-0-131-93 kubenswrapper[2564]: I0729 20:11:29.244825 2564 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["openshift-ingress/e2e-ipp-cleanup-5d7141-openshift-default-6678d486d-gbdmq"] Jul 29 20:11:30.783182 ip-10-0-131-93 kubenswrapper[2564]: I0729 20:11:30.783146 2564 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="b107bd8a-8f6f-4db0-8e9e-44a29271b1d5" path="/var/lib/kubelet/pods/b107bd8a-8f6f-4db0-8e9e-44a29271b1d5/volumes" Jul 29 20:11:42.314609 ip-10-0-131-93 kubenswrapper[2564]: I0729 20:11:42.314577 2564 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-444jx_faa8c546-c75b-4e10-93b4-661aa2b5f53e/konnectivity-agent/0.log" limit=1024 Jul 29 20:11:42.450998 ip-10-0-131-93 kubenswrapper[2564]: I0729 20:11:42.450967 2564 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-444jx_faa8c546-c75b-4e10-93b4-661aa2b5f53e/konnectivity-agent/0.log" limit=1024 Jul 29 20:11:42.582220 ip-10-0-131-93 kubenswrapper[2564]: I0729 20:11:42.568049 2564 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-444jx_faa8c546-c75b-4e10-93b4-661aa2b5f53e/konnectivity-agent/0.log" limit=1024 Jul 29 20:11:42.672446 ip-10-0-131-93 kubenswrapper[2564]: I0729 20:11:42.672411 2564 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-444jx_faa8c546-c75b-4e10-93b4-661aa2b5f53e/konnectivity-agent/0.log" limit=1024 Jul 29 20:11:42.772134 ip-10-0-131-93 kubenswrapper[2564]: I0729 20:11:42.772101 2564 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-444jx_faa8c546-c75b-4e10-93b4-661aa2b5f53e/konnectivity-agent/0.log" limit=1024 Jul 29 20:11:43.157072 ip-10-0-131-93 kubenswrapper[2564]: I0729 20:11:43.157031 2564 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-444jx_faa8c546-c75b-4e10-93b4-661aa2b5f53e/konnectivity-agent/0.log" limit=1024 Jul 29 20:11:48.192088 ip-10-0-131-93 kubenswrapper[2564]: I0729 20:11:48.192056 2564 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-444jx_faa8c546-c75b-4e10-93b4-661aa2b5f53e/konnectivity-agent/0.log" limit=1024 Jul 29 20:12:25.399171 ip-10-0-131-93 kubenswrapper[2564]: I0729 20:12:25.399120 2564 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-444jx_faa8c546-c75b-4e10-93b4-661aa2b5f53e/konnectivity-agent/0.log" limit=1024 Jul 29 20:12:26.162293 ip-10-0-131-93 kubenswrapper[2564]: I0729 20:12:26.162254 2564 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-444jx_faa8c546-c75b-4e10-93b4-661aa2b5f53e/konnectivity-agent/0.log" limit=1024 Jul 29 20:12:31.777143 ip-10-0-131-93 kubenswrapper[2564]: I0729 20:12:31.777110 2564 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-444jx_faa8c546-c75b-4e10-93b4-661aa2b5f53e/konnectivity-agent/0.log" limit=1024 Jul 29 20:12:31.850738 ip-10-0-131-93 kubenswrapper[2564]: I0729 20:12:31.850701 2564 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-444jx_faa8c546-c75b-4e10-93b4-661aa2b5f53e/konnectivity-agent/0.log" limit=1024 Jul 29 20:12:32.039045 ip-10-0-131-93 kubenswrapper[2564]: I0729 20:12:32.038965 2564 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-ingress/e2e-ipp-a-f0b78f-openshift-default-6d5b4668cb-lmfd2"] Jul 29 20:12:32.039218 ip-10-0-131-93 kubenswrapper[2564]: I0729 20:12:32.039193 2564 kuberuntime_container.go:873] "Killing container with a grace period" pod="openshift-ingress/e2e-ipp-a-f0b78f-openshift-default-6d5b4668cb-lmfd2" podUID="94a75046-fecd-413c-ad52-daf45c5732cd" containerName="istio-proxy" containerID="cri-o://fbb6df48fa33ee33dbbcbeabe66f88dd8deddc549cb48be59f5b6178789705bf" gracePeriod=30 Jul 29 20:12:32.055099 ip-10-0-131-93 kubenswrapper[2564]: I0729 20:12:32.055067 2564 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-444jx_faa8c546-c75b-4e10-93b4-661aa2b5f53e/konnectivity-agent/0.log" limit=1024 Jul 29 20:12:32.307198 ip-10-0-131-93 kubenswrapper[2564]: I0729 20:12:32.307108 2564 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-444jx_faa8c546-c75b-4e10-93b4-661aa2b5f53e/konnectivity-agent/0.log" limit=1024 Jul 29 20:12:36.215317 ip-10-0-131-93 kubenswrapper[2564]: I0729 20:12:36.215283 2564 patch_prober.go:28] interesting pod/e2e-ipp-a-f0b78f-openshift-default-6d5b4668cb-lmfd2 container/istio-proxy namespace/openshift-ingress: Readiness probe status=failure output="HTTP probe failed with statuscode: 503" start-of-body= Jul 29 20:12:36.215720 ip-10-0-131-93 kubenswrapper[2564]: I0729 20:12:36.215365 2564 prober.go:120] "Probe failed" probeType="Readiness" pod="openshift-ingress/e2e-ipp-a-f0b78f-openshift-default-6d5b4668cb-lmfd2" podUID="94a75046-fecd-413c-ad52-daf45c5732cd" containerName="istio-proxy" probeResult="failure" output="HTTP probe failed with statuscode: 503" Jul 29 20:12:37.295087 ip-10-0-131-93 kubenswrapper[2564]: I0729 20:12:37.295061 2564 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-ipp-a-f0b78f-openshift-default-6d5b4668cb-lmfd2" Jul 29 20:12:37.446999 ip-10-0-131-93 kubenswrapper[2564]: I0729 20:12:37.446968 2564 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/94a75046-fecd-413c-ad52-daf45c5732cd-istio-podinfo\") pod \"94a75046-fecd-413c-ad52-daf45c5732cd\" (UID: \"94a75046-fecd-413c-ad52-daf45c5732cd\") " Jul 29 20:12:37.447154 ip-10-0-131-93 kubenswrapper[2564]: I0729 20:12:37.447015 2564 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/94a75046-fecd-413c-ad52-daf45c5732cd-workload-certs\") pod \"94a75046-fecd-413c-ad52-daf45c5732cd\" (UID: \"94a75046-fecd-413c-ad52-daf45c5732cd\") " Jul 29 20:12:37.447154 ip-10-0-131-93 kubenswrapper[2564]: I0729 20:12:37.447058 2564 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/94a75046-fecd-413c-ad52-daf45c5732cd-istio-data\") pod \"94a75046-fecd-413c-ad52-daf45c5732cd\" (UID: \"94a75046-fecd-413c-ad52-daf45c5732cd\") " Jul 29 20:12:37.447154 ip-10-0-131-93 kubenswrapper[2564]: I0729 20:12:37.447085 2564 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/94a75046-fecd-413c-ad52-daf45c5732cd-workload-socket\") pod \"94a75046-fecd-413c-ad52-daf45c5732cd\" (UID: \"94a75046-fecd-413c-ad52-daf45c5732cd\") " Jul 29 20:12:37.447154 ip-10-0-131-93 kubenswrapper[2564]: I0729 20:12:37.447149 2564 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/94a75046-fecd-413c-ad52-daf45c5732cd-istio-token\") pod \"94a75046-fecd-413c-ad52-daf45c5732cd\" (UID: \"94a75046-fecd-413c-ad52-daf45c5732cd\") " Jul 29 20:12:37.447336 ip-10-0-131-93 kubenswrapper[2564]: I0729 20:12:37.447174 2564 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-b9sfg\" (UniqueName: \"kubernetes.io/projected/94a75046-fecd-413c-ad52-daf45c5732cd-kube-api-access-b9sfg\") pod \"94a75046-fecd-413c-ad52-daf45c5732cd\" (UID: \"94a75046-fecd-413c-ad52-daf45c5732cd\") " Jul 29 20:12:37.447336 ip-10-0-131-93 kubenswrapper[2564]: I0729 20:12:37.447195 2564 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/94a75046-fecd-413c-ad52-daf45c5732cd-istiod-ca-cert\") pod \"94a75046-fecd-413c-ad52-daf45c5732cd\" (UID: \"94a75046-fecd-413c-ad52-daf45c5732cd\") " Jul 29 20:12:37.447336 ip-10-0-131-93 kubenswrapper[2564]: I0729 20:12:37.447223 2564 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/94a75046-fecd-413c-ad52-daf45c5732cd-istio-envoy\") pod \"94a75046-fecd-413c-ad52-daf45c5732cd\" (UID: \"94a75046-fecd-413c-ad52-daf45c5732cd\") " Jul 29 20:12:37.447510 ip-10-0-131-93 kubenswrapper[2564]: I0729 20:12:37.447401 2564 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/94a75046-fecd-413c-ad52-daf45c5732cd-workload-certs" (OuterVolumeSpecName: "workload-certs") pod "94a75046-fecd-413c-ad52-daf45c5732cd" (UID: "94a75046-fecd-413c-ad52-daf45c5732cd"). InnerVolumeSpecName "workload-certs". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Jul 29 20:12:37.447574 ip-10-0-131-93 kubenswrapper[2564]: I0729 20:12:37.447503 2564 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/94a75046-fecd-413c-ad52-daf45c5732cd-credential-socket\") pod \"94a75046-fecd-413c-ad52-daf45c5732cd\" (UID: \"94a75046-fecd-413c-ad52-daf45c5732cd\") " Jul 29 20:12:37.447648 ip-10-0-131-93 kubenswrapper[2564]: I0729 20:12:37.447622 2564 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/94a75046-fecd-413c-ad52-daf45c5732cd-istiod-ca-cert" (OuterVolumeSpecName: "istiod-ca-cert") pod "94a75046-fecd-413c-ad52-daf45c5732cd" (UID: "94a75046-fecd-413c-ad52-daf45c5732cd"). InnerVolumeSpecName "istiod-ca-cert". PluginName "kubernetes.io/configmap", VolumeGIDValue "" Jul 29 20:12:37.447708 ip-10-0-131-93 kubenswrapper[2564]: I0729 20:12:37.447636 2564 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/94a75046-fecd-413c-ad52-daf45c5732cd-workload-socket" (OuterVolumeSpecName: "workload-socket") pod "94a75046-fecd-413c-ad52-daf45c5732cd" (UID: "94a75046-fecd-413c-ad52-daf45c5732cd"). InnerVolumeSpecName "workload-socket". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Jul 29 20:12:37.447914 ip-10-0-131-93 kubenswrapper[2564]: I0729 20:12:37.447883 2564 reconciler_common.go:299] "Volume detached for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/94a75046-fecd-413c-ad52-daf45c5732cd-istiod-ca-cert\") on node \"ip-10-0-131-93.ec2.internal\" DevicePath \"\"" Jul 29 20:12:37.448050 ip-10-0-131-93 kubenswrapper[2564]: I0729 20:12:37.447921 2564 reconciler_common.go:299] "Volume detached for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/94a75046-fecd-413c-ad52-daf45c5732cd-workload-certs\") on node \"ip-10-0-131-93.ec2.internal\" DevicePath \"\"" Jul 29 20:12:37.448050 ip-10-0-131-93 kubenswrapper[2564]: I0729 20:12:37.447940 2564 reconciler_common.go:299] "Volume detached for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/94a75046-fecd-413c-ad52-daf45c5732cd-workload-socket\") on node \"ip-10-0-131-93.ec2.internal\" DevicePath \"\"" Jul 29 20:12:37.448050 ip-10-0-131-93 kubenswrapper[2564]: I0729 20:12:37.447914 2564 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/94a75046-fecd-413c-ad52-daf45c5732cd-credential-socket" (OuterVolumeSpecName: "credential-socket") pod "94a75046-fecd-413c-ad52-daf45c5732cd" (UID: "94a75046-fecd-413c-ad52-daf45c5732cd"). InnerVolumeSpecName "credential-socket". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Jul 29 20:12:37.448050 ip-10-0-131-93 kubenswrapper[2564]: I0729 20:12:37.447992 2564 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/94a75046-fecd-413c-ad52-daf45c5732cd-istio-data" (OuterVolumeSpecName: "istio-data") pod "94a75046-fecd-413c-ad52-daf45c5732cd" (UID: "94a75046-fecd-413c-ad52-daf45c5732cd"). InnerVolumeSpecName "istio-data". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Jul 29 20:12:37.449262 ip-10-0-131-93 kubenswrapper[2564]: I0729 20:12:37.449238 2564 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/downward-api/94a75046-fecd-413c-ad52-daf45c5732cd-istio-podinfo" (OuterVolumeSpecName: "istio-podinfo") pod "94a75046-fecd-413c-ad52-daf45c5732cd" (UID: "94a75046-fecd-413c-ad52-daf45c5732cd"). InnerVolumeSpecName "istio-podinfo". PluginName "kubernetes.io/downward-api", VolumeGIDValue "" Jul 29 20:12:37.449389 ip-10-0-131-93 kubenswrapper[2564]: I0729 20:12:37.449289 2564 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/94a75046-fecd-413c-ad52-daf45c5732cd-istio-envoy" (OuterVolumeSpecName: "istio-envoy") pod "94a75046-fecd-413c-ad52-daf45c5732cd" (UID: "94a75046-fecd-413c-ad52-daf45c5732cd"). InnerVolumeSpecName "istio-envoy". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Jul 29 20:12:37.449462 ip-10-0-131-93 kubenswrapper[2564]: I0729 20:12:37.449438 2564 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/94a75046-fecd-413c-ad52-daf45c5732cd-istio-token" (OuterVolumeSpecName: "istio-token") pod "94a75046-fecd-413c-ad52-daf45c5732cd" (UID: "94a75046-fecd-413c-ad52-daf45c5732cd"). InnerVolumeSpecName "istio-token". PluginName "kubernetes.io/projected", VolumeGIDValue "" Jul 29 20:12:37.449573 ip-10-0-131-93 kubenswrapper[2564]: I0729 20:12:37.449555 2564 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/94a75046-fecd-413c-ad52-daf45c5732cd-kube-api-access-b9sfg" (OuterVolumeSpecName: "kube-api-access-b9sfg") pod "94a75046-fecd-413c-ad52-daf45c5732cd" (UID: "94a75046-fecd-413c-ad52-daf45c5732cd"). InnerVolumeSpecName "kube-api-access-b9sfg". PluginName "kubernetes.io/projected", VolumeGIDValue "" Jul 29 20:12:37.549293 ip-10-0-131-93 kubenswrapper[2564]: I0729 20:12:37.549251 2564 reconciler_common.go:299] "Volume detached for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/94a75046-fecd-413c-ad52-daf45c5732cd-istio-token\") on node \"ip-10-0-131-93.ec2.internal\" DevicePath \"\"" Jul 29 20:12:37.549293 ip-10-0-131-93 kubenswrapper[2564]: I0729 20:12:37.549291 2564 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-b9sfg\" (UniqueName: \"kubernetes.io/projected/94a75046-fecd-413c-ad52-daf45c5732cd-kube-api-access-b9sfg\") on node \"ip-10-0-131-93.ec2.internal\" DevicePath \"\"" Jul 29 20:12:37.549293 ip-10-0-131-93 kubenswrapper[2564]: I0729 20:12:37.549302 2564 reconciler_common.go:299] "Volume detached for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/94a75046-fecd-413c-ad52-daf45c5732cd-istio-envoy\") on node \"ip-10-0-131-93.ec2.internal\" DevicePath \"\"" Jul 29 20:12:37.549541 ip-10-0-131-93 kubenswrapper[2564]: I0729 20:12:37.549311 2564 reconciler_common.go:299] "Volume detached for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/94a75046-fecd-413c-ad52-daf45c5732cd-credential-socket\") on node \"ip-10-0-131-93.ec2.internal\" DevicePath \"\"" Jul 29 20:12:37.549541 ip-10-0-131-93 kubenswrapper[2564]: I0729 20:12:37.549319 2564 reconciler_common.go:299] "Volume detached for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/94a75046-fecd-413c-ad52-daf45c5732cd-istio-podinfo\") on node \"ip-10-0-131-93.ec2.internal\" DevicePath \"\"" Jul 29 20:12:37.549541 ip-10-0-131-93 kubenswrapper[2564]: I0729 20:12:37.549327 2564 reconciler_common.go:299] "Volume detached for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/94a75046-fecd-413c-ad52-daf45c5732cd-istio-data\") on node \"ip-10-0-131-93.ec2.internal\" DevicePath \"\"" Jul 29 20:12:37.580946 ip-10-0-131-93 kubenswrapper[2564]: I0729 20:12:37.580913 2564 generic.go:356] "Generic (PLEG): container finished" podID="94a75046-fecd-413c-ad52-daf45c5732cd" containerID="fbb6df48fa33ee33dbbcbeabe66f88dd8deddc549cb48be59f5b6178789705bf" exitCode=0 Jul 29 20:12:37.581097 ip-10-0-131-93 kubenswrapper[2564]: I0729 20:12:37.580971 2564 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-ipp-a-f0b78f-openshift-default-6d5b4668cb-lmfd2" Jul 29 20:12:37.581097 ip-10-0-131-93 kubenswrapper[2564]: I0729 20:12:37.581001 2564 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-ipp-a-f0b78f-openshift-default-6d5b4668cb-lmfd2" event={"ID":"94a75046-fecd-413c-ad52-daf45c5732cd","Type":"ContainerDied","Data":"fbb6df48fa33ee33dbbcbeabe66f88dd8deddc549cb48be59f5b6178789705bf"} Jul 29 20:12:37.581097 ip-10-0-131-93 kubenswrapper[2564]: I0729 20:12:37.581070 2564 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-ipp-a-f0b78f-openshift-default-6d5b4668cb-lmfd2" event={"ID":"94a75046-fecd-413c-ad52-daf45c5732cd","Type":"ContainerDied","Data":"f875df0c6c0e6e7f0a1b12fae7a99e5fa710c280d6befae522bb5b0d1da375c2"} Jul 29 20:12:37.581097 ip-10-0-131-93 kubenswrapper[2564]: I0729 20:12:37.581092 2564 scope.go:117] "RemoveContainer" containerID="fbb6df48fa33ee33dbbcbeabe66f88dd8deddc549cb48be59f5b6178789705bf" Jul 29 20:12:37.591026 ip-10-0-131-93 kubenswrapper[2564]: I0729 20:12:37.591008 2564 scope.go:117] "RemoveContainer" containerID="b76bf1acb7a868229a2c8e7da52658015fc10f09c4fb9ff14e66fefaa78c4efd" Jul 29 20:12:37.598998 ip-10-0-131-93 kubenswrapper[2564]: I0729 20:12:37.598983 2564 scope.go:117] "RemoveContainer" containerID="fbb6df48fa33ee33dbbcbeabe66f88dd8deddc549cb48be59f5b6178789705bf" Jul 29 20:12:37.599230 ip-10-0-131-93 kubenswrapper[2564]: E0729 20:12:37.599210 2564 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"fbb6df48fa33ee33dbbcbeabe66f88dd8deddc549cb48be59f5b6178789705bf\": container with ID starting with fbb6df48fa33ee33dbbcbeabe66f88dd8deddc549cb48be59f5b6178789705bf not found: ID does not exist" containerID="fbb6df48fa33ee33dbbcbeabe66f88dd8deddc549cb48be59f5b6178789705bf" Jul 29 20:12:37.599297 ip-10-0-131-93 kubenswrapper[2564]: I0729 20:12:37.599241 2564 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"fbb6df48fa33ee33dbbcbeabe66f88dd8deddc549cb48be59f5b6178789705bf"} err="failed to get container status \"fbb6df48fa33ee33dbbcbeabe66f88dd8deddc549cb48be59f5b6178789705bf\": rpc error: code = NotFound desc = could not find container \"fbb6df48fa33ee33dbbcbeabe66f88dd8deddc549cb48be59f5b6178789705bf\": container with ID starting with fbb6df48fa33ee33dbbcbeabe66f88dd8deddc549cb48be59f5b6178789705bf not found: ID does not exist" Jul 29 20:12:37.599297 ip-10-0-131-93 kubenswrapper[2564]: I0729 20:12:37.599262 2564 scope.go:117] "RemoveContainer" containerID="b76bf1acb7a868229a2c8e7da52658015fc10f09c4fb9ff14e66fefaa78c4efd" Jul 29 20:12:37.599513 ip-10-0-131-93 kubenswrapper[2564]: E0729 20:12:37.599498 2564 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"b76bf1acb7a868229a2c8e7da52658015fc10f09c4fb9ff14e66fefaa78c4efd\": container with ID starting with b76bf1acb7a868229a2c8e7da52658015fc10f09c4fb9ff14e66fefaa78c4efd not found: ID does not exist" containerID="b76bf1acb7a868229a2c8e7da52658015fc10f09c4fb9ff14e66fefaa78c4efd" Jul 29 20:12:37.599572 ip-10-0-131-93 kubenswrapper[2564]: I0729 20:12:37.599519 2564 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"b76bf1acb7a868229a2c8e7da52658015fc10f09c4fb9ff14e66fefaa78c4efd"} err="failed to get container status \"b76bf1acb7a868229a2c8e7da52658015fc10f09c4fb9ff14e66fefaa78c4efd\": rpc error: code = NotFound desc = could not find container \"b76bf1acb7a868229a2c8e7da52658015fc10f09c4fb9ff14e66fefaa78c4efd\": container with ID starting with b76bf1acb7a868229a2c8e7da52658015fc10f09c4fb9ff14e66fefaa78c4efd not found: ID does not exist" Jul 29 20:12:37.605194 ip-10-0-131-93 kubenswrapper[2564]: I0729 20:12:37.605171 2564 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-ingress/e2e-ipp-a-f0b78f-openshift-default-6d5b4668cb-lmfd2"] Jul 29 20:12:37.608617 ip-10-0-131-93 kubenswrapper[2564]: I0729 20:12:37.608597 2564 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["openshift-ingress/e2e-ipp-a-f0b78f-openshift-default-6d5b4668cb-lmfd2"] Jul 29 20:12:38.782861 ip-10-0-131-93 kubenswrapper[2564]: I0729 20:12:38.782830 2564 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="94a75046-fecd-413c-ad52-daf45c5732cd" path="/var/lib/kubelet/pods/94a75046-fecd-413c-ad52-daf45c5732cd/volumes" Jul 29 20:12:42.979887 ip-10-0-131-93 kubenswrapper[2564]: I0729 20:12:42.979841 2564 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-444jx_faa8c546-c75b-4e10-93b4-661aa2b5f53e/konnectivity-agent/0.log" limit=1024 Jul 29 20:12:43.023404 ip-10-0-131-93 kubenswrapper[2564]: I0729 20:12:43.023354 2564 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-ingress/payload-pre-processing-e2e-ipp-b-2d9c39-99cc99448-78rp6"] Jul 29 20:12:43.023617 ip-10-0-131-93 kubenswrapper[2564]: I0729 20:12:43.023585 2564 kuberuntime_container.go:873] "Killing container with a grace period" pod="openshift-ingress/payload-pre-processing-e2e-ipp-b-2d9c39-99cc99448-78rp6" podUID="252a57d2-9e17-4d68-83bb-fff34a75bcd9" containerName="payload-pre-processing" containerID="cri-o://3e4e696f42dc7fe126bc7cb502b57d5dbf6b5131286b94e4629e0e5f19da2c87" gracePeriod=30 Jul 29 20:12:43.052324 ip-10-0-131-93 kubenswrapper[2564]: I0729 20:12:43.052298 2564 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-444jx_faa8c546-c75b-4e10-93b4-661aa2b5f53e/konnectivity-agent/0.log" limit=1024 Jul 29 20:12:43.131031 ip-10-0-131-93 kubenswrapper[2564]: I0729 20:12:43.131005 2564 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-444jx_faa8c546-c75b-4e10-93b4-661aa2b5f53e/konnectivity-agent/0.log" limit=1024 Jul 29 20:12:43.225588 ip-10-0-131-93 kubenswrapper[2564]: I0729 20:12:43.220702 2564 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-444jx_faa8c546-c75b-4e10-93b4-661aa2b5f53e/konnectivity-agent/0.log" limit=1024 Jul 29 20:12:43.292865 ip-10-0-131-93 kubenswrapper[2564]: I0729 20:12:43.292841 2564 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-pre-processing-e2e-ipp-b-2d9c39-99cc99448-78rp6" Jul 29 20:12:43.396537 ip-10-0-131-93 kubenswrapper[2564]: I0729 20:12:43.396502 2564 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-drrcn\" (UniqueName: \"kubernetes.io/projected/252a57d2-9e17-4d68-83bb-fff34a75bcd9-kube-api-access-drrcn\") pod \"252a57d2-9e17-4d68-83bb-fff34a75bcd9\" (UID: \"252a57d2-9e17-4d68-83bb-fff34a75bcd9\") " Jul 29 20:12:43.396723 ip-10-0-131-93 kubenswrapper[2564]: I0729 20:12:43.396585 2564 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/252a57d2-9e17-4d68-83bb-fff34a75bcd9-plugins-config-volume\") pod \"252a57d2-9e17-4d68-83bb-fff34a75bcd9\" (UID: \"252a57d2-9e17-4d68-83bb-fff34a75bcd9\") " Jul 29 20:12:43.396938 ip-10-0-131-93 kubenswrapper[2564]: I0729 20:12:43.396915 2564 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/252a57d2-9e17-4d68-83bb-fff34a75bcd9-plugins-config-volume" (OuterVolumeSpecName: "plugins-config-volume") pod "252a57d2-9e17-4d68-83bb-fff34a75bcd9" (UID: "252a57d2-9e17-4d68-83bb-fff34a75bcd9"). InnerVolumeSpecName "plugins-config-volume". PluginName "kubernetes.io/configmap", VolumeGIDValue "" Jul 29 20:12:43.398669 ip-10-0-131-93 kubenswrapper[2564]: I0729 20:12:43.398639 2564 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/252a57d2-9e17-4d68-83bb-fff34a75bcd9-kube-api-access-drrcn" (OuterVolumeSpecName: "kube-api-access-drrcn") pod "252a57d2-9e17-4d68-83bb-fff34a75bcd9" (UID: "252a57d2-9e17-4d68-83bb-fff34a75bcd9"). InnerVolumeSpecName "kube-api-access-drrcn". PluginName "kubernetes.io/projected", VolumeGIDValue "" Jul 29 20:12:43.497597 ip-10-0-131-93 kubenswrapper[2564]: I0729 20:12:43.497565 2564 reconciler_common.go:299] "Volume detached for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/252a57d2-9e17-4d68-83bb-fff34a75bcd9-plugins-config-volume\") on node \"ip-10-0-131-93.ec2.internal\" DevicePath \"\"" Jul 29 20:12:43.497597 ip-10-0-131-93 kubenswrapper[2564]: I0729 20:12:43.497592 2564 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-drrcn\" (UniqueName: \"kubernetes.io/projected/252a57d2-9e17-4d68-83bb-fff34a75bcd9-kube-api-access-drrcn\") on node \"ip-10-0-131-93.ec2.internal\" DevicePath \"\"" Jul 29 20:12:43.610891 ip-10-0-131-93 kubenswrapper[2564]: I0729 20:12:43.610806 2564 generic.go:356] "Generic (PLEG): container finished" podID="252a57d2-9e17-4d68-83bb-fff34a75bcd9" containerID="3e4e696f42dc7fe126bc7cb502b57d5dbf6b5131286b94e4629e0e5f19da2c87" exitCode=0 Jul 29 20:12:43.610891 ip-10-0-131-93 kubenswrapper[2564]: I0729 20:12:43.610858 2564 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-pre-processing-e2e-ipp-b-2d9c39-99cc99448-78rp6" Jul 29 20:12:43.611108 ip-10-0-131-93 kubenswrapper[2564]: I0729 20:12:43.610892 2564 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-pre-processing-e2e-ipp-b-2d9c39-99cc99448-78rp6" event={"ID":"252a57d2-9e17-4d68-83bb-fff34a75bcd9","Type":"ContainerDied","Data":"3e4e696f42dc7fe126bc7cb502b57d5dbf6b5131286b94e4629e0e5f19da2c87"} Jul 29 20:12:43.611108 ip-10-0-131-93 kubenswrapper[2564]: I0729 20:12:43.610943 2564 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-pre-processing-e2e-ipp-b-2d9c39-99cc99448-78rp6" event={"ID":"252a57d2-9e17-4d68-83bb-fff34a75bcd9","Type":"ContainerDied","Data":"b8458dfd2b6f46c5a4fbc5b0162b1bd387d5d2189c078e333f1ad8c255091721"} Jul 29 20:12:43.611108 ip-10-0-131-93 kubenswrapper[2564]: I0729 20:12:43.610968 2564 scope.go:117] "RemoveContainer" containerID="3e4e696f42dc7fe126bc7cb502b57d5dbf6b5131286b94e4629e0e5f19da2c87" Jul 29 20:12:43.626004 ip-10-0-131-93 kubenswrapper[2564]: I0729 20:12:43.625980 2564 scope.go:117] "RemoveContainer" containerID="3e4e696f42dc7fe126bc7cb502b57d5dbf6b5131286b94e4629e0e5f19da2c87" Jul 29 20:12:43.626288 ip-10-0-131-93 kubenswrapper[2564]: E0729 20:12:43.626263 2564 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"3e4e696f42dc7fe126bc7cb502b57d5dbf6b5131286b94e4629e0e5f19da2c87\": container with ID starting with 3e4e696f42dc7fe126bc7cb502b57d5dbf6b5131286b94e4629e0e5f19da2c87 not found: ID does not exist" containerID="3e4e696f42dc7fe126bc7cb502b57d5dbf6b5131286b94e4629e0e5f19da2c87" Jul 29 20:12:43.626335 ip-10-0-131-93 kubenswrapper[2564]: I0729 20:12:43.626300 2564 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"3e4e696f42dc7fe126bc7cb502b57d5dbf6b5131286b94e4629e0e5f19da2c87"} err="failed to get container status \"3e4e696f42dc7fe126bc7cb502b57d5dbf6b5131286b94e4629e0e5f19da2c87\": rpc error: code = NotFound desc = could not find container \"3e4e696f42dc7fe126bc7cb502b57d5dbf6b5131286b94e4629e0e5f19da2c87\": container with ID starting with 3e4e696f42dc7fe126bc7cb502b57d5dbf6b5131286b94e4629e0e5f19da2c87 not found: ID does not exist" Jul 29 20:12:43.650647 ip-10-0-131-93 kubenswrapper[2564]: I0729 20:12:43.650624 2564 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-ingress/payload-pre-processing-e2e-ipp-b-2d9c39-99cc99448-78rp6"] Jul 29 20:12:43.661000 ip-10-0-131-93 kubenswrapper[2564]: I0729 20:12:43.660979 2564 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["openshift-ingress/payload-pre-processing-e2e-ipp-b-2d9c39-99cc99448-78rp6"] Jul 29 20:12:43.663632 ip-10-0-131-93 kubenswrapper[2564]: I0729 20:12:43.663613 2564 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-444jx_faa8c546-c75b-4e10-93b4-661aa2b5f53e/konnectivity-agent/0.log" limit=1024 Jul 29 20:12:44.782768 ip-10-0-131-93 kubenswrapper[2564]: I0729 20:12:44.782732 2564 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="252a57d2-9e17-4d68-83bb-fff34a75bcd9" path="/var/lib/kubelet/pods/252a57d2-9e17-4d68-83bb-fff34a75bcd9/volumes" Jul 29 20:12:49.597907 ip-10-0-131-93 kubenswrapper[2564]: I0729 20:12:49.597873 2564 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-444jx_faa8c546-c75b-4e10-93b4-661aa2b5f53e/konnectivity-agent/0.log" limit=1024 Jul 29 20:12:49.700708 ip-10-0-131-93 kubenswrapper[2564]: I0729 20:12:49.700676 2564 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-444jx_faa8c546-c75b-4e10-93b4-661aa2b5f53e/konnectivity-agent/0.log" limit=1024 Jul 29 20:12:49.870040 ip-10-0-131-93 kubenswrapper[2564]: I0729 20:12:49.869963 2564 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-444jx_faa8c546-c75b-4e10-93b4-661aa2b5f53e/konnectivity-agent/0.log" limit=1024 Jul 29 20:12:50.088826 ip-10-0-131-93 kubenswrapper[2564]: I0729 20:12:50.088795 2564 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-444jx_faa8c546-c75b-4e10-93b4-661aa2b5f53e/konnectivity-agent/0.log" limit=1024 Jul 29 20:13:00.995340 ip-10-0-131-93 kubenswrapper[2564]: I0729 20:13:00.995306 2564 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-444jx_faa8c546-c75b-4e10-93b4-661aa2b5f53e/konnectivity-agent/0.log" limit=1024 Jul 29 20:13:02.858070 ip-10-0-131-93 kubenswrapper[2564]: I0729 20:13:02.858037 2564 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-444jx_faa8c546-c75b-4e10-93b4-661aa2b5f53e/konnectivity-agent/0.log" limit=1024 Jul 29 20:13:07.808968 ip-10-0-131-93 kubenswrapper[2564]: I0729 20:13:07.808935 2564 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-444jx_faa8c546-c75b-4e10-93b4-661aa2b5f53e/konnectivity-agent/0.log" limit=1024 Jul 29 20:13:07.910087 ip-10-0-131-93 kubenswrapper[2564]: I0729 20:13:07.910058 2564 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-444jx_faa8c546-c75b-4e10-93b4-661aa2b5f53e/konnectivity-agent/0.log" limit=1024 Jul 29 20:13:08.036325 ip-10-0-131-93 kubenswrapper[2564]: I0729 20:13:08.036293 2564 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-ingress/e2e-shared-a-641e39-openshift-default-7898445948-2z4ct"] Jul 29 20:13:08.036639 ip-10-0-131-93 kubenswrapper[2564]: I0729 20:13:08.036603 2564 kuberuntime_container.go:873] "Killing container with a grace period" pod="openshift-ingress/e2e-shared-a-641e39-openshift-default-7898445948-2z4ct" podUID="c9bc01d1-d52b-4476-942e-68c5a3641d5f" containerName="istio-proxy" containerID="cri-o://56035649b95b5c07685ccb8f7e916ce8f650b2af416e214a04399a3350a20e78" gracePeriod=30 Jul 29 20:13:08.067038 ip-10-0-131-93 kubenswrapper[2564]: I0729 20:13:08.066966 2564 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-444jx_faa8c546-c75b-4e10-93b4-661aa2b5f53e/konnectivity-agent/0.log" limit=1024 Jul 29 20:13:08.286286 ip-10-0-131-93 kubenswrapper[2564]: I0729 20:13:08.286253 2564 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-444jx_faa8c546-c75b-4e10-93b4-661aa2b5f53e/konnectivity-agent/0.log" limit=1024 Jul 29 20:13:13.291562 ip-10-0-131-93 kubenswrapper[2564]: I0729 20:13:13.291538 2564 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-shared-a-641e39-openshift-default-7898445948-2z4ct" Jul 29 20:13:13.353692 ip-10-0-131-93 kubenswrapper[2564]: I0729 20:13:13.353617 2564 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/c9bc01d1-d52b-4476-942e-68c5a3641d5f-credential-socket\") pod \"c9bc01d1-d52b-4476-942e-68c5a3641d5f\" (UID: \"c9bc01d1-d52b-4476-942e-68c5a3641d5f\") " Jul 29 20:13:13.353692 ip-10-0-131-93 kubenswrapper[2564]: I0729 20:13:13.353653 2564 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/c9bc01d1-d52b-4476-942e-68c5a3641d5f-istio-data\") pod \"c9bc01d1-d52b-4476-942e-68c5a3641d5f\" (UID: \"c9bc01d1-d52b-4476-942e-68c5a3641d5f\") " Jul 29 20:13:13.353692 ip-10-0-131-93 kubenswrapper[2564]: I0729 20:13:13.353680 2564 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-zl4jb\" (UniqueName: \"kubernetes.io/projected/c9bc01d1-d52b-4476-942e-68c5a3641d5f-kube-api-access-zl4jb\") pod \"c9bc01d1-d52b-4476-942e-68c5a3641d5f\" (UID: \"c9bc01d1-d52b-4476-942e-68c5a3641d5f\") " Jul 29 20:13:13.353951 ip-10-0-131-93 kubenswrapper[2564]: I0729 20:13:13.353713 2564 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/c9bc01d1-d52b-4476-942e-68c5a3641d5f-istio-envoy\") pod \"c9bc01d1-d52b-4476-942e-68c5a3641d5f\" (UID: \"c9bc01d1-d52b-4476-942e-68c5a3641d5f\") " Jul 29 20:13:13.353951 ip-10-0-131-93 kubenswrapper[2564]: I0729 20:13:13.353740 2564 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/c9bc01d1-d52b-4476-942e-68c5a3641d5f-workload-socket\") pod \"c9bc01d1-d52b-4476-942e-68c5a3641d5f\" (UID: \"c9bc01d1-d52b-4476-942e-68c5a3641d5f\") " Jul 29 20:13:13.353951 ip-10-0-131-93 kubenswrapper[2564]: I0729 20:13:13.353782 2564 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/c9bc01d1-d52b-4476-942e-68c5a3641d5f-istio-token\") pod \"c9bc01d1-d52b-4476-942e-68c5a3641d5f\" (UID: \"c9bc01d1-d52b-4476-942e-68c5a3641d5f\") " Jul 29 20:13:13.353951 ip-10-0-131-93 kubenswrapper[2564]: I0729 20:13:13.353818 2564 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/c9bc01d1-d52b-4476-942e-68c5a3641d5f-workload-certs\") pod \"c9bc01d1-d52b-4476-942e-68c5a3641d5f\" (UID: \"c9bc01d1-d52b-4476-942e-68c5a3641d5f\") " Jul 29 20:13:13.353951 ip-10-0-131-93 kubenswrapper[2564]: I0729 20:13:13.353833 2564 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/c9bc01d1-d52b-4476-942e-68c5a3641d5f-istiod-ca-cert\") pod \"c9bc01d1-d52b-4476-942e-68c5a3641d5f\" (UID: \"c9bc01d1-d52b-4476-942e-68c5a3641d5f\") " Jul 29 20:13:13.353951 ip-10-0-131-93 kubenswrapper[2564]: I0729 20:13:13.353902 2564 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/c9bc01d1-d52b-4476-942e-68c5a3641d5f-istio-podinfo\") pod \"c9bc01d1-d52b-4476-942e-68c5a3641d5f\" (UID: \"c9bc01d1-d52b-4476-942e-68c5a3641d5f\") " Jul 29 20:13:13.354256 ip-10-0-131-93 kubenswrapper[2564]: I0729 20:13:13.354092 2564 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/c9bc01d1-d52b-4476-942e-68c5a3641d5f-workload-socket" (OuterVolumeSpecName: "workload-socket") pod "c9bc01d1-d52b-4476-942e-68c5a3641d5f" (UID: "c9bc01d1-d52b-4476-942e-68c5a3641d5f"). InnerVolumeSpecName "workload-socket". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Jul 29 20:13:13.354256 ip-10-0-131-93 kubenswrapper[2564]: I0729 20:13:13.354128 2564 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/c9bc01d1-d52b-4476-942e-68c5a3641d5f-workload-certs" (OuterVolumeSpecName: "workload-certs") pod "c9bc01d1-d52b-4476-942e-68c5a3641d5f" (UID: "c9bc01d1-d52b-4476-942e-68c5a3641d5f"). InnerVolumeSpecName "workload-certs". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Jul 29 20:13:13.354256 ip-10-0-131-93 kubenswrapper[2564]: I0729 20:13:13.354192 2564 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/c9bc01d1-d52b-4476-942e-68c5a3641d5f-credential-socket" (OuterVolumeSpecName: "credential-socket") pod "c9bc01d1-d52b-4476-942e-68c5a3641d5f" (UID: "c9bc01d1-d52b-4476-942e-68c5a3641d5f"). InnerVolumeSpecName "credential-socket". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Jul 29 20:13:13.354446 ip-10-0-131-93 kubenswrapper[2564]: I0729 20:13:13.354423 2564 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/c9bc01d1-d52b-4476-942e-68c5a3641d5f-istiod-ca-cert" (OuterVolumeSpecName: "istiod-ca-cert") pod "c9bc01d1-d52b-4476-942e-68c5a3641d5f" (UID: "c9bc01d1-d52b-4476-942e-68c5a3641d5f"). InnerVolumeSpecName "istiod-ca-cert". PluginName "kubernetes.io/configmap", VolumeGIDValue "" Jul 29 20:13:13.354639 ip-10-0-131-93 kubenswrapper[2564]: I0729 20:13:13.354610 2564 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/c9bc01d1-d52b-4476-942e-68c5a3641d5f-istio-data" (OuterVolumeSpecName: "istio-data") pod "c9bc01d1-d52b-4476-942e-68c5a3641d5f" (UID: "c9bc01d1-d52b-4476-942e-68c5a3641d5f"). InnerVolumeSpecName "istio-data". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Jul 29 20:13:13.356066 ip-10-0-131-93 kubenswrapper[2564]: I0729 20:13:13.356033 2564 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/c9bc01d1-d52b-4476-942e-68c5a3641d5f-istio-envoy" (OuterVolumeSpecName: "istio-envoy") pod "c9bc01d1-d52b-4476-942e-68c5a3641d5f" (UID: "c9bc01d1-d52b-4476-942e-68c5a3641d5f"). InnerVolumeSpecName "istio-envoy". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Jul 29 20:13:13.356066 ip-10-0-131-93 kubenswrapper[2564]: I0729 20:13:13.356050 2564 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/c9bc01d1-d52b-4476-942e-68c5a3641d5f-istio-token" (OuterVolumeSpecName: "istio-token") pod "c9bc01d1-d52b-4476-942e-68c5a3641d5f" (UID: "c9bc01d1-d52b-4476-942e-68c5a3641d5f"). InnerVolumeSpecName "istio-token". PluginName "kubernetes.io/projected", VolumeGIDValue "" Jul 29 20:13:13.356224 ip-10-0-131-93 kubenswrapper[2564]: I0729 20:13:13.356109 2564 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/c9bc01d1-d52b-4476-942e-68c5a3641d5f-kube-api-access-zl4jb" (OuterVolumeSpecName: "kube-api-access-zl4jb") pod "c9bc01d1-d52b-4476-942e-68c5a3641d5f" (UID: "c9bc01d1-d52b-4476-942e-68c5a3641d5f"). InnerVolumeSpecName "kube-api-access-zl4jb". PluginName "kubernetes.io/projected", VolumeGIDValue "" Jul 29 20:13:13.356224 ip-10-0-131-93 kubenswrapper[2564]: I0729 20:13:13.356113 2564 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/downward-api/c9bc01d1-d52b-4476-942e-68c5a3641d5f-istio-podinfo" (OuterVolumeSpecName: "istio-podinfo") pod "c9bc01d1-d52b-4476-942e-68c5a3641d5f" (UID: "c9bc01d1-d52b-4476-942e-68c5a3641d5f"). InnerVolumeSpecName "istio-podinfo". PluginName "kubernetes.io/downward-api", VolumeGIDValue "" Jul 29 20:13:13.455500 ip-10-0-131-93 kubenswrapper[2564]: I0729 20:13:13.455458 2564 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-zl4jb\" (UniqueName: \"kubernetes.io/projected/c9bc01d1-d52b-4476-942e-68c5a3641d5f-kube-api-access-zl4jb\") on node \"ip-10-0-131-93.ec2.internal\" DevicePath \"\"" Jul 29 20:13:13.455500 ip-10-0-131-93 kubenswrapper[2564]: I0729 20:13:13.455492 2564 reconciler_common.go:299] "Volume detached for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/c9bc01d1-d52b-4476-942e-68c5a3641d5f-istio-envoy\") on node \"ip-10-0-131-93.ec2.internal\" DevicePath \"\"" Jul 29 20:13:13.455500 ip-10-0-131-93 kubenswrapper[2564]: I0729 20:13:13.455505 2564 reconciler_common.go:299] "Volume detached for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/c9bc01d1-d52b-4476-942e-68c5a3641d5f-workload-socket\") on node \"ip-10-0-131-93.ec2.internal\" DevicePath \"\"" Jul 29 20:13:13.455749 ip-10-0-131-93 kubenswrapper[2564]: I0729 20:13:13.455517 2564 reconciler_common.go:299] "Volume detached for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/c9bc01d1-d52b-4476-942e-68c5a3641d5f-istio-token\") on node \"ip-10-0-131-93.ec2.internal\" DevicePath \"\"" Jul 29 20:13:13.455749 ip-10-0-131-93 kubenswrapper[2564]: I0729 20:13:13.455528 2564 reconciler_common.go:299] "Volume detached for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/c9bc01d1-d52b-4476-942e-68c5a3641d5f-workload-certs\") on node \"ip-10-0-131-93.ec2.internal\" DevicePath \"\"" Jul 29 20:13:13.455749 ip-10-0-131-93 kubenswrapper[2564]: I0729 20:13:13.455539 2564 reconciler_common.go:299] "Volume detached for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/c9bc01d1-d52b-4476-942e-68c5a3641d5f-istiod-ca-cert\") on node \"ip-10-0-131-93.ec2.internal\" DevicePath \"\"" Jul 29 20:13:13.455749 ip-10-0-131-93 kubenswrapper[2564]: I0729 20:13:13.455549 2564 reconciler_common.go:299] "Volume detached for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/c9bc01d1-d52b-4476-942e-68c5a3641d5f-istio-podinfo\") on node \"ip-10-0-131-93.ec2.internal\" DevicePath \"\"" Jul 29 20:13:13.455749 ip-10-0-131-93 kubenswrapper[2564]: I0729 20:13:13.455560 2564 reconciler_common.go:299] "Volume detached for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/c9bc01d1-d52b-4476-942e-68c5a3641d5f-credential-socket\") on node \"ip-10-0-131-93.ec2.internal\" DevicePath \"\"" Jul 29 20:13:13.455749 ip-10-0-131-93 kubenswrapper[2564]: I0729 20:13:13.455570 2564 reconciler_common.go:299] "Volume detached for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/c9bc01d1-d52b-4476-942e-68c5a3641d5f-istio-data\") on node \"ip-10-0-131-93.ec2.internal\" DevicePath \"\"" Jul 29 20:13:13.758940 ip-10-0-131-93 kubenswrapper[2564]: I0729 20:13:13.758903 2564 generic.go:356] "Generic (PLEG): container finished" podID="c9bc01d1-d52b-4476-942e-68c5a3641d5f" containerID="56035649b95b5c07685ccb8f7e916ce8f650b2af416e214a04399a3350a20e78" exitCode=0 Jul 29 20:13:13.759116 ip-10-0-131-93 kubenswrapper[2564]: I0729 20:13:13.758955 2564 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-shared-a-641e39-openshift-default-7898445948-2z4ct" Jul 29 20:13:13.759116 ip-10-0-131-93 kubenswrapper[2564]: I0729 20:13:13.758974 2564 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-shared-a-641e39-openshift-default-7898445948-2z4ct" event={"ID":"c9bc01d1-d52b-4476-942e-68c5a3641d5f","Type":"ContainerDied","Data":"56035649b95b5c07685ccb8f7e916ce8f650b2af416e214a04399a3350a20e78"} Jul 29 20:13:13.759116 ip-10-0-131-93 kubenswrapper[2564]: I0729 20:13:13.759029 2564 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-shared-a-641e39-openshift-default-7898445948-2z4ct" event={"ID":"c9bc01d1-d52b-4476-942e-68c5a3641d5f","Type":"ContainerDied","Data":"36f793126db2b7b471f1c907435363a76c52ab2cc3b88b8fafbbb4d58f93acbe"} Jul 29 20:13:13.759116 ip-10-0-131-93 kubenswrapper[2564]: I0729 20:13:13.759061 2564 scope.go:117] "RemoveContainer" containerID="56035649b95b5c07685ccb8f7e916ce8f650b2af416e214a04399a3350a20e78" Jul 29 20:13:13.768227 ip-10-0-131-93 kubenswrapper[2564]: I0729 20:13:13.768209 2564 scope.go:117] "RemoveContainer" containerID="902c38b67ac57d88a03146f71d38d5110d4192d6f459be20582131647463183b" Jul 29 20:13:13.775746 ip-10-0-131-93 kubenswrapper[2564]: I0729 20:13:13.775729 2564 scope.go:117] "RemoveContainer" containerID="56035649b95b5c07685ccb8f7e916ce8f650b2af416e214a04399a3350a20e78" Jul 29 20:13:13.775994 ip-10-0-131-93 kubenswrapper[2564]: E0729 20:13:13.775972 2564 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"56035649b95b5c07685ccb8f7e916ce8f650b2af416e214a04399a3350a20e78\": container with ID starting with 56035649b95b5c07685ccb8f7e916ce8f650b2af416e214a04399a3350a20e78 not found: ID does not exist" containerID="56035649b95b5c07685ccb8f7e916ce8f650b2af416e214a04399a3350a20e78" Jul 29 20:13:13.776041 ip-10-0-131-93 kubenswrapper[2564]: I0729 20:13:13.776002 2564 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"56035649b95b5c07685ccb8f7e916ce8f650b2af416e214a04399a3350a20e78"} err="failed to get container status \"56035649b95b5c07685ccb8f7e916ce8f650b2af416e214a04399a3350a20e78\": rpc error: code = NotFound desc = could not find container \"56035649b95b5c07685ccb8f7e916ce8f650b2af416e214a04399a3350a20e78\": container with ID starting with 56035649b95b5c07685ccb8f7e916ce8f650b2af416e214a04399a3350a20e78 not found: ID does not exist" Jul 29 20:13:13.776041 ip-10-0-131-93 kubenswrapper[2564]: I0729 20:13:13.776019 2564 scope.go:117] "RemoveContainer" containerID="902c38b67ac57d88a03146f71d38d5110d4192d6f459be20582131647463183b" Jul 29 20:13:13.776250 ip-10-0-131-93 kubenswrapper[2564]: E0729 20:13:13.776228 2564 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"902c38b67ac57d88a03146f71d38d5110d4192d6f459be20582131647463183b\": container with ID starting with 902c38b67ac57d88a03146f71d38d5110d4192d6f459be20582131647463183b not found: ID does not exist" containerID="902c38b67ac57d88a03146f71d38d5110d4192d6f459be20582131647463183b" Jul 29 20:13:13.776326 ip-10-0-131-93 kubenswrapper[2564]: I0729 20:13:13.776250 2564 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"902c38b67ac57d88a03146f71d38d5110d4192d6f459be20582131647463183b"} err="failed to get container status \"902c38b67ac57d88a03146f71d38d5110d4192d6f459be20582131647463183b\": rpc error: code = NotFound desc = could not find container \"902c38b67ac57d88a03146f71d38d5110d4192d6f459be20582131647463183b\": container with ID starting with 902c38b67ac57d88a03146f71d38d5110d4192d6f459be20582131647463183b not found: ID does not exist" Jul 29 20:13:13.783519 ip-10-0-131-93 kubenswrapper[2564]: I0729 20:13:13.783489 2564 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-ingress/e2e-shared-a-641e39-openshift-default-7898445948-2z4ct"] Jul 29 20:13:13.788295 ip-10-0-131-93 kubenswrapper[2564]: I0729 20:13:13.788274 2564 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["openshift-ingress/e2e-shared-a-641e39-openshift-default-7898445948-2z4ct"] Jul 29 20:13:14.782633 ip-10-0-131-93 kubenswrapper[2564]: I0729 20:13:14.782603 2564 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="c9bc01d1-d52b-4476-942e-68c5a3641d5f" path="/var/lib/kubelet/pods/c9bc01d1-d52b-4476-942e-68c5a3641d5f/volumes" Jul 29 20:13:19.058634 ip-10-0-131-93 kubenswrapper[2564]: I0729 20:13:19.058599 2564 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-444jx_faa8c546-c75b-4e10-93b4-661aa2b5f53e/konnectivity-agent/0.log" limit=1024 Jul 29 20:13:19.158637 ip-10-0-131-93 kubenswrapper[2564]: I0729 20:13:19.158585 2564 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-444jx_faa8c546-c75b-4e10-93b4-661aa2b5f53e/konnectivity-agent/0.log" limit=1024 Jul 29 20:13:19.257931 ip-10-0-131-93 kubenswrapper[2564]: I0729 20:13:19.257894 2564 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-444jx_faa8c546-c75b-4e10-93b4-661aa2b5f53e/konnectivity-agent/0.log" limit=1024 Jul 29 20:13:19.356707 ip-10-0-131-93 kubenswrapper[2564]: I0729 20:13:19.356614 2564 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-444jx_faa8c546-c75b-4e10-93b4-661aa2b5f53e/konnectivity-agent/0.log" limit=1024 Jul 29 20:13:19.624180 ip-10-0-131-93 kubenswrapper[2564]: I0729 20:13:19.624100 2564 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-444jx_faa8c546-c75b-4e10-93b4-661aa2b5f53e/konnectivity-agent/0.log" limit=1024 Jul 29 20:13:25.920301 ip-10-0-131-93 kubenswrapper[2564]: I0729 20:13:25.920265 2564 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-444jx_faa8c546-c75b-4e10-93b4-661aa2b5f53e/konnectivity-agent/0.log" limit=1024 Jul 29 20:13:26.014535 ip-10-0-131-93 kubenswrapper[2564]: I0729 20:13:26.014502 2564 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-444jx_faa8c546-c75b-4e10-93b4-661aa2b5f53e/konnectivity-agent/0.log" limit=1024 Jul 29 20:13:26.159090 ip-10-0-131-93 kubenswrapper[2564]: I0729 20:13:26.159059 2564 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-ingress/e2e-shared-b-4c787c-openshift-default-66676df75b-f49kx"] Jul 29 20:13:26.159303 ip-10-0-131-93 kubenswrapper[2564]: I0729 20:13:26.159282 2564 kuberuntime_container.go:873] "Killing container with a grace period" pod="openshift-ingress/e2e-shared-b-4c787c-openshift-default-66676df75b-f49kx" podUID="15f29128-6b6b-4023-b057-184a58fe96cc" containerName="istio-proxy" containerID="cri-o://6777daae1396096745b9593744e33d0f5ceb2e44a014d3532e60a488ab259e97" gracePeriod=30 Jul 29 20:13:26.183816 ip-10-0-131-93 kubenswrapper[2564]: I0729 20:13:26.183745 2564 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-444jx_faa8c546-c75b-4e10-93b4-661aa2b5f53e/konnectivity-agent/0.log" limit=1024 Jul 29 20:13:26.416356 ip-10-0-131-93 kubenswrapper[2564]: I0729 20:13:26.416324 2564 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-444jx_faa8c546-c75b-4e10-93b4-661aa2b5f53e/konnectivity-agent/0.log" limit=1024 Jul 29 20:13:31.402527 ip-10-0-131-93 kubenswrapper[2564]: I0729 20:13:31.402505 2564 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-shared-b-4c787c-openshift-default-66676df75b-f49kx" Jul 29 20:13:31.510594 ip-10-0-131-93 kubenswrapper[2564]: I0729 20:13:31.510560 2564 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/15f29128-6b6b-4023-b057-184a58fe96cc-istio-data\") pod \"15f29128-6b6b-4023-b057-184a58fe96cc\" (UID: \"15f29128-6b6b-4023-b057-184a58fe96cc\") " Jul 29 20:13:31.510790 ip-10-0-131-93 kubenswrapper[2564]: I0729 20:13:31.510603 2564 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-tsrpp\" (UniqueName: \"kubernetes.io/projected/15f29128-6b6b-4023-b057-184a58fe96cc-kube-api-access-tsrpp\") pod \"15f29128-6b6b-4023-b057-184a58fe96cc\" (UID: \"15f29128-6b6b-4023-b057-184a58fe96cc\") " Jul 29 20:13:31.510790 ip-10-0-131-93 kubenswrapper[2564]: I0729 20:13:31.510644 2564 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/15f29128-6b6b-4023-b057-184a58fe96cc-credential-socket\") pod \"15f29128-6b6b-4023-b057-184a58fe96cc\" (UID: \"15f29128-6b6b-4023-b057-184a58fe96cc\") " Jul 29 20:13:31.510790 ip-10-0-131-93 kubenswrapper[2564]: I0729 20:13:31.510685 2564 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/15f29128-6b6b-4023-b057-184a58fe96cc-istiod-ca-cert\") pod \"15f29128-6b6b-4023-b057-184a58fe96cc\" (UID: \"15f29128-6b6b-4023-b057-184a58fe96cc\") " Jul 29 20:13:31.510965 ip-10-0-131-93 kubenswrapper[2564]: I0729 20:13:31.510897 2564 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/15f29128-6b6b-4023-b057-184a58fe96cc-workload-certs\") pod \"15f29128-6b6b-4023-b057-184a58fe96cc\" (UID: \"15f29128-6b6b-4023-b057-184a58fe96cc\") " Jul 29 20:13:31.510965 ip-10-0-131-93 kubenswrapper[2564]: I0729 20:13:31.510949 2564 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/15f29128-6b6b-4023-b057-184a58fe96cc-istio-token\") pod \"15f29128-6b6b-4023-b057-184a58fe96cc\" (UID: \"15f29128-6b6b-4023-b057-184a58fe96cc\") " Jul 29 20:13:31.511075 ip-10-0-131-93 kubenswrapper[2564]: I0729 20:13:31.510974 2564 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/15f29128-6b6b-4023-b057-184a58fe96cc-istio-envoy\") pod \"15f29128-6b6b-4023-b057-184a58fe96cc\" (UID: \"15f29128-6b6b-4023-b057-184a58fe96cc\") " Jul 29 20:13:31.511075 ip-10-0-131-93 kubenswrapper[2564]: I0729 20:13:31.510996 2564 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/15f29128-6b6b-4023-b057-184a58fe96cc-credential-socket" (OuterVolumeSpecName: "credential-socket") pod "15f29128-6b6b-4023-b057-184a58fe96cc" (UID: "15f29128-6b6b-4023-b057-184a58fe96cc"). InnerVolumeSpecName "credential-socket". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Jul 29 20:13:31.511075 ip-10-0-131-93 kubenswrapper[2564]: I0729 20:13:31.511017 2564 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/15f29128-6b6b-4023-b057-184a58fe96cc-istio-podinfo\") pod \"15f29128-6b6b-4023-b057-184a58fe96cc\" (UID: \"15f29128-6b6b-4023-b057-184a58fe96cc\") " Jul 29 20:13:31.511075 ip-10-0-131-93 kubenswrapper[2564]: I0729 20:13:31.511059 2564 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/15f29128-6b6b-4023-b057-184a58fe96cc-istiod-ca-cert" (OuterVolumeSpecName: "istiod-ca-cert") pod "15f29128-6b6b-4023-b057-184a58fe96cc" (UID: "15f29128-6b6b-4023-b057-184a58fe96cc"). InnerVolumeSpecName "istiod-ca-cert". PluginName "kubernetes.io/configmap", VolumeGIDValue "" Jul 29 20:13:31.511282 ip-10-0-131-93 kubenswrapper[2564]: I0729 20:13:31.511076 2564 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/15f29128-6b6b-4023-b057-184a58fe96cc-workload-socket\") pod \"15f29128-6b6b-4023-b057-184a58fe96cc\" (UID: \"15f29128-6b6b-4023-b057-184a58fe96cc\") " Jul 29 20:13:31.511282 ip-10-0-131-93 kubenswrapper[2564]: I0729 20:13:31.511152 2564 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/15f29128-6b6b-4023-b057-184a58fe96cc-workload-certs" (OuterVolumeSpecName: "workload-certs") pod "15f29128-6b6b-4023-b057-184a58fe96cc" (UID: "15f29128-6b6b-4023-b057-184a58fe96cc"). InnerVolumeSpecName "workload-certs". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Jul 29 20:13:31.511431 ip-10-0-131-93 kubenswrapper[2564]: I0729 20:13:31.511349 2564 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/15f29128-6b6b-4023-b057-184a58fe96cc-workload-socket" (OuterVolumeSpecName: "workload-socket") pod "15f29128-6b6b-4023-b057-184a58fe96cc" (UID: "15f29128-6b6b-4023-b057-184a58fe96cc"). InnerVolumeSpecName "workload-socket". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Jul 29 20:13:31.511497 ip-10-0-131-93 kubenswrapper[2564]: I0729 20:13:31.511430 2564 reconciler_common.go:299] "Volume detached for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/15f29128-6b6b-4023-b057-184a58fe96cc-credential-socket\") on node \"ip-10-0-131-93.ec2.internal\" DevicePath \"\"" Jul 29 20:13:31.511497 ip-10-0-131-93 kubenswrapper[2564]: I0729 20:13:31.511455 2564 reconciler_common.go:299] "Volume detached for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/15f29128-6b6b-4023-b057-184a58fe96cc-istiod-ca-cert\") on node \"ip-10-0-131-93.ec2.internal\" DevicePath \"\"" Jul 29 20:13:31.511497 ip-10-0-131-93 kubenswrapper[2564]: I0729 20:13:31.511460 2564 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/15f29128-6b6b-4023-b057-184a58fe96cc-istio-data" (OuterVolumeSpecName: "istio-data") pod "15f29128-6b6b-4023-b057-184a58fe96cc" (UID: "15f29128-6b6b-4023-b057-184a58fe96cc"). InnerVolumeSpecName "istio-data". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Jul 29 20:13:31.511497 ip-10-0-131-93 kubenswrapper[2564]: I0729 20:13:31.511470 2564 reconciler_common.go:299] "Volume detached for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/15f29128-6b6b-4023-b057-184a58fe96cc-workload-certs\") on node \"ip-10-0-131-93.ec2.internal\" DevicePath \"\"" Jul 29 20:13:31.513080 ip-10-0-131-93 kubenswrapper[2564]: I0729 20:13:31.513056 2564 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/15f29128-6b6b-4023-b057-184a58fe96cc-kube-api-access-tsrpp" (OuterVolumeSpecName: "kube-api-access-tsrpp") pod "15f29128-6b6b-4023-b057-184a58fe96cc" (UID: "15f29128-6b6b-4023-b057-184a58fe96cc"). InnerVolumeSpecName "kube-api-access-tsrpp". PluginName "kubernetes.io/projected", VolumeGIDValue "" Jul 29 20:13:31.513181 ip-10-0-131-93 kubenswrapper[2564]: I0729 20:13:31.513113 2564 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/15f29128-6b6b-4023-b057-184a58fe96cc-istio-envoy" (OuterVolumeSpecName: "istio-envoy") pod "15f29128-6b6b-4023-b057-184a58fe96cc" (UID: "15f29128-6b6b-4023-b057-184a58fe96cc"). InnerVolumeSpecName "istio-envoy". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Jul 29 20:13:31.513181 ip-10-0-131-93 kubenswrapper[2564]: I0729 20:13:31.513121 2564 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/downward-api/15f29128-6b6b-4023-b057-184a58fe96cc-istio-podinfo" (OuterVolumeSpecName: "istio-podinfo") pod "15f29128-6b6b-4023-b057-184a58fe96cc" (UID: "15f29128-6b6b-4023-b057-184a58fe96cc"). InnerVolumeSpecName "istio-podinfo". PluginName "kubernetes.io/downward-api", VolumeGIDValue "" Jul 29 20:13:31.513181 ip-10-0-131-93 kubenswrapper[2564]: I0729 20:13:31.513133 2564 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/15f29128-6b6b-4023-b057-184a58fe96cc-istio-token" (OuterVolumeSpecName: "istio-token") pod "15f29128-6b6b-4023-b057-184a58fe96cc" (UID: "15f29128-6b6b-4023-b057-184a58fe96cc"). InnerVolumeSpecName "istio-token". PluginName "kubernetes.io/projected", VolumeGIDValue "" Jul 29 20:13:31.611925 ip-10-0-131-93 kubenswrapper[2564]: I0729 20:13:31.611892 2564 reconciler_common.go:299] "Volume detached for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/15f29128-6b6b-4023-b057-184a58fe96cc-istio-token\") on node \"ip-10-0-131-93.ec2.internal\" DevicePath \"\"" Jul 29 20:13:31.611925 ip-10-0-131-93 kubenswrapper[2564]: I0729 20:13:31.611919 2564 reconciler_common.go:299] "Volume detached for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/15f29128-6b6b-4023-b057-184a58fe96cc-istio-envoy\") on node \"ip-10-0-131-93.ec2.internal\" DevicePath \"\"" Jul 29 20:13:31.611925 ip-10-0-131-93 kubenswrapper[2564]: I0729 20:13:31.611928 2564 reconciler_common.go:299] "Volume detached for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/15f29128-6b6b-4023-b057-184a58fe96cc-istio-podinfo\") on node \"ip-10-0-131-93.ec2.internal\" DevicePath \"\"" Jul 29 20:13:31.611925 ip-10-0-131-93 kubenswrapper[2564]: I0729 20:13:31.611936 2564 reconciler_common.go:299] "Volume detached for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/15f29128-6b6b-4023-b057-184a58fe96cc-workload-socket\") on node \"ip-10-0-131-93.ec2.internal\" DevicePath \"\"" Jul 29 20:13:31.612193 ip-10-0-131-93 kubenswrapper[2564]: I0729 20:13:31.611944 2564 reconciler_common.go:299] "Volume detached for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/15f29128-6b6b-4023-b057-184a58fe96cc-istio-data\") on node \"ip-10-0-131-93.ec2.internal\" DevicePath \"\"" Jul 29 20:13:31.612193 ip-10-0-131-93 kubenswrapper[2564]: I0729 20:13:31.611954 2564 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-tsrpp\" (UniqueName: \"kubernetes.io/projected/15f29128-6b6b-4023-b057-184a58fe96cc-kube-api-access-tsrpp\") on node \"ip-10-0-131-93.ec2.internal\" DevicePath \"\"" Jul 29 20:13:31.847069 ip-10-0-131-93 kubenswrapper[2564]: I0729 20:13:31.846986 2564 generic.go:356] "Generic (PLEG): container finished" podID="15f29128-6b6b-4023-b057-184a58fe96cc" containerID="6777daae1396096745b9593744e33d0f5ceb2e44a014d3532e60a488ab259e97" exitCode=0 Jul 29 20:13:31.847069 ip-10-0-131-93 kubenswrapper[2564]: I0729 20:13:31.847032 2564 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-shared-b-4c787c-openshift-default-66676df75b-f49kx" Jul 29 20:13:31.847069 ip-10-0-131-93 kubenswrapper[2564]: I0729 20:13:31.847062 2564 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-shared-b-4c787c-openshift-default-66676df75b-f49kx" event={"ID":"15f29128-6b6b-4023-b057-184a58fe96cc","Type":"ContainerDied","Data":"6777daae1396096745b9593744e33d0f5ceb2e44a014d3532e60a488ab259e97"} Jul 29 20:13:31.847314 ip-10-0-131-93 kubenswrapper[2564]: I0729 20:13:31.847107 2564 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-shared-b-4c787c-openshift-default-66676df75b-f49kx" event={"ID":"15f29128-6b6b-4023-b057-184a58fe96cc","Type":"ContainerDied","Data":"413990922ed5586d8e2b3b366f5e1b45c857d6ba5e423a264023df3215f54f37"} Jul 29 20:13:31.847314 ip-10-0-131-93 kubenswrapper[2564]: I0729 20:13:31.847132 2564 scope.go:117] "RemoveContainer" containerID="6777daae1396096745b9593744e33d0f5ceb2e44a014d3532e60a488ab259e97" Jul 29 20:13:31.856213 ip-10-0-131-93 kubenswrapper[2564]: I0729 20:13:31.856192 2564 scope.go:117] "RemoveContainer" containerID="61942868a82e4552e3c5485e29efe49c8886ea19ab46b357eb2af8f7de8ab63a" Jul 29 20:13:31.863979 ip-10-0-131-93 kubenswrapper[2564]: I0729 20:13:31.863957 2564 scope.go:117] "RemoveContainer" containerID="6777daae1396096745b9593744e33d0f5ceb2e44a014d3532e60a488ab259e97" Jul 29 20:13:31.864261 ip-10-0-131-93 kubenswrapper[2564]: E0729 20:13:31.864239 2564 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"6777daae1396096745b9593744e33d0f5ceb2e44a014d3532e60a488ab259e97\": container with ID starting with 6777daae1396096745b9593744e33d0f5ceb2e44a014d3532e60a488ab259e97 not found: ID does not exist" containerID="6777daae1396096745b9593744e33d0f5ceb2e44a014d3532e60a488ab259e97" Jul 29 20:13:31.864317 ip-10-0-131-93 kubenswrapper[2564]: I0729 20:13:31.864272 2564 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"6777daae1396096745b9593744e33d0f5ceb2e44a014d3532e60a488ab259e97"} err="failed to get container status \"6777daae1396096745b9593744e33d0f5ceb2e44a014d3532e60a488ab259e97\": rpc error: code = NotFound desc = could not find container \"6777daae1396096745b9593744e33d0f5ceb2e44a014d3532e60a488ab259e97\": container with ID starting with 6777daae1396096745b9593744e33d0f5ceb2e44a014d3532e60a488ab259e97 not found: ID does not exist" Jul 29 20:13:31.864317 ip-10-0-131-93 kubenswrapper[2564]: I0729 20:13:31.864289 2564 scope.go:117] "RemoveContainer" containerID="61942868a82e4552e3c5485e29efe49c8886ea19ab46b357eb2af8f7de8ab63a" Jul 29 20:13:31.864549 ip-10-0-131-93 kubenswrapper[2564]: E0729 20:13:31.864531 2564 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"61942868a82e4552e3c5485e29efe49c8886ea19ab46b357eb2af8f7de8ab63a\": container with ID starting with 61942868a82e4552e3c5485e29efe49c8886ea19ab46b357eb2af8f7de8ab63a not found: ID does not exist" containerID="61942868a82e4552e3c5485e29efe49c8886ea19ab46b357eb2af8f7de8ab63a" Jul 29 20:13:31.864599 ip-10-0-131-93 kubenswrapper[2564]: I0729 20:13:31.864555 2564 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"61942868a82e4552e3c5485e29efe49c8886ea19ab46b357eb2af8f7de8ab63a"} err="failed to get container status \"61942868a82e4552e3c5485e29efe49c8886ea19ab46b357eb2af8f7de8ab63a\": rpc error: code = NotFound desc = could not find container \"61942868a82e4552e3c5485e29efe49c8886ea19ab46b357eb2af8f7de8ab63a\": container with ID starting with 61942868a82e4552e3c5485e29efe49c8886ea19ab46b357eb2af8f7de8ab63a not found: ID does not exist" Jul 29 20:13:31.870057 ip-10-0-131-93 kubenswrapper[2564]: I0729 20:13:31.870031 2564 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-ingress/e2e-shared-b-4c787c-openshift-default-66676df75b-f49kx"] Jul 29 20:13:31.877550 ip-10-0-131-93 kubenswrapper[2564]: I0729 20:13:31.877526 2564 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["openshift-ingress/e2e-shared-b-4c787c-openshift-default-66676df75b-f49kx"] Jul 29 20:13:32.169295 ip-10-0-131-93 kubenswrapper[2564]: I0729 20:13:32.169213 2564 log.go:25] "Finished parsing log file" path="/var/log/pods/opendatahub_maas-controller-865c54c777-mdwcz_a624f438-2b48-4e07-ae3f-aa7b3cd3d6a8/manager/0.log" Jul 29 20:13:32.546814 ip-10-0-131-93 kubenswrapper[2564]: I0729 20:13:32.546784 2564 log.go:25] "Finished parsing log file" path="/var/log/pods/opendatahub_opendatahub-operator-controller-manager-d84699fcf-k4tz9_ed963675-70ff-4f05-84ae-1e2b2702100c/copy-manifests/0.log" Jul 29 20:13:32.559112 ip-10-0-131-93 kubenswrapper[2564]: I0729 20:13:32.559090 2564 log.go:25] "Finished parsing log file" path="/var/log/pods/opendatahub_opendatahub-operator-controller-manager-d84699fcf-k4tz9_ed963675-70ff-4f05-84ae-1e2b2702100c/manager/0.log" Jul 29 20:13:32.783538 ip-10-0-131-93 kubenswrapper[2564]: I0729 20:13:32.783507 2564 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="15f29128-6b6b-4023-b057-184a58fe96cc" path="/var/lib/kubelet/pods/15f29128-6b6b-4023-b057-184a58fe96cc/volumes" Jul 29 20:13:33.150263 ip-10-0-131-93 kubenswrapper[2564]: I0729 20:13:33.150236 2564 log.go:25] "Finished parsing log file" path="/var/log/pods/odh-ai-gateway-infra_maas-api-7899cfcd6f-cmvs7_af405bb4-058a-46df-84b0-1b5154a7f124/maas-api/0.log" Jul 29 20:13:34.316596 ip-10-0-131-93 kubenswrapper[2564]: I0729 20:13:34.316565 2564 log.go:25] "Finished parsing log file" path="/var/log/pods/kuadrant-system_0acee64185f523d1d1272e9af2e4d9333e0dcde792ba30e1fa9605b7596mj7k_32a5fe7d-49f0-44cb-af5a-e4a06e29b452/pull/0.log" Jul 29 20:13:34.323207 ip-10-0-131-93 kubenswrapper[2564]: I0729 20:13:34.323187 2564 log.go:25] "Finished parsing log file" path="/var/log/pods/kuadrant-system_0acee64185f523d1d1272e9af2e4d9333e0dcde792ba30e1fa9605b7596mj7k_32a5fe7d-49f0-44cb-af5a-e4a06e29b452/extract/0.log" Jul 29 20:13:34.331288 ip-10-0-131-93 kubenswrapper[2564]: I0729 20:13:34.331271 2564 log.go:25] "Finished parsing log file" path="/var/log/pods/kuadrant-system_0acee64185f523d1d1272e9af2e4d9333e0dcde792ba30e1fa9605b7596mj7k_32a5fe7d-49f0-44cb-af5a-e4a06e29b452/util/0.log" Jul 29 20:13:34.438405 ip-10-0-131-93 kubenswrapper[2564]: I0729 20:13:34.438354 2564 log.go:25] "Finished parsing log file" path="/var/log/pods/kuadrant-system_19cb86e64775c5699d5aacf881a09c2d51e7f55f9e1ff096f2a667c5e0cwwkj_9e24b906-6ade-47ef-818c-28a1c450e0be/util/0.log" Jul 29 20:13:34.445069 ip-10-0-131-93 kubenswrapper[2564]: I0729 20:13:34.445049 2564 log.go:25] "Finished parsing log file" path="/var/log/pods/kuadrant-system_19cb86e64775c5699d5aacf881a09c2d51e7f55f9e1ff096f2a667c5e0cwwkj_9e24b906-6ade-47ef-818c-28a1c450e0be/pull/0.log" Jul 29 20:13:34.451881 ip-10-0-131-93 kubenswrapper[2564]: I0729 20:13:34.451864 2564 log.go:25] "Finished parsing log file" path="/var/log/pods/kuadrant-system_19cb86e64775c5699d5aacf881a09c2d51e7f55f9e1ff096f2a667c5e0cwwkj_9e24b906-6ade-47ef-818c-28a1c450e0be/extract/0.log" Jul 29 20:13:34.551675 ip-10-0-131-93 kubenswrapper[2564]: I0729 20:13:34.551646 2564 log.go:25] "Finished parsing log file" path="/var/log/pods/kuadrant-system_5fc36eb1065777914bfe0ff85f9a202e2a3bafefb563d9e2994474ed736jqbs_83b6dd40-61a5-45e0-b860-ef5f84491141/util/0.log" Jul 29 20:13:34.557767 ip-10-0-131-93 kubenswrapper[2564]: I0729 20:13:34.557748 2564 log.go:25] "Finished parsing log file" path="/var/log/pods/kuadrant-system_5fc36eb1065777914bfe0ff85f9a202e2a3bafefb563d9e2994474ed736jqbs_83b6dd40-61a5-45e0-b860-ef5f84491141/pull/0.log" Jul 29 20:13:34.563928 ip-10-0-131-93 kubenswrapper[2564]: I0729 20:13:34.563911 2564 log.go:25] "Finished parsing log file" path="/var/log/pods/kuadrant-system_5fc36eb1065777914bfe0ff85f9a202e2a3bafefb563d9e2994474ed736jqbs_83b6dd40-61a5-45e0-b860-ef5f84491141/extract/0.log" Jul 29 20:13:34.690086 ip-10-0-131-93 kubenswrapper[2564]: I0729 20:13:34.690060 2564 log.go:25] "Finished parsing log file" path="/var/log/pods/kuadrant-system_9438e18bbad664a016701a3153a6b421e2d977be7eb0117f80bcf45ef1g46qj_dcd5230c-d041-4831-8280-c00a332f038d/util/0.log" Jul 29 20:13:34.698199 ip-10-0-131-93 kubenswrapper[2564]: I0729 20:13:34.698178 2564 log.go:25] "Finished parsing log file" path="/var/log/pods/kuadrant-system_9438e18bbad664a016701a3153a6b421e2d977be7eb0117f80bcf45ef1g46qj_dcd5230c-d041-4831-8280-c00a332f038d/pull/0.log" Jul 29 20:13:34.708201 ip-10-0-131-93 kubenswrapper[2564]: I0729 20:13:34.708184 2564 log.go:25] "Finished parsing log file" path="/var/log/pods/kuadrant-system_9438e18bbad664a016701a3153a6b421e2d977be7eb0117f80bcf45ef1g46qj_dcd5230c-d041-4831-8280-c00a332f038d/extract/0.log" Jul 29 20:13:35.916969 ip-10-0-131-93 kubenswrapper[2564]: I0729 20:13:35.916939 2564 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ingress_data-science-gateway-data-science-gateway-class-d66cd8c7c-jzkd8_ed425122-f06f-4c32-aaeb-dbe3595cfd27/istio-proxy/0.log" Jul 29 20:13:36.051564 ip-10-0-131-93 kubenswrapper[2564]: I0729 20:13:36.051534 2564 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ingress_istiod-openshift-gateway-5677ccb49b-xdczl_6178ee97-b1fe-4de9-9fc6-2101b33978fe/discovery/0.log" Jul 29 20:13:36.598037 ip-10-0-131-93 kubenswrapper[2564]: I0729 20:13:36.598011 2564 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ingress_payload-processing-5cc6cd444b-zf8l2_21cbaa33-9f8b-47c3-8cc1-5a368b477eb2/payload-processing/0.log" Jul 29 20:13:37.242324 ip-10-0-131-93 kubenswrapper[2564]: I0729 20:13:37.242295 2564 log.go:25] "Finished parsing log file" path="/var/log/pods/llm_e2e-trlp-test-simulated-kserve-598466548d-m8bd5_9c2ff71a-f846-43de-92b0-5b8975ff575d/main/0.log" Jul 29 20:13:37.344561 ip-10-0-131-93 kubenswrapper[2564]: I0729 20:13:37.344534 2564 log.go:25] "Finished parsing log file" path="/var/log/pods/llm_e2e-unconfigured-facebook-opt-125m-simulated-kserve-85b68f6cd4t_9c140dd1-b52c-4615-9145-b0c0e4a53e43/main/0.log" Jul 29 20:13:39.458130 ip-10-0-131-93 kubenswrapper[2564]: I0729 20:13:39.458098 2564 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-444jx_faa8c546-c75b-4e10-93b4-661aa2b5f53e/konnectivity-agent/0.log" limit=1024 Jul 29 20:13:42.768401 ip-10-0-131-93 kubenswrapper[2564]: I0729 20:13:42.768358 2564 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-444jx_faa8c546-c75b-4e10-93b4-661aa2b5f53e/konnectivity-agent/0.log" limit=1024 Jul 29 20:13:42.858882 ip-10-0-131-93 kubenswrapper[2564]: I0729 20:13:42.858850 2564 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-444jx_faa8c546-c75b-4e10-93b4-661aa2b5f53e/konnectivity-agent/0.log" limit=1024 Jul 29 20:13:45.761011 ip-10-0-131-93 kubenswrapper[2564]: I0729 20:13:45.760984 2564 log.go:25] "Finished parsing log file" path="/var/log/pods/kube-system_global-pull-secret-syncer-zcgr9_5c0eec03-fbbc-40db-9c78-43c8f98534c7/global-pull-secret-syncer/0.log" Jul 29 20:13:45.828799 ip-10-0-131-93 kubenswrapper[2564]: I0729 20:13:45.828767 2564 log.go:25] "Finished parsing log file" path="/var/log/pods/kube-system_konnectivity-agent-444jx_faa8c546-c75b-4e10-93b4-661aa2b5f53e/konnectivity-agent/0.log" Jul 29 20:13:45.962246 ip-10-0-131-93 kubenswrapper[2564]: I0729 20:13:45.962217 2564 log.go:25] "Finished parsing log file" path="/var/log/pods/kube-system_kube-apiserver-proxy-ip-10-0-131-93.ec2.internal_a1b5cf65d676393f097a7d0896c452c5/haproxy/0.log" Jul 29 20:13:49.208711 ip-10-0-131-93 kubenswrapper[2564]: I0729 20:13:49.208683 2564 log.go:25] "Finished parsing log file" path="/var/log/pods/kuadrant-system_0acee64185f523d1d1272e9af2e4d9333e0dcde792ba30e1fa9605b7596mj7k_32a5fe7d-49f0-44cb-af5a-e4a06e29b452/extract/0.log" Jul 29 20:13:49.235519 ip-10-0-131-93 kubenswrapper[2564]: I0729 20:13:49.235494 2564 log.go:25] "Finished parsing log file" path="/var/log/pods/kuadrant-system_0acee64185f523d1d1272e9af2e4d9333e0dcde792ba30e1fa9605b7596mj7k_32a5fe7d-49f0-44cb-af5a-e4a06e29b452/util/0.log" Jul 29 20:13:49.263042 ip-10-0-131-93 kubenswrapper[2564]: I0729 20:13:49.263017 2564 log.go:25] "Finished parsing log file" path="/var/log/pods/kuadrant-system_0acee64185f523d1d1272e9af2e4d9333e0dcde792ba30e1fa9605b7596mj7k_32a5fe7d-49f0-44cb-af5a-e4a06e29b452/pull/0.log" Jul 29 20:13:49.292769 ip-10-0-131-93 kubenswrapper[2564]: I0729 20:13:49.292741 2564 log.go:25] "Finished parsing log file" path="/var/log/pods/kuadrant-system_19cb86e64775c5699d5aacf881a09c2d51e7f55f9e1ff096f2a667c5e0cwwkj_9e24b906-6ade-47ef-818c-28a1c450e0be/extract/0.log" Jul 29 20:13:49.331781 ip-10-0-131-93 kubenswrapper[2564]: I0729 20:13:49.331751 2564 log.go:25] "Finished parsing log file" path="/var/log/pods/kuadrant-system_19cb86e64775c5699d5aacf881a09c2d51e7f55f9e1ff096f2a667c5e0cwwkj_9e24b906-6ade-47ef-818c-28a1c450e0be/util/0.log" Jul 29 20:13:49.359767 ip-10-0-131-93 kubenswrapper[2564]: I0729 20:13:49.359744 2564 log.go:25] "Finished parsing log file" path="/var/log/pods/kuadrant-system_19cb86e64775c5699d5aacf881a09c2d51e7f55f9e1ff096f2a667c5e0cwwkj_9e24b906-6ade-47ef-818c-28a1c450e0be/pull/0.log" Jul 29 20:13:49.393893 ip-10-0-131-93 kubenswrapper[2564]: I0729 20:13:49.393869 2564 log.go:25] "Finished parsing log file" path="/var/log/pods/kuadrant-system_5fc36eb1065777914bfe0ff85f9a202e2a3bafefb563d9e2994474ed736jqbs_83b6dd40-61a5-45e0-b860-ef5f84491141/extract/0.log" Jul 29 20:13:49.419553 ip-10-0-131-93 kubenswrapper[2564]: I0729 20:13:49.419529 2564 log.go:25] "Finished parsing log file" path="/var/log/pods/kuadrant-system_5fc36eb1065777914bfe0ff85f9a202e2a3bafefb563d9e2994474ed736jqbs_83b6dd40-61a5-45e0-b860-ef5f84491141/util/0.log" Jul 29 20:13:49.448149 ip-10-0-131-93 kubenswrapper[2564]: I0729 20:13:49.448127 2564 log.go:25] "Finished parsing log file" path="/var/log/pods/kuadrant-system_5fc36eb1065777914bfe0ff85f9a202e2a3bafefb563d9e2994474ed736jqbs_83b6dd40-61a5-45e0-b860-ef5f84491141/pull/0.log" Jul 29 20:13:49.485591 ip-10-0-131-93 kubenswrapper[2564]: I0729 20:13:49.485524 2564 log.go:25] "Finished parsing log file" path="/var/log/pods/kuadrant-system_9438e18bbad664a016701a3153a6b421e2d977be7eb0117f80bcf45ef1g46qj_dcd5230c-d041-4831-8280-c00a332f038d/extract/0.log" Jul 29 20:13:49.512718 ip-10-0-131-93 kubenswrapper[2564]: I0729 20:13:49.512694 2564 log.go:25] "Finished parsing log file" path="/var/log/pods/kuadrant-system_9438e18bbad664a016701a3153a6b421e2d977be7eb0117f80bcf45ef1g46qj_dcd5230c-d041-4831-8280-c00a332f038d/util/0.log" Jul 29 20:13:49.540882 ip-10-0-131-93 kubenswrapper[2564]: I0729 20:13:49.540857 2564 log.go:25] "Finished parsing log file" path="/var/log/pods/kuadrant-system_9438e18bbad664a016701a3153a6b421e2d977be7eb0117f80bcf45ef1g46qj_dcd5230c-d041-4831-8280-c00a332f038d/pull/0.log" Jul 29 20:13:50.693388 ip-10-0-131-93 kubenswrapper[2564]: I0729 20:13:50.693335 2564 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-444jx_faa8c546-c75b-4e10-93b4-661aa2b5f53e/konnectivity-agent/0.log" limit=1024 Jul 29 20:13:52.479620 ip-10-0-131-93 kubenswrapper[2564]: I0729 20:13:52.479585 2564 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_kube-state-metrics-56d9f944b4-mb46f_b36c4189-c565-4c30-97c8-8b8811a881b2/kube-state-metrics/0.log" Jul 29 20:13:52.507807 ip-10-0-131-93 kubenswrapper[2564]: I0729 20:13:52.507775 2564 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_kube-state-metrics-56d9f944b4-mb46f_b36c4189-c565-4c30-97c8-8b8811a881b2/kube-rbac-proxy-main/0.log" Jul 29 20:13:52.529820 ip-10-0-131-93 kubenswrapper[2564]: I0729 20:13:52.529795 2564 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_kube-state-metrics-56d9f944b4-mb46f_b36c4189-c565-4c30-97c8-8b8811a881b2/kube-rbac-proxy-self/0.log" Jul 29 20:13:52.756673 ip-10-0-131-93 kubenswrapper[2564]: I0729 20:13:52.756603 2564 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_node-exporter-95bmx_09be503d-5e55-4ef7-9339-7f1a9b8c2029/node-exporter/0.log" Jul 29 20:13:52.778761 ip-10-0-131-93 kubenswrapper[2564]: I0729 20:13:52.778737 2564 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_node-exporter-95bmx_09be503d-5e55-4ef7-9339-7f1a9b8c2029/kube-rbac-proxy/0.log" Jul 29 20:13:52.811850 ip-10-0-131-93 kubenswrapper[2564]: I0729 20:13:52.811827 2564 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_node-exporter-95bmx_09be503d-5e55-4ef7-9339-7f1a9b8c2029/init-textfile/0.log" Jul 29 20:13:56.209054 ip-10-0-131-93 kubenswrapper[2564]: I0729 20:13:56.208975 2564 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-must-gather-7np8j/perf-node-gather-daemonset-bhlkn"] Jul 29 20:13:56.209524 ip-10-0-131-93 kubenswrapper[2564]: I0729 20:13:56.209274 2564 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="c9bc01d1-d52b-4476-942e-68c5a3641d5f" containerName="istio-proxy" Jul 29 20:13:56.209524 ip-10-0-131-93 kubenswrapper[2564]: I0729 20:13:56.209285 2564 state_mem.go:107] "Deleted CPUSet assignment" podUID="c9bc01d1-d52b-4476-942e-68c5a3641d5f" containerName="istio-proxy" Jul 29 20:13:56.209524 ip-10-0-131-93 kubenswrapper[2564]: I0729 20:13:56.209296 2564 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="b107bd8a-8f6f-4db0-8e9e-44a29271b1d5" containerName="istio-proxy" Jul 29 20:13:56.209524 ip-10-0-131-93 kubenswrapper[2564]: I0729 20:13:56.209302 2564 state_mem.go:107] "Deleted CPUSet assignment" podUID="b107bd8a-8f6f-4db0-8e9e-44a29271b1d5" containerName="istio-proxy" Jul 29 20:13:56.209524 ip-10-0-131-93 kubenswrapper[2564]: I0729 20:13:56.209314 2564 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="252a57d2-9e17-4d68-83bb-fff34a75bcd9" containerName="payload-pre-processing" Jul 29 20:13:56.209524 ip-10-0-131-93 kubenswrapper[2564]: I0729 20:13:56.209319 2564 state_mem.go:107] "Deleted CPUSet assignment" podUID="252a57d2-9e17-4d68-83bb-fff34a75bcd9" containerName="payload-pre-processing" Jul 29 20:13:56.209524 ip-10-0-131-93 kubenswrapper[2564]: I0729 20:13:56.209326 2564 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="b107bd8a-8f6f-4db0-8e9e-44a29271b1d5" containerName="istio-proxy" Jul 29 20:13:56.209524 ip-10-0-131-93 kubenswrapper[2564]: I0729 20:13:56.209333 2564 state_mem.go:107] "Deleted CPUSet assignment" podUID="b107bd8a-8f6f-4db0-8e9e-44a29271b1d5" containerName="istio-proxy" Jul 29 20:13:56.209524 ip-10-0-131-93 kubenswrapper[2564]: I0729 20:13:56.209341 2564 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="15f29128-6b6b-4023-b057-184a58fe96cc" containerName="istio-proxy" Jul 29 20:13:56.209524 ip-10-0-131-93 kubenswrapper[2564]: I0729 20:13:56.209346 2564 state_mem.go:107] "Deleted CPUSet assignment" podUID="15f29128-6b6b-4023-b057-184a58fe96cc" containerName="istio-proxy" Jul 29 20:13:56.209524 ip-10-0-131-93 kubenswrapper[2564]: I0729 20:13:56.209352 2564 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="00002c08-1197-4913-9b28-4442dcb6b959" containerName="maas-api" Jul 29 20:13:56.209524 ip-10-0-131-93 kubenswrapper[2564]: I0729 20:13:56.209358 2564 state_mem.go:107] "Deleted CPUSet assignment" podUID="00002c08-1197-4913-9b28-4442dcb6b959" containerName="maas-api" Jul 29 20:13:56.209524 ip-10-0-131-93 kubenswrapper[2564]: I0729 20:13:56.209367 2564 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="b7200237-7be7-4661-ba46-f13ec3fc68a8" containerName="payload-pre-processing" Jul 29 20:13:56.209524 ip-10-0-131-93 kubenswrapper[2564]: I0729 20:13:56.209386 2564 state_mem.go:107] "Deleted CPUSet assignment" podUID="b7200237-7be7-4661-ba46-f13ec3fc68a8" containerName="payload-pre-processing" Jul 29 20:13:56.209524 ip-10-0-131-93 kubenswrapper[2564]: I0729 20:13:56.209393 2564 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="94a75046-fecd-413c-ad52-daf45c5732cd" containerName="istio-proxy" Jul 29 20:13:56.209524 ip-10-0-131-93 kubenswrapper[2564]: I0729 20:13:56.209399 2564 state_mem.go:107] "Deleted CPUSet assignment" podUID="94a75046-fecd-413c-ad52-daf45c5732cd" containerName="istio-proxy" Jul 29 20:13:56.209524 ip-10-0-131-93 kubenswrapper[2564]: I0729 20:13:56.209406 2564 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="94a75046-fecd-413c-ad52-daf45c5732cd" containerName="istio-proxy" Jul 29 20:13:56.209524 ip-10-0-131-93 kubenswrapper[2564]: I0729 20:13:56.209411 2564 state_mem.go:107] "Deleted CPUSet assignment" podUID="94a75046-fecd-413c-ad52-daf45c5732cd" containerName="istio-proxy" Jul 29 20:13:56.209524 ip-10-0-131-93 kubenswrapper[2564]: I0729 20:13:56.209459 2564 memory_manager.go:368] "RemoveStaleState removing state" podUID="94a75046-fecd-413c-ad52-daf45c5732cd" containerName="istio-proxy" Jul 29 20:13:56.209524 ip-10-0-131-93 kubenswrapper[2564]: I0729 20:13:56.209466 2564 memory_manager.go:368] "RemoveStaleState removing state" podUID="00002c08-1197-4913-9b28-4442dcb6b959" containerName="maas-api" Jul 29 20:13:56.209524 ip-10-0-131-93 kubenswrapper[2564]: I0729 20:13:56.209473 2564 memory_manager.go:368] "RemoveStaleState removing state" podUID="94a75046-fecd-413c-ad52-daf45c5732cd" containerName="istio-proxy" Jul 29 20:13:56.209524 ip-10-0-131-93 kubenswrapper[2564]: I0729 20:13:56.209479 2564 memory_manager.go:368] "RemoveStaleState removing state" podUID="b107bd8a-8f6f-4db0-8e9e-44a29271b1d5" containerName="istio-proxy" Jul 29 20:13:56.209524 ip-10-0-131-93 kubenswrapper[2564]: I0729 20:13:56.209513 2564 memory_manager.go:368] "RemoveStaleState removing state" podUID="15f29128-6b6b-4023-b057-184a58fe96cc" containerName="istio-proxy" Jul 29 20:13:56.209524 ip-10-0-131-93 kubenswrapper[2564]: I0729 20:13:56.209519 2564 memory_manager.go:368] "RemoveStaleState removing state" podUID="b7200237-7be7-4661-ba46-f13ec3fc68a8" containerName="payload-pre-processing" Jul 29 20:13:56.209524 ip-10-0-131-93 kubenswrapper[2564]: I0729 20:13:56.209525 2564 memory_manager.go:368] "RemoveStaleState removing state" podUID="252a57d2-9e17-4d68-83bb-fff34a75bcd9" containerName="payload-pre-processing" Jul 29 20:13:56.209524 ip-10-0-131-93 kubenswrapper[2564]: I0729 20:13:56.209531 2564 memory_manager.go:368] "RemoveStaleState removing state" podUID="c9bc01d1-d52b-4476-942e-68c5a3641d5f" containerName="istio-proxy" Jul 29 20:13:56.209524 ip-10-0-131-93 kubenswrapper[2564]: I0729 20:13:56.209536 2564 memory_manager.go:368] "RemoveStaleState removing state" podUID="c9bc01d1-d52b-4476-942e-68c5a3641d5f" containerName="istio-proxy" Jul 29 20:13:56.212602 ip-10-0-131-93 kubenswrapper[2564]: I0729 20:13:56.212587 2564 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-must-gather-7np8j/perf-node-gather-daemonset-bhlkn" Jul 29 20:13:56.214955 ip-10-0-131-93 kubenswrapper[2564]: I0729 20:13:56.214921 2564 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-must-gather-7np8j\"/\"default-dockercfg-xfh45\"" Jul 29 20:13:56.215721 ip-10-0-131-93 kubenswrapper[2564]: I0729 20:13:56.215702 2564 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-must-gather-7np8j\"/\"kube-root-ca.crt\"" Jul 29 20:13:56.216054 ip-10-0-131-93 kubenswrapper[2564]: I0729 20:13:56.216031 2564 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-must-gather-7np8j\"/\"openshift-service-ca.crt\"" Jul 29 20:13:56.222169 ip-10-0-131-93 kubenswrapper[2564]: I0729 20:13:56.222148 2564 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-must-gather-7np8j/perf-node-gather-daemonset-bhlkn"] Jul 29 20:13:56.303415 ip-10-0-131-93 kubenswrapper[2564]: I0729 20:13:56.303365 2564 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"proc\" (UniqueName: \"kubernetes.io/host-path/d5329786-d964-4630-9161-388e013a5d66-proc\") pod \"perf-node-gather-daemonset-bhlkn\" (UID: \"d5329786-d964-4630-9161-388e013a5d66\") " pod="openshift-must-gather-7np8j/perf-node-gather-daemonset-bhlkn" Jul 29 20:13:56.303587 ip-10-0-131-93 kubenswrapper[2564]: I0729 20:13:56.303426 2564 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"lib-modules\" (UniqueName: \"kubernetes.io/host-path/d5329786-d964-4630-9161-388e013a5d66-lib-modules\") pod \"perf-node-gather-daemonset-bhlkn\" (UID: \"d5329786-d964-4630-9161-388e013a5d66\") " pod="openshift-must-gather-7np8j/perf-node-gather-daemonset-bhlkn" Jul 29 20:13:56.303587 ip-10-0-131-93 kubenswrapper[2564]: I0729 20:13:56.303485 2564 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"podres\" (UniqueName: \"kubernetes.io/host-path/d5329786-d964-4630-9161-388e013a5d66-podres\") pod \"perf-node-gather-daemonset-bhlkn\" (UID: \"d5329786-d964-4630-9161-388e013a5d66\") " pod="openshift-must-gather-7np8j/perf-node-gather-daemonset-bhlkn" Jul 29 20:13:56.303587 ip-10-0-131-93 kubenswrapper[2564]: I0729 20:13:56.303541 2564 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"sys\" (UniqueName: \"kubernetes.io/host-path/d5329786-d964-4630-9161-388e013a5d66-sys\") pod \"perf-node-gather-daemonset-bhlkn\" (UID: \"d5329786-d964-4630-9161-388e013a5d66\") " pod="openshift-must-gather-7np8j/perf-node-gather-daemonset-bhlkn" Jul 29 20:13:56.303587 ip-10-0-131-93 kubenswrapper[2564]: I0729 20:13:56.303567 2564 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-nlqnf\" (UniqueName: \"kubernetes.io/projected/d5329786-d964-4630-9161-388e013a5d66-kube-api-access-nlqnf\") pod \"perf-node-gather-daemonset-bhlkn\" (UID: \"d5329786-d964-4630-9161-388e013a5d66\") " pod="openshift-must-gather-7np8j/perf-node-gather-daemonset-bhlkn" Jul 29 20:13:56.404808 ip-10-0-131-93 kubenswrapper[2564]: I0729 20:13:56.404767 2564 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"podres\" (UniqueName: \"kubernetes.io/host-path/d5329786-d964-4630-9161-388e013a5d66-podres\") pod \"perf-node-gather-daemonset-bhlkn\" (UID: \"d5329786-d964-4630-9161-388e013a5d66\") " pod="openshift-must-gather-7np8j/perf-node-gather-daemonset-bhlkn" Jul 29 20:13:56.405012 ip-10-0-131-93 kubenswrapper[2564]: I0729 20:13:56.404824 2564 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"sys\" (UniqueName: \"kubernetes.io/host-path/d5329786-d964-4630-9161-388e013a5d66-sys\") pod \"perf-node-gather-daemonset-bhlkn\" (UID: \"d5329786-d964-4630-9161-388e013a5d66\") " pod="openshift-must-gather-7np8j/perf-node-gather-daemonset-bhlkn" Jul 29 20:13:56.405012 ip-10-0-131-93 kubenswrapper[2564]: I0729 20:13:56.404844 2564 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-nlqnf\" (UniqueName: \"kubernetes.io/projected/d5329786-d964-4630-9161-388e013a5d66-kube-api-access-nlqnf\") pod \"perf-node-gather-daemonset-bhlkn\" (UID: \"d5329786-d964-4630-9161-388e013a5d66\") " pod="openshift-must-gather-7np8j/perf-node-gather-daemonset-bhlkn" Jul 29 20:13:56.405012 ip-10-0-131-93 kubenswrapper[2564]: I0729 20:13:56.404880 2564 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"proc\" (UniqueName: \"kubernetes.io/host-path/d5329786-d964-4630-9161-388e013a5d66-proc\") pod \"perf-node-gather-daemonset-bhlkn\" (UID: \"d5329786-d964-4630-9161-388e013a5d66\") " pod="openshift-must-gather-7np8j/perf-node-gather-daemonset-bhlkn" Jul 29 20:13:56.405012 ip-10-0-131-93 kubenswrapper[2564]: I0729 20:13:56.404896 2564 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"lib-modules\" (UniqueName: \"kubernetes.io/host-path/d5329786-d964-4630-9161-388e013a5d66-lib-modules\") pod \"perf-node-gather-daemonset-bhlkn\" (UID: \"d5329786-d964-4630-9161-388e013a5d66\") " pod="openshift-must-gather-7np8j/perf-node-gather-daemonset-bhlkn" Jul 29 20:13:56.405012 ip-10-0-131-93 kubenswrapper[2564]: I0729 20:13:56.404912 2564 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"sys\" (UniqueName: \"kubernetes.io/host-path/d5329786-d964-4630-9161-388e013a5d66-sys\") pod \"perf-node-gather-daemonset-bhlkn\" (UID: \"d5329786-d964-4630-9161-388e013a5d66\") " pod="openshift-must-gather-7np8j/perf-node-gather-daemonset-bhlkn" Jul 29 20:13:56.405012 ip-10-0-131-93 kubenswrapper[2564]: I0729 20:13:56.404963 2564 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"podres\" (UniqueName: \"kubernetes.io/host-path/d5329786-d964-4630-9161-388e013a5d66-podres\") pod \"perf-node-gather-daemonset-bhlkn\" (UID: \"d5329786-d964-4630-9161-388e013a5d66\") " pod="openshift-must-gather-7np8j/perf-node-gather-daemonset-bhlkn" Jul 29 20:13:56.405012 ip-10-0-131-93 kubenswrapper[2564]: I0729 20:13:56.404983 2564 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"proc\" (UniqueName: \"kubernetes.io/host-path/d5329786-d964-4630-9161-388e013a5d66-proc\") pod \"perf-node-gather-daemonset-bhlkn\" (UID: \"d5329786-d964-4630-9161-388e013a5d66\") " pod="openshift-must-gather-7np8j/perf-node-gather-daemonset-bhlkn" Jul 29 20:13:56.405264 ip-10-0-131-93 kubenswrapper[2564]: I0729 20:13:56.405034 2564 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"lib-modules\" (UniqueName: \"kubernetes.io/host-path/d5329786-d964-4630-9161-388e013a5d66-lib-modules\") pod \"perf-node-gather-daemonset-bhlkn\" (UID: \"d5329786-d964-4630-9161-388e013a5d66\") " pod="openshift-must-gather-7np8j/perf-node-gather-daemonset-bhlkn" Jul 29 20:13:56.414202 ip-10-0-131-93 kubenswrapper[2564]: I0729 20:13:56.414176 2564 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-nlqnf\" (UniqueName: \"kubernetes.io/projected/d5329786-d964-4630-9161-388e013a5d66-kube-api-access-nlqnf\") pod \"perf-node-gather-daemonset-bhlkn\" (UID: \"d5329786-d964-4630-9161-388e013a5d66\") " pod="openshift-must-gather-7np8j/perf-node-gather-daemonset-bhlkn" Jul 29 20:13:56.522826 ip-10-0-131-93 kubenswrapper[2564]: I0729 20:13:56.522732 2564 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-must-gather-7np8j/perf-node-gather-daemonset-bhlkn" Jul 29 20:13:56.652920 ip-10-0-131-93 kubenswrapper[2564]: I0729 20:13:56.652896 2564 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-must-gather-7np8j/perf-node-gather-daemonset-bhlkn"] Jul 29 20:13:56.655172 ip-10-0-131-93 kubenswrapper[2564]: W0729 20:13:56.655146 2564 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-podd5329786_d964_4630_9161_388e013a5d66.slice/crio-de38d0018ee61c0afa723be7fd30c788acdc7c01c6ac9fec55f8a12d602516ac WatchSource:0}: Error finding container de38d0018ee61c0afa723be7fd30c788acdc7c01c6ac9fec55f8a12d602516ac: Status 404 returned error can't find the container with id de38d0018ee61c0afa723be7fd30c788acdc7c01c6ac9fec55f8a12d602516ac Jul 29 20:13:56.970857 ip-10-0-131-93 kubenswrapper[2564]: I0729 20:13:56.970822 2564 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-must-gather-7np8j/perf-node-gather-daemonset-bhlkn" event={"ID":"d5329786-d964-4630-9161-388e013a5d66","Type":"ContainerStarted","Data":"0f73facf1e24304a3166825ca9cd6317b4212d17cc5d313c0d31f7d286a6a044"} Jul 29 20:13:56.971033 ip-10-0-131-93 kubenswrapper[2564]: I0729 20:13:56.970873 2564 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-must-gather-7np8j/perf-node-gather-daemonset-bhlkn" event={"ID":"d5329786-d964-4630-9161-388e013a5d66","Type":"ContainerStarted","Data":"de38d0018ee61c0afa723be7fd30c788acdc7c01c6ac9fec55f8a12d602516ac"} Jul 29 20:13:56.971033 ip-10-0-131-93 kubenswrapper[2564]: I0729 20:13:56.970967 2564 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-must-gather-7np8j/perf-node-gather-daemonset-bhlkn" Jul 29 20:13:57.970524 ip-10-0-131-93 kubenswrapper[2564]: I0729 20:13:57.970481 2564 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-dns_dns-default-kkmn5_d5e5a7da-26f5-4d8d-8e9e-4e8991392e8e/dns/0.log" Jul 29 20:13:57.995275 ip-10-0-131-93 kubenswrapper[2564]: I0729 20:13:57.995251 2564 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-dns_dns-default-kkmn5_d5e5a7da-26f5-4d8d-8e9e-4e8991392e8e/kube-rbac-proxy/0.log" Jul 29 20:13:58.123081 ip-10-0-131-93 kubenswrapper[2564]: I0729 20:13:58.123052 2564 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-dns_node-resolver-lzsmp_ffe05666-ed11-4573-9442-a9646b974e61/dns-node-resolver/0.log" Jul 29 20:13:58.775460 ip-10-0-131-93 kubenswrapper[2564]: I0729 20:13:58.775427 2564 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-image-registry_node-ca-2lv4n_edfe2676-f3b8-4eaa-ad17-ecf737b9f786/node-ca/0.log" Jul 29 20:13:59.789417 ip-10-0-131-93 kubenswrapper[2564]: I0729 20:13:59.789365 2564 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ingress_data-science-gateway-data-science-gateway-class-d66cd8c7c-jzkd8_ed425122-f06f-4c32-aaeb-dbe3595cfd27/istio-proxy/0.log" Jul 29 20:13:59.933063 ip-10-0-131-93 kubenswrapper[2564]: I0729 20:13:59.933030 2564 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ingress_istiod-openshift-gateway-5677ccb49b-xdczl_6178ee97-b1fe-4de9-9fc6-2101b33978fe/discovery/0.log" Jul 29 20:14:00.099505 ip-10-0-131-93 kubenswrapper[2564]: I0729 20:14:00.099405 2564 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ingress_payload-processing-5cc6cd444b-zf8l2_21cbaa33-9f8b-47c3-8cc1-5a368b477eb2/payload-processing/0.log" Jul 29 20:14:00.884528 ip-10-0-131-93 kubenswrapper[2564]: I0729 20:14:00.884499 2564 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-444jx_faa8c546-c75b-4e10-93b4-661aa2b5f53e/konnectivity-agent/0.log" limit=1024 Jul 29 20:14:01.022879 ip-10-0-131-93 kubenswrapper[2564]: I0729 20:14:01.022853 2564 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ingress-canary_ingress-canary-jbktw_3832137d-c339-44b6-8ddc-af8c959c659e/serve-healthcheck-canary/0.log" Jul 29 20:14:01.721533 ip-10-0-131-93 kubenswrapper[2564]: I0729 20:14:01.721502 2564 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-insights_insights-runtime-extractor-q4bqd_9887221a-35c3-4cb7-8f93-d03a8e1c0e8e/kube-rbac-proxy/0.log" Jul 29 20:14:01.746362 ip-10-0-131-93 kubenswrapper[2564]: I0729 20:14:01.746318 2564 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-insights_insights-runtime-extractor-q4bqd_9887221a-35c3-4cb7-8f93-d03a8e1c0e8e/exporter/0.log" Jul 29 20:14:01.770306 ip-10-0-131-93 kubenswrapper[2564]: I0729 20:14:01.770284 2564 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-insights_insights-runtime-extractor-q4bqd_9887221a-35c3-4cb7-8f93-d03a8e1c0e8e/extractor/0.log" Jul 29 20:14:02.985624 ip-10-0-131-93 kubenswrapper[2564]: I0729 20:14:02.985596 2564 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-must-gather-7np8j/perf-node-gather-daemonset-bhlkn" Jul 29 20:14:03.010359 ip-10-0-131-93 kubenswrapper[2564]: I0729 20:14:03.010302 2564 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-must-gather-7np8j/perf-node-gather-daemonset-bhlkn" podStartSLOduration=7.010286145 podStartE2EDuration="7.010286145s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=true podCreationTimestamp="2026-07-29 20:13:56 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-29 20:13:57.008883079 +0000 UTC m=+3412.732942770" watchObservedRunningTime="2026-07-29 20:14:03.010286145 +0000 UTC m=+3418.734345837" Jul 29 20:14:04.339793 ip-10-0-131-93 kubenswrapper[2564]: I0729 20:14:04.339752 2564 log.go:25] "Finished parsing log file" path="/var/log/pods/opendatahub_maas-controller-865c54c777-mdwcz_a624f438-2b48-4e07-ae3f-aa7b3cd3d6a8/manager/0.log" Jul 29 20:14:04.466049 ip-10-0-131-93 kubenswrapper[2564]: I0729 20:14:04.466022 2564 log.go:25] "Finished parsing log file" path="/var/log/pods/opendatahub_opendatahub-operator-controller-manager-d84699fcf-k4tz9_ed963675-70ff-4f05-84ae-1e2b2702100c/manager/0.log" Jul 29 20:14:04.489081 ip-10-0-131-93 kubenswrapper[2564]: I0729 20:14:04.489055 2564 log.go:25] "Finished parsing log file" path="/var/log/pods/opendatahub_opendatahub-operator-controller-manager-d84699fcf-k4tz9_ed963675-70ff-4f05-84ae-1e2b2702100c/copy-manifests/0.log" Jul 29 20:14:06.160136 ip-10-0-131-93 kubenswrapper[2564]: I0729 20:14:06.160031 2564 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-lws-operator_lws-controller-manager-ffdbfbbc8-d65tt_d369ea92-58ba-436e-891e-b85d304eaf09/manager/0.log" Jul 29 20:14:06.248810 ip-10-0-131-93 kubenswrapper[2564]: I0729 20:14:06.248774 2564 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ingress_data-science-gateway-data-science-gateway-class-d66cd8c7c-jzkd8_ed425122-f06f-4c32-aaeb-dbe3595cfd27/istio-proxy/0.log" Jul 29 20:14:06.406989 ip-10-0-131-93 kubenswrapper[2564]: I0729 20:14:06.406951 2564 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ingress_istiod-openshift-gateway-5677ccb49b-xdczl_6178ee97-b1fe-4de9-9fc6-2101b33978fe/discovery/0.log" Jul 29 20:14:06.544896 ip-10-0-131-93 kubenswrapper[2564]: I0729 20:14:06.544866 2564 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ingress_payload-processing-5cc6cd444b-zf8l2_21cbaa33-9f8b-47c3-8cc1-5a368b477eb2/payload-processing/0.log" Jul 29 20:14:07.950070 ip-10-0-131-93 kubenswrapper[2564]: I0729 20:14:07.950041 2564 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ingress_data-science-gateway-data-science-gateway-class-d66cd8c7c-jzkd8_ed425122-f06f-4c32-aaeb-dbe3595cfd27/istio-proxy/0.log" Jul 29 20:14:08.090023 ip-10-0-131-93 kubenswrapper[2564]: I0729 20:14:08.089993 2564 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ingress_istiod-openshift-gateway-5677ccb49b-xdczl_6178ee97-b1fe-4de9-9fc6-2101b33978fe/discovery/0.log" Jul 29 20:14:08.220117 ip-10-0-131-93 kubenswrapper[2564]: I0729 20:14:08.220040 2564 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ingress_payload-processing-5cc6cd444b-zf8l2_21cbaa33-9f8b-47c3-8cc1-5a368b477eb2/payload-processing/0.log" Jul 29 20:14:11.254905 ip-10-0-131-93 kubenswrapper[2564]: I0729 20:14:11.254873 2564 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-kube-storage-version-migrator_migrator-6865c9967-g6r5w_969f8591-6cf6-4c21-ac1e-4d8b6d827810/migrator/0.log" Jul 29 20:14:11.278633 ip-10-0-131-93 kubenswrapper[2564]: I0729 20:14:11.278597 2564 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-kube-storage-version-migrator_migrator-6865c9967-g6r5w_969f8591-6cf6-4c21-ac1e-4d8b6d827810/graceful-termination/0.log" Jul 29 20:14:12.584134 ip-10-0-131-93 kubenswrapper[2564]: I0729 20:14:12.584106 2564 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-multus_multus-additional-cni-plugins-nmm7d_01073023-f0f3-4517-affc-d5c86021c64d/kube-multus-additional-cni-plugins/0.log" Jul 29 20:14:12.617456 ip-10-0-131-93 kubenswrapper[2564]: I0729 20:14:12.617434 2564 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-multus_multus-additional-cni-plugins-nmm7d_01073023-f0f3-4517-affc-d5c86021c64d/egress-router-binary-copy/0.log" Jul 29 20:14:12.639737 ip-10-0-131-93 kubenswrapper[2564]: I0729 20:14:12.639717 2564 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-multus_multus-additional-cni-plugins-nmm7d_01073023-f0f3-4517-affc-d5c86021c64d/cni-plugins/0.log" Jul 29 20:14:12.660589 ip-10-0-131-93 kubenswrapper[2564]: I0729 20:14:12.660567 2564 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-multus_multus-additional-cni-plugins-nmm7d_01073023-f0f3-4517-affc-d5c86021c64d/bond-cni-plugin/0.log" Jul 29 20:14:12.685355 ip-10-0-131-93 kubenswrapper[2564]: I0729 20:14:12.685337 2564 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-multus_multus-additional-cni-plugins-nmm7d_01073023-f0f3-4517-affc-d5c86021c64d/routeoverride-cni/0.log" Jul 29 20:14:12.705511 ip-10-0-131-93 kubenswrapper[2564]: I0729 20:14:12.705496 2564 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-multus_multus-additional-cni-plugins-nmm7d_01073023-f0f3-4517-affc-d5c86021c64d/whereabouts-cni-bincopy/0.log" Jul 29 20:14:12.725052 ip-10-0-131-93 kubenswrapper[2564]: I0729 20:14:12.725034 2564 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-multus_multus-additional-cni-plugins-nmm7d_01073023-f0f3-4517-affc-d5c86021c64d/whereabouts-cni/0.log" Jul 29 20:14:13.105754 ip-10-0-131-93 kubenswrapper[2564]: I0729 20:14:13.105728 2564 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-multus_multus-ccv7v_6f081e41-049f-44dc-9aa4-47c2c509d0ff/kube-multus/0.log" Jul 29 20:14:13.203839 ip-10-0-131-93 kubenswrapper[2564]: I0729 20:14:13.203813 2564 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-multus_network-metrics-daemon-hkxvx_ae220db3-299c-481a-9591-01a8f3873d9a/network-metrics-daemon/0.log" Jul 29 20:14:13.222752 ip-10-0-131-93 kubenswrapper[2564]: I0729 20:14:13.222728 2564 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-multus_network-metrics-daemon-hkxvx_ae220db3-299c-481a-9591-01a8f3873d9a/kube-rbac-proxy/0.log"