Jul 03 11:10:12.167278 ip-10-0-136-0 systemd[1]: kubelet.service: Failed to load environment files: No such file or directory Jul 03 11:10:12.167290 ip-10-0-136-0 systemd[1]: kubelet.service: Failed to run 'start-pre' task: No such file or directory Jul 03 11:10:12.167297 ip-10-0-136-0 systemd[1]: kubelet.service: Failed with result 'resources'. Jul 03 11:10:12.167605 ip-10-0-136-0 systemd[1]: Failed to start Kubernetes Kubelet. Jul 03 11:10:22.402547 ip-10-0-136-0 systemd[1]: kubelet.service: Failed to schedule restart job: Unit crio.service not found. Jul 03 11:10:22.402568 ip-10-0-136-0 systemd[1]: kubelet.service: Failed with result 'resources'. -- Boot 2d8f8a0ec9cd4fbebb0304ca8597936b -- Jul 03 11:12:23.285565 ip-10-0-136-0 systemd[1]: Starting Kubernetes Kubelet... Jul 03 11:12:23.708091 ip-10-0-136-0 kubenswrapper[2579]: Flag --container-runtime-endpoint has been deprecated, This parameter should be set via the config file specified by the Kubelet's --config flag. See https://kubernetes.io/docs/tasks/administer-cluster/kubelet-config-file/ for more information. Jul 03 11:12:23.708091 ip-10-0-136-0 kubenswrapper[2579]: Flag --minimum-container-ttl-duration has been deprecated, Use --eviction-hard or --eviction-soft instead. Will be removed in a future version. Jul 03 11:12:23.708091 ip-10-0-136-0 kubenswrapper[2579]: Flag --volume-plugin-dir has been deprecated, This parameter should be set via the config file specified by the Kubelet's --config flag. See https://kubernetes.io/docs/tasks/administer-cluster/kubelet-config-file/ for more information. Jul 03 11:12:23.708091 ip-10-0-136-0 kubenswrapper[2579]: Flag --pod-infra-container-image has been deprecated, will be removed in 1.35. Image garbage collector will get sandbox image information from CRI. Jul 03 11:12:23.708091 ip-10-0-136-0 kubenswrapper[2579]: Flag --system-reserved has been deprecated, This parameter should be set via the config file specified by the Kubelet's --config flag. See https://kubernetes.io/docs/tasks/administer-cluster/kubelet-config-file/ for more information. Jul 03 11:12:23.709962 ip-10-0-136-0 kubenswrapper[2579]: I0703 11:12:23.709866 2579 server.go:213] "--pod-infra-container-image will not be pruned by the image garbage collector in kubelet and should also be set in the remote runtime" Jul 03 11:12:23.716554 ip-10-0-136-0 kubenswrapper[2579]: W0703 11:12:23.716515 2579 feature_gate.go:328] unrecognized feature gate: DyanmicServiceEndpointIBMCloud Jul 03 11:12:23.716554 ip-10-0-136-0 kubenswrapper[2579]: W0703 11:12:23.716545 2579 feature_gate.go:328] unrecognized feature gate: OpenShiftPodSecurityAdmission Jul 03 11:12:23.716554 ip-10-0-136-0 kubenswrapper[2579]: W0703 11:12:23.716549 2579 feature_gate.go:328] unrecognized feature gate: PinnedImages Jul 03 11:12:23.716554 ip-10-0-136-0 kubenswrapper[2579]: W0703 11:12:23.716552 2579 feature_gate.go:328] unrecognized feature gate: ManagedBootImages Jul 03 11:12:23.716554 ip-10-0-136-0 kubenswrapper[2579]: W0703 11:12:23.716556 2579 feature_gate.go:328] unrecognized feature gate: InsightsConfig Jul 03 11:12:23.716554 ip-10-0-136-0 kubenswrapper[2579]: W0703 11:12:23.716560 2579 feature_gate.go:328] unrecognized feature gate: MixedCPUsAllocation Jul 03 11:12:23.716554 ip-10-0-136-0 kubenswrapper[2579]: W0703 11:12:23.716563 2579 feature_gate.go:328] unrecognized feature gate: NewOLMBoxCutterRuntime Jul 03 11:12:23.716554 ip-10-0-136-0 kubenswrapper[2579]: W0703 11:12:23.716565 2579 feature_gate.go:328] unrecognized feature gate: KMSEncryption Jul 03 11:12:23.716874 ip-10-0-136-0 kubenswrapper[2579]: W0703 11:12:23.716570 2579 feature_gate.go:351] Setting GA feature gate VolumeAttributesClass=true. It will be removed in a future release. Jul 03 11:12:23.716874 ip-10-0-136-0 kubenswrapper[2579]: W0703 11:12:23.716585 2579 feature_gate.go:328] unrecognized feature gate: GCPClusterHostedDNSInstall Jul 03 11:12:23.716874 ip-10-0-136-0 kubenswrapper[2579]: W0703 11:12:23.716589 2579 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesAWS Jul 03 11:12:23.716874 ip-10-0-136-0 kubenswrapper[2579]: W0703 11:12:23.716592 2579 feature_gate.go:328] unrecognized feature gate: OSStreams Jul 03 11:12:23.716874 ip-10-0-136-0 kubenswrapper[2579]: W0703 11:12:23.716595 2579 feature_gate.go:328] unrecognized feature gate: EventTTL Jul 03 11:12:23.716874 ip-10-0-136-0 kubenswrapper[2579]: W0703 11:12:23.716598 2579 feature_gate.go:328] unrecognized feature gate: AzureWorkloadIdentity Jul 03 11:12:23.716874 ip-10-0-136-0 kubenswrapper[2579]: W0703 11:12:23.716601 2579 feature_gate.go:328] unrecognized feature gate: NewOLM Jul 03 11:12:23.716874 ip-10-0-136-0 kubenswrapper[2579]: W0703 11:12:23.716603 2579 feature_gate.go:328] unrecognized feature gate: ConsolePluginContentSecurityPolicy Jul 03 11:12:23.716874 ip-10-0-136-0 kubenswrapper[2579]: W0703 11:12:23.716606 2579 feature_gate.go:328] unrecognized feature gate: GCPDualStackInstall Jul 03 11:12:23.716874 ip-10-0-136-0 kubenswrapper[2579]: W0703 11:12:23.716609 2579 feature_gate.go:328] unrecognized feature gate: HyperShiftOnlyDynamicResourceAllocation Jul 03 11:12:23.716874 ip-10-0-136-0 kubenswrapper[2579]: W0703 11:12:23.716612 2579 feature_gate.go:328] unrecognized feature gate: ClusterMonitoringConfig Jul 03 11:12:23.716874 ip-10-0-136-0 kubenswrapper[2579]: W0703 11:12:23.716615 2579 feature_gate.go:328] unrecognized feature gate: ClusterAPIInstall Jul 03 11:12:23.716874 ip-10-0-136-0 kubenswrapper[2579]: W0703 11:12:23.716617 2579 feature_gate.go:328] unrecognized feature gate: VSphereMixedNodeEnv Jul 03 11:12:23.716874 ip-10-0-136-0 kubenswrapper[2579]: W0703 11:12:23.716624 2579 feature_gate.go:328] unrecognized feature gate: AWSClusterHostedDNS Jul 03 11:12:23.716874 ip-10-0-136-0 kubenswrapper[2579]: W0703 11:12:23.716626 2579 feature_gate.go:328] unrecognized feature gate: NutanixMultiSubnets Jul 03 11:12:23.716874 ip-10-0-136-0 kubenswrapper[2579]: W0703 11:12:23.716629 2579 feature_gate.go:328] unrecognized feature gate: AWSClusterHostedDNSInstall Jul 03 11:12:23.716874 ip-10-0-136-0 kubenswrapper[2579]: W0703 11:12:23.716631 2579 feature_gate.go:328] unrecognized feature gate: SigstoreImageVerificationPKI Jul 03 11:12:23.716874 ip-10-0-136-0 kubenswrapper[2579]: W0703 11:12:23.716635 2579 feature_gate.go:351] Setting GA feature gate ServiceAccountTokenNodeBinding=true. It will be removed in a future release. Jul 03 11:12:23.716874 ip-10-0-136-0 kubenswrapper[2579]: W0703 11:12:23.716639 2579 feature_gate.go:328] unrecognized feature gate: ExternalSnapshotMetadata Jul 03 11:12:23.717357 ip-10-0-136-0 kubenswrapper[2579]: W0703 11:12:23.716642 2579 feature_gate.go:328] unrecognized feature gate: EtcdBackendQuota Jul 03 11:12:23.717357 ip-10-0-136-0 kubenswrapper[2579]: W0703 11:12:23.716645 2579 feature_gate.go:328] unrecognized feature gate: KMSEncryptionProvider Jul 03 11:12:23.717357 ip-10-0-136-0 kubenswrapper[2579]: W0703 11:12:23.716648 2579 feature_gate.go:328] unrecognized feature gate: MetricsCollectionProfiles Jul 03 11:12:23.717357 ip-10-0-136-0 kubenswrapper[2579]: W0703 11:12:23.716650 2579 feature_gate.go:328] unrecognized feature gate: CPMSMachineNamePrefix Jul 03 11:12:23.717357 ip-10-0-136-0 kubenswrapper[2579]: W0703 11:12:23.716653 2579 feature_gate.go:328] unrecognized feature gate: VSphereMultiNetworks Jul 03 11:12:23.717357 ip-10-0-136-0 kubenswrapper[2579]: W0703 11:12:23.716656 2579 feature_gate.go:328] unrecognized feature gate: OnPremDNSRecords Jul 03 11:12:23.717357 ip-10-0-136-0 kubenswrapper[2579]: W0703 11:12:23.716661 2579 feature_gate.go:328] unrecognized feature gate: VSphereConfigurableMaxAllowedBlockVolumesPerNode Jul 03 11:12:23.717357 ip-10-0-136-0 kubenswrapper[2579]: W0703 11:12:23.716663 2579 feature_gate.go:328] unrecognized feature gate: VSphereMultiDisk Jul 03 11:12:23.717357 ip-10-0-136-0 kubenswrapper[2579]: W0703 11:12:23.716666 2579 feature_gate.go:328] unrecognized feature gate: NewOLMCatalogdAPIV1Metas Jul 03 11:12:23.717357 ip-10-0-136-0 kubenswrapper[2579]: W0703 11:12:23.716668 2579 feature_gate.go:328] unrecognized feature gate: VSphereHostVMGroupZonal Jul 03 11:12:23.717357 ip-10-0-136-0 kubenswrapper[2579]: W0703 11:12:23.716671 2579 feature_gate.go:328] unrecognized feature gate: MultiDiskSetup Jul 03 11:12:23.717357 ip-10-0-136-0 kubenswrapper[2579]: W0703 11:12:23.716674 2579 feature_gate.go:328] unrecognized feature gate: GatewayAPIWithoutOLM Jul 03 11:12:23.717357 ip-10-0-136-0 kubenswrapper[2579]: W0703 11:12:23.716676 2579 feature_gate.go:328] unrecognized feature gate: CRDCompatibilityRequirementOperator Jul 03 11:12:23.717357 ip-10-0-136-0 kubenswrapper[2579]: W0703 11:12:23.716680 2579 feature_gate.go:328] unrecognized feature gate: MachineAPIOperatorDisableMachineHealthCheckController Jul 03 11:12:23.717357 ip-10-0-136-0 kubenswrapper[2579]: W0703 11:12:23.716684 2579 feature_gate.go:328] unrecognized feature gate: Example Jul 03 11:12:23.717357 ip-10-0-136-0 kubenswrapper[2579]: W0703 11:12:23.716693 2579 feature_gate.go:328] unrecognized feature gate: HighlyAvailableArbiter Jul 03 11:12:23.717357 ip-10-0-136-0 kubenswrapper[2579]: W0703 11:12:23.716696 2579 feature_gate.go:328] unrecognized feature gate: MultiArchInstallAzure Jul 03 11:12:23.717357 ip-10-0-136-0 kubenswrapper[2579]: W0703 11:12:23.716699 2579 feature_gate.go:328] unrecognized feature gate: AWSDualStackInstall Jul 03 11:12:23.717357 ip-10-0-136-0 kubenswrapper[2579]: W0703 11:12:23.716704 2579 feature_gate.go:328] unrecognized feature gate: ExternalOIDC Jul 03 11:12:23.717836 ip-10-0-136-0 kubenswrapper[2579]: W0703 11:12:23.716708 2579 feature_gate.go:349] Setting deprecated feature gate KMSv1=true. It will be removed in a future release. Jul 03 11:12:23.717836 ip-10-0-136-0 kubenswrapper[2579]: W0703 11:12:23.716711 2579 feature_gate.go:328] unrecognized feature gate: BuildCSIVolumes Jul 03 11:12:23.717836 ip-10-0-136-0 kubenswrapper[2579]: W0703 11:12:23.716714 2579 feature_gate.go:328] unrecognized feature gate: GatewayAPIController Jul 03 11:12:23.717836 ip-10-0-136-0 kubenswrapper[2579]: W0703 11:12:23.716717 2579 feature_gate.go:328] unrecognized feature gate: MachineAPIMigration Jul 03 11:12:23.717836 ip-10-0-136-0 kubenswrapper[2579]: W0703 11:12:23.716719 2579 feature_gate.go:328] unrecognized feature gate: AzureDualStackInstall Jul 03 11:12:23.717836 ip-10-0-136-0 kubenswrapper[2579]: W0703 11:12:23.716722 2579 feature_gate.go:328] unrecognized feature gate: InsightsOnDemandDataGather Jul 03 11:12:23.717836 ip-10-0-136-0 kubenswrapper[2579]: W0703 11:12:23.716725 2579 feature_gate.go:328] unrecognized feature gate: AlibabaPlatform Jul 03 11:12:23.717836 ip-10-0-136-0 kubenswrapper[2579]: W0703 11:12:23.716727 2579 feature_gate.go:328] unrecognized feature gate: UpgradeStatus Jul 03 11:12:23.717836 ip-10-0-136-0 kubenswrapper[2579]: W0703 11:12:23.716730 2579 feature_gate.go:328] unrecognized feature gate: AzureClusterHostedDNSInstall Jul 03 11:12:23.717836 ip-10-0-136-0 kubenswrapper[2579]: W0703 11:12:23.716733 2579 feature_gate.go:328] unrecognized feature gate: OVNObservability Jul 03 11:12:23.717836 ip-10-0-136-0 kubenswrapper[2579]: W0703 11:12:23.716735 2579 feature_gate.go:328] unrecognized feature gate: NetworkDiagnosticsConfig Jul 03 11:12:23.717836 ip-10-0-136-0 kubenswrapper[2579]: W0703 11:12:23.716740 2579 feature_gate.go:328] unrecognized feature gate: PreconfiguredUDNAddresses Jul 03 11:12:23.717836 ip-10-0-136-0 kubenswrapper[2579]: W0703 11:12:23.716743 2579 feature_gate.go:328] unrecognized feature gate: VolumeGroupSnapshot Jul 03 11:12:23.717836 ip-10-0-136-0 kubenswrapper[2579]: W0703 11:12:23.716745 2579 feature_gate.go:328] unrecognized feature gate: GCPCustomAPIEndpoints Jul 03 11:12:23.717836 ip-10-0-136-0 kubenswrapper[2579]: W0703 11:12:23.716748 2579 feature_gate.go:328] unrecognized feature gate: NewOLMOwnSingleNamespace Jul 03 11:12:23.717836 ip-10-0-136-0 kubenswrapper[2579]: W0703 11:12:23.716750 2579 feature_gate.go:328] unrecognized feature gate: AWSServiceLBNetworkSecurityGroup Jul 03 11:12:23.717836 ip-10-0-136-0 kubenswrapper[2579]: W0703 11:12:23.716753 2579 feature_gate.go:328] unrecognized feature gate: ClusterAPIMachineManagementVSphere Jul 03 11:12:23.717836 ip-10-0-136-0 kubenswrapper[2579]: W0703 11:12:23.716755 2579 feature_gate.go:328] unrecognized feature gate: AWSDedicatedHosts Jul 03 11:12:23.717836 ip-10-0-136-0 kubenswrapper[2579]: W0703 11:12:23.716758 2579 feature_gate.go:328] unrecognized feature gate: ShortCertRotation Jul 03 11:12:23.717836 ip-10-0-136-0 kubenswrapper[2579]: W0703 11:12:23.716760 2579 feature_gate.go:328] unrecognized feature gate: ProvisioningRequestAvailable Jul 03 11:12:23.718374 ip-10-0-136-0 kubenswrapper[2579]: W0703 11:12:23.716763 2579 feature_gate.go:328] unrecognized feature gate: NetworkLiveMigration Jul 03 11:12:23.718374 ip-10-0-136-0 kubenswrapper[2579]: W0703 11:12:23.716766 2579 feature_gate.go:328] unrecognized feature gate: AdminNetworkPolicy Jul 03 11:12:23.718374 ip-10-0-136-0 kubenswrapper[2579]: W0703 11:12:23.716768 2579 feature_gate.go:328] unrecognized feature gate: ExternalOIDCWithUIDAndExtraClaimMappings Jul 03 11:12:23.718374 ip-10-0-136-0 kubenswrapper[2579]: W0703 11:12:23.716771 2579 feature_gate.go:328] unrecognized feature gate: AzureMultiDisk Jul 03 11:12:23.718374 ip-10-0-136-0 kubenswrapper[2579]: W0703 11:12:23.716776 2579 feature_gate.go:328] unrecognized feature gate: DNSNameResolver Jul 03 11:12:23.718374 ip-10-0-136-0 kubenswrapper[2579]: W0703 11:12:23.716779 2579 feature_gate.go:328] unrecognized feature gate: RouteAdvertisements Jul 03 11:12:23.718374 ip-10-0-136-0 kubenswrapper[2579]: W0703 11:12:23.716783 2579 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesAzure Jul 03 11:12:23.718374 ip-10-0-136-0 kubenswrapper[2579]: W0703 11:12:23.716787 2579 feature_gate.go:328] unrecognized feature gate: BootcNodeManagement Jul 03 11:12:23.718374 ip-10-0-136-0 kubenswrapper[2579]: W0703 11:12:23.716789 2579 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesCPMS Jul 03 11:12:23.718374 ip-10-0-136-0 kubenswrapper[2579]: W0703 11:12:23.716799 2579 feature_gate.go:328] unrecognized feature gate: SignatureStores Jul 03 11:12:23.718374 ip-10-0-136-0 kubenswrapper[2579]: W0703 11:12:23.716802 2579 feature_gate.go:328] unrecognized feature gate: ImageStreamImportMode Jul 03 11:12:23.718374 ip-10-0-136-0 kubenswrapper[2579]: W0703 11:12:23.716805 2579 feature_gate.go:328] unrecognized feature gate: SigstoreImageVerification Jul 03 11:12:23.718374 ip-10-0-136-0 kubenswrapper[2579]: W0703 11:12:23.716807 2579 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesvSphere Jul 03 11:12:23.718374 ip-10-0-136-0 kubenswrapper[2579]: W0703 11:12:23.716810 2579 feature_gate.go:328] unrecognized feature gate: AzureDedicatedHosts Jul 03 11:12:23.718374 ip-10-0-136-0 kubenswrapper[2579]: W0703 11:12:23.716813 2579 feature_gate.go:328] unrecognized feature gate: GCPClusterHostedDNS Jul 03 11:12:23.718374 ip-10-0-136-0 kubenswrapper[2579]: W0703 11:12:23.716816 2579 feature_gate.go:328] unrecognized feature gate: BootImageSkewEnforcement Jul 03 11:12:23.718374 ip-10-0-136-0 kubenswrapper[2579]: W0703 11:12:23.716821 2579 feature_gate.go:328] unrecognized feature gate: ClusterVersionOperatorConfiguration Jul 03 11:12:23.718374 ip-10-0-136-0 kubenswrapper[2579]: W0703 11:12:23.716824 2579 feature_gate.go:328] unrecognized feature gate: Example2 Jul 03 11:12:23.718374 ip-10-0-136-0 kubenswrapper[2579]: W0703 11:12:23.716826 2579 feature_gate.go:328] unrecognized feature gate: NoRegistryClusterInstall Jul 03 11:12:23.718374 ip-10-0-136-0 kubenswrapper[2579]: W0703 11:12:23.716829 2579 feature_gate.go:328] unrecognized feature gate: AdditionalRoutingCapabilities Jul 03 11:12:23.718892 ip-10-0-136-0 kubenswrapper[2579]: W0703 11:12:23.716832 2579 feature_gate.go:328] unrecognized feature gate: AutomatedEtcdBackup Jul 03 11:12:23.718892 ip-10-0-136-0 kubenswrapper[2579]: W0703 11:12:23.716835 2579 feature_gate.go:328] unrecognized feature gate: MachineConfigNodes Jul 03 11:12:23.718892 ip-10-0-136-0 kubenswrapper[2579]: W0703 11:12:23.716838 2579 feature_gate.go:328] unrecognized feature gate: ImageModeStatusReporting Jul 03 11:12:23.718892 ip-10-0-136-0 kubenswrapper[2579]: W0703 11:12:23.716840 2579 feature_gate.go:328] unrecognized feature gate: NewOLMPreflightPermissionChecks Jul 03 11:12:23.718892 ip-10-0-136-0 kubenswrapper[2579]: W0703 11:12:23.716842 2579 feature_gate.go:328] unrecognized feature gate: GatewayAPI Jul 03 11:12:23.718892 ip-10-0-136-0 kubenswrapper[2579]: W0703 11:12:23.716845 2579 feature_gate.go:328] unrecognized feature gate: NetworkSegmentation Jul 03 11:12:23.718892 ip-10-0-136-0 kubenswrapper[2579]: W0703 11:12:23.716848 2579 feature_gate.go:328] unrecognized feature gate: NewOLMWebhookProviderOpenshiftServiceCA Jul 03 11:12:23.718892 ip-10-0-136-0 kubenswrapper[2579]: W0703 11:12:23.716850 2579 feature_gate.go:328] unrecognized feature gate: ClusterAPIMachineManagement Jul 03 11:12:23.718892 ip-10-0-136-0 kubenswrapper[2579]: W0703 11:12:23.716855 2579 feature_gate.go:328] unrecognized feature gate: GCPCustomAPIEndpointsInstall Jul 03 11:12:23.718892 ip-10-0-136-0 kubenswrapper[2579]: W0703 11:12:23.716857 2579 feature_gate.go:328] unrecognized feature gate: ClusterAPIInstallIBMCloud Jul 03 11:12:23.718892 ip-10-0-136-0 kubenswrapper[2579]: W0703 11:12:23.716860 2579 feature_gate.go:328] unrecognized feature gate: IrreconcilableMachineConfig Jul 03 11:12:23.718892 ip-10-0-136-0 kubenswrapper[2579]: W0703 11:12:23.716863 2579 feature_gate.go:328] unrecognized feature gate: DualReplica Jul 03 11:12:23.718892 ip-10-0-136-0 kubenswrapper[2579]: W0703 11:12:23.716866 2579 feature_gate.go:328] unrecognized feature gate: IngressControllerDynamicConfigurationManager Jul 03 11:12:23.718892 ip-10-0-136-0 kubenswrapper[2579]: W0703 11:12:23.718245 2579 feature_gate.go:328] unrecognized feature gate: KMSEncryption Jul 03 11:12:23.718892 ip-10-0-136-0 kubenswrapper[2579]: W0703 11:12:23.718257 2579 feature_gate.go:328] unrecognized feature gate: AzureClusterHostedDNSInstall Jul 03 11:12:23.718892 ip-10-0-136-0 kubenswrapper[2579]: W0703 11:12:23.718261 2579 feature_gate.go:328] unrecognized feature gate: GCPClusterHostedDNSInstall Jul 03 11:12:23.718892 ip-10-0-136-0 kubenswrapper[2579]: W0703 11:12:23.718264 2579 feature_gate.go:328] unrecognized feature gate: AWSClusterHostedDNSInstall Jul 03 11:12:23.718892 ip-10-0-136-0 kubenswrapper[2579]: W0703 11:12:23.718268 2579 feature_gate.go:328] unrecognized feature gate: RouteAdvertisements Jul 03 11:12:23.718892 ip-10-0-136-0 kubenswrapper[2579]: W0703 11:12:23.718271 2579 feature_gate.go:328] unrecognized feature gate: MetricsCollectionProfiles Jul 03 11:12:23.719418 ip-10-0-136-0 kubenswrapper[2579]: W0703 11:12:23.718274 2579 feature_gate.go:328] unrecognized feature gate: VolumeGroupSnapshot Jul 03 11:12:23.719418 ip-10-0-136-0 kubenswrapper[2579]: W0703 11:12:23.718277 2579 feature_gate.go:328] unrecognized feature gate: CRDCompatibilityRequirementOperator Jul 03 11:12:23.719418 ip-10-0-136-0 kubenswrapper[2579]: W0703 11:12:23.718280 2579 feature_gate.go:328] unrecognized feature gate: GCPDualStackInstall Jul 03 11:12:23.719418 ip-10-0-136-0 kubenswrapper[2579]: W0703 11:12:23.718289 2579 feature_gate.go:328] unrecognized feature gate: MachineAPIMigration Jul 03 11:12:23.719418 ip-10-0-136-0 kubenswrapper[2579]: W0703 11:12:23.718295 2579 feature_gate.go:328] unrecognized feature gate: ConsolePluginContentSecurityPolicy Jul 03 11:12:23.719418 ip-10-0-136-0 kubenswrapper[2579]: W0703 11:12:23.718299 2579 feature_gate.go:328] unrecognized feature gate: PreconfiguredUDNAddresses Jul 03 11:12:23.719418 ip-10-0-136-0 kubenswrapper[2579]: W0703 11:12:23.718301 2579 feature_gate.go:328] unrecognized feature gate: PinnedImages Jul 03 11:12:23.719418 ip-10-0-136-0 kubenswrapper[2579]: W0703 11:12:23.718304 2579 feature_gate.go:328] unrecognized feature gate: AzureMultiDisk Jul 03 11:12:23.719418 ip-10-0-136-0 kubenswrapper[2579]: W0703 11:12:23.718307 2579 feature_gate.go:328] unrecognized feature gate: HighlyAvailableArbiter Jul 03 11:12:23.719418 ip-10-0-136-0 kubenswrapper[2579]: W0703 11:12:23.718309 2579 feature_gate.go:328] unrecognized feature gate: OnPremDNSRecords Jul 03 11:12:23.719418 ip-10-0-136-0 kubenswrapper[2579]: W0703 11:12:23.718312 2579 feature_gate.go:328] unrecognized feature gate: MachineAPIOperatorDisableMachineHealthCheckController Jul 03 11:12:23.719418 ip-10-0-136-0 kubenswrapper[2579]: W0703 11:12:23.718315 2579 feature_gate.go:328] unrecognized feature gate: AutomatedEtcdBackup Jul 03 11:12:23.719418 ip-10-0-136-0 kubenswrapper[2579]: W0703 11:12:23.718318 2579 feature_gate.go:328] unrecognized feature gate: GCPClusterHostedDNS Jul 03 11:12:23.719418 ip-10-0-136-0 kubenswrapper[2579]: W0703 11:12:23.718321 2579 feature_gate.go:328] unrecognized feature gate: NewOLMPreflightPermissionChecks Jul 03 11:12:23.719418 ip-10-0-136-0 kubenswrapper[2579]: W0703 11:12:23.718323 2579 feature_gate.go:328] unrecognized feature gate: AzureWorkloadIdentity Jul 03 11:12:23.719418 ip-10-0-136-0 kubenswrapper[2579]: W0703 11:12:23.718326 2579 feature_gate.go:328] unrecognized feature gate: BootcNodeManagement Jul 03 11:12:23.719418 ip-10-0-136-0 kubenswrapper[2579]: W0703 11:12:23.718331 2579 feature_gate.go:328] unrecognized feature gate: OSStreams Jul 03 11:12:23.719418 ip-10-0-136-0 kubenswrapper[2579]: W0703 11:12:23.718334 2579 feature_gate.go:328] unrecognized feature gate: GatewayAPIController Jul 03 11:12:23.719418 ip-10-0-136-0 kubenswrapper[2579]: W0703 11:12:23.718336 2579 feature_gate.go:328] unrecognized feature gate: AWSDualStackInstall Jul 03 11:12:23.719418 ip-10-0-136-0 kubenswrapper[2579]: W0703 11:12:23.718340 2579 feature_gate.go:328] unrecognized feature gate: ExternalSnapshotMetadata Jul 03 11:12:23.720032 ip-10-0-136-0 kubenswrapper[2579]: W0703 11:12:23.718343 2579 feature_gate.go:328] unrecognized feature gate: ProvisioningRequestAvailable Jul 03 11:12:23.720032 ip-10-0-136-0 kubenswrapper[2579]: W0703 11:12:23.718345 2579 feature_gate.go:328] unrecognized feature gate: CPMSMachineNamePrefix Jul 03 11:12:23.720032 ip-10-0-136-0 kubenswrapper[2579]: W0703 11:12:23.718348 2579 feature_gate.go:328] unrecognized feature gate: NewOLMBoxCutterRuntime Jul 03 11:12:23.720032 ip-10-0-136-0 kubenswrapper[2579]: W0703 11:12:23.718351 2579 feature_gate.go:328] unrecognized feature gate: NewOLMOwnSingleNamespace Jul 03 11:12:23.720032 ip-10-0-136-0 kubenswrapper[2579]: W0703 11:12:23.718354 2579 feature_gate.go:328] unrecognized feature gate: ClusterAPIInstall Jul 03 11:12:23.720032 ip-10-0-136-0 kubenswrapper[2579]: W0703 11:12:23.718356 2579 feature_gate.go:328] unrecognized feature gate: MixedCPUsAllocation Jul 03 11:12:23.720032 ip-10-0-136-0 kubenswrapper[2579]: W0703 11:12:23.718359 2579 feature_gate.go:328] unrecognized feature gate: AzureDualStackInstall Jul 03 11:12:23.720032 ip-10-0-136-0 kubenswrapper[2579]: W0703 11:12:23.718362 2579 feature_gate.go:328] unrecognized feature gate: AzureDedicatedHosts Jul 03 11:12:23.720032 ip-10-0-136-0 kubenswrapper[2579]: W0703 11:12:23.718365 2579 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesAzure Jul 03 11:12:23.720032 ip-10-0-136-0 kubenswrapper[2579]: W0703 11:12:23.718370 2579 feature_gate.go:328] unrecognized feature gate: ClusterAPIMachineManagement Jul 03 11:12:23.720032 ip-10-0-136-0 kubenswrapper[2579]: W0703 11:12:23.718373 2579 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesCPMS Jul 03 11:12:23.720032 ip-10-0-136-0 kubenswrapper[2579]: W0703 11:12:23.718376 2579 feature_gate.go:328] unrecognized feature gate: IrreconcilableMachineConfig Jul 03 11:12:23.720032 ip-10-0-136-0 kubenswrapper[2579]: W0703 11:12:23.718378 2579 feature_gate.go:328] unrecognized feature gate: HyperShiftOnlyDynamicResourceAllocation Jul 03 11:12:23.720032 ip-10-0-136-0 kubenswrapper[2579]: W0703 11:12:23.718381 2579 feature_gate.go:328] unrecognized feature gate: UpgradeStatus Jul 03 11:12:23.720032 ip-10-0-136-0 kubenswrapper[2579]: W0703 11:12:23.718384 2579 feature_gate.go:328] unrecognized feature gate: VSphereHostVMGroupZonal Jul 03 11:12:23.720032 ip-10-0-136-0 kubenswrapper[2579]: W0703 11:12:23.718387 2579 feature_gate.go:328] unrecognized feature gate: ManagedBootImages Jul 03 11:12:23.720032 ip-10-0-136-0 kubenswrapper[2579]: W0703 11:12:23.718396 2579 feature_gate.go:328] unrecognized feature gate: ImageModeStatusReporting Jul 03 11:12:23.720032 ip-10-0-136-0 kubenswrapper[2579]: W0703 11:12:23.718399 2579 feature_gate.go:328] unrecognized feature gate: VSphereMixedNodeEnv Jul 03 11:12:23.720032 ip-10-0-136-0 kubenswrapper[2579]: W0703 11:12:23.718402 2579 feature_gate.go:328] unrecognized feature gate: Example2 Jul 03 11:12:23.720032 ip-10-0-136-0 kubenswrapper[2579]: W0703 11:12:23.718404 2579 feature_gate.go:328] unrecognized feature gate: AdditionalRoutingCapabilities Jul 03 11:12:23.721016 ip-10-0-136-0 kubenswrapper[2579]: W0703 11:12:23.718407 2579 feature_gate.go:328] unrecognized feature gate: GatewayAPI Jul 03 11:12:23.721016 ip-10-0-136-0 kubenswrapper[2579]: W0703 11:12:23.718412 2579 feature_gate.go:328] unrecognized feature gate: AlibabaPlatform Jul 03 11:12:23.721016 ip-10-0-136-0 kubenswrapper[2579]: W0703 11:12:23.718415 2579 feature_gate.go:328] unrecognized feature gate: DyanmicServiceEndpointIBMCloud Jul 03 11:12:23.721016 ip-10-0-136-0 kubenswrapper[2579]: W0703 11:12:23.718419 2579 feature_gate.go:349] Setting deprecated feature gate KMSv1=true. It will be removed in a future release. Jul 03 11:12:23.721016 ip-10-0-136-0 kubenswrapper[2579]: W0703 11:12:23.718423 2579 feature_gate.go:328] unrecognized feature gate: ClusterMonitoringConfig Jul 03 11:12:23.721016 ip-10-0-136-0 kubenswrapper[2579]: W0703 11:12:23.718426 2579 feature_gate.go:328] unrecognized feature gate: BuildCSIVolumes Jul 03 11:12:23.721016 ip-10-0-136-0 kubenswrapper[2579]: W0703 11:12:23.718428 2579 feature_gate.go:328] unrecognized feature gate: VSphereMultiDisk Jul 03 11:12:23.721016 ip-10-0-136-0 kubenswrapper[2579]: W0703 11:12:23.718431 2579 feature_gate.go:328] unrecognized feature gate: GCPCustomAPIEndpointsInstall Jul 03 11:12:23.721016 ip-10-0-136-0 kubenswrapper[2579]: W0703 11:12:23.718434 2579 feature_gate.go:328] unrecognized feature gate: MachineConfigNodes Jul 03 11:12:23.721016 ip-10-0-136-0 kubenswrapper[2579]: W0703 11:12:23.718436 2579 feature_gate.go:328] unrecognized feature gate: GCPCustomAPIEndpoints Jul 03 11:12:23.721016 ip-10-0-136-0 kubenswrapper[2579]: W0703 11:12:23.718440 2579 feature_gate.go:328] unrecognized feature gate: NewOLM Jul 03 11:12:23.721016 ip-10-0-136-0 kubenswrapper[2579]: W0703 11:12:23.718448 2579 feature_gate.go:328] unrecognized feature gate: ExternalOIDC Jul 03 11:12:23.721016 ip-10-0-136-0 kubenswrapper[2579]: W0703 11:12:23.718453 2579 feature_gate.go:328] unrecognized feature gate: Example Jul 03 11:12:23.721016 ip-10-0-136-0 kubenswrapper[2579]: W0703 11:12:23.718456 2579 feature_gate.go:328] unrecognized feature gate: NutanixMultiSubnets Jul 03 11:12:23.721016 ip-10-0-136-0 kubenswrapper[2579]: W0703 11:12:23.718459 2579 feature_gate.go:328] unrecognized feature gate: DNSNameResolver Jul 03 11:12:23.721016 ip-10-0-136-0 kubenswrapper[2579]: W0703 11:12:23.718462 2579 feature_gate.go:328] unrecognized feature gate: OVNObservability Jul 03 11:12:23.721016 ip-10-0-136-0 kubenswrapper[2579]: W0703 11:12:23.718465 2579 feature_gate.go:328] unrecognized feature gate: NoRegistryClusterInstall Jul 03 11:12:23.721016 ip-10-0-136-0 kubenswrapper[2579]: W0703 11:12:23.718467 2579 feature_gate.go:328] unrecognized feature gate: NewOLMWebhookProviderOpenshiftServiceCA Jul 03 11:12:23.721016 ip-10-0-136-0 kubenswrapper[2579]: W0703 11:12:23.718470 2579 feature_gate.go:328] unrecognized feature gate: InsightsOnDemandDataGather Jul 03 11:12:23.721016 ip-10-0-136-0 kubenswrapper[2579]: W0703 11:12:23.718473 2579 feature_gate.go:328] unrecognized feature gate: SigstoreImageVerification Jul 03 11:12:23.721910 ip-10-0-136-0 kubenswrapper[2579]: W0703 11:12:23.718476 2579 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesAWS Jul 03 11:12:23.721910 ip-10-0-136-0 kubenswrapper[2579]: W0703 11:12:23.718479 2579 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesvSphere Jul 03 11:12:23.721910 ip-10-0-136-0 kubenswrapper[2579]: W0703 11:12:23.718481 2579 feature_gate.go:328] unrecognized feature gate: AdminNetworkPolicy Jul 03 11:12:23.721910 ip-10-0-136-0 kubenswrapper[2579]: W0703 11:12:23.718484 2579 feature_gate.go:328] unrecognized feature gate: KMSEncryptionProvider Jul 03 11:12:23.721910 ip-10-0-136-0 kubenswrapper[2579]: W0703 11:12:23.718486 2579 feature_gate.go:328] unrecognized feature gate: VSphereMultiNetworks Jul 03 11:12:23.721910 ip-10-0-136-0 kubenswrapper[2579]: W0703 11:12:23.718492 2579 feature_gate.go:328] unrecognized feature gate: InsightsConfig Jul 03 11:12:23.721910 ip-10-0-136-0 kubenswrapper[2579]: W0703 11:12:23.718494 2579 feature_gate.go:328] unrecognized feature gate: IngressControllerDynamicConfigurationManager Jul 03 11:12:23.721910 ip-10-0-136-0 kubenswrapper[2579]: W0703 11:12:23.718497 2579 feature_gate.go:328] unrecognized feature gate: OpenShiftPodSecurityAdmission Jul 03 11:12:23.721910 ip-10-0-136-0 kubenswrapper[2579]: W0703 11:12:23.718499 2579 feature_gate.go:328] unrecognized feature gate: EventTTL Jul 03 11:12:23.721910 ip-10-0-136-0 kubenswrapper[2579]: W0703 11:12:23.718508 2579 feature_gate.go:328] unrecognized feature gate: EtcdBackendQuota Jul 03 11:12:23.721910 ip-10-0-136-0 kubenswrapper[2579]: W0703 11:12:23.718511 2579 feature_gate.go:328] unrecognized feature gate: SigstoreImageVerificationPKI Jul 03 11:12:23.721910 ip-10-0-136-0 kubenswrapper[2579]: W0703 11:12:23.718514 2579 feature_gate.go:328] unrecognized feature gate: BootImageSkewEnforcement Jul 03 11:12:23.721910 ip-10-0-136-0 kubenswrapper[2579]: W0703 11:12:23.718516 2579 feature_gate.go:328] unrecognized feature gate: AWSDedicatedHosts Jul 03 11:12:23.721910 ip-10-0-136-0 kubenswrapper[2579]: W0703 11:12:23.718519 2579 feature_gate.go:328] unrecognized feature gate: ShortCertRotation Jul 03 11:12:23.721910 ip-10-0-136-0 kubenswrapper[2579]: W0703 11:12:23.718521 2579 feature_gate.go:328] unrecognized feature gate: ClusterVersionOperatorConfiguration Jul 03 11:12:23.721910 ip-10-0-136-0 kubenswrapper[2579]: W0703 11:12:23.718524 2579 feature_gate.go:328] unrecognized feature gate: ExternalOIDCWithUIDAndExtraClaimMappings Jul 03 11:12:23.721910 ip-10-0-136-0 kubenswrapper[2579]: W0703 11:12:23.718526 2579 feature_gate.go:328] unrecognized feature gate: AWSServiceLBNetworkSecurityGroup Jul 03 11:12:23.721910 ip-10-0-136-0 kubenswrapper[2579]: W0703 11:12:23.718529 2579 feature_gate.go:328] unrecognized feature gate: NewOLMCatalogdAPIV1Metas Jul 03 11:12:23.721910 ip-10-0-136-0 kubenswrapper[2579]: W0703 11:12:23.718534 2579 feature_gate.go:328] unrecognized feature gate: NetworkLiveMigration Jul 03 11:12:23.722508 ip-10-0-136-0 kubenswrapper[2579]: W0703 11:12:23.718536 2579 feature_gate.go:328] unrecognized feature gate: ClusterAPIMachineManagementVSphere Jul 03 11:12:23.722508 ip-10-0-136-0 kubenswrapper[2579]: W0703 11:12:23.718539 2579 feature_gate.go:328] unrecognized feature gate: VSphereConfigurableMaxAllowedBlockVolumesPerNode Jul 03 11:12:23.722508 ip-10-0-136-0 kubenswrapper[2579]: W0703 11:12:23.718543 2579 feature_gate.go:351] Setting GA feature gate ServiceAccountTokenNodeBinding=true. It will be removed in a future release. Jul 03 11:12:23.722508 ip-10-0-136-0 kubenswrapper[2579]: W0703 11:12:23.718547 2579 feature_gate.go:328] unrecognized feature gate: NetworkDiagnosticsConfig Jul 03 11:12:23.722508 ip-10-0-136-0 kubenswrapper[2579]: W0703 11:12:23.718550 2579 feature_gate.go:328] unrecognized feature gate: DualReplica Jul 03 11:12:23.722508 ip-10-0-136-0 kubenswrapper[2579]: W0703 11:12:23.718553 2579 feature_gate.go:328] unrecognized feature gate: MultiArchInstallAzure Jul 03 11:12:23.722508 ip-10-0-136-0 kubenswrapper[2579]: W0703 11:12:23.718555 2579 feature_gate.go:328] unrecognized feature gate: AWSClusterHostedDNS Jul 03 11:12:23.722508 ip-10-0-136-0 kubenswrapper[2579]: W0703 11:12:23.718558 2579 feature_gate.go:328] unrecognized feature gate: ClusterAPIInstallIBMCloud Jul 03 11:12:23.722508 ip-10-0-136-0 kubenswrapper[2579]: W0703 11:12:23.718562 2579 feature_gate.go:328] unrecognized feature gate: GatewayAPIWithoutOLM Jul 03 11:12:23.722508 ip-10-0-136-0 kubenswrapper[2579]: W0703 11:12:23.718566 2579 feature_gate.go:351] Setting GA feature gate VolumeAttributesClass=true. It will be removed in a future release. Jul 03 11:12:23.722508 ip-10-0-136-0 kubenswrapper[2579]: W0703 11:12:23.718569 2579 feature_gate.go:328] unrecognized feature gate: ImageStreamImportMode Jul 03 11:12:23.722508 ip-10-0-136-0 kubenswrapper[2579]: W0703 11:12:23.718574 2579 feature_gate.go:328] unrecognized feature gate: NetworkSegmentation Jul 03 11:12:23.722508 ip-10-0-136-0 kubenswrapper[2579]: W0703 11:12:23.718578 2579 feature_gate.go:328] unrecognized feature gate: SignatureStores Jul 03 11:12:23.722508 ip-10-0-136-0 kubenswrapper[2579]: W0703 11:12:23.718581 2579 feature_gate.go:328] unrecognized feature gate: MultiDiskSetup Jul 03 11:12:23.722508 ip-10-0-136-0 kubenswrapper[2579]: W0703 11:12:23.719371 2579 feature_gate.go:328] unrecognized feature gate: VolumeGroupSnapshot Jul 03 11:12:23.722508 ip-10-0-136-0 kubenswrapper[2579]: W0703 11:12:23.719380 2579 feature_gate.go:328] unrecognized feature gate: BootImageSkewEnforcement Jul 03 11:12:23.722508 ip-10-0-136-0 kubenswrapper[2579]: W0703 11:12:23.719384 2579 feature_gate.go:328] unrecognized feature gate: ClusterAPIMachineManagement Jul 03 11:12:23.722508 ip-10-0-136-0 kubenswrapper[2579]: W0703 11:12:23.719387 2579 feature_gate.go:328] unrecognized feature gate: MetricsCollectionProfiles Jul 03 11:12:23.722508 ip-10-0-136-0 kubenswrapper[2579]: W0703 11:12:23.719390 2579 feature_gate.go:328] unrecognized feature gate: AzureDualStackInstall Jul 03 11:12:23.723105 ip-10-0-136-0 kubenswrapper[2579]: W0703 11:12:23.719393 2579 feature_gate.go:328] unrecognized feature gate: GatewayAPIWithoutOLM Jul 03 11:12:23.723105 ip-10-0-136-0 kubenswrapper[2579]: W0703 11:12:23.719396 2579 feature_gate.go:328] unrecognized feature gate: NewOLMPreflightPermissionChecks Jul 03 11:12:23.723105 ip-10-0-136-0 kubenswrapper[2579]: W0703 11:12:23.719399 2579 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesAWS Jul 03 11:12:23.723105 ip-10-0-136-0 kubenswrapper[2579]: W0703 11:12:23.719408 2579 feature_gate.go:328] unrecognized feature gate: NewOLM Jul 03 11:12:23.723105 ip-10-0-136-0 kubenswrapper[2579]: W0703 11:12:23.719411 2579 feature_gate.go:328] unrecognized feature gate: AWSDedicatedHosts Jul 03 11:12:23.723105 ip-10-0-136-0 kubenswrapper[2579]: W0703 11:12:23.719418 2579 feature_gate.go:351] Setting GA feature gate VolumeAttributesClass=true. It will be removed in a future release. Jul 03 11:12:23.723105 ip-10-0-136-0 kubenswrapper[2579]: W0703 11:12:23.719421 2579 feature_gate.go:328] unrecognized feature gate: ClusterAPIInstall Jul 03 11:12:23.723105 ip-10-0-136-0 kubenswrapper[2579]: W0703 11:12:23.719425 2579 feature_gate.go:328] unrecognized feature gate: BuildCSIVolumes Jul 03 11:12:23.723105 ip-10-0-136-0 kubenswrapper[2579]: W0703 11:12:23.719428 2579 feature_gate.go:328] unrecognized feature gate: SigstoreImageVerificationPKI Jul 03 11:12:23.723105 ip-10-0-136-0 kubenswrapper[2579]: W0703 11:12:23.719431 2579 feature_gate.go:328] unrecognized feature gate: Example Jul 03 11:12:23.723105 ip-10-0-136-0 kubenswrapper[2579]: W0703 11:12:23.719434 2579 feature_gate.go:328] unrecognized feature gate: NutanixMultiSubnets Jul 03 11:12:23.723105 ip-10-0-136-0 kubenswrapper[2579]: W0703 11:12:23.719436 2579 feature_gate.go:328] unrecognized feature gate: ClusterAPIMachineManagementVSphere Jul 03 11:12:23.723105 ip-10-0-136-0 kubenswrapper[2579]: W0703 11:12:23.719439 2579 feature_gate.go:328] unrecognized feature gate: AzureClusterHostedDNSInstall Jul 03 11:12:23.723105 ip-10-0-136-0 kubenswrapper[2579]: W0703 11:12:23.719442 2579 feature_gate.go:328] unrecognized feature gate: VSphereConfigurableMaxAllowedBlockVolumesPerNode Jul 03 11:12:23.723105 ip-10-0-136-0 kubenswrapper[2579]: W0703 11:12:23.719445 2579 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesCPMS Jul 03 11:12:23.723105 ip-10-0-136-0 kubenswrapper[2579]: W0703 11:12:23.719447 2579 feature_gate.go:328] unrecognized feature gate: VSphereMixedNodeEnv Jul 03 11:12:23.723105 ip-10-0-136-0 kubenswrapper[2579]: W0703 11:12:23.719450 2579 feature_gate.go:328] unrecognized feature gate: VSphereMultiNetworks Jul 03 11:12:23.723105 ip-10-0-136-0 kubenswrapper[2579]: W0703 11:12:23.719452 2579 feature_gate.go:328] unrecognized feature gate: GCPCustomAPIEndpointsInstall Jul 03 11:12:23.723105 ip-10-0-136-0 kubenswrapper[2579]: W0703 11:12:23.719457 2579 feature_gate.go:328] unrecognized feature gate: AWSServiceLBNetworkSecurityGroup Jul 03 11:12:23.723703 ip-10-0-136-0 kubenswrapper[2579]: W0703 11:12:23.719460 2579 feature_gate.go:328] unrecognized feature gate: AdditionalRoutingCapabilities Jul 03 11:12:23.723703 ip-10-0-136-0 kubenswrapper[2579]: W0703 11:12:23.719463 2579 feature_gate.go:328] unrecognized feature gate: SignatureStores Jul 03 11:12:23.723703 ip-10-0-136-0 kubenswrapper[2579]: W0703 11:12:23.719468 2579 feature_gate.go:328] unrecognized feature gate: ClusterMonitoringConfig Jul 03 11:12:23.723703 ip-10-0-136-0 kubenswrapper[2579]: W0703 11:12:23.719474 2579 feature_gate.go:328] unrecognized feature gate: MultiDiskSetup Jul 03 11:12:23.723703 ip-10-0-136-0 kubenswrapper[2579]: W0703 11:12:23.719477 2579 feature_gate.go:328] unrecognized feature gate: KMSEncryption Jul 03 11:12:23.723703 ip-10-0-136-0 kubenswrapper[2579]: W0703 11:12:23.719481 2579 feature_gate.go:351] Setting GA feature gate ServiceAccountTokenNodeBinding=true. It will be removed in a future release. Jul 03 11:12:23.723703 ip-10-0-136-0 kubenswrapper[2579]: W0703 11:12:23.719486 2579 feature_gate.go:328] unrecognized feature gate: GatewayAPIController Jul 03 11:12:23.723703 ip-10-0-136-0 kubenswrapper[2579]: W0703 11:12:23.719489 2579 feature_gate.go:328] unrecognized feature gate: HyperShiftOnlyDynamicResourceAllocation Jul 03 11:12:23.723703 ip-10-0-136-0 kubenswrapper[2579]: W0703 11:12:23.719492 2579 feature_gate.go:328] unrecognized feature gate: CRDCompatibilityRequirementOperator Jul 03 11:12:23.723703 ip-10-0-136-0 kubenswrapper[2579]: W0703 11:12:23.719495 2579 feature_gate.go:328] unrecognized feature gate: EventTTL Jul 03 11:12:23.723703 ip-10-0-136-0 kubenswrapper[2579]: W0703 11:12:23.719500 2579 feature_gate.go:328] unrecognized feature gate: PreconfiguredUDNAddresses Jul 03 11:12:23.723703 ip-10-0-136-0 kubenswrapper[2579]: W0703 11:12:23.719503 2579 feature_gate.go:328] unrecognized feature gate: DualReplica Jul 03 11:12:23.723703 ip-10-0-136-0 kubenswrapper[2579]: W0703 11:12:23.719505 2579 feature_gate.go:328] unrecognized feature gate: NetworkDiagnosticsConfig Jul 03 11:12:23.723703 ip-10-0-136-0 kubenswrapper[2579]: W0703 11:12:23.719508 2579 feature_gate.go:328] unrecognized feature gate: AWSClusterHostedDNSInstall Jul 03 11:12:23.723703 ip-10-0-136-0 kubenswrapper[2579]: W0703 11:12:23.719521 2579 feature_gate.go:328] unrecognized feature gate: RouteAdvertisements Jul 03 11:12:23.723703 ip-10-0-136-0 kubenswrapper[2579]: W0703 11:12:23.719524 2579 feature_gate.go:328] unrecognized feature gate: AdminNetworkPolicy Jul 03 11:12:23.723703 ip-10-0-136-0 kubenswrapper[2579]: W0703 11:12:23.719526 2579 feature_gate.go:328] unrecognized feature gate: IrreconcilableMachineConfig Jul 03 11:12:23.723703 ip-10-0-136-0 kubenswrapper[2579]: W0703 11:12:23.719538 2579 feature_gate.go:328] unrecognized feature gate: OVNObservability Jul 03 11:12:23.723703 ip-10-0-136-0 kubenswrapper[2579]: W0703 11:12:23.719541 2579 feature_gate.go:328] unrecognized feature gate: VSphereMultiDisk Jul 03 11:12:23.723703 ip-10-0-136-0 kubenswrapper[2579]: W0703 11:12:23.719544 2579 feature_gate.go:328] unrecognized feature gate: ImageModeStatusReporting Jul 03 11:12:23.724422 ip-10-0-136-0 kubenswrapper[2579]: W0703 11:12:23.719547 2579 feature_gate.go:328] unrecognized feature gate: DyanmicServiceEndpointIBMCloud Jul 03 11:12:23.724422 ip-10-0-136-0 kubenswrapper[2579]: W0703 11:12:23.719550 2579 feature_gate.go:328] unrecognized feature gate: OSStreams Jul 03 11:12:23.724422 ip-10-0-136-0 kubenswrapper[2579]: W0703 11:12:23.719555 2579 feature_gate.go:328] unrecognized feature gate: CPMSMachineNamePrefix Jul 03 11:12:23.724422 ip-10-0-136-0 kubenswrapper[2579]: W0703 11:12:23.719557 2579 feature_gate.go:328] unrecognized feature gate: NewOLMWebhookProviderOpenshiftServiceCA Jul 03 11:12:23.724422 ip-10-0-136-0 kubenswrapper[2579]: W0703 11:12:23.719560 2579 feature_gate.go:328] unrecognized feature gate: ClusterAPIInstallIBMCloud Jul 03 11:12:23.724422 ip-10-0-136-0 kubenswrapper[2579]: W0703 11:12:23.719562 2579 feature_gate.go:328] unrecognized feature gate: AzureWorkloadIdentity Jul 03 11:12:23.724422 ip-10-0-136-0 kubenswrapper[2579]: W0703 11:12:23.719565 2579 feature_gate.go:328] unrecognized feature gate: ManagedBootImages Jul 03 11:12:23.724422 ip-10-0-136-0 kubenswrapper[2579]: W0703 11:12:23.719568 2579 feature_gate.go:328] unrecognized feature gate: NewOLMCatalogdAPIV1Metas Jul 03 11:12:23.724422 ip-10-0-136-0 kubenswrapper[2579]: W0703 11:12:23.719570 2579 feature_gate.go:328] unrecognized feature gate: NetworkLiveMigration Jul 03 11:12:23.724422 ip-10-0-136-0 kubenswrapper[2579]: W0703 11:12:23.719573 2579 feature_gate.go:328] unrecognized feature gate: MixedCPUsAllocation Jul 03 11:12:23.724422 ip-10-0-136-0 kubenswrapper[2579]: W0703 11:12:23.719576 2579 feature_gate.go:328] unrecognized feature gate: AutomatedEtcdBackup Jul 03 11:12:23.724422 ip-10-0-136-0 kubenswrapper[2579]: W0703 11:12:23.719578 2579 feature_gate.go:328] unrecognized feature gate: AWSClusterHostedDNS Jul 03 11:12:23.724422 ip-10-0-136-0 kubenswrapper[2579]: W0703 11:12:23.719581 2579 feature_gate.go:328] unrecognized feature gate: NewOLMOwnSingleNamespace Jul 03 11:12:23.724422 ip-10-0-136-0 kubenswrapper[2579]: W0703 11:12:23.719584 2579 feature_gate.go:328] unrecognized feature gate: VSphereHostVMGroupZonal Jul 03 11:12:23.724422 ip-10-0-136-0 kubenswrapper[2579]: W0703 11:12:23.719587 2579 feature_gate.go:328] unrecognized feature gate: OpenShiftPodSecurityAdmission Jul 03 11:12:23.724422 ip-10-0-136-0 kubenswrapper[2579]: W0703 11:12:23.719592 2579 feature_gate.go:328] unrecognized feature gate: MachineAPIOperatorDisableMachineHealthCheckController Jul 03 11:12:23.724422 ip-10-0-136-0 kubenswrapper[2579]: W0703 11:12:23.719594 2579 feature_gate.go:328] unrecognized feature gate: ConsolePluginContentSecurityPolicy Jul 03 11:12:23.724422 ip-10-0-136-0 kubenswrapper[2579]: W0703 11:12:23.719597 2579 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesvSphere Jul 03 11:12:23.724422 ip-10-0-136-0 kubenswrapper[2579]: W0703 11:12:23.719599 2579 feature_gate.go:328] unrecognized feature gate: PinnedImages Jul 03 11:12:23.724901 ip-10-0-136-0 kubenswrapper[2579]: W0703 11:12:23.719602 2579 feature_gate.go:328] unrecognized feature gate: AlibabaPlatform Jul 03 11:12:23.724901 ip-10-0-136-0 kubenswrapper[2579]: W0703 11:12:23.719604 2579 feature_gate.go:328] unrecognized feature gate: ShortCertRotation Jul 03 11:12:23.724901 ip-10-0-136-0 kubenswrapper[2579]: W0703 11:12:23.719607 2579 feature_gate.go:328] unrecognized feature gate: ProvisioningRequestAvailable Jul 03 11:12:23.724901 ip-10-0-136-0 kubenswrapper[2579]: W0703 11:12:23.719609 2579 feature_gate.go:328] unrecognized feature gate: NewOLMBoxCutterRuntime Jul 03 11:12:23.724901 ip-10-0-136-0 kubenswrapper[2579]: W0703 11:12:23.719612 2579 feature_gate.go:328] unrecognized feature gate: GatewayAPI Jul 03 11:12:23.724901 ip-10-0-136-0 kubenswrapper[2579]: W0703 11:12:23.719614 2579 feature_gate.go:328] unrecognized feature gate: GCPCustomAPIEndpoints Jul 03 11:12:23.724901 ip-10-0-136-0 kubenswrapper[2579]: W0703 11:12:23.719617 2579 feature_gate.go:328] unrecognized feature gate: HighlyAvailableArbiter Jul 03 11:12:23.724901 ip-10-0-136-0 kubenswrapper[2579]: W0703 11:12:23.719620 2579 feature_gate.go:328] unrecognized feature gate: ClusterVersionOperatorConfiguration Jul 03 11:12:23.724901 ip-10-0-136-0 kubenswrapper[2579]: W0703 11:12:23.719622 2579 feature_gate.go:328] unrecognized feature gate: AzureDedicatedHosts Jul 03 11:12:23.724901 ip-10-0-136-0 kubenswrapper[2579]: W0703 11:12:23.719627 2579 feature_gate.go:328] unrecognized feature gate: NoRegistryClusterInstall Jul 03 11:12:23.724901 ip-10-0-136-0 kubenswrapper[2579]: W0703 11:12:23.719630 2579 feature_gate.go:328] unrecognized feature gate: GCPClusterHostedDNS Jul 03 11:12:23.724901 ip-10-0-136-0 kubenswrapper[2579]: W0703 11:12:23.719638 2579 feature_gate.go:328] unrecognized feature gate: IngressControllerDynamicConfigurationManager Jul 03 11:12:23.724901 ip-10-0-136-0 kubenswrapper[2579]: W0703 11:12:23.719641 2579 feature_gate.go:328] unrecognized feature gate: DNSNameResolver Jul 03 11:12:23.724901 ip-10-0-136-0 kubenswrapper[2579]: W0703 11:12:23.719643 2579 feature_gate.go:328] unrecognized feature gate: NetworkSegmentation Jul 03 11:12:23.724901 ip-10-0-136-0 kubenswrapper[2579]: W0703 11:12:23.719646 2579 feature_gate.go:328] unrecognized feature gate: MachineConfigNodes Jul 03 11:12:23.724901 ip-10-0-136-0 kubenswrapper[2579]: W0703 11:12:23.719648 2579 feature_gate.go:328] unrecognized feature gate: MachineAPIMigration Jul 03 11:12:23.724901 ip-10-0-136-0 kubenswrapper[2579]: W0703 11:12:23.719651 2579 feature_gate.go:328] unrecognized feature gate: UpgradeStatus Jul 03 11:12:23.724901 ip-10-0-136-0 kubenswrapper[2579]: W0703 11:12:23.719653 2579 feature_gate.go:328] unrecognized feature gate: EtcdBackendQuota Jul 03 11:12:23.724901 ip-10-0-136-0 kubenswrapper[2579]: W0703 11:12:23.719657 2579 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesAzure Jul 03 11:12:23.724901 ip-10-0-136-0 kubenswrapper[2579]: W0703 11:12:23.719660 2579 feature_gate.go:328] unrecognized feature gate: GCPDualStackInstall Jul 03 11:12:23.725515 ip-10-0-136-0 kubenswrapper[2579]: W0703 11:12:23.719662 2579 feature_gate.go:328] unrecognized feature gate: OnPremDNSRecords Jul 03 11:12:23.725515 ip-10-0-136-0 kubenswrapper[2579]: W0703 11:12:23.719667 2579 feature_gate.go:328] unrecognized feature gate: BootcNodeManagement Jul 03 11:12:23.725515 ip-10-0-136-0 kubenswrapper[2579]: W0703 11:12:23.719670 2579 feature_gate.go:328] unrecognized feature gate: ExternalSnapshotMetadata Jul 03 11:12:23.725515 ip-10-0-136-0 kubenswrapper[2579]: W0703 11:12:23.719672 2579 feature_gate.go:328] unrecognized feature gate: SigstoreImageVerification Jul 03 11:12:23.725515 ip-10-0-136-0 kubenswrapper[2579]: W0703 11:12:23.719678 2579 feature_gate.go:328] unrecognized feature gate: AzureMultiDisk Jul 03 11:12:23.725515 ip-10-0-136-0 kubenswrapper[2579]: W0703 11:12:23.719680 2579 feature_gate.go:328] unrecognized feature gate: InsightsConfig Jul 03 11:12:23.725515 ip-10-0-136-0 kubenswrapper[2579]: W0703 11:12:23.719683 2579 feature_gate.go:328] unrecognized feature gate: ImageStreamImportMode Jul 03 11:12:23.725515 ip-10-0-136-0 kubenswrapper[2579]: W0703 11:12:23.719685 2579 feature_gate.go:328] unrecognized feature gate: KMSEncryptionProvider Jul 03 11:12:23.725515 ip-10-0-136-0 kubenswrapper[2579]: W0703 11:12:23.719688 2579 feature_gate.go:328] unrecognized feature gate: Example2 Jul 03 11:12:23.725515 ip-10-0-136-0 kubenswrapper[2579]: W0703 11:12:23.719691 2579 feature_gate.go:328] unrecognized feature gate: ExternalOIDCWithUIDAndExtraClaimMappings Jul 03 11:12:23.725515 ip-10-0-136-0 kubenswrapper[2579]: W0703 11:12:23.719695 2579 feature_gate.go:349] Setting deprecated feature gate KMSv1=true. It will be removed in a future release. Jul 03 11:12:23.725515 ip-10-0-136-0 kubenswrapper[2579]: W0703 11:12:23.719698 2579 feature_gate.go:328] unrecognized feature gate: ExternalOIDC Jul 03 11:12:23.725515 ip-10-0-136-0 kubenswrapper[2579]: W0703 11:12:23.719700 2579 feature_gate.go:328] unrecognized feature gate: MultiArchInstallAzure Jul 03 11:12:23.725515 ip-10-0-136-0 kubenswrapper[2579]: W0703 11:12:23.719705 2579 feature_gate.go:328] unrecognized feature gate: AWSDualStackInstall Jul 03 11:12:23.725515 ip-10-0-136-0 kubenswrapper[2579]: W0703 11:12:23.719708 2579 feature_gate.go:328] unrecognized feature gate: GCPClusterHostedDNSInstall Jul 03 11:12:23.725515 ip-10-0-136-0 kubenswrapper[2579]: W0703 11:12:23.719711 2579 feature_gate.go:328] unrecognized feature gate: InsightsOnDemandDataGather Jul 03 11:12:23.725515 ip-10-0-136-0 kubenswrapper[2579]: I0703 11:12:23.720257 2579 flags.go:64] FLAG: --address="0.0.0.0" Jul 03 11:12:23.725515 ip-10-0-136-0 kubenswrapper[2579]: I0703 11:12:23.720295 2579 flags.go:64] FLAG: --allowed-unsafe-sysctls="[]" Jul 03 11:12:23.725515 ip-10-0-136-0 kubenswrapper[2579]: I0703 11:12:23.720317 2579 flags.go:64] FLAG: --anonymous-auth="true" Jul 03 11:12:23.725515 ip-10-0-136-0 kubenswrapper[2579]: I0703 11:12:23.720326 2579 flags.go:64] FLAG: --application-metrics-count-limit="100" Jul 03 11:12:23.725515 ip-10-0-136-0 kubenswrapper[2579]: I0703 11:12:23.720333 2579 flags.go:64] FLAG: --authentication-token-webhook="false" Jul 03 11:12:23.726072 ip-10-0-136-0 kubenswrapper[2579]: I0703 11:12:23.720339 2579 flags.go:64] FLAG: --authentication-token-webhook-cache-ttl="2m0s" Jul 03 11:12:23.726072 ip-10-0-136-0 kubenswrapper[2579]: I0703 11:12:23.720347 2579 flags.go:64] FLAG: --authorization-mode="AlwaysAllow" Jul 03 11:12:23.726072 ip-10-0-136-0 kubenswrapper[2579]: I0703 11:12:23.720355 2579 flags.go:64] FLAG: --authorization-webhook-cache-authorized-ttl="5m0s" Jul 03 11:12:23.726072 ip-10-0-136-0 kubenswrapper[2579]: I0703 11:12:23.720362 2579 flags.go:64] FLAG: --authorization-webhook-cache-unauthorized-ttl="30s" Jul 03 11:12:23.726072 ip-10-0-136-0 kubenswrapper[2579]: I0703 11:12:23.720367 2579 flags.go:64] FLAG: --boot-id-file="/proc/sys/kernel/random/boot_id" Jul 03 11:12:23.726072 ip-10-0-136-0 kubenswrapper[2579]: I0703 11:12:23.720373 2579 flags.go:64] FLAG: --bootstrap-kubeconfig="/etc/kubernetes/kubeconfig" Jul 03 11:12:23.726072 ip-10-0-136-0 kubenswrapper[2579]: I0703 11:12:23.720378 2579 flags.go:64] FLAG: --cert-dir="/var/lib/kubelet/pki" Jul 03 11:12:23.726072 ip-10-0-136-0 kubenswrapper[2579]: I0703 11:12:23.720384 2579 flags.go:64] FLAG: --cgroup-driver="cgroupfs" Jul 03 11:12:23.726072 ip-10-0-136-0 kubenswrapper[2579]: I0703 11:12:23.720392 2579 flags.go:64] FLAG: --cgroup-root="" Jul 03 11:12:23.726072 ip-10-0-136-0 kubenswrapper[2579]: I0703 11:12:23.720401 2579 flags.go:64] FLAG: --cgroups-per-qos="true" Jul 03 11:12:23.726072 ip-10-0-136-0 kubenswrapper[2579]: I0703 11:12:23.720409 2579 flags.go:64] FLAG: --client-ca-file="" Jul 03 11:12:23.726072 ip-10-0-136-0 kubenswrapper[2579]: I0703 11:12:23.720415 2579 flags.go:64] FLAG: --cloud-provider="external" Jul 03 11:12:23.726072 ip-10-0-136-0 kubenswrapper[2579]: I0703 11:12:23.720420 2579 flags.go:64] FLAG: --cluster-dns="[]" Jul 03 11:12:23.726072 ip-10-0-136-0 kubenswrapper[2579]: I0703 11:12:23.720433 2579 flags.go:64] FLAG: --cluster-domain="" Jul 03 11:12:23.726072 ip-10-0-136-0 kubenswrapper[2579]: I0703 11:12:23.720441 2579 flags.go:64] FLAG: --config="/etc/kubernetes/kubelet.conf" Jul 03 11:12:23.726072 ip-10-0-136-0 kubenswrapper[2579]: I0703 11:12:23.720449 2579 flags.go:64] FLAG: --config-dir="/etc/openshift/kubelet.conf.d" Jul 03 11:12:23.726072 ip-10-0-136-0 kubenswrapper[2579]: I0703 11:12:23.720456 2579 flags.go:64] FLAG: --container-hints="/etc/cadvisor/container_hints.json" Jul 03 11:12:23.726072 ip-10-0-136-0 kubenswrapper[2579]: I0703 11:12:23.720463 2579 flags.go:64] FLAG: --container-log-max-files="5" Jul 03 11:12:23.726072 ip-10-0-136-0 kubenswrapper[2579]: I0703 11:12:23.720470 2579 flags.go:64] FLAG: --container-log-max-size="10Mi" Jul 03 11:12:23.726072 ip-10-0-136-0 kubenswrapper[2579]: I0703 11:12:23.720475 2579 flags.go:64] FLAG: --container-runtime-endpoint="/var/run/crio/crio.sock" Jul 03 11:12:23.726072 ip-10-0-136-0 kubenswrapper[2579]: I0703 11:12:23.720480 2579 flags.go:64] FLAG: --containerd="/run/containerd/containerd.sock" Jul 03 11:12:23.726072 ip-10-0-136-0 kubenswrapper[2579]: I0703 11:12:23.720485 2579 flags.go:64] FLAG: --containerd-namespace="k8s.io" Jul 03 11:12:23.726072 ip-10-0-136-0 kubenswrapper[2579]: I0703 11:12:23.720490 2579 flags.go:64] FLAG: --contention-profiling="false" Jul 03 11:12:23.726072 ip-10-0-136-0 kubenswrapper[2579]: I0703 11:12:23.720494 2579 flags.go:64] FLAG: --cpu-cfs-quota="true" Jul 03 11:12:23.726707 ip-10-0-136-0 kubenswrapper[2579]: I0703 11:12:23.720499 2579 flags.go:64] FLAG: --cpu-cfs-quota-period="100ms" Jul 03 11:12:23.726707 ip-10-0-136-0 kubenswrapper[2579]: I0703 11:12:23.720504 2579 flags.go:64] FLAG: --cpu-manager-policy="none" Jul 03 11:12:23.726707 ip-10-0-136-0 kubenswrapper[2579]: I0703 11:12:23.720509 2579 flags.go:64] FLAG: --cpu-manager-policy-options="" Jul 03 11:12:23.726707 ip-10-0-136-0 kubenswrapper[2579]: I0703 11:12:23.720516 2579 flags.go:64] FLAG: --cpu-manager-reconcile-period="10s" Jul 03 11:12:23.726707 ip-10-0-136-0 kubenswrapper[2579]: I0703 11:12:23.720523 2579 flags.go:64] FLAG: --enable-controller-attach-detach="true" Jul 03 11:12:23.726707 ip-10-0-136-0 kubenswrapper[2579]: I0703 11:12:23.720528 2579 flags.go:64] FLAG: --enable-debugging-handlers="true" Jul 03 11:12:23.726707 ip-10-0-136-0 kubenswrapper[2579]: I0703 11:12:23.720535 2579 flags.go:64] FLAG: --enable-load-reader="false" Jul 03 11:12:23.726707 ip-10-0-136-0 kubenswrapper[2579]: I0703 11:12:23.720541 2579 flags.go:64] FLAG: --enable-server="true" Jul 03 11:12:23.726707 ip-10-0-136-0 kubenswrapper[2579]: I0703 11:12:23.720546 2579 flags.go:64] FLAG: --enforce-node-allocatable="[pods]" Jul 03 11:12:23.726707 ip-10-0-136-0 kubenswrapper[2579]: I0703 11:12:23.720556 2579 flags.go:64] FLAG: --event-burst="100" Jul 03 11:12:23.726707 ip-10-0-136-0 kubenswrapper[2579]: I0703 11:12:23.720561 2579 flags.go:64] FLAG: --event-qps="50" Jul 03 11:12:23.726707 ip-10-0-136-0 kubenswrapper[2579]: I0703 11:12:23.720566 2579 flags.go:64] FLAG: --event-storage-age-limit="default=0" Jul 03 11:12:23.726707 ip-10-0-136-0 kubenswrapper[2579]: I0703 11:12:23.720571 2579 flags.go:64] FLAG: --event-storage-event-limit="default=0" Jul 03 11:12:23.726707 ip-10-0-136-0 kubenswrapper[2579]: I0703 11:12:23.720576 2579 flags.go:64] FLAG: --eviction-hard="" Jul 03 11:12:23.726707 ip-10-0-136-0 kubenswrapper[2579]: I0703 11:12:23.720583 2579 flags.go:64] FLAG: --eviction-max-pod-grace-period="0" Jul 03 11:12:23.726707 ip-10-0-136-0 kubenswrapper[2579]: I0703 11:12:23.720588 2579 flags.go:64] FLAG: --eviction-minimum-reclaim="" Jul 03 11:12:23.726707 ip-10-0-136-0 kubenswrapper[2579]: I0703 11:12:23.720593 2579 flags.go:64] FLAG: --eviction-pressure-transition-period="5m0s" Jul 03 11:12:23.726707 ip-10-0-136-0 kubenswrapper[2579]: I0703 11:12:23.720599 2579 flags.go:64] FLAG: --eviction-soft="" Jul 03 11:12:23.726707 ip-10-0-136-0 kubenswrapper[2579]: I0703 11:12:23.720605 2579 flags.go:64] FLAG: --eviction-soft-grace-period="" Jul 03 11:12:23.726707 ip-10-0-136-0 kubenswrapper[2579]: I0703 11:12:23.720609 2579 flags.go:64] FLAG: --exit-on-lock-contention="false" Jul 03 11:12:23.726707 ip-10-0-136-0 kubenswrapper[2579]: I0703 11:12:23.720614 2579 flags.go:64] FLAG: --experimental-allocatable-ignore-eviction="false" Jul 03 11:12:23.726707 ip-10-0-136-0 kubenswrapper[2579]: I0703 11:12:23.720621 2579 flags.go:64] FLAG: --experimental-mounter-path="" Jul 03 11:12:23.726707 ip-10-0-136-0 kubenswrapper[2579]: I0703 11:12:23.720626 2579 flags.go:64] FLAG: --fail-cgroupv1="false" Jul 03 11:12:23.726707 ip-10-0-136-0 kubenswrapper[2579]: I0703 11:12:23.720630 2579 flags.go:64] FLAG: --fail-swap-on="true" Jul 03 11:12:23.726707 ip-10-0-136-0 kubenswrapper[2579]: I0703 11:12:23.720635 2579 flags.go:64] FLAG: --feature-gates="" Jul 03 11:12:23.727434 ip-10-0-136-0 kubenswrapper[2579]: I0703 11:12:23.720642 2579 flags.go:64] FLAG: --file-check-frequency="20s" Jul 03 11:12:23.727434 ip-10-0-136-0 kubenswrapper[2579]: I0703 11:12:23.720647 2579 flags.go:64] FLAG: --global-housekeeping-interval="1m0s" Jul 03 11:12:23.727434 ip-10-0-136-0 kubenswrapper[2579]: I0703 11:12:23.720653 2579 flags.go:64] FLAG: --hairpin-mode="promiscuous-bridge" Jul 03 11:12:23.727434 ip-10-0-136-0 kubenswrapper[2579]: I0703 11:12:23.720659 2579 flags.go:64] FLAG: --healthz-bind-address="127.0.0.1" Jul 03 11:12:23.727434 ip-10-0-136-0 kubenswrapper[2579]: I0703 11:12:23.720664 2579 flags.go:64] FLAG: --healthz-port="10248" Jul 03 11:12:23.727434 ip-10-0-136-0 kubenswrapper[2579]: I0703 11:12:23.720670 2579 flags.go:64] FLAG: --help="false" Jul 03 11:12:23.727434 ip-10-0-136-0 kubenswrapper[2579]: I0703 11:12:23.720675 2579 flags.go:64] FLAG: --hostname-override="ip-10-0-136-0.ec2.internal" Jul 03 11:12:23.727434 ip-10-0-136-0 kubenswrapper[2579]: I0703 11:12:23.720680 2579 flags.go:64] FLAG: --housekeeping-interval="10s" Jul 03 11:12:23.727434 ip-10-0-136-0 kubenswrapper[2579]: I0703 11:12:23.720685 2579 flags.go:64] FLAG: --http-check-frequency="20s" Jul 03 11:12:23.727434 ip-10-0-136-0 kubenswrapper[2579]: I0703 11:12:23.720691 2579 flags.go:64] FLAG: --image-credential-provider-bin-dir="/usr/libexec/kubelet-image-credential-provider-plugins" Jul 03 11:12:23.727434 ip-10-0-136-0 kubenswrapper[2579]: I0703 11:12:23.720697 2579 flags.go:64] FLAG: --image-credential-provider-config="/etc/kubernetes/credential-providers/ecr-credential-provider.yaml" Jul 03 11:12:23.727434 ip-10-0-136-0 kubenswrapper[2579]: I0703 11:12:23.720707 2579 flags.go:64] FLAG: --image-gc-high-threshold="85" Jul 03 11:12:23.727434 ip-10-0-136-0 kubenswrapper[2579]: I0703 11:12:23.720712 2579 flags.go:64] FLAG: --image-gc-low-threshold="80" Jul 03 11:12:23.727434 ip-10-0-136-0 kubenswrapper[2579]: I0703 11:12:23.720717 2579 flags.go:64] FLAG: --image-service-endpoint="" Jul 03 11:12:23.727434 ip-10-0-136-0 kubenswrapper[2579]: I0703 11:12:23.720721 2579 flags.go:64] FLAG: --kernel-memcg-notification="false" Jul 03 11:12:23.727434 ip-10-0-136-0 kubenswrapper[2579]: I0703 11:12:23.720727 2579 flags.go:64] FLAG: --kube-api-burst="100" Jul 03 11:12:23.727434 ip-10-0-136-0 kubenswrapper[2579]: I0703 11:12:23.720733 2579 flags.go:64] FLAG: --kube-api-content-type="application/vnd.kubernetes.protobuf" Jul 03 11:12:23.727434 ip-10-0-136-0 kubenswrapper[2579]: I0703 11:12:23.720740 2579 flags.go:64] FLAG: --kube-api-qps="50" Jul 03 11:12:23.727434 ip-10-0-136-0 kubenswrapper[2579]: I0703 11:12:23.720745 2579 flags.go:64] FLAG: --kube-reserved="" Jul 03 11:12:23.727434 ip-10-0-136-0 kubenswrapper[2579]: I0703 11:12:23.720750 2579 flags.go:64] FLAG: --kube-reserved-cgroup="" Jul 03 11:12:23.727434 ip-10-0-136-0 kubenswrapper[2579]: I0703 11:12:23.720755 2579 flags.go:64] FLAG: --kubeconfig="/var/lib/kubelet/kubeconfig" Jul 03 11:12:23.727434 ip-10-0-136-0 kubenswrapper[2579]: I0703 11:12:23.720760 2579 flags.go:64] FLAG: --kubelet-cgroups="" Jul 03 11:12:23.727434 ip-10-0-136-0 kubenswrapper[2579]: I0703 11:12:23.720765 2579 flags.go:64] FLAG: --local-storage-capacity-isolation="true" Jul 03 11:12:23.727434 ip-10-0-136-0 kubenswrapper[2579]: I0703 11:12:23.720771 2579 flags.go:64] FLAG: --lock-file="" Jul 03 11:12:23.728037 ip-10-0-136-0 kubenswrapper[2579]: I0703 11:12:23.720776 2579 flags.go:64] FLAG: --log-cadvisor-usage="false" Jul 03 11:12:23.728037 ip-10-0-136-0 kubenswrapper[2579]: I0703 11:12:23.720782 2579 flags.go:64] FLAG: --log-flush-frequency="5s" Jul 03 11:12:23.728037 ip-10-0-136-0 kubenswrapper[2579]: I0703 11:12:23.720787 2579 flags.go:64] FLAG: --log-json-info-buffer-size="0" Jul 03 11:12:23.728037 ip-10-0-136-0 kubenswrapper[2579]: I0703 11:12:23.720798 2579 flags.go:64] FLAG: --log-json-split-stream="false" Jul 03 11:12:23.728037 ip-10-0-136-0 kubenswrapper[2579]: I0703 11:12:23.720816 2579 flags.go:64] FLAG: --log-text-info-buffer-size="0" Jul 03 11:12:23.728037 ip-10-0-136-0 kubenswrapper[2579]: I0703 11:12:23.720823 2579 flags.go:64] FLAG: --log-text-split-stream="false" Jul 03 11:12:23.728037 ip-10-0-136-0 kubenswrapper[2579]: I0703 11:12:23.720828 2579 flags.go:64] FLAG: --logging-format="text" Jul 03 11:12:23.728037 ip-10-0-136-0 kubenswrapper[2579]: I0703 11:12:23.720833 2579 flags.go:64] FLAG: --machine-id-file="/etc/machine-id,/var/lib/dbus/machine-id" Jul 03 11:12:23.728037 ip-10-0-136-0 kubenswrapper[2579]: I0703 11:12:23.720839 2579 flags.go:64] FLAG: --make-iptables-util-chains="true" Jul 03 11:12:23.728037 ip-10-0-136-0 kubenswrapper[2579]: I0703 11:12:23.720844 2579 flags.go:64] FLAG: --manifest-url="" Jul 03 11:12:23.728037 ip-10-0-136-0 kubenswrapper[2579]: I0703 11:12:23.720849 2579 flags.go:64] FLAG: --manifest-url-header="" Jul 03 11:12:23.728037 ip-10-0-136-0 kubenswrapper[2579]: I0703 11:12:23.720860 2579 flags.go:64] FLAG: --max-housekeeping-interval="15s" Jul 03 11:12:23.728037 ip-10-0-136-0 kubenswrapper[2579]: I0703 11:12:23.720865 2579 flags.go:64] FLAG: --max-open-files="1000000" Jul 03 11:12:23.728037 ip-10-0-136-0 kubenswrapper[2579]: I0703 11:12:23.720872 2579 flags.go:64] FLAG: --max-pods="110" Jul 03 11:12:23.728037 ip-10-0-136-0 kubenswrapper[2579]: I0703 11:12:23.720878 2579 flags.go:64] FLAG: --maximum-dead-containers="-1" Jul 03 11:12:23.728037 ip-10-0-136-0 kubenswrapper[2579]: I0703 11:12:23.720883 2579 flags.go:64] FLAG: --maximum-dead-containers-per-container="1" Jul 03 11:12:23.728037 ip-10-0-136-0 kubenswrapper[2579]: I0703 11:12:23.720887 2579 flags.go:64] FLAG: --memory-manager-policy="None" Jul 03 11:12:23.728037 ip-10-0-136-0 kubenswrapper[2579]: I0703 11:12:23.720893 2579 flags.go:64] FLAG: --minimum-container-ttl-duration="6m0s" Jul 03 11:12:23.728037 ip-10-0-136-0 kubenswrapper[2579]: I0703 11:12:23.720898 2579 flags.go:64] FLAG: --minimum-image-ttl-duration="2m0s" Jul 03 11:12:23.728037 ip-10-0-136-0 kubenswrapper[2579]: I0703 11:12:23.720904 2579 flags.go:64] FLAG: --node-ip="0.0.0.0" Jul 03 11:12:23.728037 ip-10-0-136-0 kubenswrapper[2579]: I0703 11:12:23.720910 2579 flags.go:64] FLAG: --node-labels="node-role.kubernetes.io/worker=,node.openshift.io/os_id=rhel" Jul 03 11:12:23.728037 ip-10-0-136-0 kubenswrapper[2579]: I0703 11:12:23.720927 2579 flags.go:64] FLAG: --node-status-max-images="50" Jul 03 11:12:23.728037 ip-10-0-136-0 kubenswrapper[2579]: I0703 11:12:23.720932 2579 flags.go:64] FLAG: --node-status-update-frequency="10s" Jul 03 11:12:23.728037 ip-10-0-136-0 kubenswrapper[2579]: I0703 11:12:23.720937 2579 flags.go:64] FLAG: --oom-score-adj="-999" Jul 03 11:12:23.728037 ip-10-0-136-0 kubenswrapper[2579]: I0703 11:12:23.720943 2579 flags.go:64] FLAG: --pod-cidr="" Jul 03 11:12:23.728683 ip-10-0-136-0 kubenswrapper[2579]: I0703 11:12:23.720949 2579 flags.go:64] FLAG: --pod-infra-container-image="quay.io/openshift-release-dev/ocp-v4.0-art-dev@sha256:ce93f6b4197b9c62a26a3131f8057ffe9c9379c5079a4f62e5919b75151394e5" Jul 03 11:12:23.728683 ip-10-0-136-0 kubenswrapper[2579]: I0703 11:12:23.720959 2579 flags.go:64] FLAG: --pod-manifest-path="" Jul 03 11:12:23.728683 ip-10-0-136-0 kubenswrapper[2579]: I0703 11:12:23.720965 2579 flags.go:64] FLAG: --pod-max-pids="-1" Jul 03 11:12:23.728683 ip-10-0-136-0 kubenswrapper[2579]: I0703 11:12:23.720970 2579 flags.go:64] FLAG: --pods-per-core="0" Jul 03 11:12:23.728683 ip-10-0-136-0 kubenswrapper[2579]: I0703 11:12:23.720975 2579 flags.go:64] FLAG: --port="10250" Jul 03 11:12:23.728683 ip-10-0-136-0 kubenswrapper[2579]: I0703 11:12:23.720981 2579 flags.go:64] FLAG: --protect-kernel-defaults="false" Jul 03 11:12:23.728683 ip-10-0-136-0 kubenswrapper[2579]: I0703 11:12:23.720986 2579 flags.go:64] FLAG: --provider-id="aws:///us-east-1a/i-076a55e9b5f1c410b" Jul 03 11:12:23.728683 ip-10-0-136-0 kubenswrapper[2579]: I0703 11:12:23.720992 2579 flags.go:64] FLAG: --qos-reserved="" Jul 03 11:12:23.728683 ip-10-0-136-0 kubenswrapper[2579]: I0703 11:12:23.720997 2579 flags.go:64] FLAG: --read-only-port="10255" Jul 03 11:12:23.728683 ip-10-0-136-0 kubenswrapper[2579]: I0703 11:12:23.721003 2579 flags.go:64] FLAG: --register-node="true" Jul 03 11:12:23.728683 ip-10-0-136-0 kubenswrapper[2579]: I0703 11:12:23.721008 2579 flags.go:64] FLAG: --register-with-taints="" Jul 03 11:12:23.728683 ip-10-0-136-0 kubenswrapper[2579]: I0703 11:12:23.721015 2579 flags.go:64] FLAG: --registry-burst="10" Jul 03 11:12:23.728683 ip-10-0-136-0 kubenswrapper[2579]: I0703 11:12:23.721028 2579 flags.go:64] FLAG: --registry-qps="5" Jul 03 11:12:23.728683 ip-10-0-136-0 kubenswrapper[2579]: I0703 11:12:23.721034 2579 flags.go:64] FLAG: --reserved-cpus="" Jul 03 11:12:23.728683 ip-10-0-136-0 kubenswrapper[2579]: I0703 11:12:23.721039 2579 flags.go:64] FLAG: --reserved-memory="" Jul 03 11:12:23.728683 ip-10-0-136-0 kubenswrapper[2579]: I0703 11:12:23.721046 2579 flags.go:64] FLAG: --resolv-conf="/etc/resolv.conf" Jul 03 11:12:23.728683 ip-10-0-136-0 kubenswrapper[2579]: I0703 11:12:23.721052 2579 flags.go:64] FLAG: --root-dir="/var/lib/kubelet" Jul 03 11:12:23.728683 ip-10-0-136-0 kubenswrapper[2579]: I0703 11:12:23.721057 2579 flags.go:64] FLAG: --rotate-certificates="false" Jul 03 11:12:23.728683 ip-10-0-136-0 kubenswrapper[2579]: I0703 11:12:23.721062 2579 flags.go:64] FLAG: --rotate-server-certificates="false" Jul 03 11:12:23.728683 ip-10-0-136-0 kubenswrapper[2579]: I0703 11:12:23.721068 2579 flags.go:64] FLAG: --runonce="false" Jul 03 11:12:23.728683 ip-10-0-136-0 kubenswrapper[2579]: I0703 11:12:23.721074 2579 flags.go:64] FLAG: --runtime-cgroups="/system.slice/crio.service" Jul 03 11:12:23.728683 ip-10-0-136-0 kubenswrapper[2579]: I0703 11:12:23.721079 2579 flags.go:64] FLAG: --runtime-request-timeout="2m0s" Jul 03 11:12:23.728683 ip-10-0-136-0 kubenswrapper[2579]: I0703 11:12:23.721085 2579 flags.go:64] FLAG: --seccomp-default="false" Jul 03 11:12:23.728683 ip-10-0-136-0 kubenswrapper[2579]: I0703 11:12:23.721090 2579 flags.go:64] FLAG: --serialize-image-pulls="true" Jul 03 11:12:23.728683 ip-10-0-136-0 kubenswrapper[2579]: I0703 11:12:23.721095 2579 flags.go:64] FLAG: --storage-driver-buffer-duration="1m0s" Jul 03 11:12:23.729470 ip-10-0-136-0 kubenswrapper[2579]: I0703 11:12:23.721100 2579 flags.go:64] FLAG: --storage-driver-db="cadvisor" Jul 03 11:12:23.729470 ip-10-0-136-0 kubenswrapper[2579]: I0703 11:12:23.721106 2579 flags.go:64] FLAG: --storage-driver-host="localhost:8086" Jul 03 11:12:23.729470 ip-10-0-136-0 kubenswrapper[2579]: I0703 11:12:23.721117 2579 flags.go:64] FLAG: --storage-driver-password="root" Jul 03 11:12:23.729470 ip-10-0-136-0 kubenswrapper[2579]: I0703 11:12:23.721123 2579 flags.go:64] FLAG: --storage-driver-secure="false" Jul 03 11:12:23.729470 ip-10-0-136-0 kubenswrapper[2579]: I0703 11:12:23.721128 2579 flags.go:64] FLAG: --storage-driver-table="stats" Jul 03 11:12:23.729470 ip-10-0-136-0 kubenswrapper[2579]: I0703 11:12:23.721133 2579 flags.go:64] FLAG: --storage-driver-user="root" Jul 03 11:12:23.729470 ip-10-0-136-0 kubenswrapper[2579]: I0703 11:12:23.721138 2579 flags.go:64] FLAG: --streaming-connection-idle-timeout="4h0m0s" Jul 03 11:12:23.729470 ip-10-0-136-0 kubenswrapper[2579]: I0703 11:12:23.721144 2579 flags.go:64] FLAG: --sync-frequency="1m0s" Jul 03 11:12:23.729470 ip-10-0-136-0 kubenswrapper[2579]: I0703 11:12:23.721149 2579 flags.go:64] FLAG: --system-cgroups="" Jul 03 11:12:23.729470 ip-10-0-136-0 kubenswrapper[2579]: I0703 11:12:23.721154 2579 flags.go:64] FLAG: --system-reserved="cpu=500m,ephemeral-storage=1Gi,memory=1Gi" Jul 03 11:12:23.729470 ip-10-0-136-0 kubenswrapper[2579]: I0703 11:12:23.721168 2579 flags.go:64] FLAG: --system-reserved-cgroup="" Jul 03 11:12:23.729470 ip-10-0-136-0 kubenswrapper[2579]: I0703 11:12:23.721174 2579 flags.go:64] FLAG: --tls-cert-file="" Jul 03 11:12:23.729470 ip-10-0-136-0 kubenswrapper[2579]: I0703 11:12:23.721178 2579 flags.go:64] FLAG: --tls-cipher-suites="[]" Jul 03 11:12:23.729470 ip-10-0-136-0 kubenswrapper[2579]: I0703 11:12:23.721211 2579 flags.go:64] FLAG: --tls-min-version="" Jul 03 11:12:23.729470 ip-10-0-136-0 kubenswrapper[2579]: I0703 11:12:23.721217 2579 flags.go:64] FLAG: --tls-private-key-file="" Jul 03 11:12:23.729470 ip-10-0-136-0 kubenswrapper[2579]: I0703 11:12:23.721222 2579 flags.go:64] FLAG: --topology-manager-policy="none" Jul 03 11:12:23.729470 ip-10-0-136-0 kubenswrapper[2579]: I0703 11:12:23.721227 2579 flags.go:64] FLAG: --topology-manager-policy-options="" Jul 03 11:12:23.729470 ip-10-0-136-0 kubenswrapper[2579]: I0703 11:12:23.721233 2579 flags.go:64] FLAG: --topology-manager-scope="container" Jul 03 11:12:23.729470 ip-10-0-136-0 kubenswrapper[2579]: I0703 11:12:23.721239 2579 flags.go:64] FLAG: --v="2" Jul 03 11:12:23.729470 ip-10-0-136-0 kubenswrapper[2579]: I0703 11:12:23.721246 2579 flags.go:64] FLAG: --version="false" Jul 03 11:12:23.729470 ip-10-0-136-0 kubenswrapper[2579]: I0703 11:12:23.721262 2579 flags.go:64] FLAG: --vmodule="" Jul 03 11:12:23.729470 ip-10-0-136-0 kubenswrapper[2579]: I0703 11:12:23.721270 2579 flags.go:64] FLAG: --volume-plugin-dir="/etc/kubernetes/kubelet-plugins/volume/exec" Jul 03 11:12:23.729470 ip-10-0-136-0 kubenswrapper[2579]: I0703 11:12:23.721276 2579 flags.go:64] FLAG: --volume-stats-agg-period="1m0s" Jul 03 11:12:23.729470 ip-10-0-136-0 kubenswrapper[2579]: W0703 11:12:23.721561 2579 feature_gate.go:328] unrecognized feature gate: MachineConfigNodes Jul 03 11:12:23.729470 ip-10-0-136-0 kubenswrapper[2579]: W0703 11:12:23.721572 2579 feature_gate.go:328] unrecognized feature gate: EtcdBackendQuota Jul 03 11:12:23.730100 ip-10-0-136-0 kubenswrapper[2579]: W0703 11:12:23.721577 2579 feature_gate.go:328] unrecognized feature gate: VSphereHostVMGroupZonal Jul 03 11:12:23.730100 ip-10-0-136-0 kubenswrapper[2579]: W0703 11:12:23.721583 2579 feature_gate.go:328] unrecognized feature gate: NutanixMultiSubnets Jul 03 11:12:23.730100 ip-10-0-136-0 kubenswrapper[2579]: W0703 11:12:23.721589 2579 feature_gate.go:328] unrecognized feature gate: MachineAPIMigration Jul 03 11:12:23.730100 ip-10-0-136-0 kubenswrapper[2579]: W0703 11:12:23.721594 2579 feature_gate.go:328] unrecognized feature gate: NewOLMCatalogdAPIV1Metas Jul 03 11:12:23.730100 ip-10-0-136-0 kubenswrapper[2579]: W0703 11:12:23.721598 2579 feature_gate.go:328] unrecognized feature gate: AWSServiceLBNetworkSecurityGroup Jul 03 11:12:23.730100 ip-10-0-136-0 kubenswrapper[2579]: W0703 11:12:23.721605 2579 feature_gate.go:349] Setting deprecated feature gate KMSv1=true. It will be removed in a future release. Jul 03 11:12:23.730100 ip-10-0-136-0 kubenswrapper[2579]: W0703 11:12:23.721612 2579 feature_gate.go:328] unrecognized feature gate: AWSClusterHostedDNS Jul 03 11:12:23.730100 ip-10-0-136-0 kubenswrapper[2579]: W0703 11:12:23.721618 2579 feature_gate.go:328] unrecognized feature gate: GCPCustomAPIEndpoints Jul 03 11:12:23.730100 ip-10-0-136-0 kubenswrapper[2579]: W0703 11:12:23.721624 2579 feature_gate.go:328] unrecognized feature gate: AlibabaPlatform Jul 03 11:12:23.730100 ip-10-0-136-0 kubenswrapper[2579]: W0703 11:12:23.721633 2579 feature_gate.go:351] Setting GA feature gate VolumeAttributesClass=true. It will be removed in a future release. Jul 03 11:12:23.730100 ip-10-0-136-0 kubenswrapper[2579]: W0703 11:12:23.721640 2579 feature_gate.go:328] unrecognized feature gate: AzureDedicatedHosts Jul 03 11:12:23.730100 ip-10-0-136-0 kubenswrapper[2579]: W0703 11:12:23.721644 2579 feature_gate.go:328] unrecognized feature gate: DualReplica Jul 03 11:12:23.730100 ip-10-0-136-0 kubenswrapper[2579]: W0703 11:12:23.721649 2579 feature_gate.go:328] unrecognized feature gate: ClusterAPIInstallIBMCloud Jul 03 11:12:23.730100 ip-10-0-136-0 kubenswrapper[2579]: W0703 11:12:23.721655 2579 feature_gate.go:328] unrecognized feature gate: AdditionalRoutingCapabilities Jul 03 11:12:23.730100 ip-10-0-136-0 kubenswrapper[2579]: W0703 11:12:23.721659 2579 feature_gate.go:328] unrecognized feature gate: PreconfiguredUDNAddresses Jul 03 11:12:23.730100 ip-10-0-136-0 kubenswrapper[2579]: W0703 11:12:23.721664 2579 feature_gate.go:328] unrecognized feature gate: MultiArchInstallAzure Jul 03 11:12:23.730100 ip-10-0-136-0 kubenswrapper[2579]: W0703 11:12:23.721669 2579 feature_gate.go:328] unrecognized feature gate: NewOLMPreflightPermissionChecks Jul 03 11:12:23.730100 ip-10-0-136-0 kubenswrapper[2579]: W0703 11:12:23.721673 2579 feature_gate.go:328] unrecognized feature gate: IngressControllerDynamicConfigurationManager Jul 03 11:12:23.730100 ip-10-0-136-0 kubenswrapper[2579]: W0703 11:12:23.721678 2579 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesAzure Jul 03 11:12:23.730611 ip-10-0-136-0 kubenswrapper[2579]: W0703 11:12:23.721682 2579 feature_gate.go:328] unrecognized feature gate: MultiDiskSetup Jul 03 11:12:23.730611 ip-10-0-136-0 kubenswrapper[2579]: W0703 11:12:23.721686 2579 feature_gate.go:328] unrecognized feature gate: ExternalOIDC Jul 03 11:12:23.730611 ip-10-0-136-0 kubenswrapper[2579]: W0703 11:12:23.721690 2579 feature_gate.go:328] unrecognized feature gate: OVNObservability Jul 03 11:12:23.730611 ip-10-0-136-0 kubenswrapper[2579]: W0703 11:12:23.721695 2579 feature_gate.go:328] unrecognized feature gate: ClusterAPIMachineManagement Jul 03 11:12:23.730611 ip-10-0-136-0 kubenswrapper[2579]: W0703 11:12:23.721699 2579 feature_gate.go:328] unrecognized feature gate: GatewayAPIWithoutOLM Jul 03 11:12:23.730611 ip-10-0-136-0 kubenswrapper[2579]: W0703 11:12:23.721703 2579 feature_gate.go:328] unrecognized feature gate: IrreconcilableMachineConfig Jul 03 11:12:23.730611 ip-10-0-136-0 kubenswrapper[2579]: W0703 11:12:23.721708 2579 feature_gate.go:328] unrecognized feature gate: VolumeGroupSnapshot Jul 03 11:12:23.730611 ip-10-0-136-0 kubenswrapper[2579]: W0703 11:12:23.721713 2579 feature_gate.go:328] unrecognized feature gate: SigstoreImageVerificationPKI Jul 03 11:12:23.730611 ip-10-0-136-0 kubenswrapper[2579]: W0703 11:12:23.721717 2579 feature_gate.go:328] unrecognized feature gate: BuildCSIVolumes Jul 03 11:12:23.730611 ip-10-0-136-0 kubenswrapper[2579]: W0703 11:12:23.721723 2579 feature_gate.go:328] unrecognized feature gate: GCPCustomAPIEndpointsInstall Jul 03 11:12:23.730611 ip-10-0-136-0 kubenswrapper[2579]: W0703 11:12:23.721728 2579 feature_gate.go:328] unrecognized feature gate: InsightsOnDemandDataGather Jul 03 11:12:23.730611 ip-10-0-136-0 kubenswrapper[2579]: W0703 11:12:23.721732 2579 feature_gate.go:328] unrecognized feature gate: AWSDualStackInstall Jul 03 11:12:23.730611 ip-10-0-136-0 kubenswrapper[2579]: W0703 11:12:23.721737 2579 feature_gate.go:328] unrecognized feature gate: MetricsCollectionProfiles Jul 03 11:12:23.730611 ip-10-0-136-0 kubenswrapper[2579]: W0703 11:12:23.721741 2579 feature_gate.go:328] unrecognized feature gate: KMSEncryptionProvider Jul 03 11:12:23.730611 ip-10-0-136-0 kubenswrapper[2579]: W0703 11:12:23.721746 2579 feature_gate.go:328] unrecognized feature gate: ProvisioningRequestAvailable Jul 03 11:12:23.730611 ip-10-0-136-0 kubenswrapper[2579]: W0703 11:12:23.721750 2579 feature_gate.go:328] unrecognized feature gate: NewOLMBoxCutterRuntime Jul 03 11:12:23.730611 ip-10-0-136-0 kubenswrapper[2579]: W0703 11:12:23.721754 2579 feature_gate.go:328] unrecognized feature gate: GCPDualStackInstall Jul 03 11:12:23.730611 ip-10-0-136-0 kubenswrapper[2579]: W0703 11:12:23.721759 2579 feature_gate.go:328] unrecognized feature gate: NetworkLiveMigration Jul 03 11:12:23.730611 ip-10-0-136-0 kubenswrapper[2579]: W0703 11:12:23.721763 2579 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesAWS Jul 03 11:12:23.730611 ip-10-0-136-0 kubenswrapper[2579]: W0703 11:12:23.721768 2579 feature_gate.go:328] unrecognized feature gate: KMSEncryption Jul 03 11:12:23.731118 ip-10-0-136-0 kubenswrapper[2579]: W0703 11:12:23.721772 2579 feature_gate.go:328] unrecognized feature gate: OSStreams Jul 03 11:12:23.731118 ip-10-0-136-0 kubenswrapper[2579]: W0703 11:12:23.721777 2579 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesvSphere Jul 03 11:12:23.731118 ip-10-0-136-0 kubenswrapper[2579]: W0703 11:12:23.721783 2579 feature_gate.go:328] unrecognized feature gate: AzureWorkloadIdentity Jul 03 11:12:23.731118 ip-10-0-136-0 kubenswrapper[2579]: W0703 11:12:23.721787 2579 feature_gate.go:328] unrecognized feature gate: OpenShiftPodSecurityAdmission Jul 03 11:12:23.731118 ip-10-0-136-0 kubenswrapper[2579]: W0703 11:12:23.721792 2579 feature_gate.go:328] unrecognized feature gate: AWSClusterHostedDNSInstall Jul 03 11:12:23.731118 ip-10-0-136-0 kubenswrapper[2579]: W0703 11:12:23.721796 2579 feature_gate.go:328] unrecognized feature gate: UpgradeStatus Jul 03 11:12:23.731118 ip-10-0-136-0 kubenswrapper[2579]: W0703 11:12:23.721800 2579 feature_gate.go:328] unrecognized feature gate: CPMSMachineNamePrefix Jul 03 11:12:23.731118 ip-10-0-136-0 kubenswrapper[2579]: W0703 11:12:23.721805 2579 feature_gate.go:328] unrecognized feature gate: NetworkSegmentation Jul 03 11:12:23.731118 ip-10-0-136-0 kubenswrapper[2579]: W0703 11:12:23.721810 2579 feature_gate.go:328] unrecognized feature gate: SigstoreImageVerification Jul 03 11:12:23.731118 ip-10-0-136-0 kubenswrapper[2579]: W0703 11:12:23.721815 2579 feature_gate.go:328] unrecognized feature gate: ClusterAPIMachineManagementVSphere Jul 03 11:12:23.731118 ip-10-0-136-0 kubenswrapper[2579]: W0703 11:12:23.721819 2579 feature_gate.go:328] unrecognized feature gate: GCPClusterHostedDNS Jul 03 11:12:23.731118 ip-10-0-136-0 kubenswrapper[2579]: W0703 11:12:23.721824 2579 feature_gate.go:328] unrecognized feature gate: HyperShiftOnlyDynamicResourceAllocation Jul 03 11:12:23.731118 ip-10-0-136-0 kubenswrapper[2579]: W0703 11:12:23.721828 2579 feature_gate.go:328] unrecognized feature gate: AzureMultiDisk Jul 03 11:12:23.731118 ip-10-0-136-0 kubenswrapper[2579]: W0703 11:12:23.721833 2579 feature_gate.go:328] unrecognized feature gate: NoRegistryClusterInstall Jul 03 11:12:23.731118 ip-10-0-136-0 kubenswrapper[2579]: W0703 11:12:23.721837 2579 feature_gate.go:328] unrecognized feature gate: ImageModeStatusReporting Jul 03 11:12:23.731118 ip-10-0-136-0 kubenswrapper[2579]: W0703 11:12:23.721842 2579 feature_gate.go:328] unrecognized feature gate: GatewayAPI Jul 03 11:12:23.731118 ip-10-0-136-0 kubenswrapper[2579]: W0703 11:12:23.721846 2579 feature_gate.go:328] unrecognized feature gate: VSphereMixedNodeEnv Jul 03 11:12:23.731118 ip-10-0-136-0 kubenswrapper[2579]: W0703 11:12:23.721850 2579 feature_gate.go:328] unrecognized feature gate: OnPremDNSRecords Jul 03 11:12:23.731118 ip-10-0-136-0 kubenswrapper[2579]: W0703 11:12:23.721855 2579 feature_gate.go:328] unrecognized feature gate: DNSNameResolver Jul 03 11:12:23.731118 ip-10-0-136-0 kubenswrapper[2579]: W0703 11:12:23.721859 2579 feature_gate.go:328] unrecognized feature gate: ClusterAPIInstall Jul 03 11:12:23.731658 ip-10-0-136-0 kubenswrapper[2579]: W0703 11:12:23.721863 2579 feature_gate.go:328] unrecognized feature gate: SignatureStores Jul 03 11:12:23.731658 ip-10-0-136-0 kubenswrapper[2579]: W0703 11:12:23.721868 2579 feature_gate.go:328] unrecognized feature gate: NetworkDiagnosticsConfig Jul 03 11:12:23.731658 ip-10-0-136-0 kubenswrapper[2579]: W0703 11:12:23.721873 2579 feature_gate.go:328] unrecognized feature gate: Example2 Jul 03 11:12:23.731658 ip-10-0-136-0 kubenswrapper[2579]: W0703 11:12:23.721878 2579 feature_gate.go:328] unrecognized feature gate: RouteAdvertisements Jul 03 11:12:23.731658 ip-10-0-136-0 kubenswrapper[2579]: W0703 11:12:23.721885 2579 feature_gate.go:351] Setting GA feature gate ServiceAccountTokenNodeBinding=true. It will be removed in a future release. Jul 03 11:12:23.731658 ip-10-0-136-0 kubenswrapper[2579]: W0703 11:12:23.721891 2579 feature_gate.go:328] unrecognized feature gate: VSphereMultiNetworks Jul 03 11:12:23.731658 ip-10-0-136-0 kubenswrapper[2579]: W0703 11:12:23.721896 2579 feature_gate.go:328] unrecognized feature gate: GCPClusterHostedDNSInstall Jul 03 11:12:23.731658 ip-10-0-136-0 kubenswrapper[2579]: W0703 11:12:23.721900 2579 feature_gate.go:328] unrecognized feature gate: ExternalOIDCWithUIDAndExtraClaimMappings Jul 03 11:12:23.731658 ip-10-0-136-0 kubenswrapper[2579]: W0703 11:12:23.721905 2579 feature_gate.go:328] unrecognized feature gate: AutomatedEtcdBackup Jul 03 11:12:23.731658 ip-10-0-136-0 kubenswrapper[2579]: W0703 11:12:23.721909 2579 feature_gate.go:328] unrecognized feature gate: HighlyAvailableArbiter Jul 03 11:12:23.731658 ip-10-0-136-0 kubenswrapper[2579]: W0703 11:12:23.721913 2579 feature_gate.go:328] unrecognized feature gate: Example Jul 03 11:12:23.731658 ip-10-0-136-0 kubenswrapper[2579]: W0703 11:12:23.721917 2579 feature_gate.go:328] unrecognized feature gate: BootImageSkewEnforcement Jul 03 11:12:23.731658 ip-10-0-136-0 kubenswrapper[2579]: W0703 11:12:23.721921 2579 feature_gate.go:328] unrecognized feature gate: MixedCPUsAllocation Jul 03 11:12:23.731658 ip-10-0-136-0 kubenswrapper[2579]: W0703 11:12:23.721926 2579 feature_gate.go:328] unrecognized feature gate: ConsolePluginContentSecurityPolicy Jul 03 11:12:23.731658 ip-10-0-136-0 kubenswrapper[2579]: W0703 11:12:23.721932 2579 feature_gate.go:328] unrecognized feature gate: AzureDualStackInstall Jul 03 11:12:23.731658 ip-10-0-136-0 kubenswrapper[2579]: W0703 11:12:23.721936 2579 feature_gate.go:328] unrecognized feature gate: CRDCompatibilityRequirementOperator Jul 03 11:12:23.731658 ip-10-0-136-0 kubenswrapper[2579]: W0703 11:12:23.721941 2579 feature_gate.go:328] unrecognized feature gate: MachineAPIOperatorDisableMachineHealthCheckController Jul 03 11:12:23.731658 ip-10-0-136-0 kubenswrapper[2579]: W0703 11:12:23.721946 2579 feature_gate.go:328] unrecognized feature gate: ImageStreamImportMode Jul 03 11:12:23.731658 ip-10-0-136-0 kubenswrapper[2579]: W0703 11:12:23.721950 2579 feature_gate.go:328] unrecognized feature gate: GatewayAPIController Jul 03 11:12:23.732134 ip-10-0-136-0 kubenswrapper[2579]: W0703 11:12:23.721955 2579 feature_gate.go:328] unrecognized feature gate: ShortCertRotation Jul 03 11:12:23.732134 ip-10-0-136-0 kubenswrapper[2579]: W0703 11:12:23.721959 2579 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesCPMS Jul 03 11:12:23.732134 ip-10-0-136-0 kubenswrapper[2579]: W0703 11:12:23.721964 2579 feature_gate.go:328] unrecognized feature gate: AWSDedicatedHosts Jul 03 11:12:23.732134 ip-10-0-136-0 kubenswrapper[2579]: W0703 11:12:23.721970 2579 feature_gate.go:328] unrecognized feature gate: ClusterMonitoringConfig Jul 03 11:12:23.732134 ip-10-0-136-0 kubenswrapper[2579]: W0703 11:12:23.721975 2579 feature_gate.go:328] unrecognized feature gate: NewOLM Jul 03 11:12:23.732134 ip-10-0-136-0 kubenswrapper[2579]: W0703 11:12:23.721980 2579 feature_gate.go:328] unrecognized feature gate: BootcNodeManagement Jul 03 11:12:23.732134 ip-10-0-136-0 kubenswrapper[2579]: W0703 11:12:23.721984 2579 feature_gate.go:328] unrecognized feature gate: EventTTL Jul 03 11:12:23.732134 ip-10-0-136-0 kubenswrapper[2579]: W0703 11:12:23.721989 2579 feature_gate.go:328] unrecognized feature gate: VSphereMultiDisk Jul 03 11:12:23.732134 ip-10-0-136-0 kubenswrapper[2579]: W0703 11:12:23.721993 2579 feature_gate.go:328] unrecognized feature gate: ManagedBootImages Jul 03 11:12:23.732134 ip-10-0-136-0 kubenswrapper[2579]: W0703 11:12:23.721998 2579 feature_gate.go:328] unrecognized feature gate: AdminNetworkPolicy Jul 03 11:12:23.732134 ip-10-0-136-0 kubenswrapper[2579]: W0703 11:12:23.722002 2579 feature_gate.go:328] unrecognized feature gate: InsightsConfig Jul 03 11:12:23.732134 ip-10-0-136-0 kubenswrapper[2579]: W0703 11:12:23.722007 2579 feature_gate.go:328] unrecognized feature gate: VSphereConfigurableMaxAllowedBlockVolumesPerNode Jul 03 11:12:23.732134 ip-10-0-136-0 kubenswrapper[2579]: W0703 11:12:23.722012 2579 feature_gate.go:328] unrecognized feature gate: NewOLMWebhookProviderOpenshiftServiceCA Jul 03 11:12:23.732134 ip-10-0-136-0 kubenswrapper[2579]: W0703 11:12:23.722016 2579 feature_gate.go:328] unrecognized feature gate: ExternalSnapshotMetadata Jul 03 11:12:23.732134 ip-10-0-136-0 kubenswrapper[2579]: W0703 11:12:23.722021 2579 feature_gate.go:328] unrecognized feature gate: AzureClusterHostedDNSInstall Jul 03 11:12:23.732134 ip-10-0-136-0 kubenswrapper[2579]: W0703 11:12:23.722026 2579 feature_gate.go:328] unrecognized feature gate: NewOLMOwnSingleNamespace Jul 03 11:12:23.732134 ip-10-0-136-0 kubenswrapper[2579]: W0703 11:12:23.722031 2579 feature_gate.go:328] unrecognized feature gate: ClusterVersionOperatorConfiguration Jul 03 11:12:23.732134 ip-10-0-136-0 kubenswrapper[2579]: W0703 11:12:23.722036 2579 feature_gate.go:328] unrecognized feature gate: PinnedImages Jul 03 11:12:23.732134 ip-10-0-136-0 kubenswrapper[2579]: W0703 11:12:23.722040 2579 feature_gate.go:328] unrecognized feature gate: DyanmicServiceEndpointIBMCloud Jul 03 11:12:23.732632 ip-10-0-136-0 kubenswrapper[2579]: I0703 11:12:23.722738 2579 feature_gate.go:384] feature gates: {map[CBORServingAndStorage:false ClientsAllowCBOR:false ClientsPreferCBOR:false EventedPLEG:false ImageVolume:true KMSv1:true MaxUnavailableStatefulSet:false MinimumKubeletVersion:false MutableCSINodeAllocatableCount:false MutatingAdmissionPolicy:false ProcMountType:true RouteExternalCertificate:true SELinuxMount:false ServiceAccountTokenNodeBinding:true StoragePerformantSecurityPolicy:true TranslateStreamCloseWebsocketRequests:false UserNamespacesPodSecurityStandards:true UserNamespacesSupport:true VolumeAttributesClass:true]} Jul 03 11:12:23.732632 ip-10-0-136-0 kubenswrapper[2579]: I0703 11:12:23.730759 2579 server.go:529] "Kubelet version" kubeletVersion="v1.34.8" Jul 03 11:12:23.732632 ip-10-0-136-0 kubenswrapper[2579]: I0703 11:12:23.730787 2579 server.go:531] "Golang settings" GOGC="" GOMAXPROCS="" GOTRACEBACK="" Jul 03 11:12:23.732632 ip-10-0-136-0 kubenswrapper[2579]: I0703 11:12:23.731536 2579 watchdog_linux.go:95] "Systemd watchdog is not enabled" Jul 03 11:12:23.732632 ip-10-0-136-0 kubenswrapper[2579]: I0703 11:12:23.731545 2579 watchdog_linux.go:137] "Systemd watchdog is not enabled or the interval is invalid, so health checking will not be started." Jul 03 11:12:23.732632 ip-10-0-136-0 kubenswrapper[2579]: W0703 11:12:23.731631 2579 feature_gate.go:328] unrecognized feature gate: AWSDedicatedHosts Jul 03 11:12:23.732632 ip-10-0-136-0 kubenswrapper[2579]: W0703 11:12:23.731639 2579 feature_gate.go:328] unrecognized feature gate: ClusterVersionOperatorConfiguration Jul 03 11:12:23.732632 ip-10-0-136-0 kubenswrapper[2579]: W0703 11:12:23.731643 2579 feature_gate.go:328] unrecognized feature gate: ManagedBootImages Jul 03 11:12:23.732632 ip-10-0-136-0 kubenswrapper[2579]: W0703 11:12:23.731646 2579 feature_gate.go:328] unrecognized feature gate: NetworkDiagnosticsConfig Jul 03 11:12:23.732632 ip-10-0-136-0 kubenswrapper[2579]: W0703 11:12:23.731649 2579 feature_gate.go:328] unrecognized feature gate: RouteAdvertisements Jul 03 11:12:23.732632 ip-10-0-136-0 kubenswrapper[2579]: W0703 11:12:23.731653 2579 feature_gate.go:328] unrecognized feature gate: NewOLMPreflightPermissionChecks Jul 03 11:12:23.732632 ip-10-0-136-0 kubenswrapper[2579]: W0703 11:12:23.731655 2579 feature_gate.go:328] unrecognized feature gate: HyperShiftOnlyDynamicResourceAllocation Jul 03 11:12:23.732632 ip-10-0-136-0 kubenswrapper[2579]: W0703 11:12:23.731658 2579 feature_gate.go:328] unrecognized feature gate: DyanmicServiceEndpointIBMCloud Jul 03 11:12:23.732632 ip-10-0-136-0 kubenswrapper[2579]: W0703 11:12:23.731661 2579 feature_gate.go:328] unrecognized feature gate: ConsolePluginContentSecurityPolicy Jul 03 11:12:23.733001 ip-10-0-136-0 kubenswrapper[2579]: W0703 11:12:23.731663 2579 feature_gate.go:328] unrecognized feature gate: BootcNodeManagement Jul 03 11:12:23.733001 ip-10-0-136-0 kubenswrapper[2579]: W0703 11:12:23.731666 2579 feature_gate.go:328] unrecognized feature gate: ImageModeStatusReporting Jul 03 11:12:23.733001 ip-10-0-136-0 kubenswrapper[2579]: W0703 11:12:23.731669 2579 feature_gate.go:328] unrecognized feature gate: VSphereHostVMGroupZonal Jul 03 11:12:23.733001 ip-10-0-136-0 kubenswrapper[2579]: W0703 11:12:23.731672 2579 feature_gate.go:328] unrecognized feature gate: MachineAPIOperatorDisableMachineHealthCheckController Jul 03 11:12:23.733001 ip-10-0-136-0 kubenswrapper[2579]: W0703 11:12:23.731674 2579 feature_gate.go:328] unrecognized feature gate: GCPCustomAPIEndpoints Jul 03 11:12:23.733001 ip-10-0-136-0 kubenswrapper[2579]: W0703 11:12:23.731677 2579 feature_gate.go:328] unrecognized feature gate: HighlyAvailableArbiter Jul 03 11:12:23.733001 ip-10-0-136-0 kubenswrapper[2579]: W0703 11:12:23.731680 2579 feature_gate.go:328] unrecognized feature gate: MachineAPIMigration Jul 03 11:12:23.733001 ip-10-0-136-0 kubenswrapper[2579]: W0703 11:12:23.731682 2579 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesCPMS Jul 03 11:12:23.733001 ip-10-0-136-0 kubenswrapper[2579]: W0703 11:12:23.731685 2579 feature_gate.go:328] unrecognized feature gate: NetworkSegmentation Jul 03 11:12:23.733001 ip-10-0-136-0 kubenswrapper[2579]: W0703 11:12:23.731687 2579 feature_gate.go:328] unrecognized feature gate: CRDCompatibilityRequirementOperator Jul 03 11:12:23.733001 ip-10-0-136-0 kubenswrapper[2579]: W0703 11:12:23.731690 2579 feature_gate.go:328] unrecognized feature gate: CPMSMachineNamePrefix Jul 03 11:12:23.733001 ip-10-0-136-0 kubenswrapper[2579]: W0703 11:12:23.731692 2579 feature_gate.go:328] unrecognized feature gate: NewOLM Jul 03 11:12:23.733001 ip-10-0-136-0 kubenswrapper[2579]: W0703 11:12:23.731694 2579 feature_gate.go:328] unrecognized feature gate: AzureWorkloadIdentity Jul 03 11:12:23.733001 ip-10-0-136-0 kubenswrapper[2579]: W0703 11:12:23.731697 2579 feature_gate.go:328] unrecognized feature gate: PreconfiguredUDNAddresses Jul 03 11:12:23.733001 ip-10-0-136-0 kubenswrapper[2579]: W0703 11:12:23.731699 2579 feature_gate.go:328] unrecognized feature gate: IrreconcilableMachineConfig Jul 03 11:12:23.733001 ip-10-0-136-0 kubenswrapper[2579]: W0703 11:12:23.731701 2579 feature_gate.go:328] unrecognized feature gate: GCPClusterHostedDNS Jul 03 11:12:23.733001 ip-10-0-136-0 kubenswrapper[2579]: W0703 11:12:23.731704 2579 feature_gate.go:328] unrecognized feature gate: AdditionalRoutingCapabilities Jul 03 11:12:23.733001 ip-10-0-136-0 kubenswrapper[2579]: W0703 11:12:23.731706 2579 feature_gate.go:328] unrecognized feature gate: UpgradeStatus Jul 03 11:12:23.733001 ip-10-0-136-0 kubenswrapper[2579]: W0703 11:12:23.731711 2579 feature_gate.go:349] Setting deprecated feature gate KMSv1=true. It will be removed in a future release. Jul 03 11:12:23.733494 ip-10-0-136-0 kubenswrapper[2579]: W0703 11:12:23.731716 2579 feature_gate.go:328] unrecognized feature gate: Example2 Jul 03 11:12:23.733494 ip-10-0-136-0 kubenswrapper[2579]: W0703 11:12:23.731721 2579 feature_gate.go:328] unrecognized feature gate: ExternalOIDCWithUIDAndExtraClaimMappings Jul 03 11:12:23.733494 ip-10-0-136-0 kubenswrapper[2579]: W0703 11:12:23.731724 2579 feature_gate.go:328] unrecognized feature gate: AlibabaPlatform Jul 03 11:12:23.733494 ip-10-0-136-0 kubenswrapper[2579]: W0703 11:12:23.731726 2579 feature_gate.go:328] unrecognized feature gate: BootImageSkewEnforcement Jul 03 11:12:23.733494 ip-10-0-136-0 kubenswrapper[2579]: W0703 11:12:23.731729 2579 feature_gate.go:328] unrecognized feature gate: ClusterAPIMachineManagement Jul 03 11:12:23.733494 ip-10-0-136-0 kubenswrapper[2579]: W0703 11:12:23.731732 2579 feature_gate.go:328] unrecognized feature gate: ClusterMonitoringConfig Jul 03 11:12:23.733494 ip-10-0-136-0 kubenswrapper[2579]: W0703 11:12:23.731734 2579 feature_gate.go:328] unrecognized feature gate: ProvisioningRequestAvailable Jul 03 11:12:23.733494 ip-10-0-136-0 kubenswrapper[2579]: W0703 11:12:23.731737 2579 feature_gate.go:328] unrecognized feature gate: GatewayAPIWithoutOLM Jul 03 11:12:23.733494 ip-10-0-136-0 kubenswrapper[2579]: W0703 11:12:23.731739 2579 feature_gate.go:328] unrecognized feature gate: AzureClusterHostedDNSInstall Jul 03 11:12:23.733494 ip-10-0-136-0 kubenswrapper[2579]: W0703 11:12:23.731742 2579 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesAzure Jul 03 11:12:23.733494 ip-10-0-136-0 kubenswrapper[2579]: W0703 11:12:23.731744 2579 feature_gate.go:328] unrecognized feature gate: KMSEncryptionProvider Jul 03 11:12:23.733494 ip-10-0-136-0 kubenswrapper[2579]: W0703 11:12:23.731747 2579 feature_gate.go:328] unrecognized feature gate: InsightsOnDemandDataGather Jul 03 11:12:23.733494 ip-10-0-136-0 kubenswrapper[2579]: W0703 11:12:23.731750 2579 feature_gate.go:328] unrecognized feature gate: VSphereConfigurableMaxAllowedBlockVolumesPerNode Jul 03 11:12:23.733494 ip-10-0-136-0 kubenswrapper[2579]: W0703 11:12:23.731752 2579 feature_gate.go:328] unrecognized feature gate: VSphereMultiNetworks Jul 03 11:12:23.733494 ip-10-0-136-0 kubenswrapper[2579]: W0703 11:12:23.731755 2579 feature_gate.go:328] unrecognized feature gate: MultiDiskSetup Jul 03 11:12:23.733494 ip-10-0-136-0 kubenswrapper[2579]: W0703 11:12:23.731758 2579 feature_gate.go:328] unrecognized feature gate: NoRegistryClusterInstall Jul 03 11:12:23.733494 ip-10-0-136-0 kubenswrapper[2579]: W0703 11:12:23.731760 2579 feature_gate.go:328] unrecognized feature gate: MachineConfigNodes Jul 03 11:12:23.733494 ip-10-0-136-0 kubenswrapper[2579]: W0703 11:12:23.731762 2579 feature_gate.go:328] unrecognized feature gate: GCPDualStackInstall Jul 03 11:12:23.733494 ip-10-0-136-0 kubenswrapper[2579]: W0703 11:12:23.731765 2579 feature_gate.go:328] unrecognized feature gate: AWSDualStackInstall Jul 03 11:12:23.733494 ip-10-0-136-0 kubenswrapper[2579]: W0703 11:12:23.731768 2579 feature_gate.go:328] unrecognized feature gate: NetworkLiveMigration Jul 03 11:12:23.734001 ip-10-0-136-0 kubenswrapper[2579]: W0703 11:12:23.731770 2579 feature_gate.go:328] unrecognized feature gate: AutomatedEtcdBackup Jul 03 11:12:23.734001 ip-10-0-136-0 kubenswrapper[2579]: W0703 11:12:23.731772 2579 feature_gate.go:328] unrecognized feature gate: EventTTL Jul 03 11:12:23.734001 ip-10-0-136-0 kubenswrapper[2579]: W0703 11:12:23.731775 2579 feature_gate.go:328] unrecognized feature gate: GatewayAPI Jul 03 11:12:23.734001 ip-10-0-136-0 kubenswrapper[2579]: W0703 11:12:23.731777 2579 feature_gate.go:328] unrecognized feature gate: AzureDedicatedHosts Jul 03 11:12:23.734001 ip-10-0-136-0 kubenswrapper[2579]: W0703 11:12:23.731780 2579 feature_gate.go:328] unrecognized feature gate: EtcdBackendQuota Jul 03 11:12:23.734001 ip-10-0-136-0 kubenswrapper[2579]: W0703 11:12:23.731783 2579 feature_gate.go:328] unrecognized feature gate: InsightsConfig Jul 03 11:12:23.734001 ip-10-0-136-0 kubenswrapper[2579]: W0703 11:12:23.731785 2579 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesAWS Jul 03 11:12:23.734001 ip-10-0-136-0 kubenswrapper[2579]: W0703 11:12:23.731790 2579 feature_gate.go:351] Setting GA feature gate ServiceAccountTokenNodeBinding=true. It will be removed in a future release. Jul 03 11:12:23.734001 ip-10-0-136-0 kubenswrapper[2579]: W0703 11:12:23.731793 2579 feature_gate.go:328] unrecognized feature gate: GatewayAPIController Jul 03 11:12:23.734001 ip-10-0-136-0 kubenswrapper[2579]: W0703 11:12:23.731796 2579 feature_gate.go:328] unrecognized feature gate: OSStreams Jul 03 11:12:23.734001 ip-10-0-136-0 kubenswrapper[2579]: W0703 11:12:23.731799 2579 feature_gate.go:328] unrecognized feature gate: NewOLMWebhookProviderOpenshiftServiceCA Jul 03 11:12:23.734001 ip-10-0-136-0 kubenswrapper[2579]: W0703 11:12:23.731801 2579 feature_gate.go:328] unrecognized feature gate: AWSClusterHostedDNS Jul 03 11:12:23.734001 ip-10-0-136-0 kubenswrapper[2579]: W0703 11:12:23.731805 2579 feature_gate.go:328] unrecognized feature gate: MultiArchInstallAzure Jul 03 11:12:23.734001 ip-10-0-136-0 kubenswrapper[2579]: W0703 11:12:23.731808 2579 feature_gate.go:328] unrecognized feature gate: ClusterAPIInstall Jul 03 11:12:23.734001 ip-10-0-136-0 kubenswrapper[2579]: W0703 11:12:23.731811 2579 feature_gate.go:328] unrecognized feature gate: NewOLMCatalogdAPIV1Metas Jul 03 11:12:23.734001 ip-10-0-136-0 kubenswrapper[2579]: W0703 11:12:23.731814 2579 feature_gate.go:328] unrecognized feature gate: BuildCSIVolumes Jul 03 11:12:23.734001 ip-10-0-136-0 kubenswrapper[2579]: W0703 11:12:23.731818 2579 feature_gate.go:328] unrecognized feature gate: PinnedImages Jul 03 11:12:23.734001 ip-10-0-136-0 kubenswrapper[2579]: W0703 11:12:23.731820 2579 feature_gate.go:328] unrecognized feature gate: ShortCertRotation Jul 03 11:12:23.734001 ip-10-0-136-0 kubenswrapper[2579]: W0703 11:12:23.731823 2579 feature_gate.go:328] unrecognized feature gate: GCPClusterHostedDNSInstall Jul 03 11:12:23.734001 ip-10-0-136-0 kubenswrapper[2579]: W0703 11:12:23.731826 2579 feature_gate.go:328] unrecognized feature gate: OnPremDNSRecords Jul 03 11:12:23.734530 ip-10-0-136-0 kubenswrapper[2579]: W0703 11:12:23.731829 2579 feature_gate.go:328] unrecognized feature gate: VSphereMixedNodeEnv Jul 03 11:12:23.734530 ip-10-0-136-0 kubenswrapper[2579]: W0703 11:12:23.731832 2579 feature_gate.go:328] unrecognized feature gate: DNSNameResolver Jul 03 11:12:23.734530 ip-10-0-136-0 kubenswrapper[2579]: W0703 11:12:23.731834 2579 feature_gate.go:328] unrecognized feature gate: ExternalSnapshotMetadata Jul 03 11:12:23.734530 ip-10-0-136-0 kubenswrapper[2579]: W0703 11:12:23.731837 2579 feature_gate.go:328] unrecognized feature gate: ExternalOIDC Jul 03 11:12:23.734530 ip-10-0-136-0 kubenswrapper[2579]: W0703 11:12:23.731839 2579 feature_gate.go:328] unrecognized feature gate: MetricsCollectionProfiles Jul 03 11:12:23.734530 ip-10-0-136-0 kubenswrapper[2579]: W0703 11:12:23.731841 2579 feature_gate.go:328] unrecognized feature gate: GCPCustomAPIEndpointsInstall Jul 03 11:12:23.734530 ip-10-0-136-0 kubenswrapper[2579]: W0703 11:12:23.731844 2579 feature_gate.go:328] unrecognized feature gate: MixedCPUsAllocation Jul 03 11:12:23.734530 ip-10-0-136-0 kubenswrapper[2579]: W0703 11:12:23.731846 2579 feature_gate.go:328] unrecognized feature gate: OVNObservability Jul 03 11:12:23.734530 ip-10-0-136-0 kubenswrapper[2579]: W0703 11:12:23.731849 2579 feature_gate.go:328] unrecognized feature gate: NewOLMBoxCutterRuntime Jul 03 11:12:23.734530 ip-10-0-136-0 kubenswrapper[2579]: W0703 11:12:23.731852 2579 feature_gate.go:328] unrecognized feature gate: KMSEncryption Jul 03 11:12:23.734530 ip-10-0-136-0 kubenswrapper[2579]: W0703 11:12:23.731854 2579 feature_gate.go:328] unrecognized feature gate: AWSClusterHostedDNSInstall Jul 03 11:12:23.734530 ip-10-0-136-0 kubenswrapper[2579]: W0703 11:12:23.731857 2579 feature_gate.go:328] unrecognized feature gate: NutanixMultiSubnets Jul 03 11:12:23.734530 ip-10-0-136-0 kubenswrapper[2579]: W0703 11:12:23.731860 2579 feature_gate.go:328] unrecognized feature gate: ClusterAPIInstallIBMCloud Jul 03 11:12:23.734530 ip-10-0-136-0 kubenswrapper[2579]: W0703 11:12:23.731863 2579 feature_gate.go:328] unrecognized feature gate: IngressControllerDynamicConfigurationManager Jul 03 11:12:23.734530 ip-10-0-136-0 kubenswrapper[2579]: W0703 11:12:23.731865 2579 feature_gate.go:328] unrecognized feature gate: ImageStreamImportMode Jul 03 11:12:23.734530 ip-10-0-136-0 kubenswrapper[2579]: W0703 11:12:23.731868 2579 feature_gate.go:328] unrecognized feature gate: SignatureStores Jul 03 11:12:23.734530 ip-10-0-136-0 kubenswrapper[2579]: W0703 11:12:23.731871 2579 feature_gate.go:328] unrecognized feature gate: VSphereMultiDisk Jul 03 11:12:23.734530 ip-10-0-136-0 kubenswrapper[2579]: W0703 11:12:23.731873 2579 feature_gate.go:328] unrecognized feature gate: AzureMultiDisk Jul 03 11:12:23.734530 ip-10-0-136-0 kubenswrapper[2579]: W0703 11:12:23.731876 2579 feature_gate.go:328] unrecognized feature gate: AdminNetworkPolicy Jul 03 11:12:23.734530 ip-10-0-136-0 kubenswrapper[2579]: W0703 11:12:23.731878 2579 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesvSphere Jul 03 11:12:23.735037 ip-10-0-136-0 kubenswrapper[2579]: W0703 11:12:23.731881 2579 feature_gate.go:328] unrecognized feature gate: SigstoreImageVerification Jul 03 11:12:23.735037 ip-10-0-136-0 kubenswrapper[2579]: W0703 11:12:23.731883 2579 feature_gate.go:328] unrecognized feature gate: AzureDualStackInstall Jul 03 11:12:23.735037 ip-10-0-136-0 kubenswrapper[2579]: W0703 11:12:23.731888 2579 feature_gate.go:351] Setting GA feature gate VolumeAttributesClass=true. It will be removed in a future release. Jul 03 11:12:23.735037 ip-10-0-136-0 kubenswrapper[2579]: W0703 11:12:23.731891 2579 feature_gate.go:328] unrecognized feature gate: SigstoreImageVerificationPKI Jul 03 11:12:23.735037 ip-10-0-136-0 kubenswrapper[2579]: W0703 11:12:23.731894 2579 feature_gate.go:328] unrecognized feature gate: Example Jul 03 11:12:23.735037 ip-10-0-136-0 kubenswrapper[2579]: W0703 11:12:23.731897 2579 feature_gate.go:328] unrecognized feature gate: AWSServiceLBNetworkSecurityGroup Jul 03 11:12:23.735037 ip-10-0-136-0 kubenswrapper[2579]: W0703 11:12:23.731900 2579 feature_gate.go:328] unrecognized feature gate: OpenShiftPodSecurityAdmission Jul 03 11:12:23.735037 ip-10-0-136-0 kubenswrapper[2579]: W0703 11:12:23.731903 2579 feature_gate.go:328] unrecognized feature gate: VolumeGroupSnapshot Jul 03 11:12:23.735037 ip-10-0-136-0 kubenswrapper[2579]: W0703 11:12:23.731906 2579 feature_gate.go:328] unrecognized feature gate: NewOLMOwnSingleNamespace Jul 03 11:12:23.735037 ip-10-0-136-0 kubenswrapper[2579]: W0703 11:12:23.731909 2579 feature_gate.go:328] unrecognized feature gate: ClusterAPIMachineManagementVSphere Jul 03 11:12:23.735037 ip-10-0-136-0 kubenswrapper[2579]: W0703 11:12:23.731911 2579 feature_gate.go:328] unrecognized feature gate: DualReplica Jul 03 11:12:23.735037 ip-10-0-136-0 kubenswrapper[2579]: I0703 11:12:23.731917 2579 feature_gate.go:384] feature gates: {map[CBORServingAndStorage:false ClientsAllowCBOR:false ClientsPreferCBOR:false EventedPLEG:false ImageVolume:true KMSv1:true MaxUnavailableStatefulSet:false MinimumKubeletVersion:false MutableCSINodeAllocatableCount:false MutatingAdmissionPolicy:false ProcMountType:true RouteExternalCertificate:true SELinuxMount:false ServiceAccountTokenNodeBinding:true StoragePerformantSecurityPolicy:true TranslateStreamCloseWebsocketRequests:false UserNamespacesPodSecurityStandards:true UserNamespacesSupport:true VolumeAttributesClass:true]} Jul 03 11:12:23.735037 ip-10-0-136-0 kubenswrapper[2579]: W0703 11:12:23.732068 2579 feature_gate.go:328] unrecognized feature gate: VSphereMixedNodeEnv Jul 03 11:12:23.735037 ip-10-0-136-0 kubenswrapper[2579]: W0703 11:12:23.732073 2579 feature_gate.go:328] unrecognized feature gate: VolumeGroupSnapshot Jul 03 11:12:23.735506 ip-10-0-136-0 kubenswrapper[2579]: W0703 11:12:23.732076 2579 feature_gate.go:328] unrecognized feature gate: SigstoreImageVerificationPKI Jul 03 11:12:23.735506 ip-10-0-136-0 kubenswrapper[2579]: W0703 11:12:23.732078 2579 feature_gate.go:328] unrecognized feature gate: ClusterAPIMachineManagementVSphere Jul 03 11:12:23.735506 ip-10-0-136-0 kubenswrapper[2579]: W0703 11:12:23.732081 2579 feature_gate.go:328] unrecognized feature gate: HighlyAvailableArbiter Jul 03 11:12:23.735506 ip-10-0-136-0 kubenswrapper[2579]: W0703 11:12:23.732084 2579 feature_gate.go:328] unrecognized feature gate: ImageModeStatusReporting Jul 03 11:12:23.735506 ip-10-0-136-0 kubenswrapper[2579]: W0703 11:12:23.732087 2579 feature_gate.go:328] unrecognized feature gate: IrreconcilableMachineConfig Jul 03 11:12:23.735506 ip-10-0-136-0 kubenswrapper[2579]: W0703 11:12:23.732089 2579 feature_gate.go:328] unrecognized feature gate: DNSNameResolver Jul 03 11:12:23.735506 ip-10-0-136-0 kubenswrapper[2579]: W0703 11:12:23.732092 2579 feature_gate.go:328] unrecognized feature gate: ClusterAPIInstallIBMCloud Jul 03 11:12:23.735506 ip-10-0-136-0 kubenswrapper[2579]: W0703 11:12:23.732095 2579 feature_gate.go:328] unrecognized feature gate: AWSClusterHostedDNSInstall Jul 03 11:12:23.735506 ip-10-0-136-0 kubenswrapper[2579]: W0703 11:12:23.732098 2579 feature_gate.go:328] unrecognized feature gate: NetworkLiveMigration Jul 03 11:12:23.735506 ip-10-0-136-0 kubenswrapper[2579]: W0703 11:12:23.732100 2579 feature_gate.go:328] unrecognized feature gate: OVNObservability Jul 03 11:12:23.735506 ip-10-0-136-0 kubenswrapper[2579]: W0703 11:12:23.732103 2579 feature_gate.go:328] unrecognized feature gate: KMSEncryptionProvider Jul 03 11:12:23.735506 ip-10-0-136-0 kubenswrapper[2579]: W0703 11:12:23.732105 2579 feature_gate.go:328] unrecognized feature gate: NewOLMBoxCutterRuntime Jul 03 11:12:23.735506 ip-10-0-136-0 kubenswrapper[2579]: W0703 11:12:23.732109 2579 feature_gate.go:328] unrecognized feature gate: HyperShiftOnlyDynamicResourceAllocation Jul 03 11:12:23.735506 ip-10-0-136-0 kubenswrapper[2579]: W0703 11:12:23.732111 2579 feature_gate.go:328] unrecognized feature gate: VSphereMultiNetworks Jul 03 11:12:23.735506 ip-10-0-136-0 kubenswrapper[2579]: W0703 11:12:23.732114 2579 feature_gate.go:328] unrecognized feature gate: BuildCSIVolumes Jul 03 11:12:23.735506 ip-10-0-136-0 kubenswrapper[2579]: W0703 11:12:23.732116 2579 feature_gate.go:328] unrecognized feature gate: MachineAPIOperatorDisableMachineHealthCheckController Jul 03 11:12:23.735506 ip-10-0-136-0 kubenswrapper[2579]: W0703 11:12:23.732119 2579 feature_gate.go:328] unrecognized feature gate: ClusterAPIMachineManagement Jul 03 11:12:23.735506 ip-10-0-136-0 kubenswrapper[2579]: W0703 11:12:23.732121 2579 feature_gate.go:328] unrecognized feature gate: PinnedImages Jul 03 11:12:23.735506 ip-10-0-136-0 kubenswrapper[2579]: W0703 11:12:23.732124 2579 feature_gate.go:328] unrecognized feature gate: ClusterMonitoringConfig Jul 03 11:12:23.735970 ip-10-0-136-0 kubenswrapper[2579]: W0703 11:12:23.732126 2579 feature_gate.go:328] unrecognized feature gate: AWSDedicatedHosts Jul 03 11:12:23.735970 ip-10-0-136-0 kubenswrapper[2579]: W0703 11:12:23.732129 2579 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesAWS Jul 03 11:12:23.735970 ip-10-0-136-0 kubenswrapper[2579]: W0703 11:12:23.732131 2579 feature_gate.go:328] unrecognized feature gate: NewOLMCatalogdAPIV1Metas Jul 03 11:12:23.735970 ip-10-0-136-0 kubenswrapper[2579]: W0703 11:12:23.732134 2579 feature_gate.go:328] unrecognized feature gate: IngressControllerDynamicConfigurationManager Jul 03 11:12:23.735970 ip-10-0-136-0 kubenswrapper[2579]: W0703 11:12:23.732137 2579 feature_gate.go:328] unrecognized feature gate: NutanixMultiSubnets Jul 03 11:12:23.735970 ip-10-0-136-0 kubenswrapper[2579]: W0703 11:12:23.732140 2579 feature_gate.go:328] unrecognized feature gate: ShortCertRotation Jul 03 11:12:23.735970 ip-10-0-136-0 kubenswrapper[2579]: W0703 11:12:23.732143 2579 feature_gate.go:328] unrecognized feature gate: NewOLMPreflightPermissionChecks Jul 03 11:12:23.735970 ip-10-0-136-0 kubenswrapper[2579]: W0703 11:12:23.732146 2579 feature_gate.go:328] unrecognized feature gate: GatewayAPIController Jul 03 11:12:23.735970 ip-10-0-136-0 kubenswrapper[2579]: W0703 11:12:23.732149 2579 feature_gate.go:328] unrecognized feature gate: Example2 Jul 03 11:12:23.735970 ip-10-0-136-0 kubenswrapper[2579]: W0703 11:12:23.732152 2579 feature_gate.go:328] unrecognized feature gate: VSphereHostVMGroupZonal Jul 03 11:12:23.735970 ip-10-0-136-0 kubenswrapper[2579]: W0703 11:12:23.732154 2579 feature_gate.go:328] unrecognized feature gate: ClusterVersionOperatorConfiguration Jul 03 11:12:23.735970 ip-10-0-136-0 kubenswrapper[2579]: W0703 11:12:23.732156 2579 feature_gate.go:328] unrecognized feature gate: MachineAPIMigration Jul 03 11:12:23.735970 ip-10-0-136-0 kubenswrapper[2579]: W0703 11:12:23.732159 2579 feature_gate.go:328] unrecognized feature gate: AWSDualStackInstall Jul 03 11:12:23.735970 ip-10-0-136-0 kubenswrapper[2579]: W0703 11:12:23.732161 2579 feature_gate.go:328] unrecognized feature gate: ConsolePluginContentSecurityPolicy Jul 03 11:12:23.735970 ip-10-0-136-0 kubenswrapper[2579]: W0703 11:12:23.732164 2579 feature_gate.go:328] unrecognized feature gate: InsightsOnDemandDataGather Jul 03 11:12:23.735970 ip-10-0-136-0 kubenswrapper[2579]: W0703 11:12:23.732166 2579 feature_gate.go:328] unrecognized feature gate: OnPremDNSRecords Jul 03 11:12:23.735970 ip-10-0-136-0 kubenswrapper[2579]: W0703 11:12:23.732169 2579 feature_gate.go:328] unrecognized feature gate: ProvisioningRequestAvailable Jul 03 11:12:23.735970 ip-10-0-136-0 kubenswrapper[2579]: W0703 11:12:23.732171 2579 feature_gate.go:328] unrecognized feature gate: ExternalOIDCWithUIDAndExtraClaimMappings Jul 03 11:12:23.735970 ip-10-0-136-0 kubenswrapper[2579]: W0703 11:12:23.732174 2579 feature_gate.go:328] unrecognized feature gate: AzureWorkloadIdentity Jul 03 11:12:23.736484 ip-10-0-136-0 kubenswrapper[2579]: W0703 11:12:23.732176 2579 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesvSphere Jul 03 11:12:23.736484 ip-10-0-136-0 kubenswrapper[2579]: W0703 11:12:23.732179 2579 feature_gate.go:328] unrecognized feature gate: GCPClusterHostedDNSInstall Jul 03 11:12:23.736484 ip-10-0-136-0 kubenswrapper[2579]: W0703 11:12:23.732181 2579 feature_gate.go:328] unrecognized feature gate: AzureDualStackInstall Jul 03 11:12:23.736484 ip-10-0-136-0 kubenswrapper[2579]: W0703 11:12:23.732184 2579 feature_gate.go:328] unrecognized feature gate: MetricsCollectionProfiles Jul 03 11:12:23.736484 ip-10-0-136-0 kubenswrapper[2579]: W0703 11:12:23.732207 2579 feature_gate.go:328] unrecognized feature gate: AzureMultiDisk Jul 03 11:12:23.736484 ip-10-0-136-0 kubenswrapper[2579]: W0703 11:12:23.732211 2579 feature_gate.go:328] unrecognized feature gate: CPMSMachineNamePrefix Jul 03 11:12:23.736484 ip-10-0-136-0 kubenswrapper[2579]: W0703 11:12:23.732214 2579 feature_gate.go:328] unrecognized feature gate: Example Jul 03 11:12:23.736484 ip-10-0-136-0 kubenswrapper[2579]: W0703 11:12:23.732217 2579 feature_gate.go:328] unrecognized feature gate: CRDCompatibilityRequirementOperator Jul 03 11:12:23.736484 ip-10-0-136-0 kubenswrapper[2579]: W0703 11:12:23.732219 2579 feature_gate.go:328] unrecognized feature gate: NoRegistryClusterInstall Jul 03 11:12:23.736484 ip-10-0-136-0 kubenswrapper[2579]: W0703 11:12:23.732222 2579 feature_gate.go:328] unrecognized feature gate: VSphereMultiDisk Jul 03 11:12:23.736484 ip-10-0-136-0 kubenswrapper[2579]: W0703 11:12:23.732224 2579 feature_gate.go:328] unrecognized feature gate: MultiDiskSetup Jul 03 11:12:23.736484 ip-10-0-136-0 kubenswrapper[2579]: W0703 11:12:23.732227 2579 feature_gate.go:328] unrecognized feature gate: NewOLMWebhookProviderOpenshiftServiceCA Jul 03 11:12:23.736484 ip-10-0-136-0 kubenswrapper[2579]: W0703 11:12:23.732229 2579 feature_gate.go:328] unrecognized feature gate: ExternalOIDC Jul 03 11:12:23.736484 ip-10-0-136-0 kubenswrapper[2579]: W0703 11:12:23.732232 2579 feature_gate.go:328] unrecognized feature gate: KMSEncryption Jul 03 11:12:23.736484 ip-10-0-136-0 kubenswrapper[2579]: W0703 11:12:23.732234 2579 feature_gate.go:328] unrecognized feature gate: PreconfiguredUDNAddresses Jul 03 11:12:23.736484 ip-10-0-136-0 kubenswrapper[2579]: W0703 11:12:23.732238 2579 feature_gate.go:349] Setting deprecated feature gate KMSv1=true. It will be removed in a future release. Jul 03 11:12:23.736484 ip-10-0-136-0 kubenswrapper[2579]: W0703 11:12:23.732241 2579 feature_gate.go:328] unrecognized feature gate: GCPClusterHostedDNS Jul 03 11:12:23.736484 ip-10-0-136-0 kubenswrapper[2579]: W0703 11:12:23.732245 2579 feature_gate.go:328] unrecognized feature gate: AzureDedicatedHosts Jul 03 11:12:23.736484 ip-10-0-136-0 kubenswrapper[2579]: W0703 11:12:23.732248 2579 feature_gate.go:328] unrecognized feature gate: AutomatedEtcdBackup Jul 03 11:12:23.736484 ip-10-0-136-0 kubenswrapper[2579]: W0703 11:12:23.732251 2579 feature_gate.go:328] unrecognized feature gate: ImageStreamImportMode Jul 03 11:12:23.736979 ip-10-0-136-0 kubenswrapper[2579]: W0703 11:12:23.732253 2579 feature_gate.go:328] unrecognized feature gate: NewOLM Jul 03 11:12:23.736979 ip-10-0-136-0 kubenswrapper[2579]: W0703 11:12:23.732257 2579 feature_gate.go:328] unrecognized feature gate: UpgradeStatus Jul 03 11:12:23.736979 ip-10-0-136-0 kubenswrapper[2579]: W0703 11:12:23.732259 2579 feature_gate.go:328] unrecognized feature gate: NetworkSegmentation Jul 03 11:12:23.736979 ip-10-0-136-0 kubenswrapper[2579]: W0703 11:12:23.732262 2579 feature_gate.go:328] unrecognized feature gate: EventTTL Jul 03 11:12:23.736979 ip-10-0-136-0 kubenswrapper[2579]: W0703 11:12:23.732264 2579 feature_gate.go:328] unrecognized feature gate: SignatureStores Jul 03 11:12:23.736979 ip-10-0-136-0 kubenswrapper[2579]: W0703 11:12:23.732266 2579 feature_gate.go:328] unrecognized feature gate: ExternalSnapshotMetadata Jul 03 11:12:23.736979 ip-10-0-136-0 kubenswrapper[2579]: W0703 11:12:23.732269 2579 feature_gate.go:328] unrecognized feature gate: BootcNodeManagement Jul 03 11:12:23.736979 ip-10-0-136-0 kubenswrapper[2579]: W0703 11:12:23.732271 2579 feature_gate.go:328] unrecognized feature gate: AzureClusterHostedDNSInstall Jul 03 11:12:23.736979 ip-10-0-136-0 kubenswrapper[2579]: W0703 11:12:23.732274 2579 feature_gate.go:328] unrecognized feature gate: RouteAdvertisements Jul 03 11:12:23.736979 ip-10-0-136-0 kubenswrapper[2579]: W0703 11:12:23.732277 2579 feature_gate.go:328] unrecognized feature gate: DualReplica Jul 03 11:12:23.736979 ip-10-0-136-0 kubenswrapper[2579]: W0703 11:12:23.732279 2579 feature_gate.go:328] unrecognized feature gate: MachineConfigNodes Jul 03 11:12:23.736979 ip-10-0-136-0 kubenswrapper[2579]: W0703 11:12:23.732282 2579 feature_gate.go:328] unrecognized feature gate: GCPCustomAPIEndpointsInstall Jul 03 11:12:23.736979 ip-10-0-136-0 kubenswrapper[2579]: W0703 11:12:23.732286 2579 feature_gate.go:328] unrecognized feature gate: InsightsConfig Jul 03 11:12:23.736979 ip-10-0-136-0 kubenswrapper[2579]: W0703 11:12:23.732289 2579 feature_gate.go:328] unrecognized feature gate: AlibabaPlatform Jul 03 11:12:23.736979 ip-10-0-136-0 kubenswrapper[2579]: W0703 11:12:23.732291 2579 feature_gate.go:328] unrecognized feature gate: VSphereConfigurableMaxAllowedBlockVolumesPerNode Jul 03 11:12:23.736979 ip-10-0-136-0 kubenswrapper[2579]: W0703 11:12:23.732294 2579 feature_gate.go:328] unrecognized feature gate: GCPDualStackInstall Jul 03 11:12:23.736979 ip-10-0-136-0 kubenswrapper[2579]: W0703 11:12:23.732297 2579 feature_gate.go:328] unrecognized feature gate: ManagedBootImages Jul 03 11:12:23.736979 ip-10-0-136-0 kubenswrapper[2579]: W0703 11:12:23.732300 2579 feature_gate.go:351] Setting GA feature gate ServiceAccountTokenNodeBinding=true. It will be removed in a future release. Jul 03 11:12:23.736979 ip-10-0-136-0 kubenswrapper[2579]: W0703 11:12:23.732303 2579 feature_gate.go:328] unrecognized feature gate: MixedCPUsAllocation Jul 03 11:12:23.736979 ip-10-0-136-0 kubenswrapper[2579]: W0703 11:12:23.732306 2579 feature_gate.go:328] unrecognized feature gate: BootImageSkewEnforcement Jul 03 11:12:23.737492 ip-10-0-136-0 kubenswrapper[2579]: W0703 11:12:23.732309 2579 feature_gate.go:328] unrecognized feature gate: AWSServiceLBNetworkSecurityGroup Jul 03 11:12:23.737492 ip-10-0-136-0 kubenswrapper[2579]: W0703 11:12:23.732311 2579 feature_gate.go:328] unrecognized feature gate: GCPCustomAPIEndpoints Jul 03 11:12:23.737492 ip-10-0-136-0 kubenswrapper[2579]: W0703 11:12:23.732315 2579 feature_gate.go:328] unrecognized feature gate: AdditionalRoutingCapabilities Jul 03 11:12:23.737492 ip-10-0-136-0 kubenswrapper[2579]: W0703 11:12:23.732317 2579 feature_gate.go:328] unrecognized feature gate: AWSClusterHostedDNS Jul 03 11:12:23.737492 ip-10-0-136-0 kubenswrapper[2579]: W0703 11:12:23.732319 2579 feature_gate.go:328] unrecognized feature gate: AdminNetworkPolicy Jul 03 11:12:23.737492 ip-10-0-136-0 kubenswrapper[2579]: W0703 11:12:23.732322 2579 feature_gate.go:328] unrecognized feature gate: NewOLMOwnSingleNamespace Jul 03 11:12:23.737492 ip-10-0-136-0 kubenswrapper[2579]: W0703 11:12:23.732324 2579 feature_gate.go:328] unrecognized feature gate: DyanmicServiceEndpointIBMCloud Jul 03 11:12:23.737492 ip-10-0-136-0 kubenswrapper[2579]: W0703 11:12:23.732327 2579 feature_gate.go:328] unrecognized feature gate: NetworkDiagnosticsConfig Jul 03 11:12:23.737492 ip-10-0-136-0 kubenswrapper[2579]: W0703 11:12:23.732330 2579 feature_gate.go:328] unrecognized feature gate: GatewayAPIWithoutOLM Jul 03 11:12:23.737492 ip-10-0-136-0 kubenswrapper[2579]: W0703 11:12:23.732332 2579 feature_gate.go:328] unrecognized feature gate: SigstoreImageVerification Jul 03 11:12:23.737492 ip-10-0-136-0 kubenswrapper[2579]: W0703 11:12:23.732335 2579 feature_gate.go:328] unrecognized feature gate: OSStreams Jul 03 11:12:23.737492 ip-10-0-136-0 kubenswrapper[2579]: W0703 11:12:23.732338 2579 feature_gate.go:328] unrecognized feature gate: GatewayAPI Jul 03 11:12:23.737492 ip-10-0-136-0 kubenswrapper[2579]: W0703 11:12:23.732340 2579 feature_gate.go:328] unrecognized feature gate: OpenShiftPodSecurityAdmission Jul 03 11:12:23.737492 ip-10-0-136-0 kubenswrapper[2579]: W0703 11:12:23.732343 2579 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesCPMS Jul 03 11:12:23.737492 ip-10-0-136-0 kubenswrapper[2579]: W0703 11:12:23.732346 2579 feature_gate.go:328] unrecognized feature gate: EtcdBackendQuota Jul 03 11:12:23.737492 ip-10-0-136-0 kubenswrapper[2579]: W0703 11:12:23.732348 2579 feature_gate.go:328] unrecognized feature gate: MultiArchInstallAzure Jul 03 11:12:23.737492 ip-10-0-136-0 kubenswrapper[2579]: W0703 11:12:23.732352 2579 feature_gate.go:351] Setting GA feature gate VolumeAttributesClass=true. It will be removed in a future release. Jul 03 11:12:23.737492 ip-10-0-136-0 kubenswrapper[2579]: W0703 11:12:23.732356 2579 feature_gate.go:328] unrecognized feature gate: ClusterAPIInstall Jul 03 11:12:23.737492 ip-10-0-136-0 kubenswrapper[2579]: W0703 11:12:23.732359 2579 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesAzure Jul 03 11:12:23.737969 ip-10-0-136-0 kubenswrapper[2579]: I0703 11:12:23.732364 2579 feature_gate.go:384] feature gates: {map[CBORServingAndStorage:false ClientsAllowCBOR:false ClientsPreferCBOR:false EventedPLEG:false ImageVolume:true KMSv1:true MaxUnavailableStatefulSet:false MinimumKubeletVersion:false MutableCSINodeAllocatableCount:false MutatingAdmissionPolicy:false ProcMountType:true RouteExternalCertificate:true SELinuxMount:false ServiceAccountTokenNodeBinding:true StoragePerformantSecurityPolicy:true TranslateStreamCloseWebsocketRequests:false UserNamespacesPodSecurityStandards:true UserNamespacesSupport:true VolumeAttributesClass:true]} Jul 03 11:12:23.737969 ip-10-0-136-0 kubenswrapper[2579]: I0703 11:12:23.732527 2579 server.go:962] "Client rotation is on, will bootstrap in background" Jul 03 11:12:23.737969 ip-10-0-136-0 kubenswrapper[2579]: I0703 11:12:23.734964 2579 bootstrap.go:101] "Use the bootstrap credentials to request a cert, and set kubeconfig to point to the certificate dir" Jul 03 11:12:23.737969 ip-10-0-136-0 kubenswrapper[2579]: I0703 11:12:23.736499 2579 server.go:1019] "Starting client certificate rotation" Jul 03 11:12:23.737969 ip-10-0-136-0 kubenswrapper[2579]: I0703 11:12:23.736638 2579 certificate_manager.go:422] "Certificate rotation is enabled" logger="kubernetes.io/kube-apiserver-client-kubelet" Jul 03 11:12:23.737969 ip-10-0-136-0 kubenswrapper[2579]: I0703 11:12:23.736684 2579 certificate_manager.go:566] "Rotating certificates" logger="kubernetes.io/kube-apiserver-client-kubelet" Jul 03 11:12:23.739348 ip-10-0-136-0 kubenswrapper[2579]: I0703 11:12:23.739328 2579 dynamic_cafile_content.go:123] "Loaded a new CA Bundle and Verifier" name="client-ca-bundle::/etc/kubernetes/kubelet-ca.crt" Jul 03 11:12:23.741063 ip-10-0-136-0 kubenswrapper[2579]: I0703 11:12:23.741043 2579 dynamic_cafile_content.go:161] "Starting controller" name="client-ca-bundle::/etc/kubernetes/kubelet-ca.crt" Jul 03 11:12:23.755563 ip-10-0-136-0 kubenswrapper[2579]: I0703 11:12:23.755531 2579 log.go:25] "Validated CRI v1 runtime API" Jul 03 11:12:23.758225 ip-10-0-136-0 kubenswrapper[2579]: I0703 11:12:23.758182 2579 log.go:25] "Validated CRI v1 image API" Jul 03 11:12:23.759364 ip-10-0-136-0 kubenswrapper[2579]: I0703 11:12:23.759338 2579 server.go:1429] "Using cgroup driver setting received from the CRI runtime" cgroupDriver="systemd" Jul 03 11:12:23.763543 ip-10-0-136-0 kubenswrapper[2579]: I0703 11:12:23.763510 2579 fs.go:135] Filesystem UUIDs: map[40c72f0b-734d-4a69-93d9-7bfd8f5445e1:/dev/nvme0n1p3 7B77-95E7:/dev/nvme0n1p2 899f90c8-74c2-4c5f-a8f2-f13ae651a0b1:/dev/nvme0n1p4] Jul 03 11:12:23.763543 ip-10-0-136-0 kubenswrapper[2579]: I0703 11:12:23.763537 2579 fs.go:136] Filesystem partitions: map[/dev/nvme0n1p3:{mountpoint:/boot major:259 minor:3 fsType:ext4 blockSize:0} /dev/nvme0n1p4:{mountpoint:/var major:259 minor:4 fsType:xfs blockSize:0} /dev/shm:{mountpoint:/dev/shm major:0 minor:22 fsType:tmpfs blockSize:0} /run:{mountpoint:/run major:0 minor:24 fsType:tmpfs blockSize:0} /tmp:{mountpoint:/tmp major:0 minor:31 fsType:tmpfs blockSize:0} composefs_0-33:{mountpoint:/ major:0 minor:33 fsType:overlay blockSize:0}] Jul 03 11:12:23.766668 ip-10-0-136-0 kubenswrapper[2579]: I0703 11:12:23.766393 2579 reflector.go:439] "Caches populated" logger="kubernetes.io/kube-apiserver-client-kubelet" type="*v1.CertificateSigningRequest" reflector="k8s.io/client-go/tools/watch/informerwatcher.go:162" Jul 03 11:12:23.771738 ip-10-0-136-0 kubenswrapper[2579]: I0703 11:12:23.771600 2579 manager.go:216] Machine: {Timestamp:2026-07-03 11:12:23.770031532 +0000 UTC m=+0.358335857 CPUVendorID:GenuineIntel NumCores:8 NumPhysicalCores:4 NumSockets:1 CpuFrequency:3099709 MemoryCapacity:32812167168 SwapCapacity:0 MemoryByType:map[] NVMInfo:{MemoryModeCapacity:0 AppDirectModeCapacity:0 AvgPowerBudget:0} HugePages:[{PageSize:1048576 NumPages:0} {PageSize:2048 NumPages:0}] MachineID:ec2423c2e58c7c6e185f952bbdfbb9f4 SystemUUID:ec2423c2-e58c-7c6e-185f-952bbdfbb9f4 BootID:2d8f8a0e-c9cd-4fbe-bb03-04ca8597936b Filesystems:[{Device:/dev/nvme0n1p4 DeviceMajor:259 DeviceMinor:4 Capacity:128243970048 Type:vfs Inodes:62651840 HasInodes:true} {Device:composefs_0-33 DeviceMajor:0 DeviceMinor:33 Capacity:6107136 Type:vfs Inodes:18446744073709551615 HasInodes:true} {Device:/tmp DeviceMajor:0 DeviceMinor:31 Capacity:16406085632 Type:vfs Inodes:1048576 HasInodes:true} {Device:/dev/nvme0n1p3 DeviceMajor:259 DeviceMinor:3 Capacity:366869504 Type:vfs Inodes:98304 HasInodes:true} {Device:/dev/shm DeviceMajor:0 DeviceMinor:22 Capacity:16406081536 Type:vfs Inodes:4005391 HasInodes:true} {Device:/run DeviceMajor:0 DeviceMinor:24 Capacity:6562435072 Type:vfs Inodes:819200 HasInodes:true}] DiskMap:map[259:0:{Name:nvme0n1 Major:259 Minor:0 Size:128849018880 Scheduler:none}] NetworkDevices:[{Name:br-ex MacAddress:02:74:5f:08:95:bd Speed:0 Mtu:9001} {Name:ens5 MacAddress:02:74:5f:08:95:bd Speed:0 Mtu:9001} {Name:ovs-system MacAddress:72:f2:f0:ed:39:53 Speed:0 Mtu:1500}] Topology:[{Id:0 Memory:32812167168 HugePages:[{PageSize:1048576 NumPages:0} {PageSize:2048 NumPages:0}] Cores:[{Id:0 Threads:[0 4] Caches:[{Id:0 Size:32768 Type:Data Level:1} {Id:0 Size:32768 Type:Instruction Level:1} {Id:0 Size:1048576 Type:Unified Level:2}] UncoreCaches:[] SocketID:0 BookID: DrawerID:} {Id:1 Threads:[1 5] Caches:[{Id:1 Size:32768 Type:Data Level:1} {Id:1 Size:32768 Type:Instruction Level:1} {Id:1 Size:1048576 Type:Unified Level:2}] UncoreCaches:[] SocketID:0 BookID: DrawerID:} {Id:2 Threads:[2 6] Caches:[{Id:2 Size:32768 Type:Data Level:1} {Id:2 Size:32768 Type:Instruction Level:1} {Id:2 Size:1048576 Type:Unified Level:2}] UncoreCaches:[] SocketID:0 BookID: DrawerID:} {Id:3 Threads:[3 7] Caches:[{Id:3 Size:32768 Type:Data Level:1} {Id:3 Size:32768 Type:Instruction Level:1} {Id:3 Size:1048576 Type:Unified Level:2}] UncoreCaches:[] SocketID:0 BookID: DrawerID:}] Caches:[{Id:0 Size:34603008 Type:Unified Level:3}] Distances:[10]}] CloudProvider:Unknown InstanceType:Unknown InstanceID:None} Jul 03 11:12:23.771738 ip-10-0-136-0 kubenswrapper[2579]: I0703 11:12:23.771721 2579 manager_no_libpfm.go:29] cAdvisor is build without cgo and/or libpfm support. Perf event counters are not available. Jul 03 11:12:23.771913 ip-10-0-136-0 kubenswrapper[2579]: I0703 11:12:23.771833 2579 manager.go:232] Version: {KernelVersion:5.14.0-570.125.1.el9_6.x86_64 ContainerOsVersion:Red Hat Enterprise Linux CoreOS 9.6.20260630-0 (Plow) DockerVersion: DockerAPIVersion: CadvisorVersion: CadvisorRevision:} Jul 03 11:12:23.775138 ip-10-0-136-0 kubenswrapper[2579]: I0703 11:12:23.775080 2579 container_manager_linux.go:273] "Container manager verified user specified cgroup-root exists" cgroupRoot=[] Jul 03 11:12:23.775348 ip-10-0-136-0 kubenswrapper[2579]: I0703 11:12:23.775142 2579 container_manager_linux.go:278] "Creating Container Manager object based on Node Config" nodeConfig={"NodeName":"ip-10-0-136-0.ec2.internal","RuntimeCgroupsName":"/system.slice/crio.service","SystemCgroupsName":"/system.slice","KubeletCgroupsName":"","KubeletOOMScoreAdj":-999,"ContainerRuntime":"","CgroupsPerQOS":true,"CgroupRoot":"/","CgroupDriver":"systemd","KubeletRootDir":"/var/lib/kubelet","ProtectKernelDefaults":true,"KubeReservedCgroupName":"","SystemReservedCgroupName":"","ReservedSystemCPUs":{},"EnforceNodeAllocatable":{"pods":{}},"KubeReserved":null,"SystemReserved":{"cpu":"500m","ephemeral-storage":"1Gi","memory":"1Gi"},"HardEvictionThresholds":[{"Signal":"imagefs.available","Operator":"LessThan","Value":{"Quantity":null,"Percentage":0.15},"GracePeriod":0,"MinReclaim":null},{"Signal":"imagefs.inodesFree","Operator":"LessThan","Value":{"Quantity":null,"Percentage":0.05},"GracePeriod":0,"MinReclaim":null},{"Signal":"memory.available","Operator":"LessThan","Value":{"Quantity":"100Mi","Percentage":0},"GracePeriod":0,"MinReclaim":null},{"Signal":"nodefs.available","Operator":"LessThan","Value":{"Quantity":null,"Percentage":0.1},"GracePeriod":0,"MinReclaim":null},{"Signal":"nodefs.inodesFree","Operator":"LessThan","Value":{"Quantity":null,"Percentage":0.05},"GracePeriod":0,"MinReclaim":null}],"QOSReserved":{},"CPUManagerPolicy":"none","CPUManagerPolicyOptions":null,"TopologyManagerScope":"container","CPUManagerReconcilePeriod":10000000000,"MemoryManagerPolicy":"None","MemoryManagerReservedMemory":null,"PodPidsLimit":4096,"EnforceCPULimits":true,"CPUCFSQuotaPeriod":100000000,"TopologyManagerPolicy":"none","TopologyManagerPolicyOptions":null,"CgroupVersion":2} Jul 03 11:12:23.776006 ip-10-0-136-0 kubenswrapper[2579]: I0703 11:12:23.775991 2579 topology_manager.go:138] "Creating topology manager with none policy" Jul 03 11:12:23.776041 ip-10-0-136-0 kubenswrapper[2579]: I0703 11:12:23.776008 2579 container_manager_linux.go:309] "Creating device plugin manager" Jul 03 11:12:23.776041 ip-10-0-136-0 kubenswrapper[2579]: I0703 11:12:23.776016 2579 manager.go:141] "Creating Device Plugin manager" path="/var/lib/kubelet/device-plugins/kubelet.sock" Jul 03 11:12:23.776749 ip-10-0-136-0 kubenswrapper[2579]: I0703 11:12:23.776732 2579 server.go:74] "Creating device plugin registration server" version="v1beta1" socket="/var/lib/kubelet/device-plugins/kubelet.sock" Jul 03 11:12:23.778226 ip-10-0-136-0 kubenswrapper[2579]: I0703 11:12:23.778210 2579 container_manager_linux.go:318] "Creating Dynamic Resource Allocation (DRA) manager" Jul 03 11:12:23.781658 ip-10-0-136-0 kubenswrapper[2579]: I0703 11:12:23.781640 2579 state_mem.go:36] "Initialized new in-memory state store" Jul 03 11:12:23.781803 ip-10-0-136-0 kubenswrapper[2579]: I0703 11:12:23.781794 2579 server.go:1240] "Using root directory" path="/var/lib/kubelet" Jul 03 11:12:23.781838 ip-10-0-136-0 kubenswrapper[2579]: I0703 11:12:23.781830 2579 kubelet.go:486] "Attempting to sync node with API server" Jul 03 11:12:23.781867 ip-10-0-136-0 kubenswrapper[2579]: I0703 11:12:23.781841 2579 kubelet.go:387] "Adding static pod path" path="/etc/kubernetes/manifests" Jul 03 11:12:23.781867 ip-10-0-136-0 kubenswrapper[2579]: I0703 11:12:23.781856 2579 file.go:69] "Watching path" path="/etc/kubernetes/manifests" Jul 03 11:12:23.781867 ip-10-0-136-0 kubenswrapper[2579]: I0703 11:12:23.781866 2579 kubelet.go:398] "Adding apiserver pod source" Jul 03 11:12:23.781968 ip-10-0-136-0 kubenswrapper[2579]: I0703 11:12:23.781875 2579 apiserver.go:42] "Waiting for node sync before watching apiserver pods" Jul 03 11:12:23.782296 ip-10-0-136-0 kubenswrapper[2579]: I0703 11:12:23.782281 2579 state_mem.go:40] "Initialized new in-memory state store for pod resource information tracking" Jul 03 11:12:23.782343 ip-10-0-136-0 kubenswrapper[2579]: I0703 11:12:23.782310 2579 state_mem.go:40] "Initialized new in-memory state store for pod resource information tracking" Jul 03 11:12:23.784690 ip-10-0-136-0 kubenswrapper[2579]: I0703 11:12:23.784670 2579 kuberuntime_manager.go:301] "Container runtime initialized" containerRuntime="cri-o" version="1.34.8-3.rhaos4.21.git91b4b0b.el9" apiVersion="v1" Jul 03 11:12:23.787026 ip-10-0-136-0 kubenswrapper[2579]: I0703 11:12:23.787009 2579 kubelet.go:960] "Not starting ClusterTrustBundle informer because we are in static kubelet mode or the ClusterTrustBundleProjection featuregate is disabled" Jul 03 11:12:23.787115 ip-10-0-136-0 kubenswrapper[2579]: I0703 11:12:23.787083 2579 kubelet.go:984] "Not starting PodCertificateRequest manager because we are in static kubelet mode or the PodCertificateProjection feature gate is disabled" Jul 03 11:12:23.788607 ip-10-0-136-0 kubenswrapper[2579]: I0703 11:12:23.788593 2579 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/portworx-volume" Jul 03 11:12:23.788678 ip-10-0-136-0 kubenswrapper[2579]: I0703 11:12:23.788611 2579 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/empty-dir" Jul 03 11:12:23.788678 ip-10-0-136-0 kubenswrapper[2579]: I0703 11:12:23.788638 2579 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/git-repo" Jul 03 11:12:23.788678 ip-10-0-136-0 kubenswrapper[2579]: I0703 11:12:23.788644 2579 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/host-path" Jul 03 11:12:23.788678 ip-10-0-136-0 kubenswrapper[2579]: I0703 11:12:23.788655 2579 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/nfs" Jul 03 11:12:23.788678 ip-10-0-136-0 kubenswrapper[2579]: I0703 11:12:23.788661 2579 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/secret" Jul 03 11:12:23.788678 ip-10-0-136-0 kubenswrapper[2579]: I0703 11:12:23.788672 2579 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/iscsi" Jul 03 11:12:23.788678 ip-10-0-136-0 kubenswrapper[2579]: I0703 11:12:23.788677 2579 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/downward-api" Jul 03 11:12:23.788869 ip-10-0-136-0 kubenswrapper[2579]: I0703 11:12:23.788684 2579 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/fc" Jul 03 11:12:23.788869 ip-10-0-136-0 kubenswrapper[2579]: I0703 11:12:23.788690 2579 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/configmap" Jul 03 11:12:23.788869 ip-10-0-136-0 kubenswrapper[2579]: I0703 11:12:23.788699 2579 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/projected" Jul 03 11:12:23.788869 ip-10-0-136-0 kubenswrapper[2579]: I0703 11:12:23.788717 2579 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/local-volume" Jul 03 11:12:23.789108 ip-10-0-136-0 kubenswrapper[2579]: I0703 11:12:23.789088 2579 csr.go:274] "Certificate signing request is approved, waiting to be issued" logger="kubernetes.io/kube-apiserver-client-kubelet" csr="csr-grsss" Jul 03 11:12:23.789432 ip-10-0-136-0 kubenswrapper[2579]: I0703 11:12:23.789421 2579 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/csi" Jul 03 11:12:23.789464 ip-10-0-136-0 kubenswrapper[2579]: I0703 11:12:23.789433 2579 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/image" Jul 03 11:12:23.790947 ip-10-0-136-0 kubenswrapper[2579]: E0703 11:12:23.790911 2579 reflector.go:205] "Failed to watch" err="failed to list *v1.Service: services is forbidden: User \"system:anonymous\" cannot list resource \"services\" in API group \"\" at the cluster scope" logger="UnhandledError" reflector="k8s.io/client-go/informers/factory.go:160" type="*v1.Service" Jul 03 11:12:23.791072 ip-10-0-136-0 kubenswrapper[2579]: E0703 11:12:23.790992 2579 reflector.go:205] "Failed to watch" err="failed to list *v1.Node: nodes \"ip-10-0-136-0.ec2.internal\" is forbidden: User \"system:anonymous\" cannot list resource \"nodes\" in API group \"\" at the cluster scope" logger="UnhandledError" reflector="k8s.io/client-go/informers/factory.go:160" type="*v1.Node" Jul 03 11:12:23.793081 ip-10-0-136-0 kubenswrapper[2579]: I0703 11:12:23.793058 2579 csi_plugin.go:990] Failed to contact API server when waiting for CSINode publishing: csinodes.storage.k8s.io "ip-10-0-136-0.ec2.internal" is forbidden: User "system:anonymous" cannot get resource "csinodes" in API group "storage.k8s.io" at the cluster scope Jul 03 11:12:23.793175 ip-10-0-136-0 kubenswrapper[2579]: I0703 11:12:23.793163 2579 server.go:1268] "Started kubelet" Jul 03 11:12:23.793274 ip-10-0-136-0 kubenswrapper[2579]: I0703 11:12:23.793244 2579 server.go:180] "Starting to listen" address="0.0.0.0" port=10250 Jul 03 11:12:23.793776 ip-10-0-136-0 kubenswrapper[2579]: I0703 11:12:23.793738 2579 ratelimit.go:56] "Setting rate limiting for endpoint" service="podresources" qps=100 burstTokens=10 Jul 03 11:12:23.793883 ip-10-0-136-0 kubenswrapper[2579]: I0703 11:12:23.793795 2579 server_v1.go:49] "podresources" method="list" useActivePods=true Jul 03 11:12:23.794293 ip-10-0-136-0 systemd[1]: Started Kubernetes Kubelet. Jul 03 11:12:23.795623 ip-10-0-136-0 kubenswrapper[2579]: I0703 11:12:23.795605 2579 server.go:249] "Starting to serve the podresources API" endpoint="unix:/var/lib/kubelet/pod-resources/kubelet.sock" Jul 03 11:12:23.796324 ip-10-0-136-0 kubenswrapper[2579]: I0703 11:12:23.796310 2579 server.go:310] "Adding debug handlers to kubelet server" Jul 03 11:12:23.797955 ip-10-0-136-0 kubenswrapper[2579]: I0703 11:12:23.797933 2579 csr.go:270] "Certificate signing request is issued" logger="kubernetes.io/kube-apiserver-client-kubelet" csr="csr-grsss" Jul 03 11:12:23.801164 ip-10-0-136-0 kubenswrapper[2579]: E0703 11:12:23.801129 2579 kubelet.go:1638] "Image garbage collection failed once. Stats initialization may not have completed yet" err="failed to get imageFs info: unable to find data in memory cache" Jul 03 11:12:23.801527 ip-10-0-136-0 kubenswrapper[2579]: E0703 11:12:23.800489 2579 event.go:359] "Server rejected event (will not retry!)" err="events is forbidden: User \"system:anonymous\" cannot create resource \"events\" in API group \"\" in the namespace \"default\"" event="&Event{ObjectMeta:{ip-10-0-136-0.ec2.internal.18bec33712e8655e default 0 0001-01-01 00:00:00 +0000 UTC map[] map[] [] [] []},InvolvedObject:ObjectReference{Kind:Node,Namespace:,Name:ip-10-0-136-0.ec2.internal,UID:ip-10-0-136-0.ec2.internal,APIVersion:,ResourceVersion:,FieldPath:,},Reason:Starting,Message:Starting kubelet.,Source:EventSource{Component:kubelet,Host:ip-10-0-136-0.ec2.internal,},FirstTimestamp:2026-07-03 11:12:23.793132894 +0000 UTC m=+0.381437204,LastTimestamp:2026-07-03 11:12:23.793132894 +0000 UTC m=+0.381437204,Count:1,Type:Normal,EventTime:0001-01-01 00:00:00 +0000 UTC,Series:nil,Action:,Related:nil,ReportingController:kubelet,ReportingInstance:ip-10-0-136-0.ec2.internal,}" Jul 03 11:12:23.802174 ip-10-0-136-0 kubenswrapper[2579]: I0703 11:12:23.802156 2579 fs_resource_analyzer.go:67] "Starting FS ResourceAnalyzer" Jul 03 11:12:23.802288 ip-10-0-136-0 kubenswrapper[2579]: I0703 11:12:23.802176 2579 certificate_manager.go:422] "Certificate rotation is enabled" logger="kubernetes.io/kubelet-serving" Jul 03 11:12:23.802288 ip-10-0-136-0 kubenswrapper[2579]: E0703 11:12:23.802256 2579 kubelet_node_status.go:454] "Error getting the current node from lister" err="node \"ip-10-0-136-0.ec2.internal\" not found" Jul 03 11:12:23.803363 ip-10-0-136-0 kubenswrapper[2579]: I0703 11:12:23.803211 2579 desired_state_of_world_populator.go:146] "Desired state populator starts to run" Jul 03 11:12:23.803479 ip-10-0-136-0 kubenswrapper[2579]: I0703 11:12:23.803217 2579 volume_manager.go:311] "The desired_state_of_world populator starts" Jul 03 11:12:23.803556 ip-10-0-136-0 kubenswrapper[2579]: I0703 11:12:23.803545 2579 volume_manager.go:313] "Starting Kubelet Volume Manager" Jul 03 11:12:23.803828 ip-10-0-136-0 kubenswrapper[2579]: I0703 11:12:23.803814 2579 reconstruct.go:97] "Volume reconstruction finished" Jul 03 11:12:23.803930 ip-10-0-136-0 kubenswrapper[2579]: I0703 11:12:23.803920 2579 reconciler.go:29] "Reconciler: start to sync state" Jul 03 11:12:23.804595 ip-10-0-136-0 kubenswrapper[2579]: I0703 11:12:23.804580 2579 factory.go:55] Registering systemd factory Jul 03 11:12:23.804719 ip-10-0-136-0 kubenswrapper[2579]: I0703 11:12:23.804710 2579 factory.go:223] Registration of the systemd container factory successfully Jul 03 11:12:23.805099 ip-10-0-136-0 kubenswrapper[2579]: I0703 11:12:23.805082 2579 factory.go:153] Registering CRI-O factory Jul 03 11:12:23.805171 ip-10-0-136-0 kubenswrapper[2579]: I0703 11:12:23.805103 2579 factory.go:223] Registration of the crio container factory successfully Jul 03 11:12:23.805171 ip-10-0-136-0 kubenswrapper[2579]: I0703 11:12:23.805168 2579 factory.go:221] Registration of the containerd container factory failed: unable to create containerd client: containerd: cannot unix dial containerd api service: dial unix /run/containerd/containerd.sock: connect: no such file or directory Jul 03 11:12:23.805297 ip-10-0-136-0 kubenswrapper[2579]: I0703 11:12:23.805223 2579 factory.go:103] Registering Raw factory Jul 03 11:12:23.805297 ip-10-0-136-0 kubenswrapper[2579]: I0703 11:12:23.805242 2579 manager.go:1196] Started watching for new ooms in manager Jul 03 11:12:23.805963 ip-10-0-136-0 kubenswrapper[2579]: I0703 11:12:23.805946 2579 manager.go:349] Starting recovery of all containers Jul 03 11:12:23.807456 ip-10-0-136-0 kubenswrapper[2579]: I0703 11:12:23.807424 2579 reflector.go:439] "Caches populated" type="*v1.CSIDriver" reflector="k8s.io/client-go/informers/factory.go:160" Jul 03 11:12:23.810515 ip-10-0-136-0 kubenswrapper[2579]: E0703 11:12:23.810479 2579 nodelease.go:49] "Failed to get node when trying to set owner ref to the node lease" err="nodes \"ip-10-0-136-0.ec2.internal\" not found" node="ip-10-0-136-0.ec2.internal" Jul 03 11:12:23.818536 ip-10-0-136-0 kubenswrapper[2579]: I0703 11:12:23.818338 2579 manager.go:354] Recovery completed Jul 03 11:12:23.823448 ip-10-0-136-0 kubenswrapper[2579]: I0703 11:12:23.823431 2579 kubelet_node_status.go:400] "Setting node annotation to enable volume controller attach/detach" Jul 03 11:12:23.826790 ip-10-0-136-0 kubenswrapper[2579]: I0703 11:12:23.826770 2579 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-136-0.ec2.internal" event="NodeHasSufficientMemory" Jul 03 11:12:23.826909 ip-10-0-136-0 kubenswrapper[2579]: I0703 11:12:23.826809 2579 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-136-0.ec2.internal" event="NodeHasNoDiskPressure" Jul 03 11:12:23.826909 ip-10-0-136-0 kubenswrapper[2579]: I0703 11:12:23.826819 2579 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-136-0.ec2.internal" event="NodeHasSufficientPID" Jul 03 11:12:23.827496 ip-10-0-136-0 kubenswrapper[2579]: I0703 11:12:23.827478 2579 cpu_manager.go:222] "Starting CPU manager" policy="none" Jul 03 11:12:23.827496 ip-10-0-136-0 kubenswrapper[2579]: I0703 11:12:23.827496 2579 cpu_manager.go:223] "Reconciling" reconcilePeriod="10s" Jul 03 11:12:23.827616 ip-10-0-136-0 kubenswrapper[2579]: I0703 11:12:23.827513 2579 kubelet_node_status.go:400] "Setting node annotation to enable volume controller attach/detach" Jul 03 11:12:23.827616 ip-10-0-136-0 kubenswrapper[2579]: I0703 11:12:23.827518 2579 state_mem.go:36] "Initialized new in-memory state store" Jul 03 11:12:23.828597 ip-10-0-136-0 kubenswrapper[2579]: I0703 11:12:23.828580 2579 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-136-0.ec2.internal" event="NodeHasSufficientMemory" Jul 03 11:12:23.828665 ip-10-0-136-0 kubenswrapper[2579]: I0703 11:12:23.828615 2579 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-136-0.ec2.internal" event="NodeHasNoDiskPressure" Jul 03 11:12:23.828665 ip-10-0-136-0 kubenswrapper[2579]: I0703 11:12:23.828629 2579 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-136-0.ec2.internal" event="NodeHasSufficientPID" Jul 03 11:12:23.828940 ip-10-0-136-0 kubenswrapper[2579]: I0703 11:12:23.828925 2579 policy_none.go:49] "None policy: Start" Jul 03 11:12:23.828993 ip-10-0-136-0 kubenswrapper[2579]: I0703 11:12:23.828946 2579 memory_manager.go:187] "Starting memorymanager" policy="None" Jul 03 11:12:23.828993 ip-10-0-136-0 kubenswrapper[2579]: I0703 11:12:23.828958 2579 state_mem.go:36] "Initializing new in-memory state store" logger="Memory Manager state checkpoint" Jul 03 11:12:23.830165 ip-10-0-136-0 kubenswrapper[2579]: I0703 11:12:23.830154 2579 policy_none.go:47] "Start" Jul 03 11:12:23.872102 ip-10-0-136-0 kubenswrapper[2579]: I0703 11:12:23.872081 2579 manager.go:337] "Starting Device Plugin manager" Jul 03 11:12:23.872269 ip-10-0-136-0 kubenswrapper[2579]: E0703 11:12:23.872126 2579 manager.go:513] "Failed to read data from checkpoint" err="checkpoint is not found" checkpoint="kubelet_internal_checkpoint" Jul 03 11:12:23.872269 ip-10-0-136-0 kubenswrapper[2579]: I0703 11:12:23.872138 2579 server.go:87] "Starting device plugin registration server" Jul 03 11:12:23.872484 ip-10-0-136-0 kubenswrapper[2579]: I0703 11:12:23.872469 2579 eviction_manager.go:189] "Eviction manager: starting control loop" Jul 03 11:12:23.872552 ip-10-0-136-0 kubenswrapper[2579]: I0703 11:12:23.872485 2579 container_log_manager.go:146] "Initializing container log rotate workers" workers=1 monitorPeriod="10s" Jul 03 11:12:23.872634 ip-10-0-136-0 kubenswrapper[2579]: I0703 11:12:23.872618 2579 plugin_watcher.go:51] "Plugin Watcher Start" path="/var/lib/kubelet/plugins_registry" Jul 03 11:12:23.872717 ip-10-0-136-0 kubenswrapper[2579]: I0703 11:12:23.872699 2579 plugin_manager.go:116] "The desired_state_of_world populator (plugin watcher) starts" Jul 03 11:12:23.872717 ip-10-0-136-0 kubenswrapper[2579]: I0703 11:12:23.872713 2579 plugin_manager.go:118] "Starting Kubelet Plugin Manager" Jul 03 11:12:23.873361 ip-10-0-136-0 kubenswrapper[2579]: E0703 11:12:23.873336 2579 eviction_manager.go:267] "eviction manager: failed to check if we have separate container filesystem. Ignoring." err="non-existent label \"crio-containers\"" Jul 03 11:12:23.873464 ip-10-0-136-0 kubenswrapper[2579]: E0703 11:12:23.873402 2579 eviction_manager.go:292] "Eviction manager: failed to get summary stats" err="failed to get node info: node \"ip-10-0-136-0.ec2.internal\" not found" Jul 03 11:12:23.973520 ip-10-0-136-0 kubenswrapper[2579]: I0703 11:12:23.973429 2579 kubelet_node_status.go:400] "Setting node annotation to enable volume controller attach/detach" Jul 03 11:12:23.974891 ip-10-0-136-0 kubenswrapper[2579]: I0703 11:12:23.974875 2579 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-136-0.ec2.internal" event="NodeHasSufficientMemory" Jul 03 11:12:23.974987 ip-10-0-136-0 kubenswrapper[2579]: I0703 11:12:23.974915 2579 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-136-0.ec2.internal" event="NodeHasNoDiskPressure" Jul 03 11:12:23.974987 ip-10-0-136-0 kubenswrapper[2579]: I0703 11:12:23.974926 2579 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-136-0.ec2.internal" event="NodeHasSufficientPID" Jul 03 11:12:23.974987 ip-10-0-136-0 kubenswrapper[2579]: I0703 11:12:23.974968 2579 kubelet_node_status.go:78] "Attempting to register node" node="ip-10-0-136-0.ec2.internal" Jul 03 11:12:23.981239 ip-10-0-136-0 kubenswrapper[2579]: I0703 11:12:23.981208 2579 kubelet_node_status.go:81] "Successfully registered node" node="ip-10-0-136-0.ec2.internal" Jul 03 11:12:23.981369 ip-10-0-136-0 kubenswrapper[2579]: E0703 11:12:23.981250 2579 kubelet_node_status.go:536] "Error updating node status, will retry" err="error getting node \"ip-10-0-136-0.ec2.internal\": node \"ip-10-0-136-0.ec2.internal\" not found" Jul 03 11:12:23.983452 ip-10-0-136-0 kubenswrapper[2579]: I0703 11:12:23.983408 2579 kubelet_network_linux.go:54] "Initialized iptables rules." protocol="IPv4" Jul 03 11:12:23.985087 ip-10-0-136-0 kubenswrapper[2579]: I0703 11:12:23.985066 2579 kubelet_network_linux.go:54] "Initialized iptables rules." protocol="IPv6" Jul 03 11:12:23.985169 ip-10-0-136-0 kubenswrapper[2579]: I0703 11:12:23.985109 2579 status_manager.go:244] "Starting to sync pod status with apiserver" Jul 03 11:12:23.985169 ip-10-0-136-0 kubenswrapper[2579]: I0703 11:12:23.985136 2579 kubelet.go:2448] "Starting kubelet main sync loop" Jul 03 11:12:23.985293 ip-10-0-136-0 kubenswrapper[2579]: E0703 11:12:23.985179 2579 kubelet.go:2472] "Skipping pod synchronization" err="PLEG is not healthy: pleg has yet to be successful" Jul 03 11:12:23.987566 ip-10-0-136-0 kubenswrapper[2579]: I0703 11:12:23.987536 2579 reflector.go:439] "Caches populated" type="*v1.RuntimeClass" reflector="k8s.io/client-go/informers/factory.go:160" Jul 03 11:12:23.999685 ip-10-0-136-0 kubenswrapper[2579]: E0703 11:12:23.999655 2579 kubelet_node_status.go:454] "Error getting the current node from lister" err="node \"ip-10-0-136-0.ec2.internal\" not found" Jul 03 11:12:24.086061 ip-10-0-136-0 kubenswrapper[2579]: I0703 11:12:24.086015 2579 kubelet.go:2535] "SyncLoop ADD" source="file" pods=["openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-136-0.ec2.internal","kube-system/kube-apiserver-proxy-ip-10-0-136-0.ec2.internal"] Jul 03 11:12:24.086277 ip-10-0-136-0 kubenswrapper[2579]: I0703 11:12:24.086142 2579 kubelet_node_status.go:400] "Setting node annotation to enable volume controller attach/detach" Jul 03 11:12:24.088020 ip-10-0-136-0 kubenswrapper[2579]: I0703 11:12:24.087990 2579 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-136-0.ec2.internal" event="NodeHasSufficientMemory" Jul 03 11:12:24.088166 ip-10-0-136-0 kubenswrapper[2579]: I0703 11:12:24.088039 2579 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-136-0.ec2.internal" event="NodeHasNoDiskPressure" Jul 03 11:12:24.088166 ip-10-0-136-0 kubenswrapper[2579]: I0703 11:12:24.088054 2579 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-136-0.ec2.internal" event="NodeHasSufficientPID" Jul 03 11:12:24.089648 ip-10-0-136-0 kubenswrapper[2579]: I0703 11:12:24.089625 2579 kubelet_node_status.go:400] "Setting node annotation to enable volume controller attach/detach" Jul 03 11:12:24.089779 ip-10-0-136-0 kubenswrapper[2579]: I0703 11:12:24.089731 2579 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-136-0.ec2.internal" Jul 03 11:12:24.089779 ip-10-0-136-0 kubenswrapper[2579]: I0703 11:12:24.089762 2579 kubelet_node_status.go:400] "Setting node annotation to enable volume controller attach/detach" Jul 03 11:12:24.090568 ip-10-0-136-0 kubenswrapper[2579]: I0703 11:12:24.090546 2579 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-136-0.ec2.internal" event="NodeHasSufficientMemory" Jul 03 11:12:24.090568 ip-10-0-136-0 kubenswrapper[2579]: I0703 11:12:24.090562 2579 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-136-0.ec2.internal" event="NodeHasSufficientMemory" Jul 03 11:12:24.090756 ip-10-0-136-0 kubenswrapper[2579]: I0703 11:12:24.090584 2579 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-136-0.ec2.internal" event="NodeHasNoDiskPressure" Jul 03 11:12:24.090756 ip-10-0-136-0 kubenswrapper[2579]: I0703 11:12:24.090585 2579 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-136-0.ec2.internal" event="NodeHasNoDiskPressure" Jul 03 11:12:24.090756 ip-10-0-136-0 kubenswrapper[2579]: I0703 11:12:24.090593 2579 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-136-0.ec2.internal" event="NodeHasSufficientPID" Jul 03 11:12:24.090756 ip-10-0-136-0 kubenswrapper[2579]: I0703 11:12:24.090600 2579 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-136-0.ec2.internal" event="NodeHasSufficientPID" Jul 03 11:12:24.091867 ip-10-0-136-0 kubenswrapper[2579]: I0703 11:12:24.091847 2579 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/kube-apiserver-proxy-ip-10-0-136-0.ec2.internal" Jul 03 11:12:24.091969 ip-10-0-136-0 kubenswrapper[2579]: I0703 11:12:24.091882 2579 kubelet_node_status.go:400] "Setting node annotation to enable volume controller attach/detach" Jul 03 11:12:24.092753 ip-10-0-136-0 kubenswrapper[2579]: I0703 11:12:24.092735 2579 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-136-0.ec2.internal" event="NodeHasSufficientMemory" Jul 03 11:12:24.092839 ip-10-0-136-0 kubenswrapper[2579]: I0703 11:12:24.092772 2579 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-136-0.ec2.internal" event="NodeHasNoDiskPressure" Jul 03 11:12:24.092839 ip-10-0-136-0 kubenswrapper[2579]: I0703 11:12:24.092786 2579 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-136-0.ec2.internal" event="NodeHasSufficientPID" Jul 03 11:12:24.099777 ip-10-0-136-0 kubenswrapper[2579]: E0703 11:12:24.099751 2579 kubelet_node_status.go:454] "Error getting the current node from lister" err="node \"ip-10-0-136-0.ec2.internal\" not found" Jul 03 11:12:24.105151 ip-10-0-136-0 kubenswrapper[2579]: I0703 11:12:24.105123 2579 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"etc-kube\" (UniqueName: \"kubernetes.io/host-path/e77ab5b2fa375ad8c98ab61ef281b548-etc-kube\") pod \"kube-rbac-proxy-crio-ip-10-0-136-0.ec2.internal\" (UID: \"e77ab5b2fa375ad8c98ab61ef281b548\") " pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-136-0.ec2.internal" Jul 03 11:12:24.105285 ip-10-0-136-0 kubenswrapper[2579]: I0703 11:12:24.105161 2579 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"var-lib-kubelet\" (UniqueName: \"kubernetes.io/host-path/e77ab5b2fa375ad8c98ab61ef281b548-var-lib-kubelet\") pod \"kube-rbac-proxy-crio-ip-10-0-136-0.ec2.internal\" (UID: \"e77ab5b2fa375ad8c98ab61ef281b548\") " pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-136-0.ec2.internal" Jul 03 11:12:24.105285 ip-10-0-136-0 kubenswrapper[2579]: I0703 11:12:24.105201 2579 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"config\" (UniqueName: \"kubernetes.io/host-path/e9fe0b17840ecce76a8985c0f09920db-config\") pod \"kube-apiserver-proxy-ip-10-0-136-0.ec2.internal\" (UID: \"e9fe0b17840ecce76a8985c0f09920db\") " pod="kube-system/kube-apiserver-proxy-ip-10-0-136-0.ec2.internal" Jul 03 11:12:24.120133 ip-10-0-136-0 kubenswrapper[2579]: E0703 11:12:24.120107 2579 kubelet.go:3244] "No need to create a mirror pod, since failed to get node info from the cluster" err="node \"ip-10-0-136-0.ec2.internal\" not found" node="ip-10-0-136-0.ec2.internal" Jul 03 11:12:24.125037 ip-10-0-136-0 kubenswrapper[2579]: E0703 11:12:24.125018 2579 kubelet.go:3244] "No need to create a mirror pod, since failed to get node info from the cluster" err="node \"ip-10-0-136-0.ec2.internal\" not found" node="ip-10-0-136-0.ec2.internal" Jul 03 11:12:24.200239 ip-10-0-136-0 kubenswrapper[2579]: E0703 11:12:24.200173 2579 kubelet_node_status.go:454] "Error getting the current node from lister" err="node \"ip-10-0-136-0.ec2.internal\" not found" Jul 03 11:12:24.205542 ip-10-0-136-0 kubenswrapper[2579]: I0703 11:12:24.205523 2579 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"etc-kube\" (UniqueName: \"kubernetes.io/host-path/e77ab5b2fa375ad8c98ab61ef281b548-etc-kube\") pod \"kube-rbac-proxy-crio-ip-10-0-136-0.ec2.internal\" (UID: \"e77ab5b2fa375ad8c98ab61ef281b548\") " pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-136-0.ec2.internal" Jul 03 11:12:24.205625 ip-10-0-136-0 kubenswrapper[2579]: I0703 11:12:24.205561 2579 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"var-lib-kubelet\" (UniqueName: \"kubernetes.io/host-path/e77ab5b2fa375ad8c98ab61ef281b548-var-lib-kubelet\") pod \"kube-rbac-proxy-crio-ip-10-0-136-0.ec2.internal\" (UID: \"e77ab5b2fa375ad8c98ab61ef281b548\") " pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-136-0.ec2.internal" Jul 03 11:12:24.205625 ip-10-0-136-0 kubenswrapper[2579]: I0703 11:12:24.205580 2579 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"config\" (UniqueName: \"kubernetes.io/host-path/e9fe0b17840ecce76a8985c0f09920db-config\") pod \"kube-apiserver-proxy-ip-10-0-136-0.ec2.internal\" (UID: \"e9fe0b17840ecce76a8985c0f09920db\") " pod="kube-system/kube-apiserver-proxy-ip-10-0-136-0.ec2.internal" Jul 03 11:12:24.205704 ip-10-0-136-0 kubenswrapper[2579]: I0703 11:12:24.205623 2579 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"config\" (UniqueName: \"kubernetes.io/host-path/e9fe0b17840ecce76a8985c0f09920db-config\") pod \"kube-apiserver-proxy-ip-10-0-136-0.ec2.internal\" (UID: \"e9fe0b17840ecce76a8985c0f09920db\") " pod="kube-system/kube-apiserver-proxy-ip-10-0-136-0.ec2.internal" Jul 03 11:12:24.205704 ip-10-0-136-0 kubenswrapper[2579]: I0703 11:12:24.205641 2579 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"etc-kube\" (UniqueName: \"kubernetes.io/host-path/e77ab5b2fa375ad8c98ab61ef281b548-etc-kube\") pod \"kube-rbac-proxy-crio-ip-10-0-136-0.ec2.internal\" (UID: \"e77ab5b2fa375ad8c98ab61ef281b548\") " pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-136-0.ec2.internal" Jul 03 11:12:24.205704 ip-10-0-136-0 kubenswrapper[2579]: I0703 11:12:24.205651 2579 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"var-lib-kubelet\" (UniqueName: \"kubernetes.io/host-path/e77ab5b2fa375ad8c98ab61ef281b548-var-lib-kubelet\") pod \"kube-rbac-proxy-crio-ip-10-0-136-0.ec2.internal\" (UID: \"e77ab5b2fa375ad8c98ab61ef281b548\") " pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-136-0.ec2.internal" Jul 03 11:12:24.300998 ip-10-0-136-0 kubenswrapper[2579]: E0703 11:12:24.300957 2579 kubelet_node_status.go:454] "Error getting the current node from lister" err="node \"ip-10-0-136-0.ec2.internal\" not found" Jul 03 11:12:24.401603 ip-10-0-136-0 kubenswrapper[2579]: E0703 11:12:24.401569 2579 kubelet_node_status.go:454] "Error getting the current node from lister" err="node \"ip-10-0-136-0.ec2.internal\" not found" Jul 03 11:12:24.424312 ip-10-0-136-0 kubenswrapper[2579]: I0703 11:12:24.424281 2579 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-136-0.ec2.internal" Jul 03 11:12:24.428958 ip-10-0-136-0 kubenswrapper[2579]: I0703 11:12:24.428927 2579 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/kube-apiserver-proxy-ip-10-0-136-0.ec2.internal" Jul 03 11:12:24.502042 ip-10-0-136-0 kubenswrapper[2579]: E0703 11:12:24.502001 2579 kubelet_node_status.go:454] "Error getting the current node from lister" err="node \"ip-10-0-136-0.ec2.internal\" not found" Jul 03 11:12:24.602639 ip-10-0-136-0 kubenswrapper[2579]: E0703 11:12:24.602544 2579 kubelet_node_status.go:454] "Error getting the current node from lister" err="node \"ip-10-0-136-0.ec2.internal\" not found" Jul 03 11:12:24.703170 ip-10-0-136-0 kubenswrapper[2579]: E0703 11:12:24.703129 2579 kubelet_node_status.go:454] "Error getting the current node from lister" err="node \"ip-10-0-136-0.ec2.internal\" not found" Jul 03 11:12:24.735945 ip-10-0-136-0 kubenswrapper[2579]: I0703 11:12:24.735893 2579 transport.go:147] "Certificate rotation detected, shutting down client connections to start using new credentials" Jul 03 11:12:24.736613 ip-10-0-136-0 kubenswrapper[2579]: I0703 11:12:24.736085 2579 reflector.go:571] "Warning: watch ended with error" reflector="k8s.io/client-go/informers/factory.go:160" type="*v1.RuntimeClass" err="very short watch: k8s.io/client-go/informers/factory.go:160: Unexpected watch close - watch lasted less than a second and no items received" Jul 03 11:12:24.736613 ip-10-0-136-0 kubenswrapper[2579]: I0703 11:12:24.736091 2579 reflector.go:571] "Warning: watch ended with error" reflector="k8s.io/client-go/informers/factory.go:160" type="*v1.CSIDriver" err="very short watch: k8s.io/client-go/informers/factory.go:160: Unexpected watch close - watch lasted less than a second and no items received" Jul 03 11:12:24.803437 ip-10-0-136-0 kubenswrapper[2579]: E0703 11:12:24.803402 2579 kubelet_node_status.go:454] "Error getting the current node from lister" err="node \"ip-10-0-136-0.ec2.internal\" not found" Jul 03 11:12:24.804602 ip-10-0-136-0 kubenswrapper[2579]: I0703 11:12:24.804561 2579 certificate_manager.go:715] "Certificate rotation deadline determined" logger="kubernetes.io/kube-apiserver-client-kubelet" expiration="2028-07-02 11:07:23 +0000 UTC" deadline="2028-02-16 18:55:18.598490859 +0000 UTC" Jul 03 11:12:24.804651 ip-10-0-136-0 kubenswrapper[2579]: I0703 11:12:24.804603 2579 certificate_manager.go:431] "Waiting for next certificate rotation" logger="kubernetes.io/kube-apiserver-client-kubelet" sleep="14239h42m53.793891651s" Jul 03 11:12:24.904210 ip-10-0-136-0 kubenswrapper[2579]: E0703 11:12:24.904168 2579 kubelet_node_status.go:454] "Error getting the current node from lister" err="node \"ip-10-0-136-0.ec2.internal\" not found" Jul 03 11:12:24.934876 ip-10-0-136-0 kubenswrapper[2579]: I0703 11:12:24.934839 2579 reflector.go:439] "Caches populated" type="*v1.Node" reflector="k8s.io/client-go/informers/factory.go:160" Jul 03 11:12:24.972914 ip-10-0-136-0 kubenswrapper[2579]: W0703 11:12:24.972875 2579 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pode77ab5b2fa375ad8c98ab61ef281b548.slice/crio-61f63fdc0a2b02fd76e76621d9b8c94b5a2fb6280a85bcc3da8d45663735caf3 WatchSource:0}: Error finding container 61f63fdc0a2b02fd76e76621d9b8c94b5a2fb6280a85bcc3da8d45663735caf3: Status 404 returned error can't find the container with id 61f63fdc0a2b02fd76e76621d9b8c94b5a2fb6280a85bcc3da8d45663735caf3 Jul 03 11:12:24.973152 ip-10-0-136-0 kubenswrapper[2579]: W0703 11:12:24.973135 2579 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pode9fe0b17840ecce76a8985c0f09920db.slice/crio-e9754b612cffc16708470bf2c7aecee9d0f777738414a867a6c2e4579927d368 WatchSource:0}: Error finding container e9754b612cffc16708470bf2c7aecee9d0f777738414a867a6c2e4579927d368: Status 404 returned error can't find the container with id e9754b612cffc16708470bf2c7aecee9d0f777738414a867a6c2e4579927d368 Jul 03 11:12:24.978715 ip-10-0-136-0 kubenswrapper[2579]: I0703 11:12:24.978698 2579 provider.go:93] Refreshing cache for provider: *credentialprovider.defaultDockerConfigProvider Jul 03 11:12:24.988995 ip-10-0-136-0 kubenswrapper[2579]: I0703 11:12:24.988944 2579 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kube-system/kube-apiserver-proxy-ip-10-0-136-0.ec2.internal" event={"ID":"e9fe0b17840ecce76a8985c0f09920db","Type":"ContainerStarted","Data":"e9754b612cffc16708470bf2c7aecee9d0f777738414a867a6c2e4579927d368"} Jul 03 11:12:24.989947 ip-10-0-136-0 kubenswrapper[2579]: I0703 11:12:24.989914 2579 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-136-0.ec2.internal" event={"ID":"e77ab5b2fa375ad8c98ab61ef281b548","Type":"ContainerStarted","Data":"61f63fdc0a2b02fd76e76621d9b8c94b5a2fb6280a85bcc3da8d45663735caf3"} Jul 03 11:12:25.002536 ip-10-0-136-0 kubenswrapper[2579]: I0703 11:12:25.002499 2579 kubelet.go:3248] "Creating a mirror pod for static pod" pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-136-0.ec2.internal" Jul 03 11:12:25.031126 ip-10-0-136-0 kubenswrapper[2579]: I0703 11:12:25.031088 2579 warnings.go:110] "Warning: metadata.name: this is used in the Pod's hostname, which can result in surprising behavior; a DNS label is recommended: [must not contain dots]" Jul 03 11:12:25.032936 ip-10-0-136-0 kubenswrapper[2579]: I0703 11:12:25.032915 2579 kubelet.go:3248] "Creating a mirror pod for static pod" pod="kube-system/kube-apiserver-proxy-ip-10-0-136-0.ec2.internal" Jul 03 11:12:25.048105 ip-10-0-136-0 kubenswrapper[2579]: I0703 11:12:25.048077 2579 warnings.go:110] "Warning: metadata.name: this is used in the Pod's hostname, which can result in surprising behavior; a DNS label is recommended: [must not contain dots]" Jul 03 11:12:25.223513 ip-10-0-136-0 kubenswrapper[2579]: I0703 11:12:25.223427 2579 reflector.go:439] "Caches populated" type="*v1.Service" reflector="k8s.io/client-go/informers/factory.go:160" Jul 03 11:12:25.783503 ip-10-0-136-0 kubenswrapper[2579]: I0703 11:12:25.783468 2579 apiserver.go:52] "Watching apiserver" Jul 03 11:12:25.792382 ip-10-0-136-0 kubenswrapper[2579]: I0703 11:12:25.792346 2579 reflector.go:439] "Caches populated" type="*v1.Pod" reflector="pkg/kubelet/config/apiserver.go:66" Jul 03 11:12:25.792748 ip-10-0-136-0 kubenswrapper[2579]: I0703 11:12:25.792721 2579 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-ovn-kubernetes/ovnkube-node-dchtv","kube-system/konnectivity-agent-5lz4n","openshift-cluster-node-tuning-operator/tuned-dwp69","openshift-image-registry/node-ca-9ff9g","openshift-multus/multus-additional-cni-plugins-52n9r","openshift-multus/multus-cszpw","openshift-network-operator/iptables-alerter-l8lpk","kube-system/global-pull-secret-syncer-9pqm8","kube-system/kube-apiserver-proxy-ip-10-0-136-0.ec2.internal","openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-fj786","openshift-dns/node-resolver-hnmtl","openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-136-0.ec2.internal","openshift-multus/network-metrics-daemon-sqxsj","openshift-network-diagnostics/network-check-target-xzlsc"] Jul 03 11:12:25.794414 ip-10-0-136-0 kubenswrapper[2579]: I0703 11:12:25.794386 2579 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-dns/node-resolver-hnmtl" Jul 03 11:12:25.796402 ip-10-0-136-0 kubenswrapper[2579]: I0703 11:12:25.796377 2579 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/konnectivity-agent-5lz4n" Jul 03 11:12:25.799205 ip-10-0-136-0 kubenswrapper[2579]: I0703 11:12:25.799163 2579 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-operator/iptables-alerter-l8lpk" Jul 03 11:12:25.800242 ip-10-0-136-0 kubenswrapper[2579]: I0703 11:12:25.800220 2579 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-dns\"/\"kube-root-ca.crt\"" Jul 03 11:12:25.800428 ip-10-0-136-0 kubenswrapper[2579]: I0703 11:12:25.800409 2579 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-dns\"/\"openshift-service-ca.crt\"" Jul 03 11:12:25.800582 ip-10-0-136-0 kubenswrapper[2579]: I0703 11:12:25.800565 2579 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-dns\"/\"node-resolver-dockercfg-z4ppc\"" Jul 03 11:12:25.801391 ip-10-0-136-0 kubenswrapper[2579]: I0703 11:12:25.801370 2579 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-image-registry/node-ca-9ff9g" Jul 03 11:12:25.802448 ip-10-0-136-0 kubenswrapper[2579]: I0703 11:12:25.802401 2579 certificate_manager.go:566] "Rotating certificates" logger="kubernetes.io/kubelet-serving" Jul 03 11:12:25.802448 ip-10-0-136-0 kubenswrapper[2579]: I0703 11:12:25.802443 2579 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"kube-system\"/\"konnectivity-ca-bundle\"" Jul 03 11:12:25.802844 ip-10-0-136-0 kubenswrapper[2579]: I0703 11:12:25.802827 2579 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/multus-additional-cni-plugins-52n9r" Jul 03 11:12:25.803760 ip-10-0-136-0 kubenswrapper[2579]: I0703 11:12:25.803736 2579 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"kube-system\"/\"konnectivity-agent\"" Jul 03 11:12:25.806724 ip-10-0-136-0 kubenswrapper[2579]: I0703 11:12:25.806695 2579 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/multus-cszpw" Jul 03 11:12:25.808607 ip-10-0-136-0 kubenswrapper[2579]: I0703 11:12:25.808580 2579 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-network-operator\"/\"iptables-alerter-dockercfg-m2dt2\"" Jul 03 11:12:25.808868 ip-10-0-136-0 kubenswrapper[2579]: I0703 11:12:25.808850 2579 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-image-registry\"/\"kube-root-ca.crt\"" Jul 03 11:12:25.808958 ip-10-0-136-0 kubenswrapper[2579]: I0703 11:12:25.808948 2579 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-network-operator\"/\"kube-root-ca.crt\"" Jul 03 11:12:25.809107 ip-10-0-136-0 kubenswrapper[2579]: I0703 11:12:25.809090 2579 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-image-registry\"/\"image-registry-certificates\"" Jul 03 11:12:25.809225 ip-10-0-136-0 kubenswrapper[2579]: I0703 11:12:25.808855 2579 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-network-operator\"/\"openshift-service-ca.crt\"" Jul 03 11:12:25.809293 ip-10-0-136-0 kubenswrapper[2579]: I0703 11:12:25.809268 2579 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-network-operator\"/\"iptables-alerter-script\"" Jul 03 11:12:25.809352 ip-10-0-136-0 kubenswrapper[2579]: I0703 11:12:25.809294 2579 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-image-registry\"/\"openshift-service-ca.crt\"" Jul 03 11:12:25.810112 ip-10-0-136-0 kubenswrapper[2579]: I0703 11:12:25.809724 2579 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ovn-kubernetes/ovnkube-node-dchtv" Jul 03 11:12:25.812211 ip-10-0-136-0 kubenswrapper[2579]: I0703 11:12:25.811942 2579 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-9pqm8" Jul 03 11:12:25.812211 ip-10-0-136-0 kubenswrapper[2579]: E0703 11:12:25.812062 2579 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-9pqm8" podUID="7813f158-0f67-411e-84d9-a2a1c51eea61" Jul 03 11:12:25.814233 ip-10-0-136-0 kubenswrapper[2579]: I0703 11:12:25.812476 2579 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-multus\"/\"kube-root-ca.crt\"" Jul 03 11:12:25.814233 ip-10-0-136-0 kubenswrapper[2579]: I0703 11:12:25.813748 2579 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-fj786" Jul 03 11:12:25.814914 ip-10-0-136-0 kubenswrapper[2579]: I0703 11:12:25.814891 2579 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-cluster-node-tuning-operator/tuned-dwp69" Jul 03 11:12:25.815320 ip-10-0-136-0 kubenswrapper[2579]: I0703 11:12:25.815267 2579 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-wkhsq\" (UniqueName: \"kubernetes.io/projected/ad5f7b2c-eac2-4ebb-b1d8-62782a621d85-kube-api-access-wkhsq\") pod \"multus-cszpw\" (UID: \"ad5f7b2c-eac2-4ebb-b1d8-62782a621d85\") " pod="openshift-multus/multus-cszpw" Jul 03 11:12:25.815320 ip-10-0-136-0 kubenswrapper[2579]: I0703 11:12:25.815304 2579 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"os-release\" (UniqueName: \"kubernetes.io/host-path/e9fc3c32-a741-4377-970b-6cb08e5b4ac9-os-release\") pod \"multus-additional-cni-plugins-52n9r\" (UID: \"e9fc3c32-a741-4377-970b-6cb08e5b4ac9\") " pod="openshift-multus/multus-additional-cni-plugins-52n9r" Jul 03 11:12:25.815469 ip-10-0-136-0 kubenswrapper[2579]: I0703 11:12:25.815331 2579 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tuning-conf-dir\" (UniqueName: \"kubernetes.io/host-path/e9fc3c32-a741-4377-970b-6cb08e5b4ac9-tuning-conf-dir\") pod \"multus-additional-cni-plugins-52n9r\" (UID: \"e9fc3c32-a741-4377-970b-6cb08e5b4ac9\") " pod="openshift-multus/multus-additional-cni-plugins-52n9r" Jul 03 11:12:25.815469 ip-10-0-136-0 kubenswrapper[2579]: I0703 11:12:25.815352 2579 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-46v9l\" (UniqueName: \"kubernetes.io/projected/9a6053f3-4885-4e53-8b1c-93150a7546c4-kube-api-access-46v9l\") pod \"node-resolver-hnmtl\" (UID: \"9a6053f3-4885-4e53-8b1c-93150a7546c4\") " pod="openshift-dns/node-resolver-hnmtl" Jul 03 11:12:25.815469 ip-10-0-136-0 kubenswrapper[2579]: I0703 11:12:25.815378 2579 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"multus-socket-dir-parent\" (UniqueName: \"kubernetes.io/host-path/ad5f7b2c-eac2-4ebb-b1d8-62782a621d85-multus-socket-dir-parent\") pod \"multus-cszpw\" (UID: \"ad5f7b2c-eac2-4ebb-b1d8-62782a621d85\") " pod="openshift-multus/multus-cszpw" Jul 03 11:12:25.815469 ip-10-0-136-0 kubenswrapper[2579]: I0703 11:12:25.815399 2579 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-run-netns\" (UniqueName: \"kubernetes.io/host-path/ad5f7b2c-eac2-4ebb-b1d8-62782a621d85-host-run-netns\") pod \"multus-cszpw\" (UID: \"ad5f7b2c-eac2-4ebb-b1d8-62782a621d85\") " pod="openshift-multus/multus-cszpw" Jul 03 11:12:25.815469 ip-10-0-136-0 kubenswrapper[2579]: I0703 11:12:25.815422 2579 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/9a6053f3-4885-4e53-8b1c-93150a7546c4-tmp-dir\") pod \"node-resolver-hnmtl\" (UID: \"9a6053f3-4885-4e53-8b1c-93150a7546c4\") " pod="openshift-dns/node-resolver-hnmtl" Jul 03 11:12:25.815469 ip-10-0-136-0 kubenswrapper[2579]: I0703 11:12:25.815454 2579 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-679vc\" (UniqueName: \"kubernetes.io/projected/a8bc35c6-22ff-4c87-802d-e44999086c38-kube-api-access-679vc\") pod \"iptables-alerter-l8lpk\" (UID: \"a8bc35c6-22ff-4c87-802d-e44999086c38\") " pod="openshift-network-operator/iptables-alerter-l8lpk" Jul 03 11:12:25.815722 ip-10-0-136-0 kubenswrapper[2579]: I0703 11:12:25.815477 2579 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-dwqcd\" (UniqueName: \"kubernetes.io/projected/f5fa8ffd-d2fc-4474-ad0d-a7af28edf0e7-kube-api-access-dwqcd\") pod \"node-ca-9ff9g\" (UID: \"f5fa8ffd-d2fc-4474-ad0d-a7af28edf0e7\") " pod="openshift-image-registry/node-ca-9ff9g" Jul 03 11:12:25.815722 ip-10-0-136-0 kubenswrapper[2579]: I0703 11:12:25.815499 2579 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"cnibin\" (UniqueName: \"kubernetes.io/host-path/ad5f7b2c-eac2-4ebb-b1d8-62782a621d85-cnibin\") pod \"multus-cszpw\" (UID: \"ad5f7b2c-eac2-4ebb-b1d8-62782a621d85\") " pod="openshift-multus/multus-cszpw" Jul 03 11:12:25.815722 ip-10-0-136-0 kubenswrapper[2579]: I0703 11:12:25.815521 2579 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"multus-cni-dir\" (UniqueName: \"kubernetes.io/host-path/ad5f7b2c-eac2-4ebb-b1d8-62782a621d85-multus-cni-dir\") pod \"multus-cszpw\" (UID: \"ad5f7b2c-eac2-4ebb-b1d8-62782a621d85\") " pod="openshift-multus/multus-cszpw" Jul 03 11:12:25.815722 ip-10-0-136-0 kubenswrapper[2579]: I0703 11:12:25.815542 2579 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"hostroot\" (UniqueName: \"kubernetes.io/host-path/ad5f7b2c-eac2-4ebb-b1d8-62782a621d85-hostroot\") pod \"multus-cszpw\" (UID: \"ad5f7b2c-eac2-4ebb-b1d8-62782a621d85\") " pod="openshift-multus/multus-cszpw" Jul 03 11:12:25.815722 ip-10-0-136-0 kubenswrapper[2579]: I0703 11:12:25.815565 2579 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"multus-daemon-config\" (UniqueName: \"kubernetes.io/configmap/ad5f7b2c-eac2-4ebb-b1d8-62782a621d85-multus-daemon-config\") pod \"multus-cszpw\" (UID: \"ad5f7b2c-eac2-4ebb-b1d8-62782a621d85\") " pod="openshift-multus/multus-cszpw" Jul 03 11:12:25.815722 ip-10-0-136-0 kubenswrapper[2579]: I0703 11:12:25.815608 2579 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-slash\" (UniqueName: \"kubernetes.io/host-path/a8bc35c6-22ff-4c87-802d-e44999086c38-host-slash\") pod \"iptables-alerter-l8lpk\" (UID: \"a8bc35c6-22ff-4c87-802d-e44999086c38\") " pod="openshift-network-operator/iptables-alerter-l8lpk" Jul 03 11:12:25.815722 ip-10-0-136-0 kubenswrapper[2579]: I0703 11:12:25.815640 2579 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host\" (UniqueName: \"kubernetes.io/host-path/f5fa8ffd-d2fc-4474-ad0d-a7af28edf0e7-host\") pod \"node-ca-9ff9g\" (UID: \"f5fa8ffd-d2fc-4474-ad0d-a7af28edf0e7\") " pod="openshift-image-registry/node-ca-9ff9g" Jul 03 11:12:25.815722 ip-10-0-136-0 kubenswrapper[2579]: I0703 11:12:25.815669 2579 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"cni-binary-copy\" (UniqueName: \"kubernetes.io/configmap/e9fc3c32-a741-4377-970b-6cb08e5b4ac9-cni-binary-copy\") pod \"multus-additional-cni-plugins-52n9r\" (UID: \"e9fc3c32-a741-4377-970b-6cb08e5b4ac9\") " pod="openshift-multus/multus-additional-cni-plugins-52n9r" Jul 03 11:12:25.815722 ip-10-0-136-0 kubenswrapper[2579]: I0703 11:12:25.815692 2579 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"os-release\" (UniqueName: \"kubernetes.io/host-path/ad5f7b2c-eac2-4ebb-b1d8-62782a621d85-os-release\") pod \"multus-cszpw\" (UID: \"ad5f7b2c-eac2-4ebb-b1d8-62782a621d85\") " pod="openshift-multus/multus-cszpw" Jul 03 11:12:25.815722 ip-10-0-136-0 kubenswrapper[2579]: I0703 11:12:25.815715 2579 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-var-lib-cni-multus\" (UniqueName: \"kubernetes.io/host-path/ad5f7b2c-eac2-4ebb-b1d8-62782a621d85-host-var-lib-cni-multus\") pod \"multus-cszpw\" (UID: \"ad5f7b2c-eac2-4ebb-b1d8-62782a621d85\") " pod="openshift-multus/multus-cszpw" Jul 03 11:12:25.816173 ip-10-0-136-0 kubenswrapper[2579]: I0703 11:12:25.815739 2579 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-var-lib-kubelet\" (UniqueName: \"kubernetes.io/host-path/ad5f7b2c-eac2-4ebb-b1d8-62782a621d85-host-var-lib-kubelet\") pod \"multus-cszpw\" (UID: \"ad5f7b2c-eac2-4ebb-b1d8-62782a621d85\") " pod="openshift-multus/multus-cszpw" Jul 03 11:12:25.816173 ip-10-0-136-0 kubenswrapper[2579]: I0703 11:12:25.815866 2579 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"hosts-file\" (UniqueName: \"kubernetes.io/host-path/9a6053f3-4885-4e53-8b1c-93150a7546c4-hosts-file\") pod \"node-resolver-hnmtl\" (UID: \"9a6053f3-4885-4e53-8b1c-93150a7546c4\") " pod="openshift-dns/node-resolver-hnmtl" Jul 03 11:12:25.816173 ip-10-0-136-0 kubenswrapper[2579]: I0703 11:12:25.815892 2579 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"system-cni-dir\" (UniqueName: \"kubernetes.io/host-path/e9fc3c32-a741-4377-970b-6cb08e5b4ac9-system-cni-dir\") pod \"multus-additional-cni-plugins-52n9r\" (UID: \"e9fc3c32-a741-4377-970b-6cb08e5b4ac9\") " pod="openshift-multus/multus-additional-cni-plugins-52n9r" Jul 03 11:12:25.816173 ip-10-0-136-0 kubenswrapper[2579]: I0703 11:12:25.815916 2579 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-cfc5q\" (UniqueName: \"kubernetes.io/projected/e9fc3c32-a741-4377-970b-6cb08e5b4ac9-kube-api-access-cfc5q\") pod \"multus-additional-cni-plugins-52n9r\" (UID: \"e9fc3c32-a741-4377-970b-6cb08e5b4ac9\") " pod="openshift-multus/multus-additional-cni-plugins-52n9r" Jul 03 11:12:25.816173 ip-10-0-136-0 kubenswrapper[2579]: I0703 11:12:25.815940 2579 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"cni-binary-copy\" (UniqueName: \"kubernetes.io/configmap/ad5f7b2c-eac2-4ebb-b1d8-62782a621d85-cni-binary-copy\") pod \"multus-cszpw\" (UID: \"ad5f7b2c-eac2-4ebb-b1d8-62782a621d85\") " pod="openshift-multus/multus-cszpw" Jul 03 11:12:25.816173 ip-10-0-136-0 kubenswrapper[2579]: I0703 11:12:25.815964 2579 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-var-lib-cni-bin\" (UniqueName: \"kubernetes.io/host-path/ad5f7b2c-eac2-4ebb-b1d8-62782a621d85-host-var-lib-cni-bin\") pod \"multus-cszpw\" (UID: \"ad5f7b2c-eac2-4ebb-b1d8-62782a621d85\") " pod="openshift-multus/multus-cszpw" Jul 03 11:12:25.816173 ip-10-0-136-0 kubenswrapper[2579]: I0703 11:12:25.815988 2579 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"etc-kubernetes\" (UniqueName: \"kubernetes.io/host-path/ad5f7b2c-eac2-4ebb-b1d8-62782a621d85-etc-kubernetes\") pod \"multus-cszpw\" (UID: \"ad5f7b2c-eac2-4ebb-b1d8-62782a621d85\") " pod="openshift-multus/multus-cszpw" Jul 03 11:12:25.816173 ip-10-0-136-0 kubenswrapper[2579]: I0703 11:12:25.816015 2579 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"konnectivity-ca\" (UniqueName: \"kubernetes.io/configmap/945b7f84-a41f-413e-a7b6-87818468360f-konnectivity-ca\") pod \"konnectivity-agent-5lz4n\" (UID: \"945b7f84-a41f-413e-a7b6-87818468360f\") " pod="kube-system/konnectivity-agent-5lz4n" Jul 03 11:12:25.816173 ip-10-0-136-0 kubenswrapper[2579]: I0703 11:12:25.816055 2579 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"system-cni-dir\" (UniqueName: \"kubernetes.io/host-path/ad5f7b2c-eac2-4ebb-b1d8-62782a621d85-system-cni-dir\") pod \"multus-cszpw\" (UID: \"ad5f7b2c-eac2-4ebb-b1d8-62782a621d85\") " pod="openshift-multus/multus-cszpw" Jul 03 11:12:25.816173 ip-10-0-136-0 kubenswrapper[2579]: I0703 11:12:25.816106 2579 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"multus-conf-dir\" (UniqueName: \"kubernetes.io/host-path/ad5f7b2c-eac2-4ebb-b1d8-62782a621d85-multus-conf-dir\") pod \"multus-cszpw\" (UID: \"ad5f7b2c-eac2-4ebb-b1d8-62782a621d85\") " pod="openshift-multus/multus-cszpw" Jul 03 11:12:25.816173 ip-10-0-136-0 kubenswrapper[2579]: I0703 11:12:25.816144 2579 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-run-multus-certs\" (UniqueName: \"kubernetes.io/host-path/ad5f7b2c-eac2-4ebb-b1d8-62782a621d85-host-run-multus-certs\") pod \"multus-cszpw\" (UID: \"ad5f7b2c-eac2-4ebb-b1d8-62782a621d85\") " pod="openshift-multus/multus-cszpw" Jul 03 11:12:25.816173 ip-10-0-136-0 kubenswrapper[2579]: I0703 11:12:25.816170 2579 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"agent-certs\" (UniqueName: \"kubernetes.io/secret/945b7f84-a41f-413e-a7b6-87818468360f-agent-certs\") pod \"konnectivity-agent-5lz4n\" (UID: \"945b7f84-a41f-413e-a7b6-87818468360f\") " pod="kube-system/konnectivity-agent-5lz4n" Jul 03 11:12:25.816745 ip-10-0-136-0 kubenswrapper[2579]: I0703 11:12:25.816224 2579 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"iptables-alerter-script\" (UniqueName: \"kubernetes.io/configmap/a8bc35c6-22ff-4c87-802d-e44999086c38-iptables-alerter-script\") pod \"iptables-alerter-l8lpk\" (UID: \"a8bc35c6-22ff-4c87-802d-e44999086c38\") " pod="openshift-network-operator/iptables-alerter-l8lpk" Jul 03 11:12:25.816745 ip-10-0-136-0 kubenswrapper[2579]: I0703 11:12:25.816250 2579 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"cni-sysctl-allowlist\" (UniqueName: \"kubernetes.io/configmap/e9fc3c32-a741-4377-970b-6cb08e5b4ac9-cni-sysctl-allowlist\") pod \"multus-additional-cni-plugins-52n9r\" (UID: \"e9fc3c32-a741-4377-970b-6cb08e5b4ac9\") " pod="openshift-multus/multus-additional-cni-plugins-52n9r" Jul 03 11:12:25.816745 ip-10-0-136-0 kubenswrapper[2579]: I0703 11:12:25.816275 2579 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"whereabouts-flatfile-configmap\" (UniqueName: \"kubernetes.io/configmap/e9fc3c32-a741-4377-970b-6cb08e5b4ac9-whereabouts-flatfile-configmap\") pod \"multus-additional-cni-plugins-52n9r\" (UID: \"e9fc3c32-a741-4377-970b-6cb08e5b4ac9\") " pod="openshift-multus/multus-additional-cni-plugins-52n9r" Jul 03 11:12:25.816745 ip-10-0-136-0 kubenswrapper[2579]: I0703 11:12:25.816307 2579 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"serviceca\" (UniqueName: \"kubernetes.io/configmap/f5fa8ffd-d2fc-4474-ad0d-a7af28edf0e7-serviceca\") pod \"node-ca-9ff9g\" (UID: \"f5fa8ffd-d2fc-4474-ad0d-a7af28edf0e7\") " pod="openshift-image-registry/node-ca-9ff9g" Jul 03 11:12:25.816745 ip-10-0-136-0 kubenswrapper[2579]: I0703 11:12:25.816332 2579 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"cnibin\" (UniqueName: \"kubernetes.io/host-path/e9fc3c32-a741-4377-970b-6cb08e5b4ac9-cnibin\") pod \"multus-additional-cni-plugins-52n9r\" (UID: \"e9fc3c32-a741-4377-970b-6cb08e5b4ac9\") " pod="openshift-multus/multus-additional-cni-plugins-52n9r" Jul 03 11:12:25.816745 ip-10-0-136-0 kubenswrapper[2579]: I0703 11:12:25.816357 2579 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-run-k8s-cni-cncf-io\" (UniqueName: \"kubernetes.io/host-path/ad5f7b2c-eac2-4ebb-b1d8-62782a621d85-host-run-k8s-cni-cncf-io\") pod \"multus-cszpw\" (UID: \"ad5f7b2c-eac2-4ebb-b1d8-62782a621d85\") " pod="openshift-multus/multus-cszpw" Jul 03 11:12:25.819702 ip-10-0-136-0 kubenswrapper[2579]: I0703 11:12:25.817594 2579 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-image-registry\"/\"node-ca-dockercfg-57wjb\"" Jul 03 11:12:25.819702 ip-10-0-136-0 kubenswrapper[2579]: I0703 11:12:25.817798 2579 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ovn-kubernetes\"/\"env-overrides\"" Jul 03 11:12:25.819702 ip-10-0-136-0 kubenswrapper[2579]: I0703 11:12:25.817984 2579 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-multus\"/\"multus-daemon-config\"" Jul 03 11:12:25.819702 ip-10-0-136-0 kubenswrapper[2579]: I0703 11:12:25.818046 2579 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-cluster-csi-drivers\"/\"kube-root-ca.crt\"" Jul 03 11:12:25.819702 ip-10-0-136-0 kubenswrapper[2579]: I0703 11:12:25.818058 2579 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-multus\"/\"whereabouts-flatfile-config\"" Jul 03 11:12:25.819702 ip-10-0-136-0 kubenswrapper[2579]: I0703 11:12:25.818267 2579 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-multus\"/\"default-cni-sysctl-allowlist\"" Jul 03 11:12:25.819702 ip-10-0-136-0 kubenswrapper[2579]: I0703 11:12:25.818297 2579 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-multus\"/\"multus-ancillary-tools-dockercfg-znzw9\"" Jul 03 11:12:25.819702 ip-10-0-136-0 kubenswrapper[2579]: I0703 11:12:25.818309 2579 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-multus\"/\"openshift-service-ca.crt\"" Jul 03 11:12:25.819702 ip-10-0-136-0 kubenswrapper[2579]: I0703 11:12:25.818502 2579 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-cluster-csi-drivers\"/\"aws-ebs-csi-driver-node-metrics-serving-cert\"" Jul 03 11:12:25.819702 ip-10-0-136-0 kubenswrapper[2579]: I0703 11:12:25.818532 2579 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-sqxsj" Jul 03 11:12:25.819702 ip-10-0-136-0 kubenswrapper[2579]: E0703 11:12:25.818623 2579 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-sqxsj" podUID="34ca9c51-4743-4efc-9aa1-eb79a741f6a4" Jul 03 11:12:25.819702 ip-10-0-136-0 kubenswrapper[2579]: I0703 11:12:25.818693 2579 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ovn-kubernetes\"/\"ovnkube-config\"" Jul 03 11:12:25.819702 ip-10-0-136-0 kubenswrapper[2579]: I0703 11:12:25.818783 2579 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ovn-kubernetes\"/\"kube-root-ca.crt\"" Jul 03 11:12:25.819702 ip-10-0-136-0 kubenswrapper[2579]: I0703 11:12:25.818814 2579 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-multus\"/\"default-dockercfg-76nlv\"" Jul 03 11:12:25.819702 ip-10-0-136-0 kubenswrapper[2579]: I0703 11:12:25.818912 2579 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-cluster-node-tuning-operator\"/\"kube-root-ca.crt\"" Jul 03 11:12:25.819702 ip-10-0-136-0 kubenswrapper[2579]: I0703 11:12:25.819031 2579 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-cluster-node-tuning-operator\"/\"tuned-dockercfg-qtl57\"" Jul 03 11:12:25.819702 ip-10-0-136-0 kubenswrapper[2579]: I0703 11:12:25.819038 2579 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-cluster-csi-drivers\"/\"aws-ebs-csi-driver-node-sa-dockercfg-v74bj\"" Jul 03 11:12:25.819702 ip-10-0-136-0 kubenswrapper[2579]: I0703 11:12:25.819158 2579 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ovn-kubernetes\"/\"ovn-node-metrics-cert\"" Jul 03 11:12:25.819702 ip-10-0-136-0 kubenswrapper[2579]: I0703 11:12:25.819149 2579 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ovn-kubernetes\"/\"openshift-service-ca.crt\"" Jul 03 11:12:25.819702 ip-10-0-136-0 kubenswrapper[2579]: I0703 11:12:25.819223 2579 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ovn-kubernetes\"/\"ovn-kubernetes-node-dockercfg-925q4\"" Jul 03 11:12:25.819702 ip-10-0-136-0 kubenswrapper[2579]: I0703 11:12:25.819349 2579 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"kube-system\"/\"default-dockercfg-l6gcs\"" Jul 03 11:12:25.819702 ip-10-0-136-0 kubenswrapper[2579]: I0703 11:12:25.819373 2579 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-cluster-node-tuning-operator\"/\"openshift-service-ca.crt\"" Jul 03 11:12:25.819702 ip-10-0-136-0 kubenswrapper[2579]: I0703 11:12:25.819544 2579 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-multus\"/\"cni-copy-resources\"" Jul 03 11:12:25.820744 ip-10-0-136-0 kubenswrapper[2579]: I0703 11:12:25.819768 2579 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-cluster-csi-drivers\"/\"openshift-service-ca.crt\"" Jul 03 11:12:25.820744 ip-10-0-136-0 kubenswrapper[2579]: I0703 11:12:25.820007 2579 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ovn-kubernetes\"/\"ovnkube-script-lib\"" Jul 03 11:12:25.820744 ip-10-0-136-0 kubenswrapper[2579]: I0703 11:12:25.820455 2579 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-xzlsc" Jul 03 11:12:25.820744 ip-10-0-136-0 kubenswrapper[2579]: E0703 11:12:25.820525 2579 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-xzlsc" podUID="2129ac30-1645-4555-b722-55963ffb8776" Jul 03 11:12:25.839744 ip-10-0-136-0 kubenswrapper[2579]: I0703 11:12:25.839697 2579 reflector.go:439] "Caches populated" logger="kubernetes.io/kubelet-serving" type="*v1.CertificateSigningRequest" reflector="k8s.io/client-go/tools/watch/informerwatcher.go:162" Jul 03 11:12:25.873259 ip-10-0-136-0 kubenswrapper[2579]: I0703 11:12:25.872281 2579 csr.go:274] "Certificate signing request is approved, waiting to be issued" logger="kubernetes.io/kubelet-serving" csr="csr-59rgz" Jul 03 11:12:25.902217 ip-10-0-136-0 kubenswrapper[2579]: I0703 11:12:25.901748 2579 csr.go:270] "Certificate signing request is issued" logger="kubernetes.io/kubelet-serving" csr="csr-59rgz" Jul 03 11:12:25.905549 ip-10-0-136-0 kubenswrapper[2579]: I0703 11:12:25.905521 2579 desired_state_of_world_populator.go:154] "Finished populating initial desired state of world" Jul 03 11:12:25.916786 ip-10-0-136-0 kubenswrapper[2579]: I0703 11:12:25.916725 2579 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"cni-binary-copy\" (UniqueName: \"kubernetes.io/configmap/ad5f7b2c-eac2-4ebb-b1d8-62782a621d85-cni-binary-copy\") pod \"multus-cszpw\" (UID: \"ad5f7b2c-eac2-4ebb-b1d8-62782a621d85\") " pod="openshift-multus/multus-cszpw" Jul 03 11:12:25.916990 ip-10-0-136-0 kubenswrapper[2579]: I0703 11:12:25.916867 2579 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-var-lib-cni-bin\" (UniqueName: \"kubernetes.io/host-path/ad5f7b2c-eac2-4ebb-b1d8-62782a621d85-host-var-lib-cni-bin\") pod \"multus-cszpw\" (UID: \"ad5f7b2c-eac2-4ebb-b1d8-62782a621d85\") " pod="openshift-multus/multus-cszpw" Jul 03 11:12:25.916990 ip-10-0-136-0 kubenswrapper[2579]: I0703 11:12:25.916905 2579 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"konnectivity-ca\" (UniqueName: \"kubernetes.io/configmap/945b7f84-a41f-413e-a7b6-87818468360f-konnectivity-ca\") pod \"konnectivity-agent-5lz4n\" (UID: \"945b7f84-a41f-413e-a7b6-87818468360f\") " pod="kube-system/konnectivity-agent-5lz4n" Jul 03 11:12:25.916990 ip-10-0-136-0 kubenswrapper[2579]: I0703 11:12:25.916934 2579 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"lib-modules\" (UniqueName: \"kubernetes.io/host-path/99eb1d1a-c732-4365-9da7-ecab8ead5794-lib-modules\") pod \"tuned-dwp69\" (UID: \"99eb1d1a-c732-4365-9da7-ecab8ead5794\") " pod="openshift-cluster-node-tuning-operator/tuned-dwp69" Jul 03 11:12:25.916990 ip-10-0-136-0 kubenswrapper[2579]: I0703 11:12:25.916989 2579 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"run-systemd\" (UniqueName: \"kubernetes.io/host-path/45f92856-212a-4190-93ae-dfec96e1864e-run-systemd\") pod \"ovnkube-node-dchtv\" (UID: \"45f92856-212a-4190-93ae-dfec96e1864e\") " pod="openshift-ovn-kubernetes/ovnkube-node-dchtv" Jul 03 11:12:25.917178 ip-10-0-136-0 kubenswrapper[2579]: I0703 11:12:25.917026 2579 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"agent-certs\" (UniqueName: \"kubernetes.io/secret/945b7f84-a41f-413e-a7b6-87818468360f-agent-certs\") pod \"konnectivity-agent-5lz4n\" (UID: \"945b7f84-a41f-413e-a7b6-87818468360f\") " pod="kube-system/konnectivity-agent-5lz4n" Jul 03 11:12:25.917178 ip-10-0-136-0 kubenswrapper[2579]: I0703 11:12:25.917070 2579 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"iptables-alerter-script\" (UniqueName: \"kubernetes.io/configmap/a8bc35c6-22ff-4c87-802d-e44999086c38-iptables-alerter-script\") pod \"iptables-alerter-l8lpk\" (UID: \"a8bc35c6-22ff-4c87-802d-e44999086c38\") " pod="openshift-network-operator/iptables-alerter-l8lpk" Jul 03 11:12:25.917178 ip-10-0-136-0 kubenswrapper[2579]: I0703 11:12:25.917094 2579 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"etc-modprobe-d\" (UniqueName: \"kubernetes.io/host-path/99eb1d1a-c732-4365-9da7-ecab8ead5794-etc-modprobe-d\") pod \"tuned-dwp69\" (UID: \"99eb1d1a-c732-4365-9da7-ecab8ead5794\") " pod="openshift-cluster-node-tuning-operator/tuned-dwp69" Jul 03 11:12:25.917178 ip-10-0-136-0 kubenswrapper[2579]: I0703 11:12:25.917130 2579 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"etc-sysconfig\" (UniqueName: \"kubernetes.io/host-path/99eb1d1a-c732-4365-9da7-ecab8ead5794-etc-sysconfig\") pod \"tuned-dwp69\" (UID: \"99eb1d1a-c732-4365-9da7-ecab8ead5794\") " pod="openshift-cluster-node-tuning-operator/tuned-dwp69" Jul 03 11:12:25.917178 ip-10-0-136-0 kubenswrapper[2579]: I0703 11:12:25.917162 2579 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"etc-sysctl-d\" (UniqueName: \"kubernetes.io/host-path/99eb1d1a-c732-4365-9da7-ecab8ead5794-etc-sysctl-d\") pod \"tuned-dwp69\" (UID: \"99eb1d1a-c732-4365-9da7-ecab8ead5794\") " pod="openshift-cluster-node-tuning-operator/tuned-dwp69" Jul 03 11:12:25.917496 ip-10-0-136-0 kubenswrapper[2579]: I0703 11:12:25.917211 2579 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"registration-dir\" (UniqueName: \"kubernetes.io/host-path/a39e260a-4eff-4acb-9b7f-3ea1f42ff3c0-registration-dir\") pod \"aws-ebs-csi-driver-node-fj786\" (UID: \"a39e260a-4eff-4acb-9b7f-3ea1f42ff3c0\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-fj786" Jul 03 11:12:25.917496 ip-10-0-136-0 kubenswrapper[2579]: I0703 11:12:25.917236 2579 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tmp\" (UniqueName: \"kubernetes.io/empty-dir/99eb1d1a-c732-4365-9da7-ecab8ead5794-tmp\") pod \"tuned-dwp69\" (UID: \"99eb1d1a-c732-4365-9da7-ecab8ead5794\") " pod="openshift-cluster-node-tuning-operator/tuned-dwp69" Jul 03 11:12:25.917496 ip-10-0-136-0 kubenswrapper[2579]: I0703 11:12:25.917258 2579 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-hdp57\" (UniqueName: \"kubernetes.io/projected/99eb1d1a-c732-4365-9da7-ecab8ead5794-kube-api-access-hdp57\") pod \"tuned-dwp69\" (UID: \"99eb1d1a-c732-4365-9da7-ecab8ead5794\") " pod="openshift-cluster-node-tuning-operator/tuned-dwp69" Jul 03 11:12:25.917496 ip-10-0-136-0 kubenswrapper[2579]: I0703 11:12:25.917282 2579 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-kubelet\" (UniqueName: \"kubernetes.io/host-path/45f92856-212a-4190-93ae-dfec96e1864e-host-kubelet\") pod \"ovnkube-node-dchtv\" (UID: \"45f92856-212a-4190-93ae-dfec96e1864e\") " pod="openshift-ovn-kubernetes/ovnkube-node-dchtv" Jul 03 11:12:25.917496 ip-10-0-136-0 kubenswrapper[2579]: I0703 11:12:25.917313 2579 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-slash\" (UniqueName: \"kubernetes.io/host-path/45f92856-212a-4190-93ae-dfec96e1864e-host-slash\") pod \"ovnkube-node-dchtv\" (UID: \"45f92856-212a-4190-93ae-dfec96e1864e\") " pod="openshift-ovn-kubernetes/ovnkube-node-dchtv" Jul 03 11:12:25.917496 ip-10-0-136-0 kubenswrapper[2579]: I0703 11:12:25.917340 2579 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-wkhsq\" (UniqueName: \"kubernetes.io/projected/ad5f7b2c-eac2-4ebb-b1d8-62782a621d85-kube-api-access-wkhsq\") pod \"multus-cszpw\" (UID: \"ad5f7b2c-eac2-4ebb-b1d8-62782a621d85\") " pod="openshift-multus/multus-cszpw" Jul 03 11:12:25.917496 ip-10-0-136-0 kubenswrapper[2579]: I0703 11:12:25.917378 2579 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"run-openvswitch\" (UniqueName: \"kubernetes.io/host-path/45f92856-212a-4190-93ae-dfec96e1864e-run-openvswitch\") pod \"ovnkube-node-dchtv\" (UID: \"45f92856-212a-4190-93ae-dfec96e1864e\") " pod="openshift-ovn-kubernetes/ovnkube-node-dchtv" Jul 03 11:12:25.917496 ip-10-0-136-0 kubenswrapper[2579]: I0703 11:12:25.917406 2579 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"dbus\" (UniqueName: \"kubernetes.io/host-path/7813f158-0f67-411e-84d9-a2a1c51eea61-dbus\") pod \"global-pull-secret-syncer-9pqm8\" (UID: \"7813f158-0f67-411e-84d9-a2a1c51eea61\") " pod="kube-system/global-pull-secret-syncer-9pqm8" Jul 03 11:12:25.917496 ip-10-0-136-0 kubenswrapper[2579]: I0703 11:12:25.917433 2579 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-flqqx\" (UniqueName: \"kubernetes.io/projected/2129ac30-1645-4555-b722-55963ffb8776-kube-api-access-flqqx\") pod \"network-check-target-xzlsc\" (UID: \"2129ac30-1645-4555-b722-55963ffb8776\") " pod="openshift-network-diagnostics/network-check-target-xzlsc" Jul 03 11:12:25.917496 ip-10-0-136-0 kubenswrapper[2579]: I0703 11:12:25.917446 2579 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"cni-binary-copy\" (UniqueName: \"kubernetes.io/configmap/ad5f7b2c-eac2-4ebb-b1d8-62782a621d85-cni-binary-copy\") pod \"multus-cszpw\" (UID: \"ad5f7b2c-eac2-4ebb-b1d8-62782a621d85\") " pod="openshift-multus/multus-cszpw" Jul 03 11:12:25.917496 ip-10-0-136-0 kubenswrapper[2579]: I0703 11:12:25.917455 2579 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host\" (UniqueName: \"kubernetes.io/host-path/99eb1d1a-c732-4365-9da7-ecab8ead5794-host\") pod \"tuned-dwp69\" (UID: \"99eb1d1a-c732-4365-9da7-ecab8ead5794\") " pod="openshift-cluster-node-tuning-operator/tuned-dwp69" Jul 03 11:12:25.917496 ip-10-0-136-0 kubenswrapper[2579]: I0703 11:12:25.917480 2579 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"systemd-units\" (UniqueName: \"kubernetes.io/host-path/45f92856-212a-4190-93ae-dfec96e1864e-systemd-units\") pod \"ovnkube-node-dchtv\" (UID: \"45f92856-212a-4190-93ae-dfec96e1864e\") " pod="openshift-ovn-kubernetes/ovnkube-node-dchtv" Jul 03 11:12:25.917863 ip-10-0-136-0 kubenswrapper[2579]: I0703 11:12:25.917505 2579 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-run-netns\" (UniqueName: \"kubernetes.io/host-path/ad5f7b2c-eac2-4ebb-b1d8-62782a621d85-host-run-netns\") pod \"multus-cszpw\" (UID: \"ad5f7b2c-eac2-4ebb-b1d8-62782a621d85\") " pod="openshift-multus/multus-cszpw" Jul 03 11:12:25.917863 ip-10-0-136-0 kubenswrapper[2579]: I0703 11:12:25.917529 2579 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-dwqcd\" (UniqueName: \"kubernetes.io/projected/f5fa8ffd-d2fc-4474-ad0d-a7af28edf0e7-kube-api-access-dwqcd\") pod \"node-ca-9ff9g\" (UID: \"f5fa8ffd-d2fc-4474-ad0d-a7af28edf0e7\") " pod="openshift-image-registry/node-ca-9ff9g" Jul 03 11:12:25.917863 ip-10-0-136-0 kubenswrapper[2579]: I0703 11:12:25.917551 2579 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"etc-selinux\" (UniqueName: \"kubernetes.io/host-path/a39e260a-4eff-4acb-9b7f-3ea1f42ff3c0-etc-selinux\") pod \"aws-ebs-csi-driver-node-fj786\" (UID: \"a39e260a-4eff-4acb-9b7f-3ea1f42ff3c0\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-fj786" Jul 03 11:12:25.917863 ip-10-0-136-0 kubenswrapper[2579]: I0703 11:12:25.917562 2579 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"host-var-lib-cni-bin\" (UniqueName: \"kubernetes.io/host-path/ad5f7b2c-eac2-4ebb-b1d8-62782a621d85-host-var-lib-cni-bin\") pod \"multus-cszpw\" (UID: \"ad5f7b2c-eac2-4ebb-b1d8-62782a621d85\") " pod="openshift-multus/multus-cszpw" Jul 03 11:12:25.917863 ip-10-0-136-0 kubenswrapper[2579]: I0703 11:12:25.917586 2579 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"ovnkube-config\" (UniqueName: \"kubernetes.io/configmap/45f92856-212a-4190-93ae-dfec96e1864e-ovnkube-config\") pod \"ovnkube-node-dchtv\" (UID: \"45f92856-212a-4190-93ae-dfec96e1864e\") " pod="openshift-ovn-kubernetes/ovnkube-node-dchtv" Jul 03 11:12:25.917863 ip-10-0-136-0 kubenswrapper[2579]: I0703 11:12:25.917625 2579 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"cnibin\" (UniqueName: \"kubernetes.io/host-path/ad5f7b2c-eac2-4ebb-b1d8-62782a621d85-cnibin\") pod \"multus-cszpw\" (UID: \"ad5f7b2c-eac2-4ebb-b1d8-62782a621d85\") " pod="openshift-multus/multus-cszpw" Jul 03 11:12:25.917863 ip-10-0-136-0 kubenswrapper[2579]: I0703 11:12:25.917660 2579 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host\" (UniqueName: \"kubernetes.io/host-path/f5fa8ffd-d2fc-4474-ad0d-a7af28edf0e7-host\") pod \"node-ca-9ff9g\" (UID: \"f5fa8ffd-d2fc-4474-ad0d-a7af28edf0e7\") " pod="openshift-image-registry/node-ca-9ff9g" Jul 03 11:12:25.917863 ip-10-0-136-0 kubenswrapper[2579]: I0703 11:12:25.917683 2579 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"etc-systemd\" (UniqueName: \"kubernetes.io/host-path/99eb1d1a-c732-4365-9da7-ecab8ead5794-etc-systemd\") pod \"tuned-dwp69\" (UID: \"99eb1d1a-c732-4365-9da7-ecab8ead5794\") " pod="openshift-cluster-node-tuning-operator/tuned-dwp69" Jul 03 11:12:25.917863 ip-10-0-136-0 kubenswrapper[2579]: I0703 11:12:25.917708 2579 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"etc-openvswitch\" (UniqueName: \"kubernetes.io/host-path/45f92856-212a-4190-93ae-dfec96e1864e-etc-openvswitch\") pod \"ovnkube-node-dchtv\" (UID: \"45f92856-212a-4190-93ae-dfec96e1864e\") " pod="openshift-ovn-kubernetes/ovnkube-node-dchtv" Jul 03 11:12:25.917863 ip-10-0-136-0 kubenswrapper[2579]: I0703 11:12:25.917738 2579 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"run-ovn\" (UniqueName: \"kubernetes.io/host-path/45f92856-212a-4190-93ae-dfec96e1864e-run-ovn\") pod \"ovnkube-node-dchtv\" (UID: \"45f92856-212a-4190-93ae-dfec96e1864e\") " pod="openshift-ovn-kubernetes/ovnkube-node-dchtv" Jul 03 11:12:25.917863 ip-10-0-136-0 kubenswrapper[2579]: I0703 11:12:25.917763 2579 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-var-lib-kubelet\" (UniqueName: \"kubernetes.io/host-path/ad5f7b2c-eac2-4ebb-b1d8-62782a621d85-host-var-lib-kubelet\") pod \"multus-cszpw\" (UID: \"ad5f7b2c-eac2-4ebb-b1d8-62782a621d85\") " pod="openshift-multus/multus-cszpw" Jul 03 11:12:25.917863 ip-10-0-136-0 kubenswrapper[2579]: I0703 11:12:25.917812 2579 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kubelet-config\" (UniqueName: \"kubernetes.io/host-path/7813f158-0f67-411e-84d9-a2a1c51eea61-kubelet-config\") pod \"global-pull-secret-syncer-9pqm8\" (UID: \"7813f158-0f67-411e-84d9-a2a1c51eea61\") " pod="kube-system/global-pull-secret-syncer-9pqm8" Jul 03 11:12:25.917863 ip-10-0-136-0 kubenswrapper[2579]: I0703 11:12:25.917837 2579 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"etc-tuned\" (UniqueName: \"kubernetes.io/empty-dir/99eb1d1a-c732-4365-9da7-ecab8ead5794-etc-tuned\") pod \"tuned-dwp69\" (UID: \"99eb1d1a-c732-4365-9da7-ecab8ead5794\") " pod="openshift-cluster-node-tuning-operator/tuned-dwp69" Jul 03 11:12:25.917863 ip-10-0-136-0 kubenswrapper[2579]: I0703 11:12:25.917865 2579 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-run-ovn-kubernetes\" (UniqueName: \"kubernetes.io/host-path/45f92856-212a-4190-93ae-dfec96e1864e-host-run-ovn-kubernetes\") pod \"ovnkube-node-dchtv\" (UID: \"45f92856-212a-4190-93ae-dfec96e1864e\") " pod="openshift-ovn-kubernetes/ovnkube-node-dchtv" Jul 03 11:12:25.918294 ip-10-0-136-0 kubenswrapper[2579]: I0703 11:12:25.917894 2579 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"system-cni-dir\" (UniqueName: \"kubernetes.io/host-path/e9fc3c32-a741-4377-970b-6cb08e5b4ac9-system-cni-dir\") pod \"multus-additional-cni-plugins-52n9r\" (UID: \"e9fc3c32-a741-4377-970b-6cb08e5b4ac9\") " pod="openshift-multus/multus-additional-cni-plugins-52n9r" Jul 03 11:12:25.918294 ip-10-0-136-0 kubenswrapper[2579]: I0703 11:12:25.917925 2579 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-cfc5q\" (UniqueName: \"kubernetes.io/projected/e9fc3c32-a741-4377-970b-6cb08e5b4ac9-kube-api-access-cfc5q\") pod \"multus-additional-cni-plugins-52n9r\" (UID: \"e9fc3c32-a741-4377-970b-6cb08e5b4ac9\") " pod="openshift-multus/multus-additional-cni-plugins-52n9r" Jul 03 11:12:25.918294 ip-10-0-136-0 kubenswrapper[2579]: I0703 11:12:25.917955 2579 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"etc-kubernetes\" (UniqueName: \"kubernetes.io/host-path/ad5f7b2c-eac2-4ebb-b1d8-62782a621d85-etc-kubernetes\") pod \"multus-cszpw\" (UID: \"ad5f7b2c-eac2-4ebb-b1d8-62782a621d85\") " pod="openshift-multus/multus-cszpw" Jul 03 11:12:25.918294 ip-10-0-136-0 kubenswrapper[2579]: I0703 11:12:25.917991 2579 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"device-dir\" (UniqueName: \"kubernetes.io/host-path/a39e260a-4eff-4acb-9b7f-3ea1f42ff3c0-device-dir\") pod \"aws-ebs-csi-driver-node-fj786\" (UID: \"a39e260a-4eff-4acb-9b7f-3ea1f42ff3c0\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-fj786" Jul 03 11:12:25.918294 ip-10-0-136-0 kubenswrapper[2579]: I0703 11:12:25.918019 2579 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/34ca9c51-4743-4efc-9aa1-eb79a741f6a4-metrics-certs\") pod \"network-metrics-daemon-sqxsj\" (UID: \"34ca9c51-4743-4efc-9aa1-eb79a741f6a4\") " pod="openshift-multus/network-metrics-daemon-sqxsj" Jul 03 11:12:25.918294 ip-10-0-136-0 kubenswrapper[2579]: I0703 11:12:25.918045 2579 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"etc-kubernetes\" (UniqueName: \"kubernetes.io/host-path/99eb1d1a-c732-4365-9da7-ecab8ead5794-etc-kubernetes\") pod \"tuned-dwp69\" (UID: \"99eb1d1a-c732-4365-9da7-ecab8ead5794\") " pod="openshift-cluster-node-tuning-operator/tuned-dwp69" Jul 03 11:12:25.918294 ip-10-0-136-0 kubenswrapper[2579]: I0703 11:12:25.918057 2579 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"konnectivity-ca\" (UniqueName: \"kubernetes.io/configmap/945b7f84-a41f-413e-a7b6-87818468360f-konnectivity-ca\") pod \"konnectivity-agent-5lz4n\" (UID: \"945b7f84-a41f-413e-a7b6-87818468360f\") " pod="kube-system/konnectivity-agent-5lz4n" Jul 03 11:12:25.918294 ip-10-0-136-0 kubenswrapper[2579]: I0703 11:12:25.918071 2579 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"ovnkube-script-lib\" (UniqueName: \"kubernetes.io/configmap/45f92856-212a-4190-93ae-dfec96e1864e-ovnkube-script-lib\") pod \"ovnkube-node-dchtv\" (UID: \"45f92856-212a-4190-93ae-dfec96e1864e\") " pod="openshift-ovn-kubernetes/ovnkube-node-dchtv" Jul 03 11:12:25.918294 ip-10-0-136-0 kubenswrapper[2579]: I0703 11:12:25.918096 2579 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"system-cni-dir\" (UniqueName: \"kubernetes.io/host-path/ad5f7b2c-eac2-4ebb-b1d8-62782a621d85-system-cni-dir\") pod \"multus-cszpw\" (UID: \"ad5f7b2c-eac2-4ebb-b1d8-62782a621d85\") " pod="openshift-multus/multus-cszpw" Jul 03 11:12:25.918294 ip-10-0-136-0 kubenswrapper[2579]: I0703 11:12:25.918255 2579 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"cnibin\" (UniqueName: \"kubernetes.io/host-path/ad5f7b2c-eac2-4ebb-b1d8-62782a621d85-cnibin\") pod \"multus-cszpw\" (UID: \"ad5f7b2c-eac2-4ebb-b1d8-62782a621d85\") " pod="openshift-multus/multus-cszpw" Jul 03 11:12:25.918584 ip-10-0-136-0 kubenswrapper[2579]: I0703 11:12:25.918306 2579 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"multus-conf-dir\" (UniqueName: \"kubernetes.io/host-path/ad5f7b2c-eac2-4ebb-b1d8-62782a621d85-multus-conf-dir\") pod \"multus-cszpw\" (UID: \"ad5f7b2c-eac2-4ebb-b1d8-62782a621d85\") " pod="openshift-multus/multus-cszpw" Jul 03 11:12:25.918584 ip-10-0-136-0 kubenswrapper[2579]: I0703 11:12:25.918333 2579 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-run-multus-certs\" (UniqueName: \"kubernetes.io/host-path/ad5f7b2c-eac2-4ebb-b1d8-62782a621d85-host-run-multus-certs\") pod \"multus-cszpw\" (UID: \"ad5f7b2c-eac2-4ebb-b1d8-62782a621d85\") " pod="openshift-multus/multus-cszpw" Jul 03 11:12:25.918584 ip-10-0-136-0 kubenswrapper[2579]: I0703 11:12:25.918361 2579 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kubelet-dir\" (UniqueName: \"kubernetes.io/host-path/a39e260a-4eff-4acb-9b7f-3ea1f42ff3c0-kubelet-dir\") pod \"aws-ebs-csi-driver-node-fj786\" (UID: \"a39e260a-4eff-4acb-9b7f-3ea1f42ff3c0\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-fj786" Jul 03 11:12:25.918584 ip-10-0-136-0 kubenswrapper[2579]: I0703 11:12:25.918393 2579 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-jhfsm\" (UniqueName: \"kubernetes.io/projected/34ca9c51-4743-4efc-9aa1-eb79a741f6a4-kube-api-access-jhfsm\") pod \"network-metrics-daemon-sqxsj\" (UID: \"34ca9c51-4743-4efc-9aa1-eb79a741f6a4\") " pod="openshift-multus/network-metrics-daemon-sqxsj" Jul 03 11:12:25.918584 ip-10-0-136-0 kubenswrapper[2579]: I0703 11:12:25.918418 2579 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"var-lib-openvswitch\" (UniqueName: \"kubernetes.io/host-path/45f92856-212a-4190-93ae-dfec96e1864e-var-lib-openvswitch\") pod \"ovnkube-node-dchtv\" (UID: \"45f92856-212a-4190-93ae-dfec96e1864e\") " pod="openshift-ovn-kubernetes/ovnkube-node-dchtv" Jul 03 11:12:25.918584 ip-10-0-136-0 kubenswrapper[2579]: I0703 11:12:25.918446 2579 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"cni-sysctl-allowlist\" (UniqueName: \"kubernetes.io/configmap/e9fc3c32-a741-4377-970b-6cb08e5b4ac9-cni-sysctl-allowlist\") pod \"multus-additional-cni-plugins-52n9r\" (UID: \"e9fc3c32-a741-4377-970b-6cb08e5b4ac9\") " pod="openshift-multus/multus-additional-cni-plugins-52n9r" Jul 03 11:12:25.918584 ip-10-0-136-0 kubenswrapper[2579]: I0703 11:12:25.918475 2579 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"whereabouts-flatfile-configmap\" (UniqueName: \"kubernetes.io/configmap/e9fc3c32-a741-4377-970b-6cb08e5b4ac9-whereabouts-flatfile-configmap\") pod \"multus-additional-cni-plugins-52n9r\" (UID: \"e9fc3c32-a741-4377-970b-6cb08e5b4ac9\") " pod="openshift-multus/multus-additional-cni-plugins-52n9r" Jul 03 11:12:25.918584 ip-10-0-136-0 kubenswrapper[2579]: I0703 11:12:25.918502 2579 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"serviceca\" (UniqueName: \"kubernetes.io/configmap/f5fa8ffd-d2fc-4474-ad0d-a7af28edf0e7-serviceca\") pod \"node-ca-9ff9g\" (UID: \"f5fa8ffd-d2fc-4474-ad0d-a7af28edf0e7\") " pod="openshift-image-registry/node-ca-9ff9g" Jul 03 11:12:25.918584 ip-10-0-136-0 kubenswrapper[2579]: I0703 11:12:25.918527 2579 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"original-pull-secret\" (UniqueName: \"kubernetes.io/secret/7813f158-0f67-411e-84d9-a2a1c51eea61-original-pull-secret\") pod \"global-pull-secret-syncer-9pqm8\" (UID: \"7813f158-0f67-411e-84d9-a2a1c51eea61\") " pod="kube-system/global-pull-secret-syncer-9pqm8" Jul 03 11:12:25.918584 ip-10-0-136-0 kubenswrapper[2579]: I0703 11:12:25.918518 2579 swap_util.go:74] "error creating dir to test if tmpfs noswap is enabled. Assuming not supported" mount path="" error="stat /var/lib/kubelet/plugins/kubernetes.io/empty-dir: no such file or directory" Jul 03 11:12:25.918584 ip-10-0-136-0 kubenswrapper[2579]: I0703 11:12:25.918555 2579 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"sys-fs\" (UniqueName: \"kubernetes.io/host-path/a39e260a-4eff-4acb-9b7f-3ea1f42ff3c0-sys-fs\") pod \"aws-ebs-csi-driver-node-fj786\" (UID: \"a39e260a-4eff-4acb-9b7f-3ea1f42ff3c0\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-fj786" Jul 03 11:12:25.918901 ip-10-0-136-0 kubenswrapper[2579]: I0703 11:12:25.918592 2579 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"node-log\" (UniqueName: \"kubernetes.io/host-path/45f92856-212a-4190-93ae-dfec96e1864e-node-log\") pod \"ovnkube-node-dchtv\" (UID: \"45f92856-212a-4190-93ae-dfec96e1864e\") " pod="openshift-ovn-kubernetes/ovnkube-node-dchtv" Jul 03 11:12:25.918901 ip-10-0-136-0 kubenswrapper[2579]: I0703 11:12:25.918616 2579 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-var-lib-cni-networks-ovn-kubernetes\" (UniqueName: \"kubernetes.io/host-path/45f92856-212a-4190-93ae-dfec96e1864e-host-var-lib-cni-networks-ovn-kubernetes\") pod \"ovnkube-node-dchtv\" (UID: \"45f92856-212a-4190-93ae-dfec96e1864e\") " pod="openshift-ovn-kubernetes/ovnkube-node-dchtv" Jul 03 11:12:25.918901 ip-10-0-136-0 kubenswrapper[2579]: I0703 11:12:25.918641 2579 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"env-overrides\" (UniqueName: \"kubernetes.io/configmap/45f92856-212a-4190-93ae-dfec96e1864e-env-overrides\") pod \"ovnkube-node-dchtv\" (UID: \"45f92856-212a-4190-93ae-dfec96e1864e\") " pod="openshift-ovn-kubernetes/ovnkube-node-dchtv" Jul 03 11:12:25.918901 ip-10-0-136-0 kubenswrapper[2579]: I0703 11:12:25.918664 2579 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"cnibin\" (UniqueName: \"kubernetes.io/host-path/e9fc3c32-a741-4377-970b-6cb08e5b4ac9-cnibin\") pod \"multus-additional-cni-plugins-52n9r\" (UID: \"e9fc3c32-a741-4377-970b-6cb08e5b4ac9\") " pod="openshift-multus/multus-additional-cni-plugins-52n9r" Jul 03 11:12:25.918901 ip-10-0-136-0 kubenswrapper[2579]: I0703 11:12:25.918691 2579 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-run-k8s-cni-cncf-io\" (UniqueName: \"kubernetes.io/host-path/ad5f7b2c-eac2-4ebb-b1d8-62782a621d85-host-run-k8s-cni-cncf-io\") pod \"multus-cszpw\" (UID: \"ad5f7b2c-eac2-4ebb-b1d8-62782a621d85\") " pod="openshift-multus/multus-cszpw" Jul 03 11:12:25.918901 ip-10-0-136-0 kubenswrapper[2579]: I0703 11:12:25.918715 2579 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"sys\" (UniqueName: \"kubernetes.io/host-path/99eb1d1a-c732-4365-9da7-ecab8ead5794-sys\") pod \"tuned-dwp69\" (UID: \"99eb1d1a-c732-4365-9da7-ecab8ead5794\") " pod="openshift-cluster-node-tuning-operator/tuned-dwp69" Jul 03 11:12:25.918901 ip-10-0-136-0 kubenswrapper[2579]: I0703 11:12:25.918741 2579 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-cni-bin\" (UniqueName: \"kubernetes.io/host-path/45f92856-212a-4190-93ae-dfec96e1864e-host-cni-bin\") pod \"ovnkube-node-dchtv\" (UID: \"45f92856-212a-4190-93ae-dfec96e1864e\") " pod="openshift-ovn-kubernetes/ovnkube-node-dchtv" Jul 03 11:12:25.918901 ip-10-0-136-0 kubenswrapper[2579]: I0703 11:12:25.918768 2579 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"ovn-node-metrics-cert\" (UniqueName: \"kubernetes.io/secret/45f92856-212a-4190-93ae-dfec96e1864e-ovn-node-metrics-cert\") pod \"ovnkube-node-dchtv\" (UID: \"45f92856-212a-4190-93ae-dfec96e1864e\") " pod="openshift-ovn-kubernetes/ovnkube-node-dchtv" Jul 03 11:12:25.918901 ip-10-0-136-0 kubenswrapper[2579]: I0703 11:12:25.918796 2579 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"os-release\" (UniqueName: \"kubernetes.io/host-path/e9fc3c32-a741-4377-970b-6cb08e5b4ac9-os-release\") pod \"multus-additional-cni-plugins-52n9r\" (UID: \"e9fc3c32-a741-4377-970b-6cb08e5b4ac9\") " pod="openshift-multus/multus-additional-cni-plugins-52n9r" Jul 03 11:12:25.918901 ip-10-0-136-0 kubenswrapper[2579]: I0703 11:12:25.918821 2579 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tuning-conf-dir\" (UniqueName: \"kubernetes.io/host-path/e9fc3c32-a741-4377-970b-6cb08e5b4ac9-tuning-conf-dir\") pod \"multus-additional-cni-plugins-52n9r\" (UID: \"e9fc3c32-a741-4377-970b-6cb08e5b4ac9\") " pod="openshift-multus/multus-additional-cni-plugins-52n9r" Jul 03 11:12:25.918901 ip-10-0-136-0 kubenswrapper[2579]: I0703 11:12:25.918846 2579 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-46v9l\" (UniqueName: \"kubernetes.io/projected/9a6053f3-4885-4e53-8b1c-93150a7546c4-kube-api-access-46v9l\") pod \"node-resolver-hnmtl\" (UID: \"9a6053f3-4885-4e53-8b1c-93150a7546c4\") " pod="openshift-dns/node-resolver-hnmtl" Jul 03 11:12:25.918901 ip-10-0-136-0 kubenswrapper[2579]: I0703 11:12:25.918871 2579 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"etc-sysctl-conf\" (UniqueName: \"kubernetes.io/host-path/99eb1d1a-c732-4365-9da7-ecab8ead5794-etc-sysctl-conf\") pod \"tuned-dwp69\" (UID: \"99eb1d1a-c732-4365-9da7-ecab8ead5794\") " pod="openshift-cluster-node-tuning-operator/tuned-dwp69" Jul 03 11:12:25.918901 ip-10-0-136-0 kubenswrapper[2579]: I0703 11:12:25.918897 2579 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"run\" (UniqueName: \"kubernetes.io/host-path/99eb1d1a-c732-4365-9da7-ecab8ead5794-run\") pod \"tuned-dwp69\" (UID: \"99eb1d1a-c732-4365-9da7-ecab8ead5794\") " pod="openshift-cluster-node-tuning-operator/tuned-dwp69" Jul 03 11:12:25.919453 ip-10-0-136-0 kubenswrapper[2579]: I0703 11:12:25.918921 2579 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"log-socket\" (UniqueName: \"kubernetes.io/host-path/45f92856-212a-4190-93ae-dfec96e1864e-log-socket\") pod \"ovnkube-node-dchtv\" (UID: \"45f92856-212a-4190-93ae-dfec96e1864e\") " pod="openshift-ovn-kubernetes/ovnkube-node-dchtv" Jul 03 11:12:25.919453 ip-10-0-136-0 kubenswrapper[2579]: I0703 11:12:25.918946 2579 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"multus-socket-dir-parent\" (UniqueName: \"kubernetes.io/host-path/ad5f7b2c-eac2-4ebb-b1d8-62782a621d85-multus-socket-dir-parent\") pod \"multus-cszpw\" (UID: \"ad5f7b2c-eac2-4ebb-b1d8-62782a621d85\") " pod="openshift-multus/multus-cszpw" Jul 03 11:12:25.919453 ip-10-0-136-0 kubenswrapper[2579]: I0703 11:12:25.918964 2579 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/9a6053f3-4885-4e53-8b1c-93150a7546c4-tmp-dir\") pod \"node-resolver-hnmtl\" (UID: \"9a6053f3-4885-4e53-8b1c-93150a7546c4\") " pod="openshift-dns/node-resolver-hnmtl" Jul 03 11:12:25.919453 ip-10-0-136-0 kubenswrapper[2579]: I0703 11:12:25.918991 2579 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-679vc\" (UniqueName: \"kubernetes.io/projected/a8bc35c6-22ff-4c87-802d-e44999086c38-kube-api-access-679vc\") pod \"iptables-alerter-l8lpk\" (UID: \"a8bc35c6-22ff-4c87-802d-e44999086c38\") " pod="openshift-network-operator/iptables-alerter-l8lpk" Jul 03 11:12:25.919453 ip-10-0-136-0 kubenswrapper[2579]: I0703 11:12:25.919059 2579 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"socket-dir\" (UniqueName: \"kubernetes.io/host-path/a39e260a-4eff-4acb-9b7f-3ea1f42ff3c0-socket-dir\") pod \"aws-ebs-csi-driver-node-fj786\" (UID: \"a39e260a-4eff-4acb-9b7f-3ea1f42ff3c0\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-fj786" Jul 03 11:12:25.919453 ip-10-0-136-0 kubenswrapper[2579]: I0703 11:12:25.919252 2579 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"host\" (UniqueName: \"kubernetes.io/host-path/f5fa8ffd-d2fc-4474-ad0d-a7af28edf0e7-host\") pod \"node-ca-9ff9g\" (UID: \"f5fa8ffd-d2fc-4474-ad0d-a7af28edf0e7\") " pod="openshift-image-registry/node-ca-9ff9g" Jul 03 11:12:25.919453 ip-10-0-136-0 kubenswrapper[2579]: I0703 11:12:25.919402 2579 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"host-var-lib-kubelet\" (UniqueName: \"kubernetes.io/host-path/ad5f7b2c-eac2-4ebb-b1d8-62782a621d85-host-var-lib-kubelet\") pod \"multus-cszpw\" (UID: \"ad5f7b2c-eac2-4ebb-b1d8-62782a621d85\") " pod="openshift-multus/multus-cszpw" Jul 03 11:12:25.920661 ip-10-0-136-0 kubenswrapper[2579]: I0703 11:12:25.920630 2579 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"cni-sysctl-allowlist\" (UniqueName: \"kubernetes.io/configmap/e9fc3c32-a741-4377-970b-6cb08e5b4ac9-cni-sysctl-allowlist\") pod \"multus-additional-cni-plugins-52n9r\" (UID: \"e9fc3c32-a741-4377-970b-6cb08e5b4ac9\") " pod="openshift-multus/multus-additional-cni-plugins-52n9r" Jul 03 11:12:25.920803 ip-10-0-136-0 kubenswrapper[2579]: I0703 11:12:25.920758 2579 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"iptables-alerter-script\" (UniqueName: \"kubernetes.io/configmap/a8bc35c6-22ff-4c87-802d-e44999086c38-iptables-alerter-script\") pod \"iptables-alerter-l8lpk\" (UID: \"a8bc35c6-22ff-4c87-802d-e44999086c38\") " pod="openshift-network-operator/iptables-alerter-l8lpk" Jul 03 11:12:25.920987 ip-10-0-136-0 kubenswrapper[2579]: I0703 11:12:25.920964 2579 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"serviceca\" (UniqueName: \"kubernetes.io/configmap/f5fa8ffd-d2fc-4474-ad0d-a7af28edf0e7-serviceca\") pod \"node-ca-9ff9g\" (UID: \"f5fa8ffd-d2fc-4474-ad0d-a7af28edf0e7\") " pod="openshift-image-registry/node-ca-9ff9g" Jul 03 11:12:25.924631 ip-10-0-136-0 kubenswrapper[2579]: I0703 11:12:25.921429 2579 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"whereabouts-flatfile-configmap\" (UniqueName: \"kubernetes.io/configmap/e9fc3c32-a741-4377-970b-6cb08e5b4ac9-whereabouts-flatfile-configmap\") pod \"multus-additional-cni-plugins-52n9r\" (UID: \"e9fc3c32-a741-4377-970b-6cb08e5b4ac9\") " pod="openshift-multus/multus-additional-cni-plugins-52n9r" Jul 03 11:12:25.924631 ip-10-0-136-0 kubenswrapper[2579]: I0703 11:12:25.921582 2579 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"os-release\" (UniqueName: \"kubernetes.io/host-path/e9fc3c32-a741-4377-970b-6cb08e5b4ac9-os-release\") pod \"multus-additional-cni-plugins-52n9r\" (UID: \"e9fc3c32-a741-4377-970b-6cb08e5b4ac9\") " pod="openshift-multus/multus-additional-cni-plugins-52n9r" Jul 03 11:12:25.924631 ip-10-0-136-0 kubenswrapper[2579]: I0703 11:12:25.921634 2579 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"multus-cni-dir\" (UniqueName: \"kubernetes.io/host-path/ad5f7b2c-eac2-4ebb-b1d8-62782a621d85-multus-cni-dir\") pod \"multus-cszpw\" (UID: \"ad5f7b2c-eac2-4ebb-b1d8-62782a621d85\") " pod="openshift-multus/multus-cszpw" Jul 03 11:12:25.924631 ip-10-0-136-0 kubenswrapper[2579]: I0703 11:12:25.921669 2579 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"hostroot\" (UniqueName: \"kubernetes.io/host-path/ad5f7b2c-eac2-4ebb-b1d8-62782a621d85-hostroot\") pod \"multus-cszpw\" (UID: \"ad5f7b2c-eac2-4ebb-b1d8-62782a621d85\") " pod="openshift-multus/multus-cszpw" Jul 03 11:12:25.924631 ip-10-0-136-0 kubenswrapper[2579]: I0703 11:12:25.921696 2579 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"multus-daemon-config\" (UniqueName: \"kubernetes.io/configmap/ad5f7b2c-eac2-4ebb-b1d8-62782a621d85-multus-daemon-config\") pod \"multus-cszpw\" (UID: \"ad5f7b2c-eac2-4ebb-b1d8-62782a621d85\") " pod="openshift-multus/multus-cszpw" Jul 03 11:12:25.924631 ip-10-0-136-0 kubenswrapper[2579]: I0703 11:12:25.921746 2579 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-slash\" (UniqueName: \"kubernetes.io/host-path/a8bc35c6-22ff-4c87-802d-e44999086c38-host-slash\") pod \"iptables-alerter-l8lpk\" (UID: \"a8bc35c6-22ff-4c87-802d-e44999086c38\") " pod="openshift-network-operator/iptables-alerter-l8lpk" Jul 03 11:12:25.924631 ip-10-0-136-0 kubenswrapper[2579]: I0703 11:12:25.921786 2579 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-8hslg\" (UniqueName: \"kubernetes.io/projected/a39e260a-4eff-4acb-9b7f-3ea1f42ff3c0-kube-api-access-8hslg\") pod \"aws-ebs-csi-driver-node-fj786\" (UID: \"a39e260a-4eff-4acb-9b7f-3ea1f42ff3c0\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-fj786" Jul 03 11:12:25.924631 ip-10-0-136-0 kubenswrapper[2579]: I0703 11:12:25.921814 2579 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-cni-netd\" (UniqueName: \"kubernetes.io/host-path/45f92856-212a-4190-93ae-dfec96e1864e-host-cni-netd\") pod \"ovnkube-node-dchtv\" (UID: \"45f92856-212a-4190-93ae-dfec96e1864e\") " pod="openshift-ovn-kubernetes/ovnkube-node-dchtv" Jul 03 11:12:25.924631 ip-10-0-136-0 kubenswrapper[2579]: I0703 11:12:25.921864 2579 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-whgx4\" (UniqueName: \"kubernetes.io/projected/45f92856-212a-4190-93ae-dfec96e1864e-kube-api-access-whgx4\") pod \"ovnkube-node-dchtv\" (UID: \"45f92856-212a-4190-93ae-dfec96e1864e\") " pod="openshift-ovn-kubernetes/ovnkube-node-dchtv" Jul 03 11:12:25.924631 ip-10-0-136-0 kubenswrapper[2579]: I0703 11:12:25.921896 2579 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"cni-binary-copy\" (UniqueName: \"kubernetes.io/configmap/e9fc3c32-a741-4377-970b-6cb08e5b4ac9-cni-binary-copy\") pod \"multus-additional-cni-plugins-52n9r\" (UID: \"e9fc3c32-a741-4377-970b-6cb08e5b4ac9\") " pod="openshift-multus/multus-additional-cni-plugins-52n9r" Jul 03 11:12:25.924631 ip-10-0-136-0 kubenswrapper[2579]: I0703 11:12:25.921927 2579 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"os-release\" (UniqueName: \"kubernetes.io/host-path/ad5f7b2c-eac2-4ebb-b1d8-62782a621d85-os-release\") pod \"multus-cszpw\" (UID: \"ad5f7b2c-eac2-4ebb-b1d8-62782a621d85\") " pod="openshift-multus/multus-cszpw" Jul 03 11:12:25.924631 ip-10-0-136-0 kubenswrapper[2579]: I0703 11:12:25.921966 2579 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-var-lib-cni-multus\" (UniqueName: \"kubernetes.io/host-path/ad5f7b2c-eac2-4ebb-b1d8-62782a621d85-host-var-lib-cni-multus\") pod \"multus-cszpw\" (UID: \"ad5f7b2c-eac2-4ebb-b1d8-62782a621d85\") " pod="openshift-multus/multus-cszpw" Jul 03 11:12:25.924631 ip-10-0-136-0 kubenswrapper[2579]: I0703 11:12:25.921992 2579 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"hosts-file\" (UniqueName: \"kubernetes.io/host-path/9a6053f3-4885-4e53-8b1c-93150a7546c4-hosts-file\") pod \"node-resolver-hnmtl\" (UID: \"9a6053f3-4885-4e53-8b1c-93150a7546c4\") " pod="openshift-dns/node-resolver-hnmtl" Jul 03 11:12:25.924631 ip-10-0-136-0 kubenswrapper[2579]: I0703 11:12:25.922022 2579 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"var-lib-kubelet\" (UniqueName: \"kubernetes.io/host-path/99eb1d1a-c732-4365-9da7-ecab8ead5794-var-lib-kubelet\") pod \"tuned-dwp69\" (UID: \"99eb1d1a-c732-4365-9da7-ecab8ead5794\") " pod="openshift-cluster-node-tuning-operator/tuned-dwp69" Jul 03 11:12:25.924631 ip-10-0-136-0 kubenswrapper[2579]: I0703 11:12:25.922051 2579 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-run-netns\" (UniqueName: \"kubernetes.io/host-path/45f92856-212a-4190-93ae-dfec96e1864e-host-run-netns\") pod \"ovnkube-node-dchtv\" (UID: \"45f92856-212a-4190-93ae-dfec96e1864e\") " pod="openshift-ovn-kubernetes/ovnkube-node-dchtv" Jul 03 11:12:25.924631 ip-10-0-136-0 kubenswrapper[2579]: I0703 11:12:25.922427 2579 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"tuning-conf-dir\" (UniqueName: \"kubernetes.io/host-path/e9fc3c32-a741-4377-970b-6cb08e5b4ac9-tuning-conf-dir\") pod \"multus-additional-cni-plugins-52n9r\" (UID: \"e9fc3c32-a741-4377-970b-6cb08e5b4ac9\") " pod="openshift-multus/multus-additional-cni-plugins-52n9r" Jul 03 11:12:25.924631 ip-10-0-136-0 kubenswrapper[2579]: I0703 11:12:25.922499 2579 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"host-run-netns\" (UniqueName: \"kubernetes.io/host-path/ad5f7b2c-eac2-4ebb-b1d8-62782a621d85-host-run-netns\") pod \"multus-cszpw\" (UID: \"ad5f7b2c-eac2-4ebb-b1d8-62782a621d85\") " pod="openshift-multus/multus-cszpw" Jul 03 11:12:25.925516 ip-10-0-136-0 kubenswrapper[2579]: I0703 11:12:25.922561 2579 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"multus-cni-dir\" (UniqueName: \"kubernetes.io/host-path/ad5f7b2c-eac2-4ebb-b1d8-62782a621d85-multus-cni-dir\") pod \"multus-cszpw\" (UID: \"ad5f7b2c-eac2-4ebb-b1d8-62782a621d85\") " pod="openshift-multus/multus-cszpw" Jul 03 11:12:25.925516 ip-10-0-136-0 kubenswrapper[2579]: I0703 11:12:25.922542 2579 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"multus-socket-dir-parent\" (UniqueName: \"kubernetes.io/host-path/ad5f7b2c-eac2-4ebb-b1d8-62782a621d85-multus-socket-dir-parent\") pod \"multus-cszpw\" (UID: \"ad5f7b2c-eac2-4ebb-b1d8-62782a621d85\") " pod="openshift-multus/multus-cszpw" Jul 03 11:12:25.925516 ip-10-0-136-0 kubenswrapper[2579]: I0703 11:12:25.922620 2579 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"hostroot\" (UniqueName: \"kubernetes.io/host-path/ad5f7b2c-eac2-4ebb-b1d8-62782a621d85-hostroot\") pod \"multus-cszpw\" (UID: \"ad5f7b2c-eac2-4ebb-b1d8-62782a621d85\") " pod="openshift-multus/multus-cszpw" Jul 03 11:12:25.925516 ip-10-0-136-0 kubenswrapper[2579]: I0703 11:12:25.922634 2579 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"multus-conf-dir\" (UniqueName: \"kubernetes.io/host-path/ad5f7b2c-eac2-4ebb-b1d8-62782a621d85-multus-conf-dir\") pod \"multus-cszpw\" (UID: \"ad5f7b2c-eac2-4ebb-b1d8-62782a621d85\") " pod="openshift-multus/multus-cszpw" Jul 03 11:12:25.925516 ip-10-0-136-0 kubenswrapper[2579]: I0703 11:12:25.922709 2579 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"system-cni-dir\" (UniqueName: \"kubernetes.io/host-path/ad5f7b2c-eac2-4ebb-b1d8-62782a621d85-system-cni-dir\") pod \"multus-cszpw\" (UID: \"ad5f7b2c-eac2-4ebb-b1d8-62782a621d85\") " pod="openshift-multus/multus-cszpw" Jul 03 11:12:25.925516 ip-10-0-136-0 kubenswrapper[2579]: I0703 11:12:25.923084 2579 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"multus-daemon-config\" (UniqueName: \"kubernetes.io/configmap/ad5f7b2c-eac2-4ebb-b1d8-62782a621d85-multus-daemon-config\") pod \"multus-cszpw\" (UID: \"ad5f7b2c-eac2-4ebb-b1d8-62782a621d85\") " pod="openshift-multus/multus-cszpw" Jul 03 11:12:25.925516 ip-10-0-136-0 kubenswrapper[2579]: I0703 11:12:25.923354 2579 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"host-var-lib-cni-multus\" (UniqueName: \"kubernetes.io/host-path/ad5f7b2c-eac2-4ebb-b1d8-62782a621d85-host-var-lib-cni-multus\") pod \"multus-cszpw\" (UID: \"ad5f7b2c-eac2-4ebb-b1d8-62782a621d85\") " pod="openshift-multus/multus-cszpw" Jul 03 11:12:25.925516 ip-10-0-136-0 kubenswrapper[2579]: I0703 11:12:25.923389 2579 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"os-release\" (UniqueName: \"kubernetes.io/host-path/ad5f7b2c-eac2-4ebb-b1d8-62782a621d85-os-release\") pod \"multus-cszpw\" (UID: \"ad5f7b2c-eac2-4ebb-b1d8-62782a621d85\") " pod="openshift-multus/multus-cszpw" Jul 03 11:12:25.925516 ip-10-0-136-0 kubenswrapper[2579]: I0703 11:12:25.923569 2579 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"host-slash\" (UniqueName: \"kubernetes.io/host-path/a8bc35c6-22ff-4c87-802d-e44999086c38-host-slash\") pod \"iptables-alerter-l8lpk\" (UID: \"a8bc35c6-22ff-4c87-802d-e44999086c38\") " pod="openshift-network-operator/iptables-alerter-l8lpk" Jul 03 11:12:25.925516 ip-10-0-136-0 kubenswrapper[2579]: I0703 11:12:25.923908 2579 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"host-run-k8s-cni-cncf-io\" (UniqueName: \"kubernetes.io/host-path/ad5f7b2c-eac2-4ebb-b1d8-62782a621d85-host-run-k8s-cni-cncf-io\") pod \"multus-cszpw\" (UID: \"ad5f7b2c-eac2-4ebb-b1d8-62782a621d85\") " pod="openshift-multus/multus-cszpw" Jul 03 11:12:25.925516 ip-10-0-136-0 kubenswrapper[2579]: I0703 11:12:25.923949 2579 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"cni-binary-copy\" (UniqueName: \"kubernetes.io/configmap/e9fc3c32-a741-4377-970b-6cb08e5b4ac9-cni-binary-copy\") pod \"multus-additional-cni-plugins-52n9r\" (UID: \"e9fc3c32-a741-4377-970b-6cb08e5b4ac9\") " pod="openshift-multus/multus-additional-cni-plugins-52n9r" Jul 03 11:12:25.925516 ip-10-0-136-0 kubenswrapper[2579]: I0703 11:12:25.924078 2579 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"host-run-multus-certs\" (UniqueName: \"kubernetes.io/host-path/ad5f7b2c-eac2-4ebb-b1d8-62782a621d85-host-run-multus-certs\") pod \"multus-cszpw\" (UID: \"ad5f7b2c-eac2-4ebb-b1d8-62782a621d85\") " pod="openshift-multus/multus-cszpw" Jul 03 11:12:25.925516 ip-10-0-136-0 kubenswrapper[2579]: I0703 11:12:25.924417 2579 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/9a6053f3-4885-4e53-8b1c-93150a7546c4-tmp-dir\") pod \"node-resolver-hnmtl\" (UID: \"9a6053f3-4885-4e53-8b1c-93150a7546c4\") " pod="openshift-dns/node-resolver-hnmtl" Jul 03 11:12:25.925516 ip-10-0-136-0 kubenswrapper[2579]: I0703 11:12:25.924491 2579 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"cnibin\" (UniqueName: \"kubernetes.io/host-path/e9fc3c32-a741-4377-970b-6cb08e5b4ac9-cnibin\") pod \"multus-additional-cni-plugins-52n9r\" (UID: \"e9fc3c32-a741-4377-970b-6cb08e5b4ac9\") " pod="openshift-multus/multus-additional-cni-plugins-52n9r" Jul 03 11:12:25.925516 ip-10-0-136-0 kubenswrapper[2579]: I0703 11:12:25.924565 2579 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"etc-kubernetes\" (UniqueName: \"kubernetes.io/host-path/ad5f7b2c-eac2-4ebb-b1d8-62782a621d85-etc-kubernetes\") pod \"multus-cszpw\" (UID: \"ad5f7b2c-eac2-4ebb-b1d8-62782a621d85\") " pod="openshift-multus/multus-cszpw" Jul 03 11:12:25.925516 ip-10-0-136-0 kubenswrapper[2579]: I0703 11:12:25.924578 2579 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"agent-certs\" (UniqueName: \"kubernetes.io/secret/945b7f84-a41f-413e-a7b6-87818468360f-agent-certs\") pod \"konnectivity-agent-5lz4n\" (UID: \"945b7f84-a41f-413e-a7b6-87818468360f\") " pod="kube-system/konnectivity-agent-5lz4n" Jul 03 11:12:25.925516 ip-10-0-136-0 kubenswrapper[2579]: I0703 11:12:25.924610 2579 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"system-cni-dir\" (UniqueName: \"kubernetes.io/host-path/e9fc3c32-a741-4377-970b-6cb08e5b4ac9-system-cni-dir\") pod \"multus-additional-cni-plugins-52n9r\" (UID: \"e9fc3c32-a741-4377-970b-6cb08e5b4ac9\") " pod="openshift-multus/multus-additional-cni-plugins-52n9r" Jul 03 11:12:25.925516 ip-10-0-136-0 kubenswrapper[2579]: I0703 11:12:25.924767 2579 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"hosts-file\" (UniqueName: \"kubernetes.io/host-path/9a6053f3-4885-4e53-8b1c-93150a7546c4-hosts-file\") pod \"node-resolver-hnmtl\" (UID: \"9a6053f3-4885-4e53-8b1c-93150a7546c4\") " pod="openshift-dns/node-resolver-hnmtl" Jul 03 11:12:25.938716 ip-10-0-136-0 kubenswrapper[2579]: I0703 11:12:25.938674 2579 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-cfc5q\" (UniqueName: \"kubernetes.io/projected/e9fc3c32-a741-4377-970b-6cb08e5b4ac9-kube-api-access-cfc5q\") pod \"multus-additional-cni-plugins-52n9r\" (UID: \"e9fc3c32-a741-4377-970b-6cb08e5b4ac9\") " pod="openshift-multus/multus-additional-cni-plugins-52n9r" Jul 03 11:12:25.941001 ip-10-0-136-0 kubenswrapper[2579]: I0703 11:12:25.940784 2579 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-46v9l\" (UniqueName: \"kubernetes.io/projected/9a6053f3-4885-4e53-8b1c-93150a7546c4-kube-api-access-46v9l\") pod \"node-resolver-hnmtl\" (UID: \"9a6053f3-4885-4e53-8b1c-93150a7546c4\") " pod="openshift-dns/node-resolver-hnmtl" Jul 03 11:12:25.942227 ip-10-0-136-0 kubenswrapper[2579]: I0703 11:12:25.941606 2579 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-wkhsq\" (UniqueName: \"kubernetes.io/projected/ad5f7b2c-eac2-4ebb-b1d8-62782a621d85-kube-api-access-wkhsq\") pod \"multus-cszpw\" (UID: \"ad5f7b2c-eac2-4ebb-b1d8-62782a621d85\") " pod="openshift-multus/multus-cszpw" Jul 03 11:12:25.942672 ip-10-0-136-0 kubenswrapper[2579]: I0703 11:12:25.942652 2579 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-dwqcd\" (UniqueName: \"kubernetes.io/projected/f5fa8ffd-d2fc-4474-ad0d-a7af28edf0e7-kube-api-access-dwqcd\") pod \"node-ca-9ff9g\" (UID: \"f5fa8ffd-d2fc-4474-ad0d-a7af28edf0e7\") " pod="openshift-image-registry/node-ca-9ff9g" Jul 03 11:12:25.944368 ip-10-0-136-0 kubenswrapper[2579]: I0703 11:12:25.944342 2579 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-679vc\" (UniqueName: \"kubernetes.io/projected/a8bc35c6-22ff-4c87-802d-e44999086c38-kube-api-access-679vc\") pod \"iptables-alerter-l8lpk\" (UID: \"a8bc35c6-22ff-4c87-802d-e44999086c38\") " pod="openshift-network-operator/iptables-alerter-l8lpk" Jul 03 11:12:25.998470 ip-10-0-136-0 kubenswrapper[2579]: I0703 11:12:25.998438 2579 reflector.go:439] "Caches populated" type="*v1.RuntimeClass" reflector="k8s.io/client-go/informers/factory.go:160" Jul 03 11:12:26.022864 ip-10-0-136-0 kubenswrapper[2579]: I0703 11:12:26.022816 2579 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"run-openvswitch\" (UniqueName: \"kubernetes.io/host-path/45f92856-212a-4190-93ae-dfec96e1864e-run-openvswitch\") pod \"ovnkube-node-dchtv\" (UID: \"45f92856-212a-4190-93ae-dfec96e1864e\") " pod="openshift-ovn-kubernetes/ovnkube-node-dchtv" Jul 03 11:12:26.022864 ip-10-0-136-0 kubenswrapper[2579]: I0703 11:12:26.022870 2579 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"dbus\" (UniqueName: \"kubernetes.io/host-path/7813f158-0f67-411e-84d9-a2a1c51eea61-dbus\") pod \"global-pull-secret-syncer-9pqm8\" (UID: \"7813f158-0f67-411e-84d9-a2a1c51eea61\") " pod="kube-system/global-pull-secret-syncer-9pqm8" Jul 03 11:12:26.023067 ip-10-0-136-0 kubenswrapper[2579]: I0703 11:12:26.022902 2579 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-flqqx\" (UniqueName: \"kubernetes.io/projected/2129ac30-1645-4555-b722-55963ffb8776-kube-api-access-flqqx\") pod \"network-check-target-xzlsc\" (UID: \"2129ac30-1645-4555-b722-55963ffb8776\") " pod="openshift-network-diagnostics/network-check-target-xzlsc" Jul 03 11:12:26.023067 ip-10-0-136-0 kubenswrapper[2579]: I0703 11:12:26.022930 2579 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host\" (UniqueName: \"kubernetes.io/host-path/99eb1d1a-c732-4365-9da7-ecab8ead5794-host\") pod \"tuned-dwp69\" (UID: \"99eb1d1a-c732-4365-9da7-ecab8ead5794\") " pod="openshift-cluster-node-tuning-operator/tuned-dwp69" Jul 03 11:12:26.023067 ip-10-0-136-0 kubenswrapper[2579]: I0703 11:12:26.022937 2579 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"run-openvswitch\" (UniqueName: \"kubernetes.io/host-path/45f92856-212a-4190-93ae-dfec96e1864e-run-openvswitch\") pod \"ovnkube-node-dchtv\" (UID: \"45f92856-212a-4190-93ae-dfec96e1864e\") " pod="openshift-ovn-kubernetes/ovnkube-node-dchtv" Jul 03 11:12:26.023067 ip-10-0-136-0 kubenswrapper[2579]: I0703 11:12:26.022960 2579 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"systemd-units\" (UniqueName: \"kubernetes.io/host-path/45f92856-212a-4190-93ae-dfec96e1864e-systemd-units\") pod \"ovnkube-node-dchtv\" (UID: \"45f92856-212a-4190-93ae-dfec96e1864e\") " pod="openshift-ovn-kubernetes/ovnkube-node-dchtv" Jul 03 11:12:26.023067 ip-10-0-136-0 kubenswrapper[2579]: I0703 11:12:26.023003 2579 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"etc-selinux\" (UniqueName: \"kubernetes.io/host-path/a39e260a-4eff-4acb-9b7f-3ea1f42ff3c0-etc-selinux\") pod \"aws-ebs-csi-driver-node-fj786\" (UID: \"a39e260a-4eff-4acb-9b7f-3ea1f42ff3c0\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-fj786" Jul 03 11:12:26.023067 ip-10-0-136-0 kubenswrapper[2579]: I0703 11:12:26.023028 2579 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"ovnkube-config\" (UniqueName: \"kubernetes.io/configmap/45f92856-212a-4190-93ae-dfec96e1864e-ovnkube-config\") pod \"ovnkube-node-dchtv\" (UID: \"45f92856-212a-4190-93ae-dfec96e1864e\") " pod="openshift-ovn-kubernetes/ovnkube-node-dchtv" Jul 03 11:12:26.023067 ip-10-0-136-0 kubenswrapper[2579]: I0703 11:12:26.023053 2579 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"etc-systemd\" (UniqueName: \"kubernetes.io/host-path/99eb1d1a-c732-4365-9da7-ecab8ead5794-etc-systemd\") pod \"tuned-dwp69\" (UID: \"99eb1d1a-c732-4365-9da7-ecab8ead5794\") " pod="openshift-cluster-node-tuning-operator/tuned-dwp69" Jul 03 11:12:26.023396 ip-10-0-136-0 kubenswrapper[2579]: I0703 11:12:26.023076 2579 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"etc-openvswitch\" (UniqueName: \"kubernetes.io/host-path/45f92856-212a-4190-93ae-dfec96e1864e-etc-openvswitch\") pod \"ovnkube-node-dchtv\" (UID: \"45f92856-212a-4190-93ae-dfec96e1864e\") " pod="openshift-ovn-kubernetes/ovnkube-node-dchtv" Jul 03 11:12:26.023396 ip-10-0-136-0 kubenswrapper[2579]: I0703 11:12:26.023099 2579 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"run-ovn\" (UniqueName: \"kubernetes.io/host-path/45f92856-212a-4190-93ae-dfec96e1864e-run-ovn\") pod \"ovnkube-node-dchtv\" (UID: \"45f92856-212a-4190-93ae-dfec96e1864e\") " pod="openshift-ovn-kubernetes/ovnkube-node-dchtv" Jul 03 11:12:26.023396 ip-10-0-136-0 kubenswrapper[2579]: I0703 11:12:26.023127 2579 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kubelet-config\" (UniqueName: \"kubernetes.io/host-path/7813f158-0f67-411e-84d9-a2a1c51eea61-kubelet-config\") pod \"global-pull-secret-syncer-9pqm8\" (UID: \"7813f158-0f67-411e-84d9-a2a1c51eea61\") " pod="kube-system/global-pull-secret-syncer-9pqm8" Jul 03 11:12:26.023396 ip-10-0-136-0 kubenswrapper[2579]: I0703 11:12:26.023154 2579 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"etc-tuned\" (UniqueName: \"kubernetes.io/empty-dir/99eb1d1a-c732-4365-9da7-ecab8ead5794-etc-tuned\") pod \"tuned-dwp69\" (UID: \"99eb1d1a-c732-4365-9da7-ecab8ead5794\") " pod="openshift-cluster-node-tuning-operator/tuned-dwp69" Jul 03 11:12:26.023396 ip-10-0-136-0 kubenswrapper[2579]: I0703 11:12:26.023178 2579 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-run-ovn-kubernetes\" (UniqueName: \"kubernetes.io/host-path/45f92856-212a-4190-93ae-dfec96e1864e-host-run-ovn-kubernetes\") pod \"ovnkube-node-dchtv\" (UID: \"45f92856-212a-4190-93ae-dfec96e1864e\") " pod="openshift-ovn-kubernetes/ovnkube-node-dchtv" Jul 03 11:12:26.023396 ip-10-0-136-0 kubenswrapper[2579]: I0703 11:12:26.023247 2579 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"device-dir\" (UniqueName: \"kubernetes.io/host-path/a39e260a-4eff-4acb-9b7f-3ea1f42ff3c0-device-dir\") pod \"aws-ebs-csi-driver-node-fj786\" (UID: \"a39e260a-4eff-4acb-9b7f-3ea1f42ff3c0\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-fj786" Jul 03 11:12:26.023396 ip-10-0-136-0 kubenswrapper[2579]: I0703 11:12:26.023281 2579 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/34ca9c51-4743-4efc-9aa1-eb79a741f6a4-metrics-certs\") pod \"network-metrics-daemon-sqxsj\" (UID: \"34ca9c51-4743-4efc-9aa1-eb79a741f6a4\") " pod="openshift-multus/network-metrics-daemon-sqxsj" Jul 03 11:12:26.023396 ip-10-0-136-0 kubenswrapper[2579]: I0703 11:12:26.023350 2579 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"etc-kubernetes\" (UniqueName: \"kubernetes.io/host-path/99eb1d1a-c732-4365-9da7-ecab8ead5794-etc-kubernetes\") pod \"tuned-dwp69\" (UID: \"99eb1d1a-c732-4365-9da7-ecab8ead5794\") " pod="openshift-cluster-node-tuning-operator/tuned-dwp69" Jul 03 11:12:26.023396 ip-10-0-136-0 kubenswrapper[2579]: I0703 11:12:26.023359 2579 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"dbus\" (UniqueName: \"kubernetes.io/host-path/7813f158-0f67-411e-84d9-a2a1c51eea61-dbus\") pod \"global-pull-secret-syncer-9pqm8\" (UID: \"7813f158-0f67-411e-84d9-a2a1c51eea61\") " pod="kube-system/global-pull-secret-syncer-9pqm8" Jul 03 11:12:26.023396 ip-10-0-136-0 kubenswrapper[2579]: I0703 11:12:26.023378 2579 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"ovnkube-script-lib\" (UniqueName: \"kubernetes.io/configmap/45f92856-212a-4190-93ae-dfec96e1864e-ovnkube-script-lib\") pod \"ovnkube-node-dchtv\" (UID: \"45f92856-212a-4190-93ae-dfec96e1864e\") " pod="openshift-ovn-kubernetes/ovnkube-node-dchtv" Jul 03 11:12:26.023669 ip-10-0-136-0 kubenswrapper[2579]: I0703 11:12:26.023414 2579 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kubelet-dir\" (UniqueName: \"kubernetes.io/host-path/a39e260a-4eff-4acb-9b7f-3ea1f42ff3c0-kubelet-dir\") pod \"aws-ebs-csi-driver-node-fj786\" (UID: \"a39e260a-4eff-4acb-9b7f-3ea1f42ff3c0\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-fj786" Jul 03 11:12:26.023669 ip-10-0-136-0 kubenswrapper[2579]: I0703 11:12:26.023433 2579 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"run-ovn\" (UniqueName: \"kubernetes.io/host-path/45f92856-212a-4190-93ae-dfec96e1864e-run-ovn\") pod \"ovnkube-node-dchtv\" (UID: \"45f92856-212a-4190-93ae-dfec96e1864e\") " pod="openshift-ovn-kubernetes/ovnkube-node-dchtv" Jul 03 11:12:26.023669 ip-10-0-136-0 kubenswrapper[2579]: I0703 11:12:26.023445 2579 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-jhfsm\" (UniqueName: \"kubernetes.io/projected/34ca9c51-4743-4efc-9aa1-eb79a741f6a4-kube-api-access-jhfsm\") pod \"network-metrics-daemon-sqxsj\" (UID: \"34ca9c51-4743-4efc-9aa1-eb79a741f6a4\") " pod="openshift-multus/network-metrics-daemon-sqxsj" Jul 03 11:12:26.023669 ip-10-0-136-0 kubenswrapper[2579]: I0703 11:12:26.023474 2579 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"var-lib-openvswitch\" (UniqueName: \"kubernetes.io/host-path/45f92856-212a-4190-93ae-dfec96e1864e-var-lib-openvswitch\") pod \"ovnkube-node-dchtv\" (UID: \"45f92856-212a-4190-93ae-dfec96e1864e\") " pod="openshift-ovn-kubernetes/ovnkube-node-dchtv" Jul 03 11:12:26.023669 ip-10-0-136-0 kubenswrapper[2579]: I0703 11:12:26.023489 2579 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"host\" (UniqueName: \"kubernetes.io/host-path/99eb1d1a-c732-4365-9da7-ecab8ead5794-host\") pod \"tuned-dwp69\" (UID: \"99eb1d1a-c732-4365-9da7-ecab8ead5794\") " pod="openshift-cluster-node-tuning-operator/tuned-dwp69" Jul 03 11:12:26.023669 ip-10-0-136-0 kubenswrapper[2579]: I0703 11:12:26.023544 2579 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"var-lib-openvswitch\" (UniqueName: \"kubernetes.io/host-path/45f92856-212a-4190-93ae-dfec96e1864e-var-lib-openvswitch\") pod \"ovnkube-node-dchtv\" (UID: \"45f92856-212a-4190-93ae-dfec96e1864e\") " pod="openshift-ovn-kubernetes/ovnkube-node-dchtv" Jul 03 11:12:26.023669 ip-10-0-136-0 kubenswrapper[2579]: I0703 11:12:26.023546 2579 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"etc-openvswitch\" (UniqueName: \"kubernetes.io/host-path/45f92856-212a-4190-93ae-dfec96e1864e-etc-openvswitch\") pod \"ovnkube-node-dchtv\" (UID: \"45f92856-212a-4190-93ae-dfec96e1864e\") " pod="openshift-ovn-kubernetes/ovnkube-node-dchtv" Jul 03 11:12:26.023669 ip-10-0-136-0 kubenswrapper[2579]: I0703 11:12:26.023575 2579 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"original-pull-secret\" (UniqueName: \"kubernetes.io/secret/7813f158-0f67-411e-84d9-a2a1c51eea61-original-pull-secret\") pod \"global-pull-secret-syncer-9pqm8\" (UID: \"7813f158-0f67-411e-84d9-a2a1c51eea61\") " pod="kube-system/global-pull-secret-syncer-9pqm8" Jul 03 11:12:26.023669 ip-10-0-136-0 kubenswrapper[2579]: I0703 11:12:26.023604 2579 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"sys-fs\" (UniqueName: \"kubernetes.io/host-path/a39e260a-4eff-4acb-9b7f-3ea1f42ff3c0-sys-fs\") pod \"aws-ebs-csi-driver-node-fj786\" (UID: \"a39e260a-4eff-4acb-9b7f-3ea1f42ff3c0\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-fj786" Jul 03 11:12:26.023669 ip-10-0-136-0 kubenswrapper[2579]: I0703 11:12:26.023630 2579 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"node-log\" (UniqueName: \"kubernetes.io/host-path/45f92856-212a-4190-93ae-dfec96e1864e-node-log\") pod \"ovnkube-node-dchtv\" (UID: \"45f92856-212a-4190-93ae-dfec96e1864e\") " pod="openshift-ovn-kubernetes/ovnkube-node-dchtv" Jul 03 11:12:26.023669 ip-10-0-136-0 kubenswrapper[2579]: I0703 11:12:26.023656 2579 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-var-lib-cni-networks-ovn-kubernetes\" (UniqueName: \"kubernetes.io/host-path/45f92856-212a-4190-93ae-dfec96e1864e-host-var-lib-cni-networks-ovn-kubernetes\") pod \"ovnkube-node-dchtv\" (UID: \"45f92856-212a-4190-93ae-dfec96e1864e\") " pod="openshift-ovn-kubernetes/ovnkube-node-dchtv" Jul 03 11:12:26.023971 ip-10-0-136-0 kubenswrapper[2579]: I0703 11:12:26.023682 2579 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"env-overrides\" (UniqueName: \"kubernetes.io/configmap/45f92856-212a-4190-93ae-dfec96e1864e-env-overrides\") pod \"ovnkube-node-dchtv\" (UID: \"45f92856-212a-4190-93ae-dfec96e1864e\") " pod="openshift-ovn-kubernetes/ovnkube-node-dchtv" Jul 03 11:12:26.023971 ip-10-0-136-0 kubenswrapper[2579]: I0703 11:12:26.023706 2579 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"sys\" (UniqueName: \"kubernetes.io/host-path/99eb1d1a-c732-4365-9da7-ecab8ead5794-sys\") pod \"tuned-dwp69\" (UID: \"99eb1d1a-c732-4365-9da7-ecab8ead5794\") " pod="openshift-cluster-node-tuning-operator/tuned-dwp69" Jul 03 11:12:26.023971 ip-10-0-136-0 kubenswrapper[2579]: I0703 11:12:26.023728 2579 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-cni-bin\" (UniqueName: \"kubernetes.io/host-path/45f92856-212a-4190-93ae-dfec96e1864e-host-cni-bin\") pod \"ovnkube-node-dchtv\" (UID: \"45f92856-212a-4190-93ae-dfec96e1864e\") " pod="openshift-ovn-kubernetes/ovnkube-node-dchtv" Jul 03 11:12:26.023971 ip-10-0-136-0 kubenswrapper[2579]: I0703 11:12:26.023752 2579 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"ovn-node-metrics-cert\" (UniqueName: \"kubernetes.io/secret/45f92856-212a-4190-93ae-dfec96e1864e-ovn-node-metrics-cert\") pod \"ovnkube-node-dchtv\" (UID: \"45f92856-212a-4190-93ae-dfec96e1864e\") " pod="openshift-ovn-kubernetes/ovnkube-node-dchtv" Jul 03 11:12:26.023971 ip-10-0-136-0 kubenswrapper[2579]: I0703 11:12:26.023780 2579 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"etc-sysctl-conf\" (UniqueName: \"kubernetes.io/host-path/99eb1d1a-c732-4365-9da7-ecab8ead5794-etc-sysctl-conf\") pod \"tuned-dwp69\" (UID: \"99eb1d1a-c732-4365-9da7-ecab8ead5794\") " pod="openshift-cluster-node-tuning-operator/tuned-dwp69" Jul 03 11:12:26.023971 ip-10-0-136-0 kubenswrapper[2579]: I0703 11:12:26.023803 2579 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"run\" (UniqueName: \"kubernetes.io/host-path/99eb1d1a-c732-4365-9da7-ecab8ead5794-run\") pod \"tuned-dwp69\" (UID: \"99eb1d1a-c732-4365-9da7-ecab8ead5794\") " pod="openshift-cluster-node-tuning-operator/tuned-dwp69" Jul 03 11:12:26.023971 ip-10-0-136-0 kubenswrapper[2579]: I0703 11:12:26.023824 2579 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"log-socket\" (UniqueName: \"kubernetes.io/host-path/45f92856-212a-4190-93ae-dfec96e1864e-log-socket\") pod \"ovnkube-node-dchtv\" (UID: \"45f92856-212a-4190-93ae-dfec96e1864e\") " pod="openshift-ovn-kubernetes/ovnkube-node-dchtv" Jul 03 11:12:26.023971 ip-10-0-136-0 kubenswrapper[2579]: I0703 11:12:26.023855 2579 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"socket-dir\" (UniqueName: \"kubernetes.io/host-path/a39e260a-4eff-4acb-9b7f-3ea1f42ff3c0-socket-dir\") pod \"aws-ebs-csi-driver-node-fj786\" (UID: \"a39e260a-4eff-4acb-9b7f-3ea1f42ff3c0\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-fj786" Jul 03 11:12:26.023971 ip-10-0-136-0 kubenswrapper[2579]: I0703 11:12:26.023890 2579 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-8hslg\" (UniqueName: \"kubernetes.io/projected/a39e260a-4eff-4acb-9b7f-3ea1f42ff3c0-kube-api-access-8hslg\") pod \"aws-ebs-csi-driver-node-fj786\" (UID: \"a39e260a-4eff-4acb-9b7f-3ea1f42ff3c0\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-fj786" Jul 03 11:12:26.023971 ip-10-0-136-0 kubenswrapper[2579]: I0703 11:12:26.023917 2579 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-cni-netd\" (UniqueName: \"kubernetes.io/host-path/45f92856-212a-4190-93ae-dfec96e1864e-host-cni-netd\") pod \"ovnkube-node-dchtv\" (UID: \"45f92856-212a-4190-93ae-dfec96e1864e\") " pod="openshift-ovn-kubernetes/ovnkube-node-dchtv" Jul 03 11:12:26.023971 ip-10-0-136-0 kubenswrapper[2579]: I0703 11:12:26.023944 2579 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-whgx4\" (UniqueName: \"kubernetes.io/projected/45f92856-212a-4190-93ae-dfec96e1864e-kube-api-access-whgx4\") pod \"ovnkube-node-dchtv\" (UID: \"45f92856-212a-4190-93ae-dfec96e1864e\") " pod="openshift-ovn-kubernetes/ovnkube-node-dchtv" Jul 03 11:12:26.024317 ip-10-0-136-0 kubenswrapper[2579]: I0703 11:12:26.023977 2579 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"var-lib-kubelet\" (UniqueName: \"kubernetes.io/host-path/99eb1d1a-c732-4365-9da7-ecab8ead5794-var-lib-kubelet\") pod \"tuned-dwp69\" (UID: \"99eb1d1a-c732-4365-9da7-ecab8ead5794\") " pod="openshift-cluster-node-tuning-operator/tuned-dwp69" Jul 03 11:12:26.024317 ip-10-0-136-0 kubenswrapper[2579]: I0703 11:12:26.024004 2579 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-run-netns\" (UniqueName: \"kubernetes.io/host-path/45f92856-212a-4190-93ae-dfec96e1864e-host-run-netns\") pod \"ovnkube-node-dchtv\" (UID: \"45f92856-212a-4190-93ae-dfec96e1864e\") " pod="openshift-ovn-kubernetes/ovnkube-node-dchtv" Jul 03 11:12:26.024317 ip-10-0-136-0 kubenswrapper[2579]: I0703 11:12:26.024032 2579 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"lib-modules\" (UniqueName: \"kubernetes.io/host-path/99eb1d1a-c732-4365-9da7-ecab8ead5794-lib-modules\") pod \"tuned-dwp69\" (UID: \"99eb1d1a-c732-4365-9da7-ecab8ead5794\") " pod="openshift-cluster-node-tuning-operator/tuned-dwp69" Jul 03 11:12:26.024317 ip-10-0-136-0 kubenswrapper[2579]: I0703 11:12:26.024079 2579 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"run-systemd\" (UniqueName: \"kubernetes.io/host-path/45f92856-212a-4190-93ae-dfec96e1864e-run-systemd\") pod \"ovnkube-node-dchtv\" (UID: \"45f92856-212a-4190-93ae-dfec96e1864e\") " pod="openshift-ovn-kubernetes/ovnkube-node-dchtv" Jul 03 11:12:26.024317 ip-10-0-136-0 kubenswrapper[2579]: I0703 11:12:26.024110 2579 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"etc-modprobe-d\" (UniqueName: \"kubernetes.io/host-path/99eb1d1a-c732-4365-9da7-ecab8ead5794-etc-modprobe-d\") pod \"tuned-dwp69\" (UID: \"99eb1d1a-c732-4365-9da7-ecab8ead5794\") " pod="openshift-cluster-node-tuning-operator/tuned-dwp69" Jul 03 11:12:26.024317 ip-10-0-136-0 kubenswrapper[2579]: I0703 11:12:26.024134 2579 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"etc-sysconfig\" (UniqueName: \"kubernetes.io/host-path/99eb1d1a-c732-4365-9da7-ecab8ead5794-etc-sysconfig\") pod \"tuned-dwp69\" (UID: \"99eb1d1a-c732-4365-9da7-ecab8ead5794\") " pod="openshift-cluster-node-tuning-operator/tuned-dwp69" Jul 03 11:12:26.024317 ip-10-0-136-0 kubenswrapper[2579]: I0703 11:12:26.024160 2579 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"etc-sysctl-d\" (UniqueName: \"kubernetes.io/host-path/99eb1d1a-c732-4365-9da7-ecab8ead5794-etc-sysctl-d\") pod \"tuned-dwp69\" (UID: \"99eb1d1a-c732-4365-9da7-ecab8ead5794\") " pod="openshift-cluster-node-tuning-operator/tuned-dwp69" Jul 03 11:12:26.024317 ip-10-0-136-0 kubenswrapper[2579]: I0703 11:12:26.024224 2579 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"registration-dir\" (UniqueName: \"kubernetes.io/host-path/a39e260a-4eff-4acb-9b7f-3ea1f42ff3c0-registration-dir\") pod \"aws-ebs-csi-driver-node-fj786\" (UID: \"a39e260a-4eff-4acb-9b7f-3ea1f42ff3c0\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-fj786" Jul 03 11:12:26.024317 ip-10-0-136-0 kubenswrapper[2579]: I0703 11:12:26.024253 2579 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tmp\" (UniqueName: \"kubernetes.io/empty-dir/99eb1d1a-c732-4365-9da7-ecab8ead5794-tmp\") pod \"tuned-dwp69\" (UID: \"99eb1d1a-c732-4365-9da7-ecab8ead5794\") " pod="openshift-cluster-node-tuning-operator/tuned-dwp69" Jul 03 11:12:26.024317 ip-10-0-136-0 kubenswrapper[2579]: I0703 11:12:26.024279 2579 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-hdp57\" (UniqueName: \"kubernetes.io/projected/99eb1d1a-c732-4365-9da7-ecab8ead5794-kube-api-access-hdp57\") pod \"tuned-dwp69\" (UID: \"99eb1d1a-c732-4365-9da7-ecab8ead5794\") " pod="openshift-cluster-node-tuning-operator/tuned-dwp69" Jul 03 11:12:26.024317 ip-10-0-136-0 kubenswrapper[2579]: I0703 11:12:26.024307 2579 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-kubelet\" (UniqueName: \"kubernetes.io/host-path/45f92856-212a-4190-93ae-dfec96e1864e-host-kubelet\") pod \"ovnkube-node-dchtv\" (UID: \"45f92856-212a-4190-93ae-dfec96e1864e\") " pod="openshift-ovn-kubernetes/ovnkube-node-dchtv" Jul 03 11:12:26.024713 ip-10-0-136-0 kubenswrapper[2579]: I0703 11:12:26.024333 2579 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-slash\" (UniqueName: \"kubernetes.io/host-path/45f92856-212a-4190-93ae-dfec96e1864e-host-slash\") pod \"ovnkube-node-dchtv\" (UID: \"45f92856-212a-4190-93ae-dfec96e1864e\") " pod="openshift-ovn-kubernetes/ovnkube-node-dchtv" Jul 03 11:12:26.024713 ip-10-0-136-0 kubenswrapper[2579]: I0703 11:12:26.023603 2579 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kubelet-config\" (UniqueName: \"kubernetes.io/host-path/7813f158-0f67-411e-84d9-a2a1c51eea61-kubelet-config\") pod \"global-pull-secret-syncer-9pqm8\" (UID: \"7813f158-0f67-411e-84d9-a2a1c51eea61\") " pod="kube-system/global-pull-secret-syncer-9pqm8" Jul 03 11:12:26.024713 ip-10-0-136-0 kubenswrapper[2579]: I0703 11:12:26.024439 2579 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"host-run-ovn-kubernetes\" (UniqueName: \"kubernetes.io/host-path/45f92856-212a-4190-93ae-dfec96e1864e-host-run-ovn-kubernetes\") pod \"ovnkube-node-dchtv\" (UID: \"45f92856-212a-4190-93ae-dfec96e1864e\") " pod="openshift-ovn-kubernetes/ovnkube-node-dchtv" Jul 03 11:12:26.024713 ip-10-0-136-0 kubenswrapper[2579]: I0703 11:12:26.024443 2579 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"host-slash\" (UniqueName: \"kubernetes.io/host-path/45f92856-212a-4190-93ae-dfec96e1864e-host-slash\") pod \"ovnkube-node-dchtv\" (UID: \"45f92856-212a-4190-93ae-dfec96e1864e\") " pod="openshift-ovn-kubernetes/ovnkube-node-dchtv" Jul 03 11:12:26.024713 ip-10-0-136-0 kubenswrapper[2579]: I0703 11:12:26.024598 2579 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"etc-selinux\" (UniqueName: \"kubernetes.io/host-path/a39e260a-4eff-4acb-9b7f-3ea1f42ff3c0-etc-selinux\") pod \"aws-ebs-csi-driver-node-fj786\" (UID: \"a39e260a-4eff-4acb-9b7f-3ea1f42ff3c0\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-fj786" Jul 03 11:12:26.024713 ip-10-0-136-0 kubenswrapper[2579]: I0703 11:12:26.024621 2579 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"host-cni-netd\" (UniqueName: \"kubernetes.io/host-path/45f92856-212a-4190-93ae-dfec96e1864e-host-cni-netd\") pod \"ovnkube-node-dchtv\" (UID: \"45f92856-212a-4190-93ae-dfec96e1864e\") " pod="openshift-ovn-kubernetes/ovnkube-node-dchtv" Jul 03 11:12:26.024713 ip-10-0-136-0 kubenswrapper[2579]: I0703 11:12:26.024675 2579 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"node-log\" (UniqueName: \"kubernetes.io/host-path/45f92856-212a-4190-93ae-dfec96e1864e-node-log\") pod \"ovnkube-node-dchtv\" (UID: \"45f92856-212a-4190-93ae-dfec96e1864e\") " pod="openshift-ovn-kubernetes/ovnkube-node-dchtv" Jul 03 11:12:26.024939 ip-10-0-136-0 kubenswrapper[2579]: E0703 11:12:26.024713 2579 secret.go:189] Couldn't get secret kube-system/original-pull-secret: object "kube-system"/"original-pull-secret" not registered Jul 03 11:12:26.024939 ip-10-0-136-0 kubenswrapper[2579]: I0703 11:12:26.024717 2579 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"host-var-lib-cni-networks-ovn-kubernetes\" (UniqueName: \"kubernetes.io/host-path/45f92856-212a-4190-93ae-dfec96e1864e-host-var-lib-cni-networks-ovn-kubernetes\") pod \"ovnkube-node-dchtv\" (UID: \"45f92856-212a-4190-93ae-dfec96e1864e\") " pod="openshift-ovn-kubernetes/ovnkube-node-dchtv" Jul 03 11:12:26.024939 ip-10-0-136-0 kubenswrapper[2579]: E0703 11:12:26.024823 2579 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/7813f158-0f67-411e-84d9-a2a1c51eea61-original-pull-secret podName:7813f158-0f67-411e-84d9-a2a1c51eea61 nodeName:}" failed. No retries permitted until 2026-07-03 11:12:26.524780299 +0000 UTC m=+3.113084613 (durationBeforeRetry 500ms). Error: MountVolume.SetUp failed for volume "original-pull-secret" (UniqueName: "kubernetes.io/secret/7813f158-0f67-411e-84d9-a2a1c51eea61-original-pull-secret") pod "global-pull-secret-syncer-9pqm8" (UID: "7813f158-0f67-411e-84d9-a2a1c51eea61") : object "kube-system"/"original-pull-secret" not registered Jul 03 11:12:26.025079 ip-10-0-136-0 kubenswrapper[2579]: I0703 11:12:26.025042 2579 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"ovnkube-config\" (UniqueName: \"kubernetes.io/configmap/45f92856-212a-4190-93ae-dfec96e1864e-ovnkube-config\") pod \"ovnkube-node-dchtv\" (UID: \"45f92856-212a-4190-93ae-dfec96e1864e\") " pod="openshift-ovn-kubernetes/ovnkube-node-dchtv" Jul 03 11:12:26.025155 ip-10-0-136-0 kubenswrapper[2579]: I0703 11:12:26.025078 2579 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"env-overrides\" (UniqueName: \"kubernetes.io/configmap/45f92856-212a-4190-93ae-dfec96e1864e-env-overrides\") pod \"ovnkube-node-dchtv\" (UID: \"45f92856-212a-4190-93ae-dfec96e1864e\") " pod="openshift-ovn-kubernetes/ovnkube-node-dchtv" Jul 03 11:12:26.025155 ip-10-0-136-0 kubenswrapper[2579]: I0703 11:12:26.025087 2579 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"device-dir\" (UniqueName: \"kubernetes.io/host-path/a39e260a-4eff-4acb-9b7f-3ea1f42ff3c0-device-dir\") pod \"aws-ebs-csi-driver-node-fj786\" (UID: \"a39e260a-4eff-4acb-9b7f-3ea1f42ff3c0\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-fj786" Jul 03 11:12:26.025155 ip-10-0-136-0 kubenswrapper[2579]: I0703 11:12:26.025121 2579 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"etc-systemd\" (UniqueName: \"kubernetes.io/host-path/99eb1d1a-c732-4365-9da7-ecab8ead5794-etc-systemd\") pod \"tuned-dwp69\" (UID: \"99eb1d1a-c732-4365-9da7-ecab8ead5794\") " pod="openshift-cluster-node-tuning-operator/tuned-dwp69" Jul 03 11:12:26.025155 ip-10-0-136-0 kubenswrapper[2579]: I0703 11:12:26.025142 2579 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"sys\" (UniqueName: \"kubernetes.io/host-path/99eb1d1a-c732-4365-9da7-ecab8ead5794-sys\") pod \"tuned-dwp69\" (UID: \"99eb1d1a-c732-4365-9da7-ecab8ead5794\") " pod="openshift-cluster-node-tuning-operator/tuned-dwp69" Jul 03 11:12:26.025366 ip-10-0-136-0 kubenswrapper[2579]: I0703 11:12:26.025207 2579 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"var-lib-kubelet\" (UniqueName: \"kubernetes.io/host-path/99eb1d1a-c732-4365-9da7-ecab8ead5794-var-lib-kubelet\") pod \"tuned-dwp69\" (UID: \"99eb1d1a-c732-4365-9da7-ecab8ead5794\") " pod="openshift-cluster-node-tuning-operator/tuned-dwp69" Jul 03 11:12:26.025366 ip-10-0-136-0 kubenswrapper[2579]: E0703 11:12:26.025209 2579 secret.go:189] Couldn't get secret openshift-multus/metrics-daemon-secret: object "openshift-multus"/"metrics-daemon-secret" not registered Jul 03 11:12:26.025366 ip-10-0-136-0 kubenswrapper[2579]: I0703 11:12:26.025261 2579 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"etc-kubernetes\" (UniqueName: \"kubernetes.io/host-path/99eb1d1a-c732-4365-9da7-ecab8ead5794-etc-kubernetes\") pod \"tuned-dwp69\" (UID: \"99eb1d1a-c732-4365-9da7-ecab8ead5794\") " pod="openshift-cluster-node-tuning-operator/tuned-dwp69" Jul 03 11:12:26.025366 ip-10-0-136-0 kubenswrapper[2579]: E0703 11:12:26.025279 2579 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/34ca9c51-4743-4efc-9aa1-eb79a741f6a4-metrics-certs podName:34ca9c51-4743-4efc-9aa1-eb79a741f6a4 nodeName:}" failed. No retries permitted until 2026-07-03 11:12:26.525261627 +0000 UTC m=+3.113565924 (durationBeforeRetry 500ms). Error: MountVolume.SetUp failed for volume "metrics-certs" (UniqueName: "kubernetes.io/secret/34ca9c51-4743-4efc-9aa1-eb79a741f6a4-metrics-certs") pod "network-metrics-daemon-sqxsj" (UID: "34ca9c51-4743-4efc-9aa1-eb79a741f6a4") : object "openshift-multus"/"metrics-daemon-secret" not registered Jul 03 11:12:26.025366 ip-10-0-136-0 kubenswrapper[2579]: I0703 11:12:26.025340 2579 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kubelet-dir\" (UniqueName: \"kubernetes.io/host-path/a39e260a-4eff-4acb-9b7f-3ea1f42ff3c0-kubelet-dir\") pod \"aws-ebs-csi-driver-node-fj786\" (UID: \"a39e260a-4eff-4acb-9b7f-3ea1f42ff3c0\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-fj786" Jul 03 11:12:26.025366 ip-10-0-136-0 kubenswrapper[2579]: I0703 11:12:26.025343 2579 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"host-run-netns\" (UniqueName: \"kubernetes.io/host-path/45f92856-212a-4190-93ae-dfec96e1864e-host-run-netns\") pod \"ovnkube-node-dchtv\" (UID: \"45f92856-212a-4190-93ae-dfec96e1864e\") " pod="openshift-ovn-kubernetes/ovnkube-node-dchtv" Jul 03 11:12:26.025589 ip-10-0-136-0 kubenswrapper[2579]: I0703 11:12:26.025389 2579 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"systemd-units\" (UniqueName: \"kubernetes.io/host-path/45f92856-212a-4190-93ae-dfec96e1864e-systemd-units\") pod \"ovnkube-node-dchtv\" (UID: \"45f92856-212a-4190-93ae-dfec96e1864e\") " pod="openshift-ovn-kubernetes/ovnkube-node-dchtv" Jul 03 11:12:26.025589 ip-10-0-136-0 kubenswrapper[2579]: I0703 11:12:26.025413 2579 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"host-cni-bin\" (UniqueName: \"kubernetes.io/host-path/45f92856-212a-4190-93ae-dfec96e1864e-host-cni-bin\") pod \"ovnkube-node-dchtv\" (UID: \"45f92856-212a-4190-93ae-dfec96e1864e\") " pod="openshift-ovn-kubernetes/ovnkube-node-dchtv" Jul 03 11:12:26.025589 ip-10-0-136-0 kubenswrapper[2579]: I0703 11:12:26.025468 2579 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"lib-modules\" (UniqueName: \"kubernetes.io/host-path/99eb1d1a-c732-4365-9da7-ecab8ead5794-lib-modules\") pod \"tuned-dwp69\" (UID: \"99eb1d1a-c732-4365-9da7-ecab8ead5794\") " pod="openshift-cluster-node-tuning-operator/tuned-dwp69" Jul 03 11:12:26.025589 ip-10-0-136-0 kubenswrapper[2579]: I0703 11:12:26.025511 2579 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"run-systemd\" (UniqueName: \"kubernetes.io/host-path/45f92856-212a-4190-93ae-dfec96e1864e-run-systemd\") pod \"ovnkube-node-dchtv\" (UID: \"45f92856-212a-4190-93ae-dfec96e1864e\") " pod="openshift-ovn-kubernetes/ovnkube-node-dchtv" Jul 03 11:12:26.025782 ip-10-0-136-0 kubenswrapper[2579]: I0703 11:12:26.025589 2579 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"etc-modprobe-d\" (UniqueName: \"kubernetes.io/host-path/99eb1d1a-c732-4365-9da7-ecab8ead5794-etc-modprobe-d\") pod \"tuned-dwp69\" (UID: \"99eb1d1a-c732-4365-9da7-ecab8ead5794\") " pod="openshift-cluster-node-tuning-operator/tuned-dwp69" Jul 03 11:12:26.025782 ip-10-0-136-0 kubenswrapper[2579]: I0703 11:12:26.025638 2579 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"etc-sysconfig\" (UniqueName: \"kubernetes.io/host-path/99eb1d1a-c732-4365-9da7-ecab8ead5794-etc-sysconfig\") pod \"tuned-dwp69\" (UID: \"99eb1d1a-c732-4365-9da7-ecab8ead5794\") " pod="openshift-cluster-node-tuning-operator/tuned-dwp69" Jul 03 11:12:26.025782 ip-10-0-136-0 kubenswrapper[2579]: I0703 11:12:26.025728 2579 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"etc-sysctl-d\" (UniqueName: \"kubernetes.io/host-path/99eb1d1a-c732-4365-9da7-ecab8ead5794-etc-sysctl-d\") pod \"tuned-dwp69\" (UID: \"99eb1d1a-c732-4365-9da7-ecab8ead5794\") " pod="openshift-cluster-node-tuning-operator/tuned-dwp69" Jul 03 11:12:26.025782 ip-10-0-136-0 kubenswrapper[2579]: I0703 11:12:26.025776 2579 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"registration-dir\" (UniqueName: \"kubernetes.io/host-path/a39e260a-4eff-4acb-9b7f-3ea1f42ff3c0-registration-dir\") pod \"aws-ebs-csi-driver-node-fj786\" (UID: \"a39e260a-4eff-4acb-9b7f-3ea1f42ff3c0\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-fj786" Jul 03 11:12:26.025998 ip-10-0-136-0 kubenswrapper[2579]: I0703 11:12:26.025820 2579 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"host-kubelet\" (UniqueName: \"kubernetes.io/host-path/45f92856-212a-4190-93ae-dfec96e1864e-host-kubelet\") pod \"ovnkube-node-dchtv\" (UID: \"45f92856-212a-4190-93ae-dfec96e1864e\") " pod="openshift-ovn-kubernetes/ovnkube-node-dchtv" Jul 03 11:12:26.025998 ip-10-0-136-0 kubenswrapper[2579]: I0703 11:12:26.025858 2579 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"log-socket\" (UniqueName: \"kubernetes.io/host-path/45f92856-212a-4190-93ae-dfec96e1864e-log-socket\") pod \"ovnkube-node-dchtv\" (UID: \"45f92856-212a-4190-93ae-dfec96e1864e\") " pod="openshift-ovn-kubernetes/ovnkube-node-dchtv" Jul 03 11:12:26.025998 ip-10-0-136-0 kubenswrapper[2579]: I0703 11:12:26.025871 2579 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"sys-fs\" (UniqueName: \"kubernetes.io/host-path/a39e260a-4eff-4acb-9b7f-3ea1f42ff3c0-sys-fs\") pod \"aws-ebs-csi-driver-node-fj786\" (UID: \"a39e260a-4eff-4acb-9b7f-3ea1f42ff3c0\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-fj786" Jul 03 11:12:26.025998 ip-10-0-136-0 kubenswrapper[2579]: I0703 11:12:26.025933 2579 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"ovnkube-script-lib\" (UniqueName: \"kubernetes.io/configmap/45f92856-212a-4190-93ae-dfec96e1864e-ovnkube-script-lib\") pod \"ovnkube-node-dchtv\" (UID: \"45f92856-212a-4190-93ae-dfec96e1864e\") " pod="openshift-ovn-kubernetes/ovnkube-node-dchtv" Jul 03 11:12:26.026136 ip-10-0-136-0 kubenswrapper[2579]: I0703 11:12:26.025998 2579 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"etc-sysctl-conf\" (UniqueName: \"kubernetes.io/host-path/99eb1d1a-c732-4365-9da7-ecab8ead5794-etc-sysctl-conf\") pod \"tuned-dwp69\" (UID: \"99eb1d1a-c732-4365-9da7-ecab8ead5794\") " pod="openshift-cluster-node-tuning-operator/tuned-dwp69" Jul 03 11:12:26.026136 ip-10-0-136-0 kubenswrapper[2579]: I0703 11:12:26.026065 2579 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"run\" (UniqueName: \"kubernetes.io/host-path/99eb1d1a-c732-4365-9da7-ecab8ead5794-run\") pod \"tuned-dwp69\" (UID: \"99eb1d1a-c732-4365-9da7-ecab8ead5794\") " pod="openshift-cluster-node-tuning-operator/tuned-dwp69" Jul 03 11:12:26.026136 ip-10-0-136-0 kubenswrapper[2579]: I0703 11:12:26.026118 2579 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"socket-dir\" (UniqueName: \"kubernetes.io/host-path/a39e260a-4eff-4acb-9b7f-3ea1f42ff3c0-socket-dir\") pod \"aws-ebs-csi-driver-node-fj786\" (UID: \"a39e260a-4eff-4acb-9b7f-3ea1f42ff3c0\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-fj786" Jul 03 11:12:26.026545 ip-10-0-136-0 kubenswrapper[2579]: I0703 11:12:26.026437 2579 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"etc-tuned\" (UniqueName: \"kubernetes.io/empty-dir/99eb1d1a-c732-4365-9da7-ecab8ead5794-etc-tuned\") pod \"tuned-dwp69\" (UID: \"99eb1d1a-c732-4365-9da7-ecab8ead5794\") " pod="openshift-cluster-node-tuning-operator/tuned-dwp69" Jul 03 11:12:26.028547 ip-10-0-136-0 kubenswrapper[2579]: I0703 11:12:26.028523 2579 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"ovn-node-metrics-cert\" (UniqueName: \"kubernetes.io/secret/45f92856-212a-4190-93ae-dfec96e1864e-ovn-node-metrics-cert\") pod \"ovnkube-node-dchtv\" (UID: \"45f92856-212a-4190-93ae-dfec96e1864e\") " pod="openshift-ovn-kubernetes/ovnkube-node-dchtv" Jul 03 11:12:26.029053 ip-10-0-136-0 kubenswrapper[2579]: I0703 11:12:26.029028 2579 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"tmp\" (UniqueName: \"kubernetes.io/empty-dir/99eb1d1a-c732-4365-9da7-ecab8ead5794-tmp\") pod \"tuned-dwp69\" (UID: \"99eb1d1a-c732-4365-9da7-ecab8ead5794\") " pod="openshift-cluster-node-tuning-operator/tuned-dwp69" Jul 03 11:12:26.040252 ip-10-0-136-0 kubenswrapper[2579]: E0703 11:12:26.040155 2579 projected.go:291] Couldn't get configMap openshift-network-diagnostics/kube-root-ca.crt: object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered Jul 03 11:12:26.040252 ip-10-0-136-0 kubenswrapper[2579]: E0703 11:12:26.040213 2579 projected.go:291] Couldn't get configMap openshift-network-diagnostics/openshift-service-ca.crt: object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered Jul 03 11:12:26.040252 ip-10-0-136-0 kubenswrapper[2579]: E0703 11:12:26.040229 2579 projected.go:196] Error preparing data for projected volume kube-api-access-flqqx for pod openshift-network-diagnostics/network-check-target-xzlsc: [object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered, object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered] Jul 03 11:12:26.040521 ip-10-0-136-0 kubenswrapper[2579]: E0703 11:12:26.040320 2579 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/projected/2129ac30-1645-4555-b722-55963ffb8776-kube-api-access-flqqx podName:2129ac30-1645-4555-b722-55963ffb8776 nodeName:}" failed. No retries permitted until 2026-07-03 11:12:26.540297927 +0000 UTC m=+3.128602228 (durationBeforeRetry 500ms). Error: MountVolume.SetUp failed for volume "kube-api-access-flqqx" (UniqueName: "kubernetes.io/projected/2129ac30-1645-4555-b722-55963ffb8776-kube-api-access-flqqx") pod "network-check-target-xzlsc" (UID: "2129ac30-1645-4555-b722-55963ffb8776") : [object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered, object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered] Jul 03 11:12:26.046895 ip-10-0-136-0 kubenswrapper[2579]: I0703 11:12:26.046850 2579 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-whgx4\" (UniqueName: \"kubernetes.io/projected/45f92856-212a-4190-93ae-dfec96e1864e-kube-api-access-whgx4\") pod \"ovnkube-node-dchtv\" (UID: \"45f92856-212a-4190-93ae-dfec96e1864e\") " pod="openshift-ovn-kubernetes/ovnkube-node-dchtv" Jul 03 11:12:26.049467 ip-10-0-136-0 kubenswrapper[2579]: I0703 11:12:26.049415 2579 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-hdp57\" (UniqueName: \"kubernetes.io/projected/99eb1d1a-c732-4365-9da7-ecab8ead5794-kube-api-access-hdp57\") pod \"tuned-dwp69\" (UID: \"99eb1d1a-c732-4365-9da7-ecab8ead5794\") " pod="openshift-cluster-node-tuning-operator/tuned-dwp69" Jul 03 11:12:26.066511 ip-10-0-136-0 kubenswrapper[2579]: I0703 11:12:26.066472 2579 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-jhfsm\" (UniqueName: \"kubernetes.io/projected/34ca9c51-4743-4efc-9aa1-eb79a741f6a4-kube-api-access-jhfsm\") pod \"network-metrics-daemon-sqxsj\" (UID: \"34ca9c51-4743-4efc-9aa1-eb79a741f6a4\") " pod="openshift-multus/network-metrics-daemon-sqxsj" Jul 03 11:12:26.067719 ip-10-0-136-0 kubenswrapper[2579]: I0703 11:12:26.067684 2579 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-8hslg\" (UniqueName: \"kubernetes.io/projected/a39e260a-4eff-4acb-9b7f-3ea1f42ff3c0-kube-api-access-8hslg\") pod \"aws-ebs-csi-driver-node-fj786\" (UID: \"a39e260a-4eff-4acb-9b7f-3ea1f42ff3c0\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-fj786" Jul 03 11:12:26.116933 ip-10-0-136-0 kubenswrapper[2579]: I0703 11:12:26.116890 2579 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-dns/node-resolver-hnmtl" Jul 03 11:12:26.127325 ip-10-0-136-0 kubenswrapper[2579]: I0703 11:12:26.127292 2579 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/multus-cszpw" Jul 03 11:12:26.137615 ip-10-0-136-0 kubenswrapper[2579]: I0703 11:12:26.137577 2579 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/konnectivity-agent-5lz4n" Jul 03 11:12:26.153336 ip-10-0-136-0 kubenswrapper[2579]: I0703 11:12:26.153299 2579 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-operator/iptables-alerter-l8lpk" Jul 03 11:12:26.161853 ip-10-0-136-0 kubenswrapper[2579]: I0703 11:12:26.161820 2579 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-image-registry/node-ca-9ff9g" Jul 03 11:12:26.169128 ip-10-0-136-0 kubenswrapper[2579]: I0703 11:12:26.169092 2579 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/multus-additional-cni-plugins-52n9r" Jul 03 11:12:26.176998 ip-10-0-136-0 kubenswrapper[2579]: I0703 11:12:26.176951 2579 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ovn-kubernetes/ovnkube-node-dchtv" Jul 03 11:12:26.182233 ip-10-0-136-0 kubenswrapper[2579]: I0703 11:12:26.182202 2579 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-fj786" Jul 03 11:12:26.189445 ip-10-0-136-0 kubenswrapper[2579]: I0703 11:12:26.189406 2579 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-cluster-node-tuning-operator/tuned-dwp69" Jul 03 11:12:26.237355 ip-10-0-136-0 kubenswrapper[2579]: I0703 11:12:26.237325 2579 reflector.go:439] "Caches populated" type="*v1.CSIDriver" reflector="k8s.io/client-go/informers/factory.go:160" Jul 03 11:12:26.527723 ip-10-0-136-0 kubenswrapper[2579]: I0703 11:12:26.527676 2579 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/34ca9c51-4743-4efc-9aa1-eb79a741f6a4-metrics-certs\") pod \"network-metrics-daemon-sqxsj\" (UID: \"34ca9c51-4743-4efc-9aa1-eb79a741f6a4\") " pod="openshift-multus/network-metrics-daemon-sqxsj" Jul 03 11:12:26.527913 ip-10-0-136-0 kubenswrapper[2579]: I0703 11:12:26.527732 2579 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"original-pull-secret\" (UniqueName: \"kubernetes.io/secret/7813f158-0f67-411e-84d9-a2a1c51eea61-original-pull-secret\") pod \"global-pull-secret-syncer-9pqm8\" (UID: \"7813f158-0f67-411e-84d9-a2a1c51eea61\") " pod="kube-system/global-pull-secret-syncer-9pqm8" Jul 03 11:12:26.527913 ip-10-0-136-0 kubenswrapper[2579]: E0703 11:12:26.527847 2579 secret.go:189] Couldn't get secret openshift-multus/metrics-daemon-secret: object "openshift-multus"/"metrics-daemon-secret" not registered Jul 03 11:12:26.527913 ip-10-0-136-0 kubenswrapper[2579]: E0703 11:12:26.527885 2579 secret.go:189] Couldn't get secret kube-system/original-pull-secret: object "kube-system"/"original-pull-secret" not registered Jul 03 11:12:26.528091 ip-10-0-136-0 kubenswrapper[2579]: E0703 11:12:26.527935 2579 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/34ca9c51-4743-4efc-9aa1-eb79a741f6a4-metrics-certs podName:34ca9c51-4743-4efc-9aa1-eb79a741f6a4 nodeName:}" failed. No retries permitted until 2026-07-03 11:12:27.527911827 +0000 UTC m=+4.116216140 (durationBeforeRetry 1s). Error: MountVolume.SetUp failed for volume "metrics-certs" (UniqueName: "kubernetes.io/secret/34ca9c51-4743-4efc-9aa1-eb79a741f6a4-metrics-certs") pod "network-metrics-daemon-sqxsj" (UID: "34ca9c51-4743-4efc-9aa1-eb79a741f6a4") : object "openshift-multus"/"metrics-daemon-secret" not registered Jul 03 11:12:26.528091 ip-10-0-136-0 kubenswrapper[2579]: E0703 11:12:26.527958 2579 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/7813f158-0f67-411e-84d9-a2a1c51eea61-original-pull-secret podName:7813f158-0f67-411e-84d9-a2a1c51eea61 nodeName:}" failed. No retries permitted until 2026-07-03 11:12:27.527948816 +0000 UTC m=+4.116253118 (durationBeforeRetry 1s). Error: MountVolume.SetUp failed for volume "original-pull-secret" (UniqueName: "kubernetes.io/secret/7813f158-0f67-411e-84d9-a2a1c51eea61-original-pull-secret") pod "global-pull-secret-syncer-9pqm8" (UID: "7813f158-0f67-411e-84d9-a2a1c51eea61") : object "kube-system"/"original-pull-secret" not registered Jul 03 11:12:26.628870 ip-10-0-136-0 kubenswrapper[2579]: I0703 11:12:26.628779 2579 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-flqqx\" (UniqueName: \"kubernetes.io/projected/2129ac30-1645-4555-b722-55963ffb8776-kube-api-access-flqqx\") pod \"network-check-target-xzlsc\" (UID: \"2129ac30-1645-4555-b722-55963ffb8776\") " pod="openshift-network-diagnostics/network-check-target-xzlsc" Jul 03 11:12:26.629060 ip-10-0-136-0 kubenswrapper[2579]: E0703 11:12:26.628990 2579 projected.go:291] Couldn't get configMap openshift-network-diagnostics/kube-root-ca.crt: object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered Jul 03 11:12:26.629060 ip-10-0-136-0 kubenswrapper[2579]: E0703 11:12:26.629018 2579 projected.go:291] Couldn't get configMap openshift-network-diagnostics/openshift-service-ca.crt: object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered Jul 03 11:12:26.629060 ip-10-0-136-0 kubenswrapper[2579]: E0703 11:12:26.629035 2579 projected.go:196] Error preparing data for projected volume kube-api-access-flqqx for pod openshift-network-diagnostics/network-check-target-xzlsc: [object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered, object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered] Jul 03 11:12:26.629247 ip-10-0-136-0 kubenswrapper[2579]: E0703 11:12:26.629131 2579 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/projected/2129ac30-1645-4555-b722-55963ffb8776-kube-api-access-flqqx podName:2129ac30-1645-4555-b722-55963ffb8776 nodeName:}" failed. No retries permitted until 2026-07-03 11:12:27.62910842 +0000 UTC m=+4.217412729 (durationBeforeRetry 1s). Error: MountVolume.SetUp failed for volume "kube-api-access-flqqx" (UniqueName: "kubernetes.io/projected/2129ac30-1645-4555-b722-55963ffb8776-kube-api-access-flqqx") pod "network-check-target-xzlsc" (UID: "2129ac30-1645-4555-b722-55963ffb8776") : [object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered, object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered] Jul 03 11:12:26.905592 ip-10-0-136-0 kubenswrapper[2579]: I0703 11:12:26.905462 2579 certificate_manager.go:715] "Certificate rotation deadline determined" logger="kubernetes.io/kubelet-serving" expiration="2028-07-02 11:07:25 +0000 UTC" deadline="2028-03-30 04:44:32.870465962 +0000 UTC" Jul 03 11:12:26.905592 ip-10-0-136-0 kubenswrapper[2579]: I0703 11:12:26.905524 2579 certificate_manager.go:431] "Waiting for next certificate rotation" logger="kubernetes.io/kubelet-serving" sleep="15257h32m5.964946664s" Jul 03 11:12:26.986459 ip-10-0-136-0 kubenswrapper[2579]: I0703 11:12:26.986415 2579 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-sqxsj" Jul 03 11:12:26.986661 ip-10-0-136-0 kubenswrapper[2579]: I0703 11:12:26.986440 2579 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-9pqm8" Jul 03 11:12:26.986661 ip-10-0-136-0 kubenswrapper[2579]: E0703 11:12:26.986585 2579 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-sqxsj" podUID="34ca9c51-4743-4efc-9aa1-eb79a741f6a4" Jul 03 11:12:26.986743 ip-10-0-136-0 kubenswrapper[2579]: E0703 11:12:26.986672 2579 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-9pqm8" podUID="7813f158-0f67-411e-84d9-a2a1c51eea61" Jul 03 11:12:27.157385 ip-10-0-136-0 kubenswrapper[2579]: W0703 11:12:27.157343 2579 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-poda39e260a_4eff_4acb_9b7f_3ea1f42ff3c0.slice/crio-df6a3c53568ee4a8f8ee971713526dc17944f4317f56a2f2ce3eecdd1227c385 WatchSource:0}: Error finding container df6a3c53568ee4a8f8ee971713526dc17944f4317f56a2f2ce3eecdd1227c385: Status 404 returned error can't find the container with id df6a3c53568ee4a8f8ee971713526dc17944f4317f56a2f2ce3eecdd1227c385 Jul 03 11:12:27.160523 ip-10-0-136-0 kubenswrapper[2579]: W0703 11:12:27.160481 2579 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podf5fa8ffd_d2fc_4474_ad0d_a7af28edf0e7.slice/crio-477aa67d3d5e09b7d01b565b53acae9914e29cbbccf6405f8bbeab50407383cc WatchSource:0}: Error finding container 477aa67d3d5e09b7d01b565b53acae9914e29cbbccf6405f8bbeab50407383cc: Status 404 returned error can't find the container with id 477aa67d3d5e09b7d01b565b53acae9914e29cbbccf6405f8bbeab50407383cc Jul 03 11:12:27.175573 ip-10-0-136-0 kubenswrapper[2579]: W0703 11:12:27.175416 2579 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod99eb1d1a_c732_4365_9da7_ecab8ead5794.slice/crio-26ec3accb484ffdcd108299270ecb834a3e62badc2244844702ae9f2d8ab1b7f WatchSource:0}: Error finding container 26ec3accb484ffdcd108299270ecb834a3e62badc2244844702ae9f2d8ab1b7f: Status 404 returned error can't find the container with id 26ec3accb484ffdcd108299270ecb834a3e62badc2244844702ae9f2d8ab1b7f Jul 03 11:12:27.535100 ip-10-0-136-0 kubenswrapper[2579]: I0703 11:12:27.535058 2579 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/34ca9c51-4743-4efc-9aa1-eb79a741f6a4-metrics-certs\") pod \"network-metrics-daemon-sqxsj\" (UID: \"34ca9c51-4743-4efc-9aa1-eb79a741f6a4\") " pod="openshift-multus/network-metrics-daemon-sqxsj" Jul 03 11:12:27.535305 ip-10-0-136-0 kubenswrapper[2579]: I0703 11:12:27.535109 2579 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"original-pull-secret\" (UniqueName: \"kubernetes.io/secret/7813f158-0f67-411e-84d9-a2a1c51eea61-original-pull-secret\") pod \"global-pull-secret-syncer-9pqm8\" (UID: \"7813f158-0f67-411e-84d9-a2a1c51eea61\") " pod="kube-system/global-pull-secret-syncer-9pqm8" Jul 03 11:12:27.535305 ip-10-0-136-0 kubenswrapper[2579]: E0703 11:12:27.535254 2579 secret.go:189] Couldn't get secret openshift-multus/metrics-daemon-secret: object "openshift-multus"/"metrics-daemon-secret" not registered Jul 03 11:12:27.535305 ip-10-0-136-0 kubenswrapper[2579]: E0703 11:12:27.535298 2579 secret.go:189] Couldn't get secret kube-system/original-pull-secret: object "kube-system"/"original-pull-secret" not registered Jul 03 11:12:27.535430 ip-10-0-136-0 kubenswrapper[2579]: E0703 11:12:27.535352 2579 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/34ca9c51-4743-4efc-9aa1-eb79a741f6a4-metrics-certs podName:34ca9c51-4743-4efc-9aa1-eb79a741f6a4 nodeName:}" failed. No retries permitted until 2026-07-03 11:12:29.535328429 +0000 UTC m=+6.123632727 (durationBeforeRetry 2s). Error: MountVolume.SetUp failed for volume "metrics-certs" (UniqueName: "kubernetes.io/secret/34ca9c51-4743-4efc-9aa1-eb79a741f6a4-metrics-certs") pod "network-metrics-daemon-sqxsj" (UID: "34ca9c51-4743-4efc-9aa1-eb79a741f6a4") : object "openshift-multus"/"metrics-daemon-secret" not registered Jul 03 11:12:27.535430 ip-10-0-136-0 kubenswrapper[2579]: E0703 11:12:27.535371 2579 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/7813f158-0f67-411e-84d9-a2a1c51eea61-original-pull-secret podName:7813f158-0f67-411e-84d9-a2a1c51eea61 nodeName:}" failed. No retries permitted until 2026-07-03 11:12:29.535362506 +0000 UTC m=+6.123666804 (durationBeforeRetry 2s). Error: MountVolume.SetUp failed for volume "original-pull-secret" (UniqueName: "kubernetes.io/secret/7813f158-0f67-411e-84d9-a2a1c51eea61-original-pull-secret") pod "global-pull-secret-syncer-9pqm8" (UID: "7813f158-0f67-411e-84d9-a2a1c51eea61") : object "kube-system"/"original-pull-secret" not registered Jul 03 11:12:27.636110 ip-10-0-136-0 kubenswrapper[2579]: I0703 11:12:27.636064 2579 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-flqqx\" (UniqueName: \"kubernetes.io/projected/2129ac30-1645-4555-b722-55963ffb8776-kube-api-access-flqqx\") pod \"network-check-target-xzlsc\" (UID: \"2129ac30-1645-4555-b722-55963ffb8776\") " pod="openshift-network-diagnostics/network-check-target-xzlsc" Jul 03 11:12:27.636326 ip-10-0-136-0 kubenswrapper[2579]: E0703 11:12:27.636310 2579 projected.go:291] Couldn't get configMap openshift-network-diagnostics/kube-root-ca.crt: object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered Jul 03 11:12:27.636393 ip-10-0-136-0 kubenswrapper[2579]: E0703 11:12:27.636340 2579 projected.go:291] Couldn't get configMap openshift-network-diagnostics/openshift-service-ca.crt: object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered Jul 03 11:12:27.636393 ip-10-0-136-0 kubenswrapper[2579]: E0703 11:12:27.636354 2579 projected.go:196] Error preparing data for projected volume kube-api-access-flqqx for pod openshift-network-diagnostics/network-check-target-xzlsc: [object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered, object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered] Jul 03 11:12:27.636505 ip-10-0-136-0 kubenswrapper[2579]: E0703 11:12:27.636425 2579 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/projected/2129ac30-1645-4555-b722-55963ffb8776-kube-api-access-flqqx podName:2129ac30-1645-4555-b722-55963ffb8776 nodeName:}" failed. No retries permitted until 2026-07-03 11:12:29.636403288 +0000 UTC m=+6.224707601 (durationBeforeRetry 2s). Error: MountVolume.SetUp failed for volume "kube-api-access-flqqx" (UniqueName: "kubernetes.io/projected/2129ac30-1645-4555-b722-55963ffb8776-kube-api-access-flqqx") pod "network-check-target-xzlsc" (UID: "2129ac30-1645-4555-b722-55963ffb8776") : [object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered, object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered] Jul 03 11:12:27.906969 ip-10-0-136-0 kubenswrapper[2579]: I0703 11:12:27.906833 2579 certificate_manager.go:715] "Certificate rotation deadline determined" logger="kubernetes.io/kubelet-serving" expiration="2028-07-02 11:07:25 +0000 UTC" deadline="2027-12-09 14:19:01.457306124 +0000 UTC" Jul 03 11:12:27.906969 ip-10-0-136-0 kubenswrapper[2579]: I0703 11:12:27.906908 2579 certificate_manager.go:431] "Waiting for next certificate rotation" logger="kubernetes.io/kubelet-serving" sleep="12579h6m33.550402165s" Jul 03 11:12:27.986925 ip-10-0-136-0 kubenswrapper[2579]: I0703 11:12:27.986851 2579 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-xzlsc" Jul 03 11:12:27.987125 ip-10-0-136-0 kubenswrapper[2579]: E0703 11:12:27.987002 2579 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-xzlsc" podUID="2129ac30-1645-4555-b722-55963ffb8776" Jul 03 11:12:28.022385 ip-10-0-136-0 kubenswrapper[2579]: I0703 11:12:28.022307 2579 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-multus/multus-cszpw" event={"ID":"ad5f7b2c-eac2-4ebb-b1d8-62782a621d85","Type":"ContainerStarted","Data":"fbdfa83feb9735a541ce6461463e06197be9d3b50f544bedc104d3664a155d52"} Jul 03 11:12:28.033545 ip-10-0-136-0 kubenswrapper[2579]: I0703 11:12:28.033450 2579 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-image-registry/node-ca-9ff9g" event={"ID":"f5fa8ffd-d2fc-4474-ad0d-a7af28edf0e7","Type":"ContainerStarted","Data":"477aa67d3d5e09b7d01b565b53acae9914e29cbbccf6405f8bbeab50407383cc"} Jul 03 11:12:28.053574 ip-10-0-136-0 kubenswrapper[2579]: I0703 11:12:28.053516 2579 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-fj786" event={"ID":"a39e260a-4eff-4acb-9b7f-3ea1f42ff3c0","Type":"ContainerStarted","Data":"df6a3c53568ee4a8f8ee971713526dc17944f4317f56a2f2ce3eecdd1227c385"} Jul 03 11:12:28.065862 ip-10-0-136-0 kubenswrapper[2579]: I0703 11:12:28.065777 2579 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kube-system/kube-apiserver-proxy-ip-10-0-136-0.ec2.internal" event={"ID":"e9fe0b17840ecce76a8985c0f09920db","Type":"ContainerStarted","Data":"6fa68dbe662f010c09c106c195d7c317c2051003c0cf3f0c697173bbcab43a1a"} Jul 03 11:12:28.089333 ip-10-0-136-0 kubenswrapper[2579]: I0703 11:12:28.089284 2579 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ovn-kubernetes/ovnkube-node-dchtv" event={"ID":"45f92856-212a-4190-93ae-dfec96e1864e","Type":"ContainerStarted","Data":"4d4276b59c1267bac308c3995a7691e2adf308187516c3f7f9de897b8f7e6b8d"} Jul 03 11:12:28.097240 ip-10-0-136-0 kubenswrapper[2579]: I0703 11:12:28.097117 2579 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-network-operator/iptables-alerter-l8lpk" event={"ID":"a8bc35c6-22ff-4c87-802d-e44999086c38","Type":"ContainerStarted","Data":"6e8036e1825495548ebc9114deb3aeb34aed5c400d3eb9e5b7bff4fa5fdef78e"} Jul 03 11:12:28.105741 ip-10-0-136-0 kubenswrapper[2579]: I0703 11:12:28.105701 2579 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-dns/node-resolver-hnmtl" event={"ID":"9a6053f3-4885-4e53-8b1c-93150a7546c4","Type":"ContainerStarted","Data":"e3a7672d0097d9d9c90208cc2046ee85453ecd145b5f13eb52c316f3db27be35"} Jul 03 11:12:28.125456 ip-10-0-136-0 kubenswrapper[2579]: I0703 11:12:28.125340 2579 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kube-system/konnectivity-agent-5lz4n" event={"ID":"945b7f84-a41f-413e-a7b6-87818468360f","Type":"ContainerStarted","Data":"4087d672721574b1cbd7783606b1d70c113c3217c7ce616206e3de3fadc24785"} Jul 03 11:12:28.146328 ip-10-0-136-0 kubenswrapper[2579]: I0703 11:12:28.146119 2579 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-multus/multus-additional-cni-plugins-52n9r" event={"ID":"e9fc3c32-a741-4377-970b-6cb08e5b4ac9","Type":"ContainerStarted","Data":"02f0956e43e76d93ca5a533cd8a6a798b04eaaf4abdcee0031127d21d18598c6"} Jul 03 11:12:28.164072 ip-10-0-136-0 kubenswrapper[2579]: I0703 11:12:28.162756 2579 generic.go:356] "Generic (PLEG): container finished" podID="e77ab5b2fa375ad8c98ab61ef281b548" containerID="5bf79dfa9e993c2cd882194520abf94472c3d6f3ce8e724c6c20ad47530fe764" exitCode=0 Jul 03 11:12:28.164072 ip-10-0-136-0 kubenswrapper[2579]: I0703 11:12:28.163104 2579 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-136-0.ec2.internal" event={"ID":"e77ab5b2fa375ad8c98ab61ef281b548","Type":"ContainerDied","Data":"5bf79dfa9e993c2cd882194520abf94472c3d6f3ce8e724c6c20ad47530fe764"} Jul 03 11:12:28.169043 ip-10-0-136-0 kubenswrapper[2579]: I0703 11:12:28.168942 2579 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-cluster-node-tuning-operator/tuned-dwp69" event={"ID":"99eb1d1a-c732-4365-9da7-ecab8ead5794","Type":"ContainerStarted","Data":"26ec3accb484ffdcd108299270ecb834a3e62badc2244844702ae9f2d8ab1b7f"} Jul 03 11:12:28.187544 ip-10-0-136-0 kubenswrapper[2579]: I0703 11:12:28.186706 2579 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="kube-system/kube-apiserver-proxy-ip-10-0-136-0.ec2.internal" podStartSLOduration=3.186657979 podStartE2EDuration="3.186657979s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=true podCreationTimestamp="2026-07-03 11:12:25 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-03 11:12:28.082164084 +0000 UTC m=+4.670468405" watchObservedRunningTime="2026-07-03 11:12:28.186657979 +0000 UTC m=+4.774962300" Jul 03 11:12:28.986037 ip-10-0-136-0 kubenswrapper[2579]: I0703 11:12:28.985999 2579 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-sqxsj" Jul 03 11:12:28.986634 ip-10-0-136-0 kubenswrapper[2579]: E0703 11:12:28.986278 2579 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-sqxsj" podUID="34ca9c51-4743-4efc-9aa1-eb79a741f6a4" Jul 03 11:12:28.986782 ip-10-0-136-0 kubenswrapper[2579]: I0703 11:12:28.986736 2579 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-9pqm8" Jul 03 11:12:28.986925 ip-10-0-136-0 kubenswrapper[2579]: E0703 11:12:28.986902 2579 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-9pqm8" podUID="7813f158-0f67-411e-84d9-a2a1c51eea61" Jul 03 11:12:29.191323 ip-10-0-136-0 kubenswrapper[2579]: I0703 11:12:29.191259 2579 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-136-0.ec2.internal" event={"ID":"e77ab5b2fa375ad8c98ab61ef281b548","Type":"ContainerStarted","Data":"c3941013f76c43b8db939475bf70350ca4f2f2d4f361f276032f44bad8461876"} Jul 03 11:12:29.208135 ip-10-0-136-0 kubenswrapper[2579]: I0703 11:12:29.207155 2579 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-136-0.ec2.internal" podStartSLOduration=4.207128192 podStartE2EDuration="4.207128192s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=true podCreationTimestamp="2026-07-03 11:12:25 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-03 11:12:29.206714504 +0000 UTC m=+5.795018824" watchObservedRunningTime="2026-07-03 11:12:29.207128192 +0000 UTC m=+5.795432517" Jul 03 11:12:29.554025 ip-10-0-136-0 kubenswrapper[2579]: I0703 11:12:29.553981 2579 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/34ca9c51-4743-4efc-9aa1-eb79a741f6a4-metrics-certs\") pod \"network-metrics-daemon-sqxsj\" (UID: \"34ca9c51-4743-4efc-9aa1-eb79a741f6a4\") " pod="openshift-multus/network-metrics-daemon-sqxsj" Jul 03 11:12:29.554025 ip-10-0-136-0 kubenswrapper[2579]: I0703 11:12:29.554021 2579 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"original-pull-secret\" (UniqueName: \"kubernetes.io/secret/7813f158-0f67-411e-84d9-a2a1c51eea61-original-pull-secret\") pod \"global-pull-secret-syncer-9pqm8\" (UID: \"7813f158-0f67-411e-84d9-a2a1c51eea61\") " pod="kube-system/global-pull-secret-syncer-9pqm8" Jul 03 11:12:29.554341 ip-10-0-136-0 kubenswrapper[2579]: E0703 11:12:29.554132 2579 secret.go:189] Couldn't get secret kube-system/original-pull-secret: object "kube-system"/"original-pull-secret" not registered Jul 03 11:12:29.554341 ip-10-0-136-0 kubenswrapper[2579]: E0703 11:12:29.554207 2579 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/7813f158-0f67-411e-84d9-a2a1c51eea61-original-pull-secret podName:7813f158-0f67-411e-84d9-a2a1c51eea61 nodeName:}" failed. No retries permitted until 2026-07-03 11:12:33.554173402 +0000 UTC m=+10.142477704 (durationBeforeRetry 4s). Error: MountVolume.SetUp failed for volume "original-pull-secret" (UniqueName: "kubernetes.io/secret/7813f158-0f67-411e-84d9-a2a1c51eea61-original-pull-secret") pod "global-pull-secret-syncer-9pqm8" (UID: "7813f158-0f67-411e-84d9-a2a1c51eea61") : object "kube-system"/"original-pull-secret" not registered Jul 03 11:12:29.554341 ip-10-0-136-0 kubenswrapper[2579]: E0703 11:12:29.554273 2579 secret.go:189] Couldn't get secret openshift-multus/metrics-daemon-secret: object "openshift-multus"/"metrics-daemon-secret" not registered Jul 03 11:12:29.554341 ip-10-0-136-0 kubenswrapper[2579]: E0703 11:12:29.554307 2579 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/34ca9c51-4743-4efc-9aa1-eb79a741f6a4-metrics-certs podName:34ca9c51-4743-4efc-9aa1-eb79a741f6a4 nodeName:}" failed. No retries permitted until 2026-07-03 11:12:33.554295638 +0000 UTC m=+10.142599946 (durationBeforeRetry 4s). Error: MountVolume.SetUp failed for volume "metrics-certs" (UniqueName: "kubernetes.io/secret/34ca9c51-4743-4efc-9aa1-eb79a741f6a4-metrics-certs") pod "network-metrics-daemon-sqxsj" (UID: "34ca9c51-4743-4efc-9aa1-eb79a741f6a4") : object "openshift-multus"/"metrics-daemon-secret" not registered Jul 03 11:12:29.655307 ip-10-0-136-0 kubenswrapper[2579]: I0703 11:12:29.655265 2579 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-flqqx\" (UniqueName: \"kubernetes.io/projected/2129ac30-1645-4555-b722-55963ffb8776-kube-api-access-flqqx\") pod \"network-check-target-xzlsc\" (UID: \"2129ac30-1645-4555-b722-55963ffb8776\") " pod="openshift-network-diagnostics/network-check-target-xzlsc" Jul 03 11:12:29.655538 ip-10-0-136-0 kubenswrapper[2579]: E0703 11:12:29.655513 2579 projected.go:291] Couldn't get configMap openshift-network-diagnostics/kube-root-ca.crt: object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered Jul 03 11:12:29.655598 ip-10-0-136-0 kubenswrapper[2579]: E0703 11:12:29.655550 2579 projected.go:291] Couldn't get configMap openshift-network-diagnostics/openshift-service-ca.crt: object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered Jul 03 11:12:29.655598 ip-10-0-136-0 kubenswrapper[2579]: E0703 11:12:29.655564 2579 projected.go:196] Error preparing data for projected volume kube-api-access-flqqx for pod openshift-network-diagnostics/network-check-target-xzlsc: [object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered, object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered] Jul 03 11:12:29.655665 ip-10-0-136-0 kubenswrapper[2579]: E0703 11:12:29.655624 2579 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/projected/2129ac30-1645-4555-b722-55963ffb8776-kube-api-access-flqqx podName:2129ac30-1645-4555-b722-55963ffb8776 nodeName:}" failed. No retries permitted until 2026-07-03 11:12:33.655605235 +0000 UTC m=+10.243909535 (durationBeforeRetry 4s). Error: MountVolume.SetUp failed for volume "kube-api-access-flqqx" (UniqueName: "kubernetes.io/projected/2129ac30-1645-4555-b722-55963ffb8776-kube-api-access-flqqx") pod "network-check-target-xzlsc" (UID: "2129ac30-1645-4555-b722-55963ffb8776") : [object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered, object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered] Jul 03 11:12:29.985853 ip-10-0-136-0 kubenswrapper[2579]: I0703 11:12:29.985540 2579 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-xzlsc" Jul 03 11:12:29.985853 ip-10-0-136-0 kubenswrapper[2579]: E0703 11:12:29.985744 2579 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-xzlsc" podUID="2129ac30-1645-4555-b722-55963ffb8776" Jul 03 11:12:30.985925 ip-10-0-136-0 kubenswrapper[2579]: I0703 11:12:30.985880 2579 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-sqxsj" Jul 03 11:12:30.986346 ip-10-0-136-0 kubenswrapper[2579]: E0703 11:12:30.986041 2579 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-sqxsj" podUID="34ca9c51-4743-4efc-9aa1-eb79a741f6a4" Jul 03 11:12:30.986569 ip-10-0-136-0 kubenswrapper[2579]: I0703 11:12:30.986535 2579 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-9pqm8" Jul 03 11:12:30.986687 ip-10-0-136-0 kubenswrapper[2579]: E0703 11:12:30.986663 2579 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-9pqm8" podUID="7813f158-0f67-411e-84d9-a2a1c51eea61" Jul 03 11:12:31.987433 ip-10-0-136-0 kubenswrapper[2579]: I0703 11:12:31.987351 2579 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-xzlsc" Jul 03 11:12:31.987830 ip-10-0-136-0 kubenswrapper[2579]: E0703 11:12:31.987507 2579 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-xzlsc" podUID="2129ac30-1645-4555-b722-55963ffb8776" Jul 03 11:12:32.986909 ip-10-0-136-0 kubenswrapper[2579]: I0703 11:12:32.986861 2579 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-sqxsj" Jul 03 11:12:32.987087 ip-10-0-136-0 kubenswrapper[2579]: E0703 11:12:32.987047 2579 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-sqxsj" podUID="34ca9c51-4743-4efc-9aa1-eb79a741f6a4" Jul 03 11:12:32.987157 ip-10-0-136-0 kubenswrapper[2579]: I0703 11:12:32.987141 2579 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-9pqm8" Jul 03 11:12:32.987300 ip-10-0-136-0 kubenswrapper[2579]: E0703 11:12:32.987276 2579 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-9pqm8" podUID="7813f158-0f67-411e-84d9-a2a1c51eea61" Jul 03 11:12:33.593499 ip-10-0-136-0 kubenswrapper[2579]: I0703 11:12:33.593458 2579 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/34ca9c51-4743-4efc-9aa1-eb79a741f6a4-metrics-certs\") pod \"network-metrics-daemon-sqxsj\" (UID: \"34ca9c51-4743-4efc-9aa1-eb79a741f6a4\") " pod="openshift-multus/network-metrics-daemon-sqxsj" Jul 03 11:12:33.593969 ip-10-0-136-0 kubenswrapper[2579]: I0703 11:12:33.593516 2579 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"original-pull-secret\" (UniqueName: \"kubernetes.io/secret/7813f158-0f67-411e-84d9-a2a1c51eea61-original-pull-secret\") pod \"global-pull-secret-syncer-9pqm8\" (UID: \"7813f158-0f67-411e-84d9-a2a1c51eea61\") " pod="kube-system/global-pull-secret-syncer-9pqm8" Jul 03 11:12:33.593969 ip-10-0-136-0 kubenswrapper[2579]: E0703 11:12:33.593665 2579 secret.go:189] Couldn't get secret kube-system/original-pull-secret: object "kube-system"/"original-pull-secret" not registered Jul 03 11:12:33.593969 ip-10-0-136-0 kubenswrapper[2579]: E0703 11:12:33.593717 2579 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/7813f158-0f67-411e-84d9-a2a1c51eea61-original-pull-secret podName:7813f158-0f67-411e-84d9-a2a1c51eea61 nodeName:}" failed. No retries permitted until 2026-07-03 11:12:41.593704639 +0000 UTC m=+18.182008950 (durationBeforeRetry 8s). Error: MountVolume.SetUp failed for volume "original-pull-secret" (UniqueName: "kubernetes.io/secret/7813f158-0f67-411e-84d9-a2a1c51eea61-original-pull-secret") pod "global-pull-secret-syncer-9pqm8" (UID: "7813f158-0f67-411e-84d9-a2a1c51eea61") : object "kube-system"/"original-pull-secret" not registered Jul 03 11:12:33.594137 ip-10-0-136-0 kubenswrapper[2579]: E0703 11:12:33.594120 2579 secret.go:189] Couldn't get secret openshift-multus/metrics-daemon-secret: object "openshift-multus"/"metrics-daemon-secret" not registered Jul 03 11:12:33.594210 ip-10-0-136-0 kubenswrapper[2579]: E0703 11:12:33.594178 2579 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/34ca9c51-4743-4efc-9aa1-eb79a741f6a4-metrics-certs podName:34ca9c51-4743-4efc-9aa1-eb79a741f6a4 nodeName:}" failed. No retries permitted until 2026-07-03 11:12:41.594158976 +0000 UTC m=+18.182463290 (durationBeforeRetry 8s). Error: MountVolume.SetUp failed for volume "metrics-certs" (UniqueName: "kubernetes.io/secret/34ca9c51-4743-4efc-9aa1-eb79a741f6a4-metrics-certs") pod "network-metrics-daemon-sqxsj" (UID: "34ca9c51-4743-4efc-9aa1-eb79a741f6a4") : object "openshift-multus"/"metrics-daemon-secret" not registered Jul 03 11:12:33.694745 ip-10-0-136-0 kubenswrapper[2579]: I0703 11:12:33.694697 2579 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-flqqx\" (UniqueName: \"kubernetes.io/projected/2129ac30-1645-4555-b722-55963ffb8776-kube-api-access-flqqx\") pod \"network-check-target-xzlsc\" (UID: \"2129ac30-1645-4555-b722-55963ffb8776\") " pod="openshift-network-diagnostics/network-check-target-xzlsc" Jul 03 11:12:33.694939 ip-10-0-136-0 kubenswrapper[2579]: E0703 11:12:33.694892 2579 projected.go:291] Couldn't get configMap openshift-network-diagnostics/kube-root-ca.crt: object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered Jul 03 11:12:33.694939 ip-10-0-136-0 kubenswrapper[2579]: E0703 11:12:33.694911 2579 projected.go:291] Couldn't get configMap openshift-network-diagnostics/openshift-service-ca.crt: object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered Jul 03 11:12:33.694939 ip-10-0-136-0 kubenswrapper[2579]: E0703 11:12:33.694927 2579 projected.go:196] Error preparing data for projected volume kube-api-access-flqqx for pod openshift-network-diagnostics/network-check-target-xzlsc: [object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered, object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered] Jul 03 11:12:33.695081 ip-10-0-136-0 kubenswrapper[2579]: E0703 11:12:33.694989 2579 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/projected/2129ac30-1645-4555-b722-55963ffb8776-kube-api-access-flqqx podName:2129ac30-1645-4555-b722-55963ffb8776 nodeName:}" failed. No retries permitted until 2026-07-03 11:12:41.694969218 +0000 UTC m=+18.283273534 (durationBeforeRetry 8s). Error: MountVolume.SetUp failed for volume "kube-api-access-flqqx" (UniqueName: "kubernetes.io/projected/2129ac30-1645-4555-b722-55963ffb8776-kube-api-access-flqqx") pod "network-check-target-xzlsc" (UID: "2129ac30-1645-4555-b722-55963ffb8776") : [object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered, object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered] Jul 03 11:12:33.987628 ip-10-0-136-0 kubenswrapper[2579]: I0703 11:12:33.987531 2579 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-xzlsc" Jul 03 11:12:33.987793 ip-10-0-136-0 kubenswrapper[2579]: E0703 11:12:33.987657 2579 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-xzlsc" podUID="2129ac30-1645-4555-b722-55963ffb8776" Jul 03 11:12:34.985726 ip-10-0-136-0 kubenswrapper[2579]: I0703 11:12:34.985682 2579 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-sqxsj" Jul 03 11:12:34.986240 ip-10-0-136-0 kubenswrapper[2579]: E0703 11:12:34.985838 2579 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-sqxsj" podUID="34ca9c51-4743-4efc-9aa1-eb79a741f6a4" Jul 03 11:12:34.986286 ip-10-0-136-0 kubenswrapper[2579]: I0703 11:12:34.986270 2579 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-9pqm8" Jul 03 11:12:34.986385 ip-10-0-136-0 kubenswrapper[2579]: E0703 11:12:34.986343 2579 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-9pqm8" podUID="7813f158-0f67-411e-84d9-a2a1c51eea61" Jul 03 11:12:35.986626 ip-10-0-136-0 kubenswrapper[2579]: I0703 11:12:35.986502 2579 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-xzlsc" Jul 03 11:12:35.987373 ip-10-0-136-0 kubenswrapper[2579]: E0703 11:12:35.986697 2579 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-xzlsc" podUID="2129ac30-1645-4555-b722-55963ffb8776" Jul 03 11:12:36.986116 ip-10-0-136-0 kubenswrapper[2579]: I0703 11:12:36.986073 2579 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-sqxsj" Jul 03 11:12:36.986285 ip-10-0-136-0 kubenswrapper[2579]: E0703 11:12:36.986252 2579 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-sqxsj" podUID="34ca9c51-4743-4efc-9aa1-eb79a741f6a4" Jul 03 11:12:36.986674 ip-10-0-136-0 kubenswrapper[2579]: I0703 11:12:36.986654 2579 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-9pqm8" Jul 03 11:12:36.987010 ip-10-0-136-0 kubenswrapper[2579]: E0703 11:12:36.986777 2579 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-9pqm8" podUID="7813f158-0f67-411e-84d9-a2a1c51eea61" Jul 03 11:12:37.985639 ip-10-0-136-0 kubenswrapper[2579]: I0703 11:12:37.985600 2579 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-xzlsc" Jul 03 11:12:37.985810 ip-10-0-136-0 kubenswrapper[2579]: E0703 11:12:37.985737 2579 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-xzlsc" podUID="2129ac30-1645-4555-b722-55963ffb8776" Jul 03 11:12:38.985770 ip-10-0-136-0 kubenswrapper[2579]: I0703 11:12:38.985723 2579 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-sqxsj" Jul 03 11:12:38.985770 ip-10-0-136-0 kubenswrapper[2579]: I0703 11:12:38.985772 2579 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-9pqm8" Jul 03 11:12:38.986338 ip-10-0-136-0 kubenswrapper[2579]: E0703 11:12:38.985876 2579 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-sqxsj" podUID="34ca9c51-4743-4efc-9aa1-eb79a741f6a4" Jul 03 11:12:38.986338 ip-10-0-136-0 kubenswrapper[2579]: E0703 11:12:38.985996 2579 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-9pqm8" podUID="7813f158-0f67-411e-84d9-a2a1c51eea61" Jul 03 11:12:39.986504 ip-10-0-136-0 kubenswrapper[2579]: I0703 11:12:39.986455 2579 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-xzlsc" Jul 03 11:12:39.987001 ip-10-0-136-0 kubenswrapper[2579]: E0703 11:12:39.986623 2579 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-xzlsc" podUID="2129ac30-1645-4555-b722-55963ffb8776" Jul 03 11:12:40.986172 ip-10-0-136-0 kubenswrapper[2579]: I0703 11:12:40.986129 2579 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-sqxsj" Jul 03 11:12:40.986353 ip-10-0-136-0 kubenswrapper[2579]: E0703 11:12:40.986296 2579 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-sqxsj" podUID="34ca9c51-4743-4efc-9aa1-eb79a741f6a4" Jul 03 11:12:40.986411 ip-10-0-136-0 kubenswrapper[2579]: I0703 11:12:40.986347 2579 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-9pqm8" Jul 03 11:12:40.986478 ip-10-0-136-0 kubenswrapper[2579]: E0703 11:12:40.986449 2579 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-9pqm8" podUID="7813f158-0f67-411e-84d9-a2a1c51eea61" Jul 03 11:12:41.666816 ip-10-0-136-0 kubenswrapper[2579]: I0703 11:12:41.666770 2579 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/34ca9c51-4743-4efc-9aa1-eb79a741f6a4-metrics-certs\") pod \"network-metrics-daemon-sqxsj\" (UID: \"34ca9c51-4743-4efc-9aa1-eb79a741f6a4\") " pod="openshift-multus/network-metrics-daemon-sqxsj" Jul 03 11:12:41.666816 ip-10-0-136-0 kubenswrapper[2579]: I0703 11:12:41.666832 2579 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"original-pull-secret\" (UniqueName: \"kubernetes.io/secret/7813f158-0f67-411e-84d9-a2a1c51eea61-original-pull-secret\") pod \"global-pull-secret-syncer-9pqm8\" (UID: \"7813f158-0f67-411e-84d9-a2a1c51eea61\") " pod="kube-system/global-pull-secret-syncer-9pqm8" Jul 03 11:12:41.667369 ip-10-0-136-0 kubenswrapper[2579]: E0703 11:12:41.666950 2579 secret.go:189] Couldn't get secret kube-system/original-pull-secret: object "kube-system"/"original-pull-secret" not registered Jul 03 11:12:41.667369 ip-10-0-136-0 kubenswrapper[2579]: E0703 11:12:41.666954 2579 secret.go:189] Couldn't get secret openshift-multus/metrics-daemon-secret: object "openshift-multus"/"metrics-daemon-secret" not registered Jul 03 11:12:41.667369 ip-10-0-136-0 kubenswrapper[2579]: E0703 11:12:41.667020 2579 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/34ca9c51-4743-4efc-9aa1-eb79a741f6a4-metrics-certs podName:34ca9c51-4743-4efc-9aa1-eb79a741f6a4 nodeName:}" failed. No retries permitted until 2026-07-03 11:12:57.667001647 +0000 UTC m=+34.255305948 (durationBeforeRetry 16s). Error: MountVolume.SetUp failed for volume "metrics-certs" (UniqueName: "kubernetes.io/secret/34ca9c51-4743-4efc-9aa1-eb79a741f6a4-metrics-certs") pod "network-metrics-daemon-sqxsj" (UID: "34ca9c51-4743-4efc-9aa1-eb79a741f6a4") : object "openshift-multus"/"metrics-daemon-secret" not registered Jul 03 11:12:41.667369 ip-10-0-136-0 kubenswrapper[2579]: E0703 11:12:41.667040 2579 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/7813f158-0f67-411e-84d9-a2a1c51eea61-original-pull-secret podName:7813f158-0f67-411e-84d9-a2a1c51eea61 nodeName:}" failed. No retries permitted until 2026-07-03 11:12:57.667030771 +0000 UTC m=+34.255335071 (durationBeforeRetry 16s). Error: MountVolume.SetUp failed for volume "original-pull-secret" (UniqueName: "kubernetes.io/secret/7813f158-0f67-411e-84d9-a2a1c51eea61-original-pull-secret") pod "global-pull-secret-syncer-9pqm8" (UID: "7813f158-0f67-411e-84d9-a2a1c51eea61") : object "kube-system"/"original-pull-secret" not registered Jul 03 11:12:41.767523 ip-10-0-136-0 kubenswrapper[2579]: I0703 11:12:41.767475 2579 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-flqqx\" (UniqueName: \"kubernetes.io/projected/2129ac30-1645-4555-b722-55963ffb8776-kube-api-access-flqqx\") pod \"network-check-target-xzlsc\" (UID: \"2129ac30-1645-4555-b722-55963ffb8776\") " pod="openshift-network-diagnostics/network-check-target-xzlsc" Jul 03 11:12:41.767728 ip-10-0-136-0 kubenswrapper[2579]: E0703 11:12:41.767654 2579 projected.go:291] Couldn't get configMap openshift-network-diagnostics/kube-root-ca.crt: object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered Jul 03 11:12:41.767728 ip-10-0-136-0 kubenswrapper[2579]: E0703 11:12:41.767678 2579 projected.go:291] Couldn't get configMap openshift-network-diagnostics/openshift-service-ca.crt: object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered Jul 03 11:12:41.767728 ip-10-0-136-0 kubenswrapper[2579]: E0703 11:12:41.767690 2579 projected.go:196] Error preparing data for projected volume kube-api-access-flqqx for pod openshift-network-diagnostics/network-check-target-xzlsc: [object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered, object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered] Jul 03 11:12:41.767882 ip-10-0-136-0 kubenswrapper[2579]: E0703 11:12:41.767755 2579 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/projected/2129ac30-1645-4555-b722-55963ffb8776-kube-api-access-flqqx podName:2129ac30-1645-4555-b722-55963ffb8776 nodeName:}" failed. No retries permitted until 2026-07-03 11:12:57.767733803 +0000 UTC m=+34.356038111 (durationBeforeRetry 16s). Error: MountVolume.SetUp failed for volume "kube-api-access-flqqx" (UniqueName: "kubernetes.io/projected/2129ac30-1645-4555-b722-55963ffb8776-kube-api-access-flqqx") pod "network-check-target-xzlsc" (UID: "2129ac30-1645-4555-b722-55963ffb8776") : [object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered, object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered] Jul 03 11:12:41.985661 ip-10-0-136-0 kubenswrapper[2579]: I0703 11:12:41.985561 2579 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-xzlsc" Jul 03 11:12:41.985828 ip-10-0-136-0 kubenswrapper[2579]: E0703 11:12:41.985712 2579 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-xzlsc" podUID="2129ac30-1645-4555-b722-55963ffb8776" Jul 03 11:12:42.985621 ip-10-0-136-0 kubenswrapper[2579]: I0703 11:12:42.985580 2579 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-9pqm8" Jul 03 11:12:42.985621 ip-10-0-136-0 kubenswrapper[2579]: I0703 11:12:42.985610 2579 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-sqxsj" Jul 03 11:12:42.986103 ip-10-0-136-0 kubenswrapper[2579]: E0703 11:12:42.985732 2579 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-9pqm8" podUID="7813f158-0f67-411e-84d9-a2a1c51eea61" Jul 03 11:12:42.986103 ip-10-0-136-0 kubenswrapper[2579]: E0703 11:12:42.985880 2579 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-sqxsj" podUID="34ca9c51-4743-4efc-9aa1-eb79a741f6a4" Jul 03 11:12:43.987121 ip-10-0-136-0 kubenswrapper[2579]: I0703 11:12:43.987078 2579 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-xzlsc" Jul 03 11:12:43.987600 ip-10-0-136-0 kubenswrapper[2579]: E0703 11:12:43.987227 2579 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-xzlsc" podUID="2129ac30-1645-4555-b722-55963ffb8776" Jul 03 11:12:44.985930 ip-10-0-136-0 kubenswrapper[2579]: I0703 11:12:44.985879 2579 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-9pqm8" Jul 03 11:12:44.986185 ip-10-0-136-0 kubenswrapper[2579]: I0703 11:12:44.985879 2579 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-sqxsj" Jul 03 11:12:44.986185 ip-10-0-136-0 kubenswrapper[2579]: E0703 11:12:44.986032 2579 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-9pqm8" podUID="7813f158-0f67-411e-84d9-a2a1c51eea61" Jul 03 11:12:44.986185 ip-10-0-136-0 kubenswrapper[2579]: E0703 11:12:44.986087 2579 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-sqxsj" podUID="34ca9c51-4743-4efc-9aa1-eb79a741f6a4" Jul 03 11:12:45.985895 ip-10-0-136-0 kubenswrapper[2579]: I0703 11:12:45.985859 2579 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-xzlsc" Jul 03 11:12:45.986438 ip-10-0-136-0 kubenswrapper[2579]: E0703 11:12:45.986011 2579 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-xzlsc" podUID="2129ac30-1645-4555-b722-55963ffb8776" Jul 03 11:12:46.986639 ip-10-0-136-0 kubenswrapper[2579]: I0703 11:12:46.986032 2579 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-sqxsj" Jul 03 11:12:46.988119 ip-10-0-136-0 kubenswrapper[2579]: I0703 11:12:46.986032 2579 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-9pqm8" Jul 03 11:12:46.988119 ip-10-0-136-0 kubenswrapper[2579]: E0703 11:12:46.986818 2579 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-sqxsj" podUID="34ca9c51-4743-4efc-9aa1-eb79a741f6a4" Jul 03 11:12:46.988119 ip-10-0-136-0 kubenswrapper[2579]: E0703 11:12:46.986932 2579 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-9pqm8" podUID="7813f158-0f67-411e-84d9-a2a1c51eea61" Jul 03 11:12:47.239683 ip-10-0-136-0 kubenswrapper[2579]: I0703 11:12:47.239649 2579 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ovn-kubernetes/ovnkube-node-dchtv" event={"ID":"45f92856-212a-4190-93ae-dfec96e1864e","Type":"ContainerStarted","Data":"20915bc5e1331feea9578d190b2ed6516d632efd5355b3b7c4713c869b26d56b"} Jul 03 11:12:47.239857 ip-10-0-136-0 kubenswrapper[2579]: I0703 11:12:47.239745 2579 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ovn-kubernetes/ovnkube-node-dchtv" event={"ID":"45f92856-212a-4190-93ae-dfec96e1864e","Type":"ContainerStarted","Data":"5a4dcd18cb27926ffa8e455112941aa7a93ca8d97b6e0db08082fa3eb38d78c8"} Jul 03 11:12:47.239857 ip-10-0-136-0 kubenswrapper[2579]: I0703 11:12:47.239795 2579 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ovn-kubernetes/ovnkube-node-dchtv" event={"ID":"45f92856-212a-4190-93ae-dfec96e1864e","Type":"ContainerStarted","Data":"9f300fff90eab6d42a774e5095234308d491ce06223a336f48dabb79172382e2"} Jul 03 11:12:47.239857 ip-10-0-136-0 kubenswrapper[2579]: I0703 11:12:47.239844 2579 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ovn-kubernetes/ovnkube-node-dchtv" event={"ID":"45f92856-212a-4190-93ae-dfec96e1864e","Type":"ContainerStarted","Data":"85204bc16bce75604eb8f7c17942b24804e23001839478692e6527ab5843317c"} Jul 03 11:12:47.239970 ip-10-0-136-0 kubenswrapper[2579]: I0703 11:12:47.239886 2579 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ovn-kubernetes/ovnkube-node-dchtv" event={"ID":"45f92856-212a-4190-93ae-dfec96e1864e","Type":"ContainerStarted","Data":"cc34b2f419a8494804b6a662ea9dc5c9a0450e49ec95f3ec2b43e9722d244be0"} Jul 03 11:12:47.241094 ip-10-0-136-0 kubenswrapper[2579]: I0703 11:12:47.241071 2579 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-dns/node-resolver-hnmtl" event={"ID":"9a6053f3-4885-4e53-8b1c-93150a7546c4","Type":"ContainerStarted","Data":"95606a6b20bfb97a3c66e5441ca274fa577a87cf299d8160ddd2f56480a97792"} Jul 03 11:12:47.242827 ip-10-0-136-0 kubenswrapper[2579]: I0703 11:12:47.242799 2579 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kube-system/konnectivity-agent-5lz4n" event={"ID":"945b7f84-a41f-413e-a7b6-87818468360f","Type":"ContainerStarted","Data":"f81832795c7a52790d6aa3f5d706f3dc9e3d1810681d5f2d7cc56328c015afef"} Jul 03 11:12:47.244467 ip-10-0-136-0 kubenswrapper[2579]: I0703 11:12:47.244437 2579 generic.go:356] "Generic (PLEG): container finished" podID="e9fc3c32-a741-4377-970b-6cb08e5b4ac9" containerID="0998bf156030e3d6898574cfc190c13c32f00e1f67fd9e2e24b3f9de182811bf" exitCode=0 Jul 03 11:12:47.244586 ip-10-0-136-0 kubenswrapper[2579]: I0703 11:12:47.244519 2579 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-multus/multus-additional-cni-plugins-52n9r" event={"ID":"e9fc3c32-a741-4377-970b-6cb08e5b4ac9","Type":"ContainerDied","Data":"0998bf156030e3d6898574cfc190c13c32f00e1f67fd9e2e24b3f9de182811bf"} Jul 03 11:12:47.246157 ip-10-0-136-0 kubenswrapper[2579]: I0703 11:12:47.246105 2579 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-cluster-node-tuning-operator/tuned-dwp69" event={"ID":"99eb1d1a-c732-4365-9da7-ecab8ead5794","Type":"ContainerStarted","Data":"609b45cb5e9a50e1e57c32f8537334306a11f304642387b88fdd3bf5d8f52eda"} Jul 03 11:12:47.248012 ip-10-0-136-0 kubenswrapper[2579]: I0703 11:12:47.247980 2579 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-multus/multus-cszpw" event={"ID":"ad5f7b2c-eac2-4ebb-b1d8-62782a621d85","Type":"ContainerStarted","Data":"e78a0f2070dad80c069ef9ff19c41113d281eedc461df545cec79cb97dbd1e30"} Jul 03 11:12:47.249608 ip-10-0-136-0 kubenswrapper[2579]: I0703 11:12:47.249570 2579 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-image-registry/node-ca-9ff9g" event={"ID":"f5fa8ffd-d2fc-4474-ad0d-a7af28edf0e7","Type":"ContainerStarted","Data":"0365a7b64b903fd2a1800e505d28dbb89fcb9227f174266962505e0d1a414626"} Jul 03 11:12:47.251241 ip-10-0-136-0 kubenswrapper[2579]: I0703 11:12:47.251216 2579 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-fj786" event={"ID":"a39e260a-4eff-4acb-9b7f-3ea1f42ff3c0","Type":"ContainerStarted","Data":"a50644a3e6ae2441f890e24c29fbb9d171ba968907c1db286048d3c1cc28b302"} Jul 03 11:12:47.271676 ip-10-0-136-0 kubenswrapper[2579]: I0703 11:12:47.271614 2579 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-dns/node-resolver-hnmtl" podStartSLOduration=5.02698611 podStartE2EDuration="24.271588935s" totalImagesPullingTime="19.244602825s" totalInitContainerRuntime="0s" isStatefulPod=true podCreationTimestamp="2026-07-03 11:12:23 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-03 11:12:47.255850745 +0000 UTC m=+23.844155064" watchObservedRunningTime="2026-07-03 11:12:47.271588935 +0000 UTC m=+23.859893314" Jul 03 11:12:47.272381 ip-10-0-136-0 kubenswrapper[2579]: I0703 11:12:47.272337 2579 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-image-registry/node-ca-9ff9g" podStartSLOduration=4.024289122 podStartE2EDuration="23.272317323s" totalImagesPullingTime="19.248028201s" totalInitContainerRuntime="0s" isStatefulPod=true podCreationTimestamp="2026-07-03 11:12:24 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-03 11:12:47.27154604 +0000 UTC m=+23.859850360" watchObservedRunningTime="2026-07-03 11:12:47.272317323 +0000 UTC m=+23.860621652" Jul 03 11:12:47.289155 ip-10-0-136-0 kubenswrapper[2579]: I0703 11:12:47.289102 2579 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-multus/multus-cszpw" podStartSLOduration=3.83790108 podStartE2EDuration="23.289082558s" totalImagesPullingTime="19.451181478s" totalInitContainerRuntime="0s" isStatefulPod=true podCreationTimestamp="2026-07-03 11:12:24 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-03 11:12:47.288632708 +0000 UTC m=+23.876937029" watchObservedRunningTime="2026-07-03 11:12:47.289082558 +0000 UTC m=+23.877386878" Jul 03 11:12:47.306970 ip-10-0-136-0 kubenswrapper[2579]: I0703 11:12:47.306918 2579 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-cluster-node-tuning-operator/tuned-dwp69" podStartSLOduration=4.069186593 podStartE2EDuration="23.306899395s" totalImagesPullingTime="19.237712802s" totalInitContainerRuntime="0s" isStatefulPod=true podCreationTimestamp="2026-07-03 11:12:24 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-03 11:12:47.306727308 +0000 UTC m=+23.895031627" watchObservedRunningTime="2026-07-03 11:12:47.306899395 +0000 UTC m=+23.895203715" Jul 03 11:12:47.324724 ip-10-0-136-0 kubenswrapper[2579]: I0703 11:12:47.324671 2579 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="kube-system/konnectivity-agent-5lz4n" podStartSLOduration=9.659800911 podStartE2EDuration="24.324652704s" totalImagesPullingTime="14.664851793s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-03 11:12:23 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-03 11:12:47.323903547 +0000 UTC m=+23.912207868" watchObservedRunningTime="2026-07-03 11:12:47.324652704 +0000 UTC m=+23.912957024" Jul 03 11:12:47.860265 ip-10-0-136-0 kubenswrapper[2579]: I0703 11:12:47.860238 2579 plugin_watcher.go:194] "Adding socket path or updating timestamp to desired state cache" path="/var/lib/kubelet/plugins_registry/ebs.csi.aws.com-reg.sock" Jul 03 11:12:47.887026 ip-10-0-136-0 kubenswrapper[2579]: I0703 11:12:47.886916 2579 reconciler.go:161] "OperationExecutor.RegisterPlugin started" plugin={"SocketPath":"/var/lib/kubelet/plugins_registry/ebs.csi.aws.com-reg.sock","Timestamp":"2026-07-03T11:12:47.860258245Z","UUID":"b43217fd-fe74-4898-a4fe-57221cf26351","Handler":null,"Name":"","Endpoint":""} Jul 03 11:12:47.888418 ip-10-0-136-0 kubenswrapper[2579]: I0703 11:12:47.888399 2579 csi_plugin.go:106] kubernetes.io/csi: Trying to validate a new CSI Driver with name: ebs.csi.aws.com endpoint: /var/lib/kubelet/plugins/ebs.csi.aws.com/csi.sock versions: 1.0.0 Jul 03 11:12:47.888534 ip-10-0-136-0 kubenswrapper[2579]: I0703 11:12:47.888441 2579 csi_plugin.go:119] kubernetes.io/csi: Register new plugin with name: ebs.csi.aws.com at endpoint: /var/lib/kubelet/plugins/ebs.csi.aws.com/csi.sock Jul 03 11:12:47.985988 ip-10-0-136-0 kubenswrapper[2579]: I0703 11:12:47.985954 2579 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-xzlsc" Jul 03 11:12:47.986176 ip-10-0-136-0 kubenswrapper[2579]: E0703 11:12:47.986114 2579 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-xzlsc" podUID="2129ac30-1645-4555-b722-55963ffb8776" Jul 03 11:12:48.254912 ip-10-0-136-0 kubenswrapper[2579]: I0703 11:12:48.254877 2579 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-network-operator/iptables-alerter-l8lpk" event={"ID":"a8bc35c6-22ff-4c87-802d-e44999086c38","Type":"ContainerStarted","Data":"30cd885d146b57672b76bb2c7513b70df98b6b1ee81005e85f1759e539f12a65"} Jul 03 11:12:48.256788 ip-10-0-136-0 kubenswrapper[2579]: I0703 11:12:48.256761 2579 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-fj786" event={"ID":"a39e260a-4eff-4acb-9b7f-3ea1f42ff3c0","Type":"ContainerStarted","Data":"9d3f9e1fd80c4f407ec42e0fb14443753ecb2d5c7c050ed95076519f474ed531"} Jul 03 11:12:48.260011 ip-10-0-136-0 kubenswrapper[2579]: I0703 11:12:48.259979 2579 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ovn-kubernetes/ovnkube-node-dchtv" event={"ID":"45f92856-212a-4190-93ae-dfec96e1864e","Type":"ContainerStarted","Data":"23a3e52ad6fe7b723acfb99eca9aa7fb26f6690c7b19229d4e460d3ca7b50927"} Jul 03 11:12:48.545940 ip-10-0-136-0 kubenswrapper[2579]: I0703 11:12:48.545840 2579 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="kube-system/konnectivity-agent-5lz4n" Jul 03 11:12:48.546632 ip-10-0-136-0 kubenswrapper[2579]: I0703 11:12:48.546604 2579 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="kube-system/konnectivity-agent-5lz4n" Jul 03 11:12:48.563624 ip-10-0-136-0 kubenswrapper[2579]: I0703 11:12:48.563564 2579 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-network-operator/iptables-alerter-l8lpk" podStartSLOduration=5.322524315 podStartE2EDuration="24.563543619s" totalImagesPullingTime="19.241019304s" totalInitContainerRuntime="0s" isStatefulPod=true podCreationTimestamp="2026-07-03 11:12:24 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-03 11:12:48.268283931 +0000 UTC m=+24.856588254" watchObservedRunningTime="2026-07-03 11:12:48.563543619 +0000 UTC m=+25.151848095" Jul 03 11:12:48.986244 ip-10-0-136-0 kubenswrapper[2579]: I0703 11:12:48.986120 2579 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-sqxsj" Jul 03 11:12:48.986244 ip-10-0-136-0 kubenswrapper[2579]: I0703 11:12:48.986164 2579 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-9pqm8" Jul 03 11:12:48.986504 ip-10-0-136-0 kubenswrapper[2579]: E0703 11:12:48.986445 2579 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-sqxsj" podUID="34ca9c51-4743-4efc-9aa1-eb79a741f6a4" Jul 03 11:12:48.986723 ip-10-0-136-0 kubenswrapper[2579]: E0703 11:12:48.986687 2579 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-9pqm8" podUID="7813f158-0f67-411e-84d9-a2a1c51eea61" Jul 03 11:12:49.265045 ip-10-0-136-0 kubenswrapper[2579]: I0703 11:12:49.264947 2579 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-fj786" event={"ID":"a39e260a-4eff-4acb-9b7f-3ea1f42ff3c0","Type":"ContainerStarted","Data":"2d0b7201aab3089e0c89e068b99b204c1db7fb7747e7966a554b5ed6f7b2eb12"} Jul 03 11:12:49.265561 ip-10-0-136-0 kubenswrapper[2579]: I0703 11:12:49.265453 2579 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="kube-system/konnectivity-agent-5lz4n" Jul 03 11:12:49.265960 ip-10-0-136-0 kubenswrapper[2579]: I0703 11:12:49.265940 2579 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="kube-system/konnectivity-agent-5lz4n" Jul 03 11:12:49.306829 ip-10-0-136-0 kubenswrapper[2579]: I0703 11:12:49.306759 2579 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-fj786" podStartSLOduration=4.094751945 podStartE2EDuration="25.306740603s" totalImagesPullingTime="21.211988658s" totalInitContainerRuntime="0s" isStatefulPod=true podCreationTimestamp="2026-07-03 11:12:24 +0000 UTC" imagePullSessionsCount=3 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-03 11:12:49.286683622 +0000 UTC m=+25.874987941" watchObservedRunningTime="2026-07-03 11:12:49.306740603 +0000 UTC m=+25.895044924" Jul 03 11:12:49.986313 ip-10-0-136-0 kubenswrapper[2579]: I0703 11:12:49.986267 2579 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-xzlsc" Jul 03 11:12:49.986507 ip-10-0-136-0 kubenswrapper[2579]: E0703 11:12:49.986424 2579 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-xzlsc" podUID="2129ac30-1645-4555-b722-55963ffb8776" Jul 03 11:12:50.272043 ip-10-0-136-0 kubenswrapper[2579]: I0703 11:12:50.271945 2579 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ovn-kubernetes/ovnkube-node-dchtv" event={"ID":"45f92856-212a-4190-93ae-dfec96e1864e","Type":"ContainerStarted","Data":"55a77de44a054438a1d304952a0efca84ed146322a56397df8a22c721d0a9db4"} Jul 03 11:12:50.986261 ip-10-0-136-0 kubenswrapper[2579]: I0703 11:12:50.985954 2579 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-9pqm8" Jul 03 11:12:50.986441 ip-10-0-136-0 kubenswrapper[2579]: I0703 11:12:50.986060 2579 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-sqxsj" Jul 03 11:12:50.986441 ip-10-0-136-0 kubenswrapper[2579]: E0703 11:12:50.986397 2579 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-9pqm8" podUID="7813f158-0f67-411e-84d9-a2a1c51eea61" Jul 03 11:12:50.986441 ip-10-0-136-0 kubenswrapper[2579]: E0703 11:12:50.986431 2579 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-sqxsj" podUID="34ca9c51-4743-4efc-9aa1-eb79a741f6a4" Jul 03 11:12:51.985797 ip-10-0-136-0 kubenswrapper[2579]: I0703 11:12:51.985754 2579 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-xzlsc" Jul 03 11:12:51.986452 ip-10-0-136-0 kubenswrapper[2579]: E0703 11:12:51.985906 2579 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-xzlsc" podUID="2129ac30-1645-4555-b722-55963ffb8776" Jul 03 11:12:52.986180 ip-10-0-136-0 kubenswrapper[2579]: I0703 11:12:52.985990 2579 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-9pqm8" Jul 03 11:12:52.986722 ip-10-0-136-0 kubenswrapper[2579]: I0703 11:12:52.986029 2579 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-sqxsj" Jul 03 11:12:52.986722 ip-10-0-136-0 kubenswrapper[2579]: E0703 11:12:52.986295 2579 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-9pqm8" podUID="7813f158-0f67-411e-84d9-a2a1c51eea61" Jul 03 11:12:52.986722 ip-10-0-136-0 kubenswrapper[2579]: E0703 11:12:52.986427 2579 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-sqxsj" podUID="34ca9c51-4743-4efc-9aa1-eb79a741f6a4" Jul 03 11:12:53.283246 ip-10-0-136-0 kubenswrapper[2579]: I0703 11:12:53.283121 2579 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ovn-kubernetes/ovnkube-node-dchtv" event={"ID":"45f92856-212a-4190-93ae-dfec96e1864e","Type":"ContainerStarted","Data":"74337d5ecde3b79e8493eae8444297606f193812c68894bc383e9d3f221024c3"} Jul 03 11:12:53.283474 ip-10-0-136-0 kubenswrapper[2579]: I0703 11:12:53.283400 2579 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ovn-kubernetes/ovnkube-node-dchtv" Jul 03 11:12:53.285417 ip-10-0-136-0 kubenswrapper[2579]: I0703 11:12:53.285387 2579 generic.go:356] "Generic (PLEG): container finished" podID="e9fc3c32-a741-4377-970b-6cb08e5b4ac9" containerID="3535bd4637448e3a558bf2f6b48979f60e2808fda37b22071793af5691c85615" exitCode=0 Jul 03 11:12:53.285585 ip-10-0-136-0 kubenswrapper[2579]: I0703 11:12:53.285423 2579 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-multus/multus-additional-cni-plugins-52n9r" event={"ID":"e9fc3c32-a741-4377-970b-6cb08e5b4ac9","Type":"ContainerDied","Data":"3535bd4637448e3a558bf2f6b48979f60e2808fda37b22071793af5691c85615"} Jul 03 11:12:53.300791 ip-10-0-136-0 kubenswrapper[2579]: I0703 11:12:53.300763 2579 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ovn-kubernetes/ovnkube-node-dchtv" Jul 03 11:12:53.309098 ip-10-0-136-0 kubenswrapper[2579]: I0703 11:12:53.309048 2579 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-ovn-kubernetes/ovnkube-node-dchtv" podStartSLOduration=9.797696189 podStartE2EDuration="29.309031788s" totalImagesPullingTime="19.511335599s" totalInitContainerRuntime="0s" isStatefulPod=true podCreationTimestamp="2026-07-03 11:12:24 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-03 11:12:53.308967585 +0000 UTC m=+29.897271918" watchObservedRunningTime="2026-07-03 11:12:53.309031788 +0000 UTC m=+29.897336154" Jul 03 11:12:53.909341 ip-10-0-136-0 kubenswrapper[2579]: I0703 11:12:53.909303 2579 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ovn-kubernetes/ovnkube-node-dchtv" Jul 03 11:12:53.987379 ip-10-0-136-0 kubenswrapper[2579]: I0703 11:12:53.987331 2579 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-xzlsc" Jul 03 11:12:53.987861 ip-10-0-136-0 kubenswrapper[2579]: E0703 11:12:53.987477 2579 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-xzlsc" podUID="2129ac30-1645-4555-b722-55963ffb8776" Jul 03 11:12:54.288233 ip-10-0-136-0 kubenswrapper[2579]: I0703 11:12:54.288113 2579 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ovn-kubernetes/ovnkube-node-dchtv" Jul 03 11:12:54.306326 ip-10-0-136-0 kubenswrapper[2579]: I0703 11:12:54.306295 2579 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ovn-kubernetes/ovnkube-node-dchtv" Jul 03 11:12:54.826517 ip-10-0-136-0 kubenswrapper[2579]: I0703 11:12:54.826034 2579 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-multus/network-metrics-daemon-sqxsj"] Jul 03 11:12:54.826517 ip-10-0-136-0 kubenswrapper[2579]: I0703 11:12:54.826209 2579 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-sqxsj" Jul 03 11:12:54.826517 ip-10-0-136-0 kubenswrapper[2579]: E0703 11:12:54.826336 2579 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-sqxsj" podUID="34ca9c51-4743-4efc-9aa1-eb79a741f6a4" Jul 03 11:12:54.827009 ip-10-0-136-0 kubenswrapper[2579]: I0703 11:12:54.826988 2579 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kube-system/global-pull-secret-syncer-9pqm8"] Jul 03 11:12:54.827121 ip-10-0-136-0 kubenswrapper[2579]: I0703 11:12:54.827107 2579 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-9pqm8" Jul 03 11:12:54.827256 ip-10-0-136-0 kubenswrapper[2579]: E0703 11:12:54.827233 2579 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-9pqm8" podUID="7813f158-0f67-411e-84d9-a2a1c51eea61" Jul 03 11:12:54.827713 ip-10-0-136-0 kubenswrapper[2579]: I0703 11:12:54.827694 2579 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-network-diagnostics/network-check-target-xzlsc"] Jul 03 11:12:54.827776 ip-10-0-136-0 kubenswrapper[2579]: I0703 11:12:54.827761 2579 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-xzlsc" Jul 03 11:12:54.827851 ip-10-0-136-0 kubenswrapper[2579]: E0703 11:12:54.827832 2579 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-xzlsc" podUID="2129ac30-1645-4555-b722-55963ffb8776" Jul 03 11:12:55.292055 ip-10-0-136-0 kubenswrapper[2579]: I0703 11:12:55.292011 2579 generic.go:356] "Generic (PLEG): container finished" podID="e9fc3c32-a741-4377-970b-6cb08e5b4ac9" containerID="2fe87a2e1dc6f349411aacd622579f4780ee06c5debda35824111965bd0ab431" exitCode=0 Jul 03 11:12:55.292552 ip-10-0-136-0 kubenswrapper[2579]: I0703 11:12:55.292093 2579 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-multus/multus-additional-cni-plugins-52n9r" event={"ID":"e9fc3c32-a741-4377-970b-6cb08e5b4ac9","Type":"ContainerDied","Data":"2fe87a2e1dc6f349411aacd622579f4780ee06c5debda35824111965bd0ab431"} Jul 03 11:12:56.986057 ip-10-0-136-0 kubenswrapper[2579]: I0703 11:12:56.985963 2579 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-xzlsc" Jul 03 11:12:56.986693 ip-10-0-136-0 kubenswrapper[2579]: I0703 11:12:56.986069 2579 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-9pqm8" Jul 03 11:12:56.986693 ip-10-0-136-0 kubenswrapper[2579]: E0703 11:12:56.986077 2579 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-xzlsc" podUID="2129ac30-1645-4555-b722-55963ffb8776" Jul 03 11:12:56.986693 ip-10-0-136-0 kubenswrapper[2579]: I0703 11:12:56.986108 2579 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-sqxsj" Jul 03 11:12:56.986693 ip-10-0-136-0 kubenswrapper[2579]: E0703 11:12:56.986226 2579 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-9pqm8" podUID="7813f158-0f67-411e-84d9-a2a1c51eea61" Jul 03 11:12:56.986693 ip-10-0-136-0 kubenswrapper[2579]: E0703 11:12:56.986336 2579 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-sqxsj" podUID="34ca9c51-4743-4efc-9aa1-eb79a741f6a4" Jul 03 11:12:57.299694 ip-10-0-136-0 kubenswrapper[2579]: I0703 11:12:57.299611 2579 generic.go:356] "Generic (PLEG): container finished" podID="e9fc3c32-a741-4377-970b-6cb08e5b4ac9" containerID="232a664bf6d64e3a9b44715c7536e7933e9e07b7fb182380dea2d0a84f507367" exitCode=0 Jul 03 11:12:57.299980 ip-10-0-136-0 kubenswrapper[2579]: I0703 11:12:57.299947 2579 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-multus/multus-additional-cni-plugins-52n9r" event={"ID":"e9fc3c32-a741-4377-970b-6cb08e5b4ac9","Type":"ContainerDied","Data":"232a664bf6d64e3a9b44715c7536e7933e9e07b7fb182380dea2d0a84f507367"} Jul 03 11:12:57.695293 ip-10-0-136-0 kubenswrapper[2579]: I0703 11:12:57.695171 2579 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/34ca9c51-4743-4efc-9aa1-eb79a741f6a4-metrics-certs\") pod \"network-metrics-daemon-sqxsj\" (UID: \"34ca9c51-4743-4efc-9aa1-eb79a741f6a4\") " pod="openshift-multus/network-metrics-daemon-sqxsj" Jul 03 11:12:57.695293 ip-10-0-136-0 kubenswrapper[2579]: I0703 11:12:57.695243 2579 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"original-pull-secret\" (UniqueName: \"kubernetes.io/secret/7813f158-0f67-411e-84d9-a2a1c51eea61-original-pull-secret\") pod \"global-pull-secret-syncer-9pqm8\" (UID: \"7813f158-0f67-411e-84d9-a2a1c51eea61\") " pod="kube-system/global-pull-secret-syncer-9pqm8" Jul 03 11:12:57.695481 ip-10-0-136-0 kubenswrapper[2579]: E0703 11:12:57.695352 2579 secret.go:189] Couldn't get secret openshift-multus/metrics-daemon-secret: object "openshift-multus"/"metrics-daemon-secret" not registered Jul 03 11:12:57.695481 ip-10-0-136-0 kubenswrapper[2579]: E0703 11:12:57.695365 2579 secret.go:189] Couldn't get secret kube-system/original-pull-secret: object "kube-system"/"original-pull-secret" not registered Jul 03 11:12:57.695481 ip-10-0-136-0 kubenswrapper[2579]: E0703 11:12:57.695429 2579 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/7813f158-0f67-411e-84d9-a2a1c51eea61-original-pull-secret podName:7813f158-0f67-411e-84d9-a2a1c51eea61 nodeName:}" failed. No retries permitted until 2026-07-03 11:13:29.695412021 +0000 UTC m=+66.283716339 (durationBeforeRetry 32s). Error: MountVolume.SetUp failed for volume "original-pull-secret" (UniqueName: "kubernetes.io/secret/7813f158-0f67-411e-84d9-a2a1c51eea61-original-pull-secret") pod "global-pull-secret-syncer-9pqm8" (UID: "7813f158-0f67-411e-84d9-a2a1c51eea61") : object "kube-system"/"original-pull-secret" not registered Jul 03 11:12:57.695481 ip-10-0-136-0 kubenswrapper[2579]: E0703 11:12:57.695448 2579 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/34ca9c51-4743-4efc-9aa1-eb79a741f6a4-metrics-certs podName:34ca9c51-4743-4efc-9aa1-eb79a741f6a4 nodeName:}" failed. No retries permitted until 2026-07-03 11:13:29.695439181 +0000 UTC m=+66.283743487 (durationBeforeRetry 32s). Error: MountVolume.SetUp failed for volume "metrics-certs" (UniqueName: "kubernetes.io/secret/34ca9c51-4743-4efc-9aa1-eb79a741f6a4-metrics-certs") pod "network-metrics-daemon-sqxsj" (UID: "34ca9c51-4743-4efc-9aa1-eb79a741f6a4") : object "openshift-multus"/"metrics-daemon-secret" not registered Jul 03 11:12:57.796672 ip-10-0-136-0 kubenswrapper[2579]: I0703 11:12:57.796634 2579 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-flqqx\" (UniqueName: \"kubernetes.io/projected/2129ac30-1645-4555-b722-55963ffb8776-kube-api-access-flqqx\") pod \"network-check-target-xzlsc\" (UID: \"2129ac30-1645-4555-b722-55963ffb8776\") " pod="openshift-network-diagnostics/network-check-target-xzlsc" Jul 03 11:12:57.796870 ip-10-0-136-0 kubenswrapper[2579]: E0703 11:12:57.796791 2579 projected.go:291] Couldn't get configMap openshift-network-diagnostics/kube-root-ca.crt: object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered Jul 03 11:12:57.796870 ip-10-0-136-0 kubenswrapper[2579]: E0703 11:12:57.796807 2579 projected.go:291] Couldn't get configMap openshift-network-diagnostics/openshift-service-ca.crt: object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered Jul 03 11:12:57.796870 ip-10-0-136-0 kubenswrapper[2579]: E0703 11:12:57.796820 2579 projected.go:196] Error preparing data for projected volume kube-api-access-flqqx for pod openshift-network-diagnostics/network-check-target-xzlsc: [object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered, object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered] Jul 03 11:12:57.797018 ip-10-0-136-0 kubenswrapper[2579]: E0703 11:12:57.796874 2579 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/projected/2129ac30-1645-4555-b722-55963ffb8776-kube-api-access-flqqx podName:2129ac30-1645-4555-b722-55963ffb8776 nodeName:}" failed. No retries permitted until 2026-07-03 11:13:29.796859173 +0000 UTC m=+66.385163479 (durationBeforeRetry 32s). Error: MountVolume.SetUp failed for volume "kube-api-access-flqqx" (UniqueName: "kubernetes.io/projected/2129ac30-1645-4555-b722-55963ffb8776-kube-api-access-flqqx") pod "network-check-target-xzlsc" (UID: "2129ac30-1645-4555-b722-55963ffb8776") : [object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered, object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered] Jul 03 11:12:58.985772 ip-10-0-136-0 kubenswrapper[2579]: I0703 11:12:58.985726 2579 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-xzlsc" Jul 03 11:12:58.986268 ip-10-0-136-0 kubenswrapper[2579]: I0703 11:12:58.985783 2579 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-sqxsj" Jul 03 11:12:58.986268 ip-10-0-136-0 kubenswrapper[2579]: I0703 11:12:58.985742 2579 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-9pqm8" Jul 03 11:12:58.986268 ip-10-0-136-0 kubenswrapper[2579]: E0703 11:12:58.985877 2579 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-xzlsc" podUID="2129ac30-1645-4555-b722-55963ffb8776" Jul 03 11:12:58.986268 ip-10-0-136-0 kubenswrapper[2579]: E0703 11:12:58.985941 2579 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-9pqm8" podUID="7813f158-0f67-411e-84d9-a2a1c51eea61" Jul 03 11:12:58.986268 ip-10-0-136-0 kubenswrapper[2579]: E0703 11:12:58.986017 2579 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-sqxsj" podUID="34ca9c51-4743-4efc-9aa1-eb79a741f6a4" Jul 03 11:12:59.643578 ip-10-0-136-0 kubenswrapper[2579]: I0703 11:12:59.643498 2579 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-136-0.ec2.internal" event="NodeReady" Jul 03 11:12:59.643755 ip-10-0-136-0 kubenswrapper[2579]: I0703 11:12:59.643661 2579 kubelet_node_status.go:489] "Fast updating node status as it just became ready" Jul 03 11:12:59.691304 ip-10-0-136-0 kubenswrapper[2579]: I0703 11:12:59.691256 2579 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-ingress-canary/ingress-canary-2nclq"] Jul 03 11:12:59.712863 ip-10-0-136-0 kubenswrapper[2579]: I0703 11:12:59.712824 2579 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-dns/dns-default-tksjt"] Jul 03 11:12:59.713050 ip-10-0-136-0 kubenswrapper[2579]: I0703 11:12:59.712961 2579 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress-canary/ingress-canary-2nclq" Jul 03 11:12:59.715498 ip-10-0-136-0 kubenswrapper[2579]: I0703 11:12:59.715463 2579 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress-canary\"/\"canary-serving-cert\"" Jul 03 11:12:59.715729 ip-10-0-136-0 kubenswrapper[2579]: I0703 11:12:59.715518 2579 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ingress-canary\"/\"openshift-service-ca.crt\"" Jul 03 11:12:59.715729 ip-10-0-136-0 kubenswrapper[2579]: I0703 11:12:59.715613 2579 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ingress-canary\"/\"kube-root-ca.crt\"" Jul 03 11:12:59.715880 ip-10-0-136-0 kubenswrapper[2579]: I0703 11:12:59.715857 2579 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress-canary\"/\"default-dockercfg-w5xwd\"" Jul 03 11:12:59.735062 ip-10-0-136-0 kubenswrapper[2579]: I0703 11:12:59.735026 2579 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-image-registry/image-registry-659bf79d58-p9gqp"] Jul 03 11:12:59.735256 ip-10-0-136-0 kubenswrapper[2579]: I0703 11:12:59.735067 2579 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-dns/dns-default-tksjt" Jul 03 11:12:59.737536 ip-10-0-136-0 kubenswrapper[2579]: I0703 11:12:59.737503 2579 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-dns\"/\"dns-default\"" Jul 03 11:12:59.737720 ip-10-0-136-0 kubenswrapper[2579]: I0703 11:12:59.737571 2579 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-dns\"/\"dns-dockercfg-kggc2\"" Jul 03 11:12:59.737720 ip-10-0-136-0 kubenswrapper[2579]: I0703 11:12:59.737503 2579 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-dns\"/\"dns-default-metrics-tls\"" Jul 03 11:12:59.760988 ip-10-0-136-0 kubenswrapper[2579]: I0703 11:12:59.760950 2579 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress-canary/ingress-canary-2nclq"] Jul 03 11:12:59.760988 ip-10-0-136-0 kubenswrapper[2579]: I0703 11:12:59.760998 2579 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-dns/dns-default-tksjt"] Jul 03 11:12:59.761250 ip-10-0-136-0 kubenswrapper[2579]: I0703 11:12:59.761025 2579 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-image-registry/image-registry-659bf79d58-p9gqp"] Jul 03 11:12:59.761250 ip-10-0-136-0 kubenswrapper[2579]: I0703 11:12:59.761098 2579 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-image-registry/image-registry-659bf79d58-p9gqp" Jul 03 11:12:59.764411 ip-10-0-136-0 kubenswrapper[2579]: I0703 11:12:59.763887 2579 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-image-registry\"/\"registry-dockercfg-58shg\"" Jul 03 11:12:59.764411 ip-10-0-136-0 kubenswrapper[2579]: I0703 11:12:59.763913 2579 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-image-registry\"/\"image-registry-tls\"" Jul 03 11:12:59.764411 ip-10-0-136-0 kubenswrapper[2579]: I0703 11:12:59.764005 2579 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-image-registry\"/\"image-registry-private-configuration\"" Jul 03 11:12:59.764411 ip-10-0-136-0 kubenswrapper[2579]: I0703 11:12:59.764276 2579 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-image-registry\"/\"installation-pull-secrets\"" Jul 03 11:12:59.769307 ip-10-0-136-0 kubenswrapper[2579]: I0703 11:12:59.769263 2579 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-image-registry\"/\"trusted-ca\"" Jul 03 11:12:59.813108 ip-10-0-136-0 kubenswrapper[2579]: I0703 11:12:59.813030 2579 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/266035ef-72fb-4911-9110-2806027e1770-tmp-dir\") pod \"dns-default-tksjt\" (UID: \"266035ef-72fb-4911-9110-2806027e1770\") " pod="openshift-dns/dns-default-tksjt" Jul 03 11:12:59.813308 ip-10-0-136-0 kubenswrapper[2579]: I0703 11:12:59.813125 2579 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-nmf6l\" (UniqueName: \"kubernetes.io/projected/868c9601-3c66-498e-8076-e7e289b9a4a0-kube-api-access-nmf6l\") pod \"ingress-canary-2nclq\" (UID: \"868c9601-3c66-498e-8076-e7e289b9a4a0\") " pod="openshift-ingress-canary/ingress-canary-2nclq" Jul 03 11:12:59.813308 ip-10-0-136-0 kubenswrapper[2579]: I0703 11:12:59.813147 2579 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"cert\" (UniqueName: \"kubernetes.io/secret/868c9601-3c66-498e-8076-e7e289b9a4a0-cert\") pod \"ingress-canary-2nclq\" (UID: \"868c9601-3c66-498e-8076-e7e289b9a4a0\") " pod="openshift-ingress-canary/ingress-canary-2nclq" Jul 03 11:12:59.813308 ip-10-0-136-0 kubenswrapper[2579]: I0703 11:12:59.813218 2579 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"config-volume\" (UniqueName: \"kubernetes.io/configmap/266035ef-72fb-4911-9110-2806027e1770-config-volume\") pod \"dns-default-tksjt\" (UID: \"266035ef-72fb-4911-9110-2806027e1770\") " pod="openshift-dns/dns-default-tksjt" Jul 03 11:12:59.813308 ip-10-0-136-0 kubenswrapper[2579]: I0703 11:12:59.813259 2579 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-5d4mh\" (UniqueName: \"kubernetes.io/projected/266035ef-72fb-4911-9110-2806027e1770-kube-api-access-5d4mh\") pod \"dns-default-tksjt\" (UID: \"266035ef-72fb-4911-9110-2806027e1770\") " pod="openshift-dns/dns-default-tksjt" Jul 03 11:12:59.813308 ip-10-0-136-0 kubenswrapper[2579]: I0703 11:12:59.813302 2579 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"metrics-tls\" (UniqueName: \"kubernetes.io/secret/266035ef-72fb-4911-9110-2806027e1770-metrics-tls\") pod \"dns-default-tksjt\" (UID: \"266035ef-72fb-4911-9110-2806027e1770\") " pod="openshift-dns/dns-default-tksjt" Jul 03 11:12:59.914354 ip-10-0-136-0 kubenswrapper[2579]: I0703 11:12:59.914262 2579 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-tls\" (UniqueName: \"kubernetes.io/secret/266035ef-72fb-4911-9110-2806027e1770-metrics-tls\") pod \"dns-default-tksjt\" (UID: \"266035ef-72fb-4911-9110-2806027e1770\") " pod="openshift-dns/dns-default-tksjt" Jul 03 11:12:59.914354 ip-10-0-136-0 kubenswrapper[2579]: I0703 11:12:59.914304 2579 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"bound-sa-token\" (UniqueName: \"kubernetes.io/projected/387cc87f-815e-457a-83cf-d016035fed5d-bound-sa-token\") pod \"image-registry-659bf79d58-p9gqp\" (UID: \"387cc87f-815e-457a-83cf-d016035fed5d\") " pod="openshift-image-registry/image-registry-659bf79d58-p9gqp" Jul 03 11:12:59.914354 ip-10-0-136-0 kubenswrapper[2579]: I0703 11:12:59.914346 2579 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/266035ef-72fb-4911-9110-2806027e1770-tmp-dir\") pod \"dns-default-tksjt\" (UID: \"266035ef-72fb-4911-9110-2806027e1770\") " pod="openshift-dns/dns-default-tksjt" Jul 03 11:12:59.914650 ip-10-0-136-0 kubenswrapper[2579]: I0703 11:12:59.914363 2579 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"registry-tls\" (UniqueName: \"kubernetes.io/projected/387cc87f-815e-457a-83cf-d016035fed5d-registry-tls\") pod \"image-registry-659bf79d58-p9gqp\" (UID: \"387cc87f-815e-457a-83cf-d016035fed5d\") " pod="openshift-image-registry/image-registry-659bf79d58-p9gqp" Jul 03 11:12:59.914650 ip-10-0-136-0 kubenswrapper[2579]: I0703 11:12:59.914383 2579 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-nmf6l\" (UniqueName: \"kubernetes.io/projected/868c9601-3c66-498e-8076-e7e289b9a4a0-kube-api-access-nmf6l\") pod \"ingress-canary-2nclq\" (UID: \"868c9601-3c66-498e-8076-e7e289b9a4a0\") " pod="openshift-ingress-canary/ingress-canary-2nclq" Jul 03 11:12:59.914650 ip-10-0-136-0 kubenswrapper[2579]: I0703 11:12:59.914411 2579 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"ca-trust-extracted\" (UniqueName: \"kubernetes.io/empty-dir/387cc87f-815e-457a-83cf-d016035fed5d-ca-trust-extracted\") pod \"image-registry-659bf79d58-p9gqp\" (UID: \"387cc87f-815e-457a-83cf-d016035fed5d\") " pod="openshift-image-registry/image-registry-659bf79d58-p9gqp" Jul 03 11:12:59.914650 ip-10-0-136-0 kubenswrapper[2579]: E0703 11:12:59.914440 2579 secret.go:189] Couldn't get secret openshift-dns/dns-default-metrics-tls: secret "dns-default-metrics-tls" not found Jul 03 11:12:59.914650 ip-10-0-136-0 kubenswrapper[2579]: I0703 11:12:59.914442 2579 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"cert\" (UniqueName: \"kubernetes.io/secret/868c9601-3c66-498e-8076-e7e289b9a4a0-cert\") pod \"ingress-canary-2nclq\" (UID: \"868c9601-3c66-498e-8076-e7e289b9a4a0\") " pod="openshift-ingress-canary/ingress-canary-2nclq" Jul 03 11:12:59.914650 ip-10-0-136-0 kubenswrapper[2579]: E0703 11:12:59.914541 2579 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/266035ef-72fb-4911-9110-2806027e1770-metrics-tls podName:266035ef-72fb-4911-9110-2806027e1770 nodeName:}" failed. No retries permitted until 2026-07-03 11:13:00.414507601 +0000 UTC m=+37.002811901 (durationBeforeRetry 500ms). Error: MountVolume.SetUp failed for volume "metrics-tls" (UniqueName: "kubernetes.io/secret/266035ef-72fb-4911-9110-2806027e1770-metrics-tls") pod "dns-default-tksjt" (UID: "266035ef-72fb-4911-9110-2806027e1770") : secret "dns-default-metrics-tls" not found Jul 03 11:12:59.914992 ip-10-0-136-0 kubenswrapper[2579]: I0703 11:12:59.914763 2579 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/266035ef-72fb-4911-9110-2806027e1770-tmp-dir\") pod \"dns-default-tksjt\" (UID: \"266035ef-72fb-4911-9110-2806027e1770\") " pod="openshift-dns/dns-default-tksjt" Jul 03 11:12:59.914992 ip-10-0-136-0 kubenswrapper[2579]: I0703 11:12:59.914825 2579 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"trusted-ca\" (UniqueName: \"kubernetes.io/configmap/387cc87f-815e-457a-83cf-d016035fed5d-trusted-ca\") pod \"image-registry-659bf79d58-p9gqp\" (UID: \"387cc87f-815e-457a-83cf-d016035fed5d\") " pod="openshift-image-registry/image-registry-659bf79d58-p9gqp" Jul 03 11:12:59.914992 ip-10-0-136-0 kubenswrapper[2579]: I0703 11:12:59.914870 2579 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"installation-pull-secrets\" (UniqueName: \"kubernetes.io/secret/387cc87f-815e-457a-83cf-d016035fed5d-installation-pull-secrets\") pod \"image-registry-659bf79d58-p9gqp\" (UID: \"387cc87f-815e-457a-83cf-d016035fed5d\") " pod="openshift-image-registry/image-registry-659bf79d58-p9gqp" Jul 03 11:12:59.914992 ip-10-0-136-0 kubenswrapper[2579]: I0703 11:12:59.914931 2579 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-9jp4s\" (UniqueName: \"kubernetes.io/projected/387cc87f-815e-457a-83cf-d016035fed5d-kube-api-access-9jp4s\") pod \"image-registry-659bf79d58-p9gqp\" (UID: \"387cc87f-815e-457a-83cf-d016035fed5d\") " pod="openshift-image-registry/image-registry-659bf79d58-p9gqp" Jul 03 11:12:59.914992 ip-10-0-136-0 kubenswrapper[2579]: I0703 11:12:59.914969 2579 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"config-volume\" (UniqueName: \"kubernetes.io/configmap/266035ef-72fb-4911-9110-2806027e1770-config-volume\") pod \"dns-default-tksjt\" (UID: \"266035ef-72fb-4911-9110-2806027e1770\") " pod="openshift-dns/dns-default-tksjt" Jul 03 11:12:59.915179 ip-10-0-136-0 kubenswrapper[2579]: I0703 11:12:59.915010 2579 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"registry-certificates\" (UniqueName: \"kubernetes.io/configmap/387cc87f-815e-457a-83cf-d016035fed5d-registry-certificates\") pod \"image-registry-659bf79d58-p9gqp\" (UID: \"387cc87f-815e-457a-83cf-d016035fed5d\") " pod="openshift-image-registry/image-registry-659bf79d58-p9gqp" Jul 03 11:12:59.915179 ip-10-0-136-0 kubenswrapper[2579]: I0703 11:12:59.915039 2579 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-5d4mh\" (UniqueName: \"kubernetes.io/projected/266035ef-72fb-4911-9110-2806027e1770-kube-api-access-5d4mh\") pod \"dns-default-tksjt\" (UID: \"266035ef-72fb-4911-9110-2806027e1770\") " pod="openshift-dns/dns-default-tksjt" Jul 03 11:12:59.915179 ip-10-0-136-0 kubenswrapper[2579]: I0703 11:12:59.915066 2579 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"image-registry-private-configuration\" (UniqueName: \"kubernetes.io/secret/387cc87f-815e-457a-83cf-d016035fed5d-image-registry-private-configuration\") pod \"image-registry-659bf79d58-p9gqp\" (UID: \"387cc87f-815e-457a-83cf-d016035fed5d\") " pod="openshift-image-registry/image-registry-659bf79d58-p9gqp" Jul 03 11:12:59.915540 ip-10-0-136-0 kubenswrapper[2579]: I0703 11:12:59.915517 2579 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"config-volume\" (UniqueName: \"kubernetes.io/configmap/266035ef-72fb-4911-9110-2806027e1770-config-volume\") pod \"dns-default-tksjt\" (UID: \"266035ef-72fb-4911-9110-2806027e1770\") " pod="openshift-dns/dns-default-tksjt" Jul 03 11:12:59.920403 ip-10-0-136-0 kubenswrapper[2579]: I0703 11:12:59.920365 2579 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"cert\" (UniqueName: \"kubernetes.io/secret/868c9601-3c66-498e-8076-e7e289b9a4a0-cert\") pod \"ingress-canary-2nclq\" (UID: \"868c9601-3c66-498e-8076-e7e289b9a4a0\") " pod="openshift-ingress-canary/ingress-canary-2nclq" Jul 03 11:12:59.924155 ip-10-0-136-0 kubenswrapper[2579]: I0703 11:12:59.923913 2579 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-5d4mh\" (UniqueName: \"kubernetes.io/projected/266035ef-72fb-4911-9110-2806027e1770-kube-api-access-5d4mh\") pod \"dns-default-tksjt\" (UID: \"266035ef-72fb-4911-9110-2806027e1770\") " pod="openshift-dns/dns-default-tksjt" Jul 03 11:12:59.924155 ip-10-0-136-0 kubenswrapper[2579]: I0703 11:12:59.924103 2579 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-nmf6l\" (UniqueName: \"kubernetes.io/projected/868c9601-3c66-498e-8076-e7e289b9a4a0-kube-api-access-nmf6l\") pod \"ingress-canary-2nclq\" (UID: \"868c9601-3c66-498e-8076-e7e289b9a4a0\") " pod="openshift-ingress-canary/ingress-canary-2nclq" Jul 03 11:13:00.015755 ip-10-0-136-0 kubenswrapper[2579]: I0703 11:13:00.015710 2579 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"registry-tls\" (UniqueName: \"kubernetes.io/projected/387cc87f-815e-457a-83cf-d016035fed5d-registry-tls\") pod \"image-registry-659bf79d58-p9gqp\" (UID: \"387cc87f-815e-457a-83cf-d016035fed5d\") " pod="openshift-image-registry/image-registry-659bf79d58-p9gqp" Jul 03 11:13:00.016363 ip-10-0-136-0 kubenswrapper[2579]: I0703 11:13:00.015764 2579 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"ca-trust-extracted\" (UniqueName: \"kubernetes.io/empty-dir/387cc87f-815e-457a-83cf-d016035fed5d-ca-trust-extracted\") pod \"image-registry-659bf79d58-p9gqp\" (UID: \"387cc87f-815e-457a-83cf-d016035fed5d\") " pod="openshift-image-registry/image-registry-659bf79d58-p9gqp" Jul 03 11:13:00.016363 ip-10-0-136-0 kubenswrapper[2579]: I0703 11:13:00.015794 2579 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"trusted-ca\" (UniqueName: \"kubernetes.io/configmap/387cc87f-815e-457a-83cf-d016035fed5d-trusted-ca\") pod \"image-registry-659bf79d58-p9gqp\" (UID: \"387cc87f-815e-457a-83cf-d016035fed5d\") " pod="openshift-image-registry/image-registry-659bf79d58-p9gqp" Jul 03 11:13:00.016363 ip-10-0-136-0 kubenswrapper[2579]: I0703 11:13:00.015817 2579 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"installation-pull-secrets\" (UniqueName: \"kubernetes.io/secret/387cc87f-815e-457a-83cf-d016035fed5d-installation-pull-secrets\") pod \"image-registry-659bf79d58-p9gqp\" (UID: \"387cc87f-815e-457a-83cf-d016035fed5d\") " pod="openshift-image-registry/image-registry-659bf79d58-p9gqp" Jul 03 11:13:00.016363 ip-10-0-136-0 kubenswrapper[2579]: E0703 11:13:00.015887 2579 projected.go:266] Couldn't get secret openshift-image-registry/image-registry-tls: secret "image-registry-tls" not found Jul 03 11:13:00.016363 ip-10-0-136-0 kubenswrapper[2579]: E0703 11:13:00.015911 2579 projected.go:196] Error preparing data for projected volume registry-tls for pod openshift-image-registry/image-registry-659bf79d58-p9gqp: secret "image-registry-tls" not found Jul 03 11:13:00.016363 ip-10-0-136-0 kubenswrapper[2579]: E0703 11:13:00.015981 2579 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/projected/387cc87f-815e-457a-83cf-d016035fed5d-registry-tls podName:387cc87f-815e-457a-83cf-d016035fed5d nodeName:}" failed. No retries permitted until 2026-07-03 11:13:00.51595607 +0000 UTC m=+37.104260369 (durationBeforeRetry 500ms). Error: MountVolume.SetUp failed for volume "registry-tls" (UniqueName: "kubernetes.io/projected/387cc87f-815e-457a-83cf-d016035fed5d-registry-tls") pod "image-registry-659bf79d58-p9gqp" (UID: "387cc87f-815e-457a-83cf-d016035fed5d") : secret "image-registry-tls" not found Jul 03 11:13:00.016363 ip-10-0-136-0 kubenswrapper[2579]: I0703 11:13:00.016038 2579 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-9jp4s\" (UniqueName: \"kubernetes.io/projected/387cc87f-815e-457a-83cf-d016035fed5d-kube-api-access-9jp4s\") pod \"image-registry-659bf79d58-p9gqp\" (UID: \"387cc87f-815e-457a-83cf-d016035fed5d\") " pod="openshift-image-registry/image-registry-659bf79d58-p9gqp" Jul 03 11:13:00.016363 ip-10-0-136-0 kubenswrapper[2579]: I0703 11:13:00.016117 2579 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"registry-certificates\" (UniqueName: \"kubernetes.io/configmap/387cc87f-815e-457a-83cf-d016035fed5d-registry-certificates\") pod \"image-registry-659bf79d58-p9gqp\" (UID: \"387cc87f-815e-457a-83cf-d016035fed5d\") " pod="openshift-image-registry/image-registry-659bf79d58-p9gqp" Jul 03 11:13:00.016363 ip-10-0-136-0 kubenswrapper[2579]: I0703 11:13:00.016147 2579 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"image-registry-private-configuration\" (UniqueName: \"kubernetes.io/secret/387cc87f-815e-457a-83cf-d016035fed5d-image-registry-private-configuration\") pod \"image-registry-659bf79d58-p9gqp\" (UID: \"387cc87f-815e-457a-83cf-d016035fed5d\") " pod="openshift-image-registry/image-registry-659bf79d58-p9gqp" Jul 03 11:13:00.016363 ip-10-0-136-0 kubenswrapper[2579]: I0703 11:13:00.016358 2579 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"bound-sa-token\" (UniqueName: \"kubernetes.io/projected/387cc87f-815e-457a-83cf-d016035fed5d-bound-sa-token\") pod \"image-registry-659bf79d58-p9gqp\" (UID: \"387cc87f-815e-457a-83cf-d016035fed5d\") " pod="openshift-image-registry/image-registry-659bf79d58-p9gqp" Jul 03 11:13:00.016773 ip-10-0-136-0 kubenswrapper[2579]: I0703 11:13:00.016442 2579 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"ca-trust-extracted\" (UniqueName: \"kubernetes.io/empty-dir/387cc87f-815e-457a-83cf-d016035fed5d-ca-trust-extracted\") pod \"image-registry-659bf79d58-p9gqp\" (UID: \"387cc87f-815e-457a-83cf-d016035fed5d\") " pod="openshift-image-registry/image-registry-659bf79d58-p9gqp" Jul 03 11:13:00.016826 ip-10-0-136-0 kubenswrapper[2579]: I0703 11:13:00.016775 2579 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"registry-certificates\" (UniqueName: \"kubernetes.io/configmap/387cc87f-815e-457a-83cf-d016035fed5d-registry-certificates\") pod \"image-registry-659bf79d58-p9gqp\" (UID: \"387cc87f-815e-457a-83cf-d016035fed5d\") " pod="openshift-image-registry/image-registry-659bf79d58-p9gqp" Jul 03 11:13:00.017235 ip-10-0-136-0 kubenswrapper[2579]: I0703 11:13:00.017180 2579 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"trusted-ca\" (UniqueName: \"kubernetes.io/configmap/387cc87f-815e-457a-83cf-d016035fed5d-trusted-ca\") pod \"image-registry-659bf79d58-p9gqp\" (UID: \"387cc87f-815e-457a-83cf-d016035fed5d\") " pod="openshift-image-registry/image-registry-659bf79d58-p9gqp" Jul 03 11:13:00.019032 ip-10-0-136-0 kubenswrapper[2579]: I0703 11:13:00.019003 2579 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"image-registry-private-configuration\" (UniqueName: \"kubernetes.io/secret/387cc87f-815e-457a-83cf-d016035fed5d-image-registry-private-configuration\") pod \"image-registry-659bf79d58-p9gqp\" (UID: \"387cc87f-815e-457a-83cf-d016035fed5d\") " pod="openshift-image-registry/image-registry-659bf79d58-p9gqp" Jul 03 11:13:00.019417 ip-10-0-136-0 kubenswrapper[2579]: I0703 11:13:00.019391 2579 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"installation-pull-secrets\" (UniqueName: \"kubernetes.io/secret/387cc87f-815e-457a-83cf-d016035fed5d-installation-pull-secrets\") pod \"image-registry-659bf79d58-p9gqp\" (UID: \"387cc87f-815e-457a-83cf-d016035fed5d\") " pod="openshift-image-registry/image-registry-659bf79d58-p9gqp" Jul 03 11:13:00.024960 ip-10-0-136-0 kubenswrapper[2579]: I0703 11:13:00.024916 2579 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress-canary/ingress-canary-2nclq" Jul 03 11:13:00.027761 ip-10-0-136-0 kubenswrapper[2579]: I0703 11:13:00.027694 2579 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-9jp4s\" (UniqueName: \"kubernetes.io/projected/387cc87f-815e-457a-83cf-d016035fed5d-kube-api-access-9jp4s\") pod \"image-registry-659bf79d58-p9gqp\" (UID: \"387cc87f-815e-457a-83cf-d016035fed5d\") " pod="openshift-image-registry/image-registry-659bf79d58-p9gqp" Jul 03 11:13:00.027925 ip-10-0-136-0 kubenswrapper[2579]: I0703 11:13:00.027855 2579 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"bound-sa-token\" (UniqueName: \"kubernetes.io/projected/387cc87f-815e-457a-83cf-d016035fed5d-bound-sa-token\") pod \"image-registry-659bf79d58-p9gqp\" (UID: \"387cc87f-815e-457a-83cf-d016035fed5d\") " pod="openshift-image-registry/image-registry-659bf79d58-p9gqp" Jul 03 11:13:00.248000 ip-10-0-136-0 kubenswrapper[2579]: I0703 11:13:00.247816 2579 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress-canary/ingress-canary-2nclq"] Jul 03 11:13:00.251678 ip-10-0-136-0 kubenswrapper[2579]: W0703 11:13:00.251654 2579 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod868c9601_3c66_498e_8076_e7e289b9a4a0.slice/crio-84a49350e4fcae68021caf84ce58b1b8bb6478200d1e5d496470fec14fbc8e92 WatchSource:0}: Error finding container 84a49350e4fcae68021caf84ce58b1b8bb6478200d1e5d496470fec14fbc8e92: Status 404 returned error can't find the container with id 84a49350e4fcae68021caf84ce58b1b8bb6478200d1e5d496470fec14fbc8e92 Jul 03 11:13:00.308531 ip-10-0-136-0 kubenswrapper[2579]: I0703 11:13:00.308484 2579 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress-canary/ingress-canary-2nclq" event={"ID":"868c9601-3c66-498e-8076-e7e289b9a4a0","Type":"ContainerStarted","Data":"84a49350e4fcae68021caf84ce58b1b8bb6478200d1e5d496470fec14fbc8e92"} Jul 03 11:13:00.420894 ip-10-0-136-0 kubenswrapper[2579]: I0703 11:13:00.420850 2579 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-tls\" (UniqueName: \"kubernetes.io/secret/266035ef-72fb-4911-9110-2806027e1770-metrics-tls\") pod \"dns-default-tksjt\" (UID: \"266035ef-72fb-4911-9110-2806027e1770\") " pod="openshift-dns/dns-default-tksjt" Jul 03 11:13:00.421091 ip-10-0-136-0 kubenswrapper[2579]: E0703 11:13:00.421027 2579 secret.go:189] Couldn't get secret openshift-dns/dns-default-metrics-tls: secret "dns-default-metrics-tls" not found Jul 03 11:13:00.421152 ip-10-0-136-0 kubenswrapper[2579]: E0703 11:13:00.421105 2579 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/266035ef-72fb-4911-9110-2806027e1770-metrics-tls podName:266035ef-72fb-4911-9110-2806027e1770 nodeName:}" failed. No retries permitted until 2026-07-03 11:13:01.42108309 +0000 UTC m=+38.009387388 (durationBeforeRetry 1s). Error: MountVolume.SetUp failed for volume "metrics-tls" (UniqueName: "kubernetes.io/secret/266035ef-72fb-4911-9110-2806027e1770-metrics-tls") pod "dns-default-tksjt" (UID: "266035ef-72fb-4911-9110-2806027e1770") : secret "dns-default-metrics-tls" not found Jul 03 11:13:00.522147 ip-10-0-136-0 kubenswrapper[2579]: I0703 11:13:00.522043 2579 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"registry-tls\" (UniqueName: \"kubernetes.io/projected/387cc87f-815e-457a-83cf-d016035fed5d-registry-tls\") pod \"image-registry-659bf79d58-p9gqp\" (UID: \"387cc87f-815e-457a-83cf-d016035fed5d\") " pod="openshift-image-registry/image-registry-659bf79d58-p9gqp" Jul 03 11:13:00.522297 ip-10-0-136-0 kubenswrapper[2579]: E0703 11:13:00.522249 2579 projected.go:266] Couldn't get secret openshift-image-registry/image-registry-tls: secret "image-registry-tls" not found Jul 03 11:13:00.522297 ip-10-0-136-0 kubenswrapper[2579]: E0703 11:13:00.522275 2579 projected.go:196] Error preparing data for projected volume registry-tls for pod openshift-image-registry/image-registry-659bf79d58-p9gqp: secret "image-registry-tls" not found Jul 03 11:13:00.522395 ip-10-0-136-0 kubenswrapper[2579]: E0703 11:13:00.522350 2579 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/projected/387cc87f-815e-457a-83cf-d016035fed5d-registry-tls podName:387cc87f-815e-457a-83cf-d016035fed5d nodeName:}" failed. No retries permitted until 2026-07-03 11:13:01.522328027 +0000 UTC m=+38.110632327 (durationBeforeRetry 1s). Error: MountVolume.SetUp failed for volume "registry-tls" (UniqueName: "kubernetes.io/projected/387cc87f-815e-457a-83cf-d016035fed5d-registry-tls") pod "image-registry-659bf79d58-p9gqp" (UID: "387cc87f-815e-457a-83cf-d016035fed5d") : secret "image-registry-tls" not found Jul 03 11:13:00.985653 ip-10-0-136-0 kubenswrapper[2579]: I0703 11:13:00.985559 2579 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-9pqm8" Jul 03 11:13:00.985830 ip-10-0-136-0 kubenswrapper[2579]: I0703 11:13:00.985558 2579 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-sqxsj" Jul 03 11:13:00.985830 ip-10-0-136-0 kubenswrapper[2579]: I0703 11:13:00.985558 2579 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-xzlsc" Jul 03 11:13:00.988357 ip-10-0-136-0 kubenswrapper[2579]: I0703 11:13:00.988324 2579 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-multus\"/\"metrics-daemon-secret\"" Jul 03 11:13:00.988357 ip-10-0-136-0 kubenswrapper[2579]: I0703 11:13:00.988356 2579 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-network-diagnostics\"/\"openshift-service-ca.crt\"" Jul 03 11:13:00.989390 ip-10-0-136-0 kubenswrapper[2579]: I0703 11:13:00.989362 2579 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-network-diagnostics\"/\"kube-root-ca.crt\"" Jul 03 11:13:00.989536 ip-10-0-136-0 kubenswrapper[2579]: I0703 11:13:00.989399 2579 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"kube-system\"/\"original-pull-secret\"" Jul 03 11:13:00.989536 ip-10-0-136-0 kubenswrapper[2579]: I0703 11:13:00.989431 2579 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-network-diagnostics\"/\"default-dockercfg-wfk2t\"" Jul 03 11:13:00.989536 ip-10-0-136-0 kubenswrapper[2579]: I0703 11:13:00.989459 2579 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-multus\"/\"metrics-daemon-sa-dockercfg-qx6zp\"" Jul 03 11:13:01.431445 ip-10-0-136-0 kubenswrapper[2579]: I0703 11:13:01.431343 2579 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-tls\" (UniqueName: \"kubernetes.io/secret/266035ef-72fb-4911-9110-2806027e1770-metrics-tls\") pod \"dns-default-tksjt\" (UID: \"266035ef-72fb-4911-9110-2806027e1770\") " pod="openshift-dns/dns-default-tksjt" Jul 03 11:13:01.431877 ip-10-0-136-0 kubenswrapper[2579]: E0703 11:13:01.431517 2579 secret.go:189] Couldn't get secret openshift-dns/dns-default-metrics-tls: secret "dns-default-metrics-tls" not found Jul 03 11:13:01.431877 ip-10-0-136-0 kubenswrapper[2579]: E0703 11:13:01.431585 2579 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/266035ef-72fb-4911-9110-2806027e1770-metrics-tls podName:266035ef-72fb-4911-9110-2806027e1770 nodeName:}" failed. No retries permitted until 2026-07-03 11:13:03.43156924 +0000 UTC m=+40.019873538 (durationBeforeRetry 2s). Error: MountVolume.SetUp failed for volume "metrics-tls" (UniqueName: "kubernetes.io/secret/266035ef-72fb-4911-9110-2806027e1770-metrics-tls") pod "dns-default-tksjt" (UID: "266035ef-72fb-4911-9110-2806027e1770") : secret "dns-default-metrics-tls" not found Jul 03 11:13:01.532159 ip-10-0-136-0 kubenswrapper[2579]: I0703 11:13:01.532102 2579 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"registry-tls\" (UniqueName: \"kubernetes.io/projected/387cc87f-815e-457a-83cf-d016035fed5d-registry-tls\") pod \"image-registry-659bf79d58-p9gqp\" (UID: \"387cc87f-815e-457a-83cf-d016035fed5d\") " pod="openshift-image-registry/image-registry-659bf79d58-p9gqp" Jul 03 11:13:01.532349 ip-10-0-136-0 kubenswrapper[2579]: E0703 11:13:01.532292 2579 projected.go:266] Couldn't get secret openshift-image-registry/image-registry-tls: secret "image-registry-tls" not found Jul 03 11:13:01.532349 ip-10-0-136-0 kubenswrapper[2579]: E0703 11:13:01.532313 2579 projected.go:196] Error preparing data for projected volume registry-tls for pod openshift-image-registry/image-registry-659bf79d58-p9gqp: secret "image-registry-tls" not found Jul 03 11:13:01.532444 ip-10-0-136-0 kubenswrapper[2579]: E0703 11:13:01.532373 2579 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/projected/387cc87f-815e-457a-83cf-d016035fed5d-registry-tls podName:387cc87f-815e-457a-83cf-d016035fed5d nodeName:}" failed. No retries permitted until 2026-07-03 11:13:03.532355476 +0000 UTC m=+40.120659791 (durationBeforeRetry 2s). Error: MountVolume.SetUp failed for volume "registry-tls" (UniqueName: "kubernetes.io/projected/387cc87f-815e-457a-83cf-d016035fed5d-registry-tls") pod "image-registry-659bf79d58-p9gqp" (UID: "387cc87f-815e-457a-83cf-d016035fed5d") : secret "image-registry-tls" not found Jul 03 11:13:02.317802 ip-10-0-136-0 kubenswrapper[2579]: I0703 11:13:02.317637 2579 generic.go:356] "Generic (PLEG): container finished" podID="e9fc3c32-a741-4377-970b-6cb08e5b4ac9" containerID="3cc5a51b3e81f2dc4307a8575da75de63f45bc0c63b14764bd5fe368498ec32f" exitCode=0 Jul 03 11:13:02.317802 ip-10-0-136-0 kubenswrapper[2579]: I0703 11:13:02.317684 2579 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-multus/multus-additional-cni-plugins-52n9r" event={"ID":"e9fc3c32-a741-4377-970b-6cb08e5b4ac9","Type":"ContainerDied","Data":"3cc5a51b3e81f2dc4307a8575da75de63f45bc0c63b14764bd5fe368498ec32f"} Jul 03 11:13:03.323636 ip-10-0-136-0 kubenswrapper[2579]: I0703 11:13:03.323598 2579 generic.go:356] "Generic (PLEG): container finished" podID="e9fc3c32-a741-4377-970b-6cb08e5b4ac9" containerID="b5a6c6e9fad92261393b87650bb6cb0b0c3a3e9e640844b492a7cffbde8b2230" exitCode=0 Jul 03 11:13:03.324127 ip-10-0-136-0 kubenswrapper[2579]: I0703 11:13:03.323654 2579 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-multus/multus-additional-cni-plugins-52n9r" event={"ID":"e9fc3c32-a741-4377-970b-6cb08e5b4ac9","Type":"ContainerDied","Data":"b5a6c6e9fad92261393b87650bb6cb0b0c3a3e9e640844b492a7cffbde8b2230"} Jul 03 11:13:03.448022 ip-10-0-136-0 kubenswrapper[2579]: I0703 11:13:03.447982 2579 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-tls\" (UniqueName: \"kubernetes.io/secret/266035ef-72fb-4911-9110-2806027e1770-metrics-tls\") pod \"dns-default-tksjt\" (UID: \"266035ef-72fb-4911-9110-2806027e1770\") " pod="openshift-dns/dns-default-tksjt" Jul 03 11:13:03.448262 ip-10-0-136-0 kubenswrapper[2579]: E0703 11:13:03.448134 2579 secret.go:189] Couldn't get secret openshift-dns/dns-default-metrics-tls: secret "dns-default-metrics-tls" not found Jul 03 11:13:03.448262 ip-10-0-136-0 kubenswrapper[2579]: E0703 11:13:03.448230 2579 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/266035ef-72fb-4911-9110-2806027e1770-metrics-tls podName:266035ef-72fb-4911-9110-2806027e1770 nodeName:}" failed. No retries permitted until 2026-07-03 11:13:07.448211057 +0000 UTC m=+44.036515380 (durationBeforeRetry 4s). Error: MountVolume.SetUp failed for volume "metrics-tls" (UniqueName: "kubernetes.io/secret/266035ef-72fb-4911-9110-2806027e1770-metrics-tls") pod "dns-default-tksjt" (UID: "266035ef-72fb-4911-9110-2806027e1770") : secret "dns-default-metrics-tls" not found Jul 03 11:13:03.549095 ip-10-0-136-0 kubenswrapper[2579]: I0703 11:13:03.549050 2579 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"registry-tls\" (UniqueName: \"kubernetes.io/projected/387cc87f-815e-457a-83cf-d016035fed5d-registry-tls\") pod \"image-registry-659bf79d58-p9gqp\" (UID: \"387cc87f-815e-457a-83cf-d016035fed5d\") " pod="openshift-image-registry/image-registry-659bf79d58-p9gqp" Jul 03 11:13:03.549310 ip-10-0-136-0 kubenswrapper[2579]: E0703 11:13:03.549265 2579 projected.go:266] Couldn't get secret openshift-image-registry/image-registry-tls: secret "image-registry-tls" not found Jul 03 11:13:03.549310 ip-10-0-136-0 kubenswrapper[2579]: E0703 11:13:03.549289 2579 projected.go:196] Error preparing data for projected volume registry-tls for pod openshift-image-registry/image-registry-659bf79d58-p9gqp: secret "image-registry-tls" not found Jul 03 11:13:03.549426 ip-10-0-136-0 kubenswrapper[2579]: E0703 11:13:03.549371 2579 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/projected/387cc87f-815e-457a-83cf-d016035fed5d-registry-tls podName:387cc87f-815e-457a-83cf-d016035fed5d nodeName:}" failed. No retries permitted until 2026-07-03 11:13:07.54934891 +0000 UTC m=+44.137653226 (durationBeforeRetry 4s). Error: MountVolume.SetUp failed for volume "registry-tls" (UniqueName: "kubernetes.io/projected/387cc87f-815e-457a-83cf-d016035fed5d-registry-tls") pod "image-registry-659bf79d58-p9gqp" (UID: "387cc87f-815e-457a-83cf-d016035fed5d") : secret "image-registry-tls" not found Jul 03 11:13:04.157709 ip-10-0-136-0 kubenswrapper[2579]: I0703 11:13:04.157677 2579 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-insights/insights-runtime-extractor-ws6w7"] Jul 03 11:13:04.160406 ip-10-0-136-0 kubenswrapper[2579]: I0703 11:13:04.160388 2579 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-insights/insights-runtime-extractor-ws6w7" Jul 03 11:13:04.164073 ip-10-0-136-0 kubenswrapper[2579]: I0703 11:13:04.164049 2579 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-insights\"/\"insights-runtime-extractor-sa-dockercfg-d5hq9\"" Jul 03 11:13:04.164329 ip-10-0-136-0 kubenswrapper[2579]: I0703 11:13:04.164307 2579 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-insights\"/\"insights-runtime-extractor-tls\"" Jul 03 11:13:04.164439 ip-10-0-136-0 kubenswrapper[2579]: I0703 11:13:04.164337 2579 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-insights\"/\"kube-root-ca.crt\"" Jul 03 11:13:04.164439 ip-10-0-136-0 kubenswrapper[2579]: I0703 11:13:04.164312 2579 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-insights\"/\"openshift-service-ca.crt\"" Jul 03 11:13:04.164932 ip-10-0-136-0 kubenswrapper[2579]: I0703 11:13:04.164915 2579 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-insights\"/\"kube-rbac-proxy\"" Jul 03 11:13:04.184836 ip-10-0-136-0 kubenswrapper[2579]: I0703 11:13:04.184803 2579 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-insights/insights-runtime-extractor-ws6w7"] Jul 03 11:13:04.254034 ip-10-0-136-0 kubenswrapper[2579]: I0703 11:13:04.253992 2579 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"data-volume\" (UniqueName: \"kubernetes.io/empty-dir/f5f5abfc-f7a4-4d0b-be33-bb9dc442904d-data-volume\") pod \"insights-runtime-extractor-ws6w7\" (UID: \"f5f5abfc-f7a4-4d0b-be33-bb9dc442904d\") " pod="openshift-insights/insights-runtime-extractor-ws6w7" Jul 03 11:13:04.254034 ip-10-0-136-0 kubenswrapper[2579]: I0703 11:13:04.254041 2579 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-rbac-proxy-cm\" (UniqueName: \"kubernetes.io/configmap/f5f5abfc-f7a4-4d0b-be33-bb9dc442904d-kube-rbac-proxy-cm\") pod \"insights-runtime-extractor-ws6w7\" (UID: \"f5f5abfc-f7a4-4d0b-be33-bb9dc442904d\") " pod="openshift-insights/insights-runtime-extractor-ws6w7" Jul 03 11:13:04.254299 ip-10-0-136-0 kubenswrapper[2579]: I0703 11:13:04.254075 2579 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"insights-runtime-extractor-tls\" (UniqueName: \"kubernetes.io/secret/f5f5abfc-f7a4-4d0b-be33-bb9dc442904d-insights-runtime-extractor-tls\") pod \"insights-runtime-extractor-ws6w7\" (UID: \"f5f5abfc-f7a4-4d0b-be33-bb9dc442904d\") " pod="openshift-insights/insights-runtime-extractor-ws6w7" Jul 03 11:13:04.254299 ip-10-0-136-0 kubenswrapper[2579]: I0703 11:13:04.254131 2579 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"crio-socket\" (UniqueName: \"kubernetes.io/host-path/f5f5abfc-f7a4-4d0b-be33-bb9dc442904d-crio-socket\") pod \"insights-runtime-extractor-ws6w7\" (UID: \"f5f5abfc-f7a4-4d0b-be33-bb9dc442904d\") " pod="openshift-insights/insights-runtime-extractor-ws6w7" Jul 03 11:13:04.254299 ip-10-0-136-0 kubenswrapper[2579]: I0703 11:13:04.254151 2579 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-dtrbk\" (UniqueName: \"kubernetes.io/projected/f5f5abfc-f7a4-4d0b-be33-bb9dc442904d-kube-api-access-dtrbk\") pod \"insights-runtime-extractor-ws6w7\" (UID: \"f5f5abfc-f7a4-4d0b-be33-bb9dc442904d\") " pod="openshift-insights/insights-runtime-extractor-ws6w7" Jul 03 11:13:04.327587 ip-10-0-136-0 kubenswrapper[2579]: I0703 11:13:04.327479 2579 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress-canary/ingress-canary-2nclq" event={"ID":"868c9601-3c66-498e-8076-e7e289b9a4a0","Type":"ContainerStarted","Data":"140d54ad2c99ebc0b4fdbd3ccf6edac04d6014aaa9ad252035c2ac4a44cebf82"} Jul 03 11:13:04.331066 ip-10-0-136-0 kubenswrapper[2579]: I0703 11:13:04.331025 2579 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-multus/multus-additional-cni-plugins-52n9r" event={"ID":"e9fc3c32-a741-4377-970b-6cb08e5b4ac9","Type":"ContainerStarted","Data":"8b8bc39d56c2dc99e5ae51400a44618803c386e79babf47fff5cdcdb1f6d2a09"} Jul 03 11:13:04.344305 ip-10-0-136-0 kubenswrapper[2579]: I0703 11:13:04.344247 2579 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-ingress-canary/ingress-canary-2nclq" podStartSLOduration=1.53668515 podStartE2EDuration="5.344224765s" totalImagesPullingTime="3.807539615s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-03 11:12:59 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-03 11:13:04.342947607 +0000 UTC m=+40.931251926" watchObservedRunningTime="2026-07-03 11:13:04.344224765 +0000 UTC m=+40.932529063" Jul 03 11:13:04.354524 ip-10-0-136-0 kubenswrapper[2579]: I0703 11:13:04.354491 2579 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"data-volume\" (UniqueName: \"kubernetes.io/empty-dir/f5f5abfc-f7a4-4d0b-be33-bb9dc442904d-data-volume\") pod \"insights-runtime-extractor-ws6w7\" (UID: \"f5f5abfc-f7a4-4d0b-be33-bb9dc442904d\") " pod="openshift-insights/insights-runtime-extractor-ws6w7" Jul 03 11:13:04.354704 ip-10-0-136-0 kubenswrapper[2579]: I0703 11:13:04.354554 2579 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-rbac-proxy-cm\" (UniqueName: \"kubernetes.io/configmap/f5f5abfc-f7a4-4d0b-be33-bb9dc442904d-kube-rbac-proxy-cm\") pod \"insights-runtime-extractor-ws6w7\" (UID: \"f5f5abfc-f7a4-4d0b-be33-bb9dc442904d\") " pod="openshift-insights/insights-runtime-extractor-ws6w7" Jul 03 11:13:04.354704 ip-10-0-136-0 kubenswrapper[2579]: I0703 11:13:04.354594 2579 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"insights-runtime-extractor-tls\" (UniqueName: \"kubernetes.io/secret/f5f5abfc-f7a4-4d0b-be33-bb9dc442904d-insights-runtime-extractor-tls\") pod \"insights-runtime-extractor-ws6w7\" (UID: \"f5f5abfc-f7a4-4d0b-be33-bb9dc442904d\") " pod="openshift-insights/insights-runtime-extractor-ws6w7" Jul 03 11:13:04.354704 ip-10-0-136-0 kubenswrapper[2579]: I0703 11:13:04.354638 2579 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"crio-socket\" (UniqueName: \"kubernetes.io/host-path/f5f5abfc-f7a4-4d0b-be33-bb9dc442904d-crio-socket\") pod \"insights-runtime-extractor-ws6w7\" (UID: \"f5f5abfc-f7a4-4d0b-be33-bb9dc442904d\") " pod="openshift-insights/insights-runtime-extractor-ws6w7" Jul 03 11:13:04.354704 ip-10-0-136-0 kubenswrapper[2579]: I0703 11:13:04.354659 2579 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-dtrbk\" (UniqueName: \"kubernetes.io/projected/f5f5abfc-f7a4-4d0b-be33-bb9dc442904d-kube-api-access-dtrbk\") pod \"insights-runtime-extractor-ws6w7\" (UID: \"f5f5abfc-f7a4-4d0b-be33-bb9dc442904d\") " pod="openshift-insights/insights-runtime-extractor-ws6w7" Jul 03 11:13:04.354954 ip-10-0-136-0 kubenswrapper[2579]: E0703 11:13:04.354760 2579 secret.go:189] Couldn't get secret openshift-insights/insights-runtime-extractor-tls: secret "insights-runtime-extractor-tls" not found Jul 03 11:13:04.354954 ip-10-0-136-0 kubenswrapper[2579]: I0703 11:13:04.354827 2579 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"crio-socket\" (UniqueName: \"kubernetes.io/host-path/f5f5abfc-f7a4-4d0b-be33-bb9dc442904d-crio-socket\") pod \"insights-runtime-extractor-ws6w7\" (UID: \"f5f5abfc-f7a4-4d0b-be33-bb9dc442904d\") " pod="openshift-insights/insights-runtime-extractor-ws6w7" Jul 03 11:13:04.354954 ip-10-0-136-0 kubenswrapper[2579]: E0703 11:13:04.354760 2579 configmap.go:193] Couldn't get configMap openshift-insights/kube-rbac-proxy: configmap "kube-rbac-proxy" not found Jul 03 11:13:04.354954 ip-10-0-136-0 kubenswrapper[2579]: I0703 11:13:04.354871 2579 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"data-volume\" (UniqueName: \"kubernetes.io/empty-dir/f5f5abfc-f7a4-4d0b-be33-bb9dc442904d-data-volume\") pod \"insights-runtime-extractor-ws6w7\" (UID: \"f5f5abfc-f7a4-4d0b-be33-bb9dc442904d\") " pod="openshift-insights/insights-runtime-extractor-ws6w7" Jul 03 11:13:04.354954 ip-10-0-136-0 kubenswrapper[2579]: E0703 11:13:04.354843 2579 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/f5f5abfc-f7a4-4d0b-be33-bb9dc442904d-insights-runtime-extractor-tls podName:f5f5abfc-f7a4-4d0b-be33-bb9dc442904d nodeName:}" failed. No retries permitted until 2026-07-03 11:13:04.85482001 +0000 UTC m=+41.443124360 (durationBeforeRetry 500ms). Error: MountVolume.SetUp failed for volume "insights-runtime-extractor-tls" (UniqueName: "kubernetes.io/secret/f5f5abfc-f7a4-4d0b-be33-bb9dc442904d-insights-runtime-extractor-tls") pod "insights-runtime-extractor-ws6w7" (UID: "f5f5abfc-f7a4-4d0b-be33-bb9dc442904d") : secret "insights-runtime-extractor-tls" not found Jul 03 11:13:04.354954 ip-10-0-136-0 kubenswrapper[2579]: E0703 11:13:04.354921 2579 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/configmap/f5f5abfc-f7a4-4d0b-be33-bb9dc442904d-kube-rbac-proxy-cm podName:f5f5abfc-f7a4-4d0b-be33-bb9dc442904d nodeName:}" failed. No retries permitted until 2026-07-03 11:13:04.854905448 +0000 UTC m=+41.443209745 (durationBeforeRetry 500ms). Error: MountVolume.SetUp failed for volume "kube-rbac-proxy-cm" (UniqueName: "kubernetes.io/configmap/f5f5abfc-f7a4-4d0b-be33-bb9dc442904d-kube-rbac-proxy-cm") pod "insights-runtime-extractor-ws6w7" (UID: "f5f5abfc-f7a4-4d0b-be33-bb9dc442904d") : configmap "kube-rbac-proxy" not found Jul 03 11:13:04.365466 ip-10-0-136-0 kubenswrapper[2579]: I0703 11:13:04.365437 2579 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-dtrbk\" (UniqueName: \"kubernetes.io/projected/f5f5abfc-f7a4-4d0b-be33-bb9dc442904d-kube-api-access-dtrbk\") pod \"insights-runtime-extractor-ws6w7\" (UID: \"f5f5abfc-f7a4-4d0b-be33-bb9dc442904d\") " pod="openshift-insights/insights-runtime-extractor-ws6w7" Jul 03 11:13:04.371437 ip-10-0-136-0 kubenswrapper[2579]: I0703 11:13:04.371376 2579 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-multus/multus-additional-cni-plugins-52n9r" podStartSLOduration=8.970340961 podStartE2EDuration="40.371355824s" totalImagesPullingTime="31.401014863s" totalInitContainerRuntime="0s" isStatefulPod=true podCreationTimestamp="2026-07-03 11:12:24 +0000 UTC" imagePullSessionsCount=5 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-03 11:13:04.370085833 +0000 UTC m=+40.958390154" watchObservedRunningTime="2026-07-03 11:13:04.371355824 +0000 UTC m=+40.959660143" Jul 03 11:13:04.858918 ip-10-0-136-0 kubenswrapper[2579]: I0703 11:13:04.858880 2579 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-rbac-proxy-cm\" (UniqueName: \"kubernetes.io/configmap/f5f5abfc-f7a4-4d0b-be33-bb9dc442904d-kube-rbac-proxy-cm\") pod \"insights-runtime-extractor-ws6w7\" (UID: \"f5f5abfc-f7a4-4d0b-be33-bb9dc442904d\") " pod="openshift-insights/insights-runtime-extractor-ws6w7" Jul 03 11:13:04.858918 ip-10-0-136-0 kubenswrapper[2579]: I0703 11:13:04.858923 2579 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"insights-runtime-extractor-tls\" (UniqueName: \"kubernetes.io/secret/f5f5abfc-f7a4-4d0b-be33-bb9dc442904d-insights-runtime-extractor-tls\") pod \"insights-runtime-extractor-ws6w7\" (UID: \"f5f5abfc-f7a4-4d0b-be33-bb9dc442904d\") " pod="openshift-insights/insights-runtime-extractor-ws6w7" Jul 03 11:13:04.859144 ip-10-0-136-0 kubenswrapper[2579]: E0703 11:13:04.859014 2579 configmap.go:193] Couldn't get configMap openshift-insights/kube-rbac-proxy: configmap "kube-rbac-proxy" not found Jul 03 11:13:04.859144 ip-10-0-136-0 kubenswrapper[2579]: E0703 11:13:04.859085 2579 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/configmap/f5f5abfc-f7a4-4d0b-be33-bb9dc442904d-kube-rbac-proxy-cm podName:f5f5abfc-f7a4-4d0b-be33-bb9dc442904d nodeName:}" failed. No retries permitted until 2026-07-03 11:13:05.859070731 +0000 UTC m=+42.447375029 (durationBeforeRetry 1s). Error: MountVolume.SetUp failed for volume "kube-rbac-proxy-cm" (UniqueName: "kubernetes.io/configmap/f5f5abfc-f7a4-4d0b-be33-bb9dc442904d-kube-rbac-proxy-cm") pod "insights-runtime-extractor-ws6w7" (UID: "f5f5abfc-f7a4-4d0b-be33-bb9dc442904d") : configmap "kube-rbac-proxy" not found Jul 03 11:13:04.859144 ip-10-0-136-0 kubenswrapper[2579]: E0703 11:13:04.859023 2579 secret.go:189] Couldn't get secret openshift-insights/insights-runtime-extractor-tls: secret "insights-runtime-extractor-tls" not found Jul 03 11:13:04.859300 ip-10-0-136-0 kubenswrapper[2579]: E0703 11:13:04.859148 2579 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/f5f5abfc-f7a4-4d0b-be33-bb9dc442904d-insights-runtime-extractor-tls podName:f5f5abfc-f7a4-4d0b-be33-bb9dc442904d nodeName:}" failed. No retries permitted until 2026-07-03 11:13:05.859136946 +0000 UTC m=+42.447441248 (durationBeforeRetry 1s). Error: MountVolume.SetUp failed for volume "insights-runtime-extractor-tls" (UniqueName: "kubernetes.io/secret/f5f5abfc-f7a4-4d0b-be33-bb9dc442904d-insights-runtime-extractor-tls") pod "insights-runtime-extractor-ws6w7" (UID: "f5f5abfc-f7a4-4d0b-be33-bb9dc442904d") : secret "insights-runtime-extractor-tls" not found Jul 03 11:13:05.868334 ip-10-0-136-0 kubenswrapper[2579]: I0703 11:13:05.868289 2579 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-rbac-proxy-cm\" (UniqueName: \"kubernetes.io/configmap/f5f5abfc-f7a4-4d0b-be33-bb9dc442904d-kube-rbac-proxy-cm\") pod \"insights-runtime-extractor-ws6w7\" (UID: \"f5f5abfc-f7a4-4d0b-be33-bb9dc442904d\") " pod="openshift-insights/insights-runtime-extractor-ws6w7" Jul 03 11:13:05.868334 ip-10-0-136-0 kubenswrapper[2579]: I0703 11:13:05.868343 2579 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"insights-runtime-extractor-tls\" (UniqueName: \"kubernetes.io/secret/f5f5abfc-f7a4-4d0b-be33-bb9dc442904d-insights-runtime-extractor-tls\") pod \"insights-runtime-extractor-ws6w7\" (UID: \"f5f5abfc-f7a4-4d0b-be33-bb9dc442904d\") " pod="openshift-insights/insights-runtime-extractor-ws6w7" Jul 03 11:13:05.868785 ip-10-0-136-0 kubenswrapper[2579]: E0703 11:13:05.868412 2579 configmap.go:193] Couldn't get configMap openshift-insights/kube-rbac-proxy: configmap "kube-rbac-proxy" not found Jul 03 11:13:05.868785 ip-10-0-136-0 kubenswrapper[2579]: E0703 11:13:05.868430 2579 secret.go:189] Couldn't get secret openshift-insights/insights-runtime-extractor-tls: secret "insights-runtime-extractor-tls" not found Jul 03 11:13:05.868785 ip-10-0-136-0 kubenswrapper[2579]: E0703 11:13:05.868475 2579 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/f5f5abfc-f7a4-4d0b-be33-bb9dc442904d-insights-runtime-extractor-tls podName:f5f5abfc-f7a4-4d0b-be33-bb9dc442904d nodeName:}" failed. No retries permitted until 2026-07-03 11:13:07.868462979 +0000 UTC m=+44.456767278 (durationBeforeRetry 2s). Error: MountVolume.SetUp failed for volume "insights-runtime-extractor-tls" (UniqueName: "kubernetes.io/secret/f5f5abfc-f7a4-4d0b-be33-bb9dc442904d-insights-runtime-extractor-tls") pod "insights-runtime-extractor-ws6w7" (UID: "f5f5abfc-f7a4-4d0b-be33-bb9dc442904d") : secret "insights-runtime-extractor-tls" not found Jul 03 11:13:05.868785 ip-10-0-136-0 kubenswrapper[2579]: E0703 11:13:05.868488 2579 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/configmap/f5f5abfc-f7a4-4d0b-be33-bb9dc442904d-kube-rbac-proxy-cm podName:f5f5abfc-f7a4-4d0b-be33-bb9dc442904d nodeName:}" failed. No retries permitted until 2026-07-03 11:13:07.868481736 +0000 UTC m=+44.456786034 (durationBeforeRetry 2s). Error: MountVolume.SetUp failed for volume "kube-rbac-proxy-cm" (UniqueName: "kubernetes.io/configmap/f5f5abfc-f7a4-4d0b-be33-bb9dc442904d-kube-rbac-proxy-cm") pod "insights-runtime-extractor-ws6w7" (UID: "f5f5abfc-f7a4-4d0b-be33-bb9dc442904d") : configmap "kube-rbac-proxy" not found Jul 03 11:13:06.306691 ip-10-0-136-0 kubenswrapper[2579]: I0703 11:13:06.306662 2579 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-dns_node-resolver-hnmtl_9a6053f3-4885-4e53-8b1c-93150a7546c4/dns-node-resolver/0.log" Jul 03 11:13:06.907015 ip-10-0-136-0 kubenswrapper[2579]: I0703 11:13:06.906987 2579 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-image-registry_node-ca-9ff9g_f5fa8ffd-d2fc-4474-ad0d-a7af28edf0e7/node-ca/0.log" Jul 03 11:13:07.012056 ip-10-0-136-0 kubenswrapper[2579]: I0703 11:13:07.012016 2579 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-service-ca/service-ca-758f599c5-jghj2"] Jul 03 11:13:07.014083 ip-10-0-136-0 kubenswrapper[2579]: I0703 11:13:07.014063 2579 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-service-ca/service-ca-758f599c5-jghj2" Jul 03 11:13:07.016475 ip-10-0-136-0 kubenswrapper[2579]: I0703 11:13:07.016446 2579 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-service-ca\"/\"kube-root-ca.crt\"" Jul 03 11:13:07.017439 ip-10-0-136-0 kubenswrapper[2579]: I0703 11:13:07.017410 2579 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-service-ca\"/\"signing-cabundle\"" Jul 03 11:13:07.017439 ip-10-0-136-0 kubenswrapper[2579]: I0703 11:13:07.017422 2579 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-service-ca\"/\"service-ca-dockercfg-9cwwt\"" Jul 03 11:13:07.017611 ip-10-0-136-0 kubenswrapper[2579]: I0703 11:13:07.017462 2579 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-service-ca\"/\"signing-key\"" Jul 03 11:13:07.017611 ip-10-0-136-0 kubenswrapper[2579]: I0703 11:13:07.017416 2579 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-service-ca\"/\"openshift-service-ca.crt\"" Jul 03 11:13:07.024013 ip-10-0-136-0 kubenswrapper[2579]: I0703 11:13:07.023983 2579 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-service-ca/service-ca-758f599c5-jghj2"] Jul 03 11:13:07.178792 ip-10-0-136-0 kubenswrapper[2579]: I0703 11:13:07.178685 2579 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"signing-cabundle\" (UniqueName: \"kubernetes.io/configmap/c7a82084-c4ef-42ba-9505-a3c3a2421a51-signing-cabundle\") pod \"service-ca-758f599c5-jghj2\" (UID: \"c7a82084-c4ef-42ba-9505-a3c3a2421a51\") " pod="openshift-service-ca/service-ca-758f599c5-jghj2" Jul 03 11:13:07.178792 ip-10-0-136-0 kubenswrapper[2579]: I0703 11:13:07.178770 2579 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-8vk9t\" (UniqueName: \"kubernetes.io/projected/c7a82084-c4ef-42ba-9505-a3c3a2421a51-kube-api-access-8vk9t\") pod \"service-ca-758f599c5-jghj2\" (UID: \"c7a82084-c4ef-42ba-9505-a3c3a2421a51\") " pod="openshift-service-ca/service-ca-758f599c5-jghj2" Jul 03 11:13:07.178985 ip-10-0-136-0 kubenswrapper[2579]: I0703 11:13:07.178820 2579 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"signing-key\" (UniqueName: \"kubernetes.io/secret/c7a82084-c4ef-42ba-9505-a3c3a2421a51-signing-key\") pod \"service-ca-758f599c5-jghj2\" (UID: \"c7a82084-c4ef-42ba-9505-a3c3a2421a51\") " pod="openshift-service-ca/service-ca-758f599c5-jghj2" Jul 03 11:13:07.280211 ip-10-0-136-0 kubenswrapper[2579]: I0703 11:13:07.280138 2579 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"signing-key\" (UniqueName: \"kubernetes.io/secret/c7a82084-c4ef-42ba-9505-a3c3a2421a51-signing-key\") pod \"service-ca-758f599c5-jghj2\" (UID: \"c7a82084-c4ef-42ba-9505-a3c3a2421a51\") " pod="openshift-service-ca/service-ca-758f599c5-jghj2" Jul 03 11:13:07.280382 ip-10-0-136-0 kubenswrapper[2579]: I0703 11:13:07.280344 2579 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"signing-cabundle\" (UniqueName: \"kubernetes.io/configmap/c7a82084-c4ef-42ba-9505-a3c3a2421a51-signing-cabundle\") pod \"service-ca-758f599c5-jghj2\" (UID: \"c7a82084-c4ef-42ba-9505-a3c3a2421a51\") " pod="openshift-service-ca/service-ca-758f599c5-jghj2" Jul 03 11:13:07.280437 ip-10-0-136-0 kubenswrapper[2579]: I0703 11:13:07.280384 2579 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-8vk9t\" (UniqueName: \"kubernetes.io/projected/c7a82084-c4ef-42ba-9505-a3c3a2421a51-kube-api-access-8vk9t\") pod \"service-ca-758f599c5-jghj2\" (UID: \"c7a82084-c4ef-42ba-9505-a3c3a2421a51\") " pod="openshift-service-ca/service-ca-758f599c5-jghj2" Jul 03 11:13:07.281052 ip-10-0-136-0 kubenswrapper[2579]: I0703 11:13:07.281019 2579 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"signing-cabundle\" (UniqueName: \"kubernetes.io/configmap/c7a82084-c4ef-42ba-9505-a3c3a2421a51-signing-cabundle\") pod \"service-ca-758f599c5-jghj2\" (UID: \"c7a82084-c4ef-42ba-9505-a3c3a2421a51\") " pod="openshift-service-ca/service-ca-758f599c5-jghj2" Jul 03 11:13:07.282728 ip-10-0-136-0 kubenswrapper[2579]: I0703 11:13:07.282699 2579 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"signing-key\" (UniqueName: \"kubernetes.io/secret/c7a82084-c4ef-42ba-9505-a3c3a2421a51-signing-key\") pod \"service-ca-758f599c5-jghj2\" (UID: \"c7a82084-c4ef-42ba-9505-a3c3a2421a51\") " pod="openshift-service-ca/service-ca-758f599c5-jghj2" Jul 03 11:13:07.289153 ip-10-0-136-0 kubenswrapper[2579]: I0703 11:13:07.289120 2579 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-8vk9t\" (UniqueName: \"kubernetes.io/projected/c7a82084-c4ef-42ba-9505-a3c3a2421a51-kube-api-access-8vk9t\") pod \"service-ca-758f599c5-jghj2\" (UID: \"c7a82084-c4ef-42ba-9505-a3c3a2421a51\") " pod="openshift-service-ca/service-ca-758f599c5-jghj2" Jul 03 11:13:07.325504 ip-10-0-136-0 kubenswrapper[2579]: I0703 11:13:07.325460 2579 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-service-ca/service-ca-758f599c5-jghj2" Jul 03 11:13:07.462766 ip-10-0-136-0 kubenswrapper[2579]: I0703 11:13:07.462675 2579 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-service-ca/service-ca-758f599c5-jghj2"] Jul 03 11:13:07.465875 ip-10-0-136-0 kubenswrapper[2579]: W0703 11:13:07.465837 2579 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podc7a82084_c4ef_42ba_9505_a3c3a2421a51.slice/crio-f515421cd1d49283616caed5e0b64343a1d9956d2a5cc6a66df43e3afd645f8d WatchSource:0}: Error finding container f515421cd1d49283616caed5e0b64343a1d9956d2a5cc6a66df43e3afd645f8d: Status 404 returned error can't find the container with id f515421cd1d49283616caed5e0b64343a1d9956d2a5cc6a66df43e3afd645f8d Jul 03 11:13:07.482398 ip-10-0-136-0 kubenswrapper[2579]: I0703 11:13:07.482365 2579 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-tls\" (UniqueName: \"kubernetes.io/secret/266035ef-72fb-4911-9110-2806027e1770-metrics-tls\") pod \"dns-default-tksjt\" (UID: \"266035ef-72fb-4911-9110-2806027e1770\") " pod="openshift-dns/dns-default-tksjt" Jul 03 11:13:07.482577 ip-10-0-136-0 kubenswrapper[2579]: E0703 11:13:07.482548 2579 secret.go:189] Couldn't get secret openshift-dns/dns-default-metrics-tls: secret "dns-default-metrics-tls" not found Jul 03 11:13:07.482636 ip-10-0-136-0 kubenswrapper[2579]: E0703 11:13:07.482624 2579 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/266035ef-72fb-4911-9110-2806027e1770-metrics-tls podName:266035ef-72fb-4911-9110-2806027e1770 nodeName:}" failed. No retries permitted until 2026-07-03 11:13:15.482606238 +0000 UTC m=+52.070910540 (durationBeforeRetry 8s). Error: MountVolume.SetUp failed for volume "metrics-tls" (UniqueName: "kubernetes.io/secret/266035ef-72fb-4911-9110-2806027e1770-metrics-tls") pod "dns-default-tksjt" (UID: "266035ef-72fb-4911-9110-2806027e1770") : secret "dns-default-metrics-tls" not found Jul 03 11:13:07.583816 ip-10-0-136-0 kubenswrapper[2579]: I0703 11:13:07.583768 2579 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"registry-tls\" (UniqueName: \"kubernetes.io/projected/387cc87f-815e-457a-83cf-d016035fed5d-registry-tls\") pod \"image-registry-659bf79d58-p9gqp\" (UID: \"387cc87f-815e-457a-83cf-d016035fed5d\") " pod="openshift-image-registry/image-registry-659bf79d58-p9gqp" Jul 03 11:13:07.583981 ip-10-0-136-0 kubenswrapper[2579]: E0703 11:13:07.583924 2579 projected.go:266] Couldn't get secret openshift-image-registry/image-registry-tls: secret "image-registry-tls" not found Jul 03 11:13:07.583981 ip-10-0-136-0 kubenswrapper[2579]: E0703 11:13:07.583945 2579 projected.go:196] Error preparing data for projected volume registry-tls for pod openshift-image-registry/image-registry-659bf79d58-p9gqp: secret "image-registry-tls" not found Jul 03 11:13:07.584064 ip-10-0-136-0 kubenswrapper[2579]: E0703 11:13:07.584053 2579 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/projected/387cc87f-815e-457a-83cf-d016035fed5d-registry-tls podName:387cc87f-815e-457a-83cf-d016035fed5d nodeName:}" failed. No retries permitted until 2026-07-03 11:13:15.584035632 +0000 UTC m=+52.172339932 (durationBeforeRetry 8s). Error: MountVolume.SetUp failed for volume "registry-tls" (UniqueName: "kubernetes.io/projected/387cc87f-815e-457a-83cf-d016035fed5d-registry-tls") pod "image-registry-659bf79d58-p9gqp" (UID: "387cc87f-815e-457a-83cf-d016035fed5d") : secret "image-registry-tls" not found Jul 03 11:13:07.707881 ip-10-0-136-0 kubenswrapper[2579]: I0703 11:13:07.707853 2579 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ingress-canary_ingress-canary-2nclq_868c9601-3c66-498e-8076-e7e289b9a4a0/serve-healthcheck-canary/0.log" Jul 03 11:13:07.886887 ip-10-0-136-0 kubenswrapper[2579]: I0703 11:13:07.886843 2579 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-rbac-proxy-cm\" (UniqueName: \"kubernetes.io/configmap/f5f5abfc-f7a4-4d0b-be33-bb9dc442904d-kube-rbac-proxy-cm\") pod \"insights-runtime-extractor-ws6w7\" (UID: \"f5f5abfc-f7a4-4d0b-be33-bb9dc442904d\") " pod="openshift-insights/insights-runtime-extractor-ws6w7" Jul 03 11:13:07.887077 ip-10-0-136-0 kubenswrapper[2579]: I0703 11:13:07.886900 2579 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"insights-runtime-extractor-tls\" (UniqueName: \"kubernetes.io/secret/f5f5abfc-f7a4-4d0b-be33-bb9dc442904d-insights-runtime-extractor-tls\") pod \"insights-runtime-extractor-ws6w7\" (UID: \"f5f5abfc-f7a4-4d0b-be33-bb9dc442904d\") " pod="openshift-insights/insights-runtime-extractor-ws6w7" Jul 03 11:13:07.887077 ip-10-0-136-0 kubenswrapper[2579]: E0703 11:13:07.886985 2579 configmap.go:193] Couldn't get configMap openshift-insights/kube-rbac-proxy: configmap "kube-rbac-proxy" not found Jul 03 11:13:07.887168 ip-10-0-136-0 kubenswrapper[2579]: E0703 11:13:07.887077 2579 secret.go:189] Couldn't get secret openshift-insights/insights-runtime-extractor-tls: secret "insights-runtime-extractor-tls" not found Jul 03 11:13:07.887168 ip-10-0-136-0 kubenswrapper[2579]: E0703 11:13:07.887073 2579 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/configmap/f5f5abfc-f7a4-4d0b-be33-bb9dc442904d-kube-rbac-proxy-cm podName:f5f5abfc-f7a4-4d0b-be33-bb9dc442904d nodeName:}" failed. No retries permitted until 2026-07-03 11:13:11.887053246 +0000 UTC m=+48.475357559 (durationBeforeRetry 4s). Error: MountVolume.SetUp failed for volume "kube-rbac-proxy-cm" (UniqueName: "kubernetes.io/configmap/f5f5abfc-f7a4-4d0b-be33-bb9dc442904d-kube-rbac-proxy-cm") pod "insights-runtime-extractor-ws6w7" (UID: "f5f5abfc-f7a4-4d0b-be33-bb9dc442904d") : configmap "kube-rbac-proxy" not found Jul 03 11:13:07.887168 ip-10-0-136-0 kubenswrapper[2579]: E0703 11:13:07.887116 2579 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/f5f5abfc-f7a4-4d0b-be33-bb9dc442904d-insights-runtime-extractor-tls podName:f5f5abfc-f7a4-4d0b-be33-bb9dc442904d nodeName:}" failed. No retries permitted until 2026-07-03 11:13:11.887106079 +0000 UTC m=+48.475410378 (durationBeforeRetry 4s). Error: MountVolume.SetUp failed for volume "insights-runtime-extractor-tls" (UniqueName: "kubernetes.io/secret/f5f5abfc-f7a4-4d0b-be33-bb9dc442904d-insights-runtime-extractor-tls") pod "insights-runtime-extractor-ws6w7" (UID: "f5f5abfc-f7a4-4d0b-be33-bb9dc442904d") : secret "insights-runtime-extractor-tls" not found Jul 03 11:13:08.343058 ip-10-0-136-0 kubenswrapper[2579]: I0703 11:13:08.343016 2579 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-service-ca/service-ca-758f599c5-jghj2" event={"ID":"c7a82084-c4ef-42ba-9505-a3c3a2421a51","Type":"ContainerStarted","Data":"f515421cd1d49283616caed5e0b64343a1d9956d2a5cc6a66df43e3afd645f8d"} Jul 03 11:13:10.350360 ip-10-0-136-0 kubenswrapper[2579]: I0703 11:13:10.350315 2579 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-service-ca/service-ca-758f599c5-jghj2" event={"ID":"c7a82084-c4ef-42ba-9505-a3c3a2421a51","Type":"ContainerStarted","Data":"8070cf88eec31dbb584ade653ee0ee98058cff2e081e9b96e028ae26b3b2d98e"} Jul 03 11:13:10.382847 ip-10-0-136-0 kubenswrapper[2579]: I0703 11:13:10.382793 2579 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-service-ca/service-ca-758f599c5-jghj2" podStartSLOduration=2.190214097 podStartE2EDuration="4.382773663s" totalImagesPullingTime="2.192559566s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-03 11:13:06 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-03 11:13:10.381269449 +0000 UTC m=+46.969573768" watchObservedRunningTime="2026-07-03 11:13:10.382773663 +0000 UTC m=+46.971077983" Jul 03 11:13:11.921561 ip-10-0-136-0 kubenswrapper[2579]: I0703 11:13:11.921512 2579 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-rbac-proxy-cm\" (UniqueName: \"kubernetes.io/configmap/f5f5abfc-f7a4-4d0b-be33-bb9dc442904d-kube-rbac-proxy-cm\") pod \"insights-runtime-extractor-ws6w7\" (UID: \"f5f5abfc-f7a4-4d0b-be33-bb9dc442904d\") " pod="openshift-insights/insights-runtime-extractor-ws6w7" Jul 03 11:13:11.921561 ip-10-0-136-0 kubenswrapper[2579]: I0703 11:13:11.921565 2579 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"insights-runtime-extractor-tls\" (UniqueName: \"kubernetes.io/secret/f5f5abfc-f7a4-4d0b-be33-bb9dc442904d-insights-runtime-extractor-tls\") pod \"insights-runtime-extractor-ws6w7\" (UID: \"f5f5abfc-f7a4-4d0b-be33-bb9dc442904d\") " pod="openshift-insights/insights-runtime-extractor-ws6w7" Jul 03 11:13:11.922005 ip-10-0-136-0 kubenswrapper[2579]: E0703 11:13:11.921696 2579 secret.go:189] Couldn't get secret openshift-insights/insights-runtime-extractor-tls: secret "insights-runtime-extractor-tls" not found Jul 03 11:13:11.922005 ip-10-0-136-0 kubenswrapper[2579]: E0703 11:13:11.921763 2579 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/f5f5abfc-f7a4-4d0b-be33-bb9dc442904d-insights-runtime-extractor-tls podName:f5f5abfc-f7a4-4d0b-be33-bb9dc442904d nodeName:}" failed. No retries permitted until 2026-07-03 11:13:19.921748854 +0000 UTC m=+56.510053152 (durationBeforeRetry 8s). Error: MountVolume.SetUp failed for volume "insights-runtime-extractor-tls" (UniqueName: "kubernetes.io/secret/f5f5abfc-f7a4-4d0b-be33-bb9dc442904d-insights-runtime-extractor-tls") pod "insights-runtime-extractor-ws6w7" (UID: "f5f5abfc-f7a4-4d0b-be33-bb9dc442904d") : secret "insights-runtime-extractor-tls" not found Jul 03 11:13:11.922081 ip-10-0-136-0 kubenswrapper[2579]: I0703 11:13:11.922038 2579 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-rbac-proxy-cm\" (UniqueName: \"kubernetes.io/configmap/f5f5abfc-f7a4-4d0b-be33-bb9dc442904d-kube-rbac-proxy-cm\") pod \"insights-runtime-extractor-ws6w7\" (UID: \"f5f5abfc-f7a4-4d0b-be33-bb9dc442904d\") " pod="openshift-insights/insights-runtime-extractor-ws6w7" Jul 03 11:13:15.552724 ip-10-0-136-0 kubenswrapper[2579]: I0703 11:13:15.552681 2579 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-tls\" (UniqueName: \"kubernetes.io/secret/266035ef-72fb-4911-9110-2806027e1770-metrics-tls\") pod \"dns-default-tksjt\" (UID: \"266035ef-72fb-4911-9110-2806027e1770\") " pod="openshift-dns/dns-default-tksjt" Jul 03 11:13:15.555324 ip-10-0-136-0 kubenswrapper[2579]: I0703 11:13:15.555298 2579 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"metrics-tls\" (UniqueName: \"kubernetes.io/secret/266035ef-72fb-4911-9110-2806027e1770-metrics-tls\") pod \"dns-default-tksjt\" (UID: \"266035ef-72fb-4911-9110-2806027e1770\") " pod="openshift-dns/dns-default-tksjt" Jul 03 11:13:15.646033 ip-10-0-136-0 kubenswrapper[2579]: I0703 11:13:15.645990 2579 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-dns/dns-default-tksjt" Jul 03 11:13:15.653320 ip-10-0-136-0 kubenswrapper[2579]: I0703 11:13:15.653284 2579 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"registry-tls\" (UniqueName: \"kubernetes.io/projected/387cc87f-815e-457a-83cf-d016035fed5d-registry-tls\") pod \"image-registry-659bf79d58-p9gqp\" (UID: \"387cc87f-815e-457a-83cf-d016035fed5d\") " pod="openshift-image-registry/image-registry-659bf79d58-p9gqp" Jul 03 11:13:15.656118 ip-10-0-136-0 kubenswrapper[2579]: I0703 11:13:15.656075 2579 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"registry-tls\" (UniqueName: \"kubernetes.io/projected/387cc87f-815e-457a-83cf-d016035fed5d-registry-tls\") pod \"image-registry-659bf79d58-p9gqp\" (UID: \"387cc87f-815e-457a-83cf-d016035fed5d\") " pod="openshift-image-registry/image-registry-659bf79d58-p9gqp" Jul 03 11:13:15.673506 ip-10-0-136-0 kubenswrapper[2579]: I0703 11:13:15.673461 2579 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-image-registry/image-registry-659bf79d58-p9gqp" Jul 03 11:13:15.822702 ip-10-0-136-0 kubenswrapper[2579]: I0703 11:13:15.822613 2579 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-dns/dns-default-tksjt"] Jul 03 11:13:15.826754 ip-10-0-136-0 kubenswrapper[2579]: W0703 11:13:15.826720 2579 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod266035ef_72fb_4911_9110_2806027e1770.slice/crio-c9ad8f5aaa9c78072c5c0aafc5702979d5460199c705de11d9d8573f82e44928 WatchSource:0}: Error finding container c9ad8f5aaa9c78072c5c0aafc5702979d5460199c705de11d9d8573f82e44928: Status 404 returned error can't find the container with id c9ad8f5aaa9c78072c5c0aafc5702979d5460199c705de11d9d8573f82e44928 Jul 03 11:13:15.863102 ip-10-0-136-0 kubenswrapper[2579]: I0703 11:13:15.863066 2579 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-image-registry/image-registry-659bf79d58-p9gqp"] Jul 03 11:13:15.866116 ip-10-0-136-0 kubenswrapper[2579]: W0703 11:13:15.866083 2579 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod387cc87f_815e_457a_83cf_d016035fed5d.slice/crio-3287ba5cc39962a6290b4de6be0f067fe5b158bcff445eaf62d07abffdbf5f7d WatchSource:0}: Error finding container 3287ba5cc39962a6290b4de6be0f067fe5b158bcff445eaf62d07abffdbf5f7d: Status 404 returned error can't find the container with id 3287ba5cc39962a6290b4de6be0f067fe5b158bcff445eaf62d07abffdbf5f7d Jul 03 11:13:16.369259 ip-10-0-136-0 kubenswrapper[2579]: I0703 11:13:16.369211 2579 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-dns/dns-default-tksjt" event={"ID":"266035ef-72fb-4911-9110-2806027e1770","Type":"ContainerStarted","Data":"c9ad8f5aaa9c78072c5c0aafc5702979d5460199c705de11d9d8573f82e44928"} Jul 03 11:13:16.370836 ip-10-0-136-0 kubenswrapper[2579]: I0703 11:13:16.370804 2579 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-image-registry/image-registry-659bf79d58-p9gqp" event={"ID":"387cc87f-815e-457a-83cf-d016035fed5d","Type":"ContainerStarted","Data":"089af8b23bb2f9a8f44bc4c414478fe3a43c64827691f9eb9089bef89987978b"} Jul 03 11:13:16.371013 ip-10-0-136-0 kubenswrapper[2579]: I0703 11:13:16.370848 2579 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-image-registry/image-registry-659bf79d58-p9gqp" event={"ID":"387cc87f-815e-457a-83cf-d016035fed5d","Type":"ContainerStarted","Data":"3287ba5cc39962a6290b4de6be0f067fe5b158bcff445eaf62d07abffdbf5f7d"} Jul 03 11:13:16.371013 ip-10-0-136-0 kubenswrapper[2579]: I0703 11:13:16.370923 2579 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-image-registry/image-registry-659bf79d58-p9gqp" Jul 03 11:13:16.392033 ip-10-0-136-0 kubenswrapper[2579]: I0703 11:13:16.391974 2579 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-image-registry/image-registry-659bf79d58-p9gqp" podStartSLOduration=41.391951188 podStartE2EDuration="41.391951188s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-03 11:12:35 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-03 11:13:16.391076902 +0000 UTC m=+52.979381232" watchObservedRunningTime="2026-07-03 11:13:16.391951188 +0000 UTC m=+52.980255489" Jul 03 11:13:18.379062 ip-10-0-136-0 kubenswrapper[2579]: I0703 11:13:18.378963 2579 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-dns/dns-default-tksjt" event={"ID":"266035ef-72fb-4911-9110-2806027e1770","Type":"ContainerStarted","Data":"b33ad1ab62c0c33e9b93031e190fc8d6b83fd6d2f5d30f250c855a8072eef837"} Jul 03 11:13:18.379062 ip-10-0-136-0 kubenswrapper[2579]: I0703 11:13:18.379029 2579 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-dns/dns-default-tksjt" event={"ID":"266035ef-72fb-4911-9110-2806027e1770","Type":"ContainerStarted","Data":"fe3361f1789a3e8a3492c82d835dac4f2044a0afbcbae80895b23d3d1d0c3f88"} Jul 03 11:13:18.379580 ip-10-0-136-0 kubenswrapper[2579]: I0703 11:13:18.379124 2579 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-dns/dns-default-tksjt" Jul 03 11:13:18.397490 ip-10-0-136-0 kubenswrapper[2579]: I0703 11:13:18.397428 2579 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-dns/dns-default-tksjt" podStartSLOduration=17.173767583 podStartE2EDuration="19.397410318s" totalImagesPullingTime="2.223642735s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-03 11:12:59 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-03 11:13:18.396613602 +0000 UTC m=+54.984917925" watchObservedRunningTime="2026-07-03 11:13:18.397410318 +0000 UTC m=+54.985714637" Jul 03 11:13:19.995317 ip-10-0-136-0 kubenswrapper[2579]: I0703 11:13:19.995266 2579 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"insights-runtime-extractor-tls\" (UniqueName: \"kubernetes.io/secret/f5f5abfc-f7a4-4d0b-be33-bb9dc442904d-insights-runtime-extractor-tls\") pod \"insights-runtime-extractor-ws6w7\" (UID: \"f5f5abfc-f7a4-4d0b-be33-bb9dc442904d\") " pod="openshift-insights/insights-runtime-extractor-ws6w7" Jul 03 11:13:19.997802 ip-10-0-136-0 kubenswrapper[2579]: I0703 11:13:19.997779 2579 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"insights-runtime-extractor-tls\" (UniqueName: \"kubernetes.io/secret/f5f5abfc-f7a4-4d0b-be33-bb9dc442904d-insights-runtime-extractor-tls\") pod \"insights-runtime-extractor-ws6w7\" (UID: \"f5f5abfc-f7a4-4d0b-be33-bb9dc442904d\") " pod="openshift-insights/insights-runtime-extractor-ws6w7" Jul 03 11:13:20.070694 ip-10-0-136-0 kubenswrapper[2579]: I0703 11:13:20.070658 2579 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-insights/insights-runtime-extractor-ws6w7" Jul 03 11:13:20.234625 ip-10-0-136-0 kubenswrapper[2579]: I0703 11:13:20.234589 2579 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-insights/insights-runtime-extractor-ws6w7"] Jul 03 11:13:20.238146 ip-10-0-136-0 kubenswrapper[2579]: W0703 11:13:20.238113 2579 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podf5f5abfc_f7a4_4d0b_be33_bb9dc442904d.slice/crio-14c507d23e7e6702984ccea53aa847d980a5aababdb7acd7d51f50a88bd9d6b8 WatchSource:0}: Error finding container 14c507d23e7e6702984ccea53aa847d980a5aababdb7acd7d51f50a88bd9d6b8: Status 404 returned error can't find the container with id 14c507d23e7e6702984ccea53aa847d980a5aababdb7acd7d51f50a88bd9d6b8 Jul 03 11:13:20.385390 ip-10-0-136-0 kubenswrapper[2579]: I0703 11:13:20.385350 2579 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-insights/insights-runtime-extractor-ws6w7" event={"ID":"f5f5abfc-f7a4-4d0b-be33-bb9dc442904d","Type":"ContainerStarted","Data":"14c507d23e7e6702984ccea53aa847d980a5aababdb7acd7d51f50a88bd9d6b8"} Jul 03 11:13:25.401866 ip-10-0-136-0 kubenswrapper[2579]: I0703 11:13:25.401823 2579 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-insights/insights-runtime-extractor-ws6w7" event={"ID":"f5f5abfc-f7a4-4d0b-be33-bb9dc442904d","Type":"ContainerStarted","Data":"6fcc8e53c8c4c6b469c65ade247d8b18f5299057422f73e52e0ee73865855575"} Jul 03 11:13:26.312580 ip-10-0-136-0 kubenswrapper[2579]: I0703 11:13:26.312548 2579 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ovn-kubernetes/ovnkube-node-dchtv" Jul 03 11:13:27.411844 ip-10-0-136-0 kubenswrapper[2579]: I0703 11:13:27.411807 2579 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-insights/insights-runtime-extractor-ws6w7" event={"ID":"f5f5abfc-f7a4-4d0b-be33-bb9dc442904d","Type":"ContainerStarted","Data":"e8996f6dba1a9091abe5d9c48c1802840637196add91b19d745e45c8808ae1db"} Jul 03 11:13:28.384458 ip-10-0-136-0 kubenswrapper[2579]: I0703 11:13:28.384423 2579 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-dns/dns-default-tksjt" Jul 03 11:13:29.420841 ip-10-0-136-0 kubenswrapper[2579]: I0703 11:13:29.420741 2579 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-insights/insights-runtime-extractor-ws6w7" event={"ID":"f5f5abfc-f7a4-4d0b-be33-bb9dc442904d","Type":"ContainerStarted","Data":"1d80d093cb417fcbed3845930f9d37315e044e4513247560d1220a8d3ebcc75a"} Jul 03 11:13:29.438915 ip-10-0-136-0 kubenswrapper[2579]: I0703 11:13:29.438855 2579 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-insights/insights-runtime-extractor-ws6w7" podStartSLOduration=16.822710615 podStartE2EDuration="25.438835712s" totalImagesPullingTime="8.616125097s" totalInitContainerRuntime="0s" isStatefulPod=true podCreationTimestamp="2026-07-03 11:13:04 +0000 UTC" imagePullSessionsCount=3 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-03 11:13:29.43764657 +0000 UTC m=+66.025950890" watchObservedRunningTime="2026-07-03 11:13:29.438835712 +0000 UTC m=+66.027140031" Jul 03 11:13:29.786116 ip-10-0-136-0 kubenswrapper[2579]: I0703 11:13:29.786069 2579 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"original-pull-secret\" (UniqueName: \"kubernetes.io/secret/7813f158-0f67-411e-84d9-a2a1c51eea61-original-pull-secret\") pod \"global-pull-secret-syncer-9pqm8\" (UID: \"7813f158-0f67-411e-84d9-a2a1c51eea61\") " pod="kube-system/global-pull-secret-syncer-9pqm8" Jul 03 11:13:29.786319 ip-10-0-136-0 kubenswrapper[2579]: I0703 11:13:29.786217 2579 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/34ca9c51-4743-4efc-9aa1-eb79a741f6a4-metrics-certs\") pod \"network-metrics-daemon-sqxsj\" (UID: \"34ca9c51-4743-4efc-9aa1-eb79a741f6a4\") " pod="openshift-multus/network-metrics-daemon-sqxsj" Jul 03 11:13:29.788603 ip-10-0-136-0 kubenswrapper[2579]: I0703 11:13:29.788580 2579 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"kube-system\"/\"original-pull-secret\"" Jul 03 11:13:29.788665 ip-10-0-136-0 kubenswrapper[2579]: I0703 11:13:29.788605 2579 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-multus\"/\"metrics-daemon-secret\"" Jul 03 11:13:29.799230 ip-10-0-136-0 kubenswrapper[2579]: I0703 11:13:29.799171 2579 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"original-pull-secret\" (UniqueName: \"kubernetes.io/secret/7813f158-0f67-411e-84d9-a2a1c51eea61-original-pull-secret\") pod \"global-pull-secret-syncer-9pqm8\" (UID: \"7813f158-0f67-411e-84d9-a2a1c51eea61\") " pod="kube-system/global-pull-secret-syncer-9pqm8" Jul 03 11:13:29.799230 ip-10-0-136-0 kubenswrapper[2579]: I0703 11:13:29.799215 2579 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/34ca9c51-4743-4efc-9aa1-eb79a741f6a4-metrics-certs\") pod \"network-metrics-daemon-sqxsj\" (UID: \"34ca9c51-4743-4efc-9aa1-eb79a741f6a4\") " pod="openshift-multus/network-metrics-daemon-sqxsj" Jul 03 11:13:29.807681 ip-10-0-136-0 kubenswrapper[2579]: I0703 11:13:29.807649 2579 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-multus\"/\"metrics-daemon-sa-dockercfg-qx6zp\"" Jul 03 11:13:29.815927 ip-10-0-136-0 kubenswrapper[2579]: I0703 11:13:29.815881 2579 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-sqxsj" Jul 03 11:13:29.887593 ip-10-0-136-0 kubenswrapper[2579]: I0703 11:13:29.887559 2579 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-flqqx\" (UniqueName: \"kubernetes.io/projected/2129ac30-1645-4555-b722-55963ffb8776-kube-api-access-flqqx\") pod \"network-check-target-xzlsc\" (UID: \"2129ac30-1645-4555-b722-55963ffb8776\") " pod="openshift-network-diagnostics/network-check-target-xzlsc" Jul 03 11:13:29.890591 ip-10-0-136-0 kubenswrapper[2579]: I0703 11:13:29.890554 2579 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-network-diagnostics\"/\"kube-root-ca.crt\"" Jul 03 11:13:29.901432 ip-10-0-136-0 kubenswrapper[2579]: I0703 11:13:29.901321 2579 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-network-diagnostics\"/\"openshift-service-ca.crt\"" Jul 03 11:13:29.912345 ip-10-0-136-0 kubenswrapper[2579]: I0703 11:13:29.912310 2579 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-flqqx\" (UniqueName: \"kubernetes.io/projected/2129ac30-1645-4555-b722-55963ffb8776-kube-api-access-flqqx\") pod \"network-check-target-xzlsc\" (UID: \"2129ac30-1645-4555-b722-55963ffb8776\") " pod="openshift-network-diagnostics/network-check-target-xzlsc" Jul 03 11:13:29.950151 ip-10-0-136-0 kubenswrapper[2579]: I0703 11:13:29.950108 2579 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-multus/network-metrics-daemon-sqxsj"] Jul 03 11:13:29.954317 ip-10-0-136-0 kubenswrapper[2579]: W0703 11:13:29.954280 2579 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod34ca9c51_4743_4efc_9aa1_eb79a741f6a4.slice/crio-e570bc32d8cb33f10dd96cf1e82010dcd5ea9d0c4aab47bf65a40bf560f9d24b WatchSource:0}: Error finding container e570bc32d8cb33f10dd96cf1e82010dcd5ea9d0c4aab47bf65a40bf560f9d24b: Status 404 returned error can't find the container with id e570bc32d8cb33f10dd96cf1e82010dcd5ea9d0c4aab47bf65a40bf560f9d24b Jul 03 11:13:30.098212 ip-10-0-136-0 kubenswrapper[2579]: I0703 11:13:30.098107 2579 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-9pqm8" Jul 03 11:13:30.114733 ip-10-0-136-0 kubenswrapper[2579]: I0703 11:13:30.114701 2579 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-network-diagnostics\"/\"default-dockercfg-wfk2t\"" Jul 03 11:13:30.122785 ip-10-0-136-0 kubenswrapper[2579]: I0703 11:13:30.122752 2579 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-xzlsc" Jul 03 11:13:30.242800 ip-10-0-136-0 kubenswrapper[2579]: I0703 11:13:30.242751 2579 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kube-system/global-pull-secret-syncer-9pqm8"] Jul 03 11:13:30.245774 ip-10-0-136-0 kubenswrapper[2579]: W0703 11:13:30.245744 2579 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod7813f158_0f67_411e_84d9_a2a1c51eea61.slice/crio-a63c40819f384806ab2a526b8b5f8e0c30301223623bf77cfe42f07e112cd401 WatchSource:0}: Error finding container a63c40819f384806ab2a526b8b5f8e0c30301223623bf77cfe42f07e112cd401: Status 404 returned error can't find the container with id a63c40819f384806ab2a526b8b5f8e0c30301223623bf77cfe42f07e112cd401 Jul 03 11:13:30.266603 ip-10-0-136-0 kubenswrapper[2579]: I0703 11:13:30.266556 2579 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-network-diagnostics/network-check-target-xzlsc"] Jul 03 11:13:30.269640 ip-10-0-136-0 kubenswrapper[2579]: W0703 11:13:30.269609 2579 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod2129ac30_1645_4555_b722_55963ffb8776.slice/crio-688a84ec0a730b349094ad2620cbafdb0adb98a522aedd40275f7eebc1ebc4ad WatchSource:0}: Error finding container 688a84ec0a730b349094ad2620cbafdb0adb98a522aedd40275f7eebc1ebc4ad: Status 404 returned error can't find the container with id 688a84ec0a730b349094ad2620cbafdb0adb98a522aedd40275f7eebc1ebc4ad Jul 03 11:13:30.426694 ip-10-0-136-0 kubenswrapper[2579]: I0703 11:13:30.426603 2579 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kube-system/global-pull-secret-syncer-9pqm8" event={"ID":"7813f158-0f67-411e-84d9-a2a1c51eea61","Type":"ContainerStarted","Data":"a63c40819f384806ab2a526b8b5f8e0c30301223623bf77cfe42f07e112cd401"} Jul 03 11:13:30.428244 ip-10-0-136-0 kubenswrapper[2579]: I0703 11:13:30.428206 2579 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-multus/network-metrics-daemon-sqxsj" event={"ID":"34ca9c51-4743-4efc-9aa1-eb79a741f6a4","Type":"ContainerStarted","Data":"e570bc32d8cb33f10dd96cf1e82010dcd5ea9d0c4aab47bf65a40bf560f9d24b"} Jul 03 11:13:30.429623 ip-10-0-136-0 kubenswrapper[2579]: I0703 11:13:30.429576 2579 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-network-diagnostics/network-check-target-xzlsc" event={"ID":"2129ac30-1645-4555-b722-55963ffb8776","Type":"ContainerStarted","Data":"688a84ec0a730b349094ad2620cbafdb0adb98a522aedd40275f7eebc1ebc4ad"} Jul 03 11:13:30.476699 ip-10-0-136-0 kubenswrapper[2579]: I0703 11:13:30.476662 2579 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["open-cluster-management-agent-addon/klusterlet-addon-workmgr-7d8dd55d49-bs5j5"] Jul 03 11:13:30.479857 ip-10-0-136-0 kubenswrapper[2579]: I0703 11:13:30.479830 2579 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="open-cluster-management-agent-addon/klusterlet-addon-workmgr-7d8dd55d49-bs5j5" Jul 03 11:13:30.482634 ip-10-0-136-0 kubenswrapper[2579]: I0703 11:13:30.482602 2579 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"open-cluster-management-agent-addon\"/\"work-manager-hub-kubeconfig\"" Jul 03 11:13:30.482809 ip-10-0-136-0 kubenswrapper[2579]: I0703 11:13:30.482786 2579 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"open-cluster-management-agent-addon\"/\"openshift-service-ca.crt\"" Jul 03 11:13:30.482982 ip-10-0-136-0 kubenswrapper[2579]: I0703 11:13:30.482965 2579 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"open-cluster-management-agent-addon\"/\"open-cluster-management-image-pull-credentials\"" Jul 03 11:13:30.483309 ip-10-0-136-0 kubenswrapper[2579]: I0703 11:13:30.483295 2579 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"open-cluster-management-agent-addon\"/\"kube-root-ca.crt\"" Jul 03 11:13:30.491127 ip-10-0-136-0 kubenswrapper[2579]: I0703 11:13:30.491100 2579 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["open-cluster-management-agent-addon/klusterlet-addon-workmgr-7d8dd55d49-bs5j5"] Jul 03 11:13:30.553036 ip-10-0-136-0 kubenswrapper[2579]: I0703 11:13:30.553000 2579 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-image-registry/image-registry-659bf79d58-p9gqp"] Jul 03 11:13:30.557565 ip-10-0-136-0 kubenswrapper[2579]: I0703 11:13:30.557531 2579 patch_prober.go:28] interesting pod/image-registry-659bf79d58-p9gqp container/registry namespace/openshift-image-registry: Readiness probe status=failure output="HTTP probe failed with statuscode: 503" start-of-body={"errors":[{"code":"UNAVAILABLE","message":"service unavailable","detail":"health check failed: please see /debug/health"}]} Jul 03 11:13:30.557736 ip-10-0-136-0 kubenswrapper[2579]: I0703 11:13:30.557591 2579 prober.go:120] "Probe failed" probeType="Readiness" pod="openshift-image-registry/image-registry-659bf79d58-p9gqp" podUID="387cc87f-815e-457a-83cf-d016035fed5d" containerName="registry" probeResult="failure" output="HTTP probe failed with statuscode: 503" Jul 03 11:13:30.582835 ip-10-0-136-0 kubenswrapper[2579]: I0703 11:13:30.582791 2579 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-console/downloads-685bfb5f64-cngk5"] Jul 03 11:13:30.585336 ip-10-0-136-0 kubenswrapper[2579]: I0703 11:13:30.585303 2579 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-console/downloads-685bfb5f64-cngk5" Jul 03 11:13:30.589890 ip-10-0-136-0 kubenswrapper[2579]: I0703 11:13:30.589618 2579 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-console\"/\"openshift-service-ca.crt\"" Jul 03 11:13:30.589890 ip-10-0-136-0 kubenswrapper[2579]: I0703 11:13:30.589619 2579 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-console\"/\"kube-root-ca.crt\"" Jul 03 11:13:30.590248 ip-10-0-136-0 kubenswrapper[2579]: I0703 11:13:30.589966 2579 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-console\"/\"default-dockercfg-kbz27\"" Jul 03 11:13:30.592533 ip-10-0-136-0 kubenswrapper[2579]: I0703 11:13:30.592487 2579 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tmp\" (UniqueName: \"kubernetes.io/empty-dir/f6db22ed-930d-4da4-8947-99879c35c697-tmp\") pod \"klusterlet-addon-workmgr-7d8dd55d49-bs5j5\" (UID: \"f6db22ed-930d-4da4-8947-99879c35c697\") " pod="open-cluster-management-agent-addon/klusterlet-addon-workmgr-7d8dd55d49-bs5j5" Jul 03 11:13:30.592533 ip-10-0-136-0 kubenswrapper[2579]: I0703 11:13:30.592539 2579 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"klusterlet-config\" (UniqueName: \"kubernetes.io/secret/f6db22ed-930d-4da4-8947-99879c35c697-klusterlet-config\") pod \"klusterlet-addon-workmgr-7d8dd55d49-bs5j5\" (UID: \"f6db22ed-930d-4da4-8947-99879c35c697\") " pod="open-cluster-management-agent-addon/klusterlet-addon-workmgr-7d8dd55d49-bs5j5" Jul 03 11:13:30.592753 ip-10-0-136-0 kubenswrapper[2579]: I0703 11:13:30.592608 2579 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-h7psz\" (UniqueName: \"kubernetes.io/projected/f6db22ed-930d-4da4-8947-99879c35c697-kube-api-access-h7psz\") pod \"klusterlet-addon-workmgr-7d8dd55d49-bs5j5\" (UID: \"f6db22ed-930d-4da4-8947-99879c35c697\") " pod="open-cluster-management-agent-addon/klusterlet-addon-workmgr-7d8dd55d49-bs5j5" Jul 03 11:13:30.603126 ip-10-0-136-0 kubenswrapper[2579]: I0703 11:13:30.603090 2579 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-console/downloads-685bfb5f64-cngk5"] Jul 03 11:13:30.681080 ip-10-0-136-0 kubenswrapper[2579]: I0703 11:13:30.680987 2579 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-image-registry/image-registry-74fd49c4c-5v265"] Jul 03 11:13:30.683427 ip-10-0-136-0 kubenswrapper[2579]: I0703 11:13:30.683398 2579 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-image-registry/image-registry-74fd49c4c-5v265" Jul 03 11:13:30.693868 ip-10-0-136-0 kubenswrapper[2579]: I0703 11:13:30.693834 2579 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tmp\" (UniqueName: \"kubernetes.io/empty-dir/f6db22ed-930d-4da4-8947-99879c35c697-tmp\") pod \"klusterlet-addon-workmgr-7d8dd55d49-bs5j5\" (UID: \"f6db22ed-930d-4da4-8947-99879c35c697\") " pod="open-cluster-management-agent-addon/klusterlet-addon-workmgr-7d8dd55d49-bs5j5" Jul 03 11:13:30.694022 ip-10-0-136-0 kubenswrapper[2579]: I0703 11:13:30.693882 2579 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"klusterlet-config\" (UniqueName: \"kubernetes.io/secret/f6db22ed-930d-4da4-8947-99879c35c697-klusterlet-config\") pod \"klusterlet-addon-workmgr-7d8dd55d49-bs5j5\" (UID: \"f6db22ed-930d-4da4-8947-99879c35c697\") " pod="open-cluster-management-agent-addon/klusterlet-addon-workmgr-7d8dd55d49-bs5j5" Jul 03 11:13:30.694022 ip-10-0-136-0 kubenswrapper[2579]: I0703 11:13:30.693921 2579 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-27d6k\" (UniqueName: \"kubernetes.io/projected/a74b2c4b-20bf-4e91-85b7-c827a9c92d1b-kube-api-access-27d6k\") pod \"downloads-685bfb5f64-cngk5\" (UID: \"a74b2c4b-20bf-4e91-85b7-c827a9c92d1b\") " pod="openshift-console/downloads-685bfb5f64-cngk5" Jul 03 11:13:30.694022 ip-10-0-136-0 kubenswrapper[2579]: I0703 11:13:30.693958 2579 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-h7psz\" (UniqueName: \"kubernetes.io/projected/f6db22ed-930d-4da4-8947-99879c35c697-kube-api-access-h7psz\") pod \"klusterlet-addon-workmgr-7d8dd55d49-bs5j5\" (UID: \"f6db22ed-930d-4da4-8947-99879c35c697\") " pod="open-cluster-management-agent-addon/klusterlet-addon-workmgr-7d8dd55d49-bs5j5" Jul 03 11:13:30.694334 ip-10-0-136-0 kubenswrapper[2579]: I0703 11:13:30.694307 2579 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"tmp\" (UniqueName: \"kubernetes.io/empty-dir/f6db22ed-930d-4da4-8947-99879c35c697-tmp\") pod \"klusterlet-addon-workmgr-7d8dd55d49-bs5j5\" (UID: \"f6db22ed-930d-4da4-8947-99879c35c697\") " pod="open-cluster-management-agent-addon/klusterlet-addon-workmgr-7d8dd55d49-bs5j5" Jul 03 11:13:30.697272 ip-10-0-136-0 kubenswrapper[2579]: I0703 11:13:30.697221 2579 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"klusterlet-config\" (UniqueName: \"kubernetes.io/secret/f6db22ed-930d-4da4-8947-99879c35c697-klusterlet-config\") pod \"klusterlet-addon-workmgr-7d8dd55d49-bs5j5\" (UID: \"f6db22ed-930d-4da4-8947-99879c35c697\") " pod="open-cluster-management-agent-addon/klusterlet-addon-workmgr-7d8dd55d49-bs5j5" Jul 03 11:13:30.701216 ip-10-0-136-0 kubenswrapper[2579]: I0703 11:13:30.701162 2579 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-image-registry/image-registry-74fd49c4c-5v265"] Jul 03 11:13:30.710338 ip-10-0-136-0 kubenswrapper[2579]: I0703 11:13:30.710269 2579 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-h7psz\" (UniqueName: \"kubernetes.io/projected/f6db22ed-930d-4da4-8947-99879c35c697-kube-api-access-h7psz\") pod \"klusterlet-addon-workmgr-7d8dd55d49-bs5j5\" (UID: \"f6db22ed-930d-4da4-8947-99879c35c697\") " pod="open-cluster-management-agent-addon/klusterlet-addon-workmgr-7d8dd55d49-bs5j5" Jul 03 11:13:30.789980 ip-10-0-136-0 kubenswrapper[2579]: I0703 11:13:30.789928 2579 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="open-cluster-management-agent-addon/klusterlet-addon-workmgr-7d8dd55d49-bs5j5" Jul 03 11:13:30.794608 ip-10-0-136-0 kubenswrapper[2579]: I0703 11:13:30.794554 2579 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"ca-trust-extracted\" (UniqueName: \"kubernetes.io/empty-dir/05698ac0-c3e7-45fe-a431-8ece3158f16d-ca-trust-extracted\") pod \"image-registry-74fd49c4c-5v265\" (UID: \"05698ac0-c3e7-45fe-a431-8ece3158f16d\") " pod="openshift-image-registry/image-registry-74fd49c4c-5v265" Jul 03 11:13:30.794776 ip-10-0-136-0 kubenswrapper[2579]: I0703 11:13:30.794627 2579 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-27d6k\" (UniqueName: \"kubernetes.io/projected/a74b2c4b-20bf-4e91-85b7-c827a9c92d1b-kube-api-access-27d6k\") pod \"downloads-685bfb5f64-cngk5\" (UID: \"a74b2c4b-20bf-4e91-85b7-c827a9c92d1b\") " pod="openshift-console/downloads-685bfb5f64-cngk5" Jul 03 11:13:30.794776 ip-10-0-136-0 kubenswrapper[2579]: I0703 11:13:30.794739 2579 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"registry-tls\" (UniqueName: \"kubernetes.io/projected/05698ac0-c3e7-45fe-a431-8ece3158f16d-registry-tls\") pod \"image-registry-74fd49c4c-5v265\" (UID: \"05698ac0-c3e7-45fe-a431-8ece3158f16d\") " pod="openshift-image-registry/image-registry-74fd49c4c-5v265" Jul 03 11:13:30.794884 ip-10-0-136-0 kubenswrapper[2579]: I0703 11:13:30.794777 2579 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"trusted-ca\" (UniqueName: \"kubernetes.io/configmap/05698ac0-c3e7-45fe-a431-8ece3158f16d-trusted-ca\") pod \"image-registry-74fd49c4c-5v265\" (UID: \"05698ac0-c3e7-45fe-a431-8ece3158f16d\") " pod="openshift-image-registry/image-registry-74fd49c4c-5v265" Jul 03 11:13:30.794884 ip-10-0-136-0 kubenswrapper[2579]: I0703 11:13:30.794807 2579 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"bound-sa-token\" (UniqueName: \"kubernetes.io/projected/05698ac0-c3e7-45fe-a431-8ece3158f16d-bound-sa-token\") pod \"image-registry-74fd49c4c-5v265\" (UID: \"05698ac0-c3e7-45fe-a431-8ece3158f16d\") " pod="openshift-image-registry/image-registry-74fd49c4c-5v265" Jul 03 11:13:30.794884 ip-10-0-136-0 kubenswrapper[2579]: I0703 11:13:30.794839 2579 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"installation-pull-secrets\" (UniqueName: \"kubernetes.io/secret/05698ac0-c3e7-45fe-a431-8ece3158f16d-installation-pull-secrets\") pod \"image-registry-74fd49c4c-5v265\" (UID: \"05698ac0-c3e7-45fe-a431-8ece3158f16d\") " pod="openshift-image-registry/image-registry-74fd49c4c-5v265" Jul 03 11:13:30.794884 ip-10-0-136-0 kubenswrapper[2579]: I0703 11:13:30.794865 2579 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"registry-certificates\" (UniqueName: \"kubernetes.io/configmap/05698ac0-c3e7-45fe-a431-8ece3158f16d-registry-certificates\") pod \"image-registry-74fd49c4c-5v265\" (UID: \"05698ac0-c3e7-45fe-a431-8ece3158f16d\") " pod="openshift-image-registry/image-registry-74fd49c4c-5v265" Jul 03 11:13:30.795083 ip-10-0-136-0 kubenswrapper[2579]: I0703 11:13:30.794989 2579 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"image-registry-private-configuration\" (UniqueName: \"kubernetes.io/secret/05698ac0-c3e7-45fe-a431-8ece3158f16d-image-registry-private-configuration\") pod \"image-registry-74fd49c4c-5v265\" (UID: \"05698ac0-c3e7-45fe-a431-8ece3158f16d\") " pod="openshift-image-registry/image-registry-74fd49c4c-5v265" Jul 03 11:13:30.795083 ip-10-0-136-0 kubenswrapper[2579]: I0703 11:13:30.795035 2579 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-76tsx\" (UniqueName: \"kubernetes.io/projected/05698ac0-c3e7-45fe-a431-8ece3158f16d-kube-api-access-76tsx\") pod \"image-registry-74fd49c4c-5v265\" (UID: \"05698ac0-c3e7-45fe-a431-8ece3158f16d\") " pod="openshift-image-registry/image-registry-74fd49c4c-5v265" Jul 03 11:13:30.807082 ip-10-0-136-0 kubenswrapper[2579]: I0703 11:13:30.807045 2579 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-27d6k\" (UniqueName: \"kubernetes.io/projected/a74b2c4b-20bf-4e91-85b7-c827a9c92d1b-kube-api-access-27d6k\") pod \"downloads-685bfb5f64-cngk5\" (UID: \"a74b2c4b-20bf-4e91-85b7-c827a9c92d1b\") " pod="openshift-console/downloads-685bfb5f64-cngk5" Jul 03 11:13:30.897756 ip-10-0-136-0 kubenswrapper[2579]: I0703 11:13:30.895983 2579 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"ca-trust-extracted\" (UniqueName: \"kubernetes.io/empty-dir/05698ac0-c3e7-45fe-a431-8ece3158f16d-ca-trust-extracted\") pod \"image-registry-74fd49c4c-5v265\" (UID: \"05698ac0-c3e7-45fe-a431-8ece3158f16d\") " pod="openshift-image-registry/image-registry-74fd49c4c-5v265" Jul 03 11:13:30.897756 ip-10-0-136-0 kubenswrapper[2579]: I0703 11:13:30.896068 2579 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"registry-tls\" (UniqueName: \"kubernetes.io/projected/05698ac0-c3e7-45fe-a431-8ece3158f16d-registry-tls\") pod \"image-registry-74fd49c4c-5v265\" (UID: \"05698ac0-c3e7-45fe-a431-8ece3158f16d\") " pod="openshift-image-registry/image-registry-74fd49c4c-5v265" Jul 03 11:13:30.897756 ip-10-0-136-0 kubenswrapper[2579]: I0703 11:13:30.896090 2579 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"trusted-ca\" (UniqueName: \"kubernetes.io/configmap/05698ac0-c3e7-45fe-a431-8ece3158f16d-trusted-ca\") pod \"image-registry-74fd49c4c-5v265\" (UID: \"05698ac0-c3e7-45fe-a431-8ece3158f16d\") " pod="openshift-image-registry/image-registry-74fd49c4c-5v265" Jul 03 11:13:30.897756 ip-10-0-136-0 kubenswrapper[2579]: I0703 11:13:30.896113 2579 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"bound-sa-token\" (UniqueName: \"kubernetes.io/projected/05698ac0-c3e7-45fe-a431-8ece3158f16d-bound-sa-token\") pod \"image-registry-74fd49c4c-5v265\" (UID: \"05698ac0-c3e7-45fe-a431-8ece3158f16d\") " pod="openshift-image-registry/image-registry-74fd49c4c-5v265" Jul 03 11:13:30.897756 ip-10-0-136-0 kubenswrapper[2579]: I0703 11:13:30.896145 2579 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"installation-pull-secrets\" (UniqueName: \"kubernetes.io/secret/05698ac0-c3e7-45fe-a431-8ece3158f16d-installation-pull-secrets\") pod \"image-registry-74fd49c4c-5v265\" (UID: \"05698ac0-c3e7-45fe-a431-8ece3158f16d\") " pod="openshift-image-registry/image-registry-74fd49c4c-5v265" Jul 03 11:13:30.897756 ip-10-0-136-0 kubenswrapper[2579]: I0703 11:13:30.896163 2579 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"registry-certificates\" (UniqueName: \"kubernetes.io/configmap/05698ac0-c3e7-45fe-a431-8ece3158f16d-registry-certificates\") pod \"image-registry-74fd49c4c-5v265\" (UID: \"05698ac0-c3e7-45fe-a431-8ece3158f16d\") " pod="openshift-image-registry/image-registry-74fd49c4c-5v265" Jul 03 11:13:30.897756 ip-10-0-136-0 kubenswrapper[2579]: I0703 11:13:30.896240 2579 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"image-registry-private-configuration\" (UniqueName: \"kubernetes.io/secret/05698ac0-c3e7-45fe-a431-8ece3158f16d-image-registry-private-configuration\") pod \"image-registry-74fd49c4c-5v265\" (UID: \"05698ac0-c3e7-45fe-a431-8ece3158f16d\") " pod="openshift-image-registry/image-registry-74fd49c4c-5v265" Jul 03 11:13:30.897756 ip-10-0-136-0 kubenswrapper[2579]: I0703 11:13:30.896269 2579 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-76tsx\" (UniqueName: \"kubernetes.io/projected/05698ac0-c3e7-45fe-a431-8ece3158f16d-kube-api-access-76tsx\") pod \"image-registry-74fd49c4c-5v265\" (UID: \"05698ac0-c3e7-45fe-a431-8ece3158f16d\") " pod="openshift-image-registry/image-registry-74fd49c4c-5v265" Jul 03 11:13:30.897756 ip-10-0-136-0 kubenswrapper[2579]: I0703 11:13:30.897027 2579 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"ca-trust-extracted\" (UniqueName: \"kubernetes.io/empty-dir/05698ac0-c3e7-45fe-a431-8ece3158f16d-ca-trust-extracted\") pod \"image-registry-74fd49c4c-5v265\" (UID: \"05698ac0-c3e7-45fe-a431-8ece3158f16d\") " pod="openshift-image-registry/image-registry-74fd49c4c-5v265" Jul 03 11:13:30.897756 ip-10-0-136-0 kubenswrapper[2579]: I0703 11:13:30.897147 2579 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-console/downloads-685bfb5f64-cngk5" Jul 03 11:13:30.900311 ip-10-0-136-0 kubenswrapper[2579]: I0703 11:13:30.899296 2579 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"trusted-ca\" (UniqueName: \"kubernetes.io/configmap/05698ac0-c3e7-45fe-a431-8ece3158f16d-trusted-ca\") pod \"image-registry-74fd49c4c-5v265\" (UID: \"05698ac0-c3e7-45fe-a431-8ece3158f16d\") " pod="openshift-image-registry/image-registry-74fd49c4c-5v265" Jul 03 11:13:30.900311 ip-10-0-136-0 kubenswrapper[2579]: I0703 11:13:30.900003 2579 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"registry-certificates\" (UniqueName: \"kubernetes.io/configmap/05698ac0-c3e7-45fe-a431-8ece3158f16d-registry-certificates\") pod \"image-registry-74fd49c4c-5v265\" (UID: \"05698ac0-c3e7-45fe-a431-8ece3158f16d\") " pod="openshift-image-registry/image-registry-74fd49c4c-5v265" Jul 03 11:13:30.900765 ip-10-0-136-0 kubenswrapper[2579]: I0703 11:13:30.900737 2579 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"image-registry-private-configuration\" (UniqueName: \"kubernetes.io/secret/05698ac0-c3e7-45fe-a431-8ece3158f16d-image-registry-private-configuration\") pod \"image-registry-74fd49c4c-5v265\" (UID: \"05698ac0-c3e7-45fe-a431-8ece3158f16d\") " pod="openshift-image-registry/image-registry-74fd49c4c-5v265" Jul 03 11:13:30.901393 ip-10-0-136-0 kubenswrapper[2579]: I0703 11:13:30.901360 2579 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"installation-pull-secrets\" (UniqueName: \"kubernetes.io/secret/05698ac0-c3e7-45fe-a431-8ece3158f16d-installation-pull-secrets\") pod \"image-registry-74fd49c4c-5v265\" (UID: \"05698ac0-c3e7-45fe-a431-8ece3158f16d\") " pod="openshift-image-registry/image-registry-74fd49c4c-5v265" Jul 03 11:13:30.905250 ip-10-0-136-0 kubenswrapper[2579]: I0703 11:13:30.904699 2579 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"registry-tls\" (UniqueName: \"kubernetes.io/projected/05698ac0-c3e7-45fe-a431-8ece3158f16d-registry-tls\") pod \"image-registry-74fd49c4c-5v265\" (UID: \"05698ac0-c3e7-45fe-a431-8ece3158f16d\") " pod="openshift-image-registry/image-registry-74fd49c4c-5v265" Jul 03 11:13:30.911177 ip-10-0-136-0 kubenswrapper[2579]: I0703 11:13:30.911139 2579 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-76tsx\" (UniqueName: \"kubernetes.io/projected/05698ac0-c3e7-45fe-a431-8ece3158f16d-kube-api-access-76tsx\") pod \"image-registry-74fd49c4c-5v265\" (UID: \"05698ac0-c3e7-45fe-a431-8ece3158f16d\") " pod="openshift-image-registry/image-registry-74fd49c4c-5v265" Jul 03 11:13:30.911846 ip-10-0-136-0 kubenswrapper[2579]: I0703 11:13:30.911784 2579 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"bound-sa-token\" (UniqueName: \"kubernetes.io/projected/05698ac0-c3e7-45fe-a431-8ece3158f16d-bound-sa-token\") pod \"image-registry-74fd49c4c-5v265\" (UID: \"05698ac0-c3e7-45fe-a431-8ece3158f16d\") " pod="openshift-image-registry/image-registry-74fd49c4c-5v265" Jul 03 11:13:30.968478 ip-10-0-136-0 kubenswrapper[2579]: I0703 11:13:30.968380 2579 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["open-cluster-management-agent-addon/klusterlet-addon-workmgr-7d8dd55d49-bs5j5"] Jul 03 11:13:30.995372 ip-10-0-136-0 kubenswrapper[2579]: I0703 11:13:30.995334 2579 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-image-registry/image-registry-74fd49c4c-5v265" Jul 03 11:13:31.220398 ip-10-0-136-0 kubenswrapper[2579]: W0703 11:13:31.220314 2579 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podf6db22ed_930d_4da4_8947_99879c35c697.slice/crio-f5392804fb292703a7a6eb9d87f8395ef0c8e54d574885d9b8b49b5f26b6213a WatchSource:0}: Error finding container f5392804fb292703a7a6eb9d87f8395ef0c8e54d574885d9b8b49b5f26b6213a: Status 404 returned error can't find the container with id f5392804fb292703a7a6eb9d87f8395ef0c8e54d574885d9b8b49b5f26b6213a Jul 03 11:13:31.408254 ip-10-0-136-0 kubenswrapper[2579]: I0703 11:13:31.408121 2579 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-image-registry/image-registry-74fd49c4c-5v265"] Jul 03 11:13:31.419290 ip-10-0-136-0 kubenswrapper[2579]: W0703 11:13:31.417486 2579 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod05698ac0_c3e7_45fe_a431_8ece3158f16d.slice/crio-576a6cc343d19c4e56477f3cfbdc6dd0bf800aef67071d2dbb7129605a2c246b WatchSource:0}: Error finding container 576a6cc343d19c4e56477f3cfbdc6dd0bf800aef67071d2dbb7129605a2c246b: Status 404 returned error can't find the container with id 576a6cc343d19c4e56477f3cfbdc6dd0bf800aef67071d2dbb7129605a2c246b Jul 03 11:13:31.428845 ip-10-0-136-0 kubenswrapper[2579]: I0703 11:13:31.428781 2579 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-console/downloads-685bfb5f64-cngk5"] Jul 03 11:13:31.457595 ip-10-0-136-0 kubenswrapper[2579]: I0703 11:13:31.457482 2579 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-image-registry/image-registry-74fd49c4c-5v265" event={"ID":"05698ac0-c3e7-45fe-a431-8ece3158f16d","Type":"ContainerStarted","Data":"576a6cc343d19c4e56477f3cfbdc6dd0bf800aef67071d2dbb7129605a2c246b"} Jul 03 11:13:31.459871 ip-10-0-136-0 kubenswrapper[2579]: I0703 11:13:31.459507 2579 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="open-cluster-management-agent-addon/klusterlet-addon-workmgr-7d8dd55d49-bs5j5" event={"ID":"f6db22ed-930d-4da4-8947-99879c35c697","Type":"ContainerStarted","Data":"f5392804fb292703a7a6eb9d87f8395ef0c8e54d574885d9b8b49b5f26b6213a"} Jul 03 11:13:32.471364 ip-10-0-136-0 kubenswrapper[2579]: I0703 11:13:32.471321 2579 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-image-registry/image-registry-74fd49c4c-5v265" event={"ID":"05698ac0-c3e7-45fe-a431-8ece3158f16d","Type":"ContainerStarted","Data":"bbd8518501bbc7ec3338614498cc94fab45941351bc1ea9630a1650b03565abb"} Jul 03 11:13:32.472226 ip-10-0-136-0 kubenswrapper[2579]: I0703 11:13:32.472179 2579 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-image-registry/image-registry-74fd49c4c-5v265" Jul 03 11:13:32.475501 ip-10-0-136-0 kubenswrapper[2579]: I0703 11:13:32.475462 2579 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-console/downloads-685bfb5f64-cngk5" event={"ID":"a74b2c4b-20bf-4e91-85b7-c827a9c92d1b","Type":"ContainerStarted","Data":"a694368f33ff7fb175340680ac7e2172c647dd171c0fab3cb72335a58eb98ea4"} Jul 03 11:13:32.484885 ip-10-0-136-0 kubenswrapper[2579]: I0703 11:13:32.484126 2579 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-multus/network-metrics-daemon-sqxsj" event={"ID":"34ca9c51-4743-4efc-9aa1-eb79a741f6a4","Type":"ContainerStarted","Data":"802b8675e1f7a0c2102b71dc47aadf92da37d1fdbb439de415199f15f37c6180"} Jul 03 11:13:32.484885 ip-10-0-136-0 kubenswrapper[2579]: I0703 11:13:32.484240 2579 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-multus/network-metrics-daemon-sqxsj" event={"ID":"34ca9c51-4743-4efc-9aa1-eb79a741f6a4","Type":"ContainerStarted","Data":"42a357f4e588580b6b0fc8cae31317558d86fb56357a6b4696a68d15641a303f"} Jul 03 11:13:32.496867 ip-10-0-136-0 kubenswrapper[2579]: I0703 11:13:32.495385 2579 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-image-registry/image-registry-74fd49c4c-5v265" podStartSLOduration=2.495365149 podStartE2EDuration="2.495365149s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-03 11:13:30 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-03 11:13:32.494895413 +0000 UTC m=+69.083199733" watchObservedRunningTime="2026-07-03 11:13:32.495365149 +0000 UTC m=+69.083669467" Jul 03 11:13:32.514869 ip-10-0-136-0 kubenswrapper[2579]: I0703 11:13:32.514805 2579 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-multus/network-metrics-daemon-sqxsj" podStartSLOduration=67.1956301 podStartE2EDuration="1m8.514782566s" totalImagesPullingTime="1.319152466s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-03 11:12:24 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-03 11:13:32.513581184 +0000 UTC m=+69.101885508" watchObservedRunningTime="2026-07-03 11:13:32.514782566 +0000 UTC m=+69.103086885" Jul 03 11:13:33.454563 ip-10-0-136-0 kubenswrapper[2579]: I0703 11:13:33.454520 2579 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-console/console-55fd9ddfdd-klc4c"] Jul 03 11:13:33.458063 ip-10-0-136-0 kubenswrapper[2579]: I0703 11:13:33.458033 2579 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-console/console-55fd9ddfdd-klc4c" Jul 03 11:13:33.462611 ip-10-0-136-0 kubenswrapper[2579]: I0703 11:13:33.462049 2579 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-console\"/\"oauth-serving-cert\"" Jul 03 11:13:33.467222 ip-10-0-136-0 kubenswrapper[2579]: I0703 11:13:33.464000 2579 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-console\"/\"service-ca\"" Jul 03 11:13:33.467222 ip-10-0-136-0 kubenswrapper[2579]: I0703 11:13:33.464427 2579 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-console\"/\"console-serving-cert\"" Jul 03 11:13:33.467222 ip-10-0-136-0 kubenswrapper[2579]: I0703 11:13:33.464635 2579 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-console\"/\"console-oauth-config\"" Jul 03 11:13:33.467222 ip-10-0-136-0 kubenswrapper[2579]: I0703 11:13:33.464795 2579 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-console\"/\"console-config\"" Jul 03 11:13:33.467222 ip-10-0-136-0 kubenswrapper[2579]: I0703 11:13:33.464964 2579 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-console\"/\"console-dockercfg-49rmj\"" Jul 03 11:13:33.468255 ip-10-0-136-0 kubenswrapper[2579]: I0703 11:13:33.468233 2579 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-console/console-55fd9ddfdd-klc4c"] Jul 03 11:13:33.523848 ip-10-0-136-0 kubenswrapper[2579]: I0703 11:13:33.523524 2579 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"console-serving-cert\" (UniqueName: \"kubernetes.io/secret/8937bcc1-fce2-40e9-ac92-e25c9a902506-console-serving-cert\") pod \"console-55fd9ddfdd-klc4c\" (UID: \"8937bcc1-fce2-40e9-ac92-e25c9a902506\") " pod="openshift-console/console-55fd9ddfdd-klc4c" Jul 03 11:13:33.523848 ip-10-0-136-0 kubenswrapper[2579]: I0703 11:13:33.523587 2579 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"oauth-serving-cert\" (UniqueName: \"kubernetes.io/configmap/8937bcc1-fce2-40e9-ac92-e25c9a902506-oauth-serving-cert\") pod \"console-55fd9ddfdd-klc4c\" (UID: \"8937bcc1-fce2-40e9-ac92-e25c9a902506\") " pod="openshift-console/console-55fd9ddfdd-klc4c" Jul 03 11:13:33.523848 ip-10-0-136-0 kubenswrapper[2579]: I0703 11:13:33.523641 2579 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"service-ca\" (UniqueName: \"kubernetes.io/configmap/8937bcc1-fce2-40e9-ac92-e25c9a902506-service-ca\") pod \"console-55fd9ddfdd-klc4c\" (UID: \"8937bcc1-fce2-40e9-ac92-e25c9a902506\") " pod="openshift-console/console-55fd9ddfdd-klc4c" Jul 03 11:13:33.523848 ip-10-0-136-0 kubenswrapper[2579]: I0703 11:13:33.523681 2579 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-z5mct\" (UniqueName: \"kubernetes.io/projected/8937bcc1-fce2-40e9-ac92-e25c9a902506-kube-api-access-z5mct\") pod \"console-55fd9ddfdd-klc4c\" (UID: \"8937bcc1-fce2-40e9-ac92-e25c9a902506\") " pod="openshift-console/console-55fd9ddfdd-klc4c" Jul 03 11:13:33.523848 ip-10-0-136-0 kubenswrapper[2579]: I0703 11:13:33.523722 2579 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"console-config\" (UniqueName: \"kubernetes.io/configmap/8937bcc1-fce2-40e9-ac92-e25c9a902506-console-config\") pod \"console-55fd9ddfdd-klc4c\" (UID: \"8937bcc1-fce2-40e9-ac92-e25c9a902506\") " pod="openshift-console/console-55fd9ddfdd-klc4c" Jul 03 11:13:33.523848 ip-10-0-136-0 kubenswrapper[2579]: I0703 11:13:33.523748 2579 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"console-oauth-config\" (UniqueName: \"kubernetes.io/secret/8937bcc1-fce2-40e9-ac92-e25c9a902506-console-oauth-config\") pod \"console-55fd9ddfdd-klc4c\" (UID: \"8937bcc1-fce2-40e9-ac92-e25c9a902506\") " pod="openshift-console/console-55fd9ddfdd-klc4c" Jul 03 11:13:33.625116 ip-10-0-136-0 kubenswrapper[2579]: I0703 11:13:33.625072 2579 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-z5mct\" (UniqueName: \"kubernetes.io/projected/8937bcc1-fce2-40e9-ac92-e25c9a902506-kube-api-access-z5mct\") pod \"console-55fd9ddfdd-klc4c\" (UID: \"8937bcc1-fce2-40e9-ac92-e25c9a902506\") " pod="openshift-console/console-55fd9ddfdd-klc4c" Jul 03 11:13:33.625328 ip-10-0-136-0 kubenswrapper[2579]: I0703 11:13:33.625148 2579 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"console-config\" (UniqueName: \"kubernetes.io/configmap/8937bcc1-fce2-40e9-ac92-e25c9a902506-console-config\") pod \"console-55fd9ddfdd-klc4c\" (UID: \"8937bcc1-fce2-40e9-ac92-e25c9a902506\") " pod="openshift-console/console-55fd9ddfdd-klc4c" Jul 03 11:13:33.625328 ip-10-0-136-0 kubenswrapper[2579]: I0703 11:13:33.625220 2579 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"console-oauth-config\" (UniqueName: \"kubernetes.io/secret/8937bcc1-fce2-40e9-ac92-e25c9a902506-console-oauth-config\") pod \"console-55fd9ddfdd-klc4c\" (UID: \"8937bcc1-fce2-40e9-ac92-e25c9a902506\") " pod="openshift-console/console-55fd9ddfdd-klc4c" Jul 03 11:13:33.625328 ip-10-0-136-0 kubenswrapper[2579]: I0703 11:13:33.625308 2579 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"console-serving-cert\" (UniqueName: \"kubernetes.io/secret/8937bcc1-fce2-40e9-ac92-e25c9a902506-console-serving-cert\") pod \"console-55fd9ddfdd-klc4c\" (UID: \"8937bcc1-fce2-40e9-ac92-e25c9a902506\") " pod="openshift-console/console-55fd9ddfdd-klc4c" Jul 03 11:13:33.625449 ip-10-0-136-0 kubenswrapper[2579]: I0703 11:13:33.625384 2579 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"oauth-serving-cert\" (UniqueName: \"kubernetes.io/configmap/8937bcc1-fce2-40e9-ac92-e25c9a902506-oauth-serving-cert\") pod \"console-55fd9ddfdd-klc4c\" (UID: \"8937bcc1-fce2-40e9-ac92-e25c9a902506\") " pod="openshift-console/console-55fd9ddfdd-klc4c" Jul 03 11:13:33.628003 ip-10-0-136-0 kubenswrapper[2579]: I0703 11:13:33.625514 2579 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"service-ca\" (UniqueName: \"kubernetes.io/configmap/8937bcc1-fce2-40e9-ac92-e25c9a902506-service-ca\") pod \"console-55fd9ddfdd-klc4c\" (UID: \"8937bcc1-fce2-40e9-ac92-e25c9a902506\") " pod="openshift-console/console-55fd9ddfdd-klc4c" Jul 03 11:13:33.628003 ip-10-0-136-0 kubenswrapper[2579]: I0703 11:13:33.626441 2579 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"service-ca\" (UniqueName: \"kubernetes.io/configmap/8937bcc1-fce2-40e9-ac92-e25c9a902506-service-ca\") pod \"console-55fd9ddfdd-klc4c\" (UID: \"8937bcc1-fce2-40e9-ac92-e25c9a902506\") " pod="openshift-console/console-55fd9ddfdd-klc4c" Jul 03 11:13:33.628003 ip-10-0-136-0 kubenswrapper[2579]: I0703 11:13:33.627501 2579 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"console-config\" (UniqueName: \"kubernetes.io/configmap/8937bcc1-fce2-40e9-ac92-e25c9a902506-console-config\") pod \"console-55fd9ddfdd-klc4c\" (UID: \"8937bcc1-fce2-40e9-ac92-e25c9a902506\") " pod="openshift-console/console-55fd9ddfdd-klc4c" Jul 03 11:13:33.631633 ip-10-0-136-0 kubenswrapper[2579]: I0703 11:13:33.630727 2579 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"oauth-serving-cert\" (UniqueName: \"kubernetes.io/configmap/8937bcc1-fce2-40e9-ac92-e25c9a902506-oauth-serving-cert\") pod \"console-55fd9ddfdd-klc4c\" (UID: \"8937bcc1-fce2-40e9-ac92-e25c9a902506\") " pod="openshift-console/console-55fd9ddfdd-klc4c" Jul 03 11:13:33.634143 ip-10-0-136-0 kubenswrapper[2579]: I0703 11:13:33.634067 2579 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"console-serving-cert\" (UniqueName: \"kubernetes.io/secret/8937bcc1-fce2-40e9-ac92-e25c9a902506-console-serving-cert\") pod \"console-55fd9ddfdd-klc4c\" (UID: \"8937bcc1-fce2-40e9-ac92-e25c9a902506\") " pod="openshift-console/console-55fd9ddfdd-klc4c" Jul 03 11:13:33.635949 ip-10-0-136-0 kubenswrapper[2579]: I0703 11:13:33.635915 2579 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"console-oauth-config\" (UniqueName: \"kubernetes.io/secret/8937bcc1-fce2-40e9-ac92-e25c9a902506-console-oauth-config\") pod \"console-55fd9ddfdd-klc4c\" (UID: \"8937bcc1-fce2-40e9-ac92-e25c9a902506\") " pod="openshift-console/console-55fd9ddfdd-klc4c" Jul 03 11:13:33.636355 ip-10-0-136-0 kubenswrapper[2579]: I0703 11:13:33.636303 2579 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-z5mct\" (UniqueName: \"kubernetes.io/projected/8937bcc1-fce2-40e9-ac92-e25c9a902506-kube-api-access-z5mct\") pod \"console-55fd9ddfdd-klc4c\" (UID: \"8937bcc1-fce2-40e9-ac92-e25c9a902506\") " pod="openshift-console/console-55fd9ddfdd-klc4c" Jul 03 11:13:33.776478 ip-10-0-136-0 kubenswrapper[2579]: I0703 11:13:33.776438 2579 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-console/console-55fd9ddfdd-klc4c" Jul 03 11:13:37.035246 ip-10-0-136-0 kubenswrapper[2579]: I0703 11:13:37.033950 2579 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-monitoring/prometheus-operator-567f7b8b7c-dft8d"] Jul 03 11:13:37.037808 ip-10-0-136-0 kubenswrapper[2579]: I0703 11:13:37.037781 2579 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-monitoring/prometheus-operator-567f7b8b7c-dft8d" Jul 03 11:13:37.040481 ip-10-0-136-0 kubenswrapper[2579]: I0703 11:13:37.040452 2579 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"prometheus-operator-kube-rbac-proxy-config\"" Jul 03 11:13:37.041549 ip-10-0-136-0 kubenswrapper[2579]: I0703 11:13:37.041517 2579 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-monitoring\"/\"openshift-service-ca.crt\"" Jul 03 11:13:37.041681 ip-10-0-136-0 kubenswrapper[2579]: I0703 11:13:37.041547 2579 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-monitoring\"/\"kube-root-ca.crt\"" Jul 03 11:13:37.041681 ip-10-0-136-0 kubenswrapper[2579]: I0703 11:13:37.041551 2579 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-monitoring\"/\"metrics-client-ca\"" Jul 03 11:13:37.041770 ip-10-0-136-0 kubenswrapper[2579]: I0703 11:13:37.041759 2579 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"prometheus-operator-dockercfg-snpb4\"" Jul 03 11:13:37.041927 ip-10-0-136-0 kubenswrapper[2579]: I0703 11:13:37.041909 2579 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"prometheus-operator-tls\"" Jul 03 11:13:37.046599 ip-10-0-136-0 kubenswrapper[2579]: I0703 11:13:37.046565 2579 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-monitoring/prometheus-operator-567f7b8b7c-dft8d"] Jul 03 11:13:37.158007 ip-10-0-136-0 kubenswrapper[2579]: I0703 11:13:37.157739 2579 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-slxnv\" (UniqueName: \"kubernetes.io/projected/4ab5f9bd-3890-4562-895a-84485c66f4ae-kube-api-access-slxnv\") pod \"prometheus-operator-567f7b8b7c-dft8d\" (UID: \"4ab5f9bd-3890-4562-895a-84485c66f4ae\") " pod="openshift-monitoring/prometheus-operator-567f7b8b7c-dft8d" Jul 03 11:13:37.158007 ip-10-0-136-0 kubenswrapper[2579]: I0703 11:13:37.157801 2579 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"prometheus-operator-tls\" (UniqueName: \"kubernetes.io/secret/4ab5f9bd-3890-4562-895a-84485c66f4ae-prometheus-operator-tls\") pod \"prometheus-operator-567f7b8b7c-dft8d\" (UID: \"4ab5f9bd-3890-4562-895a-84485c66f4ae\") " pod="openshift-monitoring/prometheus-operator-567f7b8b7c-dft8d" Jul 03 11:13:37.158007 ip-10-0-136-0 kubenswrapper[2579]: I0703 11:13:37.157833 2579 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"metrics-client-ca\" (UniqueName: \"kubernetes.io/configmap/4ab5f9bd-3890-4562-895a-84485c66f4ae-metrics-client-ca\") pod \"prometheus-operator-567f7b8b7c-dft8d\" (UID: \"4ab5f9bd-3890-4562-895a-84485c66f4ae\") " pod="openshift-monitoring/prometheus-operator-567f7b8b7c-dft8d" Jul 03 11:13:37.158007 ip-10-0-136-0 kubenswrapper[2579]: I0703 11:13:37.157892 2579 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"prometheus-operator-kube-rbac-proxy-config\" (UniqueName: \"kubernetes.io/secret/4ab5f9bd-3890-4562-895a-84485c66f4ae-prometheus-operator-kube-rbac-proxy-config\") pod \"prometheus-operator-567f7b8b7c-dft8d\" (UID: \"4ab5f9bd-3890-4562-895a-84485c66f4ae\") " pod="openshift-monitoring/prometheus-operator-567f7b8b7c-dft8d" Jul 03 11:13:37.258417 ip-10-0-136-0 kubenswrapper[2579]: I0703 11:13:37.258375 2579 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-slxnv\" (UniqueName: \"kubernetes.io/projected/4ab5f9bd-3890-4562-895a-84485c66f4ae-kube-api-access-slxnv\") pod \"prometheus-operator-567f7b8b7c-dft8d\" (UID: \"4ab5f9bd-3890-4562-895a-84485c66f4ae\") " pod="openshift-monitoring/prometheus-operator-567f7b8b7c-dft8d" Jul 03 11:13:37.258631 ip-10-0-136-0 kubenswrapper[2579]: I0703 11:13:37.258440 2579 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"prometheus-operator-tls\" (UniqueName: \"kubernetes.io/secret/4ab5f9bd-3890-4562-895a-84485c66f4ae-prometheus-operator-tls\") pod \"prometheus-operator-567f7b8b7c-dft8d\" (UID: \"4ab5f9bd-3890-4562-895a-84485c66f4ae\") " pod="openshift-monitoring/prometheus-operator-567f7b8b7c-dft8d" Jul 03 11:13:37.258631 ip-10-0-136-0 kubenswrapper[2579]: I0703 11:13:37.258473 2579 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-client-ca\" (UniqueName: \"kubernetes.io/configmap/4ab5f9bd-3890-4562-895a-84485c66f4ae-metrics-client-ca\") pod \"prometheus-operator-567f7b8b7c-dft8d\" (UID: \"4ab5f9bd-3890-4562-895a-84485c66f4ae\") " pod="openshift-monitoring/prometheus-operator-567f7b8b7c-dft8d" Jul 03 11:13:37.258758 ip-10-0-136-0 kubenswrapper[2579]: E0703 11:13:37.258630 2579 secret.go:189] Couldn't get secret openshift-monitoring/prometheus-operator-tls: secret "prometheus-operator-tls" not found Jul 03 11:13:37.258758 ip-10-0-136-0 kubenswrapper[2579]: E0703 11:13:37.258694 2579 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/4ab5f9bd-3890-4562-895a-84485c66f4ae-prometheus-operator-tls podName:4ab5f9bd-3890-4562-895a-84485c66f4ae nodeName:}" failed. No retries permitted until 2026-07-03 11:13:37.758677447 +0000 UTC m=+74.346981748 (durationBeforeRetry 500ms). Error: MountVolume.SetUp failed for volume "prometheus-operator-tls" (UniqueName: "kubernetes.io/secret/4ab5f9bd-3890-4562-895a-84485c66f4ae-prometheus-operator-tls") pod "prometheus-operator-567f7b8b7c-dft8d" (UID: "4ab5f9bd-3890-4562-895a-84485c66f4ae") : secret "prometheus-operator-tls" not found Jul 03 11:13:37.258867 ip-10-0-136-0 kubenswrapper[2579]: I0703 11:13:37.258796 2579 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"prometheus-operator-kube-rbac-proxy-config\" (UniqueName: \"kubernetes.io/secret/4ab5f9bd-3890-4562-895a-84485c66f4ae-prometheus-operator-kube-rbac-proxy-config\") pod \"prometheus-operator-567f7b8b7c-dft8d\" (UID: \"4ab5f9bd-3890-4562-895a-84485c66f4ae\") " pod="openshift-monitoring/prometheus-operator-567f7b8b7c-dft8d" Jul 03 11:13:37.259316 ip-10-0-136-0 kubenswrapper[2579]: I0703 11:13:37.259289 2579 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"metrics-client-ca\" (UniqueName: \"kubernetes.io/configmap/4ab5f9bd-3890-4562-895a-84485c66f4ae-metrics-client-ca\") pod \"prometheus-operator-567f7b8b7c-dft8d\" (UID: \"4ab5f9bd-3890-4562-895a-84485c66f4ae\") " pod="openshift-monitoring/prometheus-operator-567f7b8b7c-dft8d" Jul 03 11:13:37.265148 ip-10-0-136-0 kubenswrapper[2579]: I0703 11:13:37.265071 2579 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"prometheus-operator-kube-rbac-proxy-config\" (UniqueName: \"kubernetes.io/secret/4ab5f9bd-3890-4562-895a-84485c66f4ae-prometheus-operator-kube-rbac-proxy-config\") pod \"prometheus-operator-567f7b8b7c-dft8d\" (UID: \"4ab5f9bd-3890-4562-895a-84485c66f4ae\") " pod="openshift-monitoring/prometheus-operator-567f7b8b7c-dft8d" Jul 03 11:13:37.269144 ip-10-0-136-0 kubenswrapper[2579]: I0703 11:13:37.269113 2579 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-slxnv\" (UniqueName: \"kubernetes.io/projected/4ab5f9bd-3890-4562-895a-84485c66f4ae-kube-api-access-slxnv\") pod \"prometheus-operator-567f7b8b7c-dft8d\" (UID: \"4ab5f9bd-3890-4562-895a-84485c66f4ae\") " pod="openshift-monitoring/prometheus-operator-567f7b8b7c-dft8d" Jul 03 11:13:37.764138 ip-10-0-136-0 kubenswrapper[2579]: I0703 11:13:37.764064 2579 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"prometheus-operator-tls\" (UniqueName: \"kubernetes.io/secret/4ab5f9bd-3890-4562-895a-84485c66f4ae-prometheus-operator-tls\") pod \"prometheus-operator-567f7b8b7c-dft8d\" (UID: \"4ab5f9bd-3890-4562-895a-84485c66f4ae\") " pod="openshift-monitoring/prometheus-operator-567f7b8b7c-dft8d" Jul 03 11:13:37.767499 ip-10-0-136-0 kubenswrapper[2579]: I0703 11:13:37.767456 2579 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"prometheus-operator-tls\" (UniqueName: \"kubernetes.io/secret/4ab5f9bd-3890-4562-895a-84485c66f4ae-prometheus-operator-tls\") pod \"prometheus-operator-567f7b8b7c-dft8d\" (UID: \"4ab5f9bd-3890-4562-895a-84485c66f4ae\") " pod="openshift-monitoring/prometheus-operator-567f7b8b7c-dft8d" Jul 03 11:13:37.952531 ip-10-0-136-0 kubenswrapper[2579]: I0703 11:13:37.952491 2579 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-monitoring/prometheus-operator-567f7b8b7c-dft8d" Jul 03 11:13:38.868206 ip-10-0-136-0 kubenswrapper[2579]: I0703 11:13:38.868099 2579 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-console/console-55fd9ddfdd-klc4c"] Jul 03 11:13:38.875208 ip-10-0-136-0 kubenswrapper[2579]: W0703 11:13:38.875107 2579 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod8937bcc1_fce2_40e9_ac92_e25c9a902506.slice/crio-bc9e330832ada0a1092b3588970773a177df4472f8dc40cfc9c69bed53b4ed66 WatchSource:0}: Error finding container bc9e330832ada0a1092b3588970773a177df4472f8dc40cfc9c69bed53b4ed66: Status 404 returned error can't find the container with id bc9e330832ada0a1092b3588970773a177df4472f8dc40cfc9c69bed53b4ed66 Jul 03 11:13:38.886650 ip-10-0-136-0 kubenswrapper[2579]: I0703 11:13:38.885411 2579 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-monitoring/prometheus-operator-567f7b8b7c-dft8d"] Jul 03 11:13:38.890296 ip-10-0-136-0 kubenswrapper[2579]: W0703 11:13:38.887896 2579 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod4ab5f9bd_3890_4562_895a_84485c66f4ae.slice/crio-a4ed194a3992adf20f7c7c923762ca567547874345ff9e962da695003ca2b688 WatchSource:0}: Error finding container a4ed194a3992adf20f7c7c923762ca567547874345ff9e962da695003ca2b688: Status 404 returned error can't find the container with id a4ed194a3992adf20f7c7c923762ca567547874345ff9e962da695003ca2b688 Jul 03 11:13:39.521219 ip-10-0-136-0 kubenswrapper[2579]: I0703 11:13:39.521144 2579 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-console/console-55fd9ddfdd-klc4c" event={"ID":"8937bcc1-fce2-40e9-ac92-e25c9a902506","Type":"ContainerStarted","Data":"bc9e330832ada0a1092b3588970773a177df4472f8dc40cfc9c69bed53b4ed66"} Jul 03 11:13:39.523351 ip-10-0-136-0 kubenswrapper[2579]: I0703 11:13:39.523308 2579 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kube-system/global-pull-secret-syncer-9pqm8" event={"ID":"7813f158-0f67-411e-84d9-a2a1c51eea61","Type":"ContainerStarted","Data":"8987910b3928a5b6eb85464372b1db780d41140a99a8adeab85cb88aeb111ad5"} Jul 03 11:13:39.525434 ip-10-0-136-0 kubenswrapper[2579]: I0703 11:13:39.525400 2579 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/prometheus-operator-567f7b8b7c-dft8d" event={"ID":"4ab5f9bd-3890-4562-895a-84485c66f4ae","Type":"ContainerStarted","Data":"a4ed194a3992adf20f7c7c923762ca567547874345ff9e962da695003ca2b688"} Jul 03 11:13:39.528486 ip-10-0-136-0 kubenswrapper[2579]: I0703 11:13:39.528436 2579 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-network-diagnostics/network-check-target-xzlsc" event={"ID":"2129ac30-1645-4555-b722-55963ffb8776","Type":"ContainerStarted","Data":"5823c7777c61826f3f6c4777a2858a711ae46eb83207752770bc6f7675092383"} Jul 03 11:13:39.528774 ip-10-0-136-0 kubenswrapper[2579]: I0703 11:13:39.528606 2579 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-network-diagnostics/network-check-target-xzlsc" Jul 03 11:13:39.531148 ip-10-0-136-0 kubenswrapper[2579]: I0703 11:13:39.531111 2579 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="open-cluster-management-agent-addon/klusterlet-addon-workmgr-7d8dd55d49-bs5j5" event={"ID":"f6db22ed-930d-4da4-8947-99879c35c697","Type":"ContainerStarted","Data":"bfde6263f2f3e20e222f7306186e15d81411c9e704f113152e827b76d2e65eae"} Jul 03 11:13:39.531344 ip-10-0-136-0 kubenswrapper[2579]: I0703 11:13:39.531318 2579 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="open-cluster-management-agent-addon/klusterlet-addon-workmgr-7d8dd55d49-bs5j5" Jul 03 11:13:39.533445 ip-10-0-136-0 kubenswrapper[2579]: I0703 11:13:39.533397 2579 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="open-cluster-management-agent-addon/klusterlet-addon-workmgr-7d8dd55d49-bs5j5" Jul 03 11:13:39.540371 ip-10-0-136-0 kubenswrapper[2579]: I0703 11:13:39.540272 2579 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="kube-system/global-pull-secret-syncer-9pqm8" podStartSLOduration=67.133994783 podStartE2EDuration="1m15.540251258s" totalImagesPullingTime="8.406256475s" totalInitContainerRuntime="0s" isStatefulPod=true podCreationTimestamp="2026-07-03 11:12:24 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-03 11:13:39.5394672 +0000 UTC m=+76.127771522" watchObservedRunningTime="2026-07-03 11:13:39.540251258 +0000 UTC m=+76.128555579" Jul 03 11:13:39.558718 ip-10-0-136-0 kubenswrapper[2579]: I0703 11:13:39.558650 2579 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-network-diagnostics/network-check-target-xzlsc" podStartSLOduration=67.175545016 podStartE2EDuration="1m15.55861469s" totalImagesPullingTime="8.383069674s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-03 11:12:24 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-03 11:13:39.557910232 +0000 UTC m=+76.146214553" watchObservedRunningTime="2026-07-03 11:13:39.55861469 +0000 UTC m=+76.146919012" Jul 03 11:13:39.581067 ip-10-0-136-0 kubenswrapper[2579]: I0703 11:13:39.580525 2579 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="open-cluster-management-agent-addon/klusterlet-addon-workmgr-7d8dd55d49-bs5j5" podStartSLOduration=2.127521498 podStartE2EDuration="9.580499581s" totalImagesPullingTime="7.452978083s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-03 11:13:30 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-03 11:13:39.575127687 +0000 UTC m=+76.163432008" watchObservedRunningTime="2026-07-03 11:13:39.580499581 +0000 UTC m=+76.168803902" Jul 03 11:13:40.560503 ip-10-0-136-0 kubenswrapper[2579]: I0703 11:13:40.560332 2579 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-image-registry/image-registry-659bf79d58-p9gqp" Jul 03 11:13:41.047855 ip-10-0-136-0 kubenswrapper[2579]: I0703 11:13:41.047819 2579 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-console/console-565d858f78-nz7js"] Jul 03 11:13:41.050493 ip-10-0-136-0 kubenswrapper[2579]: I0703 11:13:41.050461 2579 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-console/console-565d858f78-nz7js" Jul 03 11:13:41.059282 ip-10-0-136-0 kubenswrapper[2579]: I0703 11:13:41.059011 2579 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-console\"/\"trusted-ca-bundle\"" Jul 03 11:13:41.062571 ip-10-0-136-0 kubenswrapper[2579]: I0703 11:13:41.062536 2579 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-console/console-565d858f78-nz7js"] Jul 03 11:13:41.195948 ip-10-0-136-0 kubenswrapper[2579]: I0703 11:13:41.195900 2579 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"console-serving-cert\" (UniqueName: \"kubernetes.io/secret/c09906a1-28fe-47f1-ae02-46a934363524-console-serving-cert\") pod \"console-565d858f78-nz7js\" (UID: \"c09906a1-28fe-47f1-ae02-46a934363524\") " pod="openshift-console/console-565d858f78-nz7js" Jul 03 11:13:41.195948 ip-10-0-136-0 kubenswrapper[2579]: I0703 11:13:41.195955 2579 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"console-config\" (UniqueName: \"kubernetes.io/configmap/c09906a1-28fe-47f1-ae02-46a934363524-console-config\") pod \"console-565d858f78-nz7js\" (UID: \"c09906a1-28fe-47f1-ae02-46a934363524\") " pod="openshift-console/console-565d858f78-nz7js" Jul 03 11:13:41.196247 ip-10-0-136-0 kubenswrapper[2579]: I0703 11:13:41.195990 2579 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"console-oauth-config\" (UniqueName: \"kubernetes.io/secret/c09906a1-28fe-47f1-ae02-46a934363524-console-oauth-config\") pod \"console-565d858f78-nz7js\" (UID: \"c09906a1-28fe-47f1-ae02-46a934363524\") " pod="openshift-console/console-565d858f78-nz7js" Jul 03 11:13:41.196247 ip-10-0-136-0 kubenswrapper[2579]: I0703 11:13:41.196015 2579 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"service-ca\" (UniqueName: \"kubernetes.io/configmap/c09906a1-28fe-47f1-ae02-46a934363524-service-ca\") pod \"console-565d858f78-nz7js\" (UID: \"c09906a1-28fe-47f1-ae02-46a934363524\") " pod="openshift-console/console-565d858f78-nz7js" Jul 03 11:13:41.196247 ip-10-0-136-0 kubenswrapper[2579]: I0703 11:13:41.196036 2579 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"oauth-serving-cert\" (UniqueName: \"kubernetes.io/configmap/c09906a1-28fe-47f1-ae02-46a934363524-oauth-serving-cert\") pod \"console-565d858f78-nz7js\" (UID: \"c09906a1-28fe-47f1-ae02-46a934363524\") " pod="openshift-console/console-565d858f78-nz7js" Jul 03 11:13:41.196247 ip-10-0-136-0 kubenswrapper[2579]: I0703 11:13:41.196053 2579 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-7dbzx\" (UniqueName: \"kubernetes.io/projected/c09906a1-28fe-47f1-ae02-46a934363524-kube-api-access-7dbzx\") pod \"console-565d858f78-nz7js\" (UID: \"c09906a1-28fe-47f1-ae02-46a934363524\") " pod="openshift-console/console-565d858f78-nz7js" Jul 03 11:13:41.196247 ip-10-0-136-0 kubenswrapper[2579]: I0703 11:13:41.196231 2579 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/c09906a1-28fe-47f1-ae02-46a934363524-trusted-ca-bundle\") pod \"console-565d858f78-nz7js\" (UID: \"c09906a1-28fe-47f1-ae02-46a934363524\") " pod="openshift-console/console-565d858f78-nz7js" Jul 03 11:13:41.297230 ip-10-0-136-0 kubenswrapper[2579]: I0703 11:13:41.297167 2579 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"console-oauth-config\" (UniqueName: \"kubernetes.io/secret/c09906a1-28fe-47f1-ae02-46a934363524-console-oauth-config\") pod \"console-565d858f78-nz7js\" (UID: \"c09906a1-28fe-47f1-ae02-46a934363524\") " pod="openshift-console/console-565d858f78-nz7js" Jul 03 11:13:41.297230 ip-10-0-136-0 kubenswrapper[2579]: I0703 11:13:41.297236 2579 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"service-ca\" (UniqueName: \"kubernetes.io/configmap/c09906a1-28fe-47f1-ae02-46a934363524-service-ca\") pod \"console-565d858f78-nz7js\" (UID: \"c09906a1-28fe-47f1-ae02-46a934363524\") " pod="openshift-console/console-565d858f78-nz7js" Jul 03 11:13:41.297502 ip-10-0-136-0 kubenswrapper[2579]: I0703 11:13:41.297265 2579 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"oauth-serving-cert\" (UniqueName: \"kubernetes.io/configmap/c09906a1-28fe-47f1-ae02-46a934363524-oauth-serving-cert\") pod \"console-565d858f78-nz7js\" (UID: \"c09906a1-28fe-47f1-ae02-46a934363524\") " pod="openshift-console/console-565d858f78-nz7js" Jul 03 11:13:41.297502 ip-10-0-136-0 kubenswrapper[2579]: I0703 11:13:41.297294 2579 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-7dbzx\" (UniqueName: \"kubernetes.io/projected/c09906a1-28fe-47f1-ae02-46a934363524-kube-api-access-7dbzx\") pod \"console-565d858f78-nz7js\" (UID: \"c09906a1-28fe-47f1-ae02-46a934363524\") " pod="openshift-console/console-565d858f78-nz7js" Jul 03 11:13:41.297502 ip-10-0-136-0 kubenswrapper[2579]: I0703 11:13:41.297328 2579 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/c09906a1-28fe-47f1-ae02-46a934363524-trusted-ca-bundle\") pod \"console-565d858f78-nz7js\" (UID: \"c09906a1-28fe-47f1-ae02-46a934363524\") " pod="openshift-console/console-565d858f78-nz7js" Jul 03 11:13:41.297502 ip-10-0-136-0 kubenswrapper[2579]: I0703 11:13:41.297426 2579 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"console-serving-cert\" (UniqueName: \"kubernetes.io/secret/c09906a1-28fe-47f1-ae02-46a934363524-console-serving-cert\") pod \"console-565d858f78-nz7js\" (UID: \"c09906a1-28fe-47f1-ae02-46a934363524\") " pod="openshift-console/console-565d858f78-nz7js" Jul 03 11:13:41.297502 ip-10-0-136-0 kubenswrapper[2579]: I0703 11:13:41.297461 2579 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"console-config\" (UniqueName: \"kubernetes.io/configmap/c09906a1-28fe-47f1-ae02-46a934363524-console-config\") pod \"console-565d858f78-nz7js\" (UID: \"c09906a1-28fe-47f1-ae02-46a934363524\") " pod="openshift-console/console-565d858f78-nz7js" Jul 03 11:13:41.298821 ip-10-0-136-0 kubenswrapper[2579]: I0703 11:13:41.298445 2579 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"oauth-serving-cert\" (UniqueName: \"kubernetes.io/configmap/c09906a1-28fe-47f1-ae02-46a934363524-oauth-serving-cert\") pod \"console-565d858f78-nz7js\" (UID: \"c09906a1-28fe-47f1-ae02-46a934363524\") " pod="openshift-console/console-565d858f78-nz7js" Jul 03 11:13:41.298821 ip-10-0-136-0 kubenswrapper[2579]: I0703 11:13:41.298492 2579 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"service-ca\" (UniqueName: \"kubernetes.io/configmap/c09906a1-28fe-47f1-ae02-46a934363524-service-ca\") pod \"console-565d858f78-nz7js\" (UID: \"c09906a1-28fe-47f1-ae02-46a934363524\") " pod="openshift-console/console-565d858f78-nz7js" Jul 03 11:13:41.298821 ip-10-0-136-0 kubenswrapper[2579]: I0703 11:13:41.298693 2579 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"console-config\" (UniqueName: \"kubernetes.io/configmap/c09906a1-28fe-47f1-ae02-46a934363524-console-config\") pod \"console-565d858f78-nz7js\" (UID: \"c09906a1-28fe-47f1-ae02-46a934363524\") " pod="openshift-console/console-565d858f78-nz7js" Jul 03 11:13:41.298821 ip-10-0-136-0 kubenswrapper[2579]: I0703 11:13:41.298740 2579 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/c09906a1-28fe-47f1-ae02-46a934363524-trusted-ca-bundle\") pod \"console-565d858f78-nz7js\" (UID: \"c09906a1-28fe-47f1-ae02-46a934363524\") " pod="openshift-console/console-565d858f78-nz7js" Jul 03 11:13:41.300964 ip-10-0-136-0 kubenswrapper[2579]: I0703 11:13:41.300941 2579 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"console-oauth-config\" (UniqueName: \"kubernetes.io/secret/c09906a1-28fe-47f1-ae02-46a934363524-console-oauth-config\") pod \"console-565d858f78-nz7js\" (UID: \"c09906a1-28fe-47f1-ae02-46a934363524\") " pod="openshift-console/console-565d858f78-nz7js" Jul 03 11:13:41.300964 ip-10-0-136-0 kubenswrapper[2579]: I0703 11:13:41.300951 2579 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"console-serving-cert\" (UniqueName: \"kubernetes.io/secret/c09906a1-28fe-47f1-ae02-46a934363524-console-serving-cert\") pod \"console-565d858f78-nz7js\" (UID: \"c09906a1-28fe-47f1-ae02-46a934363524\") " pod="openshift-console/console-565d858f78-nz7js" Jul 03 11:13:41.312903 ip-10-0-136-0 kubenswrapper[2579]: I0703 11:13:41.312832 2579 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-7dbzx\" (UniqueName: \"kubernetes.io/projected/c09906a1-28fe-47f1-ae02-46a934363524-kube-api-access-7dbzx\") pod \"console-565d858f78-nz7js\" (UID: \"c09906a1-28fe-47f1-ae02-46a934363524\") " pod="openshift-console/console-565d858f78-nz7js" Jul 03 11:13:41.367165 ip-10-0-136-0 kubenswrapper[2579]: I0703 11:13:41.366886 2579 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-console/console-565d858f78-nz7js" Jul 03 11:13:41.542855 ip-10-0-136-0 kubenswrapper[2579]: I0703 11:13:41.542803 2579 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/prometheus-operator-567f7b8b7c-dft8d" event={"ID":"4ab5f9bd-3890-4562-895a-84485c66f4ae","Type":"ContainerStarted","Data":"2e46ea517b8a16b8a81993fd388ec83607a2c57a6ad14b087684644f5140bb6b"} Jul 03 11:13:41.543046 ip-10-0-136-0 kubenswrapper[2579]: I0703 11:13:41.542885 2579 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/prometheus-operator-567f7b8b7c-dft8d" event={"ID":"4ab5f9bd-3890-4562-895a-84485c66f4ae","Type":"ContainerStarted","Data":"8e39df7db48c576b968db0f2f13267b3539245aa247b4ffd28500798b4c1c2f6"} Jul 03 11:13:42.411161 ip-10-0-136-0 kubenswrapper[2579]: I0703 11:13:42.410704 2579 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-monitoring/prometheus-operator-567f7b8b7c-dft8d" podStartSLOduration=3.826902823 podStartE2EDuration="5.410686004s" totalImagesPullingTime="1.583783181s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-03 11:13:37 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-03 11:13:41.562646058 +0000 UTC m=+78.150950360" watchObservedRunningTime="2026-07-03 11:13:42.410686004 +0000 UTC m=+78.998990301" Jul 03 11:13:42.411161 ip-10-0-136-0 kubenswrapper[2579]: I0703 11:13:42.411097 2579 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-console/console-565d858f78-nz7js"] Jul 03 11:13:42.414403 ip-10-0-136-0 kubenswrapper[2579]: W0703 11:13:42.414366 2579 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podc09906a1_28fe_47f1_ae02_46a934363524.slice/crio-0eb31c92accb70ccfc4e84d8f8f0ab3dd2ef45973db51637006fd476c3ba69fa WatchSource:0}: Error finding container 0eb31c92accb70ccfc4e84d8f8f0ab3dd2ef45973db51637006fd476c3ba69fa: Status 404 returned error can't find the container with id 0eb31c92accb70ccfc4e84d8f8f0ab3dd2ef45973db51637006fd476c3ba69fa Jul 03 11:13:42.550772 ip-10-0-136-0 kubenswrapper[2579]: I0703 11:13:42.550701 2579 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-console/console-55fd9ddfdd-klc4c" event={"ID":"8937bcc1-fce2-40e9-ac92-e25c9a902506","Type":"ContainerStarted","Data":"f69b69694a1d8034197965404d7de6d86b54707bce0d7861b11309f83a0b3bad"} Jul 03 11:13:42.552940 ip-10-0-136-0 kubenswrapper[2579]: I0703 11:13:42.552849 2579 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-console/console-565d858f78-nz7js" event={"ID":"c09906a1-28fe-47f1-ae02-46a934363524","Type":"ContainerStarted","Data":"e7e31782a6fd35b6c33c7868ef65f5aab8f4d96d696016cf637c652ca58f146a"} Jul 03 11:13:42.552940 ip-10-0-136-0 kubenswrapper[2579]: I0703 11:13:42.552909 2579 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-console/console-565d858f78-nz7js" event={"ID":"c09906a1-28fe-47f1-ae02-46a934363524","Type":"ContainerStarted","Data":"0eb31c92accb70ccfc4e84d8f8f0ab3dd2ef45973db51637006fd476c3ba69fa"} Jul 03 11:13:42.571129 ip-10-0-136-0 kubenswrapper[2579]: I0703 11:13:42.571058 2579 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-console/console-55fd9ddfdd-klc4c" podStartSLOduration=6.161438911 podStartE2EDuration="9.571031321s" totalImagesPullingTime="3.40959241s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-03 11:13:33 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-03 11:13:42.569640387 +0000 UTC m=+79.157944707" watchObservedRunningTime="2026-07-03 11:13:42.571031321 +0000 UTC m=+79.159335642" Jul 03 11:13:42.588477 ip-10-0-136-0 kubenswrapper[2579]: I0703 11:13:42.588406 2579 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-console/console-565d858f78-nz7js" podStartSLOduration=1.588380127 podStartE2EDuration="1.588380127s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-03 11:13:41 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-03 11:13:42.58811486 +0000 UTC m=+79.176419203" watchObservedRunningTime="2026-07-03 11:13:42.588380127 +0000 UTC m=+79.176684449" Jul 03 11:13:43.430390 ip-10-0-136-0 kubenswrapper[2579]: I0703 11:13:43.430351 2579 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-monitoring/node-exporter-xj7rs"] Jul 03 11:13:43.433447 ip-10-0-136-0 kubenswrapper[2579]: I0703 11:13:43.433417 2579 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-monitoring/node-exporter-xj7rs" Jul 03 11:13:43.437879 ip-10-0-136-0 kubenswrapper[2579]: I0703 11:13:43.437844 2579 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"node-exporter-kube-rbac-proxy-config\"" Jul 03 11:13:43.438215 ip-10-0-136-0 kubenswrapper[2579]: I0703 11:13:43.438180 2579 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"node-exporter-dockercfg-s5r7q\"" Jul 03 11:13:43.438456 ip-10-0-136-0 kubenswrapper[2579]: I0703 11:13:43.438441 2579 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-monitoring\"/\"node-exporter-accelerators-collector-config\"" Jul 03 11:13:43.438644 ip-10-0-136-0 kubenswrapper[2579]: I0703 11:13:43.438630 2579 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"node-exporter-tls\"" Jul 03 11:13:43.522462 ip-10-0-136-0 kubenswrapper[2579]: I0703 11:13:43.522179 2579 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"sys\" (UniqueName: \"kubernetes.io/host-path/b2866e40-a413-4cc8-978c-65b100d290ac-sys\") pod \"node-exporter-xj7rs\" (UID: \"b2866e40-a413-4cc8-978c-65b100d290ac\") " pod="openshift-monitoring/node-exporter-xj7rs" Jul 03 11:13:43.522462 ip-10-0-136-0 kubenswrapper[2579]: I0703 11:13:43.522309 2579 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"node-exporter-accelerators-collector-config\" (UniqueName: \"kubernetes.io/configmap/b2866e40-a413-4cc8-978c-65b100d290ac-node-exporter-accelerators-collector-config\") pod \"node-exporter-xj7rs\" (UID: \"b2866e40-a413-4cc8-978c-65b100d290ac\") " pod="openshift-monitoring/node-exporter-xj7rs" Jul 03 11:13:43.522462 ip-10-0-136-0 kubenswrapper[2579]: I0703 11:13:43.522350 2579 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"node-exporter-tls\" (UniqueName: \"kubernetes.io/secret/b2866e40-a413-4cc8-978c-65b100d290ac-node-exporter-tls\") pod \"node-exporter-xj7rs\" (UID: \"b2866e40-a413-4cc8-978c-65b100d290ac\") " pod="openshift-monitoring/node-exporter-xj7rs" Jul 03 11:13:43.522462 ip-10-0-136-0 kubenswrapper[2579]: I0703 11:13:43.522378 2579 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"node-exporter-wtmp\" (UniqueName: \"kubernetes.io/host-path/b2866e40-a413-4cc8-978c-65b100d290ac-node-exporter-wtmp\") pod \"node-exporter-xj7rs\" (UID: \"b2866e40-a413-4cc8-978c-65b100d290ac\") " pod="openshift-monitoring/node-exporter-xj7rs" Jul 03 11:13:43.522462 ip-10-0-136-0 kubenswrapper[2579]: I0703 11:13:43.522411 2579 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"root\" (UniqueName: \"kubernetes.io/host-path/b2866e40-a413-4cc8-978c-65b100d290ac-root\") pod \"node-exporter-xj7rs\" (UID: \"b2866e40-a413-4cc8-978c-65b100d290ac\") " pod="openshift-monitoring/node-exporter-xj7rs" Jul 03 11:13:43.523263 ip-10-0-136-0 kubenswrapper[2579]: I0703 11:13:43.522515 2579 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"node-exporter-kube-rbac-proxy-config\" (UniqueName: \"kubernetes.io/secret/b2866e40-a413-4cc8-978c-65b100d290ac-node-exporter-kube-rbac-proxy-config\") pod \"node-exporter-xj7rs\" (UID: \"b2866e40-a413-4cc8-978c-65b100d290ac\") " pod="openshift-monitoring/node-exporter-xj7rs" Jul 03 11:13:43.523263 ip-10-0-136-0 kubenswrapper[2579]: I0703 11:13:43.522572 2579 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"node-exporter-textfile\" (UniqueName: \"kubernetes.io/empty-dir/b2866e40-a413-4cc8-978c-65b100d290ac-node-exporter-textfile\") pod \"node-exporter-xj7rs\" (UID: \"b2866e40-a413-4cc8-978c-65b100d290ac\") " pod="openshift-monitoring/node-exporter-xj7rs" Jul 03 11:13:43.523263 ip-10-0-136-0 kubenswrapper[2579]: I0703 11:13:43.522601 2579 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-ljccv\" (UniqueName: \"kubernetes.io/projected/b2866e40-a413-4cc8-978c-65b100d290ac-kube-api-access-ljccv\") pod \"node-exporter-xj7rs\" (UID: \"b2866e40-a413-4cc8-978c-65b100d290ac\") " pod="openshift-monitoring/node-exporter-xj7rs" Jul 03 11:13:43.523263 ip-10-0-136-0 kubenswrapper[2579]: I0703 11:13:43.522634 2579 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"metrics-client-ca\" (UniqueName: \"kubernetes.io/configmap/b2866e40-a413-4cc8-978c-65b100d290ac-metrics-client-ca\") pod \"node-exporter-xj7rs\" (UID: \"b2866e40-a413-4cc8-978c-65b100d290ac\") " pod="openshift-monitoring/node-exporter-xj7rs" Jul 03 11:13:43.623639 ip-10-0-136-0 kubenswrapper[2579]: I0703 11:13:43.623599 2579 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"node-exporter-textfile\" (UniqueName: \"kubernetes.io/empty-dir/b2866e40-a413-4cc8-978c-65b100d290ac-node-exporter-textfile\") pod \"node-exporter-xj7rs\" (UID: \"b2866e40-a413-4cc8-978c-65b100d290ac\") " pod="openshift-monitoring/node-exporter-xj7rs" Jul 03 11:13:43.623821 ip-10-0-136-0 kubenswrapper[2579]: I0703 11:13:43.623648 2579 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-ljccv\" (UniqueName: \"kubernetes.io/projected/b2866e40-a413-4cc8-978c-65b100d290ac-kube-api-access-ljccv\") pod \"node-exporter-xj7rs\" (UID: \"b2866e40-a413-4cc8-978c-65b100d290ac\") " pod="openshift-monitoring/node-exporter-xj7rs" Jul 03 11:13:43.623821 ip-10-0-136-0 kubenswrapper[2579]: I0703 11:13:43.623677 2579 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-client-ca\" (UniqueName: \"kubernetes.io/configmap/b2866e40-a413-4cc8-978c-65b100d290ac-metrics-client-ca\") pod \"node-exporter-xj7rs\" (UID: \"b2866e40-a413-4cc8-978c-65b100d290ac\") " pod="openshift-monitoring/node-exporter-xj7rs" Jul 03 11:13:43.623821 ip-10-0-136-0 kubenswrapper[2579]: I0703 11:13:43.623713 2579 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"sys\" (UniqueName: \"kubernetes.io/host-path/b2866e40-a413-4cc8-978c-65b100d290ac-sys\") pod \"node-exporter-xj7rs\" (UID: \"b2866e40-a413-4cc8-978c-65b100d290ac\") " pod="openshift-monitoring/node-exporter-xj7rs" Jul 03 11:13:43.623821 ip-10-0-136-0 kubenswrapper[2579]: I0703 11:13:43.623761 2579 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"node-exporter-accelerators-collector-config\" (UniqueName: \"kubernetes.io/configmap/b2866e40-a413-4cc8-978c-65b100d290ac-node-exporter-accelerators-collector-config\") pod \"node-exporter-xj7rs\" (UID: \"b2866e40-a413-4cc8-978c-65b100d290ac\") " pod="openshift-monitoring/node-exporter-xj7rs" Jul 03 11:13:43.623821 ip-10-0-136-0 kubenswrapper[2579]: I0703 11:13:43.623799 2579 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"node-exporter-tls\" (UniqueName: \"kubernetes.io/secret/b2866e40-a413-4cc8-978c-65b100d290ac-node-exporter-tls\") pod \"node-exporter-xj7rs\" (UID: \"b2866e40-a413-4cc8-978c-65b100d290ac\") " pod="openshift-monitoring/node-exporter-xj7rs" Jul 03 11:13:43.624080 ip-10-0-136-0 kubenswrapper[2579]: I0703 11:13:43.623820 2579 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"node-exporter-wtmp\" (UniqueName: \"kubernetes.io/host-path/b2866e40-a413-4cc8-978c-65b100d290ac-node-exporter-wtmp\") pod \"node-exporter-xj7rs\" (UID: \"b2866e40-a413-4cc8-978c-65b100d290ac\") " pod="openshift-monitoring/node-exporter-xj7rs" Jul 03 11:13:43.624080 ip-10-0-136-0 kubenswrapper[2579]: I0703 11:13:43.623857 2579 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"root\" (UniqueName: \"kubernetes.io/host-path/b2866e40-a413-4cc8-978c-65b100d290ac-root\") pod \"node-exporter-xj7rs\" (UID: \"b2866e40-a413-4cc8-978c-65b100d290ac\") " pod="openshift-monitoring/node-exporter-xj7rs" Jul 03 11:13:43.625868 ip-10-0-136-0 kubenswrapper[2579]: I0703 11:13:43.625754 2579 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"node-exporter-kube-rbac-proxy-config\" (UniqueName: \"kubernetes.io/secret/b2866e40-a413-4cc8-978c-65b100d290ac-node-exporter-kube-rbac-proxy-config\") pod \"node-exporter-xj7rs\" (UID: \"b2866e40-a413-4cc8-978c-65b100d290ac\") " pod="openshift-monitoring/node-exporter-xj7rs" Jul 03 11:13:43.625868 ip-10-0-136-0 kubenswrapper[2579]: I0703 11:13:43.624316 2579 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"node-exporter-textfile\" (UniqueName: \"kubernetes.io/empty-dir/b2866e40-a413-4cc8-978c-65b100d290ac-node-exporter-textfile\") pod \"node-exporter-xj7rs\" (UID: \"b2866e40-a413-4cc8-978c-65b100d290ac\") " pod="openshift-monitoring/node-exporter-xj7rs" Jul 03 11:13:43.625868 ip-10-0-136-0 kubenswrapper[2579]: I0703 11:13:43.625113 2579 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"metrics-client-ca\" (UniqueName: \"kubernetes.io/configmap/b2866e40-a413-4cc8-978c-65b100d290ac-metrics-client-ca\") pod \"node-exporter-xj7rs\" (UID: \"b2866e40-a413-4cc8-978c-65b100d290ac\") " pod="openshift-monitoring/node-exporter-xj7rs" Jul 03 11:13:43.625868 ip-10-0-136-0 kubenswrapper[2579]: I0703 11:13:43.625150 2579 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"sys\" (UniqueName: \"kubernetes.io/host-path/b2866e40-a413-4cc8-978c-65b100d290ac-sys\") pod \"node-exporter-xj7rs\" (UID: \"b2866e40-a413-4cc8-978c-65b100d290ac\") " pod="openshift-monitoring/node-exporter-xj7rs" Jul 03 11:13:43.626174 ip-10-0-136-0 kubenswrapper[2579]: I0703 11:13:43.625526 2579 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"node-exporter-accelerators-collector-config\" (UniqueName: \"kubernetes.io/configmap/b2866e40-a413-4cc8-978c-65b100d290ac-node-exporter-accelerators-collector-config\") pod \"node-exporter-xj7rs\" (UID: \"b2866e40-a413-4cc8-978c-65b100d290ac\") " pod="openshift-monitoring/node-exporter-xj7rs" Jul 03 11:13:43.626174 ip-10-0-136-0 kubenswrapper[2579]: E0703 11:13:43.625601 2579 secret.go:189] Couldn't get secret openshift-monitoring/node-exporter-tls: secret "node-exporter-tls" not found Jul 03 11:13:43.626401 ip-10-0-136-0 kubenswrapper[2579]: E0703 11:13:43.626250 2579 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/b2866e40-a413-4cc8-978c-65b100d290ac-node-exporter-tls podName:b2866e40-a413-4cc8-978c-65b100d290ac nodeName:}" failed. No retries permitted until 2026-07-03 11:13:44.126227454 +0000 UTC m=+80.714531783 (durationBeforeRetry 500ms). Error: MountVolume.SetUp failed for volume "node-exporter-tls" (UniqueName: "kubernetes.io/secret/b2866e40-a413-4cc8-978c-65b100d290ac-node-exporter-tls") pod "node-exporter-xj7rs" (UID: "b2866e40-a413-4cc8-978c-65b100d290ac") : secret "node-exporter-tls" not found Jul 03 11:13:43.626505 ip-10-0-136-0 kubenswrapper[2579]: I0703 11:13:43.625690 2579 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"node-exporter-wtmp\" (UniqueName: \"kubernetes.io/host-path/b2866e40-a413-4cc8-978c-65b100d290ac-node-exporter-wtmp\") pod \"node-exporter-xj7rs\" (UID: \"b2866e40-a413-4cc8-978c-65b100d290ac\") " pod="openshift-monitoring/node-exporter-xj7rs" Jul 03 11:13:43.626505 ip-10-0-136-0 kubenswrapper[2579]: I0703 11:13:43.623930 2579 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"root\" (UniqueName: \"kubernetes.io/host-path/b2866e40-a413-4cc8-978c-65b100d290ac-root\") pod \"node-exporter-xj7rs\" (UID: \"b2866e40-a413-4cc8-978c-65b100d290ac\") " pod="openshift-monitoring/node-exporter-xj7rs" Jul 03 11:13:43.629309 ip-10-0-136-0 kubenswrapper[2579]: I0703 11:13:43.629278 2579 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"node-exporter-kube-rbac-proxy-config\" (UniqueName: \"kubernetes.io/secret/b2866e40-a413-4cc8-978c-65b100d290ac-node-exporter-kube-rbac-proxy-config\") pod \"node-exporter-xj7rs\" (UID: \"b2866e40-a413-4cc8-978c-65b100d290ac\") " pod="openshift-monitoring/node-exporter-xj7rs" Jul 03 11:13:43.636294 ip-10-0-136-0 kubenswrapper[2579]: I0703 11:13:43.636177 2579 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-ljccv\" (UniqueName: \"kubernetes.io/projected/b2866e40-a413-4cc8-978c-65b100d290ac-kube-api-access-ljccv\") pod \"node-exporter-xj7rs\" (UID: \"b2866e40-a413-4cc8-978c-65b100d290ac\") " pod="openshift-monitoring/node-exporter-xj7rs" Jul 03 11:13:43.777742 ip-10-0-136-0 kubenswrapper[2579]: I0703 11:13:43.777699 2579 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="openshift-console/console-55fd9ddfdd-klc4c" Jul 03 11:13:43.777962 ip-10-0-136-0 kubenswrapper[2579]: I0703 11:13:43.777807 2579 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-console/console-55fd9ddfdd-klc4c" Jul 03 11:13:43.784312 ip-10-0-136-0 kubenswrapper[2579]: I0703 11:13:43.784236 2579 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="openshift-console/console-55fd9ddfdd-klc4c" Jul 03 11:13:44.134455 ip-10-0-136-0 kubenswrapper[2579]: I0703 11:13:44.134343 2579 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"node-exporter-tls\" (UniqueName: \"kubernetes.io/secret/b2866e40-a413-4cc8-978c-65b100d290ac-node-exporter-tls\") pod \"node-exporter-xj7rs\" (UID: \"b2866e40-a413-4cc8-978c-65b100d290ac\") " pod="openshift-monitoring/node-exporter-xj7rs" Jul 03 11:13:44.137555 ip-10-0-136-0 kubenswrapper[2579]: I0703 11:13:44.137520 2579 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"node-exporter-tls\" (UniqueName: \"kubernetes.io/secret/b2866e40-a413-4cc8-978c-65b100d290ac-node-exporter-tls\") pod \"node-exporter-xj7rs\" (UID: \"b2866e40-a413-4cc8-978c-65b100d290ac\") " pod="openshift-monitoring/node-exporter-xj7rs" Jul 03 11:13:44.350965 ip-10-0-136-0 kubenswrapper[2579]: I0703 11:13:44.350921 2579 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-monitoring/node-exporter-xj7rs" Jul 03 11:13:44.567847 ip-10-0-136-0 kubenswrapper[2579]: I0703 11:13:44.567798 2579 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-console/console-55fd9ddfdd-klc4c" Jul 03 11:13:47.817706 ip-10-0-136-0 kubenswrapper[2579]: I0703 11:13:47.817653 2579 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-monitoring/metrics-server-548b5c6d96-hzzhk"] Jul 03 11:13:47.955782 ip-10-0-136-0 kubenswrapper[2579]: I0703 11:13:47.955741 2579 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-monitoring/metrics-server-548b5c6d96-hzzhk"] Jul 03 11:13:47.955995 ip-10-0-136-0 kubenswrapper[2579]: I0703 11:13:47.955876 2579 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-monitoring/metrics-server-548b5c6d96-hzzhk" Jul 03 11:13:47.959677 ip-10-0-136-0 kubenswrapper[2579]: I0703 11:13:47.959646 2579 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"metrics-server-dockercfg-9fxtx\"" Jul 03 11:13:47.959874 ip-10-0-136-0 kubenswrapper[2579]: I0703 11:13:47.959646 2579 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-monitoring\"/\"metrics-server-audit-profiles\"" Jul 03 11:13:47.959874 ip-10-0-136-0 kubenswrapper[2579]: I0703 11:13:47.959646 2579 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-monitoring\"/\"kubelet-serving-ca-bundle\"" Jul 03 11:13:47.959874 ip-10-0-136-0 kubenswrapper[2579]: I0703 11:13:47.959646 2579 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"metrics-server-f0i4781fnardn\"" Jul 03 11:13:47.959874 ip-10-0-136-0 kubenswrapper[2579]: I0703 11:13:47.959646 2579 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"metrics-server-tls\"" Jul 03 11:13:47.959874 ip-10-0-136-0 kubenswrapper[2579]: I0703 11:13:47.959646 2579 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"metrics-server-client-certs\"" Jul 03 11:13:48.072366 ip-10-0-136-0 kubenswrapper[2579]: I0703 11:13:48.072086 2579 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"audit-log\" (UniqueName: \"kubernetes.io/empty-dir/29ee781d-d24c-460a-af83-39d31e9dfa0b-audit-log\") pod \"metrics-server-548b5c6d96-hzzhk\" (UID: \"29ee781d-d24c-460a-af83-39d31e9dfa0b\") " pod="openshift-monitoring/metrics-server-548b5c6d96-hzzhk" Jul 03 11:13:48.072683 ip-10-0-136-0 kubenswrapper[2579]: I0703 11:13:48.072617 2579 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"secret-metrics-server-tls\" (UniqueName: \"kubernetes.io/secret/29ee781d-d24c-460a-af83-39d31e9dfa0b-secret-metrics-server-tls\") pod \"metrics-server-548b5c6d96-hzzhk\" (UID: \"29ee781d-d24c-460a-af83-39d31e9dfa0b\") " pod="openshift-monitoring/metrics-server-548b5c6d96-hzzhk" Jul 03 11:13:48.072790 ip-10-0-136-0 kubenswrapper[2579]: I0703 11:13:48.072697 2579 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"configmap-kubelet-serving-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/29ee781d-d24c-460a-af83-39d31e9dfa0b-configmap-kubelet-serving-ca-bundle\") pod \"metrics-server-548b5c6d96-hzzhk\" (UID: \"29ee781d-d24c-460a-af83-39d31e9dfa0b\") " pod="openshift-monitoring/metrics-server-548b5c6d96-hzzhk" Jul 03 11:13:48.072790 ip-10-0-136-0 kubenswrapper[2579]: I0703 11:13:48.072743 2579 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"client-ca-bundle\" (UniqueName: \"kubernetes.io/secret/29ee781d-d24c-460a-af83-39d31e9dfa0b-client-ca-bundle\") pod \"metrics-server-548b5c6d96-hzzhk\" (UID: \"29ee781d-d24c-460a-af83-39d31e9dfa0b\") " pod="openshift-monitoring/metrics-server-548b5c6d96-hzzhk" Jul 03 11:13:48.072790 ip-10-0-136-0 kubenswrapper[2579]: I0703 11:13:48.072783 2579 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"metrics-server-audit-profiles\" (UniqueName: \"kubernetes.io/configmap/29ee781d-d24c-460a-af83-39d31e9dfa0b-metrics-server-audit-profiles\") pod \"metrics-server-548b5c6d96-hzzhk\" (UID: \"29ee781d-d24c-460a-af83-39d31e9dfa0b\") " pod="openshift-monitoring/metrics-server-548b5c6d96-hzzhk" Jul 03 11:13:48.072891 ip-10-0-136-0 kubenswrapper[2579]: I0703 11:13:48.072848 2579 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"secret-metrics-server-client-certs\" (UniqueName: \"kubernetes.io/secret/29ee781d-d24c-460a-af83-39d31e9dfa0b-secret-metrics-server-client-certs\") pod \"metrics-server-548b5c6d96-hzzhk\" (UID: \"29ee781d-d24c-460a-af83-39d31e9dfa0b\") " pod="openshift-monitoring/metrics-server-548b5c6d96-hzzhk" Jul 03 11:13:48.072924 ip-10-0-136-0 kubenswrapper[2579]: I0703 11:13:48.072903 2579 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-qbdks\" (UniqueName: \"kubernetes.io/projected/29ee781d-d24c-460a-af83-39d31e9dfa0b-kube-api-access-qbdks\") pod \"metrics-server-548b5c6d96-hzzhk\" (UID: \"29ee781d-d24c-460a-af83-39d31e9dfa0b\") " pod="openshift-monitoring/metrics-server-548b5c6d96-hzzhk" Jul 03 11:13:48.174379 ip-10-0-136-0 kubenswrapper[2579]: I0703 11:13:48.174338 2579 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"secret-metrics-server-tls\" (UniqueName: \"kubernetes.io/secret/29ee781d-d24c-460a-af83-39d31e9dfa0b-secret-metrics-server-tls\") pod \"metrics-server-548b5c6d96-hzzhk\" (UID: \"29ee781d-d24c-460a-af83-39d31e9dfa0b\") " pod="openshift-monitoring/metrics-server-548b5c6d96-hzzhk" Jul 03 11:13:48.174593 ip-10-0-136-0 kubenswrapper[2579]: I0703 11:13:48.174392 2579 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"configmap-kubelet-serving-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/29ee781d-d24c-460a-af83-39d31e9dfa0b-configmap-kubelet-serving-ca-bundle\") pod \"metrics-server-548b5c6d96-hzzhk\" (UID: \"29ee781d-d24c-460a-af83-39d31e9dfa0b\") " pod="openshift-monitoring/metrics-server-548b5c6d96-hzzhk" Jul 03 11:13:48.174593 ip-10-0-136-0 kubenswrapper[2579]: I0703 11:13:48.174426 2579 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"client-ca-bundle\" (UniqueName: \"kubernetes.io/secret/29ee781d-d24c-460a-af83-39d31e9dfa0b-client-ca-bundle\") pod \"metrics-server-548b5c6d96-hzzhk\" (UID: \"29ee781d-d24c-460a-af83-39d31e9dfa0b\") " pod="openshift-monitoring/metrics-server-548b5c6d96-hzzhk" Jul 03 11:13:48.174593 ip-10-0-136-0 kubenswrapper[2579]: I0703 11:13:48.174450 2579 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-server-audit-profiles\" (UniqueName: \"kubernetes.io/configmap/29ee781d-d24c-460a-af83-39d31e9dfa0b-metrics-server-audit-profiles\") pod \"metrics-server-548b5c6d96-hzzhk\" (UID: \"29ee781d-d24c-460a-af83-39d31e9dfa0b\") " pod="openshift-monitoring/metrics-server-548b5c6d96-hzzhk" Jul 03 11:13:48.174593 ip-10-0-136-0 kubenswrapper[2579]: I0703 11:13:48.174480 2579 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"secret-metrics-server-client-certs\" (UniqueName: \"kubernetes.io/secret/29ee781d-d24c-460a-af83-39d31e9dfa0b-secret-metrics-server-client-certs\") pod \"metrics-server-548b5c6d96-hzzhk\" (UID: \"29ee781d-d24c-460a-af83-39d31e9dfa0b\") " pod="openshift-monitoring/metrics-server-548b5c6d96-hzzhk" Jul 03 11:13:48.174593 ip-10-0-136-0 kubenswrapper[2579]: I0703 11:13:48.174505 2579 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-qbdks\" (UniqueName: \"kubernetes.io/projected/29ee781d-d24c-460a-af83-39d31e9dfa0b-kube-api-access-qbdks\") pod \"metrics-server-548b5c6d96-hzzhk\" (UID: \"29ee781d-d24c-460a-af83-39d31e9dfa0b\") " pod="openshift-monitoring/metrics-server-548b5c6d96-hzzhk" Jul 03 11:13:48.174593 ip-10-0-136-0 kubenswrapper[2579]: I0703 11:13:48.174524 2579 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"audit-log\" (UniqueName: \"kubernetes.io/empty-dir/29ee781d-d24c-460a-af83-39d31e9dfa0b-audit-log\") pod \"metrics-server-548b5c6d96-hzzhk\" (UID: \"29ee781d-d24c-460a-af83-39d31e9dfa0b\") " pod="openshift-monitoring/metrics-server-548b5c6d96-hzzhk" Jul 03 11:13:48.175326 ip-10-0-136-0 kubenswrapper[2579]: I0703 11:13:48.175295 2579 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"configmap-kubelet-serving-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/29ee781d-d24c-460a-af83-39d31e9dfa0b-configmap-kubelet-serving-ca-bundle\") pod \"metrics-server-548b5c6d96-hzzhk\" (UID: \"29ee781d-d24c-460a-af83-39d31e9dfa0b\") " pod="openshift-monitoring/metrics-server-548b5c6d96-hzzhk" Jul 03 11:13:48.176168 ip-10-0-136-0 kubenswrapper[2579]: I0703 11:13:48.176142 2579 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"metrics-server-audit-profiles\" (UniqueName: \"kubernetes.io/configmap/29ee781d-d24c-460a-af83-39d31e9dfa0b-metrics-server-audit-profiles\") pod \"metrics-server-548b5c6d96-hzzhk\" (UID: \"29ee781d-d24c-460a-af83-39d31e9dfa0b\") " pod="openshift-monitoring/metrics-server-548b5c6d96-hzzhk" Jul 03 11:13:48.178342 ip-10-0-136-0 kubenswrapper[2579]: I0703 11:13:48.177812 2579 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"client-ca-bundle\" (UniqueName: \"kubernetes.io/secret/29ee781d-d24c-460a-af83-39d31e9dfa0b-client-ca-bundle\") pod \"metrics-server-548b5c6d96-hzzhk\" (UID: \"29ee781d-d24c-460a-af83-39d31e9dfa0b\") " pod="openshift-monitoring/metrics-server-548b5c6d96-hzzhk" Jul 03 11:13:48.178342 ip-10-0-136-0 kubenswrapper[2579]: I0703 11:13:48.177938 2579 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"secret-metrics-server-tls\" (UniqueName: \"kubernetes.io/secret/29ee781d-d24c-460a-af83-39d31e9dfa0b-secret-metrics-server-tls\") pod \"metrics-server-548b5c6d96-hzzhk\" (UID: \"29ee781d-d24c-460a-af83-39d31e9dfa0b\") " pod="openshift-monitoring/metrics-server-548b5c6d96-hzzhk" Jul 03 11:13:48.178544 ip-10-0-136-0 kubenswrapper[2579]: I0703 11:13:48.178520 2579 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"secret-metrics-server-client-certs\" (UniqueName: \"kubernetes.io/secret/29ee781d-d24c-460a-af83-39d31e9dfa0b-secret-metrics-server-client-certs\") pod \"metrics-server-548b5c6d96-hzzhk\" (UID: \"29ee781d-d24c-460a-af83-39d31e9dfa0b\") " pod="openshift-monitoring/metrics-server-548b5c6d96-hzzhk" Jul 03 11:13:48.178797 ip-10-0-136-0 kubenswrapper[2579]: I0703 11:13:48.178773 2579 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"audit-log\" (UniqueName: \"kubernetes.io/empty-dir/29ee781d-d24c-460a-af83-39d31e9dfa0b-audit-log\") pod \"metrics-server-548b5c6d96-hzzhk\" (UID: \"29ee781d-d24c-460a-af83-39d31e9dfa0b\") " pod="openshift-monitoring/metrics-server-548b5c6d96-hzzhk" Jul 03 11:13:48.188552 ip-10-0-136-0 kubenswrapper[2579]: I0703 11:13:48.188513 2579 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-qbdks\" (UniqueName: \"kubernetes.io/projected/29ee781d-d24c-460a-af83-39d31e9dfa0b-kube-api-access-qbdks\") pod \"metrics-server-548b5c6d96-hzzhk\" (UID: \"29ee781d-d24c-460a-af83-39d31e9dfa0b\") " pod="openshift-monitoring/metrics-server-548b5c6d96-hzzhk" Jul 03 11:13:48.192668 ip-10-0-136-0 kubenswrapper[2579]: I0703 11:13:48.192636 2579 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-monitoring/monitoring-plugin-d5dcff7cf-nhbzk"] Jul 03 11:13:48.217919 ip-10-0-136-0 kubenswrapper[2579]: I0703 11:13:48.217490 2579 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-monitoring/monitoring-plugin-d5dcff7cf-nhbzk"] Jul 03 11:13:48.217919 ip-10-0-136-0 kubenswrapper[2579]: I0703 11:13:48.217532 2579 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-monitoring/monitoring-plugin-d5dcff7cf-nhbzk" Jul 03 11:13:48.220342 ip-10-0-136-0 kubenswrapper[2579]: I0703 11:13:48.220314 2579 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"monitoring-plugin-cert\"" Jul 03 11:13:48.220499 ip-10-0-136-0 kubenswrapper[2579]: I0703 11:13:48.220344 2579 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"monitoring-plugin-dockercfg-r5x8z\"" Jul 03 11:13:48.268255 ip-10-0-136-0 kubenswrapper[2579]: I0703 11:13:48.268219 2579 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-monitoring/metrics-server-548b5c6d96-hzzhk" Jul 03 11:13:48.376079 ip-10-0-136-0 kubenswrapper[2579]: I0703 11:13:48.375980 2579 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"monitoring-plugin-cert\" (UniqueName: \"kubernetes.io/secret/ba918183-7a45-4db4-ba86-5cd0455ed4d9-monitoring-plugin-cert\") pod \"monitoring-plugin-d5dcff7cf-nhbzk\" (UID: \"ba918183-7a45-4db4-ba86-5cd0455ed4d9\") " pod="openshift-monitoring/monitoring-plugin-d5dcff7cf-nhbzk" Jul 03 11:13:48.376271 ip-10-0-136-0 kubenswrapper[2579]: I0703 11:13:48.376108 2579 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-7fpfh\" (UniqueName: \"kubernetes.io/projected/ba918183-7a45-4db4-ba86-5cd0455ed4d9-kube-api-access-7fpfh\") pod \"monitoring-plugin-d5dcff7cf-nhbzk\" (UID: \"ba918183-7a45-4db4-ba86-5cd0455ed4d9\") " pod="openshift-monitoring/monitoring-plugin-d5dcff7cf-nhbzk" Jul 03 11:13:48.447767 ip-10-0-136-0 kubenswrapper[2579]: I0703 11:13:48.447724 2579 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-console/console-565d858f78-nz7js"] Jul 03 11:13:48.477088 ip-10-0-136-0 kubenswrapper[2579]: I0703 11:13:48.477043 2579 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"monitoring-plugin-cert\" (UniqueName: \"kubernetes.io/secret/ba918183-7a45-4db4-ba86-5cd0455ed4d9-monitoring-plugin-cert\") pod \"monitoring-plugin-d5dcff7cf-nhbzk\" (UID: \"ba918183-7a45-4db4-ba86-5cd0455ed4d9\") " pod="openshift-monitoring/monitoring-plugin-d5dcff7cf-nhbzk" Jul 03 11:13:48.477321 ip-10-0-136-0 kubenswrapper[2579]: I0703 11:13:48.477169 2579 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-7fpfh\" (UniqueName: \"kubernetes.io/projected/ba918183-7a45-4db4-ba86-5cd0455ed4d9-kube-api-access-7fpfh\") pod \"monitoring-plugin-d5dcff7cf-nhbzk\" (UID: \"ba918183-7a45-4db4-ba86-5cd0455ed4d9\") " pod="openshift-monitoring/monitoring-plugin-d5dcff7cf-nhbzk" Jul 03 11:13:48.480346 ip-10-0-136-0 kubenswrapper[2579]: I0703 11:13:48.480311 2579 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"monitoring-plugin-cert\" (UniqueName: \"kubernetes.io/secret/ba918183-7a45-4db4-ba86-5cd0455ed4d9-monitoring-plugin-cert\") pod \"monitoring-plugin-d5dcff7cf-nhbzk\" (UID: \"ba918183-7a45-4db4-ba86-5cd0455ed4d9\") " pod="openshift-monitoring/monitoring-plugin-d5dcff7cf-nhbzk" Jul 03 11:13:48.493151 ip-10-0-136-0 kubenswrapper[2579]: I0703 11:13:48.493110 2579 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-7fpfh\" (UniqueName: \"kubernetes.io/projected/ba918183-7a45-4db4-ba86-5cd0455ed4d9-kube-api-access-7fpfh\") pod \"monitoring-plugin-d5dcff7cf-nhbzk\" (UID: \"ba918183-7a45-4db4-ba86-5cd0455ed4d9\") " pod="openshift-monitoring/monitoring-plugin-d5dcff7cf-nhbzk" Jul 03 11:13:48.531535 ip-10-0-136-0 kubenswrapper[2579]: I0703 11:13:48.531493 2579 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-monitoring/monitoring-plugin-d5dcff7cf-nhbzk" Jul 03 11:13:51.368005 ip-10-0-136-0 kubenswrapper[2579]: I0703 11:13:51.367968 2579 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-console/console-565d858f78-nz7js" Jul 03 11:13:51.548173 ip-10-0-136-0 kubenswrapper[2579]: W0703 11:13:51.548138 2579 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podb2866e40_a413_4cc8_978c_65b100d290ac.slice/crio-fa2a22c4f4699d94e72b8fb409c9c83545990a46ec63dc8a68ac9567a708c903 WatchSource:0}: Error finding container fa2a22c4f4699d94e72b8fb409c9c83545990a46ec63dc8a68ac9567a708c903: Status 404 returned error can't find the container with id fa2a22c4f4699d94e72b8fb409c9c83545990a46ec63dc8a68ac9567a708c903 Jul 03 11:13:51.596984 ip-10-0-136-0 kubenswrapper[2579]: I0703 11:13:51.595448 2579 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/node-exporter-xj7rs" event={"ID":"b2866e40-a413-4cc8-978c-65b100d290ac","Type":"ContainerStarted","Data":"fa2a22c4f4699d94e72b8fb409c9c83545990a46ec63dc8a68ac9567a708c903"} Jul 03 11:13:51.686297 ip-10-0-136-0 kubenswrapper[2579]: I0703 11:13:51.686250 2579 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-monitoring/monitoring-plugin-d5dcff7cf-nhbzk"] Jul 03 11:13:51.689342 ip-10-0-136-0 kubenswrapper[2579]: W0703 11:13:51.689279 2579 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podba918183_7a45_4db4_ba86_5cd0455ed4d9.slice/crio-5c45e33e113e658f69d71502e92de9414783005e7ce10b7c15a9eb13fd466010 WatchSource:0}: Error finding container 5c45e33e113e658f69d71502e92de9414783005e7ce10b7c15a9eb13fd466010: Status 404 returned error can't find the container with id 5c45e33e113e658f69d71502e92de9414783005e7ce10b7c15a9eb13fd466010 Jul 03 11:13:51.708695 ip-10-0-136-0 kubenswrapper[2579]: I0703 11:13:51.708660 2579 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-monitoring/metrics-server-548b5c6d96-hzzhk"] Jul 03 11:13:51.712012 ip-10-0-136-0 kubenswrapper[2579]: W0703 11:13:51.711967 2579 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod29ee781d_d24c_460a_af83_39d31e9dfa0b.slice/crio-9ed5ab403455c39e7e1f305b76bdfc2f9705643655a6f2913f01bc55e8cb5c96 WatchSource:0}: Error finding container 9ed5ab403455c39e7e1f305b76bdfc2f9705643655a6f2913f01bc55e8cb5c96: Status 404 returned error can't find the container with id 9ed5ab403455c39e7e1f305b76bdfc2f9705643655a6f2913f01bc55e8cb5c96 Jul 03 11:13:52.602567 ip-10-0-136-0 kubenswrapper[2579]: I0703 11:13:52.602520 2579 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-console/downloads-685bfb5f64-cngk5" event={"ID":"a74b2c4b-20bf-4e91-85b7-c827a9c92d1b","Type":"ContainerStarted","Data":"1ad7cf16e7c56fc8a644375a56356bf86c9bb812dd224d47b2b1463bffe8ead4"} Jul 03 11:13:52.603135 ip-10-0-136-0 kubenswrapper[2579]: I0703 11:13:52.602773 2579 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-console/downloads-685bfb5f64-cngk5" Jul 03 11:13:52.604577 ip-10-0-136-0 kubenswrapper[2579]: I0703 11:13:52.604532 2579 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/monitoring-plugin-d5dcff7cf-nhbzk" event={"ID":"ba918183-7a45-4db4-ba86-5cd0455ed4d9","Type":"ContainerStarted","Data":"5c45e33e113e658f69d71502e92de9414783005e7ce10b7c15a9eb13fd466010"} Jul 03 11:13:52.606138 ip-10-0-136-0 kubenswrapper[2579]: I0703 11:13:52.606105 2579 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/metrics-server-548b5c6d96-hzzhk" event={"ID":"29ee781d-d24c-460a-af83-39d31e9dfa0b","Type":"ContainerStarted","Data":"9ed5ab403455c39e7e1f305b76bdfc2f9705643655a6f2913f01bc55e8cb5c96"} Jul 03 11:13:52.616759 ip-10-0-136-0 kubenswrapper[2579]: I0703 11:13:52.616720 2579 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-console/downloads-685bfb5f64-cngk5" Jul 03 11:13:52.627395 ip-10-0-136-0 kubenswrapper[2579]: I0703 11:13:52.624614 2579 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-console/downloads-685bfb5f64-cngk5" podStartSLOduration=2.116081641 podStartE2EDuration="22.624589111s" totalImagesPullingTime="20.50850747s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-03 11:13:30 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-03 11:13:52.621467155 +0000 UTC m=+89.209771503" watchObservedRunningTime="2026-07-03 11:13:52.624589111 +0000 UTC m=+89.212893430" Jul 03 11:13:53.616022 ip-10-0-136-0 kubenswrapper[2579]: I0703 11:13:53.615967 2579 generic.go:356] "Generic (PLEG): container finished" podID="b2866e40-a413-4cc8-978c-65b100d290ac" containerID="1bf53931c99864c8fbd859f54db700bd97f25be84a1d107ebaf807feba4529b7" exitCode=0 Jul 03 11:13:53.616646 ip-10-0-136-0 kubenswrapper[2579]: I0703 11:13:53.616118 2579 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/node-exporter-xj7rs" event={"ID":"b2866e40-a413-4cc8-978c-65b100d290ac","Type":"ContainerDied","Data":"1bf53931c99864c8fbd859f54db700bd97f25be84a1d107ebaf807feba4529b7"} Jul 03 11:13:54.501566 ip-10-0-136-0 kubenswrapper[2579]: I0703 11:13:54.501532 2579 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-image-registry/image-registry-74fd49c4c-5v265" Jul 03 11:13:54.626862 ip-10-0-136-0 kubenswrapper[2579]: I0703 11:13:54.626269 2579 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/node-exporter-xj7rs" event={"ID":"b2866e40-a413-4cc8-978c-65b100d290ac","Type":"ContainerStarted","Data":"1627fc84c4d30c6f32df8fa677781e5df428d49268bc6be6cb4801cf75ecc7d0"} Jul 03 11:13:54.626862 ip-10-0-136-0 kubenswrapper[2579]: I0703 11:13:54.626419 2579 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/node-exporter-xj7rs" event={"ID":"b2866e40-a413-4cc8-978c-65b100d290ac","Type":"ContainerStarted","Data":"c8e1ad241075272e7d1abdb06d7d6e42a5bcfe6b74425efad2faa83547385fb5"} Jul 03 11:13:55.574085 ip-10-0-136-0 kubenswrapper[2579]: I0703 11:13:55.574028 2579 kuberuntime_container.go:872] "Killing container with a grace period" pod="openshift-image-registry/image-registry-659bf79d58-p9gqp" podUID="387cc87f-815e-457a-83cf-d016035fed5d" containerName="registry" containerID="cri-o://089af8b23bb2f9a8f44bc4c414478fe3a43c64827691f9eb9089bef89987978b" gracePeriod=30 Jul 03 11:13:55.952482 ip-10-0-136-0 kubenswrapper[2579]: I0703 11:13:55.952439 2579 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-image-registry/image-registry-659bf79d58-p9gqp" Jul 03 11:13:55.974203 ip-10-0-136-0 kubenswrapper[2579]: I0703 11:13:55.974035 2579 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-monitoring/node-exporter-xj7rs" podStartSLOduration=11.496112524 podStartE2EDuration="12.974011144s" totalImagesPullingTime="1.309942698s" totalInitContainerRuntime="167.955922ms" isStatefulPod=true podCreationTimestamp="2026-07-03 11:13:43 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-03 11:13:54.650274708 +0000 UTC m=+91.238579028" watchObservedRunningTime="2026-07-03 11:13:55.974011144 +0000 UTC m=+92.562315465" Jul 03 11:13:56.061700 ip-10-0-136-0 kubenswrapper[2579]: I0703 11:13:56.061643 2579 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"installation-pull-secrets\" (UniqueName: \"kubernetes.io/secret/387cc87f-815e-457a-83cf-d016035fed5d-installation-pull-secrets\") pod \"387cc87f-815e-457a-83cf-d016035fed5d\" (UID: \"387cc87f-815e-457a-83cf-d016035fed5d\") " Jul 03 11:13:56.061907 ip-10-0-136-0 kubenswrapper[2579]: I0703 11:13:56.061722 2579 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"registry-tls\" (UniqueName: \"kubernetes.io/projected/387cc87f-815e-457a-83cf-d016035fed5d-registry-tls\") pod \"387cc87f-815e-457a-83cf-d016035fed5d\" (UID: \"387cc87f-815e-457a-83cf-d016035fed5d\") " Jul 03 11:13:56.061907 ip-10-0-136-0 kubenswrapper[2579]: I0703 11:13:56.061749 2579 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"bound-sa-token\" (UniqueName: \"kubernetes.io/projected/387cc87f-815e-457a-83cf-d016035fed5d-bound-sa-token\") pod \"387cc87f-815e-457a-83cf-d016035fed5d\" (UID: \"387cc87f-815e-457a-83cf-d016035fed5d\") " Jul 03 11:13:56.061907 ip-10-0-136-0 kubenswrapper[2579]: I0703 11:13:56.061792 2579 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"image-registry-private-configuration\" (UniqueName: \"kubernetes.io/secret/387cc87f-815e-457a-83cf-d016035fed5d-image-registry-private-configuration\") pod \"387cc87f-815e-457a-83cf-d016035fed5d\" (UID: \"387cc87f-815e-457a-83cf-d016035fed5d\") " Jul 03 11:13:56.061907 ip-10-0-136-0 kubenswrapper[2579]: I0703 11:13:56.061896 2579 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"ca-trust-extracted\" (UniqueName: \"kubernetes.io/empty-dir/387cc87f-815e-457a-83cf-d016035fed5d-ca-trust-extracted\") pod \"387cc87f-815e-457a-83cf-d016035fed5d\" (UID: \"387cc87f-815e-457a-83cf-d016035fed5d\") " Jul 03 11:13:56.062126 ip-10-0-136-0 kubenswrapper[2579]: I0703 11:13:56.061938 2579 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"registry-certificates\" (UniqueName: \"kubernetes.io/configmap/387cc87f-815e-457a-83cf-d016035fed5d-registry-certificates\") pod \"387cc87f-815e-457a-83cf-d016035fed5d\" (UID: \"387cc87f-815e-457a-83cf-d016035fed5d\") " Jul 03 11:13:56.062126 ip-10-0-136-0 kubenswrapper[2579]: I0703 11:13:56.061969 2579 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"trusted-ca\" (UniqueName: \"kubernetes.io/configmap/387cc87f-815e-457a-83cf-d016035fed5d-trusted-ca\") pod \"387cc87f-815e-457a-83cf-d016035fed5d\" (UID: \"387cc87f-815e-457a-83cf-d016035fed5d\") " Jul 03 11:13:56.062126 ip-10-0-136-0 kubenswrapper[2579]: I0703 11:13:56.062012 2579 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-9jp4s\" (UniqueName: \"kubernetes.io/projected/387cc87f-815e-457a-83cf-d016035fed5d-kube-api-access-9jp4s\") pod \"387cc87f-815e-457a-83cf-d016035fed5d\" (UID: \"387cc87f-815e-457a-83cf-d016035fed5d\") " Jul 03 11:13:56.062945 ip-10-0-136-0 kubenswrapper[2579]: I0703 11:13:56.062901 2579 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/387cc87f-815e-457a-83cf-d016035fed5d-registry-certificates" (OuterVolumeSpecName: "registry-certificates") pod "387cc87f-815e-457a-83cf-d016035fed5d" (UID: "387cc87f-815e-457a-83cf-d016035fed5d"). InnerVolumeSpecName "registry-certificates". PluginName "kubernetes.io/configmap", VolumeGIDValue "" Jul 03 11:13:56.063560 ip-10-0-136-0 kubenswrapper[2579]: I0703 11:13:56.063518 2579 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/387cc87f-815e-457a-83cf-d016035fed5d-trusted-ca" (OuterVolumeSpecName: "trusted-ca") pod "387cc87f-815e-457a-83cf-d016035fed5d" (UID: "387cc87f-815e-457a-83cf-d016035fed5d"). InnerVolumeSpecName "trusted-ca". PluginName "kubernetes.io/configmap", VolumeGIDValue "" Jul 03 11:13:56.067900 ip-10-0-136-0 kubenswrapper[2579]: I0703 11:13:56.067812 2579 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/387cc87f-815e-457a-83cf-d016035fed5d-installation-pull-secrets" (OuterVolumeSpecName: "installation-pull-secrets") pod "387cc87f-815e-457a-83cf-d016035fed5d" (UID: "387cc87f-815e-457a-83cf-d016035fed5d"). InnerVolumeSpecName "installation-pull-secrets". PluginName "kubernetes.io/secret", VolumeGIDValue "" Jul 03 11:13:56.068089 ip-10-0-136-0 kubenswrapper[2579]: I0703 11:13:56.067955 2579 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/387cc87f-815e-457a-83cf-d016035fed5d-bound-sa-token" (OuterVolumeSpecName: "bound-sa-token") pod "387cc87f-815e-457a-83cf-d016035fed5d" (UID: "387cc87f-815e-457a-83cf-d016035fed5d"). InnerVolumeSpecName "bound-sa-token". PluginName "kubernetes.io/projected", VolumeGIDValue "" Jul 03 11:13:56.068089 ip-10-0-136-0 kubenswrapper[2579]: I0703 11:13:56.068019 2579 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/387cc87f-815e-457a-83cf-d016035fed5d-image-registry-private-configuration" (OuterVolumeSpecName: "image-registry-private-configuration") pod "387cc87f-815e-457a-83cf-d016035fed5d" (UID: "387cc87f-815e-457a-83cf-d016035fed5d"). InnerVolumeSpecName "image-registry-private-configuration". PluginName "kubernetes.io/secret", VolumeGIDValue "" Jul 03 11:13:56.068089 ip-10-0-136-0 kubenswrapper[2579]: I0703 11:13:56.068076 2579 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/387cc87f-815e-457a-83cf-d016035fed5d-registry-tls" (OuterVolumeSpecName: "registry-tls") pod "387cc87f-815e-457a-83cf-d016035fed5d" (UID: "387cc87f-815e-457a-83cf-d016035fed5d"). InnerVolumeSpecName "registry-tls". PluginName "kubernetes.io/projected", VolumeGIDValue "" Jul 03 11:13:56.070040 ip-10-0-136-0 kubenswrapper[2579]: I0703 11:13:56.069993 2579 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/387cc87f-815e-457a-83cf-d016035fed5d-kube-api-access-9jp4s" (OuterVolumeSpecName: "kube-api-access-9jp4s") pod "387cc87f-815e-457a-83cf-d016035fed5d" (UID: "387cc87f-815e-457a-83cf-d016035fed5d"). InnerVolumeSpecName "kube-api-access-9jp4s". PluginName "kubernetes.io/projected", VolumeGIDValue "" Jul 03 11:13:56.076683 ip-10-0-136-0 kubenswrapper[2579]: I0703 11:13:56.076619 2579 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/387cc87f-815e-457a-83cf-d016035fed5d-ca-trust-extracted" (OuterVolumeSpecName: "ca-trust-extracted") pod "387cc87f-815e-457a-83cf-d016035fed5d" (UID: "387cc87f-815e-457a-83cf-d016035fed5d"). InnerVolumeSpecName "ca-trust-extracted". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Jul 03 11:13:56.163336 ip-10-0-136-0 kubenswrapper[2579]: I0703 11:13:56.163244 2579 reconciler_common.go:299] "Volume detached for volume \"installation-pull-secrets\" (UniqueName: \"kubernetes.io/secret/387cc87f-815e-457a-83cf-d016035fed5d-installation-pull-secrets\") on node \"ip-10-0-136-0.ec2.internal\" DevicePath \"\"" Jul 03 11:13:56.163336 ip-10-0-136-0 kubenswrapper[2579]: I0703 11:13:56.163286 2579 reconciler_common.go:299] "Volume detached for volume \"registry-tls\" (UniqueName: \"kubernetes.io/projected/387cc87f-815e-457a-83cf-d016035fed5d-registry-tls\") on node \"ip-10-0-136-0.ec2.internal\" DevicePath \"\"" Jul 03 11:13:56.163336 ip-10-0-136-0 kubenswrapper[2579]: I0703 11:13:56.163301 2579 reconciler_common.go:299] "Volume detached for volume \"bound-sa-token\" (UniqueName: \"kubernetes.io/projected/387cc87f-815e-457a-83cf-d016035fed5d-bound-sa-token\") on node \"ip-10-0-136-0.ec2.internal\" DevicePath \"\"" Jul 03 11:13:56.163336 ip-10-0-136-0 kubenswrapper[2579]: I0703 11:13:56.163316 2579 reconciler_common.go:299] "Volume detached for volume \"image-registry-private-configuration\" (UniqueName: \"kubernetes.io/secret/387cc87f-815e-457a-83cf-d016035fed5d-image-registry-private-configuration\") on node \"ip-10-0-136-0.ec2.internal\" DevicePath \"\"" Jul 03 11:13:56.163336 ip-10-0-136-0 kubenswrapper[2579]: I0703 11:13:56.163330 2579 reconciler_common.go:299] "Volume detached for volume \"ca-trust-extracted\" (UniqueName: \"kubernetes.io/empty-dir/387cc87f-815e-457a-83cf-d016035fed5d-ca-trust-extracted\") on node \"ip-10-0-136-0.ec2.internal\" DevicePath \"\"" Jul 03 11:13:56.163708 ip-10-0-136-0 kubenswrapper[2579]: I0703 11:13:56.163344 2579 reconciler_common.go:299] "Volume detached for volume \"registry-certificates\" (UniqueName: \"kubernetes.io/configmap/387cc87f-815e-457a-83cf-d016035fed5d-registry-certificates\") on node \"ip-10-0-136-0.ec2.internal\" DevicePath \"\"" Jul 03 11:13:56.163708 ip-10-0-136-0 kubenswrapper[2579]: I0703 11:13:56.163357 2579 reconciler_common.go:299] "Volume detached for volume \"trusted-ca\" (UniqueName: \"kubernetes.io/configmap/387cc87f-815e-457a-83cf-d016035fed5d-trusted-ca\") on node \"ip-10-0-136-0.ec2.internal\" DevicePath \"\"" Jul 03 11:13:56.163708 ip-10-0-136-0 kubenswrapper[2579]: I0703 11:13:56.163369 2579 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-9jp4s\" (UniqueName: \"kubernetes.io/projected/387cc87f-815e-457a-83cf-d016035fed5d-kube-api-access-9jp4s\") on node \"ip-10-0-136-0.ec2.internal\" DevicePath \"\"" Jul 03 11:13:56.638219 ip-10-0-136-0 kubenswrapper[2579]: I0703 11:13:56.638158 2579 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/metrics-server-548b5c6d96-hzzhk" event={"ID":"29ee781d-d24c-460a-af83-39d31e9dfa0b","Type":"ContainerStarted","Data":"5d70f3df3efbd75f24831b35cc2385620c69491d759f5ffcdd50288b7e46a266"} Jul 03 11:13:56.640172 ip-10-0-136-0 kubenswrapper[2579]: I0703 11:13:56.640132 2579 generic.go:356] "Generic (PLEG): container finished" podID="387cc87f-815e-457a-83cf-d016035fed5d" containerID="089af8b23bb2f9a8f44bc4c414478fe3a43c64827691f9eb9089bef89987978b" exitCode=0 Jul 03 11:13:56.640388 ip-10-0-136-0 kubenswrapper[2579]: I0703 11:13:56.640176 2579 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-image-registry/image-registry-659bf79d58-p9gqp" event={"ID":"387cc87f-815e-457a-83cf-d016035fed5d","Type":"ContainerDied","Data":"089af8b23bb2f9a8f44bc4c414478fe3a43c64827691f9eb9089bef89987978b"} Jul 03 11:13:56.640388 ip-10-0-136-0 kubenswrapper[2579]: I0703 11:13:56.640215 2579 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-image-registry/image-registry-659bf79d58-p9gqp" Jul 03 11:13:56.640388 ip-10-0-136-0 kubenswrapper[2579]: I0703 11:13:56.640258 2579 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-image-registry/image-registry-659bf79d58-p9gqp" event={"ID":"387cc87f-815e-457a-83cf-d016035fed5d","Type":"ContainerDied","Data":"3287ba5cc39962a6290b4de6be0f067fe5b158bcff445eaf62d07abffdbf5f7d"} Jul 03 11:13:56.640388 ip-10-0-136-0 kubenswrapper[2579]: I0703 11:13:56.640286 2579 scope.go:117] "RemoveContainer" containerID="089af8b23bb2f9a8f44bc4c414478fe3a43c64827691f9eb9089bef89987978b" Jul 03 11:13:56.644273 ip-10-0-136-0 kubenswrapper[2579]: I0703 11:13:56.644231 2579 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/monitoring-plugin-d5dcff7cf-nhbzk" event={"ID":"ba918183-7a45-4db4-ba86-5cd0455ed4d9","Type":"ContainerStarted","Data":"e0d2530ad3c60a207e8010d56513ae087353b954ed24ab70eb31ed39854aef56"} Jul 03 11:13:56.645405 ip-10-0-136-0 kubenswrapper[2579]: I0703 11:13:56.645375 2579 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-monitoring/monitoring-plugin-d5dcff7cf-nhbzk" Jul 03 11:13:56.652507 ip-10-0-136-0 kubenswrapper[2579]: I0703 11:13:56.652467 2579 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-monitoring/monitoring-plugin-d5dcff7cf-nhbzk" Jul 03 11:13:56.656219 ip-10-0-136-0 kubenswrapper[2579]: I0703 11:13:56.656150 2579 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-monitoring/metrics-server-548b5c6d96-hzzhk" podStartSLOduration=5.961837974 podStartE2EDuration="9.656130897s" totalImagesPullingTime="3.694292923s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-03 11:13:47 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-03 11:13:56.654531539 +0000 UTC m=+93.242835861" watchObservedRunningTime="2026-07-03 11:13:56.656130897 +0000 UTC m=+93.244435215" Jul 03 11:13:56.667108 ip-10-0-136-0 kubenswrapper[2579]: I0703 11:13:56.667081 2579 scope.go:117] "RemoveContainer" containerID="089af8b23bb2f9a8f44bc4c414478fe3a43c64827691f9eb9089bef89987978b" Jul 03 11:13:56.667560 ip-10-0-136-0 kubenswrapper[2579]: E0703 11:13:56.667531 2579 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"089af8b23bb2f9a8f44bc4c414478fe3a43c64827691f9eb9089bef89987978b\": container with ID starting with 089af8b23bb2f9a8f44bc4c414478fe3a43c64827691f9eb9089bef89987978b not found: ID does not exist" containerID="089af8b23bb2f9a8f44bc4c414478fe3a43c64827691f9eb9089bef89987978b" Jul 03 11:13:56.667661 ip-10-0-136-0 kubenswrapper[2579]: I0703 11:13:56.667576 2579 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"089af8b23bb2f9a8f44bc4c414478fe3a43c64827691f9eb9089bef89987978b"} err="failed to get container status \"089af8b23bb2f9a8f44bc4c414478fe3a43c64827691f9eb9089bef89987978b\": rpc error: code = NotFound desc = could not find container \"089af8b23bb2f9a8f44bc4c414478fe3a43c64827691f9eb9089bef89987978b\": container with ID starting with 089af8b23bb2f9a8f44bc4c414478fe3a43c64827691f9eb9089bef89987978b not found: ID does not exist" Jul 03 11:13:56.671387 ip-10-0-136-0 kubenswrapper[2579]: I0703 11:13:56.671325 2579 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-monitoring/monitoring-plugin-d5dcff7cf-nhbzk" podStartSLOduration=4.756283054 podStartE2EDuration="8.671301941s" totalImagesPullingTime="3.915018887s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-03 11:13:48 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-03 11:13:56.670163365 +0000 UTC m=+93.258467697" watchObservedRunningTime="2026-07-03 11:13:56.671301941 +0000 UTC m=+93.259606254" Jul 03 11:13:56.686882 ip-10-0-136-0 kubenswrapper[2579]: I0703 11:13:56.686839 2579 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-image-registry/image-registry-659bf79d58-p9gqp"] Jul 03 11:13:56.695206 ip-10-0-136-0 kubenswrapper[2579]: I0703 11:13:56.695138 2579 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["openshift-image-registry/image-registry-659bf79d58-p9gqp"] Jul 03 11:13:57.992701 ip-10-0-136-0 kubenswrapper[2579]: I0703 11:13:57.992656 2579 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="387cc87f-815e-457a-83cf-d016035fed5d" path="/var/lib/kubelet/pods/387cc87f-815e-457a-83cf-d016035fed5d/volumes" Jul 03 11:13:59.844733 ip-10-0-136-0 kubenswrapper[2579]: I0703 11:13:59.844694 2579 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-console/console-55fd9ddfdd-klc4c"] Jul 03 11:14:08.268754 ip-10-0-136-0 kubenswrapper[2579]: I0703 11:14:08.268708 2579 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="openshift-monitoring/metrics-server-548b5c6d96-hzzhk" Jul 03 11:14:08.269246 ip-10-0-136-0 kubenswrapper[2579]: I0703 11:14:08.268777 2579 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-monitoring/metrics-server-548b5c6d96-hzzhk" Jul 03 11:14:10.539063 ip-10-0-136-0 kubenswrapper[2579]: I0703 11:14:10.539031 2579 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-network-diagnostics/network-check-target-xzlsc" Jul 03 11:14:13.473315 ip-10-0-136-0 kubenswrapper[2579]: I0703 11:14:13.473229 2579 kuberuntime_container.go:872] "Killing container with a grace period" pod="openshift-console/console-565d858f78-nz7js" podUID="c09906a1-28fe-47f1-ae02-46a934363524" containerName="console" containerID="cri-o://e7e31782a6fd35b6c33c7868ef65f5aab8f4d96d696016cf637c652ca58f146a" gracePeriod=15 Jul 03 11:14:13.713727 ip-10-0-136-0 kubenswrapper[2579]: I0703 11:14:13.713691 2579 generic.go:356] "Generic (PLEG): container finished" podID="c09906a1-28fe-47f1-ae02-46a934363524" containerID="e7e31782a6fd35b6c33c7868ef65f5aab8f4d96d696016cf637c652ca58f146a" exitCode=0 Jul 03 11:14:13.713897 ip-10-0-136-0 kubenswrapper[2579]: I0703 11:14:13.713760 2579 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-console/console-565d858f78-nz7js" event={"ID":"c09906a1-28fe-47f1-ae02-46a934363524","Type":"ContainerDied","Data":"e7e31782a6fd35b6c33c7868ef65f5aab8f4d96d696016cf637c652ca58f146a"} Jul 03 11:14:13.757430 ip-10-0-136-0 kubenswrapper[2579]: I0703 11:14:13.757392 2579 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-console/console-565d858f78-nz7js" Jul 03 11:14:13.824152 ip-10-0-136-0 kubenswrapper[2579]: I0703 11:14:13.824114 2579 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-7dbzx\" (UniqueName: \"kubernetes.io/projected/c09906a1-28fe-47f1-ae02-46a934363524-kube-api-access-7dbzx\") pod \"c09906a1-28fe-47f1-ae02-46a934363524\" (UID: \"c09906a1-28fe-47f1-ae02-46a934363524\") " Jul 03 11:14:13.824355 ip-10-0-136-0 kubenswrapper[2579]: I0703 11:14:13.824179 2579 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"service-ca\" (UniqueName: \"kubernetes.io/configmap/c09906a1-28fe-47f1-ae02-46a934363524-service-ca\") pod \"c09906a1-28fe-47f1-ae02-46a934363524\" (UID: \"c09906a1-28fe-47f1-ae02-46a934363524\") " Jul 03 11:14:13.824355 ip-10-0-136-0 kubenswrapper[2579]: I0703 11:14:13.824224 2579 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"oauth-serving-cert\" (UniqueName: \"kubernetes.io/configmap/c09906a1-28fe-47f1-ae02-46a934363524-oauth-serving-cert\") pod \"c09906a1-28fe-47f1-ae02-46a934363524\" (UID: \"c09906a1-28fe-47f1-ae02-46a934363524\") " Jul 03 11:14:13.824355 ip-10-0-136-0 kubenswrapper[2579]: I0703 11:14:13.824254 2579 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"console-oauth-config\" (UniqueName: \"kubernetes.io/secret/c09906a1-28fe-47f1-ae02-46a934363524-console-oauth-config\") pod \"c09906a1-28fe-47f1-ae02-46a934363524\" (UID: \"c09906a1-28fe-47f1-ae02-46a934363524\") " Jul 03 11:14:13.824355 ip-10-0-136-0 kubenswrapper[2579]: I0703 11:14:13.824298 2579 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"console-serving-cert\" (UniqueName: \"kubernetes.io/secret/c09906a1-28fe-47f1-ae02-46a934363524-console-serving-cert\") pod \"c09906a1-28fe-47f1-ae02-46a934363524\" (UID: \"c09906a1-28fe-47f1-ae02-46a934363524\") " Jul 03 11:14:13.824355 ip-10-0-136-0 kubenswrapper[2579]: I0703 11:14:13.824332 2579 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"console-config\" (UniqueName: \"kubernetes.io/configmap/c09906a1-28fe-47f1-ae02-46a934363524-console-config\") pod \"c09906a1-28fe-47f1-ae02-46a934363524\" (UID: \"c09906a1-28fe-47f1-ae02-46a934363524\") " Jul 03 11:14:13.824562 ip-10-0-136-0 kubenswrapper[2579]: I0703 11:14:13.824359 2579 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/c09906a1-28fe-47f1-ae02-46a934363524-trusted-ca-bundle\") pod \"c09906a1-28fe-47f1-ae02-46a934363524\" (UID: \"c09906a1-28fe-47f1-ae02-46a934363524\") " Jul 03 11:14:13.824819 ip-10-0-136-0 kubenswrapper[2579]: I0703 11:14:13.824762 2579 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/c09906a1-28fe-47f1-ae02-46a934363524-oauth-serving-cert" (OuterVolumeSpecName: "oauth-serving-cert") pod "c09906a1-28fe-47f1-ae02-46a934363524" (UID: "c09906a1-28fe-47f1-ae02-46a934363524"). InnerVolumeSpecName "oauth-serving-cert". PluginName "kubernetes.io/configmap", VolumeGIDValue "" Jul 03 11:14:13.824819 ip-10-0-136-0 kubenswrapper[2579]: I0703 11:14:13.824726 2579 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/c09906a1-28fe-47f1-ae02-46a934363524-service-ca" (OuterVolumeSpecName: "service-ca") pod "c09906a1-28fe-47f1-ae02-46a934363524" (UID: "c09906a1-28fe-47f1-ae02-46a934363524"). InnerVolumeSpecName "service-ca". PluginName "kubernetes.io/configmap", VolumeGIDValue "" Jul 03 11:14:13.825209 ip-10-0-136-0 kubenswrapper[2579]: I0703 11:14:13.824873 2579 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/c09906a1-28fe-47f1-ae02-46a934363524-console-config" (OuterVolumeSpecName: "console-config") pod "c09906a1-28fe-47f1-ae02-46a934363524" (UID: "c09906a1-28fe-47f1-ae02-46a934363524"). InnerVolumeSpecName "console-config". PluginName "kubernetes.io/configmap", VolumeGIDValue "" Jul 03 11:14:13.825209 ip-10-0-136-0 kubenswrapper[2579]: I0703 11:14:13.824902 2579 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/c09906a1-28fe-47f1-ae02-46a934363524-trusted-ca-bundle" (OuterVolumeSpecName: "trusted-ca-bundle") pod "c09906a1-28fe-47f1-ae02-46a934363524" (UID: "c09906a1-28fe-47f1-ae02-46a934363524"). InnerVolumeSpecName "trusted-ca-bundle". PluginName "kubernetes.io/configmap", VolumeGIDValue "" Jul 03 11:14:13.826790 ip-10-0-136-0 kubenswrapper[2579]: I0703 11:14:13.826729 2579 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/c09906a1-28fe-47f1-ae02-46a934363524-kube-api-access-7dbzx" (OuterVolumeSpecName: "kube-api-access-7dbzx") pod "c09906a1-28fe-47f1-ae02-46a934363524" (UID: "c09906a1-28fe-47f1-ae02-46a934363524"). InnerVolumeSpecName "kube-api-access-7dbzx". PluginName "kubernetes.io/projected", VolumeGIDValue "" Jul 03 11:14:13.827104 ip-10-0-136-0 kubenswrapper[2579]: I0703 11:14:13.826820 2579 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/c09906a1-28fe-47f1-ae02-46a934363524-console-serving-cert" (OuterVolumeSpecName: "console-serving-cert") pod "c09906a1-28fe-47f1-ae02-46a934363524" (UID: "c09906a1-28fe-47f1-ae02-46a934363524"). InnerVolumeSpecName "console-serving-cert". PluginName "kubernetes.io/secret", VolumeGIDValue "" Jul 03 11:14:13.828115 ip-10-0-136-0 kubenswrapper[2579]: I0703 11:14:13.828088 2579 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/c09906a1-28fe-47f1-ae02-46a934363524-console-oauth-config" (OuterVolumeSpecName: "console-oauth-config") pod "c09906a1-28fe-47f1-ae02-46a934363524" (UID: "c09906a1-28fe-47f1-ae02-46a934363524"). InnerVolumeSpecName "console-oauth-config". PluginName "kubernetes.io/secret", VolumeGIDValue "" Jul 03 11:14:13.925659 ip-10-0-136-0 kubenswrapper[2579]: I0703 11:14:13.925616 2579 reconciler_common.go:299] "Volume detached for volume \"service-ca\" (UniqueName: \"kubernetes.io/configmap/c09906a1-28fe-47f1-ae02-46a934363524-service-ca\") on node \"ip-10-0-136-0.ec2.internal\" DevicePath \"\"" Jul 03 11:14:13.925659 ip-10-0-136-0 kubenswrapper[2579]: I0703 11:14:13.925649 2579 reconciler_common.go:299] "Volume detached for volume \"oauth-serving-cert\" (UniqueName: \"kubernetes.io/configmap/c09906a1-28fe-47f1-ae02-46a934363524-oauth-serving-cert\") on node \"ip-10-0-136-0.ec2.internal\" DevicePath \"\"" Jul 03 11:14:13.925659 ip-10-0-136-0 kubenswrapper[2579]: I0703 11:14:13.925658 2579 reconciler_common.go:299] "Volume detached for volume \"console-oauth-config\" (UniqueName: \"kubernetes.io/secret/c09906a1-28fe-47f1-ae02-46a934363524-console-oauth-config\") on node \"ip-10-0-136-0.ec2.internal\" DevicePath \"\"" Jul 03 11:14:13.925659 ip-10-0-136-0 kubenswrapper[2579]: I0703 11:14:13.925668 2579 reconciler_common.go:299] "Volume detached for volume \"console-serving-cert\" (UniqueName: \"kubernetes.io/secret/c09906a1-28fe-47f1-ae02-46a934363524-console-serving-cert\") on node \"ip-10-0-136-0.ec2.internal\" DevicePath \"\"" Jul 03 11:14:13.925659 ip-10-0-136-0 kubenswrapper[2579]: I0703 11:14:13.925675 2579 reconciler_common.go:299] "Volume detached for volume \"console-config\" (UniqueName: \"kubernetes.io/configmap/c09906a1-28fe-47f1-ae02-46a934363524-console-config\") on node \"ip-10-0-136-0.ec2.internal\" DevicePath \"\"" Jul 03 11:14:13.925969 ip-10-0-136-0 kubenswrapper[2579]: I0703 11:14:13.925683 2579 reconciler_common.go:299] "Volume detached for volume \"trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/c09906a1-28fe-47f1-ae02-46a934363524-trusted-ca-bundle\") on node \"ip-10-0-136-0.ec2.internal\" DevicePath \"\"" Jul 03 11:14:13.925969 ip-10-0-136-0 kubenswrapper[2579]: I0703 11:14:13.925693 2579 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-7dbzx\" (UniqueName: \"kubernetes.io/projected/c09906a1-28fe-47f1-ae02-46a934363524-kube-api-access-7dbzx\") on node \"ip-10-0-136-0.ec2.internal\" DevicePath \"\"" Jul 03 11:14:14.719345 ip-10-0-136-0 kubenswrapper[2579]: I0703 11:14:14.719299 2579 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-console/console-565d858f78-nz7js" event={"ID":"c09906a1-28fe-47f1-ae02-46a934363524","Type":"ContainerDied","Data":"0eb31c92accb70ccfc4e84d8f8f0ab3dd2ef45973db51637006fd476c3ba69fa"} Jul 03 11:14:14.719872 ip-10-0-136-0 kubenswrapper[2579]: I0703 11:14:14.719403 2579 scope.go:117] "RemoveContainer" containerID="e7e31782a6fd35b6c33c7868ef65f5aab8f4d96d696016cf637c652ca58f146a" Jul 03 11:14:14.719872 ip-10-0-136-0 kubenswrapper[2579]: I0703 11:14:14.719326 2579 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-console/console-565d858f78-nz7js" Jul 03 11:14:14.739344 ip-10-0-136-0 kubenswrapper[2579]: I0703 11:14:14.739310 2579 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-console/console-565d858f78-nz7js"] Jul 03 11:14:14.741510 ip-10-0-136-0 kubenswrapper[2579]: I0703 11:14:14.741482 2579 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["openshift-console/console-565d858f78-nz7js"] Jul 03 11:14:15.991055 ip-10-0-136-0 kubenswrapper[2579]: I0703 11:14:15.991013 2579 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="c09906a1-28fe-47f1-ae02-46a934363524" path="/var/lib/kubelet/pods/c09906a1-28fe-47f1-ae02-46a934363524/volumes" Jul 03 11:14:24.870503 ip-10-0-136-0 kubenswrapper[2579]: I0703 11:14:24.870434 2579 kuberuntime_container.go:872] "Killing container with a grace period" pod="openshift-console/console-55fd9ddfdd-klc4c" podUID="8937bcc1-fce2-40e9-ac92-e25c9a902506" containerName="console" containerID="cri-o://f69b69694a1d8034197965404d7de6d86b54707bce0d7861b11309f83a0b3bad" gracePeriod=15 Jul 03 11:14:25.145771 ip-10-0-136-0 kubenswrapper[2579]: I0703 11:14:25.145680 2579 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-console/console-55fd9ddfdd-klc4c" Jul 03 11:14:25.330389 ip-10-0-136-0 kubenswrapper[2579]: I0703 11:14:25.330354 2579 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"oauth-serving-cert\" (UniqueName: \"kubernetes.io/configmap/8937bcc1-fce2-40e9-ac92-e25c9a902506-oauth-serving-cert\") pod \"8937bcc1-fce2-40e9-ac92-e25c9a902506\" (UID: \"8937bcc1-fce2-40e9-ac92-e25c9a902506\") " Jul 03 11:14:25.330637 ip-10-0-136-0 kubenswrapper[2579]: I0703 11:14:25.330413 2579 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"service-ca\" (UniqueName: \"kubernetes.io/configmap/8937bcc1-fce2-40e9-ac92-e25c9a902506-service-ca\") pod \"8937bcc1-fce2-40e9-ac92-e25c9a902506\" (UID: \"8937bcc1-fce2-40e9-ac92-e25c9a902506\") " Jul 03 11:14:25.330637 ip-10-0-136-0 kubenswrapper[2579]: I0703 11:14:25.330441 2579 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"console-serving-cert\" (UniqueName: \"kubernetes.io/secret/8937bcc1-fce2-40e9-ac92-e25c9a902506-console-serving-cert\") pod \"8937bcc1-fce2-40e9-ac92-e25c9a902506\" (UID: \"8937bcc1-fce2-40e9-ac92-e25c9a902506\") " Jul 03 11:14:25.330637 ip-10-0-136-0 kubenswrapper[2579]: I0703 11:14:25.330491 2579 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"console-oauth-config\" (UniqueName: \"kubernetes.io/secret/8937bcc1-fce2-40e9-ac92-e25c9a902506-console-oauth-config\") pod \"8937bcc1-fce2-40e9-ac92-e25c9a902506\" (UID: \"8937bcc1-fce2-40e9-ac92-e25c9a902506\") " Jul 03 11:14:25.330637 ip-10-0-136-0 kubenswrapper[2579]: I0703 11:14:25.330512 2579 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"console-config\" (UniqueName: \"kubernetes.io/configmap/8937bcc1-fce2-40e9-ac92-e25c9a902506-console-config\") pod \"8937bcc1-fce2-40e9-ac92-e25c9a902506\" (UID: \"8937bcc1-fce2-40e9-ac92-e25c9a902506\") " Jul 03 11:14:25.330637 ip-10-0-136-0 kubenswrapper[2579]: I0703 11:14:25.330550 2579 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-z5mct\" (UniqueName: \"kubernetes.io/projected/8937bcc1-fce2-40e9-ac92-e25c9a902506-kube-api-access-z5mct\") pod \"8937bcc1-fce2-40e9-ac92-e25c9a902506\" (UID: \"8937bcc1-fce2-40e9-ac92-e25c9a902506\") " Jul 03 11:14:25.330904 ip-10-0-136-0 kubenswrapper[2579]: I0703 11:14:25.330863 2579 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/8937bcc1-fce2-40e9-ac92-e25c9a902506-oauth-serving-cert" (OuterVolumeSpecName: "oauth-serving-cert") pod "8937bcc1-fce2-40e9-ac92-e25c9a902506" (UID: "8937bcc1-fce2-40e9-ac92-e25c9a902506"). InnerVolumeSpecName "oauth-serving-cert". PluginName "kubernetes.io/configmap", VolumeGIDValue "" Jul 03 11:14:25.330964 ip-10-0-136-0 kubenswrapper[2579]: I0703 11:14:25.330890 2579 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/8937bcc1-fce2-40e9-ac92-e25c9a902506-service-ca" (OuterVolumeSpecName: "service-ca") pod "8937bcc1-fce2-40e9-ac92-e25c9a902506" (UID: "8937bcc1-fce2-40e9-ac92-e25c9a902506"). InnerVolumeSpecName "service-ca". PluginName "kubernetes.io/configmap", VolumeGIDValue "" Jul 03 11:14:25.331094 ip-10-0-136-0 kubenswrapper[2579]: I0703 11:14:25.331066 2579 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/8937bcc1-fce2-40e9-ac92-e25c9a902506-console-config" (OuterVolumeSpecName: "console-config") pod "8937bcc1-fce2-40e9-ac92-e25c9a902506" (UID: "8937bcc1-fce2-40e9-ac92-e25c9a902506"). InnerVolumeSpecName "console-config". PluginName "kubernetes.io/configmap", VolumeGIDValue "" Jul 03 11:14:25.332989 ip-10-0-136-0 kubenswrapper[2579]: I0703 11:14:25.332954 2579 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/8937bcc1-fce2-40e9-ac92-e25c9a902506-console-serving-cert" (OuterVolumeSpecName: "console-serving-cert") pod "8937bcc1-fce2-40e9-ac92-e25c9a902506" (UID: "8937bcc1-fce2-40e9-ac92-e25c9a902506"). InnerVolumeSpecName "console-serving-cert". PluginName "kubernetes.io/secret", VolumeGIDValue "" Jul 03 11:14:25.333122 ip-10-0-136-0 kubenswrapper[2579]: I0703 11:14:25.333015 2579 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/8937bcc1-fce2-40e9-ac92-e25c9a902506-console-oauth-config" (OuterVolumeSpecName: "console-oauth-config") pod "8937bcc1-fce2-40e9-ac92-e25c9a902506" (UID: "8937bcc1-fce2-40e9-ac92-e25c9a902506"). InnerVolumeSpecName "console-oauth-config". PluginName "kubernetes.io/secret", VolumeGIDValue "" Jul 03 11:14:25.333215 ip-10-0-136-0 kubenswrapper[2579]: I0703 11:14:25.333176 2579 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/8937bcc1-fce2-40e9-ac92-e25c9a902506-kube-api-access-z5mct" (OuterVolumeSpecName: "kube-api-access-z5mct") pod "8937bcc1-fce2-40e9-ac92-e25c9a902506" (UID: "8937bcc1-fce2-40e9-ac92-e25c9a902506"). InnerVolumeSpecName "kube-api-access-z5mct". PluginName "kubernetes.io/projected", VolumeGIDValue "" Jul 03 11:14:25.431620 ip-10-0-136-0 kubenswrapper[2579]: I0703 11:14:25.431522 2579 reconciler_common.go:299] "Volume detached for volume \"console-oauth-config\" (UniqueName: \"kubernetes.io/secret/8937bcc1-fce2-40e9-ac92-e25c9a902506-console-oauth-config\") on node \"ip-10-0-136-0.ec2.internal\" DevicePath \"\"" Jul 03 11:14:25.431620 ip-10-0-136-0 kubenswrapper[2579]: I0703 11:14:25.431556 2579 reconciler_common.go:299] "Volume detached for volume \"console-config\" (UniqueName: \"kubernetes.io/configmap/8937bcc1-fce2-40e9-ac92-e25c9a902506-console-config\") on node \"ip-10-0-136-0.ec2.internal\" DevicePath \"\"" Jul 03 11:14:25.431620 ip-10-0-136-0 kubenswrapper[2579]: I0703 11:14:25.431566 2579 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-z5mct\" (UniqueName: \"kubernetes.io/projected/8937bcc1-fce2-40e9-ac92-e25c9a902506-kube-api-access-z5mct\") on node \"ip-10-0-136-0.ec2.internal\" DevicePath \"\"" Jul 03 11:14:25.431620 ip-10-0-136-0 kubenswrapper[2579]: I0703 11:14:25.431576 2579 reconciler_common.go:299] "Volume detached for volume \"oauth-serving-cert\" (UniqueName: \"kubernetes.io/configmap/8937bcc1-fce2-40e9-ac92-e25c9a902506-oauth-serving-cert\") on node \"ip-10-0-136-0.ec2.internal\" DevicePath \"\"" Jul 03 11:14:25.431620 ip-10-0-136-0 kubenswrapper[2579]: I0703 11:14:25.431585 2579 reconciler_common.go:299] "Volume detached for volume \"service-ca\" (UniqueName: \"kubernetes.io/configmap/8937bcc1-fce2-40e9-ac92-e25c9a902506-service-ca\") on node \"ip-10-0-136-0.ec2.internal\" DevicePath \"\"" Jul 03 11:14:25.431620 ip-10-0-136-0 kubenswrapper[2579]: I0703 11:14:25.431595 2579 reconciler_common.go:299] "Volume detached for volume \"console-serving-cert\" (UniqueName: \"kubernetes.io/secret/8937bcc1-fce2-40e9-ac92-e25c9a902506-console-serving-cert\") on node \"ip-10-0-136-0.ec2.internal\" DevicePath \"\"" Jul 03 11:14:25.760949 ip-10-0-136-0 kubenswrapper[2579]: I0703 11:14:25.760843 2579 generic.go:356] "Generic (PLEG): container finished" podID="8937bcc1-fce2-40e9-ac92-e25c9a902506" containerID="f69b69694a1d8034197965404d7de6d86b54707bce0d7861b11309f83a0b3bad" exitCode=0 Jul 03 11:14:25.760949 ip-10-0-136-0 kubenswrapper[2579]: I0703 11:14:25.760898 2579 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-console/console-55fd9ddfdd-klc4c" Jul 03 11:14:25.760949 ip-10-0-136-0 kubenswrapper[2579]: I0703 11:14:25.760940 2579 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-console/console-55fd9ddfdd-klc4c" event={"ID":"8937bcc1-fce2-40e9-ac92-e25c9a902506","Type":"ContainerDied","Data":"f69b69694a1d8034197965404d7de6d86b54707bce0d7861b11309f83a0b3bad"} Jul 03 11:14:25.761265 ip-10-0-136-0 kubenswrapper[2579]: I0703 11:14:25.760989 2579 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-console/console-55fd9ddfdd-klc4c" event={"ID":"8937bcc1-fce2-40e9-ac92-e25c9a902506","Type":"ContainerDied","Data":"bc9e330832ada0a1092b3588970773a177df4472f8dc40cfc9c69bed53b4ed66"} Jul 03 11:14:25.761265 ip-10-0-136-0 kubenswrapper[2579]: I0703 11:14:25.761008 2579 scope.go:117] "RemoveContainer" containerID="f69b69694a1d8034197965404d7de6d86b54707bce0d7861b11309f83a0b3bad" Jul 03 11:14:25.769850 ip-10-0-136-0 kubenswrapper[2579]: I0703 11:14:25.769830 2579 scope.go:117] "RemoveContainer" containerID="f69b69694a1d8034197965404d7de6d86b54707bce0d7861b11309f83a0b3bad" Jul 03 11:14:25.770150 ip-10-0-136-0 kubenswrapper[2579]: E0703 11:14:25.770126 2579 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"f69b69694a1d8034197965404d7de6d86b54707bce0d7861b11309f83a0b3bad\": container with ID starting with f69b69694a1d8034197965404d7de6d86b54707bce0d7861b11309f83a0b3bad not found: ID does not exist" containerID="f69b69694a1d8034197965404d7de6d86b54707bce0d7861b11309f83a0b3bad" Jul 03 11:14:25.770343 ip-10-0-136-0 kubenswrapper[2579]: I0703 11:14:25.770164 2579 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"f69b69694a1d8034197965404d7de6d86b54707bce0d7861b11309f83a0b3bad"} err="failed to get container status \"f69b69694a1d8034197965404d7de6d86b54707bce0d7861b11309f83a0b3bad\": rpc error: code = NotFound desc = could not find container \"f69b69694a1d8034197965404d7de6d86b54707bce0d7861b11309f83a0b3bad\": container with ID starting with f69b69694a1d8034197965404d7de6d86b54707bce0d7861b11309f83a0b3bad not found: ID does not exist" Jul 03 11:14:25.783757 ip-10-0-136-0 kubenswrapper[2579]: I0703 11:14:25.783713 2579 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-console/console-55fd9ddfdd-klc4c"] Jul 03 11:14:25.787932 ip-10-0-136-0 kubenswrapper[2579]: I0703 11:14:25.787882 2579 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["openshift-console/console-55fd9ddfdd-klc4c"] Jul 03 11:14:25.990723 ip-10-0-136-0 kubenswrapper[2579]: I0703 11:14:25.990683 2579 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="8937bcc1-fce2-40e9-ac92-e25c9a902506" path="/var/lib/kubelet/pods/8937bcc1-fce2-40e9-ac92-e25c9a902506/volumes" Jul 03 11:14:28.274142 ip-10-0-136-0 kubenswrapper[2579]: I0703 11:14:28.274108 2579 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="openshift-monitoring/metrics-server-548b5c6d96-hzzhk" Jul 03 11:14:28.278495 ip-10-0-136-0 kubenswrapper[2579]: I0703 11:14:28.278459 2579 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-monitoring/metrics-server-548b5c6d96-hzzhk" Jul 03 11:14:29.517243 ip-10-0-136-0 kubenswrapper[2579]: I0703 11:14:29.517216 2579 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ingress-canary_ingress-canary-2nclq_868c9601-3c66-498e-8076-e7e289b9a4a0/serve-healthcheck-canary/0.log" Jul 03 11:15:07.748345 ip-10-0-136-0 kubenswrapper[2579]: I0703 11:15:07.748307 2579 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-monitoring/telemeter-client-65586ccccd-lcgbt"] Jul 03 11:15:07.748772 ip-10-0-136-0 kubenswrapper[2579]: I0703 11:15:07.748550 2579 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="8937bcc1-fce2-40e9-ac92-e25c9a902506" containerName="console" Jul 03 11:15:07.748772 ip-10-0-136-0 kubenswrapper[2579]: I0703 11:15:07.748560 2579 state_mem.go:107] "Deleted CPUSet assignment" podUID="8937bcc1-fce2-40e9-ac92-e25c9a902506" containerName="console" Jul 03 11:15:07.748772 ip-10-0-136-0 kubenswrapper[2579]: I0703 11:15:07.748570 2579 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="c09906a1-28fe-47f1-ae02-46a934363524" containerName="console" Jul 03 11:15:07.748772 ip-10-0-136-0 kubenswrapper[2579]: I0703 11:15:07.748575 2579 state_mem.go:107] "Deleted CPUSet assignment" podUID="c09906a1-28fe-47f1-ae02-46a934363524" containerName="console" Jul 03 11:15:07.748772 ip-10-0-136-0 kubenswrapper[2579]: I0703 11:15:07.748583 2579 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="387cc87f-815e-457a-83cf-d016035fed5d" containerName="registry" Jul 03 11:15:07.748772 ip-10-0-136-0 kubenswrapper[2579]: I0703 11:15:07.748589 2579 state_mem.go:107] "Deleted CPUSet assignment" podUID="387cc87f-815e-457a-83cf-d016035fed5d" containerName="registry" Jul 03 11:15:07.748772 ip-10-0-136-0 kubenswrapper[2579]: I0703 11:15:07.748641 2579 memory_manager.go:368] "RemoveStaleState removing state" podUID="8937bcc1-fce2-40e9-ac92-e25c9a902506" containerName="console" Jul 03 11:15:07.748772 ip-10-0-136-0 kubenswrapper[2579]: I0703 11:15:07.748648 2579 memory_manager.go:368] "RemoveStaleState removing state" podUID="387cc87f-815e-457a-83cf-d016035fed5d" containerName="registry" Jul 03 11:15:07.748772 ip-10-0-136-0 kubenswrapper[2579]: I0703 11:15:07.748655 2579 memory_manager.go:368] "RemoveStaleState removing state" podUID="c09906a1-28fe-47f1-ae02-46a934363524" containerName="console" Jul 03 11:15:07.751853 ip-10-0-136-0 kubenswrapper[2579]: I0703 11:15:07.751821 2579 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-monitoring/telemeter-client-65586ccccd-lcgbt" Jul 03 11:15:07.754870 ip-10-0-136-0 kubenswrapper[2579]: I0703 11:15:07.754838 2579 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"telemeter-client-tls\"" Jul 03 11:15:07.755036 ip-10-0-136-0 kubenswrapper[2579]: I0703 11:15:07.754886 2579 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"telemeter-client-kube-rbac-proxy-config\"" Jul 03 11:15:07.755098 ip-10-0-136-0 kubenswrapper[2579]: I0703 11:15:07.755035 2579 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"telemeter-client-dockercfg-k4w2v\"" Jul 03 11:15:07.755235 ip-10-0-136-0 kubenswrapper[2579]: I0703 11:15:07.755215 2579 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"telemeter-client\"" Jul 03 11:15:07.755301 ip-10-0-136-0 kubenswrapper[2579]: I0703 11:15:07.755219 2579 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"federate-client-certs\"" Jul 03 11:15:07.755880 ip-10-0-136-0 kubenswrapper[2579]: I0703 11:15:07.755861 2579 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-monitoring\"/\"telemeter-client-serving-certs-ca-bundle\"" Jul 03 11:15:07.761881 ip-10-0-136-0 kubenswrapper[2579]: I0703 11:15:07.761631 2579 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-monitoring\"/\"telemeter-trusted-ca-bundle-8i12ta5c71j38\"" Jul 03 11:15:07.773084 ip-10-0-136-0 kubenswrapper[2579]: I0703 11:15:07.773045 2579 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-monitoring/telemeter-client-65586ccccd-lcgbt"] Jul 03 11:15:07.858794 ip-10-0-136-0 kubenswrapper[2579]: I0703 11:15:07.858751 2579 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"secret-telemeter-client\" (UniqueName: \"kubernetes.io/secret/ace7baa1-5d40-4395-80ba-37e4a232eb20-secret-telemeter-client\") pod \"telemeter-client-65586ccccd-lcgbt\" (UID: \"ace7baa1-5d40-4395-80ba-37e4a232eb20\") " pod="openshift-monitoring/telemeter-client-65586ccccd-lcgbt" Jul 03 11:15:07.858794 ip-10-0-136-0 kubenswrapper[2579]: I0703 11:15:07.858796 2579 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"federate-client-tls\" (UniqueName: \"kubernetes.io/secret/ace7baa1-5d40-4395-80ba-37e4a232eb20-federate-client-tls\") pod \"telemeter-client-65586ccccd-lcgbt\" (UID: \"ace7baa1-5d40-4395-80ba-37e4a232eb20\") " pod="openshift-monitoring/telemeter-client-65586ccccd-lcgbt" Jul 03 11:15:07.859005 ip-10-0-136-0 kubenswrapper[2579]: I0703 11:15:07.858859 2579 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"telemeter-client-tls\" (UniqueName: \"kubernetes.io/secret/ace7baa1-5d40-4395-80ba-37e4a232eb20-telemeter-client-tls\") pod \"telemeter-client-65586ccccd-lcgbt\" (UID: \"ace7baa1-5d40-4395-80ba-37e4a232eb20\") " pod="openshift-monitoring/telemeter-client-65586ccccd-lcgbt" Jul 03 11:15:07.859005 ip-10-0-136-0 kubenswrapper[2579]: I0703 11:15:07.858898 2579 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"telemeter-trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/ace7baa1-5d40-4395-80ba-37e4a232eb20-telemeter-trusted-ca-bundle\") pod \"telemeter-client-65586ccccd-lcgbt\" (UID: \"ace7baa1-5d40-4395-80ba-37e4a232eb20\") " pod="openshift-monitoring/telemeter-client-65586ccccd-lcgbt" Jul 03 11:15:07.859005 ip-10-0-136-0 kubenswrapper[2579]: I0703 11:15:07.858928 2579 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"metrics-client-ca\" (UniqueName: \"kubernetes.io/configmap/ace7baa1-5d40-4395-80ba-37e4a232eb20-metrics-client-ca\") pod \"telemeter-client-65586ccccd-lcgbt\" (UID: \"ace7baa1-5d40-4395-80ba-37e4a232eb20\") " pod="openshift-monitoring/telemeter-client-65586ccccd-lcgbt" Jul 03 11:15:07.859005 ip-10-0-136-0 kubenswrapper[2579]: I0703 11:15:07.858945 2579 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"serving-certs-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/ace7baa1-5d40-4395-80ba-37e4a232eb20-serving-certs-ca-bundle\") pod \"telemeter-client-65586ccccd-lcgbt\" (UID: \"ace7baa1-5d40-4395-80ba-37e4a232eb20\") " pod="openshift-monitoring/telemeter-client-65586ccccd-lcgbt" Jul 03 11:15:07.859130 ip-10-0-136-0 kubenswrapper[2579]: I0703 11:15:07.859010 2579 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-vzc7b\" (UniqueName: \"kubernetes.io/projected/ace7baa1-5d40-4395-80ba-37e4a232eb20-kube-api-access-vzc7b\") pod \"telemeter-client-65586ccccd-lcgbt\" (UID: \"ace7baa1-5d40-4395-80ba-37e4a232eb20\") " pod="openshift-monitoring/telemeter-client-65586ccccd-lcgbt" Jul 03 11:15:07.859130 ip-10-0-136-0 kubenswrapper[2579]: I0703 11:15:07.859078 2579 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"secret-telemeter-client-kube-rbac-proxy-config\" (UniqueName: \"kubernetes.io/secret/ace7baa1-5d40-4395-80ba-37e4a232eb20-secret-telemeter-client-kube-rbac-proxy-config\") pod \"telemeter-client-65586ccccd-lcgbt\" (UID: \"ace7baa1-5d40-4395-80ba-37e4a232eb20\") " pod="openshift-monitoring/telemeter-client-65586ccccd-lcgbt" Jul 03 11:15:07.960523 ip-10-0-136-0 kubenswrapper[2579]: I0703 11:15:07.960474 2579 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"federate-client-tls\" (UniqueName: \"kubernetes.io/secret/ace7baa1-5d40-4395-80ba-37e4a232eb20-federate-client-tls\") pod \"telemeter-client-65586ccccd-lcgbt\" (UID: \"ace7baa1-5d40-4395-80ba-37e4a232eb20\") " pod="openshift-monitoring/telemeter-client-65586ccccd-lcgbt" Jul 03 11:15:07.960725 ip-10-0-136-0 kubenswrapper[2579]: I0703 11:15:07.960536 2579 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"telemeter-client-tls\" (UniqueName: \"kubernetes.io/secret/ace7baa1-5d40-4395-80ba-37e4a232eb20-telemeter-client-tls\") pod \"telemeter-client-65586ccccd-lcgbt\" (UID: \"ace7baa1-5d40-4395-80ba-37e4a232eb20\") " pod="openshift-monitoring/telemeter-client-65586ccccd-lcgbt" Jul 03 11:15:07.960725 ip-10-0-136-0 kubenswrapper[2579]: I0703 11:15:07.960584 2579 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"telemeter-trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/ace7baa1-5d40-4395-80ba-37e4a232eb20-telemeter-trusted-ca-bundle\") pod \"telemeter-client-65586ccccd-lcgbt\" (UID: \"ace7baa1-5d40-4395-80ba-37e4a232eb20\") " pod="openshift-monitoring/telemeter-client-65586ccccd-lcgbt" Jul 03 11:15:07.960725 ip-10-0-136-0 kubenswrapper[2579]: I0703 11:15:07.960615 2579 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-client-ca\" (UniqueName: \"kubernetes.io/configmap/ace7baa1-5d40-4395-80ba-37e4a232eb20-metrics-client-ca\") pod \"telemeter-client-65586ccccd-lcgbt\" (UID: \"ace7baa1-5d40-4395-80ba-37e4a232eb20\") " pod="openshift-monitoring/telemeter-client-65586ccccd-lcgbt" Jul 03 11:15:07.960725 ip-10-0-136-0 kubenswrapper[2579]: I0703 11:15:07.960639 2579 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"serving-certs-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/ace7baa1-5d40-4395-80ba-37e4a232eb20-serving-certs-ca-bundle\") pod \"telemeter-client-65586ccccd-lcgbt\" (UID: \"ace7baa1-5d40-4395-80ba-37e4a232eb20\") " pod="openshift-monitoring/telemeter-client-65586ccccd-lcgbt" Jul 03 11:15:07.960725 ip-10-0-136-0 kubenswrapper[2579]: I0703 11:15:07.960668 2579 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-vzc7b\" (UniqueName: \"kubernetes.io/projected/ace7baa1-5d40-4395-80ba-37e4a232eb20-kube-api-access-vzc7b\") pod \"telemeter-client-65586ccccd-lcgbt\" (UID: \"ace7baa1-5d40-4395-80ba-37e4a232eb20\") " pod="openshift-monitoring/telemeter-client-65586ccccd-lcgbt" Jul 03 11:15:07.960725 ip-10-0-136-0 kubenswrapper[2579]: I0703 11:15:07.960718 2579 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"secret-telemeter-client-kube-rbac-proxy-config\" (UniqueName: \"kubernetes.io/secret/ace7baa1-5d40-4395-80ba-37e4a232eb20-secret-telemeter-client-kube-rbac-proxy-config\") pod \"telemeter-client-65586ccccd-lcgbt\" (UID: \"ace7baa1-5d40-4395-80ba-37e4a232eb20\") " pod="openshift-monitoring/telemeter-client-65586ccccd-lcgbt" Jul 03 11:15:07.960953 ip-10-0-136-0 kubenswrapper[2579]: I0703 11:15:07.960750 2579 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"secret-telemeter-client\" (UniqueName: \"kubernetes.io/secret/ace7baa1-5d40-4395-80ba-37e4a232eb20-secret-telemeter-client\") pod \"telemeter-client-65586ccccd-lcgbt\" (UID: \"ace7baa1-5d40-4395-80ba-37e4a232eb20\") " pod="openshift-monitoring/telemeter-client-65586ccccd-lcgbt" Jul 03 11:15:07.961842 ip-10-0-136-0 kubenswrapper[2579]: I0703 11:15:07.961809 2579 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"telemeter-trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/ace7baa1-5d40-4395-80ba-37e4a232eb20-telemeter-trusted-ca-bundle\") pod \"telemeter-client-65586ccccd-lcgbt\" (UID: \"ace7baa1-5d40-4395-80ba-37e4a232eb20\") " pod="openshift-monitoring/telemeter-client-65586ccccd-lcgbt" Jul 03 11:15:07.962040 ip-10-0-136-0 kubenswrapper[2579]: I0703 11:15:07.961976 2579 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"serving-certs-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/ace7baa1-5d40-4395-80ba-37e4a232eb20-serving-certs-ca-bundle\") pod \"telemeter-client-65586ccccd-lcgbt\" (UID: \"ace7baa1-5d40-4395-80ba-37e4a232eb20\") " pod="openshift-monitoring/telemeter-client-65586ccccd-lcgbt" Jul 03 11:15:07.962259 ip-10-0-136-0 kubenswrapper[2579]: I0703 11:15:07.962233 2579 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"metrics-client-ca\" (UniqueName: \"kubernetes.io/configmap/ace7baa1-5d40-4395-80ba-37e4a232eb20-metrics-client-ca\") pod \"telemeter-client-65586ccccd-lcgbt\" (UID: \"ace7baa1-5d40-4395-80ba-37e4a232eb20\") " pod="openshift-monitoring/telemeter-client-65586ccccd-lcgbt" Jul 03 11:15:07.963682 ip-10-0-136-0 kubenswrapper[2579]: I0703 11:15:07.963650 2579 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"federate-client-tls\" (UniqueName: \"kubernetes.io/secret/ace7baa1-5d40-4395-80ba-37e4a232eb20-federate-client-tls\") pod \"telemeter-client-65586ccccd-lcgbt\" (UID: \"ace7baa1-5d40-4395-80ba-37e4a232eb20\") " pod="openshift-monitoring/telemeter-client-65586ccccd-lcgbt" Jul 03 11:15:07.963979 ip-10-0-136-0 kubenswrapper[2579]: I0703 11:15:07.963953 2579 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"secret-telemeter-client-kube-rbac-proxy-config\" (UniqueName: \"kubernetes.io/secret/ace7baa1-5d40-4395-80ba-37e4a232eb20-secret-telemeter-client-kube-rbac-proxy-config\") pod \"telemeter-client-65586ccccd-lcgbt\" (UID: \"ace7baa1-5d40-4395-80ba-37e4a232eb20\") " pod="openshift-monitoring/telemeter-client-65586ccccd-lcgbt" Jul 03 11:15:07.964214 ip-10-0-136-0 kubenswrapper[2579]: I0703 11:15:07.964166 2579 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"telemeter-client-tls\" (UniqueName: \"kubernetes.io/secret/ace7baa1-5d40-4395-80ba-37e4a232eb20-telemeter-client-tls\") pod \"telemeter-client-65586ccccd-lcgbt\" (UID: \"ace7baa1-5d40-4395-80ba-37e4a232eb20\") " pod="openshift-monitoring/telemeter-client-65586ccccd-lcgbt" Jul 03 11:15:07.964280 ip-10-0-136-0 kubenswrapper[2579]: I0703 11:15:07.964211 2579 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"secret-telemeter-client\" (UniqueName: \"kubernetes.io/secret/ace7baa1-5d40-4395-80ba-37e4a232eb20-secret-telemeter-client\") pod \"telemeter-client-65586ccccd-lcgbt\" (UID: \"ace7baa1-5d40-4395-80ba-37e4a232eb20\") " pod="openshift-monitoring/telemeter-client-65586ccccd-lcgbt" Jul 03 11:15:07.971282 ip-10-0-136-0 kubenswrapper[2579]: I0703 11:15:07.971251 2579 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-vzc7b\" (UniqueName: \"kubernetes.io/projected/ace7baa1-5d40-4395-80ba-37e4a232eb20-kube-api-access-vzc7b\") pod \"telemeter-client-65586ccccd-lcgbt\" (UID: \"ace7baa1-5d40-4395-80ba-37e4a232eb20\") " pod="openshift-monitoring/telemeter-client-65586ccccd-lcgbt" Jul 03 11:15:08.064831 ip-10-0-136-0 kubenswrapper[2579]: I0703 11:15:08.064781 2579 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-monitoring/telemeter-client-65586ccccd-lcgbt" Jul 03 11:15:08.237123 ip-10-0-136-0 kubenswrapper[2579]: I0703 11:15:08.237081 2579 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-monitoring/telemeter-client-65586ccccd-lcgbt"] Jul 03 11:15:08.240461 ip-10-0-136-0 kubenswrapper[2579]: W0703 11:15:08.240424 2579 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podace7baa1_5d40_4395_80ba_37e4a232eb20.slice/crio-0b66a7b675c69fb27371160d10af16895943b1c9f81ef95a12992c83fdc08a4c WatchSource:0}: Error finding container 0b66a7b675c69fb27371160d10af16895943b1c9f81ef95a12992c83fdc08a4c: Status 404 returned error can't find the container with id 0b66a7b675c69fb27371160d10af16895943b1c9f81ef95a12992c83fdc08a4c Jul 03 11:15:08.925559 ip-10-0-136-0 kubenswrapper[2579]: I0703 11:15:08.925520 2579 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/telemeter-client-65586ccccd-lcgbt" event={"ID":"ace7baa1-5d40-4395-80ba-37e4a232eb20","Type":"ContainerStarted","Data":"0b66a7b675c69fb27371160d10af16895943b1c9f81ef95a12992c83fdc08a4c"} Jul 03 11:15:10.937253 ip-10-0-136-0 kubenswrapper[2579]: I0703 11:15:10.937216 2579 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/telemeter-client-65586ccccd-lcgbt" event={"ID":"ace7baa1-5d40-4395-80ba-37e4a232eb20","Type":"ContainerStarted","Data":"3de395b11a7423d3484ff7e4631613a30ddb8e49be06896e0b7551f30d5f0b7f"} Jul 03 11:15:11.942734 ip-10-0-136-0 kubenswrapper[2579]: I0703 11:15:11.942678 2579 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/telemeter-client-65586ccccd-lcgbt" event={"ID":"ace7baa1-5d40-4395-80ba-37e4a232eb20","Type":"ContainerStarted","Data":"455a1b1953499deab88ffcd701774fd29b23a8ca60edb390cd46074ebd4588f4"} Jul 03 11:15:11.943272 ip-10-0-136-0 kubenswrapper[2579]: I0703 11:15:11.942758 2579 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/telemeter-client-65586ccccd-lcgbt" event={"ID":"ace7baa1-5d40-4395-80ba-37e4a232eb20","Type":"ContainerStarted","Data":"673305e262e6429a0a86d3305bcd439ce1861dd75b6f9d3b1aceb5ee57bb1f2f"} Jul 03 11:15:11.968087 ip-10-0-136-0 kubenswrapper[2579]: I0703 11:15:11.968029 2579 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-monitoring/telemeter-client-65586ccccd-lcgbt" podStartSLOduration=1.826932352 podStartE2EDuration="4.96800964s" totalImagesPullingTime="3.141077288s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-03 11:15:07 +0000 UTC" imagePullSessionsCount=2 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-03 11:15:11.966608689 +0000 UTC m=+168.554913006" watchObservedRunningTime="2026-07-03 11:15:11.96800964 +0000 UTC m=+168.556313979" Jul 03 11:15:12.720363 ip-10-0-136-0 kubenswrapper[2579]: I0703 11:15:12.720308 2579 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-console/console-764db899bc-8xjvj"] Jul 03 11:15:12.723575 ip-10-0-136-0 kubenswrapper[2579]: I0703 11:15:12.723539 2579 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-console/console-764db899bc-8xjvj" Jul 03 11:15:12.726472 ip-10-0-136-0 kubenswrapper[2579]: I0703 11:15:12.726441 2579 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-console\"/\"console-config\"" Jul 03 11:15:12.726776 ip-10-0-136-0 kubenswrapper[2579]: I0703 11:15:12.726441 2579 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-console\"/\"oauth-serving-cert\"" Jul 03 11:15:12.726852 ip-10-0-136-0 kubenswrapper[2579]: I0703 11:15:12.726783 2579 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-console\"/\"console-oauth-config\"" Jul 03 11:15:12.727517 ip-10-0-136-0 kubenswrapper[2579]: I0703 11:15:12.727494 2579 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-console\"/\"service-ca\"" Jul 03 11:15:12.728530 ip-10-0-136-0 kubenswrapper[2579]: I0703 11:15:12.728511 2579 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-console\"/\"console-dockercfg-49rmj\"" Jul 03 11:15:12.730981 ip-10-0-136-0 kubenswrapper[2579]: I0703 11:15:12.730839 2579 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-console\"/\"console-serving-cert\"" Jul 03 11:15:12.732402 ip-10-0-136-0 kubenswrapper[2579]: I0703 11:15:12.732380 2579 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-console\"/\"trusted-ca-bundle\"" Jul 03 11:15:12.737458 ip-10-0-136-0 kubenswrapper[2579]: I0703 11:15:12.737430 2579 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-console/console-764db899bc-8xjvj"] Jul 03 11:15:12.797250 ip-10-0-136-0 kubenswrapper[2579]: I0703 11:15:12.797207 2579 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-nkfk2\" (UniqueName: \"kubernetes.io/projected/c92e8a04-9147-45f4-8a71-9a9bd95671f5-kube-api-access-nkfk2\") pod \"console-764db899bc-8xjvj\" (UID: \"c92e8a04-9147-45f4-8a71-9a9bd95671f5\") " pod="openshift-console/console-764db899bc-8xjvj" Jul 03 11:15:12.797250 ip-10-0-136-0 kubenswrapper[2579]: I0703 11:15:12.797255 2579 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"console-serving-cert\" (UniqueName: \"kubernetes.io/secret/c92e8a04-9147-45f4-8a71-9a9bd95671f5-console-serving-cert\") pod \"console-764db899bc-8xjvj\" (UID: \"c92e8a04-9147-45f4-8a71-9a9bd95671f5\") " pod="openshift-console/console-764db899bc-8xjvj" Jul 03 11:15:12.797449 ip-10-0-136-0 kubenswrapper[2579]: I0703 11:15:12.797286 2579 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"console-config\" (UniqueName: \"kubernetes.io/configmap/c92e8a04-9147-45f4-8a71-9a9bd95671f5-console-config\") pod \"console-764db899bc-8xjvj\" (UID: \"c92e8a04-9147-45f4-8a71-9a9bd95671f5\") " pod="openshift-console/console-764db899bc-8xjvj" Jul 03 11:15:12.797449 ip-10-0-136-0 kubenswrapper[2579]: I0703 11:15:12.797314 2579 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"oauth-serving-cert\" (UniqueName: \"kubernetes.io/configmap/c92e8a04-9147-45f4-8a71-9a9bd95671f5-oauth-serving-cert\") pod \"console-764db899bc-8xjvj\" (UID: \"c92e8a04-9147-45f4-8a71-9a9bd95671f5\") " pod="openshift-console/console-764db899bc-8xjvj" Jul 03 11:15:12.797517 ip-10-0-136-0 kubenswrapper[2579]: I0703 11:15:12.797458 2579 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"service-ca\" (UniqueName: \"kubernetes.io/configmap/c92e8a04-9147-45f4-8a71-9a9bd95671f5-service-ca\") pod \"console-764db899bc-8xjvj\" (UID: \"c92e8a04-9147-45f4-8a71-9a9bd95671f5\") " pod="openshift-console/console-764db899bc-8xjvj" Jul 03 11:15:12.797517 ip-10-0-136-0 kubenswrapper[2579]: I0703 11:15:12.797507 2579 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"console-oauth-config\" (UniqueName: \"kubernetes.io/secret/c92e8a04-9147-45f4-8a71-9a9bd95671f5-console-oauth-config\") pod \"console-764db899bc-8xjvj\" (UID: \"c92e8a04-9147-45f4-8a71-9a9bd95671f5\") " pod="openshift-console/console-764db899bc-8xjvj" Jul 03 11:15:12.797576 ip-10-0-136-0 kubenswrapper[2579]: I0703 11:15:12.797534 2579 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/c92e8a04-9147-45f4-8a71-9a9bd95671f5-trusted-ca-bundle\") pod \"console-764db899bc-8xjvj\" (UID: \"c92e8a04-9147-45f4-8a71-9a9bd95671f5\") " pod="openshift-console/console-764db899bc-8xjvj" Jul 03 11:15:12.898670 ip-10-0-136-0 kubenswrapper[2579]: I0703 11:15:12.898626 2579 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"service-ca\" (UniqueName: \"kubernetes.io/configmap/c92e8a04-9147-45f4-8a71-9a9bd95671f5-service-ca\") pod \"console-764db899bc-8xjvj\" (UID: \"c92e8a04-9147-45f4-8a71-9a9bd95671f5\") " pod="openshift-console/console-764db899bc-8xjvj" Jul 03 11:15:12.898991 ip-10-0-136-0 kubenswrapper[2579]: I0703 11:15:12.898967 2579 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"console-oauth-config\" (UniqueName: \"kubernetes.io/secret/c92e8a04-9147-45f4-8a71-9a9bd95671f5-console-oauth-config\") pod \"console-764db899bc-8xjvj\" (UID: \"c92e8a04-9147-45f4-8a71-9a9bd95671f5\") " pod="openshift-console/console-764db899bc-8xjvj" Jul 03 11:15:12.899112 ip-10-0-136-0 kubenswrapper[2579]: I0703 11:15:12.899099 2579 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/c92e8a04-9147-45f4-8a71-9a9bd95671f5-trusted-ca-bundle\") pod \"console-764db899bc-8xjvj\" (UID: \"c92e8a04-9147-45f4-8a71-9a9bd95671f5\") " pod="openshift-console/console-764db899bc-8xjvj" Jul 03 11:15:12.899258 ip-10-0-136-0 kubenswrapper[2579]: I0703 11:15:12.899245 2579 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-nkfk2\" (UniqueName: \"kubernetes.io/projected/c92e8a04-9147-45f4-8a71-9a9bd95671f5-kube-api-access-nkfk2\") pod \"console-764db899bc-8xjvj\" (UID: \"c92e8a04-9147-45f4-8a71-9a9bd95671f5\") " pod="openshift-console/console-764db899bc-8xjvj" Jul 03 11:15:12.899629 ip-10-0-136-0 kubenswrapper[2579]: I0703 11:15:12.899609 2579 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"console-serving-cert\" (UniqueName: \"kubernetes.io/secret/c92e8a04-9147-45f4-8a71-9a9bd95671f5-console-serving-cert\") pod \"console-764db899bc-8xjvj\" (UID: \"c92e8a04-9147-45f4-8a71-9a9bd95671f5\") " pod="openshift-console/console-764db899bc-8xjvj" Jul 03 11:15:12.899767 ip-10-0-136-0 kubenswrapper[2579]: I0703 11:15:12.899751 2579 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"console-config\" (UniqueName: \"kubernetes.io/configmap/c92e8a04-9147-45f4-8a71-9a9bd95671f5-console-config\") pod \"console-764db899bc-8xjvj\" (UID: \"c92e8a04-9147-45f4-8a71-9a9bd95671f5\") " pod="openshift-console/console-764db899bc-8xjvj" Jul 03 11:15:12.899891 ip-10-0-136-0 kubenswrapper[2579]: I0703 11:15:12.899877 2579 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"oauth-serving-cert\" (UniqueName: \"kubernetes.io/configmap/c92e8a04-9147-45f4-8a71-9a9bd95671f5-oauth-serving-cert\") pod \"console-764db899bc-8xjvj\" (UID: \"c92e8a04-9147-45f4-8a71-9a9bd95671f5\") " pod="openshift-console/console-764db899bc-8xjvj" Jul 03 11:15:12.900932 ip-10-0-136-0 kubenswrapper[2579]: I0703 11:15:12.900907 2579 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"oauth-serving-cert\" (UniqueName: \"kubernetes.io/configmap/c92e8a04-9147-45f4-8a71-9a9bd95671f5-oauth-serving-cert\") pod \"console-764db899bc-8xjvj\" (UID: \"c92e8a04-9147-45f4-8a71-9a9bd95671f5\") " pod="openshift-console/console-764db899bc-8xjvj" Jul 03 11:15:12.901048 ip-10-0-136-0 kubenswrapper[2579]: I0703 11:15:12.900930 2579 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"service-ca\" (UniqueName: \"kubernetes.io/configmap/c92e8a04-9147-45f4-8a71-9a9bd95671f5-service-ca\") pod \"console-764db899bc-8xjvj\" (UID: \"c92e8a04-9147-45f4-8a71-9a9bd95671f5\") " pod="openshift-console/console-764db899bc-8xjvj" Jul 03 11:15:12.901048 ip-10-0-136-0 kubenswrapper[2579]: I0703 11:15:12.900953 2579 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"console-config\" (UniqueName: \"kubernetes.io/configmap/c92e8a04-9147-45f4-8a71-9a9bd95671f5-console-config\") pod \"console-764db899bc-8xjvj\" (UID: \"c92e8a04-9147-45f4-8a71-9a9bd95671f5\") " pod="openshift-console/console-764db899bc-8xjvj" Jul 03 11:15:12.901144 ip-10-0-136-0 kubenswrapper[2579]: I0703 11:15:12.901066 2579 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/c92e8a04-9147-45f4-8a71-9a9bd95671f5-trusted-ca-bundle\") pod \"console-764db899bc-8xjvj\" (UID: \"c92e8a04-9147-45f4-8a71-9a9bd95671f5\") " pod="openshift-console/console-764db899bc-8xjvj" Jul 03 11:15:12.902511 ip-10-0-136-0 kubenswrapper[2579]: I0703 11:15:12.902487 2579 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"console-oauth-config\" (UniqueName: \"kubernetes.io/secret/c92e8a04-9147-45f4-8a71-9a9bd95671f5-console-oauth-config\") pod \"console-764db899bc-8xjvj\" (UID: \"c92e8a04-9147-45f4-8a71-9a9bd95671f5\") " pod="openshift-console/console-764db899bc-8xjvj" Jul 03 11:15:12.902595 ip-10-0-136-0 kubenswrapper[2579]: I0703 11:15:12.902530 2579 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"console-serving-cert\" (UniqueName: \"kubernetes.io/secret/c92e8a04-9147-45f4-8a71-9a9bd95671f5-console-serving-cert\") pod \"console-764db899bc-8xjvj\" (UID: \"c92e8a04-9147-45f4-8a71-9a9bd95671f5\") " pod="openshift-console/console-764db899bc-8xjvj" Jul 03 11:15:12.908723 ip-10-0-136-0 kubenswrapper[2579]: I0703 11:15:12.908686 2579 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-nkfk2\" (UniqueName: \"kubernetes.io/projected/c92e8a04-9147-45f4-8a71-9a9bd95671f5-kube-api-access-nkfk2\") pod \"console-764db899bc-8xjvj\" (UID: \"c92e8a04-9147-45f4-8a71-9a9bd95671f5\") " pod="openshift-console/console-764db899bc-8xjvj" Jul 03 11:15:13.033833 ip-10-0-136-0 kubenswrapper[2579]: I0703 11:15:13.033788 2579 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-console/console-764db899bc-8xjvj" Jul 03 11:15:13.186092 ip-10-0-136-0 kubenswrapper[2579]: I0703 11:15:13.186047 2579 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-console/console-764db899bc-8xjvj"] Jul 03 11:15:13.189694 ip-10-0-136-0 kubenswrapper[2579]: W0703 11:15:13.189661 2579 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podc92e8a04_9147_45f4_8a71_9a9bd95671f5.slice/crio-85c5c6f8eb4aa5ea8eb636d0b1d43a3bd7b195f2916f86fb996cd8f0a4f7a055 WatchSource:0}: Error finding container 85c5c6f8eb4aa5ea8eb636d0b1d43a3bd7b195f2916f86fb996cd8f0a4f7a055: Status 404 returned error can't find the container with id 85c5c6f8eb4aa5ea8eb636d0b1d43a3bd7b195f2916f86fb996cd8f0a4f7a055 Jul 03 11:15:13.951618 ip-10-0-136-0 kubenswrapper[2579]: I0703 11:15:13.951515 2579 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-console/console-764db899bc-8xjvj" event={"ID":"c92e8a04-9147-45f4-8a71-9a9bd95671f5","Type":"ContainerStarted","Data":"fc3e1dee119f320f85a150b4c2a0028e4913f8dad5225b264c206580a417faf5"} Jul 03 11:15:13.951618 ip-10-0-136-0 kubenswrapper[2579]: I0703 11:15:13.951581 2579 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-console/console-764db899bc-8xjvj" event={"ID":"c92e8a04-9147-45f4-8a71-9a9bd95671f5","Type":"ContainerStarted","Data":"85c5c6f8eb4aa5ea8eb636d0b1d43a3bd7b195f2916f86fb996cd8f0a4f7a055"} Jul 03 11:15:13.973240 ip-10-0-136-0 kubenswrapper[2579]: I0703 11:15:13.973167 2579 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-console/console-764db899bc-8xjvj" podStartSLOduration=1.973147284 podStartE2EDuration="1.973147284s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-03 11:15:12 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-03 11:15:13.972176441 +0000 UTC m=+170.560480761" watchObservedRunningTime="2026-07-03 11:15:13.973147284 +0000 UTC m=+170.561451602" Jul 03 11:15:23.034670 ip-10-0-136-0 kubenswrapper[2579]: I0703 11:15:23.034594 2579 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="openshift-console/console-764db899bc-8xjvj" Jul 03 11:15:23.035148 ip-10-0-136-0 kubenswrapper[2579]: I0703 11:15:23.034697 2579 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-console/console-764db899bc-8xjvj" Jul 03 11:15:23.040269 ip-10-0-136-0 kubenswrapper[2579]: I0703 11:15:23.040239 2579 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="openshift-console/console-764db899bc-8xjvj" Jul 03 11:15:23.994513 ip-10-0-136-0 kubenswrapper[2579]: I0703 11:15:23.994483 2579 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-console/console-764db899bc-8xjvj" Jul 03 11:16:22.228750 ip-10-0-136-0 kubenswrapper[2579]: I0703 11:16:22.228708 2579 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-console/console-799d957b5b-vrh8b"] Jul 03 11:16:22.232010 ip-10-0-136-0 kubenswrapper[2579]: I0703 11:16:22.231977 2579 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-console/console-799d957b5b-vrh8b" Jul 03 11:16:22.249390 ip-10-0-136-0 kubenswrapper[2579]: I0703 11:16:22.249355 2579 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-console/console-799d957b5b-vrh8b"] Jul 03 11:16:22.368076 ip-10-0-136-0 kubenswrapper[2579]: I0703 11:16:22.368009 2579 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-h9csr\" (UniqueName: \"kubernetes.io/projected/baaa910d-775c-4be2-8d26-04897721653a-kube-api-access-h9csr\") pod \"console-799d957b5b-vrh8b\" (UID: \"baaa910d-775c-4be2-8d26-04897721653a\") " pod="openshift-console/console-799d957b5b-vrh8b" Jul 03 11:16:22.368307 ip-10-0-136-0 kubenswrapper[2579]: I0703 11:16:22.368102 2579 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"console-oauth-config\" (UniqueName: \"kubernetes.io/secret/baaa910d-775c-4be2-8d26-04897721653a-console-oauth-config\") pod \"console-799d957b5b-vrh8b\" (UID: \"baaa910d-775c-4be2-8d26-04897721653a\") " pod="openshift-console/console-799d957b5b-vrh8b" Jul 03 11:16:22.368307 ip-10-0-136-0 kubenswrapper[2579]: I0703 11:16:22.368130 2579 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"oauth-serving-cert\" (UniqueName: \"kubernetes.io/configmap/baaa910d-775c-4be2-8d26-04897721653a-oauth-serving-cert\") pod \"console-799d957b5b-vrh8b\" (UID: \"baaa910d-775c-4be2-8d26-04897721653a\") " pod="openshift-console/console-799d957b5b-vrh8b" Jul 03 11:16:22.368307 ip-10-0-136-0 kubenswrapper[2579]: I0703 11:16:22.368150 2579 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"console-serving-cert\" (UniqueName: \"kubernetes.io/secret/baaa910d-775c-4be2-8d26-04897721653a-console-serving-cert\") pod \"console-799d957b5b-vrh8b\" (UID: \"baaa910d-775c-4be2-8d26-04897721653a\") " pod="openshift-console/console-799d957b5b-vrh8b" Jul 03 11:16:22.368307 ip-10-0-136-0 kubenswrapper[2579]: I0703 11:16:22.368217 2579 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"console-config\" (UniqueName: \"kubernetes.io/configmap/baaa910d-775c-4be2-8d26-04897721653a-console-config\") pod \"console-799d957b5b-vrh8b\" (UID: \"baaa910d-775c-4be2-8d26-04897721653a\") " pod="openshift-console/console-799d957b5b-vrh8b" Jul 03 11:16:22.368307 ip-10-0-136-0 kubenswrapper[2579]: I0703 11:16:22.368250 2579 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"service-ca\" (UniqueName: \"kubernetes.io/configmap/baaa910d-775c-4be2-8d26-04897721653a-service-ca\") pod \"console-799d957b5b-vrh8b\" (UID: \"baaa910d-775c-4be2-8d26-04897721653a\") " pod="openshift-console/console-799d957b5b-vrh8b" Jul 03 11:16:22.368307 ip-10-0-136-0 kubenswrapper[2579]: I0703 11:16:22.368290 2579 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/baaa910d-775c-4be2-8d26-04897721653a-trusted-ca-bundle\") pod \"console-799d957b5b-vrh8b\" (UID: \"baaa910d-775c-4be2-8d26-04897721653a\") " pod="openshift-console/console-799d957b5b-vrh8b" Jul 03 11:16:22.469276 ip-10-0-136-0 kubenswrapper[2579]: I0703 11:16:22.469218 2579 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"console-oauth-config\" (UniqueName: \"kubernetes.io/secret/baaa910d-775c-4be2-8d26-04897721653a-console-oauth-config\") pod \"console-799d957b5b-vrh8b\" (UID: \"baaa910d-775c-4be2-8d26-04897721653a\") " pod="openshift-console/console-799d957b5b-vrh8b" Jul 03 11:16:22.469276 ip-10-0-136-0 kubenswrapper[2579]: I0703 11:16:22.469278 2579 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"oauth-serving-cert\" (UniqueName: \"kubernetes.io/configmap/baaa910d-775c-4be2-8d26-04897721653a-oauth-serving-cert\") pod \"console-799d957b5b-vrh8b\" (UID: \"baaa910d-775c-4be2-8d26-04897721653a\") " pod="openshift-console/console-799d957b5b-vrh8b" Jul 03 11:16:22.469276 ip-10-0-136-0 kubenswrapper[2579]: I0703 11:16:22.469297 2579 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"console-serving-cert\" (UniqueName: \"kubernetes.io/secret/baaa910d-775c-4be2-8d26-04897721653a-console-serving-cert\") pod \"console-799d957b5b-vrh8b\" (UID: \"baaa910d-775c-4be2-8d26-04897721653a\") " pod="openshift-console/console-799d957b5b-vrh8b" Jul 03 11:16:22.469594 ip-10-0-136-0 kubenswrapper[2579]: I0703 11:16:22.469315 2579 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"console-config\" (UniqueName: \"kubernetes.io/configmap/baaa910d-775c-4be2-8d26-04897721653a-console-config\") pod \"console-799d957b5b-vrh8b\" (UID: \"baaa910d-775c-4be2-8d26-04897721653a\") " pod="openshift-console/console-799d957b5b-vrh8b" Jul 03 11:16:22.469594 ip-10-0-136-0 kubenswrapper[2579]: I0703 11:16:22.469332 2579 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"service-ca\" (UniqueName: \"kubernetes.io/configmap/baaa910d-775c-4be2-8d26-04897721653a-service-ca\") pod \"console-799d957b5b-vrh8b\" (UID: \"baaa910d-775c-4be2-8d26-04897721653a\") " pod="openshift-console/console-799d957b5b-vrh8b" Jul 03 11:16:22.469594 ip-10-0-136-0 kubenswrapper[2579]: I0703 11:16:22.469359 2579 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/baaa910d-775c-4be2-8d26-04897721653a-trusted-ca-bundle\") pod \"console-799d957b5b-vrh8b\" (UID: \"baaa910d-775c-4be2-8d26-04897721653a\") " pod="openshift-console/console-799d957b5b-vrh8b" Jul 03 11:16:22.469594 ip-10-0-136-0 kubenswrapper[2579]: I0703 11:16:22.469401 2579 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-h9csr\" (UniqueName: \"kubernetes.io/projected/baaa910d-775c-4be2-8d26-04897721653a-kube-api-access-h9csr\") pod \"console-799d957b5b-vrh8b\" (UID: \"baaa910d-775c-4be2-8d26-04897721653a\") " pod="openshift-console/console-799d957b5b-vrh8b" Jul 03 11:16:22.470300 ip-10-0-136-0 kubenswrapper[2579]: I0703 11:16:22.470269 2579 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"service-ca\" (UniqueName: \"kubernetes.io/configmap/baaa910d-775c-4be2-8d26-04897721653a-service-ca\") pod \"console-799d957b5b-vrh8b\" (UID: \"baaa910d-775c-4be2-8d26-04897721653a\") " pod="openshift-console/console-799d957b5b-vrh8b" Jul 03 11:16:22.470426 ip-10-0-136-0 kubenswrapper[2579]: I0703 11:16:22.470269 2579 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"oauth-serving-cert\" (UniqueName: \"kubernetes.io/configmap/baaa910d-775c-4be2-8d26-04897721653a-oauth-serving-cert\") pod \"console-799d957b5b-vrh8b\" (UID: \"baaa910d-775c-4be2-8d26-04897721653a\") " pod="openshift-console/console-799d957b5b-vrh8b" Jul 03 11:16:22.470426 ip-10-0-136-0 kubenswrapper[2579]: I0703 11:16:22.470315 2579 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/baaa910d-775c-4be2-8d26-04897721653a-trusted-ca-bundle\") pod \"console-799d957b5b-vrh8b\" (UID: \"baaa910d-775c-4be2-8d26-04897721653a\") " pod="openshift-console/console-799d957b5b-vrh8b" Jul 03 11:16:22.470511 ip-10-0-136-0 kubenswrapper[2579]: I0703 11:16:22.470489 2579 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"console-config\" (UniqueName: \"kubernetes.io/configmap/baaa910d-775c-4be2-8d26-04897721653a-console-config\") pod \"console-799d957b5b-vrh8b\" (UID: \"baaa910d-775c-4be2-8d26-04897721653a\") " pod="openshift-console/console-799d957b5b-vrh8b" Jul 03 11:16:22.472134 ip-10-0-136-0 kubenswrapper[2579]: I0703 11:16:22.472096 2579 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"console-serving-cert\" (UniqueName: \"kubernetes.io/secret/baaa910d-775c-4be2-8d26-04897721653a-console-serving-cert\") pod \"console-799d957b5b-vrh8b\" (UID: \"baaa910d-775c-4be2-8d26-04897721653a\") " pod="openshift-console/console-799d957b5b-vrh8b" Jul 03 11:16:22.472134 ip-10-0-136-0 kubenswrapper[2579]: I0703 11:16:22.472109 2579 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"console-oauth-config\" (UniqueName: \"kubernetes.io/secret/baaa910d-775c-4be2-8d26-04897721653a-console-oauth-config\") pod \"console-799d957b5b-vrh8b\" (UID: \"baaa910d-775c-4be2-8d26-04897721653a\") " pod="openshift-console/console-799d957b5b-vrh8b" Jul 03 11:16:22.478467 ip-10-0-136-0 kubenswrapper[2579]: I0703 11:16:22.478442 2579 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-h9csr\" (UniqueName: \"kubernetes.io/projected/baaa910d-775c-4be2-8d26-04897721653a-kube-api-access-h9csr\") pod \"console-799d957b5b-vrh8b\" (UID: \"baaa910d-775c-4be2-8d26-04897721653a\") " pod="openshift-console/console-799d957b5b-vrh8b" Jul 03 11:16:22.542110 ip-10-0-136-0 kubenswrapper[2579]: I0703 11:16:22.542066 2579 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-console/console-799d957b5b-vrh8b" Jul 03 11:16:22.689177 ip-10-0-136-0 kubenswrapper[2579]: I0703 11:16:22.689071 2579 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-console/console-799d957b5b-vrh8b"] Jul 03 11:16:22.691984 ip-10-0-136-0 kubenswrapper[2579]: W0703 11:16:22.691952 2579 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podbaaa910d_775c_4be2_8d26_04897721653a.slice/crio-a3b14309c311367ee1d69a52cba8b947ad3b1f811f6ee8df456b21ddf7c81611 WatchSource:0}: Error finding container a3b14309c311367ee1d69a52cba8b947ad3b1f811f6ee8df456b21ddf7c81611: Status 404 returned error can't find the container with id a3b14309c311367ee1d69a52cba8b947ad3b1f811f6ee8df456b21ddf7c81611 Jul 03 11:16:23.206065 ip-10-0-136-0 kubenswrapper[2579]: I0703 11:16:23.206026 2579 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-console/console-799d957b5b-vrh8b" event={"ID":"baaa910d-775c-4be2-8d26-04897721653a","Type":"ContainerStarted","Data":"90ccd314b814cc45d99ae531fa903877474b29e1da27f94db87f2313933a9fbf"} Jul 03 11:16:23.206318 ip-10-0-136-0 kubenswrapper[2579]: I0703 11:16:23.206080 2579 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-console/console-799d957b5b-vrh8b" event={"ID":"baaa910d-775c-4be2-8d26-04897721653a","Type":"ContainerStarted","Data":"a3b14309c311367ee1d69a52cba8b947ad3b1f811f6ee8df456b21ddf7c81611"} Jul 03 11:16:23.227647 ip-10-0-136-0 kubenswrapper[2579]: I0703 11:16:23.227586 2579 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-console/console-799d957b5b-vrh8b" podStartSLOduration=1.227566591 podStartE2EDuration="1.227566591s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-03 11:16:22 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-03 11:16:23.226125152 +0000 UTC m=+239.814429473" watchObservedRunningTime="2026-07-03 11:16:23.227566591 +0000 UTC m=+239.815870910" Jul 03 11:16:32.542296 ip-10-0-136-0 kubenswrapper[2579]: I0703 11:16:32.542254 2579 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-console/console-799d957b5b-vrh8b" Jul 03 11:16:32.542831 ip-10-0-136-0 kubenswrapper[2579]: I0703 11:16:32.542357 2579 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="openshift-console/console-799d957b5b-vrh8b" Jul 03 11:16:32.547490 ip-10-0-136-0 kubenswrapper[2579]: I0703 11:16:32.547460 2579 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="openshift-console/console-799d957b5b-vrh8b" Jul 03 11:16:33.247580 ip-10-0-136-0 kubenswrapper[2579]: I0703 11:16:33.247539 2579 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-console/console-799d957b5b-vrh8b" Jul 03 11:16:33.310841 ip-10-0-136-0 kubenswrapper[2579]: I0703 11:16:33.310800 2579 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-console/console-764db899bc-8xjvj"] Jul 03 11:16:58.334782 ip-10-0-136-0 kubenswrapper[2579]: I0703 11:16:58.334671 2579 kuberuntime_container.go:872] "Killing container with a grace period" pod="openshift-console/console-764db899bc-8xjvj" podUID="c92e8a04-9147-45f4-8a71-9a9bd95671f5" containerName="console" containerID="cri-o://fc3e1dee119f320f85a150b4c2a0028e4913f8dad5225b264c206580a417faf5" gracePeriod=15 Jul 03 11:16:58.582569 ip-10-0-136-0 kubenswrapper[2579]: I0703 11:16:58.582533 2579 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-console/console-764db899bc-8xjvj" Jul 03 11:16:58.687180 ip-10-0-136-0 kubenswrapper[2579]: I0703 11:16:58.687073 2579 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"console-serving-cert\" (UniqueName: \"kubernetes.io/secret/c92e8a04-9147-45f4-8a71-9a9bd95671f5-console-serving-cert\") pod \"c92e8a04-9147-45f4-8a71-9a9bd95671f5\" (UID: \"c92e8a04-9147-45f4-8a71-9a9bd95671f5\") " Jul 03 11:16:58.687180 ip-10-0-136-0 kubenswrapper[2579]: I0703 11:16:58.687118 2579 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"oauth-serving-cert\" (UniqueName: \"kubernetes.io/configmap/c92e8a04-9147-45f4-8a71-9a9bd95671f5-oauth-serving-cert\") pod \"c92e8a04-9147-45f4-8a71-9a9bd95671f5\" (UID: \"c92e8a04-9147-45f4-8a71-9a9bd95671f5\") " Jul 03 11:16:58.687180 ip-10-0-136-0 kubenswrapper[2579]: I0703 11:16:58.687152 2579 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/c92e8a04-9147-45f4-8a71-9a9bd95671f5-trusted-ca-bundle\") pod \"c92e8a04-9147-45f4-8a71-9a9bd95671f5\" (UID: \"c92e8a04-9147-45f4-8a71-9a9bd95671f5\") " Jul 03 11:16:58.687496 ip-10-0-136-0 kubenswrapper[2579]: I0703 11:16:58.687244 2579 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"console-config\" (UniqueName: \"kubernetes.io/configmap/c92e8a04-9147-45f4-8a71-9a9bd95671f5-console-config\") pod \"c92e8a04-9147-45f4-8a71-9a9bd95671f5\" (UID: \"c92e8a04-9147-45f4-8a71-9a9bd95671f5\") " Jul 03 11:16:58.687496 ip-10-0-136-0 kubenswrapper[2579]: I0703 11:16:58.687271 2579 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"service-ca\" (UniqueName: \"kubernetes.io/configmap/c92e8a04-9147-45f4-8a71-9a9bd95671f5-service-ca\") pod \"c92e8a04-9147-45f4-8a71-9a9bd95671f5\" (UID: \"c92e8a04-9147-45f4-8a71-9a9bd95671f5\") " Jul 03 11:16:58.687496 ip-10-0-136-0 kubenswrapper[2579]: I0703 11:16:58.687295 2579 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"console-oauth-config\" (UniqueName: \"kubernetes.io/secret/c92e8a04-9147-45f4-8a71-9a9bd95671f5-console-oauth-config\") pod \"c92e8a04-9147-45f4-8a71-9a9bd95671f5\" (UID: \"c92e8a04-9147-45f4-8a71-9a9bd95671f5\") " Jul 03 11:16:58.687496 ip-10-0-136-0 kubenswrapper[2579]: I0703 11:16:58.687320 2579 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-nkfk2\" (UniqueName: \"kubernetes.io/projected/c92e8a04-9147-45f4-8a71-9a9bd95671f5-kube-api-access-nkfk2\") pod \"c92e8a04-9147-45f4-8a71-9a9bd95671f5\" (UID: \"c92e8a04-9147-45f4-8a71-9a9bd95671f5\") " Jul 03 11:16:58.687698 ip-10-0-136-0 kubenswrapper[2579]: I0703 11:16:58.687634 2579 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/c92e8a04-9147-45f4-8a71-9a9bd95671f5-oauth-serving-cert" (OuterVolumeSpecName: "oauth-serving-cert") pod "c92e8a04-9147-45f4-8a71-9a9bd95671f5" (UID: "c92e8a04-9147-45f4-8a71-9a9bd95671f5"). InnerVolumeSpecName "oauth-serving-cert". PluginName "kubernetes.io/configmap", VolumeGIDValue "" Jul 03 11:16:58.687698 ip-10-0-136-0 kubenswrapper[2579]: I0703 11:16:58.687655 2579 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/c92e8a04-9147-45f4-8a71-9a9bd95671f5-trusted-ca-bundle" (OuterVolumeSpecName: "trusted-ca-bundle") pod "c92e8a04-9147-45f4-8a71-9a9bd95671f5" (UID: "c92e8a04-9147-45f4-8a71-9a9bd95671f5"). InnerVolumeSpecName "trusted-ca-bundle". PluginName "kubernetes.io/configmap", VolumeGIDValue "" Jul 03 11:16:58.687806 ip-10-0-136-0 kubenswrapper[2579]: I0703 11:16:58.687691 2579 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/c92e8a04-9147-45f4-8a71-9a9bd95671f5-service-ca" (OuterVolumeSpecName: "service-ca") pod "c92e8a04-9147-45f4-8a71-9a9bd95671f5" (UID: "c92e8a04-9147-45f4-8a71-9a9bd95671f5"). InnerVolumeSpecName "service-ca". PluginName "kubernetes.io/configmap", VolumeGIDValue "" Jul 03 11:16:58.687806 ip-10-0-136-0 kubenswrapper[2579]: I0703 11:16:58.687713 2579 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/c92e8a04-9147-45f4-8a71-9a9bd95671f5-console-config" (OuterVolumeSpecName: "console-config") pod "c92e8a04-9147-45f4-8a71-9a9bd95671f5" (UID: "c92e8a04-9147-45f4-8a71-9a9bd95671f5"). InnerVolumeSpecName "console-config". PluginName "kubernetes.io/configmap", VolumeGIDValue "" Jul 03 11:16:58.687985 ip-10-0-136-0 kubenswrapper[2579]: I0703 11:16:58.687911 2579 reconciler_common.go:299] "Volume detached for volume \"console-config\" (UniqueName: \"kubernetes.io/configmap/c92e8a04-9147-45f4-8a71-9a9bd95671f5-console-config\") on node \"ip-10-0-136-0.ec2.internal\" DevicePath \"\"" Jul 03 11:16:58.687985 ip-10-0-136-0 kubenswrapper[2579]: I0703 11:16:58.687938 2579 reconciler_common.go:299] "Volume detached for volume \"service-ca\" (UniqueName: \"kubernetes.io/configmap/c92e8a04-9147-45f4-8a71-9a9bd95671f5-service-ca\") on node \"ip-10-0-136-0.ec2.internal\" DevicePath \"\"" Jul 03 11:16:58.687985 ip-10-0-136-0 kubenswrapper[2579]: I0703 11:16:58.687953 2579 reconciler_common.go:299] "Volume detached for volume \"oauth-serving-cert\" (UniqueName: \"kubernetes.io/configmap/c92e8a04-9147-45f4-8a71-9a9bd95671f5-oauth-serving-cert\") on node \"ip-10-0-136-0.ec2.internal\" DevicePath \"\"" Jul 03 11:16:58.687985 ip-10-0-136-0 kubenswrapper[2579]: I0703 11:16:58.687966 2579 reconciler_common.go:299] "Volume detached for volume \"trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/c92e8a04-9147-45f4-8a71-9a9bd95671f5-trusted-ca-bundle\") on node \"ip-10-0-136-0.ec2.internal\" DevicePath \"\"" Jul 03 11:16:58.689716 ip-10-0-136-0 kubenswrapper[2579]: I0703 11:16:58.689688 2579 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/c92e8a04-9147-45f4-8a71-9a9bd95671f5-console-serving-cert" (OuterVolumeSpecName: "console-serving-cert") pod "c92e8a04-9147-45f4-8a71-9a9bd95671f5" (UID: "c92e8a04-9147-45f4-8a71-9a9bd95671f5"). InnerVolumeSpecName "console-serving-cert". PluginName "kubernetes.io/secret", VolumeGIDValue "" Jul 03 11:16:58.690147 ip-10-0-136-0 kubenswrapper[2579]: I0703 11:16:58.690118 2579 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/c92e8a04-9147-45f4-8a71-9a9bd95671f5-kube-api-access-nkfk2" (OuterVolumeSpecName: "kube-api-access-nkfk2") pod "c92e8a04-9147-45f4-8a71-9a9bd95671f5" (UID: "c92e8a04-9147-45f4-8a71-9a9bd95671f5"). InnerVolumeSpecName "kube-api-access-nkfk2". PluginName "kubernetes.io/projected", VolumeGIDValue "" Jul 03 11:16:58.690147 ip-10-0-136-0 kubenswrapper[2579]: I0703 11:16:58.690123 2579 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/c92e8a04-9147-45f4-8a71-9a9bd95671f5-console-oauth-config" (OuterVolumeSpecName: "console-oauth-config") pod "c92e8a04-9147-45f4-8a71-9a9bd95671f5" (UID: "c92e8a04-9147-45f4-8a71-9a9bd95671f5"). InnerVolumeSpecName "console-oauth-config". PluginName "kubernetes.io/secret", VolumeGIDValue "" Jul 03 11:16:58.788836 ip-10-0-136-0 kubenswrapper[2579]: I0703 11:16:58.788786 2579 reconciler_common.go:299] "Volume detached for volume \"console-oauth-config\" (UniqueName: \"kubernetes.io/secret/c92e8a04-9147-45f4-8a71-9a9bd95671f5-console-oauth-config\") on node \"ip-10-0-136-0.ec2.internal\" DevicePath \"\"" Jul 03 11:16:58.788836 ip-10-0-136-0 kubenswrapper[2579]: I0703 11:16:58.788825 2579 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-nkfk2\" (UniqueName: \"kubernetes.io/projected/c92e8a04-9147-45f4-8a71-9a9bd95671f5-kube-api-access-nkfk2\") on node \"ip-10-0-136-0.ec2.internal\" DevicePath \"\"" Jul 03 11:16:58.788836 ip-10-0-136-0 kubenswrapper[2579]: I0703 11:16:58.788839 2579 reconciler_common.go:299] "Volume detached for volume \"console-serving-cert\" (UniqueName: \"kubernetes.io/secret/c92e8a04-9147-45f4-8a71-9a9bd95671f5-console-serving-cert\") on node \"ip-10-0-136-0.ec2.internal\" DevicePath \"\"" Jul 03 11:16:59.347669 ip-10-0-136-0 kubenswrapper[2579]: I0703 11:16:59.347633 2579 generic.go:356] "Generic (PLEG): container finished" podID="c92e8a04-9147-45f4-8a71-9a9bd95671f5" containerID="fc3e1dee119f320f85a150b4c2a0028e4913f8dad5225b264c206580a417faf5" exitCode=0 Jul 03 11:16:59.348155 ip-10-0-136-0 kubenswrapper[2579]: I0703 11:16:59.347682 2579 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-console/console-764db899bc-8xjvj" Jul 03 11:16:59.348155 ip-10-0-136-0 kubenswrapper[2579]: I0703 11:16:59.347730 2579 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-console/console-764db899bc-8xjvj" event={"ID":"c92e8a04-9147-45f4-8a71-9a9bd95671f5","Type":"ContainerDied","Data":"fc3e1dee119f320f85a150b4c2a0028e4913f8dad5225b264c206580a417faf5"} Jul 03 11:16:59.348155 ip-10-0-136-0 kubenswrapper[2579]: I0703 11:16:59.347795 2579 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-console/console-764db899bc-8xjvj" event={"ID":"c92e8a04-9147-45f4-8a71-9a9bd95671f5","Type":"ContainerDied","Data":"85c5c6f8eb4aa5ea8eb636d0b1d43a3bd7b195f2916f86fb996cd8f0a4f7a055"} Jul 03 11:16:59.348155 ip-10-0-136-0 kubenswrapper[2579]: I0703 11:16:59.347829 2579 scope.go:117] "RemoveContainer" containerID="fc3e1dee119f320f85a150b4c2a0028e4913f8dad5225b264c206580a417faf5" Jul 03 11:16:59.357177 ip-10-0-136-0 kubenswrapper[2579]: I0703 11:16:59.357157 2579 scope.go:117] "RemoveContainer" containerID="fc3e1dee119f320f85a150b4c2a0028e4913f8dad5225b264c206580a417faf5" Jul 03 11:16:59.357544 ip-10-0-136-0 kubenswrapper[2579]: E0703 11:16:59.357525 2579 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"fc3e1dee119f320f85a150b4c2a0028e4913f8dad5225b264c206580a417faf5\": container with ID starting with fc3e1dee119f320f85a150b4c2a0028e4913f8dad5225b264c206580a417faf5 not found: ID does not exist" containerID="fc3e1dee119f320f85a150b4c2a0028e4913f8dad5225b264c206580a417faf5" Jul 03 11:16:59.357589 ip-10-0-136-0 kubenswrapper[2579]: I0703 11:16:59.357553 2579 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"fc3e1dee119f320f85a150b4c2a0028e4913f8dad5225b264c206580a417faf5"} err="failed to get container status \"fc3e1dee119f320f85a150b4c2a0028e4913f8dad5225b264c206580a417faf5\": rpc error: code = NotFound desc = could not find container \"fc3e1dee119f320f85a150b4c2a0028e4913f8dad5225b264c206580a417faf5\": container with ID starting with fc3e1dee119f320f85a150b4c2a0028e4913f8dad5225b264c206580a417faf5 not found: ID does not exist" Jul 03 11:16:59.370765 ip-10-0-136-0 kubenswrapper[2579]: I0703 11:16:59.370730 2579 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-console/console-764db899bc-8xjvj"] Jul 03 11:16:59.376950 ip-10-0-136-0 kubenswrapper[2579]: I0703 11:16:59.376920 2579 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["openshift-console/console-764db899bc-8xjvj"] Jul 03 11:16:59.990391 ip-10-0-136-0 kubenswrapper[2579]: I0703 11:16:59.990350 2579 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="c92e8a04-9147-45f4-8a71-9a9bd95671f5" path="/var/lib/kubelet/pods/c92e8a04-9147-45f4-8a71-9a9bd95671f5/volumes" Jul 03 11:17:23.824248 ip-10-0-136-0 kubenswrapper[2579]: I0703 11:17:23.824217 2579 kubelet.go:1648] "Image garbage collection succeeded" Jul 03 11:17:26.233138 ip-10-0-136-0 kubenswrapper[2579]: I0703 11:17:26.233095 2579 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-marketplace/685f1ed643dcce69727e2826bbf9be585096b4cf774051b5daa5e67540552hs"] Jul 03 11:17:26.235449 ip-10-0-136-0 kubenswrapper[2579]: I0703 11:17:26.233513 2579 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="c92e8a04-9147-45f4-8a71-9a9bd95671f5" containerName="console" Jul 03 11:17:26.235449 ip-10-0-136-0 kubenswrapper[2579]: I0703 11:17:26.233532 2579 state_mem.go:107] "Deleted CPUSet assignment" podUID="c92e8a04-9147-45f4-8a71-9a9bd95671f5" containerName="console" Jul 03 11:17:26.235449 ip-10-0-136-0 kubenswrapper[2579]: I0703 11:17:26.233634 2579 memory_manager.go:368] "RemoveStaleState removing state" podUID="c92e8a04-9147-45f4-8a71-9a9bd95671f5" containerName="console" Jul 03 11:17:26.236514 ip-10-0-136-0 kubenswrapper[2579]: I0703 11:17:26.236487 2579 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-marketplace/685f1ed643dcce69727e2826bbf9be585096b4cf774051b5daa5e67540552hs" Jul 03 11:17:26.239471 ip-10-0-136-0 kubenswrapper[2579]: I0703 11:17:26.239440 2579 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-marketplace\"/\"default-dockercfg-6mvjw\"" Jul 03 11:17:26.239831 ip-10-0-136-0 kubenswrapper[2579]: I0703 11:17:26.239814 2579 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-marketplace\"/\"kube-root-ca.crt\"" Jul 03 11:17:26.240292 ip-10-0-136-0 kubenswrapper[2579]: I0703 11:17:26.240278 2579 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-marketplace\"/\"openshift-service-ca.crt\"" Jul 03 11:17:26.247740 ip-10-0-136-0 kubenswrapper[2579]: I0703 11:17:26.247701 2579 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-marketplace/685f1ed643dcce69727e2826bbf9be585096b4cf774051b5daa5e67540552hs"] Jul 03 11:17:26.316363 ip-10-0-136-0 kubenswrapper[2579]: I0703 11:17:26.316299 2579 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-72xhz\" (UniqueName: \"kubernetes.io/projected/b5774fc9-f799-4ca6-875a-7edd14a534a7-kube-api-access-72xhz\") pod \"685f1ed643dcce69727e2826bbf9be585096b4cf774051b5daa5e67540552hs\" (UID: \"b5774fc9-f799-4ca6-875a-7edd14a534a7\") " pod="openshift-marketplace/685f1ed643dcce69727e2826bbf9be585096b4cf774051b5daa5e67540552hs" Jul 03 11:17:26.316363 ip-10-0-136-0 kubenswrapper[2579]: I0703 11:17:26.316364 2579 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"util\" (UniqueName: \"kubernetes.io/empty-dir/b5774fc9-f799-4ca6-875a-7edd14a534a7-util\") pod \"685f1ed643dcce69727e2826bbf9be585096b4cf774051b5daa5e67540552hs\" (UID: \"b5774fc9-f799-4ca6-875a-7edd14a534a7\") " pod="openshift-marketplace/685f1ed643dcce69727e2826bbf9be585096b4cf774051b5daa5e67540552hs" Jul 03 11:17:26.316640 ip-10-0-136-0 kubenswrapper[2579]: I0703 11:17:26.316444 2579 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"bundle\" (UniqueName: \"kubernetes.io/empty-dir/b5774fc9-f799-4ca6-875a-7edd14a534a7-bundle\") pod \"685f1ed643dcce69727e2826bbf9be585096b4cf774051b5daa5e67540552hs\" (UID: \"b5774fc9-f799-4ca6-875a-7edd14a534a7\") " pod="openshift-marketplace/685f1ed643dcce69727e2826bbf9be585096b4cf774051b5daa5e67540552hs" Jul 03 11:17:26.417727 ip-10-0-136-0 kubenswrapper[2579]: I0703 11:17:26.417687 2579 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"bundle\" (UniqueName: \"kubernetes.io/empty-dir/b5774fc9-f799-4ca6-875a-7edd14a534a7-bundle\") pod \"685f1ed643dcce69727e2826bbf9be585096b4cf774051b5daa5e67540552hs\" (UID: \"b5774fc9-f799-4ca6-875a-7edd14a534a7\") " pod="openshift-marketplace/685f1ed643dcce69727e2826bbf9be585096b4cf774051b5daa5e67540552hs" Jul 03 11:17:26.417908 ip-10-0-136-0 kubenswrapper[2579]: I0703 11:17:26.417746 2579 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-72xhz\" (UniqueName: \"kubernetes.io/projected/b5774fc9-f799-4ca6-875a-7edd14a534a7-kube-api-access-72xhz\") pod \"685f1ed643dcce69727e2826bbf9be585096b4cf774051b5daa5e67540552hs\" (UID: \"b5774fc9-f799-4ca6-875a-7edd14a534a7\") " pod="openshift-marketplace/685f1ed643dcce69727e2826bbf9be585096b4cf774051b5daa5e67540552hs" Jul 03 11:17:26.417908 ip-10-0-136-0 kubenswrapper[2579]: I0703 11:17:26.417798 2579 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"util\" (UniqueName: \"kubernetes.io/empty-dir/b5774fc9-f799-4ca6-875a-7edd14a534a7-util\") pod \"685f1ed643dcce69727e2826bbf9be585096b4cf774051b5daa5e67540552hs\" (UID: \"b5774fc9-f799-4ca6-875a-7edd14a534a7\") " pod="openshift-marketplace/685f1ed643dcce69727e2826bbf9be585096b4cf774051b5daa5e67540552hs" Jul 03 11:17:26.418108 ip-10-0-136-0 kubenswrapper[2579]: I0703 11:17:26.418093 2579 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"util\" (UniqueName: \"kubernetes.io/empty-dir/b5774fc9-f799-4ca6-875a-7edd14a534a7-util\") pod \"685f1ed643dcce69727e2826bbf9be585096b4cf774051b5daa5e67540552hs\" (UID: \"b5774fc9-f799-4ca6-875a-7edd14a534a7\") " pod="openshift-marketplace/685f1ed643dcce69727e2826bbf9be585096b4cf774051b5daa5e67540552hs" Jul 03 11:17:26.418219 ip-10-0-136-0 kubenswrapper[2579]: I0703 11:17:26.418168 2579 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"bundle\" (UniqueName: \"kubernetes.io/empty-dir/b5774fc9-f799-4ca6-875a-7edd14a534a7-bundle\") pod \"685f1ed643dcce69727e2826bbf9be585096b4cf774051b5daa5e67540552hs\" (UID: \"b5774fc9-f799-4ca6-875a-7edd14a534a7\") " pod="openshift-marketplace/685f1ed643dcce69727e2826bbf9be585096b4cf774051b5daa5e67540552hs" Jul 03 11:17:26.426989 ip-10-0-136-0 kubenswrapper[2579]: I0703 11:17:26.426960 2579 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-72xhz\" (UniqueName: \"kubernetes.io/projected/b5774fc9-f799-4ca6-875a-7edd14a534a7-kube-api-access-72xhz\") pod \"685f1ed643dcce69727e2826bbf9be585096b4cf774051b5daa5e67540552hs\" (UID: \"b5774fc9-f799-4ca6-875a-7edd14a534a7\") " pod="openshift-marketplace/685f1ed643dcce69727e2826bbf9be585096b4cf774051b5daa5e67540552hs" Jul 03 11:17:26.546335 ip-10-0-136-0 kubenswrapper[2579]: I0703 11:17:26.546300 2579 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-marketplace/685f1ed643dcce69727e2826bbf9be585096b4cf774051b5daa5e67540552hs" Jul 03 11:17:26.713393 ip-10-0-136-0 kubenswrapper[2579]: I0703 11:17:26.713352 2579 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-marketplace/685f1ed643dcce69727e2826bbf9be585096b4cf774051b5daa5e67540552hs"] Jul 03 11:17:26.716823 ip-10-0-136-0 kubenswrapper[2579]: W0703 11:17:26.716787 2579 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podb5774fc9_f799_4ca6_875a_7edd14a534a7.slice/crio-8c01e70cc91c430e5acf7aaad00f2169c2783489889221b7dc8f61a01854282d WatchSource:0}: Error finding container 8c01e70cc91c430e5acf7aaad00f2169c2783489889221b7dc8f61a01854282d: Status 404 returned error can't find the container with id 8c01e70cc91c430e5acf7aaad00f2169c2783489889221b7dc8f61a01854282d Jul 03 11:17:26.718860 ip-10-0-136-0 kubenswrapper[2579]: I0703 11:17:26.718836 2579 provider.go:93] Refreshing cache for provider: *credentialprovider.defaultDockerConfigProvider Jul 03 11:17:27.454203 ip-10-0-136-0 kubenswrapper[2579]: I0703 11:17:27.454141 2579 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-marketplace/685f1ed643dcce69727e2826bbf9be585096b4cf774051b5daa5e67540552hs" event={"ID":"b5774fc9-f799-4ca6-875a-7edd14a534a7","Type":"ContainerStarted","Data":"8c01e70cc91c430e5acf7aaad00f2169c2783489889221b7dc8f61a01854282d"} Jul 03 11:17:33.482115 ip-10-0-136-0 kubenswrapper[2579]: I0703 11:17:33.481991 2579 generic.go:356] "Generic (PLEG): container finished" podID="b5774fc9-f799-4ca6-875a-7edd14a534a7" containerID="bb25d01e61a5ecb57eea43227c98fdff3dc4ea573adf61da3fb161338a8d3cbd" exitCode=0 Jul 03 11:17:33.482115 ip-10-0-136-0 kubenswrapper[2579]: I0703 11:17:33.482089 2579 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-marketplace/685f1ed643dcce69727e2826bbf9be585096b4cf774051b5daa5e67540552hs" event={"ID":"b5774fc9-f799-4ca6-875a-7edd14a534a7","Type":"ContainerDied","Data":"bb25d01e61a5ecb57eea43227c98fdff3dc4ea573adf61da3fb161338a8d3cbd"} Jul 03 11:17:35.496055 ip-10-0-136-0 kubenswrapper[2579]: I0703 11:17:35.495996 2579 generic.go:356] "Generic (PLEG): container finished" podID="b5774fc9-f799-4ca6-875a-7edd14a534a7" containerID="652c6d4ea0dba08f5f0e11d1ac8cf464575a0d3dfef34e38630d50a99064d1b7" exitCode=0 Jul 03 11:17:35.496823 ip-10-0-136-0 kubenswrapper[2579]: I0703 11:17:35.496111 2579 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-marketplace/685f1ed643dcce69727e2826bbf9be585096b4cf774051b5daa5e67540552hs" event={"ID":"b5774fc9-f799-4ca6-875a-7edd14a534a7","Type":"ContainerDied","Data":"652c6d4ea0dba08f5f0e11d1ac8cf464575a0d3dfef34e38630d50a99064d1b7"} Jul 03 11:17:42.530729 ip-10-0-136-0 kubenswrapper[2579]: I0703 11:17:42.530678 2579 generic.go:356] "Generic (PLEG): container finished" podID="b5774fc9-f799-4ca6-875a-7edd14a534a7" containerID="d27d80b9e411bf89f94838e1c7a0b734df250ae2a0c262589efe0c02be4f3c92" exitCode=0 Jul 03 11:17:42.531170 ip-10-0-136-0 kubenswrapper[2579]: I0703 11:17:42.530763 2579 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-marketplace/685f1ed643dcce69727e2826bbf9be585096b4cf774051b5daa5e67540552hs" event={"ID":"b5774fc9-f799-4ca6-875a-7edd14a534a7","Type":"ContainerDied","Data":"d27d80b9e411bf89f94838e1c7a0b734df250ae2a0c262589efe0c02be4f3c92"} Jul 03 11:17:43.667977 ip-10-0-136-0 kubenswrapper[2579]: I0703 11:17:43.667942 2579 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-marketplace/685f1ed643dcce69727e2826bbf9be585096b4cf774051b5daa5e67540552hs" Jul 03 11:17:43.682352 ip-10-0-136-0 kubenswrapper[2579]: I0703 11:17:43.682322 2579 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-72xhz\" (UniqueName: \"kubernetes.io/projected/b5774fc9-f799-4ca6-875a-7edd14a534a7-kube-api-access-72xhz\") pod \"b5774fc9-f799-4ca6-875a-7edd14a534a7\" (UID: \"b5774fc9-f799-4ca6-875a-7edd14a534a7\") " Jul 03 11:17:43.682462 ip-10-0-136-0 kubenswrapper[2579]: I0703 11:17:43.682360 2579 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"util\" (UniqueName: \"kubernetes.io/empty-dir/b5774fc9-f799-4ca6-875a-7edd14a534a7-util\") pod \"b5774fc9-f799-4ca6-875a-7edd14a534a7\" (UID: \"b5774fc9-f799-4ca6-875a-7edd14a534a7\") " Jul 03 11:17:43.682462 ip-10-0-136-0 kubenswrapper[2579]: I0703 11:17:43.682386 2579 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"bundle\" (UniqueName: \"kubernetes.io/empty-dir/b5774fc9-f799-4ca6-875a-7edd14a534a7-bundle\") pod \"b5774fc9-f799-4ca6-875a-7edd14a534a7\" (UID: \"b5774fc9-f799-4ca6-875a-7edd14a534a7\") " Jul 03 11:17:43.683113 ip-10-0-136-0 kubenswrapper[2579]: I0703 11:17:43.683040 2579 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/b5774fc9-f799-4ca6-875a-7edd14a534a7-bundle" (OuterVolumeSpecName: "bundle") pod "b5774fc9-f799-4ca6-875a-7edd14a534a7" (UID: "b5774fc9-f799-4ca6-875a-7edd14a534a7"). InnerVolumeSpecName "bundle". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Jul 03 11:17:43.684767 ip-10-0-136-0 kubenswrapper[2579]: I0703 11:17:43.684736 2579 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/b5774fc9-f799-4ca6-875a-7edd14a534a7-kube-api-access-72xhz" (OuterVolumeSpecName: "kube-api-access-72xhz") pod "b5774fc9-f799-4ca6-875a-7edd14a534a7" (UID: "b5774fc9-f799-4ca6-875a-7edd14a534a7"). InnerVolumeSpecName "kube-api-access-72xhz". PluginName "kubernetes.io/projected", VolumeGIDValue "" Jul 03 11:17:43.686464 ip-10-0-136-0 kubenswrapper[2579]: I0703 11:17:43.686441 2579 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/b5774fc9-f799-4ca6-875a-7edd14a534a7-util" (OuterVolumeSpecName: "util") pod "b5774fc9-f799-4ca6-875a-7edd14a534a7" (UID: "b5774fc9-f799-4ca6-875a-7edd14a534a7"). InnerVolumeSpecName "util". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Jul 03 11:17:43.783398 ip-10-0-136-0 kubenswrapper[2579]: I0703 11:17:43.783352 2579 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-72xhz\" (UniqueName: \"kubernetes.io/projected/b5774fc9-f799-4ca6-875a-7edd14a534a7-kube-api-access-72xhz\") on node \"ip-10-0-136-0.ec2.internal\" DevicePath \"\"" Jul 03 11:17:43.783398 ip-10-0-136-0 kubenswrapper[2579]: I0703 11:17:43.783387 2579 reconciler_common.go:299] "Volume detached for volume \"util\" (UniqueName: \"kubernetes.io/empty-dir/b5774fc9-f799-4ca6-875a-7edd14a534a7-util\") on node \"ip-10-0-136-0.ec2.internal\" DevicePath \"\"" Jul 03 11:17:43.783398 ip-10-0-136-0 kubenswrapper[2579]: I0703 11:17:43.783396 2579 reconciler_common.go:299] "Volume detached for volume \"bundle\" (UniqueName: \"kubernetes.io/empty-dir/b5774fc9-f799-4ca6-875a-7edd14a534a7-bundle\") on node \"ip-10-0-136-0.ec2.internal\" DevicePath \"\"" Jul 03 11:17:44.540461 ip-10-0-136-0 kubenswrapper[2579]: I0703 11:17:44.540426 2579 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-marketplace/685f1ed643dcce69727e2826bbf9be585096b4cf774051b5daa5e67540552hs" event={"ID":"b5774fc9-f799-4ca6-875a-7edd14a534a7","Type":"ContainerDied","Data":"8c01e70cc91c430e5acf7aaad00f2169c2783489889221b7dc8f61a01854282d"} Jul 03 11:17:44.540651 ip-10-0-136-0 kubenswrapper[2579]: I0703 11:17:44.540474 2579 pod_container_deletor.go:80] "Container not found in pod's containers" containerID="8c01e70cc91c430e5acf7aaad00f2169c2783489889221b7dc8f61a01854282d" Jul 03 11:17:44.540651 ip-10-0-136-0 kubenswrapper[2579]: I0703 11:17:44.540437 2579 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-marketplace/685f1ed643dcce69727e2826bbf9be585096b4cf774051b5daa5e67540552hs" Jul 03 11:17:50.917444 ip-10-0-136-0 kubenswrapper[2579]: I0703 11:17:50.917404 2579 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["cert-manager-operator/cert-manager-operator-controller-manager-55c444bf86-8hlnt"] Jul 03 11:17:50.917967 ip-10-0-136-0 kubenswrapper[2579]: I0703 11:17:50.917708 2579 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="b5774fc9-f799-4ca6-875a-7edd14a534a7" containerName="pull" Jul 03 11:17:50.917967 ip-10-0-136-0 kubenswrapper[2579]: I0703 11:17:50.917720 2579 state_mem.go:107] "Deleted CPUSet assignment" podUID="b5774fc9-f799-4ca6-875a-7edd14a534a7" containerName="pull" Jul 03 11:17:50.917967 ip-10-0-136-0 kubenswrapper[2579]: I0703 11:17:50.917729 2579 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="b5774fc9-f799-4ca6-875a-7edd14a534a7" containerName="util" Jul 03 11:17:50.917967 ip-10-0-136-0 kubenswrapper[2579]: I0703 11:17:50.917736 2579 state_mem.go:107] "Deleted CPUSet assignment" podUID="b5774fc9-f799-4ca6-875a-7edd14a534a7" containerName="util" Jul 03 11:17:50.917967 ip-10-0-136-0 kubenswrapper[2579]: I0703 11:17:50.917746 2579 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="b5774fc9-f799-4ca6-875a-7edd14a534a7" containerName="extract" Jul 03 11:17:50.917967 ip-10-0-136-0 kubenswrapper[2579]: I0703 11:17:50.917751 2579 state_mem.go:107] "Deleted CPUSet assignment" podUID="b5774fc9-f799-4ca6-875a-7edd14a534a7" containerName="extract" Jul 03 11:17:50.917967 ip-10-0-136-0 kubenswrapper[2579]: I0703 11:17:50.917796 2579 memory_manager.go:368] "RemoveStaleState removing state" podUID="b5774fc9-f799-4ca6-875a-7edd14a534a7" containerName="extract" Jul 03 11:17:50.921675 ip-10-0-136-0 kubenswrapper[2579]: I0703 11:17:50.921646 2579 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="cert-manager-operator/cert-manager-operator-controller-manager-55c444bf86-8hlnt" Jul 03 11:17:50.925852 ip-10-0-136-0 kubenswrapper[2579]: I0703 11:17:50.925819 2579 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"cert-manager-operator\"/\"kube-root-ca.crt\"" Jul 03 11:17:50.925852 ip-10-0-136-0 kubenswrapper[2579]: I0703 11:17:50.925845 2579 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"cert-manager-operator\"/\"cert-manager-operator-controller-manager-dockercfg-dx2s9\"" Jul 03 11:17:50.926317 ip-10-0-136-0 kubenswrapper[2579]: I0703 11:17:50.926300 2579 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"cert-manager-operator\"/\"openshift-service-ca.crt\"" Jul 03 11:17:50.942068 ip-10-0-136-0 kubenswrapper[2579]: I0703 11:17:50.942030 2579 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-nwp4c\" (UniqueName: \"kubernetes.io/projected/d07645c3-58c8-42fc-9444-874ab92e9ff3-kube-api-access-nwp4c\") pod \"cert-manager-operator-controller-manager-55c444bf86-8hlnt\" (UID: \"d07645c3-58c8-42fc-9444-874ab92e9ff3\") " pod="cert-manager-operator/cert-manager-operator-controller-manager-55c444bf86-8hlnt" Jul 03 11:17:50.942284 ip-10-0-136-0 kubenswrapper[2579]: I0703 11:17:50.942093 2579 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tmp\" (UniqueName: \"kubernetes.io/empty-dir/d07645c3-58c8-42fc-9444-874ab92e9ff3-tmp\") pod \"cert-manager-operator-controller-manager-55c444bf86-8hlnt\" (UID: \"d07645c3-58c8-42fc-9444-874ab92e9ff3\") " pod="cert-manager-operator/cert-manager-operator-controller-manager-55c444bf86-8hlnt" Jul 03 11:17:50.945834 ip-10-0-136-0 kubenswrapper[2579]: I0703 11:17:50.945800 2579 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["cert-manager-operator/cert-manager-operator-controller-manager-55c444bf86-8hlnt"] Jul 03 11:17:51.042770 ip-10-0-136-0 kubenswrapper[2579]: I0703 11:17:51.042711 2579 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-nwp4c\" (UniqueName: \"kubernetes.io/projected/d07645c3-58c8-42fc-9444-874ab92e9ff3-kube-api-access-nwp4c\") pod \"cert-manager-operator-controller-manager-55c444bf86-8hlnt\" (UID: \"d07645c3-58c8-42fc-9444-874ab92e9ff3\") " pod="cert-manager-operator/cert-manager-operator-controller-manager-55c444bf86-8hlnt" Jul 03 11:17:51.042953 ip-10-0-136-0 kubenswrapper[2579]: I0703 11:17:51.042809 2579 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tmp\" (UniqueName: \"kubernetes.io/empty-dir/d07645c3-58c8-42fc-9444-874ab92e9ff3-tmp\") pod \"cert-manager-operator-controller-manager-55c444bf86-8hlnt\" (UID: \"d07645c3-58c8-42fc-9444-874ab92e9ff3\") " pod="cert-manager-operator/cert-manager-operator-controller-manager-55c444bf86-8hlnt" Jul 03 11:17:51.043257 ip-10-0-136-0 kubenswrapper[2579]: I0703 11:17:51.043236 2579 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"tmp\" (UniqueName: \"kubernetes.io/empty-dir/d07645c3-58c8-42fc-9444-874ab92e9ff3-tmp\") pod \"cert-manager-operator-controller-manager-55c444bf86-8hlnt\" (UID: \"d07645c3-58c8-42fc-9444-874ab92e9ff3\") " pod="cert-manager-operator/cert-manager-operator-controller-manager-55c444bf86-8hlnt" Jul 03 11:17:51.055509 ip-10-0-136-0 kubenswrapper[2579]: I0703 11:17:51.055472 2579 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-nwp4c\" (UniqueName: \"kubernetes.io/projected/d07645c3-58c8-42fc-9444-874ab92e9ff3-kube-api-access-nwp4c\") pod \"cert-manager-operator-controller-manager-55c444bf86-8hlnt\" (UID: \"d07645c3-58c8-42fc-9444-874ab92e9ff3\") " pod="cert-manager-operator/cert-manager-operator-controller-manager-55c444bf86-8hlnt" Jul 03 11:17:51.231745 ip-10-0-136-0 kubenswrapper[2579]: I0703 11:17:51.231646 2579 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="cert-manager-operator/cert-manager-operator-controller-manager-55c444bf86-8hlnt" Jul 03 11:17:51.411795 ip-10-0-136-0 kubenswrapper[2579]: I0703 11:17:51.411749 2579 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["cert-manager-operator/cert-manager-operator-controller-manager-55c444bf86-8hlnt"] Jul 03 11:17:51.417024 ip-10-0-136-0 kubenswrapper[2579]: W0703 11:17:51.416992 2579 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podd07645c3_58c8_42fc_9444_874ab92e9ff3.slice/crio-200b4f716940c8d9e2e9e36a7963a11a7c78dd435b35b3a8791499ad4ff6704f WatchSource:0}: Error finding container 200b4f716940c8d9e2e9e36a7963a11a7c78dd435b35b3a8791499ad4ff6704f: Status 404 returned error can't find the container with id 200b4f716940c8d9e2e9e36a7963a11a7c78dd435b35b3a8791499ad4ff6704f Jul 03 11:17:51.570243 ip-10-0-136-0 kubenswrapper[2579]: I0703 11:17:51.570202 2579 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="cert-manager-operator/cert-manager-operator-controller-manager-55c444bf86-8hlnt" event={"ID":"d07645c3-58c8-42fc-9444-874ab92e9ff3","Type":"ContainerStarted","Data":"200b4f716940c8d9e2e9e36a7963a11a7c78dd435b35b3a8791499ad4ff6704f"} Jul 03 11:18:06.640758 ip-10-0-136-0 kubenswrapper[2579]: I0703 11:18:06.640719 2579 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="cert-manager-operator/cert-manager-operator-controller-manager-55c444bf86-8hlnt" event={"ID":"d07645c3-58c8-42fc-9444-874ab92e9ff3","Type":"ContainerStarted","Data":"9af1c3d6d9478150971d7b7f0aa0f96bec6a708b08e538dac439561fba87d3e8"} Jul 03 11:18:07.912512 ip-10-0-136-0 kubenswrapper[2579]: I0703 11:18:07.912413 2579 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="cert-manager-operator/cert-manager-operator-controller-manager-55c444bf86-8hlnt" podStartSLOduration=3.499814932 podStartE2EDuration="17.912383477s" totalImagesPullingTime="14.412568545s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-03 11:17:50 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-03 11:18:06.681589247 +0000 UTC m=+343.269893579" watchObservedRunningTime="2026-07-03 11:18:07.912383477 +0000 UTC m=+344.500687796" Jul 03 11:18:07.913651 ip-10-0-136-0 kubenswrapper[2579]: I0703 11:18:07.913616 2579 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["cert-manager/cert-manager-webhook-6486ffd997-xw44p"] Jul 03 11:18:07.916589 ip-10-0-136-0 kubenswrapper[2579]: I0703 11:18:07.916546 2579 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="cert-manager/cert-manager-webhook-6486ffd997-xw44p" Jul 03 11:18:07.930209 ip-10-0-136-0 kubenswrapper[2579]: I0703 11:18:07.930141 2579 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"cert-manager\"/\"kube-root-ca.crt\"" Jul 03 11:18:07.931064 ip-10-0-136-0 kubenswrapper[2579]: I0703 11:18:07.931036 2579 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"cert-manager\"/\"cert-manager-webhook-dockercfg-rrshn\"" Jul 03 11:18:07.932847 ip-10-0-136-0 kubenswrapper[2579]: I0703 11:18:07.932808 2579 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"cert-manager\"/\"openshift-service-ca.crt\"" Jul 03 11:18:07.938316 ip-10-0-136-0 kubenswrapper[2579]: I0703 11:18:07.938277 2579 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["cert-manager/cert-manager-webhook-6486ffd997-xw44p"] Jul 03 11:18:07.983220 ip-10-0-136-0 kubenswrapper[2579]: I0703 11:18:07.983138 2579 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-xx7xz\" (UniqueName: \"kubernetes.io/projected/ad5cb49d-8c34-4726-946a-544c6a3a41e4-kube-api-access-xx7xz\") pod \"cert-manager-webhook-6486ffd997-xw44p\" (UID: \"ad5cb49d-8c34-4726-946a-544c6a3a41e4\") " pod="cert-manager/cert-manager-webhook-6486ffd997-xw44p" Jul 03 11:18:07.983482 ip-10-0-136-0 kubenswrapper[2579]: I0703 11:18:07.983352 2579 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"bound-sa-token\" (UniqueName: \"kubernetes.io/projected/ad5cb49d-8c34-4726-946a-544c6a3a41e4-bound-sa-token\") pod \"cert-manager-webhook-6486ffd997-xw44p\" (UID: \"ad5cb49d-8c34-4726-946a-544c6a3a41e4\") " pod="cert-manager/cert-manager-webhook-6486ffd997-xw44p" Jul 03 11:18:08.084717 ip-10-0-136-0 kubenswrapper[2579]: I0703 11:18:08.084671 2579 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-xx7xz\" (UniqueName: \"kubernetes.io/projected/ad5cb49d-8c34-4726-946a-544c6a3a41e4-kube-api-access-xx7xz\") pod \"cert-manager-webhook-6486ffd997-xw44p\" (UID: \"ad5cb49d-8c34-4726-946a-544c6a3a41e4\") " pod="cert-manager/cert-manager-webhook-6486ffd997-xw44p" Jul 03 11:18:08.084923 ip-10-0-136-0 kubenswrapper[2579]: I0703 11:18:08.084767 2579 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"bound-sa-token\" (UniqueName: \"kubernetes.io/projected/ad5cb49d-8c34-4726-946a-544c6a3a41e4-bound-sa-token\") pod \"cert-manager-webhook-6486ffd997-xw44p\" (UID: \"ad5cb49d-8c34-4726-946a-544c6a3a41e4\") " pod="cert-manager/cert-manager-webhook-6486ffd997-xw44p" Jul 03 11:18:08.104450 ip-10-0-136-0 kubenswrapper[2579]: I0703 11:18:08.104410 2579 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"bound-sa-token\" (UniqueName: \"kubernetes.io/projected/ad5cb49d-8c34-4726-946a-544c6a3a41e4-bound-sa-token\") pod \"cert-manager-webhook-6486ffd997-xw44p\" (UID: \"ad5cb49d-8c34-4726-946a-544c6a3a41e4\") " pod="cert-manager/cert-manager-webhook-6486ffd997-xw44p" Jul 03 11:18:08.124956 ip-10-0-136-0 kubenswrapper[2579]: I0703 11:18:08.124917 2579 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-xx7xz\" (UniqueName: \"kubernetes.io/projected/ad5cb49d-8c34-4726-946a-544c6a3a41e4-kube-api-access-xx7xz\") pod \"cert-manager-webhook-6486ffd997-xw44p\" (UID: \"ad5cb49d-8c34-4726-946a-544c6a3a41e4\") " pod="cert-manager/cert-manager-webhook-6486ffd997-xw44p" Jul 03 11:18:08.241883 ip-10-0-136-0 kubenswrapper[2579]: I0703 11:18:08.241785 2579 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="cert-manager/cert-manager-webhook-6486ffd997-xw44p" Jul 03 11:18:08.407222 ip-10-0-136-0 kubenswrapper[2579]: I0703 11:18:08.407173 2579 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["cert-manager/cert-manager-webhook-6486ffd997-xw44p"] Jul 03 11:18:08.409498 ip-10-0-136-0 kubenswrapper[2579]: W0703 11:18:08.409470 2579 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-besteffort.slice/kubepods-besteffort-podad5cb49d_8c34_4726_946a_544c6a3a41e4.slice/crio-9a3463da0669c2d97df4a0f9f4fc0158c9fbac012fb1fb77f0996725a4a383ef WatchSource:0}: Error finding container 9a3463da0669c2d97df4a0f9f4fc0158c9fbac012fb1fb77f0996725a4a383ef: Status 404 returned error can't find the container with id 9a3463da0669c2d97df4a0f9f4fc0158c9fbac012fb1fb77f0996725a4a383ef Jul 03 11:18:08.650969 ip-10-0-136-0 kubenswrapper[2579]: I0703 11:18:08.650925 2579 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="cert-manager/cert-manager-webhook-6486ffd997-xw44p" event={"ID":"ad5cb49d-8c34-4726-946a-544c6a3a41e4","Type":"ContainerStarted","Data":"9a3463da0669c2d97df4a0f9f4fc0158c9fbac012fb1fb77f0996725a4a383ef"} Jul 03 11:18:08.782119 ip-10-0-136-0 kubenswrapper[2579]: I0703 11:18:08.782072 2579 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-marketplace/77defbb6647882b321b46d16bdefe62633c3425354d3d93571a1d6a87f2s8tt"] Jul 03 11:18:08.785939 ip-10-0-136-0 kubenswrapper[2579]: I0703 11:18:08.785904 2579 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-marketplace/77defbb6647882b321b46d16bdefe62633c3425354d3d93571a1d6a87f2s8tt" Jul 03 11:18:08.791905 ip-10-0-136-0 kubenswrapper[2579]: I0703 11:18:08.791875 2579 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-marketplace\"/\"kube-root-ca.crt\"" Jul 03 11:18:08.792096 ip-10-0-136-0 kubenswrapper[2579]: I0703 11:18:08.791875 2579 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-marketplace\"/\"openshift-service-ca.crt\"" Jul 03 11:18:08.792096 ip-10-0-136-0 kubenswrapper[2579]: I0703 11:18:08.791885 2579 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-marketplace\"/\"default-dockercfg-6mvjw\"" Jul 03 11:18:08.812000 ip-10-0-136-0 kubenswrapper[2579]: I0703 11:18:08.811960 2579 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-marketplace/77defbb6647882b321b46d16bdefe62633c3425354d3d93571a1d6a87f2s8tt"] Jul 03 11:18:08.893399 ip-10-0-136-0 kubenswrapper[2579]: I0703 11:18:08.893347 2579 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-sczhk\" (UniqueName: \"kubernetes.io/projected/cfd46090-3072-4c99-ab42-83750ed0a0c7-kube-api-access-sczhk\") pod \"77defbb6647882b321b46d16bdefe62633c3425354d3d93571a1d6a87f2s8tt\" (UID: \"cfd46090-3072-4c99-ab42-83750ed0a0c7\") " pod="openshift-marketplace/77defbb6647882b321b46d16bdefe62633c3425354d3d93571a1d6a87f2s8tt" Jul 03 11:18:08.893607 ip-10-0-136-0 kubenswrapper[2579]: I0703 11:18:08.893454 2579 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"util\" (UniqueName: \"kubernetes.io/empty-dir/cfd46090-3072-4c99-ab42-83750ed0a0c7-util\") pod \"77defbb6647882b321b46d16bdefe62633c3425354d3d93571a1d6a87f2s8tt\" (UID: \"cfd46090-3072-4c99-ab42-83750ed0a0c7\") " pod="openshift-marketplace/77defbb6647882b321b46d16bdefe62633c3425354d3d93571a1d6a87f2s8tt" Jul 03 11:18:08.893607 ip-10-0-136-0 kubenswrapper[2579]: I0703 11:18:08.893480 2579 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"bundle\" (UniqueName: \"kubernetes.io/empty-dir/cfd46090-3072-4c99-ab42-83750ed0a0c7-bundle\") pod \"77defbb6647882b321b46d16bdefe62633c3425354d3d93571a1d6a87f2s8tt\" (UID: \"cfd46090-3072-4c99-ab42-83750ed0a0c7\") " pod="openshift-marketplace/77defbb6647882b321b46d16bdefe62633c3425354d3d93571a1d6a87f2s8tt" Jul 03 11:18:08.994316 ip-10-0-136-0 kubenswrapper[2579]: I0703 11:18:08.994211 2579 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-sczhk\" (UniqueName: \"kubernetes.io/projected/cfd46090-3072-4c99-ab42-83750ed0a0c7-kube-api-access-sczhk\") pod \"77defbb6647882b321b46d16bdefe62633c3425354d3d93571a1d6a87f2s8tt\" (UID: \"cfd46090-3072-4c99-ab42-83750ed0a0c7\") " pod="openshift-marketplace/77defbb6647882b321b46d16bdefe62633c3425354d3d93571a1d6a87f2s8tt" Jul 03 11:18:08.994316 ip-10-0-136-0 kubenswrapper[2579]: I0703 11:18:08.994300 2579 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"util\" (UniqueName: \"kubernetes.io/empty-dir/cfd46090-3072-4c99-ab42-83750ed0a0c7-util\") pod \"77defbb6647882b321b46d16bdefe62633c3425354d3d93571a1d6a87f2s8tt\" (UID: \"cfd46090-3072-4c99-ab42-83750ed0a0c7\") " pod="openshift-marketplace/77defbb6647882b321b46d16bdefe62633c3425354d3d93571a1d6a87f2s8tt" Jul 03 11:18:08.994843 ip-10-0-136-0 kubenswrapper[2579]: I0703 11:18:08.994334 2579 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"bundle\" (UniqueName: \"kubernetes.io/empty-dir/cfd46090-3072-4c99-ab42-83750ed0a0c7-bundle\") pod \"77defbb6647882b321b46d16bdefe62633c3425354d3d93571a1d6a87f2s8tt\" (UID: \"cfd46090-3072-4c99-ab42-83750ed0a0c7\") " pod="openshift-marketplace/77defbb6647882b321b46d16bdefe62633c3425354d3d93571a1d6a87f2s8tt" Jul 03 11:18:08.994843 ip-10-0-136-0 kubenswrapper[2579]: I0703 11:18:08.994785 2579 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"bundle\" (UniqueName: \"kubernetes.io/empty-dir/cfd46090-3072-4c99-ab42-83750ed0a0c7-bundle\") pod \"77defbb6647882b321b46d16bdefe62633c3425354d3d93571a1d6a87f2s8tt\" (UID: \"cfd46090-3072-4c99-ab42-83750ed0a0c7\") " pod="openshift-marketplace/77defbb6647882b321b46d16bdefe62633c3425354d3d93571a1d6a87f2s8tt" Jul 03 11:18:08.994939 ip-10-0-136-0 kubenswrapper[2579]: I0703 11:18:08.994842 2579 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"util\" (UniqueName: \"kubernetes.io/empty-dir/cfd46090-3072-4c99-ab42-83750ed0a0c7-util\") pod \"77defbb6647882b321b46d16bdefe62633c3425354d3d93571a1d6a87f2s8tt\" (UID: \"cfd46090-3072-4c99-ab42-83750ed0a0c7\") " pod="openshift-marketplace/77defbb6647882b321b46d16bdefe62633c3425354d3d93571a1d6a87f2s8tt" Jul 03 11:18:09.010238 ip-10-0-136-0 kubenswrapper[2579]: I0703 11:18:09.010177 2579 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-sczhk\" (UniqueName: \"kubernetes.io/projected/cfd46090-3072-4c99-ab42-83750ed0a0c7-kube-api-access-sczhk\") pod \"77defbb6647882b321b46d16bdefe62633c3425354d3d93571a1d6a87f2s8tt\" (UID: \"cfd46090-3072-4c99-ab42-83750ed0a0c7\") " pod="openshift-marketplace/77defbb6647882b321b46d16bdefe62633c3425354d3d93571a1d6a87f2s8tt" Jul 03 11:18:09.096128 ip-10-0-136-0 kubenswrapper[2579]: I0703 11:18:09.096079 2579 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-marketplace/77defbb6647882b321b46d16bdefe62633c3425354d3d93571a1d6a87f2s8tt" Jul 03 11:18:09.267296 ip-10-0-136-0 kubenswrapper[2579]: I0703 11:18:09.267260 2579 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-marketplace/77defbb6647882b321b46d16bdefe62633c3425354d3d93571a1d6a87f2s8tt"] Jul 03 11:18:09.275171 ip-10-0-136-0 kubenswrapper[2579]: W0703 11:18:09.275130 2579 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podcfd46090_3072_4c99_ab42_83750ed0a0c7.slice/crio-88b9478584cb9e557f569bcf378de2b4e46a98566496d2b501a3c8d5ad89dd64 WatchSource:0}: Error finding container 88b9478584cb9e557f569bcf378de2b4e46a98566496d2b501a3c8d5ad89dd64: Status 404 returned error can't find the container with id 88b9478584cb9e557f569bcf378de2b4e46a98566496d2b501a3c8d5ad89dd64 Jul 03 11:18:09.658621 ip-10-0-136-0 kubenswrapper[2579]: I0703 11:18:09.658514 2579 generic.go:356] "Generic (PLEG): container finished" podID="cfd46090-3072-4c99-ab42-83750ed0a0c7" containerID="914dc33209ad2abf3f36a5b256d99ccb5bf298a915f6790442ca571c134d85c1" exitCode=0 Jul 03 11:18:09.658868 ip-10-0-136-0 kubenswrapper[2579]: I0703 11:18:09.658613 2579 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-marketplace/77defbb6647882b321b46d16bdefe62633c3425354d3d93571a1d6a87f2s8tt" event={"ID":"cfd46090-3072-4c99-ab42-83750ed0a0c7","Type":"ContainerDied","Data":"914dc33209ad2abf3f36a5b256d99ccb5bf298a915f6790442ca571c134d85c1"} Jul 03 11:18:09.658868 ip-10-0-136-0 kubenswrapper[2579]: I0703 11:18:09.658696 2579 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-marketplace/77defbb6647882b321b46d16bdefe62633c3425354d3d93571a1d6a87f2s8tt" event={"ID":"cfd46090-3072-4c99-ab42-83750ed0a0c7","Type":"ContainerStarted","Data":"88b9478584cb9e557f569bcf378de2b4e46a98566496d2b501a3c8d5ad89dd64"} Jul 03 11:18:11.671394 ip-10-0-136-0 kubenswrapper[2579]: I0703 11:18:11.671358 2579 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="cert-manager/cert-manager-webhook-6486ffd997-xw44p" event={"ID":"ad5cb49d-8c34-4726-946a-544c6a3a41e4","Type":"ContainerStarted","Data":"c555f9c80dc8460244bff8ae419954ee9ae3d511d9a7ec17837c722b62b0eab1"} Jul 03 11:18:11.671862 ip-10-0-136-0 kubenswrapper[2579]: I0703 11:18:11.671426 2579 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="cert-manager/cert-manager-webhook-6486ffd997-xw44p" Jul 03 11:18:11.695603 ip-10-0-136-0 kubenswrapper[2579]: I0703 11:18:11.695543 2579 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="cert-manager/cert-manager-webhook-6486ffd997-xw44p" podStartSLOduration=1.8571959150000001 podStartE2EDuration="4.695524089s" totalImagesPullingTime="2.838328174s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-03 11:18:07 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-03 11:18:11.693323134 +0000 UTC m=+348.281627455" watchObservedRunningTime="2026-07-03 11:18:11.695524089 +0000 UTC m=+348.283828408" Jul 03 11:18:13.684380 ip-10-0-136-0 kubenswrapper[2579]: I0703 11:18:13.684270 2579 generic.go:356] "Generic (PLEG): container finished" podID="cfd46090-3072-4c99-ab42-83750ed0a0c7" containerID="070e18b9a8853d632945b311f0ce338483aa4312877a77bf2d3e53ab75e90167" exitCode=0 Jul 03 11:18:13.684860 ip-10-0-136-0 kubenswrapper[2579]: I0703 11:18:13.684363 2579 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-marketplace/77defbb6647882b321b46d16bdefe62633c3425354d3d93571a1d6a87f2s8tt" event={"ID":"cfd46090-3072-4c99-ab42-83750ed0a0c7","Type":"ContainerDied","Data":"070e18b9a8853d632945b311f0ce338483aa4312877a77bf2d3e53ab75e90167"} Jul 03 11:18:14.691878 ip-10-0-136-0 kubenswrapper[2579]: I0703 11:18:14.691842 2579 generic.go:356] "Generic (PLEG): container finished" podID="cfd46090-3072-4c99-ab42-83750ed0a0c7" containerID="4bcd7bdbee648c4755d12c459ee0814cbe5e6c9c4fa2fc69614f1ac57246eb93" exitCode=0 Jul 03 11:18:14.692312 ip-10-0-136-0 kubenswrapper[2579]: I0703 11:18:14.691935 2579 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-marketplace/77defbb6647882b321b46d16bdefe62633c3425354d3d93571a1d6a87f2s8tt" event={"ID":"cfd46090-3072-4c99-ab42-83750ed0a0c7","Type":"ContainerDied","Data":"4bcd7bdbee648c4755d12c459ee0814cbe5e6c9c4fa2fc69614f1ac57246eb93"} Jul 03 11:18:14.949823 ip-10-0-136-0 kubenswrapper[2579]: I0703 11:18:14.949723 2579 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["cert-manager/cert-manager-cainjector-559cdd8c87-dfbbv"] Jul 03 11:18:14.970455 ip-10-0-136-0 kubenswrapper[2579]: I0703 11:18:14.970416 2579 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["cert-manager/cert-manager-cainjector-559cdd8c87-dfbbv"] Jul 03 11:18:14.970632 ip-10-0-136-0 kubenswrapper[2579]: I0703 11:18:14.970550 2579 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="cert-manager/cert-manager-cainjector-559cdd8c87-dfbbv" Jul 03 11:18:14.973939 ip-10-0-136-0 kubenswrapper[2579]: I0703 11:18:14.973909 2579 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"cert-manager\"/\"cert-manager-cainjector-dockercfg-c5zf4\"" Jul 03 11:18:15.051986 ip-10-0-136-0 kubenswrapper[2579]: I0703 11:18:15.051939 2579 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-74jr2\" (UniqueName: \"kubernetes.io/projected/86c887e7-3f7a-4e0f-bd56-90455d142336-kube-api-access-74jr2\") pod \"cert-manager-cainjector-559cdd8c87-dfbbv\" (UID: \"86c887e7-3f7a-4e0f-bd56-90455d142336\") " pod="cert-manager/cert-manager-cainjector-559cdd8c87-dfbbv" Jul 03 11:18:15.052223 ip-10-0-136-0 kubenswrapper[2579]: I0703 11:18:15.052110 2579 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"bound-sa-token\" (UniqueName: \"kubernetes.io/projected/86c887e7-3f7a-4e0f-bd56-90455d142336-bound-sa-token\") pod \"cert-manager-cainjector-559cdd8c87-dfbbv\" (UID: \"86c887e7-3f7a-4e0f-bd56-90455d142336\") " pod="cert-manager/cert-manager-cainjector-559cdd8c87-dfbbv" Jul 03 11:18:15.153517 ip-10-0-136-0 kubenswrapper[2579]: I0703 11:18:15.153459 2579 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-74jr2\" (UniqueName: \"kubernetes.io/projected/86c887e7-3f7a-4e0f-bd56-90455d142336-kube-api-access-74jr2\") pod \"cert-manager-cainjector-559cdd8c87-dfbbv\" (UID: \"86c887e7-3f7a-4e0f-bd56-90455d142336\") " pod="cert-manager/cert-manager-cainjector-559cdd8c87-dfbbv" Jul 03 11:18:15.153781 ip-10-0-136-0 kubenswrapper[2579]: I0703 11:18:15.153536 2579 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"bound-sa-token\" (UniqueName: \"kubernetes.io/projected/86c887e7-3f7a-4e0f-bd56-90455d142336-bound-sa-token\") pod \"cert-manager-cainjector-559cdd8c87-dfbbv\" (UID: \"86c887e7-3f7a-4e0f-bd56-90455d142336\") " pod="cert-manager/cert-manager-cainjector-559cdd8c87-dfbbv" Jul 03 11:18:15.163825 ip-10-0-136-0 kubenswrapper[2579]: I0703 11:18:15.163792 2579 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"bound-sa-token\" (UniqueName: \"kubernetes.io/projected/86c887e7-3f7a-4e0f-bd56-90455d142336-bound-sa-token\") pod \"cert-manager-cainjector-559cdd8c87-dfbbv\" (UID: \"86c887e7-3f7a-4e0f-bd56-90455d142336\") " pod="cert-manager/cert-manager-cainjector-559cdd8c87-dfbbv" Jul 03 11:18:15.165965 ip-10-0-136-0 kubenswrapper[2579]: I0703 11:18:15.165925 2579 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-74jr2\" (UniqueName: \"kubernetes.io/projected/86c887e7-3f7a-4e0f-bd56-90455d142336-kube-api-access-74jr2\") pod \"cert-manager-cainjector-559cdd8c87-dfbbv\" (UID: \"86c887e7-3f7a-4e0f-bd56-90455d142336\") " pod="cert-manager/cert-manager-cainjector-559cdd8c87-dfbbv" Jul 03 11:18:15.280901 ip-10-0-136-0 kubenswrapper[2579]: I0703 11:18:15.280864 2579 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="cert-manager/cert-manager-cainjector-559cdd8c87-dfbbv" Jul 03 11:18:15.457523 ip-10-0-136-0 kubenswrapper[2579]: I0703 11:18:15.457483 2579 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["cert-manager/cert-manager-cainjector-559cdd8c87-dfbbv"] Jul 03 11:18:15.460853 ip-10-0-136-0 kubenswrapper[2579]: W0703 11:18:15.460819 2579 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-besteffort.slice/kubepods-besteffort-pod86c887e7_3f7a_4e0f_bd56_90455d142336.slice/crio-3817669d3a88b30ce0e7f6c728d39293030492da4c16cf79fdd6952a21e985d6 WatchSource:0}: Error finding container 3817669d3a88b30ce0e7f6c728d39293030492da4c16cf79fdd6952a21e985d6: Status 404 returned error can't find the container with id 3817669d3a88b30ce0e7f6c728d39293030492da4c16cf79fdd6952a21e985d6 Jul 03 11:18:15.697974 ip-10-0-136-0 kubenswrapper[2579]: I0703 11:18:15.697934 2579 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="cert-manager/cert-manager-cainjector-559cdd8c87-dfbbv" event={"ID":"86c887e7-3f7a-4e0f-bd56-90455d142336","Type":"ContainerStarted","Data":"72c9c05bc1c6bf1536f007054d2de7d1b46c454e49e58d15a07fb9c0dff27141"} Jul 03 11:18:15.698436 ip-10-0-136-0 kubenswrapper[2579]: I0703 11:18:15.697989 2579 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="cert-manager/cert-manager-cainjector-559cdd8c87-dfbbv" event={"ID":"86c887e7-3f7a-4e0f-bd56-90455d142336","Type":"ContainerStarted","Data":"3817669d3a88b30ce0e7f6c728d39293030492da4c16cf79fdd6952a21e985d6"} Jul 03 11:18:15.737083 ip-10-0-136-0 kubenswrapper[2579]: I0703 11:18:15.737021 2579 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="cert-manager/cert-manager-cainjector-559cdd8c87-dfbbv" podStartSLOduration=1.7370011060000001 podStartE2EDuration="1.737001106s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-03 11:18:14 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-03 11:18:15.730604369 +0000 UTC m=+352.318908686" watchObservedRunningTime="2026-07-03 11:18:15.737001106 +0000 UTC m=+352.325305426" Jul 03 11:18:15.872474 ip-10-0-136-0 kubenswrapper[2579]: I0703 11:18:15.872430 2579 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-marketplace/77defbb6647882b321b46d16bdefe62633c3425354d3d93571a1d6a87f2s8tt" Jul 03 11:18:15.959816 ip-10-0-136-0 kubenswrapper[2579]: I0703 11:18:15.959767 2579 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"util\" (UniqueName: \"kubernetes.io/empty-dir/cfd46090-3072-4c99-ab42-83750ed0a0c7-util\") pod \"cfd46090-3072-4c99-ab42-83750ed0a0c7\" (UID: \"cfd46090-3072-4c99-ab42-83750ed0a0c7\") " Jul 03 11:18:15.960040 ip-10-0-136-0 kubenswrapper[2579]: I0703 11:18:15.959893 2579 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"bundle\" (UniqueName: \"kubernetes.io/empty-dir/cfd46090-3072-4c99-ab42-83750ed0a0c7-bundle\") pod \"cfd46090-3072-4c99-ab42-83750ed0a0c7\" (UID: \"cfd46090-3072-4c99-ab42-83750ed0a0c7\") " Jul 03 11:18:15.960040 ip-10-0-136-0 kubenswrapper[2579]: I0703 11:18:15.959940 2579 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-sczhk\" (UniqueName: \"kubernetes.io/projected/cfd46090-3072-4c99-ab42-83750ed0a0c7-kube-api-access-sczhk\") pod \"cfd46090-3072-4c99-ab42-83750ed0a0c7\" (UID: \"cfd46090-3072-4c99-ab42-83750ed0a0c7\") " Jul 03 11:18:15.960338 ip-10-0-136-0 kubenswrapper[2579]: I0703 11:18:15.960313 2579 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/cfd46090-3072-4c99-ab42-83750ed0a0c7-bundle" (OuterVolumeSpecName: "bundle") pod "cfd46090-3072-4c99-ab42-83750ed0a0c7" (UID: "cfd46090-3072-4c99-ab42-83750ed0a0c7"). InnerVolumeSpecName "bundle". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Jul 03 11:18:15.962539 ip-10-0-136-0 kubenswrapper[2579]: I0703 11:18:15.962490 2579 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/cfd46090-3072-4c99-ab42-83750ed0a0c7-kube-api-access-sczhk" (OuterVolumeSpecName: "kube-api-access-sczhk") pod "cfd46090-3072-4c99-ab42-83750ed0a0c7" (UID: "cfd46090-3072-4c99-ab42-83750ed0a0c7"). InnerVolumeSpecName "kube-api-access-sczhk". PluginName "kubernetes.io/projected", VolumeGIDValue "" Jul 03 11:18:15.964003 ip-10-0-136-0 kubenswrapper[2579]: I0703 11:18:15.963967 2579 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/cfd46090-3072-4c99-ab42-83750ed0a0c7-util" (OuterVolumeSpecName: "util") pod "cfd46090-3072-4c99-ab42-83750ed0a0c7" (UID: "cfd46090-3072-4c99-ab42-83750ed0a0c7"). InnerVolumeSpecName "util". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Jul 03 11:18:16.060949 ip-10-0-136-0 kubenswrapper[2579]: I0703 11:18:16.060837 2579 reconciler_common.go:299] "Volume detached for volume \"bundle\" (UniqueName: \"kubernetes.io/empty-dir/cfd46090-3072-4c99-ab42-83750ed0a0c7-bundle\") on node \"ip-10-0-136-0.ec2.internal\" DevicePath \"\"" Jul 03 11:18:16.060949 ip-10-0-136-0 kubenswrapper[2579]: I0703 11:18:16.060873 2579 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-sczhk\" (UniqueName: \"kubernetes.io/projected/cfd46090-3072-4c99-ab42-83750ed0a0c7-kube-api-access-sczhk\") on node \"ip-10-0-136-0.ec2.internal\" DevicePath \"\"" Jul 03 11:18:16.060949 ip-10-0-136-0 kubenswrapper[2579]: I0703 11:18:16.060889 2579 reconciler_common.go:299] "Volume detached for volume \"util\" (UniqueName: \"kubernetes.io/empty-dir/cfd46090-3072-4c99-ab42-83750ed0a0c7-util\") on node \"ip-10-0-136-0.ec2.internal\" DevicePath \"\"" Jul 03 11:18:16.706590 ip-10-0-136-0 kubenswrapper[2579]: I0703 11:18:16.706550 2579 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-marketplace/77defbb6647882b321b46d16bdefe62633c3425354d3d93571a1d6a87f2s8tt" Jul 03 11:18:16.706590 ip-10-0-136-0 kubenswrapper[2579]: I0703 11:18:16.706560 2579 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-marketplace/77defbb6647882b321b46d16bdefe62633c3425354d3d93571a1d6a87f2s8tt" event={"ID":"cfd46090-3072-4c99-ab42-83750ed0a0c7","Type":"ContainerDied","Data":"88b9478584cb9e557f569bcf378de2b4e46a98566496d2b501a3c8d5ad89dd64"} Jul 03 11:18:16.707120 ip-10-0-136-0 kubenswrapper[2579]: I0703 11:18:16.706653 2579 pod_container_deletor.go:80] "Container not found in pod's containers" containerID="88b9478584cb9e557f569bcf378de2b4e46a98566496d2b501a3c8d5ad89dd64" Jul 03 11:18:17.678949 ip-10-0-136-0 kubenswrapper[2579]: I0703 11:18:17.678916 2579 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="cert-manager/cert-manager-webhook-6486ffd997-xw44p" Jul 03 11:18:21.432700 ip-10-0-136-0 kubenswrapper[2579]: I0703 11:18:21.432652 2579 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-lws-operator/openshift-lws-operator-fd8ccff4c-zb7hd"] Jul 03 11:18:21.433218 ip-10-0-136-0 kubenswrapper[2579]: I0703 11:18:21.432955 2579 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="cfd46090-3072-4c99-ab42-83750ed0a0c7" containerName="util" Jul 03 11:18:21.433218 ip-10-0-136-0 kubenswrapper[2579]: I0703 11:18:21.432968 2579 state_mem.go:107] "Deleted CPUSet assignment" podUID="cfd46090-3072-4c99-ab42-83750ed0a0c7" containerName="util" Jul 03 11:18:21.433218 ip-10-0-136-0 kubenswrapper[2579]: I0703 11:18:21.432986 2579 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="cfd46090-3072-4c99-ab42-83750ed0a0c7" containerName="pull" Jul 03 11:18:21.433218 ip-10-0-136-0 kubenswrapper[2579]: I0703 11:18:21.432993 2579 state_mem.go:107] "Deleted CPUSet assignment" podUID="cfd46090-3072-4c99-ab42-83750ed0a0c7" containerName="pull" Jul 03 11:18:21.433218 ip-10-0-136-0 kubenswrapper[2579]: I0703 11:18:21.433001 2579 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="cfd46090-3072-4c99-ab42-83750ed0a0c7" containerName="extract" Jul 03 11:18:21.433218 ip-10-0-136-0 kubenswrapper[2579]: I0703 11:18:21.433005 2579 state_mem.go:107] "Deleted CPUSet assignment" podUID="cfd46090-3072-4c99-ab42-83750ed0a0c7" containerName="extract" Jul 03 11:18:21.433218 ip-10-0-136-0 kubenswrapper[2579]: I0703 11:18:21.433051 2579 memory_manager.go:368] "RemoveStaleState removing state" podUID="cfd46090-3072-4c99-ab42-83750ed0a0c7" containerName="extract" Jul 03 11:18:21.435171 ip-10-0-136-0 kubenswrapper[2579]: I0703 11:18:21.435151 2579 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-lws-operator/openshift-lws-operator-fd8ccff4c-zb7hd" Jul 03 11:18:21.438719 ip-10-0-136-0 kubenswrapper[2579]: I0703 11:18:21.438687 2579 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-lws-operator\"/\"kube-root-ca.crt\"" Jul 03 11:18:21.439701 ip-10-0-136-0 kubenswrapper[2579]: I0703 11:18:21.439665 2579 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-lws-operator\"/\"openshift-service-ca.crt\"" Jul 03 11:18:21.439857 ip-10-0-136-0 kubenswrapper[2579]: I0703 11:18:21.439678 2579 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-lws-operator\"/\"openshift-lws-operator-dockercfg-wdcdt\"" Jul 03 11:18:21.447814 ip-10-0-136-0 kubenswrapper[2579]: I0703 11:18:21.447778 2579 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-lws-operator/openshift-lws-operator-fd8ccff4c-zb7hd"] Jul 03 11:18:21.509440 ip-10-0-136-0 kubenswrapper[2579]: I0703 11:18:21.509392 2579 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tmp\" (UniqueName: \"kubernetes.io/empty-dir/64beee32-e069-4d34-b14c-f08e6f50dd74-tmp\") pod \"openshift-lws-operator-fd8ccff4c-zb7hd\" (UID: \"64beee32-e069-4d34-b14c-f08e6f50dd74\") " pod="openshift-lws-operator/openshift-lws-operator-fd8ccff4c-zb7hd" Jul 03 11:18:21.509622 ip-10-0-136-0 kubenswrapper[2579]: I0703 11:18:21.509462 2579 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-2t7pt\" (UniqueName: \"kubernetes.io/projected/64beee32-e069-4d34-b14c-f08e6f50dd74-kube-api-access-2t7pt\") pod \"openshift-lws-operator-fd8ccff4c-zb7hd\" (UID: \"64beee32-e069-4d34-b14c-f08e6f50dd74\") " pod="openshift-lws-operator/openshift-lws-operator-fd8ccff4c-zb7hd" Jul 03 11:18:21.610222 ip-10-0-136-0 kubenswrapper[2579]: I0703 11:18:21.610148 2579 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tmp\" (UniqueName: \"kubernetes.io/empty-dir/64beee32-e069-4d34-b14c-f08e6f50dd74-tmp\") pod \"openshift-lws-operator-fd8ccff4c-zb7hd\" (UID: \"64beee32-e069-4d34-b14c-f08e6f50dd74\") " pod="openshift-lws-operator/openshift-lws-operator-fd8ccff4c-zb7hd" Jul 03 11:18:21.610387 ip-10-0-136-0 kubenswrapper[2579]: I0703 11:18:21.610237 2579 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-2t7pt\" (UniqueName: \"kubernetes.io/projected/64beee32-e069-4d34-b14c-f08e6f50dd74-kube-api-access-2t7pt\") pod \"openshift-lws-operator-fd8ccff4c-zb7hd\" (UID: \"64beee32-e069-4d34-b14c-f08e6f50dd74\") " pod="openshift-lws-operator/openshift-lws-operator-fd8ccff4c-zb7hd" Jul 03 11:18:21.610550 ip-10-0-136-0 kubenswrapper[2579]: I0703 11:18:21.610530 2579 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"tmp\" (UniqueName: \"kubernetes.io/empty-dir/64beee32-e069-4d34-b14c-f08e6f50dd74-tmp\") pod \"openshift-lws-operator-fd8ccff4c-zb7hd\" (UID: \"64beee32-e069-4d34-b14c-f08e6f50dd74\") " pod="openshift-lws-operator/openshift-lws-operator-fd8ccff4c-zb7hd" Jul 03 11:18:21.620590 ip-10-0-136-0 kubenswrapper[2579]: I0703 11:18:21.620556 2579 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-2t7pt\" (UniqueName: \"kubernetes.io/projected/64beee32-e069-4d34-b14c-f08e6f50dd74-kube-api-access-2t7pt\") pod \"openshift-lws-operator-fd8ccff4c-zb7hd\" (UID: \"64beee32-e069-4d34-b14c-f08e6f50dd74\") " pod="openshift-lws-operator/openshift-lws-operator-fd8ccff4c-zb7hd" Jul 03 11:18:21.745308 ip-10-0-136-0 kubenswrapper[2579]: I0703 11:18:21.745202 2579 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-lws-operator/openshift-lws-operator-fd8ccff4c-zb7hd" Jul 03 11:18:21.906743 ip-10-0-136-0 kubenswrapper[2579]: I0703 11:18:21.906703 2579 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-lws-operator/openshift-lws-operator-fd8ccff4c-zb7hd"] Jul 03 11:18:21.910127 ip-10-0-136-0 kubenswrapper[2579]: W0703 11:18:21.910091 2579 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-besteffort.slice/kubepods-besteffort-pod64beee32_e069_4d34_b14c_f08e6f50dd74.slice/crio-73f44381e66ed05f85f6e2e1c95df6ab202c38b83c021c520635b681fbd8a265 WatchSource:0}: Error finding container 73f44381e66ed05f85f6e2e1c95df6ab202c38b83c021c520635b681fbd8a265: Status 404 returned error can't find the container with id 73f44381e66ed05f85f6e2e1c95df6ab202c38b83c021c520635b681fbd8a265 Jul 03 11:18:22.736456 ip-10-0-136-0 kubenswrapper[2579]: I0703 11:18:22.736418 2579 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-lws-operator/openshift-lws-operator-fd8ccff4c-zb7hd" event={"ID":"64beee32-e069-4d34-b14c-f08e6f50dd74","Type":"ContainerStarted","Data":"73f44381e66ed05f85f6e2e1c95df6ab202c38b83c021c520635b681fbd8a265"} Jul 03 11:18:26.756665 ip-10-0-136-0 kubenswrapper[2579]: I0703 11:18:26.756618 2579 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-lws-operator/openshift-lws-operator-fd8ccff4c-zb7hd" event={"ID":"64beee32-e069-4d34-b14c-f08e6f50dd74","Type":"ContainerStarted","Data":"af3aef45019b7b2440af9a0a918c4f98a1ebb1bd3fc318ca4d920cd82d3a02a8"} Jul 03 11:18:26.786891 ip-10-0-136-0 kubenswrapper[2579]: I0703 11:18:26.786823 2579 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-lws-operator/openshift-lws-operator-fd8ccff4c-zb7hd" podStartSLOduration=1.566951885 podStartE2EDuration="5.786803086s" totalImagesPullingTime="4.219851201s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-03 11:18:21 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-03 11:18:26.785979241 +0000 UTC m=+363.374283559" watchObservedRunningTime="2026-07-03 11:18:26.786803086 +0000 UTC m=+363.375107406" Jul 03 11:18:29.008534 ip-10-0-136-0 kubenswrapper[2579]: I0703 11:18:29.008488 2579 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-marketplace/629fd74b02459b8d9ee224b8bdbb513172290d80b07a5bf327840796afz4k7n"] Jul 03 11:18:29.011225 ip-10-0-136-0 kubenswrapper[2579]: I0703 11:18:29.011180 2579 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-marketplace/629fd74b02459b8d9ee224b8bdbb513172290d80b07a5bf327840796afz4k7n" Jul 03 11:18:29.013950 ip-10-0-136-0 kubenswrapper[2579]: I0703 11:18:29.013921 2579 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-marketplace\"/\"openshift-service-ca.crt\"" Jul 03 11:18:29.014109 ip-10-0-136-0 kubenswrapper[2579]: I0703 11:18:29.014081 2579 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-marketplace\"/\"kube-root-ca.crt\"" Jul 03 11:18:29.014261 ip-10-0-136-0 kubenswrapper[2579]: I0703 11:18:29.014247 2579 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-marketplace\"/\"default-dockercfg-6mvjw\"" Jul 03 11:18:29.026467 ip-10-0-136-0 kubenswrapper[2579]: I0703 11:18:29.026427 2579 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-marketplace/629fd74b02459b8d9ee224b8bdbb513172290d80b07a5bf327840796afz4k7n"] Jul 03 11:18:29.177181 ip-10-0-136-0 kubenswrapper[2579]: I0703 11:18:29.177133 2579 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-zbldc\" (UniqueName: \"kubernetes.io/projected/509683d2-efbb-42d5-98eb-76f51372f4cf-kube-api-access-zbldc\") pod \"629fd74b02459b8d9ee224b8bdbb513172290d80b07a5bf327840796afz4k7n\" (UID: \"509683d2-efbb-42d5-98eb-76f51372f4cf\") " pod="openshift-marketplace/629fd74b02459b8d9ee224b8bdbb513172290d80b07a5bf327840796afz4k7n" Jul 03 11:18:29.177395 ip-10-0-136-0 kubenswrapper[2579]: I0703 11:18:29.177262 2579 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"bundle\" (UniqueName: \"kubernetes.io/empty-dir/509683d2-efbb-42d5-98eb-76f51372f4cf-bundle\") pod \"629fd74b02459b8d9ee224b8bdbb513172290d80b07a5bf327840796afz4k7n\" (UID: \"509683d2-efbb-42d5-98eb-76f51372f4cf\") " pod="openshift-marketplace/629fd74b02459b8d9ee224b8bdbb513172290d80b07a5bf327840796afz4k7n" Jul 03 11:18:29.177395 ip-10-0-136-0 kubenswrapper[2579]: I0703 11:18:29.177307 2579 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"util\" (UniqueName: \"kubernetes.io/empty-dir/509683d2-efbb-42d5-98eb-76f51372f4cf-util\") pod \"629fd74b02459b8d9ee224b8bdbb513172290d80b07a5bf327840796afz4k7n\" (UID: \"509683d2-efbb-42d5-98eb-76f51372f4cf\") " pod="openshift-marketplace/629fd74b02459b8d9ee224b8bdbb513172290d80b07a5bf327840796afz4k7n" Jul 03 11:18:29.278096 ip-10-0-136-0 kubenswrapper[2579]: I0703 11:18:29.278056 2579 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-zbldc\" (UniqueName: \"kubernetes.io/projected/509683d2-efbb-42d5-98eb-76f51372f4cf-kube-api-access-zbldc\") pod \"629fd74b02459b8d9ee224b8bdbb513172290d80b07a5bf327840796afz4k7n\" (UID: \"509683d2-efbb-42d5-98eb-76f51372f4cf\") " pod="openshift-marketplace/629fd74b02459b8d9ee224b8bdbb513172290d80b07a5bf327840796afz4k7n" Jul 03 11:18:29.278342 ip-10-0-136-0 kubenswrapper[2579]: I0703 11:18:29.278117 2579 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"bundle\" (UniqueName: \"kubernetes.io/empty-dir/509683d2-efbb-42d5-98eb-76f51372f4cf-bundle\") pod \"629fd74b02459b8d9ee224b8bdbb513172290d80b07a5bf327840796afz4k7n\" (UID: \"509683d2-efbb-42d5-98eb-76f51372f4cf\") " pod="openshift-marketplace/629fd74b02459b8d9ee224b8bdbb513172290d80b07a5bf327840796afz4k7n" Jul 03 11:18:29.278342 ip-10-0-136-0 kubenswrapper[2579]: I0703 11:18:29.278147 2579 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"util\" (UniqueName: \"kubernetes.io/empty-dir/509683d2-efbb-42d5-98eb-76f51372f4cf-util\") pod \"629fd74b02459b8d9ee224b8bdbb513172290d80b07a5bf327840796afz4k7n\" (UID: \"509683d2-efbb-42d5-98eb-76f51372f4cf\") " pod="openshift-marketplace/629fd74b02459b8d9ee224b8bdbb513172290d80b07a5bf327840796afz4k7n" Jul 03 11:18:29.278526 ip-10-0-136-0 kubenswrapper[2579]: I0703 11:18:29.278508 2579 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"util\" (UniqueName: \"kubernetes.io/empty-dir/509683d2-efbb-42d5-98eb-76f51372f4cf-util\") pod \"629fd74b02459b8d9ee224b8bdbb513172290d80b07a5bf327840796afz4k7n\" (UID: \"509683d2-efbb-42d5-98eb-76f51372f4cf\") " pod="openshift-marketplace/629fd74b02459b8d9ee224b8bdbb513172290d80b07a5bf327840796afz4k7n" Jul 03 11:18:29.278570 ip-10-0-136-0 kubenswrapper[2579]: I0703 11:18:29.278537 2579 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"bundle\" (UniqueName: \"kubernetes.io/empty-dir/509683d2-efbb-42d5-98eb-76f51372f4cf-bundle\") pod \"629fd74b02459b8d9ee224b8bdbb513172290d80b07a5bf327840796afz4k7n\" (UID: \"509683d2-efbb-42d5-98eb-76f51372f4cf\") " pod="openshift-marketplace/629fd74b02459b8d9ee224b8bdbb513172290d80b07a5bf327840796afz4k7n" Jul 03 11:18:29.287206 ip-10-0-136-0 kubenswrapper[2579]: I0703 11:18:29.287149 2579 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-zbldc\" (UniqueName: \"kubernetes.io/projected/509683d2-efbb-42d5-98eb-76f51372f4cf-kube-api-access-zbldc\") pod \"629fd74b02459b8d9ee224b8bdbb513172290d80b07a5bf327840796afz4k7n\" (UID: \"509683d2-efbb-42d5-98eb-76f51372f4cf\") " pod="openshift-marketplace/629fd74b02459b8d9ee224b8bdbb513172290d80b07a5bf327840796afz4k7n" Jul 03 11:18:29.321129 ip-10-0-136-0 kubenswrapper[2579]: I0703 11:18:29.321081 2579 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-marketplace/629fd74b02459b8d9ee224b8bdbb513172290d80b07a5bf327840796afz4k7n" Jul 03 11:18:29.463364 ip-10-0-136-0 kubenswrapper[2579]: I0703 11:18:29.463326 2579 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-marketplace/629fd74b02459b8d9ee224b8bdbb513172290d80b07a5bf327840796afz4k7n"] Jul 03 11:18:29.465815 ip-10-0-136-0 kubenswrapper[2579]: W0703 11:18:29.465786 2579 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod509683d2_efbb_42d5_98eb_76f51372f4cf.slice/crio-e92c1c10039bab8c6652714dd8b16592036beee195077e8a28302025ade68df6 WatchSource:0}: Error finding container e92c1c10039bab8c6652714dd8b16592036beee195077e8a28302025ade68df6: Status 404 returned error can't find the container with id e92c1c10039bab8c6652714dd8b16592036beee195077e8a28302025ade68df6 Jul 03 11:18:29.772348 ip-10-0-136-0 kubenswrapper[2579]: I0703 11:18:29.772316 2579 generic.go:356] "Generic (PLEG): container finished" podID="509683d2-efbb-42d5-98eb-76f51372f4cf" containerID="7fc344291c515526938e5f551d4e0d83ab53ad48af4d45a2921b3418f41ca653" exitCode=0 Jul 03 11:18:29.772487 ip-10-0-136-0 kubenswrapper[2579]: I0703 11:18:29.772376 2579 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-marketplace/629fd74b02459b8d9ee224b8bdbb513172290d80b07a5bf327840796afz4k7n" event={"ID":"509683d2-efbb-42d5-98eb-76f51372f4cf","Type":"ContainerDied","Data":"7fc344291c515526938e5f551d4e0d83ab53ad48af4d45a2921b3418f41ca653"} Jul 03 11:18:29.772487 ip-10-0-136-0 kubenswrapper[2579]: I0703 11:18:29.772414 2579 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-marketplace/629fd74b02459b8d9ee224b8bdbb513172290d80b07a5bf327840796afz4k7n" event={"ID":"509683d2-efbb-42d5-98eb-76f51372f4cf","Type":"ContainerStarted","Data":"e92c1c10039bab8c6652714dd8b16592036beee195077e8a28302025ade68df6"} Jul 03 11:18:30.779068 ip-10-0-136-0 kubenswrapper[2579]: I0703 11:18:30.779032 2579 generic.go:356] "Generic (PLEG): container finished" podID="509683d2-efbb-42d5-98eb-76f51372f4cf" containerID="09106674412bf42d76fa86f3827724c983b6b37c137bdbfab461c4df0feb5019" exitCode=0 Jul 03 11:18:30.779527 ip-10-0-136-0 kubenswrapper[2579]: I0703 11:18:30.779122 2579 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-marketplace/629fd74b02459b8d9ee224b8bdbb513172290d80b07a5bf327840796afz4k7n" event={"ID":"509683d2-efbb-42d5-98eb-76f51372f4cf","Type":"ContainerDied","Data":"09106674412bf42d76fa86f3827724c983b6b37c137bdbfab461c4df0feb5019"} Jul 03 11:18:31.786730 ip-10-0-136-0 kubenswrapper[2579]: I0703 11:18:31.786689 2579 generic.go:356] "Generic (PLEG): container finished" podID="509683d2-efbb-42d5-98eb-76f51372f4cf" containerID="334cae8dbd1d0324e09b25e2ba6b8d6d0521bfe7f2f6fb052cac7c88d977a9a7" exitCode=0 Jul 03 11:18:31.787145 ip-10-0-136-0 kubenswrapper[2579]: I0703 11:18:31.786767 2579 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-marketplace/629fd74b02459b8d9ee224b8bdbb513172290d80b07a5bf327840796afz4k7n" event={"ID":"509683d2-efbb-42d5-98eb-76f51372f4cf","Type":"ContainerDied","Data":"334cae8dbd1d0324e09b25e2ba6b8d6d0521bfe7f2f6fb052cac7c88d977a9a7"} Jul 03 11:18:32.932479 ip-10-0-136-0 kubenswrapper[2579]: I0703 11:18:32.932440 2579 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-marketplace/629fd74b02459b8d9ee224b8bdbb513172290d80b07a5bf327840796afz4k7n" Jul 03 11:18:33.108259 ip-10-0-136-0 kubenswrapper[2579]: I0703 11:18:33.108130 2579 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"util\" (UniqueName: \"kubernetes.io/empty-dir/509683d2-efbb-42d5-98eb-76f51372f4cf-util\") pod \"509683d2-efbb-42d5-98eb-76f51372f4cf\" (UID: \"509683d2-efbb-42d5-98eb-76f51372f4cf\") " Jul 03 11:18:33.108440 ip-10-0-136-0 kubenswrapper[2579]: I0703 11:18:33.108265 2579 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"bundle\" (UniqueName: \"kubernetes.io/empty-dir/509683d2-efbb-42d5-98eb-76f51372f4cf-bundle\") pod \"509683d2-efbb-42d5-98eb-76f51372f4cf\" (UID: \"509683d2-efbb-42d5-98eb-76f51372f4cf\") " Jul 03 11:18:33.108440 ip-10-0-136-0 kubenswrapper[2579]: I0703 11:18:33.108342 2579 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-zbldc\" (UniqueName: \"kubernetes.io/projected/509683d2-efbb-42d5-98eb-76f51372f4cf-kube-api-access-zbldc\") pod \"509683d2-efbb-42d5-98eb-76f51372f4cf\" (UID: \"509683d2-efbb-42d5-98eb-76f51372f4cf\") " Jul 03 11:18:33.109139 ip-10-0-136-0 kubenswrapper[2579]: I0703 11:18:33.109014 2579 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/509683d2-efbb-42d5-98eb-76f51372f4cf-bundle" (OuterVolumeSpecName: "bundle") pod "509683d2-efbb-42d5-98eb-76f51372f4cf" (UID: "509683d2-efbb-42d5-98eb-76f51372f4cf"). InnerVolumeSpecName "bundle". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Jul 03 11:18:33.110830 ip-10-0-136-0 kubenswrapper[2579]: I0703 11:18:33.110797 2579 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/509683d2-efbb-42d5-98eb-76f51372f4cf-kube-api-access-zbldc" (OuterVolumeSpecName: "kube-api-access-zbldc") pod "509683d2-efbb-42d5-98eb-76f51372f4cf" (UID: "509683d2-efbb-42d5-98eb-76f51372f4cf"). InnerVolumeSpecName "kube-api-access-zbldc". PluginName "kubernetes.io/projected", VolumeGIDValue "" Jul 03 11:18:33.114468 ip-10-0-136-0 kubenswrapper[2579]: I0703 11:18:33.114429 2579 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/509683d2-efbb-42d5-98eb-76f51372f4cf-util" (OuterVolumeSpecName: "util") pod "509683d2-efbb-42d5-98eb-76f51372f4cf" (UID: "509683d2-efbb-42d5-98eb-76f51372f4cf"). InnerVolumeSpecName "util". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Jul 03 11:18:33.209785 ip-10-0-136-0 kubenswrapper[2579]: I0703 11:18:33.209736 2579 reconciler_common.go:299] "Volume detached for volume \"util\" (UniqueName: \"kubernetes.io/empty-dir/509683d2-efbb-42d5-98eb-76f51372f4cf-util\") on node \"ip-10-0-136-0.ec2.internal\" DevicePath \"\"" Jul 03 11:18:33.209785 ip-10-0-136-0 kubenswrapper[2579]: I0703 11:18:33.209782 2579 reconciler_common.go:299] "Volume detached for volume \"bundle\" (UniqueName: \"kubernetes.io/empty-dir/509683d2-efbb-42d5-98eb-76f51372f4cf-bundle\") on node \"ip-10-0-136-0.ec2.internal\" DevicePath \"\"" Jul 03 11:18:33.210014 ip-10-0-136-0 kubenswrapper[2579]: I0703 11:18:33.209796 2579 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-zbldc\" (UniqueName: \"kubernetes.io/projected/509683d2-efbb-42d5-98eb-76f51372f4cf-kube-api-access-zbldc\") on node \"ip-10-0-136-0.ec2.internal\" DevicePath \"\"" Jul 03 11:18:33.799234 ip-10-0-136-0 kubenswrapper[2579]: I0703 11:18:33.799200 2579 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-marketplace/629fd74b02459b8d9ee224b8bdbb513172290d80b07a5bf327840796afz4k7n" Jul 03 11:18:33.799234 ip-10-0-136-0 kubenswrapper[2579]: I0703 11:18:33.799185 2579 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-marketplace/629fd74b02459b8d9ee224b8bdbb513172290d80b07a5bf327840796afz4k7n" event={"ID":"509683d2-efbb-42d5-98eb-76f51372f4cf","Type":"ContainerDied","Data":"e92c1c10039bab8c6652714dd8b16592036beee195077e8a28302025ade68df6"} Jul 03 11:18:33.799452 ip-10-0-136-0 kubenswrapper[2579]: I0703 11:18:33.799252 2579 pod_container_deletor.go:80] "Container not found in pod's containers" containerID="e92c1c10039bab8c6652714dd8b16592036beee195077e8a28302025ade68df6" Jul 03 11:18:43.089940 ip-10-0-136-0 kubenswrapper[2579]: I0703 11:18:43.089903 2579 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["opendatahub/opendatahub-operator-controller-manager-b8c56cd69-ds56h"] Jul 03 11:18:43.090487 ip-10-0-136-0 kubenswrapper[2579]: I0703 11:18:43.090264 2579 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="509683d2-efbb-42d5-98eb-76f51372f4cf" containerName="pull" Jul 03 11:18:43.090487 ip-10-0-136-0 kubenswrapper[2579]: I0703 11:18:43.090279 2579 state_mem.go:107] "Deleted CPUSet assignment" podUID="509683d2-efbb-42d5-98eb-76f51372f4cf" containerName="pull" Jul 03 11:18:43.090487 ip-10-0-136-0 kubenswrapper[2579]: I0703 11:18:43.090288 2579 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="509683d2-efbb-42d5-98eb-76f51372f4cf" containerName="extract" Jul 03 11:18:43.090487 ip-10-0-136-0 kubenswrapper[2579]: I0703 11:18:43.090293 2579 state_mem.go:107] "Deleted CPUSet assignment" podUID="509683d2-efbb-42d5-98eb-76f51372f4cf" containerName="extract" Jul 03 11:18:43.090487 ip-10-0-136-0 kubenswrapper[2579]: I0703 11:18:43.090305 2579 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="509683d2-efbb-42d5-98eb-76f51372f4cf" containerName="util" Jul 03 11:18:43.090487 ip-10-0-136-0 kubenswrapper[2579]: I0703 11:18:43.090311 2579 state_mem.go:107] "Deleted CPUSet assignment" podUID="509683d2-efbb-42d5-98eb-76f51372f4cf" containerName="util" Jul 03 11:18:43.090487 ip-10-0-136-0 kubenswrapper[2579]: I0703 11:18:43.090425 2579 memory_manager.go:368] "RemoveStaleState removing state" podUID="509683d2-efbb-42d5-98eb-76f51372f4cf" containerName="extract" Jul 03 11:18:43.096428 ip-10-0-136-0 kubenswrapper[2579]: I0703 11:18:43.096396 2579 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="opendatahub/opendatahub-operator-controller-manager-b8c56cd69-ds56h" Jul 03 11:18:43.101345 ip-10-0-136-0 kubenswrapper[2579]: I0703 11:18:43.101318 2579 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"opendatahub\"/\"opendatahub-operator-controller-webhook-cert\"" Jul 03 11:18:43.101517 ip-10-0-136-0 kubenswrapper[2579]: I0703 11:18:43.101323 2579 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"opendatahub\"/\"kube-root-ca.crt\"" Jul 03 11:18:43.101936 ip-10-0-136-0 kubenswrapper[2579]: I0703 11:18:43.101914 2579 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"opendatahub\"/\"openshift-service-ca.crt\"" Jul 03 11:18:43.102047 ip-10-0-136-0 kubenswrapper[2579]: I0703 11:18:43.101940 2579 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"opendatahub\"/\"opendatahub-operator-controller-manager-service-cert\"" Jul 03 11:18:43.102047 ip-10-0-136-0 kubenswrapper[2579]: I0703 11:18:43.101924 2579 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"opendatahub\"/\"opendatahub-operator-controller-manager-dockercfg-nbwwk\"" Jul 03 11:18:43.122599 ip-10-0-136-0 kubenswrapper[2579]: I0703 11:18:43.122552 2579 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["opendatahub/opendatahub-operator-controller-manager-b8c56cd69-ds56h"] Jul 03 11:18:43.192553 ip-10-0-136-0 kubenswrapper[2579]: I0703 11:18:43.192516 2579 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"webhook-cert\" (UniqueName: \"kubernetes.io/secret/ff9ecf9d-c9e1-4807-88bc-c5773606274c-webhook-cert\") pod \"opendatahub-operator-controller-manager-b8c56cd69-ds56h\" (UID: \"ff9ecf9d-c9e1-4807-88bc-c5773606274c\") " pod="opendatahub/opendatahub-operator-controller-manager-b8c56cd69-ds56h" Jul 03 11:18:43.192736 ip-10-0-136-0 kubenswrapper[2579]: I0703 11:18:43.192571 2579 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tmp\" (UniqueName: \"kubernetes.io/empty-dir/ff9ecf9d-c9e1-4807-88bc-c5773606274c-tmp\") pod \"opendatahub-operator-controller-manager-b8c56cd69-ds56h\" (UID: \"ff9ecf9d-c9e1-4807-88bc-c5773606274c\") " pod="opendatahub/opendatahub-operator-controller-manager-b8c56cd69-ds56h" Jul 03 11:18:43.192736 ip-10-0-136-0 kubenswrapper[2579]: I0703 11:18:43.192593 2579 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"manifests\" (UniqueName: \"kubernetes.io/empty-dir/ff9ecf9d-c9e1-4807-88bc-c5773606274c-manifests\") pod \"opendatahub-operator-controller-manager-b8c56cd69-ds56h\" (UID: \"ff9ecf9d-c9e1-4807-88bc-c5773606274c\") " pod="opendatahub/opendatahub-operator-controller-manager-b8c56cd69-ds56h" Jul 03 11:18:43.192736 ip-10-0-136-0 kubenswrapper[2579]: I0703 11:18:43.192630 2579 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-24t67\" (UniqueName: \"kubernetes.io/projected/ff9ecf9d-c9e1-4807-88bc-c5773606274c-kube-api-access-24t67\") pod \"opendatahub-operator-controller-manager-b8c56cd69-ds56h\" (UID: \"ff9ecf9d-c9e1-4807-88bc-c5773606274c\") " pod="opendatahub/opendatahub-operator-controller-manager-b8c56cd69-ds56h" Jul 03 11:18:43.192736 ip-10-0-136-0 kubenswrapper[2579]: I0703 11:18:43.192653 2579 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"apiservice-cert\" (UniqueName: \"kubernetes.io/secret/ff9ecf9d-c9e1-4807-88bc-c5773606274c-apiservice-cert\") pod \"opendatahub-operator-controller-manager-b8c56cd69-ds56h\" (UID: \"ff9ecf9d-c9e1-4807-88bc-c5773606274c\") " pod="opendatahub/opendatahub-operator-controller-manager-b8c56cd69-ds56h" Jul 03 11:18:43.293560 ip-10-0-136-0 kubenswrapper[2579]: I0703 11:18:43.293512 2579 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-24t67\" (UniqueName: \"kubernetes.io/projected/ff9ecf9d-c9e1-4807-88bc-c5773606274c-kube-api-access-24t67\") pod \"opendatahub-operator-controller-manager-b8c56cd69-ds56h\" (UID: \"ff9ecf9d-c9e1-4807-88bc-c5773606274c\") " pod="opendatahub/opendatahub-operator-controller-manager-b8c56cd69-ds56h" Jul 03 11:18:43.293767 ip-10-0-136-0 kubenswrapper[2579]: I0703 11:18:43.293570 2579 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"apiservice-cert\" (UniqueName: \"kubernetes.io/secret/ff9ecf9d-c9e1-4807-88bc-c5773606274c-apiservice-cert\") pod \"opendatahub-operator-controller-manager-b8c56cd69-ds56h\" (UID: \"ff9ecf9d-c9e1-4807-88bc-c5773606274c\") " pod="opendatahub/opendatahub-operator-controller-manager-b8c56cd69-ds56h" Jul 03 11:18:43.294115 ip-10-0-136-0 kubenswrapper[2579]: I0703 11:18:43.293665 2579 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"webhook-cert\" (UniqueName: \"kubernetes.io/secret/ff9ecf9d-c9e1-4807-88bc-c5773606274c-webhook-cert\") pod \"opendatahub-operator-controller-manager-b8c56cd69-ds56h\" (UID: \"ff9ecf9d-c9e1-4807-88bc-c5773606274c\") " pod="opendatahub/opendatahub-operator-controller-manager-b8c56cd69-ds56h" Jul 03 11:18:43.294248 ip-10-0-136-0 kubenswrapper[2579]: I0703 11:18:43.294226 2579 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tmp\" (UniqueName: \"kubernetes.io/empty-dir/ff9ecf9d-c9e1-4807-88bc-c5773606274c-tmp\") pod \"opendatahub-operator-controller-manager-b8c56cd69-ds56h\" (UID: \"ff9ecf9d-c9e1-4807-88bc-c5773606274c\") " pod="opendatahub/opendatahub-operator-controller-manager-b8c56cd69-ds56h" Jul 03 11:18:43.294323 ip-10-0-136-0 kubenswrapper[2579]: I0703 11:18:43.294287 2579 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"manifests\" (UniqueName: \"kubernetes.io/empty-dir/ff9ecf9d-c9e1-4807-88bc-c5773606274c-manifests\") pod \"opendatahub-operator-controller-manager-b8c56cd69-ds56h\" (UID: \"ff9ecf9d-c9e1-4807-88bc-c5773606274c\") " pod="opendatahub/opendatahub-operator-controller-manager-b8c56cd69-ds56h" Jul 03 11:18:43.294812 ip-10-0-136-0 kubenswrapper[2579]: I0703 11:18:43.294786 2579 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"manifests\" (UniqueName: \"kubernetes.io/empty-dir/ff9ecf9d-c9e1-4807-88bc-c5773606274c-manifests\") pod \"opendatahub-operator-controller-manager-b8c56cd69-ds56h\" (UID: \"ff9ecf9d-c9e1-4807-88bc-c5773606274c\") " pod="opendatahub/opendatahub-operator-controller-manager-b8c56cd69-ds56h" Jul 03 11:18:43.298823 ip-10-0-136-0 kubenswrapper[2579]: I0703 11:18:43.297779 2579 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"apiservice-cert\" (UniqueName: \"kubernetes.io/secret/ff9ecf9d-c9e1-4807-88bc-c5773606274c-apiservice-cert\") pod \"opendatahub-operator-controller-manager-b8c56cd69-ds56h\" (UID: \"ff9ecf9d-c9e1-4807-88bc-c5773606274c\") " pod="opendatahub/opendatahub-operator-controller-manager-b8c56cd69-ds56h" Jul 03 11:18:43.300344 ip-10-0-136-0 kubenswrapper[2579]: I0703 11:18:43.300286 2579 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"webhook-cert\" (UniqueName: \"kubernetes.io/secret/ff9ecf9d-c9e1-4807-88bc-c5773606274c-webhook-cert\") pod \"opendatahub-operator-controller-manager-b8c56cd69-ds56h\" (UID: \"ff9ecf9d-c9e1-4807-88bc-c5773606274c\") " pod="opendatahub/opendatahub-operator-controller-manager-b8c56cd69-ds56h" Jul 03 11:18:43.302842 ip-10-0-136-0 kubenswrapper[2579]: I0703 11:18:43.302794 2579 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"tmp\" (UniqueName: \"kubernetes.io/empty-dir/ff9ecf9d-c9e1-4807-88bc-c5773606274c-tmp\") pod \"opendatahub-operator-controller-manager-b8c56cd69-ds56h\" (UID: \"ff9ecf9d-c9e1-4807-88bc-c5773606274c\") " pod="opendatahub/opendatahub-operator-controller-manager-b8c56cd69-ds56h" Jul 03 11:18:43.312403 ip-10-0-136-0 kubenswrapper[2579]: I0703 11:18:43.312366 2579 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-24t67\" (UniqueName: \"kubernetes.io/projected/ff9ecf9d-c9e1-4807-88bc-c5773606274c-kube-api-access-24t67\") pod \"opendatahub-operator-controller-manager-b8c56cd69-ds56h\" (UID: \"ff9ecf9d-c9e1-4807-88bc-c5773606274c\") " pod="opendatahub/opendatahub-operator-controller-manager-b8c56cd69-ds56h" Jul 03 11:18:43.407715 ip-10-0-136-0 kubenswrapper[2579]: I0703 11:18:43.407584 2579 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="opendatahub/opendatahub-operator-controller-manager-b8c56cd69-ds56h" Jul 03 11:18:43.602876 ip-10-0-136-0 kubenswrapper[2579]: I0703 11:18:43.602830 2579 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["opendatahub/opendatahub-operator-controller-manager-b8c56cd69-ds56h"] Jul 03 11:18:43.607045 ip-10-0-136-0 kubenswrapper[2579]: W0703 11:18:43.607006 2579 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podff9ecf9d_c9e1_4807_88bc_c5773606274c.slice/crio-d441e82cdb3fbb676dbdff848c8d950f73428ded26f04443f0c96712d0c556ad WatchSource:0}: Error finding container d441e82cdb3fbb676dbdff848c8d950f73428ded26f04443f0c96712d0c556ad: Status 404 returned error can't find the container with id d441e82cdb3fbb676dbdff848c8d950f73428ded26f04443f0c96712d0c556ad Jul 03 11:18:43.844891 ip-10-0-136-0 kubenswrapper[2579]: I0703 11:18:43.844845 2579 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="opendatahub/opendatahub-operator-controller-manager-b8c56cd69-ds56h" event={"ID":"ff9ecf9d-c9e1-4807-88bc-c5773606274c","Type":"ContainerStarted","Data":"d441e82cdb3fbb676dbdff848c8d950f73428ded26f04443f0c96712d0c556ad"} Jul 03 11:18:46.863940 ip-10-0-136-0 kubenswrapper[2579]: I0703 11:18:46.863897 2579 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="opendatahub/opendatahub-operator-controller-manager-b8c56cd69-ds56h" event={"ID":"ff9ecf9d-c9e1-4807-88bc-c5773606274c","Type":"ContainerStarted","Data":"7d43160f95cb23e8a58546af222548ea9153bc3cc12c0edf9db0a568a97f2189"} Jul 03 11:18:47.869988 ip-10-0-136-0 kubenswrapper[2579]: I0703 11:18:47.869950 2579 generic.go:356] "Generic (PLEG): container finished" podID="ff9ecf9d-c9e1-4807-88bc-c5773606274c" containerID="7d43160f95cb23e8a58546af222548ea9153bc3cc12c0edf9db0a568a97f2189" exitCode=0 Jul 03 11:18:47.870464 ip-10-0-136-0 kubenswrapper[2579]: I0703 11:18:47.870038 2579 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="opendatahub/opendatahub-operator-controller-manager-b8c56cd69-ds56h" event={"ID":"ff9ecf9d-c9e1-4807-88bc-c5773606274c","Type":"ContainerDied","Data":"7d43160f95cb23e8a58546af222548ea9153bc3cc12c0edf9db0a568a97f2189"} Jul 03 11:18:48.879133 ip-10-0-136-0 kubenswrapper[2579]: I0703 11:18:48.879065 2579 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="opendatahub/opendatahub-operator-controller-manager-b8c56cd69-ds56h" event={"ID":"ff9ecf9d-c9e1-4807-88bc-c5773606274c","Type":"ContainerStarted","Data":"e9360fdb07704ea6f376db0d09882bf17975555c61ee2cfac6ee8ea4ebf8c7f0"} Jul 03 11:18:48.879660 ip-10-0-136-0 kubenswrapper[2579]: I0703 11:18:48.879228 2579 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="opendatahub/opendatahub-operator-controller-manager-b8c56cd69-ds56h" Jul 03 11:18:48.910181 ip-10-0-136-0 kubenswrapper[2579]: I0703 11:18:48.910112 2579 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="opendatahub/opendatahub-operator-controller-manager-b8c56cd69-ds56h" podStartSLOduration=1.349657319 podStartE2EDuration="5.910094012s" totalImagesPullingTime="3.458566975s" totalInitContainerRuntime="1.101869718s" isStatefulPod=false podCreationTimestamp="2026-07-03 11:18:43 +0000 UTC" imagePullSessionsCount=2 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-03 11:18:48.908810058 +0000 UTC m=+385.497114390" watchObservedRunningTime="2026-07-03 11:18:48.910094012 +0000 UTC m=+385.498398331" Jul 03 11:18:59.886451 ip-10-0-136-0 kubenswrapper[2579]: I0703 11:18:59.886416 2579 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="opendatahub/opendatahub-operator-controller-manager-b8c56cd69-ds56h" Jul 03 11:19:03.160001 ip-10-0-136-0 kubenswrapper[2579]: I0703 11:19:03.159954 2579 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-ingress/kube-auth-proxy-5cb49b9d79-hl6j7"] Jul 03 11:19:03.169889 ip-10-0-136-0 kubenswrapper[2579]: I0703 11:19:03.169859 2579 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/kube-auth-proxy-5cb49b9d79-hl6j7" Jul 03 11:19:03.173563 ip-10-0-136-0 kubenswrapper[2579]: I0703 11:19:03.173524 2579 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ingress\"/\"kube-root-ca.crt\"" Jul 03 11:19:03.174714 ip-10-0-136-0 kubenswrapper[2579]: I0703 11:19:03.174656 2579 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress\"/\"kube-auth-proxy-dockercfg-wtv2j\"" Jul 03 11:19:03.174873 ip-10-0-136-0 kubenswrapper[2579]: I0703 11:19:03.174747 2579 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress\"/\"kube-auth-proxy-tls\"" Jul 03 11:19:03.174873 ip-10-0-136-0 kubenswrapper[2579]: I0703 11:19:03.174747 2579 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ingress\"/\"openshift-service-ca.crt\"" Jul 03 11:19:03.174873 ip-10-0-136-0 kubenswrapper[2579]: I0703 11:19:03.174756 2579 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress\"/\"kube-auth-proxy-creds\"" Jul 03 11:19:03.196682 ip-10-0-136-0 kubenswrapper[2579]: I0703 11:19:03.196638 2579 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/kube-auth-proxy-5cb49b9d79-hl6j7"] Jul 03 11:19:03.246177 ip-10-0-136-0 kubenswrapper[2579]: I0703 11:19:03.246125 2579 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["opendatahub/odh-model-controller-7998c67dc4-wsz76"] Jul 03 11:19:03.250291 ip-10-0-136-0 kubenswrapper[2579]: I0703 11:19:03.250257 2579 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="opendatahub/odh-model-controller-7998c67dc4-wsz76" Jul 03 11:19:03.253433 ip-10-0-136-0 kubenswrapper[2579]: I0703 11:19:03.253398 2579 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"opendatahub\"/\"odh-model-controller-webhook-cert\"" Jul 03 11:19:03.255408 ip-10-0-136-0 kubenswrapper[2579]: I0703 11:19:03.255381 2579 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"opendatahub\"/\"odh-model-controller-dockercfg-nzr46\"" Jul 03 11:19:03.265795 ip-10-0-136-0 kubenswrapper[2579]: I0703 11:19:03.265756 2579 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["opendatahub/odh-model-controller-7998c67dc4-wsz76"] Jul 03 11:19:03.270954 ip-10-0-136-0 kubenswrapper[2579]: I0703 11:19:03.270915 2579 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tmp\" (UniqueName: \"kubernetes.io/empty-dir/29ccc316-f283-4be9-b247-23d424fbdf1d-tmp\") pod \"kube-auth-proxy-5cb49b9d79-hl6j7\" (UID: \"29ccc316-f283-4be9-b247-23d424fbdf1d\") " pod="openshift-ingress/kube-auth-proxy-5cb49b9d79-hl6j7" Jul 03 11:19:03.271153 ip-10-0-136-0 kubenswrapper[2579]: I0703 11:19:03.270962 2579 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/29ccc316-f283-4be9-b247-23d424fbdf1d-tls-certs\") pod \"kube-auth-proxy-5cb49b9d79-hl6j7\" (UID: \"29ccc316-f283-4be9-b247-23d424fbdf1d\") " pod="openshift-ingress/kube-auth-proxy-5cb49b9d79-hl6j7" Jul 03 11:19:03.271153 ip-10-0-136-0 kubenswrapper[2579]: I0703 11:19:03.270990 2579 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-gqr4v\" (UniqueName: \"kubernetes.io/projected/29ccc316-f283-4be9-b247-23d424fbdf1d-kube-api-access-gqr4v\") pod \"kube-auth-proxy-5cb49b9d79-hl6j7\" (UID: \"29ccc316-f283-4be9-b247-23d424fbdf1d\") " pod="openshift-ingress/kube-auth-proxy-5cb49b9d79-hl6j7" Jul 03 11:19:03.372486 ip-10-0-136-0 kubenswrapper[2579]: I0703 11:19:03.372441 2579 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"cert\" (UniqueName: \"kubernetes.io/secret/4fcd4587-9ab7-43ce-92f6-620f50b11af4-cert\") pod \"odh-model-controller-7998c67dc4-wsz76\" (UID: \"4fcd4587-9ab7-43ce-92f6-620f50b11af4\") " pod="opendatahub/odh-model-controller-7998c67dc4-wsz76" Jul 03 11:19:03.372718 ip-10-0-136-0 kubenswrapper[2579]: I0703 11:19:03.372540 2579 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tmp\" (UniqueName: \"kubernetes.io/empty-dir/29ccc316-f283-4be9-b247-23d424fbdf1d-tmp\") pod \"kube-auth-proxy-5cb49b9d79-hl6j7\" (UID: \"29ccc316-f283-4be9-b247-23d424fbdf1d\") " pod="openshift-ingress/kube-auth-proxy-5cb49b9d79-hl6j7" Jul 03 11:19:03.372718 ip-10-0-136-0 kubenswrapper[2579]: I0703 11:19:03.372571 2579 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/29ccc316-f283-4be9-b247-23d424fbdf1d-tls-certs\") pod \"kube-auth-proxy-5cb49b9d79-hl6j7\" (UID: \"29ccc316-f283-4be9-b247-23d424fbdf1d\") " pod="openshift-ingress/kube-auth-proxy-5cb49b9d79-hl6j7" Jul 03 11:19:03.372718 ip-10-0-136-0 kubenswrapper[2579]: I0703 11:19:03.372588 2579 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-gqr4v\" (UniqueName: \"kubernetes.io/projected/29ccc316-f283-4be9-b247-23d424fbdf1d-kube-api-access-gqr4v\") pod \"kube-auth-proxy-5cb49b9d79-hl6j7\" (UID: \"29ccc316-f283-4be9-b247-23d424fbdf1d\") " pod="openshift-ingress/kube-auth-proxy-5cb49b9d79-hl6j7" Jul 03 11:19:03.372718 ip-10-0-136-0 kubenswrapper[2579]: I0703 11:19:03.372629 2579 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-nl8s2\" (UniqueName: \"kubernetes.io/projected/4fcd4587-9ab7-43ce-92f6-620f50b11af4-kube-api-access-nl8s2\") pod \"odh-model-controller-7998c67dc4-wsz76\" (UID: \"4fcd4587-9ab7-43ce-92f6-620f50b11af4\") " pod="opendatahub/odh-model-controller-7998c67dc4-wsz76" Jul 03 11:19:03.375229 ip-10-0-136-0 kubenswrapper[2579]: I0703 11:19:03.375170 2579 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"tmp\" (UniqueName: \"kubernetes.io/empty-dir/29ccc316-f283-4be9-b247-23d424fbdf1d-tmp\") pod \"kube-auth-proxy-5cb49b9d79-hl6j7\" (UID: \"29ccc316-f283-4be9-b247-23d424fbdf1d\") " pod="openshift-ingress/kube-auth-proxy-5cb49b9d79-hl6j7" Jul 03 11:19:03.375559 ip-10-0-136-0 kubenswrapper[2579]: I0703 11:19:03.375538 2579 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/29ccc316-f283-4be9-b247-23d424fbdf1d-tls-certs\") pod \"kube-auth-proxy-5cb49b9d79-hl6j7\" (UID: \"29ccc316-f283-4be9-b247-23d424fbdf1d\") " pod="openshift-ingress/kube-auth-proxy-5cb49b9d79-hl6j7" Jul 03 11:19:03.387340 ip-10-0-136-0 kubenswrapper[2579]: I0703 11:19:03.387260 2579 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-gqr4v\" (UniqueName: \"kubernetes.io/projected/29ccc316-f283-4be9-b247-23d424fbdf1d-kube-api-access-gqr4v\") pod \"kube-auth-proxy-5cb49b9d79-hl6j7\" (UID: \"29ccc316-f283-4be9-b247-23d424fbdf1d\") " pod="openshift-ingress/kube-auth-proxy-5cb49b9d79-hl6j7" Jul 03 11:19:03.474140 ip-10-0-136-0 kubenswrapper[2579]: I0703 11:19:03.474023 2579 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-nl8s2\" (UniqueName: \"kubernetes.io/projected/4fcd4587-9ab7-43ce-92f6-620f50b11af4-kube-api-access-nl8s2\") pod \"odh-model-controller-7998c67dc4-wsz76\" (UID: \"4fcd4587-9ab7-43ce-92f6-620f50b11af4\") " pod="opendatahub/odh-model-controller-7998c67dc4-wsz76" Jul 03 11:19:03.474369 ip-10-0-136-0 kubenswrapper[2579]: I0703 11:19:03.474166 2579 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"cert\" (UniqueName: \"kubernetes.io/secret/4fcd4587-9ab7-43ce-92f6-620f50b11af4-cert\") pod \"odh-model-controller-7998c67dc4-wsz76\" (UID: \"4fcd4587-9ab7-43ce-92f6-620f50b11af4\") " pod="opendatahub/odh-model-controller-7998c67dc4-wsz76" Jul 03 11:19:03.474369 ip-10-0-136-0 kubenswrapper[2579]: E0703 11:19:03.474339 2579 secret.go:189] Couldn't get secret opendatahub/odh-model-controller-webhook-cert: secret "odh-model-controller-webhook-cert" not found Jul 03 11:19:03.474484 ip-10-0-136-0 kubenswrapper[2579]: E0703 11:19:03.474434 2579 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/4fcd4587-9ab7-43ce-92f6-620f50b11af4-cert podName:4fcd4587-9ab7-43ce-92f6-620f50b11af4 nodeName:}" failed. No retries permitted until 2026-07-03 11:19:03.974411664 +0000 UTC m=+400.562715964 (durationBeforeRetry 500ms). Error: MountVolume.SetUp failed for volume "cert" (UniqueName: "kubernetes.io/secret/4fcd4587-9ab7-43ce-92f6-620f50b11af4-cert") pod "odh-model-controller-7998c67dc4-wsz76" (UID: "4fcd4587-9ab7-43ce-92f6-620f50b11af4") : secret "odh-model-controller-webhook-cert" not found Jul 03 11:19:03.480867 ip-10-0-136-0 kubenswrapper[2579]: I0703 11:19:03.480827 2579 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/kube-auth-proxy-5cb49b9d79-hl6j7" Jul 03 11:19:03.489216 ip-10-0-136-0 kubenswrapper[2579]: I0703 11:19:03.489158 2579 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-nl8s2\" (UniqueName: \"kubernetes.io/projected/4fcd4587-9ab7-43ce-92f6-620f50b11af4-kube-api-access-nl8s2\") pod \"odh-model-controller-7998c67dc4-wsz76\" (UID: \"4fcd4587-9ab7-43ce-92f6-620f50b11af4\") " pod="opendatahub/odh-model-controller-7998c67dc4-wsz76" Jul 03 11:19:03.640570 ip-10-0-136-0 kubenswrapper[2579]: I0703 11:19:03.640543 2579 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/kube-auth-proxy-5cb49b9d79-hl6j7"] Jul 03 11:19:03.642573 ip-10-0-136-0 kubenswrapper[2579]: W0703 11:19:03.642547 2579 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod29ccc316_f283_4be9_b247_23d424fbdf1d.slice/crio-e890ee7f24a0e879fd717d9b92420caa5278057f6287a452d0c9bcb7dd6c468d WatchSource:0}: Error finding container e890ee7f24a0e879fd717d9b92420caa5278057f6287a452d0c9bcb7dd6c468d: Status 404 returned error can't find the container with id e890ee7f24a0e879fd717d9b92420caa5278057f6287a452d0c9bcb7dd6c468d Jul 03 11:19:03.958796 ip-10-0-136-0 kubenswrapper[2579]: I0703 11:19:03.958752 2579 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/kube-auth-proxy-5cb49b9d79-hl6j7" event={"ID":"29ccc316-f283-4be9-b247-23d424fbdf1d","Type":"ContainerStarted","Data":"e890ee7f24a0e879fd717d9b92420caa5278057f6287a452d0c9bcb7dd6c468d"} Jul 03 11:19:03.978738 ip-10-0-136-0 kubenswrapper[2579]: I0703 11:19:03.978694 2579 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"cert\" (UniqueName: \"kubernetes.io/secret/4fcd4587-9ab7-43ce-92f6-620f50b11af4-cert\") pod \"odh-model-controller-7998c67dc4-wsz76\" (UID: \"4fcd4587-9ab7-43ce-92f6-620f50b11af4\") " pod="opendatahub/odh-model-controller-7998c67dc4-wsz76" Jul 03 11:19:03.981925 ip-10-0-136-0 kubenswrapper[2579]: I0703 11:19:03.981891 2579 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"cert\" (UniqueName: \"kubernetes.io/secret/4fcd4587-9ab7-43ce-92f6-620f50b11af4-cert\") pod \"odh-model-controller-7998c67dc4-wsz76\" (UID: \"4fcd4587-9ab7-43ce-92f6-620f50b11af4\") " pod="opendatahub/odh-model-controller-7998c67dc4-wsz76" Jul 03 11:19:04.087643 ip-10-0-136-0 kubenswrapper[2579]: I0703 11:19:04.087598 2579 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["opendatahub/model-serving-api-54d89bd5fc-djwk2"] Jul 03 11:19:04.092402 ip-10-0-136-0 kubenswrapper[2579]: I0703 11:19:04.092369 2579 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="opendatahub/model-serving-api-54d89bd5fc-djwk2" Jul 03 11:19:04.104073 ip-10-0-136-0 kubenswrapper[2579]: I0703 11:19:04.104037 2579 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"opendatahub\"/\"model-serving-api-tls\"" Jul 03 11:19:04.124044 ip-10-0-136-0 kubenswrapper[2579]: I0703 11:19:04.124001 2579 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["opendatahub/model-serving-api-54d89bd5fc-djwk2"] Jul 03 11:19:04.162130 ip-10-0-136-0 kubenswrapper[2579]: I0703 11:19:04.162082 2579 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="opendatahub/odh-model-controller-7998c67dc4-wsz76" Jul 03 11:19:04.180822 ip-10-0-136-0 kubenswrapper[2579]: I0703 11:19:04.180774 2579 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/bd1ed4fa-6517-40ea-aef4-14b015e858ab-tls-certs\") pod \"model-serving-api-54d89bd5fc-djwk2\" (UID: \"bd1ed4fa-6517-40ea-aef4-14b015e858ab\") " pod="opendatahub/model-serving-api-54d89bd5fc-djwk2" Jul 03 11:19:04.181017 ip-10-0-136-0 kubenswrapper[2579]: I0703 11:19:04.180833 2579 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-vg85j\" (UniqueName: \"kubernetes.io/projected/bd1ed4fa-6517-40ea-aef4-14b015e858ab-kube-api-access-vg85j\") pod \"model-serving-api-54d89bd5fc-djwk2\" (UID: \"bd1ed4fa-6517-40ea-aef4-14b015e858ab\") " pod="opendatahub/model-serving-api-54d89bd5fc-djwk2" Jul 03 11:19:04.281462 ip-10-0-136-0 kubenswrapper[2579]: I0703 11:19:04.281422 2579 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/bd1ed4fa-6517-40ea-aef4-14b015e858ab-tls-certs\") pod \"model-serving-api-54d89bd5fc-djwk2\" (UID: \"bd1ed4fa-6517-40ea-aef4-14b015e858ab\") " pod="opendatahub/model-serving-api-54d89bd5fc-djwk2" Jul 03 11:19:04.281703 ip-10-0-136-0 kubenswrapper[2579]: I0703 11:19:04.281496 2579 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-vg85j\" (UniqueName: \"kubernetes.io/projected/bd1ed4fa-6517-40ea-aef4-14b015e858ab-kube-api-access-vg85j\") pod \"model-serving-api-54d89bd5fc-djwk2\" (UID: \"bd1ed4fa-6517-40ea-aef4-14b015e858ab\") " pod="opendatahub/model-serving-api-54d89bd5fc-djwk2" Jul 03 11:19:04.281703 ip-10-0-136-0 kubenswrapper[2579]: E0703 11:19:04.281625 2579 secret.go:189] Couldn't get secret opendatahub/model-serving-api-tls: secret "model-serving-api-tls" not found Jul 03 11:19:04.281842 ip-10-0-136-0 kubenswrapper[2579]: E0703 11:19:04.281720 2579 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/bd1ed4fa-6517-40ea-aef4-14b015e858ab-tls-certs podName:bd1ed4fa-6517-40ea-aef4-14b015e858ab nodeName:}" failed. No retries permitted until 2026-07-03 11:19:04.781696956 +0000 UTC m=+401.370001255 (durationBeforeRetry 500ms). Error: MountVolume.SetUp failed for volume "tls-certs" (UniqueName: "kubernetes.io/secret/bd1ed4fa-6517-40ea-aef4-14b015e858ab-tls-certs") pod "model-serving-api-54d89bd5fc-djwk2" (UID: "bd1ed4fa-6517-40ea-aef4-14b015e858ab") : secret "model-serving-api-tls" not found Jul 03 11:19:04.306131 ip-10-0-136-0 kubenswrapper[2579]: I0703 11:19:04.305246 2579 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-vg85j\" (UniqueName: \"kubernetes.io/projected/bd1ed4fa-6517-40ea-aef4-14b015e858ab-kube-api-access-vg85j\") pod \"model-serving-api-54d89bd5fc-djwk2\" (UID: \"bd1ed4fa-6517-40ea-aef4-14b015e858ab\") " pod="opendatahub/model-serving-api-54d89bd5fc-djwk2" Jul 03 11:19:04.363524 ip-10-0-136-0 kubenswrapper[2579]: I0703 11:19:04.363346 2579 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["opendatahub/odh-model-controller-7998c67dc4-wsz76"] Jul 03 11:19:04.367286 ip-10-0-136-0 kubenswrapper[2579]: W0703 11:19:04.367231 2579 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod4fcd4587_9ab7_43ce_92f6_620f50b11af4.slice/crio-9980a94c79732fb0caeb173b5554ea96ffc9c21d0ea605a3d71b6d53b77d1cc2 WatchSource:0}: Error finding container 9980a94c79732fb0caeb173b5554ea96ffc9c21d0ea605a3d71b6d53b77d1cc2: Status 404 returned error can't find the container with id 9980a94c79732fb0caeb173b5554ea96ffc9c21d0ea605a3d71b6d53b77d1cc2 Jul 03 11:19:04.787985 ip-10-0-136-0 kubenswrapper[2579]: I0703 11:19:04.787910 2579 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/bd1ed4fa-6517-40ea-aef4-14b015e858ab-tls-certs\") pod \"model-serving-api-54d89bd5fc-djwk2\" (UID: \"bd1ed4fa-6517-40ea-aef4-14b015e858ab\") " pod="opendatahub/model-serving-api-54d89bd5fc-djwk2" Jul 03 11:19:04.792331 ip-10-0-136-0 kubenswrapper[2579]: I0703 11:19:04.792284 2579 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/bd1ed4fa-6517-40ea-aef4-14b015e858ab-tls-certs\") pod \"model-serving-api-54d89bd5fc-djwk2\" (UID: \"bd1ed4fa-6517-40ea-aef4-14b015e858ab\") " pod="opendatahub/model-serving-api-54d89bd5fc-djwk2" Jul 03 11:19:04.966635 ip-10-0-136-0 kubenswrapper[2579]: I0703 11:19:04.966559 2579 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="opendatahub/odh-model-controller-7998c67dc4-wsz76" event={"ID":"4fcd4587-9ab7-43ce-92f6-620f50b11af4","Type":"ContainerStarted","Data":"9980a94c79732fb0caeb173b5554ea96ffc9c21d0ea605a3d71b6d53b77d1cc2"} Jul 03 11:19:05.007635 ip-10-0-136-0 kubenswrapper[2579]: I0703 11:19:05.007574 2579 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="opendatahub/model-serving-api-54d89bd5fc-djwk2" Jul 03 11:19:05.164247 ip-10-0-136-0 kubenswrapper[2579]: I0703 11:19:05.164218 2579 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["opendatahub/model-serving-api-54d89bd5fc-djwk2"] Jul 03 11:19:05.166743 ip-10-0-136-0 kubenswrapper[2579]: W0703 11:19:05.166707 2579 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podbd1ed4fa_6517_40ea_aef4_14b015e858ab.slice/crio-5b5d64cf3461a239eb1098c478f3bccd148c3fa3a5fec41cf9727944f7d44f00 WatchSource:0}: Error finding container 5b5d64cf3461a239eb1098c478f3bccd148c3fa3a5fec41cf9727944f7d44f00: Status 404 returned error can't find the container with id 5b5d64cf3461a239eb1098c478f3bccd148c3fa3a5fec41cf9727944f7d44f00 Jul 03 11:19:05.376649 ip-10-0-136-0 kubenswrapper[2579]: I0703 11:19:05.376546 2579 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-ingress/istiod-openshift-gateway-7574459656-m7vpm"] Jul 03 11:19:05.380609 ip-10-0-136-0 kubenswrapper[2579]: I0703 11:19:05.380575 2579 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/istiod-openshift-gateway-7574459656-m7vpm" Jul 03 11:19:05.385949 ip-10-0-136-0 kubenswrapper[2579]: I0703 11:19:05.385903 2579 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress\"/\"cacerts\"" Jul 03 11:19:05.386375 ip-10-0-136-0 kubenswrapper[2579]: I0703 11:19:05.386354 2579 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress\"/\"istiod-openshift-gateway-dockercfg-r5vhc\"" Jul 03 11:19:05.386662 ip-10-0-136-0 kubenswrapper[2579]: I0703 11:19:05.386636 2579 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress\"/\"istiod-tls\"" Jul 03 11:19:05.386934 ip-10-0-136-0 kubenswrapper[2579]: I0703 11:19:05.386911 2579 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress\"/\"istio-kubeconfig\"" Jul 03 11:19:05.386934 ip-10-0-136-0 kubenswrapper[2579]: I0703 11:19:05.386925 2579 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ingress\"/\"openshift-gw-ca-root-cert\"" Jul 03 11:19:05.406728 ip-10-0-136-0 kubenswrapper[2579]: I0703 11:19:05.406691 2579 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/istiod-openshift-gateway-7574459656-m7vpm"] Jul 03 11:19:05.496273 ip-10-0-136-0 kubenswrapper[2579]: I0703 11:19:05.495815 2579 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-csr-ca-configmap\" (UniqueName: \"kubernetes.io/configmap/05548561-66e3-4667-bffc-da5b0b45237b-istio-csr-ca-configmap\") pod \"istiod-openshift-gateway-7574459656-m7vpm\" (UID: \"05548561-66e3-4667-bffc-da5b0b45237b\") " pod="openshift-ingress/istiod-openshift-gateway-7574459656-m7vpm" Jul 03 11:19:05.496273 ip-10-0-136-0 kubenswrapper[2579]: I0703 11:19:05.495871 2579 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"local-certs\" (UniqueName: \"kubernetes.io/empty-dir/05548561-66e3-4667-bffc-da5b0b45237b-local-certs\") pod \"istiod-openshift-gateway-7574459656-m7vpm\" (UID: \"05548561-66e3-4667-bffc-da5b0b45237b\") " pod="openshift-ingress/istiod-openshift-gateway-7574459656-m7vpm" Jul 03 11:19:05.496273 ip-10-0-136-0 kubenswrapper[2579]: I0703 11:19:05.495932 2579 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"cacerts\" (UniqueName: \"kubernetes.io/secret/05548561-66e3-4667-bffc-da5b0b45237b-cacerts\") pod \"istiod-openshift-gateway-7574459656-m7vpm\" (UID: \"05548561-66e3-4667-bffc-da5b0b45237b\") " pod="openshift-ingress/istiod-openshift-gateway-7574459656-m7vpm" Jul 03 11:19:05.496273 ip-10-0-136-0 kubenswrapper[2579]: I0703 11:19:05.495984 2579 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/05548561-66e3-4667-bffc-da5b0b45237b-istio-token\") pod \"istiod-openshift-gateway-7574459656-m7vpm\" (UID: \"05548561-66e3-4667-bffc-da5b0b45237b\") " pod="openshift-ingress/istiod-openshift-gateway-7574459656-m7vpm" Jul 03 11:19:05.496273 ip-10-0-136-0 kubenswrapper[2579]: I0703 11:19:05.496020 2579 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-kubeconfig\" (UniqueName: \"kubernetes.io/secret/05548561-66e3-4667-bffc-da5b0b45237b-istio-kubeconfig\") pod \"istiod-openshift-gateway-7574459656-m7vpm\" (UID: \"05548561-66e3-4667-bffc-da5b0b45237b\") " pod="openshift-ingress/istiod-openshift-gateway-7574459656-m7vpm" Jul 03 11:19:05.496273 ip-10-0-136-0 kubenswrapper[2579]: I0703 11:19:05.496055 2579 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-csr-dns-cert\" (UniqueName: \"kubernetes.io/secret/05548561-66e3-4667-bffc-da5b0b45237b-istio-csr-dns-cert\") pod \"istiod-openshift-gateway-7574459656-m7vpm\" (UID: \"05548561-66e3-4667-bffc-da5b0b45237b\") " pod="openshift-ingress/istiod-openshift-gateway-7574459656-m7vpm" Jul 03 11:19:05.496273 ip-10-0-136-0 kubenswrapper[2579]: I0703 11:19:05.496122 2579 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-b775n\" (UniqueName: \"kubernetes.io/projected/05548561-66e3-4667-bffc-da5b0b45237b-kube-api-access-b775n\") pod \"istiod-openshift-gateway-7574459656-m7vpm\" (UID: \"05548561-66e3-4667-bffc-da5b0b45237b\") " pod="openshift-ingress/istiod-openshift-gateway-7574459656-m7vpm" Jul 03 11:19:05.598220 ip-10-0-136-0 kubenswrapper[2579]: I0703 11:19:05.597162 2579 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"cacerts\" (UniqueName: \"kubernetes.io/secret/05548561-66e3-4667-bffc-da5b0b45237b-cacerts\") pod \"istiod-openshift-gateway-7574459656-m7vpm\" (UID: \"05548561-66e3-4667-bffc-da5b0b45237b\") " pod="openshift-ingress/istiod-openshift-gateway-7574459656-m7vpm" Jul 03 11:19:05.598220 ip-10-0-136-0 kubenswrapper[2579]: I0703 11:19:05.597273 2579 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/05548561-66e3-4667-bffc-da5b0b45237b-istio-token\") pod \"istiod-openshift-gateway-7574459656-m7vpm\" (UID: \"05548561-66e3-4667-bffc-da5b0b45237b\") " pod="openshift-ingress/istiod-openshift-gateway-7574459656-m7vpm" Jul 03 11:19:05.598220 ip-10-0-136-0 kubenswrapper[2579]: I0703 11:19:05.597316 2579 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-kubeconfig\" (UniqueName: \"kubernetes.io/secret/05548561-66e3-4667-bffc-da5b0b45237b-istio-kubeconfig\") pod \"istiod-openshift-gateway-7574459656-m7vpm\" (UID: \"05548561-66e3-4667-bffc-da5b0b45237b\") " pod="openshift-ingress/istiod-openshift-gateway-7574459656-m7vpm" Jul 03 11:19:05.598220 ip-10-0-136-0 kubenswrapper[2579]: I0703 11:19:05.597351 2579 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-csr-dns-cert\" (UniqueName: \"kubernetes.io/secret/05548561-66e3-4667-bffc-da5b0b45237b-istio-csr-dns-cert\") pod \"istiod-openshift-gateway-7574459656-m7vpm\" (UID: \"05548561-66e3-4667-bffc-da5b0b45237b\") " pod="openshift-ingress/istiod-openshift-gateway-7574459656-m7vpm" Jul 03 11:19:05.598220 ip-10-0-136-0 kubenswrapper[2579]: I0703 11:19:05.597424 2579 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-b775n\" (UniqueName: \"kubernetes.io/projected/05548561-66e3-4667-bffc-da5b0b45237b-kube-api-access-b775n\") pod \"istiod-openshift-gateway-7574459656-m7vpm\" (UID: \"05548561-66e3-4667-bffc-da5b0b45237b\") " pod="openshift-ingress/istiod-openshift-gateway-7574459656-m7vpm" Jul 03 11:19:05.598220 ip-10-0-136-0 kubenswrapper[2579]: I0703 11:19:05.597467 2579 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-csr-ca-configmap\" (UniqueName: \"kubernetes.io/configmap/05548561-66e3-4667-bffc-da5b0b45237b-istio-csr-ca-configmap\") pod \"istiod-openshift-gateway-7574459656-m7vpm\" (UID: \"05548561-66e3-4667-bffc-da5b0b45237b\") " pod="openshift-ingress/istiod-openshift-gateway-7574459656-m7vpm" Jul 03 11:19:05.598220 ip-10-0-136-0 kubenswrapper[2579]: I0703 11:19:05.597504 2579 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"local-certs\" (UniqueName: \"kubernetes.io/empty-dir/05548561-66e3-4667-bffc-da5b0b45237b-local-certs\") pod \"istiod-openshift-gateway-7574459656-m7vpm\" (UID: \"05548561-66e3-4667-bffc-da5b0b45237b\") " pod="openshift-ingress/istiod-openshift-gateway-7574459656-m7vpm" Jul 03 11:19:05.600335 ip-10-0-136-0 kubenswrapper[2579]: I0703 11:19:05.599583 2579 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istio-csr-ca-configmap\" (UniqueName: \"kubernetes.io/configmap/05548561-66e3-4667-bffc-da5b0b45237b-istio-csr-ca-configmap\") pod \"istiod-openshift-gateway-7574459656-m7vpm\" (UID: \"05548561-66e3-4667-bffc-da5b0b45237b\") " pod="openshift-ingress/istiod-openshift-gateway-7574459656-m7vpm" Jul 03 11:19:05.601632 ip-10-0-136-0 kubenswrapper[2579]: I0703 11:19:05.601597 2579 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"local-certs\" (UniqueName: \"kubernetes.io/empty-dir/05548561-66e3-4667-bffc-da5b0b45237b-local-certs\") pod \"istiod-openshift-gateway-7574459656-m7vpm\" (UID: \"05548561-66e3-4667-bffc-da5b0b45237b\") " pod="openshift-ingress/istiod-openshift-gateway-7574459656-m7vpm" Jul 03 11:19:05.603142 ip-10-0-136-0 kubenswrapper[2579]: I0703 11:19:05.603112 2579 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istio-kubeconfig\" (UniqueName: \"kubernetes.io/secret/05548561-66e3-4667-bffc-da5b0b45237b-istio-kubeconfig\") pod \"istiod-openshift-gateway-7574459656-m7vpm\" (UID: \"05548561-66e3-4667-bffc-da5b0b45237b\") " pod="openshift-ingress/istiod-openshift-gateway-7574459656-m7vpm" Jul 03 11:19:05.604875 ip-10-0-136-0 kubenswrapper[2579]: I0703 11:19:05.604846 2579 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istio-csr-dns-cert\" (UniqueName: \"kubernetes.io/secret/05548561-66e3-4667-bffc-da5b0b45237b-istio-csr-dns-cert\") pod \"istiod-openshift-gateway-7574459656-m7vpm\" (UID: \"05548561-66e3-4667-bffc-da5b0b45237b\") " pod="openshift-ingress/istiod-openshift-gateway-7574459656-m7vpm" Jul 03 11:19:05.608902 ip-10-0-136-0 kubenswrapper[2579]: I0703 11:19:05.608779 2579 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"cacerts\" (UniqueName: \"kubernetes.io/secret/05548561-66e3-4667-bffc-da5b0b45237b-cacerts\") pod \"istiod-openshift-gateway-7574459656-m7vpm\" (UID: \"05548561-66e3-4667-bffc-da5b0b45237b\") " pod="openshift-ingress/istiod-openshift-gateway-7574459656-m7vpm" Jul 03 11:19:05.617731 ip-10-0-136-0 kubenswrapper[2579]: I0703 11:19:05.617646 2579 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-b775n\" (UniqueName: \"kubernetes.io/projected/05548561-66e3-4667-bffc-da5b0b45237b-kube-api-access-b775n\") pod \"istiod-openshift-gateway-7574459656-m7vpm\" (UID: \"05548561-66e3-4667-bffc-da5b0b45237b\") " pod="openshift-ingress/istiod-openshift-gateway-7574459656-m7vpm" Jul 03 11:19:05.620234 ip-10-0-136-0 kubenswrapper[2579]: I0703 11:19:05.620123 2579 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/05548561-66e3-4667-bffc-da5b0b45237b-istio-token\") pod \"istiod-openshift-gateway-7574459656-m7vpm\" (UID: \"05548561-66e3-4667-bffc-da5b0b45237b\") " pod="openshift-ingress/istiod-openshift-gateway-7574459656-m7vpm" Jul 03 11:19:05.700896 ip-10-0-136-0 kubenswrapper[2579]: I0703 11:19:05.700794 2579 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/istiod-openshift-gateway-7574459656-m7vpm" Jul 03 11:19:05.946471 ip-10-0-136-0 kubenswrapper[2579]: I0703 11:19:05.946433 2579 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/istiod-openshift-gateway-7574459656-m7vpm"] Jul 03 11:19:05.949781 ip-10-0-136-0 kubenswrapper[2579]: W0703 11:19:05.949717 2579 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod05548561_66e3_4667_bffc_da5b0b45237b.slice/crio-379875108a5e0c09b1cc4bfc669c5aa8b8fed81ac393b8413532b3d70f5cbbf6 WatchSource:0}: Error finding container 379875108a5e0c09b1cc4bfc669c5aa8b8fed81ac393b8413532b3d70f5cbbf6: Status 404 returned error can't find the container with id 379875108a5e0c09b1cc4bfc669c5aa8b8fed81ac393b8413532b3d70f5cbbf6 Jul 03 11:19:05.977485 ip-10-0-136-0 kubenswrapper[2579]: I0703 11:19:05.977427 2579 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/istiod-openshift-gateway-7574459656-m7vpm" event={"ID":"05548561-66e3-4667-bffc-da5b0b45237b","Type":"ContainerStarted","Data":"379875108a5e0c09b1cc4bfc669c5aa8b8fed81ac393b8413532b3d70f5cbbf6"} Jul 03 11:19:05.979557 ip-10-0-136-0 kubenswrapper[2579]: I0703 11:19:05.979509 2579 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="opendatahub/model-serving-api-54d89bd5fc-djwk2" event={"ID":"bd1ed4fa-6517-40ea-aef4-14b015e858ab","Type":"ContainerStarted","Data":"5b5d64cf3461a239eb1098c478f3bccd148c3fa3a5fec41cf9727944f7d44f00"} Jul 03 11:19:09.001506 ip-10-0-136-0 kubenswrapper[2579]: I0703 11:19:09.001464 2579 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/kube-auth-proxy-5cb49b9d79-hl6j7" event={"ID":"29ccc316-f283-4be9-b247-23d424fbdf1d","Type":"ContainerStarted","Data":"11a2ef94cec7b1cf437dba1798b6cd3484f72467dca0b787aa789f7ba9854e23"} Jul 03 11:19:09.003271 ip-10-0-136-0 kubenswrapper[2579]: I0703 11:19:09.003236 2579 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="opendatahub/odh-model-controller-7998c67dc4-wsz76" event={"ID":"4fcd4587-9ab7-43ce-92f6-620f50b11af4","Type":"ContainerStarted","Data":"983dc519b3d3368ae202746db65aaf9e2e381b4bf41cace65b865a8cbf985756"} Jul 03 11:19:09.003460 ip-10-0-136-0 kubenswrapper[2579]: I0703 11:19:09.003312 2579 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="opendatahub/odh-model-controller-7998c67dc4-wsz76" Jul 03 11:19:09.005037 ip-10-0-136-0 kubenswrapper[2579]: I0703 11:19:09.005008 2579 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="opendatahub/model-serving-api-54d89bd5fc-djwk2" event={"ID":"bd1ed4fa-6517-40ea-aef4-14b015e858ab","Type":"ContainerStarted","Data":"525268ce7105e4bb7d58d803b135199709c9e5f2abe0b9f8d374a8269d28ae2e"} Jul 03 11:19:09.005221 ip-10-0-136-0 kubenswrapper[2579]: I0703 11:19:09.005155 2579 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="opendatahub/model-serving-api-54d89bd5fc-djwk2" Jul 03 11:19:09.020852 ip-10-0-136-0 kubenswrapper[2579]: I0703 11:19:09.020761 2579 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-ingress/kube-auth-proxy-5cb49b9d79-hl6j7" podStartSLOduration=0.8761837 podStartE2EDuration="6.020738822s" totalImagesPullingTime="5.144555122s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-03 11:19:03 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-03 11:19:09.018890714 +0000 UTC m=+405.607195036" watchObservedRunningTime="2026-07-03 11:19:09.020738822 +0000 UTC m=+405.609043142" Jul 03 11:19:09.040086 ip-10-0-136-0 kubenswrapper[2579]: I0703 11:19:09.040017 2579 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="opendatahub/odh-model-controller-7998c67dc4-wsz76" podStartSLOduration=1.623052374 podStartE2EDuration="6.039994364s" totalImagesPullingTime="4.41694199s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-03 11:19:03 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-03 11:19:09.037867665 +0000 UTC m=+405.626171985" watchObservedRunningTime="2026-07-03 11:19:09.039994364 +0000 UTC m=+405.628298693" Jul 03 11:19:09.058754 ip-10-0-136-0 kubenswrapper[2579]: I0703 11:19:09.058695 2579 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="opendatahub/model-serving-api-54d89bd5fc-djwk2" podStartSLOduration=1.438408882 podStartE2EDuration="5.058676868s" totalImagesPullingTime="3.620267986s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-03 11:19:04 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-03 11:19:09.055664103 +0000 UTC m=+405.643968424" watchObservedRunningTime="2026-07-03 11:19:09.058676868 +0000 UTC m=+405.646981191" Jul 03 11:19:16.559714 ip-10-0-136-0 kubenswrapper[2579]: I0703 11:19:16.559668 2579 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["opendatahub/llmisvc-controller-manager-777c6bd7d5-rsv6f"] Jul 03 11:19:16.563322 ip-10-0-136-0 kubenswrapper[2579]: I0703 11:19:16.563291 2579 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="opendatahub/llmisvc-controller-manager-777c6bd7d5-rsv6f" Jul 03 11:19:16.566768 ip-10-0-136-0 kubenswrapper[2579]: I0703 11:19:16.566741 2579 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"opendatahub\"/\"llmisvc-controller-manager-dockercfg-s9w6t\"" Jul 03 11:19:16.567125 ip-10-0-136-0 kubenswrapper[2579]: I0703 11:19:16.567102 2579 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"opendatahub\"/\"llmisvc-webhook-server-cert\"" Jul 03 11:19:16.570435 ip-10-0-136-0 kubenswrapper[2579]: I0703 11:19:16.570391 2579 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["opendatahub/kserve-controller-manager-d6f4b65b8-sq7gp"] Jul 03 11:19:16.574403 ip-10-0-136-0 kubenswrapper[2579]: I0703 11:19:16.574377 2579 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="opendatahub/kserve-controller-manager-d6f4b65b8-sq7gp" Jul 03 11:19:16.578343 ip-10-0-136-0 kubenswrapper[2579]: I0703 11:19:16.578311 2579 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"opendatahub\"/\"kserve-webhook-server-cert\"" Jul 03 11:19:16.578343 ip-10-0-136-0 kubenswrapper[2579]: I0703 11:19:16.578335 2579 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"opendatahub\"/\"kserve-controller-manager-dockercfg-jgddv\"" Jul 03 11:19:16.589951 ip-10-0-136-0 kubenswrapper[2579]: I0703 11:19:16.589907 2579 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["opendatahub/llmisvc-controller-manager-777c6bd7d5-rsv6f"] Jul 03 11:19:16.621643 ip-10-0-136-0 kubenswrapper[2579]: I0703 11:19:16.621607 2579 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["opendatahub/kserve-controller-manager-d6f4b65b8-sq7gp"] Jul 03 11:19:16.705237 ip-10-0-136-0 kubenswrapper[2579]: I0703 11:19:16.705183 2579 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"cert\" (UniqueName: \"kubernetes.io/secret/ad9d71bf-a6e6-42ed-8997-2ddb98b6f21f-cert\") pod \"llmisvc-controller-manager-777c6bd7d5-rsv6f\" (UID: \"ad9d71bf-a6e6-42ed-8997-2ddb98b6f21f\") " pod="opendatahub/llmisvc-controller-manager-777c6bd7d5-rsv6f" Jul 03 11:19:16.705433 ip-10-0-136-0 kubenswrapper[2579]: I0703 11:19:16.705249 2579 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-dbj9p\" (UniqueName: \"kubernetes.io/projected/ad9d71bf-a6e6-42ed-8997-2ddb98b6f21f-kube-api-access-dbj9p\") pod \"llmisvc-controller-manager-777c6bd7d5-rsv6f\" (UID: \"ad9d71bf-a6e6-42ed-8997-2ddb98b6f21f\") " pod="opendatahub/llmisvc-controller-manager-777c6bd7d5-rsv6f" Jul 03 11:19:16.705433 ip-10-0-136-0 kubenswrapper[2579]: I0703 11:19:16.705363 2579 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-s2swm\" (UniqueName: \"kubernetes.io/projected/0c97772e-8b5b-4733-bfcc-3a0fd479537d-kube-api-access-s2swm\") pod \"kserve-controller-manager-d6f4b65b8-sq7gp\" (UID: \"0c97772e-8b5b-4733-bfcc-3a0fd479537d\") " pod="opendatahub/kserve-controller-manager-d6f4b65b8-sq7gp" Jul 03 11:19:16.705433 ip-10-0-136-0 kubenswrapper[2579]: I0703 11:19:16.705407 2579 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"cert\" (UniqueName: \"kubernetes.io/secret/0c97772e-8b5b-4733-bfcc-3a0fd479537d-cert\") pod \"kserve-controller-manager-d6f4b65b8-sq7gp\" (UID: \"0c97772e-8b5b-4733-bfcc-3a0fd479537d\") " pod="opendatahub/kserve-controller-manager-d6f4b65b8-sq7gp" Jul 03 11:19:16.806922 ip-10-0-136-0 kubenswrapper[2579]: I0703 11:19:16.806863 2579 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"cert\" (UniqueName: \"kubernetes.io/secret/ad9d71bf-a6e6-42ed-8997-2ddb98b6f21f-cert\") pod \"llmisvc-controller-manager-777c6bd7d5-rsv6f\" (UID: \"ad9d71bf-a6e6-42ed-8997-2ddb98b6f21f\") " pod="opendatahub/llmisvc-controller-manager-777c6bd7d5-rsv6f" Jul 03 11:19:16.806922 ip-10-0-136-0 kubenswrapper[2579]: I0703 11:19:16.806933 2579 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-dbj9p\" (UniqueName: \"kubernetes.io/projected/ad9d71bf-a6e6-42ed-8997-2ddb98b6f21f-kube-api-access-dbj9p\") pod \"llmisvc-controller-manager-777c6bd7d5-rsv6f\" (UID: \"ad9d71bf-a6e6-42ed-8997-2ddb98b6f21f\") " pod="opendatahub/llmisvc-controller-manager-777c6bd7d5-rsv6f" Jul 03 11:19:16.807235 ip-10-0-136-0 kubenswrapper[2579]: I0703 11:19:16.806969 2579 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-s2swm\" (UniqueName: \"kubernetes.io/projected/0c97772e-8b5b-4733-bfcc-3a0fd479537d-kube-api-access-s2swm\") pod \"kserve-controller-manager-d6f4b65b8-sq7gp\" (UID: \"0c97772e-8b5b-4733-bfcc-3a0fd479537d\") " pod="opendatahub/kserve-controller-manager-d6f4b65b8-sq7gp" Jul 03 11:19:16.807235 ip-10-0-136-0 kubenswrapper[2579]: I0703 11:19:16.806995 2579 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"cert\" (UniqueName: \"kubernetes.io/secret/0c97772e-8b5b-4733-bfcc-3a0fd479537d-cert\") pod \"kserve-controller-manager-d6f4b65b8-sq7gp\" (UID: \"0c97772e-8b5b-4733-bfcc-3a0fd479537d\") " pod="opendatahub/kserve-controller-manager-d6f4b65b8-sq7gp" Jul 03 11:19:16.809773 ip-10-0-136-0 kubenswrapper[2579]: I0703 11:19:16.809705 2579 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"cert\" (UniqueName: \"kubernetes.io/secret/0c97772e-8b5b-4733-bfcc-3a0fd479537d-cert\") pod \"kserve-controller-manager-d6f4b65b8-sq7gp\" (UID: \"0c97772e-8b5b-4733-bfcc-3a0fd479537d\") " pod="opendatahub/kserve-controller-manager-d6f4b65b8-sq7gp" Jul 03 11:19:16.809773 ip-10-0-136-0 kubenswrapper[2579]: I0703 11:19:16.809707 2579 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"cert\" (UniqueName: \"kubernetes.io/secret/ad9d71bf-a6e6-42ed-8997-2ddb98b6f21f-cert\") pod \"llmisvc-controller-manager-777c6bd7d5-rsv6f\" (UID: \"ad9d71bf-a6e6-42ed-8997-2ddb98b6f21f\") " pod="opendatahub/llmisvc-controller-manager-777c6bd7d5-rsv6f" Jul 03 11:19:16.829015 ip-10-0-136-0 kubenswrapper[2579]: I0703 11:19:16.828978 2579 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-s2swm\" (UniqueName: \"kubernetes.io/projected/0c97772e-8b5b-4733-bfcc-3a0fd479537d-kube-api-access-s2swm\") pod \"kserve-controller-manager-d6f4b65b8-sq7gp\" (UID: \"0c97772e-8b5b-4733-bfcc-3a0fd479537d\") " pod="opendatahub/kserve-controller-manager-d6f4b65b8-sq7gp" Jul 03 11:19:16.831121 ip-10-0-136-0 kubenswrapper[2579]: I0703 11:19:16.831091 2579 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-dbj9p\" (UniqueName: \"kubernetes.io/projected/ad9d71bf-a6e6-42ed-8997-2ddb98b6f21f-kube-api-access-dbj9p\") pod \"llmisvc-controller-manager-777c6bd7d5-rsv6f\" (UID: \"ad9d71bf-a6e6-42ed-8997-2ddb98b6f21f\") " pod="opendatahub/llmisvc-controller-manager-777c6bd7d5-rsv6f" Jul 03 11:19:16.875043 ip-10-0-136-0 kubenswrapper[2579]: I0703 11:19:16.874988 2579 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="opendatahub/llmisvc-controller-manager-777c6bd7d5-rsv6f" Jul 03 11:19:16.885483 ip-10-0-136-0 kubenswrapper[2579]: I0703 11:19:16.885444 2579 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="opendatahub/kserve-controller-manager-d6f4b65b8-sq7gp" Jul 03 11:19:17.164132 ip-10-0-136-0 kubenswrapper[2579]: I0703 11:19:17.164039 2579 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["opendatahub/llmisvc-controller-manager-777c6bd7d5-rsv6f"] Jul 03 11:19:17.205087 ip-10-0-136-0 kubenswrapper[2579]: I0703 11:19:17.205033 2579 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["opendatahub/kserve-controller-manager-d6f4b65b8-sq7gp"] Jul 03 11:19:17.430951 ip-10-0-136-0 kubenswrapper[2579]: W0703 11:19:17.430917 2579 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-podad9d71bf_a6e6_42ed_8997_2ddb98b6f21f.slice/crio-65c0f2ebbc48d365d3933fbd8eec0fa3cb3b6bbec1ada149b1751d2b685677cb WatchSource:0}: Error finding container 65c0f2ebbc48d365d3933fbd8eec0fa3cb3b6bbec1ada149b1751d2b685677cb: Status 404 returned error can't find the container with id 65c0f2ebbc48d365d3933fbd8eec0fa3cb3b6bbec1ada149b1751d2b685677cb Jul 03 11:19:17.431343 ip-10-0-136-0 kubenswrapper[2579]: W0703 11:19:17.431309 2579 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod0c97772e_8b5b_4733_bfcc_3a0fd479537d.slice/crio-afe37ac286c24d592f8ed8593c777b59272cb1b42342dfd9a0b189b72b73bebc WatchSource:0}: Error finding container afe37ac286c24d592f8ed8593c777b59272cb1b42342dfd9a0b189b72b73bebc: Status 404 returned error can't find the container with id afe37ac286c24d592f8ed8593c777b59272cb1b42342dfd9a0b189b72b73bebc Jul 03 11:19:18.053417 ip-10-0-136-0 kubenswrapper[2579]: I0703 11:19:18.053377 2579 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="opendatahub/kserve-controller-manager-d6f4b65b8-sq7gp" event={"ID":"0c97772e-8b5b-4733-bfcc-3a0fd479537d","Type":"ContainerStarted","Data":"afe37ac286c24d592f8ed8593c777b59272cb1b42342dfd9a0b189b72b73bebc"} Jul 03 11:19:18.055036 ip-10-0-136-0 kubenswrapper[2579]: I0703 11:19:18.054995 2579 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="opendatahub/llmisvc-controller-manager-777c6bd7d5-rsv6f" event={"ID":"ad9d71bf-a6e6-42ed-8997-2ddb98b6f21f","Type":"ContainerStarted","Data":"65c0f2ebbc48d365d3933fbd8eec0fa3cb3b6bbec1ada149b1751d2b685677cb"} Jul 03 11:19:18.077762 ip-10-0-136-0 kubenswrapper[2579]: I0703 11:19:18.077700 2579 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-ingress/data-science-gateway-data-science-gateway-class-6576859cf4tjmjj"] Jul 03 11:19:18.090009 ip-10-0-136-0 kubenswrapper[2579]: I0703 11:19:18.089967 2579 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/data-science-gateway-data-science-gateway-class-6576859cf4tjmjj" Jul 03 11:19:18.093521 ip-10-0-136-0 kubenswrapper[2579]: I0703 11:19:18.093479 2579 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress\"/\"data-science-gateway-data-science-gateway-class-dockercfg-cxgth\"" Jul 03 11:19:18.101812 ip-10-0-136-0 kubenswrapper[2579]: I0703 11:19:18.101778 2579 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/data-science-gateway-data-science-gateway-class-6576859cf4tjmjj"] Jul 03 11:19:18.221183 ip-10-0-136-0 kubenswrapper[2579]: I0703 11:19:18.221130 2579 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/a8b74011-2e4c-43be-bb38-bb662cd50ff3-istio-data\") pod \"data-science-gateway-data-science-gateway-class-6576859cf4tjmjj\" (UID: \"a8b74011-2e4c-43be-bb38-bb662cd50ff3\") " pod="openshift-ingress/data-science-gateway-data-science-gateway-class-6576859cf4tjmjj" Jul 03 11:19:18.221400 ip-10-0-136-0 kubenswrapper[2579]: I0703 11:19:18.221211 2579 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/a8b74011-2e4c-43be-bb38-bb662cd50ff3-istiod-ca-cert\") pod \"data-science-gateway-data-science-gateway-class-6576859cf4tjmjj\" (UID: \"a8b74011-2e4c-43be-bb38-bb662cd50ff3\") " pod="openshift-ingress/data-science-gateway-data-science-gateway-class-6576859cf4tjmjj" Jul 03 11:19:18.221400 ip-10-0-136-0 kubenswrapper[2579]: I0703 11:19:18.221286 2579 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/a8b74011-2e4c-43be-bb38-bb662cd50ff3-istio-envoy\") pod \"data-science-gateway-data-science-gateway-class-6576859cf4tjmjj\" (UID: \"a8b74011-2e4c-43be-bb38-bb662cd50ff3\") " pod="openshift-ingress/data-science-gateway-data-science-gateway-class-6576859cf4tjmjj" Jul 03 11:19:18.221400 ip-10-0-136-0 kubenswrapper[2579]: I0703 11:19:18.221375 2579 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/a8b74011-2e4c-43be-bb38-bb662cd50ff3-credential-socket\") pod \"data-science-gateway-data-science-gateway-class-6576859cf4tjmjj\" (UID: \"a8b74011-2e4c-43be-bb38-bb662cd50ff3\") " pod="openshift-ingress/data-science-gateway-data-science-gateway-class-6576859cf4tjmjj" Jul 03 11:19:18.221530 ip-10-0-136-0 kubenswrapper[2579]: I0703 11:19:18.221392 2579 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/a8b74011-2e4c-43be-bb38-bb662cd50ff3-istio-podinfo\") pod \"data-science-gateway-data-science-gateway-class-6576859cf4tjmjj\" (UID: \"a8b74011-2e4c-43be-bb38-bb662cd50ff3\") " pod="openshift-ingress/data-science-gateway-data-science-gateway-class-6576859cf4tjmjj" Jul 03 11:19:18.221530 ip-10-0-136-0 kubenswrapper[2579]: I0703 11:19:18.221450 2579 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/a8b74011-2e4c-43be-bb38-bb662cd50ff3-workload-socket\") pod \"data-science-gateway-data-science-gateway-class-6576859cf4tjmjj\" (UID: \"a8b74011-2e4c-43be-bb38-bb662cd50ff3\") " pod="openshift-ingress/data-science-gateway-data-science-gateway-class-6576859cf4tjmjj" Jul 03 11:19:18.221596 ip-10-0-136-0 kubenswrapper[2579]: I0703 11:19:18.221527 2579 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/a8b74011-2e4c-43be-bb38-bb662cd50ff3-istio-token\") pod \"data-science-gateway-data-science-gateway-class-6576859cf4tjmjj\" (UID: \"a8b74011-2e4c-43be-bb38-bb662cd50ff3\") " pod="openshift-ingress/data-science-gateway-data-science-gateway-class-6576859cf4tjmjj" Jul 03 11:19:18.221596 ip-10-0-136-0 kubenswrapper[2579]: I0703 11:19:18.221577 2579 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/a8b74011-2e4c-43be-bb38-bb662cd50ff3-workload-certs\") pod \"data-science-gateway-data-science-gateway-class-6576859cf4tjmjj\" (UID: \"a8b74011-2e4c-43be-bb38-bb662cd50ff3\") " pod="openshift-ingress/data-science-gateway-data-science-gateway-class-6576859cf4tjmjj" Jul 03 11:19:18.221669 ip-10-0-136-0 kubenswrapper[2579]: I0703 11:19:18.221647 2579 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-bn7pk\" (UniqueName: \"kubernetes.io/projected/a8b74011-2e4c-43be-bb38-bb662cd50ff3-kube-api-access-bn7pk\") pod \"data-science-gateway-data-science-gateway-class-6576859cf4tjmjj\" (UID: \"a8b74011-2e4c-43be-bb38-bb662cd50ff3\") " pod="openshift-ingress/data-science-gateway-data-science-gateway-class-6576859cf4tjmjj" Jul 03 11:19:18.322957 ip-10-0-136-0 kubenswrapper[2579]: I0703 11:19:18.322854 2579 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/a8b74011-2e4c-43be-bb38-bb662cd50ff3-credential-socket\") pod \"data-science-gateway-data-science-gateway-class-6576859cf4tjmjj\" (UID: \"a8b74011-2e4c-43be-bb38-bb662cd50ff3\") " pod="openshift-ingress/data-science-gateway-data-science-gateway-class-6576859cf4tjmjj" Jul 03 11:19:18.322957 ip-10-0-136-0 kubenswrapper[2579]: I0703 11:19:18.322906 2579 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/a8b74011-2e4c-43be-bb38-bb662cd50ff3-istio-podinfo\") pod \"data-science-gateway-data-science-gateway-class-6576859cf4tjmjj\" (UID: \"a8b74011-2e4c-43be-bb38-bb662cd50ff3\") " pod="openshift-ingress/data-science-gateway-data-science-gateway-class-6576859cf4tjmjj" Jul 03 11:19:18.322957 ip-10-0-136-0 kubenswrapper[2579]: I0703 11:19:18.322944 2579 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/a8b74011-2e4c-43be-bb38-bb662cd50ff3-workload-socket\") pod \"data-science-gateway-data-science-gateway-class-6576859cf4tjmjj\" (UID: \"a8b74011-2e4c-43be-bb38-bb662cd50ff3\") " pod="openshift-ingress/data-science-gateway-data-science-gateway-class-6576859cf4tjmjj" Jul 03 11:19:18.323291 ip-10-0-136-0 kubenswrapper[2579]: I0703 11:19:18.322985 2579 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/a8b74011-2e4c-43be-bb38-bb662cd50ff3-istio-token\") pod \"data-science-gateway-data-science-gateway-class-6576859cf4tjmjj\" (UID: \"a8b74011-2e4c-43be-bb38-bb662cd50ff3\") " pod="openshift-ingress/data-science-gateway-data-science-gateway-class-6576859cf4tjmjj" Jul 03 11:19:18.323291 ip-10-0-136-0 kubenswrapper[2579]: I0703 11:19:18.323024 2579 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/a8b74011-2e4c-43be-bb38-bb662cd50ff3-workload-certs\") pod \"data-science-gateway-data-science-gateway-class-6576859cf4tjmjj\" (UID: \"a8b74011-2e4c-43be-bb38-bb662cd50ff3\") " pod="openshift-ingress/data-science-gateway-data-science-gateway-class-6576859cf4tjmjj" Jul 03 11:19:18.323291 ip-10-0-136-0 kubenswrapper[2579]: I0703 11:19:18.323064 2579 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-bn7pk\" (UniqueName: \"kubernetes.io/projected/a8b74011-2e4c-43be-bb38-bb662cd50ff3-kube-api-access-bn7pk\") pod \"data-science-gateway-data-science-gateway-class-6576859cf4tjmjj\" (UID: \"a8b74011-2e4c-43be-bb38-bb662cd50ff3\") " pod="openshift-ingress/data-science-gateway-data-science-gateway-class-6576859cf4tjmjj" Jul 03 11:19:18.323291 ip-10-0-136-0 kubenswrapper[2579]: I0703 11:19:18.323096 2579 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/a8b74011-2e4c-43be-bb38-bb662cd50ff3-istio-data\") pod \"data-science-gateway-data-science-gateway-class-6576859cf4tjmjj\" (UID: \"a8b74011-2e4c-43be-bb38-bb662cd50ff3\") " pod="openshift-ingress/data-science-gateway-data-science-gateway-class-6576859cf4tjmjj" Jul 03 11:19:18.323291 ip-10-0-136-0 kubenswrapper[2579]: I0703 11:19:18.323135 2579 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/a8b74011-2e4c-43be-bb38-bb662cd50ff3-istiod-ca-cert\") pod \"data-science-gateway-data-science-gateway-class-6576859cf4tjmjj\" (UID: \"a8b74011-2e4c-43be-bb38-bb662cd50ff3\") " pod="openshift-ingress/data-science-gateway-data-science-gateway-class-6576859cf4tjmjj" Jul 03 11:19:18.323291 ip-10-0-136-0 kubenswrapper[2579]: I0703 11:19:18.323171 2579 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/a8b74011-2e4c-43be-bb38-bb662cd50ff3-istio-envoy\") pod \"data-science-gateway-data-science-gateway-class-6576859cf4tjmjj\" (UID: \"a8b74011-2e4c-43be-bb38-bb662cd50ff3\") " pod="openshift-ingress/data-science-gateway-data-science-gateway-class-6576859cf4tjmjj" Jul 03 11:19:18.323702 ip-10-0-136-0 kubenswrapper[2579]: I0703 11:19:18.323500 2579 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/a8b74011-2e4c-43be-bb38-bb662cd50ff3-workload-certs\") pod \"data-science-gateway-data-science-gateway-class-6576859cf4tjmjj\" (UID: \"a8b74011-2e4c-43be-bb38-bb662cd50ff3\") " pod="openshift-ingress/data-science-gateway-data-science-gateway-class-6576859cf4tjmjj" Jul 03 11:19:18.324898 ip-10-0-136-0 kubenswrapper[2579]: I0703 11:19:18.324020 2579 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/a8b74011-2e4c-43be-bb38-bb662cd50ff3-istio-data\") pod \"data-science-gateway-data-science-gateway-class-6576859cf4tjmjj\" (UID: \"a8b74011-2e4c-43be-bb38-bb662cd50ff3\") " pod="openshift-ingress/data-science-gateway-data-science-gateway-class-6576859cf4tjmjj" Jul 03 11:19:18.324898 ip-10-0-136-0 kubenswrapper[2579]: I0703 11:19:18.324596 2579 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/a8b74011-2e4c-43be-bb38-bb662cd50ff3-workload-socket\") pod \"data-science-gateway-data-science-gateway-class-6576859cf4tjmjj\" (UID: \"a8b74011-2e4c-43be-bb38-bb662cd50ff3\") " pod="openshift-ingress/data-science-gateway-data-science-gateway-class-6576859cf4tjmjj" Jul 03 11:19:18.324898 ip-10-0-136-0 kubenswrapper[2579]: I0703 11:19:18.324626 2579 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/a8b74011-2e4c-43be-bb38-bb662cd50ff3-istiod-ca-cert\") pod \"data-science-gateway-data-science-gateway-class-6576859cf4tjmjj\" (UID: \"a8b74011-2e4c-43be-bb38-bb662cd50ff3\") " pod="openshift-ingress/data-science-gateway-data-science-gateway-class-6576859cf4tjmjj" Jul 03 11:19:18.324898 ip-10-0-136-0 kubenswrapper[2579]: I0703 11:19:18.324786 2579 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/a8b74011-2e4c-43be-bb38-bb662cd50ff3-credential-socket\") pod \"data-science-gateway-data-science-gateway-class-6576859cf4tjmjj\" (UID: \"a8b74011-2e4c-43be-bb38-bb662cd50ff3\") " pod="openshift-ingress/data-science-gateway-data-science-gateway-class-6576859cf4tjmjj" Jul 03 11:19:18.326699 ip-10-0-136-0 kubenswrapper[2579]: I0703 11:19:18.326576 2579 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/a8b74011-2e4c-43be-bb38-bb662cd50ff3-istio-podinfo\") pod \"data-science-gateway-data-science-gateway-class-6576859cf4tjmjj\" (UID: \"a8b74011-2e4c-43be-bb38-bb662cd50ff3\") " pod="openshift-ingress/data-science-gateway-data-science-gateway-class-6576859cf4tjmjj" Jul 03 11:19:18.327179 ip-10-0-136-0 kubenswrapper[2579]: I0703 11:19:18.327157 2579 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/a8b74011-2e4c-43be-bb38-bb662cd50ff3-istio-envoy\") pod \"data-science-gateway-data-science-gateway-class-6576859cf4tjmjj\" (UID: \"a8b74011-2e4c-43be-bb38-bb662cd50ff3\") " pod="openshift-ingress/data-science-gateway-data-science-gateway-class-6576859cf4tjmjj" Jul 03 11:19:18.343684 ip-10-0-136-0 kubenswrapper[2579]: I0703 11:19:18.343647 2579 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/a8b74011-2e4c-43be-bb38-bb662cd50ff3-istio-token\") pod \"data-science-gateway-data-science-gateway-class-6576859cf4tjmjj\" (UID: \"a8b74011-2e4c-43be-bb38-bb662cd50ff3\") " pod="openshift-ingress/data-science-gateway-data-science-gateway-class-6576859cf4tjmjj" Jul 03 11:19:18.349578 ip-10-0-136-0 kubenswrapper[2579]: I0703 11:19:18.349535 2579 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-bn7pk\" (UniqueName: \"kubernetes.io/projected/a8b74011-2e4c-43be-bb38-bb662cd50ff3-kube-api-access-bn7pk\") pod \"data-science-gateway-data-science-gateway-class-6576859cf4tjmjj\" (UID: \"a8b74011-2e4c-43be-bb38-bb662cd50ff3\") " pod="openshift-ingress/data-science-gateway-data-science-gateway-class-6576859cf4tjmjj" Jul 03 11:19:18.406513 ip-10-0-136-0 kubenswrapper[2579]: I0703 11:19:18.406463 2579 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/data-science-gateway-data-science-gateway-class-6576859cf4tjmjj" Jul 03 11:19:18.807447 ip-10-0-136-0 kubenswrapper[2579]: I0703 11:19:18.807326 2579 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/data-science-gateway-data-science-gateway-class-6576859cf4tjmjj"] Jul 03 11:19:18.912437 ip-10-0-136-0 kubenswrapper[2579]: W0703 11:19:18.912361 2579 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-poda8b74011_2e4c_43be_bb38_bb662cd50ff3.slice/crio-ec954471783721063ca6145798b203c18ecf9611a6458e7ca0bf13f1eff1076e WatchSource:0}: Error finding container ec954471783721063ca6145798b203c18ecf9611a6458e7ca0bf13f1eff1076e: Status 404 returned error can't find the container with id ec954471783721063ca6145798b203c18ecf9611a6458e7ca0bf13f1eff1076e Jul 03 11:19:19.066125 ip-10-0-136-0 kubenswrapper[2579]: I0703 11:19:19.066070 2579 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/data-science-gateway-data-science-gateway-class-6576859cf4tjmjj" event={"ID":"a8b74011-2e4c-43be-bb38-bb662cd50ff3","Type":"ContainerStarted","Data":"ec954471783721063ca6145798b203c18ecf9611a6458e7ca0bf13f1eff1076e"} Jul 03 11:19:19.777388 ip-10-0-136-0 kubenswrapper[2579]: I0703 11:19:19.777347 2579 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-ingress/istiod-openshift-gateway-7574459656-m7vpm"] Jul 03 11:19:20.013545 ip-10-0-136-0 kubenswrapper[2579]: I0703 11:19:20.013515 2579 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="opendatahub/odh-model-controller-7998c67dc4-wsz76" Jul 03 11:19:20.016287 ip-10-0-136-0 kubenswrapper[2579]: I0703 11:19:20.016259 2579 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="opendatahub/model-serving-api-54d89bd5fc-djwk2" Jul 03 11:19:21.080401 ip-10-0-136-0 kubenswrapper[2579]: I0703 11:19:21.080357 2579 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="opendatahub/kserve-controller-manager-d6f4b65b8-sq7gp" event={"ID":"0c97772e-8b5b-4733-bfcc-3a0fd479537d","Type":"ContainerStarted","Data":"e05e8d5dd4274b259cbd361d262e05649b1b9eacf5479a3ce246e784c5309d2a"} Jul 03 11:19:21.080894 ip-10-0-136-0 kubenswrapper[2579]: I0703 11:19:21.080803 2579 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="opendatahub/kserve-controller-manager-d6f4b65b8-sq7gp" Jul 03 11:19:21.106148 ip-10-0-136-0 kubenswrapper[2579]: I0703 11:19:21.106080 2579 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="opendatahub/kserve-controller-manager-d6f4b65b8-sq7gp" podStartSLOduration=2.451931164 podStartE2EDuration="5.106056852s" totalImagesPullingTime="2.654125688s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-03 11:19:16 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-03 11:19:21.106000824 +0000 UTC m=+417.694305143" watchObservedRunningTime="2026-07-03 11:19:21.106056852 +0000 UTC m=+417.694361172" Jul 03 11:19:22.088527 ip-10-0-136-0 kubenswrapper[2579]: I0703 11:19:22.088478 2579 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="opendatahub/llmisvc-controller-manager-777c6bd7d5-rsv6f" event={"ID":"ad9d71bf-a6e6-42ed-8997-2ddb98b6f21f","Type":"ContainerStarted","Data":"4541845effabac9c4b855bc7251b944508755db93aa16fdc597af65dc00569ef"} Jul 03 11:19:22.150270 ip-10-0-136-0 kubenswrapper[2579]: I0703 11:19:22.149773 2579 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="opendatahub/llmisvc-controller-manager-777c6bd7d5-rsv6f" podStartSLOduration=2.212853099 podStartE2EDuration="6.149747758s" totalImagesPullingTime="3.936894659s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-03 11:19:16 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-03 11:19:22.1480142 +0000 UTC m=+418.736318514" watchObservedRunningTime="2026-07-03 11:19:22.149747758 +0000 UTC m=+418.738052078" Jul 03 11:19:22.600656 ip-10-0-136-0 kubenswrapper[2579]: I0703 11:19:22.600604 2579 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"30892156Ki","pods":"250"} Jul 03 11:19:22.600833 ip-10-0-136-0 kubenswrapper[2579]: I0703 11:19:22.600703 2579 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"30892156Ki","pods":"250"} Jul 03 11:19:22.600833 ip-10-0-136-0 kubenswrapper[2579]: I0703 11:19:22.600768 2579 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"30892156Ki","pods":"250"} Jul 03 11:19:23.095729 ip-10-0-136-0 kubenswrapper[2579]: I0703 11:19:23.095687 2579 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/data-science-gateway-data-science-gateway-class-6576859cf4tjmjj" event={"ID":"a8b74011-2e4c-43be-bb38-bb662cd50ff3","Type":"ContainerStarted","Data":"1a49edc70a330dc13ba9d59005f660033051ac9fe9d1ddb5abb6427639050767"} Jul 03 11:19:23.096265 ip-10-0-136-0 kubenswrapper[2579]: I0703 11:19:23.095840 2579 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="opendatahub/llmisvc-controller-manager-777c6bd7d5-rsv6f" Jul 03 11:19:23.124856 ip-10-0-136-0 kubenswrapper[2579]: I0703 11:19:23.124789 2579 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-ingress/data-science-gateway-data-science-gateway-class-6576859cf4tjmjj" podStartSLOduration=1.440043604 podStartE2EDuration="5.124763332s" totalImagesPullingTime="3.684719728s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-03 11:19:18 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-03 11:19:23.122595974 +0000 UTC m=+419.710900314" watchObservedRunningTime="2026-07-03 11:19:23.124763332 +0000 UTC m=+419.713067687" Jul 03 11:19:23.407112 ip-10-0-136-0 kubenswrapper[2579]: I0703 11:19:23.407000 2579 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="openshift-ingress/data-science-gateway-data-science-gateway-class-6576859cf4tjmjj" Jul 03 11:19:23.412792 ip-10-0-136-0 kubenswrapper[2579]: I0703 11:19:23.412756 2579 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="openshift-ingress/data-science-gateway-data-science-gateway-class-6576859cf4tjmjj" Jul 03 11:19:24.100819 ip-10-0-136-0 kubenswrapper[2579]: I0703 11:19:24.100783 2579 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/data-science-gateway-data-science-gateway-class-6576859cf4tjmjj" Jul 03 11:19:24.101826 ip-10-0-136-0 kubenswrapper[2579]: I0703 11:19:24.101806 2579 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/data-science-gateway-data-science-gateway-class-6576859cf4tjmjj" Jul 03 11:19:34.038526 ip-10-0-136-0 kubenswrapper[2579]: I0703 11:19:34.038485 2579 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"30892156Ki","pods":"250"} Jul 03 11:19:34.038896 ip-10-0-136-0 kubenswrapper[2579]: I0703 11:19:34.038553 2579 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"30892156Ki","pods":"250"} Jul 03 11:19:34.155455 ip-10-0-136-0 kubenswrapper[2579]: I0703 11:19:34.155414 2579 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/istiod-openshift-gateway-7574459656-m7vpm" event={"ID":"05548561-66e3-4667-bffc-da5b0b45237b","Type":"ContainerStarted","Data":"3da13cf12f1ed1e32cfcacec03dcc3be2c16654ae5a6a53c9f28854711199986"} Jul 03 11:19:34.155677 ip-10-0-136-0 kubenswrapper[2579]: I0703 11:19:34.155482 2579 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/istiod-openshift-gateway-7574459656-m7vpm" Jul 03 11:19:34.155677 ip-10-0-136-0 kubenswrapper[2579]: I0703 11:19:34.155504 2579 kuberuntime_container.go:872] "Killing container with a grace period" pod="openshift-ingress/istiod-openshift-gateway-7574459656-m7vpm" podUID="05548561-66e3-4667-bffc-da5b0b45237b" containerName="discovery" containerID="cri-o://3da13cf12f1ed1e32cfcacec03dcc3be2c16654ae5a6a53c9f28854711199986" gracePeriod=30 Jul 03 11:19:34.190716 ip-10-0-136-0 kubenswrapper[2579]: I0703 11:19:34.190642 2579 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-ingress/istiod-openshift-gateway-7574459656-m7vpm" podStartSLOduration=1.107077706 podStartE2EDuration="29.190613434s" totalImagesPullingTime="28.083535728s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-03 11:19:05 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-03 11:19:34.190142349 +0000 UTC m=+430.778446668" watchObservedRunningTime="2026-07-03 11:19:34.190613434 +0000 UTC m=+430.778917753" Jul 03 11:19:34.415473 ip-10-0-136-0 kubenswrapper[2579]: I0703 11:19:34.415434 2579 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/istiod-openshift-gateway-7574459656-m7vpm" Jul 03 11:19:34.475468 ip-10-0-136-0 kubenswrapper[2579]: I0703 11:19:34.475425 2579 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-b775n\" (UniqueName: \"kubernetes.io/projected/05548561-66e3-4667-bffc-da5b0b45237b-kube-api-access-b775n\") pod \"05548561-66e3-4667-bffc-da5b0b45237b\" (UID: \"05548561-66e3-4667-bffc-da5b0b45237b\") " Jul 03 11:19:34.475659 ip-10-0-136-0 kubenswrapper[2579]: I0703 11:19:34.475492 2579 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/05548561-66e3-4667-bffc-da5b0b45237b-istio-token\") pod \"05548561-66e3-4667-bffc-da5b0b45237b\" (UID: \"05548561-66e3-4667-bffc-da5b0b45237b\") " Jul 03 11:19:34.475659 ip-10-0-136-0 kubenswrapper[2579]: I0703 11:19:34.475568 2579 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"local-certs\" (UniqueName: \"kubernetes.io/empty-dir/05548561-66e3-4667-bffc-da5b0b45237b-local-certs\") pod \"05548561-66e3-4667-bffc-da5b0b45237b\" (UID: \"05548561-66e3-4667-bffc-da5b0b45237b\") " Jul 03 11:19:34.475659 ip-10-0-136-0 kubenswrapper[2579]: I0703 11:19:34.475623 2579 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istio-csr-ca-configmap\" (UniqueName: \"kubernetes.io/configmap/05548561-66e3-4667-bffc-da5b0b45237b-istio-csr-ca-configmap\") pod \"05548561-66e3-4667-bffc-da5b0b45237b\" (UID: \"05548561-66e3-4667-bffc-da5b0b45237b\") " Jul 03 11:19:34.475659 ip-10-0-136-0 kubenswrapper[2579]: I0703 11:19:34.475650 2579 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istio-kubeconfig\" (UniqueName: \"kubernetes.io/secret/05548561-66e3-4667-bffc-da5b0b45237b-istio-kubeconfig\") pod \"05548561-66e3-4667-bffc-da5b0b45237b\" (UID: \"05548561-66e3-4667-bffc-da5b0b45237b\") " Jul 03 11:19:34.475876 ip-10-0-136-0 kubenswrapper[2579]: I0703 11:19:34.475678 2579 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"cacerts\" (UniqueName: \"kubernetes.io/secret/05548561-66e3-4667-bffc-da5b0b45237b-cacerts\") pod \"05548561-66e3-4667-bffc-da5b0b45237b\" (UID: \"05548561-66e3-4667-bffc-da5b0b45237b\") " Jul 03 11:19:34.475876 ip-10-0-136-0 kubenswrapper[2579]: I0703 11:19:34.475732 2579 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istio-csr-dns-cert\" (UniqueName: \"kubernetes.io/secret/05548561-66e3-4667-bffc-da5b0b45237b-istio-csr-dns-cert\") pod \"05548561-66e3-4667-bffc-da5b0b45237b\" (UID: \"05548561-66e3-4667-bffc-da5b0b45237b\") " Jul 03 11:19:34.476434 ip-10-0-136-0 kubenswrapper[2579]: I0703 11:19:34.476401 2579 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/05548561-66e3-4667-bffc-da5b0b45237b-istio-csr-ca-configmap" (OuterVolumeSpecName: "istio-csr-ca-configmap") pod "05548561-66e3-4667-bffc-da5b0b45237b" (UID: "05548561-66e3-4667-bffc-da5b0b45237b"). InnerVolumeSpecName "istio-csr-ca-configmap". PluginName "kubernetes.io/configmap", VolumeGIDValue "" Jul 03 11:19:34.478840 ip-10-0-136-0 kubenswrapper[2579]: I0703 11:19:34.478768 2579 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/05548561-66e3-4667-bffc-da5b0b45237b-istio-csr-dns-cert" (OuterVolumeSpecName: "istio-csr-dns-cert") pod "05548561-66e3-4667-bffc-da5b0b45237b" (UID: "05548561-66e3-4667-bffc-da5b0b45237b"). InnerVolumeSpecName "istio-csr-dns-cert". PluginName "kubernetes.io/secret", VolumeGIDValue "" Jul 03 11:19:34.478840 ip-10-0-136-0 kubenswrapper[2579]: I0703 11:19:34.478834 2579 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/05548561-66e3-4667-bffc-da5b0b45237b-kube-api-access-b775n" (OuterVolumeSpecName: "kube-api-access-b775n") pod "05548561-66e3-4667-bffc-da5b0b45237b" (UID: "05548561-66e3-4667-bffc-da5b0b45237b"). InnerVolumeSpecName "kube-api-access-b775n". PluginName "kubernetes.io/projected", VolumeGIDValue "" Jul 03 11:19:34.479011 ip-10-0-136-0 kubenswrapper[2579]: I0703 11:19:34.478836 2579 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/05548561-66e3-4667-bffc-da5b0b45237b-local-certs" (OuterVolumeSpecName: "local-certs") pod "05548561-66e3-4667-bffc-da5b0b45237b" (UID: "05548561-66e3-4667-bffc-da5b0b45237b"). InnerVolumeSpecName "local-certs". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Jul 03 11:19:34.479011 ip-10-0-136-0 kubenswrapper[2579]: I0703 11:19:34.478891 2579 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/05548561-66e3-4667-bffc-da5b0b45237b-cacerts" (OuterVolumeSpecName: "cacerts") pod "05548561-66e3-4667-bffc-da5b0b45237b" (UID: "05548561-66e3-4667-bffc-da5b0b45237b"). InnerVolumeSpecName "cacerts". PluginName "kubernetes.io/secret", VolumeGIDValue "" Jul 03 11:19:34.479011 ip-10-0-136-0 kubenswrapper[2579]: I0703 11:19:34.478904 2579 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/05548561-66e3-4667-bffc-da5b0b45237b-istio-token" (OuterVolumeSpecName: "istio-token") pod "05548561-66e3-4667-bffc-da5b0b45237b" (UID: "05548561-66e3-4667-bffc-da5b0b45237b"). InnerVolumeSpecName "istio-token". PluginName "kubernetes.io/projected", VolumeGIDValue "" Jul 03 11:19:34.479011 ip-10-0-136-0 kubenswrapper[2579]: I0703 11:19:34.478904 2579 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/05548561-66e3-4667-bffc-da5b0b45237b-istio-kubeconfig" (OuterVolumeSpecName: "istio-kubeconfig") pod "05548561-66e3-4667-bffc-da5b0b45237b" (UID: "05548561-66e3-4667-bffc-da5b0b45237b"). InnerVolumeSpecName "istio-kubeconfig". PluginName "kubernetes.io/secret", VolumeGIDValue "" Jul 03 11:19:34.576664 ip-10-0-136-0 kubenswrapper[2579]: I0703 11:19:34.576557 2579 reconciler_common.go:299] "Volume detached for volume \"local-certs\" (UniqueName: \"kubernetes.io/empty-dir/05548561-66e3-4667-bffc-da5b0b45237b-local-certs\") on node \"ip-10-0-136-0.ec2.internal\" DevicePath \"\"" Jul 03 11:19:34.576664 ip-10-0-136-0 kubenswrapper[2579]: I0703 11:19:34.576592 2579 reconciler_common.go:299] "Volume detached for volume \"istio-csr-ca-configmap\" (UniqueName: \"kubernetes.io/configmap/05548561-66e3-4667-bffc-da5b0b45237b-istio-csr-ca-configmap\") on node \"ip-10-0-136-0.ec2.internal\" DevicePath \"\"" Jul 03 11:19:34.576664 ip-10-0-136-0 kubenswrapper[2579]: I0703 11:19:34.576605 2579 reconciler_common.go:299] "Volume detached for volume \"istio-kubeconfig\" (UniqueName: \"kubernetes.io/secret/05548561-66e3-4667-bffc-da5b0b45237b-istio-kubeconfig\") on node \"ip-10-0-136-0.ec2.internal\" DevicePath \"\"" Jul 03 11:19:34.576664 ip-10-0-136-0 kubenswrapper[2579]: I0703 11:19:34.576614 2579 reconciler_common.go:299] "Volume detached for volume \"cacerts\" (UniqueName: \"kubernetes.io/secret/05548561-66e3-4667-bffc-da5b0b45237b-cacerts\") on node \"ip-10-0-136-0.ec2.internal\" DevicePath \"\"" Jul 03 11:19:34.576664 ip-10-0-136-0 kubenswrapper[2579]: I0703 11:19:34.576622 2579 reconciler_common.go:299] "Volume detached for volume \"istio-csr-dns-cert\" (UniqueName: \"kubernetes.io/secret/05548561-66e3-4667-bffc-da5b0b45237b-istio-csr-dns-cert\") on node \"ip-10-0-136-0.ec2.internal\" DevicePath \"\"" Jul 03 11:19:34.576664 ip-10-0-136-0 kubenswrapper[2579]: I0703 11:19:34.576631 2579 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-b775n\" (UniqueName: \"kubernetes.io/projected/05548561-66e3-4667-bffc-da5b0b45237b-kube-api-access-b775n\") on node \"ip-10-0-136-0.ec2.internal\" DevicePath \"\"" Jul 03 11:19:34.576664 ip-10-0-136-0 kubenswrapper[2579]: I0703 11:19:34.576639 2579 reconciler_common.go:299] "Volume detached for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/05548561-66e3-4667-bffc-da5b0b45237b-istio-token\") on node \"ip-10-0-136-0.ec2.internal\" DevicePath \"\"" Jul 03 11:19:35.161854 ip-10-0-136-0 kubenswrapper[2579]: I0703 11:19:35.161808 2579 generic.go:356] "Generic (PLEG): container finished" podID="05548561-66e3-4667-bffc-da5b0b45237b" containerID="3da13cf12f1ed1e32cfcacec03dcc3be2c16654ae5a6a53c9f28854711199986" exitCode=2 Jul 03 11:19:35.162364 ip-10-0-136-0 kubenswrapper[2579]: I0703 11:19:35.161864 2579 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/istiod-openshift-gateway-7574459656-m7vpm" Jul 03 11:19:35.162364 ip-10-0-136-0 kubenswrapper[2579]: I0703 11:19:35.161864 2579 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/istiod-openshift-gateway-7574459656-m7vpm" event={"ID":"05548561-66e3-4667-bffc-da5b0b45237b","Type":"ContainerDied","Data":"3da13cf12f1ed1e32cfcacec03dcc3be2c16654ae5a6a53c9f28854711199986"} Jul 03 11:19:35.162364 ip-10-0-136-0 kubenswrapper[2579]: I0703 11:19:35.162022 2579 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/istiod-openshift-gateway-7574459656-m7vpm" event={"ID":"05548561-66e3-4667-bffc-da5b0b45237b","Type":"ContainerDied","Data":"379875108a5e0c09b1cc4bfc669c5aa8b8fed81ac393b8413532b3d70f5cbbf6"} Jul 03 11:19:35.162364 ip-10-0-136-0 kubenswrapper[2579]: I0703 11:19:35.162050 2579 scope.go:117] "RemoveContainer" containerID="3da13cf12f1ed1e32cfcacec03dcc3be2c16654ae5a6a53c9f28854711199986" Jul 03 11:19:35.172401 ip-10-0-136-0 kubenswrapper[2579]: I0703 11:19:35.172376 2579 scope.go:117] "RemoveContainer" containerID="3da13cf12f1ed1e32cfcacec03dcc3be2c16654ae5a6a53c9f28854711199986" Jul 03 11:19:35.172752 ip-10-0-136-0 kubenswrapper[2579]: E0703 11:19:35.172729 2579 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"3da13cf12f1ed1e32cfcacec03dcc3be2c16654ae5a6a53c9f28854711199986\": container with ID starting with 3da13cf12f1ed1e32cfcacec03dcc3be2c16654ae5a6a53c9f28854711199986 not found: ID does not exist" containerID="3da13cf12f1ed1e32cfcacec03dcc3be2c16654ae5a6a53c9f28854711199986" Jul 03 11:19:35.172804 ip-10-0-136-0 kubenswrapper[2579]: I0703 11:19:35.172763 2579 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"3da13cf12f1ed1e32cfcacec03dcc3be2c16654ae5a6a53c9f28854711199986"} err="failed to get container status \"3da13cf12f1ed1e32cfcacec03dcc3be2c16654ae5a6a53c9f28854711199986\": rpc error: code = NotFound desc = could not find container \"3da13cf12f1ed1e32cfcacec03dcc3be2c16654ae5a6a53c9f28854711199986\": container with ID starting with 3da13cf12f1ed1e32cfcacec03dcc3be2c16654ae5a6a53c9f28854711199986 not found: ID does not exist" Jul 03 11:19:35.186109 ip-10-0-136-0 kubenswrapper[2579]: I0703 11:19:35.186060 2579 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-ingress/istiod-openshift-gateway-7574459656-m7vpm"] Jul 03 11:19:35.190563 ip-10-0-136-0 kubenswrapper[2579]: I0703 11:19:35.190523 2579 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["openshift-ingress/istiod-openshift-gateway-7574459656-m7vpm"] Jul 03 11:19:35.991944 ip-10-0-136-0 kubenswrapper[2579]: I0703 11:19:35.991897 2579 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="05548561-66e3-4667-bffc-da5b0b45237b" path="/var/lib/kubelet/pods/05548561-66e3-4667-bffc-da5b0b45237b/volumes" Jul 03 11:19:52.094803 ip-10-0-136-0 kubenswrapper[2579]: I0703 11:19:52.094699 2579 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="opendatahub/kserve-controller-manager-d6f4b65b8-sq7gp" Jul 03 11:19:54.103360 ip-10-0-136-0 kubenswrapper[2579]: I0703 11:19:54.103321 2579 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="opendatahub/llmisvc-controller-manager-777c6bd7d5-rsv6f" Jul 03 11:20:03.805956 ip-10-0-136-0 kubenswrapper[2579]: I0703 11:20:03.805909 2579 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["kuadrant-system/kuadrant-operator-catalog-5wvb2"] Jul 03 11:20:03.806550 ip-10-0-136-0 kubenswrapper[2579]: I0703 11:20:03.806316 2579 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="05548561-66e3-4667-bffc-da5b0b45237b" containerName="discovery" Jul 03 11:20:03.806550 ip-10-0-136-0 kubenswrapper[2579]: I0703 11:20:03.806328 2579 state_mem.go:107] "Deleted CPUSet assignment" podUID="05548561-66e3-4667-bffc-da5b0b45237b" containerName="discovery" Jul 03 11:20:03.806550 ip-10-0-136-0 kubenswrapper[2579]: I0703 11:20:03.806388 2579 memory_manager.go:368] "RemoveStaleState removing state" podUID="05548561-66e3-4667-bffc-da5b0b45237b" containerName="discovery" Jul 03 11:20:03.814845 ip-10-0-136-0 kubenswrapper[2579]: I0703 11:20:03.814812 2579 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/kuadrant-operator-catalog-5wvb2" Jul 03 11:20:03.819052 ip-10-0-136-0 kubenswrapper[2579]: I0703 11:20:03.819021 2579 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"kuadrant-system\"/\"kuadrant-operator-catalog-dockercfg-lp6nc\"" Jul 03 11:20:03.819266 ip-10-0-136-0 kubenswrapper[2579]: I0703 11:20:03.819207 2579 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"kuadrant-system\"/\"openshift-service-ca.crt\"" Jul 03 11:20:03.819359 ip-10-0-136-0 kubenswrapper[2579]: I0703 11:20:03.819037 2579 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"kuadrant-system\"/\"kube-root-ca.crt\"" Jul 03 11:20:03.821822 ip-10-0-136-0 kubenswrapper[2579]: I0703 11:20:03.821788 2579 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/kuadrant-operator-catalog-5wvb2"] Jul 03 11:20:03.959004 ip-10-0-136-0 kubenswrapper[2579]: I0703 11:20:03.958956 2579 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-dhtqj\" (UniqueName: \"kubernetes.io/projected/8eac4fd4-584f-4773-91b3-9132820e227a-kube-api-access-dhtqj\") pod \"kuadrant-operator-catalog-5wvb2\" (UID: \"8eac4fd4-584f-4773-91b3-9132820e227a\") " pod="kuadrant-system/kuadrant-operator-catalog-5wvb2" Jul 03 11:20:04.059790 ip-10-0-136-0 kubenswrapper[2579]: I0703 11:20:04.059674 2579 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-dhtqj\" (UniqueName: \"kubernetes.io/projected/8eac4fd4-584f-4773-91b3-9132820e227a-kube-api-access-dhtqj\") pod \"kuadrant-operator-catalog-5wvb2\" (UID: \"8eac4fd4-584f-4773-91b3-9132820e227a\") " pod="kuadrant-system/kuadrant-operator-catalog-5wvb2" Jul 03 11:20:04.072958 ip-10-0-136-0 kubenswrapper[2579]: I0703 11:20:04.072912 2579 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-dhtqj\" (UniqueName: \"kubernetes.io/projected/8eac4fd4-584f-4773-91b3-9132820e227a-kube-api-access-dhtqj\") pod \"kuadrant-operator-catalog-5wvb2\" (UID: \"8eac4fd4-584f-4773-91b3-9132820e227a\") " pod="kuadrant-system/kuadrant-operator-catalog-5wvb2" Jul 03 11:20:04.129263 ip-10-0-136-0 kubenswrapper[2579]: I0703 11:20:04.129179 2579 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/kuadrant-operator-catalog-5wvb2" Jul 03 11:20:04.142302 ip-10-0-136-0 kubenswrapper[2579]: I0703 11:20:04.142257 2579 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["kuadrant-system/kuadrant-operator-catalog-5wvb2"] Jul 03 11:20:04.301396 ip-10-0-136-0 kubenswrapper[2579]: I0703 11:20:04.301359 2579 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["kuadrant-system/kuadrant-operator-catalog-5wvb2"] Jul 03 11:20:04.304641 ip-10-0-136-0 kubenswrapper[2579]: W0703 11:20:04.304597 2579 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod8eac4fd4_584f_4773_91b3_9132820e227a.slice/crio-f80dff146c0e37898243cdc57b18553c6b043565f7c6e44e81b858f16875f607 WatchSource:0}: Error finding container f80dff146c0e37898243cdc57b18553c6b043565f7c6e44e81b858f16875f607: Status 404 returned error can't find the container with id f80dff146c0e37898243cdc57b18553c6b043565f7c6e44e81b858f16875f607 Jul 03 11:20:04.355784 ip-10-0-136-0 kubenswrapper[2579]: I0703 11:20:04.355694 2579 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["kuadrant-system/kuadrant-operator-catalog-j29hp"] Jul 03 11:20:04.358824 ip-10-0-136-0 kubenswrapper[2579]: I0703 11:20:04.358799 2579 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/kuadrant-operator-catalog-j29hp" Jul 03 11:20:04.371930 ip-10-0-136-0 kubenswrapper[2579]: I0703 11:20:04.371896 2579 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/kuadrant-operator-catalog-j29hp"] Jul 03 11:20:04.463955 ip-10-0-136-0 kubenswrapper[2579]: I0703 11:20:04.463904 2579 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-kklxd\" (UniqueName: \"kubernetes.io/projected/b26c8c7f-99b4-4644-9aee-b33bc9985fe5-kube-api-access-kklxd\") pod \"kuadrant-operator-catalog-j29hp\" (UID: \"b26c8c7f-99b4-4644-9aee-b33bc9985fe5\") " pod="kuadrant-system/kuadrant-operator-catalog-j29hp" Jul 03 11:20:04.564631 ip-10-0-136-0 kubenswrapper[2579]: I0703 11:20:04.564592 2579 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-kklxd\" (UniqueName: \"kubernetes.io/projected/b26c8c7f-99b4-4644-9aee-b33bc9985fe5-kube-api-access-kklxd\") pod \"kuadrant-operator-catalog-j29hp\" (UID: \"b26c8c7f-99b4-4644-9aee-b33bc9985fe5\") " pod="kuadrant-system/kuadrant-operator-catalog-j29hp" Jul 03 11:20:04.579212 ip-10-0-136-0 kubenswrapper[2579]: I0703 11:20:04.579161 2579 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-kklxd\" (UniqueName: \"kubernetes.io/projected/b26c8c7f-99b4-4644-9aee-b33bc9985fe5-kube-api-access-kklxd\") pod \"kuadrant-operator-catalog-j29hp\" (UID: \"b26c8c7f-99b4-4644-9aee-b33bc9985fe5\") " pod="kuadrant-system/kuadrant-operator-catalog-j29hp" Jul 03 11:20:04.670688 ip-10-0-136-0 kubenswrapper[2579]: I0703 11:20:04.670561 2579 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/kuadrant-operator-catalog-j29hp" Jul 03 11:20:04.862653 ip-10-0-136-0 kubenswrapper[2579]: I0703 11:20:04.862614 2579 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/kuadrant-operator-catalog-j29hp"] Jul 03 11:20:04.865760 ip-10-0-136-0 kubenswrapper[2579]: W0703 11:20:04.865715 2579 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podb26c8c7f_99b4_4644_9aee_b33bc9985fe5.slice/crio-5842315686d184f0919ea3e213217924297f6636efafd763d8b61989dedfe5d0 WatchSource:0}: Error finding container 5842315686d184f0919ea3e213217924297f6636efafd763d8b61989dedfe5d0: Status 404 returned error can't find the container with id 5842315686d184f0919ea3e213217924297f6636efafd763d8b61989dedfe5d0 Jul 03 11:20:05.319018 ip-10-0-136-0 kubenswrapper[2579]: I0703 11:20:05.318971 2579 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/kuadrant-operator-catalog-5wvb2" event={"ID":"8eac4fd4-584f-4773-91b3-9132820e227a","Type":"ContainerStarted","Data":"f80dff146c0e37898243cdc57b18553c6b043565f7c6e44e81b858f16875f607"} Jul 03 11:20:05.320824 ip-10-0-136-0 kubenswrapper[2579]: I0703 11:20:05.320773 2579 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/kuadrant-operator-catalog-j29hp" event={"ID":"b26c8c7f-99b4-4644-9aee-b33bc9985fe5","Type":"ContainerStarted","Data":"5842315686d184f0919ea3e213217924297f6636efafd763d8b61989dedfe5d0"} Jul 03 11:20:07.342180 ip-10-0-136-0 kubenswrapper[2579]: I0703 11:20:07.342072 2579 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/kuadrant-operator-catalog-5wvb2" event={"ID":"8eac4fd4-584f-4773-91b3-9132820e227a","Type":"ContainerStarted","Data":"d9f187c7352d9c45bc46ac1c55857c859d5d23d18917004166b83c406f036299"} Jul 03 11:20:07.342180 ip-10-0-136-0 kubenswrapper[2579]: I0703 11:20:07.342115 2579 kuberuntime_container.go:872] "Killing container with a grace period" pod="kuadrant-system/kuadrant-operator-catalog-5wvb2" podUID="8eac4fd4-584f-4773-91b3-9132820e227a" containerName="registry-server" containerID="cri-o://d9f187c7352d9c45bc46ac1c55857c859d5d23d18917004166b83c406f036299" gracePeriod=2 Jul 03 11:20:07.344178 ip-10-0-136-0 kubenswrapper[2579]: I0703 11:20:07.344146 2579 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/kuadrant-operator-catalog-j29hp" event={"ID":"b26c8c7f-99b4-4644-9aee-b33bc9985fe5","Type":"ContainerStarted","Data":"751fe6b591fe89449c8e9a5780ae68da27591b95083921b640b83050ee3a4d76"} Jul 03 11:20:07.359959 ip-10-0-136-0 kubenswrapper[2579]: I0703 11:20:07.359890 2579 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="kuadrant-system/kuadrant-operator-catalog-5wvb2" podStartSLOduration=1.56910014 podStartE2EDuration="4.359868663s" totalImagesPullingTime="2.790768523s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-03 11:20:03 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-03 11:20:07.357794566 +0000 UTC m=+463.946098885" watchObservedRunningTime="2026-07-03 11:20:07.359868663 +0000 UTC m=+463.948172984" Jul 03 11:20:07.377943 ip-10-0-136-0 kubenswrapper[2579]: I0703 11:20:07.377870 2579 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="kuadrant-system/kuadrant-operator-catalog-j29hp" podStartSLOduration=1.143585566 podStartE2EDuration="3.377850556s" totalImagesPullingTime="2.23426499s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-03 11:20:04 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-03 11:20:07.376156365 +0000 UTC m=+463.964460700" watchObservedRunningTime="2026-07-03 11:20:07.377850556 +0000 UTC m=+463.966154955" Jul 03 11:20:07.594540 ip-10-0-136-0 kubenswrapper[2579]: I0703 11:20:07.594463 2579 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/kuadrant-operator-catalog-5wvb2" Jul 03 11:20:07.696875 ip-10-0-136-0 kubenswrapper[2579]: I0703 11:20:07.696834 2579 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-dhtqj\" (UniqueName: \"kubernetes.io/projected/8eac4fd4-584f-4773-91b3-9132820e227a-kube-api-access-dhtqj\") pod \"8eac4fd4-584f-4773-91b3-9132820e227a\" (UID: \"8eac4fd4-584f-4773-91b3-9132820e227a\") " Jul 03 11:20:07.699632 ip-10-0-136-0 kubenswrapper[2579]: I0703 11:20:07.699587 2579 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/8eac4fd4-584f-4773-91b3-9132820e227a-kube-api-access-dhtqj" (OuterVolumeSpecName: "kube-api-access-dhtqj") pod "8eac4fd4-584f-4773-91b3-9132820e227a" (UID: "8eac4fd4-584f-4773-91b3-9132820e227a"). InnerVolumeSpecName "kube-api-access-dhtqj". PluginName "kubernetes.io/projected", VolumeGIDValue "" Jul 03 11:20:07.798028 ip-10-0-136-0 kubenswrapper[2579]: I0703 11:20:07.797992 2579 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-dhtqj\" (UniqueName: \"kubernetes.io/projected/8eac4fd4-584f-4773-91b3-9132820e227a-kube-api-access-dhtqj\") on node \"ip-10-0-136-0.ec2.internal\" DevicePath \"\"" Jul 03 11:20:08.351513 ip-10-0-136-0 kubenswrapper[2579]: I0703 11:20:08.351471 2579 generic.go:356] "Generic (PLEG): container finished" podID="8eac4fd4-584f-4773-91b3-9132820e227a" containerID="d9f187c7352d9c45bc46ac1c55857c859d5d23d18917004166b83c406f036299" exitCode=0 Jul 03 11:20:08.351995 ip-10-0-136-0 kubenswrapper[2579]: I0703 11:20:08.351523 2579 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/kuadrant-operator-catalog-5wvb2" Jul 03 11:20:08.351995 ip-10-0-136-0 kubenswrapper[2579]: I0703 11:20:08.351558 2579 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/kuadrant-operator-catalog-5wvb2" event={"ID":"8eac4fd4-584f-4773-91b3-9132820e227a","Type":"ContainerDied","Data":"d9f187c7352d9c45bc46ac1c55857c859d5d23d18917004166b83c406f036299"} Jul 03 11:20:08.351995 ip-10-0-136-0 kubenswrapper[2579]: I0703 11:20:08.351614 2579 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/kuadrant-operator-catalog-5wvb2" event={"ID":"8eac4fd4-584f-4773-91b3-9132820e227a","Type":"ContainerDied","Data":"f80dff146c0e37898243cdc57b18553c6b043565f7c6e44e81b858f16875f607"} Jul 03 11:20:08.351995 ip-10-0-136-0 kubenswrapper[2579]: I0703 11:20:08.351642 2579 scope.go:117] "RemoveContainer" containerID="d9f187c7352d9c45bc46ac1c55857c859d5d23d18917004166b83c406f036299" Jul 03 11:20:08.361929 ip-10-0-136-0 kubenswrapper[2579]: I0703 11:20:08.361904 2579 scope.go:117] "RemoveContainer" containerID="d9f187c7352d9c45bc46ac1c55857c859d5d23d18917004166b83c406f036299" Jul 03 11:20:08.362302 ip-10-0-136-0 kubenswrapper[2579]: E0703 11:20:08.362281 2579 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"d9f187c7352d9c45bc46ac1c55857c859d5d23d18917004166b83c406f036299\": container with ID starting with d9f187c7352d9c45bc46ac1c55857c859d5d23d18917004166b83c406f036299 not found: ID does not exist" containerID="d9f187c7352d9c45bc46ac1c55857c859d5d23d18917004166b83c406f036299" Jul 03 11:20:08.362371 ip-10-0-136-0 kubenswrapper[2579]: I0703 11:20:08.362313 2579 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"d9f187c7352d9c45bc46ac1c55857c859d5d23d18917004166b83c406f036299"} err="failed to get container status \"d9f187c7352d9c45bc46ac1c55857c859d5d23d18917004166b83c406f036299\": rpc error: code = NotFound desc = could not find container \"d9f187c7352d9c45bc46ac1c55857c859d5d23d18917004166b83c406f036299\": container with ID starting with d9f187c7352d9c45bc46ac1c55857c859d5d23d18917004166b83c406f036299 not found: ID does not exist" Jul 03 11:20:08.378923 ip-10-0-136-0 kubenswrapper[2579]: I0703 11:20:08.378873 2579 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["kuadrant-system/kuadrant-operator-catalog-5wvb2"] Jul 03 11:20:08.385789 ip-10-0-136-0 kubenswrapper[2579]: I0703 11:20:08.385750 2579 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["kuadrant-system/kuadrant-operator-catalog-5wvb2"] Jul 03 11:20:09.992177 ip-10-0-136-0 kubenswrapper[2579]: I0703 11:20:09.992132 2579 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="8eac4fd4-584f-4773-91b3-9132820e227a" path="/var/lib/kubelet/pods/8eac4fd4-584f-4773-91b3-9132820e227a/volumes" Jul 03 11:20:14.671126 ip-10-0-136-0 kubenswrapper[2579]: I0703 11:20:14.671082 2579 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="kuadrant-system/kuadrant-operator-catalog-j29hp" Jul 03 11:20:14.671591 ip-10-0-136-0 kubenswrapper[2579]: I0703 11:20:14.671161 2579 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="kuadrant-system/kuadrant-operator-catalog-j29hp" Jul 03 11:20:14.696098 ip-10-0-136-0 kubenswrapper[2579]: I0703 11:20:14.696064 2579 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="kuadrant-system/kuadrant-operator-catalog-j29hp" Jul 03 11:20:15.414632 ip-10-0-136-0 kubenswrapper[2579]: I0703 11:20:15.414597 2579 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="kuadrant-system/kuadrant-operator-catalog-j29hp" Jul 03 11:20:19.424746 ip-10-0-136-0 kubenswrapper[2579]: I0703 11:20:19.424681 2579 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["kuadrant-system/9438e18bbad664a016701a3153a6b421e2d977be7eb0117f80bcf45ef1chc8g"] Jul 03 11:20:19.425236 ip-10-0-136-0 kubenswrapper[2579]: I0703 11:20:19.425125 2579 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="8eac4fd4-584f-4773-91b3-9132820e227a" containerName="registry-server" Jul 03 11:20:19.425236 ip-10-0-136-0 kubenswrapper[2579]: I0703 11:20:19.425138 2579 state_mem.go:107] "Deleted CPUSet assignment" podUID="8eac4fd4-584f-4773-91b3-9132820e227a" containerName="registry-server" Jul 03 11:20:19.425321 ip-10-0-136-0 kubenswrapper[2579]: I0703 11:20:19.425236 2579 memory_manager.go:368] "RemoveStaleState removing state" podUID="8eac4fd4-584f-4773-91b3-9132820e227a" containerName="registry-server" Jul 03 11:20:19.428622 ip-10-0-136-0 kubenswrapper[2579]: I0703 11:20:19.428597 2579 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/9438e18bbad664a016701a3153a6b421e2d977be7eb0117f80bcf45ef1chc8g" Jul 03 11:20:19.431605 ip-10-0-136-0 kubenswrapper[2579]: I0703 11:20:19.431575 2579 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"kuadrant-system\"/\"default-dockercfg-wfhwf\"" Jul 03 11:20:19.442420 ip-10-0-136-0 kubenswrapper[2579]: I0703 11:20:19.442373 2579 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/9438e18bbad664a016701a3153a6b421e2d977be7eb0117f80bcf45ef1chc8g"] Jul 03 11:20:19.501517 ip-10-0-136-0 kubenswrapper[2579]: I0703 11:20:19.501469 2579 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"util\" (UniqueName: \"kubernetes.io/empty-dir/5a64e43f-33ee-4779-bc1e-366dc5ef0cc0-util\") pod \"9438e18bbad664a016701a3153a6b421e2d977be7eb0117f80bcf45ef1chc8g\" (UID: \"5a64e43f-33ee-4779-bc1e-366dc5ef0cc0\") " pod="kuadrant-system/9438e18bbad664a016701a3153a6b421e2d977be7eb0117f80bcf45ef1chc8g" Jul 03 11:20:19.501723 ip-10-0-136-0 kubenswrapper[2579]: I0703 11:20:19.501553 2579 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"bundle\" (UniqueName: \"kubernetes.io/empty-dir/5a64e43f-33ee-4779-bc1e-366dc5ef0cc0-bundle\") pod \"9438e18bbad664a016701a3153a6b421e2d977be7eb0117f80bcf45ef1chc8g\" (UID: \"5a64e43f-33ee-4779-bc1e-366dc5ef0cc0\") " pod="kuadrant-system/9438e18bbad664a016701a3153a6b421e2d977be7eb0117f80bcf45ef1chc8g" Jul 03 11:20:19.501723 ip-10-0-136-0 kubenswrapper[2579]: I0703 11:20:19.501678 2579 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-p6qjp\" (UniqueName: \"kubernetes.io/projected/5a64e43f-33ee-4779-bc1e-366dc5ef0cc0-kube-api-access-p6qjp\") pod \"9438e18bbad664a016701a3153a6b421e2d977be7eb0117f80bcf45ef1chc8g\" (UID: \"5a64e43f-33ee-4779-bc1e-366dc5ef0cc0\") " pod="kuadrant-system/9438e18bbad664a016701a3153a6b421e2d977be7eb0117f80bcf45ef1chc8g" Jul 03 11:20:19.602447 ip-10-0-136-0 kubenswrapper[2579]: I0703 11:20:19.602390 2579 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-p6qjp\" (UniqueName: \"kubernetes.io/projected/5a64e43f-33ee-4779-bc1e-366dc5ef0cc0-kube-api-access-p6qjp\") pod \"9438e18bbad664a016701a3153a6b421e2d977be7eb0117f80bcf45ef1chc8g\" (UID: \"5a64e43f-33ee-4779-bc1e-366dc5ef0cc0\") " pod="kuadrant-system/9438e18bbad664a016701a3153a6b421e2d977be7eb0117f80bcf45ef1chc8g" Jul 03 11:20:19.602447 ip-10-0-136-0 kubenswrapper[2579]: I0703 11:20:19.602443 2579 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"util\" (UniqueName: \"kubernetes.io/empty-dir/5a64e43f-33ee-4779-bc1e-366dc5ef0cc0-util\") pod \"9438e18bbad664a016701a3153a6b421e2d977be7eb0117f80bcf45ef1chc8g\" (UID: \"5a64e43f-33ee-4779-bc1e-366dc5ef0cc0\") " pod="kuadrant-system/9438e18bbad664a016701a3153a6b421e2d977be7eb0117f80bcf45ef1chc8g" Jul 03 11:20:19.602693 ip-10-0-136-0 kubenswrapper[2579]: I0703 11:20:19.602476 2579 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"bundle\" (UniqueName: \"kubernetes.io/empty-dir/5a64e43f-33ee-4779-bc1e-366dc5ef0cc0-bundle\") pod \"9438e18bbad664a016701a3153a6b421e2d977be7eb0117f80bcf45ef1chc8g\" (UID: \"5a64e43f-33ee-4779-bc1e-366dc5ef0cc0\") " pod="kuadrant-system/9438e18bbad664a016701a3153a6b421e2d977be7eb0117f80bcf45ef1chc8g" Jul 03 11:20:19.602842 ip-10-0-136-0 kubenswrapper[2579]: I0703 11:20:19.602825 2579 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"bundle\" (UniqueName: \"kubernetes.io/empty-dir/5a64e43f-33ee-4779-bc1e-366dc5ef0cc0-bundle\") pod \"9438e18bbad664a016701a3153a6b421e2d977be7eb0117f80bcf45ef1chc8g\" (UID: \"5a64e43f-33ee-4779-bc1e-366dc5ef0cc0\") " pod="kuadrant-system/9438e18bbad664a016701a3153a6b421e2d977be7eb0117f80bcf45ef1chc8g" Jul 03 11:20:19.602883 ip-10-0-136-0 kubenswrapper[2579]: I0703 11:20:19.602864 2579 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"util\" (UniqueName: \"kubernetes.io/empty-dir/5a64e43f-33ee-4779-bc1e-366dc5ef0cc0-util\") pod \"9438e18bbad664a016701a3153a6b421e2d977be7eb0117f80bcf45ef1chc8g\" (UID: \"5a64e43f-33ee-4779-bc1e-366dc5ef0cc0\") " pod="kuadrant-system/9438e18bbad664a016701a3153a6b421e2d977be7eb0117f80bcf45ef1chc8g" Jul 03 11:20:19.616105 ip-10-0-136-0 kubenswrapper[2579]: I0703 11:20:19.616074 2579 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-p6qjp\" (UniqueName: \"kubernetes.io/projected/5a64e43f-33ee-4779-bc1e-366dc5ef0cc0-kube-api-access-p6qjp\") pod \"9438e18bbad664a016701a3153a6b421e2d977be7eb0117f80bcf45ef1chc8g\" (UID: \"5a64e43f-33ee-4779-bc1e-366dc5ef0cc0\") " pod="kuadrant-system/9438e18bbad664a016701a3153a6b421e2d977be7eb0117f80bcf45ef1chc8g" Jul 03 11:20:19.739755 ip-10-0-136-0 kubenswrapper[2579]: I0703 11:20:19.739662 2579 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/9438e18bbad664a016701a3153a6b421e2d977be7eb0117f80bcf45ef1chc8g" Jul 03 11:20:19.833920 ip-10-0-136-0 kubenswrapper[2579]: I0703 11:20:19.833884 2579 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["kuadrant-system/19cb86e64775c5699d5aacf881a09c2d51e7f55f9e1ff096f2a667c5e0x6knz"] Jul 03 11:20:19.839090 ip-10-0-136-0 kubenswrapper[2579]: I0703 11:20:19.839054 2579 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/19cb86e64775c5699d5aacf881a09c2d51e7f55f9e1ff096f2a667c5e0x6knz" Jul 03 11:20:19.858306 ip-10-0-136-0 kubenswrapper[2579]: I0703 11:20:19.858272 2579 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/19cb86e64775c5699d5aacf881a09c2d51e7f55f9e1ff096f2a667c5e0x6knz"] Jul 03 11:20:19.904360 ip-10-0-136-0 kubenswrapper[2579]: I0703 11:20:19.904322 2579 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/9438e18bbad664a016701a3153a6b421e2d977be7eb0117f80bcf45ef1chc8g"] Jul 03 11:20:19.905414 ip-10-0-136-0 kubenswrapper[2579]: I0703 11:20:19.905390 2579 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-wf584\" (UniqueName: \"kubernetes.io/projected/1742e1ed-aaec-4579-b360-fca0512b5b72-kube-api-access-wf584\") pod \"19cb86e64775c5699d5aacf881a09c2d51e7f55f9e1ff096f2a667c5e0x6knz\" (UID: \"1742e1ed-aaec-4579-b360-fca0512b5b72\") " pod="kuadrant-system/19cb86e64775c5699d5aacf881a09c2d51e7f55f9e1ff096f2a667c5e0x6knz" Jul 03 11:20:19.905519 ip-10-0-136-0 kubenswrapper[2579]: I0703 11:20:19.905502 2579 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"bundle\" (UniqueName: \"kubernetes.io/empty-dir/1742e1ed-aaec-4579-b360-fca0512b5b72-bundle\") pod \"19cb86e64775c5699d5aacf881a09c2d51e7f55f9e1ff096f2a667c5e0x6knz\" (UID: \"1742e1ed-aaec-4579-b360-fca0512b5b72\") " pod="kuadrant-system/19cb86e64775c5699d5aacf881a09c2d51e7f55f9e1ff096f2a667c5e0x6knz" Jul 03 11:20:19.905593 ip-10-0-136-0 kubenswrapper[2579]: I0703 11:20:19.905571 2579 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"util\" (UniqueName: \"kubernetes.io/empty-dir/1742e1ed-aaec-4579-b360-fca0512b5b72-util\") pod \"19cb86e64775c5699d5aacf881a09c2d51e7f55f9e1ff096f2a667c5e0x6knz\" (UID: \"1742e1ed-aaec-4579-b360-fca0512b5b72\") " pod="kuadrant-system/19cb86e64775c5699d5aacf881a09c2d51e7f55f9e1ff096f2a667c5e0x6knz" Jul 03 11:20:19.908182 ip-10-0-136-0 kubenswrapper[2579]: W0703 11:20:19.908151 2579 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod5a64e43f_33ee_4779_bc1e_366dc5ef0cc0.slice/crio-17311a1bb4c798a9c2e240e98be7cd378fe89252baa37ce7e102fd3243e90bcd WatchSource:0}: Error finding container 17311a1bb4c798a9c2e240e98be7cd378fe89252baa37ce7e102fd3243e90bcd: Status 404 returned error can't find the container with id 17311a1bb4c798a9c2e240e98be7cd378fe89252baa37ce7e102fd3243e90bcd Jul 03 11:20:20.006935 ip-10-0-136-0 kubenswrapper[2579]: I0703 11:20:20.006838 2579 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"bundle\" (UniqueName: \"kubernetes.io/empty-dir/1742e1ed-aaec-4579-b360-fca0512b5b72-bundle\") pod \"19cb86e64775c5699d5aacf881a09c2d51e7f55f9e1ff096f2a667c5e0x6knz\" (UID: \"1742e1ed-aaec-4579-b360-fca0512b5b72\") " pod="kuadrant-system/19cb86e64775c5699d5aacf881a09c2d51e7f55f9e1ff096f2a667c5e0x6knz" Jul 03 11:20:20.006935 ip-10-0-136-0 kubenswrapper[2579]: I0703 11:20:20.006891 2579 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"util\" (UniqueName: \"kubernetes.io/empty-dir/1742e1ed-aaec-4579-b360-fca0512b5b72-util\") pod \"19cb86e64775c5699d5aacf881a09c2d51e7f55f9e1ff096f2a667c5e0x6knz\" (UID: \"1742e1ed-aaec-4579-b360-fca0512b5b72\") " pod="kuadrant-system/19cb86e64775c5699d5aacf881a09c2d51e7f55f9e1ff096f2a667c5e0x6knz" Jul 03 11:20:20.007146 ip-10-0-136-0 kubenswrapper[2579]: I0703 11:20:20.006955 2579 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-wf584\" (UniqueName: \"kubernetes.io/projected/1742e1ed-aaec-4579-b360-fca0512b5b72-kube-api-access-wf584\") pod \"19cb86e64775c5699d5aacf881a09c2d51e7f55f9e1ff096f2a667c5e0x6knz\" (UID: \"1742e1ed-aaec-4579-b360-fca0512b5b72\") " pod="kuadrant-system/19cb86e64775c5699d5aacf881a09c2d51e7f55f9e1ff096f2a667c5e0x6knz" Jul 03 11:20:20.007475 ip-10-0-136-0 kubenswrapper[2579]: I0703 11:20:20.007455 2579 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"util\" (UniqueName: \"kubernetes.io/empty-dir/1742e1ed-aaec-4579-b360-fca0512b5b72-util\") pod \"19cb86e64775c5699d5aacf881a09c2d51e7f55f9e1ff096f2a667c5e0x6knz\" (UID: \"1742e1ed-aaec-4579-b360-fca0512b5b72\") " pod="kuadrant-system/19cb86e64775c5699d5aacf881a09c2d51e7f55f9e1ff096f2a667c5e0x6knz" Jul 03 11:20:20.007523 ip-10-0-136-0 kubenswrapper[2579]: I0703 11:20:20.007457 2579 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"bundle\" (UniqueName: \"kubernetes.io/empty-dir/1742e1ed-aaec-4579-b360-fca0512b5b72-bundle\") pod \"19cb86e64775c5699d5aacf881a09c2d51e7f55f9e1ff096f2a667c5e0x6knz\" (UID: \"1742e1ed-aaec-4579-b360-fca0512b5b72\") " pod="kuadrant-system/19cb86e64775c5699d5aacf881a09c2d51e7f55f9e1ff096f2a667c5e0x6knz" Jul 03 11:20:20.018335 ip-10-0-136-0 kubenswrapper[2579]: I0703 11:20:20.018294 2579 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-wf584\" (UniqueName: \"kubernetes.io/projected/1742e1ed-aaec-4579-b360-fca0512b5b72-kube-api-access-wf584\") pod \"19cb86e64775c5699d5aacf881a09c2d51e7f55f9e1ff096f2a667c5e0x6knz\" (UID: \"1742e1ed-aaec-4579-b360-fca0512b5b72\") " pod="kuadrant-system/19cb86e64775c5699d5aacf881a09c2d51e7f55f9e1ff096f2a667c5e0x6knz" Jul 03 11:20:20.154292 ip-10-0-136-0 kubenswrapper[2579]: I0703 11:20:20.154258 2579 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/19cb86e64775c5699d5aacf881a09c2d51e7f55f9e1ff096f2a667c5e0x6knz" Jul 03 11:20:20.254733 ip-10-0-136-0 kubenswrapper[2579]: I0703 11:20:20.254689 2579 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["kuadrant-system/5fc36eb1065777914bfe0ff85f9a202e2a3bafefb563d9e2994474ed73mkshk"] Jul 03 11:20:20.260449 ip-10-0-136-0 kubenswrapper[2579]: I0703 11:20:20.260348 2579 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/5fc36eb1065777914bfe0ff85f9a202e2a3bafefb563d9e2994474ed73mkshk" Jul 03 11:20:20.277012 ip-10-0-136-0 kubenswrapper[2579]: I0703 11:20:20.276976 2579 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/5fc36eb1065777914bfe0ff85f9a202e2a3bafefb563d9e2994474ed73mkshk"] Jul 03 11:20:20.310815 ip-10-0-136-0 kubenswrapper[2579]: I0703 11:20:20.310765 2579 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"bundle\" (UniqueName: \"kubernetes.io/empty-dir/e6f9e082-5d6a-4289-9883-301dbffe6151-bundle\") pod \"5fc36eb1065777914bfe0ff85f9a202e2a3bafefb563d9e2994474ed73mkshk\" (UID: \"e6f9e082-5d6a-4289-9883-301dbffe6151\") " pod="kuadrant-system/5fc36eb1065777914bfe0ff85f9a202e2a3bafefb563d9e2994474ed73mkshk" Jul 03 11:20:20.311038 ip-10-0-136-0 kubenswrapper[2579]: I0703 11:20:20.310836 2579 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"util\" (UniqueName: \"kubernetes.io/empty-dir/e6f9e082-5d6a-4289-9883-301dbffe6151-util\") pod \"5fc36eb1065777914bfe0ff85f9a202e2a3bafefb563d9e2994474ed73mkshk\" (UID: \"e6f9e082-5d6a-4289-9883-301dbffe6151\") " pod="kuadrant-system/5fc36eb1065777914bfe0ff85f9a202e2a3bafefb563d9e2994474ed73mkshk" Jul 03 11:20:20.311038 ip-10-0-136-0 kubenswrapper[2579]: I0703 11:20:20.310964 2579 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-588gr\" (UniqueName: \"kubernetes.io/projected/e6f9e082-5d6a-4289-9883-301dbffe6151-kube-api-access-588gr\") pod \"5fc36eb1065777914bfe0ff85f9a202e2a3bafefb563d9e2994474ed73mkshk\" (UID: \"e6f9e082-5d6a-4289-9883-301dbffe6151\") " pod="kuadrant-system/5fc36eb1065777914bfe0ff85f9a202e2a3bafefb563d9e2994474ed73mkshk" Jul 03 11:20:20.320128 ip-10-0-136-0 kubenswrapper[2579]: I0703 11:20:20.320097 2579 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/19cb86e64775c5699d5aacf881a09c2d51e7f55f9e1ff096f2a667c5e0x6knz"] Jul 03 11:20:20.322474 ip-10-0-136-0 kubenswrapper[2579]: W0703 11:20:20.322442 2579 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod1742e1ed_aaec_4579_b360_fca0512b5b72.slice/crio-0d02a9d8eda70059f2e902266559c4dabb484ea885446c4ab8eee3965bd965bd WatchSource:0}: Error finding container 0d02a9d8eda70059f2e902266559c4dabb484ea885446c4ab8eee3965bd965bd: Status 404 returned error can't find the container with id 0d02a9d8eda70059f2e902266559c4dabb484ea885446c4ab8eee3965bd965bd Jul 03 11:20:20.411694 ip-10-0-136-0 kubenswrapper[2579]: I0703 11:20:20.411645 2579 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"bundle\" (UniqueName: \"kubernetes.io/empty-dir/e6f9e082-5d6a-4289-9883-301dbffe6151-bundle\") pod \"5fc36eb1065777914bfe0ff85f9a202e2a3bafefb563d9e2994474ed73mkshk\" (UID: \"e6f9e082-5d6a-4289-9883-301dbffe6151\") " pod="kuadrant-system/5fc36eb1065777914bfe0ff85f9a202e2a3bafefb563d9e2994474ed73mkshk" Jul 03 11:20:20.411883 ip-10-0-136-0 kubenswrapper[2579]: I0703 11:20:20.411715 2579 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"util\" (UniqueName: \"kubernetes.io/empty-dir/e6f9e082-5d6a-4289-9883-301dbffe6151-util\") pod \"5fc36eb1065777914bfe0ff85f9a202e2a3bafefb563d9e2994474ed73mkshk\" (UID: \"e6f9e082-5d6a-4289-9883-301dbffe6151\") " pod="kuadrant-system/5fc36eb1065777914bfe0ff85f9a202e2a3bafefb563d9e2994474ed73mkshk" Jul 03 11:20:20.411883 ip-10-0-136-0 kubenswrapper[2579]: I0703 11:20:20.411759 2579 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-588gr\" (UniqueName: \"kubernetes.io/projected/e6f9e082-5d6a-4289-9883-301dbffe6151-kube-api-access-588gr\") pod \"5fc36eb1065777914bfe0ff85f9a202e2a3bafefb563d9e2994474ed73mkshk\" (UID: \"e6f9e082-5d6a-4289-9883-301dbffe6151\") " pod="kuadrant-system/5fc36eb1065777914bfe0ff85f9a202e2a3bafefb563d9e2994474ed73mkshk" Jul 03 11:20:20.412147 ip-10-0-136-0 kubenswrapper[2579]: I0703 11:20:20.412115 2579 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"util\" (UniqueName: \"kubernetes.io/empty-dir/e6f9e082-5d6a-4289-9883-301dbffe6151-util\") pod \"5fc36eb1065777914bfe0ff85f9a202e2a3bafefb563d9e2994474ed73mkshk\" (UID: \"e6f9e082-5d6a-4289-9883-301dbffe6151\") " pod="kuadrant-system/5fc36eb1065777914bfe0ff85f9a202e2a3bafefb563d9e2994474ed73mkshk" Jul 03 11:20:20.412147 ip-10-0-136-0 kubenswrapper[2579]: I0703 11:20:20.412127 2579 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"bundle\" (UniqueName: \"kubernetes.io/empty-dir/e6f9e082-5d6a-4289-9883-301dbffe6151-bundle\") pod \"5fc36eb1065777914bfe0ff85f9a202e2a3bafefb563d9e2994474ed73mkshk\" (UID: \"e6f9e082-5d6a-4289-9883-301dbffe6151\") " pod="kuadrant-system/5fc36eb1065777914bfe0ff85f9a202e2a3bafefb563d9e2994474ed73mkshk" Jul 03 11:20:20.419674 ip-10-0-136-0 kubenswrapper[2579]: I0703 11:20:20.419621 2579 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/19cb86e64775c5699d5aacf881a09c2d51e7f55f9e1ff096f2a667c5e0x6knz" event={"ID":"1742e1ed-aaec-4579-b360-fca0512b5b72","Type":"ContainerStarted","Data":"ea54d854e4510cb4404d82ba00b2e9aa710d0974cd1a4696ad3560abfab9631c"} Jul 03 11:20:20.419945 ip-10-0-136-0 kubenswrapper[2579]: I0703 11:20:20.419706 2579 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/19cb86e64775c5699d5aacf881a09c2d51e7f55f9e1ff096f2a667c5e0x6knz" event={"ID":"1742e1ed-aaec-4579-b360-fca0512b5b72","Type":"ContainerStarted","Data":"0d02a9d8eda70059f2e902266559c4dabb484ea885446c4ab8eee3965bd965bd"} Jul 03 11:20:20.421730 ip-10-0-136-0 kubenswrapper[2579]: I0703 11:20:20.421697 2579 generic.go:356] "Generic (PLEG): container finished" podID="5a64e43f-33ee-4779-bc1e-366dc5ef0cc0" containerID="0916ef416c969fa90c732790491dd0ff0f4043e8fc777da6114427ff04a5894a" exitCode=0 Jul 03 11:20:20.421911 ip-10-0-136-0 kubenswrapper[2579]: I0703 11:20:20.421768 2579 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/9438e18bbad664a016701a3153a6b421e2d977be7eb0117f80bcf45ef1chc8g" event={"ID":"5a64e43f-33ee-4779-bc1e-366dc5ef0cc0","Type":"ContainerDied","Data":"0916ef416c969fa90c732790491dd0ff0f4043e8fc777da6114427ff04a5894a"} Jul 03 11:20:20.421911 ip-10-0-136-0 kubenswrapper[2579]: I0703 11:20:20.421823 2579 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/9438e18bbad664a016701a3153a6b421e2d977be7eb0117f80bcf45ef1chc8g" event={"ID":"5a64e43f-33ee-4779-bc1e-366dc5ef0cc0","Type":"ContainerStarted","Data":"17311a1bb4c798a9c2e240e98be7cd378fe89252baa37ce7e102fd3243e90bcd"} Jul 03 11:20:20.423059 ip-10-0-136-0 kubenswrapper[2579]: I0703 11:20:20.423034 2579 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-588gr\" (UniqueName: \"kubernetes.io/projected/e6f9e082-5d6a-4289-9883-301dbffe6151-kube-api-access-588gr\") pod \"5fc36eb1065777914bfe0ff85f9a202e2a3bafefb563d9e2994474ed73mkshk\" (UID: \"e6f9e082-5d6a-4289-9883-301dbffe6151\") " pod="kuadrant-system/5fc36eb1065777914bfe0ff85f9a202e2a3bafefb563d9e2994474ed73mkshk" Jul 03 11:20:20.575048 ip-10-0-136-0 kubenswrapper[2579]: I0703 11:20:20.574999 2579 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/5fc36eb1065777914bfe0ff85f9a202e2a3bafefb563d9e2994474ed73mkshk" Jul 03 11:20:20.633015 ip-10-0-136-0 kubenswrapper[2579]: I0703 11:20:20.632972 2579 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["kuadrant-system/0acee64185f523d1d1272e9af2e4d9333e0dcde792ba30e1fa9605b7599bmjs"] Jul 03 11:20:20.638857 ip-10-0-136-0 kubenswrapper[2579]: I0703 11:20:20.638819 2579 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/0acee64185f523d1d1272e9af2e4d9333e0dcde792ba30e1fa9605b7599bmjs" Jul 03 11:20:20.656045 ip-10-0-136-0 kubenswrapper[2579]: I0703 11:20:20.656009 2579 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/0acee64185f523d1d1272e9af2e4d9333e0dcde792ba30e1fa9605b7599bmjs"] Jul 03 11:20:20.714573 ip-10-0-136-0 kubenswrapper[2579]: I0703 11:20:20.714532 2579 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"util\" (UniqueName: \"kubernetes.io/empty-dir/805645ad-622e-4f54-8870-f15cd93c64f0-util\") pod \"0acee64185f523d1d1272e9af2e4d9333e0dcde792ba30e1fa9605b7599bmjs\" (UID: \"805645ad-622e-4f54-8870-f15cd93c64f0\") " pod="kuadrant-system/0acee64185f523d1d1272e9af2e4d9333e0dcde792ba30e1fa9605b7599bmjs" Jul 03 11:20:20.714856 ip-10-0-136-0 kubenswrapper[2579]: I0703 11:20:20.714594 2579 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-w6tlr\" (UniqueName: \"kubernetes.io/projected/805645ad-622e-4f54-8870-f15cd93c64f0-kube-api-access-w6tlr\") pod \"0acee64185f523d1d1272e9af2e4d9333e0dcde792ba30e1fa9605b7599bmjs\" (UID: \"805645ad-622e-4f54-8870-f15cd93c64f0\") " pod="kuadrant-system/0acee64185f523d1d1272e9af2e4d9333e0dcde792ba30e1fa9605b7599bmjs" Jul 03 11:20:20.714856 ip-10-0-136-0 kubenswrapper[2579]: I0703 11:20:20.714645 2579 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"bundle\" (UniqueName: \"kubernetes.io/empty-dir/805645ad-622e-4f54-8870-f15cd93c64f0-bundle\") pod \"0acee64185f523d1d1272e9af2e4d9333e0dcde792ba30e1fa9605b7599bmjs\" (UID: \"805645ad-622e-4f54-8870-f15cd93c64f0\") " pod="kuadrant-system/0acee64185f523d1d1272e9af2e4d9333e0dcde792ba30e1fa9605b7599bmjs" Jul 03 11:20:20.740950 ip-10-0-136-0 kubenswrapper[2579]: I0703 11:20:20.740917 2579 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/5fc36eb1065777914bfe0ff85f9a202e2a3bafefb563d9e2994474ed73mkshk"] Jul 03 11:20:20.743237 ip-10-0-136-0 kubenswrapper[2579]: W0703 11:20:20.743177 2579 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pode6f9e082_5d6a_4289_9883_301dbffe6151.slice/crio-aa711c4e06c3eaaa31161cdf60c760e2b29ea98d4b1b1e433bdafc2cde1e9cbe WatchSource:0}: Error finding container aa711c4e06c3eaaa31161cdf60c760e2b29ea98d4b1b1e433bdafc2cde1e9cbe: Status 404 returned error can't find the container with id aa711c4e06c3eaaa31161cdf60c760e2b29ea98d4b1b1e433bdafc2cde1e9cbe Jul 03 11:20:20.815354 ip-10-0-136-0 kubenswrapper[2579]: I0703 11:20:20.815312 2579 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"util\" (UniqueName: \"kubernetes.io/empty-dir/805645ad-622e-4f54-8870-f15cd93c64f0-util\") pod \"0acee64185f523d1d1272e9af2e4d9333e0dcde792ba30e1fa9605b7599bmjs\" (UID: \"805645ad-622e-4f54-8870-f15cd93c64f0\") " pod="kuadrant-system/0acee64185f523d1d1272e9af2e4d9333e0dcde792ba30e1fa9605b7599bmjs" Jul 03 11:20:20.815760 ip-10-0-136-0 kubenswrapper[2579]: I0703 11:20:20.815387 2579 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-w6tlr\" (UniqueName: \"kubernetes.io/projected/805645ad-622e-4f54-8870-f15cd93c64f0-kube-api-access-w6tlr\") pod \"0acee64185f523d1d1272e9af2e4d9333e0dcde792ba30e1fa9605b7599bmjs\" (UID: \"805645ad-622e-4f54-8870-f15cd93c64f0\") " pod="kuadrant-system/0acee64185f523d1d1272e9af2e4d9333e0dcde792ba30e1fa9605b7599bmjs" Jul 03 11:20:20.815760 ip-10-0-136-0 kubenswrapper[2579]: I0703 11:20:20.815429 2579 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"bundle\" (UniqueName: \"kubernetes.io/empty-dir/805645ad-622e-4f54-8870-f15cd93c64f0-bundle\") pod \"0acee64185f523d1d1272e9af2e4d9333e0dcde792ba30e1fa9605b7599bmjs\" (UID: \"805645ad-622e-4f54-8870-f15cd93c64f0\") " pod="kuadrant-system/0acee64185f523d1d1272e9af2e4d9333e0dcde792ba30e1fa9605b7599bmjs" Jul 03 11:20:20.815760 ip-10-0-136-0 kubenswrapper[2579]: I0703 11:20:20.815737 2579 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"util\" (UniqueName: \"kubernetes.io/empty-dir/805645ad-622e-4f54-8870-f15cd93c64f0-util\") pod \"0acee64185f523d1d1272e9af2e4d9333e0dcde792ba30e1fa9605b7599bmjs\" (UID: \"805645ad-622e-4f54-8870-f15cd93c64f0\") " pod="kuadrant-system/0acee64185f523d1d1272e9af2e4d9333e0dcde792ba30e1fa9605b7599bmjs" Jul 03 11:20:20.815898 ip-10-0-136-0 kubenswrapper[2579]: I0703 11:20:20.815760 2579 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"bundle\" (UniqueName: \"kubernetes.io/empty-dir/805645ad-622e-4f54-8870-f15cd93c64f0-bundle\") pod \"0acee64185f523d1d1272e9af2e4d9333e0dcde792ba30e1fa9605b7599bmjs\" (UID: \"805645ad-622e-4f54-8870-f15cd93c64f0\") " pod="kuadrant-system/0acee64185f523d1d1272e9af2e4d9333e0dcde792ba30e1fa9605b7599bmjs" Jul 03 11:20:20.827457 ip-10-0-136-0 kubenswrapper[2579]: I0703 11:20:20.827374 2579 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-w6tlr\" (UniqueName: \"kubernetes.io/projected/805645ad-622e-4f54-8870-f15cd93c64f0-kube-api-access-w6tlr\") pod \"0acee64185f523d1d1272e9af2e4d9333e0dcde792ba30e1fa9605b7599bmjs\" (UID: \"805645ad-622e-4f54-8870-f15cd93c64f0\") " pod="kuadrant-system/0acee64185f523d1d1272e9af2e4d9333e0dcde792ba30e1fa9605b7599bmjs" Jul 03 11:20:20.951451 ip-10-0-136-0 kubenswrapper[2579]: I0703 11:20:20.951406 2579 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/0acee64185f523d1d1272e9af2e4d9333e0dcde792ba30e1fa9605b7599bmjs" Jul 03 11:20:21.112714 ip-10-0-136-0 kubenswrapper[2579]: I0703 11:20:21.112687 2579 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/0acee64185f523d1d1272e9af2e4d9333e0dcde792ba30e1fa9605b7599bmjs"] Jul 03 11:20:21.115084 ip-10-0-136-0 kubenswrapper[2579]: W0703 11:20:21.115048 2579 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod805645ad_622e_4f54_8870_f15cd93c64f0.slice/crio-a1dcebb875d92230ae11659e5263368be0c313f8c1d04dfac1b4855eee1e6ba8 WatchSource:0}: Error finding container a1dcebb875d92230ae11659e5263368be0c313f8c1d04dfac1b4855eee1e6ba8: Status 404 returned error can't find the container with id a1dcebb875d92230ae11659e5263368be0c313f8c1d04dfac1b4855eee1e6ba8 Jul 03 11:20:21.430114 ip-10-0-136-0 kubenswrapper[2579]: I0703 11:20:21.430068 2579 generic.go:356] "Generic (PLEG): container finished" podID="1742e1ed-aaec-4579-b360-fca0512b5b72" containerID="ea54d854e4510cb4404d82ba00b2e9aa710d0974cd1a4696ad3560abfab9631c" exitCode=0 Jul 03 11:20:21.430468 ip-10-0-136-0 kubenswrapper[2579]: I0703 11:20:21.430114 2579 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/19cb86e64775c5699d5aacf881a09c2d51e7f55f9e1ff096f2a667c5e0x6knz" event={"ID":"1742e1ed-aaec-4579-b360-fca0512b5b72","Type":"ContainerDied","Data":"ea54d854e4510cb4404d82ba00b2e9aa710d0974cd1a4696ad3560abfab9631c"} Jul 03 11:20:21.432268 ip-10-0-136-0 kubenswrapper[2579]: I0703 11:20:21.432234 2579 generic.go:356] "Generic (PLEG): container finished" podID="805645ad-622e-4f54-8870-f15cd93c64f0" containerID="e95a724a3eac7b6ad20f378c231722de618822c1833e9062481033142649f2f7" exitCode=0 Jul 03 11:20:21.432394 ip-10-0-136-0 kubenswrapper[2579]: I0703 11:20:21.432345 2579 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/0acee64185f523d1d1272e9af2e4d9333e0dcde792ba30e1fa9605b7599bmjs" event={"ID":"805645ad-622e-4f54-8870-f15cd93c64f0","Type":"ContainerDied","Data":"e95a724a3eac7b6ad20f378c231722de618822c1833e9062481033142649f2f7"} Jul 03 11:20:21.432452 ip-10-0-136-0 kubenswrapper[2579]: I0703 11:20:21.432409 2579 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/0acee64185f523d1d1272e9af2e4d9333e0dcde792ba30e1fa9605b7599bmjs" event={"ID":"805645ad-622e-4f54-8870-f15cd93c64f0","Type":"ContainerStarted","Data":"a1dcebb875d92230ae11659e5263368be0c313f8c1d04dfac1b4855eee1e6ba8"} Jul 03 11:20:21.434516 ip-10-0-136-0 kubenswrapper[2579]: I0703 11:20:21.434486 2579 generic.go:356] "Generic (PLEG): container finished" podID="e6f9e082-5d6a-4289-9883-301dbffe6151" containerID="a692faf94438ddcdffb1b1c3b7cb4f7b2a9c1ba9f6aa245974b9ec1b6452242a" exitCode=0 Jul 03 11:20:21.434645 ip-10-0-136-0 kubenswrapper[2579]: I0703 11:20:21.434576 2579 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/5fc36eb1065777914bfe0ff85f9a202e2a3bafefb563d9e2994474ed73mkshk" event={"ID":"e6f9e082-5d6a-4289-9883-301dbffe6151","Type":"ContainerDied","Data":"a692faf94438ddcdffb1b1c3b7cb4f7b2a9c1ba9f6aa245974b9ec1b6452242a"} Jul 03 11:20:21.434645 ip-10-0-136-0 kubenswrapper[2579]: I0703 11:20:21.434653 2579 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/5fc36eb1065777914bfe0ff85f9a202e2a3bafefb563d9e2994474ed73mkshk" event={"ID":"e6f9e082-5d6a-4289-9883-301dbffe6151","Type":"ContainerStarted","Data":"aa711c4e06c3eaaa31161cdf60c760e2b29ea98d4b1b1e433bdafc2cde1e9cbe"} Jul 03 11:20:21.437049 ip-10-0-136-0 kubenswrapper[2579]: I0703 11:20:21.437019 2579 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/9438e18bbad664a016701a3153a6b421e2d977be7eb0117f80bcf45ef1chc8g" event={"ID":"5a64e43f-33ee-4779-bc1e-366dc5ef0cc0","Type":"ContainerStarted","Data":"5dc72b423ca8250de46d9d652f9abadfb4a1ae05c95f498c4428fd8c18660342"} Jul 03 11:20:22.444657 ip-10-0-136-0 kubenswrapper[2579]: I0703 11:20:22.444621 2579 generic.go:356] "Generic (PLEG): container finished" podID="1742e1ed-aaec-4579-b360-fca0512b5b72" containerID="4b1c06e74aafeea8da6f74ce78c7fad56ddcc7b99c0b04d704f6b9c309ee58a1" exitCode=0 Jul 03 11:20:22.445059 ip-10-0-136-0 kubenswrapper[2579]: I0703 11:20:22.444708 2579 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/19cb86e64775c5699d5aacf881a09c2d51e7f55f9e1ff096f2a667c5e0x6knz" event={"ID":"1742e1ed-aaec-4579-b360-fca0512b5b72","Type":"ContainerDied","Data":"4b1c06e74aafeea8da6f74ce78c7fad56ddcc7b99c0b04d704f6b9c309ee58a1"} Jul 03 11:20:22.447001 ip-10-0-136-0 kubenswrapper[2579]: I0703 11:20:22.446946 2579 generic.go:356] "Generic (PLEG): container finished" podID="5a64e43f-33ee-4779-bc1e-366dc5ef0cc0" containerID="5dc72b423ca8250de46d9d652f9abadfb4a1ae05c95f498c4428fd8c18660342" exitCode=0 Jul 03 11:20:22.447164 ip-10-0-136-0 kubenswrapper[2579]: I0703 11:20:22.447084 2579 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/9438e18bbad664a016701a3153a6b421e2d977be7eb0117f80bcf45ef1chc8g" event={"ID":"5a64e43f-33ee-4779-bc1e-366dc5ef0cc0","Type":"ContainerDied","Data":"5dc72b423ca8250de46d9d652f9abadfb4a1ae05c95f498c4428fd8c18660342"} Jul 03 11:20:23.455503 ip-10-0-136-0 kubenswrapper[2579]: I0703 11:20:23.455461 2579 generic.go:356] "Generic (PLEG): container finished" podID="1742e1ed-aaec-4579-b360-fca0512b5b72" containerID="dccebda3bf5bda61717e5b6eb27849a079baec8d0c838125d3577d92f82d7121" exitCode=0 Jul 03 11:20:23.455981 ip-10-0-136-0 kubenswrapper[2579]: I0703 11:20:23.455550 2579 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/19cb86e64775c5699d5aacf881a09c2d51e7f55f9e1ff096f2a667c5e0x6knz" event={"ID":"1742e1ed-aaec-4579-b360-fca0512b5b72","Type":"ContainerDied","Data":"dccebda3bf5bda61717e5b6eb27849a079baec8d0c838125d3577d92f82d7121"} Jul 03 11:20:23.457768 ip-10-0-136-0 kubenswrapper[2579]: I0703 11:20:23.457734 2579 generic.go:356] "Generic (PLEG): container finished" podID="805645ad-622e-4f54-8870-f15cd93c64f0" containerID="9ba539961ba8cd5314d17c9e89853805401bd4918ff6abd90d2153a138791523" exitCode=0 Jul 03 11:20:23.457927 ip-10-0-136-0 kubenswrapper[2579]: I0703 11:20:23.457801 2579 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/0acee64185f523d1d1272e9af2e4d9333e0dcde792ba30e1fa9605b7599bmjs" event={"ID":"805645ad-622e-4f54-8870-f15cd93c64f0","Type":"ContainerDied","Data":"9ba539961ba8cd5314d17c9e89853805401bd4918ff6abd90d2153a138791523"} Jul 03 11:20:23.459904 ip-10-0-136-0 kubenswrapper[2579]: I0703 11:20:23.459876 2579 generic.go:356] "Generic (PLEG): container finished" podID="e6f9e082-5d6a-4289-9883-301dbffe6151" containerID="5b60d7660e05aa6aad8ce0ef51b9244958af77b083089f779f19d80b3415cda3" exitCode=0 Jul 03 11:20:23.460023 ip-10-0-136-0 kubenswrapper[2579]: I0703 11:20:23.460003 2579 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/5fc36eb1065777914bfe0ff85f9a202e2a3bafefb563d9e2994474ed73mkshk" event={"ID":"e6f9e082-5d6a-4289-9883-301dbffe6151","Type":"ContainerDied","Data":"5b60d7660e05aa6aad8ce0ef51b9244958af77b083089f779f19d80b3415cda3"} Jul 03 11:20:23.462426 ip-10-0-136-0 kubenswrapper[2579]: I0703 11:20:23.462401 2579 generic.go:356] "Generic (PLEG): container finished" podID="5a64e43f-33ee-4779-bc1e-366dc5ef0cc0" containerID="eea75db49347cb128ee84eebfced50272ae43433e29beee2587a0e505605e293" exitCode=0 Jul 03 11:20:23.462513 ip-10-0-136-0 kubenswrapper[2579]: I0703 11:20:23.462466 2579 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/9438e18bbad664a016701a3153a6b421e2d977be7eb0117f80bcf45ef1chc8g" event={"ID":"5a64e43f-33ee-4779-bc1e-366dc5ef0cc0","Type":"ContainerDied","Data":"eea75db49347cb128ee84eebfced50272ae43433e29beee2587a0e505605e293"} Jul 03 11:20:24.471521 ip-10-0-136-0 kubenswrapper[2579]: I0703 11:20:24.471480 2579 generic.go:356] "Generic (PLEG): container finished" podID="805645ad-622e-4f54-8870-f15cd93c64f0" containerID="26925b2cf1aba2cbda4a1a22e9c44d0dfca1af7b69f8c4444a1c916188a0ee6b" exitCode=0 Jul 03 11:20:24.471973 ip-10-0-136-0 kubenswrapper[2579]: I0703 11:20:24.471574 2579 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/0acee64185f523d1d1272e9af2e4d9333e0dcde792ba30e1fa9605b7599bmjs" event={"ID":"805645ad-622e-4f54-8870-f15cd93c64f0","Type":"ContainerDied","Data":"26925b2cf1aba2cbda4a1a22e9c44d0dfca1af7b69f8c4444a1c916188a0ee6b"} Jul 03 11:20:24.473796 ip-10-0-136-0 kubenswrapper[2579]: I0703 11:20:24.473765 2579 generic.go:356] "Generic (PLEG): container finished" podID="e6f9e082-5d6a-4289-9883-301dbffe6151" containerID="9342c6ea73af69c54a3b1ab7a4392f167de24e374d19614982925655781ac744" exitCode=0 Jul 03 11:20:24.473939 ip-10-0-136-0 kubenswrapper[2579]: I0703 11:20:24.473847 2579 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/5fc36eb1065777914bfe0ff85f9a202e2a3bafefb563d9e2994474ed73mkshk" event={"ID":"e6f9e082-5d6a-4289-9883-301dbffe6151","Type":"ContainerDied","Data":"9342c6ea73af69c54a3b1ab7a4392f167de24e374d19614982925655781ac744"} Jul 03 11:20:24.627561 ip-10-0-136-0 kubenswrapper[2579]: I0703 11:20:24.627525 2579 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/19cb86e64775c5699d5aacf881a09c2d51e7f55f9e1ff096f2a667c5e0x6knz" Jul 03 11:20:24.655152 ip-10-0-136-0 kubenswrapper[2579]: I0703 11:20:24.655121 2579 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/9438e18bbad664a016701a3153a6b421e2d977be7eb0117f80bcf45ef1chc8g" Jul 03 11:20:24.756385 ip-10-0-136-0 kubenswrapper[2579]: I0703 11:20:24.756275 2579 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"bundle\" (UniqueName: \"kubernetes.io/empty-dir/1742e1ed-aaec-4579-b360-fca0512b5b72-bundle\") pod \"1742e1ed-aaec-4579-b360-fca0512b5b72\" (UID: \"1742e1ed-aaec-4579-b360-fca0512b5b72\") " Jul 03 11:20:24.756385 ip-10-0-136-0 kubenswrapper[2579]: I0703 11:20:24.756334 2579 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"util\" (UniqueName: \"kubernetes.io/empty-dir/1742e1ed-aaec-4579-b360-fca0512b5b72-util\") pod \"1742e1ed-aaec-4579-b360-fca0512b5b72\" (UID: \"1742e1ed-aaec-4579-b360-fca0512b5b72\") " Jul 03 11:20:24.756385 ip-10-0-136-0 kubenswrapper[2579]: I0703 11:20:24.756376 2579 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"bundle\" (UniqueName: \"kubernetes.io/empty-dir/5a64e43f-33ee-4779-bc1e-366dc5ef0cc0-bundle\") pod \"5a64e43f-33ee-4779-bc1e-366dc5ef0cc0\" (UID: \"5a64e43f-33ee-4779-bc1e-366dc5ef0cc0\") " Jul 03 11:20:24.756651 ip-10-0-136-0 kubenswrapper[2579]: I0703 11:20:24.756419 2579 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"util\" (UniqueName: \"kubernetes.io/empty-dir/5a64e43f-33ee-4779-bc1e-366dc5ef0cc0-util\") pod \"5a64e43f-33ee-4779-bc1e-366dc5ef0cc0\" (UID: \"5a64e43f-33ee-4779-bc1e-366dc5ef0cc0\") " Jul 03 11:20:24.756651 ip-10-0-136-0 kubenswrapper[2579]: I0703 11:20:24.756457 2579 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-wf584\" (UniqueName: \"kubernetes.io/projected/1742e1ed-aaec-4579-b360-fca0512b5b72-kube-api-access-wf584\") pod \"1742e1ed-aaec-4579-b360-fca0512b5b72\" (UID: \"1742e1ed-aaec-4579-b360-fca0512b5b72\") " Jul 03 11:20:24.756651 ip-10-0-136-0 kubenswrapper[2579]: I0703 11:20:24.756509 2579 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-p6qjp\" (UniqueName: \"kubernetes.io/projected/5a64e43f-33ee-4779-bc1e-366dc5ef0cc0-kube-api-access-p6qjp\") pod \"5a64e43f-33ee-4779-bc1e-366dc5ef0cc0\" (UID: \"5a64e43f-33ee-4779-bc1e-366dc5ef0cc0\") " Jul 03 11:20:24.756956 ip-10-0-136-0 kubenswrapper[2579]: I0703 11:20:24.756911 2579 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/1742e1ed-aaec-4579-b360-fca0512b5b72-bundle" (OuterVolumeSpecName: "bundle") pod "1742e1ed-aaec-4579-b360-fca0512b5b72" (UID: "1742e1ed-aaec-4579-b360-fca0512b5b72"). InnerVolumeSpecName "bundle". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Jul 03 11:20:24.757089 ip-10-0-136-0 kubenswrapper[2579]: I0703 11:20:24.757015 2579 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/5a64e43f-33ee-4779-bc1e-366dc5ef0cc0-bundle" (OuterVolumeSpecName: "bundle") pod "5a64e43f-33ee-4779-bc1e-366dc5ef0cc0" (UID: "5a64e43f-33ee-4779-bc1e-366dc5ef0cc0"). InnerVolumeSpecName "bundle". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Jul 03 11:20:24.759310 ip-10-0-136-0 kubenswrapper[2579]: I0703 11:20:24.759270 2579 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/5a64e43f-33ee-4779-bc1e-366dc5ef0cc0-kube-api-access-p6qjp" (OuterVolumeSpecName: "kube-api-access-p6qjp") pod "5a64e43f-33ee-4779-bc1e-366dc5ef0cc0" (UID: "5a64e43f-33ee-4779-bc1e-366dc5ef0cc0"). InnerVolumeSpecName "kube-api-access-p6qjp". PluginName "kubernetes.io/projected", VolumeGIDValue "" Jul 03 11:20:24.759484 ip-10-0-136-0 kubenswrapper[2579]: I0703 11:20:24.759415 2579 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/1742e1ed-aaec-4579-b360-fca0512b5b72-kube-api-access-wf584" (OuterVolumeSpecName: "kube-api-access-wf584") pod "1742e1ed-aaec-4579-b360-fca0512b5b72" (UID: "1742e1ed-aaec-4579-b360-fca0512b5b72"). InnerVolumeSpecName "kube-api-access-wf584". PluginName "kubernetes.io/projected", VolumeGIDValue "" Jul 03 11:20:24.762616 ip-10-0-136-0 kubenswrapper[2579]: I0703 11:20:24.762576 2579 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/5a64e43f-33ee-4779-bc1e-366dc5ef0cc0-util" (OuterVolumeSpecName: "util") pod "5a64e43f-33ee-4779-bc1e-366dc5ef0cc0" (UID: "5a64e43f-33ee-4779-bc1e-366dc5ef0cc0"). InnerVolumeSpecName "util". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Jul 03 11:20:24.762950 ip-10-0-136-0 kubenswrapper[2579]: I0703 11:20:24.762927 2579 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/1742e1ed-aaec-4579-b360-fca0512b5b72-util" (OuterVolumeSpecName: "util") pod "1742e1ed-aaec-4579-b360-fca0512b5b72" (UID: "1742e1ed-aaec-4579-b360-fca0512b5b72"). InnerVolumeSpecName "util". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Jul 03 11:20:24.857620 ip-10-0-136-0 kubenswrapper[2579]: I0703 11:20:24.857575 2579 reconciler_common.go:299] "Volume detached for volume \"bundle\" (UniqueName: \"kubernetes.io/empty-dir/1742e1ed-aaec-4579-b360-fca0512b5b72-bundle\") on node \"ip-10-0-136-0.ec2.internal\" DevicePath \"\"" Jul 03 11:20:24.857620 ip-10-0-136-0 kubenswrapper[2579]: I0703 11:20:24.857612 2579 reconciler_common.go:299] "Volume detached for volume \"util\" (UniqueName: \"kubernetes.io/empty-dir/1742e1ed-aaec-4579-b360-fca0512b5b72-util\") on node \"ip-10-0-136-0.ec2.internal\" DevicePath \"\"" Jul 03 11:20:24.857620 ip-10-0-136-0 kubenswrapper[2579]: I0703 11:20:24.857621 2579 reconciler_common.go:299] "Volume detached for volume \"bundle\" (UniqueName: \"kubernetes.io/empty-dir/5a64e43f-33ee-4779-bc1e-366dc5ef0cc0-bundle\") on node \"ip-10-0-136-0.ec2.internal\" DevicePath \"\"" Jul 03 11:20:24.857620 ip-10-0-136-0 kubenswrapper[2579]: I0703 11:20:24.857629 2579 reconciler_common.go:299] "Volume detached for volume \"util\" (UniqueName: \"kubernetes.io/empty-dir/5a64e43f-33ee-4779-bc1e-366dc5ef0cc0-util\") on node \"ip-10-0-136-0.ec2.internal\" DevicePath \"\"" Jul 03 11:20:24.857890 ip-10-0-136-0 kubenswrapper[2579]: I0703 11:20:24.857638 2579 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-wf584\" (UniqueName: \"kubernetes.io/projected/1742e1ed-aaec-4579-b360-fca0512b5b72-kube-api-access-wf584\") on node \"ip-10-0-136-0.ec2.internal\" DevicePath \"\"" Jul 03 11:20:24.857890 ip-10-0-136-0 kubenswrapper[2579]: I0703 11:20:24.857654 2579 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-p6qjp\" (UniqueName: \"kubernetes.io/projected/5a64e43f-33ee-4779-bc1e-366dc5ef0cc0-kube-api-access-p6qjp\") on node \"ip-10-0-136-0.ec2.internal\" DevicePath \"\"" Jul 03 11:20:25.481075 ip-10-0-136-0 kubenswrapper[2579]: I0703 11:20:25.481041 2579 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/19cb86e64775c5699d5aacf881a09c2d51e7f55f9e1ff096f2a667c5e0x6knz" Jul 03 11:20:25.481075 ip-10-0-136-0 kubenswrapper[2579]: I0703 11:20:25.481053 2579 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/19cb86e64775c5699d5aacf881a09c2d51e7f55f9e1ff096f2a667c5e0x6knz" event={"ID":"1742e1ed-aaec-4579-b360-fca0512b5b72","Type":"ContainerDied","Data":"0d02a9d8eda70059f2e902266559c4dabb484ea885446c4ab8eee3965bd965bd"} Jul 03 11:20:25.481620 ip-10-0-136-0 kubenswrapper[2579]: I0703 11:20:25.481109 2579 pod_container_deletor.go:80] "Container not found in pod's containers" containerID="0d02a9d8eda70059f2e902266559c4dabb484ea885446c4ab8eee3965bd965bd" Jul 03 11:20:25.483422 ip-10-0-136-0 kubenswrapper[2579]: I0703 11:20:25.483392 2579 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/9438e18bbad664a016701a3153a6b421e2d977be7eb0117f80bcf45ef1chc8g" Jul 03 11:20:25.483422 ip-10-0-136-0 kubenswrapper[2579]: I0703 11:20:25.483397 2579 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/9438e18bbad664a016701a3153a6b421e2d977be7eb0117f80bcf45ef1chc8g" event={"ID":"5a64e43f-33ee-4779-bc1e-366dc5ef0cc0","Type":"ContainerDied","Data":"17311a1bb4c798a9c2e240e98be7cd378fe89252baa37ce7e102fd3243e90bcd"} Jul 03 11:20:25.483644 ip-10-0-136-0 kubenswrapper[2579]: I0703 11:20:25.483456 2579 pod_container_deletor.go:80] "Container not found in pod's containers" containerID="17311a1bb4c798a9c2e240e98be7cd378fe89252baa37ce7e102fd3243e90bcd" Jul 03 11:20:25.634836 ip-10-0-136-0 kubenswrapper[2579]: I0703 11:20:25.634797 2579 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/5fc36eb1065777914bfe0ff85f9a202e2a3bafefb563d9e2994474ed73mkshk" Jul 03 11:20:25.658900 ip-10-0-136-0 kubenswrapper[2579]: I0703 11:20:25.658858 2579 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-console/console-67c8fbdc89-swl6g"] Jul 03 11:20:25.659757 ip-10-0-136-0 kubenswrapper[2579]: I0703 11:20:25.659730 2579 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="e6f9e082-5d6a-4289-9883-301dbffe6151" containerName="pull" Jul 03 11:20:25.659901 ip-10-0-136-0 kubenswrapper[2579]: I0703 11:20:25.659770 2579 state_mem.go:107] "Deleted CPUSet assignment" podUID="e6f9e082-5d6a-4289-9883-301dbffe6151" containerName="pull" Jul 03 11:20:25.659901 ip-10-0-136-0 kubenswrapper[2579]: I0703 11:20:25.659788 2579 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="5a64e43f-33ee-4779-bc1e-366dc5ef0cc0" containerName="util" Jul 03 11:20:25.659901 ip-10-0-136-0 kubenswrapper[2579]: I0703 11:20:25.659799 2579 state_mem.go:107] "Deleted CPUSet assignment" podUID="5a64e43f-33ee-4779-bc1e-366dc5ef0cc0" containerName="util" Jul 03 11:20:25.659901 ip-10-0-136-0 kubenswrapper[2579]: I0703 11:20:25.659818 2579 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="5a64e43f-33ee-4779-bc1e-366dc5ef0cc0" containerName="pull" Jul 03 11:20:25.659901 ip-10-0-136-0 kubenswrapper[2579]: I0703 11:20:25.659829 2579 state_mem.go:107] "Deleted CPUSet assignment" podUID="5a64e43f-33ee-4779-bc1e-366dc5ef0cc0" containerName="pull" Jul 03 11:20:25.659901 ip-10-0-136-0 kubenswrapper[2579]: I0703 11:20:25.659839 2579 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="5a64e43f-33ee-4779-bc1e-366dc5ef0cc0" containerName="extract" Jul 03 11:20:25.659901 ip-10-0-136-0 kubenswrapper[2579]: I0703 11:20:25.659849 2579 state_mem.go:107] "Deleted CPUSet assignment" podUID="5a64e43f-33ee-4779-bc1e-366dc5ef0cc0" containerName="extract" Jul 03 11:20:25.659901 ip-10-0-136-0 kubenswrapper[2579]: I0703 11:20:25.659890 2579 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="1742e1ed-aaec-4579-b360-fca0512b5b72" containerName="pull" Jul 03 11:20:25.659901 ip-10-0-136-0 kubenswrapper[2579]: I0703 11:20:25.659899 2579 state_mem.go:107] "Deleted CPUSet assignment" podUID="1742e1ed-aaec-4579-b360-fca0512b5b72" containerName="pull" Jul 03 11:20:25.660314 ip-10-0-136-0 kubenswrapper[2579]: I0703 11:20:25.659924 2579 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="1742e1ed-aaec-4579-b360-fca0512b5b72" containerName="util" Jul 03 11:20:25.660314 ip-10-0-136-0 kubenswrapper[2579]: I0703 11:20:25.659965 2579 state_mem.go:107] "Deleted CPUSet assignment" podUID="1742e1ed-aaec-4579-b360-fca0512b5b72" containerName="util" Jul 03 11:20:25.660314 ip-10-0-136-0 kubenswrapper[2579]: I0703 11:20:25.659977 2579 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="e6f9e082-5d6a-4289-9883-301dbffe6151" containerName="util" Jul 03 11:20:25.660314 ip-10-0-136-0 kubenswrapper[2579]: I0703 11:20:25.659986 2579 state_mem.go:107] "Deleted CPUSet assignment" podUID="e6f9e082-5d6a-4289-9883-301dbffe6151" containerName="util" Jul 03 11:20:25.660314 ip-10-0-136-0 kubenswrapper[2579]: I0703 11:20:25.659996 2579 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="1742e1ed-aaec-4579-b360-fca0512b5b72" containerName="extract" Jul 03 11:20:25.660314 ip-10-0-136-0 kubenswrapper[2579]: I0703 11:20:25.660007 2579 state_mem.go:107] "Deleted CPUSet assignment" podUID="1742e1ed-aaec-4579-b360-fca0512b5b72" containerName="extract" Jul 03 11:20:25.660314 ip-10-0-136-0 kubenswrapper[2579]: I0703 11:20:25.660024 2579 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="e6f9e082-5d6a-4289-9883-301dbffe6151" containerName="extract" Jul 03 11:20:25.660314 ip-10-0-136-0 kubenswrapper[2579]: I0703 11:20:25.660029 2579 state_mem.go:107] "Deleted CPUSet assignment" podUID="e6f9e082-5d6a-4289-9883-301dbffe6151" containerName="extract" Jul 03 11:20:25.660314 ip-10-0-136-0 kubenswrapper[2579]: I0703 11:20:25.660164 2579 memory_manager.go:368] "RemoveStaleState removing state" podUID="e6f9e082-5d6a-4289-9883-301dbffe6151" containerName="extract" Jul 03 11:20:25.660314 ip-10-0-136-0 kubenswrapper[2579]: I0703 11:20:25.660174 2579 memory_manager.go:368] "RemoveStaleState removing state" podUID="1742e1ed-aaec-4579-b360-fca0512b5b72" containerName="extract" Jul 03 11:20:25.660314 ip-10-0-136-0 kubenswrapper[2579]: I0703 11:20:25.660204 2579 memory_manager.go:368] "RemoveStaleState removing state" podUID="5a64e43f-33ee-4779-bc1e-366dc5ef0cc0" containerName="extract" Jul 03 11:20:25.663771 ip-10-0-136-0 kubenswrapper[2579]: I0703 11:20:25.663743 2579 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-console/console-67c8fbdc89-swl6g" Jul 03 11:20:25.669442 ip-10-0-136-0 kubenswrapper[2579]: I0703 11:20:25.669408 2579 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/0acee64185f523d1d1272e9af2e4d9333e0dcde792ba30e1fa9605b7599bmjs" Jul 03 11:20:25.682336 ip-10-0-136-0 kubenswrapper[2579]: I0703 11:20:25.682299 2579 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-console/console-67c8fbdc89-swl6g"] Jul 03 11:20:25.766229 ip-10-0-136-0 kubenswrapper[2579]: I0703 11:20:25.766089 2579 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"util\" (UniqueName: \"kubernetes.io/empty-dir/805645ad-622e-4f54-8870-f15cd93c64f0-util\") pod \"805645ad-622e-4f54-8870-f15cd93c64f0\" (UID: \"805645ad-622e-4f54-8870-f15cd93c64f0\") " Jul 03 11:20:25.766229 ip-10-0-136-0 kubenswrapper[2579]: I0703 11:20:25.766176 2579 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"util\" (UniqueName: \"kubernetes.io/empty-dir/e6f9e082-5d6a-4289-9883-301dbffe6151-util\") pod \"e6f9e082-5d6a-4289-9883-301dbffe6151\" (UID: \"e6f9e082-5d6a-4289-9883-301dbffe6151\") " Jul 03 11:20:25.766474 ip-10-0-136-0 kubenswrapper[2579]: I0703 11:20:25.766235 2579 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-w6tlr\" (UniqueName: \"kubernetes.io/projected/805645ad-622e-4f54-8870-f15cd93c64f0-kube-api-access-w6tlr\") pod \"805645ad-622e-4f54-8870-f15cd93c64f0\" (UID: \"805645ad-622e-4f54-8870-f15cd93c64f0\") " Jul 03 11:20:25.766474 ip-10-0-136-0 kubenswrapper[2579]: I0703 11:20:25.766253 2579 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-588gr\" (UniqueName: \"kubernetes.io/projected/e6f9e082-5d6a-4289-9883-301dbffe6151-kube-api-access-588gr\") pod \"e6f9e082-5d6a-4289-9883-301dbffe6151\" (UID: \"e6f9e082-5d6a-4289-9883-301dbffe6151\") " Jul 03 11:20:25.766474 ip-10-0-136-0 kubenswrapper[2579]: I0703 11:20:25.766285 2579 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"bundle\" (UniqueName: \"kubernetes.io/empty-dir/e6f9e082-5d6a-4289-9883-301dbffe6151-bundle\") pod \"e6f9e082-5d6a-4289-9883-301dbffe6151\" (UID: \"e6f9e082-5d6a-4289-9883-301dbffe6151\") " Jul 03 11:20:25.766474 ip-10-0-136-0 kubenswrapper[2579]: I0703 11:20:25.766362 2579 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"bundle\" (UniqueName: \"kubernetes.io/empty-dir/805645ad-622e-4f54-8870-f15cd93c64f0-bundle\") pod \"805645ad-622e-4f54-8870-f15cd93c64f0\" (UID: \"805645ad-622e-4f54-8870-f15cd93c64f0\") " Jul 03 11:20:25.766700 ip-10-0-136-0 kubenswrapper[2579]: I0703 11:20:25.766547 2579 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"service-ca\" (UniqueName: \"kubernetes.io/configmap/db7db60d-f370-4f46-b2c7-bb2af9788553-service-ca\") pod \"console-67c8fbdc89-swl6g\" (UID: \"db7db60d-f370-4f46-b2c7-bb2af9788553\") " pod="openshift-console/console-67c8fbdc89-swl6g" Jul 03 11:20:25.766700 ip-10-0-136-0 kubenswrapper[2579]: I0703 11:20:25.766593 2579 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/db7db60d-f370-4f46-b2c7-bb2af9788553-trusted-ca-bundle\") pod \"console-67c8fbdc89-swl6g\" (UID: \"db7db60d-f370-4f46-b2c7-bb2af9788553\") " pod="openshift-console/console-67c8fbdc89-swl6g" Jul 03 11:20:25.766700 ip-10-0-136-0 kubenswrapper[2579]: I0703 11:20:25.766656 2579 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"oauth-serving-cert\" (UniqueName: \"kubernetes.io/configmap/db7db60d-f370-4f46-b2c7-bb2af9788553-oauth-serving-cert\") pod \"console-67c8fbdc89-swl6g\" (UID: \"db7db60d-f370-4f46-b2c7-bb2af9788553\") " pod="openshift-console/console-67c8fbdc89-swl6g" Jul 03 11:20:25.766849 ip-10-0-136-0 kubenswrapper[2579]: I0703 11:20:25.766748 2579 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-clhcp\" (UniqueName: \"kubernetes.io/projected/db7db60d-f370-4f46-b2c7-bb2af9788553-kube-api-access-clhcp\") pod \"console-67c8fbdc89-swl6g\" (UID: \"db7db60d-f370-4f46-b2c7-bb2af9788553\") " pod="openshift-console/console-67c8fbdc89-swl6g" Jul 03 11:20:25.766849 ip-10-0-136-0 kubenswrapper[2579]: I0703 11:20:25.766826 2579 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"console-oauth-config\" (UniqueName: \"kubernetes.io/secret/db7db60d-f370-4f46-b2c7-bb2af9788553-console-oauth-config\") pod \"console-67c8fbdc89-swl6g\" (UID: \"db7db60d-f370-4f46-b2c7-bb2af9788553\") " pod="openshift-console/console-67c8fbdc89-swl6g" Jul 03 11:20:25.766945 ip-10-0-136-0 kubenswrapper[2579]: I0703 11:20:25.766894 2579 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"console-serving-cert\" (UniqueName: \"kubernetes.io/secret/db7db60d-f370-4f46-b2c7-bb2af9788553-console-serving-cert\") pod \"console-67c8fbdc89-swl6g\" (UID: \"db7db60d-f370-4f46-b2c7-bb2af9788553\") " pod="openshift-console/console-67c8fbdc89-swl6g" Jul 03 11:20:25.766998 ip-10-0-136-0 kubenswrapper[2579]: I0703 11:20:25.766952 2579 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"console-config\" (UniqueName: \"kubernetes.io/configmap/db7db60d-f370-4f46-b2c7-bb2af9788553-console-config\") pod \"console-67c8fbdc89-swl6g\" (UID: \"db7db60d-f370-4f46-b2c7-bb2af9788553\") " pod="openshift-console/console-67c8fbdc89-swl6g" Jul 03 11:20:25.767048 ip-10-0-136-0 kubenswrapper[2579]: I0703 11:20:25.767027 2579 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/805645ad-622e-4f54-8870-f15cd93c64f0-bundle" (OuterVolumeSpecName: "bundle") pod "805645ad-622e-4f54-8870-f15cd93c64f0" (UID: "805645ad-622e-4f54-8870-f15cd93c64f0"). InnerVolumeSpecName "bundle". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Jul 03 11:20:25.767330 ip-10-0-136-0 kubenswrapper[2579]: I0703 11:20:25.767280 2579 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/e6f9e082-5d6a-4289-9883-301dbffe6151-bundle" (OuterVolumeSpecName: "bundle") pod "e6f9e082-5d6a-4289-9883-301dbffe6151" (UID: "e6f9e082-5d6a-4289-9883-301dbffe6151"). InnerVolumeSpecName "bundle". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Jul 03 11:20:25.768829 ip-10-0-136-0 kubenswrapper[2579]: I0703 11:20:25.768786 2579 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/e6f9e082-5d6a-4289-9883-301dbffe6151-kube-api-access-588gr" (OuterVolumeSpecName: "kube-api-access-588gr") pod "e6f9e082-5d6a-4289-9883-301dbffe6151" (UID: "e6f9e082-5d6a-4289-9883-301dbffe6151"). InnerVolumeSpecName "kube-api-access-588gr". PluginName "kubernetes.io/projected", VolumeGIDValue "" Jul 03 11:20:25.769334 ip-10-0-136-0 kubenswrapper[2579]: I0703 11:20:25.769301 2579 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/805645ad-622e-4f54-8870-f15cd93c64f0-kube-api-access-w6tlr" (OuterVolumeSpecName: "kube-api-access-w6tlr") pod "805645ad-622e-4f54-8870-f15cd93c64f0" (UID: "805645ad-622e-4f54-8870-f15cd93c64f0"). InnerVolumeSpecName "kube-api-access-w6tlr". PluginName "kubernetes.io/projected", VolumeGIDValue "" Jul 03 11:20:25.773083 ip-10-0-136-0 kubenswrapper[2579]: I0703 11:20:25.773017 2579 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/e6f9e082-5d6a-4289-9883-301dbffe6151-util" (OuterVolumeSpecName: "util") pod "e6f9e082-5d6a-4289-9883-301dbffe6151" (UID: "e6f9e082-5d6a-4289-9883-301dbffe6151"). InnerVolumeSpecName "util". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Jul 03 11:20:25.773894 ip-10-0-136-0 kubenswrapper[2579]: I0703 11:20:25.773861 2579 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/805645ad-622e-4f54-8870-f15cd93c64f0-util" (OuterVolumeSpecName: "util") pod "805645ad-622e-4f54-8870-f15cd93c64f0" (UID: "805645ad-622e-4f54-8870-f15cd93c64f0"). InnerVolumeSpecName "util". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Jul 03 11:20:25.868018 ip-10-0-136-0 kubenswrapper[2579]: I0703 11:20:25.867977 2579 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-clhcp\" (UniqueName: \"kubernetes.io/projected/db7db60d-f370-4f46-b2c7-bb2af9788553-kube-api-access-clhcp\") pod \"console-67c8fbdc89-swl6g\" (UID: \"db7db60d-f370-4f46-b2c7-bb2af9788553\") " pod="openshift-console/console-67c8fbdc89-swl6g" Jul 03 11:20:25.868018 ip-10-0-136-0 kubenswrapper[2579]: I0703 11:20:25.868037 2579 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"console-oauth-config\" (UniqueName: \"kubernetes.io/secret/db7db60d-f370-4f46-b2c7-bb2af9788553-console-oauth-config\") pod \"console-67c8fbdc89-swl6g\" (UID: \"db7db60d-f370-4f46-b2c7-bb2af9788553\") " pod="openshift-console/console-67c8fbdc89-swl6g" Jul 03 11:20:25.868362 ip-10-0-136-0 kubenswrapper[2579]: I0703 11:20:25.868070 2579 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"console-serving-cert\" (UniqueName: \"kubernetes.io/secret/db7db60d-f370-4f46-b2c7-bb2af9788553-console-serving-cert\") pod \"console-67c8fbdc89-swl6g\" (UID: \"db7db60d-f370-4f46-b2c7-bb2af9788553\") " pod="openshift-console/console-67c8fbdc89-swl6g" Jul 03 11:20:25.868362 ip-10-0-136-0 kubenswrapper[2579]: I0703 11:20:25.868100 2579 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"console-config\" (UniqueName: \"kubernetes.io/configmap/db7db60d-f370-4f46-b2c7-bb2af9788553-console-config\") pod \"console-67c8fbdc89-swl6g\" (UID: \"db7db60d-f370-4f46-b2c7-bb2af9788553\") " pod="openshift-console/console-67c8fbdc89-swl6g" Jul 03 11:20:25.868362 ip-10-0-136-0 kubenswrapper[2579]: I0703 11:20:25.868122 2579 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"service-ca\" (UniqueName: \"kubernetes.io/configmap/db7db60d-f370-4f46-b2c7-bb2af9788553-service-ca\") pod \"console-67c8fbdc89-swl6g\" (UID: \"db7db60d-f370-4f46-b2c7-bb2af9788553\") " pod="openshift-console/console-67c8fbdc89-swl6g" Jul 03 11:20:25.868362 ip-10-0-136-0 kubenswrapper[2579]: I0703 11:20:25.868146 2579 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/db7db60d-f370-4f46-b2c7-bb2af9788553-trusted-ca-bundle\") pod \"console-67c8fbdc89-swl6g\" (UID: \"db7db60d-f370-4f46-b2c7-bb2af9788553\") " pod="openshift-console/console-67c8fbdc89-swl6g" Jul 03 11:20:25.868582 ip-10-0-136-0 kubenswrapper[2579]: I0703 11:20:25.868358 2579 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"oauth-serving-cert\" (UniqueName: \"kubernetes.io/configmap/db7db60d-f370-4f46-b2c7-bb2af9788553-oauth-serving-cert\") pod \"console-67c8fbdc89-swl6g\" (UID: \"db7db60d-f370-4f46-b2c7-bb2af9788553\") " pod="openshift-console/console-67c8fbdc89-swl6g" Jul 03 11:20:25.868582 ip-10-0-136-0 kubenswrapper[2579]: I0703 11:20:25.868524 2579 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-w6tlr\" (UniqueName: \"kubernetes.io/projected/805645ad-622e-4f54-8870-f15cd93c64f0-kube-api-access-w6tlr\") on node \"ip-10-0-136-0.ec2.internal\" DevicePath \"\"" Jul 03 11:20:25.868582 ip-10-0-136-0 kubenswrapper[2579]: I0703 11:20:25.868544 2579 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-588gr\" (UniqueName: \"kubernetes.io/projected/e6f9e082-5d6a-4289-9883-301dbffe6151-kube-api-access-588gr\") on node \"ip-10-0-136-0.ec2.internal\" DevicePath \"\"" Jul 03 11:20:25.868582 ip-10-0-136-0 kubenswrapper[2579]: I0703 11:20:25.868561 2579 reconciler_common.go:299] "Volume detached for volume \"bundle\" (UniqueName: \"kubernetes.io/empty-dir/e6f9e082-5d6a-4289-9883-301dbffe6151-bundle\") on node \"ip-10-0-136-0.ec2.internal\" DevicePath \"\"" Jul 03 11:20:25.868582 ip-10-0-136-0 kubenswrapper[2579]: I0703 11:20:25.868578 2579 reconciler_common.go:299] "Volume detached for volume \"bundle\" (UniqueName: \"kubernetes.io/empty-dir/805645ad-622e-4f54-8870-f15cd93c64f0-bundle\") on node \"ip-10-0-136-0.ec2.internal\" DevicePath \"\"" Jul 03 11:20:25.868823 ip-10-0-136-0 kubenswrapper[2579]: I0703 11:20:25.868591 2579 reconciler_common.go:299] "Volume detached for volume \"util\" (UniqueName: \"kubernetes.io/empty-dir/805645ad-622e-4f54-8870-f15cd93c64f0-util\") on node \"ip-10-0-136-0.ec2.internal\" DevicePath \"\"" Jul 03 11:20:25.868823 ip-10-0-136-0 kubenswrapper[2579]: I0703 11:20:25.868606 2579 reconciler_common.go:299] "Volume detached for volume \"util\" (UniqueName: \"kubernetes.io/empty-dir/e6f9e082-5d6a-4289-9883-301dbffe6151-util\") on node \"ip-10-0-136-0.ec2.internal\" DevicePath \"\"" Jul 03 11:20:25.869017 ip-10-0-136-0 kubenswrapper[2579]: I0703 11:20:25.868992 2579 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"service-ca\" (UniqueName: \"kubernetes.io/configmap/db7db60d-f370-4f46-b2c7-bb2af9788553-service-ca\") pod \"console-67c8fbdc89-swl6g\" (UID: \"db7db60d-f370-4f46-b2c7-bb2af9788553\") " pod="openshift-console/console-67c8fbdc89-swl6g" Jul 03 11:20:25.869104 ip-10-0-136-0 kubenswrapper[2579]: I0703 11:20:25.868995 2579 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"console-config\" (UniqueName: \"kubernetes.io/configmap/db7db60d-f370-4f46-b2c7-bb2af9788553-console-config\") pod \"console-67c8fbdc89-swl6g\" (UID: \"db7db60d-f370-4f46-b2c7-bb2af9788553\") " pod="openshift-console/console-67c8fbdc89-swl6g" Jul 03 11:20:25.869343 ip-10-0-136-0 kubenswrapper[2579]: I0703 11:20:25.869320 2579 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"oauth-serving-cert\" (UniqueName: \"kubernetes.io/configmap/db7db60d-f370-4f46-b2c7-bb2af9788553-oauth-serving-cert\") pod \"console-67c8fbdc89-swl6g\" (UID: \"db7db60d-f370-4f46-b2c7-bb2af9788553\") " pod="openshift-console/console-67c8fbdc89-swl6g" Jul 03 11:20:25.869460 ip-10-0-136-0 kubenswrapper[2579]: I0703 11:20:25.869438 2579 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/db7db60d-f370-4f46-b2c7-bb2af9788553-trusted-ca-bundle\") pod \"console-67c8fbdc89-swl6g\" (UID: \"db7db60d-f370-4f46-b2c7-bb2af9788553\") " pod="openshift-console/console-67c8fbdc89-swl6g" Jul 03 11:20:25.870932 ip-10-0-136-0 kubenswrapper[2579]: I0703 11:20:25.870908 2579 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"console-oauth-config\" (UniqueName: \"kubernetes.io/secret/db7db60d-f370-4f46-b2c7-bb2af9788553-console-oauth-config\") pod \"console-67c8fbdc89-swl6g\" (UID: \"db7db60d-f370-4f46-b2c7-bb2af9788553\") " pod="openshift-console/console-67c8fbdc89-swl6g" Jul 03 11:20:25.871046 ip-10-0-136-0 kubenswrapper[2579]: I0703 11:20:25.871024 2579 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"console-serving-cert\" (UniqueName: \"kubernetes.io/secret/db7db60d-f370-4f46-b2c7-bb2af9788553-console-serving-cert\") pod \"console-67c8fbdc89-swl6g\" (UID: \"db7db60d-f370-4f46-b2c7-bb2af9788553\") " pod="openshift-console/console-67c8fbdc89-swl6g" Jul 03 11:20:25.879879 ip-10-0-136-0 kubenswrapper[2579]: I0703 11:20:25.879835 2579 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-clhcp\" (UniqueName: \"kubernetes.io/projected/db7db60d-f370-4f46-b2c7-bb2af9788553-kube-api-access-clhcp\") pod \"console-67c8fbdc89-swl6g\" (UID: \"db7db60d-f370-4f46-b2c7-bb2af9788553\") " pod="openshift-console/console-67c8fbdc89-swl6g" Jul 03 11:20:25.974844 ip-10-0-136-0 kubenswrapper[2579]: I0703 11:20:25.974793 2579 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-console/console-67c8fbdc89-swl6g" Jul 03 11:20:26.141374 ip-10-0-136-0 kubenswrapper[2579]: I0703 11:20:26.141337 2579 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-console/console-67c8fbdc89-swl6g"] Jul 03 11:20:26.143207 ip-10-0-136-0 kubenswrapper[2579]: W0703 11:20:26.143162 2579 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-poddb7db60d_f370_4f46_b2c7_bb2af9788553.slice/crio-7eae39358198bb9e3804c76846b9587214500b680d1e162201b34d75c298244b WatchSource:0}: Error finding container 7eae39358198bb9e3804c76846b9587214500b680d1e162201b34d75c298244b: Status 404 returned error can't find the container with id 7eae39358198bb9e3804c76846b9587214500b680d1e162201b34d75c298244b Jul 03 11:20:26.492443 ip-10-0-136-0 kubenswrapper[2579]: I0703 11:20:26.492351 2579 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/0acee64185f523d1d1272e9af2e4d9333e0dcde792ba30e1fa9605b7599bmjs" Jul 03 11:20:26.492443 ip-10-0-136-0 kubenswrapper[2579]: I0703 11:20:26.492352 2579 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/0acee64185f523d1d1272e9af2e4d9333e0dcde792ba30e1fa9605b7599bmjs" event={"ID":"805645ad-622e-4f54-8870-f15cd93c64f0","Type":"ContainerDied","Data":"a1dcebb875d92230ae11659e5263368be0c313f8c1d04dfac1b4855eee1e6ba8"} Jul 03 11:20:26.492893 ip-10-0-136-0 kubenswrapper[2579]: I0703 11:20:26.492479 2579 pod_container_deletor.go:80] "Container not found in pod's containers" containerID="a1dcebb875d92230ae11659e5263368be0c313f8c1d04dfac1b4855eee1e6ba8" Jul 03 11:20:26.494878 ip-10-0-136-0 kubenswrapper[2579]: I0703 11:20:26.494822 2579 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/5fc36eb1065777914bfe0ff85f9a202e2a3bafefb563d9e2994474ed73mkshk" event={"ID":"e6f9e082-5d6a-4289-9883-301dbffe6151","Type":"ContainerDied","Data":"aa711c4e06c3eaaa31161cdf60c760e2b29ea98d4b1b1e433bdafc2cde1e9cbe"} Jul 03 11:20:26.495064 ip-10-0-136-0 kubenswrapper[2579]: I0703 11:20:26.494898 2579 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/5fc36eb1065777914bfe0ff85f9a202e2a3bafefb563d9e2994474ed73mkshk" Jul 03 11:20:26.495064 ip-10-0-136-0 kubenswrapper[2579]: I0703 11:20:26.494903 2579 pod_container_deletor.go:80] "Container not found in pod's containers" containerID="aa711c4e06c3eaaa31161cdf60c760e2b29ea98d4b1b1e433bdafc2cde1e9cbe" Jul 03 11:20:26.496993 ip-10-0-136-0 kubenswrapper[2579]: I0703 11:20:26.496954 2579 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-console/console-67c8fbdc89-swl6g" event={"ID":"db7db60d-f370-4f46-b2c7-bb2af9788553","Type":"ContainerStarted","Data":"379c1fa7bfba3d1012687d63a7ec6f2f7209fac5a24d2090160dc6d12e4765bf"} Jul 03 11:20:26.497162 ip-10-0-136-0 kubenswrapper[2579]: I0703 11:20:26.497019 2579 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-console/console-67c8fbdc89-swl6g" event={"ID":"db7db60d-f370-4f46-b2c7-bb2af9788553","Type":"ContainerStarted","Data":"7eae39358198bb9e3804c76846b9587214500b680d1e162201b34d75c298244b"} Jul 03 11:20:26.561221 ip-10-0-136-0 kubenswrapper[2579]: I0703 11:20:26.561149 2579 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-console/console-67c8fbdc89-swl6g" podStartSLOduration=1.5611305500000001 podStartE2EDuration="1.56113055s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-03 11:20:25 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-03 11:20:26.561006805 +0000 UTC m=+483.149311125" watchObservedRunningTime="2026-07-03 11:20:26.56113055 +0000 UTC m=+483.149434869" Jul 03 11:20:35.874148 ip-10-0-136-0 kubenswrapper[2579]: I0703 11:20:35.874090 2579 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["kuadrant-system/dns-operator-controller-manager-864dd78c4c-wz8cc"] Jul 03 11:20:35.874724 ip-10-0-136-0 kubenswrapper[2579]: I0703 11:20:35.874695 2579 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="805645ad-622e-4f54-8870-f15cd93c64f0" containerName="extract" Jul 03 11:20:35.874724 ip-10-0-136-0 kubenswrapper[2579]: I0703 11:20:35.874722 2579 state_mem.go:107] "Deleted CPUSet assignment" podUID="805645ad-622e-4f54-8870-f15cd93c64f0" containerName="extract" Jul 03 11:20:35.874838 ip-10-0-136-0 kubenswrapper[2579]: I0703 11:20:35.874748 2579 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="805645ad-622e-4f54-8870-f15cd93c64f0" containerName="pull" Jul 03 11:20:35.874838 ip-10-0-136-0 kubenswrapper[2579]: I0703 11:20:35.874755 2579 state_mem.go:107] "Deleted CPUSet assignment" podUID="805645ad-622e-4f54-8870-f15cd93c64f0" containerName="pull" Jul 03 11:20:35.874838 ip-10-0-136-0 kubenswrapper[2579]: I0703 11:20:35.874766 2579 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="805645ad-622e-4f54-8870-f15cd93c64f0" containerName="util" Jul 03 11:20:35.874838 ip-10-0-136-0 kubenswrapper[2579]: I0703 11:20:35.874773 2579 state_mem.go:107] "Deleted CPUSet assignment" podUID="805645ad-622e-4f54-8870-f15cd93c64f0" containerName="util" Jul 03 11:20:35.874961 ip-10-0-136-0 kubenswrapper[2579]: I0703 11:20:35.874841 2579 memory_manager.go:368] "RemoveStaleState removing state" podUID="805645ad-622e-4f54-8870-f15cd93c64f0" containerName="extract" Jul 03 11:20:35.882045 ip-10-0-136-0 kubenswrapper[2579]: I0703 11:20:35.882014 2579 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/dns-operator-controller-manager-864dd78c4c-wz8cc" Jul 03 11:20:35.885884 ip-10-0-136-0 kubenswrapper[2579]: I0703 11:20:35.885854 2579 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"kuadrant-system\"/\"dns-operator-controller-env\"" Jul 03 11:20:35.886034 ip-10-0-136-0 kubenswrapper[2579]: I0703 11:20:35.885854 2579 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"kuadrant-system\"/\"dns-operator-controller-manager-dockercfg-lqprr\"" Jul 03 11:20:35.891233 ip-10-0-136-0 kubenswrapper[2579]: I0703 11:20:35.891203 2579 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/dns-operator-controller-manager-864dd78c4c-wz8cc"] Jul 03 11:20:35.966418 ip-10-0-136-0 kubenswrapper[2579]: I0703 11:20:35.966378 2579 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-w8tts\" (UniqueName: \"kubernetes.io/projected/763fbd50-f445-4a0e-8555-3ffa10d05e56-kube-api-access-w8tts\") pod \"dns-operator-controller-manager-864dd78c4c-wz8cc\" (UID: \"763fbd50-f445-4a0e-8555-3ffa10d05e56\") " pod="kuadrant-system/dns-operator-controller-manager-864dd78c4c-wz8cc" Jul 03 11:20:35.974979 ip-10-0-136-0 kubenswrapper[2579]: I0703 11:20:35.974942 2579 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="openshift-console/console-67c8fbdc89-swl6g" Jul 03 11:20:35.975156 ip-10-0-136-0 kubenswrapper[2579]: I0703 11:20:35.975004 2579 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-console/console-67c8fbdc89-swl6g" Jul 03 11:20:35.980366 ip-10-0-136-0 kubenswrapper[2579]: I0703 11:20:35.980336 2579 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="openshift-console/console-67c8fbdc89-swl6g" Jul 03 11:20:36.067419 ip-10-0-136-0 kubenswrapper[2579]: I0703 11:20:36.067377 2579 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-w8tts\" (UniqueName: \"kubernetes.io/projected/763fbd50-f445-4a0e-8555-3ffa10d05e56-kube-api-access-w8tts\") pod \"dns-operator-controller-manager-864dd78c4c-wz8cc\" (UID: \"763fbd50-f445-4a0e-8555-3ffa10d05e56\") " pod="kuadrant-system/dns-operator-controller-manager-864dd78c4c-wz8cc" Jul 03 11:20:36.087462 ip-10-0-136-0 kubenswrapper[2579]: I0703 11:20:36.087426 2579 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-w8tts\" (UniqueName: \"kubernetes.io/projected/763fbd50-f445-4a0e-8555-3ffa10d05e56-kube-api-access-w8tts\") pod \"dns-operator-controller-manager-864dd78c4c-wz8cc\" (UID: \"763fbd50-f445-4a0e-8555-3ffa10d05e56\") " pod="kuadrant-system/dns-operator-controller-manager-864dd78c4c-wz8cc" Jul 03 11:20:36.195655 ip-10-0-136-0 kubenswrapper[2579]: I0703 11:20:36.195556 2579 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/dns-operator-controller-manager-864dd78c4c-wz8cc" Jul 03 11:20:36.359947 ip-10-0-136-0 kubenswrapper[2579]: I0703 11:20:36.359912 2579 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/dns-operator-controller-manager-864dd78c4c-wz8cc"] Jul 03 11:20:36.362889 ip-10-0-136-0 kubenswrapper[2579]: W0703 11:20:36.362836 2579 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod763fbd50_f445_4a0e_8555_3ffa10d05e56.slice/crio-7a8843d61f5e35d91c10c28d082ceec2f2ddfa37c74064b1870180bfcd83589a WatchSource:0}: Error finding container 7a8843d61f5e35d91c10c28d082ceec2f2ddfa37c74064b1870180bfcd83589a: Status 404 returned error can't find the container with id 7a8843d61f5e35d91c10c28d082ceec2f2ddfa37c74064b1870180bfcd83589a Jul 03 11:20:36.552712 ip-10-0-136-0 kubenswrapper[2579]: I0703 11:20:36.552664 2579 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/dns-operator-controller-manager-864dd78c4c-wz8cc" event={"ID":"763fbd50-f445-4a0e-8555-3ffa10d05e56","Type":"ContainerStarted","Data":"7a8843d61f5e35d91c10c28d082ceec2f2ddfa37c74064b1870180bfcd83589a"} Jul 03 11:20:36.556950 ip-10-0-136-0 kubenswrapper[2579]: I0703 11:20:36.556923 2579 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-console/console-67c8fbdc89-swl6g" Jul 03 11:20:36.646977 ip-10-0-136-0 kubenswrapper[2579]: I0703 11:20:36.646933 2579 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-console/console-799d957b5b-vrh8b"] Jul 03 11:20:38.568326 ip-10-0-136-0 kubenswrapper[2579]: I0703 11:20:38.568225 2579 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/dns-operator-controller-manager-864dd78c4c-wz8cc" event={"ID":"763fbd50-f445-4a0e-8555-3ffa10d05e56","Type":"ContainerStarted","Data":"a1428a4781ba129212a5f535650f0dfa8de3207754e70d5bd9ea655cd2f45edb"} Jul 03 11:20:38.568326 ip-10-0-136-0 kubenswrapper[2579]: I0703 11:20:38.568310 2579 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="kuadrant-system/dns-operator-controller-manager-864dd78c4c-wz8cc" Jul 03 11:20:38.595958 ip-10-0-136-0 kubenswrapper[2579]: I0703 11:20:38.595887 2579 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="kuadrant-system/dns-operator-controller-manager-864dd78c4c-wz8cc" podStartSLOduration=1.6425924809999999 podStartE2EDuration="3.595862737s" totalImagesPullingTime="1.953270256s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-03 11:20:35 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-03 11:20:38.592958688 +0000 UTC m=+495.181263008" watchObservedRunningTime="2026-07-03 11:20:38.595862737 +0000 UTC m=+495.184167089" Jul 03 11:20:39.326130 ip-10-0-136-0 kubenswrapper[2579]: I0703 11:20:39.326092 2579 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["kuadrant-system/limitador-operator-controller-manager-859d7b78bd-zc6jx"] Jul 03 11:20:39.333710 ip-10-0-136-0 kubenswrapper[2579]: I0703 11:20:39.331044 2579 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/limitador-operator-controller-manager-859d7b78bd-zc6jx" Jul 03 11:20:39.335238 ip-10-0-136-0 kubenswrapper[2579]: I0703 11:20:39.335213 2579 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"kuadrant-system\"/\"limitador-operator-controller-manager-dockercfg-mg2dq\"" Jul 03 11:20:39.352271 ip-10-0-136-0 kubenswrapper[2579]: I0703 11:20:39.352225 2579 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-operator-controller-manager-859d7b78bd-zc6jx"] Jul 03 11:20:39.502087 ip-10-0-136-0 kubenswrapper[2579]: I0703 11:20:39.502040 2579 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-qs8ft\" (UniqueName: \"kubernetes.io/projected/b73c6525-f1be-4a04-a0b8-76bc1635b185-kube-api-access-qs8ft\") pod \"limitador-operator-controller-manager-859d7b78bd-zc6jx\" (UID: \"b73c6525-f1be-4a04-a0b8-76bc1635b185\") " pod="kuadrant-system/limitador-operator-controller-manager-859d7b78bd-zc6jx" Jul 03 11:20:39.603288 ip-10-0-136-0 kubenswrapper[2579]: I0703 11:20:39.603145 2579 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-qs8ft\" (UniqueName: \"kubernetes.io/projected/b73c6525-f1be-4a04-a0b8-76bc1635b185-kube-api-access-qs8ft\") pod \"limitador-operator-controller-manager-859d7b78bd-zc6jx\" (UID: \"b73c6525-f1be-4a04-a0b8-76bc1635b185\") " pod="kuadrant-system/limitador-operator-controller-manager-859d7b78bd-zc6jx" Jul 03 11:20:39.616241 ip-10-0-136-0 kubenswrapper[2579]: I0703 11:20:39.616175 2579 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-qs8ft\" (UniqueName: \"kubernetes.io/projected/b73c6525-f1be-4a04-a0b8-76bc1635b185-kube-api-access-qs8ft\") pod \"limitador-operator-controller-manager-859d7b78bd-zc6jx\" (UID: \"b73c6525-f1be-4a04-a0b8-76bc1635b185\") " pod="kuadrant-system/limitador-operator-controller-manager-859d7b78bd-zc6jx" Jul 03 11:20:39.644167 ip-10-0-136-0 kubenswrapper[2579]: I0703 11:20:39.644126 2579 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/limitador-operator-controller-manager-859d7b78bd-zc6jx" Jul 03 11:20:39.808498 ip-10-0-136-0 kubenswrapper[2579]: I0703 11:20:39.808466 2579 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-operator-controller-manager-859d7b78bd-zc6jx"] Jul 03 11:20:39.810666 ip-10-0-136-0 kubenswrapper[2579]: W0703 11:20:39.810622 2579 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podb73c6525_f1be_4a04_a0b8_76bc1635b185.slice/crio-940a806fb998e23960dd78c170a7dd8998325be49bb77fdd1193bdbab0b54573 WatchSource:0}: Error finding container 940a806fb998e23960dd78c170a7dd8998325be49bb77fdd1193bdbab0b54573: Status 404 returned error can't find the container with id 940a806fb998e23960dd78c170a7dd8998325be49bb77fdd1193bdbab0b54573 Jul 03 11:20:40.582887 ip-10-0-136-0 kubenswrapper[2579]: I0703 11:20:40.582843 2579 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/limitador-operator-controller-manager-859d7b78bd-zc6jx" event={"ID":"b73c6525-f1be-4a04-a0b8-76bc1635b185","Type":"ContainerStarted","Data":"940a806fb998e23960dd78c170a7dd8998325be49bb77fdd1193bdbab0b54573"} Jul 03 11:20:42.596337 ip-10-0-136-0 kubenswrapper[2579]: I0703 11:20:42.596299 2579 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/limitador-operator-controller-manager-859d7b78bd-zc6jx" event={"ID":"b73c6525-f1be-4a04-a0b8-76bc1635b185","Type":"ContainerStarted","Data":"cfb5766892ad38b5fcc9c28b9c1db409eda3e757626b9399ef4c0ef6bba6ed12"} Jul 03 11:20:42.596770 ip-10-0-136-0 kubenswrapper[2579]: I0703 11:20:42.596427 2579 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="kuadrant-system/limitador-operator-controller-manager-859d7b78bd-zc6jx" Jul 03 11:20:42.618056 ip-10-0-136-0 kubenswrapper[2579]: I0703 11:20:42.617975 2579 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="kuadrant-system/limitador-operator-controller-manager-859d7b78bd-zc6jx" podStartSLOduration=1.6401050910000001 podStartE2EDuration="3.617952771s" totalImagesPullingTime="1.97784768s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-03 11:20:39 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-03 11:20:42.616005659 +0000 UTC m=+499.204309992" watchObservedRunningTime="2026-07-03 11:20:42.617952771 +0000 UTC m=+499.206257092" Jul 03 11:20:43.146440 ip-10-0-136-0 kubenswrapper[2579]: I0703 11:20:43.146401 2579 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["kuadrant-system/authorino-operator-75cc8d5cb7-nh9mf"] Jul 03 11:20:43.150065 ip-10-0-136-0 kubenswrapper[2579]: I0703 11:20:43.150036 2579 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/authorino-operator-75cc8d5cb7-nh9mf" Jul 03 11:20:43.153803 ip-10-0-136-0 kubenswrapper[2579]: I0703 11:20:43.153780 2579 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"kuadrant-system\"/\"authorino-operator-dockercfg-q9lzr\"" Jul 03 11:20:43.169149 ip-10-0-136-0 kubenswrapper[2579]: I0703 11:20:43.169113 2579 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/authorino-operator-75cc8d5cb7-nh9mf"] Jul 03 11:20:43.240429 ip-10-0-136-0 kubenswrapper[2579]: I0703 11:20:43.240392 2579 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-x6ncq\" (UniqueName: \"kubernetes.io/projected/0bcdb275-72ef-409c-804c-ca58380c460e-kube-api-access-x6ncq\") pod \"authorino-operator-75cc8d5cb7-nh9mf\" (UID: \"0bcdb275-72ef-409c-804c-ca58380c460e\") " pod="kuadrant-system/authorino-operator-75cc8d5cb7-nh9mf" Jul 03 11:20:43.341024 ip-10-0-136-0 kubenswrapper[2579]: I0703 11:20:43.340992 2579 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-x6ncq\" (UniqueName: \"kubernetes.io/projected/0bcdb275-72ef-409c-804c-ca58380c460e-kube-api-access-x6ncq\") pod \"authorino-operator-75cc8d5cb7-nh9mf\" (UID: \"0bcdb275-72ef-409c-804c-ca58380c460e\") " pod="kuadrant-system/authorino-operator-75cc8d5cb7-nh9mf" Jul 03 11:20:43.368677 ip-10-0-136-0 kubenswrapper[2579]: I0703 11:20:43.368633 2579 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-x6ncq\" (UniqueName: \"kubernetes.io/projected/0bcdb275-72ef-409c-804c-ca58380c460e-kube-api-access-x6ncq\") pod \"authorino-operator-75cc8d5cb7-nh9mf\" (UID: \"0bcdb275-72ef-409c-804c-ca58380c460e\") " pod="kuadrant-system/authorino-operator-75cc8d5cb7-nh9mf" Jul 03 11:20:43.461236 ip-10-0-136-0 kubenswrapper[2579]: I0703 11:20:43.461109 2579 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/authorino-operator-75cc8d5cb7-nh9mf" Jul 03 11:20:43.623481 ip-10-0-136-0 kubenswrapper[2579]: I0703 11:20:43.623451 2579 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/authorino-operator-75cc8d5cb7-nh9mf"] Jul 03 11:20:43.625991 ip-10-0-136-0 kubenswrapper[2579]: W0703 11:20:43.625945 2579 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod0bcdb275_72ef_409c_804c_ca58380c460e.slice/crio-7daf8e23bee7961fc727cb1174a00ab5772fd29f7b23ce4254c72fea6613eaa3 WatchSource:0}: Error finding container 7daf8e23bee7961fc727cb1174a00ab5772fd29f7b23ce4254c72fea6613eaa3: Status 404 returned error can't find the container with id 7daf8e23bee7961fc727cb1174a00ab5772fd29f7b23ce4254c72fea6613eaa3 Jul 03 11:20:44.609912 ip-10-0-136-0 kubenswrapper[2579]: I0703 11:20:44.609871 2579 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/authorino-operator-75cc8d5cb7-nh9mf" event={"ID":"0bcdb275-72ef-409c-804c-ca58380c460e","Type":"ContainerStarted","Data":"7daf8e23bee7961fc727cb1174a00ab5772fd29f7b23ce4254c72fea6613eaa3"} Jul 03 11:20:47.632763 ip-10-0-136-0 kubenswrapper[2579]: I0703 11:20:47.632721 2579 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/authorino-operator-75cc8d5cb7-nh9mf" event={"ID":"0bcdb275-72ef-409c-804c-ca58380c460e","Type":"ContainerStarted","Data":"7118803144da325258fa32d82b5e94ec2e9104074f177e66deb56f3eff81a573"} Jul 03 11:20:47.633184 ip-10-0-136-0 kubenswrapper[2579]: I0703 11:20:47.632787 2579 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="kuadrant-system/authorino-operator-75cc8d5cb7-nh9mf" Jul 03 11:20:47.652281 ip-10-0-136-0 kubenswrapper[2579]: I0703 11:20:47.652210 2579 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="kuadrant-system/authorino-operator-75cc8d5cb7-nh9mf" podStartSLOduration=1.713856212 podStartE2EDuration="4.652174707s" totalImagesPullingTime="2.938318495s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-03 11:20:43 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-03 11:20:47.649962912 +0000 UTC m=+504.238267246" watchObservedRunningTime="2026-07-03 11:20:47.652174707 +0000 UTC m=+504.240479027" Jul 03 11:20:48.174873 ip-10-0-136-0 kubenswrapper[2579]: I0703 11:20:48.174825 2579 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["kuadrant-system/kuadrant-operator-controller-manager-8684c9ff49-fk95g"] Jul 03 11:20:48.178526 ip-10-0-136-0 kubenswrapper[2579]: I0703 11:20:48.178493 2579 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/kuadrant-operator-controller-manager-8684c9ff49-fk95g" Jul 03 11:20:48.181823 ip-10-0-136-0 kubenswrapper[2579]: I0703 11:20:48.181800 2579 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"kuadrant-system\"/\"kuadrant-operator-controller-manager-dockercfg-d8jfw\"" Jul 03 11:20:48.199128 ip-10-0-136-0 kubenswrapper[2579]: I0703 11:20:48.199081 2579 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/kuadrant-operator-controller-manager-8684c9ff49-fk95g"] Jul 03 11:20:48.288160 ip-10-0-136-0 kubenswrapper[2579]: I0703 11:20:48.288104 2579 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"extensions-socket-volume\" (UniqueName: \"kubernetes.io/empty-dir/0fd7b987-9598-4f1f-961b-097b517e0fbe-extensions-socket-volume\") pod \"kuadrant-operator-controller-manager-8684c9ff49-fk95g\" (UID: \"0fd7b987-9598-4f1f-961b-097b517e0fbe\") " pod="kuadrant-system/kuadrant-operator-controller-manager-8684c9ff49-fk95g" Jul 03 11:20:48.288375 ip-10-0-136-0 kubenswrapper[2579]: I0703 11:20:48.288203 2579 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-nvxmw\" (UniqueName: \"kubernetes.io/projected/0fd7b987-9598-4f1f-961b-097b517e0fbe-kube-api-access-nvxmw\") pod \"kuadrant-operator-controller-manager-8684c9ff49-fk95g\" (UID: \"0fd7b987-9598-4f1f-961b-097b517e0fbe\") " pod="kuadrant-system/kuadrant-operator-controller-manager-8684c9ff49-fk95g" Jul 03 11:20:48.389019 ip-10-0-136-0 kubenswrapper[2579]: I0703 11:20:48.388977 2579 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"extensions-socket-volume\" (UniqueName: \"kubernetes.io/empty-dir/0fd7b987-9598-4f1f-961b-097b517e0fbe-extensions-socket-volume\") pod \"kuadrant-operator-controller-manager-8684c9ff49-fk95g\" (UID: \"0fd7b987-9598-4f1f-961b-097b517e0fbe\") " pod="kuadrant-system/kuadrant-operator-controller-manager-8684c9ff49-fk95g" Jul 03 11:20:48.389019 ip-10-0-136-0 kubenswrapper[2579]: I0703 11:20:48.389033 2579 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-nvxmw\" (UniqueName: \"kubernetes.io/projected/0fd7b987-9598-4f1f-961b-097b517e0fbe-kube-api-access-nvxmw\") pod \"kuadrant-operator-controller-manager-8684c9ff49-fk95g\" (UID: \"0fd7b987-9598-4f1f-961b-097b517e0fbe\") " pod="kuadrant-system/kuadrant-operator-controller-manager-8684c9ff49-fk95g" Jul 03 11:20:48.389499 ip-10-0-136-0 kubenswrapper[2579]: I0703 11:20:48.389474 2579 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"extensions-socket-volume\" (UniqueName: \"kubernetes.io/empty-dir/0fd7b987-9598-4f1f-961b-097b517e0fbe-extensions-socket-volume\") pod \"kuadrant-operator-controller-manager-8684c9ff49-fk95g\" (UID: \"0fd7b987-9598-4f1f-961b-097b517e0fbe\") " pod="kuadrant-system/kuadrant-operator-controller-manager-8684c9ff49-fk95g" Jul 03 11:20:48.399640 ip-10-0-136-0 kubenswrapper[2579]: I0703 11:20:48.399602 2579 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-nvxmw\" (UniqueName: \"kubernetes.io/projected/0fd7b987-9598-4f1f-961b-097b517e0fbe-kube-api-access-nvxmw\") pod \"kuadrant-operator-controller-manager-8684c9ff49-fk95g\" (UID: \"0fd7b987-9598-4f1f-961b-097b517e0fbe\") " pod="kuadrant-system/kuadrant-operator-controller-manager-8684c9ff49-fk95g" Jul 03 11:20:48.489370 ip-10-0-136-0 kubenswrapper[2579]: I0703 11:20:48.489264 2579 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/kuadrant-operator-controller-manager-8684c9ff49-fk95g" Jul 03 11:20:48.673329 ip-10-0-136-0 kubenswrapper[2579]: I0703 11:20:48.673265 2579 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/kuadrant-operator-controller-manager-8684c9ff49-fk95g"] Jul 03 11:20:48.675887 ip-10-0-136-0 kubenswrapper[2579]: W0703 11:20:48.675836 2579 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod0fd7b987_9598_4f1f_961b_097b517e0fbe.slice/crio-2851420f08d86f9be71ba43e064909722b46c7209e56aa7735bc43e604b033a2 WatchSource:0}: Error finding container 2851420f08d86f9be71ba43e064909722b46c7209e56aa7735bc43e604b033a2: Status 404 returned error can't find the container with id 2851420f08d86f9be71ba43e064909722b46c7209e56aa7735bc43e604b033a2 Jul 03 11:20:49.577128 ip-10-0-136-0 kubenswrapper[2579]: I0703 11:20:49.577096 2579 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="kuadrant-system/dns-operator-controller-manager-864dd78c4c-wz8cc" Jul 03 11:20:49.651334 ip-10-0-136-0 kubenswrapper[2579]: I0703 11:20:49.651291 2579 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/kuadrant-operator-controller-manager-8684c9ff49-fk95g" event={"ID":"0fd7b987-9598-4f1f-961b-097b517e0fbe","Type":"ContainerStarted","Data":"2851420f08d86f9be71ba43e064909722b46c7209e56aa7735bc43e604b033a2"} Jul 03 11:20:53.605445 ip-10-0-136-0 kubenswrapper[2579]: I0703 11:20:53.605405 2579 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="kuadrant-system/limitador-operator-controller-manager-859d7b78bd-zc6jx" Jul 03 11:20:54.685285 ip-10-0-136-0 kubenswrapper[2579]: I0703 11:20:54.685227 2579 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/kuadrant-operator-controller-manager-8684c9ff49-fk95g" event={"ID":"0fd7b987-9598-4f1f-961b-097b517e0fbe","Type":"ContainerStarted","Data":"8e15a681075af5e4034ebd28bdfad855f405bed1f9e692bdebc04ab349086cb7"} Jul 03 11:20:54.685871 ip-10-0-136-0 kubenswrapper[2579]: I0703 11:20:54.685320 2579 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="kuadrant-system/kuadrant-operator-controller-manager-8684c9ff49-fk95g" Jul 03 11:20:54.729441 ip-10-0-136-0 kubenswrapper[2579]: I0703 11:20:54.729313 2579 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="kuadrant-system/kuadrant-operator-controller-manager-8684c9ff49-fk95g" podStartSLOduration=0.940032601 podStartE2EDuration="6.729289759s" totalImagesPullingTime="5.789257158s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-03 11:20:48 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-03 11:20:54.725655543 +0000 UTC m=+511.313959866" watchObservedRunningTime="2026-07-03 11:20:54.729289759 +0000 UTC m=+511.317594079" Jul 03 11:20:58.641956 ip-10-0-136-0 kubenswrapper[2579]: I0703 11:20:58.641909 2579 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="kuadrant-system/authorino-operator-75cc8d5cb7-nh9mf" Jul 03 11:21:01.669109 ip-10-0-136-0 kubenswrapper[2579]: I0703 11:21:01.669037 2579 kuberuntime_container.go:872] "Killing container with a grace period" pod="openshift-console/console-799d957b5b-vrh8b" podUID="baaa910d-775c-4be2-8d26-04897721653a" containerName="console" containerID="cri-o://90ccd314b814cc45d99ae531fa903877474b29e1da27f94db87f2313933a9fbf" gracePeriod=15 Jul 03 11:21:01.941398 ip-10-0-136-0 kubenswrapper[2579]: I0703 11:21:01.941364 2579 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-console/console-799d957b5b-vrh8b" Jul 03 11:21:02.008314 ip-10-0-136-0 kubenswrapper[2579]: I0703 11:21:02.008270 2579 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"console-oauth-config\" (UniqueName: \"kubernetes.io/secret/baaa910d-775c-4be2-8d26-04897721653a-console-oauth-config\") pod \"baaa910d-775c-4be2-8d26-04897721653a\" (UID: \"baaa910d-775c-4be2-8d26-04897721653a\") " Jul 03 11:21:02.008314 ip-10-0-136-0 kubenswrapper[2579]: I0703 11:21:02.008320 2579 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-h9csr\" (UniqueName: \"kubernetes.io/projected/baaa910d-775c-4be2-8d26-04897721653a-kube-api-access-h9csr\") pod \"baaa910d-775c-4be2-8d26-04897721653a\" (UID: \"baaa910d-775c-4be2-8d26-04897721653a\") " Jul 03 11:21:02.008571 ip-10-0-136-0 kubenswrapper[2579]: I0703 11:21:02.008350 2579 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"console-serving-cert\" (UniqueName: \"kubernetes.io/secret/baaa910d-775c-4be2-8d26-04897721653a-console-serving-cert\") pod \"baaa910d-775c-4be2-8d26-04897721653a\" (UID: \"baaa910d-775c-4be2-8d26-04897721653a\") " Jul 03 11:21:02.008571 ip-10-0-136-0 kubenswrapper[2579]: I0703 11:21:02.008373 2579 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"console-config\" (UniqueName: \"kubernetes.io/configmap/baaa910d-775c-4be2-8d26-04897721653a-console-config\") pod \"baaa910d-775c-4be2-8d26-04897721653a\" (UID: \"baaa910d-775c-4be2-8d26-04897721653a\") " Jul 03 11:21:02.008571 ip-10-0-136-0 kubenswrapper[2579]: I0703 11:21:02.008409 2579 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"oauth-serving-cert\" (UniqueName: \"kubernetes.io/configmap/baaa910d-775c-4be2-8d26-04897721653a-oauth-serving-cert\") pod \"baaa910d-775c-4be2-8d26-04897721653a\" (UID: \"baaa910d-775c-4be2-8d26-04897721653a\") " Jul 03 11:21:02.008571 ip-10-0-136-0 kubenswrapper[2579]: I0703 11:21:02.008445 2579 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/baaa910d-775c-4be2-8d26-04897721653a-trusted-ca-bundle\") pod \"baaa910d-775c-4be2-8d26-04897721653a\" (UID: \"baaa910d-775c-4be2-8d26-04897721653a\") " Jul 03 11:21:02.008571 ip-10-0-136-0 kubenswrapper[2579]: I0703 11:21:02.008477 2579 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"service-ca\" (UniqueName: \"kubernetes.io/configmap/baaa910d-775c-4be2-8d26-04897721653a-service-ca\") pod \"baaa910d-775c-4be2-8d26-04897721653a\" (UID: \"baaa910d-775c-4be2-8d26-04897721653a\") " Jul 03 11:21:02.008984 ip-10-0-136-0 kubenswrapper[2579]: I0703 11:21:02.008889 2579 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/baaa910d-775c-4be2-8d26-04897721653a-console-config" (OuterVolumeSpecName: "console-config") pod "baaa910d-775c-4be2-8d26-04897721653a" (UID: "baaa910d-775c-4be2-8d26-04897721653a"). InnerVolumeSpecName "console-config". PluginName "kubernetes.io/configmap", VolumeGIDValue "" Jul 03 11:21:02.008984 ip-10-0-136-0 kubenswrapper[2579]: I0703 11:21:02.008897 2579 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/baaa910d-775c-4be2-8d26-04897721653a-oauth-serving-cert" (OuterVolumeSpecName: "oauth-serving-cert") pod "baaa910d-775c-4be2-8d26-04897721653a" (UID: "baaa910d-775c-4be2-8d26-04897721653a"). InnerVolumeSpecName "oauth-serving-cert". PluginName "kubernetes.io/configmap", VolumeGIDValue "" Jul 03 11:21:02.008984 ip-10-0-136-0 kubenswrapper[2579]: I0703 11:21:02.008930 2579 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/baaa910d-775c-4be2-8d26-04897721653a-service-ca" (OuterVolumeSpecName: "service-ca") pod "baaa910d-775c-4be2-8d26-04897721653a" (UID: "baaa910d-775c-4be2-8d26-04897721653a"). InnerVolumeSpecName "service-ca". PluginName "kubernetes.io/configmap", VolumeGIDValue "" Jul 03 11:21:02.009317 ip-10-0-136-0 kubenswrapper[2579]: I0703 11:21:02.009293 2579 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/baaa910d-775c-4be2-8d26-04897721653a-trusted-ca-bundle" (OuterVolumeSpecName: "trusted-ca-bundle") pod "baaa910d-775c-4be2-8d26-04897721653a" (UID: "baaa910d-775c-4be2-8d26-04897721653a"). InnerVolumeSpecName "trusted-ca-bundle". PluginName "kubernetes.io/configmap", VolumeGIDValue "" Jul 03 11:21:02.010917 ip-10-0-136-0 kubenswrapper[2579]: I0703 11:21:02.010894 2579 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/baaa910d-775c-4be2-8d26-04897721653a-kube-api-access-h9csr" (OuterVolumeSpecName: "kube-api-access-h9csr") pod "baaa910d-775c-4be2-8d26-04897721653a" (UID: "baaa910d-775c-4be2-8d26-04897721653a"). InnerVolumeSpecName "kube-api-access-h9csr". PluginName "kubernetes.io/projected", VolumeGIDValue "" Jul 03 11:21:02.011407 ip-10-0-136-0 kubenswrapper[2579]: I0703 11:21:02.011379 2579 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/baaa910d-775c-4be2-8d26-04897721653a-console-serving-cert" (OuterVolumeSpecName: "console-serving-cert") pod "baaa910d-775c-4be2-8d26-04897721653a" (UID: "baaa910d-775c-4be2-8d26-04897721653a"). InnerVolumeSpecName "console-serving-cert". PluginName "kubernetes.io/secret", VolumeGIDValue "" Jul 03 11:21:02.011407 ip-10-0-136-0 kubenswrapper[2579]: I0703 11:21:02.011395 2579 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/baaa910d-775c-4be2-8d26-04897721653a-console-oauth-config" (OuterVolumeSpecName: "console-oauth-config") pod "baaa910d-775c-4be2-8d26-04897721653a" (UID: "baaa910d-775c-4be2-8d26-04897721653a"). InnerVolumeSpecName "console-oauth-config". PluginName "kubernetes.io/secret", VolumeGIDValue "" Jul 03 11:21:02.109284 ip-10-0-136-0 kubenswrapper[2579]: I0703 11:21:02.109248 2579 reconciler_common.go:299] "Volume detached for volume \"trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/baaa910d-775c-4be2-8d26-04897721653a-trusted-ca-bundle\") on node \"ip-10-0-136-0.ec2.internal\" DevicePath \"\"" Jul 03 11:21:02.109284 ip-10-0-136-0 kubenswrapper[2579]: I0703 11:21:02.109277 2579 reconciler_common.go:299] "Volume detached for volume \"service-ca\" (UniqueName: \"kubernetes.io/configmap/baaa910d-775c-4be2-8d26-04897721653a-service-ca\") on node \"ip-10-0-136-0.ec2.internal\" DevicePath \"\"" Jul 03 11:21:02.109284 ip-10-0-136-0 kubenswrapper[2579]: I0703 11:21:02.109288 2579 reconciler_common.go:299] "Volume detached for volume \"console-oauth-config\" (UniqueName: \"kubernetes.io/secret/baaa910d-775c-4be2-8d26-04897721653a-console-oauth-config\") on node \"ip-10-0-136-0.ec2.internal\" DevicePath \"\"" Jul 03 11:21:02.109528 ip-10-0-136-0 kubenswrapper[2579]: I0703 11:21:02.109298 2579 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-h9csr\" (UniqueName: \"kubernetes.io/projected/baaa910d-775c-4be2-8d26-04897721653a-kube-api-access-h9csr\") on node \"ip-10-0-136-0.ec2.internal\" DevicePath \"\"" Jul 03 11:21:02.109528 ip-10-0-136-0 kubenswrapper[2579]: I0703 11:21:02.109307 2579 reconciler_common.go:299] "Volume detached for volume \"console-serving-cert\" (UniqueName: \"kubernetes.io/secret/baaa910d-775c-4be2-8d26-04897721653a-console-serving-cert\") on node \"ip-10-0-136-0.ec2.internal\" DevicePath \"\"" Jul 03 11:21:02.109528 ip-10-0-136-0 kubenswrapper[2579]: I0703 11:21:02.109319 2579 reconciler_common.go:299] "Volume detached for volume \"console-config\" (UniqueName: \"kubernetes.io/configmap/baaa910d-775c-4be2-8d26-04897721653a-console-config\") on node \"ip-10-0-136-0.ec2.internal\" DevicePath \"\"" Jul 03 11:21:02.109528 ip-10-0-136-0 kubenswrapper[2579]: I0703 11:21:02.109327 2579 reconciler_common.go:299] "Volume detached for volume \"oauth-serving-cert\" (UniqueName: \"kubernetes.io/configmap/baaa910d-775c-4be2-8d26-04897721653a-oauth-serving-cert\") on node \"ip-10-0-136-0.ec2.internal\" DevicePath \"\"" Jul 03 11:21:02.731417 ip-10-0-136-0 kubenswrapper[2579]: I0703 11:21:02.731381 2579 generic.go:356] "Generic (PLEG): container finished" podID="baaa910d-775c-4be2-8d26-04897721653a" containerID="90ccd314b814cc45d99ae531fa903877474b29e1da27f94db87f2313933a9fbf" exitCode=0 Jul 03 11:21:02.731844 ip-10-0-136-0 kubenswrapper[2579]: I0703 11:21:02.731427 2579 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-console/console-799d957b5b-vrh8b" Jul 03 11:21:02.731844 ip-10-0-136-0 kubenswrapper[2579]: I0703 11:21:02.731467 2579 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-console/console-799d957b5b-vrh8b" event={"ID":"baaa910d-775c-4be2-8d26-04897721653a","Type":"ContainerDied","Data":"90ccd314b814cc45d99ae531fa903877474b29e1da27f94db87f2313933a9fbf"} Jul 03 11:21:02.731844 ip-10-0-136-0 kubenswrapper[2579]: I0703 11:21:02.731524 2579 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-console/console-799d957b5b-vrh8b" event={"ID":"baaa910d-775c-4be2-8d26-04897721653a","Type":"ContainerDied","Data":"a3b14309c311367ee1d69a52cba8b947ad3b1f811f6ee8df456b21ddf7c81611"} Jul 03 11:21:02.731844 ip-10-0-136-0 kubenswrapper[2579]: I0703 11:21:02.731546 2579 scope.go:117] "RemoveContainer" containerID="90ccd314b814cc45d99ae531fa903877474b29e1da27f94db87f2313933a9fbf" Jul 03 11:21:02.742303 ip-10-0-136-0 kubenswrapper[2579]: I0703 11:21:02.742279 2579 scope.go:117] "RemoveContainer" containerID="90ccd314b814cc45d99ae531fa903877474b29e1da27f94db87f2313933a9fbf" Jul 03 11:21:02.742676 ip-10-0-136-0 kubenswrapper[2579]: E0703 11:21:02.742640 2579 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"90ccd314b814cc45d99ae531fa903877474b29e1da27f94db87f2313933a9fbf\": container with ID starting with 90ccd314b814cc45d99ae531fa903877474b29e1da27f94db87f2313933a9fbf not found: ID does not exist" containerID="90ccd314b814cc45d99ae531fa903877474b29e1da27f94db87f2313933a9fbf" Jul 03 11:21:02.742752 ip-10-0-136-0 kubenswrapper[2579]: I0703 11:21:02.742687 2579 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"90ccd314b814cc45d99ae531fa903877474b29e1da27f94db87f2313933a9fbf"} err="failed to get container status \"90ccd314b814cc45d99ae531fa903877474b29e1da27f94db87f2313933a9fbf\": rpc error: code = NotFound desc = could not find container \"90ccd314b814cc45d99ae531fa903877474b29e1da27f94db87f2313933a9fbf\": container with ID starting with 90ccd314b814cc45d99ae531fa903877474b29e1da27f94db87f2313933a9fbf not found: ID does not exist" Jul 03 11:21:02.756134 ip-10-0-136-0 kubenswrapper[2579]: I0703 11:21:02.756092 2579 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-console/console-799d957b5b-vrh8b"] Jul 03 11:21:02.758702 ip-10-0-136-0 kubenswrapper[2579]: I0703 11:21:02.758660 2579 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["openshift-console/console-799d957b5b-vrh8b"] Jul 03 11:21:03.992605 ip-10-0-136-0 kubenswrapper[2579]: I0703 11:21:03.992572 2579 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="baaa910d-775c-4be2-8d26-04897721653a" path="/var/lib/kubelet/pods/baaa910d-775c-4be2-8d26-04897721653a/volumes" Jul 03 11:21:05.693526 ip-10-0-136-0 kubenswrapper[2579]: I0703 11:21:05.693490 2579 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="kuadrant-system/kuadrant-operator-controller-manager-8684c9ff49-fk95g" Jul 03 11:21:07.529534 ip-10-0-136-0 kubenswrapper[2579]: I0703 11:21:07.529480 2579 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["kuadrant-system/kuadrant-operator-controller-manager-8684c9ff49-fk95g"] Jul 03 11:21:07.529966 ip-10-0-136-0 kubenswrapper[2579]: I0703 11:21:07.529705 2579 kuberuntime_container.go:872] "Killing container with a grace period" pod="kuadrant-system/kuadrant-operator-controller-manager-8684c9ff49-fk95g" podUID="0fd7b987-9598-4f1f-961b-097b517e0fbe" containerName="manager" containerID="cri-o://8e15a681075af5e4034ebd28bdfad855f405bed1f9e692bdebc04ab349086cb7" gracePeriod=2 Jul 03 11:21:07.538182 ip-10-0-136-0 kubenswrapper[2579]: E0703 11:21:07.538130 2579 status_manager.go:1041] "Failed to update status for pod" err="failed to patch status \"{\\\"metadata\\\":{\\\"uid\\\":\\\"0fd7b987-9598-4f1f-961b-097b517e0fbe\\\"},\\\"status\\\":{\\\"$setElementOrder/conditions\\\":[{\\\"type\\\":\\\"PodReadyToStartContainers\\\"},{\\\"type\\\":\\\"Initialized\\\"},{\\\"type\\\":\\\"Ready\\\"},{\\\"type\\\":\\\"ContainersReady\\\"},{\\\"type\\\":\\\"PodScheduled\\\"}],\\\"conditions\\\":[{\\\"observedGeneration\\\":2,\\\"type\\\":\\\"PodReadyToStartContainers\\\"},{\\\"observedGeneration\\\":2,\\\"type\\\":\\\"Initialized\\\"},{\\\"observedGeneration\\\":2,\\\"type\\\":\\\"Ready\\\"},{\\\"observedGeneration\\\":2,\\\"type\\\":\\\"ContainersReady\\\"},{\\\"observedGeneration\\\":2,\\\"type\\\":\\\"PodScheduled\\\"}],\\\"observedGeneration\\\":2}}\" for pod \"kuadrant-system\"/\"kuadrant-operator-controller-manager-8684c9ff49-fk95g\": pods \"kuadrant-operator-controller-manager-8684c9ff49-fk95g\" not found" pod="kuadrant-system/kuadrant-operator-controller-manager-8684c9ff49-fk95g" Jul 03 11:21:07.539398 ip-10-0-136-0 kubenswrapper[2579]: I0703 11:21:07.539362 2579 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["kuadrant-system/kuadrant-operator-controller-manager-8684c9ff49-fk95g"] Jul 03 11:21:07.549046 ip-10-0-136-0 kubenswrapper[2579]: I0703 11:21:07.548988 2579 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["kuadrant-system/limitador-operator-controller-manager-859d7b78bd-zc6jx"] Jul 03 11:21:07.549766 ip-10-0-136-0 kubenswrapper[2579]: I0703 11:21:07.549301 2579 kuberuntime_container.go:872] "Killing container with a grace period" pod="kuadrant-system/limitador-operator-controller-manager-859d7b78bd-zc6jx" podUID="b73c6525-f1be-4a04-a0b8-76bc1635b185" containerName="manager" containerID="cri-o://cfb5766892ad38b5fcc9c28b9c1db409eda3e757626b9399ef4c0ef6bba6ed12" gracePeriod=2 Jul 03 11:21:07.558638 ip-10-0-136-0 kubenswrapper[2579]: I0703 11:21:07.558596 2579 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["kuadrant-system/limitador-operator-controller-manager-859d7b78bd-zc6jx"] Jul 03 11:21:07.559087 ip-10-0-136-0 kubenswrapper[2579]: E0703 11:21:07.559053 2579 status_manager.go:1018] "Failed to get status for pod" err="pods \"kuadrant-operator-controller-manager-8684c9ff49-fk95g\" is forbidden: User \"system:node:ip-10-0-136-0.ec2.internal\" cannot get resource \"pods\" in API group \"\" in the namespace \"kuadrant-system\": no relationship found between node 'ip-10-0-136-0.ec2.internal' and this object" podUID="0fd7b987-9598-4f1f-961b-097b517e0fbe" pod="kuadrant-system/kuadrant-operator-controller-manager-8684c9ff49-fk95g" Jul 03 11:21:07.571589 ip-10-0-136-0 kubenswrapper[2579]: I0703 11:21:07.571540 2579 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["kuadrant-system/kuadrant-operator-controller-manager-8684c9ff49-srkrl"] Jul 03 11:21:07.572032 ip-10-0-136-0 kubenswrapper[2579]: I0703 11:21:07.572011 2579 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="0fd7b987-9598-4f1f-961b-097b517e0fbe" containerName="manager" Jul 03 11:21:07.572126 ip-10-0-136-0 kubenswrapper[2579]: I0703 11:21:07.572036 2579 state_mem.go:107] "Deleted CPUSet assignment" podUID="0fd7b987-9598-4f1f-961b-097b517e0fbe" containerName="manager" Jul 03 11:21:07.572126 ip-10-0-136-0 kubenswrapper[2579]: I0703 11:21:07.572048 2579 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="b73c6525-f1be-4a04-a0b8-76bc1635b185" containerName="manager" Jul 03 11:21:07.572126 ip-10-0-136-0 kubenswrapper[2579]: I0703 11:21:07.572056 2579 state_mem.go:107] "Deleted CPUSet assignment" podUID="b73c6525-f1be-4a04-a0b8-76bc1635b185" containerName="manager" Jul 03 11:21:07.572126 ip-10-0-136-0 kubenswrapper[2579]: I0703 11:21:07.572074 2579 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="baaa910d-775c-4be2-8d26-04897721653a" containerName="console" Jul 03 11:21:07.572126 ip-10-0-136-0 kubenswrapper[2579]: I0703 11:21:07.572082 2579 state_mem.go:107] "Deleted CPUSet assignment" podUID="baaa910d-775c-4be2-8d26-04897721653a" containerName="console" Jul 03 11:21:07.572401 ip-10-0-136-0 kubenswrapper[2579]: I0703 11:21:07.572174 2579 memory_manager.go:368] "RemoveStaleState removing state" podUID="baaa910d-775c-4be2-8d26-04897721653a" containerName="console" Jul 03 11:21:07.572401 ip-10-0-136-0 kubenswrapper[2579]: I0703 11:21:07.572209 2579 memory_manager.go:368] "RemoveStaleState removing state" podUID="b73c6525-f1be-4a04-a0b8-76bc1635b185" containerName="manager" Jul 03 11:21:07.572401 ip-10-0-136-0 kubenswrapper[2579]: I0703 11:21:07.572222 2579 memory_manager.go:368] "RemoveStaleState removing state" podUID="0fd7b987-9598-4f1f-961b-097b517e0fbe" containerName="manager" Jul 03 11:21:07.575791 ip-10-0-136-0 kubenswrapper[2579]: I0703 11:21:07.575755 2579 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/kuadrant-operator-controller-manager-8684c9ff49-srkrl" Jul 03 11:21:07.578476 ip-10-0-136-0 kubenswrapper[2579]: E0703 11:21:07.578421 2579 status_manager.go:1018] "Failed to get status for pod" err="pods \"kuadrant-operator-controller-manager-8684c9ff49-fk95g\" is forbidden: User \"system:node:ip-10-0-136-0.ec2.internal\" cannot get resource \"pods\" in API group \"\" in the namespace \"kuadrant-system\": no relationship found between node 'ip-10-0-136-0.ec2.internal' and this object" podUID="0fd7b987-9598-4f1f-961b-097b517e0fbe" pod="kuadrant-system/kuadrant-operator-controller-manager-8684c9ff49-fk95g" Jul 03 11:21:07.581067 ip-10-0-136-0 kubenswrapper[2579]: E0703 11:21:07.581023 2579 status_manager.go:1018] "Failed to get status for pod" err="pods \"limitador-operator-controller-manager-859d7b78bd-zc6jx\" is forbidden: User \"system:node:ip-10-0-136-0.ec2.internal\" cannot get resource \"pods\" in API group \"\" in the namespace \"kuadrant-system\": no relationship found between node 'ip-10-0-136-0.ec2.internal' and this object" podUID="b73c6525-f1be-4a04-a0b8-76bc1635b185" pod="kuadrant-system/limitador-operator-controller-manager-859d7b78bd-zc6jx" Jul 03 11:21:07.583860 ip-10-0-136-0 kubenswrapper[2579]: I0703 11:21:07.583818 2579 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["kuadrant-system/limitador-operator-controller-manager-859d7b78bd-qk6jn"] Jul 03 11:21:07.587476 ip-10-0-136-0 kubenswrapper[2579]: I0703 11:21:07.587448 2579 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/limitador-operator-controller-manager-859d7b78bd-qk6jn" Jul 03 11:21:07.593608 ip-10-0-136-0 kubenswrapper[2579]: I0703 11:21:07.593567 2579 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/kuadrant-operator-controller-manager-8684c9ff49-srkrl"] Jul 03 11:21:07.604334 ip-10-0-136-0 kubenswrapper[2579]: I0703 11:21:07.604290 2579 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-operator-controller-manager-859d7b78bd-qk6jn"] Jul 03 11:21:07.608434 ip-10-0-136-0 kubenswrapper[2579]: I0703 11:21:07.608386 2579 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["kuadrant-system/kuadrant-operator-controller-manager-86878c769f-ppdg8"] Jul 03 11:21:07.612712 ip-10-0-136-0 kubenswrapper[2579]: I0703 11:21:07.612682 2579 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/kuadrant-operator-controller-manager-86878c769f-ppdg8" Jul 03 11:21:07.614406 ip-10-0-136-0 kubenswrapper[2579]: E0703 11:21:07.614346 2579 status_manager.go:1018] "Failed to get status for pod" err="pods \"limitador-operator-controller-manager-859d7b78bd-zc6jx\" is forbidden: User \"system:node:ip-10-0-136-0.ec2.internal\" cannot get resource \"pods\" in API group \"\" in the namespace \"kuadrant-system\": no relationship found between node 'ip-10-0-136-0.ec2.internal' and this object" podUID="b73c6525-f1be-4a04-a0b8-76bc1635b185" pod="kuadrant-system/limitador-operator-controller-manager-859d7b78bd-zc6jx" Jul 03 11:21:07.634521 ip-10-0-136-0 kubenswrapper[2579]: I0703 11:21:07.634474 2579 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/kuadrant-operator-controller-manager-86878c769f-ppdg8"] Jul 03 11:21:07.653369 ip-10-0-136-0 kubenswrapper[2579]: E0703 11:21:07.653325 2579 status_manager.go:1018] "Failed to get status for pod" err="pods \"kuadrant-operator-controller-manager-8684c9ff49-fk95g\" is forbidden: User \"system:node:ip-10-0-136-0.ec2.internal\" cannot get resource \"pods\" in API group \"\" in the namespace \"kuadrant-system\": no relationship found between node 'ip-10-0-136-0.ec2.internal' and this object" podUID="0fd7b987-9598-4f1f-961b-097b517e0fbe" pod="kuadrant-system/kuadrant-operator-controller-manager-8684c9ff49-fk95g" Jul 03 11:21:07.655721 ip-10-0-136-0 kubenswrapper[2579]: E0703 11:21:07.655681 2579 status_manager.go:1018] "Failed to get status for pod" err="pods \"limitador-operator-controller-manager-859d7b78bd-zc6jx\" is forbidden: User \"system:node:ip-10-0-136-0.ec2.internal\" cannot get resource \"pods\" in API group \"\" in the namespace \"kuadrant-system\": no relationship found between node 'ip-10-0-136-0.ec2.internal' and this object" podUID="b73c6525-f1be-4a04-a0b8-76bc1635b185" pod="kuadrant-system/limitador-operator-controller-manager-859d7b78bd-zc6jx" Jul 03 11:21:07.658082 ip-10-0-136-0 kubenswrapper[2579]: E0703 11:21:07.658004 2579 status_manager.go:1018] "Failed to get status for pod" err="pods \"kuadrant-operator-controller-manager-8684c9ff49-fk95g\" is forbidden: User \"system:node:ip-10-0-136-0.ec2.internal\" cannot get resource \"pods\" in API group \"\" in the namespace \"kuadrant-system\": no relationship found between node 'ip-10-0-136-0.ec2.internal' and this object" podUID="0fd7b987-9598-4f1f-961b-097b517e0fbe" pod="kuadrant-system/kuadrant-operator-controller-manager-8684c9ff49-fk95g" Jul 03 11:21:07.665645 ip-10-0-136-0 kubenswrapper[2579]: I0703 11:21:07.665602 2579 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-b9grv\" (UniqueName: \"kubernetes.io/projected/d52d4600-7249-41a7-8106-76ce87bef3c6-kube-api-access-b9grv\") pod \"kuadrant-operator-controller-manager-8684c9ff49-srkrl\" (UID: \"d52d4600-7249-41a7-8106-76ce87bef3c6\") " pod="kuadrant-system/kuadrant-operator-controller-manager-8684c9ff49-srkrl" Jul 03 11:21:07.665823 ip-10-0-136-0 kubenswrapper[2579]: I0703 11:21:07.665669 2579 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"extensions-socket-volume\" (UniqueName: \"kubernetes.io/empty-dir/a7680483-0b55-4ead-b345-9467eda81e16-extensions-socket-volume\") pod \"kuadrant-operator-controller-manager-86878c769f-ppdg8\" (UID: \"a7680483-0b55-4ead-b345-9467eda81e16\") " pod="kuadrant-system/kuadrant-operator-controller-manager-86878c769f-ppdg8" Jul 03 11:21:07.665823 ip-10-0-136-0 kubenswrapper[2579]: I0703 11:21:07.665758 2579 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"extensions-socket-volume\" (UniqueName: \"kubernetes.io/empty-dir/d52d4600-7249-41a7-8106-76ce87bef3c6-extensions-socket-volume\") pod \"kuadrant-operator-controller-manager-8684c9ff49-srkrl\" (UID: \"d52d4600-7249-41a7-8106-76ce87bef3c6\") " pod="kuadrant-system/kuadrant-operator-controller-manager-8684c9ff49-srkrl" Jul 03 11:21:07.665823 ip-10-0-136-0 kubenswrapper[2579]: I0703 11:21:07.665775 2579 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-rpx5d\" (UniqueName: \"kubernetes.io/projected/73128bfa-b0b4-47ee-b529-ba648905789e-kube-api-access-rpx5d\") pod \"limitador-operator-controller-manager-859d7b78bd-qk6jn\" (UID: \"73128bfa-b0b4-47ee-b529-ba648905789e\") " pod="kuadrant-system/limitador-operator-controller-manager-859d7b78bd-qk6jn" Jul 03 11:21:07.665823 ip-10-0-136-0 kubenswrapper[2579]: I0703 11:21:07.665798 2579 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-6jhdh\" (UniqueName: \"kubernetes.io/projected/a7680483-0b55-4ead-b345-9467eda81e16-kube-api-access-6jhdh\") pod \"kuadrant-operator-controller-manager-86878c769f-ppdg8\" (UID: \"a7680483-0b55-4ead-b345-9467eda81e16\") " pod="kuadrant-system/kuadrant-operator-controller-manager-86878c769f-ppdg8" Jul 03 11:21:07.764814 ip-10-0-136-0 kubenswrapper[2579]: I0703 11:21:07.764781 2579 generic.go:356] "Generic (PLEG): container finished" podID="0fd7b987-9598-4f1f-961b-097b517e0fbe" containerID="8e15a681075af5e4034ebd28bdfad855f405bed1f9e692bdebc04ab349086cb7" exitCode=0 Jul 03 11:21:07.766280 ip-10-0-136-0 kubenswrapper[2579]: I0703 11:21:07.766242 2579 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"extensions-socket-volume\" (UniqueName: \"kubernetes.io/empty-dir/d52d4600-7249-41a7-8106-76ce87bef3c6-extensions-socket-volume\") pod \"kuadrant-operator-controller-manager-8684c9ff49-srkrl\" (UID: \"d52d4600-7249-41a7-8106-76ce87bef3c6\") " pod="kuadrant-system/kuadrant-operator-controller-manager-8684c9ff49-srkrl" Jul 03 11:21:07.766470 ip-10-0-136-0 kubenswrapper[2579]: I0703 11:21:07.766292 2579 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-rpx5d\" (UniqueName: \"kubernetes.io/projected/73128bfa-b0b4-47ee-b529-ba648905789e-kube-api-access-rpx5d\") pod \"limitador-operator-controller-manager-859d7b78bd-qk6jn\" (UID: \"73128bfa-b0b4-47ee-b529-ba648905789e\") " pod="kuadrant-system/limitador-operator-controller-manager-859d7b78bd-qk6jn" Jul 03 11:21:07.766470 ip-10-0-136-0 kubenswrapper[2579]: I0703 11:21:07.766332 2579 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-6jhdh\" (UniqueName: \"kubernetes.io/projected/a7680483-0b55-4ead-b345-9467eda81e16-kube-api-access-6jhdh\") pod \"kuadrant-operator-controller-manager-86878c769f-ppdg8\" (UID: \"a7680483-0b55-4ead-b345-9467eda81e16\") " pod="kuadrant-system/kuadrant-operator-controller-manager-86878c769f-ppdg8" Jul 03 11:21:07.766470 ip-10-0-136-0 kubenswrapper[2579]: I0703 11:21:07.766376 2579 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-b9grv\" (UniqueName: \"kubernetes.io/projected/d52d4600-7249-41a7-8106-76ce87bef3c6-kube-api-access-b9grv\") pod \"kuadrant-operator-controller-manager-8684c9ff49-srkrl\" (UID: \"d52d4600-7249-41a7-8106-76ce87bef3c6\") " pod="kuadrant-system/kuadrant-operator-controller-manager-8684c9ff49-srkrl" Jul 03 11:21:07.766470 ip-10-0-136-0 kubenswrapper[2579]: I0703 11:21:07.766433 2579 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"extensions-socket-volume\" (UniqueName: \"kubernetes.io/empty-dir/a7680483-0b55-4ead-b345-9467eda81e16-extensions-socket-volume\") pod \"kuadrant-operator-controller-manager-86878c769f-ppdg8\" (UID: \"a7680483-0b55-4ead-b345-9467eda81e16\") " pod="kuadrant-system/kuadrant-operator-controller-manager-86878c769f-ppdg8" Jul 03 11:21:07.766739 ip-10-0-136-0 kubenswrapper[2579]: I0703 11:21:07.766716 2579 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"extensions-socket-volume\" (UniqueName: \"kubernetes.io/empty-dir/d52d4600-7249-41a7-8106-76ce87bef3c6-extensions-socket-volume\") pod \"kuadrant-operator-controller-manager-8684c9ff49-srkrl\" (UID: \"d52d4600-7249-41a7-8106-76ce87bef3c6\") " pod="kuadrant-system/kuadrant-operator-controller-manager-8684c9ff49-srkrl" Jul 03 11:21:07.767022 ip-10-0-136-0 kubenswrapper[2579]: I0703 11:21:07.766832 2579 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"extensions-socket-volume\" (UniqueName: \"kubernetes.io/empty-dir/a7680483-0b55-4ead-b345-9467eda81e16-extensions-socket-volume\") pod \"kuadrant-operator-controller-manager-86878c769f-ppdg8\" (UID: \"a7680483-0b55-4ead-b345-9467eda81e16\") " pod="kuadrant-system/kuadrant-operator-controller-manager-86878c769f-ppdg8" Jul 03 11:21:07.767351 ip-10-0-136-0 kubenswrapper[2579]: I0703 11:21:07.767325 2579 generic.go:356] "Generic (PLEG): container finished" podID="b73c6525-f1be-4a04-a0b8-76bc1635b185" containerID="cfb5766892ad38b5fcc9c28b9c1db409eda3e757626b9399ef4c0ef6bba6ed12" exitCode=0 Jul 03 11:21:07.782938 ip-10-0-136-0 kubenswrapper[2579]: I0703 11:21:07.782849 2579 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-6jhdh\" (UniqueName: \"kubernetes.io/projected/a7680483-0b55-4ead-b345-9467eda81e16-kube-api-access-6jhdh\") pod \"kuadrant-operator-controller-manager-86878c769f-ppdg8\" (UID: \"a7680483-0b55-4ead-b345-9467eda81e16\") " pod="kuadrant-system/kuadrant-operator-controller-manager-86878c769f-ppdg8" Jul 03 11:21:07.782938 ip-10-0-136-0 kubenswrapper[2579]: I0703 11:21:07.782873 2579 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-b9grv\" (UniqueName: \"kubernetes.io/projected/d52d4600-7249-41a7-8106-76ce87bef3c6-kube-api-access-b9grv\") pod \"kuadrant-operator-controller-manager-8684c9ff49-srkrl\" (UID: \"d52d4600-7249-41a7-8106-76ce87bef3c6\") " pod="kuadrant-system/kuadrant-operator-controller-manager-8684c9ff49-srkrl" Jul 03 11:21:07.782938 ip-10-0-136-0 kubenswrapper[2579]: I0703 11:21:07.782873 2579 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-rpx5d\" (UniqueName: \"kubernetes.io/projected/73128bfa-b0b4-47ee-b529-ba648905789e-kube-api-access-rpx5d\") pod \"limitador-operator-controller-manager-859d7b78bd-qk6jn\" (UID: \"73128bfa-b0b4-47ee-b529-ba648905789e\") " pod="kuadrant-system/limitador-operator-controller-manager-859d7b78bd-qk6jn" Jul 03 11:21:07.821513 ip-10-0-136-0 kubenswrapper[2579]: I0703 11:21:07.821483 2579 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/kuadrant-operator-controller-manager-8684c9ff49-fk95g" Jul 03 11:21:07.824052 ip-10-0-136-0 kubenswrapper[2579]: I0703 11:21:07.824021 2579 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/limitador-operator-controller-manager-859d7b78bd-zc6jx" Jul 03 11:21:07.824209 ip-10-0-136-0 kubenswrapper[2579]: E0703 11:21:07.824019 2579 status_manager.go:1018] "Failed to get status for pod" err="pods \"kuadrant-operator-controller-manager-8684c9ff49-fk95g\" is forbidden: User \"system:node:ip-10-0-136-0.ec2.internal\" cannot get resource \"pods\" in API group \"\" in the namespace \"kuadrant-system\": no relationship found between node 'ip-10-0-136-0.ec2.internal' and this object" podUID="0fd7b987-9598-4f1f-961b-097b517e0fbe" pod="kuadrant-system/kuadrant-operator-controller-manager-8684c9ff49-fk95g" Jul 03 11:21:07.826374 ip-10-0-136-0 kubenswrapper[2579]: E0703 11:21:07.826338 2579 status_manager.go:1018] "Failed to get status for pod" err="pods \"limitador-operator-controller-manager-859d7b78bd-zc6jx\" is forbidden: User \"system:node:ip-10-0-136-0.ec2.internal\" cannot get resource \"pods\" in API group \"\" in the namespace \"kuadrant-system\": no relationship found between node 'ip-10-0-136-0.ec2.internal' and this object" podUID="b73c6525-f1be-4a04-a0b8-76bc1635b185" pod="kuadrant-system/limitador-operator-controller-manager-859d7b78bd-zc6jx" Jul 03 11:21:07.828719 ip-10-0-136-0 kubenswrapper[2579]: E0703 11:21:07.828684 2579 status_manager.go:1018] "Failed to get status for pod" err="pods \"kuadrant-operator-controller-manager-8684c9ff49-fk95g\" is forbidden: User \"system:node:ip-10-0-136-0.ec2.internal\" cannot get resource \"pods\" in API group \"\" in the namespace \"kuadrant-system\": no relationship found between node 'ip-10-0-136-0.ec2.internal' and this object" podUID="0fd7b987-9598-4f1f-961b-097b517e0fbe" pod="kuadrant-system/kuadrant-operator-controller-manager-8684c9ff49-fk95g" Jul 03 11:21:07.831085 ip-10-0-136-0 kubenswrapper[2579]: E0703 11:21:07.831056 2579 status_manager.go:1018] "Failed to get status for pod" err="pods \"limitador-operator-controller-manager-859d7b78bd-zc6jx\" is forbidden: User \"system:node:ip-10-0-136-0.ec2.internal\" cannot get resource \"pods\" in API group \"\" in the namespace \"kuadrant-system\": no relationship found between node 'ip-10-0-136-0.ec2.internal' and this object" podUID="b73c6525-f1be-4a04-a0b8-76bc1635b185" pod="kuadrant-system/limitador-operator-controller-manager-859d7b78bd-zc6jx" Jul 03 11:21:07.867648 ip-10-0-136-0 kubenswrapper[2579]: I0703 11:21:07.867603 2579 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-nvxmw\" (UniqueName: \"kubernetes.io/projected/0fd7b987-9598-4f1f-961b-097b517e0fbe-kube-api-access-nvxmw\") pod \"0fd7b987-9598-4f1f-961b-097b517e0fbe\" (UID: \"0fd7b987-9598-4f1f-961b-097b517e0fbe\") " Jul 03 11:21:07.867839 ip-10-0-136-0 kubenswrapper[2579]: I0703 11:21:07.867706 2579 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-qs8ft\" (UniqueName: \"kubernetes.io/projected/b73c6525-f1be-4a04-a0b8-76bc1635b185-kube-api-access-qs8ft\") pod \"b73c6525-f1be-4a04-a0b8-76bc1635b185\" (UID: \"b73c6525-f1be-4a04-a0b8-76bc1635b185\") " Jul 03 11:21:07.867839 ip-10-0-136-0 kubenswrapper[2579]: I0703 11:21:07.867738 2579 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"extensions-socket-volume\" (UniqueName: \"kubernetes.io/empty-dir/0fd7b987-9598-4f1f-961b-097b517e0fbe-extensions-socket-volume\") pod \"0fd7b987-9598-4f1f-961b-097b517e0fbe\" (UID: \"0fd7b987-9598-4f1f-961b-097b517e0fbe\") " Jul 03 11:21:07.868435 ip-10-0-136-0 kubenswrapper[2579]: I0703 11:21:07.868323 2579 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/0fd7b987-9598-4f1f-961b-097b517e0fbe-extensions-socket-volume" (OuterVolumeSpecName: "extensions-socket-volume") pod "0fd7b987-9598-4f1f-961b-097b517e0fbe" (UID: "0fd7b987-9598-4f1f-961b-097b517e0fbe"). InnerVolumeSpecName "extensions-socket-volume". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Jul 03 11:21:07.870133 ip-10-0-136-0 kubenswrapper[2579]: I0703 11:21:07.870104 2579 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/0fd7b987-9598-4f1f-961b-097b517e0fbe-kube-api-access-nvxmw" (OuterVolumeSpecName: "kube-api-access-nvxmw") pod "0fd7b987-9598-4f1f-961b-097b517e0fbe" (UID: "0fd7b987-9598-4f1f-961b-097b517e0fbe"). InnerVolumeSpecName "kube-api-access-nvxmw". PluginName "kubernetes.io/projected", VolumeGIDValue "" Jul 03 11:21:07.870289 ip-10-0-136-0 kubenswrapper[2579]: I0703 11:21:07.870142 2579 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/b73c6525-f1be-4a04-a0b8-76bc1635b185-kube-api-access-qs8ft" (OuterVolumeSpecName: "kube-api-access-qs8ft") pod "b73c6525-f1be-4a04-a0b8-76bc1635b185" (UID: "b73c6525-f1be-4a04-a0b8-76bc1635b185"). InnerVolumeSpecName "kube-api-access-qs8ft". PluginName "kubernetes.io/projected", VolumeGIDValue "" Jul 03 11:21:07.969297 ip-10-0-136-0 kubenswrapper[2579]: I0703 11:21:07.969251 2579 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-nvxmw\" (UniqueName: \"kubernetes.io/projected/0fd7b987-9598-4f1f-961b-097b517e0fbe-kube-api-access-nvxmw\") on node \"ip-10-0-136-0.ec2.internal\" DevicePath \"\"" Jul 03 11:21:07.969297 ip-10-0-136-0 kubenswrapper[2579]: I0703 11:21:07.969287 2579 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-qs8ft\" (UniqueName: \"kubernetes.io/projected/b73c6525-f1be-4a04-a0b8-76bc1635b185-kube-api-access-qs8ft\") on node \"ip-10-0-136-0.ec2.internal\" DevicePath \"\"" Jul 03 11:21:07.969297 ip-10-0-136-0 kubenswrapper[2579]: I0703 11:21:07.969297 2579 reconciler_common.go:299] "Volume detached for volume \"extensions-socket-volume\" (UniqueName: \"kubernetes.io/empty-dir/0fd7b987-9598-4f1f-961b-097b517e0fbe-extensions-socket-volume\") on node \"ip-10-0-136-0.ec2.internal\" DevicePath \"\"" Jul 03 11:21:07.979132 ip-10-0-136-0 kubenswrapper[2579]: I0703 11:21:07.979090 2579 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/kuadrant-operator-controller-manager-8684c9ff49-srkrl" Jul 03 11:21:07.991347 ip-10-0-136-0 kubenswrapper[2579]: I0703 11:21:07.991310 2579 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/limitador-operator-controller-manager-859d7b78bd-qk6jn" Jul 03 11:21:07.992459 ip-10-0-136-0 kubenswrapper[2579]: I0703 11:21:07.992433 2579 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="0fd7b987-9598-4f1f-961b-097b517e0fbe" path="/var/lib/kubelet/pods/0fd7b987-9598-4f1f-961b-097b517e0fbe/volumes" Jul 03 11:21:07.992864 ip-10-0-136-0 kubenswrapper[2579]: I0703 11:21:07.992847 2579 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="b73c6525-f1be-4a04-a0b8-76bc1635b185" path="/var/lib/kubelet/pods/b73c6525-f1be-4a04-a0b8-76bc1635b185/volumes" Jul 03 11:21:07.999089 ip-10-0-136-0 kubenswrapper[2579]: I0703 11:21:07.999054 2579 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/kuadrant-operator-controller-manager-86878c769f-ppdg8" Jul 03 11:21:08.188570 ip-10-0-136-0 kubenswrapper[2579]: I0703 11:21:08.188537 2579 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/kuadrant-operator-controller-manager-8684c9ff49-srkrl"] Jul 03 11:21:08.425414 ip-10-0-136-0 kubenswrapper[2579]: I0703 11:21:08.425381 2579 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/kuadrant-operator-controller-manager-86878c769f-ppdg8"] Jul 03 11:21:08.426782 ip-10-0-136-0 kubenswrapper[2579]: W0703 11:21:08.426751 2579 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-poda7680483_0b55_4ead_b345_9467eda81e16.slice/crio-6e55279c24f2af2ba050fa0706b11e76017d9dab490fab25dae9c732e5c5963b WatchSource:0}: Error finding container 6e55279c24f2af2ba050fa0706b11e76017d9dab490fab25dae9c732e5c5963b: Status 404 returned error can't find the container with id 6e55279c24f2af2ba050fa0706b11e76017d9dab490fab25dae9c732e5c5963b Jul 03 11:21:08.440580 ip-10-0-136-0 kubenswrapper[2579]: I0703 11:21:08.440541 2579 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-operator-controller-manager-859d7b78bd-qk6jn"] Jul 03 11:21:08.442456 ip-10-0-136-0 kubenswrapper[2579]: W0703 11:21:08.442425 2579 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod73128bfa_b0b4_47ee_b529_ba648905789e.slice/crio-fdcbfcd937549aa986f4fce633e0dde309aae55ce05256c30b2eab020a3c3b92 WatchSource:0}: Error finding container fdcbfcd937549aa986f4fce633e0dde309aae55ce05256c30b2eab020a3c3b92: Status 404 returned error can't find the container with id fdcbfcd937549aa986f4fce633e0dde309aae55ce05256c30b2eab020a3c3b92 Jul 03 11:21:08.776455 ip-10-0-136-0 kubenswrapper[2579]: I0703 11:21:08.776421 2579 scope.go:117] "RemoveContainer" containerID="8e15a681075af5e4034ebd28bdfad855f405bed1f9e692bdebc04ab349086cb7" Jul 03 11:21:08.776963 ip-10-0-136-0 kubenswrapper[2579]: I0703 11:21:08.776426 2579 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/kuadrant-operator-controller-manager-8684c9ff49-fk95g" Jul 03 11:21:08.778760 ip-10-0-136-0 kubenswrapper[2579]: I0703 11:21:08.778615 2579 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/limitador-operator-controller-manager-859d7b78bd-qk6jn" event={"ID":"73128bfa-b0b4-47ee-b529-ba648905789e","Type":"ContainerStarted","Data":"c8176ff8acbb1c943affd61a99ecff489b18e30c9f233888437c734e540c8d78"} Jul 03 11:21:08.778760 ip-10-0-136-0 kubenswrapper[2579]: I0703 11:21:08.778696 2579 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/limitador-operator-controller-manager-859d7b78bd-qk6jn" event={"ID":"73128bfa-b0b4-47ee-b529-ba648905789e","Type":"ContainerStarted","Data":"fdcbfcd937549aa986f4fce633e0dde309aae55ce05256c30b2eab020a3c3b92"} Jul 03 11:21:08.778760 ip-10-0-136-0 kubenswrapper[2579]: I0703 11:21:08.778729 2579 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="kuadrant-system/limitador-operator-controller-manager-859d7b78bd-qk6jn" Jul 03 11:21:08.780981 ip-10-0-136-0 kubenswrapper[2579]: I0703 11:21:08.780947 2579 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/kuadrant-operator-controller-manager-86878c769f-ppdg8" event={"ID":"a7680483-0b55-4ead-b345-9467eda81e16","Type":"ContainerStarted","Data":"549207d79b521dd59998441b8c5f2efbb35cdc452e428c14eec0f1387d601d00"} Jul 03 11:21:08.781141 ip-10-0-136-0 kubenswrapper[2579]: I0703 11:21:08.781005 2579 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/kuadrant-operator-controller-manager-86878c769f-ppdg8" event={"ID":"a7680483-0b55-4ead-b345-9467eda81e16","Type":"ContainerStarted","Data":"6e55279c24f2af2ba050fa0706b11e76017d9dab490fab25dae9c732e5c5963b"} Jul 03 11:21:08.781141 ip-10-0-136-0 kubenswrapper[2579]: I0703 11:21:08.781046 2579 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="kuadrant-system/kuadrant-operator-controller-manager-86878c769f-ppdg8" Jul 03 11:21:08.782573 ip-10-0-136-0 kubenswrapper[2579]: I0703 11:21:08.782539 2579 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/limitador-operator-controller-manager-859d7b78bd-zc6jx" Jul 03 11:21:08.784687 ip-10-0-136-0 kubenswrapper[2579]: I0703 11:21:08.784645 2579 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/kuadrant-operator-controller-manager-8684c9ff49-srkrl" event={"ID":"d52d4600-7249-41a7-8106-76ce87bef3c6","Type":"ContainerStarted","Data":"f583e932ede600f98e90927b60f1e5fa1c8732559993f8c72a72f46fe16035cf"} Jul 03 11:21:08.784866 ip-10-0-136-0 kubenswrapper[2579]: I0703 11:21:08.784750 2579 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/kuadrant-operator-controller-manager-8684c9ff49-srkrl" event={"ID":"d52d4600-7249-41a7-8106-76ce87bef3c6","Type":"ContainerStarted","Data":"f64703febec71dc05901a638ad6cbac2f5467483db8fa54d2610fc41b54f53c8"} Jul 03 11:21:08.785014 ip-10-0-136-0 kubenswrapper[2579]: I0703 11:21:08.784995 2579 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="kuadrant-system/kuadrant-operator-controller-manager-8684c9ff49-srkrl" Jul 03 11:21:08.787646 ip-10-0-136-0 kubenswrapper[2579]: I0703 11:21:08.787618 2579 scope.go:117] "RemoveContainer" containerID="cfb5766892ad38b5fcc9c28b9c1db409eda3e757626b9399ef4c0ef6bba6ed12" Jul 03 11:21:08.819062 ip-10-0-136-0 kubenswrapper[2579]: I0703 11:21:08.818999 2579 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="kuadrant-system/limitador-operator-controller-manager-859d7b78bd-qk6jn" podStartSLOduration=1.8189821419999999 podStartE2EDuration="1.818982142s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-03 11:21:07 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-03 11:21:08.817798261 +0000 UTC m=+525.406102581" watchObservedRunningTime="2026-07-03 11:21:08.818982142 +0000 UTC m=+525.407286463" Jul 03 11:21:08.882769 ip-10-0-136-0 kubenswrapper[2579]: I0703 11:21:08.882701 2579 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="kuadrant-system/kuadrant-operator-controller-manager-8684c9ff49-srkrl" podStartSLOduration=1.8826793450000001 podStartE2EDuration="1.882679345s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-03 11:21:07 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-03 11:21:08.880259767 +0000 UTC m=+525.468564087" watchObservedRunningTime="2026-07-03 11:21:08.882679345 +0000 UTC m=+525.470983662" Jul 03 11:21:08.957594 ip-10-0-136-0 kubenswrapper[2579]: I0703 11:21:08.957515 2579 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="kuadrant-system/kuadrant-operator-controller-manager-86878c769f-ppdg8" podStartSLOduration=1.957495558 podStartE2EDuration="1.957495558s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-03 11:21:07 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-03 11:21:08.955168412 +0000 UTC m=+525.543472735" watchObservedRunningTime="2026-07-03 11:21:08.957495558 +0000 UTC m=+525.545799878" Jul 03 11:21:19.795937 ip-10-0-136-0 kubenswrapper[2579]: I0703 11:21:19.795850 2579 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="kuadrant-system/limitador-operator-controller-manager-859d7b78bd-qk6jn" Jul 03 11:21:19.796510 ip-10-0-136-0 kubenswrapper[2579]: I0703 11:21:19.795957 2579 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="kuadrant-system/kuadrant-operator-controller-manager-86878c769f-ppdg8" Jul 03 11:21:19.796510 ip-10-0-136-0 kubenswrapper[2579]: I0703 11:21:19.796010 2579 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="kuadrant-system/kuadrant-operator-controller-manager-8684c9ff49-srkrl" Jul 03 11:21:19.925984 ip-10-0-136-0 kubenswrapper[2579]: I0703 11:21:19.925947 2579 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["kuadrant-system/kuadrant-operator-controller-manager-8684c9ff49-srkrl"] Jul 03 11:21:19.926214 ip-10-0-136-0 kubenswrapper[2579]: I0703 11:21:19.926164 2579 kuberuntime_container.go:872] "Killing container with a grace period" pod="kuadrant-system/kuadrant-operator-controller-manager-8684c9ff49-srkrl" podUID="d52d4600-7249-41a7-8106-76ce87bef3c6" containerName="manager" containerID="cri-o://f583e932ede600f98e90927b60f1e5fa1c8732559993f8c72a72f46fe16035cf" gracePeriod=10 Jul 03 11:21:20.187892 ip-10-0-136-0 kubenswrapper[2579]: I0703 11:21:20.187856 2579 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/kuadrant-operator-controller-manager-8684c9ff49-srkrl" Jul 03 11:21:20.290235 ip-10-0-136-0 kubenswrapper[2579]: I0703 11:21:20.290166 2579 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-b9grv\" (UniqueName: \"kubernetes.io/projected/d52d4600-7249-41a7-8106-76ce87bef3c6-kube-api-access-b9grv\") pod \"d52d4600-7249-41a7-8106-76ce87bef3c6\" (UID: \"d52d4600-7249-41a7-8106-76ce87bef3c6\") " Jul 03 11:21:20.290456 ip-10-0-136-0 kubenswrapper[2579]: I0703 11:21:20.290289 2579 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"extensions-socket-volume\" (UniqueName: \"kubernetes.io/empty-dir/d52d4600-7249-41a7-8106-76ce87bef3c6-extensions-socket-volume\") pod \"d52d4600-7249-41a7-8106-76ce87bef3c6\" (UID: \"d52d4600-7249-41a7-8106-76ce87bef3c6\") " Jul 03 11:21:20.290745 ip-10-0-136-0 kubenswrapper[2579]: I0703 11:21:20.290717 2579 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/d52d4600-7249-41a7-8106-76ce87bef3c6-extensions-socket-volume" (OuterVolumeSpecName: "extensions-socket-volume") pod "d52d4600-7249-41a7-8106-76ce87bef3c6" (UID: "d52d4600-7249-41a7-8106-76ce87bef3c6"). InnerVolumeSpecName "extensions-socket-volume". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Jul 03 11:21:20.292726 ip-10-0-136-0 kubenswrapper[2579]: I0703 11:21:20.292684 2579 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/d52d4600-7249-41a7-8106-76ce87bef3c6-kube-api-access-b9grv" (OuterVolumeSpecName: "kube-api-access-b9grv") pod "d52d4600-7249-41a7-8106-76ce87bef3c6" (UID: "d52d4600-7249-41a7-8106-76ce87bef3c6"). InnerVolumeSpecName "kube-api-access-b9grv". PluginName "kubernetes.io/projected", VolumeGIDValue "" Jul 03 11:21:20.391551 ip-10-0-136-0 kubenswrapper[2579]: I0703 11:21:20.391450 2579 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-b9grv\" (UniqueName: \"kubernetes.io/projected/d52d4600-7249-41a7-8106-76ce87bef3c6-kube-api-access-b9grv\") on node \"ip-10-0-136-0.ec2.internal\" DevicePath \"\"" Jul 03 11:21:20.391551 ip-10-0-136-0 kubenswrapper[2579]: I0703 11:21:20.391484 2579 reconciler_common.go:299] "Volume detached for volume \"extensions-socket-volume\" (UniqueName: \"kubernetes.io/empty-dir/d52d4600-7249-41a7-8106-76ce87bef3c6-extensions-socket-volume\") on node \"ip-10-0-136-0.ec2.internal\" DevicePath \"\"" Jul 03 11:21:20.865290 ip-10-0-136-0 kubenswrapper[2579]: I0703 11:21:20.865250 2579 generic.go:356] "Generic (PLEG): container finished" podID="d52d4600-7249-41a7-8106-76ce87bef3c6" containerID="f583e932ede600f98e90927b60f1e5fa1c8732559993f8c72a72f46fe16035cf" exitCode=0 Jul 03 11:21:20.865791 ip-10-0-136-0 kubenswrapper[2579]: I0703 11:21:20.865330 2579 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/kuadrant-operator-controller-manager-8684c9ff49-srkrl" Jul 03 11:21:20.865791 ip-10-0-136-0 kubenswrapper[2579]: I0703 11:21:20.865332 2579 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/kuadrant-operator-controller-manager-8684c9ff49-srkrl" event={"ID":"d52d4600-7249-41a7-8106-76ce87bef3c6","Type":"ContainerDied","Data":"f583e932ede600f98e90927b60f1e5fa1c8732559993f8c72a72f46fe16035cf"} Jul 03 11:21:20.865791 ip-10-0-136-0 kubenswrapper[2579]: I0703 11:21:20.865396 2579 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/kuadrant-operator-controller-manager-8684c9ff49-srkrl" event={"ID":"d52d4600-7249-41a7-8106-76ce87bef3c6","Type":"ContainerDied","Data":"f64703febec71dc05901a638ad6cbac2f5467483db8fa54d2610fc41b54f53c8"} Jul 03 11:21:20.865791 ip-10-0-136-0 kubenswrapper[2579]: I0703 11:21:20.865419 2579 scope.go:117] "RemoveContainer" containerID="f583e932ede600f98e90927b60f1e5fa1c8732559993f8c72a72f46fe16035cf" Jul 03 11:21:20.876499 ip-10-0-136-0 kubenswrapper[2579]: I0703 11:21:20.876462 2579 scope.go:117] "RemoveContainer" containerID="f583e932ede600f98e90927b60f1e5fa1c8732559993f8c72a72f46fe16035cf" Jul 03 11:21:20.876865 ip-10-0-136-0 kubenswrapper[2579]: E0703 11:21:20.876845 2579 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"f583e932ede600f98e90927b60f1e5fa1c8732559993f8c72a72f46fe16035cf\": container with ID starting with f583e932ede600f98e90927b60f1e5fa1c8732559993f8c72a72f46fe16035cf not found: ID does not exist" containerID="f583e932ede600f98e90927b60f1e5fa1c8732559993f8c72a72f46fe16035cf" Jul 03 11:21:20.876924 ip-10-0-136-0 kubenswrapper[2579]: I0703 11:21:20.876876 2579 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"f583e932ede600f98e90927b60f1e5fa1c8732559993f8c72a72f46fe16035cf"} err="failed to get container status \"f583e932ede600f98e90927b60f1e5fa1c8732559993f8c72a72f46fe16035cf\": rpc error: code = NotFound desc = could not find container \"f583e932ede600f98e90927b60f1e5fa1c8732559993f8c72a72f46fe16035cf\": container with ID starting with f583e932ede600f98e90927b60f1e5fa1c8732559993f8c72a72f46fe16035cf not found: ID does not exist" Jul 03 11:21:20.900587 ip-10-0-136-0 kubenswrapper[2579]: I0703 11:21:20.900552 2579 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["kuadrant-system/kuadrant-operator-controller-manager-8684c9ff49-srkrl"] Jul 03 11:21:20.910350 ip-10-0-136-0 kubenswrapper[2579]: I0703 11:21:20.910309 2579 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["kuadrant-system/kuadrant-operator-controller-manager-8684c9ff49-srkrl"] Jul 03 11:21:21.991991 ip-10-0-136-0 kubenswrapper[2579]: I0703 11:21:21.991946 2579 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="d52d4600-7249-41a7-8106-76ce87bef3c6" path="/var/lib/kubelet/pods/d52d4600-7249-41a7-8106-76ce87bef3c6/volumes" Jul 03 11:21:36.273039 ip-10-0-136-0 kubenswrapper[2579]: I0703 11:21:36.272992 2579 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-ingress/maas-default-gateway-openshift-default-56654dcf7-fbkb8"] Jul 03 11:21:36.273544 ip-10-0-136-0 kubenswrapper[2579]: I0703 11:21:36.273523 2579 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="d52d4600-7249-41a7-8106-76ce87bef3c6" containerName="manager" Jul 03 11:21:36.273544 ip-10-0-136-0 kubenswrapper[2579]: I0703 11:21:36.273546 2579 state_mem.go:107] "Deleted CPUSet assignment" podUID="d52d4600-7249-41a7-8106-76ce87bef3c6" containerName="manager" Jul 03 11:21:36.273621 ip-10-0-136-0 kubenswrapper[2579]: I0703 11:21:36.273616 2579 memory_manager.go:368] "RemoveStaleState removing state" podUID="d52d4600-7249-41a7-8106-76ce87bef3c6" containerName="manager" Jul 03 11:21:36.279570 ip-10-0-136-0 kubenswrapper[2579]: I0703 11:21:36.279541 2579 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/maas-default-gateway-openshift-default-56654dcf7-fbkb8" Jul 03 11:21:36.283019 ip-10-0-136-0 kubenswrapper[2579]: I0703 11:21:36.282997 2579 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress\"/\"maas-default-gateway-openshift-default-dockercfg-rdd9q\"" Jul 03 11:21:36.290225 ip-10-0-136-0 kubenswrapper[2579]: I0703 11:21:36.290163 2579 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/maas-default-gateway-openshift-default-56654dcf7-fbkb8"] Jul 03 11:21:36.446499 ip-10-0-136-0 kubenswrapper[2579]: I0703 11:21:36.446454 2579 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-q6pd8\" (UniqueName: \"kubernetes.io/projected/b8f8d2d9-d723-4d69-b7a5-938e70db757f-kube-api-access-q6pd8\") pod \"maas-default-gateway-openshift-default-56654dcf7-fbkb8\" (UID: \"b8f8d2d9-d723-4d69-b7a5-938e70db757f\") " pod="openshift-ingress/maas-default-gateway-openshift-default-56654dcf7-fbkb8" Jul 03 11:21:36.446499 ip-10-0-136-0 kubenswrapper[2579]: I0703 11:21:36.446510 2579 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/b8f8d2d9-d723-4d69-b7a5-938e70db757f-istio-podinfo\") pod \"maas-default-gateway-openshift-default-56654dcf7-fbkb8\" (UID: \"b8f8d2d9-d723-4d69-b7a5-938e70db757f\") " pod="openshift-ingress/maas-default-gateway-openshift-default-56654dcf7-fbkb8" Jul 03 11:21:36.446819 ip-10-0-136-0 kubenswrapper[2579]: I0703 11:21:36.446659 2579 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/b8f8d2d9-d723-4d69-b7a5-938e70db757f-workload-certs\") pod \"maas-default-gateway-openshift-default-56654dcf7-fbkb8\" (UID: \"b8f8d2d9-d723-4d69-b7a5-938e70db757f\") " pod="openshift-ingress/maas-default-gateway-openshift-default-56654dcf7-fbkb8" Jul 03 11:21:36.446819 ip-10-0-136-0 kubenswrapper[2579]: I0703 11:21:36.446705 2579 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/b8f8d2d9-d723-4d69-b7a5-938e70db757f-istio-envoy\") pod \"maas-default-gateway-openshift-default-56654dcf7-fbkb8\" (UID: \"b8f8d2d9-d723-4d69-b7a5-938e70db757f\") " pod="openshift-ingress/maas-default-gateway-openshift-default-56654dcf7-fbkb8" Jul 03 11:21:36.446819 ip-10-0-136-0 kubenswrapper[2579]: I0703 11:21:36.446748 2579 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/b8f8d2d9-d723-4d69-b7a5-938e70db757f-workload-socket\") pod \"maas-default-gateway-openshift-default-56654dcf7-fbkb8\" (UID: \"b8f8d2d9-d723-4d69-b7a5-938e70db757f\") " pod="openshift-ingress/maas-default-gateway-openshift-default-56654dcf7-fbkb8" Jul 03 11:21:36.446819 ip-10-0-136-0 kubenswrapper[2579]: I0703 11:21:36.446787 2579 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/b8f8d2d9-d723-4d69-b7a5-938e70db757f-istio-token\") pod \"maas-default-gateway-openshift-default-56654dcf7-fbkb8\" (UID: \"b8f8d2d9-d723-4d69-b7a5-938e70db757f\") " pod="openshift-ingress/maas-default-gateway-openshift-default-56654dcf7-fbkb8" Jul 03 11:21:36.446819 ip-10-0-136-0 kubenswrapper[2579]: I0703 11:21:36.446812 2579 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/b8f8d2d9-d723-4d69-b7a5-938e70db757f-istiod-ca-cert\") pod \"maas-default-gateway-openshift-default-56654dcf7-fbkb8\" (UID: \"b8f8d2d9-d723-4d69-b7a5-938e70db757f\") " pod="openshift-ingress/maas-default-gateway-openshift-default-56654dcf7-fbkb8" Jul 03 11:21:36.447015 ip-10-0-136-0 kubenswrapper[2579]: I0703 11:21:36.446842 2579 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/b8f8d2d9-d723-4d69-b7a5-938e70db757f-istio-data\") pod \"maas-default-gateway-openshift-default-56654dcf7-fbkb8\" (UID: \"b8f8d2d9-d723-4d69-b7a5-938e70db757f\") " pod="openshift-ingress/maas-default-gateway-openshift-default-56654dcf7-fbkb8" Jul 03 11:21:36.447015 ip-10-0-136-0 kubenswrapper[2579]: I0703 11:21:36.446901 2579 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/b8f8d2d9-d723-4d69-b7a5-938e70db757f-credential-socket\") pod \"maas-default-gateway-openshift-default-56654dcf7-fbkb8\" (UID: \"b8f8d2d9-d723-4d69-b7a5-938e70db757f\") " pod="openshift-ingress/maas-default-gateway-openshift-default-56654dcf7-fbkb8" Jul 03 11:21:36.548077 ip-10-0-136-0 kubenswrapper[2579]: I0703 11:21:36.547963 2579 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/b8f8d2d9-d723-4d69-b7a5-938e70db757f-workload-certs\") pod \"maas-default-gateway-openshift-default-56654dcf7-fbkb8\" (UID: \"b8f8d2d9-d723-4d69-b7a5-938e70db757f\") " pod="openshift-ingress/maas-default-gateway-openshift-default-56654dcf7-fbkb8" Jul 03 11:21:36.548077 ip-10-0-136-0 kubenswrapper[2579]: I0703 11:21:36.548023 2579 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/b8f8d2d9-d723-4d69-b7a5-938e70db757f-istio-envoy\") pod \"maas-default-gateway-openshift-default-56654dcf7-fbkb8\" (UID: \"b8f8d2d9-d723-4d69-b7a5-938e70db757f\") " pod="openshift-ingress/maas-default-gateway-openshift-default-56654dcf7-fbkb8" Jul 03 11:21:36.548077 ip-10-0-136-0 kubenswrapper[2579]: I0703 11:21:36.548058 2579 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/b8f8d2d9-d723-4d69-b7a5-938e70db757f-workload-socket\") pod \"maas-default-gateway-openshift-default-56654dcf7-fbkb8\" (UID: \"b8f8d2d9-d723-4d69-b7a5-938e70db757f\") " pod="openshift-ingress/maas-default-gateway-openshift-default-56654dcf7-fbkb8" Jul 03 11:21:36.548380 ip-10-0-136-0 kubenswrapper[2579]: I0703 11:21:36.548090 2579 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/b8f8d2d9-d723-4d69-b7a5-938e70db757f-istio-token\") pod \"maas-default-gateway-openshift-default-56654dcf7-fbkb8\" (UID: \"b8f8d2d9-d723-4d69-b7a5-938e70db757f\") " pod="openshift-ingress/maas-default-gateway-openshift-default-56654dcf7-fbkb8" Jul 03 11:21:36.548380 ip-10-0-136-0 kubenswrapper[2579]: I0703 11:21:36.548115 2579 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/b8f8d2d9-d723-4d69-b7a5-938e70db757f-istiod-ca-cert\") pod \"maas-default-gateway-openshift-default-56654dcf7-fbkb8\" (UID: \"b8f8d2d9-d723-4d69-b7a5-938e70db757f\") " pod="openshift-ingress/maas-default-gateway-openshift-default-56654dcf7-fbkb8" Jul 03 11:21:36.548380 ip-10-0-136-0 kubenswrapper[2579]: I0703 11:21:36.548144 2579 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/b8f8d2d9-d723-4d69-b7a5-938e70db757f-istio-data\") pod \"maas-default-gateway-openshift-default-56654dcf7-fbkb8\" (UID: \"b8f8d2d9-d723-4d69-b7a5-938e70db757f\") " pod="openshift-ingress/maas-default-gateway-openshift-default-56654dcf7-fbkb8" Jul 03 11:21:36.548380 ip-10-0-136-0 kubenswrapper[2579]: I0703 11:21:36.548176 2579 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/b8f8d2d9-d723-4d69-b7a5-938e70db757f-credential-socket\") pod \"maas-default-gateway-openshift-default-56654dcf7-fbkb8\" (UID: \"b8f8d2d9-d723-4d69-b7a5-938e70db757f\") " pod="openshift-ingress/maas-default-gateway-openshift-default-56654dcf7-fbkb8" Jul 03 11:21:36.548380 ip-10-0-136-0 kubenswrapper[2579]: I0703 11:21:36.548275 2579 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-q6pd8\" (UniqueName: \"kubernetes.io/projected/b8f8d2d9-d723-4d69-b7a5-938e70db757f-kube-api-access-q6pd8\") pod \"maas-default-gateway-openshift-default-56654dcf7-fbkb8\" (UID: \"b8f8d2d9-d723-4d69-b7a5-938e70db757f\") " pod="openshift-ingress/maas-default-gateway-openshift-default-56654dcf7-fbkb8" Jul 03 11:21:36.548380 ip-10-0-136-0 kubenswrapper[2579]: I0703 11:21:36.548311 2579 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/b8f8d2d9-d723-4d69-b7a5-938e70db757f-istio-podinfo\") pod \"maas-default-gateway-openshift-default-56654dcf7-fbkb8\" (UID: \"b8f8d2d9-d723-4d69-b7a5-938e70db757f\") " pod="openshift-ingress/maas-default-gateway-openshift-default-56654dcf7-fbkb8" Jul 03 11:21:36.548717 ip-10-0-136-0 kubenswrapper[2579]: I0703 11:21:36.548477 2579 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/b8f8d2d9-d723-4d69-b7a5-938e70db757f-workload-certs\") pod \"maas-default-gateway-openshift-default-56654dcf7-fbkb8\" (UID: \"b8f8d2d9-d723-4d69-b7a5-938e70db757f\") " pod="openshift-ingress/maas-default-gateway-openshift-default-56654dcf7-fbkb8" Jul 03 11:21:36.548717 ip-10-0-136-0 kubenswrapper[2579]: I0703 11:21:36.548485 2579 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/b8f8d2d9-d723-4d69-b7a5-938e70db757f-workload-socket\") pod \"maas-default-gateway-openshift-default-56654dcf7-fbkb8\" (UID: \"b8f8d2d9-d723-4d69-b7a5-938e70db757f\") " pod="openshift-ingress/maas-default-gateway-openshift-default-56654dcf7-fbkb8" Jul 03 11:21:36.548832 ip-10-0-136-0 kubenswrapper[2579]: I0703 11:21:36.548726 2579 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/b8f8d2d9-d723-4d69-b7a5-938e70db757f-credential-socket\") pod \"maas-default-gateway-openshift-default-56654dcf7-fbkb8\" (UID: \"b8f8d2d9-d723-4d69-b7a5-938e70db757f\") " pod="openshift-ingress/maas-default-gateway-openshift-default-56654dcf7-fbkb8" Jul 03 11:21:36.548832 ip-10-0-136-0 kubenswrapper[2579]: I0703 11:21:36.548792 2579 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/b8f8d2d9-d723-4d69-b7a5-938e70db757f-istio-data\") pod \"maas-default-gateway-openshift-default-56654dcf7-fbkb8\" (UID: \"b8f8d2d9-d723-4d69-b7a5-938e70db757f\") " pod="openshift-ingress/maas-default-gateway-openshift-default-56654dcf7-fbkb8" Jul 03 11:21:36.549003 ip-10-0-136-0 kubenswrapper[2579]: I0703 11:21:36.548979 2579 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/b8f8d2d9-d723-4d69-b7a5-938e70db757f-istiod-ca-cert\") pod \"maas-default-gateway-openshift-default-56654dcf7-fbkb8\" (UID: \"b8f8d2d9-d723-4d69-b7a5-938e70db757f\") " pod="openshift-ingress/maas-default-gateway-openshift-default-56654dcf7-fbkb8" Jul 03 11:21:36.550670 ip-10-0-136-0 kubenswrapper[2579]: I0703 11:21:36.550630 2579 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/b8f8d2d9-d723-4d69-b7a5-938e70db757f-istio-envoy\") pod \"maas-default-gateway-openshift-default-56654dcf7-fbkb8\" (UID: \"b8f8d2d9-d723-4d69-b7a5-938e70db757f\") " pod="openshift-ingress/maas-default-gateway-openshift-default-56654dcf7-fbkb8" Jul 03 11:21:36.551388 ip-10-0-136-0 kubenswrapper[2579]: I0703 11:21:36.551362 2579 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/b8f8d2d9-d723-4d69-b7a5-938e70db757f-istio-podinfo\") pod \"maas-default-gateway-openshift-default-56654dcf7-fbkb8\" (UID: \"b8f8d2d9-d723-4d69-b7a5-938e70db757f\") " pod="openshift-ingress/maas-default-gateway-openshift-default-56654dcf7-fbkb8" Jul 03 11:21:36.563697 ip-10-0-136-0 kubenswrapper[2579]: I0703 11:21:36.563659 2579 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/b8f8d2d9-d723-4d69-b7a5-938e70db757f-istio-token\") pod \"maas-default-gateway-openshift-default-56654dcf7-fbkb8\" (UID: \"b8f8d2d9-d723-4d69-b7a5-938e70db757f\") " pod="openshift-ingress/maas-default-gateway-openshift-default-56654dcf7-fbkb8" Jul 03 11:21:36.563926 ip-10-0-136-0 kubenswrapper[2579]: I0703 11:21:36.563898 2579 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-q6pd8\" (UniqueName: \"kubernetes.io/projected/b8f8d2d9-d723-4d69-b7a5-938e70db757f-kube-api-access-q6pd8\") pod \"maas-default-gateway-openshift-default-56654dcf7-fbkb8\" (UID: \"b8f8d2d9-d723-4d69-b7a5-938e70db757f\") " pod="openshift-ingress/maas-default-gateway-openshift-default-56654dcf7-fbkb8" Jul 03 11:21:36.595632 ip-10-0-136-0 kubenswrapper[2579]: I0703 11:21:36.595591 2579 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/maas-default-gateway-openshift-default-56654dcf7-fbkb8" Jul 03 11:21:36.804881 ip-10-0-136-0 kubenswrapper[2579]: I0703 11:21:36.804841 2579 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/maas-default-gateway-openshift-default-56654dcf7-fbkb8"] Jul 03 11:21:36.806412 ip-10-0-136-0 kubenswrapper[2579]: W0703 11:21:36.806384 2579 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podb8f8d2d9_d723_4d69_b7a5_938e70db757f.slice/crio-4a2b43d52f8154a7b478ffd5b73af25e60d68720e162dd9f04ab16e5f7e51a50 WatchSource:0}: Error finding container 4a2b43d52f8154a7b478ffd5b73af25e60d68720e162dd9f04ab16e5f7e51a50: Status 404 returned error can't find the container with id 4a2b43d52f8154a7b478ffd5b73af25e60d68720e162dd9f04ab16e5f7e51a50 Jul 03 11:21:36.809180 ip-10-0-136-0 kubenswrapper[2579]: I0703 11:21:36.809142 2579 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"30892156Ki","pods":"250"} Jul 03 11:21:36.809320 ip-10-0-136-0 kubenswrapper[2579]: I0703 11:21:36.809247 2579 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"30892156Ki","pods":"250"} Jul 03 11:21:36.809320 ip-10-0-136-0 kubenswrapper[2579]: I0703 11:21:36.809291 2579 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"30892156Ki","pods":"250"} Jul 03 11:21:36.960657 ip-10-0-136-0 kubenswrapper[2579]: I0703 11:21:36.960620 2579 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/maas-default-gateway-openshift-default-56654dcf7-fbkb8" event={"ID":"b8f8d2d9-d723-4d69-b7a5-938e70db757f","Type":"ContainerStarted","Data":"f96ee7ec696d8c8edf044768ec0355f0aeb207330b737eaf555ec2b71de60a8c"} Jul 03 11:21:36.960880 ip-10-0-136-0 kubenswrapper[2579]: I0703 11:21:36.960685 2579 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/maas-default-gateway-openshift-default-56654dcf7-fbkb8" event={"ID":"b8f8d2d9-d723-4d69-b7a5-938e70db757f","Type":"ContainerStarted","Data":"4a2b43d52f8154a7b478ffd5b73af25e60d68720e162dd9f04ab16e5f7e51a50"} Jul 03 11:21:36.986237 ip-10-0-136-0 kubenswrapper[2579]: I0703 11:21:36.986129 2579 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-ingress/maas-default-gateway-openshift-default-56654dcf7-fbkb8" podStartSLOduration=0.986107389 podStartE2EDuration="986.107389ms" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-03 11:21:36 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-03 11:21:36.985752221 +0000 UTC m=+553.574056557" watchObservedRunningTime="2026-07-03 11:21:36.986107389 +0000 UTC m=+553.574411710" Jul 03 11:21:37.595948 ip-10-0-136-0 kubenswrapper[2579]: I0703 11:21:37.595899 2579 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="openshift-ingress/maas-default-gateway-openshift-default-56654dcf7-fbkb8" Jul 03 11:21:37.597778 ip-10-0-136-0 kubenswrapper[2579]: I0703 11:21:37.597745 2579 patch_prober.go:28] interesting pod/maas-default-gateway-openshift-default-56654dcf7-fbkb8 container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.133.0.51:15021/healthz/ready\": dial tcp 10.133.0.51:15021: connect: connection refused" start-of-body= Jul 03 11:21:37.597924 ip-10-0-136-0 kubenswrapper[2579]: I0703 11:21:37.597808 2579 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/maas-default-gateway-openshift-default-56654dcf7-fbkb8" podUID="b8f8d2d9-d723-4d69-b7a5-938e70db757f" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.133.0.51:15021/healthz/ready\": dial tcp 10.133.0.51:15021: connect: connection refused" Jul 03 11:21:38.600877 ip-10-0-136-0 kubenswrapper[2579]: I0703 11:21:38.600824 2579 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="openshift-ingress/maas-default-gateway-openshift-default-56654dcf7-fbkb8" Jul 03 11:21:38.972848 ip-10-0-136-0 kubenswrapper[2579]: I0703 11:21:38.972743 2579 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/maas-default-gateway-openshift-default-56654dcf7-fbkb8" Jul 03 11:21:38.974101 ip-10-0-136-0 kubenswrapper[2579]: I0703 11:21:38.974076 2579 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/maas-default-gateway-openshift-default-56654dcf7-fbkb8" Jul 03 11:21:39.104307 ip-10-0-136-0 kubenswrapper[2579]: I0703 11:21:39.104269 2579 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["kuadrant-system/authorino-58cd955675-p27zg"] Jul 03 11:21:39.108375 ip-10-0-136-0 kubenswrapper[2579]: I0703 11:21:39.108346 2579 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/authorino-58cd955675-p27zg" Jul 03 11:21:39.113600 ip-10-0-136-0 kubenswrapper[2579]: I0703 11:21:39.113570 2579 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"kuadrant-system\"/\"authorino-authorino-dockercfg-v4s65\"" Jul 03 11:21:39.127033 ip-10-0-136-0 kubenswrapper[2579]: I0703 11:21:39.126997 2579 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/authorino-58cd955675-p27zg"] Jul 03 11:21:39.144790 ip-10-0-136-0 kubenswrapper[2579]: I0703 11:21:39.144731 2579 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["kuadrant-system/authorino-75d556bd77-b2p87"] Jul 03 11:21:39.148740 ip-10-0-136-0 kubenswrapper[2579]: I0703 11:21:39.148708 2579 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/authorino-75d556bd77-b2p87" Jul 03 11:21:39.165423 ip-10-0-136-0 kubenswrapper[2579]: I0703 11:21:39.165387 2579 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/authorino-75d556bd77-b2p87"] Jul 03 11:21:39.171752 ip-10-0-136-0 kubenswrapper[2579]: I0703 11:21:39.171715 2579 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-xsgvt\" (UniqueName: \"kubernetes.io/projected/63001d04-53b7-4ad9-99a2-05b4c9e80f86-kube-api-access-xsgvt\") pod \"authorino-58cd955675-p27zg\" (UID: \"63001d04-53b7-4ad9-99a2-05b4c9e80f86\") " pod="kuadrant-system/authorino-58cd955675-p27zg" Jul 03 11:21:39.171942 ip-10-0-136-0 kubenswrapper[2579]: I0703 11:21:39.171775 2579 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-k5hqj\" (UniqueName: \"kubernetes.io/projected/63bd541d-8ed8-47b7-8bed-c584554ffc02-kube-api-access-k5hqj\") pod \"authorino-75d556bd77-b2p87\" (UID: \"63bd541d-8ed8-47b7-8bed-c584554ffc02\") " pod="kuadrant-system/authorino-75d556bd77-b2p87" Jul 03 11:21:39.272280 ip-10-0-136-0 kubenswrapper[2579]: I0703 11:21:39.272235 2579 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-xsgvt\" (UniqueName: \"kubernetes.io/projected/63001d04-53b7-4ad9-99a2-05b4c9e80f86-kube-api-access-xsgvt\") pod \"authorino-58cd955675-p27zg\" (UID: \"63001d04-53b7-4ad9-99a2-05b4c9e80f86\") " pod="kuadrant-system/authorino-58cd955675-p27zg" Jul 03 11:21:39.272513 ip-10-0-136-0 kubenswrapper[2579]: I0703 11:21:39.272308 2579 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-k5hqj\" (UniqueName: \"kubernetes.io/projected/63bd541d-8ed8-47b7-8bed-c584554ffc02-kube-api-access-k5hqj\") pod \"authorino-75d556bd77-b2p87\" (UID: \"63bd541d-8ed8-47b7-8bed-c584554ffc02\") " pod="kuadrant-system/authorino-75d556bd77-b2p87" Jul 03 11:21:39.282489 ip-10-0-136-0 kubenswrapper[2579]: I0703 11:21:39.282453 2579 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-k5hqj\" (UniqueName: \"kubernetes.io/projected/63bd541d-8ed8-47b7-8bed-c584554ffc02-kube-api-access-k5hqj\") pod \"authorino-75d556bd77-b2p87\" (UID: \"63bd541d-8ed8-47b7-8bed-c584554ffc02\") " pod="kuadrant-system/authorino-75d556bd77-b2p87" Jul 03 11:21:39.282489 ip-10-0-136-0 kubenswrapper[2579]: I0703 11:21:39.282479 2579 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-xsgvt\" (UniqueName: \"kubernetes.io/projected/63001d04-53b7-4ad9-99a2-05b4c9e80f86-kube-api-access-xsgvt\") pod \"authorino-58cd955675-p27zg\" (UID: \"63001d04-53b7-4ad9-99a2-05b4c9e80f86\") " pod="kuadrant-system/authorino-58cd955675-p27zg" Jul 03 11:21:39.419062 ip-10-0-136-0 kubenswrapper[2579]: I0703 11:21:39.419020 2579 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/authorino-58cd955675-p27zg" Jul 03 11:21:39.461618 ip-10-0-136-0 kubenswrapper[2579]: I0703 11:21:39.461581 2579 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/authorino-75d556bd77-b2p87" Jul 03 11:21:39.606657 ip-10-0-136-0 kubenswrapper[2579]: I0703 11:21:39.606608 2579 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/authorino-58cd955675-p27zg"] Jul 03 11:21:39.607410 ip-10-0-136-0 kubenswrapper[2579]: W0703 11:21:39.607376 2579 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-besteffort.slice/kubepods-besteffort-pod63001d04_53b7_4ad9_99a2_05b4c9e80f86.slice/crio-55e020fd4ca8872b428d470bf4602d99eb855858cc9a4ba701532495c9cdd1ac WatchSource:0}: Error finding container 55e020fd4ca8872b428d470bf4602d99eb855858cc9a4ba701532495c9cdd1ac: Status 404 returned error can't find the container with id 55e020fd4ca8872b428d470bf4602d99eb855858cc9a4ba701532495c9cdd1ac Jul 03 11:21:39.651338 ip-10-0-136-0 kubenswrapper[2579]: I0703 11:21:39.651303 2579 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/authorino-75d556bd77-b2p87"] Jul 03 11:21:39.654150 ip-10-0-136-0 kubenswrapper[2579]: W0703 11:21:39.654112 2579 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-besteffort.slice/kubepods-besteffort-pod63bd541d_8ed8_47b7_8bed_c584554ffc02.slice/crio-81250143d239f90cd6705a67611191c5749d490b6bd4249c2084f93d9195421c WatchSource:0}: Error finding container 81250143d239f90cd6705a67611191c5749d490b6bd4249c2084f93d9195421c: Status 404 returned error can't find the container with id 81250143d239f90cd6705a67611191c5749d490b6bd4249c2084f93d9195421c Jul 03 11:21:39.979129 ip-10-0-136-0 kubenswrapper[2579]: I0703 11:21:39.979029 2579 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/authorino-75d556bd77-b2p87" event={"ID":"63bd541d-8ed8-47b7-8bed-c584554ffc02","Type":"ContainerStarted","Data":"81250143d239f90cd6705a67611191c5749d490b6bd4249c2084f93d9195421c"} Jul 03 11:21:39.980271 ip-10-0-136-0 kubenswrapper[2579]: I0703 11:21:39.980238 2579 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/authorino-58cd955675-p27zg" event={"ID":"63001d04-53b7-4ad9-99a2-05b4c9e80f86","Type":"ContainerStarted","Data":"55e020fd4ca8872b428d470bf4602d99eb855858cc9a4ba701532495c9cdd1ac"} Jul 03 11:21:44.018175 ip-10-0-136-0 kubenswrapper[2579]: I0703 11:21:44.018111 2579 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/authorino-75d556bd77-b2p87" event={"ID":"63bd541d-8ed8-47b7-8bed-c584554ffc02","Type":"ContainerStarted","Data":"3b1f049e8383abfd1f16ae140ed93c70495ada25320e7f2fb3e7f12cf7012bd9"} Jul 03 11:21:44.020144 ip-10-0-136-0 kubenswrapper[2579]: I0703 11:21:44.020099 2579 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/authorino-58cd955675-p27zg" event={"ID":"63001d04-53b7-4ad9-99a2-05b4c9e80f86","Type":"ContainerStarted","Data":"3dbcbc94aa6accf886f138f417ca198741ab22cf6cae6f06a5430c9f3c98e091"} Jul 03 11:21:44.038703 ip-10-0-136-0 kubenswrapper[2579]: I0703 11:21:44.038621 2579 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="kuadrant-system/authorino-75d556bd77-b2p87" podStartSLOduration=1.436200003 podStartE2EDuration="5.038602634s" totalImagesPullingTime="3.602402631s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-03 11:21:39 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-03 11:21:44.036576047 +0000 UTC m=+560.624880367" watchObservedRunningTime="2026-07-03 11:21:44.038602634 +0000 UTC m=+560.626906932" Jul 03 11:21:44.060051 ip-10-0-136-0 kubenswrapper[2579]: I0703 11:21:44.059988 2579 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="kuadrant-system/authorino-58cd955675-p27zg" podStartSLOduration=1.398315652 podStartE2EDuration="5.059966534s" totalImagesPullingTime="3.661650882s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-03 11:21:39 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-03 11:21:44.05715812 +0000 UTC m=+560.645462434" watchObservedRunningTime="2026-07-03 11:21:44.059966534 +0000 UTC m=+560.648270854" Jul 03 11:21:44.104074 ip-10-0-136-0 kubenswrapper[2579]: I0703 11:21:44.104020 2579 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["kuadrant-system/authorino-58cd955675-p27zg"] Jul 03 11:21:46.031415 ip-10-0-136-0 kubenswrapper[2579]: I0703 11:21:46.031371 2579 kuberuntime_container.go:872] "Killing container with a grace period" pod="kuadrant-system/authorino-58cd955675-p27zg" podUID="63001d04-53b7-4ad9-99a2-05b4c9e80f86" containerName="authorino" containerID="cri-o://3dbcbc94aa6accf886f138f417ca198741ab22cf6cae6f06a5430c9f3c98e091" gracePeriod=30 Jul 03 11:21:46.305280 ip-10-0-136-0 kubenswrapper[2579]: I0703 11:21:46.305239 2579 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/authorino-58cd955675-p27zg" Jul 03 11:21:46.337863 ip-10-0-136-0 kubenswrapper[2579]: I0703 11:21:46.337826 2579 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-xsgvt\" (UniqueName: \"kubernetes.io/projected/63001d04-53b7-4ad9-99a2-05b4c9e80f86-kube-api-access-xsgvt\") pod \"63001d04-53b7-4ad9-99a2-05b4c9e80f86\" (UID: \"63001d04-53b7-4ad9-99a2-05b4c9e80f86\") " Jul 03 11:21:46.340306 ip-10-0-136-0 kubenswrapper[2579]: I0703 11:21:46.340268 2579 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/63001d04-53b7-4ad9-99a2-05b4c9e80f86-kube-api-access-xsgvt" (OuterVolumeSpecName: "kube-api-access-xsgvt") pod "63001d04-53b7-4ad9-99a2-05b4c9e80f86" (UID: "63001d04-53b7-4ad9-99a2-05b4c9e80f86"). InnerVolumeSpecName "kube-api-access-xsgvt". PluginName "kubernetes.io/projected", VolumeGIDValue "" Jul 03 11:21:46.438952 ip-10-0-136-0 kubenswrapper[2579]: I0703 11:21:46.438911 2579 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-xsgvt\" (UniqueName: \"kubernetes.io/projected/63001d04-53b7-4ad9-99a2-05b4c9e80f86-kube-api-access-xsgvt\") on node \"ip-10-0-136-0.ec2.internal\" DevicePath \"\"" Jul 03 11:21:47.039014 ip-10-0-136-0 kubenswrapper[2579]: I0703 11:21:47.038971 2579 generic.go:356] "Generic (PLEG): container finished" podID="63001d04-53b7-4ad9-99a2-05b4c9e80f86" containerID="3dbcbc94aa6accf886f138f417ca198741ab22cf6cae6f06a5430c9f3c98e091" exitCode=0 Jul 03 11:21:47.039014 ip-10-0-136-0 kubenswrapper[2579]: I0703 11:21:47.039020 2579 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/authorino-58cd955675-p27zg" Jul 03 11:21:47.039496 ip-10-0-136-0 kubenswrapper[2579]: I0703 11:21:47.039058 2579 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/authorino-58cd955675-p27zg" event={"ID":"63001d04-53b7-4ad9-99a2-05b4c9e80f86","Type":"ContainerDied","Data":"3dbcbc94aa6accf886f138f417ca198741ab22cf6cae6f06a5430c9f3c98e091"} Jul 03 11:21:47.039496 ip-10-0-136-0 kubenswrapper[2579]: I0703 11:21:47.039111 2579 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/authorino-58cd955675-p27zg" event={"ID":"63001d04-53b7-4ad9-99a2-05b4c9e80f86","Type":"ContainerDied","Data":"55e020fd4ca8872b428d470bf4602d99eb855858cc9a4ba701532495c9cdd1ac"} Jul 03 11:21:47.039496 ip-10-0-136-0 kubenswrapper[2579]: I0703 11:21:47.039132 2579 scope.go:117] "RemoveContainer" containerID="3dbcbc94aa6accf886f138f417ca198741ab22cf6cae6f06a5430c9f3c98e091" Jul 03 11:21:47.049520 ip-10-0-136-0 kubenswrapper[2579]: I0703 11:21:47.049503 2579 scope.go:117] "RemoveContainer" containerID="3dbcbc94aa6accf886f138f417ca198741ab22cf6cae6f06a5430c9f3c98e091" Jul 03 11:21:47.049840 ip-10-0-136-0 kubenswrapper[2579]: E0703 11:21:47.049819 2579 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"3dbcbc94aa6accf886f138f417ca198741ab22cf6cae6f06a5430c9f3c98e091\": container with ID starting with 3dbcbc94aa6accf886f138f417ca198741ab22cf6cae6f06a5430c9f3c98e091 not found: ID does not exist" containerID="3dbcbc94aa6accf886f138f417ca198741ab22cf6cae6f06a5430c9f3c98e091" Jul 03 11:21:47.049892 ip-10-0-136-0 kubenswrapper[2579]: I0703 11:21:47.049851 2579 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"3dbcbc94aa6accf886f138f417ca198741ab22cf6cae6f06a5430c9f3c98e091"} err="failed to get container status \"3dbcbc94aa6accf886f138f417ca198741ab22cf6cae6f06a5430c9f3c98e091\": rpc error: code = NotFound desc = could not find container \"3dbcbc94aa6accf886f138f417ca198741ab22cf6cae6f06a5430c9f3c98e091\": container with ID starting with 3dbcbc94aa6accf886f138f417ca198741ab22cf6cae6f06a5430c9f3c98e091 not found: ID does not exist" Jul 03 11:21:47.070795 ip-10-0-136-0 kubenswrapper[2579]: I0703 11:21:47.070746 2579 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["kuadrant-system/authorino-58cd955675-p27zg"] Jul 03 11:21:47.082792 ip-10-0-136-0 kubenswrapper[2579]: I0703 11:21:47.082741 2579 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["kuadrant-system/authorino-58cd955675-p27zg"] Jul 03 11:21:47.992604 ip-10-0-136-0 kubenswrapper[2579]: I0703 11:21:47.992569 2579 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="63001d04-53b7-4ad9-99a2-05b4c9e80f86" path="/var/lib/kubelet/pods/63001d04-53b7-4ad9-99a2-05b4c9e80f86/volumes" Jul 03 11:22:11.658715 ip-10-0-136-0 kubenswrapper[2579]: I0703 11:22:11.658672 2579 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["kuadrant-system/authorino-5d4b67c446-48hz9"] Jul 03 11:22:11.659159 ip-10-0-136-0 kubenswrapper[2579]: I0703 11:22:11.659111 2579 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="63001d04-53b7-4ad9-99a2-05b4c9e80f86" containerName="authorino" Jul 03 11:22:11.659159 ip-10-0-136-0 kubenswrapper[2579]: I0703 11:22:11.659137 2579 state_mem.go:107] "Deleted CPUSet assignment" podUID="63001d04-53b7-4ad9-99a2-05b4c9e80f86" containerName="authorino" Jul 03 11:22:11.659301 ip-10-0-136-0 kubenswrapper[2579]: I0703 11:22:11.659276 2579 memory_manager.go:368] "RemoveStaleState removing state" podUID="63001d04-53b7-4ad9-99a2-05b4c9e80f86" containerName="authorino" Jul 03 11:22:11.670057 ip-10-0-136-0 kubenswrapper[2579]: I0703 11:22:11.670014 2579 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/authorino-5d4b67c446-48hz9" Jul 03 11:22:11.672114 ip-10-0-136-0 kubenswrapper[2579]: I0703 11:22:11.672079 2579 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/authorino-5d4b67c446-48hz9"] Jul 03 11:22:11.781325 ip-10-0-136-0 kubenswrapper[2579]: I0703 11:22:11.781276 2579 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-v4pk6\" (UniqueName: \"kubernetes.io/projected/18296ccd-989e-4cb4-b419-5c7ea449a616-kube-api-access-v4pk6\") pod \"authorino-5d4b67c446-48hz9\" (UID: \"18296ccd-989e-4cb4-b419-5c7ea449a616\") " pod="kuadrant-system/authorino-5d4b67c446-48hz9" Jul 03 11:22:11.882835 ip-10-0-136-0 kubenswrapper[2579]: I0703 11:22:11.882794 2579 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-v4pk6\" (UniqueName: \"kubernetes.io/projected/18296ccd-989e-4cb4-b419-5c7ea449a616-kube-api-access-v4pk6\") pod \"authorino-5d4b67c446-48hz9\" (UID: \"18296ccd-989e-4cb4-b419-5c7ea449a616\") " pod="kuadrant-system/authorino-5d4b67c446-48hz9" Jul 03 11:22:11.893057 ip-10-0-136-0 kubenswrapper[2579]: I0703 11:22:11.893013 2579 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-v4pk6\" (UniqueName: \"kubernetes.io/projected/18296ccd-989e-4cb4-b419-5c7ea449a616-kube-api-access-v4pk6\") pod \"authorino-5d4b67c446-48hz9\" (UID: \"18296ccd-989e-4cb4-b419-5c7ea449a616\") " pod="kuadrant-system/authorino-5d4b67c446-48hz9" Jul 03 11:22:11.903862 ip-10-0-136-0 kubenswrapper[2579]: I0703 11:22:11.903823 2579 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["kuadrant-system/authorino-5d4b67c446-48hz9"] Jul 03 11:22:11.904133 ip-10-0-136-0 kubenswrapper[2579]: I0703 11:22:11.904119 2579 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/authorino-5d4b67c446-48hz9" Jul 03 11:22:11.935594 ip-10-0-136-0 kubenswrapper[2579]: I0703 11:22:11.935558 2579 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["kuadrant-system/authorino-7d96bcdd4c-647s7"] Jul 03 11:22:11.938631 ip-10-0-136-0 kubenswrapper[2579]: I0703 11:22:11.938601 2579 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/authorino-7d96bcdd4c-647s7" Jul 03 11:22:11.948302 ip-10-0-136-0 kubenswrapper[2579]: I0703 11:22:11.948269 2579 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/authorino-7d96bcdd4c-647s7"] Jul 03 11:22:12.070162 ip-10-0-136-0 kubenswrapper[2579]: I0703 11:22:12.070134 2579 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["kuadrant-system/authorino-5d4b67c446-48hz9"] Jul 03 11:22:12.071954 ip-10-0-136-0 kubenswrapper[2579]: W0703 11:22:12.071910 2579 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-besteffort.slice/kubepods-besteffort-pod18296ccd_989e_4cb4_b419_5c7ea449a616.slice/crio-4a606c9b87f3bd2800818734c3631bd06f4e392e13e1c35f5f9dea174f6bde45 WatchSource:0}: Error finding container 4a606c9b87f3bd2800818734c3631bd06f4e392e13e1c35f5f9dea174f6bde45: Status 404 returned error can't find the container with id 4a606c9b87f3bd2800818734c3631bd06f4e392e13e1c35f5f9dea174f6bde45 Jul 03 11:22:12.084913 ip-10-0-136-0 kubenswrapper[2579]: I0703 11:22:12.084875 2579 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-nrrfr\" (UniqueName: \"kubernetes.io/projected/5581d124-b076-4dce-a5bb-a5c7558da1a4-kube-api-access-nrrfr\") pod \"authorino-7d96bcdd4c-647s7\" (UID: \"5581d124-b076-4dce-a5bb-a5c7558da1a4\") " pod="kuadrant-system/authorino-7d96bcdd4c-647s7" Jul 03 11:22:12.176161 ip-10-0-136-0 kubenswrapper[2579]: I0703 11:22:12.176056 2579 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["kuadrant-system/authorino-7d96bcdd4c-647s7"] Jul 03 11:22:12.176432 ip-10-0-136-0 kubenswrapper[2579]: E0703 11:22:12.176404 2579 pod_workers.go:1324] "Error syncing pod, skipping" err="unmounted volumes=[kube-api-access-nrrfr], unattached volumes=[], failed to process volumes=[]: context canceled" pod="kuadrant-system/authorino-7d96bcdd4c-647s7" podUID="5581d124-b076-4dce-a5bb-a5c7558da1a4" Jul 03 11:22:12.185343 ip-10-0-136-0 kubenswrapper[2579]: I0703 11:22:12.185294 2579 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/authorino-5d4b67c446-48hz9" event={"ID":"18296ccd-989e-4cb4-b419-5c7ea449a616","Type":"ContainerStarted","Data":"4a606c9b87f3bd2800818734c3631bd06f4e392e13e1c35f5f9dea174f6bde45"} Jul 03 11:22:12.185533 ip-10-0-136-0 kubenswrapper[2579]: I0703 11:22:12.185307 2579 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/authorino-7d96bcdd4c-647s7" Jul 03 11:22:12.185533 ip-10-0-136-0 kubenswrapper[2579]: I0703 11:22:12.185472 2579 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-nrrfr\" (UniqueName: \"kubernetes.io/projected/5581d124-b076-4dce-a5bb-a5c7558da1a4-kube-api-access-nrrfr\") pod \"authorino-7d96bcdd4c-647s7\" (UID: \"5581d124-b076-4dce-a5bb-a5c7558da1a4\") " pod="kuadrant-system/authorino-7d96bcdd4c-647s7" Jul 03 11:22:12.192418 ip-10-0-136-0 kubenswrapper[2579]: I0703 11:22:12.192389 2579 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/authorino-7d96bcdd4c-647s7" Jul 03 11:22:12.207547 ip-10-0-136-0 kubenswrapper[2579]: I0703 11:22:12.207510 2579 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-nrrfr\" (UniqueName: \"kubernetes.io/projected/5581d124-b076-4dce-a5bb-a5c7558da1a4-kube-api-access-nrrfr\") pod \"authorino-7d96bcdd4c-647s7\" (UID: \"5581d124-b076-4dce-a5bb-a5c7558da1a4\") " pod="kuadrant-system/authorino-7d96bcdd4c-647s7" Jul 03 11:22:12.218558 ip-10-0-136-0 kubenswrapper[2579]: I0703 11:22:12.218522 2579 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["kuadrant-system/authorino-5df584787-qjzxd"] Jul 03 11:22:12.221506 ip-10-0-136-0 kubenswrapper[2579]: I0703 11:22:12.221475 2579 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/authorino-5df584787-qjzxd" Jul 03 11:22:12.224936 ip-10-0-136-0 kubenswrapper[2579]: I0703 11:22:12.224909 2579 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"kuadrant-system\"/\"authorino-server-cert\"" Jul 03 11:22:12.229377 ip-10-0-136-0 kubenswrapper[2579]: I0703 11:22:12.229343 2579 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/authorino-5df584787-qjzxd"] Jul 03 11:22:12.286919 ip-10-0-136-0 kubenswrapper[2579]: I0703 11:22:12.286883 2579 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-nrrfr\" (UniqueName: \"kubernetes.io/projected/5581d124-b076-4dce-a5bb-a5c7558da1a4-kube-api-access-nrrfr\") pod \"5581d124-b076-4dce-a5bb-a5c7558da1a4\" (UID: \"5581d124-b076-4dce-a5bb-a5c7558da1a4\") " Jul 03 11:22:12.289352 ip-10-0-136-0 kubenswrapper[2579]: I0703 11:22:12.289322 2579 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/5581d124-b076-4dce-a5bb-a5c7558da1a4-kube-api-access-nrrfr" (OuterVolumeSpecName: "kube-api-access-nrrfr") pod "5581d124-b076-4dce-a5bb-a5c7558da1a4" (UID: "5581d124-b076-4dce-a5bb-a5c7558da1a4"). InnerVolumeSpecName "kube-api-access-nrrfr". PluginName "kubernetes.io/projected", VolumeGIDValue "" Jul 03 11:22:12.388377 ip-10-0-136-0 kubenswrapper[2579]: I0703 11:22:12.388337 2579 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-2ghmd\" (UniqueName: \"kubernetes.io/projected/bd887377-3185-4dda-9a7e-b8c0d968b6ae-kube-api-access-2ghmd\") pod \"authorino-5df584787-qjzxd\" (UID: \"bd887377-3185-4dda-9a7e-b8c0d968b6ae\") " pod="kuadrant-system/authorino-5df584787-qjzxd" Jul 03 11:22:12.388377 ip-10-0-136-0 kubenswrapper[2579]: I0703 11:22:12.388381 2579 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tls-cert\" (UniqueName: \"kubernetes.io/secret/bd887377-3185-4dda-9a7e-b8c0d968b6ae-tls-cert\") pod \"authorino-5df584787-qjzxd\" (UID: \"bd887377-3185-4dda-9a7e-b8c0d968b6ae\") " pod="kuadrant-system/authorino-5df584787-qjzxd" Jul 03 11:22:12.388602 ip-10-0-136-0 kubenswrapper[2579]: I0703 11:22:12.388420 2579 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-nrrfr\" (UniqueName: \"kubernetes.io/projected/5581d124-b076-4dce-a5bb-a5c7558da1a4-kube-api-access-nrrfr\") on node \"ip-10-0-136-0.ec2.internal\" DevicePath \"\"" Jul 03 11:22:12.489385 ip-10-0-136-0 kubenswrapper[2579]: I0703 11:22:12.489280 2579 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-2ghmd\" (UniqueName: \"kubernetes.io/projected/bd887377-3185-4dda-9a7e-b8c0d968b6ae-kube-api-access-2ghmd\") pod \"authorino-5df584787-qjzxd\" (UID: \"bd887377-3185-4dda-9a7e-b8c0d968b6ae\") " pod="kuadrant-system/authorino-5df584787-qjzxd" Jul 03 11:22:12.489385 ip-10-0-136-0 kubenswrapper[2579]: I0703 11:22:12.489332 2579 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tls-cert\" (UniqueName: \"kubernetes.io/secret/bd887377-3185-4dda-9a7e-b8c0d968b6ae-tls-cert\") pod \"authorino-5df584787-qjzxd\" (UID: \"bd887377-3185-4dda-9a7e-b8c0d968b6ae\") " pod="kuadrant-system/authorino-5df584787-qjzxd" Jul 03 11:22:12.491986 ip-10-0-136-0 kubenswrapper[2579]: I0703 11:22:12.491964 2579 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"tls-cert\" (UniqueName: \"kubernetes.io/secret/bd887377-3185-4dda-9a7e-b8c0d968b6ae-tls-cert\") pod \"authorino-5df584787-qjzxd\" (UID: \"bd887377-3185-4dda-9a7e-b8c0d968b6ae\") " pod="kuadrant-system/authorino-5df584787-qjzxd" Jul 03 11:22:12.498090 ip-10-0-136-0 kubenswrapper[2579]: I0703 11:22:12.498061 2579 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-2ghmd\" (UniqueName: \"kubernetes.io/projected/bd887377-3185-4dda-9a7e-b8c0d968b6ae-kube-api-access-2ghmd\") pod \"authorino-5df584787-qjzxd\" (UID: \"bd887377-3185-4dda-9a7e-b8c0d968b6ae\") " pod="kuadrant-system/authorino-5df584787-qjzxd" Jul 03 11:22:12.534086 ip-10-0-136-0 kubenswrapper[2579]: I0703 11:22:12.534045 2579 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/authorino-5df584787-qjzxd" Jul 03 11:22:12.687752 ip-10-0-136-0 kubenswrapper[2579]: I0703 11:22:12.687711 2579 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/authorino-5df584787-qjzxd"] Jul 03 11:22:12.688595 ip-10-0-136-0 kubenswrapper[2579]: W0703 11:22:12.688551 2579 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-besteffort.slice/kubepods-besteffort-podbd887377_3185_4dda_9a7e_b8c0d968b6ae.slice/crio-d39438c0a96f7d72f6dc4469feeed4c51c5da967b67c28bf088e9b6910962ef6 WatchSource:0}: Error finding container d39438c0a96f7d72f6dc4469feeed4c51c5da967b67c28bf088e9b6910962ef6: Status 404 returned error can't find the container with id d39438c0a96f7d72f6dc4469feeed4c51c5da967b67c28bf088e9b6910962ef6 Jul 03 11:22:13.192501 ip-10-0-136-0 kubenswrapper[2579]: I0703 11:22:13.192461 2579 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/authorino-5d4b67c446-48hz9" event={"ID":"18296ccd-989e-4cb4-b419-5c7ea449a616","Type":"ContainerStarted","Data":"24009d0a062164304ae88429d74872da5eb1a22f0b9795ae9da29e0544830a2c"} Jul 03 11:22:13.192710 ip-10-0-136-0 kubenswrapper[2579]: I0703 11:22:13.192556 2579 kuberuntime_container.go:872] "Killing container with a grace period" pod="kuadrant-system/authorino-5d4b67c446-48hz9" podUID="18296ccd-989e-4cb4-b419-5c7ea449a616" containerName="authorino" containerID="cri-o://24009d0a062164304ae88429d74872da5eb1a22f0b9795ae9da29e0544830a2c" gracePeriod=30 Jul 03 11:22:13.194887 ip-10-0-136-0 kubenswrapper[2579]: I0703 11:22:13.194856 2579 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/authorino-7d96bcdd4c-647s7" Jul 03 11:22:13.194887 ip-10-0-136-0 kubenswrapper[2579]: I0703 11:22:13.194874 2579 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/authorino-5df584787-qjzxd" event={"ID":"bd887377-3185-4dda-9a7e-b8c0d968b6ae","Type":"ContainerStarted","Data":"0f53618cdfd2ba0856906c4229dd63b3ffeef1b53e618260f401aacb2f53dd00"} Jul 03 11:22:13.195139 ip-10-0-136-0 kubenswrapper[2579]: I0703 11:22:13.194918 2579 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/authorino-5df584787-qjzxd" event={"ID":"bd887377-3185-4dda-9a7e-b8c0d968b6ae","Type":"ContainerStarted","Data":"d39438c0a96f7d72f6dc4469feeed4c51c5da967b67c28bf088e9b6910962ef6"} Jul 03 11:22:13.211255 ip-10-0-136-0 kubenswrapper[2579]: I0703 11:22:13.211199 2579 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="kuadrant-system/authorino-5d4b67c446-48hz9" podStartSLOduration=1.520198718 podStartE2EDuration="2.211161103s" totalImagesPullingTime="690.962385ms" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-03 11:22:11 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-03 11:22:13.208775256 +0000 UTC m=+589.797079588" watchObservedRunningTime="2026-07-03 11:22:13.211161103 +0000 UTC m=+589.799465424" Jul 03 11:22:13.236058 ip-10-0-136-0 kubenswrapper[2579]: I0703 11:22:13.236001 2579 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="kuadrant-system/authorino-5df584787-qjzxd" podStartSLOduration=0.85113241 podStartE2EDuration="1.235983814s" totalImagesPullingTime="384.851404ms" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-03 11:22:12 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-03 11:22:13.233477618 +0000 UTC m=+589.821781938" watchObservedRunningTime="2026-07-03 11:22:13.235983814 +0000 UTC m=+589.824288133" Jul 03 11:22:13.266097 ip-10-0-136-0 kubenswrapper[2579]: I0703 11:22:13.266059 2579 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["kuadrant-system/authorino-75d556bd77-b2p87"] Jul 03 11:22:13.266299 ip-10-0-136-0 kubenswrapper[2579]: I0703 11:22:13.266263 2579 kuberuntime_container.go:872] "Killing container with a grace period" pod="kuadrant-system/authorino-75d556bd77-b2p87" podUID="63bd541d-8ed8-47b7-8bed-c584554ffc02" containerName="authorino" containerID="cri-o://3b1f049e8383abfd1f16ae140ed93c70495ada25320e7f2fb3e7f12cf7012bd9" gracePeriod=30 Jul 03 11:22:13.288998 ip-10-0-136-0 kubenswrapper[2579]: I0703 11:22:13.288955 2579 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["kuadrant-system/authorino-7d96bcdd4c-647s7"] Jul 03 11:22:13.292368 ip-10-0-136-0 kubenswrapper[2579]: I0703 11:22:13.292330 2579 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["kuadrant-system/authorino-7d96bcdd4c-647s7"] Jul 03 11:22:13.532226 ip-10-0-136-0 kubenswrapper[2579]: I0703 11:22:13.532181 2579 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/authorino-5d4b67c446-48hz9" Jul 03 11:22:13.547887 ip-10-0-136-0 kubenswrapper[2579]: I0703 11:22:13.547856 2579 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/authorino-75d556bd77-b2p87" Jul 03 11:22:13.701314 ip-10-0-136-0 kubenswrapper[2579]: I0703 11:22:13.701172 2579 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-k5hqj\" (UniqueName: \"kubernetes.io/projected/63bd541d-8ed8-47b7-8bed-c584554ffc02-kube-api-access-k5hqj\") pod \"63bd541d-8ed8-47b7-8bed-c584554ffc02\" (UID: \"63bd541d-8ed8-47b7-8bed-c584554ffc02\") " Jul 03 11:22:13.701761 ip-10-0-136-0 kubenswrapper[2579]: I0703 11:22:13.701356 2579 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-v4pk6\" (UniqueName: \"kubernetes.io/projected/18296ccd-989e-4cb4-b419-5c7ea449a616-kube-api-access-v4pk6\") pod \"18296ccd-989e-4cb4-b419-5c7ea449a616\" (UID: \"18296ccd-989e-4cb4-b419-5c7ea449a616\") " Jul 03 11:22:13.703804 ip-10-0-136-0 kubenswrapper[2579]: I0703 11:22:13.703768 2579 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/18296ccd-989e-4cb4-b419-5c7ea449a616-kube-api-access-v4pk6" (OuterVolumeSpecName: "kube-api-access-v4pk6") pod "18296ccd-989e-4cb4-b419-5c7ea449a616" (UID: "18296ccd-989e-4cb4-b419-5c7ea449a616"). InnerVolumeSpecName "kube-api-access-v4pk6". PluginName "kubernetes.io/projected", VolumeGIDValue "" Jul 03 11:22:13.703941 ip-10-0-136-0 kubenswrapper[2579]: I0703 11:22:13.703775 2579 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/63bd541d-8ed8-47b7-8bed-c584554ffc02-kube-api-access-k5hqj" (OuterVolumeSpecName: "kube-api-access-k5hqj") pod "63bd541d-8ed8-47b7-8bed-c584554ffc02" (UID: "63bd541d-8ed8-47b7-8bed-c584554ffc02"). InnerVolumeSpecName "kube-api-access-k5hqj". PluginName "kubernetes.io/projected", VolumeGIDValue "" Jul 03 11:22:13.802889 ip-10-0-136-0 kubenswrapper[2579]: I0703 11:22:13.802841 2579 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-v4pk6\" (UniqueName: \"kubernetes.io/projected/18296ccd-989e-4cb4-b419-5c7ea449a616-kube-api-access-v4pk6\") on node \"ip-10-0-136-0.ec2.internal\" DevicePath \"\"" Jul 03 11:22:13.802889 ip-10-0-136-0 kubenswrapper[2579]: I0703 11:22:13.802878 2579 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-k5hqj\" (UniqueName: \"kubernetes.io/projected/63bd541d-8ed8-47b7-8bed-c584554ffc02-kube-api-access-k5hqj\") on node \"ip-10-0-136-0.ec2.internal\" DevicePath \"\"" Jul 03 11:22:13.992872 ip-10-0-136-0 kubenswrapper[2579]: I0703 11:22:13.992786 2579 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="5581d124-b076-4dce-a5bb-a5c7558da1a4" path="/var/lib/kubelet/pods/5581d124-b076-4dce-a5bb-a5c7558da1a4/volumes" Jul 03 11:22:14.203338 ip-10-0-136-0 kubenswrapper[2579]: I0703 11:22:14.203298 2579 generic.go:356] "Generic (PLEG): container finished" podID="18296ccd-989e-4cb4-b419-5c7ea449a616" containerID="24009d0a062164304ae88429d74872da5eb1a22f0b9795ae9da29e0544830a2c" exitCode=0 Jul 03 11:22:14.203556 ip-10-0-136-0 kubenswrapper[2579]: I0703 11:22:14.203367 2579 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/authorino-5d4b67c446-48hz9" Jul 03 11:22:14.203556 ip-10-0-136-0 kubenswrapper[2579]: I0703 11:22:14.203391 2579 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/authorino-5d4b67c446-48hz9" event={"ID":"18296ccd-989e-4cb4-b419-5c7ea449a616","Type":"ContainerDied","Data":"24009d0a062164304ae88429d74872da5eb1a22f0b9795ae9da29e0544830a2c"} Jul 03 11:22:14.203556 ip-10-0-136-0 kubenswrapper[2579]: I0703 11:22:14.203446 2579 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/authorino-5d4b67c446-48hz9" event={"ID":"18296ccd-989e-4cb4-b419-5c7ea449a616","Type":"ContainerDied","Data":"4a606c9b87f3bd2800818734c3631bd06f4e392e13e1c35f5f9dea174f6bde45"} Jul 03 11:22:14.203556 ip-10-0-136-0 kubenswrapper[2579]: I0703 11:22:14.203467 2579 scope.go:117] "RemoveContainer" containerID="24009d0a062164304ae88429d74872da5eb1a22f0b9795ae9da29e0544830a2c" Jul 03 11:22:14.205179 ip-10-0-136-0 kubenswrapper[2579]: I0703 11:22:14.205151 2579 generic.go:356] "Generic (PLEG): container finished" podID="63bd541d-8ed8-47b7-8bed-c584554ffc02" containerID="3b1f049e8383abfd1f16ae140ed93c70495ada25320e7f2fb3e7f12cf7012bd9" exitCode=0 Jul 03 11:22:14.205332 ip-10-0-136-0 kubenswrapper[2579]: I0703 11:22:14.205245 2579 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/authorino-75d556bd77-b2p87" event={"ID":"63bd541d-8ed8-47b7-8bed-c584554ffc02","Type":"ContainerDied","Data":"3b1f049e8383abfd1f16ae140ed93c70495ada25320e7f2fb3e7f12cf7012bd9"} Jul 03 11:22:14.205332 ip-10-0-136-0 kubenswrapper[2579]: I0703 11:22:14.205290 2579 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/authorino-75d556bd77-b2p87" Jul 03 11:22:14.205332 ip-10-0-136-0 kubenswrapper[2579]: I0703 11:22:14.205293 2579 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/authorino-75d556bd77-b2p87" event={"ID":"63bd541d-8ed8-47b7-8bed-c584554ffc02","Type":"ContainerDied","Data":"81250143d239f90cd6705a67611191c5749d490b6bd4249c2084f93d9195421c"} Jul 03 11:22:14.213387 ip-10-0-136-0 kubenswrapper[2579]: I0703 11:22:14.213365 2579 scope.go:117] "RemoveContainer" containerID="24009d0a062164304ae88429d74872da5eb1a22f0b9795ae9da29e0544830a2c" Jul 03 11:22:14.213730 ip-10-0-136-0 kubenswrapper[2579]: E0703 11:22:14.213710 2579 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"24009d0a062164304ae88429d74872da5eb1a22f0b9795ae9da29e0544830a2c\": container with ID starting with 24009d0a062164304ae88429d74872da5eb1a22f0b9795ae9da29e0544830a2c not found: ID does not exist" containerID="24009d0a062164304ae88429d74872da5eb1a22f0b9795ae9da29e0544830a2c" Jul 03 11:22:14.213787 ip-10-0-136-0 kubenswrapper[2579]: I0703 11:22:14.213744 2579 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"24009d0a062164304ae88429d74872da5eb1a22f0b9795ae9da29e0544830a2c"} err="failed to get container status \"24009d0a062164304ae88429d74872da5eb1a22f0b9795ae9da29e0544830a2c\": rpc error: code = NotFound desc = could not find container \"24009d0a062164304ae88429d74872da5eb1a22f0b9795ae9da29e0544830a2c\": container with ID starting with 24009d0a062164304ae88429d74872da5eb1a22f0b9795ae9da29e0544830a2c not found: ID does not exist" Jul 03 11:22:14.213787 ip-10-0-136-0 kubenswrapper[2579]: I0703 11:22:14.213765 2579 scope.go:117] "RemoveContainer" containerID="3b1f049e8383abfd1f16ae140ed93c70495ada25320e7f2fb3e7f12cf7012bd9" Jul 03 11:22:14.224550 ip-10-0-136-0 kubenswrapper[2579]: I0703 11:22:14.224500 2579 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["kuadrant-system/authorino-75d556bd77-b2p87"] Jul 03 11:22:14.225747 ip-10-0-136-0 kubenswrapper[2579]: I0703 11:22:14.225723 2579 scope.go:117] "RemoveContainer" containerID="3b1f049e8383abfd1f16ae140ed93c70495ada25320e7f2fb3e7f12cf7012bd9" Jul 03 11:22:14.226182 ip-10-0-136-0 kubenswrapper[2579]: E0703 11:22:14.226144 2579 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"3b1f049e8383abfd1f16ae140ed93c70495ada25320e7f2fb3e7f12cf7012bd9\": container with ID starting with 3b1f049e8383abfd1f16ae140ed93c70495ada25320e7f2fb3e7f12cf7012bd9 not found: ID does not exist" containerID="3b1f049e8383abfd1f16ae140ed93c70495ada25320e7f2fb3e7f12cf7012bd9" Jul 03 11:22:14.226182 ip-10-0-136-0 kubenswrapper[2579]: I0703 11:22:14.226202 2579 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"3b1f049e8383abfd1f16ae140ed93c70495ada25320e7f2fb3e7f12cf7012bd9"} err="failed to get container status \"3b1f049e8383abfd1f16ae140ed93c70495ada25320e7f2fb3e7f12cf7012bd9\": rpc error: code = NotFound desc = could not find container \"3b1f049e8383abfd1f16ae140ed93c70495ada25320e7f2fb3e7f12cf7012bd9\": container with ID starting with 3b1f049e8383abfd1f16ae140ed93c70495ada25320e7f2fb3e7f12cf7012bd9 not found: ID does not exist" Jul 03 11:22:14.227735 ip-10-0-136-0 kubenswrapper[2579]: I0703 11:22:14.227714 2579 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["kuadrant-system/authorino-75d556bd77-b2p87"] Jul 03 11:22:14.241092 ip-10-0-136-0 kubenswrapper[2579]: I0703 11:22:14.241060 2579 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["kuadrant-system/authorino-5d4b67c446-48hz9"] Jul 03 11:22:14.243530 ip-10-0-136-0 kubenswrapper[2579]: I0703 11:22:14.243468 2579 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["kuadrant-system/authorino-5d4b67c446-48hz9"] Jul 03 11:22:15.992011 ip-10-0-136-0 kubenswrapper[2579]: I0703 11:22:15.991972 2579 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="18296ccd-989e-4cb4-b419-5c7ea449a616" path="/var/lib/kubelet/pods/18296ccd-989e-4cb4-b419-5c7ea449a616/volumes" Jul 03 11:22:15.992516 ip-10-0-136-0 kubenswrapper[2579]: I0703 11:22:15.992354 2579 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="63bd541d-8ed8-47b7-8bed-c584554ffc02" path="/var/lib/kubelet/pods/63bd541d-8ed8-47b7-8bed-c584554ffc02/volumes" Jul 03 11:22:17.765152 ip-10-0-136-0 kubenswrapper[2579]: I0703 11:22:17.765108 2579 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["opendatahub/maas-controller-7b6bc6f48f-dldkg"] Jul 03 11:22:17.765571 ip-10-0-136-0 kubenswrapper[2579]: I0703 11:22:17.765559 2579 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="18296ccd-989e-4cb4-b419-5c7ea449a616" containerName="authorino" Jul 03 11:22:17.765571 ip-10-0-136-0 kubenswrapper[2579]: I0703 11:22:17.765572 2579 state_mem.go:107] "Deleted CPUSet assignment" podUID="18296ccd-989e-4cb4-b419-5c7ea449a616" containerName="authorino" Jul 03 11:22:17.765653 ip-10-0-136-0 kubenswrapper[2579]: I0703 11:22:17.765585 2579 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="63bd541d-8ed8-47b7-8bed-c584554ffc02" containerName="authorino" Jul 03 11:22:17.765653 ip-10-0-136-0 kubenswrapper[2579]: I0703 11:22:17.765591 2579 state_mem.go:107] "Deleted CPUSet assignment" podUID="63bd541d-8ed8-47b7-8bed-c584554ffc02" containerName="authorino" Jul 03 11:22:17.765653 ip-10-0-136-0 kubenswrapper[2579]: I0703 11:22:17.765645 2579 memory_manager.go:368] "RemoveStaleState removing state" podUID="18296ccd-989e-4cb4-b419-5c7ea449a616" containerName="authorino" Jul 03 11:22:17.765653 ip-10-0-136-0 kubenswrapper[2579]: I0703 11:22:17.765652 2579 memory_manager.go:368] "RemoveStaleState removing state" podUID="63bd541d-8ed8-47b7-8bed-c584554ffc02" containerName="authorino" Jul 03 11:22:17.769419 ip-10-0-136-0 kubenswrapper[2579]: I0703 11:22:17.769386 2579 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="opendatahub/maas-controller-7b6bc6f48f-dldkg" Jul 03 11:22:17.772365 ip-10-0-136-0 kubenswrapper[2579]: I0703 11:22:17.772337 2579 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"opendatahub\"/\"maas-controller-webhook-cert\"" Jul 03 11:22:17.772365 ip-10-0-136-0 kubenswrapper[2579]: I0703 11:22:17.772353 2579 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"opendatahub\"/\"maas-parameters\"" Jul 03 11:22:17.773324 ip-10-0-136-0 kubenswrapper[2579]: I0703 11:22:17.773302 2579 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"opendatahub\"/\"maas-controller-dockercfg-mvflh\"" Jul 03 11:22:17.785638 ip-10-0-136-0 kubenswrapper[2579]: I0703 11:22:17.785601 2579 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["opendatahub/maas-controller-7b6bc6f48f-dldkg"] Jul 03 11:22:17.947992 ip-10-0-136-0 kubenswrapper[2579]: I0703 11:22:17.947926 2579 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-f2t6f\" (UniqueName: \"kubernetes.io/projected/c5324bdc-22d4-4bc3-a9fe-2f673ad11c58-kube-api-access-f2t6f\") pod \"maas-controller-7b6bc6f48f-dldkg\" (UID: \"c5324bdc-22d4-4bc3-a9fe-2f673ad11c58\") " pod="opendatahub/maas-controller-7b6bc6f48f-dldkg" Jul 03 11:22:17.948271 ip-10-0-136-0 kubenswrapper[2579]: I0703 11:22:17.948109 2579 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"webhook-cert\" (UniqueName: \"kubernetes.io/secret/c5324bdc-22d4-4bc3-a9fe-2f673ad11c58-webhook-cert\") pod \"maas-controller-7b6bc6f48f-dldkg\" (UID: \"c5324bdc-22d4-4bc3-a9fe-2f673ad11c58\") " pod="opendatahub/maas-controller-7b6bc6f48f-dldkg" Jul 03 11:22:18.048890 ip-10-0-136-0 kubenswrapper[2579]: I0703 11:22:18.048848 2579 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"webhook-cert\" (UniqueName: \"kubernetes.io/secret/c5324bdc-22d4-4bc3-a9fe-2f673ad11c58-webhook-cert\") pod \"maas-controller-7b6bc6f48f-dldkg\" (UID: \"c5324bdc-22d4-4bc3-a9fe-2f673ad11c58\") " pod="opendatahub/maas-controller-7b6bc6f48f-dldkg" Jul 03 11:22:18.049093 ip-10-0-136-0 kubenswrapper[2579]: I0703 11:22:18.048905 2579 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-f2t6f\" (UniqueName: \"kubernetes.io/projected/c5324bdc-22d4-4bc3-a9fe-2f673ad11c58-kube-api-access-f2t6f\") pod \"maas-controller-7b6bc6f48f-dldkg\" (UID: \"c5324bdc-22d4-4bc3-a9fe-2f673ad11c58\") " pod="opendatahub/maas-controller-7b6bc6f48f-dldkg" Jul 03 11:22:18.051752 ip-10-0-136-0 kubenswrapper[2579]: I0703 11:22:18.051724 2579 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"webhook-cert\" (UniqueName: \"kubernetes.io/secret/c5324bdc-22d4-4bc3-a9fe-2f673ad11c58-webhook-cert\") pod \"maas-controller-7b6bc6f48f-dldkg\" (UID: \"c5324bdc-22d4-4bc3-a9fe-2f673ad11c58\") " pod="opendatahub/maas-controller-7b6bc6f48f-dldkg" Jul 03 11:22:18.058151 ip-10-0-136-0 kubenswrapper[2579]: I0703 11:22:18.058118 2579 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-f2t6f\" (UniqueName: \"kubernetes.io/projected/c5324bdc-22d4-4bc3-a9fe-2f673ad11c58-kube-api-access-f2t6f\") pod \"maas-controller-7b6bc6f48f-dldkg\" (UID: \"c5324bdc-22d4-4bc3-a9fe-2f673ad11c58\") " pod="opendatahub/maas-controller-7b6bc6f48f-dldkg" Jul 03 11:22:18.081122 ip-10-0-136-0 kubenswrapper[2579]: I0703 11:22:18.081072 2579 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="opendatahub/maas-controller-7b6bc6f48f-dldkg" Jul 03 11:22:18.242410 ip-10-0-136-0 kubenswrapper[2579]: I0703 11:22:18.242376 2579 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["opendatahub/maas-controller-7b6bc6f48f-dldkg"] Jul 03 11:22:18.244466 ip-10-0-136-0 kubenswrapper[2579]: W0703 11:22:18.244424 2579 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podc5324bdc_22d4_4bc3_a9fe_2f673ad11c58.slice/crio-579530daa3b6912816b38764216cd35036eb48f63808bb40e8e2fc61eca81eea WatchSource:0}: Error finding container 579530daa3b6912816b38764216cd35036eb48f63808bb40e8e2fc61eca81eea: Status 404 returned error can't find the container with id 579530daa3b6912816b38764216cd35036eb48f63808bb40e8e2fc61eca81eea Jul 03 11:22:19.240794 ip-10-0-136-0 kubenswrapper[2579]: I0703 11:22:19.240747 2579 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="opendatahub/maas-controller-7b6bc6f48f-dldkg" event={"ID":"c5324bdc-22d4-4bc3-a9fe-2f673ad11c58","Type":"ContainerStarted","Data":"579530daa3b6912816b38764216cd35036eb48f63808bb40e8e2fc61eca81eea"} Jul 03 11:22:21.259648 ip-10-0-136-0 kubenswrapper[2579]: I0703 11:22:21.259604 2579 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="opendatahub/maas-controller-7b6bc6f48f-dldkg" event={"ID":"c5324bdc-22d4-4bc3-a9fe-2f673ad11c58","Type":"ContainerStarted","Data":"1802c17c7e83d4f5a418d573ef32fc9ef0027869085995d1e64cb25f75dea4e4"} Jul 03 11:22:21.260174 ip-10-0-136-0 kubenswrapper[2579]: I0703 11:22:21.259731 2579 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="opendatahub/maas-controller-7b6bc6f48f-dldkg" Jul 03 11:22:21.281763 ip-10-0-136-0 kubenswrapper[2579]: I0703 11:22:21.281698 2579 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="opendatahub/maas-controller-7b6bc6f48f-dldkg" podStartSLOduration=2.040991299 podStartE2EDuration="4.28167948s" totalImagesPullingTime="2.240688181s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-03 11:22:17 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-03 11:22:21.279569334 +0000 UTC m=+597.867873654" watchObservedRunningTime="2026-07-03 11:22:21.28167948 +0000 UTC m=+597.869983846" Jul 03 11:22:32.271713 ip-10-0-136-0 kubenswrapper[2579]: I0703 11:22:32.271678 2579 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="opendatahub/maas-controller-7b6bc6f48f-dldkg" Jul 03 11:22:53.091549 ip-10-0-136-0 kubenswrapper[2579]: I0703 11:22:53.091463 2579 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-ingress/payload-pre-processing-7964457696-ncrll"] Jul 03 11:22:53.097346 ip-10-0-136-0 kubenswrapper[2579]: I0703 11:22:53.097318 2579 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-pre-processing-7964457696-ncrll" Jul 03 11:22:53.100535 ip-10-0-136-0 kubenswrapper[2579]: I0703 11:22:53.100512 2579 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress\"/\"payload-processing-dockercfg-szcjt\"" Jul 03 11:22:53.100535 ip-10-0-136-0 kubenswrapper[2579]: I0703 11:22:53.100526 2579 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ingress\"/\"payload-processing-plugins\"" Jul 03 11:22:53.107928 ip-10-0-136-0 kubenswrapper[2579]: I0703 11:22:53.107597 2579 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/payload-pre-processing-7964457696-ncrll"] Jul 03 11:22:53.179276 ip-10-0-136-0 kubenswrapper[2579]: I0703 11:22:53.179233 2579 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-9c4s8\" (UniqueName: \"kubernetes.io/projected/2c315c89-d57d-4d52-9426-06e88f95bdfb-kube-api-access-9c4s8\") pod \"payload-pre-processing-7964457696-ncrll\" (UID: \"2c315c89-d57d-4d52-9426-06e88f95bdfb\") " pod="openshift-ingress/payload-pre-processing-7964457696-ncrll" Jul 03 11:22:53.179484 ip-10-0-136-0 kubenswrapper[2579]: I0703 11:22:53.179342 2579 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/2c315c89-d57d-4d52-9426-06e88f95bdfb-plugins-config-volume\") pod \"payload-pre-processing-7964457696-ncrll\" (UID: \"2c315c89-d57d-4d52-9426-06e88f95bdfb\") " pod="openshift-ingress/payload-pre-processing-7964457696-ncrll" Jul 03 11:22:53.280852 ip-10-0-136-0 kubenswrapper[2579]: I0703 11:22:53.280810 2579 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/2c315c89-d57d-4d52-9426-06e88f95bdfb-plugins-config-volume\") pod \"payload-pre-processing-7964457696-ncrll\" (UID: \"2c315c89-d57d-4d52-9426-06e88f95bdfb\") " pod="openshift-ingress/payload-pre-processing-7964457696-ncrll" Jul 03 11:22:53.281109 ip-10-0-136-0 kubenswrapper[2579]: I0703 11:22:53.280890 2579 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-9c4s8\" (UniqueName: \"kubernetes.io/projected/2c315c89-d57d-4d52-9426-06e88f95bdfb-kube-api-access-9c4s8\") pod \"payload-pre-processing-7964457696-ncrll\" (UID: \"2c315c89-d57d-4d52-9426-06e88f95bdfb\") " pod="openshift-ingress/payload-pre-processing-7964457696-ncrll" Jul 03 11:22:53.281764 ip-10-0-136-0 kubenswrapper[2579]: I0703 11:22:53.281735 2579 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/2c315c89-d57d-4d52-9426-06e88f95bdfb-plugins-config-volume\") pod \"payload-pre-processing-7964457696-ncrll\" (UID: \"2c315c89-d57d-4d52-9426-06e88f95bdfb\") " pod="openshift-ingress/payload-pre-processing-7964457696-ncrll" Jul 03 11:22:53.289256 ip-10-0-136-0 kubenswrapper[2579]: I0703 11:22:53.289214 2579 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-9c4s8\" (UniqueName: \"kubernetes.io/projected/2c315c89-d57d-4d52-9426-06e88f95bdfb-kube-api-access-9c4s8\") pod \"payload-pre-processing-7964457696-ncrll\" (UID: \"2c315c89-d57d-4d52-9426-06e88f95bdfb\") " pod="openshift-ingress/payload-pre-processing-7964457696-ncrll" Jul 03 11:22:53.410972 ip-10-0-136-0 kubenswrapper[2579]: I0703 11:22:53.410856 2579 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-pre-processing-7964457696-ncrll" Jul 03 11:22:53.575503 ip-10-0-136-0 kubenswrapper[2579]: I0703 11:22:53.575366 2579 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/payload-pre-processing-7964457696-ncrll"] Jul 03 11:22:53.578912 ip-10-0-136-0 kubenswrapper[2579]: W0703 11:22:53.578878 2579 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod2c315c89_d57d_4d52_9426_06e88f95bdfb.slice/crio-d24b1364529c2395a24f514695f245714004502d300891af6889ab41b2d31ffc WatchSource:0}: Error finding container d24b1364529c2395a24f514695f245714004502d300891af6889ab41b2d31ffc: Status 404 returned error can't find the container with id d24b1364529c2395a24f514695f245714004502d300891af6889ab41b2d31ffc Jul 03 11:22:53.581539 ip-10-0-136-0 kubenswrapper[2579]: I0703 11:22:53.581516 2579 provider.go:93] Refreshing cache for provider: *credentialprovider.defaultDockerConfigProvider Jul 03 11:22:54.463168 ip-10-0-136-0 kubenswrapper[2579]: I0703 11:22:54.463110 2579 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-pre-processing-7964457696-ncrll" event={"ID":"2c315c89-d57d-4d52-9426-06e88f95bdfb","Type":"ContainerStarted","Data":"d24b1364529c2395a24f514695f245714004502d300891af6889ab41b2d31ffc"} Jul 03 11:22:55.472720 ip-10-0-136-0 kubenswrapper[2579]: I0703 11:22:55.472591 2579 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-pre-processing-7964457696-ncrll" event={"ID":"2c315c89-d57d-4d52-9426-06e88f95bdfb","Type":"ContainerStarted","Data":"3bd2764059e90faf27433458b1d120eb0326df23620e1374a132cc03fb9c5c7b"} Jul 03 11:22:55.472720 ip-10-0-136-0 kubenswrapper[2579]: I0703 11:22:55.472682 2579 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/payload-pre-processing-7964457696-ncrll" Jul 03 11:22:55.500806 ip-10-0-136-0 kubenswrapper[2579]: I0703 11:22:55.500736 2579 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-ingress/payload-pre-processing-7964457696-ncrll" podStartSLOduration=1.002745263 podStartE2EDuration="2.500713337s" totalImagesPullingTime="1.497968074s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-03 11:22:53 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-03 11:22:55.500673596 +0000 UTC m=+632.088977917" watchObservedRunningTime="2026-07-03 11:22:55.500713337 +0000 UTC m=+632.089017657" Jul 03 11:23:01.509259 ip-10-0-136-0 kubenswrapper[2579]: I0703 11:23:01.509183 2579 generic.go:356] "Generic (PLEG): container finished" podID="b8f8d2d9-d723-4d69-b7a5-938e70db757f" containerID="f96ee7ec696d8c8edf044768ec0355f0aeb207330b737eaf555ec2b71de60a8c" exitCode=0 Jul 03 11:23:01.509690 ip-10-0-136-0 kubenswrapper[2579]: I0703 11:23:01.509257 2579 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/maas-default-gateway-openshift-default-56654dcf7-fbkb8" event={"ID":"b8f8d2d9-d723-4d69-b7a5-938e70db757f","Type":"ContainerDied","Data":"f96ee7ec696d8c8edf044768ec0355f0aeb207330b737eaf555ec2b71de60a8c"} Jul 03 11:23:01.509756 ip-10-0-136-0 kubenswrapper[2579]: I0703 11:23:01.509738 2579 scope.go:117] "RemoveContainer" containerID="f96ee7ec696d8c8edf044768ec0355f0aeb207330b737eaf555ec2b71de60a8c" Jul 03 11:23:01.511044 ip-10-0-136-0 kubenswrapper[2579]: I0703 11:23:01.511016 2579 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"30892156Ki","pods":"250"} Jul 03 11:23:01.511110 ip-10-0-136-0 kubenswrapper[2579]: I0703 11:23:01.511089 2579 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"30892156Ki","pods":"250"} Jul 03 11:23:01.511146 ip-10-0-136-0 kubenswrapper[2579]: I0703 11:23:01.511121 2579 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"30892156Ki","pods":"250"} Jul 03 11:23:02.517552 ip-10-0-136-0 kubenswrapper[2579]: I0703 11:23:02.517509 2579 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/maas-default-gateway-openshift-default-56654dcf7-fbkb8" event={"ID":"b8f8d2d9-d723-4d69-b7a5-938e70db757f","Type":"ContainerStarted","Data":"8edf654f106b649286af08cdb32ec22728562bcbe308c7c8c159319c74051f55"} Jul 03 11:23:02.596290 ip-10-0-136-0 kubenswrapper[2579]: I0703 11:23:02.596182 2579 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="openshift-ingress/maas-default-gateway-openshift-default-56654dcf7-fbkb8" Jul 03 11:23:02.596562 ip-10-0-136-0 kubenswrapper[2579]: I0703 11:23:02.596534 2579 patch_prober.go:28] interesting pod/maas-default-gateway-openshift-default-56654dcf7-fbkb8 container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.133.0.51:15021/healthz/ready\": dial tcp 10.133.0.51:15021: connect: connection refused" start-of-body= Jul 03 11:23:02.596648 ip-10-0-136-0 kubenswrapper[2579]: I0703 11:23:02.596602 2579 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/maas-default-gateway-openshift-default-56654dcf7-fbkb8" podUID="b8f8d2d9-d723-4d69-b7a5-938e70db757f" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.133.0.51:15021/healthz/ready\": dial tcp 10.133.0.51:15021: connect: connection refused" Jul 03 11:23:03.596267 ip-10-0-136-0 kubenswrapper[2579]: I0703 11:23:03.596228 2579 patch_prober.go:28] interesting pod/maas-default-gateway-openshift-default-56654dcf7-fbkb8 container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.133.0.51:15021/healthz/ready\": dial tcp 10.133.0.51:15021: connect: connection refused" start-of-body= Jul 03 11:23:03.596712 ip-10-0-136-0 kubenswrapper[2579]: I0703 11:23:03.596292 2579 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/maas-default-gateway-openshift-default-56654dcf7-fbkb8" podUID="b8f8d2d9-d723-4d69-b7a5-938e70db757f" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.133.0.51:15021/healthz/ready\": dial tcp 10.133.0.51:15021: connect: connection refused" Jul 03 11:23:04.596705 ip-10-0-136-0 kubenswrapper[2579]: I0703 11:23:04.596658 2579 patch_prober.go:28] interesting pod/maas-default-gateway-openshift-default-56654dcf7-fbkb8 container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.133.0.51:15021/healthz/ready\": dial tcp 10.133.0.51:15021: connect: connection refused" start-of-body= Jul 03 11:23:04.597150 ip-10-0-136-0 kubenswrapper[2579]: I0703 11:23:04.596739 2579 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/maas-default-gateway-openshift-default-56654dcf7-fbkb8" podUID="b8f8d2d9-d723-4d69-b7a5-938e70db757f" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.133.0.51:15021/healthz/ready\": dial tcp 10.133.0.51:15021: connect: connection refused" Jul 03 11:23:05.596664 ip-10-0-136-0 kubenswrapper[2579]: I0703 11:23:05.596620 2579 patch_prober.go:28] interesting pod/maas-default-gateway-openshift-default-56654dcf7-fbkb8 container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.133.0.51:15021/healthz/ready\": dial tcp 10.133.0.51:15021: connect: connection refused" start-of-body= Jul 03 11:23:05.596899 ip-10-0-136-0 kubenswrapper[2579]: I0703 11:23:05.596686 2579 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/maas-default-gateway-openshift-default-56654dcf7-fbkb8" podUID="b8f8d2d9-d723-4d69-b7a5-938e70db757f" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.133.0.51:15021/healthz/ready\": dial tcp 10.133.0.51:15021: connect: connection refused" Jul 03 11:23:06.482490 ip-10-0-136-0 kubenswrapper[2579]: I0703 11:23:06.482435 2579 prober.go:120] "Probe failed" probeType="Readiness" pod="openshift-ingress/payload-pre-processing-7964457696-ncrll" podUID="2c315c89-d57d-4d52-9426-06e88f95bdfb" containerName="payload-pre-processing" probeResult="failure" output="dial tcp 10.133.0.58:9004: connect: connection refused" Jul 03 11:23:06.601575 ip-10-0-136-0 kubenswrapper[2579]: I0703 11:23:06.601531 2579 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="openshift-ingress/maas-default-gateway-openshift-default-56654dcf7-fbkb8" Jul 03 11:23:06.601957 ip-10-0-136-0 kubenswrapper[2579]: I0703 11:23:06.601920 2579 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/maas-default-gateway-openshift-default-56654dcf7-fbkb8" Jul 03 11:23:06.602914 ip-10-0-136-0 kubenswrapper[2579]: I0703 11:23:06.602895 2579 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/maas-default-gateway-openshift-default-56654dcf7-fbkb8" Jul 03 11:23:16.481610 ip-10-0-136-0 kubenswrapper[2579]: I0703 11:23:16.481575 2579 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/payload-pre-processing-7964457696-ncrll" Jul 03 11:23:53.002979 ip-10-0-136-0 kubenswrapper[2579]: I0703 11:23:53.002936 2579 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["llm/e2e-unconfigured-facebook-opt-125m-simulated-kserve-6b86dbqpm6s"] Jul 03 11:23:53.006989 ip-10-0-136-0 kubenswrapper[2579]: I0703 11:23:53.006954 2579 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="llm/e2e-unconfigured-facebook-opt-125m-simulated-kserve-6b86dbqpm6s" Jul 03 11:23:53.009580 ip-10-0-136-0 kubenswrapper[2579]: I0703 11:23:53.009553 2579 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"llm\"/\"default-dockercfg-6qpt2\"" Jul 03 11:23:53.009749 ip-10-0-136-0 kubenswrapper[2579]: I0703 11:23:53.009580 2579 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"llm\"/\"kube-root-ca.crt\"" Jul 03 11:23:53.010840 ip-10-0-136-0 kubenswrapper[2579]: I0703 11:23:53.010815 2579 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"llm\"/\"openshift-service-ca.crt\"" Jul 03 11:23:53.010969 ip-10-0-136-0 kubenswrapper[2579]: I0703 11:23:53.010889 2579 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"llm\"/\"e2e-unab60ef4d3a239b5143b412cab04acac3-kserve-self-signed-certs\"" Jul 03 11:23:53.019658 ip-10-0-136-0 kubenswrapper[2579]: I0703 11:23:53.019622 2579 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["llm/e2e-unconfigured-facebook-opt-125m-simulated-kserve-6b86dbqpm6s"] Jul 03 11:23:53.039071 ip-10-0-136-0 kubenswrapper[2579]: I0703 11:23:53.039027 2579 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-cj7lv\" (UniqueName: \"kubernetes.io/projected/72272306-4c89-4f32-9d23-296f71c9daf4-kube-api-access-cj7lv\") pod \"e2e-unconfigured-facebook-opt-125m-simulated-kserve-6b86dbqpm6s\" (UID: \"72272306-4c89-4f32-9d23-296f71c9daf4\") " pod="llm/e2e-unconfigured-facebook-opt-125m-simulated-kserve-6b86dbqpm6s" Jul 03 11:23:53.039284 ip-10-0-136-0 kubenswrapper[2579]: I0703 11:23:53.039135 2579 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/72272306-4c89-4f32-9d23-296f71c9daf4-tmp-dir\") pod \"e2e-unconfigured-facebook-opt-125m-simulated-kserve-6b86dbqpm6s\" (UID: \"72272306-4c89-4f32-9d23-296f71c9daf4\") " pod="llm/e2e-unconfigured-facebook-opt-125m-simulated-kserve-6b86dbqpm6s" Jul 03 11:23:53.039284 ip-10-0-136-0 kubenswrapper[2579]: I0703 11:23:53.039168 2579 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/72272306-4c89-4f32-9d23-296f71c9daf4-home\") pod \"e2e-unconfigured-facebook-opt-125m-simulated-kserve-6b86dbqpm6s\" (UID: \"72272306-4c89-4f32-9d23-296f71c9daf4\") " pod="llm/e2e-unconfigured-facebook-opt-125m-simulated-kserve-6b86dbqpm6s" Jul 03 11:23:53.039284 ip-10-0-136-0 kubenswrapper[2579]: I0703 11:23:53.039240 2579 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/72272306-4c89-4f32-9d23-296f71c9daf4-tls-certs\") pod \"e2e-unconfigured-facebook-opt-125m-simulated-kserve-6b86dbqpm6s\" (UID: \"72272306-4c89-4f32-9d23-296f71c9daf4\") " pod="llm/e2e-unconfigured-facebook-opt-125m-simulated-kserve-6b86dbqpm6s" Jul 03 11:23:53.039284 ip-10-0-136-0 kubenswrapper[2579]: I0703 11:23:53.039273 2579 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/72272306-4c89-4f32-9d23-296f71c9daf4-model-cache\") pod \"e2e-unconfigured-facebook-opt-125m-simulated-kserve-6b86dbqpm6s\" (UID: \"72272306-4c89-4f32-9d23-296f71c9daf4\") " pod="llm/e2e-unconfigured-facebook-opt-125m-simulated-kserve-6b86dbqpm6s" Jul 03 11:23:53.039437 ip-10-0-136-0 kubenswrapper[2579]: I0703 11:23:53.039305 2579 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/72272306-4c89-4f32-9d23-296f71c9daf4-dshm\") pod \"e2e-unconfigured-facebook-opt-125m-simulated-kserve-6b86dbqpm6s\" (UID: \"72272306-4c89-4f32-9d23-296f71c9daf4\") " pod="llm/e2e-unconfigured-facebook-opt-125m-simulated-kserve-6b86dbqpm6s" Jul 03 11:23:53.140541 ip-10-0-136-0 kubenswrapper[2579]: I0703 11:23:53.140490 2579 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/72272306-4c89-4f32-9d23-296f71c9daf4-dshm\") pod \"e2e-unconfigured-facebook-opt-125m-simulated-kserve-6b86dbqpm6s\" (UID: \"72272306-4c89-4f32-9d23-296f71c9daf4\") " pod="llm/e2e-unconfigured-facebook-opt-125m-simulated-kserve-6b86dbqpm6s" Jul 03 11:23:53.140541 ip-10-0-136-0 kubenswrapper[2579]: I0703 11:23:53.140545 2579 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-cj7lv\" (UniqueName: \"kubernetes.io/projected/72272306-4c89-4f32-9d23-296f71c9daf4-kube-api-access-cj7lv\") pod \"e2e-unconfigured-facebook-opt-125m-simulated-kserve-6b86dbqpm6s\" (UID: \"72272306-4c89-4f32-9d23-296f71c9daf4\") " pod="llm/e2e-unconfigured-facebook-opt-125m-simulated-kserve-6b86dbqpm6s" Jul 03 11:23:53.140863 ip-10-0-136-0 kubenswrapper[2579]: I0703 11:23:53.140641 2579 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/72272306-4c89-4f32-9d23-296f71c9daf4-tmp-dir\") pod \"e2e-unconfigured-facebook-opt-125m-simulated-kserve-6b86dbqpm6s\" (UID: \"72272306-4c89-4f32-9d23-296f71c9daf4\") " pod="llm/e2e-unconfigured-facebook-opt-125m-simulated-kserve-6b86dbqpm6s" Jul 03 11:23:53.140863 ip-10-0-136-0 kubenswrapper[2579]: I0703 11:23:53.140685 2579 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/72272306-4c89-4f32-9d23-296f71c9daf4-home\") pod \"e2e-unconfigured-facebook-opt-125m-simulated-kserve-6b86dbqpm6s\" (UID: \"72272306-4c89-4f32-9d23-296f71c9daf4\") " pod="llm/e2e-unconfigured-facebook-opt-125m-simulated-kserve-6b86dbqpm6s" Jul 03 11:23:53.140863 ip-10-0-136-0 kubenswrapper[2579]: I0703 11:23:53.140717 2579 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/72272306-4c89-4f32-9d23-296f71c9daf4-tls-certs\") pod \"e2e-unconfigured-facebook-opt-125m-simulated-kserve-6b86dbqpm6s\" (UID: \"72272306-4c89-4f32-9d23-296f71c9daf4\") " pod="llm/e2e-unconfigured-facebook-opt-125m-simulated-kserve-6b86dbqpm6s" Jul 03 11:23:53.140863 ip-10-0-136-0 kubenswrapper[2579]: I0703 11:23:53.140762 2579 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/72272306-4c89-4f32-9d23-296f71c9daf4-model-cache\") pod \"e2e-unconfigured-facebook-opt-125m-simulated-kserve-6b86dbqpm6s\" (UID: \"72272306-4c89-4f32-9d23-296f71c9daf4\") " pod="llm/e2e-unconfigured-facebook-opt-125m-simulated-kserve-6b86dbqpm6s" Jul 03 11:23:53.141144 ip-10-0-136-0 kubenswrapper[2579]: I0703 11:23:53.141112 2579 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/72272306-4c89-4f32-9d23-296f71c9daf4-tmp-dir\") pod \"e2e-unconfigured-facebook-opt-125m-simulated-kserve-6b86dbqpm6s\" (UID: \"72272306-4c89-4f32-9d23-296f71c9daf4\") " pod="llm/e2e-unconfigured-facebook-opt-125m-simulated-kserve-6b86dbqpm6s" Jul 03 11:23:53.141374 ip-10-0-136-0 kubenswrapper[2579]: I0703 11:23:53.141288 2579 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/72272306-4c89-4f32-9d23-296f71c9daf4-model-cache\") pod \"e2e-unconfigured-facebook-opt-125m-simulated-kserve-6b86dbqpm6s\" (UID: \"72272306-4c89-4f32-9d23-296f71c9daf4\") " pod="llm/e2e-unconfigured-facebook-opt-125m-simulated-kserve-6b86dbqpm6s" Jul 03 11:23:53.141509 ip-10-0-136-0 kubenswrapper[2579]: I0703 11:23:53.141486 2579 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/72272306-4c89-4f32-9d23-296f71c9daf4-home\") pod \"e2e-unconfigured-facebook-opt-125m-simulated-kserve-6b86dbqpm6s\" (UID: \"72272306-4c89-4f32-9d23-296f71c9daf4\") " pod="llm/e2e-unconfigured-facebook-opt-125m-simulated-kserve-6b86dbqpm6s" Jul 03 11:23:53.143634 ip-10-0-136-0 kubenswrapper[2579]: I0703 11:23:53.143599 2579 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/72272306-4c89-4f32-9d23-296f71c9daf4-dshm\") pod \"e2e-unconfigured-facebook-opt-125m-simulated-kserve-6b86dbqpm6s\" (UID: \"72272306-4c89-4f32-9d23-296f71c9daf4\") " pod="llm/e2e-unconfigured-facebook-opt-125m-simulated-kserve-6b86dbqpm6s" Jul 03 11:23:53.143773 ip-10-0-136-0 kubenswrapper[2579]: I0703 11:23:53.143643 2579 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/72272306-4c89-4f32-9d23-296f71c9daf4-tls-certs\") pod \"e2e-unconfigured-facebook-opt-125m-simulated-kserve-6b86dbqpm6s\" (UID: \"72272306-4c89-4f32-9d23-296f71c9daf4\") " pod="llm/e2e-unconfigured-facebook-opt-125m-simulated-kserve-6b86dbqpm6s" Jul 03 11:23:53.149412 ip-10-0-136-0 kubenswrapper[2579]: I0703 11:23:53.149381 2579 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-cj7lv\" (UniqueName: \"kubernetes.io/projected/72272306-4c89-4f32-9d23-296f71c9daf4-kube-api-access-cj7lv\") pod \"e2e-unconfigured-facebook-opt-125m-simulated-kserve-6b86dbqpm6s\" (UID: \"72272306-4c89-4f32-9d23-296f71c9daf4\") " pod="llm/e2e-unconfigured-facebook-opt-125m-simulated-kserve-6b86dbqpm6s" Jul 03 11:23:53.322106 ip-10-0-136-0 kubenswrapper[2579]: I0703 11:23:53.322063 2579 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="llm/e2e-unconfigured-facebook-opt-125m-simulated-kserve-6b86dbqpm6s" Jul 03 11:23:53.485468 ip-10-0-136-0 kubenswrapper[2579]: I0703 11:23:53.485440 2579 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["llm/e2e-unconfigured-facebook-opt-125m-simulated-kserve-6b86dbqpm6s"] Jul 03 11:23:53.488411 ip-10-0-136-0 kubenswrapper[2579]: W0703 11:23:53.488372 2579 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod72272306_4c89_4f32_9d23_296f71c9daf4.slice/crio-e9642e58acc4f8c541b768da64dc49c44808a22f12aa1d8f4881fea88dc0c343 WatchSource:0}: Error finding container e9642e58acc4f8c541b768da64dc49c44808a22f12aa1d8f4881fea88dc0c343: Status 404 returned error can't find the container with id e9642e58acc4f8c541b768da64dc49c44808a22f12aa1d8f4881fea88dc0c343 Jul 03 11:23:53.826737 ip-10-0-136-0 kubenswrapper[2579]: I0703 11:23:53.826678 2579 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="llm/e2e-unconfigured-facebook-opt-125m-simulated-kserve-6b86dbqpm6s" event={"ID":"72272306-4c89-4f32-9d23-296f71c9daf4","Type":"ContainerStarted","Data":"e9642e58acc4f8c541b768da64dc49c44808a22f12aa1d8f4881fea88dc0c343"} Jul 03 11:23:54.834874 ip-10-0-136-0 kubenswrapper[2579]: I0703 11:23:54.834832 2579 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="llm/e2e-unconfigured-facebook-opt-125m-simulated-kserve-6b86dbqpm6s" event={"ID":"72272306-4c89-4f32-9d23-296f71c9daf4","Type":"ContainerStarted","Data":"9b6a53b4a351f2796eb7cc9e758546284aba626ba0e16f29d1717e3cd10e7b53"} Jul 03 11:23:54.856241 ip-10-0-136-0 kubenswrapper[2579]: I0703 11:23:54.856157 2579 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="llm/e2e-unconfigured-facebook-opt-125m-simulated-kserve-6b86dbqpm6s" podStartSLOduration=1.731889561 podStartE2EDuration="2.856138633s" totalImagesPullingTime="1.124249072s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-03 11:23:52 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-03 11:23:54.853677106 +0000 UTC m=+691.441981433" watchObservedRunningTime="2026-07-03 11:23:54.856138633 +0000 UTC m=+691.444442952" Jul 03 11:24:03.323248 ip-10-0-136-0 kubenswrapper[2579]: I0703 11:24:03.323175 2579 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="llm/e2e-unconfigured-facebook-opt-125m-simulated-kserve-6b86dbqpm6s" Jul 03 11:24:03.323779 ip-10-0-136-0 kubenswrapper[2579]: I0703 11:24:03.323282 2579 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="llm/e2e-unconfigured-facebook-opt-125m-simulated-kserve-6b86dbqpm6s" Jul 03 11:24:03.336553 ip-10-0-136-0 kubenswrapper[2579]: I0703 11:24:03.336516 2579 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="llm/e2e-unconfigured-facebook-opt-125m-simulated-kserve-6b86dbqpm6s" Jul 03 11:24:03.903504 ip-10-0-136-0 kubenswrapper[2579]: I0703 11:24:03.903468 2579 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="llm/e2e-unconfigured-facebook-opt-125m-simulated-kserve-6b86dbqpm6s" Jul 03 11:24:36.779158 ip-10-0-136-0 kubenswrapper[2579]: I0703 11:24:36.779065 2579 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["llm/premium-simulated-simulated-premium-kserve-5fc6fb456f-qzj65"] Jul 03 11:24:36.783285 ip-10-0-136-0 kubenswrapper[2579]: I0703 11:24:36.783257 2579 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="llm/premium-simulated-simulated-premium-kserve-5fc6fb456f-qzj65" Jul 03 11:24:36.786260 ip-10-0-136-0 kubenswrapper[2579]: I0703 11:24:36.786234 2579 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"llm\"/\"premium-simulated-simulated-premium-kserve-self-signed-certs\"" Jul 03 11:24:36.793306 ip-10-0-136-0 kubenswrapper[2579]: I0703 11:24:36.793279 2579 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["llm/premium-simulated-simulated-premium-kserve-5fc6fb456f-qzj65"] Jul 03 11:24:36.865698 ip-10-0-136-0 kubenswrapper[2579]: I0703 11:24:36.865647 2579 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/2328f303-ef30-4846-beb3-7f2124b9de08-model-cache\") pod \"premium-simulated-simulated-premium-kserve-5fc6fb456f-qzj65\" (UID: \"2328f303-ef30-4846-beb3-7f2124b9de08\") " pod="llm/premium-simulated-simulated-premium-kserve-5fc6fb456f-qzj65" Jul 03 11:24:36.865698 ip-10-0-136-0 kubenswrapper[2579]: I0703 11:24:36.865699 2579 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/2328f303-ef30-4846-beb3-7f2124b9de08-dshm\") pod \"premium-simulated-simulated-premium-kserve-5fc6fb456f-qzj65\" (UID: \"2328f303-ef30-4846-beb3-7f2124b9de08\") " pod="llm/premium-simulated-simulated-premium-kserve-5fc6fb456f-qzj65" Jul 03 11:24:36.865965 ip-10-0-136-0 kubenswrapper[2579]: I0703 11:24:36.865747 2579 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/2328f303-ef30-4846-beb3-7f2124b9de08-home\") pod \"premium-simulated-simulated-premium-kserve-5fc6fb456f-qzj65\" (UID: \"2328f303-ef30-4846-beb3-7f2124b9de08\") " pod="llm/premium-simulated-simulated-premium-kserve-5fc6fb456f-qzj65" Jul 03 11:24:36.865965 ip-10-0-136-0 kubenswrapper[2579]: I0703 11:24:36.865820 2579 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/2328f303-ef30-4846-beb3-7f2124b9de08-tmp-dir\") pod \"premium-simulated-simulated-premium-kserve-5fc6fb456f-qzj65\" (UID: \"2328f303-ef30-4846-beb3-7f2124b9de08\") " pod="llm/premium-simulated-simulated-premium-kserve-5fc6fb456f-qzj65" Jul 03 11:24:36.865965 ip-10-0-136-0 kubenswrapper[2579]: I0703 11:24:36.865883 2579 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/2328f303-ef30-4846-beb3-7f2124b9de08-tls-certs\") pod \"premium-simulated-simulated-premium-kserve-5fc6fb456f-qzj65\" (UID: \"2328f303-ef30-4846-beb3-7f2124b9de08\") " pod="llm/premium-simulated-simulated-premium-kserve-5fc6fb456f-qzj65" Jul 03 11:24:36.866088 ip-10-0-136-0 kubenswrapper[2579]: I0703 11:24:36.865959 2579 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-87l79\" (UniqueName: \"kubernetes.io/projected/2328f303-ef30-4846-beb3-7f2124b9de08-kube-api-access-87l79\") pod \"premium-simulated-simulated-premium-kserve-5fc6fb456f-qzj65\" (UID: \"2328f303-ef30-4846-beb3-7f2124b9de08\") " pod="llm/premium-simulated-simulated-premium-kserve-5fc6fb456f-qzj65" Jul 03 11:24:36.966528 ip-10-0-136-0 kubenswrapper[2579]: I0703 11:24:36.966483 2579 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/2328f303-ef30-4846-beb3-7f2124b9de08-tmp-dir\") pod \"premium-simulated-simulated-premium-kserve-5fc6fb456f-qzj65\" (UID: \"2328f303-ef30-4846-beb3-7f2124b9de08\") " pod="llm/premium-simulated-simulated-premium-kserve-5fc6fb456f-qzj65" Jul 03 11:24:36.966746 ip-10-0-136-0 kubenswrapper[2579]: I0703 11:24:36.966537 2579 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/2328f303-ef30-4846-beb3-7f2124b9de08-tls-certs\") pod \"premium-simulated-simulated-premium-kserve-5fc6fb456f-qzj65\" (UID: \"2328f303-ef30-4846-beb3-7f2124b9de08\") " pod="llm/premium-simulated-simulated-premium-kserve-5fc6fb456f-qzj65" Jul 03 11:24:36.966746 ip-10-0-136-0 kubenswrapper[2579]: I0703 11:24:36.966569 2579 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-87l79\" (UniqueName: \"kubernetes.io/projected/2328f303-ef30-4846-beb3-7f2124b9de08-kube-api-access-87l79\") pod \"premium-simulated-simulated-premium-kserve-5fc6fb456f-qzj65\" (UID: \"2328f303-ef30-4846-beb3-7f2124b9de08\") " pod="llm/premium-simulated-simulated-premium-kserve-5fc6fb456f-qzj65" Jul 03 11:24:36.966746 ip-10-0-136-0 kubenswrapper[2579]: I0703 11:24:36.966607 2579 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/2328f303-ef30-4846-beb3-7f2124b9de08-model-cache\") pod \"premium-simulated-simulated-premium-kserve-5fc6fb456f-qzj65\" (UID: \"2328f303-ef30-4846-beb3-7f2124b9de08\") " pod="llm/premium-simulated-simulated-premium-kserve-5fc6fb456f-qzj65" Jul 03 11:24:36.966746 ip-10-0-136-0 kubenswrapper[2579]: I0703 11:24:36.966625 2579 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/2328f303-ef30-4846-beb3-7f2124b9de08-dshm\") pod \"premium-simulated-simulated-premium-kserve-5fc6fb456f-qzj65\" (UID: \"2328f303-ef30-4846-beb3-7f2124b9de08\") " pod="llm/premium-simulated-simulated-premium-kserve-5fc6fb456f-qzj65" Jul 03 11:24:36.966746 ip-10-0-136-0 kubenswrapper[2579]: I0703 11:24:36.966646 2579 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/2328f303-ef30-4846-beb3-7f2124b9de08-home\") pod \"premium-simulated-simulated-premium-kserve-5fc6fb456f-qzj65\" (UID: \"2328f303-ef30-4846-beb3-7f2124b9de08\") " pod="llm/premium-simulated-simulated-premium-kserve-5fc6fb456f-qzj65" Jul 03 11:24:36.967012 ip-10-0-136-0 kubenswrapper[2579]: I0703 11:24:36.966970 2579 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/2328f303-ef30-4846-beb3-7f2124b9de08-model-cache\") pod \"premium-simulated-simulated-premium-kserve-5fc6fb456f-qzj65\" (UID: \"2328f303-ef30-4846-beb3-7f2124b9de08\") " pod="llm/premium-simulated-simulated-premium-kserve-5fc6fb456f-qzj65" Jul 03 11:24:36.967012 ip-10-0-136-0 kubenswrapper[2579]: I0703 11:24:36.967002 2579 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/2328f303-ef30-4846-beb3-7f2124b9de08-tmp-dir\") pod \"premium-simulated-simulated-premium-kserve-5fc6fb456f-qzj65\" (UID: \"2328f303-ef30-4846-beb3-7f2124b9de08\") " pod="llm/premium-simulated-simulated-premium-kserve-5fc6fb456f-qzj65" Jul 03 11:24:36.967099 ip-10-0-136-0 kubenswrapper[2579]: I0703 11:24:36.967079 2579 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/2328f303-ef30-4846-beb3-7f2124b9de08-home\") pod \"premium-simulated-simulated-premium-kserve-5fc6fb456f-qzj65\" (UID: \"2328f303-ef30-4846-beb3-7f2124b9de08\") " pod="llm/premium-simulated-simulated-premium-kserve-5fc6fb456f-qzj65" Jul 03 11:24:36.969144 ip-10-0-136-0 kubenswrapper[2579]: I0703 11:24:36.969106 2579 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/2328f303-ef30-4846-beb3-7f2124b9de08-dshm\") pod \"premium-simulated-simulated-premium-kserve-5fc6fb456f-qzj65\" (UID: \"2328f303-ef30-4846-beb3-7f2124b9de08\") " pod="llm/premium-simulated-simulated-premium-kserve-5fc6fb456f-qzj65" Jul 03 11:24:36.969345 ip-10-0-136-0 kubenswrapper[2579]: I0703 11:24:36.969261 2579 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/2328f303-ef30-4846-beb3-7f2124b9de08-tls-certs\") pod \"premium-simulated-simulated-premium-kserve-5fc6fb456f-qzj65\" (UID: \"2328f303-ef30-4846-beb3-7f2124b9de08\") " pod="llm/premium-simulated-simulated-premium-kserve-5fc6fb456f-qzj65" Jul 03 11:24:36.975486 ip-10-0-136-0 kubenswrapper[2579]: I0703 11:24:36.975451 2579 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-87l79\" (UniqueName: \"kubernetes.io/projected/2328f303-ef30-4846-beb3-7f2124b9de08-kube-api-access-87l79\") pod \"premium-simulated-simulated-premium-kserve-5fc6fb456f-qzj65\" (UID: \"2328f303-ef30-4846-beb3-7f2124b9de08\") " pod="llm/premium-simulated-simulated-premium-kserve-5fc6fb456f-qzj65" Jul 03 11:24:37.094422 ip-10-0-136-0 kubenswrapper[2579]: I0703 11:24:37.094317 2579 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="llm/premium-simulated-simulated-premium-kserve-5fc6fb456f-qzj65" Jul 03 11:24:37.257232 ip-10-0-136-0 kubenswrapper[2579]: I0703 11:24:37.255288 2579 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["llm/premium-simulated-simulated-premium-kserve-5fc6fb456f-qzj65"] Jul 03 11:24:38.100956 ip-10-0-136-0 kubenswrapper[2579]: I0703 11:24:38.100922 2579 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="llm/premium-simulated-simulated-premium-kserve-5fc6fb456f-qzj65" event={"ID":"2328f303-ef30-4846-beb3-7f2124b9de08","Type":"ContainerStarted","Data":"1fbc2d74cbe47e64166bd47b196685a3367f348bc9960f57bdcff37fb1e81e1a"} Jul 03 11:24:38.101368 ip-10-0-136-0 kubenswrapper[2579]: I0703 11:24:38.100962 2579 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="llm/premium-simulated-simulated-premium-kserve-5fc6fb456f-qzj65" event={"ID":"2328f303-ef30-4846-beb3-7f2124b9de08","Type":"ContainerStarted","Data":"fcf3548833e85f1bffeaa957c3df6948b01ecbc60f10a0dd3583805265d7c3bb"} Jul 03 11:24:38.121695 ip-10-0-136-0 kubenswrapper[2579]: I0703 11:24:38.121636 2579 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="llm/premium-simulated-simulated-premium-kserve-5fc6fb456f-qzj65" podStartSLOduration=1.905286184 podStartE2EDuration="2.121616671s" totalImagesPullingTime="216.330487ms" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-03 11:24:36 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-03 11:24:38.120503859 +0000 UTC m=+734.708808190" watchObservedRunningTime="2026-07-03 11:24:38.121616671 +0000 UTC m=+734.709921010" Jul 03 11:24:47.095373 ip-10-0-136-0 kubenswrapper[2579]: I0703 11:24:47.095324 2579 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="llm/premium-simulated-simulated-premium-kserve-5fc6fb456f-qzj65" Jul 03 11:24:47.095817 ip-10-0-136-0 kubenswrapper[2579]: I0703 11:24:47.095390 2579 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="llm/premium-simulated-simulated-premium-kserve-5fc6fb456f-qzj65" Jul 03 11:24:47.108569 ip-10-0-136-0 kubenswrapper[2579]: I0703 11:24:47.108533 2579 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="llm/premium-simulated-simulated-premium-kserve-5fc6fb456f-qzj65" Jul 03 11:24:47.176046 ip-10-0-136-0 kubenswrapper[2579]: I0703 11:24:47.176012 2579 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="llm/premium-simulated-simulated-premium-kserve-5fc6fb456f-qzj65" Jul 03 11:25:32.968785 ip-10-0-136-0 kubenswrapper[2579]: I0703 11:25:32.968744 2579 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["kuadrant-system/authorino-55cdcbbc74-wx8bc"] Jul 03 11:25:32.972405 ip-10-0-136-0 kubenswrapper[2579]: I0703 11:25:32.972380 2579 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/authorino-55cdcbbc74-wx8bc" Jul 03 11:25:32.980411 ip-10-0-136-0 kubenswrapper[2579]: I0703 11:25:32.980381 2579 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/authorino-55cdcbbc74-wx8bc"] Jul 03 11:25:33.007906 ip-10-0-136-0 kubenswrapper[2579]: I0703 11:25:33.007865 2579 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tls-cert\" (UniqueName: \"kubernetes.io/secret/49857fe6-d325-4c68-9283-4fbad85410e6-tls-cert\") pod \"authorino-55cdcbbc74-wx8bc\" (UID: \"49857fe6-d325-4c68-9283-4fbad85410e6\") " pod="kuadrant-system/authorino-55cdcbbc74-wx8bc" Jul 03 11:25:33.008088 ip-10-0-136-0 kubenswrapper[2579]: I0703 11:25:33.007921 2579 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-kjvmq\" (UniqueName: \"kubernetes.io/projected/49857fe6-d325-4c68-9283-4fbad85410e6-kube-api-access-kjvmq\") pod \"authorino-55cdcbbc74-wx8bc\" (UID: \"49857fe6-d325-4c68-9283-4fbad85410e6\") " pod="kuadrant-system/authorino-55cdcbbc74-wx8bc" Jul 03 11:25:33.109178 ip-10-0-136-0 kubenswrapper[2579]: I0703 11:25:33.109125 2579 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tls-cert\" (UniqueName: \"kubernetes.io/secret/49857fe6-d325-4c68-9283-4fbad85410e6-tls-cert\") pod \"authorino-55cdcbbc74-wx8bc\" (UID: \"49857fe6-d325-4c68-9283-4fbad85410e6\") " pod="kuadrant-system/authorino-55cdcbbc74-wx8bc" Jul 03 11:25:33.109403 ip-10-0-136-0 kubenswrapper[2579]: I0703 11:25:33.109224 2579 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-kjvmq\" (UniqueName: \"kubernetes.io/projected/49857fe6-d325-4c68-9283-4fbad85410e6-kube-api-access-kjvmq\") pod \"authorino-55cdcbbc74-wx8bc\" (UID: \"49857fe6-d325-4c68-9283-4fbad85410e6\") " pod="kuadrant-system/authorino-55cdcbbc74-wx8bc" Jul 03 11:25:33.112044 ip-10-0-136-0 kubenswrapper[2579]: I0703 11:25:33.112012 2579 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"tls-cert\" (UniqueName: \"kubernetes.io/secret/49857fe6-d325-4c68-9283-4fbad85410e6-tls-cert\") pod \"authorino-55cdcbbc74-wx8bc\" (UID: \"49857fe6-d325-4c68-9283-4fbad85410e6\") " pod="kuadrant-system/authorino-55cdcbbc74-wx8bc" Jul 03 11:25:33.117297 ip-10-0-136-0 kubenswrapper[2579]: I0703 11:25:33.117266 2579 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-kjvmq\" (UniqueName: \"kubernetes.io/projected/49857fe6-d325-4c68-9283-4fbad85410e6-kube-api-access-kjvmq\") pod \"authorino-55cdcbbc74-wx8bc\" (UID: \"49857fe6-d325-4c68-9283-4fbad85410e6\") " pod="kuadrant-system/authorino-55cdcbbc74-wx8bc" Jul 03 11:25:33.283320 ip-10-0-136-0 kubenswrapper[2579]: I0703 11:25:33.283279 2579 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/authorino-55cdcbbc74-wx8bc" Jul 03 11:25:33.459819 ip-10-0-136-0 kubenswrapper[2579]: I0703 11:25:33.459784 2579 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/authorino-55cdcbbc74-wx8bc"] Jul 03 11:25:33.461700 ip-10-0-136-0 kubenswrapper[2579]: W0703 11:25:33.461660 2579 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-besteffort.slice/kubepods-besteffort-pod49857fe6_d325_4c68_9283_4fbad85410e6.slice/crio-ec7e3ccaec85886ee95b41e8e068fa2c0843f83943fb0a622d9864cdaa8150de WatchSource:0}: Error finding container ec7e3ccaec85886ee95b41e8e068fa2c0843f83943fb0a622d9864cdaa8150de: Status 404 returned error can't find the container with id ec7e3ccaec85886ee95b41e8e068fa2c0843f83943fb0a622d9864cdaa8150de Jul 03 11:25:34.447557 ip-10-0-136-0 kubenswrapper[2579]: I0703 11:25:34.447506 2579 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/authorino-55cdcbbc74-wx8bc" event={"ID":"49857fe6-d325-4c68-9283-4fbad85410e6","Type":"ContainerStarted","Data":"c4340998a94331b23134d935645c4dcca535111c41045c3a65c59702b5967057"} Jul 03 11:25:34.448073 ip-10-0-136-0 kubenswrapper[2579]: I0703 11:25:34.447585 2579 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/authorino-55cdcbbc74-wx8bc" event={"ID":"49857fe6-d325-4c68-9283-4fbad85410e6","Type":"ContainerStarted","Data":"ec7e3ccaec85886ee95b41e8e068fa2c0843f83943fb0a622d9864cdaa8150de"} Jul 03 11:25:34.469848 ip-10-0-136-0 kubenswrapper[2579]: I0703 11:25:34.469762 2579 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="kuadrant-system/authorino-55cdcbbc74-wx8bc" podStartSLOduration=2.104676235 podStartE2EDuration="2.46973219s" totalImagesPullingTime="365.055955ms" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-03 11:25:32 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-03 11:25:34.464733775 +0000 UTC m=+791.053038096" watchObservedRunningTime="2026-07-03 11:25:34.46973219 +0000 UTC m=+791.058036511" Jul 03 11:25:34.502914 ip-10-0-136-0 kubenswrapper[2579]: I0703 11:25:34.502867 2579 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["kuadrant-system/authorino-5df584787-qjzxd"] Jul 03 11:25:34.503492 ip-10-0-136-0 kubenswrapper[2579]: I0703 11:25:34.503134 2579 kuberuntime_container.go:872] "Killing container with a grace period" pod="kuadrant-system/authorino-5df584787-qjzxd" podUID="bd887377-3185-4dda-9a7e-b8c0d968b6ae" containerName="authorino" containerID="cri-o://0f53618cdfd2ba0856906c4229dd63b3ffeef1b53e618260f401aacb2f53dd00" gracePeriod=30 Jul 03 11:25:34.805128 ip-10-0-136-0 kubenswrapper[2579]: I0703 11:25:34.805092 2579 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/authorino-5df584787-qjzxd" Jul 03 11:25:34.928964 ip-10-0-136-0 kubenswrapper[2579]: I0703 11:25:34.928924 2579 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-2ghmd\" (UniqueName: \"kubernetes.io/projected/bd887377-3185-4dda-9a7e-b8c0d968b6ae-kube-api-access-2ghmd\") pod \"bd887377-3185-4dda-9a7e-b8c0d968b6ae\" (UID: \"bd887377-3185-4dda-9a7e-b8c0d968b6ae\") " Jul 03 11:25:34.929141 ip-10-0-136-0 kubenswrapper[2579]: I0703 11:25:34.929052 2579 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"tls-cert\" (UniqueName: \"kubernetes.io/secret/bd887377-3185-4dda-9a7e-b8c0d968b6ae-tls-cert\") pod \"bd887377-3185-4dda-9a7e-b8c0d968b6ae\" (UID: \"bd887377-3185-4dda-9a7e-b8c0d968b6ae\") " Jul 03 11:25:34.931464 ip-10-0-136-0 kubenswrapper[2579]: I0703 11:25:34.931419 2579 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/bd887377-3185-4dda-9a7e-b8c0d968b6ae-kube-api-access-2ghmd" (OuterVolumeSpecName: "kube-api-access-2ghmd") pod "bd887377-3185-4dda-9a7e-b8c0d968b6ae" (UID: "bd887377-3185-4dda-9a7e-b8c0d968b6ae"). InnerVolumeSpecName "kube-api-access-2ghmd". PluginName "kubernetes.io/projected", VolumeGIDValue "" Jul 03 11:25:34.944364 ip-10-0-136-0 kubenswrapper[2579]: I0703 11:25:34.944318 2579 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/bd887377-3185-4dda-9a7e-b8c0d968b6ae-tls-cert" (OuterVolumeSpecName: "tls-cert") pod "bd887377-3185-4dda-9a7e-b8c0d968b6ae" (UID: "bd887377-3185-4dda-9a7e-b8c0d968b6ae"). InnerVolumeSpecName "tls-cert". PluginName "kubernetes.io/secret", VolumeGIDValue "" Jul 03 11:25:35.030318 ip-10-0-136-0 kubenswrapper[2579]: I0703 11:25:35.030279 2579 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-2ghmd\" (UniqueName: \"kubernetes.io/projected/bd887377-3185-4dda-9a7e-b8c0d968b6ae-kube-api-access-2ghmd\") on node \"ip-10-0-136-0.ec2.internal\" DevicePath \"\"" Jul 03 11:25:35.030318 ip-10-0-136-0 kubenswrapper[2579]: I0703 11:25:35.030310 2579 reconciler_common.go:299] "Volume detached for volume \"tls-cert\" (UniqueName: \"kubernetes.io/secret/bd887377-3185-4dda-9a7e-b8c0d968b6ae-tls-cert\") on node \"ip-10-0-136-0.ec2.internal\" DevicePath \"\"" Jul 03 11:25:35.455330 ip-10-0-136-0 kubenswrapper[2579]: I0703 11:25:35.455219 2579 generic.go:356] "Generic (PLEG): container finished" podID="bd887377-3185-4dda-9a7e-b8c0d968b6ae" containerID="0f53618cdfd2ba0856906c4229dd63b3ffeef1b53e618260f401aacb2f53dd00" exitCode=0 Jul 03 11:25:35.455330 ip-10-0-136-0 kubenswrapper[2579]: I0703 11:25:35.455268 2579 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/authorino-5df584787-qjzxd" Jul 03 11:25:35.455330 ip-10-0-136-0 kubenswrapper[2579]: I0703 11:25:35.455312 2579 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/authorino-5df584787-qjzxd" event={"ID":"bd887377-3185-4dda-9a7e-b8c0d968b6ae","Type":"ContainerDied","Data":"0f53618cdfd2ba0856906c4229dd63b3ffeef1b53e618260f401aacb2f53dd00"} Jul 03 11:25:35.455921 ip-10-0-136-0 kubenswrapper[2579]: I0703 11:25:35.455366 2579 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/authorino-5df584787-qjzxd" event={"ID":"bd887377-3185-4dda-9a7e-b8c0d968b6ae","Type":"ContainerDied","Data":"d39438c0a96f7d72f6dc4469feeed4c51c5da967b67c28bf088e9b6910962ef6"} Jul 03 11:25:35.455921 ip-10-0-136-0 kubenswrapper[2579]: I0703 11:25:35.455392 2579 scope.go:117] "RemoveContainer" containerID="0f53618cdfd2ba0856906c4229dd63b3ffeef1b53e618260f401aacb2f53dd00" Jul 03 11:25:35.468472 ip-10-0-136-0 kubenswrapper[2579]: I0703 11:25:35.468439 2579 scope.go:117] "RemoveContainer" containerID="0f53618cdfd2ba0856906c4229dd63b3ffeef1b53e618260f401aacb2f53dd00" Jul 03 11:25:35.468862 ip-10-0-136-0 kubenswrapper[2579]: E0703 11:25:35.468835 2579 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"0f53618cdfd2ba0856906c4229dd63b3ffeef1b53e618260f401aacb2f53dd00\": container with ID starting with 0f53618cdfd2ba0856906c4229dd63b3ffeef1b53e618260f401aacb2f53dd00 not found: ID does not exist" containerID="0f53618cdfd2ba0856906c4229dd63b3ffeef1b53e618260f401aacb2f53dd00" Jul 03 11:25:35.468971 ip-10-0-136-0 kubenswrapper[2579]: I0703 11:25:35.468870 2579 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"0f53618cdfd2ba0856906c4229dd63b3ffeef1b53e618260f401aacb2f53dd00"} err="failed to get container status \"0f53618cdfd2ba0856906c4229dd63b3ffeef1b53e618260f401aacb2f53dd00\": rpc error: code = NotFound desc = could not find container \"0f53618cdfd2ba0856906c4229dd63b3ffeef1b53e618260f401aacb2f53dd00\": container with ID starting with 0f53618cdfd2ba0856906c4229dd63b3ffeef1b53e618260f401aacb2f53dd00 not found: ID does not exist" Jul 03 11:25:35.488238 ip-10-0-136-0 kubenswrapper[2579]: I0703 11:25:35.488158 2579 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["kuadrant-system/authorino-5df584787-qjzxd"] Jul 03 11:25:35.490889 ip-10-0-136-0 kubenswrapper[2579]: I0703 11:25:35.490852 2579 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["kuadrant-system/authorino-5df584787-qjzxd"] Jul 03 11:25:35.994543 ip-10-0-136-0 kubenswrapper[2579]: I0703 11:25:35.994494 2579 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="bd887377-3185-4dda-9a7e-b8c0d968b6ae" path="/var/lib/kubelet/pods/bd887377-3185-4dda-9a7e-b8c0d968b6ae/volumes" Jul 03 11:26:58.731058 ip-10-0-136-0 kubenswrapper[2579]: I0703 11:26:58.731014 2579 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["opendatahub/maas-controller-7b6bc6f48f-dldkg"] Jul 03 11:26:58.731690 ip-10-0-136-0 kubenswrapper[2579]: I0703 11:26:58.731279 2579 kuberuntime_container.go:872] "Killing container with a grace period" pod="opendatahub/maas-controller-7b6bc6f48f-dldkg" podUID="c5324bdc-22d4-4bc3-a9fe-2f673ad11c58" containerName="manager" containerID="cri-o://1802c17c7e83d4f5a418d573ef32fc9ef0027869085995d1e64cb25f75dea4e4" gracePeriod=30 Jul 03 11:26:58.991338 ip-10-0-136-0 kubenswrapper[2579]: I0703 11:26:58.991241 2579 generic.go:356] "Generic (PLEG): container finished" podID="c5324bdc-22d4-4bc3-a9fe-2f673ad11c58" containerID="1802c17c7e83d4f5a418d573ef32fc9ef0027869085995d1e64cb25f75dea4e4" exitCode=0 Jul 03 11:26:58.991338 ip-10-0-136-0 kubenswrapper[2579]: I0703 11:26:58.991304 2579 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="opendatahub/maas-controller-7b6bc6f48f-dldkg" event={"ID":"c5324bdc-22d4-4bc3-a9fe-2f673ad11c58","Type":"ContainerDied","Data":"1802c17c7e83d4f5a418d573ef32fc9ef0027869085995d1e64cb25f75dea4e4"} Jul 03 11:26:58.991535 ip-10-0-136-0 kubenswrapper[2579]: I0703 11:26:58.991356 2579 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="opendatahub/maas-controller-7b6bc6f48f-dldkg" event={"ID":"c5324bdc-22d4-4bc3-a9fe-2f673ad11c58","Type":"ContainerDied","Data":"579530daa3b6912816b38764216cd35036eb48f63808bb40e8e2fc61eca81eea"} Jul 03 11:26:58.991535 ip-10-0-136-0 kubenswrapper[2579]: I0703 11:26:58.991374 2579 pod_container_deletor.go:80] "Container not found in pod's containers" containerID="579530daa3b6912816b38764216cd35036eb48f63808bb40e8e2fc61eca81eea" Jul 03 11:26:58.996698 ip-10-0-136-0 kubenswrapper[2579]: I0703 11:26:58.996662 2579 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="opendatahub/maas-controller-7b6bc6f48f-dldkg" Jul 03 11:26:59.069144 ip-10-0-136-0 kubenswrapper[2579]: I0703 11:26:59.069105 2579 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"webhook-cert\" (UniqueName: \"kubernetes.io/secret/c5324bdc-22d4-4bc3-a9fe-2f673ad11c58-webhook-cert\") pod \"c5324bdc-22d4-4bc3-a9fe-2f673ad11c58\" (UID: \"c5324bdc-22d4-4bc3-a9fe-2f673ad11c58\") " Jul 03 11:26:59.069365 ip-10-0-136-0 kubenswrapper[2579]: I0703 11:26:59.069255 2579 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-f2t6f\" (UniqueName: \"kubernetes.io/projected/c5324bdc-22d4-4bc3-a9fe-2f673ad11c58-kube-api-access-f2t6f\") pod \"c5324bdc-22d4-4bc3-a9fe-2f673ad11c58\" (UID: \"c5324bdc-22d4-4bc3-a9fe-2f673ad11c58\") " Jul 03 11:26:59.071909 ip-10-0-136-0 kubenswrapper[2579]: I0703 11:26:59.071867 2579 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/c5324bdc-22d4-4bc3-a9fe-2f673ad11c58-webhook-cert" (OuterVolumeSpecName: "webhook-cert") pod "c5324bdc-22d4-4bc3-a9fe-2f673ad11c58" (UID: "c5324bdc-22d4-4bc3-a9fe-2f673ad11c58"). InnerVolumeSpecName "webhook-cert". PluginName "kubernetes.io/secret", VolumeGIDValue "" Jul 03 11:26:59.071909 ip-10-0-136-0 kubenswrapper[2579]: I0703 11:26:59.071875 2579 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/c5324bdc-22d4-4bc3-a9fe-2f673ad11c58-kube-api-access-f2t6f" (OuterVolumeSpecName: "kube-api-access-f2t6f") pod "c5324bdc-22d4-4bc3-a9fe-2f673ad11c58" (UID: "c5324bdc-22d4-4bc3-a9fe-2f673ad11c58"). InnerVolumeSpecName "kube-api-access-f2t6f". PluginName "kubernetes.io/projected", VolumeGIDValue "" Jul 03 11:26:59.170487 ip-10-0-136-0 kubenswrapper[2579]: I0703 11:26:59.170430 2579 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-f2t6f\" (UniqueName: \"kubernetes.io/projected/c5324bdc-22d4-4bc3-a9fe-2f673ad11c58-kube-api-access-f2t6f\") on node \"ip-10-0-136-0.ec2.internal\" DevicePath \"\"" Jul 03 11:26:59.170487 ip-10-0-136-0 kubenswrapper[2579]: I0703 11:26:59.170479 2579 reconciler_common.go:299] "Volume detached for volume \"webhook-cert\" (UniqueName: \"kubernetes.io/secret/c5324bdc-22d4-4bc3-a9fe-2f673ad11c58-webhook-cert\") on node \"ip-10-0-136-0.ec2.internal\" DevicePath \"\"" Jul 03 11:26:59.998372 ip-10-0-136-0 kubenswrapper[2579]: I0703 11:26:59.998324 2579 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="opendatahub/maas-controller-7b6bc6f48f-dldkg" Jul 03 11:27:00.024122 ip-10-0-136-0 kubenswrapper[2579]: I0703 11:27:00.024060 2579 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["opendatahub/maas-controller-7b6bc6f48f-dldkg"] Jul 03 11:27:00.032028 ip-10-0-136-0 kubenswrapper[2579]: I0703 11:27:00.031988 2579 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["opendatahub/maas-controller-7b6bc6f48f-dldkg"] Jul 03 11:27:01.992635 ip-10-0-136-0 kubenswrapper[2579]: I0703 11:27:01.992597 2579 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="c5324bdc-22d4-4bc3-a9fe-2f673ad11c58" path="/var/lib/kubelet/pods/c5324bdc-22d4-4bc3-a9fe-2f673ad11c58/volumes" Jul 03 11:28:24.007231 ip-10-0-136-0 kubenswrapper[2579]: I0703 11:28:24.007171 2579 scope.go:117] "RemoveContainer" containerID="1802c17c7e83d4f5a418d573ef32fc9ef0027869085995d1e64cb25f75dea4e4" Jul 03 11:31:55.720071 ip-10-0-136-0 kubenswrapper[2579]: I0703 11:31:55.719959 2579 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["kuadrant-system/kuadrant-operator-controller-manager-86878c769f-ppdg8"] Jul 03 11:31:55.720695 ip-10-0-136-0 kubenswrapper[2579]: I0703 11:31:55.720260 2579 kuberuntime_container.go:872] "Killing container with a grace period" pod="kuadrant-system/kuadrant-operator-controller-manager-86878c769f-ppdg8" podUID="a7680483-0b55-4ead-b345-9467eda81e16" containerName="manager" containerID="cri-o://549207d79b521dd59998441b8c5f2efbb35cdc452e428c14eec0f1387d601d00" gracePeriod=10 Jul 03 11:31:58.873026 ip-10-0-136-0 kubenswrapper[2579]: I0703 11:31:58.872988 2579 generic.go:356] "Generic (PLEG): container finished" podID="a7680483-0b55-4ead-b345-9467eda81e16" containerID="549207d79b521dd59998441b8c5f2efbb35cdc452e428c14eec0f1387d601d00" exitCode=0 Jul 03 11:31:58.873462 ip-10-0-136-0 kubenswrapper[2579]: I0703 11:31:58.873068 2579 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/kuadrant-operator-controller-manager-86878c769f-ppdg8" event={"ID":"a7680483-0b55-4ead-b345-9467eda81e16","Type":"ContainerDied","Data":"549207d79b521dd59998441b8c5f2efbb35cdc452e428c14eec0f1387d601d00"} Jul 03 11:31:58.999170 ip-10-0-136-0 kubenswrapper[2579]: I0703 11:31:58.999137 2579 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/kuadrant-operator-controller-manager-86878c769f-ppdg8" Jul 03 11:31:59.105742 ip-10-0-136-0 kubenswrapper[2579]: I0703 11:31:59.105630 2579 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-6jhdh\" (UniqueName: \"kubernetes.io/projected/a7680483-0b55-4ead-b345-9467eda81e16-kube-api-access-6jhdh\") pod \"a7680483-0b55-4ead-b345-9467eda81e16\" (UID: \"a7680483-0b55-4ead-b345-9467eda81e16\") " Jul 03 11:31:59.105742 ip-10-0-136-0 kubenswrapper[2579]: I0703 11:31:59.105737 2579 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"extensions-socket-volume\" (UniqueName: \"kubernetes.io/empty-dir/a7680483-0b55-4ead-b345-9467eda81e16-extensions-socket-volume\") pod \"a7680483-0b55-4ead-b345-9467eda81e16\" (UID: \"a7680483-0b55-4ead-b345-9467eda81e16\") " Jul 03 11:31:59.106242 ip-10-0-136-0 kubenswrapper[2579]: I0703 11:31:59.106177 2579 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/a7680483-0b55-4ead-b345-9467eda81e16-extensions-socket-volume" (OuterVolumeSpecName: "extensions-socket-volume") pod "a7680483-0b55-4ead-b345-9467eda81e16" (UID: "a7680483-0b55-4ead-b345-9467eda81e16"). InnerVolumeSpecName "extensions-socket-volume". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Jul 03 11:31:59.108054 ip-10-0-136-0 kubenswrapper[2579]: I0703 11:31:59.108029 2579 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/a7680483-0b55-4ead-b345-9467eda81e16-kube-api-access-6jhdh" (OuterVolumeSpecName: "kube-api-access-6jhdh") pod "a7680483-0b55-4ead-b345-9467eda81e16" (UID: "a7680483-0b55-4ead-b345-9467eda81e16"). InnerVolumeSpecName "kube-api-access-6jhdh". PluginName "kubernetes.io/projected", VolumeGIDValue "" Jul 03 11:31:59.207052 ip-10-0-136-0 kubenswrapper[2579]: I0703 11:31:59.207001 2579 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-6jhdh\" (UniqueName: \"kubernetes.io/projected/a7680483-0b55-4ead-b345-9467eda81e16-kube-api-access-6jhdh\") on node \"ip-10-0-136-0.ec2.internal\" DevicePath \"\"" Jul 03 11:31:59.207052 ip-10-0-136-0 kubenswrapper[2579]: I0703 11:31:59.207038 2579 reconciler_common.go:299] "Volume detached for volume \"extensions-socket-volume\" (UniqueName: \"kubernetes.io/empty-dir/a7680483-0b55-4ead-b345-9467eda81e16-extensions-socket-volume\") on node \"ip-10-0-136-0.ec2.internal\" DevicePath \"\"" Jul 03 11:31:59.886989 ip-10-0-136-0 kubenswrapper[2579]: I0703 11:31:59.886950 2579 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/kuadrant-operator-controller-manager-86878c769f-ppdg8" Jul 03 11:31:59.886989 ip-10-0-136-0 kubenswrapper[2579]: I0703 11:31:59.886960 2579 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/kuadrant-operator-controller-manager-86878c769f-ppdg8" event={"ID":"a7680483-0b55-4ead-b345-9467eda81e16","Type":"ContainerDied","Data":"6e55279c24f2af2ba050fa0706b11e76017d9dab490fab25dae9c732e5c5963b"} Jul 03 11:31:59.887526 ip-10-0-136-0 kubenswrapper[2579]: I0703 11:31:59.887057 2579 scope.go:117] "RemoveContainer" containerID="549207d79b521dd59998441b8c5f2efbb35cdc452e428c14eec0f1387d601d00" Jul 03 11:31:59.915504 ip-10-0-136-0 kubenswrapper[2579]: I0703 11:31:59.915471 2579 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["kuadrant-system/kuadrant-operator-controller-manager-86878c769f-ppdg8"] Jul 03 11:31:59.918065 ip-10-0-136-0 kubenswrapper[2579]: I0703 11:31:59.918031 2579 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["kuadrant-system/kuadrant-operator-controller-manager-86878c769f-ppdg8"] Jul 03 11:31:59.993208 ip-10-0-136-0 kubenswrapper[2579]: I0703 11:31:59.993155 2579 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="a7680483-0b55-4ead-b345-9467eda81e16" path="/var/lib/kubelet/pods/a7680483-0b55-4ead-b345-9467eda81e16/volumes" Jul 03 11:32:56.869053 ip-10-0-136-0 kubenswrapper[2579]: I0703 11:32:56.869002 2579 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["kuadrant-system/kuadrant-operator-controller-manager-86878c769f-dcjh8"] Jul 03 11:32:56.869627 ip-10-0-136-0 kubenswrapper[2579]: I0703 11:32:56.869466 2579 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="a7680483-0b55-4ead-b345-9467eda81e16" containerName="manager" Jul 03 11:32:56.869627 ip-10-0-136-0 kubenswrapper[2579]: I0703 11:32:56.869480 2579 state_mem.go:107] "Deleted CPUSet assignment" podUID="a7680483-0b55-4ead-b345-9467eda81e16" containerName="manager" Jul 03 11:32:56.869627 ip-10-0-136-0 kubenswrapper[2579]: I0703 11:32:56.869489 2579 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="c5324bdc-22d4-4bc3-a9fe-2f673ad11c58" containerName="manager" Jul 03 11:32:56.869627 ip-10-0-136-0 kubenswrapper[2579]: I0703 11:32:56.869496 2579 state_mem.go:107] "Deleted CPUSet assignment" podUID="c5324bdc-22d4-4bc3-a9fe-2f673ad11c58" containerName="manager" Jul 03 11:32:56.869627 ip-10-0-136-0 kubenswrapper[2579]: I0703 11:32:56.869512 2579 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="bd887377-3185-4dda-9a7e-b8c0d968b6ae" containerName="authorino" Jul 03 11:32:56.869627 ip-10-0-136-0 kubenswrapper[2579]: I0703 11:32:56.869519 2579 state_mem.go:107] "Deleted CPUSet assignment" podUID="bd887377-3185-4dda-9a7e-b8c0d968b6ae" containerName="authorino" Jul 03 11:32:56.869627 ip-10-0-136-0 kubenswrapper[2579]: I0703 11:32:56.869583 2579 memory_manager.go:368] "RemoveStaleState removing state" podUID="a7680483-0b55-4ead-b345-9467eda81e16" containerName="manager" Jul 03 11:32:56.869627 ip-10-0-136-0 kubenswrapper[2579]: I0703 11:32:56.869593 2579 memory_manager.go:368] "RemoveStaleState removing state" podUID="c5324bdc-22d4-4bc3-a9fe-2f673ad11c58" containerName="manager" Jul 03 11:32:56.869627 ip-10-0-136-0 kubenswrapper[2579]: I0703 11:32:56.869601 2579 memory_manager.go:368] "RemoveStaleState removing state" podUID="bd887377-3185-4dda-9a7e-b8c0d968b6ae" containerName="authorino" Jul 03 11:32:56.872853 ip-10-0-136-0 kubenswrapper[2579]: I0703 11:32:56.872825 2579 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/kuadrant-operator-controller-manager-86878c769f-dcjh8" Jul 03 11:32:56.877445 ip-10-0-136-0 kubenswrapper[2579]: I0703 11:32:56.877419 2579 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"kuadrant-system\"/\"kuadrant-operator-controller-manager-dockercfg-d8jfw\"" Jul 03 11:32:56.896058 ip-10-0-136-0 kubenswrapper[2579]: I0703 11:32:56.896017 2579 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/kuadrant-operator-controller-manager-86878c769f-dcjh8"] Jul 03 11:32:56.971315 ip-10-0-136-0 kubenswrapper[2579]: I0703 11:32:56.971266 2579 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-jpln8\" (UniqueName: \"kubernetes.io/projected/0bbde1f4-7dff-48af-92a8-26c3f6aea3c9-kube-api-access-jpln8\") pod \"kuadrant-operator-controller-manager-86878c769f-dcjh8\" (UID: \"0bbde1f4-7dff-48af-92a8-26c3f6aea3c9\") " pod="kuadrant-system/kuadrant-operator-controller-manager-86878c769f-dcjh8" Jul 03 11:32:56.971501 ip-10-0-136-0 kubenswrapper[2579]: I0703 11:32:56.971422 2579 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"extensions-socket-volume\" (UniqueName: \"kubernetes.io/empty-dir/0bbde1f4-7dff-48af-92a8-26c3f6aea3c9-extensions-socket-volume\") pod \"kuadrant-operator-controller-manager-86878c769f-dcjh8\" (UID: \"0bbde1f4-7dff-48af-92a8-26c3f6aea3c9\") " pod="kuadrant-system/kuadrant-operator-controller-manager-86878c769f-dcjh8" Jul 03 11:32:57.071930 ip-10-0-136-0 kubenswrapper[2579]: I0703 11:32:57.071878 2579 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"extensions-socket-volume\" (UniqueName: \"kubernetes.io/empty-dir/0bbde1f4-7dff-48af-92a8-26c3f6aea3c9-extensions-socket-volume\") pod \"kuadrant-operator-controller-manager-86878c769f-dcjh8\" (UID: \"0bbde1f4-7dff-48af-92a8-26c3f6aea3c9\") " pod="kuadrant-system/kuadrant-operator-controller-manager-86878c769f-dcjh8" Jul 03 11:32:57.072111 ip-10-0-136-0 kubenswrapper[2579]: I0703 11:32:57.071960 2579 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-jpln8\" (UniqueName: \"kubernetes.io/projected/0bbde1f4-7dff-48af-92a8-26c3f6aea3c9-kube-api-access-jpln8\") pod \"kuadrant-operator-controller-manager-86878c769f-dcjh8\" (UID: \"0bbde1f4-7dff-48af-92a8-26c3f6aea3c9\") " pod="kuadrant-system/kuadrant-operator-controller-manager-86878c769f-dcjh8" Jul 03 11:32:57.072367 ip-10-0-136-0 kubenswrapper[2579]: I0703 11:32:57.072344 2579 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"extensions-socket-volume\" (UniqueName: \"kubernetes.io/empty-dir/0bbde1f4-7dff-48af-92a8-26c3f6aea3c9-extensions-socket-volume\") pod \"kuadrant-operator-controller-manager-86878c769f-dcjh8\" (UID: \"0bbde1f4-7dff-48af-92a8-26c3f6aea3c9\") " pod="kuadrant-system/kuadrant-operator-controller-manager-86878c769f-dcjh8" Jul 03 11:32:57.091334 ip-10-0-136-0 kubenswrapper[2579]: I0703 11:32:57.091292 2579 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-jpln8\" (UniqueName: \"kubernetes.io/projected/0bbde1f4-7dff-48af-92a8-26c3f6aea3c9-kube-api-access-jpln8\") pod \"kuadrant-operator-controller-manager-86878c769f-dcjh8\" (UID: \"0bbde1f4-7dff-48af-92a8-26c3f6aea3c9\") " pod="kuadrant-system/kuadrant-operator-controller-manager-86878c769f-dcjh8" Jul 03 11:32:57.185290 ip-10-0-136-0 kubenswrapper[2579]: I0703 11:32:57.185129 2579 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/kuadrant-operator-controller-manager-86878c769f-dcjh8" Jul 03 11:32:57.372820 ip-10-0-136-0 kubenswrapper[2579]: I0703 11:32:57.372790 2579 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/kuadrant-operator-controller-manager-86878c769f-dcjh8"] Jul 03 11:32:57.375594 ip-10-0-136-0 kubenswrapper[2579]: W0703 11:32:57.375556 2579 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod0bbde1f4_7dff_48af_92a8_26c3f6aea3c9.slice/crio-dfd66d043009156184960ac47f346b2fd82f319a192d7716d5905e53ca36c74e WatchSource:0}: Error finding container dfd66d043009156184960ac47f346b2fd82f319a192d7716d5905e53ca36c74e: Status 404 returned error can't find the container with id dfd66d043009156184960ac47f346b2fd82f319a192d7716d5905e53ca36c74e Jul 03 11:32:58.241349 ip-10-0-136-0 kubenswrapper[2579]: I0703 11:32:58.241302 2579 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/kuadrant-operator-controller-manager-86878c769f-dcjh8" event={"ID":"0bbde1f4-7dff-48af-92a8-26c3f6aea3c9","Type":"ContainerStarted","Data":"425fda95dba0e155afb5746e60d95601f417bd95485731634fc1585efa42a77b"} Jul 03 11:32:58.241766 ip-10-0-136-0 kubenswrapper[2579]: I0703 11:32:58.241373 2579 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="kuadrant-system/kuadrant-operator-controller-manager-86878c769f-dcjh8" Jul 03 11:32:58.241766 ip-10-0-136-0 kubenswrapper[2579]: I0703 11:32:58.241399 2579 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/kuadrant-operator-controller-manager-86878c769f-dcjh8" event={"ID":"0bbde1f4-7dff-48af-92a8-26c3f6aea3c9","Type":"ContainerStarted","Data":"dfd66d043009156184960ac47f346b2fd82f319a192d7716d5905e53ca36c74e"} Jul 03 11:32:58.276672 ip-10-0-136-0 kubenswrapper[2579]: I0703 11:32:58.276612 2579 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="kuadrant-system/kuadrant-operator-controller-manager-86878c769f-dcjh8" podStartSLOduration=2.276592243 podStartE2EDuration="2.276592243s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-03 11:32:56 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-03 11:32:58.275554898 +0000 UTC m=+1234.863859217" watchObservedRunningTime="2026-07-03 11:32:58.276592243 +0000 UTC m=+1234.864896563" Jul 03 11:33:09.250169 ip-10-0-136-0 kubenswrapper[2579]: I0703 11:33:09.250081 2579 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="kuadrant-system/kuadrant-operator-controller-manager-86878c769f-dcjh8" Jul 03 11:39:18.994062 ip-10-0-136-0 kubenswrapper[2579]: I0703 11:39:18.994017 2579 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-ingress/e2e-ait-0715e8cc-openshift-default-79f6496779-svpzb"] Jul 03 11:39:19.000660 ip-10-0-136-0 kubenswrapper[2579]: I0703 11:39:19.000609 2579 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-ait-0715e8cc-openshift-default-79f6496779-svpzb" Jul 03 11:39:19.005608 ip-10-0-136-0 kubenswrapper[2579]: I0703 11:39:19.005577 2579 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress\"/\"e2e-ait-0715e8cc-openshift-default-dockercfg-9p8t2\"" Jul 03 11:39:19.031010 ip-10-0-136-0 kubenswrapper[2579]: I0703 11:39:19.030967 2579 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/abb6b8d7-f179-48d1-b9d9-e33cabc05901-credential-socket\") pod \"e2e-ait-0715e8cc-openshift-default-79f6496779-svpzb\" (UID: \"abb6b8d7-f179-48d1-b9d9-e33cabc05901\") " pod="openshift-ingress/e2e-ait-0715e8cc-openshift-default-79f6496779-svpzb" Jul 03 11:39:19.031247 ip-10-0-136-0 kubenswrapper[2579]: I0703 11:39:19.031029 2579 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/abb6b8d7-f179-48d1-b9d9-e33cabc05901-istio-token\") pod \"e2e-ait-0715e8cc-openshift-default-79f6496779-svpzb\" (UID: \"abb6b8d7-f179-48d1-b9d9-e33cabc05901\") " pod="openshift-ingress/e2e-ait-0715e8cc-openshift-default-79f6496779-svpzb" Jul 03 11:39:19.031247 ip-10-0-136-0 kubenswrapper[2579]: I0703 11:39:19.031085 2579 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/abb6b8d7-f179-48d1-b9d9-e33cabc05901-istio-envoy\") pod \"e2e-ait-0715e8cc-openshift-default-79f6496779-svpzb\" (UID: \"abb6b8d7-f179-48d1-b9d9-e33cabc05901\") " pod="openshift-ingress/e2e-ait-0715e8cc-openshift-default-79f6496779-svpzb" Jul 03 11:39:19.031247 ip-10-0-136-0 kubenswrapper[2579]: I0703 11:39:19.031112 2579 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/abb6b8d7-f179-48d1-b9d9-e33cabc05901-workload-socket\") pod \"e2e-ait-0715e8cc-openshift-default-79f6496779-svpzb\" (UID: \"abb6b8d7-f179-48d1-b9d9-e33cabc05901\") " pod="openshift-ingress/e2e-ait-0715e8cc-openshift-default-79f6496779-svpzb" Jul 03 11:39:19.031247 ip-10-0-136-0 kubenswrapper[2579]: I0703 11:39:19.031142 2579 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-4lkcc\" (UniqueName: \"kubernetes.io/projected/abb6b8d7-f179-48d1-b9d9-e33cabc05901-kube-api-access-4lkcc\") pod \"e2e-ait-0715e8cc-openshift-default-79f6496779-svpzb\" (UID: \"abb6b8d7-f179-48d1-b9d9-e33cabc05901\") " pod="openshift-ingress/e2e-ait-0715e8cc-openshift-default-79f6496779-svpzb" Jul 03 11:39:19.031247 ip-10-0-136-0 kubenswrapper[2579]: I0703 11:39:19.031173 2579 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/abb6b8d7-f179-48d1-b9d9-e33cabc05901-istio-data\") pod \"e2e-ait-0715e8cc-openshift-default-79f6496779-svpzb\" (UID: \"abb6b8d7-f179-48d1-b9d9-e33cabc05901\") " pod="openshift-ingress/e2e-ait-0715e8cc-openshift-default-79f6496779-svpzb" Jul 03 11:39:19.031543 ip-10-0-136-0 kubenswrapper[2579]: I0703 11:39:19.031327 2579 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/abb6b8d7-f179-48d1-b9d9-e33cabc05901-workload-certs\") pod \"e2e-ait-0715e8cc-openshift-default-79f6496779-svpzb\" (UID: \"abb6b8d7-f179-48d1-b9d9-e33cabc05901\") " pod="openshift-ingress/e2e-ait-0715e8cc-openshift-default-79f6496779-svpzb" Jul 03 11:39:19.031543 ip-10-0-136-0 kubenswrapper[2579]: I0703 11:39:19.031360 2579 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/abb6b8d7-f179-48d1-b9d9-e33cabc05901-istio-podinfo\") pod \"e2e-ait-0715e8cc-openshift-default-79f6496779-svpzb\" (UID: \"abb6b8d7-f179-48d1-b9d9-e33cabc05901\") " pod="openshift-ingress/e2e-ait-0715e8cc-openshift-default-79f6496779-svpzb" Jul 03 11:39:19.031543 ip-10-0-136-0 kubenswrapper[2579]: I0703 11:39:19.031389 2579 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/abb6b8d7-f179-48d1-b9d9-e33cabc05901-istiod-ca-cert\") pod \"e2e-ait-0715e8cc-openshift-default-79f6496779-svpzb\" (UID: \"abb6b8d7-f179-48d1-b9d9-e33cabc05901\") " pod="openshift-ingress/e2e-ait-0715e8cc-openshift-default-79f6496779-svpzb" Jul 03 11:39:19.122799 ip-10-0-136-0 kubenswrapper[2579]: I0703 11:39:19.122761 2579 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/e2e-ait-0715e8cc-openshift-default-79f6496779-svpzb"] Jul 03 11:39:19.132303 ip-10-0-136-0 kubenswrapper[2579]: I0703 11:39:19.132267 2579 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/abb6b8d7-f179-48d1-b9d9-e33cabc05901-istio-token\") pod \"e2e-ait-0715e8cc-openshift-default-79f6496779-svpzb\" (UID: \"abb6b8d7-f179-48d1-b9d9-e33cabc05901\") " pod="openshift-ingress/e2e-ait-0715e8cc-openshift-default-79f6496779-svpzb" Jul 03 11:39:19.132531 ip-10-0-136-0 kubenswrapper[2579]: I0703 11:39:19.132351 2579 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/abb6b8d7-f179-48d1-b9d9-e33cabc05901-istio-envoy\") pod \"e2e-ait-0715e8cc-openshift-default-79f6496779-svpzb\" (UID: \"abb6b8d7-f179-48d1-b9d9-e33cabc05901\") " pod="openshift-ingress/e2e-ait-0715e8cc-openshift-default-79f6496779-svpzb" Jul 03 11:39:19.132531 ip-10-0-136-0 kubenswrapper[2579]: I0703 11:39:19.132383 2579 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/abb6b8d7-f179-48d1-b9d9-e33cabc05901-workload-socket\") pod \"e2e-ait-0715e8cc-openshift-default-79f6496779-svpzb\" (UID: \"abb6b8d7-f179-48d1-b9d9-e33cabc05901\") " pod="openshift-ingress/e2e-ait-0715e8cc-openshift-default-79f6496779-svpzb" Jul 03 11:39:19.132531 ip-10-0-136-0 kubenswrapper[2579]: I0703 11:39:19.132417 2579 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-4lkcc\" (UniqueName: \"kubernetes.io/projected/abb6b8d7-f179-48d1-b9d9-e33cabc05901-kube-api-access-4lkcc\") pod \"e2e-ait-0715e8cc-openshift-default-79f6496779-svpzb\" (UID: \"abb6b8d7-f179-48d1-b9d9-e33cabc05901\") " pod="openshift-ingress/e2e-ait-0715e8cc-openshift-default-79f6496779-svpzb" Jul 03 11:39:19.132531 ip-10-0-136-0 kubenswrapper[2579]: I0703 11:39:19.132450 2579 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/abb6b8d7-f179-48d1-b9d9-e33cabc05901-istio-data\") pod \"e2e-ait-0715e8cc-openshift-default-79f6496779-svpzb\" (UID: \"abb6b8d7-f179-48d1-b9d9-e33cabc05901\") " pod="openshift-ingress/e2e-ait-0715e8cc-openshift-default-79f6496779-svpzb" Jul 03 11:39:19.132531 ip-10-0-136-0 kubenswrapper[2579]: I0703 11:39:19.132520 2579 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/abb6b8d7-f179-48d1-b9d9-e33cabc05901-workload-certs\") pod \"e2e-ait-0715e8cc-openshift-default-79f6496779-svpzb\" (UID: \"abb6b8d7-f179-48d1-b9d9-e33cabc05901\") " pod="openshift-ingress/e2e-ait-0715e8cc-openshift-default-79f6496779-svpzb" Jul 03 11:39:19.132794 ip-10-0-136-0 kubenswrapper[2579]: I0703 11:39:19.132550 2579 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/abb6b8d7-f179-48d1-b9d9-e33cabc05901-istio-podinfo\") pod \"e2e-ait-0715e8cc-openshift-default-79f6496779-svpzb\" (UID: \"abb6b8d7-f179-48d1-b9d9-e33cabc05901\") " pod="openshift-ingress/e2e-ait-0715e8cc-openshift-default-79f6496779-svpzb" Jul 03 11:39:19.132794 ip-10-0-136-0 kubenswrapper[2579]: I0703 11:39:19.132582 2579 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/abb6b8d7-f179-48d1-b9d9-e33cabc05901-istiod-ca-cert\") pod \"e2e-ait-0715e8cc-openshift-default-79f6496779-svpzb\" (UID: \"abb6b8d7-f179-48d1-b9d9-e33cabc05901\") " pod="openshift-ingress/e2e-ait-0715e8cc-openshift-default-79f6496779-svpzb" Jul 03 11:39:19.132794 ip-10-0-136-0 kubenswrapper[2579]: I0703 11:39:19.132631 2579 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/abb6b8d7-f179-48d1-b9d9-e33cabc05901-credential-socket\") pod \"e2e-ait-0715e8cc-openshift-default-79f6496779-svpzb\" (UID: \"abb6b8d7-f179-48d1-b9d9-e33cabc05901\") " pod="openshift-ingress/e2e-ait-0715e8cc-openshift-default-79f6496779-svpzb" Jul 03 11:39:19.133169 ip-10-0-136-0 kubenswrapper[2579]: I0703 11:39:19.133142 2579 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/abb6b8d7-f179-48d1-b9d9-e33cabc05901-credential-socket\") pod \"e2e-ait-0715e8cc-openshift-default-79f6496779-svpzb\" (UID: \"abb6b8d7-f179-48d1-b9d9-e33cabc05901\") " pod="openshift-ingress/e2e-ait-0715e8cc-openshift-default-79f6496779-svpzb" Jul 03 11:39:19.133355 ip-10-0-136-0 kubenswrapper[2579]: I0703 11:39:19.133170 2579 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/abb6b8d7-f179-48d1-b9d9-e33cabc05901-istio-data\") pod \"e2e-ait-0715e8cc-openshift-default-79f6496779-svpzb\" (UID: \"abb6b8d7-f179-48d1-b9d9-e33cabc05901\") " pod="openshift-ingress/e2e-ait-0715e8cc-openshift-default-79f6496779-svpzb" Jul 03 11:39:19.133454 ip-10-0-136-0 kubenswrapper[2579]: I0703 11:39:19.133254 2579 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/abb6b8d7-f179-48d1-b9d9-e33cabc05901-workload-socket\") pod \"e2e-ait-0715e8cc-openshift-default-79f6496779-svpzb\" (UID: \"abb6b8d7-f179-48d1-b9d9-e33cabc05901\") " pod="openshift-ingress/e2e-ait-0715e8cc-openshift-default-79f6496779-svpzb" Jul 03 11:39:19.133880 ip-10-0-136-0 kubenswrapper[2579]: I0703 11:39:19.133846 2579 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/abb6b8d7-f179-48d1-b9d9-e33cabc05901-workload-certs\") pod \"e2e-ait-0715e8cc-openshift-default-79f6496779-svpzb\" (UID: \"abb6b8d7-f179-48d1-b9d9-e33cabc05901\") " pod="openshift-ingress/e2e-ait-0715e8cc-openshift-default-79f6496779-svpzb" Jul 03 11:39:19.134571 ip-10-0-136-0 kubenswrapper[2579]: I0703 11:39:19.134546 2579 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/abb6b8d7-f179-48d1-b9d9-e33cabc05901-istiod-ca-cert\") pod \"e2e-ait-0715e8cc-openshift-default-79f6496779-svpzb\" (UID: \"abb6b8d7-f179-48d1-b9d9-e33cabc05901\") " pod="openshift-ingress/e2e-ait-0715e8cc-openshift-default-79f6496779-svpzb" Jul 03 11:39:19.136315 ip-10-0-136-0 kubenswrapper[2579]: I0703 11:39:19.136287 2579 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/abb6b8d7-f179-48d1-b9d9-e33cabc05901-istio-envoy\") pod \"e2e-ait-0715e8cc-openshift-default-79f6496779-svpzb\" (UID: \"abb6b8d7-f179-48d1-b9d9-e33cabc05901\") " pod="openshift-ingress/e2e-ait-0715e8cc-openshift-default-79f6496779-svpzb" Jul 03 11:39:19.136775 ip-10-0-136-0 kubenswrapper[2579]: I0703 11:39:19.136750 2579 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/abb6b8d7-f179-48d1-b9d9-e33cabc05901-istio-podinfo\") pod \"e2e-ait-0715e8cc-openshift-default-79f6496779-svpzb\" (UID: \"abb6b8d7-f179-48d1-b9d9-e33cabc05901\") " pod="openshift-ingress/e2e-ait-0715e8cc-openshift-default-79f6496779-svpzb" Jul 03 11:39:19.144120 ip-10-0-136-0 kubenswrapper[2579]: I0703 11:39:19.144070 2579 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-4lkcc\" (UniqueName: \"kubernetes.io/projected/abb6b8d7-f179-48d1-b9d9-e33cabc05901-kube-api-access-4lkcc\") pod \"e2e-ait-0715e8cc-openshift-default-79f6496779-svpzb\" (UID: \"abb6b8d7-f179-48d1-b9d9-e33cabc05901\") " pod="openshift-ingress/e2e-ait-0715e8cc-openshift-default-79f6496779-svpzb" Jul 03 11:39:19.144771 ip-10-0-136-0 kubenswrapper[2579]: I0703 11:39:19.144736 2579 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/abb6b8d7-f179-48d1-b9d9-e33cabc05901-istio-token\") pod \"e2e-ait-0715e8cc-openshift-default-79f6496779-svpzb\" (UID: \"abb6b8d7-f179-48d1-b9d9-e33cabc05901\") " pod="openshift-ingress/e2e-ait-0715e8cc-openshift-default-79f6496779-svpzb" Jul 03 11:39:19.315850 ip-10-0-136-0 kubenswrapper[2579]: I0703 11:39:19.315805 2579 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-ait-0715e8cc-openshift-default-79f6496779-svpzb" Jul 03 11:39:19.480794 ip-10-0-136-0 kubenswrapper[2579]: I0703 11:39:19.480701 2579 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/e2e-ait-0715e8cc-openshift-default-79f6496779-svpzb"] Jul 03 11:39:19.483869 ip-10-0-136-0 kubenswrapper[2579]: W0703 11:39:19.483830 2579 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podabb6b8d7_f179_48d1_b9d9_e33cabc05901.slice/crio-da7930fa95fa99f00606d67635c3389954f46ed8497fd064fa9868f4684ed609 WatchSource:0}: Error finding container da7930fa95fa99f00606d67635c3389954f46ed8497fd064fa9868f4684ed609: Status 404 returned error can't find the container with id da7930fa95fa99f00606d67635c3389954f46ed8497fd064fa9868f4684ed609 Jul 03 11:39:19.486326 ip-10-0-136-0 kubenswrapper[2579]: I0703 11:39:19.486285 2579 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"30892156Ki","pods":"250"} Jul 03 11:39:19.486433 ip-10-0-136-0 kubenswrapper[2579]: I0703 11:39:19.486366 2579 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"30892156Ki","pods":"250"} Jul 03 11:39:19.486433 ip-10-0-136-0 kubenswrapper[2579]: I0703 11:39:19.486396 2579 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"30892156Ki","pods":"250"} Jul 03 11:39:19.612028 ip-10-0-136-0 kubenswrapper[2579]: I0703 11:39:19.611921 2579 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-ait-0715e8cc-openshift-default-79f6496779-svpzb" event={"ID":"abb6b8d7-f179-48d1-b9d9-e33cabc05901","Type":"ContainerStarted","Data":"dea9522a24d2d5b56a87e0ea617157b45bc7895b4674411c9c0002992b47e790"} Jul 03 11:39:19.612028 ip-10-0-136-0 kubenswrapper[2579]: I0703 11:39:19.611990 2579 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-ait-0715e8cc-openshift-default-79f6496779-svpzb" event={"ID":"abb6b8d7-f179-48d1-b9d9-e33cabc05901","Type":"ContainerStarted","Data":"da7930fa95fa99f00606d67635c3389954f46ed8497fd064fa9868f4684ed609"} Jul 03 11:39:19.642578 ip-10-0-136-0 kubenswrapper[2579]: I0703 11:39:19.642514 2579 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-ingress/e2e-ait-0715e8cc-openshift-default-79f6496779-svpzb" podStartSLOduration=1.642491801 podStartE2EDuration="1.642491801s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-03 11:39:18 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-03 11:39:19.635350841 +0000 UTC m=+1616.223655161" watchObservedRunningTime="2026-07-03 11:39:19.642491801 +0000 UTC m=+1616.230796180" Jul 03 11:39:20.316689 ip-10-0-136-0 kubenswrapper[2579]: I0703 11:39:20.316648 2579 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="openshift-ingress/e2e-ait-0715e8cc-openshift-default-79f6496779-svpzb" Jul 03 11:39:20.322827 ip-10-0-136-0 kubenswrapper[2579]: I0703 11:39:20.322797 2579 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="openshift-ingress/e2e-ait-0715e8cc-openshift-default-79f6496779-svpzb" Jul 03 11:39:20.619009 ip-10-0-136-0 kubenswrapper[2579]: I0703 11:39:20.618913 2579 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/e2e-ait-0715e8cc-openshift-default-79f6496779-svpzb" Jul 03 11:39:20.620110 ip-10-0-136-0 kubenswrapper[2579]: I0703 11:39:20.620085 2579 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/e2e-ait-0715e8cc-openshift-default-79f6496779-svpzb" Jul 03 11:39:25.575500 ip-10-0-136-0 kubenswrapper[2579]: I0703 11:39:25.575451 2579 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-ingress/e2e-ait-0715e8cc-openshift-default-79f6496779-svpzb"] Jul 03 11:39:25.575936 ip-10-0-136-0 kubenswrapper[2579]: I0703 11:39:25.575671 2579 kuberuntime_container.go:872] "Killing container with a grace period" pod="openshift-ingress/e2e-ait-0715e8cc-openshift-default-79f6496779-svpzb" podUID="abb6b8d7-f179-48d1-b9d9-e33cabc05901" containerName="istio-proxy" containerID="cri-o://dea9522a24d2d5b56a87e0ea617157b45bc7895b4674411c9c0002992b47e790" gracePeriod=30 Jul 03 11:39:30.853835 ip-10-0-136-0 kubenswrapper[2579]: I0703 11:39:30.853801 2579 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-ait-0715e8cc-openshift-default-79f6496779-svpzb" Jul 03 11:39:30.954105 ip-10-0-136-0 kubenswrapper[2579]: I0703 11:39:30.953993 2579 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/abb6b8d7-f179-48d1-b9d9-e33cabc05901-istio-data\") pod \"abb6b8d7-f179-48d1-b9d9-e33cabc05901\" (UID: \"abb6b8d7-f179-48d1-b9d9-e33cabc05901\") " Jul 03 11:39:30.954105 ip-10-0-136-0 kubenswrapper[2579]: I0703 11:39:30.954074 2579 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/abb6b8d7-f179-48d1-b9d9-e33cabc05901-istio-podinfo\") pod \"abb6b8d7-f179-48d1-b9d9-e33cabc05901\" (UID: \"abb6b8d7-f179-48d1-b9d9-e33cabc05901\") " Jul 03 11:39:30.954105 ip-10-0-136-0 kubenswrapper[2579]: I0703 11:39:30.954104 2579 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/abb6b8d7-f179-48d1-b9d9-e33cabc05901-istio-token\") pod \"abb6b8d7-f179-48d1-b9d9-e33cabc05901\" (UID: \"abb6b8d7-f179-48d1-b9d9-e33cabc05901\") " Jul 03 11:39:30.954456 ip-10-0-136-0 kubenswrapper[2579]: I0703 11:39:30.954127 2579 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/abb6b8d7-f179-48d1-b9d9-e33cabc05901-credential-socket\") pod \"abb6b8d7-f179-48d1-b9d9-e33cabc05901\" (UID: \"abb6b8d7-f179-48d1-b9d9-e33cabc05901\") " Jul 03 11:39:30.954456 ip-10-0-136-0 kubenswrapper[2579]: I0703 11:39:30.954174 2579 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/abb6b8d7-f179-48d1-b9d9-e33cabc05901-istio-envoy\") pod \"abb6b8d7-f179-48d1-b9d9-e33cabc05901\" (UID: \"abb6b8d7-f179-48d1-b9d9-e33cabc05901\") " Jul 03 11:39:30.954562 ip-10-0-136-0 kubenswrapper[2579]: I0703 11:39:30.954443 2579 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/abb6b8d7-f179-48d1-b9d9-e33cabc05901-credential-socket" (OuterVolumeSpecName: "credential-socket") pod "abb6b8d7-f179-48d1-b9d9-e33cabc05901" (UID: "abb6b8d7-f179-48d1-b9d9-e33cabc05901"). InnerVolumeSpecName "credential-socket". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Jul 03 11:39:30.954562 ip-10-0-136-0 kubenswrapper[2579]: I0703 11:39:30.954481 2579 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-4lkcc\" (UniqueName: \"kubernetes.io/projected/abb6b8d7-f179-48d1-b9d9-e33cabc05901-kube-api-access-4lkcc\") pod \"abb6b8d7-f179-48d1-b9d9-e33cabc05901\" (UID: \"abb6b8d7-f179-48d1-b9d9-e33cabc05901\") " Jul 03 11:39:30.954562 ip-10-0-136-0 kubenswrapper[2579]: I0703 11:39:30.954544 2579 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/abb6b8d7-f179-48d1-b9d9-e33cabc05901-workload-socket\") pod \"abb6b8d7-f179-48d1-b9d9-e33cabc05901\" (UID: \"abb6b8d7-f179-48d1-b9d9-e33cabc05901\") " Jul 03 11:39:30.954727 ip-10-0-136-0 kubenswrapper[2579]: I0703 11:39:30.954567 2579 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/abb6b8d7-f179-48d1-b9d9-e33cabc05901-istiod-ca-cert\") pod \"abb6b8d7-f179-48d1-b9d9-e33cabc05901\" (UID: \"abb6b8d7-f179-48d1-b9d9-e33cabc05901\") " Jul 03 11:39:30.954727 ip-10-0-136-0 kubenswrapper[2579]: I0703 11:39:30.954599 2579 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/abb6b8d7-f179-48d1-b9d9-e33cabc05901-workload-certs\") pod \"abb6b8d7-f179-48d1-b9d9-e33cabc05901\" (UID: \"abb6b8d7-f179-48d1-b9d9-e33cabc05901\") " Jul 03 11:39:30.955315 ip-10-0-136-0 kubenswrapper[2579]: I0703 11:39:30.955016 2579 reconciler_common.go:299] "Volume detached for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/abb6b8d7-f179-48d1-b9d9-e33cabc05901-credential-socket\") on node \"ip-10-0-136-0.ec2.internal\" DevicePath \"\"" Jul 03 11:39:30.955315 ip-10-0-136-0 kubenswrapper[2579]: I0703 11:39:30.955056 2579 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/abb6b8d7-f179-48d1-b9d9-e33cabc05901-istiod-ca-cert" (OuterVolumeSpecName: "istiod-ca-cert") pod "abb6b8d7-f179-48d1-b9d9-e33cabc05901" (UID: "abb6b8d7-f179-48d1-b9d9-e33cabc05901"). InnerVolumeSpecName "istiod-ca-cert". PluginName "kubernetes.io/configmap", VolumeGIDValue "" Jul 03 11:39:30.955315 ip-10-0-136-0 kubenswrapper[2579]: I0703 11:39:30.955264 2579 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/abb6b8d7-f179-48d1-b9d9-e33cabc05901-workload-certs" (OuterVolumeSpecName: "workload-certs") pod "abb6b8d7-f179-48d1-b9d9-e33cabc05901" (UID: "abb6b8d7-f179-48d1-b9d9-e33cabc05901"). InnerVolumeSpecName "workload-certs". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Jul 03 11:39:30.955537 ip-10-0-136-0 kubenswrapper[2579]: I0703 11:39:30.955147 2579 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/abb6b8d7-f179-48d1-b9d9-e33cabc05901-workload-socket" (OuterVolumeSpecName: "workload-socket") pod "abb6b8d7-f179-48d1-b9d9-e33cabc05901" (UID: "abb6b8d7-f179-48d1-b9d9-e33cabc05901"). InnerVolumeSpecName "workload-socket". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Jul 03 11:39:30.955537 ip-10-0-136-0 kubenswrapper[2579]: I0703 11:39:30.955465 2579 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/abb6b8d7-f179-48d1-b9d9-e33cabc05901-istio-data" (OuterVolumeSpecName: "istio-data") pod "abb6b8d7-f179-48d1-b9d9-e33cabc05901" (UID: "abb6b8d7-f179-48d1-b9d9-e33cabc05901"). InnerVolumeSpecName "istio-data". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Jul 03 11:39:30.956936 ip-10-0-136-0 kubenswrapper[2579]: I0703 11:39:30.956910 2579 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/downward-api/abb6b8d7-f179-48d1-b9d9-e33cabc05901-istio-podinfo" (OuterVolumeSpecName: "istio-podinfo") pod "abb6b8d7-f179-48d1-b9d9-e33cabc05901" (UID: "abb6b8d7-f179-48d1-b9d9-e33cabc05901"). InnerVolumeSpecName "istio-podinfo". PluginName "kubernetes.io/downward-api", VolumeGIDValue "" Jul 03 11:39:30.957086 ip-10-0-136-0 kubenswrapper[2579]: I0703 11:39:30.957065 2579 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/abb6b8d7-f179-48d1-b9d9-e33cabc05901-istio-token" (OuterVolumeSpecName: "istio-token") pod "abb6b8d7-f179-48d1-b9d9-e33cabc05901" (UID: "abb6b8d7-f179-48d1-b9d9-e33cabc05901"). InnerVolumeSpecName "istio-token". PluginName "kubernetes.io/projected", VolumeGIDValue "" Jul 03 11:39:30.957162 ip-10-0-136-0 kubenswrapper[2579]: I0703 11:39:30.957147 2579 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/abb6b8d7-f179-48d1-b9d9-e33cabc05901-kube-api-access-4lkcc" (OuterVolumeSpecName: "kube-api-access-4lkcc") pod "abb6b8d7-f179-48d1-b9d9-e33cabc05901" (UID: "abb6b8d7-f179-48d1-b9d9-e33cabc05901"). InnerVolumeSpecName "kube-api-access-4lkcc". PluginName "kubernetes.io/projected", VolumeGIDValue "" Jul 03 11:39:30.957241 ip-10-0-136-0 kubenswrapper[2579]: I0703 11:39:30.957174 2579 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/abb6b8d7-f179-48d1-b9d9-e33cabc05901-istio-envoy" (OuterVolumeSpecName: "istio-envoy") pod "abb6b8d7-f179-48d1-b9d9-e33cabc05901" (UID: "abb6b8d7-f179-48d1-b9d9-e33cabc05901"). InnerVolumeSpecName "istio-envoy". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Jul 03 11:39:31.056437 ip-10-0-136-0 kubenswrapper[2579]: I0703 11:39:31.056397 2579 reconciler_common.go:299] "Volume detached for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/abb6b8d7-f179-48d1-b9d9-e33cabc05901-workload-certs\") on node \"ip-10-0-136-0.ec2.internal\" DevicePath \"\"" Jul 03 11:39:31.056437 ip-10-0-136-0 kubenswrapper[2579]: I0703 11:39:31.056429 2579 reconciler_common.go:299] "Volume detached for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/abb6b8d7-f179-48d1-b9d9-e33cabc05901-istio-data\") on node \"ip-10-0-136-0.ec2.internal\" DevicePath \"\"" Jul 03 11:39:31.056437 ip-10-0-136-0 kubenswrapper[2579]: I0703 11:39:31.056438 2579 reconciler_common.go:299] "Volume detached for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/abb6b8d7-f179-48d1-b9d9-e33cabc05901-istio-podinfo\") on node \"ip-10-0-136-0.ec2.internal\" DevicePath \"\"" Jul 03 11:39:31.056437 ip-10-0-136-0 kubenswrapper[2579]: I0703 11:39:31.056446 2579 reconciler_common.go:299] "Volume detached for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/abb6b8d7-f179-48d1-b9d9-e33cabc05901-istio-token\") on node \"ip-10-0-136-0.ec2.internal\" DevicePath \"\"" Jul 03 11:39:31.056437 ip-10-0-136-0 kubenswrapper[2579]: I0703 11:39:31.056454 2579 reconciler_common.go:299] "Volume detached for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/abb6b8d7-f179-48d1-b9d9-e33cabc05901-istio-envoy\") on node \"ip-10-0-136-0.ec2.internal\" DevicePath \"\"" Jul 03 11:39:31.056758 ip-10-0-136-0 kubenswrapper[2579]: I0703 11:39:31.056464 2579 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-4lkcc\" (UniqueName: \"kubernetes.io/projected/abb6b8d7-f179-48d1-b9d9-e33cabc05901-kube-api-access-4lkcc\") on node \"ip-10-0-136-0.ec2.internal\" DevicePath \"\"" Jul 03 11:39:31.056758 ip-10-0-136-0 kubenswrapper[2579]: I0703 11:39:31.056478 2579 reconciler_common.go:299] "Volume detached for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/abb6b8d7-f179-48d1-b9d9-e33cabc05901-workload-socket\") on node \"ip-10-0-136-0.ec2.internal\" DevicePath \"\"" Jul 03 11:39:31.056758 ip-10-0-136-0 kubenswrapper[2579]: I0703 11:39:31.056492 2579 reconciler_common.go:299] "Volume detached for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/abb6b8d7-f179-48d1-b9d9-e33cabc05901-istiod-ca-cert\") on node \"ip-10-0-136-0.ec2.internal\" DevicePath \"\"" Jul 03 11:39:31.697083 ip-10-0-136-0 kubenswrapper[2579]: I0703 11:39:31.697041 2579 generic.go:356] "Generic (PLEG): container finished" podID="abb6b8d7-f179-48d1-b9d9-e33cabc05901" containerID="dea9522a24d2d5b56a87e0ea617157b45bc7895b4674411c9c0002992b47e790" exitCode=0 Jul 03 11:39:31.697295 ip-10-0-136-0 kubenswrapper[2579]: I0703 11:39:31.697096 2579 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-ait-0715e8cc-openshift-default-79f6496779-svpzb" Jul 03 11:39:31.697295 ip-10-0-136-0 kubenswrapper[2579]: I0703 11:39:31.697126 2579 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-ait-0715e8cc-openshift-default-79f6496779-svpzb" event={"ID":"abb6b8d7-f179-48d1-b9d9-e33cabc05901","Type":"ContainerDied","Data":"dea9522a24d2d5b56a87e0ea617157b45bc7895b4674411c9c0002992b47e790"} Jul 03 11:39:31.697295 ip-10-0-136-0 kubenswrapper[2579]: I0703 11:39:31.697229 2579 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-ait-0715e8cc-openshift-default-79f6496779-svpzb" event={"ID":"abb6b8d7-f179-48d1-b9d9-e33cabc05901","Type":"ContainerDied","Data":"da7930fa95fa99f00606d67635c3389954f46ed8497fd064fa9868f4684ed609"} Jul 03 11:39:31.697295 ip-10-0-136-0 kubenswrapper[2579]: I0703 11:39:31.697263 2579 scope.go:117] "RemoveContainer" containerID="dea9522a24d2d5b56a87e0ea617157b45bc7895b4674411c9c0002992b47e790" Jul 03 11:39:31.707872 ip-10-0-136-0 kubenswrapper[2579]: I0703 11:39:31.707850 2579 scope.go:117] "RemoveContainer" containerID="dea9522a24d2d5b56a87e0ea617157b45bc7895b4674411c9c0002992b47e790" Jul 03 11:39:31.708242 ip-10-0-136-0 kubenswrapper[2579]: E0703 11:39:31.708217 2579 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"dea9522a24d2d5b56a87e0ea617157b45bc7895b4674411c9c0002992b47e790\": container with ID starting with dea9522a24d2d5b56a87e0ea617157b45bc7895b4674411c9c0002992b47e790 not found: ID does not exist" containerID="dea9522a24d2d5b56a87e0ea617157b45bc7895b4674411c9c0002992b47e790" Jul 03 11:39:31.708307 ip-10-0-136-0 kubenswrapper[2579]: I0703 11:39:31.708257 2579 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"dea9522a24d2d5b56a87e0ea617157b45bc7895b4674411c9c0002992b47e790"} err="failed to get container status \"dea9522a24d2d5b56a87e0ea617157b45bc7895b4674411c9c0002992b47e790\": rpc error: code = NotFound desc = could not find container \"dea9522a24d2d5b56a87e0ea617157b45bc7895b4674411c9c0002992b47e790\": container with ID starting with dea9522a24d2d5b56a87e0ea617157b45bc7895b4674411c9c0002992b47e790 not found: ID does not exist" Jul 03 11:39:31.723086 ip-10-0-136-0 kubenswrapper[2579]: I0703 11:39:31.723036 2579 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-ingress/e2e-ait-0715e8cc-openshift-default-79f6496779-svpzb"] Jul 03 11:39:31.732894 ip-10-0-136-0 kubenswrapper[2579]: I0703 11:39:31.732856 2579 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["openshift-ingress/e2e-ait-0715e8cc-openshift-default-79f6496779-svpzb"] Jul 03 11:39:31.913980 ip-10-0-136-0 kubenswrapper[2579]: I0703 11:39:31.913945 2579 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-ingress/e2e-ait-af3804fd-openshift-default-fdf7f96b7-62hrf"] Jul 03 11:39:31.914419 ip-10-0-136-0 kubenswrapper[2579]: I0703 11:39:31.914384 2579 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="abb6b8d7-f179-48d1-b9d9-e33cabc05901" containerName="istio-proxy" Jul 03 11:39:31.914419 ip-10-0-136-0 kubenswrapper[2579]: I0703 11:39:31.914398 2579 state_mem.go:107] "Deleted CPUSet assignment" podUID="abb6b8d7-f179-48d1-b9d9-e33cabc05901" containerName="istio-proxy" Jul 03 11:39:31.914498 ip-10-0-136-0 kubenswrapper[2579]: I0703 11:39:31.914485 2579 memory_manager.go:368] "RemoveStaleState removing state" podUID="abb6b8d7-f179-48d1-b9d9-e33cabc05901" containerName="istio-proxy" Jul 03 11:39:31.919857 ip-10-0-136-0 kubenswrapper[2579]: I0703 11:39:31.919831 2579 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-ait-af3804fd-openshift-default-fdf7f96b7-62hrf" Jul 03 11:39:31.922640 ip-10-0-136-0 kubenswrapper[2579]: I0703 11:39:31.922610 2579 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress\"/\"e2e-ait-af3804fd-openshift-default-dockercfg-25xwk\"" Jul 03 11:39:31.931243 ip-10-0-136-0 kubenswrapper[2579]: I0703 11:39:31.931154 2579 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/e2e-ait-af3804fd-openshift-default-fdf7f96b7-62hrf"] Jul 03 11:39:31.992717 ip-10-0-136-0 kubenswrapper[2579]: I0703 11:39:31.992623 2579 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="abb6b8d7-f179-48d1-b9d9-e33cabc05901" path="/var/lib/kubelet/pods/abb6b8d7-f179-48d1-b9d9-e33cabc05901/volumes" Jul 03 11:39:32.067260 ip-10-0-136-0 kubenswrapper[2579]: I0703 11:39:32.067160 2579 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/dab277ee-9da6-490f-ab7c-61e68e66476d-workload-socket\") pod \"e2e-ait-af3804fd-openshift-default-fdf7f96b7-62hrf\" (UID: \"dab277ee-9da6-490f-ab7c-61e68e66476d\") " pod="openshift-ingress/e2e-ait-af3804fd-openshift-default-fdf7f96b7-62hrf" Jul 03 11:39:32.067518 ip-10-0-136-0 kubenswrapper[2579]: I0703 11:39:32.067334 2579 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/dab277ee-9da6-490f-ab7c-61e68e66476d-istio-token\") pod \"e2e-ait-af3804fd-openshift-default-fdf7f96b7-62hrf\" (UID: \"dab277ee-9da6-490f-ab7c-61e68e66476d\") " pod="openshift-ingress/e2e-ait-af3804fd-openshift-default-fdf7f96b7-62hrf" Jul 03 11:39:32.067518 ip-10-0-136-0 kubenswrapper[2579]: I0703 11:39:32.067385 2579 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/dab277ee-9da6-490f-ab7c-61e68e66476d-istio-data\") pod \"e2e-ait-af3804fd-openshift-default-fdf7f96b7-62hrf\" (UID: \"dab277ee-9da6-490f-ab7c-61e68e66476d\") " pod="openshift-ingress/e2e-ait-af3804fd-openshift-default-fdf7f96b7-62hrf" Jul 03 11:39:32.067518 ip-10-0-136-0 kubenswrapper[2579]: I0703 11:39:32.067406 2579 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/dab277ee-9da6-490f-ab7c-61e68e66476d-istiod-ca-cert\") pod \"e2e-ait-af3804fd-openshift-default-fdf7f96b7-62hrf\" (UID: \"dab277ee-9da6-490f-ab7c-61e68e66476d\") " pod="openshift-ingress/e2e-ait-af3804fd-openshift-default-fdf7f96b7-62hrf" Jul 03 11:39:32.067689 ip-10-0-136-0 kubenswrapper[2579]: I0703 11:39:32.067530 2579 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/dab277ee-9da6-490f-ab7c-61e68e66476d-credential-socket\") pod \"e2e-ait-af3804fd-openshift-default-fdf7f96b7-62hrf\" (UID: \"dab277ee-9da6-490f-ab7c-61e68e66476d\") " pod="openshift-ingress/e2e-ait-af3804fd-openshift-default-fdf7f96b7-62hrf" Jul 03 11:39:32.067689 ip-10-0-136-0 kubenswrapper[2579]: I0703 11:39:32.067664 2579 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/dab277ee-9da6-490f-ab7c-61e68e66476d-istio-podinfo\") pod \"e2e-ait-af3804fd-openshift-default-fdf7f96b7-62hrf\" (UID: \"dab277ee-9da6-490f-ab7c-61e68e66476d\") " pod="openshift-ingress/e2e-ait-af3804fd-openshift-default-fdf7f96b7-62hrf" Jul 03 11:39:32.067790 ip-10-0-136-0 kubenswrapper[2579]: I0703 11:39:32.067707 2579 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-6qfdp\" (UniqueName: \"kubernetes.io/projected/dab277ee-9da6-490f-ab7c-61e68e66476d-kube-api-access-6qfdp\") pod \"e2e-ait-af3804fd-openshift-default-fdf7f96b7-62hrf\" (UID: \"dab277ee-9da6-490f-ab7c-61e68e66476d\") " pod="openshift-ingress/e2e-ait-af3804fd-openshift-default-fdf7f96b7-62hrf" Jul 03 11:39:32.067836 ip-10-0-136-0 kubenswrapper[2579]: I0703 11:39:32.067799 2579 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/dab277ee-9da6-490f-ab7c-61e68e66476d-istio-envoy\") pod \"e2e-ait-af3804fd-openshift-default-fdf7f96b7-62hrf\" (UID: \"dab277ee-9da6-490f-ab7c-61e68e66476d\") " pod="openshift-ingress/e2e-ait-af3804fd-openshift-default-fdf7f96b7-62hrf" Jul 03 11:39:32.067894 ip-10-0-136-0 kubenswrapper[2579]: I0703 11:39:32.067856 2579 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/dab277ee-9da6-490f-ab7c-61e68e66476d-workload-certs\") pod \"e2e-ait-af3804fd-openshift-default-fdf7f96b7-62hrf\" (UID: \"dab277ee-9da6-490f-ab7c-61e68e66476d\") " pod="openshift-ingress/e2e-ait-af3804fd-openshift-default-fdf7f96b7-62hrf" Jul 03 11:39:32.169022 ip-10-0-136-0 kubenswrapper[2579]: I0703 11:39:32.168976 2579 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/dab277ee-9da6-490f-ab7c-61e68e66476d-istio-envoy\") pod \"e2e-ait-af3804fd-openshift-default-fdf7f96b7-62hrf\" (UID: \"dab277ee-9da6-490f-ab7c-61e68e66476d\") " pod="openshift-ingress/e2e-ait-af3804fd-openshift-default-fdf7f96b7-62hrf" Jul 03 11:39:32.169277 ip-10-0-136-0 kubenswrapper[2579]: I0703 11:39:32.169039 2579 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/dab277ee-9da6-490f-ab7c-61e68e66476d-workload-certs\") pod \"e2e-ait-af3804fd-openshift-default-fdf7f96b7-62hrf\" (UID: \"dab277ee-9da6-490f-ab7c-61e68e66476d\") " pod="openshift-ingress/e2e-ait-af3804fd-openshift-default-fdf7f96b7-62hrf" Jul 03 11:39:32.169277 ip-10-0-136-0 kubenswrapper[2579]: I0703 11:39:32.169066 2579 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/dab277ee-9da6-490f-ab7c-61e68e66476d-workload-socket\") pod \"e2e-ait-af3804fd-openshift-default-fdf7f96b7-62hrf\" (UID: \"dab277ee-9da6-490f-ab7c-61e68e66476d\") " pod="openshift-ingress/e2e-ait-af3804fd-openshift-default-fdf7f96b7-62hrf" Jul 03 11:39:32.169277 ip-10-0-136-0 kubenswrapper[2579]: I0703 11:39:32.169098 2579 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/dab277ee-9da6-490f-ab7c-61e68e66476d-istio-token\") pod \"e2e-ait-af3804fd-openshift-default-fdf7f96b7-62hrf\" (UID: \"dab277ee-9da6-490f-ab7c-61e68e66476d\") " pod="openshift-ingress/e2e-ait-af3804fd-openshift-default-fdf7f96b7-62hrf" Jul 03 11:39:32.169277 ip-10-0-136-0 kubenswrapper[2579]: I0703 11:39:32.169116 2579 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/dab277ee-9da6-490f-ab7c-61e68e66476d-istio-data\") pod \"e2e-ait-af3804fd-openshift-default-fdf7f96b7-62hrf\" (UID: \"dab277ee-9da6-490f-ab7c-61e68e66476d\") " pod="openshift-ingress/e2e-ait-af3804fd-openshift-default-fdf7f96b7-62hrf" Jul 03 11:39:32.169277 ip-10-0-136-0 kubenswrapper[2579]: I0703 11:39:32.169134 2579 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/dab277ee-9da6-490f-ab7c-61e68e66476d-istiod-ca-cert\") pod \"e2e-ait-af3804fd-openshift-default-fdf7f96b7-62hrf\" (UID: \"dab277ee-9da6-490f-ab7c-61e68e66476d\") " pod="openshift-ingress/e2e-ait-af3804fd-openshift-default-fdf7f96b7-62hrf" Jul 03 11:39:32.169277 ip-10-0-136-0 kubenswrapper[2579]: I0703 11:39:32.169172 2579 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/dab277ee-9da6-490f-ab7c-61e68e66476d-credential-socket\") pod \"e2e-ait-af3804fd-openshift-default-fdf7f96b7-62hrf\" (UID: \"dab277ee-9da6-490f-ab7c-61e68e66476d\") " pod="openshift-ingress/e2e-ait-af3804fd-openshift-default-fdf7f96b7-62hrf" Jul 03 11:39:32.169277 ip-10-0-136-0 kubenswrapper[2579]: I0703 11:39:32.169253 2579 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/dab277ee-9da6-490f-ab7c-61e68e66476d-istio-podinfo\") pod \"e2e-ait-af3804fd-openshift-default-fdf7f96b7-62hrf\" (UID: \"dab277ee-9da6-490f-ab7c-61e68e66476d\") " pod="openshift-ingress/e2e-ait-af3804fd-openshift-default-fdf7f96b7-62hrf" Jul 03 11:39:32.169277 ip-10-0-136-0 kubenswrapper[2579]: I0703 11:39:32.169280 2579 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-6qfdp\" (UniqueName: \"kubernetes.io/projected/dab277ee-9da6-490f-ab7c-61e68e66476d-kube-api-access-6qfdp\") pod \"e2e-ait-af3804fd-openshift-default-fdf7f96b7-62hrf\" (UID: \"dab277ee-9da6-490f-ab7c-61e68e66476d\") " pod="openshift-ingress/e2e-ait-af3804fd-openshift-default-fdf7f96b7-62hrf" Jul 03 11:39:32.169702 ip-10-0-136-0 kubenswrapper[2579]: I0703 11:39:32.169595 2579 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/dab277ee-9da6-490f-ab7c-61e68e66476d-workload-certs\") pod \"e2e-ait-af3804fd-openshift-default-fdf7f96b7-62hrf\" (UID: \"dab277ee-9da6-490f-ab7c-61e68e66476d\") " pod="openshift-ingress/e2e-ait-af3804fd-openshift-default-fdf7f96b7-62hrf" Jul 03 11:39:32.169702 ip-10-0-136-0 kubenswrapper[2579]: I0703 11:39:32.169623 2579 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/dab277ee-9da6-490f-ab7c-61e68e66476d-workload-socket\") pod \"e2e-ait-af3804fd-openshift-default-fdf7f96b7-62hrf\" (UID: \"dab277ee-9da6-490f-ab7c-61e68e66476d\") " pod="openshift-ingress/e2e-ait-af3804fd-openshift-default-fdf7f96b7-62hrf" Jul 03 11:39:32.169779 ip-10-0-136-0 kubenswrapper[2579]: I0703 11:39:32.169742 2579 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/dab277ee-9da6-490f-ab7c-61e68e66476d-istio-data\") pod \"e2e-ait-af3804fd-openshift-default-fdf7f96b7-62hrf\" (UID: \"dab277ee-9da6-490f-ab7c-61e68e66476d\") " pod="openshift-ingress/e2e-ait-af3804fd-openshift-default-fdf7f96b7-62hrf" Jul 03 11:39:32.169999 ip-10-0-136-0 kubenswrapper[2579]: I0703 11:39:32.169971 2579 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/dab277ee-9da6-490f-ab7c-61e68e66476d-credential-socket\") pod \"e2e-ait-af3804fd-openshift-default-fdf7f96b7-62hrf\" (UID: \"dab277ee-9da6-490f-ab7c-61e68e66476d\") " pod="openshift-ingress/e2e-ait-af3804fd-openshift-default-fdf7f96b7-62hrf" Jul 03 11:39:32.170099 ip-10-0-136-0 kubenswrapper[2579]: I0703 11:39:32.170007 2579 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/dab277ee-9da6-490f-ab7c-61e68e66476d-istiod-ca-cert\") pod \"e2e-ait-af3804fd-openshift-default-fdf7f96b7-62hrf\" (UID: \"dab277ee-9da6-490f-ab7c-61e68e66476d\") " pod="openshift-ingress/e2e-ait-af3804fd-openshift-default-fdf7f96b7-62hrf" Jul 03 11:39:32.171521 ip-10-0-136-0 kubenswrapper[2579]: I0703 11:39:32.171500 2579 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/dab277ee-9da6-490f-ab7c-61e68e66476d-istio-envoy\") pod \"e2e-ait-af3804fd-openshift-default-fdf7f96b7-62hrf\" (UID: \"dab277ee-9da6-490f-ab7c-61e68e66476d\") " pod="openshift-ingress/e2e-ait-af3804fd-openshift-default-fdf7f96b7-62hrf" Jul 03 11:39:32.171923 ip-10-0-136-0 kubenswrapper[2579]: I0703 11:39:32.171904 2579 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/dab277ee-9da6-490f-ab7c-61e68e66476d-istio-podinfo\") pod \"e2e-ait-af3804fd-openshift-default-fdf7f96b7-62hrf\" (UID: \"dab277ee-9da6-490f-ab7c-61e68e66476d\") " pod="openshift-ingress/e2e-ait-af3804fd-openshift-default-fdf7f96b7-62hrf" Jul 03 11:39:32.181323 ip-10-0-136-0 kubenswrapper[2579]: I0703 11:39:32.181288 2579 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/dab277ee-9da6-490f-ab7c-61e68e66476d-istio-token\") pod \"e2e-ait-af3804fd-openshift-default-fdf7f96b7-62hrf\" (UID: \"dab277ee-9da6-490f-ab7c-61e68e66476d\") " pod="openshift-ingress/e2e-ait-af3804fd-openshift-default-fdf7f96b7-62hrf" Jul 03 11:39:32.181491 ip-10-0-136-0 kubenswrapper[2579]: I0703 11:39:32.181380 2579 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-6qfdp\" (UniqueName: \"kubernetes.io/projected/dab277ee-9da6-490f-ab7c-61e68e66476d-kube-api-access-6qfdp\") pod \"e2e-ait-af3804fd-openshift-default-fdf7f96b7-62hrf\" (UID: \"dab277ee-9da6-490f-ab7c-61e68e66476d\") " pod="openshift-ingress/e2e-ait-af3804fd-openshift-default-fdf7f96b7-62hrf" Jul 03 11:39:32.233261 ip-10-0-136-0 kubenswrapper[2579]: I0703 11:39:32.233205 2579 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-ait-af3804fd-openshift-default-fdf7f96b7-62hrf" Jul 03 11:39:32.416335 ip-10-0-136-0 kubenswrapper[2579]: I0703 11:39:32.416303 2579 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/e2e-ait-af3804fd-openshift-default-fdf7f96b7-62hrf"] Jul 03 11:39:32.418949 ip-10-0-136-0 kubenswrapper[2579]: W0703 11:39:32.418906 2579 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-poddab277ee_9da6_490f_ab7c_61e68e66476d.slice/crio-c937b2375bbd818353832174599ad081c33161dbe15bdd9df2b9988391175893 WatchSource:0}: Error finding container c937b2375bbd818353832174599ad081c33161dbe15bdd9df2b9988391175893: Status 404 returned error can't find the container with id c937b2375bbd818353832174599ad081c33161dbe15bdd9df2b9988391175893 Jul 03 11:39:32.421176 ip-10-0-136-0 kubenswrapper[2579]: I0703 11:39:32.421137 2579 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"30892156Ki","pods":"250"} Jul 03 11:39:32.421337 ip-10-0-136-0 kubenswrapper[2579]: I0703 11:39:32.421247 2579 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"30892156Ki","pods":"250"} Jul 03 11:39:32.421337 ip-10-0-136-0 kubenswrapper[2579]: I0703 11:39:32.421294 2579 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"30892156Ki","pods":"250"} Jul 03 11:39:32.705893 ip-10-0-136-0 kubenswrapper[2579]: I0703 11:39:32.705850 2579 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-ait-af3804fd-openshift-default-fdf7f96b7-62hrf" event={"ID":"dab277ee-9da6-490f-ab7c-61e68e66476d","Type":"ContainerStarted","Data":"e93f82a825e3bff435c432744293628379705935a9fc3dc76d7269d8d91d9980"} Jul 03 11:39:32.706117 ip-10-0-136-0 kubenswrapper[2579]: I0703 11:39:32.705918 2579 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-ait-af3804fd-openshift-default-fdf7f96b7-62hrf" event={"ID":"dab277ee-9da6-490f-ab7c-61e68e66476d","Type":"ContainerStarted","Data":"c937b2375bbd818353832174599ad081c33161dbe15bdd9df2b9988391175893"} Jul 03 11:39:32.730331 ip-10-0-136-0 kubenswrapper[2579]: I0703 11:39:32.730271 2579 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-ingress/e2e-ait-af3804fd-openshift-default-fdf7f96b7-62hrf" podStartSLOduration=1.730250555 podStartE2EDuration="1.730250555s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-03 11:39:31 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-03 11:39:32.728128376 +0000 UTC m=+1629.316432697" watchObservedRunningTime="2026-07-03 11:39:32.730250555 +0000 UTC m=+1629.318554874" Jul 03 11:39:33.234287 ip-10-0-136-0 kubenswrapper[2579]: I0703 11:39:33.234210 2579 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="openshift-ingress/e2e-ait-af3804fd-openshift-default-fdf7f96b7-62hrf" Jul 03 11:39:33.242239 ip-10-0-136-0 kubenswrapper[2579]: I0703 11:39:33.242168 2579 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="openshift-ingress/e2e-ait-af3804fd-openshift-default-fdf7f96b7-62hrf" Jul 03 11:39:33.714426 ip-10-0-136-0 kubenswrapper[2579]: I0703 11:39:33.714384 2579 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/e2e-ait-af3804fd-openshift-default-fdf7f96b7-62hrf" Jul 03 11:39:33.715894 ip-10-0-136-0 kubenswrapper[2579]: I0703 11:39:33.715858 2579 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/e2e-ait-af3804fd-openshift-default-fdf7f96b7-62hrf" Jul 03 11:39:44.594629 ip-10-0-136-0 kubenswrapper[2579]: I0703 11:39:44.594576 2579 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-ingress/e2e-ait-af3804fd-openshift-default-fdf7f96b7-62hrf"] Jul 03 11:39:44.595231 ip-10-0-136-0 kubenswrapper[2579]: I0703 11:39:44.594829 2579 kuberuntime_container.go:872] "Killing container with a grace period" pod="openshift-ingress/e2e-ait-af3804fd-openshift-default-fdf7f96b7-62hrf" podUID="dab277ee-9da6-490f-ab7c-61e68e66476d" containerName="istio-proxy" containerID="cri-o://e93f82a825e3bff435c432744293628379705935a9fc3dc76d7269d8d91d9980" gracePeriod=30 Jul 03 11:39:46.266269 ip-10-0-136-0 kubenswrapper[2579]: I0703 11:39:46.266233 2579 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-ait-af3804fd-openshift-default-fdf7f96b7-62hrf" Jul 03 11:39:46.339504 ip-10-0-136-0 kubenswrapper[2579]: I0703 11:39:46.339464 2579 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/dab277ee-9da6-490f-ab7c-61e68e66476d-workload-socket\") pod \"dab277ee-9da6-490f-ab7c-61e68e66476d\" (UID: \"dab277ee-9da6-490f-ab7c-61e68e66476d\") " Jul 03 11:39:46.339732 ip-10-0-136-0 kubenswrapper[2579]: I0703 11:39:46.339556 2579 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/dab277ee-9da6-490f-ab7c-61e68e66476d-istio-data\") pod \"dab277ee-9da6-490f-ab7c-61e68e66476d\" (UID: \"dab277ee-9da6-490f-ab7c-61e68e66476d\") " Jul 03 11:39:46.339732 ip-10-0-136-0 kubenswrapper[2579]: I0703 11:39:46.339590 2579 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/dab277ee-9da6-490f-ab7c-61e68e66476d-credential-socket\") pod \"dab277ee-9da6-490f-ab7c-61e68e66476d\" (UID: \"dab277ee-9da6-490f-ab7c-61e68e66476d\") " Jul 03 11:39:46.339732 ip-10-0-136-0 kubenswrapper[2579]: I0703 11:39:46.339616 2579 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/dab277ee-9da6-490f-ab7c-61e68e66476d-istio-podinfo\") pod \"dab277ee-9da6-490f-ab7c-61e68e66476d\" (UID: \"dab277ee-9da6-490f-ab7c-61e68e66476d\") " Jul 03 11:39:46.339732 ip-10-0-136-0 kubenswrapper[2579]: I0703 11:39:46.339643 2579 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/dab277ee-9da6-490f-ab7c-61e68e66476d-workload-certs\") pod \"dab277ee-9da6-490f-ab7c-61e68e66476d\" (UID: \"dab277ee-9da6-490f-ab7c-61e68e66476d\") " Jul 03 11:39:46.339732 ip-10-0-136-0 kubenswrapper[2579]: I0703 11:39:46.339683 2579 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-6qfdp\" (UniqueName: \"kubernetes.io/projected/dab277ee-9da6-490f-ab7c-61e68e66476d-kube-api-access-6qfdp\") pod \"dab277ee-9da6-490f-ab7c-61e68e66476d\" (UID: \"dab277ee-9da6-490f-ab7c-61e68e66476d\") " Jul 03 11:39:46.339732 ip-10-0-136-0 kubenswrapper[2579]: I0703 11:39:46.339718 2579 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/dab277ee-9da6-490f-ab7c-61e68e66476d-istio-token\") pod \"dab277ee-9da6-490f-ab7c-61e68e66476d\" (UID: \"dab277ee-9da6-490f-ab7c-61e68e66476d\") " Jul 03 11:39:46.340065 ip-10-0-136-0 kubenswrapper[2579]: I0703 11:39:46.339768 2579 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/dab277ee-9da6-490f-ab7c-61e68e66476d-istiod-ca-cert\") pod \"dab277ee-9da6-490f-ab7c-61e68e66476d\" (UID: \"dab277ee-9da6-490f-ab7c-61e68e66476d\") " Jul 03 11:39:46.340065 ip-10-0-136-0 kubenswrapper[2579]: I0703 11:39:46.339859 2579 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/dab277ee-9da6-490f-ab7c-61e68e66476d-istio-envoy\") pod \"dab277ee-9da6-490f-ab7c-61e68e66476d\" (UID: \"dab277ee-9da6-490f-ab7c-61e68e66476d\") " Jul 03 11:39:46.340166 ip-10-0-136-0 kubenswrapper[2579]: I0703 11:39:46.340113 2579 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/dab277ee-9da6-490f-ab7c-61e68e66476d-workload-certs" (OuterVolumeSpecName: "workload-certs") pod "dab277ee-9da6-490f-ab7c-61e68e66476d" (UID: "dab277ee-9da6-490f-ab7c-61e68e66476d"). InnerVolumeSpecName "workload-certs". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Jul 03 11:39:46.340278 ip-10-0-136-0 kubenswrapper[2579]: I0703 11:39:46.340252 2579 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/dab277ee-9da6-490f-ab7c-61e68e66476d-workload-socket" (OuterVolumeSpecName: "workload-socket") pod "dab277ee-9da6-490f-ab7c-61e68e66476d" (UID: "dab277ee-9da6-490f-ab7c-61e68e66476d"). InnerVolumeSpecName "workload-socket". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Jul 03 11:39:46.340344 ip-10-0-136-0 kubenswrapper[2579]: I0703 11:39:46.340287 2579 reconciler_common.go:299] "Volume detached for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/dab277ee-9da6-490f-ab7c-61e68e66476d-workload-certs\") on node \"ip-10-0-136-0.ec2.internal\" DevicePath \"\"" Jul 03 11:39:46.340403 ip-10-0-136-0 kubenswrapper[2579]: I0703 11:39:46.340342 2579 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/dab277ee-9da6-490f-ab7c-61e68e66476d-credential-socket" (OuterVolumeSpecName: "credential-socket") pod "dab277ee-9da6-490f-ab7c-61e68e66476d" (UID: "dab277ee-9da6-490f-ab7c-61e68e66476d"). InnerVolumeSpecName "credential-socket". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Jul 03 11:39:46.340460 ip-10-0-136-0 kubenswrapper[2579]: I0703 11:39:46.340434 2579 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/dab277ee-9da6-490f-ab7c-61e68e66476d-istiod-ca-cert" (OuterVolumeSpecName: "istiod-ca-cert") pod "dab277ee-9da6-490f-ab7c-61e68e66476d" (UID: "dab277ee-9da6-490f-ab7c-61e68e66476d"). InnerVolumeSpecName "istiod-ca-cert". PluginName "kubernetes.io/configmap", VolumeGIDValue "" Jul 03 11:39:46.340573 ip-10-0-136-0 kubenswrapper[2579]: I0703 11:39:46.340542 2579 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/dab277ee-9da6-490f-ab7c-61e68e66476d-istio-data" (OuterVolumeSpecName: "istio-data") pod "dab277ee-9da6-490f-ab7c-61e68e66476d" (UID: "dab277ee-9da6-490f-ab7c-61e68e66476d"). InnerVolumeSpecName "istio-data". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Jul 03 11:39:46.343523 ip-10-0-136-0 kubenswrapper[2579]: I0703 11:39:46.343484 2579 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/downward-api/dab277ee-9da6-490f-ab7c-61e68e66476d-istio-podinfo" (OuterVolumeSpecName: "istio-podinfo") pod "dab277ee-9da6-490f-ab7c-61e68e66476d" (UID: "dab277ee-9da6-490f-ab7c-61e68e66476d"). InnerVolumeSpecName "istio-podinfo". PluginName "kubernetes.io/downward-api", VolumeGIDValue "" Jul 03 11:39:46.343523 ip-10-0-136-0 kubenswrapper[2579]: I0703 11:39:46.343501 2579 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/dab277ee-9da6-490f-ab7c-61e68e66476d-kube-api-access-6qfdp" (OuterVolumeSpecName: "kube-api-access-6qfdp") pod "dab277ee-9da6-490f-ab7c-61e68e66476d" (UID: "dab277ee-9da6-490f-ab7c-61e68e66476d"). InnerVolumeSpecName "kube-api-access-6qfdp". PluginName "kubernetes.io/projected", VolumeGIDValue "" Jul 03 11:39:46.343523 ip-10-0-136-0 kubenswrapper[2579]: I0703 11:39:46.343516 2579 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/dab277ee-9da6-490f-ab7c-61e68e66476d-istio-envoy" (OuterVolumeSpecName: "istio-envoy") pod "dab277ee-9da6-490f-ab7c-61e68e66476d" (UID: "dab277ee-9da6-490f-ab7c-61e68e66476d"). InnerVolumeSpecName "istio-envoy". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Jul 03 11:39:46.343761 ip-10-0-136-0 kubenswrapper[2579]: I0703 11:39:46.343627 2579 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/dab277ee-9da6-490f-ab7c-61e68e66476d-istio-token" (OuterVolumeSpecName: "istio-token") pod "dab277ee-9da6-490f-ab7c-61e68e66476d" (UID: "dab277ee-9da6-490f-ab7c-61e68e66476d"). InnerVolumeSpecName "istio-token". PluginName "kubernetes.io/projected", VolumeGIDValue "" Jul 03 11:39:46.441516 ip-10-0-136-0 kubenswrapper[2579]: I0703 11:39:46.441474 2579 reconciler_common.go:299] "Volume detached for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/dab277ee-9da6-490f-ab7c-61e68e66476d-istiod-ca-cert\") on node \"ip-10-0-136-0.ec2.internal\" DevicePath \"\"" Jul 03 11:39:46.441516 ip-10-0-136-0 kubenswrapper[2579]: I0703 11:39:46.441509 2579 reconciler_common.go:299] "Volume detached for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/dab277ee-9da6-490f-ab7c-61e68e66476d-istio-envoy\") on node \"ip-10-0-136-0.ec2.internal\" DevicePath \"\"" Jul 03 11:39:46.441516 ip-10-0-136-0 kubenswrapper[2579]: I0703 11:39:46.441517 2579 reconciler_common.go:299] "Volume detached for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/dab277ee-9da6-490f-ab7c-61e68e66476d-workload-socket\") on node \"ip-10-0-136-0.ec2.internal\" DevicePath \"\"" Jul 03 11:39:46.441516 ip-10-0-136-0 kubenswrapper[2579]: I0703 11:39:46.441526 2579 reconciler_common.go:299] "Volume detached for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/dab277ee-9da6-490f-ab7c-61e68e66476d-istio-data\") on node \"ip-10-0-136-0.ec2.internal\" DevicePath \"\"" Jul 03 11:39:46.441802 ip-10-0-136-0 kubenswrapper[2579]: I0703 11:39:46.441534 2579 reconciler_common.go:299] "Volume detached for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/dab277ee-9da6-490f-ab7c-61e68e66476d-credential-socket\") on node \"ip-10-0-136-0.ec2.internal\" DevicePath \"\"" Jul 03 11:39:46.441802 ip-10-0-136-0 kubenswrapper[2579]: I0703 11:39:46.441545 2579 reconciler_common.go:299] "Volume detached for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/dab277ee-9da6-490f-ab7c-61e68e66476d-istio-podinfo\") on node \"ip-10-0-136-0.ec2.internal\" DevicePath \"\"" Jul 03 11:39:46.441802 ip-10-0-136-0 kubenswrapper[2579]: I0703 11:39:46.441554 2579 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-6qfdp\" (UniqueName: \"kubernetes.io/projected/dab277ee-9da6-490f-ab7c-61e68e66476d-kube-api-access-6qfdp\") on node \"ip-10-0-136-0.ec2.internal\" DevicePath \"\"" Jul 03 11:39:46.441802 ip-10-0-136-0 kubenswrapper[2579]: I0703 11:39:46.441563 2579 reconciler_common.go:299] "Volume detached for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/dab277ee-9da6-490f-ab7c-61e68e66476d-istio-token\") on node \"ip-10-0-136-0.ec2.internal\" DevicePath \"\"" Jul 03 11:39:46.800475 ip-10-0-136-0 kubenswrapper[2579]: I0703 11:39:46.800441 2579 generic.go:356] "Generic (PLEG): container finished" podID="dab277ee-9da6-490f-ab7c-61e68e66476d" containerID="e93f82a825e3bff435c432744293628379705935a9fc3dc76d7269d8d91d9980" exitCode=0 Jul 03 11:39:46.800687 ip-10-0-136-0 kubenswrapper[2579]: I0703 11:39:46.800491 2579 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-ait-af3804fd-openshift-default-fdf7f96b7-62hrf" Jul 03 11:39:46.800687 ip-10-0-136-0 kubenswrapper[2579]: I0703 11:39:46.800529 2579 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-ait-af3804fd-openshift-default-fdf7f96b7-62hrf" event={"ID":"dab277ee-9da6-490f-ab7c-61e68e66476d","Type":"ContainerDied","Data":"e93f82a825e3bff435c432744293628379705935a9fc3dc76d7269d8d91d9980"} Jul 03 11:39:46.800687 ip-10-0-136-0 kubenswrapper[2579]: I0703 11:39:46.800597 2579 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-ait-af3804fd-openshift-default-fdf7f96b7-62hrf" event={"ID":"dab277ee-9da6-490f-ab7c-61e68e66476d","Type":"ContainerDied","Data":"c937b2375bbd818353832174599ad081c33161dbe15bdd9df2b9988391175893"} Jul 03 11:39:46.800687 ip-10-0-136-0 kubenswrapper[2579]: I0703 11:39:46.800635 2579 scope.go:117] "RemoveContainer" containerID="e93f82a825e3bff435c432744293628379705935a9fc3dc76d7269d8d91d9980" Jul 03 11:39:46.812264 ip-10-0-136-0 kubenswrapper[2579]: I0703 11:39:46.812240 2579 scope.go:117] "RemoveContainer" containerID="e93f82a825e3bff435c432744293628379705935a9fc3dc76d7269d8d91d9980" Jul 03 11:39:46.812638 ip-10-0-136-0 kubenswrapper[2579]: E0703 11:39:46.812618 2579 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"e93f82a825e3bff435c432744293628379705935a9fc3dc76d7269d8d91d9980\": container with ID starting with e93f82a825e3bff435c432744293628379705935a9fc3dc76d7269d8d91d9980 not found: ID does not exist" containerID="e93f82a825e3bff435c432744293628379705935a9fc3dc76d7269d8d91d9980" Jul 03 11:39:46.812698 ip-10-0-136-0 kubenswrapper[2579]: I0703 11:39:46.812647 2579 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"e93f82a825e3bff435c432744293628379705935a9fc3dc76d7269d8d91d9980"} err="failed to get container status \"e93f82a825e3bff435c432744293628379705935a9fc3dc76d7269d8d91d9980\": rpc error: code = NotFound desc = could not find container \"e93f82a825e3bff435c432744293628379705935a9fc3dc76d7269d8d91d9980\": container with ID starting with e93f82a825e3bff435c432744293628379705935a9fc3dc76d7269d8d91d9980 not found: ID does not exist" Jul 03 11:39:46.827039 ip-10-0-136-0 kubenswrapper[2579]: I0703 11:39:46.827002 2579 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-ingress/e2e-ait-af3804fd-openshift-default-fdf7f96b7-62hrf"] Jul 03 11:39:46.832442 ip-10-0-136-0 kubenswrapper[2579]: I0703 11:39:46.832398 2579 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["openshift-ingress/e2e-ait-af3804fd-openshift-default-fdf7f96b7-62hrf"] Jul 03 11:39:47.994052 ip-10-0-136-0 kubenswrapper[2579]: I0703 11:39:47.994000 2579 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="dab277ee-9da6-490f-ab7c-61e68e66476d" path="/var/lib/kubelet/pods/dab277ee-9da6-490f-ab7c-61e68e66476d/volumes" Jul 03 11:39:51.472032 ip-10-0-136-0 kubenswrapper[2579]: I0703 11:39:51.471996 2579 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-ingress/e2e-derive-658af5a0-openshift-default-6cb45b6dc5-fwwrm"] Jul 03 11:39:51.472557 ip-10-0-136-0 kubenswrapper[2579]: I0703 11:39:51.472465 2579 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="dab277ee-9da6-490f-ab7c-61e68e66476d" containerName="istio-proxy" Jul 03 11:39:51.472557 ip-10-0-136-0 kubenswrapper[2579]: I0703 11:39:51.472479 2579 state_mem.go:107] "Deleted CPUSet assignment" podUID="dab277ee-9da6-490f-ab7c-61e68e66476d" containerName="istio-proxy" Jul 03 11:39:51.472860 ip-10-0-136-0 kubenswrapper[2579]: I0703 11:39:51.472563 2579 memory_manager.go:368] "RemoveStaleState removing state" podUID="dab277ee-9da6-490f-ab7c-61e68e66476d" containerName="istio-proxy" Jul 03 11:39:51.475710 ip-10-0-136-0 kubenswrapper[2579]: I0703 11:39:51.475686 2579 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-derive-658af5a0-openshift-default-6cb45b6dc5-fwwrm" Jul 03 11:39:51.483345 ip-10-0-136-0 kubenswrapper[2579]: I0703 11:39:51.483290 2579 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress\"/\"e2e-derive-658af5a0-openshift-default-dockercfg-v79jh\"" Jul 03 11:39:51.498872 ip-10-0-136-0 kubenswrapper[2579]: I0703 11:39:51.498826 2579 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/e2e-derive-658af5a0-openshift-default-6cb45b6dc5-fwwrm"] Jul 03 11:39:51.596624 ip-10-0-136-0 kubenswrapper[2579]: I0703 11:39:51.596589 2579 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/de29e5f0-022c-4c52-83e3-cc86ef01d137-istio-data\") pod \"e2e-derive-658af5a0-openshift-default-6cb45b6dc5-fwwrm\" (UID: \"de29e5f0-022c-4c52-83e3-cc86ef01d137\") " pod="openshift-ingress/e2e-derive-658af5a0-openshift-default-6cb45b6dc5-fwwrm" Jul 03 11:39:51.596866 ip-10-0-136-0 kubenswrapper[2579]: I0703 11:39:51.596648 2579 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/de29e5f0-022c-4c52-83e3-cc86ef01d137-workload-certs\") pod \"e2e-derive-658af5a0-openshift-default-6cb45b6dc5-fwwrm\" (UID: \"de29e5f0-022c-4c52-83e3-cc86ef01d137\") " pod="openshift-ingress/e2e-derive-658af5a0-openshift-default-6cb45b6dc5-fwwrm" Jul 03 11:39:51.596866 ip-10-0-136-0 kubenswrapper[2579]: I0703 11:39:51.596697 2579 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-g4nrs\" (UniqueName: \"kubernetes.io/projected/de29e5f0-022c-4c52-83e3-cc86ef01d137-kube-api-access-g4nrs\") pod \"e2e-derive-658af5a0-openshift-default-6cb45b6dc5-fwwrm\" (UID: \"de29e5f0-022c-4c52-83e3-cc86ef01d137\") " pod="openshift-ingress/e2e-derive-658af5a0-openshift-default-6cb45b6dc5-fwwrm" Jul 03 11:39:51.596866 ip-10-0-136-0 kubenswrapper[2579]: I0703 11:39:51.596802 2579 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/de29e5f0-022c-4c52-83e3-cc86ef01d137-istio-podinfo\") pod \"e2e-derive-658af5a0-openshift-default-6cb45b6dc5-fwwrm\" (UID: \"de29e5f0-022c-4c52-83e3-cc86ef01d137\") " pod="openshift-ingress/e2e-derive-658af5a0-openshift-default-6cb45b6dc5-fwwrm" Jul 03 11:39:51.596866 ip-10-0-136-0 kubenswrapper[2579]: I0703 11:39:51.596847 2579 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/de29e5f0-022c-4c52-83e3-cc86ef01d137-istio-token\") pod \"e2e-derive-658af5a0-openshift-default-6cb45b6dc5-fwwrm\" (UID: \"de29e5f0-022c-4c52-83e3-cc86ef01d137\") " pod="openshift-ingress/e2e-derive-658af5a0-openshift-default-6cb45b6dc5-fwwrm" Jul 03 11:39:51.597010 ip-10-0-136-0 kubenswrapper[2579]: I0703 11:39:51.596873 2579 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/de29e5f0-022c-4c52-83e3-cc86ef01d137-workload-socket\") pod \"e2e-derive-658af5a0-openshift-default-6cb45b6dc5-fwwrm\" (UID: \"de29e5f0-022c-4c52-83e3-cc86ef01d137\") " pod="openshift-ingress/e2e-derive-658af5a0-openshift-default-6cb45b6dc5-fwwrm" Jul 03 11:39:51.597010 ip-10-0-136-0 kubenswrapper[2579]: I0703 11:39:51.596925 2579 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/de29e5f0-022c-4c52-83e3-cc86ef01d137-istiod-ca-cert\") pod \"e2e-derive-658af5a0-openshift-default-6cb45b6dc5-fwwrm\" (UID: \"de29e5f0-022c-4c52-83e3-cc86ef01d137\") " pod="openshift-ingress/e2e-derive-658af5a0-openshift-default-6cb45b6dc5-fwwrm" Jul 03 11:39:51.597010 ip-10-0-136-0 kubenswrapper[2579]: I0703 11:39:51.596960 2579 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/de29e5f0-022c-4c52-83e3-cc86ef01d137-credential-socket\") pod \"e2e-derive-658af5a0-openshift-default-6cb45b6dc5-fwwrm\" (UID: \"de29e5f0-022c-4c52-83e3-cc86ef01d137\") " pod="openshift-ingress/e2e-derive-658af5a0-openshift-default-6cb45b6dc5-fwwrm" Jul 03 11:39:51.597010 ip-10-0-136-0 kubenswrapper[2579]: I0703 11:39:51.596991 2579 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/de29e5f0-022c-4c52-83e3-cc86ef01d137-istio-envoy\") pod \"e2e-derive-658af5a0-openshift-default-6cb45b6dc5-fwwrm\" (UID: \"de29e5f0-022c-4c52-83e3-cc86ef01d137\") " pod="openshift-ingress/e2e-derive-658af5a0-openshift-default-6cb45b6dc5-fwwrm" Jul 03 11:39:51.698265 ip-10-0-136-0 kubenswrapper[2579]: I0703 11:39:51.698217 2579 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/de29e5f0-022c-4c52-83e3-cc86ef01d137-workload-certs\") pod \"e2e-derive-658af5a0-openshift-default-6cb45b6dc5-fwwrm\" (UID: \"de29e5f0-022c-4c52-83e3-cc86ef01d137\") " pod="openshift-ingress/e2e-derive-658af5a0-openshift-default-6cb45b6dc5-fwwrm" Jul 03 11:39:51.698265 ip-10-0-136-0 kubenswrapper[2579]: I0703 11:39:51.698268 2579 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-g4nrs\" (UniqueName: \"kubernetes.io/projected/de29e5f0-022c-4c52-83e3-cc86ef01d137-kube-api-access-g4nrs\") pod \"e2e-derive-658af5a0-openshift-default-6cb45b6dc5-fwwrm\" (UID: \"de29e5f0-022c-4c52-83e3-cc86ef01d137\") " pod="openshift-ingress/e2e-derive-658af5a0-openshift-default-6cb45b6dc5-fwwrm" Jul 03 11:39:51.698561 ip-10-0-136-0 kubenswrapper[2579]: I0703 11:39:51.698320 2579 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/de29e5f0-022c-4c52-83e3-cc86ef01d137-istio-podinfo\") pod \"e2e-derive-658af5a0-openshift-default-6cb45b6dc5-fwwrm\" (UID: \"de29e5f0-022c-4c52-83e3-cc86ef01d137\") " pod="openshift-ingress/e2e-derive-658af5a0-openshift-default-6cb45b6dc5-fwwrm" Jul 03 11:39:51.698561 ip-10-0-136-0 kubenswrapper[2579]: I0703 11:39:51.698347 2579 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/de29e5f0-022c-4c52-83e3-cc86ef01d137-istio-token\") pod \"e2e-derive-658af5a0-openshift-default-6cb45b6dc5-fwwrm\" (UID: \"de29e5f0-022c-4c52-83e3-cc86ef01d137\") " pod="openshift-ingress/e2e-derive-658af5a0-openshift-default-6cb45b6dc5-fwwrm" Jul 03 11:39:51.698561 ip-10-0-136-0 kubenswrapper[2579]: I0703 11:39:51.698381 2579 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/de29e5f0-022c-4c52-83e3-cc86ef01d137-workload-socket\") pod \"e2e-derive-658af5a0-openshift-default-6cb45b6dc5-fwwrm\" (UID: \"de29e5f0-022c-4c52-83e3-cc86ef01d137\") " pod="openshift-ingress/e2e-derive-658af5a0-openshift-default-6cb45b6dc5-fwwrm" Jul 03 11:39:51.698561 ip-10-0-136-0 kubenswrapper[2579]: I0703 11:39:51.698437 2579 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/de29e5f0-022c-4c52-83e3-cc86ef01d137-istiod-ca-cert\") pod \"e2e-derive-658af5a0-openshift-default-6cb45b6dc5-fwwrm\" (UID: \"de29e5f0-022c-4c52-83e3-cc86ef01d137\") " pod="openshift-ingress/e2e-derive-658af5a0-openshift-default-6cb45b6dc5-fwwrm" Jul 03 11:39:51.698561 ip-10-0-136-0 kubenswrapper[2579]: I0703 11:39:51.698480 2579 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/de29e5f0-022c-4c52-83e3-cc86ef01d137-credential-socket\") pod \"e2e-derive-658af5a0-openshift-default-6cb45b6dc5-fwwrm\" (UID: \"de29e5f0-022c-4c52-83e3-cc86ef01d137\") " pod="openshift-ingress/e2e-derive-658af5a0-openshift-default-6cb45b6dc5-fwwrm" Jul 03 11:39:51.698561 ip-10-0-136-0 kubenswrapper[2579]: I0703 11:39:51.698523 2579 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/de29e5f0-022c-4c52-83e3-cc86ef01d137-istio-envoy\") pod \"e2e-derive-658af5a0-openshift-default-6cb45b6dc5-fwwrm\" (UID: \"de29e5f0-022c-4c52-83e3-cc86ef01d137\") " pod="openshift-ingress/e2e-derive-658af5a0-openshift-default-6cb45b6dc5-fwwrm" Jul 03 11:39:51.698892 ip-10-0-136-0 kubenswrapper[2579]: I0703 11:39:51.698602 2579 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/de29e5f0-022c-4c52-83e3-cc86ef01d137-istio-data\") pod \"e2e-derive-658af5a0-openshift-default-6cb45b6dc5-fwwrm\" (UID: \"de29e5f0-022c-4c52-83e3-cc86ef01d137\") " pod="openshift-ingress/e2e-derive-658af5a0-openshift-default-6cb45b6dc5-fwwrm" Jul 03 11:39:51.698892 ip-10-0-136-0 kubenswrapper[2579]: I0703 11:39:51.698769 2579 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/de29e5f0-022c-4c52-83e3-cc86ef01d137-workload-certs\") pod \"e2e-derive-658af5a0-openshift-default-6cb45b6dc5-fwwrm\" (UID: \"de29e5f0-022c-4c52-83e3-cc86ef01d137\") " pod="openshift-ingress/e2e-derive-658af5a0-openshift-default-6cb45b6dc5-fwwrm" Jul 03 11:39:51.699007 ip-10-0-136-0 kubenswrapper[2579]: I0703 11:39:51.698944 2579 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/de29e5f0-022c-4c52-83e3-cc86ef01d137-istio-data\") pod \"e2e-derive-658af5a0-openshift-default-6cb45b6dc5-fwwrm\" (UID: \"de29e5f0-022c-4c52-83e3-cc86ef01d137\") " pod="openshift-ingress/e2e-derive-658af5a0-openshift-default-6cb45b6dc5-fwwrm" Jul 03 11:39:51.699007 ip-10-0-136-0 kubenswrapper[2579]: I0703 11:39:51.698972 2579 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/de29e5f0-022c-4c52-83e3-cc86ef01d137-workload-socket\") pod \"e2e-derive-658af5a0-openshift-default-6cb45b6dc5-fwwrm\" (UID: \"de29e5f0-022c-4c52-83e3-cc86ef01d137\") " pod="openshift-ingress/e2e-derive-658af5a0-openshift-default-6cb45b6dc5-fwwrm" Jul 03 11:39:51.699113 ip-10-0-136-0 kubenswrapper[2579]: I0703 11:39:51.699050 2579 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/de29e5f0-022c-4c52-83e3-cc86ef01d137-credential-socket\") pod \"e2e-derive-658af5a0-openshift-default-6cb45b6dc5-fwwrm\" (UID: \"de29e5f0-022c-4c52-83e3-cc86ef01d137\") " pod="openshift-ingress/e2e-derive-658af5a0-openshift-default-6cb45b6dc5-fwwrm" Jul 03 11:39:51.699310 ip-10-0-136-0 kubenswrapper[2579]: I0703 11:39:51.699279 2579 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/de29e5f0-022c-4c52-83e3-cc86ef01d137-istiod-ca-cert\") pod \"e2e-derive-658af5a0-openshift-default-6cb45b6dc5-fwwrm\" (UID: \"de29e5f0-022c-4c52-83e3-cc86ef01d137\") " pod="openshift-ingress/e2e-derive-658af5a0-openshift-default-6cb45b6dc5-fwwrm" Jul 03 11:39:51.701160 ip-10-0-136-0 kubenswrapper[2579]: I0703 11:39:51.701134 2579 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/de29e5f0-022c-4c52-83e3-cc86ef01d137-istio-envoy\") pod \"e2e-derive-658af5a0-openshift-default-6cb45b6dc5-fwwrm\" (UID: \"de29e5f0-022c-4c52-83e3-cc86ef01d137\") " pod="openshift-ingress/e2e-derive-658af5a0-openshift-default-6cb45b6dc5-fwwrm" Jul 03 11:39:51.701325 ip-10-0-136-0 kubenswrapper[2579]: I0703 11:39:51.701305 2579 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/de29e5f0-022c-4c52-83e3-cc86ef01d137-istio-podinfo\") pod \"e2e-derive-658af5a0-openshift-default-6cb45b6dc5-fwwrm\" (UID: \"de29e5f0-022c-4c52-83e3-cc86ef01d137\") " pod="openshift-ingress/e2e-derive-658af5a0-openshift-default-6cb45b6dc5-fwwrm" Jul 03 11:39:51.712545 ip-10-0-136-0 kubenswrapper[2579]: I0703 11:39:51.712514 2579 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/de29e5f0-022c-4c52-83e3-cc86ef01d137-istio-token\") pod \"e2e-derive-658af5a0-openshift-default-6cb45b6dc5-fwwrm\" (UID: \"de29e5f0-022c-4c52-83e3-cc86ef01d137\") " pod="openshift-ingress/e2e-derive-658af5a0-openshift-default-6cb45b6dc5-fwwrm" Jul 03 11:39:51.712734 ip-10-0-136-0 kubenswrapper[2579]: I0703 11:39:51.712712 2579 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-g4nrs\" (UniqueName: \"kubernetes.io/projected/de29e5f0-022c-4c52-83e3-cc86ef01d137-kube-api-access-g4nrs\") pod \"e2e-derive-658af5a0-openshift-default-6cb45b6dc5-fwwrm\" (UID: \"de29e5f0-022c-4c52-83e3-cc86ef01d137\") " pod="openshift-ingress/e2e-derive-658af5a0-openshift-default-6cb45b6dc5-fwwrm" Jul 03 11:39:51.787737 ip-10-0-136-0 kubenswrapper[2579]: I0703 11:39:51.787693 2579 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-derive-658af5a0-openshift-default-6cb45b6dc5-fwwrm" Jul 03 11:39:51.979396 ip-10-0-136-0 kubenswrapper[2579]: I0703 11:39:51.979365 2579 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/e2e-derive-658af5a0-openshift-default-6cb45b6dc5-fwwrm"] Jul 03 11:39:51.982013 ip-10-0-136-0 kubenswrapper[2579]: W0703 11:39:51.981979 2579 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podde29e5f0_022c_4c52_83e3_cc86ef01d137.slice/crio-e3cc6116f0a1485d2145f2e5997cb0ebb1e23d342b12362d201301c83f975db1 WatchSource:0}: Error finding container e3cc6116f0a1485d2145f2e5997cb0ebb1e23d342b12362d201301c83f975db1: Status 404 returned error can't find the container with id e3cc6116f0a1485d2145f2e5997cb0ebb1e23d342b12362d201301c83f975db1 Jul 03 11:39:51.984301 ip-10-0-136-0 kubenswrapper[2579]: I0703 11:39:51.984265 2579 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"30892156Ki","pods":"250"} Jul 03 11:39:51.984405 ip-10-0-136-0 kubenswrapper[2579]: I0703 11:39:51.984340 2579 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"30892156Ki","pods":"250"} Jul 03 11:39:51.984405 ip-10-0-136-0 kubenswrapper[2579]: I0703 11:39:51.984386 2579 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"30892156Ki","pods":"250"} Jul 03 11:39:52.842396 ip-10-0-136-0 kubenswrapper[2579]: I0703 11:39:52.842348 2579 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-derive-658af5a0-openshift-default-6cb45b6dc5-fwwrm" event={"ID":"de29e5f0-022c-4c52-83e3-cc86ef01d137","Type":"ContainerStarted","Data":"3d808355fa27b5d470bcf1fe55468c93b3820444126e7d6694ff1221908503c4"} Jul 03 11:39:52.842880 ip-10-0-136-0 kubenswrapper[2579]: I0703 11:39:52.842427 2579 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-derive-658af5a0-openshift-default-6cb45b6dc5-fwwrm" event={"ID":"de29e5f0-022c-4c52-83e3-cc86ef01d137","Type":"ContainerStarted","Data":"e3cc6116f0a1485d2145f2e5997cb0ebb1e23d342b12362d201301c83f975db1"} Jul 03 11:39:52.880412 ip-10-0-136-0 kubenswrapper[2579]: I0703 11:39:52.880346 2579 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-ingress/e2e-derive-658af5a0-openshift-default-6cb45b6dc5-fwwrm" podStartSLOduration=1.880327963 podStartE2EDuration="1.880327963s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-03 11:39:51 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-03 11:39:52.878419982 +0000 UTC m=+1649.466724329" watchObservedRunningTime="2026-07-03 11:39:52.880327963 +0000 UTC m=+1649.468632281" Jul 03 11:39:53.788672 ip-10-0-136-0 kubenswrapper[2579]: I0703 11:39:53.788621 2579 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="openshift-ingress/e2e-derive-658af5a0-openshift-default-6cb45b6dc5-fwwrm" Jul 03 11:39:53.794323 ip-10-0-136-0 kubenswrapper[2579]: I0703 11:39:53.794289 2579 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="openshift-ingress/e2e-derive-658af5a0-openshift-default-6cb45b6dc5-fwwrm" Jul 03 11:39:53.855429 ip-10-0-136-0 kubenswrapper[2579]: I0703 11:39:53.855388 2579 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/e2e-derive-658af5a0-openshift-default-6cb45b6dc5-fwwrm" Jul 03 11:39:53.856516 ip-10-0-136-0 kubenswrapper[2579]: I0703 11:39:53.856489 2579 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/e2e-derive-658af5a0-openshift-default-6cb45b6dc5-fwwrm" Jul 03 11:39:57.401937 ip-10-0-136-0 kubenswrapper[2579]: I0703 11:39:57.401889 2579 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-ingress/e2e-derive-658af5a0-openshift-default-6cb45b6dc5-fwwrm"] Jul 03 11:39:57.402413 ip-10-0-136-0 kubenswrapper[2579]: I0703 11:39:57.402096 2579 kuberuntime_container.go:872] "Killing container with a grace period" pod="openshift-ingress/e2e-derive-658af5a0-openshift-default-6cb45b6dc5-fwwrm" podUID="de29e5f0-022c-4c52-83e3-cc86ef01d137" containerName="istio-proxy" containerID="cri-o://3d808355fa27b5d470bcf1fe55468c93b3820444126e7d6694ff1221908503c4" gracePeriod=30 Jul 03 11:40:02.709650 ip-10-0-136-0 kubenswrapper[2579]: I0703 11:40:02.709609 2579 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-derive-658af5a0-openshift-default-6cb45b6dc5-fwwrm" Jul 03 11:40:02.819261 ip-10-0-136-0 kubenswrapper[2579]: I0703 11:40:02.819182 2579 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/de29e5f0-022c-4c52-83e3-cc86ef01d137-istio-data\") pod \"de29e5f0-022c-4c52-83e3-cc86ef01d137\" (UID: \"de29e5f0-022c-4c52-83e3-cc86ef01d137\") " Jul 03 11:40:02.819463 ip-10-0-136-0 kubenswrapper[2579]: I0703 11:40:02.819288 2579 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/de29e5f0-022c-4c52-83e3-cc86ef01d137-istio-token\") pod \"de29e5f0-022c-4c52-83e3-cc86ef01d137\" (UID: \"de29e5f0-022c-4c52-83e3-cc86ef01d137\") " Jul 03 11:40:02.819463 ip-10-0-136-0 kubenswrapper[2579]: I0703 11:40:02.819315 2579 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/de29e5f0-022c-4c52-83e3-cc86ef01d137-workload-socket\") pod \"de29e5f0-022c-4c52-83e3-cc86ef01d137\" (UID: \"de29e5f0-022c-4c52-83e3-cc86ef01d137\") " Jul 03 11:40:02.819463 ip-10-0-136-0 kubenswrapper[2579]: I0703 11:40:02.819346 2579 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/de29e5f0-022c-4c52-83e3-cc86ef01d137-istio-podinfo\") pod \"de29e5f0-022c-4c52-83e3-cc86ef01d137\" (UID: \"de29e5f0-022c-4c52-83e3-cc86ef01d137\") " Jul 03 11:40:02.819463 ip-10-0-136-0 kubenswrapper[2579]: I0703 11:40:02.819375 2579 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/de29e5f0-022c-4c52-83e3-cc86ef01d137-credential-socket\") pod \"de29e5f0-022c-4c52-83e3-cc86ef01d137\" (UID: \"de29e5f0-022c-4c52-83e3-cc86ef01d137\") " Jul 03 11:40:02.819463 ip-10-0-136-0 kubenswrapper[2579]: I0703 11:40:02.819432 2579 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/de29e5f0-022c-4c52-83e3-cc86ef01d137-istiod-ca-cert\") pod \"de29e5f0-022c-4c52-83e3-cc86ef01d137\" (UID: \"de29e5f0-022c-4c52-83e3-cc86ef01d137\") " Jul 03 11:40:02.819725 ip-10-0-136-0 kubenswrapper[2579]: I0703 11:40:02.819522 2579 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/de29e5f0-022c-4c52-83e3-cc86ef01d137-workload-certs\") pod \"de29e5f0-022c-4c52-83e3-cc86ef01d137\" (UID: \"de29e5f0-022c-4c52-83e3-cc86ef01d137\") " Jul 03 11:40:02.819725 ip-10-0-136-0 kubenswrapper[2579]: I0703 11:40:02.819537 2579 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/de29e5f0-022c-4c52-83e3-cc86ef01d137-istio-data" (OuterVolumeSpecName: "istio-data") pod "de29e5f0-022c-4c52-83e3-cc86ef01d137" (UID: "de29e5f0-022c-4c52-83e3-cc86ef01d137"). InnerVolumeSpecName "istio-data". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Jul 03 11:40:02.819725 ip-10-0-136-0 kubenswrapper[2579]: I0703 11:40:02.819620 2579 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/de29e5f0-022c-4c52-83e3-cc86ef01d137-workload-socket" (OuterVolumeSpecName: "workload-socket") pod "de29e5f0-022c-4c52-83e3-cc86ef01d137" (UID: "de29e5f0-022c-4c52-83e3-cc86ef01d137"). InnerVolumeSpecName "workload-socket". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Jul 03 11:40:02.819725 ip-10-0-136-0 kubenswrapper[2579]: I0703 11:40:02.819645 2579 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/de29e5f0-022c-4c52-83e3-cc86ef01d137-istio-envoy\") pod \"de29e5f0-022c-4c52-83e3-cc86ef01d137\" (UID: \"de29e5f0-022c-4c52-83e3-cc86ef01d137\") " Jul 03 11:40:02.819953 ip-10-0-136-0 kubenswrapper[2579]: I0703 11:40:02.819760 2579 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-g4nrs\" (UniqueName: \"kubernetes.io/projected/de29e5f0-022c-4c52-83e3-cc86ef01d137-kube-api-access-g4nrs\") pod \"de29e5f0-022c-4c52-83e3-cc86ef01d137\" (UID: \"de29e5f0-022c-4c52-83e3-cc86ef01d137\") " Jul 03 11:40:02.819953 ip-10-0-136-0 kubenswrapper[2579]: I0703 11:40:02.819815 2579 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/de29e5f0-022c-4c52-83e3-cc86ef01d137-workload-certs" (OuterVolumeSpecName: "workload-certs") pod "de29e5f0-022c-4c52-83e3-cc86ef01d137" (UID: "de29e5f0-022c-4c52-83e3-cc86ef01d137"). InnerVolumeSpecName "workload-certs". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Jul 03 11:40:02.819953 ip-10-0-136-0 kubenswrapper[2579]: I0703 11:40:02.819825 2579 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/de29e5f0-022c-4c52-83e3-cc86ef01d137-credential-socket" (OuterVolumeSpecName: "credential-socket") pod "de29e5f0-022c-4c52-83e3-cc86ef01d137" (UID: "de29e5f0-022c-4c52-83e3-cc86ef01d137"). InnerVolumeSpecName "credential-socket". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Jul 03 11:40:02.820110 ip-10-0-136-0 kubenswrapper[2579]: I0703 11:40:02.819921 2579 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/de29e5f0-022c-4c52-83e3-cc86ef01d137-istiod-ca-cert" (OuterVolumeSpecName: "istiod-ca-cert") pod "de29e5f0-022c-4c52-83e3-cc86ef01d137" (UID: "de29e5f0-022c-4c52-83e3-cc86ef01d137"). InnerVolumeSpecName "istiod-ca-cert". PluginName "kubernetes.io/configmap", VolumeGIDValue "" Jul 03 11:40:02.820371 ip-10-0-136-0 kubenswrapper[2579]: I0703 11:40:02.820305 2579 reconciler_common.go:299] "Volume detached for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/de29e5f0-022c-4c52-83e3-cc86ef01d137-istiod-ca-cert\") on node \"ip-10-0-136-0.ec2.internal\" DevicePath \"\"" Jul 03 11:40:02.820371 ip-10-0-136-0 kubenswrapper[2579]: I0703 11:40:02.820328 2579 reconciler_common.go:299] "Volume detached for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/de29e5f0-022c-4c52-83e3-cc86ef01d137-workload-certs\") on node \"ip-10-0-136-0.ec2.internal\" DevicePath \"\"" Jul 03 11:40:02.820371 ip-10-0-136-0 kubenswrapper[2579]: I0703 11:40:02.820339 2579 reconciler_common.go:299] "Volume detached for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/de29e5f0-022c-4c52-83e3-cc86ef01d137-istio-data\") on node \"ip-10-0-136-0.ec2.internal\" DevicePath \"\"" Jul 03 11:40:02.820371 ip-10-0-136-0 kubenswrapper[2579]: I0703 11:40:02.820347 2579 reconciler_common.go:299] "Volume detached for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/de29e5f0-022c-4c52-83e3-cc86ef01d137-workload-socket\") on node \"ip-10-0-136-0.ec2.internal\" DevicePath \"\"" Jul 03 11:40:02.820371 ip-10-0-136-0 kubenswrapper[2579]: I0703 11:40:02.820355 2579 reconciler_common.go:299] "Volume detached for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/de29e5f0-022c-4c52-83e3-cc86ef01d137-credential-socket\") on node \"ip-10-0-136-0.ec2.internal\" DevicePath \"\"" Jul 03 11:40:02.821858 ip-10-0-136-0 kubenswrapper[2579]: I0703 11:40:02.821829 2579 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/de29e5f0-022c-4c52-83e3-cc86ef01d137-istio-token" (OuterVolumeSpecName: "istio-token") pod "de29e5f0-022c-4c52-83e3-cc86ef01d137" (UID: "de29e5f0-022c-4c52-83e3-cc86ef01d137"). InnerVolumeSpecName "istio-token". PluginName "kubernetes.io/projected", VolumeGIDValue "" Jul 03 11:40:02.821986 ip-10-0-136-0 kubenswrapper[2579]: I0703 11:40:02.821877 2579 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/downward-api/de29e5f0-022c-4c52-83e3-cc86ef01d137-istio-podinfo" (OuterVolumeSpecName: "istio-podinfo") pod "de29e5f0-022c-4c52-83e3-cc86ef01d137" (UID: "de29e5f0-022c-4c52-83e3-cc86ef01d137"). InnerVolumeSpecName "istio-podinfo". PluginName "kubernetes.io/downward-api", VolumeGIDValue "" Jul 03 11:40:02.822321 ip-10-0-136-0 kubenswrapper[2579]: I0703 11:40:02.822299 2579 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/de29e5f0-022c-4c52-83e3-cc86ef01d137-istio-envoy" (OuterVolumeSpecName: "istio-envoy") pod "de29e5f0-022c-4c52-83e3-cc86ef01d137" (UID: "de29e5f0-022c-4c52-83e3-cc86ef01d137"). InnerVolumeSpecName "istio-envoy". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Jul 03 11:40:02.822378 ip-10-0-136-0 kubenswrapper[2579]: I0703 11:40:02.822303 2579 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/de29e5f0-022c-4c52-83e3-cc86ef01d137-kube-api-access-g4nrs" (OuterVolumeSpecName: "kube-api-access-g4nrs") pod "de29e5f0-022c-4c52-83e3-cc86ef01d137" (UID: "de29e5f0-022c-4c52-83e3-cc86ef01d137"). InnerVolumeSpecName "kube-api-access-g4nrs". PluginName "kubernetes.io/projected", VolumeGIDValue "" Jul 03 11:40:02.910636 ip-10-0-136-0 kubenswrapper[2579]: I0703 11:40:02.910598 2579 generic.go:356] "Generic (PLEG): container finished" podID="de29e5f0-022c-4c52-83e3-cc86ef01d137" containerID="3d808355fa27b5d470bcf1fe55468c93b3820444126e7d6694ff1221908503c4" exitCode=0 Jul 03 11:40:02.910820 ip-10-0-136-0 kubenswrapper[2579]: I0703 11:40:02.910648 2579 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-derive-658af5a0-openshift-default-6cb45b6dc5-fwwrm" Jul 03 11:40:02.910820 ip-10-0-136-0 kubenswrapper[2579]: I0703 11:40:02.910685 2579 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-derive-658af5a0-openshift-default-6cb45b6dc5-fwwrm" event={"ID":"de29e5f0-022c-4c52-83e3-cc86ef01d137","Type":"ContainerDied","Data":"3d808355fa27b5d470bcf1fe55468c93b3820444126e7d6694ff1221908503c4"} Jul 03 11:40:02.910820 ip-10-0-136-0 kubenswrapper[2579]: I0703 11:40:02.910769 2579 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-derive-658af5a0-openshift-default-6cb45b6dc5-fwwrm" event={"ID":"de29e5f0-022c-4c52-83e3-cc86ef01d137","Type":"ContainerDied","Data":"e3cc6116f0a1485d2145f2e5997cb0ebb1e23d342b12362d201301c83f975db1"} Jul 03 11:40:02.910820 ip-10-0-136-0 kubenswrapper[2579]: I0703 11:40:02.910800 2579 scope.go:117] "RemoveContainer" containerID="3d808355fa27b5d470bcf1fe55468c93b3820444126e7d6694ff1221908503c4" Jul 03 11:40:02.921722 ip-10-0-136-0 kubenswrapper[2579]: I0703 11:40:02.921690 2579 reconciler_common.go:299] "Volume detached for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/de29e5f0-022c-4c52-83e3-cc86ef01d137-istio-envoy\") on node \"ip-10-0-136-0.ec2.internal\" DevicePath \"\"" Jul 03 11:40:02.921722 ip-10-0-136-0 kubenswrapper[2579]: I0703 11:40:02.921715 2579 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-g4nrs\" (UniqueName: \"kubernetes.io/projected/de29e5f0-022c-4c52-83e3-cc86ef01d137-kube-api-access-g4nrs\") on node \"ip-10-0-136-0.ec2.internal\" DevicePath \"\"" Jul 03 11:40:02.921722 ip-10-0-136-0 kubenswrapper[2579]: I0703 11:40:02.921725 2579 reconciler_common.go:299] "Volume detached for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/de29e5f0-022c-4c52-83e3-cc86ef01d137-istio-token\") on node \"ip-10-0-136-0.ec2.internal\" DevicePath \"\"" Jul 03 11:40:02.921940 ip-10-0-136-0 kubenswrapper[2579]: I0703 11:40:02.921737 2579 reconciler_common.go:299] "Volume detached for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/de29e5f0-022c-4c52-83e3-cc86ef01d137-istio-podinfo\") on node \"ip-10-0-136-0.ec2.internal\" DevicePath \"\"" Jul 03 11:40:02.922050 ip-10-0-136-0 kubenswrapper[2579]: I0703 11:40:02.922033 2579 scope.go:117] "RemoveContainer" containerID="3d808355fa27b5d470bcf1fe55468c93b3820444126e7d6694ff1221908503c4" Jul 03 11:40:02.922398 ip-10-0-136-0 kubenswrapper[2579]: E0703 11:40:02.922379 2579 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"3d808355fa27b5d470bcf1fe55468c93b3820444126e7d6694ff1221908503c4\": container with ID starting with 3d808355fa27b5d470bcf1fe55468c93b3820444126e7d6694ff1221908503c4 not found: ID does not exist" containerID="3d808355fa27b5d470bcf1fe55468c93b3820444126e7d6694ff1221908503c4" Jul 03 11:40:02.922463 ip-10-0-136-0 kubenswrapper[2579]: I0703 11:40:02.922408 2579 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"3d808355fa27b5d470bcf1fe55468c93b3820444126e7d6694ff1221908503c4"} err="failed to get container status \"3d808355fa27b5d470bcf1fe55468c93b3820444126e7d6694ff1221908503c4\": rpc error: code = NotFound desc = could not find container \"3d808355fa27b5d470bcf1fe55468c93b3820444126e7d6694ff1221908503c4\": container with ID starting with 3d808355fa27b5d470bcf1fe55468c93b3820444126e7d6694ff1221908503c4 not found: ID does not exist" Jul 03 11:40:02.935668 ip-10-0-136-0 kubenswrapper[2579]: I0703 11:40:02.935621 2579 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-ingress/e2e-derive-658af5a0-openshift-default-6cb45b6dc5-fwwrm"] Jul 03 11:40:02.938269 ip-10-0-136-0 kubenswrapper[2579]: I0703 11:40:02.938163 2579 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["openshift-ingress/e2e-derive-658af5a0-openshift-default-6cb45b6dc5-fwwrm"] Jul 03 11:40:03.992988 ip-10-0-136-0 kubenswrapper[2579]: I0703 11:40:03.992947 2579 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="de29e5f0-022c-4c52-83e3-cc86ef01d137" path="/var/lib/kubelet/pods/de29e5f0-022c-4c52-83e3-cc86ef01d137/volumes" Jul 03 11:41:42.234762 ip-10-0-136-0 kubenswrapper[2579]: I0703 11:41:42.234710 2579 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-ingress/e2e-mt-c6d643e5-openshift-default-5db8d64d55-jb4z8"] Jul 03 11:41:42.235401 ip-10-0-136-0 kubenswrapper[2579]: I0703 11:41:42.235383 2579 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="de29e5f0-022c-4c52-83e3-cc86ef01d137" containerName="istio-proxy" Jul 03 11:41:42.235463 ip-10-0-136-0 kubenswrapper[2579]: I0703 11:41:42.235405 2579 state_mem.go:107] "Deleted CPUSet assignment" podUID="de29e5f0-022c-4c52-83e3-cc86ef01d137" containerName="istio-proxy" Jul 03 11:41:42.235527 ip-10-0-136-0 kubenswrapper[2579]: I0703 11:41:42.235515 2579 memory_manager.go:368] "RemoveStaleState removing state" podUID="de29e5f0-022c-4c52-83e3-cc86ef01d137" containerName="istio-proxy" Jul 03 11:41:42.239381 ip-10-0-136-0 kubenswrapper[2579]: I0703 11:41:42.239342 2579 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-mt-c6d643e5-openshift-default-5db8d64d55-jb4z8" Jul 03 11:41:42.244539 ip-10-0-136-0 kubenswrapper[2579]: I0703 11:41:42.244510 2579 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress\"/\"e2e-mt-c6d643e5-openshift-default-dockercfg-vk44d\"" Jul 03 11:41:42.370774 ip-10-0-136-0 kubenswrapper[2579]: I0703 11:41:42.370725 2579 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/e2e-mt-c6d643e5-openshift-default-5db8d64d55-jb4z8"] Jul 03 11:41:42.398779 ip-10-0-136-0 kubenswrapper[2579]: I0703 11:41:42.398731 2579 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/2ad22828-4f7a-4ceb-9d0c-5f96b842878b-credential-socket\") pod \"e2e-mt-c6d643e5-openshift-default-5db8d64d55-jb4z8\" (UID: \"2ad22828-4f7a-4ceb-9d0c-5f96b842878b\") " pod="openshift-ingress/e2e-mt-c6d643e5-openshift-default-5db8d64d55-jb4z8" Jul 03 11:41:42.398974 ip-10-0-136-0 kubenswrapper[2579]: I0703 11:41:42.398796 2579 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/2ad22828-4f7a-4ceb-9d0c-5f96b842878b-istio-token\") pod \"e2e-mt-c6d643e5-openshift-default-5db8d64d55-jb4z8\" (UID: \"2ad22828-4f7a-4ceb-9d0c-5f96b842878b\") " pod="openshift-ingress/e2e-mt-c6d643e5-openshift-default-5db8d64d55-jb4z8" Jul 03 11:41:42.398974 ip-10-0-136-0 kubenswrapper[2579]: I0703 11:41:42.398934 2579 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/2ad22828-4f7a-4ceb-9d0c-5f96b842878b-istio-podinfo\") pod \"e2e-mt-c6d643e5-openshift-default-5db8d64d55-jb4z8\" (UID: \"2ad22828-4f7a-4ceb-9d0c-5f96b842878b\") " pod="openshift-ingress/e2e-mt-c6d643e5-openshift-default-5db8d64d55-jb4z8" Jul 03 11:41:42.399079 ip-10-0-136-0 kubenswrapper[2579]: I0703 11:41:42.398977 2579 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/2ad22828-4f7a-4ceb-9d0c-5f96b842878b-istiod-ca-cert\") pod \"e2e-mt-c6d643e5-openshift-default-5db8d64d55-jb4z8\" (UID: \"2ad22828-4f7a-4ceb-9d0c-5f96b842878b\") " pod="openshift-ingress/e2e-mt-c6d643e5-openshift-default-5db8d64d55-jb4z8" Jul 03 11:41:42.399079 ip-10-0-136-0 kubenswrapper[2579]: I0703 11:41:42.398996 2579 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-kptwc\" (UniqueName: \"kubernetes.io/projected/2ad22828-4f7a-4ceb-9d0c-5f96b842878b-kube-api-access-kptwc\") pod \"e2e-mt-c6d643e5-openshift-default-5db8d64d55-jb4z8\" (UID: \"2ad22828-4f7a-4ceb-9d0c-5f96b842878b\") " pod="openshift-ingress/e2e-mt-c6d643e5-openshift-default-5db8d64d55-jb4z8" Jul 03 11:41:42.399079 ip-10-0-136-0 kubenswrapper[2579]: I0703 11:41:42.399060 2579 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/2ad22828-4f7a-4ceb-9d0c-5f96b842878b-workload-socket\") pod \"e2e-mt-c6d643e5-openshift-default-5db8d64d55-jb4z8\" (UID: \"2ad22828-4f7a-4ceb-9d0c-5f96b842878b\") " pod="openshift-ingress/e2e-mt-c6d643e5-openshift-default-5db8d64d55-jb4z8" Jul 03 11:41:42.399218 ip-10-0-136-0 kubenswrapper[2579]: I0703 11:41:42.399090 2579 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/2ad22828-4f7a-4ceb-9d0c-5f96b842878b-istio-envoy\") pod \"e2e-mt-c6d643e5-openshift-default-5db8d64d55-jb4z8\" (UID: \"2ad22828-4f7a-4ceb-9d0c-5f96b842878b\") " pod="openshift-ingress/e2e-mt-c6d643e5-openshift-default-5db8d64d55-jb4z8" Jul 03 11:41:42.399218 ip-10-0-136-0 kubenswrapper[2579]: I0703 11:41:42.399111 2579 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/2ad22828-4f7a-4ceb-9d0c-5f96b842878b-workload-certs\") pod \"e2e-mt-c6d643e5-openshift-default-5db8d64d55-jb4z8\" (UID: \"2ad22828-4f7a-4ceb-9d0c-5f96b842878b\") " pod="openshift-ingress/e2e-mt-c6d643e5-openshift-default-5db8d64d55-jb4z8" Jul 03 11:41:42.399218 ip-10-0-136-0 kubenswrapper[2579]: I0703 11:41:42.399130 2579 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/2ad22828-4f7a-4ceb-9d0c-5f96b842878b-istio-data\") pod \"e2e-mt-c6d643e5-openshift-default-5db8d64d55-jb4z8\" (UID: \"2ad22828-4f7a-4ceb-9d0c-5f96b842878b\") " pod="openshift-ingress/e2e-mt-c6d643e5-openshift-default-5db8d64d55-jb4z8" Jul 03 11:41:42.500741 ip-10-0-136-0 kubenswrapper[2579]: I0703 11:41:42.500625 2579 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/2ad22828-4f7a-4ceb-9d0c-5f96b842878b-istio-podinfo\") pod \"e2e-mt-c6d643e5-openshift-default-5db8d64d55-jb4z8\" (UID: \"2ad22828-4f7a-4ceb-9d0c-5f96b842878b\") " pod="openshift-ingress/e2e-mt-c6d643e5-openshift-default-5db8d64d55-jb4z8" Jul 03 11:41:42.500741 ip-10-0-136-0 kubenswrapper[2579]: I0703 11:41:42.500684 2579 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/2ad22828-4f7a-4ceb-9d0c-5f96b842878b-istiod-ca-cert\") pod \"e2e-mt-c6d643e5-openshift-default-5db8d64d55-jb4z8\" (UID: \"2ad22828-4f7a-4ceb-9d0c-5f96b842878b\") " pod="openshift-ingress/e2e-mt-c6d643e5-openshift-default-5db8d64d55-jb4z8" Jul 03 11:41:42.501012 ip-10-0-136-0 kubenswrapper[2579]: I0703 11:41:42.500843 2579 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-kptwc\" (UniqueName: \"kubernetes.io/projected/2ad22828-4f7a-4ceb-9d0c-5f96b842878b-kube-api-access-kptwc\") pod \"e2e-mt-c6d643e5-openshift-default-5db8d64d55-jb4z8\" (UID: \"2ad22828-4f7a-4ceb-9d0c-5f96b842878b\") " pod="openshift-ingress/e2e-mt-c6d643e5-openshift-default-5db8d64d55-jb4z8" Jul 03 11:41:42.501012 ip-10-0-136-0 kubenswrapper[2579]: I0703 11:41:42.500891 2579 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/2ad22828-4f7a-4ceb-9d0c-5f96b842878b-workload-socket\") pod \"e2e-mt-c6d643e5-openshift-default-5db8d64d55-jb4z8\" (UID: \"2ad22828-4f7a-4ceb-9d0c-5f96b842878b\") " pod="openshift-ingress/e2e-mt-c6d643e5-openshift-default-5db8d64d55-jb4z8" Jul 03 11:41:42.501012 ip-10-0-136-0 kubenswrapper[2579]: I0703 11:41:42.500914 2579 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/2ad22828-4f7a-4ceb-9d0c-5f96b842878b-istio-envoy\") pod \"e2e-mt-c6d643e5-openshift-default-5db8d64d55-jb4z8\" (UID: \"2ad22828-4f7a-4ceb-9d0c-5f96b842878b\") " pod="openshift-ingress/e2e-mt-c6d643e5-openshift-default-5db8d64d55-jb4z8" Jul 03 11:41:42.501012 ip-10-0-136-0 kubenswrapper[2579]: I0703 11:41:42.500933 2579 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/2ad22828-4f7a-4ceb-9d0c-5f96b842878b-workload-certs\") pod \"e2e-mt-c6d643e5-openshift-default-5db8d64d55-jb4z8\" (UID: \"2ad22828-4f7a-4ceb-9d0c-5f96b842878b\") " pod="openshift-ingress/e2e-mt-c6d643e5-openshift-default-5db8d64d55-jb4z8" Jul 03 11:41:42.501012 ip-10-0-136-0 kubenswrapper[2579]: I0703 11:41:42.500958 2579 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/2ad22828-4f7a-4ceb-9d0c-5f96b842878b-istio-data\") pod \"e2e-mt-c6d643e5-openshift-default-5db8d64d55-jb4z8\" (UID: \"2ad22828-4f7a-4ceb-9d0c-5f96b842878b\") " pod="openshift-ingress/e2e-mt-c6d643e5-openshift-default-5db8d64d55-jb4z8" Jul 03 11:41:42.501012 ip-10-0-136-0 kubenswrapper[2579]: I0703 11:41:42.501005 2579 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/2ad22828-4f7a-4ceb-9d0c-5f96b842878b-credential-socket\") pod \"e2e-mt-c6d643e5-openshift-default-5db8d64d55-jb4z8\" (UID: \"2ad22828-4f7a-4ceb-9d0c-5f96b842878b\") " pod="openshift-ingress/e2e-mt-c6d643e5-openshift-default-5db8d64d55-jb4z8" Jul 03 11:41:42.501012 ip-10-0-136-0 kubenswrapper[2579]: I0703 11:41:42.501052 2579 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/2ad22828-4f7a-4ceb-9d0c-5f96b842878b-istio-token\") pod \"e2e-mt-c6d643e5-openshift-default-5db8d64d55-jb4z8\" (UID: \"2ad22828-4f7a-4ceb-9d0c-5f96b842878b\") " pod="openshift-ingress/e2e-mt-c6d643e5-openshift-default-5db8d64d55-jb4z8" Jul 03 11:41:42.501629 ip-10-0-136-0 kubenswrapper[2579]: I0703 11:41:42.501454 2579 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/2ad22828-4f7a-4ceb-9d0c-5f96b842878b-workload-certs\") pod \"e2e-mt-c6d643e5-openshift-default-5db8d64d55-jb4z8\" (UID: \"2ad22828-4f7a-4ceb-9d0c-5f96b842878b\") " pod="openshift-ingress/e2e-mt-c6d643e5-openshift-default-5db8d64d55-jb4z8" Jul 03 11:41:42.501629 ip-10-0-136-0 kubenswrapper[2579]: I0703 11:41:42.501620 2579 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/2ad22828-4f7a-4ceb-9d0c-5f96b842878b-workload-socket\") pod \"e2e-mt-c6d643e5-openshift-default-5db8d64d55-jb4z8\" (UID: \"2ad22828-4f7a-4ceb-9d0c-5f96b842878b\") " pod="openshift-ingress/e2e-mt-c6d643e5-openshift-default-5db8d64d55-jb4z8" Jul 03 11:41:42.501748 ip-10-0-136-0 kubenswrapper[2579]: I0703 11:41:42.501682 2579 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/2ad22828-4f7a-4ceb-9d0c-5f96b842878b-istiod-ca-cert\") pod \"e2e-mt-c6d643e5-openshift-default-5db8d64d55-jb4z8\" (UID: \"2ad22828-4f7a-4ceb-9d0c-5f96b842878b\") " pod="openshift-ingress/e2e-mt-c6d643e5-openshift-default-5db8d64d55-jb4z8" Jul 03 11:41:42.501748 ip-10-0-136-0 kubenswrapper[2579]: I0703 11:41:42.501714 2579 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/2ad22828-4f7a-4ceb-9d0c-5f96b842878b-istio-data\") pod \"e2e-mt-c6d643e5-openshift-default-5db8d64d55-jb4z8\" (UID: \"2ad22828-4f7a-4ceb-9d0c-5f96b842878b\") " pod="openshift-ingress/e2e-mt-c6d643e5-openshift-default-5db8d64d55-jb4z8" Jul 03 11:41:42.501827 ip-10-0-136-0 kubenswrapper[2579]: I0703 11:41:42.501748 2579 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/2ad22828-4f7a-4ceb-9d0c-5f96b842878b-credential-socket\") pod \"e2e-mt-c6d643e5-openshift-default-5db8d64d55-jb4z8\" (UID: \"2ad22828-4f7a-4ceb-9d0c-5f96b842878b\") " pod="openshift-ingress/e2e-mt-c6d643e5-openshift-default-5db8d64d55-jb4z8" Jul 03 11:41:42.504107 ip-10-0-136-0 kubenswrapper[2579]: I0703 11:41:42.504077 2579 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/2ad22828-4f7a-4ceb-9d0c-5f96b842878b-istio-envoy\") pod \"e2e-mt-c6d643e5-openshift-default-5db8d64d55-jb4z8\" (UID: \"2ad22828-4f7a-4ceb-9d0c-5f96b842878b\") " pod="openshift-ingress/e2e-mt-c6d643e5-openshift-default-5db8d64d55-jb4z8" Jul 03 11:41:42.504272 ip-10-0-136-0 kubenswrapper[2579]: I0703 11:41:42.504251 2579 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/2ad22828-4f7a-4ceb-9d0c-5f96b842878b-istio-podinfo\") pod \"e2e-mt-c6d643e5-openshift-default-5db8d64d55-jb4z8\" (UID: \"2ad22828-4f7a-4ceb-9d0c-5f96b842878b\") " pod="openshift-ingress/e2e-mt-c6d643e5-openshift-default-5db8d64d55-jb4z8" Jul 03 11:41:42.511887 ip-10-0-136-0 kubenswrapper[2579]: I0703 11:41:42.511858 2579 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/2ad22828-4f7a-4ceb-9d0c-5f96b842878b-istio-token\") pod \"e2e-mt-c6d643e5-openshift-default-5db8d64d55-jb4z8\" (UID: \"2ad22828-4f7a-4ceb-9d0c-5f96b842878b\") " pod="openshift-ingress/e2e-mt-c6d643e5-openshift-default-5db8d64d55-jb4z8" Jul 03 11:41:42.512179 ip-10-0-136-0 kubenswrapper[2579]: I0703 11:41:42.512156 2579 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-kptwc\" (UniqueName: \"kubernetes.io/projected/2ad22828-4f7a-4ceb-9d0c-5f96b842878b-kube-api-access-kptwc\") pod \"e2e-mt-c6d643e5-openshift-default-5db8d64d55-jb4z8\" (UID: \"2ad22828-4f7a-4ceb-9d0c-5f96b842878b\") " pod="openshift-ingress/e2e-mt-c6d643e5-openshift-default-5db8d64d55-jb4z8" Jul 03 11:41:42.554674 ip-10-0-136-0 kubenswrapper[2579]: I0703 11:41:42.554630 2579 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-mt-c6d643e5-openshift-default-5db8d64d55-jb4z8" Jul 03 11:41:42.736008 ip-10-0-136-0 kubenswrapper[2579]: I0703 11:41:42.735970 2579 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/e2e-mt-c6d643e5-openshift-default-5db8d64d55-jb4z8"] Jul 03 11:41:42.736443 ip-10-0-136-0 kubenswrapper[2579]: W0703 11:41:42.736393 2579 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod2ad22828_4f7a_4ceb_9d0c_5f96b842878b.slice/crio-665e997ce998ffd87c3b2d92c765b2981b5a0c622aaa3855773cbe8b76457d43 WatchSource:0}: Error finding container 665e997ce998ffd87c3b2d92c765b2981b5a0c622aaa3855773cbe8b76457d43: Status 404 returned error can't find the container with id 665e997ce998ffd87c3b2d92c765b2981b5a0c622aaa3855773cbe8b76457d43 Jul 03 11:41:42.739297 ip-10-0-136-0 kubenswrapper[2579]: I0703 11:41:42.739261 2579 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"30892156Ki","pods":"250"} Jul 03 11:41:42.739414 ip-10-0-136-0 kubenswrapper[2579]: I0703 11:41:42.739337 2579 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"30892156Ki","pods":"250"} Jul 03 11:41:42.739414 ip-10-0-136-0 kubenswrapper[2579]: I0703 11:41:42.739373 2579 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"30892156Ki","pods":"250"} Jul 03 11:41:43.524287 ip-10-0-136-0 kubenswrapper[2579]: I0703 11:41:43.524245 2579 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-mt-c6d643e5-openshift-default-5db8d64d55-jb4z8" event={"ID":"2ad22828-4f7a-4ceb-9d0c-5f96b842878b","Type":"ContainerStarted","Data":"8af45627b2fa83aab3554abf3555bccb865d28babe5f8d2415cfb43a65465c8b"} Jul 03 11:41:43.524984 ip-10-0-136-0 kubenswrapper[2579]: I0703 11:41:43.524307 2579 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-mt-c6d643e5-openshift-default-5db8d64d55-jb4z8" event={"ID":"2ad22828-4f7a-4ceb-9d0c-5f96b842878b","Type":"ContainerStarted","Data":"665e997ce998ffd87c3b2d92c765b2981b5a0c622aaa3855773cbe8b76457d43"} Jul 03 11:41:43.555516 ip-10-0-136-0 kubenswrapper[2579]: I0703 11:41:43.555476 2579 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="openshift-ingress/e2e-mt-c6d643e5-openshift-default-5db8d64d55-jb4z8" Jul 03 11:41:43.557629 ip-10-0-136-0 kubenswrapper[2579]: I0703 11:41:43.557566 2579 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-ingress/e2e-mt-c6d643e5-openshift-default-5db8d64d55-jb4z8" podStartSLOduration=1.55754117 podStartE2EDuration="1.55754117s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-03 11:41:42 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-03 11:41:43.553674154 +0000 UTC m=+1760.141978476" watchObservedRunningTime="2026-07-03 11:41:43.55754117 +0000 UTC m=+1760.145845491" Jul 03 11:41:43.561484 ip-10-0-136-0 kubenswrapper[2579]: I0703 11:41:43.561455 2579 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="openshift-ingress/e2e-mt-c6d643e5-openshift-default-5db8d64d55-jb4z8" Jul 03 11:41:44.541436 ip-10-0-136-0 kubenswrapper[2579]: I0703 11:41:44.541085 2579 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/e2e-mt-c6d643e5-openshift-default-5db8d64d55-jb4z8" Jul 03 11:41:44.542688 ip-10-0-136-0 kubenswrapper[2579]: I0703 11:41:44.542632 2579 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/e2e-mt-c6d643e5-openshift-default-5db8d64d55-jb4z8" Jul 03 11:42:00.582441 ip-10-0-136-0 kubenswrapper[2579]: I0703 11:42:00.582377 2579 patch_prober.go:28] interesting pod/e2e-mt-c6d643e5-openshift-default-5db8d64d55-jb4z8 container/istio-proxy namespace/openshift-ingress: Readiness probe status=failure output="Get \"http://10.133.0.66:15021/healthz/ready\": context deadline exceeded (Client.Timeout exceeded while awaiting headers)" start-of-body= Jul 03 11:42:00.582928 ip-10-0-136-0 kubenswrapper[2579]: I0703 11:42:00.582505 2579 prober.go:120] "Probe failed" probeType="Readiness" pod="openshift-ingress/e2e-mt-c6d643e5-openshift-default-5db8d64d55-jb4z8" podUID="2ad22828-4f7a-4ceb-9d0c-5f96b842878b" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.133.0.66:15021/healthz/ready\": context deadline exceeded (Client.Timeout exceeded while awaiting headers)" Jul 03 11:42:01.646655 ip-10-0-136-0 kubenswrapper[2579]: I0703 11:42:01.646612 2579 generic.go:356] "Generic (PLEG): container finished" podID="2ad22828-4f7a-4ceb-9d0c-5f96b842878b" containerID="8af45627b2fa83aab3554abf3555bccb865d28babe5f8d2415cfb43a65465c8b" exitCode=0 Jul 03 11:42:01.647214 ip-10-0-136-0 kubenswrapper[2579]: I0703 11:42:01.646685 2579 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-mt-c6d643e5-openshift-default-5db8d64d55-jb4z8" event={"ID":"2ad22828-4f7a-4ceb-9d0c-5f96b842878b","Type":"ContainerDied","Data":"8af45627b2fa83aab3554abf3555bccb865d28babe5f8d2415cfb43a65465c8b"} Jul 03 11:42:01.647291 ip-10-0-136-0 kubenswrapper[2579]: I0703 11:42:01.647253 2579 scope.go:117] "RemoveContainer" containerID="8af45627b2fa83aab3554abf3555bccb865d28babe5f8d2415cfb43a65465c8b" Jul 03 11:42:01.648614 ip-10-0-136-0 kubenswrapper[2579]: I0703 11:42:01.648577 2579 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"30892156Ki","pods":"250"} Jul 03 11:42:01.648718 ip-10-0-136-0 kubenswrapper[2579]: I0703 11:42:01.648672 2579 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"30892156Ki","pods":"250"} Jul 03 11:42:01.648718 ip-10-0-136-0 kubenswrapper[2579]: I0703 11:42:01.648709 2579 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"30892156Ki","pods":"250"} Jul 03 11:42:02.659585 ip-10-0-136-0 kubenswrapper[2579]: I0703 11:42:02.659535 2579 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-mt-c6d643e5-openshift-default-5db8d64d55-jb4z8" event={"ID":"2ad22828-4f7a-4ceb-9d0c-5f96b842878b","Type":"ContainerStarted","Data":"c010e26da508b421e60279f9bed3417ac2854a8a626c065dabb848aa776dda0b"} Jul 03 11:42:03.555208 ip-10-0-136-0 kubenswrapper[2579]: I0703 11:42:03.555137 2579 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="openshift-ingress/e2e-mt-c6d643e5-openshift-default-5db8d64d55-jb4z8" Jul 03 11:42:03.555440 ip-10-0-136-0 kubenswrapper[2579]: I0703 11:42:03.555310 2579 patch_prober.go:28] interesting pod/e2e-mt-c6d643e5-openshift-default-5db8d64d55-jb4z8 container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.133.0.66:15021/healthz/ready\": dial tcp 10.133.0.66:15021: connect: connection refused" start-of-body= Jul 03 11:42:03.555440 ip-10-0-136-0 kubenswrapper[2579]: I0703 11:42:03.555361 2579 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-mt-c6d643e5-openshift-default-5db8d64d55-jb4z8" podUID="2ad22828-4f7a-4ceb-9d0c-5f96b842878b" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.133.0.66:15021/healthz/ready\": dial tcp 10.133.0.66:15021: connect: connection refused" Jul 03 11:42:04.556257 ip-10-0-136-0 kubenswrapper[2579]: I0703 11:42:04.556184 2579 patch_prober.go:28] interesting pod/e2e-mt-c6d643e5-openshift-default-5db8d64d55-jb4z8 container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.133.0.66:15021/healthz/ready\": dial tcp 10.133.0.66:15021: connect: connection refused" start-of-body= Jul 03 11:42:04.556805 ip-10-0-136-0 kubenswrapper[2579]: I0703 11:42:04.556327 2579 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-mt-c6d643e5-openshift-default-5db8d64d55-jb4z8" podUID="2ad22828-4f7a-4ceb-9d0c-5f96b842878b" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.133.0.66:15021/healthz/ready\": dial tcp 10.133.0.66:15021: connect: connection refused" Jul 03 11:42:05.555225 ip-10-0-136-0 kubenswrapper[2579]: I0703 11:42:05.555161 2579 patch_prober.go:28] interesting pod/e2e-mt-c6d643e5-openshift-default-5db8d64d55-jb4z8 container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.133.0.66:15021/healthz/ready\": dial tcp 10.133.0.66:15021: connect: connection refused" start-of-body= Jul 03 11:42:05.556227 ip-10-0-136-0 kubenswrapper[2579]: I0703 11:42:05.556161 2579 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-mt-c6d643e5-openshift-default-5db8d64d55-jb4z8" podUID="2ad22828-4f7a-4ceb-9d0c-5f96b842878b" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.133.0.66:15021/healthz/ready\": dial tcp 10.133.0.66:15021: connect: connection refused" Jul 03 11:42:06.142219 ip-10-0-136-0 kubenswrapper[2579]: I0703 11:42:06.140235 2579 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-ingress/e2e-mt-c6d643e5-openshift-default-5db8d64d55-jb4z8"] Jul 03 11:42:06.142219 ip-10-0-136-0 kubenswrapper[2579]: I0703 11:42:06.140825 2579 kuberuntime_container.go:872] "Killing container with a grace period" pod="openshift-ingress/e2e-mt-c6d643e5-openshift-default-5db8d64d55-jb4z8" podUID="2ad22828-4f7a-4ceb-9d0c-5f96b842878b" containerName="istio-proxy" containerID="cri-o://c010e26da508b421e60279f9bed3417ac2854a8a626c065dabb848aa776dda0b" gracePeriod=30 Jul 03 11:42:11.923725 ip-10-0-136-0 kubenswrapper[2579]: I0703 11:42:11.923690 2579 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-mt-c6d643e5-openshift-default-5db8d64d55-jb4z8" Jul 03 11:42:12.024704 ip-10-0-136-0 kubenswrapper[2579]: I0703 11:42:12.024663 2579 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/2ad22828-4f7a-4ceb-9d0c-5f96b842878b-workload-certs\") pod \"2ad22828-4f7a-4ceb-9d0c-5f96b842878b\" (UID: \"2ad22828-4f7a-4ceb-9d0c-5f96b842878b\") " Jul 03 11:42:12.024704 ip-10-0-136-0 kubenswrapper[2579]: I0703 11:42:12.024712 2579 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-kptwc\" (UniqueName: \"kubernetes.io/projected/2ad22828-4f7a-4ceb-9d0c-5f96b842878b-kube-api-access-kptwc\") pod \"2ad22828-4f7a-4ceb-9d0c-5f96b842878b\" (UID: \"2ad22828-4f7a-4ceb-9d0c-5f96b842878b\") " Jul 03 11:42:12.024940 ip-10-0-136-0 kubenswrapper[2579]: I0703 11:42:12.024738 2579 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/2ad22828-4f7a-4ceb-9d0c-5f96b842878b-istio-data\") pod \"2ad22828-4f7a-4ceb-9d0c-5f96b842878b\" (UID: \"2ad22828-4f7a-4ceb-9d0c-5f96b842878b\") " Jul 03 11:42:12.024940 ip-10-0-136-0 kubenswrapper[2579]: I0703 11:42:12.024787 2579 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/2ad22828-4f7a-4ceb-9d0c-5f96b842878b-workload-socket\") pod \"2ad22828-4f7a-4ceb-9d0c-5f96b842878b\" (UID: \"2ad22828-4f7a-4ceb-9d0c-5f96b842878b\") " Jul 03 11:42:12.024940 ip-10-0-136-0 kubenswrapper[2579]: I0703 11:42:12.024855 2579 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/2ad22828-4f7a-4ceb-9d0c-5f96b842878b-credential-socket\") pod \"2ad22828-4f7a-4ceb-9d0c-5f96b842878b\" (UID: \"2ad22828-4f7a-4ceb-9d0c-5f96b842878b\") " Jul 03 11:42:12.024940 ip-10-0-136-0 kubenswrapper[2579]: I0703 11:42:12.024893 2579 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/2ad22828-4f7a-4ceb-9d0c-5f96b842878b-istio-envoy\") pod \"2ad22828-4f7a-4ceb-9d0c-5f96b842878b\" (UID: \"2ad22828-4f7a-4ceb-9d0c-5f96b842878b\") " Jul 03 11:42:12.025263 ip-10-0-136-0 kubenswrapper[2579]: I0703 11:42:12.024925 2579 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/2ad22828-4f7a-4ceb-9d0c-5f96b842878b-istio-podinfo\") pod \"2ad22828-4f7a-4ceb-9d0c-5f96b842878b\" (UID: \"2ad22828-4f7a-4ceb-9d0c-5f96b842878b\") " Jul 03 11:42:12.025358 ip-10-0-136-0 kubenswrapper[2579]: I0703 11:42:12.025288 2579 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/2ad22828-4f7a-4ceb-9d0c-5f96b842878b-istio-token\") pod \"2ad22828-4f7a-4ceb-9d0c-5f96b842878b\" (UID: \"2ad22828-4f7a-4ceb-9d0c-5f96b842878b\") " Jul 03 11:42:12.025358 ip-10-0-136-0 kubenswrapper[2579]: I0703 11:42:12.025000 2579 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/2ad22828-4f7a-4ceb-9d0c-5f96b842878b-workload-certs" (OuterVolumeSpecName: "workload-certs") pod "2ad22828-4f7a-4ceb-9d0c-5f96b842878b" (UID: "2ad22828-4f7a-4ceb-9d0c-5f96b842878b"). InnerVolumeSpecName "workload-certs". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Jul 03 11:42:12.025358 ip-10-0-136-0 kubenswrapper[2579]: I0703 11:42:12.025176 2579 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/2ad22828-4f7a-4ceb-9d0c-5f96b842878b-credential-socket" (OuterVolumeSpecName: "credential-socket") pod "2ad22828-4f7a-4ceb-9d0c-5f96b842878b" (UID: "2ad22828-4f7a-4ceb-9d0c-5f96b842878b"). InnerVolumeSpecName "credential-socket". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Jul 03 11:42:12.025513 ip-10-0-136-0 kubenswrapper[2579]: I0703 11:42:12.025216 2579 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/2ad22828-4f7a-4ceb-9d0c-5f96b842878b-workload-socket" (OuterVolumeSpecName: "workload-socket") pod "2ad22828-4f7a-4ceb-9d0c-5f96b842878b" (UID: "2ad22828-4f7a-4ceb-9d0c-5f96b842878b"). InnerVolumeSpecName "workload-socket". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Jul 03 11:42:12.025513 ip-10-0-136-0 kubenswrapper[2579]: I0703 11:42:12.025389 2579 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/2ad22828-4f7a-4ceb-9d0c-5f96b842878b-istiod-ca-cert\") pod \"2ad22828-4f7a-4ceb-9d0c-5f96b842878b\" (UID: \"2ad22828-4f7a-4ceb-9d0c-5f96b842878b\") " Jul 03 11:42:12.025741 ip-10-0-136-0 kubenswrapper[2579]: I0703 11:42:12.025714 2579 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/2ad22828-4f7a-4ceb-9d0c-5f96b842878b-istio-data" (OuterVolumeSpecName: "istio-data") pod "2ad22828-4f7a-4ceb-9d0c-5f96b842878b" (UID: "2ad22828-4f7a-4ceb-9d0c-5f96b842878b"). InnerVolumeSpecName "istio-data". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Jul 03 11:42:12.025839 ip-10-0-136-0 kubenswrapper[2579]: I0703 11:42:12.025821 2579 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/2ad22828-4f7a-4ceb-9d0c-5f96b842878b-istiod-ca-cert" (OuterVolumeSpecName: "istiod-ca-cert") pod "2ad22828-4f7a-4ceb-9d0c-5f96b842878b" (UID: "2ad22828-4f7a-4ceb-9d0c-5f96b842878b"). InnerVolumeSpecName "istiod-ca-cert". PluginName "kubernetes.io/configmap", VolumeGIDValue "" Jul 03 11:42:12.026020 ip-10-0-136-0 kubenswrapper[2579]: I0703 11:42:12.025994 2579 reconciler_common.go:299] "Volume detached for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/2ad22828-4f7a-4ceb-9d0c-5f96b842878b-istiod-ca-cert\") on node \"ip-10-0-136-0.ec2.internal\" DevicePath \"\"" Jul 03 11:42:12.026020 ip-10-0-136-0 kubenswrapper[2579]: I0703 11:42:12.026018 2579 reconciler_common.go:299] "Volume detached for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/2ad22828-4f7a-4ceb-9d0c-5f96b842878b-workload-certs\") on node \"ip-10-0-136-0.ec2.internal\" DevicePath \"\"" Jul 03 11:42:12.026020 ip-10-0-136-0 kubenswrapper[2579]: I0703 11:42:12.026027 2579 reconciler_common.go:299] "Volume detached for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/2ad22828-4f7a-4ceb-9d0c-5f96b842878b-istio-data\") on node \"ip-10-0-136-0.ec2.internal\" DevicePath \"\"" Jul 03 11:42:12.026313 ip-10-0-136-0 kubenswrapper[2579]: I0703 11:42:12.026037 2579 reconciler_common.go:299] "Volume detached for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/2ad22828-4f7a-4ceb-9d0c-5f96b842878b-workload-socket\") on node \"ip-10-0-136-0.ec2.internal\" DevicePath \"\"" Jul 03 11:42:12.026313 ip-10-0-136-0 kubenswrapper[2579]: I0703 11:42:12.026050 2579 reconciler_common.go:299] "Volume detached for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/2ad22828-4f7a-4ceb-9d0c-5f96b842878b-credential-socket\") on node \"ip-10-0-136-0.ec2.internal\" DevicePath \"\"" Jul 03 11:42:12.027634 ip-10-0-136-0 kubenswrapper[2579]: I0703 11:42:12.027582 2579 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/2ad22828-4f7a-4ceb-9d0c-5f96b842878b-istio-envoy" (OuterVolumeSpecName: "istio-envoy") pod "2ad22828-4f7a-4ceb-9d0c-5f96b842878b" (UID: "2ad22828-4f7a-4ceb-9d0c-5f96b842878b"). InnerVolumeSpecName "istio-envoy". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Jul 03 11:42:12.027773 ip-10-0-136-0 kubenswrapper[2579]: I0703 11:42:12.027686 2579 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/2ad22828-4f7a-4ceb-9d0c-5f96b842878b-istio-token" (OuterVolumeSpecName: "istio-token") pod "2ad22828-4f7a-4ceb-9d0c-5f96b842878b" (UID: "2ad22828-4f7a-4ceb-9d0c-5f96b842878b"). InnerVolumeSpecName "istio-token". PluginName "kubernetes.io/projected", VolumeGIDValue "" Jul 03 11:42:12.027827 ip-10-0-136-0 kubenswrapper[2579]: I0703 11:42:12.027782 2579 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/downward-api/2ad22828-4f7a-4ceb-9d0c-5f96b842878b-istio-podinfo" (OuterVolumeSpecName: "istio-podinfo") pod "2ad22828-4f7a-4ceb-9d0c-5f96b842878b" (UID: "2ad22828-4f7a-4ceb-9d0c-5f96b842878b"). InnerVolumeSpecName "istio-podinfo". PluginName "kubernetes.io/downward-api", VolumeGIDValue "" Jul 03 11:42:12.027827 ip-10-0-136-0 kubenswrapper[2579]: I0703 11:42:12.027787 2579 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/2ad22828-4f7a-4ceb-9d0c-5f96b842878b-kube-api-access-kptwc" (OuterVolumeSpecName: "kube-api-access-kptwc") pod "2ad22828-4f7a-4ceb-9d0c-5f96b842878b" (UID: "2ad22828-4f7a-4ceb-9d0c-5f96b842878b"). InnerVolumeSpecName "kube-api-access-kptwc". PluginName "kubernetes.io/projected", VolumeGIDValue "" Jul 03 11:42:12.127588 ip-10-0-136-0 kubenswrapper[2579]: I0703 11:42:12.127542 2579 reconciler_common.go:299] "Volume detached for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/2ad22828-4f7a-4ceb-9d0c-5f96b842878b-istio-envoy\") on node \"ip-10-0-136-0.ec2.internal\" DevicePath \"\"" Jul 03 11:42:12.127588 ip-10-0-136-0 kubenswrapper[2579]: I0703 11:42:12.127579 2579 reconciler_common.go:299] "Volume detached for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/2ad22828-4f7a-4ceb-9d0c-5f96b842878b-istio-podinfo\") on node \"ip-10-0-136-0.ec2.internal\" DevicePath \"\"" Jul 03 11:42:12.127588 ip-10-0-136-0 kubenswrapper[2579]: I0703 11:42:12.127587 2579 reconciler_common.go:299] "Volume detached for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/2ad22828-4f7a-4ceb-9d0c-5f96b842878b-istio-token\") on node \"ip-10-0-136-0.ec2.internal\" DevicePath \"\"" Jul 03 11:42:12.127588 ip-10-0-136-0 kubenswrapper[2579]: I0703 11:42:12.127598 2579 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-kptwc\" (UniqueName: \"kubernetes.io/projected/2ad22828-4f7a-4ceb-9d0c-5f96b842878b-kube-api-access-kptwc\") on node \"ip-10-0-136-0.ec2.internal\" DevicePath \"\"" Jul 03 11:42:12.725248 ip-10-0-136-0 kubenswrapper[2579]: I0703 11:42:12.725157 2579 generic.go:356] "Generic (PLEG): container finished" podID="2ad22828-4f7a-4ceb-9d0c-5f96b842878b" containerID="c010e26da508b421e60279f9bed3417ac2854a8a626c065dabb848aa776dda0b" exitCode=0 Jul 03 11:42:12.725248 ip-10-0-136-0 kubenswrapper[2579]: I0703 11:42:12.725239 2579 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-mt-c6d643e5-openshift-default-5db8d64d55-jb4z8" Jul 03 11:42:12.725248 ip-10-0-136-0 kubenswrapper[2579]: I0703 11:42:12.725236 2579 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-mt-c6d643e5-openshift-default-5db8d64d55-jb4z8" event={"ID":"2ad22828-4f7a-4ceb-9d0c-5f96b842878b","Type":"ContainerDied","Data":"c010e26da508b421e60279f9bed3417ac2854a8a626c065dabb848aa776dda0b"} Jul 03 11:42:12.725539 ip-10-0-136-0 kubenswrapper[2579]: I0703 11:42:12.725302 2579 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-mt-c6d643e5-openshift-default-5db8d64d55-jb4z8" event={"ID":"2ad22828-4f7a-4ceb-9d0c-5f96b842878b","Type":"ContainerDied","Data":"665e997ce998ffd87c3b2d92c765b2981b5a0c622aaa3855773cbe8b76457d43"} Jul 03 11:42:12.725539 ip-10-0-136-0 kubenswrapper[2579]: I0703 11:42:12.725327 2579 scope.go:117] "RemoveContainer" containerID="c010e26da508b421e60279f9bed3417ac2854a8a626c065dabb848aa776dda0b" Jul 03 11:42:12.739940 ip-10-0-136-0 kubenswrapper[2579]: I0703 11:42:12.739912 2579 scope.go:117] "RemoveContainer" containerID="8af45627b2fa83aab3554abf3555bccb865d28babe5f8d2415cfb43a65465c8b" Jul 03 11:42:12.750150 ip-10-0-136-0 kubenswrapper[2579]: I0703 11:42:12.750120 2579 scope.go:117] "RemoveContainer" containerID="c010e26da508b421e60279f9bed3417ac2854a8a626c065dabb848aa776dda0b" Jul 03 11:42:12.750555 ip-10-0-136-0 kubenswrapper[2579]: E0703 11:42:12.750532 2579 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"c010e26da508b421e60279f9bed3417ac2854a8a626c065dabb848aa776dda0b\": container with ID starting with c010e26da508b421e60279f9bed3417ac2854a8a626c065dabb848aa776dda0b not found: ID does not exist" containerID="c010e26da508b421e60279f9bed3417ac2854a8a626c065dabb848aa776dda0b" Jul 03 11:42:12.750662 ip-10-0-136-0 kubenswrapper[2579]: I0703 11:42:12.750564 2579 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"c010e26da508b421e60279f9bed3417ac2854a8a626c065dabb848aa776dda0b"} err="failed to get container status \"c010e26da508b421e60279f9bed3417ac2854a8a626c065dabb848aa776dda0b\": rpc error: code = NotFound desc = could not find container \"c010e26da508b421e60279f9bed3417ac2854a8a626c065dabb848aa776dda0b\": container with ID starting with c010e26da508b421e60279f9bed3417ac2854a8a626c065dabb848aa776dda0b not found: ID does not exist" Jul 03 11:42:12.750662 ip-10-0-136-0 kubenswrapper[2579]: I0703 11:42:12.750585 2579 scope.go:117] "RemoveContainer" containerID="8af45627b2fa83aab3554abf3555bccb865d28babe5f8d2415cfb43a65465c8b" Jul 03 11:42:12.750901 ip-10-0-136-0 kubenswrapper[2579]: E0703 11:42:12.750879 2579 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"8af45627b2fa83aab3554abf3555bccb865d28babe5f8d2415cfb43a65465c8b\": container with ID starting with 8af45627b2fa83aab3554abf3555bccb865d28babe5f8d2415cfb43a65465c8b not found: ID does not exist" containerID="8af45627b2fa83aab3554abf3555bccb865d28babe5f8d2415cfb43a65465c8b" Jul 03 11:42:12.750967 ip-10-0-136-0 kubenswrapper[2579]: I0703 11:42:12.750912 2579 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"8af45627b2fa83aab3554abf3555bccb865d28babe5f8d2415cfb43a65465c8b"} err="failed to get container status \"8af45627b2fa83aab3554abf3555bccb865d28babe5f8d2415cfb43a65465c8b\": rpc error: code = NotFound desc = could not find container \"8af45627b2fa83aab3554abf3555bccb865d28babe5f8d2415cfb43a65465c8b\": container with ID starting with 8af45627b2fa83aab3554abf3555bccb865d28babe5f8d2415cfb43a65465c8b not found: ID does not exist" Jul 03 11:42:12.756962 ip-10-0-136-0 kubenswrapper[2579]: I0703 11:42:12.756916 2579 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-ingress/e2e-mt-c6d643e5-openshift-default-5db8d64d55-jb4z8"] Jul 03 11:42:12.767949 ip-10-0-136-0 kubenswrapper[2579]: I0703 11:42:12.767913 2579 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["openshift-ingress/e2e-mt-c6d643e5-openshift-default-5db8d64d55-jb4z8"] Jul 03 11:42:13.993386 ip-10-0-136-0 kubenswrapper[2579]: I0703 11:42:13.993305 2579 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="2ad22828-4f7a-4ceb-9d0c-5f96b842878b" path="/var/lib/kubelet/pods/2ad22828-4f7a-4ceb-9d0c-5f96b842878b/volumes" Jul 03 11:42:28.980268 ip-10-0-136-0 kubenswrapper[2579]: I0703 11:42:28.980225 2579 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-ingress/e2e-mt-83f02be8-openshift-default-6f677c6cdc-zk8xp"] Jul 03 11:42:28.980795 ip-10-0-136-0 kubenswrapper[2579]: I0703 11:42:28.980686 2579 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="2ad22828-4f7a-4ceb-9d0c-5f96b842878b" containerName="istio-proxy" Jul 03 11:42:28.980795 ip-10-0-136-0 kubenswrapper[2579]: I0703 11:42:28.980702 2579 state_mem.go:107] "Deleted CPUSet assignment" podUID="2ad22828-4f7a-4ceb-9d0c-5f96b842878b" containerName="istio-proxy" Jul 03 11:42:28.980795 ip-10-0-136-0 kubenswrapper[2579]: I0703 11:42:28.980781 2579 memory_manager.go:368] "RemoveStaleState removing state" podUID="2ad22828-4f7a-4ceb-9d0c-5f96b842878b" containerName="istio-proxy" Jul 03 11:42:28.980795 ip-10-0-136-0 kubenswrapper[2579]: I0703 11:42:28.980791 2579 memory_manager.go:368] "RemoveStaleState removing state" podUID="2ad22828-4f7a-4ceb-9d0c-5f96b842878b" containerName="istio-proxy" Jul 03 11:42:28.985089 ip-10-0-136-0 kubenswrapper[2579]: I0703 11:42:28.985026 2579 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-mt-83f02be8-openshift-default-6f677c6cdc-zk8xp" Jul 03 11:42:28.987725 ip-10-0-136-0 kubenswrapper[2579]: I0703 11:42:28.987694 2579 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress\"/\"e2e-mt-83f02be8-openshift-default-dockercfg-fg89b\"" Jul 03 11:42:29.024611 ip-10-0-136-0 kubenswrapper[2579]: I0703 11:42:29.024567 2579 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/e2e-mt-83f02be8-openshift-default-6f677c6cdc-zk8xp"] Jul 03 11:42:29.081450 ip-10-0-136-0 kubenswrapper[2579]: I0703 11:42:29.081411 2579 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/a3199277-65dd-4682-bad6-056dba00d4b8-credential-socket\") pod \"e2e-mt-83f02be8-openshift-default-6f677c6cdc-zk8xp\" (UID: \"a3199277-65dd-4682-bad6-056dba00d4b8\") " pod="openshift-ingress/e2e-mt-83f02be8-openshift-default-6f677c6cdc-zk8xp" Jul 03 11:42:29.081660 ip-10-0-136-0 kubenswrapper[2579]: I0703 11:42:29.081472 2579 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/a3199277-65dd-4682-bad6-056dba00d4b8-istio-podinfo\") pod \"e2e-mt-83f02be8-openshift-default-6f677c6cdc-zk8xp\" (UID: \"a3199277-65dd-4682-bad6-056dba00d4b8\") " pod="openshift-ingress/e2e-mt-83f02be8-openshift-default-6f677c6cdc-zk8xp" Jul 03 11:42:29.081660 ip-10-0-136-0 kubenswrapper[2579]: I0703 11:42:29.081548 2579 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/a3199277-65dd-4682-bad6-056dba00d4b8-workload-socket\") pod \"e2e-mt-83f02be8-openshift-default-6f677c6cdc-zk8xp\" (UID: \"a3199277-65dd-4682-bad6-056dba00d4b8\") " pod="openshift-ingress/e2e-mt-83f02be8-openshift-default-6f677c6cdc-zk8xp" Jul 03 11:42:29.081660 ip-10-0-136-0 kubenswrapper[2579]: I0703 11:42:29.081594 2579 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/a3199277-65dd-4682-bad6-056dba00d4b8-istio-token\") pod \"e2e-mt-83f02be8-openshift-default-6f677c6cdc-zk8xp\" (UID: \"a3199277-65dd-4682-bad6-056dba00d4b8\") " pod="openshift-ingress/e2e-mt-83f02be8-openshift-default-6f677c6cdc-zk8xp" Jul 03 11:42:29.081660 ip-10-0-136-0 kubenswrapper[2579]: I0703 11:42:29.081645 2579 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/a3199277-65dd-4682-bad6-056dba00d4b8-istio-envoy\") pod \"e2e-mt-83f02be8-openshift-default-6f677c6cdc-zk8xp\" (UID: \"a3199277-65dd-4682-bad6-056dba00d4b8\") " pod="openshift-ingress/e2e-mt-83f02be8-openshift-default-6f677c6cdc-zk8xp" Jul 03 11:42:29.081889 ip-10-0-136-0 kubenswrapper[2579]: I0703 11:42:29.081665 2579 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/a3199277-65dd-4682-bad6-056dba00d4b8-istio-data\") pod \"e2e-mt-83f02be8-openshift-default-6f677c6cdc-zk8xp\" (UID: \"a3199277-65dd-4682-bad6-056dba00d4b8\") " pod="openshift-ingress/e2e-mt-83f02be8-openshift-default-6f677c6cdc-zk8xp" Jul 03 11:42:29.081889 ip-10-0-136-0 kubenswrapper[2579]: I0703 11:42:29.081699 2579 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/a3199277-65dd-4682-bad6-056dba00d4b8-istiod-ca-cert\") pod \"e2e-mt-83f02be8-openshift-default-6f677c6cdc-zk8xp\" (UID: \"a3199277-65dd-4682-bad6-056dba00d4b8\") " pod="openshift-ingress/e2e-mt-83f02be8-openshift-default-6f677c6cdc-zk8xp" Jul 03 11:42:29.081889 ip-10-0-136-0 kubenswrapper[2579]: I0703 11:42:29.081817 2579 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/a3199277-65dd-4682-bad6-056dba00d4b8-workload-certs\") pod \"e2e-mt-83f02be8-openshift-default-6f677c6cdc-zk8xp\" (UID: \"a3199277-65dd-4682-bad6-056dba00d4b8\") " pod="openshift-ingress/e2e-mt-83f02be8-openshift-default-6f677c6cdc-zk8xp" Jul 03 11:42:29.082020 ip-10-0-136-0 kubenswrapper[2579]: I0703 11:42:29.081886 2579 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-ndfwk\" (UniqueName: \"kubernetes.io/projected/a3199277-65dd-4682-bad6-056dba00d4b8-kube-api-access-ndfwk\") pod \"e2e-mt-83f02be8-openshift-default-6f677c6cdc-zk8xp\" (UID: \"a3199277-65dd-4682-bad6-056dba00d4b8\") " pod="openshift-ingress/e2e-mt-83f02be8-openshift-default-6f677c6cdc-zk8xp" Jul 03 11:42:29.182399 ip-10-0-136-0 kubenswrapper[2579]: I0703 11:42:29.182354 2579 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/a3199277-65dd-4682-bad6-056dba00d4b8-istio-envoy\") pod \"e2e-mt-83f02be8-openshift-default-6f677c6cdc-zk8xp\" (UID: \"a3199277-65dd-4682-bad6-056dba00d4b8\") " pod="openshift-ingress/e2e-mt-83f02be8-openshift-default-6f677c6cdc-zk8xp" Jul 03 11:42:29.182399 ip-10-0-136-0 kubenswrapper[2579]: I0703 11:42:29.182402 2579 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/a3199277-65dd-4682-bad6-056dba00d4b8-istio-data\") pod \"e2e-mt-83f02be8-openshift-default-6f677c6cdc-zk8xp\" (UID: \"a3199277-65dd-4682-bad6-056dba00d4b8\") " pod="openshift-ingress/e2e-mt-83f02be8-openshift-default-6f677c6cdc-zk8xp" Jul 03 11:42:29.182705 ip-10-0-136-0 kubenswrapper[2579]: I0703 11:42:29.182436 2579 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/a3199277-65dd-4682-bad6-056dba00d4b8-istiod-ca-cert\") pod \"e2e-mt-83f02be8-openshift-default-6f677c6cdc-zk8xp\" (UID: \"a3199277-65dd-4682-bad6-056dba00d4b8\") " pod="openshift-ingress/e2e-mt-83f02be8-openshift-default-6f677c6cdc-zk8xp" Jul 03 11:42:29.182705 ip-10-0-136-0 kubenswrapper[2579]: I0703 11:42:29.182475 2579 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/a3199277-65dd-4682-bad6-056dba00d4b8-workload-certs\") pod \"e2e-mt-83f02be8-openshift-default-6f677c6cdc-zk8xp\" (UID: \"a3199277-65dd-4682-bad6-056dba00d4b8\") " pod="openshift-ingress/e2e-mt-83f02be8-openshift-default-6f677c6cdc-zk8xp" Jul 03 11:42:29.182705 ip-10-0-136-0 kubenswrapper[2579]: I0703 11:42:29.182526 2579 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-ndfwk\" (UniqueName: \"kubernetes.io/projected/a3199277-65dd-4682-bad6-056dba00d4b8-kube-api-access-ndfwk\") pod \"e2e-mt-83f02be8-openshift-default-6f677c6cdc-zk8xp\" (UID: \"a3199277-65dd-4682-bad6-056dba00d4b8\") " pod="openshift-ingress/e2e-mt-83f02be8-openshift-default-6f677c6cdc-zk8xp" Jul 03 11:42:29.182705 ip-10-0-136-0 kubenswrapper[2579]: I0703 11:42:29.182552 2579 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/a3199277-65dd-4682-bad6-056dba00d4b8-credential-socket\") pod \"e2e-mt-83f02be8-openshift-default-6f677c6cdc-zk8xp\" (UID: \"a3199277-65dd-4682-bad6-056dba00d4b8\") " pod="openshift-ingress/e2e-mt-83f02be8-openshift-default-6f677c6cdc-zk8xp" Jul 03 11:42:29.182705 ip-10-0-136-0 kubenswrapper[2579]: I0703 11:42:29.182593 2579 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/a3199277-65dd-4682-bad6-056dba00d4b8-istio-podinfo\") pod \"e2e-mt-83f02be8-openshift-default-6f677c6cdc-zk8xp\" (UID: \"a3199277-65dd-4682-bad6-056dba00d4b8\") " pod="openshift-ingress/e2e-mt-83f02be8-openshift-default-6f677c6cdc-zk8xp" Jul 03 11:42:29.182705 ip-10-0-136-0 kubenswrapper[2579]: I0703 11:42:29.182630 2579 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/a3199277-65dd-4682-bad6-056dba00d4b8-workload-socket\") pod \"e2e-mt-83f02be8-openshift-default-6f677c6cdc-zk8xp\" (UID: \"a3199277-65dd-4682-bad6-056dba00d4b8\") " pod="openshift-ingress/e2e-mt-83f02be8-openshift-default-6f677c6cdc-zk8xp" Jul 03 11:42:29.182705 ip-10-0-136-0 kubenswrapper[2579]: I0703 11:42:29.182665 2579 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/a3199277-65dd-4682-bad6-056dba00d4b8-istio-token\") pod \"e2e-mt-83f02be8-openshift-default-6f677c6cdc-zk8xp\" (UID: \"a3199277-65dd-4682-bad6-056dba00d4b8\") " pod="openshift-ingress/e2e-mt-83f02be8-openshift-default-6f677c6cdc-zk8xp" Jul 03 11:42:29.183107 ip-10-0-136-0 kubenswrapper[2579]: I0703 11:42:29.182930 2579 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/a3199277-65dd-4682-bad6-056dba00d4b8-istio-data\") pod \"e2e-mt-83f02be8-openshift-default-6f677c6cdc-zk8xp\" (UID: \"a3199277-65dd-4682-bad6-056dba00d4b8\") " pod="openshift-ingress/e2e-mt-83f02be8-openshift-default-6f677c6cdc-zk8xp" Jul 03 11:42:29.183107 ip-10-0-136-0 kubenswrapper[2579]: I0703 11:42:29.182946 2579 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/a3199277-65dd-4682-bad6-056dba00d4b8-workload-certs\") pod \"e2e-mt-83f02be8-openshift-default-6f677c6cdc-zk8xp\" (UID: \"a3199277-65dd-4682-bad6-056dba00d4b8\") " pod="openshift-ingress/e2e-mt-83f02be8-openshift-default-6f677c6cdc-zk8xp" Jul 03 11:42:29.183269 ip-10-0-136-0 kubenswrapper[2579]: I0703 11:42:29.183225 2579 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/a3199277-65dd-4682-bad6-056dba00d4b8-workload-socket\") pod \"e2e-mt-83f02be8-openshift-default-6f677c6cdc-zk8xp\" (UID: \"a3199277-65dd-4682-bad6-056dba00d4b8\") " pod="openshift-ingress/e2e-mt-83f02be8-openshift-default-6f677c6cdc-zk8xp" Jul 03 11:42:29.183407 ip-10-0-136-0 kubenswrapper[2579]: I0703 11:42:29.183273 2579 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/a3199277-65dd-4682-bad6-056dba00d4b8-credential-socket\") pod \"e2e-mt-83f02be8-openshift-default-6f677c6cdc-zk8xp\" (UID: \"a3199277-65dd-4682-bad6-056dba00d4b8\") " pod="openshift-ingress/e2e-mt-83f02be8-openshift-default-6f677c6cdc-zk8xp" Jul 03 11:42:29.183866 ip-10-0-136-0 kubenswrapper[2579]: I0703 11:42:29.183829 2579 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/a3199277-65dd-4682-bad6-056dba00d4b8-istiod-ca-cert\") pod \"e2e-mt-83f02be8-openshift-default-6f677c6cdc-zk8xp\" (UID: \"a3199277-65dd-4682-bad6-056dba00d4b8\") " pod="openshift-ingress/e2e-mt-83f02be8-openshift-default-6f677c6cdc-zk8xp" Jul 03 11:42:29.185100 ip-10-0-136-0 kubenswrapper[2579]: I0703 11:42:29.185074 2579 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/a3199277-65dd-4682-bad6-056dba00d4b8-istio-envoy\") pod \"e2e-mt-83f02be8-openshift-default-6f677c6cdc-zk8xp\" (UID: \"a3199277-65dd-4682-bad6-056dba00d4b8\") " pod="openshift-ingress/e2e-mt-83f02be8-openshift-default-6f677c6cdc-zk8xp" Jul 03 11:42:29.185592 ip-10-0-136-0 kubenswrapper[2579]: I0703 11:42:29.185571 2579 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/a3199277-65dd-4682-bad6-056dba00d4b8-istio-podinfo\") pod \"e2e-mt-83f02be8-openshift-default-6f677c6cdc-zk8xp\" (UID: \"a3199277-65dd-4682-bad6-056dba00d4b8\") " pod="openshift-ingress/e2e-mt-83f02be8-openshift-default-6f677c6cdc-zk8xp" Jul 03 11:42:29.216422 ip-10-0-136-0 kubenswrapper[2579]: I0703 11:42:29.216386 2579 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/a3199277-65dd-4682-bad6-056dba00d4b8-istio-token\") pod \"e2e-mt-83f02be8-openshift-default-6f677c6cdc-zk8xp\" (UID: \"a3199277-65dd-4682-bad6-056dba00d4b8\") " pod="openshift-ingress/e2e-mt-83f02be8-openshift-default-6f677c6cdc-zk8xp" Jul 03 11:42:29.217471 ip-10-0-136-0 kubenswrapper[2579]: I0703 11:42:29.217439 2579 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-ndfwk\" (UniqueName: \"kubernetes.io/projected/a3199277-65dd-4682-bad6-056dba00d4b8-kube-api-access-ndfwk\") pod \"e2e-mt-83f02be8-openshift-default-6f677c6cdc-zk8xp\" (UID: \"a3199277-65dd-4682-bad6-056dba00d4b8\") " pod="openshift-ingress/e2e-mt-83f02be8-openshift-default-6f677c6cdc-zk8xp" Jul 03 11:42:29.298894 ip-10-0-136-0 kubenswrapper[2579]: I0703 11:42:29.298851 2579 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-mt-83f02be8-openshift-default-6f677c6cdc-zk8xp" Jul 03 11:42:29.470685 ip-10-0-136-0 kubenswrapper[2579]: I0703 11:42:29.470638 2579 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/e2e-mt-83f02be8-openshift-default-6f677c6cdc-zk8xp"] Jul 03 11:42:29.472532 ip-10-0-136-0 kubenswrapper[2579]: W0703 11:42:29.472491 2579 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-poda3199277_65dd_4682_bad6_056dba00d4b8.slice/crio-0713c02226bdbfc124080be1f1ca7cad2f6d1595abc65cae278e2110a4c95149 WatchSource:0}: Error finding container 0713c02226bdbfc124080be1f1ca7cad2f6d1595abc65cae278e2110a4c95149: Status 404 returned error can't find the container with id 0713c02226bdbfc124080be1f1ca7cad2f6d1595abc65cae278e2110a4c95149 Jul 03 11:42:29.475337 ip-10-0-136-0 kubenswrapper[2579]: I0703 11:42:29.475302 2579 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"30892156Ki","pods":"250"} Jul 03 11:42:29.475441 ip-10-0-136-0 kubenswrapper[2579]: I0703 11:42:29.475388 2579 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"30892156Ki","pods":"250"} Jul 03 11:42:29.475441 ip-10-0-136-0 kubenswrapper[2579]: I0703 11:42:29.475423 2579 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"30892156Ki","pods":"250"} Jul 03 11:42:29.831956 ip-10-0-136-0 kubenswrapper[2579]: I0703 11:42:29.831910 2579 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-mt-83f02be8-openshift-default-6f677c6cdc-zk8xp" event={"ID":"a3199277-65dd-4682-bad6-056dba00d4b8","Type":"ContainerStarted","Data":"4265fd58e1d8cf384d366434cc2cb17f59db91578d25f3ff5619754bdbdb53d4"} Jul 03 11:42:29.832218 ip-10-0-136-0 kubenswrapper[2579]: I0703 11:42:29.831984 2579 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-mt-83f02be8-openshift-default-6f677c6cdc-zk8xp" event={"ID":"a3199277-65dd-4682-bad6-056dba00d4b8","Type":"ContainerStarted","Data":"0713c02226bdbfc124080be1f1ca7cad2f6d1595abc65cae278e2110a4c95149"} Jul 03 11:42:29.879267 ip-10-0-136-0 kubenswrapper[2579]: I0703 11:42:29.879144 2579 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-ingress/e2e-mt-83f02be8-openshift-default-6f677c6cdc-zk8xp" podStartSLOduration=1.8791228530000001 podStartE2EDuration="1.879122853s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-03 11:42:28 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-03 11:42:29.878724728 +0000 UTC m=+1806.467029048" watchObservedRunningTime="2026-07-03 11:42:29.879122853 +0000 UTC m=+1806.467427177" Jul 03 11:42:30.300078 ip-10-0-136-0 kubenswrapper[2579]: I0703 11:42:30.300032 2579 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="openshift-ingress/e2e-mt-83f02be8-openshift-default-6f677c6cdc-zk8xp" Jul 03 11:42:30.301744 ip-10-0-136-0 kubenswrapper[2579]: I0703 11:42:30.301718 2579 patch_prober.go:28] interesting pod/e2e-mt-83f02be8-openshift-default-6f677c6cdc-zk8xp container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.133.0.67:15021/healthz/ready\": dial tcp 10.133.0.67:15021: connect: connection refused" start-of-body= Jul 03 11:42:30.301814 ip-10-0-136-0 kubenswrapper[2579]: I0703 11:42:30.301778 2579 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-mt-83f02be8-openshift-default-6f677c6cdc-zk8xp" podUID="a3199277-65dd-4682-bad6-056dba00d4b8" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.133.0.67:15021/healthz/ready\": dial tcp 10.133.0.67:15021: connect: connection refused" Jul 03 11:42:31.304701 ip-10-0-136-0 kubenswrapper[2579]: I0703 11:42:31.304664 2579 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="openshift-ingress/e2e-mt-83f02be8-openshift-default-6f677c6cdc-zk8xp" Jul 03 11:42:31.845086 ip-10-0-136-0 kubenswrapper[2579]: I0703 11:42:31.845050 2579 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/e2e-mt-83f02be8-openshift-default-6f677c6cdc-zk8xp" Jul 03 11:42:31.846288 ip-10-0-136-0 kubenswrapper[2579]: I0703 11:42:31.846259 2579 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/e2e-mt-83f02be8-openshift-default-6f677c6cdc-zk8xp" Jul 03 11:42:47.110122 ip-10-0-136-0 kubenswrapper[2579]: I0703 11:42:47.109737 2579 patch_prober.go:28] interesting pod/e2e-mt-83f02be8-openshift-default-6f677c6cdc-zk8xp container/istio-proxy namespace/openshift-ingress: Readiness probe status=failure output="Get \"http://10.133.0.67:15021/healthz/ready\": read tcp 10.133.0.2:55428->10.133.0.67:15021: read: connection reset by peer" start-of-body= Jul 03 11:42:47.110122 ip-10-0-136-0 kubenswrapper[2579]: I0703 11:42:47.110020 2579 prober.go:120] "Probe failed" probeType="Readiness" pod="openshift-ingress/e2e-mt-83f02be8-openshift-default-6f677c6cdc-zk8xp" podUID="a3199277-65dd-4682-bad6-056dba00d4b8" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.133.0.67:15021/healthz/ready\": read tcp 10.133.0.2:55428->10.133.0.67:15021: read: connection reset by peer" Jul 03 11:42:47.946847 ip-10-0-136-0 kubenswrapper[2579]: I0703 11:42:47.946805 2579 generic.go:356] "Generic (PLEG): container finished" podID="a3199277-65dd-4682-bad6-056dba00d4b8" containerID="4265fd58e1d8cf384d366434cc2cb17f59db91578d25f3ff5619754bdbdb53d4" exitCode=0 Jul 03 11:42:47.947070 ip-10-0-136-0 kubenswrapper[2579]: I0703 11:42:47.946886 2579 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-mt-83f02be8-openshift-default-6f677c6cdc-zk8xp" event={"ID":"a3199277-65dd-4682-bad6-056dba00d4b8","Type":"ContainerDied","Data":"4265fd58e1d8cf384d366434cc2cb17f59db91578d25f3ff5619754bdbdb53d4"} Jul 03 11:42:47.947369 ip-10-0-136-0 kubenswrapper[2579]: I0703 11:42:47.947354 2579 scope.go:117] "RemoveContainer" containerID="4265fd58e1d8cf384d366434cc2cb17f59db91578d25f3ff5619754bdbdb53d4" Jul 03 11:42:47.948655 ip-10-0-136-0 kubenswrapper[2579]: I0703 11:42:47.948623 2579 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"30892156Ki","pods":"250"} Jul 03 11:42:47.948777 ip-10-0-136-0 kubenswrapper[2579]: I0703 11:42:47.948701 2579 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"30892156Ki","pods":"250"} Jul 03 11:42:47.948777 ip-10-0-136-0 kubenswrapper[2579]: I0703 11:42:47.948735 2579 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"30892156Ki","pods":"250"} Jul 03 11:42:48.961009 ip-10-0-136-0 kubenswrapper[2579]: I0703 11:42:48.960961 2579 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-mt-83f02be8-openshift-default-6f677c6cdc-zk8xp" event={"ID":"a3199277-65dd-4682-bad6-056dba00d4b8","Type":"ContainerStarted","Data":"ad421fa0dca2b3eab0defd319fcfbf071351e8ff89678f99ad70e8f70905c727"} Jul 03 11:42:49.299981 ip-10-0-136-0 kubenswrapper[2579]: I0703 11:42:49.299937 2579 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="openshift-ingress/e2e-mt-83f02be8-openshift-default-6f677c6cdc-zk8xp" Jul 03 11:42:49.300298 ip-10-0-136-0 kubenswrapper[2579]: I0703 11:42:49.300271 2579 patch_prober.go:28] interesting pod/e2e-mt-83f02be8-openshift-default-6f677c6cdc-zk8xp container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.133.0.67:15021/healthz/ready\": dial tcp 10.133.0.67:15021: connect: connection refused" start-of-body= Jul 03 11:42:49.300402 ip-10-0-136-0 kubenswrapper[2579]: I0703 11:42:49.300333 2579 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-mt-83f02be8-openshift-default-6f677c6cdc-zk8xp" podUID="a3199277-65dd-4682-bad6-056dba00d4b8" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.133.0.67:15021/healthz/ready\": dial tcp 10.133.0.67:15021: connect: connection refused" Jul 03 11:42:50.299623 ip-10-0-136-0 kubenswrapper[2579]: I0703 11:42:50.299584 2579 patch_prober.go:28] interesting pod/e2e-mt-83f02be8-openshift-default-6f677c6cdc-zk8xp container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.133.0.67:15021/healthz/ready\": dial tcp 10.133.0.67:15021: connect: connection refused" start-of-body= Jul 03 11:42:50.300063 ip-10-0-136-0 kubenswrapper[2579]: I0703 11:42:50.299647 2579 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-mt-83f02be8-openshift-default-6f677c6cdc-zk8xp" podUID="a3199277-65dd-4682-bad6-056dba00d4b8" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.133.0.67:15021/healthz/ready\": dial tcp 10.133.0.67:15021: connect: connection refused" Jul 03 11:42:51.299334 ip-10-0-136-0 kubenswrapper[2579]: I0703 11:42:51.299291 2579 patch_prober.go:28] interesting pod/e2e-mt-83f02be8-openshift-default-6f677c6cdc-zk8xp container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.133.0.67:15021/healthz/ready\": dial tcp 10.133.0.67:15021: connect: connection refused" start-of-body= Jul 03 11:42:51.299505 ip-10-0-136-0 kubenswrapper[2579]: I0703 11:42:51.299355 2579 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-mt-83f02be8-openshift-default-6f677c6cdc-zk8xp" podUID="a3199277-65dd-4682-bad6-056dba00d4b8" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.133.0.67:15021/healthz/ready\": dial tcp 10.133.0.67:15021: connect: connection refused" Jul 03 11:42:52.300078 ip-10-0-136-0 kubenswrapper[2579]: I0703 11:42:52.300037 2579 patch_prober.go:28] interesting pod/e2e-mt-83f02be8-openshift-default-6f677c6cdc-zk8xp container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.133.0.67:15021/healthz/ready\": dial tcp 10.133.0.67:15021: connect: connection refused" start-of-body= Jul 03 11:42:52.300557 ip-10-0-136-0 kubenswrapper[2579]: I0703 11:42:52.300126 2579 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-mt-83f02be8-openshift-default-6f677c6cdc-zk8xp" podUID="a3199277-65dd-4682-bad6-056dba00d4b8" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.133.0.67:15021/healthz/ready\": dial tcp 10.133.0.67:15021: connect: connection refused" Jul 03 11:42:53.303971 ip-10-0-136-0 kubenswrapper[2579]: I0703 11:42:53.303938 2579 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="openshift-ingress/e2e-mt-83f02be8-openshift-default-6f677c6cdc-zk8xp" Jul 03 11:42:53.304401 ip-10-0-136-0 kubenswrapper[2579]: I0703 11:42:53.304225 2579 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/e2e-mt-83f02be8-openshift-default-6f677c6cdc-zk8xp" Jul 03 11:42:53.305084 ip-10-0-136-0 kubenswrapper[2579]: I0703 11:42:53.305061 2579 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/e2e-mt-83f02be8-openshift-default-6f677c6cdc-zk8xp" Jul 03 11:44:30.264182 ip-10-0-136-0 kubenswrapper[2579]: I0703 11:44:30.264136 2579 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-ingress/e2e-mt-83f02be8-openshift-default-6f677c6cdc-zk8xp"] Jul 03 11:44:30.264810 ip-10-0-136-0 kubenswrapper[2579]: I0703 11:44:30.264368 2579 kuberuntime_container.go:872] "Killing container with a grace period" pod="openshift-ingress/e2e-mt-83f02be8-openshift-default-6f677c6cdc-zk8xp" podUID="a3199277-65dd-4682-bad6-056dba00d4b8" containerName="istio-proxy" containerID="cri-o://ad421fa0dca2b3eab0defd319fcfbf071351e8ff89678f99ad70e8f70905c727" gracePeriod=30 Jul 03 11:44:35.541086 ip-10-0-136-0 kubenswrapper[2579]: I0703 11:44:35.541001 2579 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-mt-83f02be8-openshift-default-6f677c6cdc-zk8xp" Jul 03 11:44:35.616045 ip-10-0-136-0 kubenswrapper[2579]: I0703 11:44:35.615971 2579 generic.go:356] "Generic (PLEG): container finished" podID="a3199277-65dd-4682-bad6-056dba00d4b8" containerID="ad421fa0dca2b3eab0defd319fcfbf071351e8ff89678f99ad70e8f70905c727" exitCode=0 Jul 03 11:44:35.616045 ip-10-0-136-0 kubenswrapper[2579]: I0703 11:44:35.616056 2579 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-mt-83f02be8-openshift-default-6f677c6cdc-zk8xp" Jul 03 11:44:35.616512 ip-10-0-136-0 kubenswrapper[2579]: I0703 11:44:35.616077 2579 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-mt-83f02be8-openshift-default-6f677c6cdc-zk8xp" event={"ID":"a3199277-65dd-4682-bad6-056dba00d4b8","Type":"ContainerDied","Data":"ad421fa0dca2b3eab0defd319fcfbf071351e8ff89678f99ad70e8f70905c727"} Jul 03 11:44:35.616512 ip-10-0-136-0 kubenswrapper[2579]: I0703 11:44:35.616161 2579 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-mt-83f02be8-openshift-default-6f677c6cdc-zk8xp" event={"ID":"a3199277-65dd-4682-bad6-056dba00d4b8","Type":"ContainerDied","Data":"0713c02226bdbfc124080be1f1ca7cad2f6d1595abc65cae278e2110a4c95149"} Jul 03 11:44:35.616512 ip-10-0-136-0 kubenswrapper[2579]: I0703 11:44:35.616206 2579 scope.go:117] "RemoveContainer" containerID="ad421fa0dca2b3eab0defd319fcfbf071351e8ff89678f99ad70e8f70905c727" Jul 03 11:44:35.625427 ip-10-0-136-0 kubenswrapper[2579]: I0703 11:44:35.625389 2579 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/a3199277-65dd-4682-bad6-056dba00d4b8-istio-token\") pod \"a3199277-65dd-4682-bad6-056dba00d4b8\" (UID: \"a3199277-65dd-4682-bad6-056dba00d4b8\") " Jul 03 11:44:35.625605 ip-10-0-136-0 kubenswrapper[2579]: I0703 11:44:35.625498 2579 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/a3199277-65dd-4682-bad6-056dba00d4b8-istio-envoy\") pod \"a3199277-65dd-4682-bad6-056dba00d4b8\" (UID: \"a3199277-65dd-4682-bad6-056dba00d4b8\") " Jul 03 11:44:35.625605 ip-10-0-136-0 kubenswrapper[2579]: I0703 11:44:35.625541 2579 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/a3199277-65dd-4682-bad6-056dba00d4b8-istio-podinfo\") pod \"a3199277-65dd-4682-bad6-056dba00d4b8\" (UID: \"a3199277-65dd-4682-bad6-056dba00d4b8\") " Jul 03 11:44:35.625605 ip-10-0-136-0 kubenswrapper[2579]: I0703 11:44:35.625584 2579 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/a3199277-65dd-4682-bad6-056dba00d4b8-credential-socket\") pod \"a3199277-65dd-4682-bad6-056dba00d4b8\" (UID: \"a3199277-65dd-4682-bad6-056dba00d4b8\") " Jul 03 11:44:35.625731 ip-10-0-136-0 kubenswrapper[2579]: I0703 11:44:35.625609 2579 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/a3199277-65dd-4682-bad6-056dba00d4b8-istio-data\") pod \"a3199277-65dd-4682-bad6-056dba00d4b8\" (UID: \"a3199277-65dd-4682-bad6-056dba00d4b8\") " Jul 03 11:44:35.625731 ip-10-0-136-0 kubenswrapper[2579]: I0703 11:44:35.625643 2579 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/a3199277-65dd-4682-bad6-056dba00d4b8-workload-socket\") pod \"a3199277-65dd-4682-bad6-056dba00d4b8\" (UID: \"a3199277-65dd-4682-bad6-056dba00d4b8\") " Jul 03 11:44:35.625731 ip-10-0-136-0 kubenswrapper[2579]: I0703 11:44:35.625674 2579 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/a3199277-65dd-4682-bad6-056dba00d4b8-workload-certs\") pod \"a3199277-65dd-4682-bad6-056dba00d4b8\" (UID: \"a3199277-65dd-4682-bad6-056dba00d4b8\") " Jul 03 11:44:35.625731 ip-10-0-136-0 kubenswrapper[2579]: I0703 11:44:35.625720 2579 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-ndfwk\" (UniqueName: \"kubernetes.io/projected/a3199277-65dd-4682-bad6-056dba00d4b8-kube-api-access-ndfwk\") pod \"a3199277-65dd-4682-bad6-056dba00d4b8\" (UID: \"a3199277-65dd-4682-bad6-056dba00d4b8\") " Jul 03 11:44:35.625931 ip-10-0-136-0 kubenswrapper[2579]: I0703 11:44:35.625791 2579 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/a3199277-65dd-4682-bad6-056dba00d4b8-istiod-ca-cert\") pod \"a3199277-65dd-4682-bad6-056dba00d4b8\" (UID: \"a3199277-65dd-4682-bad6-056dba00d4b8\") " Jul 03 11:44:35.625991 ip-10-0-136-0 kubenswrapper[2579]: I0703 11:44:35.625948 2579 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/a3199277-65dd-4682-bad6-056dba00d4b8-credential-socket" (OuterVolumeSpecName: "credential-socket") pod "a3199277-65dd-4682-bad6-056dba00d4b8" (UID: "a3199277-65dd-4682-bad6-056dba00d4b8"). InnerVolumeSpecName "credential-socket". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Jul 03 11:44:35.626400 ip-10-0-136-0 kubenswrapper[2579]: I0703 11:44:35.626067 2579 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/a3199277-65dd-4682-bad6-056dba00d4b8-workload-certs" (OuterVolumeSpecName: "workload-certs") pod "a3199277-65dd-4682-bad6-056dba00d4b8" (UID: "a3199277-65dd-4682-bad6-056dba00d4b8"). InnerVolumeSpecName "workload-certs". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Jul 03 11:44:35.626400 ip-10-0-136-0 kubenswrapper[2579]: I0703 11:44:35.626170 2579 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/a3199277-65dd-4682-bad6-056dba00d4b8-workload-socket" (OuterVolumeSpecName: "workload-socket") pod "a3199277-65dd-4682-bad6-056dba00d4b8" (UID: "a3199277-65dd-4682-bad6-056dba00d4b8"). InnerVolumeSpecName "workload-socket". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Jul 03 11:44:35.626577 ip-10-0-136-0 kubenswrapper[2579]: I0703 11:44:35.626443 2579 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/a3199277-65dd-4682-bad6-056dba00d4b8-istiod-ca-cert" (OuterVolumeSpecName: "istiod-ca-cert") pod "a3199277-65dd-4682-bad6-056dba00d4b8" (UID: "a3199277-65dd-4682-bad6-056dba00d4b8"). InnerVolumeSpecName "istiod-ca-cert". PluginName "kubernetes.io/configmap", VolumeGIDValue "" Jul 03 11:44:35.626577 ip-10-0-136-0 kubenswrapper[2579]: I0703 11:44:35.626555 2579 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/a3199277-65dd-4682-bad6-056dba00d4b8-istio-data" (OuterVolumeSpecName: "istio-data") pod "a3199277-65dd-4682-bad6-056dba00d4b8" (UID: "a3199277-65dd-4682-bad6-056dba00d4b8"). InnerVolumeSpecName "istio-data". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Jul 03 11:44:35.626700 ip-10-0-136-0 kubenswrapper[2579]: I0703 11:44:35.626586 2579 reconciler_common.go:299] "Volume detached for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/a3199277-65dd-4682-bad6-056dba00d4b8-workload-socket\") on node \"ip-10-0-136-0.ec2.internal\" DevicePath \"\"" Jul 03 11:44:35.626700 ip-10-0-136-0 kubenswrapper[2579]: I0703 11:44:35.626609 2579 reconciler_common.go:299] "Volume detached for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/a3199277-65dd-4682-bad6-056dba00d4b8-workload-certs\") on node \"ip-10-0-136-0.ec2.internal\" DevicePath \"\"" Jul 03 11:44:35.626700 ip-10-0-136-0 kubenswrapper[2579]: I0703 11:44:35.626624 2579 reconciler_common.go:299] "Volume detached for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/a3199277-65dd-4682-bad6-056dba00d4b8-istiod-ca-cert\") on node \"ip-10-0-136-0.ec2.internal\" DevicePath \"\"" Jul 03 11:44:35.626700 ip-10-0-136-0 kubenswrapper[2579]: I0703 11:44:35.626637 2579 reconciler_common.go:299] "Volume detached for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/a3199277-65dd-4682-bad6-056dba00d4b8-credential-socket\") on node \"ip-10-0-136-0.ec2.internal\" DevicePath \"\"" Jul 03 11:44:35.627746 ip-10-0-136-0 kubenswrapper[2579]: I0703 11:44:35.627717 2579 scope.go:117] "RemoveContainer" containerID="4265fd58e1d8cf384d366434cc2cb17f59db91578d25f3ff5619754bdbdb53d4" Jul 03 11:44:35.628154 ip-10-0-136-0 kubenswrapper[2579]: I0703 11:44:35.628128 2579 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/downward-api/a3199277-65dd-4682-bad6-056dba00d4b8-istio-podinfo" (OuterVolumeSpecName: "istio-podinfo") pod "a3199277-65dd-4682-bad6-056dba00d4b8" (UID: "a3199277-65dd-4682-bad6-056dba00d4b8"). InnerVolumeSpecName "istio-podinfo". PluginName "kubernetes.io/downward-api", VolumeGIDValue "" Jul 03 11:44:35.628604 ip-10-0-136-0 kubenswrapper[2579]: I0703 11:44:35.628581 2579 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/a3199277-65dd-4682-bad6-056dba00d4b8-istio-envoy" (OuterVolumeSpecName: "istio-envoy") pod "a3199277-65dd-4682-bad6-056dba00d4b8" (UID: "a3199277-65dd-4682-bad6-056dba00d4b8"). InnerVolumeSpecName "istio-envoy". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Jul 03 11:44:35.628783 ip-10-0-136-0 kubenswrapper[2579]: I0703 11:44:35.628748 2579 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/a3199277-65dd-4682-bad6-056dba00d4b8-istio-token" (OuterVolumeSpecName: "istio-token") pod "a3199277-65dd-4682-bad6-056dba00d4b8" (UID: "a3199277-65dd-4682-bad6-056dba00d4b8"). InnerVolumeSpecName "istio-token". PluginName "kubernetes.io/projected", VolumeGIDValue "" Jul 03 11:44:35.628898 ip-10-0-136-0 kubenswrapper[2579]: I0703 11:44:35.628832 2579 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/a3199277-65dd-4682-bad6-056dba00d4b8-kube-api-access-ndfwk" (OuterVolumeSpecName: "kube-api-access-ndfwk") pod "a3199277-65dd-4682-bad6-056dba00d4b8" (UID: "a3199277-65dd-4682-bad6-056dba00d4b8"). InnerVolumeSpecName "kube-api-access-ndfwk". PluginName "kubernetes.io/projected", VolumeGIDValue "" Jul 03 11:44:35.656544 ip-10-0-136-0 kubenswrapper[2579]: I0703 11:44:35.656518 2579 scope.go:117] "RemoveContainer" containerID="ad421fa0dca2b3eab0defd319fcfbf071351e8ff89678f99ad70e8f70905c727" Jul 03 11:44:35.656953 ip-10-0-136-0 kubenswrapper[2579]: E0703 11:44:35.656926 2579 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"ad421fa0dca2b3eab0defd319fcfbf071351e8ff89678f99ad70e8f70905c727\": container with ID starting with ad421fa0dca2b3eab0defd319fcfbf071351e8ff89678f99ad70e8f70905c727 not found: ID does not exist" containerID="ad421fa0dca2b3eab0defd319fcfbf071351e8ff89678f99ad70e8f70905c727" Jul 03 11:44:35.657021 ip-10-0-136-0 kubenswrapper[2579]: I0703 11:44:35.656963 2579 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"ad421fa0dca2b3eab0defd319fcfbf071351e8ff89678f99ad70e8f70905c727"} err="failed to get container status \"ad421fa0dca2b3eab0defd319fcfbf071351e8ff89678f99ad70e8f70905c727\": rpc error: code = NotFound desc = could not find container \"ad421fa0dca2b3eab0defd319fcfbf071351e8ff89678f99ad70e8f70905c727\": container with ID starting with ad421fa0dca2b3eab0defd319fcfbf071351e8ff89678f99ad70e8f70905c727 not found: ID does not exist" Jul 03 11:44:35.657021 ip-10-0-136-0 kubenswrapper[2579]: I0703 11:44:35.656984 2579 scope.go:117] "RemoveContainer" containerID="4265fd58e1d8cf384d366434cc2cb17f59db91578d25f3ff5619754bdbdb53d4" Jul 03 11:44:35.657335 ip-10-0-136-0 kubenswrapper[2579]: E0703 11:44:35.657296 2579 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"4265fd58e1d8cf384d366434cc2cb17f59db91578d25f3ff5619754bdbdb53d4\": container with ID starting with 4265fd58e1d8cf384d366434cc2cb17f59db91578d25f3ff5619754bdbdb53d4 not found: ID does not exist" containerID="4265fd58e1d8cf384d366434cc2cb17f59db91578d25f3ff5619754bdbdb53d4" Jul 03 11:44:35.657452 ip-10-0-136-0 kubenswrapper[2579]: I0703 11:44:35.657345 2579 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"4265fd58e1d8cf384d366434cc2cb17f59db91578d25f3ff5619754bdbdb53d4"} err="failed to get container status \"4265fd58e1d8cf384d366434cc2cb17f59db91578d25f3ff5619754bdbdb53d4\": rpc error: code = NotFound desc = could not find container \"4265fd58e1d8cf384d366434cc2cb17f59db91578d25f3ff5619754bdbdb53d4\": container with ID starting with 4265fd58e1d8cf384d366434cc2cb17f59db91578d25f3ff5619754bdbdb53d4 not found: ID does not exist" Jul 03 11:44:35.727978 ip-10-0-136-0 kubenswrapper[2579]: I0703 11:44:35.727936 2579 reconciler_common.go:299] "Volume detached for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/a3199277-65dd-4682-bad6-056dba00d4b8-istio-envoy\") on node \"ip-10-0-136-0.ec2.internal\" DevicePath \"\"" Jul 03 11:44:35.727978 ip-10-0-136-0 kubenswrapper[2579]: I0703 11:44:35.727969 2579 reconciler_common.go:299] "Volume detached for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/a3199277-65dd-4682-bad6-056dba00d4b8-istio-podinfo\") on node \"ip-10-0-136-0.ec2.internal\" DevicePath \"\"" Jul 03 11:44:35.727978 ip-10-0-136-0 kubenswrapper[2579]: I0703 11:44:35.727978 2579 reconciler_common.go:299] "Volume detached for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/a3199277-65dd-4682-bad6-056dba00d4b8-istio-data\") on node \"ip-10-0-136-0.ec2.internal\" DevicePath \"\"" Jul 03 11:44:35.727978 ip-10-0-136-0 kubenswrapper[2579]: I0703 11:44:35.727988 2579 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-ndfwk\" (UniqueName: \"kubernetes.io/projected/a3199277-65dd-4682-bad6-056dba00d4b8-kube-api-access-ndfwk\") on node \"ip-10-0-136-0.ec2.internal\" DevicePath \"\"" Jul 03 11:44:35.728327 ip-10-0-136-0 kubenswrapper[2579]: I0703 11:44:35.727999 2579 reconciler_common.go:299] "Volume detached for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/a3199277-65dd-4682-bad6-056dba00d4b8-istio-token\") on node \"ip-10-0-136-0.ec2.internal\" DevicePath \"\"" Jul 03 11:44:35.941013 ip-10-0-136-0 kubenswrapper[2579]: I0703 11:44:35.940967 2579 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-ingress/e2e-mt-83f02be8-openshift-default-6f677c6cdc-zk8xp"] Jul 03 11:44:35.948420 ip-10-0-136-0 kubenswrapper[2579]: I0703 11:44:35.948387 2579 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["openshift-ingress/e2e-mt-83f02be8-openshift-default-6f677c6cdc-zk8xp"] Jul 03 11:44:35.993946 ip-10-0-136-0 kubenswrapper[2579]: I0703 11:44:35.993902 2579 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="a3199277-65dd-4682-bad6-056dba00d4b8" path="/var/lib/kubelet/pods/a3199277-65dd-4682-bad6-056dba00d4b8/volumes" Jul 03 11:45:27.179622 ip-10-0-136-0 kubenswrapper[2579]: I0703 11:45:27.179585 2579 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-ingress/e2e-inf-a-017645-openshift-default-6cc6c59dff-blk4t"] Jul 03 11:45:27.180065 ip-10-0-136-0 kubenswrapper[2579]: I0703 11:45:27.179996 2579 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="2ad22828-4f7a-4ceb-9d0c-5f96b842878b" containerName="istio-proxy" Jul 03 11:45:27.180065 ip-10-0-136-0 kubenswrapper[2579]: I0703 11:45:27.180006 2579 state_mem.go:107] "Deleted CPUSet assignment" podUID="2ad22828-4f7a-4ceb-9d0c-5f96b842878b" containerName="istio-proxy" Jul 03 11:45:27.180065 ip-10-0-136-0 kubenswrapper[2579]: I0703 11:45:27.180019 2579 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="a3199277-65dd-4682-bad6-056dba00d4b8" containerName="istio-proxy" Jul 03 11:45:27.180065 ip-10-0-136-0 kubenswrapper[2579]: I0703 11:45:27.180025 2579 state_mem.go:107] "Deleted CPUSet assignment" podUID="a3199277-65dd-4682-bad6-056dba00d4b8" containerName="istio-proxy" Jul 03 11:45:27.180065 ip-10-0-136-0 kubenswrapper[2579]: I0703 11:45:27.180039 2579 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="a3199277-65dd-4682-bad6-056dba00d4b8" containerName="istio-proxy" Jul 03 11:45:27.180065 ip-10-0-136-0 kubenswrapper[2579]: I0703 11:45:27.180045 2579 state_mem.go:107] "Deleted CPUSet assignment" podUID="a3199277-65dd-4682-bad6-056dba00d4b8" containerName="istio-proxy" Jul 03 11:45:27.180305 ip-10-0-136-0 kubenswrapper[2579]: I0703 11:45:27.180104 2579 memory_manager.go:368] "RemoveStaleState removing state" podUID="a3199277-65dd-4682-bad6-056dba00d4b8" containerName="istio-proxy" Jul 03 11:45:27.180305 ip-10-0-136-0 kubenswrapper[2579]: I0703 11:45:27.180111 2579 memory_manager.go:368] "RemoveStaleState removing state" podUID="a3199277-65dd-4682-bad6-056dba00d4b8" containerName="istio-proxy" Jul 03 11:45:27.183250 ip-10-0-136-0 kubenswrapper[2579]: I0703 11:45:27.183228 2579 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-inf-a-017645-openshift-default-6cc6c59dff-blk4t" Jul 03 11:45:27.186135 ip-10-0-136-0 kubenswrapper[2579]: I0703 11:45:27.186110 2579 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress\"/\"e2e-inf-a-017645-openshift-default-dockercfg-xl2gl\"" Jul 03 11:45:27.200099 ip-10-0-136-0 kubenswrapper[2579]: I0703 11:45:27.200063 2579 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/e2e-inf-a-017645-openshift-default-6cc6c59dff-blk4t"] Jul 03 11:45:27.320499 ip-10-0-136-0 kubenswrapper[2579]: I0703 11:45:27.320448 2579 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/f1b7686d-4c68-4b24-8ae1-a8cde82640cd-istio-podinfo\") pod \"e2e-inf-a-017645-openshift-default-6cc6c59dff-blk4t\" (UID: \"f1b7686d-4c68-4b24-8ae1-a8cde82640cd\") " pod="openshift-ingress/e2e-inf-a-017645-openshift-default-6cc6c59dff-blk4t" Jul 03 11:45:27.320700 ip-10-0-136-0 kubenswrapper[2579]: I0703 11:45:27.320523 2579 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-w92df\" (UniqueName: \"kubernetes.io/projected/f1b7686d-4c68-4b24-8ae1-a8cde82640cd-kube-api-access-w92df\") pod \"e2e-inf-a-017645-openshift-default-6cc6c59dff-blk4t\" (UID: \"f1b7686d-4c68-4b24-8ae1-a8cde82640cd\") " pod="openshift-ingress/e2e-inf-a-017645-openshift-default-6cc6c59dff-blk4t" Jul 03 11:45:27.320700 ip-10-0-136-0 kubenswrapper[2579]: I0703 11:45:27.320623 2579 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/f1b7686d-4c68-4b24-8ae1-a8cde82640cd-workload-socket\") pod \"e2e-inf-a-017645-openshift-default-6cc6c59dff-blk4t\" (UID: \"f1b7686d-4c68-4b24-8ae1-a8cde82640cd\") " pod="openshift-ingress/e2e-inf-a-017645-openshift-default-6cc6c59dff-blk4t" Jul 03 11:45:27.320700 ip-10-0-136-0 kubenswrapper[2579]: I0703 11:45:27.320687 2579 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/f1b7686d-4c68-4b24-8ae1-a8cde82640cd-workload-certs\") pod \"e2e-inf-a-017645-openshift-default-6cc6c59dff-blk4t\" (UID: \"f1b7686d-4c68-4b24-8ae1-a8cde82640cd\") " pod="openshift-ingress/e2e-inf-a-017645-openshift-default-6cc6c59dff-blk4t" Jul 03 11:45:27.320853 ip-10-0-136-0 kubenswrapper[2579]: I0703 11:45:27.320732 2579 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/f1b7686d-4c68-4b24-8ae1-a8cde82640cd-credential-socket\") pod \"e2e-inf-a-017645-openshift-default-6cc6c59dff-blk4t\" (UID: \"f1b7686d-4c68-4b24-8ae1-a8cde82640cd\") " pod="openshift-ingress/e2e-inf-a-017645-openshift-default-6cc6c59dff-blk4t" Jul 03 11:45:27.320853 ip-10-0-136-0 kubenswrapper[2579]: I0703 11:45:27.320763 2579 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/f1b7686d-4c68-4b24-8ae1-a8cde82640cd-istio-envoy\") pod \"e2e-inf-a-017645-openshift-default-6cc6c59dff-blk4t\" (UID: \"f1b7686d-4c68-4b24-8ae1-a8cde82640cd\") " pod="openshift-ingress/e2e-inf-a-017645-openshift-default-6cc6c59dff-blk4t" Jul 03 11:45:27.320960 ip-10-0-136-0 kubenswrapper[2579]: I0703 11:45:27.320850 2579 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/f1b7686d-4c68-4b24-8ae1-a8cde82640cd-istio-data\") pod \"e2e-inf-a-017645-openshift-default-6cc6c59dff-blk4t\" (UID: \"f1b7686d-4c68-4b24-8ae1-a8cde82640cd\") " pod="openshift-ingress/e2e-inf-a-017645-openshift-default-6cc6c59dff-blk4t" Jul 03 11:45:27.320960 ip-10-0-136-0 kubenswrapper[2579]: I0703 11:45:27.320905 2579 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/f1b7686d-4c68-4b24-8ae1-a8cde82640cd-istio-token\") pod \"e2e-inf-a-017645-openshift-default-6cc6c59dff-blk4t\" (UID: \"f1b7686d-4c68-4b24-8ae1-a8cde82640cd\") " pod="openshift-ingress/e2e-inf-a-017645-openshift-default-6cc6c59dff-blk4t" Jul 03 11:45:27.320960 ip-10-0-136-0 kubenswrapper[2579]: I0703 11:45:27.320941 2579 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/f1b7686d-4c68-4b24-8ae1-a8cde82640cd-istiod-ca-cert\") pod \"e2e-inf-a-017645-openshift-default-6cc6c59dff-blk4t\" (UID: \"f1b7686d-4c68-4b24-8ae1-a8cde82640cd\") " pod="openshift-ingress/e2e-inf-a-017645-openshift-default-6cc6c59dff-blk4t" Jul 03 11:45:27.422338 ip-10-0-136-0 kubenswrapper[2579]: I0703 11:45:27.422284 2579 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/f1b7686d-4c68-4b24-8ae1-a8cde82640cd-istio-data\") pod \"e2e-inf-a-017645-openshift-default-6cc6c59dff-blk4t\" (UID: \"f1b7686d-4c68-4b24-8ae1-a8cde82640cd\") " pod="openshift-ingress/e2e-inf-a-017645-openshift-default-6cc6c59dff-blk4t" Jul 03 11:45:27.422546 ip-10-0-136-0 kubenswrapper[2579]: I0703 11:45:27.422355 2579 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/f1b7686d-4c68-4b24-8ae1-a8cde82640cd-istio-token\") pod \"e2e-inf-a-017645-openshift-default-6cc6c59dff-blk4t\" (UID: \"f1b7686d-4c68-4b24-8ae1-a8cde82640cd\") " pod="openshift-ingress/e2e-inf-a-017645-openshift-default-6cc6c59dff-blk4t" Jul 03 11:45:27.422546 ip-10-0-136-0 kubenswrapper[2579]: I0703 11:45:27.422387 2579 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/f1b7686d-4c68-4b24-8ae1-a8cde82640cd-istiod-ca-cert\") pod \"e2e-inf-a-017645-openshift-default-6cc6c59dff-blk4t\" (UID: \"f1b7686d-4c68-4b24-8ae1-a8cde82640cd\") " pod="openshift-ingress/e2e-inf-a-017645-openshift-default-6cc6c59dff-blk4t" Jul 03 11:45:27.422546 ip-10-0-136-0 kubenswrapper[2579]: I0703 11:45:27.422416 2579 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/f1b7686d-4c68-4b24-8ae1-a8cde82640cd-istio-podinfo\") pod \"e2e-inf-a-017645-openshift-default-6cc6c59dff-blk4t\" (UID: \"f1b7686d-4c68-4b24-8ae1-a8cde82640cd\") " pod="openshift-ingress/e2e-inf-a-017645-openshift-default-6cc6c59dff-blk4t" Jul 03 11:45:27.422546 ip-10-0-136-0 kubenswrapper[2579]: I0703 11:45:27.422466 2579 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-w92df\" (UniqueName: \"kubernetes.io/projected/f1b7686d-4c68-4b24-8ae1-a8cde82640cd-kube-api-access-w92df\") pod \"e2e-inf-a-017645-openshift-default-6cc6c59dff-blk4t\" (UID: \"f1b7686d-4c68-4b24-8ae1-a8cde82640cd\") " pod="openshift-ingress/e2e-inf-a-017645-openshift-default-6cc6c59dff-blk4t" Jul 03 11:45:27.422546 ip-10-0-136-0 kubenswrapper[2579]: I0703 11:45:27.422503 2579 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/f1b7686d-4c68-4b24-8ae1-a8cde82640cd-workload-socket\") pod \"e2e-inf-a-017645-openshift-default-6cc6c59dff-blk4t\" (UID: \"f1b7686d-4c68-4b24-8ae1-a8cde82640cd\") " pod="openshift-ingress/e2e-inf-a-017645-openshift-default-6cc6c59dff-blk4t" Jul 03 11:45:27.422546 ip-10-0-136-0 kubenswrapper[2579]: I0703 11:45:27.422539 2579 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/f1b7686d-4c68-4b24-8ae1-a8cde82640cd-workload-certs\") pod \"e2e-inf-a-017645-openshift-default-6cc6c59dff-blk4t\" (UID: \"f1b7686d-4c68-4b24-8ae1-a8cde82640cd\") " pod="openshift-ingress/e2e-inf-a-017645-openshift-default-6cc6c59dff-blk4t" Jul 03 11:45:27.422874 ip-10-0-136-0 kubenswrapper[2579]: I0703 11:45:27.422592 2579 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/f1b7686d-4c68-4b24-8ae1-a8cde82640cd-credential-socket\") pod \"e2e-inf-a-017645-openshift-default-6cc6c59dff-blk4t\" (UID: \"f1b7686d-4c68-4b24-8ae1-a8cde82640cd\") " pod="openshift-ingress/e2e-inf-a-017645-openshift-default-6cc6c59dff-blk4t" Jul 03 11:45:27.422874 ip-10-0-136-0 kubenswrapper[2579]: I0703 11:45:27.422624 2579 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/f1b7686d-4c68-4b24-8ae1-a8cde82640cd-istio-envoy\") pod \"e2e-inf-a-017645-openshift-default-6cc6c59dff-blk4t\" (UID: \"f1b7686d-4c68-4b24-8ae1-a8cde82640cd\") " pod="openshift-ingress/e2e-inf-a-017645-openshift-default-6cc6c59dff-blk4t" Jul 03 11:45:27.422874 ip-10-0-136-0 kubenswrapper[2579]: I0703 11:45:27.422819 2579 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/f1b7686d-4c68-4b24-8ae1-a8cde82640cd-istio-data\") pod \"e2e-inf-a-017645-openshift-default-6cc6c59dff-blk4t\" (UID: \"f1b7686d-4c68-4b24-8ae1-a8cde82640cd\") " pod="openshift-ingress/e2e-inf-a-017645-openshift-default-6cc6c59dff-blk4t" Jul 03 11:45:27.423034 ip-10-0-136-0 kubenswrapper[2579]: I0703 11:45:27.422902 2579 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/f1b7686d-4c68-4b24-8ae1-a8cde82640cd-workload-socket\") pod \"e2e-inf-a-017645-openshift-default-6cc6c59dff-blk4t\" (UID: \"f1b7686d-4c68-4b24-8ae1-a8cde82640cd\") " pod="openshift-ingress/e2e-inf-a-017645-openshift-default-6cc6c59dff-blk4t" Jul 03 11:45:27.423034 ip-10-0-136-0 kubenswrapper[2579]: I0703 11:45:27.422958 2579 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/f1b7686d-4c68-4b24-8ae1-a8cde82640cd-workload-certs\") pod \"e2e-inf-a-017645-openshift-default-6cc6c59dff-blk4t\" (UID: \"f1b7686d-4c68-4b24-8ae1-a8cde82640cd\") " pod="openshift-ingress/e2e-inf-a-017645-openshift-default-6cc6c59dff-blk4t" Jul 03 11:45:27.423114 ip-10-0-136-0 kubenswrapper[2579]: I0703 11:45:27.423072 2579 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/f1b7686d-4c68-4b24-8ae1-a8cde82640cd-credential-socket\") pod \"e2e-inf-a-017645-openshift-default-6cc6c59dff-blk4t\" (UID: \"f1b7686d-4c68-4b24-8ae1-a8cde82640cd\") " pod="openshift-ingress/e2e-inf-a-017645-openshift-default-6cc6c59dff-blk4t" Jul 03 11:45:27.423298 ip-10-0-136-0 kubenswrapper[2579]: I0703 11:45:27.423279 2579 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/f1b7686d-4c68-4b24-8ae1-a8cde82640cd-istiod-ca-cert\") pod \"e2e-inf-a-017645-openshift-default-6cc6c59dff-blk4t\" (UID: \"f1b7686d-4c68-4b24-8ae1-a8cde82640cd\") " pod="openshift-ingress/e2e-inf-a-017645-openshift-default-6cc6c59dff-blk4t" Jul 03 11:45:27.424981 ip-10-0-136-0 kubenswrapper[2579]: I0703 11:45:27.424958 2579 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/f1b7686d-4c68-4b24-8ae1-a8cde82640cd-istio-envoy\") pod \"e2e-inf-a-017645-openshift-default-6cc6c59dff-blk4t\" (UID: \"f1b7686d-4c68-4b24-8ae1-a8cde82640cd\") " pod="openshift-ingress/e2e-inf-a-017645-openshift-default-6cc6c59dff-blk4t" Jul 03 11:45:27.425284 ip-10-0-136-0 kubenswrapper[2579]: I0703 11:45:27.425257 2579 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/f1b7686d-4c68-4b24-8ae1-a8cde82640cd-istio-podinfo\") pod \"e2e-inf-a-017645-openshift-default-6cc6c59dff-blk4t\" (UID: \"f1b7686d-4c68-4b24-8ae1-a8cde82640cd\") " pod="openshift-ingress/e2e-inf-a-017645-openshift-default-6cc6c59dff-blk4t" Jul 03 11:45:27.432508 ip-10-0-136-0 kubenswrapper[2579]: I0703 11:45:27.432423 2579 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/f1b7686d-4c68-4b24-8ae1-a8cde82640cd-istio-token\") pod \"e2e-inf-a-017645-openshift-default-6cc6c59dff-blk4t\" (UID: \"f1b7686d-4c68-4b24-8ae1-a8cde82640cd\") " pod="openshift-ingress/e2e-inf-a-017645-openshift-default-6cc6c59dff-blk4t" Jul 03 11:45:27.432744 ip-10-0-136-0 kubenswrapper[2579]: I0703 11:45:27.432442 2579 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-w92df\" (UniqueName: \"kubernetes.io/projected/f1b7686d-4c68-4b24-8ae1-a8cde82640cd-kube-api-access-w92df\") pod \"e2e-inf-a-017645-openshift-default-6cc6c59dff-blk4t\" (UID: \"f1b7686d-4c68-4b24-8ae1-a8cde82640cd\") " pod="openshift-ingress/e2e-inf-a-017645-openshift-default-6cc6c59dff-blk4t" Jul 03 11:45:27.494740 ip-10-0-136-0 kubenswrapper[2579]: I0703 11:45:27.494702 2579 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-inf-a-017645-openshift-default-6cc6c59dff-blk4t" Jul 03 11:45:27.663660 ip-10-0-136-0 kubenswrapper[2579]: I0703 11:45:27.663626 2579 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/e2e-inf-a-017645-openshift-default-6cc6c59dff-blk4t"] Jul 03 11:45:27.665106 ip-10-0-136-0 kubenswrapper[2579]: W0703 11:45:27.665078 2579 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podf1b7686d_4c68_4b24_8ae1_a8cde82640cd.slice/crio-4647de9da687bd236c82113dd6ef8a35ee79164aebd3a2e01e8cbea04d66ea01 WatchSource:0}: Error finding container 4647de9da687bd236c82113dd6ef8a35ee79164aebd3a2e01e8cbea04d66ea01: Status 404 returned error can't find the container with id 4647de9da687bd236c82113dd6ef8a35ee79164aebd3a2e01e8cbea04d66ea01 Jul 03 11:45:27.667655 ip-10-0-136-0 kubenswrapper[2579]: I0703 11:45:27.667604 2579 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"30892156Ki","pods":"250"} Jul 03 11:45:27.667767 ip-10-0-136-0 kubenswrapper[2579]: I0703 11:45:27.667713 2579 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"30892156Ki","pods":"250"} Jul 03 11:45:27.667810 ip-10-0-136-0 kubenswrapper[2579]: I0703 11:45:27.667769 2579 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"30892156Ki","pods":"250"} Jul 03 11:45:27.930940 ip-10-0-136-0 kubenswrapper[2579]: I0703 11:45:27.930899 2579 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-inf-a-017645-openshift-default-6cc6c59dff-blk4t" event={"ID":"f1b7686d-4c68-4b24-8ae1-a8cde82640cd","Type":"ContainerStarted","Data":"816339a42db6956f3ce9c7ce34ecca89b8d3322a61e878e59138ad28d1a6ad75"} Jul 03 11:45:27.931120 ip-10-0-136-0 kubenswrapper[2579]: I0703 11:45:27.930969 2579 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-inf-a-017645-openshift-default-6cc6c59dff-blk4t" event={"ID":"f1b7686d-4c68-4b24-8ae1-a8cde82640cd","Type":"ContainerStarted","Data":"4647de9da687bd236c82113dd6ef8a35ee79164aebd3a2e01e8cbea04d66ea01"} Jul 03 11:45:27.961833 ip-10-0-136-0 kubenswrapper[2579]: I0703 11:45:27.961708 2579 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-ingress/e2e-inf-a-017645-openshift-default-6cc6c59dff-blk4t" podStartSLOduration=0.961687572 podStartE2EDuration="961.687572ms" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-03 11:45:27 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-03 11:45:27.959659826 +0000 UTC m=+1984.547964146" watchObservedRunningTime="2026-07-03 11:45:27.961687572 +0000 UTC m=+1984.549991891" Jul 03 11:45:28.495134 ip-10-0-136-0 kubenswrapper[2579]: I0703 11:45:28.495089 2579 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="openshift-ingress/e2e-inf-a-017645-openshift-default-6cc6c59dff-blk4t" Jul 03 11:45:28.500750 ip-10-0-136-0 kubenswrapper[2579]: I0703 11:45:28.500717 2579 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="openshift-ingress/e2e-inf-a-017645-openshift-default-6cc6c59dff-blk4t" Jul 03 11:45:28.939380 ip-10-0-136-0 kubenswrapper[2579]: I0703 11:45:28.939341 2579 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/e2e-inf-a-017645-openshift-default-6cc6c59dff-blk4t" Jul 03 11:45:28.940543 ip-10-0-136-0 kubenswrapper[2579]: I0703 11:45:28.940517 2579 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/e2e-inf-a-017645-openshift-default-6cc6c59dff-blk4t" Jul 03 11:45:35.158994 ip-10-0-136-0 kubenswrapper[2579]: I0703 11:45:35.158949 2579 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-ingress/e2e-inf-b-44ab65-openshift-default-5946888467-59l9p"] Jul 03 11:45:35.163051 ip-10-0-136-0 kubenswrapper[2579]: I0703 11:45:35.163022 2579 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-inf-b-44ab65-openshift-default-5946888467-59l9p" Jul 03 11:45:35.166702 ip-10-0-136-0 kubenswrapper[2579]: I0703 11:45:35.166643 2579 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress\"/\"e2e-inf-b-44ab65-openshift-default-dockercfg-s7v8w\"" Jul 03 11:45:35.186364 ip-10-0-136-0 kubenswrapper[2579]: I0703 11:45:35.186323 2579 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/e2e-inf-b-44ab65-openshift-default-5946888467-59l9p"] Jul 03 11:45:35.308177 ip-10-0-136-0 kubenswrapper[2579]: I0703 11:45:35.308137 2579 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/623e6c46-3354-47da-8157-83d80d524b3d-workload-socket\") pod \"e2e-inf-b-44ab65-openshift-default-5946888467-59l9p\" (UID: \"623e6c46-3354-47da-8157-83d80d524b3d\") " pod="openshift-ingress/e2e-inf-b-44ab65-openshift-default-5946888467-59l9p" Jul 03 11:45:35.308400 ip-10-0-136-0 kubenswrapper[2579]: I0703 11:45:35.308228 2579 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/623e6c46-3354-47da-8157-83d80d524b3d-istio-data\") pod \"e2e-inf-b-44ab65-openshift-default-5946888467-59l9p\" (UID: \"623e6c46-3354-47da-8157-83d80d524b3d\") " pod="openshift-ingress/e2e-inf-b-44ab65-openshift-default-5946888467-59l9p" Jul 03 11:45:35.308400 ip-10-0-136-0 kubenswrapper[2579]: I0703 11:45:35.308272 2579 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-6rjj7\" (UniqueName: \"kubernetes.io/projected/623e6c46-3354-47da-8157-83d80d524b3d-kube-api-access-6rjj7\") pod \"e2e-inf-b-44ab65-openshift-default-5946888467-59l9p\" (UID: \"623e6c46-3354-47da-8157-83d80d524b3d\") " pod="openshift-ingress/e2e-inf-b-44ab65-openshift-default-5946888467-59l9p" Jul 03 11:45:35.308400 ip-10-0-136-0 kubenswrapper[2579]: I0703 11:45:35.308301 2579 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/623e6c46-3354-47da-8157-83d80d524b3d-workload-certs\") pod \"e2e-inf-b-44ab65-openshift-default-5946888467-59l9p\" (UID: \"623e6c46-3354-47da-8157-83d80d524b3d\") " pod="openshift-ingress/e2e-inf-b-44ab65-openshift-default-5946888467-59l9p" Jul 03 11:45:35.308400 ip-10-0-136-0 kubenswrapper[2579]: I0703 11:45:35.308322 2579 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/623e6c46-3354-47da-8157-83d80d524b3d-istio-envoy\") pod \"e2e-inf-b-44ab65-openshift-default-5946888467-59l9p\" (UID: \"623e6c46-3354-47da-8157-83d80d524b3d\") " pod="openshift-ingress/e2e-inf-b-44ab65-openshift-default-5946888467-59l9p" Jul 03 11:45:35.308537 ip-10-0-136-0 kubenswrapper[2579]: I0703 11:45:35.308409 2579 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/623e6c46-3354-47da-8157-83d80d524b3d-istio-token\") pod \"e2e-inf-b-44ab65-openshift-default-5946888467-59l9p\" (UID: \"623e6c46-3354-47da-8157-83d80d524b3d\") " pod="openshift-ingress/e2e-inf-b-44ab65-openshift-default-5946888467-59l9p" Jul 03 11:45:35.308537 ip-10-0-136-0 kubenswrapper[2579]: I0703 11:45:35.308508 2579 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/623e6c46-3354-47da-8157-83d80d524b3d-istio-podinfo\") pod \"e2e-inf-b-44ab65-openshift-default-5946888467-59l9p\" (UID: \"623e6c46-3354-47da-8157-83d80d524b3d\") " pod="openshift-ingress/e2e-inf-b-44ab65-openshift-default-5946888467-59l9p" Jul 03 11:45:35.308602 ip-10-0-136-0 kubenswrapper[2579]: I0703 11:45:35.308568 2579 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/623e6c46-3354-47da-8157-83d80d524b3d-credential-socket\") pod \"e2e-inf-b-44ab65-openshift-default-5946888467-59l9p\" (UID: \"623e6c46-3354-47da-8157-83d80d524b3d\") " pod="openshift-ingress/e2e-inf-b-44ab65-openshift-default-5946888467-59l9p" Jul 03 11:45:35.308645 ip-10-0-136-0 kubenswrapper[2579]: I0703 11:45:35.308629 2579 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/623e6c46-3354-47da-8157-83d80d524b3d-istiod-ca-cert\") pod \"e2e-inf-b-44ab65-openshift-default-5946888467-59l9p\" (UID: \"623e6c46-3354-47da-8157-83d80d524b3d\") " pod="openshift-ingress/e2e-inf-b-44ab65-openshift-default-5946888467-59l9p" Jul 03 11:45:35.409750 ip-10-0-136-0 kubenswrapper[2579]: I0703 11:45:35.409639 2579 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/623e6c46-3354-47da-8157-83d80d524b3d-istio-data\") pod \"e2e-inf-b-44ab65-openshift-default-5946888467-59l9p\" (UID: \"623e6c46-3354-47da-8157-83d80d524b3d\") " pod="openshift-ingress/e2e-inf-b-44ab65-openshift-default-5946888467-59l9p" Jul 03 11:45:35.409750 ip-10-0-136-0 kubenswrapper[2579]: I0703 11:45:35.409697 2579 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-6rjj7\" (UniqueName: \"kubernetes.io/projected/623e6c46-3354-47da-8157-83d80d524b3d-kube-api-access-6rjj7\") pod \"e2e-inf-b-44ab65-openshift-default-5946888467-59l9p\" (UID: \"623e6c46-3354-47da-8157-83d80d524b3d\") " pod="openshift-ingress/e2e-inf-b-44ab65-openshift-default-5946888467-59l9p" Jul 03 11:45:35.409750 ip-10-0-136-0 kubenswrapper[2579]: I0703 11:45:35.409727 2579 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/623e6c46-3354-47da-8157-83d80d524b3d-workload-certs\") pod \"e2e-inf-b-44ab65-openshift-default-5946888467-59l9p\" (UID: \"623e6c46-3354-47da-8157-83d80d524b3d\") " pod="openshift-ingress/e2e-inf-b-44ab65-openshift-default-5946888467-59l9p" Jul 03 11:45:35.409750 ip-10-0-136-0 kubenswrapper[2579]: I0703 11:45:35.409750 2579 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/623e6c46-3354-47da-8157-83d80d524b3d-istio-envoy\") pod \"e2e-inf-b-44ab65-openshift-default-5946888467-59l9p\" (UID: \"623e6c46-3354-47da-8157-83d80d524b3d\") " pod="openshift-ingress/e2e-inf-b-44ab65-openshift-default-5946888467-59l9p" Jul 03 11:45:35.410116 ip-10-0-136-0 kubenswrapper[2579]: I0703 11:45:35.409787 2579 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/623e6c46-3354-47da-8157-83d80d524b3d-istio-token\") pod \"e2e-inf-b-44ab65-openshift-default-5946888467-59l9p\" (UID: \"623e6c46-3354-47da-8157-83d80d524b3d\") " pod="openshift-ingress/e2e-inf-b-44ab65-openshift-default-5946888467-59l9p" Jul 03 11:45:35.410116 ip-10-0-136-0 kubenswrapper[2579]: I0703 11:45:35.409875 2579 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/623e6c46-3354-47da-8157-83d80d524b3d-istio-podinfo\") pod \"e2e-inf-b-44ab65-openshift-default-5946888467-59l9p\" (UID: \"623e6c46-3354-47da-8157-83d80d524b3d\") " pod="openshift-ingress/e2e-inf-b-44ab65-openshift-default-5946888467-59l9p" Jul 03 11:45:35.410116 ip-10-0-136-0 kubenswrapper[2579]: I0703 11:45:35.409917 2579 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/623e6c46-3354-47da-8157-83d80d524b3d-credential-socket\") pod \"e2e-inf-b-44ab65-openshift-default-5946888467-59l9p\" (UID: \"623e6c46-3354-47da-8157-83d80d524b3d\") " pod="openshift-ingress/e2e-inf-b-44ab65-openshift-default-5946888467-59l9p" Jul 03 11:45:35.410116 ip-10-0-136-0 kubenswrapper[2579]: I0703 11:45:35.409961 2579 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/623e6c46-3354-47da-8157-83d80d524b3d-istiod-ca-cert\") pod \"e2e-inf-b-44ab65-openshift-default-5946888467-59l9p\" (UID: \"623e6c46-3354-47da-8157-83d80d524b3d\") " pod="openshift-ingress/e2e-inf-b-44ab65-openshift-default-5946888467-59l9p" Jul 03 11:45:35.410116 ip-10-0-136-0 kubenswrapper[2579]: I0703 11:45:35.409986 2579 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/623e6c46-3354-47da-8157-83d80d524b3d-workload-socket\") pod \"e2e-inf-b-44ab65-openshift-default-5946888467-59l9p\" (UID: \"623e6c46-3354-47da-8157-83d80d524b3d\") " pod="openshift-ingress/e2e-inf-b-44ab65-openshift-default-5946888467-59l9p" Jul 03 11:45:35.410414 ip-10-0-136-0 kubenswrapper[2579]: I0703 11:45:35.410221 2579 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/623e6c46-3354-47da-8157-83d80d524b3d-istio-data\") pod \"e2e-inf-b-44ab65-openshift-default-5946888467-59l9p\" (UID: \"623e6c46-3354-47da-8157-83d80d524b3d\") " pod="openshift-ingress/e2e-inf-b-44ab65-openshift-default-5946888467-59l9p" Jul 03 11:45:35.410523 ip-10-0-136-0 kubenswrapper[2579]: I0703 11:45:35.410498 2579 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/623e6c46-3354-47da-8157-83d80d524b3d-workload-certs\") pod \"e2e-inf-b-44ab65-openshift-default-5946888467-59l9p\" (UID: \"623e6c46-3354-47da-8157-83d80d524b3d\") " pod="openshift-ingress/e2e-inf-b-44ab65-openshift-default-5946888467-59l9p" Jul 03 11:45:35.410740 ip-10-0-136-0 kubenswrapper[2579]: I0703 11:45:35.410711 2579 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/623e6c46-3354-47da-8157-83d80d524b3d-workload-socket\") pod \"e2e-inf-b-44ab65-openshift-default-5946888467-59l9p\" (UID: \"623e6c46-3354-47da-8157-83d80d524b3d\") " pod="openshift-ingress/e2e-inf-b-44ab65-openshift-default-5946888467-59l9p" Jul 03 11:45:35.410874 ip-10-0-136-0 kubenswrapper[2579]: I0703 11:45:35.410742 2579 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/623e6c46-3354-47da-8157-83d80d524b3d-credential-socket\") pod \"e2e-inf-b-44ab65-openshift-default-5946888467-59l9p\" (UID: \"623e6c46-3354-47da-8157-83d80d524b3d\") " pod="openshift-ingress/e2e-inf-b-44ab65-openshift-default-5946888467-59l9p" Jul 03 11:45:35.411164 ip-10-0-136-0 kubenswrapper[2579]: I0703 11:45:35.411139 2579 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/623e6c46-3354-47da-8157-83d80d524b3d-istiod-ca-cert\") pod \"e2e-inf-b-44ab65-openshift-default-5946888467-59l9p\" (UID: \"623e6c46-3354-47da-8157-83d80d524b3d\") " pod="openshift-ingress/e2e-inf-b-44ab65-openshift-default-5946888467-59l9p" Jul 03 11:45:35.413003 ip-10-0-136-0 kubenswrapper[2579]: I0703 11:45:35.412974 2579 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/623e6c46-3354-47da-8157-83d80d524b3d-istio-podinfo\") pod \"e2e-inf-b-44ab65-openshift-default-5946888467-59l9p\" (UID: \"623e6c46-3354-47da-8157-83d80d524b3d\") " pod="openshift-ingress/e2e-inf-b-44ab65-openshift-default-5946888467-59l9p" Jul 03 11:45:35.413122 ip-10-0-136-0 kubenswrapper[2579]: I0703 11:45:35.412980 2579 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/623e6c46-3354-47da-8157-83d80d524b3d-istio-envoy\") pod \"e2e-inf-b-44ab65-openshift-default-5946888467-59l9p\" (UID: \"623e6c46-3354-47da-8157-83d80d524b3d\") " pod="openshift-ingress/e2e-inf-b-44ab65-openshift-default-5946888467-59l9p" Jul 03 11:45:35.419461 ip-10-0-136-0 kubenswrapper[2579]: I0703 11:45:35.419431 2579 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/623e6c46-3354-47da-8157-83d80d524b3d-istio-token\") pod \"e2e-inf-b-44ab65-openshift-default-5946888467-59l9p\" (UID: \"623e6c46-3354-47da-8157-83d80d524b3d\") " pod="openshift-ingress/e2e-inf-b-44ab65-openshift-default-5946888467-59l9p" Jul 03 11:45:35.419667 ip-10-0-136-0 kubenswrapper[2579]: I0703 11:45:35.419646 2579 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-6rjj7\" (UniqueName: \"kubernetes.io/projected/623e6c46-3354-47da-8157-83d80d524b3d-kube-api-access-6rjj7\") pod \"e2e-inf-b-44ab65-openshift-default-5946888467-59l9p\" (UID: \"623e6c46-3354-47da-8157-83d80d524b3d\") " pod="openshift-ingress/e2e-inf-b-44ab65-openshift-default-5946888467-59l9p" Jul 03 11:45:35.476148 ip-10-0-136-0 kubenswrapper[2579]: I0703 11:45:35.476096 2579 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-inf-b-44ab65-openshift-default-5946888467-59l9p" Jul 03 11:45:35.652765 ip-10-0-136-0 kubenswrapper[2579]: I0703 11:45:35.652732 2579 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/e2e-inf-b-44ab65-openshift-default-5946888467-59l9p"] Jul 03 11:45:35.655324 ip-10-0-136-0 kubenswrapper[2579]: W0703 11:45:35.655282 2579 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod623e6c46_3354_47da_8157_83d80d524b3d.slice/crio-5479cea7c11bd4fad7b83f5d08e0ec98a70760efa6447564f4a65f681bcc7cf5 WatchSource:0}: Error finding container 5479cea7c11bd4fad7b83f5d08e0ec98a70760efa6447564f4a65f681bcc7cf5: Status 404 returned error can't find the container with id 5479cea7c11bd4fad7b83f5d08e0ec98a70760efa6447564f4a65f681bcc7cf5 Jul 03 11:45:35.658053 ip-10-0-136-0 kubenswrapper[2579]: I0703 11:45:35.658018 2579 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"30892156Ki","pods":"250"} Jul 03 11:45:35.658162 ip-10-0-136-0 kubenswrapper[2579]: I0703 11:45:35.658110 2579 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"30892156Ki","pods":"250"} Jul 03 11:45:35.658275 ip-10-0-136-0 kubenswrapper[2579]: I0703 11:45:35.658182 2579 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"30892156Ki","pods":"250"} Jul 03 11:45:35.992279 ip-10-0-136-0 kubenswrapper[2579]: I0703 11:45:35.992166 2579 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-inf-b-44ab65-openshift-default-5946888467-59l9p" event={"ID":"623e6c46-3354-47da-8157-83d80d524b3d","Type":"ContainerStarted","Data":"c1b12e28d61306bb0a67db516619ec4164bc2ee14f2c7b469d1c7e9b74be1692"} Jul 03 11:45:35.992279 ip-10-0-136-0 kubenswrapper[2579]: I0703 11:45:35.992247 2579 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-inf-b-44ab65-openshift-default-5946888467-59l9p" event={"ID":"623e6c46-3354-47da-8157-83d80d524b3d","Type":"ContainerStarted","Data":"5479cea7c11bd4fad7b83f5d08e0ec98a70760efa6447564f4a65f681bcc7cf5"} Jul 03 11:45:36.016407 ip-10-0-136-0 kubenswrapper[2579]: I0703 11:45:36.016350 2579 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-ingress/e2e-inf-b-44ab65-openshift-default-5946888467-59l9p" podStartSLOduration=1.016327022 podStartE2EDuration="1.016327022s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-03 11:45:35 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-03 11:45:36.010415389 +0000 UTC m=+1992.598719710" watchObservedRunningTime="2026-07-03 11:45:36.016327022 +0000 UTC m=+1992.604631342" Jul 03 11:45:36.477304 ip-10-0-136-0 kubenswrapper[2579]: I0703 11:45:36.477259 2579 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="openshift-ingress/e2e-inf-b-44ab65-openshift-default-5946888467-59l9p" Jul 03 11:45:36.482855 ip-10-0-136-0 kubenswrapper[2579]: I0703 11:45:36.482822 2579 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="openshift-ingress/e2e-inf-b-44ab65-openshift-default-5946888467-59l9p" Jul 03 11:45:36.991684 ip-10-0-136-0 kubenswrapper[2579]: I0703 11:45:36.991650 2579 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/e2e-inf-b-44ab65-openshift-default-5946888467-59l9p" Jul 03 11:45:36.992693 ip-10-0-136-0 kubenswrapper[2579]: I0703 11:45:36.992671 2579 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/e2e-inf-b-44ab65-openshift-default-5946888467-59l9p" Jul 03 11:45:48.081273 ip-10-0-136-0 kubenswrapper[2579]: I0703 11:45:48.081232 2579 generic.go:356] "Generic (PLEG): container finished" podID="f1b7686d-4c68-4b24-8ae1-a8cde82640cd" containerID="816339a42db6956f3ce9c7ce34ecca89b8d3322a61e878e59138ad28d1a6ad75" exitCode=0 Jul 03 11:45:48.081753 ip-10-0-136-0 kubenswrapper[2579]: I0703 11:45:48.081283 2579 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-inf-a-017645-openshift-default-6cc6c59dff-blk4t" event={"ID":"f1b7686d-4c68-4b24-8ae1-a8cde82640cd","Type":"ContainerDied","Data":"816339a42db6956f3ce9c7ce34ecca89b8d3322a61e878e59138ad28d1a6ad75"} Jul 03 11:45:48.081753 ip-10-0-136-0 kubenswrapper[2579]: I0703 11:45:48.081738 2579 scope.go:117] "RemoveContainer" containerID="816339a42db6956f3ce9c7ce34ecca89b8d3322a61e878e59138ad28d1a6ad75" Jul 03 11:45:48.082967 ip-10-0-136-0 kubenswrapper[2579]: I0703 11:45:48.082938 2579 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"30892156Ki","pods":"250"} Jul 03 11:45:48.083092 ip-10-0-136-0 kubenswrapper[2579]: I0703 11:45:48.083052 2579 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"30892156Ki","pods":"250"} Jul 03 11:45:48.083156 ip-10-0-136-0 kubenswrapper[2579]: I0703 11:45:48.083104 2579 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"30892156Ki","pods":"250"} Jul 03 11:45:49.089738 ip-10-0-136-0 kubenswrapper[2579]: I0703 11:45:49.089692 2579 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-inf-a-017645-openshift-default-6cc6c59dff-blk4t" event={"ID":"f1b7686d-4c68-4b24-8ae1-a8cde82640cd","Type":"ContainerStarted","Data":"b4721786d3a066aaf43776587bebbe2578201c5ee90e46fe39dc18847ff81455"} Jul 03 11:45:49.496176 ip-10-0-136-0 kubenswrapper[2579]: I0703 11:45:49.496061 2579 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="openshift-ingress/e2e-inf-a-017645-openshift-default-6cc6c59dff-blk4t" Jul 03 11:45:49.496399 ip-10-0-136-0 kubenswrapper[2579]: I0703 11:45:49.496276 2579 patch_prober.go:28] interesting pod/e2e-inf-a-017645-openshift-default-6cc6c59dff-blk4t container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.133.0.68:15021/healthz/ready\": dial tcp 10.133.0.68:15021: connect: connection refused" start-of-body= Jul 03 11:45:49.496399 ip-10-0-136-0 kubenswrapper[2579]: I0703 11:45:49.496327 2579 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-inf-a-017645-openshift-default-6cc6c59dff-blk4t" podUID="f1b7686d-4c68-4b24-8ae1-a8cde82640cd" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.133.0.68:15021/healthz/ready\": dial tcp 10.133.0.68:15021: connect: connection refused" Jul 03 11:45:50.496331 ip-10-0-136-0 kubenswrapper[2579]: I0703 11:45:50.496282 2579 patch_prober.go:28] interesting pod/e2e-inf-a-017645-openshift-default-6cc6c59dff-blk4t container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.133.0.68:15021/healthz/ready\": dial tcp 10.133.0.68:15021: connect: connection refused" start-of-body= Jul 03 11:45:50.496791 ip-10-0-136-0 kubenswrapper[2579]: I0703 11:45:50.496355 2579 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-inf-a-017645-openshift-default-6cc6c59dff-blk4t" podUID="f1b7686d-4c68-4b24-8ae1-a8cde82640cd" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.133.0.68:15021/healthz/ready\": dial tcp 10.133.0.68:15021: connect: connection refused" Jul 03 11:45:51.496620 ip-10-0-136-0 kubenswrapper[2579]: I0703 11:45:51.496582 2579 patch_prober.go:28] interesting pod/e2e-inf-a-017645-openshift-default-6cc6c59dff-blk4t container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.133.0.68:15021/healthz/ready\": dial tcp 10.133.0.68:15021: connect: connection refused" start-of-body= Jul 03 11:45:51.497047 ip-10-0-136-0 kubenswrapper[2579]: I0703 11:45:51.496639 2579 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-inf-a-017645-openshift-default-6cc6c59dff-blk4t" podUID="f1b7686d-4c68-4b24-8ae1-a8cde82640cd" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.133.0.68:15021/healthz/ready\": dial tcp 10.133.0.68:15021: connect: connection refused" Jul 03 11:45:52.495947 ip-10-0-136-0 kubenswrapper[2579]: I0703 11:45:52.495901 2579 patch_prober.go:28] interesting pod/e2e-inf-a-017645-openshift-default-6cc6c59dff-blk4t container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.133.0.68:15021/healthz/ready\": dial tcp 10.133.0.68:15021: connect: connection refused" start-of-body= Jul 03 11:45:52.496160 ip-10-0-136-0 kubenswrapper[2579]: I0703 11:45:52.495965 2579 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-inf-a-017645-openshift-default-6cc6c59dff-blk4t" podUID="f1b7686d-4c68-4b24-8ae1-a8cde82640cd" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.133.0.68:15021/healthz/ready\": dial tcp 10.133.0.68:15021: connect: connection refused" Jul 03 11:45:53.549284 ip-10-0-136-0 kubenswrapper[2579]: I0703 11:45:53.549250 2579 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="openshift-ingress/e2e-inf-a-017645-openshift-default-6cc6c59dff-blk4t" Jul 03 11:45:53.549840 ip-10-0-136-0 kubenswrapper[2579]: I0703 11:45:53.549693 2579 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/e2e-inf-a-017645-openshift-default-6cc6c59dff-blk4t" Jul 03 11:45:53.550779 ip-10-0-136-0 kubenswrapper[2579]: I0703 11:45:53.550749 2579 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/e2e-inf-a-017645-openshift-default-6cc6c59dff-blk4t" Jul 03 11:45:56.139505 ip-10-0-136-0 kubenswrapper[2579]: I0703 11:45:56.139464 2579 generic.go:356] "Generic (PLEG): container finished" podID="623e6c46-3354-47da-8157-83d80d524b3d" containerID="c1b12e28d61306bb0a67db516619ec4164bc2ee14f2c7b469d1c7e9b74be1692" exitCode=0 Jul 03 11:45:56.139956 ip-10-0-136-0 kubenswrapper[2579]: I0703 11:45:56.139538 2579 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-inf-b-44ab65-openshift-default-5946888467-59l9p" event={"ID":"623e6c46-3354-47da-8157-83d80d524b3d","Type":"ContainerDied","Data":"c1b12e28d61306bb0a67db516619ec4164bc2ee14f2c7b469d1c7e9b74be1692"} Jul 03 11:45:56.140047 ip-10-0-136-0 kubenswrapper[2579]: I0703 11:45:56.140028 2579 scope.go:117] "RemoveContainer" containerID="c1b12e28d61306bb0a67db516619ec4164bc2ee14f2c7b469d1c7e9b74be1692" Jul 03 11:45:56.141241 ip-10-0-136-0 kubenswrapper[2579]: I0703 11:45:56.141174 2579 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"30892156Ki","pods":"250"} Jul 03 11:45:56.141354 ip-10-0-136-0 kubenswrapper[2579]: I0703 11:45:56.141284 2579 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"30892156Ki","pods":"250"} Jul 03 11:45:56.141354 ip-10-0-136-0 kubenswrapper[2579]: I0703 11:45:56.141323 2579 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"30892156Ki","pods":"250"} Jul 03 11:45:57.148600 ip-10-0-136-0 kubenswrapper[2579]: I0703 11:45:57.148561 2579 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-inf-b-44ab65-openshift-default-5946888467-59l9p" event={"ID":"623e6c46-3354-47da-8157-83d80d524b3d","Type":"ContainerStarted","Data":"00b2d88be3ba88496c4f88bb2fb3043304e201e572edc51ec56e0a7fb1def652"} Jul 03 11:45:57.476753 ip-10-0-136-0 kubenswrapper[2579]: I0703 11:45:57.476641 2579 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="openshift-ingress/e2e-inf-b-44ab65-openshift-default-5946888467-59l9p" Jul 03 11:45:57.476958 ip-10-0-136-0 kubenswrapper[2579]: I0703 11:45:57.476931 2579 patch_prober.go:28] interesting pod/e2e-inf-b-44ab65-openshift-default-5946888467-59l9p container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.133.0.69:15021/healthz/ready\": dial tcp 10.133.0.69:15021: connect: connection refused" start-of-body= Jul 03 11:45:57.477018 ip-10-0-136-0 kubenswrapper[2579]: I0703 11:45:57.476990 2579 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-inf-b-44ab65-openshift-default-5946888467-59l9p" podUID="623e6c46-3354-47da-8157-83d80d524b3d" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.133.0.69:15021/healthz/ready\": dial tcp 10.133.0.69:15021: connect: connection refused" Jul 03 11:45:58.477122 ip-10-0-136-0 kubenswrapper[2579]: I0703 11:45:58.477079 2579 patch_prober.go:28] interesting pod/e2e-inf-b-44ab65-openshift-default-5946888467-59l9p container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.133.0.69:15021/healthz/ready\": dial tcp 10.133.0.69:15021: connect: connection refused" start-of-body= Jul 03 11:45:58.477546 ip-10-0-136-0 kubenswrapper[2579]: I0703 11:45:58.477151 2579 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-inf-b-44ab65-openshift-default-5946888467-59l9p" podUID="623e6c46-3354-47da-8157-83d80d524b3d" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.133.0.69:15021/healthz/ready\": dial tcp 10.133.0.69:15021: connect: connection refused" Jul 03 11:45:59.477317 ip-10-0-136-0 kubenswrapper[2579]: I0703 11:45:59.477275 2579 patch_prober.go:28] interesting pod/e2e-inf-b-44ab65-openshift-default-5946888467-59l9p container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.133.0.69:15021/healthz/ready\": dial tcp 10.133.0.69:15021: connect: connection refused" start-of-body= Jul 03 11:45:59.477726 ip-10-0-136-0 kubenswrapper[2579]: I0703 11:45:59.477336 2579 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-inf-b-44ab65-openshift-default-5946888467-59l9p" podUID="623e6c46-3354-47da-8157-83d80d524b3d" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.133.0.69:15021/healthz/ready\": dial tcp 10.133.0.69:15021: connect: connection refused" Jul 03 11:46:00.476952 ip-10-0-136-0 kubenswrapper[2579]: I0703 11:46:00.476905 2579 patch_prober.go:28] interesting pod/e2e-inf-b-44ab65-openshift-default-5946888467-59l9p container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.133.0.69:15021/healthz/ready\": dial tcp 10.133.0.69:15021: connect: connection refused" start-of-body= Jul 03 11:46:00.477135 ip-10-0-136-0 kubenswrapper[2579]: I0703 11:46:00.476972 2579 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-inf-b-44ab65-openshift-default-5946888467-59l9p" podUID="623e6c46-3354-47da-8157-83d80d524b3d" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.133.0.69:15021/healthz/ready\": dial tcp 10.133.0.69:15021: connect: connection refused" Jul 03 11:46:01.476865 ip-10-0-136-0 kubenswrapper[2579]: I0703 11:46:01.476822 2579 patch_prober.go:28] interesting pod/e2e-inf-b-44ab65-openshift-default-5946888467-59l9p container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.133.0.69:15021/healthz/ready\": dial tcp 10.133.0.69:15021: connect: connection refused" start-of-body= Jul 03 11:46:01.477383 ip-10-0-136-0 kubenswrapper[2579]: I0703 11:46:01.476897 2579 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-inf-b-44ab65-openshift-default-5946888467-59l9p" podUID="623e6c46-3354-47da-8157-83d80d524b3d" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.133.0.69:15021/healthz/ready\": dial tcp 10.133.0.69:15021: connect: connection refused" Jul 03 11:46:02.481922 ip-10-0-136-0 kubenswrapper[2579]: I0703 11:46:02.481889 2579 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="openshift-ingress/e2e-inf-b-44ab65-openshift-default-5946888467-59l9p" Jul 03 11:46:02.482453 ip-10-0-136-0 kubenswrapper[2579]: I0703 11:46:02.482299 2579 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/e2e-inf-b-44ab65-openshift-default-5946888467-59l9p" Jul 03 11:46:02.483159 ip-10-0-136-0 kubenswrapper[2579]: I0703 11:46:02.483138 2579 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/e2e-inf-b-44ab65-openshift-default-5946888467-59l9p" Jul 03 11:46:26.247592 ip-10-0-136-0 kubenswrapper[2579]: I0703 11:46:26.247551 2579 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["ai-tenant-e2e-inf-b-44ab65/test-model-44ab65-kserve-bfc77b86f-sdz9j"] Jul 03 11:46:26.251094 ip-10-0-136-0 kubenswrapper[2579]: I0703 11:46:26.251063 2579 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="ai-tenant-e2e-inf-b-44ab65/test-model-44ab65-kserve-bfc77b86f-sdz9j" Jul 03 11:46:26.255142 ip-10-0-136-0 kubenswrapper[2579]: I0703 11:46:26.255105 2579 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"ai-tenant-e2e-inf-b-44ab65\"/\"openshift-service-ca.crt\"" Jul 03 11:46:26.255373 ip-10-0-136-0 kubenswrapper[2579]: I0703 11:46:26.255120 2579 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"ai-tenant-e2e-inf-b-44ab65\"/\"kube-root-ca.crt\"" Jul 03 11:46:26.255373 ip-10-0-136-0 kubenswrapper[2579]: I0703 11:46:26.255206 2579 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"ai-tenant-e2e-inf-b-44ab65\"/\"default-dockercfg-5ncxv\"" Jul 03 11:46:26.255518 ip-10-0-136-0 kubenswrapper[2579]: I0703 11:46:26.255499 2579 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"ai-tenant-e2e-inf-b-44ab65\"/\"test-model-44ab65-kserve-self-signed-certs\"" Jul 03 11:46:26.273284 ip-10-0-136-0 kubenswrapper[2579]: I0703 11:46:26.273249 2579 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["ai-tenant-e2e-inf-b-44ab65/test-model-44ab65-kserve-bfc77b86f-sdz9j"] Jul 03 11:46:26.382412 ip-10-0-136-0 kubenswrapper[2579]: I0703 11:46:26.382374 2579 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/d8df6c2a-e6cf-4a9e-88d6-86ec7f7461e1-home\") pod \"test-model-44ab65-kserve-bfc77b86f-sdz9j\" (UID: \"d8df6c2a-e6cf-4a9e-88d6-86ec7f7461e1\") " pod="ai-tenant-e2e-inf-b-44ab65/test-model-44ab65-kserve-bfc77b86f-sdz9j" Jul 03 11:46:26.382644 ip-10-0-136-0 kubenswrapper[2579]: I0703 11:46:26.382434 2579 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/d8df6c2a-e6cf-4a9e-88d6-86ec7f7461e1-tmp-dir\") pod \"test-model-44ab65-kserve-bfc77b86f-sdz9j\" (UID: \"d8df6c2a-e6cf-4a9e-88d6-86ec7f7461e1\") " pod="ai-tenant-e2e-inf-b-44ab65/test-model-44ab65-kserve-bfc77b86f-sdz9j" Jul 03 11:46:26.382644 ip-10-0-136-0 kubenswrapper[2579]: I0703 11:46:26.382506 2579 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/d8df6c2a-e6cf-4a9e-88d6-86ec7f7461e1-tls-certs\") pod \"test-model-44ab65-kserve-bfc77b86f-sdz9j\" (UID: \"d8df6c2a-e6cf-4a9e-88d6-86ec7f7461e1\") " pod="ai-tenant-e2e-inf-b-44ab65/test-model-44ab65-kserve-bfc77b86f-sdz9j" Jul 03 11:46:26.382644 ip-10-0-136-0 kubenswrapper[2579]: I0703 11:46:26.382581 2579 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/d8df6c2a-e6cf-4a9e-88d6-86ec7f7461e1-dshm\") pod \"test-model-44ab65-kserve-bfc77b86f-sdz9j\" (UID: \"d8df6c2a-e6cf-4a9e-88d6-86ec7f7461e1\") " pod="ai-tenant-e2e-inf-b-44ab65/test-model-44ab65-kserve-bfc77b86f-sdz9j" Jul 03 11:46:26.382644 ip-10-0-136-0 kubenswrapper[2579]: I0703 11:46:26.382621 2579 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-dvpcm\" (UniqueName: \"kubernetes.io/projected/d8df6c2a-e6cf-4a9e-88d6-86ec7f7461e1-kube-api-access-dvpcm\") pod \"test-model-44ab65-kserve-bfc77b86f-sdz9j\" (UID: \"d8df6c2a-e6cf-4a9e-88d6-86ec7f7461e1\") " pod="ai-tenant-e2e-inf-b-44ab65/test-model-44ab65-kserve-bfc77b86f-sdz9j" Jul 03 11:46:26.382897 ip-10-0-136-0 kubenswrapper[2579]: I0703 11:46:26.382727 2579 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/d8df6c2a-e6cf-4a9e-88d6-86ec7f7461e1-model-cache\") pod \"test-model-44ab65-kserve-bfc77b86f-sdz9j\" (UID: \"d8df6c2a-e6cf-4a9e-88d6-86ec7f7461e1\") " pod="ai-tenant-e2e-inf-b-44ab65/test-model-44ab65-kserve-bfc77b86f-sdz9j" Jul 03 11:46:26.483643 ip-10-0-136-0 kubenswrapper[2579]: I0703 11:46:26.483600 2579 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/d8df6c2a-e6cf-4a9e-88d6-86ec7f7461e1-home\") pod \"test-model-44ab65-kserve-bfc77b86f-sdz9j\" (UID: \"d8df6c2a-e6cf-4a9e-88d6-86ec7f7461e1\") " pod="ai-tenant-e2e-inf-b-44ab65/test-model-44ab65-kserve-bfc77b86f-sdz9j" Jul 03 11:46:26.483835 ip-10-0-136-0 kubenswrapper[2579]: I0703 11:46:26.483654 2579 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/d8df6c2a-e6cf-4a9e-88d6-86ec7f7461e1-tmp-dir\") pod \"test-model-44ab65-kserve-bfc77b86f-sdz9j\" (UID: \"d8df6c2a-e6cf-4a9e-88d6-86ec7f7461e1\") " pod="ai-tenant-e2e-inf-b-44ab65/test-model-44ab65-kserve-bfc77b86f-sdz9j" Jul 03 11:46:26.483835 ip-10-0-136-0 kubenswrapper[2579]: I0703 11:46:26.483674 2579 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/d8df6c2a-e6cf-4a9e-88d6-86ec7f7461e1-tls-certs\") pod \"test-model-44ab65-kserve-bfc77b86f-sdz9j\" (UID: \"d8df6c2a-e6cf-4a9e-88d6-86ec7f7461e1\") " pod="ai-tenant-e2e-inf-b-44ab65/test-model-44ab65-kserve-bfc77b86f-sdz9j" Jul 03 11:46:26.483835 ip-10-0-136-0 kubenswrapper[2579]: I0703 11:46:26.483710 2579 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/d8df6c2a-e6cf-4a9e-88d6-86ec7f7461e1-dshm\") pod \"test-model-44ab65-kserve-bfc77b86f-sdz9j\" (UID: \"d8df6c2a-e6cf-4a9e-88d6-86ec7f7461e1\") " pod="ai-tenant-e2e-inf-b-44ab65/test-model-44ab65-kserve-bfc77b86f-sdz9j" Jul 03 11:46:26.483835 ip-10-0-136-0 kubenswrapper[2579]: I0703 11:46:26.483737 2579 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-dvpcm\" (UniqueName: \"kubernetes.io/projected/d8df6c2a-e6cf-4a9e-88d6-86ec7f7461e1-kube-api-access-dvpcm\") pod \"test-model-44ab65-kserve-bfc77b86f-sdz9j\" (UID: \"d8df6c2a-e6cf-4a9e-88d6-86ec7f7461e1\") " pod="ai-tenant-e2e-inf-b-44ab65/test-model-44ab65-kserve-bfc77b86f-sdz9j" Jul 03 11:46:26.483835 ip-10-0-136-0 kubenswrapper[2579]: I0703 11:46:26.483773 2579 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/d8df6c2a-e6cf-4a9e-88d6-86ec7f7461e1-model-cache\") pod \"test-model-44ab65-kserve-bfc77b86f-sdz9j\" (UID: \"d8df6c2a-e6cf-4a9e-88d6-86ec7f7461e1\") " pod="ai-tenant-e2e-inf-b-44ab65/test-model-44ab65-kserve-bfc77b86f-sdz9j" Jul 03 11:46:26.484211 ip-10-0-136-0 kubenswrapper[2579]: I0703 11:46:26.484170 2579 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/d8df6c2a-e6cf-4a9e-88d6-86ec7f7461e1-tmp-dir\") pod \"test-model-44ab65-kserve-bfc77b86f-sdz9j\" (UID: \"d8df6c2a-e6cf-4a9e-88d6-86ec7f7461e1\") " pod="ai-tenant-e2e-inf-b-44ab65/test-model-44ab65-kserve-bfc77b86f-sdz9j" Jul 03 11:46:26.484268 ip-10-0-136-0 kubenswrapper[2579]: I0703 11:46:26.484232 2579 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/d8df6c2a-e6cf-4a9e-88d6-86ec7f7461e1-home\") pod \"test-model-44ab65-kserve-bfc77b86f-sdz9j\" (UID: \"d8df6c2a-e6cf-4a9e-88d6-86ec7f7461e1\") " pod="ai-tenant-e2e-inf-b-44ab65/test-model-44ab65-kserve-bfc77b86f-sdz9j" Jul 03 11:46:26.484268 ip-10-0-136-0 kubenswrapper[2579]: I0703 11:46:26.484243 2579 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/d8df6c2a-e6cf-4a9e-88d6-86ec7f7461e1-model-cache\") pod \"test-model-44ab65-kserve-bfc77b86f-sdz9j\" (UID: \"d8df6c2a-e6cf-4a9e-88d6-86ec7f7461e1\") " pod="ai-tenant-e2e-inf-b-44ab65/test-model-44ab65-kserve-bfc77b86f-sdz9j" Jul 03 11:46:26.486351 ip-10-0-136-0 kubenswrapper[2579]: I0703 11:46:26.486330 2579 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/d8df6c2a-e6cf-4a9e-88d6-86ec7f7461e1-dshm\") pod \"test-model-44ab65-kserve-bfc77b86f-sdz9j\" (UID: \"d8df6c2a-e6cf-4a9e-88d6-86ec7f7461e1\") " pod="ai-tenant-e2e-inf-b-44ab65/test-model-44ab65-kserve-bfc77b86f-sdz9j" Jul 03 11:46:26.486676 ip-10-0-136-0 kubenswrapper[2579]: I0703 11:46:26.486638 2579 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/d8df6c2a-e6cf-4a9e-88d6-86ec7f7461e1-tls-certs\") pod \"test-model-44ab65-kserve-bfc77b86f-sdz9j\" (UID: \"d8df6c2a-e6cf-4a9e-88d6-86ec7f7461e1\") " pod="ai-tenant-e2e-inf-b-44ab65/test-model-44ab65-kserve-bfc77b86f-sdz9j" Jul 03 11:46:26.493613 ip-10-0-136-0 kubenswrapper[2579]: I0703 11:46:26.493579 2579 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-dvpcm\" (UniqueName: \"kubernetes.io/projected/d8df6c2a-e6cf-4a9e-88d6-86ec7f7461e1-kube-api-access-dvpcm\") pod \"test-model-44ab65-kserve-bfc77b86f-sdz9j\" (UID: \"d8df6c2a-e6cf-4a9e-88d6-86ec7f7461e1\") " pod="ai-tenant-e2e-inf-b-44ab65/test-model-44ab65-kserve-bfc77b86f-sdz9j" Jul 03 11:46:26.562294 ip-10-0-136-0 kubenswrapper[2579]: I0703 11:46:26.562259 2579 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="ai-tenant-e2e-inf-b-44ab65/test-model-44ab65-kserve-bfc77b86f-sdz9j" Jul 03 11:46:26.729621 ip-10-0-136-0 kubenswrapper[2579]: I0703 11:46:26.729592 2579 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["ai-tenant-e2e-inf-b-44ab65/test-model-44ab65-kserve-bfc77b86f-sdz9j"] Jul 03 11:46:26.730880 ip-10-0-136-0 kubenswrapper[2579]: W0703 11:46:26.730851 2579 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-besteffort.slice/kubepods-besteffort-podd8df6c2a_e6cf_4a9e_88d6_86ec7f7461e1.slice/crio-c0533aa0c9fb699e77a0ad2716637c4c9ec114359c0da646f9c0184690e972d8 WatchSource:0}: Error finding container c0533aa0c9fb699e77a0ad2716637c4c9ec114359c0da646f9c0184690e972d8: Status 404 returned error can't find the container with id c0533aa0c9fb699e77a0ad2716637c4c9ec114359c0da646f9c0184690e972d8 Jul 03 11:46:27.338318 ip-10-0-136-0 kubenswrapper[2579]: I0703 11:46:27.338284 2579 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="ai-tenant-e2e-inf-b-44ab65/test-model-44ab65-kserve-bfc77b86f-sdz9j" event={"ID":"d8df6c2a-e6cf-4a9e-88d6-86ec7f7461e1","Type":"ContainerStarted","Data":"3ed7ea7a10c99283e8849f902725fdbe03c354cfa5d17fc15fd07d57edb02ec8"} Jul 03 11:46:27.338318 ip-10-0-136-0 kubenswrapper[2579]: I0703 11:46:27.338332 2579 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="ai-tenant-e2e-inf-b-44ab65/test-model-44ab65-kserve-bfc77b86f-sdz9j" event={"ID":"d8df6c2a-e6cf-4a9e-88d6-86ec7f7461e1","Type":"ContainerStarted","Data":"c0533aa0c9fb699e77a0ad2716637c4c9ec114359c0da646f9c0184690e972d8"} Jul 03 11:46:27.357909 ip-10-0-136-0 kubenswrapper[2579]: I0703 11:46:27.357837 2579 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="ai-tenant-e2e-inf-b-44ab65/test-model-44ab65-kserve-bfc77b86f-sdz9j" podStartSLOduration=1.357815826 podStartE2EDuration="1.357815826s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-03 11:46:26 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-03 11:46:27.355606858 +0000 UTC m=+2043.943911182" watchObservedRunningTime="2026-07-03 11:46:27.357815826 +0000 UTC m=+2043.946120147" Jul 03 11:46:36.563173 ip-10-0-136-0 kubenswrapper[2579]: I0703 11:46:36.563122 2579 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="ai-tenant-e2e-inf-b-44ab65/test-model-44ab65-kserve-bfc77b86f-sdz9j" Jul 03 11:46:36.563719 ip-10-0-136-0 kubenswrapper[2579]: I0703 11:46:36.563224 2579 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="ai-tenant-e2e-inf-b-44ab65/test-model-44ab65-kserve-bfc77b86f-sdz9j" Jul 03 11:46:36.577011 ip-10-0-136-0 kubenswrapper[2579]: I0703 11:46:36.576974 2579 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="ai-tenant-e2e-inf-b-44ab65/test-model-44ab65-kserve-bfc77b86f-sdz9j" Jul 03 11:46:37.428795 ip-10-0-136-0 kubenswrapper[2579]: I0703 11:46:37.428754 2579 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="ai-tenant-e2e-inf-b-44ab65/test-model-44ab65-kserve-bfc77b86f-sdz9j" Jul 03 11:46:47.522056 ip-10-0-136-0 kubenswrapper[2579]: I0703 11:46:47.521973 2579 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["ai-tenant-e2e-inf-b-44ab65/test-model-44ab65-kserve-bfc77b86f-sdz9j"] Jul 03 11:47:02.545048 ip-10-0-136-0 kubenswrapper[2579]: I0703 11:47:02.544986 2579 kuberuntime_container.go:872] "Killing container with a grace period" pod="ai-tenant-e2e-inf-b-44ab65/test-model-44ab65-kserve-bfc77b86f-sdz9j" podUID="d8df6c2a-e6cf-4a9e-88d6-86ec7f7461e1" containerName="main" containerID="cri-o://3ed7ea7a10c99283e8849f902725fdbe03c354cfa5d17fc15fd07d57edb02ec8" gracePeriod=45 Jul 03 11:47:02.818417 ip-10-0-136-0 kubenswrapper[2579]: I0703 11:47:02.818379 2579 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="ai-tenant-e2e-inf-b-44ab65/test-model-44ab65-kserve-bfc77b86f-sdz9j" Jul 03 11:47:02.878920 ip-10-0-136-0 kubenswrapper[2579]: I0703 11:47:02.878883 2579 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/d8df6c2a-e6cf-4a9e-88d6-86ec7f7461e1-dshm\") pod \"d8df6c2a-e6cf-4a9e-88d6-86ec7f7461e1\" (UID: \"d8df6c2a-e6cf-4a9e-88d6-86ec7f7461e1\") " Jul 03 11:47:02.879104 ip-10-0-136-0 kubenswrapper[2579]: I0703 11:47:02.878982 2579 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/d8df6c2a-e6cf-4a9e-88d6-86ec7f7461e1-model-cache\") pod \"d8df6c2a-e6cf-4a9e-88d6-86ec7f7461e1\" (UID: \"d8df6c2a-e6cf-4a9e-88d6-86ec7f7461e1\") " Jul 03 11:47:02.879104 ip-10-0-136-0 kubenswrapper[2579]: I0703 11:47:02.879014 2579 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/d8df6c2a-e6cf-4a9e-88d6-86ec7f7461e1-tmp-dir\") pod \"d8df6c2a-e6cf-4a9e-88d6-86ec7f7461e1\" (UID: \"d8df6c2a-e6cf-4a9e-88d6-86ec7f7461e1\") " Jul 03 11:47:02.879104 ip-10-0-136-0 kubenswrapper[2579]: I0703 11:47:02.879054 2579 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/d8df6c2a-e6cf-4a9e-88d6-86ec7f7461e1-tls-certs\") pod \"d8df6c2a-e6cf-4a9e-88d6-86ec7f7461e1\" (UID: \"d8df6c2a-e6cf-4a9e-88d6-86ec7f7461e1\") " Jul 03 11:47:02.879287 ip-10-0-136-0 kubenswrapper[2579]: I0703 11:47:02.879105 2579 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/d8df6c2a-e6cf-4a9e-88d6-86ec7f7461e1-home\") pod \"d8df6c2a-e6cf-4a9e-88d6-86ec7f7461e1\" (UID: \"d8df6c2a-e6cf-4a9e-88d6-86ec7f7461e1\") " Jul 03 11:47:02.879287 ip-10-0-136-0 kubenswrapper[2579]: I0703 11:47:02.879136 2579 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-dvpcm\" (UniqueName: \"kubernetes.io/projected/d8df6c2a-e6cf-4a9e-88d6-86ec7f7461e1-kube-api-access-dvpcm\") pod \"d8df6c2a-e6cf-4a9e-88d6-86ec7f7461e1\" (UID: \"d8df6c2a-e6cf-4a9e-88d6-86ec7f7461e1\") " Jul 03 11:47:02.879400 ip-10-0-136-0 kubenswrapper[2579]: I0703 11:47:02.879295 2579 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/d8df6c2a-e6cf-4a9e-88d6-86ec7f7461e1-model-cache" (OuterVolumeSpecName: "model-cache") pod "d8df6c2a-e6cf-4a9e-88d6-86ec7f7461e1" (UID: "d8df6c2a-e6cf-4a9e-88d6-86ec7f7461e1"). InnerVolumeSpecName "model-cache". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Jul 03 11:47:02.879400 ip-10-0-136-0 kubenswrapper[2579]: I0703 11:47:02.879345 2579 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/d8df6c2a-e6cf-4a9e-88d6-86ec7f7461e1-tmp-dir" (OuterVolumeSpecName: "tmp-dir") pod "d8df6c2a-e6cf-4a9e-88d6-86ec7f7461e1" (UID: "d8df6c2a-e6cf-4a9e-88d6-86ec7f7461e1"). InnerVolumeSpecName "tmp-dir". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Jul 03 11:47:02.879511 ip-10-0-136-0 kubenswrapper[2579]: I0703 11:47:02.879476 2579 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/d8df6c2a-e6cf-4a9e-88d6-86ec7f7461e1-home" (OuterVolumeSpecName: "home") pod "d8df6c2a-e6cf-4a9e-88d6-86ec7f7461e1" (UID: "d8df6c2a-e6cf-4a9e-88d6-86ec7f7461e1"). InnerVolumeSpecName "home". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Jul 03 11:47:02.879557 ip-10-0-136-0 kubenswrapper[2579]: I0703 11:47:02.879539 2579 reconciler_common.go:299] "Volume detached for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/d8df6c2a-e6cf-4a9e-88d6-86ec7f7461e1-model-cache\") on node \"ip-10-0-136-0.ec2.internal\" DevicePath \"\"" Jul 03 11:47:02.879602 ip-10-0-136-0 kubenswrapper[2579]: I0703 11:47:02.879567 2579 reconciler_common.go:299] "Volume detached for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/d8df6c2a-e6cf-4a9e-88d6-86ec7f7461e1-tmp-dir\") on node \"ip-10-0-136-0.ec2.internal\" DevicePath \"\"" Jul 03 11:47:02.881516 ip-10-0-136-0 kubenswrapper[2579]: I0703 11:47:02.881476 2579 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/d8df6c2a-e6cf-4a9e-88d6-86ec7f7461e1-tls-certs" (OuterVolumeSpecName: "tls-certs") pod "d8df6c2a-e6cf-4a9e-88d6-86ec7f7461e1" (UID: "d8df6c2a-e6cf-4a9e-88d6-86ec7f7461e1"). InnerVolumeSpecName "tls-certs". PluginName "kubernetes.io/secret", VolumeGIDValue "" Jul 03 11:47:02.881641 ip-10-0-136-0 kubenswrapper[2579]: I0703 11:47:02.881521 2579 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/d8df6c2a-e6cf-4a9e-88d6-86ec7f7461e1-dshm" (OuterVolumeSpecName: "dshm") pod "d8df6c2a-e6cf-4a9e-88d6-86ec7f7461e1" (UID: "d8df6c2a-e6cf-4a9e-88d6-86ec7f7461e1"). InnerVolumeSpecName "dshm". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Jul 03 11:47:02.881641 ip-10-0-136-0 kubenswrapper[2579]: I0703 11:47:02.881532 2579 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/d8df6c2a-e6cf-4a9e-88d6-86ec7f7461e1-kube-api-access-dvpcm" (OuterVolumeSpecName: "kube-api-access-dvpcm") pod "d8df6c2a-e6cf-4a9e-88d6-86ec7f7461e1" (UID: "d8df6c2a-e6cf-4a9e-88d6-86ec7f7461e1"). InnerVolumeSpecName "kube-api-access-dvpcm". PluginName "kubernetes.io/projected", VolumeGIDValue "" Jul 03 11:47:02.980585 ip-10-0-136-0 kubenswrapper[2579]: I0703 11:47:02.980537 2579 reconciler_common.go:299] "Volume detached for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/d8df6c2a-e6cf-4a9e-88d6-86ec7f7461e1-tls-certs\") on node \"ip-10-0-136-0.ec2.internal\" DevicePath \"\"" Jul 03 11:47:02.980585 ip-10-0-136-0 kubenswrapper[2579]: I0703 11:47:02.980572 2579 reconciler_common.go:299] "Volume detached for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/d8df6c2a-e6cf-4a9e-88d6-86ec7f7461e1-home\") on node \"ip-10-0-136-0.ec2.internal\" DevicePath \"\"" Jul 03 11:47:02.980585 ip-10-0-136-0 kubenswrapper[2579]: I0703 11:47:02.980582 2579 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-dvpcm\" (UniqueName: \"kubernetes.io/projected/d8df6c2a-e6cf-4a9e-88d6-86ec7f7461e1-kube-api-access-dvpcm\") on node \"ip-10-0-136-0.ec2.internal\" DevicePath \"\"" Jul 03 11:47:02.980585 ip-10-0-136-0 kubenswrapper[2579]: I0703 11:47:02.980593 2579 reconciler_common.go:299] "Volume detached for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/d8df6c2a-e6cf-4a9e-88d6-86ec7f7461e1-dshm\") on node \"ip-10-0-136-0.ec2.internal\" DevicePath \"\"" Jul 03 11:47:03.576802 ip-10-0-136-0 kubenswrapper[2579]: I0703 11:47:03.576762 2579 generic.go:356] "Generic (PLEG): container finished" podID="d8df6c2a-e6cf-4a9e-88d6-86ec7f7461e1" containerID="3ed7ea7a10c99283e8849f902725fdbe03c354cfa5d17fc15fd07d57edb02ec8" exitCode=0 Jul 03 11:47:03.577289 ip-10-0-136-0 kubenswrapper[2579]: I0703 11:47:03.576816 2579 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="ai-tenant-e2e-inf-b-44ab65/test-model-44ab65-kserve-bfc77b86f-sdz9j" Jul 03 11:47:03.577289 ip-10-0-136-0 kubenswrapper[2579]: I0703 11:47:03.576845 2579 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="ai-tenant-e2e-inf-b-44ab65/test-model-44ab65-kserve-bfc77b86f-sdz9j" event={"ID":"d8df6c2a-e6cf-4a9e-88d6-86ec7f7461e1","Type":"ContainerDied","Data":"3ed7ea7a10c99283e8849f902725fdbe03c354cfa5d17fc15fd07d57edb02ec8"} Jul 03 11:47:03.577289 ip-10-0-136-0 kubenswrapper[2579]: I0703 11:47:03.576893 2579 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="ai-tenant-e2e-inf-b-44ab65/test-model-44ab65-kserve-bfc77b86f-sdz9j" event={"ID":"d8df6c2a-e6cf-4a9e-88d6-86ec7f7461e1","Type":"ContainerDied","Data":"c0533aa0c9fb699e77a0ad2716637c4c9ec114359c0da646f9c0184690e972d8"} Jul 03 11:47:03.577289 ip-10-0-136-0 kubenswrapper[2579]: I0703 11:47:03.576919 2579 scope.go:117] "RemoveContainer" containerID="3ed7ea7a10c99283e8849f902725fdbe03c354cfa5d17fc15fd07d57edb02ec8" Jul 03 11:47:03.587557 ip-10-0-136-0 kubenswrapper[2579]: I0703 11:47:03.587531 2579 scope.go:117] "RemoveContainer" containerID="3ed7ea7a10c99283e8849f902725fdbe03c354cfa5d17fc15fd07d57edb02ec8" Jul 03 11:47:03.587894 ip-10-0-136-0 kubenswrapper[2579]: E0703 11:47:03.587872 2579 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"3ed7ea7a10c99283e8849f902725fdbe03c354cfa5d17fc15fd07d57edb02ec8\": container with ID starting with 3ed7ea7a10c99283e8849f902725fdbe03c354cfa5d17fc15fd07d57edb02ec8 not found: ID does not exist" containerID="3ed7ea7a10c99283e8849f902725fdbe03c354cfa5d17fc15fd07d57edb02ec8" Jul 03 11:47:03.587957 ip-10-0-136-0 kubenswrapper[2579]: I0703 11:47:03.587905 2579 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"3ed7ea7a10c99283e8849f902725fdbe03c354cfa5d17fc15fd07d57edb02ec8"} err="failed to get container status \"3ed7ea7a10c99283e8849f902725fdbe03c354cfa5d17fc15fd07d57edb02ec8\": rpc error: code = NotFound desc = could not find container \"3ed7ea7a10c99283e8849f902725fdbe03c354cfa5d17fc15fd07d57edb02ec8\": container with ID starting with 3ed7ea7a10c99283e8849f902725fdbe03c354cfa5d17fc15fd07d57edb02ec8 not found: ID does not exist" Jul 03 11:47:03.621582 ip-10-0-136-0 kubenswrapper[2579]: I0703 11:47:03.621540 2579 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["ai-tenant-e2e-inf-b-44ab65/test-model-44ab65-kserve-bfc77b86f-sdz9j"] Jul 03 11:47:03.636974 ip-10-0-136-0 kubenswrapper[2579]: I0703 11:47:03.636928 2579 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["ai-tenant-e2e-inf-b-44ab65/test-model-44ab65-kserve-bfc77b86f-sdz9j"] Jul 03 11:47:03.993427 ip-10-0-136-0 kubenswrapper[2579]: I0703 11:47:03.993320 2579 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="d8df6c2a-e6cf-4a9e-88d6-86ec7f7461e1" path="/var/lib/kubelet/pods/d8df6c2a-e6cf-4a9e-88d6-86ec7f7461e1/volumes" Jul 03 11:47:31.934534 ip-10-0-136-0 kubenswrapper[2579]: I0703 11:47:31.934490 2579 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-ingress/e2e-inf-a-017645-openshift-default-6cc6c59dff-blk4t"] Jul 03 11:47:31.935126 ip-10-0-136-0 kubenswrapper[2579]: I0703 11:47:31.934714 2579 kuberuntime_container.go:872] "Killing container with a grace period" pod="openshift-ingress/e2e-inf-a-017645-openshift-default-6cc6c59dff-blk4t" podUID="f1b7686d-4c68-4b24-8ae1-a8cde82640cd" containerName="istio-proxy" containerID="cri-o://b4721786d3a066aaf43776587bebbe2578201c5ee90e46fe39dc18847ff81455" gracePeriod=30 Jul 03 11:47:37.230801 ip-10-0-136-0 kubenswrapper[2579]: I0703 11:47:37.230743 2579 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-inf-a-017645-openshift-default-6cc6c59dff-blk4t" Jul 03 11:47:37.339519 ip-10-0-136-0 kubenswrapper[2579]: I0703 11:47:37.339475 2579 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/f1b7686d-4c68-4b24-8ae1-a8cde82640cd-istio-data\") pod \"f1b7686d-4c68-4b24-8ae1-a8cde82640cd\" (UID: \"f1b7686d-4c68-4b24-8ae1-a8cde82640cd\") " Jul 03 11:47:37.339729 ip-10-0-136-0 kubenswrapper[2579]: I0703 11:47:37.339556 2579 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/f1b7686d-4c68-4b24-8ae1-a8cde82640cd-istio-envoy\") pod \"f1b7686d-4c68-4b24-8ae1-a8cde82640cd\" (UID: \"f1b7686d-4c68-4b24-8ae1-a8cde82640cd\") " Jul 03 11:47:37.339729 ip-10-0-136-0 kubenswrapper[2579]: I0703 11:47:37.339585 2579 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/f1b7686d-4c68-4b24-8ae1-a8cde82640cd-istio-token\") pod \"f1b7686d-4c68-4b24-8ae1-a8cde82640cd\" (UID: \"f1b7686d-4c68-4b24-8ae1-a8cde82640cd\") " Jul 03 11:47:37.339729 ip-10-0-136-0 kubenswrapper[2579]: I0703 11:47:37.339607 2579 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/f1b7686d-4c68-4b24-8ae1-a8cde82640cd-istiod-ca-cert\") pod \"f1b7686d-4c68-4b24-8ae1-a8cde82640cd\" (UID: \"f1b7686d-4c68-4b24-8ae1-a8cde82640cd\") " Jul 03 11:47:37.339729 ip-10-0-136-0 kubenswrapper[2579]: I0703 11:47:37.339652 2579 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/f1b7686d-4c68-4b24-8ae1-a8cde82640cd-workload-certs\") pod \"f1b7686d-4c68-4b24-8ae1-a8cde82640cd\" (UID: \"f1b7686d-4c68-4b24-8ae1-a8cde82640cd\") " Jul 03 11:47:37.339971 ip-10-0-136-0 kubenswrapper[2579]: I0703 11:47:37.339868 2579 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/f1b7686d-4c68-4b24-8ae1-a8cde82640cd-workload-socket\") pod \"f1b7686d-4c68-4b24-8ae1-a8cde82640cd\" (UID: \"f1b7686d-4c68-4b24-8ae1-a8cde82640cd\") " Jul 03 11:47:37.339971 ip-10-0-136-0 kubenswrapper[2579]: I0703 11:47:37.339958 2579 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/f1b7686d-4c68-4b24-8ae1-a8cde82640cd-istio-podinfo\") pod \"f1b7686d-4c68-4b24-8ae1-a8cde82640cd\" (UID: \"f1b7686d-4c68-4b24-8ae1-a8cde82640cd\") " Jul 03 11:47:37.340073 ip-10-0-136-0 kubenswrapper[2579]: I0703 11:47:37.339992 2579 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/f1b7686d-4c68-4b24-8ae1-a8cde82640cd-workload-certs" (OuterVolumeSpecName: "workload-certs") pod "f1b7686d-4c68-4b24-8ae1-a8cde82640cd" (UID: "f1b7686d-4c68-4b24-8ae1-a8cde82640cd"). InnerVolumeSpecName "workload-certs". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Jul 03 11:47:37.340073 ip-10-0-136-0 kubenswrapper[2579]: I0703 11:47:37.340006 2579 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-w92df\" (UniqueName: \"kubernetes.io/projected/f1b7686d-4c68-4b24-8ae1-a8cde82640cd-kube-api-access-w92df\") pod \"f1b7686d-4c68-4b24-8ae1-a8cde82640cd\" (UID: \"f1b7686d-4c68-4b24-8ae1-a8cde82640cd\") " Jul 03 11:47:37.340232 ip-10-0-136-0 kubenswrapper[2579]: I0703 11:47:37.340080 2579 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/f1b7686d-4c68-4b24-8ae1-a8cde82640cd-workload-socket" (OuterVolumeSpecName: "workload-socket") pod "f1b7686d-4c68-4b24-8ae1-a8cde82640cd" (UID: "f1b7686d-4c68-4b24-8ae1-a8cde82640cd"). InnerVolumeSpecName "workload-socket". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Jul 03 11:47:37.340232 ip-10-0-136-0 kubenswrapper[2579]: I0703 11:47:37.340087 2579 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/f1b7686d-4c68-4b24-8ae1-a8cde82640cd-istiod-ca-cert" (OuterVolumeSpecName: "istiod-ca-cert") pod "f1b7686d-4c68-4b24-8ae1-a8cde82640cd" (UID: "f1b7686d-4c68-4b24-8ae1-a8cde82640cd"). InnerVolumeSpecName "istiod-ca-cert". PluginName "kubernetes.io/configmap", VolumeGIDValue "" Jul 03 11:47:37.340232 ip-10-0-136-0 kubenswrapper[2579]: I0703 11:47:37.340084 2579 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/f1b7686d-4c68-4b24-8ae1-a8cde82640cd-credential-socket\") pod \"f1b7686d-4c68-4b24-8ae1-a8cde82640cd\" (UID: \"f1b7686d-4c68-4b24-8ae1-a8cde82640cd\") " Jul 03 11:47:37.340392 ip-10-0-136-0 kubenswrapper[2579]: I0703 11:47:37.340309 2579 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/f1b7686d-4c68-4b24-8ae1-a8cde82640cd-credential-socket" (OuterVolumeSpecName: "credential-socket") pod "f1b7686d-4c68-4b24-8ae1-a8cde82640cd" (UID: "f1b7686d-4c68-4b24-8ae1-a8cde82640cd"). InnerVolumeSpecName "credential-socket". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Jul 03 11:47:37.340622 ip-10-0-136-0 kubenswrapper[2579]: I0703 11:47:37.340597 2579 reconciler_common.go:299] "Volume detached for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/f1b7686d-4c68-4b24-8ae1-a8cde82640cd-workload-socket\") on node \"ip-10-0-136-0.ec2.internal\" DevicePath \"\"" Jul 03 11:47:37.340622 ip-10-0-136-0 kubenswrapper[2579]: I0703 11:47:37.340624 2579 reconciler_common.go:299] "Volume detached for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/f1b7686d-4c68-4b24-8ae1-a8cde82640cd-credential-socket\") on node \"ip-10-0-136-0.ec2.internal\" DevicePath \"\"" Jul 03 11:47:37.340977 ip-10-0-136-0 kubenswrapper[2579]: I0703 11:47:37.340636 2579 reconciler_common.go:299] "Volume detached for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/f1b7686d-4c68-4b24-8ae1-a8cde82640cd-istiod-ca-cert\") on node \"ip-10-0-136-0.ec2.internal\" DevicePath \"\"" Jul 03 11:47:37.340977 ip-10-0-136-0 kubenswrapper[2579]: I0703 11:47:37.340650 2579 reconciler_common.go:299] "Volume detached for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/f1b7686d-4c68-4b24-8ae1-a8cde82640cd-workload-certs\") on node \"ip-10-0-136-0.ec2.internal\" DevicePath \"\"" Jul 03 11:47:37.340977 ip-10-0-136-0 kubenswrapper[2579]: I0703 11:47:37.340889 2579 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/f1b7686d-4c68-4b24-8ae1-a8cde82640cd-istio-data" (OuterVolumeSpecName: "istio-data") pod "f1b7686d-4c68-4b24-8ae1-a8cde82640cd" (UID: "f1b7686d-4c68-4b24-8ae1-a8cde82640cd"). InnerVolumeSpecName "istio-data". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Jul 03 11:47:37.342223 ip-10-0-136-0 kubenswrapper[2579]: I0703 11:47:37.342169 2579 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/f1b7686d-4c68-4b24-8ae1-a8cde82640cd-istio-token" (OuterVolumeSpecName: "istio-token") pod "f1b7686d-4c68-4b24-8ae1-a8cde82640cd" (UID: "f1b7686d-4c68-4b24-8ae1-a8cde82640cd"). InnerVolumeSpecName "istio-token". PluginName "kubernetes.io/projected", VolumeGIDValue "" Jul 03 11:47:37.342465 ip-10-0-136-0 kubenswrapper[2579]: I0703 11:47:37.342435 2579 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/f1b7686d-4c68-4b24-8ae1-a8cde82640cd-istio-envoy" (OuterVolumeSpecName: "istio-envoy") pod "f1b7686d-4c68-4b24-8ae1-a8cde82640cd" (UID: "f1b7686d-4c68-4b24-8ae1-a8cde82640cd"). InnerVolumeSpecName "istio-envoy". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Jul 03 11:47:37.342822 ip-10-0-136-0 kubenswrapper[2579]: I0703 11:47:37.342790 2579 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/downward-api/f1b7686d-4c68-4b24-8ae1-a8cde82640cd-istio-podinfo" (OuterVolumeSpecName: "istio-podinfo") pod "f1b7686d-4c68-4b24-8ae1-a8cde82640cd" (UID: "f1b7686d-4c68-4b24-8ae1-a8cde82640cd"). InnerVolumeSpecName "istio-podinfo". PluginName "kubernetes.io/downward-api", VolumeGIDValue "" Jul 03 11:47:37.342896 ip-10-0-136-0 kubenswrapper[2579]: I0703 11:47:37.342881 2579 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/f1b7686d-4c68-4b24-8ae1-a8cde82640cd-kube-api-access-w92df" (OuterVolumeSpecName: "kube-api-access-w92df") pod "f1b7686d-4c68-4b24-8ae1-a8cde82640cd" (UID: "f1b7686d-4c68-4b24-8ae1-a8cde82640cd"). InnerVolumeSpecName "kube-api-access-w92df". PluginName "kubernetes.io/projected", VolumeGIDValue "" Jul 03 11:47:37.441485 ip-10-0-136-0 kubenswrapper[2579]: I0703 11:47:37.441443 2579 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-w92df\" (UniqueName: \"kubernetes.io/projected/f1b7686d-4c68-4b24-8ae1-a8cde82640cd-kube-api-access-w92df\") on node \"ip-10-0-136-0.ec2.internal\" DevicePath \"\"" Jul 03 11:47:37.441485 ip-10-0-136-0 kubenswrapper[2579]: I0703 11:47:37.441484 2579 reconciler_common.go:299] "Volume detached for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/f1b7686d-4c68-4b24-8ae1-a8cde82640cd-istio-data\") on node \"ip-10-0-136-0.ec2.internal\" DevicePath \"\"" Jul 03 11:47:37.441756 ip-10-0-136-0 kubenswrapper[2579]: I0703 11:47:37.441497 2579 reconciler_common.go:299] "Volume detached for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/f1b7686d-4c68-4b24-8ae1-a8cde82640cd-istio-envoy\") on node \"ip-10-0-136-0.ec2.internal\" DevicePath \"\"" Jul 03 11:47:37.441756 ip-10-0-136-0 kubenswrapper[2579]: I0703 11:47:37.441507 2579 reconciler_common.go:299] "Volume detached for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/f1b7686d-4c68-4b24-8ae1-a8cde82640cd-istio-token\") on node \"ip-10-0-136-0.ec2.internal\" DevicePath \"\"" Jul 03 11:47:37.441756 ip-10-0-136-0 kubenswrapper[2579]: I0703 11:47:37.441519 2579 reconciler_common.go:299] "Volume detached for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/f1b7686d-4c68-4b24-8ae1-a8cde82640cd-istio-podinfo\") on node \"ip-10-0-136-0.ec2.internal\" DevicePath \"\"" Jul 03 11:47:37.789733 ip-10-0-136-0 kubenswrapper[2579]: I0703 11:47:37.789690 2579 generic.go:356] "Generic (PLEG): container finished" podID="f1b7686d-4c68-4b24-8ae1-a8cde82640cd" containerID="b4721786d3a066aaf43776587bebbe2578201c5ee90e46fe39dc18847ff81455" exitCode=0 Jul 03 11:47:37.789960 ip-10-0-136-0 kubenswrapper[2579]: I0703 11:47:37.789747 2579 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-inf-a-017645-openshift-default-6cc6c59dff-blk4t" Jul 03 11:47:37.789960 ip-10-0-136-0 kubenswrapper[2579]: I0703 11:47:37.789785 2579 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-inf-a-017645-openshift-default-6cc6c59dff-blk4t" event={"ID":"f1b7686d-4c68-4b24-8ae1-a8cde82640cd","Type":"ContainerDied","Data":"b4721786d3a066aaf43776587bebbe2578201c5ee90e46fe39dc18847ff81455"} Jul 03 11:47:37.789960 ip-10-0-136-0 kubenswrapper[2579]: I0703 11:47:37.789847 2579 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-inf-a-017645-openshift-default-6cc6c59dff-blk4t" event={"ID":"f1b7686d-4c68-4b24-8ae1-a8cde82640cd","Type":"ContainerDied","Data":"4647de9da687bd236c82113dd6ef8a35ee79164aebd3a2e01e8cbea04d66ea01"} Jul 03 11:47:37.789960 ip-10-0-136-0 kubenswrapper[2579]: I0703 11:47:37.789872 2579 scope.go:117] "RemoveContainer" containerID="b4721786d3a066aaf43776587bebbe2578201c5ee90e46fe39dc18847ff81455" Jul 03 11:47:37.801108 ip-10-0-136-0 kubenswrapper[2579]: I0703 11:47:37.801080 2579 scope.go:117] "RemoveContainer" containerID="816339a42db6956f3ce9c7ce34ecca89b8d3322a61e878e59138ad28d1a6ad75" Jul 03 11:47:37.810940 ip-10-0-136-0 kubenswrapper[2579]: I0703 11:47:37.810915 2579 scope.go:117] "RemoveContainer" containerID="b4721786d3a066aaf43776587bebbe2578201c5ee90e46fe39dc18847ff81455" Jul 03 11:47:37.811336 ip-10-0-136-0 kubenswrapper[2579]: E0703 11:47:37.811299 2579 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"b4721786d3a066aaf43776587bebbe2578201c5ee90e46fe39dc18847ff81455\": container with ID starting with b4721786d3a066aaf43776587bebbe2578201c5ee90e46fe39dc18847ff81455 not found: ID does not exist" containerID="b4721786d3a066aaf43776587bebbe2578201c5ee90e46fe39dc18847ff81455" Jul 03 11:47:37.811469 ip-10-0-136-0 kubenswrapper[2579]: I0703 11:47:37.811340 2579 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"b4721786d3a066aaf43776587bebbe2578201c5ee90e46fe39dc18847ff81455"} err="failed to get container status \"b4721786d3a066aaf43776587bebbe2578201c5ee90e46fe39dc18847ff81455\": rpc error: code = NotFound desc = could not find container \"b4721786d3a066aaf43776587bebbe2578201c5ee90e46fe39dc18847ff81455\": container with ID starting with b4721786d3a066aaf43776587bebbe2578201c5ee90e46fe39dc18847ff81455 not found: ID does not exist" Jul 03 11:47:37.811469 ip-10-0-136-0 kubenswrapper[2579]: I0703 11:47:37.811362 2579 scope.go:117] "RemoveContainer" containerID="816339a42db6956f3ce9c7ce34ecca89b8d3322a61e878e59138ad28d1a6ad75" Jul 03 11:47:37.811666 ip-10-0-136-0 kubenswrapper[2579]: E0703 11:47:37.811645 2579 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"816339a42db6956f3ce9c7ce34ecca89b8d3322a61e878e59138ad28d1a6ad75\": container with ID starting with 816339a42db6956f3ce9c7ce34ecca89b8d3322a61e878e59138ad28d1a6ad75 not found: ID does not exist" containerID="816339a42db6956f3ce9c7ce34ecca89b8d3322a61e878e59138ad28d1a6ad75" Jul 03 11:47:37.811717 ip-10-0-136-0 kubenswrapper[2579]: I0703 11:47:37.811670 2579 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"816339a42db6956f3ce9c7ce34ecca89b8d3322a61e878e59138ad28d1a6ad75"} err="failed to get container status \"816339a42db6956f3ce9c7ce34ecca89b8d3322a61e878e59138ad28d1a6ad75\": rpc error: code = NotFound desc = could not find container \"816339a42db6956f3ce9c7ce34ecca89b8d3322a61e878e59138ad28d1a6ad75\": container with ID starting with 816339a42db6956f3ce9c7ce34ecca89b8d3322a61e878e59138ad28d1a6ad75 not found: ID does not exist" Jul 03 11:47:37.828453 ip-10-0-136-0 kubenswrapper[2579]: I0703 11:47:37.828394 2579 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-ingress/e2e-inf-a-017645-openshift-default-6cc6c59dff-blk4t"] Jul 03 11:47:37.834818 ip-10-0-136-0 kubenswrapper[2579]: I0703 11:47:37.834765 2579 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["openshift-ingress/e2e-inf-a-017645-openshift-default-6cc6c59dff-blk4t"] Jul 03 11:47:37.992636 ip-10-0-136-0 kubenswrapper[2579]: I0703 11:47:37.992586 2579 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="f1b7686d-4c68-4b24-8ae1-a8cde82640cd" path="/var/lib/kubelet/pods/f1b7686d-4c68-4b24-8ae1-a8cde82640cd/volumes" Jul 03 11:47:39.940016 ip-10-0-136-0 kubenswrapper[2579]: I0703 11:47:39.939967 2579 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-ingress/e2e-inf-b-44ab65-openshift-default-5946888467-59l9p"] Jul 03 11:47:39.940443 ip-10-0-136-0 kubenswrapper[2579]: I0703 11:47:39.940253 2579 kuberuntime_container.go:872] "Killing container with a grace period" pod="openshift-ingress/e2e-inf-b-44ab65-openshift-default-5946888467-59l9p" podUID="623e6c46-3354-47da-8157-83d80d524b3d" containerName="istio-proxy" containerID="cri-o://00b2d88be3ba88496c4f88bb2fb3043304e201e572edc51ec56e0a7fb1def652" gracePeriod=30 Jul 03 11:47:40.962496 ip-10-0-136-0 kubenswrapper[2579]: I0703 11:47:40.962448 2579 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-ingress/e2e-api-a-515460-openshift-default-897bbd7c-xhjtn"] Jul 03 11:47:40.962984 ip-10-0-136-0 kubenswrapper[2579]: I0703 11:47:40.962968 2579 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="f1b7686d-4c68-4b24-8ae1-a8cde82640cd" containerName="istio-proxy" Jul 03 11:47:40.963038 ip-10-0-136-0 kubenswrapper[2579]: I0703 11:47:40.962987 2579 state_mem.go:107] "Deleted CPUSet assignment" podUID="f1b7686d-4c68-4b24-8ae1-a8cde82640cd" containerName="istio-proxy" Jul 03 11:47:40.963038 ip-10-0-136-0 kubenswrapper[2579]: I0703 11:47:40.963004 2579 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="d8df6c2a-e6cf-4a9e-88d6-86ec7f7461e1" containerName="main" Jul 03 11:47:40.963038 ip-10-0-136-0 kubenswrapper[2579]: I0703 11:47:40.963009 2579 state_mem.go:107] "Deleted CPUSet assignment" podUID="d8df6c2a-e6cf-4a9e-88d6-86ec7f7461e1" containerName="main" Jul 03 11:47:40.963140 ip-10-0-136-0 kubenswrapper[2579]: I0703 11:47:40.963107 2579 memory_manager.go:368] "RemoveStaleState removing state" podUID="f1b7686d-4c68-4b24-8ae1-a8cde82640cd" containerName="istio-proxy" Jul 03 11:47:40.963140 ip-10-0-136-0 kubenswrapper[2579]: I0703 11:47:40.963114 2579 memory_manager.go:368] "RemoveStaleState removing state" podUID="f1b7686d-4c68-4b24-8ae1-a8cde82640cd" containerName="istio-proxy" Jul 03 11:47:40.963140 ip-10-0-136-0 kubenswrapper[2579]: I0703 11:47:40.963124 2579 memory_manager.go:368] "RemoveStaleState removing state" podUID="d8df6c2a-e6cf-4a9e-88d6-86ec7f7461e1" containerName="main" Jul 03 11:47:40.968315 ip-10-0-136-0 kubenswrapper[2579]: I0703 11:47:40.968285 2579 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-api-a-515460-openshift-default-897bbd7c-xhjtn" Jul 03 11:47:40.974025 ip-10-0-136-0 kubenswrapper[2579]: I0703 11:47:40.973994 2579 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress\"/\"e2e-api-a-515460-openshift-default-dockercfg-86p8r\"" Jul 03 11:47:41.027536 ip-10-0-136-0 kubenswrapper[2579]: I0703 11:47:41.027476 2579 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/e2e-api-a-515460-openshift-default-897bbd7c-xhjtn"] Jul 03 11:47:41.076892 ip-10-0-136-0 kubenswrapper[2579]: I0703 11:47:41.076840 2579 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/75a2fbf5-528e-4910-b92b-e575c5ccf0f0-workload-socket\") pod \"e2e-api-a-515460-openshift-default-897bbd7c-xhjtn\" (UID: \"75a2fbf5-528e-4910-b92b-e575c5ccf0f0\") " pod="openshift-ingress/e2e-api-a-515460-openshift-default-897bbd7c-xhjtn" Jul 03 11:47:41.077115 ip-10-0-136-0 kubenswrapper[2579]: I0703 11:47:41.076904 2579 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-5nspb\" (UniqueName: \"kubernetes.io/projected/75a2fbf5-528e-4910-b92b-e575c5ccf0f0-kube-api-access-5nspb\") pod \"e2e-api-a-515460-openshift-default-897bbd7c-xhjtn\" (UID: \"75a2fbf5-528e-4910-b92b-e575c5ccf0f0\") " pod="openshift-ingress/e2e-api-a-515460-openshift-default-897bbd7c-xhjtn" Jul 03 11:47:41.077115 ip-10-0-136-0 kubenswrapper[2579]: I0703 11:47:41.076946 2579 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/75a2fbf5-528e-4910-b92b-e575c5ccf0f0-workload-certs\") pod \"e2e-api-a-515460-openshift-default-897bbd7c-xhjtn\" (UID: \"75a2fbf5-528e-4910-b92b-e575c5ccf0f0\") " pod="openshift-ingress/e2e-api-a-515460-openshift-default-897bbd7c-xhjtn" Jul 03 11:47:41.077115 ip-10-0-136-0 kubenswrapper[2579]: I0703 11:47:41.076970 2579 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/75a2fbf5-528e-4910-b92b-e575c5ccf0f0-istio-token\") pod \"e2e-api-a-515460-openshift-default-897bbd7c-xhjtn\" (UID: \"75a2fbf5-528e-4910-b92b-e575c5ccf0f0\") " pod="openshift-ingress/e2e-api-a-515460-openshift-default-897bbd7c-xhjtn" Jul 03 11:47:41.077115 ip-10-0-136-0 kubenswrapper[2579]: I0703 11:47:41.077036 2579 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/75a2fbf5-528e-4910-b92b-e575c5ccf0f0-credential-socket\") pod \"e2e-api-a-515460-openshift-default-897bbd7c-xhjtn\" (UID: \"75a2fbf5-528e-4910-b92b-e575c5ccf0f0\") " pod="openshift-ingress/e2e-api-a-515460-openshift-default-897bbd7c-xhjtn" Jul 03 11:47:41.077115 ip-10-0-136-0 kubenswrapper[2579]: I0703 11:47:41.077094 2579 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/75a2fbf5-528e-4910-b92b-e575c5ccf0f0-istio-envoy\") pod \"e2e-api-a-515460-openshift-default-897bbd7c-xhjtn\" (UID: \"75a2fbf5-528e-4910-b92b-e575c5ccf0f0\") " pod="openshift-ingress/e2e-api-a-515460-openshift-default-897bbd7c-xhjtn" Jul 03 11:47:41.077399 ip-10-0-136-0 kubenswrapper[2579]: I0703 11:47:41.077135 2579 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/75a2fbf5-528e-4910-b92b-e575c5ccf0f0-istio-podinfo\") pod \"e2e-api-a-515460-openshift-default-897bbd7c-xhjtn\" (UID: \"75a2fbf5-528e-4910-b92b-e575c5ccf0f0\") " pod="openshift-ingress/e2e-api-a-515460-openshift-default-897bbd7c-xhjtn" Jul 03 11:47:41.077399 ip-10-0-136-0 kubenswrapper[2579]: I0703 11:47:41.077158 2579 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/75a2fbf5-528e-4910-b92b-e575c5ccf0f0-istiod-ca-cert\") pod \"e2e-api-a-515460-openshift-default-897bbd7c-xhjtn\" (UID: \"75a2fbf5-528e-4910-b92b-e575c5ccf0f0\") " pod="openshift-ingress/e2e-api-a-515460-openshift-default-897bbd7c-xhjtn" Jul 03 11:47:41.077399 ip-10-0-136-0 kubenswrapper[2579]: I0703 11:47:41.077248 2579 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/75a2fbf5-528e-4910-b92b-e575c5ccf0f0-istio-data\") pod \"e2e-api-a-515460-openshift-default-897bbd7c-xhjtn\" (UID: \"75a2fbf5-528e-4910-b92b-e575c5ccf0f0\") " pod="openshift-ingress/e2e-api-a-515460-openshift-default-897bbd7c-xhjtn" Jul 03 11:47:41.178536 ip-10-0-136-0 kubenswrapper[2579]: I0703 11:47:41.178487 2579 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/75a2fbf5-528e-4910-b92b-e575c5ccf0f0-workload-socket\") pod \"e2e-api-a-515460-openshift-default-897bbd7c-xhjtn\" (UID: \"75a2fbf5-528e-4910-b92b-e575c5ccf0f0\") " pod="openshift-ingress/e2e-api-a-515460-openshift-default-897bbd7c-xhjtn" Jul 03 11:47:41.178536 ip-10-0-136-0 kubenswrapper[2579]: I0703 11:47:41.178542 2579 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-5nspb\" (UniqueName: \"kubernetes.io/projected/75a2fbf5-528e-4910-b92b-e575c5ccf0f0-kube-api-access-5nspb\") pod \"e2e-api-a-515460-openshift-default-897bbd7c-xhjtn\" (UID: \"75a2fbf5-528e-4910-b92b-e575c5ccf0f0\") " pod="openshift-ingress/e2e-api-a-515460-openshift-default-897bbd7c-xhjtn" Jul 03 11:47:41.178843 ip-10-0-136-0 kubenswrapper[2579]: I0703 11:47:41.178571 2579 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/75a2fbf5-528e-4910-b92b-e575c5ccf0f0-workload-certs\") pod \"e2e-api-a-515460-openshift-default-897bbd7c-xhjtn\" (UID: \"75a2fbf5-528e-4910-b92b-e575c5ccf0f0\") " pod="openshift-ingress/e2e-api-a-515460-openshift-default-897bbd7c-xhjtn" Jul 03 11:47:41.178843 ip-10-0-136-0 kubenswrapper[2579]: I0703 11:47:41.178591 2579 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/75a2fbf5-528e-4910-b92b-e575c5ccf0f0-istio-token\") pod \"e2e-api-a-515460-openshift-default-897bbd7c-xhjtn\" (UID: \"75a2fbf5-528e-4910-b92b-e575c5ccf0f0\") " pod="openshift-ingress/e2e-api-a-515460-openshift-default-897bbd7c-xhjtn" Jul 03 11:47:41.178843 ip-10-0-136-0 kubenswrapper[2579]: I0703 11:47:41.178609 2579 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/75a2fbf5-528e-4910-b92b-e575c5ccf0f0-credential-socket\") pod \"e2e-api-a-515460-openshift-default-897bbd7c-xhjtn\" (UID: \"75a2fbf5-528e-4910-b92b-e575c5ccf0f0\") " pod="openshift-ingress/e2e-api-a-515460-openshift-default-897bbd7c-xhjtn" Jul 03 11:47:41.178843 ip-10-0-136-0 kubenswrapper[2579]: I0703 11:47:41.178647 2579 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/75a2fbf5-528e-4910-b92b-e575c5ccf0f0-istio-envoy\") pod \"e2e-api-a-515460-openshift-default-897bbd7c-xhjtn\" (UID: \"75a2fbf5-528e-4910-b92b-e575c5ccf0f0\") " pod="openshift-ingress/e2e-api-a-515460-openshift-default-897bbd7c-xhjtn" Jul 03 11:47:41.178843 ip-10-0-136-0 kubenswrapper[2579]: I0703 11:47:41.178687 2579 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/75a2fbf5-528e-4910-b92b-e575c5ccf0f0-istio-podinfo\") pod \"e2e-api-a-515460-openshift-default-897bbd7c-xhjtn\" (UID: \"75a2fbf5-528e-4910-b92b-e575c5ccf0f0\") " pod="openshift-ingress/e2e-api-a-515460-openshift-default-897bbd7c-xhjtn" Jul 03 11:47:41.178843 ip-10-0-136-0 kubenswrapper[2579]: I0703 11:47:41.178722 2579 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/75a2fbf5-528e-4910-b92b-e575c5ccf0f0-istiod-ca-cert\") pod \"e2e-api-a-515460-openshift-default-897bbd7c-xhjtn\" (UID: \"75a2fbf5-528e-4910-b92b-e575c5ccf0f0\") " pod="openshift-ingress/e2e-api-a-515460-openshift-default-897bbd7c-xhjtn" Jul 03 11:47:41.178843 ip-10-0-136-0 kubenswrapper[2579]: I0703 11:47:41.178780 2579 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/75a2fbf5-528e-4910-b92b-e575c5ccf0f0-istio-data\") pod \"e2e-api-a-515460-openshift-default-897bbd7c-xhjtn\" (UID: \"75a2fbf5-528e-4910-b92b-e575c5ccf0f0\") " pod="openshift-ingress/e2e-api-a-515460-openshift-default-897bbd7c-xhjtn" Jul 03 11:47:41.179158 ip-10-0-136-0 kubenswrapper[2579]: I0703 11:47:41.179016 2579 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/75a2fbf5-528e-4910-b92b-e575c5ccf0f0-workload-socket\") pod \"e2e-api-a-515460-openshift-default-897bbd7c-xhjtn\" (UID: \"75a2fbf5-528e-4910-b92b-e575c5ccf0f0\") " pod="openshift-ingress/e2e-api-a-515460-openshift-default-897bbd7c-xhjtn" Jul 03 11:47:41.179158 ip-10-0-136-0 kubenswrapper[2579]: I0703 11:47:41.179083 2579 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/75a2fbf5-528e-4910-b92b-e575c5ccf0f0-credential-socket\") pod \"e2e-api-a-515460-openshift-default-897bbd7c-xhjtn\" (UID: \"75a2fbf5-528e-4910-b92b-e575c5ccf0f0\") " pod="openshift-ingress/e2e-api-a-515460-openshift-default-897bbd7c-xhjtn" Jul 03 11:47:41.179348 ip-10-0-136-0 kubenswrapper[2579]: I0703 11:47:41.179233 2579 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/75a2fbf5-528e-4910-b92b-e575c5ccf0f0-istio-data\") pod \"e2e-api-a-515460-openshift-default-897bbd7c-xhjtn\" (UID: \"75a2fbf5-528e-4910-b92b-e575c5ccf0f0\") " pod="openshift-ingress/e2e-api-a-515460-openshift-default-897bbd7c-xhjtn" Jul 03 11:47:41.179348 ip-10-0-136-0 kubenswrapper[2579]: I0703 11:47:41.179330 2579 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/75a2fbf5-528e-4910-b92b-e575c5ccf0f0-workload-certs\") pod \"e2e-api-a-515460-openshift-default-897bbd7c-xhjtn\" (UID: \"75a2fbf5-528e-4910-b92b-e575c5ccf0f0\") " pod="openshift-ingress/e2e-api-a-515460-openshift-default-897bbd7c-xhjtn" Jul 03 11:47:41.179685 ip-10-0-136-0 kubenswrapper[2579]: I0703 11:47:41.179586 2579 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/75a2fbf5-528e-4910-b92b-e575c5ccf0f0-istiod-ca-cert\") pod \"e2e-api-a-515460-openshift-default-897bbd7c-xhjtn\" (UID: \"75a2fbf5-528e-4910-b92b-e575c5ccf0f0\") " pod="openshift-ingress/e2e-api-a-515460-openshift-default-897bbd7c-xhjtn" Jul 03 11:47:41.181342 ip-10-0-136-0 kubenswrapper[2579]: I0703 11:47:41.181310 2579 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/75a2fbf5-528e-4910-b92b-e575c5ccf0f0-istio-envoy\") pod \"e2e-api-a-515460-openshift-default-897bbd7c-xhjtn\" (UID: \"75a2fbf5-528e-4910-b92b-e575c5ccf0f0\") " pod="openshift-ingress/e2e-api-a-515460-openshift-default-897bbd7c-xhjtn" Jul 03 11:47:41.181576 ip-10-0-136-0 kubenswrapper[2579]: I0703 11:47:41.181555 2579 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/75a2fbf5-528e-4910-b92b-e575c5ccf0f0-istio-podinfo\") pod \"e2e-api-a-515460-openshift-default-897bbd7c-xhjtn\" (UID: \"75a2fbf5-528e-4910-b92b-e575c5ccf0f0\") " pod="openshift-ingress/e2e-api-a-515460-openshift-default-897bbd7c-xhjtn" Jul 03 11:47:41.194431 ip-10-0-136-0 kubenswrapper[2579]: I0703 11:47:41.194399 2579 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-5nspb\" (UniqueName: \"kubernetes.io/projected/75a2fbf5-528e-4910-b92b-e575c5ccf0f0-kube-api-access-5nspb\") pod \"e2e-api-a-515460-openshift-default-897bbd7c-xhjtn\" (UID: \"75a2fbf5-528e-4910-b92b-e575c5ccf0f0\") " pod="openshift-ingress/e2e-api-a-515460-openshift-default-897bbd7c-xhjtn" Jul 03 11:47:41.194601 ip-10-0-136-0 kubenswrapper[2579]: I0703 11:47:41.194398 2579 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/75a2fbf5-528e-4910-b92b-e575c5ccf0f0-istio-token\") pod \"e2e-api-a-515460-openshift-default-897bbd7c-xhjtn\" (UID: \"75a2fbf5-528e-4910-b92b-e575c5ccf0f0\") " pod="openshift-ingress/e2e-api-a-515460-openshift-default-897bbd7c-xhjtn" Jul 03 11:47:41.280261 ip-10-0-136-0 kubenswrapper[2579]: I0703 11:47:41.280216 2579 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-api-a-515460-openshift-default-897bbd7c-xhjtn" Jul 03 11:47:41.479543 ip-10-0-136-0 kubenswrapper[2579]: I0703 11:47:41.479509 2579 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/e2e-api-a-515460-openshift-default-897bbd7c-xhjtn"] Jul 03 11:47:41.480048 ip-10-0-136-0 kubenswrapper[2579]: W0703 11:47:41.480024 2579 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod75a2fbf5_528e_4910_b92b_e575c5ccf0f0.slice/crio-7e44d9699fe732bb5c59ab10f2e036fab09281da9de8f89b6de38839792d06cb WatchSource:0}: Error finding container 7e44d9699fe732bb5c59ab10f2e036fab09281da9de8f89b6de38839792d06cb: Status 404 returned error can't find the container with id 7e44d9699fe732bb5c59ab10f2e036fab09281da9de8f89b6de38839792d06cb Jul 03 11:47:41.482327 ip-10-0-136-0 kubenswrapper[2579]: I0703 11:47:41.482290 2579 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"30892156Ki","pods":"250"} Jul 03 11:47:41.482434 ip-10-0-136-0 kubenswrapper[2579]: I0703 11:47:41.482368 2579 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"30892156Ki","pods":"250"} Jul 03 11:47:41.482434 ip-10-0-136-0 kubenswrapper[2579]: I0703 11:47:41.482398 2579 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"30892156Ki","pods":"250"} Jul 03 11:47:41.820552 ip-10-0-136-0 kubenswrapper[2579]: I0703 11:47:41.820447 2579 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-api-a-515460-openshift-default-897bbd7c-xhjtn" event={"ID":"75a2fbf5-528e-4910-b92b-e575c5ccf0f0","Type":"ContainerStarted","Data":"19cad7f406870be1a95f8e2be936db1ea845975d6c74789ee75b6f610ab44c44"} Jul 03 11:47:41.820552 ip-10-0-136-0 kubenswrapper[2579]: I0703 11:47:41.820504 2579 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-api-a-515460-openshift-default-897bbd7c-xhjtn" event={"ID":"75a2fbf5-528e-4910-b92b-e575c5ccf0f0","Type":"ContainerStarted","Data":"7e44d9699fe732bb5c59ab10f2e036fab09281da9de8f89b6de38839792d06cb"} Jul 03 11:47:41.862547 ip-10-0-136-0 kubenswrapper[2579]: I0703 11:47:41.862491 2579 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-ingress/e2e-api-a-515460-openshift-default-897bbd7c-xhjtn" podStartSLOduration=1.862473156 podStartE2EDuration="1.862473156s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-03 11:47:40 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-03 11:47:41.860838315 +0000 UTC m=+2118.449142668" watchObservedRunningTime="2026-07-03 11:47:41.862473156 +0000 UTC m=+2118.450777518" Jul 03 11:47:42.281044 ip-10-0-136-0 kubenswrapper[2579]: I0703 11:47:42.280997 2579 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="openshift-ingress/e2e-api-a-515460-openshift-default-897bbd7c-xhjtn" Jul 03 11:47:42.287059 ip-10-0-136-0 kubenswrapper[2579]: I0703 11:47:42.287025 2579 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="openshift-ingress/e2e-api-a-515460-openshift-default-897bbd7c-xhjtn" Jul 03 11:47:42.834485 ip-10-0-136-0 kubenswrapper[2579]: I0703 11:47:42.834446 2579 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/e2e-api-a-515460-openshift-default-897bbd7c-xhjtn" Jul 03 11:47:42.835723 ip-10-0-136-0 kubenswrapper[2579]: I0703 11:47:42.835704 2579 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/e2e-api-a-515460-openshift-default-897bbd7c-xhjtn" Jul 03 11:47:45.235353 ip-10-0-136-0 kubenswrapper[2579]: I0703 11:47:45.235317 2579 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-inf-b-44ab65-openshift-default-5946888467-59l9p" Jul 03 11:47:45.324272 ip-10-0-136-0 kubenswrapper[2579]: I0703 11:47:45.324181 2579 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/623e6c46-3354-47da-8157-83d80d524b3d-istio-data\") pod \"623e6c46-3354-47da-8157-83d80d524b3d\" (UID: \"623e6c46-3354-47da-8157-83d80d524b3d\") " Jul 03 11:47:45.324476 ip-10-0-136-0 kubenswrapper[2579]: I0703 11:47:45.324309 2579 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/623e6c46-3354-47da-8157-83d80d524b3d-istio-podinfo\") pod \"623e6c46-3354-47da-8157-83d80d524b3d\" (UID: \"623e6c46-3354-47da-8157-83d80d524b3d\") " Jul 03 11:47:45.324476 ip-10-0-136-0 kubenswrapper[2579]: I0703 11:47:45.324331 2579 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/623e6c46-3354-47da-8157-83d80d524b3d-istio-token\") pod \"623e6c46-3354-47da-8157-83d80d524b3d\" (UID: \"623e6c46-3354-47da-8157-83d80d524b3d\") " Jul 03 11:47:45.324476 ip-10-0-136-0 kubenswrapper[2579]: I0703 11:47:45.324459 2579 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/623e6c46-3354-47da-8157-83d80d524b3d-workload-socket\") pod \"623e6c46-3354-47da-8157-83d80d524b3d\" (UID: \"623e6c46-3354-47da-8157-83d80d524b3d\") " Jul 03 11:47:45.324651 ip-10-0-136-0 kubenswrapper[2579]: I0703 11:47:45.324572 2579 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-6rjj7\" (UniqueName: \"kubernetes.io/projected/623e6c46-3354-47da-8157-83d80d524b3d-kube-api-access-6rjj7\") pod \"623e6c46-3354-47da-8157-83d80d524b3d\" (UID: \"623e6c46-3354-47da-8157-83d80d524b3d\") " Jul 03 11:47:45.324651 ip-10-0-136-0 kubenswrapper[2579]: I0703 11:47:45.324613 2579 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/623e6c46-3354-47da-8157-83d80d524b3d-istiod-ca-cert\") pod \"623e6c46-3354-47da-8157-83d80d524b3d\" (UID: \"623e6c46-3354-47da-8157-83d80d524b3d\") " Jul 03 11:47:45.324651 ip-10-0-136-0 kubenswrapper[2579]: I0703 11:47:45.324644 2579 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/623e6c46-3354-47da-8157-83d80d524b3d-istio-envoy\") pod \"623e6c46-3354-47da-8157-83d80d524b3d\" (UID: \"623e6c46-3354-47da-8157-83d80d524b3d\") " Jul 03 11:47:45.324796 ip-10-0-136-0 kubenswrapper[2579]: I0703 11:47:45.324751 2579 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/623e6c46-3354-47da-8157-83d80d524b3d-credential-socket\") pod \"623e6c46-3354-47da-8157-83d80d524b3d\" (UID: \"623e6c46-3354-47da-8157-83d80d524b3d\") " Jul 03 11:47:45.324796 ip-10-0-136-0 kubenswrapper[2579]: I0703 11:47:45.324751 2579 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/623e6c46-3354-47da-8157-83d80d524b3d-workload-socket" (OuterVolumeSpecName: "workload-socket") pod "623e6c46-3354-47da-8157-83d80d524b3d" (UID: "623e6c46-3354-47da-8157-83d80d524b3d"). InnerVolumeSpecName "workload-socket". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Jul 03 11:47:45.324796 ip-10-0-136-0 kubenswrapper[2579]: I0703 11:47:45.324776 2579 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/623e6c46-3354-47da-8157-83d80d524b3d-workload-certs\") pod \"623e6c46-3354-47da-8157-83d80d524b3d\" (UID: \"623e6c46-3354-47da-8157-83d80d524b3d\") " Jul 03 11:47:45.325102 ip-10-0-136-0 kubenswrapper[2579]: I0703 11:47:45.325064 2579 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/623e6c46-3354-47da-8157-83d80d524b3d-credential-socket" (OuterVolumeSpecName: "credential-socket") pod "623e6c46-3354-47da-8157-83d80d524b3d" (UID: "623e6c46-3354-47da-8157-83d80d524b3d"). InnerVolumeSpecName "credential-socket". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Jul 03 11:47:45.325276 ip-10-0-136-0 kubenswrapper[2579]: I0703 11:47:45.325130 2579 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/623e6c46-3354-47da-8157-83d80d524b3d-istio-data" (OuterVolumeSpecName: "istio-data") pod "623e6c46-3354-47da-8157-83d80d524b3d" (UID: "623e6c46-3354-47da-8157-83d80d524b3d"). InnerVolumeSpecName "istio-data". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Jul 03 11:47:45.325276 ip-10-0-136-0 kubenswrapper[2579]: I0703 11:47:45.325162 2579 reconciler_common.go:299] "Volume detached for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/623e6c46-3354-47da-8157-83d80d524b3d-credential-socket\") on node \"ip-10-0-136-0.ec2.internal\" DevicePath \"\"" Jul 03 11:47:45.325276 ip-10-0-136-0 kubenswrapper[2579]: I0703 11:47:45.325227 2579 reconciler_common.go:299] "Volume detached for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/623e6c46-3354-47da-8157-83d80d524b3d-workload-socket\") on node \"ip-10-0-136-0.ec2.internal\" DevicePath \"\"" Jul 03 11:47:45.325451 ip-10-0-136-0 kubenswrapper[2579]: I0703 11:47:45.325344 2579 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/623e6c46-3354-47da-8157-83d80d524b3d-istiod-ca-cert" (OuterVolumeSpecName: "istiod-ca-cert") pod "623e6c46-3354-47da-8157-83d80d524b3d" (UID: "623e6c46-3354-47da-8157-83d80d524b3d"). InnerVolumeSpecName "istiod-ca-cert". PluginName "kubernetes.io/configmap", VolumeGIDValue "" Jul 03 11:47:45.325451 ip-10-0-136-0 kubenswrapper[2579]: I0703 11:47:45.325395 2579 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/623e6c46-3354-47da-8157-83d80d524b3d-workload-certs" (OuterVolumeSpecName: "workload-certs") pod "623e6c46-3354-47da-8157-83d80d524b3d" (UID: "623e6c46-3354-47da-8157-83d80d524b3d"). InnerVolumeSpecName "workload-certs". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Jul 03 11:47:45.327537 ip-10-0-136-0 kubenswrapper[2579]: I0703 11:47:45.327506 2579 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/downward-api/623e6c46-3354-47da-8157-83d80d524b3d-istio-podinfo" (OuterVolumeSpecName: "istio-podinfo") pod "623e6c46-3354-47da-8157-83d80d524b3d" (UID: "623e6c46-3354-47da-8157-83d80d524b3d"). InnerVolumeSpecName "istio-podinfo". PluginName "kubernetes.io/downward-api", VolumeGIDValue "" Jul 03 11:47:45.327658 ip-10-0-136-0 kubenswrapper[2579]: I0703 11:47:45.327510 2579 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/623e6c46-3354-47da-8157-83d80d524b3d-istio-token" (OuterVolumeSpecName: "istio-token") pod "623e6c46-3354-47da-8157-83d80d524b3d" (UID: "623e6c46-3354-47da-8157-83d80d524b3d"). InnerVolumeSpecName "istio-token". PluginName "kubernetes.io/projected", VolumeGIDValue "" Jul 03 11:47:45.327658 ip-10-0-136-0 kubenswrapper[2579]: I0703 11:47:45.327570 2579 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/623e6c46-3354-47da-8157-83d80d524b3d-istio-envoy" (OuterVolumeSpecName: "istio-envoy") pod "623e6c46-3354-47da-8157-83d80d524b3d" (UID: "623e6c46-3354-47da-8157-83d80d524b3d"). InnerVolumeSpecName "istio-envoy". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Jul 03 11:47:45.327658 ip-10-0-136-0 kubenswrapper[2579]: I0703 11:47:45.327644 2579 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/623e6c46-3354-47da-8157-83d80d524b3d-kube-api-access-6rjj7" (OuterVolumeSpecName: "kube-api-access-6rjj7") pod "623e6c46-3354-47da-8157-83d80d524b3d" (UID: "623e6c46-3354-47da-8157-83d80d524b3d"). InnerVolumeSpecName "kube-api-access-6rjj7". PluginName "kubernetes.io/projected", VolumeGIDValue "" Jul 03 11:47:45.425970 ip-10-0-136-0 kubenswrapper[2579]: I0703 11:47:45.425934 2579 reconciler_common.go:299] "Volume detached for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/623e6c46-3354-47da-8157-83d80d524b3d-istio-token\") on node \"ip-10-0-136-0.ec2.internal\" DevicePath \"\"" Jul 03 11:47:45.425970 ip-10-0-136-0 kubenswrapper[2579]: I0703 11:47:45.425966 2579 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-6rjj7\" (UniqueName: \"kubernetes.io/projected/623e6c46-3354-47da-8157-83d80d524b3d-kube-api-access-6rjj7\") on node \"ip-10-0-136-0.ec2.internal\" DevicePath \"\"" Jul 03 11:47:45.425970 ip-10-0-136-0 kubenswrapper[2579]: I0703 11:47:45.425978 2579 reconciler_common.go:299] "Volume detached for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/623e6c46-3354-47da-8157-83d80d524b3d-istiod-ca-cert\") on node \"ip-10-0-136-0.ec2.internal\" DevicePath \"\"" Jul 03 11:47:45.426281 ip-10-0-136-0 kubenswrapper[2579]: I0703 11:47:45.425986 2579 reconciler_common.go:299] "Volume detached for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/623e6c46-3354-47da-8157-83d80d524b3d-istio-envoy\") on node \"ip-10-0-136-0.ec2.internal\" DevicePath \"\"" Jul 03 11:47:45.426281 ip-10-0-136-0 kubenswrapper[2579]: I0703 11:47:45.425995 2579 reconciler_common.go:299] "Volume detached for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/623e6c46-3354-47da-8157-83d80d524b3d-workload-certs\") on node \"ip-10-0-136-0.ec2.internal\" DevicePath \"\"" Jul 03 11:47:45.426281 ip-10-0-136-0 kubenswrapper[2579]: I0703 11:47:45.426002 2579 reconciler_common.go:299] "Volume detached for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/623e6c46-3354-47da-8157-83d80d524b3d-istio-data\") on node \"ip-10-0-136-0.ec2.internal\" DevicePath \"\"" Jul 03 11:47:45.426281 ip-10-0-136-0 kubenswrapper[2579]: I0703 11:47:45.426010 2579 reconciler_common.go:299] "Volume detached for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/623e6c46-3354-47da-8157-83d80d524b3d-istio-podinfo\") on node \"ip-10-0-136-0.ec2.internal\" DevicePath \"\"" Jul 03 11:47:45.857547 ip-10-0-136-0 kubenswrapper[2579]: I0703 11:47:45.857514 2579 generic.go:356] "Generic (PLEG): container finished" podID="623e6c46-3354-47da-8157-83d80d524b3d" containerID="00b2d88be3ba88496c4f88bb2fb3043304e201e572edc51ec56e0a7fb1def652" exitCode=0 Jul 03 11:47:45.857547 ip-10-0-136-0 kubenswrapper[2579]: I0703 11:47:45.857554 2579 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-inf-b-44ab65-openshift-default-5946888467-59l9p" event={"ID":"623e6c46-3354-47da-8157-83d80d524b3d","Type":"ContainerDied","Data":"00b2d88be3ba88496c4f88bb2fb3043304e201e572edc51ec56e0a7fb1def652"} Jul 03 11:47:45.857794 ip-10-0-136-0 kubenswrapper[2579]: I0703 11:47:45.857577 2579 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-inf-b-44ab65-openshift-default-5946888467-59l9p" Jul 03 11:47:45.857794 ip-10-0-136-0 kubenswrapper[2579]: I0703 11:47:45.857594 2579 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-inf-b-44ab65-openshift-default-5946888467-59l9p" event={"ID":"623e6c46-3354-47da-8157-83d80d524b3d","Type":"ContainerDied","Data":"5479cea7c11bd4fad7b83f5d08e0ec98a70760efa6447564f4a65f681bcc7cf5"} Jul 03 11:47:45.857794 ip-10-0-136-0 kubenswrapper[2579]: I0703 11:47:45.857614 2579 scope.go:117] "RemoveContainer" containerID="00b2d88be3ba88496c4f88bb2fb3043304e201e572edc51ec56e0a7fb1def652" Jul 03 11:47:45.868720 ip-10-0-136-0 kubenswrapper[2579]: I0703 11:47:45.868693 2579 scope.go:117] "RemoveContainer" containerID="c1b12e28d61306bb0a67db516619ec4164bc2ee14f2c7b469d1c7e9b74be1692" Jul 03 11:47:45.879140 ip-10-0-136-0 kubenswrapper[2579]: I0703 11:47:45.879118 2579 scope.go:117] "RemoveContainer" containerID="00b2d88be3ba88496c4f88bb2fb3043304e201e572edc51ec56e0a7fb1def652" Jul 03 11:47:45.879524 ip-10-0-136-0 kubenswrapper[2579]: E0703 11:47:45.879499 2579 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"00b2d88be3ba88496c4f88bb2fb3043304e201e572edc51ec56e0a7fb1def652\": container with ID starting with 00b2d88be3ba88496c4f88bb2fb3043304e201e572edc51ec56e0a7fb1def652 not found: ID does not exist" containerID="00b2d88be3ba88496c4f88bb2fb3043304e201e572edc51ec56e0a7fb1def652" Jul 03 11:47:45.879623 ip-10-0-136-0 kubenswrapper[2579]: I0703 11:47:45.879539 2579 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"00b2d88be3ba88496c4f88bb2fb3043304e201e572edc51ec56e0a7fb1def652"} err="failed to get container status \"00b2d88be3ba88496c4f88bb2fb3043304e201e572edc51ec56e0a7fb1def652\": rpc error: code = NotFound desc = could not find container \"00b2d88be3ba88496c4f88bb2fb3043304e201e572edc51ec56e0a7fb1def652\": container with ID starting with 00b2d88be3ba88496c4f88bb2fb3043304e201e572edc51ec56e0a7fb1def652 not found: ID does not exist" Jul 03 11:47:45.879623 ip-10-0-136-0 kubenswrapper[2579]: I0703 11:47:45.879584 2579 scope.go:117] "RemoveContainer" containerID="c1b12e28d61306bb0a67db516619ec4164bc2ee14f2c7b469d1c7e9b74be1692" Jul 03 11:47:45.879914 ip-10-0-136-0 kubenswrapper[2579]: E0703 11:47:45.879881 2579 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"c1b12e28d61306bb0a67db516619ec4164bc2ee14f2c7b469d1c7e9b74be1692\": container with ID starting with c1b12e28d61306bb0a67db516619ec4164bc2ee14f2c7b469d1c7e9b74be1692 not found: ID does not exist" containerID="c1b12e28d61306bb0a67db516619ec4164bc2ee14f2c7b469d1c7e9b74be1692" Jul 03 11:47:45.879955 ip-10-0-136-0 kubenswrapper[2579]: I0703 11:47:45.879922 2579 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"c1b12e28d61306bb0a67db516619ec4164bc2ee14f2c7b469d1c7e9b74be1692"} err="failed to get container status \"c1b12e28d61306bb0a67db516619ec4164bc2ee14f2c7b469d1c7e9b74be1692\": rpc error: code = NotFound desc = could not find container \"c1b12e28d61306bb0a67db516619ec4164bc2ee14f2c7b469d1c7e9b74be1692\": container with ID starting with c1b12e28d61306bb0a67db516619ec4164bc2ee14f2c7b469d1c7e9b74be1692 not found: ID does not exist" Jul 03 11:47:45.889121 ip-10-0-136-0 kubenswrapper[2579]: I0703 11:47:45.889084 2579 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-ingress/e2e-inf-b-44ab65-openshift-default-5946888467-59l9p"] Jul 03 11:47:45.892982 ip-10-0-136-0 kubenswrapper[2579]: I0703 11:47:45.892947 2579 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["openshift-ingress/e2e-inf-b-44ab65-openshift-default-5946888467-59l9p"] Jul 03 11:47:45.992744 ip-10-0-136-0 kubenswrapper[2579]: I0703 11:47:45.992690 2579 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="623e6c46-3354-47da-8157-83d80d524b3d" path="/var/lib/kubelet/pods/623e6c46-3354-47da-8157-83d80d524b3d/volumes" Jul 03 11:47:49.267548 ip-10-0-136-0 kubenswrapper[2579]: I0703 11:47:49.267507 2579 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-ingress/e2e-api-b-8730be-openshift-default-58f6b5485f-vf9cs"] Jul 03 11:47:49.268005 ip-10-0-136-0 kubenswrapper[2579]: I0703 11:47:49.267990 2579 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="623e6c46-3354-47da-8157-83d80d524b3d" containerName="istio-proxy" Jul 03 11:47:49.268050 ip-10-0-136-0 kubenswrapper[2579]: I0703 11:47:49.268007 2579 state_mem.go:107] "Deleted CPUSet assignment" podUID="623e6c46-3354-47da-8157-83d80d524b3d" containerName="istio-proxy" Jul 03 11:47:49.268050 ip-10-0-136-0 kubenswrapper[2579]: I0703 11:47:49.268016 2579 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="623e6c46-3354-47da-8157-83d80d524b3d" containerName="istio-proxy" Jul 03 11:47:49.268050 ip-10-0-136-0 kubenswrapper[2579]: I0703 11:47:49.268022 2579 state_mem.go:107] "Deleted CPUSet assignment" podUID="623e6c46-3354-47da-8157-83d80d524b3d" containerName="istio-proxy" Jul 03 11:47:49.268050 ip-10-0-136-0 kubenswrapper[2579]: I0703 11:47:49.268029 2579 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="f1b7686d-4c68-4b24-8ae1-a8cde82640cd" containerName="istio-proxy" Jul 03 11:47:49.268050 ip-10-0-136-0 kubenswrapper[2579]: I0703 11:47:49.268034 2579 state_mem.go:107] "Deleted CPUSet assignment" podUID="f1b7686d-4c68-4b24-8ae1-a8cde82640cd" containerName="istio-proxy" Jul 03 11:47:49.268233 ip-10-0-136-0 kubenswrapper[2579]: I0703 11:47:49.268090 2579 memory_manager.go:368] "RemoveStaleState removing state" podUID="623e6c46-3354-47da-8157-83d80d524b3d" containerName="istio-proxy" Jul 03 11:47:49.268233 ip-10-0-136-0 kubenswrapper[2579]: I0703 11:47:49.268100 2579 memory_manager.go:368] "RemoveStaleState removing state" podUID="623e6c46-3354-47da-8157-83d80d524b3d" containerName="istio-proxy" Jul 03 11:47:49.272801 ip-10-0-136-0 kubenswrapper[2579]: I0703 11:47:49.272770 2579 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-api-b-8730be-openshift-default-58f6b5485f-vf9cs" Jul 03 11:47:49.277245 ip-10-0-136-0 kubenswrapper[2579]: I0703 11:47:49.277175 2579 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress\"/\"e2e-api-b-8730be-openshift-default-dockercfg-pskhx\"" Jul 03 11:47:49.296572 ip-10-0-136-0 kubenswrapper[2579]: I0703 11:47:49.296522 2579 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/e2e-api-b-8730be-openshift-default-58f6b5485f-vf9cs"] Jul 03 11:47:49.368433 ip-10-0-136-0 kubenswrapper[2579]: I0703 11:47:49.368367 2579 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/df8a6db7-c558-410f-b596-51f30826429a-istio-data\") pod \"e2e-api-b-8730be-openshift-default-58f6b5485f-vf9cs\" (UID: \"df8a6db7-c558-410f-b596-51f30826429a\") " pod="openshift-ingress/e2e-api-b-8730be-openshift-default-58f6b5485f-vf9cs" Jul 03 11:47:49.368648 ip-10-0-136-0 kubenswrapper[2579]: I0703 11:47:49.368444 2579 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/df8a6db7-c558-410f-b596-51f30826429a-workload-socket\") pod \"e2e-api-b-8730be-openshift-default-58f6b5485f-vf9cs\" (UID: \"df8a6db7-c558-410f-b596-51f30826429a\") " pod="openshift-ingress/e2e-api-b-8730be-openshift-default-58f6b5485f-vf9cs" Jul 03 11:47:49.368648 ip-10-0-136-0 kubenswrapper[2579]: I0703 11:47:49.368531 2579 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/df8a6db7-c558-410f-b596-51f30826429a-istio-podinfo\") pod \"e2e-api-b-8730be-openshift-default-58f6b5485f-vf9cs\" (UID: \"df8a6db7-c558-410f-b596-51f30826429a\") " pod="openshift-ingress/e2e-api-b-8730be-openshift-default-58f6b5485f-vf9cs" Jul 03 11:47:49.368648 ip-10-0-136-0 kubenswrapper[2579]: I0703 11:47:49.368586 2579 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-zwj8c\" (UniqueName: \"kubernetes.io/projected/df8a6db7-c558-410f-b596-51f30826429a-kube-api-access-zwj8c\") pod \"e2e-api-b-8730be-openshift-default-58f6b5485f-vf9cs\" (UID: \"df8a6db7-c558-410f-b596-51f30826429a\") " pod="openshift-ingress/e2e-api-b-8730be-openshift-default-58f6b5485f-vf9cs" Jul 03 11:47:49.368648 ip-10-0-136-0 kubenswrapper[2579]: I0703 11:47:49.368640 2579 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/df8a6db7-c558-410f-b596-51f30826429a-istio-envoy\") pod \"e2e-api-b-8730be-openshift-default-58f6b5485f-vf9cs\" (UID: \"df8a6db7-c558-410f-b596-51f30826429a\") " pod="openshift-ingress/e2e-api-b-8730be-openshift-default-58f6b5485f-vf9cs" Jul 03 11:47:49.368863 ip-10-0-136-0 kubenswrapper[2579]: I0703 11:47:49.368702 2579 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/df8a6db7-c558-410f-b596-51f30826429a-workload-certs\") pod \"e2e-api-b-8730be-openshift-default-58f6b5485f-vf9cs\" (UID: \"df8a6db7-c558-410f-b596-51f30826429a\") " pod="openshift-ingress/e2e-api-b-8730be-openshift-default-58f6b5485f-vf9cs" Jul 03 11:47:49.368863 ip-10-0-136-0 kubenswrapper[2579]: I0703 11:47:49.368737 2579 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/df8a6db7-c558-410f-b596-51f30826429a-istiod-ca-cert\") pod \"e2e-api-b-8730be-openshift-default-58f6b5485f-vf9cs\" (UID: \"df8a6db7-c558-410f-b596-51f30826429a\") " pod="openshift-ingress/e2e-api-b-8730be-openshift-default-58f6b5485f-vf9cs" Jul 03 11:47:49.368863 ip-10-0-136-0 kubenswrapper[2579]: I0703 11:47:49.368779 2579 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/df8a6db7-c558-410f-b596-51f30826429a-istio-token\") pod \"e2e-api-b-8730be-openshift-default-58f6b5485f-vf9cs\" (UID: \"df8a6db7-c558-410f-b596-51f30826429a\") " pod="openshift-ingress/e2e-api-b-8730be-openshift-default-58f6b5485f-vf9cs" Jul 03 11:47:49.368863 ip-10-0-136-0 kubenswrapper[2579]: I0703 11:47:49.368837 2579 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/df8a6db7-c558-410f-b596-51f30826429a-credential-socket\") pod \"e2e-api-b-8730be-openshift-default-58f6b5485f-vf9cs\" (UID: \"df8a6db7-c558-410f-b596-51f30826429a\") " pod="openshift-ingress/e2e-api-b-8730be-openshift-default-58f6b5485f-vf9cs" Jul 03 11:47:49.470445 ip-10-0-136-0 kubenswrapper[2579]: I0703 11:47:49.470388 2579 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/df8a6db7-c558-410f-b596-51f30826429a-istio-token\") pod \"e2e-api-b-8730be-openshift-default-58f6b5485f-vf9cs\" (UID: \"df8a6db7-c558-410f-b596-51f30826429a\") " pod="openshift-ingress/e2e-api-b-8730be-openshift-default-58f6b5485f-vf9cs" Jul 03 11:47:49.470445 ip-10-0-136-0 kubenswrapper[2579]: I0703 11:47:49.470451 2579 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/df8a6db7-c558-410f-b596-51f30826429a-credential-socket\") pod \"e2e-api-b-8730be-openshift-default-58f6b5485f-vf9cs\" (UID: \"df8a6db7-c558-410f-b596-51f30826429a\") " pod="openshift-ingress/e2e-api-b-8730be-openshift-default-58f6b5485f-vf9cs" Jul 03 11:47:49.470795 ip-10-0-136-0 kubenswrapper[2579]: I0703 11:47:49.470499 2579 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/df8a6db7-c558-410f-b596-51f30826429a-istio-data\") pod \"e2e-api-b-8730be-openshift-default-58f6b5485f-vf9cs\" (UID: \"df8a6db7-c558-410f-b596-51f30826429a\") " pod="openshift-ingress/e2e-api-b-8730be-openshift-default-58f6b5485f-vf9cs" Jul 03 11:47:49.470795 ip-10-0-136-0 kubenswrapper[2579]: I0703 11:47:49.470532 2579 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/df8a6db7-c558-410f-b596-51f30826429a-workload-socket\") pod \"e2e-api-b-8730be-openshift-default-58f6b5485f-vf9cs\" (UID: \"df8a6db7-c558-410f-b596-51f30826429a\") " pod="openshift-ingress/e2e-api-b-8730be-openshift-default-58f6b5485f-vf9cs" Jul 03 11:47:49.470795 ip-10-0-136-0 kubenswrapper[2579]: I0703 11:47:49.470613 2579 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/df8a6db7-c558-410f-b596-51f30826429a-istio-podinfo\") pod \"e2e-api-b-8730be-openshift-default-58f6b5485f-vf9cs\" (UID: \"df8a6db7-c558-410f-b596-51f30826429a\") " pod="openshift-ingress/e2e-api-b-8730be-openshift-default-58f6b5485f-vf9cs" Jul 03 11:47:49.470795 ip-10-0-136-0 kubenswrapper[2579]: I0703 11:47:49.470653 2579 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-zwj8c\" (UniqueName: \"kubernetes.io/projected/df8a6db7-c558-410f-b596-51f30826429a-kube-api-access-zwj8c\") pod \"e2e-api-b-8730be-openshift-default-58f6b5485f-vf9cs\" (UID: \"df8a6db7-c558-410f-b596-51f30826429a\") " pod="openshift-ingress/e2e-api-b-8730be-openshift-default-58f6b5485f-vf9cs" Jul 03 11:47:49.470795 ip-10-0-136-0 kubenswrapper[2579]: I0703 11:47:49.470707 2579 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/df8a6db7-c558-410f-b596-51f30826429a-istio-envoy\") pod \"e2e-api-b-8730be-openshift-default-58f6b5485f-vf9cs\" (UID: \"df8a6db7-c558-410f-b596-51f30826429a\") " pod="openshift-ingress/e2e-api-b-8730be-openshift-default-58f6b5485f-vf9cs" Jul 03 11:47:49.470795 ip-10-0-136-0 kubenswrapper[2579]: I0703 11:47:49.470767 2579 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/df8a6db7-c558-410f-b596-51f30826429a-workload-certs\") pod \"e2e-api-b-8730be-openshift-default-58f6b5485f-vf9cs\" (UID: \"df8a6db7-c558-410f-b596-51f30826429a\") " pod="openshift-ingress/e2e-api-b-8730be-openshift-default-58f6b5485f-vf9cs" Jul 03 11:47:49.471119 ip-10-0-136-0 kubenswrapper[2579]: I0703 11:47:49.470796 2579 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/df8a6db7-c558-410f-b596-51f30826429a-istiod-ca-cert\") pod \"e2e-api-b-8730be-openshift-default-58f6b5485f-vf9cs\" (UID: \"df8a6db7-c558-410f-b596-51f30826429a\") " pod="openshift-ingress/e2e-api-b-8730be-openshift-default-58f6b5485f-vf9cs" Jul 03 11:47:49.471119 ip-10-0-136-0 kubenswrapper[2579]: I0703 11:47:49.470985 2579 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/df8a6db7-c558-410f-b596-51f30826429a-credential-socket\") pod \"e2e-api-b-8730be-openshift-default-58f6b5485f-vf9cs\" (UID: \"df8a6db7-c558-410f-b596-51f30826429a\") " pod="openshift-ingress/e2e-api-b-8730be-openshift-default-58f6b5485f-vf9cs" Jul 03 11:47:49.471119 ip-10-0-136-0 kubenswrapper[2579]: I0703 11:47:49.471017 2579 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/df8a6db7-c558-410f-b596-51f30826429a-workload-socket\") pod \"e2e-api-b-8730be-openshift-default-58f6b5485f-vf9cs\" (UID: \"df8a6db7-c558-410f-b596-51f30826429a\") " pod="openshift-ingress/e2e-api-b-8730be-openshift-default-58f6b5485f-vf9cs" Jul 03 11:47:49.471119 ip-10-0-136-0 kubenswrapper[2579]: I0703 11:47:49.471060 2579 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/df8a6db7-c558-410f-b596-51f30826429a-istio-data\") pod \"e2e-api-b-8730be-openshift-default-58f6b5485f-vf9cs\" (UID: \"df8a6db7-c558-410f-b596-51f30826429a\") " pod="openshift-ingress/e2e-api-b-8730be-openshift-default-58f6b5485f-vf9cs" Jul 03 11:47:49.471353 ip-10-0-136-0 kubenswrapper[2579]: I0703 11:47:49.471177 2579 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/df8a6db7-c558-410f-b596-51f30826429a-workload-certs\") pod \"e2e-api-b-8730be-openshift-default-58f6b5485f-vf9cs\" (UID: \"df8a6db7-c558-410f-b596-51f30826429a\") " pod="openshift-ingress/e2e-api-b-8730be-openshift-default-58f6b5485f-vf9cs" Jul 03 11:47:49.471756 ip-10-0-136-0 kubenswrapper[2579]: I0703 11:47:49.471729 2579 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/df8a6db7-c558-410f-b596-51f30826429a-istiod-ca-cert\") pod \"e2e-api-b-8730be-openshift-default-58f6b5485f-vf9cs\" (UID: \"df8a6db7-c558-410f-b596-51f30826429a\") " pod="openshift-ingress/e2e-api-b-8730be-openshift-default-58f6b5485f-vf9cs" Jul 03 11:47:49.473496 ip-10-0-136-0 kubenswrapper[2579]: I0703 11:47:49.473455 2579 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/df8a6db7-c558-410f-b596-51f30826429a-istio-envoy\") pod \"e2e-api-b-8730be-openshift-default-58f6b5485f-vf9cs\" (UID: \"df8a6db7-c558-410f-b596-51f30826429a\") " pod="openshift-ingress/e2e-api-b-8730be-openshift-default-58f6b5485f-vf9cs" Jul 03 11:47:49.473747 ip-10-0-136-0 kubenswrapper[2579]: I0703 11:47:49.473722 2579 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/df8a6db7-c558-410f-b596-51f30826429a-istio-podinfo\") pod \"e2e-api-b-8730be-openshift-default-58f6b5485f-vf9cs\" (UID: \"df8a6db7-c558-410f-b596-51f30826429a\") " pod="openshift-ingress/e2e-api-b-8730be-openshift-default-58f6b5485f-vf9cs" Jul 03 11:47:49.482890 ip-10-0-136-0 kubenswrapper[2579]: I0703 11:47:49.482855 2579 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/df8a6db7-c558-410f-b596-51f30826429a-istio-token\") pod \"e2e-api-b-8730be-openshift-default-58f6b5485f-vf9cs\" (UID: \"df8a6db7-c558-410f-b596-51f30826429a\") " pod="openshift-ingress/e2e-api-b-8730be-openshift-default-58f6b5485f-vf9cs" Jul 03 11:47:49.483376 ip-10-0-136-0 kubenswrapper[2579]: I0703 11:47:49.483349 2579 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-zwj8c\" (UniqueName: \"kubernetes.io/projected/df8a6db7-c558-410f-b596-51f30826429a-kube-api-access-zwj8c\") pod \"e2e-api-b-8730be-openshift-default-58f6b5485f-vf9cs\" (UID: \"df8a6db7-c558-410f-b596-51f30826429a\") " pod="openshift-ingress/e2e-api-b-8730be-openshift-default-58f6b5485f-vf9cs" Jul 03 11:47:49.585872 ip-10-0-136-0 kubenswrapper[2579]: I0703 11:47:49.585775 2579 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-api-b-8730be-openshift-default-58f6b5485f-vf9cs" Jul 03 11:47:49.770235 ip-10-0-136-0 kubenswrapper[2579]: I0703 11:47:49.770180 2579 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/e2e-api-b-8730be-openshift-default-58f6b5485f-vf9cs"] Jul 03 11:47:49.771042 ip-10-0-136-0 kubenswrapper[2579]: W0703 11:47:49.771007 2579 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-poddf8a6db7_c558_410f_b596_51f30826429a.slice/crio-cc81b8d9c3a39a3458b9fe8ba348308e96148d9bf6b361bad0f1e8d22116e396 WatchSource:0}: Error finding container cc81b8d9c3a39a3458b9fe8ba348308e96148d9bf6b361bad0f1e8d22116e396: Status 404 returned error can't find the container with id cc81b8d9c3a39a3458b9fe8ba348308e96148d9bf6b361bad0f1e8d22116e396 Jul 03 11:47:49.773547 ip-10-0-136-0 kubenswrapper[2579]: I0703 11:47:49.773502 2579 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"30892156Ki","pods":"250"} Jul 03 11:47:49.773680 ip-10-0-136-0 kubenswrapper[2579]: I0703 11:47:49.773583 2579 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"30892156Ki","pods":"250"} Jul 03 11:47:49.773680 ip-10-0-136-0 kubenswrapper[2579]: I0703 11:47:49.773621 2579 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"30892156Ki","pods":"250"} Jul 03 11:47:49.888672 ip-10-0-136-0 kubenswrapper[2579]: I0703 11:47:49.888629 2579 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-api-b-8730be-openshift-default-58f6b5485f-vf9cs" event={"ID":"df8a6db7-c558-410f-b596-51f30826429a","Type":"ContainerStarted","Data":"c8e7603e199155a730a3f67e6056972cc97fbe618453dd01a4f42c207b565792"} Jul 03 11:47:49.889180 ip-10-0-136-0 kubenswrapper[2579]: I0703 11:47:49.888699 2579 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-api-b-8730be-openshift-default-58f6b5485f-vf9cs" event={"ID":"df8a6db7-c558-410f-b596-51f30826429a","Type":"ContainerStarted","Data":"cc81b8d9c3a39a3458b9fe8ba348308e96148d9bf6b361bad0f1e8d22116e396"} Jul 03 11:47:49.922534 ip-10-0-136-0 kubenswrapper[2579]: I0703 11:47:49.922469 2579 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-ingress/e2e-api-b-8730be-openshift-default-58f6b5485f-vf9cs" podStartSLOduration=0.922449982 podStartE2EDuration="922.449982ms" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-03 11:47:49 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-03 11:47:49.920162622 +0000 UTC m=+2126.508466941" watchObservedRunningTime="2026-07-03 11:47:49.922449982 +0000 UTC m=+2126.510754299" Jul 03 11:47:50.586168 ip-10-0-136-0 kubenswrapper[2579]: I0703 11:47:50.586115 2579 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="openshift-ingress/e2e-api-b-8730be-openshift-default-58f6b5485f-vf9cs" Jul 03 11:47:50.591831 ip-10-0-136-0 kubenswrapper[2579]: I0703 11:47:50.591802 2579 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="openshift-ingress/e2e-api-b-8730be-openshift-default-58f6b5485f-vf9cs" Jul 03 11:47:50.895079 ip-10-0-136-0 kubenswrapper[2579]: I0703 11:47:50.894987 2579 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/e2e-api-b-8730be-openshift-default-58f6b5485f-vf9cs" Jul 03 11:47:50.896302 ip-10-0-136-0 kubenswrapper[2579]: I0703 11:47:50.896272 2579 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/e2e-api-b-8730be-openshift-default-58f6b5485f-vf9cs" Jul 03 11:47:58.876449 ip-10-0-136-0 kubenswrapper[2579]: I0703 11:47:58.876400 2579 patch_prober.go:28] interesting pod/e2e-api-a-515460-openshift-default-897bbd7c-xhjtn container/istio-proxy namespace/openshift-ingress: Readiness probe status=failure output="Get \"http://10.133.0.71:15021/healthz/ready\": context deadline exceeded (Client.Timeout exceeded while awaiting headers)" start-of-body= Jul 03 11:47:58.876843 ip-10-0-136-0 kubenswrapper[2579]: I0703 11:47:58.876509 2579 prober.go:120] "Probe failed" probeType="Readiness" pod="openshift-ingress/e2e-api-a-515460-openshift-default-897bbd7c-xhjtn" podUID="75a2fbf5-528e-4910-b92b-e575c5ccf0f0" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.133.0.71:15021/healthz/ready\": context deadline exceeded (Client.Timeout exceeded while awaiting headers)" Jul 03 11:48:00.968085 ip-10-0-136-0 kubenswrapper[2579]: I0703 11:48:00.968045 2579 generic.go:356] "Generic (PLEG): container finished" podID="75a2fbf5-528e-4910-b92b-e575c5ccf0f0" containerID="19cad7f406870be1a95f8e2be936db1ea845975d6c74789ee75b6f610ab44c44" exitCode=0 Jul 03 11:48:00.968582 ip-10-0-136-0 kubenswrapper[2579]: I0703 11:48:00.968124 2579 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-api-a-515460-openshift-default-897bbd7c-xhjtn" event={"ID":"75a2fbf5-528e-4910-b92b-e575c5ccf0f0","Type":"ContainerDied","Data":"19cad7f406870be1a95f8e2be936db1ea845975d6c74789ee75b6f610ab44c44"} Jul 03 11:48:00.968743 ip-10-0-136-0 kubenswrapper[2579]: I0703 11:48:00.968724 2579 scope.go:117] "RemoveContainer" containerID="19cad7f406870be1a95f8e2be936db1ea845975d6c74789ee75b6f610ab44c44" Jul 03 11:48:00.969963 ip-10-0-136-0 kubenswrapper[2579]: I0703 11:48:00.969930 2579 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"30892156Ki","pods":"250"} Jul 03 11:48:00.970047 ip-10-0-136-0 kubenswrapper[2579]: I0703 11:48:00.970029 2579 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"30892156Ki","pods":"250"} Jul 03 11:48:00.970090 ip-10-0-136-0 kubenswrapper[2579]: I0703 11:48:00.970081 2579 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"30892156Ki","pods":"250"} Jul 03 11:48:01.995596 ip-10-0-136-0 kubenswrapper[2579]: I0703 11:48:01.995551 2579 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-api-a-515460-openshift-default-897bbd7c-xhjtn" event={"ID":"75a2fbf5-528e-4910-b92b-e575c5ccf0f0","Type":"ContainerStarted","Data":"fae9c063c680239c53dbcd45c9329a5d8541d1ef6ff0ed0557a40ec5b732e45a"} Jul 03 11:48:02.281296 ip-10-0-136-0 kubenswrapper[2579]: I0703 11:48:02.281243 2579 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="openshift-ingress/e2e-api-a-515460-openshift-default-897bbd7c-xhjtn" Jul 03 11:48:02.281765 ip-10-0-136-0 kubenswrapper[2579]: I0703 11:48:02.281728 2579 patch_prober.go:28] interesting pod/e2e-api-a-515460-openshift-default-897bbd7c-xhjtn container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.133.0.71:15021/healthz/ready\": dial tcp 10.133.0.71:15021: connect: connection refused" start-of-body= Jul 03 11:48:02.281891 ip-10-0-136-0 kubenswrapper[2579]: I0703 11:48:02.281812 2579 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-api-a-515460-openshift-default-897bbd7c-xhjtn" podUID="75a2fbf5-528e-4910-b92b-e575c5ccf0f0" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.133.0.71:15021/healthz/ready\": dial tcp 10.133.0.71:15021: connect: connection refused" Jul 03 11:48:03.281251 ip-10-0-136-0 kubenswrapper[2579]: I0703 11:48:03.281171 2579 patch_prober.go:28] interesting pod/e2e-api-a-515460-openshift-default-897bbd7c-xhjtn container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.133.0.71:15021/healthz/ready\": dial tcp 10.133.0.71:15021: connect: connection refused" start-of-body= Jul 03 11:48:03.281880 ip-10-0-136-0 kubenswrapper[2579]: I0703 11:48:03.281281 2579 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-api-a-515460-openshift-default-897bbd7c-xhjtn" podUID="75a2fbf5-528e-4910-b92b-e575c5ccf0f0" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.133.0.71:15021/healthz/ready\": dial tcp 10.133.0.71:15021: connect: connection refused" Jul 03 11:48:04.280903 ip-10-0-136-0 kubenswrapper[2579]: I0703 11:48:04.280848 2579 patch_prober.go:28] interesting pod/e2e-api-a-515460-openshift-default-897bbd7c-xhjtn container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.133.0.71:15021/healthz/ready\": dial tcp 10.133.0.71:15021: connect: connection refused" start-of-body= Jul 03 11:48:04.281144 ip-10-0-136-0 kubenswrapper[2579]: I0703 11:48:04.280925 2579 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-api-a-515460-openshift-default-897bbd7c-xhjtn" podUID="75a2fbf5-528e-4910-b92b-e575c5ccf0f0" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.133.0.71:15021/healthz/ready\": dial tcp 10.133.0.71:15021: connect: connection refused" Jul 03 11:48:05.280946 ip-10-0-136-0 kubenswrapper[2579]: I0703 11:48:05.280901 2579 patch_prober.go:28] interesting pod/e2e-api-a-515460-openshift-default-897bbd7c-xhjtn container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.133.0.71:15021/healthz/ready\": dial tcp 10.133.0.71:15021: connect: connection refused" start-of-body= Jul 03 11:48:05.281385 ip-10-0-136-0 kubenswrapper[2579]: I0703 11:48:05.280978 2579 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-api-a-515460-openshift-default-897bbd7c-xhjtn" podUID="75a2fbf5-528e-4910-b92b-e575c5ccf0f0" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.133.0.71:15021/healthz/ready\": dial tcp 10.133.0.71:15021: connect: connection refused" Jul 03 11:48:06.285879 ip-10-0-136-0 kubenswrapper[2579]: I0703 11:48:06.285841 2579 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="openshift-ingress/e2e-api-a-515460-openshift-default-897bbd7c-xhjtn" Jul 03 11:48:06.286392 ip-10-0-136-0 kubenswrapper[2579]: I0703 11:48:06.286126 2579 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/e2e-api-a-515460-openshift-default-897bbd7c-xhjtn" Jul 03 11:48:06.287338 ip-10-0-136-0 kubenswrapper[2579]: I0703 11:48:06.287315 2579 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/e2e-api-a-515460-openshift-default-897bbd7c-xhjtn" Jul 03 11:48:06.938429 ip-10-0-136-0 kubenswrapper[2579]: I0703 11:48:06.938376 2579 patch_prober.go:28] interesting pod/e2e-api-b-8730be-openshift-default-58f6b5485f-vf9cs container/istio-proxy namespace/openshift-ingress: Readiness probe status=failure output="Get \"http://10.133.0.72:15021/healthz/ready\": context deadline exceeded (Client.Timeout exceeded while awaiting headers)" start-of-body= Jul 03 11:48:06.938646 ip-10-0-136-0 kubenswrapper[2579]: I0703 11:48:06.938459 2579 prober.go:120] "Probe failed" probeType="Readiness" pod="openshift-ingress/e2e-api-b-8730be-openshift-default-58f6b5485f-vf9cs" podUID="df8a6db7-c558-410f-b596-51f30826429a" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.133.0.72:15021/healthz/ready\": context deadline exceeded (Client.Timeout exceeded while awaiting headers)" Jul 03 11:48:09.034042 ip-10-0-136-0 kubenswrapper[2579]: I0703 11:48:09.034007 2579 generic.go:356] "Generic (PLEG): container finished" podID="df8a6db7-c558-410f-b596-51f30826429a" containerID="c8e7603e199155a730a3f67e6056972cc97fbe618453dd01a4f42c207b565792" exitCode=0 Jul 03 11:48:09.034497 ip-10-0-136-0 kubenswrapper[2579]: I0703 11:48:09.034080 2579 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-api-b-8730be-openshift-default-58f6b5485f-vf9cs" event={"ID":"df8a6db7-c558-410f-b596-51f30826429a","Type":"ContainerDied","Data":"c8e7603e199155a730a3f67e6056972cc97fbe618453dd01a4f42c207b565792"} Jul 03 11:48:09.034632 ip-10-0-136-0 kubenswrapper[2579]: I0703 11:48:09.034614 2579 scope.go:117] "RemoveContainer" containerID="c8e7603e199155a730a3f67e6056972cc97fbe618453dd01a4f42c207b565792" Jul 03 11:48:09.035839 ip-10-0-136-0 kubenswrapper[2579]: I0703 11:48:09.035808 2579 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"30892156Ki","pods":"250"} Jul 03 11:48:09.035913 ip-10-0-136-0 kubenswrapper[2579]: I0703 11:48:09.035894 2579 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"30892156Ki","pods":"250"} Jul 03 11:48:09.035950 ip-10-0-136-0 kubenswrapper[2579]: I0703 11:48:09.035923 2579 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"30892156Ki","pods":"250"} Jul 03 11:48:10.048880 ip-10-0-136-0 kubenswrapper[2579]: I0703 11:48:10.048827 2579 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-api-b-8730be-openshift-default-58f6b5485f-vf9cs" event={"ID":"df8a6db7-c558-410f-b596-51f30826429a","Type":"ContainerStarted","Data":"d2d027d214cff4180aa14e3c6dbb9ff75336bca8d1ac65f91c7ee938e5ac3a6b"} Jul 03 11:48:10.586591 ip-10-0-136-0 kubenswrapper[2579]: I0703 11:48:10.586540 2579 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="openshift-ingress/e2e-api-b-8730be-openshift-default-58f6b5485f-vf9cs" Jul 03 11:48:10.586898 ip-10-0-136-0 kubenswrapper[2579]: I0703 11:48:10.586868 2579 patch_prober.go:28] interesting pod/e2e-api-b-8730be-openshift-default-58f6b5485f-vf9cs container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.133.0.72:15021/healthz/ready\": dial tcp 10.133.0.72:15021: connect: connection refused" start-of-body= Jul 03 11:48:10.586999 ip-10-0-136-0 kubenswrapper[2579]: I0703 11:48:10.586940 2579 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-api-b-8730be-openshift-default-58f6b5485f-vf9cs" podUID="df8a6db7-c558-410f-b596-51f30826429a" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.133.0.72:15021/healthz/ready\": dial tcp 10.133.0.72:15021: connect: connection refused" Jul 03 11:48:11.587060 ip-10-0-136-0 kubenswrapper[2579]: I0703 11:48:11.587021 2579 patch_prober.go:28] interesting pod/e2e-api-b-8730be-openshift-default-58f6b5485f-vf9cs container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.133.0.72:15021/healthz/ready\": dial tcp 10.133.0.72:15021: connect: connection refused" start-of-body= Jul 03 11:48:11.587630 ip-10-0-136-0 kubenswrapper[2579]: I0703 11:48:11.587081 2579 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-api-b-8730be-openshift-default-58f6b5485f-vf9cs" podUID="df8a6db7-c558-410f-b596-51f30826429a" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.133.0.72:15021/healthz/ready\": dial tcp 10.133.0.72:15021: connect: connection refused" Jul 03 11:48:11.597083 ip-10-0-136-0 kubenswrapper[2579]: I0703 11:48:11.597039 2579 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-ingress/e2e-api-a-515460-openshift-default-897bbd7c-xhjtn"] Jul 03 11:48:11.597322 ip-10-0-136-0 kubenswrapper[2579]: I0703 11:48:11.597293 2579 kuberuntime_container.go:872] "Killing container with a grace period" pod="openshift-ingress/e2e-api-a-515460-openshift-default-897bbd7c-xhjtn" podUID="75a2fbf5-528e-4910-b92b-e575c5ccf0f0" containerName="istio-proxy" containerID="cri-o://fae9c063c680239c53dbcd45c9329a5d8541d1ef6ff0ed0557a40ec5b732e45a" gracePeriod=30 Jul 03 11:48:12.587351 ip-10-0-136-0 kubenswrapper[2579]: I0703 11:48:12.587307 2579 patch_prober.go:28] interesting pod/e2e-api-b-8730be-openshift-default-58f6b5485f-vf9cs container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.133.0.72:15021/healthz/ready\": dial tcp 10.133.0.72:15021: connect: connection refused" start-of-body= Jul 03 11:48:12.587800 ip-10-0-136-0 kubenswrapper[2579]: I0703 11:48:12.587372 2579 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-api-b-8730be-openshift-default-58f6b5485f-vf9cs" podUID="df8a6db7-c558-410f-b596-51f30826429a" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.133.0.72:15021/healthz/ready\": dial tcp 10.133.0.72:15021: connect: connection refused" Jul 03 11:48:13.586264 ip-10-0-136-0 kubenswrapper[2579]: I0703 11:48:13.586174 2579 patch_prober.go:28] interesting pod/e2e-api-b-8730be-openshift-default-58f6b5485f-vf9cs container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.133.0.72:15021/healthz/ready\": dial tcp 10.133.0.72:15021: connect: connection refused" start-of-body= Jul 03 11:48:13.586477 ip-10-0-136-0 kubenswrapper[2579]: I0703 11:48:13.586277 2579 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-api-b-8730be-openshift-default-58f6b5485f-vf9cs" podUID="df8a6db7-c558-410f-b596-51f30826429a" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.133.0.72:15021/healthz/ready\": dial tcp 10.133.0.72:15021: connect: connection refused" Jul 03 11:48:14.591028 ip-10-0-136-0 kubenswrapper[2579]: I0703 11:48:14.590989 2579 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="openshift-ingress/e2e-api-b-8730be-openshift-default-58f6b5485f-vf9cs" Jul 03 11:48:14.591539 ip-10-0-136-0 kubenswrapper[2579]: I0703 11:48:14.591517 2579 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/e2e-api-b-8730be-openshift-default-58f6b5485f-vf9cs" Jul 03 11:48:14.592557 ip-10-0-136-0 kubenswrapper[2579]: I0703 11:48:14.592524 2579 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/e2e-api-b-8730be-openshift-default-58f6b5485f-vf9cs" Jul 03 11:48:16.877583 ip-10-0-136-0 kubenswrapper[2579]: I0703 11:48:16.877548 2579 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-api-a-515460-openshift-default-897bbd7c-xhjtn" Jul 03 11:48:16.988162 ip-10-0-136-0 kubenswrapper[2579]: I0703 11:48:16.988123 2579 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/75a2fbf5-528e-4910-b92b-e575c5ccf0f0-workload-certs\") pod \"75a2fbf5-528e-4910-b92b-e575c5ccf0f0\" (UID: \"75a2fbf5-528e-4910-b92b-e575c5ccf0f0\") " Jul 03 11:48:16.988162 ip-10-0-136-0 kubenswrapper[2579]: I0703 11:48:16.988161 2579 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/75a2fbf5-528e-4910-b92b-e575c5ccf0f0-istio-podinfo\") pod \"75a2fbf5-528e-4910-b92b-e575c5ccf0f0\" (UID: \"75a2fbf5-528e-4910-b92b-e575c5ccf0f0\") " Jul 03 11:48:16.988455 ip-10-0-136-0 kubenswrapper[2579]: I0703 11:48:16.988245 2579 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/75a2fbf5-528e-4910-b92b-e575c5ccf0f0-credential-socket\") pod \"75a2fbf5-528e-4910-b92b-e575c5ccf0f0\" (UID: \"75a2fbf5-528e-4910-b92b-e575c5ccf0f0\") " Jul 03 11:48:16.988455 ip-10-0-136-0 kubenswrapper[2579]: I0703 11:48:16.988311 2579 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/75a2fbf5-528e-4910-b92b-e575c5ccf0f0-workload-socket\") pod \"75a2fbf5-528e-4910-b92b-e575c5ccf0f0\" (UID: \"75a2fbf5-528e-4910-b92b-e575c5ccf0f0\") " Jul 03 11:48:16.988455 ip-10-0-136-0 kubenswrapper[2579]: I0703 11:48:16.988379 2579 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/75a2fbf5-528e-4910-b92b-e575c5ccf0f0-istio-token\") pod \"75a2fbf5-528e-4910-b92b-e575c5ccf0f0\" (UID: \"75a2fbf5-528e-4910-b92b-e575c5ccf0f0\") " Jul 03 11:48:16.988455 ip-10-0-136-0 kubenswrapper[2579]: I0703 11:48:16.988419 2579 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-5nspb\" (UniqueName: \"kubernetes.io/projected/75a2fbf5-528e-4910-b92b-e575c5ccf0f0-kube-api-access-5nspb\") pod \"75a2fbf5-528e-4910-b92b-e575c5ccf0f0\" (UID: \"75a2fbf5-528e-4910-b92b-e575c5ccf0f0\") " Jul 03 11:48:16.988653 ip-10-0-136-0 kubenswrapper[2579]: I0703 11:48:16.988465 2579 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/75a2fbf5-528e-4910-b92b-e575c5ccf0f0-istiod-ca-cert\") pod \"75a2fbf5-528e-4910-b92b-e575c5ccf0f0\" (UID: \"75a2fbf5-528e-4910-b92b-e575c5ccf0f0\") " Jul 03 11:48:16.988653 ip-10-0-136-0 kubenswrapper[2579]: I0703 11:48:16.988486 2579 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/75a2fbf5-528e-4910-b92b-e575c5ccf0f0-workload-certs" (OuterVolumeSpecName: "workload-certs") pod "75a2fbf5-528e-4910-b92b-e575c5ccf0f0" (UID: "75a2fbf5-528e-4910-b92b-e575c5ccf0f0"). InnerVolumeSpecName "workload-certs". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Jul 03 11:48:16.988653 ip-10-0-136-0 kubenswrapper[2579]: I0703 11:48:16.988497 2579 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/75a2fbf5-528e-4910-b92b-e575c5ccf0f0-istio-envoy\") pod \"75a2fbf5-528e-4910-b92b-e575c5ccf0f0\" (UID: \"75a2fbf5-528e-4910-b92b-e575c5ccf0f0\") " Jul 03 11:48:16.988653 ip-10-0-136-0 kubenswrapper[2579]: I0703 11:48:16.988555 2579 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/75a2fbf5-528e-4910-b92b-e575c5ccf0f0-istio-data\") pod \"75a2fbf5-528e-4910-b92b-e575c5ccf0f0\" (UID: \"75a2fbf5-528e-4910-b92b-e575c5ccf0f0\") " Jul 03 11:48:16.988933 ip-10-0-136-0 kubenswrapper[2579]: I0703 11:48:16.988666 2579 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/75a2fbf5-528e-4910-b92b-e575c5ccf0f0-credential-socket" (OuterVolumeSpecName: "credential-socket") pod "75a2fbf5-528e-4910-b92b-e575c5ccf0f0" (UID: "75a2fbf5-528e-4910-b92b-e575c5ccf0f0"). InnerVolumeSpecName "credential-socket". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Jul 03 11:48:16.988933 ip-10-0-136-0 kubenswrapper[2579]: I0703 11:48:16.988680 2579 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/75a2fbf5-528e-4910-b92b-e575c5ccf0f0-workload-socket" (OuterVolumeSpecName: "workload-socket") pod "75a2fbf5-528e-4910-b92b-e575c5ccf0f0" (UID: "75a2fbf5-528e-4910-b92b-e575c5ccf0f0"). InnerVolumeSpecName "workload-socket". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Jul 03 11:48:16.989044 ip-10-0-136-0 kubenswrapper[2579]: I0703 11:48:16.988973 2579 reconciler_common.go:299] "Volume detached for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/75a2fbf5-528e-4910-b92b-e575c5ccf0f0-workload-certs\") on node \"ip-10-0-136-0.ec2.internal\" DevicePath \"\"" Jul 03 11:48:16.989044 ip-10-0-136-0 kubenswrapper[2579]: I0703 11:48:16.988994 2579 reconciler_common.go:299] "Volume detached for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/75a2fbf5-528e-4910-b92b-e575c5ccf0f0-credential-socket\") on node \"ip-10-0-136-0.ec2.internal\" DevicePath \"\"" Jul 03 11:48:16.989044 ip-10-0-136-0 kubenswrapper[2579]: I0703 11:48:16.989009 2579 reconciler_common.go:299] "Volume detached for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/75a2fbf5-528e-4910-b92b-e575c5ccf0f0-workload-socket\") on node \"ip-10-0-136-0.ec2.internal\" DevicePath \"\"" Jul 03 11:48:16.989044 ip-10-0-136-0 kubenswrapper[2579]: I0703 11:48:16.988983 2579 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/75a2fbf5-528e-4910-b92b-e575c5ccf0f0-istiod-ca-cert" (OuterVolumeSpecName: "istiod-ca-cert") pod "75a2fbf5-528e-4910-b92b-e575c5ccf0f0" (UID: "75a2fbf5-528e-4910-b92b-e575c5ccf0f0"). InnerVolumeSpecName "istiod-ca-cert". PluginName "kubernetes.io/configmap", VolumeGIDValue "" Jul 03 11:48:16.989903 ip-10-0-136-0 kubenswrapper[2579]: I0703 11:48:16.989868 2579 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/75a2fbf5-528e-4910-b92b-e575c5ccf0f0-istio-data" (OuterVolumeSpecName: "istio-data") pod "75a2fbf5-528e-4910-b92b-e575c5ccf0f0" (UID: "75a2fbf5-528e-4910-b92b-e575c5ccf0f0"). InnerVolumeSpecName "istio-data". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Jul 03 11:48:16.990760 ip-10-0-136-0 kubenswrapper[2579]: I0703 11:48:16.990719 2579 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/75a2fbf5-528e-4910-b92b-e575c5ccf0f0-istio-token" (OuterVolumeSpecName: "istio-token") pod "75a2fbf5-528e-4910-b92b-e575c5ccf0f0" (UID: "75a2fbf5-528e-4910-b92b-e575c5ccf0f0"). InnerVolumeSpecName "istio-token". PluginName "kubernetes.io/projected", VolumeGIDValue "" Jul 03 11:48:16.990760 ip-10-0-136-0 kubenswrapper[2579]: I0703 11:48:16.990750 2579 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/downward-api/75a2fbf5-528e-4910-b92b-e575c5ccf0f0-istio-podinfo" (OuterVolumeSpecName: "istio-podinfo") pod "75a2fbf5-528e-4910-b92b-e575c5ccf0f0" (UID: "75a2fbf5-528e-4910-b92b-e575c5ccf0f0"). InnerVolumeSpecName "istio-podinfo". PluginName "kubernetes.io/downward-api", VolumeGIDValue "" Jul 03 11:48:16.990931 ip-10-0-136-0 kubenswrapper[2579]: I0703 11:48:16.990911 2579 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/75a2fbf5-528e-4910-b92b-e575c5ccf0f0-istio-envoy" (OuterVolumeSpecName: "istio-envoy") pod "75a2fbf5-528e-4910-b92b-e575c5ccf0f0" (UID: "75a2fbf5-528e-4910-b92b-e575c5ccf0f0"). InnerVolumeSpecName "istio-envoy". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Jul 03 11:48:16.991124 ip-10-0-136-0 kubenswrapper[2579]: I0703 11:48:16.991101 2579 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/75a2fbf5-528e-4910-b92b-e575c5ccf0f0-kube-api-access-5nspb" (OuterVolumeSpecName: "kube-api-access-5nspb") pod "75a2fbf5-528e-4910-b92b-e575c5ccf0f0" (UID: "75a2fbf5-528e-4910-b92b-e575c5ccf0f0"). InnerVolumeSpecName "kube-api-access-5nspb". PluginName "kubernetes.io/projected", VolumeGIDValue "" Jul 03 11:48:17.090018 ip-10-0-136-0 kubenswrapper[2579]: I0703 11:48:17.089977 2579 reconciler_common.go:299] "Volume detached for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/75a2fbf5-528e-4910-b92b-e575c5ccf0f0-istio-token\") on node \"ip-10-0-136-0.ec2.internal\" DevicePath \"\"" Jul 03 11:48:17.090018 ip-10-0-136-0 kubenswrapper[2579]: I0703 11:48:17.090020 2579 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-5nspb\" (UniqueName: \"kubernetes.io/projected/75a2fbf5-528e-4910-b92b-e575c5ccf0f0-kube-api-access-5nspb\") on node \"ip-10-0-136-0.ec2.internal\" DevicePath \"\"" Jul 03 11:48:17.090328 ip-10-0-136-0 kubenswrapper[2579]: I0703 11:48:17.090036 2579 reconciler_common.go:299] "Volume detached for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/75a2fbf5-528e-4910-b92b-e575c5ccf0f0-istiod-ca-cert\") on node \"ip-10-0-136-0.ec2.internal\" DevicePath \"\"" Jul 03 11:48:17.090328 ip-10-0-136-0 kubenswrapper[2579]: I0703 11:48:17.090048 2579 reconciler_common.go:299] "Volume detached for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/75a2fbf5-528e-4910-b92b-e575c5ccf0f0-istio-envoy\") on node \"ip-10-0-136-0.ec2.internal\" DevicePath \"\"" Jul 03 11:48:17.090328 ip-10-0-136-0 kubenswrapper[2579]: I0703 11:48:17.090059 2579 reconciler_common.go:299] "Volume detached for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/75a2fbf5-528e-4910-b92b-e575c5ccf0f0-istio-data\") on node \"ip-10-0-136-0.ec2.internal\" DevicePath \"\"" Jul 03 11:48:17.090328 ip-10-0-136-0 kubenswrapper[2579]: I0703 11:48:17.090071 2579 reconciler_common.go:299] "Volume detached for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/75a2fbf5-528e-4910-b92b-e575c5ccf0f0-istio-podinfo\") on node \"ip-10-0-136-0.ec2.internal\" DevicePath \"\"" Jul 03 11:48:17.099676 ip-10-0-136-0 kubenswrapper[2579]: I0703 11:48:17.099636 2579 generic.go:356] "Generic (PLEG): container finished" podID="75a2fbf5-528e-4910-b92b-e575c5ccf0f0" containerID="fae9c063c680239c53dbcd45c9329a5d8541d1ef6ff0ed0557a40ec5b732e45a" exitCode=0 Jul 03 11:48:17.099882 ip-10-0-136-0 kubenswrapper[2579]: I0703 11:48:17.099691 2579 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-api-a-515460-openshift-default-897bbd7c-xhjtn" Jul 03 11:48:17.099882 ip-10-0-136-0 kubenswrapper[2579]: I0703 11:48:17.099721 2579 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-api-a-515460-openshift-default-897bbd7c-xhjtn" event={"ID":"75a2fbf5-528e-4910-b92b-e575c5ccf0f0","Type":"ContainerDied","Data":"fae9c063c680239c53dbcd45c9329a5d8541d1ef6ff0ed0557a40ec5b732e45a"} Jul 03 11:48:17.099882 ip-10-0-136-0 kubenswrapper[2579]: I0703 11:48:17.099792 2579 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-api-a-515460-openshift-default-897bbd7c-xhjtn" event={"ID":"75a2fbf5-528e-4910-b92b-e575c5ccf0f0","Type":"ContainerDied","Data":"7e44d9699fe732bb5c59ab10f2e036fab09281da9de8f89b6de38839792d06cb"} Jul 03 11:48:17.099882 ip-10-0-136-0 kubenswrapper[2579]: I0703 11:48:17.099826 2579 scope.go:117] "RemoveContainer" containerID="fae9c063c680239c53dbcd45c9329a5d8541d1ef6ff0ed0557a40ec5b732e45a" Jul 03 11:48:17.111330 ip-10-0-136-0 kubenswrapper[2579]: I0703 11:48:17.111305 2579 scope.go:117] "RemoveContainer" containerID="19cad7f406870be1a95f8e2be936db1ea845975d6c74789ee75b6f610ab44c44" Jul 03 11:48:17.121137 ip-10-0-136-0 kubenswrapper[2579]: I0703 11:48:17.121114 2579 scope.go:117] "RemoveContainer" containerID="fae9c063c680239c53dbcd45c9329a5d8541d1ef6ff0ed0557a40ec5b732e45a" Jul 03 11:48:17.121563 ip-10-0-136-0 kubenswrapper[2579]: E0703 11:48:17.121540 2579 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"fae9c063c680239c53dbcd45c9329a5d8541d1ef6ff0ed0557a40ec5b732e45a\": container with ID starting with fae9c063c680239c53dbcd45c9329a5d8541d1ef6ff0ed0557a40ec5b732e45a not found: ID does not exist" containerID="fae9c063c680239c53dbcd45c9329a5d8541d1ef6ff0ed0557a40ec5b732e45a" Jul 03 11:48:17.121618 ip-10-0-136-0 kubenswrapper[2579]: I0703 11:48:17.121574 2579 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"fae9c063c680239c53dbcd45c9329a5d8541d1ef6ff0ed0557a40ec5b732e45a"} err="failed to get container status \"fae9c063c680239c53dbcd45c9329a5d8541d1ef6ff0ed0557a40ec5b732e45a\": rpc error: code = NotFound desc = could not find container \"fae9c063c680239c53dbcd45c9329a5d8541d1ef6ff0ed0557a40ec5b732e45a\": container with ID starting with fae9c063c680239c53dbcd45c9329a5d8541d1ef6ff0ed0557a40ec5b732e45a not found: ID does not exist" Jul 03 11:48:17.121618 ip-10-0-136-0 kubenswrapper[2579]: I0703 11:48:17.121594 2579 scope.go:117] "RemoveContainer" containerID="19cad7f406870be1a95f8e2be936db1ea845975d6c74789ee75b6f610ab44c44" Jul 03 11:48:17.121869 ip-10-0-136-0 kubenswrapper[2579]: E0703 11:48:17.121846 2579 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"19cad7f406870be1a95f8e2be936db1ea845975d6c74789ee75b6f610ab44c44\": container with ID starting with 19cad7f406870be1a95f8e2be936db1ea845975d6c74789ee75b6f610ab44c44 not found: ID does not exist" containerID="19cad7f406870be1a95f8e2be936db1ea845975d6c74789ee75b6f610ab44c44" Jul 03 11:48:17.121974 ip-10-0-136-0 kubenswrapper[2579]: I0703 11:48:17.121872 2579 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"19cad7f406870be1a95f8e2be936db1ea845975d6c74789ee75b6f610ab44c44"} err="failed to get container status \"19cad7f406870be1a95f8e2be936db1ea845975d6c74789ee75b6f610ab44c44\": rpc error: code = NotFound desc = could not find container \"19cad7f406870be1a95f8e2be936db1ea845975d6c74789ee75b6f610ab44c44\": container with ID starting with 19cad7f406870be1a95f8e2be936db1ea845975d6c74789ee75b6f610ab44c44 not found: ID does not exist" Jul 03 11:48:17.127885 ip-10-0-136-0 kubenswrapper[2579]: I0703 11:48:17.127841 2579 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-ingress/e2e-api-a-515460-openshift-default-897bbd7c-xhjtn"] Jul 03 11:48:17.130595 ip-10-0-136-0 kubenswrapper[2579]: I0703 11:48:17.130566 2579 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["openshift-ingress/e2e-api-a-515460-openshift-default-897bbd7c-xhjtn"] Jul 03 11:48:17.993314 ip-10-0-136-0 kubenswrapper[2579]: I0703 11:48:17.993278 2579 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="75a2fbf5-528e-4910-b92b-e575c5ccf0f0" path="/var/lib/kubelet/pods/75a2fbf5-528e-4910-b92b-e575c5ccf0f0/volumes" Jul 03 11:48:19.921458 ip-10-0-136-0 kubenswrapper[2579]: I0703 11:48:19.921416 2579 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-ingress/e2e-api-b-8730be-openshift-default-58f6b5485f-vf9cs"] Jul 03 11:48:19.921872 ip-10-0-136-0 kubenswrapper[2579]: I0703 11:48:19.921674 2579 kuberuntime_container.go:872] "Killing container with a grace period" pod="openshift-ingress/e2e-api-b-8730be-openshift-default-58f6b5485f-vf9cs" podUID="df8a6db7-c558-410f-b596-51f30826429a" containerName="istio-proxy" containerID="cri-o://d2d027d214cff4180aa14e3c6dbb9ff75336bca8d1ac65f91c7ee938e5ac3a6b" gracePeriod=30 Jul 03 11:48:20.808570 ip-10-0-136-0 kubenswrapper[2579]: I0703 11:48:20.808525 2579 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-ingress/e2e-shared-a-9c2097-openshift-default-6d687665ff-25rvz"] Jul 03 11:48:20.808998 ip-10-0-136-0 kubenswrapper[2579]: I0703 11:48:20.808983 2579 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="75a2fbf5-528e-4910-b92b-e575c5ccf0f0" containerName="istio-proxy" Jul 03 11:48:20.809059 ip-10-0-136-0 kubenswrapper[2579]: I0703 11:48:20.809002 2579 state_mem.go:107] "Deleted CPUSet assignment" podUID="75a2fbf5-528e-4910-b92b-e575c5ccf0f0" containerName="istio-proxy" Jul 03 11:48:20.809095 ip-10-0-136-0 kubenswrapper[2579]: I0703 11:48:20.809085 2579 memory_manager.go:368] "RemoveStaleState removing state" podUID="75a2fbf5-528e-4910-b92b-e575c5ccf0f0" containerName="istio-proxy" Jul 03 11:48:20.809129 ip-10-0-136-0 kubenswrapper[2579]: I0703 11:48:20.809098 2579 memory_manager.go:368] "RemoveStaleState removing state" podUID="75a2fbf5-528e-4910-b92b-e575c5ccf0f0" containerName="istio-proxy" Jul 03 11:48:20.812614 ip-10-0-136-0 kubenswrapper[2579]: I0703 11:48:20.812586 2579 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-shared-a-9c2097-openshift-default-6d687665ff-25rvz" Jul 03 11:48:20.817096 ip-10-0-136-0 kubenswrapper[2579]: I0703 11:48:20.817063 2579 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress\"/\"e2e-shared-a-9c2097-openshift-default-dockercfg-pjxpq\"" Jul 03 11:48:20.831053 ip-10-0-136-0 kubenswrapper[2579]: I0703 11:48:20.831007 2579 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/f4cf29c7-812e-45ca-bf86-80a5cff794ac-istiod-ca-cert\") pod \"e2e-shared-a-9c2097-openshift-default-6d687665ff-25rvz\" (UID: \"f4cf29c7-812e-45ca-bf86-80a5cff794ac\") " pod="openshift-ingress/e2e-shared-a-9c2097-openshift-default-6d687665ff-25rvz" Jul 03 11:48:20.831319 ip-10-0-136-0 kubenswrapper[2579]: I0703 11:48:20.831074 2579 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/f4cf29c7-812e-45ca-bf86-80a5cff794ac-istio-token\") pod \"e2e-shared-a-9c2097-openshift-default-6d687665ff-25rvz\" (UID: \"f4cf29c7-812e-45ca-bf86-80a5cff794ac\") " pod="openshift-ingress/e2e-shared-a-9c2097-openshift-default-6d687665ff-25rvz" Jul 03 11:48:20.831319 ip-10-0-136-0 kubenswrapper[2579]: I0703 11:48:20.831118 2579 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/f4cf29c7-812e-45ca-bf86-80a5cff794ac-credential-socket\") pod \"e2e-shared-a-9c2097-openshift-default-6d687665ff-25rvz\" (UID: \"f4cf29c7-812e-45ca-bf86-80a5cff794ac\") " pod="openshift-ingress/e2e-shared-a-9c2097-openshift-default-6d687665ff-25rvz" Jul 03 11:48:20.831319 ip-10-0-136-0 kubenswrapper[2579]: I0703 11:48:20.831282 2579 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/f4cf29c7-812e-45ca-bf86-80a5cff794ac-workload-socket\") pod \"e2e-shared-a-9c2097-openshift-default-6d687665ff-25rvz\" (UID: \"f4cf29c7-812e-45ca-bf86-80a5cff794ac\") " pod="openshift-ingress/e2e-shared-a-9c2097-openshift-default-6d687665ff-25rvz" Jul 03 11:48:20.831319 ip-10-0-136-0 kubenswrapper[2579]: I0703 11:48:20.831318 2579 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-vxql8\" (UniqueName: \"kubernetes.io/projected/f4cf29c7-812e-45ca-bf86-80a5cff794ac-kube-api-access-vxql8\") pod \"e2e-shared-a-9c2097-openshift-default-6d687665ff-25rvz\" (UID: \"f4cf29c7-812e-45ca-bf86-80a5cff794ac\") " pod="openshift-ingress/e2e-shared-a-9c2097-openshift-default-6d687665ff-25rvz" Jul 03 11:48:20.831549 ip-10-0-136-0 kubenswrapper[2579]: I0703 11:48:20.831352 2579 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/f4cf29c7-812e-45ca-bf86-80a5cff794ac-istio-envoy\") pod \"e2e-shared-a-9c2097-openshift-default-6d687665ff-25rvz\" (UID: \"f4cf29c7-812e-45ca-bf86-80a5cff794ac\") " pod="openshift-ingress/e2e-shared-a-9c2097-openshift-default-6d687665ff-25rvz" Jul 03 11:48:20.831549 ip-10-0-136-0 kubenswrapper[2579]: I0703 11:48:20.831386 2579 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/f4cf29c7-812e-45ca-bf86-80a5cff794ac-istio-data\") pod \"e2e-shared-a-9c2097-openshift-default-6d687665ff-25rvz\" (UID: \"f4cf29c7-812e-45ca-bf86-80a5cff794ac\") " pod="openshift-ingress/e2e-shared-a-9c2097-openshift-default-6d687665ff-25rvz" Jul 03 11:48:20.831549 ip-10-0-136-0 kubenswrapper[2579]: I0703 11:48:20.831438 2579 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/f4cf29c7-812e-45ca-bf86-80a5cff794ac-workload-certs\") pod \"e2e-shared-a-9c2097-openshift-default-6d687665ff-25rvz\" (UID: \"f4cf29c7-812e-45ca-bf86-80a5cff794ac\") " pod="openshift-ingress/e2e-shared-a-9c2097-openshift-default-6d687665ff-25rvz" Jul 03 11:48:20.831549 ip-10-0-136-0 kubenswrapper[2579]: I0703 11:48:20.831511 2579 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/f4cf29c7-812e-45ca-bf86-80a5cff794ac-istio-podinfo\") pod \"e2e-shared-a-9c2097-openshift-default-6d687665ff-25rvz\" (UID: \"f4cf29c7-812e-45ca-bf86-80a5cff794ac\") " pod="openshift-ingress/e2e-shared-a-9c2097-openshift-default-6d687665ff-25rvz" Jul 03 11:48:20.835686 ip-10-0-136-0 kubenswrapper[2579]: I0703 11:48:20.835646 2579 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/e2e-shared-a-9c2097-openshift-default-6d687665ff-25rvz"] Jul 03 11:48:20.932807 ip-10-0-136-0 kubenswrapper[2579]: I0703 11:48:20.932759 2579 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/f4cf29c7-812e-45ca-bf86-80a5cff794ac-istio-podinfo\") pod \"e2e-shared-a-9c2097-openshift-default-6d687665ff-25rvz\" (UID: \"f4cf29c7-812e-45ca-bf86-80a5cff794ac\") " pod="openshift-ingress/e2e-shared-a-9c2097-openshift-default-6d687665ff-25rvz" Jul 03 11:48:20.933390 ip-10-0-136-0 kubenswrapper[2579]: I0703 11:48:20.932845 2579 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/f4cf29c7-812e-45ca-bf86-80a5cff794ac-istiod-ca-cert\") pod \"e2e-shared-a-9c2097-openshift-default-6d687665ff-25rvz\" (UID: \"f4cf29c7-812e-45ca-bf86-80a5cff794ac\") " pod="openshift-ingress/e2e-shared-a-9c2097-openshift-default-6d687665ff-25rvz" Jul 03 11:48:20.933390 ip-10-0-136-0 kubenswrapper[2579]: I0703 11:48:20.932875 2579 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/f4cf29c7-812e-45ca-bf86-80a5cff794ac-istio-token\") pod \"e2e-shared-a-9c2097-openshift-default-6d687665ff-25rvz\" (UID: \"f4cf29c7-812e-45ca-bf86-80a5cff794ac\") " pod="openshift-ingress/e2e-shared-a-9c2097-openshift-default-6d687665ff-25rvz" Jul 03 11:48:20.933390 ip-10-0-136-0 kubenswrapper[2579]: I0703 11:48:20.932901 2579 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/f4cf29c7-812e-45ca-bf86-80a5cff794ac-credential-socket\") pod \"e2e-shared-a-9c2097-openshift-default-6d687665ff-25rvz\" (UID: \"f4cf29c7-812e-45ca-bf86-80a5cff794ac\") " pod="openshift-ingress/e2e-shared-a-9c2097-openshift-default-6d687665ff-25rvz" Jul 03 11:48:20.933390 ip-10-0-136-0 kubenswrapper[2579]: I0703 11:48:20.933097 2579 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/f4cf29c7-812e-45ca-bf86-80a5cff794ac-workload-socket\") pod \"e2e-shared-a-9c2097-openshift-default-6d687665ff-25rvz\" (UID: \"f4cf29c7-812e-45ca-bf86-80a5cff794ac\") " pod="openshift-ingress/e2e-shared-a-9c2097-openshift-default-6d687665ff-25rvz" Jul 03 11:48:20.933390 ip-10-0-136-0 kubenswrapper[2579]: I0703 11:48:20.933138 2579 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-vxql8\" (UniqueName: \"kubernetes.io/projected/f4cf29c7-812e-45ca-bf86-80a5cff794ac-kube-api-access-vxql8\") pod \"e2e-shared-a-9c2097-openshift-default-6d687665ff-25rvz\" (UID: \"f4cf29c7-812e-45ca-bf86-80a5cff794ac\") " pod="openshift-ingress/e2e-shared-a-9c2097-openshift-default-6d687665ff-25rvz" Jul 03 11:48:20.933390 ip-10-0-136-0 kubenswrapper[2579]: I0703 11:48:20.933174 2579 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/f4cf29c7-812e-45ca-bf86-80a5cff794ac-istio-envoy\") pod \"e2e-shared-a-9c2097-openshift-default-6d687665ff-25rvz\" (UID: \"f4cf29c7-812e-45ca-bf86-80a5cff794ac\") " pod="openshift-ingress/e2e-shared-a-9c2097-openshift-default-6d687665ff-25rvz" Jul 03 11:48:20.933838 ip-10-0-136-0 kubenswrapper[2579]: I0703 11:48:20.933443 2579 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/f4cf29c7-812e-45ca-bf86-80a5cff794ac-istio-data\") pod \"e2e-shared-a-9c2097-openshift-default-6d687665ff-25rvz\" (UID: \"f4cf29c7-812e-45ca-bf86-80a5cff794ac\") " pod="openshift-ingress/e2e-shared-a-9c2097-openshift-default-6d687665ff-25rvz" Jul 03 11:48:20.933838 ip-10-0-136-0 kubenswrapper[2579]: I0703 11:48:20.933523 2579 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/f4cf29c7-812e-45ca-bf86-80a5cff794ac-workload-certs\") pod \"e2e-shared-a-9c2097-openshift-default-6d687665ff-25rvz\" (UID: \"f4cf29c7-812e-45ca-bf86-80a5cff794ac\") " pod="openshift-ingress/e2e-shared-a-9c2097-openshift-default-6d687665ff-25rvz" Jul 03 11:48:20.933838 ip-10-0-136-0 kubenswrapper[2579]: I0703 11:48:20.933552 2579 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/f4cf29c7-812e-45ca-bf86-80a5cff794ac-workload-socket\") pod \"e2e-shared-a-9c2097-openshift-default-6d687665ff-25rvz\" (UID: \"f4cf29c7-812e-45ca-bf86-80a5cff794ac\") " pod="openshift-ingress/e2e-shared-a-9c2097-openshift-default-6d687665ff-25rvz" Jul 03 11:48:20.934081 ip-10-0-136-0 kubenswrapper[2579]: I0703 11:48:20.933855 2579 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/f4cf29c7-812e-45ca-bf86-80a5cff794ac-istiod-ca-cert\") pod \"e2e-shared-a-9c2097-openshift-default-6d687665ff-25rvz\" (UID: \"f4cf29c7-812e-45ca-bf86-80a5cff794ac\") " pod="openshift-ingress/e2e-shared-a-9c2097-openshift-default-6d687665ff-25rvz" Jul 03 11:48:20.934081 ip-10-0-136-0 kubenswrapper[2579]: I0703 11:48:20.933975 2579 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/f4cf29c7-812e-45ca-bf86-80a5cff794ac-istio-data\") pod \"e2e-shared-a-9c2097-openshift-default-6d687665ff-25rvz\" (UID: \"f4cf29c7-812e-45ca-bf86-80a5cff794ac\") " pod="openshift-ingress/e2e-shared-a-9c2097-openshift-default-6d687665ff-25rvz" Jul 03 11:48:20.934081 ip-10-0-136-0 kubenswrapper[2579]: I0703 11:48:20.933997 2579 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/f4cf29c7-812e-45ca-bf86-80a5cff794ac-workload-certs\") pod \"e2e-shared-a-9c2097-openshift-default-6d687665ff-25rvz\" (UID: \"f4cf29c7-812e-45ca-bf86-80a5cff794ac\") " pod="openshift-ingress/e2e-shared-a-9c2097-openshift-default-6d687665ff-25rvz" Jul 03 11:48:20.934081 ip-10-0-136-0 kubenswrapper[2579]: I0703 11:48:20.934056 2579 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/f4cf29c7-812e-45ca-bf86-80a5cff794ac-credential-socket\") pod \"e2e-shared-a-9c2097-openshift-default-6d687665ff-25rvz\" (UID: \"f4cf29c7-812e-45ca-bf86-80a5cff794ac\") " pod="openshift-ingress/e2e-shared-a-9c2097-openshift-default-6d687665ff-25rvz" Jul 03 11:48:20.935700 ip-10-0-136-0 kubenswrapper[2579]: I0703 11:48:20.935676 2579 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/f4cf29c7-812e-45ca-bf86-80a5cff794ac-istio-envoy\") pod \"e2e-shared-a-9c2097-openshift-default-6d687665ff-25rvz\" (UID: \"f4cf29c7-812e-45ca-bf86-80a5cff794ac\") " pod="openshift-ingress/e2e-shared-a-9c2097-openshift-default-6d687665ff-25rvz" Jul 03 11:48:20.935913 ip-10-0-136-0 kubenswrapper[2579]: I0703 11:48:20.935897 2579 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/f4cf29c7-812e-45ca-bf86-80a5cff794ac-istio-podinfo\") pod \"e2e-shared-a-9c2097-openshift-default-6d687665ff-25rvz\" (UID: \"f4cf29c7-812e-45ca-bf86-80a5cff794ac\") " pod="openshift-ingress/e2e-shared-a-9c2097-openshift-default-6d687665ff-25rvz" Jul 03 11:48:20.945616 ip-10-0-136-0 kubenswrapper[2579]: I0703 11:48:20.945571 2579 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/f4cf29c7-812e-45ca-bf86-80a5cff794ac-istio-token\") pod \"e2e-shared-a-9c2097-openshift-default-6d687665ff-25rvz\" (UID: \"f4cf29c7-812e-45ca-bf86-80a5cff794ac\") " pod="openshift-ingress/e2e-shared-a-9c2097-openshift-default-6d687665ff-25rvz" Jul 03 11:48:20.955420 ip-10-0-136-0 kubenswrapper[2579]: I0703 11:48:20.955385 2579 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-vxql8\" (UniqueName: \"kubernetes.io/projected/f4cf29c7-812e-45ca-bf86-80a5cff794ac-kube-api-access-vxql8\") pod \"e2e-shared-a-9c2097-openshift-default-6d687665ff-25rvz\" (UID: \"f4cf29c7-812e-45ca-bf86-80a5cff794ac\") " pod="openshift-ingress/e2e-shared-a-9c2097-openshift-default-6d687665ff-25rvz" Jul 03 11:48:21.127108 ip-10-0-136-0 kubenswrapper[2579]: I0703 11:48:21.126998 2579 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-shared-a-9c2097-openshift-default-6d687665ff-25rvz" Jul 03 11:48:21.312974 ip-10-0-136-0 kubenswrapper[2579]: I0703 11:48:21.312943 2579 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/e2e-shared-a-9c2097-openshift-default-6d687665ff-25rvz"] Jul 03 11:48:21.315678 ip-10-0-136-0 kubenswrapper[2579]: W0703 11:48:21.315646 2579 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podf4cf29c7_812e_45ca_bf86_80a5cff794ac.slice/crio-564d5e71a7aacef936215de629c849bdbaf861e7a600b62aeb496299044c3aa8 WatchSource:0}: Error finding container 564d5e71a7aacef936215de629c849bdbaf861e7a600b62aeb496299044c3aa8: Status 404 returned error can't find the container with id 564d5e71a7aacef936215de629c849bdbaf861e7a600b62aeb496299044c3aa8 Jul 03 11:48:21.318011 ip-10-0-136-0 kubenswrapper[2579]: I0703 11:48:21.317971 2579 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"30892156Ki","pods":"250"} Jul 03 11:48:21.318081 ip-10-0-136-0 kubenswrapper[2579]: I0703 11:48:21.318061 2579 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"30892156Ki","pods":"250"} Jul 03 11:48:21.318126 ip-10-0-136-0 kubenswrapper[2579]: I0703 11:48:21.318095 2579 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"30892156Ki","pods":"250"} Jul 03 11:48:22.133935 ip-10-0-136-0 kubenswrapper[2579]: I0703 11:48:22.133890 2579 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-shared-a-9c2097-openshift-default-6d687665ff-25rvz" event={"ID":"f4cf29c7-812e-45ca-bf86-80a5cff794ac","Type":"ContainerStarted","Data":"693748e656f6194cc63b93d7d5c369f12d3d882ae6cd31d22e02cbf80c21791d"} Jul 03 11:48:22.134513 ip-10-0-136-0 kubenswrapper[2579]: I0703 11:48:22.133968 2579 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-shared-a-9c2097-openshift-default-6d687665ff-25rvz" event={"ID":"f4cf29c7-812e-45ca-bf86-80a5cff794ac","Type":"ContainerStarted","Data":"564d5e71a7aacef936215de629c849bdbaf861e7a600b62aeb496299044c3aa8"} Jul 03 11:48:22.167574 ip-10-0-136-0 kubenswrapper[2579]: I0703 11:48:22.167513 2579 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-ingress/e2e-shared-a-9c2097-openshift-default-6d687665ff-25rvz" podStartSLOduration=2.167492413 podStartE2EDuration="2.167492413s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-03 11:48:20 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-03 11:48:22.166222528 +0000 UTC m=+2158.754526847" watchObservedRunningTime="2026-07-03 11:48:22.167492413 +0000 UTC m=+2158.755796732" Jul 03 11:48:23.127920 ip-10-0-136-0 kubenswrapper[2579]: I0703 11:48:23.127883 2579 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="openshift-ingress/e2e-shared-a-9c2097-openshift-default-6d687665ff-25rvz" Jul 03 11:48:23.133843 ip-10-0-136-0 kubenswrapper[2579]: I0703 11:48:23.133809 2579 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="openshift-ingress/e2e-shared-a-9c2097-openshift-default-6d687665ff-25rvz" Jul 03 11:48:23.140047 ip-10-0-136-0 kubenswrapper[2579]: I0703 11:48:23.140006 2579 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/e2e-shared-a-9c2097-openshift-default-6d687665ff-25rvz" Jul 03 11:48:23.141267 ip-10-0-136-0 kubenswrapper[2579]: I0703 11:48:23.141244 2579 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/e2e-shared-a-9c2097-openshift-default-6d687665ff-25rvz" Jul 03 11:48:24.961050 ip-10-0-136-0 kubenswrapper[2579]: E0703 11:48:24.960995 2579 cadvisor_stats_provider.go:567] "Partial failure issuing cadvisor.ContainerInfoV2" err="partial failures: [\"/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-poddf8a6db7_c558_410f_b596_51f30826429a.slice/crio-conmon-d2d027d214cff4180aa14e3c6dbb9ff75336bca8d1ac65f91c7ee938e5ac3a6b.scope\": RecentStats: unable to find data in memory cache]" Jul 03 11:48:24.961642 ip-10-0-136-0 kubenswrapper[2579]: E0703 11:48:24.961010 2579 cadvisor_stats_provider.go:567] "Partial failure issuing cadvisor.ContainerInfoV2" err="partial failures: [\"/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-poddf8a6db7_c558_410f_b596_51f30826429a.slice/crio-d2d027d214cff4180aa14e3c6dbb9ff75336bca8d1ac65f91c7ee938e5ac3a6b.scope\": RecentStats: unable to find data in memory cache], [\"/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-poddf8a6db7_c558_410f_b596_51f30826429a.slice/crio-conmon-d2d027d214cff4180aa14e3c6dbb9ff75336bca8d1ac65f91c7ee938e5ac3a6b.scope\": RecentStats: unable to find data in memory cache]" Jul 03 11:48:25.154445 ip-10-0-136-0 kubenswrapper[2579]: I0703 11:48:25.154403 2579 generic.go:356] "Generic (PLEG): container finished" podID="df8a6db7-c558-410f-b596-51f30826429a" containerID="d2d027d214cff4180aa14e3c6dbb9ff75336bca8d1ac65f91c7ee938e5ac3a6b" exitCode=0 Jul 03 11:48:25.154598 ip-10-0-136-0 kubenswrapper[2579]: I0703 11:48:25.154472 2579 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-api-b-8730be-openshift-default-58f6b5485f-vf9cs" event={"ID":"df8a6db7-c558-410f-b596-51f30826429a","Type":"ContainerDied","Data":"d2d027d214cff4180aa14e3c6dbb9ff75336bca8d1ac65f91c7ee938e5ac3a6b"} Jul 03 11:48:25.154598 ip-10-0-136-0 kubenswrapper[2579]: I0703 11:48:25.154576 2579 scope.go:117] "RemoveContainer" containerID="c8e7603e199155a730a3f67e6056972cc97fbe618453dd01a4f42c207b565792" Jul 03 11:48:25.202587 ip-10-0-136-0 kubenswrapper[2579]: I0703 11:48:25.202551 2579 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-api-b-8730be-openshift-default-58f6b5485f-vf9cs" Jul 03 11:48:25.282088 ip-10-0-136-0 kubenswrapper[2579]: I0703 11:48:25.282047 2579 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/df8a6db7-c558-410f-b596-51f30826429a-credential-socket\") pod \"df8a6db7-c558-410f-b596-51f30826429a\" (UID: \"df8a6db7-c558-410f-b596-51f30826429a\") " Jul 03 11:48:25.282323 ip-10-0-136-0 kubenswrapper[2579]: I0703 11:48:25.282107 2579 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/df8a6db7-c558-410f-b596-51f30826429a-istio-envoy\") pod \"df8a6db7-c558-410f-b596-51f30826429a\" (UID: \"df8a6db7-c558-410f-b596-51f30826429a\") " Jul 03 11:48:25.282323 ip-10-0-136-0 kubenswrapper[2579]: I0703 11:48:25.282127 2579 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/df8a6db7-c558-410f-b596-51f30826429a-istiod-ca-cert\") pod \"df8a6db7-c558-410f-b596-51f30826429a\" (UID: \"df8a6db7-c558-410f-b596-51f30826429a\") " Jul 03 11:48:25.282323 ip-10-0-136-0 kubenswrapper[2579]: I0703 11:48:25.282165 2579 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/df8a6db7-c558-410f-b596-51f30826429a-istio-podinfo\") pod \"df8a6db7-c558-410f-b596-51f30826429a\" (UID: \"df8a6db7-c558-410f-b596-51f30826429a\") " Jul 03 11:48:25.282323 ip-10-0-136-0 kubenswrapper[2579]: I0703 11:48:25.282211 2579 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/df8a6db7-c558-410f-b596-51f30826429a-workload-socket\") pod \"df8a6db7-c558-410f-b596-51f30826429a\" (UID: \"df8a6db7-c558-410f-b596-51f30826429a\") " Jul 03 11:48:25.282323 ip-10-0-136-0 kubenswrapper[2579]: I0703 11:48:25.282240 2579 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/df8a6db7-c558-410f-b596-51f30826429a-istio-token\") pod \"df8a6db7-c558-410f-b596-51f30826429a\" (UID: \"df8a6db7-c558-410f-b596-51f30826429a\") " Jul 03 11:48:25.282637 ip-10-0-136-0 kubenswrapper[2579]: I0703 11:48:25.282329 2579 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/df8a6db7-c558-410f-b596-51f30826429a-istio-data\") pod \"df8a6db7-c558-410f-b596-51f30826429a\" (UID: \"df8a6db7-c558-410f-b596-51f30826429a\") " Jul 03 11:48:25.282637 ip-10-0-136-0 kubenswrapper[2579]: I0703 11:48:25.282356 2579 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-zwj8c\" (UniqueName: \"kubernetes.io/projected/df8a6db7-c558-410f-b596-51f30826429a-kube-api-access-zwj8c\") pod \"df8a6db7-c558-410f-b596-51f30826429a\" (UID: \"df8a6db7-c558-410f-b596-51f30826429a\") " Jul 03 11:48:25.282637 ip-10-0-136-0 kubenswrapper[2579]: I0703 11:48:25.282395 2579 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/df8a6db7-c558-410f-b596-51f30826429a-workload-certs\") pod \"df8a6db7-c558-410f-b596-51f30826429a\" (UID: \"df8a6db7-c558-410f-b596-51f30826429a\") " Jul 03 11:48:25.282637 ip-10-0-136-0 kubenswrapper[2579]: I0703 11:48:25.282441 2579 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/df8a6db7-c558-410f-b596-51f30826429a-credential-socket" (OuterVolumeSpecName: "credential-socket") pod "df8a6db7-c558-410f-b596-51f30826429a" (UID: "df8a6db7-c558-410f-b596-51f30826429a"). InnerVolumeSpecName "credential-socket". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Jul 03 11:48:25.282875 ip-10-0-136-0 kubenswrapper[2579]: I0703 11:48:25.282777 2579 reconciler_common.go:299] "Volume detached for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/df8a6db7-c558-410f-b596-51f30826429a-credential-socket\") on node \"ip-10-0-136-0.ec2.internal\" DevicePath \"\"" Jul 03 11:48:25.282875 ip-10-0-136-0 kubenswrapper[2579]: I0703 11:48:25.282772 2579 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/df8a6db7-c558-410f-b596-51f30826429a-workload-certs" (OuterVolumeSpecName: "workload-certs") pod "df8a6db7-c558-410f-b596-51f30826429a" (UID: "df8a6db7-c558-410f-b596-51f30826429a"). InnerVolumeSpecName "workload-certs". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Jul 03 11:48:25.283047 ip-10-0-136-0 kubenswrapper[2579]: I0703 11:48:25.282999 2579 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/df8a6db7-c558-410f-b596-51f30826429a-workload-socket" (OuterVolumeSpecName: "workload-socket") pod "df8a6db7-c558-410f-b596-51f30826429a" (UID: "df8a6db7-c558-410f-b596-51f30826429a"). InnerVolumeSpecName "workload-socket". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Jul 03 11:48:25.283132 ip-10-0-136-0 kubenswrapper[2579]: I0703 11:48:25.283050 2579 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/df8a6db7-c558-410f-b596-51f30826429a-istiod-ca-cert" (OuterVolumeSpecName: "istiod-ca-cert") pod "df8a6db7-c558-410f-b596-51f30826429a" (UID: "df8a6db7-c558-410f-b596-51f30826429a"). InnerVolumeSpecName "istiod-ca-cert". PluginName "kubernetes.io/configmap", VolumeGIDValue "" Jul 03 11:48:25.283684 ip-10-0-136-0 kubenswrapper[2579]: I0703 11:48:25.283596 2579 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/df8a6db7-c558-410f-b596-51f30826429a-istio-data" (OuterVolumeSpecName: "istio-data") pod "df8a6db7-c558-410f-b596-51f30826429a" (UID: "df8a6db7-c558-410f-b596-51f30826429a"). InnerVolumeSpecName "istio-data". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Jul 03 11:48:25.284898 ip-10-0-136-0 kubenswrapper[2579]: I0703 11:48:25.284861 2579 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/df8a6db7-c558-410f-b596-51f30826429a-istio-token" (OuterVolumeSpecName: "istio-token") pod "df8a6db7-c558-410f-b596-51f30826429a" (UID: "df8a6db7-c558-410f-b596-51f30826429a"). InnerVolumeSpecName "istio-token". PluginName "kubernetes.io/projected", VolumeGIDValue "" Jul 03 11:48:25.285146 ip-10-0-136-0 kubenswrapper[2579]: I0703 11:48:25.285127 2579 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/downward-api/df8a6db7-c558-410f-b596-51f30826429a-istio-podinfo" (OuterVolumeSpecName: "istio-podinfo") pod "df8a6db7-c558-410f-b596-51f30826429a" (UID: "df8a6db7-c558-410f-b596-51f30826429a"). InnerVolumeSpecName "istio-podinfo". PluginName "kubernetes.io/downward-api", VolumeGIDValue "" Jul 03 11:48:25.285409 ip-10-0-136-0 kubenswrapper[2579]: I0703 11:48:25.285341 2579 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/df8a6db7-c558-410f-b596-51f30826429a-istio-envoy" (OuterVolumeSpecName: "istio-envoy") pod "df8a6db7-c558-410f-b596-51f30826429a" (UID: "df8a6db7-c558-410f-b596-51f30826429a"). InnerVolumeSpecName "istio-envoy". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Jul 03 11:48:25.285409 ip-10-0-136-0 kubenswrapper[2579]: I0703 11:48:25.285356 2579 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/df8a6db7-c558-410f-b596-51f30826429a-kube-api-access-zwj8c" (OuterVolumeSpecName: "kube-api-access-zwj8c") pod "df8a6db7-c558-410f-b596-51f30826429a" (UID: "df8a6db7-c558-410f-b596-51f30826429a"). InnerVolumeSpecName "kube-api-access-zwj8c". PluginName "kubernetes.io/projected", VolumeGIDValue "" Jul 03 11:48:25.383499 ip-10-0-136-0 kubenswrapper[2579]: I0703 11:48:25.383463 2579 reconciler_common.go:299] "Volume detached for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/df8a6db7-c558-410f-b596-51f30826429a-istio-data\") on node \"ip-10-0-136-0.ec2.internal\" DevicePath \"\"" Jul 03 11:48:25.383499 ip-10-0-136-0 kubenswrapper[2579]: I0703 11:48:25.383495 2579 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-zwj8c\" (UniqueName: \"kubernetes.io/projected/df8a6db7-c558-410f-b596-51f30826429a-kube-api-access-zwj8c\") on node \"ip-10-0-136-0.ec2.internal\" DevicePath \"\"" Jul 03 11:48:25.383499 ip-10-0-136-0 kubenswrapper[2579]: I0703 11:48:25.383507 2579 reconciler_common.go:299] "Volume detached for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/df8a6db7-c558-410f-b596-51f30826429a-workload-certs\") on node \"ip-10-0-136-0.ec2.internal\" DevicePath \"\"" Jul 03 11:48:25.383745 ip-10-0-136-0 kubenswrapper[2579]: I0703 11:48:25.383515 2579 reconciler_common.go:299] "Volume detached for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/df8a6db7-c558-410f-b596-51f30826429a-istio-envoy\") on node \"ip-10-0-136-0.ec2.internal\" DevicePath \"\"" Jul 03 11:48:25.383745 ip-10-0-136-0 kubenswrapper[2579]: I0703 11:48:25.383523 2579 reconciler_common.go:299] "Volume detached for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/df8a6db7-c558-410f-b596-51f30826429a-istiod-ca-cert\") on node \"ip-10-0-136-0.ec2.internal\" DevicePath \"\"" Jul 03 11:48:25.383745 ip-10-0-136-0 kubenswrapper[2579]: I0703 11:48:25.383530 2579 reconciler_common.go:299] "Volume detached for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/df8a6db7-c558-410f-b596-51f30826429a-istio-podinfo\") on node \"ip-10-0-136-0.ec2.internal\" DevicePath \"\"" Jul 03 11:48:25.383745 ip-10-0-136-0 kubenswrapper[2579]: I0703 11:48:25.383537 2579 reconciler_common.go:299] "Volume detached for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/df8a6db7-c558-410f-b596-51f30826429a-workload-socket\") on node \"ip-10-0-136-0.ec2.internal\" DevicePath \"\"" Jul 03 11:48:25.383745 ip-10-0-136-0 kubenswrapper[2579]: I0703 11:48:25.383544 2579 reconciler_common.go:299] "Volume detached for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/df8a6db7-c558-410f-b596-51f30826429a-istio-token\") on node \"ip-10-0-136-0.ec2.internal\" DevicePath \"\"" Jul 03 11:48:26.162860 ip-10-0-136-0 kubenswrapper[2579]: I0703 11:48:26.162815 2579 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-api-b-8730be-openshift-default-58f6b5485f-vf9cs" event={"ID":"df8a6db7-c558-410f-b596-51f30826429a","Type":"ContainerDied","Data":"cc81b8d9c3a39a3458b9fe8ba348308e96148d9bf6b361bad0f1e8d22116e396"} Jul 03 11:48:26.162860 ip-10-0-136-0 kubenswrapper[2579]: I0703 11:48:26.162838 2579 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-api-b-8730be-openshift-default-58f6b5485f-vf9cs" Jul 03 11:48:26.163430 ip-10-0-136-0 kubenswrapper[2579]: I0703 11:48:26.162893 2579 scope.go:117] "RemoveContainer" containerID="d2d027d214cff4180aa14e3c6dbb9ff75336bca8d1ac65f91c7ee938e5ac3a6b" Jul 03 11:48:26.188182 ip-10-0-136-0 kubenswrapper[2579]: I0703 11:48:26.188142 2579 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-ingress/e2e-api-b-8730be-openshift-default-58f6b5485f-vf9cs"] Jul 03 11:48:26.198377 ip-10-0-136-0 kubenswrapper[2579]: I0703 11:48:26.198345 2579 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["openshift-ingress/e2e-api-b-8730be-openshift-default-58f6b5485f-vf9cs"] Jul 03 11:48:27.993329 ip-10-0-136-0 kubenswrapper[2579]: I0703 11:48:27.993291 2579 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="df8a6db7-c558-410f-b596-51f30826429a" path="/var/lib/kubelet/pods/df8a6db7-c558-410f-b596-51f30826429a/volumes" Jul 03 11:48:29.447217 ip-10-0-136-0 kubenswrapper[2579]: I0703 11:48:29.445919 2579 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-ingress/e2e-shared-b-f7a699-openshift-default-ddcd795dc-k2fz7"] Jul 03 11:48:29.447217 ip-10-0-136-0 kubenswrapper[2579]: I0703 11:48:29.446572 2579 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="75a2fbf5-528e-4910-b92b-e575c5ccf0f0" containerName="istio-proxy" Jul 03 11:48:29.447217 ip-10-0-136-0 kubenswrapper[2579]: I0703 11:48:29.446600 2579 state_mem.go:107] "Deleted CPUSet assignment" podUID="75a2fbf5-528e-4910-b92b-e575c5ccf0f0" containerName="istio-proxy" Jul 03 11:48:29.447217 ip-10-0-136-0 kubenswrapper[2579]: I0703 11:48:29.446641 2579 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="df8a6db7-c558-410f-b596-51f30826429a" containerName="istio-proxy" Jul 03 11:48:29.447217 ip-10-0-136-0 kubenswrapper[2579]: I0703 11:48:29.446649 2579 state_mem.go:107] "Deleted CPUSet assignment" podUID="df8a6db7-c558-410f-b596-51f30826429a" containerName="istio-proxy" Jul 03 11:48:29.447217 ip-10-0-136-0 kubenswrapper[2579]: I0703 11:48:29.446659 2579 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="df8a6db7-c558-410f-b596-51f30826429a" containerName="istio-proxy" Jul 03 11:48:29.447217 ip-10-0-136-0 kubenswrapper[2579]: I0703 11:48:29.446668 2579 state_mem.go:107] "Deleted CPUSet assignment" podUID="df8a6db7-c558-410f-b596-51f30826429a" containerName="istio-proxy" Jul 03 11:48:29.447217 ip-10-0-136-0 kubenswrapper[2579]: I0703 11:48:29.446795 2579 memory_manager.go:368] "RemoveStaleState removing state" podUID="df8a6db7-c558-410f-b596-51f30826429a" containerName="istio-proxy" Jul 03 11:48:29.447217 ip-10-0-136-0 kubenswrapper[2579]: I0703 11:48:29.446806 2579 memory_manager.go:368] "RemoveStaleState removing state" podUID="df8a6db7-c558-410f-b596-51f30826429a" containerName="istio-proxy" Jul 03 11:48:29.457117 ip-10-0-136-0 kubenswrapper[2579]: I0703 11:48:29.457068 2579 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-shared-b-f7a699-openshift-default-ddcd795dc-k2fz7" Jul 03 11:48:29.463275 ip-10-0-136-0 kubenswrapper[2579]: I0703 11:48:29.461826 2579 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress\"/\"e2e-shared-b-f7a699-openshift-default-dockercfg-tzbth\"" Jul 03 11:48:29.463275 ip-10-0-136-0 kubenswrapper[2579]: I0703 11:48:29.462157 2579 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/e2e-shared-b-f7a699-openshift-default-ddcd795dc-k2fz7"] Jul 03 11:48:29.531236 ip-10-0-136-0 kubenswrapper[2579]: I0703 11:48:29.531161 2579 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/11f2fb16-2d82-4f7c-b197-08414de5a06e-workload-certs\") pod \"e2e-shared-b-f7a699-openshift-default-ddcd795dc-k2fz7\" (UID: \"11f2fb16-2d82-4f7c-b197-08414de5a06e\") " pod="openshift-ingress/e2e-shared-b-f7a699-openshift-default-ddcd795dc-k2fz7" Jul 03 11:48:29.531599 ip-10-0-136-0 kubenswrapper[2579]: I0703 11:48:29.531564 2579 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/11f2fb16-2d82-4f7c-b197-08414de5a06e-istio-envoy\") pod \"e2e-shared-b-f7a699-openshift-default-ddcd795dc-k2fz7\" (UID: \"11f2fb16-2d82-4f7c-b197-08414de5a06e\") " pod="openshift-ingress/e2e-shared-b-f7a699-openshift-default-ddcd795dc-k2fz7" Jul 03 11:48:29.531724 ip-10-0-136-0 kubenswrapper[2579]: I0703 11:48:29.531709 2579 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/11f2fb16-2d82-4f7c-b197-08414de5a06e-workload-socket\") pod \"e2e-shared-b-f7a699-openshift-default-ddcd795dc-k2fz7\" (UID: \"11f2fb16-2d82-4f7c-b197-08414de5a06e\") " pod="openshift-ingress/e2e-shared-b-f7a699-openshift-default-ddcd795dc-k2fz7" Jul 03 11:48:29.531944 ip-10-0-136-0 kubenswrapper[2579]: I0703 11:48:29.531926 2579 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/11f2fb16-2d82-4f7c-b197-08414de5a06e-istio-data\") pod \"e2e-shared-b-f7a699-openshift-default-ddcd795dc-k2fz7\" (UID: \"11f2fb16-2d82-4f7c-b197-08414de5a06e\") " pod="openshift-ingress/e2e-shared-b-f7a699-openshift-default-ddcd795dc-k2fz7" Jul 03 11:48:29.532101 ip-10-0-136-0 kubenswrapper[2579]: I0703 11:48:29.532085 2579 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/11f2fb16-2d82-4f7c-b197-08414de5a06e-istiod-ca-cert\") pod \"e2e-shared-b-f7a699-openshift-default-ddcd795dc-k2fz7\" (UID: \"11f2fb16-2d82-4f7c-b197-08414de5a06e\") " pod="openshift-ingress/e2e-shared-b-f7a699-openshift-default-ddcd795dc-k2fz7" Jul 03 11:48:29.536362 ip-10-0-136-0 kubenswrapper[2579]: I0703 11:48:29.536316 2579 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/11f2fb16-2d82-4f7c-b197-08414de5a06e-istio-podinfo\") pod \"e2e-shared-b-f7a699-openshift-default-ddcd795dc-k2fz7\" (UID: \"11f2fb16-2d82-4f7c-b197-08414de5a06e\") " pod="openshift-ingress/e2e-shared-b-f7a699-openshift-default-ddcd795dc-k2fz7" Jul 03 11:48:29.536645 ip-10-0-136-0 kubenswrapper[2579]: I0703 11:48:29.536625 2579 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-jb24j\" (UniqueName: \"kubernetes.io/projected/11f2fb16-2d82-4f7c-b197-08414de5a06e-kube-api-access-jb24j\") pod \"e2e-shared-b-f7a699-openshift-default-ddcd795dc-k2fz7\" (UID: \"11f2fb16-2d82-4f7c-b197-08414de5a06e\") " pod="openshift-ingress/e2e-shared-b-f7a699-openshift-default-ddcd795dc-k2fz7" Jul 03 11:48:29.536783 ip-10-0-136-0 kubenswrapper[2579]: I0703 11:48:29.536765 2579 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/11f2fb16-2d82-4f7c-b197-08414de5a06e-credential-socket\") pod \"e2e-shared-b-f7a699-openshift-default-ddcd795dc-k2fz7\" (UID: \"11f2fb16-2d82-4f7c-b197-08414de5a06e\") " pod="openshift-ingress/e2e-shared-b-f7a699-openshift-default-ddcd795dc-k2fz7" Jul 03 11:48:29.536908 ip-10-0-136-0 kubenswrapper[2579]: I0703 11:48:29.536894 2579 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/11f2fb16-2d82-4f7c-b197-08414de5a06e-istio-token\") pod \"e2e-shared-b-f7a699-openshift-default-ddcd795dc-k2fz7\" (UID: \"11f2fb16-2d82-4f7c-b197-08414de5a06e\") " pod="openshift-ingress/e2e-shared-b-f7a699-openshift-default-ddcd795dc-k2fz7" Jul 03 11:48:29.640571 ip-10-0-136-0 kubenswrapper[2579]: I0703 11:48:29.638221 2579 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/11f2fb16-2d82-4f7c-b197-08414de5a06e-istio-data\") pod \"e2e-shared-b-f7a699-openshift-default-ddcd795dc-k2fz7\" (UID: \"11f2fb16-2d82-4f7c-b197-08414de5a06e\") " pod="openshift-ingress/e2e-shared-b-f7a699-openshift-default-ddcd795dc-k2fz7" Jul 03 11:48:29.640571 ip-10-0-136-0 kubenswrapper[2579]: I0703 11:48:29.638313 2579 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/11f2fb16-2d82-4f7c-b197-08414de5a06e-istiod-ca-cert\") pod \"e2e-shared-b-f7a699-openshift-default-ddcd795dc-k2fz7\" (UID: \"11f2fb16-2d82-4f7c-b197-08414de5a06e\") " pod="openshift-ingress/e2e-shared-b-f7a699-openshift-default-ddcd795dc-k2fz7" Jul 03 11:48:29.640571 ip-10-0-136-0 kubenswrapper[2579]: I0703 11:48:29.638362 2579 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/11f2fb16-2d82-4f7c-b197-08414de5a06e-istio-podinfo\") pod \"e2e-shared-b-f7a699-openshift-default-ddcd795dc-k2fz7\" (UID: \"11f2fb16-2d82-4f7c-b197-08414de5a06e\") " pod="openshift-ingress/e2e-shared-b-f7a699-openshift-default-ddcd795dc-k2fz7" Jul 03 11:48:29.640571 ip-10-0-136-0 kubenswrapper[2579]: I0703 11:48:29.638390 2579 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-jb24j\" (UniqueName: \"kubernetes.io/projected/11f2fb16-2d82-4f7c-b197-08414de5a06e-kube-api-access-jb24j\") pod \"e2e-shared-b-f7a699-openshift-default-ddcd795dc-k2fz7\" (UID: \"11f2fb16-2d82-4f7c-b197-08414de5a06e\") " pod="openshift-ingress/e2e-shared-b-f7a699-openshift-default-ddcd795dc-k2fz7" Jul 03 11:48:29.640571 ip-10-0-136-0 kubenswrapper[2579]: I0703 11:48:29.638431 2579 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/11f2fb16-2d82-4f7c-b197-08414de5a06e-credential-socket\") pod \"e2e-shared-b-f7a699-openshift-default-ddcd795dc-k2fz7\" (UID: \"11f2fb16-2d82-4f7c-b197-08414de5a06e\") " pod="openshift-ingress/e2e-shared-b-f7a699-openshift-default-ddcd795dc-k2fz7" Jul 03 11:48:29.640571 ip-10-0-136-0 kubenswrapper[2579]: I0703 11:48:29.638470 2579 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/11f2fb16-2d82-4f7c-b197-08414de5a06e-istio-token\") pod \"e2e-shared-b-f7a699-openshift-default-ddcd795dc-k2fz7\" (UID: \"11f2fb16-2d82-4f7c-b197-08414de5a06e\") " pod="openshift-ingress/e2e-shared-b-f7a699-openshift-default-ddcd795dc-k2fz7" Jul 03 11:48:29.640571 ip-10-0-136-0 kubenswrapper[2579]: I0703 11:48:29.638533 2579 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/11f2fb16-2d82-4f7c-b197-08414de5a06e-workload-certs\") pod \"e2e-shared-b-f7a699-openshift-default-ddcd795dc-k2fz7\" (UID: \"11f2fb16-2d82-4f7c-b197-08414de5a06e\") " pod="openshift-ingress/e2e-shared-b-f7a699-openshift-default-ddcd795dc-k2fz7" Jul 03 11:48:29.640571 ip-10-0-136-0 kubenswrapper[2579]: I0703 11:48:29.638587 2579 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/11f2fb16-2d82-4f7c-b197-08414de5a06e-istio-envoy\") pod \"e2e-shared-b-f7a699-openshift-default-ddcd795dc-k2fz7\" (UID: \"11f2fb16-2d82-4f7c-b197-08414de5a06e\") " pod="openshift-ingress/e2e-shared-b-f7a699-openshift-default-ddcd795dc-k2fz7" Jul 03 11:48:29.640571 ip-10-0-136-0 kubenswrapper[2579]: I0703 11:48:29.638614 2579 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/11f2fb16-2d82-4f7c-b197-08414de5a06e-workload-socket\") pod \"e2e-shared-b-f7a699-openshift-default-ddcd795dc-k2fz7\" (UID: \"11f2fb16-2d82-4f7c-b197-08414de5a06e\") " pod="openshift-ingress/e2e-shared-b-f7a699-openshift-default-ddcd795dc-k2fz7" Jul 03 11:48:29.640571 ip-10-0-136-0 kubenswrapper[2579]: I0703 11:48:29.639150 2579 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/11f2fb16-2d82-4f7c-b197-08414de5a06e-workload-socket\") pod \"e2e-shared-b-f7a699-openshift-default-ddcd795dc-k2fz7\" (UID: \"11f2fb16-2d82-4f7c-b197-08414de5a06e\") " pod="openshift-ingress/e2e-shared-b-f7a699-openshift-default-ddcd795dc-k2fz7" Jul 03 11:48:29.640571 ip-10-0-136-0 kubenswrapper[2579]: I0703 11:48:29.639457 2579 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/11f2fb16-2d82-4f7c-b197-08414de5a06e-istio-data\") pod \"e2e-shared-b-f7a699-openshift-default-ddcd795dc-k2fz7\" (UID: \"11f2fb16-2d82-4f7c-b197-08414de5a06e\") " pod="openshift-ingress/e2e-shared-b-f7a699-openshift-default-ddcd795dc-k2fz7" Jul 03 11:48:29.640571 ip-10-0-136-0 kubenswrapper[2579]: I0703 11:48:29.640112 2579 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/11f2fb16-2d82-4f7c-b197-08414de5a06e-istiod-ca-cert\") pod \"e2e-shared-b-f7a699-openshift-default-ddcd795dc-k2fz7\" (UID: \"11f2fb16-2d82-4f7c-b197-08414de5a06e\") " pod="openshift-ingress/e2e-shared-b-f7a699-openshift-default-ddcd795dc-k2fz7" Jul 03 11:48:29.641443 ip-10-0-136-0 kubenswrapper[2579]: I0703 11:48:29.641411 2579 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/11f2fb16-2d82-4f7c-b197-08414de5a06e-credential-socket\") pod \"e2e-shared-b-f7a699-openshift-default-ddcd795dc-k2fz7\" (UID: \"11f2fb16-2d82-4f7c-b197-08414de5a06e\") " pod="openshift-ingress/e2e-shared-b-f7a699-openshift-default-ddcd795dc-k2fz7" Jul 03 11:48:29.642130 ip-10-0-136-0 kubenswrapper[2579]: I0703 11:48:29.641787 2579 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/11f2fb16-2d82-4f7c-b197-08414de5a06e-workload-certs\") pod \"e2e-shared-b-f7a699-openshift-default-ddcd795dc-k2fz7\" (UID: \"11f2fb16-2d82-4f7c-b197-08414de5a06e\") " pod="openshift-ingress/e2e-shared-b-f7a699-openshift-default-ddcd795dc-k2fz7" Jul 03 11:48:29.645293 ip-10-0-136-0 kubenswrapper[2579]: I0703 11:48:29.644554 2579 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/11f2fb16-2d82-4f7c-b197-08414de5a06e-istio-envoy\") pod \"e2e-shared-b-f7a699-openshift-default-ddcd795dc-k2fz7\" (UID: \"11f2fb16-2d82-4f7c-b197-08414de5a06e\") " pod="openshift-ingress/e2e-shared-b-f7a699-openshift-default-ddcd795dc-k2fz7" Jul 03 11:48:29.653220 ip-10-0-136-0 kubenswrapper[2579]: I0703 11:48:29.651717 2579 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-jb24j\" (UniqueName: \"kubernetes.io/projected/11f2fb16-2d82-4f7c-b197-08414de5a06e-kube-api-access-jb24j\") pod \"e2e-shared-b-f7a699-openshift-default-ddcd795dc-k2fz7\" (UID: \"11f2fb16-2d82-4f7c-b197-08414de5a06e\") " pod="openshift-ingress/e2e-shared-b-f7a699-openshift-default-ddcd795dc-k2fz7" Jul 03 11:48:29.654628 ip-10-0-136-0 kubenswrapper[2579]: I0703 11:48:29.653750 2579 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/11f2fb16-2d82-4f7c-b197-08414de5a06e-istio-token\") pod \"e2e-shared-b-f7a699-openshift-default-ddcd795dc-k2fz7\" (UID: \"11f2fb16-2d82-4f7c-b197-08414de5a06e\") " pod="openshift-ingress/e2e-shared-b-f7a699-openshift-default-ddcd795dc-k2fz7" Jul 03 11:48:29.659218 ip-10-0-136-0 kubenswrapper[2579]: I0703 11:48:29.657867 2579 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/11f2fb16-2d82-4f7c-b197-08414de5a06e-istio-podinfo\") pod \"e2e-shared-b-f7a699-openshift-default-ddcd795dc-k2fz7\" (UID: \"11f2fb16-2d82-4f7c-b197-08414de5a06e\") " pod="openshift-ingress/e2e-shared-b-f7a699-openshift-default-ddcd795dc-k2fz7" Jul 03 11:48:29.776408 ip-10-0-136-0 kubenswrapper[2579]: I0703 11:48:29.776364 2579 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-shared-b-f7a699-openshift-default-ddcd795dc-k2fz7" Jul 03 11:48:30.006531 ip-10-0-136-0 kubenswrapper[2579]: I0703 11:48:30.006492 2579 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/e2e-shared-b-f7a699-openshift-default-ddcd795dc-k2fz7"] Jul 03 11:48:30.008344 ip-10-0-136-0 kubenswrapper[2579]: W0703 11:48:30.008306 2579 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod11f2fb16_2d82_4f7c_b197_08414de5a06e.slice/crio-4d58bd160601c76d546e6b7655575b3fec4f7b61ae7282861ee7bbca0a3d9d49 WatchSource:0}: Error finding container 4d58bd160601c76d546e6b7655575b3fec4f7b61ae7282861ee7bbca0a3d9d49: Status 404 returned error can't find the container with id 4d58bd160601c76d546e6b7655575b3fec4f7b61ae7282861ee7bbca0a3d9d49 Jul 03 11:48:30.010716 ip-10-0-136-0 kubenswrapper[2579]: I0703 11:48:30.010665 2579 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"30892156Ki","pods":"250"} Jul 03 11:48:30.010826 ip-10-0-136-0 kubenswrapper[2579]: I0703 11:48:30.010794 2579 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"30892156Ki","pods":"250"} Jul 03 11:48:30.010882 ip-10-0-136-0 kubenswrapper[2579]: I0703 11:48:30.010840 2579 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"30892156Ki","pods":"250"} Jul 03 11:48:30.193859 ip-10-0-136-0 kubenswrapper[2579]: I0703 11:48:30.193799 2579 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-shared-b-f7a699-openshift-default-ddcd795dc-k2fz7" event={"ID":"11f2fb16-2d82-4f7c-b197-08414de5a06e","Type":"ContainerStarted","Data":"0cb7f2f9c50fc7b1447022eabcde82c7bc9d13765f4a5dcf7af0a7c7b782c995"} Jul 03 11:48:30.194077 ip-10-0-136-0 kubenswrapper[2579]: I0703 11:48:30.193877 2579 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-shared-b-f7a699-openshift-default-ddcd795dc-k2fz7" event={"ID":"11f2fb16-2d82-4f7c-b197-08414de5a06e","Type":"ContainerStarted","Data":"4d58bd160601c76d546e6b7655575b3fec4f7b61ae7282861ee7bbca0a3d9d49"} Jul 03 11:48:30.219011 ip-10-0-136-0 kubenswrapper[2579]: I0703 11:48:30.218952 2579 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-ingress/e2e-shared-b-f7a699-openshift-default-ddcd795dc-k2fz7" podStartSLOduration=1.218933678 podStartE2EDuration="1.218933678s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-03 11:48:29 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-03 11:48:30.217035613 +0000 UTC m=+2166.805339935" watchObservedRunningTime="2026-07-03 11:48:30.218933678 +0000 UTC m=+2166.807237998" Jul 03 11:48:30.777493 ip-10-0-136-0 kubenswrapper[2579]: I0703 11:48:30.777443 2579 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="openshift-ingress/e2e-shared-b-f7a699-openshift-default-ddcd795dc-k2fz7" Jul 03 11:48:31.784553 ip-10-0-136-0 kubenswrapper[2579]: I0703 11:48:31.784515 2579 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="openshift-ingress/e2e-shared-b-f7a699-openshift-default-ddcd795dc-k2fz7" Jul 03 11:48:32.207750 ip-10-0-136-0 kubenswrapper[2579]: I0703 11:48:32.207619 2579 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/e2e-shared-b-f7a699-openshift-default-ddcd795dc-k2fz7" Jul 03 11:48:32.209899 ip-10-0-136-0 kubenswrapper[2579]: I0703 11:48:32.209855 2579 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/e2e-shared-b-f7a699-openshift-default-ddcd795dc-k2fz7" Jul 03 11:48:36.238863 ip-10-0-136-0 kubenswrapper[2579]: I0703 11:48:36.238766 2579 generic.go:356] "Generic (PLEG): container finished" podID="f4cf29c7-812e-45ca-bf86-80a5cff794ac" containerID="693748e656f6194cc63b93d7d5c369f12d3d882ae6cd31d22e02cbf80c21791d" exitCode=0 Jul 03 11:48:36.238863 ip-10-0-136-0 kubenswrapper[2579]: I0703 11:48:36.238844 2579 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-shared-a-9c2097-openshift-default-6d687665ff-25rvz" event={"ID":"f4cf29c7-812e-45ca-bf86-80a5cff794ac","Type":"ContainerDied","Data":"693748e656f6194cc63b93d7d5c369f12d3d882ae6cd31d22e02cbf80c21791d"} Jul 03 11:48:36.239424 ip-10-0-136-0 kubenswrapper[2579]: I0703 11:48:36.239406 2579 scope.go:117] "RemoveContainer" containerID="693748e656f6194cc63b93d7d5c369f12d3d882ae6cd31d22e02cbf80c21791d" Jul 03 11:48:36.240848 ip-10-0-136-0 kubenswrapper[2579]: I0703 11:48:36.240813 2579 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"30892156Ki","pods":"250"} Jul 03 11:48:36.240973 ip-10-0-136-0 kubenswrapper[2579]: I0703 11:48:36.240916 2579 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"30892156Ki","pods":"250"} Jul 03 11:48:36.240973 ip-10-0-136-0 kubenswrapper[2579]: I0703 11:48:36.240965 2579 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"30892156Ki","pods":"250"} Jul 03 11:48:37.255811 ip-10-0-136-0 kubenswrapper[2579]: I0703 11:48:37.255763 2579 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-shared-a-9c2097-openshift-default-6d687665ff-25rvz" event={"ID":"f4cf29c7-812e-45ca-bf86-80a5cff794ac","Type":"ContainerStarted","Data":"b8845547f8938f6cc98ff7a4f84abdd2bd1a47f066ae1c610dad12c3f44c2cc6"} Jul 03 11:48:38.127650 ip-10-0-136-0 kubenswrapper[2579]: I0703 11:48:38.127598 2579 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="openshift-ingress/e2e-shared-a-9c2097-openshift-default-6d687665ff-25rvz" Jul 03 11:48:38.129070 ip-10-0-136-0 kubenswrapper[2579]: I0703 11:48:38.129034 2579 patch_prober.go:28] interesting pod/e2e-shared-a-9c2097-openshift-default-6d687665ff-25rvz container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.133.0.73:15021/healthz/ready\": dial tcp 10.133.0.73:15021: connect: connection refused" start-of-body= Jul 03 11:48:38.129254 ip-10-0-136-0 kubenswrapper[2579]: I0703 11:48:38.129093 2579 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-shared-a-9c2097-openshift-default-6d687665ff-25rvz" podUID="f4cf29c7-812e-45ca-bf86-80a5cff794ac" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.133.0.73:15021/healthz/ready\": dial tcp 10.133.0.73:15021: connect: connection refused" Jul 03 11:48:38.140922 ip-10-0-136-0 kubenswrapper[2579]: I0703 11:48:38.140860 2579 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/e2e-shared-a-9c2097-openshift-default-6d687665ff-25rvz" Jul 03 11:48:39.127751 ip-10-0-136-0 kubenswrapper[2579]: I0703 11:48:39.127715 2579 patch_prober.go:28] interesting pod/e2e-shared-a-9c2097-openshift-default-6d687665ff-25rvz container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.133.0.73:15021/healthz/ready\": dial tcp 10.133.0.73:15021: connect: connection refused" start-of-body= Jul 03 11:48:39.128237 ip-10-0-136-0 kubenswrapper[2579]: I0703 11:48:39.127774 2579 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-shared-a-9c2097-openshift-default-6d687665ff-25rvz" podUID="f4cf29c7-812e-45ca-bf86-80a5cff794ac" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.133.0.73:15021/healthz/ready\": dial tcp 10.133.0.73:15021: connect: connection refused" Jul 03 11:48:40.128370 ip-10-0-136-0 kubenswrapper[2579]: I0703 11:48:40.128320 2579 patch_prober.go:28] interesting pod/e2e-shared-a-9c2097-openshift-default-6d687665ff-25rvz container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.133.0.73:15021/healthz/ready\": dial tcp 10.133.0.73:15021: connect: connection refused" start-of-body= Jul 03 11:48:40.128823 ip-10-0-136-0 kubenswrapper[2579]: I0703 11:48:40.128409 2579 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-shared-a-9c2097-openshift-default-6d687665ff-25rvz" podUID="f4cf29c7-812e-45ca-bf86-80a5cff794ac" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.133.0.73:15021/healthz/ready\": dial tcp 10.133.0.73:15021: connect: connection refused" Jul 03 11:48:41.127531 ip-10-0-136-0 kubenswrapper[2579]: I0703 11:48:41.127485 2579 patch_prober.go:28] interesting pod/e2e-shared-a-9c2097-openshift-default-6d687665ff-25rvz container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.133.0.73:15021/healthz/ready\": dial tcp 10.133.0.73:15021: connect: connection refused" start-of-body= Jul 03 11:48:41.127734 ip-10-0-136-0 kubenswrapper[2579]: I0703 11:48:41.127551 2579 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-shared-a-9c2097-openshift-default-6d687665ff-25rvz" podUID="f4cf29c7-812e-45ca-bf86-80a5cff794ac" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.133.0.73:15021/healthz/ready\": dial tcp 10.133.0.73:15021: connect: connection refused" Jul 03 11:48:42.132252 ip-10-0-136-0 kubenswrapper[2579]: I0703 11:48:42.132181 2579 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="openshift-ingress/e2e-shared-a-9c2097-openshift-default-6d687665ff-25rvz" Jul 03 11:48:42.133723 ip-10-0-136-0 kubenswrapper[2579]: I0703 11:48:42.133695 2579 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/e2e-shared-a-9c2097-openshift-default-6d687665ff-25rvz" Jul 03 11:48:48.249387 ip-10-0-136-0 kubenswrapper[2579]: I0703 11:48:48.249336 2579 patch_prober.go:28] interesting pod/e2e-shared-b-f7a699-openshift-default-ddcd795dc-k2fz7 container/istio-proxy namespace/openshift-ingress: Readiness probe status=failure output="Get \"http://10.133.0.74:15021/healthz/ready\": context deadline exceeded (Client.Timeout exceeded while awaiting headers)" start-of-body= Jul 03 11:48:48.249888 ip-10-0-136-0 kubenswrapper[2579]: I0703 11:48:48.249420 2579 prober.go:120] "Probe failed" probeType="Readiness" pod="openshift-ingress/e2e-shared-b-f7a699-openshift-default-ddcd795dc-k2fz7" podUID="11f2fb16-2d82-4f7c-b197-08414de5a06e" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.133.0.74:15021/healthz/ready\": context deadline exceeded (Client.Timeout exceeded while awaiting headers)" Jul 03 11:48:51.348779 ip-10-0-136-0 kubenswrapper[2579]: I0703 11:48:51.348741 2579 generic.go:356] "Generic (PLEG): container finished" podID="11f2fb16-2d82-4f7c-b197-08414de5a06e" containerID="0cb7f2f9c50fc7b1447022eabcde82c7bc9d13765f4a5dcf7af0a7c7b782c995" exitCode=0 Jul 03 11:48:51.349280 ip-10-0-136-0 kubenswrapper[2579]: I0703 11:48:51.348828 2579 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-shared-b-f7a699-openshift-default-ddcd795dc-k2fz7" event={"ID":"11f2fb16-2d82-4f7c-b197-08414de5a06e","Type":"ContainerDied","Data":"0cb7f2f9c50fc7b1447022eabcde82c7bc9d13765f4a5dcf7af0a7c7b782c995"} Jul 03 11:48:51.349480 ip-10-0-136-0 kubenswrapper[2579]: I0703 11:48:51.349461 2579 scope.go:117] "RemoveContainer" containerID="0cb7f2f9c50fc7b1447022eabcde82c7bc9d13765f4a5dcf7af0a7c7b782c995" Jul 03 11:48:51.350624 ip-10-0-136-0 kubenswrapper[2579]: I0703 11:48:51.350592 2579 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"30892156Ki","pods":"250"} Jul 03 11:48:51.350742 ip-10-0-136-0 kubenswrapper[2579]: I0703 11:48:51.350666 2579 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"30892156Ki","pods":"250"} Jul 03 11:48:51.350742 ip-10-0-136-0 kubenswrapper[2579]: I0703 11:48:51.350705 2579 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"30892156Ki","pods":"250"} Jul 03 11:48:52.362516 ip-10-0-136-0 kubenswrapper[2579]: I0703 11:48:52.362473 2579 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-shared-b-f7a699-openshift-default-ddcd795dc-k2fz7" event={"ID":"11f2fb16-2d82-4f7c-b197-08414de5a06e","Type":"ContainerStarted","Data":"1e111a048a9451f4c306399f6a3b1775c9521f88cf94d6dab9de833d0faa04a3"} Jul 03 11:48:52.777168 ip-10-0-136-0 kubenswrapper[2579]: I0703 11:48:52.777116 2579 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="openshift-ingress/e2e-shared-b-f7a699-openshift-default-ddcd795dc-k2fz7" Jul 03 11:48:52.777553 ip-10-0-136-0 kubenswrapper[2579]: I0703 11:48:52.777522 2579 patch_prober.go:28] interesting pod/e2e-shared-b-f7a699-openshift-default-ddcd795dc-k2fz7 container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.133.0.74:15021/healthz/ready\": dial tcp 10.133.0.74:15021: connect: connection refused" start-of-body= Jul 03 11:48:52.777619 ip-10-0-136-0 kubenswrapper[2579]: I0703 11:48:52.777599 2579 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-shared-b-f7a699-openshift-default-ddcd795dc-k2fz7" podUID="11f2fb16-2d82-4f7c-b197-08414de5a06e" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.133.0.74:15021/healthz/ready\": dial tcp 10.133.0.74:15021: connect: connection refused" Jul 03 11:48:53.777426 ip-10-0-136-0 kubenswrapper[2579]: I0703 11:48:53.777391 2579 patch_prober.go:28] interesting pod/e2e-shared-b-f7a699-openshift-default-ddcd795dc-k2fz7 container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.133.0.74:15021/healthz/ready\": dial tcp 10.133.0.74:15021: connect: connection refused" start-of-body= Jul 03 11:48:53.777861 ip-10-0-136-0 kubenswrapper[2579]: I0703 11:48:53.777448 2579 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-shared-b-f7a699-openshift-default-ddcd795dc-k2fz7" podUID="11f2fb16-2d82-4f7c-b197-08414de5a06e" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.133.0.74:15021/healthz/ready\": dial tcp 10.133.0.74:15021: connect: connection refused" Jul 03 11:48:54.776890 ip-10-0-136-0 kubenswrapper[2579]: I0703 11:48:54.776851 2579 patch_prober.go:28] interesting pod/e2e-shared-b-f7a699-openshift-default-ddcd795dc-k2fz7 container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.133.0.74:15021/healthz/ready\": dial tcp 10.133.0.74:15021: connect: connection refused" start-of-body= Jul 03 11:48:54.777086 ip-10-0-136-0 kubenswrapper[2579]: I0703 11:48:54.776919 2579 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-shared-b-f7a699-openshift-default-ddcd795dc-k2fz7" podUID="11f2fb16-2d82-4f7c-b197-08414de5a06e" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.133.0.74:15021/healthz/ready\": dial tcp 10.133.0.74:15021: connect: connection refused" Jul 03 11:48:55.777608 ip-10-0-136-0 kubenswrapper[2579]: I0703 11:48:55.777564 2579 patch_prober.go:28] interesting pod/e2e-shared-b-f7a699-openshift-default-ddcd795dc-k2fz7 container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.133.0.74:15021/healthz/ready\": dial tcp 10.133.0.74:15021: connect: connection refused" start-of-body= Jul 03 11:48:55.778027 ip-10-0-136-0 kubenswrapper[2579]: I0703 11:48:55.777626 2579 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-shared-b-f7a699-openshift-default-ddcd795dc-k2fz7" podUID="11f2fb16-2d82-4f7c-b197-08414de5a06e" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.133.0.74:15021/healthz/ready\": dial tcp 10.133.0.74:15021: connect: connection refused" Jul 03 11:48:56.781242 ip-10-0-136-0 kubenswrapper[2579]: I0703 11:48:56.781204 2579 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="openshift-ingress/e2e-shared-b-f7a699-openshift-default-ddcd795dc-k2fz7" Jul 03 11:48:56.781636 ip-10-0-136-0 kubenswrapper[2579]: I0703 11:48:56.781591 2579 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/e2e-shared-b-f7a699-openshift-default-ddcd795dc-k2fz7" Jul 03 11:48:56.782889 ip-10-0-136-0 kubenswrapper[2579]: I0703 11:48:56.782859 2579 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/e2e-shared-b-f7a699-openshift-default-ddcd795dc-k2fz7" Jul 03 11:51:32.613632 ip-10-0-136-0 kubenswrapper[2579]: I0703 11:51:32.613530 2579 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["ai-tenant-e2e-shared-a-9c2097/sub-test-model-9c2097-kserve-665fdbffff-9xbs4"] Jul 03 11:51:32.617307 ip-10-0-136-0 kubenswrapper[2579]: I0703 11:51:32.617283 2579 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="ai-tenant-e2e-shared-a-9c2097/sub-test-model-9c2097-kserve-665fdbffff-9xbs4" Jul 03 11:51:32.620809 ip-10-0-136-0 kubenswrapper[2579]: I0703 11:51:32.620778 2579 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"ai-tenant-e2e-shared-a-9c2097\"/\"default-dockercfg-8jfcx\"" Jul 03 11:51:32.621475 ip-10-0-136-0 kubenswrapper[2579]: I0703 11:51:32.621451 2579 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"ai-tenant-e2e-shared-a-9c2097\"/\"kube-root-ca.crt\"" Jul 03 11:51:32.621584 ip-10-0-136-0 kubenswrapper[2579]: I0703 11:51:32.621511 2579 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"ai-tenant-e2e-shared-a-9c2097\"/\"openshift-service-ca.crt\"" Jul 03 11:51:32.621748 ip-10-0-136-0 kubenswrapper[2579]: I0703 11:51:32.621733 2579 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"ai-tenant-e2e-shared-a-9c2097\"/\"sub-test-model-9c2097-kserve-self-signed-certs\"" Jul 03 11:51:32.631275 ip-10-0-136-0 kubenswrapper[2579]: I0703 11:51:32.631233 2579 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["ai-tenant-e2e-shared-a-9c2097/sub-test-model-9c2097-kserve-665fdbffff-9xbs4"] Jul 03 11:51:32.690000 ip-10-0-136-0 kubenswrapper[2579]: I0703 11:51:32.689952 2579 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-cktvf\" (UniqueName: \"kubernetes.io/projected/cdcdda17-90dd-43c7-bb78-8c681de85357-kube-api-access-cktvf\") pod \"sub-test-model-9c2097-kserve-665fdbffff-9xbs4\" (UID: \"cdcdda17-90dd-43c7-bb78-8c681de85357\") " pod="ai-tenant-e2e-shared-a-9c2097/sub-test-model-9c2097-kserve-665fdbffff-9xbs4" Jul 03 11:51:32.690216 ip-10-0-136-0 kubenswrapper[2579]: I0703 11:51:32.690013 2579 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/cdcdda17-90dd-43c7-bb78-8c681de85357-dshm\") pod \"sub-test-model-9c2097-kserve-665fdbffff-9xbs4\" (UID: \"cdcdda17-90dd-43c7-bb78-8c681de85357\") " pod="ai-tenant-e2e-shared-a-9c2097/sub-test-model-9c2097-kserve-665fdbffff-9xbs4" Jul 03 11:51:32.690216 ip-10-0-136-0 kubenswrapper[2579]: I0703 11:51:32.690052 2579 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/cdcdda17-90dd-43c7-bb78-8c681de85357-model-cache\") pod \"sub-test-model-9c2097-kserve-665fdbffff-9xbs4\" (UID: \"cdcdda17-90dd-43c7-bb78-8c681de85357\") " pod="ai-tenant-e2e-shared-a-9c2097/sub-test-model-9c2097-kserve-665fdbffff-9xbs4" Jul 03 11:51:32.690216 ip-10-0-136-0 kubenswrapper[2579]: I0703 11:51:32.690145 2579 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/cdcdda17-90dd-43c7-bb78-8c681de85357-home\") pod \"sub-test-model-9c2097-kserve-665fdbffff-9xbs4\" (UID: \"cdcdda17-90dd-43c7-bb78-8c681de85357\") " pod="ai-tenant-e2e-shared-a-9c2097/sub-test-model-9c2097-kserve-665fdbffff-9xbs4" Jul 03 11:51:32.690216 ip-10-0-136-0 kubenswrapper[2579]: I0703 11:51:32.690203 2579 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/cdcdda17-90dd-43c7-bb78-8c681de85357-tmp-dir\") pod \"sub-test-model-9c2097-kserve-665fdbffff-9xbs4\" (UID: \"cdcdda17-90dd-43c7-bb78-8c681de85357\") " pod="ai-tenant-e2e-shared-a-9c2097/sub-test-model-9c2097-kserve-665fdbffff-9xbs4" Jul 03 11:51:32.690403 ip-10-0-136-0 kubenswrapper[2579]: I0703 11:51:32.690225 2579 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/cdcdda17-90dd-43c7-bb78-8c681de85357-tls-certs\") pod \"sub-test-model-9c2097-kserve-665fdbffff-9xbs4\" (UID: \"cdcdda17-90dd-43c7-bb78-8c681de85357\") " pod="ai-tenant-e2e-shared-a-9c2097/sub-test-model-9c2097-kserve-665fdbffff-9xbs4" Jul 03 11:51:32.791041 ip-10-0-136-0 kubenswrapper[2579]: I0703 11:51:32.790985 2579 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/cdcdda17-90dd-43c7-bb78-8c681de85357-tmp-dir\") pod \"sub-test-model-9c2097-kserve-665fdbffff-9xbs4\" (UID: \"cdcdda17-90dd-43c7-bb78-8c681de85357\") " pod="ai-tenant-e2e-shared-a-9c2097/sub-test-model-9c2097-kserve-665fdbffff-9xbs4" Jul 03 11:51:32.791284 ip-10-0-136-0 kubenswrapper[2579]: I0703 11:51:32.791063 2579 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/cdcdda17-90dd-43c7-bb78-8c681de85357-tls-certs\") pod \"sub-test-model-9c2097-kserve-665fdbffff-9xbs4\" (UID: \"cdcdda17-90dd-43c7-bb78-8c681de85357\") " pod="ai-tenant-e2e-shared-a-9c2097/sub-test-model-9c2097-kserve-665fdbffff-9xbs4" Jul 03 11:51:32.791284 ip-10-0-136-0 kubenswrapper[2579]: I0703 11:51:32.791150 2579 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-cktvf\" (UniqueName: \"kubernetes.io/projected/cdcdda17-90dd-43c7-bb78-8c681de85357-kube-api-access-cktvf\") pod \"sub-test-model-9c2097-kserve-665fdbffff-9xbs4\" (UID: \"cdcdda17-90dd-43c7-bb78-8c681de85357\") " pod="ai-tenant-e2e-shared-a-9c2097/sub-test-model-9c2097-kserve-665fdbffff-9xbs4" Jul 03 11:51:32.791284 ip-10-0-136-0 kubenswrapper[2579]: I0703 11:51:32.791249 2579 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/cdcdda17-90dd-43c7-bb78-8c681de85357-dshm\") pod \"sub-test-model-9c2097-kserve-665fdbffff-9xbs4\" (UID: \"cdcdda17-90dd-43c7-bb78-8c681de85357\") " pod="ai-tenant-e2e-shared-a-9c2097/sub-test-model-9c2097-kserve-665fdbffff-9xbs4" Jul 03 11:51:32.791284 ip-10-0-136-0 kubenswrapper[2579]: I0703 11:51:32.791276 2579 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/cdcdda17-90dd-43c7-bb78-8c681de85357-model-cache\") pod \"sub-test-model-9c2097-kserve-665fdbffff-9xbs4\" (UID: \"cdcdda17-90dd-43c7-bb78-8c681de85357\") " pod="ai-tenant-e2e-shared-a-9c2097/sub-test-model-9c2097-kserve-665fdbffff-9xbs4" Jul 03 11:51:32.791491 ip-10-0-136-0 kubenswrapper[2579]: I0703 11:51:32.791343 2579 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/cdcdda17-90dd-43c7-bb78-8c681de85357-home\") pod \"sub-test-model-9c2097-kserve-665fdbffff-9xbs4\" (UID: \"cdcdda17-90dd-43c7-bb78-8c681de85357\") " pod="ai-tenant-e2e-shared-a-9c2097/sub-test-model-9c2097-kserve-665fdbffff-9xbs4" Jul 03 11:51:32.791617 ip-10-0-136-0 kubenswrapper[2579]: I0703 11:51:32.791585 2579 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/cdcdda17-90dd-43c7-bb78-8c681de85357-tmp-dir\") pod \"sub-test-model-9c2097-kserve-665fdbffff-9xbs4\" (UID: \"cdcdda17-90dd-43c7-bb78-8c681de85357\") " pod="ai-tenant-e2e-shared-a-9c2097/sub-test-model-9c2097-kserve-665fdbffff-9xbs4" Jul 03 11:51:32.791796 ip-10-0-136-0 kubenswrapper[2579]: I0703 11:51:32.791771 2579 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/cdcdda17-90dd-43c7-bb78-8c681de85357-home\") pod \"sub-test-model-9c2097-kserve-665fdbffff-9xbs4\" (UID: \"cdcdda17-90dd-43c7-bb78-8c681de85357\") " pod="ai-tenant-e2e-shared-a-9c2097/sub-test-model-9c2097-kserve-665fdbffff-9xbs4" Jul 03 11:51:32.791861 ip-10-0-136-0 kubenswrapper[2579]: I0703 11:51:32.791827 2579 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/cdcdda17-90dd-43c7-bb78-8c681de85357-model-cache\") pod \"sub-test-model-9c2097-kserve-665fdbffff-9xbs4\" (UID: \"cdcdda17-90dd-43c7-bb78-8c681de85357\") " pod="ai-tenant-e2e-shared-a-9c2097/sub-test-model-9c2097-kserve-665fdbffff-9xbs4" Jul 03 11:51:32.793972 ip-10-0-136-0 kubenswrapper[2579]: I0703 11:51:32.793938 2579 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/cdcdda17-90dd-43c7-bb78-8c681de85357-dshm\") pod \"sub-test-model-9c2097-kserve-665fdbffff-9xbs4\" (UID: \"cdcdda17-90dd-43c7-bb78-8c681de85357\") " pod="ai-tenant-e2e-shared-a-9c2097/sub-test-model-9c2097-kserve-665fdbffff-9xbs4" Jul 03 11:51:32.794167 ip-10-0-136-0 kubenswrapper[2579]: I0703 11:51:32.794144 2579 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/cdcdda17-90dd-43c7-bb78-8c681de85357-tls-certs\") pod \"sub-test-model-9c2097-kserve-665fdbffff-9xbs4\" (UID: \"cdcdda17-90dd-43c7-bb78-8c681de85357\") " pod="ai-tenant-e2e-shared-a-9c2097/sub-test-model-9c2097-kserve-665fdbffff-9xbs4" Jul 03 11:51:32.800807 ip-10-0-136-0 kubenswrapper[2579]: I0703 11:51:32.800769 2579 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-cktvf\" (UniqueName: \"kubernetes.io/projected/cdcdda17-90dd-43c7-bb78-8c681de85357-kube-api-access-cktvf\") pod \"sub-test-model-9c2097-kserve-665fdbffff-9xbs4\" (UID: \"cdcdda17-90dd-43c7-bb78-8c681de85357\") " pod="ai-tenant-e2e-shared-a-9c2097/sub-test-model-9c2097-kserve-665fdbffff-9xbs4" Jul 03 11:51:32.928821 ip-10-0-136-0 kubenswrapper[2579]: I0703 11:51:32.928708 2579 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="ai-tenant-e2e-shared-a-9c2097/sub-test-model-9c2097-kserve-665fdbffff-9xbs4" Jul 03 11:51:33.107047 ip-10-0-136-0 kubenswrapper[2579]: I0703 11:51:33.107009 2579 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["ai-tenant-e2e-shared-a-9c2097/sub-test-model-9c2097-kserve-665fdbffff-9xbs4"] Jul 03 11:51:33.107784 ip-10-0-136-0 kubenswrapper[2579]: W0703 11:51:33.107753 2579 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-besteffort.slice/kubepods-besteffort-podcdcdda17_90dd_43c7_bb78_8c681de85357.slice/crio-218330f8752ebdd2d48d81fe6f4a7ff2b45e1bddf1a771f87487d4bb87c1919c WatchSource:0}: Error finding container 218330f8752ebdd2d48d81fe6f4a7ff2b45e1bddf1a771f87487d4bb87c1919c: Status 404 returned error can't find the container with id 218330f8752ebdd2d48d81fe6f4a7ff2b45e1bddf1a771f87487d4bb87c1919c Jul 03 11:51:33.385892 ip-10-0-136-0 kubenswrapper[2579]: I0703 11:51:33.385851 2579 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="ai-tenant-e2e-shared-a-9c2097/sub-test-model-9c2097-kserve-665fdbffff-9xbs4" event={"ID":"cdcdda17-90dd-43c7-bb78-8c681de85357","Type":"ContainerStarted","Data":"df7fdd0c994606976be798bfc74d15110f267ef1fa1c81f7ce8098fd326fc593"} Jul 03 11:51:33.385892 ip-10-0-136-0 kubenswrapper[2579]: I0703 11:51:33.385900 2579 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="ai-tenant-e2e-shared-a-9c2097/sub-test-model-9c2097-kserve-665fdbffff-9xbs4" event={"ID":"cdcdda17-90dd-43c7-bb78-8c681de85357","Type":"ContainerStarted","Data":"218330f8752ebdd2d48d81fe6f4a7ff2b45e1bddf1a771f87487d4bb87c1919c"} Jul 03 11:51:33.439613 ip-10-0-136-0 kubenswrapper[2579]: I0703 11:51:33.439470 2579 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="ai-tenant-e2e-shared-a-9c2097/sub-test-model-9c2097-kserve-665fdbffff-9xbs4" podStartSLOduration=1.439452409 podStartE2EDuration="1.439452409s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-03 11:51:32 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-03 11:51:33.437742438 +0000 UTC m=+2350.026046760" watchObservedRunningTime="2026-07-03 11:51:33.439452409 +0000 UTC m=+2350.027756728" Jul 03 11:51:42.929259 ip-10-0-136-0 kubenswrapper[2579]: I0703 11:51:42.929173 2579 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="ai-tenant-e2e-shared-a-9c2097/sub-test-model-9c2097-kserve-665fdbffff-9xbs4" Jul 03 11:51:42.929755 ip-10-0-136-0 kubenswrapper[2579]: I0703 11:51:42.929287 2579 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="ai-tenant-e2e-shared-a-9c2097/sub-test-model-9c2097-kserve-665fdbffff-9xbs4" Jul 03 11:51:42.945827 ip-10-0-136-0 kubenswrapper[2579]: I0703 11:51:42.945792 2579 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="ai-tenant-e2e-shared-a-9c2097/sub-test-model-9c2097-kserve-665fdbffff-9xbs4" Jul 03 11:51:43.466499 ip-10-0-136-0 kubenswrapper[2579]: I0703 11:51:43.466464 2579 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="ai-tenant-e2e-shared-a-9c2097/sub-test-model-9c2097-kserve-665fdbffff-9xbs4" Jul 03 11:52:42.120418 ip-10-0-136-0 kubenswrapper[2579]: I0703 11:52:42.120377 2579 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["ai-tenant-e2e-shared-a-9c2097/sub-test-model-9c2097-kserve-665fdbffff-9xbs4"] Jul 03 11:52:57.142642 ip-10-0-136-0 kubenswrapper[2579]: I0703 11:52:57.142535 2579 kuberuntime_container.go:872] "Killing container with a grace period" pod="ai-tenant-e2e-shared-a-9c2097/sub-test-model-9c2097-kserve-665fdbffff-9xbs4" podUID="cdcdda17-90dd-43c7-bb78-8c681de85357" containerName="main" containerID="cri-o://df7fdd0c994606976be798bfc74d15110f267ef1fa1c81f7ce8098fd326fc593" gracePeriod=45 Jul 03 11:52:57.418502 ip-10-0-136-0 kubenswrapper[2579]: I0703 11:52:57.418468 2579 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="ai-tenant-e2e-shared-a-9c2097/sub-test-model-9c2097-kserve-665fdbffff-9xbs4" Jul 03 11:52:57.465282 ip-10-0-136-0 kubenswrapper[2579]: I0703 11:52:57.465238 2579 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-cktvf\" (UniqueName: \"kubernetes.io/projected/cdcdda17-90dd-43c7-bb78-8c681de85357-kube-api-access-cktvf\") pod \"cdcdda17-90dd-43c7-bb78-8c681de85357\" (UID: \"cdcdda17-90dd-43c7-bb78-8c681de85357\") " Jul 03 11:52:57.465503 ip-10-0-136-0 kubenswrapper[2579]: I0703 11:52:57.465323 2579 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/cdcdda17-90dd-43c7-bb78-8c681de85357-model-cache\") pod \"cdcdda17-90dd-43c7-bb78-8c681de85357\" (UID: \"cdcdda17-90dd-43c7-bb78-8c681de85357\") " Jul 03 11:52:57.465503 ip-10-0-136-0 kubenswrapper[2579]: I0703 11:52:57.465400 2579 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/cdcdda17-90dd-43c7-bb78-8c681de85357-home\") pod \"cdcdda17-90dd-43c7-bb78-8c681de85357\" (UID: \"cdcdda17-90dd-43c7-bb78-8c681de85357\") " Jul 03 11:52:57.465503 ip-10-0-136-0 kubenswrapper[2579]: I0703 11:52:57.465418 2579 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/cdcdda17-90dd-43c7-bb78-8c681de85357-tls-certs\") pod \"cdcdda17-90dd-43c7-bb78-8c681de85357\" (UID: \"cdcdda17-90dd-43c7-bb78-8c681de85357\") " Jul 03 11:52:57.465503 ip-10-0-136-0 kubenswrapper[2579]: I0703 11:52:57.465437 2579 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/cdcdda17-90dd-43c7-bb78-8c681de85357-dshm\") pod \"cdcdda17-90dd-43c7-bb78-8c681de85357\" (UID: \"cdcdda17-90dd-43c7-bb78-8c681de85357\") " Jul 03 11:52:57.465503 ip-10-0-136-0 kubenswrapper[2579]: I0703 11:52:57.465471 2579 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/cdcdda17-90dd-43c7-bb78-8c681de85357-tmp-dir\") pod \"cdcdda17-90dd-43c7-bb78-8c681de85357\" (UID: \"cdcdda17-90dd-43c7-bb78-8c681de85357\") " Jul 03 11:52:57.465878 ip-10-0-136-0 kubenswrapper[2579]: I0703 11:52:57.465798 2579 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/cdcdda17-90dd-43c7-bb78-8c681de85357-home" (OuterVolumeSpecName: "home") pod "cdcdda17-90dd-43c7-bb78-8c681de85357" (UID: "cdcdda17-90dd-43c7-bb78-8c681de85357"). InnerVolumeSpecName "home". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Jul 03 11:52:57.466015 ip-10-0-136-0 kubenswrapper[2579]: I0703 11:52:57.465907 2579 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/cdcdda17-90dd-43c7-bb78-8c681de85357-tmp-dir" (OuterVolumeSpecName: "tmp-dir") pod "cdcdda17-90dd-43c7-bb78-8c681de85357" (UID: "cdcdda17-90dd-43c7-bb78-8c681de85357"). InnerVolumeSpecName "tmp-dir". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Jul 03 11:52:57.466015 ip-10-0-136-0 kubenswrapper[2579]: I0703 11:52:57.465972 2579 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/cdcdda17-90dd-43c7-bb78-8c681de85357-model-cache" (OuterVolumeSpecName: "model-cache") pod "cdcdda17-90dd-43c7-bb78-8c681de85357" (UID: "cdcdda17-90dd-43c7-bb78-8c681de85357"). InnerVolumeSpecName "model-cache". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Jul 03 11:52:57.468435 ip-10-0-136-0 kubenswrapper[2579]: I0703 11:52:57.468394 2579 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/cdcdda17-90dd-43c7-bb78-8c681de85357-tls-certs" (OuterVolumeSpecName: "tls-certs") pod "cdcdda17-90dd-43c7-bb78-8c681de85357" (UID: "cdcdda17-90dd-43c7-bb78-8c681de85357"). InnerVolumeSpecName "tls-certs". PluginName "kubernetes.io/secret", VolumeGIDValue "" Jul 03 11:52:57.468603 ip-10-0-136-0 kubenswrapper[2579]: I0703 11:52:57.468439 2579 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/cdcdda17-90dd-43c7-bb78-8c681de85357-dshm" (OuterVolumeSpecName: "dshm") pod "cdcdda17-90dd-43c7-bb78-8c681de85357" (UID: "cdcdda17-90dd-43c7-bb78-8c681de85357"). InnerVolumeSpecName "dshm". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Jul 03 11:52:57.468603 ip-10-0-136-0 kubenswrapper[2579]: I0703 11:52:57.468321 2579 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/cdcdda17-90dd-43c7-bb78-8c681de85357-kube-api-access-cktvf" (OuterVolumeSpecName: "kube-api-access-cktvf") pod "cdcdda17-90dd-43c7-bb78-8c681de85357" (UID: "cdcdda17-90dd-43c7-bb78-8c681de85357"). InnerVolumeSpecName "kube-api-access-cktvf". PluginName "kubernetes.io/projected", VolumeGIDValue "" Jul 03 11:52:57.567581 ip-10-0-136-0 kubenswrapper[2579]: I0703 11:52:57.567519 2579 reconciler_common.go:299] "Volume detached for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/cdcdda17-90dd-43c7-bb78-8c681de85357-home\") on node \"ip-10-0-136-0.ec2.internal\" DevicePath \"\"" Jul 03 11:52:57.567581 ip-10-0-136-0 kubenswrapper[2579]: I0703 11:52:57.567570 2579 reconciler_common.go:299] "Volume detached for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/cdcdda17-90dd-43c7-bb78-8c681de85357-tls-certs\") on node \"ip-10-0-136-0.ec2.internal\" DevicePath \"\"" Jul 03 11:52:57.567581 ip-10-0-136-0 kubenswrapper[2579]: I0703 11:52:57.567582 2579 reconciler_common.go:299] "Volume detached for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/cdcdda17-90dd-43c7-bb78-8c681de85357-dshm\") on node \"ip-10-0-136-0.ec2.internal\" DevicePath \"\"" Jul 03 11:52:57.567581 ip-10-0-136-0 kubenswrapper[2579]: I0703 11:52:57.567590 2579 reconciler_common.go:299] "Volume detached for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/cdcdda17-90dd-43c7-bb78-8c681de85357-tmp-dir\") on node \"ip-10-0-136-0.ec2.internal\" DevicePath \"\"" Jul 03 11:52:57.567581 ip-10-0-136-0 kubenswrapper[2579]: I0703 11:52:57.567600 2579 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-cktvf\" (UniqueName: \"kubernetes.io/projected/cdcdda17-90dd-43c7-bb78-8c681de85357-kube-api-access-cktvf\") on node \"ip-10-0-136-0.ec2.internal\" DevicePath \"\"" Jul 03 11:52:57.568025 ip-10-0-136-0 kubenswrapper[2579]: I0703 11:52:57.567613 2579 reconciler_common.go:299] "Volume detached for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/cdcdda17-90dd-43c7-bb78-8c681de85357-model-cache\") on node \"ip-10-0-136-0.ec2.internal\" DevicePath \"\"" Jul 03 11:52:57.922873 ip-10-0-136-0 kubenswrapper[2579]: I0703 11:52:57.922836 2579 generic.go:356] "Generic (PLEG): container finished" podID="cdcdda17-90dd-43c7-bb78-8c681de85357" containerID="df7fdd0c994606976be798bfc74d15110f267ef1fa1c81f7ce8098fd326fc593" exitCode=0 Jul 03 11:52:57.922873 ip-10-0-136-0 kubenswrapper[2579]: I0703 11:52:57.922875 2579 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="ai-tenant-e2e-shared-a-9c2097/sub-test-model-9c2097-kserve-665fdbffff-9xbs4" Jul 03 11:52:57.923105 ip-10-0-136-0 kubenswrapper[2579]: I0703 11:52:57.922893 2579 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="ai-tenant-e2e-shared-a-9c2097/sub-test-model-9c2097-kserve-665fdbffff-9xbs4" event={"ID":"cdcdda17-90dd-43c7-bb78-8c681de85357","Type":"ContainerDied","Data":"df7fdd0c994606976be798bfc74d15110f267ef1fa1c81f7ce8098fd326fc593"} Jul 03 11:52:57.923105 ip-10-0-136-0 kubenswrapper[2579]: I0703 11:52:57.922934 2579 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="ai-tenant-e2e-shared-a-9c2097/sub-test-model-9c2097-kserve-665fdbffff-9xbs4" event={"ID":"cdcdda17-90dd-43c7-bb78-8c681de85357","Type":"ContainerDied","Data":"218330f8752ebdd2d48d81fe6f4a7ff2b45e1bddf1a771f87487d4bb87c1919c"} Jul 03 11:52:57.923105 ip-10-0-136-0 kubenswrapper[2579]: I0703 11:52:57.922954 2579 scope.go:117] "RemoveContainer" containerID="df7fdd0c994606976be798bfc74d15110f267ef1fa1c81f7ce8098fd326fc593" Jul 03 11:52:57.934249 ip-10-0-136-0 kubenswrapper[2579]: I0703 11:52:57.934177 2579 scope.go:117] "RemoveContainer" containerID="df7fdd0c994606976be798bfc74d15110f267ef1fa1c81f7ce8098fd326fc593" Jul 03 11:52:57.934608 ip-10-0-136-0 kubenswrapper[2579]: E0703 11:52:57.934574 2579 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"df7fdd0c994606976be798bfc74d15110f267ef1fa1c81f7ce8098fd326fc593\": container with ID starting with df7fdd0c994606976be798bfc74d15110f267ef1fa1c81f7ce8098fd326fc593 not found: ID does not exist" containerID="df7fdd0c994606976be798bfc74d15110f267ef1fa1c81f7ce8098fd326fc593" Jul 03 11:52:57.934685 ip-10-0-136-0 kubenswrapper[2579]: I0703 11:52:57.934625 2579 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"df7fdd0c994606976be798bfc74d15110f267ef1fa1c81f7ce8098fd326fc593"} err="failed to get container status \"df7fdd0c994606976be798bfc74d15110f267ef1fa1c81f7ce8098fd326fc593\": rpc error: code = NotFound desc = could not find container \"df7fdd0c994606976be798bfc74d15110f267ef1fa1c81f7ce8098fd326fc593\": container with ID starting with df7fdd0c994606976be798bfc74d15110f267ef1fa1c81f7ce8098fd326fc593 not found: ID does not exist" Jul 03 11:52:57.946789 ip-10-0-136-0 kubenswrapper[2579]: I0703 11:52:57.946743 2579 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["ai-tenant-e2e-shared-a-9c2097/sub-test-model-9c2097-kserve-665fdbffff-9xbs4"] Jul 03 11:52:57.957133 ip-10-0-136-0 kubenswrapper[2579]: I0703 11:52:57.957075 2579 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["ai-tenant-e2e-shared-a-9c2097/sub-test-model-9c2097-kserve-665fdbffff-9xbs4"] Jul 03 11:52:57.993454 ip-10-0-136-0 kubenswrapper[2579]: I0703 11:52:57.993415 2579 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="cdcdda17-90dd-43c7-bb78-8c681de85357" path="/var/lib/kubelet/pods/cdcdda17-90dd-43c7-bb78-8c681de85357/volumes" Jul 03 11:52:59.509032 ip-10-0-136-0 kubenswrapper[2579]: I0703 11:52:59.508991 2579 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["ai-tenant-e2e-shared-a-9c2097/rate-test-model-9c2097-kserve-76b44d97f5-9vh7w"] Jul 03 11:52:59.509494 ip-10-0-136-0 kubenswrapper[2579]: I0703 11:52:59.509478 2579 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="cdcdda17-90dd-43c7-bb78-8c681de85357" containerName="main" Jul 03 11:52:59.509541 ip-10-0-136-0 kubenswrapper[2579]: I0703 11:52:59.509496 2579 state_mem.go:107] "Deleted CPUSet assignment" podUID="cdcdda17-90dd-43c7-bb78-8c681de85357" containerName="main" Jul 03 11:52:59.509577 ip-10-0-136-0 kubenswrapper[2579]: I0703 11:52:59.509564 2579 memory_manager.go:368] "RemoveStaleState removing state" podUID="cdcdda17-90dd-43c7-bb78-8c681de85357" containerName="main" Jul 03 11:52:59.514084 ip-10-0-136-0 kubenswrapper[2579]: I0703 11:52:59.514054 2579 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="ai-tenant-e2e-shared-a-9c2097/rate-test-model-9c2097-kserve-76b44d97f5-9vh7w" Jul 03 11:52:59.517299 ip-10-0-136-0 kubenswrapper[2579]: I0703 11:52:59.517269 2579 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"ai-tenant-e2e-shared-a-9c2097\"/\"kube-root-ca.crt\"" Jul 03 11:52:59.518409 ip-10-0-136-0 kubenswrapper[2579]: I0703 11:52:59.518374 2579 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"ai-tenant-e2e-shared-a-9c2097\"/\"openshift-service-ca.crt\"" Jul 03 11:52:59.518409 ip-10-0-136-0 kubenswrapper[2579]: I0703 11:52:59.518405 2579 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"ai-tenant-e2e-shared-a-9c2097\"/\"default-dockercfg-8jfcx\"" Jul 03 11:52:59.518677 ip-10-0-136-0 kubenswrapper[2579]: I0703 11:52:59.518602 2579 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"ai-tenant-e2e-shared-a-9c2097\"/\"rate-test-model-9c2097-kserve-self-signed-certs\"" Jul 03 11:52:59.523058 ip-10-0-136-0 kubenswrapper[2579]: I0703 11:52:59.522999 2579 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["ai-tenant-e2e-shared-a-9c2097/rate-test-model-9c2097-kserve-76b44d97f5-9vh7w"] Jul 03 11:52:59.589737 ip-10-0-136-0 kubenswrapper[2579]: I0703 11:52:59.589688 2579 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/6e24dbf3-9a6a-450c-b4f0-0831612d36e6-tls-certs\") pod \"rate-test-model-9c2097-kserve-76b44d97f5-9vh7w\" (UID: \"6e24dbf3-9a6a-450c-b4f0-0831612d36e6\") " pod="ai-tenant-e2e-shared-a-9c2097/rate-test-model-9c2097-kserve-76b44d97f5-9vh7w" Jul 03 11:52:59.589737 ip-10-0-136-0 kubenswrapper[2579]: I0703 11:52:59.589743 2579 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/6e24dbf3-9a6a-450c-b4f0-0831612d36e6-model-cache\") pod \"rate-test-model-9c2097-kserve-76b44d97f5-9vh7w\" (UID: \"6e24dbf3-9a6a-450c-b4f0-0831612d36e6\") " pod="ai-tenant-e2e-shared-a-9c2097/rate-test-model-9c2097-kserve-76b44d97f5-9vh7w" Jul 03 11:52:59.590029 ip-10-0-136-0 kubenswrapper[2579]: I0703 11:52:59.589775 2579 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/6e24dbf3-9a6a-450c-b4f0-0831612d36e6-dshm\") pod \"rate-test-model-9c2097-kserve-76b44d97f5-9vh7w\" (UID: \"6e24dbf3-9a6a-450c-b4f0-0831612d36e6\") " pod="ai-tenant-e2e-shared-a-9c2097/rate-test-model-9c2097-kserve-76b44d97f5-9vh7w" Jul 03 11:52:59.590029 ip-10-0-136-0 kubenswrapper[2579]: I0703 11:52:59.589887 2579 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/6e24dbf3-9a6a-450c-b4f0-0831612d36e6-tmp-dir\") pod \"rate-test-model-9c2097-kserve-76b44d97f5-9vh7w\" (UID: \"6e24dbf3-9a6a-450c-b4f0-0831612d36e6\") " pod="ai-tenant-e2e-shared-a-9c2097/rate-test-model-9c2097-kserve-76b44d97f5-9vh7w" Jul 03 11:52:59.590029 ip-10-0-136-0 kubenswrapper[2579]: I0703 11:52:59.589950 2579 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-8dm8z\" (UniqueName: \"kubernetes.io/projected/6e24dbf3-9a6a-450c-b4f0-0831612d36e6-kube-api-access-8dm8z\") pod \"rate-test-model-9c2097-kserve-76b44d97f5-9vh7w\" (UID: \"6e24dbf3-9a6a-450c-b4f0-0831612d36e6\") " pod="ai-tenant-e2e-shared-a-9c2097/rate-test-model-9c2097-kserve-76b44d97f5-9vh7w" Jul 03 11:52:59.590165 ip-10-0-136-0 kubenswrapper[2579]: I0703 11:52:59.590055 2579 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/6e24dbf3-9a6a-450c-b4f0-0831612d36e6-home\") pod \"rate-test-model-9c2097-kserve-76b44d97f5-9vh7w\" (UID: \"6e24dbf3-9a6a-450c-b4f0-0831612d36e6\") " pod="ai-tenant-e2e-shared-a-9c2097/rate-test-model-9c2097-kserve-76b44d97f5-9vh7w" Jul 03 11:52:59.691368 ip-10-0-136-0 kubenswrapper[2579]: I0703 11:52:59.691324 2579 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/6e24dbf3-9a6a-450c-b4f0-0831612d36e6-tmp-dir\") pod \"rate-test-model-9c2097-kserve-76b44d97f5-9vh7w\" (UID: \"6e24dbf3-9a6a-450c-b4f0-0831612d36e6\") " pod="ai-tenant-e2e-shared-a-9c2097/rate-test-model-9c2097-kserve-76b44d97f5-9vh7w" Jul 03 11:52:59.691560 ip-10-0-136-0 kubenswrapper[2579]: I0703 11:52:59.691383 2579 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-8dm8z\" (UniqueName: \"kubernetes.io/projected/6e24dbf3-9a6a-450c-b4f0-0831612d36e6-kube-api-access-8dm8z\") pod \"rate-test-model-9c2097-kserve-76b44d97f5-9vh7w\" (UID: \"6e24dbf3-9a6a-450c-b4f0-0831612d36e6\") " pod="ai-tenant-e2e-shared-a-9c2097/rate-test-model-9c2097-kserve-76b44d97f5-9vh7w" Jul 03 11:52:59.691560 ip-10-0-136-0 kubenswrapper[2579]: I0703 11:52:59.691437 2579 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/6e24dbf3-9a6a-450c-b4f0-0831612d36e6-home\") pod \"rate-test-model-9c2097-kserve-76b44d97f5-9vh7w\" (UID: \"6e24dbf3-9a6a-450c-b4f0-0831612d36e6\") " pod="ai-tenant-e2e-shared-a-9c2097/rate-test-model-9c2097-kserve-76b44d97f5-9vh7w" Jul 03 11:52:59.691560 ip-10-0-136-0 kubenswrapper[2579]: I0703 11:52:59.691493 2579 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/6e24dbf3-9a6a-450c-b4f0-0831612d36e6-tls-certs\") pod \"rate-test-model-9c2097-kserve-76b44d97f5-9vh7w\" (UID: \"6e24dbf3-9a6a-450c-b4f0-0831612d36e6\") " pod="ai-tenant-e2e-shared-a-9c2097/rate-test-model-9c2097-kserve-76b44d97f5-9vh7w" Jul 03 11:52:59.691560 ip-10-0-136-0 kubenswrapper[2579]: I0703 11:52:59.691511 2579 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/6e24dbf3-9a6a-450c-b4f0-0831612d36e6-model-cache\") pod \"rate-test-model-9c2097-kserve-76b44d97f5-9vh7w\" (UID: \"6e24dbf3-9a6a-450c-b4f0-0831612d36e6\") " pod="ai-tenant-e2e-shared-a-9c2097/rate-test-model-9c2097-kserve-76b44d97f5-9vh7w" Jul 03 11:52:59.691560 ip-10-0-136-0 kubenswrapper[2579]: I0703 11:52:59.691529 2579 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/6e24dbf3-9a6a-450c-b4f0-0831612d36e6-dshm\") pod \"rate-test-model-9c2097-kserve-76b44d97f5-9vh7w\" (UID: \"6e24dbf3-9a6a-450c-b4f0-0831612d36e6\") " pod="ai-tenant-e2e-shared-a-9c2097/rate-test-model-9c2097-kserve-76b44d97f5-9vh7w" Jul 03 11:52:59.691848 ip-10-0-136-0 kubenswrapper[2579]: I0703 11:52:59.691761 2579 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/6e24dbf3-9a6a-450c-b4f0-0831612d36e6-tmp-dir\") pod \"rate-test-model-9c2097-kserve-76b44d97f5-9vh7w\" (UID: \"6e24dbf3-9a6a-450c-b4f0-0831612d36e6\") " pod="ai-tenant-e2e-shared-a-9c2097/rate-test-model-9c2097-kserve-76b44d97f5-9vh7w" Jul 03 11:52:59.691898 ip-10-0-136-0 kubenswrapper[2579]: I0703 11:52:59.691878 2579 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/6e24dbf3-9a6a-450c-b4f0-0831612d36e6-model-cache\") pod \"rate-test-model-9c2097-kserve-76b44d97f5-9vh7w\" (UID: \"6e24dbf3-9a6a-450c-b4f0-0831612d36e6\") " pod="ai-tenant-e2e-shared-a-9c2097/rate-test-model-9c2097-kserve-76b44d97f5-9vh7w" Jul 03 11:52:59.691934 ip-10-0-136-0 kubenswrapper[2579]: I0703 11:52:59.691906 2579 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/6e24dbf3-9a6a-450c-b4f0-0831612d36e6-home\") pod \"rate-test-model-9c2097-kserve-76b44d97f5-9vh7w\" (UID: \"6e24dbf3-9a6a-450c-b4f0-0831612d36e6\") " pod="ai-tenant-e2e-shared-a-9c2097/rate-test-model-9c2097-kserve-76b44d97f5-9vh7w" Jul 03 11:52:59.694182 ip-10-0-136-0 kubenswrapper[2579]: I0703 11:52:59.694150 2579 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/6e24dbf3-9a6a-450c-b4f0-0831612d36e6-dshm\") pod \"rate-test-model-9c2097-kserve-76b44d97f5-9vh7w\" (UID: \"6e24dbf3-9a6a-450c-b4f0-0831612d36e6\") " pod="ai-tenant-e2e-shared-a-9c2097/rate-test-model-9c2097-kserve-76b44d97f5-9vh7w" Jul 03 11:52:59.694408 ip-10-0-136-0 kubenswrapper[2579]: I0703 11:52:59.694388 2579 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/6e24dbf3-9a6a-450c-b4f0-0831612d36e6-tls-certs\") pod \"rate-test-model-9c2097-kserve-76b44d97f5-9vh7w\" (UID: \"6e24dbf3-9a6a-450c-b4f0-0831612d36e6\") " pod="ai-tenant-e2e-shared-a-9c2097/rate-test-model-9c2097-kserve-76b44d97f5-9vh7w" Jul 03 11:52:59.704043 ip-10-0-136-0 kubenswrapper[2579]: I0703 11:52:59.704009 2579 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-8dm8z\" (UniqueName: \"kubernetes.io/projected/6e24dbf3-9a6a-450c-b4f0-0831612d36e6-kube-api-access-8dm8z\") pod \"rate-test-model-9c2097-kserve-76b44d97f5-9vh7w\" (UID: \"6e24dbf3-9a6a-450c-b4f0-0831612d36e6\") " pod="ai-tenant-e2e-shared-a-9c2097/rate-test-model-9c2097-kserve-76b44d97f5-9vh7w" Jul 03 11:52:59.826234 ip-10-0-136-0 kubenswrapper[2579]: I0703 11:52:59.826172 2579 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="ai-tenant-e2e-shared-a-9c2097/rate-test-model-9c2097-kserve-76b44d97f5-9vh7w" Jul 03 11:53:00.001483 ip-10-0-136-0 kubenswrapper[2579]: I0703 11:53:00.001452 2579 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["ai-tenant-e2e-shared-a-9c2097/rate-test-model-9c2097-kserve-76b44d97f5-9vh7w"] Jul 03 11:53:00.002884 ip-10-0-136-0 kubenswrapper[2579]: W0703 11:53:00.002855 2579 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-besteffort.slice/kubepods-besteffort-pod6e24dbf3_9a6a_450c_b4f0_0831612d36e6.slice/crio-70447688bfc71d89595b1c90a8b09a9f0fdc52b77b585af0cb6bf9e8b528e412 WatchSource:0}: Error finding container 70447688bfc71d89595b1c90a8b09a9f0fdc52b77b585af0cb6bf9e8b528e412: Status 404 returned error can't find the container with id 70447688bfc71d89595b1c90a8b09a9f0fdc52b77b585af0cb6bf9e8b528e412 Jul 03 11:53:00.950279 ip-10-0-136-0 kubenswrapper[2579]: I0703 11:53:00.950236 2579 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="ai-tenant-e2e-shared-a-9c2097/rate-test-model-9c2097-kserve-76b44d97f5-9vh7w" event={"ID":"6e24dbf3-9a6a-450c-b4f0-0831612d36e6","Type":"ContainerStarted","Data":"75353135de6d9c0174aca29cf2f38ca68dd518aa2180129172232f4082787985"} Jul 03 11:53:00.950697 ip-10-0-136-0 kubenswrapper[2579]: I0703 11:53:00.950293 2579 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="ai-tenant-e2e-shared-a-9c2097/rate-test-model-9c2097-kserve-76b44d97f5-9vh7w" event={"ID":"6e24dbf3-9a6a-450c-b4f0-0831612d36e6","Type":"ContainerStarted","Data":"70447688bfc71d89595b1c90a8b09a9f0fdc52b77b585af0cb6bf9e8b528e412"} Jul 03 11:53:00.971121 ip-10-0-136-0 kubenswrapper[2579]: I0703 11:53:00.971055 2579 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="ai-tenant-e2e-shared-a-9c2097/rate-test-model-9c2097-kserve-76b44d97f5-9vh7w" podStartSLOduration=1.971038582 podStartE2EDuration="1.971038582s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-03 11:52:59 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-03 11:53:00.96943316 +0000 UTC m=+2437.557737501" watchObservedRunningTime="2026-07-03 11:53:00.971038582 +0000 UTC m=+2437.559342903" Jul 03 11:53:09.826874 ip-10-0-136-0 kubenswrapper[2579]: I0703 11:53:09.826774 2579 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="ai-tenant-e2e-shared-a-9c2097/rate-test-model-9c2097-kserve-76b44d97f5-9vh7w" Jul 03 11:53:09.827439 ip-10-0-136-0 kubenswrapper[2579]: I0703 11:53:09.826895 2579 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="ai-tenant-e2e-shared-a-9c2097/rate-test-model-9c2097-kserve-76b44d97f5-9vh7w" Jul 03 11:53:09.840753 ip-10-0-136-0 kubenswrapper[2579]: I0703 11:53:09.840724 2579 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="ai-tenant-e2e-shared-a-9c2097/rate-test-model-9c2097-kserve-76b44d97f5-9vh7w" Jul 03 11:53:10.021663 ip-10-0-136-0 kubenswrapper[2579]: I0703 11:53:10.021625 2579 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="ai-tenant-e2e-shared-a-9c2097/rate-test-model-9c2097-kserve-76b44d97f5-9vh7w" Jul 03 11:54:17.701429 ip-10-0-136-0 kubenswrapper[2579]: I0703 11:54:17.701340 2579 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["ai-tenant-e2e-shared-a-9c2097/rate-test-model-9c2097-kserve-76b44d97f5-9vh7w"] Jul 03 11:54:32.723351 ip-10-0-136-0 kubenswrapper[2579]: I0703 11:54:32.723294 2579 kuberuntime_container.go:872] "Killing container with a grace period" pod="ai-tenant-e2e-shared-a-9c2097/rate-test-model-9c2097-kserve-76b44d97f5-9vh7w" podUID="6e24dbf3-9a6a-450c-b4f0-0831612d36e6" containerName="main" containerID="cri-o://75353135de6d9c0174aca29cf2f38ca68dd518aa2180129172232f4082787985" gracePeriod=45 Jul 03 11:54:32.986586 ip-10-0-136-0 kubenswrapper[2579]: I0703 11:54:32.986500 2579 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="ai-tenant-e2e-shared-a-9c2097/rate-test-model-9c2097-kserve-76b44d97f5-9vh7w" Jul 03 11:54:33.072346 ip-10-0-136-0 kubenswrapper[2579]: I0703 11:54:33.072310 2579 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/6e24dbf3-9a6a-450c-b4f0-0831612d36e6-tmp-dir\") pod \"6e24dbf3-9a6a-450c-b4f0-0831612d36e6\" (UID: \"6e24dbf3-9a6a-450c-b4f0-0831612d36e6\") " Jul 03 11:54:33.072346 ip-10-0-136-0 kubenswrapper[2579]: I0703 11:54:33.072357 2579 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/6e24dbf3-9a6a-450c-b4f0-0831612d36e6-model-cache\") pod \"6e24dbf3-9a6a-450c-b4f0-0831612d36e6\" (UID: \"6e24dbf3-9a6a-450c-b4f0-0831612d36e6\") " Jul 03 11:54:33.072607 ip-10-0-136-0 kubenswrapper[2579]: I0703 11:54:33.072400 2579 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-8dm8z\" (UniqueName: \"kubernetes.io/projected/6e24dbf3-9a6a-450c-b4f0-0831612d36e6-kube-api-access-8dm8z\") pod \"6e24dbf3-9a6a-450c-b4f0-0831612d36e6\" (UID: \"6e24dbf3-9a6a-450c-b4f0-0831612d36e6\") " Jul 03 11:54:33.072607 ip-10-0-136-0 kubenswrapper[2579]: I0703 11:54:33.072437 2579 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/6e24dbf3-9a6a-450c-b4f0-0831612d36e6-tls-certs\") pod \"6e24dbf3-9a6a-450c-b4f0-0831612d36e6\" (UID: \"6e24dbf3-9a6a-450c-b4f0-0831612d36e6\") " Jul 03 11:54:33.072607 ip-10-0-136-0 kubenswrapper[2579]: I0703 11:54:33.072462 2579 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/6e24dbf3-9a6a-450c-b4f0-0831612d36e6-home\") pod \"6e24dbf3-9a6a-450c-b4f0-0831612d36e6\" (UID: \"6e24dbf3-9a6a-450c-b4f0-0831612d36e6\") " Jul 03 11:54:33.072771 ip-10-0-136-0 kubenswrapper[2579]: I0703 11:54:33.072707 2579 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/6e24dbf3-9a6a-450c-b4f0-0831612d36e6-tmp-dir" (OuterVolumeSpecName: "tmp-dir") pod "6e24dbf3-9a6a-450c-b4f0-0831612d36e6" (UID: "6e24dbf3-9a6a-450c-b4f0-0831612d36e6"). InnerVolumeSpecName "tmp-dir". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Jul 03 11:54:33.072841 ip-10-0-136-0 kubenswrapper[2579]: I0703 11:54:33.072783 2579 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/6e24dbf3-9a6a-450c-b4f0-0831612d36e6-model-cache" (OuterVolumeSpecName: "model-cache") pod "6e24dbf3-9a6a-450c-b4f0-0831612d36e6" (UID: "6e24dbf3-9a6a-450c-b4f0-0831612d36e6"). InnerVolumeSpecName "model-cache". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Jul 03 11:54:33.072897 ip-10-0-136-0 kubenswrapper[2579]: I0703 11:54:33.072857 2579 reconciler_common.go:299] "Volume detached for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/6e24dbf3-9a6a-450c-b4f0-0831612d36e6-tmp-dir\") on node \"ip-10-0-136-0.ec2.internal\" DevicePath \"\"" Jul 03 11:54:33.072897 ip-10-0-136-0 kubenswrapper[2579]: I0703 11:54:33.072857 2579 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/6e24dbf3-9a6a-450c-b4f0-0831612d36e6-home" (OuterVolumeSpecName: "home") pod "6e24dbf3-9a6a-450c-b4f0-0831612d36e6" (UID: "6e24dbf3-9a6a-450c-b4f0-0831612d36e6"). InnerVolumeSpecName "home". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Jul 03 11:54:33.074925 ip-10-0-136-0 kubenswrapper[2579]: I0703 11:54:33.074886 2579 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/6e24dbf3-9a6a-450c-b4f0-0831612d36e6-kube-api-access-8dm8z" (OuterVolumeSpecName: "kube-api-access-8dm8z") pod "6e24dbf3-9a6a-450c-b4f0-0831612d36e6" (UID: "6e24dbf3-9a6a-450c-b4f0-0831612d36e6"). InnerVolumeSpecName "kube-api-access-8dm8z". PluginName "kubernetes.io/projected", VolumeGIDValue "" Jul 03 11:54:33.075070 ip-10-0-136-0 kubenswrapper[2579]: I0703 11:54:33.074925 2579 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/6e24dbf3-9a6a-450c-b4f0-0831612d36e6-tls-certs" (OuterVolumeSpecName: "tls-certs") pod "6e24dbf3-9a6a-450c-b4f0-0831612d36e6" (UID: "6e24dbf3-9a6a-450c-b4f0-0831612d36e6"). InnerVolumeSpecName "tls-certs". PluginName "kubernetes.io/secret", VolumeGIDValue "" Jul 03 11:54:33.173448 ip-10-0-136-0 kubenswrapper[2579]: I0703 11:54:33.173406 2579 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/6e24dbf3-9a6a-450c-b4f0-0831612d36e6-dshm\") pod \"6e24dbf3-9a6a-450c-b4f0-0831612d36e6\" (UID: \"6e24dbf3-9a6a-450c-b4f0-0831612d36e6\") " Jul 03 11:54:33.173681 ip-10-0-136-0 kubenswrapper[2579]: I0703 11:54:33.173666 2579 reconciler_common.go:299] "Volume detached for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/6e24dbf3-9a6a-450c-b4f0-0831612d36e6-model-cache\") on node \"ip-10-0-136-0.ec2.internal\" DevicePath \"\"" Jul 03 11:54:33.173758 ip-10-0-136-0 kubenswrapper[2579]: I0703 11:54:33.173682 2579 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-8dm8z\" (UniqueName: \"kubernetes.io/projected/6e24dbf3-9a6a-450c-b4f0-0831612d36e6-kube-api-access-8dm8z\") on node \"ip-10-0-136-0.ec2.internal\" DevicePath \"\"" Jul 03 11:54:33.173758 ip-10-0-136-0 kubenswrapper[2579]: I0703 11:54:33.173691 2579 reconciler_common.go:299] "Volume detached for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/6e24dbf3-9a6a-450c-b4f0-0831612d36e6-tls-certs\") on node \"ip-10-0-136-0.ec2.internal\" DevicePath \"\"" Jul 03 11:54:33.173758 ip-10-0-136-0 kubenswrapper[2579]: I0703 11:54:33.173699 2579 reconciler_common.go:299] "Volume detached for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/6e24dbf3-9a6a-450c-b4f0-0831612d36e6-home\") on node \"ip-10-0-136-0.ec2.internal\" DevicePath \"\"" Jul 03 11:54:33.175736 ip-10-0-136-0 kubenswrapper[2579]: I0703 11:54:33.175715 2579 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/6e24dbf3-9a6a-450c-b4f0-0831612d36e6-dshm" (OuterVolumeSpecName: "dshm") pod "6e24dbf3-9a6a-450c-b4f0-0831612d36e6" (UID: "6e24dbf3-9a6a-450c-b4f0-0831612d36e6"). InnerVolumeSpecName "dshm". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Jul 03 11:54:33.274446 ip-10-0-136-0 kubenswrapper[2579]: I0703 11:54:33.274414 2579 reconciler_common.go:299] "Volume detached for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/6e24dbf3-9a6a-450c-b4f0-0831612d36e6-dshm\") on node \"ip-10-0-136-0.ec2.internal\" DevicePath \"\"" Jul 03 11:54:33.538276 ip-10-0-136-0 kubenswrapper[2579]: I0703 11:54:33.538143 2579 generic.go:356] "Generic (PLEG): container finished" podID="6e24dbf3-9a6a-450c-b4f0-0831612d36e6" containerID="75353135de6d9c0174aca29cf2f38ca68dd518aa2180129172232f4082787985" exitCode=0 Jul 03 11:54:33.538276 ip-10-0-136-0 kubenswrapper[2579]: I0703 11:54:33.538213 2579 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="ai-tenant-e2e-shared-a-9c2097/rate-test-model-9c2097-kserve-76b44d97f5-9vh7w" Jul 03 11:54:33.538276 ip-10-0-136-0 kubenswrapper[2579]: I0703 11:54:33.538237 2579 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="ai-tenant-e2e-shared-a-9c2097/rate-test-model-9c2097-kserve-76b44d97f5-9vh7w" event={"ID":"6e24dbf3-9a6a-450c-b4f0-0831612d36e6","Type":"ContainerDied","Data":"75353135de6d9c0174aca29cf2f38ca68dd518aa2180129172232f4082787985"} Jul 03 11:54:33.538549 ip-10-0-136-0 kubenswrapper[2579]: I0703 11:54:33.538286 2579 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="ai-tenant-e2e-shared-a-9c2097/rate-test-model-9c2097-kserve-76b44d97f5-9vh7w" event={"ID":"6e24dbf3-9a6a-450c-b4f0-0831612d36e6","Type":"ContainerDied","Data":"70447688bfc71d89595b1c90a8b09a9f0fdc52b77b585af0cb6bf9e8b528e412"} Jul 03 11:54:33.538549 ip-10-0-136-0 kubenswrapper[2579]: I0703 11:54:33.538306 2579 scope.go:117] "RemoveContainer" containerID="75353135de6d9c0174aca29cf2f38ca68dd518aa2180129172232f4082787985" Jul 03 11:54:33.549425 ip-10-0-136-0 kubenswrapper[2579]: I0703 11:54:33.549401 2579 scope.go:117] "RemoveContainer" containerID="75353135de6d9c0174aca29cf2f38ca68dd518aa2180129172232f4082787985" Jul 03 11:54:33.549805 ip-10-0-136-0 kubenswrapper[2579]: E0703 11:54:33.549783 2579 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"75353135de6d9c0174aca29cf2f38ca68dd518aa2180129172232f4082787985\": container with ID starting with 75353135de6d9c0174aca29cf2f38ca68dd518aa2180129172232f4082787985 not found: ID does not exist" containerID="75353135de6d9c0174aca29cf2f38ca68dd518aa2180129172232f4082787985" Jul 03 11:54:33.549890 ip-10-0-136-0 kubenswrapper[2579]: I0703 11:54:33.549820 2579 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"75353135de6d9c0174aca29cf2f38ca68dd518aa2180129172232f4082787985"} err="failed to get container status \"75353135de6d9c0174aca29cf2f38ca68dd518aa2180129172232f4082787985\": rpc error: code = NotFound desc = could not find container \"75353135de6d9c0174aca29cf2f38ca68dd518aa2180129172232f4082787985\": container with ID starting with 75353135de6d9c0174aca29cf2f38ca68dd518aa2180129172232f4082787985 not found: ID does not exist" Jul 03 11:54:33.564081 ip-10-0-136-0 kubenswrapper[2579]: I0703 11:54:33.564042 2579 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["ai-tenant-e2e-shared-a-9c2097/rate-test-model-9c2097-kserve-76b44d97f5-9vh7w"] Jul 03 11:54:33.566756 ip-10-0-136-0 kubenswrapper[2579]: I0703 11:54:33.566731 2579 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["ai-tenant-e2e-shared-a-9c2097/rate-test-model-9c2097-kserve-76b44d97f5-9vh7w"] Jul 03 11:54:33.993024 ip-10-0-136-0 kubenswrapper[2579]: I0703 11:54:33.992933 2579 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="6e24dbf3-9a6a-450c-b4f0-0831612d36e6" path="/var/lib/kubelet/pods/6e24dbf3-9a6a-450c-b4f0-0831612d36e6/volumes" Jul 03 11:55:02.624348 ip-10-0-136-0 kubenswrapper[2579]: I0703 11:55:02.624304 2579 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-ingress/e2e-shared-a-9c2097-openshift-default-6d687665ff-25rvz"] Jul 03 11:55:02.624962 ip-10-0-136-0 kubenswrapper[2579]: I0703 11:55:02.624620 2579 kuberuntime_container.go:872] "Killing container with a grace period" pod="openshift-ingress/e2e-shared-a-9c2097-openshift-default-6d687665ff-25rvz" podUID="f4cf29c7-812e-45ca-bf86-80a5cff794ac" containerName="istio-proxy" containerID="cri-o://b8845547f8938f6cc98ff7a4f84abdd2bd1a47f066ae1c610dad12c3f44c2cc6" gracePeriod=30 Jul 03 11:55:07.762518 ip-10-0-136-0 kubenswrapper[2579]: I0703 11:55:07.762483 2579 generic.go:356] "Generic (PLEG): container finished" podID="f4cf29c7-812e-45ca-bf86-80a5cff794ac" containerID="b8845547f8938f6cc98ff7a4f84abdd2bd1a47f066ae1c610dad12c3f44c2cc6" exitCode=0 Jul 03 11:55:07.762962 ip-10-0-136-0 kubenswrapper[2579]: I0703 11:55:07.762563 2579 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-shared-a-9c2097-openshift-default-6d687665ff-25rvz" event={"ID":"f4cf29c7-812e-45ca-bf86-80a5cff794ac","Type":"ContainerDied","Data":"b8845547f8938f6cc98ff7a4f84abdd2bd1a47f066ae1c610dad12c3f44c2cc6"} Jul 03 11:55:07.762962 ip-10-0-136-0 kubenswrapper[2579]: I0703 11:55:07.762672 2579 scope.go:117] "RemoveContainer" containerID="693748e656f6194cc63b93d7d5c369f12d3d882ae6cd31d22e02cbf80c21791d" Jul 03 11:55:07.901511 ip-10-0-136-0 kubenswrapper[2579]: I0703 11:55:07.901426 2579 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-shared-a-9c2097-openshift-default-6d687665ff-25rvz" Jul 03 11:55:08.039717 ip-10-0-136-0 kubenswrapper[2579]: I0703 11:55:08.039674 2579 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/f4cf29c7-812e-45ca-bf86-80a5cff794ac-istiod-ca-cert\") pod \"f4cf29c7-812e-45ca-bf86-80a5cff794ac\" (UID: \"f4cf29c7-812e-45ca-bf86-80a5cff794ac\") " Jul 03 11:55:08.039954 ip-10-0-136-0 kubenswrapper[2579]: I0703 11:55:08.039742 2579 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/f4cf29c7-812e-45ca-bf86-80a5cff794ac-istio-token\") pod \"f4cf29c7-812e-45ca-bf86-80a5cff794ac\" (UID: \"f4cf29c7-812e-45ca-bf86-80a5cff794ac\") " Jul 03 11:55:08.039954 ip-10-0-136-0 kubenswrapper[2579]: I0703 11:55:08.039762 2579 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/f4cf29c7-812e-45ca-bf86-80a5cff794ac-workload-socket\") pod \"f4cf29c7-812e-45ca-bf86-80a5cff794ac\" (UID: \"f4cf29c7-812e-45ca-bf86-80a5cff794ac\") " Jul 03 11:55:08.039954 ip-10-0-136-0 kubenswrapper[2579]: I0703 11:55:08.039779 2579 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-vxql8\" (UniqueName: \"kubernetes.io/projected/f4cf29c7-812e-45ca-bf86-80a5cff794ac-kube-api-access-vxql8\") pod \"f4cf29c7-812e-45ca-bf86-80a5cff794ac\" (UID: \"f4cf29c7-812e-45ca-bf86-80a5cff794ac\") " Jul 03 11:55:08.039954 ip-10-0-136-0 kubenswrapper[2579]: I0703 11:55:08.039800 2579 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/f4cf29c7-812e-45ca-bf86-80a5cff794ac-istio-data\") pod \"f4cf29c7-812e-45ca-bf86-80a5cff794ac\" (UID: \"f4cf29c7-812e-45ca-bf86-80a5cff794ac\") " Jul 03 11:55:08.039954 ip-10-0-136-0 kubenswrapper[2579]: I0703 11:55:08.039839 2579 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/f4cf29c7-812e-45ca-bf86-80a5cff794ac-istio-podinfo\") pod \"f4cf29c7-812e-45ca-bf86-80a5cff794ac\" (UID: \"f4cf29c7-812e-45ca-bf86-80a5cff794ac\") " Jul 03 11:55:08.039954 ip-10-0-136-0 kubenswrapper[2579]: I0703 11:55:08.039863 2579 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/f4cf29c7-812e-45ca-bf86-80a5cff794ac-credential-socket\") pod \"f4cf29c7-812e-45ca-bf86-80a5cff794ac\" (UID: \"f4cf29c7-812e-45ca-bf86-80a5cff794ac\") " Jul 03 11:55:08.039954 ip-10-0-136-0 kubenswrapper[2579]: I0703 11:55:08.039903 2579 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/f4cf29c7-812e-45ca-bf86-80a5cff794ac-workload-certs\") pod \"f4cf29c7-812e-45ca-bf86-80a5cff794ac\" (UID: \"f4cf29c7-812e-45ca-bf86-80a5cff794ac\") " Jul 03 11:55:08.039954 ip-10-0-136-0 kubenswrapper[2579]: I0703 11:55:08.039932 2579 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/f4cf29c7-812e-45ca-bf86-80a5cff794ac-istio-envoy\") pod \"f4cf29c7-812e-45ca-bf86-80a5cff794ac\" (UID: \"f4cf29c7-812e-45ca-bf86-80a5cff794ac\") " Jul 03 11:55:08.040435 ip-10-0-136-0 kubenswrapper[2579]: I0703 11:55:08.040111 2579 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/f4cf29c7-812e-45ca-bf86-80a5cff794ac-workload-socket" (OuterVolumeSpecName: "workload-socket") pod "f4cf29c7-812e-45ca-bf86-80a5cff794ac" (UID: "f4cf29c7-812e-45ca-bf86-80a5cff794ac"). InnerVolumeSpecName "workload-socket". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Jul 03 11:55:08.040435 ip-10-0-136-0 kubenswrapper[2579]: I0703 11:55:08.040245 2579 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/f4cf29c7-812e-45ca-bf86-80a5cff794ac-istiod-ca-cert" (OuterVolumeSpecName: "istiod-ca-cert") pod "f4cf29c7-812e-45ca-bf86-80a5cff794ac" (UID: "f4cf29c7-812e-45ca-bf86-80a5cff794ac"). InnerVolumeSpecName "istiod-ca-cert". PluginName "kubernetes.io/configmap", VolumeGIDValue "" Jul 03 11:55:08.041167 ip-10-0-136-0 kubenswrapper[2579]: I0703 11:55:08.041118 2579 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/f4cf29c7-812e-45ca-bf86-80a5cff794ac-credential-socket" (OuterVolumeSpecName: "credential-socket") pod "f4cf29c7-812e-45ca-bf86-80a5cff794ac" (UID: "f4cf29c7-812e-45ca-bf86-80a5cff794ac"). InnerVolumeSpecName "credential-socket". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Jul 03 11:55:08.047268 ip-10-0-136-0 kubenswrapper[2579]: I0703 11:55:08.043273 2579 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/f4cf29c7-812e-45ca-bf86-80a5cff794ac-workload-certs" (OuterVolumeSpecName: "workload-certs") pod "f4cf29c7-812e-45ca-bf86-80a5cff794ac" (UID: "f4cf29c7-812e-45ca-bf86-80a5cff794ac"). InnerVolumeSpecName "workload-certs". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Jul 03 11:55:08.047268 ip-10-0-136-0 kubenswrapper[2579]: I0703 11:55:08.043803 2579 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/f4cf29c7-812e-45ca-bf86-80a5cff794ac-istio-data" (OuterVolumeSpecName: "istio-data") pod "f4cf29c7-812e-45ca-bf86-80a5cff794ac" (UID: "f4cf29c7-812e-45ca-bf86-80a5cff794ac"). InnerVolumeSpecName "istio-data". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Jul 03 11:55:08.047268 ip-10-0-136-0 kubenswrapper[2579]: I0703 11:55:08.045242 2579 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/f4cf29c7-812e-45ca-bf86-80a5cff794ac-kube-api-access-vxql8" (OuterVolumeSpecName: "kube-api-access-vxql8") pod "f4cf29c7-812e-45ca-bf86-80a5cff794ac" (UID: "f4cf29c7-812e-45ca-bf86-80a5cff794ac"). InnerVolumeSpecName "kube-api-access-vxql8". PluginName "kubernetes.io/projected", VolumeGIDValue "" Jul 03 11:55:08.051020 ip-10-0-136-0 kubenswrapper[2579]: I0703 11:55:08.050977 2579 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/f4cf29c7-812e-45ca-bf86-80a5cff794ac-istio-envoy" (OuterVolumeSpecName: "istio-envoy") pod "f4cf29c7-812e-45ca-bf86-80a5cff794ac" (UID: "f4cf29c7-812e-45ca-bf86-80a5cff794ac"). InnerVolumeSpecName "istio-envoy". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Jul 03 11:55:08.051020 ip-10-0-136-0 kubenswrapper[2579]: I0703 11:55:08.050991 2579 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/f4cf29c7-812e-45ca-bf86-80a5cff794ac-istio-token" (OuterVolumeSpecName: "istio-token") pod "f4cf29c7-812e-45ca-bf86-80a5cff794ac" (UID: "f4cf29c7-812e-45ca-bf86-80a5cff794ac"). InnerVolumeSpecName "istio-token". PluginName "kubernetes.io/projected", VolumeGIDValue "" Jul 03 11:55:08.051414 ip-10-0-136-0 kubenswrapper[2579]: I0703 11:55:08.051394 2579 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/downward-api/f4cf29c7-812e-45ca-bf86-80a5cff794ac-istio-podinfo" (OuterVolumeSpecName: "istio-podinfo") pod "f4cf29c7-812e-45ca-bf86-80a5cff794ac" (UID: "f4cf29c7-812e-45ca-bf86-80a5cff794ac"). InnerVolumeSpecName "istio-podinfo". PluginName "kubernetes.io/downward-api", VolumeGIDValue "" Jul 03 11:55:08.142530 ip-10-0-136-0 kubenswrapper[2579]: I0703 11:55:08.142486 2579 reconciler_common.go:299] "Volume detached for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/f4cf29c7-812e-45ca-bf86-80a5cff794ac-workload-certs\") on node \"ip-10-0-136-0.ec2.internal\" DevicePath \"\"" Jul 03 11:55:08.142530 ip-10-0-136-0 kubenswrapper[2579]: I0703 11:55:08.142521 2579 reconciler_common.go:299] "Volume detached for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/f4cf29c7-812e-45ca-bf86-80a5cff794ac-istio-envoy\") on node \"ip-10-0-136-0.ec2.internal\" DevicePath \"\"" Jul 03 11:55:08.142530 ip-10-0-136-0 kubenswrapper[2579]: I0703 11:55:08.142531 2579 reconciler_common.go:299] "Volume detached for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/f4cf29c7-812e-45ca-bf86-80a5cff794ac-istiod-ca-cert\") on node \"ip-10-0-136-0.ec2.internal\" DevicePath \"\"" Jul 03 11:55:08.142530 ip-10-0-136-0 kubenswrapper[2579]: I0703 11:55:08.142539 2579 reconciler_common.go:299] "Volume detached for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/f4cf29c7-812e-45ca-bf86-80a5cff794ac-istio-token\") on node \"ip-10-0-136-0.ec2.internal\" DevicePath \"\"" Jul 03 11:55:08.142824 ip-10-0-136-0 kubenswrapper[2579]: I0703 11:55:08.142548 2579 reconciler_common.go:299] "Volume detached for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/f4cf29c7-812e-45ca-bf86-80a5cff794ac-workload-socket\") on node \"ip-10-0-136-0.ec2.internal\" DevicePath \"\"" Jul 03 11:55:08.142824 ip-10-0-136-0 kubenswrapper[2579]: I0703 11:55:08.142558 2579 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-vxql8\" (UniqueName: \"kubernetes.io/projected/f4cf29c7-812e-45ca-bf86-80a5cff794ac-kube-api-access-vxql8\") on node \"ip-10-0-136-0.ec2.internal\" DevicePath \"\"" Jul 03 11:55:08.142824 ip-10-0-136-0 kubenswrapper[2579]: I0703 11:55:08.142567 2579 reconciler_common.go:299] "Volume detached for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/f4cf29c7-812e-45ca-bf86-80a5cff794ac-istio-data\") on node \"ip-10-0-136-0.ec2.internal\" DevicePath \"\"" Jul 03 11:55:08.142824 ip-10-0-136-0 kubenswrapper[2579]: I0703 11:55:08.142574 2579 reconciler_common.go:299] "Volume detached for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/f4cf29c7-812e-45ca-bf86-80a5cff794ac-istio-podinfo\") on node \"ip-10-0-136-0.ec2.internal\" DevicePath \"\"" Jul 03 11:55:08.142824 ip-10-0-136-0 kubenswrapper[2579]: I0703 11:55:08.142582 2579 reconciler_common.go:299] "Volume detached for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/f4cf29c7-812e-45ca-bf86-80a5cff794ac-credential-socket\") on node \"ip-10-0-136-0.ec2.internal\" DevicePath \"\"" Jul 03 11:55:08.770054 ip-10-0-136-0 kubenswrapper[2579]: I0703 11:55:08.770022 2579 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-shared-a-9c2097-openshift-default-6d687665ff-25rvz" Jul 03 11:55:08.770483 ip-10-0-136-0 kubenswrapper[2579]: I0703 11:55:08.770026 2579 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-shared-a-9c2097-openshift-default-6d687665ff-25rvz" event={"ID":"f4cf29c7-812e-45ca-bf86-80a5cff794ac","Type":"ContainerDied","Data":"564d5e71a7aacef936215de629c849bdbaf861e7a600b62aeb496299044c3aa8"} Jul 03 11:55:08.770483 ip-10-0-136-0 kubenswrapper[2579]: I0703 11:55:08.770184 2579 scope.go:117] "RemoveContainer" containerID="b8845547f8938f6cc98ff7a4f84abdd2bd1a47f066ae1c610dad12c3f44c2cc6" Jul 03 11:55:08.795089 ip-10-0-136-0 kubenswrapper[2579]: I0703 11:55:08.795036 2579 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-ingress/e2e-shared-a-9c2097-openshift-default-6d687665ff-25rvz"] Jul 03 11:55:08.801569 ip-10-0-136-0 kubenswrapper[2579]: I0703 11:55:08.801523 2579 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["openshift-ingress/e2e-shared-a-9c2097-openshift-default-6d687665ff-25rvz"] Jul 03 11:55:09.992807 ip-10-0-136-0 kubenswrapper[2579]: I0703 11:55:09.992764 2579 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="f4cf29c7-812e-45ca-bf86-80a5cff794ac" path="/var/lib/kubelet/pods/f4cf29c7-812e-45ca-bf86-80a5cff794ac/volumes" Jul 03 11:55:10.413607 ip-10-0-136-0 kubenswrapper[2579]: I0703 11:55:10.413567 2579 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-ingress/e2e-shared-b-f7a699-openshift-default-ddcd795dc-k2fz7"] Jul 03 11:55:10.413890 ip-10-0-136-0 kubenswrapper[2579]: I0703 11:55:10.413856 2579 kuberuntime_container.go:872] "Killing container with a grace period" pod="openshift-ingress/e2e-shared-b-f7a699-openshift-default-ddcd795dc-k2fz7" podUID="11f2fb16-2d82-4f7c-b197-08414de5a06e" containerName="istio-proxy" containerID="cri-o://1e111a048a9451f4c306399f6a3b1775c9521f88cf94d6dab9de833d0faa04a3" gracePeriod=30 Jul 03 11:55:11.108011 ip-10-0-136-0 kubenswrapper[2579]: I0703 11:55:11.107975 2579 log.go:25] "Finished parsing log file" path="/var/log/pods/kuadrant-system_authorino-55cdcbbc74-wx8bc_49857fe6-d325-4c68-9283-4fbad85410e6/authorino/0.log" Jul 03 11:55:11.784295 ip-10-0-136-0 kubenswrapper[2579]: I0703 11:55:11.784250 2579 patch_prober.go:28] interesting pod/e2e-shared-b-f7a699-openshift-default-ddcd795dc-k2fz7 container/istio-proxy namespace/openshift-ingress: Readiness probe status=failure output="HTTP probe failed with statuscode: 503" start-of-body= Jul 03 11:55:11.784481 ip-10-0-136-0 kubenswrapper[2579]: I0703 11:55:11.784313 2579 prober.go:120] "Probe failed" probeType="Readiness" pod="openshift-ingress/e2e-shared-b-f7a699-openshift-default-ddcd795dc-k2fz7" podUID="11f2fb16-2d82-4f7c-b197-08414de5a06e" containerName="istio-proxy" probeResult="failure" output="HTTP probe failed with statuscode: 503" Jul 03 11:55:15.700134 ip-10-0-136-0 kubenswrapper[2579]: I0703 11:55:15.700097 2579 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-shared-b-f7a699-openshift-default-ddcd795dc-k2fz7" Jul 03 11:55:15.704701 ip-10-0-136-0 kubenswrapper[2579]: I0703 11:55:15.704655 2579 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/11f2fb16-2d82-4f7c-b197-08414de5a06e-workload-certs\") pod \"11f2fb16-2d82-4f7c-b197-08414de5a06e\" (UID: \"11f2fb16-2d82-4f7c-b197-08414de5a06e\") " Jul 03 11:55:15.704944 ip-10-0-136-0 kubenswrapper[2579]: I0703 11:55:15.704744 2579 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/11f2fb16-2d82-4f7c-b197-08414de5a06e-istio-podinfo\") pod \"11f2fb16-2d82-4f7c-b197-08414de5a06e\" (UID: \"11f2fb16-2d82-4f7c-b197-08414de5a06e\") " Jul 03 11:55:15.704944 ip-10-0-136-0 kubenswrapper[2579]: I0703 11:55:15.704772 2579 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/11f2fb16-2d82-4f7c-b197-08414de5a06e-credential-socket\") pod \"11f2fb16-2d82-4f7c-b197-08414de5a06e\" (UID: \"11f2fb16-2d82-4f7c-b197-08414de5a06e\") " Jul 03 11:55:15.704944 ip-10-0-136-0 kubenswrapper[2579]: I0703 11:55:15.704811 2579 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/11f2fb16-2d82-4f7c-b197-08414de5a06e-istiod-ca-cert\") pod \"11f2fb16-2d82-4f7c-b197-08414de5a06e\" (UID: \"11f2fb16-2d82-4f7c-b197-08414de5a06e\") " Jul 03 11:55:15.704944 ip-10-0-136-0 kubenswrapper[2579]: I0703 11:55:15.704835 2579 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/11f2fb16-2d82-4f7c-b197-08414de5a06e-istio-token\") pod \"11f2fb16-2d82-4f7c-b197-08414de5a06e\" (UID: \"11f2fb16-2d82-4f7c-b197-08414de5a06e\") " Jul 03 11:55:15.704944 ip-10-0-136-0 kubenswrapper[2579]: I0703 11:55:15.704887 2579 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/11f2fb16-2d82-4f7c-b197-08414de5a06e-istio-envoy\") pod \"11f2fb16-2d82-4f7c-b197-08414de5a06e\" (UID: \"11f2fb16-2d82-4f7c-b197-08414de5a06e\") " Jul 03 11:55:15.704944 ip-10-0-136-0 kubenswrapper[2579]: I0703 11:55:15.704920 2579 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/11f2fb16-2d82-4f7c-b197-08414de5a06e-istio-data\") pod \"11f2fb16-2d82-4f7c-b197-08414de5a06e\" (UID: \"11f2fb16-2d82-4f7c-b197-08414de5a06e\") " Jul 03 11:55:15.705265 ip-10-0-136-0 kubenswrapper[2579]: I0703 11:55:15.704946 2579 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-jb24j\" (UniqueName: \"kubernetes.io/projected/11f2fb16-2d82-4f7c-b197-08414de5a06e-kube-api-access-jb24j\") pod \"11f2fb16-2d82-4f7c-b197-08414de5a06e\" (UID: \"11f2fb16-2d82-4f7c-b197-08414de5a06e\") " Jul 03 11:55:15.705265 ip-10-0-136-0 kubenswrapper[2579]: I0703 11:55:15.704985 2579 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/11f2fb16-2d82-4f7c-b197-08414de5a06e-workload-socket\") pod \"11f2fb16-2d82-4f7c-b197-08414de5a06e\" (UID: \"11f2fb16-2d82-4f7c-b197-08414de5a06e\") " Jul 03 11:55:15.705265 ip-10-0-136-0 kubenswrapper[2579]: I0703 11:55:15.705114 2579 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/11f2fb16-2d82-4f7c-b197-08414de5a06e-workload-certs" (OuterVolumeSpecName: "workload-certs") pod "11f2fb16-2d82-4f7c-b197-08414de5a06e" (UID: "11f2fb16-2d82-4f7c-b197-08414de5a06e"). InnerVolumeSpecName "workload-certs". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Jul 03 11:55:15.705392 ip-10-0-136-0 kubenswrapper[2579]: I0703 11:55:15.705303 2579 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/11f2fb16-2d82-4f7c-b197-08414de5a06e-credential-socket" (OuterVolumeSpecName: "credential-socket") pod "11f2fb16-2d82-4f7c-b197-08414de5a06e" (UID: "11f2fb16-2d82-4f7c-b197-08414de5a06e"). InnerVolumeSpecName "credential-socket". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Jul 03 11:55:15.705444 ip-10-0-136-0 kubenswrapper[2579]: I0703 11:55:15.705402 2579 reconciler_common.go:299] "Volume detached for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/11f2fb16-2d82-4f7c-b197-08414de5a06e-workload-certs\") on node \"ip-10-0-136-0.ec2.internal\" DevicePath \"\"" Jul 03 11:55:15.705501 ip-10-0-136-0 kubenswrapper[2579]: I0703 11:55:15.705463 2579 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/11f2fb16-2d82-4f7c-b197-08414de5a06e-istiod-ca-cert" (OuterVolumeSpecName: "istiod-ca-cert") pod "11f2fb16-2d82-4f7c-b197-08414de5a06e" (UID: "11f2fb16-2d82-4f7c-b197-08414de5a06e"). InnerVolumeSpecName "istiod-ca-cert". PluginName "kubernetes.io/configmap", VolumeGIDValue "" Jul 03 11:55:15.705615 ip-10-0-136-0 kubenswrapper[2579]: I0703 11:55:15.705590 2579 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/11f2fb16-2d82-4f7c-b197-08414de5a06e-workload-socket" (OuterVolumeSpecName: "workload-socket") pod "11f2fb16-2d82-4f7c-b197-08414de5a06e" (UID: "11f2fb16-2d82-4f7c-b197-08414de5a06e"). InnerVolumeSpecName "workload-socket". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Jul 03 11:55:15.706642 ip-10-0-136-0 kubenswrapper[2579]: I0703 11:55:15.706583 2579 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/11f2fb16-2d82-4f7c-b197-08414de5a06e-istio-data" (OuterVolumeSpecName: "istio-data") pod "11f2fb16-2d82-4f7c-b197-08414de5a06e" (UID: "11f2fb16-2d82-4f7c-b197-08414de5a06e"). InnerVolumeSpecName "istio-data". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Jul 03 11:55:15.706894 ip-10-0-136-0 kubenswrapper[2579]: I0703 11:55:15.706869 2579 log.go:25] "Finished parsing log file" path="/var/log/pods/opendatahub_kserve-controller-manager-d6f4b65b8-sq7gp_0c97772e-8b5b-4733-bfcc-3a0fd479537d/manager/0.log" Jul 03 11:55:15.707950 ip-10-0-136-0 kubenswrapper[2579]: I0703 11:55:15.707889 2579 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/11f2fb16-2d82-4f7c-b197-08414de5a06e-istio-token" (OuterVolumeSpecName: "istio-token") pod "11f2fb16-2d82-4f7c-b197-08414de5a06e" (UID: "11f2fb16-2d82-4f7c-b197-08414de5a06e"). InnerVolumeSpecName "istio-token". PluginName "kubernetes.io/projected", VolumeGIDValue "" Jul 03 11:55:15.707950 ip-10-0-136-0 kubenswrapper[2579]: I0703 11:55:15.707904 2579 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/downward-api/11f2fb16-2d82-4f7c-b197-08414de5a06e-istio-podinfo" (OuterVolumeSpecName: "istio-podinfo") pod "11f2fb16-2d82-4f7c-b197-08414de5a06e" (UID: "11f2fb16-2d82-4f7c-b197-08414de5a06e"). InnerVolumeSpecName "istio-podinfo". PluginName "kubernetes.io/downward-api", VolumeGIDValue "" Jul 03 11:55:15.708109 ip-10-0-136-0 kubenswrapper[2579]: I0703 11:55:15.707968 2579 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/11f2fb16-2d82-4f7c-b197-08414de5a06e-istio-envoy" (OuterVolumeSpecName: "istio-envoy") pod "11f2fb16-2d82-4f7c-b197-08414de5a06e" (UID: "11f2fb16-2d82-4f7c-b197-08414de5a06e"). InnerVolumeSpecName "istio-envoy". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Jul 03 11:55:15.708411 ip-10-0-136-0 kubenswrapper[2579]: I0703 11:55:15.708387 2579 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/11f2fb16-2d82-4f7c-b197-08414de5a06e-kube-api-access-jb24j" (OuterVolumeSpecName: "kube-api-access-jb24j") pod "11f2fb16-2d82-4f7c-b197-08414de5a06e" (UID: "11f2fb16-2d82-4f7c-b197-08414de5a06e"). InnerVolumeSpecName "kube-api-access-jb24j". PluginName "kubernetes.io/projected", VolumeGIDValue "" Jul 03 11:55:15.806507 ip-10-0-136-0 kubenswrapper[2579]: I0703 11:55:15.806459 2579 reconciler_common.go:299] "Volume detached for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/11f2fb16-2d82-4f7c-b197-08414de5a06e-istio-podinfo\") on node \"ip-10-0-136-0.ec2.internal\" DevicePath \"\"" Jul 03 11:55:15.806507 ip-10-0-136-0 kubenswrapper[2579]: I0703 11:55:15.806493 2579 reconciler_common.go:299] "Volume detached for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/11f2fb16-2d82-4f7c-b197-08414de5a06e-credential-socket\") on node \"ip-10-0-136-0.ec2.internal\" DevicePath \"\"" Jul 03 11:55:15.806507 ip-10-0-136-0 kubenswrapper[2579]: I0703 11:55:15.806503 2579 reconciler_common.go:299] "Volume detached for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/11f2fb16-2d82-4f7c-b197-08414de5a06e-istiod-ca-cert\") on node \"ip-10-0-136-0.ec2.internal\" DevicePath \"\"" Jul 03 11:55:15.806507 ip-10-0-136-0 kubenswrapper[2579]: I0703 11:55:15.806511 2579 reconciler_common.go:299] "Volume detached for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/11f2fb16-2d82-4f7c-b197-08414de5a06e-istio-token\") on node \"ip-10-0-136-0.ec2.internal\" DevicePath \"\"" Jul 03 11:55:15.806507 ip-10-0-136-0 kubenswrapper[2579]: I0703 11:55:15.806519 2579 reconciler_common.go:299] "Volume detached for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/11f2fb16-2d82-4f7c-b197-08414de5a06e-istio-envoy\") on node \"ip-10-0-136-0.ec2.internal\" DevicePath \"\"" Jul 03 11:55:15.806507 ip-10-0-136-0 kubenswrapper[2579]: I0703 11:55:15.806526 2579 reconciler_common.go:299] "Volume detached for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/11f2fb16-2d82-4f7c-b197-08414de5a06e-istio-data\") on node \"ip-10-0-136-0.ec2.internal\" DevicePath \"\"" Jul 03 11:55:15.806880 ip-10-0-136-0 kubenswrapper[2579]: I0703 11:55:15.806534 2579 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-jb24j\" (UniqueName: \"kubernetes.io/projected/11f2fb16-2d82-4f7c-b197-08414de5a06e-kube-api-access-jb24j\") on node \"ip-10-0-136-0.ec2.internal\" DevicePath \"\"" Jul 03 11:55:15.806880 ip-10-0-136-0 kubenswrapper[2579]: I0703 11:55:15.806544 2579 reconciler_common.go:299] "Volume detached for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/11f2fb16-2d82-4f7c-b197-08414de5a06e-workload-socket\") on node \"ip-10-0-136-0.ec2.internal\" DevicePath \"\"" Jul 03 11:55:15.817582 ip-10-0-136-0 kubenswrapper[2579]: I0703 11:55:15.817539 2579 generic.go:356] "Generic (PLEG): container finished" podID="11f2fb16-2d82-4f7c-b197-08414de5a06e" containerID="1e111a048a9451f4c306399f6a3b1775c9521f88cf94d6dab9de833d0faa04a3" exitCode=0 Jul 03 11:55:15.817833 ip-10-0-136-0 kubenswrapper[2579]: I0703 11:55:15.817593 2579 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-shared-b-f7a699-openshift-default-ddcd795dc-k2fz7" Jul 03 11:55:15.817833 ip-10-0-136-0 kubenswrapper[2579]: I0703 11:55:15.817624 2579 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-shared-b-f7a699-openshift-default-ddcd795dc-k2fz7" event={"ID":"11f2fb16-2d82-4f7c-b197-08414de5a06e","Type":"ContainerDied","Data":"1e111a048a9451f4c306399f6a3b1775c9521f88cf94d6dab9de833d0faa04a3"} Jul 03 11:55:15.817833 ip-10-0-136-0 kubenswrapper[2579]: I0703 11:55:15.817679 2579 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-shared-b-f7a699-openshift-default-ddcd795dc-k2fz7" event={"ID":"11f2fb16-2d82-4f7c-b197-08414de5a06e","Type":"ContainerDied","Data":"4d58bd160601c76d546e6b7655575b3fec4f7b61ae7282861ee7bbca0a3d9d49"} Jul 03 11:55:15.817833 ip-10-0-136-0 kubenswrapper[2579]: I0703 11:55:15.817707 2579 scope.go:117] "RemoveContainer" containerID="1e111a048a9451f4c306399f6a3b1775c9521f88cf94d6dab9de833d0faa04a3" Jul 03 11:55:15.826374 ip-10-0-136-0 kubenswrapper[2579]: I0703 11:55:15.826340 2579 log.go:25] "Finished parsing log file" path="/var/log/pods/opendatahub_llmisvc-controller-manager-777c6bd7d5-rsv6f_ad9d71bf-a6e6-42ed-8997-2ddb98b6f21f/manager/0.log" Jul 03 11:55:15.830155 ip-10-0-136-0 kubenswrapper[2579]: I0703 11:55:15.830131 2579 scope.go:117] "RemoveContainer" containerID="0cb7f2f9c50fc7b1447022eabcde82c7bc9d13765f4a5dcf7af0a7c7b782c995" Jul 03 11:55:15.841385 ip-10-0-136-0 kubenswrapper[2579]: I0703 11:55:15.841359 2579 scope.go:117] "RemoveContainer" containerID="1e111a048a9451f4c306399f6a3b1775c9521f88cf94d6dab9de833d0faa04a3" Jul 03 11:55:15.841737 ip-10-0-136-0 kubenswrapper[2579]: E0703 11:55:15.841703 2579 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"1e111a048a9451f4c306399f6a3b1775c9521f88cf94d6dab9de833d0faa04a3\": container with ID starting with 1e111a048a9451f4c306399f6a3b1775c9521f88cf94d6dab9de833d0faa04a3 not found: ID does not exist" containerID="1e111a048a9451f4c306399f6a3b1775c9521f88cf94d6dab9de833d0faa04a3" Jul 03 11:55:15.841847 ip-10-0-136-0 kubenswrapper[2579]: I0703 11:55:15.841745 2579 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"1e111a048a9451f4c306399f6a3b1775c9521f88cf94d6dab9de833d0faa04a3"} err="failed to get container status \"1e111a048a9451f4c306399f6a3b1775c9521f88cf94d6dab9de833d0faa04a3\": rpc error: code = NotFound desc = could not find container \"1e111a048a9451f4c306399f6a3b1775c9521f88cf94d6dab9de833d0faa04a3\": container with ID starting with 1e111a048a9451f4c306399f6a3b1775c9521f88cf94d6dab9de833d0faa04a3 not found: ID does not exist" Jul 03 11:55:15.841847 ip-10-0-136-0 kubenswrapper[2579]: I0703 11:55:15.841771 2579 scope.go:117] "RemoveContainer" containerID="0cb7f2f9c50fc7b1447022eabcde82c7bc9d13765f4a5dcf7af0a7c7b782c995" Jul 03 11:55:15.842176 ip-10-0-136-0 kubenswrapper[2579]: E0703 11:55:15.842145 2579 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"0cb7f2f9c50fc7b1447022eabcde82c7bc9d13765f4a5dcf7af0a7c7b782c995\": container with ID starting with 0cb7f2f9c50fc7b1447022eabcde82c7bc9d13765f4a5dcf7af0a7c7b782c995 not found: ID does not exist" containerID="0cb7f2f9c50fc7b1447022eabcde82c7bc9d13765f4a5dcf7af0a7c7b782c995" Jul 03 11:55:15.842289 ip-10-0-136-0 kubenswrapper[2579]: I0703 11:55:15.842182 2579 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"0cb7f2f9c50fc7b1447022eabcde82c7bc9d13765f4a5dcf7af0a7c7b782c995"} err="failed to get container status \"0cb7f2f9c50fc7b1447022eabcde82c7bc9d13765f4a5dcf7af0a7c7b782c995\": rpc error: code = NotFound desc = could not find container \"0cb7f2f9c50fc7b1447022eabcde82c7bc9d13765f4a5dcf7af0a7c7b782c995\": container with ID starting with 0cb7f2f9c50fc7b1447022eabcde82c7bc9d13765f4a5dcf7af0a7c7b782c995 not found: ID does not exist" Jul 03 11:55:15.844275 ip-10-0-136-0 kubenswrapper[2579]: I0703 11:55:15.844248 2579 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-ingress/e2e-shared-b-f7a699-openshift-default-ddcd795dc-k2fz7"] Jul 03 11:55:15.847725 ip-10-0-136-0 kubenswrapper[2579]: I0703 11:55:15.847691 2579 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["openshift-ingress/e2e-shared-b-f7a699-openshift-default-ddcd795dc-k2fz7"] Jul 03 11:55:15.996170 ip-10-0-136-0 kubenswrapper[2579]: I0703 11:55:15.996077 2579 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="11f2fb16-2d82-4f7c-b197-08414de5a06e" path="/var/lib/kubelet/pods/11f2fb16-2d82-4f7c-b197-08414de5a06e/volumes" Jul 03 11:55:16.412071 ip-10-0-136-0 kubenswrapper[2579]: I0703 11:55:16.412041 2579 log.go:25] "Finished parsing log file" path="/var/log/pods/opendatahub_model-serving-api-54d89bd5fc-djwk2_bd1ed4fa-6517-40ea-aef4-14b015e858ab/server/0.log" Jul 03 11:55:16.531873 ip-10-0-136-0 kubenswrapper[2579]: I0703 11:55:16.531838 2579 log.go:25] "Finished parsing log file" path="/var/log/pods/opendatahub_odh-model-controller-7998c67dc4-wsz76_4fcd4587-9ab7-43ce-92f6-620f50b11af4/manager/0.log" Jul 03 11:55:16.647733 ip-10-0-136-0 kubenswrapper[2579]: I0703 11:55:16.647700 2579 log.go:25] "Finished parsing log file" path="/var/log/pods/opendatahub_opendatahub-operator-controller-manager-b8c56cd69-ds56h_ff9ecf9d-c9e1-4807-88bc-c5773606274c/copy-manifests/0.log" Jul 03 11:55:16.663621 ip-10-0-136-0 kubenswrapper[2579]: I0703 11:55:16.663531 2579 log.go:25] "Finished parsing log file" path="/var/log/pods/opendatahub_opendatahub-operator-controller-manager-b8c56cd69-ds56h_ff9ecf9d-c9e1-4807-88bc-c5773606274c/manager/0.log" Jul 03 11:55:17.797467 ip-10-0-136-0 kubenswrapper[2579]: I0703 11:55:17.797430 2579 log.go:25] "Finished parsing log file" path="/var/log/pods/kuadrant-system_0acee64185f523d1d1272e9af2e4d9333e0dcde792ba30e1fa9605b7599bmjs_805645ad-622e-4f54-8870-f15cd93c64f0/util/0.log" Jul 03 11:55:17.806682 ip-10-0-136-0 kubenswrapper[2579]: I0703 11:55:17.806649 2579 log.go:25] "Finished parsing log file" path="/var/log/pods/kuadrant-system_0acee64185f523d1d1272e9af2e4d9333e0dcde792ba30e1fa9605b7599bmjs_805645ad-622e-4f54-8870-f15cd93c64f0/pull/0.log" Jul 03 11:55:17.815095 ip-10-0-136-0 kubenswrapper[2579]: I0703 11:55:17.815067 2579 log.go:25] "Finished parsing log file" path="/var/log/pods/kuadrant-system_0acee64185f523d1d1272e9af2e4d9333e0dcde792ba30e1fa9605b7599bmjs_805645ad-622e-4f54-8870-f15cd93c64f0/extract/0.log" Jul 03 11:55:17.933378 ip-10-0-136-0 kubenswrapper[2579]: I0703 11:55:17.933342 2579 log.go:25] "Finished parsing log file" path="/var/log/pods/kuadrant-system_19cb86e64775c5699d5aacf881a09c2d51e7f55f9e1ff096f2a667c5e0x6knz_1742e1ed-aaec-4579-b360-fca0512b5b72/util/0.log" Jul 03 11:55:17.941730 ip-10-0-136-0 kubenswrapper[2579]: I0703 11:55:17.941699 2579 log.go:25] "Finished parsing log file" path="/var/log/pods/kuadrant-system_19cb86e64775c5699d5aacf881a09c2d51e7f55f9e1ff096f2a667c5e0x6knz_1742e1ed-aaec-4579-b360-fca0512b5b72/pull/0.log" Jul 03 11:55:17.949860 ip-10-0-136-0 kubenswrapper[2579]: I0703 11:55:17.949835 2579 log.go:25] "Finished parsing log file" path="/var/log/pods/kuadrant-system_19cb86e64775c5699d5aacf881a09c2d51e7f55f9e1ff096f2a667c5e0x6knz_1742e1ed-aaec-4579-b360-fca0512b5b72/extract/0.log" Jul 03 11:55:18.066229 ip-10-0-136-0 kubenswrapper[2579]: I0703 11:55:18.066090 2579 log.go:25] "Finished parsing log file" path="/var/log/pods/kuadrant-system_5fc36eb1065777914bfe0ff85f9a202e2a3bafefb563d9e2994474ed73mkshk_e6f9e082-5d6a-4289-9883-301dbffe6151/util/0.log" Jul 03 11:55:18.074612 ip-10-0-136-0 kubenswrapper[2579]: I0703 11:55:18.074584 2579 log.go:25] "Finished parsing log file" path="/var/log/pods/kuadrant-system_5fc36eb1065777914bfe0ff85f9a202e2a3bafefb563d9e2994474ed73mkshk_e6f9e082-5d6a-4289-9883-301dbffe6151/pull/0.log" Jul 03 11:55:18.082327 ip-10-0-136-0 kubenswrapper[2579]: I0703 11:55:18.082300 2579 log.go:25] "Finished parsing log file" path="/var/log/pods/kuadrant-system_5fc36eb1065777914bfe0ff85f9a202e2a3bafefb563d9e2994474ed73mkshk_e6f9e082-5d6a-4289-9883-301dbffe6151/extract/0.log" Jul 03 11:55:18.193832 ip-10-0-136-0 kubenswrapper[2579]: I0703 11:55:18.193797 2579 log.go:25] "Finished parsing log file" path="/var/log/pods/kuadrant-system_9438e18bbad664a016701a3153a6b421e2d977be7eb0117f80bcf45ef1chc8g_5a64e43f-33ee-4779-bc1e-366dc5ef0cc0/util/0.log" Jul 03 11:55:18.201253 ip-10-0-136-0 kubenswrapper[2579]: I0703 11:55:18.201223 2579 log.go:25] "Finished parsing log file" path="/var/log/pods/kuadrant-system_9438e18bbad664a016701a3153a6b421e2d977be7eb0117f80bcf45ef1chc8g_5a64e43f-33ee-4779-bc1e-366dc5ef0cc0/pull/0.log" Jul 03 11:55:18.208991 ip-10-0-136-0 kubenswrapper[2579]: I0703 11:55:18.208967 2579 log.go:25] "Finished parsing log file" path="/var/log/pods/kuadrant-system_9438e18bbad664a016701a3153a6b421e2d977be7eb0117f80bcf45ef1chc8g_5a64e43f-33ee-4779-bc1e-366dc5ef0cc0/extract/0.log" Jul 03 11:55:18.348662 ip-10-0-136-0 kubenswrapper[2579]: I0703 11:55:18.348567 2579 log.go:25] "Finished parsing log file" path="/var/log/pods/kuadrant-system_authorino-55cdcbbc74-wx8bc_49857fe6-d325-4c68-9283-4fbad85410e6/authorino/0.log" Jul 03 11:55:18.483118 ip-10-0-136-0 kubenswrapper[2579]: I0703 11:55:18.483081 2579 log.go:25] "Finished parsing log file" path="/var/log/pods/kuadrant-system_authorino-operator-75cc8d5cb7-nh9mf_0bcdb275-72ef-409c-804c-ca58380c460e/manager/0.log" Jul 03 11:55:18.597487 ip-10-0-136-0 kubenswrapper[2579]: I0703 11:55:18.597460 2579 log.go:25] "Finished parsing log file" path="/var/log/pods/kuadrant-system_dns-operator-controller-manager-864dd78c4c-wz8cc_763fbd50-f445-4a0e-8555-3ffa10d05e56/manager/0.log" Jul 03 11:55:18.822504 ip-10-0-136-0 kubenswrapper[2579]: I0703 11:55:18.822466 2579 log.go:25] "Finished parsing log file" path="/var/log/pods/kuadrant-system_kuadrant-operator-catalog-j29hp_b26c8c7f-99b4-4644-9aee-b33bc9985fe5/registry-server/0.log" Jul 03 11:55:18.942493 ip-10-0-136-0 kubenswrapper[2579]: I0703 11:55:18.942460 2579 log.go:25] "Finished parsing log file" path="/var/log/pods/kuadrant-system_kuadrant-operator-controller-manager-86878c769f-dcjh8_0bbde1f4-7dff-48af-92a8-26c3f6aea3c9/manager/0.log" Jul 03 11:55:19.183940 ip-10-0-136-0 kubenswrapper[2579]: I0703 11:55:19.183847 2579 log.go:25] "Finished parsing log file" path="/var/log/pods/kuadrant-system_limitador-operator-controller-manager-859d7b78bd-qk6jn_73128bfa-b0b4-47ee-b529-ba648905789e/manager/0.log" Jul 03 11:55:19.539248 ip-10-0-136-0 kubenswrapper[2579]: I0703 11:55:19.539214 2579 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ingress_data-science-gateway-data-science-gateway-class-6576859cf4tjmjj_a8b74011-2e4c-43be-bb38-bb662cd50ff3/istio-proxy/0.log" Jul 03 11:55:19.774453 ip-10-0-136-0 kubenswrapper[2579]: I0703 11:55:19.774420 2579 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ingress_kube-auth-proxy-5cb49b9d79-hl6j7_29ccc316-f283-4be9-b247-23d424fbdf1d/kube-auth-proxy/0.log" Jul 03 11:55:20.021249 ip-10-0-136-0 kubenswrapper[2579]: I0703 11:55:20.021218 2579 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ingress_maas-default-gateway-openshift-default-56654dcf7-fbkb8_b8f8d2d9-d723-4d69-b7a5-938e70db757f/istio-proxy/1.log" Jul 03 11:55:20.146163 ip-10-0-136-0 kubenswrapper[2579]: I0703 11:55:20.146129 2579 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ingress_payload-pre-processing-7964457696-ncrll_2c315c89-d57d-4d52-9426-06e88f95bdfb/payload-pre-processing/0.log" Jul 03 11:55:21.038948 ip-10-0-136-0 kubenswrapper[2579]: I0703 11:55:21.038915 2579 log.go:25] "Finished parsing log file" path="/var/log/pods/llm_e2e-unconfigured-facebook-opt-125m-simulated-kserve-6b86dbqpm6s_72272306-4c89-4f32-9d23-296f71c9daf4/main/0.log" Jul 03 11:55:21.264797 ip-10-0-136-0 kubenswrapper[2579]: I0703 11:55:21.264763 2579 log.go:25] "Finished parsing log file" path="/var/log/pods/llm_premium-simulated-simulated-premium-kserve-5fc6fb456f-qzj65_2328f303-ef30-4846-beb3-7f2124b9de08/main/0.log" Jul 03 11:55:27.978210 ip-10-0-136-0 kubenswrapper[2579]: I0703 11:55:27.978156 2579 log.go:25] "Finished parsing log file" path="/var/log/pods/kube-system_global-pull-secret-syncer-9pqm8_7813f158-0f67-411e-84d9-a2a1c51eea61/global-pull-secret-syncer/0.log" Jul 03 11:55:28.178027 ip-10-0-136-0 kubenswrapper[2579]: I0703 11:55:28.177966 2579 log.go:25] "Finished parsing log file" path="/var/log/pods/kube-system_konnectivity-agent-5lz4n_945b7f84-a41f-413e-a7b6-87818468360f/konnectivity-agent/0.log" Jul 03 11:55:28.234990 ip-10-0-136-0 kubenswrapper[2579]: I0703 11:55:28.234898 2579 log.go:25] "Finished parsing log file" path="/var/log/pods/kube-system_kube-apiserver-proxy-ip-10-0-136-0.ec2.internal_e9fe0b17840ecce76a8985c0f09920db/haproxy/0.log" Jul 03 11:55:33.127885 ip-10-0-136-0 kubenswrapper[2579]: I0703 11:55:33.127850 2579 log.go:25] "Finished parsing log file" path="/var/log/pods/kuadrant-system_0acee64185f523d1d1272e9af2e4d9333e0dcde792ba30e1fa9605b7599bmjs_805645ad-622e-4f54-8870-f15cd93c64f0/extract/0.log" Jul 03 11:55:33.157586 ip-10-0-136-0 kubenswrapper[2579]: I0703 11:55:33.157549 2579 log.go:25] "Finished parsing log file" path="/var/log/pods/kuadrant-system_0acee64185f523d1d1272e9af2e4d9333e0dcde792ba30e1fa9605b7599bmjs_805645ad-622e-4f54-8870-f15cd93c64f0/util/0.log" Jul 03 11:55:33.185199 ip-10-0-136-0 kubenswrapper[2579]: I0703 11:55:33.185168 2579 log.go:25] "Finished parsing log file" path="/var/log/pods/kuadrant-system_0acee64185f523d1d1272e9af2e4d9333e0dcde792ba30e1fa9605b7599bmjs_805645ad-622e-4f54-8870-f15cd93c64f0/pull/0.log" Jul 03 11:55:33.219361 ip-10-0-136-0 kubenswrapper[2579]: I0703 11:55:33.219327 2579 log.go:25] "Finished parsing log file" path="/var/log/pods/kuadrant-system_19cb86e64775c5699d5aacf881a09c2d51e7f55f9e1ff096f2a667c5e0x6knz_1742e1ed-aaec-4579-b360-fca0512b5b72/extract/0.log" Jul 03 11:55:33.254990 ip-10-0-136-0 kubenswrapper[2579]: I0703 11:55:33.254955 2579 log.go:25] "Finished parsing log file" path="/var/log/pods/kuadrant-system_19cb86e64775c5699d5aacf881a09c2d51e7f55f9e1ff096f2a667c5e0x6knz_1742e1ed-aaec-4579-b360-fca0512b5b72/util/0.log" Jul 03 11:55:33.293629 ip-10-0-136-0 kubenswrapper[2579]: I0703 11:55:33.293598 2579 log.go:25] "Finished parsing log file" path="/var/log/pods/kuadrant-system_19cb86e64775c5699d5aacf881a09c2d51e7f55f9e1ff096f2a667c5e0x6knz_1742e1ed-aaec-4579-b360-fca0512b5b72/pull/0.log" Jul 03 11:55:33.330658 ip-10-0-136-0 kubenswrapper[2579]: I0703 11:55:33.330612 2579 log.go:25] "Finished parsing log file" path="/var/log/pods/kuadrant-system_5fc36eb1065777914bfe0ff85f9a202e2a3bafefb563d9e2994474ed73mkshk_e6f9e082-5d6a-4289-9883-301dbffe6151/extract/0.log" Jul 03 11:55:33.360313 ip-10-0-136-0 kubenswrapper[2579]: I0703 11:55:33.360281 2579 log.go:25] "Finished parsing log file" path="/var/log/pods/kuadrant-system_5fc36eb1065777914bfe0ff85f9a202e2a3bafefb563d9e2994474ed73mkshk_e6f9e082-5d6a-4289-9883-301dbffe6151/util/0.log" Jul 03 11:55:33.387153 ip-10-0-136-0 kubenswrapper[2579]: I0703 11:55:33.387069 2579 log.go:25] "Finished parsing log file" path="/var/log/pods/kuadrant-system_5fc36eb1065777914bfe0ff85f9a202e2a3bafefb563d9e2994474ed73mkshk_e6f9e082-5d6a-4289-9883-301dbffe6151/pull/0.log" Jul 03 11:55:33.421717 ip-10-0-136-0 kubenswrapper[2579]: I0703 11:55:33.421683 2579 log.go:25] "Finished parsing log file" path="/var/log/pods/kuadrant-system_9438e18bbad664a016701a3153a6b421e2d977be7eb0117f80bcf45ef1chc8g_5a64e43f-33ee-4779-bc1e-366dc5ef0cc0/extract/0.log" Jul 03 11:55:33.450509 ip-10-0-136-0 kubenswrapper[2579]: I0703 11:55:33.450476 2579 log.go:25] "Finished parsing log file" path="/var/log/pods/kuadrant-system_9438e18bbad664a016701a3153a6b421e2d977be7eb0117f80bcf45ef1chc8g_5a64e43f-33ee-4779-bc1e-366dc5ef0cc0/util/0.log" Jul 03 11:55:33.481104 ip-10-0-136-0 kubenswrapper[2579]: I0703 11:55:33.481070 2579 log.go:25] "Finished parsing log file" path="/var/log/pods/kuadrant-system_9438e18bbad664a016701a3153a6b421e2d977be7eb0117f80bcf45ef1chc8g_5a64e43f-33ee-4779-bc1e-366dc5ef0cc0/pull/0.log" Jul 03 11:55:33.822031 ip-10-0-136-0 kubenswrapper[2579]: I0703 11:55:33.821990 2579 log.go:25] "Finished parsing log file" path="/var/log/pods/kuadrant-system_authorino-55cdcbbc74-wx8bc_49857fe6-d325-4c68-9283-4fbad85410e6/authorino/0.log" Jul 03 11:55:33.856151 ip-10-0-136-0 kubenswrapper[2579]: I0703 11:55:33.856114 2579 log.go:25] "Finished parsing log file" path="/var/log/pods/kuadrant-system_authorino-operator-75cc8d5cb7-nh9mf_0bcdb275-72ef-409c-804c-ca58380c460e/manager/0.log" Jul 03 11:55:33.885305 ip-10-0-136-0 kubenswrapper[2579]: I0703 11:55:33.885270 2579 log.go:25] "Finished parsing log file" path="/var/log/pods/kuadrant-system_dns-operator-controller-manager-864dd78c4c-wz8cc_763fbd50-f445-4a0e-8555-3ffa10d05e56/manager/0.log" Jul 03 11:55:33.968617 ip-10-0-136-0 kubenswrapper[2579]: I0703 11:55:33.968575 2579 log.go:25] "Finished parsing log file" path="/var/log/pods/kuadrant-system_kuadrant-operator-catalog-j29hp_b26c8c7f-99b4-4644-9aee-b33bc9985fe5/registry-server/0.log" Jul 03 11:55:34.099731 ip-10-0-136-0 kubenswrapper[2579]: I0703 11:55:34.099613 2579 log.go:25] "Finished parsing log file" path="/var/log/pods/kuadrant-system_kuadrant-operator-controller-manager-86878c769f-dcjh8_0bbde1f4-7dff-48af-92a8-26c3f6aea3c9/manager/0.log" Jul 03 11:55:34.252241 ip-10-0-136-0 kubenswrapper[2579]: I0703 11:55:34.252182 2579 log.go:25] "Finished parsing log file" path="/var/log/pods/kuadrant-system_limitador-operator-controller-manager-859d7b78bd-qk6jn_73128bfa-b0b4-47ee-b529-ba648905789e/manager/0.log" Jul 03 11:55:36.195403 ip-10-0-136-0 kubenswrapper[2579]: I0703 11:55:36.195355 2579 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_metrics-server-548b5c6d96-hzzhk_29ee781d-d24c-460a-af83-39d31e9dfa0b/metrics-server/0.log" Jul 03 11:55:36.235717 ip-10-0-136-0 kubenswrapper[2579]: I0703 11:55:36.235681 2579 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_monitoring-plugin-d5dcff7cf-nhbzk_ba918183-7a45-4db4-ba86-5cd0455ed4d9/monitoring-plugin/0.log" Jul 03 11:55:36.496569 ip-10-0-136-0 kubenswrapper[2579]: I0703 11:55:36.496475 2579 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_node-exporter-xj7rs_b2866e40-a413-4cc8-978c-65b100d290ac/node-exporter/0.log" Jul 03 11:55:36.533579 ip-10-0-136-0 kubenswrapper[2579]: I0703 11:55:36.533546 2579 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_node-exporter-xj7rs_b2866e40-a413-4cc8-978c-65b100d290ac/kube-rbac-proxy/0.log" Jul 03 11:55:36.559799 ip-10-0-136-0 kubenswrapper[2579]: I0703 11:55:36.559769 2579 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_node-exporter-xj7rs_b2866e40-a413-4cc8-978c-65b100d290ac/init-textfile/0.log" Jul 03 11:55:36.891919 ip-10-0-136-0 kubenswrapper[2579]: I0703 11:55:36.891882 2579 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_prometheus-operator-567f7b8b7c-dft8d_4ab5f9bd-3890-4562-895a-84485c66f4ae/prometheus-operator/0.log" Jul 03 11:55:36.916392 ip-10-0-136-0 kubenswrapper[2579]: I0703 11:55:36.916358 2579 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_prometheus-operator-567f7b8b7c-dft8d_4ab5f9bd-3890-4562-895a-84485c66f4ae/kube-rbac-proxy/0.log" Jul 03 11:55:36.975922 ip-10-0-136-0 kubenswrapper[2579]: I0703 11:55:36.975888 2579 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_telemeter-client-65586ccccd-lcgbt_ace7baa1-5d40-4395-80ba-37e4a232eb20/telemeter-client/0.log" Jul 03 11:55:37.005113 ip-10-0-136-0 kubenswrapper[2579]: I0703 11:55:37.005086 2579 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_telemeter-client-65586ccccd-lcgbt_ace7baa1-5d40-4395-80ba-37e4a232eb20/reload/0.log" Jul 03 11:55:37.031572 ip-10-0-136-0 kubenswrapper[2579]: I0703 11:55:37.031533 2579 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_telemeter-client-65586ccccd-lcgbt_ace7baa1-5d40-4395-80ba-37e4a232eb20/kube-rbac-proxy/0.log" Jul 03 11:55:37.603589 ip-10-0-136-0 kubenswrapper[2579]: I0703 11:55:37.603546 2579 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-must-gather-fgj8l/perf-node-gather-daemonset-l79dj"] Jul 03 11:55:37.603974 ip-10-0-136-0 kubenswrapper[2579]: I0703 11:55:37.603961 2579 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="f4cf29c7-812e-45ca-bf86-80a5cff794ac" containerName="istio-proxy" Jul 03 11:55:37.604018 ip-10-0-136-0 kubenswrapper[2579]: I0703 11:55:37.603978 2579 state_mem.go:107] "Deleted CPUSet assignment" podUID="f4cf29c7-812e-45ca-bf86-80a5cff794ac" containerName="istio-proxy" Jul 03 11:55:37.604018 ip-10-0-136-0 kubenswrapper[2579]: I0703 11:55:37.603991 2579 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="f4cf29c7-812e-45ca-bf86-80a5cff794ac" containerName="istio-proxy" Jul 03 11:55:37.604018 ip-10-0-136-0 kubenswrapper[2579]: I0703 11:55:37.603996 2579 state_mem.go:107] "Deleted CPUSet assignment" podUID="f4cf29c7-812e-45ca-bf86-80a5cff794ac" containerName="istio-proxy" Jul 03 11:55:37.604018 ip-10-0-136-0 kubenswrapper[2579]: I0703 11:55:37.604002 2579 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="11f2fb16-2d82-4f7c-b197-08414de5a06e" containerName="istio-proxy" Jul 03 11:55:37.604018 ip-10-0-136-0 kubenswrapper[2579]: I0703 11:55:37.604007 2579 state_mem.go:107] "Deleted CPUSet assignment" podUID="11f2fb16-2d82-4f7c-b197-08414de5a06e" containerName="istio-proxy" Jul 03 11:55:37.604018 ip-10-0-136-0 kubenswrapper[2579]: I0703 11:55:37.604020 2579 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="6e24dbf3-9a6a-450c-b4f0-0831612d36e6" containerName="main" Jul 03 11:55:37.604217 ip-10-0-136-0 kubenswrapper[2579]: I0703 11:55:37.604026 2579 state_mem.go:107] "Deleted CPUSet assignment" podUID="6e24dbf3-9a6a-450c-b4f0-0831612d36e6" containerName="main" Jul 03 11:55:37.604217 ip-10-0-136-0 kubenswrapper[2579]: I0703 11:55:37.604037 2579 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="11f2fb16-2d82-4f7c-b197-08414de5a06e" containerName="istio-proxy" Jul 03 11:55:37.604217 ip-10-0-136-0 kubenswrapper[2579]: I0703 11:55:37.604043 2579 state_mem.go:107] "Deleted CPUSet assignment" podUID="11f2fb16-2d82-4f7c-b197-08414de5a06e" containerName="istio-proxy" Jul 03 11:55:37.604217 ip-10-0-136-0 kubenswrapper[2579]: I0703 11:55:37.604102 2579 memory_manager.go:368] "RemoveStaleState removing state" podUID="f4cf29c7-812e-45ca-bf86-80a5cff794ac" containerName="istio-proxy" Jul 03 11:55:37.604217 ip-10-0-136-0 kubenswrapper[2579]: I0703 11:55:37.604109 2579 memory_manager.go:368] "RemoveStaleState removing state" podUID="6e24dbf3-9a6a-450c-b4f0-0831612d36e6" containerName="main" Jul 03 11:55:37.604217 ip-10-0-136-0 kubenswrapper[2579]: I0703 11:55:37.604116 2579 memory_manager.go:368] "RemoveStaleState removing state" podUID="11f2fb16-2d82-4f7c-b197-08414de5a06e" containerName="istio-proxy" Jul 03 11:55:37.607443 ip-10-0-136-0 kubenswrapper[2579]: I0703 11:55:37.607418 2579 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-must-gather-fgj8l/perf-node-gather-daemonset-l79dj" Jul 03 11:55:37.609798 ip-10-0-136-0 kubenswrapper[2579]: I0703 11:55:37.609769 2579 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-must-gather-fgj8l\"/\"kube-root-ca.crt\"" Jul 03 11:55:37.609979 ip-10-0-136-0 kubenswrapper[2579]: I0703 11:55:37.609821 2579 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-must-gather-fgj8l\"/\"default-dockercfg-z5qdj\"" Jul 03 11:55:37.610848 ip-10-0-136-0 kubenswrapper[2579]: I0703 11:55:37.610824 2579 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-must-gather-fgj8l\"/\"openshift-service-ca.crt\"" Jul 03 11:55:37.618480 ip-10-0-136-0 kubenswrapper[2579]: I0703 11:55:37.618447 2579 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-must-gather-fgj8l/perf-node-gather-daemonset-l79dj"] Jul 03 11:55:37.709668 ip-10-0-136-0 kubenswrapper[2579]: I0703 11:55:37.709479 2579 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"sys\" (UniqueName: \"kubernetes.io/host-path/f50b538c-54f3-48c8-b0f3-e19dd6f9f8df-sys\") pod \"perf-node-gather-daemonset-l79dj\" (UID: \"f50b538c-54f3-48c8-b0f3-e19dd6f9f8df\") " pod="openshift-must-gather-fgj8l/perf-node-gather-daemonset-l79dj" Jul 03 11:55:37.709668 ip-10-0-136-0 kubenswrapper[2579]: I0703 11:55:37.709565 2579 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"lib-modules\" (UniqueName: \"kubernetes.io/host-path/f50b538c-54f3-48c8-b0f3-e19dd6f9f8df-lib-modules\") pod \"perf-node-gather-daemonset-l79dj\" (UID: \"f50b538c-54f3-48c8-b0f3-e19dd6f9f8df\") " pod="openshift-must-gather-fgj8l/perf-node-gather-daemonset-l79dj" Jul 03 11:55:37.709668 ip-10-0-136-0 kubenswrapper[2579]: I0703 11:55:37.709648 2579 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-hsp2n\" (UniqueName: \"kubernetes.io/projected/f50b538c-54f3-48c8-b0f3-e19dd6f9f8df-kube-api-access-hsp2n\") pod \"perf-node-gather-daemonset-l79dj\" (UID: \"f50b538c-54f3-48c8-b0f3-e19dd6f9f8df\") " pod="openshift-must-gather-fgj8l/perf-node-gather-daemonset-l79dj" Jul 03 11:55:37.709668 ip-10-0-136-0 kubenswrapper[2579]: I0703 11:55:37.709674 2579 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"proc\" (UniqueName: \"kubernetes.io/host-path/f50b538c-54f3-48c8-b0f3-e19dd6f9f8df-proc\") pod \"perf-node-gather-daemonset-l79dj\" (UID: \"f50b538c-54f3-48c8-b0f3-e19dd6f9f8df\") " pod="openshift-must-gather-fgj8l/perf-node-gather-daemonset-l79dj" Jul 03 11:55:37.710081 ip-10-0-136-0 kubenswrapper[2579]: I0703 11:55:37.709715 2579 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"podres\" (UniqueName: \"kubernetes.io/host-path/f50b538c-54f3-48c8-b0f3-e19dd6f9f8df-podres\") pod \"perf-node-gather-daemonset-l79dj\" (UID: \"f50b538c-54f3-48c8-b0f3-e19dd6f9f8df\") " pod="openshift-must-gather-fgj8l/perf-node-gather-daemonset-l79dj" Jul 03 11:55:37.811397 ip-10-0-136-0 kubenswrapper[2579]: I0703 11:55:37.811320 2579 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-hsp2n\" (UniqueName: \"kubernetes.io/projected/f50b538c-54f3-48c8-b0f3-e19dd6f9f8df-kube-api-access-hsp2n\") pod \"perf-node-gather-daemonset-l79dj\" (UID: \"f50b538c-54f3-48c8-b0f3-e19dd6f9f8df\") " pod="openshift-must-gather-fgj8l/perf-node-gather-daemonset-l79dj" Jul 03 11:55:37.811397 ip-10-0-136-0 kubenswrapper[2579]: I0703 11:55:37.811403 2579 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"proc\" (UniqueName: \"kubernetes.io/host-path/f50b538c-54f3-48c8-b0f3-e19dd6f9f8df-proc\") pod \"perf-node-gather-daemonset-l79dj\" (UID: \"f50b538c-54f3-48c8-b0f3-e19dd6f9f8df\") " pod="openshift-must-gather-fgj8l/perf-node-gather-daemonset-l79dj" Jul 03 11:55:37.811740 ip-10-0-136-0 kubenswrapper[2579]: I0703 11:55:37.811458 2579 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"podres\" (UniqueName: \"kubernetes.io/host-path/f50b538c-54f3-48c8-b0f3-e19dd6f9f8df-podres\") pod \"perf-node-gather-daemonset-l79dj\" (UID: \"f50b538c-54f3-48c8-b0f3-e19dd6f9f8df\") " pod="openshift-must-gather-fgj8l/perf-node-gather-daemonset-l79dj" Jul 03 11:55:37.811740 ip-10-0-136-0 kubenswrapper[2579]: I0703 11:55:37.811558 2579 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"proc\" (UniqueName: \"kubernetes.io/host-path/f50b538c-54f3-48c8-b0f3-e19dd6f9f8df-proc\") pod \"perf-node-gather-daemonset-l79dj\" (UID: \"f50b538c-54f3-48c8-b0f3-e19dd6f9f8df\") " pod="openshift-must-gather-fgj8l/perf-node-gather-daemonset-l79dj" Jul 03 11:55:37.811740 ip-10-0-136-0 kubenswrapper[2579]: I0703 11:55:37.811573 2579 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"sys\" (UniqueName: \"kubernetes.io/host-path/f50b538c-54f3-48c8-b0f3-e19dd6f9f8df-sys\") pod \"perf-node-gather-daemonset-l79dj\" (UID: \"f50b538c-54f3-48c8-b0f3-e19dd6f9f8df\") " pod="openshift-must-gather-fgj8l/perf-node-gather-daemonset-l79dj" Jul 03 11:55:37.811740 ip-10-0-136-0 kubenswrapper[2579]: I0703 11:55:37.811631 2579 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"sys\" (UniqueName: \"kubernetes.io/host-path/f50b538c-54f3-48c8-b0f3-e19dd6f9f8df-sys\") pod \"perf-node-gather-daemonset-l79dj\" (UID: \"f50b538c-54f3-48c8-b0f3-e19dd6f9f8df\") " pod="openshift-must-gather-fgj8l/perf-node-gather-daemonset-l79dj" Jul 03 11:55:37.811740 ip-10-0-136-0 kubenswrapper[2579]: I0703 11:55:37.811671 2579 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"lib-modules\" (UniqueName: \"kubernetes.io/host-path/f50b538c-54f3-48c8-b0f3-e19dd6f9f8df-lib-modules\") pod \"perf-node-gather-daemonset-l79dj\" (UID: \"f50b538c-54f3-48c8-b0f3-e19dd6f9f8df\") " pod="openshift-must-gather-fgj8l/perf-node-gather-daemonset-l79dj" Jul 03 11:55:37.811740 ip-10-0-136-0 kubenswrapper[2579]: I0703 11:55:37.811705 2579 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"podres\" (UniqueName: \"kubernetes.io/host-path/f50b538c-54f3-48c8-b0f3-e19dd6f9f8df-podres\") pod \"perf-node-gather-daemonset-l79dj\" (UID: \"f50b538c-54f3-48c8-b0f3-e19dd6f9f8df\") " pod="openshift-must-gather-fgj8l/perf-node-gather-daemonset-l79dj" Jul 03 11:55:37.811953 ip-10-0-136-0 kubenswrapper[2579]: I0703 11:55:37.811824 2579 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"lib-modules\" (UniqueName: \"kubernetes.io/host-path/f50b538c-54f3-48c8-b0f3-e19dd6f9f8df-lib-modules\") pod \"perf-node-gather-daemonset-l79dj\" (UID: \"f50b538c-54f3-48c8-b0f3-e19dd6f9f8df\") " pod="openshift-must-gather-fgj8l/perf-node-gather-daemonset-l79dj" Jul 03 11:55:37.829231 ip-10-0-136-0 kubenswrapper[2579]: I0703 11:55:37.829177 2579 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-hsp2n\" (UniqueName: \"kubernetes.io/projected/f50b538c-54f3-48c8-b0f3-e19dd6f9f8df-kube-api-access-hsp2n\") pod \"perf-node-gather-daemonset-l79dj\" (UID: \"f50b538c-54f3-48c8-b0f3-e19dd6f9f8df\") " pod="openshift-must-gather-fgj8l/perf-node-gather-daemonset-l79dj" Jul 03 11:55:37.919527 ip-10-0-136-0 kubenswrapper[2579]: I0703 11:55:37.919478 2579 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-must-gather-fgj8l/perf-node-gather-daemonset-l79dj" Jul 03 11:55:38.078062 ip-10-0-136-0 kubenswrapper[2579]: I0703 11:55:38.078026 2579 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-must-gather-fgj8l/perf-node-gather-daemonset-l79dj"] Jul 03 11:55:38.080458 ip-10-0-136-0 kubenswrapper[2579]: W0703 11:55:38.080415 2579 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-podf50b538c_54f3_48c8_b0f3_e19dd6f9f8df.slice/crio-9665850b148f58e09582c364ef54f913cd759cff1b4bfc6a25adca643ee6ec83 WatchSource:0}: Error finding container 9665850b148f58e09582c364ef54f913cd759cff1b4bfc6a25adca643ee6ec83: Status 404 returned error can't find the container with id 9665850b148f58e09582c364ef54f913cd759cff1b4bfc6a25adca643ee6ec83 Jul 03 11:55:38.968079 ip-10-0-136-0 kubenswrapper[2579]: I0703 11:55:38.968040 2579 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-must-gather-fgj8l/perf-node-gather-daemonset-l79dj" event={"ID":"f50b538c-54f3-48c8-b0f3-e19dd6f9f8df","Type":"ContainerStarted","Data":"589c9a0e9627879da3468d4a484743a557dbcd2f1d8d823385066653acb491b2"} Jul 03 11:55:38.968527 ip-10-0-136-0 kubenswrapper[2579]: I0703 11:55:38.968098 2579 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-must-gather-fgj8l/perf-node-gather-daemonset-l79dj" event={"ID":"f50b538c-54f3-48c8-b0f3-e19dd6f9f8df","Type":"ContainerStarted","Data":"9665850b148f58e09582c364ef54f913cd759cff1b4bfc6a25adca643ee6ec83"} Jul 03 11:55:38.968527 ip-10-0-136-0 kubenswrapper[2579]: I0703 11:55:38.968176 2579 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-must-gather-fgj8l/perf-node-gather-daemonset-l79dj" Jul 03 11:55:38.987602 ip-10-0-136-0 kubenswrapper[2579]: I0703 11:55:38.987532 2579 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-must-gather-fgj8l/perf-node-gather-daemonset-l79dj" podStartSLOduration=1.9875138589999999 podStartE2EDuration="1.987513859s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=true podCreationTimestamp="2026-07-03 11:55:37 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-03 11:55:38.9841553 +0000 UTC m=+2595.572459622" watchObservedRunningTime="2026-07-03 11:55:38.987513859 +0000 UTC m=+2595.575818185" Jul 03 11:55:39.442222 ip-10-0-136-0 kubenswrapper[2579]: I0703 11:55:39.442153 2579 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-console_console-67c8fbdc89-swl6g_db7db60d-f370-4f46-b2c7-bb2af9788553/console/0.log" Jul 03 11:55:39.478423 ip-10-0-136-0 kubenswrapper[2579]: I0703 11:55:39.478383 2579 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-console_downloads-685bfb5f64-cngk5_a74b2c4b-20bf-4e91-85b7-c827a9c92d1b/download-server/0.log" Jul 03 11:55:41.400595 ip-10-0-136-0 kubenswrapper[2579]: I0703 11:55:41.400568 2579 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-dns_dns-default-tksjt_266035ef-72fb-4911-9110-2806027e1770/dns/0.log" Jul 03 11:55:41.427839 ip-10-0-136-0 kubenswrapper[2579]: I0703 11:55:41.427812 2579 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-dns_dns-default-tksjt_266035ef-72fb-4911-9110-2806027e1770/kube-rbac-proxy/0.log" Jul 03 11:55:41.515771 ip-10-0-136-0 kubenswrapper[2579]: I0703 11:55:41.515741 2579 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-dns_node-resolver-hnmtl_9a6053f3-4885-4e53-8b1c-93150a7546c4/dns-node-resolver/0.log" Jul 03 11:55:42.086806 ip-10-0-136-0 kubenswrapper[2579]: I0703 11:55:42.086767 2579 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-image-registry_image-registry-74fd49c4c-5v265_05698ac0-c3e7-45fe-a431-8ece3158f16d/registry/0.log" Jul 03 11:55:42.111867 ip-10-0-136-0 kubenswrapper[2579]: I0703 11:55:42.111840 2579 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-image-registry_node-ca-9ff9g_f5fa8ffd-d2fc-4474-ad0d-a7af28edf0e7/node-ca/0.log" Jul 03 11:55:43.188228 ip-10-0-136-0 kubenswrapper[2579]: I0703 11:55:43.188173 2579 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ingress_data-science-gateway-data-science-gateway-class-6576859cf4tjmjj_a8b74011-2e4c-43be-bb38-bb662cd50ff3/istio-proxy/0.log" Jul 03 11:55:43.354482 ip-10-0-136-0 kubenswrapper[2579]: I0703 11:55:43.354452 2579 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ingress_kube-auth-proxy-5cb49b9d79-hl6j7_29ccc316-f283-4be9-b247-23d424fbdf1d/kube-auth-proxy/0.log" Jul 03 11:55:43.442852 ip-10-0-136-0 kubenswrapper[2579]: I0703 11:55:43.442770 2579 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ingress_maas-default-gateway-openshift-default-56654dcf7-fbkb8_b8f8d2d9-d723-4d69-b7a5-938e70db757f/istio-proxy/0.log" Jul 03 11:55:43.471889 ip-10-0-136-0 kubenswrapper[2579]: I0703 11:55:43.471850 2579 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ingress_maas-default-gateway-openshift-default-56654dcf7-fbkb8_b8f8d2d9-d723-4d69-b7a5-938e70db757f/istio-proxy/1.log" Jul 03 11:55:43.508313 ip-10-0-136-0 kubenswrapper[2579]: I0703 11:55:43.508272 2579 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ingress_payload-pre-processing-7964457696-ncrll_2c315c89-d57d-4d52-9426-06e88f95bdfb/payload-pre-processing/0.log" Jul 03 11:55:44.287757 ip-10-0-136-0 kubenswrapper[2579]: I0703 11:55:44.287725 2579 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ingress-canary_ingress-canary-2nclq_868c9601-3c66-498e-8076-e7e289b9a4a0/serve-healthcheck-canary/0.log" Jul 03 11:55:44.950266 ip-10-0-136-0 kubenswrapper[2579]: I0703 11:55:44.950228 2579 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-insights_insights-runtime-extractor-ws6w7_f5f5abfc-f7a4-4d0b-be33-bb9dc442904d/kube-rbac-proxy/0.log" Jul 03 11:55:44.978810 ip-10-0-136-0 kubenswrapper[2579]: I0703 11:55:44.978774 2579 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-insights_insights-runtime-extractor-ws6w7_f5f5abfc-f7a4-4d0b-be33-bb9dc442904d/exporter/0.log" Jul 03 11:55:44.985770 ip-10-0-136-0 kubenswrapper[2579]: I0703 11:55:44.985739 2579 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-must-gather-fgj8l/perf-node-gather-daemonset-l79dj" Jul 03 11:55:45.009873 ip-10-0-136-0 kubenswrapper[2579]: I0703 11:55:45.009849 2579 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-insights_insights-runtime-extractor-ws6w7_f5f5abfc-f7a4-4d0b-be33-bb9dc442904d/extractor/0.log" Jul 03 11:55:47.518078 ip-10-0-136-0 kubenswrapper[2579]: I0703 11:55:47.518046 2579 log.go:25] "Finished parsing log file" path="/var/log/pods/opendatahub_kserve-controller-manager-d6f4b65b8-sq7gp_0c97772e-8b5b-4733-bfcc-3a0fd479537d/manager/0.log" Jul 03 11:55:47.675529 ip-10-0-136-0 kubenswrapper[2579]: I0703 11:55:47.675389 2579 log.go:25] "Finished parsing log file" path="/var/log/pods/opendatahub_llmisvc-controller-manager-777c6bd7d5-rsv6f_ad9d71bf-a6e6-42ed-8997-2ddb98b6f21f/manager/0.log" Jul 03 11:55:47.920260 ip-10-0-136-0 kubenswrapper[2579]: I0703 11:55:47.920225 2579 log.go:25] "Finished parsing log file" path="/var/log/pods/opendatahub_model-serving-api-54d89bd5fc-djwk2_bd1ed4fa-6517-40ea-aef4-14b015e858ab/server/0.log" Jul 03 11:55:47.996132 ip-10-0-136-0 kubenswrapper[2579]: I0703 11:55:47.996088 2579 log.go:25] "Finished parsing log file" path="/var/log/pods/opendatahub_odh-model-controller-7998c67dc4-wsz76_4fcd4587-9ab7-43ce-92f6-620f50b11af4/manager/0.log" Jul 03 11:55:48.085815 ip-10-0-136-0 kubenswrapper[2579]: I0703 11:55:48.085727 2579 log.go:25] "Finished parsing log file" path="/var/log/pods/opendatahub_opendatahub-operator-controller-manager-b8c56cd69-ds56h_ff9ecf9d-c9e1-4807-88bc-c5773606274c/manager/0.log" Jul 03 11:55:48.112100 ip-10-0-136-0 kubenswrapper[2579]: I0703 11:55:48.112060 2579 log.go:25] "Finished parsing log file" path="/var/log/pods/opendatahub_opendatahub-operator-controller-manager-b8c56cd69-ds56h_ff9ecf9d-c9e1-4807-88bc-c5773606274c/copy-manifests/0.log" Jul 03 11:55:48.392504 ip-10-0-136-0 kubenswrapper[2579]: I0703 11:55:48.392413 2579 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ingress_data-science-gateway-data-science-gateway-class-6576859cf4tjmjj_a8b74011-2e4c-43be-bb38-bb662cd50ff3/istio-proxy/0.log" Jul 03 11:55:48.523712 ip-10-0-136-0 kubenswrapper[2579]: I0703 11:55:48.523682 2579 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ingress_kube-auth-proxy-5cb49b9d79-hl6j7_29ccc316-f283-4be9-b247-23d424fbdf1d/kube-auth-proxy/0.log" Jul 03 11:55:48.568136 ip-10-0-136-0 kubenswrapper[2579]: I0703 11:55:48.568100 2579 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ingress_maas-default-gateway-openshift-default-56654dcf7-fbkb8_b8f8d2d9-d723-4d69-b7a5-938e70db757f/istio-proxy/0.log" Jul 03 11:55:48.601330 ip-10-0-136-0 kubenswrapper[2579]: I0703 11:55:48.601294 2579 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ingress_maas-default-gateway-openshift-default-56654dcf7-fbkb8_b8f8d2d9-d723-4d69-b7a5-938e70db757f/istio-proxy/1.log" Jul 03 11:55:48.625108 ip-10-0-136-0 kubenswrapper[2579]: I0703 11:55:48.625071 2579 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ingress_payload-pre-processing-7964457696-ncrll_2c315c89-d57d-4d52-9426-06e88f95bdfb/payload-pre-processing/0.log" Jul 03 11:55:49.789702 ip-10-0-136-0 kubenswrapper[2579]: I0703 11:55:49.789675 2579 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-lws-operator_openshift-lws-operator-fd8ccff4c-zb7hd_64beee32-e069-4d34-b14c-f08e6f50dd74/openshift-lws-operator/0.log" Jul 03 11:55:49.930241 ip-10-0-136-0 kubenswrapper[2579]: I0703 11:55:49.930130 2579 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ingress_data-science-gateway-data-science-gateway-class-6576859cf4tjmjj_a8b74011-2e4c-43be-bb38-bb662cd50ff3/istio-proxy/0.log" Jul 03 11:55:50.059530 ip-10-0-136-0 kubenswrapper[2579]: I0703 11:55:50.059501 2579 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ingress_kube-auth-proxy-5cb49b9d79-hl6j7_29ccc316-f283-4be9-b247-23d424fbdf1d/kube-auth-proxy/0.log" Jul 03 11:55:50.104214 ip-10-0-136-0 kubenswrapper[2579]: I0703 11:55:50.104162 2579 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ingress_maas-default-gateway-openshift-default-56654dcf7-fbkb8_b8f8d2d9-d723-4d69-b7a5-938e70db757f/istio-proxy/0.log" Jul 03 11:55:50.143921 ip-10-0-136-0 kubenswrapper[2579]: I0703 11:55:50.143879 2579 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ingress_maas-default-gateway-openshift-default-56654dcf7-fbkb8_b8f8d2d9-d723-4d69-b7a5-938e70db757f/istio-proxy/1.log" Jul 03 11:55:50.169987 ip-10-0-136-0 kubenswrapper[2579]: I0703 11:55:50.169953 2579 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ingress_payload-pre-processing-7964457696-ncrll_2c315c89-d57d-4d52-9426-06e88f95bdfb/payload-pre-processing/0.log"