Jul 10 18:37:58.460389 ip-10-0-143-17 systemd[1]: kubelet.service: Failed to load environment files: No such file or directory Jul 10 18:37:58.460400 ip-10-0-143-17 systemd[1]: kubelet.service: Failed to run 'start-pre' task: No such file or directory Jul 10 18:37:58.460409 ip-10-0-143-17 systemd[1]: kubelet.service: Failed with result 'resources'. Jul 10 18:37:58.460700 ip-10-0-143-17 systemd[1]: Failed to start Kubernetes Kubelet. Jul 10 18:38:08.638549 ip-10-0-143-17 systemd[1]: kubelet.service: Failed to schedule restart job: Unit crio.service not found. Jul 10 18:38:08.638565 ip-10-0-143-17 systemd[1]: kubelet.service: Failed with result 'resources'. -- Boot c93341347feb419faf3c533eefe6b0a4 -- Jul 10 18:39:45.287473 ip-10-0-143-17 systemd[1]: Starting Kubernetes Kubelet... Jul 10 18:39:45.789159 ip-10-0-143-17 kubenswrapper[2585]: Flag --container-runtime-endpoint has been deprecated, This parameter should be set via the config file specified by the Kubelet's --config flag. See https://kubernetes.io/docs/tasks/administer-cluster/kubelet-config-file/ for more information. Jul 10 18:39:45.789159 ip-10-0-143-17 kubenswrapper[2585]: Flag --minimum-container-ttl-duration has been deprecated, Use --eviction-hard or --eviction-soft instead. Will be removed in a future version. Jul 10 18:39:45.789159 ip-10-0-143-17 kubenswrapper[2585]: Flag --volume-plugin-dir has been deprecated, This parameter should be set via the config file specified by the Kubelet's --config flag. See https://kubernetes.io/docs/tasks/administer-cluster/kubelet-config-file/ for more information. Jul 10 18:39:45.789159 ip-10-0-143-17 kubenswrapper[2585]: Flag --pod-infra-container-image has been deprecated, will be removed in 1.35. Image garbage collector will get sandbox image information from CRI. Jul 10 18:39:45.789159 ip-10-0-143-17 kubenswrapper[2585]: Flag --system-reserved has been deprecated, This parameter should be set via the config file specified by the Kubelet's --config flag. See https://kubernetes.io/docs/tasks/administer-cluster/kubelet-config-file/ for more information. Jul 10 18:39:45.789159 ip-10-0-143-17 kubenswrapper[2585]: I0710 18:39:45.735581 2585 server.go:213] "--pod-infra-container-image will not be pruned by the image garbage collector in kubelet and should also be set in the remote runtime" Jul 10 18:39:45.789159 ip-10-0-143-17 kubenswrapper[2585]: W0710 18:39:45.742571 2585 feature_gate.go:328] unrecognized feature gate: NewOLMCatalogdAPIV1Metas Jul 10 18:39:45.789159 ip-10-0-143-17 kubenswrapper[2585]: W0710 18:39:45.742583 2585 feature_gate.go:328] unrecognized feature gate: Example Jul 10 18:39:45.789159 ip-10-0-143-17 kubenswrapper[2585]: W0710 18:39:45.742588 2585 feature_gate.go:328] unrecognized feature gate: MultiArchInstallAzure Jul 10 18:39:45.789159 ip-10-0-143-17 kubenswrapper[2585]: W0710 18:39:45.742600 2585 feature_gate.go:328] unrecognized feature gate: ManagedBootImages Jul 10 18:39:45.789159 ip-10-0-143-17 kubenswrapper[2585]: W0710 18:39:45.742604 2585 feature_gate.go:328] unrecognized feature gate: ClusterAPIMachineManagementVSphere Jul 10 18:39:45.789159 ip-10-0-143-17 kubenswrapper[2585]: W0710 18:39:45.742607 2585 feature_gate.go:328] unrecognized feature gate: ExternalSnapshotMetadata Jul 10 18:39:45.789159 ip-10-0-143-17 kubenswrapper[2585]: W0710 18:39:45.742610 2585 feature_gate.go:328] unrecognized feature gate: AzureMultiDisk Jul 10 18:39:45.789159 ip-10-0-143-17 kubenswrapper[2585]: W0710 18:39:45.742613 2585 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesvSphere Jul 10 18:39:45.809038 ip-10-0-143-17 kubenswrapper[2585]: W0710 18:39:45.742616 2585 feature_gate.go:328] unrecognized feature gate: HyperShiftOnlyDynamicResourceAllocation Jul 10 18:39:45.809038 ip-10-0-143-17 kubenswrapper[2585]: W0710 18:39:45.742619 2585 feature_gate.go:328] unrecognized feature gate: CPMSMachineNamePrefix Jul 10 18:39:45.809038 ip-10-0-143-17 kubenswrapper[2585]: W0710 18:39:45.742622 2585 feature_gate.go:328] unrecognized feature gate: ExternalOIDC Jul 10 18:39:45.809038 ip-10-0-143-17 kubenswrapper[2585]: W0710 18:39:45.742625 2585 feature_gate.go:328] unrecognized feature gate: DualReplica Jul 10 18:39:45.809038 ip-10-0-143-17 kubenswrapper[2585]: W0710 18:39:45.742629 2585 feature_gate.go:328] unrecognized feature gate: MetricsCollectionProfiles Jul 10 18:39:45.809038 ip-10-0-143-17 kubenswrapper[2585]: W0710 18:39:45.742631 2585 feature_gate.go:328] unrecognized feature gate: HighlyAvailableArbiter Jul 10 18:39:45.809038 ip-10-0-143-17 kubenswrapper[2585]: W0710 18:39:45.742634 2585 feature_gate.go:328] unrecognized feature gate: AWSClusterHostedDNSInstall Jul 10 18:39:45.809038 ip-10-0-143-17 kubenswrapper[2585]: W0710 18:39:45.742636 2585 feature_gate.go:328] unrecognized feature gate: AzureWorkloadIdentity Jul 10 18:39:45.809038 ip-10-0-143-17 kubenswrapper[2585]: W0710 18:39:45.742639 2585 feature_gate.go:328] unrecognized feature gate: SigstoreImageVerificationPKI Jul 10 18:39:45.809038 ip-10-0-143-17 kubenswrapper[2585]: W0710 18:39:45.742642 2585 feature_gate.go:328] unrecognized feature gate: ClusterAPIMachineManagement Jul 10 18:39:45.809038 ip-10-0-143-17 kubenswrapper[2585]: W0710 18:39:45.742647 2585 feature_gate.go:351] Setting GA feature gate ServiceAccountTokenNodeBinding=true. It will be removed in a future release. Jul 10 18:39:45.809038 ip-10-0-143-17 kubenswrapper[2585]: W0710 18:39:45.742650 2585 feature_gate.go:328] unrecognized feature gate: InsightsOnDemandDataGather Jul 10 18:39:45.809038 ip-10-0-143-17 kubenswrapper[2585]: W0710 18:39:45.742654 2585 feature_gate.go:328] unrecognized feature gate: IrreconcilableMachineConfig Jul 10 18:39:45.809038 ip-10-0-143-17 kubenswrapper[2585]: W0710 18:39:45.742662 2585 feature_gate.go:328] unrecognized feature gate: NutanixMultiSubnets Jul 10 18:39:45.809038 ip-10-0-143-17 kubenswrapper[2585]: W0710 18:39:45.742666 2585 feature_gate.go:328] unrecognized feature gate: AdminNetworkPolicy Jul 10 18:39:45.809038 ip-10-0-143-17 kubenswrapper[2585]: W0710 18:39:45.742669 2585 feature_gate.go:328] unrecognized feature gate: PreconfiguredUDNAddresses Jul 10 18:39:45.809038 ip-10-0-143-17 kubenswrapper[2585]: W0710 18:39:45.742672 2585 feature_gate.go:328] unrecognized feature gate: VSphereMultiDisk Jul 10 18:39:45.809038 ip-10-0-143-17 kubenswrapper[2585]: W0710 18:39:45.742675 2585 feature_gate.go:328] unrecognized feature gate: BootImageSkewEnforcement Jul 10 18:39:45.809038 ip-10-0-143-17 kubenswrapper[2585]: W0710 18:39:45.742677 2585 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesAWS Jul 10 18:39:45.807145 ip-10-0-143-17 systemd[1]: Started Kubernetes Kubelet. Jul 10 18:39:46.020991 ip-10-0-143-17 kubenswrapper[2585]: W0710 18:39:45.742680 2585 feature_gate.go:328] unrecognized feature gate: SigstoreImageVerification Jul 10 18:39:46.020991 ip-10-0-143-17 kubenswrapper[2585]: W0710 18:39:45.742683 2585 feature_gate.go:328] unrecognized feature gate: EventTTL Jul 10 18:39:46.020991 ip-10-0-143-17 kubenswrapper[2585]: W0710 18:39:45.742686 2585 feature_gate.go:328] unrecognized feature gate: VSphereConfigurableMaxAllowedBlockVolumesPerNode Jul 10 18:39:46.020991 ip-10-0-143-17 kubenswrapper[2585]: W0710 18:39:45.742688 2585 feature_gate.go:328] unrecognized feature gate: DyanmicServiceEndpointIBMCloud Jul 10 18:39:46.020991 ip-10-0-143-17 kubenswrapper[2585]: W0710 18:39:45.742691 2585 feature_gate.go:328] unrecognized feature gate: UpgradeStatus Jul 10 18:39:46.020991 ip-10-0-143-17 kubenswrapper[2585]: W0710 18:39:45.742693 2585 feature_gate.go:328] unrecognized feature gate: ClusterMonitoringConfig Jul 10 18:39:46.020991 ip-10-0-143-17 kubenswrapper[2585]: W0710 18:39:45.742696 2585 feature_gate.go:328] unrecognized feature gate: AWSClusterHostedDNS Jul 10 18:39:46.020991 ip-10-0-143-17 kubenswrapper[2585]: W0710 18:39:45.742704 2585 feature_gate.go:328] unrecognized feature gate: GCPClusterHostedDNS Jul 10 18:39:46.020991 ip-10-0-143-17 kubenswrapper[2585]: W0710 18:39:45.742707 2585 feature_gate.go:328] unrecognized feature gate: AzureClusterHostedDNSInstall Jul 10 18:39:46.020991 ip-10-0-143-17 kubenswrapper[2585]: W0710 18:39:45.742710 2585 feature_gate.go:328] unrecognized feature gate: NewOLM Jul 10 18:39:46.020991 ip-10-0-143-17 kubenswrapper[2585]: W0710 18:39:45.742712 2585 feature_gate.go:328] unrecognized feature gate: EtcdBackendQuota Jul 10 18:39:46.020991 ip-10-0-143-17 kubenswrapper[2585]: W0710 18:39:45.742715 2585 feature_gate.go:328] unrecognized feature gate: ClusterAPIInstall Jul 10 18:39:46.020991 ip-10-0-143-17 kubenswrapper[2585]: W0710 18:39:45.742717 2585 feature_gate.go:328] unrecognized feature gate: AWSServiceLBNetworkSecurityGroup Jul 10 18:39:46.020991 ip-10-0-143-17 kubenswrapper[2585]: W0710 18:39:45.742720 2585 feature_gate.go:328] unrecognized feature gate: MachineAPIOperatorDisableMachineHealthCheckController Jul 10 18:39:46.020991 ip-10-0-143-17 kubenswrapper[2585]: W0710 18:39:45.742722 2585 feature_gate.go:328] unrecognized feature gate: KMSEncryptionProvider Jul 10 18:39:46.020991 ip-10-0-143-17 kubenswrapper[2585]: W0710 18:39:45.742725 2585 feature_gate.go:328] unrecognized feature gate: NewOLMBoxCutterRuntime Jul 10 18:39:46.020991 ip-10-0-143-17 kubenswrapper[2585]: W0710 18:39:45.742728 2585 feature_gate.go:328] unrecognized feature gate: AWSDualStackInstall Jul 10 18:39:46.020991 ip-10-0-143-17 kubenswrapper[2585]: W0710 18:39:45.742730 2585 feature_gate.go:328] unrecognized feature gate: ImageModeStatusReporting Jul 10 18:39:46.020991 ip-10-0-143-17 kubenswrapper[2585]: W0710 18:39:45.742733 2585 feature_gate.go:328] unrecognized feature gate: AzureDualStackInstall Jul 10 18:39:46.020991 ip-10-0-143-17 kubenswrapper[2585]: W0710 18:39:45.742735 2585 feature_gate.go:328] unrecognized feature gate: MultiDiskSetup Jul 10 18:39:46.030001 ip-10-0-143-17 kubenswrapper[2585]: W0710 18:39:45.742738 2585 feature_gate.go:328] unrecognized feature gate: KMSEncryption Jul 10 18:39:46.030001 ip-10-0-143-17 kubenswrapper[2585]: W0710 18:39:45.742741 2585 feature_gate.go:328] unrecognized feature gate: GCPCustomAPIEndpoints Jul 10 18:39:46.030001 ip-10-0-143-17 kubenswrapper[2585]: W0710 18:39:45.742746 2585 feature_gate.go:349] Setting deprecated feature gate KMSv1=true. It will be removed in a future release. Jul 10 18:39:46.030001 ip-10-0-143-17 kubenswrapper[2585]: W0710 18:39:45.742750 2585 feature_gate.go:328] unrecognized feature gate: GCPClusterHostedDNSInstall Jul 10 18:39:46.030001 ip-10-0-143-17 kubenswrapper[2585]: W0710 18:39:45.742754 2585 feature_gate.go:351] Setting GA feature gate VolumeAttributesClass=true. It will be removed in a future release. Jul 10 18:39:46.030001 ip-10-0-143-17 kubenswrapper[2585]: W0710 18:39:45.742758 2585 feature_gate.go:328] unrecognized feature gate: BootcNodeManagement Jul 10 18:39:46.030001 ip-10-0-143-17 kubenswrapper[2585]: W0710 18:39:45.742761 2585 feature_gate.go:328] unrecognized feature gate: ImageStreamImportMode Jul 10 18:39:46.030001 ip-10-0-143-17 kubenswrapper[2585]: W0710 18:39:45.742764 2585 feature_gate.go:328] unrecognized feature gate: NewOLMPreflightPermissionChecks Jul 10 18:39:46.030001 ip-10-0-143-17 kubenswrapper[2585]: W0710 18:39:45.742768 2585 feature_gate.go:328] unrecognized feature gate: PinnedImages Jul 10 18:39:46.030001 ip-10-0-143-17 kubenswrapper[2585]: W0710 18:39:45.742771 2585 feature_gate.go:328] unrecognized feature gate: AlibabaPlatform Jul 10 18:39:46.030001 ip-10-0-143-17 kubenswrapper[2585]: W0710 18:39:45.742774 2585 feature_gate.go:328] unrecognized feature gate: SignatureStores Jul 10 18:39:46.030001 ip-10-0-143-17 kubenswrapper[2585]: W0710 18:39:45.742776 2585 feature_gate.go:328] unrecognized feature gate: ClusterAPIInstallIBMCloud Jul 10 18:39:46.030001 ip-10-0-143-17 kubenswrapper[2585]: W0710 18:39:45.742779 2585 feature_gate.go:328] unrecognized feature gate: VSphereMultiNetworks Jul 10 18:39:46.030001 ip-10-0-143-17 kubenswrapper[2585]: W0710 18:39:45.742781 2585 feature_gate.go:328] unrecognized feature gate: RouteAdvertisements Jul 10 18:39:46.030001 ip-10-0-143-17 kubenswrapper[2585]: W0710 18:39:45.742784 2585 feature_gate.go:328] unrecognized feature gate: Example2 Jul 10 18:39:46.030001 ip-10-0-143-17 kubenswrapper[2585]: W0710 18:39:45.742787 2585 feature_gate.go:328] unrecognized feature gate: GatewayAPI Jul 10 18:39:46.030001 ip-10-0-143-17 kubenswrapper[2585]: W0710 18:39:45.742790 2585 feature_gate.go:328] unrecognized feature gate: AdditionalRoutingCapabilities Jul 10 18:39:46.030001 ip-10-0-143-17 kubenswrapper[2585]: W0710 18:39:45.742793 2585 feature_gate.go:328] unrecognized feature gate: NetworkSegmentation Jul 10 18:39:46.030001 ip-10-0-143-17 kubenswrapper[2585]: W0710 18:39:45.742796 2585 feature_gate.go:328] unrecognized feature gate: VolumeGroupSnapshot Jul 10 18:39:46.030001 ip-10-0-143-17 kubenswrapper[2585]: W0710 18:39:45.742798 2585 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesCPMS Jul 10 18:39:46.033707 ip-10-0-143-17 kubenswrapper[2585]: W0710 18:39:45.742801 2585 feature_gate.go:328] unrecognized feature gate: NewOLMOwnSingleNamespace Jul 10 18:39:46.033707 ip-10-0-143-17 kubenswrapper[2585]: W0710 18:39:45.742804 2585 feature_gate.go:328] unrecognized feature gate: NetworkDiagnosticsConfig Jul 10 18:39:46.033707 ip-10-0-143-17 kubenswrapper[2585]: W0710 18:39:45.742806 2585 feature_gate.go:328] unrecognized feature gate: CRDCompatibilityRequirementOperator Jul 10 18:39:46.033707 ip-10-0-143-17 kubenswrapper[2585]: W0710 18:39:45.742809 2585 feature_gate.go:328] unrecognized feature gate: OpenShiftPodSecurityAdmission Jul 10 18:39:46.033707 ip-10-0-143-17 kubenswrapper[2585]: W0710 18:39:45.742812 2585 feature_gate.go:328] unrecognized feature gate: OSStreams Jul 10 18:39:46.033707 ip-10-0-143-17 kubenswrapper[2585]: W0710 18:39:45.742814 2585 feature_gate.go:328] unrecognized feature gate: MixedCPUsAllocation Jul 10 18:39:46.033707 ip-10-0-143-17 kubenswrapper[2585]: W0710 18:39:45.742817 2585 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesAzure Jul 10 18:39:46.033707 ip-10-0-143-17 kubenswrapper[2585]: W0710 18:39:45.742819 2585 feature_gate.go:328] unrecognized feature gate: VSphereHostVMGroupZonal Jul 10 18:39:46.033707 ip-10-0-143-17 kubenswrapper[2585]: W0710 18:39:45.742822 2585 feature_gate.go:328] unrecognized feature gate: ShortCertRotation Jul 10 18:39:46.033707 ip-10-0-143-17 kubenswrapper[2585]: W0710 18:39:45.742825 2585 feature_gate.go:328] unrecognized feature gate: MachineConfigNodes Jul 10 18:39:46.033707 ip-10-0-143-17 kubenswrapper[2585]: W0710 18:39:45.742827 2585 feature_gate.go:328] unrecognized feature gate: BuildCSIVolumes Jul 10 18:39:46.033707 ip-10-0-143-17 kubenswrapper[2585]: W0710 18:39:45.742830 2585 feature_gate.go:328] unrecognized feature gate: GCPCustomAPIEndpointsInstall Jul 10 18:39:46.033707 ip-10-0-143-17 kubenswrapper[2585]: W0710 18:39:45.742833 2585 feature_gate.go:328] unrecognized feature gate: ProvisioningRequestAvailable Jul 10 18:39:46.033707 ip-10-0-143-17 kubenswrapper[2585]: W0710 18:39:45.742835 2585 feature_gate.go:328] unrecognized feature gate: MachineAPIMigration Jul 10 18:39:46.033707 ip-10-0-143-17 kubenswrapper[2585]: W0710 18:39:45.742837 2585 feature_gate.go:328] unrecognized feature gate: ClusterVersionOperatorConfiguration Jul 10 18:39:46.033707 ip-10-0-143-17 kubenswrapper[2585]: W0710 18:39:45.742840 2585 feature_gate.go:328] unrecognized feature gate: VSphereMixedNodeEnv Jul 10 18:39:46.033707 ip-10-0-143-17 kubenswrapper[2585]: W0710 18:39:45.742842 2585 feature_gate.go:328] unrecognized feature gate: OVNObservability Jul 10 18:39:46.033707 ip-10-0-143-17 kubenswrapper[2585]: W0710 18:39:45.742845 2585 feature_gate.go:328] unrecognized feature gate: AzureDedicatedHosts Jul 10 18:39:46.033707 ip-10-0-143-17 kubenswrapper[2585]: W0710 18:39:45.742847 2585 feature_gate.go:328] unrecognized feature gate: IngressControllerDynamicConfigurationManager Jul 10 18:39:46.035960 ip-10-0-143-17 kubenswrapper[2585]: W0710 18:39:45.742850 2585 feature_gate.go:328] unrecognized feature gate: ConsolePluginContentSecurityPolicy Jul 10 18:39:46.035960 ip-10-0-143-17 kubenswrapper[2585]: W0710 18:39:45.742853 2585 feature_gate.go:328] unrecognized feature gate: NetworkLiveMigration Jul 10 18:39:46.035960 ip-10-0-143-17 kubenswrapper[2585]: W0710 18:39:45.742855 2585 feature_gate.go:328] unrecognized feature gate: GCPDualStackInstall Jul 10 18:39:46.035960 ip-10-0-143-17 kubenswrapper[2585]: W0710 18:39:45.742858 2585 feature_gate.go:328] unrecognized feature gate: OnPremDNSRecords Jul 10 18:39:46.035960 ip-10-0-143-17 kubenswrapper[2585]: W0710 18:39:45.742860 2585 feature_gate.go:328] unrecognized feature gate: ExternalOIDCWithUIDAndExtraClaimMappings Jul 10 18:39:46.035960 ip-10-0-143-17 kubenswrapper[2585]: W0710 18:39:45.742863 2585 feature_gate.go:328] unrecognized feature gate: GatewayAPIController Jul 10 18:39:46.035960 ip-10-0-143-17 kubenswrapper[2585]: W0710 18:39:45.742866 2585 feature_gate.go:328] unrecognized feature gate: NewOLMWebhookProviderOpenshiftServiceCA Jul 10 18:39:46.035960 ip-10-0-143-17 kubenswrapper[2585]: W0710 18:39:45.742868 2585 feature_gate.go:328] unrecognized feature gate: AutomatedEtcdBackup Jul 10 18:39:46.035960 ip-10-0-143-17 kubenswrapper[2585]: W0710 18:39:45.742871 2585 feature_gate.go:328] unrecognized feature gate: InsightsConfig Jul 10 18:39:46.035960 ip-10-0-143-17 kubenswrapper[2585]: W0710 18:39:45.742873 2585 feature_gate.go:328] unrecognized feature gate: AWSDedicatedHosts Jul 10 18:39:46.035960 ip-10-0-143-17 kubenswrapper[2585]: W0710 18:39:45.742876 2585 feature_gate.go:328] unrecognized feature gate: DNSNameResolver Jul 10 18:39:46.035960 ip-10-0-143-17 kubenswrapper[2585]: W0710 18:39:45.742878 2585 feature_gate.go:328] unrecognized feature gate: GatewayAPIWithoutOLM Jul 10 18:39:46.035960 ip-10-0-143-17 kubenswrapper[2585]: W0710 18:39:45.742881 2585 feature_gate.go:328] unrecognized feature gate: NoRegistryClusterInstall Jul 10 18:39:46.035960 ip-10-0-143-17 kubenswrapper[2585]: W0710 18:39:45.743628 2585 feature_gate.go:328] unrecognized feature gate: BootcNodeManagement Jul 10 18:39:46.035960 ip-10-0-143-17 kubenswrapper[2585]: W0710 18:39:45.743635 2585 feature_gate.go:328] unrecognized feature gate: AWSClusterHostedDNS Jul 10 18:39:46.035960 ip-10-0-143-17 kubenswrapper[2585]: W0710 18:39:45.743639 2585 feature_gate.go:328] unrecognized feature gate: MultiArchInstallAzure Jul 10 18:39:46.035960 ip-10-0-143-17 kubenswrapper[2585]: W0710 18:39:45.743642 2585 feature_gate.go:328] unrecognized feature gate: MixedCPUsAllocation Jul 10 18:39:46.035960 ip-10-0-143-17 kubenswrapper[2585]: W0710 18:39:45.743645 2585 feature_gate.go:328] unrecognized feature gate: BootImageSkewEnforcement Jul 10 18:39:46.035960 ip-10-0-143-17 kubenswrapper[2585]: W0710 18:39:45.743648 2585 feature_gate.go:328] unrecognized feature gate: NewOLMBoxCutterRuntime Jul 10 18:39:46.035960 ip-10-0-143-17 kubenswrapper[2585]: W0710 18:39:45.743652 2585 feature_gate.go:328] unrecognized feature gate: GatewayAPI Jul 10 18:39:46.036673 ip-10-0-143-17 kubenswrapper[2585]: W0710 18:39:45.743655 2585 feature_gate.go:328] unrecognized feature gate: KMSEncryptionProvider Jul 10 18:39:46.036673 ip-10-0-143-17 kubenswrapper[2585]: W0710 18:39:45.743658 2585 feature_gate.go:328] unrecognized feature gate: MachineConfigNodes Jul 10 18:39:46.036673 ip-10-0-143-17 kubenswrapper[2585]: W0710 18:39:45.743661 2585 feature_gate.go:328] unrecognized feature gate: GCPDualStackInstall Jul 10 18:39:46.036673 ip-10-0-143-17 kubenswrapper[2585]: W0710 18:39:45.743664 2585 feature_gate.go:328] unrecognized feature gate: SigstoreImageVerification Jul 10 18:39:46.036673 ip-10-0-143-17 kubenswrapper[2585]: W0710 18:39:45.743666 2585 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesAWS Jul 10 18:39:46.036673 ip-10-0-143-17 kubenswrapper[2585]: W0710 18:39:45.743670 2585 feature_gate.go:328] unrecognized feature gate: MachineAPIMigration Jul 10 18:39:46.036673 ip-10-0-143-17 kubenswrapper[2585]: W0710 18:39:45.743673 2585 feature_gate.go:328] unrecognized feature gate: OVNObservability Jul 10 18:39:46.036673 ip-10-0-143-17 kubenswrapper[2585]: W0710 18:39:45.743676 2585 feature_gate.go:328] unrecognized feature gate: ClusterAPIMachineManagementVSphere Jul 10 18:39:46.036673 ip-10-0-143-17 kubenswrapper[2585]: W0710 18:39:45.743678 2585 feature_gate.go:328] unrecognized feature gate: NewOLM Jul 10 18:39:46.036673 ip-10-0-143-17 kubenswrapper[2585]: W0710 18:39:45.743681 2585 feature_gate.go:328] unrecognized feature gate: ShortCertRotation Jul 10 18:39:46.036673 ip-10-0-143-17 kubenswrapper[2585]: W0710 18:39:45.743684 2585 feature_gate.go:328] unrecognized feature gate: AzureMultiDisk Jul 10 18:39:46.036673 ip-10-0-143-17 kubenswrapper[2585]: W0710 18:39:45.743687 2585 feature_gate.go:328] unrecognized feature gate: AWSDedicatedHosts Jul 10 18:39:46.036673 ip-10-0-143-17 kubenswrapper[2585]: W0710 18:39:45.743690 2585 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesvSphere Jul 10 18:39:46.036673 ip-10-0-143-17 kubenswrapper[2585]: W0710 18:39:45.743692 2585 feature_gate.go:328] unrecognized feature gate: VSphereMixedNodeEnv Jul 10 18:39:46.036673 ip-10-0-143-17 kubenswrapper[2585]: W0710 18:39:45.743696 2585 feature_gate.go:328] unrecognized feature gate: ExternalOIDC Jul 10 18:39:46.036673 ip-10-0-143-17 kubenswrapper[2585]: W0710 18:39:45.743698 2585 feature_gate.go:328] unrecognized feature gate: ExternalSnapshotMetadata Jul 10 18:39:46.036673 ip-10-0-143-17 kubenswrapper[2585]: W0710 18:39:45.743701 2585 feature_gate.go:328] unrecognized feature gate: ClusterAPIInstallIBMCloud Jul 10 18:39:46.036673 ip-10-0-143-17 kubenswrapper[2585]: W0710 18:39:45.743704 2585 feature_gate.go:328] unrecognized feature gate: PinnedImages Jul 10 18:39:46.036673 ip-10-0-143-17 kubenswrapper[2585]: W0710 18:39:45.743706 2585 feature_gate.go:328] unrecognized feature gate: AdditionalRoutingCapabilities Jul 10 18:39:46.036673 ip-10-0-143-17 kubenswrapper[2585]: W0710 18:39:45.743709 2585 feature_gate.go:328] unrecognized feature gate: AzureDualStackInstall Jul 10 18:39:46.037331 ip-10-0-143-17 kubenswrapper[2585]: W0710 18:39:45.743711 2585 feature_gate.go:328] unrecognized feature gate: ClusterMonitoringConfig Jul 10 18:39:46.037331 ip-10-0-143-17 kubenswrapper[2585]: W0710 18:39:45.743714 2585 feature_gate.go:328] unrecognized feature gate: KMSEncryption Jul 10 18:39:46.037331 ip-10-0-143-17 kubenswrapper[2585]: W0710 18:39:45.743716 2585 feature_gate.go:328] unrecognized feature gate: AWSServiceLBNetworkSecurityGroup Jul 10 18:39:46.037331 ip-10-0-143-17 kubenswrapper[2585]: W0710 18:39:45.743720 2585 feature_gate.go:351] Setting GA feature gate VolumeAttributesClass=true. It will be removed in a future release. Jul 10 18:39:46.037331 ip-10-0-143-17 kubenswrapper[2585]: W0710 18:39:45.743723 2585 feature_gate.go:328] unrecognized feature gate: MultiDiskSetup Jul 10 18:39:46.037331 ip-10-0-143-17 kubenswrapper[2585]: W0710 18:39:45.743726 2585 feature_gate.go:328] unrecognized feature gate: ProvisioningRequestAvailable Jul 10 18:39:46.037331 ip-10-0-143-17 kubenswrapper[2585]: W0710 18:39:45.743729 2585 feature_gate.go:328] unrecognized feature gate: ImageModeStatusReporting Jul 10 18:39:46.037331 ip-10-0-143-17 kubenswrapper[2585]: W0710 18:39:45.743732 2585 feature_gate.go:328] unrecognized feature gate: DualReplica Jul 10 18:39:46.037331 ip-10-0-143-17 kubenswrapper[2585]: W0710 18:39:45.743734 2585 feature_gate.go:328] unrecognized feature gate: MachineAPIOperatorDisableMachineHealthCheckController Jul 10 18:39:46.037331 ip-10-0-143-17 kubenswrapper[2585]: W0710 18:39:45.743738 2585 feature_gate.go:328] unrecognized feature gate: GatewayAPIWithoutOLM Jul 10 18:39:46.037331 ip-10-0-143-17 kubenswrapper[2585]: W0710 18:39:45.743741 2585 feature_gate.go:328] unrecognized feature gate: NewOLMPreflightPermissionChecks Jul 10 18:39:46.037331 ip-10-0-143-17 kubenswrapper[2585]: W0710 18:39:45.743744 2585 feature_gate.go:328] unrecognized feature gate: ClusterAPIMachineManagement Jul 10 18:39:46.037331 ip-10-0-143-17 kubenswrapper[2585]: W0710 18:39:45.743747 2585 feature_gate.go:328] unrecognized feature gate: DNSNameResolver Jul 10 18:39:46.037331 ip-10-0-143-17 kubenswrapper[2585]: W0710 18:39:45.743749 2585 feature_gate.go:328] unrecognized feature gate: GCPCustomAPIEndpoints Jul 10 18:39:46.037331 ip-10-0-143-17 kubenswrapper[2585]: W0710 18:39:45.743752 2585 feature_gate.go:328] unrecognized feature gate: OpenShiftPodSecurityAdmission Jul 10 18:39:46.037331 ip-10-0-143-17 kubenswrapper[2585]: W0710 18:39:45.743755 2585 feature_gate.go:328] unrecognized feature gate: ManagedBootImages Jul 10 18:39:46.037331 ip-10-0-143-17 kubenswrapper[2585]: W0710 18:39:45.743758 2585 feature_gate.go:328] unrecognized feature gate: NewOLMOwnSingleNamespace Jul 10 18:39:46.037331 ip-10-0-143-17 kubenswrapper[2585]: W0710 18:39:45.743761 2585 feature_gate.go:328] unrecognized feature gate: GatewayAPIController Jul 10 18:39:46.037331 ip-10-0-143-17 kubenswrapper[2585]: W0710 18:39:45.743763 2585 feature_gate.go:328] unrecognized feature gate: DyanmicServiceEndpointIBMCloud Jul 10 18:39:46.037898 ip-10-0-143-17 kubenswrapper[2585]: W0710 18:39:45.743765 2585 feature_gate.go:328] unrecognized feature gate: ClusterAPIInstall Jul 10 18:39:46.037898 ip-10-0-143-17 kubenswrapper[2585]: W0710 18:39:45.743768 2585 feature_gate.go:328] unrecognized feature gate: IrreconcilableMachineConfig Jul 10 18:39:46.037898 ip-10-0-143-17 kubenswrapper[2585]: W0710 18:39:45.743770 2585 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesAzure Jul 10 18:39:46.037898 ip-10-0-143-17 kubenswrapper[2585]: W0710 18:39:45.743773 2585 feature_gate.go:328] unrecognized feature gate: UpgradeStatus Jul 10 18:39:46.037898 ip-10-0-143-17 kubenswrapper[2585]: W0710 18:39:45.743776 2585 feature_gate.go:328] unrecognized feature gate: HighlyAvailableArbiter Jul 10 18:39:46.037898 ip-10-0-143-17 kubenswrapper[2585]: W0710 18:39:45.743779 2585 feature_gate.go:328] unrecognized feature gate: VSphereMultiNetworks Jul 10 18:39:46.037898 ip-10-0-143-17 kubenswrapper[2585]: W0710 18:39:45.743781 2585 feature_gate.go:328] unrecognized feature gate: CRDCompatibilityRequirementOperator Jul 10 18:39:46.037898 ip-10-0-143-17 kubenswrapper[2585]: W0710 18:39:45.743783 2585 feature_gate.go:328] unrecognized feature gate: NetworkLiveMigration Jul 10 18:39:46.037898 ip-10-0-143-17 kubenswrapper[2585]: W0710 18:39:45.743822 2585 feature_gate.go:328] unrecognized feature gate: ConsolePluginContentSecurityPolicy Jul 10 18:39:46.037898 ip-10-0-143-17 kubenswrapper[2585]: W0710 18:39:45.743862 2585 feature_gate.go:328] unrecognized feature gate: CPMSMachineNamePrefix Jul 10 18:39:46.037898 ip-10-0-143-17 kubenswrapper[2585]: W0710 18:39:45.743867 2585 feature_gate.go:328] unrecognized feature gate: OnPremDNSRecords Jul 10 18:39:46.037898 ip-10-0-143-17 kubenswrapper[2585]: W0710 18:39:45.743873 2585 feature_gate.go:328] unrecognized feature gate: GCPClusterHostedDNS Jul 10 18:39:46.037898 ip-10-0-143-17 kubenswrapper[2585]: W0710 18:39:45.744868 2585 feature_gate.go:328] unrecognized feature gate: AzureClusterHostedDNSInstall Jul 10 18:39:46.037898 ip-10-0-143-17 kubenswrapper[2585]: W0710 18:39:45.744876 2585 feature_gate.go:328] unrecognized feature gate: RouteAdvertisements Jul 10 18:39:46.037898 ip-10-0-143-17 kubenswrapper[2585]: W0710 18:39:45.744880 2585 feature_gate.go:328] unrecognized feature gate: VSphereConfigurableMaxAllowedBlockVolumesPerNode Jul 10 18:39:46.037898 ip-10-0-143-17 kubenswrapper[2585]: W0710 18:39:45.744883 2585 feature_gate.go:328] unrecognized feature gate: VolumeGroupSnapshot Jul 10 18:39:46.037898 ip-10-0-143-17 kubenswrapper[2585]: W0710 18:39:45.744887 2585 feature_gate.go:328] unrecognized feature gate: VSphereMultiDisk Jul 10 18:39:46.037898 ip-10-0-143-17 kubenswrapper[2585]: W0710 18:39:45.744890 2585 feature_gate.go:328] unrecognized feature gate: VSphereHostVMGroupZonal Jul 10 18:39:46.037898 ip-10-0-143-17 kubenswrapper[2585]: W0710 18:39:45.744896 2585 feature_gate.go:328] unrecognized feature gate: NutanixMultiSubnets Jul 10 18:39:46.037898 ip-10-0-143-17 kubenswrapper[2585]: W0710 18:39:45.744900 2585 feature_gate.go:328] unrecognized feature gate: Example Jul 10 18:39:46.038554 ip-10-0-143-17 kubenswrapper[2585]: W0710 18:39:45.744904 2585 feature_gate.go:349] Setting deprecated feature gate KMSv1=true. It will be removed in a future release. Jul 10 18:39:46.038554 ip-10-0-143-17 kubenswrapper[2585]: W0710 18:39:45.744912 2585 feature_gate.go:328] unrecognized feature gate: OSStreams Jul 10 18:39:46.038554 ip-10-0-143-17 kubenswrapper[2585]: W0710 18:39:45.744916 2585 feature_gate.go:328] unrecognized feature gate: NewOLMCatalogdAPIV1Metas Jul 10 18:39:46.038554 ip-10-0-143-17 kubenswrapper[2585]: W0710 18:39:45.744919 2585 feature_gate.go:328] unrecognized feature gate: EtcdBackendQuota Jul 10 18:39:46.038554 ip-10-0-143-17 kubenswrapper[2585]: W0710 18:39:45.744923 2585 feature_gate.go:328] unrecognized feature gate: ClusterVersionOperatorConfiguration Jul 10 18:39:46.038554 ip-10-0-143-17 kubenswrapper[2585]: W0710 18:39:45.744925 2585 feature_gate.go:328] unrecognized feature gate: ImageStreamImportMode Jul 10 18:39:46.038554 ip-10-0-143-17 kubenswrapper[2585]: W0710 18:39:45.744928 2585 feature_gate.go:328] unrecognized feature gate: InsightsConfig Jul 10 18:39:46.038554 ip-10-0-143-17 kubenswrapper[2585]: W0710 18:39:45.744931 2585 feature_gate.go:328] unrecognized feature gate: NewOLMWebhookProviderOpenshiftServiceCA Jul 10 18:39:46.038554 ip-10-0-143-17 kubenswrapper[2585]: W0710 18:39:45.744934 2585 feature_gate.go:328] unrecognized feature gate: SigstoreImageVerificationPKI Jul 10 18:39:46.038554 ip-10-0-143-17 kubenswrapper[2585]: W0710 18:39:45.744939 2585 feature_gate.go:328] unrecognized feature gate: AutomatedEtcdBackup Jul 10 18:39:46.038554 ip-10-0-143-17 kubenswrapper[2585]: W0710 18:39:45.744943 2585 feature_gate.go:328] unrecognized feature gate: AdminNetworkPolicy Jul 10 18:39:46.038554 ip-10-0-143-17 kubenswrapper[2585]: W0710 18:39:45.744948 2585 feature_gate.go:328] unrecognized feature gate: AWSClusterHostedDNSInstall Jul 10 18:39:46.038554 ip-10-0-143-17 kubenswrapper[2585]: W0710 18:39:45.744952 2585 feature_gate.go:328] unrecognized feature gate: InsightsOnDemandDataGather Jul 10 18:39:46.038554 ip-10-0-143-17 kubenswrapper[2585]: W0710 18:39:45.744986 2585 feature_gate.go:328] unrecognized feature gate: EventTTL Jul 10 18:39:46.038554 ip-10-0-143-17 kubenswrapper[2585]: W0710 18:39:45.745024 2585 feature_gate.go:328] unrecognized feature gate: HyperShiftOnlyDynamicResourceAllocation Jul 10 18:39:46.038554 ip-10-0-143-17 kubenswrapper[2585]: W0710 18:39:45.745140 2585 feature_gate.go:328] unrecognized feature gate: NetworkSegmentation Jul 10 18:39:46.038554 ip-10-0-143-17 kubenswrapper[2585]: W0710 18:39:45.745147 2585 feature_gate.go:351] Setting GA feature gate ServiceAccountTokenNodeBinding=true. It will be removed in a future release. Jul 10 18:39:46.038554 ip-10-0-143-17 kubenswrapper[2585]: W0710 18:39:45.745153 2585 feature_gate.go:328] unrecognized feature gate: BuildCSIVolumes Jul 10 18:39:46.038554 ip-10-0-143-17 kubenswrapper[2585]: W0710 18:39:45.745156 2585 feature_gate.go:328] unrecognized feature gate: ExternalOIDCWithUIDAndExtraClaimMappings Jul 10 18:39:46.039119 ip-10-0-143-17 kubenswrapper[2585]: W0710 18:39:45.745159 2585 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesCPMS Jul 10 18:39:46.039119 ip-10-0-143-17 kubenswrapper[2585]: W0710 18:39:45.745162 2585 feature_gate.go:328] unrecognized feature gate: IngressControllerDynamicConfigurationManager Jul 10 18:39:46.039119 ip-10-0-143-17 kubenswrapper[2585]: W0710 18:39:45.745165 2585 feature_gate.go:328] unrecognized feature gate: GCPClusterHostedDNSInstall Jul 10 18:39:46.039119 ip-10-0-143-17 kubenswrapper[2585]: W0710 18:39:45.745168 2585 feature_gate.go:328] unrecognized feature gate: Example2 Jul 10 18:39:46.039119 ip-10-0-143-17 kubenswrapper[2585]: W0710 18:39:45.745171 2585 feature_gate.go:328] unrecognized feature gate: AzureWorkloadIdentity Jul 10 18:39:46.039119 ip-10-0-143-17 kubenswrapper[2585]: W0710 18:39:45.745174 2585 feature_gate.go:328] unrecognized feature gate: NetworkDiagnosticsConfig Jul 10 18:39:46.039119 ip-10-0-143-17 kubenswrapper[2585]: W0710 18:39:45.745177 2585 feature_gate.go:328] unrecognized feature gate: AlibabaPlatform Jul 10 18:39:46.039119 ip-10-0-143-17 kubenswrapper[2585]: W0710 18:39:45.745180 2585 feature_gate.go:328] unrecognized feature gate: AzureDedicatedHosts Jul 10 18:39:46.039119 ip-10-0-143-17 kubenswrapper[2585]: W0710 18:39:45.745182 2585 feature_gate.go:328] unrecognized feature gate: GCPCustomAPIEndpointsInstall Jul 10 18:39:46.039119 ip-10-0-143-17 kubenswrapper[2585]: W0710 18:39:45.745185 2585 feature_gate.go:328] unrecognized feature gate: NoRegistryClusterInstall Jul 10 18:39:46.039119 ip-10-0-143-17 kubenswrapper[2585]: W0710 18:39:45.745187 2585 feature_gate.go:328] unrecognized feature gate: MetricsCollectionProfiles Jul 10 18:39:46.039119 ip-10-0-143-17 kubenswrapper[2585]: W0710 18:39:45.745190 2585 feature_gate.go:328] unrecognized feature gate: PreconfiguredUDNAddresses Jul 10 18:39:46.039119 ip-10-0-143-17 kubenswrapper[2585]: W0710 18:39:45.745193 2585 feature_gate.go:328] unrecognized feature gate: SignatureStores Jul 10 18:39:46.039119 ip-10-0-143-17 kubenswrapper[2585]: W0710 18:39:45.745196 2585 feature_gate.go:328] unrecognized feature gate: AWSDualStackInstall Jul 10 18:39:46.039119 ip-10-0-143-17 kubenswrapper[2585]: W0710 18:39:45.745622 2585 feature_gate.go:328] unrecognized feature gate: MultiDiskSetup Jul 10 18:39:46.039119 ip-10-0-143-17 kubenswrapper[2585]: W0710 18:39:45.745628 2585 feature_gate.go:328] unrecognized feature gate: NutanixMultiSubnets Jul 10 18:39:46.039119 ip-10-0-143-17 kubenswrapper[2585]: W0710 18:39:45.745631 2585 feature_gate.go:328] unrecognized feature gate: SignatureStores Jul 10 18:39:46.039119 ip-10-0-143-17 kubenswrapper[2585]: W0710 18:39:45.745637 2585 feature_gate.go:328] unrecognized feature gate: NetworkLiveMigration Jul 10 18:39:46.039119 ip-10-0-143-17 kubenswrapper[2585]: W0710 18:39:45.745640 2585 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesvSphere Jul 10 18:39:46.039119 ip-10-0-143-17 kubenswrapper[2585]: W0710 18:39:45.745643 2585 feature_gate.go:328] unrecognized feature gate: GCPClusterHostedDNS Jul 10 18:39:46.039751 ip-10-0-143-17 kubenswrapper[2585]: W0710 18:39:45.745648 2585 feature_gate.go:351] Setting GA feature gate ServiceAccountTokenNodeBinding=true. It will be removed in a future release. Jul 10 18:39:46.039751 ip-10-0-143-17 kubenswrapper[2585]: W0710 18:39:45.745652 2585 feature_gate.go:328] unrecognized feature gate: ExternalOIDC Jul 10 18:39:46.039751 ip-10-0-143-17 kubenswrapper[2585]: W0710 18:39:45.745655 2585 feature_gate.go:328] unrecognized feature gate: GCPCustomAPIEndpointsInstall Jul 10 18:39:46.039751 ip-10-0-143-17 kubenswrapper[2585]: W0710 18:39:45.745659 2585 feature_gate.go:328] unrecognized feature gate: ExternalOIDCWithUIDAndExtraClaimMappings Jul 10 18:39:46.039751 ip-10-0-143-17 kubenswrapper[2585]: W0710 18:39:45.745662 2585 feature_gate.go:328] unrecognized feature gate: AzureDualStackInstall Jul 10 18:39:46.039751 ip-10-0-143-17 kubenswrapper[2585]: W0710 18:39:45.745665 2585 feature_gate.go:328] unrecognized feature gate: OVNObservability Jul 10 18:39:46.039751 ip-10-0-143-17 kubenswrapper[2585]: W0710 18:39:45.745668 2585 feature_gate.go:328] unrecognized feature gate: NoRegistryClusterInstall Jul 10 18:39:46.039751 ip-10-0-143-17 kubenswrapper[2585]: W0710 18:39:45.745672 2585 feature_gate.go:328] unrecognized feature gate: NewOLMOwnSingleNamespace Jul 10 18:39:46.039751 ip-10-0-143-17 kubenswrapper[2585]: W0710 18:39:45.745675 2585 feature_gate.go:328] unrecognized feature gate: GCPCustomAPIEndpoints Jul 10 18:39:46.039751 ip-10-0-143-17 kubenswrapper[2585]: W0710 18:39:45.745677 2585 feature_gate.go:328] unrecognized feature gate: ManagedBootImages Jul 10 18:39:46.039751 ip-10-0-143-17 kubenswrapper[2585]: W0710 18:39:45.745680 2585 feature_gate.go:328] unrecognized feature gate: KMSEncryption Jul 10 18:39:46.039751 ip-10-0-143-17 kubenswrapper[2585]: W0710 18:39:45.745683 2585 feature_gate.go:328] unrecognized feature gate: EventTTL Jul 10 18:39:46.039751 ip-10-0-143-17 kubenswrapper[2585]: W0710 18:39:45.745685 2585 feature_gate.go:328] unrecognized feature gate: IngressControllerDynamicConfigurationManager Jul 10 18:39:46.039751 ip-10-0-143-17 kubenswrapper[2585]: W0710 18:39:45.745688 2585 feature_gate.go:328] unrecognized feature gate: ClusterAPIMachineManagementVSphere Jul 10 18:39:46.039751 ip-10-0-143-17 kubenswrapper[2585]: W0710 18:39:45.745690 2585 feature_gate.go:328] unrecognized feature gate: ProvisioningRequestAvailable Jul 10 18:39:46.039751 ip-10-0-143-17 kubenswrapper[2585]: W0710 18:39:45.745693 2585 feature_gate.go:328] unrecognized feature gate: NetworkDiagnosticsConfig Jul 10 18:39:46.039751 ip-10-0-143-17 kubenswrapper[2585]: W0710 18:39:45.745695 2585 feature_gate.go:328] unrecognized feature gate: CPMSMachineNamePrefix Jul 10 18:39:46.039751 ip-10-0-143-17 kubenswrapper[2585]: W0710 18:39:45.745698 2585 feature_gate.go:328] unrecognized feature gate: ImageStreamImportMode Jul 10 18:39:46.039751 ip-10-0-143-17 kubenswrapper[2585]: W0710 18:39:45.745700 2585 feature_gate.go:328] unrecognized feature gate: AzureWorkloadIdentity Jul 10 18:39:46.040337 ip-10-0-143-17 kubenswrapper[2585]: W0710 18:39:45.745703 2585 feature_gate.go:328] unrecognized feature gate: ImageModeStatusReporting Jul 10 18:39:46.040337 ip-10-0-143-17 kubenswrapper[2585]: W0710 18:39:45.745705 2585 feature_gate.go:328] unrecognized feature gate: NewOLMWebhookProviderOpenshiftServiceCA Jul 10 18:39:46.040337 ip-10-0-143-17 kubenswrapper[2585]: W0710 18:39:45.745708 2585 feature_gate.go:328] unrecognized feature gate: PinnedImages Jul 10 18:39:46.040337 ip-10-0-143-17 kubenswrapper[2585]: W0710 18:39:45.745710 2585 feature_gate.go:328] unrecognized feature gate: VSphereHostVMGroupZonal Jul 10 18:39:46.040337 ip-10-0-143-17 kubenswrapper[2585]: W0710 18:39:45.745713 2585 feature_gate.go:328] unrecognized feature gate: NewOLMBoxCutterRuntime Jul 10 18:39:46.040337 ip-10-0-143-17 kubenswrapper[2585]: W0710 18:39:45.745715 2585 feature_gate.go:328] unrecognized feature gate: MetricsCollectionProfiles Jul 10 18:39:46.040337 ip-10-0-143-17 kubenswrapper[2585]: W0710 18:39:45.745718 2585 feature_gate.go:328] unrecognized feature gate: GatewayAPIController Jul 10 18:39:46.040337 ip-10-0-143-17 kubenswrapper[2585]: W0710 18:39:45.745720 2585 feature_gate.go:328] unrecognized feature gate: OpenShiftPodSecurityAdmission Jul 10 18:39:46.040337 ip-10-0-143-17 kubenswrapper[2585]: W0710 18:39:45.745723 2585 feature_gate.go:328] unrecognized feature gate: AdminNetworkPolicy Jul 10 18:39:46.040337 ip-10-0-143-17 kubenswrapper[2585]: W0710 18:39:45.745725 2585 feature_gate.go:328] unrecognized feature gate: BootcNodeManagement Jul 10 18:39:46.040337 ip-10-0-143-17 kubenswrapper[2585]: W0710 18:39:45.745729 2585 feature_gate.go:328] unrecognized feature gate: OnPremDNSRecords Jul 10 18:39:46.040337 ip-10-0-143-17 kubenswrapper[2585]: W0710 18:39:45.745732 2585 feature_gate.go:328] unrecognized feature gate: AWSClusterHostedDNSInstall Jul 10 18:39:46.040337 ip-10-0-143-17 kubenswrapper[2585]: W0710 18:39:45.745735 2585 feature_gate.go:328] unrecognized feature gate: Example2 Jul 10 18:39:46.040337 ip-10-0-143-17 kubenswrapper[2585]: W0710 18:39:45.745738 2585 feature_gate.go:328] unrecognized feature gate: UpgradeStatus Jul 10 18:39:46.040337 ip-10-0-143-17 kubenswrapper[2585]: W0710 18:39:45.745741 2585 feature_gate.go:328] unrecognized feature gate: AWSClusterHostedDNS Jul 10 18:39:46.040337 ip-10-0-143-17 kubenswrapper[2585]: W0710 18:39:45.745743 2585 feature_gate.go:328] unrecognized feature gate: RouteAdvertisements Jul 10 18:39:46.040337 ip-10-0-143-17 kubenswrapper[2585]: W0710 18:39:45.745746 2585 feature_gate.go:328] unrecognized feature gate: NewOLMPreflightPermissionChecks Jul 10 18:39:46.040337 ip-10-0-143-17 kubenswrapper[2585]: W0710 18:39:45.745749 2585 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesAWS Jul 10 18:39:46.040337 ip-10-0-143-17 kubenswrapper[2585]: W0710 18:39:45.745753 2585 feature_gate.go:349] Setting deprecated feature gate KMSv1=true. It will be removed in a future release. Jul 10 18:39:46.040337 ip-10-0-143-17 kubenswrapper[2585]: W0710 18:39:45.745756 2585 feature_gate.go:328] unrecognized feature gate: ClusterAPIInstallIBMCloud Jul 10 18:39:46.040941 ip-10-0-143-17 kubenswrapper[2585]: W0710 18:39:45.745759 2585 feature_gate.go:328] unrecognized feature gate: CRDCompatibilityRequirementOperator Jul 10 18:39:46.040941 ip-10-0-143-17 kubenswrapper[2585]: W0710 18:39:45.745762 2585 feature_gate.go:328] unrecognized feature gate: AlibabaPlatform Jul 10 18:39:46.040941 ip-10-0-143-17 kubenswrapper[2585]: W0710 18:39:45.745765 2585 feature_gate.go:328] unrecognized feature gate: AzureMultiDisk Jul 10 18:39:46.040941 ip-10-0-143-17 kubenswrapper[2585]: W0710 18:39:45.745767 2585 feature_gate.go:328] unrecognized feature gate: ExternalSnapshotMetadata Jul 10 18:39:46.040941 ip-10-0-143-17 kubenswrapper[2585]: W0710 18:39:45.745770 2585 feature_gate.go:328] unrecognized feature gate: HighlyAvailableArbiter Jul 10 18:39:46.040941 ip-10-0-143-17 kubenswrapper[2585]: W0710 18:39:45.745772 2585 feature_gate.go:328] unrecognized feature gate: InsightsConfig Jul 10 18:39:46.040941 ip-10-0-143-17 kubenswrapper[2585]: W0710 18:39:45.745775 2585 feature_gate.go:328] unrecognized feature gate: VolumeGroupSnapshot Jul 10 18:39:46.040941 ip-10-0-143-17 kubenswrapper[2585]: W0710 18:39:45.745778 2585 feature_gate.go:328] unrecognized feature gate: MachineConfigNodes Jul 10 18:39:46.040941 ip-10-0-143-17 kubenswrapper[2585]: W0710 18:39:45.745780 2585 feature_gate.go:328] unrecognized feature gate: MachineAPIMigration Jul 10 18:39:46.040941 ip-10-0-143-17 kubenswrapper[2585]: W0710 18:39:45.745783 2585 feature_gate.go:328] unrecognized feature gate: AzureClusterHostedDNSInstall Jul 10 18:39:46.040941 ip-10-0-143-17 kubenswrapper[2585]: W0710 18:39:45.745785 2585 feature_gate.go:328] unrecognized feature gate: ShortCertRotation Jul 10 18:39:46.040941 ip-10-0-143-17 kubenswrapper[2585]: W0710 18:39:45.745788 2585 feature_gate.go:328] unrecognized feature gate: VSphereConfigurableMaxAllowedBlockVolumesPerNode Jul 10 18:39:46.040941 ip-10-0-143-17 kubenswrapper[2585]: W0710 18:39:45.745791 2585 feature_gate.go:328] unrecognized feature gate: BuildCSIVolumes Jul 10 18:39:46.040941 ip-10-0-143-17 kubenswrapper[2585]: W0710 18:39:45.745793 2585 feature_gate.go:328] unrecognized feature gate: AWSServiceLBNetworkSecurityGroup Jul 10 18:39:46.040941 ip-10-0-143-17 kubenswrapper[2585]: W0710 18:39:45.745796 2585 feature_gate.go:328] unrecognized feature gate: SigstoreImageVerificationPKI Jul 10 18:39:46.040941 ip-10-0-143-17 kubenswrapper[2585]: W0710 18:39:45.745798 2585 feature_gate.go:328] unrecognized feature gate: GCPDualStackInstall Jul 10 18:39:46.040941 ip-10-0-143-17 kubenswrapper[2585]: W0710 18:39:45.745801 2585 feature_gate.go:328] unrecognized feature gate: GCPClusterHostedDNSInstall Jul 10 18:39:46.040941 ip-10-0-143-17 kubenswrapper[2585]: W0710 18:39:45.745803 2585 feature_gate.go:328] unrecognized feature gate: ClusterVersionOperatorConfiguration Jul 10 18:39:46.040941 ip-10-0-143-17 kubenswrapper[2585]: W0710 18:39:45.745806 2585 feature_gate.go:328] unrecognized feature gate: AzureDedicatedHosts Jul 10 18:39:46.040941 ip-10-0-143-17 kubenswrapper[2585]: W0710 18:39:45.745808 2585 feature_gate.go:328] unrecognized feature gate: VSphereMixedNodeEnv Jul 10 18:39:46.041512 ip-10-0-143-17 kubenswrapper[2585]: W0710 18:39:45.745810 2585 feature_gate.go:328] unrecognized feature gate: AWSDedicatedHosts Jul 10 18:39:46.041512 ip-10-0-143-17 kubenswrapper[2585]: W0710 18:39:45.745813 2585 feature_gate.go:328] unrecognized feature gate: GatewayAPI Jul 10 18:39:46.041512 ip-10-0-143-17 kubenswrapper[2585]: W0710 18:39:45.745817 2585 feature_gate.go:328] unrecognized feature gate: IrreconcilableMachineConfig Jul 10 18:39:46.041512 ip-10-0-143-17 kubenswrapper[2585]: W0710 18:39:45.745820 2585 feature_gate.go:328] unrecognized feature gate: DyanmicServiceEndpointIBMCloud Jul 10 18:39:46.041512 ip-10-0-143-17 kubenswrapper[2585]: W0710 18:39:45.745823 2585 feature_gate.go:328] unrecognized feature gate: OSStreams Jul 10 18:39:46.041512 ip-10-0-143-17 kubenswrapper[2585]: W0710 18:39:45.745828 2585 feature_gate.go:328] unrecognized feature gate: GatewayAPIWithoutOLM Jul 10 18:39:46.041512 ip-10-0-143-17 kubenswrapper[2585]: W0710 18:39:45.745831 2585 feature_gate.go:328] unrecognized feature gate: SigstoreImageVerification Jul 10 18:39:46.041512 ip-10-0-143-17 kubenswrapper[2585]: W0710 18:39:45.745833 2585 feature_gate.go:328] unrecognized feature gate: BootImageSkewEnforcement Jul 10 18:39:46.041512 ip-10-0-143-17 kubenswrapper[2585]: W0710 18:39:45.745836 2585 feature_gate.go:328] unrecognized feature gate: ClusterAPIMachineManagement Jul 10 18:39:46.041512 ip-10-0-143-17 kubenswrapper[2585]: W0710 18:39:45.745838 2585 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesCPMS Jul 10 18:39:46.041512 ip-10-0-143-17 kubenswrapper[2585]: W0710 18:39:45.745841 2585 feature_gate.go:328] unrecognized feature gate: HyperShiftOnlyDynamicResourceAllocation Jul 10 18:39:46.041512 ip-10-0-143-17 kubenswrapper[2585]: W0710 18:39:45.745843 2585 feature_gate.go:328] unrecognized feature gate: MixedCPUsAllocation Jul 10 18:39:46.041512 ip-10-0-143-17 kubenswrapper[2585]: W0710 18:39:45.745846 2585 feature_gate.go:328] unrecognized feature gate: NetworkSegmentation Jul 10 18:39:46.041512 ip-10-0-143-17 kubenswrapper[2585]: W0710 18:39:45.745849 2585 feature_gate.go:328] unrecognized feature gate: DNSNameResolver Jul 10 18:39:46.041512 ip-10-0-143-17 kubenswrapper[2585]: W0710 18:39:45.745851 2585 feature_gate.go:328] unrecognized feature gate: NewOLMCatalogdAPIV1Metas Jul 10 18:39:46.041512 ip-10-0-143-17 kubenswrapper[2585]: W0710 18:39:45.745854 2585 feature_gate.go:328] unrecognized feature gate: EtcdBackendQuota Jul 10 18:39:46.041512 ip-10-0-143-17 kubenswrapper[2585]: W0710 18:39:45.745856 2585 feature_gate.go:328] unrecognized feature gate: DualReplica Jul 10 18:39:46.041512 ip-10-0-143-17 kubenswrapper[2585]: W0710 18:39:45.745859 2585 feature_gate.go:328] unrecognized feature gate: AutomatedEtcdBackup Jul 10 18:39:46.041512 ip-10-0-143-17 kubenswrapper[2585]: W0710 18:39:45.745861 2585 feature_gate.go:328] unrecognized feature gate: AWSDualStackInstall Jul 10 18:39:46.041512 ip-10-0-143-17 kubenswrapper[2585]: W0710 18:39:45.745864 2585 feature_gate.go:328] unrecognized feature gate: VSphereMultiNetworks Jul 10 18:39:46.042059 ip-10-0-143-17 kubenswrapper[2585]: W0710 18:39:45.745866 2585 feature_gate.go:328] unrecognized feature gate: ClusterAPIInstall Jul 10 18:39:46.042059 ip-10-0-143-17 kubenswrapper[2585]: W0710 18:39:45.745869 2585 feature_gate.go:328] unrecognized feature gate: AdditionalRoutingCapabilities Jul 10 18:39:46.042059 ip-10-0-143-17 kubenswrapper[2585]: W0710 18:39:45.745871 2585 feature_gate.go:328] unrecognized feature gate: MultiArchInstallAzure Jul 10 18:39:46.042059 ip-10-0-143-17 kubenswrapper[2585]: W0710 18:39:45.745873 2585 feature_gate.go:328] unrecognized feature gate: PreconfiguredUDNAddresses Jul 10 18:39:46.042059 ip-10-0-143-17 kubenswrapper[2585]: W0710 18:39:45.745876 2585 feature_gate.go:328] unrecognized feature gate: NewOLM Jul 10 18:39:46.042059 ip-10-0-143-17 kubenswrapper[2585]: W0710 18:39:45.745879 2585 feature_gate.go:328] unrecognized feature gate: MachineAPIOperatorDisableMachineHealthCheckController Jul 10 18:39:46.042059 ip-10-0-143-17 kubenswrapper[2585]: W0710 18:39:45.745883 2585 feature_gate.go:351] Setting GA feature gate VolumeAttributesClass=true. It will be removed in a future release. Jul 10 18:39:46.042059 ip-10-0-143-17 kubenswrapper[2585]: W0710 18:39:45.745886 2585 feature_gate.go:328] unrecognized feature gate: VSphereMultiDisk Jul 10 18:39:46.042059 ip-10-0-143-17 kubenswrapper[2585]: W0710 18:39:45.745889 2585 feature_gate.go:328] unrecognized feature gate: ClusterMonitoringConfig Jul 10 18:39:46.042059 ip-10-0-143-17 kubenswrapper[2585]: W0710 18:39:45.745892 2585 feature_gate.go:328] unrecognized feature gate: InsightsOnDemandDataGather Jul 10 18:39:46.042059 ip-10-0-143-17 kubenswrapper[2585]: W0710 18:39:45.745895 2585 feature_gate.go:328] unrecognized feature gate: Example Jul 10 18:39:46.042059 ip-10-0-143-17 kubenswrapper[2585]: W0710 18:39:45.745898 2585 feature_gate.go:328] unrecognized feature gate: ConsolePluginContentSecurityPolicy Jul 10 18:39:46.042059 ip-10-0-143-17 kubenswrapper[2585]: W0710 18:39:45.745900 2585 feature_gate.go:328] unrecognized feature gate: KMSEncryptionProvider Jul 10 18:39:46.042059 ip-10-0-143-17 kubenswrapper[2585]: W0710 18:39:45.745903 2585 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesAzure Jul 10 18:39:46.042059 ip-10-0-143-17 kubenswrapper[2585]: I0710 18:39:45.745979 2585 flags.go:64] FLAG: --address="0.0.0.0" Jul 10 18:39:46.042059 ip-10-0-143-17 kubenswrapper[2585]: I0710 18:39:45.745986 2585 flags.go:64] FLAG: --allowed-unsafe-sysctls="[]" Jul 10 18:39:46.042059 ip-10-0-143-17 kubenswrapper[2585]: I0710 18:39:45.746008 2585 flags.go:64] FLAG: --anonymous-auth="true" Jul 10 18:39:46.042059 ip-10-0-143-17 kubenswrapper[2585]: I0710 18:39:45.746016 2585 flags.go:64] FLAG: --application-metrics-count-limit="100" Jul 10 18:39:46.042059 ip-10-0-143-17 kubenswrapper[2585]: I0710 18:39:45.746025 2585 flags.go:64] FLAG: --authentication-token-webhook="false" Jul 10 18:39:46.042059 ip-10-0-143-17 kubenswrapper[2585]: I0710 18:39:45.746029 2585 flags.go:64] FLAG: --authentication-token-webhook-cache-ttl="2m0s" Jul 10 18:39:46.042059 ip-10-0-143-17 kubenswrapper[2585]: I0710 18:39:45.746033 2585 flags.go:64] FLAG: --authorization-mode="AlwaysAllow" Jul 10 18:39:46.042679 ip-10-0-143-17 kubenswrapper[2585]: I0710 18:39:45.746040 2585 flags.go:64] FLAG: --authorization-webhook-cache-authorized-ttl="5m0s" Jul 10 18:39:46.042679 ip-10-0-143-17 kubenswrapper[2585]: I0710 18:39:45.746043 2585 flags.go:64] FLAG: --authorization-webhook-cache-unauthorized-ttl="30s" Jul 10 18:39:46.042679 ip-10-0-143-17 kubenswrapper[2585]: I0710 18:39:45.746046 2585 flags.go:64] FLAG: --boot-id-file="/proc/sys/kernel/random/boot_id" Jul 10 18:39:46.042679 ip-10-0-143-17 kubenswrapper[2585]: I0710 18:39:45.746049 2585 flags.go:64] FLAG: --bootstrap-kubeconfig="/etc/kubernetes/kubeconfig" Jul 10 18:39:46.042679 ip-10-0-143-17 kubenswrapper[2585]: I0710 18:39:45.746053 2585 flags.go:64] FLAG: --cert-dir="/var/lib/kubelet/pki" Jul 10 18:39:46.042679 ip-10-0-143-17 kubenswrapper[2585]: I0710 18:39:45.746056 2585 flags.go:64] FLAG: --cgroup-driver="cgroupfs" Jul 10 18:39:46.042679 ip-10-0-143-17 kubenswrapper[2585]: I0710 18:39:45.746059 2585 flags.go:64] FLAG: --cgroup-root="" Jul 10 18:39:46.042679 ip-10-0-143-17 kubenswrapper[2585]: I0710 18:39:45.746061 2585 flags.go:64] FLAG: --cgroups-per-qos="true" Jul 10 18:39:46.042679 ip-10-0-143-17 kubenswrapper[2585]: I0710 18:39:45.746065 2585 flags.go:64] FLAG: --client-ca-file="" Jul 10 18:39:46.042679 ip-10-0-143-17 kubenswrapper[2585]: I0710 18:39:45.746068 2585 flags.go:64] FLAG: --cloud-provider="external" Jul 10 18:39:46.042679 ip-10-0-143-17 kubenswrapper[2585]: I0710 18:39:45.746070 2585 flags.go:64] FLAG: --cluster-dns="[]" Jul 10 18:39:46.042679 ip-10-0-143-17 kubenswrapper[2585]: I0710 18:39:45.746074 2585 flags.go:64] FLAG: --cluster-domain="" Jul 10 18:39:46.042679 ip-10-0-143-17 kubenswrapper[2585]: I0710 18:39:45.746077 2585 flags.go:64] FLAG: --config="/etc/kubernetes/kubelet.conf" Jul 10 18:39:46.042679 ip-10-0-143-17 kubenswrapper[2585]: I0710 18:39:45.746080 2585 flags.go:64] FLAG: --config-dir="/etc/openshift/kubelet.conf.d" Jul 10 18:39:46.042679 ip-10-0-143-17 kubenswrapper[2585]: I0710 18:39:45.746084 2585 flags.go:64] FLAG: --container-hints="/etc/cadvisor/container_hints.json" Jul 10 18:39:46.042679 ip-10-0-143-17 kubenswrapper[2585]: I0710 18:39:45.746087 2585 flags.go:64] FLAG: --container-log-max-files="5" Jul 10 18:39:46.042679 ip-10-0-143-17 kubenswrapper[2585]: I0710 18:39:45.746091 2585 flags.go:64] FLAG: --container-log-max-size="10Mi" Jul 10 18:39:46.042679 ip-10-0-143-17 kubenswrapper[2585]: I0710 18:39:45.746094 2585 flags.go:64] FLAG: --container-runtime-endpoint="/var/run/crio/crio.sock" Jul 10 18:39:46.042679 ip-10-0-143-17 kubenswrapper[2585]: I0710 18:39:45.746097 2585 flags.go:64] FLAG: --containerd="/run/containerd/containerd.sock" Jul 10 18:39:46.042679 ip-10-0-143-17 kubenswrapper[2585]: I0710 18:39:45.746100 2585 flags.go:64] FLAG: --containerd-namespace="k8s.io" Jul 10 18:39:46.042679 ip-10-0-143-17 kubenswrapper[2585]: I0710 18:39:45.746103 2585 flags.go:64] FLAG: --contention-profiling="false" Jul 10 18:39:46.042679 ip-10-0-143-17 kubenswrapper[2585]: I0710 18:39:45.746111 2585 flags.go:64] FLAG: --cpu-cfs-quota="true" Jul 10 18:39:46.042679 ip-10-0-143-17 kubenswrapper[2585]: I0710 18:39:45.746114 2585 flags.go:64] FLAG: --cpu-cfs-quota-period="100ms" Jul 10 18:39:46.042679 ip-10-0-143-17 kubenswrapper[2585]: I0710 18:39:45.746118 2585 flags.go:64] FLAG: --cpu-manager-policy="none" Jul 10 18:39:46.043350 ip-10-0-143-17 kubenswrapper[2585]: I0710 18:39:45.746121 2585 flags.go:64] FLAG: --cpu-manager-policy-options="" Jul 10 18:39:46.043350 ip-10-0-143-17 kubenswrapper[2585]: I0710 18:39:45.746127 2585 flags.go:64] FLAG: --cpu-manager-reconcile-period="10s" Jul 10 18:39:46.043350 ip-10-0-143-17 kubenswrapper[2585]: I0710 18:39:45.746131 2585 flags.go:64] FLAG: --enable-controller-attach-detach="true" Jul 10 18:39:46.043350 ip-10-0-143-17 kubenswrapper[2585]: I0710 18:39:45.746133 2585 flags.go:64] FLAG: --enable-debugging-handlers="true" Jul 10 18:39:46.043350 ip-10-0-143-17 kubenswrapper[2585]: I0710 18:39:45.746138 2585 flags.go:64] FLAG: --enable-load-reader="false" Jul 10 18:39:46.043350 ip-10-0-143-17 kubenswrapper[2585]: I0710 18:39:45.746141 2585 flags.go:64] FLAG: --enable-server="true" Jul 10 18:39:46.043350 ip-10-0-143-17 kubenswrapper[2585]: I0710 18:39:45.746144 2585 flags.go:64] FLAG: --enforce-node-allocatable="[pods]" Jul 10 18:39:46.043350 ip-10-0-143-17 kubenswrapper[2585]: I0710 18:39:45.746150 2585 flags.go:64] FLAG: --event-burst="100" Jul 10 18:39:46.043350 ip-10-0-143-17 kubenswrapper[2585]: I0710 18:39:45.746153 2585 flags.go:64] FLAG: --event-qps="50" Jul 10 18:39:46.043350 ip-10-0-143-17 kubenswrapper[2585]: I0710 18:39:45.746156 2585 flags.go:64] FLAG: --event-storage-age-limit="default=0" Jul 10 18:39:46.043350 ip-10-0-143-17 kubenswrapper[2585]: I0710 18:39:45.746159 2585 flags.go:64] FLAG: --event-storage-event-limit="default=0" Jul 10 18:39:46.043350 ip-10-0-143-17 kubenswrapper[2585]: I0710 18:39:45.746162 2585 flags.go:64] FLAG: --eviction-hard="" Jul 10 18:39:46.043350 ip-10-0-143-17 kubenswrapper[2585]: I0710 18:39:45.746165 2585 flags.go:64] FLAG: --eviction-max-pod-grace-period="0" Jul 10 18:39:46.043350 ip-10-0-143-17 kubenswrapper[2585]: I0710 18:39:45.746168 2585 flags.go:64] FLAG: --eviction-minimum-reclaim="" Jul 10 18:39:46.043350 ip-10-0-143-17 kubenswrapper[2585]: I0710 18:39:45.746171 2585 flags.go:64] FLAG: --eviction-pressure-transition-period="5m0s" Jul 10 18:39:46.043350 ip-10-0-143-17 kubenswrapper[2585]: I0710 18:39:45.746174 2585 flags.go:64] FLAG: --eviction-soft="" Jul 10 18:39:46.043350 ip-10-0-143-17 kubenswrapper[2585]: I0710 18:39:45.746177 2585 flags.go:64] FLAG: --eviction-soft-grace-period="" Jul 10 18:39:46.043350 ip-10-0-143-17 kubenswrapper[2585]: I0710 18:39:45.746180 2585 flags.go:64] FLAG: --exit-on-lock-contention="false" Jul 10 18:39:46.043350 ip-10-0-143-17 kubenswrapper[2585]: I0710 18:39:45.746183 2585 flags.go:64] FLAG: --experimental-allocatable-ignore-eviction="false" Jul 10 18:39:46.043350 ip-10-0-143-17 kubenswrapper[2585]: I0710 18:39:45.746186 2585 flags.go:64] FLAG: --experimental-mounter-path="" Jul 10 18:39:46.043350 ip-10-0-143-17 kubenswrapper[2585]: I0710 18:39:45.746188 2585 flags.go:64] FLAG: --fail-cgroupv1="false" Jul 10 18:39:46.043350 ip-10-0-143-17 kubenswrapper[2585]: I0710 18:39:45.746191 2585 flags.go:64] FLAG: --fail-swap-on="true" Jul 10 18:39:46.043350 ip-10-0-143-17 kubenswrapper[2585]: I0710 18:39:45.746194 2585 flags.go:64] FLAG: --feature-gates="" Jul 10 18:39:46.043350 ip-10-0-143-17 kubenswrapper[2585]: I0710 18:39:45.746198 2585 flags.go:64] FLAG: --file-check-frequency="20s" Jul 10 18:39:46.043350 ip-10-0-143-17 kubenswrapper[2585]: I0710 18:39:45.746200 2585 flags.go:64] FLAG: --global-housekeeping-interval="1m0s" Jul 10 18:39:46.044027 ip-10-0-143-17 kubenswrapper[2585]: I0710 18:39:45.746217 2585 flags.go:64] FLAG: --hairpin-mode="promiscuous-bridge" Jul 10 18:39:46.044027 ip-10-0-143-17 kubenswrapper[2585]: I0710 18:39:45.746220 2585 flags.go:64] FLAG: --healthz-bind-address="127.0.0.1" Jul 10 18:39:46.044027 ip-10-0-143-17 kubenswrapper[2585]: I0710 18:39:45.746223 2585 flags.go:64] FLAG: --healthz-port="10248" Jul 10 18:39:46.044027 ip-10-0-143-17 kubenswrapper[2585]: I0710 18:39:45.746227 2585 flags.go:64] FLAG: --help="false" Jul 10 18:39:46.044027 ip-10-0-143-17 kubenswrapper[2585]: I0710 18:39:45.746229 2585 flags.go:64] FLAG: --hostname-override="ip-10-0-143-17.ec2.internal" Jul 10 18:39:46.044027 ip-10-0-143-17 kubenswrapper[2585]: I0710 18:39:45.746232 2585 flags.go:64] FLAG: --housekeeping-interval="10s" Jul 10 18:39:46.044027 ip-10-0-143-17 kubenswrapper[2585]: I0710 18:39:45.746235 2585 flags.go:64] FLAG: --http-check-frequency="20s" Jul 10 18:39:46.044027 ip-10-0-143-17 kubenswrapper[2585]: I0710 18:39:45.746238 2585 flags.go:64] FLAG: --image-credential-provider-bin-dir="/usr/libexec/kubelet-image-credential-provider-plugins" Jul 10 18:39:46.044027 ip-10-0-143-17 kubenswrapper[2585]: I0710 18:39:45.746244 2585 flags.go:64] FLAG: --image-credential-provider-config="/etc/kubernetes/credential-providers/ecr-credential-provider.yaml" Jul 10 18:39:46.044027 ip-10-0-143-17 kubenswrapper[2585]: I0710 18:39:45.746248 2585 flags.go:64] FLAG: --image-gc-high-threshold="85" Jul 10 18:39:46.044027 ip-10-0-143-17 kubenswrapper[2585]: I0710 18:39:45.746250 2585 flags.go:64] FLAG: --image-gc-low-threshold="80" Jul 10 18:39:46.044027 ip-10-0-143-17 kubenswrapper[2585]: I0710 18:39:45.746254 2585 flags.go:64] FLAG: --image-service-endpoint="" Jul 10 18:39:46.044027 ip-10-0-143-17 kubenswrapper[2585]: I0710 18:39:45.746257 2585 flags.go:64] FLAG: --kernel-memcg-notification="false" Jul 10 18:39:46.044027 ip-10-0-143-17 kubenswrapper[2585]: I0710 18:39:45.746261 2585 flags.go:64] FLAG: --kube-api-burst="100" Jul 10 18:39:46.044027 ip-10-0-143-17 kubenswrapper[2585]: I0710 18:39:45.746265 2585 flags.go:64] FLAG: --kube-api-content-type="application/vnd.kubernetes.protobuf" Jul 10 18:39:46.044027 ip-10-0-143-17 kubenswrapper[2585]: I0710 18:39:45.746268 2585 flags.go:64] FLAG: --kube-api-qps="50" Jul 10 18:39:46.044027 ip-10-0-143-17 kubenswrapper[2585]: I0710 18:39:45.746271 2585 flags.go:64] FLAG: --kube-reserved="" Jul 10 18:39:46.044027 ip-10-0-143-17 kubenswrapper[2585]: I0710 18:39:45.746274 2585 flags.go:64] FLAG: --kube-reserved-cgroup="" Jul 10 18:39:46.044027 ip-10-0-143-17 kubenswrapper[2585]: I0710 18:39:45.746276 2585 flags.go:64] FLAG: --kubeconfig="/var/lib/kubelet/kubeconfig" Jul 10 18:39:46.044027 ip-10-0-143-17 kubenswrapper[2585]: I0710 18:39:45.746279 2585 flags.go:64] FLAG: --kubelet-cgroups="" Jul 10 18:39:46.044027 ip-10-0-143-17 kubenswrapper[2585]: I0710 18:39:45.746282 2585 flags.go:64] FLAG: --local-storage-capacity-isolation="true" Jul 10 18:39:46.044027 ip-10-0-143-17 kubenswrapper[2585]: I0710 18:39:45.746285 2585 flags.go:64] FLAG: --lock-file="" Jul 10 18:39:46.044027 ip-10-0-143-17 kubenswrapper[2585]: I0710 18:39:45.746288 2585 flags.go:64] FLAG: --log-cadvisor-usage="false" Jul 10 18:39:46.044027 ip-10-0-143-17 kubenswrapper[2585]: I0710 18:39:45.746291 2585 flags.go:64] FLAG: --log-flush-frequency="5s" Jul 10 18:39:46.044719 ip-10-0-143-17 kubenswrapper[2585]: I0710 18:39:45.746294 2585 flags.go:64] FLAG: --log-json-info-buffer-size="0" Jul 10 18:39:46.044719 ip-10-0-143-17 kubenswrapper[2585]: I0710 18:39:45.746299 2585 flags.go:64] FLAG: --log-json-split-stream="false" Jul 10 18:39:46.044719 ip-10-0-143-17 kubenswrapper[2585]: I0710 18:39:45.746302 2585 flags.go:64] FLAG: --log-text-info-buffer-size="0" Jul 10 18:39:46.044719 ip-10-0-143-17 kubenswrapper[2585]: I0710 18:39:45.746305 2585 flags.go:64] FLAG: --log-text-split-stream="false" Jul 10 18:39:46.044719 ip-10-0-143-17 kubenswrapper[2585]: I0710 18:39:45.746307 2585 flags.go:64] FLAG: --logging-format="text" Jul 10 18:39:46.044719 ip-10-0-143-17 kubenswrapper[2585]: I0710 18:39:45.746310 2585 flags.go:64] FLAG: --machine-id-file="/etc/machine-id,/var/lib/dbus/machine-id" Jul 10 18:39:46.044719 ip-10-0-143-17 kubenswrapper[2585]: I0710 18:39:45.746313 2585 flags.go:64] FLAG: --make-iptables-util-chains="true" Jul 10 18:39:46.044719 ip-10-0-143-17 kubenswrapper[2585]: I0710 18:39:45.746316 2585 flags.go:64] FLAG: --manifest-url="" Jul 10 18:39:46.044719 ip-10-0-143-17 kubenswrapper[2585]: I0710 18:39:45.746319 2585 flags.go:64] FLAG: --manifest-url-header="" Jul 10 18:39:46.044719 ip-10-0-143-17 kubenswrapper[2585]: I0710 18:39:45.746323 2585 flags.go:64] FLAG: --max-housekeeping-interval="15s" Jul 10 18:39:46.044719 ip-10-0-143-17 kubenswrapper[2585]: I0710 18:39:45.746326 2585 flags.go:64] FLAG: --max-open-files="1000000" Jul 10 18:39:46.044719 ip-10-0-143-17 kubenswrapper[2585]: I0710 18:39:45.746331 2585 flags.go:64] FLAG: --max-pods="110" Jul 10 18:39:46.044719 ip-10-0-143-17 kubenswrapper[2585]: I0710 18:39:45.746334 2585 flags.go:64] FLAG: --maximum-dead-containers="-1" Jul 10 18:39:46.044719 ip-10-0-143-17 kubenswrapper[2585]: I0710 18:39:45.746337 2585 flags.go:64] FLAG: --maximum-dead-containers-per-container="1" Jul 10 18:39:46.044719 ip-10-0-143-17 kubenswrapper[2585]: I0710 18:39:45.746340 2585 flags.go:64] FLAG: --memory-manager-policy="None" Jul 10 18:39:46.044719 ip-10-0-143-17 kubenswrapper[2585]: I0710 18:39:45.746343 2585 flags.go:64] FLAG: --minimum-container-ttl-duration="6m0s" Jul 10 18:39:46.044719 ip-10-0-143-17 kubenswrapper[2585]: I0710 18:39:45.746347 2585 flags.go:64] FLAG: --minimum-image-ttl-duration="2m0s" Jul 10 18:39:46.044719 ip-10-0-143-17 kubenswrapper[2585]: I0710 18:39:45.746350 2585 flags.go:64] FLAG: --node-ip="0.0.0.0" Jul 10 18:39:46.044719 ip-10-0-143-17 kubenswrapper[2585]: I0710 18:39:45.746354 2585 flags.go:64] FLAG: --node-labels="node-role.kubernetes.io/worker=,node.openshift.io/os_id=rhel" Jul 10 18:39:46.044719 ip-10-0-143-17 kubenswrapper[2585]: I0710 18:39:45.746362 2585 flags.go:64] FLAG: --node-status-max-images="50" Jul 10 18:39:46.044719 ip-10-0-143-17 kubenswrapper[2585]: I0710 18:39:45.746365 2585 flags.go:64] FLAG: --node-status-update-frequency="10s" Jul 10 18:39:46.044719 ip-10-0-143-17 kubenswrapper[2585]: I0710 18:39:45.746368 2585 flags.go:64] FLAG: --oom-score-adj="-999" Jul 10 18:39:46.044719 ip-10-0-143-17 kubenswrapper[2585]: I0710 18:39:45.746371 2585 flags.go:64] FLAG: --pod-cidr="" Jul 10 18:39:46.045370 ip-10-0-143-17 kubenswrapper[2585]: I0710 18:39:45.746374 2585 flags.go:64] FLAG: --pod-infra-container-image="quay.io/openshift-release-dev/ocp-v4.0-art-dev@sha256:b04c0b1f73af4d021dd8f0b51bfeb302c24482c093de5f2c43654ab56c506926" Jul 10 18:39:46.045370 ip-10-0-143-17 kubenswrapper[2585]: I0710 18:39:45.746380 2585 flags.go:64] FLAG: --pod-manifest-path="" Jul 10 18:39:46.045370 ip-10-0-143-17 kubenswrapper[2585]: I0710 18:39:45.746383 2585 flags.go:64] FLAG: --pod-max-pids="-1" Jul 10 18:39:46.045370 ip-10-0-143-17 kubenswrapper[2585]: I0710 18:39:45.746386 2585 flags.go:64] FLAG: --pods-per-core="0" Jul 10 18:39:46.045370 ip-10-0-143-17 kubenswrapper[2585]: I0710 18:39:45.746389 2585 flags.go:64] FLAG: --port="10250" Jul 10 18:39:46.045370 ip-10-0-143-17 kubenswrapper[2585]: I0710 18:39:45.746392 2585 flags.go:64] FLAG: --protect-kernel-defaults="false" Jul 10 18:39:46.045370 ip-10-0-143-17 kubenswrapper[2585]: I0710 18:39:45.746395 2585 flags.go:64] FLAG: --provider-id="aws:///us-east-1a/i-06f225d7ec07d0be9" Jul 10 18:39:46.045370 ip-10-0-143-17 kubenswrapper[2585]: I0710 18:39:45.746398 2585 flags.go:64] FLAG: --qos-reserved="" Jul 10 18:39:46.045370 ip-10-0-143-17 kubenswrapper[2585]: I0710 18:39:45.746401 2585 flags.go:64] FLAG: --read-only-port="10255" Jul 10 18:39:46.045370 ip-10-0-143-17 kubenswrapper[2585]: I0710 18:39:45.746404 2585 flags.go:64] FLAG: --register-node="true" Jul 10 18:39:46.045370 ip-10-0-143-17 kubenswrapper[2585]: I0710 18:39:45.746407 2585 flags.go:64] FLAG: --register-with-taints="" Jul 10 18:39:46.045370 ip-10-0-143-17 kubenswrapper[2585]: I0710 18:39:45.746410 2585 flags.go:64] FLAG: --registry-burst="10" Jul 10 18:39:46.045370 ip-10-0-143-17 kubenswrapper[2585]: I0710 18:39:45.746413 2585 flags.go:64] FLAG: --registry-qps="5" Jul 10 18:39:46.045370 ip-10-0-143-17 kubenswrapper[2585]: I0710 18:39:45.746416 2585 flags.go:64] FLAG: --reserved-cpus="" Jul 10 18:39:46.045370 ip-10-0-143-17 kubenswrapper[2585]: I0710 18:39:45.746419 2585 flags.go:64] FLAG: --reserved-memory="" Jul 10 18:39:46.045370 ip-10-0-143-17 kubenswrapper[2585]: I0710 18:39:45.746423 2585 flags.go:64] FLAG: --resolv-conf="/etc/resolv.conf" Jul 10 18:39:46.045370 ip-10-0-143-17 kubenswrapper[2585]: I0710 18:39:45.746426 2585 flags.go:64] FLAG: --root-dir="/var/lib/kubelet" Jul 10 18:39:46.045370 ip-10-0-143-17 kubenswrapper[2585]: I0710 18:39:45.746429 2585 flags.go:64] FLAG: --rotate-certificates="false" Jul 10 18:39:46.045370 ip-10-0-143-17 kubenswrapper[2585]: I0710 18:39:45.746432 2585 flags.go:64] FLAG: --rotate-server-certificates="false" Jul 10 18:39:46.045370 ip-10-0-143-17 kubenswrapper[2585]: I0710 18:39:45.746435 2585 flags.go:64] FLAG: --runonce="false" Jul 10 18:39:46.045370 ip-10-0-143-17 kubenswrapper[2585]: I0710 18:39:45.746437 2585 flags.go:64] FLAG: --runtime-cgroups="/system.slice/crio.service" Jul 10 18:39:46.045370 ip-10-0-143-17 kubenswrapper[2585]: I0710 18:39:45.746440 2585 flags.go:64] FLAG: --runtime-request-timeout="2m0s" Jul 10 18:39:46.045370 ip-10-0-143-17 kubenswrapper[2585]: I0710 18:39:45.746443 2585 flags.go:64] FLAG: --seccomp-default="false" Jul 10 18:39:46.045370 ip-10-0-143-17 kubenswrapper[2585]: I0710 18:39:45.746446 2585 flags.go:64] FLAG: --serialize-image-pulls="true" Jul 10 18:39:46.045370 ip-10-0-143-17 kubenswrapper[2585]: I0710 18:39:45.746449 2585 flags.go:64] FLAG: --storage-driver-buffer-duration="1m0s" Jul 10 18:39:46.046069 ip-10-0-143-17 kubenswrapper[2585]: I0710 18:39:45.746452 2585 flags.go:64] FLAG: --storage-driver-db="cadvisor" Jul 10 18:39:46.046069 ip-10-0-143-17 kubenswrapper[2585]: I0710 18:39:45.746456 2585 flags.go:64] FLAG: --storage-driver-host="localhost:8086" Jul 10 18:39:46.046069 ip-10-0-143-17 kubenswrapper[2585]: I0710 18:39:45.746459 2585 flags.go:64] FLAG: --storage-driver-password="root" Jul 10 18:39:46.046069 ip-10-0-143-17 kubenswrapper[2585]: I0710 18:39:45.746463 2585 flags.go:64] FLAG: --storage-driver-secure="false" Jul 10 18:39:46.046069 ip-10-0-143-17 kubenswrapper[2585]: I0710 18:39:45.746466 2585 flags.go:64] FLAG: --storage-driver-table="stats" Jul 10 18:39:46.046069 ip-10-0-143-17 kubenswrapper[2585]: I0710 18:39:45.746468 2585 flags.go:64] FLAG: --storage-driver-user="root" Jul 10 18:39:46.046069 ip-10-0-143-17 kubenswrapper[2585]: I0710 18:39:45.746471 2585 flags.go:64] FLAG: --streaming-connection-idle-timeout="4h0m0s" Jul 10 18:39:46.046069 ip-10-0-143-17 kubenswrapper[2585]: I0710 18:39:45.746474 2585 flags.go:64] FLAG: --sync-frequency="1m0s" Jul 10 18:39:46.046069 ip-10-0-143-17 kubenswrapper[2585]: I0710 18:39:45.746478 2585 flags.go:64] FLAG: --system-cgroups="" Jul 10 18:39:46.046069 ip-10-0-143-17 kubenswrapper[2585]: I0710 18:39:45.746480 2585 flags.go:64] FLAG: --system-reserved="cpu=500m,ephemeral-storage=1Gi,memory=1Gi" Jul 10 18:39:46.046069 ip-10-0-143-17 kubenswrapper[2585]: I0710 18:39:45.746486 2585 flags.go:64] FLAG: --system-reserved-cgroup="" Jul 10 18:39:46.046069 ip-10-0-143-17 kubenswrapper[2585]: I0710 18:39:45.746488 2585 flags.go:64] FLAG: --tls-cert-file="" Jul 10 18:39:46.046069 ip-10-0-143-17 kubenswrapper[2585]: I0710 18:39:45.746492 2585 flags.go:64] FLAG: --tls-cipher-suites="[]" Jul 10 18:39:46.046069 ip-10-0-143-17 kubenswrapper[2585]: I0710 18:39:45.746496 2585 flags.go:64] FLAG: --tls-min-version="" Jul 10 18:39:46.046069 ip-10-0-143-17 kubenswrapper[2585]: I0710 18:39:45.746499 2585 flags.go:64] FLAG: --tls-private-key-file="" Jul 10 18:39:46.046069 ip-10-0-143-17 kubenswrapper[2585]: I0710 18:39:45.746501 2585 flags.go:64] FLAG: --topology-manager-policy="none" Jul 10 18:39:46.046069 ip-10-0-143-17 kubenswrapper[2585]: I0710 18:39:45.746504 2585 flags.go:64] FLAG: --topology-manager-policy-options="" Jul 10 18:39:46.046069 ip-10-0-143-17 kubenswrapper[2585]: I0710 18:39:45.746507 2585 flags.go:64] FLAG: --topology-manager-scope="container" Jul 10 18:39:46.046069 ip-10-0-143-17 kubenswrapper[2585]: I0710 18:39:45.746510 2585 flags.go:64] FLAG: --v="2" Jul 10 18:39:46.046069 ip-10-0-143-17 kubenswrapper[2585]: I0710 18:39:45.746514 2585 flags.go:64] FLAG: --version="false" Jul 10 18:39:46.046069 ip-10-0-143-17 kubenswrapper[2585]: I0710 18:39:45.746518 2585 flags.go:64] FLAG: --vmodule="" Jul 10 18:39:46.046069 ip-10-0-143-17 kubenswrapper[2585]: I0710 18:39:45.746521 2585 flags.go:64] FLAG: --volume-plugin-dir="/etc/kubernetes/kubelet-plugins/volume/exec" Jul 10 18:39:46.046069 ip-10-0-143-17 kubenswrapper[2585]: I0710 18:39:45.746524 2585 flags.go:64] FLAG: --volume-stats-agg-period="1m0s" Jul 10 18:39:46.046069 ip-10-0-143-17 kubenswrapper[2585]: W0710 18:39:45.746648 2585 feature_gate.go:328] unrecognized feature gate: InsightsConfig Jul 10 18:39:46.046069 ip-10-0-143-17 kubenswrapper[2585]: W0710 18:39:45.746654 2585 feature_gate.go:328] unrecognized feature gate: AzureClusterHostedDNSInstall Jul 10 18:39:46.051537 ip-10-0-143-17 kubenswrapper[2585]: W0710 18:39:45.746657 2585 feature_gate.go:328] unrecognized feature gate: AzureMultiDisk Jul 10 18:39:46.051537 ip-10-0-143-17 kubenswrapper[2585]: W0710 18:39:45.746660 2585 feature_gate.go:328] unrecognized feature gate: AzureDualStackInstall Jul 10 18:39:46.051537 ip-10-0-143-17 kubenswrapper[2585]: W0710 18:39:45.746663 2585 feature_gate.go:328] unrecognized feature gate: NewOLMBoxCutterRuntime Jul 10 18:39:46.051537 ip-10-0-143-17 kubenswrapper[2585]: W0710 18:39:45.746665 2585 feature_gate.go:328] unrecognized feature gate: NewOLMWebhookProviderOpenshiftServiceCA Jul 10 18:39:46.051537 ip-10-0-143-17 kubenswrapper[2585]: W0710 18:39:45.746668 2585 feature_gate.go:328] unrecognized feature gate: GCPClusterHostedDNS Jul 10 18:39:46.051537 ip-10-0-143-17 kubenswrapper[2585]: W0710 18:39:45.746671 2585 feature_gate.go:328] unrecognized feature gate: ConsolePluginContentSecurityPolicy Jul 10 18:39:46.051537 ip-10-0-143-17 kubenswrapper[2585]: W0710 18:39:45.746674 2585 feature_gate.go:351] Setting GA feature gate VolumeAttributesClass=true. It will be removed in a future release. Jul 10 18:39:46.051537 ip-10-0-143-17 kubenswrapper[2585]: W0710 18:39:45.746678 2585 feature_gate.go:328] unrecognized feature gate: MachineAPIMigration Jul 10 18:39:46.051537 ip-10-0-143-17 kubenswrapper[2585]: W0710 18:39:45.746682 2585 feature_gate.go:328] unrecognized feature gate: VSphereConfigurableMaxAllowedBlockVolumesPerNode Jul 10 18:39:46.051537 ip-10-0-143-17 kubenswrapper[2585]: W0710 18:39:45.746685 2585 feature_gate.go:328] unrecognized feature gate: AWSClusterHostedDNS Jul 10 18:39:46.051537 ip-10-0-143-17 kubenswrapper[2585]: W0710 18:39:45.746690 2585 feature_gate.go:328] unrecognized feature gate: MetricsCollectionProfiles Jul 10 18:39:46.051537 ip-10-0-143-17 kubenswrapper[2585]: W0710 18:39:45.746692 2585 feature_gate.go:328] unrecognized feature gate: ClusterVersionOperatorConfiguration Jul 10 18:39:46.051537 ip-10-0-143-17 kubenswrapper[2585]: W0710 18:39:45.746695 2585 feature_gate.go:328] unrecognized feature gate: DualReplica Jul 10 18:39:46.051537 ip-10-0-143-17 kubenswrapper[2585]: W0710 18:39:45.746698 2585 feature_gate.go:328] unrecognized feature gate: ExternalOIDCWithUIDAndExtraClaimMappings Jul 10 18:39:46.051537 ip-10-0-143-17 kubenswrapper[2585]: W0710 18:39:45.746702 2585 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesAWS Jul 10 18:39:46.051537 ip-10-0-143-17 kubenswrapper[2585]: W0710 18:39:45.746704 2585 feature_gate.go:328] unrecognized feature gate: ImageStreamImportMode Jul 10 18:39:46.051537 ip-10-0-143-17 kubenswrapper[2585]: W0710 18:39:45.746708 2585 feature_gate.go:328] unrecognized feature gate: NetworkLiveMigration Jul 10 18:39:46.051537 ip-10-0-143-17 kubenswrapper[2585]: W0710 18:39:45.746711 2585 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesvSphere Jul 10 18:39:46.051537 ip-10-0-143-17 kubenswrapper[2585]: W0710 18:39:45.746713 2585 feature_gate.go:328] unrecognized feature gate: ExternalSnapshotMetadata Jul 10 18:39:46.052106 ip-10-0-143-17 kubenswrapper[2585]: W0710 18:39:45.746716 2585 feature_gate.go:328] unrecognized feature gate: AutomatedEtcdBackup Jul 10 18:39:46.052106 ip-10-0-143-17 kubenswrapper[2585]: W0710 18:39:45.746718 2585 feature_gate.go:328] unrecognized feature gate: NewOLMCatalogdAPIV1Metas Jul 10 18:39:46.052106 ip-10-0-143-17 kubenswrapper[2585]: W0710 18:39:45.746721 2585 feature_gate.go:328] unrecognized feature gate: GatewayAPIController Jul 10 18:39:46.052106 ip-10-0-143-17 kubenswrapper[2585]: W0710 18:39:45.746724 2585 feature_gate.go:328] unrecognized feature gate: SigstoreImageVerification Jul 10 18:39:46.052106 ip-10-0-143-17 kubenswrapper[2585]: W0710 18:39:45.746726 2585 feature_gate.go:328] unrecognized feature gate: SigstoreImageVerificationPKI Jul 10 18:39:46.052106 ip-10-0-143-17 kubenswrapper[2585]: W0710 18:39:45.746729 2585 feature_gate.go:328] unrecognized feature gate: ClusterAPIMachineManagementVSphere Jul 10 18:39:46.052106 ip-10-0-143-17 kubenswrapper[2585]: W0710 18:39:45.746731 2585 feature_gate.go:328] unrecognized feature gate: DyanmicServiceEndpointIBMCloud Jul 10 18:39:46.052106 ip-10-0-143-17 kubenswrapper[2585]: W0710 18:39:45.746734 2585 feature_gate.go:328] unrecognized feature gate: AlibabaPlatform Jul 10 18:39:46.052106 ip-10-0-143-17 kubenswrapper[2585]: W0710 18:39:45.746736 2585 feature_gate.go:328] unrecognized feature gate: NetworkDiagnosticsConfig Jul 10 18:39:46.052106 ip-10-0-143-17 kubenswrapper[2585]: W0710 18:39:45.746739 2585 feature_gate.go:328] unrecognized feature gate: KMSEncryption Jul 10 18:39:46.052106 ip-10-0-143-17 kubenswrapper[2585]: W0710 18:39:45.746742 2585 feature_gate.go:328] unrecognized feature gate: KMSEncryptionProvider Jul 10 18:39:46.052106 ip-10-0-143-17 kubenswrapper[2585]: W0710 18:39:45.746744 2585 feature_gate.go:328] unrecognized feature gate: DNSNameResolver Jul 10 18:39:46.052106 ip-10-0-143-17 kubenswrapper[2585]: W0710 18:39:45.746746 2585 feature_gate.go:328] unrecognized feature gate: ExternalOIDC Jul 10 18:39:46.052106 ip-10-0-143-17 kubenswrapper[2585]: W0710 18:39:45.746749 2585 feature_gate.go:328] unrecognized feature gate: PinnedImages Jul 10 18:39:46.052106 ip-10-0-143-17 kubenswrapper[2585]: W0710 18:39:45.746753 2585 feature_gate.go:349] Setting deprecated feature gate KMSv1=true. It will be removed in a future release. Jul 10 18:39:46.052106 ip-10-0-143-17 kubenswrapper[2585]: W0710 18:39:45.746756 2585 feature_gate.go:328] unrecognized feature gate: AzureDedicatedHosts Jul 10 18:39:46.052106 ip-10-0-143-17 kubenswrapper[2585]: W0710 18:39:45.746759 2585 feature_gate.go:328] unrecognized feature gate: HyperShiftOnlyDynamicResourceAllocation Jul 10 18:39:46.052106 ip-10-0-143-17 kubenswrapper[2585]: W0710 18:39:45.746761 2585 feature_gate.go:328] unrecognized feature gate: AWSDualStackInstall Jul 10 18:39:46.052106 ip-10-0-143-17 kubenswrapper[2585]: W0710 18:39:45.746764 2585 feature_gate.go:328] unrecognized feature gate: VolumeGroupSnapshot Jul 10 18:39:46.052106 ip-10-0-143-17 kubenswrapper[2585]: W0710 18:39:45.746766 2585 feature_gate.go:328] unrecognized feature gate: BuildCSIVolumes Jul 10 18:39:46.259682 ip-10-0-143-17 kubenswrapper[2585]: W0710 18:39:45.746769 2585 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesCPMS Jul 10 18:39:46.259682 ip-10-0-143-17 kubenswrapper[2585]: W0710 18:39:45.746773 2585 feature_gate.go:328] unrecognized feature gate: MixedCPUsAllocation Jul 10 18:39:46.259682 ip-10-0-143-17 kubenswrapper[2585]: W0710 18:39:45.746775 2585 feature_gate.go:328] unrecognized feature gate: OnPremDNSRecords Jul 10 18:39:46.259682 ip-10-0-143-17 kubenswrapper[2585]: W0710 18:39:45.746779 2585 feature_gate.go:328] unrecognized feature gate: NoRegistryClusterInstall Jul 10 18:39:46.259682 ip-10-0-143-17 kubenswrapper[2585]: W0710 18:39:45.746782 2585 feature_gate.go:328] unrecognized feature gate: MachineConfigNodes Jul 10 18:39:46.259682 ip-10-0-143-17 kubenswrapper[2585]: W0710 18:39:45.746784 2585 feature_gate.go:328] unrecognized feature gate: EventTTL Jul 10 18:39:46.259682 ip-10-0-143-17 kubenswrapper[2585]: W0710 18:39:45.746787 2585 feature_gate.go:328] unrecognized feature gate: GCPCustomAPIEndpoints Jul 10 18:39:46.259682 ip-10-0-143-17 kubenswrapper[2585]: W0710 18:39:45.746790 2585 feature_gate.go:328] unrecognized feature gate: NetworkSegmentation Jul 10 18:39:46.259682 ip-10-0-143-17 kubenswrapper[2585]: W0710 18:39:45.746792 2585 feature_gate.go:328] unrecognized feature gate: MultiDiskSetup Jul 10 18:39:46.259682 ip-10-0-143-17 kubenswrapper[2585]: W0710 18:39:45.746794 2585 feature_gate.go:328] unrecognized feature gate: AzureWorkloadIdentity Jul 10 18:39:46.259682 ip-10-0-143-17 kubenswrapper[2585]: W0710 18:39:45.746798 2585 feature_gate.go:328] unrecognized feature gate: GCPClusterHostedDNSInstall Jul 10 18:39:46.259682 ip-10-0-143-17 kubenswrapper[2585]: W0710 18:39:45.746800 2585 feature_gate.go:328] unrecognized feature gate: BootImageSkewEnforcement Jul 10 18:39:46.259682 ip-10-0-143-17 kubenswrapper[2585]: W0710 18:39:45.746803 2585 feature_gate.go:328] unrecognized feature gate: Example2 Jul 10 18:39:46.259682 ip-10-0-143-17 kubenswrapper[2585]: W0710 18:39:45.746805 2585 feature_gate.go:328] unrecognized feature gate: OVNObservability Jul 10 18:39:46.259682 ip-10-0-143-17 kubenswrapper[2585]: W0710 18:39:45.746808 2585 feature_gate.go:328] unrecognized feature gate: ProvisioningRequestAvailable Jul 10 18:39:46.259682 ip-10-0-143-17 kubenswrapper[2585]: W0710 18:39:45.746810 2585 feature_gate.go:328] unrecognized feature gate: VSphereHostVMGroupZonal Jul 10 18:39:46.259682 ip-10-0-143-17 kubenswrapper[2585]: W0710 18:39:45.746813 2585 feature_gate.go:328] unrecognized feature gate: ClusterMonitoringConfig Jul 10 18:39:46.259682 ip-10-0-143-17 kubenswrapper[2585]: W0710 18:39:45.746815 2585 feature_gate.go:328] unrecognized feature gate: ImageModeStatusReporting Jul 10 18:39:46.259682 ip-10-0-143-17 kubenswrapper[2585]: W0710 18:39:45.746818 2585 feature_gate.go:328] unrecognized feature gate: OSStreams Jul 10 18:39:46.259682 ip-10-0-143-17 kubenswrapper[2585]: W0710 18:39:45.746820 2585 feature_gate.go:328] unrecognized feature gate: AdminNetworkPolicy Jul 10 18:39:46.260390 ip-10-0-143-17 kubenswrapper[2585]: W0710 18:39:45.746823 2585 feature_gate.go:328] unrecognized feature gate: AWSServiceLBNetworkSecurityGroup Jul 10 18:39:46.260390 ip-10-0-143-17 kubenswrapper[2585]: W0710 18:39:45.746826 2585 feature_gate.go:328] unrecognized feature gate: GatewayAPI Jul 10 18:39:46.260390 ip-10-0-143-17 kubenswrapper[2585]: W0710 18:39:45.746828 2585 feature_gate.go:328] unrecognized feature gate: MachineAPIOperatorDisableMachineHealthCheckController Jul 10 18:39:46.260390 ip-10-0-143-17 kubenswrapper[2585]: W0710 18:39:45.746831 2585 feature_gate.go:328] unrecognized feature gate: ShortCertRotation Jul 10 18:39:46.260390 ip-10-0-143-17 kubenswrapper[2585]: W0710 18:39:45.746834 2585 feature_gate.go:328] unrecognized feature gate: GCPDualStackInstall Jul 10 18:39:46.260390 ip-10-0-143-17 kubenswrapper[2585]: W0710 18:39:45.746836 2585 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesAzure Jul 10 18:39:46.260390 ip-10-0-143-17 kubenswrapper[2585]: W0710 18:39:45.746839 2585 feature_gate.go:328] unrecognized feature gate: NutanixMultiSubnets Jul 10 18:39:46.260390 ip-10-0-143-17 kubenswrapper[2585]: W0710 18:39:45.746842 2585 feature_gate.go:328] unrecognized feature gate: ManagedBootImages Jul 10 18:39:46.260390 ip-10-0-143-17 kubenswrapper[2585]: W0710 18:39:45.746844 2585 feature_gate.go:328] unrecognized feature gate: HighlyAvailableArbiter Jul 10 18:39:46.260390 ip-10-0-143-17 kubenswrapper[2585]: W0710 18:39:45.746847 2585 feature_gate.go:328] unrecognized feature gate: GCPCustomAPIEndpointsInstall Jul 10 18:39:46.260390 ip-10-0-143-17 kubenswrapper[2585]: W0710 18:39:45.746849 2585 feature_gate.go:328] unrecognized feature gate: EtcdBackendQuota Jul 10 18:39:46.260390 ip-10-0-143-17 kubenswrapper[2585]: W0710 18:39:45.746851 2585 feature_gate.go:328] unrecognized feature gate: NewOLM Jul 10 18:39:46.260390 ip-10-0-143-17 kubenswrapper[2585]: W0710 18:39:45.746854 2585 feature_gate.go:328] unrecognized feature gate: VSphereMixedNodeEnv Jul 10 18:39:46.260390 ip-10-0-143-17 kubenswrapper[2585]: W0710 18:39:45.746857 2585 feature_gate.go:328] unrecognized feature gate: CPMSMachineNamePrefix Jul 10 18:39:46.260390 ip-10-0-143-17 kubenswrapper[2585]: W0710 18:39:45.746860 2585 feature_gate.go:328] unrecognized feature gate: SignatureStores Jul 10 18:39:46.260390 ip-10-0-143-17 kubenswrapper[2585]: W0710 18:39:45.746864 2585 feature_gate.go:328] unrecognized feature gate: NewOLMOwnSingleNamespace Jul 10 18:39:46.260390 ip-10-0-143-17 kubenswrapper[2585]: W0710 18:39:45.746867 2585 feature_gate.go:328] unrecognized feature gate: Example Jul 10 18:39:46.260390 ip-10-0-143-17 kubenswrapper[2585]: W0710 18:39:45.746869 2585 feature_gate.go:328] unrecognized feature gate: CRDCompatibilityRequirementOperator Jul 10 18:39:46.260390 ip-10-0-143-17 kubenswrapper[2585]: W0710 18:39:45.746872 2585 feature_gate.go:328] unrecognized feature gate: MultiArchInstallAzure Jul 10 18:39:46.260390 ip-10-0-143-17 kubenswrapper[2585]: W0710 18:39:45.746874 2585 feature_gate.go:328] unrecognized feature gate: IrreconcilableMachineConfig Jul 10 18:39:46.260969 ip-10-0-143-17 kubenswrapper[2585]: W0710 18:39:45.746877 2585 feature_gate.go:328] unrecognized feature gate: GatewayAPIWithoutOLM Jul 10 18:39:46.260969 ip-10-0-143-17 kubenswrapper[2585]: W0710 18:39:45.746879 2585 feature_gate.go:328] unrecognized feature gate: ClusterAPIMachineManagement Jul 10 18:39:46.260969 ip-10-0-143-17 kubenswrapper[2585]: W0710 18:39:45.746882 2585 feature_gate.go:328] unrecognized feature gate: PreconfiguredUDNAddresses Jul 10 18:39:46.260969 ip-10-0-143-17 kubenswrapper[2585]: W0710 18:39:45.746891 2585 feature_gate.go:328] unrecognized feature gate: IngressControllerDynamicConfigurationManager Jul 10 18:39:46.260969 ip-10-0-143-17 kubenswrapper[2585]: W0710 18:39:45.746893 2585 feature_gate.go:328] unrecognized feature gate: VSphereMultiDisk Jul 10 18:39:46.260969 ip-10-0-143-17 kubenswrapper[2585]: W0710 18:39:45.746896 2585 feature_gate.go:328] unrecognized feature gate: UpgradeStatus Jul 10 18:39:46.260969 ip-10-0-143-17 kubenswrapper[2585]: W0710 18:39:45.746899 2585 feature_gate.go:328] unrecognized feature gate: RouteAdvertisements Jul 10 18:39:46.260969 ip-10-0-143-17 kubenswrapper[2585]: W0710 18:39:45.746903 2585 feature_gate.go:351] Setting GA feature gate ServiceAccountTokenNodeBinding=true. It will be removed in a future release. Jul 10 18:39:46.260969 ip-10-0-143-17 kubenswrapper[2585]: W0710 18:39:45.746906 2585 feature_gate.go:328] unrecognized feature gate: AdditionalRoutingCapabilities Jul 10 18:39:46.260969 ip-10-0-143-17 kubenswrapper[2585]: W0710 18:39:45.746909 2585 feature_gate.go:328] unrecognized feature gate: NewOLMPreflightPermissionChecks Jul 10 18:39:46.260969 ip-10-0-143-17 kubenswrapper[2585]: W0710 18:39:45.746912 2585 feature_gate.go:328] unrecognized feature gate: ClusterAPIInstallIBMCloud Jul 10 18:39:46.260969 ip-10-0-143-17 kubenswrapper[2585]: W0710 18:39:45.746915 2585 feature_gate.go:328] unrecognized feature gate: AWSDedicatedHosts Jul 10 18:39:46.260969 ip-10-0-143-17 kubenswrapper[2585]: W0710 18:39:45.746918 2585 feature_gate.go:328] unrecognized feature gate: InsightsOnDemandDataGather Jul 10 18:39:46.260969 ip-10-0-143-17 kubenswrapper[2585]: W0710 18:39:45.746921 2585 feature_gate.go:328] unrecognized feature gate: ClusterAPIInstall Jul 10 18:39:46.260969 ip-10-0-143-17 kubenswrapper[2585]: W0710 18:39:45.746923 2585 feature_gate.go:328] unrecognized feature gate: VSphereMultiNetworks Jul 10 18:39:46.260969 ip-10-0-143-17 kubenswrapper[2585]: W0710 18:39:45.746926 2585 feature_gate.go:328] unrecognized feature gate: AWSClusterHostedDNSInstall Jul 10 18:39:46.260969 ip-10-0-143-17 kubenswrapper[2585]: W0710 18:39:45.746929 2585 feature_gate.go:328] unrecognized feature gate: BootcNodeManagement Jul 10 18:39:46.260969 ip-10-0-143-17 kubenswrapper[2585]: W0710 18:39:45.746931 2585 feature_gate.go:328] unrecognized feature gate: OpenShiftPodSecurityAdmission Jul 10 18:39:46.261626 ip-10-0-143-17 kubenswrapper[2585]: I0710 18:39:45.747640 2585 feature_gate.go:384] feature gates: {map[CBORServingAndStorage:false ClientsAllowCBOR:false ClientsPreferCBOR:false EventedPLEG:false ImageVolume:true KMSv1:true MaxUnavailableStatefulSet:false MinimumKubeletVersion:false MutableCSINodeAllocatableCount:false MutatingAdmissionPolicy:false ProcMountType:true RouteExternalCertificate:true SELinuxMount:false ServiceAccountTokenNodeBinding:true StoragePerformantSecurityPolicy:true TranslateStreamCloseWebsocketRequests:false UserNamespacesPodSecurityStandards:true UserNamespacesSupport:true VolumeAttributesClass:true]} Jul 10 18:39:46.261626 ip-10-0-143-17 kubenswrapper[2585]: I0710 18:39:45.755119 2585 server.go:529] "Kubelet version" kubeletVersion="v1.34.9" Jul 10 18:39:46.261626 ip-10-0-143-17 kubenswrapper[2585]: I0710 18:39:45.755135 2585 server.go:531] "Golang settings" GOGC="" GOMAXPROCS="" GOTRACEBACK="" Jul 10 18:39:46.261626 ip-10-0-143-17 kubenswrapper[2585]: I0710 18:39:45.755155 2585 watchdog_linux.go:95] "Systemd watchdog is not enabled" Jul 10 18:39:46.261626 ip-10-0-143-17 kubenswrapper[2585]: I0710 18:39:45.755160 2585 watchdog_linux.go:137] "Systemd watchdog is not enabled or the interval is invalid, so health checking will not be started." Jul 10 18:39:46.261626 ip-10-0-143-17 kubenswrapper[2585]: W0710 18:39:45.755243 2585 feature_gate.go:328] unrecognized feature gate: BuildCSIVolumes Jul 10 18:39:46.261626 ip-10-0-143-17 kubenswrapper[2585]: W0710 18:39:45.755248 2585 feature_gate.go:328] unrecognized feature gate: ClusterAPIInstall Jul 10 18:39:46.261626 ip-10-0-143-17 kubenswrapper[2585]: W0710 18:39:45.755252 2585 feature_gate.go:328] unrecognized feature gate: ExternalOIDC Jul 10 18:39:46.261626 ip-10-0-143-17 kubenswrapper[2585]: W0710 18:39:45.755255 2585 feature_gate.go:328] unrecognized feature gate: AzureMultiDisk Jul 10 18:39:46.261626 ip-10-0-143-17 kubenswrapper[2585]: W0710 18:39:45.755258 2585 feature_gate.go:328] unrecognized feature gate: KMSEncryptionProvider Jul 10 18:39:46.261626 ip-10-0-143-17 kubenswrapper[2585]: W0710 18:39:45.755261 2585 feature_gate.go:328] unrecognized feature gate: NetworkDiagnosticsConfig Jul 10 18:39:46.261626 ip-10-0-143-17 kubenswrapper[2585]: W0710 18:39:45.755264 2585 feature_gate.go:328] unrecognized feature gate: ClusterVersionOperatorConfiguration Jul 10 18:39:46.261626 ip-10-0-143-17 kubenswrapper[2585]: W0710 18:39:45.755267 2585 feature_gate.go:328] unrecognized feature gate: NutanixMultiSubnets Jul 10 18:39:46.261626 ip-10-0-143-17 kubenswrapper[2585]: W0710 18:39:45.755270 2585 feature_gate.go:328] unrecognized feature gate: GCPCustomAPIEndpointsInstall Jul 10 18:39:46.261626 ip-10-0-143-17 kubenswrapper[2585]: W0710 18:39:45.755273 2585 feature_gate.go:328] unrecognized feature gate: GatewayAPIController Jul 10 18:39:46.262074 ip-10-0-143-17 kubenswrapper[2585]: W0710 18:39:45.755275 2585 feature_gate.go:328] unrecognized feature gate: NewOLMCatalogdAPIV1Metas Jul 10 18:39:46.262074 ip-10-0-143-17 kubenswrapper[2585]: W0710 18:39:45.755279 2585 feature_gate.go:328] unrecognized feature gate: ConsolePluginContentSecurityPolicy Jul 10 18:39:46.262074 ip-10-0-143-17 kubenswrapper[2585]: W0710 18:39:45.755283 2585 feature_gate.go:328] unrecognized feature gate: MultiDiskSetup Jul 10 18:39:46.262074 ip-10-0-143-17 kubenswrapper[2585]: W0710 18:39:45.755285 2585 feature_gate.go:328] unrecognized feature gate: OVNObservability Jul 10 18:39:46.262074 ip-10-0-143-17 kubenswrapper[2585]: W0710 18:39:45.755288 2585 feature_gate.go:328] unrecognized feature gate: CPMSMachineNamePrefix Jul 10 18:39:46.262074 ip-10-0-143-17 kubenswrapper[2585]: W0710 18:39:45.755290 2585 feature_gate.go:328] unrecognized feature gate: ImageStreamImportMode Jul 10 18:39:46.262074 ip-10-0-143-17 kubenswrapper[2585]: W0710 18:39:45.755293 2585 feature_gate.go:328] unrecognized feature gate: MetricsCollectionProfiles Jul 10 18:39:46.262074 ip-10-0-143-17 kubenswrapper[2585]: W0710 18:39:45.755296 2585 feature_gate.go:328] unrecognized feature gate: AWSClusterHostedDNSInstall Jul 10 18:39:46.262074 ip-10-0-143-17 kubenswrapper[2585]: W0710 18:39:45.755298 2585 feature_gate.go:328] unrecognized feature gate: BootImageSkewEnforcement Jul 10 18:39:46.262074 ip-10-0-143-17 kubenswrapper[2585]: W0710 18:39:45.755301 2585 feature_gate.go:328] unrecognized feature gate: AdditionalRoutingCapabilities Jul 10 18:39:46.262074 ip-10-0-143-17 kubenswrapper[2585]: W0710 18:39:45.755303 2585 feature_gate.go:328] unrecognized feature gate: PreconfiguredUDNAddresses Jul 10 18:39:46.262074 ip-10-0-143-17 kubenswrapper[2585]: W0710 18:39:45.755306 2585 feature_gate.go:328] unrecognized feature gate: IngressControllerDynamicConfigurationManager Jul 10 18:39:46.262074 ip-10-0-143-17 kubenswrapper[2585]: W0710 18:39:45.755308 2585 feature_gate.go:328] unrecognized feature gate: VSphereHostVMGroupZonal Jul 10 18:39:46.262074 ip-10-0-143-17 kubenswrapper[2585]: W0710 18:39:45.755311 2585 feature_gate.go:328] unrecognized feature gate: AzureClusterHostedDNSInstall Jul 10 18:39:46.262074 ip-10-0-143-17 kubenswrapper[2585]: W0710 18:39:45.755314 2585 feature_gate.go:328] unrecognized feature gate: DNSNameResolver Jul 10 18:39:46.262074 ip-10-0-143-17 kubenswrapper[2585]: W0710 18:39:45.755316 2585 feature_gate.go:328] unrecognized feature gate: AWSDualStackInstall Jul 10 18:39:46.262074 ip-10-0-143-17 kubenswrapper[2585]: W0710 18:39:45.755319 2585 feature_gate.go:328] unrecognized feature gate: Example Jul 10 18:39:46.262074 ip-10-0-143-17 kubenswrapper[2585]: W0710 18:39:45.755324 2585 feature_gate.go:328] unrecognized feature gate: ProvisioningRequestAvailable Jul 10 18:39:46.262074 ip-10-0-143-17 kubenswrapper[2585]: W0710 18:39:45.755327 2585 feature_gate.go:328] unrecognized feature gate: ExternalSnapshotMetadata Jul 10 18:39:46.262074 ip-10-0-143-17 kubenswrapper[2585]: W0710 18:39:45.755329 2585 feature_gate.go:328] unrecognized feature gate: InsightsConfig Jul 10 18:39:46.262674 ip-10-0-143-17 kubenswrapper[2585]: W0710 18:39:45.755334 2585 feature_gate.go:351] Setting GA feature gate VolumeAttributesClass=true. It will be removed in a future release. Jul 10 18:39:46.262674 ip-10-0-143-17 kubenswrapper[2585]: W0710 18:39:45.755338 2585 feature_gate.go:328] unrecognized feature gate: VSphereMultiDisk Jul 10 18:39:46.262674 ip-10-0-143-17 kubenswrapper[2585]: W0710 18:39:45.755341 2585 feature_gate.go:328] unrecognized feature gate: NetworkLiveMigration Jul 10 18:39:46.262674 ip-10-0-143-17 kubenswrapper[2585]: W0710 18:39:45.755343 2585 feature_gate.go:328] unrecognized feature gate: CRDCompatibilityRequirementOperator Jul 10 18:39:46.262674 ip-10-0-143-17 kubenswrapper[2585]: W0710 18:39:45.755346 2585 feature_gate.go:328] unrecognized feature gate: VSphereMixedNodeEnv Jul 10 18:39:46.262674 ip-10-0-143-17 kubenswrapper[2585]: W0710 18:39:45.755349 2585 feature_gate.go:328] unrecognized feature gate: AzureDualStackInstall Jul 10 18:39:46.262674 ip-10-0-143-17 kubenswrapper[2585]: W0710 18:39:45.755352 2585 feature_gate.go:328] unrecognized feature gate: GCPClusterHostedDNS Jul 10 18:39:46.262674 ip-10-0-143-17 kubenswrapper[2585]: W0710 18:39:45.755354 2585 feature_gate.go:328] unrecognized feature gate: AWSDedicatedHosts Jul 10 18:39:46.262674 ip-10-0-143-17 kubenswrapper[2585]: W0710 18:39:45.755357 2585 feature_gate.go:328] unrecognized feature gate: OnPremDNSRecords Jul 10 18:39:46.262674 ip-10-0-143-17 kubenswrapper[2585]: W0710 18:39:45.755360 2585 feature_gate.go:328] unrecognized feature gate: VSphereConfigurableMaxAllowedBlockVolumesPerNode Jul 10 18:39:46.262674 ip-10-0-143-17 kubenswrapper[2585]: W0710 18:39:45.755363 2585 feature_gate.go:328] unrecognized feature gate: OpenShiftPodSecurityAdmission Jul 10 18:39:46.262674 ip-10-0-143-17 kubenswrapper[2585]: W0710 18:39:45.755366 2585 feature_gate.go:328] unrecognized feature gate: Example2 Jul 10 18:39:46.262674 ip-10-0-143-17 kubenswrapper[2585]: W0710 18:39:45.755370 2585 feature_gate.go:349] Setting deprecated feature gate KMSv1=true. It will be removed in a future release. Jul 10 18:39:46.262674 ip-10-0-143-17 kubenswrapper[2585]: W0710 18:39:45.755373 2585 feature_gate.go:328] unrecognized feature gate: MachineAPIMigration Jul 10 18:39:46.262674 ip-10-0-143-17 kubenswrapper[2585]: W0710 18:39:45.755376 2585 feature_gate.go:328] unrecognized feature gate: GCPClusterHostedDNSInstall Jul 10 18:39:46.262674 ip-10-0-143-17 kubenswrapper[2585]: W0710 18:39:45.755379 2585 feature_gate.go:328] unrecognized feature gate: KMSEncryption Jul 10 18:39:46.262674 ip-10-0-143-17 kubenswrapper[2585]: W0710 18:39:45.755381 2585 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesAWS Jul 10 18:39:46.262674 ip-10-0-143-17 kubenswrapper[2585]: W0710 18:39:45.755384 2585 feature_gate.go:328] unrecognized feature gate: HighlyAvailableArbiter Jul 10 18:39:46.262674 ip-10-0-143-17 kubenswrapper[2585]: W0710 18:39:45.755386 2585 feature_gate.go:328] unrecognized feature gate: OSStreams Jul 10 18:39:46.263179 ip-10-0-143-17 kubenswrapper[2585]: W0710 18:39:45.755389 2585 feature_gate.go:328] unrecognized feature gate: ClusterAPIMachineManagement Jul 10 18:39:46.263179 ip-10-0-143-17 kubenswrapper[2585]: W0710 18:39:45.755391 2585 feature_gate.go:328] unrecognized feature gate: MachineConfigNodes Jul 10 18:39:46.263179 ip-10-0-143-17 kubenswrapper[2585]: W0710 18:39:45.755394 2585 feature_gate.go:328] unrecognized feature gate: GatewayAPI Jul 10 18:39:46.263179 ip-10-0-143-17 kubenswrapper[2585]: W0710 18:39:45.755397 2585 feature_gate.go:328] unrecognized feature gate: HyperShiftOnlyDynamicResourceAllocation Jul 10 18:39:46.263179 ip-10-0-143-17 kubenswrapper[2585]: W0710 18:39:45.755399 2585 feature_gate.go:328] unrecognized feature gate: MixedCPUsAllocation Jul 10 18:39:46.263179 ip-10-0-143-17 kubenswrapper[2585]: W0710 18:39:45.755401 2585 feature_gate.go:328] unrecognized feature gate: AutomatedEtcdBackup Jul 10 18:39:46.263179 ip-10-0-143-17 kubenswrapper[2585]: W0710 18:39:45.755404 2585 feature_gate.go:328] unrecognized feature gate: ClusterAPIInstallIBMCloud Jul 10 18:39:46.263179 ip-10-0-143-17 kubenswrapper[2585]: W0710 18:39:45.755406 2585 feature_gate.go:328] unrecognized feature gate: AlibabaPlatform Jul 10 18:39:46.263179 ip-10-0-143-17 kubenswrapper[2585]: W0710 18:39:45.755409 2585 feature_gate.go:328] unrecognized feature gate: AdminNetworkPolicy Jul 10 18:39:46.263179 ip-10-0-143-17 kubenswrapper[2585]: W0710 18:39:45.755411 2585 feature_gate.go:328] unrecognized feature gate: NewOLM Jul 10 18:39:46.263179 ip-10-0-143-17 kubenswrapper[2585]: W0710 18:39:45.755417 2585 feature_gate.go:328] unrecognized feature gate: InsightsOnDemandDataGather Jul 10 18:39:46.263179 ip-10-0-143-17 kubenswrapper[2585]: W0710 18:39:45.755419 2585 feature_gate.go:328] unrecognized feature gate: EventTTL Jul 10 18:39:46.263179 ip-10-0-143-17 kubenswrapper[2585]: W0710 18:39:45.755422 2585 feature_gate.go:328] unrecognized feature gate: AzureWorkloadIdentity Jul 10 18:39:46.263179 ip-10-0-143-17 kubenswrapper[2585]: W0710 18:39:45.755425 2585 feature_gate.go:328] unrecognized feature gate: AzureDedicatedHosts Jul 10 18:39:46.263179 ip-10-0-143-17 kubenswrapper[2585]: W0710 18:39:45.755427 2585 feature_gate.go:328] unrecognized feature gate: GatewayAPIWithoutOLM Jul 10 18:39:46.263179 ip-10-0-143-17 kubenswrapper[2585]: W0710 18:39:45.755430 2585 feature_gate.go:328] unrecognized feature gate: MachineAPIOperatorDisableMachineHealthCheckController Jul 10 18:39:46.263179 ip-10-0-143-17 kubenswrapper[2585]: W0710 18:39:45.755432 2585 feature_gate.go:328] unrecognized feature gate: ImageModeStatusReporting Jul 10 18:39:46.263179 ip-10-0-143-17 kubenswrapper[2585]: W0710 18:39:45.755435 2585 feature_gate.go:328] unrecognized feature gate: NewOLMBoxCutterRuntime Jul 10 18:39:46.263179 ip-10-0-143-17 kubenswrapper[2585]: W0710 18:39:45.755437 2585 feature_gate.go:328] unrecognized feature gate: RouteAdvertisements Jul 10 18:39:46.263179 ip-10-0-143-17 kubenswrapper[2585]: W0710 18:39:45.755440 2585 feature_gate.go:328] unrecognized feature gate: NewOLMPreflightPermissionChecks Jul 10 18:39:46.263758 ip-10-0-143-17 kubenswrapper[2585]: W0710 18:39:45.755442 2585 feature_gate.go:328] unrecognized feature gate: SigstoreImageVerificationPKI Jul 10 18:39:46.263758 ip-10-0-143-17 kubenswrapper[2585]: W0710 18:39:45.755445 2585 feature_gate.go:328] unrecognized feature gate: SigstoreImageVerification Jul 10 18:39:46.263758 ip-10-0-143-17 kubenswrapper[2585]: W0710 18:39:45.755447 2585 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesvSphere Jul 10 18:39:46.263758 ip-10-0-143-17 kubenswrapper[2585]: W0710 18:39:45.755452 2585 feature_gate.go:328] unrecognized feature gate: VolumeGroupSnapshot Jul 10 18:39:46.263758 ip-10-0-143-17 kubenswrapper[2585]: W0710 18:39:45.755455 2585 feature_gate.go:328] unrecognized feature gate: ShortCertRotation Jul 10 18:39:46.263758 ip-10-0-143-17 kubenswrapper[2585]: W0710 18:39:45.755457 2585 feature_gate.go:328] unrecognized feature gate: DualReplica Jul 10 18:39:46.263758 ip-10-0-143-17 kubenswrapper[2585]: W0710 18:39:45.755460 2585 feature_gate.go:328] unrecognized feature gate: VSphereMultiNetworks Jul 10 18:39:46.263758 ip-10-0-143-17 kubenswrapper[2585]: W0710 18:39:45.755462 2585 feature_gate.go:328] unrecognized feature gate: MultiArchInstallAzure Jul 10 18:39:46.263758 ip-10-0-143-17 kubenswrapper[2585]: W0710 18:39:45.755465 2585 feature_gate.go:328] unrecognized feature gate: NewOLMOwnSingleNamespace Jul 10 18:39:46.263758 ip-10-0-143-17 kubenswrapper[2585]: W0710 18:39:45.755467 2585 feature_gate.go:328] unrecognized feature gate: NewOLMWebhookProviderOpenshiftServiceCA Jul 10 18:39:46.263758 ip-10-0-143-17 kubenswrapper[2585]: W0710 18:39:45.755470 2585 feature_gate.go:328] unrecognized feature gate: GCPCustomAPIEndpoints Jul 10 18:39:46.263758 ip-10-0-143-17 kubenswrapper[2585]: W0710 18:39:45.755473 2585 feature_gate.go:328] unrecognized feature gate: ManagedBootImages Jul 10 18:39:46.263758 ip-10-0-143-17 kubenswrapper[2585]: W0710 18:39:45.755475 2585 feature_gate.go:328] unrecognized feature gate: BootcNodeManagement Jul 10 18:39:46.263758 ip-10-0-143-17 kubenswrapper[2585]: W0710 18:39:45.755478 2585 feature_gate.go:328] unrecognized feature gate: SignatureStores Jul 10 18:39:46.263758 ip-10-0-143-17 kubenswrapper[2585]: W0710 18:39:45.755480 2585 feature_gate.go:328] unrecognized feature gate: IrreconcilableMachineConfig Jul 10 18:39:46.263758 ip-10-0-143-17 kubenswrapper[2585]: W0710 18:39:45.755483 2585 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesCPMS Jul 10 18:39:46.263758 ip-10-0-143-17 kubenswrapper[2585]: W0710 18:39:45.755485 2585 feature_gate.go:328] unrecognized feature gate: EtcdBackendQuota Jul 10 18:39:46.263758 ip-10-0-143-17 kubenswrapper[2585]: W0710 18:39:45.755488 2585 feature_gate.go:328] unrecognized feature gate: ExternalOIDCWithUIDAndExtraClaimMappings Jul 10 18:39:46.263758 ip-10-0-143-17 kubenswrapper[2585]: W0710 18:39:45.755490 2585 feature_gate.go:328] unrecognized feature gate: ClusterAPIMachineManagementVSphere Jul 10 18:39:46.263758 ip-10-0-143-17 kubenswrapper[2585]: W0710 18:39:45.755493 2585 feature_gate.go:328] unrecognized feature gate: UpgradeStatus Jul 10 18:39:46.264293 ip-10-0-143-17 kubenswrapper[2585]: W0710 18:39:45.755495 2585 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesAzure Jul 10 18:39:46.264293 ip-10-0-143-17 kubenswrapper[2585]: W0710 18:39:45.755498 2585 feature_gate.go:328] unrecognized feature gate: NoRegistryClusterInstall Jul 10 18:39:46.264293 ip-10-0-143-17 kubenswrapper[2585]: W0710 18:39:45.755501 2585 feature_gate.go:328] unrecognized feature gate: NetworkSegmentation Jul 10 18:39:46.264293 ip-10-0-143-17 kubenswrapper[2585]: W0710 18:39:45.755504 2585 feature_gate.go:328] unrecognized feature gate: ClusterMonitoringConfig Jul 10 18:39:46.264293 ip-10-0-143-17 kubenswrapper[2585]: W0710 18:39:45.755508 2585 feature_gate.go:351] Setting GA feature gate ServiceAccountTokenNodeBinding=true. It will be removed in a future release. Jul 10 18:39:46.264293 ip-10-0-143-17 kubenswrapper[2585]: W0710 18:39:45.755511 2585 feature_gate.go:328] unrecognized feature gate: AWSClusterHostedDNS Jul 10 18:39:46.264293 ip-10-0-143-17 kubenswrapper[2585]: W0710 18:39:45.755514 2585 feature_gate.go:328] unrecognized feature gate: AWSServiceLBNetworkSecurityGroup Jul 10 18:39:46.264293 ip-10-0-143-17 kubenswrapper[2585]: W0710 18:39:45.755517 2585 feature_gate.go:328] unrecognized feature gate: GCPDualStackInstall Jul 10 18:39:46.264293 ip-10-0-143-17 kubenswrapper[2585]: W0710 18:39:45.755520 2585 feature_gate.go:328] unrecognized feature gate: DyanmicServiceEndpointIBMCloud Jul 10 18:39:46.264293 ip-10-0-143-17 kubenswrapper[2585]: W0710 18:39:45.755523 2585 feature_gate.go:328] unrecognized feature gate: PinnedImages Jul 10 18:39:46.264293 ip-10-0-143-17 kubenswrapper[2585]: I0710 18:39:45.755528 2585 feature_gate.go:384] feature gates: {map[CBORServingAndStorage:false ClientsAllowCBOR:false ClientsPreferCBOR:false EventedPLEG:false ImageVolume:true KMSv1:true MaxUnavailableStatefulSet:false MinimumKubeletVersion:false MutableCSINodeAllocatableCount:false MutatingAdmissionPolicy:false ProcMountType:true RouteExternalCertificate:true SELinuxMount:false ServiceAccountTokenNodeBinding:true StoragePerformantSecurityPolicy:true TranslateStreamCloseWebsocketRequests:false UserNamespacesPodSecurityStandards:true UserNamespacesSupport:true VolumeAttributesClass:true]} Jul 10 18:39:46.264293 ip-10-0-143-17 kubenswrapper[2585]: W0710 18:39:45.755637 2585 feature_gate.go:328] unrecognized feature gate: ClusterAPIMachineManagement Jul 10 18:39:46.264293 ip-10-0-143-17 kubenswrapper[2585]: W0710 18:39:45.755641 2585 feature_gate.go:328] unrecognized feature gate: SigstoreImageVerification Jul 10 18:39:46.264293 ip-10-0-143-17 kubenswrapper[2585]: W0710 18:39:45.755644 2585 feature_gate.go:328] unrecognized feature gate: VSphereMultiDisk Jul 10 18:39:46.264712 ip-10-0-143-17 kubenswrapper[2585]: W0710 18:39:45.755647 2585 feature_gate.go:328] unrecognized feature gate: NetworkDiagnosticsConfig Jul 10 18:39:46.264712 ip-10-0-143-17 kubenswrapper[2585]: W0710 18:39:45.755653 2585 feature_gate.go:351] Setting GA feature gate VolumeAttributesClass=true. It will be removed in a future release. Jul 10 18:39:46.264712 ip-10-0-143-17 kubenswrapper[2585]: W0710 18:39:45.755656 2585 feature_gate.go:328] unrecognized feature gate: ExternalOIDCWithUIDAndExtraClaimMappings Jul 10 18:39:46.264712 ip-10-0-143-17 kubenswrapper[2585]: W0710 18:39:45.755659 2585 feature_gate.go:328] unrecognized feature gate: OVNObservability Jul 10 18:39:46.264712 ip-10-0-143-17 kubenswrapper[2585]: W0710 18:39:45.755662 2585 feature_gate.go:328] unrecognized feature gate: NetworkLiveMigration Jul 10 18:39:46.264712 ip-10-0-143-17 kubenswrapper[2585]: W0710 18:39:45.755666 2585 feature_gate.go:328] unrecognized feature gate: MachineAPIMigration Jul 10 18:39:46.264712 ip-10-0-143-17 kubenswrapper[2585]: W0710 18:39:45.755668 2585 feature_gate.go:328] unrecognized feature gate: ExternalSnapshotMetadata Jul 10 18:39:46.264712 ip-10-0-143-17 kubenswrapper[2585]: W0710 18:39:45.755671 2585 feature_gate.go:328] unrecognized feature gate: AzureWorkloadIdentity Jul 10 18:39:46.264712 ip-10-0-143-17 kubenswrapper[2585]: W0710 18:39:45.755675 2585 feature_gate.go:351] Setting GA feature gate ServiceAccountTokenNodeBinding=true. It will be removed in a future release. Jul 10 18:39:46.264712 ip-10-0-143-17 kubenswrapper[2585]: W0710 18:39:45.755678 2585 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesAzure Jul 10 18:39:46.264712 ip-10-0-143-17 kubenswrapper[2585]: W0710 18:39:45.755682 2585 feature_gate.go:328] unrecognized feature gate: MultiDiskSetup Jul 10 18:39:46.264712 ip-10-0-143-17 kubenswrapper[2585]: W0710 18:39:45.755685 2585 feature_gate.go:328] unrecognized feature gate: AdditionalRoutingCapabilities Jul 10 18:39:46.264712 ip-10-0-143-17 kubenswrapper[2585]: W0710 18:39:45.755688 2585 feature_gate.go:328] unrecognized feature gate: MetricsCollectionProfiles Jul 10 18:39:46.264712 ip-10-0-143-17 kubenswrapper[2585]: W0710 18:39:45.755690 2585 feature_gate.go:328] unrecognized feature gate: OSStreams Jul 10 18:39:46.264712 ip-10-0-143-17 kubenswrapper[2585]: W0710 18:39:45.755693 2585 feature_gate.go:328] unrecognized feature gate: AWSServiceLBNetworkSecurityGroup Jul 10 18:39:46.264712 ip-10-0-143-17 kubenswrapper[2585]: W0710 18:39:45.755696 2585 feature_gate.go:328] unrecognized feature gate: AWSDedicatedHosts Jul 10 18:39:46.264712 ip-10-0-143-17 kubenswrapper[2585]: W0710 18:39:45.755698 2585 feature_gate.go:328] unrecognized feature gate: DNSNameResolver Jul 10 18:39:46.264712 ip-10-0-143-17 kubenswrapper[2585]: W0710 18:39:45.755701 2585 feature_gate.go:328] unrecognized feature gate: KMSEncryption Jul 10 18:39:46.264712 ip-10-0-143-17 kubenswrapper[2585]: W0710 18:39:45.755703 2585 feature_gate.go:328] unrecognized feature gate: NoRegistryClusterInstall Jul 10 18:39:46.265231 ip-10-0-143-17 kubenswrapper[2585]: W0710 18:39:45.755706 2585 feature_gate.go:328] unrecognized feature gate: ClusterAPIInstall Jul 10 18:39:46.265231 ip-10-0-143-17 kubenswrapper[2585]: W0710 18:39:45.755709 2585 feature_gate.go:328] unrecognized feature gate: NewOLMBoxCutterRuntime Jul 10 18:39:46.265231 ip-10-0-143-17 kubenswrapper[2585]: W0710 18:39:45.755713 2585 feature_gate.go:328] unrecognized feature gate: VSphereConfigurableMaxAllowedBlockVolumesPerNode Jul 10 18:39:46.265231 ip-10-0-143-17 kubenswrapper[2585]: W0710 18:39:45.755715 2585 feature_gate.go:328] unrecognized feature gate: AdminNetworkPolicy Jul 10 18:39:46.265231 ip-10-0-143-17 kubenswrapper[2585]: W0710 18:39:45.755718 2585 feature_gate.go:328] unrecognized feature gate: NewOLMOwnSingleNamespace Jul 10 18:39:46.265231 ip-10-0-143-17 kubenswrapper[2585]: W0710 18:39:45.755720 2585 feature_gate.go:328] unrecognized feature gate: NutanixMultiSubnets Jul 10 18:39:46.265231 ip-10-0-143-17 kubenswrapper[2585]: W0710 18:39:45.755723 2585 feature_gate.go:328] unrecognized feature gate: ClusterAPIMachineManagementVSphere Jul 10 18:39:46.265231 ip-10-0-143-17 kubenswrapper[2585]: W0710 18:39:45.755726 2585 feature_gate.go:328] unrecognized feature gate: PinnedImages Jul 10 18:39:46.265231 ip-10-0-143-17 kubenswrapper[2585]: W0710 18:39:45.755728 2585 feature_gate.go:328] unrecognized feature gate: Example Jul 10 18:39:46.265231 ip-10-0-143-17 kubenswrapper[2585]: W0710 18:39:45.755731 2585 feature_gate.go:328] unrecognized feature gate: AWSClusterHostedDNSInstall Jul 10 18:39:46.265231 ip-10-0-143-17 kubenswrapper[2585]: W0710 18:39:45.755733 2585 feature_gate.go:328] unrecognized feature gate: AWSClusterHostedDNS Jul 10 18:39:46.265231 ip-10-0-143-17 kubenswrapper[2585]: W0710 18:39:45.755736 2585 feature_gate.go:328] unrecognized feature gate: KMSEncryptionProvider Jul 10 18:39:46.265231 ip-10-0-143-17 kubenswrapper[2585]: W0710 18:39:45.755738 2585 feature_gate.go:328] unrecognized feature gate: MixedCPUsAllocation Jul 10 18:39:46.265231 ip-10-0-143-17 kubenswrapper[2585]: W0710 18:39:45.755741 2585 feature_gate.go:328] unrecognized feature gate: GCPClusterHostedDNSInstall Jul 10 18:39:46.265231 ip-10-0-143-17 kubenswrapper[2585]: W0710 18:39:45.755745 2585 feature_gate.go:328] unrecognized feature gate: RouteAdvertisements Jul 10 18:39:46.265231 ip-10-0-143-17 kubenswrapper[2585]: W0710 18:39:45.755747 2585 feature_gate.go:328] unrecognized feature gate: GatewayAPIWithoutOLM Jul 10 18:39:46.265231 ip-10-0-143-17 kubenswrapper[2585]: W0710 18:39:45.755750 2585 feature_gate.go:328] unrecognized feature gate: NewOLM Jul 10 18:39:46.265231 ip-10-0-143-17 kubenswrapper[2585]: W0710 18:39:45.755753 2585 feature_gate.go:328] unrecognized feature gate: GCPClusterHostedDNS Jul 10 18:39:46.265231 ip-10-0-143-17 kubenswrapper[2585]: W0710 18:39:45.755755 2585 feature_gate.go:328] unrecognized feature gate: ImageStreamImportMode Jul 10 18:39:46.265231 ip-10-0-143-17 kubenswrapper[2585]: W0710 18:39:45.755759 2585 feature_gate.go:349] Setting deprecated feature gate KMSv1=true. It will be removed in a future release. Jul 10 18:39:46.265830 ip-10-0-143-17 kubenswrapper[2585]: W0710 18:39:45.755762 2585 feature_gate.go:328] unrecognized feature gate: MultiArchInstallAzure Jul 10 18:39:46.265830 ip-10-0-143-17 kubenswrapper[2585]: W0710 18:39:45.755764 2585 feature_gate.go:328] unrecognized feature gate: OpenShiftPodSecurityAdmission Jul 10 18:39:46.265830 ip-10-0-143-17 kubenswrapper[2585]: W0710 18:39:45.755767 2585 feature_gate.go:328] unrecognized feature gate: AlibabaPlatform Jul 10 18:39:46.265830 ip-10-0-143-17 kubenswrapper[2585]: W0710 18:39:45.755770 2585 feature_gate.go:328] unrecognized feature gate: SigstoreImageVerificationPKI Jul 10 18:39:46.265830 ip-10-0-143-17 kubenswrapper[2585]: W0710 18:39:45.755772 2585 feature_gate.go:328] unrecognized feature gate: CRDCompatibilityRequirementOperator Jul 10 18:39:46.265830 ip-10-0-143-17 kubenswrapper[2585]: W0710 18:39:45.755775 2585 feature_gate.go:328] unrecognized feature gate: GCPCustomAPIEndpointsInstall Jul 10 18:39:46.265830 ip-10-0-143-17 kubenswrapper[2585]: W0710 18:39:45.755777 2585 feature_gate.go:328] unrecognized feature gate: VSphereHostVMGroupZonal Jul 10 18:39:46.265830 ip-10-0-143-17 kubenswrapper[2585]: W0710 18:39:45.755779 2585 feature_gate.go:328] unrecognized feature gate: DyanmicServiceEndpointIBMCloud Jul 10 18:39:46.265830 ip-10-0-143-17 kubenswrapper[2585]: W0710 18:39:45.755782 2585 feature_gate.go:328] unrecognized feature gate: AWSDualStackInstall Jul 10 18:39:46.265830 ip-10-0-143-17 kubenswrapper[2585]: W0710 18:39:45.755785 2585 feature_gate.go:328] unrecognized feature gate: ProvisioningRequestAvailable Jul 10 18:39:46.265830 ip-10-0-143-17 kubenswrapper[2585]: W0710 18:39:45.755787 2585 feature_gate.go:328] unrecognized feature gate: UpgradeStatus Jul 10 18:39:46.265830 ip-10-0-143-17 kubenswrapper[2585]: W0710 18:39:45.755790 2585 feature_gate.go:328] unrecognized feature gate: GatewayAPIController Jul 10 18:39:46.265830 ip-10-0-143-17 kubenswrapper[2585]: W0710 18:39:45.755793 2585 feature_gate.go:328] unrecognized feature gate: AzureClusterHostedDNSInstall Jul 10 18:39:46.265830 ip-10-0-143-17 kubenswrapper[2585]: W0710 18:39:45.755795 2585 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesAWS Jul 10 18:39:46.265830 ip-10-0-143-17 kubenswrapper[2585]: W0710 18:39:45.755799 2585 feature_gate.go:328] unrecognized feature gate: ShortCertRotation Jul 10 18:39:46.265830 ip-10-0-143-17 kubenswrapper[2585]: W0710 18:39:45.755802 2585 feature_gate.go:328] unrecognized feature gate: OnPremDNSRecords Jul 10 18:39:46.265830 ip-10-0-143-17 kubenswrapper[2585]: W0710 18:39:45.755804 2585 feature_gate.go:328] unrecognized feature gate: ClusterMonitoringConfig Jul 10 18:39:46.265830 ip-10-0-143-17 kubenswrapper[2585]: W0710 18:39:45.755807 2585 feature_gate.go:328] unrecognized feature gate: Example2 Jul 10 18:39:46.265830 ip-10-0-143-17 kubenswrapper[2585]: W0710 18:39:45.755809 2585 feature_gate.go:328] unrecognized feature gate: BootcNodeManagement Jul 10 18:39:46.265830 ip-10-0-143-17 kubenswrapper[2585]: W0710 18:39:45.755812 2585 feature_gate.go:328] unrecognized feature gate: MachineAPIOperatorDisableMachineHealthCheckController Jul 10 18:39:46.266396 ip-10-0-143-17 kubenswrapper[2585]: W0710 18:39:45.755815 2585 feature_gate.go:328] unrecognized feature gate: AutomatedEtcdBackup Jul 10 18:39:46.266396 ip-10-0-143-17 kubenswrapper[2585]: W0710 18:39:45.755817 2585 feature_gate.go:328] unrecognized feature gate: VSphereMultiNetworks Jul 10 18:39:46.266396 ip-10-0-143-17 kubenswrapper[2585]: W0710 18:39:45.755820 2585 feature_gate.go:328] unrecognized feature gate: GatewayAPI Jul 10 18:39:46.266396 ip-10-0-143-17 kubenswrapper[2585]: W0710 18:39:45.755822 2585 feature_gate.go:328] unrecognized feature gate: VolumeGroupSnapshot Jul 10 18:39:46.266396 ip-10-0-143-17 kubenswrapper[2585]: W0710 18:39:45.755825 2585 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesvSphere Jul 10 18:39:46.266396 ip-10-0-143-17 kubenswrapper[2585]: W0710 18:39:45.755827 2585 feature_gate.go:328] unrecognized feature gate: GCPDualStackInstall Jul 10 18:39:46.266396 ip-10-0-143-17 kubenswrapper[2585]: W0710 18:39:45.755831 2585 feature_gate.go:328] unrecognized feature gate: MachineConfigNodes Jul 10 18:39:46.266396 ip-10-0-143-17 kubenswrapper[2585]: W0710 18:39:45.755834 2585 feature_gate.go:328] unrecognized feature gate: EventTTL Jul 10 18:39:46.266396 ip-10-0-143-17 kubenswrapper[2585]: W0710 18:39:45.755836 2585 feature_gate.go:328] unrecognized feature gate: HyperShiftOnlyDynamicResourceAllocation Jul 10 18:39:46.266396 ip-10-0-143-17 kubenswrapper[2585]: W0710 18:39:45.755839 2585 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesCPMS Jul 10 18:39:46.266396 ip-10-0-143-17 kubenswrapper[2585]: W0710 18:39:45.755841 2585 feature_gate.go:328] unrecognized feature gate: ExternalOIDC Jul 10 18:39:46.266396 ip-10-0-143-17 kubenswrapper[2585]: W0710 18:39:45.755844 2585 feature_gate.go:328] unrecognized feature gate: ManagedBootImages Jul 10 18:39:46.266396 ip-10-0-143-17 kubenswrapper[2585]: W0710 18:39:45.755846 2585 feature_gate.go:328] unrecognized feature gate: ClusterAPIInstallIBMCloud Jul 10 18:39:46.266396 ip-10-0-143-17 kubenswrapper[2585]: W0710 18:39:45.755849 2585 feature_gate.go:328] unrecognized feature gate: AzureDualStackInstall Jul 10 18:39:46.266396 ip-10-0-143-17 kubenswrapper[2585]: W0710 18:39:45.755852 2585 feature_gate.go:328] unrecognized feature gate: AzureDedicatedHosts Jul 10 18:39:46.266396 ip-10-0-143-17 kubenswrapper[2585]: W0710 18:39:45.755855 2585 feature_gate.go:328] unrecognized feature gate: DualReplica Jul 10 18:39:46.266396 ip-10-0-143-17 kubenswrapper[2585]: W0710 18:39:45.755857 2585 feature_gate.go:328] unrecognized feature gate: ImageModeStatusReporting Jul 10 18:39:46.266396 ip-10-0-143-17 kubenswrapper[2585]: W0710 18:39:45.755860 2585 feature_gate.go:328] unrecognized feature gate: HighlyAvailableArbiter Jul 10 18:39:46.266396 ip-10-0-143-17 kubenswrapper[2585]: W0710 18:39:45.755862 2585 feature_gate.go:328] unrecognized feature gate: InsightsOnDemandDataGather Jul 10 18:39:46.266396 ip-10-0-143-17 kubenswrapper[2585]: W0710 18:39:45.755865 2585 feature_gate.go:328] unrecognized feature gate: BuildCSIVolumes Jul 10 18:39:46.266971 ip-10-0-143-17 kubenswrapper[2585]: W0710 18:39:45.755867 2585 feature_gate.go:328] unrecognized feature gate: PreconfiguredUDNAddresses Jul 10 18:39:46.266971 ip-10-0-143-17 kubenswrapper[2585]: W0710 18:39:45.755870 2585 feature_gate.go:328] unrecognized feature gate: IngressControllerDynamicConfigurationManager Jul 10 18:39:46.266971 ip-10-0-143-17 kubenswrapper[2585]: W0710 18:39:45.755872 2585 feature_gate.go:328] unrecognized feature gate: SignatureStores Jul 10 18:39:46.266971 ip-10-0-143-17 kubenswrapper[2585]: W0710 18:39:45.755875 2585 feature_gate.go:328] unrecognized feature gate: ConsolePluginContentSecurityPolicy Jul 10 18:39:46.266971 ip-10-0-143-17 kubenswrapper[2585]: W0710 18:39:45.755877 2585 feature_gate.go:328] unrecognized feature gate: InsightsConfig Jul 10 18:39:46.266971 ip-10-0-143-17 kubenswrapper[2585]: W0710 18:39:45.755880 2585 feature_gate.go:328] unrecognized feature gate: VSphereMixedNodeEnv Jul 10 18:39:46.266971 ip-10-0-143-17 kubenswrapper[2585]: W0710 18:39:45.755884 2585 feature_gate.go:328] unrecognized feature gate: CPMSMachineNamePrefix Jul 10 18:39:46.266971 ip-10-0-143-17 kubenswrapper[2585]: W0710 18:39:45.755886 2585 feature_gate.go:328] unrecognized feature gate: NewOLMWebhookProviderOpenshiftServiceCA Jul 10 18:39:46.266971 ip-10-0-143-17 kubenswrapper[2585]: W0710 18:39:45.755889 2585 feature_gate.go:328] unrecognized feature gate: EtcdBackendQuota Jul 10 18:39:46.266971 ip-10-0-143-17 kubenswrapper[2585]: W0710 18:39:45.755891 2585 feature_gate.go:328] unrecognized feature gate: GCPCustomAPIEndpoints Jul 10 18:39:46.266971 ip-10-0-143-17 kubenswrapper[2585]: W0710 18:39:45.755894 2585 feature_gate.go:328] unrecognized feature gate: NewOLMPreflightPermissionChecks Jul 10 18:39:46.266971 ip-10-0-143-17 kubenswrapper[2585]: W0710 18:39:45.755896 2585 feature_gate.go:328] unrecognized feature gate: AzureMultiDisk Jul 10 18:39:46.266971 ip-10-0-143-17 kubenswrapper[2585]: W0710 18:39:45.755899 2585 feature_gate.go:328] unrecognized feature gate: ClusterVersionOperatorConfiguration Jul 10 18:39:46.266971 ip-10-0-143-17 kubenswrapper[2585]: W0710 18:39:45.755902 2585 feature_gate.go:328] unrecognized feature gate: NewOLMCatalogdAPIV1Metas Jul 10 18:39:46.266971 ip-10-0-143-17 kubenswrapper[2585]: W0710 18:39:45.755904 2585 feature_gate.go:328] unrecognized feature gate: BootImageSkewEnforcement Jul 10 18:39:46.266971 ip-10-0-143-17 kubenswrapper[2585]: W0710 18:39:45.755906 2585 feature_gate.go:328] unrecognized feature gate: IrreconcilableMachineConfig Jul 10 18:39:46.266971 ip-10-0-143-17 kubenswrapper[2585]: W0710 18:39:45.755909 2585 feature_gate.go:328] unrecognized feature gate: NetworkSegmentation Jul 10 18:39:46.267488 ip-10-0-143-17 kubenswrapper[2585]: I0710 18:39:45.755913 2585 feature_gate.go:384] feature gates: {map[CBORServingAndStorage:false ClientsAllowCBOR:false ClientsPreferCBOR:false EventedPLEG:false ImageVolume:true KMSv1:true MaxUnavailableStatefulSet:false MinimumKubeletVersion:false MutableCSINodeAllocatableCount:false MutatingAdmissionPolicy:false ProcMountType:true RouteExternalCertificate:true SELinuxMount:false ServiceAccountTokenNodeBinding:true StoragePerformantSecurityPolicy:true TranslateStreamCloseWebsocketRequests:false UserNamespacesPodSecurityStandards:true UserNamespacesSupport:true VolumeAttributesClass:true]} Jul 10 18:39:46.267488 ip-10-0-143-17 kubenswrapper[2585]: I0710 18:39:45.756012 2585 server.go:962] "Client rotation is on, will bootstrap in background" Jul 10 18:39:46.267488 ip-10-0-143-17 kubenswrapper[2585]: I0710 18:39:45.758097 2585 bootstrap.go:101] "Use the bootstrap credentials to request a cert, and set kubeconfig to point to the certificate dir" Jul 10 18:39:46.267488 ip-10-0-143-17 kubenswrapper[2585]: I0710 18:39:45.759570 2585 server.go:1019] "Starting client certificate rotation" Jul 10 18:39:46.267488 ip-10-0-143-17 kubenswrapper[2585]: I0710 18:39:45.759673 2585 certificate_manager.go:422] "Certificate rotation is enabled" logger="kubernetes.io/kube-apiserver-client-kubelet" Jul 10 18:39:46.267488 ip-10-0-143-17 kubenswrapper[2585]: I0710 18:39:45.759712 2585 certificate_manager.go:566] "Rotating certificates" logger="kubernetes.io/kube-apiserver-client-kubelet" Jul 10 18:39:46.267488 ip-10-0-143-17 kubenswrapper[2585]: I0710 18:39:45.762103 2585 dynamic_cafile_content.go:123] "Loaded a new CA Bundle and Verifier" name="client-ca-bundle::/etc/kubernetes/kubelet-ca.crt" Jul 10 18:39:46.267488 ip-10-0-143-17 kubenswrapper[2585]: I0710 18:39:45.763250 2585 dynamic_cafile_content.go:161] "Starting controller" name="client-ca-bundle::/etc/kubernetes/kubelet-ca.crt" Jul 10 18:39:46.267488 ip-10-0-143-17 kubenswrapper[2585]: I0710 18:39:45.773018 2585 log.go:25] "Validated CRI v1 runtime API" Jul 10 18:39:46.267488 ip-10-0-143-17 kubenswrapper[2585]: I0710 18:39:45.776927 2585 log.go:25] "Validated CRI v1 image API" Jul 10 18:39:46.267488 ip-10-0-143-17 kubenswrapper[2585]: I0710 18:39:45.777704 2585 server.go:1429] "Using cgroup driver setting received from the CRI runtime" cgroupDriver="systemd" Jul 10 18:39:46.267488 ip-10-0-143-17 kubenswrapper[2585]: I0710 18:39:45.780952 2585 fs.go:135] Filesystem UUIDs: map[7B77-95E7:/dev/nvme0n1p2 7f4ef599-c577-463f-8d0f-c951da544420:/dev/nvme0n1p4 f90603b0-b11d-453b-89ef-ec76ef3251a9:/dev/nvme0n1p3] Jul 10 18:39:46.267857 ip-10-0-143-17 kubenswrapper[2585]: I0710 18:39:45.780967 2585 fs.go:136] Filesystem partitions: map[/dev/nvme0n1p3:{mountpoint:/boot major:259 minor:3 fsType:ext4 blockSize:0} /dev/nvme0n1p4:{mountpoint:/var major:259 minor:4 fsType:xfs blockSize:0} /dev/shm:{mountpoint:/dev/shm major:0 minor:22 fsType:tmpfs blockSize:0} /run:{mountpoint:/run major:0 minor:24 fsType:tmpfs blockSize:0} /tmp:{mountpoint:/tmp major:0 minor:31 fsType:tmpfs blockSize:0} composefs_0-33:{mountpoint:/ major:0 minor:33 fsType:overlay blockSize:0}] Jul 10 18:39:46.267925 ip-10-0-143-17 kubenswrapper[2585]: I0710 18:39:45.785237 2585 manager.go:216] Machine: {Timestamp:2026-07-10 18:39:45.784746551 +0000 UTC m=+0.360437117 CPUVendorID:GenuineIntel NumCores:8 NumPhysicalCores:4 NumSockets:1 CpuFrequency:3093252 MemoryCapacity:33164492800 SwapCapacity:0 MemoryByType:map[] NVMInfo:{MemoryModeCapacity:0 AppDirectModeCapacity:0 AvgPowerBudget:0} HugePages:[{PageSize:1048576 NumPages:0} {PageSize:2048 NumPages:0}] MachineID:ec2ecc2a8c761da9db16180badd6b725 SystemUUID:ec2ecc2a-8c76-1da9-db16-180badd6b725 BootID:c9334134-7feb-419f-af3c-533eefe6b0a4 Filesystems:[{Device:/dev/shm DeviceMajor:0 DeviceMinor:22 Capacity:16582246400 Type:vfs Inodes:4048400 HasInodes:true} {Device:/run DeviceMajor:0 DeviceMinor:24 Capacity:6632898560 Type:vfs Inodes:819200 HasInodes:true} {Device:/dev/nvme0n1p4 DeviceMajor:259 DeviceMinor:4 Capacity:128243970048 Type:vfs Inodes:62651840 HasInodes:true} {Device:composefs_0-33 DeviceMajor:0 DeviceMinor:33 Capacity:6107136 Type:vfs Inodes:18446744073709551615 HasInodes:true} {Device:/tmp DeviceMajor:0 DeviceMinor:31 Capacity:16582246400 Type:vfs Inodes:1048576 HasInodes:true} {Device:/dev/nvme0n1p3 DeviceMajor:259 DeviceMinor:3 Capacity:366869504 Type:vfs Inodes:98304 HasInodes:true}] DiskMap:map[259:0:{Name:nvme0n1 Major:259 Minor:0 Size:128849018880 Scheduler:none}] NetworkDevices:[{Name:br-ex MacAddress:02:cb:ac:0b:9e:b3 Speed:0 Mtu:9001} {Name:ens5 MacAddress:02:cb:ac:0b:9e:b3 Speed:0 Mtu:9001} {Name:ovs-system MacAddress:32:74:2a:39:76:11 Speed:0 Mtu:1500}] Topology:[{Id:0 Memory:33164492800 HugePages:[{PageSize:1048576 NumPages:0} {PageSize:2048 NumPages:0}] Cores:[{Id:0 Threads:[0 4] Caches:[{Id:0 Size:32768 Type:Data Level:1} {Id:0 Size:32768 Type:Instruction Level:1} {Id:0 Size:1048576 Type:Unified Level:2}] UncoreCaches:[] SocketID:0 BookID: DrawerID:} {Id:1 Threads:[1 5] Caches:[{Id:1 Size:32768 Type:Data Level:1} {Id:1 Size:32768 Type:Instruction Level:1} {Id:1 Size:1048576 Type:Unified Level:2}] UncoreCaches:[] SocketID:0 BookID: DrawerID:} {Id:2 Threads:[2 6] Caches:[{Id:2 Size:32768 Type:Data Level:1} {Id:2 Size:32768 Type:Instruction Level:1} {Id:2 Size:1048576 Type:Unified Level:2}] UncoreCaches:[] SocketID:0 BookID: DrawerID:} {Id:3 Threads:[3 7] Caches:[{Id:3 Size:32768 Type:Data Level:1} {Id:3 Size:32768 Type:Instruction Level:1} {Id:3 Size:1048576 Type:Unified Level:2}] UncoreCaches:[] SocketID:0 BookID: DrawerID:}] Caches:[{Id:0 Size:37486592 Type:Unified Level:3}] Distances:[10]}] CloudProvider:Unknown InstanceType:Unknown InstanceID:None} Jul 10 18:39:46.267925 ip-10-0-143-17 kubenswrapper[2585]: I0710 18:39:45.785332 2585 manager_no_libpfm.go:29] cAdvisor is build without cgo and/or libpfm support. Perf event counters are not available. Jul 10 18:39:46.267925 ip-10-0-143-17 kubenswrapper[2585]: I0710 18:39:45.785435 2585 manager.go:232] Version: {KernelVersion:5.14.0-570.126.1.el9_6.x86_64 ContainerOsVersion:Red Hat Enterprise Linux CoreOS 9.6.20260702-0 (Plow) DockerVersion: DockerAPIVersion: CadvisorVersion: CadvisorRevision:} Jul 10 18:39:46.267925 ip-10-0-143-17 kubenswrapper[2585]: I0710 18:39:45.788710 2585 reflector.go:439] "Caches populated" logger="kubernetes.io/kube-apiserver-client-kubelet" type="*v1.CertificateSigningRequest" reflector="k8s.io/client-go/tools/watch/informerwatcher.go:162" Jul 10 18:39:46.267925 ip-10-0-143-17 kubenswrapper[2585]: I0710 18:39:45.789317 2585 container_manager_linux.go:273] "Container manager verified user specified cgroup-root exists" cgroupRoot=[] Jul 10 18:39:46.267925 ip-10-0-143-17 kubenswrapper[2585]: I0710 18:39:45.789349 2585 container_manager_linux.go:278] "Creating Container Manager object based on Node Config" nodeConfig={"NodeName":"ip-10-0-143-17.ec2.internal","RuntimeCgroupsName":"/system.slice/crio.service","SystemCgroupsName":"/system.slice","KubeletCgroupsName":"","KubeletOOMScoreAdj":-999,"ContainerRuntime":"","CgroupsPerQOS":true,"CgroupRoot":"/","CgroupDriver":"systemd","KubeletRootDir":"/var/lib/kubelet","ProtectKernelDefaults":true,"KubeReservedCgroupName":"","SystemReservedCgroupName":"","ReservedSystemCPUs":{},"EnforceNodeAllocatable":{"pods":{}},"KubeReserved":null,"SystemReserved":{"cpu":"500m","ephemeral-storage":"1Gi","memory":"1Gi"},"HardEvictionThresholds":[{"Signal":"imagefs.available","Operator":"LessThan","Value":{"Quantity":null,"Percentage":0.15},"GracePeriod":0,"MinReclaim":null},{"Signal":"imagefs.inodesFree","Operator":"LessThan","Value":{"Quantity":null,"Percentage":0.05},"GracePeriod":0,"MinReclaim":null},{"Signal":"memory.available","Operator":"LessThan","Value":{"Quantity":"100Mi","Percentage":0},"GracePeriod":0,"MinReclaim":null},{"Signal":"nodefs.available","Operator":"LessThan","Value":{"Quantity":null,"Percentage":0.1},"GracePeriod":0,"MinReclaim":null},{"Signal":"nodefs.inodesFree","Operator":"LessThan","Value":{"Quantity":null,"Percentage":0.05},"GracePeriod":0,"MinReclaim":null}],"QOSReserved":{},"CPUManagerPolicy":"none","CPUManagerPolicyOptions":null,"TopologyManagerScope":"container","CPUManagerReconcilePeriod":10000000000,"MemoryManagerPolicy":"None","MemoryManagerReservedMemory":null,"PodPidsLimit":4096,"EnforceCPULimits":true,"CPUCFSQuotaPeriod":100000000,"TopologyManagerPolicy":"none","TopologyManagerPolicyOptions":null,"CgroupVersion":2} Jul 10 18:39:46.267925 ip-10-0-143-17 kubenswrapper[2585]: I0710 18:39:45.790068 2585 topology_manager.go:138] "Creating topology manager with none policy" Jul 10 18:39:46.268217 ip-10-0-143-17 kubenswrapper[2585]: I0710 18:39:45.790078 2585 container_manager_linux.go:309] "Creating device plugin manager" Jul 10 18:39:46.268217 ip-10-0-143-17 kubenswrapper[2585]: I0710 18:39:45.790085 2585 manager.go:141] "Creating Device Plugin manager" path="/var/lib/kubelet/device-plugins/kubelet.sock" Jul 10 18:39:46.268217 ip-10-0-143-17 kubenswrapper[2585]: I0710 18:39:45.791023 2585 server.go:74] "Creating device plugin registration server" version="v1beta1" socket="/var/lib/kubelet/device-plugins/kubelet.sock" Jul 10 18:39:46.268217 ip-10-0-143-17 kubenswrapper[2585]: I0710 18:39:45.791953 2585 container_manager_linux.go:318] "Creating Dynamic Resource Allocation (DRA) manager" Jul 10 18:39:46.268217 ip-10-0-143-17 kubenswrapper[2585]: I0710 18:39:45.794776 2585 state_mem.go:36] "Initialized new in-memory state store" Jul 10 18:39:46.268217 ip-10-0-143-17 kubenswrapper[2585]: I0710 18:39:45.794890 2585 server.go:1240] "Using root directory" path="/var/lib/kubelet" Jul 10 18:39:46.268217 ip-10-0-143-17 kubenswrapper[2585]: I0710 18:39:45.794916 2585 kubelet.go:486] "Attempting to sync node with API server" Jul 10 18:39:46.268217 ip-10-0-143-17 kubenswrapper[2585]: I0710 18:39:45.794926 2585 kubelet.go:387] "Adding static pod path" path="/etc/kubernetes/manifests" Jul 10 18:39:46.268217 ip-10-0-143-17 kubenswrapper[2585]: I0710 18:39:45.794945 2585 file.go:69] "Watching path" path="/etc/kubernetes/manifests" Jul 10 18:39:46.268217 ip-10-0-143-17 kubenswrapper[2585]: I0710 18:39:45.794955 2585 kubelet.go:398] "Adding apiserver pod source" Jul 10 18:39:46.268217 ip-10-0-143-17 kubenswrapper[2585]: I0710 18:39:45.794963 2585 apiserver.go:42] "Waiting for node sync before watching apiserver pods" Jul 10 18:39:46.268217 ip-10-0-143-17 kubenswrapper[2585]: I0710 18:39:45.795274 2585 state_mem.go:40] "Initialized new in-memory state store for pod resource information tracking" Jul 10 18:39:46.268217 ip-10-0-143-17 kubenswrapper[2585]: I0710 18:39:45.795288 2585 state_mem.go:40] "Initialized new in-memory state store for pod resource information tracking" Jul 10 18:39:46.268217 ip-10-0-143-17 kubenswrapper[2585]: I0710 18:39:45.798891 2585 kuberuntime_manager.go:301] "Container runtime initialized" containerRuntime="cri-o" version="1.34.10-2.rhaos4.21.git83ee25b.el9" apiVersion="v1" Jul 10 18:39:46.268217 ip-10-0-143-17 kubenswrapper[2585]: I0710 18:39:45.800656 2585 kubelet.go:960] "Not starting ClusterTrustBundle informer because we are in static kubelet mode or the ClusterTrustBundleProjection featuregate is disabled" Jul 10 18:39:46.268217 ip-10-0-143-17 kubenswrapper[2585]: I0710 18:39:45.800684 2585 kubelet.go:984] "Not starting PodCertificateRequest manager because we are in static kubelet mode or the PodCertificateProjection feature gate is disabled" Jul 10 18:39:46.268217 ip-10-0-143-17 kubenswrapper[2585]: I0710 18:39:45.801438 2585 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/portworx-volume" Jul 10 18:39:46.268217 ip-10-0-143-17 kubenswrapper[2585]: I0710 18:39:45.801456 2585 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/empty-dir" Jul 10 18:39:46.268217 ip-10-0-143-17 kubenswrapper[2585]: I0710 18:39:45.801465 2585 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/git-repo" Jul 10 18:39:46.268217 ip-10-0-143-17 kubenswrapper[2585]: I0710 18:39:45.801473 2585 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/host-path" Jul 10 18:39:46.268217 ip-10-0-143-17 kubenswrapper[2585]: I0710 18:39:45.801484 2585 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/nfs" Jul 10 18:39:46.268217 ip-10-0-143-17 kubenswrapper[2585]: I0710 18:39:45.801491 2585 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/secret" Jul 10 18:39:46.268217 ip-10-0-143-17 kubenswrapper[2585]: I0710 18:39:45.801529 2585 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/iscsi" Jul 10 18:39:46.268217 ip-10-0-143-17 kubenswrapper[2585]: I0710 18:39:45.801537 2585 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/downward-api" Jul 10 18:39:46.268217 ip-10-0-143-17 kubenswrapper[2585]: I0710 18:39:45.801545 2585 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/fc" Jul 10 18:39:46.268217 ip-10-0-143-17 kubenswrapper[2585]: I0710 18:39:45.801553 2585 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/configmap" Jul 10 18:39:46.268217 ip-10-0-143-17 kubenswrapper[2585]: I0710 18:39:45.801565 2585 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/projected" Jul 10 18:39:46.268217 ip-10-0-143-17 kubenswrapper[2585]: I0710 18:39:45.801576 2585 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/local-volume" Jul 10 18:39:46.268217 ip-10-0-143-17 kubenswrapper[2585]: I0710 18:39:45.802269 2585 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/csi" Jul 10 18:39:46.268217 ip-10-0-143-17 kubenswrapper[2585]: I0710 18:39:45.802279 2585 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/image" Jul 10 18:39:46.268217 ip-10-0-143-17 kubenswrapper[2585]: I0710 18:39:45.805593 2585 csi_plugin.go:990] Failed to contact API server when waiting for CSINode publishing: csinodes.storage.k8s.io "ip-10-0-143-17.ec2.internal" is forbidden: User "system:anonymous" cannot get resource "csinodes" in API group "storage.k8s.io" at the cluster scope Jul 10 18:39:46.268217 ip-10-0-143-17 kubenswrapper[2585]: E0710 18:39:45.805754 2585 reflector.go:205] "Failed to watch" err="failed to list *v1.Service: services is forbidden: User \"system:anonymous\" cannot list resource \"services\" in API group \"\" at the cluster scope" logger="UnhandledError" reflector="k8s.io/client-go/informers/factory.go:160" type="*v1.Service" Jul 10 18:39:46.268217 ip-10-0-143-17 kubenswrapper[2585]: E0710 18:39:45.805820 2585 reflector.go:205] "Failed to watch" err="failed to list *v1.Node: nodes \"ip-10-0-143-17.ec2.internal\" is forbidden: User \"system:anonymous\" cannot list resource \"nodes\" in API group \"\" at the cluster scope" logger="UnhandledError" reflector="k8s.io/client-go/informers/factory.go:160" type="*v1.Node" Jul 10 18:39:46.268217 ip-10-0-143-17 kubenswrapper[2585]: I0710 18:39:45.806364 2585 server.go:1268] "Started kubelet" Jul 10 18:39:46.268217 ip-10-0-143-17 kubenswrapper[2585]: I0710 18:39:45.806414 2585 server.go:180] "Starting to listen" address="0.0.0.0" port=10250 Jul 10 18:39:46.268217 ip-10-0-143-17 kubenswrapper[2585]: I0710 18:39:45.806448 2585 ratelimit.go:56] "Setting rate limiting for endpoint" service="podresources" qps=100 burstTokens=10 Jul 10 18:39:46.268217 ip-10-0-143-17 kubenswrapper[2585]: I0710 18:39:45.806510 2585 server_v1.go:49] "podresources" method="list" useActivePods=true Jul 10 18:39:46.268217 ip-10-0-143-17 kubenswrapper[2585]: I0710 18:39:45.810852 2585 server.go:249] "Starting to serve the podresources API" endpoint="unix:/var/lib/kubelet/pod-resources/kubelet.sock" Jul 10 18:39:46.268217 ip-10-0-143-17 kubenswrapper[2585]: I0710 18:39:45.812815 2585 server.go:310] "Adding debug handlers to kubelet server" Jul 10 18:39:46.268217 ip-10-0-143-17 kubenswrapper[2585]: I0710 18:39:45.812936 2585 csr.go:274] "Certificate signing request is approved, waiting to be issued" logger="kubernetes.io/kube-apiserver-client-kubelet" csr="csr-cgcsw" Jul 10 18:39:46.269402 ip-10-0-143-17 kubenswrapper[2585]: E0710 18:39:45.813055 2585 kubelet.go:1638] "Image garbage collection failed once. Stats initialization may not have completed yet" err="failed to get imageFs info: unable to find data in memory cache" Jul 10 18:39:46.269402 ip-10-0-143-17 kubenswrapper[2585]: E0710 18:39:45.814782 2585 event.go:359] "Server rejected event (will not retry!)" err="events is forbidden: User \"system:anonymous\" cannot create resource \"events\" in API group \"\" in the namespace \"default\"" event="&Event{ObjectMeta:{ip-10-0-143-17.ec2.internal.18c101b0b0b6de5f default 0 0001-01-01 00:00:00 +0000 UTC map[] map[] [] [] []},InvolvedObject:ObjectReference{Kind:Node,Namespace:,Name:ip-10-0-143-17.ec2.internal,UID:ip-10-0-143-17.ec2.internal,APIVersion:,ResourceVersion:,FieldPath:,},Reason:Starting,Message:Starting kubelet.,Source:EventSource{Component:kubelet,Host:ip-10-0-143-17.ec2.internal,},FirstTimestamp:2026-07-10 18:39:45.806073439 +0000 UTC m=+0.381763987,LastTimestamp:2026-07-10 18:39:45.806073439 +0000 UTC m=+0.381763987,Count:1,Type:Normal,EventTime:0001-01-01 00:00:00 +0000 UTC,Series:nil,Action:,Related:nil,ReportingController:kubelet,ReportingInstance:ip-10-0-143-17.ec2.internal,}" Jul 10 18:39:46.269402 ip-10-0-143-17 kubenswrapper[2585]: I0710 18:39:45.816573 2585 fs_resource_analyzer.go:67] "Starting FS ResourceAnalyzer" Jul 10 18:39:46.269402 ip-10-0-143-17 kubenswrapper[2585]: I0710 18:39:45.816587 2585 certificate_manager.go:422] "Certificate rotation is enabled" logger="kubernetes.io/kubelet-serving" Jul 10 18:39:46.269402 ip-10-0-143-17 kubenswrapper[2585]: E0710 18:39:45.816701 2585 kubelet_node_status.go:454] "Error getting the current node from lister" err="node \"ip-10-0-143-17.ec2.internal\" not found" Jul 10 18:39:46.269402 ip-10-0-143-17 kubenswrapper[2585]: I0710 18:39:45.817344 2585 desired_state_of_world_populator.go:146] "Desired state populator starts to run" Jul 10 18:39:46.269402 ip-10-0-143-17 kubenswrapper[2585]: I0710 18:39:45.817378 2585 volume_manager.go:311] "The desired_state_of_world populator starts" Jul 10 18:39:46.269402 ip-10-0-143-17 kubenswrapper[2585]: I0710 18:39:45.817391 2585 volume_manager.go:313] "Starting Kubelet Volume Manager" Jul 10 18:39:46.269402 ip-10-0-143-17 kubenswrapper[2585]: I0710 18:39:45.817407 2585 factory.go:55] Registering systemd factory Jul 10 18:39:46.269402 ip-10-0-143-17 kubenswrapper[2585]: I0710 18:39:45.817419 2585 factory.go:223] Registration of the systemd container factory successfully Jul 10 18:39:46.269402 ip-10-0-143-17 kubenswrapper[2585]: I0710 18:39:45.817475 2585 reconstruct.go:97] "Volume reconstruction finished" Jul 10 18:39:46.269402 ip-10-0-143-17 kubenswrapper[2585]: I0710 18:39:45.817483 2585 reconciler.go:29] "Reconciler: start to sync state" Jul 10 18:39:46.269402 ip-10-0-143-17 kubenswrapper[2585]: I0710 18:39:45.817596 2585 factory.go:153] Registering CRI-O factory Jul 10 18:39:46.269402 ip-10-0-143-17 kubenswrapper[2585]: I0710 18:39:45.817608 2585 factory.go:223] Registration of the crio container factory successfully Jul 10 18:39:46.269402 ip-10-0-143-17 kubenswrapper[2585]: I0710 18:39:45.817661 2585 factory.go:221] Registration of the containerd container factory failed: unable to create containerd client: containerd: cannot unix dial containerd api service: dial unix /run/containerd/containerd.sock: connect: no such file or directory Jul 10 18:39:46.269402 ip-10-0-143-17 kubenswrapper[2585]: I0710 18:39:45.817682 2585 factory.go:103] Registering Raw factory Jul 10 18:39:46.269402 ip-10-0-143-17 kubenswrapper[2585]: I0710 18:39:45.817696 2585 manager.go:1196] Started watching for new ooms in manager Jul 10 18:39:46.269402 ip-10-0-143-17 kubenswrapper[2585]: I0710 18:39:45.818080 2585 manager.go:349] Starting recovery of all containers Jul 10 18:39:46.269402 ip-10-0-143-17 kubenswrapper[2585]: I0710 18:39:45.819494 2585 csr.go:270] "Certificate signing request is issued" logger="kubernetes.io/kube-apiserver-client-kubelet" csr="csr-cgcsw" Jul 10 18:39:46.269402 ip-10-0-143-17 kubenswrapper[2585]: E0710 18:39:45.819620 2585 controller.go:145] "Failed to ensure lease exists, will retry" err="leases.coordination.k8s.io \"ip-10-0-143-17.ec2.internal\" is forbidden: User \"system:anonymous\" cannot get resource \"leases\" in API group \"coordination.k8s.io\" in the namespace \"kube-node-lease\"" interval="200ms" Jul 10 18:39:46.269402 ip-10-0-143-17 kubenswrapper[2585]: E0710 18:39:45.819690 2585 reflector.go:205] "Failed to watch" err="failed to list *v1.CSIDriver: csidrivers.storage.k8s.io is forbidden: User \"system:anonymous\" cannot list resource \"csidrivers\" in API group \"storage.k8s.io\" at the cluster scope" logger="UnhandledError" reflector="k8s.io/client-go/informers/factory.go:160" type="*v1.CSIDriver" Jul 10 18:39:46.269402 ip-10-0-143-17 kubenswrapper[2585]: I0710 18:39:45.829938 2585 manager.go:354] Recovery completed Jul 10 18:39:46.269402 ip-10-0-143-17 kubenswrapper[2585]: I0710 18:39:45.834004 2585 kubelet_node_status.go:400] "Setting node annotation to enable volume controller attach/detach" Jul 10 18:39:46.269402 ip-10-0-143-17 kubenswrapper[2585]: I0710 18:39:45.835975 2585 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-143-17.ec2.internal" event="NodeHasSufficientMemory" Jul 10 18:39:46.269402 ip-10-0-143-17 kubenswrapper[2585]: I0710 18:39:45.836003 2585 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-143-17.ec2.internal" event="NodeHasNoDiskPressure" Jul 10 18:39:46.269402 ip-10-0-143-17 kubenswrapper[2585]: I0710 18:39:45.836016 2585 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-143-17.ec2.internal" event="NodeHasSufficientPID" Jul 10 18:39:46.269402 ip-10-0-143-17 kubenswrapper[2585]: I0710 18:39:45.838569 2585 cpu_manager.go:222] "Starting CPU manager" policy="none" Jul 10 18:39:46.269402 ip-10-0-143-17 kubenswrapper[2585]: I0710 18:39:45.838583 2585 cpu_manager.go:223] "Reconciling" reconcilePeriod="10s" Jul 10 18:39:46.269402 ip-10-0-143-17 kubenswrapper[2585]: I0710 18:39:45.838612 2585 state_mem.go:36] "Initialized new in-memory state store" Jul 10 18:39:46.269402 ip-10-0-143-17 kubenswrapper[2585]: I0710 18:39:45.838638 2585 kubelet_node_status.go:400] "Setting node annotation to enable volume controller attach/detach" Jul 10 18:39:46.269402 ip-10-0-143-17 kubenswrapper[2585]: I0710 18:39:45.839450 2585 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-143-17.ec2.internal" event="NodeHasSufficientMemory" Jul 10 18:39:46.269402 ip-10-0-143-17 kubenswrapper[2585]: I0710 18:39:45.839481 2585 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-143-17.ec2.internal" event="NodeHasNoDiskPressure" Jul 10 18:39:46.270299 ip-10-0-143-17 kubenswrapper[2585]: I0710 18:39:45.839491 2585 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-143-17.ec2.internal" event="NodeHasSufficientPID" Jul 10 18:39:46.270299 ip-10-0-143-17 kubenswrapper[2585]: I0710 18:39:45.839742 2585 policy_none.go:49] "None policy: Start" Jul 10 18:39:46.270299 ip-10-0-143-17 kubenswrapper[2585]: I0710 18:39:45.839758 2585 memory_manager.go:187] "Starting memorymanager" policy="None" Jul 10 18:39:46.270299 ip-10-0-143-17 kubenswrapper[2585]: I0710 18:39:45.839772 2585 state_mem.go:36] "Initializing new in-memory state store" logger="Memory Manager state checkpoint" Jul 10 18:39:46.270299 ip-10-0-143-17 kubenswrapper[2585]: I0710 18:39:45.840894 2585 policy_none.go:47] "Start" Jul 10 18:39:46.270299 ip-10-0-143-17 kubenswrapper[2585]: I0710 18:39:45.888829 2585 manager.go:337] "Starting Device Plugin manager" Jul 10 18:39:46.270299 ip-10-0-143-17 kubenswrapper[2585]: E0710 18:39:45.888858 2585 manager.go:513] "Failed to read data from checkpoint" err="checkpoint is not found" checkpoint="kubelet_internal_checkpoint" Jul 10 18:39:46.270299 ip-10-0-143-17 kubenswrapper[2585]: I0710 18:39:45.888868 2585 server.go:87] "Starting device plugin registration server" Jul 10 18:39:46.270299 ip-10-0-143-17 kubenswrapper[2585]: I0710 18:39:45.889098 2585 eviction_manager.go:189] "Eviction manager: starting control loop" Jul 10 18:39:46.270299 ip-10-0-143-17 kubenswrapper[2585]: I0710 18:39:45.889111 2585 container_log_manager.go:146] "Initializing container log rotate workers" workers=1 monitorPeriod="10s" Jul 10 18:39:46.270299 ip-10-0-143-17 kubenswrapper[2585]: I0710 18:39:45.889215 2585 plugin_watcher.go:51] "Plugin Watcher Start" path="/var/lib/kubelet/plugins_registry" Jul 10 18:39:46.270299 ip-10-0-143-17 kubenswrapper[2585]: I0710 18:39:45.889290 2585 plugin_manager.go:116] "The desired_state_of_world populator (plugin watcher) starts" Jul 10 18:39:46.270299 ip-10-0-143-17 kubenswrapper[2585]: I0710 18:39:45.889299 2585 plugin_manager.go:118] "Starting Kubelet Plugin Manager" Jul 10 18:39:46.270299 ip-10-0-143-17 kubenswrapper[2585]: E0710 18:39:45.889756 2585 eviction_manager.go:267] "eviction manager: failed to check if we have separate container filesystem. Ignoring." err="non-existent label \"crio-containers\"" Jul 10 18:39:46.270299 ip-10-0-143-17 kubenswrapper[2585]: E0710 18:39:45.889798 2585 eviction_manager.go:292] "Eviction manager: failed to get summary stats" err="failed to get node info: node \"ip-10-0-143-17.ec2.internal\" not found" Jul 10 18:39:46.270299 ip-10-0-143-17 kubenswrapper[2585]: I0710 18:39:45.989330 2585 kubelet_node_status.go:400] "Setting node annotation to enable volume controller attach/detach" Jul 10 18:39:46.270299 ip-10-0-143-17 kubenswrapper[2585]: I0710 18:39:45.990281 2585 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-143-17.ec2.internal" event="NodeHasSufficientMemory" Jul 10 18:39:46.270299 ip-10-0-143-17 kubenswrapper[2585]: I0710 18:39:45.990308 2585 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-143-17.ec2.internal" event="NodeHasNoDiskPressure" Jul 10 18:39:46.270299 ip-10-0-143-17 kubenswrapper[2585]: I0710 18:39:45.990323 2585 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-143-17.ec2.internal" event="NodeHasSufficientPID" Jul 10 18:39:46.270299 ip-10-0-143-17 kubenswrapper[2585]: I0710 18:39:45.990349 2585 kubelet_node_status.go:78] "Attempting to register node" node="ip-10-0-143-17.ec2.internal" Jul 10 18:39:46.270299 ip-10-0-143-17 kubenswrapper[2585]: I0710 18:39:45.998073 2585 kubelet_node_status.go:81] "Successfully registered node" node="ip-10-0-143-17.ec2.internal" Jul 10 18:39:46.270299 ip-10-0-143-17 kubenswrapper[2585]: E0710 18:39:45.998089 2585 kubelet_node_status.go:536] "Error updating node status, will retry" err="error getting node \"ip-10-0-143-17.ec2.internal\": node \"ip-10-0-143-17.ec2.internal\" not found" Jul 10 18:39:46.270299 ip-10-0-143-17 kubenswrapper[2585]: E0710 18:39:46.031314 2585 kubelet_node_status.go:454] "Error getting the current node from lister" err="node \"ip-10-0-143-17.ec2.internal\" not found" Jul 10 18:39:46.270299 ip-10-0-143-17 kubenswrapper[2585]: I0710 18:39:46.090137 2585 kubelet_network_linux.go:54] "Initialized iptables rules." protocol="IPv4" Jul 10 18:39:46.270299 ip-10-0-143-17 kubenswrapper[2585]: I0710 18:39:46.091467 2585 kubelet_network_linux.go:54] "Initialized iptables rules." protocol="IPv6" Jul 10 18:39:46.270299 ip-10-0-143-17 kubenswrapper[2585]: I0710 18:39:46.091501 2585 status_manager.go:244] "Starting to sync pod status with apiserver" Jul 10 18:39:46.270299 ip-10-0-143-17 kubenswrapper[2585]: I0710 18:39:46.091519 2585 kubelet.go:2448] "Starting kubelet main sync loop" Jul 10 18:39:46.270299 ip-10-0-143-17 kubenswrapper[2585]: E0710 18:39:46.091555 2585 kubelet.go:2472] "Skipping pod synchronization" err="PLEG is not healthy: pleg has yet to be successful" Jul 10 18:39:46.270299 ip-10-0-143-17 kubenswrapper[2585]: I0710 18:39:46.093739 2585 reflector.go:439] "Caches populated" type="*v1.RuntimeClass" reflector="k8s.io/client-go/informers/factory.go:160" Jul 10 18:39:46.270299 ip-10-0-143-17 kubenswrapper[2585]: E0710 18:39:46.132165 2585 kubelet_node_status.go:454] "Error getting the current node from lister" err="node \"ip-10-0-143-17.ec2.internal\" not found" Jul 10 18:39:46.270299 ip-10-0-143-17 kubenswrapper[2585]: I0710 18:39:46.192351 2585 kubelet.go:2535] "SyncLoop ADD" source="file" pods=["openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-143-17.ec2.internal","kube-system/kube-apiserver-proxy-ip-10-0-143-17.ec2.internal"] Jul 10 18:39:46.270299 ip-10-0-143-17 kubenswrapper[2585]: I0710 18:39:46.192444 2585 kubelet_node_status.go:400] "Setting node annotation to enable volume controller attach/detach" Jul 10 18:39:46.270299 ip-10-0-143-17 kubenswrapper[2585]: I0710 18:39:46.193508 2585 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-143-17.ec2.internal" event="NodeHasSufficientMemory" Jul 10 18:39:46.270299 ip-10-0-143-17 kubenswrapper[2585]: I0710 18:39:46.193537 2585 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-143-17.ec2.internal" event="NodeHasNoDiskPressure" Jul 10 18:39:46.270299 ip-10-0-143-17 kubenswrapper[2585]: I0710 18:39:46.193547 2585 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-143-17.ec2.internal" event="NodeHasSufficientPID" Jul 10 18:39:46.270299 ip-10-0-143-17 kubenswrapper[2585]: I0710 18:39:46.194889 2585 kubelet_node_status.go:400] "Setting node annotation to enable volume controller attach/detach" Jul 10 18:39:46.270299 ip-10-0-143-17 kubenswrapper[2585]: I0710 18:39:46.195006 2585 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-143-17.ec2.internal" Jul 10 18:39:46.270299 ip-10-0-143-17 kubenswrapper[2585]: I0710 18:39:46.195034 2585 kubelet_node_status.go:400] "Setting node annotation to enable volume controller attach/detach" Jul 10 18:39:46.275414 ip-10-0-143-17 kubenswrapper[2585]: I0710 18:39:46.195573 2585 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-143-17.ec2.internal" event="NodeHasSufficientMemory" Jul 10 18:39:46.275414 ip-10-0-143-17 kubenswrapper[2585]: I0710 18:39:46.195599 2585 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-143-17.ec2.internal" event="NodeHasNoDiskPressure" Jul 10 18:39:46.275414 ip-10-0-143-17 kubenswrapper[2585]: I0710 18:39:46.195610 2585 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-143-17.ec2.internal" event="NodeHasSufficientPID" Jul 10 18:39:46.275414 ip-10-0-143-17 kubenswrapper[2585]: I0710 18:39:46.195612 2585 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-143-17.ec2.internal" event="NodeHasSufficientMemory" Jul 10 18:39:46.275414 ip-10-0-143-17 kubenswrapper[2585]: I0710 18:39:46.195630 2585 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-143-17.ec2.internal" event="NodeHasNoDiskPressure" Jul 10 18:39:46.275414 ip-10-0-143-17 kubenswrapper[2585]: I0710 18:39:46.195641 2585 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-143-17.ec2.internal" event="NodeHasSufficientPID" Jul 10 18:39:46.275414 ip-10-0-143-17 kubenswrapper[2585]: I0710 18:39:46.196779 2585 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/kube-apiserver-proxy-ip-10-0-143-17.ec2.internal" Jul 10 18:39:46.275414 ip-10-0-143-17 kubenswrapper[2585]: I0710 18:39:46.196801 2585 kubelet_node_status.go:400] "Setting node annotation to enable volume controller attach/detach" Jul 10 18:39:46.275414 ip-10-0-143-17 kubenswrapper[2585]: I0710 18:39:46.197471 2585 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-143-17.ec2.internal" event="NodeHasSufficientMemory" Jul 10 18:39:46.275414 ip-10-0-143-17 kubenswrapper[2585]: I0710 18:39:46.197496 2585 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-143-17.ec2.internal" event="NodeHasNoDiskPressure" Jul 10 18:39:46.275414 ip-10-0-143-17 kubenswrapper[2585]: I0710 18:39:46.197506 2585 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-143-17.ec2.internal" event="NodeHasSufficientPID" Jul 10 18:39:46.275414 ip-10-0-143-17 kubenswrapper[2585]: I0710 18:39:46.219348 2585 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"etc-kube\" (UniqueName: \"kubernetes.io/host-path/3813618e37881b79cb3d5b62e1c14d5b-etc-kube\") pod \"kube-rbac-proxy-crio-ip-10-0-143-17.ec2.internal\" (UID: \"3813618e37881b79cb3d5b62e1c14d5b\") " pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-143-17.ec2.internal" Jul 10 18:39:46.275414 ip-10-0-143-17 kubenswrapper[2585]: I0710 18:39:46.219372 2585 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"var-lib-kubelet\" (UniqueName: \"kubernetes.io/host-path/3813618e37881b79cb3d5b62e1c14d5b-var-lib-kubelet\") pod \"kube-rbac-proxy-crio-ip-10-0-143-17.ec2.internal\" (UID: \"3813618e37881b79cb3d5b62e1c14d5b\") " pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-143-17.ec2.internal" Jul 10 18:39:46.275414 ip-10-0-143-17 kubenswrapper[2585]: I0710 18:39:46.219390 2585 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"config\" (UniqueName: \"kubernetes.io/host-path/23dbc229ee3e12019dd90879f1a0df34-config\") pod \"kube-apiserver-proxy-ip-10-0-143-17.ec2.internal\" (UID: \"23dbc229ee3e12019dd90879f1a0df34\") " pod="kube-system/kube-apiserver-proxy-ip-10-0-143-17.ec2.internal" Jul 10 18:39:46.275414 ip-10-0-143-17 kubenswrapper[2585]: E0710 18:39:46.232549 2585 kubelet_node_status.go:454] "Error getting the current node from lister" err="node \"ip-10-0-143-17.ec2.internal\" not found" Jul 10 18:39:46.275414 ip-10-0-143-17 kubenswrapper[2585]: E0710 18:39:46.233949 2585 kubelet.go:3244] "No need to create a mirror pod, since failed to get node info from the cluster" err="node \"ip-10-0-143-17.ec2.internal\" not found" node="ip-10-0-143-17.ec2.internal" Jul 10 18:39:46.275414 ip-10-0-143-17 kubenswrapper[2585]: E0710 18:39:46.238531 2585 kubelet.go:3244] "No need to create a mirror pod, since failed to get node info from the cluster" err="node \"ip-10-0-143-17.ec2.internal\" not found" node="ip-10-0-143-17.ec2.internal" Jul 10 18:39:46.429225 ip-10-0-143-17 kubenswrapper[2585]: I0710 18:39:46.319487 2585 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"etc-kube\" (UniqueName: \"kubernetes.io/host-path/3813618e37881b79cb3d5b62e1c14d5b-etc-kube\") pod \"kube-rbac-proxy-crio-ip-10-0-143-17.ec2.internal\" (UID: \"3813618e37881b79cb3d5b62e1c14d5b\") " pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-143-17.ec2.internal" Jul 10 18:39:46.429225 ip-10-0-143-17 kubenswrapper[2585]: I0710 18:39:46.319517 2585 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"var-lib-kubelet\" (UniqueName: \"kubernetes.io/host-path/3813618e37881b79cb3d5b62e1c14d5b-var-lib-kubelet\") pod \"kube-rbac-proxy-crio-ip-10-0-143-17.ec2.internal\" (UID: \"3813618e37881b79cb3d5b62e1c14d5b\") " pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-143-17.ec2.internal" Jul 10 18:39:46.429225 ip-10-0-143-17 kubenswrapper[2585]: I0710 18:39:46.319534 2585 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"config\" (UniqueName: \"kubernetes.io/host-path/23dbc229ee3e12019dd90879f1a0df34-config\") pod \"kube-apiserver-proxy-ip-10-0-143-17.ec2.internal\" (UID: \"23dbc229ee3e12019dd90879f1a0df34\") " pod="kube-system/kube-apiserver-proxy-ip-10-0-143-17.ec2.internal" Jul 10 18:39:46.429225 ip-10-0-143-17 kubenswrapper[2585]: I0710 18:39:46.319576 2585 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"config\" (UniqueName: \"kubernetes.io/host-path/23dbc229ee3e12019dd90879f1a0df34-config\") pod \"kube-apiserver-proxy-ip-10-0-143-17.ec2.internal\" (UID: \"23dbc229ee3e12019dd90879f1a0df34\") " pod="kube-system/kube-apiserver-proxy-ip-10-0-143-17.ec2.internal" Jul 10 18:39:46.429225 ip-10-0-143-17 kubenswrapper[2585]: I0710 18:39:46.319578 2585 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"var-lib-kubelet\" (UniqueName: \"kubernetes.io/host-path/3813618e37881b79cb3d5b62e1c14d5b-var-lib-kubelet\") pod \"kube-rbac-proxy-crio-ip-10-0-143-17.ec2.internal\" (UID: \"3813618e37881b79cb3d5b62e1c14d5b\") " pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-143-17.ec2.internal" Jul 10 18:39:46.429225 ip-10-0-143-17 kubenswrapper[2585]: I0710 18:39:46.319577 2585 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"etc-kube\" (UniqueName: \"kubernetes.io/host-path/3813618e37881b79cb3d5b62e1c14d5b-etc-kube\") pod \"kube-rbac-proxy-crio-ip-10-0-143-17.ec2.internal\" (UID: \"3813618e37881b79cb3d5b62e1c14d5b\") " pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-143-17.ec2.internal" Jul 10 18:39:46.429225 ip-10-0-143-17 kubenswrapper[2585]: E0710 18:39:46.333578 2585 kubelet_node_status.go:454] "Error getting the current node from lister" err="node \"ip-10-0-143-17.ec2.internal\" not found" Jul 10 18:39:46.545608 ip-10-0-143-17 kubenswrapper[2585]: E0710 18:39:46.434100 2585 kubelet_node_status.go:454] "Error getting the current node from lister" err="node \"ip-10-0-143-17.ec2.internal\" not found" Jul 10 18:39:46.545608 ip-10-0-143-17 kubenswrapper[2585]: E0710 18:39:46.534547 2585 kubelet_node_status.go:454] "Error getting the current node from lister" err="node \"ip-10-0-143-17.ec2.internal\" not found" Jul 10 18:39:46.545608 ip-10-0-143-17 kubenswrapper[2585]: I0710 18:39:46.538710 2585 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-143-17.ec2.internal" Jul 10 18:39:46.546792 ip-10-0-143-17 kubenswrapper[2585]: I0710 18:39:46.546575 2585 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/kube-apiserver-proxy-ip-10-0-143-17.ec2.internal" Jul 10 18:39:47.821186 ip-10-0-143-17 kubenswrapper[2585]: E0710 18:39:46.635306 2585 kubelet_node_status.go:454] "Error getting the current node from lister" err="node \"ip-10-0-143-17.ec2.internal\" not found" Jul 10 18:39:47.946156 ip-10-0-143-17 kubenswrapper[2585]: E0710 18:39:47.854106 2585 kubelet_node_status.go:454] "Error getting the current node from lister" err="node \"ip-10-0-143-17.ec2.internal\" not found" Jul 10 18:39:47.946156 ip-10-0-143-17 kubenswrapper[2585]: I0710 18:39:47.877260 2585 transport.go:147] "Certificate rotation detected, shutting down client connections to start using new credentials" Jul 10 18:39:47.946156 ip-10-0-143-17 kubenswrapper[2585]: I0710 18:39:47.877402 2585 reflector.go:571] "Warning: watch ended with error" reflector="k8s.io/client-go/informers/factory.go:160" type="*v1.RuntimeClass" err="very short watch: k8s.io/client-go/informers/factory.go:160: Unexpected watch close - watch lasted less than a second and no items received" Jul 10 18:39:47.946156 ip-10-0-143-17 kubenswrapper[2585]: I0710 18:39:47.941521 2585 certificate_manager.go:715] "Certificate rotation deadline determined" logger="kubernetes.io/kube-apiserver-client-kubelet" expiration="2028-07-09 18:34:46 +0000 UTC" deadline="2027-12-12 09:47:13.927971095 +0000 UTC" Jul 10 18:39:47.946156 ip-10-0-143-17 kubenswrapper[2585]: I0710 18:39:47.941587 2585 certificate_manager.go:431] "Waiting for next certificate rotation" logger="kubernetes.io/kube-apiserver-client-kubelet" sleep="12471h7m25.986388595s" Jul 10 18:39:48.075072 ip-10-0-143-17 kubenswrapper[2585]: E0710 18:39:47.954765 2585 kubelet_node_status.go:454] "Error getting the current node from lister" err="node \"ip-10-0-143-17.ec2.internal\" not found" Jul 10 18:39:48.075072 ip-10-0-143-17 kubenswrapper[2585]: I0710 18:39:47.961803 2585 reflector.go:439] "Caches populated" type="*v1.Service" reflector="k8s.io/client-go/informers/factory.go:160" Jul 10 18:39:48.075072 ip-10-0-143-17 kubenswrapper[2585]: I0710 18:39:48.023967 2585 reflector.go:439] "Caches populated" type="*v1.Node" reflector="k8s.io/client-go/informers/factory.go:160" Jul 10 18:39:48.075072 ip-10-0-143-17 kubenswrapper[2585]: I0710 18:39:48.035852 2585 kubelet.go:3248] "Creating a mirror pod for static pod" pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-143-17.ec2.internal" Jul 10 18:39:48.075072 ip-10-0-143-17 kubenswrapper[2585]: I0710 18:39:48.047502 2585 warnings.go:110] "Warning: metadata.name: this is used in the Pod's hostname, which can result in surprising behavior; a DNS label is recommended: [must not contain dots]" Jul 10 18:39:48.075072 ip-10-0-143-17 kubenswrapper[2585]: I0710 18:39:48.049312 2585 kubelet.go:3248] "Creating a mirror pod for static pod" pod="kube-system/kube-apiserver-proxy-ip-10-0-143-17.ec2.internal" Jul 10 18:39:48.075072 ip-10-0-143-17 kubenswrapper[2585]: I0710 18:39:48.055898 2585 warnings.go:110] "Warning: metadata.name: this is used in the Pod's hostname, which can result in surprising behavior; a DNS label is recommended: [must not contain dots]" Jul 10 18:39:48.302575 ip-10-0-143-17 kubenswrapper[2585]: W0710 18:39:48.252486 2585 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod23dbc229ee3e12019dd90879f1a0df34.slice/crio-af2f0caa4063978d98bfaeeaf7e2d5ba540017447c5414838c333a3203e91c5c WatchSource:0}: Error finding container af2f0caa4063978d98bfaeeaf7e2d5ba540017447c5414838c333a3203e91c5c: Status 404 returned error can't find the container with id af2f0caa4063978d98bfaeeaf7e2d5ba540017447c5414838c333a3203e91c5c Jul 10 18:39:48.302575 ip-10-0-143-17 kubenswrapper[2585]: W0710 18:39:48.263290 2585 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod3813618e37881b79cb3d5b62e1c14d5b.slice/crio-c8dbcf9c9570e82db6d2885b49f22e5e04a85f9a3089f34f710c411ab50ba1b2 WatchSource:0}: Error finding container c8dbcf9c9570e82db6d2885b49f22e5e04a85f9a3089f34f710c411ab50ba1b2: Status 404 returned error can't find the container with id c8dbcf9c9570e82db6d2885b49f22e5e04a85f9a3089f34f710c411ab50ba1b2 Jul 10 18:39:48.302575 ip-10-0-143-17 kubenswrapper[2585]: I0710 18:39:48.264369 2585 provider.go:93] Refreshing cache for provider: *credentialprovider.defaultDockerConfigProvider Jul 10 18:39:48.474779 ip-10-0-143-17 kubenswrapper[2585]: I0710 18:39:48.318485 2585 reflector.go:439] "Caches populated" type="*v1.CSIDriver" reflector="k8s.io/client-go/informers/factory.go:160" Jul 10 18:39:48.960665 ip-10-0-143-17 kubenswrapper[2585]: I0710 18:39:48.914147 2585 apiserver.go:52] "Watching apiserver" Jul 10 18:39:48.960665 ip-10-0-143-17 kubenswrapper[2585]: I0710 18:39:48.925214 2585 reflector.go:439] "Caches populated" type="*v1.Pod" reflector="pkg/kubelet/config/apiserver.go:66" Jul 10 18:39:48.960665 ip-10-0-143-17 kubenswrapper[2585]: I0710 18:39:48.926446 2585 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-143-17.ec2.internal","openshift-multus/multus-additional-cni-plugins-w4cxp","openshift-network-operator/iptables-alerter-m5446","openshift-ovn-kubernetes/ovnkube-node-xg6bm","kube-system/global-pull-secret-syncer-hhpnn","kube-system/kube-apiserver-proxy-ip-10-0-143-17.ec2.internal","openshift-cluster-node-tuning-operator/tuned-w6ds9","openshift-image-registry/node-ca-crbxd","openshift-multus/multus-h4nv5","openshift-multus/network-metrics-daemon-7rwwk","openshift-network-diagnostics/network-check-target-7tsng","kube-system/konnectivity-agent-x7zf6","openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-cw6g2","openshift-dns/node-resolver-ndwqb"] Jul 10 18:39:48.960665 ip-10-0-143-17 kubenswrapper[2585]: I0710 18:39:48.928202 2585 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-cluster-node-tuning-operator/tuned-w6ds9" Jul 10 18:39:48.960665 ip-10-0-143-17 kubenswrapper[2585]: I0710 18:39:48.930786 2585 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-cluster-node-tuning-operator\"/\"openshift-service-ca.crt\"" Jul 10 18:39:48.960665 ip-10-0-143-17 kubenswrapper[2585]: I0710 18:39:48.931353 2585 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-cw6g2" Jul 10 18:39:48.960665 ip-10-0-143-17 kubenswrapper[2585]: I0710 18:39:48.931693 2585 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-cluster-node-tuning-operator\"/\"kube-root-ca.crt\"" Jul 10 18:39:48.960665 ip-10-0-143-17 kubenswrapper[2585]: I0710 18:39:48.931701 2585 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-cluster-node-tuning-operator\"/\"tuned-dockercfg-9swvg\"" Jul 10 18:39:48.960665 ip-10-0-143-17 kubenswrapper[2585]: I0710 18:39:48.932622 2585 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-operator/iptables-alerter-m5446" Jul 10 18:39:48.960665 ip-10-0-143-17 kubenswrapper[2585]: I0710 18:39:48.934027 2585 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-cluster-csi-drivers\"/\"aws-ebs-csi-driver-node-metrics-serving-cert\"" Jul 10 18:39:48.960665 ip-10-0-143-17 kubenswrapper[2585]: I0710 18:39:48.934079 2585 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ovn-kubernetes/ovnkube-node-xg6bm" Jul 10 18:39:48.960665 ip-10-0-143-17 kubenswrapper[2585]: I0710 18:39:48.934159 2585 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-cluster-csi-drivers\"/\"kube-root-ca.crt\"" Jul 10 18:39:48.960665 ip-10-0-143-17 kubenswrapper[2585]: I0710 18:39:48.934473 2585 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-cluster-csi-drivers\"/\"openshift-service-ca.crt\"" Jul 10 18:39:48.960665 ip-10-0-143-17 kubenswrapper[2585]: I0710 18:39:48.934562 2585 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-cluster-csi-drivers\"/\"aws-ebs-csi-driver-node-sa-dockercfg-vwjw7\"" Jul 10 18:39:48.960665 ip-10-0-143-17 kubenswrapper[2585]: I0710 18:39:48.935769 2585 certificate_manager.go:566] "Rotating certificates" logger="kubernetes.io/kubelet-serving" Jul 10 18:39:48.960665 ip-10-0-143-17 kubenswrapper[2585]: I0710 18:39:48.935881 2585 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-network-operator\"/\"openshift-service-ca.crt\"" Jul 10 18:39:48.960665 ip-10-0-143-17 kubenswrapper[2585]: I0710 18:39:48.935995 2585 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-network-operator\"/\"kube-root-ca.crt\"" Jul 10 18:39:48.960665 ip-10-0-143-17 kubenswrapper[2585]: I0710 18:39:48.936164 2585 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-network-operator\"/\"iptables-alerter-script\"" Jul 10 18:39:48.960665 ip-10-0-143-17 kubenswrapper[2585]: I0710 18:39:48.936297 2585 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-network-operator\"/\"iptables-alerter-dockercfg-frq6l\"" Jul 10 18:39:48.960665 ip-10-0-143-17 kubenswrapper[2585]: I0710 18:39:48.937402 2585 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-hhpnn" Jul 10 18:39:48.960665 ip-10-0-143-17 kubenswrapper[2585]: E0710 18:39:48.937476 2585 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-hhpnn" podUID="15a0ea9c-d294-4065-aed5-a182a76a7d7f" Jul 10 18:39:48.960665 ip-10-0-143-17 kubenswrapper[2585]: I0710 18:39:48.940101 2585 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-dns/node-resolver-ndwqb" Jul 10 18:39:48.960665 ip-10-0-143-17 kubenswrapper[2585]: I0710 18:39:48.940565 2585 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/multus-h4nv5" Jul 10 18:39:48.960665 ip-10-0-143-17 kubenswrapper[2585]: I0710 18:39:48.941116 2585 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-image-registry/node-ca-crbxd" Jul 10 18:39:48.960665 ip-10-0-143-17 kubenswrapper[2585]: I0710 18:39:48.943181 2585 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ovn-kubernetes\"/\"kube-root-ca.crt\"" Jul 10 18:39:48.960665 ip-10-0-143-17 kubenswrapper[2585]: I0710 18:39:48.943422 2585 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ovn-kubernetes\"/\"openshift-service-ca.crt\"" Jul 10 18:39:48.960665 ip-10-0-143-17 kubenswrapper[2585]: I0710 18:39:48.943617 2585 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ovn-kubernetes\"/\"env-overrides\"" Jul 10 18:39:48.962585 ip-10-0-143-17 kubenswrapper[2585]: I0710 18:39:48.943769 2585 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ovn-kubernetes\"/\"ovnkube-config\"" Jul 10 18:39:48.962585 ip-10-0-143-17 kubenswrapper[2585]: I0710 18:39:48.944110 2585 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ovn-kubernetes\"/\"ovn-node-metrics-cert\"" Jul 10 18:39:48.962585 ip-10-0-143-17 kubenswrapper[2585]: I0710 18:39:48.944281 2585 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ovn-kubernetes\"/\"ovn-kubernetes-node-dockercfg-9rffz\"" Jul 10 18:39:48.962585 ip-10-0-143-17 kubenswrapper[2585]: I0710 18:39:48.944656 2585 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-7rwwk" Jul 10 18:39:48.962585 ip-10-0-143-17 kubenswrapper[2585]: E0710 18:39:48.944721 2585 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-7rwwk" podUID="6516579e-a521-4350-bc6b-c4d698b2f845" Jul 10 18:39:48.962585 ip-10-0-143-17 kubenswrapper[2585]: I0710 18:39:48.945131 2585 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/multus-additional-cni-plugins-w4cxp" Jul 10 18:39:48.962585 ip-10-0-143-17 kubenswrapper[2585]: I0710 18:39:48.945343 2585 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-dns\"/\"node-resolver-dockercfg-gfc7j\"" Jul 10 18:39:48.962585 ip-10-0-143-17 kubenswrapper[2585]: I0710 18:39:48.945673 2585 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-multus\"/\"kube-root-ca.crt\"" Jul 10 18:39:48.962585 ip-10-0-143-17 kubenswrapper[2585]: I0710 18:39:48.946141 2585 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-7tsng" Jul 10 18:39:48.962585 ip-10-0-143-17 kubenswrapper[2585]: I0710 18:39:48.946167 2585 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-multus\"/\"multus-daemon-config\"" Jul 10 18:39:48.962585 ip-10-0-143-17 kubenswrapper[2585]: E0710 18:39:48.946193 2585 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-7tsng" podUID="2166f472-2dd7-47f0-a15e-72d2abf71118" Jul 10 18:39:48.962585 ip-10-0-143-17 kubenswrapper[2585]: I0710 18:39:48.946278 2585 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-dns\"/\"openshift-service-ca.crt\"" Jul 10 18:39:48.962585 ip-10-0-143-17 kubenswrapper[2585]: I0710 18:39:48.946277 2585 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-multus\"/\"default-dockercfg-ksgkx\"" Jul 10 18:39:48.962585 ip-10-0-143-17 kubenswrapper[2585]: I0710 18:39:48.946341 2585 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-multus\"/\"openshift-service-ca.crt\"" Jul 10 18:39:48.962585 ip-10-0-143-17 kubenswrapper[2585]: I0710 18:39:48.946816 2585 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-image-registry\"/\"openshift-service-ca.crt\"" Jul 10 18:39:48.962585 ip-10-0-143-17 kubenswrapper[2585]: I0710 18:39:48.947148 2585 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ovn-kubernetes\"/\"ovnkube-script-lib\"" Jul 10 18:39:48.962585 ip-10-0-143-17 kubenswrapper[2585]: I0710 18:39:48.947242 2585 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/konnectivity-agent-x7zf6" Jul 10 18:39:48.962585 ip-10-0-143-17 kubenswrapper[2585]: I0710 18:39:48.947438 2585 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-image-registry\"/\"image-registry-certificates\"" Jul 10 18:39:48.962585 ip-10-0-143-17 kubenswrapper[2585]: I0710 18:39:48.947660 2585 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-multus\"/\"cni-copy-resources\"" Jul 10 18:39:48.962585 ip-10-0-143-17 kubenswrapper[2585]: I0710 18:39:48.947745 2585 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-image-registry\"/\"kube-root-ca.crt\"" Jul 10 18:39:48.962585 ip-10-0-143-17 kubenswrapper[2585]: I0710 18:39:48.947827 2585 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-image-registry\"/\"node-ca-dockercfg-5jrmj\"" Jul 10 18:39:48.962585 ip-10-0-143-17 kubenswrapper[2585]: I0710 18:39:48.948091 2585 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-dns\"/\"kube-root-ca.crt\"" Jul 10 18:39:48.962585 ip-10-0-143-17 kubenswrapper[2585]: I0710 18:39:48.949630 2585 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-multus\"/\"whereabouts-flatfile-config\"" Jul 10 18:39:48.962585 ip-10-0-143-17 kubenswrapper[2585]: I0710 18:39:48.949778 2585 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"kube-system\"/\"konnectivity-ca-bundle\"" Jul 10 18:39:48.962585 ip-10-0-143-17 kubenswrapper[2585]: I0710 18:39:48.950038 2585 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-multus\"/\"default-cni-sysctl-allowlist\"" Jul 10 18:39:48.962585 ip-10-0-143-17 kubenswrapper[2585]: I0710 18:39:48.950425 2585 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"kube-system\"/\"konnectivity-agent\"" Jul 10 18:39:48.962585 ip-10-0-143-17 kubenswrapper[2585]: I0710 18:39:48.950436 2585 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-multus\"/\"multus-ancillary-tools-dockercfg-6xsv7\"" Jul 10 18:39:48.962585 ip-10-0-143-17 kubenswrapper[2585]: I0710 18:39:48.959505 2585 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"kube-system\"/\"default-dockercfg-k62x8\"" Jul 10 18:39:48.964514 ip-10-0-143-17 kubenswrapper[2585]: I0710 18:39:48.964191 2585 reflector.go:439] "Caches populated" logger="kubernetes.io/kubelet-serving" type="*v1.CertificateSigningRequest" reflector="k8s.io/client-go/tools/watch/informerwatcher.go:162" Jul 10 18:39:48.972862 ip-10-0-143-17 kubenswrapper[2585]: I0710 18:39:48.970908 2585 csr.go:274] "Certificate signing request is approved, waiting to be issued" logger="kubernetes.io/kubelet-serving" csr="csr-29nx8" Jul 10 18:39:49.048980 ip-10-0-143-17 kubenswrapper[2585]: I0710 18:39:48.979516 2585 csr.go:270] "Certificate signing request is issued" logger="kubernetes.io/kubelet-serving" csr="csr-29nx8" Jul 10 18:39:49.048980 ip-10-0-143-17 kubenswrapper[2585]: I0710 18:39:48.980894 2585 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"ovnkube-config\" (UniqueName: \"kubernetes.io/configmap/d863cfea-f2f3-4404-b62a-d41785525f3f-ovnkube-config\") pod \"ovnkube-node-xg6bm\" (UID: \"d863cfea-f2f3-4404-b62a-d41785525f3f\") " pod="openshift-ovn-kubernetes/ovnkube-node-xg6bm" Jul 10 18:39:49.048980 ip-10-0-143-17 kubenswrapper[2585]: I0710 18:39:48.980987 2585 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-rz4xs\" (UniqueName: \"kubernetes.io/projected/d863cfea-f2f3-4404-b62a-d41785525f3f-kube-api-access-rz4xs\") pod \"ovnkube-node-xg6bm\" (UID: \"d863cfea-f2f3-4404-b62a-d41785525f3f\") " pod="openshift-ovn-kubernetes/ovnkube-node-xg6bm" Jul 10 18:39:49.048980 ip-10-0-143-17 kubenswrapper[2585]: I0710 18:39:48.982953 2585 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"socket-dir\" (UniqueName: \"kubernetes.io/host-path/bd56ef17-098b-40ba-8500-8cb05e293dc5-socket-dir\") pod \"aws-ebs-csi-driver-node-cw6g2\" (UID: \"bd56ef17-098b-40ba-8500-8cb05e293dc5\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-cw6g2" Jul 10 18:39:49.048980 ip-10-0-143-17 kubenswrapper[2585]: I0710 18:39:48.983187 2585 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"etc-selinux\" (UniqueName: \"kubernetes.io/host-path/bd56ef17-098b-40ba-8500-8cb05e293dc5-etc-selinux\") pod \"aws-ebs-csi-driver-node-cw6g2\" (UID: \"bd56ef17-098b-40ba-8500-8cb05e293dc5\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-cw6g2" Jul 10 18:39:49.048980 ip-10-0-143-17 kubenswrapper[2585]: I0710 18:39:48.983211 2585 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-cb4h7\" (UniqueName: \"kubernetes.io/projected/bd56ef17-098b-40ba-8500-8cb05e293dc5-kube-api-access-cb4h7\") pod \"aws-ebs-csi-driver-node-cw6g2\" (UID: \"bd56ef17-098b-40ba-8500-8cb05e293dc5\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-cw6g2" Jul 10 18:39:49.048980 ip-10-0-143-17 kubenswrapper[2585]: I0710 18:39:48.983237 2585 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"iptables-alerter-script\" (UniqueName: \"kubernetes.io/configmap/03ab0623-6f5f-49c6-b999-6dc99fc53667-iptables-alerter-script\") pod \"iptables-alerter-m5446\" (UID: \"03ab0623-6f5f-49c6-b999-6dc99fc53667\") " pod="openshift-network-operator/iptables-alerter-m5446" Jul 10 18:39:49.048980 ip-10-0-143-17 kubenswrapper[2585]: I0710 18:39:48.983262 2585 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"var-lib-openvswitch\" (UniqueName: \"kubernetes.io/host-path/d863cfea-f2f3-4404-b62a-d41785525f3f-var-lib-openvswitch\") pod \"ovnkube-node-xg6bm\" (UID: \"d863cfea-f2f3-4404-b62a-d41785525f3f\") " pod="openshift-ovn-kubernetes/ovnkube-node-xg6bm" Jul 10 18:39:49.048980 ip-10-0-143-17 kubenswrapper[2585]: I0710 18:39:48.983285 2585 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"env-overrides\" (UniqueName: \"kubernetes.io/configmap/d863cfea-f2f3-4404-b62a-d41785525f3f-env-overrides\") pod \"ovnkube-node-xg6bm\" (UID: \"d863cfea-f2f3-4404-b62a-d41785525f3f\") " pod="openshift-ovn-kubernetes/ovnkube-node-xg6bm" Jul 10 18:39:49.048980 ip-10-0-143-17 kubenswrapper[2585]: I0710 18:39:48.983308 2585 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"etc-systemd\" (UniqueName: \"kubernetes.io/host-path/fdd15e8f-ae22-4144-b9bb-b4e4f09be270-etc-systemd\") pod \"tuned-w6ds9\" (UID: \"fdd15e8f-ae22-4144-b9bb-b4e4f09be270\") " pod="openshift-cluster-node-tuning-operator/tuned-w6ds9" Jul 10 18:39:49.048980 ip-10-0-143-17 kubenswrapper[2585]: I0710 18:39:48.983329 2585 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"sys\" (UniqueName: \"kubernetes.io/host-path/fdd15e8f-ae22-4144-b9bb-b4e4f09be270-sys\") pod \"tuned-w6ds9\" (UID: \"fdd15e8f-ae22-4144-b9bb-b4e4f09be270\") " pod="openshift-cluster-node-tuning-operator/tuned-w6ds9" Jul 10 18:39:49.048980 ip-10-0-143-17 kubenswrapper[2585]: I0710 18:39:48.983349 2585 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-zjxqn\" (UniqueName: \"kubernetes.io/projected/fdd15e8f-ae22-4144-b9bb-b4e4f09be270-kube-api-access-zjxqn\") pod \"tuned-w6ds9\" (UID: \"fdd15e8f-ae22-4144-b9bb-b4e4f09be270\") " pod="openshift-cluster-node-tuning-operator/tuned-w6ds9" Jul 10 18:39:49.048980 ip-10-0-143-17 kubenswrapper[2585]: I0710 18:39:48.983373 2585 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kubelet-dir\" (UniqueName: \"kubernetes.io/host-path/bd56ef17-098b-40ba-8500-8cb05e293dc5-kubelet-dir\") pod \"aws-ebs-csi-driver-node-cw6g2\" (UID: \"bd56ef17-098b-40ba-8500-8cb05e293dc5\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-cw6g2" Jul 10 18:39:49.048980 ip-10-0-143-17 kubenswrapper[2585]: I0710 18:39:48.983404 2585 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-slash\" (UniqueName: \"kubernetes.io/host-path/03ab0623-6f5f-49c6-b999-6dc99fc53667-host-slash\") pod \"iptables-alerter-m5446\" (UID: \"03ab0623-6f5f-49c6-b999-6dc99fc53667\") " pod="openshift-network-operator/iptables-alerter-m5446" Jul 10 18:39:49.159062 ip-10-0-143-17 kubenswrapper[2585]: I0710 18:39:48.983442 2585 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-run-netns\" (UniqueName: \"kubernetes.io/host-path/d863cfea-f2f3-4404-b62a-d41785525f3f-host-run-netns\") pod \"ovnkube-node-xg6bm\" (UID: \"d863cfea-f2f3-4404-b62a-d41785525f3f\") " pod="openshift-ovn-kubernetes/ovnkube-node-xg6bm" Jul 10 18:39:49.159062 ip-10-0-143-17 kubenswrapper[2585]: I0710 18:39:48.983477 2585 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"etc-openvswitch\" (UniqueName: \"kubernetes.io/host-path/d863cfea-f2f3-4404-b62a-d41785525f3f-etc-openvswitch\") pod \"ovnkube-node-xg6bm\" (UID: \"d863cfea-f2f3-4404-b62a-d41785525f3f\") " pod="openshift-ovn-kubernetes/ovnkube-node-xg6bm" Jul 10 18:39:49.159062 ip-10-0-143-17 kubenswrapper[2585]: I0710 18:39:48.983502 2585 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"run-ovn\" (UniqueName: \"kubernetes.io/host-path/d863cfea-f2f3-4404-b62a-d41785525f3f-run-ovn\") pod \"ovnkube-node-xg6bm\" (UID: \"d863cfea-f2f3-4404-b62a-d41785525f3f\") " pod="openshift-ovn-kubernetes/ovnkube-node-xg6bm" Jul 10 18:39:49.159062 ip-10-0-143-17 kubenswrapper[2585]: I0710 18:39:48.983525 2585 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"etc-tuned\" (UniqueName: \"kubernetes.io/empty-dir/fdd15e8f-ae22-4144-b9bb-b4e4f09be270-etc-tuned\") pod \"tuned-w6ds9\" (UID: \"fdd15e8f-ae22-4144-b9bb-b4e4f09be270\") " pod="openshift-cluster-node-tuning-operator/tuned-w6ds9" Jul 10 18:39:49.159062 ip-10-0-143-17 kubenswrapper[2585]: I0710 18:39:48.983562 2585 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"node-log\" (UniqueName: \"kubernetes.io/host-path/d863cfea-f2f3-4404-b62a-d41785525f3f-node-log\") pod \"ovnkube-node-xg6bm\" (UID: \"d863cfea-f2f3-4404-b62a-d41785525f3f\") " pod="openshift-ovn-kubernetes/ovnkube-node-xg6bm" Jul 10 18:39:49.159062 ip-10-0-143-17 kubenswrapper[2585]: I0710 18:39:48.983594 2585 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"dbus\" (UniqueName: \"kubernetes.io/host-path/15a0ea9c-d294-4065-aed5-a182a76a7d7f-dbus\") pod \"global-pull-secret-syncer-hhpnn\" (UID: \"15a0ea9c-d294-4065-aed5-a182a76a7d7f\") " pod="kube-system/global-pull-secret-syncer-hhpnn" Jul 10 18:39:49.159062 ip-10-0-143-17 kubenswrapper[2585]: I0710 18:39:48.983618 2585 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"original-pull-secret\" (UniqueName: \"kubernetes.io/secret/15a0ea9c-d294-4065-aed5-a182a76a7d7f-original-pull-secret\") pod \"global-pull-secret-syncer-hhpnn\" (UID: \"15a0ea9c-d294-4065-aed5-a182a76a7d7f\") " pod="kube-system/global-pull-secret-syncer-hhpnn" Jul 10 18:39:49.159062 ip-10-0-143-17 kubenswrapper[2585]: I0710 18:39:48.983704 2585 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"run-systemd\" (UniqueName: \"kubernetes.io/host-path/d863cfea-f2f3-4404-b62a-d41785525f3f-run-systemd\") pod \"ovnkube-node-xg6bm\" (UID: \"d863cfea-f2f3-4404-b62a-d41785525f3f\") " pod="openshift-ovn-kubernetes/ovnkube-node-xg6bm" Jul 10 18:39:49.159062 ip-10-0-143-17 kubenswrapper[2585]: I0710 18:39:48.983737 2585 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"etc-modprobe-d\" (UniqueName: \"kubernetes.io/host-path/fdd15e8f-ae22-4144-b9bb-b4e4f09be270-etc-modprobe-d\") pod \"tuned-w6ds9\" (UID: \"fdd15e8f-ae22-4144-b9bb-b4e4f09be270\") " pod="openshift-cluster-node-tuning-operator/tuned-w6ds9" Jul 10 18:39:49.159062 ip-10-0-143-17 kubenswrapper[2585]: I0710 18:39:48.983755 2585 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host\" (UniqueName: \"kubernetes.io/host-path/fdd15e8f-ae22-4144-b9bb-b4e4f09be270-host\") pod \"tuned-w6ds9\" (UID: \"fdd15e8f-ae22-4144-b9bb-b4e4f09be270\") " pod="openshift-cluster-node-tuning-operator/tuned-w6ds9" Jul 10 18:39:49.159062 ip-10-0-143-17 kubenswrapper[2585]: I0710 18:39:48.983772 2585 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"systemd-units\" (UniqueName: \"kubernetes.io/host-path/d863cfea-f2f3-4404-b62a-d41785525f3f-systemd-units\") pod \"ovnkube-node-xg6bm\" (UID: \"d863cfea-f2f3-4404-b62a-d41785525f3f\") " pod="openshift-ovn-kubernetes/ovnkube-node-xg6bm" Jul 10 18:39:49.159062 ip-10-0-143-17 kubenswrapper[2585]: I0710 18:39:48.983788 2585 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"run-openvswitch\" (UniqueName: \"kubernetes.io/host-path/d863cfea-f2f3-4404-b62a-d41785525f3f-run-openvswitch\") pod \"ovnkube-node-xg6bm\" (UID: \"d863cfea-f2f3-4404-b62a-d41785525f3f\") " pod="openshift-ovn-kubernetes/ovnkube-node-xg6bm" Jul 10 18:39:49.159062 ip-10-0-143-17 kubenswrapper[2585]: I0710 18:39:48.983811 2585 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-var-lib-cni-networks-ovn-kubernetes\" (UniqueName: \"kubernetes.io/host-path/d863cfea-f2f3-4404-b62a-d41785525f3f-host-var-lib-cni-networks-ovn-kubernetes\") pod \"ovnkube-node-xg6bm\" (UID: \"d863cfea-f2f3-4404-b62a-d41785525f3f\") " pod="openshift-ovn-kubernetes/ovnkube-node-xg6bm" Jul 10 18:39:49.159062 ip-10-0-143-17 kubenswrapper[2585]: I0710 18:39:48.983843 2585 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"ovn-node-metrics-cert\" (UniqueName: \"kubernetes.io/secret/d863cfea-f2f3-4404-b62a-d41785525f3f-ovn-node-metrics-cert\") pod \"ovnkube-node-xg6bm\" (UID: \"d863cfea-f2f3-4404-b62a-d41785525f3f\") " pod="openshift-ovn-kubernetes/ovnkube-node-xg6bm" Jul 10 18:39:49.160277 ip-10-0-143-17 kubenswrapper[2585]: I0710 18:39:48.983889 2585 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-n5dbk\" (UniqueName: \"kubernetes.io/projected/03ab0623-6f5f-49c6-b999-6dc99fc53667-kube-api-access-n5dbk\") pod \"iptables-alerter-m5446\" (UID: \"03ab0623-6f5f-49c6-b999-6dc99fc53667\") " pod="openshift-network-operator/iptables-alerter-m5446" Jul 10 18:39:49.160277 ip-10-0-143-17 kubenswrapper[2585]: I0710 18:39:48.983938 2585 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"etc-sysconfig\" (UniqueName: \"kubernetes.io/host-path/fdd15e8f-ae22-4144-b9bb-b4e4f09be270-etc-sysconfig\") pod \"tuned-w6ds9\" (UID: \"fdd15e8f-ae22-4144-b9bb-b4e4f09be270\") " pod="openshift-cluster-node-tuning-operator/tuned-w6ds9" Jul 10 18:39:49.160277 ip-10-0-143-17 kubenswrapper[2585]: I0710 18:39:48.983967 2585 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"run\" (UniqueName: \"kubernetes.io/host-path/fdd15e8f-ae22-4144-b9bb-b4e4f09be270-run\") pod \"tuned-w6ds9\" (UID: \"fdd15e8f-ae22-4144-b9bb-b4e4f09be270\") " pod="openshift-cluster-node-tuning-operator/tuned-w6ds9" Jul 10 18:39:49.160277 ip-10-0-143-17 kubenswrapper[2585]: I0710 18:39:48.983990 2585 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"lib-modules\" (UniqueName: \"kubernetes.io/host-path/fdd15e8f-ae22-4144-b9bb-b4e4f09be270-lib-modules\") pod \"tuned-w6ds9\" (UID: \"fdd15e8f-ae22-4144-b9bb-b4e4f09be270\") " pod="openshift-cluster-node-tuning-operator/tuned-w6ds9" Jul 10 18:39:49.160277 ip-10-0-143-17 kubenswrapper[2585]: I0710 18:39:48.984018 2585 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-cni-netd\" (UniqueName: \"kubernetes.io/host-path/d863cfea-f2f3-4404-b62a-d41785525f3f-host-cni-netd\") pod \"ovnkube-node-xg6bm\" (UID: \"d863cfea-f2f3-4404-b62a-d41785525f3f\") " pod="openshift-ovn-kubernetes/ovnkube-node-xg6bm" Jul 10 18:39:49.160277 ip-10-0-143-17 kubenswrapper[2585]: I0710 18:39:48.984057 2585 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"ovnkube-script-lib\" (UniqueName: \"kubernetes.io/configmap/d863cfea-f2f3-4404-b62a-d41785525f3f-ovnkube-script-lib\") pod \"ovnkube-node-xg6bm\" (UID: \"d863cfea-f2f3-4404-b62a-d41785525f3f\") " pod="openshift-ovn-kubernetes/ovnkube-node-xg6bm" Jul 10 18:39:49.160277 ip-10-0-143-17 kubenswrapper[2585]: I0710 18:39:48.984112 2585 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kubelet-config\" (UniqueName: \"kubernetes.io/host-path/15a0ea9c-d294-4065-aed5-a182a76a7d7f-kubelet-config\") pod \"global-pull-secret-syncer-hhpnn\" (UID: \"15a0ea9c-d294-4065-aed5-a182a76a7d7f\") " pod="kube-system/global-pull-secret-syncer-hhpnn" Jul 10 18:39:49.160277 ip-10-0-143-17 kubenswrapper[2585]: I0710 18:39:48.984174 2585 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"etc-kubernetes\" (UniqueName: \"kubernetes.io/host-path/fdd15e8f-ae22-4144-b9bb-b4e4f09be270-etc-kubernetes\") pod \"tuned-w6ds9\" (UID: \"fdd15e8f-ae22-4144-b9bb-b4e4f09be270\") " pod="openshift-cluster-node-tuning-operator/tuned-w6ds9" Jul 10 18:39:49.160277 ip-10-0-143-17 kubenswrapper[2585]: I0710 18:39:48.984220 2585 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"etc-sysctl-d\" (UniqueName: \"kubernetes.io/host-path/fdd15e8f-ae22-4144-b9bb-b4e4f09be270-etc-sysctl-d\") pod \"tuned-w6ds9\" (UID: \"fdd15e8f-ae22-4144-b9bb-b4e4f09be270\") " pod="openshift-cluster-node-tuning-operator/tuned-w6ds9" Jul 10 18:39:49.160277 ip-10-0-143-17 kubenswrapper[2585]: I0710 18:39:48.984242 2585 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"etc-sysctl-conf\" (UniqueName: \"kubernetes.io/host-path/fdd15e8f-ae22-4144-b9bb-b4e4f09be270-etc-sysctl-conf\") pod \"tuned-w6ds9\" (UID: \"fdd15e8f-ae22-4144-b9bb-b4e4f09be270\") " pod="openshift-cluster-node-tuning-operator/tuned-w6ds9" Jul 10 18:39:49.160277 ip-10-0-143-17 kubenswrapper[2585]: I0710 18:39:48.984296 2585 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tmp\" (UniqueName: \"kubernetes.io/empty-dir/fdd15e8f-ae22-4144-b9bb-b4e4f09be270-tmp\") pod \"tuned-w6ds9\" (UID: \"fdd15e8f-ae22-4144-b9bb-b4e4f09be270\") " pod="openshift-cluster-node-tuning-operator/tuned-w6ds9" Jul 10 18:39:49.160277 ip-10-0-143-17 kubenswrapper[2585]: I0710 18:39:48.984320 2585 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-kubelet\" (UniqueName: \"kubernetes.io/host-path/d863cfea-f2f3-4404-b62a-d41785525f3f-host-kubelet\") pod \"ovnkube-node-xg6bm\" (UID: \"d863cfea-f2f3-4404-b62a-d41785525f3f\") " pod="openshift-ovn-kubernetes/ovnkube-node-xg6bm" Jul 10 18:39:49.160277 ip-10-0-143-17 kubenswrapper[2585]: I0710 18:39:48.984336 2585 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-slash\" (UniqueName: \"kubernetes.io/host-path/d863cfea-f2f3-4404-b62a-d41785525f3f-host-slash\") pod \"ovnkube-node-xg6bm\" (UID: \"d863cfea-f2f3-4404-b62a-d41785525f3f\") " pod="openshift-ovn-kubernetes/ovnkube-node-xg6bm" Jul 10 18:39:49.160277 ip-10-0-143-17 kubenswrapper[2585]: I0710 18:39:48.984364 2585 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"log-socket\" (UniqueName: \"kubernetes.io/host-path/d863cfea-f2f3-4404-b62a-d41785525f3f-log-socket\") pod \"ovnkube-node-xg6bm\" (UID: \"d863cfea-f2f3-4404-b62a-d41785525f3f\") " pod="openshift-ovn-kubernetes/ovnkube-node-xg6bm" Jul 10 18:39:49.160908 ip-10-0-143-17 kubenswrapper[2585]: I0710 18:39:48.984395 2585 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-run-ovn-kubernetes\" (UniqueName: \"kubernetes.io/host-path/d863cfea-f2f3-4404-b62a-d41785525f3f-host-run-ovn-kubernetes\") pod \"ovnkube-node-xg6bm\" (UID: \"d863cfea-f2f3-4404-b62a-d41785525f3f\") " pod="openshift-ovn-kubernetes/ovnkube-node-xg6bm" Jul 10 18:39:49.160908 ip-10-0-143-17 kubenswrapper[2585]: I0710 18:39:48.984426 2585 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"var-lib-kubelet\" (UniqueName: \"kubernetes.io/host-path/fdd15e8f-ae22-4144-b9bb-b4e4f09be270-var-lib-kubelet\") pod \"tuned-w6ds9\" (UID: \"fdd15e8f-ae22-4144-b9bb-b4e4f09be270\") " pod="openshift-cluster-node-tuning-operator/tuned-w6ds9" Jul 10 18:39:49.160908 ip-10-0-143-17 kubenswrapper[2585]: I0710 18:39:48.984463 2585 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"registration-dir\" (UniqueName: \"kubernetes.io/host-path/bd56ef17-098b-40ba-8500-8cb05e293dc5-registration-dir\") pod \"aws-ebs-csi-driver-node-cw6g2\" (UID: \"bd56ef17-098b-40ba-8500-8cb05e293dc5\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-cw6g2" Jul 10 18:39:49.160908 ip-10-0-143-17 kubenswrapper[2585]: I0710 18:39:48.984477 2585 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"device-dir\" (UniqueName: \"kubernetes.io/host-path/bd56ef17-098b-40ba-8500-8cb05e293dc5-device-dir\") pod \"aws-ebs-csi-driver-node-cw6g2\" (UID: \"bd56ef17-098b-40ba-8500-8cb05e293dc5\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-cw6g2" Jul 10 18:39:49.160908 ip-10-0-143-17 kubenswrapper[2585]: I0710 18:39:48.984492 2585 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"sys-fs\" (UniqueName: \"kubernetes.io/host-path/bd56ef17-098b-40ba-8500-8cb05e293dc5-sys-fs\") pod \"aws-ebs-csi-driver-node-cw6g2\" (UID: \"bd56ef17-098b-40ba-8500-8cb05e293dc5\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-cw6g2" Jul 10 18:39:49.160908 ip-10-0-143-17 kubenswrapper[2585]: I0710 18:39:48.984512 2585 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-cni-bin\" (UniqueName: \"kubernetes.io/host-path/d863cfea-f2f3-4404-b62a-d41785525f3f-host-cni-bin\") pod \"ovnkube-node-xg6bm\" (UID: \"d863cfea-f2f3-4404-b62a-d41785525f3f\") " pod="openshift-ovn-kubernetes/ovnkube-node-xg6bm" Jul 10 18:39:49.160908 ip-10-0-143-17 kubenswrapper[2585]: I0710 18:39:49.037489 2585 desired_state_of_world_populator.go:154] "Finished populating initial desired state of world" Jul 10 18:39:49.160908 ip-10-0-143-17 kubenswrapper[2585]: I0710 18:39:49.037923 2585 reflector.go:439] "Caches populated" type="*v1.RuntimeClass" reflector="k8s.io/client-go/informers/factory.go:160" Jul 10 18:39:49.160908 ip-10-0-143-17 kubenswrapper[2585]: I0710 18:39:49.084713 2585 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-run-netns\" (UniqueName: \"kubernetes.io/host-path/d0da579f-0565-4704-942a-402e5bf46135-host-run-netns\") pod \"multus-h4nv5\" (UID: \"d0da579f-0565-4704-942a-402e5bf46135\") " pod="openshift-multus/multus-h4nv5" Jul 10 18:39:49.160908 ip-10-0-143-17 kubenswrapper[2585]: I0710 18:39:49.084781 2585 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"serviceca\" (UniqueName: \"kubernetes.io/configmap/ad44fb01-da7b-40ac-906e-5a7badc87f54-serviceca\") pod \"node-ca-crbxd\" (UID: \"ad44fb01-da7b-40ac-906e-5a7badc87f54\") " pod="openshift-image-registry/node-ca-crbxd" Jul 10 18:39:49.160908 ip-10-0-143-17 kubenswrapper[2585]: I0710 18:39:49.084814 2585 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"original-pull-secret\" (UniqueName: \"kubernetes.io/secret/15a0ea9c-d294-4065-aed5-a182a76a7d7f-original-pull-secret\") pod \"global-pull-secret-syncer-hhpnn\" (UID: \"15a0ea9c-d294-4065-aed5-a182a76a7d7f\") " pod="kube-system/global-pull-secret-syncer-hhpnn" Jul 10 18:39:49.160908 ip-10-0-143-17 kubenswrapper[2585]: I0710 18:39:49.084838 2585 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"etc-kubernetes\" (UniqueName: \"kubernetes.io/host-path/d0da579f-0565-4704-942a-402e5bf46135-etc-kubernetes\") pod \"multus-h4nv5\" (UID: \"d0da579f-0565-4704-942a-402e5bf46135\") " pod="openshift-multus/multus-h4nv5" Jul 10 18:39:49.160908 ip-10-0-143-17 kubenswrapper[2585]: I0710 18:39:49.084864 2585 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"os-release\" (UniqueName: \"kubernetes.io/host-path/c3be1d6d-b0aa-4613-9d9a-e308693cc790-os-release\") pod \"multus-additional-cni-plugins-w4cxp\" (UID: \"c3be1d6d-b0aa-4613-9d9a-e308693cc790\") " pod="openshift-multus/multus-additional-cni-plugins-w4cxp" Jul 10 18:39:49.160908 ip-10-0-143-17 kubenswrapper[2585]: I0710 18:39:49.084888 2585 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-rxhcr\" (UniqueName: \"kubernetes.io/projected/c3be1d6d-b0aa-4613-9d9a-e308693cc790-kube-api-access-rxhcr\") pod \"multus-additional-cni-plugins-w4cxp\" (UID: \"c3be1d6d-b0aa-4613-9d9a-e308693cc790\") " pod="openshift-multus/multus-additional-cni-plugins-w4cxp" Jul 10 18:39:49.160908 ip-10-0-143-17 kubenswrapper[2585]: E0710 18:39:49.084965 2585 secret.go:189] Couldn't get secret kube-system/original-pull-secret: object "kube-system"/"original-pull-secret" not registered Jul 10 18:39:49.162366 ip-10-0-143-17 kubenswrapper[2585]: I0710 18:39:49.084993 2585 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-82qmj\" (UniqueName: \"kubernetes.io/projected/2166f472-2dd7-47f0-a15e-72d2abf71118-kube-api-access-82qmj\") pod \"network-check-target-7tsng\" (UID: \"2166f472-2dd7-47f0-a15e-72d2abf71118\") " pod="openshift-network-diagnostics/network-check-target-7tsng" Jul 10 18:39:49.162366 ip-10-0-143-17 kubenswrapper[2585]: E0710 18:39:49.085064 2585 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/15a0ea9c-d294-4065-aed5-a182a76a7d7f-original-pull-secret podName:15a0ea9c-d294-4065-aed5-a182a76a7d7f nodeName:}" failed. No retries permitted until 2026-07-10 18:39:49.585023928 +0000 UTC m=+3.042383854 (durationBeforeRetry 500ms). Error: MountVolume.SetUp failed for volume "original-pull-secret" (UniqueName: "kubernetes.io/secret/15a0ea9c-d294-4065-aed5-a182a76a7d7f-original-pull-secret") pod "global-pull-secret-syncer-hhpnn" (UID: "15a0ea9c-d294-4065-aed5-a182a76a7d7f") : object "kube-system"/"original-pull-secret" not registered Jul 10 18:39:49.162366 ip-10-0-143-17 kubenswrapper[2585]: I0710 18:39:49.085085 2585 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"run-systemd\" (UniqueName: \"kubernetes.io/host-path/d863cfea-f2f3-4404-b62a-d41785525f3f-run-systemd\") pod \"ovnkube-node-xg6bm\" (UID: \"d863cfea-f2f3-4404-b62a-d41785525f3f\") " pod="openshift-ovn-kubernetes/ovnkube-node-xg6bm" Jul 10 18:39:49.162366 ip-10-0-143-17 kubenswrapper[2585]: I0710 18:39:49.085112 2585 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"system-cni-dir\" (UniqueName: \"kubernetes.io/host-path/d0da579f-0565-4704-942a-402e5bf46135-system-cni-dir\") pod \"multus-h4nv5\" (UID: \"d0da579f-0565-4704-942a-402e5bf46135\") " pod="openshift-multus/multus-h4nv5" Jul 10 18:39:49.162366 ip-10-0-143-17 kubenswrapper[2585]: I0710 18:39:49.085135 2585 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-4rrz6\" (UniqueName: \"kubernetes.io/projected/ad44fb01-da7b-40ac-906e-5a7badc87f54-kube-api-access-4rrz6\") pod \"node-ca-crbxd\" (UID: \"ad44fb01-da7b-40ac-906e-5a7badc87f54\") " pod="openshift-image-registry/node-ca-crbxd" Jul 10 18:39:49.162366 ip-10-0-143-17 kubenswrapper[2585]: I0710 18:39:49.085162 2585 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tuning-conf-dir\" (UniqueName: \"kubernetes.io/host-path/c3be1d6d-b0aa-4613-9d9a-e308693cc790-tuning-conf-dir\") pod \"multus-additional-cni-plugins-w4cxp\" (UID: \"c3be1d6d-b0aa-4613-9d9a-e308693cc790\") " pod="openshift-multus/multus-additional-cni-plugins-w4cxp" Jul 10 18:39:49.162366 ip-10-0-143-17 kubenswrapper[2585]: I0710 18:39:49.085172 2585 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"run-systemd\" (UniqueName: \"kubernetes.io/host-path/d863cfea-f2f3-4404-b62a-d41785525f3f-run-systemd\") pod \"ovnkube-node-xg6bm\" (UID: \"d863cfea-f2f3-4404-b62a-d41785525f3f\") " pod="openshift-ovn-kubernetes/ovnkube-node-xg6bm" Jul 10 18:39:49.162366 ip-10-0-143-17 kubenswrapper[2585]: I0710 18:39:49.085193 2585 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"cnibin\" (UniqueName: \"kubernetes.io/host-path/d0da579f-0565-4704-942a-402e5bf46135-cnibin\") pod \"multus-h4nv5\" (UID: \"d0da579f-0565-4704-942a-402e5bf46135\") " pod="openshift-multus/multus-h4nv5" Jul 10 18:39:49.162366 ip-10-0-143-17 kubenswrapper[2585]: I0710 18:39:49.085229 2585 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-var-lib-cni-networks-ovn-kubernetes\" (UniqueName: \"kubernetes.io/host-path/d863cfea-f2f3-4404-b62a-d41785525f3f-host-var-lib-cni-networks-ovn-kubernetes\") pod \"ovnkube-node-xg6bm\" (UID: \"d863cfea-f2f3-4404-b62a-d41785525f3f\") " pod="openshift-ovn-kubernetes/ovnkube-node-xg6bm" Jul 10 18:39:49.162366 ip-10-0-143-17 kubenswrapper[2585]: I0710 18:39:49.085259 2585 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"ovn-node-metrics-cert\" (UniqueName: \"kubernetes.io/secret/d863cfea-f2f3-4404-b62a-d41785525f3f-ovn-node-metrics-cert\") pod \"ovnkube-node-xg6bm\" (UID: \"d863cfea-f2f3-4404-b62a-d41785525f3f\") " pod="openshift-ovn-kubernetes/ovnkube-node-xg6bm" Jul 10 18:39:49.162366 ip-10-0-143-17 kubenswrapper[2585]: I0710 18:39:49.085282 2585 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/5a594168-14da-41cb-9871-edbbf08720f7-tmp-dir\") pod \"node-resolver-ndwqb\" (UID: \"5a594168-14da-41cb-9871-edbbf08720f7\") " pod="openshift-dns/node-resolver-ndwqb" Jul 10 18:39:49.162366 ip-10-0-143-17 kubenswrapper[2585]: I0710 18:39:49.085307 2585 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"etc-sysconfig\" (UniqueName: \"kubernetes.io/host-path/fdd15e8f-ae22-4144-b9bb-b4e4f09be270-etc-sysconfig\") pod \"tuned-w6ds9\" (UID: \"fdd15e8f-ae22-4144-b9bb-b4e4f09be270\") " pod="openshift-cluster-node-tuning-operator/tuned-w6ds9" Jul 10 18:39:49.162366 ip-10-0-143-17 kubenswrapper[2585]: I0710 18:39:49.085308 2585 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"host-var-lib-cni-networks-ovn-kubernetes\" (UniqueName: \"kubernetes.io/host-path/d863cfea-f2f3-4404-b62a-d41785525f3f-host-var-lib-cni-networks-ovn-kubernetes\") pod \"ovnkube-node-xg6bm\" (UID: \"d863cfea-f2f3-4404-b62a-d41785525f3f\") " pod="openshift-ovn-kubernetes/ovnkube-node-xg6bm" Jul 10 18:39:49.162929 ip-10-0-143-17 kubenswrapper[2585]: I0710 18:39:49.085328 2585 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"run\" (UniqueName: \"kubernetes.io/host-path/fdd15e8f-ae22-4144-b9bb-b4e4f09be270-run\") pod \"tuned-w6ds9\" (UID: \"fdd15e8f-ae22-4144-b9bb-b4e4f09be270\") " pod="openshift-cluster-node-tuning-operator/tuned-w6ds9" Jul 10 18:39:49.162929 ip-10-0-143-17 kubenswrapper[2585]: I0710 18:39:49.085351 2585 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"os-release\" (UniqueName: \"kubernetes.io/host-path/d0da579f-0565-4704-942a-402e5bf46135-os-release\") pod \"multus-h4nv5\" (UID: \"d0da579f-0565-4704-942a-402e5bf46135\") " pod="openshift-multus/multus-h4nv5" Jul 10 18:39:49.162929 ip-10-0-143-17 kubenswrapper[2585]: I0710 18:39:49.085370 2585 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"hostroot\" (UniqueName: \"kubernetes.io/host-path/d0da579f-0565-4704-942a-402e5bf46135-hostroot\") pod \"multus-h4nv5\" (UID: \"d0da579f-0565-4704-942a-402e5bf46135\") " pod="openshift-multus/multus-h4nv5" Jul 10 18:39:49.162929 ip-10-0-143-17 kubenswrapper[2585]: I0710 18:39:49.085376 2585 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"etc-sysconfig\" (UniqueName: \"kubernetes.io/host-path/fdd15e8f-ae22-4144-b9bb-b4e4f09be270-etc-sysconfig\") pod \"tuned-w6ds9\" (UID: \"fdd15e8f-ae22-4144-b9bb-b4e4f09be270\") " pod="openshift-cluster-node-tuning-operator/tuned-w6ds9" Jul 10 18:39:49.162929 ip-10-0-143-17 kubenswrapper[2585]: I0710 18:39:49.085391 2585 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-bzp9h\" (UniqueName: \"kubernetes.io/projected/d0da579f-0565-4704-942a-402e5bf46135-kube-api-access-bzp9h\") pod \"multus-h4nv5\" (UID: \"d0da579f-0565-4704-942a-402e5bf46135\") " pod="openshift-multus/multus-h4nv5" Jul 10 18:39:49.162929 ip-10-0-143-17 kubenswrapper[2585]: I0710 18:39:49.085430 2585 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"whereabouts-flatfile-configmap\" (UniqueName: \"kubernetes.io/configmap/c3be1d6d-b0aa-4613-9d9a-e308693cc790-whereabouts-flatfile-configmap\") pod \"multus-additional-cni-plugins-w4cxp\" (UID: \"c3be1d6d-b0aa-4613-9d9a-e308693cc790\") " pod="openshift-multus/multus-additional-cni-plugins-w4cxp" Jul 10 18:39:49.162929 ip-10-0-143-17 kubenswrapper[2585]: I0710 18:39:49.085452 2585 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tmp\" (UniqueName: \"kubernetes.io/empty-dir/fdd15e8f-ae22-4144-b9bb-b4e4f09be270-tmp\") pod \"tuned-w6ds9\" (UID: \"fdd15e8f-ae22-4144-b9bb-b4e4f09be270\") " pod="openshift-cluster-node-tuning-operator/tuned-w6ds9" Jul 10 18:39:49.162929 ip-10-0-143-17 kubenswrapper[2585]: I0710 18:39:49.085474 2585 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-kubelet\" (UniqueName: \"kubernetes.io/host-path/d863cfea-f2f3-4404-b62a-d41785525f3f-host-kubelet\") pod \"ovnkube-node-xg6bm\" (UID: \"d863cfea-f2f3-4404-b62a-d41785525f3f\") " pod="openshift-ovn-kubernetes/ovnkube-node-xg6bm" Jul 10 18:39:49.162929 ip-10-0-143-17 kubenswrapper[2585]: I0710 18:39:49.085474 2585 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"run\" (UniqueName: \"kubernetes.io/host-path/fdd15e8f-ae22-4144-b9bb-b4e4f09be270-run\") pod \"tuned-w6ds9\" (UID: \"fdd15e8f-ae22-4144-b9bb-b4e4f09be270\") " pod="openshift-cluster-node-tuning-operator/tuned-w6ds9" Jul 10 18:39:49.162929 ip-10-0-143-17 kubenswrapper[2585]: I0710 18:39:49.085509 2585 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-slash\" (UniqueName: \"kubernetes.io/host-path/d863cfea-f2f3-4404-b62a-d41785525f3f-host-slash\") pod \"ovnkube-node-xg6bm\" (UID: \"d863cfea-f2f3-4404-b62a-d41785525f3f\") " pod="openshift-ovn-kubernetes/ovnkube-node-xg6bm" Jul 10 18:39:49.162929 ip-10-0-143-17 kubenswrapper[2585]: I0710 18:39:49.085515 2585 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"host-kubelet\" (UniqueName: \"kubernetes.io/host-path/d863cfea-f2f3-4404-b62a-d41785525f3f-host-kubelet\") pod \"ovnkube-node-xg6bm\" (UID: \"d863cfea-f2f3-4404-b62a-d41785525f3f\") " pod="openshift-ovn-kubernetes/ovnkube-node-xg6bm" Jul 10 18:39:49.162929 ip-10-0-143-17 kubenswrapper[2585]: I0710 18:39:49.085555 2585 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"log-socket\" (UniqueName: \"kubernetes.io/host-path/d863cfea-f2f3-4404-b62a-d41785525f3f-log-socket\") pod \"ovnkube-node-xg6bm\" (UID: \"d863cfea-f2f3-4404-b62a-d41785525f3f\") " pod="openshift-ovn-kubernetes/ovnkube-node-xg6bm" Jul 10 18:39:49.162929 ip-10-0-143-17 kubenswrapper[2585]: I0710 18:39:49.085571 2585 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"host-slash\" (UniqueName: \"kubernetes.io/host-path/d863cfea-f2f3-4404-b62a-d41785525f3f-host-slash\") pod \"ovnkube-node-xg6bm\" (UID: \"d863cfea-f2f3-4404-b62a-d41785525f3f\") " pod="openshift-ovn-kubernetes/ovnkube-node-xg6bm" Jul 10 18:39:49.162929 ip-10-0-143-17 kubenswrapper[2585]: I0710 18:39:49.085582 2585 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-run-ovn-kubernetes\" (UniqueName: \"kubernetes.io/host-path/d863cfea-f2f3-4404-b62a-d41785525f3f-host-run-ovn-kubernetes\") pod \"ovnkube-node-xg6bm\" (UID: \"d863cfea-f2f3-4404-b62a-d41785525f3f\") " pod="openshift-ovn-kubernetes/ovnkube-node-xg6bm" Jul 10 18:39:49.163769 ip-10-0-143-17 kubenswrapper[2585]: I0710 18:39:49.085612 2585 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"cni-sysctl-allowlist\" (UniqueName: \"kubernetes.io/configmap/c3be1d6d-b0aa-4613-9d9a-e308693cc790-cni-sysctl-allowlist\") pod \"multus-additional-cni-plugins-w4cxp\" (UID: \"c3be1d6d-b0aa-4613-9d9a-e308693cc790\") " pod="openshift-multus/multus-additional-cni-plugins-w4cxp" Jul 10 18:39:49.163769 ip-10-0-143-17 kubenswrapper[2585]: I0710 18:39:49.085614 2585 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"host-run-ovn-kubernetes\" (UniqueName: \"kubernetes.io/host-path/d863cfea-f2f3-4404-b62a-d41785525f3f-host-run-ovn-kubernetes\") pod \"ovnkube-node-xg6bm\" (UID: \"d863cfea-f2f3-4404-b62a-d41785525f3f\") " pod="openshift-ovn-kubernetes/ovnkube-node-xg6bm" Jul 10 18:39:49.163769 ip-10-0-143-17 kubenswrapper[2585]: I0710 18:39:49.085639 2585 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"registration-dir\" (UniqueName: \"kubernetes.io/host-path/bd56ef17-098b-40ba-8500-8cb05e293dc5-registration-dir\") pod \"aws-ebs-csi-driver-node-cw6g2\" (UID: \"bd56ef17-098b-40ba-8500-8cb05e293dc5\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-cw6g2" Jul 10 18:39:49.163769 ip-10-0-143-17 kubenswrapper[2585]: I0710 18:39:49.085663 2585 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"device-dir\" (UniqueName: \"kubernetes.io/host-path/bd56ef17-098b-40ba-8500-8cb05e293dc5-device-dir\") pod \"aws-ebs-csi-driver-node-cw6g2\" (UID: \"bd56ef17-098b-40ba-8500-8cb05e293dc5\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-cw6g2" Jul 10 18:39:49.163769 ip-10-0-143-17 kubenswrapper[2585]: I0710 18:39:49.085635 2585 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"log-socket\" (UniqueName: \"kubernetes.io/host-path/d863cfea-f2f3-4404-b62a-d41785525f3f-log-socket\") pod \"ovnkube-node-xg6bm\" (UID: \"d863cfea-f2f3-4404-b62a-d41785525f3f\") " pod="openshift-ovn-kubernetes/ovnkube-node-xg6bm" Jul 10 18:39:49.163769 ip-10-0-143-17 kubenswrapper[2585]: I0710 18:39:49.085686 2585 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"sys-fs\" (UniqueName: \"kubernetes.io/host-path/bd56ef17-098b-40ba-8500-8cb05e293dc5-sys-fs\") pod \"aws-ebs-csi-driver-node-cw6g2\" (UID: \"bd56ef17-098b-40ba-8500-8cb05e293dc5\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-cw6g2" Jul 10 18:39:49.163769 ip-10-0-143-17 kubenswrapper[2585]: I0710 18:39:49.085706 2585 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"registration-dir\" (UniqueName: \"kubernetes.io/host-path/bd56ef17-098b-40ba-8500-8cb05e293dc5-registration-dir\") pod \"aws-ebs-csi-driver-node-cw6g2\" (UID: \"bd56ef17-098b-40ba-8500-8cb05e293dc5\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-cw6g2" Jul 10 18:39:49.163769 ip-10-0-143-17 kubenswrapper[2585]: I0710 18:39:49.085724 2585 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-cni-bin\" (UniqueName: \"kubernetes.io/host-path/d863cfea-f2f3-4404-b62a-d41785525f3f-host-cni-bin\") pod \"ovnkube-node-xg6bm\" (UID: \"d863cfea-f2f3-4404-b62a-d41785525f3f\") " pod="openshift-ovn-kubernetes/ovnkube-node-xg6bm" Jul 10 18:39:49.163769 ip-10-0-143-17 kubenswrapper[2585]: I0710 18:39:49.085737 2585 swap_util.go:74] "error creating dir to test if tmpfs noswap is enabled. Assuming not supported" mount path="" error="stat /var/lib/kubelet/plugins/kubernetes.io/empty-dir: no such file or directory" Jul 10 18:39:49.163769 ip-10-0-143-17 kubenswrapper[2585]: I0710 18:39:49.085749 2585 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"multus-cni-dir\" (UniqueName: \"kubernetes.io/host-path/d0da579f-0565-4704-942a-402e5bf46135-multus-cni-dir\") pod \"multus-h4nv5\" (UID: \"d0da579f-0565-4704-942a-402e5bf46135\") " pod="openshift-multus/multus-h4nv5" Jul 10 18:39:49.163769 ip-10-0-143-17 kubenswrapper[2585]: I0710 18:39:49.085771 2585 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"sys-fs\" (UniqueName: \"kubernetes.io/host-path/bd56ef17-098b-40ba-8500-8cb05e293dc5-sys-fs\") pod \"aws-ebs-csi-driver-node-cw6g2\" (UID: \"bd56ef17-098b-40ba-8500-8cb05e293dc5\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-cw6g2" Jul 10 18:39:49.163769 ip-10-0-143-17 kubenswrapper[2585]: I0710 18:39:49.085776 2585 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"system-cni-dir\" (UniqueName: \"kubernetes.io/host-path/c3be1d6d-b0aa-4613-9d9a-e308693cc790-system-cni-dir\") pod \"multus-additional-cni-plugins-w4cxp\" (UID: \"c3be1d6d-b0aa-4613-9d9a-e308693cc790\") " pod="openshift-multus/multus-additional-cni-plugins-w4cxp" Jul 10 18:39:49.163769 ip-10-0-143-17 kubenswrapper[2585]: I0710 18:39:49.085741 2585 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"device-dir\" (UniqueName: \"kubernetes.io/host-path/bd56ef17-098b-40ba-8500-8cb05e293dc5-device-dir\") pod \"aws-ebs-csi-driver-node-cw6g2\" (UID: \"bd56ef17-098b-40ba-8500-8cb05e293dc5\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-cw6g2" Jul 10 18:39:49.163769 ip-10-0-143-17 kubenswrapper[2585]: I0710 18:39:49.085779 2585 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"host-cni-bin\" (UniqueName: \"kubernetes.io/host-path/d863cfea-f2f3-4404-b62a-d41785525f3f-host-cni-bin\") pod \"ovnkube-node-xg6bm\" (UID: \"d863cfea-f2f3-4404-b62a-d41785525f3f\") " pod="openshift-ovn-kubernetes/ovnkube-node-xg6bm" Jul 10 18:39:49.164368 ip-10-0-143-17 kubenswrapper[2585]: I0710 18:39:49.085802 2585 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"cnibin\" (UniqueName: \"kubernetes.io/host-path/c3be1d6d-b0aa-4613-9d9a-e308693cc790-cnibin\") pod \"multus-additional-cni-plugins-w4cxp\" (UID: \"c3be1d6d-b0aa-4613-9d9a-e308693cc790\") " pod="openshift-multus/multus-additional-cni-plugins-w4cxp" Jul 10 18:39:49.164368 ip-10-0-143-17 kubenswrapper[2585]: I0710 18:39:49.085828 2585 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"etc-selinux\" (UniqueName: \"kubernetes.io/host-path/bd56ef17-098b-40ba-8500-8cb05e293dc5-etc-selinux\") pod \"aws-ebs-csi-driver-node-cw6g2\" (UID: \"bd56ef17-098b-40ba-8500-8cb05e293dc5\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-cw6g2" Jul 10 18:39:49.164368 ip-10-0-143-17 kubenswrapper[2585]: I0710 18:39:49.085889 2585 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-cb4h7\" (UniqueName: \"kubernetes.io/projected/bd56ef17-098b-40ba-8500-8cb05e293dc5-kube-api-access-cb4h7\") pod \"aws-ebs-csi-driver-node-cw6g2\" (UID: \"bd56ef17-098b-40ba-8500-8cb05e293dc5\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-cw6g2" Jul 10 18:39:49.164368 ip-10-0-143-17 kubenswrapper[2585]: I0710 18:39:49.085913 2585 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"dbus\" (UniqueName: \"kubernetes.io/host-path/15a0ea9c-d294-4065-aed5-a182a76a7d7f-dbus\") pod \"global-pull-secret-syncer-hhpnn\" (UID: \"15a0ea9c-d294-4065-aed5-a182a76a7d7f\") " pod="kube-system/global-pull-secret-syncer-hhpnn" Jul 10 18:39:49.164368 ip-10-0-143-17 kubenswrapper[2585]: I0710 18:39:49.085946 2585 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-fwnk6\" (UniqueName: \"kubernetes.io/projected/5a594168-14da-41cb-9871-edbbf08720f7-kube-api-access-fwnk6\") pod \"node-resolver-ndwqb\" (UID: \"5a594168-14da-41cb-9871-edbbf08720f7\") " pod="openshift-dns/node-resolver-ndwqb" Jul 10 18:39:49.164368 ip-10-0-143-17 kubenswrapper[2585]: I0710 18:39:49.085975 2585 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"etc-systemd\" (UniqueName: \"kubernetes.io/host-path/fdd15e8f-ae22-4144-b9bb-b4e4f09be270-etc-systemd\") pod \"tuned-w6ds9\" (UID: \"fdd15e8f-ae22-4144-b9bb-b4e4f09be270\") " pod="openshift-cluster-node-tuning-operator/tuned-w6ds9" Jul 10 18:39:49.164368 ip-10-0-143-17 kubenswrapper[2585]: I0710 18:39:49.085981 2585 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"etc-selinux\" (UniqueName: \"kubernetes.io/host-path/bd56ef17-098b-40ba-8500-8cb05e293dc5-etc-selinux\") pod \"aws-ebs-csi-driver-node-cw6g2\" (UID: \"bd56ef17-098b-40ba-8500-8cb05e293dc5\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-cw6g2" Jul 10 18:39:49.164368 ip-10-0-143-17 kubenswrapper[2585]: I0710 18:39:49.085999 2585 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-slash\" (UniqueName: \"kubernetes.io/host-path/03ab0623-6f5f-49c6-b999-6dc99fc53667-host-slash\") pod \"iptables-alerter-m5446\" (UID: \"03ab0623-6f5f-49c6-b999-6dc99fc53667\") " pod="openshift-network-operator/iptables-alerter-m5446" Jul 10 18:39:49.164368 ip-10-0-143-17 kubenswrapper[2585]: I0710 18:39:49.086045 2585 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"host-slash\" (UniqueName: \"kubernetes.io/host-path/03ab0623-6f5f-49c6-b999-6dc99fc53667-host-slash\") pod \"iptables-alerter-m5446\" (UID: \"03ab0623-6f5f-49c6-b999-6dc99fc53667\") " pod="openshift-network-operator/iptables-alerter-m5446" Jul 10 18:39:49.164368 ip-10-0-143-17 kubenswrapper[2585]: I0710 18:39:49.086096 2585 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"etc-systemd\" (UniqueName: \"kubernetes.io/host-path/fdd15e8f-ae22-4144-b9bb-b4e4f09be270-etc-systemd\") pod \"tuned-w6ds9\" (UID: \"fdd15e8f-ae22-4144-b9bb-b4e4f09be270\") " pod="openshift-cluster-node-tuning-operator/tuned-w6ds9" Jul 10 18:39:49.164368 ip-10-0-143-17 kubenswrapper[2585]: I0710 18:39:49.086106 2585 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"dbus\" (UniqueName: \"kubernetes.io/host-path/15a0ea9c-d294-4065-aed5-a182a76a7d7f-dbus\") pod \"global-pull-secret-syncer-hhpnn\" (UID: \"15a0ea9c-d294-4065-aed5-a182a76a7d7f\") " pod="kube-system/global-pull-secret-syncer-hhpnn" Jul 10 18:39:49.164368 ip-10-0-143-17 kubenswrapper[2585]: I0710 18:39:49.086164 2585 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"etc-openvswitch\" (UniqueName: \"kubernetes.io/host-path/d863cfea-f2f3-4404-b62a-d41785525f3f-etc-openvswitch\") pod \"ovnkube-node-xg6bm\" (UID: \"d863cfea-f2f3-4404-b62a-d41785525f3f\") " pod="openshift-ovn-kubernetes/ovnkube-node-xg6bm" Jul 10 18:39:49.164368 ip-10-0-143-17 kubenswrapper[2585]: I0710 18:39:49.086215 2585 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-var-lib-cni-multus\" (UniqueName: \"kubernetes.io/host-path/d0da579f-0565-4704-942a-402e5bf46135-host-var-lib-cni-multus\") pod \"multus-h4nv5\" (UID: \"d0da579f-0565-4704-942a-402e5bf46135\") " pod="openshift-multus/multus-h4nv5" Jul 10 18:39:49.164368 ip-10-0-143-17 kubenswrapper[2585]: I0710 18:39:49.086241 2585 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"etc-tuned\" (UniqueName: \"kubernetes.io/empty-dir/fdd15e8f-ae22-4144-b9bb-b4e4f09be270-etc-tuned\") pod \"tuned-w6ds9\" (UID: \"fdd15e8f-ae22-4144-b9bb-b4e4f09be270\") " pod="openshift-cluster-node-tuning-operator/tuned-w6ds9" Jul 10 18:39:49.165306 ip-10-0-143-17 kubenswrapper[2585]: I0710 18:39:49.086249 2585 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"etc-openvswitch\" (UniqueName: \"kubernetes.io/host-path/d863cfea-f2f3-4404-b62a-d41785525f3f-etc-openvswitch\") pod \"ovnkube-node-xg6bm\" (UID: \"d863cfea-f2f3-4404-b62a-d41785525f3f\") " pod="openshift-ovn-kubernetes/ovnkube-node-xg6bm" Jul 10 18:39:49.165306 ip-10-0-143-17 kubenswrapper[2585]: I0710 18:39:49.086264 2585 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"hosts-file\" (UniqueName: \"kubernetes.io/host-path/5a594168-14da-41cb-9871-edbbf08720f7-hosts-file\") pod \"node-resolver-ndwqb\" (UID: \"5a594168-14da-41cb-9871-edbbf08720f7\") " pod="openshift-dns/node-resolver-ndwqb" Jul 10 18:39:49.165306 ip-10-0-143-17 kubenswrapper[2585]: I0710 18:39:49.086305 2585 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-run-k8s-cni-cncf-io\" (UniqueName: \"kubernetes.io/host-path/d0da579f-0565-4704-942a-402e5bf46135-host-run-k8s-cni-cncf-io\") pod \"multus-h4nv5\" (UID: \"d0da579f-0565-4704-942a-402e5bf46135\") " pod="openshift-multus/multus-h4nv5" Jul 10 18:39:49.165306 ip-10-0-143-17 kubenswrapper[2585]: I0710 18:39:49.086331 2585 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"cni-binary-copy\" (UniqueName: \"kubernetes.io/configmap/d0da579f-0565-4704-942a-402e5bf46135-cni-binary-copy\") pod \"multus-h4nv5\" (UID: \"d0da579f-0565-4704-942a-402e5bf46135\") " pod="openshift-multus/multus-h4nv5" Jul 10 18:39:49.165306 ip-10-0-143-17 kubenswrapper[2585]: I0710 18:39:49.086352 2585 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host\" (UniqueName: \"kubernetes.io/host-path/ad44fb01-da7b-40ac-906e-5a7badc87f54-host\") pod \"node-ca-crbxd\" (UID: \"ad44fb01-da7b-40ac-906e-5a7badc87f54\") " pod="openshift-image-registry/node-ca-crbxd" Jul 10 18:39:49.165306 ip-10-0-143-17 kubenswrapper[2585]: I0710 18:39:49.086417 2585 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"etc-modprobe-d\" (UniqueName: \"kubernetes.io/host-path/fdd15e8f-ae22-4144-b9bb-b4e4f09be270-etc-modprobe-d\") pod \"tuned-w6ds9\" (UID: \"fdd15e8f-ae22-4144-b9bb-b4e4f09be270\") " pod="openshift-cluster-node-tuning-operator/tuned-w6ds9" Jul 10 18:39:49.165306 ip-10-0-143-17 kubenswrapper[2585]: I0710 18:39:49.086470 2585 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host\" (UniqueName: \"kubernetes.io/host-path/fdd15e8f-ae22-4144-b9bb-b4e4f09be270-host\") pod \"tuned-w6ds9\" (UID: \"fdd15e8f-ae22-4144-b9bb-b4e4f09be270\") " pod="openshift-cluster-node-tuning-operator/tuned-w6ds9" Jul 10 18:39:49.165306 ip-10-0-143-17 kubenswrapper[2585]: I0710 18:39:49.086495 2585 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"systemd-units\" (UniqueName: \"kubernetes.io/host-path/d863cfea-f2f3-4404-b62a-d41785525f3f-systemd-units\") pod \"ovnkube-node-xg6bm\" (UID: \"d863cfea-f2f3-4404-b62a-d41785525f3f\") " pod="openshift-ovn-kubernetes/ovnkube-node-xg6bm" Jul 10 18:39:49.165306 ip-10-0-143-17 kubenswrapper[2585]: I0710 18:39:49.086517 2585 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"run-openvswitch\" (UniqueName: \"kubernetes.io/host-path/d863cfea-f2f3-4404-b62a-d41785525f3f-run-openvswitch\") pod \"ovnkube-node-xg6bm\" (UID: \"d863cfea-f2f3-4404-b62a-d41785525f3f\") " pod="openshift-ovn-kubernetes/ovnkube-node-xg6bm" Jul 10 18:39:49.165306 ip-10-0-143-17 kubenswrapper[2585]: I0710 18:39:49.086590 2585 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"ovnkube-script-lib\" (UniqueName: \"kubernetes.io/configmap/d863cfea-f2f3-4404-b62a-d41785525f3f-ovnkube-script-lib\") pod \"ovnkube-node-xg6bm\" (UID: \"d863cfea-f2f3-4404-b62a-d41785525f3f\") " pod="openshift-ovn-kubernetes/ovnkube-node-xg6bm" Jul 10 18:39:49.165306 ip-10-0-143-17 kubenswrapper[2585]: I0710 18:39:49.086607 2585 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"etc-modprobe-d\" (UniqueName: \"kubernetes.io/host-path/fdd15e8f-ae22-4144-b9bb-b4e4f09be270-etc-modprobe-d\") pod \"tuned-w6ds9\" (UID: \"fdd15e8f-ae22-4144-b9bb-b4e4f09be270\") " pod="openshift-cluster-node-tuning-operator/tuned-w6ds9" Jul 10 18:39:49.165306 ip-10-0-143-17 kubenswrapper[2585]: I0710 18:39:49.086617 2585 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kubelet-config\" (UniqueName: \"kubernetes.io/host-path/15a0ea9c-d294-4065-aed5-a182a76a7d7f-kubelet-config\") pod \"global-pull-secret-syncer-hhpnn\" (UID: \"15a0ea9c-d294-4065-aed5-a182a76a7d7f\") " pod="kube-system/global-pull-secret-syncer-hhpnn" Jul 10 18:39:49.165306 ip-10-0-143-17 kubenswrapper[2585]: I0710 18:39:49.086668 2585 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"run-openvswitch\" (UniqueName: \"kubernetes.io/host-path/d863cfea-f2f3-4404-b62a-d41785525f3f-run-openvswitch\") pod \"ovnkube-node-xg6bm\" (UID: \"d863cfea-f2f3-4404-b62a-d41785525f3f\") " pod="openshift-ovn-kubernetes/ovnkube-node-xg6bm" Jul 10 18:39:49.165306 ip-10-0-143-17 kubenswrapper[2585]: I0710 18:39:49.086661 2585 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"host\" (UniqueName: \"kubernetes.io/host-path/fdd15e8f-ae22-4144-b9bb-b4e4f09be270-host\") pod \"tuned-w6ds9\" (UID: \"fdd15e8f-ae22-4144-b9bb-b4e4f09be270\") " pod="openshift-cluster-node-tuning-operator/tuned-w6ds9" Jul 10 18:39:49.166199 ip-10-0-143-17 kubenswrapper[2585]: I0710 18:39:49.086700 2585 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-n5dbk\" (UniqueName: \"kubernetes.io/projected/03ab0623-6f5f-49c6-b999-6dc99fc53667-kube-api-access-n5dbk\") pod \"iptables-alerter-m5446\" (UID: \"03ab0623-6f5f-49c6-b999-6dc99fc53667\") " pod="openshift-network-operator/iptables-alerter-m5446" Jul 10 18:39:49.166199 ip-10-0-143-17 kubenswrapper[2585]: I0710 18:39:49.086702 2585 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kubelet-config\" (UniqueName: \"kubernetes.io/host-path/15a0ea9c-d294-4065-aed5-a182a76a7d7f-kubelet-config\") pod \"global-pull-secret-syncer-hhpnn\" (UID: \"15a0ea9c-d294-4065-aed5-a182a76a7d7f\") " pod="kube-system/global-pull-secret-syncer-hhpnn" Jul 10 18:39:49.166199 ip-10-0-143-17 kubenswrapper[2585]: I0710 18:39:49.086794 2585 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"lib-modules\" (UniqueName: \"kubernetes.io/host-path/fdd15e8f-ae22-4144-b9bb-b4e4f09be270-lib-modules\") pod \"tuned-w6ds9\" (UID: \"fdd15e8f-ae22-4144-b9bb-b4e4f09be270\") " pod="openshift-cluster-node-tuning-operator/tuned-w6ds9" Jul 10 18:39:49.166199 ip-10-0-143-17 kubenswrapper[2585]: I0710 18:39:49.086914 2585 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"systemd-units\" (UniqueName: \"kubernetes.io/host-path/d863cfea-f2f3-4404-b62a-d41785525f3f-systemd-units\") pod \"ovnkube-node-xg6bm\" (UID: \"d863cfea-f2f3-4404-b62a-d41785525f3f\") " pod="openshift-ovn-kubernetes/ovnkube-node-xg6bm" Jul 10 18:39:49.166199 ip-10-0-143-17 kubenswrapper[2585]: I0710 18:39:49.086955 2585 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-cni-netd\" (UniqueName: \"kubernetes.io/host-path/d863cfea-f2f3-4404-b62a-d41785525f3f-host-cni-netd\") pod \"ovnkube-node-xg6bm\" (UID: \"d863cfea-f2f3-4404-b62a-d41785525f3f\") " pod="openshift-ovn-kubernetes/ovnkube-node-xg6bm" Jul 10 18:39:49.166199 ip-10-0-143-17 kubenswrapper[2585]: I0710 18:39:49.087043 2585 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"lib-modules\" (UniqueName: \"kubernetes.io/host-path/fdd15e8f-ae22-4144-b9bb-b4e4f09be270-lib-modules\") pod \"tuned-w6ds9\" (UID: \"fdd15e8f-ae22-4144-b9bb-b4e4f09be270\") " pod="openshift-cluster-node-tuning-operator/tuned-w6ds9" Jul 10 18:39:49.166199 ip-10-0-143-17 kubenswrapper[2585]: I0710 18:39:49.087033 2585 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-run-multus-certs\" (UniqueName: \"kubernetes.io/host-path/d0da579f-0565-4704-942a-402e5bf46135-host-run-multus-certs\") pod \"multus-h4nv5\" (UID: \"d0da579f-0565-4704-942a-402e5bf46135\") " pod="openshift-multus/multus-h4nv5" Jul 10 18:39:49.166199 ip-10-0-143-17 kubenswrapper[2585]: I0710 18:39:49.087089 2585 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/6516579e-a521-4350-bc6b-c4d698b2f845-metrics-certs\") pod \"network-metrics-daemon-7rwwk\" (UID: \"6516579e-a521-4350-bc6b-c4d698b2f845\") " pod="openshift-multus/network-metrics-daemon-7rwwk" Jul 10 18:39:49.166199 ip-10-0-143-17 kubenswrapper[2585]: I0710 18:39:49.087056 2585 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"host-cni-netd\" (UniqueName: \"kubernetes.io/host-path/d863cfea-f2f3-4404-b62a-d41785525f3f-host-cni-netd\") pod \"ovnkube-node-xg6bm\" (UID: \"d863cfea-f2f3-4404-b62a-d41785525f3f\") " pod="openshift-ovn-kubernetes/ovnkube-node-xg6bm" Jul 10 18:39:49.166199 ip-10-0-143-17 kubenswrapper[2585]: I0710 18:39:49.087147 2585 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-74lfk\" (UniqueName: \"kubernetes.io/projected/6516579e-a521-4350-bc6b-c4d698b2f845-kube-api-access-74lfk\") pod \"network-metrics-daemon-7rwwk\" (UID: \"6516579e-a521-4350-bc6b-c4d698b2f845\") " pod="openshift-multus/network-metrics-daemon-7rwwk" Jul 10 18:39:49.166199 ip-10-0-143-17 kubenswrapper[2585]: I0710 18:39:49.087217 2585 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"agent-certs\" (UniqueName: \"kubernetes.io/secret/11f2f17b-9176-4b2b-b4a4-836025deed92-agent-certs\") pod \"konnectivity-agent-x7zf6\" (UID: \"11f2f17b-9176-4b2b-b4a4-836025deed92\") " pod="kube-system/konnectivity-agent-x7zf6" Jul 10 18:39:49.166199 ip-10-0-143-17 kubenswrapper[2585]: I0710 18:39:49.087268 2585 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"etc-kubernetes\" (UniqueName: \"kubernetes.io/host-path/fdd15e8f-ae22-4144-b9bb-b4e4f09be270-etc-kubernetes\") pod \"tuned-w6ds9\" (UID: \"fdd15e8f-ae22-4144-b9bb-b4e4f09be270\") " pod="openshift-cluster-node-tuning-operator/tuned-w6ds9" Jul 10 18:39:49.166199 ip-10-0-143-17 kubenswrapper[2585]: I0710 18:39:49.087356 2585 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"etc-sysctl-d\" (UniqueName: \"kubernetes.io/host-path/fdd15e8f-ae22-4144-b9bb-b4e4f09be270-etc-sysctl-d\") pod \"tuned-w6ds9\" (UID: \"fdd15e8f-ae22-4144-b9bb-b4e4f09be270\") " pod="openshift-cluster-node-tuning-operator/tuned-w6ds9" Jul 10 18:39:49.166199 ip-10-0-143-17 kubenswrapper[2585]: I0710 18:39:49.087379 2585 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"etc-sysctl-conf\" (UniqueName: \"kubernetes.io/host-path/fdd15e8f-ae22-4144-b9bb-b4e4f09be270-etc-sysctl-conf\") pod \"tuned-w6ds9\" (UID: \"fdd15e8f-ae22-4144-b9bb-b4e4f09be270\") " pod="openshift-cluster-node-tuning-operator/tuned-w6ds9" Jul 10 18:39:49.166678 ip-10-0-143-17 kubenswrapper[2585]: I0710 18:39:49.087402 2585 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"ovnkube-config\" (UniqueName: \"kubernetes.io/configmap/d863cfea-f2f3-4404-b62a-d41785525f3f-ovnkube-config\") pod \"ovnkube-node-xg6bm\" (UID: \"d863cfea-f2f3-4404-b62a-d41785525f3f\") " pod="openshift-ovn-kubernetes/ovnkube-node-xg6bm" Jul 10 18:39:49.166678 ip-10-0-143-17 kubenswrapper[2585]: I0710 18:39:49.087454 2585 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"var-lib-kubelet\" (UniqueName: \"kubernetes.io/host-path/fdd15e8f-ae22-4144-b9bb-b4e4f09be270-var-lib-kubelet\") pod \"tuned-w6ds9\" (UID: \"fdd15e8f-ae22-4144-b9bb-b4e4f09be270\") " pod="openshift-cluster-node-tuning-operator/tuned-w6ds9" Jul 10 18:39:49.166678 ip-10-0-143-17 kubenswrapper[2585]: I0710 18:39:49.087455 2585 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"etc-sysctl-d\" (UniqueName: \"kubernetes.io/host-path/fdd15e8f-ae22-4144-b9bb-b4e4f09be270-etc-sysctl-d\") pod \"tuned-w6ds9\" (UID: \"fdd15e8f-ae22-4144-b9bb-b4e4f09be270\") " pod="openshift-cluster-node-tuning-operator/tuned-w6ds9" Jul 10 18:39:49.166678 ip-10-0-143-17 kubenswrapper[2585]: I0710 18:39:49.087448 2585 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"etc-kubernetes\" (UniqueName: \"kubernetes.io/host-path/fdd15e8f-ae22-4144-b9bb-b4e4f09be270-etc-kubernetes\") pod \"tuned-w6ds9\" (UID: \"fdd15e8f-ae22-4144-b9bb-b4e4f09be270\") " pod="openshift-cluster-node-tuning-operator/tuned-w6ds9" Jul 10 18:39:49.166678 ip-10-0-143-17 kubenswrapper[2585]: I0710 18:39:49.087482 2585 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-rz4xs\" (UniqueName: \"kubernetes.io/projected/d863cfea-f2f3-4404-b62a-d41785525f3f-kube-api-access-rz4xs\") pod \"ovnkube-node-xg6bm\" (UID: \"d863cfea-f2f3-4404-b62a-d41785525f3f\") " pod="openshift-ovn-kubernetes/ovnkube-node-xg6bm" Jul 10 18:39:49.166678 ip-10-0-143-17 kubenswrapper[2585]: I0710 18:39:49.087507 2585 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"ovnkube-script-lib\" (UniqueName: \"kubernetes.io/configmap/d863cfea-f2f3-4404-b62a-d41785525f3f-ovnkube-script-lib\") pod \"ovnkube-node-xg6bm\" (UID: \"d863cfea-f2f3-4404-b62a-d41785525f3f\") " pod="openshift-ovn-kubernetes/ovnkube-node-xg6bm" Jul 10 18:39:49.166678 ip-10-0-143-17 kubenswrapper[2585]: I0710 18:39:49.087523 2585 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"etc-sysctl-conf\" (UniqueName: \"kubernetes.io/host-path/fdd15e8f-ae22-4144-b9bb-b4e4f09be270-etc-sysctl-conf\") pod \"tuned-w6ds9\" (UID: \"fdd15e8f-ae22-4144-b9bb-b4e4f09be270\") " pod="openshift-cluster-node-tuning-operator/tuned-w6ds9" Jul 10 18:39:49.166678 ip-10-0-143-17 kubenswrapper[2585]: I0710 18:39:49.087572 2585 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"var-lib-kubelet\" (UniqueName: \"kubernetes.io/host-path/fdd15e8f-ae22-4144-b9bb-b4e4f09be270-var-lib-kubelet\") pod \"tuned-w6ds9\" (UID: \"fdd15e8f-ae22-4144-b9bb-b4e4f09be270\") " pod="openshift-cluster-node-tuning-operator/tuned-w6ds9" Jul 10 18:39:49.166678 ip-10-0-143-17 kubenswrapper[2585]: I0710 18:39:49.087531 2585 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-var-lib-kubelet\" (UniqueName: \"kubernetes.io/host-path/d0da579f-0565-4704-942a-402e5bf46135-host-var-lib-kubelet\") pod \"multus-h4nv5\" (UID: \"d0da579f-0565-4704-942a-402e5bf46135\") " pod="openshift-multus/multus-h4nv5" Jul 10 18:39:49.166678 ip-10-0-143-17 kubenswrapper[2585]: I0710 18:39:49.087633 2585 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"cni-binary-copy\" (UniqueName: \"kubernetes.io/configmap/c3be1d6d-b0aa-4613-9d9a-e308693cc790-cni-binary-copy\") pod \"multus-additional-cni-plugins-w4cxp\" (UID: \"c3be1d6d-b0aa-4613-9d9a-e308693cc790\") " pod="openshift-multus/multus-additional-cni-plugins-w4cxp" Jul 10 18:39:49.166678 ip-10-0-143-17 kubenswrapper[2585]: I0710 18:39:49.087666 2585 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"konnectivity-ca\" (UniqueName: \"kubernetes.io/configmap/11f2f17b-9176-4b2b-b4a4-836025deed92-konnectivity-ca\") pod \"konnectivity-agent-x7zf6\" (UID: \"11f2f17b-9176-4b2b-b4a4-836025deed92\") " pod="kube-system/konnectivity-agent-x7zf6" Jul 10 18:39:49.166678 ip-10-0-143-17 kubenswrapper[2585]: I0710 18:39:49.087695 2585 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"socket-dir\" (UniqueName: \"kubernetes.io/host-path/bd56ef17-098b-40ba-8500-8cb05e293dc5-socket-dir\") pod \"aws-ebs-csi-driver-node-cw6g2\" (UID: \"bd56ef17-098b-40ba-8500-8cb05e293dc5\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-cw6g2" Jul 10 18:39:49.166678 ip-10-0-143-17 kubenswrapper[2585]: I0710 18:39:49.087725 2585 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"iptables-alerter-script\" (UniqueName: \"kubernetes.io/configmap/03ab0623-6f5f-49c6-b999-6dc99fc53667-iptables-alerter-script\") pod \"iptables-alerter-m5446\" (UID: \"03ab0623-6f5f-49c6-b999-6dc99fc53667\") " pod="openshift-network-operator/iptables-alerter-m5446" Jul 10 18:39:49.166678 ip-10-0-143-17 kubenswrapper[2585]: I0710 18:39:49.087766 2585 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"var-lib-openvswitch\" (UniqueName: \"kubernetes.io/host-path/d863cfea-f2f3-4404-b62a-d41785525f3f-var-lib-openvswitch\") pod \"ovnkube-node-xg6bm\" (UID: \"d863cfea-f2f3-4404-b62a-d41785525f3f\") " pod="openshift-ovn-kubernetes/ovnkube-node-xg6bm" Jul 10 18:39:49.167103 ip-10-0-143-17 kubenswrapper[2585]: I0710 18:39:49.087794 2585 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"env-overrides\" (UniqueName: \"kubernetes.io/configmap/d863cfea-f2f3-4404-b62a-d41785525f3f-env-overrides\") pod \"ovnkube-node-xg6bm\" (UID: \"d863cfea-f2f3-4404-b62a-d41785525f3f\") " pod="openshift-ovn-kubernetes/ovnkube-node-xg6bm" Jul 10 18:39:49.167103 ip-10-0-143-17 kubenswrapper[2585]: I0710 18:39:49.087819 2585 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"multus-socket-dir-parent\" (UniqueName: \"kubernetes.io/host-path/d0da579f-0565-4704-942a-402e5bf46135-multus-socket-dir-parent\") pod \"multus-h4nv5\" (UID: \"d0da579f-0565-4704-942a-402e5bf46135\") " pod="openshift-multus/multus-h4nv5" Jul 10 18:39:49.167103 ip-10-0-143-17 kubenswrapper[2585]: I0710 18:39:49.087844 2585 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"socket-dir\" (UniqueName: \"kubernetes.io/host-path/bd56ef17-098b-40ba-8500-8cb05e293dc5-socket-dir\") pod \"aws-ebs-csi-driver-node-cw6g2\" (UID: \"bd56ef17-098b-40ba-8500-8cb05e293dc5\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-cw6g2" Jul 10 18:39:49.167103 ip-10-0-143-17 kubenswrapper[2585]: I0710 18:39:49.087857 2585 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-var-lib-cni-bin\" (UniqueName: \"kubernetes.io/host-path/d0da579f-0565-4704-942a-402e5bf46135-host-var-lib-cni-bin\") pod \"multus-h4nv5\" (UID: \"d0da579f-0565-4704-942a-402e5bf46135\") " pod="openshift-multus/multus-h4nv5" Jul 10 18:39:49.167103 ip-10-0-143-17 kubenswrapper[2585]: I0710 18:39:49.087855 2585 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"var-lib-openvswitch\" (UniqueName: \"kubernetes.io/host-path/d863cfea-f2f3-4404-b62a-d41785525f3f-var-lib-openvswitch\") pod \"ovnkube-node-xg6bm\" (UID: \"d863cfea-f2f3-4404-b62a-d41785525f3f\") " pod="openshift-ovn-kubernetes/ovnkube-node-xg6bm" Jul 10 18:39:49.167103 ip-10-0-143-17 kubenswrapper[2585]: I0710 18:39:49.087884 2585 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"multus-conf-dir\" (UniqueName: \"kubernetes.io/host-path/d0da579f-0565-4704-942a-402e5bf46135-multus-conf-dir\") pod \"multus-h4nv5\" (UID: \"d0da579f-0565-4704-942a-402e5bf46135\") " pod="openshift-multus/multus-h4nv5" Jul 10 18:39:49.167103 ip-10-0-143-17 kubenswrapper[2585]: I0710 18:39:49.087942 2585 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"multus-daemon-config\" (UniqueName: \"kubernetes.io/configmap/d0da579f-0565-4704-942a-402e5bf46135-multus-daemon-config\") pod \"multus-h4nv5\" (UID: \"d0da579f-0565-4704-942a-402e5bf46135\") " pod="openshift-multus/multus-h4nv5" Jul 10 18:39:49.167103 ip-10-0-143-17 kubenswrapper[2585]: I0710 18:39:49.087990 2585 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"node-log\" (UniqueName: \"kubernetes.io/host-path/d863cfea-f2f3-4404-b62a-d41785525f3f-node-log\") pod \"ovnkube-node-xg6bm\" (UID: \"d863cfea-f2f3-4404-b62a-d41785525f3f\") " pod="openshift-ovn-kubernetes/ovnkube-node-xg6bm" Jul 10 18:39:49.167103 ip-10-0-143-17 kubenswrapper[2585]: I0710 18:39:49.088027 2585 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"ovnkube-config\" (UniqueName: \"kubernetes.io/configmap/d863cfea-f2f3-4404-b62a-d41785525f3f-ovnkube-config\") pod \"ovnkube-node-xg6bm\" (UID: \"d863cfea-f2f3-4404-b62a-d41785525f3f\") " pod="openshift-ovn-kubernetes/ovnkube-node-xg6bm" Jul 10 18:39:49.167103 ip-10-0-143-17 kubenswrapper[2585]: I0710 18:39:49.088038 2585 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"sys\" (UniqueName: \"kubernetes.io/host-path/fdd15e8f-ae22-4144-b9bb-b4e4f09be270-sys\") pod \"tuned-w6ds9\" (UID: \"fdd15e8f-ae22-4144-b9bb-b4e4f09be270\") " pod="openshift-cluster-node-tuning-operator/tuned-w6ds9" Jul 10 18:39:49.167103 ip-10-0-143-17 kubenswrapper[2585]: I0710 18:39:49.088065 2585 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-zjxqn\" (UniqueName: \"kubernetes.io/projected/fdd15e8f-ae22-4144-b9bb-b4e4f09be270-kube-api-access-zjxqn\") pod \"tuned-w6ds9\" (UID: \"fdd15e8f-ae22-4144-b9bb-b4e4f09be270\") " pod="openshift-cluster-node-tuning-operator/tuned-w6ds9" Jul 10 18:39:49.167103 ip-10-0-143-17 kubenswrapper[2585]: I0710 18:39:49.088100 2585 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kubelet-dir\" (UniqueName: \"kubernetes.io/host-path/bd56ef17-098b-40ba-8500-8cb05e293dc5-kubelet-dir\") pod \"aws-ebs-csi-driver-node-cw6g2\" (UID: \"bd56ef17-098b-40ba-8500-8cb05e293dc5\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-cw6g2" Jul 10 18:39:49.167103 ip-10-0-143-17 kubenswrapper[2585]: I0710 18:39:49.088132 2585 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-run-netns\" (UniqueName: \"kubernetes.io/host-path/d863cfea-f2f3-4404-b62a-d41785525f3f-host-run-netns\") pod \"ovnkube-node-xg6bm\" (UID: \"d863cfea-f2f3-4404-b62a-d41785525f3f\") " pod="openshift-ovn-kubernetes/ovnkube-node-xg6bm" Jul 10 18:39:49.167103 ip-10-0-143-17 kubenswrapper[2585]: I0710 18:39:49.088140 2585 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"sys\" (UniqueName: \"kubernetes.io/host-path/fdd15e8f-ae22-4144-b9bb-b4e4f09be270-sys\") pod \"tuned-w6ds9\" (UID: \"fdd15e8f-ae22-4144-b9bb-b4e4f09be270\") " pod="openshift-cluster-node-tuning-operator/tuned-w6ds9" Jul 10 18:39:49.167525 ip-10-0-143-17 kubenswrapper[2585]: I0710 18:39:49.088160 2585 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"run-ovn\" (UniqueName: \"kubernetes.io/host-path/d863cfea-f2f3-4404-b62a-d41785525f3f-run-ovn\") pod \"ovnkube-node-xg6bm\" (UID: \"d863cfea-f2f3-4404-b62a-d41785525f3f\") " pod="openshift-ovn-kubernetes/ovnkube-node-xg6bm" Jul 10 18:39:49.167525 ip-10-0-143-17 kubenswrapper[2585]: I0710 18:39:49.088206 2585 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kubelet-dir\" (UniqueName: \"kubernetes.io/host-path/bd56ef17-098b-40ba-8500-8cb05e293dc5-kubelet-dir\") pod \"aws-ebs-csi-driver-node-cw6g2\" (UID: \"bd56ef17-098b-40ba-8500-8cb05e293dc5\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-cw6g2" Jul 10 18:39:49.167525 ip-10-0-143-17 kubenswrapper[2585]: I0710 18:39:49.088263 2585 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"host-run-netns\" (UniqueName: \"kubernetes.io/host-path/d863cfea-f2f3-4404-b62a-d41785525f3f-host-run-netns\") pod \"ovnkube-node-xg6bm\" (UID: \"d863cfea-f2f3-4404-b62a-d41785525f3f\") " pod="openshift-ovn-kubernetes/ovnkube-node-xg6bm" Jul 10 18:39:49.167525 ip-10-0-143-17 kubenswrapper[2585]: I0710 18:39:49.088303 2585 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"run-ovn\" (UniqueName: \"kubernetes.io/host-path/d863cfea-f2f3-4404-b62a-d41785525f3f-run-ovn\") pod \"ovnkube-node-xg6bm\" (UID: \"d863cfea-f2f3-4404-b62a-d41785525f3f\") " pod="openshift-ovn-kubernetes/ovnkube-node-xg6bm" Jul 10 18:39:49.167525 ip-10-0-143-17 kubenswrapper[2585]: I0710 18:39:49.088300 2585 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"node-log\" (UniqueName: \"kubernetes.io/host-path/d863cfea-f2f3-4404-b62a-d41785525f3f-node-log\") pod \"ovnkube-node-xg6bm\" (UID: \"d863cfea-f2f3-4404-b62a-d41785525f3f\") " pod="openshift-ovn-kubernetes/ovnkube-node-xg6bm" Jul 10 18:39:49.167525 ip-10-0-143-17 kubenswrapper[2585]: I0710 18:39:49.088342 2585 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"iptables-alerter-script\" (UniqueName: \"kubernetes.io/configmap/03ab0623-6f5f-49c6-b999-6dc99fc53667-iptables-alerter-script\") pod \"iptables-alerter-m5446\" (UID: \"03ab0623-6f5f-49c6-b999-6dc99fc53667\") " pod="openshift-network-operator/iptables-alerter-m5446" Jul 10 18:39:49.167525 ip-10-0-143-17 kubenswrapper[2585]: I0710 18:39:49.088452 2585 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"env-overrides\" (UniqueName: \"kubernetes.io/configmap/d863cfea-f2f3-4404-b62a-d41785525f3f-env-overrides\") pod \"ovnkube-node-xg6bm\" (UID: \"d863cfea-f2f3-4404-b62a-d41785525f3f\") " pod="openshift-ovn-kubernetes/ovnkube-node-xg6bm" Jul 10 18:39:49.167525 ip-10-0-143-17 kubenswrapper[2585]: I0710 18:39:49.090640 2585 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"etc-tuned\" (UniqueName: \"kubernetes.io/empty-dir/fdd15e8f-ae22-4144-b9bb-b4e4f09be270-etc-tuned\") pod \"tuned-w6ds9\" (UID: \"fdd15e8f-ae22-4144-b9bb-b4e4f09be270\") " pod="openshift-cluster-node-tuning-operator/tuned-w6ds9" Jul 10 18:39:49.167525 ip-10-0-143-17 kubenswrapper[2585]: I0710 18:39:49.090809 2585 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"tmp\" (UniqueName: \"kubernetes.io/empty-dir/fdd15e8f-ae22-4144-b9bb-b4e4f09be270-tmp\") pod \"tuned-w6ds9\" (UID: \"fdd15e8f-ae22-4144-b9bb-b4e4f09be270\") " pod="openshift-cluster-node-tuning-operator/tuned-w6ds9" Jul 10 18:39:49.167525 ip-10-0-143-17 kubenswrapper[2585]: I0710 18:39:49.091436 2585 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"ovn-node-metrics-cert\" (UniqueName: \"kubernetes.io/secret/d863cfea-f2f3-4404-b62a-d41785525f3f-ovn-node-metrics-cert\") pod \"ovnkube-node-xg6bm\" (UID: \"d863cfea-f2f3-4404-b62a-d41785525f3f\") " pod="openshift-ovn-kubernetes/ovnkube-node-xg6bm" Jul 10 18:39:49.167525 ip-10-0-143-17 kubenswrapper[2585]: I0710 18:39:49.095040 2585 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-n5dbk\" (UniqueName: \"kubernetes.io/projected/03ab0623-6f5f-49c6-b999-6dc99fc53667-kube-api-access-n5dbk\") pod \"iptables-alerter-m5446\" (UID: \"03ab0623-6f5f-49c6-b999-6dc99fc53667\") " pod="openshift-network-operator/iptables-alerter-m5446" Jul 10 18:39:49.167525 ip-10-0-143-17 kubenswrapper[2585]: I0710 18:39:49.095383 2585 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-cb4h7\" (UniqueName: \"kubernetes.io/projected/bd56ef17-098b-40ba-8500-8cb05e293dc5-kube-api-access-cb4h7\") pod \"aws-ebs-csi-driver-node-cw6g2\" (UID: \"bd56ef17-098b-40ba-8500-8cb05e293dc5\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-cw6g2" Jul 10 18:39:49.167525 ip-10-0-143-17 kubenswrapper[2585]: I0710 18:39:49.096499 2585 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-rz4xs\" (UniqueName: \"kubernetes.io/projected/d863cfea-f2f3-4404-b62a-d41785525f3f-kube-api-access-rz4xs\") pod \"ovnkube-node-xg6bm\" (UID: \"d863cfea-f2f3-4404-b62a-d41785525f3f\") " pod="openshift-ovn-kubernetes/ovnkube-node-xg6bm" Jul 10 18:39:49.167525 ip-10-0-143-17 kubenswrapper[2585]: I0710 18:39:49.096815 2585 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-zjxqn\" (UniqueName: \"kubernetes.io/projected/fdd15e8f-ae22-4144-b9bb-b4e4f09be270-kube-api-access-zjxqn\") pod \"tuned-w6ds9\" (UID: \"fdd15e8f-ae22-4144-b9bb-b4e4f09be270\") " pod="openshift-cluster-node-tuning-operator/tuned-w6ds9" Jul 10 18:39:49.189277 ip-10-0-143-17 kubenswrapper[2585]: I0710 18:39:49.189236 2585 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-run-multus-certs\" (UniqueName: \"kubernetes.io/host-path/d0da579f-0565-4704-942a-402e5bf46135-host-run-multus-certs\") pod \"multus-h4nv5\" (UID: \"d0da579f-0565-4704-942a-402e5bf46135\") " pod="openshift-multus/multus-h4nv5" Jul 10 18:39:49.189417 ip-10-0-143-17 kubenswrapper[2585]: I0710 18:39:49.189285 2585 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/6516579e-a521-4350-bc6b-c4d698b2f845-metrics-certs\") pod \"network-metrics-daemon-7rwwk\" (UID: \"6516579e-a521-4350-bc6b-c4d698b2f845\") " pod="openshift-multus/network-metrics-daemon-7rwwk" Jul 10 18:39:49.189417 ip-10-0-143-17 kubenswrapper[2585]: I0710 18:39:49.189305 2585 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"host-run-multus-certs\" (UniqueName: \"kubernetes.io/host-path/d0da579f-0565-4704-942a-402e5bf46135-host-run-multus-certs\") pod \"multus-h4nv5\" (UID: \"d0da579f-0565-4704-942a-402e5bf46135\") " pod="openshift-multus/multus-h4nv5" Jul 10 18:39:49.189417 ip-10-0-143-17 kubenswrapper[2585]: I0710 18:39:49.189312 2585 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-74lfk\" (UniqueName: \"kubernetes.io/projected/6516579e-a521-4350-bc6b-c4d698b2f845-kube-api-access-74lfk\") pod \"network-metrics-daemon-7rwwk\" (UID: \"6516579e-a521-4350-bc6b-c4d698b2f845\") " pod="openshift-multus/network-metrics-daemon-7rwwk" Jul 10 18:39:49.189417 ip-10-0-143-17 kubenswrapper[2585]: I0710 18:39:49.189361 2585 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"agent-certs\" (UniqueName: \"kubernetes.io/secret/11f2f17b-9176-4b2b-b4a4-836025deed92-agent-certs\") pod \"konnectivity-agent-x7zf6\" (UID: \"11f2f17b-9176-4b2b-b4a4-836025deed92\") " pod="kube-system/konnectivity-agent-x7zf6" Jul 10 18:39:49.189417 ip-10-0-143-17 kubenswrapper[2585]: I0710 18:39:49.189389 2585 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-var-lib-kubelet\" (UniqueName: \"kubernetes.io/host-path/d0da579f-0565-4704-942a-402e5bf46135-host-var-lib-kubelet\") pod \"multus-h4nv5\" (UID: \"d0da579f-0565-4704-942a-402e5bf46135\") " pod="openshift-multus/multus-h4nv5" Jul 10 18:39:49.189417 ip-10-0-143-17 kubenswrapper[2585]: I0710 18:39:49.189412 2585 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"cni-binary-copy\" (UniqueName: \"kubernetes.io/configmap/c3be1d6d-b0aa-4613-9d9a-e308693cc790-cni-binary-copy\") pod \"multus-additional-cni-plugins-w4cxp\" (UID: \"c3be1d6d-b0aa-4613-9d9a-e308693cc790\") " pod="openshift-multus/multus-additional-cni-plugins-w4cxp" Jul 10 18:39:49.189749 ip-10-0-143-17 kubenswrapper[2585]: I0710 18:39:49.189437 2585 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"konnectivity-ca\" (UniqueName: \"kubernetes.io/configmap/11f2f17b-9176-4b2b-b4a4-836025deed92-konnectivity-ca\") pod \"konnectivity-agent-x7zf6\" (UID: \"11f2f17b-9176-4b2b-b4a4-836025deed92\") " pod="kube-system/konnectivity-agent-x7zf6" Jul 10 18:39:49.189749 ip-10-0-143-17 kubenswrapper[2585]: I0710 18:39:49.189462 2585 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"multus-socket-dir-parent\" (UniqueName: \"kubernetes.io/host-path/d0da579f-0565-4704-942a-402e5bf46135-multus-socket-dir-parent\") pod \"multus-h4nv5\" (UID: \"d0da579f-0565-4704-942a-402e5bf46135\") " pod="openshift-multus/multus-h4nv5" Jul 10 18:39:49.189749 ip-10-0-143-17 kubenswrapper[2585]: I0710 18:39:49.189482 2585 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-var-lib-cni-bin\" (UniqueName: \"kubernetes.io/host-path/d0da579f-0565-4704-942a-402e5bf46135-host-var-lib-cni-bin\") pod \"multus-h4nv5\" (UID: \"d0da579f-0565-4704-942a-402e5bf46135\") " pod="openshift-multus/multus-h4nv5" Jul 10 18:39:49.189749 ip-10-0-143-17 kubenswrapper[2585]: I0710 18:39:49.189505 2585 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"multus-conf-dir\" (UniqueName: \"kubernetes.io/host-path/d0da579f-0565-4704-942a-402e5bf46135-multus-conf-dir\") pod \"multus-h4nv5\" (UID: \"d0da579f-0565-4704-942a-402e5bf46135\") " pod="openshift-multus/multus-h4nv5" Jul 10 18:39:49.189749 ip-10-0-143-17 kubenswrapper[2585]: I0710 18:39:49.189525 2585 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"multus-daemon-config\" (UniqueName: \"kubernetes.io/configmap/d0da579f-0565-4704-942a-402e5bf46135-multus-daemon-config\") pod \"multus-h4nv5\" (UID: \"d0da579f-0565-4704-942a-402e5bf46135\") " pod="openshift-multus/multus-h4nv5" Jul 10 18:39:49.189749 ip-10-0-143-17 kubenswrapper[2585]: I0710 18:39:49.189571 2585 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-run-netns\" (UniqueName: \"kubernetes.io/host-path/d0da579f-0565-4704-942a-402e5bf46135-host-run-netns\") pod \"multus-h4nv5\" (UID: \"d0da579f-0565-4704-942a-402e5bf46135\") " pod="openshift-multus/multus-h4nv5" Jul 10 18:39:49.189749 ip-10-0-143-17 kubenswrapper[2585]: I0710 18:39:49.189594 2585 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"serviceca\" (UniqueName: \"kubernetes.io/configmap/ad44fb01-da7b-40ac-906e-5a7badc87f54-serviceca\") pod \"node-ca-crbxd\" (UID: \"ad44fb01-da7b-40ac-906e-5a7badc87f54\") " pod="openshift-image-registry/node-ca-crbxd" Jul 10 18:39:49.189749 ip-10-0-143-17 kubenswrapper[2585]: I0710 18:39:49.189629 2585 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"etc-kubernetes\" (UniqueName: \"kubernetes.io/host-path/d0da579f-0565-4704-942a-402e5bf46135-etc-kubernetes\") pod \"multus-h4nv5\" (UID: \"d0da579f-0565-4704-942a-402e5bf46135\") " pod="openshift-multus/multus-h4nv5" Jul 10 18:39:49.189749 ip-10-0-143-17 kubenswrapper[2585]: E0710 18:39:49.189649 2585 secret.go:189] Couldn't get secret openshift-multus/metrics-daemon-secret: object "openshift-multus"/"metrics-daemon-secret" not registered Jul 10 18:39:49.189749 ip-10-0-143-17 kubenswrapper[2585]: I0710 18:39:49.189653 2585 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"os-release\" (UniqueName: \"kubernetes.io/host-path/c3be1d6d-b0aa-4613-9d9a-e308693cc790-os-release\") pod \"multus-additional-cni-plugins-w4cxp\" (UID: \"c3be1d6d-b0aa-4613-9d9a-e308693cc790\") " pod="openshift-multus/multus-additional-cni-plugins-w4cxp" Jul 10 18:39:49.189749 ip-10-0-143-17 kubenswrapper[2585]: I0710 18:39:49.189676 2585 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-rxhcr\" (UniqueName: \"kubernetes.io/projected/c3be1d6d-b0aa-4613-9d9a-e308693cc790-kube-api-access-rxhcr\") pod \"multus-additional-cni-plugins-w4cxp\" (UID: \"c3be1d6d-b0aa-4613-9d9a-e308693cc790\") " pod="openshift-multus/multus-additional-cni-plugins-w4cxp" Jul 10 18:39:49.189749 ip-10-0-143-17 kubenswrapper[2585]: I0710 18:39:49.189703 2585 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-82qmj\" (UniqueName: \"kubernetes.io/projected/2166f472-2dd7-47f0-a15e-72d2abf71118-kube-api-access-82qmj\") pod \"network-check-target-7tsng\" (UID: \"2166f472-2dd7-47f0-a15e-72d2abf71118\") " pod="openshift-network-diagnostics/network-check-target-7tsng" Jul 10 18:39:49.189749 ip-10-0-143-17 kubenswrapper[2585]: E0710 18:39:49.189717 2585 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/6516579e-a521-4350-bc6b-c4d698b2f845-metrics-certs podName:6516579e-a521-4350-bc6b-c4d698b2f845 nodeName:}" failed. No retries permitted until 2026-07-10 18:39:49.689698297 +0000 UTC m=+3.147058229 (durationBeforeRetry 500ms). Error: MountVolume.SetUp failed for volume "metrics-certs" (UniqueName: "kubernetes.io/secret/6516579e-a521-4350-bc6b-c4d698b2f845-metrics-certs") pod "network-metrics-daemon-7rwwk" (UID: "6516579e-a521-4350-bc6b-c4d698b2f845") : object "openshift-multus"/"metrics-daemon-secret" not registered Jul 10 18:39:49.190263 ip-10-0-143-17 kubenswrapper[2585]: I0710 18:39:49.189756 2585 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"system-cni-dir\" (UniqueName: \"kubernetes.io/host-path/d0da579f-0565-4704-942a-402e5bf46135-system-cni-dir\") pod \"multus-h4nv5\" (UID: \"d0da579f-0565-4704-942a-402e5bf46135\") " pod="openshift-multus/multus-h4nv5" Jul 10 18:39:49.190263 ip-10-0-143-17 kubenswrapper[2585]: I0710 18:39:49.189780 2585 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-4rrz6\" (UniqueName: \"kubernetes.io/projected/ad44fb01-da7b-40ac-906e-5a7badc87f54-kube-api-access-4rrz6\") pod \"node-ca-crbxd\" (UID: \"ad44fb01-da7b-40ac-906e-5a7badc87f54\") " pod="openshift-image-registry/node-ca-crbxd" Jul 10 18:39:49.190263 ip-10-0-143-17 kubenswrapper[2585]: I0710 18:39:49.189804 2585 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tuning-conf-dir\" (UniqueName: \"kubernetes.io/host-path/c3be1d6d-b0aa-4613-9d9a-e308693cc790-tuning-conf-dir\") pod \"multus-additional-cni-plugins-w4cxp\" (UID: \"c3be1d6d-b0aa-4613-9d9a-e308693cc790\") " pod="openshift-multus/multus-additional-cni-plugins-w4cxp" Jul 10 18:39:49.190263 ip-10-0-143-17 kubenswrapper[2585]: I0710 18:39:49.189827 2585 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"cnibin\" (UniqueName: \"kubernetes.io/host-path/d0da579f-0565-4704-942a-402e5bf46135-cnibin\") pod \"multus-h4nv5\" (UID: \"d0da579f-0565-4704-942a-402e5bf46135\") " pod="openshift-multus/multus-h4nv5" Jul 10 18:39:49.190263 ip-10-0-143-17 kubenswrapper[2585]: I0710 18:39:49.189852 2585 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/5a594168-14da-41cb-9871-edbbf08720f7-tmp-dir\") pod \"node-resolver-ndwqb\" (UID: \"5a594168-14da-41cb-9871-edbbf08720f7\") " pod="openshift-dns/node-resolver-ndwqb" Jul 10 18:39:49.190263 ip-10-0-143-17 kubenswrapper[2585]: I0710 18:39:49.189877 2585 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"os-release\" (UniqueName: \"kubernetes.io/host-path/d0da579f-0565-4704-942a-402e5bf46135-os-release\") pod \"multus-h4nv5\" (UID: \"d0da579f-0565-4704-942a-402e5bf46135\") " pod="openshift-multus/multus-h4nv5" Jul 10 18:39:49.190263 ip-10-0-143-17 kubenswrapper[2585]: I0710 18:39:49.189901 2585 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"hostroot\" (UniqueName: \"kubernetes.io/host-path/d0da579f-0565-4704-942a-402e5bf46135-hostroot\") pod \"multus-h4nv5\" (UID: \"d0da579f-0565-4704-942a-402e5bf46135\") " pod="openshift-multus/multus-h4nv5" Jul 10 18:39:49.190263 ip-10-0-143-17 kubenswrapper[2585]: I0710 18:39:49.189925 2585 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-bzp9h\" (UniqueName: \"kubernetes.io/projected/d0da579f-0565-4704-942a-402e5bf46135-kube-api-access-bzp9h\") pod \"multus-h4nv5\" (UID: \"d0da579f-0565-4704-942a-402e5bf46135\") " pod="openshift-multus/multus-h4nv5" Jul 10 18:39:49.190263 ip-10-0-143-17 kubenswrapper[2585]: I0710 18:39:49.189948 2585 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"whereabouts-flatfile-configmap\" (UniqueName: \"kubernetes.io/configmap/c3be1d6d-b0aa-4613-9d9a-e308693cc790-whereabouts-flatfile-configmap\") pod \"multus-additional-cni-plugins-w4cxp\" (UID: \"c3be1d6d-b0aa-4613-9d9a-e308693cc790\") " pod="openshift-multus/multus-additional-cni-plugins-w4cxp" Jul 10 18:39:49.190263 ip-10-0-143-17 kubenswrapper[2585]: I0710 18:39:49.189981 2585 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"cni-sysctl-allowlist\" (UniqueName: \"kubernetes.io/configmap/c3be1d6d-b0aa-4613-9d9a-e308693cc790-cni-sysctl-allowlist\") pod \"multus-additional-cni-plugins-w4cxp\" (UID: \"c3be1d6d-b0aa-4613-9d9a-e308693cc790\") " pod="openshift-multus/multus-additional-cni-plugins-w4cxp" Jul 10 18:39:49.190263 ip-10-0-143-17 kubenswrapper[2585]: I0710 18:39:49.190008 2585 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"multus-cni-dir\" (UniqueName: \"kubernetes.io/host-path/d0da579f-0565-4704-942a-402e5bf46135-multus-cni-dir\") pod \"multus-h4nv5\" (UID: \"d0da579f-0565-4704-942a-402e5bf46135\") " pod="openshift-multus/multus-h4nv5" Jul 10 18:39:49.190263 ip-10-0-143-17 kubenswrapper[2585]: I0710 18:39:49.190031 2585 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"system-cni-dir\" (UniqueName: \"kubernetes.io/host-path/c3be1d6d-b0aa-4613-9d9a-e308693cc790-system-cni-dir\") pod \"multus-additional-cni-plugins-w4cxp\" (UID: \"c3be1d6d-b0aa-4613-9d9a-e308693cc790\") " pod="openshift-multus/multus-additional-cni-plugins-w4cxp" Jul 10 18:39:49.190263 ip-10-0-143-17 kubenswrapper[2585]: I0710 18:39:49.190057 2585 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"cnibin\" (UniqueName: \"kubernetes.io/host-path/c3be1d6d-b0aa-4613-9d9a-e308693cc790-cnibin\") pod \"multus-additional-cni-plugins-w4cxp\" (UID: \"c3be1d6d-b0aa-4613-9d9a-e308693cc790\") " pod="openshift-multus/multus-additional-cni-plugins-w4cxp" Jul 10 18:39:49.190263 ip-10-0-143-17 kubenswrapper[2585]: I0710 18:39:49.190085 2585 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-fwnk6\" (UniqueName: \"kubernetes.io/projected/5a594168-14da-41cb-9871-edbbf08720f7-kube-api-access-fwnk6\") pod \"node-resolver-ndwqb\" (UID: \"5a594168-14da-41cb-9871-edbbf08720f7\") " pod="openshift-dns/node-resolver-ndwqb" Jul 10 18:39:49.192080 ip-10-0-143-17 kubenswrapper[2585]: I0710 18:39:49.190112 2585 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-var-lib-cni-multus\" (UniqueName: \"kubernetes.io/host-path/d0da579f-0565-4704-942a-402e5bf46135-host-var-lib-cni-multus\") pod \"multus-h4nv5\" (UID: \"d0da579f-0565-4704-942a-402e5bf46135\") " pod="openshift-multus/multus-h4nv5" Jul 10 18:39:49.192080 ip-10-0-143-17 kubenswrapper[2585]: I0710 18:39:49.190137 2585 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"hosts-file\" (UniqueName: \"kubernetes.io/host-path/5a594168-14da-41cb-9871-edbbf08720f7-hosts-file\") pod \"node-resolver-ndwqb\" (UID: \"5a594168-14da-41cb-9871-edbbf08720f7\") " pod="openshift-dns/node-resolver-ndwqb" Jul 10 18:39:49.192080 ip-10-0-143-17 kubenswrapper[2585]: I0710 18:39:49.190161 2585 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-run-k8s-cni-cncf-io\" (UniqueName: \"kubernetes.io/host-path/d0da579f-0565-4704-942a-402e5bf46135-host-run-k8s-cni-cncf-io\") pod \"multus-h4nv5\" (UID: \"d0da579f-0565-4704-942a-402e5bf46135\") " pod="openshift-multus/multus-h4nv5" Jul 10 18:39:49.192080 ip-10-0-143-17 kubenswrapper[2585]: I0710 18:39:49.190187 2585 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"cni-binary-copy\" (UniqueName: \"kubernetes.io/configmap/d0da579f-0565-4704-942a-402e5bf46135-cni-binary-copy\") pod \"multus-h4nv5\" (UID: \"d0da579f-0565-4704-942a-402e5bf46135\") " pod="openshift-multus/multus-h4nv5" Jul 10 18:39:49.192080 ip-10-0-143-17 kubenswrapper[2585]: I0710 18:39:49.190209 2585 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host\" (UniqueName: \"kubernetes.io/host-path/ad44fb01-da7b-40ac-906e-5a7badc87f54-host\") pod \"node-ca-crbxd\" (UID: \"ad44fb01-da7b-40ac-906e-5a7badc87f54\") " pod="openshift-image-registry/node-ca-crbxd" Jul 10 18:39:49.192080 ip-10-0-143-17 kubenswrapper[2585]: I0710 18:39:49.190218 2585 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"multus-conf-dir\" (UniqueName: \"kubernetes.io/host-path/d0da579f-0565-4704-942a-402e5bf46135-multus-conf-dir\") pod \"multus-h4nv5\" (UID: \"d0da579f-0565-4704-942a-402e5bf46135\") " pod="openshift-multus/multus-h4nv5" Jul 10 18:39:49.192080 ip-10-0-143-17 kubenswrapper[2585]: I0710 18:39:49.190290 2585 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"host\" (UniqueName: \"kubernetes.io/host-path/ad44fb01-da7b-40ac-906e-5a7badc87f54-host\") pod \"node-ca-crbxd\" (UID: \"ad44fb01-da7b-40ac-906e-5a7badc87f54\") " pod="openshift-image-registry/node-ca-crbxd" Jul 10 18:39:49.192080 ip-10-0-143-17 kubenswrapper[2585]: I0710 18:39:49.190294 2585 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"multus-socket-dir-parent\" (UniqueName: \"kubernetes.io/host-path/d0da579f-0565-4704-942a-402e5bf46135-multus-socket-dir-parent\") pod \"multus-h4nv5\" (UID: \"d0da579f-0565-4704-942a-402e5bf46135\") " pod="openshift-multus/multus-h4nv5" Jul 10 18:39:49.192080 ip-10-0-143-17 kubenswrapper[2585]: I0710 18:39:49.190357 2585 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"host-var-lib-cni-bin\" (UniqueName: \"kubernetes.io/host-path/d0da579f-0565-4704-942a-402e5bf46135-host-var-lib-cni-bin\") pod \"multus-h4nv5\" (UID: \"d0da579f-0565-4704-942a-402e5bf46135\") " pod="openshift-multus/multus-h4nv5" Jul 10 18:39:49.192080 ip-10-0-143-17 kubenswrapper[2585]: I0710 18:39:49.190367 2585 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"multus-cni-dir\" (UniqueName: \"kubernetes.io/host-path/d0da579f-0565-4704-942a-402e5bf46135-multus-cni-dir\") pod \"multus-h4nv5\" (UID: \"d0da579f-0565-4704-942a-402e5bf46135\") " pod="openshift-multus/multus-h4nv5" Jul 10 18:39:49.192080 ip-10-0-143-17 kubenswrapper[2585]: I0710 18:39:49.190405 2585 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"host-var-lib-cni-multus\" (UniqueName: \"kubernetes.io/host-path/d0da579f-0565-4704-942a-402e5bf46135-host-var-lib-cni-multus\") pod \"multus-h4nv5\" (UID: \"d0da579f-0565-4704-942a-402e5bf46135\") " pod="openshift-multus/multus-h4nv5" Jul 10 18:39:49.192080 ip-10-0-143-17 kubenswrapper[2585]: I0710 18:39:49.190424 2585 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"hosts-file\" (UniqueName: \"kubernetes.io/host-path/5a594168-14da-41cb-9871-edbbf08720f7-hosts-file\") pod \"node-resolver-ndwqb\" (UID: \"5a594168-14da-41cb-9871-edbbf08720f7\") " pod="openshift-dns/node-resolver-ndwqb" Jul 10 18:39:49.192080 ip-10-0-143-17 kubenswrapper[2585]: I0710 18:39:49.190437 2585 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"host-var-lib-kubelet\" (UniqueName: \"kubernetes.io/host-path/d0da579f-0565-4704-942a-402e5bf46135-host-var-lib-kubelet\") pod \"multus-h4nv5\" (UID: \"d0da579f-0565-4704-942a-402e5bf46135\") " pod="openshift-multus/multus-h4nv5" Jul 10 18:39:49.192080 ip-10-0-143-17 kubenswrapper[2585]: I0710 18:39:49.190485 2585 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"system-cni-dir\" (UniqueName: \"kubernetes.io/host-path/c3be1d6d-b0aa-4613-9d9a-e308693cc790-system-cni-dir\") pod \"multus-additional-cni-plugins-w4cxp\" (UID: \"c3be1d6d-b0aa-4613-9d9a-e308693cc790\") " pod="openshift-multus/multus-additional-cni-plugins-w4cxp" Jul 10 18:39:49.192080 ip-10-0-143-17 kubenswrapper[2585]: I0710 18:39:49.190528 2585 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"cnibin\" (UniqueName: \"kubernetes.io/host-path/d0da579f-0565-4704-942a-402e5bf46135-cnibin\") pod \"multus-h4nv5\" (UID: \"d0da579f-0565-4704-942a-402e5bf46135\") " pod="openshift-multus/multus-h4nv5" Jul 10 18:39:49.192801 ip-10-0-143-17 kubenswrapper[2585]: I0710 18:39:49.190590 2585 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"cnibin\" (UniqueName: \"kubernetes.io/host-path/c3be1d6d-b0aa-4613-9d9a-e308693cc790-cnibin\") pod \"multus-additional-cni-plugins-w4cxp\" (UID: \"c3be1d6d-b0aa-4613-9d9a-e308693cc790\") " pod="openshift-multus/multus-additional-cni-plugins-w4cxp" Jul 10 18:39:49.192801 ip-10-0-143-17 kubenswrapper[2585]: I0710 18:39:49.190688 2585 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"os-release\" (UniqueName: \"kubernetes.io/host-path/d0da579f-0565-4704-942a-402e5bf46135-os-release\") pod \"multus-h4nv5\" (UID: \"d0da579f-0565-4704-942a-402e5bf46135\") " pod="openshift-multus/multus-h4nv5" Jul 10 18:39:49.192801 ip-10-0-143-17 kubenswrapper[2585]: I0710 18:39:49.190742 2585 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"hostroot\" (UniqueName: \"kubernetes.io/host-path/d0da579f-0565-4704-942a-402e5bf46135-hostroot\") pod \"multus-h4nv5\" (UID: \"d0da579f-0565-4704-942a-402e5bf46135\") " pod="openshift-multus/multus-h4nv5" Jul 10 18:39:49.192801 ip-10-0-143-17 kubenswrapper[2585]: I0710 18:39:49.190773 2585 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"system-cni-dir\" (UniqueName: \"kubernetes.io/host-path/d0da579f-0565-4704-942a-402e5bf46135-system-cni-dir\") pod \"multus-h4nv5\" (UID: \"d0da579f-0565-4704-942a-402e5bf46135\") " pod="openshift-multus/multus-h4nv5" Jul 10 18:39:49.192801 ip-10-0-143-17 kubenswrapper[2585]: I0710 18:39:49.190945 2585 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"cni-binary-copy\" (UniqueName: \"kubernetes.io/configmap/c3be1d6d-b0aa-4613-9d9a-e308693cc790-cni-binary-copy\") pod \"multus-additional-cni-plugins-w4cxp\" (UID: \"c3be1d6d-b0aa-4613-9d9a-e308693cc790\") " pod="openshift-multus/multus-additional-cni-plugins-w4cxp" Jul 10 18:39:49.192801 ip-10-0-143-17 kubenswrapper[2585]: I0710 18:39:49.190982 2585 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"cni-binary-copy\" (UniqueName: \"kubernetes.io/configmap/d0da579f-0565-4704-942a-402e5bf46135-cni-binary-copy\") pod \"multus-h4nv5\" (UID: \"d0da579f-0565-4704-942a-402e5bf46135\") " pod="openshift-multus/multus-h4nv5" Jul 10 18:39:49.192801 ip-10-0-143-17 kubenswrapper[2585]: I0710 18:39:49.191264 2585 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/5a594168-14da-41cb-9871-edbbf08720f7-tmp-dir\") pod \"node-resolver-ndwqb\" (UID: \"5a594168-14da-41cb-9871-edbbf08720f7\") " pod="openshift-dns/node-resolver-ndwqb" Jul 10 18:39:49.192801 ip-10-0-143-17 kubenswrapper[2585]: I0710 18:39:49.191297 2585 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"cni-sysctl-allowlist\" (UniqueName: \"kubernetes.io/configmap/c3be1d6d-b0aa-4613-9d9a-e308693cc790-cni-sysctl-allowlist\") pod \"multus-additional-cni-plugins-w4cxp\" (UID: \"c3be1d6d-b0aa-4613-9d9a-e308693cc790\") " pod="openshift-multus/multus-additional-cni-plugins-w4cxp" Jul 10 18:39:49.192801 ip-10-0-143-17 kubenswrapper[2585]: I0710 18:39:49.191327 2585 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"whereabouts-flatfile-configmap\" (UniqueName: \"kubernetes.io/configmap/c3be1d6d-b0aa-4613-9d9a-e308693cc790-whereabouts-flatfile-configmap\") pod \"multus-additional-cni-plugins-w4cxp\" (UID: \"c3be1d6d-b0aa-4613-9d9a-e308693cc790\") " pod="openshift-multus/multus-additional-cni-plugins-w4cxp" Jul 10 18:39:49.192801 ip-10-0-143-17 kubenswrapper[2585]: I0710 18:39:49.191417 2585 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"tuning-conf-dir\" (UniqueName: \"kubernetes.io/host-path/c3be1d6d-b0aa-4613-9d9a-e308693cc790-tuning-conf-dir\") pod \"multus-additional-cni-plugins-w4cxp\" (UID: \"c3be1d6d-b0aa-4613-9d9a-e308693cc790\") " pod="openshift-multus/multus-additional-cni-plugins-w4cxp" Jul 10 18:39:49.192801 ip-10-0-143-17 kubenswrapper[2585]: I0710 18:39:49.191422 2585 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"os-release\" (UniqueName: \"kubernetes.io/host-path/c3be1d6d-b0aa-4613-9d9a-e308693cc790-os-release\") pod \"multus-additional-cni-plugins-w4cxp\" (UID: \"c3be1d6d-b0aa-4613-9d9a-e308693cc790\") " pod="openshift-multus/multus-additional-cni-plugins-w4cxp" Jul 10 18:39:49.192801 ip-10-0-143-17 kubenswrapper[2585]: I0710 18:39:49.191442 2585 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"serviceca\" (UniqueName: \"kubernetes.io/configmap/ad44fb01-da7b-40ac-906e-5a7badc87f54-serviceca\") pod \"node-ca-crbxd\" (UID: \"ad44fb01-da7b-40ac-906e-5a7badc87f54\") " pod="openshift-image-registry/node-ca-crbxd" Jul 10 18:39:49.192801 ip-10-0-143-17 kubenswrapper[2585]: I0710 18:39:49.191446 2585 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"konnectivity-ca\" (UniqueName: \"kubernetes.io/configmap/11f2f17b-9176-4b2b-b4a4-836025deed92-konnectivity-ca\") pod \"konnectivity-agent-x7zf6\" (UID: \"11f2f17b-9176-4b2b-b4a4-836025deed92\") " pod="kube-system/konnectivity-agent-x7zf6" Jul 10 18:39:49.192801 ip-10-0-143-17 kubenswrapper[2585]: I0710 18:39:49.191471 2585 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"etc-kubernetes\" (UniqueName: \"kubernetes.io/host-path/d0da579f-0565-4704-942a-402e5bf46135-etc-kubernetes\") pod \"multus-h4nv5\" (UID: \"d0da579f-0565-4704-942a-402e5bf46135\") " pod="openshift-multus/multus-h4nv5" Jul 10 18:39:49.192801 ip-10-0-143-17 kubenswrapper[2585]: I0710 18:39:49.191509 2585 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"host-run-k8s-cni-cncf-io\" (UniqueName: \"kubernetes.io/host-path/d0da579f-0565-4704-942a-402e5bf46135-host-run-k8s-cni-cncf-io\") pod \"multus-h4nv5\" (UID: \"d0da579f-0565-4704-942a-402e5bf46135\") " pod="openshift-multus/multus-h4nv5" Jul 10 18:39:49.193423 ip-10-0-143-17 kubenswrapper[2585]: I0710 18:39:49.191512 2585 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"host-run-netns\" (UniqueName: \"kubernetes.io/host-path/d0da579f-0565-4704-942a-402e5bf46135-host-run-netns\") pod \"multus-h4nv5\" (UID: \"d0da579f-0565-4704-942a-402e5bf46135\") " pod="openshift-multus/multus-h4nv5" Jul 10 18:39:49.193423 ip-10-0-143-17 kubenswrapper[2585]: I0710 18:39:49.191784 2585 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"multus-daemon-config\" (UniqueName: \"kubernetes.io/configmap/d0da579f-0565-4704-942a-402e5bf46135-multus-daemon-config\") pod \"multus-h4nv5\" (UID: \"d0da579f-0565-4704-942a-402e5bf46135\") " pod="openshift-multus/multus-h4nv5" Jul 10 18:39:49.193423 ip-10-0-143-17 kubenswrapper[2585]: I0710 18:39:49.193177 2585 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"agent-certs\" (UniqueName: \"kubernetes.io/secret/11f2f17b-9176-4b2b-b4a4-836025deed92-agent-certs\") pod \"konnectivity-agent-x7zf6\" (UID: \"11f2f17b-9176-4b2b-b4a4-836025deed92\") " pod="kube-system/konnectivity-agent-x7zf6" Jul 10 18:39:49.197342 ip-10-0-143-17 kubenswrapper[2585]: E0710 18:39:49.197320 2585 projected.go:291] Couldn't get configMap openshift-network-diagnostics/kube-root-ca.crt: object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered Jul 10 18:39:49.197458 ip-10-0-143-17 kubenswrapper[2585]: E0710 18:39:49.197348 2585 projected.go:291] Couldn't get configMap openshift-network-diagnostics/openshift-service-ca.crt: object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered Jul 10 18:39:49.197458 ip-10-0-143-17 kubenswrapper[2585]: E0710 18:39:49.197363 2585 projected.go:196] Error preparing data for projected volume kube-api-access-82qmj for pod openshift-network-diagnostics/network-check-target-7tsng: [object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered, object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered] Jul 10 18:39:49.197458 ip-10-0-143-17 kubenswrapper[2585]: E0710 18:39:49.197419 2585 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/projected/2166f472-2dd7-47f0-a15e-72d2abf71118-kube-api-access-82qmj podName:2166f472-2dd7-47f0-a15e-72d2abf71118 nodeName:}" failed. No retries permitted until 2026-07-10 18:39:49.697401889 +0000 UTC m=+3.154761817 (durationBeforeRetry 500ms). Error: MountVolume.SetUp failed for volume "kube-api-access-82qmj" (UniqueName: "kubernetes.io/projected/2166f472-2dd7-47f0-a15e-72d2abf71118-kube-api-access-82qmj") pod "network-check-target-7tsng" (UID: "2166f472-2dd7-47f0-a15e-72d2abf71118") : [object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered, object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered] Jul 10 18:39:49.199355 ip-10-0-143-17 kubenswrapper[2585]: I0710 18:39:49.199299 2585 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-fwnk6\" (UniqueName: \"kubernetes.io/projected/5a594168-14da-41cb-9871-edbbf08720f7-kube-api-access-fwnk6\") pod \"node-resolver-ndwqb\" (UID: \"5a594168-14da-41cb-9871-edbbf08720f7\") " pod="openshift-dns/node-resolver-ndwqb" Jul 10 18:39:49.199475 ip-10-0-143-17 kubenswrapper[2585]: I0710 18:39:49.199435 2585 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-74lfk\" (UniqueName: \"kubernetes.io/projected/6516579e-a521-4350-bc6b-c4d698b2f845-kube-api-access-74lfk\") pod \"network-metrics-daemon-7rwwk\" (UID: \"6516579e-a521-4350-bc6b-c4d698b2f845\") " pod="openshift-multus/network-metrics-daemon-7rwwk" Jul 10 18:39:49.203311 ip-10-0-143-17 kubenswrapper[2585]: I0710 18:39:49.203268 2585 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-4rrz6\" (UniqueName: \"kubernetes.io/projected/ad44fb01-da7b-40ac-906e-5a7badc87f54-kube-api-access-4rrz6\") pod \"node-ca-crbxd\" (UID: \"ad44fb01-da7b-40ac-906e-5a7badc87f54\") " pod="openshift-image-registry/node-ca-crbxd" Jul 10 18:39:49.204147 ip-10-0-143-17 kubenswrapper[2585]: I0710 18:39:49.204122 2585 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-rxhcr\" (UniqueName: \"kubernetes.io/projected/c3be1d6d-b0aa-4613-9d9a-e308693cc790-kube-api-access-rxhcr\") pod \"multus-additional-cni-plugins-w4cxp\" (UID: \"c3be1d6d-b0aa-4613-9d9a-e308693cc790\") " pod="openshift-multus/multus-additional-cni-plugins-w4cxp" Jul 10 18:39:49.204324 ip-10-0-143-17 kubenswrapper[2585]: I0710 18:39:49.204303 2585 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-bzp9h\" (UniqueName: \"kubernetes.io/projected/d0da579f-0565-4704-942a-402e5bf46135-kube-api-access-bzp9h\") pod \"multus-h4nv5\" (UID: \"d0da579f-0565-4704-942a-402e5bf46135\") " pod="openshift-multus/multus-h4nv5" Jul 10 18:39:49.220632 ip-10-0-143-17 kubenswrapper[2585]: I0710 18:39:49.220578 2585 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-143-17.ec2.internal" event={"ID":"3813618e37881b79cb3d5b62e1c14d5b","Type":"ContainerStarted","Data":"c8dbcf9c9570e82db6d2885b49f22e5e04a85f9a3089f34f710c411ab50ba1b2"} Jul 10 18:39:49.221829 ip-10-0-143-17 kubenswrapper[2585]: I0710 18:39:49.221778 2585 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kube-system/kube-apiserver-proxy-ip-10-0-143-17.ec2.internal" event={"ID":"23dbc229ee3e12019dd90879f1a0df34","Type":"ContainerStarted","Data":"af2f0caa4063978d98bfaeeaf7e2d5ba540017447c5414838c333a3203e91c5c"} Jul 10 18:39:49.243425 ip-10-0-143-17 kubenswrapper[2585]: I0710 18:39:49.243346 2585 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-cluster-node-tuning-operator/tuned-w6ds9" Jul 10 18:39:49.256284 ip-10-0-143-17 kubenswrapper[2585]: I0710 18:39:49.256262 2585 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-cw6g2" Jul 10 18:39:49.267124 ip-10-0-143-17 kubenswrapper[2585]: I0710 18:39:49.267100 2585 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-operator/iptables-alerter-m5446" Jul 10 18:39:49.274818 ip-10-0-143-17 kubenswrapper[2585]: I0710 18:39:49.274800 2585 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ovn-kubernetes/ovnkube-node-xg6bm" Jul 10 18:39:49.286504 ip-10-0-143-17 kubenswrapper[2585]: I0710 18:39:49.286484 2585 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-dns/node-resolver-ndwqb" Jul 10 18:39:49.293303 ip-10-0-143-17 kubenswrapper[2585]: I0710 18:39:49.293284 2585 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/multus-h4nv5" Jul 10 18:39:49.300252 ip-10-0-143-17 kubenswrapper[2585]: I0710 18:39:49.300234 2585 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-image-registry/node-ca-crbxd" Jul 10 18:39:49.307732 ip-10-0-143-17 kubenswrapper[2585]: I0710 18:39:49.307715 2585 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/multus-additional-cni-plugins-w4cxp" Jul 10 18:39:49.316325 ip-10-0-143-17 kubenswrapper[2585]: I0710 18:39:49.316301 2585 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/konnectivity-agent-x7zf6" Jul 10 18:39:49.593424 ip-10-0-143-17 kubenswrapper[2585]: I0710 18:39:49.593325 2585 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"original-pull-secret\" (UniqueName: \"kubernetes.io/secret/15a0ea9c-d294-4065-aed5-a182a76a7d7f-original-pull-secret\") pod \"global-pull-secret-syncer-hhpnn\" (UID: \"15a0ea9c-d294-4065-aed5-a182a76a7d7f\") " pod="kube-system/global-pull-secret-syncer-hhpnn" Jul 10 18:39:49.593596 ip-10-0-143-17 kubenswrapper[2585]: E0710 18:39:49.593506 2585 secret.go:189] Couldn't get secret kube-system/original-pull-secret: object "kube-system"/"original-pull-secret" not registered Jul 10 18:39:49.593648 ip-10-0-143-17 kubenswrapper[2585]: E0710 18:39:49.593611 2585 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/15a0ea9c-d294-4065-aed5-a182a76a7d7f-original-pull-secret podName:15a0ea9c-d294-4065-aed5-a182a76a7d7f nodeName:}" failed. No retries permitted until 2026-07-10 18:39:50.593596105 +0000 UTC m=+4.050956031 (durationBeforeRetry 1s). Error: MountVolume.SetUp failed for volume "original-pull-secret" (UniqueName: "kubernetes.io/secret/15a0ea9c-d294-4065-aed5-a182a76a7d7f-original-pull-secret") pod "global-pull-secret-syncer-hhpnn" (UID: "15a0ea9c-d294-4065-aed5-a182a76a7d7f") : object "kube-system"/"original-pull-secret" not registered Jul 10 18:39:49.693950 ip-10-0-143-17 kubenswrapper[2585]: I0710 18:39:49.693904 2585 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/6516579e-a521-4350-bc6b-c4d698b2f845-metrics-certs\") pod \"network-metrics-daemon-7rwwk\" (UID: \"6516579e-a521-4350-bc6b-c4d698b2f845\") " pod="openshift-multus/network-metrics-daemon-7rwwk" Jul 10 18:39:49.694136 ip-10-0-143-17 kubenswrapper[2585]: E0710 18:39:49.694055 2585 secret.go:189] Couldn't get secret openshift-multus/metrics-daemon-secret: object "openshift-multus"/"metrics-daemon-secret" not registered Jul 10 18:39:49.694136 ip-10-0-143-17 kubenswrapper[2585]: E0710 18:39:49.694120 2585 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/6516579e-a521-4350-bc6b-c4d698b2f845-metrics-certs podName:6516579e-a521-4350-bc6b-c4d698b2f845 nodeName:}" failed. No retries permitted until 2026-07-10 18:39:50.694106475 +0000 UTC m=+4.151466387 (durationBeforeRetry 1s). Error: MountVolume.SetUp failed for volume "metrics-certs" (UniqueName: "kubernetes.io/secret/6516579e-a521-4350-bc6b-c4d698b2f845-metrics-certs") pod "network-metrics-daemon-7rwwk" (UID: "6516579e-a521-4350-bc6b-c4d698b2f845") : object "openshift-multus"/"metrics-daemon-secret" not registered Jul 10 18:39:49.795305 ip-10-0-143-17 kubenswrapper[2585]: I0710 18:39:49.795265 2585 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-82qmj\" (UniqueName: \"kubernetes.io/projected/2166f472-2dd7-47f0-a15e-72d2abf71118-kube-api-access-82qmj\") pod \"network-check-target-7tsng\" (UID: \"2166f472-2dd7-47f0-a15e-72d2abf71118\") " pod="openshift-network-diagnostics/network-check-target-7tsng" Jul 10 18:39:49.795493 ip-10-0-143-17 kubenswrapper[2585]: E0710 18:39:49.795431 2585 projected.go:291] Couldn't get configMap openshift-network-diagnostics/kube-root-ca.crt: object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered Jul 10 18:39:49.795493 ip-10-0-143-17 kubenswrapper[2585]: E0710 18:39:49.795448 2585 projected.go:291] Couldn't get configMap openshift-network-diagnostics/openshift-service-ca.crt: object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered Jul 10 18:39:49.795493 ip-10-0-143-17 kubenswrapper[2585]: E0710 18:39:49.795459 2585 projected.go:196] Error preparing data for projected volume kube-api-access-82qmj for pod openshift-network-diagnostics/network-check-target-7tsng: [object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered, object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered] Jul 10 18:39:49.795625 ip-10-0-143-17 kubenswrapper[2585]: E0710 18:39:49.795516 2585 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/projected/2166f472-2dd7-47f0-a15e-72d2abf71118-kube-api-access-82qmj podName:2166f472-2dd7-47f0-a15e-72d2abf71118 nodeName:}" failed. No retries permitted until 2026-07-10 18:39:50.795499923 +0000 UTC m=+4.252859856 (durationBeforeRetry 1s). Error: MountVolume.SetUp failed for volume "kube-api-access-82qmj" (UniqueName: "kubernetes.io/projected/2166f472-2dd7-47f0-a15e-72d2abf71118-kube-api-access-82qmj") pod "network-check-target-7tsng" (UID: "2166f472-2dd7-47f0-a15e-72d2abf71118") : [object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered, object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered] Jul 10 18:39:49.982242 ip-10-0-143-17 kubenswrapper[2585]: I0710 18:39:49.982194 2585 certificate_manager.go:715] "Certificate rotation deadline determined" logger="kubernetes.io/kubelet-serving" expiration="2028-07-09 18:34:48 +0000 UTC" deadline="2028-04-22 17:15:00.471408376 +0000 UTC" Jul 10 18:39:49.982242 ip-10-0-143-17 kubenswrapper[2585]: I0710 18:39:49.982238 2585 certificate_manager.go:431] "Waiting for next certificate rotation" logger="kubernetes.io/kubelet-serving" sleep="15646h35m10.489174909s" Jul 10 18:39:50.171656 ip-10-0-143-17 kubenswrapper[2585]: W0710 18:39:50.171624 2585 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podd0da579f_0565_4704_942a_402e5bf46135.slice/crio-6fc9bbbe7f1a0dc90ca02e7fc2b72fe28eb84729c92d61f2afe8ffb8674540af WatchSource:0}: Error finding container 6fc9bbbe7f1a0dc90ca02e7fc2b72fe28eb84729c92d61f2afe8ffb8674540af: Status 404 returned error can't find the container with id 6fc9bbbe7f1a0dc90ca02e7fc2b72fe28eb84729c92d61f2afe8ffb8674540af Jul 10 18:39:50.172527 ip-10-0-143-17 kubenswrapper[2585]: W0710 18:39:50.172497 2585 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podfdd15e8f_ae22_4144_b9bb_b4e4f09be270.slice/crio-23c1f11244644d13f47d28fa359105d553aab495b97e45ee8940f8b3aa783b5f WatchSource:0}: Error finding container 23c1f11244644d13f47d28fa359105d553aab495b97e45ee8940f8b3aa783b5f: Status 404 returned error can't find the container with id 23c1f11244644d13f47d28fa359105d553aab495b97e45ee8940f8b3aa783b5f Jul 10 18:39:50.173938 ip-10-0-143-17 kubenswrapper[2585]: W0710 18:39:50.173377 2585 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod03ab0623_6f5f_49c6_b999_6dc99fc53667.slice/crio-92f7f571b6f737933e24766efcbf7a9b8414a9fee1b6b679c9b0504fbefb74da WatchSource:0}: Error finding container 92f7f571b6f737933e24766efcbf7a9b8414a9fee1b6b679c9b0504fbefb74da: Status 404 returned error can't find the container with id 92f7f571b6f737933e24766efcbf7a9b8414a9fee1b6b679c9b0504fbefb74da Jul 10 18:39:50.174146 ip-10-0-143-17 kubenswrapper[2585]: W0710 18:39:50.174105 2585 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podad44fb01_da7b_40ac_906e_5a7badc87f54.slice/crio-c40619bd815e61d42e96a2a9cbdc5ef9ec6fe7aa116c59ca232b9cfde3f8d4eb WatchSource:0}: Error finding container c40619bd815e61d42e96a2a9cbdc5ef9ec6fe7aa116c59ca232b9cfde3f8d4eb: Status 404 returned error can't find the container with id c40619bd815e61d42e96a2a9cbdc5ef9ec6fe7aa116c59ca232b9cfde3f8d4eb Jul 10 18:39:50.195059 ip-10-0-143-17 kubenswrapper[2585]: W0710 18:39:50.195028 2585 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podbd56ef17_098b_40ba_8500_8cb05e293dc5.slice/crio-4db3985ffd3bfc3a33dc0fb782253c00384a52876879f40af8ac527706721844 WatchSource:0}: Error finding container 4db3985ffd3bfc3a33dc0fb782253c00384a52876879f40af8ac527706721844: Status 404 returned error can't find the container with id 4db3985ffd3bfc3a33dc0fb782253c00384a52876879f40af8ac527706721844 Jul 10 18:39:50.195860 ip-10-0-143-17 kubenswrapper[2585]: W0710 18:39:50.195813 2585 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podd863cfea_f2f3_4404_b62a_d41785525f3f.slice/crio-0f71fd4cd39c61b32698ca609ea529d23e8dbbcb4adc55cbcae4bee584714dc2 WatchSource:0}: Error finding container 0f71fd4cd39c61b32698ca609ea529d23e8dbbcb4adc55cbcae4bee584714dc2: Status 404 returned error can't find the container with id 0f71fd4cd39c61b32698ca609ea529d23e8dbbcb4adc55cbcae4bee584714dc2 Jul 10 18:39:50.196722 ip-10-0-143-17 kubenswrapper[2585]: W0710 18:39:50.196697 2585 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod5a594168_14da_41cb_9871_edbbf08720f7.slice/crio-c401307c8e58529aceb8dacb35b4e83adbf43aee2611c9c22179be3b2c2e3a26 WatchSource:0}: Error finding container c401307c8e58529aceb8dacb35b4e83adbf43aee2611c9c22179be3b2c2e3a26: Status 404 returned error can't find the container with id c401307c8e58529aceb8dacb35b4e83adbf43aee2611c9c22179be3b2c2e3a26 Jul 10 18:39:50.199795 ip-10-0-143-17 kubenswrapper[2585]: W0710 18:39:50.199774 2585 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod11f2f17b_9176_4b2b_b4a4_836025deed92.slice/crio-e0d6dd547a09d80d48be6865dd7adf8ecf1e5b37d70269108770c6ffc09bd7fe WatchSource:0}: Error finding container e0d6dd547a09d80d48be6865dd7adf8ecf1e5b37d70269108770c6ffc09bd7fe: Status 404 returned error can't find the container with id e0d6dd547a09d80d48be6865dd7adf8ecf1e5b37d70269108770c6ffc09bd7fe Jul 10 18:39:50.210048 ip-10-0-143-17 kubenswrapper[2585]: I0710 18:39:50.210009 2585 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-7rwwk" Jul 10 18:39:50.210161 ip-10-0-143-17 kubenswrapper[2585]: E0710 18:39:50.210139 2585 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-7rwwk" podUID="6516579e-a521-4350-bc6b-c4d698b2f845" Jul 10 18:39:50.224413 ip-10-0-143-17 kubenswrapper[2585]: I0710 18:39:50.224385 2585 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-cw6g2" event={"ID":"bd56ef17-098b-40ba-8500-8cb05e293dc5","Type":"ContainerStarted","Data":"4db3985ffd3bfc3a33dc0fb782253c00384a52876879f40af8ac527706721844"} Jul 10 18:39:50.225434 ip-10-0-143-17 kubenswrapper[2585]: I0710 18:39:50.225409 2585 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-image-registry/node-ca-crbxd" event={"ID":"ad44fb01-da7b-40ac-906e-5a7badc87f54","Type":"ContainerStarted","Data":"c40619bd815e61d42e96a2a9cbdc5ef9ec6fe7aa116c59ca232b9cfde3f8d4eb"} Jul 10 18:39:50.227876 ip-10-0-143-17 kubenswrapper[2585]: I0710 18:39:50.227852 2585 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-dns/node-resolver-ndwqb" event={"ID":"5a594168-14da-41cb-9871-edbbf08720f7","Type":"ContainerStarted","Data":"c401307c8e58529aceb8dacb35b4e83adbf43aee2611c9c22179be3b2c2e3a26"} Jul 10 18:39:50.233910 ip-10-0-143-17 kubenswrapper[2585]: I0710 18:39:50.233886 2585 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kube-system/konnectivity-agent-x7zf6" event={"ID":"11f2f17b-9176-4b2b-b4a4-836025deed92","Type":"ContainerStarted","Data":"e0d6dd547a09d80d48be6865dd7adf8ecf1e5b37d70269108770c6ffc09bd7fe"} Jul 10 18:39:50.236372 ip-10-0-143-17 kubenswrapper[2585]: I0710 18:39:50.236347 2585 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-cluster-node-tuning-operator/tuned-w6ds9" event={"ID":"fdd15e8f-ae22-4144-b9bb-b4e4f09be270","Type":"ContainerStarted","Data":"23c1f11244644d13f47d28fa359105d553aab495b97e45ee8940f8b3aa783b5f"} Jul 10 18:39:50.237569 ip-10-0-143-17 kubenswrapper[2585]: I0710 18:39:50.237514 2585 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ovn-kubernetes/ovnkube-node-xg6bm" event={"ID":"d863cfea-f2f3-4404-b62a-d41785525f3f","Type":"ContainerStarted","Data":"0f71fd4cd39c61b32698ca609ea529d23e8dbbcb4adc55cbcae4bee584714dc2"} Jul 10 18:39:50.238711 ip-10-0-143-17 kubenswrapper[2585]: I0710 18:39:50.238688 2585 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-network-operator/iptables-alerter-m5446" event={"ID":"03ab0623-6f5f-49c6-b999-6dc99fc53667","Type":"ContainerStarted","Data":"92f7f571b6f737933e24766efcbf7a9b8414a9fee1b6b679c9b0504fbefb74da"} Jul 10 18:39:50.239881 ip-10-0-143-17 kubenswrapper[2585]: I0710 18:39:50.239845 2585 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-multus/multus-h4nv5" event={"ID":"d0da579f-0565-4704-942a-402e5bf46135","Type":"ContainerStarted","Data":"6fc9bbbe7f1a0dc90ca02e7fc2b72fe28eb84729c92d61f2afe8ffb8674540af"} Jul 10 18:39:50.241146 ip-10-0-143-17 kubenswrapper[2585]: I0710 18:39:50.241123 2585 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-multus/multus-additional-cni-plugins-w4cxp" event={"ID":"c3be1d6d-b0aa-4613-9d9a-e308693cc790","Type":"ContainerStarted","Data":"b4c417da33ee50b45ef323a03cc9606d698d820e33c9768f55cf2617440eec5e"} Jul 10 18:39:50.602921 ip-10-0-143-17 kubenswrapper[2585]: I0710 18:39:50.602513 2585 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"original-pull-secret\" (UniqueName: \"kubernetes.io/secret/15a0ea9c-d294-4065-aed5-a182a76a7d7f-original-pull-secret\") pod \"global-pull-secret-syncer-hhpnn\" (UID: \"15a0ea9c-d294-4065-aed5-a182a76a7d7f\") " pod="kube-system/global-pull-secret-syncer-hhpnn" Jul 10 18:39:50.602921 ip-10-0-143-17 kubenswrapper[2585]: E0710 18:39:50.602784 2585 secret.go:189] Couldn't get secret kube-system/original-pull-secret: object "kube-system"/"original-pull-secret" not registered Jul 10 18:39:50.603123 ip-10-0-143-17 kubenswrapper[2585]: E0710 18:39:50.602968 2585 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/15a0ea9c-d294-4065-aed5-a182a76a7d7f-original-pull-secret podName:15a0ea9c-d294-4065-aed5-a182a76a7d7f nodeName:}" failed. No retries permitted until 2026-07-10 18:39:52.602945222 +0000 UTC m=+6.060305151 (durationBeforeRetry 2s). Error: MountVolume.SetUp failed for volume "original-pull-secret" (UniqueName: "kubernetes.io/secret/15a0ea9c-d294-4065-aed5-a182a76a7d7f-original-pull-secret") pod "global-pull-secret-syncer-hhpnn" (UID: "15a0ea9c-d294-4065-aed5-a182a76a7d7f") : object "kube-system"/"original-pull-secret" not registered Jul 10 18:39:50.703381 ip-10-0-143-17 kubenswrapper[2585]: I0710 18:39:50.703341 2585 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/6516579e-a521-4350-bc6b-c4d698b2f845-metrics-certs\") pod \"network-metrics-daemon-7rwwk\" (UID: \"6516579e-a521-4350-bc6b-c4d698b2f845\") " pod="openshift-multus/network-metrics-daemon-7rwwk" Jul 10 18:39:50.703594 ip-10-0-143-17 kubenswrapper[2585]: E0710 18:39:50.703506 2585 secret.go:189] Couldn't get secret openshift-multus/metrics-daemon-secret: object "openshift-multus"/"metrics-daemon-secret" not registered Jul 10 18:39:50.703671 ip-10-0-143-17 kubenswrapper[2585]: E0710 18:39:50.703614 2585 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/6516579e-a521-4350-bc6b-c4d698b2f845-metrics-certs podName:6516579e-a521-4350-bc6b-c4d698b2f845 nodeName:}" failed. No retries permitted until 2026-07-10 18:39:52.703592801 +0000 UTC m=+6.160952730 (durationBeforeRetry 2s). Error: MountVolume.SetUp failed for volume "metrics-certs" (UniqueName: "kubernetes.io/secret/6516579e-a521-4350-bc6b-c4d698b2f845-metrics-certs") pod "network-metrics-daemon-7rwwk" (UID: "6516579e-a521-4350-bc6b-c4d698b2f845") : object "openshift-multus"/"metrics-daemon-secret" not registered Jul 10 18:39:50.804189 ip-10-0-143-17 kubenswrapper[2585]: I0710 18:39:50.804146 2585 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-82qmj\" (UniqueName: \"kubernetes.io/projected/2166f472-2dd7-47f0-a15e-72d2abf71118-kube-api-access-82qmj\") pod \"network-check-target-7tsng\" (UID: \"2166f472-2dd7-47f0-a15e-72d2abf71118\") " pod="openshift-network-diagnostics/network-check-target-7tsng" Jul 10 18:39:50.804357 ip-10-0-143-17 kubenswrapper[2585]: E0710 18:39:50.804325 2585 projected.go:291] Couldn't get configMap openshift-network-diagnostics/kube-root-ca.crt: object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered Jul 10 18:39:50.804357 ip-10-0-143-17 kubenswrapper[2585]: E0710 18:39:50.804350 2585 projected.go:291] Couldn't get configMap openshift-network-diagnostics/openshift-service-ca.crt: object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered Jul 10 18:39:50.804459 ip-10-0-143-17 kubenswrapper[2585]: E0710 18:39:50.804364 2585 projected.go:196] Error preparing data for projected volume kube-api-access-82qmj for pod openshift-network-diagnostics/network-check-target-7tsng: [object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered, object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered] Jul 10 18:39:50.804459 ip-10-0-143-17 kubenswrapper[2585]: E0710 18:39:50.804422 2585 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/projected/2166f472-2dd7-47f0-a15e-72d2abf71118-kube-api-access-82qmj podName:2166f472-2dd7-47f0-a15e-72d2abf71118 nodeName:}" failed. No retries permitted until 2026-07-10 18:39:52.804403564 +0000 UTC m=+6.261763479 (durationBeforeRetry 2s). Error: MountVolume.SetUp failed for volume "kube-api-access-82qmj" (UniqueName: "kubernetes.io/projected/2166f472-2dd7-47f0-a15e-72d2abf71118-kube-api-access-82qmj") pod "network-check-target-7tsng" (UID: "2166f472-2dd7-47f0-a15e-72d2abf71118") : [object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered, object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered] Jul 10 18:39:50.982926 ip-10-0-143-17 kubenswrapper[2585]: I0710 18:39:50.982857 2585 certificate_manager.go:715] "Certificate rotation deadline determined" logger="kubernetes.io/kubelet-serving" expiration="2028-07-09 18:34:48 +0000 UTC" deadline="2028-03-18 11:45:25.302010298 +0000 UTC" Jul 10 18:39:50.982926 ip-10-0-143-17 kubenswrapper[2585]: I0710 18:39:50.982921 2585 certificate_manager.go:431] "Waiting for next certificate rotation" logger="kubernetes.io/kubelet-serving" sleep="14801h5m34.319093508s" Jul 10 18:39:51.210688 ip-10-0-143-17 kubenswrapper[2585]: I0710 18:39:51.210656 2585 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-hhpnn" Jul 10 18:39:51.210874 ip-10-0-143-17 kubenswrapper[2585]: E0710 18:39:51.210786 2585 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-hhpnn" podUID="15a0ea9c-d294-4065-aed5-a182a76a7d7f" Jul 10 18:39:51.211651 ip-10-0-143-17 kubenswrapper[2585]: I0710 18:39:51.211630 2585 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-7tsng" Jul 10 18:39:51.211760 ip-10-0-143-17 kubenswrapper[2585]: E0710 18:39:51.211737 2585 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-7tsng" podUID="2166f472-2dd7-47f0-a15e-72d2abf71118" Jul 10 18:39:51.248813 ip-10-0-143-17 kubenswrapper[2585]: I0710 18:39:51.248722 2585 generic.go:356] "Generic (PLEG): container finished" podID="3813618e37881b79cb3d5b62e1c14d5b" containerID="23ef57e1937eefe0e52aaca56cc022bc6c88b7b8e80b8e460f9bfd006002e9a0" exitCode=0 Jul 10 18:39:51.248973 ip-10-0-143-17 kubenswrapper[2585]: I0710 18:39:51.248823 2585 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-143-17.ec2.internal" event={"ID":"3813618e37881b79cb3d5b62e1c14d5b","Type":"ContainerDied","Data":"23ef57e1937eefe0e52aaca56cc022bc6c88b7b8e80b8e460f9bfd006002e9a0"} Jul 10 18:39:51.275202 ip-10-0-143-17 kubenswrapper[2585]: I0710 18:39:51.275158 2585 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kube-system/kube-apiserver-proxy-ip-10-0-143-17.ec2.internal" event={"ID":"23dbc229ee3e12019dd90879f1a0df34","Type":"ContainerStarted","Data":"120c178b260ea715dfb16d77a464b9fa15d5479dc73a917a42c7e2f236ea1fd1"} Jul 10 18:39:52.210823 ip-10-0-143-17 kubenswrapper[2585]: I0710 18:39:52.210788 2585 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-7rwwk" Jul 10 18:39:52.211266 ip-10-0-143-17 kubenswrapper[2585]: E0710 18:39:52.210938 2585 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-7rwwk" podUID="6516579e-a521-4350-bc6b-c4d698b2f845" Jul 10 18:39:52.304451 ip-10-0-143-17 kubenswrapper[2585]: I0710 18:39:52.304368 2585 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-143-17.ec2.internal" event={"ID":"3813618e37881b79cb3d5b62e1c14d5b","Type":"ContainerStarted","Data":"5d7fc21a5299f79562e2d6d680dbe4db9a89cdc5b1398bd9ac661f9c5ba8313f"} Jul 10 18:39:52.321367 ip-10-0-143-17 kubenswrapper[2585]: I0710 18:39:52.320600 2585 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="kube-system/kube-apiserver-proxy-ip-10-0-143-17.ec2.internal" podStartSLOduration=4.320579034 podStartE2EDuration="4.320579034s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=true podCreationTimestamp="2026-07-10 18:39:48 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-10 18:39:51.29098178 +0000 UTC m=+4.748341733" watchObservedRunningTime="2026-07-10 18:39:52.320579034 +0000 UTC m=+5.777938972" Jul 10 18:39:52.618764 ip-10-0-143-17 kubenswrapper[2585]: I0710 18:39:52.618682 2585 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"original-pull-secret\" (UniqueName: \"kubernetes.io/secret/15a0ea9c-d294-4065-aed5-a182a76a7d7f-original-pull-secret\") pod \"global-pull-secret-syncer-hhpnn\" (UID: \"15a0ea9c-d294-4065-aed5-a182a76a7d7f\") " pod="kube-system/global-pull-secret-syncer-hhpnn" Jul 10 18:39:52.618915 ip-10-0-143-17 kubenswrapper[2585]: E0710 18:39:52.618875 2585 secret.go:189] Couldn't get secret kube-system/original-pull-secret: object "kube-system"/"original-pull-secret" not registered Jul 10 18:39:52.618965 ip-10-0-143-17 kubenswrapper[2585]: E0710 18:39:52.618940 2585 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/15a0ea9c-d294-4065-aed5-a182a76a7d7f-original-pull-secret podName:15a0ea9c-d294-4065-aed5-a182a76a7d7f nodeName:}" failed. No retries permitted until 2026-07-10 18:39:56.618919891 +0000 UTC m=+10.076279822 (durationBeforeRetry 4s). Error: MountVolume.SetUp failed for volume "original-pull-secret" (UniqueName: "kubernetes.io/secret/15a0ea9c-d294-4065-aed5-a182a76a7d7f-original-pull-secret") pod "global-pull-secret-syncer-hhpnn" (UID: "15a0ea9c-d294-4065-aed5-a182a76a7d7f") : object "kube-system"/"original-pull-secret" not registered Jul 10 18:39:52.719510 ip-10-0-143-17 kubenswrapper[2585]: I0710 18:39:52.719474 2585 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/6516579e-a521-4350-bc6b-c4d698b2f845-metrics-certs\") pod \"network-metrics-daemon-7rwwk\" (UID: \"6516579e-a521-4350-bc6b-c4d698b2f845\") " pod="openshift-multus/network-metrics-daemon-7rwwk" Jul 10 18:39:52.719718 ip-10-0-143-17 kubenswrapper[2585]: E0710 18:39:52.719700 2585 secret.go:189] Couldn't get secret openshift-multus/metrics-daemon-secret: object "openshift-multus"/"metrics-daemon-secret" not registered Jul 10 18:39:52.719777 ip-10-0-143-17 kubenswrapper[2585]: E0710 18:39:52.719767 2585 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/6516579e-a521-4350-bc6b-c4d698b2f845-metrics-certs podName:6516579e-a521-4350-bc6b-c4d698b2f845 nodeName:}" failed. No retries permitted until 2026-07-10 18:39:56.719750077 +0000 UTC m=+10.177109993 (durationBeforeRetry 4s). Error: MountVolume.SetUp failed for volume "metrics-certs" (UniqueName: "kubernetes.io/secret/6516579e-a521-4350-bc6b-c4d698b2f845-metrics-certs") pod "network-metrics-daemon-7rwwk" (UID: "6516579e-a521-4350-bc6b-c4d698b2f845") : object "openshift-multus"/"metrics-daemon-secret" not registered Jul 10 18:39:52.820488 ip-10-0-143-17 kubenswrapper[2585]: I0710 18:39:52.820446 2585 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-82qmj\" (UniqueName: \"kubernetes.io/projected/2166f472-2dd7-47f0-a15e-72d2abf71118-kube-api-access-82qmj\") pod \"network-check-target-7tsng\" (UID: \"2166f472-2dd7-47f0-a15e-72d2abf71118\") " pod="openshift-network-diagnostics/network-check-target-7tsng" Jul 10 18:39:52.820686 ip-10-0-143-17 kubenswrapper[2585]: E0710 18:39:52.820667 2585 projected.go:291] Couldn't get configMap openshift-network-diagnostics/kube-root-ca.crt: object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered Jul 10 18:39:52.820757 ip-10-0-143-17 kubenswrapper[2585]: E0710 18:39:52.820696 2585 projected.go:291] Couldn't get configMap openshift-network-diagnostics/openshift-service-ca.crt: object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered Jul 10 18:39:52.820757 ip-10-0-143-17 kubenswrapper[2585]: E0710 18:39:52.820710 2585 projected.go:196] Error preparing data for projected volume kube-api-access-82qmj for pod openshift-network-diagnostics/network-check-target-7tsng: [object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered, object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered] Jul 10 18:39:52.820863 ip-10-0-143-17 kubenswrapper[2585]: E0710 18:39:52.820772 2585 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/projected/2166f472-2dd7-47f0-a15e-72d2abf71118-kube-api-access-82qmj podName:2166f472-2dd7-47f0-a15e-72d2abf71118 nodeName:}" failed. No retries permitted until 2026-07-10 18:39:56.820753353 +0000 UTC m=+10.278113267 (durationBeforeRetry 4s). Error: MountVolume.SetUp failed for volume "kube-api-access-82qmj" (UniqueName: "kubernetes.io/projected/2166f472-2dd7-47f0-a15e-72d2abf71118-kube-api-access-82qmj") pod "network-check-target-7tsng" (UID: "2166f472-2dd7-47f0-a15e-72d2abf71118") : [object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered, object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered] Jul 10 18:39:53.210691 ip-10-0-143-17 kubenswrapper[2585]: I0710 18:39:53.210652 2585 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-hhpnn" Jul 10 18:39:53.210869 ip-10-0-143-17 kubenswrapper[2585]: E0710 18:39:53.210797 2585 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-hhpnn" podUID="15a0ea9c-d294-4065-aed5-a182a76a7d7f" Jul 10 18:39:53.211236 ip-10-0-143-17 kubenswrapper[2585]: I0710 18:39:53.211207 2585 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-7tsng" Jul 10 18:39:53.211504 ip-10-0-143-17 kubenswrapper[2585]: E0710 18:39:53.211304 2585 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-7tsng" podUID="2166f472-2dd7-47f0-a15e-72d2abf71118" Jul 10 18:39:54.210578 ip-10-0-143-17 kubenswrapper[2585]: I0710 18:39:54.210525 2585 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-7rwwk" Jul 10 18:39:54.210755 ip-10-0-143-17 kubenswrapper[2585]: E0710 18:39:54.210688 2585 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-7rwwk" podUID="6516579e-a521-4350-bc6b-c4d698b2f845" Jul 10 18:39:55.211214 ip-10-0-143-17 kubenswrapper[2585]: I0710 18:39:55.210963 2585 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-hhpnn" Jul 10 18:39:55.211214 ip-10-0-143-17 kubenswrapper[2585]: I0710 18:39:55.211008 2585 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-7tsng" Jul 10 18:39:55.211214 ip-10-0-143-17 kubenswrapper[2585]: E0710 18:39:55.211100 2585 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-hhpnn" podUID="15a0ea9c-d294-4065-aed5-a182a76a7d7f" Jul 10 18:39:55.211837 ip-10-0-143-17 kubenswrapper[2585]: E0710 18:39:55.211524 2585 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-7tsng" podUID="2166f472-2dd7-47f0-a15e-72d2abf71118" Jul 10 18:39:56.210858 ip-10-0-143-17 kubenswrapper[2585]: I0710 18:39:56.210802 2585 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-7rwwk" Jul 10 18:39:56.211055 ip-10-0-143-17 kubenswrapper[2585]: E0710 18:39:56.210945 2585 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-7rwwk" podUID="6516579e-a521-4350-bc6b-c4d698b2f845" Jul 10 18:39:56.655333 ip-10-0-143-17 kubenswrapper[2585]: I0710 18:39:56.655246 2585 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"original-pull-secret\" (UniqueName: \"kubernetes.io/secret/15a0ea9c-d294-4065-aed5-a182a76a7d7f-original-pull-secret\") pod \"global-pull-secret-syncer-hhpnn\" (UID: \"15a0ea9c-d294-4065-aed5-a182a76a7d7f\") " pod="kube-system/global-pull-secret-syncer-hhpnn" Jul 10 18:39:56.655831 ip-10-0-143-17 kubenswrapper[2585]: E0710 18:39:56.655402 2585 secret.go:189] Couldn't get secret kube-system/original-pull-secret: object "kube-system"/"original-pull-secret" not registered Jul 10 18:39:56.655831 ip-10-0-143-17 kubenswrapper[2585]: E0710 18:39:56.655470 2585 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/15a0ea9c-d294-4065-aed5-a182a76a7d7f-original-pull-secret podName:15a0ea9c-d294-4065-aed5-a182a76a7d7f nodeName:}" failed. No retries permitted until 2026-07-10 18:40:04.655449218 +0000 UTC m=+18.112809144 (durationBeforeRetry 8s). Error: MountVolume.SetUp failed for volume "original-pull-secret" (UniqueName: "kubernetes.io/secret/15a0ea9c-d294-4065-aed5-a182a76a7d7f-original-pull-secret") pod "global-pull-secret-syncer-hhpnn" (UID: "15a0ea9c-d294-4065-aed5-a182a76a7d7f") : object "kube-system"/"original-pull-secret" not registered Jul 10 18:39:56.756280 ip-10-0-143-17 kubenswrapper[2585]: I0710 18:39:56.756231 2585 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/6516579e-a521-4350-bc6b-c4d698b2f845-metrics-certs\") pod \"network-metrics-daemon-7rwwk\" (UID: \"6516579e-a521-4350-bc6b-c4d698b2f845\") " pod="openshift-multus/network-metrics-daemon-7rwwk" Jul 10 18:39:56.756448 ip-10-0-143-17 kubenswrapper[2585]: E0710 18:39:56.756410 2585 secret.go:189] Couldn't get secret openshift-multus/metrics-daemon-secret: object "openshift-multus"/"metrics-daemon-secret" not registered Jul 10 18:39:56.756522 ip-10-0-143-17 kubenswrapper[2585]: E0710 18:39:56.756478 2585 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/6516579e-a521-4350-bc6b-c4d698b2f845-metrics-certs podName:6516579e-a521-4350-bc6b-c4d698b2f845 nodeName:}" failed. No retries permitted until 2026-07-10 18:40:04.756458325 +0000 UTC m=+18.213818256 (durationBeforeRetry 8s). Error: MountVolume.SetUp failed for volume "metrics-certs" (UniqueName: "kubernetes.io/secret/6516579e-a521-4350-bc6b-c4d698b2f845-metrics-certs") pod "network-metrics-daemon-7rwwk" (UID: "6516579e-a521-4350-bc6b-c4d698b2f845") : object "openshift-multus"/"metrics-daemon-secret" not registered Jul 10 18:39:56.857609 ip-10-0-143-17 kubenswrapper[2585]: I0710 18:39:56.857564 2585 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-82qmj\" (UniqueName: \"kubernetes.io/projected/2166f472-2dd7-47f0-a15e-72d2abf71118-kube-api-access-82qmj\") pod \"network-check-target-7tsng\" (UID: \"2166f472-2dd7-47f0-a15e-72d2abf71118\") " pod="openshift-network-diagnostics/network-check-target-7tsng" Jul 10 18:39:56.857802 ip-10-0-143-17 kubenswrapper[2585]: E0710 18:39:56.857741 2585 projected.go:291] Couldn't get configMap openshift-network-diagnostics/kube-root-ca.crt: object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered Jul 10 18:39:56.857802 ip-10-0-143-17 kubenswrapper[2585]: E0710 18:39:56.857761 2585 projected.go:291] Couldn't get configMap openshift-network-diagnostics/openshift-service-ca.crt: object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered Jul 10 18:39:56.857802 ip-10-0-143-17 kubenswrapper[2585]: E0710 18:39:56.857774 2585 projected.go:196] Error preparing data for projected volume kube-api-access-82qmj for pod openshift-network-diagnostics/network-check-target-7tsng: [object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered, object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered] Jul 10 18:39:56.857963 ip-10-0-143-17 kubenswrapper[2585]: E0710 18:39:56.857832 2585 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/projected/2166f472-2dd7-47f0-a15e-72d2abf71118-kube-api-access-82qmj podName:2166f472-2dd7-47f0-a15e-72d2abf71118 nodeName:}" failed. No retries permitted until 2026-07-10 18:40:04.857813935 +0000 UTC m=+18.315173877 (durationBeforeRetry 8s). Error: MountVolume.SetUp failed for volume "kube-api-access-82qmj" (UniqueName: "kubernetes.io/projected/2166f472-2dd7-47f0-a15e-72d2abf71118-kube-api-access-82qmj") pod "network-check-target-7tsng" (UID: "2166f472-2dd7-47f0-a15e-72d2abf71118") : [object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered, object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered] Jul 10 18:39:57.211737 ip-10-0-143-17 kubenswrapper[2585]: I0710 18:39:57.211552 2585 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-hhpnn" Jul 10 18:39:57.211737 ip-10-0-143-17 kubenswrapper[2585]: I0710 18:39:57.211598 2585 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-7tsng" Jul 10 18:39:57.211737 ip-10-0-143-17 kubenswrapper[2585]: E0710 18:39:57.211689 2585 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-hhpnn" podUID="15a0ea9c-d294-4065-aed5-a182a76a7d7f" Jul 10 18:39:57.212022 ip-10-0-143-17 kubenswrapper[2585]: E0710 18:39:57.211803 2585 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-7tsng" podUID="2166f472-2dd7-47f0-a15e-72d2abf71118" Jul 10 18:39:58.210956 ip-10-0-143-17 kubenswrapper[2585]: I0710 18:39:58.210918 2585 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-7rwwk" Jul 10 18:39:58.211342 ip-10-0-143-17 kubenswrapper[2585]: E0710 18:39:58.211077 2585 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-7rwwk" podUID="6516579e-a521-4350-bc6b-c4d698b2f845" Jul 10 18:39:59.210445 ip-10-0-143-17 kubenswrapper[2585]: I0710 18:39:59.210410 2585 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-7tsng" Jul 10 18:39:59.210662 ip-10-0-143-17 kubenswrapper[2585]: E0710 18:39:59.210560 2585 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-7tsng" podUID="2166f472-2dd7-47f0-a15e-72d2abf71118" Jul 10 18:39:59.210793 ip-10-0-143-17 kubenswrapper[2585]: I0710 18:39:59.210770 2585 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-hhpnn" Jul 10 18:39:59.210924 ip-10-0-143-17 kubenswrapper[2585]: E0710 18:39:59.210890 2585 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-hhpnn" podUID="15a0ea9c-d294-4065-aed5-a182a76a7d7f" Jul 10 18:40:00.210363 ip-10-0-143-17 kubenswrapper[2585]: I0710 18:40:00.210329 2585 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-7rwwk" Jul 10 18:40:00.210797 ip-10-0-143-17 kubenswrapper[2585]: E0710 18:40:00.210452 2585 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-7rwwk" podUID="6516579e-a521-4350-bc6b-c4d698b2f845" Jul 10 18:40:01.210546 ip-10-0-143-17 kubenswrapper[2585]: I0710 18:40:01.210497 2585 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-7tsng" Jul 10 18:40:01.210546 ip-10-0-143-17 kubenswrapper[2585]: I0710 18:40:01.210524 2585 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-hhpnn" Jul 10 18:40:01.211043 ip-10-0-143-17 kubenswrapper[2585]: E0710 18:40:01.210659 2585 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-7tsng" podUID="2166f472-2dd7-47f0-a15e-72d2abf71118" Jul 10 18:40:01.211043 ip-10-0-143-17 kubenswrapper[2585]: E0710 18:40:01.210795 2585 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-hhpnn" podUID="15a0ea9c-d294-4065-aed5-a182a76a7d7f" Jul 10 18:40:02.210278 ip-10-0-143-17 kubenswrapper[2585]: I0710 18:40:02.210239 2585 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-7rwwk" Jul 10 18:40:02.210466 ip-10-0-143-17 kubenswrapper[2585]: E0710 18:40:02.210393 2585 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-7rwwk" podUID="6516579e-a521-4350-bc6b-c4d698b2f845" Jul 10 18:40:03.210158 ip-10-0-143-17 kubenswrapper[2585]: I0710 18:40:03.210117 2585 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-hhpnn" Jul 10 18:40:03.210614 ip-10-0-143-17 kubenswrapper[2585]: I0710 18:40:03.210170 2585 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-7tsng" Jul 10 18:40:03.210614 ip-10-0-143-17 kubenswrapper[2585]: E0710 18:40:03.210264 2585 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-hhpnn" podUID="15a0ea9c-d294-4065-aed5-a182a76a7d7f" Jul 10 18:40:03.210614 ip-10-0-143-17 kubenswrapper[2585]: E0710 18:40:03.210394 2585 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-7tsng" podUID="2166f472-2dd7-47f0-a15e-72d2abf71118" Jul 10 18:40:04.210769 ip-10-0-143-17 kubenswrapper[2585]: I0710 18:40:04.210733 2585 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-7rwwk" Jul 10 18:40:04.211251 ip-10-0-143-17 kubenswrapper[2585]: E0710 18:40:04.210881 2585 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-7rwwk" podUID="6516579e-a521-4350-bc6b-c4d698b2f845" Jul 10 18:40:04.720077 ip-10-0-143-17 kubenswrapper[2585]: I0710 18:40:04.720040 2585 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"original-pull-secret\" (UniqueName: \"kubernetes.io/secret/15a0ea9c-d294-4065-aed5-a182a76a7d7f-original-pull-secret\") pod \"global-pull-secret-syncer-hhpnn\" (UID: \"15a0ea9c-d294-4065-aed5-a182a76a7d7f\") " pod="kube-system/global-pull-secret-syncer-hhpnn" Jul 10 18:40:04.720259 ip-10-0-143-17 kubenswrapper[2585]: E0710 18:40:04.720199 2585 secret.go:189] Couldn't get secret kube-system/original-pull-secret: object "kube-system"/"original-pull-secret" not registered Jul 10 18:40:04.720316 ip-10-0-143-17 kubenswrapper[2585]: E0710 18:40:04.720280 2585 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/15a0ea9c-d294-4065-aed5-a182a76a7d7f-original-pull-secret podName:15a0ea9c-d294-4065-aed5-a182a76a7d7f nodeName:}" failed. No retries permitted until 2026-07-10 18:40:20.720258202 +0000 UTC m=+34.177618131 (durationBeforeRetry 16s). Error: MountVolume.SetUp failed for volume "original-pull-secret" (UniqueName: "kubernetes.io/secret/15a0ea9c-d294-4065-aed5-a182a76a7d7f-original-pull-secret") pod "global-pull-secret-syncer-hhpnn" (UID: "15a0ea9c-d294-4065-aed5-a182a76a7d7f") : object "kube-system"/"original-pull-secret" not registered Jul 10 18:40:04.821359 ip-10-0-143-17 kubenswrapper[2585]: I0710 18:40:04.821326 2585 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/6516579e-a521-4350-bc6b-c4d698b2f845-metrics-certs\") pod \"network-metrics-daemon-7rwwk\" (UID: \"6516579e-a521-4350-bc6b-c4d698b2f845\") " pod="openshift-multus/network-metrics-daemon-7rwwk" Jul 10 18:40:04.821568 ip-10-0-143-17 kubenswrapper[2585]: E0710 18:40:04.821463 2585 secret.go:189] Couldn't get secret openshift-multus/metrics-daemon-secret: object "openshift-multus"/"metrics-daemon-secret" not registered Jul 10 18:40:04.821568 ip-10-0-143-17 kubenswrapper[2585]: E0710 18:40:04.821528 2585 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/6516579e-a521-4350-bc6b-c4d698b2f845-metrics-certs podName:6516579e-a521-4350-bc6b-c4d698b2f845 nodeName:}" failed. No retries permitted until 2026-07-10 18:40:20.821511087 +0000 UTC m=+34.278871001 (durationBeforeRetry 16s). Error: MountVolume.SetUp failed for volume "metrics-certs" (UniqueName: "kubernetes.io/secret/6516579e-a521-4350-bc6b-c4d698b2f845-metrics-certs") pod "network-metrics-daemon-7rwwk" (UID: "6516579e-a521-4350-bc6b-c4d698b2f845") : object "openshift-multus"/"metrics-daemon-secret" not registered Jul 10 18:40:04.922600 ip-10-0-143-17 kubenswrapper[2585]: I0710 18:40:04.922569 2585 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-82qmj\" (UniqueName: \"kubernetes.io/projected/2166f472-2dd7-47f0-a15e-72d2abf71118-kube-api-access-82qmj\") pod \"network-check-target-7tsng\" (UID: \"2166f472-2dd7-47f0-a15e-72d2abf71118\") " pod="openshift-network-diagnostics/network-check-target-7tsng" Jul 10 18:40:04.922754 ip-10-0-143-17 kubenswrapper[2585]: E0710 18:40:04.922684 2585 projected.go:291] Couldn't get configMap openshift-network-diagnostics/kube-root-ca.crt: object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered Jul 10 18:40:04.922754 ip-10-0-143-17 kubenswrapper[2585]: E0710 18:40:04.922696 2585 projected.go:291] Couldn't get configMap openshift-network-diagnostics/openshift-service-ca.crt: object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered Jul 10 18:40:04.922754 ip-10-0-143-17 kubenswrapper[2585]: E0710 18:40:04.922704 2585 projected.go:196] Error preparing data for projected volume kube-api-access-82qmj for pod openshift-network-diagnostics/network-check-target-7tsng: [object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered, object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered] Jul 10 18:40:04.922754 ip-10-0-143-17 kubenswrapper[2585]: E0710 18:40:04.922750 2585 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/projected/2166f472-2dd7-47f0-a15e-72d2abf71118-kube-api-access-82qmj podName:2166f472-2dd7-47f0-a15e-72d2abf71118 nodeName:}" failed. No retries permitted until 2026-07-10 18:40:20.922737499 +0000 UTC m=+34.380097411 (durationBeforeRetry 16s). Error: MountVolume.SetUp failed for volume "kube-api-access-82qmj" (UniqueName: "kubernetes.io/projected/2166f472-2dd7-47f0-a15e-72d2abf71118-kube-api-access-82qmj") pod "network-check-target-7tsng" (UID: "2166f472-2dd7-47f0-a15e-72d2abf71118") : [object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered, object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered] Jul 10 18:40:05.210332 ip-10-0-143-17 kubenswrapper[2585]: I0710 18:40:05.210243 2585 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-hhpnn" Jul 10 18:40:05.210492 ip-10-0-143-17 kubenswrapper[2585]: E0710 18:40:05.210385 2585 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-hhpnn" podUID="15a0ea9c-d294-4065-aed5-a182a76a7d7f" Jul 10 18:40:05.210492 ip-10-0-143-17 kubenswrapper[2585]: I0710 18:40:05.210454 2585 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-7tsng" Jul 10 18:40:05.210611 ip-10-0-143-17 kubenswrapper[2585]: E0710 18:40:05.210572 2585 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-7tsng" podUID="2166f472-2dd7-47f0-a15e-72d2abf71118" Jul 10 18:40:06.210324 ip-10-0-143-17 kubenswrapper[2585]: I0710 18:40:06.210287 2585 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-7rwwk" Jul 10 18:40:06.210791 ip-10-0-143-17 kubenswrapper[2585]: E0710 18:40:06.210432 2585 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-7rwwk" podUID="6516579e-a521-4350-bc6b-c4d698b2f845" Jul 10 18:40:07.211899 ip-10-0-143-17 kubenswrapper[2585]: I0710 18:40:07.211799 2585 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-7tsng" Jul 10 18:40:07.212361 ip-10-0-143-17 kubenswrapper[2585]: E0710 18:40:07.211952 2585 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-7tsng" podUID="2166f472-2dd7-47f0-a15e-72d2abf71118" Jul 10 18:40:07.212361 ip-10-0-143-17 kubenswrapper[2585]: I0710 18:40:07.211993 2585 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-hhpnn" Jul 10 18:40:07.212361 ip-10-0-143-17 kubenswrapper[2585]: E0710 18:40:07.212066 2585 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-hhpnn" podUID="15a0ea9c-d294-4065-aed5-a182a76a7d7f" Jul 10 18:40:08.210690 ip-10-0-143-17 kubenswrapper[2585]: I0710 18:40:08.210651 2585 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-7rwwk" Jul 10 18:40:08.210896 ip-10-0-143-17 kubenswrapper[2585]: E0710 18:40:08.210792 2585 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-7rwwk" podUID="6516579e-a521-4350-bc6b-c4d698b2f845" Jul 10 18:40:09.210751 ip-10-0-143-17 kubenswrapper[2585]: I0710 18:40:09.210730 2585 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-hhpnn" Jul 10 18:40:09.211392 ip-10-0-143-17 kubenswrapper[2585]: E0710 18:40:09.211365 2585 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-hhpnn" podUID="15a0ea9c-d294-4065-aed5-a182a76a7d7f" Jul 10 18:40:09.211733 ip-10-0-143-17 kubenswrapper[2585]: I0710 18:40:09.211402 2585 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-7tsng" Jul 10 18:40:09.212000 ip-10-0-143-17 kubenswrapper[2585]: E0710 18:40:09.211966 2585 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-7tsng" podUID="2166f472-2dd7-47f0-a15e-72d2abf71118" Jul 10 18:40:09.338562 ip-10-0-143-17 kubenswrapper[2585]: I0710 18:40:09.338513 2585 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kube-system/konnectivity-agent-x7zf6" event={"ID":"11f2f17b-9176-4b2b-b4a4-836025deed92","Type":"ContainerStarted","Data":"0bef0adacfee9177524fe85da4c23344d84fe3cca873068da9af6044e85e7bda"} Jul 10 18:40:09.339980 ip-10-0-143-17 kubenswrapper[2585]: I0710 18:40:09.339947 2585 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-cluster-node-tuning-operator/tuned-w6ds9" event={"ID":"fdd15e8f-ae22-4144-b9bb-b4e4f09be270","Type":"ContainerStarted","Data":"1cf4c7561293ca25307c8ef3454720277b53474f694962f168eef11865dadef8"} Jul 10 18:40:09.341393 ip-10-0-143-17 kubenswrapper[2585]: I0710 18:40:09.341372 2585 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ovn-kubernetes/ovnkube-node-xg6bm" event={"ID":"d863cfea-f2f3-4404-b62a-d41785525f3f","Type":"ContainerStarted","Data":"1272aa2c31f3111d35536402e7b89684e8d366fad9b444c3a090bf71ff0cbe67"} Jul 10 18:40:09.342953 ip-10-0-143-17 kubenswrapper[2585]: I0710 18:40:09.342928 2585 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-multus/multus-h4nv5" event={"ID":"d0da579f-0565-4704-942a-402e5bf46135","Type":"ContainerStarted","Data":"5ac6adbca64f2237aa84434b6acf67e8e800cdd6a8f0c4d1aadf5b2f8d26178d"} Jul 10 18:40:09.344256 ip-10-0-143-17 kubenswrapper[2585]: I0710 18:40:09.344234 2585 generic.go:356] "Generic (PLEG): container finished" podID="c3be1d6d-b0aa-4613-9d9a-e308693cc790" containerID="953e3e844095dec0048a6f1ae8aac6c9bf3faaf2b6115e2681224747aeaac5f9" exitCode=0 Jul 10 18:40:09.344358 ip-10-0-143-17 kubenswrapper[2585]: I0710 18:40:09.344262 2585 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-multus/multus-additional-cni-plugins-w4cxp" event={"ID":"c3be1d6d-b0aa-4613-9d9a-e308693cc790","Type":"ContainerDied","Data":"953e3e844095dec0048a6f1ae8aac6c9bf3faaf2b6115e2681224747aeaac5f9"} Jul 10 18:40:09.346037 ip-10-0-143-17 kubenswrapper[2585]: I0710 18:40:09.345936 2585 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-cw6g2" event={"ID":"bd56ef17-098b-40ba-8500-8cb05e293dc5","Type":"ContainerStarted","Data":"9de579896790bfe3639adfb5a5eebd1705e32325709fd920b3b13547bb54d068"} Jul 10 18:40:09.347711 ip-10-0-143-17 kubenswrapper[2585]: I0710 18:40:09.347679 2585 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-image-registry/node-ca-crbxd" event={"ID":"ad44fb01-da7b-40ac-906e-5a7badc87f54","Type":"ContainerStarted","Data":"cf4acbe39505d8aa087225f560901b14e99412ba25c16c50cec0d68fdb1a1711"} Jul 10 18:40:09.349074 ip-10-0-143-17 kubenswrapper[2585]: I0710 18:40:09.349054 2585 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-dns/node-resolver-ndwqb" event={"ID":"5a594168-14da-41cb-9871-edbbf08720f7","Type":"ContainerStarted","Data":"0cc301014fd957afd1ebc3bae6eac9983240941f4071b0f55b6e86eafa11fd20"} Jul 10 18:40:09.370878 ip-10-0-143-17 kubenswrapper[2585]: I0710 18:40:09.370825 2585 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-143-17.ec2.internal" podStartSLOduration=21.370806041 podStartE2EDuration="21.370806041s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=true podCreationTimestamp="2026-07-10 18:39:48 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-10 18:39:52.32134912 +0000 UTC m=+5.778709055" watchObservedRunningTime="2026-07-10 18:40:09.370806041 +0000 UTC m=+22.828165979" Jul 10 18:40:09.385160 ip-10-0-143-17 kubenswrapper[2585]: I0710 18:40:09.385119 2585 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="kube-system/konnectivity-agent-x7zf6" podStartSLOduration=11.831861573 podStartE2EDuration="22.385105641s" totalImagesPullingTime="10.553244068s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-10 18:39:47 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-10 18:40:09.38480813 +0000 UTC m=+22.842168064" watchObservedRunningTime="2026-07-10 18:40:09.385105641 +0000 UTC m=+22.842465576" Jul 10 18:40:09.401666 ip-10-0-143-17 kubenswrapper[2585]: I0710 18:40:09.401625 2585 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-cluster-node-tuning-operator/tuned-w6ds9" podStartSLOduration=3.5440741940000002 podStartE2EDuration="22.401611918s" totalImagesPullingTime="18.857537724s" totalInitContainerRuntime="0s" isStatefulPod=true podCreationTimestamp="2026-07-10 18:39:47 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-10 18:40:09.401173541 +0000 UTC m=+22.858533496" watchObservedRunningTime="2026-07-10 18:40:09.401611918 +0000 UTC m=+22.858971853" Jul 10 18:40:09.416014 ip-10-0-143-17 kubenswrapper[2585]: I0710 18:40:09.415961 2585 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-dns/node-resolver-ndwqb" podStartSLOduration=3.567281544 podStartE2EDuration="22.415945443s" totalImagesPullingTime="18.848663899s" totalInitContainerRuntime="0s" isStatefulPod=true podCreationTimestamp="2026-07-10 18:39:47 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-10 18:40:09.415519557 +0000 UTC m=+22.872879491" watchObservedRunningTime="2026-07-10 18:40:09.415945443 +0000 UTC m=+22.873305378" Jul 10 18:40:09.432629 ip-10-0-143-17 kubenswrapper[2585]: I0710 18:40:09.432588 2585 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-multus/multus-h4nv5" podStartSLOduration=3.515593005 podStartE2EDuration="22.432574478s" totalImagesPullingTime="18.916981473s" totalInitContainerRuntime="0s" isStatefulPod=true podCreationTimestamp="2026-07-10 18:39:47 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-10 18:40:09.432085404 +0000 UTC m=+22.889445340" watchObservedRunningTime="2026-07-10 18:40:09.432574478 +0000 UTC m=+22.889934414" Jul 10 18:40:09.450219 ip-10-0-143-17 kubenswrapper[2585]: I0710 18:40:09.450179 2585 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-image-registry/node-ca-crbxd" podStartSLOduration=3.593883548 podStartE2EDuration="22.45016477s" totalImagesPullingTime="18.856281222s" totalInitContainerRuntime="0s" isStatefulPod=true podCreationTimestamp="2026-07-10 18:39:47 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-10 18:40:09.449909699 +0000 UTC m=+22.907269634" watchObservedRunningTime="2026-07-10 18:40:09.45016477 +0000 UTC m=+22.907524713" Jul 10 18:40:10.210552 ip-10-0-143-17 kubenswrapper[2585]: I0710 18:40:10.210307 2585 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-7rwwk" Jul 10 18:40:10.210746 ip-10-0-143-17 kubenswrapper[2585]: E0710 18:40:10.210684 2585 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-7rwwk" podUID="6516579e-a521-4350-bc6b-c4d698b2f845" Jul 10 18:40:10.354988 ip-10-0-143-17 kubenswrapper[2585]: I0710 18:40:10.354920 2585 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ovn-kubernetes_ovnkube-node-xg6bm_d863cfea-f2f3-4404-b62a-d41785525f3f/ovn-acl-logging/0.log" Jul 10 18:40:10.355876 ip-10-0-143-17 kubenswrapper[2585]: I0710 18:40:10.355285 2585 generic.go:356] "Generic (PLEG): container finished" podID="d863cfea-f2f3-4404-b62a-d41785525f3f" containerID="11d5e1e2a255181bb9fae134397b646a0832fdcd35d5f3617b9df6e1f63cc78f" exitCode=1 Jul 10 18:40:10.355876 ip-10-0-143-17 kubenswrapper[2585]: I0710 18:40:10.355384 2585 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ovn-kubernetes/ovnkube-node-xg6bm" event={"ID":"d863cfea-f2f3-4404-b62a-d41785525f3f","Type":"ContainerStarted","Data":"f2a0515c2cf4f6f9a5d3b8985524d4b2b99b2cfd91ea3f363286b45d0796eb4f"} Jul 10 18:40:10.355876 ip-10-0-143-17 kubenswrapper[2585]: I0710 18:40:10.355456 2585 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ovn-kubernetes/ovnkube-node-xg6bm" event={"ID":"d863cfea-f2f3-4404-b62a-d41785525f3f","Type":"ContainerStarted","Data":"49dabf3f114fa717c5c0da1fa69903cc22fa9041d89988898959649cd6a99d36"} Jul 10 18:40:10.355876 ip-10-0-143-17 kubenswrapper[2585]: I0710 18:40:10.355486 2585 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ovn-kubernetes/ovnkube-node-xg6bm" event={"ID":"d863cfea-f2f3-4404-b62a-d41785525f3f","Type":"ContainerStarted","Data":"16188ca414e5f2d8f68581193309a6c9998ba051184d5f5f7bd3eabd2ae046a4"} Jul 10 18:40:10.355876 ip-10-0-143-17 kubenswrapper[2585]: I0710 18:40:10.355515 2585 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ovn-kubernetes/ovnkube-node-xg6bm" event={"ID":"d863cfea-f2f3-4404-b62a-d41785525f3f","Type":"ContainerStarted","Data":"e96bd9523f73b984610a10eb304203e64a246905bf64cf1f0491d391f6481704"} Jul 10 18:40:10.355876 ip-10-0-143-17 kubenswrapper[2585]: I0710 18:40:10.355583 2585 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ovn-kubernetes/ovnkube-node-xg6bm" event={"ID":"d863cfea-f2f3-4404-b62a-d41785525f3f","Type":"ContainerDied","Data":"11d5e1e2a255181bb9fae134397b646a0832fdcd35d5f3617b9df6e1f63cc78f"} Jul 10 18:40:10.734644 ip-10-0-143-17 kubenswrapper[2585]: I0710 18:40:10.734614 2585 plugin_watcher.go:194] "Adding socket path or updating timestamp to desired state cache" path="/var/lib/kubelet/plugins_registry/ebs.csi.aws.com-reg.sock" Jul 10 18:40:11.021300 ip-10-0-143-17 kubenswrapper[2585]: I0710 18:40:11.021211 2585 reconciler.go:161] "OperationExecutor.RegisterPlugin started" plugin={"SocketPath":"/var/lib/kubelet/plugins_registry/ebs.csi.aws.com-reg.sock","Timestamp":"2026-07-10T18:40:10.73463879Z","UUID":"2287bfd6-3ea2-45d4-8a1e-8ae97b9ada5e","Handler":null,"Name":"","Endpoint":""} Jul 10 18:40:11.022990 ip-10-0-143-17 kubenswrapper[2585]: I0710 18:40:11.022962 2585 csi_plugin.go:106] kubernetes.io/csi: Trying to validate a new CSI Driver with name: ebs.csi.aws.com endpoint: /var/lib/kubelet/plugins/ebs.csi.aws.com/csi.sock versions: 1.0.0 Jul 10 18:40:11.023125 ip-10-0-143-17 kubenswrapper[2585]: I0710 18:40:11.023001 2585 csi_plugin.go:119] kubernetes.io/csi: Register new plugin with name: ebs.csi.aws.com at endpoint: /var/lib/kubelet/plugins/ebs.csi.aws.com/csi.sock Jul 10 18:40:11.210863 ip-10-0-143-17 kubenswrapper[2585]: I0710 18:40:11.210830 2585 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-7tsng" Jul 10 18:40:11.211039 ip-10-0-143-17 kubenswrapper[2585]: E0710 18:40:11.210959 2585 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-7tsng" podUID="2166f472-2dd7-47f0-a15e-72d2abf71118" Jul 10 18:40:11.211442 ip-10-0-143-17 kubenswrapper[2585]: I0710 18:40:11.211414 2585 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-hhpnn" Jul 10 18:40:11.211592 ip-10-0-143-17 kubenswrapper[2585]: E0710 18:40:11.211529 2585 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-hhpnn" podUID="15a0ea9c-d294-4065-aed5-a182a76a7d7f" Jul 10 18:40:11.319732 ip-10-0-143-17 kubenswrapper[2585]: I0710 18:40:11.319665 2585 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="kube-system/konnectivity-agent-x7zf6" Jul 10 18:40:11.320515 ip-10-0-143-17 kubenswrapper[2585]: I0710 18:40:11.320457 2585 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="kube-system/konnectivity-agent-x7zf6" Jul 10 18:40:11.360590 ip-10-0-143-17 kubenswrapper[2585]: I0710 18:40:11.360560 2585 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-cw6g2" event={"ID":"bd56ef17-098b-40ba-8500-8cb05e293dc5","Type":"ContainerStarted","Data":"68eae1c0bed181b7d88e17cb5e8394eff5f8ce758fca668ba547d773b98b6d7b"} Jul 10 18:40:11.362453 ip-10-0-143-17 kubenswrapper[2585]: I0710 18:40:11.362425 2585 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-network-operator/iptables-alerter-m5446" event={"ID":"03ab0623-6f5f-49c6-b999-6dc99fc53667","Type":"ContainerStarted","Data":"49c1a6f3a3e74dd4060c818546d9893f3d55e2eeb46621a983fe169d0a2cf49d"} Jul 10 18:40:11.362816 ip-10-0-143-17 kubenswrapper[2585]: I0710 18:40:11.362772 2585 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="kube-system/konnectivity-agent-x7zf6" Jul 10 18:40:11.363382 ip-10-0-143-17 kubenswrapper[2585]: I0710 18:40:11.363364 2585 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="kube-system/konnectivity-agent-x7zf6" Jul 10 18:40:11.382053 ip-10-0-143-17 kubenswrapper[2585]: I0710 18:40:11.382012 2585 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-network-operator/iptables-alerter-m5446" podStartSLOduration=5.525780414 podStartE2EDuration="24.381997001s" totalImagesPullingTime="18.856216587s" totalInitContainerRuntime="0s" isStatefulPod=true podCreationTimestamp="2026-07-10 18:39:47 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-10 18:40:11.381451195 +0000 UTC m=+24.838811130" watchObservedRunningTime="2026-07-10 18:40:11.381997001 +0000 UTC m=+24.839356936" Jul 10 18:40:12.210504 ip-10-0-143-17 kubenswrapper[2585]: I0710 18:40:12.210422 2585 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-7rwwk" Jul 10 18:40:12.210785 ip-10-0-143-17 kubenswrapper[2585]: E0710 18:40:12.210576 2585 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-7rwwk" podUID="6516579e-a521-4350-bc6b-c4d698b2f845" Jul 10 18:40:12.366954 ip-10-0-143-17 kubenswrapper[2585]: I0710 18:40:12.366916 2585 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-cw6g2" event={"ID":"bd56ef17-098b-40ba-8500-8cb05e293dc5","Type":"ContainerStarted","Data":"e952efb55d6c5877fdb532167fda0fe5c3696eb3d29fea3db40f95fd1b9e1449"} Jul 10 18:40:12.389665 ip-10-0-143-17 kubenswrapper[2585]: I0710 18:40:12.389613 2585 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-cw6g2" podStartSLOduration=4.218316942 podStartE2EDuration="25.389598501s" totalImagesPullingTime="21.171281559s" totalInitContainerRuntime="0s" isStatefulPod=true podCreationTimestamp="2026-07-10 18:39:47 +0000 UTC" imagePullSessionsCount=3 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-10 18:40:12.388045934 +0000 UTC m=+25.845405879" watchObservedRunningTime="2026-07-10 18:40:12.389598501 +0000 UTC m=+25.846958434" Jul 10 18:40:13.211250 ip-10-0-143-17 kubenswrapper[2585]: I0710 18:40:13.211062 2585 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-hhpnn" Jul 10 18:40:13.211441 ip-10-0-143-17 kubenswrapper[2585]: I0710 18:40:13.211106 2585 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-7tsng" Jul 10 18:40:13.211441 ip-10-0-143-17 kubenswrapper[2585]: E0710 18:40:13.211359 2585 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-hhpnn" podUID="15a0ea9c-d294-4065-aed5-a182a76a7d7f" Jul 10 18:40:13.211441 ip-10-0-143-17 kubenswrapper[2585]: E0710 18:40:13.211402 2585 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-7tsng" podUID="2166f472-2dd7-47f0-a15e-72d2abf71118" Jul 10 18:40:14.210214 ip-10-0-143-17 kubenswrapper[2585]: I0710 18:40:14.210160 2585 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-7rwwk" Jul 10 18:40:14.210749 ip-10-0-143-17 kubenswrapper[2585]: E0710 18:40:14.210264 2585 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-7rwwk" podUID="6516579e-a521-4350-bc6b-c4d698b2f845" Jul 10 18:40:14.374827 ip-10-0-143-17 kubenswrapper[2585]: I0710 18:40:14.374805 2585 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ovn-kubernetes_ovnkube-node-xg6bm_d863cfea-f2f3-4404-b62a-d41785525f3f/ovn-acl-logging/0.log" Jul 10 18:40:14.375189 ip-10-0-143-17 kubenswrapper[2585]: I0710 18:40:14.375165 2585 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ovn-kubernetes/ovnkube-node-xg6bm" event={"ID":"d863cfea-f2f3-4404-b62a-d41785525f3f","Type":"ContainerStarted","Data":"55c8c42c99a3f675cb6f7d6c877829ef7e61a700dcdcc74f80532c77b620d5a1"} Jul 10 18:40:14.376750 ip-10-0-143-17 kubenswrapper[2585]: I0710 18:40:14.376733 2585 generic.go:356] "Generic (PLEG): container finished" podID="c3be1d6d-b0aa-4613-9d9a-e308693cc790" containerID="36735b250f4556330b2598b7f8349f13805e7cf81548ee2bd7ca5b16981fcdd3" exitCode=0 Jul 10 18:40:14.376850 ip-10-0-143-17 kubenswrapper[2585]: I0710 18:40:14.376769 2585 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-multus/multus-additional-cni-plugins-w4cxp" event={"ID":"c3be1d6d-b0aa-4613-9d9a-e308693cc790","Type":"ContainerDied","Data":"36735b250f4556330b2598b7f8349f13805e7cf81548ee2bd7ca5b16981fcdd3"} Jul 10 18:40:15.210746 ip-10-0-143-17 kubenswrapper[2585]: I0710 18:40:15.210722 2585 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-hhpnn" Jul 10 18:40:15.211081 ip-10-0-143-17 kubenswrapper[2585]: E0710 18:40:15.210817 2585 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-hhpnn" podUID="15a0ea9c-d294-4065-aed5-a182a76a7d7f" Jul 10 18:40:15.211081 ip-10-0-143-17 kubenswrapper[2585]: I0710 18:40:15.210856 2585 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-7tsng" Jul 10 18:40:15.211081 ip-10-0-143-17 kubenswrapper[2585]: E0710 18:40:15.210904 2585 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-7tsng" podUID="2166f472-2dd7-47f0-a15e-72d2abf71118" Jul 10 18:40:16.210941 ip-10-0-143-17 kubenswrapper[2585]: I0710 18:40:16.210783 2585 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-7rwwk" Jul 10 18:40:16.211274 ip-10-0-143-17 kubenswrapper[2585]: E0710 18:40:16.211012 2585 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-7rwwk" podUID="6516579e-a521-4350-bc6b-c4d698b2f845" Jul 10 18:40:16.382434 ip-10-0-143-17 kubenswrapper[2585]: I0710 18:40:16.382376 2585 generic.go:356] "Generic (PLEG): container finished" podID="c3be1d6d-b0aa-4613-9d9a-e308693cc790" containerID="154d08509bdf5b9fce237a534ae668d7e1384cf7faf10aa484509c752fe4b08e" exitCode=0 Jul 10 18:40:16.382527 ip-10-0-143-17 kubenswrapper[2585]: I0710 18:40:16.382455 2585 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-multus/multus-additional-cni-plugins-w4cxp" event={"ID":"c3be1d6d-b0aa-4613-9d9a-e308693cc790","Type":"ContainerDied","Data":"154d08509bdf5b9fce237a534ae668d7e1384cf7faf10aa484509c752fe4b08e"} Jul 10 18:40:16.385956 ip-10-0-143-17 kubenswrapper[2585]: I0710 18:40:16.385939 2585 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ovn-kubernetes_ovnkube-node-xg6bm_d863cfea-f2f3-4404-b62a-d41785525f3f/ovn-acl-logging/0.log" Jul 10 18:40:16.386273 ip-10-0-143-17 kubenswrapper[2585]: I0710 18:40:16.386251 2585 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ovn-kubernetes/ovnkube-node-xg6bm" event={"ID":"d863cfea-f2f3-4404-b62a-d41785525f3f","Type":"ContainerStarted","Data":"b5e0d309474a3097f680aacc25b7377f338f740e3b04255a9eba029505f7d73e"} Jul 10 18:40:16.386515 ip-10-0-143-17 kubenswrapper[2585]: I0710 18:40:16.386502 2585 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ovn-kubernetes/ovnkube-node-xg6bm" Jul 10 18:40:16.386591 ip-10-0-143-17 kubenswrapper[2585]: I0710 18:40:16.386584 2585 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ovn-kubernetes/ovnkube-node-xg6bm" Jul 10 18:40:16.386643 ip-10-0-143-17 kubenswrapper[2585]: I0710 18:40:16.386633 2585 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ovn-kubernetes/ovnkube-node-xg6bm" Jul 10 18:40:16.386722 ip-10-0-143-17 kubenswrapper[2585]: I0710 18:40:16.386706 2585 scope.go:117] "RemoveContainer" containerID="11d5e1e2a255181bb9fae134397b646a0832fdcd35d5f3617b9df6e1f63cc78f" Jul 10 18:40:16.402269 ip-10-0-143-17 kubenswrapper[2585]: I0710 18:40:16.402252 2585 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ovn-kubernetes/ovnkube-node-xg6bm" Jul 10 18:40:16.402389 ip-10-0-143-17 kubenswrapper[2585]: I0710 18:40:16.402376 2585 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ovn-kubernetes/ovnkube-node-xg6bm" Jul 10 18:40:17.211478 ip-10-0-143-17 kubenswrapper[2585]: I0710 18:40:17.211448 2585 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-hhpnn" Jul 10 18:40:17.211899 ip-10-0-143-17 kubenswrapper[2585]: I0710 18:40:17.211512 2585 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-7tsng" Jul 10 18:40:17.211899 ip-10-0-143-17 kubenswrapper[2585]: E0710 18:40:17.211529 2585 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-hhpnn" podUID="15a0ea9c-d294-4065-aed5-a182a76a7d7f" Jul 10 18:40:17.211899 ip-10-0-143-17 kubenswrapper[2585]: E0710 18:40:17.211606 2585 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-7tsng" podUID="2166f472-2dd7-47f0-a15e-72d2abf71118" Jul 10 18:40:17.392608 ip-10-0-143-17 kubenswrapper[2585]: I0710 18:40:17.392584 2585 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ovn-kubernetes_ovnkube-node-xg6bm_d863cfea-f2f3-4404-b62a-d41785525f3f/ovn-acl-logging/0.log" Jul 10 18:40:17.392943 ip-10-0-143-17 kubenswrapper[2585]: I0710 18:40:17.392904 2585 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ovn-kubernetes/ovnkube-node-xg6bm" event={"ID":"d863cfea-f2f3-4404-b62a-d41785525f3f","Type":"ContainerStarted","Data":"5625496a5914d91c01f933307cb1d2a6adbea9bdc4cd414189f8668f7aa9dd00"} Jul 10 18:40:17.824818 ip-10-0-143-17 kubenswrapper[2585]: I0710 18:40:17.824509 2585 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-ovn-kubernetes/ovnkube-node-xg6bm" podStartSLOduration=11.826620467 podStartE2EDuration="30.824487495s" totalImagesPullingTime="18.997867028s" totalInitContainerRuntime="0s" isStatefulPod=true podCreationTimestamp="2026-07-10 18:39:47 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-10 18:40:17.423872495 +0000 UTC m=+30.881232455" watchObservedRunningTime="2026-07-10 18:40:17.824487495 +0000 UTC m=+31.281847430" Jul 10 18:40:17.825501 ip-10-0-143-17 kubenswrapper[2585]: I0710 18:40:17.825480 2585 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-multus/network-metrics-daemon-7rwwk"] Jul 10 18:40:17.825622 ip-10-0-143-17 kubenswrapper[2585]: I0710 18:40:17.825610 2585 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-7rwwk" Jul 10 18:40:17.825716 ip-10-0-143-17 kubenswrapper[2585]: E0710 18:40:17.825699 2585 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-7rwwk" podUID="6516579e-a521-4350-bc6b-c4d698b2f845" Jul 10 18:40:17.827763 ip-10-0-143-17 kubenswrapper[2585]: I0710 18:40:17.827731 2585 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kube-system/global-pull-secret-syncer-hhpnn"] Jul 10 18:40:17.827884 ip-10-0-143-17 kubenswrapper[2585]: I0710 18:40:17.827821 2585 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-hhpnn" Jul 10 18:40:17.827942 ip-10-0-143-17 kubenswrapper[2585]: E0710 18:40:17.827901 2585 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-hhpnn" podUID="15a0ea9c-d294-4065-aed5-a182a76a7d7f" Jul 10 18:40:17.828446 ip-10-0-143-17 kubenswrapper[2585]: I0710 18:40:17.828425 2585 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-network-diagnostics/network-check-target-7tsng"] Jul 10 18:40:17.828558 ip-10-0-143-17 kubenswrapper[2585]: I0710 18:40:17.828510 2585 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-7tsng" Jul 10 18:40:17.828644 ip-10-0-143-17 kubenswrapper[2585]: E0710 18:40:17.828619 2585 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-7tsng" podUID="2166f472-2dd7-47f0-a15e-72d2abf71118" Jul 10 18:40:18.397981 ip-10-0-143-17 kubenswrapper[2585]: I0710 18:40:18.397944 2585 generic.go:356] "Generic (PLEG): container finished" podID="c3be1d6d-b0aa-4613-9d9a-e308693cc790" containerID="120a4ee1c90fafc536d8a102cca03bd5c0cc04d1f47b8e5b54a76b6da25e5c52" exitCode=0 Jul 10 18:40:18.398491 ip-10-0-143-17 kubenswrapper[2585]: I0710 18:40:18.398000 2585 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-multus/multus-additional-cni-plugins-w4cxp" event={"ID":"c3be1d6d-b0aa-4613-9d9a-e308693cc790","Type":"ContainerDied","Data":"120a4ee1c90fafc536d8a102cca03bd5c0cc04d1f47b8e5b54a76b6da25e5c52"} Jul 10 18:40:19.210285 ip-10-0-143-17 kubenswrapper[2585]: I0710 18:40:19.210251 2585 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-7tsng" Jul 10 18:40:19.210424 ip-10-0-143-17 kubenswrapper[2585]: E0710 18:40:19.210360 2585 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-7tsng" podUID="2166f472-2dd7-47f0-a15e-72d2abf71118" Jul 10 18:40:20.210071 ip-10-0-143-17 kubenswrapper[2585]: I0710 18:40:20.210039 2585 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-7rwwk" Jul 10 18:40:20.210621 ip-10-0-143-17 kubenswrapper[2585]: I0710 18:40:20.210039 2585 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-hhpnn" Jul 10 18:40:20.210621 ip-10-0-143-17 kubenswrapper[2585]: E0710 18:40:20.210174 2585 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-7rwwk" podUID="6516579e-a521-4350-bc6b-c4d698b2f845" Jul 10 18:40:20.210621 ip-10-0-143-17 kubenswrapper[2585]: E0710 18:40:20.210223 2585 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-hhpnn" podUID="15a0ea9c-d294-4065-aed5-a182a76a7d7f" Jul 10 18:40:20.750171 ip-10-0-143-17 kubenswrapper[2585]: I0710 18:40:20.750134 2585 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"original-pull-secret\" (UniqueName: \"kubernetes.io/secret/15a0ea9c-d294-4065-aed5-a182a76a7d7f-original-pull-secret\") pod \"global-pull-secret-syncer-hhpnn\" (UID: \"15a0ea9c-d294-4065-aed5-a182a76a7d7f\") " pod="kube-system/global-pull-secret-syncer-hhpnn" Jul 10 18:40:20.750333 ip-10-0-143-17 kubenswrapper[2585]: E0710 18:40:20.750271 2585 secret.go:189] Couldn't get secret kube-system/original-pull-secret: object "kube-system"/"original-pull-secret" not registered Jul 10 18:40:20.750375 ip-10-0-143-17 kubenswrapper[2585]: E0710 18:40:20.750334 2585 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/15a0ea9c-d294-4065-aed5-a182a76a7d7f-original-pull-secret podName:15a0ea9c-d294-4065-aed5-a182a76a7d7f nodeName:}" failed. No retries permitted until 2026-07-10 18:40:52.750315421 +0000 UTC m=+66.207675336 (durationBeforeRetry 32s). Error: MountVolume.SetUp failed for volume "original-pull-secret" (UniqueName: "kubernetes.io/secret/15a0ea9c-d294-4065-aed5-a182a76a7d7f-original-pull-secret") pod "global-pull-secret-syncer-hhpnn" (UID: "15a0ea9c-d294-4065-aed5-a182a76a7d7f") : object "kube-system"/"original-pull-secret" not registered Jul 10 18:40:20.851350 ip-10-0-143-17 kubenswrapper[2585]: I0710 18:40:20.851318 2585 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/6516579e-a521-4350-bc6b-c4d698b2f845-metrics-certs\") pod \"network-metrics-daemon-7rwwk\" (UID: \"6516579e-a521-4350-bc6b-c4d698b2f845\") " pod="openshift-multus/network-metrics-daemon-7rwwk" Jul 10 18:40:20.851492 ip-10-0-143-17 kubenswrapper[2585]: E0710 18:40:20.851454 2585 secret.go:189] Couldn't get secret openshift-multus/metrics-daemon-secret: object "openshift-multus"/"metrics-daemon-secret" not registered Jul 10 18:40:20.851576 ip-10-0-143-17 kubenswrapper[2585]: E0710 18:40:20.851523 2585 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/6516579e-a521-4350-bc6b-c4d698b2f845-metrics-certs podName:6516579e-a521-4350-bc6b-c4d698b2f845 nodeName:}" failed. No retries permitted until 2026-07-10 18:40:52.851503052 +0000 UTC m=+66.308862973 (durationBeforeRetry 32s). Error: MountVolume.SetUp failed for volume "metrics-certs" (UniqueName: "kubernetes.io/secret/6516579e-a521-4350-bc6b-c4d698b2f845-metrics-certs") pod "network-metrics-daemon-7rwwk" (UID: "6516579e-a521-4350-bc6b-c4d698b2f845") : object "openshift-multus"/"metrics-daemon-secret" not registered Jul 10 18:40:20.952083 ip-10-0-143-17 kubenswrapper[2585]: I0710 18:40:20.952058 2585 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-82qmj\" (UniqueName: \"kubernetes.io/projected/2166f472-2dd7-47f0-a15e-72d2abf71118-kube-api-access-82qmj\") pod \"network-check-target-7tsng\" (UID: \"2166f472-2dd7-47f0-a15e-72d2abf71118\") " pod="openshift-network-diagnostics/network-check-target-7tsng" Jul 10 18:40:20.952199 ip-10-0-143-17 kubenswrapper[2585]: E0710 18:40:20.952161 2585 projected.go:291] Couldn't get configMap openshift-network-diagnostics/kube-root-ca.crt: object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered Jul 10 18:40:20.952199 ip-10-0-143-17 kubenswrapper[2585]: E0710 18:40:20.952173 2585 projected.go:291] Couldn't get configMap openshift-network-diagnostics/openshift-service-ca.crt: object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered Jul 10 18:40:20.952199 ip-10-0-143-17 kubenswrapper[2585]: E0710 18:40:20.952182 2585 projected.go:196] Error preparing data for projected volume kube-api-access-82qmj for pod openshift-network-diagnostics/network-check-target-7tsng: [object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered, object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered] Jul 10 18:40:20.952296 ip-10-0-143-17 kubenswrapper[2585]: E0710 18:40:20.952221 2585 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/projected/2166f472-2dd7-47f0-a15e-72d2abf71118-kube-api-access-82qmj podName:2166f472-2dd7-47f0-a15e-72d2abf71118 nodeName:}" failed. No retries permitted until 2026-07-10 18:40:52.952211098 +0000 UTC m=+66.409571011 (durationBeforeRetry 32s). Error: MountVolume.SetUp failed for volume "kube-api-access-82qmj" (UniqueName: "kubernetes.io/projected/2166f472-2dd7-47f0-a15e-72d2abf71118-kube-api-access-82qmj") pod "network-check-target-7tsng" (UID: "2166f472-2dd7-47f0-a15e-72d2abf71118") : [object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered, object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered] Jul 10 18:40:21.211184 ip-10-0-143-17 kubenswrapper[2585]: I0710 18:40:21.211103 2585 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-7tsng" Jul 10 18:40:21.211792 ip-10-0-143-17 kubenswrapper[2585]: E0710 18:40:21.211232 2585 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-7tsng" podUID="2166f472-2dd7-47f0-a15e-72d2abf71118" Jul 10 18:40:21.845734 ip-10-0-143-17 kubenswrapper[2585]: I0710 18:40:21.845663 2585 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-143-17.ec2.internal" event="NodeReady" Jul 10 18:40:21.845874 ip-10-0-143-17 kubenswrapper[2585]: I0710 18:40:21.845799 2585 kubelet_node_status.go:489] "Fast updating node status as it just became ready" Jul 10 18:40:21.890591 ip-10-0-143-17 kubenswrapper[2585]: I0710 18:40:21.890564 2585 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-dns/dns-default-bwks6"] Jul 10 18:40:21.918526 ip-10-0-143-17 kubenswrapper[2585]: I0710 18:40:21.918501 2585 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-ingress-canary/ingress-canary-g7fx6"] Jul 10 18:40:21.918689 ip-10-0-143-17 kubenswrapper[2585]: I0710 18:40:21.918610 2585 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-dns/dns-default-bwks6" Jul 10 18:40:21.921321 ip-10-0-143-17 kubenswrapper[2585]: I0710 18:40:21.921298 2585 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-dns\"/\"dns-default-metrics-tls\"" Jul 10 18:40:21.921436 ip-10-0-143-17 kubenswrapper[2585]: I0710 18:40:21.921334 2585 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-dns\"/\"dns-default\"" Jul 10 18:40:21.921436 ip-10-0-143-17 kubenswrapper[2585]: I0710 18:40:21.921344 2585 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-dns\"/\"dns-dockercfg-4x2b4\"" Jul 10 18:40:21.933702 ip-10-0-143-17 kubenswrapper[2585]: I0710 18:40:21.933681 2585 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress-canary/ingress-canary-g7fx6"] Jul 10 18:40:21.933797 ip-10-0-143-17 kubenswrapper[2585]: I0710 18:40:21.933710 2585 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-dns/dns-default-bwks6"] Jul 10 18:40:21.933851 ip-10-0-143-17 kubenswrapper[2585]: I0710 18:40:21.933804 2585 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress-canary/ingress-canary-g7fx6" Jul 10 18:40:21.936686 ip-10-0-143-17 kubenswrapper[2585]: I0710 18:40:21.936649 2585 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ingress-canary\"/\"openshift-service-ca.crt\"" Jul 10 18:40:21.937668 ip-10-0-143-17 kubenswrapper[2585]: I0710 18:40:21.937639 2585 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress-canary\"/\"default-dockercfg-g2pth\"" Jul 10 18:40:21.937776 ip-10-0-143-17 kubenswrapper[2585]: I0710 18:40:21.937759 2585 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress-canary\"/\"canary-serving-cert\"" Jul 10 18:40:21.938591 ip-10-0-143-17 kubenswrapper[2585]: I0710 18:40:21.938570 2585 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ingress-canary\"/\"kube-root-ca.crt\"" Jul 10 18:40:21.959138 ip-10-0-143-17 kubenswrapper[2585]: I0710 18:40:21.959118 2585 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-5s49j\" (UniqueName: \"kubernetes.io/projected/3c50e2a2-37c9-45b8-9ad0-fc624a151d61-kube-api-access-5s49j\") pod \"dns-default-bwks6\" (UID: \"3c50e2a2-37c9-45b8-9ad0-fc624a151d61\") " pod="openshift-dns/dns-default-bwks6" Jul 10 18:40:21.959234 ip-10-0-143-17 kubenswrapper[2585]: I0710 18:40:21.959154 2585 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"config-volume\" (UniqueName: \"kubernetes.io/configmap/3c50e2a2-37c9-45b8-9ad0-fc624a151d61-config-volume\") pod \"dns-default-bwks6\" (UID: \"3c50e2a2-37c9-45b8-9ad0-fc624a151d61\") " pod="openshift-dns/dns-default-bwks6" Jul 10 18:40:21.959234 ip-10-0-143-17 kubenswrapper[2585]: I0710 18:40:21.959215 2585 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"metrics-tls\" (UniqueName: \"kubernetes.io/secret/3c50e2a2-37c9-45b8-9ad0-fc624a151d61-metrics-tls\") pod \"dns-default-bwks6\" (UID: \"3c50e2a2-37c9-45b8-9ad0-fc624a151d61\") " pod="openshift-dns/dns-default-bwks6" Jul 10 18:40:21.959340 ip-10-0-143-17 kubenswrapper[2585]: I0710 18:40:21.959238 2585 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/3c50e2a2-37c9-45b8-9ad0-fc624a151d61-tmp-dir\") pod \"dns-default-bwks6\" (UID: \"3c50e2a2-37c9-45b8-9ad0-fc624a151d61\") " pod="openshift-dns/dns-default-bwks6" Jul 10 18:40:21.959340 ip-10-0-143-17 kubenswrapper[2585]: I0710 18:40:21.959275 2585 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"cert\" (UniqueName: \"kubernetes.io/secret/8c0d7178-8c0c-4f9c-b534-fe044b79a208-cert\") pod \"ingress-canary-g7fx6\" (UID: \"8c0d7178-8c0c-4f9c-b534-fe044b79a208\") " pod="openshift-ingress-canary/ingress-canary-g7fx6" Jul 10 18:40:21.959409 ip-10-0-143-17 kubenswrapper[2585]: I0710 18:40:21.959354 2585 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-gfnvk\" (UniqueName: \"kubernetes.io/projected/8c0d7178-8c0c-4f9c-b534-fe044b79a208-kube-api-access-gfnvk\") pod \"ingress-canary-g7fx6\" (UID: \"8c0d7178-8c0c-4f9c-b534-fe044b79a208\") " pod="openshift-ingress-canary/ingress-canary-g7fx6" Jul 10 18:40:22.060330 ip-10-0-143-17 kubenswrapper[2585]: I0710 18:40:22.060289 2585 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-5s49j\" (UniqueName: \"kubernetes.io/projected/3c50e2a2-37c9-45b8-9ad0-fc624a151d61-kube-api-access-5s49j\") pod \"dns-default-bwks6\" (UID: \"3c50e2a2-37c9-45b8-9ad0-fc624a151d61\") " pod="openshift-dns/dns-default-bwks6" Jul 10 18:40:22.060459 ip-10-0-143-17 kubenswrapper[2585]: I0710 18:40:22.060333 2585 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"config-volume\" (UniqueName: \"kubernetes.io/configmap/3c50e2a2-37c9-45b8-9ad0-fc624a151d61-config-volume\") pod \"dns-default-bwks6\" (UID: \"3c50e2a2-37c9-45b8-9ad0-fc624a151d61\") " pod="openshift-dns/dns-default-bwks6" Jul 10 18:40:22.060459 ip-10-0-143-17 kubenswrapper[2585]: I0710 18:40:22.060392 2585 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-tls\" (UniqueName: \"kubernetes.io/secret/3c50e2a2-37c9-45b8-9ad0-fc624a151d61-metrics-tls\") pod \"dns-default-bwks6\" (UID: \"3c50e2a2-37c9-45b8-9ad0-fc624a151d61\") " pod="openshift-dns/dns-default-bwks6" Jul 10 18:40:22.060459 ip-10-0-143-17 kubenswrapper[2585]: I0710 18:40:22.060414 2585 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/3c50e2a2-37c9-45b8-9ad0-fc624a151d61-tmp-dir\") pod \"dns-default-bwks6\" (UID: \"3c50e2a2-37c9-45b8-9ad0-fc624a151d61\") " pod="openshift-dns/dns-default-bwks6" Jul 10 18:40:22.060459 ip-10-0-143-17 kubenswrapper[2585]: I0710 18:40:22.060452 2585 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"cert\" (UniqueName: \"kubernetes.io/secret/8c0d7178-8c0c-4f9c-b534-fe044b79a208-cert\") pod \"ingress-canary-g7fx6\" (UID: \"8c0d7178-8c0c-4f9c-b534-fe044b79a208\") " pod="openshift-ingress-canary/ingress-canary-g7fx6" Jul 10 18:40:22.060653 ip-10-0-143-17 kubenswrapper[2585]: I0710 18:40:22.060488 2585 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-gfnvk\" (UniqueName: \"kubernetes.io/projected/8c0d7178-8c0c-4f9c-b534-fe044b79a208-kube-api-access-gfnvk\") pod \"ingress-canary-g7fx6\" (UID: \"8c0d7178-8c0c-4f9c-b534-fe044b79a208\") " pod="openshift-ingress-canary/ingress-canary-g7fx6" Jul 10 18:40:22.060699 ip-10-0-143-17 kubenswrapper[2585]: E0710 18:40:22.060664 2585 secret.go:189] Couldn't get secret openshift-dns/dns-default-metrics-tls: secret "dns-default-metrics-tls" not found Jul 10 18:40:22.060733 ip-10-0-143-17 kubenswrapper[2585]: E0710 18:40:22.060714 2585 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/3c50e2a2-37c9-45b8-9ad0-fc624a151d61-metrics-tls podName:3c50e2a2-37c9-45b8-9ad0-fc624a151d61 nodeName:}" failed. No retries permitted until 2026-07-10 18:40:22.560699938 +0000 UTC m=+36.018059851 (durationBeforeRetry 500ms). Error: MountVolume.SetUp failed for volume "metrics-tls" (UniqueName: "kubernetes.io/secret/3c50e2a2-37c9-45b8-9ad0-fc624a151d61-metrics-tls") pod "dns-default-bwks6" (UID: "3c50e2a2-37c9-45b8-9ad0-fc624a151d61") : secret "dns-default-metrics-tls" not found Jul 10 18:40:22.060841 ip-10-0-143-17 kubenswrapper[2585]: I0710 18:40:22.060823 2585 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/3c50e2a2-37c9-45b8-9ad0-fc624a151d61-tmp-dir\") pod \"dns-default-bwks6\" (UID: \"3c50e2a2-37c9-45b8-9ad0-fc624a151d61\") " pod="openshift-dns/dns-default-bwks6" Jul 10 18:40:22.061038 ip-10-0-143-17 kubenswrapper[2585]: I0710 18:40:22.061015 2585 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"config-volume\" (UniqueName: \"kubernetes.io/configmap/3c50e2a2-37c9-45b8-9ad0-fc624a151d61-config-volume\") pod \"dns-default-bwks6\" (UID: \"3c50e2a2-37c9-45b8-9ad0-fc624a151d61\") " pod="openshift-dns/dns-default-bwks6" Jul 10 18:40:22.065497 ip-10-0-143-17 kubenswrapper[2585]: I0710 18:40:22.065479 2585 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"cert\" (UniqueName: \"kubernetes.io/secret/8c0d7178-8c0c-4f9c-b534-fe044b79a208-cert\") pod \"ingress-canary-g7fx6\" (UID: \"8c0d7178-8c0c-4f9c-b534-fe044b79a208\") " pod="openshift-ingress-canary/ingress-canary-g7fx6" Jul 10 18:40:22.068831 ip-10-0-143-17 kubenswrapper[2585]: I0710 18:40:22.068813 2585 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-5s49j\" (UniqueName: \"kubernetes.io/projected/3c50e2a2-37c9-45b8-9ad0-fc624a151d61-kube-api-access-5s49j\") pod \"dns-default-bwks6\" (UID: \"3c50e2a2-37c9-45b8-9ad0-fc624a151d61\") " pod="openshift-dns/dns-default-bwks6" Jul 10 18:40:22.069129 ip-10-0-143-17 kubenswrapper[2585]: I0710 18:40:22.069110 2585 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-gfnvk\" (UniqueName: \"kubernetes.io/projected/8c0d7178-8c0c-4f9c-b534-fe044b79a208-kube-api-access-gfnvk\") pod \"ingress-canary-g7fx6\" (UID: \"8c0d7178-8c0c-4f9c-b534-fe044b79a208\") " pod="openshift-ingress-canary/ingress-canary-g7fx6" Jul 10 18:40:22.210959 ip-10-0-143-17 kubenswrapper[2585]: I0710 18:40:22.210882 2585 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-hhpnn" Jul 10 18:40:22.211117 ip-10-0-143-17 kubenswrapper[2585]: I0710 18:40:22.210886 2585 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-7rwwk" Jul 10 18:40:22.213826 ip-10-0-143-17 kubenswrapper[2585]: I0710 18:40:22.213806 2585 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"kube-system\"/\"original-pull-secret\"" Jul 10 18:40:22.214319 ip-10-0-143-17 kubenswrapper[2585]: I0710 18:40:22.213808 2585 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-multus\"/\"metrics-daemon-secret\"" Jul 10 18:40:22.214319 ip-10-0-143-17 kubenswrapper[2585]: I0710 18:40:22.213827 2585 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-multus\"/\"metrics-daemon-sa-dockercfg-hzzqm\"" Jul 10 18:40:22.245123 ip-10-0-143-17 kubenswrapper[2585]: I0710 18:40:22.245106 2585 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress-canary/ingress-canary-g7fx6" Jul 10 18:40:22.565423 ip-10-0-143-17 kubenswrapper[2585]: I0710 18:40:22.565394 2585 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-tls\" (UniqueName: \"kubernetes.io/secret/3c50e2a2-37c9-45b8-9ad0-fc624a151d61-metrics-tls\") pod \"dns-default-bwks6\" (UID: \"3c50e2a2-37c9-45b8-9ad0-fc624a151d61\") " pod="openshift-dns/dns-default-bwks6" Jul 10 18:40:22.565562 ip-10-0-143-17 kubenswrapper[2585]: E0710 18:40:22.565520 2585 secret.go:189] Couldn't get secret openshift-dns/dns-default-metrics-tls: secret "dns-default-metrics-tls" not found Jul 10 18:40:22.565611 ip-10-0-143-17 kubenswrapper[2585]: E0710 18:40:22.565585 2585 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/3c50e2a2-37c9-45b8-9ad0-fc624a151d61-metrics-tls podName:3c50e2a2-37c9-45b8-9ad0-fc624a151d61 nodeName:}" failed. No retries permitted until 2026-07-10 18:40:23.565570251 +0000 UTC m=+37.022930163 (durationBeforeRetry 1s). Error: MountVolume.SetUp failed for volume "metrics-tls" (UniqueName: "kubernetes.io/secret/3c50e2a2-37c9-45b8-9ad0-fc624a151d61-metrics-tls") pod "dns-default-bwks6" (UID: "3c50e2a2-37c9-45b8-9ad0-fc624a151d61") : secret "dns-default-metrics-tls" not found Jul 10 18:40:22.696228 ip-10-0-143-17 kubenswrapper[2585]: I0710 18:40:22.696044 2585 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress-canary/ingress-canary-g7fx6"] Jul 10 18:40:22.728355 ip-10-0-143-17 kubenswrapper[2585]: W0710 18:40:22.728324 2585 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod8c0d7178_8c0c_4f9c_b534_fe044b79a208.slice/crio-c7586b49458f7ef7d522b77c9770ac0fcedb443f982a9a459c45780eb94ce142 WatchSource:0}: Error finding container c7586b49458f7ef7d522b77c9770ac0fcedb443f982a9a459c45780eb94ce142: Status 404 returned error can't find the container with id c7586b49458f7ef7d522b77c9770ac0fcedb443f982a9a459c45780eb94ce142 Jul 10 18:40:23.210404 ip-10-0-143-17 kubenswrapper[2585]: I0710 18:40:23.210339 2585 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-7tsng" Jul 10 18:40:23.213221 ip-10-0-143-17 kubenswrapper[2585]: I0710 18:40:23.213197 2585 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-network-diagnostics\"/\"kube-root-ca.crt\"" Jul 10 18:40:23.213341 ip-10-0-143-17 kubenswrapper[2585]: I0710 18:40:23.213211 2585 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-network-diagnostics\"/\"default-dockercfg-s52z2\"" Jul 10 18:40:23.214290 ip-10-0-143-17 kubenswrapper[2585]: I0710 18:40:23.214270 2585 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-network-diagnostics\"/\"openshift-service-ca.crt\"" Jul 10 18:40:23.413597 ip-10-0-143-17 kubenswrapper[2585]: I0710 18:40:23.413564 2585 generic.go:356] "Generic (PLEG): container finished" podID="c3be1d6d-b0aa-4613-9d9a-e308693cc790" containerID="7845c17d5b939ec158b88d840376a4de7cf6f40ac84599224aca509a54aff0d4" exitCode=0 Jul 10 18:40:23.413810 ip-10-0-143-17 kubenswrapper[2585]: I0710 18:40:23.413644 2585 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-multus/multus-additional-cni-plugins-w4cxp" event={"ID":"c3be1d6d-b0aa-4613-9d9a-e308693cc790","Type":"ContainerDied","Data":"7845c17d5b939ec158b88d840376a4de7cf6f40ac84599224aca509a54aff0d4"} Jul 10 18:40:23.414907 ip-10-0-143-17 kubenswrapper[2585]: I0710 18:40:23.414884 2585 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress-canary/ingress-canary-g7fx6" event={"ID":"8c0d7178-8c0c-4f9c-b534-fe044b79a208","Type":"ContainerStarted","Data":"c7586b49458f7ef7d522b77c9770ac0fcedb443f982a9a459c45780eb94ce142"} Jul 10 18:40:23.571889 ip-10-0-143-17 kubenswrapper[2585]: I0710 18:40:23.571850 2585 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-tls\" (UniqueName: \"kubernetes.io/secret/3c50e2a2-37c9-45b8-9ad0-fc624a151d61-metrics-tls\") pod \"dns-default-bwks6\" (UID: \"3c50e2a2-37c9-45b8-9ad0-fc624a151d61\") " pod="openshift-dns/dns-default-bwks6" Jul 10 18:40:23.572031 ip-10-0-143-17 kubenswrapper[2585]: E0710 18:40:23.572006 2585 secret.go:189] Couldn't get secret openshift-dns/dns-default-metrics-tls: secret "dns-default-metrics-tls" not found Jul 10 18:40:23.572136 ip-10-0-143-17 kubenswrapper[2585]: E0710 18:40:23.572077 2585 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/3c50e2a2-37c9-45b8-9ad0-fc624a151d61-metrics-tls podName:3c50e2a2-37c9-45b8-9ad0-fc624a151d61 nodeName:}" failed. No retries permitted until 2026-07-10 18:40:25.572061694 +0000 UTC m=+39.029421608 (durationBeforeRetry 2s). Error: MountVolume.SetUp failed for volume "metrics-tls" (UniqueName: "kubernetes.io/secret/3c50e2a2-37c9-45b8-9ad0-fc624a151d61-metrics-tls") pod "dns-default-bwks6" (UID: "3c50e2a2-37c9-45b8-9ad0-fc624a151d61") : secret "dns-default-metrics-tls" not found Jul 10 18:40:24.421229 ip-10-0-143-17 kubenswrapper[2585]: I0710 18:40:24.421191 2585 generic.go:356] "Generic (PLEG): container finished" podID="c3be1d6d-b0aa-4613-9d9a-e308693cc790" containerID="11583501e0c6c916b1620f7487296bcefeb834e30d946d183454dfe9838a3878" exitCode=0 Jul 10 18:40:24.421690 ip-10-0-143-17 kubenswrapper[2585]: I0710 18:40:24.421250 2585 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-multus/multus-additional-cni-plugins-w4cxp" event={"ID":"c3be1d6d-b0aa-4613-9d9a-e308693cc790","Type":"ContainerDied","Data":"11583501e0c6c916b1620f7487296bcefeb834e30d946d183454dfe9838a3878"} Jul 10 18:40:25.425361 ip-10-0-143-17 kubenswrapper[2585]: I0710 18:40:25.425136 2585 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress-canary/ingress-canary-g7fx6" event={"ID":"8c0d7178-8c0c-4f9c-b534-fe044b79a208","Type":"ContainerStarted","Data":"fdd700168b2e396934f0be9b6dff12ad2272058e80124668b082aa30fe8d654c"} Jul 10 18:40:25.428324 ip-10-0-143-17 kubenswrapper[2585]: I0710 18:40:25.428299 2585 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-multus/multus-additional-cni-plugins-w4cxp" event={"ID":"c3be1d6d-b0aa-4613-9d9a-e308693cc790","Type":"ContainerStarted","Data":"5b013872fece0e5cf4bf62d54aa5203c7ca97009ad821f842620ca2311a9e3ab"} Jul 10 18:40:25.441387 ip-10-0-143-17 kubenswrapper[2585]: I0710 18:40:25.441348 2585 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-ingress-canary/ingress-canary-g7fx6" podStartSLOduration=2.3459830999999998 podStartE2EDuration="4.441336631s" totalImagesPullingTime="2.095353531s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-10 18:40:21 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-10 18:40:25.441270093 +0000 UTC m=+38.898630050" watchObservedRunningTime="2026-07-10 18:40:25.441336631 +0000 UTC m=+38.898696556" Jul 10 18:40:25.463170 ip-10-0-143-17 kubenswrapper[2585]: I0710 18:40:25.463136 2585 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-multus/multus-additional-cni-plugins-w4cxp" podStartSLOduration=8.103688473 podStartE2EDuration="38.463123559s" totalImagesPullingTime="30.359435086s" totalInitContainerRuntime="0s" isStatefulPod=true podCreationTimestamp="2026-07-10 18:39:47 +0000 UTC" imagePullSessionsCount=5 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-10 18:40:25.463022611 +0000 UTC m=+38.920382556" watchObservedRunningTime="2026-07-10 18:40:25.463123559 +0000 UTC m=+38.920483488" Jul 10 18:40:25.585931 ip-10-0-143-17 kubenswrapper[2585]: I0710 18:40:25.585907 2585 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-tls\" (UniqueName: \"kubernetes.io/secret/3c50e2a2-37c9-45b8-9ad0-fc624a151d61-metrics-tls\") pod \"dns-default-bwks6\" (UID: \"3c50e2a2-37c9-45b8-9ad0-fc624a151d61\") " pod="openshift-dns/dns-default-bwks6" Jul 10 18:40:25.586053 ip-10-0-143-17 kubenswrapper[2585]: E0710 18:40:25.586017 2585 secret.go:189] Couldn't get secret openshift-dns/dns-default-metrics-tls: secret "dns-default-metrics-tls" not found Jul 10 18:40:25.586115 ip-10-0-143-17 kubenswrapper[2585]: E0710 18:40:25.586086 2585 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/3c50e2a2-37c9-45b8-9ad0-fc624a151d61-metrics-tls podName:3c50e2a2-37c9-45b8-9ad0-fc624a151d61 nodeName:}" failed. No retries permitted until 2026-07-10 18:40:29.586068394 +0000 UTC m=+43.043428322 (durationBeforeRetry 4s). Error: MountVolume.SetUp failed for volume "metrics-tls" (UniqueName: "kubernetes.io/secret/3c50e2a2-37c9-45b8-9ad0-fc624a151d61-metrics-tls") pod "dns-default-bwks6" (UID: "3c50e2a2-37c9-45b8-9ad0-fc624a151d61") : secret "dns-default-metrics-tls" not found Jul 10 18:40:29.614145 ip-10-0-143-17 kubenswrapper[2585]: I0710 18:40:29.614111 2585 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-tls\" (UniqueName: \"kubernetes.io/secret/3c50e2a2-37c9-45b8-9ad0-fc624a151d61-metrics-tls\") pod \"dns-default-bwks6\" (UID: \"3c50e2a2-37c9-45b8-9ad0-fc624a151d61\") " pod="openshift-dns/dns-default-bwks6" Jul 10 18:40:29.614623 ip-10-0-143-17 kubenswrapper[2585]: E0710 18:40:29.614229 2585 secret.go:189] Couldn't get secret openshift-dns/dns-default-metrics-tls: secret "dns-default-metrics-tls" not found Jul 10 18:40:29.614623 ip-10-0-143-17 kubenswrapper[2585]: E0710 18:40:29.614279 2585 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/3c50e2a2-37c9-45b8-9ad0-fc624a151d61-metrics-tls podName:3c50e2a2-37c9-45b8-9ad0-fc624a151d61 nodeName:}" failed. No retries permitted until 2026-07-10 18:40:37.614264721 +0000 UTC m=+51.071624634 (durationBeforeRetry 8s). Error: MountVolume.SetUp failed for volume "metrics-tls" (UniqueName: "kubernetes.io/secret/3c50e2a2-37c9-45b8-9ad0-fc624a151d61-metrics-tls") pod "dns-default-bwks6" (UID: "3c50e2a2-37c9-45b8-9ad0-fc624a151d61") : secret "dns-default-metrics-tls" not found Jul 10 18:40:37.667680 ip-10-0-143-17 kubenswrapper[2585]: I0710 18:40:37.667643 2585 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-tls\" (UniqueName: \"kubernetes.io/secret/3c50e2a2-37c9-45b8-9ad0-fc624a151d61-metrics-tls\") pod \"dns-default-bwks6\" (UID: \"3c50e2a2-37c9-45b8-9ad0-fc624a151d61\") " pod="openshift-dns/dns-default-bwks6" Jul 10 18:40:37.668029 ip-10-0-143-17 kubenswrapper[2585]: E0710 18:40:37.667749 2585 secret.go:189] Couldn't get secret openshift-dns/dns-default-metrics-tls: secret "dns-default-metrics-tls" not found Jul 10 18:40:37.668029 ip-10-0-143-17 kubenswrapper[2585]: E0710 18:40:37.667797 2585 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/3c50e2a2-37c9-45b8-9ad0-fc624a151d61-metrics-tls podName:3c50e2a2-37c9-45b8-9ad0-fc624a151d61 nodeName:}" failed. No retries permitted until 2026-07-10 18:40:53.667784445 +0000 UTC m=+67.125144358 (durationBeforeRetry 16s). Error: MountVolume.SetUp failed for volume "metrics-tls" (UniqueName: "kubernetes.io/secret/3c50e2a2-37c9-45b8-9ad0-fc624a151d61-metrics-tls") pod "dns-default-bwks6" (UID: "3c50e2a2-37c9-45b8-9ad0-fc624a151d61") : secret "dns-default-metrics-tls" not found Jul 10 18:40:48.408603 ip-10-0-143-17 kubenswrapper[2585]: I0710 18:40:48.408574 2585 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ovn-kubernetes/ovnkube-node-xg6bm" Jul 10 18:40:52.764815 ip-10-0-143-17 kubenswrapper[2585]: I0710 18:40:52.764766 2585 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"original-pull-secret\" (UniqueName: \"kubernetes.io/secret/15a0ea9c-d294-4065-aed5-a182a76a7d7f-original-pull-secret\") pod \"global-pull-secret-syncer-hhpnn\" (UID: \"15a0ea9c-d294-4065-aed5-a182a76a7d7f\") " pod="kube-system/global-pull-secret-syncer-hhpnn" Jul 10 18:40:52.767672 ip-10-0-143-17 kubenswrapper[2585]: I0710 18:40:52.767650 2585 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"kube-system\"/\"original-pull-secret\"" Jul 10 18:40:52.779552 ip-10-0-143-17 kubenswrapper[2585]: I0710 18:40:52.779514 2585 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"original-pull-secret\" (UniqueName: \"kubernetes.io/secret/15a0ea9c-d294-4065-aed5-a182a76a7d7f-original-pull-secret\") pod \"global-pull-secret-syncer-hhpnn\" (UID: \"15a0ea9c-d294-4065-aed5-a182a76a7d7f\") " pod="kube-system/global-pull-secret-syncer-hhpnn" Jul 10 18:40:52.822609 ip-10-0-143-17 kubenswrapper[2585]: I0710 18:40:52.822588 2585 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-hhpnn" Jul 10 18:40:52.865379 ip-10-0-143-17 kubenswrapper[2585]: I0710 18:40:52.865347 2585 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/6516579e-a521-4350-bc6b-c4d698b2f845-metrics-certs\") pod \"network-metrics-daemon-7rwwk\" (UID: \"6516579e-a521-4350-bc6b-c4d698b2f845\") " pod="openshift-multus/network-metrics-daemon-7rwwk" Jul 10 18:40:52.868060 ip-10-0-143-17 kubenswrapper[2585]: I0710 18:40:52.868033 2585 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-multus\"/\"metrics-daemon-secret\"" Jul 10 18:40:52.875594 ip-10-0-143-17 kubenswrapper[2585]: E0710 18:40:52.875570 2585 secret.go:189] Couldn't get secret openshift-multus/metrics-daemon-secret: secret "metrics-daemon-secret" not found Jul 10 18:40:52.875710 ip-10-0-143-17 kubenswrapper[2585]: E0710 18:40:52.875647 2585 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/6516579e-a521-4350-bc6b-c4d698b2f845-metrics-certs podName:6516579e-a521-4350-bc6b-c4d698b2f845 nodeName:}" failed. No retries permitted until 2026-07-10 18:41:56.875630445 +0000 UTC m=+130.332990371 (durationBeforeRetry 1m4s). Error: MountVolume.SetUp failed for volume "metrics-certs" (UniqueName: "kubernetes.io/secret/6516579e-a521-4350-bc6b-c4d698b2f845-metrics-certs") pod "network-metrics-daemon-7rwwk" (UID: "6516579e-a521-4350-bc6b-c4d698b2f845") : secret "metrics-daemon-secret" not found Jul 10 18:40:52.940588 ip-10-0-143-17 kubenswrapper[2585]: I0710 18:40:52.940561 2585 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kube-system/global-pull-secret-syncer-hhpnn"] Jul 10 18:40:52.966555 ip-10-0-143-17 kubenswrapper[2585]: I0710 18:40:52.966513 2585 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-82qmj\" (UniqueName: \"kubernetes.io/projected/2166f472-2dd7-47f0-a15e-72d2abf71118-kube-api-access-82qmj\") pod \"network-check-target-7tsng\" (UID: \"2166f472-2dd7-47f0-a15e-72d2abf71118\") " pod="openshift-network-diagnostics/network-check-target-7tsng" Jul 10 18:40:52.969410 ip-10-0-143-17 kubenswrapper[2585]: I0710 18:40:52.969393 2585 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-network-diagnostics\"/\"kube-root-ca.crt\"" Jul 10 18:40:52.980285 ip-10-0-143-17 kubenswrapper[2585]: I0710 18:40:52.980270 2585 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-network-diagnostics\"/\"openshift-service-ca.crt\"" Jul 10 18:40:52.990615 ip-10-0-143-17 kubenswrapper[2585]: I0710 18:40:52.990597 2585 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-82qmj\" (UniqueName: \"kubernetes.io/projected/2166f472-2dd7-47f0-a15e-72d2abf71118-kube-api-access-82qmj\") pod \"network-check-target-7tsng\" (UID: \"2166f472-2dd7-47f0-a15e-72d2abf71118\") " pod="openshift-network-diagnostics/network-check-target-7tsng" Jul 10 18:40:53.119237 ip-10-0-143-17 kubenswrapper[2585]: I0710 18:40:53.119188 2585 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-insights/insights-operator-68d9dd94cf-57mwk"] Jul 10 18:40:53.121787 ip-10-0-143-17 kubenswrapper[2585]: I0710 18:40:53.121772 2585 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-insights/insights-operator-68d9dd94cf-57mwk" Jul 10 18:40:53.124648 ip-10-0-143-17 kubenswrapper[2585]: I0710 18:40:53.124628 2585 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-insights\"/\"openshift-service-ca.crt\"" Jul 10 18:40:53.124648 ip-10-0-143-17 kubenswrapper[2585]: I0710 18:40:53.124642 2585 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-insights\"/\"operator-dockercfg-b2gff\"" Jul 10 18:40:53.124761 ip-10-0-143-17 kubenswrapper[2585]: I0710 18:40:53.124628 2585 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-insights\"/\"service-ca-bundle\"" Jul 10 18:40:53.125733 ip-10-0-143-17 kubenswrapper[2585]: I0710 18:40:53.125715 2585 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-insights\"/\"openshift-insights-serving-cert\"" Jul 10 18:40:53.125820 ip-10-0-143-17 kubenswrapper[2585]: I0710 18:40:53.125803 2585 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-insights\"/\"kube-root-ca.crt\"" Jul 10 18:40:53.130172 ip-10-0-143-17 kubenswrapper[2585]: I0710 18:40:53.130156 2585 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-insights\"/\"trusted-ca-bundle\"" Jul 10 18:40:53.133024 ip-10-0-143-17 kubenswrapper[2585]: I0710 18:40:53.133004 2585 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-insights/insights-operator-68d9dd94cf-57mwk"] Jul 10 18:40:53.167808 ip-10-0-143-17 kubenswrapper[2585]: I0710 18:40:53.167790 2585 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/3041e309-0ff4-41d8-9123-a82c98b5dad3-trusted-ca-bundle\") pod \"insights-operator-68d9dd94cf-57mwk\" (UID: \"3041e309-0ff4-41d8-9123-a82c98b5dad3\") " pod="openshift-insights/insights-operator-68d9dd94cf-57mwk" Jul 10 18:40:53.167892 ip-10-0-143-17 kubenswrapper[2585]: I0710 18:40:53.167819 2585 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tmp\" (UniqueName: \"kubernetes.io/empty-dir/3041e309-0ff4-41d8-9123-a82c98b5dad3-tmp\") pod \"insights-operator-68d9dd94cf-57mwk\" (UID: \"3041e309-0ff4-41d8-9123-a82c98b5dad3\") " pod="openshift-insights/insights-operator-68d9dd94cf-57mwk" Jul 10 18:40:53.167892 ip-10-0-143-17 kubenswrapper[2585]: I0710 18:40:53.167843 2585 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"service-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/3041e309-0ff4-41d8-9123-a82c98b5dad3-service-ca-bundle\") pod \"insights-operator-68d9dd94cf-57mwk\" (UID: \"3041e309-0ff4-41d8-9123-a82c98b5dad3\") " pod="openshift-insights/insights-operator-68d9dd94cf-57mwk" Jul 10 18:40:53.167892 ip-10-0-143-17 kubenswrapper[2585]: I0710 18:40:53.167861 2585 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"serving-cert\" (UniqueName: \"kubernetes.io/secret/3041e309-0ff4-41d8-9123-a82c98b5dad3-serving-cert\") pod \"insights-operator-68d9dd94cf-57mwk\" (UID: \"3041e309-0ff4-41d8-9123-a82c98b5dad3\") " pod="openshift-insights/insights-operator-68d9dd94cf-57mwk" Jul 10 18:40:53.167892 ip-10-0-143-17 kubenswrapper[2585]: I0710 18:40:53.167877 2585 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-srl98\" (UniqueName: \"kubernetes.io/projected/3041e309-0ff4-41d8-9123-a82c98b5dad3-kube-api-access-srl98\") pod \"insights-operator-68d9dd94cf-57mwk\" (UID: \"3041e309-0ff4-41d8-9123-a82c98b5dad3\") " pod="openshift-insights/insights-operator-68d9dd94cf-57mwk" Jul 10 18:40:53.168022 ip-10-0-143-17 kubenswrapper[2585]: I0710 18:40:53.167957 2585 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"snapshots\" (UniqueName: \"kubernetes.io/empty-dir/3041e309-0ff4-41d8-9123-a82c98b5dad3-snapshots\") pod \"insights-operator-68d9dd94cf-57mwk\" (UID: \"3041e309-0ff4-41d8-9123-a82c98b5dad3\") " pod="openshift-insights/insights-operator-68d9dd94cf-57mwk" Jul 10 18:40:53.223919 ip-10-0-143-17 kubenswrapper[2585]: I0710 18:40:53.223899 2585 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-network-diagnostics\"/\"default-dockercfg-s52z2\"" Jul 10 18:40:53.230368 ip-10-0-143-17 kubenswrapper[2585]: I0710 18:40:53.230349 2585 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-ingress/router-default-5d998f6f9d-t8frx"] Jul 10 18:40:53.232090 ip-10-0-143-17 kubenswrapper[2585]: I0710 18:40:53.232075 2585 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-7tsng" Jul 10 18:40:53.232159 ip-10-0-143-17 kubenswrapper[2585]: I0710 18:40:53.232100 2585 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/router-default-5d998f6f9d-t8frx" Jul 10 18:40:53.234718 ip-10-0-143-17 kubenswrapper[2585]: I0710 18:40:53.234697 2585 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress\"/\"router-stats-default\"" Jul 10 18:40:53.234845 ip-10-0-143-17 kubenswrapper[2585]: I0710 18:40:53.234827 2585 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ingress\"/\"kube-root-ca.crt\"" Jul 10 18:40:53.234895 ip-10-0-143-17 kubenswrapper[2585]: I0710 18:40:53.234832 2585 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress\"/\"default-ingress-cert\"" Jul 10 18:40:53.235131 ip-10-0-143-17 kubenswrapper[2585]: I0710 18:40:53.235106 2585 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ingress\"/\"openshift-service-ca.crt\"" Jul 10 18:40:53.235231 ip-10-0-143-17 kubenswrapper[2585]: I0710 18:40:53.235214 2585 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress\"/\"router-dockercfg-4w9ln\"" Jul 10 18:40:53.235342 ip-10-0-143-17 kubenswrapper[2585]: I0710 18:40:53.235329 2585 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress\"/\"router-metrics-certs-default\"" Jul 10 18:40:53.235747 ip-10-0-143-17 kubenswrapper[2585]: I0710 18:40:53.235730 2585 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ingress\"/\"service-ca-bundle\"" Jul 10 18:40:53.246575 ip-10-0-143-17 kubenswrapper[2585]: I0710 18:40:53.246552 2585 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/router-default-5d998f6f9d-t8frx"] Jul 10 18:40:53.268628 ip-10-0-143-17 kubenswrapper[2585]: I0710 18:40:53.268594 2585 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tmp\" (UniqueName: \"kubernetes.io/empty-dir/3041e309-0ff4-41d8-9123-a82c98b5dad3-tmp\") pod \"insights-operator-68d9dd94cf-57mwk\" (UID: \"3041e309-0ff4-41d8-9123-a82c98b5dad3\") " pod="openshift-insights/insights-operator-68d9dd94cf-57mwk" Jul 10 18:40:53.268755 ip-10-0-143-17 kubenswrapper[2585]: I0710 18:40:53.268634 2585 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"service-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/3041e309-0ff4-41d8-9123-a82c98b5dad3-service-ca-bundle\") pod \"insights-operator-68d9dd94cf-57mwk\" (UID: \"3041e309-0ff4-41d8-9123-a82c98b5dad3\") " pod="openshift-insights/insights-operator-68d9dd94cf-57mwk" Jul 10 18:40:53.268755 ip-10-0-143-17 kubenswrapper[2585]: I0710 18:40:53.268664 2585 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"serving-cert\" (UniqueName: \"kubernetes.io/secret/3041e309-0ff4-41d8-9123-a82c98b5dad3-serving-cert\") pod \"insights-operator-68d9dd94cf-57mwk\" (UID: \"3041e309-0ff4-41d8-9123-a82c98b5dad3\") " pod="openshift-insights/insights-operator-68d9dd94cf-57mwk" Jul 10 18:40:53.268755 ip-10-0-143-17 kubenswrapper[2585]: I0710 18:40:53.268689 2585 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-srl98\" (UniqueName: \"kubernetes.io/projected/3041e309-0ff4-41d8-9123-a82c98b5dad3-kube-api-access-srl98\") pod \"insights-operator-68d9dd94cf-57mwk\" (UID: \"3041e309-0ff4-41d8-9123-a82c98b5dad3\") " pod="openshift-insights/insights-operator-68d9dd94cf-57mwk" Jul 10 18:40:53.268755 ip-10-0-143-17 kubenswrapper[2585]: I0710 18:40:53.268737 2585 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"snapshots\" (UniqueName: \"kubernetes.io/empty-dir/3041e309-0ff4-41d8-9123-a82c98b5dad3-snapshots\") pod \"insights-operator-68d9dd94cf-57mwk\" (UID: \"3041e309-0ff4-41d8-9123-a82c98b5dad3\") " pod="openshift-insights/insights-operator-68d9dd94cf-57mwk" Jul 10 18:40:53.268972 ip-10-0-143-17 kubenswrapper[2585]: I0710 18:40:53.268798 2585 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/3041e309-0ff4-41d8-9123-a82c98b5dad3-trusted-ca-bundle\") pod \"insights-operator-68d9dd94cf-57mwk\" (UID: \"3041e309-0ff4-41d8-9123-a82c98b5dad3\") " pod="openshift-insights/insights-operator-68d9dd94cf-57mwk" Jul 10 18:40:53.269104 ip-10-0-143-17 kubenswrapper[2585]: I0710 18:40:53.269077 2585 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"tmp\" (UniqueName: \"kubernetes.io/empty-dir/3041e309-0ff4-41d8-9123-a82c98b5dad3-tmp\") pod \"insights-operator-68d9dd94cf-57mwk\" (UID: \"3041e309-0ff4-41d8-9123-a82c98b5dad3\") " pod="openshift-insights/insights-operator-68d9dd94cf-57mwk" Jul 10 18:40:53.269404 ip-10-0-143-17 kubenswrapper[2585]: I0710 18:40:53.269379 2585 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"snapshots\" (UniqueName: \"kubernetes.io/empty-dir/3041e309-0ff4-41d8-9123-a82c98b5dad3-snapshots\") pod \"insights-operator-68d9dd94cf-57mwk\" (UID: \"3041e309-0ff4-41d8-9123-a82c98b5dad3\") " pod="openshift-insights/insights-operator-68d9dd94cf-57mwk" Jul 10 18:40:53.269529 ip-10-0-143-17 kubenswrapper[2585]: I0710 18:40:53.269511 2585 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"service-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/3041e309-0ff4-41d8-9123-a82c98b5dad3-service-ca-bundle\") pod \"insights-operator-68d9dd94cf-57mwk\" (UID: \"3041e309-0ff4-41d8-9123-a82c98b5dad3\") " pod="openshift-insights/insights-operator-68d9dd94cf-57mwk" Jul 10 18:40:53.269756 ip-10-0-143-17 kubenswrapper[2585]: I0710 18:40:53.269737 2585 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/3041e309-0ff4-41d8-9123-a82c98b5dad3-trusted-ca-bundle\") pod \"insights-operator-68d9dd94cf-57mwk\" (UID: \"3041e309-0ff4-41d8-9123-a82c98b5dad3\") " pod="openshift-insights/insights-operator-68d9dd94cf-57mwk" Jul 10 18:40:53.271249 ip-10-0-143-17 kubenswrapper[2585]: I0710 18:40:53.271221 2585 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"serving-cert\" (UniqueName: \"kubernetes.io/secret/3041e309-0ff4-41d8-9123-a82c98b5dad3-serving-cert\") pod \"insights-operator-68d9dd94cf-57mwk\" (UID: \"3041e309-0ff4-41d8-9123-a82c98b5dad3\") " pod="openshift-insights/insights-operator-68d9dd94cf-57mwk" Jul 10 18:40:53.280257 ip-10-0-143-17 kubenswrapper[2585]: I0710 18:40:53.280214 2585 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-srl98\" (UniqueName: \"kubernetes.io/projected/3041e309-0ff4-41d8-9123-a82c98b5dad3-kube-api-access-srl98\") pod \"insights-operator-68d9dd94cf-57mwk\" (UID: \"3041e309-0ff4-41d8-9123-a82c98b5dad3\") " pod="openshift-insights/insights-operator-68d9dd94cf-57mwk" Jul 10 18:40:53.298072 ip-10-0-143-17 kubenswrapper[2585]: I0710 18:40:53.298048 2585 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-cluster-storage-operator/volume-data-source-validator-78495597f6-86v9g"] Jul 10 18:40:53.303161 ip-10-0-143-17 kubenswrapper[2585]: I0710 18:40:53.303138 2585 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-cluster-storage-operator/volume-data-source-validator-78495597f6-86v9g" Jul 10 18:40:53.306050 ip-10-0-143-17 kubenswrapper[2585]: I0710 18:40:53.306027 2585 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-cluster-storage-operator\"/\"openshift-service-ca.crt\"" Jul 10 18:40:53.306403 ip-10-0-143-17 kubenswrapper[2585]: I0710 18:40:53.306384 2585 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-cluster-storage-operator\"/\"volume-data-source-validator-dockercfg-mzkcd\"" Jul 10 18:40:53.306724 ip-10-0-143-17 kubenswrapper[2585]: I0710 18:40:53.306531 2585 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-cluster-storage-operator\"/\"kube-root-ca.crt\"" Jul 10 18:40:53.313000 ip-10-0-143-17 kubenswrapper[2585]: I0710 18:40:53.312981 2585 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-cluster-storage-operator/volume-data-source-validator-78495597f6-86v9g"] Jul 10 18:40:53.329251 ip-10-0-143-17 kubenswrapper[2585]: I0710 18:40:53.328425 2585 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-cluster-samples-operator/cluster-samples-operator-65db858894-74x7s"] Jul 10 18:40:53.331072 ip-10-0-143-17 kubenswrapper[2585]: I0710 18:40:53.330958 2585 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-monitoring/cluster-monitoring-operator-5f7f494dd4-q52h2"] Jul 10 18:40:53.331160 ip-10-0-143-17 kubenswrapper[2585]: I0710 18:40:53.331111 2585 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-cluster-samples-operator/cluster-samples-operator-65db858894-74x7s" Jul 10 18:40:53.333307 ip-10-0-143-17 kubenswrapper[2585]: I0710 18:40:53.333280 2585 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-monitoring/cluster-monitoring-operator-5f7f494dd4-q52h2" Jul 10 18:40:53.334890 ip-10-0-143-17 kubenswrapper[2585]: I0710 18:40:53.334455 2585 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-cluster-samples-operator\"/\"cluster-samples-operator-dockercfg-v9j2d\"" Jul 10 18:40:53.334890 ip-10-0-143-17 kubenswrapper[2585]: I0710 18:40:53.334734 2585 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-cluster-samples-operator\"/\"openshift-service-ca.crt\"" Jul 10 18:40:53.335119 ip-10-0-143-17 kubenswrapper[2585]: I0710 18:40:53.335100 2585 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-cluster-samples-operator\"/\"samples-operator-tls\"" Jul 10 18:40:53.335209 ip-10-0-143-17 kubenswrapper[2585]: I0710 18:40:53.335141 2585 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-cluster-samples-operator\"/\"kube-root-ca.crt\"" Jul 10 18:40:53.336253 ip-10-0-143-17 kubenswrapper[2585]: I0710 18:40:53.336110 2585 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"cluster-monitoring-operator-dockercfg-cfwmv\"" Jul 10 18:40:53.336253 ip-10-0-143-17 kubenswrapper[2585]: I0710 18:40:53.336121 2585 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-monitoring\"/\"openshift-service-ca.crt\"" Jul 10 18:40:53.336253 ip-10-0-143-17 kubenswrapper[2585]: I0710 18:40:53.336154 2585 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-monitoring\"/\"kube-root-ca.crt\"" Jul 10 18:40:53.336467 ip-10-0-143-17 kubenswrapper[2585]: I0710 18:40:53.336403 2585 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"cluster-monitoring-operator-tls\"" Jul 10 18:40:53.338284 ip-10-0-143-17 kubenswrapper[2585]: I0710 18:40:53.338090 2585 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-monitoring\"/\"telemetry-config\"" Jul 10 18:40:53.347592 ip-10-0-143-17 kubenswrapper[2585]: I0710 18:40:53.347570 2585 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-cluster-samples-operator/cluster-samples-operator-65db858894-74x7s"] Jul 10 18:40:53.348843 ip-10-0-143-17 kubenswrapper[2585]: I0710 18:40:53.348738 2585 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-monitoring/cluster-monitoring-operator-5f7f494dd4-q52h2"] Jul 10 18:40:53.369120 ip-10-0-143-17 kubenswrapper[2585]: I0710 18:40:53.369097 2585 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/bf853b15-a7e2-4f66-ace3-b1f1fc9323de-metrics-certs\") pod \"router-default-5d998f6f9d-t8frx\" (UID: \"bf853b15-a7e2-4f66-ace3-b1f1fc9323de\") " pod="openshift-ingress/router-default-5d998f6f9d-t8frx" Jul 10 18:40:53.369221 ip-10-0-143-17 kubenswrapper[2585]: I0710 18:40:53.369140 2585 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-4x5l2\" (UniqueName: \"kubernetes.io/projected/04ed092b-56ca-425f-bc70-cafaec37f312-kube-api-access-4x5l2\") pod \"cluster-samples-operator-65db858894-74x7s\" (UID: \"04ed092b-56ca-425f-bc70-cafaec37f312\") " pod="openshift-cluster-samples-operator/cluster-samples-operator-65db858894-74x7s" Jul 10 18:40:53.369221 ip-10-0-143-17 kubenswrapper[2585]: I0710 18:40:53.369193 2585 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"samples-operator-tls\" (UniqueName: \"kubernetes.io/secret/04ed092b-56ca-425f-bc70-cafaec37f312-samples-operator-tls\") pod \"cluster-samples-operator-65db858894-74x7s\" (UID: \"04ed092b-56ca-425f-bc70-cafaec37f312\") " pod="openshift-cluster-samples-operator/cluster-samples-operator-65db858894-74x7s" Jul 10 18:40:53.369349 ip-10-0-143-17 kubenswrapper[2585]: I0710 18:40:53.369221 2585 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"stats-auth\" (UniqueName: \"kubernetes.io/secret/bf853b15-a7e2-4f66-ace3-b1f1fc9323de-stats-auth\") pod \"router-default-5d998f6f9d-t8frx\" (UID: \"bf853b15-a7e2-4f66-ace3-b1f1fc9323de\") " pod="openshift-ingress/router-default-5d998f6f9d-t8frx" Jul 10 18:40:53.369349 ip-10-0-143-17 kubenswrapper[2585]: I0710 18:40:53.369245 2585 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-v288k\" (UniqueName: \"kubernetes.io/projected/bf853b15-a7e2-4f66-ace3-b1f1fc9323de-kube-api-access-v288k\") pod \"router-default-5d998f6f9d-t8frx\" (UID: \"bf853b15-a7e2-4f66-ace3-b1f1fc9323de\") " pod="openshift-ingress/router-default-5d998f6f9d-t8frx" Jul 10 18:40:53.369349 ip-10-0-143-17 kubenswrapper[2585]: I0710 18:40:53.369299 2585 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"default-certificate\" (UniqueName: \"kubernetes.io/secret/bf853b15-a7e2-4f66-ace3-b1f1fc9323de-default-certificate\") pod \"router-default-5d998f6f9d-t8frx\" (UID: \"bf853b15-a7e2-4f66-ace3-b1f1fc9323de\") " pod="openshift-ingress/router-default-5d998f6f9d-t8frx" Jul 10 18:40:53.369349 ip-10-0-143-17 kubenswrapper[2585]: I0710 18:40:53.369322 2585 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"service-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/bf853b15-a7e2-4f66-ace3-b1f1fc9323de-service-ca-bundle\") pod \"router-default-5d998f6f9d-t8frx\" (UID: \"bf853b15-a7e2-4f66-ace3-b1f1fc9323de\") " pod="openshift-ingress/router-default-5d998f6f9d-t8frx" Jul 10 18:40:53.369570 ip-10-0-143-17 kubenswrapper[2585]: I0710 18:40:53.369358 2585 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-7zb48\" (UniqueName: \"kubernetes.io/projected/aba57a3c-b29f-4038-9ee1-0dcfdf4c5d5f-kube-api-access-7zb48\") pod \"volume-data-source-validator-78495597f6-86v9g\" (UID: \"aba57a3c-b29f-4038-9ee1-0dcfdf4c5d5f\") " pod="openshift-cluster-storage-operator/volume-data-source-validator-78495597f6-86v9g" Jul 10 18:40:53.369846 ip-10-0-143-17 kubenswrapper[2585]: I0710 18:40:53.369827 2585 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-network-diagnostics/network-check-target-7tsng"] Jul 10 18:40:53.372761 ip-10-0-143-17 kubenswrapper[2585]: W0710 18:40:53.372741 2585 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod2166f472_2dd7_47f0_a15e_72d2abf71118.slice/crio-a2d36077f05d50c67079aedd184861e76a9b4e4f057a20f4837075a1c0343e96 WatchSource:0}: Error finding container a2d36077f05d50c67079aedd184861e76a9b4e4f057a20f4837075a1c0343e96: Status 404 returned error can't find the container with id a2d36077f05d50c67079aedd184861e76a9b4e4f057a20f4837075a1c0343e96 Jul 10 18:40:53.430494 ip-10-0-143-17 kubenswrapper[2585]: I0710 18:40:53.430473 2585 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-insights/insights-operator-68d9dd94cf-57mwk" Jul 10 18:40:53.469692 ip-10-0-143-17 kubenswrapper[2585]: I0710 18:40:53.469662 2585 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"default-certificate\" (UniqueName: \"kubernetes.io/secret/bf853b15-a7e2-4f66-ace3-b1f1fc9323de-default-certificate\") pod \"router-default-5d998f6f9d-t8frx\" (UID: \"bf853b15-a7e2-4f66-ace3-b1f1fc9323de\") " pod="openshift-ingress/router-default-5d998f6f9d-t8frx" Jul 10 18:40:53.469817 ip-10-0-143-17 kubenswrapper[2585]: I0710 18:40:53.469703 2585 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"service-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/bf853b15-a7e2-4f66-ace3-b1f1fc9323de-service-ca-bundle\") pod \"router-default-5d998f6f9d-t8frx\" (UID: \"bf853b15-a7e2-4f66-ace3-b1f1fc9323de\") " pod="openshift-ingress/router-default-5d998f6f9d-t8frx" Jul 10 18:40:53.469817 ip-10-0-143-17 kubenswrapper[2585]: I0710 18:40:53.469737 2585 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-9hcvm\" (UniqueName: \"kubernetes.io/projected/45e87367-8dde-4c37-b642-f38ead4d2b04-kube-api-access-9hcvm\") pod \"cluster-monitoring-operator-5f7f494dd4-q52h2\" (UID: \"45e87367-8dde-4c37-b642-f38ead4d2b04\") " pod="openshift-monitoring/cluster-monitoring-operator-5f7f494dd4-q52h2" Jul 10 18:40:53.469817 ip-10-0-143-17 kubenswrapper[2585]: I0710 18:40:53.469762 2585 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-7zb48\" (UniqueName: \"kubernetes.io/projected/aba57a3c-b29f-4038-9ee1-0dcfdf4c5d5f-kube-api-access-7zb48\") pod \"volume-data-source-validator-78495597f6-86v9g\" (UID: \"aba57a3c-b29f-4038-9ee1-0dcfdf4c5d5f\") " pod="openshift-cluster-storage-operator/volume-data-source-validator-78495597f6-86v9g" Jul 10 18:40:53.469817 ip-10-0-143-17 kubenswrapper[2585]: I0710 18:40:53.469791 2585 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/bf853b15-a7e2-4f66-ace3-b1f1fc9323de-metrics-certs\") pod \"router-default-5d998f6f9d-t8frx\" (UID: \"bf853b15-a7e2-4f66-ace3-b1f1fc9323de\") " pod="openshift-ingress/router-default-5d998f6f9d-t8frx" Jul 10 18:40:53.469817 ip-10-0-143-17 kubenswrapper[2585]: I0710 18:40:53.469812 2585 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-4x5l2\" (UniqueName: \"kubernetes.io/projected/04ed092b-56ca-425f-bc70-cafaec37f312-kube-api-access-4x5l2\") pod \"cluster-samples-operator-65db858894-74x7s\" (UID: \"04ed092b-56ca-425f-bc70-cafaec37f312\") " pod="openshift-cluster-samples-operator/cluster-samples-operator-65db858894-74x7s" Jul 10 18:40:53.470082 ip-10-0-143-17 kubenswrapper[2585]: I0710 18:40:53.469829 2585 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"cluster-monitoring-operator-tls\" (UniqueName: \"kubernetes.io/secret/45e87367-8dde-4c37-b642-f38ead4d2b04-cluster-monitoring-operator-tls\") pod \"cluster-monitoring-operator-5f7f494dd4-q52h2\" (UID: \"45e87367-8dde-4c37-b642-f38ead4d2b04\") " pod="openshift-monitoring/cluster-monitoring-operator-5f7f494dd4-q52h2" Jul 10 18:40:53.470082 ip-10-0-143-17 kubenswrapper[2585]: I0710 18:40:53.469862 2585 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"samples-operator-tls\" (UniqueName: \"kubernetes.io/secret/04ed092b-56ca-425f-bc70-cafaec37f312-samples-operator-tls\") pod \"cluster-samples-operator-65db858894-74x7s\" (UID: \"04ed092b-56ca-425f-bc70-cafaec37f312\") " pod="openshift-cluster-samples-operator/cluster-samples-operator-65db858894-74x7s" Jul 10 18:40:53.470082 ip-10-0-143-17 kubenswrapper[2585]: I0710 18:40:53.469879 2585 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"stats-auth\" (UniqueName: \"kubernetes.io/secret/bf853b15-a7e2-4f66-ace3-b1f1fc9323de-stats-auth\") pod \"router-default-5d998f6f9d-t8frx\" (UID: \"bf853b15-a7e2-4f66-ace3-b1f1fc9323de\") " pod="openshift-ingress/router-default-5d998f6f9d-t8frx" Jul 10 18:40:53.470082 ip-10-0-143-17 kubenswrapper[2585]: E0710 18:40:53.469989 2585 secret.go:189] Couldn't get secret openshift-cluster-samples-operator/samples-operator-tls: secret "samples-operator-tls" not found Jul 10 18:40:53.470082 ip-10-0-143-17 kubenswrapper[2585]: E0710 18:40:53.470013 2585 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/configmap/bf853b15-a7e2-4f66-ace3-b1f1fc9323de-service-ca-bundle podName:bf853b15-a7e2-4f66-ace3-b1f1fc9323de nodeName:}" failed. No retries permitted until 2026-07-10 18:40:53.969994212 +0000 UTC m=+67.427354144 (durationBeforeRetry 500ms). Error: MountVolume.SetUp failed for volume "service-ca-bundle" (UniqueName: "kubernetes.io/configmap/bf853b15-a7e2-4f66-ace3-b1f1fc9323de-service-ca-bundle") pod "router-default-5d998f6f9d-t8frx" (UID: "bf853b15-a7e2-4f66-ace3-b1f1fc9323de") : configmap references non-existent config key: service-ca.crt Jul 10 18:40:53.470082 ip-10-0-143-17 kubenswrapper[2585]: E0710 18:40:53.470033 2585 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/04ed092b-56ca-425f-bc70-cafaec37f312-samples-operator-tls podName:04ed092b-56ca-425f-bc70-cafaec37f312 nodeName:}" failed. No retries permitted until 2026-07-10 18:40:53.970024298 +0000 UTC m=+67.427384214 (durationBeforeRetry 500ms). Error: MountVolume.SetUp failed for volume "samples-operator-tls" (UniqueName: "kubernetes.io/secret/04ed092b-56ca-425f-bc70-cafaec37f312-samples-operator-tls") pod "cluster-samples-operator-65db858894-74x7s" (UID: "04ed092b-56ca-425f-bc70-cafaec37f312") : secret "samples-operator-tls" not found Jul 10 18:40:53.470378 ip-10-0-143-17 kubenswrapper[2585]: E0710 18:40:53.470134 2585 secret.go:189] Couldn't get secret openshift-ingress/router-metrics-certs-default: secret "router-metrics-certs-default" not found Jul 10 18:40:53.470378 ip-10-0-143-17 kubenswrapper[2585]: E0710 18:40:53.470183 2585 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/bf853b15-a7e2-4f66-ace3-b1f1fc9323de-metrics-certs podName:bf853b15-a7e2-4f66-ace3-b1f1fc9323de nodeName:}" failed. No retries permitted until 2026-07-10 18:40:53.970167671 +0000 UTC m=+67.427527584 (durationBeforeRetry 500ms). Error: MountVolume.SetUp failed for volume "metrics-certs" (UniqueName: "kubernetes.io/secret/bf853b15-a7e2-4f66-ace3-b1f1fc9323de-metrics-certs") pod "router-default-5d998f6f9d-t8frx" (UID: "bf853b15-a7e2-4f66-ace3-b1f1fc9323de") : secret "router-metrics-certs-default" not found Jul 10 18:40:53.470378 ip-10-0-143-17 kubenswrapper[2585]: I0710 18:40:53.470355 2585 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-v288k\" (UniqueName: \"kubernetes.io/projected/bf853b15-a7e2-4f66-ace3-b1f1fc9323de-kube-api-access-v288k\") pod \"router-default-5d998f6f9d-t8frx\" (UID: \"bf853b15-a7e2-4f66-ace3-b1f1fc9323de\") " pod="openshift-ingress/router-default-5d998f6f9d-t8frx" Jul 10 18:40:53.470561 ip-10-0-143-17 kubenswrapper[2585]: I0710 18:40:53.470385 2585 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"telemetry-config\" (UniqueName: \"kubernetes.io/configmap/45e87367-8dde-4c37-b642-f38ead4d2b04-telemetry-config\") pod \"cluster-monitoring-operator-5f7f494dd4-q52h2\" (UID: \"45e87367-8dde-4c37-b642-f38ead4d2b04\") " pod="openshift-monitoring/cluster-monitoring-operator-5f7f494dd4-q52h2" Jul 10 18:40:53.472888 ip-10-0-143-17 kubenswrapper[2585]: I0710 18:40:53.472846 2585 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"default-certificate\" (UniqueName: \"kubernetes.io/secret/bf853b15-a7e2-4f66-ace3-b1f1fc9323de-default-certificate\") pod \"router-default-5d998f6f9d-t8frx\" (UID: \"bf853b15-a7e2-4f66-ace3-b1f1fc9323de\") " pod="openshift-ingress/router-default-5d998f6f9d-t8frx" Jul 10 18:40:53.474943 ip-10-0-143-17 kubenswrapper[2585]: I0710 18:40:53.474873 2585 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"stats-auth\" (UniqueName: \"kubernetes.io/secret/bf853b15-a7e2-4f66-ace3-b1f1fc9323de-stats-auth\") pod \"router-default-5d998f6f9d-t8frx\" (UID: \"bf853b15-a7e2-4f66-ace3-b1f1fc9323de\") " pod="openshift-ingress/router-default-5d998f6f9d-t8frx" Jul 10 18:40:53.479265 ip-10-0-143-17 kubenswrapper[2585]: I0710 18:40:53.479243 2585 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-4x5l2\" (UniqueName: \"kubernetes.io/projected/04ed092b-56ca-425f-bc70-cafaec37f312-kube-api-access-4x5l2\") pod \"cluster-samples-operator-65db858894-74x7s\" (UID: \"04ed092b-56ca-425f-bc70-cafaec37f312\") " pod="openshift-cluster-samples-operator/cluster-samples-operator-65db858894-74x7s" Jul 10 18:40:53.480106 ip-10-0-143-17 kubenswrapper[2585]: I0710 18:40:53.480084 2585 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-7zb48\" (UniqueName: \"kubernetes.io/projected/aba57a3c-b29f-4038-9ee1-0dcfdf4c5d5f-kube-api-access-7zb48\") pod \"volume-data-source-validator-78495597f6-86v9g\" (UID: \"aba57a3c-b29f-4038-9ee1-0dcfdf4c5d5f\") " pod="openshift-cluster-storage-operator/volume-data-source-validator-78495597f6-86v9g" Jul 10 18:40:53.485253 ip-10-0-143-17 kubenswrapper[2585]: I0710 18:40:53.484243 2585 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-v288k\" (UniqueName: \"kubernetes.io/projected/bf853b15-a7e2-4f66-ace3-b1f1fc9323de-kube-api-access-v288k\") pod \"router-default-5d998f6f9d-t8frx\" (UID: \"bf853b15-a7e2-4f66-ace3-b1f1fc9323de\") " pod="openshift-ingress/router-default-5d998f6f9d-t8frx" Jul 10 18:40:53.492286 ip-10-0-143-17 kubenswrapper[2585]: I0710 18:40:53.492228 2585 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kube-system/global-pull-secret-syncer-hhpnn" event={"ID":"15a0ea9c-d294-4065-aed5-a182a76a7d7f","Type":"ContainerStarted","Data":"2111676aa081d5eae5566124aeb08e220304ccc3de32d8977b044fd45501b68d"} Jul 10 18:40:53.495797 ip-10-0-143-17 kubenswrapper[2585]: I0710 18:40:53.494796 2585 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-image-registry/image-registry-5bd858b6bc-fkzlq"] Jul 10 18:40:53.497552 ip-10-0-143-17 kubenswrapper[2585]: I0710 18:40:53.497496 2585 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-network-diagnostics/network-check-target-7tsng" event={"ID":"2166f472-2dd7-47f0-a15e-72d2abf71118","Type":"ContainerStarted","Data":"a2d36077f05d50c67079aedd184861e76a9b4e4f057a20f4837075a1c0343e96"} Jul 10 18:40:53.497642 ip-10-0-143-17 kubenswrapper[2585]: I0710 18:40:53.497611 2585 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-image-registry/image-registry-5bd858b6bc-fkzlq" Jul 10 18:40:53.500753 ip-10-0-143-17 kubenswrapper[2585]: I0710 18:40:53.500693 2585 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-image-registry\"/\"image-registry-private-configuration\"" Jul 10 18:40:53.500856 ip-10-0-143-17 kubenswrapper[2585]: I0710 18:40:53.500791 2585 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-image-registry\"/\"registry-dockercfg-nkq5k\"" Jul 10 18:40:53.500947 ip-10-0-143-17 kubenswrapper[2585]: I0710 18:40:53.500929 2585 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-image-registry\"/\"image-registry-tls\"" Jul 10 18:40:53.501179 ip-10-0-143-17 kubenswrapper[2585]: I0710 18:40:53.501056 2585 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-image-registry\"/\"installation-pull-secrets\"" Jul 10 18:40:53.506410 ip-10-0-143-17 kubenswrapper[2585]: I0710 18:40:53.506378 2585 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-image-registry\"/\"trusted-ca\"" Jul 10 18:40:53.509841 ip-10-0-143-17 kubenswrapper[2585]: I0710 18:40:53.509817 2585 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-image-registry/image-registry-5bd858b6bc-fkzlq"] Jul 10 18:40:53.569311 ip-10-0-143-17 kubenswrapper[2585]: I0710 18:40:53.569283 2585 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-insights/insights-operator-68d9dd94cf-57mwk"] Jul 10 18:40:53.571094 ip-10-0-143-17 kubenswrapper[2585]: I0710 18:40:53.571072 2585 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"image-registry-private-configuration\" (UniqueName: \"kubernetes.io/secret/f361ac0b-29f3-49e4-910a-1b98ff415751-image-registry-private-configuration\") pod \"image-registry-5bd858b6bc-fkzlq\" (UID: \"f361ac0b-29f3-49e4-910a-1b98ff415751\") " pod="openshift-image-registry/image-registry-5bd858b6bc-fkzlq" Jul 10 18:40:53.571206 ip-10-0-143-17 kubenswrapper[2585]: I0710 18:40:53.571127 2585 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"cluster-monitoring-operator-tls\" (UniqueName: \"kubernetes.io/secret/45e87367-8dde-4c37-b642-f38ead4d2b04-cluster-monitoring-operator-tls\") pod \"cluster-monitoring-operator-5f7f494dd4-q52h2\" (UID: \"45e87367-8dde-4c37-b642-f38ead4d2b04\") " pod="openshift-monitoring/cluster-monitoring-operator-5f7f494dd4-q52h2" Jul 10 18:40:53.571206 ip-10-0-143-17 kubenswrapper[2585]: I0710 18:40:53.571157 2585 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"telemetry-config\" (UniqueName: \"kubernetes.io/configmap/45e87367-8dde-4c37-b642-f38ead4d2b04-telemetry-config\") pod \"cluster-monitoring-operator-5f7f494dd4-q52h2\" (UID: \"45e87367-8dde-4c37-b642-f38ead4d2b04\") " pod="openshift-monitoring/cluster-monitoring-operator-5f7f494dd4-q52h2" Jul 10 18:40:53.571206 ip-10-0-143-17 kubenswrapper[2585]: I0710 18:40:53.571185 2585 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"installation-pull-secrets\" (UniqueName: \"kubernetes.io/secret/f361ac0b-29f3-49e4-910a-1b98ff415751-installation-pull-secrets\") pod \"image-registry-5bd858b6bc-fkzlq\" (UID: \"f361ac0b-29f3-49e4-910a-1b98ff415751\") " pod="openshift-image-registry/image-registry-5bd858b6bc-fkzlq" Jul 10 18:40:53.571363 ip-10-0-143-17 kubenswrapper[2585]: I0710 18:40:53.571212 2585 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-9hcvm\" (UniqueName: \"kubernetes.io/projected/45e87367-8dde-4c37-b642-f38ead4d2b04-kube-api-access-9hcvm\") pod \"cluster-monitoring-operator-5f7f494dd4-q52h2\" (UID: \"45e87367-8dde-4c37-b642-f38ead4d2b04\") " pod="openshift-monitoring/cluster-monitoring-operator-5f7f494dd4-q52h2" Jul 10 18:40:53.571363 ip-10-0-143-17 kubenswrapper[2585]: E0710 18:40:53.571251 2585 secret.go:189] Couldn't get secret openshift-monitoring/cluster-monitoring-operator-tls: secret "cluster-monitoring-operator-tls" not found Jul 10 18:40:53.571363 ip-10-0-143-17 kubenswrapper[2585]: E0710 18:40:53.571309 2585 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/45e87367-8dde-4c37-b642-f38ead4d2b04-cluster-monitoring-operator-tls podName:45e87367-8dde-4c37-b642-f38ead4d2b04 nodeName:}" failed. No retries permitted until 2026-07-10 18:40:54.071291696 +0000 UTC m=+67.528651626 (durationBeforeRetry 500ms). Error: MountVolume.SetUp failed for volume "cluster-monitoring-operator-tls" (UniqueName: "kubernetes.io/secret/45e87367-8dde-4c37-b642-f38ead4d2b04-cluster-monitoring-operator-tls") pod "cluster-monitoring-operator-5f7f494dd4-q52h2" (UID: "45e87367-8dde-4c37-b642-f38ead4d2b04") : secret "cluster-monitoring-operator-tls" not found Jul 10 18:40:53.571651 ip-10-0-143-17 kubenswrapper[2585]: I0710 18:40:53.571393 2585 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"registry-tls\" (UniqueName: \"kubernetes.io/projected/f361ac0b-29f3-49e4-910a-1b98ff415751-registry-tls\") pod \"image-registry-5bd858b6bc-fkzlq\" (UID: \"f361ac0b-29f3-49e4-910a-1b98ff415751\") " pod="openshift-image-registry/image-registry-5bd858b6bc-fkzlq" Jul 10 18:40:53.571651 ip-10-0-143-17 kubenswrapper[2585]: I0710 18:40:53.571429 2585 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"bound-sa-token\" (UniqueName: \"kubernetes.io/projected/f361ac0b-29f3-49e4-910a-1b98ff415751-bound-sa-token\") pod \"image-registry-5bd858b6bc-fkzlq\" (UID: \"f361ac0b-29f3-49e4-910a-1b98ff415751\") " pod="openshift-image-registry/image-registry-5bd858b6bc-fkzlq" Jul 10 18:40:53.571651 ip-10-0-143-17 kubenswrapper[2585]: I0710 18:40:53.571457 2585 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"ca-trust-extracted\" (UniqueName: \"kubernetes.io/empty-dir/f361ac0b-29f3-49e4-910a-1b98ff415751-ca-trust-extracted\") pod \"image-registry-5bd858b6bc-fkzlq\" (UID: \"f361ac0b-29f3-49e4-910a-1b98ff415751\") " pod="openshift-image-registry/image-registry-5bd858b6bc-fkzlq" Jul 10 18:40:53.571651 ip-10-0-143-17 kubenswrapper[2585]: I0710 18:40:53.571496 2585 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"registry-certificates\" (UniqueName: \"kubernetes.io/configmap/f361ac0b-29f3-49e4-910a-1b98ff415751-registry-certificates\") pod \"image-registry-5bd858b6bc-fkzlq\" (UID: \"f361ac0b-29f3-49e4-910a-1b98ff415751\") " pod="openshift-image-registry/image-registry-5bd858b6bc-fkzlq" Jul 10 18:40:53.571833 ip-10-0-143-17 kubenswrapper[2585]: I0710 18:40:53.571686 2585 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"trusted-ca\" (UniqueName: \"kubernetes.io/configmap/f361ac0b-29f3-49e4-910a-1b98ff415751-trusted-ca\") pod \"image-registry-5bd858b6bc-fkzlq\" (UID: \"f361ac0b-29f3-49e4-910a-1b98ff415751\") " pod="openshift-image-registry/image-registry-5bd858b6bc-fkzlq" Jul 10 18:40:53.571833 ip-10-0-143-17 kubenswrapper[2585]: I0710 18:40:53.571735 2585 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-jbmkw\" (UniqueName: \"kubernetes.io/projected/f361ac0b-29f3-49e4-910a-1b98ff415751-kube-api-access-jbmkw\") pod \"image-registry-5bd858b6bc-fkzlq\" (UID: \"f361ac0b-29f3-49e4-910a-1b98ff415751\") " pod="openshift-image-registry/image-registry-5bd858b6bc-fkzlq" Jul 10 18:40:53.572036 ip-10-0-143-17 kubenswrapper[2585]: I0710 18:40:53.572014 2585 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"telemetry-config\" (UniqueName: \"kubernetes.io/configmap/45e87367-8dde-4c37-b642-f38ead4d2b04-telemetry-config\") pod \"cluster-monitoring-operator-5f7f494dd4-q52h2\" (UID: \"45e87367-8dde-4c37-b642-f38ead4d2b04\") " pod="openshift-monitoring/cluster-monitoring-operator-5f7f494dd4-q52h2" Jul 10 18:40:53.573527 ip-10-0-143-17 kubenswrapper[2585]: W0710 18:40:53.573495 2585 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod3041e309_0ff4_41d8_9123_a82c98b5dad3.slice/crio-2060abf9bcb72cc585dc10e10ffd24adadb2ee0e78b71d1c85093a8704012d49 WatchSource:0}: Error finding container 2060abf9bcb72cc585dc10e10ffd24adadb2ee0e78b71d1c85093a8704012d49: Status 404 returned error can't find the container with id 2060abf9bcb72cc585dc10e10ffd24adadb2ee0e78b71d1c85093a8704012d49 Jul 10 18:40:53.580449 ip-10-0-143-17 kubenswrapper[2585]: I0710 18:40:53.580430 2585 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-9hcvm\" (UniqueName: \"kubernetes.io/projected/45e87367-8dde-4c37-b642-f38ead4d2b04-kube-api-access-9hcvm\") pod \"cluster-monitoring-operator-5f7f494dd4-q52h2\" (UID: \"45e87367-8dde-4c37-b642-f38ead4d2b04\") " pod="openshift-monitoring/cluster-monitoring-operator-5f7f494dd4-q52h2" Jul 10 18:40:53.614864 ip-10-0-143-17 kubenswrapper[2585]: I0710 18:40:53.614827 2585 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-cluster-storage-operator/volume-data-source-validator-78495597f6-86v9g" Jul 10 18:40:53.672928 ip-10-0-143-17 kubenswrapper[2585]: I0710 18:40:53.672881 2585 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"trusted-ca\" (UniqueName: \"kubernetes.io/configmap/f361ac0b-29f3-49e4-910a-1b98ff415751-trusted-ca\") pod \"image-registry-5bd858b6bc-fkzlq\" (UID: \"f361ac0b-29f3-49e4-910a-1b98ff415751\") " pod="openshift-image-registry/image-registry-5bd858b6bc-fkzlq" Jul 10 18:40:53.673040 ip-10-0-143-17 kubenswrapper[2585]: I0710 18:40:53.672963 2585 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-jbmkw\" (UniqueName: \"kubernetes.io/projected/f361ac0b-29f3-49e4-910a-1b98ff415751-kube-api-access-jbmkw\") pod \"image-registry-5bd858b6bc-fkzlq\" (UID: \"f361ac0b-29f3-49e4-910a-1b98ff415751\") " pod="openshift-image-registry/image-registry-5bd858b6bc-fkzlq" Jul 10 18:40:53.673040 ip-10-0-143-17 kubenswrapper[2585]: I0710 18:40:53.673012 2585 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-tls\" (UniqueName: \"kubernetes.io/secret/3c50e2a2-37c9-45b8-9ad0-fc624a151d61-metrics-tls\") pod \"dns-default-bwks6\" (UID: \"3c50e2a2-37c9-45b8-9ad0-fc624a151d61\") " pod="openshift-dns/dns-default-bwks6" Jul 10 18:40:53.673146 ip-10-0-143-17 kubenswrapper[2585]: I0710 18:40:53.673037 2585 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"image-registry-private-configuration\" (UniqueName: \"kubernetes.io/secret/f361ac0b-29f3-49e4-910a-1b98ff415751-image-registry-private-configuration\") pod \"image-registry-5bd858b6bc-fkzlq\" (UID: \"f361ac0b-29f3-49e4-910a-1b98ff415751\") " pod="openshift-image-registry/image-registry-5bd858b6bc-fkzlq" Jul 10 18:40:53.673146 ip-10-0-143-17 kubenswrapper[2585]: I0710 18:40:53.673113 2585 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"installation-pull-secrets\" (UniqueName: \"kubernetes.io/secret/f361ac0b-29f3-49e4-910a-1b98ff415751-installation-pull-secrets\") pod \"image-registry-5bd858b6bc-fkzlq\" (UID: \"f361ac0b-29f3-49e4-910a-1b98ff415751\") " pod="openshift-image-registry/image-registry-5bd858b6bc-fkzlq" Jul 10 18:40:53.673248 ip-10-0-143-17 kubenswrapper[2585]: I0710 18:40:53.673143 2585 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"registry-tls\" (UniqueName: \"kubernetes.io/projected/f361ac0b-29f3-49e4-910a-1b98ff415751-registry-tls\") pod \"image-registry-5bd858b6bc-fkzlq\" (UID: \"f361ac0b-29f3-49e4-910a-1b98ff415751\") " pod="openshift-image-registry/image-registry-5bd858b6bc-fkzlq" Jul 10 18:40:53.673248 ip-10-0-143-17 kubenswrapper[2585]: I0710 18:40:53.673178 2585 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"bound-sa-token\" (UniqueName: \"kubernetes.io/projected/f361ac0b-29f3-49e4-910a-1b98ff415751-bound-sa-token\") pod \"image-registry-5bd858b6bc-fkzlq\" (UID: \"f361ac0b-29f3-49e4-910a-1b98ff415751\") " pod="openshift-image-registry/image-registry-5bd858b6bc-fkzlq" Jul 10 18:40:53.673248 ip-10-0-143-17 kubenswrapper[2585]: I0710 18:40:53.673204 2585 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"ca-trust-extracted\" (UniqueName: \"kubernetes.io/empty-dir/f361ac0b-29f3-49e4-910a-1b98ff415751-ca-trust-extracted\") pod \"image-registry-5bd858b6bc-fkzlq\" (UID: \"f361ac0b-29f3-49e4-910a-1b98ff415751\") " pod="openshift-image-registry/image-registry-5bd858b6bc-fkzlq" Jul 10 18:40:53.673248 ip-10-0-143-17 kubenswrapper[2585]: I0710 18:40:53.673243 2585 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"registry-certificates\" (UniqueName: \"kubernetes.io/configmap/f361ac0b-29f3-49e4-910a-1b98ff415751-registry-certificates\") pod \"image-registry-5bd858b6bc-fkzlq\" (UID: \"f361ac0b-29f3-49e4-910a-1b98ff415751\") " pod="openshift-image-registry/image-registry-5bd858b6bc-fkzlq" Jul 10 18:40:53.674570 ip-10-0-143-17 kubenswrapper[2585]: E0710 18:40:53.673577 2585 projected.go:266] Couldn't get secret openshift-image-registry/image-registry-tls: secret "image-registry-tls" not found Jul 10 18:40:53.674570 ip-10-0-143-17 kubenswrapper[2585]: E0710 18:40:53.673602 2585 projected.go:196] Error preparing data for projected volume registry-tls for pod openshift-image-registry/image-registry-5bd858b6bc-fkzlq: secret "image-registry-tls" not found Jul 10 18:40:53.674570 ip-10-0-143-17 kubenswrapper[2585]: I0710 18:40:53.674140 2585 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"registry-certificates\" (UniqueName: \"kubernetes.io/configmap/f361ac0b-29f3-49e4-910a-1b98ff415751-registry-certificates\") pod \"image-registry-5bd858b6bc-fkzlq\" (UID: \"f361ac0b-29f3-49e4-910a-1b98ff415751\") " pod="openshift-image-registry/image-registry-5bd858b6bc-fkzlq" Jul 10 18:40:53.674570 ip-10-0-143-17 kubenswrapper[2585]: I0710 18:40:53.674227 2585 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"ca-trust-extracted\" (UniqueName: \"kubernetes.io/empty-dir/f361ac0b-29f3-49e4-910a-1b98ff415751-ca-trust-extracted\") pod \"image-registry-5bd858b6bc-fkzlq\" (UID: \"f361ac0b-29f3-49e4-910a-1b98ff415751\") " pod="openshift-image-registry/image-registry-5bd858b6bc-fkzlq" Jul 10 18:40:53.674570 ip-10-0-143-17 kubenswrapper[2585]: E0710 18:40:53.674452 2585 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/projected/f361ac0b-29f3-49e4-910a-1b98ff415751-registry-tls podName:f361ac0b-29f3-49e4-910a-1b98ff415751 nodeName:}" failed. No retries permitted until 2026-07-10 18:40:54.173641278 +0000 UTC m=+67.631001418 (durationBeforeRetry 500ms). Error: MountVolume.SetUp failed for volume "registry-tls" (UniqueName: "kubernetes.io/projected/f361ac0b-29f3-49e4-910a-1b98ff415751-registry-tls") pod "image-registry-5bd858b6bc-fkzlq" (UID: "f361ac0b-29f3-49e4-910a-1b98ff415751") : secret "image-registry-tls" not found Jul 10 18:40:53.674975 ip-10-0-143-17 kubenswrapper[2585]: E0710 18:40:53.674889 2585 secret.go:189] Couldn't get secret openshift-dns/dns-default-metrics-tls: secret "dns-default-metrics-tls" not found Jul 10 18:40:53.674975 ip-10-0-143-17 kubenswrapper[2585]: E0710 18:40:53.674957 2585 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/3c50e2a2-37c9-45b8-9ad0-fc624a151d61-metrics-tls podName:3c50e2a2-37c9-45b8-9ad0-fc624a151d61 nodeName:}" failed. No retries permitted until 2026-07-10 18:41:25.674935447 +0000 UTC m=+99.132295374 (durationBeforeRetry 32s). Error: MountVolume.SetUp failed for volume "metrics-tls" (UniqueName: "kubernetes.io/secret/3c50e2a2-37c9-45b8-9ad0-fc624a151d61-metrics-tls") pod "dns-default-bwks6" (UID: "3c50e2a2-37c9-45b8-9ad0-fc624a151d61") : secret "dns-default-metrics-tls" not found Jul 10 18:40:53.675329 ip-10-0-143-17 kubenswrapper[2585]: I0710 18:40:53.675305 2585 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"trusted-ca\" (UniqueName: \"kubernetes.io/configmap/f361ac0b-29f3-49e4-910a-1b98ff415751-trusted-ca\") pod \"image-registry-5bd858b6bc-fkzlq\" (UID: \"f361ac0b-29f3-49e4-910a-1b98ff415751\") " pod="openshift-image-registry/image-registry-5bd858b6bc-fkzlq" Jul 10 18:40:53.676354 ip-10-0-143-17 kubenswrapper[2585]: I0710 18:40:53.676327 2585 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"image-registry-private-configuration\" (UniqueName: \"kubernetes.io/secret/f361ac0b-29f3-49e4-910a-1b98ff415751-image-registry-private-configuration\") pod \"image-registry-5bd858b6bc-fkzlq\" (UID: \"f361ac0b-29f3-49e4-910a-1b98ff415751\") " pod="openshift-image-registry/image-registry-5bd858b6bc-fkzlq" Jul 10 18:40:53.676833 ip-10-0-143-17 kubenswrapper[2585]: I0710 18:40:53.676813 2585 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"installation-pull-secrets\" (UniqueName: \"kubernetes.io/secret/f361ac0b-29f3-49e4-910a-1b98ff415751-installation-pull-secrets\") pod \"image-registry-5bd858b6bc-fkzlq\" (UID: \"f361ac0b-29f3-49e4-910a-1b98ff415751\") " pod="openshift-image-registry/image-registry-5bd858b6bc-fkzlq" Jul 10 18:40:53.689752 ip-10-0-143-17 kubenswrapper[2585]: I0710 18:40:53.689729 2585 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"bound-sa-token\" (UniqueName: \"kubernetes.io/projected/f361ac0b-29f3-49e4-910a-1b98ff415751-bound-sa-token\") pod \"image-registry-5bd858b6bc-fkzlq\" (UID: \"f361ac0b-29f3-49e4-910a-1b98ff415751\") " pod="openshift-image-registry/image-registry-5bd858b6bc-fkzlq" Jul 10 18:40:53.691053 ip-10-0-143-17 kubenswrapper[2585]: I0710 18:40:53.691027 2585 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-jbmkw\" (UniqueName: \"kubernetes.io/projected/f361ac0b-29f3-49e4-910a-1b98ff415751-kube-api-access-jbmkw\") pod \"image-registry-5bd858b6bc-fkzlq\" (UID: \"f361ac0b-29f3-49e4-910a-1b98ff415751\") " pod="openshift-image-registry/image-registry-5bd858b6bc-fkzlq" Jul 10 18:40:53.761242 ip-10-0-143-17 kubenswrapper[2585]: I0710 18:40:53.761214 2585 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-cluster-storage-operator/volume-data-source-validator-78495597f6-86v9g"] Jul 10 18:40:53.763887 ip-10-0-143-17 kubenswrapper[2585]: W0710 18:40:53.763859 2585 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podaba57a3c_b29f_4038_9ee1_0dcfdf4c5d5f.slice/crio-809fe0beccc9110a37fbb3dfef194182e98d30a784b88cc5971d10d21f22ee96 WatchSource:0}: Error finding container 809fe0beccc9110a37fbb3dfef194182e98d30a784b88cc5971d10d21f22ee96: Status 404 returned error can't find the container with id 809fe0beccc9110a37fbb3dfef194182e98d30a784b88cc5971d10d21f22ee96 Jul 10 18:40:53.976055 ip-10-0-143-17 kubenswrapper[2585]: I0710 18:40:53.976016 2585 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/bf853b15-a7e2-4f66-ace3-b1f1fc9323de-metrics-certs\") pod \"router-default-5d998f6f9d-t8frx\" (UID: \"bf853b15-a7e2-4f66-ace3-b1f1fc9323de\") " pod="openshift-ingress/router-default-5d998f6f9d-t8frx" Jul 10 18:40:53.976501 ip-10-0-143-17 kubenswrapper[2585]: I0710 18:40:53.976086 2585 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"samples-operator-tls\" (UniqueName: \"kubernetes.io/secret/04ed092b-56ca-425f-bc70-cafaec37f312-samples-operator-tls\") pod \"cluster-samples-operator-65db858894-74x7s\" (UID: \"04ed092b-56ca-425f-bc70-cafaec37f312\") " pod="openshift-cluster-samples-operator/cluster-samples-operator-65db858894-74x7s" Jul 10 18:40:53.976501 ip-10-0-143-17 kubenswrapper[2585]: I0710 18:40:53.976143 2585 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"service-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/bf853b15-a7e2-4f66-ace3-b1f1fc9323de-service-ca-bundle\") pod \"router-default-5d998f6f9d-t8frx\" (UID: \"bf853b15-a7e2-4f66-ace3-b1f1fc9323de\") " pod="openshift-ingress/router-default-5d998f6f9d-t8frx" Jul 10 18:40:53.976501 ip-10-0-143-17 kubenswrapper[2585]: E0710 18:40:53.976205 2585 secret.go:189] Couldn't get secret openshift-ingress/router-metrics-certs-default: secret "router-metrics-certs-default" not found Jul 10 18:40:53.976501 ip-10-0-143-17 kubenswrapper[2585]: E0710 18:40:53.976284 2585 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/bf853b15-a7e2-4f66-ace3-b1f1fc9323de-metrics-certs podName:bf853b15-a7e2-4f66-ace3-b1f1fc9323de nodeName:}" failed. No retries permitted until 2026-07-10 18:40:54.97625722 +0000 UTC m=+68.433617146 (durationBeforeRetry 1s). Error: MountVolume.SetUp failed for volume "metrics-certs" (UniqueName: "kubernetes.io/secret/bf853b15-a7e2-4f66-ace3-b1f1fc9323de-metrics-certs") pod "router-default-5d998f6f9d-t8frx" (UID: "bf853b15-a7e2-4f66-ace3-b1f1fc9323de") : secret "router-metrics-certs-default" not found Jul 10 18:40:53.976501 ip-10-0-143-17 kubenswrapper[2585]: E0710 18:40:53.976326 2585 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/configmap/bf853b15-a7e2-4f66-ace3-b1f1fc9323de-service-ca-bundle podName:bf853b15-a7e2-4f66-ace3-b1f1fc9323de nodeName:}" failed. No retries permitted until 2026-07-10 18:40:54.976305983 +0000 UTC m=+68.433665910 (durationBeforeRetry 1s). Error: MountVolume.SetUp failed for volume "service-ca-bundle" (UniqueName: "kubernetes.io/configmap/bf853b15-a7e2-4f66-ace3-b1f1fc9323de-service-ca-bundle") pod "router-default-5d998f6f9d-t8frx" (UID: "bf853b15-a7e2-4f66-ace3-b1f1fc9323de") : configmap references non-existent config key: service-ca.crt Jul 10 18:40:53.976501 ip-10-0-143-17 kubenswrapper[2585]: E0710 18:40:53.976332 2585 secret.go:189] Couldn't get secret openshift-cluster-samples-operator/samples-operator-tls: secret "samples-operator-tls" not found Jul 10 18:40:53.976501 ip-10-0-143-17 kubenswrapper[2585]: E0710 18:40:53.976408 2585 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/04ed092b-56ca-425f-bc70-cafaec37f312-samples-operator-tls podName:04ed092b-56ca-425f-bc70-cafaec37f312 nodeName:}" failed. No retries permitted until 2026-07-10 18:40:54.976377317 +0000 UTC m=+68.433737245 (durationBeforeRetry 1s). Error: MountVolume.SetUp failed for volume "samples-operator-tls" (UniqueName: "kubernetes.io/secret/04ed092b-56ca-425f-bc70-cafaec37f312-samples-operator-tls") pod "cluster-samples-operator-65db858894-74x7s" (UID: "04ed092b-56ca-425f-bc70-cafaec37f312") : secret "samples-operator-tls" not found Jul 10 18:40:54.076920 ip-10-0-143-17 kubenswrapper[2585]: I0710 18:40:54.076884 2585 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"cluster-monitoring-operator-tls\" (UniqueName: \"kubernetes.io/secret/45e87367-8dde-4c37-b642-f38ead4d2b04-cluster-monitoring-operator-tls\") pod \"cluster-monitoring-operator-5f7f494dd4-q52h2\" (UID: \"45e87367-8dde-4c37-b642-f38ead4d2b04\") " pod="openshift-monitoring/cluster-monitoring-operator-5f7f494dd4-q52h2" Jul 10 18:40:54.077085 ip-10-0-143-17 kubenswrapper[2585]: E0710 18:40:54.077045 2585 secret.go:189] Couldn't get secret openshift-monitoring/cluster-monitoring-operator-tls: secret "cluster-monitoring-operator-tls" not found Jul 10 18:40:54.077480 ip-10-0-143-17 kubenswrapper[2585]: E0710 18:40:54.077430 2585 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/45e87367-8dde-4c37-b642-f38ead4d2b04-cluster-monitoring-operator-tls podName:45e87367-8dde-4c37-b642-f38ead4d2b04 nodeName:}" failed. No retries permitted until 2026-07-10 18:40:55.077407206 +0000 UTC m=+68.534767184 (durationBeforeRetry 1s). Error: MountVolume.SetUp failed for volume "cluster-monitoring-operator-tls" (UniqueName: "kubernetes.io/secret/45e87367-8dde-4c37-b642-f38ead4d2b04-cluster-monitoring-operator-tls") pod "cluster-monitoring-operator-5f7f494dd4-q52h2" (UID: "45e87367-8dde-4c37-b642-f38ead4d2b04") : secret "cluster-monitoring-operator-tls" not found Jul 10 18:40:54.177491 ip-10-0-143-17 kubenswrapper[2585]: I0710 18:40:54.177457 2585 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"registry-tls\" (UniqueName: \"kubernetes.io/projected/f361ac0b-29f3-49e4-910a-1b98ff415751-registry-tls\") pod \"image-registry-5bd858b6bc-fkzlq\" (UID: \"f361ac0b-29f3-49e4-910a-1b98ff415751\") " pod="openshift-image-registry/image-registry-5bd858b6bc-fkzlq" Jul 10 18:40:54.177681 ip-10-0-143-17 kubenswrapper[2585]: E0710 18:40:54.177616 2585 projected.go:266] Couldn't get secret openshift-image-registry/image-registry-tls: secret "image-registry-tls" not found Jul 10 18:40:54.177681 ip-10-0-143-17 kubenswrapper[2585]: E0710 18:40:54.177639 2585 projected.go:196] Error preparing data for projected volume registry-tls for pod openshift-image-registry/image-registry-5bd858b6bc-fkzlq: secret "image-registry-tls" not found Jul 10 18:40:54.177791 ip-10-0-143-17 kubenswrapper[2585]: E0710 18:40:54.177701 2585 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/projected/f361ac0b-29f3-49e4-910a-1b98ff415751-registry-tls podName:f361ac0b-29f3-49e4-910a-1b98ff415751 nodeName:}" failed. No retries permitted until 2026-07-10 18:40:55.177685619 +0000 UTC m=+68.635045544 (durationBeforeRetry 1s). Error: MountVolume.SetUp failed for volume "registry-tls" (UniqueName: "kubernetes.io/projected/f361ac0b-29f3-49e4-910a-1b98ff415751-registry-tls") pod "image-registry-5bd858b6bc-fkzlq" (UID: "f361ac0b-29f3-49e4-910a-1b98ff415751") : secret "image-registry-tls" not found Jul 10 18:40:54.501012 ip-10-0-143-17 kubenswrapper[2585]: I0710 18:40:54.500889 2585 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-cluster-storage-operator/volume-data-source-validator-78495597f6-86v9g" event={"ID":"aba57a3c-b29f-4038-9ee1-0dcfdf4c5d5f","Type":"ContainerStarted","Data":"809fe0beccc9110a37fbb3dfef194182e98d30a784b88cc5971d10d21f22ee96"} Jul 10 18:40:54.502307 ip-10-0-143-17 kubenswrapper[2585]: I0710 18:40:54.502248 2585 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-insights/insights-operator-68d9dd94cf-57mwk" event={"ID":"3041e309-0ff4-41d8-9123-a82c98b5dad3","Type":"ContainerStarted","Data":"2060abf9bcb72cc585dc10e10ffd24adadb2ee0e78b71d1c85093a8704012d49"} Jul 10 18:40:54.984849 ip-10-0-143-17 kubenswrapper[2585]: I0710 18:40:54.984682 2585 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/bf853b15-a7e2-4f66-ace3-b1f1fc9323de-metrics-certs\") pod \"router-default-5d998f6f9d-t8frx\" (UID: \"bf853b15-a7e2-4f66-ace3-b1f1fc9323de\") " pod="openshift-ingress/router-default-5d998f6f9d-t8frx" Jul 10 18:40:54.984849 ip-10-0-143-17 kubenswrapper[2585]: I0710 18:40:54.984749 2585 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"samples-operator-tls\" (UniqueName: \"kubernetes.io/secret/04ed092b-56ca-425f-bc70-cafaec37f312-samples-operator-tls\") pod \"cluster-samples-operator-65db858894-74x7s\" (UID: \"04ed092b-56ca-425f-bc70-cafaec37f312\") " pod="openshift-cluster-samples-operator/cluster-samples-operator-65db858894-74x7s" Jul 10 18:40:54.984849 ip-10-0-143-17 kubenswrapper[2585]: I0710 18:40:54.984805 2585 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"service-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/bf853b15-a7e2-4f66-ace3-b1f1fc9323de-service-ca-bundle\") pod \"router-default-5d998f6f9d-t8frx\" (UID: \"bf853b15-a7e2-4f66-ace3-b1f1fc9323de\") " pod="openshift-ingress/router-default-5d998f6f9d-t8frx" Jul 10 18:40:54.984849 ip-10-0-143-17 kubenswrapper[2585]: E0710 18:40:54.984835 2585 secret.go:189] Couldn't get secret openshift-ingress/router-metrics-certs-default: secret "router-metrics-certs-default" not found Jul 10 18:40:54.985453 ip-10-0-143-17 kubenswrapper[2585]: E0710 18:40:54.984900 2585 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/bf853b15-a7e2-4f66-ace3-b1f1fc9323de-metrics-certs podName:bf853b15-a7e2-4f66-ace3-b1f1fc9323de nodeName:}" failed. No retries permitted until 2026-07-10 18:40:56.98488144 +0000 UTC m=+70.442241368 (durationBeforeRetry 2s). Error: MountVolume.SetUp failed for volume "metrics-certs" (UniqueName: "kubernetes.io/secret/bf853b15-a7e2-4f66-ace3-b1f1fc9323de-metrics-certs") pod "router-default-5d998f6f9d-t8frx" (UID: "bf853b15-a7e2-4f66-ace3-b1f1fc9323de") : secret "router-metrics-certs-default" not found Jul 10 18:40:54.985453 ip-10-0-143-17 kubenswrapper[2585]: E0710 18:40:54.984962 2585 secret.go:189] Couldn't get secret openshift-cluster-samples-operator/samples-operator-tls: secret "samples-operator-tls" not found Jul 10 18:40:54.985453 ip-10-0-143-17 kubenswrapper[2585]: E0710 18:40:54.985027 2585 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/04ed092b-56ca-425f-bc70-cafaec37f312-samples-operator-tls podName:04ed092b-56ca-425f-bc70-cafaec37f312 nodeName:}" failed. No retries permitted until 2026-07-10 18:40:56.985009384 +0000 UTC m=+70.442369312 (durationBeforeRetry 2s). Error: MountVolume.SetUp failed for volume "samples-operator-tls" (UniqueName: "kubernetes.io/secret/04ed092b-56ca-425f-bc70-cafaec37f312-samples-operator-tls") pod "cluster-samples-operator-65db858894-74x7s" (UID: "04ed092b-56ca-425f-bc70-cafaec37f312") : secret "samples-operator-tls" not found Jul 10 18:40:54.985453 ip-10-0-143-17 kubenswrapper[2585]: E0710 18:40:54.985098 2585 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/configmap/bf853b15-a7e2-4f66-ace3-b1f1fc9323de-service-ca-bundle podName:bf853b15-a7e2-4f66-ace3-b1f1fc9323de nodeName:}" failed. No retries permitted until 2026-07-10 18:40:56.985088221 +0000 UTC m=+70.442448153 (durationBeforeRetry 2s). Error: MountVolume.SetUp failed for volume "service-ca-bundle" (UniqueName: "kubernetes.io/configmap/bf853b15-a7e2-4f66-ace3-b1f1fc9323de-service-ca-bundle") pod "router-default-5d998f6f9d-t8frx" (UID: "bf853b15-a7e2-4f66-ace3-b1f1fc9323de") : configmap references non-existent config key: service-ca.crt Jul 10 18:40:55.085477 ip-10-0-143-17 kubenswrapper[2585]: I0710 18:40:55.085440 2585 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"cluster-monitoring-operator-tls\" (UniqueName: \"kubernetes.io/secret/45e87367-8dde-4c37-b642-f38ead4d2b04-cluster-monitoring-operator-tls\") pod \"cluster-monitoring-operator-5f7f494dd4-q52h2\" (UID: \"45e87367-8dde-4c37-b642-f38ead4d2b04\") " pod="openshift-monitoring/cluster-monitoring-operator-5f7f494dd4-q52h2" Jul 10 18:40:55.085704 ip-10-0-143-17 kubenswrapper[2585]: E0710 18:40:55.085620 2585 secret.go:189] Couldn't get secret openshift-monitoring/cluster-monitoring-operator-tls: secret "cluster-monitoring-operator-tls" not found Jul 10 18:40:55.085704 ip-10-0-143-17 kubenswrapper[2585]: E0710 18:40:55.085698 2585 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/45e87367-8dde-4c37-b642-f38ead4d2b04-cluster-monitoring-operator-tls podName:45e87367-8dde-4c37-b642-f38ead4d2b04 nodeName:}" failed. No retries permitted until 2026-07-10 18:40:57.08567874 +0000 UTC m=+70.543038671 (durationBeforeRetry 2s). Error: MountVolume.SetUp failed for volume "cluster-monitoring-operator-tls" (UniqueName: "kubernetes.io/secret/45e87367-8dde-4c37-b642-f38ead4d2b04-cluster-monitoring-operator-tls") pod "cluster-monitoring-operator-5f7f494dd4-q52h2" (UID: "45e87367-8dde-4c37-b642-f38ead4d2b04") : secret "cluster-monitoring-operator-tls" not found Jul 10 18:40:55.187158 ip-10-0-143-17 kubenswrapper[2585]: I0710 18:40:55.186997 2585 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"registry-tls\" (UniqueName: \"kubernetes.io/projected/f361ac0b-29f3-49e4-910a-1b98ff415751-registry-tls\") pod \"image-registry-5bd858b6bc-fkzlq\" (UID: \"f361ac0b-29f3-49e4-910a-1b98ff415751\") " pod="openshift-image-registry/image-registry-5bd858b6bc-fkzlq" Jul 10 18:40:55.187277 ip-10-0-143-17 kubenswrapper[2585]: E0710 18:40:55.187229 2585 projected.go:266] Couldn't get secret openshift-image-registry/image-registry-tls: secret "image-registry-tls" not found Jul 10 18:40:55.187277 ip-10-0-143-17 kubenswrapper[2585]: E0710 18:40:55.187245 2585 projected.go:196] Error preparing data for projected volume registry-tls for pod openshift-image-registry/image-registry-5bd858b6bc-fkzlq: secret "image-registry-tls" not found Jul 10 18:40:55.187393 ip-10-0-143-17 kubenswrapper[2585]: E0710 18:40:55.187301 2585 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/projected/f361ac0b-29f3-49e4-910a-1b98ff415751-registry-tls podName:f361ac0b-29f3-49e4-910a-1b98ff415751 nodeName:}" failed. No retries permitted until 2026-07-10 18:40:57.187282499 +0000 UTC m=+70.644642435 (durationBeforeRetry 2s). Error: MountVolume.SetUp failed for volume "registry-tls" (UniqueName: "kubernetes.io/projected/f361ac0b-29f3-49e4-910a-1b98ff415751-registry-tls") pod "image-registry-5bd858b6bc-fkzlq" (UID: "f361ac0b-29f3-49e4-910a-1b98ff415751") : secret "image-registry-tls" not found Jul 10 18:40:55.507076 ip-10-0-143-17 kubenswrapper[2585]: I0710 18:40:55.507037 2585 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-cluster-storage-operator/volume-data-source-validator-78495597f6-86v9g" event={"ID":"aba57a3c-b29f-4038-9ee1-0dcfdf4c5d5f","Type":"ContainerStarted","Data":"8fc5884610fe285a0d987520b720d1823a7cbcabdd021f5f1f28f9ea7e321aca"} Jul 10 18:40:55.532817 ip-10-0-143-17 kubenswrapper[2585]: I0710 18:40:55.532610 2585 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-cluster-storage-operator/volume-data-source-validator-78495597f6-86v9g" podStartSLOduration=1.233825663 podStartE2EDuration="2.532591756s" totalImagesPullingTime="1.298766093s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-10 18:40:53 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-10 18:40:55.530604646 +0000 UTC m=+68.987964606" watchObservedRunningTime="2026-07-10 18:40:55.532591756 +0000 UTC m=+68.989951694" Jul 10 18:40:57.004841 ip-10-0-143-17 kubenswrapper[2585]: I0710 18:40:57.004803 2585 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/bf853b15-a7e2-4f66-ace3-b1f1fc9323de-metrics-certs\") pod \"router-default-5d998f6f9d-t8frx\" (UID: \"bf853b15-a7e2-4f66-ace3-b1f1fc9323de\") " pod="openshift-ingress/router-default-5d998f6f9d-t8frx" Jul 10 18:40:57.005287 ip-10-0-143-17 kubenswrapper[2585]: I0710 18:40:57.004870 2585 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"samples-operator-tls\" (UniqueName: \"kubernetes.io/secret/04ed092b-56ca-425f-bc70-cafaec37f312-samples-operator-tls\") pod \"cluster-samples-operator-65db858894-74x7s\" (UID: \"04ed092b-56ca-425f-bc70-cafaec37f312\") " pod="openshift-cluster-samples-operator/cluster-samples-operator-65db858894-74x7s" Jul 10 18:40:57.005287 ip-10-0-143-17 kubenswrapper[2585]: I0710 18:40:57.004936 2585 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"service-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/bf853b15-a7e2-4f66-ace3-b1f1fc9323de-service-ca-bundle\") pod \"router-default-5d998f6f9d-t8frx\" (UID: \"bf853b15-a7e2-4f66-ace3-b1f1fc9323de\") " pod="openshift-ingress/router-default-5d998f6f9d-t8frx" Jul 10 18:40:57.005287 ip-10-0-143-17 kubenswrapper[2585]: E0710 18:40:57.004967 2585 secret.go:189] Couldn't get secret openshift-ingress/router-metrics-certs-default: secret "router-metrics-certs-default" not found Jul 10 18:40:57.005287 ip-10-0-143-17 kubenswrapper[2585]: E0710 18:40:57.005044 2585 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/bf853b15-a7e2-4f66-ace3-b1f1fc9323de-metrics-certs podName:bf853b15-a7e2-4f66-ace3-b1f1fc9323de nodeName:}" failed. No retries permitted until 2026-07-10 18:41:01.005022876 +0000 UTC m=+74.462382789 (durationBeforeRetry 4s). Error: MountVolume.SetUp failed for volume "metrics-certs" (UniqueName: "kubernetes.io/secret/bf853b15-a7e2-4f66-ace3-b1f1fc9323de-metrics-certs") pod "router-default-5d998f6f9d-t8frx" (UID: "bf853b15-a7e2-4f66-ace3-b1f1fc9323de") : secret "router-metrics-certs-default" not found Jul 10 18:40:57.005287 ip-10-0-143-17 kubenswrapper[2585]: E0710 18:40:57.005043 2585 secret.go:189] Couldn't get secret openshift-cluster-samples-operator/samples-operator-tls: secret "samples-operator-tls" not found Jul 10 18:40:57.005287 ip-10-0-143-17 kubenswrapper[2585]: E0710 18:40:57.005075 2585 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/configmap/bf853b15-a7e2-4f66-ace3-b1f1fc9323de-service-ca-bundle podName:bf853b15-a7e2-4f66-ace3-b1f1fc9323de nodeName:}" failed. No retries permitted until 2026-07-10 18:41:01.005057255 +0000 UTC m=+74.462417182 (durationBeforeRetry 4s). Error: MountVolume.SetUp failed for volume "service-ca-bundle" (UniqueName: "kubernetes.io/configmap/bf853b15-a7e2-4f66-ace3-b1f1fc9323de-service-ca-bundle") pod "router-default-5d998f6f9d-t8frx" (UID: "bf853b15-a7e2-4f66-ace3-b1f1fc9323de") : configmap references non-existent config key: service-ca.crt Jul 10 18:40:57.005287 ip-10-0-143-17 kubenswrapper[2585]: E0710 18:40:57.005096 2585 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/04ed092b-56ca-425f-bc70-cafaec37f312-samples-operator-tls podName:04ed092b-56ca-425f-bc70-cafaec37f312 nodeName:}" failed. No retries permitted until 2026-07-10 18:41:01.005083416 +0000 UTC m=+74.462443348 (durationBeforeRetry 4s). Error: MountVolume.SetUp failed for volume "samples-operator-tls" (UniqueName: "kubernetes.io/secret/04ed092b-56ca-425f-bc70-cafaec37f312-samples-operator-tls") pod "cluster-samples-operator-65db858894-74x7s" (UID: "04ed092b-56ca-425f-bc70-cafaec37f312") : secret "samples-operator-tls" not found Jul 10 18:40:57.105658 ip-10-0-143-17 kubenswrapper[2585]: I0710 18:40:57.105618 2585 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"cluster-monitoring-operator-tls\" (UniqueName: \"kubernetes.io/secret/45e87367-8dde-4c37-b642-f38ead4d2b04-cluster-monitoring-operator-tls\") pod \"cluster-monitoring-operator-5f7f494dd4-q52h2\" (UID: \"45e87367-8dde-4c37-b642-f38ead4d2b04\") " pod="openshift-monitoring/cluster-monitoring-operator-5f7f494dd4-q52h2" Jul 10 18:40:57.105792 ip-10-0-143-17 kubenswrapper[2585]: E0710 18:40:57.105753 2585 secret.go:189] Couldn't get secret openshift-monitoring/cluster-monitoring-operator-tls: secret "cluster-monitoring-operator-tls" not found Jul 10 18:40:57.105854 ip-10-0-143-17 kubenswrapper[2585]: E0710 18:40:57.105814 2585 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/45e87367-8dde-4c37-b642-f38ead4d2b04-cluster-monitoring-operator-tls podName:45e87367-8dde-4c37-b642-f38ead4d2b04 nodeName:}" failed. No retries permitted until 2026-07-10 18:41:01.105799822 +0000 UTC m=+74.563159753 (durationBeforeRetry 4s). Error: MountVolume.SetUp failed for volume "cluster-monitoring-operator-tls" (UniqueName: "kubernetes.io/secret/45e87367-8dde-4c37-b642-f38ead4d2b04-cluster-monitoring-operator-tls") pod "cluster-monitoring-operator-5f7f494dd4-q52h2" (UID: "45e87367-8dde-4c37-b642-f38ead4d2b04") : secret "cluster-monitoring-operator-tls" not found Jul 10 18:40:57.206508 ip-10-0-143-17 kubenswrapper[2585]: I0710 18:40:57.206473 2585 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"registry-tls\" (UniqueName: \"kubernetes.io/projected/f361ac0b-29f3-49e4-910a-1b98ff415751-registry-tls\") pod \"image-registry-5bd858b6bc-fkzlq\" (UID: \"f361ac0b-29f3-49e4-910a-1b98ff415751\") " pod="openshift-image-registry/image-registry-5bd858b6bc-fkzlq" Jul 10 18:40:57.206681 ip-10-0-143-17 kubenswrapper[2585]: E0710 18:40:57.206600 2585 projected.go:266] Couldn't get secret openshift-image-registry/image-registry-tls: secret "image-registry-tls" not found Jul 10 18:40:57.206681 ip-10-0-143-17 kubenswrapper[2585]: E0710 18:40:57.206613 2585 projected.go:196] Error preparing data for projected volume registry-tls for pod openshift-image-registry/image-registry-5bd858b6bc-fkzlq: secret "image-registry-tls" not found Jul 10 18:40:57.206681 ip-10-0-143-17 kubenswrapper[2585]: E0710 18:40:57.206660 2585 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/projected/f361ac0b-29f3-49e4-910a-1b98ff415751-registry-tls podName:f361ac0b-29f3-49e4-910a-1b98ff415751 nodeName:}" failed. No retries permitted until 2026-07-10 18:41:01.206643384 +0000 UTC m=+74.664003318 (durationBeforeRetry 4s). Error: MountVolume.SetUp failed for volume "registry-tls" (UniqueName: "kubernetes.io/projected/f361ac0b-29f3-49e4-910a-1b98ff415751-registry-tls") pod "image-registry-5bd858b6bc-fkzlq" (UID: "f361ac0b-29f3-49e4-910a-1b98ff415751") : secret "image-registry-tls" not found Jul 10 18:40:59.465065 ip-10-0-143-17 kubenswrapper[2585]: I0710 18:40:59.465031 2585 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-insights/insights-runtime-extractor-rd8jt"] Jul 10 18:40:59.467339 ip-10-0-143-17 kubenswrapper[2585]: I0710 18:40:59.467323 2585 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-insights/insights-runtime-extractor-rd8jt" Jul 10 18:40:59.498001 ip-10-0-143-17 kubenswrapper[2585]: I0710 18:40:59.497978 2585 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-insights\"/\"insights-runtime-extractor-sa-dockercfg-nfnkx\"" Jul 10 18:40:59.498808 ip-10-0-143-17 kubenswrapper[2585]: I0710 18:40:59.498788 2585 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-insights\"/\"kube-rbac-proxy\"" Jul 10 18:40:59.516362 ip-10-0-143-17 kubenswrapper[2585]: I0710 18:40:59.516337 2585 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-insights\"/\"insights-runtime-extractor-tls\"" Jul 10 18:40:59.519846 ip-10-0-143-17 kubenswrapper[2585]: I0710 18:40:59.519823 2585 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kube-system/global-pull-secret-syncer-hhpnn" event={"ID":"15a0ea9c-d294-4065-aed5-a182a76a7d7f","Type":"ContainerStarted","Data":"77b95895d1909ab3f3c8f23a80815f2e1e10b781e7cf88f8dd11856b997b9a77"} Jul 10 18:40:59.521418 ip-10-0-143-17 kubenswrapper[2585]: I0710 18:40:59.521395 2585 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-insights/insights-operator-68d9dd94cf-57mwk" event={"ID":"3041e309-0ff4-41d8-9123-a82c98b5dad3","Type":"ContainerStarted","Data":"1f4b21e1ac086d5f08b8bb6c3b55e398f5400e413dbf6f25905163e09f5b294a"} Jul 10 18:40:59.522815 ip-10-0-143-17 kubenswrapper[2585]: I0710 18:40:59.522794 2585 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-network-diagnostics/network-check-target-7tsng" event={"ID":"2166f472-2dd7-47f0-a15e-72d2abf71118","Type":"ContainerStarted","Data":"6a8010b9492be018a1a47086cce3fae59115de68949b2270cab2c27e7b698812"} Jul 10 18:40:59.522930 ip-10-0-143-17 kubenswrapper[2585]: I0710 18:40:59.522917 2585 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-network-diagnostics/network-check-target-7tsng" Jul 10 18:40:59.525269 ip-10-0-143-17 kubenswrapper[2585]: I0710 18:40:59.525250 2585 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"insights-runtime-extractor-tls\" (UniqueName: \"kubernetes.io/secret/06325764-d12d-4aa8-9314-71a3a2c49d12-insights-runtime-extractor-tls\") pod \"insights-runtime-extractor-rd8jt\" (UID: \"06325764-d12d-4aa8-9314-71a3a2c49d12\") " pod="openshift-insights/insights-runtime-extractor-rd8jt" Jul 10 18:40:59.525341 ip-10-0-143-17 kubenswrapper[2585]: I0710 18:40:59.525291 2585 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-rbac-proxy-cm\" (UniqueName: \"kubernetes.io/configmap/06325764-d12d-4aa8-9314-71a3a2c49d12-kube-rbac-proxy-cm\") pod \"insights-runtime-extractor-rd8jt\" (UID: \"06325764-d12d-4aa8-9314-71a3a2c49d12\") " pod="openshift-insights/insights-runtime-extractor-rd8jt" Jul 10 18:40:59.525341 ip-10-0-143-17 kubenswrapper[2585]: I0710 18:40:59.525314 2585 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"data-volume\" (UniqueName: \"kubernetes.io/empty-dir/06325764-d12d-4aa8-9314-71a3a2c49d12-data-volume\") pod \"insights-runtime-extractor-rd8jt\" (UID: \"06325764-d12d-4aa8-9314-71a3a2c49d12\") " pod="openshift-insights/insights-runtime-extractor-rd8jt" Jul 10 18:40:59.525407 ip-10-0-143-17 kubenswrapper[2585]: I0710 18:40:59.525388 2585 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-hbhxm\" (UniqueName: \"kubernetes.io/projected/06325764-d12d-4aa8-9314-71a3a2c49d12-kube-api-access-hbhxm\") pod \"insights-runtime-extractor-rd8jt\" (UID: \"06325764-d12d-4aa8-9314-71a3a2c49d12\") " pod="openshift-insights/insights-runtime-extractor-rd8jt" Jul 10 18:40:59.525460 ip-10-0-143-17 kubenswrapper[2585]: I0710 18:40:59.525449 2585 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"crio-socket\" (UniqueName: \"kubernetes.io/host-path/06325764-d12d-4aa8-9314-71a3a2c49d12-crio-socket\") pod \"insights-runtime-extractor-rd8jt\" (UID: \"06325764-d12d-4aa8-9314-71a3a2c49d12\") " pod="openshift-insights/insights-runtime-extractor-rd8jt" Jul 10 18:40:59.553907 ip-10-0-143-17 kubenswrapper[2585]: I0710 18:40:59.553886 2585 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-insights/insights-runtime-extractor-rd8jt"] Jul 10 18:40:59.623997 ip-10-0-143-17 kubenswrapper[2585]: I0710 18:40:59.623957 2585 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="kube-system/global-pull-secret-syncer-hhpnn" podStartSLOduration=67.04493886 podStartE2EDuration="1m12.62394293s" totalImagesPullingTime="5.57900407s" totalInitContainerRuntime="0s" isStatefulPod=true podCreationTimestamp="2026-07-10 18:39:47 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-10 18:40:59.623628821 +0000 UTC m=+73.080988759" watchObservedRunningTime="2026-07-10 18:40:59.62394293 +0000 UTC m=+73.081302865" Jul 10 18:40:59.625879 ip-10-0-143-17 kubenswrapper[2585]: I0710 18:40:59.625860 2585 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"crio-socket\" (UniqueName: \"kubernetes.io/host-path/06325764-d12d-4aa8-9314-71a3a2c49d12-crio-socket\") pod \"insights-runtime-extractor-rd8jt\" (UID: \"06325764-d12d-4aa8-9314-71a3a2c49d12\") " pod="openshift-insights/insights-runtime-extractor-rd8jt" Jul 10 18:40:59.626011 ip-10-0-143-17 kubenswrapper[2585]: I0710 18:40:59.625987 2585 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"crio-socket\" (UniqueName: \"kubernetes.io/host-path/06325764-d12d-4aa8-9314-71a3a2c49d12-crio-socket\") pod \"insights-runtime-extractor-rd8jt\" (UID: \"06325764-d12d-4aa8-9314-71a3a2c49d12\") " pod="openshift-insights/insights-runtime-extractor-rd8jt" Jul 10 18:40:59.626100 ip-10-0-143-17 kubenswrapper[2585]: I0710 18:40:59.626075 2585 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"insights-runtime-extractor-tls\" (UniqueName: \"kubernetes.io/secret/06325764-d12d-4aa8-9314-71a3a2c49d12-insights-runtime-extractor-tls\") pod \"insights-runtime-extractor-rd8jt\" (UID: \"06325764-d12d-4aa8-9314-71a3a2c49d12\") " pod="openshift-insights/insights-runtime-extractor-rd8jt" Jul 10 18:40:59.626170 ip-10-0-143-17 kubenswrapper[2585]: I0710 18:40:59.626155 2585 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-rbac-proxy-cm\" (UniqueName: \"kubernetes.io/configmap/06325764-d12d-4aa8-9314-71a3a2c49d12-kube-rbac-proxy-cm\") pod \"insights-runtime-extractor-rd8jt\" (UID: \"06325764-d12d-4aa8-9314-71a3a2c49d12\") " pod="openshift-insights/insights-runtime-extractor-rd8jt" Jul 10 18:40:59.626230 ip-10-0-143-17 kubenswrapper[2585]: I0710 18:40:59.626182 2585 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"data-volume\" (UniqueName: \"kubernetes.io/empty-dir/06325764-d12d-4aa8-9314-71a3a2c49d12-data-volume\") pod \"insights-runtime-extractor-rd8jt\" (UID: \"06325764-d12d-4aa8-9314-71a3a2c49d12\") " pod="openshift-insights/insights-runtime-extractor-rd8jt" Jul 10 18:40:59.626230 ip-10-0-143-17 kubenswrapper[2585]: I0710 18:40:59.626216 2585 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-hbhxm\" (UniqueName: \"kubernetes.io/projected/06325764-d12d-4aa8-9314-71a3a2c49d12-kube-api-access-hbhxm\") pod \"insights-runtime-extractor-rd8jt\" (UID: \"06325764-d12d-4aa8-9314-71a3a2c49d12\") " pod="openshift-insights/insights-runtime-extractor-rd8jt" Jul 10 18:40:59.626352 ip-10-0-143-17 kubenswrapper[2585]: E0710 18:40:59.626333 2585 configmap.go:193] Couldn't get configMap openshift-insights/kube-rbac-proxy: configmap "kube-rbac-proxy" not found Jul 10 18:40:59.626463 ip-10-0-143-17 kubenswrapper[2585]: E0710 18:40:59.626349 2585 secret.go:189] Couldn't get secret openshift-insights/insights-runtime-extractor-tls: secret "insights-runtime-extractor-tls" not found Jul 10 18:40:59.626526 ip-10-0-143-17 kubenswrapper[2585]: E0710 18:40:59.626487 2585 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/configmap/06325764-d12d-4aa8-9314-71a3a2c49d12-kube-rbac-proxy-cm podName:06325764-d12d-4aa8-9314-71a3a2c49d12 nodeName:}" failed. No retries permitted until 2026-07-10 18:41:00.126424375 +0000 UTC m=+73.583784289 (durationBeforeRetry 500ms). Error: MountVolume.SetUp failed for volume "kube-rbac-proxy-cm" (UniqueName: "kubernetes.io/configmap/06325764-d12d-4aa8-9314-71a3a2c49d12-kube-rbac-proxy-cm") pod "insights-runtime-extractor-rd8jt" (UID: "06325764-d12d-4aa8-9314-71a3a2c49d12") : configmap "kube-rbac-proxy" not found Jul 10 18:40:59.626526 ip-10-0-143-17 kubenswrapper[2585]: E0710 18:40:59.626507 2585 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/06325764-d12d-4aa8-9314-71a3a2c49d12-insights-runtime-extractor-tls podName:06325764-d12d-4aa8-9314-71a3a2c49d12 nodeName:}" failed. No retries permitted until 2026-07-10 18:41:00.12649596 +0000 UTC m=+73.583855878 (durationBeforeRetry 500ms). Error: MountVolume.SetUp failed for volume "insights-runtime-extractor-tls" (UniqueName: "kubernetes.io/secret/06325764-d12d-4aa8-9314-71a3a2c49d12-insights-runtime-extractor-tls") pod "insights-runtime-extractor-rd8jt" (UID: "06325764-d12d-4aa8-9314-71a3a2c49d12") : secret "insights-runtime-extractor-tls" not found Jul 10 18:40:59.627103 ip-10-0-143-17 kubenswrapper[2585]: I0710 18:40:59.627083 2585 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"data-volume\" (UniqueName: \"kubernetes.io/empty-dir/06325764-d12d-4aa8-9314-71a3a2c49d12-data-volume\") pod \"insights-runtime-extractor-rd8jt\" (UID: \"06325764-d12d-4aa8-9314-71a3a2c49d12\") " pod="openshift-insights/insights-runtime-extractor-rd8jt" Jul 10 18:40:59.653070 ip-10-0-143-17 kubenswrapper[2585]: I0710 18:40:59.653043 2585 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-hbhxm\" (UniqueName: \"kubernetes.io/projected/06325764-d12d-4aa8-9314-71a3a2c49d12-kube-api-access-hbhxm\") pod \"insights-runtime-extractor-rd8jt\" (UID: \"06325764-d12d-4aa8-9314-71a3a2c49d12\") " pod="openshift-insights/insights-runtime-extractor-rd8jt" Jul 10 18:40:59.653150 ip-10-0-143-17 kubenswrapper[2585]: I0710 18:40:59.653084 2585 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-insights/insights-operator-68d9dd94cf-57mwk" podStartSLOduration=1.721274526 podStartE2EDuration="6.653072308s" totalImagesPullingTime="4.931797782s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-10 18:40:53 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-10 18:40:59.652236938 +0000 UTC m=+73.109596872" watchObservedRunningTime="2026-07-10 18:40:59.653072308 +0000 UTC m=+73.110432243" Jul 10 18:40:59.693509 ip-10-0-143-17 kubenswrapper[2585]: I0710 18:40:59.693479 2585 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-network-diagnostics/network-check-target-7tsng" podStartSLOduration=67.543907962 podStartE2EDuration="1m12.693467987s" totalImagesPullingTime="5.149560025s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-10 18:39:47 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-10 18:40:59.691845408 +0000 UTC m=+73.149205342" watchObservedRunningTime="2026-07-10 18:40:59.693467987 +0000 UTC m=+73.150827921" Jul 10 18:41:00.128724 ip-10-0-143-17 kubenswrapper[2585]: I0710 18:41:00.128698 2585 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"insights-runtime-extractor-tls\" (UniqueName: \"kubernetes.io/secret/06325764-d12d-4aa8-9314-71a3a2c49d12-insights-runtime-extractor-tls\") pod \"insights-runtime-extractor-rd8jt\" (UID: \"06325764-d12d-4aa8-9314-71a3a2c49d12\") " pod="openshift-insights/insights-runtime-extractor-rd8jt" Jul 10 18:41:00.128864 ip-10-0-143-17 kubenswrapper[2585]: I0710 18:41:00.128745 2585 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-rbac-proxy-cm\" (UniqueName: \"kubernetes.io/configmap/06325764-d12d-4aa8-9314-71a3a2c49d12-kube-rbac-proxy-cm\") pod \"insights-runtime-extractor-rd8jt\" (UID: \"06325764-d12d-4aa8-9314-71a3a2c49d12\") " pod="openshift-insights/insights-runtime-extractor-rd8jt" Jul 10 18:41:00.128864 ip-10-0-143-17 kubenswrapper[2585]: E0710 18:41:00.128846 2585 secret.go:189] Couldn't get secret openshift-insights/insights-runtime-extractor-tls: secret "insights-runtime-extractor-tls" not found Jul 10 18:41:00.128940 ip-10-0-143-17 kubenswrapper[2585]: E0710 18:41:00.128864 2585 configmap.go:193] Couldn't get configMap openshift-insights/kube-rbac-proxy: configmap "kube-rbac-proxy" not found Jul 10 18:41:00.128940 ip-10-0-143-17 kubenswrapper[2585]: E0710 18:41:00.128905 2585 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/configmap/06325764-d12d-4aa8-9314-71a3a2c49d12-kube-rbac-proxy-cm podName:06325764-d12d-4aa8-9314-71a3a2c49d12 nodeName:}" failed. No retries permitted until 2026-07-10 18:41:01.128893383 +0000 UTC m=+74.586253309 (durationBeforeRetry 1s). Error: MountVolume.SetUp failed for volume "kube-rbac-proxy-cm" (UniqueName: "kubernetes.io/configmap/06325764-d12d-4aa8-9314-71a3a2c49d12-kube-rbac-proxy-cm") pod "insights-runtime-extractor-rd8jt" (UID: "06325764-d12d-4aa8-9314-71a3a2c49d12") : configmap "kube-rbac-proxy" not found Jul 10 18:41:00.128940 ip-10-0-143-17 kubenswrapper[2585]: E0710 18:41:00.128919 2585 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/06325764-d12d-4aa8-9314-71a3a2c49d12-insights-runtime-extractor-tls podName:06325764-d12d-4aa8-9314-71a3a2c49d12 nodeName:}" failed. No retries permitted until 2026-07-10 18:41:01.128911707 +0000 UTC m=+74.586271620 (durationBeforeRetry 1s). Error: MountVolume.SetUp failed for volume "insights-runtime-extractor-tls" (UniqueName: "kubernetes.io/secret/06325764-d12d-4aa8-9314-71a3a2c49d12-insights-runtime-extractor-tls") pod "insights-runtime-extractor-rd8jt" (UID: "06325764-d12d-4aa8-9314-71a3a2c49d12") : secret "insights-runtime-extractor-tls" not found Jul 10 18:41:01.036861 ip-10-0-143-17 kubenswrapper[2585]: I0710 18:41:01.036829 2585 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/bf853b15-a7e2-4f66-ace3-b1f1fc9323de-metrics-certs\") pod \"router-default-5d998f6f9d-t8frx\" (UID: \"bf853b15-a7e2-4f66-ace3-b1f1fc9323de\") " pod="openshift-ingress/router-default-5d998f6f9d-t8frx" Jul 10 18:41:01.037275 ip-10-0-143-17 kubenswrapper[2585]: I0710 18:41:01.036877 2585 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"samples-operator-tls\" (UniqueName: \"kubernetes.io/secret/04ed092b-56ca-425f-bc70-cafaec37f312-samples-operator-tls\") pod \"cluster-samples-operator-65db858894-74x7s\" (UID: \"04ed092b-56ca-425f-bc70-cafaec37f312\") " pod="openshift-cluster-samples-operator/cluster-samples-operator-65db858894-74x7s" Jul 10 18:41:01.037275 ip-10-0-143-17 kubenswrapper[2585]: E0710 18:41:01.036952 2585 secret.go:189] Couldn't get secret openshift-ingress/router-metrics-certs-default: secret "router-metrics-certs-default" not found Jul 10 18:41:01.037275 ip-10-0-143-17 kubenswrapper[2585]: E0710 18:41:01.037016 2585 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/bf853b15-a7e2-4f66-ace3-b1f1fc9323de-metrics-certs podName:bf853b15-a7e2-4f66-ace3-b1f1fc9323de nodeName:}" failed. No retries permitted until 2026-07-10 18:41:09.037002968 +0000 UTC m=+82.494362881 (durationBeforeRetry 8s). Error: MountVolume.SetUp failed for volume "metrics-certs" (UniqueName: "kubernetes.io/secret/bf853b15-a7e2-4f66-ace3-b1f1fc9323de-metrics-certs") pod "router-default-5d998f6f9d-t8frx" (UID: "bf853b15-a7e2-4f66-ace3-b1f1fc9323de") : secret "router-metrics-certs-default" not found Jul 10 18:41:01.037275 ip-10-0-143-17 kubenswrapper[2585]: E0710 18:41:01.037016 2585 secret.go:189] Couldn't get secret openshift-cluster-samples-operator/samples-operator-tls: secret "samples-operator-tls" not found Jul 10 18:41:01.037275 ip-10-0-143-17 kubenswrapper[2585]: E0710 18:41:01.037038 2585 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/configmap/bf853b15-a7e2-4f66-ace3-b1f1fc9323de-service-ca-bundle podName:bf853b15-a7e2-4f66-ace3-b1f1fc9323de nodeName:}" failed. No retries permitted until 2026-07-10 18:41:09.037023765 +0000 UTC m=+82.494383683 (durationBeforeRetry 8s). Error: MountVolume.SetUp failed for volume "service-ca-bundle" (UniqueName: "kubernetes.io/configmap/bf853b15-a7e2-4f66-ace3-b1f1fc9323de-service-ca-bundle") pod "router-default-5d998f6f9d-t8frx" (UID: "bf853b15-a7e2-4f66-ace3-b1f1fc9323de") : configmap references non-existent config key: service-ca.crt Jul 10 18:41:01.037275 ip-10-0-143-17 kubenswrapper[2585]: I0710 18:41:01.036949 2585 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"service-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/bf853b15-a7e2-4f66-ace3-b1f1fc9323de-service-ca-bundle\") pod \"router-default-5d998f6f9d-t8frx\" (UID: \"bf853b15-a7e2-4f66-ace3-b1f1fc9323de\") " pod="openshift-ingress/router-default-5d998f6f9d-t8frx" Jul 10 18:41:01.037275 ip-10-0-143-17 kubenswrapper[2585]: E0710 18:41:01.037068 2585 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/04ed092b-56ca-425f-bc70-cafaec37f312-samples-operator-tls podName:04ed092b-56ca-425f-bc70-cafaec37f312 nodeName:}" failed. No retries permitted until 2026-07-10 18:41:09.037057679 +0000 UTC m=+82.494417592 (durationBeforeRetry 8s). Error: MountVolume.SetUp failed for volume "samples-operator-tls" (UniqueName: "kubernetes.io/secret/04ed092b-56ca-425f-bc70-cafaec37f312-samples-operator-tls") pod "cluster-samples-operator-65db858894-74x7s" (UID: "04ed092b-56ca-425f-bc70-cafaec37f312") : secret "samples-operator-tls" not found Jul 10 18:41:01.137666 ip-10-0-143-17 kubenswrapper[2585]: I0710 18:41:01.137641 2585 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"cluster-monitoring-operator-tls\" (UniqueName: \"kubernetes.io/secret/45e87367-8dde-4c37-b642-f38ead4d2b04-cluster-monitoring-operator-tls\") pod \"cluster-monitoring-operator-5f7f494dd4-q52h2\" (UID: \"45e87367-8dde-4c37-b642-f38ead4d2b04\") " pod="openshift-monitoring/cluster-monitoring-operator-5f7f494dd4-q52h2" Jul 10 18:41:01.137802 ip-10-0-143-17 kubenswrapper[2585]: I0710 18:41:01.137675 2585 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"insights-runtime-extractor-tls\" (UniqueName: \"kubernetes.io/secret/06325764-d12d-4aa8-9314-71a3a2c49d12-insights-runtime-extractor-tls\") pod \"insights-runtime-extractor-rd8jt\" (UID: \"06325764-d12d-4aa8-9314-71a3a2c49d12\") " pod="openshift-insights/insights-runtime-extractor-rd8jt" Jul 10 18:41:01.137802 ip-10-0-143-17 kubenswrapper[2585]: I0710 18:41:01.137706 2585 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-rbac-proxy-cm\" (UniqueName: \"kubernetes.io/configmap/06325764-d12d-4aa8-9314-71a3a2c49d12-kube-rbac-proxy-cm\") pod \"insights-runtime-extractor-rd8jt\" (UID: \"06325764-d12d-4aa8-9314-71a3a2c49d12\") " pod="openshift-insights/insights-runtime-extractor-rd8jt" Jul 10 18:41:01.137802 ip-10-0-143-17 kubenswrapper[2585]: E0710 18:41:01.137782 2585 secret.go:189] Couldn't get secret openshift-monitoring/cluster-monitoring-operator-tls: secret "cluster-monitoring-operator-tls" not found Jul 10 18:41:01.137802 ip-10-0-143-17 kubenswrapper[2585]: E0710 18:41:01.137795 2585 secret.go:189] Couldn't get secret openshift-insights/insights-runtime-extractor-tls: secret "insights-runtime-extractor-tls" not found Jul 10 18:41:01.138014 ip-10-0-143-17 kubenswrapper[2585]: E0710 18:41:01.137834 2585 configmap.go:193] Couldn't get configMap openshift-insights/kube-rbac-proxy: configmap "kube-rbac-proxy" not found Jul 10 18:41:01.138014 ip-10-0-143-17 kubenswrapper[2585]: E0710 18:41:01.137838 2585 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/45e87367-8dde-4c37-b642-f38ead4d2b04-cluster-monitoring-operator-tls podName:45e87367-8dde-4c37-b642-f38ead4d2b04 nodeName:}" failed. No retries permitted until 2026-07-10 18:41:09.137823287 +0000 UTC m=+82.595183219 (durationBeforeRetry 8s). Error: MountVolume.SetUp failed for volume "cluster-monitoring-operator-tls" (UniqueName: "kubernetes.io/secret/45e87367-8dde-4c37-b642-f38ead4d2b04-cluster-monitoring-operator-tls") pod "cluster-monitoring-operator-5f7f494dd4-q52h2" (UID: "45e87367-8dde-4c37-b642-f38ead4d2b04") : secret "cluster-monitoring-operator-tls" not found Jul 10 18:41:01.138014 ip-10-0-143-17 kubenswrapper[2585]: E0710 18:41:01.137881 2585 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/06325764-d12d-4aa8-9314-71a3a2c49d12-insights-runtime-extractor-tls podName:06325764-d12d-4aa8-9314-71a3a2c49d12 nodeName:}" failed. No retries permitted until 2026-07-10 18:41:03.137871111 +0000 UTC m=+76.595231023 (durationBeforeRetry 2s). Error: MountVolume.SetUp failed for volume "insights-runtime-extractor-tls" (UniqueName: "kubernetes.io/secret/06325764-d12d-4aa8-9314-71a3a2c49d12-insights-runtime-extractor-tls") pod "insights-runtime-extractor-rd8jt" (UID: "06325764-d12d-4aa8-9314-71a3a2c49d12") : secret "insights-runtime-extractor-tls" not found Jul 10 18:41:01.138014 ip-10-0-143-17 kubenswrapper[2585]: E0710 18:41:01.137892 2585 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/configmap/06325764-d12d-4aa8-9314-71a3a2c49d12-kube-rbac-proxy-cm podName:06325764-d12d-4aa8-9314-71a3a2c49d12 nodeName:}" failed. No retries permitted until 2026-07-10 18:41:03.137886476 +0000 UTC m=+76.595246389 (durationBeforeRetry 2s). Error: MountVolume.SetUp failed for volume "kube-rbac-proxy-cm" (UniqueName: "kubernetes.io/configmap/06325764-d12d-4aa8-9314-71a3a2c49d12-kube-rbac-proxy-cm") pod "insights-runtime-extractor-rd8jt" (UID: "06325764-d12d-4aa8-9314-71a3a2c49d12") : configmap "kube-rbac-proxy" not found Jul 10 18:41:01.238517 ip-10-0-143-17 kubenswrapper[2585]: I0710 18:41:01.238492 2585 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"registry-tls\" (UniqueName: \"kubernetes.io/projected/f361ac0b-29f3-49e4-910a-1b98ff415751-registry-tls\") pod \"image-registry-5bd858b6bc-fkzlq\" (UID: \"f361ac0b-29f3-49e4-910a-1b98ff415751\") " pod="openshift-image-registry/image-registry-5bd858b6bc-fkzlq" Jul 10 18:41:01.238655 ip-10-0-143-17 kubenswrapper[2585]: E0710 18:41:01.238634 2585 projected.go:266] Couldn't get secret openshift-image-registry/image-registry-tls: secret "image-registry-tls" not found Jul 10 18:41:01.238655 ip-10-0-143-17 kubenswrapper[2585]: E0710 18:41:01.238649 2585 projected.go:196] Error preparing data for projected volume registry-tls for pod openshift-image-registry/image-registry-5bd858b6bc-fkzlq: secret "image-registry-tls" not found Jul 10 18:41:01.238766 ip-10-0-143-17 kubenswrapper[2585]: E0710 18:41:01.238699 2585 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/projected/f361ac0b-29f3-49e4-910a-1b98ff415751-registry-tls podName:f361ac0b-29f3-49e4-910a-1b98ff415751 nodeName:}" failed. No retries permitted until 2026-07-10 18:41:09.238684545 +0000 UTC m=+82.696044480 (durationBeforeRetry 8s). Error: MountVolume.SetUp failed for volume "registry-tls" (UniqueName: "kubernetes.io/projected/f361ac0b-29f3-49e4-910a-1b98ff415751-registry-tls") pod "image-registry-5bd858b6bc-fkzlq" (UID: "f361ac0b-29f3-49e4-910a-1b98ff415751") : secret "image-registry-tls" not found Jul 10 18:41:01.977205 ip-10-0-143-17 kubenswrapper[2585]: I0710 18:41:01.977173 2585 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-dns_node-resolver-ndwqb_5a594168-14da-41cb-9871-edbbf08720f7/dns-node-resolver/0.log" Jul 10 18:41:03.153874 ip-10-0-143-17 kubenswrapper[2585]: I0710 18:41:03.153839 2585 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"insights-runtime-extractor-tls\" (UniqueName: \"kubernetes.io/secret/06325764-d12d-4aa8-9314-71a3a2c49d12-insights-runtime-extractor-tls\") pod \"insights-runtime-extractor-rd8jt\" (UID: \"06325764-d12d-4aa8-9314-71a3a2c49d12\") " pod="openshift-insights/insights-runtime-extractor-rd8jt" Jul 10 18:41:03.154252 ip-10-0-143-17 kubenswrapper[2585]: I0710 18:41:03.153886 2585 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-rbac-proxy-cm\" (UniqueName: \"kubernetes.io/configmap/06325764-d12d-4aa8-9314-71a3a2c49d12-kube-rbac-proxy-cm\") pod \"insights-runtime-extractor-rd8jt\" (UID: \"06325764-d12d-4aa8-9314-71a3a2c49d12\") " pod="openshift-insights/insights-runtime-extractor-rd8jt" Jul 10 18:41:03.154252 ip-10-0-143-17 kubenswrapper[2585]: E0710 18:41:03.153984 2585 secret.go:189] Couldn't get secret openshift-insights/insights-runtime-extractor-tls: secret "insights-runtime-extractor-tls" not found Jul 10 18:41:03.154252 ip-10-0-143-17 kubenswrapper[2585]: E0710 18:41:03.154045 2585 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/06325764-d12d-4aa8-9314-71a3a2c49d12-insights-runtime-extractor-tls podName:06325764-d12d-4aa8-9314-71a3a2c49d12 nodeName:}" failed. No retries permitted until 2026-07-10 18:41:07.154030869 +0000 UTC m=+80.611390782 (durationBeforeRetry 4s). Error: MountVolume.SetUp failed for volume "insights-runtime-extractor-tls" (UniqueName: "kubernetes.io/secret/06325764-d12d-4aa8-9314-71a3a2c49d12-insights-runtime-extractor-tls") pod "insights-runtime-extractor-rd8jt" (UID: "06325764-d12d-4aa8-9314-71a3a2c49d12") : secret "insights-runtime-extractor-tls" not found Jul 10 18:41:03.154252 ip-10-0-143-17 kubenswrapper[2585]: E0710 18:41:03.153991 2585 configmap.go:193] Couldn't get configMap openshift-insights/kube-rbac-proxy: configmap "kube-rbac-proxy" not found Jul 10 18:41:03.154252 ip-10-0-143-17 kubenswrapper[2585]: E0710 18:41:03.154078 2585 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/configmap/06325764-d12d-4aa8-9314-71a3a2c49d12-kube-rbac-proxy-cm podName:06325764-d12d-4aa8-9314-71a3a2c49d12 nodeName:}" failed. No retries permitted until 2026-07-10 18:41:07.154071526 +0000 UTC m=+80.611431438 (durationBeforeRetry 4s). Error: MountVolume.SetUp failed for volume "kube-rbac-proxy-cm" (UniqueName: "kubernetes.io/configmap/06325764-d12d-4aa8-9314-71a3a2c49d12-kube-rbac-proxy-cm") pod "insights-runtime-extractor-rd8jt" (UID: "06325764-d12d-4aa8-9314-71a3a2c49d12") : configmap "kube-rbac-proxy" not found Jul 10 18:41:03.376323 ip-10-0-143-17 kubenswrapper[2585]: I0710 18:41:03.376297 2585 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-image-registry_node-ca-crbxd_ad44fb01-da7b-40ac-906e-5a7badc87f54/node-ca/0.log" Jul 10 18:41:04.177608 ip-10-0-143-17 kubenswrapper[2585]: I0710 18:41:04.177585 2585 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ingress-canary_ingress-canary-g7fx6_8c0d7178-8c0c-4f9c-b534-fe044b79a208/serve-healthcheck-canary/0.log" Jul 10 18:41:07.186211 ip-10-0-143-17 kubenswrapper[2585]: I0710 18:41:07.186177 2585 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"insights-runtime-extractor-tls\" (UniqueName: \"kubernetes.io/secret/06325764-d12d-4aa8-9314-71a3a2c49d12-insights-runtime-extractor-tls\") pod \"insights-runtime-extractor-rd8jt\" (UID: \"06325764-d12d-4aa8-9314-71a3a2c49d12\") " pod="openshift-insights/insights-runtime-extractor-rd8jt" Jul 10 18:41:07.186632 ip-10-0-143-17 kubenswrapper[2585]: I0710 18:41:07.186221 2585 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-rbac-proxy-cm\" (UniqueName: \"kubernetes.io/configmap/06325764-d12d-4aa8-9314-71a3a2c49d12-kube-rbac-proxy-cm\") pod \"insights-runtime-extractor-rd8jt\" (UID: \"06325764-d12d-4aa8-9314-71a3a2c49d12\") " pod="openshift-insights/insights-runtime-extractor-rd8jt" Jul 10 18:41:07.186632 ip-10-0-143-17 kubenswrapper[2585]: E0710 18:41:07.186289 2585 configmap.go:193] Couldn't get configMap openshift-insights/kube-rbac-proxy: configmap "kube-rbac-proxy" not found Jul 10 18:41:07.186632 ip-10-0-143-17 kubenswrapper[2585]: E0710 18:41:07.186341 2585 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/configmap/06325764-d12d-4aa8-9314-71a3a2c49d12-kube-rbac-proxy-cm podName:06325764-d12d-4aa8-9314-71a3a2c49d12 nodeName:}" failed. No retries permitted until 2026-07-10 18:41:15.186323898 +0000 UTC m=+88.643683811 (durationBeforeRetry 8s). Error: MountVolume.SetUp failed for volume "kube-rbac-proxy-cm" (UniqueName: "kubernetes.io/configmap/06325764-d12d-4aa8-9314-71a3a2c49d12-kube-rbac-proxy-cm") pod "insights-runtime-extractor-rd8jt" (UID: "06325764-d12d-4aa8-9314-71a3a2c49d12") : configmap "kube-rbac-proxy" not found Jul 10 18:41:07.186632 ip-10-0-143-17 kubenswrapper[2585]: E0710 18:41:07.186356 2585 secret.go:189] Couldn't get secret openshift-insights/insights-runtime-extractor-tls: secret "insights-runtime-extractor-tls" not found Jul 10 18:41:07.186632 ip-10-0-143-17 kubenswrapper[2585]: E0710 18:41:07.186416 2585 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/06325764-d12d-4aa8-9314-71a3a2c49d12-insights-runtime-extractor-tls podName:06325764-d12d-4aa8-9314-71a3a2c49d12 nodeName:}" failed. No retries permitted until 2026-07-10 18:41:15.186397636 +0000 UTC m=+88.643757562 (durationBeforeRetry 8s). Error: MountVolume.SetUp failed for volume "insights-runtime-extractor-tls" (UniqueName: "kubernetes.io/secret/06325764-d12d-4aa8-9314-71a3a2c49d12-insights-runtime-extractor-tls") pod "insights-runtime-extractor-rd8jt" (UID: "06325764-d12d-4aa8-9314-71a3a2c49d12") : secret "insights-runtime-extractor-tls" not found Jul 10 18:41:09.097877 ip-10-0-143-17 kubenswrapper[2585]: I0710 18:41:09.097840 2585 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/bf853b15-a7e2-4f66-ace3-b1f1fc9323de-metrics-certs\") pod \"router-default-5d998f6f9d-t8frx\" (UID: \"bf853b15-a7e2-4f66-ace3-b1f1fc9323de\") " pod="openshift-ingress/router-default-5d998f6f9d-t8frx" Jul 10 18:41:09.097877 ip-10-0-143-17 kubenswrapper[2585]: I0710 18:41:09.097888 2585 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"samples-operator-tls\" (UniqueName: \"kubernetes.io/secret/04ed092b-56ca-425f-bc70-cafaec37f312-samples-operator-tls\") pod \"cluster-samples-operator-65db858894-74x7s\" (UID: \"04ed092b-56ca-425f-bc70-cafaec37f312\") " pod="openshift-cluster-samples-operator/cluster-samples-operator-65db858894-74x7s" Jul 10 18:41:09.098448 ip-10-0-143-17 kubenswrapper[2585]: I0710 18:41:09.097928 2585 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"service-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/bf853b15-a7e2-4f66-ace3-b1f1fc9323de-service-ca-bundle\") pod \"router-default-5d998f6f9d-t8frx\" (UID: \"bf853b15-a7e2-4f66-ace3-b1f1fc9323de\") " pod="openshift-ingress/router-default-5d998f6f9d-t8frx" Jul 10 18:41:09.098448 ip-10-0-143-17 kubenswrapper[2585]: E0710 18:41:09.098013 2585 secret.go:189] Couldn't get secret openshift-ingress/router-metrics-certs-default: secret "router-metrics-certs-default" not found Jul 10 18:41:09.098448 ip-10-0-143-17 kubenswrapper[2585]: E0710 18:41:09.098088 2585 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/configmap/bf853b15-a7e2-4f66-ace3-b1f1fc9323de-service-ca-bundle podName:bf853b15-a7e2-4f66-ace3-b1f1fc9323de nodeName:}" failed. No retries permitted until 2026-07-10 18:41:25.098066395 +0000 UTC m=+98.555426324 (durationBeforeRetry 16s). Error: MountVolume.SetUp failed for volume "service-ca-bundle" (UniqueName: "kubernetes.io/configmap/bf853b15-a7e2-4f66-ace3-b1f1fc9323de-service-ca-bundle") pod "router-default-5d998f6f9d-t8frx" (UID: "bf853b15-a7e2-4f66-ace3-b1f1fc9323de") : configmap references non-existent config key: service-ca.crt Jul 10 18:41:09.098448 ip-10-0-143-17 kubenswrapper[2585]: E0710 18:41:09.098109 2585 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/bf853b15-a7e2-4f66-ace3-b1f1fc9323de-metrics-certs podName:bf853b15-a7e2-4f66-ace3-b1f1fc9323de nodeName:}" failed. No retries permitted until 2026-07-10 18:41:25.098100592 +0000 UTC m=+98.555460505 (durationBeforeRetry 16s). Error: MountVolume.SetUp failed for volume "metrics-certs" (UniqueName: "kubernetes.io/secret/bf853b15-a7e2-4f66-ace3-b1f1fc9323de-metrics-certs") pod "router-default-5d998f6f9d-t8frx" (UID: "bf853b15-a7e2-4f66-ace3-b1f1fc9323de") : secret "router-metrics-certs-default" not found Jul 10 18:41:09.100727 ip-10-0-143-17 kubenswrapper[2585]: I0710 18:41:09.100700 2585 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"samples-operator-tls\" (UniqueName: \"kubernetes.io/secret/04ed092b-56ca-425f-bc70-cafaec37f312-samples-operator-tls\") pod \"cluster-samples-operator-65db858894-74x7s\" (UID: \"04ed092b-56ca-425f-bc70-cafaec37f312\") " pod="openshift-cluster-samples-operator/cluster-samples-operator-65db858894-74x7s" Jul 10 18:41:09.198295 ip-10-0-143-17 kubenswrapper[2585]: I0710 18:41:09.198264 2585 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"cluster-monitoring-operator-tls\" (UniqueName: \"kubernetes.io/secret/45e87367-8dde-4c37-b642-f38ead4d2b04-cluster-monitoring-operator-tls\") pod \"cluster-monitoring-operator-5f7f494dd4-q52h2\" (UID: \"45e87367-8dde-4c37-b642-f38ead4d2b04\") " pod="openshift-monitoring/cluster-monitoring-operator-5f7f494dd4-q52h2" Jul 10 18:41:09.198445 ip-10-0-143-17 kubenswrapper[2585]: E0710 18:41:09.198374 2585 secret.go:189] Couldn't get secret openshift-monitoring/cluster-monitoring-operator-tls: secret "cluster-monitoring-operator-tls" not found Jul 10 18:41:09.198445 ip-10-0-143-17 kubenswrapper[2585]: E0710 18:41:09.198425 2585 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/45e87367-8dde-4c37-b642-f38ead4d2b04-cluster-monitoring-operator-tls podName:45e87367-8dde-4c37-b642-f38ead4d2b04 nodeName:}" failed. No retries permitted until 2026-07-10 18:41:25.1984083 +0000 UTC m=+98.655768228 (durationBeforeRetry 16s). Error: MountVolume.SetUp failed for volume "cluster-monitoring-operator-tls" (UniqueName: "kubernetes.io/secret/45e87367-8dde-4c37-b642-f38ead4d2b04-cluster-monitoring-operator-tls") pod "cluster-monitoring-operator-5f7f494dd4-q52h2" (UID: "45e87367-8dde-4c37-b642-f38ead4d2b04") : secret "cluster-monitoring-operator-tls" not found Jul 10 18:41:09.248626 ip-10-0-143-17 kubenswrapper[2585]: I0710 18:41:09.248601 2585 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-cluster-samples-operator/cluster-samples-operator-65db858894-74x7s" Jul 10 18:41:09.299447 ip-10-0-143-17 kubenswrapper[2585]: I0710 18:41:09.299218 2585 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"registry-tls\" (UniqueName: \"kubernetes.io/projected/f361ac0b-29f3-49e4-910a-1b98ff415751-registry-tls\") pod \"image-registry-5bd858b6bc-fkzlq\" (UID: \"f361ac0b-29f3-49e4-910a-1b98ff415751\") " pod="openshift-image-registry/image-registry-5bd858b6bc-fkzlq" Jul 10 18:41:09.302397 ip-10-0-143-17 kubenswrapper[2585]: I0710 18:41:09.302371 2585 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"registry-tls\" (UniqueName: \"kubernetes.io/projected/f361ac0b-29f3-49e4-910a-1b98ff415751-registry-tls\") pod \"image-registry-5bd858b6bc-fkzlq\" (UID: \"f361ac0b-29f3-49e4-910a-1b98ff415751\") " pod="openshift-image-registry/image-registry-5bd858b6bc-fkzlq" Jul 10 18:41:09.376365 ip-10-0-143-17 kubenswrapper[2585]: I0710 18:41:09.376316 2585 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-cluster-samples-operator/cluster-samples-operator-65db858894-74x7s"] Jul 10 18:41:09.408670 ip-10-0-143-17 kubenswrapper[2585]: I0710 18:41:09.408650 2585 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-image-registry/image-registry-5bd858b6bc-fkzlq" Jul 10 18:41:09.529766 ip-10-0-143-17 kubenswrapper[2585]: I0710 18:41:09.529742 2585 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-image-registry/image-registry-5bd858b6bc-fkzlq"] Jul 10 18:41:09.532583 ip-10-0-143-17 kubenswrapper[2585]: W0710 18:41:09.532564 2585 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podf361ac0b_29f3_49e4_910a_1b98ff415751.slice/crio-b2298d146de5704e86c2fdd60a2848fbc899cd745ba20c9e066ee5924c11579a WatchSource:0}: Error finding container b2298d146de5704e86c2fdd60a2848fbc899cd745ba20c9e066ee5924c11579a: Status 404 returned error can't find the container with id b2298d146de5704e86c2fdd60a2848fbc899cd745ba20c9e066ee5924c11579a Jul 10 18:41:09.554131 ip-10-0-143-17 kubenswrapper[2585]: I0710 18:41:09.554099 2585 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-image-registry/image-registry-5bd858b6bc-fkzlq" event={"ID":"f361ac0b-29f3-49e4-910a-1b98ff415751","Type":"ContainerStarted","Data":"b2298d146de5704e86c2fdd60a2848fbc899cd745ba20c9e066ee5924c11579a"} Jul 10 18:41:09.555158 ip-10-0-143-17 kubenswrapper[2585]: I0710 18:41:09.555138 2585 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-cluster-samples-operator/cluster-samples-operator-65db858894-74x7s" event={"ID":"04ed092b-56ca-425f-bc70-cafaec37f312","Type":"ContainerStarted","Data":"5ada103ecf34dbcc352d16f757fdda3c71e2810fe6eaff003b6da3482cc7aebb"} Jul 10 18:41:10.559581 ip-10-0-143-17 kubenswrapper[2585]: I0710 18:41:10.559530 2585 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-image-registry/image-registry-5bd858b6bc-fkzlq" event={"ID":"f361ac0b-29f3-49e4-910a-1b98ff415751","Type":"ContainerStarted","Data":"ceb21d1f1cc836b4afec41d50d339f32c289358ba2aa8fa1cc181345a0421fc2"} Jul 10 18:41:10.560040 ip-10-0-143-17 kubenswrapper[2585]: I0710 18:41:10.559668 2585 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-image-registry/image-registry-5bd858b6bc-fkzlq" Jul 10 18:41:10.580552 ip-10-0-143-17 kubenswrapper[2585]: I0710 18:41:10.580492 2585 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-image-registry/image-registry-5bd858b6bc-fkzlq" podStartSLOduration=17.580476838 podStartE2EDuration="17.580476838s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-10 18:40:53 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-10 18:41:10.579473913 +0000 UTC m=+84.036833869" watchObservedRunningTime="2026-07-10 18:41:10.580476838 +0000 UTC m=+84.037836773" Jul 10 18:41:11.564220 ip-10-0-143-17 kubenswrapper[2585]: I0710 18:41:11.564187 2585 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-cluster-samples-operator/cluster-samples-operator-65db858894-74x7s" event={"ID":"04ed092b-56ca-425f-bc70-cafaec37f312","Type":"ContainerStarted","Data":"0aac5e3c21c46a4486f8fe60af799ea90125243bf810e547d486686552a66c47"} Jul 10 18:41:11.564591 ip-10-0-143-17 kubenswrapper[2585]: I0710 18:41:11.564245 2585 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-cluster-samples-operator/cluster-samples-operator-65db858894-74x7s" event={"ID":"04ed092b-56ca-425f-bc70-cafaec37f312","Type":"ContainerStarted","Data":"e8d2b7329ba2d59fdec134fcdb2124e6e9f496e918846b537a1b75251976e4c3"} Jul 10 18:41:11.585172 ip-10-0-143-17 kubenswrapper[2585]: I0710 18:41:11.585126 2585 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-cluster-samples-operator/cluster-samples-operator-65db858894-74x7s" podStartSLOduration=17.007617807 podStartE2EDuration="18.585109751s" totalImagesPullingTime="1.577491944s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-10 18:40:53 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-10 18:41:11.584074617 +0000 UTC m=+85.041434571" watchObservedRunningTime="2026-07-10 18:41:11.585109751 +0000 UTC m=+85.042469685" Jul 10 18:41:15.246736 ip-10-0-143-17 kubenswrapper[2585]: I0710 18:41:15.246705 2585 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"insights-runtime-extractor-tls\" (UniqueName: \"kubernetes.io/secret/06325764-d12d-4aa8-9314-71a3a2c49d12-insights-runtime-extractor-tls\") pod \"insights-runtime-extractor-rd8jt\" (UID: \"06325764-d12d-4aa8-9314-71a3a2c49d12\") " pod="openshift-insights/insights-runtime-extractor-rd8jt" Jul 10 18:41:15.247192 ip-10-0-143-17 kubenswrapper[2585]: I0710 18:41:15.246746 2585 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-rbac-proxy-cm\" (UniqueName: \"kubernetes.io/configmap/06325764-d12d-4aa8-9314-71a3a2c49d12-kube-rbac-proxy-cm\") pod \"insights-runtime-extractor-rd8jt\" (UID: \"06325764-d12d-4aa8-9314-71a3a2c49d12\") " pod="openshift-insights/insights-runtime-extractor-rd8jt" Jul 10 18:41:15.247192 ip-10-0-143-17 kubenswrapper[2585]: E0710 18:41:15.246866 2585 secret.go:189] Couldn't get secret openshift-insights/insights-runtime-extractor-tls: secret "insights-runtime-extractor-tls" not found Jul 10 18:41:15.247192 ip-10-0-143-17 kubenswrapper[2585]: E0710 18:41:15.246925 2585 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/06325764-d12d-4aa8-9314-71a3a2c49d12-insights-runtime-extractor-tls podName:06325764-d12d-4aa8-9314-71a3a2c49d12 nodeName:}" failed. No retries permitted until 2026-07-10 18:41:31.246911183 +0000 UTC m=+104.704271129 (durationBeforeRetry 16s). Error: MountVolume.SetUp failed for volume "insights-runtime-extractor-tls" (UniqueName: "kubernetes.io/secret/06325764-d12d-4aa8-9314-71a3a2c49d12-insights-runtime-extractor-tls") pod "insights-runtime-extractor-rd8jt" (UID: "06325764-d12d-4aa8-9314-71a3a2c49d12") : secret "insights-runtime-extractor-tls" not found Jul 10 18:41:15.247403 ip-10-0-143-17 kubenswrapper[2585]: I0710 18:41:15.247377 2585 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-rbac-proxy-cm\" (UniqueName: \"kubernetes.io/configmap/06325764-d12d-4aa8-9314-71a3a2c49d12-kube-rbac-proxy-cm\") pod \"insights-runtime-extractor-rd8jt\" (UID: \"06325764-d12d-4aa8-9314-71a3a2c49d12\") " pod="openshift-insights/insights-runtime-extractor-rd8jt" Jul 10 18:41:19.952650 ip-10-0-143-17 kubenswrapper[2585]: I0710 18:41:19.952618 2585 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-image-registry/image-registry-5bd858b6bc-fkzlq"] Jul 10 18:41:20.063049 ip-10-0-143-17 kubenswrapper[2585]: I0710 18:41:20.063023 2585 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-image-registry/image-registry-bb888c97d-64x5s"] Jul 10 18:41:20.066810 ip-10-0-143-17 kubenswrapper[2585]: I0710 18:41:20.066796 2585 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-image-registry/image-registry-bb888c97d-64x5s" Jul 10 18:41:20.092031 ip-10-0-143-17 kubenswrapper[2585]: I0710 18:41:20.092009 2585 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-image-registry/image-registry-bb888c97d-64x5s"] Jul 10 18:41:20.186844 ip-10-0-143-17 kubenswrapper[2585]: I0710 18:41:20.186811 2585 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"registry-certificates\" (UniqueName: \"kubernetes.io/configmap/b623c78b-f3ba-4fe7-b967-ab6cabb56ec7-registry-certificates\") pod \"image-registry-bb888c97d-64x5s\" (UID: \"b623c78b-f3ba-4fe7-b967-ab6cabb56ec7\") " pod="openshift-image-registry/image-registry-bb888c97d-64x5s" Jul 10 18:41:20.186965 ip-10-0-143-17 kubenswrapper[2585]: I0710 18:41:20.186849 2585 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"ca-trust-extracted\" (UniqueName: \"kubernetes.io/empty-dir/b623c78b-f3ba-4fe7-b967-ab6cabb56ec7-ca-trust-extracted\") pod \"image-registry-bb888c97d-64x5s\" (UID: \"b623c78b-f3ba-4fe7-b967-ab6cabb56ec7\") " pod="openshift-image-registry/image-registry-bb888c97d-64x5s" Jul 10 18:41:20.186965 ip-10-0-143-17 kubenswrapper[2585]: I0710 18:41:20.186893 2585 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"bound-sa-token\" (UniqueName: \"kubernetes.io/projected/b623c78b-f3ba-4fe7-b967-ab6cabb56ec7-bound-sa-token\") pod \"image-registry-bb888c97d-64x5s\" (UID: \"b623c78b-f3ba-4fe7-b967-ab6cabb56ec7\") " pod="openshift-image-registry/image-registry-bb888c97d-64x5s" Jul 10 18:41:20.186965 ip-10-0-143-17 kubenswrapper[2585]: I0710 18:41:20.186945 2585 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"trusted-ca\" (UniqueName: \"kubernetes.io/configmap/b623c78b-f3ba-4fe7-b967-ab6cabb56ec7-trusted-ca\") pod \"image-registry-bb888c97d-64x5s\" (UID: \"b623c78b-f3ba-4fe7-b967-ab6cabb56ec7\") " pod="openshift-image-registry/image-registry-bb888c97d-64x5s" Jul 10 18:41:20.187106 ip-10-0-143-17 kubenswrapper[2585]: I0710 18:41:20.186992 2585 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"image-registry-private-configuration\" (UniqueName: \"kubernetes.io/secret/b623c78b-f3ba-4fe7-b967-ab6cabb56ec7-image-registry-private-configuration\") pod \"image-registry-bb888c97d-64x5s\" (UID: \"b623c78b-f3ba-4fe7-b967-ab6cabb56ec7\") " pod="openshift-image-registry/image-registry-bb888c97d-64x5s" Jul 10 18:41:20.187106 ip-10-0-143-17 kubenswrapper[2585]: I0710 18:41:20.187048 2585 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-n6pd7\" (UniqueName: \"kubernetes.io/projected/b623c78b-f3ba-4fe7-b967-ab6cabb56ec7-kube-api-access-n6pd7\") pod \"image-registry-bb888c97d-64x5s\" (UID: \"b623c78b-f3ba-4fe7-b967-ab6cabb56ec7\") " pod="openshift-image-registry/image-registry-bb888c97d-64x5s" Jul 10 18:41:20.187106 ip-10-0-143-17 kubenswrapper[2585]: I0710 18:41:20.187085 2585 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"installation-pull-secrets\" (UniqueName: \"kubernetes.io/secret/b623c78b-f3ba-4fe7-b967-ab6cabb56ec7-installation-pull-secrets\") pod \"image-registry-bb888c97d-64x5s\" (UID: \"b623c78b-f3ba-4fe7-b967-ab6cabb56ec7\") " pod="openshift-image-registry/image-registry-bb888c97d-64x5s" Jul 10 18:41:20.187209 ip-10-0-143-17 kubenswrapper[2585]: I0710 18:41:20.187125 2585 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"registry-tls\" (UniqueName: \"kubernetes.io/projected/b623c78b-f3ba-4fe7-b967-ab6cabb56ec7-registry-tls\") pod \"image-registry-bb888c97d-64x5s\" (UID: \"b623c78b-f3ba-4fe7-b967-ab6cabb56ec7\") " pod="openshift-image-registry/image-registry-bb888c97d-64x5s" Jul 10 18:41:20.288052 ip-10-0-143-17 kubenswrapper[2585]: I0710 18:41:20.288024 2585 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"registry-certificates\" (UniqueName: \"kubernetes.io/configmap/b623c78b-f3ba-4fe7-b967-ab6cabb56ec7-registry-certificates\") pod \"image-registry-bb888c97d-64x5s\" (UID: \"b623c78b-f3ba-4fe7-b967-ab6cabb56ec7\") " pod="openshift-image-registry/image-registry-bb888c97d-64x5s" Jul 10 18:41:20.288179 ip-10-0-143-17 kubenswrapper[2585]: I0710 18:41:20.288055 2585 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"ca-trust-extracted\" (UniqueName: \"kubernetes.io/empty-dir/b623c78b-f3ba-4fe7-b967-ab6cabb56ec7-ca-trust-extracted\") pod \"image-registry-bb888c97d-64x5s\" (UID: \"b623c78b-f3ba-4fe7-b967-ab6cabb56ec7\") " pod="openshift-image-registry/image-registry-bb888c97d-64x5s" Jul 10 18:41:20.288179 ip-10-0-143-17 kubenswrapper[2585]: I0710 18:41:20.288084 2585 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"bound-sa-token\" (UniqueName: \"kubernetes.io/projected/b623c78b-f3ba-4fe7-b967-ab6cabb56ec7-bound-sa-token\") pod \"image-registry-bb888c97d-64x5s\" (UID: \"b623c78b-f3ba-4fe7-b967-ab6cabb56ec7\") " pod="openshift-image-registry/image-registry-bb888c97d-64x5s" Jul 10 18:41:20.288179 ip-10-0-143-17 kubenswrapper[2585]: I0710 18:41:20.288107 2585 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"trusted-ca\" (UniqueName: \"kubernetes.io/configmap/b623c78b-f3ba-4fe7-b967-ab6cabb56ec7-trusted-ca\") pod \"image-registry-bb888c97d-64x5s\" (UID: \"b623c78b-f3ba-4fe7-b967-ab6cabb56ec7\") " pod="openshift-image-registry/image-registry-bb888c97d-64x5s" Jul 10 18:41:20.288179 ip-10-0-143-17 kubenswrapper[2585]: I0710 18:41:20.288135 2585 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"image-registry-private-configuration\" (UniqueName: \"kubernetes.io/secret/b623c78b-f3ba-4fe7-b967-ab6cabb56ec7-image-registry-private-configuration\") pod \"image-registry-bb888c97d-64x5s\" (UID: \"b623c78b-f3ba-4fe7-b967-ab6cabb56ec7\") " pod="openshift-image-registry/image-registry-bb888c97d-64x5s" Jul 10 18:41:20.288179 ip-10-0-143-17 kubenswrapper[2585]: I0710 18:41:20.288153 2585 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-n6pd7\" (UniqueName: \"kubernetes.io/projected/b623c78b-f3ba-4fe7-b967-ab6cabb56ec7-kube-api-access-n6pd7\") pod \"image-registry-bb888c97d-64x5s\" (UID: \"b623c78b-f3ba-4fe7-b967-ab6cabb56ec7\") " pod="openshift-image-registry/image-registry-bb888c97d-64x5s" Jul 10 18:41:20.288179 ip-10-0-143-17 kubenswrapper[2585]: I0710 18:41:20.288171 2585 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"installation-pull-secrets\" (UniqueName: \"kubernetes.io/secret/b623c78b-f3ba-4fe7-b967-ab6cabb56ec7-installation-pull-secrets\") pod \"image-registry-bb888c97d-64x5s\" (UID: \"b623c78b-f3ba-4fe7-b967-ab6cabb56ec7\") " pod="openshift-image-registry/image-registry-bb888c97d-64x5s" Jul 10 18:41:20.288469 ip-10-0-143-17 kubenswrapper[2585]: I0710 18:41:20.288208 2585 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"registry-tls\" (UniqueName: \"kubernetes.io/projected/b623c78b-f3ba-4fe7-b967-ab6cabb56ec7-registry-tls\") pod \"image-registry-bb888c97d-64x5s\" (UID: \"b623c78b-f3ba-4fe7-b967-ab6cabb56ec7\") " pod="openshift-image-registry/image-registry-bb888c97d-64x5s" Jul 10 18:41:20.288524 ip-10-0-143-17 kubenswrapper[2585]: I0710 18:41:20.288501 2585 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"ca-trust-extracted\" (UniqueName: \"kubernetes.io/empty-dir/b623c78b-f3ba-4fe7-b967-ab6cabb56ec7-ca-trust-extracted\") pod \"image-registry-bb888c97d-64x5s\" (UID: \"b623c78b-f3ba-4fe7-b967-ab6cabb56ec7\") " pod="openshift-image-registry/image-registry-bb888c97d-64x5s" Jul 10 18:41:20.288928 ip-10-0-143-17 kubenswrapper[2585]: I0710 18:41:20.288901 2585 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"registry-certificates\" (UniqueName: \"kubernetes.io/configmap/b623c78b-f3ba-4fe7-b967-ab6cabb56ec7-registry-certificates\") pod \"image-registry-bb888c97d-64x5s\" (UID: \"b623c78b-f3ba-4fe7-b967-ab6cabb56ec7\") " pod="openshift-image-registry/image-registry-bb888c97d-64x5s" Jul 10 18:41:20.289224 ip-10-0-143-17 kubenswrapper[2585]: I0710 18:41:20.289203 2585 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"trusted-ca\" (UniqueName: \"kubernetes.io/configmap/b623c78b-f3ba-4fe7-b967-ab6cabb56ec7-trusted-ca\") pod \"image-registry-bb888c97d-64x5s\" (UID: \"b623c78b-f3ba-4fe7-b967-ab6cabb56ec7\") " pod="openshift-image-registry/image-registry-bb888c97d-64x5s" Jul 10 18:41:20.290622 ip-10-0-143-17 kubenswrapper[2585]: I0710 18:41:20.290604 2585 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"registry-tls\" (UniqueName: \"kubernetes.io/projected/b623c78b-f3ba-4fe7-b967-ab6cabb56ec7-registry-tls\") pod \"image-registry-bb888c97d-64x5s\" (UID: \"b623c78b-f3ba-4fe7-b967-ab6cabb56ec7\") " pod="openshift-image-registry/image-registry-bb888c97d-64x5s" Jul 10 18:41:20.290768 ip-10-0-143-17 kubenswrapper[2585]: I0710 18:41:20.290746 2585 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"image-registry-private-configuration\" (UniqueName: \"kubernetes.io/secret/b623c78b-f3ba-4fe7-b967-ab6cabb56ec7-image-registry-private-configuration\") pod \"image-registry-bb888c97d-64x5s\" (UID: \"b623c78b-f3ba-4fe7-b967-ab6cabb56ec7\") " pod="openshift-image-registry/image-registry-bb888c97d-64x5s" Jul 10 18:41:20.290768 ip-10-0-143-17 kubenswrapper[2585]: I0710 18:41:20.290759 2585 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"installation-pull-secrets\" (UniqueName: \"kubernetes.io/secret/b623c78b-f3ba-4fe7-b967-ab6cabb56ec7-installation-pull-secrets\") pod \"image-registry-bb888c97d-64x5s\" (UID: \"b623c78b-f3ba-4fe7-b967-ab6cabb56ec7\") " pod="openshift-image-registry/image-registry-bb888c97d-64x5s" Jul 10 18:41:20.297097 ip-10-0-143-17 kubenswrapper[2585]: I0710 18:41:20.297075 2585 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-n6pd7\" (UniqueName: \"kubernetes.io/projected/b623c78b-f3ba-4fe7-b967-ab6cabb56ec7-kube-api-access-n6pd7\") pod \"image-registry-bb888c97d-64x5s\" (UID: \"b623c78b-f3ba-4fe7-b967-ab6cabb56ec7\") " pod="openshift-image-registry/image-registry-bb888c97d-64x5s" Jul 10 18:41:20.307170 ip-10-0-143-17 kubenswrapper[2585]: I0710 18:41:20.307150 2585 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"bound-sa-token\" (UniqueName: \"kubernetes.io/projected/b623c78b-f3ba-4fe7-b967-ab6cabb56ec7-bound-sa-token\") pod \"image-registry-bb888c97d-64x5s\" (UID: \"b623c78b-f3ba-4fe7-b967-ab6cabb56ec7\") " pod="openshift-image-registry/image-registry-bb888c97d-64x5s" Jul 10 18:41:20.375580 ip-10-0-143-17 kubenswrapper[2585]: I0710 18:41:20.375559 2585 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-image-registry/image-registry-bb888c97d-64x5s" Jul 10 18:41:20.518137 ip-10-0-143-17 kubenswrapper[2585]: I0710 18:41:20.518107 2585 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-image-registry/image-registry-bb888c97d-64x5s"] Jul 10 18:41:20.521013 ip-10-0-143-17 kubenswrapper[2585]: W0710 18:41:20.520980 2585 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podb623c78b_f3ba_4fe7_b967_ab6cabb56ec7.slice/crio-9f65e5d8b3289176d7f9760d8d7550790396d696f99088d8c0e4d4ca8ff386c0 WatchSource:0}: Error finding container 9f65e5d8b3289176d7f9760d8d7550790396d696f99088d8c0e4d4ca8ff386c0: Status 404 returned error can't find the container with id 9f65e5d8b3289176d7f9760d8d7550790396d696f99088d8c0e4d4ca8ff386c0 Jul 10 18:41:20.592660 ip-10-0-143-17 kubenswrapper[2585]: I0710 18:41:20.592630 2585 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-image-registry/image-registry-bb888c97d-64x5s" event={"ID":"b623c78b-f3ba-4fe7-b967-ab6cabb56ec7","Type":"ContainerStarted","Data":"d7b50319140c0c577a9a51d2349180684fbac78839f14f0fac13033f2a572c78"} Jul 10 18:41:20.592763 ip-10-0-143-17 kubenswrapper[2585]: I0710 18:41:20.592681 2585 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-image-registry/image-registry-bb888c97d-64x5s" event={"ID":"b623c78b-f3ba-4fe7-b967-ab6cabb56ec7","Type":"ContainerStarted","Data":"9f65e5d8b3289176d7f9760d8d7550790396d696f99088d8c0e4d4ca8ff386c0"} Jul 10 18:41:20.592763 ip-10-0-143-17 kubenswrapper[2585]: I0710 18:41:20.592721 2585 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-image-registry/image-registry-bb888c97d-64x5s" Jul 10 18:41:20.616391 ip-10-0-143-17 kubenswrapper[2585]: I0710 18:41:20.616334 2585 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-image-registry/image-registry-bb888c97d-64x5s" podStartSLOduration=0.616320815 podStartE2EDuration="616.320815ms" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-10 18:41:20 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-10 18:41:20.615848223 +0000 UTC m=+94.073208178" watchObservedRunningTime="2026-07-10 18:41:20.616320815 +0000 UTC m=+94.073680750" Jul 10 18:41:25.125309 ip-10-0-143-17 kubenswrapper[2585]: I0710 18:41:25.125271 2585 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/bf853b15-a7e2-4f66-ace3-b1f1fc9323de-metrics-certs\") pod \"router-default-5d998f6f9d-t8frx\" (UID: \"bf853b15-a7e2-4f66-ace3-b1f1fc9323de\") " pod="openshift-ingress/router-default-5d998f6f9d-t8frx" Jul 10 18:41:25.125692 ip-10-0-143-17 kubenswrapper[2585]: I0710 18:41:25.125336 2585 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"service-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/bf853b15-a7e2-4f66-ace3-b1f1fc9323de-service-ca-bundle\") pod \"router-default-5d998f6f9d-t8frx\" (UID: \"bf853b15-a7e2-4f66-ace3-b1f1fc9323de\") " pod="openshift-ingress/router-default-5d998f6f9d-t8frx" Jul 10 18:41:25.125906 ip-10-0-143-17 kubenswrapper[2585]: I0710 18:41:25.125887 2585 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"service-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/bf853b15-a7e2-4f66-ace3-b1f1fc9323de-service-ca-bundle\") pod \"router-default-5d998f6f9d-t8frx\" (UID: \"bf853b15-a7e2-4f66-ace3-b1f1fc9323de\") " pod="openshift-ingress/router-default-5d998f6f9d-t8frx" Jul 10 18:41:25.127489 ip-10-0-143-17 kubenswrapper[2585]: I0710 18:41:25.127467 2585 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/bf853b15-a7e2-4f66-ace3-b1f1fc9323de-metrics-certs\") pod \"router-default-5d998f6f9d-t8frx\" (UID: \"bf853b15-a7e2-4f66-ace3-b1f1fc9323de\") " pod="openshift-ingress/router-default-5d998f6f9d-t8frx" Jul 10 18:41:25.226436 ip-10-0-143-17 kubenswrapper[2585]: I0710 18:41:25.226409 2585 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"cluster-monitoring-operator-tls\" (UniqueName: \"kubernetes.io/secret/45e87367-8dde-4c37-b642-f38ead4d2b04-cluster-monitoring-operator-tls\") pod \"cluster-monitoring-operator-5f7f494dd4-q52h2\" (UID: \"45e87367-8dde-4c37-b642-f38ead4d2b04\") " pod="openshift-monitoring/cluster-monitoring-operator-5f7f494dd4-q52h2" Jul 10 18:41:25.228624 ip-10-0-143-17 kubenswrapper[2585]: I0710 18:41:25.228595 2585 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"cluster-monitoring-operator-tls\" (UniqueName: \"kubernetes.io/secret/45e87367-8dde-4c37-b642-f38ead4d2b04-cluster-monitoring-operator-tls\") pod \"cluster-monitoring-operator-5f7f494dd4-q52h2\" (UID: \"45e87367-8dde-4c37-b642-f38ead4d2b04\") " pod="openshift-monitoring/cluster-monitoring-operator-5f7f494dd4-q52h2" Jul 10 18:41:25.340005 ip-10-0-143-17 kubenswrapper[2585]: I0710 18:41:25.339973 2585 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/router-default-5d998f6f9d-t8frx" Jul 10 18:41:25.453081 ip-10-0-143-17 kubenswrapper[2585]: I0710 18:41:25.453054 2585 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-monitoring/cluster-monitoring-operator-5f7f494dd4-q52h2" Jul 10 18:41:25.468070 ip-10-0-143-17 kubenswrapper[2585]: I0710 18:41:25.468042 2585 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/router-default-5d998f6f9d-t8frx"] Jul 10 18:41:25.470850 ip-10-0-143-17 kubenswrapper[2585]: W0710 18:41:25.470826 2585 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podbf853b15_a7e2_4f66_ace3_b1f1fc9323de.slice/crio-c1dce5563f364e6ac8391fd81770376cf97ffa042256c3e9c15698e086375ebf WatchSource:0}: Error finding container c1dce5563f364e6ac8391fd81770376cf97ffa042256c3e9c15698e086375ebf: Status 404 returned error can't find the container with id c1dce5563f364e6ac8391fd81770376cf97ffa042256c3e9c15698e086375ebf Jul 10 18:41:25.581896 ip-10-0-143-17 kubenswrapper[2585]: I0710 18:41:25.581866 2585 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-monitoring/cluster-monitoring-operator-5f7f494dd4-q52h2"] Jul 10 18:41:25.584759 ip-10-0-143-17 kubenswrapper[2585]: W0710 18:41:25.584729 2585 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod45e87367_8dde_4c37_b642_f38ead4d2b04.slice/crio-70ea5c756c9ef5a98147a62fd387aca17d4cd21e231d06c8cbea38b62430ce68 WatchSource:0}: Error finding container 70ea5c756c9ef5a98147a62fd387aca17d4cd21e231d06c8cbea38b62430ce68: Status 404 returned error can't find the container with id 70ea5c756c9ef5a98147a62fd387aca17d4cd21e231d06c8cbea38b62430ce68 Jul 10 18:41:25.608905 ip-10-0-143-17 kubenswrapper[2585]: I0710 18:41:25.608877 2585 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/cluster-monitoring-operator-5f7f494dd4-q52h2" event={"ID":"45e87367-8dde-4c37-b642-f38ead4d2b04","Type":"ContainerStarted","Data":"70ea5c756c9ef5a98147a62fd387aca17d4cd21e231d06c8cbea38b62430ce68"} Jul 10 18:41:25.610201 ip-10-0-143-17 kubenswrapper[2585]: I0710 18:41:25.610180 2585 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/router-default-5d998f6f9d-t8frx" event={"ID":"bf853b15-a7e2-4f66-ace3-b1f1fc9323de","Type":"ContainerStarted","Data":"3a71616f7cc416e7f2ac2fa3f86119d92b5e7ed84136d5b6a8b7437d7a6ebc38"} Jul 10 18:41:25.610294 ip-10-0-143-17 kubenswrapper[2585]: I0710 18:41:25.610222 2585 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/router-default-5d998f6f9d-t8frx" event={"ID":"bf853b15-a7e2-4f66-ace3-b1f1fc9323de","Type":"ContainerStarted","Data":"c1dce5563f364e6ac8391fd81770376cf97ffa042256c3e9c15698e086375ebf"} Jul 10 18:41:25.630997 ip-10-0-143-17 kubenswrapper[2585]: I0710 18:41:25.630939 2585 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-ingress/router-default-5d998f6f9d-t8frx" podStartSLOduration=32.630920522 podStartE2EDuration="32.630920522s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-10 18:40:53 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-10 18:41:25.62947405 +0000 UTC m=+99.086833984" watchObservedRunningTime="2026-07-10 18:41:25.630920522 +0000 UTC m=+99.088280459" Jul 10 18:41:25.729715 ip-10-0-143-17 kubenswrapper[2585]: I0710 18:41:25.729683 2585 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-tls\" (UniqueName: \"kubernetes.io/secret/3c50e2a2-37c9-45b8-9ad0-fc624a151d61-metrics-tls\") pod \"dns-default-bwks6\" (UID: \"3c50e2a2-37c9-45b8-9ad0-fc624a151d61\") " pod="openshift-dns/dns-default-bwks6" Jul 10 18:41:25.731800 ip-10-0-143-17 kubenswrapper[2585]: I0710 18:41:25.731779 2585 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"metrics-tls\" (UniqueName: \"kubernetes.io/secret/3c50e2a2-37c9-45b8-9ad0-fc624a151d61-metrics-tls\") pod \"dns-default-bwks6\" (UID: \"3c50e2a2-37c9-45b8-9ad0-fc624a151d61\") " pod="openshift-dns/dns-default-bwks6" Jul 10 18:41:25.833344 ip-10-0-143-17 kubenswrapper[2585]: I0710 18:41:25.833316 2585 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-dns\"/\"dns-dockercfg-4x2b4\"" Jul 10 18:41:25.841242 ip-10-0-143-17 kubenswrapper[2585]: I0710 18:41:25.841226 2585 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-dns/dns-default-bwks6" Jul 10 18:41:25.965995 ip-10-0-143-17 kubenswrapper[2585]: I0710 18:41:25.965851 2585 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-dns/dns-default-bwks6"] Jul 10 18:41:25.968291 ip-10-0-143-17 kubenswrapper[2585]: W0710 18:41:25.968266 2585 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod3c50e2a2_37c9_45b8_9ad0_fc624a151d61.slice/crio-6561e9797ef34ac2323122fdfb23943846cd5db8adc9ea81cdf865e0fc06c9e7 WatchSource:0}: Error finding container 6561e9797ef34ac2323122fdfb23943846cd5db8adc9ea81cdf865e0fc06c9e7: Status 404 returned error can't find the container with id 6561e9797ef34ac2323122fdfb23943846cd5db8adc9ea81cdf865e0fc06c9e7 Jul 10 18:41:26.341113 ip-10-0-143-17 kubenswrapper[2585]: I0710 18:41:26.341078 2585 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="openshift-ingress/router-default-5d998f6f9d-t8frx" Jul 10 18:41:26.343860 ip-10-0-143-17 kubenswrapper[2585]: I0710 18:41:26.343838 2585 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="openshift-ingress/router-default-5d998f6f9d-t8frx" Jul 10 18:41:26.615601 ip-10-0-143-17 kubenswrapper[2585]: I0710 18:41:26.615495 2585 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-dns/dns-default-bwks6" event={"ID":"3c50e2a2-37c9-45b8-9ad0-fc624a151d61","Type":"ContainerStarted","Data":"6561e9797ef34ac2323122fdfb23943846cd5db8adc9ea81cdf865e0fc06c9e7"} Jul 10 18:41:26.615808 ip-10-0-143-17 kubenswrapper[2585]: I0710 18:41:26.615794 2585 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/router-default-5d998f6f9d-t8frx" Jul 10 18:41:26.617153 ip-10-0-143-17 kubenswrapper[2585]: I0710 18:41:26.617132 2585 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/router-default-5d998f6f9d-t8frx" Jul 10 18:41:28.534137 ip-10-0-143-17 kubenswrapper[2585]: I0710 18:41:28.534099 2585 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-monitoring/prometheus-operator-admission-webhook-77d98bc786-287c6"] Jul 10 18:41:28.537423 ip-10-0-143-17 kubenswrapper[2585]: I0710 18:41:28.537402 2585 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-monitoring/prometheus-operator-admission-webhook-77d98bc786-287c6" Jul 10 18:41:28.541483 ip-10-0-143-17 kubenswrapper[2585]: I0710 18:41:28.541457 2585 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"prometheus-operator-admission-webhook-tls\"" Jul 10 18:41:28.541483 ip-10-0-143-17 kubenswrapper[2585]: I0710 18:41:28.541459 2585 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"prometheus-operator-admission-webhook-dockercfg-f7f6c\"" Jul 10 18:41:28.551888 ip-10-0-143-17 kubenswrapper[2585]: I0710 18:41:28.551871 2585 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-monitoring/prometheus-operator-admission-webhook-77d98bc786-287c6"] Jul 10 18:41:28.624682 ip-10-0-143-17 kubenswrapper[2585]: I0710 18:41:28.624653 2585 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/cluster-monitoring-operator-5f7f494dd4-q52h2" event={"ID":"45e87367-8dde-4c37-b642-f38ead4d2b04","Type":"ContainerStarted","Data":"aaed68fbe783b080f43ec246fa14b279509a4bc26920560cef1b5e79f1411942"} Jul 10 18:41:28.626306 ip-10-0-143-17 kubenswrapper[2585]: I0710 18:41:28.626285 2585 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-dns/dns-default-bwks6" event={"ID":"3c50e2a2-37c9-45b8-9ad0-fc624a151d61","Type":"ContainerStarted","Data":"3c7f3f35ff6b4f798c5b33cbed26b718f89ca41ff61d42940dc20960f931c11c"} Jul 10 18:41:28.626407 ip-10-0-143-17 kubenswrapper[2585]: I0710 18:41:28.626332 2585 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-dns/dns-default-bwks6" event={"ID":"3c50e2a2-37c9-45b8-9ad0-fc624a151d61","Type":"ContainerStarted","Data":"cb590e1870f08bba4b28381207758181a352d5a9477540f4f41ebab57e85b66b"} Jul 10 18:41:28.650381 ip-10-0-143-17 kubenswrapper[2585]: I0710 18:41:28.650335 2585 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-monitoring/cluster-monitoring-operator-5f7f494dd4-q52h2" podStartSLOduration=33.314051046 podStartE2EDuration="35.650321375s" totalImagesPullingTime="2.336270329s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-10 18:40:53 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-10 18:41:28.650194691 +0000 UTC m=+102.107554625" watchObservedRunningTime="2026-07-10 18:41:28.650321375 +0000 UTC m=+102.107681307" Jul 10 18:41:28.654444 ip-10-0-143-17 kubenswrapper[2585]: I0710 18:41:28.654423 2585 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tls-certificates\" (UniqueName: \"kubernetes.io/secret/062015e4-0bbf-4327-ab65-bd64b305da35-tls-certificates\") pod \"prometheus-operator-admission-webhook-77d98bc786-287c6\" (UID: \"062015e4-0bbf-4327-ab65-bd64b305da35\") " pod="openshift-monitoring/prometheus-operator-admission-webhook-77d98bc786-287c6" Jul 10 18:41:28.677714 ip-10-0-143-17 kubenswrapper[2585]: I0710 18:41:28.677677 2585 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-dns/dns-default-bwks6" podStartSLOduration=65.726366797 podStartE2EDuration="1m7.677665112s" totalImagesPullingTime="1.951298315s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-10 18:40:21 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-10 18:41:28.676726051 +0000 UTC m=+102.134085985" watchObservedRunningTime="2026-07-10 18:41:28.677665112 +0000 UTC m=+102.135025046" Jul 10 18:41:28.755740 ip-10-0-143-17 kubenswrapper[2585]: I0710 18:41:28.755699 2585 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tls-certificates\" (UniqueName: \"kubernetes.io/secret/062015e4-0bbf-4327-ab65-bd64b305da35-tls-certificates\") pod \"prometheus-operator-admission-webhook-77d98bc786-287c6\" (UID: \"062015e4-0bbf-4327-ab65-bd64b305da35\") " pod="openshift-monitoring/prometheus-operator-admission-webhook-77d98bc786-287c6" Jul 10 18:41:28.758002 ip-10-0-143-17 kubenswrapper[2585]: I0710 18:41:28.757981 2585 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"tls-certificates\" (UniqueName: \"kubernetes.io/secret/062015e4-0bbf-4327-ab65-bd64b305da35-tls-certificates\") pod \"prometheus-operator-admission-webhook-77d98bc786-287c6\" (UID: \"062015e4-0bbf-4327-ab65-bd64b305da35\") " pod="openshift-monitoring/prometheus-operator-admission-webhook-77d98bc786-287c6" Jul 10 18:41:28.846138 ip-10-0-143-17 kubenswrapper[2585]: I0710 18:41:28.846085 2585 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-monitoring/prometheus-operator-admission-webhook-77d98bc786-287c6" Jul 10 18:41:28.975410 ip-10-0-143-17 kubenswrapper[2585]: I0710 18:41:28.975388 2585 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-monitoring/prometheus-operator-admission-webhook-77d98bc786-287c6"] Jul 10 18:41:28.976656 ip-10-0-143-17 kubenswrapper[2585]: W0710 18:41:28.976629 2585 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod062015e4_0bbf_4327_ab65_bd64b305da35.slice/crio-85ec909c8f3af3a415bd2725b24123795918a0c8ba458ac9f5217c5565bbfb73 WatchSource:0}: Error finding container 85ec909c8f3af3a415bd2725b24123795918a0c8ba458ac9f5217c5565bbfb73: Status 404 returned error can't find the container with id 85ec909c8f3af3a415bd2725b24123795918a0c8ba458ac9f5217c5565bbfb73 Jul 10 18:41:29.630408 ip-10-0-143-17 kubenswrapper[2585]: I0710 18:41:29.630376 2585 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/prometheus-operator-admission-webhook-77d98bc786-287c6" event={"ID":"062015e4-0bbf-4327-ab65-bd64b305da35","Type":"ContainerStarted","Data":"85ec909c8f3af3a415bd2725b24123795918a0c8ba458ac9f5217c5565bbfb73"} Jul 10 18:41:29.630762 ip-10-0-143-17 kubenswrapper[2585]: I0710 18:41:29.630741 2585 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-dns/dns-default-bwks6" Jul 10 18:41:29.958606 ip-10-0-143-17 kubenswrapper[2585]: I0710 18:41:29.958523 2585 patch_prober.go:28] interesting pod/image-registry-5bd858b6bc-fkzlq container/registry namespace/openshift-image-registry: Readiness probe status=failure output="HTTP probe failed with statuscode: 503" start-of-body={"errors":[{"code":"UNAVAILABLE","message":"service unavailable","detail":"health check failed: please see /debug/health"}]} Jul 10 18:41:29.958606 ip-10-0-143-17 kubenswrapper[2585]: I0710 18:41:29.958586 2585 prober.go:120] "Probe failed" probeType="Readiness" pod="openshift-image-registry/image-registry-5bd858b6bc-fkzlq" podUID="f361ac0b-29f3-49e4-910a-1b98ff415751" containerName="registry" probeResult="failure" output="HTTP probe failed with statuscode: 503" Jul 10 18:41:30.333877 ip-10-0-143-17 kubenswrapper[2585]: I0710 18:41:30.333851 2585 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-console/console-7dcf5c576-vqjdl"] Jul 10 18:41:30.351460 ip-10-0-143-17 kubenswrapper[2585]: I0710 18:41:30.351434 2585 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-console/console-7dcf5c576-vqjdl"] Jul 10 18:41:30.351579 ip-10-0-143-17 kubenswrapper[2585]: I0710 18:41:30.351513 2585 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-console/console-7dcf5c576-vqjdl" Jul 10 18:41:30.354739 ip-10-0-143-17 kubenswrapper[2585]: I0710 18:41:30.354706 2585 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-console\"/\"oauth-serving-cert\"" Jul 10 18:41:30.354820 ip-10-0-143-17 kubenswrapper[2585]: I0710 18:41:30.354708 2585 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-console\"/\"console-serving-cert\"" Jul 10 18:41:30.354971 ip-10-0-143-17 kubenswrapper[2585]: I0710 18:41:30.354954 2585 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-console\"/\"kube-root-ca.crt\"" Jul 10 18:41:30.355785 ip-10-0-143-17 kubenswrapper[2585]: I0710 18:41:30.355768 2585 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-console\"/\"console-oauth-config\"" Jul 10 18:41:30.355889 ip-10-0-143-17 kubenswrapper[2585]: I0710 18:41:30.355768 2585 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-console\"/\"console-config\"" Jul 10 18:41:30.355889 ip-10-0-143-17 kubenswrapper[2585]: I0710 18:41:30.355770 2585 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-console\"/\"openshift-service-ca.crt\"" Jul 10 18:41:30.356048 ip-10-0-143-17 kubenswrapper[2585]: I0710 18:41:30.355770 2585 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-console\"/\"console-dockercfg-m6xbs\"" Jul 10 18:41:30.356048 ip-10-0-143-17 kubenswrapper[2585]: I0710 18:41:30.355823 2585 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-console\"/\"service-ca\"" Jul 10 18:41:30.468492 ip-10-0-143-17 kubenswrapper[2585]: I0710 18:41:30.468467 2585 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"console-config\" (UniqueName: \"kubernetes.io/configmap/dc58c93b-5a2c-4f55-ae5e-f9cddf66101e-console-config\") pod \"console-7dcf5c576-vqjdl\" (UID: \"dc58c93b-5a2c-4f55-ae5e-f9cddf66101e\") " pod="openshift-console/console-7dcf5c576-vqjdl" Jul 10 18:41:30.468604 ip-10-0-143-17 kubenswrapper[2585]: I0710 18:41:30.468508 2585 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"console-serving-cert\" (UniqueName: \"kubernetes.io/secret/dc58c93b-5a2c-4f55-ae5e-f9cddf66101e-console-serving-cert\") pod \"console-7dcf5c576-vqjdl\" (UID: \"dc58c93b-5a2c-4f55-ae5e-f9cddf66101e\") " pod="openshift-console/console-7dcf5c576-vqjdl" Jul 10 18:41:30.468604 ip-10-0-143-17 kubenswrapper[2585]: I0710 18:41:30.468529 2585 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"oauth-serving-cert\" (UniqueName: \"kubernetes.io/configmap/dc58c93b-5a2c-4f55-ae5e-f9cddf66101e-oauth-serving-cert\") pod \"console-7dcf5c576-vqjdl\" (UID: \"dc58c93b-5a2c-4f55-ae5e-f9cddf66101e\") " pod="openshift-console/console-7dcf5c576-vqjdl" Jul 10 18:41:30.468604 ip-10-0-143-17 kubenswrapper[2585]: I0710 18:41:30.468582 2585 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"service-ca\" (UniqueName: \"kubernetes.io/configmap/dc58c93b-5a2c-4f55-ae5e-f9cddf66101e-service-ca\") pod \"console-7dcf5c576-vqjdl\" (UID: \"dc58c93b-5a2c-4f55-ae5e-f9cddf66101e\") " pod="openshift-console/console-7dcf5c576-vqjdl" Jul 10 18:41:30.468714 ip-10-0-143-17 kubenswrapper[2585]: I0710 18:41:30.468625 2585 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-tzf5j\" (UniqueName: \"kubernetes.io/projected/dc58c93b-5a2c-4f55-ae5e-f9cddf66101e-kube-api-access-tzf5j\") pod \"console-7dcf5c576-vqjdl\" (UID: \"dc58c93b-5a2c-4f55-ae5e-f9cddf66101e\") " pod="openshift-console/console-7dcf5c576-vqjdl" Jul 10 18:41:30.468714 ip-10-0-143-17 kubenswrapper[2585]: I0710 18:41:30.468676 2585 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"console-oauth-config\" (UniqueName: \"kubernetes.io/secret/dc58c93b-5a2c-4f55-ae5e-f9cddf66101e-console-oauth-config\") pod \"console-7dcf5c576-vqjdl\" (UID: \"dc58c93b-5a2c-4f55-ae5e-f9cddf66101e\") " pod="openshift-console/console-7dcf5c576-vqjdl" Jul 10 18:41:30.528168 ip-10-0-143-17 kubenswrapper[2585]: I0710 18:41:30.528148 2585 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-network-diagnostics/network-check-target-7tsng" Jul 10 18:41:30.570049 ip-10-0-143-17 kubenswrapper[2585]: I0710 18:41:30.570024 2585 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"console-oauth-config\" (UniqueName: \"kubernetes.io/secret/dc58c93b-5a2c-4f55-ae5e-f9cddf66101e-console-oauth-config\") pod \"console-7dcf5c576-vqjdl\" (UID: \"dc58c93b-5a2c-4f55-ae5e-f9cddf66101e\") " pod="openshift-console/console-7dcf5c576-vqjdl" Jul 10 18:41:30.570186 ip-10-0-143-17 kubenswrapper[2585]: I0710 18:41:30.570168 2585 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"console-config\" (UniqueName: \"kubernetes.io/configmap/dc58c93b-5a2c-4f55-ae5e-f9cddf66101e-console-config\") pod \"console-7dcf5c576-vqjdl\" (UID: \"dc58c93b-5a2c-4f55-ae5e-f9cddf66101e\") " pod="openshift-console/console-7dcf5c576-vqjdl" Jul 10 18:41:30.570233 ip-10-0-143-17 kubenswrapper[2585]: I0710 18:41:30.570223 2585 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"console-serving-cert\" (UniqueName: \"kubernetes.io/secret/dc58c93b-5a2c-4f55-ae5e-f9cddf66101e-console-serving-cert\") pod \"console-7dcf5c576-vqjdl\" (UID: \"dc58c93b-5a2c-4f55-ae5e-f9cddf66101e\") " pod="openshift-console/console-7dcf5c576-vqjdl" Jul 10 18:41:30.570265 ip-10-0-143-17 kubenswrapper[2585]: I0710 18:41:30.570252 2585 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"oauth-serving-cert\" (UniqueName: \"kubernetes.io/configmap/dc58c93b-5a2c-4f55-ae5e-f9cddf66101e-oauth-serving-cert\") pod \"console-7dcf5c576-vqjdl\" (UID: \"dc58c93b-5a2c-4f55-ae5e-f9cddf66101e\") " pod="openshift-console/console-7dcf5c576-vqjdl" Jul 10 18:41:30.570297 ip-10-0-143-17 kubenswrapper[2585]: I0710 18:41:30.570277 2585 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"service-ca\" (UniqueName: \"kubernetes.io/configmap/dc58c93b-5a2c-4f55-ae5e-f9cddf66101e-service-ca\") pod \"console-7dcf5c576-vqjdl\" (UID: \"dc58c93b-5a2c-4f55-ae5e-f9cddf66101e\") " pod="openshift-console/console-7dcf5c576-vqjdl" Jul 10 18:41:30.570341 ip-10-0-143-17 kubenswrapper[2585]: I0710 18:41:30.570316 2585 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-tzf5j\" (UniqueName: \"kubernetes.io/projected/dc58c93b-5a2c-4f55-ae5e-f9cddf66101e-kube-api-access-tzf5j\") pod \"console-7dcf5c576-vqjdl\" (UID: \"dc58c93b-5a2c-4f55-ae5e-f9cddf66101e\") " pod="openshift-console/console-7dcf5c576-vqjdl" Jul 10 18:41:30.570998 ip-10-0-143-17 kubenswrapper[2585]: I0710 18:41:30.570979 2585 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"oauth-serving-cert\" (UniqueName: \"kubernetes.io/configmap/dc58c93b-5a2c-4f55-ae5e-f9cddf66101e-oauth-serving-cert\") pod \"console-7dcf5c576-vqjdl\" (UID: \"dc58c93b-5a2c-4f55-ae5e-f9cddf66101e\") " pod="openshift-console/console-7dcf5c576-vqjdl" Jul 10 18:41:30.571103 ip-10-0-143-17 kubenswrapper[2585]: I0710 18:41:30.571023 2585 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"service-ca\" (UniqueName: \"kubernetes.io/configmap/dc58c93b-5a2c-4f55-ae5e-f9cddf66101e-service-ca\") pod \"console-7dcf5c576-vqjdl\" (UID: \"dc58c93b-5a2c-4f55-ae5e-f9cddf66101e\") " pod="openshift-console/console-7dcf5c576-vqjdl" Jul 10 18:41:30.571103 ip-10-0-143-17 kubenswrapper[2585]: I0710 18:41:30.571030 2585 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"console-config\" (UniqueName: \"kubernetes.io/configmap/dc58c93b-5a2c-4f55-ae5e-f9cddf66101e-console-config\") pod \"console-7dcf5c576-vqjdl\" (UID: \"dc58c93b-5a2c-4f55-ae5e-f9cddf66101e\") " pod="openshift-console/console-7dcf5c576-vqjdl" Jul 10 18:41:30.572390 ip-10-0-143-17 kubenswrapper[2585]: I0710 18:41:30.572372 2585 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"console-oauth-config\" (UniqueName: \"kubernetes.io/secret/dc58c93b-5a2c-4f55-ae5e-f9cddf66101e-console-oauth-config\") pod \"console-7dcf5c576-vqjdl\" (UID: \"dc58c93b-5a2c-4f55-ae5e-f9cddf66101e\") " pod="openshift-console/console-7dcf5c576-vqjdl" Jul 10 18:41:30.572891 ip-10-0-143-17 kubenswrapper[2585]: I0710 18:41:30.572870 2585 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"console-serving-cert\" (UniqueName: \"kubernetes.io/secret/dc58c93b-5a2c-4f55-ae5e-f9cddf66101e-console-serving-cert\") pod \"console-7dcf5c576-vqjdl\" (UID: \"dc58c93b-5a2c-4f55-ae5e-f9cddf66101e\") " pod="openshift-console/console-7dcf5c576-vqjdl" Jul 10 18:41:30.579961 ip-10-0-143-17 kubenswrapper[2585]: I0710 18:41:30.579913 2585 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-tzf5j\" (UniqueName: \"kubernetes.io/projected/dc58c93b-5a2c-4f55-ae5e-f9cddf66101e-kube-api-access-tzf5j\") pod \"console-7dcf5c576-vqjdl\" (UID: \"dc58c93b-5a2c-4f55-ae5e-f9cddf66101e\") " pod="openshift-console/console-7dcf5c576-vqjdl" Jul 10 18:41:30.635373 ip-10-0-143-17 kubenswrapper[2585]: I0710 18:41:30.635338 2585 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/prometheus-operator-admission-webhook-77d98bc786-287c6" event={"ID":"062015e4-0bbf-4327-ab65-bd64b305da35","Type":"ContainerStarted","Data":"cf313eea94574358ad4a1c2e513c2a15a57094d99ec333dfefa0af24ecdb21da"} Jul 10 18:41:30.635711 ip-10-0-143-17 kubenswrapper[2585]: I0710 18:41:30.635497 2585 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-monitoring/prometheus-operator-admission-webhook-77d98bc786-287c6" Jul 10 18:41:30.639924 ip-10-0-143-17 kubenswrapper[2585]: I0710 18:41:30.639902 2585 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-monitoring/prometheus-operator-admission-webhook-77d98bc786-287c6" Jul 10 18:41:30.654367 ip-10-0-143-17 kubenswrapper[2585]: I0710 18:41:30.654331 2585 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-monitoring/prometheus-operator-admission-webhook-77d98bc786-287c6" podStartSLOduration=1.301633743 podStartE2EDuration="2.654318739s" totalImagesPullingTime="1.352684996s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-10 18:41:28 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-10 18:41:30.653857841 +0000 UTC m=+104.111217792" watchObservedRunningTime="2026-07-10 18:41:30.654318739 +0000 UTC m=+104.111678675" Jul 10 18:41:30.668585 ip-10-0-143-17 kubenswrapper[2585]: I0710 18:41:30.668566 2585 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-console/console-7dcf5c576-vqjdl" Jul 10 18:41:30.823492 ip-10-0-143-17 kubenswrapper[2585]: I0710 18:41:30.823469 2585 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-console/console-7dcf5c576-vqjdl"] Jul 10 18:41:30.825803 ip-10-0-143-17 kubenswrapper[2585]: W0710 18:41:30.825776 2585 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-poddc58c93b_5a2c_4f55_ae5e_f9cddf66101e.slice/crio-7b1cb2a245e10fcc3433b153d3068c749d1e1b66d09473c9dfc3b77d0e946bb8 WatchSource:0}: Error finding container 7b1cb2a245e10fcc3433b153d3068c749d1e1b66d09473c9dfc3b77d0e946bb8: Status 404 returned error can't find the container with id 7b1cb2a245e10fcc3433b153d3068c749d1e1b66d09473c9dfc3b77d0e946bb8 Jul 10 18:41:31.276315 ip-10-0-143-17 kubenswrapper[2585]: I0710 18:41:31.276285 2585 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"insights-runtime-extractor-tls\" (UniqueName: \"kubernetes.io/secret/06325764-d12d-4aa8-9314-71a3a2c49d12-insights-runtime-extractor-tls\") pod \"insights-runtime-extractor-rd8jt\" (UID: \"06325764-d12d-4aa8-9314-71a3a2c49d12\") " pod="openshift-insights/insights-runtime-extractor-rd8jt" Jul 10 18:41:31.278405 ip-10-0-143-17 kubenswrapper[2585]: I0710 18:41:31.278387 2585 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"insights-runtime-extractor-tls\" (UniqueName: \"kubernetes.io/secret/06325764-d12d-4aa8-9314-71a3a2c49d12-insights-runtime-extractor-tls\") pod \"insights-runtime-extractor-rd8jt\" (UID: \"06325764-d12d-4aa8-9314-71a3a2c49d12\") " pod="openshift-insights/insights-runtime-extractor-rd8jt" Jul 10 18:41:31.575934 ip-10-0-143-17 kubenswrapper[2585]: I0710 18:41:31.575845 2585 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-insights/insights-runtime-extractor-rd8jt" Jul 10 18:41:31.642231 ip-10-0-143-17 kubenswrapper[2585]: I0710 18:41:31.642193 2585 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-console/console-7dcf5c576-vqjdl" event={"ID":"dc58c93b-5a2c-4f55-ae5e-f9cddf66101e","Type":"ContainerStarted","Data":"7b1cb2a245e10fcc3433b153d3068c749d1e1b66d09473c9dfc3b77d0e946bb8"} Jul 10 18:41:31.679688 ip-10-0-143-17 kubenswrapper[2585]: I0710 18:41:31.679654 2585 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-monitoring/prometheus-operator-64785b6675-vcwsb"] Jul 10 18:41:31.711837 ip-10-0-143-17 kubenswrapper[2585]: I0710 18:41:31.711812 2585 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-monitoring/prometheus-operator-64785b6675-vcwsb"] Jul 10 18:41:31.711958 ip-10-0-143-17 kubenswrapper[2585]: I0710 18:41:31.711937 2585 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-monitoring/prometheus-operator-64785b6675-vcwsb" Jul 10 18:41:31.714904 ip-10-0-143-17 kubenswrapper[2585]: I0710 18:41:31.714883 2585 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-monitoring\"/\"metrics-client-ca\"" Jul 10 18:41:31.716465 ip-10-0-143-17 kubenswrapper[2585]: I0710 18:41:31.716444 2585 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"prometheus-operator-tls\"" Jul 10 18:41:31.716609 ip-10-0-143-17 kubenswrapper[2585]: I0710 18:41:31.716474 2585 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"prometheus-operator-dockercfg-6rjtl\"" Jul 10 18:41:31.716684 ip-10-0-143-17 kubenswrapper[2585]: I0710 18:41:31.716636 2585 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"prometheus-operator-kube-rbac-proxy-config\"" Jul 10 18:41:31.763421 ip-10-0-143-17 kubenswrapper[2585]: I0710 18:41:31.763312 2585 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-insights/insights-runtime-extractor-rd8jt"] Jul 10 18:41:31.766365 ip-10-0-143-17 kubenswrapper[2585]: W0710 18:41:31.766343 2585 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod06325764_d12d_4aa8_9314_71a3a2c49d12.slice/crio-91621de3619e7287cc4d225a32fd0be88fdf90b9cacf01549e57b094b5e1a0f5 WatchSource:0}: Error finding container 91621de3619e7287cc4d225a32fd0be88fdf90b9cacf01549e57b094b5e1a0f5: Status 404 returned error can't find the container with id 91621de3619e7287cc4d225a32fd0be88fdf90b9cacf01549e57b094b5e1a0f5 Jul 10 18:41:31.881044 ip-10-0-143-17 kubenswrapper[2585]: I0710 18:41:31.880972 2585 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-mm2wl\" (UniqueName: \"kubernetes.io/projected/34730388-7ab6-4c38-a961-a7b2ca5f2b98-kube-api-access-mm2wl\") pod \"prometheus-operator-64785b6675-vcwsb\" (UID: \"34730388-7ab6-4c38-a961-a7b2ca5f2b98\") " pod="openshift-monitoring/prometheus-operator-64785b6675-vcwsb" Jul 10 18:41:31.881044 ip-10-0-143-17 kubenswrapper[2585]: I0710 18:41:31.881030 2585 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"prometheus-operator-kube-rbac-proxy-config\" (UniqueName: \"kubernetes.io/secret/34730388-7ab6-4c38-a961-a7b2ca5f2b98-prometheus-operator-kube-rbac-proxy-config\") pod \"prometheus-operator-64785b6675-vcwsb\" (UID: \"34730388-7ab6-4c38-a961-a7b2ca5f2b98\") " pod="openshift-monitoring/prometheus-operator-64785b6675-vcwsb" Jul 10 18:41:31.881210 ip-10-0-143-17 kubenswrapper[2585]: I0710 18:41:31.881060 2585 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"metrics-client-ca\" (UniqueName: \"kubernetes.io/configmap/34730388-7ab6-4c38-a961-a7b2ca5f2b98-metrics-client-ca\") pod \"prometheus-operator-64785b6675-vcwsb\" (UID: \"34730388-7ab6-4c38-a961-a7b2ca5f2b98\") " pod="openshift-monitoring/prometheus-operator-64785b6675-vcwsb" Jul 10 18:41:31.881210 ip-10-0-143-17 kubenswrapper[2585]: I0710 18:41:31.881141 2585 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"prometheus-operator-tls\" (UniqueName: \"kubernetes.io/secret/34730388-7ab6-4c38-a961-a7b2ca5f2b98-prometheus-operator-tls\") pod \"prometheus-operator-64785b6675-vcwsb\" (UID: \"34730388-7ab6-4c38-a961-a7b2ca5f2b98\") " pod="openshift-monitoring/prometheus-operator-64785b6675-vcwsb" Jul 10 18:41:31.981489 ip-10-0-143-17 kubenswrapper[2585]: I0710 18:41:31.981467 2585 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-mm2wl\" (UniqueName: \"kubernetes.io/projected/34730388-7ab6-4c38-a961-a7b2ca5f2b98-kube-api-access-mm2wl\") pod \"prometheus-operator-64785b6675-vcwsb\" (UID: \"34730388-7ab6-4c38-a961-a7b2ca5f2b98\") " pod="openshift-monitoring/prometheus-operator-64785b6675-vcwsb" Jul 10 18:41:31.981605 ip-10-0-143-17 kubenswrapper[2585]: I0710 18:41:31.981503 2585 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"prometheus-operator-kube-rbac-proxy-config\" (UniqueName: \"kubernetes.io/secret/34730388-7ab6-4c38-a961-a7b2ca5f2b98-prometheus-operator-kube-rbac-proxy-config\") pod \"prometheus-operator-64785b6675-vcwsb\" (UID: \"34730388-7ab6-4c38-a961-a7b2ca5f2b98\") " pod="openshift-monitoring/prometheus-operator-64785b6675-vcwsb" Jul 10 18:41:31.981605 ip-10-0-143-17 kubenswrapper[2585]: I0710 18:41:31.981522 2585 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-client-ca\" (UniqueName: \"kubernetes.io/configmap/34730388-7ab6-4c38-a961-a7b2ca5f2b98-metrics-client-ca\") pod \"prometheus-operator-64785b6675-vcwsb\" (UID: \"34730388-7ab6-4c38-a961-a7b2ca5f2b98\") " pod="openshift-monitoring/prometheus-operator-64785b6675-vcwsb" Jul 10 18:41:31.981790 ip-10-0-143-17 kubenswrapper[2585]: I0710 18:41:31.981713 2585 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"prometheus-operator-tls\" (UniqueName: \"kubernetes.io/secret/34730388-7ab6-4c38-a961-a7b2ca5f2b98-prometheus-operator-tls\") pod \"prometheus-operator-64785b6675-vcwsb\" (UID: \"34730388-7ab6-4c38-a961-a7b2ca5f2b98\") " pod="openshift-monitoring/prometheus-operator-64785b6675-vcwsb" Jul 10 18:41:31.981853 ip-10-0-143-17 kubenswrapper[2585]: E0710 18:41:31.981831 2585 secret.go:189] Couldn't get secret openshift-monitoring/prometheus-operator-tls: secret "prometheus-operator-tls" not found Jul 10 18:41:31.981907 ip-10-0-143-17 kubenswrapper[2585]: E0710 18:41:31.981896 2585 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/34730388-7ab6-4c38-a961-a7b2ca5f2b98-prometheus-operator-tls podName:34730388-7ab6-4c38-a961-a7b2ca5f2b98 nodeName:}" failed. No retries permitted until 2026-07-10 18:41:32.481877669 +0000 UTC m=+105.939237591 (durationBeforeRetry 500ms). Error: MountVolume.SetUp failed for volume "prometheus-operator-tls" (UniqueName: "kubernetes.io/secret/34730388-7ab6-4c38-a961-a7b2ca5f2b98-prometheus-operator-tls") pod "prometheus-operator-64785b6675-vcwsb" (UID: "34730388-7ab6-4c38-a961-a7b2ca5f2b98") : secret "prometheus-operator-tls" not found Jul 10 18:41:31.982084 ip-10-0-143-17 kubenswrapper[2585]: I0710 18:41:31.982065 2585 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"metrics-client-ca\" (UniqueName: \"kubernetes.io/configmap/34730388-7ab6-4c38-a961-a7b2ca5f2b98-metrics-client-ca\") pod \"prometheus-operator-64785b6675-vcwsb\" (UID: \"34730388-7ab6-4c38-a961-a7b2ca5f2b98\") " pod="openshift-monitoring/prometheus-operator-64785b6675-vcwsb" Jul 10 18:41:31.983764 ip-10-0-143-17 kubenswrapper[2585]: I0710 18:41:31.983742 2585 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"prometheus-operator-kube-rbac-proxy-config\" (UniqueName: \"kubernetes.io/secret/34730388-7ab6-4c38-a961-a7b2ca5f2b98-prometheus-operator-kube-rbac-proxy-config\") pod \"prometheus-operator-64785b6675-vcwsb\" (UID: \"34730388-7ab6-4c38-a961-a7b2ca5f2b98\") " pod="openshift-monitoring/prometheus-operator-64785b6675-vcwsb" Jul 10 18:41:31.994647 ip-10-0-143-17 kubenswrapper[2585]: I0710 18:41:31.994621 2585 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-mm2wl\" (UniqueName: \"kubernetes.io/projected/34730388-7ab6-4c38-a961-a7b2ca5f2b98-kube-api-access-mm2wl\") pod \"prometheus-operator-64785b6675-vcwsb\" (UID: \"34730388-7ab6-4c38-a961-a7b2ca5f2b98\") " pod="openshift-monitoring/prometheus-operator-64785b6675-vcwsb" Jul 10 18:41:32.487778 ip-10-0-143-17 kubenswrapper[2585]: I0710 18:41:32.487734 2585 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"prometheus-operator-tls\" (UniqueName: \"kubernetes.io/secret/34730388-7ab6-4c38-a961-a7b2ca5f2b98-prometheus-operator-tls\") pod \"prometheus-operator-64785b6675-vcwsb\" (UID: \"34730388-7ab6-4c38-a961-a7b2ca5f2b98\") " pod="openshift-monitoring/prometheus-operator-64785b6675-vcwsb" Jul 10 18:41:32.490886 ip-10-0-143-17 kubenswrapper[2585]: I0710 18:41:32.490857 2585 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"prometheus-operator-tls\" (UniqueName: \"kubernetes.io/secret/34730388-7ab6-4c38-a961-a7b2ca5f2b98-prometheus-operator-tls\") pod \"prometheus-operator-64785b6675-vcwsb\" (UID: \"34730388-7ab6-4c38-a961-a7b2ca5f2b98\") " pod="openshift-monitoring/prometheus-operator-64785b6675-vcwsb" Jul 10 18:41:32.624038 ip-10-0-143-17 kubenswrapper[2585]: I0710 18:41:32.624002 2585 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-monitoring/prometheus-operator-64785b6675-vcwsb" Jul 10 18:41:32.648827 ip-10-0-143-17 kubenswrapper[2585]: I0710 18:41:32.648744 2585 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-insights/insights-runtime-extractor-rd8jt" event={"ID":"06325764-d12d-4aa8-9314-71a3a2c49d12","Type":"ContainerStarted","Data":"91621de3619e7287cc4d225a32fd0be88fdf90b9cacf01549e57b094b5e1a0f5"} Jul 10 18:41:32.824255 ip-10-0-143-17 kubenswrapper[2585]: I0710 18:41:32.824112 2585 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-monitoring/prometheus-operator-64785b6675-vcwsb"] Jul 10 18:41:32.830995 ip-10-0-143-17 kubenswrapper[2585]: W0710 18:41:32.829773 2585 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod34730388_7ab6_4c38_a961_a7b2ca5f2b98.slice/crio-f37b75c4354ec9751d4da9b222345aa24a841cb5690c46a558a0623ff0fbc401 WatchSource:0}: Error finding container f37b75c4354ec9751d4da9b222345aa24a841cb5690c46a558a0623ff0fbc401: Status 404 returned error can't find the container with id f37b75c4354ec9751d4da9b222345aa24a841cb5690c46a558a0623ff0fbc401 Jul 10 18:41:33.653872 ip-10-0-143-17 kubenswrapper[2585]: I0710 18:41:33.653824 2585 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/prometheus-operator-64785b6675-vcwsb" event={"ID":"34730388-7ab6-4c38-a961-a7b2ca5f2b98","Type":"ContainerStarted","Data":"f37b75c4354ec9751d4da9b222345aa24a841cb5690c46a558a0623ff0fbc401"} Jul 10 18:41:37.670289 ip-10-0-143-17 kubenswrapper[2585]: I0710 18:41:37.670260 2585 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-insights/insights-runtime-extractor-rd8jt" event={"ID":"06325764-d12d-4aa8-9314-71a3a2c49d12","Type":"ContainerStarted","Data":"df68241457e4a22b9fc7639f62ed8a87e26b797d5ba6d990e5a5f4800e90c099"} Jul 10 18:41:37.670650 ip-10-0-143-17 kubenswrapper[2585]: I0710 18:41:37.670350 2585 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-insights/insights-runtime-extractor-rd8jt" event={"ID":"06325764-d12d-4aa8-9314-71a3a2c49d12","Type":"ContainerStarted","Data":"68e911fe192aaaa6aec59829d4eb182cc524e74495e3ab61963abf3dc48a7c6c"} Jul 10 18:41:37.671907 ip-10-0-143-17 kubenswrapper[2585]: I0710 18:41:37.671882 2585 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/prometheus-operator-64785b6675-vcwsb" event={"ID":"34730388-7ab6-4c38-a961-a7b2ca5f2b98","Type":"ContainerStarted","Data":"d4d64ff4a2b4cd03a723c672a4cff666501edba2033973cda649c88fc4d00e09"} Jul 10 18:41:37.672016 ip-10-0-143-17 kubenswrapper[2585]: I0710 18:41:37.671937 2585 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/prometheus-operator-64785b6675-vcwsb" event={"ID":"34730388-7ab6-4c38-a961-a7b2ca5f2b98","Type":"ContainerStarted","Data":"863c00dbbf5cbf11aa5e6c5ca5262ec391def45771aa443516d2f19071a2b6fe"} Jul 10 18:41:37.673395 ip-10-0-143-17 kubenswrapper[2585]: I0710 18:41:37.673373 2585 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-console/console-7dcf5c576-vqjdl" event={"ID":"dc58c93b-5a2c-4f55-ae5e-f9cddf66101e","Type":"ContainerStarted","Data":"879a6cc4411ed8e54e788f74c065e43973ceaf0b7a3395edc2f13b11e071aeb0"} Jul 10 18:41:37.696975 ip-10-0-143-17 kubenswrapper[2585]: I0710 18:41:37.696927 2585 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-monitoring/prometheus-operator-64785b6675-vcwsb" podStartSLOduration=2.664904452 podStartE2EDuration="6.696914703s" totalImagesPullingTime="4.032010251s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-10 18:41:31 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-10 18:41:37.694701251 +0000 UTC m=+111.152061186" watchObservedRunningTime="2026-07-10 18:41:37.696914703 +0000 UTC m=+111.154274637" Jul 10 18:41:37.714866 ip-10-0-143-17 kubenswrapper[2585]: I0710 18:41:37.714828 2585 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-console/console-7dcf5c576-vqjdl" podStartSLOduration=2.012695939 podStartE2EDuration="7.714815244s" totalImagesPullingTime="5.702119305s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-10 18:41:30 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-10 18:41:37.714422048 +0000 UTC m=+111.171781983" watchObservedRunningTime="2026-07-10 18:41:37.714815244 +0000 UTC m=+111.172175179" Jul 10 18:41:39.638239 ip-10-0-143-17 kubenswrapper[2585]: I0710 18:41:39.638206 2585 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-dns/dns-default-bwks6" Jul 10 18:41:39.958021 ip-10-0-143-17 kubenswrapper[2585]: I0710 18:41:39.957944 2585 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-image-registry/image-registry-5bd858b6bc-fkzlq" Jul 10 18:41:40.312397 ip-10-0-143-17 kubenswrapper[2585]: I0710 18:41:40.312369 2585 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-monitoring/node-exporter-st6xg"] Jul 10 18:41:40.314612 ip-10-0-143-17 kubenswrapper[2585]: I0710 18:41:40.314593 2585 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-monitoring/node-exporter-st6xg" Jul 10 18:41:40.318329 ip-10-0-143-17 kubenswrapper[2585]: I0710 18:41:40.318307 2585 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"node-exporter-tls\"" Jul 10 18:41:40.318441 ip-10-0-143-17 kubenswrapper[2585]: I0710 18:41:40.318365 2585 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"node-exporter-dockercfg-5ggl7\"" Jul 10 18:41:40.318833 ip-10-0-143-17 kubenswrapper[2585]: I0710 18:41:40.318817 2585 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-monitoring\"/\"node-exporter-accelerators-collector-config\"" Jul 10 18:41:40.321861 ip-10-0-143-17 kubenswrapper[2585]: I0710 18:41:40.321847 2585 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"node-exporter-kube-rbac-proxy-config\"" Jul 10 18:41:40.353548 ip-10-0-143-17 kubenswrapper[2585]: I0710 18:41:40.353511 2585 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"sys\" (UniqueName: \"kubernetes.io/host-path/61826ce1-4adb-4270-ae29-d777b005c6d0-sys\") pod \"node-exporter-st6xg\" (UID: \"61826ce1-4adb-4270-ae29-d777b005c6d0\") " pod="openshift-monitoring/node-exporter-st6xg" Jul 10 18:41:40.353727 ip-10-0-143-17 kubenswrapper[2585]: I0710 18:41:40.353587 2585 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"node-exporter-accelerators-collector-config\" (UniqueName: \"kubernetes.io/configmap/61826ce1-4adb-4270-ae29-d777b005c6d0-node-exporter-accelerators-collector-config\") pod \"node-exporter-st6xg\" (UID: \"61826ce1-4adb-4270-ae29-d777b005c6d0\") " pod="openshift-monitoring/node-exporter-st6xg" Jul 10 18:41:40.353727 ip-10-0-143-17 kubenswrapper[2585]: I0710 18:41:40.353634 2585 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"root\" (UniqueName: \"kubernetes.io/host-path/61826ce1-4adb-4270-ae29-d777b005c6d0-root\") pod \"node-exporter-st6xg\" (UID: \"61826ce1-4adb-4270-ae29-d777b005c6d0\") " pod="openshift-monitoring/node-exporter-st6xg" Jul 10 18:41:40.353727 ip-10-0-143-17 kubenswrapper[2585]: I0710 18:41:40.353663 2585 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"node-exporter-kube-rbac-proxy-config\" (UniqueName: \"kubernetes.io/secret/61826ce1-4adb-4270-ae29-d777b005c6d0-node-exporter-kube-rbac-proxy-config\") pod \"node-exporter-st6xg\" (UID: \"61826ce1-4adb-4270-ae29-d777b005c6d0\") " pod="openshift-monitoring/node-exporter-st6xg" Jul 10 18:41:40.353727 ip-10-0-143-17 kubenswrapper[2585]: I0710 18:41:40.353716 2585 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"node-exporter-tls\" (UniqueName: \"kubernetes.io/secret/61826ce1-4adb-4270-ae29-d777b005c6d0-node-exporter-tls\") pod \"node-exporter-st6xg\" (UID: \"61826ce1-4adb-4270-ae29-d777b005c6d0\") " pod="openshift-monitoring/node-exporter-st6xg" Jul 10 18:41:40.353940 ip-10-0-143-17 kubenswrapper[2585]: I0710 18:41:40.353739 2585 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"node-exporter-textfile\" (UniqueName: \"kubernetes.io/empty-dir/61826ce1-4adb-4270-ae29-d777b005c6d0-node-exporter-textfile\") pod \"node-exporter-st6xg\" (UID: \"61826ce1-4adb-4270-ae29-d777b005c6d0\") " pod="openshift-monitoring/node-exporter-st6xg" Jul 10 18:41:40.353940 ip-10-0-143-17 kubenswrapper[2585]: I0710 18:41:40.353757 2585 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"metrics-client-ca\" (UniqueName: \"kubernetes.io/configmap/61826ce1-4adb-4270-ae29-d777b005c6d0-metrics-client-ca\") pod \"node-exporter-st6xg\" (UID: \"61826ce1-4adb-4270-ae29-d777b005c6d0\") " pod="openshift-monitoring/node-exporter-st6xg" Jul 10 18:41:40.353940 ip-10-0-143-17 kubenswrapper[2585]: I0710 18:41:40.353779 2585 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-xcnwv\" (UniqueName: \"kubernetes.io/projected/61826ce1-4adb-4270-ae29-d777b005c6d0-kube-api-access-xcnwv\") pod \"node-exporter-st6xg\" (UID: \"61826ce1-4adb-4270-ae29-d777b005c6d0\") " pod="openshift-monitoring/node-exporter-st6xg" Jul 10 18:41:40.353940 ip-10-0-143-17 kubenswrapper[2585]: I0710 18:41:40.353795 2585 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"node-exporter-wtmp\" (UniqueName: \"kubernetes.io/host-path/61826ce1-4adb-4270-ae29-d777b005c6d0-node-exporter-wtmp\") pod \"node-exporter-st6xg\" (UID: \"61826ce1-4adb-4270-ae29-d777b005c6d0\") " pod="openshift-monitoring/node-exporter-st6xg" Jul 10 18:41:40.420136 ip-10-0-143-17 kubenswrapper[2585]: I0710 18:41:40.420109 2585 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-console/console-8c67b5665-lqvvr"] Jul 10 18:41:40.422251 ip-10-0-143-17 kubenswrapper[2585]: I0710 18:41:40.422234 2585 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-console/console-8c67b5665-lqvvr" Jul 10 18:41:40.434629 ip-10-0-143-17 kubenswrapper[2585]: I0710 18:41:40.434584 2585 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-console\"/\"trusted-ca-bundle\"" Jul 10 18:41:40.436569 ip-10-0-143-17 kubenswrapper[2585]: I0710 18:41:40.436528 2585 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-console/console-8c67b5665-lqvvr"] Jul 10 18:41:40.454532 ip-10-0-143-17 kubenswrapper[2585]: I0710 18:41:40.454503 2585 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-client-ca\" (UniqueName: \"kubernetes.io/configmap/61826ce1-4adb-4270-ae29-d777b005c6d0-metrics-client-ca\") pod \"node-exporter-st6xg\" (UID: \"61826ce1-4adb-4270-ae29-d777b005c6d0\") " pod="openshift-monitoring/node-exporter-st6xg" Jul 10 18:41:40.454634 ip-10-0-143-17 kubenswrapper[2585]: I0710 18:41:40.454548 2585 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"console-config\" (UniqueName: \"kubernetes.io/configmap/f750992a-c8d1-42a2-81e1-c657404d690f-console-config\") pod \"console-8c67b5665-lqvvr\" (UID: \"f750992a-c8d1-42a2-81e1-c657404d690f\") " pod="openshift-console/console-8c67b5665-lqvvr" Jul 10 18:41:40.454634 ip-10-0-143-17 kubenswrapper[2585]: I0710 18:41:40.454580 2585 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-xcnwv\" (UniqueName: \"kubernetes.io/projected/61826ce1-4adb-4270-ae29-d777b005c6d0-kube-api-access-xcnwv\") pod \"node-exporter-st6xg\" (UID: \"61826ce1-4adb-4270-ae29-d777b005c6d0\") " pod="openshift-monitoring/node-exporter-st6xg" Jul 10 18:41:40.454634 ip-10-0-143-17 kubenswrapper[2585]: I0710 18:41:40.454606 2585 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"console-oauth-config\" (UniqueName: \"kubernetes.io/secret/f750992a-c8d1-42a2-81e1-c657404d690f-console-oauth-config\") pod \"console-8c67b5665-lqvvr\" (UID: \"f750992a-c8d1-42a2-81e1-c657404d690f\") " pod="openshift-console/console-8c67b5665-lqvvr" Jul 10 18:41:40.454730 ip-10-0-143-17 kubenswrapper[2585]: I0710 18:41:40.454660 2585 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"node-exporter-wtmp\" (UniqueName: \"kubernetes.io/host-path/61826ce1-4adb-4270-ae29-d777b005c6d0-node-exporter-wtmp\") pod \"node-exporter-st6xg\" (UID: \"61826ce1-4adb-4270-ae29-d777b005c6d0\") " pod="openshift-monitoring/node-exporter-st6xg" Jul 10 18:41:40.454730 ip-10-0-143-17 kubenswrapper[2585]: I0710 18:41:40.454724 2585 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"oauth-serving-cert\" (UniqueName: \"kubernetes.io/configmap/f750992a-c8d1-42a2-81e1-c657404d690f-oauth-serving-cert\") pod \"console-8c67b5665-lqvvr\" (UID: \"f750992a-c8d1-42a2-81e1-c657404d690f\") " pod="openshift-console/console-8c67b5665-lqvvr" Jul 10 18:41:40.454795 ip-10-0-143-17 kubenswrapper[2585]: I0710 18:41:40.454756 2585 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"sys\" (UniqueName: \"kubernetes.io/host-path/61826ce1-4adb-4270-ae29-d777b005c6d0-sys\") pod \"node-exporter-st6xg\" (UID: \"61826ce1-4adb-4270-ae29-d777b005c6d0\") " pod="openshift-monitoring/node-exporter-st6xg" Jul 10 18:41:40.454795 ip-10-0-143-17 kubenswrapper[2585]: I0710 18:41:40.454779 2585 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"console-serving-cert\" (UniqueName: \"kubernetes.io/secret/f750992a-c8d1-42a2-81e1-c657404d690f-console-serving-cert\") pod \"console-8c67b5665-lqvvr\" (UID: \"f750992a-c8d1-42a2-81e1-c657404d690f\") " pod="openshift-console/console-8c67b5665-lqvvr" Jul 10 18:41:40.454887 ip-10-0-143-17 kubenswrapper[2585]: I0710 18:41:40.454824 2585 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"node-exporter-wtmp\" (UniqueName: \"kubernetes.io/host-path/61826ce1-4adb-4270-ae29-d777b005c6d0-node-exporter-wtmp\") pod \"node-exporter-st6xg\" (UID: \"61826ce1-4adb-4270-ae29-d777b005c6d0\") " pod="openshift-monitoring/node-exporter-st6xg" Jul 10 18:41:40.454887 ip-10-0-143-17 kubenswrapper[2585]: I0710 18:41:40.454839 2585 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/f750992a-c8d1-42a2-81e1-c657404d690f-trusted-ca-bundle\") pod \"console-8c67b5665-lqvvr\" (UID: \"f750992a-c8d1-42a2-81e1-c657404d690f\") " pod="openshift-console/console-8c67b5665-lqvvr" Jul 10 18:41:40.454887 ip-10-0-143-17 kubenswrapper[2585]: I0710 18:41:40.454866 2585 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"sys\" (UniqueName: \"kubernetes.io/host-path/61826ce1-4adb-4270-ae29-d777b005c6d0-sys\") pod \"node-exporter-st6xg\" (UID: \"61826ce1-4adb-4270-ae29-d777b005c6d0\") " pod="openshift-monitoring/node-exporter-st6xg" Jul 10 18:41:40.455029 ip-10-0-143-17 kubenswrapper[2585]: I0710 18:41:40.454961 2585 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"node-exporter-accelerators-collector-config\" (UniqueName: \"kubernetes.io/configmap/61826ce1-4adb-4270-ae29-d777b005c6d0-node-exporter-accelerators-collector-config\") pod \"node-exporter-st6xg\" (UID: \"61826ce1-4adb-4270-ae29-d777b005c6d0\") " pod="openshift-monitoring/node-exporter-st6xg" Jul 10 18:41:40.455029 ip-10-0-143-17 kubenswrapper[2585]: I0710 18:41:40.454993 2585 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"root\" (UniqueName: \"kubernetes.io/host-path/61826ce1-4adb-4270-ae29-d777b005c6d0-root\") pod \"node-exporter-st6xg\" (UID: \"61826ce1-4adb-4270-ae29-d777b005c6d0\") " pod="openshift-monitoring/node-exporter-st6xg" Jul 10 18:41:40.455128 ip-10-0-143-17 kubenswrapper[2585]: I0710 18:41:40.455026 2585 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-zs45k\" (UniqueName: \"kubernetes.io/projected/f750992a-c8d1-42a2-81e1-c657404d690f-kube-api-access-zs45k\") pod \"console-8c67b5665-lqvvr\" (UID: \"f750992a-c8d1-42a2-81e1-c657404d690f\") " pod="openshift-console/console-8c67b5665-lqvvr" Jul 10 18:41:40.455128 ip-10-0-143-17 kubenswrapper[2585]: I0710 18:41:40.455043 2585 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"root\" (UniqueName: \"kubernetes.io/host-path/61826ce1-4adb-4270-ae29-d777b005c6d0-root\") pod \"node-exporter-st6xg\" (UID: \"61826ce1-4adb-4270-ae29-d777b005c6d0\") " pod="openshift-monitoring/node-exporter-st6xg" Jul 10 18:41:40.455128 ip-10-0-143-17 kubenswrapper[2585]: I0710 18:41:40.455058 2585 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"node-exporter-kube-rbac-proxy-config\" (UniqueName: \"kubernetes.io/secret/61826ce1-4adb-4270-ae29-d777b005c6d0-node-exporter-kube-rbac-proxy-config\") pod \"node-exporter-st6xg\" (UID: \"61826ce1-4adb-4270-ae29-d777b005c6d0\") " pod="openshift-monitoring/node-exporter-st6xg" Jul 10 18:41:40.455128 ip-10-0-143-17 kubenswrapper[2585]: I0710 18:41:40.455085 2585 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"metrics-client-ca\" (UniqueName: \"kubernetes.io/configmap/61826ce1-4adb-4270-ae29-d777b005c6d0-metrics-client-ca\") pod \"node-exporter-st6xg\" (UID: \"61826ce1-4adb-4270-ae29-d777b005c6d0\") " pod="openshift-monitoring/node-exporter-st6xg" Jul 10 18:41:40.455128 ip-10-0-143-17 kubenswrapper[2585]: I0710 18:41:40.455103 2585 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"node-exporter-tls\" (UniqueName: \"kubernetes.io/secret/61826ce1-4adb-4270-ae29-d777b005c6d0-node-exporter-tls\") pod \"node-exporter-st6xg\" (UID: \"61826ce1-4adb-4270-ae29-d777b005c6d0\") " pod="openshift-monitoring/node-exporter-st6xg" Jul 10 18:41:40.455325 ip-10-0-143-17 kubenswrapper[2585]: I0710 18:41:40.455134 2585 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"node-exporter-textfile\" (UniqueName: \"kubernetes.io/empty-dir/61826ce1-4adb-4270-ae29-d777b005c6d0-node-exporter-textfile\") pod \"node-exporter-st6xg\" (UID: \"61826ce1-4adb-4270-ae29-d777b005c6d0\") " pod="openshift-monitoring/node-exporter-st6xg" Jul 10 18:41:40.455325 ip-10-0-143-17 kubenswrapper[2585]: I0710 18:41:40.455159 2585 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"service-ca\" (UniqueName: \"kubernetes.io/configmap/f750992a-c8d1-42a2-81e1-c657404d690f-service-ca\") pod \"console-8c67b5665-lqvvr\" (UID: \"f750992a-c8d1-42a2-81e1-c657404d690f\") " pod="openshift-console/console-8c67b5665-lqvvr" Jul 10 18:41:40.455325 ip-10-0-143-17 kubenswrapper[2585]: I0710 18:41:40.455321 2585 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"node-exporter-textfile\" (UniqueName: \"kubernetes.io/empty-dir/61826ce1-4adb-4270-ae29-d777b005c6d0-node-exporter-textfile\") pod \"node-exporter-st6xg\" (UID: \"61826ce1-4adb-4270-ae29-d777b005c6d0\") " pod="openshift-monitoring/node-exporter-st6xg" Jul 10 18:41:40.455465 ip-10-0-143-17 kubenswrapper[2585]: I0710 18:41:40.455448 2585 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"node-exporter-accelerators-collector-config\" (UniqueName: \"kubernetes.io/configmap/61826ce1-4adb-4270-ae29-d777b005c6d0-node-exporter-accelerators-collector-config\") pod \"node-exporter-st6xg\" (UID: \"61826ce1-4adb-4270-ae29-d777b005c6d0\") " pod="openshift-monitoring/node-exporter-st6xg" Jul 10 18:41:40.457129 ip-10-0-143-17 kubenswrapper[2585]: I0710 18:41:40.457112 2585 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"node-exporter-kube-rbac-proxy-config\" (UniqueName: \"kubernetes.io/secret/61826ce1-4adb-4270-ae29-d777b005c6d0-node-exporter-kube-rbac-proxy-config\") pod \"node-exporter-st6xg\" (UID: \"61826ce1-4adb-4270-ae29-d777b005c6d0\") " pod="openshift-monitoring/node-exporter-st6xg" Jul 10 18:41:40.457273 ip-10-0-143-17 kubenswrapper[2585]: I0710 18:41:40.457255 2585 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"node-exporter-tls\" (UniqueName: \"kubernetes.io/secret/61826ce1-4adb-4270-ae29-d777b005c6d0-node-exporter-tls\") pod \"node-exporter-st6xg\" (UID: \"61826ce1-4adb-4270-ae29-d777b005c6d0\") " pod="openshift-monitoring/node-exporter-st6xg" Jul 10 18:41:40.468352 ip-10-0-143-17 kubenswrapper[2585]: I0710 18:41:40.468331 2585 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-xcnwv\" (UniqueName: \"kubernetes.io/projected/61826ce1-4adb-4270-ae29-d777b005c6d0-kube-api-access-xcnwv\") pod \"node-exporter-st6xg\" (UID: \"61826ce1-4adb-4270-ae29-d777b005c6d0\") " pod="openshift-monitoring/node-exporter-st6xg" Jul 10 18:41:40.556385 ip-10-0-143-17 kubenswrapper[2585]: I0710 18:41:40.556352 2585 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"console-oauth-config\" (UniqueName: \"kubernetes.io/secret/f750992a-c8d1-42a2-81e1-c657404d690f-console-oauth-config\") pod \"console-8c67b5665-lqvvr\" (UID: \"f750992a-c8d1-42a2-81e1-c657404d690f\") " pod="openshift-console/console-8c67b5665-lqvvr" Jul 10 18:41:40.556595 ip-10-0-143-17 kubenswrapper[2585]: I0710 18:41:40.556413 2585 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"oauth-serving-cert\" (UniqueName: \"kubernetes.io/configmap/f750992a-c8d1-42a2-81e1-c657404d690f-oauth-serving-cert\") pod \"console-8c67b5665-lqvvr\" (UID: \"f750992a-c8d1-42a2-81e1-c657404d690f\") " pod="openshift-console/console-8c67b5665-lqvvr" Jul 10 18:41:40.556595 ip-10-0-143-17 kubenswrapper[2585]: I0710 18:41:40.556442 2585 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"console-serving-cert\" (UniqueName: \"kubernetes.io/secret/f750992a-c8d1-42a2-81e1-c657404d690f-console-serving-cert\") pod \"console-8c67b5665-lqvvr\" (UID: \"f750992a-c8d1-42a2-81e1-c657404d690f\") " pod="openshift-console/console-8c67b5665-lqvvr" Jul 10 18:41:40.556595 ip-10-0-143-17 kubenswrapper[2585]: I0710 18:41:40.556463 2585 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/f750992a-c8d1-42a2-81e1-c657404d690f-trusted-ca-bundle\") pod \"console-8c67b5665-lqvvr\" (UID: \"f750992a-c8d1-42a2-81e1-c657404d690f\") " pod="openshift-console/console-8c67b5665-lqvvr" Jul 10 18:41:40.556595 ip-10-0-143-17 kubenswrapper[2585]: I0710 18:41:40.556495 2585 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-zs45k\" (UniqueName: \"kubernetes.io/projected/f750992a-c8d1-42a2-81e1-c657404d690f-kube-api-access-zs45k\") pod \"console-8c67b5665-lqvvr\" (UID: \"f750992a-c8d1-42a2-81e1-c657404d690f\") " pod="openshift-console/console-8c67b5665-lqvvr" Jul 10 18:41:40.556595 ip-10-0-143-17 kubenswrapper[2585]: I0710 18:41:40.556580 2585 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"service-ca\" (UniqueName: \"kubernetes.io/configmap/f750992a-c8d1-42a2-81e1-c657404d690f-service-ca\") pod \"console-8c67b5665-lqvvr\" (UID: \"f750992a-c8d1-42a2-81e1-c657404d690f\") " pod="openshift-console/console-8c67b5665-lqvvr" Jul 10 18:41:40.556875 ip-10-0-143-17 kubenswrapper[2585]: I0710 18:41:40.556621 2585 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"console-config\" (UniqueName: \"kubernetes.io/configmap/f750992a-c8d1-42a2-81e1-c657404d690f-console-config\") pod \"console-8c67b5665-lqvvr\" (UID: \"f750992a-c8d1-42a2-81e1-c657404d690f\") " pod="openshift-console/console-8c67b5665-lqvvr" Jul 10 18:41:40.557365 ip-10-0-143-17 kubenswrapper[2585]: I0710 18:41:40.557335 2585 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"console-config\" (UniqueName: \"kubernetes.io/configmap/f750992a-c8d1-42a2-81e1-c657404d690f-console-config\") pod \"console-8c67b5665-lqvvr\" (UID: \"f750992a-c8d1-42a2-81e1-c657404d690f\") " pod="openshift-console/console-8c67b5665-lqvvr" Jul 10 18:41:40.557470 ip-10-0-143-17 kubenswrapper[2585]: I0710 18:41:40.557419 2585 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/f750992a-c8d1-42a2-81e1-c657404d690f-trusted-ca-bundle\") pod \"console-8c67b5665-lqvvr\" (UID: \"f750992a-c8d1-42a2-81e1-c657404d690f\") " pod="openshift-console/console-8c67b5665-lqvvr" Jul 10 18:41:40.557470 ip-10-0-143-17 kubenswrapper[2585]: I0710 18:41:40.557446 2585 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"service-ca\" (UniqueName: \"kubernetes.io/configmap/f750992a-c8d1-42a2-81e1-c657404d690f-service-ca\") pod \"console-8c67b5665-lqvvr\" (UID: \"f750992a-c8d1-42a2-81e1-c657404d690f\") " pod="openshift-console/console-8c67b5665-lqvvr" Jul 10 18:41:40.557470 ip-10-0-143-17 kubenswrapper[2585]: I0710 18:41:40.557457 2585 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"oauth-serving-cert\" (UniqueName: \"kubernetes.io/configmap/f750992a-c8d1-42a2-81e1-c657404d690f-oauth-serving-cert\") pod \"console-8c67b5665-lqvvr\" (UID: \"f750992a-c8d1-42a2-81e1-c657404d690f\") " pod="openshift-console/console-8c67b5665-lqvvr" Jul 10 18:41:40.558910 ip-10-0-143-17 kubenswrapper[2585]: I0710 18:41:40.558888 2585 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"console-oauth-config\" (UniqueName: \"kubernetes.io/secret/f750992a-c8d1-42a2-81e1-c657404d690f-console-oauth-config\") pod \"console-8c67b5665-lqvvr\" (UID: \"f750992a-c8d1-42a2-81e1-c657404d690f\") " pod="openshift-console/console-8c67b5665-lqvvr" Jul 10 18:41:40.559008 ip-10-0-143-17 kubenswrapper[2585]: I0710 18:41:40.558993 2585 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"console-serving-cert\" (UniqueName: \"kubernetes.io/secret/f750992a-c8d1-42a2-81e1-c657404d690f-console-serving-cert\") pod \"console-8c67b5665-lqvvr\" (UID: \"f750992a-c8d1-42a2-81e1-c657404d690f\") " pod="openshift-console/console-8c67b5665-lqvvr" Jul 10 18:41:40.570134 ip-10-0-143-17 kubenswrapper[2585]: I0710 18:41:40.570085 2585 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-zs45k\" (UniqueName: \"kubernetes.io/projected/f750992a-c8d1-42a2-81e1-c657404d690f-kube-api-access-zs45k\") pod \"console-8c67b5665-lqvvr\" (UID: \"f750992a-c8d1-42a2-81e1-c657404d690f\") " pod="openshift-console/console-8c67b5665-lqvvr" Jul 10 18:41:40.623282 ip-10-0-143-17 kubenswrapper[2585]: I0710 18:41:40.623257 2585 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-monitoring/node-exporter-st6xg" Jul 10 18:41:40.633923 ip-10-0-143-17 kubenswrapper[2585]: W0710 18:41:40.633896 2585 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod61826ce1_4adb_4270_ae29_d777b005c6d0.slice/crio-b5c325121f5a77a579759558241cf6fb6218fad709e77c0ffb7b23d4f0bd16dc WatchSource:0}: Error finding container b5c325121f5a77a579759558241cf6fb6218fad709e77c0ffb7b23d4f0bd16dc: Status 404 returned error can't find the container with id b5c325121f5a77a579759558241cf6fb6218fad709e77c0ffb7b23d4f0bd16dc Jul 10 18:41:40.668756 ip-10-0-143-17 kubenswrapper[2585]: I0710 18:41:40.668734 2585 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-console/console-7dcf5c576-vqjdl" Jul 10 18:41:40.669035 ip-10-0-143-17 kubenswrapper[2585]: I0710 18:41:40.668787 2585 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="openshift-console/console-7dcf5c576-vqjdl" Jul 10 18:41:40.673140 ip-10-0-143-17 kubenswrapper[2585]: I0710 18:41:40.673123 2585 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="openshift-console/console-7dcf5c576-vqjdl" Jul 10 18:41:40.686925 ip-10-0-143-17 kubenswrapper[2585]: I0710 18:41:40.686900 2585 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/node-exporter-st6xg" event={"ID":"61826ce1-4adb-4270-ae29-d777b005c6d0","Type":"ContainerStarted","Data":"b5c325121f5a77a579759558241cf6fb6218fad709e77c0ffb7b23d4f0bd16dc"} Jul 10 18:41:40.689094 ip-10-0-143-17 kubenswrapper[2585]: I0710 18:41:40.689064 2585 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-insights/insights-runtime-extractor-rd8jt" event={"ID":"06325764-d12d-4aa8-9314-71a3a2c49d12","Type":"ContainerStarted","Data":"e9d99b8e1ebe328d3937f1d38e455a9d2985024d466464f83a3c1f9caecec685"} Jul 10 18:41:40.693145 ip-10-0-143-17 kubenswrapper[2585]: I0710 18:41:40.693126 2585 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-console/console-7dcf5c576-vqjdl" Jul 10 18:41:40.722049 ip-10-0-143-17 kubenswrapper[2585]: I0710 18:41:40.722006 2585 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-insights/insights-runtime-extractor-rd8jt" podStartSLOduration=33.498488479 podStartE2EDuration="41.721994538s" totalImagesPullingTime="8.223506059s" totalInitContainerRuntime="0s" isStatefulPod=true podCreationTimestamp="2026-07-10 18:40:59 +0000 UTC" imagePullSessionsCount=3 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-10 18:41:40.720813432 +0000 UTC m=+114.178173367" watchObservedRunningTime="2026-07-10 18:41:40.721994538 +0000 UTC m=+114.179354472" Jul 10 18:41:40.731019 ip-10-0-143-17 kubenswrapper[2585]: I0710 18:41:40.730994 2585 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-console/console-8c67b5665-lqvvr" Jul 10 18:41:40.863050 ip-10-0-143-17 kubenswrapper[2585]: I0710 18:41:40.862985 2585 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-console/console-8c67b5665-lqvvr"] Jul 10 18:41:40.865778 ip-10-0-143-17 kubenswrapper[2585]: W0710 18:41:40.865754 2585 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podf750992a_c8d1_42a2_81e1_c657404d690f.slice/crio-766eff61f895ed023cbba80139bed951f68e47dc3d512473c115b2a855f47109 WatchSource:0}: Error finding container 766eff61f895ed023cbba80139bed951f68e47dc3d512473c115b2a855f47109: Status 404 returned error can't find the container with id 766eff61f895ed023cbba80139bed951f68e47dc3d512473c115b2a855f47109 Jul 10 18:41:41.601408 ip-10-0-143-17 kubenswrapper[2585]: I0710 18:41:41.601377 2585 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-image-registry/image-registry-bb888c97d-64x5s" Jul 10 18:41:41.697897 ip-10-0-143-17 kubenswrapper[2585]: I0710 18:41:41.697869 2585 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-console/console-8c67b5665-lqvvr" event={"ID":"f750992a-c8d1-42a2-81e1-c657404d690f","Type":"ContainerStarted","Data":"f82b6e47a48ab876dfcf95aae72602cfcca6f9a0e2ffd58cd376d3bf2f95be3c"} Jul 10 18:41:41.698290 ip-10-0-143-17 kubenswrapper[2585]: I0710 18:41:41.697923 2585 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-console/console-8c67b5665-lqvvr" event={"ID":"f750992a-c8d1-42a2-81e1-c657404d690f","Type":"ContainerStarted","Data":"766eff61f895ed023cbba80139bed951f68e47dc3d512473c115b2a855f47109"} Jul 10 18:41:41.741553 ip-10-0-143-17 kubenswrapper[2585]: I0710 18:41:41.741489 2585 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-console/console-8c67b5665-lqvvr" podStartSLOduration=1.741469156 podStartE2EDuration="1.741469156s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-10 18:41:40 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-10 18:41:41.7405882 +0000 UTC m=+115.197948135" watchObservedRunningTime="2026-07-10 18:41:41.741469156 +0000 UTC m=+115.198829092" Jul 10 18:41:42.706871 ip-10-0-143-17 kubenswrapper[2585]: I0710 18:41:42.706834 2585 generic.go:356] "Generic (PLEG): container finished" podID="61826ce1-4adb-4270-ae29-d777b005c6d0" containerID="64abdb2a09fbe18cf98c444b745a12df4508851bb036bf3a1c7f7b3809abe722" exitCode=0 Jul 10 18:41:42.707251 ip-10-0-143-17 kubenswrapper[2585]: I0710 18:41:42.706917 2585 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/node-exporter-st6xg" event={"ID":"61826ce1-4adb-4270-ae29-d777b005c6d0","Type":"ContainerDied","Data":"64abdb2a09fbe18cf98c444b745a12df4508851bb036bf3a1c7f7b3809abe722"} Jul 10 18:41:43.712505 ip-10-0-143-17 kubenswrapper[2585]: I0710 18:41:43.712475 2585 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/node-exporter-st6xg" event={"ID":"61826ce1-4adb-4270-ae29-d777b005c6d0","Type":"ContainerStarted","Data":"70dc8e85cdc6d226f3a918e50aa5f142ecb074ec21e7d45f423904417a56b0fc"} Jul 10 18:41:43.712868 ip-10-0-143-17 kubenswrapper[2585]: I0710 18:41:43.712527 2585 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/node-exporter-st6xg" event={"ID":"61826ce1-4adb-4270-ae29-d777b005c6d0","Type":"ContainerStarted","Data":"6b4cf06f037905045954eb474de30e8795c2ac26b7dd728e780b82a30f7ac46a"} Jul 10 18:41:43.747628 ip-10-0-143-17 kubenswrapper[2585]: I0710 18:41:43.747579 2585 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-monitoring/node-exporter-st6xg" podStartSLOduration=2.688585685 podStartE2EDuration="3.747564581s" totalImagesPullingTime="991.632545ms" totalInitContainerRuntime="67.346351ms" isStatefulPod=true podCreationTimestamp="2026-07-10 18:41:40 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-10 18:41:43.746176073 +0000 UTC m=+117.203536007" watchObservedRunningTime="2026-07-10 18:41:43.747564581 +0000 UTC m=+117.204924550" Jul 10 18:41:44.760386 ip-10-0-143-17 kubenswrapper[2585]: I0710 18:41:44.760354 2585 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-monitoring/metrics-server-688c68676d-bdllx"] Jul 10 18:41:44.762443 ip-10-0-143-17 kubenswrapper[2585]: I0710 18:41:44.762427 2585 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-monitoring/metrics-server-688c68676d-bdllx" Jul 10 18:41:44.766571 ip-10-0-143-17 kubenswrapper[2585]: I0710 18:41:44.766533 2585 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-monitoring\"/\"kubelet-serving-ca-bundle\"" Jul 10 18:41:44.767836 ip-10-0-143-17 kubenswrapper[2585]: I0710 18:41:44.767784 2585 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"metrics-server-8571u5860p67d\"" Jul 10 18:41:44.768042 ip-10-0-143-17 kubenswrapper[2585]: I0710 18:41:44.767820 2585 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-monitoring\"/\"metrics-server-audit-profiles\"" Jul 10 18:41:44.768042 ip-10-0-143-17 kubenswrapper[2585]: I0710 18:41:44.767837 2585 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"metrics-server-dockercfg-77j4w\"" Jul 10 18:41:44.768188 ip-10-0-143-17 kubenswrapper[2585]: I0710 18:41:44.767884 2585 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"metrics-server-client-certs\"" Jul 10 18:41:44.768188 ip-10-0-143-17 kubenswrapper[2585]: I0710 18:41:44.767905 2585 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"metrics-server-tls\"" Jul 10 18:41:44.780431 ip-10-0-143-17 kubenswrapper[2585]: I0710 18:41:44.780412 2585 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-monitoring/metrics-server-688c68676d-bdllx"] Jul 10 18:41:44.791086 ip-10-0-143-17 kubenswrapper[2585]: I0710 18:41:44.791061 2585 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"client-ca-bundle\" (UniqueName: \"kubernetes.io/secret/b44dd6d6-9d73-4626-bfe4-04cf7215ebed-client-ca-bundle\") pod \"metrics-server-688c68676d-bdllx\" (UID: \"b44dd6d6-9d73-4626-bfe4-04cf7215ebed\") " pod="openshift-monitoring/metrics-server-688c68676d-bdllx" Jul 10 18:41:44.791185 ip-10-0-143-17 kubenswrapper[2585]: I0710 18:41:44.791168 2585 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"secret-metrics-server-client-certs\" (UniqueName: \"kubernetes.io/secret/b44dd6d6-9d73-4626-bfe4-04cf7215ebed-secret-metrics-server-client-certs\") pod \"metrics-server-688c68676d-bdllx\" (UID: \"b44dd6d6-9d73-4626-bfe4-04cf7215ebed\") " pod="openshift-monitoring/metrics-server-688c68676d-bdllx" Jul 10 18:41:44.791322 ip-10-0-143-17 kubenswrapper[2585]: I0710 18:41:44.791195 2585 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"audit-log\" (UniqueName: \"kubernetes.io/empty-dir/b44dd6d6-9d73-4626-bfe4-04cf7215ebed-audit-log\") pod \"metrics-server-688c68676d-bdllx\" (UID: \"b44dd6d6-9d73-4626-bfe4-04cf7215ebed\") " pod="openshift-monitoring/metrics-server-688c68676d-bdllx" Jul 10 18:41:44.791322 ip-10-0-143-17 kubenswrapper[2585]: I0710 18:41:44.791279 2585 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"metrics-server-audit-profiles\" (UniqueName: \"kubernetes.io/configmap/b44dd6d6-9d73-4626-bfe4-04cf7215ebed-metrics-server-audit-profiles\") pod \"metrics-server-688c68676d-bdllx\" (UID: \"b44dd6d6-9d73-4626-bfe4-04cf7215ebed\") " pod="openshift-monitoring/metrics-server-688c68676d-bdllx" Jul 10 18:41:44.791463 ip-10-0-143-17 kubenswrapper[2585]: I0710 18:41:44.791351 2585 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"configmap-kubelet-serving-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/b44dd6d6-9d73-4626-bfe4-04cf7215ebed-configmap-kubelet-serving-ca-bundle\") pod \"metrics-server-688c68676d-bdllx\" (UID: \"b44dd6d6-9d73-4626-bfe4-04cf7215ebed\") " pod="openshift-monitoring/metrics-server-688c68676d-bdllx" Jul 10 18:41:44.791463 ip-10-0-143-17 kubenswrapper[2585]: I0710 18:41:44.791433 2585 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"secret-metrics-server-tls\" (UniqueName: \"kubernetes.io/secret/b44dd6d6-9d73-4626-bfe4-04cf7215ebed-secret-metrics-server-tls\") pod \"metrics-server-688c68676d-bdllx\" (UID: \"b44dd6d6-9d73-4626-bfe4-04cf7215ebed\") " pod="openshift-monitoring/metrics-server-688c68676d-bdllx" Jul 10 18:41:44.791608 ip-10-0-143-17 kubenswrapper[2585]: I0710 18:41:44.791460 2585 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-p4kzs\" (UniqueName: \"kubernetes.io/projected/b44dd6d6-9d73-4626-bfe4-04cf7215ebed-kube-api-access-p4kzs\") pod \"metrics-server-688c68676d-bdllx\" (UID: \"b44dd6d6-9d73-4626-bfe4-04cf7215ebed\") " pod="openshift-monitoring/metrics-server-688c68676d-bdllx" Jul 10 18:41:44.892818 ip-10-0-143-17 kubenswrapper[2585]: I0710 18:41:44.892788 2585 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"client-ca-bundle\" (UniqueName: \"kubernetes.io/secret/b44dd6d6-9d73-4626-bfe4-04cf7215ebed-client-ca-bundle\") pod \"metrics-server-688c68676d-bdllx\" (UID: \"b44dd6d6-9d73-4626-bfe4-04cf7215ebed\") " pod="openshift-monitoring/metrics-server-688c68676d-bdllx" Jul 10 18:41:44.892937 ip-10-0-143-17 kubenswrapper[2585]: I0710 18:41:44.892834 2585 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"secret-metrics-server-client-certs\" (UniqueName: \"kubernetes.io/secret/b44dd6d6-9d73-4626-bfe4-04cf7215ebed-secret-metrics-server-client-certs\") pod \"metrics-server-688c68676d-bdllx\" (UID: \"b44dd6d6-9d73-4626-bfe4-04cf7215ebed\") " pod="openshift-monitoring/metrics-server-688c68676d-bdllx" Jul 10 18:41:44.892937 ip-10-0-143-17 kubenswrapper[2585]: I0710 18:41:44.892853 2585 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"audit-log\" (UniqueName: \"kubernetes.io/empty-dir/b44dd6d6-9d73-4626-bfe4-04cf7215ebed-audit-log\") pod \"metrics-server-688c68676d-bdllx\" (UID: \"b44dd6d6-9d73-4626-bfe4-04cf7215ebed\") " pod="openshift-monitoring/metrics-server-688c68676d-bdllx" Jul 10 18:41:44.892937 ip-10-0-143-17 kubenswrapper[2585]: I0710 18:41:44.892873 2585 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-server-audit-profiles\" (UniqueName: \"kubernetes.io/configmap/b44dd6d6-9d73-4626-bfe4-04cf7215ebed-metrics-server-audit-profiles\") pod \"metrics-server-688c68676d-bdllx\" (UID: \"b44dd6d6-9d73-4626-bfe4-04cf7215ebed\") " pod="openshift-monitoring/metrics-server-688c68676d-bdllx" Jul 10 18:41:44.892937 ip-10-0-143-17 kubenswrapper[2585]: I0710 18:41:44.892897 2585 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"configmap-kubelet-serving-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/b44dd6d6-9d73-4626-bfe4-04cf7215ebed-configmap-kubelet-serving-ca-bundle\") pod \"metrics-server-688c68676d-bdllx\" (UID: \"b44dd6d6-9d73-4626-bfe4-04cf7215ebed\") " pod="openshift-monitoring/metrics-server-688c68676d-bdllx" Jul 10 18:41:44.892937 ip-10-0-143-17 kubenswrapper[2585]: I0710 18:41:44.892931 2585 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"secret-metrics-server-tls\" (UniqueName: \"kubernetes.io/secret/b44dd6d6-9d73-4626-bfe4-04cf7215ebed-secret-metrics-server-tls\") pod \"metrics-server-688c68676d-bdllx\" (UID: \"b44dd6d6-9d73-4626-bfe4-04cf7215ebed\") " pod="openshift-monitoring/metrics-server-688c68676d-bdllx" Jul 10 18:41:44.893150 ip-10-0-143-17 kubenswrapper[2585]: I0710 18:41:44.892946 2585 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-p4kzs\" (UniqueName: \"kubernetes.io/projected/b44dd6d6-9d73-4626-bfe4-04cf7215ebed-kube-api-access-p4kzs\") pod \"metrics-server-688c68676d-bdllx\" (UID: \"b44dd6d6-9d73-4626-bfe4-04cf7215ebed\") " pod="openshift-monitoring/metrics-server-688c68676d-bdllx" Jul 10 18:41:44.893316 ip-10-0-143-17 kubenswrapper[2585]: I0710 18:41:44.893285 2585 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"audit-log\" (UniqueName: \"kubernetes.io/empty-dir/b44dd6d6-9d73-4626-bfe4-04cf7215ebed-audit-log\") pod \"metrics-server-688c68676d-bdllx\" (UID: \"b44dd6d6-9d73-4626-bfe4-04cf7215ebed\") " pod="openshift-monitoring/metrics-server-688c68676d-bdllx" Jul 10 18:41:44.893729 ip-10-0-143-17 kubenswrapper[2585]: I0710 18:41:44.893706 2585 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"configmap-kubelet-serving-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/b44dd6d6-9d73-4626-bfe4-04cf7215ebed-configmap-kubelet-serving-ca-bundle\") pod \"metrics-server-688c68676d-bdllx\" (UID: \"b44dd6d6-9d73-4626-bfe4-04cf7215ebed\") " pod="openshift-monitoring/metrics-server-688c68676d-bdllx" Jul 10 18:41:44.894063 ip-10-0-143-17 kubenswrapper[2585]: I0710 18:41:44.894038 2585 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"metrics-server-audit-profiles\" (UniqueName: \"kubernetes.io/configmap/b44dd6d6-9d73-4626-bfe4-04cf7215ebed-metrics-server-audit-profiles\") pod \"metrics-server-688c68676d-bdllx\" (UID: \"b44dd6d6-9d73-4626-bfe4-04cf7215ebed\") " pod="openshift-monitoring/metrics-server-688c68676d-bdllx" Jul 10 18:41:44.895303 ip-10-0-143-17 kubenswrapper[2585]: I0710 18:41:44.895278 2585 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"secret-metrics-server-client-certs\" (UniqueName: \"kubernetes.io/secret/b44dd6d6-9d73-4626-bfe4-04cf7215ebed-secret-metrics-server-client-certs\") pod \"metrics-server-688c68676d-bdllx\" (UID: \"b44dd6d6-9d73-4626-bfe4-04cf7215ebed\") " pod="openshift-monitoring/metrics-server-688c68676d-bdllx" Jul 10 18:41:44.895563 ip-10-0-143-17 kubenswrapper[2585]: I0710 18:41:44.895521 2585 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"secret-metrics-server-tls\" (UniqueName: \"kubernetes.io/secret/b44dd6d6-9d73-4626-bfe4-04cf7215ebed-secret-metrics-server-tls\") pod \"metrics-server-688c68676d-bdllx\" (UID: \"b44dd6d6-9d73-4626-bfe4-04cf7215ebed\") " pod="openshift-monitoring/metrics-server-688c68676d-bdllx" Jul 10 18:41:44.895650 ip-10-0-143-17 kubenswrapper[2585]: I0710 18:41:44.895569 2585 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"client-ca-bundle\" (UniqueName: \"kubernetes.io/secret/b44dd6d6-9d73-4626-bfe4-04cf7215ebed-client-ca-bundle\") pod \"metrics-server-688c68676d-bdllx\" (UID: \"b44dd6d6-9d73-4626-bfe4-04cf7215ebed\") " pod="openshift-monitoring/metrics-server-688c68676d-bdllx" Jul 10 18:41:44.902382 ip-10-0-143-17 kubenswrapper[2585]: I0710 18:41:44.902362 2585 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-p4kzs\" (UniqueName: \"kubernetes.io/projected/b44dd6d6-9d73-4626-bfe4-04cf7215ebed-kube-api-access-p4kzs\") pod \"metrics-server-688c68676d-bdllx\" (UID: \"b44dd6d6-9d73-4626-bfe4-04cf7215ebed\") " pod="openshift-monitoring/metrics-server-688c68676d-bdllx" Jul 10 18:41:44.972170 ip-10-0-143-17 kubenswrapper[2585]: I0710 18:41:44.972136 2585 kuberuntime_container.go:873] "Killing container with a grace period" pod="openshift-image-registry/image-registry-5bd858b6bc-fkzlq" podUID="f361ac0b-29f3-49e4-910a-1b98ff415751" containerName="registry" containerID="cri-o://ceb21d1f1cc836b4afec41d50d339f32c289358ba2aa8fa1cc181345a0421fc2" gracePeriod=30 Jul 10 18:41:45.071586 ip-10-0-143-17 kubenswrapper[2585]: I0710 18:41:45.071513 2585 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-monitoring/metrics-server-688c68676d-bdllx" Jul 10 18:41:45.226783 ip-10-0-143-17 kubenswrapper[2585]: I0710 18:41:45.226762 2585 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-image-registry/image-registry-5bd858b6bc-fkzlq" Jul 10 18:41:45.295755 ip-10-0-143-17 kubenswrapper[2585]: I0710 18:41:45.295728 2585 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"image-registry-private-configuration\" (UniqueName: \"kubernetes.io/secret/f361ac0b-29f3-49e4-910a-1b98ff415751-image-registry-private-configuration\") pod \"f361ac0b-29f3-49e4-910a-1b98ff415751\" (UID: \"f361ac0b-29f3-49e4-910a-1b98ff415751\") " Jul 10 18:41:45.295847 ip-10-0-143-17 kubenswrapper[2585]: I0710 18:41:45.295764 2585 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"installation-pull-secrets\" (UniqueName: \"kubernetes.io/secret/f361ac0b-29f3-49e4-910a-1b98ff415751-installation-pull-secrets\") pod \"f361ac0b-29f3-49e4-910a-1b98ff415751\" (UID: \"f361ac0b-29f3-49e4-910a-1b98ff415751\") " Jul 10 18:41:45.295847 ip-10-0-143-17 kubenswrapper[2585]: I0710 18:41:45.295811 2585 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-jbmkw\" (UniqueName: \"kubernetes.io/projected/f361ac0b-29f3-49e4-910a-1b98ff415751-kube-api-access-jbmkw\") pod \"f361ac0b-29f3-49e4-910a-1b98ff415751\" (UID: \"f361ac0b-29f3-49e4-910a-1b98ff415751\") " Jul 10 18:41:45.295934 ip-10-0-143-17 kubenswrapper[2585]: I0710 18:41:45.295868 2585 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"registry-tls\" (UniqueName: \"kubernetes.io/projected/f361ac0b-29f3-49e4-910a-1b98ff415751-registry-tls\") pod \"f361ac0b-29f3-49e4-910a-1b98ff415751\" (UID: \"f361ac0b-29f3-49e4-910a-1b98ff415751\") " Jul 10 18:41:45.296082 ip-10-0-143-17 kubenswrapper[2585]: I0710 18:41:45.296063 2585 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"ca-trust-extracted\" (UniqueName: \"kubernetes.io/empty-dir/f361ac0b-29f3-49e4-910a-1b98ff415751-ca-trust-extracted\") pod \"f361ac0b-29f3-49e4-910a-1b98ff415751\" (UID: \"f361ac0b-29f3-49e4-910a-1b98ff415751\") " Jul 10 18:41:45.296155 ip-10-0-143-17 kubenswrapper[2585]: I0710 18:41:45.296139 2585 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"trusted-ca\" (UniqueName: \"kubernetes.io/configmap/f361ac0b-29f3-49e4-910a-1b98ff415751-trusted-ca\") pod \"f361ac0b-29f3-49e4-910a-1b98ff415751\" (UID: \"f361ac0b-29f3-49e4-910a-1b98ff415751\") " Jul 10 18:41:45.296206 ip-10-0-143-17 kubenswrapper[2585]: I0710 18:41:45.296192 2585 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"registry-certificates\" (UniqueName: \"kubernetes.io/configmap/f361ac0b-29f3-49e4-910a-1b98ff415751-registry-certificates\") pod \"f361ac0b-29f3-49e4-910a-1b98ff415751\" (UID: \"f361ac0b-29f3-49e4-910a-1b98ff415751\") " Jul 10 18:41:45.296258 ip-10-0-143-17 kubenswrapper[2585]: I0710 18:41:45.296226 2585 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"bound-sa-token\" (UniqueName: \"kubernetes.io/projected/f361ac0b-29f3-49e4-910a-1b98ff415751-bound-sa-token\") pod \"f361ac0b-29f3-49e4-910a-1b98ff415751\" (UID: \"f361ac0b-29f3-49e4-910a-1b98ff415751\") " Jul 10 18:41:45.296777 ip-10-0-143-17 kubenswrapper[2585]: I0710 18:41:45.296745 2585 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/f361ac0b-29f3-49e4-910a-1b98ff415751-registry-certificates" (OuterVolumeSpecName: "registry-certificates") pod "f361ac0b-29f3-49e4-910a-1b98ff415751" (UID: "f361ac0b-29f3-49e4-910a-1b98ff415751"). InnerVolumeSpecName "registry-certificates". PluginName "kubernetes.io/configmap", VolumeGIDValue "" Jul 10 18:41:45.296890 ip-10-0-143-17 kubenswrapper[2585]: I0710 18:41:45.296811 2585 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/f361ac0b-29f3-49e4-910a-1b98ff415751-trusted-ca" (OuterVolumeSpecName: "trusted-ca") pod "f361ac0b-29f3-49e4-910a-1b98ff415751" (UID: "f361ac0b-29f3-49e4-910a-1b98ff415751"). InnerVolumeSpecName "trusted-ca". PluginName "kubernetes.io/configmap", VolumeGIDValue "" Jul 10 18:41:45.298289 ip-10-0-143-17 kubenswrapper[2585]: I0710 18:41:45.298247 2585 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/f361ac0b-29f3-49e4-910a-1b98ff415751-registry-tls" (OuterVolumeSpecName: "registry-tls") pod "f361ac0b-29f3-49e4-910a-1b98ff415751" (UID: "f361ac0b-29f3-49e4-910a-1b98ff415751"). InnerVolumeSpecName "registry-tls". PluginName "kubernetes.io/projected", VolumeGIDValue "" Jul 10 18:41:45.298553 ip-10-0-143-17 kubenswrapper[2585]: I0710 18:41:45.298498 2585 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/f361ac0b-29f3-49e4-910a-1b98ff415751-image-registry-private-configuration" (OuterVolumeSpecName: "image-registry-private-configuration") pod "f361ac0b-29f3-49e4-910a-1b98ff415751" (UID: "f361ac0b-29f3-49e4-910a-1b98ff415751"). InnerVolumeSpecName "image-registry-private-configuration". PluginName "kubernetes.io/secret", VolumeGIDValue "" Jul 10 18:41:45.298645 ip-10-0-143-17 kubenswrapper[2585]: I0710 18:41:45.298585 2585 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/f361ac0b-29f3-49e4-910a-1b98ff415751-kube-api-access-jbmkw" (OuterVolumeSpecName: "kube-api-access-jbmkw") pod "f361ac0b-29f3-49e4-910a-1b98ff415751" (UID: "f361ac0b-29f3-49e4-910a-1b98ff415751"). InnerVolumeSpecName "kube-api-access-jbmkw". PluginName "kubernetes.io/projected", VolumeGIDValue "" Jul 10 18:41:45.298754 ip-10-0-143-17 kubenswrapper[2585]: I0710 18:41:45.298728 2585 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/f361ac0b-29f3-49e4-910a-1b98ff415751-installation-pull-secrets" (OuterVolumeSpecName: "installation-pull-secrets") pod "f361ac0b-29f3-49e4-910a-1b98ff415751" (UID: "f361ac0b-29f3-49e4-910a-1b98ff415751"). InnerVolumeSpecName "installation-pull-secrets". PluginName "kubernetes.io/secret", VolumeGIDValue "" Jul 10 18:41:45.298846 ip-10-0-143-17 kubenswrapper[2585]: I0710 18:41:45.298757 2585 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/f361ac0b-29f3-49e4-910a-1b98ff415751-bound-sa-token" (OuterVolumeSpecName: "bound-sa-token") pod "f361ac0b-29f3-49e4-910a-1b98ff415751" (UID: "f361ac0b-29f3-49e4-910a-1b98ff415751"). InnerVolumeSpecName "bound-sa-token". PluginName "kubernetes.io/projected", VolumeGIDValue "" Jul 10 18:41:45.304775 ip-10-0-143-17 kubenswrapper[2585]: I0710 18:41:45.304754 2585 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/f361ac0b-29f3-49e4-910a-1b98ff415751-ca-trust-extracted" (OuterVolumeSpecName: "ca-trust-extracted") pod "f361ac0b-29f3-49e4-910a-1b98ff415751" (UID: "f361ac0b-29f3-49e4-910a-1b98ff415751"). InnerVolumeSpecName "ca-trust-extracted". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Jul 10 18:41:45.309437 ip-10-0-143-17 kubenswrapper[2585]: I0710 18:41:45.309390 2585 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-monitoring/metrics-server-688c68676d-bdllx"] Jul 10 18:41:45.311882 ip-10-0-143-17 kubenswrapper[2585]: W0710 18:41:45.311859 2585 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podb44dd6d6_9d73_4626_bfe4_04cf7215ebed.slice/crio-c4766daa569a1dab4e31b28a969503d1c6a6b36b0e4c800f6eada6e6b1eee36e WatchSource:0}: Error finding container c4766daa569a1dab4e31b28a969503d1c6a6b36b0e4c800f6eada6e6b1eee36e: Status 404 returned error can't find the container with id c4766daa569a1dab4e31b28a969503d1c6a6b36b0e4c800f6eada6e6b1eee36e Jul 10 18:41:45.397416 ip-10-0-143-17 kubenswrapper[2585]: I0710 18:41:45.397352 2585 reconciler_common.go:299] "Volume detached for volume \"registry-tls\" (UniqueName: \"kubernetes.io/projected/f361ac0b-29f3-49e4-910a-1b98ff415751-registry-tls\") on node \"ip-10-0-143-17.ec2.internal\" DevicePath \"\"" Jul 10 18:41:45.397416 ip-10-0-143-17 kubenswrapper[2585]: I0710 18:41:45.397375 2585 reconciler_common.go:299] "Volume detached for volume \"ca-trust-extracted\" (UniqueName: \"kubernetes.io/empty-dir/f361ac0b-29f3-49e4-910a-1b98ff415751-ca-trust-extracted\") on node \"ip-10-0-143-17.ec2.internal\" DevicePath \"\"" Jul 10 18:41:45.397416 ip-10-0-143-17 kubenswrapper[2585]: I0710 18:41:45.397386 2585 reconciler_common.go:299] "Volume detached for volume \"trusted-ca\" (UniqueName: \"kubernetes.io/configmap/f361ac0b-29f3-49e4-910a-1b98ff415751-trusted-ca\") on node \"ip-10-0-143-17.ec2.internal\" DevicePath \"\"" Jul 10 18:41:45.397416 ip-10-0-143-17 kubenswrapper[2585]: I0710 18:41:45.397396 2585 reconciler_common.go:299] "Volume detached for volume \"registry-certificates\" (UniqueName: \"kubernetes.io/configmap/f361ac0b-29f3-49e4-910a-1b98ff415751-registry-certificates\") on node \"ip-10-0-143-17.ec2.internal\" DevicePath \"\"" Jul 10 18:41:45.397416 ip-10-0-143-17 kubenswrapper[2585]: I0710 18:41:45.397404 2585 reconciler_common.go:299] "Volume detached for volume \"bound-sa-token\" (UniqueName: \"kubernetes.io/projected/f361ac0b-29f3-49e4-910a-1b98ff415751-bound-sa-token\") on node \"ip-10-0-143-17.ec2.internal\" DevicePath \"\"" Jul 10 18:41:45.397416 ip-10-0-143-17 kubenswrapper[2585]: I0710 18:41:45.397412 2585 reconciler_common.go:299] "Volume detached for volume \"image-registry-private-configuration\" (UniqueName: \"kubernetes.io/secret/f361ac0b-29f3-49e4-910a-1b98ff415751-image-registry-private-configuration\") on node \"ip-10-0-143-17.ec2.internal\" DevicePath \"\"" Jul 10 18:41:45.397416 ip-10-0-143-17 kubenswrapper[2585]: I0710 18:41:45.397420 2585 reconciler_common.go:299] "Volume detached for volume \"installation-pull-secrets\" (UniqueName: \"kubernetes.io/secret/f361ac0b-29f3-49e4-910a-1b98ff415751-installation-pull-secrets\") on node \"ip-10-0-143-17.ec2.internal\" DevicePath \"\"" Jul 10 18:41:45.397735 ip-10-0-143-17 kubenswrapper[2585]: I0710 18:41:45.397429 2585 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-jbmkw\" (UniqueName: \"kubernetes.io/projected/f361ac0b-29f3-49e4-910a-1b98ff415751-kube-api-access-jbmkw\") on node \"ip-10-0-143-17.ec2.internal\" DevicePath \"\"" Jul 10 18:41:45.720527 ip-10-0-143-17 kubenswrapper[2585]: I0710 18:41:45.720493 2585 generic.go:356] "Generic (PLEG): container finished" podID="f361ac0b-29f3-49e4-910a-1b98ff415751" containerID="ceb21d1f1cc836b4afec41d50d339f32c289358ba2aa8fa1cc181345a0421fc2" exitCode=0 Jul 10 18:41:45.720699 ip-10-0-143-17 kubenswrapper[2585]: I0710 18:41:45.720556 2585 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-image-registry/image-registry-5bd858b6bc-fkzlq" Jul 10 18:41:45.720699 ip-10-0-143-17 kubenswrapper[2585]: I0710 18:41:45.720585 2585 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-image-registry/image-registry-5bd858b6bc-fkzlq" event={"ID":"f361ac0b-29f3-49e4-910a-1b98ff415751","Type":"ContainerDied","Data":"ceb21d1f1cc836b4afec41d50d339f32c289358ba2aa8fa1cc181345a0421fc2"} Jul 10 18:41:45.720699 ip-10-0-143-17 kubenswrapper[2585]: I0710 18:41:45.720637 2585 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-image-registry/image-registry-5bd858b6bc-fkzlq" event={"ID":"f361ac0b-29f3-49e4-910a-1b98ff415751","Type":"ContainerDied","Data":"b2298d146de5704e86c2fdd60a2848fbc899cd745ba20c9e066ee5924c11579a"} Jul 10 18:41:45.720699 ip-10-0-143-17 kubenswrapper[2585]: I0710 18:41:45.720666 2585 scope.go:117] "RemoveContainer" containerID="ceb21d1f1cc836b4afec41d50d339f32c289358ba2aa8fa1cc181345a0421fc2" Jul 10 18:41:45.726690 ip-10-0-143-17 kubenswrapper[2585]: I0710 18:41:45.726656 2585 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/metrics-server-688c68676d-bdllx" event={"ID":"b44dd6d6-9d73-4626-bfe4-04cf7215ebed","Type":"ContainerStarted","Data":"c4766daa569a1dab4e31b28a969503d1c6a6b36b0e4c800f6eada6e6b1eee36e"} Jul 10 18:41:45.733958 ip-10-0-143-17 kubenswrapper[2585]: I0710 18:41:45.733939 2585 scope.go:117] "RemoveContainer" containerID="ceb21d1f1cc836b4afec41d50d339f32c289358ba2aa8fa1cc181345a0421fc2" Jul 10 18:41:45.734209 ip-10-0-143-17 kubenswrapper[2585]: E0710 18:41:45.734191 2585 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"ceb21d1f1cc836b4afec41d50d339f32c289358ba2aa8fa1cc181345a0421fc2\": container with ID starting with ceb21d1f1cc836b4afec41d50d339f32c289358ba2aa8fa1cc181345a0421fc2 not found: ID does not exist" containerID="ceb21d1f1cc836b4afec41d50d339f32c289358ba2aa8fa1cc181345a0421fc2" Jul 10 18:41:45.734274 ip-10-0-143-17 kubenswrapper[2585]: I0710 18:41:45.734220 2585 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"ceb21d1f1cc836b4afec41d50d339f32c289358ba2aa8fa1cc181345a0421fc2"} err="failed to get container status \"ceb21d1f1cc836b4afec41d50d339f32c289358ba2aa8fa1cc181345a0421fc2\": rpc error: code = NotFound desc = could not find container \"ceb21d1f1cc836b4afec41d50d339f32c289358ba2aa8fa1cc181345a0421fc2\": container with ID starting with ceb21d1f1cc836b4afec41d50d339f32c289358ba2aa8fa1cc181345a0421fc2 not found: ID does not exist" Jul 10 18:41:45.749779 ip-10-0-143-17 kubenswrapper[2585]: I0710 18:41:45.749757 2585 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-image-registry/image-registry-5bd858b6bc-fkzlq"] Jul 10 18:41:45.753598 ip-10-0-143-17 kubenswrapper[2585]: I0710 18:41:45.753579 2585 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["openshift-image-registry/image-registry-5bd858b6bc-fkzlq"] Jul 10 18:41:46.735230 ip-10-0-143-17 kubenswrapper[2585]: I0710 18:41:46.735200 2585 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/metrics-server-688c68676d-bdllx" event={"ID":"b44dd6d6-9d73-4626-bfe4-04cf7215ebed","Type":"ContainerStarted","Data":"ba68805a1e38091d00eb70d92c0e4674a08e370b1c674e5e93e658d1cbd18797"} Jul 10 18:41:46.769561 ip-10-0-143-17 kubenswrapper[2585]: I0710 18:41:46.769496 2585 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-monitoring/metrics-server-688c68676d-bdllx" podStartSLOduration=1.422613347 podStartE2EDuration="2.76946491s" totalImagesPullingTime="1.346851563s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-10 18:41:44 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-10 18:41:46.769261769 +0000 UTC m=+120.226621721" watchObservedRunningTime="2026-07-10 18:41:46.76946491 +0000 UTC m=+120.226824846" Jul 10 18:41:47.140849 ip-10-0-143-17 kubenswrapper[2585]: I0710 18:41:47.140823 2585 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-console/console-8c67b5665-lqvvr"] Jul 10 18:41:47.189142 ip-10-0-143-17 kubenswrapper[2585]: I0710 18:41:47.189106 2585 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-console/console-5c555b848f-4jxfq"] Jul 10 18:41:47.189430 ip-10-0-143-17 kubenswrapper[2585]: I0710 18:41:47.189414 2585 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="f361ac0b-29f3-49e4-910a-1b98ff415751" containerName="registry" Jul 10 18:41:47.189509 ip-10-0-143-17 kubenswrapper[2585]: I0710 18:41:47.189433 2585 state_mem.go:107] "Deleted CPUSet assignment" podUID="f361ac0b-29f3-49e4-910a-1b98ff415751" containerName="registry" Jul 10 18:41:47.189589 ip-10-0-143-17 kubenswrapper[2585]: I0710 18:41:47.189517 2585 memory_manager.go:368] "RemoveStaleState removing state" podUID="f361ac0b-29f3-49e4-910a-1b98ff415751" containerName="registry" Jul 10 18:41:47.192235 ip-10-0-143-17 kubenswrapper[2585]: I0710 18:41:47.192180 2585 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-console/console-5c555b848f-4jxfq" Jul 10 18:41:47.208102 ip-10-0-143-17 kubenswrapper[2585]: I0710 18:41:47.208077 2585 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-console/console-5c555b848f-4jxfq"] Jul 10 18:41:47.215979 ip-10-0-143-17 kubenswrapper[2585]: I0710 18:41:47.215955 2585 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="f361ac0b-29f3-49e4-910a-1b98ff415751" path="/var/lib/kubelet/pods/f361ac0b-29f3-49e4-910a-1b98ff415751/volumes" Jul 10 18:41:47.313095 ip-10-0-143-17 kubenswrapper[2585]: I0710 18:41:47.313066 2585 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-4bg6l\" (UniqueName: \"kubernetes.io/projected/f449c3a6-ffbf-45f2-90b0-0ada0f1af9f3-kube-api-access-4bg6l\") pod \"console-5c555b848f-4jxfq\" (UID: \"f449c3a6-ffbf-45f2-90b0-0ada0f1af9f3\") " pod="openshift-console/console-5c555b848f-4jxfq" Jul 10 18:41:47.313231 ip-10-0-143-17 kubenswrapper[2585]: I0710 18:41:47.313113 2585 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"console-oauth-config\" (UniqueName: \"kubernetes.io/secret/f449c3a6-ffbf-45f2-90b0-0ada0f1af9f3-console-oauth-config\") pod \"console-5c555b848f-4jxfq\" (UID: \"f449c3a6-ffbf-45f2-90b0-0ada0f1af9f3\") " pod="openshift-console/console-5c555b848f-4jxfq" Jul 10 18:41:47.313231 ip-10-0-143-17 kubenswrapper[2585]: I0710 18:41:47.313143 2585 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"service-ca\" (UniqueName: \"kubernetes.io/configmap/f449c3a6-ffbf-45f2-90b0-0ada0f1af9f3-service-ca\") pod \"console-5c555b848f-4jxfq\" (UID: \"f449c3a6-ffbf-45f2-90b0-0ada0f1af9f3\") " pod="openshift-console/console-5c555b848f-4jxfq" Jul 10 18:41:47.313231 ip-10-0-143-17 kubenswrapper[2585]: I0710 18:41:47.313220 2585 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/f449c3a6-ffbf-45f2-90b0-0ada0f1af9f3-trusted-ca-bundle\") pod \"console-5c555b848f-4jxfq\" (UID: \"f449c3a6-ffbf-45f2-90b0-0ada0f1af9f3\") " pod="openshift-console/console-5c555b848f-4jxfq" Jul 10 18:41:47.313348 ip-10-0-143-17 kubenswrapper[2585]: I0710 18:41:47.313263 2585 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"console-config\" (UniqueName: \"kubernetes.io/configmap/f449c3a6-ffbf-45f2-90b0-0ada0f1af9f3-console-config\") pod \"console-5c555b848f-4jxfq\" (UID: \"f449c3a6-ffbf-45f2-90b0-0ada0f1af9f3\") " pod="openshift-console/console-5c555b848f-4jxfq" Jul 10 18:41:47.313348 ip-10-0-143-17 kubenswrapper[2585]: I0710 18:41:47.313291 2585 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"oauth-serving-cert\" (UniqueName: \"kubernetes.io/configmap/f449c3a6-ffbf-45f2-90b0-0ada0f1af9f3-oauth-serving-cert\") pod \"console-5c555b848f-4jxfq\" (UID: \"f449c3a6-ffbf-45f2-90b0-0ada0f1af9f3\") " pod="openshift-console/console-5c555b848f-4jxfq" Jul 10 18:41:47.313348 ip-10-0-143-17 kubenswrapper[2585]: I0710 18:41:47.313331 2585 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"console-serving-cert\" (UniqueName: \"kubernetes.io/secret/f449c3a6-ffbf-45f2-90b0-0ada0f1af9f3-console-serving-cert\") pod \"console-5c555b848f-4jxfq\" (UID: \"f449c3a6-ffbf-45f2-90b0-0ada0f1af9f3\") " pod="openshift-console/console-5c555b848f-4jxfq" Jul 10 18:41:47.414587 ip-10-0-143-17 kubenswrapper[2585]: I0710 18:41:47.414553 2585 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/f449c3a6-ffbf-45f2-90b0-0ada0f1af9f3-trusted-ca-bundle\") pod \"console-5c555b848f-4jxfq\" (UID: \"f449c3a6-ffbf-45f2-90b0-0ada0f1af9f3\") " pod="openshift-console/console-5c555b848f-4jxfq" Jul 10 18:41:47.414733 ip-10-0-143-17 kubenswrapper[2585]: I0710 18:41:47.414599 2585 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"console-config\" (UniqueName: \"kubernetes.io/configmap/f449c3a6-ffbf-45f2-90b0-0ada0f1af9f3-console-config\") pod \"console-5c555b848f-4jxfq\" (UID: \"f449c3a6-ffbf-45f2-90b0-0ada0f1af9f3\") " pod="openshift-console/console-5c555b848f-4jxfq" Jul 10 18:41:47.414733 ip-10-0-143-17 kubenswrapper[2585]: I0710 18:41:47.414627 2585 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"oauth-serving-cert\" (UniqueName: \"kubernetes.io/configmap/f449c3a6-ffbf-45f2-90b0-0ada0f1af9f3-oauth-serving-cert\") pod \"console-5c555b848f-4jxfq\" (UID: \"f449c3a6-ffbf-45f2-90b0-0ada0f1af9f3\") " pod="openshift-console/console-5c555b848f-4jxfq" Jul 10 18:41:47.414733 ip-10-0-143-17 kubenswrapper[2585]: I0710 18:41:47.414675 2585 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"console-serving-cert\" (UniqueName: \"kubernetes.io/secret/f449c3a6-ffbf-45f2-90b0-0ada0f1af9f3-console-serving-cert\") pod \"console-5c555b848f-4jxfq\" (UID: \"f449c3a6-ffbf-45f2-90b0-0ada0f1af9f3\") " pod="openshift-console/console-5c555b848f-4jxfq" Jul 10 18:41:47.414733 ip-10-0-143-17 kubenswrapper[2585]: I0710 18:41:47.414726 2585 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-4bg6l\" (UniqueName: \"kubernetes.io/projected/f449c3a6-ffbf-45f2-90b0-0ada0f1af9f3-kube-api-access-4bg6l\") pod \"console-5c555b848f-4jxfq\" (UID: \"f449c3a6-ffbf-45f2-90b0-0ada0f1af9f3\") " pod="openshift-console/console-5c555b848f-4jxfq" Jul 10 18:41:47.414928 ip-10-0-143-17 kubenswrapper[2585]: I0710 18:41:47.414797 2585 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"console-oauth-config\" (UniqueName: \"kubernetes.io/secret/f449c3a6-ffbf-45f2-90b0-0ada0f1af9f3-console-oauth-config\") pod \"console-5c555b848f-4jxfq\" (UID: \"f449c3a6-ffbf-45f2-90b0-0ada0f1af9f3\") " pod="openshift-console/console-5c555b848f-4jxfq" Jul 10 18:41:47.414928 ip-10-0-143-17 kubenswrapper[2585]: I0710 18:41:47.414846 2585 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"service-ca\" (UniqueName: \"kubernetes.io/configmap/f449c3a6-ffbf-45f2-90b0-0ada0f1af9f3-service-ca\") pod \"console-5c555b848f-4jxfq\" (UID: \"f449c3a6-ffbf-45f2-90b0-0ada0f1af9f3\") " pod="openshift-console/console-5c555b848f-4jxfq" Jul 10 18:41:47.415381 ip-10-0-143-17 kubenswrapper[2585]: I0710 18:41:47.415358 2585 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"console-config\" (UniqueName: \"kubernetes.io/configmap/f449c3a6-ffbf-45f2-90b0-0ada0f1af9f3-console-config\") pod \"console-5c555b848f-4jxfq\" (UID: \"f449c3a6-ffbf-45f2-90b0-0ada0f1af9f3\") " pod="openshift-console/console-5c555b848f-4jxfq" Jul 10 18:41:47.415441 ip-10-0-143-17 kubenswrapper[2585]: I0710 18:41:47.415423 2585 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"oauth-serving-cert\" (UniqueName: \"kubernetes.io/configmap/f449c3a6-ffbf-45f2-90b0-0ada0f1af9f3-oauth-serving-cert\") pod \"console-5c555b848f-4jxfq\" (UID: \"f449c3a6-ffbf-45f2-90b0-0ada0f1af9f3\") " pod="openshift-console/console-5c555b848f-4jxfq" Jul 10 18:41:47.415495 ip-10-0-143-17 kubenswrapper[2585]: I0710 18:41:47.415432 2585 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/f449c3a6-ffbf-45f2-90b0-0ada0f1af9f3-trusted-ca-bundle\") pod \"console-5c555b848f-4jxfq\" (UID: \"f449c3a6-ffbf-45f2-90b0-0ada0f1af9f3\") " pod="openshift-console/console-5c555b848f-4jxfq" Jul 10 18:41:47.415572 ip-10-0-143-17 kubenswrapper[2585]: I0710 18:41:47.415515 2585 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"service-ca\" (UniqueName: \"kubernetes.io/configmap/f449c3a6-ffbf-45f2-90b0-0ada0f1af9f3-service-ca\") pod \"console-5c555b848f-4jxfq\" (UID: \"f449c3a6-ffbf-45f2-90b0-0ada0f1af9f3\") " pod="openshift-console/console-5c555b848f-4jxfq" Jul 10 18:41:47.417254 ip-10-0-143-17 kubenswrapper[2585]: I0710 18:41:47.417231 2585 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"console-oauth-config\" (UniqueName: \"kubernetes.io/secret/f449c3a6-ffbf-45f2-90b0-0ada0f1af9f3-console-oauth-config\") pod \"console-5c555b848f-4jxfq\" (UID: \"f449c3a6-ffbf-45f2-90b0-0ada0f1af9f3\") " pod="openshift-console/console-5c555b848f-4jxfq" Jul 10 18:41:47.417418 ip-10-0-143-17 kubenswrapper[2585]: I0710 18:41:47.417399 2585 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"console-serving-cert\" (UniqueName: \"kubernetes.io/secret/f449c3a6-ffbf-45f2-90b0-0ada0f1af9f3-console-serving-cert\") pod \"console-5c555b848f-4jxfq\" (UID: \"f449c3a6-ffbf-45f2-90b0-0ada0f1af9f3\") " pod="openshift-console/console-5c555b848f-4jxfq" Jul 10 18:41:47.423876 ip-10-0-143-17 kubenswrapper[2585]: I0710 18:41:47.423856 2585 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-4bg6l\" (UniqueName: \"kubernetes.io/projected/f449c3a6-ffbf-45f2-90b0-0ada0f1af9f3-kube-api-access-4bg6l\") pod \"console-5c555b848f-4jxfq\" (UID: \"f449c3a6-ffbf-45f2-90b0-0ada0f1af9f3\") " pod="openshift-console/console-5c555b848f-4jxfq" Jul 10 18:41:47.505917 ip-10-0-143-17 kubenswrapper[2585]: I0710 18:41:47.505882 2585 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-console/console-5c555b848f-4jxfq" Jul 10 18:41:47.641757 ip-10-0-143-17 kubenswrapper[2585]: I0710 18:41:47.641735 2585 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-console/console-5c555b848f-4jxfq"] Jul 10 18:41:47.643825 ip-10-0-143-17 kubenswrapper[2585]: W0710 18:41:47.643798 2585 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podf449c3a6_ffbf_45f2_90b0_0ada0f1af9f3.slice/crio-b2fdddcad409dd4d1ba98314fb9a0cfdf3f644e200d93e745d1bc0085d3d51db WatchSource:0}: Error finding container b2fdddcad409dd4d1ba98314fb9a0cfdf3f644e200d93e745d1bc0085d3d51db: Status 404 returned error can't find the container with id b2fdddcad409dd4d1ba98314fb9a0cfdf3f644e200d93e745d1bc0085d3d51db Jul 10 18:41:47.741006 ip-10-0-143-17 kubenswrapper[2585]: I0710 18:41:47.740975 2585 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-console/console-5c555b848f-4jxfq" event={"ID":"f449c3a6-ffbf-45f2-90b0-0ada0f1af9f3","Type":"ContainerStarted","Data":"f134fcc8fc20876b1a629ada0d0079e7b1bd7db327d3d33b6ff28f17ae8408b2"} Jul 10 18:41:47.741429 ip-10-0-143-17 kubenswrapper[2585]: I0710 18:41:47.741027 2585 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-console/console-5c555b848f-4jxfq" event={"ID":"f449c3a6-ffbf-45f2-90b0-0ada0f1af9f3","Type":"ContainerStarted","Data":"b2fdddcad409dd4d1ba98314fb9a0cfdf3f644e200d93e745d1bc0085d3d51db"} Jul 10 18:41:47.770240 ip-10-0-143-17 kubenswrapper[2585]: I0710 18:41:47.770160 2585 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-console/console-5c555b848f-4jxfq" podStartSLOduration=0.770145954 podStartE2EDuration="770.145954ms" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-10 18:41:47 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-10 18:41:47.769792742 +0000 UTC m=+121.227152677" watchObservedRunningTime="2026-07-10 18:41:47.770145954 +0000 UTC m=+121.227505889" Jul 10 18:41:50.731983 ip-10-0-143-17 kubenswrapper[2585]: I0710 18:41:50.731951 2585 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-console/console-8c67b5665-lqvvr" Jul 10 18:41:56.884959 ip-10-0-143-17 kubenswrapper[2585]: I0710 18:41:56.884916 2585 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/6516579e-a521-4350-bc6b-c4d698b2f845-metrics-certs\") pod \"network-metrics-daemon-7rwwk\" (UID: \"6516579e-a521-4350-bc6b-c4d698b2f845\") " pod="openshift-multus/network-metrics-daemon-7rwwk" Jul 10 18:41:56.887100 ip-10-0-143-17 kubenswrapper[2585]: I0710 18:41:56.887080 2585 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/6516579e-a521-4350-bc6b-c4d698b2f845-metrics-certs\") pod \"network-metrics-daemon-7rwwk\" (UID: \"6516579e-a521-4350-bc6b-c4d698b2f845\") " pod="openshift-multus/network-metrics-daemon-7rwwk" Jul 10 18:41:57.030709 ip-10-0-143-17 kubenswrapper[2585]: I0710 18:41:57.030681 2585 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-multus\"/\"metrics-daemon-sa-dockercfg-hzzqm\"" Jul 10 18:41:57.038332 ip-10-0-143-17 kubenswrapper[2585]: I0710 18:41:57.038313 2585 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-7rwwk" Jul 10 18:41:57.171513 ip-10-0-143-17 kubenswrapper[2585]: I0710 18:41:57.171447 2585 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-multus/network-metrics-daemon-7rwwk"] Jul 10 18:41:57.174435 ip-10-0-143-17 kubenswrapper[2585]: W0710 18:41:57.174409 2585 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod6516579e_a521_4350_bc6b_c4d698b2f845.slice/crio-7d0b6070f8b2221c34b504e3628a4c90fc60bc3aff2a96690ded07efa97f7531 WatchSource:0}: Error finding container 7d0b6070f8b2221c34b504e3628a4c90fc60bc3aff2a96690ded07efa97f7531: Status 404 returned error can't find the container with id 7d0b6070f8b2221c34b504e3628a4c90fc60bc3aff2a96690ded07efa97f7531 Jul 10 18:41:57.506641 ip-10-0-143-17 kubenswrapper[2585]: I0710 18:41:57.506614 2585 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="openshift-console/console-5c555b848f-4jxfq" Jul 10 18:41:57.506847 ip-10-0-143-17 kubenswrapper[2585]: I0710 18:41:57.506662 2585 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-console/console-5c555b848f-4jxfq" Jul 10 18:41:57.512222 ip-10-0-143-17 kubenswrapper[2585]: I0710 18:41:57.512201 2585 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="openshift-console/console-5c555b848f-4jxfq" Jul 10 18:41:57.778094 ip-10-0-143-17 kubenswrapper[2585]: I0710 18:41:57.778012 2585 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-multus/network-metrics-daemon-7rwwk" event={"ID":"6516579e-a521-4350-bc6b-c4d698b2f845","Type":"ContainerStarted","Data":"7d0b6070f8b2221c34b504e3628a4c90fc60bc3aff2a96690ded07efa97f7531"} Jul 10 18:41:57.782065 ip-10-0-143-17 kubenswrapper[2585]: I0710 18:41:57.782041 2585 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-console/console-5c555b848f-4jxfq" Jul 10 18:41:57.837854 ip-10-0-143-17 kubenswrapper[2585]: I0710 18:41:57.837825 2585 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-console/console-7dcf5c576-vqjdl"] Jul 10 18:41:58.783699 ip-10-0-143-17 kubenswrapper[2585]: I0710 18:41:58.783650 2585 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-multus/network-metrics-daemon-7rwwk" event={"ID":"6516579e-a521-4350-bc6b-c4d698b2f845","Type":"ContainerStarted","Data":"cbca7c7ab5b5dd0cca4e84733a81690e21c610270c3c2eb6a637aa06f6464e99"} Jul 10 18:41:58.784128 ip-10-0-143-17 kubenswrapper[2585]: I0710 18:41:58.783770 2585 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-multus/network-metrics-daemon-7rwwk" event={"ID":"6516579e-a521-4350-bc6b-c4d698b2f845","Type":"ContainerStarted","Data":"9da98f7130fb4e2b3391b0256dabce21e4a67456e17d1dbd6d46d4c5773d3936"} Jul 10 18:41:58.812532 ip-10-0-143-17 kubenswrapper[2585]: I0710 18:41:58.812483 2585 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-multus/network-metrics-daemon-7rwwk" podStartSLOduration=130.972509602 podStartE2EDuration="2m11.812469521s" totalImagesPullingTime="839.959919ms" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-10 18:39:47 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-10 18:41:58.81218005 +0000 UTC m=+132.269539985" watchObservedRunningTime="2026-07-10 18:41:58.812469521 +0000 UTC m=+132.269829462" Jul 10 18:42:05.072217 ip-10-0-143-17 kubenswrapper[2585]: I0710 18:42:05.072179 2585 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-monitoring/metrics-server-688c68676d-bdllx" Jul 10 18:42:05.072745 ip-10-0-143-17 kubenswrapper[2585]: I0710 18:42:05.072249 2585 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="openshift-monitoring/metrics-server-688c68676d-bdllx" Jul 10 18:42:09.825977 ip-10-0-143-17 kubenswrapper[2585]: I0710 18:42:09.825944 2585 generic.go:356] "Generic (PLEG): container finished" podID="3041e309-0ff4-41d8-9123-a82c98b5dad3" containerID="1f4b21e1ac086d5f08b8bb6c3b55e398f5400e413dbf6f25905163e09f5b294a" exitCode=0 Jul 10 18:42:09.826335 ip-10-0-143-17 kubenswrapper[2585]: I0710 18:42:09.826021 2585 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-insights/insights-operator-68d9dd94cf-57mwk" event={"ID":"3041e309-0ff4-41d8-9123-a82c98b5dad3","Type":"ContainerDied","Data":"1f4b21e1ac086d5f08b8bb6c3b55e398f5400e413dbf6f25905163e09f5b294a"} Jul 10 18:42:09.826385 ip-10-0-143-17 kubenswrapper[2585]: I0710 18:42:09.826352 2585 scope.go:117] "RemoveContainer" containerID="1f4b21e1ac086d5f08b8bb6c3b55e398f5400e413dbf6f25905163e09f5b294a" Jul 10 18:42:10.831572 ip-10-0-143-17 kubenswrapper[2585]: I0710 18:42:10.831518 2585 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-insights/insights-operator-68d9dd94cf-57mwk" event={"ID":"3041e309-0ff4-41d8-9123-a82c98b5dad3","Type":"ContainerStarted","Data":"1fa2729fca2134ee8a53f53445ef14d5dae5628244c9d58d792845fbb780c624"} Jul 10 18:42:12.160063 ip-10-0-143-17 kubenswrapper[2585]: I0710 18:42:12.160021 2585 kuberuntime_container.go:873] "Killing container with a grace period" pod="openshift-console/console-8c67b5665-lqvvr" podUID="f750992a-c8d1-42a2-81e1-c657404d690f" containerName="console" containerID="cri-o://f82b6e47a48ab876dfcf95aae72602cfcca6f9a0e2ffd58cd376d3bf2f95be3c" gracePeriod=15 Jul 10 18:42:12.425382 ip-10-0-143-17 kubenswrapper[2585]: I0710 18:42:12.425329 2585 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-console/console-8c67b5665-lqvvr" Jul 10 18:42:12.503867 ip-10-0-143-17 kubenswrapper[2585]: I0710 18:42:12.503839 2585 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-zs45k\" (UniqueName: \"kubernetes.io/projected/f750992a-c8d1-42a2-81e1-c657404d690f-kube-api-access-zs45k\") pod \"f750992a-c8d1-42a2-81e1-c657404d690f\" (UID: \"f750992a-c8d1-42a2-81e1-c657404d690f\") " Jul 10 18:42:12.503987 ip-10-0-143-17 kubenswrapper[2585]: I0710 18:42:12.503875 2585 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/f750992a-c8d1-42a2-81e1-c657404d690f-trusted-ca-bundle\") pod \"f750992a-c8d1-42a2-81e1-c657404d690f\" (UID: \"f750992a-c8d1-42a2-81e1-c657404d690f\") " Jul 10 18:42:12.503987 ip-10-0-143-17 kubenswrapper[2585]: I0710 18:42:12.503893 2585 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"service-ca\" (UniqueName: \"kubernetes.io/configmap/f750992a-c8d1-42a2-81e1-c657404d690f-service-ca\") pod \"f750992a-c8d1-42a2-81e1-c657404d690f\" (UID: \"f750992a-c8d1-42a2-81e1-c657404d690f\") " Jul 10 18:42:12.503987 ip-10-0-143-17 kubenswrapper[2585]: I0710 18:42:12.503934 2585 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"console-serving-cert\" (UniqueName: \"kubernetes.io/secret/f750992a-c8d1-42a2-81e1-c657404d690f-console-serving-cert\") pod \"f750992a-c8d1-42a2-81e1-c657404d690f\" (UID: \"f750992a-c8d1-42a2-81e1-c657404d690f\") " Jul 10 18:42:12.503987 ip-10-0-143-17 kubenswrapper[2585]: I0710 18:42:12.503962 2585 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"oauth-serving-cert\" (UniqueName: \"kubernetes.io/configmap/f750992a-c8d1-42a2-81e1-c657404d690f-oauth-serving-cert\") pod \"f750992a-c8d1-42a2-81e1-c657404d690f\" (UID: \"f750992a-c8d1-42a2-81e1-c657404d690f\") " Jul 10 18:42:12.503987 ip-10-0-143-17 kubenswrapper[2585]: I0710 18:42:12.503981 2585 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"console-config\" (UniqueName: \"kubernetes.io/configmap/f750992a-c8d1-42a2-81e1-c657404d690f-console-config\") pod \"f750992a-c8d1-42a2-81e1-c657404d690f\" (UID: \"f750992a-c8d1-42a2-81e1-c657404d690f\") " Jul 10 18:42:12.504233 ip-10-0-143-17 kubenswrapper[2585]: I0710 18:42:12.504013 2585 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"console-oauth-config\" (UniqueName: \"kubernetes.io/secret/f750992a-c8d1-42a2-81e1-c657404d690f-console-oauth-config\") pod \"f750992a-c8d1-42a2-81e1-c657404d690f\" (UID: \"f750992a-c8d1-42a2-81e1-c657404d690f\") " Jul 10 18:42:12.504373 ip-10-0-143-17 kubenswrapper[2585]: I0710 18:42:12.504337 2585 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/f750992a-c8d1-42a2-81e1-c657404d690f-trusted-ca-bundle" (OuterVolumeSpecName: "trusted-ca-bundle") pod "f750992a-c8d1-42a2-81e1-c657404d690f" (UID: "f750992a-c8d1-42a2-81e1-c657404d690f"). InnerVolumeSpecName "trusted-ca-bundle". PluginName "kubernetes.io/configmap", VolumeGIDValue "" Jul 10 18:42:12.504497 ip-10-0-143-17 kubenswrapper[2585]: I0710 18:42:12.504473 2585 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/f750992a-c8d1-42a2-81e1-c657404d690f-oauth-serving-cert" (OuterVolumeSpecName: "oauth-serving-cert") pod "f750992a-c8d1-42a2-81e1-c657404d690f" (UID: "f750992a-c8d1-42a2-81e1-c657404d690f"). InnerVolumeSpecName "oauth-serving-cert". PluginName "kubernetes.io/configmap", VolumeGIDValue "" Jul 10 18:42:12.504497 ip-10-0-143-17 kubenswrapper[2585]: I0710 18:42:12.504466 2585 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/f750992a-c8d1-42a2-81e1-c657404d690f-console-config" (OuterVolumeSpecName: "console-config") pod "f750992a-c8d1-42a2-81e1-c657404d690f" (UID: "f750992a-c8d1-42a2-81e1-c657404d690f"). InnerVolumeSpecName "console-config". PluginName "kubernetes.io/configmap", VolumeGIDValue "" Jul 10 18:42:12.504634 ip-10-0-143-17 kubenswrapper[2585]: I0710 18:42:12.504484 2585 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/f750992a-c8d1-42a2-81e1-c657404d690f-service-ca" (OuterVolumeSpecName: "service-ca") pod "f750992a-c8d1-42a2-81e1-c657404d690f" (UID: "f750992a-c8d1-42a2-81e1-c657404d690f"). InnerVolumeSpecName "service-ca". PluginName "kubernetes.io/configmap", VolumeGIDValue "" Jul 10 18:42:12.506149 ip-10-0-143-17 kubenswrapper[2585]: I0710 18:42:12.506123 2585 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/f750992a-c8d1-42a2-81e1-c657404d690f-kube-api-access-zs45k" (OuterVolumeSpecName: "kube-api-access-zs45k") pod "f750992a-c8d1-42a2-81e1-c657404d690f" (UID: "f750992a-c8d1-42a2-81e1-c657404d690f"). InnerVolumeSpecName "kube-api-access-zs45k". PluginName "kubernetes.io/projected", VolumeGIDValue "" Jul 10 18:42:12.506224 ip-10-0-143-17 kubenswrapper[2585]: I0710 18:42:12.506151 2585 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/f750992a-c8d1-42a2-81e1-c657404d690f-console-serving-cert" (OuterVolumeSpecName: "console-serving-cert") pod "f750992a-c8d1-42a2-81e1-c657404d690f" (UID: "f750992a-c8d1-42a2-81e1-c657404d690f"). InnerVolumeSpecName "console-serving-cert". PluginName "kubernetes.io/secret", VolumeGIDValue "" Jul 10 18:42:12.506224 ip-10-0-143-17 kubenswrapper[2585]: I0710 18:42:12.506171 2585 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/f750992a-c8d1-42a2-81e1-c657404d690f-console-oauth-config" (OuterVolumeSpecName: "console-oauth-config") pod "f750992a-c8d1-42a2-81e1-c657404d690f" (UID: "f750992a-c8d1-42a2-81e1-c657404d690f"). InnerVolumeSpecName "console-oauth-config". PluginName "kubernetes.io/secret", VolumeGIDValue "" Jul 10 18:42:12.605311 ip-10-0-143-17 kubenswrapper[2585]: I0710 18:42:12.605287 2585 reconciler_common.go:299] "Volume detached for volume \"service-ca\" (UniqueName: \"kubernetes.io/configmap/f750992a-c8d1-42a2-81e1-c657404d690f-service-ca\") on node \"ip-10-0-143-17.ec2.internal\" DevicePath \"\"" Jul 10 18:42:12.605311 ip-10-0-143-17 kubenswrapper[2585]: I0710 18:42:12.605307 2585 reconciler_common.go:299] "Volume detached for volume \"console-serving-cert\" (UniqueName: \"kubernetes.io/secret/f750992a-c8d1-42a2-81e1-c657404d690f-console-serving-cert\") on node \"ip-10-0-143-17.ec2.internal\" DevicePath \"\"" Jul 10 18:42:12.605447 ip-10-0-143-17 kubenswrapper[2585]: I0710 18:42:12.605319 2585 reconciler_common.go:299] "Volume detached for volume \"oauth-serving-cert\" (UniqueName: \"kubernetes.io/configmap/f750992a-c8d1-42a2-81e1-c657404d690f-oauth-serving-cert\") on node \"ip-10-0-143-17.ec2.internal\" DevicePath \"\"" Jul 10 18:42:12.605447 ip-10-0-143-17 kubenswrapper[2585]: I0710 18:42:12.605328 2585 reconciler_common.go:299] "Volume detached for volume \"console-config\" (UniqueName: \"kubernetes.io/configmap/f750992a-c8d1-42a2-81e1-c657404d690f-console-config\") on node \"ip-10-0-143-17.ec2.internal\" DevicePath \"\"" Jul 10 18:42:12.605447 ip-10-0-143-17 kubenswrapper[2585]: I0710 18:42:12.605338 2585 reconciler_common.go:299] "Volume detached for volume \"console-oauth-config\" (UniqueName: \"kubernetes.io/secret/f750992a-c8d1-42a2-81e1-c657404d690f-console-oauth-config\") on node \"ip-10-0-143-17.ec2.internal\" DevicePath \"\"" Jul 10 18:42:12.605447 ip-10-0-143-17 kubenswrapper[2585]: I0710 18:42:12.605347 2585 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-zs45k\" (UniqueName: \"kubernetes.io/projected/f750992a-c8d1-42a2-81e1-c657404d690f-kube-api-access-zs45k\") on node \"ip-10-0-143-17.ec2.internal\" DevicePath \"\"" Jul 10 18:42:12.605447 ip-10-0-143-17 kubenswrapper[2585]: I0710 18:42:12.605354 2585 reconciler_common.go:299] "Volume detached for volume \"trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/f750992a-c8d1-42a2-81e1-c657404d690f-trusted-ca-bundle\") on node \"ip-10-0-143-17.ec2.internal\" DevicePath \"\"" Jul 10 18:42:12.842052 ip-10-0-143-17 kubenswrapper[2585]: I0710 18:42:12.842026 2585 generic.go:356] "Generic (PLEG): container finished" podID="f750992a-c8d1-42a2-81e1-c657404d690f" containerID="f82b6e47a48ab876dfcf95aae72602cfcca6f9a0e2ffd58cd376d3bf2f95be3c" exitCode=0 Jul 10 18:42:12.842232 ip-10-0-143-17 kubenswrapper[2585]: I0710 18:42:12.842075 2585 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-console/console-8c67b5665-lqvvr" Jul 10 18:42:12.842232 ip-10-0-143-17 kubenswrapper[2585]: I0710 18:42:12.842088 2585 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-console/console-8c67b5665-lqvvr" event={"ID":"f750992a-c8d1-42a2-81e1-c657404d690f","Type":"ContainerDied","Data":"f82b6e47a48ab876dfcf95aae72602cfcca6f9a0e2ffd58cd376d3bf2f95be3c"} Jul 10 18:42:12.842232 ip-10-0-143-17 kubenswrapper[2585]: I0710 18:42:12.842131 2585 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-console/console-8c67b5665-lqvvr" event={"ID":"f750992a-c8d1-42a2-81e1-c657404d690f","Type":"ContainerDied","Data":"766eff61f895ed023cbba80139bed951f68e47dc3d512473c115b2a855f47109"} Jul 10 18:42:12.842232 ip-10-0-143-17 kubenswrapper[2585]: I0710 18:42:12.842152 2585 scope.go:117] "RemoveContainer" containerID="f82b6e47a48ab876dfcf95aae72602cfcca6f9a0e2ffd58cd376d3bf2f95be3c" Jul 10 18:42:12.850941 ip-10-0-143-17 kubenswrapper[2585]: I0710 18:42:12.850925 2585 scope.go:117] "RemoveContainer" containerID="f82b6e47a48ab876dfcf95aae72602cfcca6f9a0e2ffd58cd376d3bf2f95be3c" Jul 10 18:42:12.851215 ip-10-0-143-17 kubenswrapper[2585]: E0710 18:42:12.851194 2585 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"f82b6e47a48ab876dfcf95aae72602cfcca6f9a0e2ffd58cd376d3bf2f95be3c\": container with ID starting with f82b6e47a48ab876dfcf95aae72602cfcca6f9a0e2ffd58cd376d3bf2f95be3c not found: ID does not exist" containerID="f82b6e47a48ab876dfcf95aae72602cfcca6f9a0e2ffd58cd376d3bf2f95be3c" Jul 10 18:42:12.851281 ip-10-0-143-17 kubenswrapper[2585]: I0710 18:42:12.851223 2585 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"f82b6e47a48ab876dfcf95aae72602cfcca6f9a0e2ffd58cd376d3bf2f95be3c"} err="failed to get container status \"f82b6e47a48ab876dfcf95aae72602cfcca6f9a0e2ffd58cd376d3bf2f95be3c\": rpc error: code = NotFound desc = could not find container \"f82b6e47a48ab876dfcf95aae72602cfcca6f9a0e2ffd58cd376d3bf2f95be3c\": container with ID starting with f82b6e47a48ab876dfcf95aae72602cfcca6f9a0e2ffd58cd376d3bf2f95be3c not found: ID does not exist" Jul 10 18:42:12.866344 ip-10-0-143-17 kubenswrapper[2585]: I0710 18:42:12.866325 2585 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-console/console-8c67b5665-lqvvr"] Jul 10 18:42:12.871362 ip-10-0-143-17 kubenswrapper[2585]: I0710 18:42:12.871343 2585 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["openshift-console/console-8c67b5665-lqvvr"] Jul 10 18:42:13.215162 ip-10-0-143-17 kubenswrapper[2585]: I0710 18:42:13.215098 2585 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="f750992a-c8d1-42a2-81e1-c657404d690f" path="/var/lib/kubelet/pods/f750992a-c8d1-42a2-81e1-c657404d690f/volumes" Jul 10 18:42:22.860992 ip-10-0-143-17 kubenswrapper[2585]: I0710 18:42:22.860781 2585 kuberuntime_container.go:873] "Killing container with a grace period" pod="openshift-console/console-7dcf5c576-vqjdl" podUID="dc58c93b-5a2c-4f55-ae5e-f9cddf66101e" containerName="console" containerID="cri-o://879a6cc4411ed8e54e788f74c065e43973ceaf0b7a3395edc2f13b11e071aeb0" gracePeriod=15 Jul 10 18:42:23.134085 ip-10-0-143-17 kubenswrapper[2585]: I0710 18:42:23.134059 2585 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-console/console-7dcf5c576-vqjdl" Jul 10 18:42:23.285294 ip-10-0-143-17 kubenswrapper[2585]: I0710 18:42:23.285250 2585 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-tzf5j\" (UniqueName: \"kubernetes.io/projected/dc58c93b-5a2c-4f55-ae5e-f9cddf66101e-kube-api-access-tzf5j\") pod \"dc58c93b-5a2c-4f55-ae5e-f9cddf66101e\" (UID: \"dc58c93b-5a2c-4f55-ae5e-f9cddf66101e\") " Jul 10 18:42:23.285485 ip-10-0-143-17 kubenswrapper[2585]: I0710 18:42:23.285331 2585 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"console-serving-cert\" (UniqueName: \"kubernetes.io/secret/dc58c93b-5a2c-4f55-ae5e-f9cddf66101e-console-serving-cert\") pod \"dc58c93b-5a2c-4f55-ae5e-f9cddf66101e\" (UID: \"dc58c93b-5a2c-4f55-ae5e-f9cddf66101e\") " Jul 10 18:42:23.285485 ip-10-0-143-17 kubenswrapper[2585]: I0710 18:42:23.285359 2585 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"oauth-serving-cert\" (UniqueName: \"kubernetes.io/configmap/dc58c93b-5a2c-4f55-ae5e-f9cddf66101e-oauth-serving-cert\") pod \"dc58c93b-5a2c-4f55-ae5e-f9cddf66101e\" (UID: \"dc58c93b-5a2c-4f55-ae5e-f9cddf66101e\") " Jul 10 18:42:23.285485 ip-10-0-143-17 kubenswrapper[2585]: I0710 18:42:23.285472 2585 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"service-ca\" (UniqueName: \"kubernetes.io/configmap/dc58c93b-5a2c-4f55-ae5e-f9cddf66101e-service-ca\") pod \"dc58c93b-5a2c-4f55-ae5e-f9cddf66101e\" (UID: \"dc58c93b-5a2c-4f55-ae5e-f9cddf66101e\") " Jul 10 18:42:23.285680 ip-10-0-143-17 kubenswrapper[2585]: I0710 18:42:23.285521 2585 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"console-oauth-config\" (UniqueName: \"kubernetes.io/secret/dc58c93b-5a2c-4f55-ae5e-f9cddf66101e-console-oauth-config\") pod \"dc58c93b-5a2c-4f55-ae5e-f9cddf66101e\" (UID: \"dc58c93b-5a2c-4f55-ae5e-f9cddf66101e\") " Jul 10 18:42:23.285680 ip-10-0-143-17 kubenswrapper[2585]: I0710 18:42:23.285596 2585 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"console-config\" (UniqueName: \"kubernetes.io/configmap/dc58c93b-5a2c-4f55-ae5e-f9cddf66101e-console-config\") pod \"dc58c93b-5a2c-4f55-ae5e-f9cddf66101e\" (UID: \"dc58c93b-5a2c-4f55-ae5e-f9cddf66101e\") " Jul 10 18:42:23.286204 ip-10-0-143-17 kubenswrapper[2585]: I0710 18:42:23.286170 2585 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/dc58c93b-5a2c-4f55-ae5e-f9cddf66101e-console-config" (OuterVolumeSpecName: "console-config") pod "dc58c93b-5a2c-4f55-ae5e-f9cddf66101e" (UID: "dc58c93b-5a2c-4f55-ae5e-f9cddf66101e"). InnerVolumeSpecName "console-config". PluginName "kubernetes.io/configmap", VolumeGIDValue "" Jul 10 18:42:23.289556 ip-10-0-143-17 kubenswrapper[2585]: I0710 18:42:23.286815 2585 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/dc58c93b-5a2c-4f55-ae5e-f9cddf66101e-oauth-serving-cert" (OuterVolumeSpecName: "oauth-serving-cert") pod "dc58c93b-5a2c-4f55-ae5e-f9cddf66101e" (UID: "dc58c93b-5a2c-4f55-ae5e-f9cddf66101e"). InnerVolumeSpecName "oauth-serving-cert". PluginName "kubernetes.io/configmap", VolumeGIDValue "" Jul 10 18:42:23.289556 ip-10-0-143-17 kubenswrapper[2585]: I0710 18:42:23.287953 2585 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/dc58c93b-5a2c-4f55-ae5e-f9cddf66101e-service-ca" (OuterVolumeSpecName: "service-ca") pod "dc58c93b-5a2c-4f55-ae5e-f9cddf66101e" (UID: "dc58c93b-5a2c-4f55-ae5e-f9cddf66101e"). InnerVolumeSpecName "service-ca". PluginName "kubernetes.io/configmap", VolumeGIDValue "" Jul 10 18:42:23.293554 ip-10-0-143-17 kubenswrapper[2585]: I0710 18:42:23.290584 2585 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/dc58c93b-5a2c-4f55-ae5e-f9cddf66101e-console-oauth-config" (OuterVolumeSpecName: "console-oauth-config") pod "dc58c93b-5a2c-4f55-ae5e-f9cddf66101e" (UID: "dc58c93b-5a2c-4f55-ae5e-f9cddf66101e"). InnerVolumeSpecName "console-oauth-config". PluginName "kubernetes.io/secret", VolumeGIDValue "" Jul 10 18:42:23.297744 ip-10-0-143-17 kubenswrapper[2585]: I0710 18:42:23.297620 2585 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/dc58c93b-5a2c-4f55-ae5e-f9cddf66101e-console-serving-cert" (OuterVolumeSpecName: "console-serving-cert") pod "dc58c93b-5a2c-4f55-ae5e-f9cddf66101e" (UID: "dc58c93b-5a2c-4f55-ae5e-f9cddf66101e"). InnerVolumeSpecName "console-serving-cert". PluginName "kubernetes.io/secret", VolumeGIDValue "" Jul 10 18:42:23.297744 ip-10-0-143-17 kubenswrapper[2585]: I0710 18:42:23.297715 2585 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/dc58c93b-5a2c-4f55-ae5e-f9cddf66101e-kube-api-access-tzf5j" (OuterVolumeSpecName: "kube-api-access-tzf5j") pod "dc58c93b-5a2c-4f55-ae5e-f9cddf66101e" (UID: "dc58c93b-5a2c-4f55-ae5e-f9cddf66101e"). InnerVolumeSpecName "kube-api-access-tzf5j". PluginName "kubernetes.io/projected", VolumeGIDValue "" Jul 10 18:42:23.386664 ip-10-0-143-17 kubenswrapper[2585]: I0710 18:42:23.386596 2585 reconciler_common.go:299] "Volume detached for volume \"console-serving-cert\" (UniqueName: \"kubernetes.io/secret/dc58c93b-5a2c-4f55-ae5e-f9cddf66101e-console-serving-cert\") on node \"ip-10-0-143-17.ec2.internal\" DevicePath \"\"" Jul 10 18:42:23.386664 ip-10-0-143-17 kubenswrapper[2585]: I0710 18:42:23.386623 2585 reconciler_common.go:299] "Volume detached for volume \"oauth-serving-cert\" (UniqueName: \"kubernetes.io/configmap/dc58c93b-5a2c-4f55-ae5e-f9cddf66101e-oauth-serving-cert\") on node \"ip-10-0-143-17.ec2.internal\" DevicePath \"\"" Jul 10 18:42:23.386664 ip-10-0-143-17 kubenswrapper[2585]: I0710 18:42:23.386633 2585 reconciler_common.go:299] "Volume detached for volume \"service-ca\" (UniqueName: \"kubernetes.io/configmap/dc58c93b-5a2c-4f55-ae5e-f9cddf66101e-service-ca\") on node \"ip-10-0-143-17.ec2.internal\" DevicePath \"\"" Jul 10 18:42:23.386664 ip-10-0-143-17 kubenswrapper[2585]: I0710 18:42:23.386641 2585 reconciler_common.go:299] "Volume detached for volume \"console-oauth-config\" (UniqueName: \"kubernetes.io/secret/dc58c93b-5a2c-4f55-ae5e-f9cddf66101e-console-oauth-config\") on node \"ip-10-0-143-17.ec2.internal\" DevicePath \"\"" Jul 10 18:42:23.386664 ip-10-0-143-17 kubenswrapper[2585]: I0710 18:42:23.386653 2585 reconciler_common.go:299] "Volume detached for volume \"console-config\" (UniqueName: \"kubernetes.io/configmap/dc58c93b-5a2c-4f55-ae5e-f9cddf66101e-console-config\") on node \"ip-10-0-143-17.ec2.internal\" DevicePath \"\"" Jul 10 18:42:23.386664 ip-10-0-143-17 kubenswrapper[2585]: I0710 18:42:23.386661 2585 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-tzf5j\" (UniqueName: \"kubernetes.io/projected/dc58c93b-5a2c-4f55-ae5e-f9cddf66101e-kube-api-access-tzf5j\") on node \"ip-10-0-143-17.ec2.internal\" DevicePath \"\"" Jul 10 18:42:23.890786 ip-10-0-143-17 kubenswrapper[2585]: I0710 18:42:23.890749 2585 generic.go:356] "Generic (PLEG): container finished" podID="dc58c93b-5a2c-4f55-ae5e-f9cddf66101e" containerID="879a6cc4411ed8e54e788f74c065e43973ceaf0b7a3395edc2f13b11e071aeb0" exitCode=0 Jul 10 18:42:23.891142 ip-10-0-143-17 kubenswrapper[2585]: I0710 18:42:23.890795 2585 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-console/console-7dcf5c576-vqjdl" Jul 10 18:42:23.891142 ip-10-0-143-17 kubenswrapper[2585]: I0710 18:42:23.890830 2585 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-console/console-7dcf5c576-vqjdl" event={"ID":"dc58c93b-5a2c-4f55-ae5e-f9cddf66101e","Type":"ContainerDied","Data":"879a6cc4411ed8e54e788f74c065e43973ceaf0b7a3395edc2f13b11e071aeb0"} Jul 10 18:42:23.891142 ip-10-0-143-17 kubenswrapper[2585]: I0710 18:42:23.890881 2585 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-console/console-7dcf5c576-vqjdl" event={"ID":"dc58c93b-5a2c-4f55-ae5e-f9cddf66101e","Type":"ContainerDied","Data":"7b1cb2a245e10fcc3433b153d3068c749d1e1b66d09473c9dfc3b77d0e946bb8"} Jul 10 18:42:23.891142 ip-10-0-143-17 kubenswrapper[2585]: I0710 18:42:23.890904 2585 scope.go:117] "RemoveContainer" containerID="879a6cc4411ed8e54e788f74c065e43973ceaf0b7a3395edc2f13b11e071aeb0" Jul 10 18:42:23.899510 ip-10-0-143-17 kubenswrapper[2585]: I0710 18:42:23.899495 2585 scope.go:117] "RemoveContainer" containerID="879a6cc4411ed8e54e788f74c065e43973ceaf0b7a3395edc2f13b11e071aeb0" Jul 10 18:42:23.899767 ip-10-0-143-17 kubenswrapper[2585]: E0710 18:42:23.899749 2585 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"879a6cc4411ed8e54e788f74c065e43973ceaf0b7a3395edc2f13b11e071aeb0\": container with ID starting with 879a6cc4411ed8e54e788f74c065e43973ceaf0b7a3395edc2f13b11e071aeb0 not found: ID does not exist" containerID="879a6cc4411ed8e54e788f74c065e43973ceaf0b7a3395edc2f13b11e071aeb0" Jul 10 18:42:23.899826 ip-10-0-143-17 kubenswrapper[2585]: I0710 18:42:23.899776 2585 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"879a6cc4411ed8e54e788f74c065e43973ceaf0b7a3395edc2f13b11e071aeb0"} err="failed to get container status \"879a6cc4411ed8e54e788f74c065e43973ceaf0b7a3395edc2f13b11e071aeb0\": rpc error: code = NotFound desc = could not find container \"879a6cc4411ed8e54e788f74c065e43973ceaf0b7a3395edc2f13b11e071aeb0\": container with ID starting with 879a6cc4411ed8e54e788f74c065e43973ceaf0b7a3395edc2f13b11e071aeb0 not found: ID does not exist" Jul 10 18:42:23.912038 ip-10-0-143-17 kubenswrapper[2585]: I0710 18:42:23.912019 2585 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-console/console-7dcf5c576-vqjdl"] Jul 10 18:42:23.915240 ip-10-0-143-17 kubenswrapper[2585]: I0710 18:42:23.915219 2585 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["openshift-console/console-7dcf5c576-vqjdl"] Jul 10 18:42:25.077123 ip-10-0-143-17 kubenswrapper[2585]: I0710 18:42:25.077089 2585 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="openshift-monitoring/metrics-server-688c68676d-bdllx" Jul 10 18:42:25.080813 ip-10-0-143-17 kubenswrapper[2585]: I0710 18:42:25.080789 2585 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-monitoring/metrics-server-688c68676d-bdllx" Jul 10 18:42:25.215211 ip-10-0-143-17 kubenswrapper[2585]: I0710 18:42:25.215184 2585 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="dc58c93b-5a2c-4f55-ae5e-f9cddf66101e" path="/var/lib/kubelet/pods/dc58c93b-5a2c-4f55-ae5e-f9cddf66101e/volumes" Jul 10 18:43:04.635745 ip-10-0-143-17 kubenswrapper[2585]: I0710 18:43:04.635707 2585 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-monitoring/telemeter-client-7449dcbd6f-vvl9q"] Jul 10 18:43:04.636210 ip-10-0-143-17 kubenswrapper[2585]: I0710 18:43:04.635975 2585 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="f750992a-c8d1-42a2-81e1-c657404d690f" containerName="console" Jul 10 18:43:04.636210 ip-10-0-143-17 kubenswrapper[2585]: I0710 18:43:04.635986 2585 state_mem.go:107] "Deleted CPUSet assignment" podUID="f750992a-c8d1-42a2-81e1-c657404d690f" containerName="console" Jul 10 18:43:04.636210 ip-10-0-143-17 kubenswrapper[2585]: I0710 18:43:04.636003 2585 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="dc58c93b-5a2c-4f55-ae5e-f9cddf66101e" containerName="console" Jul 10 18:43:04.636210 ip-10-0-143-17 kubenswrapper[2585]: I0710 18:43:04.636008 2585 state_mem.go:107] "Deleted CPUSet assignment" podUID="dc58c93b-5a2c-4f55-ae5e-f9cddf66101e" containerName="console" Jul 10 18:43:04.636210 ip-10-0-143-17 kubenswrapper[2585]: I0710 18:43:04.636052 2585 memory_manager.go:368] "RemoveStaleState removing state" podUID="dc58c93b-5a2c-4f55-ae5e-f9cddf66101e" containerName="console" Jul 10 18:43:04.636210 ip-10-0-143-17 kubenswrapper[2585]: I0710 18:43:04.636062 2585 memory_manager.go:368] "RemoveStaleState removing state" podUID="f750992a-c8d1-42a2-81e1-c657404d690f" containerName="console" Jul 10 18:43:04.639051 ip-10-0-143-17 kubenswrapper[2585]: I0710 18:43:04.639026 2585 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-monitoring/telemeter-client-7449dcbd6f-vvl9q" Jul 10 18:43:04.644864 ip-10-0-143-17 kubenswrapper[2585]: I0710 18:43:04.644824 2585 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"telemeter-client\"" Jul 10 18:43:04.645021 ip-10-0-143-17 kubenswrapper[2585]: I0710 18:43:04.644844 2585 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"federate-client-certs\"" Jul 10 18:43:04.645021 ip-10-0-143-17 kubenswrapper[2585]: I0710 18:43:04.644939 2585 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"telemeter-client-tls\"" Jul 10 18:43:04.645341 ip-10-0-143-17 kubenswrapper[2585]: I0710 18:43:04.645318 2585 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"telemeter-client-kube-rbac-proxy-config\"" Jul 10 18:43:04.646572 ip-10-0-143-17 kubenswrapper[2585]: I0710 18:43:04.646553 2585 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-monitoring\"/\"telemeter-client-serving-certs-ca-bundle\"" Jul 10 18:43:04.656714 ip-10-0-143-17 kubenswrapper[2585]: I0710 18:43:04.656699 2585 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"telemeter-client-dockercfg-mrn8m\"" Jul 10 18:43:04.660637 ip-10-0-143-17 kubenswrapper[2585]: I0710 18:43:04.660619 2585 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-monitoring\"/\"telemeter-trusted-ca-bundle-8i12ta5c71j38\"" Jul 10 18:43:04.665524 ip-10-0-143-17 kubenswrapper[2585]: I0710 18:43:04.665502 2585 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"telemeter-trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/68290b3e-a6e8-42c4-aab9-9147abee2039-telemeter-trusted-ca-bundle\") pod \"telemeter-client-7449dcbd6f-vvl9q\" (UID: \"68290b3e-a6e8-42c4-aab9-9147abee2039\") " pod="openshift-monitoring/telemeter-client-7449dcbd6f-vvl9q" Jul 10 18:43:04.665617 ip-10-0-143-17 kubenswrapper[2585]: I0710 18:43:04.665556 2585 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"federate-client-tls\" (UniqueName: \"kubernetes.io/secret/68290b3e-a6e8-42c4-aab9-9147abee2039-federate-client-tls\") pod \"telemeter-client-7449dcbd6f-vvl9q\" (UID: \"68290b3e-a6e8-42c4-aab9-9147abee2039\") " pod="openshift-monitoring/telemeter-client-7449dcbd6f-vvl9q" Jul 10 18:43:04.665617 ip-10-0-143-17 kubenswrapper[2585]: I0710 18:43:04.665593 2585 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"metrics-client-ca\" (UniqueName: \"kubernetes.io/configmap/68290b3e-a6e8-42c4-aab9-9147abee2039-metrics-client-ca\") pod \"telemeter-client-7449dcbd6f-vvl9q\" (UID: \"68290b3e-a6e8-42c4-aab9-9147abee2039\") " pod="openshift-monitoring/telemeter-client-7449dcbd6f-vvl9q" Jul 10 18:43:04.665705 ip-10-0-143-17 kubenswrapper[2585]: I0710 18:43:04.665659 2585 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"serving-certs-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/68290b3e-a6e8-42c4-aab9-9147abee2039-serving-certs-ca-bundle\") pod \"telemeter-client-7449dcbd6f-vvl9q\" (UID: \"68290b3e-a6e8-42c4-aab9-9147abee2039\") " pod="openshift-monitoring/telemeter-client-7449dcbd6f-vvl9q" Jul 10 18:43:04.665705 ip-10-0-143-17 kubenswrapper[2585]: I0710 18:43:04.665686 2585 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"secret-telemeter-client\" (UniqueName: \"kubernetes.io/secret/68290b3e-a6e8-42c4-aab9-9147abee2039-secret-telemeter-client\") pod \"telemeter-client-7449dcbd6f-vvl9q\" (UID: \"68290b3e-a6e8-42c4-aab9-9147abee2039\") " pod="openshift-monitoring/telemeter-client-7449dcbd6f-vvl9q" Jul 10 18:43:04.665797 ip-10-0-143-17 kubenswrapper[2585]: I0710 18:43:04.665723 2585 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-2vn46\" (UniqueName: \"kubernetes.io/projected/68290b3e-a6e8-42c4-aab9-9147abee2039-kube-api-access-2vn46\") pod \"telemeter-client-7449dcbd6f-vvl9q\" (UID: \"68290b3e-a6e8-42c4-aab9-9147abee2039\") " pod="openshift-monitoring/telemeter-client-7449dcbd6f-vvl9q" Jul 10 18:43:04.665842 ip-10-0-143-17 kubenswrapper[2585]: I0710 18:43:04.665798 2585 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"telemeter-client-tls\" (UniqueName: \"kubernetes.io/secret/68290b3e-a6e8-42c4-aab9-9147abee2039-telemeter-client-tls\") pod \"telemeter-client-7449dcbd6f-vvl9q\" (UID: \"68290b3e-a6e8-42c4-aab9-9147abee2039\") " pod="openshift-monitoring/telemeter-client-7449dcbd6f-vvl9q" Jul 10 18:43:04.665930 ip-10-0-143-17 kubenswrapper[2585]: I0710 18:43:04.665884 2585 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"secret-telemeter-client-kube-rbac-proxy-config\" (UniqueName: \"kubernetes.io/secret/68290b3e-a6e8-42c4-aab9-9147abee2039-secret-telemeter-client-kube-rbac-proxy-config\") pod \"telemeter-client-7449dcbd6f-vvl9q\" (UID: \"68290b3e-a6e8-42c4-aab9-9147abee2039\") " pod="openshift-monitoring/telemeter-client-7449dcbd6f-vvl9q" Jul 10 18:43:04.678745 ip-10-0-143-17 kubenswrapper[2585]: I0710 18:43:04.678703 2585 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-monitoring/telemeter-client-7449dcbd6f-vvl9q"] Jul 10 18:43:04.766970 ip-10-0-143-17 kubenswrapper[2585]: I0710 18:43:04.766923 2585 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-2vn46\" (UniqueName: \"kubernetes.io/projected/68290b3e-a6e8-42c4-aab9-9147abee2039-kube-api-access-2vn46\") pod \"telemeter-client-7449dcbd6f-vvl9q\" (UID: \"68290b3e-a6e8-42c4-aab9-9147abee2039\") " pod="openshift-monitoring/telemeter-client-7449dcbd6f-vvl9q" Jul 10 18:43:04.767100 ip-10-0-143-17 kubenswrapper[2585]: I0710 18:43:04.766986 2585 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"telemeter-client-tls\" (UniqueName: \"kubernetes.io/secret/68290b3e-a6e8-42c4-aab9-9147abee2039-telemeter-client-tls\") pod \"telemeter-client-7449dcbd6f-vvl9q\" (UID: \"68290b3e-a6e8-42c4-aab9-9147abee2039\") " pod="openshift-monitoring/telemeter-client-7449dcbd6f-vvl9q" Jul 10 18:43:04.767100 ip-10-0-143-17 kubenswrapper[2585]: I0710 18:43:04.767034 2585 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"secret-telemeter-client-kube-rbac-proxy-config\" (UniqueName: \"kubernetes.io/secret/68290b3e-a6e8-42c4-aab9-9147abee2039-secret-telemeter-client-kube-rbac-proxy-config\") pod \"telemeter-client-7449dcbd6f-vvl9q\" (UID: \"68290b3e-a6e8-42c4-aab9-9147abee2039\") " pod="openshift-monitoring/telemeter-client-7449dcbd6f-vvl9q" Jul 10 18:43:04.767298 ip-10-0-143-17 kubenswrapper[2585]: I0710 18:43:04.767278 2585 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"telemeter-trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/68290b3e-a6e8-42c4-aab9-9147abee2039-telemeter-trusted-ca-bundle\") pod \"telemeter-client-7449dcbd6f-vvl9q\" (UID: \"68290b3e-a6e8-42c4-aab9-9147abee2039\") " pod="openshift-monitoring/telemeter-client-7449dcbd6f-vvl9q" Jul 10 18:43:04.767366 ip-10-0-143-17 kubenswrapper[2585]: I0710 18:43:04.767315 2585 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"federate-client-tls\" (UniqueName: \"kubernetes.io/secret/68290b3e-a6e8-42c4-aab9-9147abee2039-federate-client-tls\") pod \"telemeter-client-7449dcbd6f-vvl9q\" (UID: \"68290b3e-a6e8-42c4-aab9-9147abee2039\") " pod="openshift-monitoring/telemeter-client-7449dcbd6f-vvl9q" Jul 10 18:43:04.767366 ip-10-0-143-17 kubenswrapper[2585]: I0710 18:43:04.767339 2585 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-client-ca\" (UniqueName: \"kubernetes.io/configmap/68290b3e-a6e8-42c4-aab9-9147abee2039-metrics-client-ca\") pod \"telemeter-client-7449dcbd6f-vvl9q\" (UID: \"68290b3e-a6e8-42c4-aab9-9147abee2039\") " pod="openshift-monitoring/telemeter-client-7449dcbd6f-vvl9q" Jul 10 18:43:04.767469 ip-10-0-143-17 kubenswrapper[2585]: I0710 18:43:04.767383 2585 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"serving-certs-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/68290b3e-a6e8-42c4-aab9-9147abee2039-serving-certs-ca-bundle\") pod \"telemeter-client-7449dcbd6f-vvl9q\" (UID: \"68290b3e-a6e8-42c4-aab9-9147abee2039\") " pod="openshift-monitoring/telemeter-client-7449dcbd6f-vvl9q" Jul 10 18:43:04.767469 ip-10-0-143-17 kubenswrapper[2585]: I0710 18:43:04.767412 2585 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"secret-telemeter-client\" (UniqueName: \"kubernetes.io/secret/68290b3e-a6e8-42c4-aab9-9147abee2039-secret-telemeter-client\") pod \"telemeter-client-7449dcbd6f-vvl9q\" (UID: \"68290b3e-a6e8-42c4-aab9-9147abee2039\") " pod="openshift-monitoring/telemeter-client-7449dcbd6f-vvl9q" Jul 10 18:43:04.768164 ip-10-0-143-17 kubenswrapper[2585]: I0710 18:43:04.768115 2585 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"serving-certs-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/68290b3e-a6e8-42c4-aab9-9147abee2039-serving-certs-ca-bundle\") pod \"telemeter-client-7449dcbd6f-vvl9q\" (UID: \"68290b3e-a6e8-42c4-aab9-9147abee2039\") " pod="openshift-monitoring/telemeter-client-7449dcbd6f-vvl9q" Jul 10 18:43:04.768330 ip-10-0-143-17 kubenswrapper[2585]: I0710 18:43:04.768309 2585 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"metrics-client-ca\" (UniqueName: \"kubernetes.io/configmap/68290b3e-a6e8-42c4-aab9-9147abee2039-metrics-client-ca\") pod \"telemeter-client-7449dcbd6f-vvl9q\" (UID: \"68290b3e-a6e8-42c4-aab9-9147abee2039\") " pod="openshift-monitoring/telemeter-client-7449dcbd6f-vvl9q" Jul 10 18:43:04.768567 ip-10-0-143-17 kubenswrapper[2585]: I0710 18:43:04.768523 2585 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"telemeter-trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/68290b3e-a6e8-42c4-aab9-9147abee2039-telemeter-trusted-ca-bundle\") pod \"telemeter-client-7449dcbd6f-vvl9q\" (UID: \"68290b3e-a6e8-42c4-aab9-9147abee2039\") " pod="openshift-monitoring/telemeter-client-7449dcbd6f-vvl9q" Jul 10 18:43:04.770191 ip-10-0-143-17 kubenswrapper[2585]: I0710 18:43:04.770119 2585 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"telemeter-client-tls\" (UniqueName: \"kubernetes.io/secret/68290b3e-a6e8-42c4-aab9-9147abee2039-telemeter-client-tls\") pod \"telemeter-client-7449dcbd6f-vvl9q\" (UID: \"68290b3e-a6e8-42c4-aab9-9147abee2039\") " pod="openshift-monitoring/telemeter-client-7449dcbd6f-vvl9q" Jul 10 18:43:04.770191 ip-10-0-143-17 kubenswrapper[2585]: I0710 18:43:04.770138 2585 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"secret-telemeter-client-kube-rbac-proxy-config\" (UniqueName: \"kubernetes.io/secret/68290b3e-a6e8-42c4-aab9-9147abee2039-secret-telemeter-client-kube-rbac-proxy-config\") pod \"telemeter-client-7449dcbd6f-vvl9q\" (UID: \"68290b3e-a6e8-42c4-aab9-9147abee2039\") " pod="openshift-monitoring/telemeter-client-7449dcbd6f-vvl9q" Jul 10 18:43:04.770319 ip-10-0-143-17 kubenswrapper[2585]: I0710 18:43:04.770214 2585 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"federate-client-tls\" (UniqueName: \"kubernetes.io/secret/68290b3e-a6e8-42c4-aab9-9147abee2039-federate-client-tls\") pod \"telemeter-client-7449dcbd6f-vvl9q\" (UID: \"68290b3e-a6e8-42c4-aab9-9147abee2039\") " pod="openshift-monitoring/telemeter-client-7449dcbd6f-vvl9q" Jul 10 18:43:04.771858 ip-10-0-143-17 kubenswrapper[2585]: I0710 18:43:04.771838 2585 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"secret-telemeter-client\" (UniqueName: \"kubernetes.io/secret/68290b3e-a6e8-42c4-aab9-9147abee2039-secret-telemeter-client\") pod \"telemeter-client-7449dcbd6f-vvl9q\" (UID: \"68290b3e-a6e8-42c4-aab9-9147abee2039\") " pod="openshift-monitoring/telemeter-client-7449dcbd6f-vvl9q" Jul 10 18:43:04.777990 ip-10-0-143-17 kubenswrapper[2585]: I0710 18:43:04.777966 2585 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-2vn46\" (UniqueName: \"kubernetes.io/projected/68290b3e-a6e8-42c4-aab9-9147abee2039-kube-api-access-2vn46\") pod \"telemeter-client-7449dcbd6f-vvl9q\" (UID: \"68290b3e-a6e8-42c4-aab9-9147abee2039\") " pod="openshift-monitoring/telemeter-client-7449dcbd6f-vvl9q" Jul 10 18:43:04.949055 ip-10-0-143-17 kubenswrapper[2585]: I0710 18:43:04.948969 2585 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-monitoring/telemeter-client-7449dcbd6f-vvl9q" Jul 10 18:43:05.105487 ip-10-0-143-17 kubenswrapper[2585]: I0710 18:43:05.105447 2585 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-monitoring/telemeter-client-7449dcbd6f-vvl9q"] Jul 10 18:43:05.108156 ip-10-0-143-17 kubenswrapper[2585]: W0710 18:43:05.108129 2585 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod68290b3e_a6e8_42c4_aab9_9147abee2039.slice/crio-54138b9830e43fff94e109f9e64ee53aa07998e32ee4a8eb5aabebbc098e9a3d WatchSource:0}: Error finding container 54138b9830e43fff94e109f9e64ee53aa07998e32ee4a8eb5aabebbc098e9a3d: Status 404 returned error can't find the container with id 54138b9830e43fff94e109f9e64ee53aa07998e32ee4a8eb5aabebbc098e9a3d Jul 10 18:43:06.049515 ip-10-0-143-17 kubenswrapper[2585]: I0710 18:43:06.049473 2585 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/telemeter-client-7449dcbd6f-vvl9q" event={"ID":"68290b3e-a6e8-42c4-aab9-9147abee2039","Type":"ContainerStarted","Data":"54138b9830e43fff94e109f9e64ee53aa07998e32ee4a8eb5aabebbc098e9a3d"} Jul 10 18:43:08.060585 ip-10-0-143-17 kubenswrapper[2585]: I0710 18:43:08.060532 2585 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/telemeter-client-7449dcbd6f-vvl9q" event={"ID":"68290b3e-a6e8-42c4-aab9-9147abee2039","Type":"ContainerStarted","Data":"a234c3c9c7f93f03b8a620e1e8f1d9d28b8778f801ce049bab3299b45167c8fe"} Jul 10 18:43:09.066653 ip-10-0-143-17 kubenswrapper[2585]: I0710 18:43:09.066618 2585 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/telemeter-client-7449dcbd6f-vvl9q" event={"ID":"68290b3e-a6e8-42c4-aab9-9147abee2039","Type":"ContainerStarted","Data":"d0dea0b7e7fdd18132031f8c3e0542077819d0071b3bad63b448372eee501b6a"} Jul 10 18:43:09.067025 ip-10-0-143-17 kubenswrapper[2585]: I0710 18:43:09.066677 2585 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/telemeter-client-7449dcbd6f-vvl9q" event={"ID":"68290b3e-a6e8-42c4-aab9-9147abee2039","Type":"ContainerStarted","Data":"2f2eed01234089650a26b6a065d04ae11eb5bafba92e9c16087ca16001a249de"} Jul 10 18:43:09.094235 ip-10-0-143-17 kubenswrapper[2585]: I0710 18:43:09.094188 2585 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-monitoring/telemeter-client-7449dcbd6f-vvl9q" podStartSLOduration=2.035327019 podStartE2EDuration="5.094168486s" totalImagesPullingTime="3.058841467s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-10 18:43:04 +0000 UTC" imagePullSessionsCount=2 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-10 18:43:09.092849763 +0000 UTC m=+202.550209719" watchObservedRunningTime="2026-07-10 18:43:09.094168486 +0000 UTC m=+202.551528420" Jul 10 18:43:09.743228 ip-10-0-143-17 kubenswrapper[2585]: I0710 18:43:09.743196 2585 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-console/console-6b797595fb-lqkgg"] Jul 10 18:43:09.746243 ip-10-0-143-17 kubenswrapper[2585]: I0710 18:43:09.746219 2585 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-console/console-6b797595fb-lqkgg" Jul 10 18:43:09.760422 ip-10-0-143-17 kubenswrapper[2585]: I0710 18:43:09.760398 2585 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-console/console-6b797595fb-lqkgg"] Jul 10 18:43:09.806442 ip-10-0-143-17 kubenswrapper[2585]: I0710 18:43:09.805442 2585 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"console-serving-cert\" (UniqueName: \"kubernetes.io/secret/26cdaf91-208c-4e42-bf79-b0a257e34dab-console-serving-cert\") pod \"console-6b797595fb-lqkgg\" (UID: \"26cdaf91-208c-4e42-bf79-b0a257e34dab\") " pod="openshift-console/console-6b797595fb-lqkgg" Jul 10 18:43:09.806442 ip-10-0-143-17 kubenswrapper[2585]: I0710 18:43:09.805482 2585 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"service-ca\" (UniqueName: \"kubernetes.io/configmap/26cdaf91-208c-4e42-bf79-b0a257e34dab-service-ca\") pod \"console-6b797595fb-lqkgg\" (UID: \"26cdaf91-208c-4e42-bf79-b0a257e34dab\") " pod="openshift-console/console-6b797595fb-lqkgg" Jul 10 18:43:09.806442 ip-10-0-143-17 kubenswrapper[2585]: I0710 18:43:09.805515 2585 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"console-config\" (UniqueName: \"kubernetes.io/configmap/26cdaf91-208c-4e42-bf79-b0a257e34dab-console-config\") pod \"console-6b797595fb-lqkgg\" (UID: \"26cdaf91-208c-4e42-bf79-b0a257e34dab\") " pod="openshift-console/console-6b797595fb-lqkgg" Jul 10 18:43:09.806442 ip-10-0-143-17 kubenswrapper[2585]: I0710 18:43:09.805564 2585 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-mfllj\" (UniqueName: \"kubernetes.io/projected/26cdaf91-208c-4e42-bf79-b0a257e34dab-kube-api-access-mfllj\") pod \"console-6b797595fb-lqkgg\" (UID: \"26cdaf91-208c-4e42-bf79-b0a257e34dab\") " pod="openshift-console/console-6b797595fb-lqkgg" Jul 10 18:43:09.806442 ip-10-0-143-17 kubenswrapper[2585]: I0710 18:43:09.805605 2585 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/26cdaf91-208c-4e42-bf79-b0a257e34dab-trusted-ca-bundle\") pod \"console-6b797595fb-lqkgg\" (UID: \"26cdaf91-208c-4e42-bf79-b0a257e34dab\") " pod="openshift-console/console-6b797595fb-lqkgg" Jul 10 18:43:09.806442 ip-10-0-143-17 kubenswrapper[2585]: I0710 18:43:09.805692 2585 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"console-oauth-config\" (UniqueName: \"kubernetes.io/secret/26cdaf91-208c-4e42-bf79-b0a257e34dab-console-oauth-config\") pod \"console-6b797595fb-lqkgg\" (UID: \"26cdaf91-208c-4e42-bf79-b0a257e34dab\") " pod="openshift-console/console-6b797595fb-lqkgg" Jul 10 18:43:09.806442 ip-10-0-143-17 kubenswrapper[2585]: I0710 18:43:09.805726 2585 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"oauth-serving-cert\" (UniqueName: \"kubernetes.io/configmap/26cdaf91-208c-4e42-bf79-b0a257e34dab-oauth-serving-cert\") pod \"console-6b797595fb-lqkgg\" (UID: \"26cdaf91-208c-4e42-bf79-b0a257e34dab\") " pod="openshift-console/console-6b797595fb-lqkgg" Jul 10 18:43:09.906673 ip-10-0-143-17 kubenswrapper[2585]: I0710 18:43:09.906634 2585 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/26cdaf91-208c-4e42-bf79-b0a257e34dab-trusted-ca-bundle\") pod \"console-6b797595fb-lqkgg\" (UID: \"26cdaf91-208c-4e42-bf79-b0a257e34dab\") " pod="openshift-console/console-6b797595fb-lqkgg" Jul 10 18:43:09.906816 ip-10-0-143-17 kubenswrapper[2585]: I0710 18:43:09.906734 2585 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"console-oauth-config\" (UniqueName: \"kubernetes.io/secret/26cdaf91-208c-4e42-bf79-b0a257e34dab-console-oauth-config\") pod \"console-6b797595fb-lqkgg\" (UID: \"26cdaf91-208c-4e42-bf79-b0a257e34dab\") " pod="openshift-console/console-6b797595fb-lqkgg" Jul 10 18:43:09.906947 ip-10-0-143-17 kubenswrapper[2585]: I0710 18:43:09.906927 2585 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"oauth-serving-cert\" (UniqueName: \"kubernetes.io/configmap/26cdaf91-208c-4e42-bf79-b0a257e34dab-oauth-serving-cert\") pod \"console-6b797595fb-lqkgg\" (UID: \"26cdaf91-208c-4e42-bf79-b0a257e34dab\") " pod="openshift-console/console-6b797595fb-lqkgg" Jul 10 18:43:09.908513 ip-10-0-143-17 kubenswrapper[2585]: I0710 18:43:09.908480 2585 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"console-serving-cert\" (UniqueName: \"kubernetes.io/secret/26cdaf91-208c-4e42-bf79-b0a257e34dab-console-serving-cert\") pod \"console-6b797595fb-lqkgg\" (UID: \"26cdaf91-208c-4e42-bf79-b0a257e34dab\") " pod="openshift-console/console-6b797595fb-lqkgg" Jul 10 18:43:09.908688 ip-10-0-143-17 kubenswrapper[2585]: I0710 18:43:09.908578 2585 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"service-ca\" (UniqueName: \"kubernetes.io/configmap/26cdaf91-208c-4e42-bf79-b0a257e34dab-service-ca\") pod \"console-6b797595fb-lqkgg\" (UID: \"26cdaf91-208c-4e42-bf79-b0a257e34dab\") " pod="openshift-console/console-6b797595fb-lqkgg" Jul 10 18:43:09.908749 ip-10-0-143-17 kubenswrapper[2585]: I0710 18:43:09.908728 2585 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"console-config\" (UniqueName: \"kubernetes.io/configmap/26cdaf91-208c-4e42-bf79-b0a257e34dab-console-config\") pod \"console-6b797595fb-lqkgg\" (UID: \"26cdaf91-208c-4e42-bf79-b0a257e34dab\") " pod="openshift-console/console-6b797595fb-lqkgg" Jul 10 18:43:09.908798 ip-10-0-143-17 kubenswrapper[2585]: I0710 18:43:09.908763 2585 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-mfllj\" (UniqueName: \"kubernetes.io/projected/26cdaf91-208c-4e42-bf79-b0a257e34dab-kube-api-access-mfllj\") pod \"console-6b797595fb-lqkgg\" (UID: \"26cdaf91-208c-4e42-bf79-b0a257e34dab\") " pod="openshift-console/console-6b797595fb-lqkgg" Jul 10 18:43:09.908874 ip-10-0-143-17 kubenswrapper[2585]: I0710 18:43:09.908847 2585 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"oauth-serving-cert\" (UniqueName: \"kubernetes.io/configmap/26cdaf91-208c-4e42-bf79-b0a257e34dab-oauth-serving-cert\") pod \"console-6b797595fb-lqkgg\" (UID: \"26cdaf91-208c-4e42-bf79-b0a257e34dab\") " pod="openshift-console/console-6b797595fb-lqkgg" Jul 10 18:43:09.913378 ip-10-0-143-17 kubenswrapper[2585]: I0710 18:43:09.909265 2585 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/26cdaf91-208c-4e42-bf79-b0a257e34dab-trusted-ca-bundle\") pod \"console-6b797595fb-lqkgg\" (UID: \"26cdaf91-208c-4e42-bf79-b0a257e34dab\") " pod="openshift-console/console-6b797595fb-lqkgg" Jul 10 18:43:09.913378 ip-10-0-143-17 kubenswrapper[2585]: I0710 18:43:09.909462 2585 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"console-config\" (UniqueName: \"kubernetes.io/configmap/26cdaf91-208c-4e42-bf79-b0a257e34dab-console-config\") pod \"console-6b797595fb-lqkgg\" (UID: \"26cdaf91-208c-4e42-bf79-b0a257e34dab\") " pod="openshift-console/console-6b797595fb-lqkgg" Jul 10 18:43:09.913378 ip-10-0-143-17 kubenswrapper[2585]: I0710 18:43:09.909668 2585 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"service-ca\" (UniqueName: \"kubernetes.io/configmap/26cdaf91-208c-4e42-bf79-b0a257e34dab-service-ca\") pod \"console-6b797595fb-lqkgg\" (UID: \"26cdaf91-208c-4e42-bf79-b0a257e34dab\") " pod="openshift-console/console-6b797595fb-lqkgg" Jul 10 18:43:09.914948 ip-10-0-143-17 kubenswrapper[2585]: I0710 18:43:09.914921 2585 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"console-oauth-config\" (UniqueName: \"kubernetes.io/secret/26cdaf91-208c-4e42-bf79-b0a257e34dab-console-oauth-config\") pod \"console-6b797595fb-lqkgg\" (UID: \"26cdaf91-208c-4e42-bf79-b0a257e34dab\") " pod="openshift-console/console-6b797595fb-lqkgg" Jul 10 18:43:09.915462 ip-10-0-143-17 kubenswrapper[2585]: I0710 18:43:09.915430 2585 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"console-serving-cert\" (UniqueName: \"kubernetes.io/secret/26cdaf91-208c-4e42-bf79-b0a257e34dab-console-serving-cert\") pod \"console-6b797595fb-lqkgg\" (UID: \"26cdaf91-208c-4e42-bf79-b0a257e34dab\") " pod="openshift-console/console-6b797595fb-lqkgg" Jul 10 18:43:09.919239 ip-10-0-143-17 kubenswrapper[2585]: I0710 18:43:09.919218 2585 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-mfllj\" (UniqueName: \"kubernetes.io/projected/26cdaf91-208c-4e42-bf79-b0a257e34dab-kube-api-access-mfllj\") pod \"console-6b797595fb-lqkgg\" (UID: \"26cdaf91-208c-4e42-bf79-b0a257e34dab\") " pod="openshift-console/console-6b797595fb-lqkgg" Jul 10 18:43:10.056486 ip-10-0-143-17 kubenswrapper[2585]: I0710 18:43:10.056391 2585 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-console/console-6b797595fb-lqkgg" Jul 10 18:43:10.198479 ip-10-0-143-17 kubenswrapper[2585]: I0710 18:43:10.198456 2585 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-console/console-6b797595fb-lqkgg"] Jul 10 18:43:10.200276 ip-10-0-143-17 kubenswrapper[2585]: W0710 18:43:10.200249 2585 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod26cdaf91_208c_4e42_bf79_b0a257e34dab.slice/crio-dfd0b193e141ba6c1943fd5ddbd5d831852d67b1059803c338b7635d520102bd WatchSource:0}: Error finding container dfd0b193e141ba6c1943fd5ddbd5d831852d67b1059803c338b7635d520102bd: Status 404 returned error can't find the container with id dfd0b193e141ba6c1943fd5ddbd5d831852d67b1059803c338b7635d520102bd Jul 10 18:43:11.075864 ip-10-0-143-17 kubenswrapper[2585]: I0710 18:43:11.075826 2585 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-console/console-6b797595fb-lqkgg" event={"ID":"26cdaf91-208c-4e42-bf79-b0a257e34dab","Type":"ContainerStarted","Data":"3eee74e3ca90292736aaf8869cb6a2d31c56e507b8866091b76d07ec019873ed"} Jul 10 18:43:11.076055 ip-10-0-143-17 kubenswrapper[2585]: I0710 18:43:11.075887 2585 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-console/console-6b797595fb-lqkgg" event={"ID":"26cdaf91-208c-4e42-bf79-b0a257e34dab","Type":"ContainerStarted","Data":"dfd0b193e141ba6c1943fd5ddbd5d831852d67b1059803c338b7635d520102bd"} Jul 10 18:43:11.099243 ip-10-0-143-17 kubenswrapper[2585]: I0710 18:43:11.099192 2585 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-console/console-6b797595fb-lqkgg" podStartSLOduration=2.099176897 podStartE2EDuration="2.099176897s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-10 18:43:09 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-10 18:43:11.09758484 +0000 UTC m=+204.554944795" watchObservedRunningTime="2026-07-10 18:43:11.099176897 +0000 UTC m=+204.556536833" Jul 10 18:43:20.056861 ip-10-0-143-17 kubenswrapper[2585]: I0710 18:43:20.056826 2585 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="openshift-console/console-6b797595fb-lqkgg" Jul 10 18:43:20.057315 ip-10-0-143-17 kubenswrapper[2585]: I0710 18:43:20.056885 2585 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-console/console-6b797595fb-lqkgg" Jul 10 18:43:20.061515 ip-10-0-143-17 kubenswrapper[2585]: I0710 18:43:20.061493 2585 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="openshift-console/console-6b797595fb-lqkgg" Jul 10 18:43:20.115240 ip-10-0-143-17 kubenswrapper[2585]: I0710 18:43:20.115216 2585 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-console/console-6b797595fb-lqkgg" Jul 10 18:43:20.172710 ip-10-0-143-17 kubenswrapper[2585]: I0710 18:43:20.172669 2585 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-console/console-5c555b848f-4jxfq"] Jul 10 18:43:45.194626 ip-10-0-143-17 kubenswrapper[2585]: I0710 18:43:45.194589 2585 kuberuntime_container.go:873] "Killing container with a grace period" pod="openshift-console/console-5c555b848f-4jxfq" podUID="f449c3a6-ffbf-45f2-90b0-0ada0f1af9f3" containerName="console" containerID="cri-o://f134fcc8fc20876b1a629ada0d0079e7b1bd7db327d3d33b6ff28f17ae8408b2" gracePeriod=15 Jul 10 18:43:45.422778 ip-10-0-143-17 kubenswrapper[2585]: I0710 18:43:45.422750 2585 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-console/console-5c555b848f-4jxfq" Jul 10 18:43:45.564335 ip-10-0-143-17 kubenswrapper[2585]: I0710 18:43:45.564310 2585 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"console-config\" (UniqueName: \"kubernetes.io/configmap/f449c3a6-ffbf-45f2-90b0-0ada0f1af9f3-console-config\") pod \"f449c3a6-ffbf-45f2-90b0-0ada0f1af9f3\" (UID: \"f449c3a6-ffbf-45f2-90b0-0ada0f1af9f3\") " Jul 10 18:43:45.564467 ip-10-0-143-17 kubenswrapper[2585]: I0710 18:43:45.564348 2585 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"console-oauth-config\" (UniqueName: \"kubernetes.io/secret/f449c3a6-ffbf-45f2-90b0-0ada0f1af9f3-console-oauth-config\") pod \"f449c3a6-ffbf-45f2-90b0-0ada0f1af9f3\" (UID: \"f449c3a6-ffbf-45f2-90b0-0ada0f1af9f3\") " Jul 10 18:43:45.564467 ip-10-0-143-17 kubenswrapper[2585]: I0710 18:43:45.564373 2585 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"service-ca\" (UniqueName: \"kubernetes.io/configmap/f449c3a6-ffbf-45f2-90b0-0ada0f1af9f3-service-ca\") pod \"f449c3a6-ffbf-45f2-90b0-0ada0f1af9f3\" (UID: \"f449c3a6-ffbf-45f2-90b0-0ada0f1af9f3\") " Jul 10 18:43:45.564467 ip-10-0-143-17 kubenswrapper[2585]: I0710 18:43:45.564415 2585 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-4bg6l\" (UniqueName: \"kubernetes.io/projected/f449c3a6-ffbf-45f2-90b0-0ada0f1af9f3-kube-api-access-4bg6l\") pod \"f449c3a6-ffbf-45f2-90b0-0ada0f1af9f3\" (UID: \"f449c3a6-ffbf-45f2-90b0-0ada0f1af9f3\") " Jul 10 18:43:45.564467 ip-10-0-143-17 kubenswrapper[2585]: I0710 18:43:45.564450 2585 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/f449c3a6-ffbf-45f2-90b0-0ada0f1af9f3-trusted-ca-bundle\") pod \"f449c3a6-ffbf-45f2-90b0-0ada0f1af9f3\" (UID: \"f449c3a6-ffbf-45f2-90b0-0ada0f1af9f3\") " Jul 10 18:43:45.564671 ip-10-0-143-17 kubenswrapper[2585]: I0710 18:43:45.564564 2585 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"console-serving-cert\" (UniqueName: \"kubernetes.io/secret/f449c3a6-ffbf-45f2-90b0-0ada0f1af9f3-console-serving-cert\") pod \"f449c3a6-ffbf-45f2-90b0-0ada0f1af9f3\" (UID: \"f449c3a6-ffbf-45f2-90b0-0ada0f1af9f3\") " Jul 10 18:43:45.564671 ip-10-0-143-17 kubenswrapper[2585]: I0710 18:43:45.564623 2585 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"oauth-serving-cert\" (UniqueName: \"kubernetes.io/configmap/f449c3a6-ffbf-45f2-90b0-0ada0f1af9f3-oauth-serving-cert\") pod \"f449c3a6-ffbf-45f2-90b0-0ada0f1af9f3\" (UID: \"f449c3a6-ffbf-45f2-90b0-0ada0f1af9f3\") " Jul 10 18:43:45.564859 ip-10-0-143-17 kubenswrapper[2585]: I0710 18:43:45.564828 2585 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/f449c3a6-ffbf-45f2-90b0-0ada0f1af9f3-console-config" (OuterVolumeSpecName: "console-config") pod "f449c3a6-ffbf-45f2-90b0-0ada0f1af9f3" (UID: "f449c3a6-ffbf-45f2-90b0-0ada0f1af9f3"). InnerVolumeSpecName "console-config". PluginName "kubernetes.io/configmap", VolumeGIDValue "" Jul 10 18:43:45.565040 ip-10-0-143-17 kubenswrapper[2585]: I0710 18:43:45.565014 2585 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/f449c3a6-ffbf-45f2-90b0-0ada0f1af9f3-trusted-ca-bundle" (OuterVolumeSpecName: "trusted-ca-bundle") pod "f449c3a6-ffbf-45f2-90b0-0ada0f1af9f3" (UID: "f449c3a6-ffbf-45f2-90b0-0ada0f1af9f3"). InnerVolumeSpecName "trusted-ca-bundle". PluginName "kubernetes.io/configmap", VolumeGIDValue "" Jul 10 18:43:45.565040 ip-10-0-143-17 kubenswrapper[2585]: I0710 18:43:45.565012 2585 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/f449c3a6-ffbf-45f2-90b0-0ada0f1af9f3-service-ca" (OuterVolumeSpecName: "service-ca") pod "f449c3a6-ffbf-45f2-90b0-0ada0f1af9f3" (UID: "f449c3a6-ffbf-45f2-90b0-0ada0f1af9f3"). InnerVolumeSpecName "service-ca". PluginName "kubernetes.io/configmap", VolumeGIDValue "" Jul 10 18:43:45.565162 ip-10-0-143-17 kubenswrapper[2585]: I0710 18:43:45.565061 2585 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/f449c3a6-ffbf-45f2-90b0-0ada0f1af9f3-oauth-serving-cert" (OuterVolumeSpecName: "oauth-serving-cert") pod "f449c3a6-ffbf-45f2-90b0-0ada0f1af9f3" (UID: "f449c3a6-ffbf-45f2-90b0-0ada0f1af9f3"). InnerVolumeSpecName "oauth-serving-cert". PluginName "kubernetes.io/configmap", VolumeGIDValue "" Jul 10 18:43:45.566503 ip-10-0-143-17 kubenswrapper[2585]: I0710 18:43:45.566487 2585 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/f449c3a6-ffbf-45f2-90b0-0ada0f1af9f3-console-oauth-config" (OuterVolumeSpecName: "console-oauth-config") pod "f449c3a6-ffbf-45f2-90b0-0ada0f1af9f3" (UID: "f449c3a6-ffbf-45f2-90b0-0ada0f1af9f3"). InnerVolumeSpecName "console-oauth-config". PluginName "kubernetes.io/secret", VolumeGIDValue "" Jul 10 18:43:45.566974 ip-10-0-143-17 kubenswrapper[2585]: I0710 18:43:45.566954 2585 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/f449c3a6-ffbf-45f2-90b0-0ada0f1af9f3-console-serving-cert" (OuterVolumeSpecName: "console-serving-cert") pod "f449c3a6-ffbf-45f2-90b0-0ada0f1af9f3" (UID: "f449c3a6-ffbf-45f2-90b0-0ada0f1af9f3"). InnerVolumeSpecName "console-serving-cert". PluginName "kubernetes.io/secret", VolumeGIDValue "" Jul 10 18:43:45.567017 ip-10-0-143-17 kubenswrapper[2585]: I0710 18:43:45.566992 2585 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/f449c3a6-ffbf-45f2-90b0-0ada0f1af9f3-kube-api-access-4bg6l" (OuterVolumeSpecName: "kube-api-access-4bg6l") pod "f449c3a6-ffbf-45f2-90b0-0ada0f1af9f3" (UID: "f449c3a6-ffbf-45f2-90b0-0ada0f1af9f3"). InnerVolumeSpecName "kube-api-access-4bg6l". PluginName "kubernetes.io/projected", VolumeGIDValue "" Jul 10 18:43:45.665771 ip-10-0-143-17 kubenswrapper[2585]: I0710 18:43:45.665748 2585 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-4bg6l\" (UniqueName: \"kubernetes.io/projected/f449c3a6-ffbf-45f2-90b0-0ada0f1af9f3-kube-api-access-4bg6l\") on node \"ip-10-0-143-17.ec2.internal\" DevicePath \"\"" Jul 10 18:43:45.665771 ip-10-0-143-17 kubenswrapper[2585]: I0710 18:43:45.665769 2585 reconciler_common.go:299] "Volume detached for volume \"trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/f449c3a6-ffbf-45f2-90b0-0ada0f1af9f3-trusted-ca-bundle\") on node \"ip-10-0-143-17.ec2.internal\" DevicePath \"\"" Jul 10 18:43:45.665895 ip-10-0-143-17 kubenswrapper[2585]: I0710 18:43:45.665777 2585 reconciler_common.go:299] "Volume detached for volume \"console-serving-cert\" (UniqueName: \"kubernetes.io/secret/f449c3a6-ffbf-45f2-90b0-0ada0f1af9f3-console-serving-cert\") on node \"ip-10-0-143-17.ec2.internal\" DevicePath \"\"" Jul 10 18:43:45.665895 ip-10-0-143-17 kubenswrapper[2585]: I0710 18:43:45.665786 2585 reconciler_common.go:299] "Volume detached for volume \"oauth-serving-cert\" (UniqueName: \"kubernetes.io/configmap/f449c3a6-ffbf-45f2-90b0-0ada0f1af9f3-oauth-serving-cert\") on node \"ip-10-0-143-17.ec2.internal\" DevicePath \"\"" Jul 10 18:43:45.665895 ip-10-0-143-17 kubenswrapper[2585]: I0710 18:43:45.665794 2585 reconciler_common.go:299] "Volume detached for volume \"console-config\" (UniqueName: \"kubernetes.io/configmap/f449c3a6-ffbf-45f2-90b0-0ada0f1af9f3-console-config\") on node \"ip-10-0-143-17.ec2.internal\" DevicePath \"\"" Jul 10 18:43:45.665895 ip-10-0-143-17 kubenswrapper[2585]: I0710 18:43:45.665803 2585 reconciler_common.go:299] "Volume detached for volume \"console-oauth-config\" (UniqueName: \"kubernetes.io/secret/f449c3a6-ffbf-45f2-90b0-0ada0f1af9f3-console-oauth-config\") on node \"ip-10-0-143-17.ec2.internal\" DevicePath \"\"" Jul 10 18:43:45.665895 ip-10-0-143-17 kubenswrapper[2585]: I0710 18:43:45.665810 2585 reconciler_common.go:299] "Volume detached for volume \"service-ca\" (UniqueName: \"kubernetes.io/configmap/f449c3a6-ffbf-45f2-90b0-0ada0f1af9f3-service-ca\") on node \"ip-10-0-143-17.ec2.internal\" DevicePath \"\"" Jul 10 18:43:46.207081 ip-10-0-143-17 kubenswrapper[2585]: I0710 18:43:46.207049 2585 generic.go:356] "Generic (PLEG): container finished" podID="f449c3a6-ffbf-45f2-90b0-0ada0f1af9f3" containerID="f134fcc8fc20876b1a629ada0d0079e7b1bd7db327d3d33b6ff28f17ae8408b2" exitCode=0 Jul 10 18:43:46.207460 ip-10-0-143-17 kubenswrapper[2585]: I0710 18:43:46.207093 2585 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-console/console-5c555b848f-4jxfq" Jul 10 18:43:46.207460 ip-10-0-143-17 kubenswrapper[2585]: I0710 18:43:46.207141 2585 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-console/console-5c555b848f-4jxfq" event={"ID":"f449c3a6-ffbf-45f2-90b0-0ada0f1af9f3","Type":"ContainerDied","Data":"f134fcc8fc20876b1a629ada0d0079e7b1bd7db327d3d33b6ff28f17ae8408b2"} Jul 10 18:43:46.207460 ip-10-0-143-17 kubenswrapper[2585]: I0710 18:43:46.207189 2585 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-console/console-5c555b848f-4jxfq" event={"ID":"f449c3a6-ffbf-45f2-90b0-0ada0f1af9f3","Type":"ContainerDied","Data":"b2fdddcad409dd4d1ba98314fb9a0cfdf3f644e200d93e745d1bc0085d3d51db"} Jul 10 18:43:46.207460 ip-10-0-143-17 kubenswrapper[2585]: I0710 18:43:46.207209 2585 scope.go:117] "RemoveContainer" containerID="f134fcc8fc20876b1a629ada0d0079e7b1bd7db327d3d33b6ff28f17ae8408b2" Jul 10 18:43:46.215502 ip-10-0-143-17 kubenswrapper[2585]: I0710 18:43:46.215487 2585 scope.go:117] "RemoveContainer" containerID="f134fcc8fc20876b1a629ada0d0079e7b1bd7db327d3d33b6ff28f17ae8408b2" Jul 10 18:43:46.215764 ip-10-0-143-17 kubenswrapper[2585]: E0710 18:43:46.215747 2585 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"f134fcc8fc20876b1a629ada0d0079e7b1bd7db327d3d33b6ff28f17ae8408b2\": container with ID starting with f134fcc8fc20876b1a629ada0d0079e7b1bd7db327d3d33b6ff28f17ae8408b2 not found: ID does not exist" containerID="f134fcc8fc20876b1a629ada0d0079e7b1bd7db327d3d33b6ff28f17ae8408b2" Jul 10 18:43:46.215830 ip-10-0-143-17 kubenswrapper[2585]: I0710 18:43:46.215772 2585 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"f134fcc8fc20876b1a629ada0d0079e7b1bd7db327d3d33b6ff28f17ae8408b2"} err="failed to get container status \"f134fcc8fc20876b1a629ada0d0079e7b1bd7db327d3d33b6ff28f17ae8408b2\": rpc error: code = NotFound desc = could not find container \"f134fcc8fc20876b1a629ada0d0079e7b1bd7db327d3d33b6ff28f17ae8408b2\": container with ID starting with f134fcc8fc20876b1a629ada0d0079e7b1bd7db327d3d33b6ff28f17ae8408b2 not found: ID does not exist" Jul 10 18:43:46.230607 ip-10-0-143-17 kubenswrapper[2585]: I0710 18:43:46.230528 2585 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-console/console-5c555b848f-4jxfq"] Jul 10 18:43:46.238371 ip-10-0-143-17 kubenswrapper[2585]: I0710 18:43:46.238353 2585 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["openshift-console/console-5c555b848f-4jxfq"] Jul 10 18:43:47.217356 ip-10-0-143-17 kubenswrapper[2585]: I0710 18:43:47.217288 2585 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="f449c3a6-ffbf-45f2-90b0-0ada0f1af9f3" path="/var/lib/kubelet/pods/f449c3a6-ffbf-45f2-90b0-0ada0f1af9f3/volumes" Jul 10 18:44:46.949015 ip-10-0-143-17 kubenswrapper[2585]: I0710 18:44:46.948984 2585 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ovn-kubernetes_ovnkube-node-xg6bm_d863cfea-f2f3-4404-b62a-d41785525f3f/ovn-acl-logging/0.log" Jul 10 18:44:46.953916 ip-10-0-143-17 kubenswrapper[2585]: I0710 18:44:46.953893 2585 kubelet.go:1648] "Image garbage collection succeeded" Jul 10 18:44:46.959226 ip-10-0-143-17 kubenswrapper[2585]: I0710 18:44:46.956777 2585 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ovn-kubernetes_ovnkube-node-xg6bm_d863cfea-f2f3-4404-b62a-d41785525f3f/ovn-acl-logging/0.log" Jul 10 18:44:59.810987 ip-10-0-143-17 kubenswrapper[2585]: I0710 18:44:59.810956 2585 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-console/console-dbc48dfcb-h89sb"] Jul 10 18:44:59.812440 ip-10-0-143-17 kubenswrapper[2585]: I0710 18:44:59.811221 2585 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="f449c3a6-ffbf-45f2-90b0-0ada0f1af9f3" containerName="console" Jul 10 18:44:59.812440 ip-10-0-143-17 kubenswrapper[2585]: I0710 18:44:59.811231 2585 state_mem.go:107] "Deleted CPUSet assignment" podUID="f449c3a6-ffbf-45f2-90b0-0ada0f1af9f3" containerName="console" Jul 10 18:44:59.812440 ip-10-0-143-17 kubenswrapper[2585]: I0710 18:44:59.811287 2585 memory_manager.go:368] "RemoveStaleState removing state" podUID="f449c3a6-ffbf-45f2-90b0-0ada0f1af9f3" containerName="console" Jul 10 18:44:59.812903 ip-10-0-143-17 kubenswrapper[2585]: I0710 18:44:59.812887 2585 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-console/console-dbc48dfcb-h89sb" Jul 10 18:44:59.825553 ip-10-0-143-17 kubenswrapper[2585]: I0710 18:44:59.825521 2585 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-console/console-dbc48dfcb-h89sb"] Jul 10 18:44:59.876381 ip-10-0-143-17 kubenswrapper[2585]: I0710 18:44:59.876353 2585 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"console-serving-cert\" (UniqueName: \"kubernetes.io/secret/d6eddcf2-4883-4ab6-9acf-65ee02d662f1-console-serving-cert\") pod \"console-dbc48dfcb-h89sb\" (UID: \"d6eddcf2-4883-4ab6-9acf-65ee02d662f1\") " pod="openshift-console/console-dbc48dfcb-h89sb" Jul 10 18:44:59.876381 ip-10-0-143-17 kubenswrapper[2585]: I0710 18:44:59.876382 2585 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"console-config\" (UniqueName: \"kubernetes.io/configmap/d6eddcf2-4883-4ab6-9acf-65ee02d662f1-console-config\") pod \"console-dbc48dfcb-h89sb\" (UID: \"d6eddcf2-4883-4ab6-9acf-65ee02d662f1\") " pod="openshift-console/console-dbc48dfcb-h89sb" Jul 10 18:44:59.876616 ip-10-0-143-17 kubenswrapper[2585]: I0710 18:44:59.876406 2585 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/d6eddcf2-4883-4ab6-9acf-65ee02d662f1-trusted-ca-bundle\") pod \"console-dbc48dfcb-h89sb\" (UID: \"d6eddcf2-4883-4ab6-9acf-65ee02d662f1\") " pod="openshift-console/console-dbc48dfcb-h89sb" Jul 10 18:44:59.876616 ip-10-0-143-17 kubenswrapper[2585]: I0710 18:44:59.876463 2585 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-stq2r\" (UniqueName: \"kubernetes.io/projected/d6eddcf2-4883-4ab6-9acf-65ee02d662f1-kube-api-access-stq2r\") pod \"console-dbc48dfcb-h89sb\" (UID: \"d6eddcf2-4883-4ab6-9acf-65ee02d662f1\") " pod="openshift-console/console-dbc48dfcb-h89sb" Jul 10 18:44:59.876616 ip-10-0-143-17 kubenswrapper[2585]: I0710 18:44:59.876555 2585 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"console-oauth-config\" (UniqueName: \"kubernetes.io/secret/d6eddcf2-4883-4ab6-9acf-65ee02d662f1-console-oauth-config\") pod \"console-dbc48dfcb-h89sb\" (UID: \"d6eddcf2-4883-4ab6-9acf-65ee02d662f1\") " pod="openshift-console/console-dbc48dfcb-h89sb" Jul 10 18:44:59.876767 ip-10-0-143-17 kubenswrapper[2585]: I0710 18:44:59.876627 2585 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"service-ca\" (UniqueName: \"kubernetes.io/configmap/d6eddcf2-4883-4ab6-9acf-65ee02d662f1-service-ca\") pod \"console-dbc48dfcb-h89sb\" (UID: \"d6eddcf2-4883-4ab6-9acf-65ee02d662f1\") " pod="openshift-console/console-dbc48dfcb-h89sb" Jul 10 18:44:59.876767 ip-10-0-143-17 kubenswrapper[2585]: I0710 18:44:59.876667 2585 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"oauth-serving-cert\" (UniqueName: \"kubernetes.io/configmap/d6eddcf2-4883-4ab6-9acf-65ee02d662f1-oauth-serving-cert\") pod \"console-dbc48dfcb-h89sb\" (UID: \"d6eddcf2-4883-4ab6-9acf-65ee02d662f1\") " pod="openshift-console/console-dbc48dfcb-h89sb" Jul 10 18:44:59.977602 ip-10-0-143-17 kubenswrapper[2585]: I0710 18:44:59.977576 2585 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"service-ca\" (UniqueName: \"kubernetes.io/configmap/d6eddcf2-4883-4ab6-9acf-65ee02d662f1-service-ca\") pod \"console-dbc48dfcb-h89sb\" (UID: \"d6eddcf2-4883-4ab6-9acf-65ee02d662f1\") " pod="openshift-console/console-dbc48dfcb-h89sb" Jul 10 18:44:59.977720 ip-10-0-143-17 kubenswrapper[2585]: I0710 18:44:59.977617 2585 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"oauth-serving-cert\" (UniqueName: \"kubernetes.io/configmap/d6eddcf2-4883-4ab6-9acf-65ee02d662f1-oauth-serving-cert\") pod \"console-dbc48dfcb-h89sb\" (UID: \"d6eddcf2-4883-4ab6-9acf-65ee02d662f1\") " pod="openshift-console/console-dbc48dfcb-h89sb" Jul 10 18:44:59.977720 ip-10-0-143-17 kubenswrapper[2585]: I0710 18:44:59.977647 2585 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"console-serving-cert\" (UniqueName: \"kubernetes.io/secret/d6eddcf2-4883-4ab6-9acf-65ee02d662f1-console-serving-cert\") pod \"console-dbc48dfcb-h89sb\" (UID: \"d6eddcf2-4883-4ab6-9acf-65ee02d662f1\") " pod="openshift-console/console-dbc48dfcb-h89sb" Jul 10 18:44:59.977720 ip-10-0-143-17 kubenswrapper[2585]: I0710 18:44:59.977671 2585 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"console-config\" (UniqueName: \"kubernetes.io/configmap/d6eddcf2-4883-4ab6-9acf-65ee02d662f1-console-config\") pod \"console-dbc48dfcb-h89sb\" (UID: \"d6eddcf2-4883-4ab6-9acf-65ee02d662f1\") " pod="openshift-console/console-dbc48dfcb-h89sb" Jul 10 18:44:59.977720 ip-10-0-143-17 kubenswrapper[2585]: I0710 18:44:59.977708 2585 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/d6eddcf2-4883-4ab6-9acf-65ee02d662f1-trusted-ca-bundle\") pod \"console-dbc48dfcb-h89sb\" (UID: \"d6eddcf2-4883-4ab6-9acf-65ee02d662f1\") " pod="openshift-console/console-dbc48dfcb-h89sb" Jul 10 18:44:59.977920 ip-10-0-143-17 kubenswrapper[2585]: I0710 18:44:59.977758 2585 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-stq2r\" (UniqueName: \"kubernetes.io/projected/d6eddcf2-4883-4ab6-9acf-65ee02d662f1-kube-api-access-stq2r\") pod \"console-dbc48dfcb-h89sb\" (UID: \"d6eddcf2-4883-4ab6-9acf-65ee02d662f1\") " pod="openshift-console/console-dbc48dfcb-h89sb" Jul 10 18:44:59.977920 ip-10-0-143-17 kubenswrapper[2585]: I0710 18:44:59.977788 2585 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"console-oauth-config\" (UniqueName: \"kubernetes.io/secret/d6eddcf2-4883-4ab6-9acf-65ee02d662f1-console-oauth-config\") pod \"console-dbc48dfcb-h89sb\" (UID: \"d6eddcf2-4883-4ab6-9acf-65ee02d662f1\") " pod="openshift-console/console-dbc48dfcb-h89sb" Jul 10 18:44:59.978297 ip-10-0-143-17 kubenswrapper[2585]: I0710 18:44:59.978269 2585 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"oauth-serving-cert\" (UniqueName: \"kubernetes.io/configmap/d6eddcf2-4883-4ab6-9acf-65ee02d662f1-oauth-serving-cert\") pod \"console-dbc48dfcb-h89sb\" (UID: \"d6eddcf2-4883-4ab6-9acf-65ee02d662f1\") " pod="openshift-console/console-dbc48dfcb-h89sb" Jul 10 18:44:59.978398 ip-10-0-143-17 kubenswrapper[2585]: I0710 18:44:59.978350 2585 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"service-ca\" (UniqueName: \"kubernetes.io/configmap/d6eddcf2-4883-4ab6-9acf-65ee02d662f1-service-ca\") pod \"console-dbc48dfcb-h89sb\" (UID: \"d6eddcf2-4883-4ab6-9acf-65ee02d662f1\") " pod="openshift-console/console-dbc48dfcb-h89sb" Jul 10 18:44:59.978610 ip-10-0-143-17 kubenswrapper[2585]: I0710 18:44:59.978588 2585 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"console-config\" (UniqueName: \"kubernetes.io/configmap/d6eddcf2-4883-4ab6-9acf-65ee02d662f1-console-config\") pod \"console-dbc48dfcb-h89sb\" (UID: \"d6eddcf2-4883-4ab6-9acf-65ee02d662f1\") " pod="openshift-console/console-dbc48dfcb-h89sb" Jul 10 18:44:59.978654 ip-10-0-143-17 kubenswrapper[2585]: I0710 18:44:59.978599 2585 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/d6eddcf2-4883-4ab6-9acf-65ee02d662f1-trusted-ca-bundle\") pod \"console-dbc48dfcb-h89sb\" (UID: \"d6eddcf2-4883-4ab6-9acf-65ee02d662f1\") " pod="openshift-console/console-dbc48dfcb-h89sb" Jul 10 18:44:59.980115 ip-10-0-143-17 kubenswrapper[2585]: I0710 18:44:59.980091 2585 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"console-serving-cert\" (UniqueName: \"kubernetes.io/secret/d6eddcf2-4883-4ab6-9acf-65ee02d662f1-console-serving-cert\") pod \"console-dbc48dfcb-h89sb\" (UID: \"d6eddcf2-4883-4ab6-9acf-65ee02d662f1\") " pod="openshift-console/console-dbc48dfcb-h89sb" Jul 10 18:44:59.980206 ip-10-0-143-17 kubenswrapper[2585]: I0710 18:44:59.980097 2585 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"console-oauth-config\" (UniqueName: \"kubernetes.io/secret/d6eddcf2-4883-4ab6-9acf-65ee02d662f1-console-oauth-config\") pod \"console-dbc48dfcb-h89sb\" (UID: \"d6eddcf2-4883-4ab6-9acf-65ee02d662f1\") " pod="openshift-console/console-dbc48dfcb-h89sb" Jul 10 18:44:59.985984 ip-10-0-143-17 kubenswrapper[2585]: I0710 18:44:59.985965 2585 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-stq2r\" (UniqueName: \"kubernetes.io/projected/d6eddcf2-4883-4ab6-9acf-65ee02d662f1-kube-api-access-stq2r\") pod \"console-dbc48dfcb-h89sb\" (UID: \"d6eddcf2-4883-4ab6-9acf-65ee02d662f1\") " pod="openshift-console/console-dbc48dfcb-h89sb" Jul 10 18:45:00.121266 ip-10-0-143-17 kubenswrapper[2585]: I0710 18:45:00.121190 2585 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-console/console-dbc48dfcb-h89sb" Jul 10 18:45:00.245721 ip-10-0-143-17 kubenswrapper[2585]: I0710 18:45:00.245694 2585 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-console/console-dbc48dfcb-h89sb"] Jul 10 18:45:00.248644 ip-10-0-143-17 kubenswrapper[2585]: W0710 18:45:00.248620 2585 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podd6eddcf2_4883_4ab6_9acf_65ee02d662f1.slice/crio-68705cc94e542a1a5d17bf827dcc8ebd3144e3399ded1fd9e1820b20d29edcaa WatchSource:0}: Error finding container 68705cc94e542a1a5d17bf827dcc8ebd3144e3399ded1fd9e1820b20d29edcaa: Status 404 returned error can't find the container with id 68705cc94e542a1a5d17bf827dcc8ebd3144e3399ded1fd9e1820b20d29edcaa Jul 10 18:45:00.477266 ip-10-0-143-17 kubenswrapper[2585]: I0710 18:45:00.477237 2585 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-console/console-dbc48dfcb-h89sb" event={"ID":"d6eddcf2-4883-4ab6-9acf-65ee02d662f1","Type":"ContainerStarted","Data":"9d8d7e7d5e693ce04df323561c2c0ad9f8c68a8b4a1506bc12423ca52b6a9ce2"} Jul 10 18:45:00.477415 ip-10-0-143-17 kubenswrapper[2585]: I0710 18:45:00.477281 2585 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-console/console-dbc48dfcb-h89sb" event={"ID":"d6eddcf2-4883-4ab6-9acf-65ee02d662f1","Type":"ContainerStarted","Data":"68705cc94e542a1a5d17bf827dcc8ebd3144e3399ded1fd9e1820b20d29edcaa"} Jul 10 18:45:00.496155 ip-10-0-143-17 kubenswrapper[2585]: I0710 18:45:00.496107 2585 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-console/console-dbc48dfcb-h89sb" podStartSLOduration=1.496093022 podStartE2EDuration="1.496093022s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-10 18:44:59 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-10 18:45:00.494632122 +0000 UTC m=+313.951992073" watchObservedRunningTime="2026-07-10 18:45:00.496093022 +0000 UTC m=+313.953452984" Jul 10 18:45:10.121924 ip-10-0-143-17 kubenswrapper[2585]: I0710 18:45:10.121894 2585 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="openshift-console/console-dbc48dfcb-h89sb" Jul 10 18:45:10.122327 ip-10-0-143-17 kubenswrapper[2585]: I0710 18:45:10.121953 2585 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-console/console-dbc48dfcb-h89sb" Jul 10 18:45:10.126268 ip-10-0-143-17 kubenswrapper[2585]: I0710 18:45:10.126243 2585 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="openshift-console/console-dbc48dfcb-h89sb" Jul 10 18:45:10.518913 ip-10-0-143-17 kubenswrapper[2585]: I0710 18:45:10.518887 2585 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-console/console-dbc48dfcb-h89sb" Jul 10 18:45:10.566714 ip-10-0-143-17 kubenswrapper[2585]: I0710 18:45:10.566685 2585 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-console/console-6b797595fb-lqkgg"] Jul 10 18:45:35.589113 ip-10-0-143-17 kubenswrapper[2585]: I0710 18:45:35.589075 2585 kuberuntime_container.go:873] "Killing container with a grace period" pod="openshift-console/console-6b797595fb-lqkgg" podUID="26cdaf91-208c-4e42-bf79-b0a257e34dab" containerName="console" containerID="cri-o://3eee74e3ca90292736aaf8869cb6a2d31c56e507b8866091b76d07ec019873ed" gracePeriod=15 Jul 10 18:45:35.824029 ip-10-0-143-17 kubenswrapper[2585]: I0710 18:45:35.824005 2585 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-console/console-6b797595fb-lqkgg" Jul 10 18:45:35.944868 ip-10-0-143-17 kubenswrapper[2585]: I0710 18:45:35.944807 2585 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"console-oauth-config\" (UniqueName: \"kubernetes.io/secret/26cdaf91-208c-4e42-bf79-b0a257e34dab-console-oauth-config\") pod \"26cdaf91-208c-4e42-bf79-b0a257e34dab\" (UID: \"26cdaf91-208c-4e42-bf79-b0a257e34dab\") " Jul 10 18:45:35.944868 ip-10-0-143-17 kubenswrapper[2585]: I0710 18:45:35.944845 2585 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/26cdaf91-208c-4e42-bf79-b0a257e34dab-trusted-ca-bundle\") pod \"26cdaf91-208c-4e42-bf79-b0a257e34dab\" (UID: \"26cdaf91-208c-4e42-bf79-b0a257e34dab\") " Jul 10 18:45:35.945057 ip-10-0-143-17 kubenswrapper[2585]: I0710 18:45:35.944880 2585 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-mfllj\" (UniqueName: \"kubernetes.io/projected/26cdaf91-208c-4e42-bf79-b0a257e34dab-kube-api-access-mfllj\") pod \"26cdaf91-208c-4e42-bf79-b0a257e34dab\" (UID: \"26cdaf91-208c-4e42-bf79-b0a257e34dab\") " Jul 10 18:45:35.945057 ip-10-0-143-17 kubenswrapper[2585]: I0710 18:45:35.944918 2585 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"console-config\" (UniqueName: \"kubernetes.io/configmap/26cdaf91-208c-4e42-bf79-b0a257e34dab-console-config\") pod \"26cdaf91-208c-4e42-bf79-b0a257e34dab\" (UID: \"26cdaf91-208c-4e42-bf79-b0a257e34dab\") " Jul 10 18:45:35.945057 ip-10-0-143-17 kubenswrapper[2585]: I0710 18:45:35.944962 2585 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"oauth-serving-cert\" (UniqueName: \"kubernetes.io/configmap/26cdaf91-208c-4e42-bf79-b0a257e34dab-oauth-serving-cert\") pod \"26cdaf91-208c-4e42-bf79-b0a257e34dab\" (UID: \"26cdaf91-208c-4e42-bf79-b0a257e34dab\") " Jul 10 18:45:35.945057 ip-10-0-143-17 kubenswrapper[2585]: I0710 18:45:35.944987 2585 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"console-serving-cert\" (UniqueName: \"kubernetes.io/secret/26cdaf91-208c-4e42-bf79-b0a257e34dab-console-serving-cert\") pod \"26cdaf91-208c-4e42-bf79-b0a257e34dab\" (UID: \"26cdaf91-208c-4e42-bf79-b0a257e34dab\") " Jul 10 18:45:35.945057 ip-10-0-143-17 kubenswrapper[2585]: I0710 18:45:35.945015 2585 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"service-ca\" (UniqueName: \"kubernetes.io/configmap/26cdaf91-208c-4e42-bf79-b0a257e34dab-service-ca\") pod \"26cdaf91-208c-4e42-bf79-b0a257e34dab\" (UID: \"26cdaf91-208c-4e42-bf79-b0a257e34dab\") " Jul 10 18:45:35.945317 ip-10-0-143-17 kubenswrapper[2585]: I0710 18:45:35.945275 2585 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/26cdaf91-208c-4e42-bf79-b0a257e34dab-trusted-ca-bundle" (OuterVolumeSpecName: "trusted-ca-bundle") pod "26cdaf91-208c-4e42-bf79-b0a257e34dab" (UID: "26cdaf91-208c-4e42-bf79-b0a257e34dab"). InnerVolumeSpecName "trusted-ca-bundle". PluginName "kubernetes.io/configmap", VolumeGIDValue "" Jul 10 18:45:35.945317 ip-10-0-143-17 kubenswrapper[2585]: I0710 18:45:35.945298 2585 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/26cdaf91-208c-4e42-bf79-b0a257e34dab-console-config" (OuterVolumeSpecName: "console-config") pod "26cdaf91-208c-4e42-bf79-b0a257e34dab" (UID: "26cdaf91-208c-4e42-bf79-b0a257e34dab"). InnerVolumeSpecName "console-config". PluginName "kubernetes.io/configmap", VolumeGIDValue "" Jul 10 18:45:35.945483 ip-10-0-143-17 kubenswrapper[2585]: I0710 18:45:35.945452 2585 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/26cdaf91-208c-4e42-bf79-b0a257e34dab-oauth-serving-cert" (OuterVolumeSpecName: "oauth-serving-cert") pod "26cdaf91-208c-4e42-bf79-b0a257e34dab" (UID: "26cdaf91-208c-4e42-bf79-b0a257e34dab"). InnerVolumeSpecName "oauth-serving-cert". PluginName "kubernetes.io/configmap", VolumeGIDValue "" Jul 10 18:45:35.945638 ip-10-0-143-17 kubenswrapper[2585]: I0710 18:45:35.945501 2585 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/26cdaf91-208c-4e42-bf79-b0a257e34dab-service-ca" (OuterVolumeSpecName: "service-ca") pod "26cdaf91-208c-4e42-bf79-b0a257e34dab" (UID: "26cdaf91-208c-4e42-bf79-b0a257e34dab"). InnerVolumeSpecName "service-ca". PluginName "kubernetes.io/configmap", VolumeGIDValue "" Jul 10 18:45:35.946968 ip-10-0-143-17 kubenswrapper[2585]: I0710 18:45:35.946946 2585 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/26cdaf91-208c-4e42-bf79-b0a257e34dab-console-oauth-config" (OuterVolumeSpecName: "console-oauth-config") pod "26cdaf91-208c-4e42-bf79-b0a257e34dab" (UID: "26cdaf91-208c-4e42-bf79-b0a257e34dab"). InnerVolumeSpecName "console-oauth-config". PluginName "kubernetes.io/secret", VolumeGIDValue "" Jul 10 18:45:35.947045 ip-10-0-143-17 kubenswrapper[2585]: I0710 18:45:35.947009 2585 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/26cdaf91-208c-4e42-bf79-b0a257e34dab-console-serving-cert" (OuterVolumeSpecName: "console-serving-cert") pod "26cdaf91-208c-4e42-bf79-b0a257e34dab" (UID: "26cdaf91-208c-4e42-bf79-b0a257e34dab"). InnerVolumeSpecName "console-serving-cert". PluginName "kubernetes.io/secret", VolumeGIDValue "" Jul 10 18:45:35.947045 ip-10-0-143-17 kubenswrapper[2585]: I0710 18:45:35.947030 2585 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/26cdaf91-208c-4e42-bf79-b0a257e34dab-kube-api-access-mfllj" (OuterVolumeSpecName: "kube-api-access-mfllj") pod "26cdaf91-208c-4e42-bf79-b0a257e34dab" (UID: "26cdaf91-208c-4e42-bf79-b0a257e34dab"). InnerVolumeSpecName "kube-api-access-mfllj". PluginName "kubernetes.io/projected", VolumeGIDValue "" Jul 10 18:45:36.046504 ip-10-0-143-17 kubenswrapper[2585]: I0710 18:45:36.046483 2585 reconciler_common.go:299] "Volume detached for volume \"oauth-serving-cert\" (UniqueName: \"kubernetes.io/configmap/26cdaf91-208c-4e42-bf79-b0a257e34dab-oauth-serving-cert\") on node \"ip-10-0-143-17.ec2.internal\" DevicePath \"\"" Jul 10 18:45:36.046504 ip-10-0-143-17 kubenswrapper[2585]: I0710 18:45:36.046503 2585 reconciler_common.go:299] "Volume detached for volume \"console-serving-cert\" (UniqueName: \"kubernetes.io/secret/26cdaf91-208c-4e42-bf79-b0a257e34dab-console-serving-cert\") on node \"ip-10-0-143-17.ec2.internal\" DevicePath \"\"" Jul 10 18:45:36.046639 ip-10-0-143-17 kubenswrapper[2585]: I0710 18:45:36.046512 2585 reconciler_common.go:299] "Volume detached for volume \"service-ca\" (UniqueName: \"kubernetes.io/configmap/26cdaf91-208c-4e42-bf79-b0a257e34dab-service-ca\") on node \"ip-10-0-143-17.ec2.internal\" DevicePath \"\"" Jul 10 18:45:36.046639 ip-10-0-143-17 kubenswrapper[2585]: I0710 18:45:36.046521 2585 reconciler_common.go:299] "Volume detached for volume \"console-oauth-config\" (UniqueName: \"kubernetes.io/secret/26cdaf91-208c-4e42-bf79-b0a257e34dab-console-oauth-config\") on node \"ip-10-0-143-17.ec2.internal\" DevicePath \"\"" Jul 10 18:45:36.046639 ip-10-0-143-17 kubenswrapper[2585]: I0710 18:45:36.046529 2585 reconciler_common.go:299] "Volume detached for volume \"trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/26cdaf91-208c-4e42-bf79-b0a257e34dab-trusted-ca-bundle\") on node \"ip-10-0-143-17.ec2.internal\" DevicePath \"\"" Jul 10 18:45:36.046639 ip-10-0-143-17 kubenswrapper[2585]: I0710 18:45:36.046550 2585 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-mfllj\" (UniqueName: \"kubernetes.io/projected/26cdaf91-208c-4e42-bf79-b0a257e34dab-kube-api-access-mfllj\") on node \"ip-10-0-143-17.ec2.internal\" DevicePath \"\"" Jul 10 18:45:36.046639 ip-10-0-143-17 kubenswrapper[2585]: I0710 18:45:36.046560 2585 reconciler_common.go:299] "Volume detached for volume \"console-config\" (UniqueName: \"kubernetes.io/configmap/26cdaf91-208c-4e42-bf79-b0a257e34dab-console-config\") on node \"ip-10-0-143-17.ec2.internal\" DevicePath \"\"" Jul 10 18:45:36.614103 ip-10-0-143-17 kubenswrapper[2585]: I0710 18:45:36.614074 2585 generic.go:356] "Generic (PLEG): container finished" podID="26cdaf91-208c-4e42-bf79-b0a257e34dab" containerID="3eee74e3ca90292736aaf8869cb6a2d31c56e507b8866091b76d07ec019873ed" exitCode=0 Jul 10 18:45:36.614455 ip-10-0-143-17 kubenswrapper[2585]: I0710 18:45:36.614119 2585 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-console/console-6b797595fb-lqkgg" Jul 10 18:45:36.614455 ip-10-0-143-17 kubenswrapper[2585]: I0710 18:45:36.614157 2585 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-console/console-6b797595fb-lqkgg" event={"ID":"26cdaf91-208c-4e42-bf79-b0a257e34dab","Type":"ContainerDied","Data":"3eee74e3ca90292736aaf8869cb6a2d31c56e507b8866091b76d07ec019873ed"} Jul 10 18:45:36.614455 ip-10-0-143-17 kubenswrapper[2585]: I0710 18:45:36.614210 2585 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-console/console-6b797595fb-lqkgg" event={"ID":"26cdaf91-208c-4e42-bf79-b0a257e34dab","Type":"ContainerDied","Data":"dfd0b193e141ba6c1943fd5ddbd5d831852d67b1059803c338b7635d520102bd"} Jul 10 18:45:36.614455 ip-10-0-143-17 kubenswrapper[2585]: I0710 18:45:36.614230 2585 scope.go:117] "RemoveContainer" containerID="3eee74e3ca90292736aaf8869cb6a2d31c56e507b8866091b76d07ec019873ed" Jul 10 18:45:36.622244 ip-10-0-143-17 kubenswrapper[2585]: I0710 18:45:36.622229 2585 scope.go:117] "RemoveContainer" containerID="3eee74e3ca90292736aaf8869cb6a2d31c56e507b8866091b76d07ec019873ed" Jul 10 18:45:36.622496 ip-10-0-143-17 kubenswrapper[2585]: E0710 18:45:36.622469 2585 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"3eee74e3ca90292736aaf8869cb6a2d31c56e507b8866091b76d07ec019873ed\": container with ID starting with 3eee74e3ca90292736aaf8869cb6a2d31c56e507b8866091b76d07ec019873ed not found: ID does not exist" containerID="3eee74e3ca90292736aaf8869cb6a2d31c56e507b8866091b76d07ec019873ed" Jul 10 18:45:36.622580 ip-10-0-143-17 kubenswrapper[2585]: I0710 18:45:36.622495 2585 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"3eee74e3ca90292736aaf8869cb6a2d31c56e507b8866091b76d07ec019873ed"} err="failed to get container status \"3eee74e3ca90292736aaf8869cb6a2d31c56e507b8866091b76d07ec019873ed\": rpc error: code = NotFound desc = could not find container \"3eee74e3ca90292736aaf8869cb6a2d31c56e507b8866091b76d07ec019873ed\": container with ID starting with 3eee74e3ca90292736aaf8869cb6a2d31c56e507b8866091b76d07ec019873ed not found: ID does not exist" Jul 10 18:45:36.634962 ip-10-0-143-17 kubenswrapper[2585]: I0710 18:45:36.634938 2585 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-console/console-6b797595fb-lqkgg"] Jul 10 18:45:36.638585 ip-10-0-143-17 kubenswrapper[2585]: I0710 18:45:36.638564 2585 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["openshift-console/console-6b797595fb-lqkgg"] Jul 10 18:45:37.214704 ip-10-0-143-17 kubenswrapper[2585]: I0710 18:45:37.214672 2585 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="26cdaf91-208c-4e42-bf79-b0a257e34dab" path="/var/lib/kubelet/pods/26cdaf91-208c-4e42-bf79-b0a257e34dab/volumes" Jul 10 18:45:38.657016 ip-10-0-143-17 kubenswrapper[2585]: I0710 18:45:38.656966 2585 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-marketplace/74e94af6ec69867d65f1d90648440bdc4f0845d5e32c4818a460ebb589v8nnz"] Jul 10 18:45:38.657402 ip-10-0-143-17 kubenswrapper[2585]: I0710 18:45:38.657296 2585 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="26cdaf91-208c-4e42-bf79-b0a257e34dab" containerName="console" Jul 10 18:45:38.657402 ip-10-0-143-17 kubenswrapper[2585]: I0710 18:45:38.657309 2585 state_mem.go:107] "Deleted CPUSet assignment" podUID="26cdaf91-208c-4e42-bf79-b0a257e34dab" containerName="console" Jul 10 18:45:38.657402 ip-10-0-143-17 kubenswrapper[2585]: I0710 18:45:38.657358 2585 memory_manager.go:368] "RemoveStaleState removing state" podUID="26cdaf91-208c-4e42-bf79-b0a257e34dab" containerName="console" Jul 10 18:45:38.662093 ip-10-0-143-17 kubenswrapper[2585]: I0710 18:45:38.662055 2585 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-marketplace/74e94af6ec69867d65f1d90648440bdc4f0845d5e32c4818a460ebb589v8nnz" Jul 10 18:45:38.665697 ip-10-0-143-17 kubenswrapper[2585]: I0710 18:45:38.665671 2585 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-marketplace\"/\"default-dockercfg-lrkqb\"" Jul 10 18:45:38.665837 ip-10-0-143-17 kubenswrapper[2585]: I0710 18:45:38.665731 2585 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-marketplace\"/\"kube-root-ca.crt\"" Jul 10 18:45:38.666622 ip-10-0-143-17 kubenswrapper[2585]: I0710 18:45:38.666601 2585 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-marketplace\"/\"openshift-service-ca.crt\"" Jul 10 18:45:38.670482 ip-10-0-143-17 kubenswrapper[2585]: I0710 18:45:38.670328 2585 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-marketplace/74e94af6ec69867d65f1d90648440bdc4f0845d5e32c4818a460ebb589v8nnz"] Jul 10 18:45:38.766325 ip-10-0-143-17 kubenswrapper[2585]: I0710 18:45:38.766298 2585 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"bundle\" (UniqueName: \"kubernetes.io/empty-dir/f2e9ab6c-c203-4f25-9e1c-b3776902cc90-bundle\") pod \"74e94af6ec69867d65f1d90648440bdc4f0845d5e32c4818a460ebb589v8nnz\" (UID: \"f2e9ab6c-c203-4f25-9e1c-b3776902cc90\") " pod="openshift-marketplace/74e94af6ec69867d65f1d90648440bdc4f0845d5e32c4818a460ebb589v8nnz" Jul 10 18:45:38.766431 ip-10-0-143-17 kubenswrapper[2585]: I0710 18:45:38.766347 2585 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"util\" (UniqueName: \"kubernetes.io/empty-dir/f2e9ab6c-c203-4f25-9e1c-b3776902cc90-util\") pod \"74e94af6ec69867d65f1d90648440bdc4f0845d5e32c4818a460ebb589v8nnz\" (UID: \"f2e9ab6c-c203-4f25-9e1c-b3776902cc90\") " pod="openshift-marketplace/74e94af6ec69867d65f1d90648440bdc4f0845d5e32c4818a460ebb589v8nnz" Jul 10 18:45:38.766492 ip-10-0-143-17 kubenswrapper[2585]: I0710 18:45:38.766429 2585 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-mhbws\" (UniqueName: \"kubernetes.io/projected/f2e9ab6c-c203-4f25-9e1c-b3776902cc90-kube-api-access-mhbws\") pod \"74e94af6ec69867d65f1d90648440bdc4f0845d5e32c4818a460ebb589v8nnz\" (UID: \"f2e9ab6c-c203-4f25-9e1c-b3776902cc90\") " pod="openshift-marketplace/74e94af6ec69867d65f1d90648440bdc4f0845d5e32c4818a460ebb589v8nnz" Jul 10 18:45:38.867566 ip-10-0-143-17 kubenswrapper[2585]: I0710 18:45:38.867514 2585 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-mhbws\" (UniqueName: \"kubernetes.io/projected/f2e9ab6c-c203-4f25-9e1c-b3776902cc90-kube-api-access-mhbws\") pod \"74e94af6ec69867d65f1d90648440bdc4f0845d5e32c4818a460ebb589v8nnz\" (UID: \"f2e9ab6c-c203-4f25-9e1c-b3776902cc90\") " pod="openshift-marketplace/74e94af6ec69867d65f1d90648440bdc4f0845d5e32c4818a460ebb589v8nnz" Jul 10 18:45:38.867664 ip-10-0-143-17 kubenswrapper[2585]: I0710 18:45:38.867577 2585 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"bundle\" (UniqueName: \"kubernetes.io/empty-dir/f2e9ab6c-c203-4f25-9e1c-b3776902cc90-bundle\") pod \"74e94af6ec69867d65f1d90648440bdc4f0845d5e32c4818a460ebb589v8nnz\" (UID: \"f2e9ab6c-c203-4f25-9e1c-b3776902cc90\") " pod="openshift-marketplace/74e94af6ec69867d65f1d90648440bdc4f0845d5e32c4818a460ebb589v8nnz" Jul 10 18:45:38.867664 ip-10-0-143-17 kubenswrapper[2585]: I0710 18:45:38.867608 2585 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"util\" (UniqueName: \"kubernetes.io/empty-dir/f2e9ab6c-c203-4f25-9e1c-b3776902cc90-util\") pod \"74e94af6ec69867d65f1d90648440bdc4f0845d5e32c4818a460ebb589v8nnz\" (UID: \"f2e9ab6c-c203-4f25-9e1c-b3776902cc90\") " pod="openshift-marketplace/74e94af6ec69867d65f1d90648440bdc4f0845d5e32c4818a460ebb589v8nnz" Jul 10 18:45:38.867931 ip-10-0-143-17 kubenswrapper[2585]: I0710 18:45:38.867913 2585 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"util\" (UniqueName: \"kubernetes.io/empty-dir/f2e9ab6c-c203-4f25-9e1c-b3776902cc90-util\") pod \"74e94af6ec69867d65f1d90648440bdc4f0845d5e32c4818a460ebb589v8nnz\" (UID: \"f2e9ab6c-c203-4f25-9e1c-b3776902cc90\") " pod="openshift-marketplace/74e94af6ec69867d65f1d90648440bdc4f0845d5e32c4818a460ebb589v8nnz" Jul 10 18:45:38.867994 ip-10-0-143-17 kubenswrapper[2585]: I0710 18:45:38.867945 2585 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"bundle\" (UniqueName: \"kubernetes.io/empty-dir/f2e9ab6c-c203-4f25-9e1c-b3776902cc90-bundle\") pod \"74e94af6ec69867d65f1d90648440bdc4f0845d5e32c4818a460ebb589v8nnz\" (UID: \"f2e9ab6c-c203-4f25-9e1c-b3776902cc90\") " pod="openshift-marketplace/74e94af6ec69867d65f1d90648440bdc4f0845d5e32c4818a460ebb589v8nnz" Jul 10 18:45:38.877354 ip-10-0-143-17 kubenswrapper[2585]: I0710 18:45:38.877335 2585 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-mhbws\" (UniqueName: \"kubernetes.io/projected/f2e9ab6c-c203-4f25-9e1c-b3776902cc90-kube-api-access-mhbws\") pod \"74e94af6ec69867d65f1d90648440bdc4f0845d5e32c4818a460ebb589v8nnz\" (UID: \"f2e9ab6c-c203-4f25-9e1c-b3776902cc90\") " pod="openshift-marketplace/74e94af6ec69867d65f1d90648440bdc4f0845d5e32c4818a460ebb589v8nnz" Jul 10 18:45:38.973694 ip-10-0-143-17 kubenswrapper[2585]: I0710 18:45:38.973673 2585 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-marketplace/74e94af6ec69867d65f1d90648440bdc4f0845d5e32c4818a460ebb589v8nnz" Jul 10 18:45:39.098352 ip-10-0-143-17 kubenswrapper[2585]: I0710 18:45:39.098328 2585 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-marketplace/74e94af6ec69867d65f1d90648440bdc4f0845d5e32c4818a460ebb589v8nnz"] Jul 10 18:45:39.100700 ip-10-0-143-17 kubenswrapper[2585]: W0710 18:45:39.100658 2585 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podf2e9ab6c_c203_4f25_9e1c_b3776902cc90.slice/crio-fed9e3f60c536b6810ca19bad2f9c32f4154c2beb2e5b439e5b29e8436efe6a2 WatchSource:0}: Error finding container fed9e3f60c536b6810ca19bad2f9c32f4154c2beb2e5b439e5b29e8436efe6a2: Status 404 returned error can't find the container with id fed9e3f60c536b6810ca19bad2f9c32f4154c2beb2e5b439e5b29e8436efe6a2 Jul 10 18:45:39.102593 ip-10-0-143-17 kubenswrapper[2585]: I0710 18:45:39.102566 2585 provider.go:93] Refreshing cache for provider: *credentialprovider.defaultDockerConfigProvider Jul 10 18:45:39.626710 ip-10-0-143-17 kubenswrapper[2585]: I0710 18:45:39.626676 2585 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-marketplace/74e94af6ec69867d65f1d90648440bdc4f0845d5e32c4818a460ebb589v8nnz" event={"ID":"f2e9ab6c-c203-4f25-9e1c-b3776902cc90","Type":"ContainerStarted","Data":"fed9e3f60c536b6810ca19bad2f9c32f4154c2beb2e5b439e5b29e8436efe6a2"} Jul 10 18:45:45.652346 ip-10-0-143-17 kubenswrapper[2585]: I0710 18:45:45.652311 2585 generic.go:356] "Generic (PLEG): container finished" podID="f2e9ab6c-c203-4f25-9e1c-b3776902cc90" containerID="8857b64daca6447e325db0767fad022abaebd6c76aca2a449140f86ac0b35c11" exitCode=0 Jul 10 18:45:45.652346 ip-10-0-143-17 kubenswrapper[2585]: I0710 18:45:45.652349 2585 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-marketplace/74e94af6ec69867d65f1d90648440bdc4f0845d5e32c4818a460ebb589v8nnz" event={"ID":"f2e9ab6c-c203-4f25-9e1c-b3776902cc90","Type":"ContainerDied","Data":"8857b64daca6447e325db0767fad022abaebd6c76aca2a449140f86ac0b35c11"} Jul 10 18:45:48.666037 ip-10-0-143-17 kubenswrapper[2585]: I0710 18:45:48.665999 2585 generic.go:356] "Generic (PLEG): container finished" podID="f2e9ab6c-c203-4f25-9e1c-b3776902cc90" containerID="baed7c1936248edf0e45b73ecad172f6b46b32c2e33e767868cc9792c4ec97c7" exitCode=0 Jul 10 18:45:48.666383 ip-10-0-143-17 kubenswrapper[2585]: I0710 18:45:48.666086 2585 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-marketplace/74e94af6ec69867d65f1d90648440bdc4f0845d5e32c4818a460ebb589v8nnz" event={"ID":"f2e9ab6c-c203-4f25-9e1c-b3776902cc90","Type":"ContainerDied","Data":"baed7c1936248edf0e45b73ecad172f6b46b32c2e33e767868cc9792c4ec97c7"} Jul 10 18:45:56.698258 ip-10-0-143-17 kubenswrapper[2585]: I0710 18:45:56.698218 2585 generic.go:356] "Generic (PLEG): container finished" podID="f2e9ab6c-c203-4f25-9e1c-b3776902cc90" containerID="db9575b064a2f5658cde78fa327cf6d51534a718577e55d50695864be08ecd5f" exitCode=0 Jul 10 18:45:56.698678 ip-10-0-143-17 kubenswrapper[2585]: I0710 18:45:56.698288 2585 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-marketplace/74e94af6ec69867d65f1d90648440bdc4f0845d5e32c4818a460ebb589v8nnz" event={"ID":"f2e9ab6c-c203-4f25-9e1c-b3776902cc90","Type":"ContainerDied","Data":"db9575b064a2f5658cde78fa327cf6d51534a718577e55d50695864be08ecd5f"} Jul 10 18:45:57.866757 ip-10-0-143-17 kubenswrapper[2585]: I0710 18:45:57.866730 2585 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-marketplace/74e94af6ec69867d65f1d90648440bdc4f0845d5e32c4818a460ebb589v8nnz" Jul 10 18:45:57.927834 ip-10-0-143-17 kubenswrapper[2585]: I0710 18:45:57.927804 2585 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"bundle\" (UniqueName: \"kubernetes.io/empty-dir/f2e9ab6c-c203-4f25-9e1c-b3776902cc90-bundle\") pod \"f2e9ab6c-c203-4f25-9e1c-b3776902cc90\" (UID: \"f2e9ab6c-c203-4f25-9e1c-b3776902cc90\") " Jul 10 18:45:57.927943 ip-10-0-143-17 kubenswrapper[2585]: I0710 18:45:57.927854 2585 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"util\" (UniqueName: \"kubernetes.io/empty-dir/f2e9ab6c-c203-4f25-9e1c-b3776902cc90-util\") pod \"f2e9ab6c-c203-4f25-9e1c-b3776902cc90\" (UID: \"f2e9ab6c-c203-4f25-9e1c-b3776902cc90\") " Jul 10 18:45:57.927943 ip-10-0-143-17 kubenswrapper[2585]: I0710 18:45:57.927879 2585 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-mhbws\" (UniqueName: \"kubernetes.io/projected/f2e9ab6c-c203-4f25-9e1c-b3776902cc90-kube-api-access-mhbws\") pod \"f2e9ab6c-c203-4f25-9e1c-b3776902cc90\" (UID: \"f2e9ab6c-c203-4f25-9e1c-b3776902cc90\") " Jul 10 18:45:57.928354 ip-10-0-143-17 kubenswrapper[2585]: I0710 18:45:57.928326 2585 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/f2e9ab6c-c203-4f25-9e1c-b3776902cc90-bundle" (OuterVolumeSpecName: "bundle") pod "f2e9ab6c-c203-4f25-9e1c-b3776902cc90" (UID: "f2e9ab6c-c203-4f25-9e1c-b3776902cc90"). InnerVolumeSpecName "bundle". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Jul 10 18:45:57.929988 ip-10-0-143-17 kubenswrapper[2585]: I0710 18:45:57.929967 2585 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/f2e9ab6c-c203-4f25-9e1c-b3776902cc90-kube-api-access-mhbws" (OuterVolumeSpecName: "kube-api-access-mhbws") pod "f2e9ab6c-c203-4f25-9e1c-b3776902cc90" (UID: "f2e9ab6c-c203-4f25-9e1c-b3776902cc90"). InnerVolumeSpecName "kube-api-access-mhbws". PluginName "kubernetes.io/projected", VolumeGIDValue "" Jul 10 18:45:57.933490 ip-10-0-143-17 kubenswrapper[2585]: I0710 18:45:57.933465 2585 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/f2e9ab6c-c203-4f25-9e1c-b3776902cc90-util" (OuterVolumeSpecName: "util") pod "f2e9ab6c-c203-4f25-9e1c-b3776902cc90" (UID: "f2e9ab6c-c203-4f25-9e1c-b3776902cc90"). InnerVolumeSpecName "util". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Jul 10 18:45:58.028926 ip-10-0-143-17 kubenswrapper[2585]: I0710 18:45:58.028903 2585 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-mhbws\" (UniqueName: \"kubernetes.io/projected/f2e9ab6c-c203-4f25-9e1c-b3776902cc90-kube-api-access-mhbws\") on node \"ip-10-0-143-17.ec2.internal\" DevicePath \"\"" Jul 10 18:45:58.028926 ip-10-0-143-17 kubenswrapper[2585]: I0710 18:45:58.028924 2585 reconciler_common.go:299] "Volume detached for volume \"bundle\" (UniqueName: \"kubernetes.io/empty-dir/f2e9ab6c-c203-4f25-9e1c-b3776902cc90-bundle\") on node \"ip-10-0-143-17.ec2.internal\" DevicePath \"\"" Jul 10 18:45:58.029061 ip-10-0-143-17 kubenswrapper[2585]: I0710 18:45:58.028933 2585 reconciler_common.go:299] "Volume detached for volume \"util\" (UniqueName: \"kubernetes.io/empty-dir/f2e9ab6c-c203-4f25-9e1c-b3776902cc90-util\") on node \"ip-10-0-143-17.ec2.internal\" DevicePath \"\"" Jul 10 18:45:58.708467 ip-10-0-143-17 kubenswrapper[2585]: I0710 18:45:58.708427 2585 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-marketplace/74e94af6ec69867d65f1d90648440bdc4f0845d5e32c4818a460ebb589v8nnz" event={"ID":"f2e9ab6c-c203-4f25-9e1c-b3776902cc90","Type":"ContainerDied","Data":"fed9e3f60c536b6810ca19bad2f9c32f4154c2beb2e5b439e5b29e8436efe6a2"} Jul 10 18:45:58.708467 ip-10-0-143-17 kubenswrapper[2585]: I0710 18:45:58.708459 2585 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-marketplace/74e94af6ec69867d65f1d90648440bdc4f0845d5e32c4818a460ebb589v8nnz" Jul 10 18:45:58.708691 ip-10-0-143-17 kubenswrapper[2585]: I0710 18:45:58.708487 2585 pod_container_deletor.go:80] "Container not found in pod's containers" containerID="fed9e3f60c536b6810ca19bad2f9c32f4154c2beb2e5b439e5b29e8436efe6a2" Jul 10 18:46:01.772698 ip-10-0-143-17 kubenswrapper[2585]: I0710 18:46:01.772666 2585 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-keda/custom-metrics-autoscaler-operator-75f7c5f8f-cjsww"] Jul 10 18:46:01.773239 ip-10-0-143-17 kubenswrapper[2585]: I0710 18:46:01.773124 2585 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="f2e9ab6c-c203-4f25-9e1c-b3776902cc90" containerName="extract" Jul 10 18:46:01.773239 ip-10-0-143-17 kubenswrapper[2585]: I0710 18:46:01.773143 2585 state_mem.go:107] "Deleted CPUSet assignment" podUID="f2e9ab6c-c203-4f25-9e1c-b3776902cc90" containerName="extract" Jul 10 18:46:01.773239 ip-10-0-143-17 kubenswrapper[2585]: I0710 18:46:01.773157 2585 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="f2e9ab6c-c203-4f25-9e1c-b3776902cc90" containerName="pull" Jul 10 18:46:01.773239 ip-10-0-143-17 kubenswrapper[2585]: I0710 18:46:01.773166 2585 state_mem.go:107] "Deleted CPUSet assignment" podUID="f2e9ab6c-c203-4f25-9e1c-b3776902cc90" containerName="pull" Jul 10 18:46:01.773239 ip-10-0-143-17 kubenswrapper[2585]: I0710 18:46:01.773191 2585 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="f2e9ab6c-c203-4f25-9e1c-b3776902cc90" containerName="util" Jul 10 18:46:01.773239 ip-10-0-143-17 kubenswrapper[2585]: I0710 18:46:01.773199 2585 state_mem.go:107] "Deleted CPUSet assignment" podUID="f2e9ab6c-c203-4f25-9e1c-b3776902cc90" containerName="util" Jul 10 18:46:01.773561 ip-10-0-143-17 kubenswrapper[2585]: I0710 18:46:01.773265 2585 memory_manager.go:368] "RemoveStaleState removing state" podUID="f2e9ab6c-c203-4f25-9e1c-b3776902cc90" containerName="extract" Jul 10 18:46:01.775313 ip-10-0-143-17 kubenswrapper[2585]: I0710 18:46:01.775292 2585 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-keda/custom-metrics-autoscaler-operator-75f7c5f8f-cjsww" Jul 10 18:46:01.778857 ip-10-0-143-17 kubenswrapper[2585]: I0710 18:46:01.778833 2585 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-keda\"/\"custom-metrics-autoscaler-operator-dockercfg-k488p\"" Jul 10 18:46:01.778957 ip-10-0-143-17 kubenswrapper[2585]: I0710 18:46:01.778898 2585 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-keda\"/\"kedaorg-certs\"" Jul 10 18:46:01.778957 ip-10-0-143-17 kubenswrapper[2585]: I0710 18:46:01.778896 2585 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-keda\"/\"openshift-service-ca.crt\"" Jul 10 18:46:01.778957 ip-10-0-143-17 kubenswrapper[2585]: I0710 18:46:01.778897 2585 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-keda\"/\"kube-root-ca.crt\"" Jul 10 18:46:01.789200 ip-10-0-143-17 kubenswrapper[2585]: I0710 18:46:01.789172 2585 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-keda/custom-metrics-autoscaler-operator-75f7c5f8f-cjsww"] Jul 10 18:46:01.860011 ip-10-0-143-17 kubenswrapper[2585]: I0710 18:46:01.859988 2585 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-m5lhl\" (UniqueName: \"kubernetes.io/projected/0c6a02de-6d97-4de1-9659-cec5c5eb56a6-kube-api-access-m5lhl\") pod \"custom-metrics-autoscaler-operator-75f7c5f8f-cjsww\" (UID: \"0c6a02de-6d97-4de1-9659-cec5c5eb56a6\") " pod="openshift-keda/custom-metrics-autoscaler-operator-75f7c5f8f-cjsww" Jul 10 18:46:01.860125 ip-10-0-143-17 kubenswrapper[2585]: I0710 18:46:01.860044 2585 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"certificates\" (UniqueName: \"kubernetes.io/secret/0c6a02de-6d97-4de1-9659-cec5c5eb56a6-certificates\") pod \"custom-metrics-autoscaler-operator-75f7c5f8f-cjsww\" (UID: \"0c6a02de-6d97-4de1-9659-cec5c5eb56a6\") " pod="openshift-keda/custom-metrics-autoscaler-operator-75f7c5f8f-cjsww" Jul 10 18:46:01.961257 ip-10-0-143-17 kubenswrapper[2585]: I0710 18:46:01.961219 2585 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-m5lhl\" (UniqueName: \"kubernetes.io/projected/0c6a02de-6d97-4de1-9659-cec5c5eb56a6-kube-api-access-m5lhl\") pod \"custom-metrics-autoscaler-operator-75f7c5f8f-cjsww\" (UID: \"0c6a02de-6d97-4de1-9659-cec5c5eb56a6\") " pod="openshift-keda/custom-metrics-autoscaler-operator-75f7c5f8f-cjsww" Jul 10 18:46:01.961353 ip-10-0-143-17 kubenswrapper[2585]: I0710 18:46:01.961294 2585 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"certificates\" (UniqueName: \"kubernetes.io/secret/0c6a02de-6d97-4de1-9659-cec5c5eb56a6-certificates\") pod \"custom-metrics-autoscaler-operator-75f7c5f8f-cjsww\" (UID: \"0c6a02de-6d97-4de1-9659-cec5c5eb56a6\") " pod="openshift-keda/custom-metrics-autoscaler-operator-75f7c5f8f-cjsww" Jul 10 18:46:01.963527 ip-10-0-143-17 kubenswrapper[2585]: I0710 18:46:01.963505 2585 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"certificates\" (UniqueName: \"kubernetes.io/secret/0c6a02de-6d97-4de1-9659-cec5c5eb56a6-certificates\") pod \"custom-metrics-autoscaler-operator-75f7c5f8f-cjsww\" (UID: \"0c6a02de-6d97-4de1-9659-cec5c5eb56a6\") " pod="openshift-keda/custom-metrics-autoscaler-operator-75f7c5f8f-cjsww" Jul 10 18:46:01.973490 ip-10-0-143-17 kubenswrapper[2585]: I0710 18:46:01.973467 2585 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-m5lhl\" (UniqueName: \"kubernetes.io/projected/0c6a02de-6d97-4de1-9659-cec5c5eb56a6-kube-api-access-m5lhl\") pod \"custom-metrics-autoscaler-operator-75f7c5f8f-cjsww\" (UID: \"0c6a02de-6d97-4de1-9659-cec5c5eb56a6\") " pod="openshift-keda/custom-metrics-autoscaler-operator-75f7c5f8f-cjsww" Jul 10 18:46:02.085175 ip-10-0-143-17 kubenswrapper[2585]: I0710 18:46:02.085121 2585 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-keda/custom-metrics-autoscaler-operator-75f7c5f8f-cjsww" Jul 10 18:46:02.216737 ip-10-0-143-17 kubenswrapper[2585]: I0710 18:46:02.216712 2585 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-keda/custom-metrics-autoscaler-operator-75f7c5f8f-cjsww"] Jul 10 18:46:02.219773 ip-10-0-143-17 kubenswrapper[2585]: W0710 18:46:02.219748 2585 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod0c6a02de_6d97_4de1_9659_cec5c5eb56a6.slice/crio-c7b797c65a2f4ecf3a4f05399a3f3ed6d92414c16961a8efc7df9e8827676a64 WatchSource:0}: Error finding container c7b797c65a2f4ecf3a4f05399a3f3ed6d92414c16961a8efc7df9e8827676a64: Status 404 returned error can't find the container with id c7b797c65a2f4ecf3a4f05399a3f3ed6d92414c16961a8efc7df9e8827676a64 Jul 10 18:46:02.725328 ip-10-0-143-17 kubenswrapper[2585]: I0710 18:46:02.725294 2585 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-keda/custom-metrics-autoscaler-operator-75f7c5f8f-cjsww" event={"ID":"0c6a02de-6d97-4de1-9659-cec5c5eb56a6","Type":"ContainerStarted","Data":"c7b797c65a2f4ecf3a4f05399a3f3ed6d92414c16961a8efc7df9e8827676a64"} Jul 10 18:46:05.740852 ip-10-0-143-17 kubenswrapper[2585]: I0710 18:46:05.740810 2585 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-keda/custom-metrics-autoscaler-operator-75f7c5f8f-cjsww" event={"ID":"0c6a02de-6d97-4de1-9659-cec5c5eb56a6","Type":"ContainerStarted","Data":"5a9aadfb81e132832126c9d7b2bf4611d9c2f61ce478e1a53ebc649259336f79"} Jul 10 18:46:05.741237 ip-10-0-143-17 kubenswrapper[2585]: I0710 18:46:05.740956 2585 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-keda/custom-metrics-autoscaler-operator-75f7c5f8f-cjsww" Jul 10 18:46:05.761856 ip-10-0-143-17 kubenswrapper[2585]: I0710 18:46:05.761809 2585 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-keda/custom-metrics-autoscaler-operator-75f7c5f8f-cjsww" podStartSLOduration=1.915826158 podStartE2EDuration="4.761796067s" totalImagesPullingTime="2.845969909s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-10 18:46:01 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-10 18:46:05.75996021 +0000 UTC m=+379.217320145" watchObservedRunningTime="2026-07-10 18:46:05.761796067 +0000 UTC m=+379.219156003" Jul 10 18:46:05.854860 ip-10-0-143-17 kubenswrapper[2585]: I0710 18:46:05.854828 2585 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-keda/keda-operator-68bff59c-kxk6f"] Jul 10 18:46:05.857450 ip-10-0-143-17 kubenswrapper[2585]: I0710 18:46:05.857429 2585 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-keda/keda-operator-68bff59c-kxk6f" Jul 10 18:46:05.860277 ip-10-0-143-17 kubenswrapper[2585]: I0710 18:46:05.860257 2585 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-keda\"/\"keda-operator-certs\"" Jul 10 18:46:05.860433 ip-10-0-143-17 kubenswrapper[2585]: I0710 18:46:05.860416 2585 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-keda\"/\"keda-ocp-cabundle\"" Jul 10 18:46:05.860491 ip-10-0-143-17 kubenswrapper[2585]: I0710 18:46:05.860444 2585 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-keda\"/\"keda-operator-dockercfg-7j7kt\"" Jul 10 18:46:05.866949 ip-10-0-143-17 kubenswrapper[2585]: I0710 18:46:05.866918 2585 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-keda/keda-operator-68bff59c-kxk6f"] Jul 10 18:46:05.994430 ip-10-0-143-17 kubenswrapper[2585]: I0710 18:46:05.994358 2585 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-blqts\" (UniqueName: \"kubernetes.io/projected/6e934b04-0f7a-48c7-bbb5-bb510edb7022-kube-api-access-blqts\") pod \"keda-operator-68bff59c-kxk6f\" (UID: \"6e934b04-0f7a-48c7-bbb5-bb510edb7022\") " pod="openshift-keda/keda-operator-68bff59c-kxk6f" Jul 10 18:46:05.994430 ip-10-0-143-17 kubenswrapper[2585]: I0710 18:46:05.994390 2585 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"certificates\" (UniqueName: \"kubernetes.io/projected/6e934b04-0f7a-48c7-bbb5-bb510edb7022-certificates\") pod \"keda-operator-68bff59c-kxk6f\" (UID: \"6e934b04-0f7a-48c7-bbb5-bb510edb7022\") " pod="openshift-keda/keda-operator-68bff59c-kxk6f" Jul 10 18:46:05.994430 ip-10-0-143-17 kubenswrapper[2585]: I0710 18:46:05.994407 2585 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"cabundle0\" (UniqueName: \"kubernetes.io/configmap/6e934b04-0f7a-48c7-bbb5-bb510edb7022-cabundle0\") pod \"keda-operator-68bff59c-kxk6f\" (UID: \"6e934b04-0f7a-48c7-bbb5-bb510edb7022\") " pod="openshift-keda/keda-operator-68bff59c-kxk6f" Jul 10 18:46:06.095702 ip-10-0-143-17 kubenswrapper[2585]: I0710 18:46:06.095660 2585 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-blqts\" (UniqueName: \"kubernetes.io/projected/6e934b04-0f7a-48c7-bbb5-bb510edb7022-kube-api-access-blqts\") pod \"keda-operator-68bff59c-kxk6f\" (UID: \"6e934b04-0f7a-48c7-bbb5-bb510edb7022\") " pod="openshift-keda/keda-operator-68bff59c-kxk6f" Jul 10 18:46:06.095702 ip-10-0-143-17 kubenswrapper[2585]: I0710 18:46:06.095702 2585 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"certificates\" (UniqueName: \"kubernetes.io/projected/6e934b04-0f7a-48c7-bbb5-bb510edb7022-certificates\") pod \"keda-operator-68bff59c-kxk6f\" (UID: \"6e934b04-0f7a-48c7-bbb5-bb510edb7022\") " pod="openshift-keda/keda-operator-68bff59c-kxk6f" Jul 10 18:46:06.095899 ip-10-0-143-17 kubenswrapper[2585]: I0710 18:46:06.095720 2585 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"cabundle0\" (UniqueName: \"kubernetes.io/configmap/6e934b04-0f7a-48c7-bbb5-bb510edb7022-cabundle0\") pod \"keda-operator-68bff59c-kxk6f\" (UID: \"6e934b04-0f7a-48c7-bbb5-bb510edb7022\") " pod="openshift-keda/keda-operator-68bff59c-kxk6f" Jul 10 18:46:06.095899 ip-10-0-143-17 kubenswrapper[2585]: E0710 18:46:06.095829 2585 secret.go:281] references non-existent secret key: ca.crt Jul 10 18:46:06.095899 ip-10-0-143-17 kubenswrapper[2585]: E0710 18:46:06.095852 2585 projected.go:279] Couldn't get secret payload openshift-keda/kedaorg-certs: references non-existent secret key: ca.crt Jul 10 18:46:06.095899 ip-10-0-143-17 kubenswrapper[2585]: E0710 18:46:06.095861 2585 projected.go:196] Error preparing data for projected volume certificates for pod openshift-keda/keda-operator-68bff59c-kxk6f: references non-existent secret key: ca.crt Jul 10 18:46:06.096018 ip-10-0-143-17 kubenswrapper[2585]: E0710 18:46:06.095912 2585 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/projected/6e934b04-0f7a-48c7-bbb5-bb510edb7022-certificates podName:6e934b04-0f7a-48c7-bbb5-bb510edb7022 nodeName:}" failed. No retries permitted until 2026-07-10 18:46:06.59589757 +0000 UTC m=+380.053257487 (durationBeforeRetry 500ms). Error: MountVolume.SetUp failed for volume "certificates" (UniqueName: "kubernetes.io/projected/6e934b04-0f7a-48c7-bbb5-bb510edb7022-certificates") pod "keda-operator-68bff59c-kxk6f" (UID: "6e934b04-0f7a-48c7-bbb5-bb510edb7022") : references non-existent secret key: ca.crt Jul 10 18:46:06.096325 ip-10-0-143-17 kubenswrapper[2585]: I0710 18:46:06.096308 2585 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"cabundle0\" (UniqueName: \"kubernetes.io/configmap/6e934b04-0f7a-48c7-bbb5-bb510edb7022-cabundle0\") pod \"keda-operator-68bff59c-kxk6f\" (UID: \"6e934b04-0f7a-48c7-bbb5-bb510edb7022\") " pod="openshift-keda/keda-operator-68bff59c-kxk6f" Jul 10 18:46:06.105417 ip-10-0-143-17 kubenswrapper[2585]: I0710 18:46:06.105395 2585 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-blqts\" (UniqueName: \"kubernetes.io/projected/6e934b04-0f7a-48c7-bbb5-bb510edb7022-kube-api-access-blqts\") pod \"keda-operator-68bff59c-kxk6f\" (UID: \"6e934b04-0f7a-48c7-bbb5-bb510edb7022\") " pod="openshift-keda/keda-operator-68bff59c-kxk6f" Jul 10 18:46:06.106443 ip-10-0-143-17 kubenswrapper[2585]: I0710 18:46:06.106421 2585 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-keda/keda-metrics-apiserver-559f75f947-457f8"] Jul 10 18:46:06.108799 ip-10-0-143-17 kubenswrapper[2585]: I0710 18:46:06.108782 2585 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-keda/keda-metrics-apiserver-559f75f947-457f8" Jul 10 18:46:06.111380 ip-10-0-143-17 kubenswrapper[2585]: I0710 18:46:06.111362 2585 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-keda\"/\"keda-metrics-apiserver-certs\"" Jul 10 18:46:06.121044 ip-10-0-143-17 kubenswrapper[2585]: I0710 18:46:06.121024 2585 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-keda/keda-metrics-apiserver-559f75f947-457f8"] Jul 10 18:46:06.196500 ip-10-0-143-17 kubenswrapper[2585]: I0710 18:46:06.196476 2585 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"certificates\" (UniqueName: \"kubernetes.io/projected/7674cb7f-b212-4731-bcd2-e401857a86f1-certificates\") pod \"keda-metrics-apiserver-559f75f947-457f8\" (UID: \"7674cb7f-b212-4731-bcd2-e401857a86f1\") " pod="openshift-keda/keda-metrics-apiserver-559f75f947-457f8" Jul 10 18:46:06.196659 ip-10-0-143-17 kubenswrapper[2585]: I0710 18:46:06.196505 2585 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-9cvqs\" (UniqueName: \"kubernetes.io/projected/7674cb7f-b212-4731-bcd2-e401857a86f1-kube-api-access-9cvqs\") pod \"keda-metrics-apiserver-559f75f947-457f8\" (UID: \"7674cb7f-b212-4731-bcd2-e401857a86f1\") " pod="openshift-keda/keda-metrics-apiserver-559f75f947-457f8" Jul 10 18:46:06.196659 ip-10-0-143-17 kubenswrapper[2585]: I0710 18:46:06.196555 2585 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"temp-vol\" (UniqueName: \"kubernetes.io/empty-dir/7674cb7f-b212-4731-bcd2-e401857a86f1-temp-vol\") pod \"keda-metrics-apiserver-559f75f947-457f8\" (UID: \"7674cb7f-b212-4731-bcd2-e401857a86f1\") " pod="openshift-keda/keda-metrics-apiserver-559f75f947-457f8" Jul 10 18:46:06.297948 ip-10-0-143-17 kubenswrapper[2585]: I0710 18:46:06.297870 2585 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"certificates\" (UniqueName: \"kubernetes.io/projected/7674cb7f-b212-4731-bcd2-e401857a86f1-certificates\") pod \"keda-metrics-apiserver-559f75f947-457f8\" (UID: \"7674cb7f-b212-4731-bcd2-e401857a86f1\") " pod="openshift-keda/keda-metrics-apiserver-559f75f947-457f8" Jul 10 18:46:06.297948 ip-10-0-143-17 kubenswrapper[2585]: I0710 18:46:06.297902 2585 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-9cvqs\" (UniqueName: \"kubernetes.io/projected/7674cb7f-b212-4731-bcd2-e401857a86f1-kube-api-access-9cvqs\") pod \"keda-metrics-apiserver-559f75f947-457f8\" (UID: \"7674cb7f-b212-4731-bcd2-e401857a86f1\") " pod="openshift-keda/keda-metrics-apiserver-559f75f947-457f8" Jul 10 18:46:06.298114 ip-10-0-143-17 kubenswrapper[2585]: I0710 18:46:06.298007 2585 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"temp-vol\" (UniqueName: \"kubernetes.io/empty-dir/7674cb7f-b212-4731-bcd2-e401857a86f1-temp-vol\") pod \"keda-metrics-apiserver-559f75f947-457f8\" (UID: \"7674cb7f-b212-4731-bcd2-e401857a86f1\") " pod="openshift-keda/keda-metrics-apiserver-559f75f947-457f8" Jul 10 18:46:06.298409 ip-10-0-143-17 kubenswrapper[2585]: I0710 18:46:06.298387 2585 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"temp-vol\" (UniqueName: \"kubernetes.io/empty-dir/7674cb7f-b212-4731-bcd2-e401857a86f1-temp-vol\") pod \"keda-metrics-apiserver-559f75f947-457f8\" (UID: \"7674cb7f-b212-4731-bcd2-e401857a86f1\") " pod="openshift-keda/keda-metrics-apiserver-559f75f947-457f8" Jul 10 18:46:06.300370 ip-10-0-143-17 kubenswrapper[2585]: I0710 18:46:06.300349 2585 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"certificates\" (UniqueName: \"kubernetes.io/projected/7674cb7f-b212-4731-bcd2-e401857a86f1-certificates\") pod \"keda-metrics-apiserver-559f75f947-457f8\" (UID: \"7674cb7f-b212-4731-bcd2-e401857a86f1\") " pod="openshift-keda/keda-metrics-apiserver-559f75f947-457f8" Jul 10 18:46:06.306312 ip-10-0-143-17 kubenswrapper[2585]: I0710 18:46:06.306292 2585 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-9cvqs\" (UniqueName: \"kubernetes.io/projected/7674cb7f-b212-4731-bcd2-e401857a86f1-kube-api-access-9cvqs\") pod \"keda-metrics-apiserver-559f75f947-457f8\" (UID: \"7674cb7f-b212-4731-bcd2-e401857a86f1\") " pod="openshift-keda/keda-metrics-apiserver-559f75f947-457f8" Jul 10 18:46:06.421188 ip-10-0-143-17 kubenswrapper[2585]: I0710 18:46:06.421162 2585 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-keda/keda-metrics-apiserver-559f75f947-457f8" Jul 10 18:46:06.429243 ip-10-0-143-17 kubenswrapper[2585]: I0710 18:46:06.429219 2585 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-keda/keda-admission-5fbd5c4644-6ntk8"] Jul 10 18:46:06.431322 ip-10-0-143-17 kubenswrapper[2585]: I0710 18:46:06.431305 2585 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-keda/keda-admission-5fbd5c4644-6ntk8" Jul 10 18:46:06.434135 ip-10-0-143-17 kubenswrapper[2585]: I0710 18:46:06.434107 2585 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-keda\"/\"keda-admission-webhooks-certs\"" Jul 10 18:46:06.442500 ip-10-0-143-17 kubenswrapper[2585]: I0710 18:46:06.442475 2585 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-keda/keda-admission-5fbd5c4644-6ntk8"] Jul 10 18:46:06.501097 ip-10-0-143-17 kubenswrapper[2585]: I0710 18:46:06.500987 2585 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-wjzhq\" (UniqueName: \"kubernetes.io/projected/1d8c39af-8ae7-4946-984f-920cafa3922e-kube-api-access-wjzhq\") pod \"keda-admission-5fbd5c4644-6ntk8\" (UID: \"1d8c39af-8ae7-4946-984f-920cafa3922e\") " pod="openshift-keda/keda-admission-5fbd5c4644-6ntk8" Jul 10 18:46:06.501097 ip-10-0-143-17 kubenswrapper[2585]: I0710 18:46:06.501044 2585 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"certificates\" (UniqueName: \"kubernetes.io/projected/1d8c39af-8ae7-4946-984f-920cafa3922e-certificates\") pod \"keda-admission-5fbd5c4644-6ntk8\" (UID: \"1d8c39af-8ae7-4946-984f-920cafa3922e\") " pod="openshift-keda/keda-admission-5fbd5c4644-6ntk8" Jul 10 18:46:06.565598 ip-10-0-143-17 kubenswrapper[2585]: I0710 18:46:06.565573 2585 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-keda/keda-metrics-apiserver-559f75f947-457f8"] Jul 10 18:46:06.602161 ip-10-0-143-17 kubenswrapper[2585]: I0710 18:46:06.602137 2585 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-wjzhq\" (UniqueName: \"kubernetes.io/projected/1d8c39af-8ae7-4946-984f-920cafa3922e-kube-api-access-wjzhq\") pod \"keda-admission-5fbd5c4644-6ntk8\" (UID: \"1d8c39af-8ae7-4946-984f-920cafa3922e\") " pod="openshift-keda/keda-admission-5fbd5c4644-6ntk8" Jul 10 18:46:06.602301 ip-10-0-143-17 kubenswrapper[2585]: I0710 18:46:06.602174 2585 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"certificates\" (UniqueName: \"kubernetes.io/projected/1d8c39af-8ae7-4946-984f-920cafa3922e-certificates\") pod \"keda-admission-5fbd5c4644-6ntk8\" (UID: \"1d8c39af-8ae7-4946-984f-920cafa3922e\") " pod="openshift-keda/keda-admission-5fbd5c4644-6ntk8" Jul 10 18:46:06.602372 ip-10-0-143-17 kubenswrapper[2585]: I0710 18:46:06.602353 2585 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"certificates\" (UniqueName: \"kubernetes.io/projected/6e934b04-0f7a-48c7-bbb5-bb510edb7022-certificates\") pod \"keda-operator-68bff59c-kxk6f\" (UID: \"6e934b04-0f7a-48c7-bbb5-bb510edb7022\") " pod="openshift-keda/keda-operator-68bff59c-kxk6f" Jul 10 18:46:06.602519 ip-10-0-143-17 kubenswrapper[2585]: E0710 18:46:06.602504 2585 projected.go:266] Couldn't get secret openshift-keda/keda-admission-webhooks-certs: secret "keda-admission-webhooks-certs" not found Jul 10 18:46:06.602607 ip-10-0-143-17 kubenswrapper[2585]: E0710 18:46:06.602529 2585 projected.go:196] Error preparing data for projected volume certificates for pod openshift-keda/keda-admission-5fbd5c4644-6ntk8: secret "keda-admission-webhooks-certs" not found Jul 10 18:46:06.602647 ip-10-0-143-17 kubenswrapper[2585]: E0710 18:46:06.602609 2585 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/projected/1d8c39af-8ae7-4946-984f-920cafa3922e-certificates podName:1d8c39af-8ae7-4946-984f-920cafa3922e nodeName:}" failed. No retries permitted until 2026-07-10 18:46:07.102590342 +0000 UTC m=+380.559950255 (durationBeforeRetry 500ms). Error: MountVolume.SetUp failed for volume "certificates" (UniqueName: "kubernetes.io/projected/1d8c39af-8ae7-4946-984f-920cafa3922e-certificates") pod "keda-admission-5fbd5c4644-6ntk8" (UID: "1d8c39af-8ae7-4946-984f-920cafa3922e") : secret "keda-admission-webhooks-certs" not found Jul 10 18:46:06.604848 ip-10-0-143-17 kubenswrapper[2585]: I0710 18:46:06.604817 2585 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"certificates\" (UniqueName: \"kubernetes.io/projected/6e934b04-0f7a-48c7-bbb5-bb510edb7022-certificates\") pod \"keda-operator-68bff59c-kxk6f\" (UID: \"6e934b04-0f7a-48c7-bbb5-bb510edb7022\") " pod="openshift-keda/keda-operator-68bff59c-kxk6f" Jul 10 18:46:06.617528 ip-10-0-143-17 kubenswrapper[2585]: I0710 18:46:06.615904 2585 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-wjzhq\" (UniqueName: \"kubernetes.io/projected/1d8c39af-8ae7-4946-984f-920cafa3922e-kube-api-access-wjzhq\") pod \"keda-admission-5fbd5c4644-6ntk8\" (UID: \"1d8c39af-8ae7-4946-984f-920cafa3922e\") " pod="openshift-keda/keda-admission-5fbd5c4644-6ntk8" Jul 10 18:46:06.746885 ip-10-0-143-17 kubenswrapper[2585]: I0710 18:46:06.746852 2585 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-keda/keda-metrics-apiserver-559f75f947-457f8" event={"ID":"7674cb7f-b212-4731-bcd2-e401857a86f1","Type":"ContainerStarted","Data":"431ab3f51ba697f9638d575e8812e8fd629e93d8d4a06d46fbae2215241323ef"} Jul 10 18:46:06.768575 ip-10-0-143-17 kubenswrapper[2585]: I0710 18:46:06.768531 2585 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-keda/keda-operator-68bff59c-kxk6f" Jul 10 18:46:06.910767 ip-10-0-143-17 kubenswrapper[2585]: I0710 18:46:06.910655 2585 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-keda/keda-operator-68bff59c-kxk6f"] Jul 10 18:46:06.912917 ip-10-0-143-17 kubenswrapper[2585]: W0710 18:46:06.912886 2585 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod6e934b04_0f7a_48c7_bbb5_bb510edb7022.slice/crio-a5e36fbf4183538adf0abfd8cc449c2374d0824b3e1698523df6ea047f782751 WatchSource:0}: Error finding container a5e36fbf4183538adf0abfd8cc449c2374d0824b3e1698523df6ea047f782751: Status 404 returned error can't find the container with id a5e36fbf4183538adf0abfd8cc449c2374d0824b3e1698523df6ea047f782751 Jul 10 18:46:07.107230 ip-10-0-143-17 kubenswrapper[2585]: I0710 18:46:07.107140 2585 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"certificates\" (UniqueName: \"kubernetes.io/projected/1d8c39af-8ae7-4946-984f-920cafa3922e-certificates\") pod \"keda-admission-5fbd5c4644-6ntk8\" (UID: \"1d8c39af-8ae7-4946-984f-920cafa3922e\") " pod="openshift-keda/keda-admission-5fbd5c4644-6ntk8" Jul 10 18:46:07.110274 ip-10-0-143-17 kubenswrapper[2585]: I0710 18:46:07.110217 2585 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"certificates\" (UniqueName: \"kubernetes.io/projected/1d8c39af-8ae7-4946-984f-920cafa3922e-certificates\") pod \"keda-admission-5fbd5c4644-6ntk8\" (UID: \"1d8c39af-8ae7-4946-984f-920cafa3922e\") " pod="openshift-keda/keda-admission-5fbd5c4644-6ntk8" Jul 10 18:46:07.351011 ip-10-0-143-17 kubenswrapper[2585]: I0710 18:46:07.350982 2585 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-keda/keda-admission-5fbd5c4644-6ntk8" Jul 10 18:46:07.509042 ip-10-0-143-17 kubenswrapper[2585]: I0710 18:46:07.509007 2585 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-keda/keda-admission-5fbd5c4644-6ntk8"] Jul 10 18:46:07.513626 ip-10-0-143-17 kubenswrapper[2585]: W0710 18:46:07.513556 2585 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod1d8c39af_8ae7_4946_984f_920cafa3922e.slice/crio-0313ab6bb0bdcdd25824648877110d7a723b0ed4b64d935b24786ce4d0831d56 WatchSource:0}: Error finding container 0313ab6bb0bdcdd25824648877110d7a723b0ed4b64d935b24786ce4d0831d56: Status 404 returned error can't find the container with id 0313ab6bb0bdcdd25824648877110d7a723b0ed4b64d935b24786ce4d0831d56 Jul 10 18:46:07.752953 ip-10-0-143-17 kubenswrapper[2585]: I0710 18:46:07.752914 2585 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-keda/keda-admission-5fbd5c4644-6ntk8" event={"ID":"1d8c39af-8ae7-4946-984f-920cafa3922e","Type":"ContainerStarted","Data":"0313ab6bb0bdcdd25824648877110d7a723b0ed4b64d935b24786ce4d0831d56"} Jul 10 18:46:07.754287 ip-10-0-143-17 kubenswrapper[2585]: I0710 18:46:07.754256 2585 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-keda/keda-operator-68bff59c-kxk6f" event={"ID":"6e934b04-0f7a-48c7-bbb5-bb510edb7022","Type":"ContainerStarted","Data":"a5e36fbf4183538adf0abfd8cc449c2374d0824b3e1698523df6ea047f782751"} Jul 10 18:46:10.770653 ip-10-0-143-17 kubenswrapper[2585]: I0710 18:46:10.770617 2585 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-keda/keda-metrics-apiserver-559f75f947-457f8" event={"ID":"7674cb7f-b212-4731-bcd2-e401857a86f1","Type":"ContainerStarted","Data":"06e230c9b6e5e8ea540d1b0eb4bc7b72da1b9413d3d5172409559dbfb681fd67"} Jul 10 18:46:10.771081 ip-10-0-143-17 kubenswrapper[2585]: I0710 18:46:10.770764 2585 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-keda/keda-metrics-apiserver-559f75f947-457f8" Jul 10 18:46:10.772083 ip-10-0-143-17 kubenswrapper[2585]: I0710 18:46:10.772062 2585 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-keda/keda-admission-5fbd5c4644-6ntk8" event={"ID":"1d8c39af-8ae7-4946-984f-920cafa3922e","Type":"ContainerStarted","Data":"45e4200c9a937b820633d5eb0f8be675903a7914df90f21d92adb35016c372ba"} Jul 10 18:46:10.772187 ip-10-0-143-17 kubenswrapper[2585]: I0710 18:46:10.772122 2585 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-keda/keda-admission-5fbd5c4644-6ntk8" Jul 10 18:46:10.773364 ip-10-0-143-17 kubenswrapper[2585]: I0710 18:46:10.773345 2585 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-keda/keda-operator-68bff59c-kxk6f" event={"ID":"6e934b04-0f7a-48c7-bbb5-bb510edb7022","Type":"ContainerStarted","Data":"57f5dd8c0d592d42527bbdf533b1dabee49dce23f727a1206e5a2c6f29b1ec01"} Jul 10 18:46:10.773476 ip-10-0-143-17 kubenswrapper[2585]: I0710 18:46:10.773463 2585 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-keda/keda-operator-68bff59c-kxk6f" Jul 10 18:46:10.789710 ip-10-0-143-17 kubenswrapper[2585]: I0710 18:46:10.789674 2585 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-keda/keda-metrics-apiserver-559f75f947-457f8" podStartSLOduration=1.419354298 podStartE2EDuration="4.78966237s" totalImagesPullingTime="3.370308072s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-10 18:46:06 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-10 18:46:10.789053731 +0000 UTC m=+384.246413692" watchObservedRunningTime="2026-07-10 18:46:10.78966237 +0000 UTC m=+384.247022305" Jul 10 18:46:10.807092 ip-10-0-143-17 kubenswrapper[2585]: I0710 18:46:10.807050 2585 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-keda/keda-operator-68bff59c-kxk6f" podStartSLOduration=2.776232413 podStartE2EDuration="5.807038541s" totalImagesPullingTime="3.030806128s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-10 18:46:05 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-10 18:46:10.805972276 +0000 UTC m=+384.263332222" watchObservedRunningTime="2026-07-10 18:46:10.807038541 +0000 UTC m=+384.264398475" Jul 10 18:46:10.823658 ip-10-0-143-17 kubenswrapper[2585]: I0710 18:46:10.823619 2585 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-keda/keda-admission-5fbd5c4644-6ntk8" podStartSLOduration=1.697285508 podStartE2EDuration="4.823607317s" totalImagesPullingTime="3.126321809s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-10 18:46:06 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-10 18:46:10.822844675 +0000 UTC m=+384.280204610" watchObservedRunningTime="2026-07-10 18:46:10.823607317 +0000 UTC m=+384.280967251" Jul 10 18:46:21.782036 ip-10-0-143-17 kubenswrapper[2585]: I0710 18:46:21.782002 2585 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-keda/keda-metrics-apiserver-559f75f947-457f8" Jul 10 18:46:26.749314 ip-10-0-143-17 kubenswrapper[2585]: I0710 18:46:26.749286 2585 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-keda/custom-metrics-autoscaler-operator-75f7c5f8f-cjsww" Jul 10 18:46:31.779719 ip-10-0-143-17 kubenswrapper[2585]: I0710 18:46:31.779683 2585 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-keda/keda-operator-68bff59c-kxk6f" Jul 10 18:46:31.780198 ip-10-0-143-17 kubenswrapper[2585]: I0710 18:46:31.779782 2585 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-keda/keda-admission-5fbd5c4644-6ntk8" Jul 10 18:46:59.508212 ip-10-0-143-17 kubenswrapper[2585]: I0710 18:46:59.508174 2585 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-marketplace/685f1ed643dcce69727e2826bbf9be585096b4cf774051b5daa5e67540lgr5b"] Jul 10 18:46:59.513807 ip-10-0-143-17 kubenswrapper[2585]: I0710 18:46:59.513785 2585 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-marketplace/685f1ed643dcce69727e2826bbf9be585096b4cf774051b5daa5e67540lgr5b" Jul 10 18:46:59.519243 ip-10-0-143-17 kubenswrapper[2585]: I0710 18:46:59.519217 2585 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-marketplace\"/\"kube-root-ca.crt\"" Jul 10 18:46:59.520272 ip-10-0-143-17 kubenswrapper[2585]: I0710 18:46:59.520203 2585 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-marketplace\"/\"default-dockercfg-lrkqb\"" Jul 10 18:46:59.520384 ip-10-0-143-17 kubenswrapper[2585]: I0710 18:46:59.520203 2585 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-marketplace\"/\"openshift-service-ca.crt\"" Jul 10 18:46:59.533425 ip-10-0-143-17 kubenswrapper[2585]: I0710 18:46:59.533405 2585 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-marketplace/685f1ed643dcce69727e2826bbf9be585096b4cf774051b5daa5e67540lgr5b"] Jul 10 18:46:59.675081 ip-10-0-143-17 kubenswrapper[2585]: I0710 18:46:59.675047 2585 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"bundle\" (UniqueName: \"kubernetes.io/empty-dir/d7fa66fd-2e92-4a50-84de-470433cebcb3-bundle\") pod \"685f1ed643dcce69727e2826bbf9be585096b4cf774051b5daa5e67540lgr5b\" (UID: \"d7fa66fd-2e92-4a50-84de-470433cebcb3\") " pod="openshift-marketplace/685f1ed643dcce69727e2826bbf9be585096b4cf774051b5daa5e67540lgr5b" Jul 10 18:46:59.675222 ip-10-0-143-17 kubenswrapper[2585]: I0710 18:46:59.675096 2585 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-rvqvf\" (UniqueName: \"kubernetes.io/projected/d7fa66fd-2e92-4a50-84de-470433cebcb3-kube-api-access-rvqvf\") pod \"685f1ed643dcce69727e2826bbf9be585096b4cf774051b5daa5e67540lgr5b\" (UID: \"d7fa66fd-2e92-4a50-84de-470433cebcb3\") " pod="openshift-marketplace/685f1ed643dcce69727e2826bbf9be585096b4cf774051b5daa5e67540lgr5b" Jul 10 18:46:59.675222 ip-10-0-143-17 kubenswrapper[2585]: I0710 18:46:59.675120 2585 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"util\" (UniqueName: \"kubernetes.io/empty-dir/d7fa66fd-2e92-4a50-84de-470433cebcb3-util\") pod \"685f1ed643dcce69727e2826bbf9be585096b4cf774051b5daa5e67540lgr5b\" (UID: \"d7fa66fd-2e92-4a50-84de-470433cebcb3\") " pod="openshift-marketplace/685f1ed643dcce69727e2826bbf9be585096b4cf774051b5daa5e67540lgr5b" Jul 10 18:46:59.776352 ip-10-0-143-17 kubenswrapper[2585]: I0710 18:46:59.776278 2585 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"bundle\" (UniqueName: \"kubernetes.io/empty-dir/d7fa66fd-2e92-4a50-84de-470433cebcb3-bundle\") pod \"685f1ed643dcce69727e2826bbf9be585096b4cf774051b5daa5e67540lgr5b\" (UID: \"d7fa66fd-2e92-4a50-84de-470433cebcb3\") " pod="openshift-marketplace/685f1ed643dcce69727e2826bbf9be585096b4cf774051b5daa5e67540lgr5b" Jul 10 18:46:59.776352 ip-10-0-143-17 kubenswrapper[2585]: I0710 18:46:59.776318 2585 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-rvqvf\" (UniqueName: \"kubernetes.io/projected/d7fa66fd-2e92-4a50-84de-470433cebcb3-kube-api-access-rvqvf\") pod \"685f1ed643dcce69727e2826bbf9be585096b4cf774051b5daa5e67540lgr5b\" (UID: \"d7fa66fd-2e92-4a50-84de-470433cebcb3\") " pod="openshift-marketplace/685f1ed643dcce69727e2826bbf9be585096b4cf774051b5daa5e67540lgr5b" Jul 10 18:46:59.776500 ip-10-0-143-17 kubenswrapper[2585]: I0710 18:46:59.776351 2585 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"util\" (UniqueName: \"kubernetes.io/empty-dir/d7fa66fd-2e92-4a50-84de-470433cebcb3-util\") pod \"685f1ed643dcce69727e2826bbf9be585096b4cf774051b5daa5e67540lgr5b\" (UID: \"d7fa66fd-2e92-4a50-84de-470433cebcb3\") " pod="openshift-marketplace/685f1ed643dcce69727e2826bbf9be585096b4cf774051b5daa5e67540lgr5b" Jul 10 18:46:59.776766 ip-10-0-143-17 kubenswrapper[2585]: I0710 18:46:59.776747 2585 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"bundle\" (UniqueName: \"kubernetes.io/empty-dir/d7fa66fd-2e92-4a50-84de-470433cebcb3-bundle\") pod \"685f1ed643dcce69727e2826bbf9be585096b4cf774051b5daa5e67540lgr5b\" (UID: \"d7fa66fd-2e92-4a50-84de-470433cebcb3\") " pod="openshift-marketplace/685f1ed643dcce69727e2826bbf9be585096b4cf774051b5daa5e67540lgr5b" Jul 10 18:46:59.776811 ip-10-0-143-17 kubenswrapper[2585]: I0710 18:46:59.776762 2585 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"util\" (UniqueName: \"kubernetes.io/empty-dir/d7fa66fd-2e92-4a50-84de-470433cebcb3-util\") pod \"685f1ed643dcce69727e2826bbf9be585096b4cf774051b5daa5e67540lgr5b\" (UID: \"d7fa66fd-2e92-4a50-84de-470433cebcb3\") " pod="openshift-marketplace/685f1ed643dcce69727e2826bbf9be585096b4cf774051b5daa5e67540lgr5b" Jul 10 18:46:59.786734 ip-10-0-143-17 kubenswrapper[2585]: I0710 18:46:59.786708 2585 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-rvqvf\" (UniqueName: \"kubernetes.io/projected/d7fa66fd-2e92-4a50-84de-470433cebcb3-kube-api-access-rvqvf\") pod \"685f1ed643dcce69727e2826bbf9be585096b4cf774051b5daa5e67540lgr5b\" (UID: \"d7fa66fd-2e92-4a50-84de-470433cebcb3\") " pod="openshift-marketplace/685f1ed643dcce69727e2826bbf9be585096b4cf774051b5daa5e67540lgr5b" Jul 10 18:46:59.822631 ip-10-0-143-17 kubenswrapper[2585]: I0710 18:46:59.822610 2585 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-marketplace/685f1ed643dcce69727e2826bbf9be585096b4cf774051b5daa5e67540lgr5b" Jul 10 18:46:59.963795 ip-10-0-143-17 kubenswrapper[2585]: I0710 18:46:59.963769 2585 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-marketplace/685f1ed643dcce69727e2826bbf9be585096b4cf774051b5daa5e67540lgr5b"] Jul 10 18:46:59.966021 ip-10-0-143-17 kubenswrapper[2585]: W0710 18:46:59.965995 2585 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podd7fa66fd_2e92_4a50_84de_470433cebcb3.slice/crio-1329ce7fbbecccd4522392e6ecfc0c51935f0dec94881fbccb23f7bba9938dbb WatchSource:0}: Error finding container 1329ce7fbbecccd4522392e6ecfc0c51935f0dec94881fbccb23f7bba9938dbb: Status 404 returned error can't find the container with id 1329ce7fbbecccd4522392e6ecfc0c51935f0dec94881fbccb23f7bba9938dbb Jul 10 18:46:59.979074 ip-10-0-143-17 kubenswrapper[2585]: I0710 18:46:59.979046 2585 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-marketplace/685f1ed643dcce69727e2826bbf9be585096b4cf774051b5daa5e67540lgr5b" event={"ID":"d7fa66fd-2e92-4a50-84de-470433cebcb3","Type":"ContainerStarted","Data":"1329ce7fbbecccd4522392e6ecfc0c51935f0dec94881fbccb23f7bba9938dbb"} Jul 10 18:47:00.984911 ip-10-0-143-17 kubenswrapper[2585]: I0710 18:47:00.984871 2585 generic.go:356] "Generic (PLEG): container finished" podID="d7fa66fd-2e92-4a50-84de-470433cebcb3" containerID="7a25707505cb38610542c81c622130ed7828a68f3b1f86664ebfc86773c183ae" exitCode=0 Jul 10 18:47:00.985255 ip-10-0-143-17 kubenswrapper[2585]: I0710 18:47:00.984941 2585 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-marketplace/685f1ed643dcce69727e2826bbf9be585096b4cf774051b5daa5e67540lgr5b" event={"ID":"d7fa66fd-2e92-4a50-84de-470433cebcb3","Type":"ContainerDied","Data":"7a25707505cb38610542c81c622130ed7828a68f3b1f86664ebfc86773c183ae"} Jul 10 18:47:04.002119 ip-10-0-143-17 kubenswrapper[2585]: I0710 18:47:04.002069 2585 generic.go:356] "Generic (PLEG): container finished" podID="d7fa66fd-2e92-4a50-84de-470433cebcb3" containerID="7011680c0d80054f60e2f9d24fc4a08c6ce5f348a4ca5247c7787a86f0d22b1c" exitCode=0 Jul 10 18:47:04.002557 ip-10-0-143-17 kubenswrapper[2585]: I0710 18:47:04.002143 2585 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-marketplace/685f1ed643dcce69727e2826bbf9be585096b4cf774051b5daa5e67540lgr5b" event={"ID":"d7fa66fd-2e92-4a50-84de-470433cebcb3","Type":"ContainerDied","Data":"7011680c0d80054f60e2f9d24fc4a08c6ce5f348a4ca5247c7787a86f0d22b1c"} Jul 10 18:47:05.008408 ip-10-0-143-17 kubenswrapper[2585]: I0710 18:47:05.008371 2585 generic.go:356] "Generic (PLEG): container finished" podID="d7fa66fd-2e92-4a50-84de-470433cebcb3" containerID="abce11198c648288772d5433b685c6e20d77b9fd234b5bdfdabf783bebb098a0" exitCode=0 Jul 10 18:47:05.008796 ip-10-0-143-17 kubenswrapper[2585]: I0710 18:47:05.008452 2585 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-marketplace/685f1ed643dcce69727e2826bbf9be585096b4cf774051b5daa5e67540lgr5b" event={"ID":"d7fa66fd-2e92-4a50-84de-470433cebcb3","Type":"ContainerDied","Data":"abce11198c648288772d5433b685c6e20d77b9fd234b5bdfdabf783bebb098a0"} Jul 10 18:47:06.144302 ip-10-0-143-17 kubenswrapper[2585]: I0710 18:47:06.144277 2585 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-marketplace/685f1ed643dcce69727e2826bbf9be585096b4cf774051b5daa5e67540lgr5b" Jul 10 18:47:06.326988 ip-10-0-143-17 kubenswrapper[2585]: I0710 18:47:06.326920 2585 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"util\" (UniqueName: \"kubernetes.io/empty-dir/d7fa66fd-2e92-4a50-84de-470433cebcb3-util\") pod \"d7fa66fd-2e92-4a50-84de-470433cebcb3\" (UID: \"d7fa66fd-2e92-4a50-84de-470433cebcb3\") " Jul 10 18:47:06.327123 ip-10-0-143-17 kubenswrapper[2585]: I0710 18:47:06.326996 2585 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-rvqvf\" (UniqueName: \"kubernetes.io/projected/d7fa66fd-2e92-4a50-84de-470433cebcb3-kube-api-access-rvqvf\") pod \"d7fa66fd-2e92-4a50-84de-470433cebcb3\" (UID: \"d7fa66fd-2e92-4a50-84de-470433cebcb3\") " Jul 10 18:47:06.327123 ip-10-0-143-17 kubenswrapper[2585]: I0710 18:47:06.327068 2585 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"bundle\" (UniqueName: \"kubernetes.io/empty-dir/d7fa66fd-2e92-4a50-84de-470433cebcb3-bundle\") pod \"d7fa66fd-2e92-4a50-84de-470433cebcb3\" (UID: \"d7fa66fd-2e92-4a50-84de-470433cebcb3\") " Jul 10 18:47:06.327922 ip-10-0-143-17 kubenswrapper[2585]: I0710 18:47:06.327886 2585 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/d7fa66fd-2e92-4a50-84de-470433cebcb3-bundle" (OuterVolumeSpecName: "bundle") pod "d7fa66fd-2e92-4a50-84de-470433cebcb3" (UID: "d7fa66fd-2e92-4a50-84de-470433cebcb3"). InnerVolumeSpecName "bundle". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Jul 10 18:47:06.328964 ip-10-0-143-17 kubenswrapper[2585]: I0710 18:47:06.328942 2585 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/d7fa66fd-2e92-4a50-84de-470433cebcb3-kube-api-access-rvqvf" (OuterVolumeSpecName: "kube-api-access-rvqvf") pod "d7fa66fd-2e92-4a50-84de-470433cebcb3" (UID: "d7fa66fd-2e92-4a50-84de-470433cebcb3"). InnerVolumeSpecName "kube-api-access-rvqvf". PluginName "kubernetes.io/projected", VolumeGIDValue "" Jul 10 18:47:06.332369 ip-10-0-143-17 kubenswrapper[2585]: I0710 18:47:06.332331 2585 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/d7fa66fd-2e92-4a50-84de-470433cebcb3-util" (OuterVolumeSpecName: "util") pod "d7fa66fd-2e92-4a50-84de-470433cebcb3" (UID: "d7fa66fd-2e92-4a50-84de-470433cebcb3"). InnerVolumeSpecName "util". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Jul 10 18:47:06.428203 ip-10-0-143-17 kubenswrapper[2585]: I0710 18:47:06.428184 2585 reconciler_common.go:299] "Volume detached for volume \"util\" (UniqueName: \"kubernetes.io/empty-dir/d7fa66fd-2e92-4a50-84de-470433cebcb3-util\") on node \"ip-10-0-143-17.ec2.internal\" DevicePath \"\"" Jul 10 18:47:06.428203 ip-10-0-143-17 kubenswrapper[2585]: I0710 18:47:06.428205 2585 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-rvqvf\" (UniqueName: \"kubernetes.io/projected/d7fa66fd-2e92-4a50-84de-470433cebcb3-kube-api-access-rvqvf\") on node \"ip-10-0-143-17.ec2.internal\" DevicePath \"\"" Jul 10 18:47:06.428340 ip-10-0-143-17 kubenswrapper[2585]: I0710 18:47:06.428216 2585 reconciler_common.go:299] "Volume detached for volume \"bundle\" (UniqueName: \"kubernetes.io/empty-dir/d7fa66fd-2e92-4a50-84de-470433cebcb3-bundle\") on node \"ip-10-0-143-17.ec2.internal\" DevicePath \"\"" Jul 10 18:47:07.021330 ip-10-0-143-17 kubenswrapper[2585]: I0710 18:47:07.021296 2585 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-marketplace/685f1ed643dcce69727e2826bbf9be585096b4cf774051b5daa5e67540lgr5b" event={"ID":"d7fa66fd-2e92-4a50-84de-470433cebcb3","Type":"ContainerDied","Data":"1329ce7fbbecccd4522392e6ecfc0c51935f0dec94881fbccb23f7bba9938dbb"} Jul 10 18:47:07.021330 ip-10-0-143-17 kubenswrapper[2585]: I0710 18:47:07.021322 2585 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-marketplace/685f1ed643dcce69727e2826bbf9be585096b4cf774051b5daa5e67540lgr5b" Jul 10 18:47:07.021504 ip-10-0-143-17 kubenswrapper[2585]: I0710 18:47:07.021339 2585 pod_container_deletor.go:80] "Container not found in pod's containers" containerID="1329ce7fbbecccd4522392e6ecfc0c51935f0dec94881fbccb23f7bba9938dbb" Jul 10 18:47:21.631895 ip-10-0-143-17 kubenswrapper[2585]: I0710 18:47:21.631816 2585 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-marketplace/77defbb6647882b321b46d16bdefe62633c3425354d3d93571a1d6a87fwqzfk"] Jul 10 18:47:21.632229 ip-10-0-143-17 kubenswrapper[2585]: I0710 18:47:21.632114 2585 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="d7fa66fd-2e92-4a50-84de-470433cebcb3" containerName="pull" Jul 10 18:47:21.632229 ip-10-0-143-17 kubenswrapper[2585]: I0710 18:47:21.632126 2585 state_mem.go:107] "Deleted CPUSet assignment" podUID="d7fa66fd-2e92-4a50-84de-470433cebcb3" containerName="pull" Jul 10 18:47:21.632229 ip-10-0-143-17 kubenswrapper[2585]: I0710 18:47:21.632140 2585 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="d7fa66fd-2e92-4a50-84de-470433cebcb3" containerName="util" Jul 10 18:47:21.632229 ip-10-0-143-17 kubenswrapper[2585]: I0710 18:47:21.632145 2585 state_mem.go:107] "Deleted CPUSet assignment" podUID="d7fa66fd-2e92-4a50-84de-470433cebcb3" containerName="util" Jul 10 18:47:21.632229 ip-10-0-143-17 kubenswrapper[2585]: I0710 18:47:21.632152 2585 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="d7fa66fd-2e92-4a50-84de-470433cebcb3" containerName="extract" Jul 10 18:47:21.632229 ip-10-0-143-17 kubenswrapper[2585]: I0710 18:47:21.632157 2585 state_mem.go:107] "Deleted CPUSet assignment" podUID="d7fa66fd-2e92-4a50-84de-470433cebcb3" containerName="extract" Jul 10 18:47:21.632229 ip-10-0-143-17 kubenswrapper[2585]: I0710 18:47:21.632213 2585 memory_manager.go:368] "RemoveStaleState removing state" podUID="d7fa66fd-2e92-4a50-84de-470433cebcb3" containerName="extract" Jul 10 18:47:21.635275 ip-10-0-143-17 kubenswrapper[2585]: I0710 18:47:21.635258 2585 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-marketplace/77defbb6647882b321b46d16bdefe62633c3425354d3d93571a1d6a87fwqzfk" Jul 10 18:47:21.638030 ip-10-0-143-17 kubenswrapper[2585]: I0710 18:47:21.638003 2585 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-marketplace\"/\"kube-root-ca.crt\"" Jul 10 18:47:21.638170 ip-10-0-143-17 kubenswrapper[2585]: I0710 18:47:21.638002 2585 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-marketplace\"/\"openshift-service-ca.crt\"" Jul 10 18:47:21.639249 ip-10-0-143-17 kubenswrapper[2585]: I0710 18:47:21.639226 2585 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-marketplace\"/\"default-dockercfg-lrkqb\"" Jul 10 18:47:21.644173 ip-10-0-143-17 kubenswrapper[2585]: I0710 18:47:21.644151 2585 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-marketplace/77defbb6647882b321b46d16bdefe62633c3425354d3d93571a1d6a87fwqzfk"] Jul 10 18:47:21.730459 ip-10-0-143-17 kubenswrapper[2585]: I0710 18:47:21.730435 2585 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-9xj7v\" (UniqueName: \"kubernetes.io/projected/dcb2868a-a425-4620-a1b8-9795816ea724-kube-api-access-9xj7v\") pod \"77defbb6647882b321b46d16bdefe62633c3425354d3d93571a1d6a87fwqzfk\" (UID: \"dcb2868a-a425-4620-a1b8-9795816ea724\") " pod="openshift-marketplace/77defbb6647882b321b46d16bdefe62633c3425354d3d93571a1d6a87fwqzfk" Jul 10 18:47:21.730590 ip-10-0-143-17 kubenswrapper[2585]: I0710 18:47:21.730470 2585 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"bundle\" (UniqueName: \"kubernetes.io/empty-dir/dcb2868a-a425-4620-a1b8-9795816ea724-bundle\") pod \"77defbb6647882b321b46d16bdefe62633c3425354d3d93571a1d6a87fwqzfk\" (UID: \"dcb2868a-a425-4620-a1b8-9795816ea724\") " pod="openshift-marketplace/77defbb6647882b321b46d16bdefe62633c3425354d3d93571a1d6a87fwqzfk" Jul 10 18:47:21.730590 ip-10-0-143-17 kubenswrapper[2585]: I0710 18:47:21.730510 2585 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"util\" (UniqueName: \"kubernetes.io/empty-dir/dcb2868a-a425-4620-a1b8-9795816ea724-util\") pod \"77defbb6647882b321b46d16bdefe62633c3425354d3d93571a1d6a87fwqzfk\" (UID: \"dcb2868a-a425-4620-a1b8-9795816ea724\") " pod="openshift-marketplace/77defbb6647882b321b46d16bdefe62633c3425354d3d93571a1d6a87fwqzfk" Jul 10 18:47:21.831199 ip-10-0-143-17 kubenswrapper[2585]: I0710 18:47:21.831173 2585 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-9xj7v\" (UniqueName: \"kubernetes.io/projected/dcb2868a-a425-4620-a1b8-9795816ea724-kube-api-access-9xj7v\") pod \"77defbb6647882b321b46d16bdefe62633c3425354d3d93571a1d6a87fwqzfk\" (UID: \"dcb2868a-a425-4620-a1b8-9795816ea724\") " pod="openshift-marketplace/77defbb6647882b321b46d16bdefe62633c3425354d3d93571a1d6a87fwqzfk" Jul 10 18:47:21.831296 ip-10-0-143-17 kubenswrapper[2585]: I0710 18:47:21.831209 2585 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"bundle\" (UniqueName: \"kubernetes.io/empty-dir/dcb2868a-a425-4620-a1b8-9795816ea724-bundle\") pod \"77defbb6647882b321b46d16bdefe62633c3425354d3d93571a1d6a87fwqzfk\" (UID: \"dcb2868a-a425-4620-a1b8-9795816ea724\") " pod="openshift-marketplace/77defbb6647882b321b46d16bdefe62633c3425354d3d93571a1d6a87fwqzfk" Jul 10 18:47:21.831296 ip-10-0-143-17 kubenswrapper[2585]: I0710 18:47:21.831261 2585 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"util\" (UniqueName: \"kubernetes.io/empty-dir/dcb2868a-a425-4620-a1b8-9795816ea724-util\") pod \"77defbb6647882b321b46d16bdefe62633c3425354d3d93571a1d6a87fwqzfk\" (UID: \"dcb2868a-a425-4620-a1b8-9795816ea724\") " pod="openshift-marketplace/77defbb6647882b321b46d16bdefe62633c3425354d3d93571a1d6a87fwqzfk" Jul 10 18:47:21.831694 ip-10-0-143-17 kubenswrapper[2585]: I0710 18:47:21.831678 2585 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"util\" (UniqueName: \"kubernetes.io/empty-dir/dcb2868a-a425-4620-a1b8-9795816ea724-util\") pod \"77defbb6647882b321b46d16bdefe62633c3425354d3d93571a1d6a87fwqzfk\" (UID: \"dcb2868a-a425-4620-a1b8-9795816ea724\") " pod="openshift-marketplace/77defbb6647882b321b46d16bdefe62633c3425354d3d93571a1d6a87fwqzfk" Jul 10 18:47:21.831747 ip-10-0-143-17 kubenswrapper[2585]: I0710 18:47:21.831701 2585 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"bundle\" (UniqueName: \"kubernetes.io/empty-dir/dcb2868a-a425-4620-a1b8-9795816ea724-bundle\") pod \"77defbb6647882b321b46d16bdefe62633c3425354d3d93571a1d6a87fwqzfk\" (UID: \"dcb2868a-a425-4620-a1b8-9795816ea724\") " pod="openshift-marketplace/77defbb6647882b321b46d16bdefe62633c3425354d3d93571a1d6a87fwqzfk" Jul 10 18:47:21.839849 ip-10-0-143-17 kubenswrapper[2585]: I0710 18:47:21.839827 2585 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-9xj7v\" (UniqueName: \"kubernetes.io/projected/dcb2868a-a425-4620-a1b8-9795816ea724-kube-api-access-9xj7v\") pod \"77defbb6647882b321b46d16bdefe62633c3425354d3d93571a1d6a87fwqzfk\" (UID: \"dcb2868a-a425-4620-a1b8-9795816ea724\") " pod="openshift-marketplace/77defbb6647882b321b46d16bdefe62633c3425354d3d93571a1d6a87fwqzfk" Jul 10 18:47:21.944229 ip-10-0-143-17 kubenswrapper[2585]: I0710 18:47:21.944171 2585 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-marketplace/77defbb6647882b321b46d16bdefe62633c3425354d3d93571a1d6a87fwqzfk" Jul 10 18:47:22.073342 ip-10-0-143-17 kubenswrapper[2585]: I0710 18:47:22.073303 2585 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-marketplace/77defbb6647882b321b46d16bdefe62633c3425354d3d93571a1d6a87fwqzfk"] Jul 10 18:47:22.075642 ip-10-0-143-17 kubenswrapper[2585]: W0710 18:47:22.075617 2585 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-poddcb2868a_a425_4620_a1b8_9795816ea724.slice/crio-6f60dbeec85df9152baf739c585fd2dc21ad2bbb24b85b5c375a19ec659bdb3b WatchSource:0}: Error finding container 6f60dbeec85df9152baf739c585fd2dc21ad2bbb24b85b5c375a19ec659bdb3b: Status 404 returned error can't find the container with id 6f60dbeec85df9152baf739c585fd2dc21ad2bbb24b85b5c375a19ec659bdb3b Jul 10 18:47:22.087237 ip-10-0-143-17 kubenswrapper[2585]: I0710 18:47:22.087214 2585 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-marketplace/77defbb6647882b321b46d16bdefe62633c3425354d3d93571a1d6a87fwqzfk" event={"ID":"dcb2868a-a425-4620-a1b8-9795816ea724","Type":"ContainerStarted","Data":"6f60dbeec85df9152baf739c585fd2dc21ad2bbb24b85b5c375a19ec659bdb3b"} Jul 10 18:47:23.093255 ip-10-0-143-17 kubenswrapper[2585]: I0710 18:47:23.093214 2585 generic.go:356] "Generic (PLEG): container finished" podID="dcb2868a-a425-4620-a1b8-9795816ea724" containerID="92a131b9a0859822ebb98cd9b77838523a2c457e3f3043c14523ca173d539099" exitCode=0 Jul 10 18:47:23.093638 ip-10-0-143-17 kubenswrapper[2585]: I0710 18:47:23.093298 2585 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-marketplace/77defbb6647882b321b46d16bdefe62633c3425354d3d93571a1d6a87fwqzfk" event={"ID":"dcb2868a-a425-4620-a1b8-9795816ea724","Type":"ContainerDied","Data":"92a131b9a0859822ebb98cd9b77838523a2c457e3f3043c14523ca173d539099"} Jul 10 18:47:29.123160 ip-10-0-143-17 kubenswrapper[2585]: I0710 18:47:29.123121 2585 generic.go:356] "Generic (PLEG): container finished" podID="dcb2868a-a425-4620-a1b8-9795816ea724" containerID="3f528751cc9a83b49a61df28632d41a418002c6fa316015289240a31b5d0b455" exitCode=0 Jul 10 18:47:29.123502 ip-10-0-143-17 kubenswrapper[2585]: I0710 18:47:29.123187 2585 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-marketplace/77defbb6647882b321b46d16bdefe62633c3425354d3d93571a1d6a87fwqzfk" event={"ID":"dcb2868a-a425-4620-a1b8-9795816ea724","Type":"ContainerDied","Data":"3f528751cc9a83b49a61df28632d41a418002c6fa316015289240a31b5d0b455"} Jul 10 18:47:30.130332 ip-10-0-143-17 kubenswrapper[2585]: I0710 18:47:30.130299 2585 generic.go:356] "Generic (PLEG): container finished" podID="dcb2868a-a425-4620-a1b8-9795816ea724" containerID="db77db4163bd907e1ffd19dd80729c27b50105a77cb356f4a7ad7364cf1f9d9e" exitCode=0 Jul 10 18:47:30.130732 ip-10-0-143-17 kubenswrapper[2585]: I0710 18:47:30.130383 2585 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-marketplace/77defbb6647882b321b46d16bdefe62633c3425354d3d93571a1d6a87fwqzfk" event={"ID":"dcb2868a-a425-4620-a1b8-9795816ea724","Type":"ContainerDied","Data":"db77db4163bd907e1ffd19dd80729c27b50105a77cb356f4a7ad7364cf1f9d9e"} Jul 10 18:47:31.264893 ip-10-0-143-17 kubenswrapper[2585]: I0710 18:47:31.264867 2585 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-marketplace/77defbb6647882b321b46d16bdefe62633c3425354d3d93571a1d6a87fwqzfk" Jul 10 18:47:31.303097 ip-10-0-143-17 kubenswrapper[2585]: I0710 18:47:31.303070 2585 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"bundle\" (UniqueName: \"kubernetes.io/empty-dir/dcb2868a-a425-4620-a1b8-9795816ea724-bundle\") pod \"dcb2868a-a425-4620-a1b8-9795816ea724\" (UID: \"dcb2868a-a425-4620-a1b8-9795816ea724\") " Jul 10 18:47:31.303226 ip-10-0-143-17 kubenswrapper[2585]: I0710 18:47:31.303135 2585 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-9xj7v\" (UniqueName: \"kubernetes.io/projected/dcb2868a-a425-4620-a1b8-9795816ea724-kube-api-access-9xj7v\") pod \"dcb2868a-a425-4620-a1b8-9795816ea724\" (UID: \"dcb2868a-a425-4620-a1b8-9795816ea724\") " Jul 10 18:47:31.303226 ip-10-0-143-17 kubenswrapper[2585]: I0710 18:47:31.303166 2585 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"util\" (UniqueName: \"kubernetes.io/empty-dir/dcb2868a-a425-4620-a1b8-9795816ea724-util\") pod \"dcb2868a-a425-4620-a1b8-9795816ea724\" (UID: \"dcb2868a-a425-4620-a1b8-9795816ea724\") " Jul 10 18:47:31.303800 ip-10-0-143-17 kubenswrapper[2585]: I0710 18:47:31.303772 2585 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/dcb2868a-a425-4620-a1b8-9795816ea724-bundle" (OuterVolumeSpecName: "bundle") pod "dcb2868a-a425-4620-a1b8-9795816ea724" (UID: "dcb2868a-a425-4620-a1b8-9795816ea724"). InnerVolumeSpecName "bundle". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Jul 10 18:47:31.305660 ip-10-0-143-17 kubenswrapper[2585]: I0710 18:47:31.305633 2585 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/dcb2868a-a425-4620-a1b8-9795816ea724-kube-api-access-9xj7v" (OuterVolumeSpecName: "kube-api-access-9xj7v") pod "dcb2868a-a425-4620-a1b8-9795816ea724" (UID: "dcb2868a-a425-4620-a1b8-9795816ea724"). InnerVolumeSpecName "kube-api-access-9xj7v". PluginName "kubernetes.io/projected", VolumeGIDValue "" Jul 10 18:47:31.308681 ip-10-0-143-17 kubenswrapper[2585]: I0710 18:47:31.308647 2585 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/dcb2868a-a425-4620-a1b8-9795816ea724-util" (OuterVolumeSpecName: "util") pod "dcb2868a-a425-4620-a1b8-9795816ea724" (UID: "dcb2868a-a425-4620-a1b8-9795816ea724"). InnerVolumeSpecName "util". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Jul 10 18:47:31.404463 ip-10-0-143-17 kubenswrapper[2585]: I0710 18:47:31.404412 2585 reconciler_common.go:299] "Volume detached for volume \"util\" (UniqueName: \"kubernetes.io/empty-dir/dcb2868a-a425-4620-a1b8-9795816ea724-util\") on node \"ip-10-0-143-17.ec2.internal\" DevicePath \"\"" Jul 10 18:47:31.404463 ip-10-0-143-17 kubenswrapper[2585]: I0710 18:47:31.404433 2585 reconciler_common.go:299] "Volume detached for volume \"bundle\" (UniqueName: \"kubernetes.io/empty-dir/dcb2868a-a425-4620-a1b8-9795816ea724-bundle\") on node \"ip-10-0-143-17.ec2.internal\" DevicePath \"\"" Jul 10 18:47:31.404463 ip-10-0-143-17 kubenswrapper[2585]: I0710 18:47:31.404442 2585 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-9xj7v\" (UniqueName: \"kubernetes.io/projected/dcb2868a-a425-4620-a1b8-9795816ea724-kube-api-access-9xj7v\") on node \"ip-10-0-143-17.ec2.internal\" DevicePath \"\"" Jul 10 18:47:32.148210 ip-10-0-143-17 kubenswrapper[2585]: I0710 18:47:32.148170 2585 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-marketplace/77defbb6647882b321b46d16bdefe62633c3425354d3d93571a1d6a87fwqzfk" event={"ID":"dcb2868a-a425-4620-a1b8-9795816ea724","Type":"ContainerDied","Data":"6f60dbeec85df9152baf739c585fd2dc21ad2bbb24b85b5c375a19ec659bdb3b"} Jul 10 18:47:32.148378 ip-10-0-143-17 kubenswrapper[2585]: I0710 18:47:32.148255 2585 pod_container_deletor.go:80] "Container not found in pod's containers" containerID="6f60dbeec85df9152baf739c585fd2dc21ad2bbb24b85b5c375a19ec659bdb3b" Jul 10 18:47:32.148378 ip-10-0-143-17 kubenswrapper[2585]: I0710 18:47:32.148173 2585 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-marketplace/77defbb6647882b321b46d16bdefe62633c3425354d3d93571a1d6a87fwqzfk" Jul 10 18:47:40.897238 ip-10-0-143-17 kubenswrapper[2585]: I0710 18:47:40.897200 2585 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-lws-operator/openshift-lws-operator-fd8ccff4c-kr5jx"] Jul 10 18:47:40.897720 ip-10-0-143-17 kubenswrapper[2585]: I0710 18:47:40.897650 2585 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="dcb2868a-a425-4620-a1b8-9795816ea724" containerName="extract" Jul 10 18:47:40.897720 ip-10-0-143-17 kubenswrapper[2585]: I0710 18:47:40.897668 2585 state_mem.go:107] "Deleted CPUSet assignment" podUID="dcb2868a-a425-4620-a1b8-9795816ea724" containerName="extract" Jul 10 18:47:40.897720 ip-10-0-143-17 kubenswrapper[2585]: I0710 18:47:40.897680 2585 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="dcb2868a-a425-4620-a1b8-9795816ea724" containerName="util" Jul 10 18:47:40.897720 ip-10-0-143-17 kubenswrapper[2585]: I0710 18:47:40.897688 2585 state_mem.go:107] "Deleted CPUSet assignment" podUID="dcb2868a-a425-4620-a1b8-9795816ea724" containerName="util" Jul 10 18:47:40.897720 ip-10-0-143-17 kubenswrapper[2585]: I0710 18:47:40.897705 2585 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="dcb2868a-a425-4620-a1b8-9795816ea724" containerName="pull" Jul 10 18:47:40.897720 ip-10-0-143-17 kubenswrapper[2585]: I0710 18:47:40.897712 2585 state_mem.go:107] "Deleted CPUSet assignment" podUID="dcb2868a-a425-4620-a1b8-9795816ea724" containerName="pull" Jul 10 18:47:40.898002 ip-10-0-143-17 kubenswrapper[2585]: I0710 18:47:40.897796 2585 memory_manager.go:368] "RemoveStaleState removing state" podUID="dcb2868a-a425-4620-a1b8-9795816ea724" containerName="extract" Jul 10 18:47:40.899716 ip-10-0-143-17 kubenswrapper[2585]: I0710 18:47:40.899695 2585 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-lws-operator/openshift-lws-operator-fd8ccff4c-kr5jx" Jul 10 18:47:40.902404 ip-10-0-143-17 kubenswrapper[2585]: I0710 18:47:40.902387 2585 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-lws-operator\"/\"openshift-lws-operator-dockercfg-qfnfw\"" Jul 10 18:47:40.903464 ip-10-0-143-17 kubenswrapper[2585]: I0710 18:47:40.903445 2585 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-lws-operator\"/\"kube-root-ca.crt\"" Jul 10 18:47:40.903549 ip-10-0-143-17 kubenswrapper[2585]: I0710 18:47:40.903445 2585 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-lws-operator\"/\"openshift-service-ca.crt\"" Jul 10 18:47:40.910857 ip-10-0-143-17 kubenswrapper[2585]: I0710 18:47:40.910838 2585 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-lws-operator/openshift-lws-operator-fd8ccff4c-kr5jx"] Jul 10 18:47:40.971357 ip-10-0-143-17 kubenswrapper[2585]: I0710 18:47:40.971329 2585 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-5pncf\" (UniqueName: \"kubernetes.io/projected/d8e11bcc-7272-4dcd-ad05-a0183cd58498-kube-api-access-5pncf\") pod \"openshift-lws-operator-fd8ccff4c-kr5jx\" (UID: \"d8e11bcc-7272-4dcd-ad05-a0183cd58498\") " pod="openshift-lws-operator/openshift-lws-operator-fd8ccff4c-kr5jx" Jul 10 18:47:40.971465 ip-10-0-143-17 kubenswrapper[2585]: I0710 18:47:40.971421 2585 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tmp\" (UniqueName: \"kubernetes.io/empty-dir/d8e11bcc-7272-4dcd-ad05-a0183cd58498-tmp\") pod \"openshift-lws-operator-fd8ccff4c-kr5jx\" (UID: \"d8e11bcc-7272-4dcd-ad05-a0183cd58498\") " pod="openshift-lws-operator/openshift-lws-operator-fd8ccff4c-kr5jx" Jul 10 18:47:41.072001 ip-10-0-143-17 kubenswrapper[2585]: I0710 18:47:41.071978 2585 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-5pncf\" (UniqueName: \"kubernetes.io/projected/d8e11bcc-7272-4dcd-ad05-a0183cd58498-kube-api-access-5pncf\") pod \"openshift-lws-operator-fd8ccff4c-kr5jx\" (UID: \"d8e11bcc-7272-4dcd-ad05-a0183cd58498\") " pod="openshift-lws-operator/openshift-lws-operator-fd8ccff4c-kr5jx" Jul 10 18:47:41.072079 ip-10-0-143-17 kubenswrapper[2585]: I0710 18:47:41.072045 2585 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tmp\" (UniqueName: \"kubernetes.io/empty-dir/d8e11bcc-7272-4dcd-ad05-a0183cd58498-tmp\") pod \"openshift-lws-operator-fd8ccff4c-kr5jx\" (UID: \"d8e11bcc-7272-4dcd-ad05-a0183cd58498\") " pod="openshift-lws-operator/openshift-lws-operator-fd8ccff4c-kr5jx" Jul 10 18:47:41.072344 ip-10-0-143-17 kubenswrapper[2585]: I0710 18:47:41.072326 2585 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"tmp\" (UniqueName: \"kubernetes.io/empty-dir/d8e11bcc-7272-4dcd-ad05-a0183cd58498-tmp\") pod \"openshift-lws-operator-fd8ccff4c-kr5jx\" (UID: \"d8e11bcc-7272-4dcd-ad05-a0183cd58498\") " pod="openshift-lws-operator/openshift-lws-operator-fd8ccff4c-kr5jx" Jul 10 18:47:41.080128 ip-10-0-143-17 kubenswrapper[2585]: I0710 18:47:41.080108 2585 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-5pncf\" (UniqueName: \"kubernetes.io/projected/d8e11bcc-7272-4dcd-ad05-a0183cd58498-kube-api-access-5pncf\") pod \"openshift-lws-operator-fd8ccff4c-kr5jx\" (UID: \"d8e11bcc-7272-4dcd-ad05-a0183cd58498\") " pod="openshift-lws-operator/openshift-lws-operator-fd8ccff4c-kr5jx" Jul 10 18:47:41.221778 ip-10-0-143-17 kubenswrapper[2585]: I0710 18:47:41.221751 2585 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-lws-operator/openshift-lws-operator-fd8ccff4c-kr5jx" Jul 10 18:47:41.349256 ip-10-0-143-17 kubenswrapper[2585]: I0710 18:47:41.349216 2585 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-lws-operator/openshift-lws-operator-fd8ccff4c-kr5jx"] Jul 10 18:47:41.352301 ip-10-0-143-17 kubenswrapper[2585]: W0710 18:47:41.352273 2585 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-besteffort.slice/kubepods-besteffort-podd8e11bcc_7272_4dcd_ad05_a0183cd58498.slice/crio-b928547ba6774235fa752cd1fe2407322275f6e36c6ab110a82f0362be721931 WatchSource:0}: Error finding container b928547ba6774235fa752cd1fe2407322275f6e36c6ab110a82f0362be721931: Status 404 returned error can't find the container with id b928547ba6774235fa752cd1fe2407322275f6e36c6ab110a82f0362be721931 Jul 10 18:47:42.191969 ip-10-0-143-17 kubenswrapper[2585]: I0710 18:47:42.191933 2585 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-lws-operator/openshift-lws-operator-fd8ccff4c-kr5jx" event={"ID":"d8e11bcc-7272-4dcd-ad05-a0183cd58498","Type":"ContainerStarted","Data":"b928547ba6774235fa752cd1fe2407322275f6e36c6ab110a82f0362be721931"} Jul 10 18:47:44.202668 ip-10-0-143-17 kubenswrapper[2585]: I0710 18:47:44.202635 2585 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-lws-operator/openshift-lws-operator-fd8ccff4c-kr5jx" event={"ID":"d8e11bcc-7272-4dcd-ad05-a0183cd58498","Type":"ContainerStarted","Data":"f8f55dfe5f4248f04ec28cf0561b0a9862a0bef515752f7bbdf55cea3d0cab51"} Jul 10 18:47:44.228696 ip-10-0-143-17 kubenswrapper[2585]: I0710 18:47:44.228647 2585 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-lws-operator/openshift-lws-operator-fd8ccff4c-kr5jx" podStartSLOduration=2.008763446 podStartE2EDuration="4.228633011s" totalImagesPullingTime="2.219869565s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-10 18:47:40 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-10 18:47:44.226277977 +0000 UTC m=+477.683637913" watchObservedRunningTime="2026-07-10 18:47:44.228633011 +0000 UTC m=+477.685993003" Jul 10 18:48:09.551466 ip-10-0-143-17 kubenswrapper[2585]: I0710 18:48:09.551434 2585 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-ingress/istiod-openshift-gateway-7574459656-kjkln"] Jul 10 18:48:09.553911 ip-10-0-143-17 kubenswrapper[2585]: I0710 18:48:09.553896 2585 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/istiod-openshift-gateway-7574459656-kjkln" Jul 10 18:48:09.556971 ip-10-0-143-17 kubenswrapper[2585]: I0710 18:48:09.556949 2585 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress\"/\"istiod-tls\"" Jul 10 18:48:09.556971 ip-10-0-143-17 kubenswrapper[2585]: I0710 18:48:09.556965 2585 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress\"/\"istiod-openshift-gateway-dockercfg-zm26b\"" Jul 10 18:48:09.557179 ip-10-0-143-17 kubenswrapper[2585]: I0710 18:48:09.556950 2585 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress\"/\"istio-kubeconfig\"" Jul 10 18:48:09.557179 ip-10-0-143-17 kubenswrapper[2585]: I0710 18:48:09.556970 2585 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ingress\"/\"openshift-gw-ca-root-cert\"" Jul 10 18:48:09.557179 ip-10-0-143-17 kubenswrapper[2585]: I0710 18:48:09.556949 2585 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress\"/\"cacerts\"" Jul 10 18:48:09.571267 ip-10-0-143-17 kubenswrapper[2585]: I0710 18:48:09.571243 2585 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/istiod-openshift-gateway-7574459656-kjkln"] Jul 10 18:48:09.670202 ip-10-0-143-17 kubenswrapper[2585]: I0710 18:48:09.670174 2585 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-kubeconfig\" (UniqueName: \"kubernetes.io/secret/1ecf8920-95e7-4eb5-91ed-a9cc8c05c108-istio-kubeconfig\") pod \"istiod-openshift-gateway-7574459656-kjkln\" (UID: \"1ecf8920-95e7-4eb5-91ed-a9cc8c05c108\") " pod="openshift-ingress/istiod-openshift-gateway-7574459656-kjkln" Jul 10 18:48:09.670202 ip-10-0-143-17 kubenswrapper[2585]: I0710 18:48:09.670203 2585 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"local-certs\" (UniqueName: \"kubernetes.io/empty-dir/1ecf8920-95e7-4eb5-91ed-a9cc8c05c108-local-certs\") pod \"istiod-openshift-gateway-7574459656-kjkln\" (UID: \"1ecf8920-95e7-4eb5-91ed-a9cc8c05c108\") " pod="openshift-ingress/istiod-openshift-gateway-7574459656-kjkln" Jul 10 18:48:09.670398 ip-10-0-143-17 kubenswrapper[2585]: I0710 18:48:09.670219 2585 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"cacerts\" (UniqueName: \"kubernetes.io/secret/1ecf8920-95e7-4eb5-91ed-a9cc8c05c108-cacerts\") pod \"istiod-openshift-gateway-7574459656-kjkln\" (UID: \"1ecf8920-95e7-4eb5-91ed-a9cc8c05c108\") " pod="openshift-ingress/istiod-openshift-gateway-7574459656-kjkln" Jul 10 18:48:09.670398 ip-10-0-143-17 kubenswrapper[2585]: I0710 18:48:09.670238 2585 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-csr-dns-cert\" (UniqueName: \"kubernetes.io/secret/1ecf8920-95e7-4eb5-91ed-a9cc8c05c108-istio-csr-dns-cert\") pod \"istiod-openshift-gateway-7574459656-kjkln\" (UID: \"1ecf8920-95e7-4eb5-91ed-a9cc8c05c108\") " pod="openshift-ingress/istiod-openshift-gateway-7574459656-kjkln" Jul 10 18:48:09.670398 ip-10-0-143-17 kubenswrapper[2585]: I0710 18:48:09.670327 2585 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-k4sgb\" (UniqueName: \"kubernetes.io/projected/1ecf8920-95e7-4eb5-91ed-a9cc8c05c108-kube-api-access-k4sgb\") pod \"istiod-openshift-gateway-7574459656-kjkln\" (UID: \"1ecf8920-95e7-4eb5-91ed-a9cc8c05c108\") " pod="openshift-ingress/istiod-openshift-gateway-7574459656-kjkln" Jul 10 18:48:09.670398 ip-10-0-143-17 kubenswrapper[2585]: I0710 18:48:09.670380 2585 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/1ecf8920-95e7-4eb5-91ed-a9cc8c05c108-istio-token\") pod \"istiod-openshift-gateway-7574459656-kjkln\" (UID: \"1ecf8920-95e7-4eb5-91ed-a9cc8c05c108\") " pod="openshift-ingress/istiod-openshift-gateway-7574459656-kjkln" Jul 10 18:48:09.670621 ip-10-0-143-17 kubenswrapper[2585]: I0710 18:48:09.670435 2585 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-csr-ca-configmap\" (UniqueName: \"kubernetes.io/configmap/1ecf8920-95e7-4eb5-91ed-a9cc8c05c108-istio-csr-ca-configmap\") pod \"istiod-openshift-gateway-7574459656-kjkln\" (UID: \"1ecf8920-95e7-4eb5-91ed-a9cc8c05c108\") " pod="openshift-ingress/istiod-openshift-gateway-7574459656-kjkln" Jul 10 18:48:09.771602 ip-10-0-143-17 kubenswrapper[2585]: I0710 18:48:09.771570 2585 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-k4sgb\" (UniqueName: \"kubernetes.io/projected/1ecf8920-95e7-4eb5-91ed-a9cc8c05c108-kube-api-access-k4sgb\") pod \"istiod-openshift-gateway-7574459656-kjkln\" (UID: \"1ecf8920-95e7-4eb5-91ed-a9cc8c05c108\") " pod="openshift-ingress/istiod-openshift-gateway-7574459656-kjkln" Jul 10 18:48:09.771721 ip-10-0-143-17 kubenswrapper[2585]: I0710 18:48:09.771615 2585 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/1ecf8920-95e7-4eb5-91ed-a9cc8c05c108-istio-token\") pod \"istiod-openshift-gateway-7574459656-kjkln\" (UID: \"1ecf8920-95e7-4eb5-91ed-a9cc8c05c108\") " pod="openshift-ingress/istiod-openshift-gateway-7574459656-kjkln" Jul 10 18:48:09.771721 ip-10-0-143-17 kubenswrapper[2585]: I0710 18:48:09.771657 2585 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-csr-ca-configmap\" (UniqueName: \"kubernetes.io/configmap/1ecf8920-95e7-4eb5-91ed-a9cc8c05c108-istio-csr-ca-configmap\") pod \"istiod-openshift-gateway-7574459656-kjkln\" (UID: \"1ecf8920-95e7-4eb5-91ed-a9cc8c05c108\") " pod="openshift-ingress/istiod-openshift-gateway-7574459656-kjkln" Jul 10 18:48:09.771721 ip-10-0-143-17 kubenswrapper[2585]: I0710 18:48:09.771718 2585 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-kubeconfig\" (UniqueName: \"kubernetes.io/secret/1ecf8920-95e7-4eb5-91ed-a9cc8c05c108-istio-kubeconfig\") pod \"istiod-openshift-gateway-7574459656-kjkln\" (UID: \"1ecf8920-95e7-4eb5-91ed-a9cc8c05c108\") " pod="openshift-ingress/istiod-openshift-gateway-7574459656-kjkln" Jul 10 18:48:09.771880 ip-10-0-143-17 kubenswrapper[2585]: I0710 18:48:09.771744 2585 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"local-certs\" (UniqueName: \"kubernetes.io/empty-dir/1ecf8920-95e7-4eb5-91ed-a9cc8c05c108-local-certs\") pod \"istiod-openshift-gateway-7574459656-kjkln\" (UID: \"1ecf8920-95e7-4eb5-91ed-a9cc8c05c108\") " pod="openshift-ingress/istiod-openshift-gateway-7574459656-kjkln" Jul 10 18:48:09.771880 ip-10-0-143-17 kubenswrapper[2585]: I0710 18:48:09.771766 2585 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"cacerts\" (UniqueName: \"kubernetes.io/secret/1ecf8920-95e7-4eb5-91ed-a9cc8c05c108-cacerts\") pod \"istiod-openshift-gateway-7574459656-kjkln\" (UID: \"1ecf8920-95e7-4eb5-91ed-a9cc8c05c108\") " pod="openshift-ingress/istiod-openshift-gateway-7574459656-kjkln" Jul 10 18:48:09.771880 ip-10-0-143-17 kubenswrapper[2585]: I0710 18:48:09.771799 2585 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-csr-dns-cert\" (UniqueName: \"kubernetes.io/secret/1ecf8920-95e7-4eb5-91ed-a9cc8c05c108-istio-csr-dns-cert\") pod \"istiod-openshift-gateway-7574459656-kjkln\" (UID: \"1ecf8920-95e7-4eb5-91ed-a9cc8c05c108\") " pod="openshift-ingress/istiod-openshift-gateway-7574459656-kjkln" Jul 10 18:48:09.772344 ip-10-0-143-17 kubenswrapper[2585]: I0710 18:48:09.772316 2585 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istio-csr-ca-configmap\" (UniqueName: \"kubernetes.io/configmap/1ecf8920-95e7-4eb5-91ed-a9cc8c05c108-istio-csr-ca-configmap\") pod \"istiod-openshift-gateway-7574459656-kjkln\" (UID: \"1ecf8920-95e7-4eb5-91ed-a9cc8c05c108\") " pod="openshift-ingress/istiod-openshift-gateway-7574459656-kjkln" Jul 10 18:48:09.774091 ip-10-0-143-17 kubenswrapper[2585]: I0710 18:48:09.774064 2585 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"local-certs\" (UniqueName: \"kubernetes.io/empty-dir/1ecf8920-95e7-4eb5-91ed-a9cc8c05c108-local-certs\") pod \"istiod-openshift-gateway-7574459656-kjkln\" (UID: \"1ecf8920-95e7-4eb5-91ed-a9cc8c05c108\") " pod="openshift-ingress/istiod-openshift-gateway-7574459656-kjkln" Jul 10 18:48:09.774191 ip-10-0-143-17 kubenswrapper[2585]: I0710 18:48:09.774127 2585 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"cacerts\" (UniqueName: \"kubernetes.io/secret/1ecf8920-95e7-4eb5-91ed-a9cc8c05c108-cacerts\") pod \"istiod-openshift-gateway-7574459656-kjkln\" (UID: \"1ecf8920-95e7-4eb5-91ed-a9cc8c05c108\") " pod="openshift-ingress/istiod-openshift-gateway-7574459656-kjkln" Jul 10 18:48:09.774282 ip-10-0-143-17 kubenswrapper[2585]: I0710 18:48:09.774261 2585 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istio-kubeconfig\" (UniqueName: \"kubernetes.io/secret/1ecf8920-95e7-4eb5-91ed-a9cc8c05c108-istio-kubeconfig\") pod \"istiod-openshift-gateway-7574459656-kjkln\" (UID: \"1ecf8920-95e7-4eb5-91ed-a9cc8c05c108\") " pod="openshift-ingress/istiod-openshift-gateway-7574459656-kjkln" Jul 10 18:48:09.774338 ip-10-0-143-17 kubenswrapper[2585]: I0710 18:48:09.774272 2585 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istio-csr-dns-cert\" (UniqueName: \"kubernetes.io/secret/1ecf8920-95e7-4eb5-91ed-a9cc8c05c108-istio-csr-dns-cert\") pod \"istiod-openshift-gateway-7574459656-kjkln\" (UID: \"1ecf8920-95e7-4eb5-91ed-a9cc8c05c108\") " pod="openshift-ingress/istiod-openshift-gateway-7574459656-kjkln" Jul 10 18:48:09.779857 ip-10-0-143-17 kubenswrapper[2585]: I0710 18:48:09.779836 2585 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/1ecf8920-95e7-4eb5-91ed-a9cc8c05c108-istio-token\") pod \"istiod-openshift-gateway-7574459656-kjkln\" (UID: \"1ecf8920-95e7-4eb5-91ed-a9cc8c05c108\") " pod="openshift-ingress/istiod-openshift-gateway-7574459656-kjkln" Jul 10 18:48:09.783381 ip-10-0-143-17 kubenswrapper[2585]: I0710 18:48:09.783361 2585 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-k4sgb\" (UniqueName: \"kubernetes.io/projected/1ecf8920-95e7-4eb5-91ed-a9cc8c05c108-kube-api-access-k4sgb\") pod \"istiod-openshift-gateway-7574459656-kjkln\" (UID: \"1ecf8920-95e7-4eb5-91ed-a9cc8c05c108\") " pod="openshift-ingress/istiod-openshift-gateway-7574459656-kjkln" Jul 10 18:48:09.862634 ip-10-0-143-17 kubenswrapper[2585]: I0710 18:48:09.862584 2585 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/istiod-openshift-gateway-7574459656-kjkln" Jul 10 18:48:10.002851 ip-10-0-143-17 kubenswrapper[2585]: I0710 18:48:10.002820 2585 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/istiod-openshift-gateway-7574459656-kjkln"] Jul 10 18:48:10.005808 ip-10-0-143-17 kubenswrapper[2585]: W0710 18:48:10.005774 2585 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod1ecf8920_95e7_4eb5_91ed_a9cc8c05c108.slice/crio-209eca4911578c7e2ce36c1ba1a2d067ccf194c74df2f6c58c4d31a829313970 WatchSource:0}: Error finding container 209eca4911578c7e2ce36c1ba1a2d067ccf194c74df2f6c58c4d31a829313970: Status 404 returned error can't find the container with id 209eca4911578c7e2ce36c1ba1a2d067ccf194c74df2f6c58c4d31a829313970 Jul 10 18:48:10.313943 ip-10-0-143-17 kubenswrapper[2585]: I0710 18:48:10.313910 2585 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/istiod-openshift-gateway-7574459656-kjkln" event={"ID":"1ecf8920-95e7-4eb5-91ed-a9cc8c05c108","Type":"ContainerStarted","Data":"209eca4911578c7e2ce36c1ba1a2d067ccf194c74df2f6c58c4d31a829313970"} Jul 10 18:48:12.981454 ip-10-0-143-17 kubenswrapper[2585]: I0710 18:48:12.981409 2585 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236224Ki","pods":"250"} Jul 10 18:48:12.981713 ip-10-0-143-17 kubenswrapper[2585]: I0710 18:48:12.981501 2585 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236224Ki","pods":"250"} Jul 10 18:48:13.330173 ip-10-0-143-17 kubenswrapper[2585]: I0710 18:48:13.330069 2585 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/istiod-openshift-gateway-7574459656-kjkln" event={"ID":"1ecf8920-95e7-4eb5-91ed-a9cc8c05c108","Type":"ContainerStarted","Data":"6a7aedfc78c03cc0277a1cd27064e4710b0d17dcde6c1b6146c7d158b9ffcbea"} Jul 10 18:48:13.330173 ip-10-0-143-17 kubenswrapper[2585]: I0710 18:48:13.330150 2585 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/istiod-openshift-gateway-7574459656-kjkln" Jul 10 18:48:13.352071 ip-10-0-143-17 kubenswrapper[2585]: I0710 18:48:13.352026 2585 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-ingress/istiod-openshift-gateway-7574459656-kjkln" podStartSLOduration=1.3785337979999999 podStartE2EDuration="4.352014427s" totalImagesPullingTime="2.973480629s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-10 18:48:09 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-10 18:48:13.350723664 +0000 UTC m=+506.808083599" watchObservedRunningTime="2026-07-10 18:48:13.352014427 +0000 UTC m=+506.809374362" Jul 10 18:48:14.335914 ip-10-0-143-17 kubenswrapper[2585]: I0710 18:48:14.335883 2585 patch_prober.go:28] interesting pod/istiod-openshift-gateway-7574459656-kjkln container/discovery namespace/openshift-ingress: Readiness probe status=failure output="HTTP probe failed with statuscode: 503" start-of-body= Jul 10 18:48:14.336254 ip-10-0-143-17 kubenswrapper[2585]: I0710 18:48:14.335943 2585 prober.go:120] "Probe failed" probeType="Readiness" pod="openshift-ingress/istiod-openshift-gateway-7574459656-kjkln" podUID="1ecf8920-95e7-4eb5-91ed-a9cc8c05c108" containerName="discovery" probeResult="failure" output="HTTP probe failed with statuscode: 503" Jul 10 18:48:17.334960 ip-10-0-143-17 kubenswrapper[2585]: I0710 18:48:17.334933 2585 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/istiod-openshift-gateway-7574459656-kjkln" Jul 10 18:48:17.679514 ip-10-0-143-17 kubenswrapper[2585]: I0710 18:48:17.679436 2585 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-ingress/openshift-ai-inference-openshift-default-6c48cf8bcc-cl5g8"] Jul 10 18:48:17.683810 ip-10-0-143-17 kubenswrapper[2585]: I0710 18:48:17.683785 2585 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/openshift-ai-inference-openshift-default-6c48cf8bcc-cl5g8" Jul 10 18:48:17.686852 ip-10-0-143-17 kubenswrapper[2585]: I0710 18:48:17.686826 2585 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress\"/\"openshift-ai-inference-openshift-default-dockercfg-cb8n9\"" Jul 10 18:48:17.700188 ip-10-0-143-17 kubenswrapper[2585]: I0710 18:48:17.700164 2585 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/openshift-ai-inference-openshift-default-6c48cf8bcc-cl5g8"] Jul 10 18:48:17.838667 ip-10-0-143-17 kubenswrapper[2585]: I0710 18:48:17.838639 2585 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-gfmzm\" (UniqueName: \"kubernetes.io/projected/27c2089c-3b42-41ba-91c3-c6f72e532db5-kube-api-access-gfmzm\") pod \"openshift-ai-inference-openshift-default-6c48cf8bcc-cl5g8\" (UID: \"27c2089c-3b42-41ba-91c3-c6f72e532db5\") " pod="openshift-ingress/openshift-ai-inference-openshift-default-6c48cf8bcc-cl5g8" Jul 10 18:48:17.838812 ip-10-0-143-17 kubenswrapper[2585]: I0710 18:48:17.838684 2585 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/27c2089c-3b42-41ba-91c3-c6f72e532db5-istiod-ca-cert\") pod \"openshift-ai-inference-openshift-default-6c48cf8bcc-cl5g8\" (UID: \"27c2089c-3b42-41ba-91c3-c6f72e532db5\") " pod="openshift-ingress/openshift-ai-inference-openshift-default-6c48cf8bcc-cl5g8" Jul 10 18:48:17.838812 ip-10-0-143-17 kubenswrapper[2585]: I0710 18:48:17.838712 2585 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/27c2089c-3b42-41ba-91c3-c6f72e532db5-workload-certs\") pod \"openshift-ai-inference-openshift-default-6c48cf8bcc-cl5g8\" (UID: \"27c2089c-3b42-41ba-91c3-c6f72e532db5\") " pod="openshift-ingress/openshift-ai-inference-openshift-default-6c48cf8bcc-cl5g8" Jul 10 18:48:17.838812 ip-10-0-143-17 kubenswrapper[2585]: I0710 18:48:17.838735 2585 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/27c2089c-3b42-41ba-91c3-c6f72e532db5-istio-podinfo\") pod \"openshift-ai-inference-openshift-default-6c48cf8bcc-cl5g8\" (UID: \"27c2089c-3b42-41ba-91c3-c6f72e532db5\") " pod="openshift-ingress/openshift-ai-inference-openshift-default-6c48cf8bcc-cl5g8" Jul 10 18:48:17.838812 ip-10-0-143-17 kubenswrapper[2585]: I0710 18:48:17.838759 2585 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/27c2089c-3b42-41ba-91c3-c6f72e532db5-credential-socket\") pod \"openshift-ai-inference-openshift-default-6c48cf8bcc-cl5g8\" (UID: \"27c2089c-3b42-41ba-91c3-c6f72e532db5\") " pod="openshift-ingress/openshift-ai-inference-openshift-default-6c48cf8bcc-cl5g8" Jul 10 18:48:17.838812 ip-10-0-143-17 kubenswrapper[2585]: I0710 18:48:17.838775 2585 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/27c2089c-3b42-41ba-91c3-c6f72e532db5-workload-socket\") pod \"openshift-ai-inference-openshift-default-6c48cf8bcc-cl5g8\" (UID: \"27c2089c-3b42-41ba-91c3-c6f72e532db5\") " pod="openshift-ingress/openshift-ai-inference-openshift-default-6c48cf8bcc-cl5g8" Jul 10 18:48:17.838996 ip-10-0-143-17 kubenswrapper[2585]: I0710 18:48:17.838840 2585 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/27c2089c-3b42-41ba-91c3-c6f72e532db5-istio-envoy\") pod \"openshift-ai-inference-openshift-default-6c48cf8bcc-cl5g8\" (UID: \"27c2089c-3b42-41ba-91c3-c6f72e532db5\") " pod="openshift-ingress/openshift-ai-inference-openshift-default-6c48cf8bcc-cl5g8" Jul 10 18:48:17.838996 ip-10-0-143-17 kubenswrapper[2585]: I0710 18:48:17.838866 2585 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/27c2089c-3b42-41ba-91c3-c6f72e532db5-istio-data\") pod \"openshift-ai-inference-openshift-default-6c48cf8bcc-cl5g8\" (UID: \"27c2089c-3b42-41ba-91c3-c6f72e532db5\") " pod="openshift-ingress/openshift-ai-inference-openshift-default-6c48cf8bcc-cl5g8" Jul 10 18:48:17.838996 ip-10-0-143-17 kubenswrapper[2585]: I0710 18:48:17.838911 2585 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/27c2089c-3b42-41ba-91c3-c6f72e532db5-istio-token\") pod \"openshift-ai-inference-openshift-default-6c48cf8bcc-cl5g8\" (UID: \"27c2089c-3b42-41ba-91c3-c6f72e532db5\") " pod="openshift-ingress/openshift-ai-inference-openshift-default-6c48cf8bcc-cl5g8" Jul 10 18:48:17.898261 ip-10-0-143-17 kubenswrapper[2585]: I0710 18:48:17.898239 2585 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-console/console-5757c989b5-d7f7l"] Jul 10 18:48:17.902758 ip-10-0-143-17 kubenswrapper[2585]: I0710 18:48:17.902740 2585 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-console/console-5757c989b5-d7f7l" Jul 10 18:48:17.914417 ip-10-0-143-17 kubenswrapper[2585]: I0710 18:48:17.914390 2585 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-console/console-5757c989b5-d7f7l"] Jul 10 18:48:17.939717 ip-10-0-143-17 kubenswrapper[2585]: I0710 18:48:17.939671 2585 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/27c2089c-3b42-41ba-91c3-c6f72e532db5-istio-token\") pod \"openshift-ai-inference-openshift-default-6c48cf8bcc-cl5g8\" (UID: \"27c2089c-3b42-41ba-91c3-c6f72e532db5\") " pod="openshift-ingress/openshift-ai-inference-openshift-default-6c48cf8bcc-cl5g8" Jul 10 18:48:17.939717 ip-10-0-143-17 kubenswrapper[2585]: I0710 18:48:17.939700 2585 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-gfmzm\" (UniqueName: \"kubernetes.io/projected/27c2089c-3b42-41ba-91c3-c6f72e532db5-kube-api-access-gfmzm\") pod \"openshift-ai-inference-openshift-default-6c48cf8bcc-cl5g8\" (UID: \"27c2089c-3b42-41ba-91c3-c6f72e532db5\") " pod="openshift-ingress/openshift-ai-inference-openshift-default-6c48cf8bcc-cl5g8" Jul 10 18:48:17.939825 ip-10-0-143-17 kubenswrapper[2585]: I0710 18:48:17.939730 2585 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/27c2089c-3b42-41ba-91c3-c6f72e532db5-istiod-ca-cert\") pod \"openshift-ai-inference-openshift-default-6c48cf8bcc-cl5g8\" (UID: \"27c2089c-3b42-41ba-91c3-c6f72e532db5\") " pod="openshift-ingress/openshift-ai-inference-openshift-default-6c48cf8bcc-cl5g8" Jul 10 18:48:17.939825 ip-10-0-143-17 kubenswrapper[2585]: I0710 18:48:17.939748 2585 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/27c2089c-3b42-41ba-91c3-c6f72e532db5-workload-certs\") pod \"openshift-ai-inference-openshift-default-6c48cf8bcc-cl5g8\" (UID: \"27c2089c-3b42-41ba-91c3-c6f72e532db5\") " pod="openshift-ingress/openshift-ai-inference-openshift-default-6c48cf8bcc-cl5g8" Jul 10 18:48:17.939825 ip-10-0-143-17 kubenswrapper[2585]: I0710 18:48:17.939773 2585 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/27c2089c-3b42-41ba-91c3-c6f72e532db5-istio-podinfo\") pod \"openshift-ai-inference-openshift-default-6c48cf8bcc-cl5g8\" (UID: \"27c2089c-3b42-41ba-91c3-c6f72e532db5\") " pod="openshift-ingress/openshift-ai-inference-openshift-default-6c48cf8bcc-cl5g8" Jul 10 18:48:17.939915 ip-10-0-143-17 kubenswrapper[2585]: I0710 18:48:17.939893 2585 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/27c2089c-3b42-41ba-91c3-c6f72e532db5-credential-socket\") pod \"openshift-ai-inference-openshift-default-6c48cf8bcc-cl5g8\" (UID: \"27c2089c-3b42-41ba-91c3-c6f72e532db5\") " pod="openshift-ingress/openshift-ai-inference-openshift-default-6c48cf8bcc-cl5g8" Jul 10 18:48:17.939956 ip-10-0-143-17 kubenswrapper[2585]: I0710 18:48:17.939937 2585 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/27c2089c-3b42-41ba-91c3-c6f72e532db5-workload-socket\") pod \"openshift-ai-inference-openshift-default-6c48cf8bcc-cl5g8\" (UID: \"27c2089c-3b42-41ba-91c3-c6f72e532db5\") " pod="openshift-ingress/openshift-ai-inference-openshift-default-6c48cf8bcc-cl5g8" Jul 10 18:48:17.939995 ip-10-0-143-17 kubenswrapper[2585]: I0710 18:48:17.939980 2585 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/27c2089c-3b42-41ba-91c3-c6f72e532db5-istio-envoy\") pod \"openshift-ai-inference-openshift-default-6c48cf8bcc-cl5g8\" (UID: \"27c2089c-3b42-41ba-91c3-c6f72e532db5\") " pod="openshift-ingress/openshift-ai-inference-openshift-default-6c48cf8bcc-cl5g8" Jul 10 18:48:17.940047 ip-10-0-143-17 kubenswrapper[2585]: I0710 18:48:17.940031 2585 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/27c2089c-3b42-41ba-91c3-c6f72e532db5-istio-data\") pod \"openshift-ai-inference-openshift-default-6c48cf8bcc-cl5g8\" (UID: \"27c2089c-3b42-41ba-91c3-c6f72e532db5\") " pod="openshift-ingress/openshift-ai-inference-openshift-default-6c48cf8bcc-cl5g8" Jul 10 18:48:17.940200 ip-10-0-143-17 kubenswrapper[2585]: I0710 18:48:17.940177 2585 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/27c2089c-3b42-41ba-91c3-c6f72e532db5-workload-certs\") pod \"openshift-ai-inference-openshift-default-6c48cf8bcc-cl5g8\" (UID: \"27c2089c-3b42-41ba-91c3-c6f72e532db5\") " pod="openshift-ingress/openshift-ai-inference-openshift-default-6c48cf8bcc-cl5g8" Jul 10 18:48:17.940282 ip-10-0-143-17 kubenswrapper[2585]: I0710 18:48:17.940217 2585 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/27c2089c-3b42-41ba-91c3-c6f72e532db5-credential-socket\") pod \"openshift-ai-inference-openshift-default-6c48cf8bcc-cl5g8\" (UID: \"27c2089c-3b42-41ba-91c3-c6f72e532db5\") " pod="openshift-ingress/openshift-ai-inference-openshift-default-6c48cf8bcc-cl5g8" Jul 10 18:48:17.940376 ip-10-0-143-17 kubenswrapper[2585]: I0710 18:48:17.940358 2585 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/27c2089c-3b42-41ba-91c3-c6f72e532db5-workload-socket\") pod \"openshift-ai-inference-openshift-default-6c48cf8bcc-cl5g8\" (UID: \"27c2089c-3b42-41ba-91c3-c6f72e532db5\") " pod="openshift-ingress/openshift-ai-inference-openshift-default-6c48cf8bcc-cl5g8" Jul 10 18:48:17.940414 ip-10-0-143-17 kubenswrapper[2585]: I0710 18:48:17.940400 2585 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/27c2089c-3b42-41ba-91c3-c6f72e532db5-istio-data\") pod \"openshift-ai-inference-openshift-default-6c48cf8bcc-cl5g8\" (UID: \"27c2089c-3b42-41ba-91c3-c6f72e532db5\") " pod="openshift-ingress/openshift-ai-inference-openshift-default-6c48cf8bcc-cl5g8" Jul 10 18:48:17.940478 ip-10-0-143-17 kubenswrapper[2585]: I0710 18:48:17.940463 2585 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/27c2089c-3b42-41ba-91c3-c6f72e532db5-istiod-ca-cert\") pod \"openshift-ai-inference-openshift-default-6c48cf8bcc-cl5g8\" (UID: \"27c2089c-3b42-41ba-91c3-c6f72e532db5\") " pod="openshift-ingress/openshift-ai-inference-openshift-default-6c48cf8bcc-cl5g8" Jul 10 18:48:17.942106 ip-10-0-143-17 kubenswrapper[2585]: I0710 18:48:17.942080 2585 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/27c2089c-3b42-41ba-91c3-c6f72e532db5-istio-envoy\") pod \"openshift-ai-inference-openshift-default-6c48cf8bcc-cl5g8\" (UID: \"27c2089c-3b42-41ba-91c3-c6f72e532db5\") " pod="openshift-ingress/openshift-ai-inference-openshift-default-6c48cf8bcc-cl5g8" Jul 10 18:48:17.942294 ip-10-0-143-17 kubenswrapper[2585]: I0710 18:48:17.942279 2585 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/27c2089c-3b42-41ba-91c3-c6f72e532db5-istio-podinfo\") pod \"openshift-ai-inference-openshift-default-6c48cf8bcc-cl5g8\" (UID: \"27c2089c-3b42-41ba-91c3-c6f72e532db5\") " pod="openshift-ingress/openshift-ai-inference-openshift-default-6c48cf8bcc-cl5g8" Jul 10 18:48:17.947629 ip-10-0-143-17 kubenswrapper[2585]: I0710 18:48:17.947601 2585 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-gfmzm\" (UniqueName: \"kubernetes.io/projected/27c2089c-3b42-41ba-91c3-c6f72e532db5-kube-api-access-gfmzm\") pod \"openshift-ai-inference-openshift-default-6c48cf8bcc-cl5g8\" (UID: \"27c2089c-3b42-41ba-91c3-c6f72e532db5\") " pod="openshift-ingress/openshift-ai-inference-openshift-default-6c48cf8bcc-cl5g8" Jul 10 18:48:17.947705 ip-10-0-143-17 kubenswrapper[2585]: I0710 18:48:17.947627 2585 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/27c2089c-3b42-41ba-91c3-c6f72e532db5-istio-token\") pod \"openshift-ai-inference-openshift-default-6c48cf8bcc-cl5g8\" (UID: \"27c2089c-3b42-41ba-91c3-c6f72e532db5\") " pod="openshift-ingress/openshift-ai-inference-openshift-default-6c48cf8bcc-cl5g8" Jul 10 18:48:17.999677 ip-10-0-143-17 kubenswrapper[2585]: I0710 18:48:17.999653 2585 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/openshift-ai-inference-openshift-default-6c48cf8bcc-cl5g8" Jul 10 18:48:18.040896 ip-10-0-143-17 kubenswrapper[2585]: I0710 18:48:18.040863 2585 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"oauth-serving-cert\" (UniqueName: \"kubernetes.io/configmap/e1138200-d6f8-4502-add0-d6c8a4f91714-oauth-serving-cert\") pod \"console-5757c989b5-d7f7l\" (UID: \"e1138200-d6f8-4502-add0-d6c8a4f91714\") " pod="openshift-console/console-5757c989b5-d7f7l" Jul 10 18:48:18.041038 ip-10-0-143-17 kubenswrapper[2585]: I0710 18:48:18.040913 2585 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"console-oauth-config\" (UniqueName: \"kubernetes.io/secret/e1138200-d6f8-4502-add0-d6c8a4f91714-console-oauth-config\") pod \"console-5757c989b5-d7f7l\" (UID: \"e1138200-d6f8-4502-add0-d6c8a4f91714\") " pod="openshift-console/console-5757c989b5-d7f7l" Jul 10 18:48:18.041038 ip-10-0-143-17 kubenswrapper[2585]: I0710 18:48:18.040957 2585 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/e1138200-d6f8-4502-add0-d6c8a4f91714-trusted-ca-bundle\") pod \"console-5757c989b5-d7f7l\" (UID: \"e1138200-d6f8-4502-add0-d6c8a4f91714\") " pod="openshift-console/console-5757c989b5-d7f7l" Jul 10 18:48:18.041038 ip-10-0-143-17 kubenswrapper[2585]: I0710 18:48:18.040992 2585 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-4rk8w\" (UniqueName: \"kubernetes.io/projected/e1138200-d6f8-4502-add0-d6c8a4f91714-kube-api-access-4rk8w\") pod \"console-5757c989b5-d7f7l\" (UID: \"e1138200-d6f8-4502-add0-d6c8a4f91714\") " pod="openshift-console/console-5757c989b5-d7f7l" Jul 10 18:48:18.041211 ip-10-0-143-17 kubenswrapper[2585]: I0710 18:48:18.041051 2585 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"service-ca\" (UniqueName: \"kubernetes.io/configmap/e1138200-d6f8-4502-add0-d6c8a4f91714-service-ca\") pod \"console-5757c989b5-d7f7l\" (UID: \"e1138200-d6f8-4502-add0-d6c8a4f91714\") " pod="openshift-console/console-5757c989b5-d7f7l" Jul 10 18:48:18.041211 ip-10-0-143-17 kubenswrapper[2585]: I0710 18:48:18.041112 2585 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"console-serving-cert\" (UniqueName: \"kubernetes.io/secret/e1138200-d6f8-4502-add0-d6c8a4f91714-console-serving-cert\") pod \"console-5757c989b5-d7f7l\" (UID: \"e1138200-d6f8-4502-add0-d6c8a4f91714\") " pod="openshift-console/console-5757c989b5-d7f7l" Jul 10 18:48:18.041211 ip-10-0-143-17 kubenswrapper[2585]: I0710 18:48:18.041134 2585 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"console-config\" (UniqueName: \"kubernetes.io/configmap/e1138200-d6f8-4502-add0-d6c8a4f91714-console-config\") pod \"console-5757c989b5-d7f7l\" (UID: \"e1138200-d6f8-4502-add0-d6c8a4f91714\") " pod="openshift-console/console-5757c989b5-d7f7l" Jul 10 18:48:18.132923 ip-10-0-143-17 kubenswrapper[2585]: I0710 18:48:18.132902 2585 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/openshift-ai-inference-openshift-default-6c48cf8bcc-cl5g8"] Jul 10 18:48:18.134969 ip-10-0-143-17 kubenswrapper[2585]: W0710 18:48:18.134939 2585 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod27c2089c_3b42_41ba_91c3_c6f72e532db5.slice/crio-d9f2cbe7ca488dbe9edd231f64171688f4c669eb75a365ecc9c80d51475fc601 WatchSource:0}: Error finding container d9f2cbe7ca488dbe9edd231f64171688f4c669eb75a365ecc9c80d51475fc601: Status 404 returned error can't find the container with id d9f2cbe7ca488dbe9edd231f64171688f4c669eb75a365ecc9c80d51475fc601 Jul 10 18:48:18.142134 ip-10-0-143-17 kubenswrapper[2585]: I0710 18:48:18.142110 2585 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"console-serving-cert\" (UniqueName: \"kubernetes.io/secret/e1138200-d6f8-4502-add0-d6c8a4f91714-console-serving-cert\") pod \"console-5757c989b5-d7f7l\" (UID: \"e1138200-d6f8-4502-add0-d6c8a4f91714\") " pod="openshift-console/console-5757c989b5-d7f7l" Jul 10 18:48:18.142237 ip-10-0-143-17 kubenswrapper[2585]: I0710 18:48:18.142139 2585 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"console-config\" (UniqueName: \"kubernetes.io/configmap/e1138200-d6f8-4502-add0-d6c8a4f91714-console-config\") pod \"console-5757c989b5-d7f7l\" (UID: \"e1138200-d6f8-4502-add0-d6c8a4f91714\") " pod="openshift-console/console-5757c989b5-d7f7l" Jul 10 18:48:18.142237 ip-10-0-143-17 kubenswrapper[2585]: I0710 18:48:18.142179 2585 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"oauth-serving-cert\" (UniqueName: \"kubernetes.io/configmap/e1138200-d6f8-4502-add0-d6c8a4f91714-oauth-serving-cert\") pod \"console-5757c989b5-d7f7l\" (UID: \"e1138200-d6f8-4502-add0-d6c8a4f91714\") " pod="openshift-console/console-5757c989b5-d7f7l" Jul 10 18:48:18.142237 ip-10-0-143-17 kubenswrapper[2585]: I0710 18:48:18.142202 2585 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"console-oauth-config\" (UniqueName: \"kubernetes.io/secret/e1138200-d6f8-4502-add0-d6c8a4f91714-console-oauth-config\") pod \"console-5757c989b5-d7f7l\" (UID: \"e1138200-d6f8-4502-add0-d6c8a4f91714\") " pod="openshift-console/console-5757c989b5-d7f7l" Jul 10 18:48:18.142393 ip-10-0-143-17 kubenswrapper[2585]: I0710 18:48:18.142262 2585 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/e1138200-d6f8-4502-add0-d6c8a4f91714-trusted-ca-bundle\") pod \"console-5757c989b5-d7f7l\" (UID: \"e1138200-d6f8-4502-add0-d6c8a4f91714\") " pod="openshift-console/console-5757c989b5-d7f7l" Jul 10 18:48:18.142393 ip-10-0-143-17 kubenswrapper[2585]: I0710 18:48:18.142292 2585 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-4rk8w\" (UniqueName: \"kubernetes.io/projected/e1138200-d6f8-4502-add0-d6c8a4f91714-kube-api-access-4rk8w\") pod \"console-5757c989b5-d7f7l\" (UID: \"e1138200-d6f8-4502-add0-d6c8a4f91714\") " pod="openshift-console/console-5757c989b5-d7f7l" Jul 10 18:48:18.142393 ip-10-0-143-17 kubenswrapper[2585]: I0710 18:48:18.142360 2585 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"service-ca\" (UniqueName: \"kubernetes.io/configmap/e1138200-d6f8-4502-add0-d6c8a4f91714-service-ca\") pod \"console-5757c989b5-d7f7l\" (UID: \"e1138200-d6f8-4502-add0-d6c8a4f91714\") " pod="openshift-console/console-5757c989b5-d7f7l" Jul 10 18:48:18.142907 ip-10-0-143-17 kubenswrapper[2585]: I0710 18:48:18.142884 2585 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"console-config\" (UniqueName: \"kubernetes.io/configmap/e1138200-d6f8-4502-add0-d6c8a4f91714-console-config\") pod \"console-5757c989b5-d7f7l\" (UID: \"e1138200-d6f8-4502-add0-d6c8a4f91714\") " pod="openshift-console/console-5757c989b5-d7f7l" Jul 10 18:48:18.143094 ip-10-0-143-17 kubenswrapper[2585]: I0710 18:48:18.143070 2585 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"oauth-serving-cert\" (UniqueName: \"kubernetes.io/configmap/e1138200-d6f8-4502-add0-d6c8a4f91714-oauth-serving-cert\") pod \"console-5757c989b5-d7f7l\" (UID: \"e1138200-d6f8-4502-add0-d6c8a4f91714\") " pod="openshift-console/console-5757c989b5-d7f7l" Jul 10 18:48:18.143145 ip-10-0-143-17 kubenswrapper[2585]: I0710 18:48:18.143110 2585 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"service-ca\" (UniqueName: \"kubernetes.io/configmap/e1138200-d6f8-4502-add0-d6c8a4f91714-service-ca\") pod \"console-5757c989b5-d7f7l\" (UID: \"e1138200-d6f8-4502-add0-d6c8a4f91714\") " pod="openshift-console/console-5757c989b5-d7f7l" Jul 10 18:48:18.143272 ip-10-0-143-17 kubenswrapper[2585]: I0710 18:48:18.143255 2585 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/e1138200-d6f8-4502-add0-d6c8a4f91714-trusted-ca-bundle\") pod \"console-5757c989b5-d7f7l\" (UID: \"e1138200-d6f8-4502-add0-d6c8a4f91714\") " pod="openshift-console/console-5757c989b5-d7f7l" Jul 10 18:48:18.145122 ip-10-0-143-17 kubenswrapper[2585]: I0710 18:48:18.145098 2585 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"console-oauth-config\" (UniqueName: \"kubernetes.io/secret/e1138200-d6f8-4502-add0-d6c8a4f91714-console-oauth-config\") pod \"console-5757c989b5-d7f7l\" (UID: \"e1138200-d6f8-4502-add0-d6c8a4f91714\") " pod="openshift-console/console-5757c989b5-d7f7l" Jul 10 18:48:18.145169 ip-10-0-143-17 kubenswrapper[2585]: I0710 18:48:18.145105 2585 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"console-serving-cert\" (UniqueName: \"kubernetes.io/secret/e1138200-d6f8-4502-add0-d6c8a4f91714-console-serving-cert\") pod \"console-5757c989b5-d7f7l\" (UID: \"e1138200-d6f8-4502-add0-d6c8a4f91714\") " pod="openshift-console/console-5757c989b5-d7f7l" Jul 10 18:48:18.149834 ip-10-0-143-17 kubenswrapper[2585]: I0710 18:48:18.149817 2585 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-4rk8w\" (UniqueName: \"kubernetes.io/projected/e1138200-d6f8-4502-add0-d6c8a4f91714-kube-api-access-4rk8w\") pod \"console-5757c989b5-d7f7l\" (UID: \"e1138200-d6f8-4502-add0-d6c8a4f91714\") " pod="openshift-console/console-5757c989b5-d7f7l" Jul 10 18:48:18.211602 ip-10-0-143-17 kubenswrapper[2585]: I0710 18:48:18.211526 2585 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-console/console-5757c989b5-d7f7l" Jul 10 18:48:18.342812 ip-10-0-143-17 kubenswrapper[2585]: I0710 18:48:18.342790 2585 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-console/console-5757c989b5-d7f7l"] Jul 10 18:48:18.344347 ip-10-0-143-17 kubenswrapper[2585]: W0710 18:48:18.344318 2585 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pode1138200_d6f8_4502_add0_d6c8a4f91714.slice/crio-b274c7024e5726662f006ff78528847e70e426fac34cfd494503e1ada536f76a WatchSource:0}: Error finding container b274c7024e5726662f006ff78528847e70e426fac34cfd494503e1ada536f76a: Status 404 returned error can't find the container with id b274c7024e5726662f006ff78528847e70e426fac34cfd494503e1ada536f76a Jul 10 18:48:18.353335 ip-10-0-143-17 kubenswrapper[2585]: I0710 18:48:18.353312 2585 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-console/console-5757c989b5-d7f7l" event={"ID":"e1138200-d6f8-4502-add0-d6c8a4f91714","Type":"ContainerStarted","Data":"b274c7024e5726662f006ff78528847e70e426fac34cfd494503e1ada536f76a"} Jul 10 18:48:18.354463 ip-10-0-143-17 kubenswrapper[2585]: I0710 18:48:18.354444 2585 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/openshift-ai-inference-openshift-default-6c48cf8bcc-cl5g8" event={"ID":"27c2089c-3b42-41ba-91c3-c6f72e532db5","Type":"ContainerStarted","Data":"d9f2cbe7ca488dbe9edd231f64171688f4c669eb75a365ecc9c80d51475fc601"} Jul 10 18:48:19.362076 ip-10-0-143-17 kubenswrapper[2585]: I0710 18:48:19.362035 2585 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-console/console-5757c989b5-d7f7l" event={"ID":"e1138200-d6f8-4502-add0-d6c8a4f91714","Type":"ContainerStarted","Data":"c5d1dfecd92abe501b7d1e1eb50a35369b7fde7038a7398c06561630ed5c1e08"} Jul 10 18:48:19.384401 ip-10-0-143-17 kubenswrapper[2585]: I0710 18:48:19.384348 2585 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-console/console-5757c989b5-d7f7l" podStartSLOduration=2.384327974 podStartE2EDuration="2.384327974s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-10 18:48:17 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-10 18:48:19.382199842 +0000 UTC m=+512.839559796" watchObservedRunningTime="2026-07-10 18:48:19.384327974 +0000 UTC m=+512.841687910" Jul 10 18:48:20.730606 ip-10-0-143-17 kubenswrapper[2585]: I0710 18:48:20.730567 2585 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236224Ki","pods":"250"} Jul 10 18:48:20.730914 ip-10-0-143-17 kubenswrapper[2585]: I0710 18:48:20.730635 2585 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236224Ki","pods":"250"} Jul 10 18:48:20.730914 ip-10-0-143-17 kubenswrapper[2585]: I0710 18:48:20.730665 2585 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236224Ki","pods":"250"} Jul 10 18:48:21.379767 ip-10-0-143-17 kubenswrapper[2585]: I0710 18:48:21.379731 2585 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/openshift-ai-inference-openshift-default-6c48cf8bcc-cl5g8" event={"ID":"27c2089c-3b42-41ba-91c3-c6f72e532db5","Type":"ContainerStarted","Data":"b51395a6803413c98cfe8f77513f733dcdaa778580d9b75c7002cc664c75e0e3"} Jul 10 18:48:21.401437 ip-10-0-143-17 kubenswrapper[2585]: I0710 18:48:21.401392 2585 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-ingress/openshift-ai-inference-openshift-default-6c48cf8bcc-cl5g8" podStartSLOduration=1.80842956 podStartE2EDuration="4.401379137s" totalImagesPullingTime="2.592949577s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-10 18:48:17 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-10 18:48:21.400282291 +0000 UTC m=+514.857642225" watchObservedRunningTime="2026-07-10 18:48:21.401379137 +0000 UTC m=+514.858739053" Jul 10 18:48:22.000557 ip-10-0-143-17 kubenswrapper[2585]: I0710 18:48:22.000513 2585 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="openshift-ingress/openshift-ai-inference-openshift-default-6c48cf8bcc-cl5g8" Jul 10 18:48:22.004944 ip-10-0-143-17 kubenswrapper[2585]: I0710 18:48:22.004922 2585 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="openshift-ingress/openshift-ai-inference-openshift-default-6c48cf8bcc-cl5g8" Jul 10 18:48:22.384428 ip-10-0-143-17 kubenswrapper[2585]: I0710 18:48:22.384354 2585 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/openshift-ai-inference-openshift-default-6c48cf8bcc-cl5g8" Jul 10 18:48:22.385237 ip-10-0-143-17 kubenswrapper[2585]: I0710 18:48:22.385217 2585 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/openshift-ai-inference-openshift-default-6c48cf8bcc-cl5g8" Jul 10 18:48:25.381593 ip-10-0-143-17 kubenswrapper[2585]: I0710 18:48:25.381559 2585 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-marketplace/af2a5cb5935dca7d5808b44f590e0d3d1363f94abbf5c1d8a71e95fcab8vttv"] Jul 10 18:48:25.385133 ip-10-0-143-17 kubenswrapper[2585]: I0710 18:48:25.385117 2585 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-marketplace/af2a5cb5935dca7d5808b44f590e0d3d1363f94abbf5c1d8a71e95fcab8vttv" Jul 10 18:48:25.387898 ip-10-0-143-17 kubenswrapper[2585]: I0710 18:48:25.387875 2585 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-marketplace\"/\"default-dockercfg-lrkqb\"" Jul 10 18:48:25.388025 ip-10-0-143-17 kubenswrapper[2585]: I0710 18:48:25.387876 2585 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-marketplace\"/\"kube-root-ca.crt\"" Jul 10 18:48:25.389069 ip-10-0-143-17 kubenswrapper[2585]: I0710 18:48:25.389048 2585 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-marketplace\"/\"openshift-service-ca.crt\"" Jul 10 18:48:25.395513 ip-10-0-143-17 kubenswrapper[2585]: I0710 18:48:25.395494 2585 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-marketplace/af2a5cb5935dca7d5808b44f590e0d3d1363f94abbf5c1d8a71e95fcab8vttv"] Jul 10 18:48:25.477803 ip-10-0-143-17 kubenswrapper[2585]: I0710 18:48:25.477780 2585 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-marketplace/f13fb8b568e19b60b32189bed3ab51e2b7cfe62efa490862cd57eead76vghm6"] Jul 10 18:48:25.481294 ip-10-0-143-17 kubenswrapper[2585]: I0710 18:48:25.481279 2585 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-marketplace/f13fb8b568e19b60b32189bed3ab51e2b7cfe62efa490862cd57eead76vghm6" Jul 10 18:48:25.489863 ip-10-0-143-17 kubenswrapper[2585]: I0710 18:48:25.489844 2585 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-marketplace/f13fb8b568e19b60b32189bed3ab51e2b7cfe62efa490862cd57eead76vghm6"] Jul 10 18:48:25.503244 ip-10-0-143-17 kubenswrapper[2585]: I0710 18:48:25.503224 2585 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"bundle\" (UniqueName: \"kubernetes.io/empty-dir/ad769c9e-a6d0-4f76-ab63-a667c8c9855e-bundle\") pod \"af2a5cb5935dca7d5808b44f590e0d3d1363f94abbf5c1d8a71e95fcab8vttv\" (UID: \"ad769c9e-a6d0-4f76-ab63-a667c8c9855e\") " pod="openshift-marketplace/af2a5cb5935dca7d5808b44f590e0d3d1363f94abbf5c1d8a71e95fcab8vttv" Jul 10 18:48:25.503329 ip-10-0-143-17 kubenswrapper[2585]: I0710 18:48:25.503265 2585 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-724lz\" (UniqueName: \"kubernetes.io/projected/ad769c9e-a6d0-4f76-ab63-a667c8c9855e-kube-api-access-724lz\") pod \"af2a5cb5935dca7d5808b44f590e0d3d1363f94abbf5c1d8a71e95fcab8vttv\" (UID: \"ad769c9e-a6d0-4f76-ab63-a667c8c9855e\") " pod="openshift-marketplace/af2a5cb5935dca7d5808b44f590e0d3d1363f94abbf5c1d8a71e95fcab8vttv" Jul 10 18:48:25.503329 ip-10-0-143-17 kubenswrapper[2585]: I0710 18:48:25.503304 2585 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"util\" (UniqueName: \"kubernetes.io/empty-dir/ad769c9e-a6d0-4f76-ab63-a667c8c9855e-util\") pod \"af2a5cb5935dca7d5808b44f590e0d3d1363f94abbf5c1d8a71e95fcab8vttv\" (UID: \"ad769c9e-a6d0-4f76-ab63-a667c8c9855e\") " pod="openshift-marketplace/af2a5cb5935dca7d5808b44f590e0d3d1363f94abbf5c1d8a71e95fcab8vttv" Jul 10 18:48:25.582012 ip-10-0-143-17 kubenswrapper[2585]: I0710 18:48:25.581988 2585 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-marketplace/299160ebe81f164e400242457c5a1580453e7d84fd8087b72a6eb317efw24cw"] Jul 10 18:48:25.585327 ip-10-0-143-17 kubenswrapper[2585]: I0710 18:48:25.585313 2585 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-marketplace/299160ebe81f164e400242457c5a1580453e7d84fd8087b72a6eb317efw24cw" Jul 10 18:48:25.593768 ip-10-0-143-17 kubenswrapper[2585]: I0710 18:48:25.593744 2585 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-marketplace/299160ebe81f164e400242457c5a1580453e7d84fd8087b72a6eb317efw24cw"] Jul 10 18:48:25.603994 ip-10-0-143-17 kubenswrapper[2585]: I0710 18:48:25.603968 2585 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"bundle\" (UniqueName: \"kubernetes.io/empty-dir/ad769c9e-a6d0-4f76-ab63-a667c8c9855e-bundle\") pod \"af2a5cb5935dca7d5808b44f590e0d3d1363f94abbf5c1d8a71e95fcab8vttv\" (UID: \"ad769c9e-a6d0-4f76-ab63-a667c8c9855e\") " pod="openshift-marketplace/af2a5cb5935dca7d5808b44f590e0d3d1363f94abbf5c1d8a71e95fcab8vttv" Jul 10 18:48:25.604093 ip-10-0-143-17 kubenswrapper[2585]: I0710 18:48:25.604030 2585 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-724lz\" (UniqueName: \"kubernetes.io/projected/ad769c9e-a6d0-4f76-ab63-a667c8c9855e-kube-api-access-724lz\") pod \"af2a5cb5935dca7d5808b44f590e0d3d1363f94abbf5c1d8a71e95fcab8vttv\" (UID: \"ad769c9e-a6d0-4f76-ab63-a667c8c9855e\") " pod="openshift-marketplace/af2a5cb5935dca7d5808b44f590e0d3d1363f94abbf5c1d8a71e95fcab8vttv" Jul 10 18:48:25.604093 ip-10-0-143-17 kubenswrapper[2585]: I0710 18:48:25.604073 2585 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"bundle\" (UniqueName: \"kubernetes.io/empty-dir/c5d7aa6a-a8cd-40fa-9d53-5a6b7b0a9c32-bundle\") pod \"f13fb8b568e19b60b32189bed3ab51e2b7cfe62efa490862cd57eead76vghm6\" (UID: \"c5d7aa6a-a8cd-40fa-9d53-5a6b7b0a9c32\") " pod="openshift-marketplace/f13fb8b568e19b60b32189bed3ab51e2b7cfe62efa490862cd57eead76vghm6" Jul 10 18:48:25.604222 ip-10-0-143-17 kubenswrapper[2585]: I0710 18:48:25.604104 2585 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"util\" (UniqueName: \"kubernetes.io/empty-dir/c5d7aa6a-a8cd-40fa-9d53-5a6b7b0a9c32-util\") pod \"f13fb8b568e19b60b32189bed3ab51e2b7cfe62efa490862cd57eead76vghm6\" (UID: \"c5d7aa6a-a8cd-40fa-9d53-5a6b7b0a9c32\") " pod="openshift-marketplace/f13fb8b568e19b60b32189bed3ab51e2b7cfe62efa490862cd57eead76vghm6" Jul 10 18:48:25.604222 ip-10-0-143-17 kubenswrapper[2585]: I0710 18:48:25.604131 2585 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-lmqqt\" (UniqueName: \"kubernetes.io/projected/c5d7aa6a-a8cd-40fa-9d53-5a6b7b0a9c32-kube-api-access-lmqqt\") pod \"f13fb8b568e19b60b32189bed3ab51e2b7cfe62efa490862cd57eead76vghm6\" (UID: \"c5d7aa6a-a8cd-40fa-9d53-5a6b7b0a9c32\") " pod="openshift-marketplace/f13fb8b568e19b60b32189bed3ab51e2b7cfe62efa490862cd57eead76vghm6" Jul 10 18:48:25.604339 ip-10-0-143-17 kubenswrapper[2585]: I0710 18:48:25.604209 2585 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"util\" (UniqueName: \"kubernetes.io/empty-dir/ad769c9e-a6d0-4f76-ab63-a667c8c9855e-util\") pod \"af2a5cb5935dca7d5808b44f590e0d3d1363f94abbf5c1d8a71e95fcab8vttv\" (UID: \"ad769c9e-a6d0-4f76-ab63-a667c8c9855e\") " pod="openshift-marketplace/af2a5cb5935dca7d5808b44f590e0d3d1363f94abbf5c1d8a71e95fcab8vttv" Jul 10 18:48:25.604407 ip-10-0-143-17 kubenswrapper[2585]: I0710 18:48:25.604389 2585 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"bundle\" (UniqueName: \"kubernetes.io/empty-dir/ad769c9e-a6d0-4f76-ab63-a667c8c9855e-bundle\") pod \"af2a5cb5935dca7d5808b44f590e0d3d1363f94abbf5c1d8a71e95fcab8vttv\" (UID: \"ad769c9e-a6d0-4f76-ab63-a667c8c9855e\") " pod="openshift-marketplace/af2a5cb5935dca7d5808b44f590e0d3d1363f94abbf5c1d8a71e95fcab8vttv" Jul 10 18:48:25.604583 ip-10-0-143-17 kubenswrapper[2585]: I0710 18:48:25.604562 2585 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"util\" (UniqueName: \"kubernetes.io/empty-dir/ad769c9e-a6d0-4f76-ab63-a667c8c9855e-util\") pod \"af2a5cb5935dca7d5808b44f590e0d3d1363f94abbf5c1d8a71e95fcab8vttv\" (UID: \"ad769c9e-a6d0-4f76-ab63-a667c8c9855e\") " pod="openshift-marketplace/af2a5cb5935dca7d5808b44f590e0d3d1363f94abbf5c1d8a71e95fcab8vttv" Jul 10 18:48:25.612690 ip-10-0-143-17 kubenswrapper[2585]: I0710 18:48:25.612673 2585 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-724lz\" (UniqueName: \"kubernetes.io/projected/ad769c9e-a6d0-4f76-ab63-a667c8c9855e-kube-api-access-724lz\") pod \"af2a5cb5935dca7d5808b44f590e0d3d1363f94abbf5c1d8a71e95fcab8vttv\" (UID: \"ad769c9e-a6d0-4f76-ab63-a667c8c9855e\") " pod="openshift-marketplace/af2a5cb5935dca7d5808b44f590e0d3d1363f94abbf5c1d8a71e95fcab8vttv" Jul 10 18:48:25.677817 ip-10-0-143-17 kubenswrapper[2585]: I0710 18:48:25.677763 2585 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-marketplace/0e9e3357d40614682d1ebdb402e8ba62e1e258fb473e6f54140aafff3644hqs"] Jul 10 18:48:25.681197 ip-10-0-143-17 kubenswrapper[2585]: I0710 18:48:25.681182 2585 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-marketplace/0e9e3357d40614682d1ebdb402e8ba62e1e258fb473e6f54140aafff3644hqs" Jul 10 18:48:25.693277 ip-10-0-143-17 kubenswrapper[2585]: I0710 18:48:25.693250 2585 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-marketplace/0e9e3357d40614682d1ebdb402e8ba62e1e258fb473e6f54140aafff3644hqs"] Jul 10 18:48:25.695026 ip-10-0-143-17 kubenswrapper[2585]: I0710 18:48:25.694592 2585 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-marketplace/af2a5cb5935dca7d5808b44f590e0d3d1363f94abbf5c1d8a71e95fcab8vttv" Jul 10 18:48:25.705634 ip-10-0-143-17 kubenswrapper[2585]: I0710 18:48:25.705613 2585 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"bundle\" (UniqueName: \"kubernetes.io/empty-dir/c5d7aa6a-a8cd-40fa-9d53-5a6b7b0a9c32-bundle\") pod \"f13fb8b568e19b60b32189bed3ab51e2b7cfe62efa490862cd57eead76vghm6\" (UID: \"c5d7aa6a-a8cd-40fa-9d53-5a6b7b0a9c32\") " pod="openshift-marketplace/f13fb8b568e19b60b32189bed3ab51e2b7cfe62efa490862cd57eead76vghm6" Jul 10 18:48:25.705728 ip-10-0-143-17 kubenswrapper[2585]: I0710 18:48:25.705641 2585 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"util\" (UniqueName: \"kubernetes.io/empty-dir/c5d7aa6a-a8cd-40fa-9d53-5a6b7b0a9c32-util\") pod \"f13fb8b568e19b60b32189bed3ab51e2b7cfe62efa490862cd57eead76vghm6\" (UID: \"c5d7aa6a-a8cd-40fa-9d53-5a6b7b0a9c32\") " pod="openshift-marketplace/f13fb8b568e19b60b32189bed3ab51e2b7cfe62efa490862cd57eead76vghm6" Jul 10 18:48:25.705728 ip-10-0-143-17 kubenswrapper[2585]: I0710 18:48:25.705661 2585 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-lmqqt\" (UniqueName: \"kubernetes.io/projected/c5d7aa6a-a8cd-40fa-9d53-5a6b7b0a9c32-kube-api-access-lmqqt\") pod \"f13fb8b568e19b60b32189bed3ab51e2b7cfe62efa490862cd57eead76vghm6\" (UID: \"c5d7aa6a-a8cd-40fa-9d53-5a6b7b0a9c32\") " pod="openshift-marketplace/f13fb8b568e19b60b32189bed3ab51e2b7cfe62efa490862cd57eead76vghm6" Jul 10 18:48:25.705728 ip-10-0-143-17 kubenswrapper[2585]: I0710 18:48:25.705700 2585 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-lbwft\" (UniqueName: \"kubernetes.io/projected/4d0d90b9-02d7-4ccf-b59e-c158c8363691-kube-api-access-lbwft\") pod \"299160ebe81f164e400242457c5a1580453e7d84fd8087b72a6eb317efw24cw\" (UID: \"4d0d90b9-02d7-4ccf-b59e-c158c8363691\") " pod="openshift-marketplace/299160ebe81f164e400242457c5a1580453e7d84fd8087b72a6eb317efw24cw" Jul 10 18:48:25.705890 ip-10-0-143-17 kubenswrapper[2585]: I0710 18:48:25.705755 2585 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"bundle\" (UniqueName: \"kubernetes.io/empty-dir/4d0d90b9-02d7-4ccf-b59e-c158c8363691-bundle\") pod \"299160ebe81f164e400242457c5a1580453e7d84fd8087b72a6eb317efw24cw\" (UID: \"4d0d90b9-02d7-4ccf-b59e-c158c8363691\") " pod="openshift-marketplace/299160ebe81f164e400242457c5a1580453e7d84fd8087b72a6eb317efw24cw" Jul 10 18:48:25.705890 ip-10-0-143-17 kubenswrapper[2585]: I0710 18:48:25.705809 2585 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"util\" (UniqueName: \"kubernetes.io/empty-dir/4d0d90b9-02d7-4ccf-b59e-c158c8363691-util\") pod \"299160ebe81f164e400242457c5a1580453e7d84fd8087b72a6eb317efw24cw\" (UID: \"4d0d90b9-02d7-4ccf-b59e-c158c8363691\") " pod="openshift-marketplace/299160ebe81f164e400242457c5a1580453e7d84fd8087b72a6eb317efw24cw" Jul 10 18:48:25.706041 ip-10-0-143-17 kubenswrapper[2585]: I0710 18:48:25.706021 2585 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"bundle\" (UniqueName: \"kubernetes.io/empty-dir/c5d7aa6a-a8cd-40fa-9d53-5a6b7b0a9c32-bundle\") pod \"f13fb8b568e19b60b32189bed3ab51e2b7cfe62efa490862cd57eead76vghm6\" (UID: \"c5d7aa6a-a8cd-40fa-9d53-5a6b7b0a9c32\") " pod="openshift-marketplace/f13fb8b568e19b60b32189bed3ab51e2b7cfe62efa490862cd57eead76vghm6" Jul 10 18:48:25.706116 ip-10-0-143-17 kubenswrapper[2585]: I0710 18:48:25.706091 2585 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"util\" (UniqueName: \"kubernetes.io/empty-dir/c5d7aa6a-a8cd-40fa-9d53-5a6b7b0a9c32-util\") pod \"f13fb8b568e19b60b32189bed3ab51e2b7cfe62efa490862cd57eead76vghm6\" (UID: \"c5d7aa6a-a8cd-40fa-9d53-5a6b7b0a9c32\") " pod="openshift-marketplace/f13fb8b568e19b60b32189bed3ab51e2b7cfe62efa490862cd57eead76vghm6" Jul 10 18:48:25.715009 ip-10-0-143-17 kubenswrapper[2585]: I0710 18:48:25.714982 2585 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-lmqqt\" (UniqueName: \"kubernetes.io/projected/c5d7aa6a-a8cd-40fa-9d53-5a6b7b0a9c32-kube-api-access-lmqqt\") pod \"f13fb8b568e19b60b32189bed3ab51e2b7cfe62efa490862cd57eead76vghm6\" (UID: \"c5d7aa6a-a8cd-40fa-9d53-5a6b7b0a9c32\") " pod="openshift-marketplace/f13fb8b568e19b60b32189bed3ab51e2b7cfe62efa490862cd57eead76vghm6" Jul 10 18:48:25.790267 ip-10-0-143-17 kubenswrapper[2585]: I0710 18:48:25.790237 2585 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-marketplace/f13fb8b568e19b60b32189bed3ab51e2b7cfe62efa490862cd57eead76vghm6" Jul 10 18:48:25.806766 ip-10-0-143-17 kubenswrapper[2585]: I0710 18:48:25.806740 2585 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-xq4bs\" (UniqueName: \"kubernetes.io/projected/000ca62e-7689-4432-9fb4-481ca5f76723-kube-api-access-xq4bs\") pod \"0e9e3357d40614682d1ebdb402e8ba62e1e258fb473e6f54140aafff3644hqs\" (UID: \"000ca62e-7689-4432-9fb4-481ca5f76723\") " pod="openshift-marketplace/0e9e3357d40614682d1ebdb402e8ba62e1e258fb473e6f54140aafff3644hqs" Jul 10 18:48:25.806889 ip-10-0-143-17 kubenswrapper[2585]: I0710 18:48:25.806778 2585 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"bundle\" (UniqueName: \"kubernetes.io/empty-dir/4d0d90b9-02d7-4ccf-b59e-c158c8363691-bundle\") pod \"299160ebe81f164e400242457c5a1580453e7d84fd8087b72a6eb317efw24cw\" (UID: \"4d0d90b9-02d7-4ccf-b59e-c158c8363691\") " pod="openshift-marketplace/299160ebe81f164e400242457c5a1580453e7d84fd8087b72a6eb317efw24cw" Jul 10 18:48:25.806889 ip-10-0-143-17 kubenswrapper[2585]: I0710 18:48:25.806803 2585 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"bundle\" (UniqueName: \"kubernetes.io/empty-dir/000ca62e-7689-4432-9fb4-481ca5f76723-bundle\") pod \"0e9e3357d40614682d1ebdb402e8ba62e1e258fb473e6f54140aafff3644hqs\" (UID: \"000ca62e-7689-4432-9fb4-481ca5f76723\") " pod="openshift-marketplace/0e9e3357d40614682d1ebdb402e8ba62e1e258fb473e6f54140aafff3644hqs" Jul 10 18:48:25.806889 ip-10-0-143-17 kubenswrapper[2585]: I0710 18:48:25.806833 2585 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"util\" (UniqueName: \"kubernetes.io/empty-dir/4d0d90b9-02d7-4ccf-b59e-c158c8363691-util\") pod \"299160ebe81f164e400242457c5a1580453e7d84fd8087b72a6eb317efw24cw\" (UID: \"4d0d90b9-02d7-4ccf-b59e-c158c8363691\") " pod="openshift-marketplace/299160ebe81f164e400242457c5a1580453e7d84fd8087b72a6eb317efw24cw" Jul 10 18:48:25.806889 ip-10-0-143-17 kubenswrapper[2585]: I0710 18:48:25.806867 2585 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"util\" (UniqueName: \"kubernetes.io/empty-dir/000ca62e-7689-4432-9fb4-481ca5f76723-util\") pod \"0e9e3357d40614682d1ebdb402e8ba62e1e258fb473e6f54140aafff3644hqs\" (UID: \"000ca62e-7689-4432-9fb4-481ca5f76723\") " pod="openshift-marketplace/0e9e3357d40614682d1ebdb402e8ba62e1e258fb473e6f54140aafff3644hqs" Jul 10 18:48:25.807774 ip-10-0-143-17 kubenswrapper[2585]: I0710 18:48:25.806924 2585 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-lbwft\" (UniqueName: \"kubernetes.io/projected/4d0d90b9-02d7-4ccf-b59e-c158c8363691-kube-api-access-lbwft\") pod \"299160ebe81f164e400242457c5a1580453e7d84fd8087b72a6eb317efw24cw\" (UID: \"4d0d90b9-02d7-4ccf-b59e-c158c8363691\") " pod="openshift-marketplace/299160ebe81f164e400242457c5a1580453e7d84fd8087b72a6eb317efw24cw" Jul 10 18:48:25.807774 ip-10-0-143-17 kubenswrapper[2585]: I0710 18:48:25.807566 2585 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"util\" (UniqueName: \"kubernetes.io/empty-dir/4d0d90b9-02d7-4ccf-b59e-c158c8363691-util\") pod \"299160ebe81f164e400242457c5a1580453e7d84fd8087b72a6eb317efw24cw\" (UID: \"4d0d90b9-02d7-4ccf-b59e-c158c8363691\") " pod="openshift-marketplace/299160ebe81f164e400242457c5a1580453e7d84fd8087b72a6eb317efw24cw" Jul 10 18:48:25.807774 ip-10-0-143-17 kubenswrapper[2585]: I0710 18:48:25.807644 2585 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"bundle\" (UniqueName: \"kubernetes.io/empty-dir/4d0d90b9-02d7-4ccf-b59e-c158c8363691-bundle\") pod \"299160ebe81f164e400242457c5a1580453e7d84fd8087b72a6eb317efw24cw\" (UID: \"4d0d90b9-02d7-4ccf-b59e-c158c8363691\") " pod="openshift-marketplace/299160ebe81f164e400242457c5a1580453e7d84fd8087b72a6eb317efw24cw" Jul 10 18:48:25.814987 ip-10-0-143-17 kubenswrapper[2585]: I0710 18:48:25.814961 2585 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-lbwft\" (UniqueName: \"kubernetes.io/projected/4d0d90b9-02d7-4ccf-b59e-c158c8363691-kube-api-access-lbwft\") pod \"299160ebe81f164e400242457c5a1580453e7d84fd8087b72a6eb317efw24cw\" (UID: \"4d0d90b9-02d7-4ccf-b59e-c158c8363691\") " pod="openshift-marketplace/299160ebe81f164e400242457c5a1580453e7d84fd8087b72a6eb317efw24cw" Jul 10 18:48:25.824755 ip-10-0-143-17 kubenswrapper[2585]: I0710 18:48:25.824732 2585 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-marketplace/af2a5cb5935dca7d5808b44f590e0d3d1363f94abbf5c1d8a71e95fcab8vttv"] Jul 10 18:48:25.826142 ip-10-0-143-17 kubenswrapper[2585]: W0710 18:48:25.826119 2585 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podad769c9e_a6d0_4f76_ab63_a667c8c9855e.slice/crio-b571b6b24048062a32d7368fbdf42a67b7fd704dbd8c7a010cca34de771a6ca7 WatchSource:0}: Error finding container b571b6b24048062a32d7368fbdf42a67b7fd704dbd8c7a010cca34de771a6ca7: Status 404 returned error can't find the container with id b571b6b24048062a32d7368fbdf42a67b7fd704dbd8c7a010cca34de771a6ca7 Jul 10 18:48:25.894574 ip-10-0-143-17 kubenswrapper[2585]: I0710 18:48:25.894550 2585 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-marketplace/299160ebe81f164e400242457c5a1580453e7d84fd8087b72a6eb317efw24cw" Jul 10 18:48:25.908183 ip-10-0-143-17 kubenswrapper[2585]: I0710 18:48:25.908151 2585 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"util\" (UniqueName: \"kubernetes.io/empty-dir/000ca62e-7689-4432-9fb4-481ca5f76723-util\") pod \"0e9e3357d40614682d1ebdb402e8ba62e1e258fb473e6f54140aafff3644hqs\" (UID: \"000ca62e-7689-4432-9fb4-481ca5f76723\") " pod="openshift-marketplace/0e9e3357d40614682d1ebdb402e8ba62e1e258fb473e6f54140aafff3644hqs" Jul 10 18:48:25.908351 ip-10-0-143-17 kubenswrapper[2585]: I0710 18:48:25.908312 2585 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-xq4bs\" (UniqueName: \"kubernetes.io/projected/000ca62e-7689-4432-9fb4-481ca5f76723-kube-api-access-xq4bs\") pod \"0e9e3357d40614682d1ebdb402e8ba62e1e258fb473e6f54140aafff3644hqs\" (UID: \"000ca62e-7689-4432-9fb4-481ca5f76723\") " pod="openshift-marketplace/0e9e3357d40614682d1ebdb402e8ba62e1e258fb473e6f54140aafff3644hqs" Jul 10 18:48:25.908447 ip-10-0-143-17 kubenswrapper[2585]: I0710 18:48:25.908369 2585 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"bundle\" (UniqueName: \"kubernetes.io/empty-dir/000ca62e-7689-4432-9fb4-481ca5f76723-bundle\") pod \"0e9e3357d40614682d1ebdb402e8ba62e1e258fb473e6f54140aafff3644hqs\" (UID: \"000ca62e-7689-4432-9fb4-481ca5f76723\") " pod="openshift-marketplace/0e9e3357d40614682d1ebdb402e8ba62e1e258fb473e6f54140aafff3644hqs" Jul 10 18:48:25.908870 ip-10-0-143-17 kubenswrapper[2585]: I0710 18:48:25.908574 2585 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"util\" (UniqueName: \"kubernetes.io/empty-dir/000ca62e-7689-4432-9fb4-481ca5f76723-util\") pod \"0e9e3357d40614682d1ebdb402e8ba62e1e258fb473e6f54140aafff3644hqs\" (UID: \"000ca62e-7689-4432-9fb4-481ca5f76723\") " pod="openshift-marketplace/0e9e3357d40614682d1ebdb402e8ba62e1e258fb473e6f54140aafff3644hqs" Jul 10 18:48:25.908870 ip-10-0-143-17 kubenswrapper[2585]: I0710 18:48:25.908758 2585 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"bundle\" (UniqueName: \"kubernetes.io/empty-dir/000ca62e-7689-4432-9fb4-481ca5f76723-bundle\") pod \"0e9e3357d40614682d1ebdb402e8ba62e1e258fb473e6f54140aafff3644hqs\" (UID: \"000ca62e-7689-4432-9fb4-481ca5f76723\") " pod="openshift-marketplace/0e9e3357d40614682d1ebdb402e8ba62e1e258fb473e6f54140aafff3644hqs" Jul 10 18:48:25.917226 ip-10-0-143-17 kubenswrapper[2585]: I0710 18:48:25.917204 2585 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-xq4bs\" (UniqueName: \"kubernetes.io/projected/000ca62e-7689-4432-9fb4-481ca5f76723-kube-api-access-xq4bs\") pod \"0e9e3357d40614682d1ebdb402e8ba62e1e258fb473e6f54140aafff3644hqs\" (UID: \"000ca62e-7689-4432-9fb4-481ca5f76723\") " pod="openshift-marketplace/0e9e3357d40614682d1ebdb402e8ba62e1e258fb473e6f54140aafff3644hqs" Jul 10 18:48:25.939396 ip-10-0-143-17 kubenswrapper[2585]: I0710 18:48:25.939365 2585 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-marketplace/f13fb8b568e19b60b32189bed3ab51e2b7cfe62efa490862cd57eead76vghm6"] Jul 10 18:48:25.995479 ip-10-0-143-17 kubenswrapper[2585]: I0710 18:48:25.995451 2585 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-marketplace/0e9e3357d40614682d1ebdb402e8ba62e1e258fb473e6f54140aafff3644hqs" Jul 10 18:48:26.041196 ip-10-0-143-17 kubenswrapper[2585]: I0710 18:48:26.041170 2585 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-marketplace/299160ebe81f164e400242457c5a1580453e7d84fd8087b72a6eb317efw24cw"] Jul 10 18:48:26.041581 ip-10-0-143-17 kubenswrapper[2585]: W0710 18:48:26.041555 2585 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod4d0d90b9_02d7_4ccf_b59e_c158c8363691.slice/crio-2bd052e0f6d30826503b9ca62b7246d53c8842d6f62b5266c2e8e7ad98230192 WatchSource:0}: Error finding container 2bd052e0f6d30826503b9ca62b7246d53c8842d6f62b5266c2e8e7ad98230192: Status 404 returned error can't find the container with id 2bd052e0f6d30826503b9ca62b7246d53c8842d6f62b5266c2e8e7ad98230192 Jul 10 18:48:26.146714 ip-10-0-143-17 kubenswrapper[2585]: I0710 18:48:26.146678 2585 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-marketplace/0e9e3357d40614682d1ebdb402e8ba62e1e258fb473e6f54140aafff3644hqs"] Jul 10 18:48:26.156550 ip-10-0-143-17 kubenswrapper[2585]: W0710 18:48:26.156513 2585 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod000ca62e_7689_4432_9fb4_481ca5f76723.slice/crio-ee5297b4d4b3f4899df25f583328a8b1db2e5792bb7d7dd030c9c4138c8d0264 WatchSource:0}: Error finding container ee5297b4d4b3f4899df25f583328a8b1db2e5792bb7d7dd030c9c4138c8d0264: Status 404 returned error can't find the container with id ee5297b4d4b3f4899df25f583328a8b1db2e5792bb7d7dd030c9c4138c8d0264 Jul 10 18:48:26.404003 ip-10-0-143-17 kubenswrapper[2585]: I0710 18:48:26.403974 2585 generic.go:356] "Generic (PLEG): container finished" podID="4d0d90b9-02d7-4ccf-b59e-c158c8363691" containerID="a740d18851b1dbdb4e59b033b59d67e906406474fb3c9c585807a8598db3b0b7" exitCode=0 Jul 10 18:48:26.404403 ip-10-0-143-17 kubenswrapper[2585]: I0710 18:48:26.404051 2585 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-marketplace/299160ebe81f164e400242457c5a1580453e7d84fd8087b72a6eb317efw24cw" event={"ID":"4d0d90b9-02d7-4ccf-b59e-c158c8363691","Type":"ContainerDied","Data":"a740d18851b1dbdb4e59b033b59d67e906406474fb3c9c585807a8598db3b0b7"} Jul 10 18:48:26.404403 ip-10-0-143-17 kubenswrapper[2585]: I0710 18:48:26.404097 2585 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-marketplace/299160ebe81f164e400242457c5a1580453e7d84fd8087b72a6eb317efw24cw" event={"ID":"4d0d90b9-02d7-4ccf-b59e-c158c8363691","Type":"ContainerStarted","Data":"2bd052e0f6d30826503b9ca62b7246d53c8842d6f62b5266c2e8e7ad98230192"} Jul 10 18:48:26.405516 ip-10-0-143-17 kubenswrapper[2585]: I0710 18:48:26.405490 2585 generic.go:356] "Generic (PLEG): container finished" podID="ad769c9e-a6d0-4f76-ab63-a667c8c9855e" containerID="d997904016f8c1c1a2411d9686a6db59304427438224509254b538bddfbbbf2a" exitCode=0 Jul 10 18:48:26.405657 ip-10-0-143-17 kubenswrapper[2585]: I0710 18:48:26.405565 2585 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-marketplace/af2a5cb5935dca7d5808b44f590e0d3d1363f94abbf5c1d8a71e95fcab8vttv" event={"ID":"ad769c9e-a6d0-4f76-ab63-a667c8c9855e","Type":"ContainerDied","Data":"d997904016f8c1c1a2411d9686a6db59304427438224509254b538bddfbbbf2a"} Jul 10 18:48:26.405657 ip-10-0-143-17 kubenswrapper[2585]: I0710 18:48:26.405605 2585 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-marketplace/af2a5cb5935dca7d5808b44f590e0d3d1363f94abbf5c1d8a71e95fcab8vttv" event={"ID":"ad769c9e-a6d0-4f76-ab63-a667c8c9855e","Type":"ContainerStarted","Data":"b571b6b24048062a32d7368fbdf42a67b7fd704dbd8c7a010cca34de771a6ca7"} Jul 10 18:48:26.407043 ip-10-0-143-17 kubenswrapper[2585]: I0710 18:48:26.407022 2585 generic.go:356] "Generic (PLEG): container finished" podID="000ca62e-7689-4432-9fb4-481ca5f76723" containerID="af9d80b765e92fbf5bdddba1449364131271a355e6079583775c89a0698685c8" exitCode=0 Jul 10 18:48:26.407132 ip-10-0-143-17 kubenswrapper[2585]: I0710 18:48:26.407102 2585 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-marketplace/0e9e3357d40614682d1ebdb402e8ba62e1e258fb473e6f54140aafff3644hqs" event={"ID":"000ca62e-7689-4432-9fb4-481ca5f76723","Type":"ContainerDied","Data":"af9d80b765e92fbf5bdddba1449364131271a355e6079583775c89a0698685c8"} Jul 10 18:48:26.407179 ip-10-0-143-17 kubenswrapper[2585]: I0710 18:48:26.407158 2585 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-marketplace/0e9e3357d40614682d1ebdb402e8ba62e1e258fb473e6f54140aafff3644hqs" event={"ID":"000ca62e-7689-4432-9fb4-481ca5f76723","Type":"ContainerStarted","Data":"ee5297b4d4b3f4899df25f583328a8b1db2e5792bb7d7dd030c9c4138c8d0264"} Jul 10 18:48:26.408836 ip-10-0-143-17 kubenswrapper[2585]: I0710 18:48:26.408812 2585 generic.go:356] "Generic (PLEG): container finished" podID="c5d7aa6a-a8cd-40fa-9d53-5a6b7b0a9c32" containerID="28a7a61778f4d81b26a8e5558c58c192a072807ad23e2da30892c7bd0579ec25" exitCode=0 Jul 10 18:48:26.408947 ip-10-0-143-17 kubenswrapper[2585]: I0710 18:48:26.408839 2585 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-marketplace/f13fb8b568e19b60b32189bed3ab51e2b7cfe62efa490862cd57eead76vghm6" event={"ID":"c5d7aa6a-a8cd-40fa-9d53-5a6b7b0a9c32","Type":"ContainerDied","Data":"28a7a61778f4d81b26a8e5558c58c192a072807ad23e2da30892c7bd0579ec25"} Jul 10 18:48:26.408947 ip-10-0-143-17 kubenswrapper[2585]: I0710 18:48:26.408885 2585 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-marketplace/f13fb8b568e19b60b32189bed3ab51e2b7cfe62efa490862cd57eead76vghm6" event={"ID":"c5d7aa6a-a8cd-40fa-9d53-5a6b7b0a9c32","Type":"ContainerStarted","Data":"a329e7b3336221c8a9fd7d2dd6d05e9d51f28e119b58712c74f34b74313b38c9"} Jul 10 18:48:27.415301 ip-10-0-143-17 kubenswrapper[2585]: I0710 18:48:27.415275 2585 generic.go:356] "Generic (PLEG): container finished" podID="c5d7aa6a-a8cd-40fa-9d53-5a6b7b0a9c32" containerID="19d0ee4d79c930e3e7cebf4f1ebbf721616b82cd3a9ff417480d9519b4b83b27" exitCode=0 Jul 10 18:48:27.415666 ip-10-0-143-17 kubenswrapper[2585]: I0710 18:48:27.415364 2585 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-marketplace/f13fb8b568e19b60b32189bed3ab51e2b7cfe62efa490862cd57eead76vghm6" event={"ID":"c5d7aa6a-a8cd-40fa-9d53-5a6b7b0a9c32","Type":"ContainerDied","Data":"19d0ee4d79c930e3e7cebf4f1ebbf721616b82cd3a9ff417480d9519b4b83b27"} Jul 10 18:48:27.417510 ip-10-0-143-17 kubenswrapper[2585]: I0710 18:48:27.417485 2585 generic.go:356] "Generic (PLEG): container finished" podID="4d0d90b9-02d7-4ccf-b59e-c158c8363691" containerID="9ff11d3eab23d6a569e35a76a1b045583b8a7d1c241366f2ea5319c039508787" exitCode=0 Jul 10 18:48:27.417615 ip-10-0-143-17 kubenswrapper[2585]: I0710 18:48:27.417576 2585 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-marketplace/299160ebe81f164e400242457c5a1580453e7d84fd8087b72a6eb317efw24cw" event={"ID":"4d0d90b9-02d7-4ccf-b59e-c158c8363691","Type":"ContainerDied","Data":"9ff11d3eab23d6a569e35a76a1b045583b8a7d1c241366f2ea5319c039508787"} Jul 10 18:48:27.419514 ip-10-0-143-17 kubenswrapper[2585]: I0710 18:48:27.419493 2585 generic.go:356] "Generic (PLEG): container finished" podID="ad769c9e-a6d0-4f76-ab63-a667c8c9855e" containerID="bf2f8df77e6102df6f58ddb2610cf748315e3659b316a7418d6547e4bde4f8a8" exitCode=0 Jul 10 18:48:27.419618 ip-10-0-143-17 kubenswrapper[2585]: I0710 18:48:27.419580 2585 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-marketplace/af2a5cb5935dca7d5808b44f590e0d3d1363f94abbf5c1d8a71e95fcab8vttv" event={"ID":"ad769c9e-a6d0-4f76-ab63-a667c8c9855e","Type":"ContainerDied","Data":"bf2f8df77e6102df6f58ddb2610cf748315e3659b316a7418d6547e4bde4f8a8"} Jul 10 18:48:27.421698 ip-10-0-143-17 kubenswrapper[2585]: I0710 18:48:27.421677 2585 generic.go:356] "Generic (PLEG): container finished" podID="000ca62e-7689-4432-9fb4-481ca5f76723" containerID="e70e59f3544e57c29337f29e39645ba19eeb2a523c9d8dc1b3b20f1dfeb148fc" exitCode=0 Jul 10 18:48:27.421807 ip-10-0-143-17 kubenswrapper[2585]: I0710 18:48:27.421706 2585 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-marketplace/0e9e3357d40614682d1ebdb402e8ba62e1e258fb473e6f54140aafff3644hqs" event={"ID":"000ca62e-7689-4432-9fb4-481ca5f76723","Type":"ContainerDied","Data":"e70e59f3544e57c29337f29e39645ba19eeb2a523c9d8dc1b3b20f1dfeb148fc"} Jul 10 18:48:28.212488 ip-10-0-143-17 kubenswrapper[2585]: I0710 18:48:28.212461 2585 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-console/console-5757c989b5-d7f7l" Jul 10 18:48:28.212698 ip-10-0-143-17 kubenswrapper[2585]: I0710 18:48:28.212572 2585 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="openshift-console/console-5757c989b5-d7f7l" Jul 10 18:48:28.216959 ip-10-0-143-17 kubenswrapper[2585]: I0710 18:48:28.216937 2585 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="openshift-console/console-5757c989b5-d7f7l" Jul 10 18:48:28.428930 ip-10-0-143-17 kubenswrapper[2585]: I0710 18:48:28.428899 2585 generic.go:356] "Generic (PLEG): container finished" podID="4d0d90b9-02d7-4ccf-b59e-c158c8363691" containerID="96e62727e84f1cf0b8029b3adb34c87ef4b4ec1d5b8212b050da26ed728e4bcd" exitCode=0 Jul 10 18:48:28.429300 ip-10-0-143-17 kubenswrapper[2585]: I0710 18:48:28.428991 2585 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-marketplace/299160ebe81f164e400242457c5a1580453e7d84fd8087b72a6eb317efw24cw" event={"ID":"4d0d90b9-02d7-4ccf-b59e-c158c8363691","Type":"ContainerDied","Data":"96e62727e84f1cf0b8029b3adb34c87ef4b4ec1d5b8212b050da26ed728e4bcd"} Jul 10 18:48:28.431063 ip-10-0-143-17 kubenswrapper[2585]: I0710 18:48:28.431044 2585 generic.go:356] "Generic (PLEG): container finished" podID="ad769c9e-a6d0-4f76-ab63-a667c8c9855e" containerID="95deddfdda5084ab78e80b84493fc48f0583f0506cc9ee1efe14c6f895eefcd1" exitCode=0 Jul 10 18:48:28.431164 ip-10-0-143-17 kubenswrapper[2585]: I0710 18:48:28.431128 2585 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-marketplace/af2a5cb5935dca7d5808b44f590e0d3d1363f94abbf5c1d8a71e95fcab8vttv" event={"ID":"ad769c9e-a6d0-4f76-ab63-a667c8c9855e","Type":"ContainerDied","Data":"95deddfdda5084ab78e80b84493fc48f0583f0506cc9ee1efe14c6f895eefcd1"} Jul 10 18:48:28.433085 ip-10-0-143-17 kubenswrapper[2585]: I0710 18:48:28.433064 2585 generic.go:356] "Generic (PLEG): container finished" podID="000ca62e-7689-4432-9fb4-481ca5f76723" containerID="5bbdff43b4f090ff241fe0c93182e4971e3bb4d98cc162db235e594121d1e09a" exitCode=0 Jul 10 18:48:28.433169 ip-10-0-143-17 kubenswrapper[2585]: I0710 18:48:28.433139 2585 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-marketplace/0e9e3357d40614682d1ebdb402e8ba62e1e258fb473e6f54140aafff3644hqs" event={"ID":"000ca62e-7689-4432-9fb4-481ca5f76723","Type":"ContainerDied","Data":"5bbdff43b4f090ff241fe0c93182e4971e3bb4d98cc162db235e594121d1e09a"} Jul 10 18:48:28.434926 ip-10-0-143-17 kubenswrapper[2585]: I0710 18:48:28.434908 2585 generic.go:356] "Generic (PLEG): container finished" podID="c5d7aa6a-a8cd-40fa-9d53-5a6b7b0a9c32" containerID="5ba917052ba327b2b3f7c1b6bfab6c2bc535b52fc1f4c5a33a96f431fbc9a80b" exitCode=0 Jul 10 18:48:28.435011 ip-10-0-143-17 kubenswrapper[2585]: I0710 18:48:28.434958 2585 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-marketplace/f13fb8b568e19b60b32189bed3ab51e2b7cfe62efa490862cd57eead76vghm6" event={"ID":"c5d7aa6a-a8cd-40fa-9d53-5a6b7b0a9c32","Type":"ContainerDied","Data":"5ba917052ba327b2b3f7c1b6bfab6c2bc535b52fc1f4c5a33a96f431fbc9a80b"} Jul 10 18:48:28.438820 ip-10-0-143-17 kubenswrapper[2585]: I0710 18:48:28.438801 2585 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-console/console-5757c989b5-d7f7l" Jul 10 18:48:28.500039 ip-10-0-143-17 kubenswrapper[2585]: I0710 18:48:28.500009 2585 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-console/console-dbc48dfcb-h89sb"] Jul 10 18:48:29.605257 ip-10-0-143-17 kubenswrapper[2585]: I0710 18:48:29.605066 2585 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-marketplace/0e9e3357d40614682d1ebdb402e8ba62e1e258fb473e6f54140aafff3644hqs" Jul 10 18:48:29.630512 ip-10-0-143-17 kubenswrapper[2585]: I0710 18:48:29.630476 2585 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-marketplace/f13fb8b568e19b60b32189bed3ab51e2b7cfe62efa490862cd57eead76vghm6" Jul 10 18:48:29.644440 ip-10-0-143-17 kubenswrapper[2585]: I0710 18:48:29.644420 2585 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-marketplace/af2a5cb5935dca7d5808b44f590e0d3d1363f94abbf5c1d8a71e95fcab8vttv" Jul 10 18:48:29.648817 ip-10-0-143-17 kubenswrapper[2585]: I0710 18:48:29.648794 2585 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-marketplace/299160ebe81f164e400242457c5a1580453e7d84fd8087b72a6eb317efw24cw" Jul 10 18:48:29.742038 ip-10-0-143-17 kubenswrapper[2585]: I0710 18:48:29.742016 2585 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"util\" (UniqueName: \"kubernetes.io/empty-dir/ad769c9e-a6d0-4f76-ab63-a667c8c9855e-util\") pod \"ad769c9e-a6d0-4f76-ab63-a667c8c9855e\" (UID: \"ad769c9e-a6d0-4f76-ab63-a667c8c9855e\") " Jul 10 18:48:29.742182 ip-10-0-143-17 kubenswrapper[2585]: I0710 18:48:29.742050 2585 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"bundle\" (UniqueName: \"kubernetes.io/empty-dir/c5d7aa6a-a8cd-40fa-9d53-5a6b7b0a9c32-bundle\") pod \"c5d7aa6a-a8cd-40fa-9d53-5a6b7b0a9c32\" (UID: \"c5d7aa6a-a8cd-40fa-9d53-5a6b7b0a9c32\") " Jul 10 18:48:29.742182 ip-10-0-143-17 kubenswrapper[2585]: I0710 18:48:29.742071 2585 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"util\" (UniqueName: \"kubernetes.io/empty-dir/4d0d90b9-02d7-4ccf-b59e-c158c8363691-util\") pod \"4d0d90b9-02d7-4ccf-b59e-c158c8363691\" (UID: \"4d0d90b9-02d7-4ccf-b59e-c158c8363691\") " Jul 10 18:48:29.742182 ip-10-0-143-17 kubenswrapper[2585]: I0710 18:48:29.742121 2585 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-lmqqt\" (UniqueName: \"kubernetes.io/projected/c5d7aa6a-a8cd-40fa-9d53-5a6b7b0a9c32-kube-api-access-lmqqt\") pod \"c5d7aa6a-a8cd-40fa-9d53-5a6b7b0a9c32\" (UID: \"c5d7aa6a-a8cd-40fa-9d53-5a6b7b0a9c32\") " Jul 10 18:48:29.742182 ip-10-0-143-17 kubenswrapper[2585]: I0710 18:48:29.742149 2585 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"bundle\" (UniqueName: \"kubernetes.io/empty-dir/ad769c9e-a6d0-4f76-ab63-a667c8c9855e-bundle\") pod \"ad769c9e-a6d0-4f76-ab63-a667c8c9855e\" (UID: \"ad769c9e-a6d0-4f76-ab63-a667c8c9855e\") " Jul 10 18:48:29.742182 ip-10-0-143-17 kubenswrapper[2585]: I0710 18:48:29.742173 2585 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"util\" (UniqueName: \"kubernetes.io/empty-dir/c5d7aa6a-a8cd-40fa-9d53-5a6b7b0a9c32-util\") pod \"c5d7aa6a-a8cd-40fa-9d53-5a6b7b0a9c32\" (UID: \"c5d7aa6a-a8cd-40fa-9d53-5a6b7b0a9c32\") " Jul 10 18:48:29.742436 ip-10-0-143-17 kubenswrapper[2585]: I0710 18:48:29.742306 2585 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-xq4bs\" (UniqueName: \"kubernetes.io/projected/000ca62e-7689-4432-9fb4-481ca5f76723-kube-api-access-xq4bs\") pod \"000ca62e-7689-4432-9fb4-481ca5f76723\" (UID: \"000ca62e-7689-4432-9fb4-481ca5f76723\") " Jul 10 18:48:29.742436 ip-10-0-143-17 kubenswrapper[2585]: I0710 18:48:29.742356 2585 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"util\" (UniqueName: \"kubernetes.io/empty-dir/000ca62e-7689-4432-9fb4-481ca5f76723-util\") pod \"000ca62e-7689-4432-9fb4-481ca5f76723\" (UID: \"000ca62e-7689-4432-9fb4-481ca5f76723\") " Jul 10 18:48:29.742436 ip-10-0-143-17 kubenswrapper[2585]: I0710 18:48:29.742401 2585 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"bundle\" (UniqueName: \"kubernetes.io/empty-dir/000ca62e-7689-4432-9fb4-481ca5f76723-bundle\") pod \"000ca62e-7689-4432-9fb4-481ca5f76723\" (UID: \"000ca62e-7689-4432-9fb4-481ca5f76723\") " Jul 10 18:48:29.742598 ip-10-0-143-17 kubenswrapper[2585]: I0710 18:48:29.742450 2585 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"bundle\" (UniqueName: \"kubernetes.io/empty-dir/4d0d90b9-02d7-4ccf-b59e-c158c8363691-bundle\") pod \"4d0d90b9-02d7-4ccf-b59e-c158c8363691\" (UID: \"4d0d90b9-02d7-4ccf-b59e-c158c8363691\") " Jul 10 18:48:29.742598 ip-10-0-143-17 kubenswrapper[2585]: I0710 18:48:29.742505 2585 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-724lz\" (UniqueName: \"kubernetes.io/projected/ad769c9e-a6d0-4f76-ab63-a667c8c9855e-kube-api-access-724lz\") pod \"ad769c9e-a6d0-4f76-ab63-a667c8c9855e\" (UID: \"ad769c9e-a6d0-4f76-ab63-a667c8c9855e\") " Jul 10 18:48:29.742598 ip-10-0-143-17 kubenswrapper[2585]: I0710 18:48:29.742563 2585 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-lbwft\" (UniqueName: \"kubernetes.io/projected/4d0d90b9-02d7-4ccf-b59e-c158c8363691-kube-api-access-lbwft\") pod \"4d0d90b9-02d7-4ccf-b59e-c158c8363691\" (UID: \"4d0d90b9-02d7-4ccf-b59e-c158c8363691\") " Jul 10 18:48:29.743245 ip-10-0-143-17 kubenswrapper[2585]: I0710 18:48:29.742954 2585 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/ad769c9e-a6d0-4f76-ab63-a667c8c9855e-bundle" (OuterVolumeSpecName: "bundle") pod "ad769c9e-a6d0-4f76-ab63-a667c8c9855e" (UID: "ad769c9e-a6d0-4f76-ab63-a667c8c9855e"). InnerVolumeSpecName "bundle". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Jul 10 18:48:29.743245 ip-10-0-143-17 kubenswrapper[2585]: I0710 18:48:29.743197 2585 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/000ca62e-7689-4432-9fb4-481ca5f76723-bundle" (OuterVolumeSpecName: "bundle") pod "000ca62e-7689-4432-9fb4-481ca5f76723" (UID: "000ca62e-7689-4432-9fb4-481ca5f76723"). InnerVolumeSpecName "bundle". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Jul 10 18:48:29.743488 ip-10-0-143-17 kubenswrapper[2585]: I0710 18:48:29.743267 2585 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/c5d7aa6a-a8cd-40fa-9d53-5a6b7b0a9c32-bundle" (OuterVolumeSpecName: "bundle") pod "c5d7aa6a-a8cd-40fa-9d53-5a6b7b0a9c32" (UID: "c5d7aa6a-a8cd-40fa-9d53-5a6b7b0a9c32"). InnerVolumeSpecName "bundle". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Jul 10 18:48:29.743920 ip-10-0-143-17 kubenswrapper[2585]: I0710 18:48:29.743897 2585 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/4d0d90b9-02d7-4ccf-b59e-c158c8363691-bundle" (OuterVolumeSpecName: "bundle") pod "4d0d90b9-02d7-4ccf-b59e-c158c8363691" (UID: "4d0d90b9-02d7-4ccf-b59e-c158c8363691"). InnerVolumeSpecName "bundle". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Jul 10 18:48:29.744797 ip-10-0-143-17 kubenswrapper[2585]: I0710 18:48:29.744768 2585 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/c5d7aa6a-a8cd-40fa-9d53-5a6b7b0a9c32-kube-api-access-lmqqt" (OuterVolumeSpecName: "kube-api-access-lmqqt") pod "c5d7aa6a-a8cd-40fa-9d53-5a6b7b0a9c32" (UID: "c5d7aa6a-a8cd-40fa-9d53-5a6b7b0a9c32"). InnerVolumeSpecName "kube-api-access-lmqqt". PluginName "kubernetes.io/projected", VolumeGIDValue "" Jul 10 18:48:29.745211 ip-10-0-143-17 kubenswrapper[2585]: I0710 18:48:29.745183 2585 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/4d0d90b9-02d7-4ccf-b59e-c158c8363691-kube-api-access-lbwft" (OuterVolumeSpecName: "kube-api-access-lbwft") pod "4d0d90b9-02d7-4ccf-b59e-c158c8363691" (UID: "4d0d90b9-02d7-4ccf-b59e-c158c8363691"). InnerVolumeSpecName "kube-api-access-lbwft". PluginName "kubernetes.io/projected", VolumeGIDValue "" Jul 10 18:48:29.745880 ip-10-0-143-17 kubenswrapper[2585]: I0710 18:48:29.745851 2585 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/000ca62e-7689-4432-9fb4-481ca5f76723-kube-api-access-xq4bs" (OuterVolumeSpecName: "kube-api-access-xq4bs") pod "000ca62e-7689-4432-9fb4-481ca5f76723" (UID: "000ca62e-7689-4432-9fb4-481ca5f76723"). InnerVolumeSpecName "kube-api-access-xq4bs". PluginName "kubernetes.io/projected", VolumeGIDValue "" Jul 10 18:48:29.746112 ip-10-0-143-17 kubenswrapper[2585]: I0710 18:48:29.746096 2585 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/ad769c9e-a6d0-4f76-ab63-a667c8c9855e-kube-api-access-724lz" (OuterVolumeSpecName: "kube-api-access-724lz") pod "ad769c9e-a6d0-4f76-ab63-a667c8c9855e" (UID: "ad769c9e-a6d0-4f76-ab63-a667c8c9855e"). InnerVolumeSpecName "kube-api-access-724lz". PluginName "kubernetes.io/projected", VolumeGIDValue "" Jul 10 18:48:29.751383 ip-10-0-143-17 kubenswrapper[2585]: I0710 18:48:29.751360 2585 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/c5d7aa6a-a8cd-40fa-9d53-5a6b7b0a9c32-util" (OuterVolumeSpecName: "util") pod "c5d7aa6a-a8cd-40fa-9d53-5a6b7b0a9c32" (UID: "c5d7aa6a-a8cd-40fa-9d53-5a6b7b0a9c32"). InnerVolumeSpecName "util". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Jul 10 18:48:29.751669 ip-10-0-143-17 kubenswrapper[2585]: I0710 18:48:29.751646 2585 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/000ca62e-7689-4432-9fb4-481ca5f76723-util" (OuterVolumeSpecName: "util") pod "000ca62e-7689-4432-9fb4-481ca5f76723" (UID: "000ca62e-7689-4432-9fb4-481ca5f76723"). InnerVolumeSpecName "util". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Jul 10 18:48:29.751722 ip-10-0-143-17 kubenswrapper[2585]: I0710 18:48:29.751708 2585 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/ad769c9e-a6d0-4f76-ab63-a667c8c9855e-util" (OuterVolumeSpecName: "util") pod "ad769c9e-a6d0-4f76-ab63-a667c8c9855e" (UID: "ad769c9e-a6d0-4f76-ab63-a667c8c9855e"). InnerVolumeSpecName "util". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Jul 10 18:48:29.751789 ip-10-0-143-17 kubenswrapper[2585]: I0710 18:48:29.751773 2585 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/4d0d90b9-02d7-4ccf-b59e-c158c8363691-util" (OuterVolumeSpecName: "util") pod "4d0d90b9-02d7-4ccf-b59e-c158c8363691" (UID: "4d0d90b9-02d7-4ccf-b59e-c158c8363691"). InnerVolumeSpecName "util". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Jul 10 18:48:29.843679 ip-10-0-143-17 kubenswrapper[2585]: I0710 18:48:29.843659 2585 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-724lz\" (UniqueName: \"kubernetes.io/projected/ad769c9e-a6d0-4f76-ab63-a667c8c9855e-kube-api-access-724lz\") on node \"ip-10-0-143-17.ec2.internal\" DevicePath \"\"" Jul 10 18:48:29.843679 ip-10-0-143-17 kubenswrapper[2585]: I0710 18:48:29.843679 2585 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-lbwft\" (UniqueName: \"kubernetes.io/projected/4d0d90b9-02d7-4ccf-b59e-c158c8363691-kube-api-access-lbwft\") on node \"ip-10-0-143-17.ec2.internal\" DevicePath \"\"" Jul 10 18:48:29.843793 ip-10-0-143-17 kubenswrapper[2585]: I0710 18:48:29.843688 2585 reconciler_common.go:299] "Volume detached for volume \"util\" (UniqueName: \"kubernetes.io/empty-dir/ad769c9e-a6d0-4f76-ab63-a667c8c9855e-util\") on node \"ip-10-0-143-17.ec2.internal\" DevicePath \"\"" Jul 10 18:48:29.843793 ip-10-0-143-17 kubenswrapper[2585]: I0710 18:48:29.843696 2585 reconciler_common.go:299] "Volume detached for volume \"bundle\" (UniqueName: \"kubernetes.io/empty-dir/c5d7aa6a-a8cd-40fa-9d53-5a6b7b0a9c32-bundle\") on node \"ip-10-0-143-17.ec2.internal\" DevicePath \"\"" Jul 10 18:48:29.843793 ip-10-0-143-17 kubenswrapper[2585]: I0710 18:48:29.843704 2585 reconciler_common.go:299] "Volume detached for volume \"util\" (UniqueName: \"kubernetes.io/empty-dir/4d0d90b9-02d7-4ccf-b59e-c158c8363691-util\") on node \"ip-10-0-143-17.ec2.internal\" DevicePath \"\"" Jul 10 18:48:29.843793 ip-10-0-143-17 kubenswrapper[2585]: I0710 18:48:29.843711 2585 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-lmqqt\" (UniqueName: \"kubernetes.io/projected/c5d7aa6a-a8cd-40fa-9d53-5a6b7b0a9c32-kube-api-access-lmqqt\") on node \"ip-10-0-143-17.ec2.internal\" DevicePath \"\"" Jul 10 18:48:29.843793 ip-10-0-143-17 kubenswrapper[2585]: I0710 18:48:29.843719 2585 reconciler_common.go:299] "Volume detached for volume \"bundle\" (UniqueName: \"kubernetes.io/empty-dir/ad769c9e-a6d0-4f76-ab63-a667c8c9855e-bundle\") on node \"ip-10-0-143-17.ec2.internal\" DevicePath \"\"" Jul 10 18:48:29.843793 ip-10-0-143-17 kubenswrapper[2585]: I0710 18:48:29.843727 2585 reconciler_common.go:299] "Volume detached for volume \"util\" (UniqueName: \"kubernetes.io/empty-dir/c5d7aa6a-a8cd-40fa-9d53-5a6b7b0a9c32-util\") on node \"ip-10-0-143-17.ec2.internal\" DevicePath \"\"" Jul 10 18:48:29.843793 ip-10-0-143-17 kubenswrapper[2585]: I0710 18:48:29.843735 2585 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-xq4bs\" (UniqueName: \"kubernetes.io/projected/000ca62e-7689-4432-9fb4-481ca5f76723-kube-api-access-xq4bs\") on node \"ip-10-0-143-17.ec2.internal\" DevicePath \"\"" Jul 10 18:48:29.843793 ip-10-0-143-17 kubenswrapper[2585]: I0710 18:48:29.843742 2585 reconciler_common.go:299] "Volume detached for volume \"util\" (UniqueName: \"kubernetes.io/empty-dir/000ca62e-7689-4432-9fb4-481ca5f76723-util\") on node \"ip-10-0-143-17.ec2.internal\" DevicePath \"\"" Jul 10 18:48:29.843793 ip-10-0-143-17 kubenswrapper[2585]: I0710 18:48:29.843750 2585 reconciler_common.go:299] "Volume detached for volume \"bundle\" (UniqueName: \"kubernetes.io/empty-dir/000ca62e-7689-4432-9fb4-481ca5f76723-bundle\") on node \"ip-10-0-143-17.ec2.internal\" DevicePath \"\"" Jul 10 18:48:29.843793 ip-10-0-143-17 kubenswrapper[2585]: I0710 18:48:29.843756 2585 reconciler_common.go:299] "Volume detached for volume \"bundle\" (UniqueName: \"kubernetes.io/empty-dir/4d0d90b9-02d7-4ccf-b59e-c158c8363691-bundle\") on node \"ip-10-0-143-17.ec2.internal\" DevicePath \"\"" Jul 10 18:48:30.446015 ip-10-0-143-17 kubenswrapper[2585]: I0710 18:48:30.445991 2585 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-marketplace/f13fb8b568e19b60b32189bed3ab51e2b7cfe62efa490862cd57eead76vghm6" Jul 10 18:48:30.446015 ip-10-0-143-17 kubenswrapper[2585]: I0710 18:48:30.446003 2585 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-marketplace/f13fb8b568e19b60b32189bed3ab51e2b7cfe62efa490862cd57eead76vghm6" event={"ID":"c5d7aa6a-a8cd-40fa-9d53-5a6b7b0a9c32","Type":"ContainerDied","Data":"a329e7b3336221c8a9fd7d2dd6d05e9d51f28e119b58712c74f34b74313b38c9"} Jul 10 18:48:30.446203 ip-10-0-143-17 kubenswrapper[2585]: I0710 18:48:30.446049 2585 pod_container_deletor.go:80] "Container not found in pod's containers" containerID="a329e7b3336221c8a9fd7d2dd6d05e9d51f28e119b58712c74f34b74313b38c9" Jul 10 18:48:30.454384 ip-10-0-143-17 kubenswrapper[2585]: I0710 18:48:30.454350 2585 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-marketplace/299160ebe81f164e400242457c5a1580453e7d84fd8087b72a6eb317efw24cw" Jul 10 18:48:30.454524 ip-10-0-143-17 kubenswrapper[2585]: I0710 18:48:30.454349 2585 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-marketplace/299160ebe81f164e400242457c5a1580453e7d84fd8087b72a6eb317efw24cw" event={"ID":"4d0d90b9-02d7-4ccf-b59e-c158c8363691","Type":"ContainerDied","Data":"2bd052e0f6d30826503b9ca62b7246d53c8842d6f62b5266c2e8e7ad98230192"} Jul 10 18:48:30.454610 ip-10-0-143-17 kubenswrapper[2585]: I0710 18:48:30.454553 2585 pod_container_deletor.go:80] "Container not found in pod's containers" containerID="2bd052e0f6d30826503b9ca62b7246d53c8842d6f62b5266c2e8e7ad98230192" Jul 10 18:48:30.456420 ip-10-0-143-17 kubenswrapper[2585]: I0710 18:48:30.456398 2585 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-marketplace/af2a5cb5935dca7d5808b44f590e0d3d1363f94abbf5c1d8a71e95fcab8vttv" Jul 10 18:48:30.456759 ip-10-0-143-17 kubenswrapper[2585]: I0710 18:48:30.456421 2585 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-marketplace/af2a5cb5935dca7d5808b44f590e0d3d1363f94abbf5c1d8a71e95fcab8vttv" event={"ID":"ad769c9e-a6d0-4f76-ab63-a667c8c9855e","Type":"ContainerDied","Data":"b571b6b24048062a32d7368fbdf42a67b7fd704dbd8c7a010cca34de771a6ca7"} Jul 10 18:48:30.456872 ip-10-0-143-17 kubenswrapper[2585]: I0710 18:48:30.456781 2585 pod_container_deletor.go:80] "Container not found in pod's containers" containerID="b571b6b24048062a32d7368fbdf42a67b7fd704dbd8c7a010cca34de771a6ca7" Jul 10 18:48:30.458795 ip-10-0-143-17 kubenswrapper[2585]: I0710 18:48:30.458777 2585 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-marketplace/0e9e3357d40614682d1ebdb402e8ba62e1e258fb473e6f54140aafff3644hqs" Jul 10 18:48:30.458885 ip-10-0-143-17 kubenswrapper[2585]: I0710 18:48:30.458812 2585 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-marketplace/0e9e3357d40614682d1ebdb402e8ba62e1e258fb473e6f54140aafff3644hqs" event={"ID":"000ca62e-7689-4432-9fb4-481ca5f76723","Type":"ContainerDied","Data":"ee5297b4d4b3f4899df25f583328a8b1db2e5792bb7d7dd030c9c4138c8d0264"} Jul 10 18:48:30.458885 ip-10-0-143-17 kubenswrapper[2585]: I0710 18:48:30.458854 2585 pod_container_deletor.go:80] "Container not found in pod's containers" containerID="ee5297b4d4b3f4899df25f583328a8b1db2e5792bb7d7dd030c9c4138c8d0264" Jul 10 18:48:39.057290 ip-10-0-143-17 kubenswrapper[2585]: I0710 18:48:39.057255 2585 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["kuadrant-system/authorino-operator-6d85f6564-hfksn"] Jul 10 18:48:39.057788 ip-10-0-143-17 kubenswrapper[2585]: I0710 18:48:39.057639 2585 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="ad769c9e-a6d0-4f76-ab63-a667c8c9855e" containerName="util" Jul 10 18:48:39.057788 ip-10-0-143-17 kubenswrapper[2585]: I0710 18:48:39.057653 2585 state_mem.go:107] "Deleted CPUSet assignment" podUID="ad769c9e-a6d0-4f76-ab63-a667c8c9855e" containerName="util" Jul 10 18:48:39.057788 ip-10-0-143-17 kubenswrapper[2585]: I0710 18:48:39.057662 2585 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="4d0d90b9-02d7-4ccf-b59e-c158c8363691" containerName="pull" Jul 10 18:48:39.057788 ip-10-0-143-17 kubenswrapper[2585]: I0710 18:48:39.057668 2585 state_mem.go:107] "Deleted CPUSet assignment" podUID="4d0d90b9-02d7-4ccf-b59e-c158c8363691" containerName="pull" Jul 10 18:48:39.057788 ip-10-0-143-17 kubenswrapper[2585]: I0710 18:48:39.057677 2585 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="000ca62e-7689-4432-9fb4-481ca5f76723" containerName="util" Jul 10 18:48:39.057788 ip-10-0-143-17 kubenswrapper[2585]: I0710 18:48:39.057683 2585 state_mem.go:107] "Deleted CPUSet assignment" podUID="000ca62e-7689-4432-9fb4-481ca5f76723" containerName="util" Jul 10 18:48:39.057788 ip-10-0-143-17 kubenswrapper[2585]: I0710 18:48:39.057696 2585 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="000ca62e-7689-4432-9fb4-481ca5f76723" containerName="pull" Jul 10 18:48:39.057788 ip-10-0-143-17 kubenswrapper[2585]: I0710 18:48:39.057702 2585 state_mem.go:107] "Deleted CPUSet assignment" podUID="000ca62e-7689-4432-9fb4-481ca5f76723" containerName="pull" Jul 10 18:48:39.057788 ip-10-0-143-17 kubenswrapper[2585]: I0710 18:48:39.057709 2585 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="4d0d90b9-02d7-4ccf-b59e-c158c8363691" containerName="util" Jul 10 18:48:39.057788 ip-10-0-143-17 kubenswrapper[2585]: I0710 18:48:39.057714 2585 state_mem.go:107] "Deleted CPUSet assignment" podUID="4d0d90b9-02d7-4ccf-b59e-c158c8363691" containerName="util" Jul 10 18:48:39.057788 ip-10-0-143-17 kubenswrapper[2585]: I0710 18:48:39.057720 2585 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="c5d7aa6a-a8cd-40fa-9d53-5a6b7b0a9c32" containerName="pull" Jul 10 18:48:39.057788 ip-10-0-143-17 kubenswrapper[2585]: I0710 18:48:39.057725 2585 state_mem.go:107] "Deleted CPUSet assignment" podUID="c5d7aa6a-a8cd-40fa-9d53-5a6b7b0a9c32" containerName="pull" Jul 10 18:48:39.057788 ip-10-0-143-17 kubenswrapper[2585]: I0710 18:48:39.057733 2585 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="ad769c9e-a6d0-4f76-ab63-a667c8c9855e" containerName="extract" Jul 10 18:48:39.057788 ip-10-0-143-17 kubenswrapper[2585]: I0710 18:48:39.057739 2585 state_mem.go:107] "Deleted CPUSet assignment" podUID="ad769c9e-a6d0-4f76-ab63-a667c8c9855e" containerName="extract" Jul 10 18:48:39.057788 ip-10-0-143-17 kubenswrapper[2585]: I0710 18:48:39.057746 2585 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="ad769c9e-a6d0-4f76-ab63-a667c8c9855e" containerName="pull" Jul 10 18:48:39.057788 ip-10-0-143-17 kubenswrapper[2585]: I0710 18:48:39.057750 2585 state_mem.go:107] "Deleted CPUSet assignment" podUID="ad769c9e-a6d0-4f76-ab63-a667c8c9855e" containerName="pull" Jul 10 18:48:39.057788 ip-10-0-143-17 kubenswrapper[2585]: I0710 18:48:39.057757 2585 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="c5d7aa6a-a8cd-40fa-9d53-5a6b7b0a9c32" containerName="util" Jul 10 18:48:39.057788 ip-10-0-143-17 kubenswrapper[2585]: I0710 18:48:39.057762 2585 state_mem.go:107] "Deleted CPUSet assignment" podUID="c5d7aa6a-a8cd-40fa-9d53-5a6b7b0a9c32" containerName="util" Jul 10 18:48:39.057788 ip-10-0-143-17 kubenswrapper[2585]: I0710 18:48:39.057771 2585 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="000ca62e-7689-4432-9fb4-481ca5f76723" containerName="extract" Jul 10 18:48:39.057788 ip-10-0-143-17 kubenswrapper[2585]: I0710 18:48:39.057776 2585 state_mem.go:107] "Deleted CPUSet assignment" podUID="000ca62e-7689-4432-9fb4-481ca5f76723" containerName="extract" Jul 10 18:48:39.057788 ip-10-0-143-17 kubenswrapper[2585]: I0710 18:48:39.057783 2585 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="4d0d90b9-02d7-4ccf-b59e-c158c8363691" containerName="extract" Jul 10 18:48:39.057788 ip-10-0-143-17 kubenswrapper[2585]: I0710 18:48:39.057788 2585 state_mem.go:107] "Deleted CPUSet assignment" podUID="4d0d90b9-02d7-4ccf-b59e-c158c8363691" containerName="extract" Jul 10 18:48:39.057788 ip-10-0-143-17 kubenswrapper[2585]: I0710 18:48:39.057794 2585 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="c5d7aa6a-a8cd-40fa-9d53-5a6b7b0a9c32" containerName="extract" Jul 10 18:48:39.057788 ip-10-0-143-17 kubenswrapper[2585]: I0710 18:48:39.057799 2585 state_mem.go:107] "Deleted CPUSet assignment" podUID="c5d7aa6a-a8cd-40fa-9d53-5a6b7b0a9c32" containerName="extract" Jul 10 18:48:39.058560 ip-10-0-143-17 kubenswrapper[2585]: I0710 18:48:39.057859 2585 memory_manager.go:368] "RemoveStaleState removing state" podUID="c5d7aa6a-a8cd-40fa-9d53-5a6b7b0a9c32" containerName="extract" Jul 10 18:48:39.058560 ip-10-0-143-17 kubenswrapper[2585]: I0710 18:48:39.057869 2585 memory_manager.go:368] "RemoveStaleState removing state" podUID="ad769c9e-a6d0-4f76-ab63-a667c8c9855e" containerName="extract" Jul 10 18:48:39.058560 ip-10-0-143-17 kubenswrapper[2585]: I0710 18:48:39.057876 2585 memory_manager.go:368] "RemoveStaleState removing state" podUID="4d0d90b9-02d7-4ccf-b59e-c158c8363691" containerName="extract" Jul 10 18:48:39.058560 ip-10-0-143-17 kubenswrapper[2585]: I0710 18:48:39.057882 2585 memory_manager.go:368] "RemoveStaleState removing state" podUID="000ca62e-7689-4432-9fb4-481ca5f76723" containerName="extract" Jul 10 18:48:39.061973 ip-10-0-143-17 kubenswrapper[2585]: I0710 18:48:39.061956 2585 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/authorino-operator-6d85f6564-hfksn" Jul 10 18:48:39.064667 ip-10-0-143-17 kubenswrapper[2585]: I0710 18:48:39.064644 2585 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"kuadrant-system\"/\"openshift-service-ca.crt\"" Jul 10 18:48:39.064829 ip-10-0-143-17 kubenswrapper[2585]: I0710 18:48:39.064679 2585 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"kuadrant-system\"/\"kube-root-ca.crt\"" Jul 10 18:48:39.064829 ip-10-0-143-17 kubenswrapper[2585]: I0710 18:48:39.064775 2585 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"kuadrant-system\"/\"authorino-operator-dockercfg-62m2f\"" Jul 10 18:48:39.074239 ip-10-0-143-17 kubenswrapper[2585]: I0710 18:48:39.074216 2585 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/authorino-operator-6d85f6564-hfksn"] Jul 10 18:48:39.214891 ip-10-0-143-17 kubenswrapper[2585]: I0710 18:48:39.214868 2585 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-966tx\" (UniqueName: \"kubernetes.io/projected/b21d8c45-c55b-4c6b-b36b-df56774a6183-kube-api-access-966tx\") pod \"authorino-operator-6d85f6564-hfksn\" (UID: \"b21d8c45-c55b-4c6b-b36b-df56774a6183\") " pod="kuadrant-system/authorino-operator-6d85f6564-hfksn" Jul 10 18:48:39.315917 ip-10-0-143-17 kubenswrapper[2585]: I0710 18:48:39.315839 2585 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-966tx\" (UniqueName: \"kubernetes.io/projected/b21d8c45-c55b-4c6b-b36b-df56774a6183-kube-api-access-966tx\") pod \"authorino-operator-6d85f6564-hfksn\" (UID: \"b21d8c45-c55b-4c6b-b36b-df56774a6183\") " pod="kuadrant-system/authorino-operator-6d85f6564-hfksn" Jul 10 18:48:39.327790 ip-10-0-143-17 kubenswrapper[2585]: I0710 18:48:39.327771 2585 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-966tx\" (UniqueName: \"kubernetes.io/projected/b21d8c45-c55b-4c6b-b36b-df56774a6183-kube-api-access-966tx\") pod \"authorino-operator-6d85f6564-hfksn\" (UID: \"b21d8c45-c55b-4c6b-b36b-df56774a6183\") " pod="kuadrant-system/authorino-operator-6d85f6564-hfksn" Jul 10 18:48:39.372719 ip-10-0-143-17 kubenswrapper[2585]: I0710 18:48:39.372683 2585 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/authorino-operator-6d85f6564-hfksn" Jul 10 18:48:39.503759 ip-10-0-143-17 kubenswrapper[2585]: I0710 18:48:39.503723 2585 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/authorino-operator-6d85f6564-hfksn"] Jul 10 18:48:39.505888 ip-10-0-143-17 kubenswrapper[2585]: W0710 18:48:39.505859 2585 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podb21d8c45_c55b_4c6b_b36b_df56774a6183.slice/crio-0bda3653505b8cabb1269c0b85eedcdd9586d32c220ea74ae90357b84c28203d WatchSource:0}: Error finding container 0bda3653505b8cabb1269c0b85eedcdd9586d32c220ea74ae90357b84c28203d: Status 404 returned error can't find the container with id 0bda3653505b8cabb1269c0b85eedcdd9586d32c220ea74ae90357b84c28203d Jul 10 18:48:40.512708 ip-10-0-143-17 kubenswrapper[2585]: I0710 18:48:40.512665 2585 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/authorino-operator-6d85f6564-hfksn" event={"ID":"b21d8c45-c55b-4c6b-b36b-df56774a6183","Type":"ContainerStarted","Data":"0bda3653505b8cabb1269c0b85eedcdd9586d32c220ea74ae90357b84c28203d"} Jul 10 18:48:42.524429 ip-10-0-143-17 kubenswrapper[2585]: I0710 18:48:42.524399 2585 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/authorino-operator-6d85f6564-hfksn" event={"ID":"b21d8c45-c55b-4c6b-b36b-df56774a6183","Type":"ContainerStarted","Data":"286d98b18e3052abc932cd7ec7a54c2f827f27443e66779b8936ed8c65114844"} Jul 10 18:48:42.524777 ip-10-0-143-17 kubenswrapper[2585]: I0710 18:48:42.524553 2585 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="kuadrant-system/authorino-operator-6d85f6564-hfksn" Jul 10 18:48:42.548971 ip-10-0-143-17 kubenswrapper[2585]: I0710 18:48:42.548923 2585 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="kuadrant-system/authorino-operator-6d85f6564-hfksn" podStartSLOduration=1.025752743 podStartE2EDuration="3.548909719s" totalImagesPullingTime="2.523156976s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-10 18:48:39 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-10 18:48:42.546922952 +0000 UTC m=+536.004282889" watchObservedRunningTime="2026-07-10 18:48:42.548909719 +0000 UTC m=+536.006269653" Jul 10 18:48:53.524832 ip-10-0-143-17 kubenswrapper[2585]: I0710 18:48:53.524772 2585 kuberuntime_container.go:873] "Killing container with a grace period" pod="openshift-console/console-dbc48dfcb-h89sb" podUID="d6eddcf2-4883-4ab6-9acf-65ee02d662f1" containerName="console" containerID="cri-o://9d8d7e7d5e693ce04df323561c2c0ad9f8c68a8b4a1506bc12423ca52b6a9ce2" gracePeriod=15 Jul 10 18:48:53.531427 ip-10-0-143-17 kubenswrapper[2585]: I0710 18:48:53.531402 2585 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="kuadrant-system/authorino-operator-6d85f6564-hfksn" Jul 10 18:48:53.776799 ip-10-0-143-17 kubenswrapper[2585]: I0710 18:48:53.776749 2585 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-console/console-dbc48dfcb-h89sb" Jul 10 18:48:53.833395 ip-10-0-143-17 kubenswrapper[2585]: I0710 18:48:53.833371 2585 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"console-oauth-config\" (UniqueName: \"kubernetes.io/secret/d6eddcf2-4883-4ab6-9acf-65ee02d662f1-console-oauth-config\") pod \"d6eddcf2-4883-4ab6-9acf-65ee02d662f1\" (UID: \"d6eddcf2-4883-4ab6-9acf-65ee02d662f1\") " Jul 10 18:48:53.833521 ip-10-0-143-17 kubenswrapper[2585]: I0710 18:48:53.833408 2585 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"console-serving-cert\" (UniqueName: \"kubernetes.io/secret/d6eddcf2-4883-4ab6-9acf-65ee02d662f1-console-serving-cert\") pod \"d6eddcf2-4883-4ab6-9acf-65ee02d662f1\" (UID: \"d6eddcf2-4883-4ab6-9acf-65ee02d662f1\") " Jul 10 18:48:53.833521 ip-10-0-143-17 kubenswrapper[2585]: I0710 18:48:53.833466 2585 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"service-ca\" (UniqueName: \"kubernetes.io/configmap/d6eddcf2-4883-4ab6-9acf-65ee02d662f1-service-ca\") pod \"d6eddcf2-4883-4ab6-9acf-65ee02d662f1\" (UID: \"d6eddcf2-4883-4ab6-9acf-65ee02d662f1\") " Jul 10 18:48:53.833521 ip-10-0-143-17 kubenswrapper[2585]: I0710 18:48:53.833490 2585 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"console-config\" (UniqueName: \"kubernetes.io/configmap/d6eddcf2-4883-4ab6-9acf-65ee02d662f1-console-config\") pod \"d6eddcf2-4883-4ab6-9acf-65ee02d662f1\" (UID: \"d6eddcf2-4883-4ab6-9acf-65ee02d662f1\") " Jul 10 18:48:53.833521 ip-10-0-143-17 kubenswrapper[2585]: I0710 18:48:53.833504 2585 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/d6eddcf2-4883-4ab6-9acf-65ee02d662f1-trusted-ca-bundle\") pod \"d6eddcf2-4883-4ab6-9acf-65ee02d662f1\" (UID: \"d6eddcf2-4883-4ab6-9acf-65ee02d662f1\") " Jul 10 18:48:53.833764 ip-10-0-143-17 kubenswrapper[2585]: I0710 18:48:53.833522 2585 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-stq2r\" (UniqueName: \"kubernetes.io/projected/d6eddcf2-4883-4ab6-9acf-65ee02d662f1-kube-api-access-stq2r\") pod \"d6eddcf2-4883-4ab6-9acf-65ee02d662f1\" (UID: \"d6eddcf2-4883-4ab6-9acf-65ee02d662f1\") " Jul 10 18:48:53.833764 ip-10-0-143-17 kubenswrapper[2585]: I0710 18:48:53.833567 2585 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"oauth-serving-cert\" (UniqueName: \"kubernetes.io/configmap/d6eddcf2-4883-4ab6-9acf-65ee02d662f1-oauth-serving-cert\") pod \"d6eddcf2-4883-4ab6-9acf-65ee02d662f1\" (UID: \"d6eddcf2-4883-4ab6-9acf-65ee02d662f1\") " Jul 10 18:48:53.833954 ip-10-0-143-17 kubenswrapper[2585]: I0710 18:48:53.833927 2585 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/d6eddcf2-4883-4ab6-9acf-65ee02d662f1-console-config" (OuterVolumeSpecName: "console-config") pod "d6eddcf2-4883-4ab6-9acf-65ee02d662f1" (UID: "d6eddcf2-4883-4ab6-9acf-65ee02d662f1"). InnerVolumeSpecName "console-config". PluginName "kubernetes.io/configmap", VolumeGIDValue "" Jul 10 18:48:53.834023 ip-10-0-143-17 kubenswrapper[2585]: I0710 18:48:53.833944 2585 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/d6eddcf2-4883-4ab6-9acf-65ee02d662f1-service-ca" (OuterVolumeSpecName: "service-ca") pod "d6eddcf2-4883-4ab6-9acf-65ee02d662f1" (UID: "d6eddcf2-4883-4ab6-9acf-65ee02d662f1"). InnerVolumeSpecName "service-ca". PluginName "kubernetes.io/configmap", VolumeGIDValue "" Jul 10 18:48:53.834023 ip-10-0-143-17 kubenswrapper[2585]: I0710 18:48:53.833947 2585 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/d6eddcf2-4883-4ab6-9acf-65ee02d662f1-trusted-ca-bundle" (OuterVolumeSpecName: "trusted-ca-bundle") pod "d6eddcf2-4883-4ab6-9acf-65ee02d662f1" (UID: "d6eddcf2-4883-4ab6-9acf-65ee02d662f1"). InnerVolumeSpecName "trusted-ca-bundle". PluginName "kubernetes.io/configmap", VolumeGIDValue "" Jul 10 18:48:53.834023 ip-10-0-143-17 kubenswrapper[2585]: I0710 18:48:53.833991 2585 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/d6eddcf2-4883-4ab6-9acf-65ee02d662f1-oauth-serving-cert" (OuterVolumeSpecName: "oauth-serving-cert") pod "d6eddcf2-4883-4ab6-9acf-65ee02d662f1" (UID: "d6eddcf2-4883-4ab6-9acf-65ee02d662f1"). InnerVolumeSpecName "oauth-serving-cert". PluginName "kubernetes.io/configmap", VolumeGIDValue "" Jul 10 18:48:53.835491 ip-10-0-143-17 kubenswrapper[2585]: I0710 18:48:53.835469 2585 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/d6eddcf2-4883-4ab6-9acf-65ee02d662f1-console-oauth-config" (OuterVolumeSpecName: "console-oauth-config") pod "d6eddcf2-4883-4ab6-9acf-65ee02d662f1" (UID: "d6eddcf2-4883-4ab6-9acf-65ee02d662f1"). InnerVolumeSpecName "console-oauth-config". PluginName "kubernetes.io/secret", VolumeGIDValue "" Jul 10 18:48:53.835583 ip-10-0-143-17 kubenswrapper[2585]: I0710 18:48:53.835500 2585 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/d6eddcf2-4883-4ab6-9acf-65ee02d662f1-console-serving-cert" (OuterVolumeSpecName: "console-serving-cert") pod "d6eddcf2-4883-4ab6-9acf-65ee02d662f1" (UID: "d6eddcf2-4883-4ab6-9acf-65ee02d662f1"). InnerVolumeSpecName "console-serving-cert". PluginName "kubernetes.io/secret", VolumeGIDValue "" Jul 10 18:48:53.835743 ip-10-0-143-17 kubenswrapper[2585]: I0710 18:48:53.835726 2585 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/d6eddcf2-4883-4ab6-9acf-65ee02d662f1-kube-api-access-stq2r" (OuterVolumeSpecName: "kube-api-access-stq2r") pod "d6eddcf2-4883-4ab6-9acf-65ee02d662f1" (UID: "d6eddcf2-4883-4ab6-9acf-65ee02d662f1"). InnerVolumeSpecName "kube-api-access-stq2r". PluginName "kubernetes.io/projected", VolumeGIDValue "" Jul 10 18:48:53.934900 ip-10-0-143-17 kubenswrapper[2585]: I0710 18:48:53.934881 2585 reconciler_common.go:299] "Volume detached for volume \"service-ca\" (UniqueName: \"kubernetes.io/configmap/d6eddcf2-4883-4ab6-9acf-65ee02d662f1-service-ca\") on node \"ip-10-0-143-17.ec2.internal\" DevicePath \"\"" Jul 10 18:48:53.934900 ip-10-0-143-17 kubenswrapper[2585]: I0710 18:48:53.934901 2585 reconciler_common.go:299] "Volume detached for volume \"console-config\" (UniqueName: \"kubernetes.io/configmap/d6eddcf2-4883-4ab6-9acf-65ee02d662f1-console-config\") on node \"ip-10-0-143-17.ec2.internal\" DevicePath \"\"" Jul 10 18:48:53.935018 ip-10-0-143-17 kubenswrapper[2585]: I0710 18:48:53.934910 2585 reconciler_common.go:299] "Volume detached for volume \"trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/d6eddcf2-4883-4ab6-9acf-65ee02d662f1-trusted-ca-bundle\") on node \"ip-10-0-143-17.ec2.internal\" DevicePath \"\"" Jul 10 18:48:53.935018 ip-10-0-143-17 kubenswrapper[2585]: I0710 18:48:53.934919 2585 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-stq2r\" (UniqueName: \"kubernetes.io/projected/d6eddcf2-4883-4ab6-9acf-65ee02d662f1-kube-api-access-stq2r\") on node \"ip-10-0-143-17.ec2.internal\" DevicePath \"\"" Jul 10 18:48:53.935018 ip-10-0-143-17 kubenswrapper[2585]: I0710 18:48:53.934928 2585 reconciler_common.go:299] "Volume detached for volume \"oauth-serving-cert\" (UniqueName: \"kubernetes.io/configmap/d6eddcf2-4883-4ab6-9acf-65ee02d662f1-oauth-serving-cert\") on node \"ip-10-0-143-17.ec2.internal\" DevicePath \"\"" Jul 10 18:48:53.935018 ip-10-0-143-17 kubenswrapper[2585]: I0710 18:48:53.934936 2585 reconciler_common.go:299] "Volume detached for volume \"console-oauth-config\" (UniqueName: \"kubernetes.io/secret/d6eddcf2-4883-4ab6-9acf-65ee02d662f1-console-oauth-config\") on node \"ip-10-0-143-17.ec2.internal\" DevicePath \"\"" Jul 10 18:48:53.935018 ip-10-0-143-17 kubenswrapper[2585]: I0710 18:48:53.934944 2585 reconciler_common.go:299] "Volume detached for volume \"console-serving-cert\" (UniqueName: \"kubernetes.io/secret/d6eddcf2-4883-4ab6-9acf-65ee02d662f1-console-serving-cert\") on node \"ip-10-0-143-17.ec2.internal\" DevicePath \"\"" Jul 10 18:48:54.583026 ip-10-0-143-17 kubenswrapper[2585]: I0710 18:48:54.582998 2585 generic.go:356] "Generic (PLEG): container finished" podID="d6eddcf2-4883-4ab6-9acf-65ee02d662f1" containerID="9d8d7e7d5e693ce04df323561c2c0ad9f8c68a8b4a1506bc12423ca52b6a9ce2" exitCode=0 Jul 10 18:48:54.583395 ip-10-0-143-17 kubenswrapper[2585]: I0710 18:48:54.583050 2585 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-console/console-dbc48dfcb-h89sb" Jul 10 18:48:54.583395 ip-10-0-143-17 kubenswrapper[2585]: I0710 18:48:54.583086 2585 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-console/console-dbc48dfcb-h89sb" event={"ID":"d6eddcf2-4883-4ab6-9acf-65ee02d662f1","Type":"ContainerDied","Data":"9d8d7e7d5e693ce04df323561c2c0ad9f8c68a8b4a1506bc12423ca52b6a9ce2"} Jul 10 18:48:54.583395 ip-10-0-143-17 kubenswrapper[2585]: I0710 18:48:54.583139 2585 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-console/console-dbc48dfcb-h89sb" event={"ID":"d6eddcf2-4883-4ab6-9acf-65ee02d662f1","Type":"ContainerDied","Data":"68705cc94e542a1a5d17bf827dcc8ebd3144e3399ded1fd9e1820b20d29edcaa"} Jul 10 18:48:54.583395 ip-10-0-143-17 kubenswrapper[2585]: I0710 18:48:54.583160 2585 scope.go:117] "RemoveContainer" containerID="9d8d7e7d5e693ce04df323561c2c0ad9f8c68a8b4a1506bc12423ca52b6a9ce2" Jul 10 18:48:54.592681 ip-10-0-143-17 kubenswrapper[2585]: I0710 18:48:54.592665 2585 scope.go:117] "RemoveContainer" containerID="9d8d7e7d5e693ce04df323561c2c0ad9f8c68a8b4a1506bc12423ca52b6a9ce2" Jul 10 18:48:54.592924 ip-10-0-143-17 kubenswrapper[2585]: E0710 18:48:54.592906 2585 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"9d8d7e7d5e693ce04df323561c2c0ad9f8c68a8b4a1506bc12423ca52b6a9ce2\": container with ID starting with 9d8d7e7d5e693ce04df323561c2c0ad9f8c68a8b4a1506bc12423ca52b6a9ce2 not found: ID does not exist" containerID="9d8d7e7d5e693ce04df323561c2c0ad9f8c68a8b4a1506bc12423ca52b6a9ce2" Jul 10 18:48:54.592983 ip-10-0-143-17 kubenswrapper[2585]: I0710 18:48:54.592937 2585 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"9d8d7e7d5e693ce04df323561c2c0ad9f8c68a8b4a1506bc12423ca52b6a9ce2"} err="failed to get container status \"9d8d7e7d5e693ce04df323561c2c0ad9f8c68a8b4a1506bc12423ca52b6a9ce2\": rpc error: code = NotFound desc = could not find container \"9d8d7e7d5e693ce04df323561c2c0ad9f8c68a8b4a1506bc12423ca52b6a9ce2\": container with ID starting with 9d8d7e7d5e693ce04df323561c2c0ad9f8c68a8b4a1506bc12423ca52b6a9ce2 not found: ID does not exist" Jul 10 18:48:54.610969 ip-10-0-143-17 kubenswrapper[2585]: I0710 18:48:54.610943 2585 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-console/console-dbc48dfcb-h89sb"] Jul 10 18:48:54.616618 ip-10-0-143-17 kubenswrapper[2585]: I0710 18:48:54.616596 2585 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["openshift-console/console-dbc48dfcb-h89sb"] Jul 10 18:48:55.217176 ip-10-0-143-17 kubenswrapper[2585]: I0710 18:48:55.217144 2585 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="d6eddcf2-4883-4ab6-9acf-65ee02d662f1" path="/var/lib/kubelet/pods/d6eddcf2-4883-4ab6-9acf-65ee02d662f1/volumes" Jul 10 18:49:36.327348 ip-10-0-143-17 kubenswrapper[2585]: I0710 18:49:36.327300 2585 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["kuadrant-system/limitador-limitador-69574b596d-x7mgm"] Jul 10 18:49:36.327799 ip-10-0-143-17 kubenswrapper[2585]: I0710 18:49:36.327669 2585 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="d6eddcf2-4883-4ab6-9acf-65ee02d662f1" containerName="console" Jul 10 18:49:36.327799 ip-10-0-143-17 kubenswrapper[2585]: I0710 18:49:36.327681 2585 state_mem.go:107] "Deleted CPUSet assignment" podUID="d6eddcf2-4883-4ab6-9acf-65ee02d662f1" containerName="console" Jul 10 18:49:36.327799 ip-10-0-143-17 kubenswrapper[2585]: I0710 18:49:36.327753 2585 memory_manager.go:368] "RemoveStaleState removing state" podUID="d6eddcf2-4883-4ab6-9acf-65ee02d662f1" containerName="console" Jul 10 18:49:36.334256 ip-10-0-143-17 kubenswrapper[2585]: I0710 18:49:36.334234 2585 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/limitador-limitador-69574b596d-x7mgm" Jul 10 18:49:36.337185 ip-10-0-143-17 kubenswrapper[2585]: I0710 18:49:36.337156 2585 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"kuadrant-system\"/\"limitador-limits-config-limitador\"" Jul 10 18:49:36.337848 ip-10-0-143-17 kubenswrapper[2585]: I0710 18:49:36.337826 2585 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"kuadrant-system\"/\"default-dockercfg-npkqq\"" Jul 10 18:49:36.339757 ip-10-0-143-17 kubenswrapper[2585]: I0710 18:49:36.339727 2585 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-69574b596d-x7mgm"] Jul 10 18:49:36.370636 ip-10-0-143-17 kubenswrapper[2585]: I0710 18:49:36.370606 2585 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-69574b596d-x7mgm"] Jul 10 18:49:36.447718 ip-10-0-143-17 kubenswrapper[2585]: I0710 18:49:36.447679 2585 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"config-file\" (UniqueName: \"kubernetes.io/configmap/2db6ff8a-77c3-4aea-a4db-057b68a32265-config-file\") pod \"limitador-limitador-69574b596d-x7mgm\" (UID: \"2db6ff8a-77c3-4aea-a4db-057b68a32265\") " pod="kuadrant-system/limitador-limitador-69574b596d-x7mgm" Jul 10 18:49:36.447894 ip-10-0-143-17 kubenswrapper[2585]: I0710 18:49:36.447736 2585 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-vjgf2\" (UniqueName: \"kubernetes.io/projected/2db6ff8a-77c3-4aea-a4db-057b68a32265-kube-api-access-vjgf2\") pod \"limitador-limitador-69574b596d-x7mgm\" (UID: \"2db6ff8a-77c3-4aea-a4db-057b68a32265\") " pod="kuadrant-system/limitador-limitador-69574b596d-x7mgm" Jul 10 18:49:36.548912 ip-10-0-143-17 kubenswrapper[2585]: I0710 18:49:36.548878 2585 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-vjgf2\" (UniqueName: \"kubernetes.io/projected/2db6ff8a-77c3-4aea-a4db-057b68a32265-kube-api-access-vjgf2\") pod \"limitador-limitador-69574b596d-x7mgm\" (UID: \"2db6ff8a-77c3-4aea-a4db-057b68a32265\") " pod="kuadrant-system/limitador-limitador-69574b596d-x7mgm" Jul 10 18:49:36.549054 ip-10-0-143-17 kubenswrapper[2585]: I0710 18:49:36.548960 2585 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"config-file\" (UniqueName: \"kubernetes.io/configmap/2db6ff8a-77c3-4aea-a4db-057b68a32265-config-file\") pod \"limitador-limitador-69574b596d-x7mgm\" (UID: \"2db6ff8a-77c3-4aea-a4db-057b68a32265\") " pod="kuadrant-system/limitador-limitador-69574b596d-x7mgm" Jul 10 18:49:36.549601 ip-10-0-143-17 kubenswrapper[2585]: I0710 18:49:36.549581 2585 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"config-file\" (UniqueName: \"kubernetes.io/configmap/2db6ff8a-77c3-4aea-a4db-057b68a32265-config-file\") pod \"limitador-limitador-69574b596d-x7mgm\" (UID: \"2db6ff8a-77c3-4aea-a4db-057b68a32265\") " pod="kuadrant-system/limitador-limitador-69574b596d-x7mgm" Jul 10 18:49:36.557675 ip-10-0-143-17 kubenswrapper[2585]: I0710 18:49:36.557654 2585 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-vjgf2\" (UniqueName: \"kubernetes.io/projected/2db6ff8a-77c3-4aea-a4db-057b68a32265-kube-api-access-vjgf2\") pod \"limitador-limitador-69574b596d-x7mgm\" (UID: \"2db6ff8a-77c3-4aea-a4db-057b68a32265\") " pod="kuadrant-system/limitador-limitador-69574b596d-x7mgm" Jul 10 18:49:36.645444 ip-10-0-143-17 kubenswrapper[2585]: I0710 18:49:36.645373 2585 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/limitador-limitador-69574b596d-x7mgm" Jul 10 18:49:36.787951 ip-10-0-143-17 kubenswrapper[2585]: I0710 18:49:36.787927 2585 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-69574b596d-x7mgm"] Jul 10 18:49:36.790052 ip-10-0-143-17 kubenswrapper[2585]: W0710 18:49:36.790011 2585 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod2db6ff8a_77c3_4aea_a4db_057b68a32265.slice/crio-67e771338df243fb6dbf559b71affc1d059088d0857c33a8848c4b1268bb0913 WatchSource:0}: Error finding container 67e771338df243fb6dbf559b71affc1d059088d0857c33a8848c4b1268bb0913: Status 404 returned error can't find the container with id 67e771338df243fb6dbf559b71affc1d059088d0857c33a8848c4b1268bb0913 Jul 10 18:49:37.797370 ip-10-0-143-17 kubenswrapper[2585]: I0710 18:49:37.797326 2585 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/limitador-limitador-69574b596d-x7mgm" event={"ID":"2db6ff8a-77c3-4aea-a4db-057b68a32265","Type":"ContainerStarted","Data":"67e771338df243fb6dbf559b71affc1d059088d0857c33a8848c4b1268bb0913"} Jul 10 18:49:40.816626 ip-10-0-143-17 kubenswrapper[2585]: I0710 18:49:40.816592 2585 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/limitador-limitador-69574b596d-x7mgm" event={"ID":"2db6ff8a-77c3-4aea-a4db-057b68a32265","Type":"ContainerStarted","Data":"388592a89cadf6e6b6e194132a33d825f4a573d9794c91e181b7117c10fad7a7"} Jul 10 18:49:40.817019 ip-10-0-143-17 kubenswrapper[2585]: I0710 18:49:40.816700 2585 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="kuadrant-system/limitador-limitador-69574b596d-x7mgm" Jul 10 18:49:40.835503 ip-10-0-143-17 kubenswrapper[2585]: I0710 18:49:40.835455 2585 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="kuadrant-system/limitador-limitador-69574b596d-x7mgm" podStartSLOduration=1.28082071 podStartE2EDuration="4.83544237s" totalImagesPullingTime="3.55462166s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-10 18:49:36 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-10 18:49:40.833919153 +0000 UTC m=+594.291279081" watchObservedRunningTime="2026-07-10 18:49:40.83544237 +0000 UTC m=+594.292802330" Jul 10 18:49:46.987514 ip-10-0-143-17 kubenswrapper[2585]: I0710 18:49:46.987486 2585 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ovn-kubernetes_ovnkube-node-xg6bm_d863cfea-f2f3-4404-b62a-d41785525f3f/ovn-acl-logging/0.log" Jul 10 18:49:46.989313 ip-10-0-143-17 kubenswrapper[2585]: I0710 18:49:46.989290 2585 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ovn-kubernetes_ovnkube-node-xg6bm_d863cfea-f2f3-4404-b62a-d41785525f3f/ovn-acl-logging/0.log" Jul 10 18:49:51.822919 ip-10-0-143-17 kubenswrapper[2585]: I0710 18:49:51.822888 2585 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="kuadrant-system/limitador-limitador-69574b596d-x7mgm" Jul 10 18:50:11.833860 ip-10-0-143-17 kubenswrapper[2585]: I0710 18:50:11.833784 2585 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-ingress/istiod-openshift-gateway-94bb8fbfd-6h99s"] Jul 10 18:50:11.836851 ip-10-0-143-17 kubenswrapper[2585]: I0710 18:50:11.836835 2585 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/istiod-openshift-gateway-94bb8fbfd-6h99s" Jul 10 18:50:11.851334 ip-10-0-143-17 kubenswrapper[2585]: I0710 18:50:11.851306 2585 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/istiod-openshift-gateway-94bb8fbfd-6h99s"] Jul 10 18:50:11.944585 ip-10-0-143-17 kubenswrapper[2585]: I0710 18:50:11.944506 2585 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"local-certs\" (UniqueName: \"kubernetes.io/empty-dir/c2545730-3e12-4efe-b88c-6300fc5510a5-local-certs\") pod \"istiod-openshift-gateway-94bb8fbfd-6h99s\" (UID: \"c2545730-3e12-4efe-b88c-6300fc5510a5\") " pod="openshift-ingress/istiod-openshift-gateway-94bb8fbfd-6h99s" Jul 10 18:50:11.944585 ip-10-0-143-17 kubenswrapper[2585]: I0710 18:50:11.944564 2585 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-csr-ca-configmap\" (UniqueName: \"kubernetes.io/configmap/c2545730-3e12-4efe-b88c-6300fc5510a5-istio-csr-ca-configmap\") pod \"istiod-openshift-gateway-94bb8fbfd-6h99s\" (UID: \"c2545730-3e12-4efe-b88c-6300fc5510a5\") " pod="openshift-ingress/istiod-openshift-gateway-94bb8fbfd-6h99s" Jul 10 18:50:11.944786 ip-10-0-143-17 kubenswrapper[2585]: I0710 18:50:11.944630 2585 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/c2545730-3e12-4efe-b88c-6300fc5510a5-istio-token\") pod \"istiod-openshift-gateway-94bb8fbfd-6h99s\" (UID: \"c2545730-3e12-4efe-b88c-6300fc5510a5\") " pod="openshift-ingress/istiod-openshift-gateway-94bb8fbfd-6h99s" Jul 10 18:50:11.944786 ip-10-0-143-17 kubenswrapper[2585]: I0710 18:50:11.944680 2585 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-jlhxv\" (UniqueName: \"kubernetes.io/projected/c2545730-3e12-4efe-b88c-6300fc5510a5-kube-api-access-jlhxv\") pod \"istiod-openshift-gateway-94bb8fbfd-6h99s\" (UID: \"c2545730-3e12-4efe-b88c-6300fc5510a5\") " pod="openshift-ingress/istiod-openshift-gateway-94bb8fbfd-6h99s" Jul 10 18:50:11.944786 ip-10-0-143-17 kubenswrapper[2585]: I0710 18:50:11.944703 2585 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-kubeconfig\" (UniqueName: \"kubernetes.io/secret/c2545730-3e12-4efe-b88c-6300fc5510a5-istio-kubeconfig\") pod \"istiod-openshift-gateway-94bb8fbfd-6h99s\" (UID: \"c2545730-3e12-4efe-b88c-6300fc5510a5\") " pod="openshift-ingress/istiod-openshift-gateway-94bb8fbfd-6h99s" Jul 10 18:50:11.944786 ip-10-0-143-17 kubenswrapper[2585]: I0710 18:50:11.944754 2585 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"cacerts\" (UniqueName: \"kubernetes.io/secret/c2545730-3e12-4efe-b88c-6300fc5510a5-cacerts\") pod \"istiod-openshift-gateway-94bb8fbfd-6h99s\" (UID: \"c2545730-3e12-4efe-b88c-6300fc5510a5\") " pod="openshift-ingress/istiod-openshift-gateway-94bb8fbfd-6h99s" Jul 10 18:50:11.944916 ip-10-0-143-17 kubenswrapper[2585]: I0710 18:50:11.944787 2585 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-csr-dns-cert\" (UniqueName: \"kubernetes.io/secret/c2545730-3e12-4efe-b88c-6300fc5510a5-istio-csr-dns-cert\") pod \"istiod-openshift-gateway-94bb8fbfd-6h99s\" (UID: \"c2545730-3e12-4efe-b88c-6300fc5510a5\") " pod="openshift-ingress/istiod-openshift-gateway-94bb8fbfd-6h99s" Jul 10 18:50:12.045339 ip-10-0-143-17 kubenswrapper[2585]: I0710 18:50:12.045302 2585 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"cacerts\" (UniqueName: \"kubernetes.io/secret/c2545730-3e12-4efe-b88c-6300fc5510a5-cacerts\") pod \"istiod-openshift-gateway-94bb8fbfd-6h99s\" (UID: \"c2545730-3e12-4efe-b88c-6300fc5510a5\") " pod="openshift-ingress/istiod-openshift-gateway-94bb8fbfd-6h99s" Jul 10 18:50:12.045531 ip-10-0-143-17 kubenswrapper[2585]: I0710 18:50:12.045359 2585 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-csr-dns-cert\" (UniqueName: \"kubernetes.io/secret/c2545730-3e12-4efe-b88c-6300fc5510a5-istio-csr-dns-cert\") pod \"istiod-openshift-gateway-94bb8fbfd-6h99s\" (UID: \"c2545730-3e12-4efe-b88c-6300fc5510a5\") " pod="openshift-ingress/istiod-openshift-gateway-94bb8fbfd-6h99s" Jul 10 18:50:12.045531 ip-10-0-143-17 kubenswrapper[2585]: I0710 18:50:12.045394 2585 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"local-certs\" (UniqueName: \"kubernetes.io/empty-dir/c2545730-3e12-4efe-b88c-6300fc5510a5-local-certs\") pod \"istiod-openshift-gateway-94bb8fbfd-6h99s\" (UID: \"c2545730-3e12-4efe-b88c-6300fc5510a5\") " pod="openshift-ingress/istiod-openshift-gateway-94bb8fbfd-6h99s" Jul 10 18:50:12.045531 ip-10-0-143-17 kubenswrapper[2585]: I0710 18:50:12.045414 2585 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-csr-ca-configmap\" (UniqueName: \"kubernetes.io/configmap/c2545730-3e12-4efe-b88c-6300fc5510a5-istio-csr-ca-configmap\") pod \"istiod-openshift-gateway-94bb8fbfd-6h99s\" (UID: \"c2545730-3e12-4efe-b88c-6300fc5510a5\") " pod="openshift-ingress/istiod-openshift-gateway-94bb8fbfd-6h99s" Jul 10 18:50:12.045531 ip-10-0-143-17 kubenswrapper[2585]: I0710 18:50:12.045435 2585 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/c2545730-3e12-4efe-b88c-6300fc5510a5-istio-token\") pod \"istiod-openshift-gateway-94bb8fbfd-6h99s\" (UID: \"c2545730-3e12-4efe-b88c-6300fc5510a5\") " pod="openshift-ingress/istiod-openshift-gateway-94bb8fbfd-6h99s" Jul 10 18:50:12.045531 ip-10-0-143-17 kubenswrapper[2585]: I0710 18:50:12.045464 2585 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-jlhxv\" (UniqueName: \"kubernetes.io/projected/c2545730-3e12-4efe-b88c-6300fc5510a5-kube-api-access-jlhxv\") pod \"istiod-openshift-gateway-94bb8fbfd-6h99s\" (UID: \"c2545730-3e12-4efe-b88c-6300fc5510a5\") " pod="openshift-ingress/istiod-openshift-gateway-94bb8fbfd-6h99s" Jul 10 18:50:12.045531 ip-10-0-143-17 kubenswrapper[2585]: I0710 18:50:12.045489 2585 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-kubeconfig\" (UniqueName: \"kubernetes.io/secret/c2545730-3e12-4efe-b88c-6300fc5510a5-istio-kubeconfig\") pod \"istiod-openshift-gateway-94bb8fbfd-6h99s\" (UID: \"c2545730-3e12-4efe-b88c-6300fc5510a5\") " pod="openshift-ingress/istiod-openshift-gateway-94bb8fbfd-6h99s" Jul 10 18:50:12.046314 ip-10-0-143-17 kubenswrapper[2585]: I0710 18:50:12.046278 2585 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istio-csr-ca-configmap\" (UniqueName: \"kubernetes.io/configmap/c2545730-3e12-4efe-b88c-6300fc5510a5-istio-csr-ca-configmap\") pod \"istiod-openshift-gateway-94bb8fbfd-6h99s\" (UID: \"c2545730-3e12-4efe-b88c-6300fc5510a5\") " pod="openshift-ingress/istiod-openshift-gateway-94bb8fbfd-6h99s" Jul 10 18:50:12.047861 ip-10-0-143-17 kubenswrapper[2585]: I0710 18:50:12.047827 2585 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istio-csr-dns-cert\" (UniqueName: \"kubernetes.io/secret/c2545730-3e12-4efe-b88c-6300fc5510a5-istio-csr-dns-cert\") pod \"istiod-openshift-gateway-94bb8fbfd-6h99s\" (UID: \"c2545730-3e12-4efe-b88c-6300fc5510a5\") " pod="openshift-ingress/istiod-openshift-gateway-94bb8fbfd-6h99s" Jul 10 18:50:12.047861 ip-10-0-143-17 kubenswrapper[2585]: I0710 18:50:12.047846 2585 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"local-certs\" (UniqueName: \"kubernetes.io/empty-dir/c2545730-3e12-4efe-b88c-6300fc5510a5-local-certs\") pod \"istiod-openshift-gateway-94bb8fbfd-6h99s\" (UID: \"c2545730-3e12-4efe-b88c-6300fc5510a5\") " pod="openshift-ingress/istiod-openshift-gateway-94bb8fbfd-6h99s" Jul 10 18:50:12.047985 ip-10-0-143-17 kubenswrapper[2585]: I0710 18:50:12.047887 2585 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"cacerts\" (UniqueName: \"kubernetes.io/secret/c2545730-3e12-4efe-b88c-6300fc5510a5-cacerts\") pod \"istiod-openshift-gateway-94bb8fbfd-6h99s\" (UID: \"c2545730-3e12-4efe-b88c-6300fc5510a5\") " pod="openshift-ingress/istiod-openshift-gateway-94bb8fbfd-6h99s" Jul 10 18:50:12.048060 ip-10-0-143-17 kubenswrapper[2585]: I0710 18:50:12.048035 2585 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istio-kubeconfig\" (UniqueName: \"kubernetes.io/secret/c2545730-3e12-4efe-b88c-6300fc5510a5-istio-kubeconfig\") pod \"istiod-openshift-gateway-94bb8fbfd-6h99s\" (UID: \"c2545730-3e12-4efe-b88c-6300fc5510a5\") " pod="openshift-ingress/istiod-openshift-gateway-94bb8fbfd-6h99s" Jul 10 18:50:12.066073 ip-10-0-143-17 kubenswrapper[2585]: I0710 18:50:12.066041 2585 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-jlhxv\" (UniqueName: \"kubernetes.io/projected/c2545730-3e12-4efe-b88c-6300fc5510a5-kube-api-access-jlhxv\") pod \"istiod-openshift-gateway-94bb8fbfd-6h99s\" (UID: \"c2545730-3e12-4efe-b88c-6300fc5510a5\") " pod="openshift-ingress/istiod-openshift-gateway-94bb8fbfd-6h99s" Jul 10 18:50:12.066555 ip-10-0-143-17 kubenswrapper[2585]: I0710 18:50:12.066516 2585 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/c2545730-3e12-4efe-b88c-6300fc5510a5-istio-token\") pod \"istiod-openshift-gateway-94bb8fbfd-6h99s\" (UID: \"c2545730-3e12-4efe-b88c-6300fc5510a5\") " pod="openshift-ingress/istiod-openshift-gateway-94bb8fbfd-6h99s" Jul 10 18:50:12.145956 ip-10-0-143-17 kubenswrapper[2585]: I0710 18:50:12.145868 2585 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/istiod-openshift-gateway-94bb8fbfd-6h99s" Jul 10 18:50:12.289918 ip-10-0-143-17 kubenswrapper[2585]: I0710 18:50:12.289894 2585 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/istiod-openshift-gateway-94bb8fbfd-6h99s"] Jul 10 18:50:12.291302 ip-10-0-143-17 kubenswrapper[2585]: W0710 18:50:12.291267 2585 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podc2545730_3e12_4efe_b88c_6300fc5510a5.slice/crio-ae0df155ab792bbdbd090277a2e5ae37052fd98a470ac0bd3d493475c575f30e WatchSource:0}: Error finding container ae0df155ab792bbdbd090277a2e5ae37052fd98a470ac0bd3d493475c575f30e: Status 404 returned error can't find the container with id ae0df155ab792bbdbd090277a2e5ae37052fd98a470ac0bd3d493475c575f30e Jul 10 18:50:12.293402 ip-10-0-143-17 kubenswrapper[2585]: I0710 18:50:12.293369 2585 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236224Ki","pods":"250"} Jul 10 18:50:12.293488 ip-10-0-143-17 kubenswrapper[2585]: I0710 18:50:12.293433 2585 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236224Ki","pods":"250"} Jul 10 18:50:12.976353 ip-10-0-143-17 kubenswrapper[2585]: I0710 18:50:12.976313 2585 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/istiod-openshift-gateway-94bb8fbfd-6h99s" event={"ID":"c2545730-3e12-4efe-b88c-6300fc5510a5","Type":"ContainerStarted","Data":"3f6a7c224613f5d3a88831c2e17541612c2e451cb2eae2cb2c0dd02ce4662e3e"} Jul 10 18:50:12.976754 ip-10-0-143-17 kubenswrapper[2585]: I0710 18:50:12.976367 2585 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/istiod-openshift-gateway-94bb8fbfd-6h99s" event={"ID":"c2545730-3e12-4efe-b88c-6300fc5510a5","Type":"ContainerStarted","Data":"ae0df155ab792bbdbd090277a2e5ae37052fd98a470ac0bd3d493475c575f30e"} Jul 10 18:50:12.976754 ip-10-0-143-17 kubenswrapper[2585]: I0710 18:50:12.976395 2585 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/istiod-openshift-gateway-94bb8fbfd-6h99s" Jul 10 18:50:13.004752 ip-10-0-143-17 kubenswrapper[2585]: I0710 18:50:13.004700 2585 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-ingress/istiod-openshift-gateway-94bb8fbfd-6h99s" podStartSLOduration=2.004683362 podStartE2EDuration="2.004683362s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-10 18:50:11 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-10 18:50:13.001779214 +0000 UTC m=+626.459139169" watchObservedRunningTime="2026-07-10 18:50:13.004683362 +0000 UTC m=+626.462043298" Jul 10 18:50:13.983348 ip-10-0-143-17 kubenswrapper[2585]: I0710 18:50:13.983316 2585 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/istiod-openshift-gateway-94bb8fbfd-6h99s" Jul 10 18:50:14.055061 ip-10-0-143-17 kubenswrapper[2585]: I0710 18:50:14.055024 2585 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-ingress/istiod-openshift-gateway-7574459656-kjkln"] Jul 10 18:50:14.055448 ip-10-0-143-17 kubenswrapper[2585]: I0710 18:50:14.055236 2585 kuberuntime_container.go:873] "Killing container with a grace period" pod="openshift-ingress/istiod-openshift-gateway-7574459656-kjkln" podUID="1ecf8920-95e7-4eb5-91ed-a9cc8c05c108" containerName="discovery" containerID="cri-o://6a7aedfc78c03cc0277a1cd27064e4710b0d17dcde6c1b6146c7d158b9ffcbea" gracePeriod=30 Jul 10 18:50:14.309420 ip-10-0-143-17 kubenswrapper[2585]: I0710 18:50:14.309392 2585 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/istiod-openshift-gateway-7574459656-kjkln" Jul 10 18:50:14.367628 ip-10-0-143-17 kubenswrapper[2585]: I0710 18:50:14.367607 2585 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/1ecf8920-95e7-4eb5-91ed-a9cc8c05c108-istio-token\") pod \"1ecf8920-95e7-4eb5-91ed-a9cc8c05c108\" (UID: \"1ecf8920-95e7-4eb5-91ed-a9cc8c05c108\") " Jul 10 18:50:14.367746 ip-10-0-143-17 kubenswrapper[2585]: I0710 18:50:14.367669 2585 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"cacerts\" (UniqueName: \"kubernetes.io/secret/1ecf8920-95e7-4eb5-91ed-a9cc8c05c108-cacerts\") pod \"1ecf8920-95e7-4eb5-91ed-a9cc8c05c108\" (UID: \"1ecf8920-95e7-4eb5-91ed-a9cc8c05c108\") " Jul 10 18:50:14.367746 ip-10-0-143-17 kubenswrapper[2585]: I0710 18:50:14.367714 2585 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-k4sgb\" (UniqueName: \"kubernetes.io/projected/1ecf8920-95e7-4eb5-91ed-a9cc8c05c108-kube-api-access-k4sgb\") pod \"1ecf8920-95e7-4eb5-91ed-a9cc8c05c108\" (UID: \"1ecf8920-95e7-4eb5-91ed-a9cc8c05c108\") " Jul 10 18:50:14.367746 ip-10-0-143-17 kubenswrapper[2585]: I0710 18:50:14.367732 2585 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"local-certs\" (UniqueName: \"kubernetes.io/empty-dir/1ecf8920-95e7-4eb5-91ed-a9cc8c05c108-local-certs\") pod \"1ecf8920-95e7-4eb5-91ed-a9cc8c05c108\" (UID: \"1ecf8920-95e7-4eb5-91ed-a9cc8c05c108\") " Jul 10 18:50:14.367905 ip-10-0-143-17 kubenswrapper[2585]: I0710 18:50:14.367759 2585 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istio-csr-dns-cert\" (UniqueName: \"kubernetes.io/secret/1ecf8920-95e7-4eb5-91ed-a9cc8c05c108-istio-csr-dns-cert\") pod \"1ecf8920-95e7-4eb5-91ed-a9cc8c05c108\" (UID: \"1ecf8920-95e7-4eb5-91ed-a9cc8c05c108\") " Jul 10 18:50:14.367905 ip-10-0-143-17 kubenswrapper[2585]: I0710 18:50:14.367799 2585 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istio-csr-ca-configmap\" (UniqueName: \"kubernetes.io/configmap/1ecf8920-95e7-4eb5-91ed-a9cc8c05c108-istio-csr-ca-configmap\") pod \"1ecf8920-95e7-4eb5-91ed-a9cc8c05c108\" (UID: \"1ecf8920-95e7-4eb5-91ed-a9cc8c05c108\") " Jul 10 18:50:14.367905 ip-10-0-143-17 kubenswrapper[2585]: I0710 18:50:14.367845 2585 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istio-kubeconfig\" (UniqueName: \"kubernetes.io/secret/1ecf8920-95e7-4eb5-91ed-a9cc8c05c108-istio-kubeconfig\") pod \"1ecf8920-95e7-4eb5-91ed-a9cc8c05c108\" (UID: \"1ecf8920-95e7-4eb5-91ed-a9cc8c05c108\") " Jul 10 18:50:14.368375 ip-10-0-143-17 kubenswrapper[2585]: I0710 18:50:14.368335 2585 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/1ecf8920-95e7-4eb5-91ed-a9cc8c05c108-istio-csr-ca-configmap" (OuterVolumeSpecName: "istio-csr-ca-configmap") pod "1ecf8920-95e7-4eb5-91ed-a9cc8c05c108" (UID: "1ecf8920-95e7-4eb5-91ed-a9cc8c05c108"). InnerVolumeSpecName "istio-csr-ca-configmap". PluginName "kubernetes.io/configmap", VolumeGIDValue "" Jul 10 18:50:14.370218 ip-10-0-143-17 kubenswrapper[2585]: I0710 18:50:14.370186 2585 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/1ecf8920-95e7-4eb5-91ed-a9cc8c05c108-istio-csr-dns-cert" (OuterVolumeSpecName: "istio-csr-dns-cert") pod "1ecf8920-95e7-4eb5-91ed-a9cc8c05c108" (UID: "1ecf8920-95e7-4eb5-91ed-a9cc8c05c108"). InnerVolumeSpecName "istio-csr-dns-cert". PluginName "kubernetes.io/secret", VolumeGIDValue "" Jul 10 18:50:14.370337 ip-10-0-143-17 kubenswrapper[2585]: I0710 18:50:14.370234 2585 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/1ecf8920-95e7-4eb5-91ed-a9cc8c05c108-kube-api-access-k4sgb" (OuterVolumeSpecName: "kube-api-access-k4sgb") pod "1ecf8920-95e7-4eb5-91ed-a9cc8c05c108" (UID: "1ecf8920-95e7-4eb5-91ed-a9cc8c05c108"). InnerVolumeSpecName "kube-api-access-k4sgb". PluginName "kubernetes.io/projected", VolumeGIDValue "" Jul 10 18:50:14.370434 ip-10-0-143-17 kubenswrapper[2585]: I0710 18:50:14.370415 2585 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/1ecf8920-95e7-4eb5-91ed-a9cc8c05c108-local-certs" (OuterVolumeSpecName: "local-certs") pod "1ecf8920-95e7-4eb5-91ed-a9cc8c05c108" (UID: "1ecf8920-95e7-4eb5-91ed-a9cc8c05c108"). InnerVolumeSpecName "local-certs". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Jul 10 18:50:14.370604 ip-10-0-143-17 kubenswrapper[2585]: I0710 18:50:14.370578 2585 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/1ecf8920-95e7-4eb5-91ed-a9cc8c05c108-cacerts" (OuterVolumeSpecName: "cacerts") pod "1ecf8920-95e7-4eb5-91ed-a9cc8c05c108" (UID: "1ecf8920-95e7-4eb5-91ed-a9cc8c05c108"). InnerVolumeSpecName "cacerts". PluginName "kubernetes.io/secret", VolumeGIDValue "" Jul 10 18:50:14.370695 ip-10-0-143-17 kubenswrapper[2585]: I0710 18:50:14.370669 2585 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/1ecf8920-95e7-4eb5-91ed-a9cc8c05c108-istio-token" (OuterVolumeSpecName: "istio-token") pod "1ecf8920-95e7-4eb5-91ed-a9cc8c05c108" (UID: "1ecf8920-95e7-4eb5-91ed-a9cc8c05c108"). InnerVolumeSpecName "istio-token". PluginName "kubernetes.io/projected", VolumeGIDValue "" Jul 10 18:50:14.370752 ip-10-0-143-17 kubenswrapper[2585]: I0710 18:50:14.370734 2585 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/1ecf8920-95e7-4eb5-91ed-a9cc8c05c108-istio-kubeconfig" (OuterVolumeSpecName: "istio-kubeconfig") pod "1ecf8920-95e7-4eb5-91ed-a9cc8c05c108" (UID: "1ecf8920-95e7-4eb5-91ed-a9cc8c05c108"). InnerVolumeSpecName "istio-kubeconfig". PluginName "kubernetes.io/secret", VolumeGIDValue "" Jul 10 18:50:14.468662 ip-10-0-143-17 kubenswrapper[2585]: I0710 18:50:14.468639 2585 reconciler_common.go:299] "Volume detached for volume \"cacerts\" (UniqueName: \"kubernetes.io/secret/1ecf8920-95e7-4eb5-91ed-a9cc8c05c108-cacerts\") on node \"ip-10-0-143-17.ec2.internal\" DevicePath \"\"" Jul 10 18:50:14.468662 ip-10-0-143-17 kubenswrapper[2585]: I0710 18:50:14.468659 2585 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-k4sgb\" (UniqueName: \"kubernetes.io/projected/1ecf8920-95e7-4eb5-91ed-a9cc8c05c108-kube-api-access-k4sgb\") on node \"ip-10-0-143-17.ec2.internal\" DevicePath \"\"" Jul 10 18:50:14.468787 ip-10-0-143-17 kubenswrapper[2585]: I0710 18:50:14.468670 2585 reconciler_common.go:299] "Volume detached for volume \"local-certs\" (UniqueName: \"kubernetes.io/empty-dir/1ecf8920-95e7-4eb5-91ed-a9cc8c05c108-local-certs\") on node \"ip-10-0-143-17.ec2.internal\" DevicePath \"\"" Jul 10 18:50:14.468787 ip-10-0-143-17 kubenswrapper[2585]: I0710 18:50:14.468679 2585 reconciler_common.go:299] "Volume detached for volume \"istio-csr-dns-cert\" (UniqueName: \"kubernetes.io/secret/1ecf8920-95e7-4eb5-91ed-a9cc8c05c108-istio-csr-dns-cert\") on node \"ip-10-0-143-17.ec2.internal\" DevicePath \"\"" Jul 10 18:50:14.468787 ip-10-0-143-17 kubenswrapper[2585]: I0710 18:50:14.468687 2585 reconciler_common.go:299] "Volume detached for volume \"istio-csr-ca-configmap\" (UniqueName: \"kubernetes.io/configmap/1ecf8920-95e7-4eb5-91ed-a9cc8c05c108-istio-csr-ca-configmap\") on node \"ip-10-0-143-17.ec2.internal\" DevicePath \"\"" Jul 10 18:50:14.468787 ip-10-0-143-17 kubenswrapper[2585]: I0710 18:50:14.468695 2585 reconciler_common.go:299] "Volume detached for volume \"istio-kubeconfig\" (UniqueName: \"kubernetes.io/secret/1ecf8920-95e7-4eb5-91ed-a9cc8c05c108-istio-kubeconfig\") on node \"ip-10-0-143-17.ec2.internal\" DevicePath \"\"" Jul 10 18:50:14.468787 ip-10-0-143-17 kubenswrapper[2585]: I0710 18:50:14.468703 2585 reconciler_common.go:299] "Volume detached for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/1ecf8920-95e7-4eb5-91ed-a9cc8c05c108-istio-token\") on node \"ip-10-0-143-17.ec2.internal\" DevicePath \"\"" Jul 10 18:50:14.987892 ip-10-0-143-17 kubenswrapper[2585]: I0710 18:50:14.987861 2585 generic.go:356] "Generic (PLEG): container finished" podID="1ecf8920-95e7-4eb5-91ed-a9cc8c05c108" containerID="6a7aedfc78c03cc0277a1cd27064e4710b0d17dcde6c1b6146c7d158b9ffcbea" exitCode=0 Jul 10 18:50:14.988317 ip-10-0-143-17 kubenswrapper[2585]: I0710 18:50:14.987909 2585 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/istiod-openshift-gateway-7574459656-kjkln" Jul 10 18:50:14.988317 ip-10-0-143-17 kubenswrapper[2585]: I0710 18:50:14.987946 2585 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/istiod-openshift-gateway-7574459656-kjkln" event={"ID":"1ecf8920-95e7-4eb5-91ed-a9cc8c05c108","Type":"ContainerDied","Data":"6a7aedfc78c03cc0277a1cd27064e4710b0d17dcde6c1b6146c7d158b9ffcbea"} Jul 10 18:50:14.988317 ip-10-0-143-17 kubenswrapper[2585]: I0710 18:50:14.987994 2585 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/istiod-openshift-gateway-7574459656-kjkln" event={"ID":"1ecf8920-95e7-4eb5-91ed-a9cc8c05c108","Type":"ContainerDied","Data":"209eca4911578c7e2ce36c1ba1a2d067ccf194c74df2f6c58c4d31a829313970"} Jul 10 18:50:14.988317 ip-10-0-143-17 kubenswrapper[2585]: I0710 18:50:14.988015 2585 scope.go:117] "RemoveContainer" containerID="6a7aedfc78c03cc0277a1cd27064e4710b0d17dcde6c1b6146c7d158b9ffcbea" Jul 10 18:50:14.997555 ip-10-0-143-17 kubenswrapper[2585]: I0710 18:50:14.997525 2585 scope.go:117] "RemoveContainer" containerID="6a7aedfc78c03cc0277a1cd27064e4710b0d17dcde6c1b6146c7d158b9ffcbea" Jul 10 18:50:14.997773 ip-10-0-143-17 kubenswrapper[2585]: E0710 18:50:14.997757 2585 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"6a7aedfc78c03cc0277a1cd27064e4710b0d17dcde6c1b6146c7d158b9ffcbea\": container with ID starting with 6a7aedfc78c03cc0277a1cd27064e4710b0d17dcde6c1b6146c7d158b9ffcbea not found: ID does not exist" containerID="6a7aedfc78c03cc0277a1cd27064e4710b0d17dcde6c1b6146c7d158b9ffcbea" Jul 10 18:50:14.997827 ip-10-0-143-17 kubenswrapper[2585]: I0710 18:50:14.997779 2585 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"6a7aedfc78c03cc0277a1cd27064e4710b0d17dcde6c1b6146c7d158b9ffcbea"} err="failed to get container status \"6a7aedfc78c03cc0277a1cd27064e4710b0d17dcde6c1b6146c7d158b9ffcbea\": rpc error: code = NotFound desc = could not find container \"6a7aedfc78c03cc0277a1cd27064e4710b0d17dcde6c1b6146c7d158b9ffcbea\": container with ID starting with 6a7aedfc78c03cc0277a1cd27064e4710b0d17dcde6c1b6146c7d158b9ffcbea not found: ID does not exist" Jul 10 18:50:15.028701 ip-10-0-143-17 kubenswrapper[2585]: I0710 18:50:15.028677 2585 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-ingress/istiod-openshift-gateway-7574459656-kjkln"] Jul 10 18:50:15.035608 ip-10-0-143-17 kubenswrapper[2585]: I0710 18:50:15.035585 2585 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["openshift-ingress/istiod-openshift-gateway-7574459656-kjkln"] Jul 10 18:50:15.216852 ip-10-0-143-17 kubenswrapper[2585]: I0710 18:50:15.216823 2585 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="1ecf8920-95e7-4eb5-91ed-a9cc8c05c108" path="/var/lib/kubelet/pods/1ecf8920-95e7-4eb5-91ed-a9cc8c05c108/volumes" Jul 10 18:50:21.521947 ip-10-0-143-17 kubenswrapper[2585]: I0710 18:50:21.521900 2585 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["kserve/seaweedfs-64568bcd49-bgwr8"] Jul 10 18:50:21.522323 ip-10-0-143-17 kubenswrapper[2585]: I0710 18:50:21.522251 2585 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="1ecf8920-95e7-4eb5-91ed-a9cc8c05c108" containerName="discovery" Jul 10 18:50:21.522323 ip-10-0-143-17 kubenswrapper[2585]: I0710 18:50:21.522261 2585 state_mem.go:107] "Deleted CPUSet assignment" podUID="1ecf8920-95e7-4eb5-91ed-a9cc8c05c108" containerName="discovery" Jul 10 18:50:21.522323 ip-10-0-143-17 kubenswrapper[2585]: I0710 18:50:21.522322 2585 memory_manager.go:368] "RemoveStaleState removing state" podUID="1ecf8920-95e7-4eb5-91ed-a9cc8c05c108" containerName="discovery" Jul 10 18:50:21.525191 ip-10-0-143-17 kubenswrapper[2585]: I0710 18:50:21.525167 2585 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kserve/seaweedfs-64568bcd49-bgwr8" Jul 10 18:50:21.528322 ip-10-0-143-17 kubenswrapper[2585]: I0710 18:50:21.528301 2585 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"kserve\"/\"mlpipeline-s3-artifact\"" Jul 10 18:50:21.528463 ip-10-0-143-17 kubenswrapper[2585]: I0710 18:50:21.528393 2585 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"kserve\"/\"default-dockercfg-rkjvq\"" Jul 10 18:50:21.528574 ip-10-0-143-17 kubenswrapper[2585]: I0710 18:50:21.528551 2585 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"kserve\"/\"openshift-service-ca.crt\"" Jul 10 18:50:21.528682 ip-10-0-143-17 kubenswrapper[2585]: I0710 18:50:21.528602 2585 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"kserve\"/\"kube-root-ca.crt\"" Jul 10 18:50:21.536971 ip-10-0-143-17 kubenswrapper[2585]: I0710 18:50:21.536947 2585 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kserve/seaweedfs-64568bcd49-bgwr8"] Jul 10 18:50:21.632725 ip-10-0-143-17 kubenswrapper[2585]: I0710 18:50:21.632683 2585 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"data\" (UniqueName: \"kubernetes.io/empty-dir/d92b43e7-be96-4435-9d79-2091eb4a84dd-data\") pod \"seaweedfs-64568bcd49-bgwr8\" (UID: \"d92b43e7-be96-4435-9d79-2091eb4a84dd\") " pod="kserve/seaweedfs-64568bcd49-bgwr8" Jul 10 18:50:21.632900 ip-10-0-143-17 kubenswrapper[2585]: I0710 18:50:21.632816 2585 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-877jw\" (UniqueName: \"kubernetes.io/projected/d92b43e7-be96-4435-9d79-2091eb4a84dd-kube-api-access-877jw\") pod \"seaweedfs-64568bcd49-bgwr8\" (UID: \"d92b43e7-be96-4435-9d79-2091eb4a84dd\") " pod="kserve/seaweedfs-64568bcd49-bgwr8" Jul 10 18:50:21.733304 ip-10-0-143-17 kubenswrapper[2585]: I0710 18:50:21.733266 2585 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-877jw\" (UniqueName: \"kubernetes.io/projected/d92b43e7-be96-4435-9d79-2091eb4a84dd-kube-api-access-877jw\") pod \"seaweedfs-64568bcd49-bgwr8\" (UID: \"d92b43e7-be96-4435-9d79-2091eb4a84dd\") " pod="kserve/seaweedfs-64568bcd49-bgwr8" Jul 10 18:50:21.733513 ip-10-0-143-17 kubenswrapper[2585]: I0710 18:50:21.733342 2585 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"data\" (UniqueName: \"kubernetes.io/empty-dir/d92b43e7-be96-4435-9d79-2091eb4a84dd-data\") pod \"seaweedfs-64568bcd49-bgwr8\" (UID: \"d92b43e7-be96-4435-9d79-2091eb4a84dd\") " pod="kserve/seaweedfs-64568bcd49-bgwr8" Jul 10 18:50:21.733811 ip-10-0-143-17 kubenswrapper[2585]: I0710 18:50:21.733789 2585 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"data\" (UniqueName: \"kubernetes.io/empty-dir/d92b43e7-be96-4435-9d79-2091eb4a84dd-data\") pod \"seaweedfs-64568bcd49-bgwr8\" (UID: \"d92b43e7-be96-4435-9d79-2091eb4a84dd\") " pod="kserve/seaweedfs-64568bcd49-bgwr8" Jul 10 18:50:21.742254 ip-10-0-143-17 kubenswrapper[2585]: I0710 18:50:21.742221 2585 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-877jw\" (UniqueName: \"kubernetes.io/projected/d92b43e7-be96-4435-9d79-2091eb4a84dd-kube-api-access-877jw\") pod \"seaweedfs-64568bcd49-bgwr8\" (UID: \"d92b43e7-be96-4435-9d79-2091eb4a84dd\") " pod="kserve/seaweedfs-64568bcd49-bgwr8" Jul 10 18:50:21.835485 ip-10-0-143-17 kubenswrapper[2585]: I0710 18:50:21.835392 2585 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kserve/seaweedfs-64568bcd49-bgwr8" Jul 10 18:50:21.977132 ip-10-0-143-17 kubenswrapper[2585]: I0710 18:50:21.977103 2585 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kserve/seaweedfs-64568bcd49-bgwr8"] Jul 10 18:50:21.979196 ip-10-0-143-17 kubenswrapper[2585]: W0710 18:50:21.979167 2585 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podd92b43e7_be96_4435_9d79_2091eb4a84dd.slice/crio-5a8d8b9e0940fa3ddb04bb56b05331fbaf5d82c75b4fb6961ee34ce53b39bdff WatchSource:0}: Error finding container 5a8d8b9e0940fa3ddb04bb56b05331fbaf5d82c75b4fb6961ee34ce53b39bdff: Status 404 returned error can't find the container with id 5a8d8b9e0940fa3ddb04bb56b05331fbaf5d82c75b4fb6961ee34ce53b39bdff Jul 10 18:50:22.026174 ip-10-0-143-17 kubenswrapper[2585]: I0710 18:50:22.026139 2585 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kserve/seaweedfs-64568bcd49-bgwr8" event={"ID":"d92b43e7-be96-4435-9d79-2091eb4a84dd","Type":"ContainerStarted","Data":"5a8d8b9e0940fa3ddb04bb56b05331fbaf5d82c75b4fb6961ee34ce53b39bdff"} Jul 10 18:50:26.049461 ip-10-0-143-17 kubenswrapper[2585]: I0710 18:50:26.049427 2585 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kserve/seaweedfs-64568bcd49-bgwr8" event={"ID":"d92b43e7-be96-4435-9d79-2091eb4a84dd","Type":"ContainerStarted","Data":"59459bca1ba64bbccb4be50b8af3f44994f172640b7462303952f147672da4c6"} Jul 10 18:50:26.049862 ip-10-0-143-17 kubenswrapper[2585]: I0710 18:50:26.049499 2585 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="kserve/seaweedfs-64568bcd49-bgwr8" Jul 10 18:50:26.070175 ip-10-0-143-17 kubenswrapper[2585]: I0710 18:50:26.070113 2585 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="kserve/seaweedfs-64568bcd49-bgwr8" podStartSLOduration=1.764464904 podStartE2EDuration="5.070098026s" totalImagesPullingTime="3.305633122s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-10 18:50:21 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-10 18:50:26.06776115 +0000 UTC m=+639.525121110" watchObservedRunningTime="2026-07-10 18:50:26.070098026 +0000 UTC m=+639.527458039" Jul 10 18:50:32.057036 ip-10-0-143-17 kubenswrapper[2585]: I0710 18:50:32.056994 2585 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="kserve/seaweedfs-64568bcd49-bgwr8" Jul 10 18:52:10.256704 ip-10-0-143-17 kubenswrapper[2585]: I0710 18:52:10.256621 2585 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["kserve-ci-e2e-test/router-gateway-1-openshift-default-799f46c59b-sf6qk"] Jul 10 18:52:10.260146 ip-10-0-143-17 kubenswrapper[2585]: I0710 18:52:10.260123 2585 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kserve-ci-e2e-test/router-gateway-1-openshift-default-799f46c59b-sf6qk" Jul 10 18:52:10.262822 ip-10-0-143-17 kubenswrapper[2585]: I0710 18:52:10.262802 2585 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"kserve-ci-e2e-test\"/\"openshift-gw-ca-root-cert\"" Jul 10 18:52:10.263074 ip-10-0-143-17 kubenswrapper[2585]: I0710 18:52:10.263047 2585 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"kserve-ci-e2e-test\"/\"router-gateway-1-openshift-default-dockercfg-25h9s\"" Jul 10 18:52:10.263180 ip-10-0-143-17 kubenswrapper[2585]: I0710 18:52:10.263061 2585 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"kserve-ci-e2e-test\"/\"kube-root-ca.crt\"" Jul 10 18:52:10.263278 ip-10-0-143-17 kubenswrapper[2585]: I0710 18:52:10.263245 2585 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"kserve-ci-e2e-test\"/\"openshift-service-ca.crt\"" Jul 10 18:52:10.281132 ip-10-0-143-17 kubenswrapper[2585]: I0710 18:52:10.281108 2585 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kserve-ci-e2e-test/router-gateway-1-openshift-default-799f46c59b-sf6qk"] Jul 10 18:52:10.404524 ip-10-0-143-17 kubenswrapper[2585]: I0710 18:52:10.404485 2585 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/10d311d2-b0ff-486f-a709-ef0f52dfa327-credential-socket\") pod \"router-gateway-1-openshift-default-799f46c59b-sf6qk\" (UID: \"10d311d2-b0ff-486f-a709-ef0f52dfa327\") " pod="kserve-ci-e2e-test/router-gateway-1-openshift-default-799f46c59b-sf6qk" Jul 10 18:52:10.404713 ip-10-0-143-17 kubenswrapper[2585]: I0710 18:52:10.404529 2585 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/10d311d2-b0ff-486f-a709-ef0f52dfa327-istio-envoy\") pod \"router-gateway-1-openshift-default-799f46c59b-sf6qk\" (UID: \"10d311d2-b0ff-486f-a709-ef0f52dfa327\") " pod="kserve-ci-e2e-test/router-gateway-1-openshift-default-799f46c59b-sf6qk" Jul 10 18:52:10.404713 ip-10-0-143-17 kubenswrapper[2585]: I0710 18:52:10.404567 2585 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/10d311d2-b0ff-486f-a709-ef0f52dfa327-workload-socket\") pod \"router-gateway-1-openshift-default-799f46c59b-sf6qk\" (UID: \"10d311d2-b0ff-486f-a709-ef0f52dfa327\") " pod="kserve-ci-e2e-test/router-gateway-1-openshift-default-799f46c59b-sf6qk" Jul 10 18:52:10.404713 ip-10-0-143-17 kubenswrapper[2585]: I0710 18:52:10.404628 2585 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/10d311d2-b0ff-486f-a709-ef0f52dfa327-workload-certs\") pod \"router-gateway-1-openshift-default-799f46c59b-sf6qk\" (UID: \"10d311d2-b0ff-486f-a709-ef0f52dfa327\") " pod="kserve-ci-e2e-test/router-gateway-1-openshift-default-799f46c59b-sf6qk" Jul 10 18:52:10.404713 ip-10-0-143-17 kubenswrapper[2585]: I0710 18:52:10.404676 2585 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/10d311d2-b0ff-486f-a709-ef0f52dfa327-istio-data\") pod \"router-gateway-1-openshift-default-799f46c59b-sf6qk\" (UID: \"10d311d2-b0ff-486f-a709-ef0f52dfa327\") " pod="kserve-ci-e2e-test/router-gateway-1-openshift-default-799f46c59b-sf6qk" Jul 10 18:52:10.404713 ip-10-0-143-17 kubenswrapper[2585]: I0710 18:52:10.404709 2585 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-6qmql\" (UniqueName: \"kubernetes.io/projected/10d311d2-b0ff-486f-a709-ef0f52dfa327-kube-api-access-6qmql\") pod \"router-gateway-1-openshift-default-799f46c59b-sf6qk\" (UID: \"10d311d2-b0ff-486f-a709-ef0f52dfa327\") " pod="kserve-ci-e2e-test/router-gateway-1-openshift-default-799f46c59b-sf6qk" Jul 10 18:52:10.404908 ip-10-0-143-17 kubenswrapper[2585]: I0710 18:52:10.404731 2585 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/10d311d2-b0ff-486f-a709-ef0f52dfa327-istio-token\") pod \"router-gateway-1-openshift-default-799f46c59b-sf6qk\" (UID: \"10d311d2-b0ff-486f-a709-ef0f52dfa327\") " pod="kserve-ci-e2e-test/router-gateway-1-openshift-default-799f46c59b-sf6qk" Jul 10 18:52:10.404908 ip-10-0-143-17 kubenswrapper[2585]: I0710 18:52:10.404764 2585 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/10d311d2-b0ff-486f-a709-ef0f52dfa327-istiod-ca-cert\") pod \"router-gateway-1-openshift-default-799f46c59b-sf6qk\" (UID: \"10d311d2-b0ff-486f-a709-ef0f52dfa327\") " pod="kserve-ci-e2e-test/router-gateway-1-openshift-default-799f46c59b-sf6qk" Jul 10 18:52:10.404908 ip-10-0-143-17 kubenswrapper[2585]: I0710 18:52:10.404794 2585 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/10d311d2-b0ff-486f-a709-ef0f52dfa327-istio-podinfo\") pod \"router-gateway-1-openshift-default-799f46c59b-sf6qk\" (UID: \"10d311d2-b0ff-486f-a709-ef0f52dfa327\") " pod="kserve-ci-e2e-test/router-gateway-1-openshift-default-799f46c59b-sf6qk" Jul 10 18:52:10.505817 ip-10-0-143-17 kubenswrapper[2585]: I0710 18:52:10.505784 2585 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-6qmql\" (UniqueName: \"kubernetes.io/projected/10d311d2-b0ff-486f-a709-ef0f52dfa327-kube-api-access-6qmql\") pod \"router-gateway-1-openshift-default-799f46c59b-sf6qk\" (UID: \"10d311d2-b0ff-486f-a709-ef0f52dfa327\") " pod="kserve-ci-e2e-test/router-gateway-1-openshift-default-799f46c59b-sf6qk" Jul 10 18:52:10.505817 ip-10-0-143-17 kubenswrapper[2585]: I0710 18:52:10.505825 2585 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/10d311d2-b0ff-486f-a709-ef0f52dfa327-istio-token\") pod \"router-gateway-1-openshift-default-799f46c59b-sf6qk\" (UID: \"10d311d2-b0ff-486f-a709-ef0f52dfa327\") " pod="kserve-ci-e2e-test/router-gateway-1-openshift-default-799f46c59b-sf6qk" Jul 10 18:52:10.506089 ip-10-0-143-17 kubenswrapper[2585]: I0710 18:52:10.505846 2585 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/10d311d2-b0ff-486f-a709-ef0f52dfa327-istiod-ca-cert\") pod \"router-gateway-1-openshift-default-799f46c59b-sf6qk\" (UID: \"10d311d2-b0ff-486f-a709-ef0f52dfa327\") " pod="kserve-ci-e2e-test/router-gateway-1-openshift-default-799f46c59b-sf6qk" Jul 10 18:52:10.506089 ip-10-0-143-17 kubenswrapper[2585]: I0710 18:52:10.505891 2585 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/10d311d2-b0ff-486f-a709-ef0f52dfa327-istio-podinfo\") pod \"router-gateway-1-openshift-default-799f46c59b-sf6qk\" (UID: \"10d311d2-b0ff-486f-a709-ef0f52dfa327\") " pod="kserve-ci-e2e-test/router-gateway-1-openshift-default-799f46c59b-sf6qk" Jul 10 18:52:10.506089 ip-10-0-143-17 kubenswrapper[2585]: I0710 18:52:10.505957 2585 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/10d311d2-b0ff-486f-a709-ef0f52dfa327-credential-socket\") pod \"router-gateway-1-openshift-default-799f46c59b-sf6qk\" (UID: \"10d311d2-b0ff-486f-a709-ef0f52dfa327\") " pod="kserve-ci-e2e-test/router-gateway-1-openshift-default-799f46c59b-sf6qk" Jul 10 18:52:10.506089 ip-10-0-143-17 kubenswrapper[2585]: I0710 18:52:10.505989 2585 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/10d311d2-b0ff-486f-a709-ef0f52dfa327-istio-envoy\") pod \"router-gateway-1-openshift-default-799f46c59b-sf6qk\" (UID: \"10d311d2-b0ff-486f-a709-ef0f52dfa327\") " pod="kserve-ci-e2e-test/router-gateway-1-openshift-default-799f46c59b-sf6qk" Jul 10 18:52:10.506089 ip-10-0-143-17 kubenswrapper[2585]: I0710 18:52:10.506027 2585 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/10d311d2-b0ff-486f-a709-ef0f52dfa327-workload-socket\") pod \"router-gateway-1-openshift-default-799f46c59b-sf6qk\" (UID: \"10d311d2-b0ff-486f-a709-ef0f52dfa327\") " pod="kserve-ci-e2e-test/router-gateway-1-openshift-default-799f46c59b-sf6qk" Jul 10 18:52:10.506352 ip-10-0-143-17 kubenswrapper[2585]: I0710 18:52:10.506178 2585 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/10d311d2-b0ff-486f-a709-ef0f52dfa327-workload-certs\") pod \"router-gateway-1-openshift-default-799f46c59b-sf6qk\" (UID: \"10d311d2-b0ff-486f-a709-ef0f52dfa327\") " pod="kserve-ci-e2e-test/router-gateway-1-openshift-default-799f46c59b-sf6qk" Jul 10 18:52:10.506352 ip-10-0-143-17 kubenswrapper[2585]: I0710 18:52:10.506229 2585 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/10d311d2-b0ff-486f-a709-ef0f52dfa327-istio-data\") pod \"router-gateway-1-openshift-default-799f46c59b-sf6qk\" (UID: \"10d311d2-b0ff-486f-a709-ef0f52dfa327\") " pod="kserve-ci-e2e-test/router-gateway-1-openshift-default-799f46c59b-sf6qk" Jul 10 18:52:10.506604 ip-10-0-143-17 kubenswrapper[2585]: I0710 18:52:10.506579 2585 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/10d311d2-b0ff-486f-a709-ef0f52dfa327-credential-socket\") pod \"router-gateway-1-openshift-default-799f46c59b-sf6qk\" (UID: \"10d311d2-b0ff-486f-a709-ef0f52dfa327\") " pod="kserve-ci-e2e-test/router-gateway-1-openshift-default-799f46c59b-sf6qk" Jul 10 18:52:10.506763 ip-10-0-143-17 kubenswrapper[2585]: I0710 18:52:10.506627 2585 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/10d311d2-b0ff-486f-a709-ef0f52dfa327-workload-certs\") pod \"router-gateway-1-openshift-default-799f46c59b-sf6qk\" (UID: \"10d311d2-b0ff-486f-a709-ef0f52dfa327\") " pod="kserve-ci-e2e-test/router-gateway-1-openshift-default-799f46c59b-sf6qk" Jul 10 18:52:10.506763 ip-10-0-143-17 kubenswrapper[2585]: I0710 18:52:10.506690 2585 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/10d311d2-b0ff-486f-a709-ef0f52dfa327-workload-socket\") pod \"router-gateway-1-openshift-default-799f46c59b-sf6qk\" (UID: \"10d311d2-b0ff-486f-a709-ef0f52dfa327\") " pod="kserve-ci-e2e-test/router-gateway-1-openshift-default-799f46c59b-sf6qk" Jul 10 18:52:10.506763 ip-10-0-143-17 kubenswrapper[2585]: I0710 18:52:10.506743 2585 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/10d311d2-b0ff-486f-a709-ef0f52dfa327-istiod-ca-cert\") pod \"router-gateway-1-openshift-default-799f46c59b-sf6qk\" (UID: \"10d311d2-b0ff-486f-a709-ef0f52dfa327\") " pod="kserve-ci-e2e-test/router-gateway-1-openshift-default-799f46c59b-sf6qk" Jul 10 18:52:10.506899 ip-10-0-143-17 kubenswrapper[2585]: I0710 18:52:10.506813 2585 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/10d311d2-b0ff-486f-a709-ef0f52dfa327-istio-data\") pod \"router-gateway-1-openshift-default-799f46c59b-sf6qk\" (UID: \"10d311d2-b0ff-486f-a709-ef0f52dfa327\") " pod="kserve-ci-e2e-test/router-gateway-1-openshift-default-799f46c59b-sf6qk" Jul 10 18:52:10.508364 ip-10-0-143-17 kubenswrapper[2585]: I0710 18:52:10.508344 2585 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/10d311d2-b0ff-486f-a709-ef0f52dfa327-istio-envoy\") pod \"router-gateway-1-openshift-default-799f46c59b-sf6qk\" (UID: \"10d311d2-b0ff-486f-a709-ef0f52dfa327\") " pod="kserve-ci-e2e-test/router-gateway-1-openshift-default-799f46c59b-sf6qk" Jul 10 18:52:10.508465 ip-10-0-143-17 kubenswrapper[2585]: I0710 18:52:10.508441 2585 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/10d311d2-b0ff-486f-a709-ef0f52dfa327-istio-podinfo\") pod \"router-gateway-1-openshift-default-799f46c59b-sf6qk\" (UID: \"10d311d2-b0ff-486f-a709-ef0f52dfa327\") " pod="kserve-ci-e2e-test/router-gateway-1-openshift-default-799f46c59b-sf6qk" Jul 10 18:52:10.514531 ip-10-0-143-17 kubenswrapper[2585]: I0710 18:52:10.514506 2585 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/10d311d2-b0ff-486f-a709-ef0f52dfa327-istio-token\") pod \"router-gateway-1-openshift-default-799f46c59b-sf6qk\" (UID: \"10d311d2-b0ff-486f-a709-ef0f52dfa327\") " pod="kserve-ci-e2e-test/router-gateway-1-openshift-default-799f46c59b-sf6qk" Jul 10 18:52:10.514531 ip-10-0-143-17 kubenswrapper[2585]: I0710 18:52:10.514529 2585 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-6qmql\" (UniqueName: \"kubernetes.io/projected/10d311d2-b0ff-486f-a709-ef0f52dfa327-kube-api-access-6qmql\") pod \"router-gateway-1-openshift-default-799f46c59b-sf6qk\" (UID: \"10d311d2-b0ff-486f-a709-ef0f52dfa327\") " pod="kserve-ci-e2e-test/router-gateway-1-openshift-default-799f46c59b-sf6qk" Jul 10 18:52:10.571574 ip-10-0-143-17 kubenswrapper[2585]: I0710 18:52:10.571520 2585 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kserve-ci-e2e-test/router-gateway-1-openshift-default-799f46c59b-sf6qk" Jul 10 18:52:10.716685 ip-10-0-143-17 kubenswrapper[2585]: I0710 18:52:10.716651 2585 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kserve-ci-e2e-test/router-gateway-1-openshift-default-799f46c59b-sf6qk"] Jul 10 18:52:10.720064 ip-10-0-143-17 kubenswrapper[2585]: W0710 18:52:10.720033 2585 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod10d311d2_b0ff_486f_a709_ef0f52dfa327.slice/crio-85795610a8eb9781c3ff8ebcd5a124c0b4dadc3269498e68e8791ecedce99a8f WatchSource:0}: Error finding container 85795610a8eb9781c3ff8ebcd5a124c0b4dadc3269498e68e8791ecedce99a8f: Status 404 returned error can't find the container with id 85795610a8eb9781c3ff8ebcd5a124c0b4dadc3269498e68e8791ecedce99a8f Jul 10 18:52:10.722060 ip-10-0-143-17 kubenswrapper[2585]: I0710 18:52:10.722026 2585 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236224Ki","pods":"250"} Jul 10 18:52:10.722132 ip-10-0-143-17 kubenswrapper[2585]: I0710 18:52:10.722098 2585 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236224Ki","pods":"250"} Jul 10 18:52:10.722169 ip-10-0-143-17 kubenswrapper[2585]: I0710 18:52:10.722132 2585 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236224Ki","pods":"250"} Jul 10 18:52:11.580978 ip-10-0-143-17 kubenswrapper[2585]: I0710 18:52:11.580944 2585 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kserve-ci-e2e-test/router-gateway-1-openshift-default-799f46c59b-sf6qk" event={"ID":"10d311d2-b0ff-486f-a709-ef0f52dfa327","Type":"ContainerStarted","Data":"282ee7209dc4660a0f48a5be87cc0dad51cfb2d9acdbc0eaa53b7515210aafbf"} Jul 10 18:52:11.581472 ip-10-0-143-17 kubenswrapper[2585]: I0710 18:52:11.580998 2585 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kserve-ci-e2e-test/router-gateway-1-openshift-default-799f46c59b-sf6qk" event={"ID":"10d311d2-b0ff-486f-a709-ef0f52dfa327","Type":"ContainerStarted","Data":"85795610a8eb9781c3ff8ebcd5a124c0b4dadc3269498e68e8791ecedce99a8f"} Jul 10 18:52:11.608141 ip-10-0-143-17 kubenswrapper[2585]: I0710 18:52:11.608094 2585 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="kserve-ci-e2e-test/router-gateway-1-openshift-default-799f46c59b-sf6qk" podStartSLOduration=1.608076949 podStartE2EDuration="1.608076949s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-10 18:52:10 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-10 18:52:11.605736545 +0000 UTC m=+745.063096481" watchObservedRunningTime="2026-07-10 18:52:11.608076949 +0000 UTC m=+745.065436883" Jul 10 18:52:12.572261 ip-10-0-143-17 kubenswrapper[2585]: I0710 18:52:12.572222 2585 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="kserve-ci-e2e-test/router-gateway-1-openshift-default-799f46c59b-sf6qk" Jul 10 18:52:12.577477 ip-10-0-143-17 kubenswrapper[2585]: I0710 18:52:12.577449 2585 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="kserve-ci-e2e-test/router-gateway-1-openshift-default-799f46c59b-sf6qk" Jul 10 18:52:12.585594 ip-10-0-143-17 kubenswrapper[2585]: I0710 18:52:12.585572 2585 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="kserve-ci-e2e-test/router-gateway-1-openshift-default-799f46c59b-sf6qk" Jul 10 18:52:12.586418 ip-10-0-143-17 kubenswrapper[2585]: I0710 18:52:12.586400 2585 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="kserve-ci-e2e-test/router-gateway-1-openshift-default-799f46c59b-sf6qk" Jul 10 18:52:52.318234 ip-10-0-143-17 kubenswrapper[2585]: I0710 18:52:52.318189 2585 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["kserve-ci-e2e-test/gw-section-name-router-with-gat-2f0a622e-kserve-798cdbd559pb2cr"] Jul 10 18:52:52.321983 ip-10-0-143-17 kubenswrapper[2585]: I0710 18:52:52.321965 2585 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kserve-ci-e2e-test/gw-section-name-router-with-gat-2f0a622e-kserve-798cdbd559pb2cr" Jul 10 18:52:52.326061 ip-10-0-143-17 kubenswrapper[2585]: I0710 18:52:52.326045 2585 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"kserve-ci-e2e-test\"/\"default-dockercfg-jjdpb\"" Jul 10 18:52:52.326188 ip-10-0-143-17 kubenswrapper[2585]: I0710 18:52:52.326057 2585 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"kserve-ci-e2e-test\"/\"gw-sec0c69dceeb48768325d1a53a749e65786-kserve-self-signed-certs\"" Jul 10 18:52:52.331710 ip-10-0-143-17 kubenswrapper[2585]: I0710 18:52:52.331690 2585 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kserve-ci-e2e-test/gw-section-name-router-with-gat-2f0a622e-kserve-798cdbd559pb2cr"] Jul 10 18:52:52.488224 ip-10-0-143-17 kubenswrapper[2585]: I0710 18:52:52.488188 2585 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-pdrds\" (UniqueName: \"kubernetes.io/projected/77ac2342-1d50-4c4b-bf5b-0528240cd532-kube-api-access-pdrds\") pod \"gw-section-name-router-with-gat-2f0a622e-kserve-798cdbd559pb2cr\" (UID: \"77ac2342-1d50-4c4b-bf5b-0528240cd532\") " pod="kserve-ci-e2e-test/gw-section-name-router-with-gat-2f0a622e-kserve-798cdbd559pb2cr" Jul 10 18:52:52.488381 ip-10-0-143-17 kubenswrapper[2585]: I0710 18:52:52.488293 2585 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/77ac2342-1d50-4c4b-bf5b-0528240cd532-model-cache\") pod \"gw-section-name-router-with-gat-2f0a622e-kserve-798cdbd559pb2cr\" (UID: \"77ac2342-1d50-4c4b-bf5b-0528240cd532\") " pod="kserve-ci-e2e-test/gw-section-name-router-with-gat-2f0a622e-kserve-798cdbd559pb2cr" Jul 10 18:52:52.488381 ip-10-0-143-17 kubenswrapper[2585]: I0710 18:52:52.488345 2585 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/77ac2342-1d50-4c4b-bf5b-0528240cd532-dshm\") pod \"gw-section-name-router-with-gat-2f0a622e-kserve-798cdbd559pb2cr\" (UID: \"77ac2342-1d50-4c4b-bf5b-0528240cd532\") " pod="kserve-ci-e2e-test/gw-section-name-router-with-gat-2f0a622e-kserve-798cdbd559pb2cr" Jul 10 18:52:52.488381 ip-10-0-143-17 kubenswrapper[2585]: I0710 18:52:52.488374 2585 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/77ac2342-1d50-4c4b-bf5b-0528240cd532-tls-certs\") pod \"gw-section-name-router-with-gat-2f0a622e-kserve-798cdbd559pb2cr\" (UID: \"77ac2342-1d50-4c4b-bf5b-0528240cd532\") " pod="kserve-ci-e2e-test/gw-section-name-router-with-gat-2f0a622e-kserve-798cdbd559pb2cr" Jul 10 18:52:52.488523 ip-10-0-143-17 kubenswrapper[2585]: I0710 18:52:52.488501 2585 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/77ac2342-1d50-4c4b-bf5b-0528240cd532-tmp-dir\") pod \"gw-section-name-router-with-gat-2f0a622e-kserve-798cdbd559pb2cr\" (UID: \"77ac2342-1d50-4c4b-bf5b-0528240cd532\") " pod="kserve-ci-e2e-test/gw-section-name-router-with-gat-2f0a622e-kserve-798cdbd559pb2cr" Jul 10 18:52:52.488582 ip-10-0-143-17 kubenswrapper[2585]: I0710 18:52:52.488569 2585 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kserve-provision-location\" (UniqueName: \"kubernetes.io/empty-dir/77ac2342-1d50-4c4b-bf5b-0528240cd532-kserve-provision-location\") pod \"gw-section-name-router-with-gat-2f0a622e-kserve-798cdbd559pb2cr\" (UID: \"77ac2342-1d50-4c4b-bf5b-0528240cd532\") " pod="kserve-ci-e2e-test/gw-section-name-router-with-gat-2f0a622e-kserve-798cdbd559pb2cr" Jul 10 18:52:52.488621 ip-10-0-143-17 kubenswrapper[2585]: I0710 18:52:52.488593 2585 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/77ac2342-1d50-4c4b-bf5b-0528240cd532-home\") pod \"gw-section-name-router-with-gat-2f0a622e-kserve-798cdbd559pb2cr\" (UID: \"77ac2342-1d50-4c4b-bf5b-0528240cd532\") " pod="kserve-ci-e2e-test/gw-section-name-router-with-gat-2f0a622e-kserve-798cdbd559pb2cr" Jul 10 18:52:52.589248 ip-10-0-143-17 kubenswrapper[2585]: I0710 18:52:52.589179 2585 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/77ac2342-1d50-4c4b-bf5b-0528240cd532-model-cache\") pod \"gw-section-name-router-with-gat-2f0a622e-kserve-798cdbd559pb2cr\" (UID: \"77ac2342-1d50-4c4b-bf5b-0528240cd532\") " pod="kserve-ci-e2e-test/gw-section-name-router-with-gat-2f0a622e-kserve-798cdbd559pb2cr" Jul 10 18:52:52.589248 ip-10-0-143-17 kubenswrapper[2585]: I0710 18:52:52.589216 2585 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/77ac2342-1d50-4c4b-bf5b-0528240cd532-dshm\") pod \"gw-section-name-router-with-gat-2f0a622e-kserve-798cdbd559pb2cr\" (UID: \"77ac2342-1d50-4c4b-bf5b-0528240cd532\") " pod="kserve-ci-e2e-test/gw-section-name-router-with-gat-2f0a622e-kserve-798cdbd559pb2cr" Jul 10 18:52:52.589248 ip-10-0-143-17 kubenswrapper[2585]: I0710 18:52:52.589237 2585 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/77ac2342-1d50-4c4b-bf5b-0528240cd532-tls-certs\") pod \"gw-section-name-router-with-gat-2f0a622e-kserve-798cdbd559pb2cr\" (UID: \"77ac2342-1d50-4c4b-bf5b-0528240cd532\") " pod="kserve-ci-e2e-test/gw-section-name-router-with-gat-2f0a622e-kserve-798cdbd559pb2cr" Jul 10 18:52:52.589525 ip-10-0-143-17 kubenswrapper[2585]: I0710 18:52:52.589273 2585 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/77ac2342-1d50-4c4b-bf5b-0528240cd532-tmp-dir\") pod \"gw-section-name-router-with-gat-2f0a622e-kserve-798cdbd559pb2cr\" (UID: \"77ac2342-1d50-4c4b-bf5b-0528240cd532\") " pod="kserve-ci-e2e-test/gw-section-name-router-with-gat-2f0a622e-kserve-798cdbd559pb2cr" Jul 10 18:52:52.589525 ip-10-0-143-17 kubenswrapper[2585]: I0710 18:52:52.589294 2585 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kserve-provision-location\" (UniqueName: \"kubernetes.io/empty-dir/77ac2342-1d50-4c4b-bf5b-0528240cd532-kserve-provision-location\") pod \"gw-section-name-router-with-gat-2f0a622e-kserve-798cdbd559pb2cr\" (UID: \"77ac2342-1d50-4c4b-bf5b-0528240cd532\") " pod="kserve-ci-e2e-test/gw-section-name-router-with-gat-2f0a622e-kserve-798cdbd559pb2cr" Jul 10 18:52:52.589525 ip-10-0-143-17 kubenswrapper[2585]: I0710 18:52:52.589314 2585 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/77ac2342-1d50-4c4b-bf5b-0528240cd532-home\") pod \"gw-section-name-router-with-gat-2f0a622e-kserve-798cdbd559pb2cr\" (UID: \"77ac2342-1d50-4c4b-bf5b-0528240cd532\") " pod="kserve-ci-e2e-test/gw-section-name-router-with-gat-2f0a622e-kserve-798cdbd559pb2cr" Jul 10 18:52:52.589525 ip-10-0-143-17 kubenswrapper[2585]: I0710 18:52:52.589336 2585 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-pdrds\" (UniqueName: \"kubernetes.io/projected/77ac2342-1d50-4c4b-bf5b-0528240cd532-kube-api-access-pdrds\") pod \"gw-section-name-router-with-gat-2f0a622e-kserve-798cdbd559pb2cr\" (UID: \"77ac2342-1d50-4c4b-bf5b-0528240cd532\") " pod="kserve-ci-e2e-test/gw-section-name-router-with-gat-2f0a622e-kserve-798cdbd559pb2cr" Jul 10 18:52:52.589767 ip-10-0-143-17 kubenswrapper[2585]: I0710 18:52:52.589617 2585 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/77ac2342-1d50-4c4b-bf5b-0528240cd532-model-cache\") pod \"gw-section-name-router-with-gat-2f0a622e-kserve-798cdbd559pb2cr\" (UID: \"77ac2342-1d50-4c4b-bf5b-0528240cd532\") " pod="kserve-ci-e2e-test/gw-section-name-router-with-gat-2f0a622e-kserve-798cdbd559pb2cr" Jul 10 18:52:52.589767 ip-10-0-143-17 kubenswrapper[2585]: I0710 18:52:52.589652 2585 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/77ac2342-1d50-4c4b-bf5b-0528240cd532-tmp-dir\") pod \"gw-section-name-router-with-gat-2f0a622e-kserve-798cdbd559pb2cr\" (UID: \"77ac2342-1d50-4c4b-bf5b-0528240cd532\") " pod="kserve-ci-e2e-test/gw-section-name-router-with-gat-2f0a622e-kserve-798cdbd559pb2cr" Jul 10 18:52:52.589767 ip-10-0-143-17 kubenswrapper[2585]: I0710 18:52:52.589761 2585 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/77ac2342-1d50-4c4b-bf5b-0528240cd532-home\") pod \"gw-section-name-router-with-gat-2f0a622e-kserve-798cdbd559pb2cr\" (UID: \"77ac2342-1d50-4c4b-bf5b-0528240cd532\") " pod="kserve-ci-e2e-test/gw-section-name-router-with-gat-2f0a622e-kserve-798cdbd559pb2cr" Jul 10 18:52:52.589866 ip-10-0-143-17 kubenswrapper[2585]: I0710 18:52:52.589768 2585 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kserve-provision-location\" (UniqueName: \"kubernetes.io/empty-dir/77ac2342-1d50-4c4b-bf5b-0528240cd532-kserve-provision-location\") pod \"gw-section-name-router-with-gat-2f0a622e-kserve-798cdbd559pb2cr\" (UID: \"77ac2342-1d50-4c4b-bf5b-0528240cd532\") " pod="kserve-ci-e2e-test/gw-section-name-router-with-gat-2f0a622e-kserve-798cdbd559pb2cr" Jul 10 18:52:52.591626 ip-10-0-143-17 kubenswrapper[2585]: I0710 18:52:52.591603 2585 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/77ac2342-1d50-4c4b-bf5b-0528240cd532-dshm\") pod \"gw-section-name-router-with-gat-2f0a622e-kserve-798cdbd559pb2cr\" (UID: \"77ac2342-1d50-4c4b-bf5b-0528240cd532\") " pod="kserve-ci-e2e-test/gw-section-name-router-with-gat-2f0a622e-kserve-798cdbd559pb2cr" Jul 10 18:52:52.591903 ip-10-0-143-17 kubenswrapper[2585]: I0710 18:52:52.591881 2585 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/77ac2342-1d50-4c4b-bf5b-0528240cd532-tls-certs\") pod \"gw-section-name-router-with-gat-2f0a622e-kserve-798cdbd559pb2cr\" (UID: \"77ac2342-1d50-4c4b-bf5b-0528240cd532\") " pod="kserve-ci-e2e-test/gw-section-name-router-with-gat-2f0a622e-kserve-798cdbd559pb2cr" Jul 10 18:52:52.599170 ip-10-0-143-17 kubenswrapper[2585]: I0710 18:52:52.599149 2585 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-pdrds\" (UniqueName: \"kubernetes.io/projected/77ac2342-1d50-4c4b-bf5b-0528240cd532-kube-api-access-pdrds\") pod \"gw-section-name-router-with-gat-2f0a622e-kserve-798cdbd559pb2cr\" (UID: \"77ac2342-1d50-4c4b-bf5b-0528240cd532\") " pod="kserve-ci-e2e-test/gw-section-name-router-with-gat-2f0a622e-kserve-798cdbd559pb2cr" Jul 10 18:52:52.632727 ip-10-0-143-17 kubenswrapper[2585]: I0710 18:52:52.632705 2585 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kserve-ci-e2e-test/gw-section-name-router-with-gat-2f0a622e-kserve-798cdbd559pb2cr" Jul 10 18:52:52.817714 ip-10-0-143-17 kubenswrapper[2585]: I0710 18:52:52.817683 2585 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kserve-ci-e2e-test/gw-section-name-router-with-gat-2f0a622e-kserve-798cdbd559pb2cr"] Jul 10 18:52:52.820853 ip-10-0-143-17 kubenswrapper[2585]: W0710 18:52:52.820827 2585 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod77ac2342_1d50_4c4b_bf5b_0528240cd532.slice/crio-51a106b21d2bec78c59538f7101035ee9c71461fe74f0ec43c7df2f231e16ae3 WatchSource:0}: Error finding container 51a106b21d2bec78c59538f7101035ee9c71461fe74f0ec43c7df2f231e16ae3: Status 404 returned error can't find the container with id 51a106b21d2bec78c59538f7101035ee9c71461fe74f0ec43c7df2f231e16ae3 Jul 10 18:52:52.822578 ip-10-0-143-17 kubenswrapper[2585]: I0710 18:52:52.822561 2585 provider.go:93] Refreshing cache for provider: *credentialprovider.defaultDockerConfigProvider Jul 10 18:52:53.806279 ip-10-0-143-17 kubenswrapper[2585]: I0710 18:52:53.806235 2585 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kserve-ci-e2e-test/gw-section-name-router-with-gat-2f0a622e-kserve-798cdbd559pb2cr" event={"ID":"77ac2342-1d50-4c4b-bf5b-0528240cd532","Type":"ContainerStarted","Data":"51a106b21d2bec78c59538f7101035ee9c71461fe74f0ec43c7df2f231e16ae3"} Jul 10 18:52:57.832452 ip-10-0-143-17 kubenswrapper[2585]: I0710 18:52:57.832371 2585 generic.go:356] "Generic (PLEG): container finished" podID="27c2089c-3b42-41ba-91c3-c6f72e532db5" containerID="b51395a6803413c98cfe8f77513f733dcdaa778580d9b75c7002cc664c75e0e3" exitCode=0 Jul 10 18:52:57.832452 ip-10-0-143-17 kubenswrapper[2585]: I0710 18:52:57.832434 2585 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/openshift-ai-inference-openshift-default-6c48cf8bcc-cl5g8" event={"ID":"27c2089c-3b42-41ba-91c3-c6f72e532db5","Type":"ContainerDied","Data":"b51395a6803413c98cfe8f77513f733dcdaa778580d9b75c7002cc664c75e0e3"} Jul 10 18:52:57.833189 ip-10-0-143-17 kubenswrapper[2585]: I0710 18:52:57.833021 2585 scope.go:117] "RemoveContainer" containerID="b51395a6803413c98cfe8f77513f733dcdaa778580d9b75c7002cc664c75e0e3" Jul 10 18:52:57.834324 ip-10-0-143-17 kubenswrapper[2585]: I0710 18:52:57.834292 2585 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236224Ki","pods":"250"} Jul 10 18:52:57.834397 ip-10-0-143-17 kubenswrapper[2585]: I0710 18:52:57.834366 2585 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236224Ki","pods":"250"} Jul 10 18:52:57.834433 ip-10-0-143-17 kubenswrapper[2585]: I0710 18:52:57.834397 2585 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236224Ki","pods":"250"} Jul 10 18:52:57.835517 ip-10-0-143-17 kubenswrapper[2585]: I0710 18:52:57.835488 2585 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kserve-ci-e2e-test/gw-section-name-router-with-gat-2f0a622e-kserve-798cdbd559pb2cr" event={"ID":"77ac2342-1d50-4c4b-bf5b-0528240cd532","Type":"ContainerStarted","Data":"1ffe78473577506bd62ac51fa0d14d3396b6b510e9f3839c93a535bf05e1e05e"} Jul 10 18:52:58.843189 ip-10-0-143-17 kubenswrapper[2585]: I0710 18:52:58.843159 2585 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/openshift-ai-inference-openshift-default-6c48cf8bcc-cl5g8" event={"ID":"27c2089c-3b42-41ba-91c3-c6f72e532db5","Type":"ContainerStarted","Data":"7a0c813ecf40e213c85ca65c1059ad1b33a3cfacd2b7129adaab027fe0f9b18c"} Jul 10 18:52:59.000033 ip-10-0-143-17 kubenswrapper[2585]: I0710 18:52:58.999985 2585 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="openshift-ingress/openshift-ai-inference-openshift-default-6c48cf8bcc-cl5g8" Jul 10 18:52:59.000203 ip-10-0-143-17 kubenswrapper[2585]: I0710 18:52:59.000170 2585 patch_prober.go:28] interesting pod/openshift-ai-inference-openshift-default-6c48cf8bcc-cl5g8 container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.133.0.34:15021/healthz/ready\": dial tcp 10.133.0.34:15021: connect: connection refused" start-of-body= Jul 10 18:52:59.000257 ip-10-0-143-17 kubenswrapper[2585]: I0710 18:52:59.000217 2585 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/openshift-ai-inference-openshift-default-6c48cf8bcc-cl5g8" podUID="27c2089c-3b42-41ba-91c3-c6f72e532db5" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.133.0.34:15021/healthz/ready\": dial tcp 10.133.0.34:15021: connect: connection refused" Jul 10 18:53:00.000580 ip-10-0-143-17 kubenswrapper[2585]: I0710 18:53:00.000526 2585 patch_prober.go:28] interesting pod/openshift-ai-inference-openshift-default-6c48cf8bcc-cl5g8 container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.133.0.34:15021/healthz/ready\": dial tcp 10.133.0.34:15021: connect: connection refused" start-of-body= Jul 10 18:53:00.001003 ip-10-0-143-17 kubenswrapper[2585]: I0710 18:53:00.000649 2585 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/openshift-ai-inference-openshift-default-6c48cf8bcc-cl5g8" podUID="27c2089c-3b42-41ba-91c3-c6f72e532db5" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.133.0.34:15021/healthz/ready\": dial tcp 10.133.0.34:15021: connect: connection refused" Jul 10 18:53:01.000713 ip-10-0-143-17 kubenswrapper[2585]: I0710 18:53:01.000667 2585 patch_prober.go:28] interesting pod/openshift-ai-inference-openshift-default-6c48cf8bcc-cl5g8 container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.133.0.34:15021/healthz/ready\": dial tcp 10.133.0.34:15021: connect: connection refused" start-of-body= Jul 10 18:53:01.001106 ip-10-0-143-17 kubenswrapper[2585]: I0710 18:53:01.000743 2585 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/openshift-ai-inference-openshift-default-6c48cf8bcc-cl5g8" podUID="27c2089c-3b42-41ba-91c3-c6f72e532db5" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.133.0.34:15021/healthz/ready\": dial tcp 10.133.0.34:15021: connect: connection refused" Jul 10 18:53:02.000245 ip-10-0-143-17 kubenswrapper[2585]: I0710 18:53:02.000201 2585 patch_prober.go:28] interesting pod/openshift-ai-inference-openshift-default-6c48cf8bcc-cl5g8 container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.133.0.34:15021/healthz/ready\": dial tcp 10.133.0.34:15021: connect: connection refused" start-of-body= Jul 10 18:53:02.000593 ip-10-0-143-17 kubenswrapper[2585]: I0710 18:53:02.000505 2585 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/openshift-ai-inference-openshift-default-6c48cf8bcc-cl5g8" podUID="27c2089c-3b42-41ba-91c3-c6f72e532db5" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.133.0.34:15021/healthz/ready\": dial tcp 10.133.0.34:15021: connect: connection refused" Jul 10 18:53:02.877521 ip-10-0-143-17 kubenswrapper[2585]: I0710 18:53:02.877416 2585 generic.go:356] "Generic (PLEG): container finished" podID="77ac2342-1d50-4c4b-bf5b-0528240cd532" containerID="1ffe78473577506bd62ac51fa0d14d3396b6b510e9f3839c93a535bf05e1e05e" exitCode=0 Jul 10 18:53:02.877521 ip-10-0-143-17 kubenswrapper[2585]: I0710 18:53:02.877495 2585 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kserve-ci-e2e-test/gw-section-name-router-with-gat-2f0a622e-kserve-798cdbd559pb2cr" event={"ID":"77ac2342-1d50-4c4b-bf5b-0528240cd532","Type":"ContainerDied","Data":"1ffe78473577506bd62ac51fa0d14d3396b6b510e9f3839c93a535bf05e1e05e"} Jul 10 18:53:02.879212 ip-10-0-143-17 kubenswrapper[2585]: I0710 18:53:02.879187 2585 generic.go:356] "Generic (PLEG): container finished" podID="10d311d2-b0ff-486f-a709-ef0f52dfa327" containerID="282ee7209dc4660a0f48a5be87cc0dad51cfb2d9acdbc0eaa53b7515210aafbf" exitCode=0 Jul 10 18:53:02.879315 ip-10-0-143-17 kubenswrapper[2585]: I0710 18:53:02.879255 2585 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kserve-ci-e2e-test/router-gateway-1-openshift-default-799f46c59b-sf6qk" event={"ID":"10d311d2-b0ff-486f-a709-ef0f52dfa327","Type":"ContainerDied","Data":"282ee7209dc4660a0f48a5be87cc0dad51cfb2d9acdbc0eaa53b7515210aafbf"} Jul 10 18:53:02.879673 ip-10-0-143-17 kubenswrapper[2585]: I0710 18:53:02.879659 2585 scope.go:117] "RemoveContainer" containerID="282ee7209dc4660a0f48a5be87cc0dad51cfb2d9acdbc0eaa53b7515210aafbf" Jul 10 18:53:02.880510 ip-10-0-143-17 kubenswrapper[2585]: I0710 18:53:02.880475 2585 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236224Ki","pods":"250"} Jul 10 18:53:02.880632 ip-10-0-143-17 kubenswrapper[2585]: I0710 18:53:02.880590 2585 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236224Ki","pods":"250"} Jul 10 18:53:02.880679 ip-10-0-143-17 kubenswrapper[2585]: I0710 18:53:02.880662 2585 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236224Ki","pods":"250"} Jul 10 18:53:03.000040 ip-10-0-143-17 kubenswrapper[2585]: I0710 18:53:02.999952 2585 patch_prober.go:28] interesting pod/openshift-ai-inference-openshift-default-6c48cf8bcc-cl5g8 container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.133.0.34:15021/healthz/ready\": dial tcp 10.133.0.34:15021: connect: connection refused" start-of-body= Jul 10 18:53:03.000040 ip-10-0-143-17 kubenswrapper[2585]: I0710 18:53:03.000018 2585 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/openshift-ai-inference-openshift-default-6c48cf8bcc-cl5g8" podUID="27c2089c-3b42-41ba-91c3-c6f72e532db5" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.133.0.34:15021/healthz/ready\": dial tcp 10.133.0.34:15021: connect: connection refused" Jul 10 18:53:03.898348 ip-10-0-143-17 kubenswrapper[2585]: I0710 18:53:03.898312 2585 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kserve-ci-e2e-test/router-gateway-1-openshift-default-799f46c59b-sf6qk" event={"ID":"10d311d2-b0ff-486f-a709-ef0f52dfa327","Type":"ContainerStarted","Data":"0528101d39e064d4090a7d6c2d31c670085c1b7d6b6b56d8e442e17cafa0e888"} Jul 10 18:53:04.000744 ip-10-0-143-17 kubenswrapper[2585]: I0710 18:53:04.000605 2585 patch_prober.go:28] interesting pod/openshift-ai-inference-openshift-default-6c48cf8bcc-cl5g8 container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.133.0.34:15021/healthz/ready\": dial tcp 10.133.0.34:15021: connect: connection refused" start-of-body= Jul 10 18:53:04.000744 ip-10-0-143-17 kubenswrapper[2585]: I0710 18:53:04.000674 2585 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/openshift-ai-inference-openshift-default-6c48cf8bcc-cl5g8" podUID="27c2089c-3b42-41ba-91c3-c6f72e532db5" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.133.0.34:15021/healthz/ready\": dial tcp 10.133.0.34:15021: connect: connection refused" Jul 10 18:53:04.572496 ip-10-0-143-17 kubenswrapper[2585]: I0710 18:53:04.572462 2585 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="kserve-ci-e2e-test/router-gateway-1-openshift-default-799f46c59b-sf6qk" Jul 10 18:53:04.572901 ip-10-0-143-17 kubenswrapper[2585]: I0710 18:53:04.572870 2585 prober.go:120] "Probe failed" probeType="Startup" pod="kserve-ci-e2e-test/router-gateway-1-openshift-default-799f46c59b-sf6qk" podUID="10d311d2-b0ff-486f-a709-ef0f52dfa327" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.133.0.44:15021/healthz/ready\": dial tcp 10.133.0.44:15021: connect: connection refused" Jul 10 18:53:04.906020 ip-10-0-143-17 kubenswrapper[2585]: I0710 18:53:04.905982 2585 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kserve-ci-e2e-test/gw-section-name-router-with-gat-2f0a622e-kserve-798cdbd559pb2cr" event={"ID":"77ac2342-1d50-4c4b-bf5b-0528240cd532","Type":"ContainerStarted","Data":"7a54dae4a18b610ab4b743a789ad8eef2e6b0653650ac239cbd8f669f571ed55"} Jul 10 18:53:04.931750 ip-10-0-143-17 kubenswrapper[2585]: I0710 18:53:04.931695 2585 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="kserve-ci-e2e-test/gw-section-name-router-with-gat-2f0a622e-kserve-798cdbd559pb2cr" podStartSLOduration=2.168426693 podStartE2EDuration="12.931680831s" totalImagesPullingTime="5.52102574s" totalInitContainerRuntime="5.242228398s" isStatefulPod=false podCreationTimestamp="2026-07-10 18:52:52 +0000 UTC" imagePullSessionsCount=2 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-10 18:53:04.929982786 +0000 UTC m=+798.387342762" watchObservedRunningTime="2026-07-10 18:53:04.931680831 +0000 UTC m=+798.389040766" Jul 10 18:53:05.004642 ip-10-0-143-17 kubenswrapper[2585]: I0710 18:53:05.004611 2585 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="openshift-ingress/openshift-ai-inference-openshift-default-6c48cf8bcc-cl5g8" Jul 10 18:53:05.004892 ip-10-0-143-17 kubenswrapper[2585]: I0710 18:53:05.004862 2585 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/openshift-ai-inference-openshift-default-6c48cf8bcc-cl5g8" Jul 10 18:53:05.005598 ip-10-0-143-17 kubenswrapper[2585]: I0710 18:53:05.005582 2585 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/openshift-ai-inference-openshift-default-6c48cf8bcc-cl5g8" Jul 10 18:53:05.572197 ip-10-0-143-17 kubenswrapper[2585]: I0710 18:53:05.572153 2585 prober.go:120] "Probe failed" probeType="Startup" pod="kserve-ci-e2e-test/router-gateway-1-openshift-default-799f46c59b-sf6qk" podUID="10d311d2-b0ff-486f-a709-ef0f52dfa327" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.133.0.44:15021/healthz/ready\": dial tcp 10.133.0.44:15021: connect: connection refused" Jul 10 18:53:06.572344 ip-10-0-143-17 kubenswrapper[2585]: I0710 18:53:06.572302 2585 prober.go:120] "Probe failed" probeType="Startup" pod="kserve-ci-e2e-test/router-gateway-1-openshift-default-799f46c59b-sf6qk" podUID="10d311d2-b0ff-486f-a709-ef0f52dfa327" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.133.0.44:15021/healthz/ready\": dial tcp 10.133.0.44:15021: connect: connection refused" Jul 10 18:53:07.572073 ip-10-0-143-17 kubenswrapper[2585]: I0710 18:53:07.572027 2585 prober.go:120] "Probe failed" probeType="Startup" pod="kserve-ci-e2e-test/router-gateway-1-openshift-default-799f46c59b-sf6qk" podUID="10d311d2-b0ff-486f-a709-ef0f52dfa327" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.133.0.44:15021/healthz/ready\": dial tcp 10.133.0.44:15021: connect: connection refused" Jul 10 18:53:08.572688 ip-10-0-143-17 kubenswrapper[2585]: I0710 18:53:08.572643 2585 prober.go:120] "Probe failed" probeType="Startup" pod="kserve-ci-e2e-test/router-gateway-1-openshift-default-799f46c59b-sf6qk" podUID="10d311d2-b0ff-486f-a709-ef0f52dfa327" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.133.0.44:15021/healthz/ready\": dial tcp 10.133.0.44:15021: connect: connection refused" Jul 10 18:53:09.576427 ip-10-0-143-17 kubenswrapper[2585]: I0710 18:53:09.576352 2585 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="kserve-ci-e2e-test/router-gateway-1-openshift-default-799f46c59b-sf6qk" Jul 10 18:53:09.576796 ip-10-0-143-17 kubenswrapper[2585]: I0710 18:53:09.576655 2585 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="kserve-ci-e2e-test/router-gateway-1-openshift-default-799f46c59b-sf6qk" Jul 10 18:53:09.577272 ip-10-0-143-17 kubenswrapper[2585]: I0710 18:53:09.577254 2585 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="kserve-ci-e2e-test/router-gateway-1-openshift-default-799f46c59b-sf6qk" Jul 10 18:53:12.005644 ip-10-0-143-17 kubenswrapper[2585]: I0710 18:53:12.005612 2585 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["kserve-ci-e2e-test/gw-section-name-router-with-gat-2f0a622e-kserve-798cdbd559pb2cr"] Jul 10 18:53:12.006009 ip-10-0-143-17 kubenswrapper[2585]: I0710 18:53:12.005830 2585 kuberuntime_container.go:873] "Killing container with a grace period" pod="kserve-ci-e2e-test/gw-section-name-router-with-gat-2f0a622e-kserve-798cdbd559pb2cr" podUID="77ac2342-1d50-4c4b-bf5b-0528240cd532" containerName="main" containerID="cri-o://7a54dae4a18b610ab4b743a789ad8eef2e6b0653650ac239cbd8f669f571ed55" gracePeriod=30 Jul 10 18:53:12.297909 ip-10-0-143-17 kubenswrapper[2585]: I0710 18:53:12.297881 2585 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="kserve-ci-e2e-test/gw-section-name-router-with-gat-2f0a622e-kserve-798cdbd559pb2cr" Jul 10 18:53:12.372233 ip-10-0-143-17 kubenswrapper[2585]: I0710 18:53:12.372204 2585 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/77ac2342-1d50-4c4b-bf5b-0528240cd532-home\") pod \"77ac2342-1d50-4c4b-bf5b-0528240cd532\" (UID: \"77ac2342-1d50-4c4b-bf5b-0528240cd532\") " Jul 10 18:53:12.372348 ip-10-0-143-17 kubenswrapper[2585]: I0710 18:53:12.372242 2585 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/77ac2342-1d50-4c4b-bf5b-0528240cd532-tmp-dir\") pod \"77ac2342-1d50-4c4b-bf5b-0528240cd532\" (UID: \"77ac2342-1d50-4c4b-bf5b-0528240cd532\") " Jul 10 18:53:12.372348 ip-10-0-143-17 kubenswrapper[2585]: I0710 18:53:12.372267 2585 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-pdrds\" (UniqueName: \"kubernetes.io/projected/77ac2342-1d50-4c4b-bf5b-0528240cd532-kube-api-access-pdrds\") pod \"77ac2342-1d50-4c4b-bf5b-0528240cd532\" (UID: \"77ac2342-1d50-4c4b-bf5b-0528240cd532\") " Jul 10 18:53:12.372348 ip-10-0-143-17 kubenswrapper[2585]: I0710 18:53:12.372300 2585 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/77ac2342-1d50-4c4b-bf5b-0528240cd532-dshm\") pod \"77ac2342-1d50-4c4b-bf5b-0528240cd532\" (UID: \"77ac2342-1d50-4c4b-bf5b-0528240cd532\") " Jul 10 18:53:12.372485 ip-10-0-143-17 kubenswrapper[2585]: I0710 18:53:12.372345 2585 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kserve-provision-location\" (UniqueName: \"kubernetes.io/empty-dir/77ac2342-1d50-4c4b-bf5b-0528240cd532-kserve-provision-location\") pod \"77ac2342-1d50-4c4b-bf5b-0528240cd532\" (UID: \"77ac2342-1d50-4c4b-bf5b-0528240cd532\") " Jul 10 18:53:12.372485 ip-10-0-143-17 kubenswrapper[2585]: I0710 18:53:12.372407 2585 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/77ac2342-1d50-4c4b-bf5b-0528240cd532-model-cache\") pod \"77ac2342-1d50-4c4b-bf5b-0528240cd532\" (UID: \"77ac2342-1d50-4c4b-bf5b-0528240cd532\") " Jul 10 18:53:12.372485 ip-10-0-143-17 kubenswrapper[2585]: I0710 18:53:12.372457 2585 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/77ac2342-1d50-4c4b-bf5b-0528240cd532-tls-certs\") pod \"77ac2342-1d50-4c4b-bf5b-0528240cd532\" (UID: \"77ac2342-1d50-4c4b-bf5b-0528240cd532\") " Jul 10 18:53:12.372670 ip-10-0-143-17 kubenswrapper[2585]: I0710 18:53:12.372489 2585 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/77ac2342-1d50-4c4b-bf5b-0528240cd532-home" (OuterVolumeSpecName: "home") pod "77ac2342-1d50-4c4b-bf5b-0528240cd532" (UID: "77ac2342-1d50-4c4b-bf5b-0528240cd532"). InnerVolumeSpecName "home". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Jul 10 18:53:12.372670 ip-10-0-143-17 kubenswrapper[2585]: I0710 18:53:12.372503 2585 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/77ac2342-1d50-4c4b-bf5b-0528240cd532-tmp-dir" (OuterVolumeSpecName: "tmp-dir") pod "77ac2342-1d50-4c4b-bf5b-0528240cd532" (UID: "77ac2342-1d50-4c4b-bf5b-0528240cd532"). InnerVolumeSpecName "tmp-dir". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Jul 10 18:53:12.372783 ip-10-0-143-17 kubenswrapper[2585]: I0710 18:53:12.372700 2585 reconciler_common.go:299] "Volume detached for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/77ac2342-1d50-4c4b-bf5b-0528240cd532-home\") on node \"ip-10-0-143-17.ec2.internal\" DevicePath \"\"" Jul 10 18:53:12.372783 ip-10-0-143-17 kubenswrapper[2585]: I0710 18:53:12.372721 2585 reconciler_common.go:299] "Volume detached for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/77ac2342-1d50-4c4b-bf5b-0528240cd532-tmp-dir\") on node \"ip-10-0-143-17.ec2.internal\" DevicePath \"\"" Jul 10 18:53:12.372783 ip-10-0-143-17 kubenswrapper[2585]: I0710 18:53:12.372756 2585 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/77ac2342-1d50-4c4b-bf5b-0528240cd532-model-cache" (OuterVolumeSpecName: "model-cache") pod "77ac2342-1d50-4c4b-bf5b-0528240cd532" (UID: "77ac2342-1d50-4c4b-bf5b-0528240cd532"). InnerVolumeSpecName "model-cache". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Jul 10 18:53:12.374615 ip-10-0-143-17 kubenswrapper[2585]: I0710 18:53:12.374584 2585 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/77ac2342-1d50-4c4b-bf5b-0528240cd532-kube-api-access-pdrds" (OuterVolumeSpecName: "kube-api-access-pdrds") pod "77ac2342-1d50-4c4b-bf5b-0528240cd532" (UID: "77ac2342-1d50-4c4b-bf5b-0528240cd532"). InnerVolumeSpecName "kube-api-access-pdrds". PluginName "kubernetes.io/projected", VolumeGIDValue "" Jul 10 18:53:12.374615 ip-10-0-143-17 kubenswrapper[2585]: I0710 18:53:12.374607 2585 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/77ac2342-1d50-4c4b-bf5b-0528240cd532-tls-certs" (OuterVolumeSpecName: "tls-certs") pod "77ac2342-1d50-4c4b-bf5b-0528240cd532" (UID: "77ac2342-1d50-4c4b-bf5b-0528240cd532"). InnerVolumeSpecName "tls-certs". PluginName "kubernetes.io/secret", VolumeGIDValue "" Jul 10 18:53:12.374761 ip-10-0-143-17 kubenswrapper[2585]: I0710 18:53:12.374645 2585 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/77ac2342-1d50-4c4b-bf5b-0528240cd532-dshm" (OuterVolumeSpecName: "dshm") pod "77ac2342-1d50-4c4b-bf5b-0528240cd532" (UID: "77ac2342-1d50-4c4b-bf5b-0528240cd532"). InnerVolumeSpecName "dshm". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Jul 10 18:53:12.435264 ip-10-0-143-17 kubenswrapper[2585]: I0710 18:53:12.435228 2585 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/77ac2342-1d50-4c4b-bf5b-0528240cd532-kserve-provision-location" (OuterVolumeSpecName: "kserve-provision-location") pod "77ac2342-1d50-4c4b-bf5b-0528240cd532" (UID: "77ac2342-1d50-4c4b-bf5b-0528240cd532"). InnerVolumeSpecName "kserve-provision-location". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Jul 10 18:53:12.473750 ip-10-0-143-17 kubenswrapper[2585]: I0710 18:53:12.473725 2585 reconciler_common.go:299] "Volume detached for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/77ac2342-1d50-4c4b-bf5b-0528240cd532-model-cache\") on node \"ip-10-0-143-17.ec2.internal\" DevicePath \"\"" Jul 10 18:53:12.473750 ip-10-0-143-17 kubenswrapper[2585]: I0710 18:53:12.473749 2585 reconciler_common.go:299] "Volume detached for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/77ac2342-1d50-4c4b-bf5b-0528240cd532-tls-certs\") on node \"ip-10-0-143-17.ec2.internal\" DevicePath \"\"" Jul 10 18:53:12.473902 ip-10-0-143-17 kubenswrapper[2585]: I0710 18:53:12.473762 2585 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-pdrds\" (UniqueName: \"kubernetes.io/projected/77ac2342-1d50-4c4b-bf5b-0528240cd532-kube-api-access-pdrds\") on node \"ip-10-0-143-17.ec2.internal\" DevicePath \"\"" Jul 10 18:53:12.473902 ip-10-0-143-17 kubenswrapper[2585]: I0710 18:53:12.473773 2585 reconciler_common.go:299] "Volume detached for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/77ac2342-1d50-4c4b-bf5b-0528240cd532-dshm\") on node \"ip-10-0-143-17.ec2.internal\" DevicePath \"\"" Jul 10 18:53:12.473902 ip-10-0-143-17 kubenswrapper[2585]: I0710 18:53:12.473782 2585 reconciler_common.go:299] "Volume detached for volume \"kserve-provision-location\" (UniqueName: \"kubernetes.io/empty-dir/77ac2342-1d50-4c4b-bf5b-0528240cd532-kserve-provision-location\") on node \"ip-10-0-143-17.ec2.internal\" DevicePath \"\"" Jul 10 18:53:12.946554 ip-10-0-143-17 kubenswrapper[2585]: I0710 18:53:12.946510 2585 generic.go:356] "Generic (PLEG): container finished" podID="77ac2342-1d50-4c4b-bf5b-0528240cd532" containerID="7a54dae4a18b610ab4b743a789ad8eef2e6b0653650ac239cbd8f669f571ed55" exitCode=0 Jul 10 18:53:12.946721 ip-10-0-143-17 kubenswrapper[2585]: I0710 18:53:12.946584 2585 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="kserve-ci-e2e-test/gw-section-name-router-with-gat-2f0a622e-kserve-798cdbd559pb2cr" Jul 10 18:53:12.946721 ip-10-0-143-17 kubenswrapper[2585]: I0710 18:53:12.946601 2585 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kserve-ci-e2e-test/gw-section-name-router-with-gat-2f0a622e-kserve-798cdbd559pb2cr" event={"ID":"77ac2342-1d50-4c4b-bf5b-0528240cd532","Type":"ContainerDied","Data":"7a54dae4a18b610ab4b743a789ad8eef2e6b0653650ac239cbd8f669f571ed55"} Jul 10 18:53:12.946721 ip-10-0-143-17 kubenswrapper[2585]: I0710 18:53:12.946655 2585 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kserve-ci-e2e-test/gw-section-name-router-with-gat-2f0a622e-kserve-798cdbd559pb2cr" event={"ID":"77ac2342-1d50-4c4b-bf5b-0528240cd532","Type":"ContainerDied","Data":"51a106b21d2bec78c59538f7101035ee9c71461fe74f0ec43c7df2f231e16ae3"} Jul 10 18:53:12.946721 ip-10-0-143-17 kubenswrapper[2585]: I0710 18:53:12.946679 2585 scope.go:117] "RemoveContainer" containerID="7a54dae4a18b610ab4b743a789ad8eef2e6b0653650ac239cbd8f669f571ed55" Jul 10 18:53:12.958236 ip-10-0-143-17 kubenswrapper[2585]: I0710 18:53:12.958211 2585 scope.go:117] "RemoveContainer" containerID="1ffe78473577506bd62ac51fa0d14d3396b6b510e9f3839c93a535bf05e1e05e" Jul 10 18:53:12.967935 ip-10-0-143-17 kubenswrapper[2585]: I0710 18:53:12.967915 2585 scope.go:117] "RemoveContainer" containerID="7a54dae4a18b610ab4b743a789ad8eef2e6b0653650ac239cbd8f669f571ed55" Jul 10 18:53:12.968160 ip-10-0-143-17 kubenswrapper[2585]: E0710 18:53:12.968144 2585 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"7a54dae4a18b610ab4b743a789ad8eef2e6b0653650ac239cbd8f669f571ed55\": container with ID starting with 7a54dae4a18b610ab4b743a789ad8eef2e6b0653650ac239cbd8f669f571ed55 not found: ID does not exist" containerID="7a54dae4a18b610ab4b743a789ad8eef2e6b0653650ac239cbd8f669f571ed55" Jul 10 18:53:12.968207 ip-10-0-143-17 kubenswrapper[2585]: I0710 18:53:12.968168 2585 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"7a54dae4a18b610ab4b743a789ad8eef2e6b0653650ac239cbd8f669f571ed55"} err="failed to get container status \"7a54dae4a18b610ab4b743a789ad8eef2e6b0653650ac239cbd8f669f571ed55\": rpc error: code = NotFound desc = could not find container \"7a54dae4a18b610ab4b743a789ad8eef2e6b0653650ac239cbd8f669f571ed55\": container with ID starting with 7a54dae4a18b610ab4b743a789ad8eef2e6b0653650ac239cbd8f669f571ed55 not found: ID does not exist" Jul 10 18:53:12.968207 ip-10-0-143-17 kubenswrapper[2585]: I0710 18:53:12.968184 2585 scope.go:117] "RemoveContainer" containerID="1ffe78473577506bd62ac51fa0d14d3396b6b510e9f3839c93a535bf05e1e05e" Jul 10 18:53:12.968420 ip-10-0-143-17 kubenswrapper[2585]: E0710 18:53:12.968403 2585 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"1ffe78473577506bd62ac51fa0d14d3396b6b510e9f3839c93a535bf05e1e05e\": container with ID starting with 1ffe78473577506bd62ac51fa0d14d3396b6b510e9f3839c93a535bf05e1e05e not found: ID does not exist" containerID="1ffe78473577506bd62ac51fa0d14d3396b6b510e9f3839c93a535bf05e1e05e" Jul 10 18:53:12.968457 ip-10-0-143-17 kubenswrapper[2585]: I0710 18:53:12.968426 2585 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"1ffe78473577506bd62ac51fa0d14d3396b6b510e9f3839c93a535bf05e1e05e"} err="failed to get container status \"1ffe78473577506bd62ac51fa0d14d3396b6b510e9f3839c93a535bf05e1e05e\": rpc error: code = NotFound desc = could not find container \"1ffe78473577506bd62ac51fa0d14d3396b6b510e9f3839c93a535bf05e1e05e\": container with ID starting with 1ffe78473577506bd62ac51fa0d14d3396b6b510e9f3839c93a535bf05e1e05e not found: ID does not exist" Jul 10 18:53:12.973033 ip-10-0-143-17 kubenswrapper[2585]: I0710 18:53:12.973006 2585 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["kserve-ci-e2e-test/gw-section-name-router-with-gat-2f0a622e-kserve-798cdbd559pb2cr"] Jul 10 18:53:12.977034 ip-10-0-143-17 kubenswrapper[2585]: I0710 18:53:12.977005 2585 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["kserve-ci-e2e-test/gw-section-name-router-with-gat-2f0a622e-kserve-798cdbd559pb2cr"] Jul 10 18:53:13.216185 ip-10-0-143-17 kubenswrapper[2585]: I0710 18:53:13.216103 2585 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="77ac2342-1d50-4c4b-bf5b-0528240cd532" path="/var/lib/kubelet/pods/77ac2342-1d50-4c4b-bf5b-0528240cd532/volumes" Jul 10 18:53:15.623466 ip-10-0-143-17 kubenswrapper[2585]: I0710 18:53:15.623431 2585 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["kserve-ci-e2e-test/gw-section-name-router-with-gat-f1d92d0f-kserve-7d7fd586fbvqjx6"] Jul 10 18:53:15.623861 ip-10-0-143-17 kubenswrapper[2585]: I0710 18:53:15.623799 2585 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="77ac2342-1d50-4c4b-bf5b-0528240cd532" containerName="storage-initializer" Jul 10 18:53:15.623861 ip-10-0-143-17 kubenswrapper[2585]: I0710 18:53:15.623812 2585 state_mem.go:107] "Deleted CPUSet assignment" podUID="77ac2342-1d50-4c4b-bf5b-0528240cd532" containerName="storage-initializer" Jul 10 18:53:15.623861 ip-10-0-143-17 kubenswrapper[2585]: I0710 18:53:15.623836 2585 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="77ac2342-1d50-4c4b-bf5b-0528240cd532" containerName="main" Jul 10 18:53:15.623861 ip-10-0-143-17 kubenswrapper[2585]: I0710 18:53:15.623841 2585 state_mem.go:107] "Deleted CPUSet assignment" podUID="77ac2342-1d50-4c4b-bf5b-0528240cd532" containerName="main" Jul 10 18:53:15.623993 ip-10-0-143-17 kubenswrapper[2585]: I0710 18:53:15.623895 2585 memory_manager.go:368] "RemoveStaleState removing state" podUID="77ac2342-1d50-4c4b-bf5b-0528240cd532" containerName="main" Jul 10 18:53:15.627038 ip-10-0-143-17 kubenswrapper[2585]: I0710 18:53:15.627015 2585 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kserve-ci-e2e-test/gw-section-name-router-with-gat-f1d92d0f-kserve-7d7fd586fbvqjx6" Jul 10 18:53:15.630744 ip-10-0-143-17 kubenswrapper[2585]: I0710 18:53:15.630723 2585 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"kserve-ci-e2e-test\"/\"gw-sec2774c263d49959f50d9eebc552e13bf9-kserve-self-signed-certs\"" Jul 10 18:53:15.630744 ip-10-0-143-17 kubenswrapper[2585]: I0710 18:53:15.630730 2585 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"kserve-ci-e2e-test\"/\"default-dockercfg-jjdpb\"" Jul 10 18:53:15.642322 ip-10-0-143-17 kubenswrapper[2585]: I0710 18:53:15.642301 2585 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kserve-ci-e2e-test/gw-section-name-router-with-gat-f1d92d0f-kserve-7d7fd586fbvqjx6"] Jul 10 18:53:15.698667 ip-10-0-143-17 kubenswrapper[2585]: I0710 18:53:15.698639 2585 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/d19db5df-0a63-48f8-83fb-26210c0cc2fe-model-cache\") pod \"gw-section-name-router-with-gat-f1d92d0f-kserve-7d7fd586fbvqjx6\" (UID: \"d19db5df-0a63-48f8-83fb-26210c0cc2fe\") " pod="kserve-ci-e2e-test/gw-section-name-router-with-gat-f1d92d0f-kserve-7d7fd586fbvqjx6" Jul 10 18:53:15.698771 ip-10-0-143-17 kubenswrapper[2585]: I0710 18:53:15.698671 2585 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/d19db5df-0a63-48f8-83fb-26210c0cc2fe-tls-certs\") pod \"gw-section-name-router-with-gat-f1d92d0f-kserve-7d7fd586fbvqjx6\" (UID: \"d19db5df-0a63-48f8-83fb-26210c0cc2fe\") " pod="kserve-ci-e2e-test/gw-section-name-router-with-gat-f1d92d0f-kserve-7d7fd586fbvqjx6" Jul 10 18:53:15.698771 ip-10-0-143-17 kubenswrapper[2585]: I0710 18:53:15.698705 2585 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/d19db5df-0a63-48f8-83fb-26210c0cc2fe-dshm\") pod \"gw-section-name-router-with-gat-f1d92d0f-kserve-7d7fd586fbvqjx6\" (UID: \"d19db5df-0a63-48f8-83fb-26210c0cc2fe\") " pod="kserve-ci-e2e-test/gw-section-name-router-with-gat-f1d92d0f-kserve-7d7fd586fbvqjx6" Jul 10 18:53:15.698771 ip-10-0-143-17 kubenswrapper[2585]: I0710 18:53:15.698740 2585 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-f4jzk\" (UniqueName: \"kubernetes.io/projected/d19db5df-0a63-48f8-83fb-26210c0cc2fe-kube-api-access-f4jzk\") pod \"gw-section-name-router-with-gat-f1d92d0f-kserve-7d7fd586fbvqjx6\" (UID: \"d19db5df-0a63-48f8-83fb-26210c0cc2fe\") " pod="kserve-ci-e2e-test/gw-section-name-router-with-gat-f1d92d0f-kserve-7d7fd586fbvqjx6" Jul 10 18:53:15.698897 ip-10-0-143-17 kubenswrapper[2585]: I0710 18:53:15.698776 2585 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/d19db5df-0a63-48f8-83fb-26210c0cc2fe-home\") pod \"gw-section-name-router-with-gat-f1d92d0f-kserve-7d7fd586fbvqjx6\" (UID: \"d19db5df-0a63-48f8-83fb-26210c0cc2fe\") " pod="kserve-ci-e2e-test/gw-section-name-router-with-gat-f1d92d0f-kserve-7d7fd586fbvqjx6" Jul 10 18:53:15.698897 ip-10-0-143-17 kubenswrapper[2585]: I0710 18:53:15.698799 2585 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kserve-provision-location\" (UniqueName: \"kubernetes.io/empty-dir/d19db5df-0a63-48f8-83fb-26210c0cc2fe-kserve-provision-location\") pod \"gw-section-name-router-with-gat-f1d92d0f-kserve-7d7fd586fbvqjx6\" (UID: \"d19db5df-0a63-48f8-83fb-26210c0cc2fe\") " pod="kserve-ci-e2e-test/gw-section-name-router-with-gat-f1d92d0f-kserve-7d7fd586fbvqjx6" Jul 10 18:53:15.698897 ip-10-0-143-17 kubenswrapper[2585]: I0710 18:53:15.698854 2585 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/d19db5df-0a63-48f8-83fb-26210c0cc2fe-tmp-dir\") pod \"gw-section-name-router-with-gat-f1d92d0f-kserve-7d7fd586fbvqjx6\" (UID: \"d19db5df-0a63-48f8-83fb-26210c0cc2fe\") " pod="kserve-ci-e2e-test/gw-section-name-router-with-gat-f1d92d0f-kserve-7d7fd586fbvqjx6" Jul 10 18:53:15.799636 ip-10-0-143-17 kubenswrapper[2585]: I0710 18:53:15.799603 2585 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/d19db5df-0a63-48f8-83fb-26210c0cc2fe-model-cache\") pod \"gw-section-name-router-with-gat-f1d92d0f-kserve-7d7fd586fbvqjx6\" (UID: \"d19db5df-0a63-48f8-83fb-26210c0cc2fe\") " pod="kserve-ci-e2e-test/gw-section-name-router-with-gat-f1d92d0f-kserve-7d7fd586fbvqjx6" Jul 10 18:53:15.799769 ip-10-0-143-17 kubenswrapper[2585]: I0710 18:53:15.799650 2585 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/d19db5df-0a63-48f8-83fb-26210c0cc2fe-tls-certs\") pod \"gw-section-name-router-with-gat-f1d92d0f-kserve-7d7fd586fbvqjx6\" (UID: \"d19db5df-0a63-48f8-83fb-26210c0cc2fe\") " pod="kserve-ci-e2e-test/gw-section-name-router-with-gat-f1d92d0f-kserve-7d7fd586fbvqjx6" Jul 10 18:53:15.799769 ip-10-0-143-17 kubenswrapper[2585]: I0710 18:53:15.799703 2585 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/d19db5df-0a63-48f8-83fb-26210c0cc2fe-dshm\") pod \"gw-section-name-router-with-gat-f1d92d0f-kserve-7d7fd586fbvqjx6\" (UID: \"d19db5df-0a63-48f8-83fb-26210c0cc2fe\") " pod="kserve-ci-e2e-test/gw-section-name-router-with-gat-f1d92d0f-kserve-7d7fd586fbvqjx6" Jul 10 18:53:15.799769 ip-10-0-143-17 kubenswrapper[2585]: I0710 18:53:15.799730 2585 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-f4jzk\" (UniqueName: \"kubernetes.io/projected/d19db5df-0a63-48f8-83fb-26210c0cc2fe-kube-api-access-f4jzk\") pod \"gw-section-name-router-with-gat-f1d92d0f-kserve-7d7fd586fbvqjx6\" (UID: \"d19db5df-0a63-48f8-83fb-26210c0cc2fe\") " pod="kserve-ci-e2e-test/gw-section-name-router-with-gat-f1d92d0f-kserve-7d7fd586fbvqjx6" Jul 10 18:53:15.799769 ip-10-0-143-17 kubenswrapper[2585]: I0710 18:53:15.799758 2585 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/d19db5df-0a63-48f8-83fb-26210c0cc2fe-home\") pod \"gw-section-name-router-with-gat-f1d92d0f-kserve-7d7fd586fbvqjx6\" (UID: \"d19db5df-0a63-48f8-83fb-26210c0cc2fe\") " pod="kserve-ci-e2e-test/gw-section-name-router-with-gat-f1d92d0f-kserve-7d7fd586fbvqjx6" Jul 10 18:53:15.799969 ip-10-0-143-17 kubenswrapper[2585]: I0710 18:53:15.799786 2585 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kserve-provision-location\" (UniqueName: \"kubernetes.io/empty-dir/d19db5df-0a63-48f8-83fb-26210c0cc2fe-kserve-provision-location\") pod \"gw-section-name-router-with-gat-f1d92d0f-kserve-7d7fd586fbvqjx6\" (UID: \"d19db5df-0a63-48f8-83fb-26210c0cc2fe\") " pod="kserve-ci-e2e-test/gw-section-name-router-with-gat-f1d92d0f-kserve-7d7fd586fbvqjx6" Jul 10 18:53:15.799969 ip-10-0-143-17 kubenswrapper[2585]: I0710 18:53:15.799866 2585 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/d19db5df-0a63-48f8-83fb-26210c0cc2fe-tmp-dir\") pod \"gw-section-name-router-with-gat-f1d92d0f-kserve-7d7fd586fbvqjx6\" (UID: \"d19db5df-0a63-48f8-83fb-26210c0cc2fe\") " pod="kserve-ci-e2e-test/gw-section-name-router-with-gat-f1d92d0f-kserve-7d7fd586fbvqjx6" Jul 10 18:53:15.800085 ip-10-0-143-17 kubenswrapper[2585]: I0710 18:53:15.800062 2585 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/d19db5df-0a63-48f8-83fb-26210c0cc2fe-model-cache\") pod \"gw-section-name-router-with-gat-f1d92d0f-kserve-7d7fd586fbvqjx6\" (UID: \"d19db5df-0a63-48f8-83fb-26210c0cc2fe\") " pod="kserve-ci-e2e-test/gw-section-name-router-with-gat-f1d92d0f-kserve-7d7fd586fbvqjx6" Jul 10 18:53:15.800187 ip-10-0-143-17 kubenswrapper[2585]: I0710 18:53:15.800167 2585 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/d19db5df-0a63-48f8-83fb-26210c0cc2fe-home\") pod \"gw-section-name-router-with-gat-f1d92d0f-kserve-7d7fd586fbvqjx6\" (UID: \"d19db5df-0a63-48f8-83fb-26210c0cc2fe\") " pod="kserve-ci-e2e-test/gw-section-name-router-with-gat-f1d92d0f-kserve-7d7fd586fbvqjx6" Jul 10 18:53:15.800359 ip-10-0-143-17 kubenswrapper[2585]: I0710 18:53:15.800340 2585 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kserve-provision-location\" (UniqueName: \"kubernetes.io/empty-dir/d19db5df-0a63-48f8-83fb-26210c0cc2fe-kserve-provision-location\") pod \"gw-section-name-router-with-gat-f1d92d0f-kserve-7d7fd586fbvqjx6\" (UID: \"d19db5df-0a63-48f8-83fb-26210c0cc2fe\") " pod="kserve-ci-e2e-test/gw-section-name-router-with-gat-f1d92d0f-kserve-7d7fd586fbvqjx6" Jul 10 18:53:15.800434 ip-10-0-143-17 kubenswrapper[2585]: I0710 18:53:15.800371 2585 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/d19db5df-0a63-48f8-83fb-26210c0cc2fe-tmp-dir\") pod \"gw-section-name-router-with-gat-f1d92d0f-kserve-7d7fd586fbvqjx6\" (UID: \"d19db5df-0a63-48f8-83fb-26210c0cc2fe\") " pod="kserve-ci-e2e-test/gw-section-name-router-with-gat-f1d92d0f-kserve-7d7fd586fbvqjx6" Jul 10 18:53:15.802651 ip-10-0-143-17 kubenswrapper[2585]: I0710 18:53:15.802630 2585 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/d19db5df-0a63-48f8-83fb-26210c0cc2fe-dshm\") pod \"gw-section-name-router-with-gat-f1d92d0f-kserve-7d7fd586fbvqjx6\" (UID: \"d19db5df-0a63-48f8-83fb-26210c0cc2fe\") " pod="kserve-ci-e2e-test/gw-section-name-router-with-gat-f1d92d0f-kserve-7d7fd586fbvqjx6" Jul 10 18:53:15.802950 ip-10-0-143-17 kubenswrapper[2585]: I0710 18:53:15.802930 2585 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/d19db5df-0a63-48f8-83fb-26210c0cc2fe-tls-certs\") pod \"gw-section-name-router-with-gat-f1d92d0f-kserve-7d7fd586fbvqjx6\" (UID: \"d19db5df-0a63-48f8-83fb-26210c0cc2fe\") " pod="kserve-ci-e2e-test/gw-section-name-router-with-gat-f1d92d0f-kserve-7d7fd586fbvqjx6" Jul 10 18:53:15.809193 ip-10-0-143-17 kubenswrapper[2585]: I0710 18:53:15.809168 2585 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-f4jzk\" (UniqueName: \"kubernetes.io/projected/d19db5df-0a63-48f8-83fb-26210c0cc2fe-kube-api-access-f4jzk\") pod \"gw-section-name-router-with-gat-f1d92d0f-kserve-7d7fd586fbvqjx6\" (UID: \"d19db5df-0a63-48f8-83fb-26210c0cc2fe\") " pod="kserve-ci-e2e-test/gw-section-name-router-with-gat-f1d92d0f-kserve-7d7fd586fbvqjx6" Jul 10 18:53:15.937490 ip-10-0-143-17 kubenswrapper[2585]: I0710 18:53:15.937420 2585 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kserve-ci-e2e-test/gw-section-name-router-with-gat-f1d92d0f-kserve-7d7fd586fbvqjx6" Jul 10 18:53:16.068239 ip-10-0-143-17 kubenswrapper[2585]: I0710 18:53:16.068213 2585 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kserve-ci-e2e-test/gw-section-name-router-with-gat-f1d92d0f-kserve-7d7fd586fbvqjx6"] Jul 10 18:53:16.069883 ip-10-0-143-17 kubenswrapper[2585]: W0710 18:53:16.069854 2585 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podd19db5df_0a63_48f8_83fb_26210c0cc2fe.slice/crio-8b7b1af6f373141887719576e97564cd40467129894f4c79377cf8d2c9173b3d WatchSource:0}: Error finding container 8b7b1af6f373141887719576e97564cd40467129894f4c79377cf8d2c9173b3d: Status 404 returned error can't find the container with id 8b7b1af6f373141887719576e97564cd40467129894f4c79377cf8d2c9173b3d Jul 10 18:53:16.970259 ip-10-0-143-17 kubenswrapper[2585]: I0710 18:53:16.970219 2585 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kserve-ci-e2e-test/gw-section-name-router-with-gat-f1d92d0f-kserve-7d7fd586fbvqjx6" event={"ID":"d19db5df-0a63-48f8-83fb-26210c0cc2fe","Type":"ContainerStarted","Data":"9666498802cb3d3b865719a3909c920c11c2a3a91856aa55e65f166cbd131930"} Jul 10 18:53:16.970651 ip-10-0-143-17 kubenswrapper[2585]: I0710 18:53:16.970281 2585 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kserve-ci-e2e-test/gw-section-name-router-with-gat-f1d92d0f-kserve-7d7fd586fbvqjx6" event={"ID":"d19db5df-0a63-48f8-83fb-26210c0cc2fe","Type":"ContainerStarted","Data":"8b7b1af6f373141887719576e97564cd40467129894f4c79377cf8d2c9173b3d"} Jul 10 18:53:19.225261 ip-10-0-143-17 kubenswrapper[2585]: I0710 18:53:19.225231 2585 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["kserve-ci-e2e-test/gw-section-name-router-with-gat-f1d92d0f-kserve-7d7fd586fbvqjx6"] Jul 10 18:53:19.225731 ip-10-0-143-17 kubenswrapper[2585]: I0710 18:53:19.225441 2585 kuberuntime_container.go:873] "Killing container with a grace period" pod="kserve-ci-e2e-test/gw-section-name-router-with-gat-f1d92d0f-kserve-7d7fd586fbvqjx6" podUID="d19db5df-0a63-48f8-83fb-26210c0cc2fe" containerName="storage-initializer" containerID="cri-o://9666498802cb3d3b865719a3909c920c11c2a3a91856aa55e65f166cbd131930" gracePeriod=30 Jul 10 18:53:20.682939 ip-10-0-143-17 kubenswrapper[2585]: I0710 18:53:20.682919 2585 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="kserve-ci-e2e-test/gw-section-name-router-with-gat-f1d92d0f-kserve-7d7fd586fbvqjx6" Jul 10 18:53:20.746447 ip-10-0-143-17 kubenswrapper[2585]: I0710 18:53:20.746423 2585 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/d19db5df-0a63-48f8-83fb-26210c0cc2fe-tls-certs\") pod \"d19db5df-0a63-48f8-83fb-26210c0cc2fe\" (UID: \"d19db5df-0a63-48f8-83fb-26210c0cc2fe\") " Jul 10 18:53:20.746567 ip-10-0-143-17 kubenswrapper[2585]: I0710 18:53:20.746457 2585 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/d19db5df-0a63-48f8-83fb-26210c0cc2fe-home\") pod \"d19db5df-0a63-48f8-83fb-26210c0cc2fe\" (UID: \"d19db5df-0a63-48f8-83fb-26210c0cc2fe\") " Jul 10 18:53:20.746567 ip-10-0-143-17 kubenswrapper[2585]: I0710 18:53:20.746477 2585 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kserve-provision-location\" (UniqueName: \"kubernetes.io/empty-dir/d19db5df-0a63-48f8-83fb-26210c0cc2fe-kserve-provision-location\") pod \"d19db5df-0a63-48f8-83fb-26210c0cc2fe\" (UID: \"d19db5df-0a63-48f8-83fb-26210c0cc2fe\") " Jul 10 18:53:20.746567 ip-10-0-143-17 kubenswrapper[2585]: I0710 18:53:20.746518 2585 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/d19db5df-0a63-48f8-83fb-26210c0cc2fe-model-cache\") pod \"d19db5df-0a63-48f8-83fb-26210c0cc2fe\" (UID: \"d19db5df-0a63-48f8-83fb-26210c0cc2fe\") " Jul 10 18:53:20.746708 ip-10-0-143-17 kubenswrapper[2585]: I0710 18:53:20.746566 2585 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/d19db5df-0a63-48f8-83fb-26210c0cc2fe-tmp-dir\") pod \"d19db5df-0a63-48f8-83fb-26210c0cc2fe\" (UID: \"d19db5df-0a63-48f8-83fb-26210c0cc2fe\") " Jul 10 18:53:20.746708 ip-10-0-143-17 kubenswrapper[2585]: I0710 18:53:20.746603 2585 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-f4jzk\" (UniqueName: \"kubernetes.io/projected/d19db5df-0a63-48f8-83fb-26210c0cc2fe-kube-api-access-f4jzk\") pod \"d19db5df-0a63-48f8-83fb-26210c0cc2fe\" (UID: \"d19db5df-0a63-48f8-83fb-26210c0cc2fe\") " Jul 10 18:53:20.746708 ip-10-0-143-17 kubenswrapper[2585]: I0710 18:53:20.746655 2585 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/d19db5df-0a63-48f8-83fb-26210c0cc2fe-dshm\") pod \"d19db5df-0a63-48f8-83fb-26210c0cc2fe\" (UID: \"d19db5df-0a63-48f8-83fb-26210c0cc2fe\") " Jul 10 18:53:20.746860 ip-10-0-143-17 kubenswrapper[2585]: I0710 18:53:20.746761 2585 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/d19db5df-0a63-48f8-83fb-26210c0cc2fe-home" (OuterVolumeSpecName: "home") pod "d19db5df-0a63-48f8-83fb-26210c0cc2fe" (UID: "d19db5df-0a63-48f8-83fb-26210c0cc2fe"). InnerVolumeSpecName "home". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Jul 10 18:53:20.746860 ip-10-0-143-17 kubenswrapper[2585]: I0710 18:53:20.746782 2585 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/d19db5df-0a63-48f8-83fb-26210c0cc2fe-model-cache" (OuterVolumeSpecName: "model-cache") pod "d19db5df-0a63-48f8-83fb-26210c0cc2fe" (UID: "d19db5df-0a63-48f8-83fb-26210c0cc2fe"). InnerVolumeSpecName "model-cache". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Jul 10 18:53:20.746961 ip-10-0-143-17 kubenswrapper[2585]: I0710 18:53:20.746869 2585 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/d19db5df-0a63-48f8-83fb-26210c0cc2fe-tmp-dir" (OuterVolumeSpecName: "tmp-dir") pod "d19db5df-0a63-48f8-83fb-26210c0cc2fe" (UID: "d19db5df-0a63-48f8-83fb-26210c0cc2fe"). InnerVolumeSpecName "tmp-dir". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Jul 10 18:53:20.747028 ip-10-0-143-17 kubenswrapper[2585]: I0710 18:53:20.747005 2585 reconciler_common.go:299] "Volume detached for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/d19db5df-0a63-48f8-83fb-26210c0cc2fe-home\") on node \"ip-10-0-143-17.ec2.internal\" DevicePath \"\"" Jul 10 18:53:20.747093 ip-10-0-143-17 kubenswrapper[2585]: I0710 18:53:20.747035 2585 reconciler_common.go:299] "Volume detached for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/d19db5df-0a63-48f8-83fb-26210c0cc2fe-model-cache\") on node \"ip-10-0-143-17.ec2.internal\" DevicePath \"\"" Jul 10 18:53:20.747093 ip-10-0-143-17 kubenswrapper[2585]: I0710 18:53:20.747051 2585 reconciler_common.go:299] "Volume detached for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/d19db5df-0a63-48f8-83fb-26210c0cc2fe-tmp-dir\") on node \"ip-10-0-143-17.ec2.internal\" DevicePath \"\"" Jul 10 18:53:20.748533 ip-10-0-143-17 kubenswrapper[2585]: I0710 18:53:20.748506 2585 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/d19db5df-0a63-48f8-83fb-26210c0cc2fe-tls-certs" (OuterVolumeSpecName: "tls-certs") pod "d19db5df-0a63-48f8-83fb-26210c0cc2fe" (UID: "d19db5df-0a63-48f8-83fb-26210c0cc2fe"). InnerVolumeSpecName "tls-certs". PluginName "kubernetes.io/secret", VolumeGIDValue "" Jul 10 18:53:20.748671 ip-10-0-143-17 kubenswrapper[2585]: I0710 18:53:20.748609 2585 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/d19db5df-0a63-48f8-83fb-26210c0cc2fe-kube-api-access-f4jzk" (OuterVolumeSpecName: "kube-api-access-f4jzk") pod "d19db5df-0a63-48f8-83fb-26210c0cc2fe" (UID: "d19db5df-0a63-48f8-83fb-26210c0cc2fe"). InnerVolumeSpecName "kube-api-access-f4jzk". PluginName "kubernetes.io/projected", VolumeGIDValue "" Jul 10 18:53:20.748792 ip-10-0-143-17 kubenswrapper[2585]: I0710 18:53:20.748778 2585 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/d19db5df-0a63-48f8-83fb-26210c0cc2fe-dshm" (OuterVolumeSpecName: "dshm") pod "d19db5df-0a63-48f8-83fb-26210c0cc2fe" (UID: "d19db5df-0a63-48f8-83fb-26210c0cc2fe"). InnerVolumeSpecName "dshm". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Jul 10 18:53:20.812132 ip-10-0-143-17 kubenswrapper[2585]: I0710 18:53:20.812075 2585 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/d19db5df-0a63-48f8-83fb-26210c0cc2fe-kserve-provision-location" (OuterVolumeSpecName: "kserve-provision-location") pod "d19db5df-0a63-48f8-83fb-26210c0cc2fe" (UID: "d19db5df-0a63-48f8-83fb-26210c0cc2fe"). InnerVolumeSpecName "kserve-provision-location". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Jul 10 18:53:20.848057 ip-10-0-143-17 kubenswrapper[2585]: I0710 18:53:20.848037 2585 reconciler_common.go:299] "Volume detached for volume \"kserve-provision-location\" (UniqueName: \"kubernetes.io/empty-dir/d19db5df-0a63-48f8-83fb-26210c0cc2fe-kserve-provision-location\") on node \"ip-10-0-143-17.ec2.internal\" DevicePath \"\"" Jul 10 18:53:20.848057 ip-10-0-143-17 kubenswrapper[2585]: I0710 18:53:20.848055 2585 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-f4jzk\" (UniqueName: \"kubernetes.io/projected/d19db5df-0a63-48f8-83fb-26210c0cc2fe-kube-api-access-f4jzk\") on node \"ip-10-0-143-17.ec2.internal\" DevicePath \"\"" Jul 10 18:53:20.848168 ip-10-0-143-17 kubenswrapper[2585]: I0710 18:53:20.848064 2585 reconciler_common.go:299] "Volume detached for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/d19db5df-0a63-48f8-83fb-26210c0cc2fe-dshm\") on node \"ip-10-0-143-17.ec2.internal\" DevicePath \"\"" Jul 10 18:53:20.848168 ip-10-0-143-17 kubenswrapper[2585]: I0710 18:53:20.848072 2585 reconciler_common.go:299] "Volume detached for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/d19db5df-0a63-48f8-83fb-26210c0cc2fe-tls-certs\") on node \"ip-10-0-143-17.ec2.internal\" DevicePath \"\"" Jul 10 18:53:20.995993 ip-10-0-143-17 kubenswrapper[2585]: I0710 18:53:20.995964 2585 generic.go:356] "Generic (PLEG): container finished" podID="d19db5df-0a63-48f8-83fb-26210c0cc2fe" containerID="9666498802cb3d3b865719a3909c920c11c2a3a91856aa55e65f166cbd131930" exitCode=0 Jul 10 18:53:20.996096 ip-10-0-143-17 kubenswrapper[2585]: I0710 18:53:20.996013 2585 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="kserve-ci-e2e-test/gw-section-name-router-with-gat-f1d92d0f-kserve-7d7fd586fbvqjx6" Jul 10 18:53:20.996096 ip-10-0-143-17 kubenswrapper[2585]: I0710 18:53:20.996046 2585 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kserve-ci-e2e-test/gw-section-name-router-with-gat-f1d92d0f-kserve-7d7fd586fbvqjx6" event={"ID":"d19db5df-0a63-48f8-83fb-26210c0cc2fe","Type":"ContainerDied","Data":"9666498802cb3d3b865719a3909c920c11c2a3a91856aa55e65f166cbd131930"} Jul 10 18:53:20.996162 ip-10-0-143-17 kubenswrapper[2585]: I0710 18:53:20.996123 2585 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kserve-ci-e2e-test/gw-section-name-router-with-gat-f1d92d0f-kserve-7d7fd586fbvqjx6" event={"ID":"d19db5df-0a63-48f8-83fb-26210c0cc2fe","Type":"ContainerDied","Data":"8b7b1af6f373141887719576e97564cd40467129894f4c79377cf8d2c9173b3d"} Jul 10 18:53:20.996162 ip-10-0-143-17 kubenswrapper[2585]: I0710 18:53:20.996148 2585 scope.go:117] "RemoveContainer" containerID="9666498802cb3d3b865719a3909c920c11c2a3a91856aa55e65f166cbd131930" Jul 10 18:53:21.034265 ip-10-0-143-17 kubenswrapper[2585]: I0710 18:53:21.034242 2585 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["kserve-ci-e2e-test/gw-section-name-router-with-gat-f1d92d0f-kserve-7d7fd586fbvqjx6"] Jul 10 18:53:21.039229 ip-10-0-143-17 kubenswrapper[2585]: I0710 18:53:21.039210 2585 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["kserve-ci-e2e-test/gw-section-name-router-with-gat-f1d92d0f-kserve-7d7fd586fbvqjx6"] Jul 10 18:53:21.081613 ip-10-0-143-17 kubenswrapper[2585]: I0710 18:53:21.081596 2585 scope.go:117] "RemoveContainer" containerID="9666498802cb3d3b865719a3909c920c11c2a3a91856aa55e65f166cbd131930" Jul 10 18:53:21.081910 ip-10-0-143-17 kubenswrapper[2585]: E0710 18:53:21.081891 2585 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"9666498802cb3d3b865719a3909c920c11c2a3a91856aa55e65f166cbd131930\": container with ID starting with 9666498802cb3d3b865719a3909c920c11c2a3a91856aa55e65f166cbd131930 not found: ID does not exist" containerID="9666498802cb3d3b865719a3909c920c11c2a3a91856aa55e65f166cbd131930" Jul 10 18:53:21.081961 ip-10-0-143-17 kubenswrapper[2585]: I0710 18:53:21.081921 2585 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"9666498802cb3d3b865719a3909c920c11c2a3a91856aa55e65f166cbd131930"} err="failed to get container status \"9666498802cb3d3b865719a3909c920c11c2a3a91856aa55e65f166cbd131930\": rpc error: code = NotFound desc = could not find container \"9666498802cb3d3b865719a3909c920c11c2a3a91856aa55e65f166cbd131930\": container with ID starting with 9666498802cb3d3b865719a3909c920c11c2a3a91856aa55e65f166cbd131930 not found: ID does not exist" Jul 10 18:53:21.216247 ip-10-0-143-17 kubenswrapper[2585]: I0710 18:53:21.216219 2585 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="d19db5df-0a63-48f8-83fb-26210c0cc2fe" path="/var/lib/kubelet/pods/d19db5df-0a63-48f8-83fb-26210c0cc2fe/volumes" Jul 10 18:53:28.812323 ip-10-0-143-17 kubenswrapper[2585]: I0710 18:53:28.812293 2585 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["kserve-ci-e2e-test/llmisvc-model-fb-opt-125m-route-6fa1027a-kserve-66b8d9fc44xrxgt"] Jul 10 18:53:28.812743 ip-10-0-143-17 kubenswrapper[2585]: I0710 18:53:28.812673 2585 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="d19db5df-0a63-48f8-83fb-26210c0cc2fe" containerName="storage-initializer" Jul 10 18:53:28.812743 ip-10-0-143-17 kubenswrapper[2585]: I0710 18:53:28.812685 2585 state_mem.go:107] "Deleted CPUSet assignment" podUID="d19db5df-0a63-48f8-83fb-26210c0cc2fe" containerName="storage-initializer" Jul 10 18:53:28.812816 ip-10-0-143-17 kubenswrapper[2585]: I0710 18:53:28.812767 2585 memory_manager.go:368] "RemoveStaleState removing state" podUID="d19db5df-0a63-48f8-83fb-26210c0cc2fe" containerName="storage-initializer" Jul 10 18:53:28.816437 ip-10-0-143-17 kubenswrapper[2585]: I0710 18:53:28.816415 2585 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kserve-ci-e2e-test/llmisvc-model-fb-opt-125m-route-6fa1027a-kserve-66b8d9fc44xrxgt" Jul 10 18:53:28.820674 ip-10-0-143-17 kubenswrapper[2585]: I0710 18:53:28.820650 2585 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"kserve-ci-e2e-test\"/\"default-dockercfg-jjdpb\"" Jul 10 18:53:28.820674 ip-10-0-143-17 kubenswrapper[2585]: I0710 18:53:28.820663 2585 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"kserve-ci-e2e-test\"/\"llmisvad71fa5348b85aebd404221bba611457-kserve-self-signed-certs\"" Jul 10 18:53:28.829528 ip-10-0-143-17 kubenswrapper[2585]: I0710 18:53:28.829505 2585 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kserve-ci-e2e-test/llmisvc-model-fb-opt-125m-route-6fa1027a-kserve-66b8d9fc44xrxgt"] Jul 10 18:53:28.910447 ip-10-0-143-17 kubenswrapper[2585]: I0710 18:53:28.910423 2585 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/7fc19211-7c48-4517-8e04-70c500d5f701-dshm\") pod \"llmisvc-model-fb-opt-125m-route-6fa1027a-kserve-66b8d9fc44xrxgt\" (UID: \"7fc19211-7c48-4517-8e04-70c500d5f701\") " pod="kserve-ci-e2e-test/llmisvc-model-fb-opt-125m-route-6fa1027a-kserve-66b8d9fc44xrxgt" Jul 10 18:53:28.910572 ip-10-0-143-17 kubenswrapper[2585]: I0710 18:53:28.910451 2585 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/7fc19211-7c48-4517-8e04-70c500d5f701-model-cache\") pod \"llmisvc-model-fb-opt-125m-route-6fa1027a-kserve-66b8d9fc44xrxgt\" (UID: \"7fc19211-7c48-4517-8e04-70c500d5f701\") " pod="kserve-ci-e2e-test/llmisvc-model-fb-opt-125m-route-6fa1027a-kserve-66b8d9fc44xrxgt" Jul 10 18:53:28.910572 ip-10-0-143-17 kubenswrapper[2585]: I0710 18:53:28.910495 2585 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kserve-provision-location\" (UniqueName: \"kubernetes.io/empty-dir/7fc19211-7c48-4517-8e04-70c500d5f701-kserve-provision-location\") pod \"llmisvc-model-fb-opt-125m-route-6fa1027a-kserve-66b8d9fc44xrxgt\" (UID: \"7fc19211-7c48-4517-8e04-70c500d5f701\") " pod="kserve-ci-e2e-test/llmisvc-model-fb-opt-125m-route-6fa1027a-kserve-66b8d9fc44xrxgt" Jul 10 18:53:28.910688 ip-10-0-143-17 kubenswrapper[2585]: I0710 18:53:28.910611 2585 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/7fc19211-7c48-4517-8e04-70c500d5f701-home\") pod \"llmisvc-model-fb-opt-125m-route-6fa1027a-kserve-66b8d9fc44xrxgt\" (UID: \"7fc19211-7c48-4517-8e04-70c500d5f701\") " pod="kserve-ci-e2e-test/llmisvc-model-fb-opt-125m-route-6fa1027a-kserve-66b8d9fc44xrxgt" Jul 10 18:53:28.910688 ip-10-0-143-17 kubenswrapper[2585]: I0710 18:53:28.910663 2585 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/7fc19211-7c48-4517-8e04-70c500d5f701-tls-certs\") pod \"llmisvc-model-fb-opt-125m-route-6fa1027a-kserve-66b8d9fc44xrxgt\" (UID: \"7fc19211-7c48-4517-8e04-70c500d5f701\") " pod="kserve-ci-e2e-test/llmisvc-model-fb-opt-125m-route-6fa1027a-kserve-66b8d9fc44xrxgt" Jul 10 18:53:28.910777 ip-10-0-143-17 kubenswrapper[2585]: I0710 18:53:28.910695 2585 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/7fc19211-7c48-4517-8e04-70c500d5f701-tmp-dir\") pod \"llmisvc-model-fb-opt-125m-route-6fa1027a-kserve-66b8d9fc44xrxgt\" (UID: \"7fc19211-7c48-4517-8e04-70c500d5f701\") " pod="kserve-ci-e2e-test/llmisvc-model-fb-opt-125m-route-6fa1027a-kserve-66b8d9fc44xrxgt" Jul 10 18:53:28.910777 ip-10-0-143-17 kubenswrapper[2585]: I0710 18:53:28.910713 2585 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-qxlng\" (UniqueName: \"kubernetes.io/projected/7fc19211-7c48-4517-8e04-70c500d5f701-kube-api-access-qxlng\") pod \"llmisvc-model-fb-opt-125m-route-6fa1027a-kserve-66b8d9fc44xrxgt\" (UID: \"7fc19211-7c48-4517-8e04-70c500d5f701\") " pod="kserve-ci-e2e-test/llmisvc-model-fb-opt-125m-route-6fa1027a-kserve-66b8d9fc44xrxgt" Jul 10 18:53:29.011413 ip-10-0-143-17 kubenswrapper[2585]: I0710 18:53:29.011382 2585 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/7fc19211-7c48-4517-8e04-70c500d5f701-home\") pod \"llmisvc-model-fb-opt-125m-route-6fa1027a-kserve-66b8d9fc44xrxgt\" (UID: \"7fc19211-7c48-4517-8e04-70c500d5f701\") " pod="kserve-ci-e2e-test/llmisvc-model-fb-opt-125m-route-6fa1027a-kserve-66b8d9fc44xrxgt" Jul 10 18:53:29.011571 ip-10-0-143-17 kubenswrapper[2585]: I0710 18:53:29.011419 2585 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/7fc19211-7c48-4517-8e04-70c500d5f701-tls-certs\") pod \"llmisvc-model-fb-opt-125m-route-6fa1027a-kserve-66b8d9fc44xrxgt\" (UID: \"7fc19211-7c48-4517-8e04-70c500d5f701\") " pod="kserve-ci-e2e-test/llmisvc-model-fb-opt-125m-route-6fa1027a-kserve-66b8d9fc44xrxgt" Jul 10 18:53:29.011571 ip-10-0-143-17 kubenswrapper[2585]: I0710 18:53:29.011461 2585 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/7fc19211-7c48-4517-8e04-70c500d5f701-tmp-dir\") pod \"llmisvc-model-fb-opt-125m-route-6fa1027a-kserve-66b8d9fc44xrxgt\" (UID: \"7fc19211-7c48-4517-8e04-70c500d5f701\") " pod="kserve-ci-e2e-test/llmisvc-model-fb-opt-125m-route-6fa1027a-kserve-66b8d9fc44xrxgt" Jul 10 18:53:29.011571 ip-10-0-143-17 kubenswrapper[2585]: I0710 18:53:29.011487 2585 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-qxlng\" (UniqueName: \"kubernetes.io/projected/7fc19211-7c48-4517-8e04-70c500d5f701-kube-api-access-qxlng\") pod \"llmisvc-model-fb-opt-125m-route-6fa1027a-kserve-66b8d9fc44xrxgt\" (UID: \"7fc19211-7c48-4517-8e04-70c500d5f701\") " pod="kserve-ci-e2e-test/llmisvc-model-fb-opt-125m-route-6fa1027a-kserve-66b8d9fc44xrxgt" Jul 10 18:53:29.011571 ip-10-0-143-17 kubenswrapper[2585]: I0710 18:53:29.011568 2585 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/7fc19211-7c48-4517-8e04-70c500d5f701-dshm\") pod \"llmisvc-model-fb-opt-125m-route-6fa1027a-kserve-66b8d9fc44xrxgt\" (UID: \"7fc19211-7c48-4517-8e04-70c500d5f701\") " pod="kserve-ci-e2e-test/llmisvc-model-fb-opt-125m-route-6fa1027a-kserve-66b8d9fc44xrxgt" Jul 10 18:53:29.011789 ip-10-0-143-17 kubenswrapper[2585]: I0710 18:53:29.011596 2585 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/7fc19211-7c48-4517-8e04-70c500d5f701-model-cache\") pod \"llmisvc-model-fb-opt-125m-route-6fa1027a-kserve-66b8d9fc44xrxgt\" (UID: \"7fc19211-7c48-4517-8e04-70c500d5f701\") " pod="kserve-ci-e2e-test/llmisvc-model-fb-opt-125m-route-6fa1027a-kserve-66b8d9fc44xrxgt" Jul 10 18:53:29.011789 ip-10-0-143-17 kubenswrapper[2585]: I0710 18:53:29.011694 2585 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kserve-provision-location\" (UniqueName: \"kubernetes.io/empty-dir/7fc19211-7c48-4517-8e04-70c500d5f701-kserve-provision-location\") pod \"llmisvc-model-fb-opt-125m-route-6fa1027a-kserve-66b8d9fc44xrxgt\" (UID: \"7fc19211-7c48-4517-8e04-70c500d5f701\") " pod="kserve-ci-e2e-test/llmisvc-model-fb-opt-125m-route-6fa1027a-kserve-66b8d9fc44xrxgt" Jul 10 18:53:29.011884 ip-10-0-143-17 kubenswrapper[2585]: I0710 18:53:29.011836 2585 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/7fc19211-7c48-4517-8e04-70c500d5f701-home\") pod \"llmisvc-model-fb-opt-125m-route-6fa1027a-kserve-66b8d9fc44xrxgt\" (UID: \"7fc19211-7c48-4517-8e04-70c500d5f701\") " pod="kserve-ci-e2e-test/llmisvc-model-fb-opt-125m-route-6fa1027a-kserve-66b8d9fc44xrxgt" Jul 10 18:53:29.011884 ip-10-0-143-17 kubenswrapper[2585]: I0710 18:53:29.011863 2585 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/7fc19211-7c48-4517-8e04-70c500d5f701-tmp-dir\") pod \"llmisvc-model-fb-opt-125m-route-6fa1027a-kserve-66b8d9fc44xrxgt\" (UID: \"7fc19211-7c48-4517-8e04-70c500d5f701\") " pod="kserve-ci-e2e-test/llmisvc-model-fb-opt-125m-route-6fa1027a-kserve-66b8d9fc44xrxgt" Jul 10 18:53:29.011997 ip-10-0-143-17 kubenswrapper[2585]: I0710 18:53:29.011964 2585 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kserve-provision-location\" (UniqueName: \"kubernetes.io/empty-dir/7fc19211-7c48-4517-8e04-70c500d5f701-kserve-provision-location\") pod \"llmisvc-model-fb-opt-125m-route-6fa1027a-kserve-66b8d9fc44xrxgt\" (UID: \"7fc19211-7c48-4517-8e04-70c500d5f701\") " pod="kserve-ci-e2e-test/llmisvc-model-fb-opt-125m-route-6fa1027a-kserve-66b8d9fc44xrxgt" Jul 10 18:53:29.012044 ip-10-0-143-17 kubenswrapper[2585]: I0710 18:53:29.012001 2585 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/7fc19211-7c48-4517-8e04-70c500d5f701-model-cache\") pod \"llmisvc-model-fb-opt-125m-route-6fa1027a-kserve-66b8d9fc44xrxgt\" (UID: \"7fc19211-7c48-4517-8e04-70c500d5f701\") " pod="kserve-ci-e2e-test/llmisvc-model-fb-opt-125m-route-6fa1027a-kserve-66b8d9fc44xrxgt" Jul 10 18:53:29.013688 ip-10-0-143-17 kubenswrapper[2585]: I0710 18:53:29.013672 2585 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/7fc19211-7c48-4517-8e04-70c500d5f701-dshm\") pod \"llmisvc-model-fb-opt-125m-route-6fa1027a-kserve-66b8d9fc44xrxgt\" (UID: \"7fc19211-7c48-4517-8e04-70c500d5f701\") " pod="kserve-ci-e2e-test/llmisvc-model-fb-opt-125m-route-6fa1027a-kserve-66b8d9fc44xrxgt" Jul 10 18:53:29.014007 ip-10-0-143-17 kubenswrapper[2585]: I0710 18:53:29.013990 2585 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/7fc19211-7c48-4517-8e04-70c500d5f701-tls-certs\") pod \"llmisvc-model-fb-opt-125m-route-6fa1027a-kserve-66b8d9fc44xrxgt\" (UID: \"7fc19211-7c48-4517-8e04-70c500d5f701\") " pod="kserve-ci-e2e-test/llmisvc-model-fb-opt-125m-route-6fa1027a-kserve-66b8d9fc44xrxgt" Jul 10 18:53:29.019425 ip-10-0-143-17 kubenswrapper[2585]: I0710 18:53:29.019403 2585 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-qxlng\" (UniqueName: \"kubernetes.io/projected/7fc19211-7c48-4517-8e04-70c500d5f701-kube-api-access-qxlng\") pod \"llmisvc-model-fb-opt-125m-route-6fa1027a-kserve-66b8d9fc44xrxgt\" (UID: \"7fc19211-7c48-4517-8e04-70c500d5f701\") " pod="kserve-ci-e2e-test/llmisvc-model-fb-opt-125m-route-6fa1027a-kserve-66b8d9fc44xrxgt" Jul 10 18:53:29.129217 ip-10-0-143-17 kubenswrapper[2585]: I0710 18:53:29.129147 2585 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kserve-ci-e2e-test/llmisvc-model-fb-opt-125m-route-6fa1027a-kserve-66b8d9fc44xrxgt" Jul 10 18:53:29.263445 ip-10-0-143-17 kubenswrapper[2585]: I0710 18:53:29.263419 2585 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kserve-ci-e2e-test/llmisvc-model-fb-opt-125m-route-6fa1027a-kserve-66b8d9fc44xrxgt"] Jul 10 18:53:29.264437 ip-10-0-143-17 kubenswrapper[2585]: W0710 18:53:29.264414 2585 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod7fc19211_7c48_4517_8e04_70c500d5f701.slice/crio-1600be912924769ed71a64aa35f72884536aef72dcc8dd5b892a970479a730ad WatchSource:0}: Error finding container 1600be912924769ed71a64aa35f72884536aef72dcc8dd5b892a970479a730ad: Status 404 returned error can't find the container with id 1600be912924769ed71a64aa35f72884536aef72dcc8dd5b892a970479a730ad Jul 10 18:53:30.045322 ip-10-0-143-17 kubenswrapper[2585]: I0710 18:53:30.045287 2585 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kserve-ci-e2e-test/llmisvc-model-fb-opt-125m-route-6fa1027a-kserve-66b8d9fc44xrxgt" event={"ID":"7fc19211-7c48-4517-8e04-70c500d5f701","Type":"ContainerStarted","Data":"1c2a519b3f48199a81cfc178c048c7c918ff4f6f8633da91e3852fccb87702d0"} Jul 10 18:53:30.045704 ip-10-0-143-17 kubenswrapper[2585]: I0710 18:53:30.045348 2585 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kserve-ci-e2e-test/llmisvc-model-fb-opt-125m-route-6fa1027a-kserve-66b8d9fc44xrxgt" event={"ID":"7fc19211-7c48-4517-8e04-70c500d5f701","Type":"ContainerStarted","Data":"1600be912924769ed71a64aa35f72884536aef72dcc8dd5b892a970479a730ad"} Jul 10 18:53:34.071273 ip-10-0-143-17 kubenswrapper[2585]: I0710 18:53:34.071242 2585 generic.go:356] "Generic (PLEG): container finished" podID="7fc19211-7c48-4517-8e04-70c500d5f701" containerID="1c2a519b3f48199a81cfc178c048c7c918ff4f6f8633da91e3852fccb87702d0" exitCode=0 Jul 10 18:53:34.071635 ip-10-0-143-17 kubenswrapper[2585]: I0710 18:53:34.071306 2585 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kserve-ci-e2e-test/llmisvc-model-fb-opt-125m-route-6fa1027a-kserve-66b8d9fc44xrxgt" event={"ID":"7fc19211-7c48-4517-8e04-70c500d5f701","Type":"ContainerDied","Data":"1c2a519b3f48199a81cfc178c048c7c918ff4f6f8633da91e3852fccb87702d0"} Jul 10 18:53:35.078187 ip-10-0-143-17 kubenswrapper[2585]: I0710 18:53:35.078149 2585 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kserve-ci-e2e-test/llmisvc-model-fb-opt-125m-route-6fa1027a-kserve-66b8d9fc44xrxgt" event={"ID":"7fc19211-7c48-4517-8e04-70c500d5f701","Type":"ContainerStarted","Data":"7d0e1c43b8963c677b5541c775c014feb9627f2d998ee0d26a90fae5ac41a149"} Jul 10 18:53:35.100667 ip-10-0-143-17 kubenswrapper[2585]: I0710 18:53:35.100619 2585 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="kserve-ci-e2e-test/llmisvc-model-fb-opt-125m-route-6fa1027a-kserve-66b8d9fc44xrxgt" podStartSLOduration=2.928291602 podStartE2EDuration="7.100604643s" totalImagesPullingTime="0s" totalInitContainerRuntime="4.172313041s" isStatefulPod=false podCreationTimestamp="2026-07-10 18:53:28 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-10 18:53:35.09817109 +0000 UTC m=+828.555531046" watchObservedRunningTime="2026-07-10 18:53:35.100604643 +0000 UTC m=+828.557964577" Jul 10 18:53:39.130247 ip-10-0-143-17 kubenswrapper[2585]: I0710 18:53:39.130213 2585 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="kserve-ci-e2e-test/llmisvc-model-fb-opt-125m-route-6fa1027a-kserve-66b8d9fc44xrxgt" Jul 10 18:53:39.130773 ip-10-0-143-17 kubenswrapper[2585]: I0710 18:53:39.130280 2585 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="kserve-ci-e2e-test/llmisvc-model-fb-opt-125m-route-6fa1027a-kserve-66b8d9fc44xrxgt" Jul 10 18:53:39.142893 ip-10-0-143-17 kubenswrapper[2585]: I0710 18:53:39.142872 2585 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="kserve-ci-e2e-test/llmisvc-model-fb-opt-125m-route-6fa1027a-kserve-66b8d9fc44xrxgt" Jul 10 18:53:40.116903 ip-10-0-143-17 kubenswrapper[2585]: I0710 18:53:40.116870 2585 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="kserve-ci-e2e-test/llmisvc-model-fb-opt-125m-route-6fa1027a-kserve-66b8d9fc44xrxgt" Jul 10 18:54:47.025736 ip-10-0-143-17 kubenswrapper[2585]: I0710 18:54:47.025662 2585 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ovn-kubernetes_ovnkube-node-xg6bm_d863cfea-f2f3-4404-b62a-d41785525f3f/ovn-acl-logging/0.log" Jul 10 18:54:47.026189 ip-10-0-143-17 kubenswrapper[2585]: I0710 18:54:47.025990 2585 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ovn-kubernetes_ovnkube-node-xg6bm_d863cfea-f2f3-4404-b62a-d41785525f3f/ovn-acl-logging/0.log" Jul 10 18:57:49.183329 ip-10-0-143-17 kubenswrapper[2585]: I0710 18:57:49.183243 2585 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["kserve-ci-e2e-test/llmisvc-model-fb-opt-125m-route-6fa1027a-kserve-66b8d9fc44xrxgt"] Jul 10 18:57:49.183838 ip-10-0-143-17 kubenswrapper[2585]: I0710 18:57:49.183479 2585 kuberuntime_container.go:873] "Killing container with a grace period" pod="kserve-ci-e2e-test/llmisvc-model-fb-opt-125m-route-6fa1027a-kserve-66b8d9fc44xrxgt" podUID="7fc19211-7c48-4517-8e04-70c500d5f701" containerName="main" containerID="cri-o://7d0e1c43b8963c677b5541c775c014feb9627f2d998ee0d26a90fae5ac41a149" gracePeriod=30 Jul 10 18:57:49.373473 ip-10-0-143-17 kubenswrapper[2585]: I0710 18:57:49.373442 2585 generic.go:356] "Generic (PLEG): container finished" podID="7fc19211-7c48-4517-8e04-70c500d5f701" containerID="7d0e1c43b8963c677b5541c775c014feb9627f2d998ee0d26a90fae5ac41a149" exitCode=0 Jul 10 18:57:49.373629 ip-10-0-143-17 kubenswrapper[2585]: I0710 18:57:49.373518 2585 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kserve-ci-e2e-test/llmisvc-model-fb-opt-125m-route-6fa1027a-kserve-66b8d9fc44xrxgt" event={"ID":"7fc19211-7c48-4517-8e04-70c500d5f701","Type":"ContainerDied","Data":"7d0e1c43b8963c677b5541c775c014feb9627f2d998ee0d26a90fae5ac41a149"} Jul 10 18:57:49.425396 ip-10-0-143-17 kubenswrapper[2585]: I0710 18:57:49.425373 2585 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="kserve-ci-e2e-test/llmisvc-model-fb-opt-125m-route-6fa1027a-kserve-66b8d9fc44xrxgt" Jul 10 18:57:49.549083 ip-10-0-143-17 kubenswrapper[2585]: I0710 18:57:49.549057 2585 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kserve-provision-location\" (UniqueName: \"kubernetes.io/empty-dir/7fc19211-7c48-4517-8e04-70c500d5f701-kserve-provision-location\") pod \"7fc19211-7c48-4517-8e04-70c500d5f701\" (UID: \"7fc19211-7c48-4517-8e04-70c500d5f701\") " Jul 10 18:57:49.549083 ip-10-0-143-17 kubenswrapper[2585]: I0710 18:57:49.549087 2585 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/7fc19211-7c48-4517-8e04-70c500d5f701-home\") pod \"7fc19211-7c48-4517-8e04-70c500d5f701\" (UID: \"7fc19211-7c48-4517-8e04-70c500d5f701\") " Jul 10 18:57:49.549279 ip-10-0-143-17 kubenswrapper[2585]: I0710 18:57:49.549114 2585 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-qxlng\" (UniqueName: \"kubernetes.io/projected/7fc19211-7c48-4517-8e04-70c500d5f701-kube-api-access-qxlng\") pod \"7fc19211-7c48-4517-8e04-70c500d5f701\" (UID: \"7fc19211-7c48-4517-8e04-70c500d5f701\") " Jul 10 18:57:49.549279 ip-10-0-143-17 kubenswrapper[2585]: I0710 18:57:49.549179 2585 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/7fc19211-7c48-4517-8e04-70c500d5f701-tmp-dir\") pod \"7fc19211-7c48-4517-8e04-70c500d5f701\" (UID: \"7fc19211-7c48-4517-8e04-70c500d5f701\") " Jul 10 18:57:49.549279 ip-10-0-143-17 kubenswrapper[2585]: I0710 18:57:49.549208 2585 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/7fc19211-7c48-4517-8e04-70c500d5f701-dshm\") pod \"7fc19211-7c48-4517-8e04-70c500d5f701\" (UID: \"7fc19211-7c48-4517-8e04-70c500d5f701\") " Jul 10 18:57:49.549279 ip-10-0-143-17 kubenswrapper[2585]: I0710 18:57:49.549230 2585 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/7fc19211-7c48-4517-8e04-70c500d5f701-tls-certs\") pod \"7fc19211-7c48-4517-8e04-70c500d5f701\" (UID: \"7fc19211-7c48-4517-8e04-70c500d5f701\") " Jul 10 18:57:49.549487 ip-10-0-143-17 kubenswrapper[2585]: I0710 18:57:49.549303 2585 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/7fc19211-7c48-4517-8e04-70c500d5f701-model-cache\") pod \"7fc19211-7c48-4517-8e04-70c500d5f701\" (UID: \"7fc19211-7c48-4517-8e04-70c500d5f701\") " Jul 10 18:57:49.549487 ip-10-0-143-17 kubenswrapper[2585]: I0710 18:57:49.549349 2585 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/7fc19211-7c48-4517-8e04-70c500d5f701-home" (OuterVolumeSpecName: "home") pod "7fc19211-7c48-4517-8e04-70c500d5f701" (UID: "7fc19211-7c48-4517-8e04-70c500d5f701"). InnerVolumeSpecName "home". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Jul 10 18:57:49.549487 ip-10-0-143-17 kubenswrapper[2585]: I0710 18:57:49.549439 2585 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/7fc19211-7c48-4517-8e04-70c500d5f701-tmp-dir" (OuterVolumeSpecName: "tmp-dir") pod "7fc19211-7c48-4517-8e04-70c500d5f701" (UID: "7fc19211-7c48-4517-8e04-70c500d5f701"). InnerVolumeSpecName "tmp-dir". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Jul 10 18:57:49.549677 ip-10-0-143-17 kubenswrapper[2585]: I0710 18:57:49.549662 2585 reconciler_common.go:299] "Volume detached for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/7fc19211-7c48-4517-8e04-70c500d5f701-home\") on node \"ip-10-0-143-17.ec2.internal\" DevicePath \"\"" Jul 10 18:57:49.549677 ip-10-0-143-17 kubenswrapper[2585]: I0710 18:57:49.549665 2585 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/7fc19211-7c48-4517-8e04-70c500d5f701-model-cache" (OuterVolumeSpecName: "model-cache") pod "7fc19211-7c48-4517-8e04-70c500d5f701" (UID: "7fc19211-7c48-4517-8e04-70c500d5f701"). InnerVolumeSpecName "model-cache". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Jul 10 18:57:49.549782 ip-10-0-143-17 kubenswrapper[2585]: I0710 18:57:49.549693 2585 reconciler_common.go:299] "Volume detached for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/7fc19211-7c48-4517-8e04-70c500d5f701-tmp-dir\") on node \"ip-10-0-143-17.ec2.internal\" DevicePath \"\"" Jul 10 18:57:49.551360 ip-10-0-143-17 kubenswrapper[2585]: I0710 18:57:49.551335 2585 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/7fc19211-7c48-4517-8e04-70c500d5f701-dshm" (OuterVolumeSpecName: "dshm") pod "7fc19211-7c48-4517-8e04-70c500d5f701" (UID: "7fc19211-7c48-4517-8e04-70c500d5f701"). InnerVolumeSpecName "dshm". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Jul 10 18:57:49.551487 ip-10-0-143-17 kubenswrapper[2585]: I0710 18:57:49.551439 2585 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/7fc19211-7c48-4517-8e04-70c500d5f701-kube-api-access-qxlng" (OuterVolumeSpecName: "kube-api-access-qxlng") pod "7fc19211-7c48-4517-8e04-70c500d5f701" (UID: "7fc19211-7c48-4517-8e04-70c500d5f701"). InnerVolumeSpecName "kube-api-access-qxlng". PluginName "kubernetes.io/projected", VolumeGIDValue "" Jul 10 18:57:49.551487 ip-10-0-143-17 kubenswrapper[2585]: I0710 18:57:49.551464 2585 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/7fc19211-7c48-4517-8e04-70c500d5f701-tls-certs" (OuterVolumeSpecName: "tls-certs") pod "7fc19211-7c48-4517-8e04-70c500d5f701" (UID: "7fc19211-7c48-4517-8e04-70c500d5f701"). InnerVolumeSpecName "tls-certs". PluginName "kubernetes.io/secret", VolumeGIDValue "" Jul 10 18:57:49.604703 ip-10-0-143-17 kubenswrapper[2585]: I0710 18:57:49.604673 2585 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/7fc19211-7c48-4517-8e04-70c500d5f701-kserve-provision-location" (OuterVolumeSpecName: "kserve-provision-location") pod "7fc19211-7c48-4517-8e04-70c500d5f701" (UID: "7fc19211-7c48-4517-8e04-70c500d5f701"). InnerVolumeSpecName "kserve-provision-location". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Jul 10 18:57:49.650548 ip-10-0-143-17 kubenswrapper[2585]: I0710 18:57:49.650521 2585 reconciler_common.go:299] "Volume detached for volume \"kserve-provision-location\" (UniqueName: \"kubernetes.io/empty-dir/7fc19211-7c48-4517-8e04-70c500d5f701-kserve-provision-location\") on node \"ip-10-0-143-17.ec2.internal\" DevicePath \"\"" Jul 10 18:57:49.650675 ip-10-0-143-17 kubenswrapper[2585]: I0710 18:57:49.650562 2585 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-qxlng\" (UniqueName: \"kubernetes.io/projected/7fc19211-7c48-4517-8e04-70c500d5f701-kube-api-access-qxlng\") on node \"ip-10-0-143-17.ec2.internal\" DevicePath \"\"" Jul 10 18:57:49.650675 ip-10-0-143-17 kubenswrapper[2585]: I0710 18:57:49.650575 2585 reconciler_common.go:299] "Volume detached for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/7fc19211-7c48-4517-8e04-70c500d5f701-dshm\") on node \"ip-10-0-143-17.ec2.internal\" DevicePath \"\"" Jul 10 18:57:49.650675 ip-10-0-143-17 kubenswrapper[2585]: I0710 18:57:49.650583 2585 reconciler_common.go:299] "Volume detached for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/7fc19211-7c48-4517-8e04-70c500d5f701-tls-certs\") on node \"ip-10-0-143-17.ec2.internal\" DevicePath \"\"" Jul 10 18:57:49.650675 ip-10-0-143-17 kubenswrapper[2585]: I0710 18:57:49.650592 2585 reconciler_common.go:299] "Volume detached for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/7fc19211-7c48-4517-8e04-70c500d5f701-model-cache\") on node \"ip-10-0-143-17.ec2.internal\" DevicePath \"\"" Jul 10 18:57:50.379758 ip-10-0-143-17 kubenswrapper[2585]: I0710 18:57:50.379732 2585 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="kserve-ci-e2e-test/llmisvc-model-fb-opt-125m-route-6fa1027a-kserve-66b8d9fc44xrxgt" Jul 10 18:57:50.380216 ip-10-0-143-17 kubenswrapper[2585]: I0710 18:57:50.379734 2585 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kserve-ci-e2e-test/llmisvc-model-fb-opt-125m-route-6fa1027a-kserve-66b8d9fc44xrxgt" event={"ID":"7fc19211-7c48-4517-8e04-70c500d5f701","Type":"ContainerDied","Data":"1600be912924769ed71a64aa35f72884536aef72dcc8dd5b892a970479a730ad"} Jul 10 18:57:50.380216 ip-10-0-143-17 kubenswrapper[2585]: I0710 18:57:50.379885 2585 scope.go:117] "RemoveContainer" containerID="7d0e1c43b8963c677b5541c775c014feb9627f2d998ee0d26a90fae5ac41a149" Jul 10 18:57:50.389586 ip-10-0-143-17 kubenswrapper[2585]: I0710 18:57:50.389569 2585 scope.go:117] "RemoveContainer" containerID="1c2a519b3f48199a81cfc178c048c7c918ff4f6f8633da91e3852fccb87702d0" Jul 10 18:57:50.412236 ip-10-0-143-17 kubenswrapper[2585]: I0710 18:57:50.412215 2585 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["kserve-ci-e2e-test/llmisvc-model-fb-opt-125m-route-6fa1027a-kserve-66b8d9fc44xrxgt"] Jul 10 18:57:50.424380 ip-10-0-143-17 kubenswrapper[2585]: I0710 18:57:50.424356 2585 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["kserve-ci-e2e-test/llmisvc-model-fb-opt-125m-route-6fa1027a-kserve-66b8d9fc44xrxgt"] Jul 10 18:57:51.216240 ip-10-0-143-17 kubenswrapper[2585]: I0710 18:57:51.216203 2585 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="7fc19211-7c48-4517-8e04-70c500d5f701" path="/var/lib/kubelet/pods/7fc19211-7c48-4517-8e04-70c500d5f701/volumes" Jul 10 18:59:47.057432 ip-10-0-143-17 kubenswrapper[2585]: I0710 18:59:47.057399 2585 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ovn-kubernetes_ovnkube-node-xg6bm_d863cfea-f2f3-4404-b62a-d41785525f3f/ovn-acl-logging/0.log" Jul 10 18:59:47.060125 ip-10-0-143-17 kubenswrapper[2585]: I0710 18:59:47.060102 2585 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ovn-kubernetes_ovnkube-node-xg6bm_d863cfea-f2f3-4404-b62a-d41785525f3f/ovn-acl-logging/0.log" Jul 10 19:00:39.823116 ip-10-0-143-17 kubenswrapper[2585]: I0710 19:00:39.823041 2585 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["kserve/llmisvc-controller-manager-645ffbd459-p959l"] Jul 10 19:00:39.823506 ip-10-0-143-17 kubenswrapper[2585]: I0710 19:00:39.823376 2585 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="7fc19211-7c48-4517-8e04-70c500d5f701" containerName="storage-initializer" Jul 10 19:00:39.823506 ip-10-0-143-17 kubenswrapper[2585]: I0710 19:00:39.823387 2585 state_mem.go:107] "Deleted CPUSet assignment" podUID="7fc19211-7c48-4517-8e04-70c500d5f701" containerName="storage-initializer" Jul 10 19:00:39.823506 ip-10-0-143-17 kubenswrapper[2585]: I0710 19:00:39.823406 2585 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="7fc19211-7c48-4517-8e04-70c500d5f701" containerName="main" Jul 10 19:00:39.823506 ip-10-0-143-17 kubenswrapper[2585]: I0710 19:00:39.823412 2585 state_mem.go:107] "Deleted CPUSet assignment" podUID="7fc19211-7c48-4517-8e04-70c500d5f701" containerName="main" Jul 10 19:00:39.823506 ip-10-0-143-17 kubenswrapper[2585]: I0710 19:00:39.823466 2585 memory_manager.go:368] "RemoveStaleState removing state" podUID="7fc19211-7c48-4517-8e04-70c500d5f701" containerName="main" Jul 10 19:00:39.825578 ip-10-0-143-17 kubenswrapper[2585]: I0710 19:00:39.825561 2585 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kserve/llmisvc-controller-manager-645ffbd459-p959l" Jul 10 19:00:39.829438 ip-10-0-143-17 kubenswrapper[2585]: I0710 19:00:39.829418 2585 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"kserve\"/\"llmisvc-webhook-server-cert\"" Jul 10 19:00:39.829607 ip-10-0-143-17 kubenswrapper[2585]: I0710 19:00:39.829469 2585 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"kserve\"/\"llmisvc-controller-manager-dockercfg-xfchk\"" Jul 10 19:00:39.837124 ip-10-0-143-17 kubenswrapper[2585]: I0710 19:00:39.837103 2585 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kserve/llmisvc-controller-manager-645ffbd459-p959l"] Jul 10 19:00:39.956087 ip-10-0-143-17 kubenswrapper[2585]: I0710 19:00:39.956046 2585 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-f6dqn\" (UniqueName: \"kubernetes.io/projected/4c4024ac-8ae6-43fa-9a94-9336e896afde-kube-api-access-f6dqn\") pod \"llmisvc-controller-manager-645ffbd459-p959l\" (UID: \"4c4024ac-8ae6-43fa-9a94-9336e896afde\") " pod="kserve/llmisvc-controller-manager-645ffbd459-p959l" Jul 10 19:00:39.956259 ip-10-0-143-17 kubenswrapper[2585]: I0710 19:00:39.956133 2585 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"cert\" (UniqueName: \"kubernetes.io/secret/4c4024ac-8ae6-43fa-9a94-9336e896afde-cert\") pod \"llmisvc-controller-manager-645ffbd459-p959l\" (UID: \"4c4024ac-8ae6-43fa-9a94-9336e896afde\") " pod="kserve/llmisvc-controller-manager-645ffbd459-p959l" Jul 10 19:00:40.057419 ip-10-0-143-17 kubenswrapper[2585]: I0710 19:00:40.057391 2585 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"cert\" (UniqueName: \"kubernetes.io/secret/4c4024ac-8ae6-43fa-9a94-9336e896afde-cert\") pod \"llmisvc-controller-manager-645ffbd459-p959l\" (UID: \"4c4024ac-8ae6-43fa-9a94-9336e896afde\") " pod="kserve/llmisvc-controller-manager-645ffbd459-p959l" Jul 10 19:00:40.057583 ip-10-0-143-17 kubenswrapper[2585]: I0710 19:00:40.057452 2585 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-f6dqn\" (UniqueName: \"kubernetes.io/projected/4c4024ac-8ae6-43fa-9a94-9336e896afde-kube-api-access-f6dqn\") pod \"llmisvc-controller-manager-645ffbd459-p959l\" (UID: \"4c4024ac-8ae6-43fa-9a94-9336e896afde\") " pod="kserve/llmisvc-controller-manager-645ffbd459-p959l" Jul 10 19:00:40.059832 ip-10-0-143-17 kubenswrapper[2585]: I0710 19:00:40.059813 2585 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"cert\" (UniqueName: \"kubernetes.io/secret/4c4024ac-8ae6-43fa-9a94-9336e896afde-cert\") pod \"llmisvc-controller-manager-645ffbd459-p959l\" (UID: \"4c4024ac-8ae6-43fa-9a94-9336e896afde\") " pod="kserve/llmisvc-controller-manager-645ffbd459-p959l" Jul 10 19:00:40.067486 ip-10-0-143-17 kubenswrapper[2585]: I0710 19:00:40.067465 2585 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-f6dqn\" (UniqueName: \"kubernetes.io/projected/4c4024ac-8ae6-43fa-9a94-9336e896afde-kube-api-access-f6dqn\") pod \"llmisvc-controller-manager-645ffbd459-p959l\" (UID: \"4c4024ac-8ae6-43fa-9a94-9336e896afde\") " pod="kserve/llmisvc-controller-manager-645ffbd459-p959l" Jul 10 19:00:40.135233 ip-10-0-143-17 kubenswrapper[2585]: I0710 19:00:40.135183 2585 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kserve/llmisvc-controller-manager-645ffbd459-p959l" Jul 10 19:00:40.474507 ip-10-0-143-17 kubenswrapper[2585]: I0710 19:00:40.474483 2585 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kserve/llmisvc-controller-manager-645ffbd459-p959l"] Jul 10 19:00:40.476113 ip-10-0-143-17 kubenswrapper[2585]: W0710 19:00:40.476087 2585 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-pod4c4024ac_8ae6_43fa_9a94_9336e896afde.slice/crio-eeb82c32f0f458d98dee80118aa72deac31024c8d2126300d37337dccad950a3 WatchSource:0}: Error finding container eeb82c32f0f458d98dee80118aa72deac31024c8d2126300d37337dccad950a3: Status 404 returned error can't find the container with id eeb82c32f0f458d98dee80118aa72deac31024c8d2126300d37337dccad950a3 Jul 10 19:00:40.477485 ip-10-0-143-17 kubenswrapper[2585]: I0710 19:00:40.477467 2585 provider.go:93] Refreshing cache for provider: *credentialprovider.defaultDockerConfigProvider Jul 10 19:00:41.273178 ip-10-0-143-17 kubenswrapper[2585]: I0710 19:00:41.273146 2585 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kserve/llmisvc-controller-manager-645ffbd459-p959l" event={"ID":"4c4024ac-8ae6-43fa-9a94-9336e896afde","Type":"ContainerStarted","Data":"eeb82c32f0f458d98dee80118aa72deac31024c8d2126300d37337dccad950a3"} Jul 10 19:00:43.285853 ip-10-0-143-17 kubenswrapper[2585]: I0710 19:00:43.285820 2585 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kserve/llmisvc-controller-manager-645ffbd459-p959l" event={"ID":"4c4024ac-8ae6-43fa-9a94-9336e896afde","Type":"ContainerStarted","Data":"ac151c81a84e90b5fa7f29554ce94e84c5c3f2ad2b66bd79239a7b4f4121d66f"} Jul 10 19:00:43.286233 ip-10-0-143-17 kubenswrapper[2585]: I0710 19:00:43.285950 2585 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="kserve/llmisvc-controller-manager-645ffbd459-p959l" Jul 10 19:00:43.311680 ip-10-0-143-17 kubenswrapper[2585]: I0710 19:00:43.311636 2585 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="kserve/llmisvc-controller-manager-645ffbd459-p959l" podStartSLOduration=1.670420163 podStartE2EDuration="4.311622083s" totalImagesPullingTime="2.64120192s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-10 19:00:39 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-10 19:00:43.30946961 +0000 UTC m=+1256.766829568" watchObservedRunningTime="2026-07-10 19:00:43.311622083 +0000 UTC m=+1256.768982050" Jul 10 19:01:14.293152 ip-10-0-143-17 kubenswrapper[2585]: I0710 19:01:14.293115 2585 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="kserve/llmisvc-controller-manager-645ffbd459-p959l" Jul 10 19:01:51.374386 ip-10-0-143-17 kubenswrapper[2585]: I0710 19:01:51.374352 2585 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["kserve-ci-e2e-test/router-gateway-2-openshift-default-54c789bdc6-66nfj"] Jul 10 19:01:51.377749 ip-10-0-143-17 kubenswrapper[2585]: I0710 19:01:51.377732 2585 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kserve-ci-e2e-test/router-gateway-2-openshift-default-54c789bdc6-66nfj" Jul 10 19:01:51.381265 ip-10-0-143-17 kubenswrapper[2585]: I0710 19:01:51.381243 2585 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"kserve-ci-e2e-test\"/\"router-gateway-2-openshift-default-dockercfg-v9mpg\"" Jul 10 19:01:51.396738 ip-10-0-143-17 kubenswrapper[2585]: I0710 19:01:51.396715 2585 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kserve-ci-e2e-test/router-gateway-2-openshift-default-54c789bdc6-66nfj"] Jul 10 19:01:51.445218 ip-10-0-143-17 kubenswrapper[2585]: I0710 19:01:51.445194 2585 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/7aa2cf75-c55d-45ff-a4ca-9bfba3d0a016-istio-token\") pod \"router-gateway-2-openshift-default-54c789bdc6-66nfj\" (UID: \"7aa2cf75-c55d-45ff-a4ca-9bfba3d0a016\") " pod="kserve-ci-e2e-test/router-gateway-2-openshift-default-54c789bdc6-66nfj" Jul 10 19:01:51.445307 ip-10-0-143-17 kubenswrapper[2585]: I0710 19:01:51.445222 2585 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/7aa2cf75-c55d-45ff-a4ca-9bfba3d0a016-istio-envoy\") pod \"router-gateway-2-openshift-default-54c789bdc6-66nfj\" (UID: \"7aa2cf75-c55d-45ff-a4ca-9bfba3d0a016\") " pod="kserve-ci-e2e-test/router-gateway-2-openshift-default-54c789bdc6-66nfj" Jul 10 19:01:51.445307 ip-10-0-143-17 kubenswrapper[2585]: I0710 19:01:51.445245 2585 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/7aa2cf75-c55d-45ff-a4ca-9bfba3d0a016-istio-data\") pod \"router-gateway-2-openshift-default-54c789bdc6-66nfj\" (UID: \"7aa2cf75-c55d-45ff-a4ca-9bfba3d0a016\") " pod="kserve-ci-e2e-test/router-gateway-2-openshift-default-54c789bdc6-66nfj" Jul 10 19:01:51.445379 ip-10-0-143-17 kubenswrapper[2585]: I0710 19:01:51.445318 2585 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/7aa2cf75-c55d-45ff-a4ca-9bfba3d0a016-istio-podinfo\") pod \"router-gateway-2-openshift-default-54c789bdc6-66nfj\" (UID: \"7aa2cf75-c55d-45ff-a4ca-9bfba3d0a016\") " pod="kserve-ci-e2e-test/router-gateway-2-openshift-default-54c789bdc6-66nfj" Jul 10 19:01:51.445379 ip-10-0-143-17 kubenswrapper[2585]: I0710 19:01:51.445358 2585 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/7aa2cf75-c55d-45ff-a4ca-9bfba3d0a016-workload-socket\") pod \"router-gateway-2-openshift-default-54c789bdc6-66nfj\" (UID: \"7aa2cf75-c55d-45ff-a4ca-9bfba3d0a016\") " pod="kserve-ci-e2e-test/router-gateway-2-openshift-default-54c789bdc6-66nfj" Jul 10 19:01:51.445379 ip-10-0-143-17 kubenswrapper[2585]: I0710 19:01:51.445377 2585 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/7aa2cf75-c55d-45ff-a4ca-9bfba3d0a016-workload-certs\") pod \"router-gateway-2-openshift-default-54c789bdc6-66nfj\" (UID: \"7aa2cf75-c55d-45ff-a4ca-9bfba3d0a016\") " pod="kserve-ci-e2e-test/router-gateway-2-openshift-default-54c789bdc6-66nfj" Jul 10 19:01:51.445477 ip-10-0-143-17 kubenswrapper[2585]: I0710 19:01:51.445440 2585 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/7aa2cf75-c55d-45ff-a4ca-9bfba3d0a016-istiod-ca-cert\") pod \"router-gateway-2-openshift-default-54c789bdc6-66nfj\" (UID: \"7aa2cf75-c55d-45ff-a4ca-9bfba3d0a016\") " pod="kserve-ci-e2e-test/router-gateway-2-openshift-default-54c789bdc6-66nfj" Jul 10 19:01:51.445477 ip-10-0-143-17 kubenswrapper[2585]: I0710 19:01:51.445466 2585 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-9p4h5\" (UniqueName: \"kubernetes.io/projected/7aa2cf75-c55d-45ff-a4ca-9bfba3d0a016-kube-api-access-9p4h5\") pod \"router-gateway-2-openshift-default-54c789bdc6-66nfj\" (UID: \"7aa2cf75-c55d-45ff-a4ca-9bfba3d0a016\") " pod="kserve-ci-e2e-test/router-gateway-2-openshift-default-54c789bdc6-66nfj" Jul 10 19:01:51.445558 ip-10-0-143-17 kubenswrapper[2585]: I0710 19:01:51.445494 2585 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/7aa2cf75-c55d-45ff-a4ca-9bfba3d0a016-credential-socket\") pod \"router-gateway-2-openshift-default-54c789bdc6-66nfj\" (UID: \"7aa2cf75-c55d-45ff-a4ca-9bfba3d0a016\") " pod="kserve-ci-e2e-test/router-gateway-2-openshift-default-54c789bdc6-66nfj" Jul 10 19:01:51.546612 ip-10-0-143-17 kubenswrapper[2585]: I0710 19:01:51.546585 2585 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/7aa2cf75-c55d-45ff-a4ca-9bfba3d0a016-istio-token\") pod \"router-gateway-2-openshift-default-54c789bdc6-66nfj\" (UID: \"7aa2cf75-c55d-45ff-a4ca-9bfba3d0a016\") " pod="kserve-ci-e2e-test/router-gateway-2-openshift-default-54c789bdc6-66nfj" Jul 10 19:01:51.546753 ip-10-0-143-17 kubenswrapper[2585]: I0710 19:01:51.546620 2585 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/7aa2cf75-c55d-45ff-a4ca-9bfba3d0a016-istio-envoy\") pod \"router-gateway-2-openshift-default-54c789bdc6-66nfj\" (UID: \"7aa2cf75-c55d-45ff-a4ca-9bfba3d0a016\") " pod="kserve-ci-e2e-test/router-gateway-2-openshift-default-54c789bdc6-66nfj" Jul 10 19:01:51.546753 ip-10-0-143-17 kubenswrapper[2585]: I0710 19:01:51.546643 2585 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/7aa2cf75-c55d-45ff-a4ca-9bfba3d0a016-istio-data\") pod \"router-gateway-2-openshift-default-54c789bdc6-66nfj\" (UID: \"7aa2cf75-c55d-45ff-a4ca-9bfba3d0a016\") " pod="kserve-ci-e2e-test/router-gateway-2-openshift-default-54c789bdc6-66nfj" Jul 10 19:01:51.546753 ip-10-0-143-17 kubenswrapper[2585]: I0710 19:01:51.546670 2585 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/7aa2cf75-c55d-45ff-a4ca-9bfba3d0a016-istio-podinfo\") pod \"router-gateway-2-openshift-default-54c789bdc6-66nfj\" (UID: \"7aa2cf75-c55d-45ff-a4ca-9bfba3d0a016\") " pod="kserve-ci-e2e-test/router-gateway-2-openshift-default-54c789bdc6-66nfj" Jul 10 19:01:51.546753 ip-10-0-143-17 kubenswrapper[2585]: I0710 19:01:51.546711 2585 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/7aa2cf75-c55d-45ff-a4ca-9bfba3d0a016-workload-socket\") pod \"router-gateway-2-openshift-default-54c789bdc6-66nfj\" (UID: \"7aa2cf75-c55d-45ff-a4ca-9bfba3d0a016\") " pod="kserve-ci-e2e-test/router-gateway-2-openshift-default-54c789bdc6-66nfj" Jul 10 19:01:51.546753 ip-10-0-143-17 kubenswrapper[2585]: I0710 19:01:51.546736 2585 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/7aa2cf75-c55d-45ff-a4ca-9bfba3d0a016-workload-certs\") pod \"router-gateway-2-openshift-default-54c789bdc6-66nfj\" (UID: \"7aa2cf75-c55d-45ff-a4ca-9bfba3d0a016\") " pod="kserve-ci-e2e-test/router-gateway-2-openshift-default-54c789bdc6-66nfj" Jul 10 19:01:51.547028 ip-10-0-143-17 kubenswrapper[2585]: I0710 19:01:51.546795 2585 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/7aa2cf75-c55d-45ff-a4ca-9bfba3d0a016-istiod-ca-cert\") pod \"router-gateway-2-openshift-default-54c789bdc6-66nfj\" (UID: \"7aa2cf75-c55d-45ff-a4ca-9bfba3d0a016\") " pod="kserve-ci-e2e-test/router-gateway-2-openshift-default-54c789bdc6-66nfj" Jul 10 19:01:51.547028 ip-10-0-143-17 kubenswrapper[2585]: I0710 19:01:51.546818 2585 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-9p4h5\" (UniqueName: \"kubernetes.io/projected/7aa2cf75-c55d-45ff-a4ca-9bfba3d0a016-kube-api-access-9p4h5\") pod \"router-gateway-2-openshift-default-54c789bdc6-66nfj\" (UID: \"7aa2cf75-c55d-45ff-a4ca-9bfba3d0a016\") " pod="kserve-ci-e2e-test/router-gateway-2-openshift-default-54c789bdc6-66nfj" Jul 10 19:01:51.547028 ip-10-0-143-17 kubenswrapper[2585]: I0710 19:01:51.546864 2585 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/7aa2cf75-c55d-45ff-a4ca-9bfba3d0a016-credential-socket\") pod \"router-gateway-2-openshift-default-54c789bdc6-66nfj\" (UID: \"7aa2cf75-c55d-45ff-a4ca-9bfba3d0a016\") " pod="kserve-ci-e2e-test/router-gateway-2-openshift-default-54c789bdc6-66nfj" Jul 10 19:01:51.547195 ip-10-0-143-17 kubenswrapper[2585]: I0710 19:01:51.547058 2585 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/7aa2cf75-c55d-45ff-a4ca-9bfba3d0a016-istio-data\") pod \"router-gateway-2-openshift-default-54c789bdc6-66nfj\" (UID: \"7aa2cf75-c55d-45ff-a4ca-9bfba3d0a016\") " pod="kserve-ci-e2e-test/router-gateway-2-openshift-default-54c789bdc6-66nfj" Jul 10 19:01:51.547195 ip-10-0-143-17 kubenswrapper[2585]: I0710 19:01:51.547146 2585 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/7aa2cf75-c55d-45ff-a4ca-9bfba3d0a016-credential-socket\") pod \"router-gateway-2-openshift-default-54c789bdc6-66nfj\" (UID: \"7aa2cf75-c55d-45ff-a4ca-9bfba3d0a016\") " pod="kserve-ci-e2e-test/router-gateway-2-openshift-default-54c789bdc6-66nfj" Jul 10 19:01:51.547515 ip-10-0-143-17 kubenswrapper[2585]: I0710 19:01:51.547491 2585 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/7aa2cf75-c55d-45ff-a4ca-9bfba3d0a016-workload-socket\") pod \"router-gateway-2-openshift-default-54c789bdc6-66nfj\" (UID: \"7aa2cf75-c55d-45ff-a4ca-9bfba3d0a016\") " pod="kserve-ci-e2e-test/router-gateway-2-openshift-default-54c789bdc6-66nfj" Jul 10 19:01:51.547653 ip-10-0-143-17 kubenswrapper[2585]: I0710 19:01:51.547577 2585 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/7aa2cf75-c55d-45ff-a4ca-9bfba3d0a016-workload-certs\") pod \"router-gateway-2-openshift-default-54c789bdc6-66nfj\" (UID: \"7aa2cf75-c55d-45ff-a4ca-9bfba3d0a016\") " pod="kserve-ci-e2e-test/router-gateway-2-openshift-default-54c789bdc6-66nfj" Jul 10 19:01:51.547946 ip-10-0-143-17 kubenswrapper[2585]: I0710 19:01:51.547923 2585 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/7aa2cf75-c55d-45ff-a4ca-9bfba3d0a016-istiod-ca-cert\") pod \"router-gateway-2-openshift-default-54c789bdc6-66nfj\" (UID: \"7aa2cf75-c55d-45ff-a4ca-9bfba3d0a016\") " pod="kserve-ci-e2e-test/router-gateway-2-openshift-default-54c789bdc6-66nfj" Jul 10 19:01:51.549293 ip-10-0-143-17 kubenswrapper[2585]: I0710 19:01:51.549271 2585 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/7aa2cf75-c55d-45ff-a4ca-9bfba3d0a016-istio-envoy\") pod \"router-gateway-2-openshift-default-54c789bdc6-66nfj\" (UID: \"7aa2cf75-c55d-45ff-a4ca-9bfba3d0a016\") " pod="kserve-ci-e2e-test/router-gateway-2-openshift-default-54c789bdc6-66nfj" Jul 10 19:01:51.549405 ip-10-0-143-17 kubenswrapper[2585]: I0710 19:01:51.549302 2585 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/7aa2cf75-c55d-45ff-a4ca-9bfba3d0a016-istio-podinfo\") pod \"router-gateway-2-openshift-default-54c789bdc6-66nfj\" (UID: \"7aa2cf75-c55d-45ff-a4ca-9bfba3d0a016\") " pod="kserve-ci-e2e-test/router-gateway-2-openshift-default-54c789bdc6-66nfj" Jul 10 19:01:51.557290 ip-10-0-143-17 kubenswrapper[2585]: I0710 19:01:51.557265 2585 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/7aa2cf75-c55d-45ff-a4ca-9bfba3d0a016-istio-token\") pod \"router-gateway-2-openshift-default-54c789bdc6-66nfj\" (UID: \"7aa2cf75-c55d-45ff-a4ca-9bfba3d0a016\") " pod="kserve-ci-e2e-test/router-gateway-2-openshift-default-54c789bdc6-66nfj" Jul 10 19:01:51.559363 ip-10-0-143-17 kubenswrapper[2585]: I0710 19:01:51.559339 2585 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-9p4h5\" (UniqueName: \"kubernetes.io/projected/7aa2cf75-c55d-45ff-a4ca-9bfba3d0a016-kube-api-access-9p4h5\") pod \"router-gateway-2-openshift-default-54c789bdc6-66nfj\" (UID: \"7aa2cf75-c55d-45ff-a4ca-9bfba3d0a016\") " pod="kserve-ci-e2e-test/router-gateway-2-openshift-default-54c789bdc6-66nfj" Jul 10 19:01:51.688307 ip-10-0-143-17 kubenswrapper[2585]: I0710 19:01:51.688234 2585 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kserve-ci-e2e-test/router-gateway-2-openshift-default-54c789bdc6-66nfj" Jul 10 19:01:51.834664 ip-10-0-143-17 kubenswrapper[2585]: I0710 19:01:51.834642 2585 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kserve-ci-e2e-test/router-gateway-2-openshift-default-54c789bdc6-66nfj"] Jul 10 19:01:51.836580 ip-10-0-143-17 kubenswrapper[2585]: W0710 19:01:51.836549 2585 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod7aa2cf75_c55d_45ff_a4ca_9bfba3d0a016.slice/crio-3b1a65bdd021a4f6063938e2cb9589c02c2149d955254c79dbbbf26a5174d2a3 WatchSource:0}: Error finding container 3b1a65bdd021a4f6063938e2cb9589c02c2149d955254c79dbbbf26a5174d2a3: Status 404 returned error can't find the container with id 3b1a65bdd021a4f6063938e2cb9589c02c2149d955254c79dbbbf26a5174d2a3 Jul 10 19:01:51.838658 ip-10-0-143-17 kubenswrapper[2585]: I0710 19:01:51.838625 2585 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236224Ki","pods":"250"} Jul 10 19:01:51.838733 ip-10-0-143-17 kubenswrapper[2585]: I0710 19:01:51.838692 2585 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236224Ki","pods":"250"} Jul 10 19:01:51.838733 ip-10-0-143-17 kubenswrapper[2585]: I0710 19:01:51.838722 2585 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236224Ki","pods":"250"} Jul 10 19:01:52.632689 ip-10-0-143-17 kubenswrapper[2585]: I0710 19:01:52.632656 2585 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kserve-ci-e2e-test/router-gateway-2-openshift-default-54c789bdc6-66nfj" event={"ID":"7aa2cf75-c55d-45ff-a4ca-9bfba3d0a016","Type":"ContainerStarted","Data":"00da77673eb7cd0a6919e833c55018568b6507be7abf2815805d4ae82bee5c11"} Jul 10 19:01:52.633135 ip-10-0-143-17 kubenswrapper[2585]: I0710 19:01:52.632707 2585 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kserve-ci-e2e-test/router-gateway-2-openshift-default-54c789bdc6-66nfj" event={"ID":"7aa2cf75-c55d-45ff-a4ca-9bfba3d0a016","Type":"ContainerStarted","Data":"3b1a65bdd021a4f6063938e2cb9589c02c2149d955254c79dbbbf26a5174d2a3"} Jul 10 19:01:52.667214 ip-10-0-143-17 kubenswrapper[2585]: I0710 19:01:52.667165 2585 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="kserve-ci-e2e-test/router-gateway-2-openshift-default-54c789bdc6-66nfj" podStartSLOduration=1.6671502999999999 podStartE2EDuration="1.6671503s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-10 19:01:51 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-10 19:01:52.666275176 +0000 UTC m=+1326.123635107" watchObservedRunningTime="2026-07-10 19:01:52.6671503 +0000 UTC m=+1326.124510308" Jul 10 19:01:52.689323 ip-10-0-143-17 kubenswrapper[2585]: I0710 19:01:52.689297 2585 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="kserve-ci-e2e-test/router-gateway-2-openshift-default-54c789bdc6-66nfj" Jul 10 19:01:52.694014 ip-10-0-143-17 kubenswrapper[2585]: I0710 19:01:52.693996 2585 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="kserve-ci-e2e-test/router-gateway-2-openshift-default-54c789bdc6-66nfj" Jul 10 19:01:53.638164 ip-10-0-143-17 kubenswrapper[2585]: I0710 19:01:53.638136 2585 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="kserve-ci-e2e-test/router-gateway-2-openshift-default-54c789bdc6-66nfj" Jul 10 19:01:53.638940 ip-10-0-143-17 kubenswrapper[2585]: I0710 19:01:53.638921 2585 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="kserve-ci-e2e-test/router-gateway-2-openshift-default-54c789bdc6-66nfj" Jul 10 19:02:03.713778 ip-10-0-143-17 kubenswrapper[2585]: I0710 19:02:03.713744 2585 generic.go:356] "Generic (PLEG): container finished" podID="7aa2cf75-c55d-45ff-a4ca-9bfba3d0a016" containerID="00da77673eb7cd0a6919e833c55018568b6507be7abf2815805d4ae82bee5c11" exitCode=0 Jul 10 19:02:03.714182 ip-10-0-143-17 kubenswrapper[2585]: I0710 19:02:03.713814 2585 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kserve-ci-e2e-test/router-gateway-2-openshift-default-54c789bdc6-66nfj" event={"ID":"7aa2cf75-c55d-45ff-a4ca-9bfba3d0a016","Type":"ContainerDied","Data":"00da77673eb7cd0a6919e833c55018568b6507be7abf2815805d4ae82bee5c11"} Jul 10 19:02:03.714182 ip-10-0-143-17 kubenswrapper[2585]: I0710 19:02:03.714126 2585 scope.go:117] "RemoveContainer" containerID="00da77673eb7cd0a6919e833c55018568b6507be7abf2815805d4ae82bee5c11" Jul 10 19:02:03.714986 ip-10-0-143-17 kubenswrapper[2585]: I0710 19:02:03.714960 2585 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236224Ki","pods":"250"} Jul 10 19:02:03.715047 ip-10-0-143-17 kubenswrapper[2585]: I0710 19:02:03.715027 2585 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236224Ki","pods":"250"} Jul 10 19:02:03.715083 ip-10-0-143-17 kubenswrapper[2585]: I0710 19:02:03.715058 2585 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236224Ki","pods":"250"} Jul 10 19:02:04.720949 ip-10-0-143-17 kubenswrapper[2585]: I0710 19:02:04.720916 2585 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kserve-ci-e2e-test/router-gateway-2-openshift-default-54c789bdc6-66nfj" event={"ID":"7aa2cf75-c55d-45ff-a4ca-9bfba3d0a016","Type":"ContainerStarted","Data":"65f06c8671488500aff9ca055d30bccb1caca8f7f37be3a73576da36220d4b2e"} Jul 10 19:02:05.689130 ip-10-0-143-17 kubenswrapper[2585]: I0710 19:02:05.689091 2585 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="kserve-ci-e2e-test/router-gateway-2-openshift-default-54c789bdc6-66nfj" Jul 10 19:02:05.690257 ip-10-0-143-17 kubenswrapper[2585]: I0710 19:02:05.690228 2585 prober.go:120] "Probe failed" probeType="Startup" pod="kserve-ci-e2e-test/router-gateway-2-openshift-default-54c789bdc6-66nfj" podUID="7aa2cf75-c55d-45ff-a4ca-9bfba3d0a016" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.133.0.49:15021/healthz/ready\": dial tcp 10.133.0.49:15021: connect: connection refused" Jul 10 19:02:06.689206 ip-10-0-143-17 kubenswrapper[2585]: I0710 19:02:06.689164 2585 prober.go:120] "Probe failed" probeType="Startup" pod="kserve-ci-e2e-test/router-gateway-2-openshift-default-54c789bdc6-66nfj" podUID="7aa2cf75-c55d-45ff-a4ca-9bfba3d0a016" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.133.0.49:15021/healthz/ready\": dial tcp 10.133.0.49:15021: connect: connection refused" Jul 10 19:02:07.689271 ip-10-0-143-17 kubenswrapper[2585]: I0710 19:02:07.689231 2585 prober.go:120] "Probe failed" probeType="Startup" pod="kserve-ci-e2e-test/router-gateway-2-openshift-default-54c789bdc6-66nfj" podUID="7aa2cf75-c55d-45ff-a4ca-9bfba3d0a016" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.133.0.49:15021/healthz/ready\": dial tcp 10.133.0.49:15021: connect: connection refused" Jul 10 19:02:08.638265 ip-10-0-143-17 kubenswrapper[2585]: I0710 19:02:08.638230 2585 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="kserve-ci-e2e-test/router-gateway-2-openshift-default-54c789bdc6-66nfj" Jul 10 19:02:08.689141 ip-10-0-143-17 kubenswrapper[2585]: I0710 19:02:08.689106 2585 prober.go:120] "Probe failed" probeType="Startup" pod="kserve-ci-e2e-test/router-gateway-2-openshift-default-54c789bdc6-66nfj" podUID="7aa2cf75-c55d-45ff-a4ca-9bfba3d0a016" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.133.0.49:15021/healthz/ready\": dial tcp 10.133.0.49:15021: connect: connection refused" Jul 10 19:02:09.689602 ip-10-0-143-17 kubenswrapper[2585]: I0710 19:02:09.689472 2585 prober.go:120] "Probe failed" probeType="Startup" pod="kserve-ci-e2e-test/router-gateway-2-openshift-default-54c789bdc6-66nfj" podUID="7aa2cf75-c55d-45ff-a4ca-9bfba3d0a016" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.133.0.49:15021/healthz/ready\": dial tcp 10.133.0.49:15021: connect: connection refused" Jul 10 19:02:10.692838 ip-10-0-143-17 kubenswrapper[2585]: I0710 19:02:10.692808 2585 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="kserve-ci-e2e-test/router-gateway-2-openshift-default-54c789bdc6-66nfj" Jul 10 19:02:10.693781 ip-10-0-143-17 kubenswrapper[2585]: I0710 19:02:10.693762 2585 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="kserve-ci-e2e-test/router-gateway-2-openshift-default-54c789bdc6-66nfj" Jul 10 19:04:47.094408 ip-10-0-143-17 kubenswrapper[2585]: I0710 19:04:47.094378 2585 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ovn-kubernetes_ovnkube-node-xg6bm_d863cfea-f2f3-4404-b62a-d41785525f3f/ovn-acl-logging/0.log" Jul 10 19:04:47.098250 ip-10-0-143-17 kubenswrapper[2585]: I0710 19:04:47.098226 2585 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ovn-kubernetes_ovnkube-node-xg6bm_d863cfea-f2f3-4404-b62a-d41785525f3f/ovn-acl-logging/0.log" Jul 10 19:09:47.126911 ip-10-0-143-17 kubenswrapper[2585]: I0710 19:09:47.126883 2585 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ovn-kubernetes_ovnkube-node-xg6bm_d863cfea-f2f3-4404-b62a-d41785525f3f/ovn-acl-logging/0.log" Jul 10 19:09:47.132632 ip-10-0-143-17 kubenswrapper[2585]: I0710 19:09:47.132612 2585 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ovn-kubernetes_ovnkube-node-xg6bm_d863cfea-f2f3-4404-b62a-d41785525f3f/ovn-acl-logging/0.log" Jul 10 19:14:47.165389 ip-10-0-143-17 kubenswrapper[2585]: I0710 19:14:47.165357 2585 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ovn-kubernetes_ovnkube-node-xg6bm_d863cfea-f2f3-4404-b62a-d41785525f3f/ovn-acl-logging/0.log" Jul 10 19:14:47.172424 ip-10-0-143-17 kubenswrapper[2585]: I0710 19:14:47.172405 2585 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ovn-kubernetes_ovnkube-node-xg6bm_d863cfea-f2f3-4404-b62a-d41785525f3f/ovn-acl-logging/0.log" Jul 10 19:19:13.689731 ip-10-0-143-17 kubenswrapper[2585]: I0710 19:19:13.689696 2585 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["kserve-ci-e2e-test/llmisvc-model-fb-opt-125m-route-f312f5ec-kserve-mn-0-1"] Jul 10 19:19:13.693300 ip-10-0-143-17 kubenswrapper[2585]: I0710 19:19:13.693284 2585 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kserve-ci-e2e-test/llmisvc-model-fb-opt-125m-route-f312f5ec-kserve-mn-0-1" Jul 10 19:19:13.697123 ip-10-0-143-17 kubenswrapper[2585]: I0710 19:19:13.697094 2585 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"kserve-ci-e2e-test\"/\"default-dockercfg-jjdpb\"" Jul 10 19:19:13.697710 ip-10-0-143-17 kubenswrapper[2585]: I0710 19:19:13.697693 2585 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"kserve-ci-e2e-test\"/\"llmisvc-model-fb-opt-125m-route-f312f5-cb7fb8cf-dockercfg-762zd\"" Jul 10 19:19:13.697863 ip-10-0-143-17 kubenswrapper[2585]: I0710 19:19:13.697846 2585 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"kserve-ci-e2e-test\"/\"llmisv4e643bc258191ffc517a31cd1d0ddd27-kserve-self-signed-certs\"" Jul 10 19:19:13.706946 ip-10-0-143-17 kubenswrapper[2585]: I0710 19:19:13.706927 2585 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kserve-ci-e2e-test/llmisvc-model-fb-opt-125m-route-f312f5ec-kserve-mn-0-1"] Jul 10 19:19:13.740725 ip-10-0-143-17 kubenswrapper[2585]: I0710 19:19:13.740698 2585 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/59ed214a-e61e-4dff-9811-87c0c02e39c5-home\") pod \"llmisvc-model-fb-opt-125m-route-f312f5ec-kserve-mn-0-1\" (UID: \"59ed214a-e61e-4dff-9811-87c0c02e39c5\") " pod="kserve-ci-e2e-test/llmisvc-model-fb-opt-125m-route-f312f5ec-kserve-mn-0-1" Jul 10 19:19:13.740824 ip-10-0-143-17 kubenswrapper[2585]: I0710 19:19:13.740742 2585 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/59ed214a-e61e-4dff-9811-87c0c02e39c5-tmp-dir\") pod \"llmisvc-model-fb-opt-125m-route-f312f5ec-kserve-mn-0-1\" (UID: \"59ed214a-e61e-4dff-9811-87c0c02e39c5\") " pod="kserve-ci-e2e-test/llmisvc-model-fb-opt-125m-route-f312f5ec-kserve-mn-0-1" Jul 10 19:19:13.740896 ip-10-0-143-17 kubenswrapper[2585]: I0710 19:19:13.740848 2585 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/59ed214a-e61e-4dff-9811-87c0c02e39c5-model-cache\") pod \"llmisvc-model-fb-opt-125m-route-f312f5ec-kserve-mn-0-1\" (UID: \"59ed214a-e61e-4dff-9811-87c0c02e39c5\") " pod="kserve-ci-e2e-test/llmisvc-model-fb-opt-125m-route-f312f5ec-kserve-mn-0-1" Jul 10 19:19:13.740896 ip-10-0-143-17 kubenswrapper[2585]: I0710 19:19:13.740885 2585 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/59ed214a-e61e-4dff-9811-87c0c02e39c5-tls-certs\") pod \"llmisvc-model-fb-opt-125m-route-f312f5ec-kserve-mn-0-1\" (UID: \"59ed214a-e61e-4dff-9811-87c0c02e39c5\") " pod="kserve-ci-e2e-test/llmisvc-model-fb-opt-125m-route-f312f5ec-kserve-mn-0-1" Jul 10 19:19:13.741001 ip-10-0-143-17 kubenswrapper[2585]: I0710 19:19:13.740909 2585 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-fmnsq\" (UniqueName: \"kubernetes.io/projected/59ed214a-e61e-4dff-9811-87c0c02e39c5-kube-api-access-fmnsq\") pod \"llmisvc-model-fb-opt-125m-route-f312f5ec-kserve-mn-0-1\" (UID: \"59ed214a-e61e-4dff-9811-87c0c02e39c5\") " pod="kserve-ci-e2e-test/llmisvc-model-fb-opt-125m-route-f312f5ec-kserve-mn-0-1" Jul 10 19:19:13.741001 ip-10-0-143-17 kubenswrapper[2585]: I0710 19:19:13.740950 2585 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/59ed214a-e61e-4dff-9811-87c0c02e39c5-dshm\") pod \"llmisvc-model-fb-opt-125m-route-f312f5ec-kserve-mn-0-1\" (UID: \"59ed214a-e61e-4dff-9811-87c0c02e39c5\") " pod="kserve-ci-e2e-test/llmisvc-model-fb-opt-125m-route-f312f5ec-kserve-mn-0-1" Jul 10 19:19:13.741001 ip-10-0-143-17 kubenswrapper[2585]: I0710 19:19:13.740973 2585 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kserve-provision-location\" (UniqueName: \"kubernetes.io/empty-dir/59ed214a-e61e-4dff-9811-87c0c02e39c5-kserve-provision-location\") pod \"llmisvc-model-fb-opt-125m-route-f312f5ec-kserve-mn-0-1\" (UID: \"59ed214a-e61e-4dff-9811-87c0c02e39c5\") " pod="kserve-ci-e2e-test/llmisvc-model-fb-opt-125m-route-f312f5ec-kserve-mn-0-1" Jul 10 19:19:13.841968 ip-10-0-143-17 kubenswrapper[2585]: I0710 19:19:13.841925 2585 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/59ed214a-e61e-4dff-9811-87c0c02e39c5-dshm\") pod \"llmisvc-model-fb-opt-125m-route-f312f5ec-kserve-mn-0-1\" (UID: \"59ed214a-e61e-4dff-9811-87c0c02e39c5\") " pod="kserve-ci-e2e-test/llmisvc-model-fb-opt-125m-route-f312f5ec-kserve-mn-0-1" Jul 10 19:19:13.842110 ip-10-0-143-17 kubenswrapper[2585]: I0710 19:19:13.842005 2585 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kserve-provision-location\" (UniqueName: \"kubernetes.io/empty-dir/59ed214a-e61e-4dff-9811-87c0c02e39c5-kserve-provision-location\") pod \"llmisvc-model-fb-opt-125m-route-f312f5ec-kserve-mn-0-1\" (UID: \"59ed214a-e61e-4dff-9811-87c0c02e39c5\") " pod="kserve-ci-e2e-test/llmisvc-model-fb-opt-125m-route-f312f5ec-kserve-mn-0-1" Jul 10 19:19:13.842110 ip-10-0-143-17 kubenswrapper[2585]: I0710 19:19:13.842096 2585 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/59ed214a-e61e-4dff-9811-87c0c02e39c5-home\") pod \"llmisvc-model-fb-opt-125m-route-f312f5ec-kserve-mn-0-1\" (UID: \"59ed214a-e61e-4dff-9811-87c0c02e39c5\") " pod="kserve-ci-e2e-test/llmisvc-model-fb-opt-125m-route-f312f5ec-kserve-mn-0-1" Jul 10 19:19:13.842239 ip-10-0-143-17 kubenswrapper[2585]: I0710 19:19:13.842142 2585 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/59ed214a-e61e-4dff-9811-87c0c02e39c5-tmp-dir\") pod \"llmisvc-model-fb-opt-125m-route-f312f5ec-kserve-mn-0-1\" (UID: \"59ed214a-e61e-4dff-9811-87c0c02e39c5\") " pod="kserve-ci-e2e-test/llmisvc-model-fb-opt-125m-route-f312f5ec-kserve-mn-0-1" Jul 10 19:19:13.842239 ip-10-0-143-17 kubenswrapper[2585]: I0710 19:19:13.842224 2585 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/59ed214a-e61e-4dff-9811-87c0c02e39c5-model-cache\") pod \"llmisvc-model-fb-opt-125m-route-f312f5ec-kserve-mn-0-1\" (UID: \"59ed214a-e61e-4dff-9811-87c0c02e39c5\") " pod="kserve-ci-e2e-test/llmisvc-model-fb-opt-125m-route-f312f5ec-kserve-mn-0-1" Jul 10 19:19:13.842357 ip-10-0-143-17 kubenswrapper[2585]: I0710 19:19:13.842272 2585 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/59ed214a-e61e-4dff-9811-87c0c02e39c5-tls-certs\") pod \"llmisvc-model-fb-opt-125m-route-f312f5ec-kserve-mn-0-1\" (UID: \"59ed214a-e61e-4dff-9811-87c0c02e39c5\") " pod="kserve-ci-e2e-test/llmisvc-model-fb-opt-125m-route-f312f5ec-kserve-mn-0-1" Jul 10 19:19:13.842357 ip-10-0-143-17 kubenswrapper[2585]: I0710 19:19:13.842299 2585 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-fmnsq\" (UniqueName: \"kubernetes.io/projected/59ed214a-e61e-4dff-9811-87c0c02e39c5-kube-api-access-fmnsq\") pod \"llmisvc-model-fb-opt-125m-route-f312f5ec-kserve-mn-0-1\" (UID: \"59ed214a-e61e-4dff-9811-87c0c02e39c5\") " pod="kserve-ci-e2e-test/llmisvc-model-fb-opt-125m-route-f312f5ec-kserve-mn-0-1" Jul 10 19:19:13.842452 ip-10-0-143-17 kubenswrapper[2585]: I0710 19:19:13.842364 2585 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kserve-provision-location\" (UniqueName: \"kubernetes.io/empty-dir/59ed214a-e61e-4dff-9811-87c0c02e39c5-kserve-provision-location\") pod \"llmisvc-model-fb-opt-125m-route-f312f5ec-kserve-mn-0-1\" (UID: \"59ed214a-e61e-4dff-9811-87c0c02e39c5\") " pod="kserve-ci-e2e-test/llmisvc-model-fb-opt-125m-route-f312f5ec-kserve-mn-0-1" Jul 10 19:19:13.842452 ip-10-0-143-17 kubenswrapper[2585]: I0710 19:19:13.842441 2585 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/59ed214a-e61e-4dff-9811-87c0c02e39c5-home\") pod \"llmisvc-model-fb-opt-125m-route-f312f5ec-kserve-mn-0-1\" (UID: \"59ed214a-e61e-4dff-9811-87c0c02e39c5\") " pod="kserve-ci-e2e-test/llmisvc-model-fb-opt-125m-route-f312f5ec-kserve-mn-0-1" Jul 10 19:19:13.842641 ip-10-0-143-17 kubenswrapper[2585]: I0710 19:19:13.842580 2585 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/59ed214a-e61e-4dff-9811-87c0c02e39c5-model-cache\") pod \"llmisvc-model-fb-opt-125m-route-f312f5ec-kserve-mn-0-1\" (UID: \"59ed214a-e61e-4dff-9811-87c0c02e39c5\") " pod="kserve-ci-e2e-test/llmisvc-model-fb-opt-125m-route-f312f5ec-kserve-mn-0-1" Jul 10 19:19:13.842641 ip-10-0-143-17 kubenswrapper[2585]: I0710 19:19:13.842591 2585 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/59ed214a-e61e-4dff-9811-87c0c02e39c5-tmp-dir\") pod \"llmisvc-model-fb-opt-125m-route-f312f5ec-kserve-mn-0-1\" (UID: \"59ed214a-e61e-4dff-9811-87c0c02e39c5\") " pod="kserve-ci-e2e-test/llmisvc-model-fb-opt-125m-route-f312f5ec-kserve-mn-0-1" Jul 10 19:19:13.844068 ip-10-0-143-17 kubenswrapper[2585]: I0710 19:19:13.844048 2585 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/59ed214a-e61e-4dff-9811-87c0c02e39c5-dshm\") pod \"llmisvc-model-fb-opt-125m-route-f312f5ec-kserve-mn-0-1\" (UID: \"59ed214a-e61e-4dff-9811-87c0c02e39c5\") " pod="kserve-ci-e2e-test/llmisvc-model-fb-opt-125m-route-f312f5ec-kserve-mn-0-1" Jul 10 19:19:13.844447 ip-10-0-143-17 kubenswrapper[2585]: I0710 19:19:13.844425 2585 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/59ed214a-e61e-4dff-9811-87c0c02e39c5-tls-certs\") pod \"llmisvc-model-fb-opt-125m-route-f312f5ec-kserve-mn-0-1\" (UID: \"59ed214a-e61e-4dff-9811-87c0c02e39c5\") " pod="kserve-ci-e2e-test/llmisvc-model-fb-opt-125m-route-f312f5ec-kserve-mn-0-1" Jul 10 19:19:13.851862 ip-10-0-143-17 kubenswrapper[2585]: I0710 19:19:13.851831 2585 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-fmnsq\" (UniqueName: \"kubernetes.io/projected/59ed214a-e61e-4dff-9811-87c0c02e39c5-kube-api-access-fmnsq\") pod \"llmisvc-model-fb-opt-125m-route-f312f5ec-kserve-mn-0-1\" (UID: \"59ed214a-e61e-4dff-9811-87c0c02e39c5\") " pod="kserve-ci-e2e-test/llmisvc-model-fb-opt-125m-route-f312f5ec-kserve-mn-0-1" Jul 10 19:19:14.004098 ip-10-0-143-17 kubenswrapper[2585]: I0710 19:19:14.004058 2585 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kserve-ci-e2e-test/llmisvc-model-fb-opt-125m-route-f312f5ec-kserve-mn-0-1" Jul 10 19:19:14.143672 ip-10-0-143-17 kubenswrapper[2585]: I0710 19:19:14.143647 2585 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kserve-ci-e2e-test/llmisvc-model-fb-opt-125m-route-f312f5ec-kserve-mn-0-1"] Jul 10 19:19:14.145074 ip-10-0-143-17 kubenswrapper[2585]: W0710 19:19:14.145027 2585 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod59ed214a_e61e_4dff_9811_87c0c02e39c5.slice/crio-1e6da6469d72846f37c795881214238bf58a4f73def6d0b9c1ba7ec31b5cb4f8 WatchSource:0}: Error finding container 1e6da6469d72846f37c795881214238bf58a4f73def6d0b9c1ba7ec31b5cb4f8: Status 404 returned error can't find the container with id 1e6da6469d72846f37c795881214238bf58a4f73def6d0b9c1ba7ec31b5cb4f8 Jul 10 19:19:15.070075 ip-10-0-143-17 kubenswrapper[2585]: I0710 19:19:15.070038 2585 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kserve-ci-e2e-test/llmisvc-model-fb-opt-125m-route-f312f5ec-kserve-mn-0-1" event={"ID":"59ed214a-e61e-4dff-9811-87c0c02e39c5","Type":"ContainerStarted","Data":"675ea93d898fda343d7a257072a86699c543a9c1d84c58817f678745efff6ef3"} Jul 10 19:19:15.070497 ip-10-0-143-17 kubenswrapper[2585]: I0710 19:19:15.070094 2585 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kserve-ci-e2e-test/llmisvc-model-fb-opt-125m-route-f312f5ec-kserve-mn-0-1" event={"ID":"59ed214a-e61e-4dff-9811-87c0c02e39c5","Type":"ContainerStarted","Data":"1e6da6469d72846f37c795881214238bf58a4f73def6d0b9c1ba7ec31b5cb4f8"} Jul 10 19:19:19.092748 ip-10-0-143-17 kubenswrapper[2585]: I0710 19:19:19.092720 2585 generic.go:356] "Generic (PLEG): container finished" podID="59ed214a-e61e-4dff-9811-87c0c02e39c5" containerID="675ea93d898fda343d7a257072a86699c543a9c1d84c58817f678745efff6ef3" exitCode=0 Jul 10 19:19:19.093126 ip-10-0-143-17 kubenswrapper[2585]: I0710 19:19:19.092799 2585 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kserve-ci-e2e-test/llmisvc-model-fb-opt-125m-route-f312f5ec-kserve-mn-0-1" event={"ID":"59ed214a-e61e-4dff-9811-87c0c02e39c5","Type":"ContainerDied","Data":"675ea93d898fda343d7a257072a86699c543a9c1d84c58817f678745efff6ef3"} Jul 10 19:19:19.093969 ip-10-0-143-17 kubenswrapper[2585]: I0710 19:19:19.093951 2585 provider.go:93] Refreshing cache for provider: *credentialprovider.defaultDockerConfigProvider Jul 10 19:19:47.223899 ip-10-0-143-17 kubenswrapper[2585]: I0710 19:19:47.223875 2585 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ovn-kubernetes_ovnkube-node-xg6bm_d863cfea-f2f3-4404-b62a-d41785525f3f/ovn-acl-logging/0.log" Jul 10 19:19:47.309933 ip-10-0-143-17 kubenswrapper[2585]: I0710 19:19:47.226468 2585 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ovn-kubernetes_ovnkube-node-xg6bm_d863cfea-f2f3-4404-b62a-d41785525f3f/ovn-acl-logging/0.log" Jul 10 19:19:47.309933 ip-10-0-143-17 kubenswrapper[2585]: I0710 19:19:47.279998 2585 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kserve-ci-e2e-test/llmisvc-model-fb-opt-125m-route-f312f5ec-kserve-mn-0-1" event={"ID":"59ed214a-e61e-4dff-9811-87c0c02e39c5","Type":"ContainerStarted","Data":"626d96447ff59b7fb9513fe8b2b3bf9b43d921296efe7963c41be3800fd3254c"} Jul 10 19:19:47.309933 ip-10-0-143-17 kubenswrapper[2585]: I0710 19:19:47.302471 2585 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="kserve-ci-e2e-test/llmisvc-model-fb-opt-125m-route-f312f5ec-kserve-mn-0-1" podStartSLOduration=2.522530582 podStartE2EDuration="34.302458031s" totalImagesPullingTime="27.385223549s" totalInitContainerRuntime="4.3947039s" isStatefulPod=false podCreationTimestamp="2026-07-10 19:19:13 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-10 19:19:47.3007272 +0000 UTC m=+2400.758087135" watchObservedRunningTime="2026-07-10 19:19:47.302458031 +0000 UTC m=+2400.759818001" Jul 10 19:20:55.227641 ip-10-0-143-17 kubenswrapper[2585]: I0710 19:20:55.227599 2585 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["kserve-ci-e2e-test/custom-route-timeout-pd-test-kserve-d9768b96b-f4kzh"] Jul 10 19:20:55.232175 ip-10-0-143-17 kubenswrapper[2585]: I0710 19:20:55.232147 2585 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kserve-ci-e2e-test/custom-route-timeout-pd-test-kserve-d9768b96b-f4kzh" Jul 10 19:20:55.234979 ip-10-0-143-17 kubenswrapper[2585]: I0710 19:20:55.234949 2585 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"kserve-ci-e2e-test\"/\"custom-route-timeout-pd-test-kserve-dockercfg-njdwj\"" Jul 10 19:20:55.235226 ip-10-0-143-17 kubenswrapper[2585]: I0710 19:20:55.235041 2585 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"kserve-ci-e2e-test\"/\"custom-route-timeout-pd-test-kserve-self-signed-certs\"" Jul 10 19:20:55.244648 ip-10-0-143-17 kubenswrapper[2585]: I0710 19:20:55.244623 2585 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["kserve-ci-e2e-test/custom-route-timeout-pd-test-kserve-prefill-7d465b7985-qljdn"] Jul 10 19:20:55.248939 ip-10-0-143-17 kubenswrapper[2585]: I0710 19:20:55.248912 2585 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kserve-ci-e2e-test/custom-route-timeout-pd-test-kserve-d9768b96b-f4kzh"] Jul 10 19:20:55.249069 ip-10-0-143-17 kubenswrapper[2585]: I0710 19:20:55.249054 2585 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kserve-ci-e2e-test/custom-route-timeout-pd-test-kserve-prefill-7d465b7985-qljdn" Jul 10 19:20:55.259640 ip-10-0-143-17 kubenswrapper[2585]: I0710 19:20:55.259604 2585 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kserve-ci-e2e-test/custom-route-timeout-pd-test-kserve-prefill-7d465b7985-qljdn"] Jul 10 19:20:55.328439 ip-10-0-143-17 kubenswrapper[2585]: I0710 19:20:55.328396 2585 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/582d08e8-3203-452c-b6e5-885db8cace55-dshm\") pod \"custom-route-timeout-pd-test-kserve-d9768b96b-f4kzh\" (UID: \"582d08e8-3203-452c-b6e5-885db8cace55\") " pod="kserve-ci-e2e-test/custom-route-timeout-pd-test-kserve-d9768b96b-f4kzh" Jul 10 19:20:55.328636 ip-10-0-143-17 kubenswrapper[2585]: I0710 19:20:55.328455 2585 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/582d08e8-3203-452c-b6e5-885db8cace55-model-cache\") pod \"custom-route-timeout-pd-test-kserve-d9768b96b-f4kzh\" (UID: \"582d08e8-3203-452c-b6e5-885db8cace55\") " pod="kserve-ci-e2e-test/custom-route-timeout-pd-test-kserve-d9768b96b-f4kzh" Jul 10 19:20:55.328636 ip-10-0-143-17 kubenswrapper[2585]: I0710 19:20:55.328480 2585 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/3b61c19c-497d-48a7-90c1-7d00ae4a0dc8-home\") pod \"custom-route-timeout-pd-test-kserve-prefill-7d465b7985-qljdn\" (UID: \"3b61c19c-497d-48a7-90c1-7d00ae4a0dc8\") " pod="kserve-ci-e2e-test/custom-route-timeout-pd-test-kserve-prefill-7d465b7985-qljdn" Jul 10 19:20:55.328636 ip-10-0-143-17 kubenswrapper[2585]: I0710 19:20:55.328509 2585 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kserve-provision-location\" (UniqueName: \"kubernetes.io/empty-dir/3b61c19c-497d-48a7-90c1-7d00ae4a0dc8-kserve-provision-location\") pod \"custom-route-timeout-pd-test-kserve-prefill-7d465b7985-qljdn\" (UID: \"3b61c19c-497d-48a7-90c1-7d00ae4a0dc8\") " pod="kserve-ci-e2e-test/custom-route-timeout-pd-test-kserve-prefill-7d465b7985-qljdn" Jul 10 19:20:55.328636 ip-10-0-143-17 kubenswrapper[2585]: I0710 19:20:55.328621 2585 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-fwmvf\" (UniqueName: \"kubernetes.io/projected/3b61c19c-497d-48a7-90c1-7d00ae4a0dc8-kube-api-access-fwmvf\") pod \"custom-route-timeout-pd-test-kserve-prefill-7d465b7985-qljdn\" (UID: \"3b61c19c-497d-48a7-90c1-7d00ae4a0dc8\") " pod="kserve-ci-e2e-test/custom-route-timeout-pd-test-kserve-prefill-7d465b7985-qljdn" Jul 10 19:20:55.328807 ip-10-0-143-17 kubenswrapper[2585]: I0710 19:20:55.328651 2585 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/3b61c19c-497d-48a7-90c1-7d00ae4a0dc8-dshm\") pod \"custom-route-timeout-pd-test-kserve-prefill-7d465b7985-qljdn\" (UID: \"3b61c19c-497d-48a7-90c1-7d00ae4a0dc8\") " pod="kserve-ci-e2e-test/custom-route-timeout-pd-test-kserve-prefill-7d465b7985-qljdn" Jul 10 19:20:55.328807 ip-10-0-143-17 kubenswrapper[2585]: I0710 19:20:55.328671 2585 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/3b61c19c-497d-48a7-90c1-7d00ae4a0dc8-model-cache\") pod \"custom-route-timeout-pd-test-kserve-prefill-7d465b7985-qljdn\" (UID: \"3b61c19c-497d-48a7-90c1-7d00ae4a0dc8\") " pod="kserve-ci-e2e-test/custom-route-timeout-pd-test-kserve-prefill-7d465b7985-qljdn" Jul 10 19:20:55.328807 ip-10-0-143-17 kubenswrapper[2585]: I0710 19:20:55.328730 2585 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/582d08e8-3203-452c-b6e5-885db8cace55-tmp-dir\") pod \"custom-route-timeout-pd-test-kserve-d9768b96b-f4kzh\" (UID: \"582d08e8-3203-452c-b6e5-885db8cace55\") " pod="kserve-ci-e2e-test/custom-route-timeout-pd-test-kserve-d9768b96b-f4kzh" Jul 10 19:20:55.328807 ip-10-0-143-17 kubenswrapper[2585]: I0710 19:20:55.328771 2585 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kserve-provision-location\" (UniqueName: \"kubernetes.io/empty-dir/582d08e8-3203-452c-b6e5-885db8cace55-kserve-provision-location\") pod \"custom-route-timeout-pd-test-kserve-d9768b96b-f4kzh\" (UID: \"582d08e8-3203-452c-b6e5-885db8cace55\") " pod="kserve-ci-e2e-test/custom-route-timeout-pd-test-kserve-d9768b96b-f4kzh" Jul 10 19:20:55.328807 ip-10-0-143-17 kubenswrapper[2585]: I0710 19:20:55.328790 2585 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/582d08e8-3203-452c-b6e5-885db8cace55-tls-certs\") pod \"custom-route-timeout-pd-test-kserve-d9768b96b-f4kzh\" (UID: \"582d08e8-3203-452c-b6e5-885db8cace55\") " pod="kserve-ci-e2e-test/custom-route-timeout-pd-test-kserve-d9768b96b-f4kzh" Jul 10 19:20:55.328807 ip-10-0-143-17 kubenswrapper[2585]: I0710 19:20:55.328806 2585 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/3b61c19c-497d-48a7-90c1-7d00ae4a0dc8-tmp-dir\") pod \"custom-route-timeout-pd-test-kserve-prefill-7d465b7985-qljdn\" (UID: \"3b61c19c-497d-48a7-90c1-7d00ae4a0dc8\") " pod="kserve-ci-e2e-test/custom-route-timeout-pd-test-kserve-prefill-7d465b7985-qljdn" Jul 10 19:20:55.329130 ip-10-0-143-17 kubenswrapper[2585]: I0710 19:20:55.328907 2585 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-zg99x\" (UniqueName: \"kubernetes.io/projected/582d08e8-3203-452c-b6e5-885db8cace55-kube-api-access-zg99x\") pod \"custom-route-timeout-pd-test-kserve-d9768b96b-f4kzh\" (UID: \"582d08e8-3203-452c-b6e5-885db8cace55\") " pod="kserve-ci-e2e-test/custom-route-timeout-pd-test-kserve-d9768b96b-f4kzh" Jul 10 19:20:55.329130 ip-10-0-143-17 kubenswrapper[2585]: I0710 19:20:55.328954 2585 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/3b61c19c-497d-48a7-90c1-7d00ae4a0dc8-tls-certs\") pod \"custom-route-timeout-pd-test-kserve-prefill-7d465b7985-qljdn\" (UID: \"3b61c19c-497d-48a7-90c1-7d00ae4a0dc8\") " pod="kserve-ci-e2e-test/custom-route-timeout-pd-test-kserve-prefill-7d465b7985-qljdn" Jul 10 19:20:55.329130 ip-10-0-143-17 kubenswrapper[2585]: I0710 19:20:55.329004 2585 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/582d08e8-3203-452c-b6e5-885db8cace55-home\") pod \"custom-route-timeout-pd-test-kserve-d9768b96b-f4kzh\" (UID: \"582d08e8-3203-452c-b6e5-885db8cace55\") " pod="kserve-ci-e2e-test/custom-route-timeout-pd-test-kserve-d9768b96b-f4kzh" Jul 10 19:20:55.430172 ip-10-0-143-17 kubenswrapper[2585]: I0710 19:20:55.430133 2585 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kserve-provision-location\" (UniqueName: \"kubernetes.io/empty-dir/3b61c19c-497d-48a7-90c1-7d00ae4a0dc8-kserve-provision-location\") pod \"custom-route-timeout-pd-test-kserve-prefill-7d465b7985-qljdn\" (UID: \"3b61c19c-497d-48a7-90c1-7d00ae4a0dc8\") " pod="kserve-ci-e2e-test/custom-route-timeout-pd-test-kserve-prefill-7d465b7985-qljdn" Jul 10 19:20:55.430340 ip-10-0-143-17 kubenswrapper[2585]: I0710 19:20:55.430180 2585 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-fwmvf\" (UniqueName: \"kubernetes.io/projected/3b61c19c-497d-48a7-90c1-7d00ae4a0dc8-kube-api-access-fwmvf\") pod \"custom-route-timeout-pd-test-kserve-prefill-7d465b7985-qljdn\" (UID: \"3b61c19c-497d-48a7-90c1-7d00ae4a0dc8\") " pod="kserve-ci-e2e-test/custom-route-timeout-pd-test-kserve-prefill-7d465b7985-qljdn" Jul 10 19:20:55.430340 ip-10-0-143-17 kubenswrapper[2585]: I0710 19:20:55.430220 2585 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/3b61c19c-497d-48a7-90c1-7d00ae4a0dc8-dshm\") pod \"custom-route-timeout-pd-test-kserve-prefill-7d465b7985-qljdn\" (UID: \"3b61c19c-497d-48a7-90c1-7d00ae4a0dc8\") " pod="kserve-ci-e2e-test/custom-route-timeout-pd-test-kserve-prefill-7d465b7985-qljdn" Jul 10 19:20:55.430340 ip-10-0-143-17 kubenswrapper[2585]: I0710 19:20:55.430244 2585 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/3b61c19c-497d-48a7-90c1-7d00ae4a0dc8-model-cache\") pod \"custom-route-timeout-pd-test-kserve-prefill-7d465b7985-qljdn\" (UID: \"3b61c19c-497d-48a7-90c1-7d00ae4a0dc8\") " pod="kserve-ci-e2e-test/custom-route-timeout-pd-test-kserve-prefill-7d465b7985-qljdn" Jul 10 19:20:55.430340 ip-10-0-143-17 kubenswrapper[2585]: I0710 19:20:55.430276 2585 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/582d08e8-3203-452c-b6e5-885db8cace55-tmp-dir\") pod \"custom-route-timeout-pd-test-kserve-d9768b96b-f4kzh\" (UID: \"582d08e8-3203-452c-b6e5-885db8cace55\") " pod="kserve-ci-e2e-test/custom-route-timeout-pd-test-kserve-d9768b96b-f4kzh" Jul 10 19:20:55.430340 ip-10-0-143-17 kubenswrapper[2585]: I0710 19:20:55.430306 2585 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kserve-provision-location\" (UniqueName: \"kubernetes.io/empty-dir/582d08e8-3203-452c-b6e5-885db8cace55-kserve-provision-location\") pod \"custom-route-timeout-pd-test-kserve-d9768b96b-f4kzh\" (UID: \"582d08e8-3203-452c-b6e5-885db8cace55\") " pod="kserve-ci-e2e-test/custom-route-timeout-pd-test-kserve-d9768b96b-f4kzh" Jul 10 19:20:55.430340 ip-10-0-143-17 kubenswrapper[2585]: I0710 19:20:55.430332 2585 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/582d08e8-3203-452c-b6e5-885db8cace55-tls-certs\") pod \"custom-route-timeout-pd-test-kserve-d9768b96b-f4kzh\" (UID: \"582d08e8-3203-452c-b6e5-885db8cace55\") " pod="kserve-ci-e2e-test/custom-route-timeout-pd-test-kserve-d9768b96b-f4kzh" Jul 10 19:20:55.430694 ip-10-0-143-17 kubenswrapper[2585]: I0710 19:20:55.430643 2585 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kserve-provision-location\" (UniqueName: \"kubernetes.io/empty-dir/3b61c19c-497d-48a7-90c1-7d00ae4a0dc8-kserve-provision-location\") pod \"custom-route-timeout-pd-test-kserve-prefill-7d465b7985-qljdn\" (UID: \"3b61c19c-497d-48a7-90c1-7d00ae4a0dc8\") " pod="kserve-ci-e2e-test/custom-route-timeout-pd-test-kserve-prefill-7d465b7985-qljdn" Jul 10 19:20:55.430694 ip-10-0-143-17 kubenswrapper[2585]: I0710 19:20:55.430675 2585 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/582d08e8-3203-452c-b6e5-885db8cace55-tmp-dir\") pod \"custom-route-timeout-pd-test-kserve-d9768b96b-f4kzh\" (UID: \"582d08e8-3203-452c-b6e5-885db8cace55\") " pod="kserve-ci-e2e-test/custom-route-timeout-pd-test-kserve-d9768b96b-f4kzh" Jul 10 19:20:55.430797 ip-10-0-143-17 kubenswrapper[2585]: I0710 19:20:55.430735 2585 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kserve-provision-location\" (UniqueName: \"kubernetes.io/empty-dir/582d08e8-3203-452c-b6e5-885db8cace55-kserve-provision-location\") pod \"custom-route-timeout-pd-test-kserve-d9768b96b-f4kzh\" (UID: \"582d08e8-3203-452c-b6e5-885db8cace55\") " pod="kserve-ci-e2e-test/custom-route-timeout-pd-test-kserve-d9768b96b-f4kzh" Jul 10 19:20:55.430797 ip-10-0-143-17 kubenswrapper[2585]: I0710 19:20:55.430743 2585 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/3b61c19c-497d-48a7-90c1-7d00ae4a0dc8-model-cache\") pod \"custom-route-timeout-pd-test-kserve-prefill-7d465b7985-qljdn\" (UID: \"3b61c19c-497d-48a7-90c1-7d00ae4a0dc8\") " pod="kserve-ci-e2e-test/custom-route-timeout-pd-test-kserve-prefill-7d465b7985-qljdn" Jul 10 19:20:55.430797 ip-10-0-143-17 kubenswrapper[2585]: I0710 19:20:55.430764 2585 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/3b61c19c-497d-48a7-90c1-7d00ae4a0dc8-tmp-dir\") pod \"custom-route-timeout-pd-test-kserve-prefill-7d465b7985-qljdn\" (UID: \"3b61c19c-497d-48a7-90c1-7d00ae4a0dc8\") " pod="kserve-ci-e2e-test/custom-route-timeout-pd-test-kserve-prefill-7d465b7985-qljdn" Jul 10 19:20:55.430922 ip-10-0-143-17 kubenswrapper[2585]: I0710 19:20:55.430842 2585 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-zg99x\" (UniqueName: \"kubernetes.io/projected/582d08e8-3203-452c-b6e5-885db8cace55-kube-api-access-zg99x\") pod \"custom-route-timeout-pd-test-kserve-d9768b96b-f4kzh\" (UID: \"582d08e8-3203-452c-b6e5-885db8cace55\") " pod="kserve-ci-e2e-test/custom-route-timeout-pd-test-kserve-d9768b96b-f4kzh" Jul 10 19:20:55.430922 ip-10-0-143-17 kubenswrapper[2585]: I0710 19:20:55.430872 2585 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/3b61c19c-497d-48a7-90c1-7d00ae4a0dc8-tls-certs\") pod \"custom-route-timeout-pd-test-kserve-prefill-7d465b7985-qljdn\" (UID: \"3b61c19c-497d-48a7-90c1-7d00ae4a0dc8\") " pod="kserve-ci-e2e-test/custom-route-timeout-pd-test-kserve-prefill-7d465b7985-qljdn" Jul 10 19:20:55.430922 ip-10-0-143-17 kubenswrapper[2585]: I0710 19:20:55.430904 2585 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/582d08e8-3203-452c-b6e5-885db8cace55-home\") pod \"custom-route-timeout-pd-test-kserve-d9768b96b-f4kzh\" (UID: \"582d08e8-3203-452c-b6e5-885db8cace55\") " pod="kserve-ci-e2e-test/custom-route-timeout-pd-test-kserve-d9768b96b-f4kzh" Jul 10 19:20:55.431078 ip-10-0-143-17 kubenswrapper[2585]: I0710 19:20:55.430972 2585 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/582d08e8-3203-452c-b6e5-885db8cace55-dshm\") pod \"custom-route-timeout-pd-test-kserve-d9768b96b-f4kzh\" (UID: \"582d08e8-3203-452c-b6e5-885db8cace55\") " pod="kserve-ci-e2e-test/custom-route-timeout-pd-test-kserve-d9768b96b-f4kzh" Jul 10 19:20:55.431078 ip-10-0-143-17 kubenswrapper[2585]: I0710 19:20:55.430999 2585 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/3b61c19c-497d-48a7-90c1-7d00ae4a0dc8-tmp-dir\") pod \"custom-route-timeout-pd-test-kserve-prefill-7d465b7985-qljdn\" (UID: \"3b61c19c-497d-48a7-90c1-7d00ae4a0dc8\") " pod="kserve-ci-e2e-test/custom-route-timeout-pd-test-kserve-prefill-7d465b7985-qljdn" Jul 10 19:20:55.431078 ip-10-0-143-17 kubenswrapper[2585]: I0710 19:20:55.431003 2585 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/582d08e8-3203-452c-b6e5-885db8cace55-model-cache\") pod \"custom-route-timeout-pd-test-kserve-d9768b96b-f4kzh\" (UID: \"582d08e8-3203-452c-b6e5-885db8cace55\") " pod="kserve-ci-e2e-test/custom-route-timeout-pd-test-kserve-d9768b96b-f4kzh" Jul 10 19:20:55.431078 ip-10-0-143-17 kubenswrapper[2585]: I0710 19:20:55.431048 2585 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/3b61c19c-497d-48a7-90c1-7d00ae4a0dc8-home\") pod \"custom-route-timeout-pd-test-kserve-prefill-7d465b7985-qljdn\" (UID: \"3b61c19c-497d-48a7-90c1-7d00ae4a0dc8\") " pod="kserve-ci-e2e-test/custom-route-timeout-pd-test-kserve-prefill-7d465b7985-qljdn" Jul 10 19:20:55.431294 ip-10-0-143-17 kubenswrapper[2585]: I0710 19:20:55.431232 2585 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/582d08e8-3203-452c-b6e5-885db8cace55-home\") pod \"custom-route-timeout-pd-test-kserve-d9768b96b-f4kzh\" (UID: \"582d08e8-3203-452c-b6e5-885db8cace55\") " pod="kserve-ci-e2e-test/custom-route-timeout-pd-test-kserve-d9768b96b-f4kzh" Jul 10 19:20:55.431421 ip-10-0-143-17 kubenswrapper[2585]: I0710 19:20:55.431393 2585 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/3b61c19c-497d-48a7-90c1-7d00ae4a0dc8-home\") pod \"custom-route-timeout-pd-test-kserve-prefill-7d465b7985-qljdn\" (UID: \"3b61c19c-497d-48a7-90c1-7d00ae4a0dc8\") " pod="kserve-ci-e2e-test/custom-route-timeout-pd-test-kserve-prefill-7d465b7985-qljdn" Jul 10 19:20:55.431498 ip-10-0-143-17 kubenswrapper[2585]: I0710 19:20:55.431427 2585 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/582d08e8-3203-452c-b6e5-885db8cace55-model-cache\") pod \"custom-route-timeout-pd-test-kserve-d9768b96b-f4kzh\" (UID: \"582d08e8-3203-452c-b6e5-885db8cace55\") " pod="kserve-ci-e2e-test/custom-route-timeout-pd-test-kserve-d9768b96b-f4kzh" Jul 10 19:20:55.432804 ip-10-0-143-17 kubenswrapper[2585]: I0710 19:20:55.432780 2585 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/3b61c19c-497d-48a7-90c1-7d00ae4a0dc8-dshm\") pod \"custom-route-timeout-pd-test-kserve-prefill-7d465b7985-qljdn\" (UID: \"3b61c19c-497d-48a7-90c1-7d00ae4a0dc8\") " pod="kserve-ci-e2e-test/custom-route-timeout-pd-test-kserve-prefill-7d465b7985-qljdn" Jul 10 19:20:55.433161 ip-10-0-143-17 kubenswrapper[2585]: I0710 19:20:55.433140 2585 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/582d08e8-3203-452c-b6e5-885db8cace55-dshm\") pod \"custom-route-timeout-pd-test-kserve-d9768b96b-f4kzh\" (UID: \"582d08e8-3203-452c-b6e5-885db8cace55\") " pod="kserve-ci-e2e-test/custom-route-timeout-pd-test-kserve-d9768b96b-f4kzh" Jul 10 19:20:55.433330 ip-10-0-143-17 kubenswrapper[2585]: I0710 19:20:55.433308 2585 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/3b61c19c-497d-48a7-90c1-7d00ae4a0dc8-tls-certs\") pod \"custom-route-timeout-pd-test-kserve-prefill-7d465b7985-qljdn\" (UID: \"3b61c19c-497d-48a7-90c1-7d00ae4a0dc8\") " pod="kserve-ci-e2e-test/custom-route-timeout-pd-test-kserve-prefill-7d465b7985-qljdn" Jul 10 19:20:55.433697 ip-10-0-143-17 kubenswrapper[2585]: I0710 19:20:55.433669 2585 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/582d08e8-3203-452c-b6e5-885db8cace55-tls-certs\") pod \"custom-route-timeout-pd-test-kserve-d9768b96b-f4kzh\" (UID: \"582d08e8-3203-452c-b6e5-885db8cace55\") " pod="kserve-ci-e2e-test/custom-route-timeout-pd-test-kserve-d9768b96b-f4kzh" Jul 10 19:20:55.443156 ip-10-0-143-17 kubenswrapper[2585]: I0710 19:20:55.443129 2585 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-zg99x\" (UniqueName: \"kubernetes.io/projected/582d08e8-3203-452c-b6e5-885db8cace55-kube-api-access-zg99x\") pod \"custom-route-timeout-pd-test-kserve-d9768b96b-f4kzh\" (UID: \"582d08e8-3203-452c-b6e5-885db8cace55\") " pod="kserve-ci-e2e-test/custom-route-timeout-pd-test-kserve-d9768b96b-f4kzh" Jul 10 19:20:55.445366 ip-10-0-143-17 kubenswrapper[2585]: I0710 19:20:55.445335 2585 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-fwmvf\" (UniqueName: \"kubernetes.io/projected/3b61c19c-497d-48a7-90c1-7d00ae4a0dc8-kube-api-access-fwmvf\") pod \"custom-route-timeout-pd-test-kserve-prefill-7d465b7985-qljdn\" (UID: \"3b61c19c-497d-48a7-90c1-7d00ae4a0dc8\") " pod="kserve-ci-e2e-test/custom-route-timeout-pd-test-kserve-prefill-7d465b7985-qljdn" Jul 10 19:20:55.545311 ip-10-0-143-17 kubenswrapper[2585]: I0710 19:20:55.545229 2585 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kserve-ci-e2e-test/custom-route-timeout-pd-test-kserve-d9768b96b-f4kzh" Jul 10 19:20:55.563841 ip-10-0-143-17 kubenswrapper[2585]: I0710 19:20:55.563814 2585 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kserve-ci-e2e-test/custom-route-timeout-pd-test-kserve-prefill-7d465b7985-qljdn" Jul 10 19:20:55.723199 ip-10-0-143-17 kubenswrapper[2585]: I0710 19:20:55.723168 2585 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kserve-ci-e2e-test/custom-route-timeout-pd-test-kserve-d9768b96b-f4kzh"] Jul 10 19:20:55.724382 ip-10-0-143-17 kubenswrapper[2585]: W0710 19:20:55.724353 2585 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod582d08e8_3203_452c_b6e5_885db8cace55.slice/crio-103ad6bd9b1864fb6d096475302907b537e2383d0652866c701f9b52462054dc WatchSource:0}: Error finding container 103ad6bd9b1864fb6d096475302907b537e2383d0652866c701f9b52462054dc: Status 404 returned error can't find the container with id 103ad6bd9b1864fb6d096475302907b537e2383d0652866c701f9b52462054dc Jul 10 19:20:55.742623 ip-10-0-143-17 kubenswrapper[2585]: I0710 19:20:55.742597 2585 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kserve-ci-e2e-test/custom-route-timeout-pd-test-kserve-prefill-7d465b7985-qljdn"] Jul 10 19:20:55.743180 ip-10-0-143-17 kubenswrapper[2585]: W0710 19:20:55.743137 2585 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod3b61c19c_497d_48a7_90c1_7d00ae4a0dc8.slice/crio-5eacd06e989ceb66497b4e48014589a79d126e2e9906e3b132c61c587678ac4c WatchSource:0}: Error finding container 5eacd06e989ceb66497b4e48014589a79d126e2e9906e3b132c61c587678ac4c: Status 404 returned error can't find the container with id 5eacd06e989ceb66497b4e48014589a79d126e2e9906e3b132c61c587678ac4c Jul 10 19:20:56.677381 ip-10-0-143-17 kubenswrapper[2585]: I0710 19:20:56.677299 2585 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kserve-ci-e2e-test/custom-route-timeout-pd-test-kserve-prefill-7d465b7985-qljdn" event={"ID":"3b61c19c-497d-48a7-90c1-7d00ae4a0dc8","Type":"ContainerStarted","Data":"11b8ee6adc85061be0807e411350c314de5428f6808641a01c46043c0ae11b19"} Jul 10 19:20:56.677955 ip-10-0-143-17 kubenswrapper[2585]: I0710 19:20:56.677412 2585 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kserve-ci-e2e-test/custom-route-timeout-pd-test-kserve-prefill-7d465b7985-qljdn" event={"ID":"3b61c19c-497d-48a7-90c1-7d00ae4a0dc8","Type":"ContainerStarted","Data":"5eacd06e989ceb66497b4e48014589a79d126e2e9906e3b132c61c587678ac4c"} Jul 10 19:20:56.679170 ip-10-0-143-17 kubenswrapper[2585]: I0710 19:20:56.679140 2585 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kserve-ci-e2e-test/custom-route-timeout-pd-test-kserve-d9768b96b-f4kzh" event={"ID":"582d08e8-3203-452c-b6e5-885db8cace55","Type":"ContainerStarted","Data":"e7960b1cffff7526408a5b7cfc5979d1c43339038fb0e841ac015ca61f37c619"} Jul 10 19:20:56.679270 ip-10-0-143-17 kubenswrapper[2585]: I0710 19:20:56.679187 2585 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kserve-ci-e2e-test/custom-route-timeout-pd-test-kserve-d9768b96b-f4kzh" event={"ID":"582d08e8-3203-452c-b6e5-885db8cace55","Type":"ContainerStarted","Data":"103ad6bd9b1864fb6d096475302907b537e2383d0652866c701f9b52462054dc"} Jul 10 19:20:56.679270 ip-10-0-143-17 kubenswrapper[2585]: I0710 19:20:56.679236 2585 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="kserve-ci-e2e-test/custom-route-timeout-pd-test-kserve-d9768b96b-f4kzh" Jul 10 19:20:57.690968 ip-10-0-143-17 kubenswrapper[2585]: I0710 19:20:57.690925 2585 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kserve-ci-e2e-test/custom-route-timeout-pd-test-kserve-d9768b96b-f4kzh" event={"ID":"582d08e8-3203-452c-b6e5-885db8cace55","Type":"ContainerStarted","Data":"e44c1bc9173dd845c87652369d82105b0ebe040d67b9c3d6f5cb4f64905480e2"} Jul 10 19:21:00.714667 ip-10-0-143-17 kubenswrapper[2585]: I0710 19:21:00.714575 2585 generic.go:356] "Generic (PLEG): container finished" podID="3b61c19c-497d-48a7-90c1-7d00ae4a0dc8" containerID="11b8ee6adc85061be0807e411350c314de5428f6808641a01c46043c0ae11b19" exitCode=0 Jul 10 19:21:00.714667 ip-10-0-143-17 kubenswrapper[2585]: I0710 19:21:00.714638 2585 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kserve-ci-e2e-test/custom-route-timeout-pd-test-kserve-prefill-7d465b7985-qljdn" event={"ID":"3b61c19c-497d-48a7-90c1-7d00ae4a0dc8","Type":"ContainerDied","Data":"11b8ee6adc85061be0807e411350c314de5428f6808641a01c46043c0ae11b19"} Jul 10 19:21:01.722765 ip-10-0-143-17 kubenswrapper[2585]: I0710 19:21:01.722725 2585 generic.go:356] "Generic (PLEG): container finished" podID="582d08e8-3203-452c-b6e5-885db8cace55" containerID="e44c1bc9173dd845c87652369d82105b0ebe040d67b9c3d6f5cb4f64905480e2" exitCode=0 Jul 10 19:21:01.723266 ip-10-0-143-17 kubenswrapper[2585]: I0710 19:21:01.722792 2585 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kserve-ci-e2e-test/custom-route-timeout-pd-test-kserve-d9768b96b-f4kzh" event={"ID":"582d08e8-3203-452c-b6e5-885db8cace55","Type":"ContainerDied","Data":"e44c1bc9173dd845c87652369d82105b0ebe040d67b9c3d6f5cb4f64905480e2"} Jul 10 19:21:01.725108 ip-10-0-143-17 kubenswrapper[2585]: I0710 19:21:01.725085 2585 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kserve-ci-e2e-test/custom-route-timeout-pd-test-kserve-prefill-7d465b7985-qljdn" event={"ID":"3b61c19c-497d-48a7-90c1-7d00ae4a0dc8","Type":"ContainerStarted","Data":"507a501cffc9d609795dac967b4bbca1b8131ab63ecb579ab00a3df6048be181"} Jul 10 19:21:01.769840 ip-10-0-143-17 kubenswrapper[2585]: I0710 19:21:01.769785 2585 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="kserve-ci-e2e-test/custom-route-timeout-pd-test-kserve-prefill-7d465b7985-qljdn" podStartSLOduration=2.314136415 podStartE2EDuration="6.769768817s" totalImagesPullingTime="0s" totalInitContainerRuntime="4.455632402s" isStatefulPod=false podCreationTimestamp="2026-07-10 19:20:55 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-10 19:21:01.766873399 +0000 UTC m=+2475.224233361" watchObservedRunningTime="2026-07-10 19:21:01.769768817 +0000 UTC m=+2475.227128751" Jul 10 19:21:02.736028 ip-10-0-143-17 kubenswrapper[2585]: I0710 19:21:02.735980 2585 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kserve-ci-e2e-test/custom-route-timeout-pd-test-kserve-d9768b96b-f4kzh" event={"ID":"582d08e8-3203-452c-b6e5-885db8cace55","Type":"ContainerStarted","Data":"8c825b781828d8a226b5d9720d6a186b412e0949e4c0fd868c8fe042fb981456"} Jul 10 19:21:02.764294 ip-10-0-143-17 kubenswrapper[2585]: I0710 19:21:02.764227 2585 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="kserve-ci-e2e-test/custom-route-timeout-pd-test-kserve-d9768b96b-f4kzh" podStartSLOduration=2.50844128 podStartE2EDuration="7.764206438s" totalImagesPullingTime="866.9659ms" totalInitContainerRuntime="4.388799258s" isStatefulPod=false podCreationTimestamp="2026-07-10 19:20:55 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-10 19:21:02.761207694 +0000 UTC m=+2476.218567677" watchObservedRunningTime="2026-07-10 19:21:02.764206438 +0000 UTC m=+2476.221566374" Jul 10 19:21:05.546422 ip-10-0-143-17 kubenswrapper[2585]: I0710 19:21:05.546381 2585 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="kserve-ci-e2e-test/custom-route-timeout-pd-test-kserve-d9768b96b-f4kzh" Jul 10 19:21:05.546906 ip-10-0-143-17 kubenswrapper[2585]: I0710 19:21:05.546471 2585 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="kserve-ci-e2e-test/custom-route-timeout-pd-test-kserve-d9768b96b-f4kzh" Jul 10 19:21:05.548420 ip-10-0-143-17 kubenswrapper[2585]: I0710 19:21:05.548386 2585 prober.go:120] "Probe failed" probeType="Startup" pod="kserve-ci-e2e-test/custom-route-timeout-pd-test-kserve-d9768b96b-f4kzh" podUID="582d08e8-3203-452c-b6e5-885db8cace55" containerName="main" probeResult="failure" output="Get \"https://10.133.0.51:8001/health\": dial tcp 10.133.0.51:8001: connect: connection refused" Jul 10 19:21:05.564022 ip-10-0-143-17 kubenswrapper[2585]: I0710 19:21:05.563986 2585 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="kserve-ci-e2e-test/custom-route-timeout-pd-test-kserve-prefill-7d465b7985-qljdn" Jul 10 19:21:05.564179 ip-10-0-143-17 kubenswrapper[2585]: I0710 19:21:05.564061 2585 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="kserve-ci-e2e-test/custom-route-timeout-pd-test-kserve-prefill-7d465b7985-qljdn" Jul 10 19:21:05.565659 ip-10-0-143-17 kubenswrapper[2585]: I0710 19:21:05.565628 2585 prober.go:120] "Probe failed" probeType="Startup" pod="kserve-ci-e2e-test/custom-route-timeout-pd-test-kserve-prefill-7d465b7985-qljdn" podUID="3b61c19c-497d-48a7-90c1-7d00ae4a0dc8" containerName="main" probeResult="failure" output="Get \"https://10.133.0.52:8000/health\": dial tcp 10.133.0.52:8000: connect: connection refused" Jul 10 19:21:15.546424 ip-10-0-143-17 kubenswrapper[2585]: I0710 19:21:15.546369 2585 prober.go:120] "Probe failed" probeType="Startup" pod="kserve-ci-e2e-test/custom-route-timeout-pd-test-kserve-d9768b96b-f4kzh" podUID="582d08e8-3203-452c-b6e5-885db8cace55" containerName="main" probeResult="failure" output="Get \"https://10.133.0.51:8001/health\": dial tcp 10.133.0.51:8001: connect: connection refused" Jul 10 19:21:15.564560 ip-10-0-143-17 kubenswrapper[2585]: I0710 19:21:15.564501 2585 prober.go:120] "Probe failed" probeType="Startup" pod="kserve-ci-e2e-test/custom-route-timeout-pd-test-kserve-prefill-7d465b7985-qljdn" podUID="3b61c19c-497d-48a7-90c1-7d00ae4a0dc8" containerName="main" probeResult="failure" output="Get \"https://10.133.0.52:8000/health\": dial tcp 10.133.0.52:8000: connect: connection refused" Jul 10 19:21:15.568878 ip-10-0-143-17 kubenswrapper[2585]: I0710 19:21:15.568851 2585 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="kserve-ci-e2e-test/custom-route-timeout-pd-test-kserve-d9768b96b-f4kzh" Jul 10 19:21:23.191029 ip-10-0-143-17 kubenswrapper[2585]: I0710 19:21:23.190982 2585 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["kserve-ci-e2e-test/llmisvc-model-fb-opt-125m-route-f312f5ec-kserve-mn-0-1"] Jul 10 19:21:23.191420 ip-10-0-143-17 kubenswrapper[2585]: I0710 19:21:23.191264 2585 kuberuntime_container.go:873] "Killing container with a grace period" pod="kserve-ci-e2e-test/llmisvc-model-fb-opt-125m-route-f312f5ec-kserve-mn-0-1" podUID="59ed214a-e61e-4dff-9811-87c0c02e39c5" containerName="main" containerID="cri-o://626d96447ff59b7fb9513fe8b2b3bf9b43d921296efe7963c41be3800fd3254c" gracePeriod=30 Jul 10 19:21:23.900866 ip-10-0-143-17 kubenswrapper[2585]: I0710 19:21:23.900831 2585 generic.go:356] "Generic (PLEG): container finished" podID="59ed214a-e61e-4dff-9811-87c0c02e39c5" containerID="626d96447ff59b7fb9513fe8b2b3bf9b43d921296efe7963c41be3800fd3254c" exitCode=0 Jul 10 19:21:23.901051 ip-10-0-143-17 kubenswrapper[2585]: I0710 19:21:23.900906 2585 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kserve-ci-e2e-test/llmisvc-model-fb-opt-125m-route-f312f5ec-kserve-mn-0-1" event={"ID":"59ed214a-e61e-4dff-9811-87c0c02e39c5","Type":"ContainerDied","Data":"626d96447ff59b7fb9513fe8b2b3bf9b43d921296efe7963c41be3800fd3254c"} Jul 10 19:21:23.950418 ip-10-0-143-17 kubenswrapper[2585]: I0710 19:21:23.950396 2585 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="kserve-ci-e2e-test/llmisvc-model-fb-opt-125m-route-f312f5ec-kserve-mn-0-1" Jul 10 19:21:24.057917 ip-10-0-143-17 kubenswrapper[2585]: I0710 19:21:24.057828 2585 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-fmnsq\" (UniqueName: \"kubernetes.io/projected/59ed214a-e61e-4dff-9811-87c0c02e39c5-kube-api-access-fmnsq\") pod \"59ed214a-e61e-4dff-9811-87c0c02e39c5\" (UID: \"59ed214a-e61e-4dff-9811-87c0c02e39c5\") " Jul 10 19:21:24.058087 ip-10-0-143-17 kubenswrapper[2585]: I0710 19:21:24.057919 2585 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/59ed214a-e61e-4dff-9811-87c0c02e39c5-tls-certs\") pod \"59ed214a-e61e-4dff-9811-87c0c02e39c5\" (UID: \"59ed214a-e61e-4dff-9811-87c0c02e39c5\") " Jul 10 19:21:24.058087 ip-10-0-143-17 kubenswrapper[2585]: I0710 19:21:24.057973 2585 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/59ed214a-e61e-4dff-9811-87c0c02e39c5-tmp-dir\") pod \"59ed214a-e61e-4dff-9811-87c0c02e39c5\" (UID: \"59ed214a-e61e-4dff-9811-87c0c02e39c5\") " Jul 10 19:21:24.058087 ip-10-0-143-17 kubenswrapper[2585]: I0710 19:21:24.058009 2585 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/59ed214a-e61e-4dff-9811-87c0c02e39c5-dshm\") pod \"59ed214a-e61e-4dff-9811-87c0c02e39c5\" (UID: \"59ed214a-e61e-4dff-9811-87c0c02e39c5\") " Jul 10 19:21:24.058087 ip-10-0-143-17 kubenswrapper[2585]: I0710 19:21:24.058050 2585 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/59ed214a-e61e-4dff-9811-87c0c02e39c5-model-cache\") pod \"59ed214a-e61e-4dff-9811-87c0c02e39c5\" (UID: \"59ed214a-e61e-4dff-9811-87c0c02e39c5\") " Jul 10 19:21:24.058087 ip-10-0-143-17 kubenswrapper[2585]: I0710 19:21:24.058085 2585 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/59ed214a-e61e-4dff-9811-87c0c02e39c5-home\") pod \"59ed214a-e61e-4dff-9811-87c0c02e39c5\" (UID: \"59ed214a-e61e-4dff-9811-87c0c02e39c5\") " Jul 10 19:21:24.058343 ip-10-0-143-17 kubenswrapper[2585]: I0710 19:21:24.058119 2585 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kserve-provision-location\" (UniqueName: \"kubernetes.io/empty-dir/59ed214a-e61e-4dff-9811-87c0c02e39c5-kserve-provision-location\") pod \"59ed214a-e61e-4dff-9811-87c0c02e39c5\" (UID: \"59ed214a-e61e-4dff-9811-87c0c02e39c5\") " Jul 10 19:21:24.058612 ip-10-0-143-17 kubenswrapper[2585]: I0710 19:21:24.058485 2585 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/59ed214a-e61e-4dff-9811-87c0c02e39c5-model-cache" (OuterVolumeSpecName: "model-cache") pod "59ed214a-e61e-4dff-9811-87c0c02e39c5" (UID: "59ed214a-e61e-4dff-9811-87c0c02e39c5"). InnerVolumeSpecName "model-cache". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Jul 10 19:21:24.059199 ip-10-0-143-17 kubenswrapper[2585]: I0710 19:21:24.059160 2585 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/59ed214a-e61e-4dff-9811-87c0c02e39c5-home" (OuterVolumeSpecName: "home") pod "59ed214a-e61e-4dff-9811-87c0c02e39c5" (UID: "59ed214a-e61e-4dff-9811-87c0c02e39c5"). InnerVolumeSpecName "home". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Jul 10 19:21:24.060576 ip-10-0-143-17 kubenswrapper[2585]: I0710 19:21:24.060529 2585 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/59ed214a-e61e-4dff-9811-87c0c02e39c5-tls-certs" (OuterVolumeSpecName: "tls-certs") pod "59ed214a-e61e-4dff-9811-87c0c02e39c5" (UID: "59ed214a-e61e-4dff-9811-87c0c02e39c5"). InnerVolumeSpecName "tls-certs". PluginName "kubernetes.io/secret", VolumeGIDValue "" Jul 10 19:21:24.060693 ip-10-0-143-17 kubenswrapper[2585]: I0710 19:21:24.060606 2585 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/59ed214a-e61e-4dff-9811-87c0c02e39c5-kube-api-access-fmnsq" (OuterVolumeSpecName: "kube-api-access-fmnsq") pod "59ed214a-e61e-4dff-9811-87c0c02e39c5" (UID: "59ed214a-e61e-4dff-9811-87c0c02e39c5"). InnerVolumeSpecName "kube-api-access-fmnsq". PluginName "kubernetes.io/projected", VolumeGIDValue "" Jul 10 19:21:24.060693 ip-10-0-143-17 kubenswrapper[2585]: I0710 19:21:24.060621 2585 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/59ed214a-e61e-4dff-9811-87c0c02e39c5-dshm" (OuterVolumeSpecName: "dshm") pod "59ed214a-e61e-4dff-9811-87c0c02e39c5" (UID: "59ed214a-e61e-4dff-9811-87c0c02e39c5"). InnerVolumeSpecName "dshm". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Jul 10 19:21:24.076923 ip-10-0-143-17 kubenswrapper[2585]: I0710 19:21:24.076890 2585 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/59ed214a-e61e-4dff-9811-87c0c02e39c5-tmp-dir" (OuterVolumeSpecName: "tmp-dir") pod "59ed214a-e61e-4dff-9811-87c0c02e39c5" (UID: "59ed214a-e61e-4dff-9811-87c0c02e39c5"). InnerVolumeSpecName "tmp-dir". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Jul 10 19:21:24.123321 ip-10-0-143-17 kubenswrapper[2585]: I0710 19:21:24.123279 2585 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/59ed214a-e61e-4dff-9811-87c0c02e39c5-kserve-provision-location" (OuterVolumeSpecName: "kserve-provision-location") pod "59ed214a-e61e-4dff-9811-87c0c02e39c5" (UID: "59ed214a-e61e-4dff-9811-87c0c02e39c5"). InnerVolumeSpecName "kserve-provision-location". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Jul 10 19:21:24.159408 ip-10-0-143-17 kubenswrapper[2585]: I0710 19:21:24.159375 2585 reconciler_common.go:299] "Volume detached for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/59ed214a-e61e-4dff-9811-87c0c02e39c5-home\") on node \"ip-10-0-143-17.ec2.internal\" DevicePath \"\"" Jul 10 19:21:24.159408 ip-10-0-143-17 kubenswrapper[2585]: I0710 19:21:24.159404 2585 reconciler_common.go:299] "Volume detached for volume \"kserve-provision-location\" (UniqueName: \"kubernetes.io/empty-dir/59ed214a-e61e-4dff-9811-87c0c02e39c5-kserve-provision-location\") on node \"ip-10-0-143-17.ec2.internal\" DevicePath \"\"" Jul 10 19:21:24.159408 ip-10-0-143-17 kubenswrapper[2585]: I0710 19:21:24.159413 2585 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-fmnsq\" (UniqueName: \"kubernetes.io/projected/59ed214a-e61e-4dff-9811-87c0c02e39c5-kube-api-access-fmnsq\") on node \"ip-10-0-143-17.ec2.internal\" DevicePath \"\"" Jul 10 19:21:24.159656 ip-10-0-143-17 kubenswrapper[2585]: I0710 19:21:24.159424 2585 reconciler_common.go:299] "Volume detached for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/59ed214a-e61e-4dff-9811-87c0c02e39c5-tls-certs\") on node \"ip-10-0-143-17.ec2.internal\" DevicePath \"\"" Jul 10 19:21:24.159656 ip-10-0-143-17 kubenswrapper[2585]: I0710 19:21:24.159432 2585 reconciler_common.go:299] "Volume detached for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/59ed214a-e61e-4dff-9811-87c0c02e39c5-tmp-dir\") on node \"ip-10-0-143-17.ec2.internal\" DevicePath \"\"" Jul 10 19:21:24.159656 ip-10-0-143-17 kubenswrapper[2585]: I0710 19:21:24.159439 2585 reconciler_common.go:299] "Volume detached for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/59ed214a-e61e-4dff-9811-87c0c02e39c5-dshm\") on node \"ip-10-0-143-17.ec2.internal\" DevicePath \"\"" Jul 10 19:21:24.159656 ip-10-0-143-17 kubenswrapper[2585]: I0710 19:21:24.159446 2585 reconciler_common.go:299] "Volume detached for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/59ed214a-e61e-4dff-9811-87c0c02e39c5-model-cache\") on node \"ip-10-0-143-17.ec2.internal\" DevicePath \"\"" Jul 10 19:21:24.909890 ip-10-0-143-17 kubenswrapper[2585]: I0710 19:21:24.909858 2585 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kserve-ci-e2e-test/llmisvc-model-fb-opt-125m-route-f312f5ec-kserve-mn-0-1" event={"ID":"59ed214a-e61e-4dff-9811-87c0c02e39c5","Type":"ContainerDied","Data":"1e6da6469d72846f37c795881214238bf58a4f73def6d0b9c1ba7ec31b5cb4f8"} Jul 10 19:21:24.910334 ip-10-0-143-17 kubenswrapper[2585]: I0710 19:21:24.909895 2585 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="kserve-ci-e2e-test/llmisvc-model-fb-opt-125m-route-f312f5ec-kserve-mn-0-1" Jul 10 19:21:24.910334 ip-10-0-143-17 kubenswrapper[2585]: I0710 19:21:24.909933 2585 scope.go:117] "RemoveContainer" containerID="626d96447ff59b7fb9513fe8b2b3bf9b43d921296efe7963c41be3800fd3254c" Jul 10 19:21:24.920993 ip-10-0-143-17 kubenswrapper[2585]: I0710 19:21:24.920587 2585 scope.go:117] "RemoveContainer" containerID="675ea93d898fda343d7a257072a86699c543a9c1d84c58817f678745efff6ef3" Jul 10 19:21:24.938605 ip-10-0-143-17 kubenswrapper[2585]: I0710 19:21:24.938585 2585 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["kserve-ci-e2e-test/llmisvc-model-fb-opt-125m-route-f312f5ec-kserve-mn-0-1"] Jul 10 19:21:24.943951 ip-10-0-143-17 kubenswrapper[2585]: I0710 19:21:24.943930 2585 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["kserve-ci-e2e-test/llmisvc-model-fb-opt-125m-route-f312f5ec-kserve-mn-0-1"] Jul 10 19:21:25.222093 ip-10-0-143-17 kubenswrapper[2585]: I0710 19:21:25.222053 2585 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="59ed214a-e61e-4dff-9811-87c0c02e39c5" path="/var/lib/kubelet/pods/59ed214a-e61e-4dff-9811-87c0c02e39c5/volumes" Jul 10 19:21:25.546684 ip-10-0-143-17 kubenswrapper[2585]: I0710 19:21:25.546579 2585 prober.go:120] "Probe failed" probeType="Startup" pod="kserve-ci-e2e-test/custom-route-timeout-pd-test-kserve-d9768b96b-f4kzh" podUID="582d08e8-3203-452c-b6e5-885db8cace55" containerName="main" probeResult="failure" output="Get \"https://10.133.0.51:8001/health\": dial tcp 10.133.0.51:8001: connect: connection refused" Jul 10 19:21:25.565993 ip-10-0-143-17 kubenswrapper[2585]: I0710 19:21:25.565921 2585 prober.go:120] "Probe failed" probeType="Startup" pod="kserve-ci-e2e-test/custom-route-timeout-pd-test-kserve-prefill-7d465b7985-qljdn" podUID="3b61c19c-497d-48a7-90c1-7d00ae4a0dc8" containerName="main" probeResult="failure" output="Get \"https://10.133.0.52:8000/health\": dial tcp 10.133.0.52:8000: connect: connection refused" Jul 10 19:21:35.546594 ip-10-0-143-17 kubenswrapper[2585]: I0710 19:21:35.546518 2585 prober.go:120] "Probe failed" probeType="Startup" pod="kserve-ci-e2e-test/custom-route-timeout-pd-test-kserve-d9768b96b-f4kzh" podUID="582d08e8-3203-452c-b6e5-885db8cace55" containerName="main" probeResult="failure" output="Get \"https://10.133.0.51:8001/health\": dial tcp 10.133.0.51:8001: connect: connection refused" Jul 10 19:21:35.565246 ip-10-0-143-17 kubenswrapper[2585]: I0710 19:21:35.565203 2585 prober.go:120] "Probe failed" probeType="Startup" pod="kserve-ci-e2e-test/custom-route-timeout-pd-test-kserve-prefill-7d465b7985-qljdn" podUID="3b61c19c-497d-48a7-90c1-7d00ae4a0dc8" containerName="main" probeResult="failure" output="Get \"https://10.133.0.52:8000/health\": dial tcp 10.133.0.52:8000: connect: connection refused" Jul 10 19:21:45.546007 ip-10-0-143-17 kubenswrapper[2585]: I0710 19:21:45.545901 2585 prober.go:120] "Probe failed" probeType="Startup" pod="kserve-ci-e2e-test/custom-route-timeout-pd-test-kserve-d9768b96b-f4kzh" podUID="582d08e8-3203-452c-b6e5-885db8cace55" containerName="main" probeResult="failure" output="Get \"https://10.133.0.51:8001/health\": dial tcp 10.133.0.51:8001: connect: connection refused" Jul 10 19:21:45.564585 ip-10-0-143-17 kubenswrapper[2585]: I0710 19:21:45.564515 2585 prober.go:120] "Probe failed" probeType="Startup" pod="kserve-ci-e2e-test/custom-route-timeout-pd-test-kserve-prefill-7d465b7985-qljdn" podUID="3b61c19c-497d-48a7-90c1-7d00ae4a0dc8" containerName="main" probeResult="failure" output="Get \"https://10.133.0.52:8000/health\": dial tcp 10.133.0.52:8000: connect: connection refused" Jul 10 19:21:55.545840 ip-10-0-143-17 kubenswrapper[2585]: I0710 19:21:55.545794 2585 prober.go:120] "Probe failed" probeType="Startup" pod="kserve-ci-e2e-test/custom-route-timeout-pd-test-kserve-d9768b96b-f4kzh" podUID="582d08e8-3203-452c-b6e5-885db8cace55" containerName="main" probeResult="failure" output="Get \"https://10.133.0.51:8001/health\": dial tcp 10.133.0.51:8001: connect: connection refused" Jul 10 19:21:55.564417 ip-10-0-143-17 kubenswrapper[2585]: I0710 19:21:55.564373 2585 prober.go:120] "Probe failed" probeType="Startup" pod="kserve-ci-e2e-test/custom-route-timeout-pd-test-kserve-prefill-7d465b7985-qljdn" podUID="3b61c19c-497d-48a7-90c1-7d00ae4a0dc8" containerName="main" probeResult="failure" output="Get \"https://10.133.0.52:8000/health\": dial tcp 10.133.0.52:8000: connect: connection refused" Jul 10 19:22:05.545717 ip-10-0-143-17 kubenswrapper[2585]: I0710 19:22:05.545668 2585 prober.go:120] "Probe failed" probeType="Startup" pod="kserve-ci-e2e-test/custom-route-timeout-pd-test-kserve-d9768b96b-f4kzh" podUID="582d08e8-3203-452c-b6e5-885db8cace55" containerName="main" probeResult="failure" output="Get \"https://10.133.0.51:8001/health\": dial tcp 10.133.0.51:8001: connect: connection refused" Jul 10 19:22:05.564597 ip-10-0-143-17 kubenswrapper[2585]: I0710 19:22:05.564562 2585 prober.go:120] "Probe failed" probeType="Startup" pod="kserve-ci-e2e-test/custom-route-timeout-pd-test-kserve-prefill-7d465b7985-qljdn" podUID="3b61c19c-497d-48a7-90c1-7d00ae4a0dc8" containerName="main" probeResult="failure" output="Get \"https://10.133.0.52:8000/health\": dial tcp 10.133.0.52:8000: connect: connection refused" Jul 10 19:22:15.545690 ip-10-0-143-17 kubenswrapper[2585]: I0710 19:22:15.545644 2585 prober.go:120] "Probe failed" probeType="Startup" pod="kserve-ci-e2e-test/custom-route-timeout-pd-test-kserve-d9768b96b-f4kzh" podUID="582d08e8-3203-452c-b6e5-885db8cace55" containerName="main" probeResult="failure" output="Get \"https://10.133.0.51:8001/health\": dial tcp 10.133.0.51:8001: connect: connection refused" Jul 10 19:22:15.564749 ip-10-0-143-17 kubenswrapper[2585]: I0710 19:22:15.564705 2585 prober.go:120] "Probe failed" probeType="Startup" pod="kserve-ci-e2e-test/custom-route-timeout-pd-test-kserve-prefill-7d465b7985-qljdn" podUID="3b61c19c-497d-48a7-90c1-7d00ae4a0dc8" containerName="main" probeResult="failure" output="Get \"https://10.133.0.52:8000/health\": dial tcp 10.133.0.52:8000: connect: connection refused" Jul 10 19:22:25.546564 ip-10-0-143-17 kubenswrapper[2585]: I0710 19:22:25.546485 2585 prober.go:120] "Probe failed" probeType="Startup" pod="kserve-ci-e2e-test/custom-route-timeout-pd-test-kserve-d9768b96b-f4kzh" podUID="582d08e8-3203-452c-b6e5-885db8cace55" containerName="main" probeResult="failure" output="Get \"https://10.133.0.51:8001/health\": dial tcp 10.133.0.51:8001: connect: connection refused" Jul 10 19:22:25.564226 ip-10-0-143-17 kubenswrapper[2585]: I0710 19:22:25.564189 2585 prober.go:120] "Probe failed" probeType="Startup" pod="kserve-ci-e2e-test/custom-route-timeout-pd-test-kserve-prefill-7d465b7985-qljdn" podUID="3b61c19c-497d-48a7-90c1-7d00ae4a0dc8" containerName="main" probeResult="failure" output="Get \"https://10.133.0.52:8000/health\": dial tcp 10.133.0.52:8000: connect: connection refused" Jul 10 19:22:35.546259 ip-10-0-143-17 kubenswrapper[2585]: I0710 19:22:35.546213 2585 prober.go:120] "Probe failed" probeType="Startup" pod="kserve-ci-e2e-test/custom-route-timeout-pd-test-kserve-d9768b96b-f4kzh" podUID="582d08e8-3203-452c-b6e5-885db8cace55" containerName="main" probeResult="failure" output="Get \"https://10.133.0.51:8001/health\": dial tcp 10.133.0.51:8001: connect: connection refused" Jul 10 19:22:35.564732 ip-10-0-143-17 kubenswrapper[2585]: I0710 19:22:35.564698 2585 prober.go:120] "Probe failed" probeType="Startup" pod="kserve-ci-e2e-test/custom-route-timeout-pd-test-kserve-prefill-7d465b7985-qljdn" podUID="3b61c19c-497d-48a7-90c1-7d00ae4a0dc8" containerName="main" probeResult="failure" output="Get \"https://10.133.0.52:8000/health\": dial tcp 10.133.0.52:8000: connect: connection refused" Jul 10 19:22:45.546022 ip-10-0-143-17 kubenswrapper[2585]: I0710 19:22:45.545978 2585 prober.go:120] "Probe failed" probeType="Startup" pod="kserve-ci-e2e-test/custom-route-timeout-pd-test-kserve-d9768b96b-f4kzh" podUID="582d08e8-3203-452c-b6e5-885db8cace55" containerName="main" probeResult="failure" output="Get \"https://10.133.0.51:8001/health\": dial tcp 10.133.0.51:8001: connect: connection refused" Jul 10 19:22:45.564353 ip-10-0-143-17 kubenswrapper[2585]: I0710 19:22:45.564312 2585 prober.go:120] "Probe failed" probeType="Startup" pod="kserve-ci-e2e-test/custom-route-timeout-pd-test-kserve-prefill-7d465b7985-qljdn" podUID="3b61c19c-497d-48a7-90c1-7d00ae4a0dc8" containerName="main" probeResult="failure" output="Get \"https://10.133.0.52:8000/health\": dial tcp 10.133.0.52:8000: connect: connection refused" Jul 10 19:22:55.546273 ip-10-0-143-17 kubenswrapper[2585]: I0710 19:22:55.546222 2585 prober.go:120] "Probe failed" probeType="Startup" pod="kserve-ci-e2e-test/custom-route-timeout-pd-test-kserve-d9768b96b-f4kzh" podUID="582d08e8-3203-452c-b6e5-885db8cace55" containerName="main" probeResult="failure" output="Get \"https://10.133.0.51:8001/health\": dial tcp 10.133.0.51:8001: connect: connection refused" Jul 10 19:22:55.564125 ip-10-0-143-17 kubenswrapper[2585]: I0710 19:22:55.564075 2585 prober.go:120] "Probe failed" probeType="Startup" pod="kserve-ci-e2e-test/custom-route-timeout-pd-test-kserve-prefill-7d465b7985-qljdn" podUID="3b61c19c-497d-48a7-90c1-7d00ae4a0dc8" containerName="main" probeResult="failure" output="Get \"https://10.133.0.52:8000/health\": dial tcp 10.133.0.52:8000: connect: connection refused" Jul 10 19:23:05.545990 ip-10-0-143-17 kubenswrapper[2585]: I0710 19:23:05.545945 2585 prober.go:120] "Probe failed" probeType="Startup" pod="kserve-ci-e2e-test/custom-route-timeout-pd-test-kserve-d9768b96b-f4kzh" podUID="582d08e8-3203-452c-b6e5-885db8cace55" containerName="main" probeResult="failure" output="Get \"https://10.133.0.51:8001/health\": dial tcp 10.133.0.51:8001: connect: connection refused" Jul 10 19:23:05.564148 ip-10-0-143-17 kubenswrapper[2585]: I0710 19:23:05.564108 2585 prober.go:120] "Probe failed" probeType="Startup" pod="kserve-ci-e2e-test/custom-route-timeout-pd-test-kserve-prefill-7d465b7985-qljdn" podUID="3b61c19c-497d-48a7-90c1-7d00ae4a0dc8" containerName="main" probeResult="failure" output="Get \"https://10.133.0.52:8000/health\": dial tcp 10.133.0.52:8000: connect: connection refused" Jul 10 19:23:15.555153 ip-10-0-143-17 kubenswrapper[2585]: I0710 19:23:15.555059 2585 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="kserve-ci-e2e-test/custom-route-timeout-pd-test-kserve-d9768b96b-f4kzh" Jul 10 19:23:15.568844 ip-10-0-143-17 kubenswrapper[2585]: I0710 19:23:15.568816 2585 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="kserve-ci-e2e-test/custom-route-timeout-pd-test-kserve-d9768b96b-f4kzh" Jul 10 19:23:15.577981 ip-10-0-143-17 kubenswrapper[2585]: I0710 19:23:15.577949 2585 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="kserve-ci-e2e-test/custom-route-timeout-pd-test-kserve-prefill-7d465b7985-qljdn" Jul 10 19:23:15.585841 ip-10-0-143-17 kubenswrapper[2585]: I0710 19:23:15.585813 2585 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="kserve-ci-e2e-test/custom-route-timeout-pd-test-kserve-prefill-7d465b7985-qljdn" Jul 10 19:23:27.319635 ip-10-0-143-17 kubenswrapper[2585]: I0710 19:23:27.319600 2585 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["kserve-ci-e2e-test/custom-route-timeout-pd-test-kserve-d9768b96b-f4kzh"] Jul 10 19:23:27.320107 ip-10-0-143-17 kubenswrapper[2585]: I0710 19:23:27.319968 2585 kuberuntime_container.go:873] "Killing container with a grace period" pod="kserve-ci-e2e-test/custom-route-timeout-pd-test-kserve-d9768b96b-f4kzh" podUID="582d08e8-3203-452c-b6e5-885db8cace55" containerName="main" containerID="cri-o://8c825b781828d8a226b5d9720d6a186b412e0949e4c0fd868c8fe042fb981456" gracePeriod=30 Jul 10 19:23:27.322515 ip-10-0-143-17 kubenswrapper[2585]: I0710 19:23:27.322488 2585 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["kserve-ci-e2e-test/custom-route-timeout-pd-test-kserve-prefill-7d465b7985-qljdn"] Jul 10 19:23:27.322723 ip-10-0-143-17 kubenswrapper[2585]: I0710 19:23:27.322686 2585 kuberuntime_container.go:873] "Killing container with a grace period" pod="kserve-ci-e2e-test/custom-route-timeout-pd-test-kserve-prefill-7d465b7985-qljdn" podUID="3b61c19c-497d-48a7-90c1-7d00ae4a0dc8" containerName="main" containerID="cri-o://507a501cffc9d609795dac967b4bbca1b8131ab63ecb579ab00a3df6048be181" gracePeriod=30 Jul 10 19:23:45.121194 ip-10-0-143-17 kubenswrapper[2585]: I0710 19:23:45.121164 2585 log.go:25] "Finished parsing log file" path="/var/log/pods/kserve-ci-e2e-test_custom-route-timeout-pd-test-kserve-d9768b96b-f4kzh_582d08e8-3203-452c-b6e5-885db8cace55/main/0.log" Jul 10 19:23:45.137722 ip-10-0-143-17 kubenswrapper[2585]: I0710 19:23:45.137687 2585 log.go:25] "Finished parsing log file" path="/var/log/pods/kserve-ci-e2e-test_custom-route-timeout-pd-test-kserve-d9768b96b-f4kzh_582d08e8-3203-452c-b6e5-885db8cace55/llm-d-routing-sidecar/0.log" Jul 10 19:23:45.149067 ip-10-0-143-17 kubenswrapper[2585]: I0710 19:23:45.149040 2585 log.go:25] "Finished parsing log file" path="/var/log/pods/kserve-ci-e2e-test_custom-route-timeout-pd-test-kserve-d9768b96b-f4kzh_582d08e8-3203-452c-b6e5-885db8cace55/storage-initializer/0.log" Jul 10 19:23:45.168647 ip-10-0-143-17 kubenswrapper[2585]: I0710 19:23:45.168616 2585 log.go:25] "Finished parsing log file" path="/var/log/pods/kserve-ci-e2e-test_custom-route-timeout-pd-test-kserve-prefill-7d465b7985-qljdn_3b61c19c-497d-48a7-90c1-7d00ae4a0dc8/main/0.log" Jul 10 19:23:45.175105 ip-10-0-143-17 kubenswrapper[2585]: I0710 19:23:45.175083 2585 log.go:25] "Finished parsing log file" path="/var/log/pods/kserve-ci-e2e-test_custom-route-timeout-pd-test-kserve-prefill-7d465b7985-qljdn_3b61c19c-497d-48a7-90c1-7d00ae4a0dc8/storage-initializer/0.log" Jul 10 19:23:45.195622 ip-10-0-143-17 kubenswrapper[2585]: I0710 19:23:45.195599 2585 log.go:25] "Finished parsing log file" path="/var/log/pods/kserve-ci-e2e-test_router-gateway-1-openshift-default-799f46c59b-sf6qk_10d311d2-b0ff-486f-a709-ef0f52dfa327/istio-proxy/1.log" Jul 10 19:23:45.195977 ip-10-0-143-17 kubenswrapper[2585]: I0710 19:23:45.195957 2585 log.go:25] "Finished parsing log file" path="/var/log/pods/kserve-ci-e2e-test_router-gateway-1-openshift-default-799f46c59b-sf6qk_10d311d2-b0ff-486f-a709-ef0f52dfa327/istio-proxy/0.log" Jul 10 19:23:45.209738 ip-10-0-143-17 kubenswrapper[2585]: I0710 19:23:45.209713 2585 log.go:25] "Finished parsing log file" path="/var/log/pods/kserve-ci-e2e-test_router-gateway-2-openshift-default-54c789bdc6-66nfj_7aa2cf75-c55d-45ff-a4ca-9bfba3d0a016/istio-proxy/1.log" Jul 10 19:23:45.211371 ip-10-0-143-17 kubenswrapper[2585]: I0710 19:23:45.211351 2585 log.go:25] "Finished parsing log file" path="/var/log/pods/kserve-ci-e2e-test_router-gateway-2-openshift-default-54c789bdc6-66nfj_7aa2cf75-c55d-45ff-a4ca-9bfba3d0a016/istio-proxy/0.log" Jul 10 19:23:46.215559 ip-10-0-143-17 kubenswrapper[2585]: I0710 19:23:46.215511 2585 log.go:25] "Finished parsing log file" path="/var/log/pods/kserve-ci-e2e-test_custom-route-timeout-pd-test-kserve-d9768b96b-f4kzh_582d08e8-3203-452c-b6e5-885db8cace55/main/0.log" Jul 10 19:23:46.227409 ip-10-0-143-17 kubenswrapper[2585]: I0710 19:23:46.227388 2585 log.go:25] "Finished parsing log file" path="/var/log/pods/kserve-ci-e2e-test_custom-route-timeout-pd-test-kserve-d9768b96b-f4kzh_582d08e8-3203-452c-b6e5-885db8cace55/llm-d-routing-sidecar/0.log" Jul 10 19:23:46.260832 ip-10-0-143-17 kubenswrapper[2585]: I0710 19:23:46.260808 2585 log.go:25] "Finished parsing log file" path="/var/log/pods/kserve-ci-e2e-test_custom-route-timeout-pd-test-kserve-d9768b96b-f4kzh_582d08e8-3203-452c-b6e5-885db8cace55/storage-initializer/0.log" Jul 10 19:23:46.297045 ip-10-0-143-17 kubenswrapper[2585]: I0710 19:23:46.297021 2585 log.go:25] "Finished parsing log file" path="/var/log/pods/kserve-ci-e2e-test_custom-route-timeout-pd-test-kserve-prefill-7d465b7985-qljdn_3b61c19c-497d-48a7-90c1-7d00ae4a0dc8/main/0.log" Jul 10 19:23:46.312395 ip-10-0-143-17 kubenswrapper[2585]: I0710 19:23:46.312373 2585 log.go:25] "Finished parsing log file" path="/var/log/pods/kserve-ci-e2e-test_custom-route-timeout-pd-test-kserve-prefill-7d465b7985-qljdn_3b61c19c-497d-48a7-90c1-7d00ae4a0dc8/storage-initializer/0.log" Jul 10 19:23:46.335807 ip-10-0-143-17 kubenswrapper[2585]: I0710 19:23:46.335780 2585 log.go:25] "Finished parsing log file" path="/var/log/pods/kserve-ci-e2e-test_router-gateway-1-openshift-default-799f46c59b-sf6qk_10d311d2-b0ff-486f-a709-ef0f52dfa327/istio-proxy/1.log" Jul 10 19:23:46.336207 ip-10-0-143-17 kubenswrapper[2585]: I0710 19:23:46.336191 2585 log.go:25] "Finished parsing log file" path="/var/log/pods/kserve-ci-e2e-test_router-gateway-1-openshift-default-799f46c59b-sf6qk_10d311d2-b0ff-486f-a709-ef0f52dfa327/istio-proxy/0.log" Jul 10 19:23:46.352039 ip-10-0-143-17 kubenswrapper[2585]: I0710 19:23:46.352016 2585 log.go:25] "Finished parsing log file" path="/var/log/pods/kserve-ci-e2e-test_router-gateway-2-openshift-default-54c789bdc6-66nfj_7aa2cf75-c55d-45ff-a4ca-9bfba3d0a016/istio-proxy/1.log" Jul 10 19:23:46.352493 ip-10-0-143-17 kubenswrapper[2585]: I0710 19:23:46.352475 2585 log.go:25] "Finished parsing log file" path="/var/log/pods/kserve-ci-e2e-test_router-gateway-2-openshift-default-54c789bdc6-66nfj_7aa2cf75-c55d-45ff-a4ca-9bfba3d0a016/istio-proxy/0.log" Jul 10 19:23:47.391393 ip-10-0-143-17 kubenswrapper[2585]: I0710 19:23:47.391335 2585 log.go:25] "Finished parsing log file" path="/var/log/pods/kserve-ci-e2e-test_custom-route-timeout-pd-test-kserve-d9768b96b-f4kzh_582d08e8-3203-452c-b6e5-885db8cace55/main/0.log" Jul 10 19:23:47.399135 ip-10-0-143-17 kubenswrapper[2585]: I0710 19:23:47.399112 2585 log.go:25] "Finished parsing log file" path="/var/log/pods/kserve-ci-e2e-test_custom-route-timeout-pd-test-kserve-d9768b96b-f4kzh_582d08e8-3203-452c-b6e5-885db8cace55/llm-d-routing-sidecar/0.log" Jul 10 19:23:47.411423 ip-10-0-143-17 kubenswrapper[2585]: I0710 19:23:47.411404 2585 log.go:25] "Finished parsing log file" path="/var/log/pods/kserve-ci-e2e-test_custom-route-timeout-pd-test-kserve-d9768b96b-f4kzh_582d08e8-3203-452c-b6e5-885db8cace55/storage-initializer/0.log" Jul 10 19:23:47.431683 ip-10-0-143-17 kubenswrapper[2585]: I0710 19:23:47.431662 2585 log.go:25] "Finished parsing log file" path="/var/log/pods/kserve-ci-e2e-test_custom-route-timeout-pd-test-kserve-prefill-7d465b7985-qljdn_3b61c19c-497d-48a7-90c1-7d00ae4a0dc8/main/0.log" Jul 10 19:23:47.439225 ip-10-0-143-17 kubenswrapper[2585]: I0710 19:23:47.439188 2585 log.go:25] "Finished parsing log file" path="/var/log/pods/kserve-ci-e2e-test_custom-route-timeout-pd-test-kserve-prefill-7d465b7985-qljdn_3b61c19c-497d-48a7-90c1-7d00ae4a0dc8/storage-initializer/0.log" Jul 10 19:23:47.458310 ip-10-0-143-17 kubenswrapper[2585]: I0710 19:23:47.458288 2585 log.go:25] "Finished parsing log file" path="/var/log/pods/kserve-ci-e2e-test_router-gateway-1-openshift-default-799f46c59b-sf6qk_10d311d2-b0ff-486f-a709-ef0f52dfa327/istio-proxy/1.log" Jul 10 19:23:47.458772 ip-10-0-143-17 kubenswrapper[2585]: I0710 19:23:47.458753 2585 log.go:25] "Finished parsing log file" path="/var/log/pods/kserve-ci-e2e-test_router-gateway-1-openshift-default-799f46c59b-sf6qk_10d311d2-b0ff-486f-a709-ef0f52dfa327/istio-proxy/0.log" Jul 10 19:23:47.474148 ip-10-0-143-17 kubenswrapper[2585]: I0710 19:23:47.474125 2585 log.go:25] "Finished parsing log file" path="/var/log/pods/kserve-ci-e2e-test_router-gateway-2-openshift-default-54c789bdc6-66nfj_7aa2cf75-c55d-45ff-a4ca-9bfba3d0a016/istio-proxy/1.log" Jul 10 19:23:47.474584 ip-10-0-143-17 kubenswrapper[2585]: I0710 19:23:47.474563 2585 log.go:25] "Finished parsing log file" path="/var/log/pods/kserve-ci-e2e-test_router-gateway-2-openshift-default-54c789bdc6-66nfj_7aa2cf75-c55d-45ff-a4ca-9bfba3d0a016/istio-proxy/0.log" Jul 10 19:23:48.498030 ip-10-0-143-17 kubenswrapper[2585]: I0710 19:23:48.498001 2585 log.go:25] "Finished parsing log file" path="/var/log/pods/kserve-ci-e2e-test_custom-route-timeout-pd-test-kserve-d9768b96b-f4kzh_582d08e8-3203-452c-b6e5-885db8cace55/main/0.log" Jul 10 19:23:48.505480 ip-10-0-143-17 kubenswrapper[2585]: I0710 19:23:48.505451 2585 log.go:25] "Finished parsing log file" path="/var/log/pods/kserve-ci-e2e-test_custom-route-timeout-pd-test-kserve-d9768b96b-f4kzh_582d08e8-3203-452c-b6e5-885db8cace55/llm-d-routing-sidecar/0.log" Jul 10 19:23:48.517719 ip-10-0-143-17 kubenswrapper[2585]: I0710 19:23:48.517689 2585 log.go:25] "Finished parsing log file" path="/var/log/pods/kserve-ci-e2e-test_custom-route-timeout-pd-test-kserve-d9768b96b-f4kzh_582d08e8-3203-452c-b6e5-885db8cace55/storage-initializer/0.log" Jul 10 19:23:48.539223 ip-10-0-143-17 kubenswrapper[2585]: I0710 19:23:48.539189 2585 log.go:25] "Finished parsing log file" path="/var/log/pods/kserve-ci-e2e-test_custom-route-timeout-pd-test-kserve-prefill-7d465b7985-qljdn_3b61c19c-497d-48a7-90c1-7d00ae4a0dc8/main/0.log" Jul 10 19:23:48.546563 ip-10-0-143-17 kubenswrapper[2585]: I0710 19:23:48.546518 2585 log.go:25] "Finished parsing log file" path="/var/log/pods/kserve-ci-e2e-test_custom-route-timeout-pd-test-kserve-prefill-7d465b7985-qljdn_3b61c19c-497d-48a7-90c1-7d00ae4a0dc8/storage-initializer/0.log" Jul 10 19:23:48.564659 ip-10-0-143-17 kubenswrapper[2585]: I0710 19:23:48.564631 2585 log.go:25] "Finished parsing log file" path="/var/log/pods/kserve-ci-e2e-test_router-gateway-1-openshift-default-799f46c59b-sf6qk_10d311d2-b0ff-486f-a709-ef0f52dfa327/istio-proxy/1.log" Jul 10 19:23:48.565085 ip-10-0-143-17 kubenswrapper[2585]: I0710 19:23:48.565067 2585 log.go:25] "Finished parsing log file" path="/var/log/pods/kserve-ci-e2e-test_router-gateway-1-openshift-default-799f46c59b-sf6qk_10d311d2-b0ff-486f-a709-ef0f52dfa327/istio-proxy/0.log" Jul 10 19:23:48.579780 ip-10-0-143-17 kubenswrapper[2585]: I0710 19:23:48.579753 2585 log.go:25] "Finished parsing log file" path="/var/log/pods/kserve-ci-e2e-test_router-gateway-2-openshift-default-54c789bdc6-66nfj_7aa2cf75-c55d-45ff-a4ca-9bfba3d0a016/istio-proxy/1.log" Jul 10 19:23:48.580264 ip-10-0-143-17 kubenswrapper[2585]: I0710 19:23:48.580247 2585 log.go:25] "Finished parsing log file" path="/var/log/pods/kserve-ci-e2e-test_router-gateway-2-openshift-default-54c789bdc6-66nfj_7aa2cf75-c55d-45ff-a4ca-9bfba3d0a016/istio-proxy/0.log" Jul 10 19:23:49.532465 ip-10-0-143-17 kubenswrapper[2585]: I0710 19:23:49.532422 2585 log.go:25] "Finished parsing log file" path="/var/log/pods/kserve-ci-e2e-test_custom-route-timeout-pd-test-kserve-d9768b96b-f4kzh_582d08e8-3203-452c-b6e5-885db8cace55/main/0.log" Jul 10 19:23:49.540187 ip-10-0-143-17 kubenswrapper[2585]: I0710 19:23:49.540161 2585 log.go:25] "Finished parsing log file" path="/var/log/pods/kserve-ci-e2e-test_custom-route-timeout-pd-test-kserve-d9768b96b-f4kzh_582d08e8-3203-452c-b6e5-885db8cace55/llm-d-routing-sidecar/0.log" Jul 10 19:23:49.555792 ip-10-0-143-17 kubenswrapper[2585]: I0710 19:23:49.555763 2585 log.go:25] "Finished parsing log file" path="/var/log/pods/kserve-ci-e2e-test_custom-route-timeout-pd-test-kserve-d9768b96b-f4kzh_582d08e8-3203-452c-b6e5-885db8cace55/storage-initializer/0.log" Jul 10 19:23:49.576669 ip-10-0-143-17 kubenswrapper[2585]: I0710 19:23:49.576645 2585 log.go:25] "Finished parsing log file" path="/var/log/pods/kserve-ci-e2e-test_custom-route-timeout-pd-test-kserve-prefill-7d465b7985-qljdn_3b61c19c-497d-48a7-90c1-7d00ae4a0dc8/main/0.log" Jul 10 19:23:49.586038 ip-10-0-143-17 kubenswrapper[2585]: I0710 19:23:49.586020 2585 log.go:25] "Finished parsing log file" path="/var/log/pods/kserve-ci-e2e-test_custom-route-timeout-pd-test-kserve-prefill-7d465b7985-qljdn_3b61c19c-497d-48a7-90c1-7d00ae4a0dc8/storage-initializer/0.log" Jul 10 19:23:49.605404 ip-10-0-143-17 kubenswrapper[2585]: I0710 19:23:49.605371 2585 log.go:25] "Finished parsing log file" path="/var/log/pods/kserve-ci-e2e-test_router-gateway-1-openshift-default-799f46c59b-sf6qk_10d311d2-b0ff-486f-a709-ef0f52dfa327/istio-proxy/1.log" Jul 10 19:23:49.605833 ip-10-0-143-17 kubenswrapper[2585]: I0710 19:23:49.605813 2585 log.go:25] "Finished parsing log file" path="/var/log/pods/kserve-ci-e2e-test_router-gateway-1-openshift-default-799f46c59b-sf6qk_10d311d2-b0ff-486f-a709-ef0f52dfa327/istio-proxy/0.log" Jul 10 19:23:49.620878 ip-10-0-143-17 kubenswrapper[2585]: I0710 19:23:49.620858 2585 log.go:25] "Finished parsing log file" path="/var/log/pods/kserve-ci-e2e-test_router-gateway-2-openshift-default-54c789bdc6-66nfj_7aa2cf75-c55d-45ff-a4ca-9bfba3d0a016/istio-proxy/1.log" Jul 10 19:23:49.621258 ip-10-0-143-17 kubenswrapper[2585]: I0710 19:23:49.621241 2585 log.go:25] "Finished parsing log file" path="/var/log/pods/kserve-ci-e2e-test_router-gateway-2-openshift-default-54c789bdc6-66nfj_7aa2cf75-c55d-45ff-a4ca-9bfba3d0a016/istio-proxy/0.log" Jul 10 19:23:50.661610 ip-10-0-143-17 kubenswrapper[2585]: I0710 19:23:50.661578 2585 log.go:25] "Finished parsing log file" path="/var/log/pods/kserve-ci-e2e-test_custom-route-timeout-pd-test-kserve-d9768b96b-f4kzh_582d08e8-3203-452c-b6e5-885db8cace55/main/0.log" Jul 10 19:23:50.669373 ip-10-0-143-17 kubenswrapper[2585]: I0710 19:23:50.669351 2585 log.go:25] "Finished parsing log file" path="/var/log/pods/kserve-ci-e2e-test_custom-route-timeout-pd-test-kserve-d9768b96b-f4kzh_582d08e8-3203-452c-b6e5-885db8cace55/llm-d-routing-sidecar/0.log" Jul 10 19:23:50.693694 ip-10-0-143-17 kubenswrapper[2585]: I0710 19:23:50.693668 2585 log.go:25] "Finished parsing log file" path="/var/log/pods/kserve-ci-e2e-test_custom-route-timeout-pd-test-kserve-d9768b96b-f4kzh_582d08e8-3203-452c-b6e5-885db8cace55/storage-initializer/0.log" Jul 10 19:23:50.718225 ip-10-0-143-17 kubenswrapper[2585]: I0710 19:23:50.718197 2585 log.go:25] "Finished parsing log file" path="/var/log/pods/kserve-ci-e2e-test_custom-route-timeout-pd-test-kserve-prefill-7d465b7985-qljdn_3b61c19c-497d-48a7-90c1-7d00ae4a0dc8/main/0.log" Jul 10 19:23:50.728281 ip-10-0-143-17 kubenswrapper[2585]: I0710 19:23:50.728262 2585 log.go:25] "Finished parsing log file" path="/var/log/pods/kserve-ci-e2e-test_custom-route-timeout-pd-test-kserve-prefill-7d465b7985-qljdn_3b61c19c-497d-48a7-90c1-7d00ae4a0dc8/storage-initializer/0.log" Jul 10 19:23:50.747464 ip-10-0-143-17 kubenswrapper[2585]: I0710 19:23:50.747444 2585 log.go:25] "Finished parsing log file" path="/var/log/pods/kserve-ci-e2e-test_router-gateway-1-openshift-default-799f46c59b-sf6qk_10d311d2-b0ff-486f-a709-ef0f52dfa327/istio-proxy/1.log" Jul 10 19:23:50.748070 ip-10-0-143-17 kubenswrapper[2585]: I0710 19:23:50.748051 2585 log.go:25] "Finished parsing log file" path="/var/log/pods/kserve-ci-e2e-test_router-gateway-1-openshift-default-799f46c59b-sf6qk_10d311d2-b0ff-486f-a709-ef0f52dfa327/istio-proxy/0.log" Jul 10 19:23:50.762201 ip-10-0-143-17 kubenswrapper[2585]: I0710 19:23:50.762177 2585 log.go:25] "Finished parsing log file" path="/var/log/pods/kserve-ci-e2e-test_router-gateway-2-openshift-default-54c789bdc6-66nfj_7aa2cf75-c55d-45ff-a4ca-9bfba3d0a016/istio-proxy/1.log" Jul 10 19:23:50.762752 ip-10-0-143-17 kubenswrapper[2585]: I0710 19:23:50.762735 2585 log.go:25] "Finished parsing log file" path="/var/log/pods/kserve-ci-e2e-test_router-gateway-2-openshift-default-54c789bdc6-66nfj_7aa2cf75-c55d-45ff-a4ca-9bfba3d0a016/istio-proxy/0.log" Jul 10 19:23:51.748109 ip-10-0-143-17 kubenswrapper[2585]: I0710 19:23:51.748074 2585 log.go:25] "Finished parsing log file" path="/var/log/pods/kserve-ci-e2e-test_custom-route-timeout-pd-test-kserve-d9768b96b-f4kzh_582d08e8-3203-452c-b6e5-885db8cace55/main/0.log" Jul 10 19:23:51.755620 ip-10-0-143-17 kubenswrapper[2585]: I0710 19:23:51.755596 2585 log.go:25] "Finished parsing log file" path="/var/log/pods/kserve-ci-e2e-test_custom-route-timeout-pd-test-kserve-d9768b96b-f4kzh_582d08e8-3203-452c-b6e5-885db8cace55/llm-d-routing-sidecar/0.log" Jul 10 19:23:51.767295 ip-10-0-143-17 kubenswrapper[2585]: I0710 19:23:51.767274 2585 log.go:25] "Finished parsing log file" path="/var/log/pods/kserve-ci-e2e-test_custom-route-timeout-pd-test-kserve-d9768b96b-f4kzh_582d08e8-3203-452c-b6e5-885db8cace55/storage-initializer/0.log" Jul 10 19:23:51.788208 ip-10-0-143-17 kubenswrapper[2585]: I0710 19:23:51.788187 2585 log.go:25] "Finished parsing log file" path="/var/log/pods/kserve-ci-e2e-test_custom-route-timeout-pd-test-kserve-prefill-7d465b7985-qljdn_3b61c19c-497d-48a7-90c1-7d00ae4a0dc8/main/0.log" Jul 10 19:23:51.801124 ip-10-0-143-17 kubenswrapper[2585]: I0710 19:23:51.801098 2585 log.go:25] "Finished parsing log file" path="/var/log/pods/kserve-ci-e2e-test_custom-route-timeout-pd-test-kserve-prefill-7d465b7985-qljdn_3b61c19c-497d-48a7-90c1-7d00ae4a0dc8/storage-initializer/0.log" Jul 10 19:23:51.820680 ip-10-0-143-17 kubenswrapper[2585]: I0710 19:23:51.820657 2585 log.go:25] "Finished parsing log file" path="/var/log/pods/kserve-ci-e2e-test_router-gateway-1-openshift-default-799f46c59b-sf6qk_10d311d2-b0ff-486f-a709-ef0f52dfa327/istio-proxy/1.log" Jul 10 19:23:51.821008 ip-10-0-143-17 kubenswrapper[2585]: I0710 19:23:51.820967 2585 log.go:25] "Finished parsing log file" path="/var/log/pods/kserve-ci-e2e-test_router-gateway-1-openshift-default-799f46c59b-sf6qk_10d311d2-b0ff-486f-a709-ef0f52dfa327/istio-proxy/0.log" Jul 10 19:23:51.836418 ip-10-0-143-17 kubenswrapper[2585]: I0710 19:23:51.836391 2585 log.go:25] "Finished parsing log file" path="/var/log/pods/kserve-ci-e2e-test_router-gateway-2-openshift-default-54c789bdc6-66nfj_7aa2cf75-c55d-45ff-a4ca-9bfba3d0a016/istio-proxy/1.log" Jul 10 19:23:51.837064 ip-10-0-143-17 kubenswrapper[2585]: I0710 19:23:51.837043 2585 log.go:25] "Finished parsing log file" path="/var/log/pods/kserve-ci-e2e-test_router-gateway-2-openshift-default-54c789bdc6-66nfj_7aa2cf75-c55d-45ff-a4ca-9bfba3d0a016/istio-proxy/0.log" Jul 10 19:23:52.776016 ip-10-0-143-17 kubenswrapper[2585]: I0710 19:23:52.775979 2585 log.go:25] "Finished parsing log file" path="/var/log/pods/kserve-ci-e2e-test_custom-route-timeout-pd-test-kserve-d9768b96b-f4kzh_582d08e8-3203-452c-b6e5-885db8cace55/main/0.log" Jul 10 19:23:52.782795 ip-10-0-143-17 kubenswrapper[2585]: I0710 19:23:52.782774 2585 log.go:25] "Finished parsing log file" path="/var/log/pods/kserve-ci-e2e-test_custom-route-timeout-pd-test-kserve-d9768b96b-f4kzh_582d08e8-3203-452c-b6e5-885db8cace55/llm-d-routing-sidecar/0.log" Jul 10 19:23:52.793214 ip-10-0-143-17 kubenswrapper[2585]: I0710 19:23:52.793184 2585 log.go:25] "Finished parsing log file" path="/var/log/pods/kserve-ci-e2e-test_custom-route-timeout-pd-test-kserve-d9768b96b-f4kzh_582d08e8-3203-452c-b6e5-885db8cace55/storage-initializer/0.log" Jul 10 19:23:52.815116 ip-10-0-143-17 kubenswrapper[2585]: I0710 19:23:52.815095 2585 log.go:25] "Finished parsing log file" path="/var/log/pods/kserve-ci-e2e-test_custom-route-timeout-pd-test-kserve-prefill-7d465b7985-qljdn_3b61c19c-497d-48a7-90c1-7d00ae4a0dc8/main/0.log" Jul 10 19:23:52.822653 ip-10-0-143-17 kubenswrapper[2585]: I0710 19:23:52.822629 2585 log.go:25] "Finished parsing log file" path="/var/log/pods/kserve-ci-e2e-test_custom-route-timeout-pd-test-kserve-prefill-7d465b7985-qljdn_3b61c19c-497d-48a7-90c1-7d00ae4a0dc8/storage-initializer/0.log" Jul 10 19:23:52.841584 ip-10-0-143-17 kubenswrapper[2585]: I0710 19:23:52.841533 2585 log.go:25] "Finished parsing log file" path="/var/log/pods/kserve-ci-e2e-test_router-gateway-1-openshift-default-799f46c59b-sf6qk_10d311d2-b0ff-486f-a709-ef0f52dfa327/istio-proxy/1.log" Jul 10 19:23:52.842016 ip-10-0-143-17 kubenswrapper[2585]: I0710 19:23:52.841995 2585 log.go:25] "Finished parsing log file" path="/var/log/pods/kserve-ci-e2e-test_router-gateway-1-openshift-default-799f46c59b-sf6qk_10d311d2-b0ff-486f-a709-ef0f52dfa327/istio-proxy/0.log" Jul 10 19:23:52.858490 ip-10-0-143-17 kubenswrapper[2585]: I0710 19:23:52.858465 2585 log.go:25] "Finished parsing log file" path="/var/log/pods/kserve-ci-e2e-test_router-gateway-2-openshift-default-54c789bdc6-66nfj_7aa2cf75-c55d-45ff-a4ca-9bfba3d0a016/istio-proxy/1.log" Jul 10 19:23:52.859175 ip-10-0-143-17 kubenswrapper[2585]: I0710 19:23:52.859157 2585 log.go:25] "Finished parsing log file" path="/var/log/pods/kserve-ci-e2e-test_router-gateway-2-openshift-default-54c789bdc6-66nfj_7aa2cf75-c55d-45ff-a4ca-9bfba3d0a016/istio-proxy/0.log" Jul 10 19:23:53.865677 ip-10-0-143-17 kubenswrapper[2585]: I0710 19:23:53.865646 2585 log.go:25] "Finished parsing log file" path="/var/log/pods/kserve-ci-e2e-test_custom-route-timeout-pd-test-kserve-d9768b96b-f4kzh_582d08e8-3203-452c-b6e5-885db8cace55/main/0.log" Jul 10 19:23:53.876167 ip-10-0-143-17 kubenswrapper[2585]: I0710 19:23:53.876142 2585 log.go:25] "Finished parsing log file" path="/var/log/pods/kserve-ci-e2e-test_custom-route-timeout-pd-test-kserve-d9768b96b-f4kzh_582d08e8-3203-452c-b6e5-885db8cace55/llm-d-routing-sidecar/0.log" Jul 10 19:23:53.895236 ip-10-0-143-17 kubenswrapper[2585]: I0710 19:23:53.895211 2585 log.go:25] "Finished parsing log file" path="/var/log/pods/kserve-ci-e2e-test_custom-route-timeout-pd-test-kserve-d9768b96b-f4kzh_582d08e8-3203-452c-b6e5-885db8cace55/storage-initializer/0.log" Jul 10 19:23:53.918651 ip-10-0-143-17 kubenswrapper[2585]: I0710 19:23:53.918630 2585 log.go:25] "Finished parsing log file" path="/var/log/pods/kserve-ci-e2e-test_custom-route-timeout-pd-test-kserve-prefill-7d465b7985-qljdn_3b61c19c-497d-48a7-90c1-7d00ae4a0dc8/main/0.log" Jul 10 19:23:53.928147 ip-10-0-143-17 kubenswrapper[2585]: I0710 19:23:53.928126 2585 log.go:25] "Finished parsing log file" path="/var/log/pods/kserve-ci-e2e-test_custom-route-timeout-pd-test-kserve-prefill-7d465b7985-qljdn_3b61c19c-497d-48a7-90c1-7d00ae4a0dc8/storage-initializer/0.log" Jul 10 19:23:53.948357 ip-10-0-143-17 kubenswrapper[2585]: I0710 19:23:53.948333 2585 log.go:25] "Finished parsing log file" path="/var/log/pods/kserve-ci-e2e-test_router-gateway-1-openshift-default-799f46c59b-sf6qk_10d311d2-b0ff-486f-a709-ef0f52dfa327/istio-proxy/1.log" Jul 10 19:23:53.948843 ip-10-0-143-17 kubenswrapper[2585]: I0710 19:23:53.948828 2585 log.go:25] "Finished parsing log file" path="/var/log/pods/kserve-ci-e2e-test_router-gateway-1-openshift-default-799f46c59b-sf6qk_10d311d2-b0ff-486f-a709-ef0f52dfa327/istio-proxy/0.log" Jul 10 19:23:53.965865 ip-10-0-143-17 kubenswrapper[2585]: I0710 19:23:53.965840 2585 log.go:25] "Finished parsing log file" path="/var/log/pods/kserve-ci-e2e-test_router-gateway-2-openshift-default-54c789bdc6-66nfj_7aa2cf75-c55d-45ff-a4ca-9bfba3d0a016/istio-proxy/1.log" Jul 10 19:23:53.966308 ip-10-0-143-17 kubenswrapper[2585]: I0710 19:23:53.966292 2585 log.go:25] "Finished parsing log file" path="/var/log/pods/kserve-ci-e2e-test_router-gateway-2-openshift-default-54c789bdc6-66nfj_7aa2cf75-c55d-45ff-a4ca-9bfba3d0a016/istio-proxy/0.log" Jul 10 19:23:55.013912 ip-10-0-143-17 kubenswrapper[2585]: I0710 19:23:55.013882 2585 log.go:25] "Finished parsing log file" path="/var/log/pods/kserve-ci-e2e-test_custom-route-timeout-pd-test-kserve-d9768b96b-f4kzh_582d08e8-3203-452c-b6e5-885db8cace55/main/0.log" Jul 10 19:23:55.056365 ip-10-0-143-17 kubenswrapper[2585]: I0710 19:23:55.056341 2585 log.go:25] "Finished parsing log file" path="/var/log/pods/kserve-ci-e2e-test_custom-route-timeout-pd-test-kserve-d9768b96b-f4kzh_582d08e8-3203-452c-b6e5-885db8cace55/llm-d-routing-sidecar/0.log" Jul 10 19:23:55.096709 ip-10-0-143-17 kubenswrapper[2585]: I0710 19:23:55.096683 2585 log.go:25] "Finished parsing log file" path="/var/log/pods/kserve-ci-e2e-test_custom-route-timeout-pd-test-kserve-d9768b96b-f4kzh_582d08e8-3203-452c-b6e5-885db8cace55/storage-initializer/0.log" Jul 10 19:23:55.124613 ip-10-0-143-17 kubenswrapper[2585]: I0710 19:23:55.124586 2585 log.go:25] "Finished parsing log file" path="/var/log/pods/kserve-ci-e2e-test_custom-route-timeout-pd-test-kserve-prefill-7d465b7985-qljdn_3b61c19c-497d-48a7-90c1-7d00ae4a0dc8/main/0.log" Jul 10 19:23:55.151449 ip-10-0-143-17 kubenswrapper[2585]: I0710 19:23:55.151430 2585 log.go:25] "Finished parsing log file" path="/var/log/pods/kserve-ci-e2e-test_custom-route-timeout-pd-test-kserve-prefill-7d465b7985-qljdn_3b61c19c-497d-48a7-90c1-7d00ae4a0dc8/storage-initializer/0.log" Jul 10 19:23:55.171508 ip-10-0-143-17 kubenswrapper[2585]: I0710 19:23:55.171476 2585 log.go:25] "Finished parsing log file" path="/var/log/pods/kserve-ci-e2e-test_router-gateway-1-openshift-default-799f46c59b-sf6qk_10d311d2-b0ff-486f-a709-ef0f52dfa327/istio-proxy/1.log" Jul 10 19:23:55.172001 ip-10-0-143-17 kubenswrapper[2585]: I0710 19:23:55.171975 2585 log.go:25] "Finished parsing log file" path="/var/log/pods/kserve-ci-e2e-test_router-gateway-1-openshift-default-799f46c59b-sf6qk_10d311d2-b0ff-486f-a709-ef0f52dfa327/istio-proxy/0.log" Jul 10 19:23:55.189395 ip-10-0-143-17 kubenswrapper[2585]: I0710 19:23:55.189375 2585 log.go:25] "Finished parsing log file" path="/var/log/pods/kserve-ci-e2e-test_router-gateway-2-openshift-default-54c789bdc6-66nfj_7aa2cf75-c55d-45ff-a4ca-9bfba3d0a016/istio-proxy/1.log" Jul 10 19:23:55.189914 ip-10-0-143-17 kubenswrapper[2585]: I0710 19:23:55.189897 2585 log.go:25] "Finished parsing log file" path="/var/log/pods/kserve-ci-e2e-test_router-gateway-2-openshift-default-54c789bdc6-66nfj_7aa2cf75-c55d-45ff-a4ca-9bfba3d0a016/istio-proxy/0.log" Jul 10 19:23:56.477095 ip-10-0-143-17 kubenswrapper[2585]: I0710 19:23:56.477068 2585 log.go:25] "Finished parsing log file" path="/var/log/pods/kserve-ci-e2e-test_custom-route-timeout-pd-test-kserve-d9768b96b-f4kzh_582d08e8-3203-452c-b6e5-885db8cace55/main/0.log" Jul 10 19:23:56.485289 ip-10-0-143-17 kubenswrapper[2585]: I0710 19:23:56.485260 2585 log.go:25] "Finished parsing log file" path="/var/log/pods/kserve-ci-e2e-test_custom-route-timeout-pd-test-kserve-d9768b96b-f4kzh_582d08e8-3203-452c-b6e5-885db8cace55/llm-d-routing-sidecar/0.log" Jul 10 19:23:56.496258 ip-10-0-143-17 kubenswrapper[2585]: I0710 19:23:56.496207 2585 log.go:25] "Finished parsing log file" path="/var/log/pods/kserve-ci-e2e-test_custom-route-timeout-pd-test-kserve-d9768b96b-f4kzh_582d08e8-3203-452c-b6e5-885db8cace55/storage-initializer/0.log" Jul 10 19:23:56.516427 ip-10-0-143-17 kubenswrapper[2585]: I0710 19:23:56.516400 2585 log.go:25] "Finished parsing log file" path="/var/log/pods/kserve-ci-e2e-test_custom-route-timeout-pd-test-kserve-prefill-7d465b7985-qljdn_3b61c19c-497d-48a7-90c1-7d00ae4a0dc8/main/0.log" Jul 10 19:23:56.523349 ip-10-0-143-17 kubenswrapper[2585]: I0710 19:23:56.523330 2585 log.go:25] "Finished parsing log file" path="/var/log/pods/kserve-ci-e2e-test_custom-route-timeout-pd-test-kserve-prefill-7d465b7985-qljdn_3b61c19c-497d-48a7-90c1-7d00ae4a0dc8/storage-initializer/0.log" Jul 10 19:23:56.544791 ip-10-0-143-17 kubenswrapper[2585]: I0710 19:23:56.544771 2585 log.go:25] "Finished parsing log file" path="/var/log/pods/kserve-ci-e2e-test_router-gateway-1-openshift-default-799f46c59b-sf6qk_10d311d2-b0ff-486f-a709-ef0f52dfa327/istio-proxy/1.log" Jul 10 19:23:56.545229 ip-10-0-143-17 kubenswrapper[2585]: I0710 19:23:56.545211 2585 log.go:25] "Finished parsing log file" path="/var/log/pods/kserve-ci-e2e-test_router-gateway-1-openshift-default-799f46c59b-sf6qk_10d311d2-b0ff-486f-a709-ef0f52dfa327/istio-proxy/0.log" Jul 10 19:23:56.558678 ip-10-0-143-17 kubenswrapper[2585]: I0710 19:23:56.558655 2585 log.go:25] "Finished parsing log file" path="/var/log/pods/kserve-ci-e2e-test_router-gateway-2-openshift-default-54c789bdc6-66nfj_7aa2cf75-c55d-45ff-a4ca-9bfba3d0a016/istio-proxy/1.log" Jul 10 19:23:56.559301 ip-10-0-143-17 kubenswrapper[2585]: I0710 19:23:56.559278 2585 log.go:25] "Finished parsing log file" path="/var/log/pods/kserve-ci-e2e-test_router-gateway-2-openshift-default-54c789bdc6-66nfj_7aa2cf75-c55d-45ff-a4ca-9bfba3d0a016/istio-proxy/0.log" Jul 10 19:23:57.321009 ip-10-0-143-17 kubenswrapper[2585]: I0710 19:23:57.320967 2585 kuberuntime_container.go:873] "Killing container with a grace period" pod="kserve-ci-e2e-test/custom-route-timeout-pd-test-kserve-d9768b96b-f4kzh" podUID="582d08e8-3203-452c-b6e5-885db8cace55" containerName="llm-d-routing-sidecar" containerID="cri-o://e7960b1cffff7526408a5b7cfc5979d1c43339038fb0e841ac015ca61f37c619" gracePeriod=2 Jul 10 19:23:57.571237 ip-10-0-143-17 kubenswrapper[2585]: I0710 19:23:57.571143 2585 log.go:25] "Finished parsing log file" path="/var/log/pods/kserve-ci-e2e-test_custom-route-timeout-pd-test-kserve-d9768b96b-f4kzh_582d08e8-3203-452c-b6e5-885db8cace55/main/0.log" Jul 10 19:23:57.581345 ip-10-0-143-17 kubenswrapper[2585]: I0710 19:23:57.581318 2585 log.go:25] "Finished parsing log file" path="/var/log/pods/kserve-ci-e2e-test_custom-route-timeout-pd-test-kserve-d9768b96b-f4kzh_582d08e8-3203-452c-b6e5-885db8cace55/llm-d-routing-sidecar/0.log" Jul 10 19:23:57.597963 ip-10-0-143-17 kubenswrapper[2585]: I0710 19:23:57.597932 2585 log.go:25] "Finished parsing log file" path="/var/log/pods/kserve-ci-e2e-test_custom-route-timeout-pd-test-kserve-d9768b96b-f4kzh_582d08e8-3203-452c-b6e5-885db8cace55/storage-initializer/0.log" Jul 10 19:23:57.619941 ip-10-0-143-17 kubenswrapper[2585]: I0710 19:23:57.619909 2585 log.go:25] "Finished parsing log file" path="/var/log/pods/kserve-ci-e2e-test_custom-route-timeout-pd-test-kserve-prefill-7d465b7985-qljdn_3b61c19c-497d-48a7-90c1-7d00ae4a0dc8/main/0.log" Jul 10 19:23:57.637169 ip-10-0-143-17 kubenswrapper[2585]: I0710 19:23:57.637141 2585 log.go:25] "Finished parsing log file" path="/var/log/pods/kserve-ci-e2e-test_custom-route-timeout-pd-test-kserve-prefill-7d465b7985-qljdn_3b61c19c-497d-48a7-90c1-7d00ae4a0dc8/storage-initializer/0.log" Jul 10 19:23:57.657767 ip-10-0-143-17 kubenswrapper[2585]: I0710 19:23:57.657710 2585 log.go:25] "Finished parsing log file" path="/var/log/pods/kserve-ci-e2e-test_router-gateway-1-openshift-default-799f46c59b-sf6qk_10d311d2-b0ff-486f-a709-ef0f52dfa327/istio-proxy/1.log" Jul 10 19:23:57.658500 ip-10-0-143-17 kubenswrapper[2585]: I0710 19:23:57.658476 2585 log.go:25] "Finished parsing log file" path="/var/log/pods/kserve-ci-e2e-test_router-gateway-1-openshift-default-799f46c59b-sf6qk_10d311d2-b0ff-486f-a709-ef0f52dfa327/istio-proxy/0.log" Jul 10 19:23:57.674266 ip-10-0-143-17 kubenswrapper[2585]: I0710 19:23:57.674186 2585 log.go:25] "Finished parsing log file" path="/var/log/pods/kserve-ci-e2e-test_router-gateway-2-openshift-default-54c789bdc6-66nfj_7aa2cf75-c55d-45ff-a4ca-9bfba3d0a016/istio-proxy/1.log" Jul 10 19:23:57.674844 ip-10-0-143-17 kubenswrapper[2585]: I0710 19:23:57.674818 2585 log.go:25] "Finished parsing log file" path="/var/log/pods/kserve-ci-e2e-test_router-gateway-2-openshift-default-54c789bdc6-66nfj_7aa2cf75-c55d-45ff-a4ca-9bfba3d0a016/istio-proxy/0.log" Jul 10 19:23:57.718073 ip-10-0-143-17 kubenswrapper[2585]: I0710 19:23:57.718052 2585 log.go:25] "Finished parsing log file" path="/var/log/pods/kserve-ci-e2e-test_custom-route-timeout-pd-test-kserve-d9768b96b-f4kzh_582d08e8-3203-452c-b6e5-885db8cace55/main/0.log" Jul 10 19:23:57.720999 ip-10-0-143-17 kubenswrapper[2585]: I0710 19:23:57.720972 2585 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="kserve-ci-e2e-test/custom-route-timeout-pd-test-kserve-d9768b96b-f4kzh" Jul 10 19:23:57.723380 ip-10-0-143-17 kubenswrapper[2585]: I0710 19:23:57.723364 2585 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="kserve-ci-e2e-test/custom-route-timeout-pd-test-kserve-prefill-7d465b7985-qljdn" Jul 10 19:23:57.831978 ip-10-0-143-17 kubenswrapper[2585]: I0710 19:23:57.831904 2585 log.go:25] "Finished parsing log file" path="/var/log/pods/kserve-ci-e2e-test_custom-route-timeout-pd-test-kserve-d9768b96b-f4kzh_582d08e8-3203-452c-b6e5-885db8cace55/main/0.log" Jul 10 19:23:57.832514 ip-10-0-143-17 kubenswrapper[2585]: I0710 19:23:57.832492 2585 generic.go:356] "Generic (PLEG): container finished" podID="582d08e8-3203-452c-b6e5-885db8cace55" containerID="8c825b781828d8a226b5d9720d6a186b412e0949e4c0fd868c8fe042fb981456" exitCode=137 Jul 10 19:23:57.832514 ip-10-0-143-17 kubenswrapper[2585]: I0710 19:23:57.832513 2585 generic.go:356] "Generic (PLEG): container finished" podID="582d08e8-3203-452c-b6e5-885db8cace55" containerID="e7960b1cffff7526408a5b7cfc5979d1c43339038fb0e841ac015ca61f37c619" exitCode=0 Jul 10 19:23:57.832694 ip-10-0-143-17 kubenswrapper[2585]: I0710 19:23:57.832571 2585 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="kserve-ci-e2e-test/custom-route-timeout-pd-test-kserve-d9768b96b-f4kzh" Jul 10 19:23:57.832694 ip-10-0-143-17 kubenswrapper[2585]: I0710 19:23:57.832569 2585 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kserve-ci-e2e-test/custom-route-timeout-pd-test-kserve-d9768b96b-f4kzh" event={"ID":"582d08e8-3203-452c-b6e5-885db8cace55","Type":"ContainerDied","Data":"8c825b781828d8a226b5d9720d6a186b412e0949e4c0fd868c8fe042fb981456"} Jul 10 19:23:57.832694 ip-10-0-143-17 kubenswrapper[2585]: I0710 19:23:57.832692 2585 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kserve-ci-e2e-test/custom-route-timeout-pd-test-kserve-d9768b96b-f4kzh" event={"ID":"582d08e8-3203-452c-b6e5-885db8cace55","Type":"ContainerDied","Data":"e7960b1cffff7526408a5b7cfc5979d1c43339038fb0e841ac015ca61f37c619"} Jul 10 19:23:57.832849 ip-10-0-143-17 kubenswrapper[2585]: I0710 19:23:57.832716 2585 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kserve-ci-e2e-test/custom-route-timeout-pd-test-kserve-d9768b96b-f4kzh" event={"ID":"582d08e8-3203-452c-b6e5-885db8cace55","Type":"ContainerDied","Data":"103ad6bd9b1864fb6d096475302907b537e2383d0652866c701f9b52462054dc"} Jul 10 19:23:57.832849 ip-10-0-143-17 kubenswrapper[2585]: I0710 19:23:57.832747 2585 scope.go:117] "RemoveContainer" containerID="8c825b781828d8a226b5d9720d6a186b412e0949e4c0fd868c8fe042fb981456" Jul 10 19:23:57.834296 ip-10-0-143-17 kubenswrapper[2585]: I0710 19:23:57.834273 2585 generic.go:356] "Generic (PLEG): container finished" podID="3b61c19c-497d-48a7-90c1-7d00ae4a0dc8" containerID="507a501cffc9d609795dac967b4bbca1b8131ab63ecb579ab00a3df6048be181" exitCode=137 Jul 10 19:23:57.834421 ip-10-0-143-17 kubenswrapper[2585]: I0710 19:23:57.834314 2585 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="kserve-ci-e2e-test/custom-route-timeout-pd-test-kserve-prefill-7d465b7985-qljdn" Jul 10 19:23:57.834421 ip-10-0-143-17 kubenswrapper[2585]: I0710 19:23:57.834343 2585 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kserve-ci-e2e-test/custom-route-timeout-pd-test-kserve-prefill-7d465b7985-qljdn" event={"ID":"3b61c19c-497d-48a7-90c1-7d00ae4a0dc8","Type":"ContainerDied","Data":"507a501cffc9d609795dac967b4bbca1b8131ab63ecb579ab00a3df6048be181"} Jul 10 19:23:57.834421 ip-10-0-143-17 kubenswrapper[2585]: I0710 19:23:57.834382 2585 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kserve-ci-e2e-test/custom-route-timeout-pd-test-kserve-prefill-7d465b7985-qljdn" event={"ID":"3b61c19c-497d-48a7-90c1-7d00ae4a0dc8","Type":"ContainerDied","Data":"5eacd06e989ceb66497b4e48014589a79d126e2e9906e3b132c61c587678ac4c"} Jul 10 19:23:57.841808 ip-10-0-143-17 kubenswrapper[2585]: I0710 19:23:57.841786 2585 scope.go:117] "RemoveContainer" containerID="e44c1bc9173dd845c87652369d82105b0ebe040d67b9c3d6f5cb4f64905480e2" Jul 10 19:23:57.898300 ip-10-0-143-17 kubenswrapper[2585]: I0710 19:23:57.898266 2585 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kserve-provision-location\" (UniqueName: \"kubernetes.io/empty-dir/582d08e8-3203-452c-b6e5-885db8cace55-kserve-provision-location\") pod \"582d08e8-3203-452c-b6e5-885db8cace55\" (UID: \"582d08e8-3203-452c-b6e5-885db8cace55\") " Jul 10 19:23:57.898466 ip-10-0-143-17 kubenswrapper[2585]: I0710 19:23:57.898321 2585 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/3b61c19c-497d-48a7-90c1-7d00ae4a0dc8-model-cache\") pod \"3b61c19c-497d-48a7-90c1-7d00ae4a0dc8\" (UID: \"3b61c19c-497d-48a7-90c1-7d00ae4a0dc8\") " Jul 10 19:23:57.898466 ip-10-0-143-17 kubenswrapper[2585]: I0710 19:23:57.898359 2585 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/3b61c19c-497d-48a7-90c1-7d00ae4a0dc8-tls-certs\") pod \"3b61c19c-497d-48a7-90c1-7d00ae4a0dc8\" (UID: \"3b61c19c-497d-48a7-90c1-7d00ae4a0dc8\") " Jul 10 19:23:57.898466 ip-10-0-143-17 kubenswrapper[2585]: I0710 19:23:57.898420 2585 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/582d08e8-3203-452c-b6e5-885db8cace55-dshm\") pod \"582d08e8-3203-452c-b6e5-885db8cace55\" (UID: \"582d08e8-3203-452c-b6e5-885db8cace55\") " Jul 10 19:23:57.898466 ip-10-0-143-17 kubenswrapper[2585]: I0710 19:23:57.898435 2585 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/3b61c19c-497d-48a7-90c1-7d00ae4a0dc8-home\") pod \"3b61c19c-497d-48a7-90c1-7d00ae4a0dc8\" (UID: \"3b61c19c-497d-48a7-90c1-7d00ae4a0dc8\") " Jul 10 19:23:57.898718 ip-10-0-143-17 kubenswrapper[2585]: I0710 19:23:57.898475 2585 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/3b61c19c-497d-48a7-90c1-7d00ae4a0dc8-tmp-dir\") pod \"3b61c19c-497d-48a7-90c1-7d00ae4a0dc8\" (UID: \"3b61c19c-497d-48a7-90c1-7d00ae4a0dc8\") " Jul 10 19:23:57.898718 ip-10-0-143-17 kubenswrapper[2585]: I0710 19:23:57.898497 2585 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kserve-provision-location\" (UniqueName: \"kubernetes.io/empty-dir/3b61c19c-497d-48a7-90c1-7d00ae4a0dc8-kserve-provision-location\") pod \"3b61c19c-497d-48a7-90c1-7d00ae4a0dc8\" (UID: \"3b61c19c-497d-48a7-90c1-7d00ae4a0dc8\") " Jul 10 19:23:57.898718 ip-10-0-143-17 kubenswrapper[2585]: I0710 19:23:57.898527 2585 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-fwmvf\" (UniqueName: \"kubernetes.io/projected/3b61c19c-497d-48a7-90c1-7d00ae4a0dc8-kube-api-access-fwmvf\") pod \"3b61c19c-497d-48a7-90c1-7d00ae4a0dc8\" (UID: \"3b61c19c-497d-48a7-90c1-7d00ae4a0dc8\") " Jul 10 19:23:57.898718 ip-10-0-143-17 kubenswrapper[2585]: I0710 19:23:57.898582 2585 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/582d08e8-3203-452c-b6e5-885db8cace55-home\") pod \"582d08e8-3203-452c-b6e5-885db8cace55\" (UID: \"582d08e8-3203-452c-b6e5-885db8cace55\") " Jul 10 19:23:57.898718 ip-10-0-143-17 kubenswrapper[2585]: I0710 19:23:57.898579 2585 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/3b61c19c-497d-48a7-90c1-7d00ae4a0dc8-model-cache" (OuterVolumeSpecName: "model-cache") pod "3b61c19c-497d-48a7-90c1-7d00ae4a0dc8" (UID: "3b61c19c-497d-48a7-90c1-7d00ae4a0dc8"). InnerVolumeSpecName "model-cache". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Jul 10 19:23:57.898718 ip-10-0-143-17 kubenswrapper[2585]: I0710 19:23:57.898622 2585 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-zg99x\" (UniqueName: \"kubernetes.io/projected/582d08e8-3203-452c-b6e5-885db8cace55-kube-api-access-zg99x\") pod \"582d08e8-3203-452c-b6e5-885db8cace55\" (UID: \"582d08e8-3203-452c-b6e5-885db8cace55\") " Jul 10 19:23:57.898718 ip-10-0-143-17 kubenswrapper[2585]: I0710 19:23:57.898657 2585 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/582d08e8-3203-452c-b6e5-885db8cace55-tls-certs\") pod \"582d08e8-3203-452c-b6e5-885db8cace55\" (UID: \"582d08e8-3203-452c-b6e5-885db8cace55\") " Jul 10 19:23:57.898718 ip-10-0-143-17 kubenswrapper[2585]: I0710 19:23:57.898683 2585 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/3b61c19c-497d-48a7-90c1-7d00ae4a0dc8-dshm\") pod \"3b61c19c-497d-48a7-90c1-7d00ae4a0dc8\" (UID: \"3b61c19c-497d-48a7-90c1-7d00ae4a0dc8\") " Jul 10 19:23:57.898718 ip-10-0-143-17 kubenswrapper[2585]: I0710 19:23:57.898716 2585 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/582d08e8-3203-452c-b6e5-885db8cace55-model-cache\") pod \"582d08e8-3203-452c-b6e5-885db8cace55\" (UID: \"582d08e8-3203-452c-b6e5-885db8cace55\") " Jul 10 19:23:57.899173 ip-10-0-143-17 kubenswrapper[2585]: I0710 19:23:57.898740 2585 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/582d08e8-3203-452c-b6e5-885db8cace55-tmp-dir\") pod \"582d08e8-3203-452c-b6e5-885db8cace55\" (UID: \"582d08e8-3203-452c-b6e5-885db8cace55\") " Jul 10 19:23:57.899173 ip-10-0-143-17 kubenswrapper[2585]: I0710 19:23:57.899107 2585 reconciler_common.go:299] "Volume detached for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/3b61c19c-497d-48a7-90c1-7d00ae4a0dc8-model-cache\") on node \"ip-10-0-143-17.ec2.internal\" DevicePath \"\"" Jul 10 19:23:57.899692 ip-10-0-143-17 kubenswrapper[2585]: I0710 19:23:57.899666 2585 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/3b61c19c-497d-48a7-90c1-7d00ae4a0dc8-home" (OuterVolumeSpecName: "home") pod "3b61c19c-497d-48a7-90c1-7d00ae4a0dc8" (UID: "3b61c19c-497d-48a7-90c1-7d00ae4a0dc8"). InnerVolumeSpecName "home". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Jul 10 19:23:57.901469 ip-10-0-143-17 kubenswrapper[2585]: I0710 19:23:57.901442 2585 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/582d08e8-3203-452c-b6e5-885db8cace55-home" (OuterVolumeSpecName: "home") pod "582d08e8-3203-452c-b6e5-885db8cace55" (UID: "582d08e8-3203-452c-b6e5-885db8cace55"). InnerVolumeSpecName "home". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Jul 10 19:23:57.901469 ip-10-0-143-17 kubenswrapper[2585]: I0710 19:23:57.901460 2585 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/582d08e8-3203-452c-b6e5-885db8cace55-dshm" (OuterVolumeSpecName: "dshm") pod "582d08e8-3203-452c-b6e5-885db8cace55" (UID: "582d08e8-3203-452c-b6e5-885db8cace55"). InnerVolumeSpecName "dshm". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Jul 10 19:23:57.902246 ip-10-0-143-17 kubenswrapper[2585]: I0710 19:23:57.902221 2585 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/3b61c19c-497d-48a7-90c1-7d00ae4a0dc8-tls-certs" (OuterVolumeSpecName: "tls-certs") pod "3b61c19c-497d-48a7-90c1-7d00ae4a0dc8" (UID: "3b61c19c-497d-48a7-90c1-7d00ae4a0dc8"). InnerVolumeSpecName "tls-certs". PluginName "kubernetes.io/secret", VolumeGIDValue "" Jul 10 19:23:57.902333 ip-10-0-143-17 kubenswrapper[2585]: I0710 19:23:57.902295 2585 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/582d08e8-3203-452c-b6e5-885db8cace55-model-cache" (OuterVolumeSpecName: "model-cache") pod "582d08e8-3203-452c-b6e5-885db8cace55" (UID: "582d08e8-3203-452c-b6e5-885db8cace55"). InnerVolumeSpecName "model-cache". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Jul 10 19:23:57.902675 ip-10-0-143-17 kubenswrapper[2585]: I0710 19:23:57.902643 2585 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/3b61c19c-497d-48a7-90c1-7d00ae4a0dc8-kube-api-access-fwmvf" (OuterVolumeSpecName: "kube-api-access-fwmvf") pod "3b61c19c-497d-48a7-90c1-7d00ae4a0dc8" (UID: "3b61c19c-497d-48a7-90c1-7d00ae4a0dc8"). InnerVolumeSpecName "kube-api-access-fwmvf". PluginName "kubernetes.io/projected", VolumeGIDValue "" Jul 10 19:23:57.903368 ip-10-0-143-17 kubenswrapper[2585]: I0710 19:23:57.903339 2585 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/582d08e8-3203-452c-b6e5-885db8cace55-kube-api-access-zg99x" (OuterVolumeSpecName: "kube-api-access-zg99x") pod "582d08e8-3203-452c-b6e5-885db8cace55" (UID: "582d08e8-3203-452c-b6e5-885db8cace55"). InnerVolumeSpecName "kube-api-access-zg99x". PluginName "kubernetes.io/projected", VolumeGIDValue "" Jul 10 19:23:57.904554 ip-10-0-143-17 kubenswrapper[2585]: I0710 19:23:57.904515 2585 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/582d08e8-3203-452c-b6e5-885db8cace55-tls-certs" (OuterVolumeSpecName: "tls-certs") pod "582d08e8-3203-452c-b6e5-885db8cace55" (UID: "582d08e8-3203-452c-b6e5-885db8cace55"). InnerVolumeSpecName "tls-certs". PluginName "kubernetes.io/secret", VolumeGIDValue "" Jul 10 19:23:57.904786 ip-10-0-143-17 kubenswrapper[2585]: I0710 19:23:57.904766 2585 scope.go:117] "RemoveContainer" containerID="e7960b1cffff7526408a5b7cfc5979d1c43339038fb0e841ac015ca61f37c619" Jul 10 19:23:57.904894 ip-10-0-143-17 kubenswrapper[2585]: I0710 19:23:57.904868 2585 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/3b61c19c-497d-48a7-90c1-7d00ae4a0dc8-dshm" (OuterVolumeSpecName: "dshm") pod "3b61c19c-497d-48a7-90c1-7d00ae4a0dc8" (UID: "3b61c19c-497d-48a7-90c1-7d00ae4a0dc8"). InnerVolumeSpecName "dshm". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Jul 10 19:23:57.913377 ip-10-0-143-17 kubenswrapper[2585]: I0710 19:23:57.913358 2585 scope.go:117] "RemoveContainer" containerID="8c825b781828d8a226b5d9720d6a186b412e0949e4c0fd868c8fe042fb981456" Jul 10 19:23:57.913705 ip-10-0-143-17 kubenswrapper[2585]: E0710 19:23:57.913684 2585 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"8c825b781828d8a226b5d9720d6a186b412e0949e4c0fd868c8fe042fb981456\": container with ID starting with 8c825b781828d8a226b5d9720d6a186b412e0949e4c0fd868c8fe042fb981456 not found: ID does not exist" containerID="8c825b781828d8a226b5d9720d6a186b412e0949e4c0fd868c8fe042fb981456" Jul 10 19:23:57.913795 ip-10-0-143-17 kubenswrapper[2585]: I0710 19:23:57.913716 2585 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"8c825b781828d8a226b5d9720d6a186b412e0949e4c0fd868c8fe042fb981456"} err="failed to get container status \"8c825b781828d8a226b5d9720d6a186b412e0949e4c0fd868c8fe042fb981456\": rpc error: code = NotFound desc = could not find container \"8c825b781828d8a226b5d9720d6a186b412e0949e4c0fd868c8fe042fb981456\": container with ID starting with 8c825b781828d8a226b5d9720d6a186b412e0949e4c0fd868c8fe042fb981456 not found: ID does not exist" Jul 10 19:23:57.913795 ip-10-0-143-17 kubenswrapper[2585]: I0710 19:23:57.913736 2585 scope.go:117] "RemoveContainer" containerID="e44c1bc9173dd845c87652369d82105b0ebe040d67b9c3d6f5cb4f64905480e2" Jul 10 19:23:57.914041 ip-10-0-143-17 kubenswrapper[2585]: E0710 19:23:57.914024 2585 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"e44c1bc9173dd845c87652369d82105b0ebe040d67b9c3d6f5cb4f64905480e2\": container with ID starting with e44c1bc9173dd845c87652369d82105b0ebe040d67b9c3d6f5cb4f64905480e2 not found: ID does not exist" containerID="e44c1bc9173dd845c87652369d82105b0ebe040d67b9c3d6f5cb4f64905480e2" Jul 10 19:23:57.914091 ip-10-0-143-17 kubenswrapper[2585]: I0710 19:23:57.914047 2585 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"e44c1bc9173dd845c87652369d82105b0ebe040d67b9c3d6f5cb4f64905480e2"} err="failed to get container status \"e44c1bc9173dd845c87652369d82105b0ebe040d67b9c3d6f5cb4f64905480e2\": rpc error: code = NotFound desc = could not find container \"e44c1bc9173dd845c87652369d82105b0ebe040d67b9c3d6f5cb4f64905480e2\": container with ID starting with e44c1bc9173dd845c87652369d82105b0ebe040d67b9c3d6f5cb4f64905480e2 not found: ID does not exist" Jul 10 19:23:57.914091 ip-10-0-143-17 kubenswrapper[2585]: I0710 19:23:57.914069 2585 scope.go:117] "RemoveContainer" containerID="e7960b1cffff7526408a5b7cfc5979d1c43339038fb0e841ac015ca61f37c619" Jul 10 19:23:57.914303 ip-10-0-143-17 kubenswrapper[2585]: E0710 19:23:57.914287 2585 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"e7960b1cffff7526408a5b7cfc5979d1c43339038fb0e841ac015ca61f37c619\": container with ID starting with e7960b1cffff7526408a5b7cfc5979d1c43339038fb0e841ac015ca61f37c619 not found: ID does not exist" containerID="e7960b1cffff7526408a5b7cfc5979d1c43339038fb0e841ac015ca61f37c619" Jul 10 19:23:57.914378 ip-10-0-143-17 kubenswrapper[2585]: I0710 19:23:57.914308 2585 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"e7960b1cffff7526408a5b7cfc5979d1c43339038fb0e841ac015ca61f37c619"} err="failed to get container status \"e7960b1cffff7526408a5b7cfc5979d1c43339038fb0e841ac015ca61f37c619\": rpc error: code = NotFound desc = could not find container \"e7960b1cffff7526408a5b7cfc5979d1c43339038fb0e841ac015ca61f37c619\": container with ID starting with e7960b1cffff7526408a5b7cfc5979d1c43339038fb0e841ac015ca61f37c619 not found: ID does not exist" Jul 10 19:23:57.914378 ip-10-0-143-17 kubenswrapper[2585]: I0710 19:23:57.914322 2585 scope.go:117] "RemoveContainer" containerID="8c825b781828d8a226b5d9720d6a186b412e0949e4c0fd868c8fe042fb981456" Jul 10 19:23:57.914605 ip-10-0-143-17 kubenswrapper[2585]: I0710 19:23:57.914582 2585 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"8c825b781828d8a226b5d9720d6a186b412e0949e4c0fd868c8fe042fb981456"} err="failed to get container status \"8c825b781828d8a226b5d9720d6a186b412e0949e4c0fd868c8fe042fb981456\": rpc error: code = NotFound desc = could not find container \"8c825b781828d8a226b5d9720d6a186b412e0949e4c0fd868c8fe042fb981456\": container with ID starting with 8c825b781828d8a226b5d9720d6a186b412e0949e4c0fd868c8fe042fb981456 not found: ID does not exist" Jul 10 19:23:57.914690 ip-10-0-143-17 kubenswrapper[2585]: I0710 19:23:57.914614 2585 scope.go:117] "RemoveContainer" containerID="e44c1bc9173dd845c87652369d82105b0ebe040d67b9c3d6f5cb4f64905480e2" Jul 10 19:23:57.914874 ip-10-0-143-17 kubenswrapper[2585]: I0710 19:23:57.914855 2585 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"e44c1bc9173dd845c87652369d82105b0ebe040d67b9c3d6f5cb4f64905480e2"} err="failed to get container status \"e44c1bc9173dd845c87652369d82105b0ebe040d67b9c3d6f5cb4f64905480e2\": rpc error: code = NotFound desc = could not find container \"e44c1bc9173dd845c87652369d82105b0ebe040d67b9c3d6f5cb4f64905480e2\": container with ID starting with e44c1bc9173dd845c87652369d82105b0ebe040d67b9c3d6f5cb4f64905480e2 not found: ID does not exist" Jul 10 19:23:57.914961 ip-10-0-143-17 kubenswrapper[2585]: I0710 19:23:57.914886 2585 scope.go:117] "RemoveContainer" containerID="e7960b1cffff7526408a5b7cfc5979d1c43339038fb0e841ac015ca61f37c619" Jul 10 19:23:57.915121 ip-10-0-143-17 kubenswrapper[2585]: I0710 19:23:57.915102 2585 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"e7960b1cffff7526408a5b7cfc5979d1c43339038fb0e841ac015ca61f37c619"} err="failed to get container status \"e7960b1cffff7526408a5b7cfc5979d1c43339038fb0e841ac015ca61f37c619\": rpc error: code = NotFound desc = could not find container \"e7960b1cffff7526408a5b7cfc5979d1c43339038fb0e841ac015ca61f37c619\": container with ID starting with e7960b1cffff7526408a5b7cfc5979d1c43339038fb0e841ac015ca61f37c619 not found: ID does not exist" Jul 10 19:23:57.915189 ip-10-0-143-17 kubenswrapper[2585]: I0710 19:23:57.915122 2585 scope.go:117] "RemoveContainer" containerID="507a501cffc9d609795dac967b4bbca1b8131ab63ecb579ab00a3df6048be181" Jul 10 19:23:57.920129 ip-10-0-143-17 kubenswrapper[2585]: I0710 19:23:57.920103 2585 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/582d08e8-3203-452c-b6e5-885db8cace55-tmp-dir" (OuterVolumeSpecName: "tmp-dir") pod "582d08e8-3203-452c-b6e5-885db8cace55" (UID: "582d08e8-3203-452c-b6e5-885db8cace55"). InnerVolumeSpecName "tmp-dir". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Jul 10 19:23:57.920479 ip-10-0-143-17 kubenswrapper[2585]: I0710 19:23:57.920456 2585 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/3b61c19c-497d-48a7-90c1-7d00ae4a0dc8-tmp-dir" (OuterVolumeSpecName: "tmp-dir") pod "3b61c19c-497d-48a7-90c1-7d00ae4a0dc8" (UID: "3b61c19c-497d-48a7-90c1-7d00ae4a0dc8"). InnerVolumeSpecName "tmp-dir". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Jul 10 19:23:57.923884 ip-10-0-143-17 kubenswrapper[2585]: I0710 19:23:57.923870 2585 scope.go:117] "RemoveContainer" containerID="11b8ee6adc85061be0807e411350c314de5428f6808641a01c46043c0ae11b19" Jul 10 19:23:57.982662 ip-10-0-143-17 kubenswrapper[2585]: I0710 19:23:57.982615 2585 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/582d08e8-3203-452c-b6e5-885db8cace55-kserve-provision-location" (OuterVolumeSpecName: "kserve-provision-location") pod "582d08e8-3203-452c-b6e5-885db8cace55" (UID: "582d08e8-3203-452c-b6e5-885db8cace55"). InnerVolumeSpecName "kserve-provision-location". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Jul 10 19:23:57.983726 ip-10-0-143-17 kubenswrapper[2585]: I0710 19:23:57.983695 2585 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/3b61c19c-497d-48a7-90c1-7d00ae4a0dc8-kserve-provision-location" (OuterVolumeSpecName: "kserve-provision-location") pod "3b61c19c-497d-48a7-90c1-7d00ae4a0dc8" (UID: "3b61c19c-497d-48a7-90c1-7d00ae4a0dc8"). InnerVolumeSpecName "kserve-provision-location". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Jul 10 19:23:57.995251 ip-10-0-143-17 kubenswrapper[2585]: I0710 19:23:57.995221 2585 scope.go:117] "RemoveContainer" containerID="507a501cffc9d609795dac967b4bbca1b8131ab63ecb579ab00a3df6048be181" Jul 10 19:23:57.995611 ip-10-0-143-17 kubenswrapper[2585]: E0710 19:23:57.995587 2585 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"507a501cffc9d609795dac967b4bbca1b8131ab63ecb579ab00a3df6048be181\": container with ID starting with 507a501cffc9d609795dac967b4bbca1b8131ab63ecb579ab00a3df6048be181 not found: ID does not exist" containerID="507a501cffc9d609795dac967b4bbca1b8131ab63ecb579ab00a3df6048be181" Jul 10 19:23:57.995689 ip-10-0-143-17 kubenswrapper[2585]: I0710 19:23:57.995624 2585 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"507a501cffc9d609795dac967b4bbca1b8131ab63ecb579ab00a3df6048be181"} err="failed to get container status \"507a501cffc9d609795dac967b4bbca1b8131ab63ecb579ab00a3df6048be181\": rpc error: code = NotFound desc = could not find container \"507a501cffc9d609795dac967b4bbca1b8131ab63ecb579ab00a3df6048be181\": container with ID starting with 507a501cffc9d609795dac967b4bbca1b8131ab63ecb579ab00a3df6048be181 not found: ID does not exist" Jul 10 19:23:57.995689 ip-10-0-143-17 kubenswrapper[2585]: I0710 19:23:57.995650 2585 scope.go:117] "RemoveContainer" containerID="11b8ee6adc85061be0807e411350c314de5428f6808641a01c46043c0ae11b19" Jul 10 19:23:57.995942 ip-10-0-143-17 kubenswrapper[2585]: E0710 19:23:57.995924 2585 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"11b8ee6adc85061be0807e411350c314de5428f6808641a01c46043c0ae11b19\": container with ID starting with 11b8ee6adc85061be0807e411350c314de5428f6808641a01c46043c0ae11b19 not found: ID does not exist" containerID="11b8ee6adc85061be0807e411350c314de5428f6808641a01c46043c0ae11b19" Jul 10 19:23:57.995979 ip-10-0-143-17 kubenswrapper[2585]: I0710 19:23:57.995948 2585 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"11b8ee6adc85061be0807e411350c314de5428f6808641a01c46043c0ae11b19"} err="failed to get container status \"11b8ee6adc85061be0807e411350c314de5428f6808641a01c46043c0ae11b19\": rpc error: code = NotFound desc = could not find container \"11b8ee6adc85061be0807e411350c314de5428f6808641a01c46043c0ae11b19\": container with ID starting with 11b8ee6adc85061be0807e411350c314de5428f6808641a01c46043c0ae11b19 not found: ID does not exist" Jul 10 19:23:58.000447 ip-10-0-143-17 kubenswrapper[2585]: I0710 19:23:58.000431 2585 reconciler_common.go:299] "Volume detached for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/3b61c19c-497d-48a7-90c1-7d00ae4a0dc8-tmp-dir\") on node \"ip-10-0-143-17.ec2.internal\" DevicePath \"\"" Jul 10 19:23:58.000506 ip-10-0-143-17 kubenswrapper[2585]: I0710 19:23:58.000449 2585 reconciler_common.go:299] "Volume detached for volume \"kserve-provision-location\" (UniqueName: \"kubernetes.io/empty-dir/3b61c19c-497d-48a7-90c1-7d00ae4a0dc8-kserve-provision-location\") on node \"ip-10-0-143-17.ec2.internal\" DevicePath \"\"" Jul 10 19:23:58.000506 ip-10-0-143-17 kubenswrapper[2585]: I0710 19:23:58.000460 2585 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-fwmvf\" (UniqueName: \"kubernetes.io/projected/3b61c19c-497d-48a7-90c1-7d00ae4a0dc8-kube-api-access-fwmvf\") on node \"ip-10-0-143-17.ec2.internal\" DevicePath \"\"" Jul 10 19:23:58.000506 ip-10-0-143-17 kubenswrapper[2585]: I0710 19:23:58.000468 2585 reconciler_common.go:299] "Volume detached for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/582d08e8-3203-452c-b6e5-885db8cace55-home\") on node \"ip-10-0-143-17.ec2.internal\" DevicePath \"\"" Jul 10 19:23:58.000506 ip-10-0-143-17 kubenswrapper[2585]: I0710 19:23:58.000477 2585 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-zg99x\" (UniqueName: \"kubernetes.io/projected/582d08e8-3203-452c-b6e5-885db8cace55-kube-api-access-zg99x\") on node \"ip-10-0-143-17.ec2.internal\" DevicePath \"\"" Jul 10 19:23:58.000506 ip-10-0-143-17 kubenswrapper[2585]: I0710 19:23:58.000485 2585 reconciler_common.go:299] "Volume detached for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/582d08e8-3203-452c-b6e5-885db8cace55-tls-certs\") on node \"ip-10-0-143-17.ec2.internal\" DevicePath \"\"" Jul 10 19:23:58.000506 ip-10-0-143-17 kubenswrapper[2585]: I0710 19:23:58.000492 2585 reconciler_common.go:299] "Volume detached for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/3b61c19c-497d-48a7-90c1-7d00ae4a0dc8-dshm\") on node \"ip-10-0-143-17.ec2.internal\" DevicePath \"\"" Jul 10 19:23:58.000506 ip-10-0-143-17 kubenswrapper[2585]: I0710 19:23:58.000499 2585 reconciler_common.go:299] "Volume detached for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/582d08e8-3203-452c-b6e5-885db8cace55-model-cache\") on node \"ip-10-0-143-17.ec2.internal\" DevicePath \"\"" Jul 10 19:23:58.000506 ip-10-0-143-17 kubenswrapper[2585]: I0710 19:23:58.000506 2585 reconciler_common.go:299] "Volume detached for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/582d08e8-3203-452c-b6e5-885db8cace55-tmp-dir\") on node \"ip-10-0-143-17.ec2.internal\" DevicePath \"\"" Jul 10 19:23:58.000844 ip-10-0-143-17 kubenswrapper[2585]: I0710 19:23:58.000513 2585 reconciler_common.go:299] "Volume detached for volume \"kserve-provision-location\" (UniqueName: \"kubernetes.io/empty-dir/582d08e8-3203-452c-b6e5-885db8cace55-kserve-provision-location\") on node \"ip-10-0-143-17.ec2.internal\" DevicePath \"\"" Jul 10 19:23:58.000844 ip-10-0-143-17 kubenswrapper[2585]: I0710 19:23:58.000521 2585 reconciler_common.go:299] "Volume detached for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/3b61c19c-497d-48a7-90c1-7d00ae4a0dc8-tls-certs\") on node \"ip-10-0-143-17.ec2.internal\" DevicePath \"\"" Jul 10 19:23:58.000844 ip-10-0-143-17 kubenswrapper[2585]: I0710 19:23:58.000528 2585 reconciler_common.go:299] "Volume detached for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/582d08e8-3203-452c-b6e5-885db8cace55-dshm\") on node \"ip-10-0-143-17.ec2.internal\" DevicePath \"\"" Jul 10 19:23:58.000844 ip-10-0-143-17 kubenswrapper[2585]: I0710 19:23:58.000556 2585 reconciler_common.go:299] "Volume detached for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/3b61c19c-497d-48a7-90c1-7d00ae4a0dc8-home\") on node \"ip-10-0-143-17.ec2.internal\" DevicePath \"\"" Jul 10 19:23:58.162763 ip-10-0-143-17 kubenswrapper[2585]: I0710 19:23:58.162737 2585 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["kserve-ci-e2e-test/custom-route-timeout-pd-test-kserve-prefill-7d465b7985-qljdn"] Jul 10 19:23:58.166869 ip-10-0-143-17 kubenswrapper[2585]: I0710 19:23:58.166848 2585 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["kserve-ci-e2e-test/custom-route-timeout-pd-test-kserve-prefill-7d465b7985-qljdn"] Jul 10 19:23:58.181106 ip-10-0-143-17 kubenswrapper[2585]: I0710 19:23:58.181079 2585 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["kserve-ci-e2e-test/custom-route-timeout-pd-test-kserve-d9768b96b-f4kzh"] Jul 10 19:23:58.184663 ip-10-0-143-17 kubenswrapper[2585]: I0710 19:23:58.184643 2585 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["kserve-ci-e2e-test/custom-route-timeout-pd-test-kserve-d9768b96b-f4kzh"] Jul 10 19:23:58.659572 ip-10-0-143-17 kubenswrapper[2585]: I0710 19:23:58.659528 2585 log.go:25] "Finished parsing log file" path="/var/log/pods/kserve-ci-e2e-test_router-gateway-1-openshift-default-799f46c59b-sf6qk_10d311d2-b0ff-486f-a709-ef0f52dfa327/istio-proxy/1.log" Jul 10 19:23:58.660019 ip-10-0-143-17 kubenswrapper[2585]: I0710 19:23:58.659986 2585 log.go:25] "Finished parsing log file" path="/var/log/pods/kserve-ci-e2e-test_router-gateway-1-openshift-default-799f46c59b-sf6qk_10d311d2-b0ff-486f-a709-ef0f52dfa327/istio-proxy/0.log" Jul 10 19:23:58.673294 ip-10-0-143-17 kubenswrapper[2585]: I0710 19:23:58.673267 2585 log.go:25] "Finished parsing log file" path="/var/log/pods/kserve-ci-e2e-test_router-gateway-2-openshift-default-54c789bdc6-66nfj_7aa2cf75-c55d-45ff-a4ca-9bfba3d0a016/istio-proxy/1.log" Jul 10 19:23:58.673825 ip-10-0-143-17 kubenswrapper[2585]: I0710 19:23:58.673804 2585 log.go:25] "Finished parsing log file" path="/var/log/pods/kserve-ci-e2e-test_router-gateway-2-openshift-default-54c789bdc6-66nfj_7aa2cf75-c55d-45ff-a4ca-9bfba3d0a016/istio-proxy/0.log" Jul 10 19:23:59.221439 ip-10-0-143-17 kubenswrapper[2585]: I0710 19:23:59.221407 2585 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="3b61c19c-497d-48a7-90c1-7d00ae4a0dc8" path="/var/lib/kubelet/pods/3b61c19c-497d-48a7-90c1-7d00ae4a0dc8/volumes" Jul 10 19:23:59.221874 ip-10-0-143-17 kubenswrapper[2585]: I0710 19:23:59.221858 2585 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="582d08e8-3203-452c-b6e5-885db8cace55" path="/var/lib/kubelet/pods/582d08e8-3203-452c-b6e5-885db8cace55/volumes" Jul 10 19:23:59.610428 ip-10-0-143-17 kubenswrapper[2585]: I0710 19:23:59.610333 2585 log.go:25] "Finished parsing log file" path="/var/log/pods/kserve-ci-e2e-test_router-gateway-1-openshift-default-799f46c59b-sf6qk_10d311d2-b0ff-486f-a709-ef0f52dfa327/istio-proxy/1.log" Jul 10 19:23:59.611101 ip-10-0-143-17 kubenswrapper[2585]: I0710 19:23:59.611072 2585 log.go:25] "Finished parsing log file" path="/var/log/pods/kserve-ci-e2e-test_router-gateway-1-openshift-default-799f46c59b-sf6qk_10d311d2-b0ff-486f-a709-ef0f52dfa327/istio-proxy/0.log" Jul 10 19:23:59.626767 ip-10-0-143-17 kubenswrapper[2585]: I0710 19:23:59.626735 2585 log.go:25] "Finished parsing log file" path="/var/log/pods/kserve-ci-e2e-test_router-gateway-2-openshift-default-54c789bdc6-66nfj_7aa2cf75-c55d-45ff-a4ca-9bfba3d0a016/istio-proxy/1.log" Jul 10 19:23:59.627377 ip-10-0-143-17 kubenswrapper[2585]: I0710 19:23:59.627350 2585 log.go:25] "Finished parsing log file" path="/var/log/pods/kserve-ci-e2e-test_router-gateway-2-openshift-default-54c789bdc6-66nfj_7aa2cf75-c55d-45ff-a4ca-9bfba3d0a016/istio-proxy/0.log" Jul 10 19:24:00.662335 ip-10-0-143-17 kubenswrapper[2585]: I0710 19:24:00.662305 2585 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ingress_istiod-openshift-gateway-94bb8fbfd-6h99s_c2545730-3e12-4efe-b88c-6300fc5510a5/discovery/0.log" Jul 10 19:24:00.693844 ip-10-0-143-17 kubenswrapper[2585]: I0710 19:24:00.693806 2585 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ingress_openshift-ai-inference-openshift-default-6c48cf8bcc-cl5g8_27c2089c-3b42-41ba-91c3-c6f72e532db5/istio-proxy/0.log" Jul 10 19:24:00.694440 ip-10-0-143-17 kubenswrapper[2585]: I0710 19:24:00.694424 2585 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ingress_openshift-ai-inference-openshift-default-6c48cf8bcc-cl5g8_27c2089c-3b42-41ba-91c3-c6f72e532db5/istio-proxy/1.log" Jul 10 19:24:00.722182 ip-10-0-143-17 kubenswrapper[2585]: I0710 19:24:00.722130 2585 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ingress_router-default-5d998f6f9d-t8frx_bf853b15-a7e2-4f66-ace3-b1f1fc9323de/router/0.log" Jul 10 19:24:01.663723 ip-10-0-143-17 kubenswrapper[2585]: I0710 19:24:01.663689 2585 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ingress_istiod-openshift-gateway-94bb8fbfd-6h99s_c2545730-3e12-4efe-b88c-6300fc5510a5/discovery/0.log" Jul 10 19:24:01.681783 ip-10-0-143-17 kubenswrapper[2585]: I0710 19:24:01.681757 2585 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ingress_openshift-ai-inference-openshift-default-6c48cf8bcc-cl5g8_27c2089c-3b42-41ba-91c3-c6f72e532db5/istio-proxy/0.log" Jul 10 19:24:01.682341 ip-10-0-143-17 kubenswrapper[2585]: I0710 19:24:01.682313 2585 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ingress_openshift-ai-inference-openshift-default-6c48cf8bcc-cl5g8_27c2089c-3b42-41ba-91c3-c6f72e532db5/istio-proxy/1.log" Jul 10 19:24:01.701844 ip-10-0-143-17 kubenswrapper[2585]: I0710 19:24:01.701821 2585 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ingress_router-default-5d998f6f9d-t8frx_bf853b15-a7e2-4f66-ace3-b1f1fc9323de/router/0.log" Jul 10 19:24:02.568854 ip-10-0-143-17 kubenswrapper[2585]: I0710 19:24:02.568829 2585 log.go:25] "Finished parsing log file" path="/var/log/pods/kuadrant-system_authorino-operator-6d85f6564-hfksn_b21d8c45-c55b-4c6b-b36b-df56774a6183/manager/0.log" Jul 10 19:24:02.767117 ip-10-0-143-17 kubenswrapper[2585]: I0710 19:24:02.767088 2585 log.go:25] "Finished parsing log file" path="/var/log/pods/kuadrant-system_limitador-limitador-69574b596d-x7mgm_2db6ff8a-77c3-4aea-a4db-057b68a32265/limitador/0.log" Jul 10 19:24:08.203353 ip-10-0-143-17 kubenswrapper[2585]: I0710 19:24:08.203314 2585 log.go:25] "Finished parsing log file" path="/var/log/pods/kube-system_global-pull-secret-syncer-hhpnn_15a0ea9c-d294-4065-aed5-a182a76a7d7f/global-pull-secret-syncer/0.log" Jul 10 19:24:08.385028 ip-10-0-143-17 kubenswrapper[2585]: I0710 19:24:08.384998 2585 log.go:25] "Finished parsing log file" path="/var/log/pods/kube-system_konnectivity-agent-x7zf6_11f2f17b-9176-4b2b-b4a4-836025deed92/konnectivity-agent/0.log" Jul 10 19:24:08.451583 ip-10-0-143-17 kubenswrapper[2585]: I0710 19:24:08.451532 2585 log.go:25] "Finished parsing log file" path="/var/log/pods/kube-system_kube-apiserver-proxy-ip-10-0-143-17.ec2.internal_23dbc229ee3e12019dd90879f1a0df34/haproxy/0.log" Jul 10 19:24:13.514704 ip-10-0-143-17 kubenswrapper[2585]: I0710 19:24:13.514670 2585 log.go:25] "Finished parsing log file" path="/var/log/pods/kuadrant-system_authorino-operator-6d85f6564-hfksn_b21d8c45-c55b-4c6b-b36b-df56774a6183/manager/0.log" Jul 10 19:24:13.749078 ip-10-0-143-17 kubenswrapper[2585]: I0710 19:24:13.749049 2585 log.go:25] "Finished parsing log file" path="/var/log/pods/kuadrant-system_limitador-limitador-69574b596d-x7mgm_2db6ff8a-77c3-4aea-a4db-057b68a32265/limitador/0.log" Jul 10 19:24:15.148432 ip-10-0-143-17 kubenswrapper[2585]: I0710 19:24:15.148402 2585 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_cluster-monitoring-operator-5f7f494dd4-q52h2_45e87367-8dde-4c37-b642-f38ead4d2b04/cluster-monitoring-operator/0.log" Jul 10 19:24:15.286623 ip-10-0-143-17 kubenswrapper[2585]: I0710 19:24:15.286594 2585 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_metrics-server-688c68676d-bdllx_b44dd6d6-9d73-4626-bfe4-04cf7215ebed/metrics-server/0.log" Jul 10 19:24:15.536566 ip-10-0-143-17 kubenswrapper[2585]: I0710 19:24:15.536525 2585 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_node-exporter-st6xg_61826ce1-4adb-4270-ae29-d777b005c6d0/node-exporter/0.log" Jul 10 19:24:15.559938 ip-10-0-143-17 kubenswrapper[2585]: I0710 19:24:15.559914 2585 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_node-exporter-st6xg_61826ce1-4adb-4270-ae29-d777b005c6d0/kube-rbac-proxy/0.log" Jul 10 19:24:15.581358 ip-10-0-143-17 kubenswrapper[2585]: I0710 19:24:15.581341 2585 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_node-exporter-st6xg_61826ce1-4adb-4270-ae29-d777b005c6d0/init-textfile/0.log" Jul 10 19:24:15.942848 ip-10-0-143-17 kubenswrapper[2585]: I0710 19:24:15.942761 2585 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_prometheus-operator-64785b6675-vcwsb_34730388-7ab6-4c38-a961-a7b2ca5f2b98/prometheus-operator/0.log" Jul 10 19:24:15.970293 ip-10-0-143-17 kubenswrapper[2585]: I0710 19:24:15.970263 2585 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_prometheus-operator-64785b6675-vcwsb_34730388-7ab6-4c38-a961-a7b2ca5f2b98/kube-rbac-proxy/0.log" Jul 10 19:24:16.001574 ip-10-0-143-17 kubenswrapper[2585]: I0710 19:24:16.001527 2585 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_prometheus-operator-admission-webhook-77d98bc786-287c6_062015e4-0bbf-4327-ab65-bd64b305da35/prometheus-operator-admission-webhook/0.log" Jul 10 19:24:16.038550 ip-10-0-143-17 kubenswrapper[2585]: I0710 19:24:16.038520 2585 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_telemeter-client-7449dcbd6f-vvl9q_68290b3e-a6e8-42c4-aab9-9147abee2039/telemeter-client/0.log" Jul 10 19:24:16.064447 ip-10-0-143-17 kubenswrapper[2585]: I0710 19:24:16.064424 2585 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_telemeter-client-7449dcbd6f-vvl9q_68290b3e-a6e8-42c4-aab9-9147abee2039/reload/0.log" Jul 10 19:24:16.091495 ip-10-0-143-17 kubenswrapper[2585]: I0710 19:24:16.091477 2585 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_telemeter-client-7449dcbd6f-vvl9q_68290b3e-a6e8-42c4-aab9-9147abee2039/kube-rbac-proxy/0.log" Jul 10 19:24:17.777910 ip-10-0-143-17 kubenswrapper[2585]: I0710 19:24:17.777877 2585 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-must-gather-l54pc/perf-node-gather-daemonset-wm84g"] Jul 10 19:24:17.778349 ip-10-0-143-17 kubenswrapper[2585]: I0710 19:24:17.778331 2585 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="59ed214a-e61e-4dff-9811-87c0c02e39c5" containerName="main" Jul 10 19:24:17.778406 ip-10-0-143-17 kubenswrapper[2585]: I0710 19:24:17.778351 2585 state_mem.go:107] "Deleted CPUSet assignment" podUID="59ed214a-e61e-4dff-9811-87c0c02e39c5" containerName="main" Jul 10 19:24:17.778406 ip-10-0-143-17 kubenswrapper[2585]: I0710 19:24:17.778363 2585 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="3b61c19c-497d-48a7-90c1-7d00ae4a0dc8" containerName="main" Jul 10 19:24:17.778406 ip-10-0-143-17 kubenswrapper[2585]: I0710 19:24:17.778369 2585 state_mem.go:107] "Deleted CPUSet assignment" podUID="3b61c19c-497d-48a7-90c1-7d00ae4a0dc8" containerName="main" Jul 10 19:24:17.778406 ip-10-0-143-17 kubenswrapper[2585]: I0710 19:24:17.778376 2585 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="582d08e8-3203-452c-b6e5-885db8cace55" containerName="llm-d-routing-sidecar" Jul 10 19:24:17.778406 ip-10-0-143-17 kubenswrapper[2585]: I0710 19:24:17.778383 2585 state_mem.go:107] "Deleted CPUSet assignment" podUID="582d08e8-3203-452c-b6e5-885db8cace55" containerName="llm-d-routing-sidecar" Jul 10 19:24:17.778406 ip-10-0-143-17 kubenswrapper[2585]: I0710 19:24:17.778394 2585 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="582d08e8-3203-452c-b6e5-885db8cace55" containerName="storage-initializer" Jul 10 19:24:17.778406 ip-10-0-143-17 kubenswrapper[2585]: I0710 19:24:17.778400 2585 state_mem.go:107] "Deleted CPUSet assignment" podUID="582d08e8-3203-452c-b6e5-885db8cace55" containerName="storage-initializer" Jul 10 19:24:17.778740 ip-10-0-143-17 kubenswrapper[2585]: I0710 19:24:17.778416 2585 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="59ed214a-e61e-4dff-9811-87c0c02e39c5" containerName="storage-initializer" Jul 10 19:24:17.778740 ip-10-0-143-17 kubenswrapper[2585]: I0710 19:24:17.778421 2585 state_mem.go:107] "Deleted CPUSet assignment" podUID="59ed214a-e61e-4dff-9811-87c0c02e39c5" containerName="storage-initializer" Jul 10 19:24:17.778740 ip-10-0-143-17 kubenswrapper[2585]: I0710 19:24:17.778435 2585 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="582d08e8-3203-452c-b6e5-885db8cace55" containerName="main" Jul 10 19:24:17.778740 ip-10-0-143-17 kubenswrapper[2585]: I0710 19:24:17.778443 2585 state_mem.go:107] "Deleted CPUSet assignment" podUID="582d08e8-3203-452c-b6e5-885db8cace55" containerName="main" Jul 10 19:24:17.778740 ip-10-0-143-17 kubenswrapper[2585]: I0710 19:24:17.778451 2585 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="3b61c19c-497d-48a7-90c1-7d00ae4a0dc8" containerName="storage-initializer" Jul 10 19:24:17.778740 ip-10-0-143-17 kubenswrapper[2585]: I0710 19:24:17.778456 2585 state_mem.go:107] "Deleted CPUSet assignment" podUID="3b61c19c-497d-48a7-90c1-7d00ae4a0dc8" containerName="storage-initializer" Jul 10 19:24:17.778740 ip-10-0-143-17 kubenswrapper[2585]: I0710 19:24:17.778510 2585 memory_manager.go:368] "RemoveStaleState removing state" podUID="3b61c19c-497d-48a7-90c1-7d00ae4a0dc8" containerName="main" Jul 10 19:24:17.778740 ip-10-0-143-17 kubenswrapper[2585]: I0710 19:24:17.778517 2585 memory_manager.go:368] "RemoveStaleState removing state" podUID="59ed214a-e61e-4dff-9811-87c0c02e39c5" containerName="main" Jul 10 19:24:17.778740 ip-10-0-143-17 kubenswrapper[2585]: I0710 19:24:17.778526 2585 memory_manager.go:368] "RemoveStaleState removing state" podUID="582d08e8-3203-452c-b6e5-885db8cace55" containerName="main" Jul 10 19:24:17.778740 ip-10-0-143-17 kubenswrapper[2585]: I0710 19:24:17.778552 2585 memory_manager.go:368] "RemoveStaleState removing state" podUID="582d08e8-3203-452c-b6e5-885db8cace55" containerName="llm-d-routing-sidecar" Jul 10 19:24:17.781961 ip-10-0-143-17 kubenswrapper[2585]: I0710 19:24:17.781940 2585 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-must-gather-l54pc/perf-node-gather-daemonset-wm84g" Jul 10 19:24:17.788162 ip-10-0-143-17 kubenswrapper[2585]: I0710 19:24:17.788140 2585 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-must-gather-l54pc\"/\"kube-root-ca.crt\"" Jul 10 19:24:17.788252 ip-10-0-143-17 kubenswrapper[2585]: I0710 19:24:17.788208 2585 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-must-gather-l54pc\"/\"openshift-service-ca.crt\"" Jul 10 19:24:17.789325 ip-10-0-143-17 kubenswrapper[2585]: I0710 19:24:17.789310 2585 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-must-gather-l54pc\"/\"default-dockercfg-zmwvt\"" Jul 10 19:24:17.796019 ip-10-0-143-17 kubenswrapper[2585]: I0710 19:24:17.795998 2585 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-must-gather-l54pc/perf-node-gather-daemonset-wm84g"] Jul 10 19:24:17.881687 ip-10-0-143-17 kubenswrapper[2585]: I0710 19:24:17.881653 2585 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"sys\" (UniqueName: \"kubernetes.io/host-path/2e76f14d-dc17-4304-9c72-71119887aec1-sys\") pod \"perf-node-gather-daemonset-wm84g\" (UID: \"2e76f14d-dc17-4304-9c72-71119887aec1\") " pod="openshift-must-gather-l54pc/perf-node-gather-daemonset-wm84g" Jul 10 19:24:17.881847 ip-10-0-143-17 kubenswrapper[2585]: I0710 19:24:17.881694 2585 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"podres\" (UniqueName: \"kubernetes.io/host-path/2e76f14d-dc17-4304-9c72-71119887aec1-podres\") pod \"perf-node-gather-daemonset-wm84g\" (UID: \"2e76f14d-dc17-4304-9c72-71119887aec1\") " pod="openshift-must-gather-l54pc/perf-node-gather-daemonset-wm84g" Jul 10 19:24:17.881847 ip-10-0-143-17 kubenswrapper[2585]: I0710 19:24:17.881732 2585 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"lib-modules\" (UniqueName: \"kubernetes.io/host-path/2e76f14d-dc17-4304-9c72-71119887aec1-lib-modules\") pod \"perf-node-gather-daemonset-wm84g\" (UID: \"2e76f14d-dc17-4304-9c72-71119887aec1\") " pod="openshift-must-gather-l54pc/perf-node-gather-daemonset-wm84g" Jul 10 19:24:17.881847 ip-10-0-143-17 kubenswrapper[2585]: I0710 19:24:17.881761 2585 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-qnbrn\" (UniqueName: \"kubernetes.io/projected/2e76f14d-dc17-4304-9c72-71119887aec1-kube-api-access-qnbrn\") pod \"perf-node-gather-daemonset-wm84g\" (UID: \"2e76f14d-dc17-4304-9c72-71119887aec1\") " pod="openshift-must-gather-l54pc/perf-node-gather-daemonset-wm84g" Jul 10 19:24:17.881847 ip-10-0-143-17 kubenswrapper[2585]: I0710 19:24:17.881784 2585 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"proc\" (UniqueName: \"kubernetes.io/host-path/2e76f14d-dc17-4304-9c72-71119887aec1-proc\") pod \"perf-node-gather-daemonset-wm84g\" (UID: \"2e76f14d-dc17-4304-9c72-71119887aec1\") " pod="openshift-must-gather-l54pc/perf-node-gather-daemonset-wm84g" Jul 10 19:24:17.982951 ip-10-0-143-17 kubenswrapper[2585]: I0710 19:24:17.982918 2585 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"proc\" (UniqueName: \"kubernetes.io/host-path/2e76f14d-dc17-4304-9c72-71119887aec1-proc\") pod \"perf-node-gather-daemonset-wm84g\" (UID: \"2e76f14d-dc17-4304-9c72-71119887aec1\") " pod="openshift-must-gather-l54pc/perf-node-gather-daemonset-wm84g" Jul 10 19:24:17.983119 ip-10-0-143-17 kubenswrapper[2585]: I0710 19:24:17.982995 2585 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"sys\" (UniqueName: \"kubernetes.io/host-path/2e76f14d-dc17-4304-9c72-71119887aec1-sys\") pod \"perf-node-gather-daemonset-wm84g\" (UID: \"2e76f14d-dc17-4304-9c72-71119887aec1\") " pod="openshift-must-gather-l54pc/perf-node-gather-daemonset-wm84g" Jul 10 19:24:17.983119 ip-10-0-143-17 kubenswrapper[2585]: I0710 19:24:17.983013 2585 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"podres\" (UniqueName: \"kubernetes.io/host-path/2e76f14d-dc17-4304-9c72-71119887aec1-podres\") pod \"perf-node-gather-daemonset-wm84g\" (UID: \"2e76f14d-dc17-4304-9c72-71119887aec1\") " pod="openshift-must-gather-l54pc/perf-node-gather-daemonset-wm84g" Jul 10 19:24:17.983119 ip-10-0-143-17 kubenswrapper[2585]: I0710 19:24:17.983047 2585 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"proc\" (UniqueName: \"kubernetes.io/host-path/2e76f14d-dc17-4304-9c72-71119887aec1-proc\") pod \"perf-node-gather-daemonset-wm84g\" (UID: \"2e76f14d-dc17-4304-9c72-71119887aec1\") " pod="openshift-must-gather-l54pc/perf-node-gather-daemonset-wm84g" Jul 10 19:24:17.983119 ip-10-0-143-17 kubenswrapper[2585]: I0710 19:24:17.983055 2585 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"lib-modules\" (UniqueName: \"kubernetes.io/host-path/2e76f14d-dc17-4304-9c72-71119887aec1-lib-modules\") pod \"perf-node-gather-daemonset-wm84g\" (UID: \"2e76f14d-dc17-4304-9c72-71119887aec1\") " pod="openshift-must-gather-l54pc/perf-node-gather-daemonset-wm84g" Jul 10 19:24:17.983119 ip-10-0-143-17 kubenswrapper[2585]: I0710 19:24:17.983106 2585 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-qnbrn\" (UniqueName: \"kubernetes.io/projected/2e76f14d-dc17-4304-9c72-71119887aec1-kube-api-access-qnbrn\") pod \"perf-node-gather-daemonset-wm84g\" (UID: \"2e76f14d-dc17-4304-9c72-71119887aec1\") " pod="openshift-must-gather-l54pc/perf-node-gather-daemonset-wm84g" Jul 10 19:24:17.983119 ip-10-0-143-17 kubenswrapper[2585]: I0710 19:24:17.983110 2585 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"sys\" (UniqueName: \"kubernetes.io/host-path/2e76f14d-dc17-4304-9c72-71119887aec1-sys\") pod \"perf-node-gather-daemonset-wm84g\" (UID: \"2e76f14d-dc17-4304-9c72-71119887aec1\") " pod="openshift-must-gather-l54pc/perf-node-gather-daemonset-wm84g" Jul 10 19:24:17.983342 ip-10-0-143-17 kubenswrapper[2585]: I0710 19:24:17.983197 2585 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"podres\" (UniqueName: \"kubernetes.io/host-path/2e76f14d-dc17-4304-9c72-71119887aec1-podres\") pod \"perf-node-gather-daemonset-wm84g\" (UID: \"2e76f14d-dc17-4304-9c72-71119887aec1\") " pod="openshift-must-gather-l54pc/perf-node-gather-daemonset-wm84g" Jul 10 19:24:17.983342 ip-10-0-143-17 kubenswrapper[2585]: I0710 19:24:17.983266 2585 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"lib-modules\" (UniqueName: \"kubernetes.io/host-path/2e76f14d-dc17-4304-9c72-71119887aec1-lib-modules\") pod \"perf-node-gather-daemonset-wm84g\" (UID: \"2e76f14d-dc17-4304-9c72-71119887aec1\") " pod="openshift-must-gather-l54pc/perf-node-gather-daemonset-wm84g" Jul 10 19:24:17.991705 ip-10-0-143-17 kubenswrapper[2585]: I0710 19:24:17.991677 2585 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-qnbrn\" (UniqueName: \"kubernetes.io/projected/2e76f14d-dc17-4304-9c72-71119887aec1-kube-api-access-qnbrn\") pod \"perf-node-gather-daemonset-wm84g\" (UID: \"2e76f14d-dc17-4304-9c72-71119887aec1\") " pod="openshift-must-gather-l54pc/perf-node-gather-daemonset-wm84g" Jul 10 19:24:18.093132 ip-10-0-143-17 kubenswrapper[2585]: I0710 19:24:18.093035 2585 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-must-gather-l54pc/perf-node-gather-daemonset-wm84g" Jul 10 19:24:18.229925 ip-10-0-143-17 kubenswrapper[2585]: I0710 19:24:18.229895 2585 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-must-gather-l54pc/perf-node-gather-daemonset-wm84g"] Jul 10 19:24:18.231482 ip-10-0-143-17 kubenswrapper[2585]: W0710 19:24:18.231451 2585 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-pod2e76f14d_dc17_4304_9c72_71119887aec1.slice/crio-693f28c9d28fd8ddfd2339b41e60d0065698895f4d32f92e339124fbbb8fa86e WatchSource:0}: Error finding container 693f28c9d28fd8ddfd2339b41e60d0065698895f4d32f92e339124fbbb8fa86e: Status 404 returned error can't find the container with id 693f28c9d28fd8ddfd2339b41e60d0065698895f4d32f92e339124fbbb8fa86e Jul 10 19:24:18.468236 ip-10-0-143-17 kubenswrapper[2585]: I0710 19:24:18.468178 2585 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-console_console-5757c989b5-d7f7l_e1138200-d6f8-4502-add0-d6c8a4f91714/console/0.log" Jul 10 19:24:18.944837 ip-10-0-143-17 kubenswrapper[2585]: I0710 19:24:18.944802 2585 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-must-gather-l54pc/perf-node-gather-daemonset-wm84g" event={"ID":"2e76f14d-dc17-4304-9c72-71119887aec1","Type":"ContainerStarted","Data":"5d0988a0e5b4cc15c7f936a8635336cb9520650f57c0317407f193f46efd852d"} Jul 10 19:24:18.945242 ip-10-0-143-17 kubenswrapper[2585]: I0710 19:24:18.944859 2585 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-must-gather-l54pc/perf-node-gather-daemonset-wm84g" event={"ID":"2e76f14d-dc17-4304-9c72-71119887aec1","Type":"ContainerStarted","Data":"693f28c9d28fd8ddfd2339b41e60d0065698895f4d32f92e339124fbbb8fa86e"} Jul 10 19:24:18.945242 ip-10-0-143-17 kubenswrapper[2585]: I0710 19:24:18.944936 2585 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-must-gather-l54pc/perf-node-gather-daemonset-wm84g" Jul 10 19:24:18.964525 ip-10-0-143-17 kubenswrapper[2585]: I0710 19:24:18.964474 2585 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-must-gather-l54pc/perf-node-gather-daemonset-wm84g" podStartSLOduration=1.964458718 podStartE2EDuration="1.964458718s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=true podCreationTimestamp="2026-07-10 19:24:17 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-10 19:24:18.963390609 +0000 UTC m=+2672.420750544" watchObservedRunningTime="2026-07-10 19:24:18.964458718 +0000 UTC m=+2672.421818653" Jul 10 19:24:19.149044 ip-10-0-143-17 kubenswrapper[2585]: I0710 19:24:19.149012 2585 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-cluster-storage-operator_volume-data-source-validator-78495597f6-86v9g_aba57a3c-b29f-4038-9ee1-0dcfdf4c5d5f/volume-data-source-validator/0.log" Jul 10 19:24:20.183055 ip-10-0-143-17 kubenswrapper[2585]: I0710 19:24:20.183026 2585 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-dns_dns-default-bwks6_3c50e2a2-37c9-45b8-9ad0-fc624a151d61/dns/0.log" Jul 10 19:24:20.204444 ip-10-0-143-17 kubenswrapper[2585]: I0710 19:24:20.204414 2585 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-dns_dns-default-bwks6_3c50e2a2-37c9-45b8-9ad0-fc624a151d61/kube-rbac-proxy/0.log" Jul 10 19:24:20.326701 ip-10-0-143-17 kubenswrapper[2585]: I0710 19:24:20.326672 2585 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-dns_node-resolver-ndwqb_5a594168-14da-41cb-9871-edbbf08720f7/dns-node-resolver/0.log" Jul 10 19:24:20.972258 ip-10-0-143-17 kubenswrapper[2585]: I0710 19:24:20.972223 2585 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-image-registry_image-registry-bb888c97d-64x5s_b623c78b-f3ba-4fe7-b967-ab6cabb56ec7/registry/0.log" Jul 10 19:24:21.046193 ip-10-0-143-17 kubenswrapper[2585]: I0710 19:24:21.046164 2585 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-image-registry_node-ca-crbxd_ad44fb01-da7b-40ac-906e-5a7badc87f54/node-ca/0.log" Jul 10 19:24:22.048196 ip-10-0-143-17 kubenswrapper[2585]: I0710 19:24:22.048166 2585 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ingress_istiod-openshift-gateway-94bb8fbfd-6h99s_c2545730-3e12-4efe-b88c-6300fc5510a5/discovery/0.log" Jul 10 19:24:22.075781 ip-10-0-143-17 kubenswrapper[2585]: I0710 19:24:22.075757 2585 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ingress_openshift-ai-inference-openshift-default-6c48cf8bcc-cl5g8_27c2089c-3b42-41ba-91c3-c6f72e532db5/istio-proxy/0.log" Jul 10 19:24:22.076530 ip-10-0-143-17 kubenswrapper[2585]: I0710 19:24:22.076504 2585 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ingress_openshift-ai-inference-openshift-default-6c48cf8bcc-cl5g8_27c2089c-3b42-41ba-91c3-c6f72e532db5/istio-proxy/1.log" Jul 10 19:24:22.097635 ip-10-0-143-17 kubenswrapper[2585]: I0710 19:24:22.097616 2585 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ingress_router-default-5d998f6f9d-t8frx_bf853b15-a7e2-4f66-ace3-b1f1fc9323de/router/0.log" Jul 10 19:24:22.718951 ip-10-0-143-17 kubenswrapper[2585]: I0710 19:24:22.718923 2585 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ingress-canary_ingress-canary-g7fx6_8c0d7178-8c0c-4f9c-b534-fe044b79a208/serve-healthcheck-canary/0.log" Jul 10 19:24:23.176212 ip-10-0-143-17 kubenswrapper[2585]: I0710 19:24:23.176182 2585 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-insights_insights-operator-68d9dd94cf-57mwk_3041e309-0ff4-41d8-9123-a82c98b5dad3/insights-operator/0.log" Jul 10 19:24:23.177244 ip-10-0-143-17 kubenswrapper[2585]: I0710 19:24:23.177219 2585 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-insights_insights-operator-68d9dd94cf-57mwk_3041e309-0ff4-41d8-9123-a82c98b5dad3/insights-operator/1.log" Jul 10 19:24:23.335054 ip-10-0-143-17 kubenswrapper[2585]: I0710 19:24:23.335029 2585 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-insights_insights-runtime-extractor-rd8jt_06325764-d12d-4aa8-9314-71a3a2c49d12/kube-rbac-proxy/0.log" Jul 10 19:24:23.357624 ip-10-0-143-17 kubenswrapper[2585]: I0710 19:24:23.357602 2585 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-insights_insights-runtime-extractor-rd8jt_06325764-d12d-4aa8-9314-71a3a2c49d12/exporter/0.log" Jul 10 19:24:23.377553 ip-10-0-143-17 kubenswrapper[2585]: I0710 19:24:23.377522 2585 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-insights_insights-runtime-extractor-rd8jt_06325764-d12d-4aa8-9314-71a3a2c49d12/extractor/0.log" Jul 10 19:24:24.960718 ip-10-0-143-17 kubenswrapper[2585]: I0710 19:24:24.960693 2585 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-must-gather-l54pc/perf-node-gather-daemonset-wm84g" Jul 10 19:24:26.322013 ip-10-0-143-17 kubenswrapper[2585]: I0710 19:24:26.321980 2585 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-lws-operator_openshift-lws-operator-fd8ccff4c-kr5jx_d8e11bcc-7272-4dcd-ad05-a0183cd58498/openshift-lws-operator/0.log" Jul 10 19:24:27.187044 ip-10-0-143-17 kubenswrapper[2585]: I0710 19:24:27.187010 2585 log.go:25] "Finished parsing log file" path="/var/log/pods/kserve_llmisvc-controller-manager-645ffbd459-p959l_4c4024ac-8ae6-43fa-9a94-9336e896afde/manager/0.log" Jul 10 19:24:27.492271 ip-10-0-143-17 kubenswrapper[2585]: I0710 19:24:27.492228 2585 log.go:25] "Finished parsing log file" path="/var/log/pods/kserve_seaweedfs-64568bcd49-bgwr8_d92b43e7-be96-4435-9d79-2091eb4a84dd/seaweedfs/0.log" Jul 10 19:24:29.178960 ip-10-0-143-17 kubenswrapper[2585]: I0710 19:24:29.178929 2585 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ingress_istiod-openshift-gateway-94bb8fbfd-6h99s_c2545730-3e12-4efe-b88c-6300fc5510a5/discovery/0.log" Jul 10 19:24:29.259930 ip-10-0-143-17 kubenswrapper[2585]: I0710 19:24:29.259896 2585 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ingress_openshift-ai-inference-openshift-default-6c48cf8bcc-cl5g8_27c2089c-3b42-41ba-91c3-c6f72e532db5/istio-proxy/0.log" Jul 10 19:24:29.260652 ip-10-0-143-17 kubenswrapper[2585]: I0710 19:24:29.260619 2585 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ingress_openshift-ai-inference-openshift-default-6c48cf8bcc-cl5g8_27c2089c-3b42-41ba-91c3-c6f72e532db5/istio-proxy/1.log" Jul 10 19:24:29.299345 ip-10-0-143-17 kubenswrapper[2585]: I0710 19:24:29.299309 2585 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ingress_router-default-5d998f6f9d-t8frx_bf853b15-a7e2-4f66-ace3-b1f1fc9323de/router/0.log" Jul 10 19:24:30.522392 ip-10-0-143-17 kubenswrapper[2585]: I0710 19:24:30.522360 2585 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ingress_istiod-openshift-gateway-94bb8fbfd-6h99s_c2545730-3e12-4efe-b88c-6300fc5510a5/discovery/0.log" Jul 10 19:24:30.542441 ip-10-0-143-17 kubenswrapper[2585]: I0710 19:24:30.542405 2585 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ingress_openshift-ai-inference-openshift-default-6c48cf8bcc-cl5g8_27c2089c-3b42-41ba-91c3-c6f72e532db5/istio-proxy/0.log" Jul 10 19:24:30.543256 ip-10-0-143-17 kubenswrapper[2585]: I0710 19:24:30.543233 2585 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ingress_openshift-ai-inference-openshift-default-6c48cf8bcc-cl5g8_27c2089c-3b42-41ba-91c3-c6f72e532db5/istio-proxy/1.log" Jul 10 19:24:30.556150 ip-10-0-143-17 kubenswrapper[2585]: I0710 19:24:30.556123 2585 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ingress_router-default-5d998f6f9d-t8frx_bf853b15-a7e2-4f66-ace3-b1f1fc9323de/router/0.log" Jul 10 19:24:34.494456 ip-10-0-143-17 kubenswrapper[2585]: I0710 19:24:34.494426 2585 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-multus_multus-additional-cni-plugins-w4cxp_c3be1d6d-b0aa-4613-9d9a-e308693cc790/kube-multus-additional-cni-plugins/0.log" Jul 10 19:24:34.516390 ip-10-0-143-17 kubenswrapper[2585]: I0710 19:24:34.516333 2585 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-multus_multus-additional-cni-plugins-w4cxp_c3be1d6d-b0aa-4613-9d9a-e308693cc790/egress-router-binary-copy/0.log" Jul 10 19:24:34.540123 ip-10-0-143-17 kubenswrapper[2585]: I0710 19:24:34.540093 2585 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-multus_multus-additional-cni-plugins-w4cxp_c3be1d6d-b0aa-4613-9d9a-e308693cc790/cni-plugins/0.log" Jul 10 19:24:34.559967 ip-10-0-143-17 kubenswrapper[2585]: I0710 19:24:34.559944 2585 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-multus_multus-additional-cni-plugins-w4cxp_c3be1d6d-b0aa-4613-9d9a-e308693cc790/bond-cni-plugin/0.log" Jul 10 19:24:34.579454 ip-10-0-143-17 kubenswrapper[2585]: I0710 19:24:34.579433 2585 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-multus_multus-additional-cni-plugins-w4cxp_c3be1d6d-b0aa-4613-9d9a-e308693cc790/routeoverride-cni/0.log" Jul 10 19:24:34.601515 ip-10-0-143-17 kubenswrapper[2585]: I0710 19:24:34.601495 2585 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-multus_multus-additional-cni-plugins-w4cxp_c3be1d6d-b0aa-4613-9d9a-e308693cc790/whereabouts-cni-bincopy/0.log" Jul 10 19:24:34.621371 ip-10-0-143-17 kubenswrapper[2585]: I0710 19:24:34.621347 2585 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-multus_multus-additional-cni-plugins-w4cxp_c3be1d6d-b0aa-4613-9d9a-e308693cc790/whereabouts-cni/0.log" Jul 10 19:24:34.810966 ip-10-0-143-17 kubenswrapper[2585]: I0710 19:24:34.810932 2585 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-multus_multus-h4nv5_d0da579f-0565-4704-942a-402e5bf46135/kube-multus/0.log" Jul 10 19:24:34.960355 ip-10-0-143-17 kubenswrapper[2585]: I0710 19:24:34.960324 2585 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-multus_network-metrics-daemon-7rwwk_6516579e-a521-4350-bc6b-c4d698b2f845/network-metrics-daemon/0.log" Jul 10 19:24:34.981361 ip-10-0-143-17 kubenswrapper[2585]: I0710 19:24:34.981314 2585 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-multus_network-metrics-daemon-7rwwk_6516579e-a521-4350-bc6b-c4d698b2f845/kube-rbac-proxy/0.log"