Jul 10 18:01:00.129317 ip-10-0-128-39 systemd[1]: kubelet.service: Failed to load environment files: No such file or directory Jul 10 18:01:00.129332 ip-10-0-128-39 systemd[1]: kubelet.service: Failed to run 'start-pre' task: No such file or directory Jul 10 18:01:00.129342 ip-10-0-128-39 systemd[1]: kubelet.service: Failed with result 'resources'. Jul 10 18:01:00.129652 ip-10-0-128-39 systemd[1]: Failed to start Kubernetes Kubelet. Jul 10 18:01:10.342759 ip-10-0-128-39 systemd[1]: kubelet.service: Failed to schedule restart job: Unit crio.service not found. Jul 10 18:01:10.342778 ip-10-0-128-39 systemd[1]: kubelet.service: Failed with result 'resources'. -- Boot 013a744bf5194048aca49d2ed26f741a -- Jul 10 18:02:50.342670 ip-10-0-128-39 systemd[1]: Starting Kubernetes Kubelet... Jul 10 18:02:51.163600 ip-10-0-128-39 kubenswrapper[2558]: Flag --container-runtime-endpoint has been deprecated, This parameter should be set via the config file specified by the Kubelet's --config flag. See https://kubernetes.io/docs/tasks/administer-cluster/kubelet-config-file/ for more information. Jul 10 18:02:51.163600 ip-10-0-128-39 kubenswrapper[2558]: Flag --minimum-container-ttl-duration has been deprecated, Use --eviction-hard or --eviction-soft instead. Will be removed in a future version. Jul 10 18:02:51.163600 ip-10-0-128-39 kubenswrapper[2558]: Flag --volume-plugin-dir has been deprecated, This parameter should be set via the config file specified by the Kubelet's --config flag. See https://kubernetes.io/docs/tasks/administer-cluster/kubelet-config-file/ for more information. Jul 10 18:02:51.163600 ip-10-0-128-39 kubenswrapper[2558]: Flag --pod-infra-container-image has been deprecated, will be removed in 1.35. Image garbage collector will get sandbox image information from CRI. Jul 10 18:02:51.163600 ip-10-0-128-39 kubenswrapper[2558]: Flag --system-reserved has been deprecated, This parameter should be set via the config file specified by the Kubelet's --config flag. See https://kubernetes.io/docs/tasks/administer-cluster/kubelet-config-file/ for more information. Jul 10 18:02:51.163600 ip-10-0-128-39 kubenswrapper[2558]: I0710 18:02:50.833234 2558 server.go:213] "--pod-infra-container-image will not be pruned by the image garbage collector in kubelet and should also be set in the remote runtime" Jul 10 18:02:51.163600 ip-10-0-128-39 kubenswrapper[2558]: W0710 18:02:50.837686 2558 feature_gate.go:328] unrecognized feature gate: MetricsCollectionProfiles Jul 10 18:02:51.163600 ip-10-0-128-39 kubenswrapper[2558]: W0710 18:02:50.837699 2558 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesAWS Jul 10 18:02:51.163600 ip-10-0-128-39 kubenswrapper[2558]: W0710 18:02:50.837703 2558 feature_gate.go:328] unrecognized feature gate: NoRegistryClusterInstall Jul 10 18:02:51.163600 ip-10-0-128-39 kubenswrapper[2558]: W0710 18:02:50.837707 2558 feature_gate.go:328] unrecognized feature gate: AWSServiceLBNetworkSecurityGroup Jul 10 18:02:51.163600 ip-10-0-128-39 kubenswrapper[2558]: W0710 18:02:50.837710 2558 feature_gate.go:328] unrecognized feature gate: Example Jul 10 18:02:51.163600 ip-10-0-128-39 kubenswrapper[2558]: W0710 18:02:50.837713 2558 feature_gate.go:328] unrecognized feature gate: ClusterVersionOperatorConfiguration Jul 10 18:02:51.163600 ip-10-0-128-39 kubenswrapper[2558]: W0710 18:02:50.837716 2558 feature_gate.go:328] unrecognized feature gate: OnPremDNSRecords Jul 10 18:02:51.163600 ip-10-0-128-39 kubenswrapper[2558]: W0710 18:02:50.837718 2558 feature_gate.go:328] unrecognized feature gate: VSphereMultiDisk Jul 10 18:02:50.904963 ip-10-0-128-39 systemd[1]: Started Kubernetes Kubelet. Jul 10 18:02:51.179616 ip-10-0-128-39 kubenswrapper[2558]: W0710 18:02:50.837721 2558 feature_gate.go:328] unrecognized feature gate: GatewayAPIController Jul 10 18:02:51.179616 ip-10-0-128-39 kubenswrapper[2558]: W0710 18:02:50.837724 2558 feature_gate.go:328] unrecognized feature gate: ShortCertRotation Jul 10 18:02:51.179616 ip-10-0-128-39 kubenswrapper[2558]: W0710 18:02:50.837727 2558 feature_gate.go:328] unrecognized feature gate: ClusterAPIMachineManagementVSphere Jul 10 18:02:51.179616 ip-10-0-128-39 kubenswrapper[2558]: W0710 18:02:50.837731 2558 feature_gate.go:351] Setting GA feature gate VolumeAttributesClass=true. It will be removed in a future release. Jul 10 18:02:51.179616 ip-10-0-128-39 kubenswrapper[2558]: W0710 18:02:50.837735 2558 feature_gate.go:328] unrecognized feature gate: AzureDedicatedHosts Jul 10 18:02:51.179616 ip-10-0-128-39 kubenswrapper[2558]: W0710 18:02:50.837739 2558 feature_gate.go:328] unrecognized feature gate: NetworkSegmentation Jul 10 18:02:51.179616 ip-10-0-128-39 kubenswrapper[2558]: W0710 18:02:50.837742 2558 feature_gate.go:328] unrecognized feature gate: GCPDualStackInstall Jul 10 18:02:51.179616 ip-10-0-128-39 kubenswrapper[2558]: W0710 18:02:50.837750 2558 feature_gate.go:328] unrecognized feature gate: KMSEncryption Jul 10 18:02:51.179616 ip-10-0-128-39 kubenswrapper[2558]: W0710 18:02:50.837753 2558 feature_gate.go:328] unrecognized feature gate: NetworkLiveMigration Jul 10 18:02:51.179616 ip-10-0-128-39 kubenswrapper[2558]: W0710 18:02:50.837756 2558 feature_gate.go:328] unrecognized feature gate: VSphereHostVMGroupZonal Jul 10 18:02:51.179616 ip-10-0-128-39 kubenswrapper[2558]: W0710 18:02:50.837758 2558 feature_gate.go:328] unrecognized feature gate: DNSNameResolver Jul 10 18:02:51.179616 ip-10-0-128-39 kubenswrapper[2558]: W0710 18:02:50.837761 2558 feature_gate.go:328] unrecognized feature gate: MultiDiskSetup Jul 10 18:02:51.179616 ip-10-0-128-39 kubenswrapper[2558]: W0710 18:02:50.837764 2558 feature_gate.go:328] unrecognized feature gate: ExternalSnapshotMetadata Jul 10 18:02:51.179616 ip-10-0-128-39 kubenswrapper[2558]: W0710 18:02:50.837767 2558 feature_gate.go:328] unrecognized feature gate: OSStreams Jul 10 18:02:51.179616 ip-10-0-128-39 kubenswrapper[2558]: W0710 18:02:50.837769 2558 feature_gate.go:328] unrecognized feature gate: HyperShiftOnlyDynamicResourceAllocation Jul 10 18:02:51.179616 ip-10-0-128-39 kubenswrapper[2558]: W0710 18:02:50.837772 2558 feature_gate.go:328] unrecognized feature gate: GatewayAPI Jul 10 18:02:51.179616 ip-10-0-128-39 kubenswrapper[2558]: W0710 18:02:50.837775 2558 feature_gate.go:328] unrecognized feature gate: IngressControllerDynamicConfigurationManager Jul 10 18:02:51.179616 ip-10-0-128-39 kubenswrapper[2558]: W0710 18:02:50.837777 2558 feature_gate.go:328] unrecognized feature gate: BootImageSkewEnforcement Jul 10 18:02:51.179616 ip-10-0-128-39 kubenswrapper[2558]: W0710 18:02:50.837780 2558 feature_gate.go:328] unrecognized feature gate: ClusterAPIInstall Jul 10 18:02:51.179616 ip-10-0-128-39 kubenswrapper[2558]: W0710 18:02:50.837783 2558 feature_gate.go:328] unrecognized feature gate: AdminNetworkPolicy Jul 10 18:02:51.180731 ip-10-0-128-39 kubenswrapper[2558]: W0710 18:02:50.837785 2558 feature_gate.go:328] unrecognized feature gate: VSphereMultiNetworks Jul 10 18:02:51.180731 ip-10-0-128-39 kubenswrapper[2558]: W0710 18:02:50.837788 2558 feature_gate.go:328] unrecognized feature gate: SigstoreImageVerificationPKI Jul 10 18:02:51.180731 ip-10-0-128-39 kubenswrapper[2558]: W0710 18:02:50.837790 2558 feature_gate.go:328] unrecognized feature gate: OpenShiftPodSecurityAdmission Jul 10 18:02:51.180731 ip-10-0-128-39 kubenswrapper[2558]: W0710 18:02:50.837793 2558 feature_gate.go:328] unrecognized feature gate: AzureDualStackInstall Jul 10 18:02:51.180731 ip-10-0-128-39 kubenswrapper[2558]: W0710 18:02:50.837795 2558 feature_gate.go:328] unrecognized feature gate: AWSDualStackInstall Jul 10 18:02:51.180731 ip-10-0-128-39 kubenswrapper[2558]: W0710 18:02:50.837797 2558 feature_gate.go:328] unrecognized feature gate: GCPClusterHostedDNS Jul 10 18:02:51.180731 ip-10-0-128-39 kubenswrapper[2558]: W0710 18:02:50.837800 2558 feature_gate.go:328] unrecognized feature gate: ClusterMonitoringConfig Jul 10 18:02:51.180731 ip-10-0-128-39 kubenswrapper[2558]: W0710 18:02:50.837804 2558 feature_gate.go:351] Setting GA feature gate ServiceAccountTokenNodeBinding=true. It will be removed in a future release. Jul 10 18:02:51.180731 ip-10-0-128-39 kubenswrapper[2558]: W0710 18:02:50.837807 2558 feature_gate.go:328] unrecognized feature gate: InsightsOnDemandDataGather Jul 10 18:02:51.180731 ip-10-0-128-39 kubenswrapper[2558]: W0710 18:02:50.837810 2558 feature_gate.go:328] unrecognized feature gate: GCPCustomAPIEndpointsInstall Jul 10 18:02:51.180731 ip-10-0-128-39 kubenswrapper[2558]: W0710 18:02:50.837814 2558 feature_gate.go:328] unrecognized feature gate: BuildCSIVolumes Jul 10 18:02:51.180731 ip-10-0-128-39 kubenswrapper[2558]: W0710 18:02:50.837817 2558 feature_gate.go:328] unrecognized feature gate: PinnedImages Jul 10 18:02:51.180731 ip-10-0-128-39 kubenswrapper[2558]: W0710 18:02:50.837820 2558 feature_gate.go:328] unrecognized feature gate: AWSClusterHostedDNS Jul 10 18:02:51.180731 ip-10-0-128-39 kubenswrapper[2558]: W0710 18:02:50.837823 2558 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesvSphere Jul 10 18:02:51.180731 ip-10-0-128-39 kubenswrapper[2558]: W0710 18:02:50.837826 2558 feature_gate.go:328] unrecognized feature gate: MachineAPIOperatorDisableMachineHealthCheckController Jul 10 18:02:51.180731 ip-10-0-128-39 kubenswrapper[2558]: W0710 18:02:50.837828 2558 feature_gate.go:328] unrecognized feature gate: BootcNodeManagement Jul 10 18:02:51.180731 ip-10-0-128-39 kubenswrapper[2558]: W0710 18:02:50.837831 2558 feature_gate.go:328] unrecognized feature gate: NewOLMWebhookProviderOpenshiftServiceCA Jul 10 18:02:51.180731 ip-10-0-128-39 kubenswrapper[2558]: W0710 18:02:50.837834 2558 feature_gate.go:328] unrecognized feature gate: Example2 Jul 10 18:02:51.180731 ip-10-0-128-39 kubenswrapper[2558]: W0710 18:02:50.837836 2558 feature_gate.go:328] unrecognized feature gate: IrreconcilableMachineConfig Jul 10 18:02:51.181783 ip-10-0-128-39 kubenswrapper[2558]: W0710 18:02:50.837839 2558 feature_gate.go:328] unrecognized feature gate: VSphereMixedNodeEnv Jul 10 18:02:51.181783 ip-10-0-128-39 kubenswrapper[2558]: W0710 18:02:50.837841 2558 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesCPMS Jul 10 18:02:51.181783 ip-10-0-128-39 kubenswrapper[2558]: W0710 18:02:50.837844 2558 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesAzure Jul 10 18:02:51.181783 ip-10-0-128-39 kubenswrapper[2558]: W0710 18:02:50.837846 2558 feature_gate.go:328] unrecognized feature gate: AutomatedEtcdBackup Jul 10 18:02:51.181783 ip-10-0-128-39 kubenswrapper[2558]: W0710 18:02:50.837850 2558 feature_gate.go:328] unrecognized feature gate: AlibabaPlatform Jul 10 18:02:51.181783 ip-10-0-128-39 kubenswrapper[2558]: W0710 18:02:50.837852 2558 feature_gate.go:328] unrecognized feature gate: ClusterAPIInstallIBMCloud Jul 10 18:02:51.181783 ip-10-0-128-39 kubenswrapper[2558]: W0710 18:02:50.837854 2558 feature_gate.go:328] unrecognized feature gate: RouteAdvertisements Jul 10 18:02:51.181783 ip-10-0-128-39 kubenswrapper[2558]: W0710 18:02:50.837857 2558 feature_gate.go:328] unrecognized feature gate: ExternalOIDC Jul 10 18:02:51.181783 ip-10-0-128-39 kubenswrapper[2558]: W0710 18:02:50.837860 2558 feature_gate.go:328] unrecognized feature gate: AzureWorkloadIdentity Jul 10 18:02:51.181783 ip-10-0-128-39 kubenswrapper[2558]: W0710 18:02:50.837863 2558 feature_gate.go:328] unrecognized feature gate: AWSDedicatedHosts Jul 10 18:02:51.181783 ip-10-0-128-39 kubenswrapper[2558]: W0710 18:02:50.837866 2558 feature_gate.go:328] unrecognized feature gate: ConsolePluginContentSecurityPolicy Jul 10 18:02:51.181783 ip-10-0-128-39 kubenswrapper[2558]: W0710 18:02:50.837868 2558 feature_gate.go:328] unrecognized feature gate: EventTTL Jul 10 18:02:51.181783 ip-10-0-128-39 kubenswrapper[2558]: W0710 18:02:50.837871 2558 feature_gate.go:328] unrecognized feature gate: UpgradeStatus Jul 10 18:02:51.181783 ip-10-0-128-39 kubenswrapper[2558]: W0710 18:02:50.837873 2558 feature_gate.go:328] unrecognized feature gate: DyanmicServiceEndpointIBMCloud Jul 10 18:02:51.181783 ip-10-0-128-39 kubenswrapper[2558]: W0710 18:02:50.837876 2558 feature_gate.go:328] unrecognized feature gate: EtcdBackendQuota Jul 10 18:02:51.181783 ip-10-0-128-39 kubenswrapper[2558]: W0710 18:02:50.837878 2558 feature_gate.go:328] unrecognized feature gate: SigstoreImageVerification Jul 10 18:02:51.181783 ip-10-0-128-39 kubenswrapper[2558]: W0710 18:02:50.837881 2558 feature_gate.go:328] unrecognized feature gate: ExternalOIDCWithUIDAndExtraClaimMappings Jul 10 18:02:51.181783 ip-10-0-128-39 kubenswrapper[2558]: W0710 18:02:50.837884 2558 feature_gate.go:328] unrecognized feature gate: MachineAPIMigration Jul 10 18:02:51.181783 ip-10-0-128-39 kubenswrapper[2558]: W0710 18:02:50.837887 2558 feature_gate.go:328] unrecognized feature gate: MultiArchInstallAzure Jul 10 18:02:51.181783 ip-10-0-128-39 kubenswrapper[2558]: W0710 18:02:50.837889 2558 feature_gate.go:328] unrecognized feature gate: CPMSMachineNamePrefix Jul 10 18:02:51.182856 ip-10-0-128-39 kubenswrapper[2558]: W0710 18:02:50.837892 2558 feature_gate.go:328] unrecognized feature gate: VSphereConfigurableMaxAllowedBlockVolumesPerNode Jul 10 18:02:51.182856 ip-10-0-128-39 kubenswrapper[2558]: W0710 18:02:50.837894 2558 feature_gate.go:328] unrecognized feature gate: AzureClusterHostedDNSInstall Jul 10 18:02:51.182856 ip-10-0-128-39 kubenswrapper[2558]: W0710 18:02:50.837896 2558 feature_gate.go:328] unrecognized feature gate: NewOLMCatalogdAPIV1Metas Jul 10 18:02:51.182856 ip-10-0-128-39 kubenswrapper[2558]: W0710 18:02:50.837899 2558 feature_gate.go:328] unrecognized feature gate: KMSEncryptionProvider Jul 10 18:02:51.182856 ip-10-0-128-39 kubenswrapper[2558]: W0710 18:02:50.837902 2558 feature_gate.go:328] unrecognized feature gate: AWSClusterHostedDNSInstall Jul 10 18:02:51.182856 ip-10-0-128-39 kubenswrapper[2558]: W0710 18:02:50.837904 2558 feature_gate.go:328] unrecognized feature gate: NewOLMPreflightPermissionChecks Jul 10 18:02:51.182856 ip-10-0-128-39 kubenswrapper[2558]: W0710 18:02:50.837907 2558 feature_gate.go:328] unrecognized feature gate: ImageStreamImportMode Jul 10 18:02:51.182856 ip-10-0-128-39 kubenswrapper[2558]: W0710 18:02:50.837910 2558 feature_gate.go:328] unrecognized feature gate: AzureMultiDisk Jul 10 18:02:51.182856 ip-10-0-128-39 kubenswrapper[2558]: W0710 18:02:50.837912 2558 feature_gate.go:328] unrecognized feature gate: GCPCustomAPIEndpoints Jul 10 18:02:51.182856 ip-10-0-128-39 kubenswrapper[2558]: W0710 18:02:50.837915 2558 feature_gate.go:328] unrecognized feature gate: VolumeGroupSnapshot Jul 10 18:02:51.182856 ip-10-0-128-39 kubenswrapper[2558]: W0710 18:02:50.837917 2558 feature_gate.go:328] unrecognized feature gate: DualReplica Jul 10 18:02:51.182856 ip-10-0-128-39 kubenswrapper[2558]: W0710 18:02:50.837920 2558 feature_gate.go:328] unrecognized feature gate: MachineConfigNodes Jul 10 18:02:51.182856 ip-10-0-128-39 kubenswrapper[2558]: W0710 18:02:50.837923 2558 feature_gate.go:328] unrecognized feature gate: NutanixMultiSubnets Jul 10 18:02:51.182856 ip-10-0-128-39 kubenswrapper[2558]: W0710 18:02:50.837926 2558 feature_gate.go:328] unrecognized feature gate: NewOLM Jul 10 18:02:51.182856 ip-10-0-128-39 kubenswrapper[2558]: W0710 18:02:50.837928 2558 feature_gate.go:328] unrecognized feature gate: ProvisioningRequestAvailable Jul 10 18:02:51.182856 ip-10-0-128-39 kubenswrapper[2558]: W0710 18:02:50.837931 2558 feature_gate.go:328] unrecognized feature gate: MixedCPUsAllocation Jul 10 18:02:51.182856 ip-10-0-128-39 kubenswrapper[2558]: W0710 18:02:50.837935 2558 feature_gate.go:328] unrecognized feature gate: InsightsConfig Jul 10 18:02:51.182856 ip-10-0-128-39 kubenswrapper[2558]: W0710 18:02:50.837938 2558 feature_gate.go:328] unrecognized feature gate: HighlyAvailableArbiter Jul 10 18:02:51.182856 ip-10-0-128-39 kubenswrapper[2558]: W0710 18:02:50.837940 2558 feature_gate.go:328] unrecognized feature gate: PreconfiguredUDNAddresses Jul 10 18:02:51.182856 ip-10-0-128-39 kubenswrapper[2558]: W0710 18:02:50.837943 2558 feature_gate.go:328] unrecognized feature gate: SignatureStores Jul 10 18:02:51.183937 ip-10-0-128-39 kubenswrapper[2558]: W0710 18:02:50.837945 2558 feature_gate.go:328] unrecognized feature gate: NewOLMBoxCutterRuntime Jul 10 18:02:51.183937 ip-10-0-128-39 kubenswrapper[2558]: W0710 18:02:50.837948 2558 feature_gate.go:328] unrecognized feature gate: AdditionalRoutingCapabilities Jul 10 18:02:51.183937 ip-10-0-128-39 kubenswrapper[2558]: W0710 18:02:50.837950 2558 feature_gate.go:328] unrecognized feature gate: GatewayAPIWithoutOLM Jul 10 18:02:51.183937 ip-10-0-128-39 kubenswrapper[2558]: W0710 18:02:50.837953 2558 feature_gate.go:328] unrecognized feature gate: NewOLMOwnSingleNamespace Jul 10 18:02:51.183937 ip-10-0-128-39 kubenswrapper[2558]: W0710 18:02:50.837956 2558 feature_gate.go:328] unrecognized feature gate: OVNObservability Jul 10 18:02:51.183937 ip-10-0-128-39 kubenswrapper[2558]: W0710 18:02:50.837958 2558 feature_gate.go:328] unrecognized feature gate: ImageModeStatusReporting Jul 10 18:02:51.183937 ip-10-0-128-39 kubenswrapper[2558]: W0710 18:02:50.837961 2558 feature_gate.go:328] unrecognized feature gate: GCPClusterHostedDNSInstall Jul 10 18:02:51.183937 ip-10-0-128-39 kubenswrapper[2558]: W0710 18:02:50.837963 2558 feature_gate.go:328] unrecognized feature gate: NetworkDiagnosticsConfig Jul 10 18:02:51.183937 ip-10-0-128-39 kubenswrapper[2558]: W0710 18:02:50.837966 2558 feature_gate.go:328] unrecognized feature gate: CRDCompatibilityRequirementOperator Jul 10 18:02:51.183937 ip-10-0-128-39 kubenswrapper[2558]: W0710 18:02:50.837970 2558 feature_gate.go:349] Setting deprecated feature gate KMSv1=true. It will be removed in a future release. Jul 10 18:02:51.183937 ip-10-0-128-39 kubenswrapper[2558]: W0710 18:02:50.837973 2558 feature_gate.go:328] unrecognized feature gate: ClusterAPIMachineManagement Jul 10 18:02:51.183937 ip-10-0-128-39 kubenswrapper[2558]: W0710 18:02:50.837976 2558 feature_gate.go:328] unrecognized feature gate: ManagedBootImages Jul 10 18:02:51.183937 ip-10-0-128-39 kubenswrapper[2558]: W0710 18:02:50.839460 2558 feature_gate.go:328] unrecognized feature gate: RouteAdvertisements Jul 10 18:02:51.183937 ip-10-0-128-39 kubenswrapper[2558]: W0710 18:02:50.839469 2558 feature_gate.go:351] Setting GA feature gate VolumeAttributesClass=true. It will be removed in a future release. Jul 10 18:02:51.183937 ip-10-0-128-39 kubenswrapper[2558]: W0710 18:02:50.839474 2558 feature_gate.go:328] unrecognized feature gate: GatewayAPIController Jul 10 18:02:51.183937 ip-10-0-128-39 kubenswrapper[2558]: W0710 18:02:50.839477 2558 feature_gate.go:328] unrecognized feature gate: AutomatedEtcdBackup Jul 10 18:02:51.183937 ip-10-0-128-39 kubenswrapper[2558]: W0710 18:02:50.839480 2558 feature_gate.go:328] unrecognized feature gate: EtcdBackendQuota Jul 10 18:02:51.183937 ip-10-0-128-39 kubenswrapper[2558]: W0710 18:02:50.839485 2558 feature_gate.go:328] unrecognized feature gate: PinnedImages Jul 10 18:02:51.183937 ip-10-0-128-39 kubenswrapper[2558]: W0710 18:02:50.839488 2558 feature_gate.go:328] unrecognized feature gate: AzureDualStackInstall Jul 10 18:02:51.185012 ip-10-0-128-39 kubenswrapper[2558]: W0710 18:02:50.839491 2558 feature_gate.go:328] unrecognized feature gate: DualReplica Jul 10 18:02:51.185012 ip-10-0-128-39 kubenswrapper[2558]: W0710 18:02:50.839494 2558 feature_gate.go:328] unrecognized feature gate: NewOLMBoxCutterRuntime Jul 10 18:02:51.185012 ip-10-0-128-39 kubenswrapper[2558]: W0710 18:02:50.839496 2558 feature_gate.go:328] unrecognized feature gate: AWSServiceLBNetworkSecurityGroup Jul 10 18:02:51.185012 ip-10-0-128-39 kubenswrapper[2558]: W0710 18:02:50.839499 2558 feature_gate.go:328] unrecognized feature gate: NetworkLiveMigration Jul 10 18:02:51.185012 ip-10-0-128-39 kubenswrapper[2558]: W0710 18:02:50.839502 2558 feature_gate.go:328] unrecognized feature gate: AWSDedicatedHosts Jul 10 18:02:51.185012 ip-10-0-128-39 kubenswrapper[2558]: W0710 18:02:50.839506 2558 feature_gate.go:351] Setting GA feature gate ServiceAccountTokenNodeBinding=true. It will be removed in a future release. Jul 10 18:02:51.185012 ip-10-0-128-39 kubenswrapper[2558]: W0710 18:02:50.839509 2558 feature_gate.go:328] unrecognized feature gate: NewOLMOwnSingleNamespace Jul 10 18:02:51.185012 ip-10-0-128-39 kubenswrapper[2558]: W0710 18:02:50.839512 2558 feature_gate.go:328] unrecognized feature gate: PreconfiguredUDNAddresses Jul 10 18:02:51.185012 ip-10-0-128-39 kubenswrapper[2558]: W0710 18:02:50.839515 2558 feature_gate.go:328] unrecognized feature gate: MachineAPIMigration Jul 10 18:02:51.185012 ip-10-0-128-39 kubenswrapper[2558]: W0710 18:02:50.839518 2558 feature_gate.go:328] unrecognized feature gate: ProvisioningRequestAvailable Jul 10 18:02:51.185012 ip-10-0-128-39 kubenswrapper[2558]: W0710 18:02:50.839521 2558 feature_gate.go:328] unrecognized feature gate: BuildCSIVolumes Jul 10 18:02:51.185012 ip-10-0-128-39 kubenswrapper[2558]: W0710 18:02:50.839524 2558 feature_gate.go:328] unrecognized feature gate: VSphereMixedNodeEnv Jul 10 18:02:51.185012 ip-10-0-128-39 kubenswrapper[2558]: W0710 18:02:50.839526 2558 feature_gate.go:328] unrecognized feature gate: AdditionalRoutingCapabilities Jul 10 18:02:51.185012 ip-10-0-128-39 kubenswrapper[2558]: W0710 18:02:50.839529 2558 feature_gate.go:328] unrecognized feature gate: ShortCertRotation Jul 10 18:02:51.185012 ip-10-0-128-39 kubenswrapper[2558]: W0710 18:02:50.839532 2558 feature_gate.go:328] unrecognized feature gate: ManagedBootImages Jul 10 18:02:51.185012 ip-10-0-128-39 kubenswrapper[2558]: W0710 18:02:50.839535 2558 feature_gate.go:328] unrecognized feature gate: MixedCPUsAllocation Jul 10 18:02:51.185012 ip-10-0-128-39 kubenswrapper[2558]: W0710 18:02:50.839537 2558 feature_gate.go:328] unrecognized feature gate: OSStreams Jul 10 18:02:51.185012 ip-10-0-128-39 kubenswrapper[2558]: W0710 18:02:50.839541 2558 feature_gate.go:328] unrecognized feature gate: MultiArchInstallAzure Jul 10 18:02:51.185012 ip-10-0-128-39 kubenswrapper[2558]: W0710 18:02:50.839544 2558 feature_gate.go:328] unrecognized feature gate: SigstoreImageVerificationPKI Jul 10 18:02:51.185012 ip-10-0-128-39 kubenswrapper[2558]: W0710 18:02:50.839546 2558 feature_gate.go:328] unrecognized feature gate: AWSDualStackInstall Jul 10 18:02:51.186513 ip-10-0-128-39 kubenswrapper[2558]: W0710 18:02:50.839548 2558 feature_gate.go:328] unrecognized feature gate: ClusterAPIMachineManagementVSphere Jul 10 18:02:51.186513 ip-10-0-128-39 kubenswrapper[2558]: W0710 18:02:50.839551 2558 feature_gate.go:328] unrecognized feature gate: GCPDualStackInstall Jul 10 18:02:51.186513 ip-10-0-128-39 kubenswrapper[2558]: W0710 18:02:50.839554 2558 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesAzure Jul 10 18:02:51.186513 ip-10-0-128-39 kubenswrapper[2558]: W0710 18:02:50.839556 2558 feature_gate.go:328] unrecognized feature gate: BootcNodeManagement Jul 10 18:02:51.186513 ip-10-0-128-39 kubenswrapper[2558]: W0710 18:02:50.839559 2558 feature_gate.go:328] unrecognized feature gate: OpenShiftPodSecurityAdmission Jul 10 18:02:51.186513 ip-10-0-128-39 kubenswrapper[2558]: W0710 18:02:50.839562 2558 feature_gate.go:328] unrecognized feature gate: ClusterMonitoringConfig Jul 10 18:02:51.186513 ip-10-0-128-39 kubenswrapper[2558]: W0710 18:02:50.839564 2558 feature_gate.go:328] unrecognized feature gate: KMSEncryptionProvider Jul 10 18:02:51.186513 ip-10-0-128-39 kubenswrapper[2558]: W0710 18:02:50.839567 2558 feature_gate.go:328] unrecognized feature gate: GCPClusterHostedDNSInstall Jul 10 18:02:51.186513 ip-10-0-128-39 kubenswrapper[2558]: W0710 18:02:50.839569 2558 feature_gate.go:328] unrecognized feature gate: ClusterVersionOperatorConfiguration Jul 10 18:02:51.186513 ip-10-0-128-39 kubenswrapper[2558]: W0710 18:02:50.839572 2558 feature_gate.go:328] unrecognized feature gate: GCPCustomAPIEndpointsInstall Jul 10 18:02:51.186513 ip-10-0-128-39 kubenswrapper[2558]: W0710 18:02:50.839575 2558 feature_gate.go:328] unrecognized feature gate: MachineConfigNodes Jul 10 18:02:51.186513 ip-10-0-128-39 kubenswrapper[2558]: W0710 18:02:50.839578 2558 feature_gate.go:328] unrecognized feature gate: AdminNetworkPolicy Jul 10 18:02:51.186513 ip-10-0-128-39 kubenswrapper[2558]: W0710 18:02:50.839580 2558 feature_gate.go:328] unrecognized feature gate: ConsolePluginContentSecurityPolicy Jul 10 18:02:51.186513 ip-10-0-128-39 kubenswrapper[2558]: W0710 18:02:50.839583 2558 feature_gate.go:328] unrecognized feature gate: ImageStreamImportMode Jul 10 18:02:51.186513 ip-10-0-128-39 kubenswrapper[2558]: W0710 18:02:50.839585 2558 feature_gate.go:328] unrecognized feature gate: NewOLMCatalogdAPIV1Metas Jul 10 18:02:51.186513 ip-10-0-128-39 kubenswrapper[2558]: W0710 18:02:50.839589 2558 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesCPMS Jul 10 18:02:51.186513 ip-10-0-128-39 kubenswrapper[2558]: W0710 18:02:50.839591 2558 feature_gate.go:328] unrecognized feature gate: ExternalOIDCWithUIDAndExtraClaimMappings Jul 10 18:02:51.186513 ip-10-0-128-39 kubenswrapper[2558]: W0710 18:02:50.839594 2558 feature_gate.go:328] unrecognized feature gate: MetricsCollectionProfiles Jul 10 18:02:51.186513 ip-10-0-128-39 kubenswrapper[2558]: W0710 18:02:50.839596 2558 feature_gate.go:328] unrecognized feature gate: NewOLMWebhookProviderOpenshiftServiceCA Jul 10 18:02:51.187565 ip-10-0-128-39 kubenswrapper[2558]: W0710 18:02:50.839599 2558 feature_gate.go:328] unrecognized feature gate: UpgradeStatus Jul 10 18:02:51.187565 ip-10-0-128-39 kubenswrapper[2558]: W0710 18:02:50.839602 2558 feature_gate.go:328] unrecognized feature gate: AzureMultiDisk Jul 10 18:02:51.187565 ip-10-0-128-39 kubenswrapper[2558]: W0710 18:02:50.839605 2558 feature_gate.go:328] unrecognized feature gate: AzureWorkloadIdentity Jul 10 18:02:51.187565 ip-10-0-128-39 kubenswrapper[2558]: W0710 18:02:50.839607 2558 feature_gate.go:328] unrecognized feature gate: ClusterAPIInstall Jul 10 18:02:51.187565 ip-10-0-128-39 kubenswrapper[2558]: W0710 18:02:50.839610 2558 feature_gate.go:328] unrecognized feature gate: VSphereMultiNetworks Jul 10 18:02:51.187565 ip-10-0-128-39 kubenswrapper[2558]: W0710 18:02:50.839612 2558 feature_gate.go:328] unrecognized feature gate: MultiDiskSetup Jul 10 18:02:51.187565 ip-10-0-128-39 kubenswrapper[2558]: W0710 18:02:50.839615 2558 feature_gate.go:328] unrecognized feature gate: NoRegistryClusterInstall Jul 10 18:02:51.187565 ip-10-0-128-39 kubenswrapper[2558]: W0710 18:02:50.839617 2558 feature_gate.go:328] unrecognized feature gate: SignatureStores Jul 10 18:02:51.187565 ip-10-0-128-39 kubenswrapper[2558]: W0710 18:02:50.839620 2558 feature_gate.go:328] unrecognized feature gate: DyanmicServiceEndpointIBMCloud Jul 10 18:02:51.187565 ip-10-0-128-39 kubenswrapper[2558]: W0710 18:02:50.839622 2558 feature_gate.go:328] unrecognized feature gate: InsightsConfig Jul 10 18:02:51.187565 ip-10-0-128-39 kubenswrapper[2558]: W0710 18:02:50.839625 2558 feature_gate.go:328] unrecognized feature gate: IrreconcilableMachineConfig Jul 10 18:02:51.187565 ip-10-0-128-39 kubenswrapper[2558]: W0710 18:02:50.839627 2558 feature_gate.go:328] unrecognized feature gate: HighlyAvailableArbiter Jul 10 18:02:51.187565 ip-10-0-128-39 kubenswrapper[2558]: W0710 18:02:50.839630 2558 feature_gate.go:328] unrecognized feature gate: NetworkSegmentation Jul 10 18:02:51.187565 ip-10-0-128-39 kubenswrapper[2558]: W0710 18:02:50.839633 2558 feature_gate.go:328] unrecognized feature gate: OVNObservability Jul 10 18:02:51.187565 ip-10-0-128-39 kubenswrapper[2558]: W0710 18:02:50.839635 2558 feature_gate.go:328] unrecognized feature gate: NewOLM Jul 10 18:02:51.187565 ip-10-0-128-39 kubenswrapper[2558]: W0710 18:02:50.839638 2558 feature_gate.go:328] unrecognized feature gate: InsightsOnDemandDataGather Jul 10 18:02:51.187565 ip-10-0-128-39 kubenswrapper[2558]: W0710 18:02:50.839641 2558 feature_gate.go:328] unrecognized feature gate: VolumeGroupSnapshot Jul 10 18:02:51.187565 ip-10-0-128-39 kubenswrapper[2558]: W0710 18:02:50.839644 2558 feature_gate.go:328] unrecognized feature gate: VSphereConfigurableMaxAllowedBlockVolumesPerNode Jul 10 18:02:51.187565 ip-10-0-128-39 kubenswrapper[2558]: W0710 18:02:50.839647 2558 feature_gate.go:328] unrecognized feature gate: SigstoreImageVerification Jul 10 18:02:51.187565 ip-10-0-128-39 kubenswrapper[2558]: W0710 18:02:50.839649 2558 feature_gate.go:328] unrecognized feature gate: DNSNameResolver Jul 10 18:02:51.188623 ip-10-0-128-39 kubenswrapper[2558]: W0710 18:02:50.839653 2558 feature_gate.go:349] Setting deprecated feature gate KMSv1=true. It will be removed in a future release. Jul 10 18:02:51.188623 ip-10-0-128-39 kubenswrapper[2558]: W0710 18:02:50.839657 2558 feature_gate.go:328] unrecognized feature gate: GatewayAPI Jul 10 18:02:51.188623 ip-10-0-128-39 kubenswrapper[2558]: W0710 18:02:50.839659 2558 feature_gate.go:328] unrecognized feature gate: ImageModeStatusReporting Jul 10 18:02:51.188623 ip-10-0-128-39 kubenswrapper[2558]: W0710 18:02:50.839662 2558 feature_gate.go:328] unrecognized feature gate: HyperShiftOnlyDynamicResourceAllocation Jul 10 18:02:51.188623 ip-10-0-128-39 kubenswrapper[2558]: W0710 18:02:50.839665 2558 feature_gate.go:328] unrecognized feature gate: VSphereMultiDisk Jul 10 18:02:51.188623 ip-10-0-128-39 kubenswrapper[2558]: W0710 18:02:50.839669 2558 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesvSphere Jul 10 18:02:51.188623 ip-10-0-128-39 kubenswrapper[2558]: W0710 18:02:50.839672 2558 feature_gate.go:328] unrecognized feature gate: AWSClusterHostedDNS Jul 10 18:02:51.188623 ip-10-0-128-39 kubenswrapper[2558]: W0710 18:02:50.839674 2558 feature_gate.go:328] unrecognized feature gate: VSphereHostVMGroupZonal Jul 10 18:02:51.188623 ip-10-0-128-39 kubenswrapper[2558]: W0710 18:02:50.839676 2558 feature_gate.go:328] unrecognized feature gate: AWSClusterHostedDNSInstall Jul 10 18:02:51.188623 ip-10-0-128-39 kubenswrapper[2558]: W0710 18:02:50.839679 2558 feature_gate.go:328] unrecognized feature gate: ClusterAPIInstallIBMCloud Jul 10 18:02:51.188623 ip-10-0-128-39 kubenswrapper[2558]: W0710 18:02:50.839681 2558 feature_gate.go:328] unrecognized feature gate: GCPClusterHostedDNS Jul 10 18:02:51.188623 ip-10-0-128-39 kubenswrapper[2558]: W0710 18:02:50.839684 2558 feature_gate.go:328] unrecognized feature gate: ExternalOIDC Jul 10 18:02:51.188623 ip-10-0-128-39 kubenswrapper[2558]: W0710 18:02:50.839686 2558 feature_gate.go:328] unrecognized feature gate: AzureClusterHostedDNSInstall Jul 10 18:02:51.188623 ip-10-0-128-39 kubenswrapper[2558]: W0710 18:02:50.839689 2558 feature_gate.go:328] unrecognized feature gate: EventTTL Jul 10 18:02:51.188623 ip-10-0-128-39 kubenswrapper[2558]: W0710 18:02:50.839691 2558 feature_gate.go:328] unrecognized feature gate: GatewayAPIWithoutOLM Jul 10 18:02:51.188623 ip-10-0-128-39 kubenswrapper[2558]: W0710 18:02:50.839694 2558 feature_gate.go:328] unrecognized feature gate: AlibabaPlatform Jul 10 18:02:51.188623 ip-10-0-128-39 kubenswrapper[2558]: W0710 18:02:50.839697 2558 feature_gate.go:328] unrecognized feature gate: NutanixMultiSubnets Jul 10 18:02:51.188623 ip-10-0-128-39 kubenswrapper[2558]: W0710 18:02:50.839699 2558 feature_gate.go:328] unrecognized feature gate: KMSEncryption Jul 10 18:02:51.188623 ip-10-0-128-39 kubenswrapper[2558]: W0710 18:02:50.839702 2558 feature_gate.go:328] unrecognized feature gate: CRDCompatibilityRequirementOperator Jul 10 18:02:51.188623 ip-10-0-128-39 kubenswrapper[2558]: W0710 18:02:50.839705 2558 feature_gate.go:328] unrecognized feature gate: AzureDedicatedHosts Jul 10 18:02:51.189484 ip-10-0-128-39 kubenswrapper[2558]: W0710 18:02:50.839707 2558 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesAWS Jul 10 18:02:51.189484 ip-10-0-128-39 kubenswrapper[2558]: W0710 18:02:50.839710 2558 feature_gate.go:328] unrecognized feature gate: ExternalSnapshotMetadata Jul 10 18:02:51.189484 ip-10-0-128-39 kubenswrapper[2558]: W0710 18:02:50.839712 2558 feature_gate.go:328] unrecognized feature gate: MachineAPIOperatorDisableMachineHealthCheckController Jul 10 18:02:51.189484 ip-10-0-128-39 kubenswrapper[2558]: W0710 18:02:50.839715 2558 feature_gate.go:328] unrecognized feature gate: CPMSMachineNamePrefix Jul 10 18:02:51.189484 ip-10-0-128-39 kubenswrapper[2558]: W0710 18:02:50.839718 2558 feature_gate.go:328] unrecognized feature gate: IngressControllerDynamicConfigurationManager Jul 10 18:02:51.189484 ip-10-0-128-39 kubenswrapper[2558]: W0710 18:02:50.839720 2558 feature_gate.go:328] unrecognized feature gate: Example Jul 10 18:02:51.189484 ip-10-0-128-39 kubenswrapper[2558]: W0710 18:02:50.839723 2558 feature_gate.go:328] unrecognized feature gate: BootImageSkewEnforcement Jul 10 18:02:51.189484 ip-10-0-128-39 kubenswrapper[2558]: W0710 18:02:50.839725 2558 feature_gate.go:328] unrecognized feature gate: NewOLMPreflightPermissionChecks Jul 10 18:02:51.189484 ip-10-0-128-39 kubenswrapper[2558]: W0710 18:02:50.839727 2558 feature_gate.go:328] unrecognized feature gate: Example2 Jul 10 18:02:51.189484 ip-10-0-128-39 kubenswrapper[2558]: W0710 18:02:50.839732 2558 feature_gate.go:328] unrecognized feature gate: OnPremDNSRecords Jul 10 18:02:51.189484 ip-10-0-128-39 kubenswrapper[2558]: W0710 18:02:50.839735 2558 feature_gate.go:328] unrecognized feature gate: GCPCustomAPIEndpoints Jul 10 18:02:51.189484 ip-10-0-128-39 kubenswrapper[2558]: W0710 18:02:50.839738 2558 feature_gate.go:328] unrecognized feature gate: ClusterAPIMachineManagement Jul 10 18:02:51.189484 ip-10-0-128-39 kubenswrapper[2558]: W0710 18:02:50.839740 2558 feature_gate.go:328] unrecognized feature gate: NetworkDiagnosticsConfig Jul 10 18:02:51.189484 ip-10-0-128-39 kubenswrapper[2558]: W0710 18:02:50.840106 2558 feature_gate.go:328] unrecognized feature gate: VSphereConfigurableMaxAllowedBlockVolumesPerNode Jul 10 18:02:51.189484 ip-10-0-128-39 kubenswrapper[2558]: W0710 18:02:50.840111 2558 feature_gate.go:328] unrecognized feature gate: AWSDualStackInstall Jul 10 18:02:51.189484 ip-10-0-128-39 kubenswrapper[2558]: W0710 18:02:50.840114 2558 feature_gate.go:328] unrecognized feature gate: SigstoreImageVerification Jul 10 18:02:51.189484 ip-10-0-128-39 kubenswrapper[2558]: W0710 18:02:50.840117 2558 feature_gate.go:328] unrecognized feature gate: MultiArchInstallAzure Jul 10 18:02:51.189484 ip-10-0-128-39 kubenswrapper[2558]: W0710 18:02:50.840120 2558 feature_gate.go:328] unrecognized feature gate: AzureMultiDisk Jul 10 18:02:51.189484 ip-10-0-128-39 kubenswrapper[2558]: W0710 18:02:50.840123 2558 feature_gate.go:328] unrecognized feature gate: AzureWorkloadIdentity Jul 10 18:02:51.190356 ip-10-0-128-39 kubenswrapper[2558]: W0710 18:02:50.840126 2558 feature_gate.go:328] unrecognized feature gate: ClusterAPIInstallIBMCloud Jul 10 18:02:51.190356 ip-10-0-128-39 kubenswrapper[2558]: W0710 18:02:50.840128 2558 feature_gate.go:328] unrecognized feature gate: HighlyAvailableArbiter Jul 10 18:02:51.190356 ip-10-0-128-39 kubenswrapper[2558]: W0710 18:02:50.840131 2558 feature_gate.go:328] unrecognized feature gate: VSphereMultiDisk Jul 10 18:02:51.190356 ip-10-0-128-39 kubenswrapper[2558]: W0710 18:02:50.840133 2558 feature_gate.go:328] unrecognized feature gate: NewOLM Jul 10 18:02:51.190356 ip-10-0-128-39 kubenswrapper[2558]: W0710 18:02:50.840136 2558 feature_gate.go:328] unrecognized feature gate: MachineAPIOperatorDisableMachineHealthCheckController Jul 10 18:02:51.190356 ip-10-0-128-39 kubenswrapper[2558]: W0710 18:02:50.840139 2558 feature_gate.go:328] unrecognized feature gate: InsightsConfig Jul 10 18:02:51.190356 ip-10-0-128-39 kubenswrapper[2558]: W0710 18:02:50.840141 2558 feature_gate.go:328] unrecognized feature gate: MultiDiskSetup Jul 10 18:02:51.190356 ip-10-0-128-39 kubenswrapper[2558]: W0710 18:02:50.840144 2558 feature_gate.go:328] unrecognized feature gate: Example Jul 10 18:02:51.190356 ip-10-0-128-39 kubenswrapper[2558]: W0710 18:02:50.840146 2558 feature_gate.go:328] unrecognized feature gate: ClusterAPIMachineManagement Jul 10 18:02:51.190356 ip-10-0-128-39 kubenswrapper[2558]: W0710 18:02:50.840149 2558 feature_gate.go:328] unrecognized feature gate: NewOLMCatalogdAPIV1Metas Jul 10 18:02:51.190356 ip-10-0-128-39 kubenswrapper[2558]: W0710 18:02:50.840152 2558 feature_gate.go:328] unrecognized feature gate: AWSDedicatedHosts Jul 10 18:02:51.190356 ip-10-0-128-39 kubenswrapper[2558]: W0710 18:02:50.840154 2558 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesAWS Jul 10 18:02:51.190356 ip-10-0-128-39 kubenswrapper[2558]: W0710 18:02:50.840157 2558 feature_gate.go:328] unrecognized feature gate: ImageModeStatusReporting Jul 10 18:02:51.190356 ip-10-0-128-39 kubenswrapper[2558]: W0710 18:02:50.840159 2558 feature_gate.go:328] unrecognized feature gate: NetworkSegmentation Jul 10 18:02:51.190356 ip-10-0-128-39 kubenswrapper[2558]: W0710 18:02:50.840162 2558 feature_gate.go:328] unrecognized feature gate: VSphereHostVMGroupZonal Jul 10 18:02:51.190356 ip-10-0-128-39 kubenswrapper[2558]: W0710 18:02:50.840164 2558 feature_gate.go:328] unrecognized feature gate: VolumeGroupSnapshot Jul 10 18:02:51.190356 ip-10-0-128-39 kubenswrapper[2558]: W0710 18:02:50.840167 2558 feature_gate.go:328] unrecognized feature gate: AzureClusterHostedDNSInstall Jul 10 18:02:51.190356 ip-10-0-128-39 kubenswrapper[2558]: W0710 18:02:50.840184 2558 feature_gate.go:328] unrecognized feature gate: NetworkLiveMigration Jul 10 18:02:51.190356 ip-10-0-128-39 kubenswrapper[2558]: W0710 18:02:50.840187 2558 feature_gate.go:328] unrecognized feature gate: OnPremDNSRecords Jul 10 18:02:51.190356 ip-10-0-128-39 kubenswrapper[2558]: W0710 18:02:50.840190 2558 feature_gate.go:328] unrecognized feature gate: MachineConfigNodes Jul 10 18:02:51.191232 ip-10-0-128-39 kubenswrapper[2558]: W0710 18:02:50.840193 2558 feature_gate.go:328] unrecognized feature gate: AWSServiceLBNetworkSecurityGroup Jul 10 18:02:51.191232 ip-10-0-128-39 kubenswrapper[2558]: W0710 18:02:50.840195 2558 feature_gate.go:328] unrecognized feature gate: NewOLMPreflightPermissionChecks Jul 10 18:02:51.191232 ip-10-0-128-39 kubenswrapper[2558]: W0710 18:02:50.840199 2558 feature_gate.go:328] unrecognized feature gate: IngressControllerDynamicConfigurationManager Jul 10 18:02:51.191232 ip-10-0-128-39 kubenswrapper[2558]: W0710 18:02:50.840202 2558 feature_gate.go:328] unrecognized feature gate: AWSClusterHostedDNSInstall Jul 10 18:02:51.191232 ip-10-0-128-39 kubenswrapper[2558]: W0710 18:02:50.840205 2558 feature_gate.go:328] unrecognized feature gate: NetworkDiagnosticsConfig Jul 10 18:02:51.191232 ip-10-0-128-39 kubenswrapper[2558]: W0710 18:02:50.840207 2558 feature_gate.go:328] unrecognized feature gate: ProvisioningRequestAvailable Jul 10 18:02:51.191232 ip-10-0-128-39 kubenswrapper[2558]: W0710 18:02:50.840210 2558 feature_gate.go:328] unrecognized feature gate: BootImageSkewEnforcement Jul 10 18:02:51.191232 ip-10-0-128-39 kubenswrapper[2558]: W0710 18:02:50.840213 2558 feature_gate.go:328] unrecognized feature gate: PinnedImages Jul 10 18:02:51.191232 ip-10-0-128-39 kubenswrapper[2558]: W0710 18:02:50.840216 2558 feature_gate.go:328] unrecognized feature gate: SigstoreImageVerificationPKI Jul 10 18:02:51.191232 ip-10-0-128-39 kubenswrapper[2558]: W0710 18:02:50.840218 2558 feature_gate.go:328] unrecognized feature gate: VSphereMixedNodeEnv Jul 10 18:02:51.191232 ip-10-0-128-39 kubenswrapper[2558]: W0710 18:02:50.840221 2558 feature_gate.go:328] unrecognized feature gate: EventTTL Jul 10 18:02:51.191232 ip-10-0-128-39 kubenswrapper[2558]: W0710 18:02:50.840224 2558 feature_gate.go:328] unrecognized feature gate: DualReplica Jul 10 18:02:51.191232 ip-10-0-128-39 kubenswrapper[2558]: W0710 18:02:50.840227 2558 feature_gate.go:328] unrecognized feature gate: ShortCertRotation Jul 10 18:02:51.191232 ip-10-0-128-39 kubenswrapper[2558]: W0710 18:02:50.840230 2558 feature_gate.go:328] unrecognized feature gate: VSphereMultiNetworks Jul 10 18:02:51.191232 ip-10-0-128-39 kubenswrapper[2558]: W0710 18:02:50.840233 2558 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesAzure Jul 10 18:02:51.191232 ip-10-0-128-39 kubenswrapper[2558]: W0710 18:02:50.840237 2558 feature_gate.go:351] Setting GA feature gate ServiceAccountTokenNodeBinding=true. It will be removed in a future release. Jul 10 18:02:51.191232 ip-10-0-128-39 kubenswrapper[2558]: W0710 18:02:50.840241 2558 feature_gate.go:328] unrecognized feature gate: NutanixMultiSubnets Jul 10 18:02:51.191232 ip-10-0-128-39 kubenswrapper[2558]: W0710 18:02:50.840244 2558 feature_gate.go:328] unrecognized feature gate: ManagedBootImages Jul 10 18:02:51.191232 ip-10-0-128-39 kubenswrapper[2558]: W0710 18:02:50.840247 2558 feature_gate.go:328] unrecognized feature gate: HyperShiftOnlyDynamicResourceAllocation Jul 10 18:02:51.192113 ip-10-0-128-39 kubenswrapper[2558]: W0710 18:02:50.840250 2558 feature_gate.go:328] unrecognized feature gate: CPMSMachineNamePrefix Jul 10 18:02:51.192113 ip-10-0-128-39 kubenswrapper[2558]: W0710 18:02:50.840253 2558 feature_gate.go:328] unrecognized feature gate: ImageStreamImportMode Jul 10 18:02:51.192113 ip-10-0-128-39 kubenswrapper[2558]: W0710 18:02:50.840256 2558 feature_gate.go:328] unrecognized feature gate: ExternalOIDCWithUIDAndExtraClaimMappings Jul 10 18:02:51.192113 ip-10-0-128-39 kubenswrapper[2558]: W0710 18:02:50.840259 2558 feature_gate.go:328] unrecognized feature gate: CRDCompatibilityRequirementOperator Jul 10 18:02:51.192113 ip-10-0-128-39 kubenswrapper[2558]: W0710 18:02:50.840261 2558 feature_gate.go:328] unrecognized feature gate: PreconfiguredUDNAddresses Jul 10 18:02:51.192113 ip-10-0-128-39 kubenswrapper[2558]: W0710 18:02:50.840264 2558 feature_gate.go:328] unrecognized feature gate: AdditionalRoutingCapabilities Jul 10 18:02:51.192113 ip-10-0-128-39 kubenswrapper[2558]: W0710 18:02:50.840266 2558 feature_gate.go:328] unrecognized feature gate: OVNObservability Jul 10 18:02:51.192113 ip-10-0-128-39 kubenswrapper[2558]: W0710 18:02:50.840269 2558 feature_gate.go:328] unrecognized feature gate: GCPDualStackInstall Jul 10 18:02:51.192113 ip-10-0-128-39 kubenswrapper[2558]: W0710 18:02:50.840271 2558 feature_gate.go:328] unrecognized feature gate: ClusterVersionOperatorConfiguration Jul 10 18:02:51.192113 ip-10-0-128-39 kubenswrapper[2558]: W0710 18:02:50.840274 2558 feature_gate.go:328] unrecognized feature gate: Example2 Jul 10 18:02:51.192113 ip-10-0-128-39 kubenswrapper[2558]: W0710 18:02:50.840277 2558 feature_gate.go:328] unrecognized feature gate: NoRegistryClusterInstall Jul 10 18:02:51.192113 ip-10-0-128-39 kubenswrapper[2558]: W0710 18:02:50.840280 2558 feature_gate.go:328] unrecognized feature gate: DyanmicServiceEndpointIBMCloud Jul 10 18:02:51.192113 ip-10-0-128-39 kubenswrapper[2558]: W0710 18:02:50.840282 2558 feature_gate.go:328] unrecognized feature gate: AutomatedEtcdBackup Jul 10 18:02:51.192113 ip-10-0-128-39 kubenswrapper[2558]: W0710 18:02:50.840285 2558 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesvSphere Jul 10 18:02:51.192113 ip-10-0-128-39 kubenswrapper[2558]: W0710 18:02:50.840287 2558 feature_gate.go:328] unrecognized feature gate: GatewayAPI Jul 10 18:02:51.192113 ip-10-0-128-39 kubenswrapper[2558]: W0710 18:02:50.840292 2558 feature_gate.go:328] unrecognized feature gate: InsightsOnDemandDataGather Jul 10 18:02:51.192113 ip-10-0-128-39 kubenswrapper[2558]: W0710 18:02:50.840295 2558 feature_gate.go:328] unrecognized feature gate: MetricsCollectionProfiles Jul 10 18:02:51.192113 ip-10-0-128-39 kubenswrapper[2558]: W0710 18:02:50.840298 2558 feature_gate.go:328] unrecognized feature gate: NewOLMWebhookProviderOpenshiftServiceCA Jul 10 18:02:51.192113 ip-10-0-128-39 kubenswrapper[2558]: W0710 18:02:50.840300 2558 feature_gate.go:328] unrecognized feature gate: ExternalOIDC Jul 10 18:02:51.193590 ip-10-0-128-39 kubenswrapper[2558]: W0710 18:02:50.840303 2558 feature_gate.go:328] unrecognized feature gate: GCPCustomAPIEndpointsInstall Jul 10 18:02:51.193590 ip-10-0-128-39 kubenswrapper[2558]: W0710 18:02:50.840306 2558 feature_gate.go:328] unrecognized feature gate: IrreconcilableMachineConfig Jul 10 18:02:51.193590 ip-10-0-128-39 kubenswrapper[2558]: W0710 18:02:50.840308 2558 feature_gate.go:328] unrecognized feature gate: GCPClusterHostedDNSInstall Jul 10 18:02:51.193590 ip-10-0-128-39 kubenswrapper[2558]: W0710 18:02:50.840311 2558 feature_gate.go:328] unrecognized feature gate: EtcdBackendQuota Jul 10 18:02:51.193590 ip-10-0-128-39 kubenswrapper[2558]: W0710 18:02:50.840313 2558 feature_gate.go:328] unrecognized feature gate: AzureDedicatedHosts Jul 10 18:02:51.193590 ip-10-0-128-39 kubenswrapper[2558]: W0710 18:02:50.840316 2558 feature_gate.go:328] unrecognized feature gate: GCPClusterHostedDNS Jul 10 18:02:51.193590 ip-10-0-128-39 kubenswrapper[2558]: W0710 18:02:50.840319 2558 feature_gate.go:328] unrecognized feature gate: NewOLMOwnSingleNamespace Jul 10 18:02:51.193590 ip-10-0-128-39 kubenswrapper[2558]: W0710 18:02:50.840321 2558 feature_gate.go:328] unrecognized feature gate: ClusterAPIInstall Jul 10 18:02:51.193590 ip-10-0-128-39 kubenswrapper[2558]: W0710 18:02:50.840324 2558 feature_gate.go:328] unrecognized feature gate: NewOLMBoxCutterRuntime Jul 10 18:02:51.193590 ip-10-0-128-39 kubenswrapper[2558]: W0710 18:02:50.840326 2558 feature_gate.go:328] unrecognized feature gate: GCPCustomAPIEndpoints Jul 10 18:02:51.193590 ip-10-0-128-39 kubenswrapper[2558]: W0710 18:02:50.840329 2558 feature_gate.go:328] unrecognized feature gate: OSStreams Jul 10 18:02:51.193590 ip-10-0-128-39 kubenswrapper[2558]: W0710 18:02:50.840331 2558 feature_gate.go:328] unrecognized feature gate: AdminNetworkPolicy Jul 10 18:02:51.193590 ip-10-0-128-39 kubenswrapper[2558]: W0710 18:02:50.840334 2558 feature_gate.go:328] unrecognized feature gate: DNSNameResolver Jul 10 18:02:51.193590 ip-10-0-128-39 kubenswrapper[2558]: W0710 18:02:50.840336 2558 feature_gate.go:328] unrecognized feature gate: GatewayAPIWithoutOLM Jul 10 18:02:51.193590 ip-10-0-128-39 kubenswrapper[2558]: W0710 18:02:50.840339 2558 feature_gate.go:328] unrecognized feature gate: AWSClusterHostedDNS Jul 10 18:02:51.193590 ip-10-0-128-39 kubenswrapper[2558]: W0710 18:02:50.840341 2558 feature_gate.go:328] unrecognized feature gate: ConsolePluginContentSecurityPolicy Jul 10 18:02:51.193590 ip-10-0-128-39 kubenswrapper[2558]: W0710 18:02:50.840345 2558 feature_gate.go:328] unrecognized feature gate: ClusterAPIMachineManagementVSphere Jul 10 18:02:51.193590 ip-10-0-128-39 kubenswrapper[2558]: W0710 18:02:50.840349 2558 feature_gate.go:349] Setting deprecated feature gate KMSv1=true. It will be removed in a future release. Jul 10 18:02:51.193590 ip-10-0-128-39 kubenswrapper[2558]: W0710 18:02:50.840353 2558 feature_gate.go:328] unrecognized feature gate: OpenShiftPodSecurityAdmission Jul 10 18:02:51.193590 ip-10-0-128-39 kubenswrapper[2558]: W0710 18:02:50.840355 2558 feature_gate.go:328] unrecognized feature gate: AzureDualStackInstall Jul 10 18:02:51.194609 ip-10-0-128-39 kubenswrapper[2558]: W0710 18:02:50.840358 2558 feature_gate.go:328] unrecognized feature gate: ExternalSnapshotMetadata Jul 10 18:02:51.194609 ip-10-0-128-39 kubenswrapper[2558]: W0710 18:02:50.840361 2558 feature_gate.go:328] unrecognized feature gate: MachineAPIMigration Jul 10 18:02:51.194609 ip-10-0-128-39 kubenswrapper[2558]: W0710 18:02:50.840363 2558 feature_gate.go:328] unrecognized feature gate: SignatureStores Jul 10 18:02:51.194609 ip-10-0-128-39 kubenswrapper[2558]: W0710 18:02:50.840367 2558 feature_gate.go:351] Setting GA feature gate VolumeAttributesClass=true. It will be removed in a future release. Jul 10 18:02:51.194609 ip-10-0-128-39 kubenswrapper[2558]: W0710 18:02:50.840370 2558 feature_gate.go:328] unrecognized feature gate: KMSEncryptionProvider Jul 10 18:02:51.194609 ip-10-0-128-39 kubenswrapper[2558]: W0710 18:02:50.840373 2558 feature_gate.go:328] unrecognized feature gate: BootcNodeManagement Jul 10 18:02:51.194609 ip-10-0-128-39 kubenswrapper[2558]: W0710 18:02:50.840376 2558 feature_gate.go:328] unrecognized feature gate: RouteAdvertisements Jul 10 18:02:51.194609 ip-10-0-128-39 kubenswrapper[2558]: W0710 18:02:50.840379 2558 feature_gate.go:328] unrecognized feature gate: AlibabaPlatform Jul 10 18:02:51.194609 ip-10-0-128-39 kubenswrapper[2558]: W0710 18:02:50.840383 2558 feature_gate.go:328] unrecognized feature gate: GatewayAPIController Jul 10 18:02:51.194609 ip-10-0-128-39 kubenswrapper[2558]: W0710 18:02:50.840385 2558 feature_gate.go:328] unrecognized feature gate: KMSEncryption Jul 10 18:02:51.194609 ip-10-0-128-39 kubenswrapper[2558]: W0710 18:02:50.840388 2558 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesCPMS Jul 10 18:02:51.194609 ip-10-0-128-39 kubenswrapper[2558]: W0710 18:02:50.840391 2558 feature_gate.go:328] unrecognized feature gate: UpgradeStatus Jul 10 18:02:51.194609 ip-10-0-128-39 kubenswrapper[2558]: W0710 18:02:50.840393 2558 feature_gate.go:328] unrecognized feature gate: BuildCSIVolumes Jul 10 18:02:51.194609 ip-10-0-128-39 kubenswrapper[2558]: W0710 18:02:50.840396 2558 feature_gate.go:328] unrecognized feature gate: MixedCPUsAllocation Jul 10 18:02:51.194609 ip-10-0-128-39 kubenswrapper[2558]: W0710 18:02:50.840398 2558 feature_gate.go:328] unrecognized feature gate: ClusterMonitoringConfig Jul 10 18:02:51.194609 ip-10-0-128-39 kubenswrapper[2558]: I0710 18:02:50.840466 2558 flags.go:64] FLAG: --address="0.0.0.0" Jul 10 18:02:51.194609 ip-10-0-128-39 kubenswrapper[2558]: I0710 18:02:50.840473 2558 flags.go:64] FLAG: --allowed-unsafe-sysctls="[]" Jul 10 18:02:51.194609 ip-10-0-128-39 kubenswrapper[2558]: I0710 18:02:50.840479 2558 flags.go:64] FLAG: --anonymous-auth="true" Jul 10 18:02:51.194609 ip-10-0-128-39 kubenswrapper[2558]: I0710 18:02:50.840484 2558 flags.go:64] FLAG: --application-metrics-count-limit="100" Jul 10 18:02:51.194609 ip-10-0-128-39 kubenswrapper[2558]: I0710 18:02:50.840489 2558 flags.go:64] FLAG: --authentication-token-webhook="false" Jul 10 18:02:51.194609 ip-10-0-128-39 kubenswrapper[2558]: I0710 18:02:50.840492 2558 flags.go:64] FLAG: --authentication-token-webhook-cache-ttl="2m0s" Jul 10 18:02:51.195482 ip-10-0-128-39 kubenswrapper[2558]: I0710 18:02:50.840496 2558 flags.go:64] FLAG: --authorization-mode="AlwaysAllow" Jul 10 18:02:51.195482 ip-10-0-128-39 kubenswrapper[2558]: I0710 18:02:50.840500 2558 flags.go:64] FLAG: --authorization-webhook-cache-authorized-ttl="5m0s" Jul 10 18:02:51.195482 ip-10-0-128-39 kubenswrapper[2558]: I0710 18:02:50.840503 2558 flags.go:64] FLAG: --authorization-webhook-cache-unauthorized-ttl="30s" Jul 10 18:02:51.195482 ip-10-0-128-39 kubenswrapper[2558]: I0710 18:02:50.840506 2558 flags.go:64] FLAG: --boot-id-file="/proc/sys/kernel/random/boot_id" Jul 10 18:02:51.195482 ip-10-0-128-39 kubenswrapper[2558]: I0710 18:02:50.840509 2558 flags.go:64] FLAG: --bootstrap-kubeconfig="/etc/kubernetes/kubeconfig" Jul 10 18:02:51.195482 ip-10-0-128-39 kubenswrapper[2558]: I0710 18:02:50.840513 2558 flags.go:64] FLAG: --cert-dir="/var/lib/kubelet/pki" Jul 10 18:02:51.195482 ip-10-0-128-39 kubenswrapper[2558]: I0710 18:02:50.840516 2558 flags.go:64] FLAG: --cgroup-driver="cgroupfs" Jul 10 18:02:51.195482 ip-10-0-128-39 kubenswrapper[2558]: I0710 18:02:50.840519 2558 flags.go:64] FLAG: --cgroup-root="" Jul 10 18:02:51.195482 ip-10-0-128-39 kubenswrapper[2558]: I0710 18:02:50.840522 2558 flags.go:64] FLAG: --cgroups-per-qos="true" Jul 10 18:02:51.195482 ip-10-0-128-39 kubenswrapper[2558]: I0710 18:02:50.840525 2558 flags.go:64] FLAG: --client-ca-file="" Jul 10 18:02:51.195482 ip-10-0-128-39 kubenswrapper[2558]: I0710 18:02:50.840528 2558 flags.go:64] FLAG: --cloud-provider="external" Jul 10 18:02:51.195482 ip-10-0-128-39 kubenswrapper[2558]: I0710 18:02:50.840530 2558 flags.go:64] FLAG: --cluster-dns="[]" Jul 10 18:02:51.195482 ip-10-0-128-39 kubenswrapper[2558]: I0710 18:02:50.840534 2558 flags.go:64] FLAG: --cluster-domain="" Jul 10 18:02:51.195482 ip-10-0-128-39 kubenswrapper[2558]: I0710 18:02:50.840537 2558 flags.go:64] FLAG: --config="/etc/kubernetes/kubelet.conf" Jul 10 18:02:51.195482 ip-10-0-128-39 kubenswrapper[2558]: I0710 18:02:50.840540 2558 flags.go:64] FLAG: --config-dir="/etc/openshift/kubelet.conf.d" Jul 10 18:02:51.195482 ip-10-0-128-39 kubenswrapper[2558]: I0710 18:02:50.840543 2558 flags.go:64] FLAG: --container-hints="/etc/cadvisor/container_hints.json" Jul 10 18:02:51.195482 ip-10-0-128-39 kubenswrapper[2558]: I0710 18:02:50.840547 2558 flags.go:64] FLAG: --container-log-max-files="5" Jul 10 18:02:51.195482 ip-10-0-128-39 kubenswrapper[2558]: I0710 18:02:50.840550 2558 flags.go:64] FLAG: --container-log-max-size="10Mi" Jul 10 18:02:51.195482 ip-10-0-128-39 kubenswrapper[2558]: I0710 18:02:50.840553 2558 flags.go:64] FLAG: --container-runtime-endpoint="/var/run/crio/crio.sock" Jul 10 18:02:51.195482 ip-10-0-128-39 kubenswrapper[2558]: I0710 18:02:50.840559 2558 flags.go:64] FLAG: --containerd="/run/containerd/containerd.sock" Jul 10 18:02:51.195482 ip-10-0-128-39 kubenswrapper[2558]: I0710 18:02:50.840563 2558 flags.go:64] FLAG: --containerd-namespace="k8s.io" Jul 10 18:02:51.195482 ip-10-0-128-39 kubenswrapper[2558]: I0710 18:02:50.840566 2558 flags.go:64] FLAG: --contention-profiling="false" Jul 10 18:02:51.195482 ip-10-0-128-39 kubenswrapper[2558]: I0710 18:02:50.840569 2558 flags.go:64] FLAG: --cpu-cfs-quota="true" Jul 10 18:02:51.195482 ip-10-0-128-39 kubenswrapper[2558]: I0710 18:02:50.840572 2558 flags.go:64] FLAG: --cpu-cfs-quota-period="100ms" Jul 10 18:02:51.196571 ip-10-0-128-39 kubenswrapper[2558]: I0710 18:02:50.840575 2558 flags.go:64] FLAG: --cpu-manager-policy="none" Jul 10 18:02:51.196571 ip-10-0-128-39 kubenswrapper[2558]: I0710 18:02:50.840577 2558 flags.go:64] FLAG: --cpu-manager-policy-options="" Jul 10 18:02:51.196571 ip-10-0-128-39 kubenswrapper[2558]: I0710 18:02:50.840582 2558 flags.go:64] FLAG: --cpu-manager-reconcile-period="10s" Jul 10 18:02:51.196571 ip-10-0-128-39 kubenswrapper[2558]: I0710 18:02:50.840584 2558 flags.go:64] FLAG: --enable-controller-attach-detach="true" Jul 10 18:02:51.196571 ip-10-0-128-39 kubenswrapper[2558]: I0710 18:02:50.840588 2558 flags.go:64] FLAG: --enable-debugging-handlers="true" Jul 10 18:02:51.196571 ip-10-0-128-39 kubenswrapper[2558]: I0710 18:02:50.840591 2558 flags.go:64] FLAG: --enable-load-reader="false" Jul 10 18:02:51.196571 ip-10-0-128-39 kubenswrapper[2558]: I0710 18:02:50.840594 2558 flags.go:64] FLAG: --enable-server="true" Jul 10 18:02:51.196571 ip-10-0-128-39 kubenswrapper[2558]: I0710 18:02:50.840597 2558 flags.go:64] FLAG: --enforce-node-allocatable="[pods]" Jul 10 18:02:51.196571 ip-10-0-128-39 kubenswrapper[2558]: I0710 18:02:50.840602 2558 flags.go:64] FLAG: --event-burst="100" Jul 10 18:02:51.196571 ip-10-0-128-39 kubenswrapper[2558]: I0710 18:02:50.840606 2558 flags.go:64] FLAG: --event-qps="50" Jul 10 18:02:51.196571 ip-10-0-128-39 kubenswrapper[2558]: I0710 18:02:50.840608 2558 flags.go:64] FLAG: --event-storage-age-limit="default=0" Jul 10 18:02:51.196571 ip-10-0-128-39 kubenswrapper[2558]: I0710 18:02:50.840612 2558 flags.go:64] FLAG: --event-storage-event-limit="default=0" Jul 10 18:02:51.196571 ip-10-0-128-39 kubenswrapper[2558]: I0710 18:02:50.840614 2558 flags.go:64] FLAG: --eviction-hard="" Jul 10 18:02:51.196571 ip-10-0-128-39 kubenswrapper[2558]: I0710 18:02:50.840618 2558 flags.go:64] FLAG: --eviction-max-pod-grace-period="0" Jul 10 18:02:51.196571 ip-10-0-128-39 kubenswrapper[2558]: I0710 18:02:50.840621 2558 flags.go:64] FLAG: --eviction-minimum-reclaim="" Jul 10 18:02:51.196571 ip-10-0-128-39 kubenswrapper[2558]: I0710 18:02:50.840626 2558 flags.go:64] FLAG: --eviction-pressure-transition-period="5m0s" Jul 10 18:02:51.196571 ip-10-0-128-39 kubenswrapper[2558]: I0710 18:02:50.840629 2558 flags.go:64] FLAG: --eviction-soft="" Jul 10 18:02:51.196571 ip-10-0-128-39 kubenswrapper[2558]: I0710 18:02:50.840632 2558 flags.go:64] FLAG: --eviction-soft-grace-period="" Jul 10 18:02:51.196571 ip-10-0-128-39 kubenswrapper[2558]: I0710 18:02:50.840635 2558 flags.go:64] FLAG: --exit-on-lock-contention="false" Jul 10 18:02:51.196571 ip-10-0-128-39 kubenswrapper[2558]: I0710 18:02:50.840638 2558 flags.go:64] FLAG: --experimental-allocatable-ignore-eviction="false" Jul 10 18:02:51.196571 ip-10-0-128-39 kubenswrapper[2558]: I0710 18:02:50.840641 2558 flags.go:64] FLAG: --experimental-mounter-path="" Jul 10 18:02:51.196571 ip-10-0-128-39 kubenswrapper[2558]: I0710 18:02:50.840644 2558 flags.go:64] FLAG: --fail-cgroupv1="false" Jul 10 18:02:51.196571 ip-10-0-128-39 kubenswrapper[2558]: I0710 18:02:50.840647 2558 flags.go:64] FLAG: --fail-swap-on="true" Jul 10 18:02:51.196571 ip-10-0-128-39 kubenswrapper[2558]: I0710 18:02:50.840650 2558 flags.go:64] FLAG: --feature-gates="" Jul 10 18:02:51.196571 ip-10-0-128-39 kubenswrapper[2558]: I0710 18:02:50.840654 2558 flags.go:64] FLAG: --file-check-frequency="20s" Jul 10 18:02:51.197647 ip-10-0-128-39 kubenswrapper[2558]: I0710 18:02:50.840657 2558 flags.go:64] FLAG: --global-housekeeping-interval="1m0s" Jul 10 18:02:51.197647 ip-10-0-128-39 kubenswrapper[2558]: I0710 18:02:50.840660 2558 flags.go:64] FLAG: --hairpin-mode="promiscuous-bridge" Jul 10 18:02:51.197647 ip-10-0-128-39 kubenswrapper[2558]: I0710 18:02:50.840665 2558 flags.go:64] FLAG: --healthz-bind-address="127.0.0.1" Jul 10 18:02:51.197647 ip-10-0-128-39 kubenswrapper[2558]: I0710 18:02:50.840668 2558 flags.go:64] FLAG: --healthz-port="10248" Jul 10 18:02:51.197647 ip-10-0-128-39 kubenswrapper[2558]: I0710 18:02:50.840671 2558 flags.go:64] FLAG: --help="false" Jul 10 18:02:51.197647 ip-10-0-128-39 kubenswrapper[2558]: I0710 18:02:50.840673 2558 flags.go:64] FLAG: --hostname-override="ip-10-0-128-39.ec2.internal" Jul 10 18:02:51.197647 ip-10-0-128-39 kubenswrapper[2558]: I0710 18:02:50.840676 2558 flags.go:64] FLAG: --housekeeping-interval="10s" Jul 10 18:02:51.197647 ip-10-0-128-39 kubenswrapper[2558]: I0710 18:02:50.840679 2558 flags.go:64] FLAG: --http-check-frequency="20s" Jul 10 18:02:51.197647 ip-10-0-128-39 kubenswrapper[2558]: I0710 18:02:50.840682 2558 flags.go:64] FLAG: --image-credential-provider-bin-dir="/usr/libexec/kubelet-image-credential-provider-plugins" Jul 10 18:02:51.197647 ip-10-0-128-39 kubenswrapper[2558]: I0710 18:02:50.840685 2558 flags.go:64] FLAG: --image-credential-provider-config="/etc/kubernetes/credential-providers/ecr-credential-provider.yaml" Jul 10 18:02:51.197647 ip-10-0-128-39 kubenswrapper[2558]: I0710 18:02:50.840688 2558 flags.go:64] FLAG: --image-gc-high-threshold="85" Jul 10 18:02:51.197647 ip-10-0-128-39 kubenswrapper[2558]: I0710 18:02:50.840691 2558 flags.go:64] FLAG: --image-gc-low-threshold="80" Jul 10 18:02:51.197647 ip-10-0-128-39 kubenswrapper[2558]: I0710 18:02:50.840694 2558 flags.go:64] FLAG: --image-service-endpoint="" Jul 10 18:02:51.197647 ip-10-0-128-39 kubenswrapper[2558]: I0710 18:02:50.840697 2558 flags.go:64] FLAG: --kernel-memcg-notification="false" Jul 10 18:02:51.197647 ip-10-0-128-39 kubenswrapper[2558]: I0710 18:02:50.840700 2558 flags.go:64] FLAG: --kube-api-burst="100" Jul 10 18:02:51.197647 ip-10-0-128-39 kubenswrapper[2558]: I0710 18:02:50.840703 2558 flags.go:64] FLAG: --kube-api-content-type="application/vnd.kubernetes.protobuf" Jul 10 18:02:51.197647 ip-10-0-128-39 kubenswrapper[2558]: I0710 18:02:50.840706 2558 flags.go:64] FLAG: --kube-api-qps="50" Jul 10 18:02:51.197647 ip-10-0-128-39 kubenswrapper[2558]: I0710 18:02:50.840709 2558 flags.go:64] FLAG: --kube-reserved="" Jul 10 18:02:51.197647 ip-10-0-128-39 kubenswrapper[2558]: I0710 18:02:50.840711 2558 flags.go:64] FLAG: --kube-reserved-cgroup="" Jul 10 18:02:51.197647 ip-10-0-128-39 kubenswrapper[2558]: I0710 18:02:50.840714 2558 flags.go:64] FLAG: --kubeconfig="/var/lib/kubelet/kubeconfig" Jul 10 18:02:51.197647 ip-10-0-128-39 kubenswrapper[2558]: I0710 18:02:50.840717 2558 flags.go:64] FLAG: --kubelet-cgroups="" Jul 10 18:02:51.197647 ip-10-0-128-39 kubenswrapper[2558]: I0710 18:02:50.840719 2558 flags.go:64] FLAG: --local-storage-capacity-isolation="true" Jul 10 18:02:51.197647 ip-10-0-128-39 kubenswrapper[2558]: I0710 18:02:50.840725 2558 flags.go:64] FLAG: --lock-file="" Jul 10 18:02:51.197647 ip-10-0-128-39 kubenswrapper[2558]: I0710 18:02:50.840728 2558 flags.go:64] FLAG: --log-cadvisor-usage="false" Jul 10 18:02:51.198735 ip-10-0-128-39 kubenswrapper[2558]: I0710 18:02:50.840730 2558 flags.go:64] FLAG: --log-flush-frequency="5s" Jul 10 18:02:51.198735 ip-10-0-128-39 kubenswrapper[2558]: I0710 18:02:50.840733 2558 flags.go:64] FLAG: --log-json-info-buffer-size="0" Jul 10 18:02:51.198735 ip-10-0-128-39 kubenswrapper[2558]: I0710 18:02:50.840738 2558 flags.go:64] FLAG: --log-json-split-stream="false" Jul 10 18:02:51.198735 ip-10-0-128-39 kubenswrapper[2558]: I0710 18:02:50.840741 2558 flags.go:64] FLAG: --log-text-info-buffer-size="0" Jul 10 18:02:51.198735 ip-10-0-128-39 kubenswrapper[2558]: I0710 18:02:50.840744 2558 flags.go:64] FLAG: --log-text-split-stream="false" Jul 10 18:02:51.198735 ip-10-0-128-39 kubenswrapper[2558]: I0710 18:02:50.840747 2558 flags.go:64] FLAG: --logging-format="text" Jul 10 18:02:51.198735 ip-10-0-128-39 kubenswrapper[2558]: I0710 18:02:50.840749 2558 flags.go:64] FLAG: --machine-id-file="/etc/machine-id,/var/lib/dbus/machine-id" Jul 10 18:02:51.198735 ip-10-0-128-39 kubenswrapper[2558]: I0710 18:02:50.840753 2558 flags.go:64] FLAG: --make-iptables-util-chains="true" Jul 10 18:02:51.198735 ip-10-0-128-39 kubenswrapper[2558]: I0710 18:02:50.840755 2558 flags.go:64] FLAG: --manifest-url="" Jul 10 18:02:51.198735 ip-10-0-128-39 kubenswrapper[2558]: I0710 18:02:50.840758 2558 flags.go:64] FLAG: --manifest-url-header="" Jul 10 18:02:51.198735 ip-10-0-128-39 kubenswrapper[2558]: I0710 18:02:50.840764 2558 flags.go:64] FLAG: --max-housekeeping-interval="15s" Jul 10 18:02:51.198735 ip-10-0-128-39 kubenswrapper[2558]: I0710 18:02:50.840767 2558 flags.go:64] FLAG: --max-open-files="1000000" Jul 10 18:02:51.198735 ip-10-0-128-39 kubenswrapper[2558]: I0710 18:02:50.840771 2558 flags.go:64] FLAG: --max-pods="110" Jul 10 18:02:51.198735 ip-10-0-128-39 kubenswrapper[2558]: I0710 18:02:50.840774 2558 flags.go:64] FLAG: --maximum-dead-containers="-1" Jul 10 18:02:51.198735 ip-10-0-128-39 kubenswrapper[2558]: I0710 18:02:50.840777 2558 flags.go:64] FLAG: --maximum-dead-containers-per-container="1" Jul 10 18:02:51.198735 ip-10-0-128-39 kubenswrapper[2558]: I0710 18:02:50.840779 2558 flags.go:64] FLAG: --memory-manager-policy="None" Jul 10 18:02:51.198735 ip-10-0-128-39 kubenswrapper[2558]: I0710 18:02:50.840782 2558 flags.go:64] FLAG: --minimum-container-ttl-duration="6m0s" Jul 10 18:02:51.198735 ip-10-0-128-39 kubenswrapper[2558]: I0710 18:02:50.840785 2558 flags.go:64] FLAG: --minimum-image-ttl-duration="2m0s" Jul 10 18:02:51.198735 ip-10-0-128-39 kubenswrapper[2558]: I0710 18:02:50.840788 2558 flags.go:64] FLAG: --node-ip="0.0.0.0" Jul 10 18:02:51.198735 ip-10-0-128-39 kubenswrapper[2558]: I0710 18:02:50.840791 2558 flags.go:64] FLAG: --node-labels="node-role.kubernetes.io/worker=,node.openshift.io/os_id=rhel" Jul 10 18:02:51.198735 ip-10-0-128-39 kubenswrapper[2558]: I0710 18:02:50.840798 2558 flags.go:64] FLAG: --node-status-max-images="50" Jul 10 18:02:51.198735 ip-10-0-128-39 kubenswrapper[2558]: I0710 18:02:50.840801 2558 flags.go:64] FLAG: --node-status-update-frequency="10s" Jul 10 18:02:51.198735 ip-10-0-128-39 kubenswrapper[2558]: I0710 18:02:50.840804 2558 flags.go:64] FLAG: --oom-score-adj="-999" Jul 10 18:02:51.198735 ip-10-0-128-39 kubenswrapper[2558]: I0710 18:02:50.840807 2558 flags.go:64] FLAG: --pod-cidr="" Jul 10 18:02:51.202068 ip-10-0-128-39 kubenswrapper[2558]: I0710 18:02:50.840810 2558 flags.go:64] FLAG: --pod-infra-container-image="quay.io/openshift-release-dev/ocp-v4.0-art-dev@sha256:b04c0b1f73af4d021dd8f0b51bfeb302c24482c093de5f2c43654ab56c506926" Jul 10 18:02:51.202068 ip-10-0-128-39 kubenswrapper[2558]: I0710 18:02:50.840816 2558 flags.go:64] FLAG: --pod-manifest-path="" Jul 10 18:02:51.202068 ip-10-0-128-39 kubenswrapper[2558]: I0710 18:02:50.840818 2558 flags.go:64] FLAG: --pod-max-pids="-1" Jul 10 18:02:51.202068 ip-10-0-128-39 kubenswrapper[2558]: I0710 18:02:50.840821 2558 flags.go:64] FLAG: --pods-per-core="0" Jul 10 18:02:51.202068 ip-10-0-128-39 kubenswrapper[2558]: I0710 18:02:50.840824 2558 flags.go:64] FLAG: --port="10250" Jul 10 18:02:51.202068 ip-10-0-128-39 kubenswrapper[2558]: I0710 18:02:50.840827 2558 flags.go:64] FLAG: --protect-kernel-defaults="false" Jul 10 18:02:51.202068 ip-10-0-128-39 kubenswrapper[2558]: I0710 18:02:50.840831 2558 flags.go:64] FLAG: --provider-id="aws:///us-east-1a/i-0f5da4a9cd16413da" Jul 10 18:02:51.202068 ip-10-0-128-39 kubenswrapper[2558]: I0710 18:02:50.840835 2558 flags.go:64] FLAG: --qos-reserved="" Jul 10 18:02:51.202068 ip-10-0-128-39 kubenswrapper[2558]: I0710 18:02:50.840837 2558 flags.go:64] FLAG: --read-only-port="10255" Jul 10 18:02:51.202068 ip-10-0-128-39 kubenswrapper[2558]: I0710 18:02:50.840840 2558 flags.go:64] FLAG: --register-node="true" Jul 10 18:02:51.202068 ip-10-0-128-39 kubenswrapper[2558]: I0710 18:02:50.840843 2558 flags.go:64] FLAG: --register-with-taints="" Jul 10 18:02:51.202068 ip-10-0-128-39 kubenswrapper[2558]: I0710 18:02:50.840847 2558 flags.go:64] FLAG: --registry-burst="10" Jul 10 18:02:51.202068 ip-10-0-128-39 kubenswrapper[2558]: I0710 18:02:50.840849 2558 flags.go:64] FLAG: --registry-qps="5" Jul 10 18:02:51.202068 ip-10-0-128-39 kubenswrapper[2558]: I0710 18:02:50.840852 2558 flags.go:64] FLAG: --reserved-cpus="" Jul 10 18:02:51.202068 ip-10-0-128-39 kubenswrapper[2558]: I0710 18:02:50.840855 2558 flags.go:64] FLAG: --reserved-memory="" Jul 10 18:02:51.202068 ip-10-0-128-39 kubenswrapper[2558]: I0710 18:02:50.840859 2558 flags.go:64] FLAG: --resolv-conf="/etc/resolv.conf" Jul 10 18:02:51.202068 ip-10-0-128-39 kubenswrapper[2558]: I0710 18:02:50.840862 2558 flags.go:64] FLAG: --root-dir="/var/lib/kubelet" Jul 10 18:02:51.202068 ip-10-0-128-39 kubenswrapper[2558]: I0710 18:02:50.840865 2558 flags.go:64] FLAG: --rotate-certificates="false" Jul 10 18:02:51.202068 ip-10-0-128-39 kubenswrapper[2558]: I0710 18:02:50.840867 2558 flags.go:64] FLAG: --rotate-server-certificates="false" Jul 10 18:02:51.202068 ip-10-0-128-39 kubenswrapper[2558]: I0710 18:02:50.840872 2558 flags.go:64] FLAG: --runonce="false" Jul 10 18:02:51.202068 ip-10-0-128-39 kubenswrapper[2558]: I0710 18:02:50.840875 2558 flags.go:64] FLAG: --runtime-cgroups="/system.slice/crio.service" Jul 10 18:02:51.202068 ip-10-0-128-39 kubenswrapper[2558]: I0710 18:02:50.840878 2558 flags.go:64] FLAG: --runtime-request-timeout="2m0s" Jul 10 18:02:51.202068 ip-10-0-128-39 kubenswrapper[2558]: I0710 18:02:50.840881 2558 flags.go:64] FLAG: --seccomp-default="false" Jul 10 18:02:51.202068 ip-10-0-128-39 kubenswrapper[2558]: I0710 18:02:50.840884 2558 flags.go:64] FLAG: --serialize-image-pulls="true" Jul 10 18:02:51.202068 ip-10-0-128-39 kubenswrapper[2558]: I0710 18:02:50.840887 2558 flags.go:64] FLAG: --storage-driver-buffer-duration="1m0s" Jul 10 18:02:51.203142 ip-10-0-128-39 kubenswrapper[2558]: I0710 18:02:50.840890 2558 flags.go:64] FLAG: --storage-driver-db="cadvisor" Jul 10 18:02:51.203142 ip-10-0-128-39 kubenswrapper[2558]: I0710 18:02:50.840893 2558 flags.go:64] FLAG: --storage-driver-host="localhost:8086" Jul 10 18:02:51.203142 ip-10-0-128-39 kubenswrapper[2558]: I0710 18:02:50.840896 2558 flags.go:64] FLAG: --storage-driver-password="root" Jul 10 18:02:51.203142 ip-10-0-128-39 kubenswrapper[2558]: I0710 18:02:50.840899 2558 flags.go:64] FLAG: --storage-driver-secure="false" Jul 10 18:02:51.203142 ip-10-0-128-39 kubenswrapper[2558]: I0710 18:02:50.840902 2558 flags.go:64] FLAG: --storage-driver-table="stats" Jul 10 18:02:51.203142 ip-10-0-128-39 kubenswrapper[2558]: I0710 18:02:50.840904 2558 flags.go:64] FLAG: --storage-driver-user="root" Jul 10 18:02:51.203142 ip-10-0-128-39 kubenswrapper[2558]: I0710 18:02:50.840907 2558 flags.go:64] FLAG: --streaming-connection-idle-timeout="4h0m0s" Jul 10 18:02:51.203142 ip-10-0-128-39 kubenswrapper[2558]: I0710 18:02:50.840910 2558 flags.go:64] FLAG: --sync-frequency="1m0s" Jul 10 18:02:51.203142 ip-10-0-128-39 kubenswrapper[2558]: I0710 18:02:50.840914 2558 flags.go:64] FLAG: --system-cgroups="" Jul 10 18:02:51.203142 ip-10-0-128-39 kubenswrapper[2558]: I0710 18:02:50.840916 2558 flags.go:64] FLAG: --system-reserved="cpu=500m,ephemeral-storage=1Gi,memory=1Gi" Jul 10 18:02:51.203142 ip-10-0-128-39 kubenswrapper[2558]: I0710 18:02:50.840922 2558 flags.go:64] FLAG: --system-reserved-cgroup="" Jul 10 18:02:51.203142 ip-10-0-128-39 kubenswrapper[2558]: I0710 18:02:50.840925 2558 flags.go:64] FLAG: --tls-cert-file="" Jul 10 18:02:51.203142 ip-10-0-128-39 kubenswrapper[2558]: I0710 18:02:50.840927 2558 flags.go:64] FLAG: --tls-cipher-suites="[]" Jul 10 18:02:51.203142 ip-10-0-128-39 kubenswrapper[2558]: I0710 18:02:50.840933 2558 flags.go:64] FLAG: --tls-min-version="" Jul 10 18:02:51.203142 ip-10-0-128-39 kubenswrapper[2558]: I0710 18:02:50.840935 2558 flags.go:64] FLAG: --tls-private-key-file="" Jul 10 18:02:51.203142 ip-10-0-128-39 kubenswrapper[2558]: I0710 18:02:50.840938 2558 flags.go:64] FLAG: --topology-manager-policy="none" Jul 10 18:02:51.203142 ip-10-0-128-39 kubenswrapper[2558]: I0710 18:02:50.840941 2558 flags.go:64] FLAG: --topology-manager-policy-options="" Jul 10 18:02:51.203142 ip-10-0-128-39 kubenswrapper[2558]: I0710 18:02:50.840943 2558 flags.go:64] FLAG: --topology-manager-scope="container" Jul 10 18:02:51.203142 ip-10-0-128-39 kubenswrapper[2558]: I0710 18:02:50.840946 2558 flags.go:64] FLAG: --v="2" Jul 10 18:02:51.203142 ip-10-0-128-39 kubenswrapper[2558]: I0710 18:02:50.840950 2558 flags.go:64] FLAG: --version="false" Jul 10 18:02:51.203142 ip-10-0-128-39 kubenswrapper[2558]: I0710 18:02:50.840954 2558 flags.go:64] FLAG: --vmodule="" Jul 10 18:02:51.203142 ip-10-0-128-39 kubenswrapper[2558]: I0710 18:02:50.840958 2558 flags.go:64] FLAG: --volume-plugin-dir="/etc/kubernetes/kubelet-plugins/volume/exec" Jul 10 18:02:51.203142 ip-10-0-128-39 kubenswrapper[2558]: I0710 18:02:50.840961 2558 flags.go:64] FLAG: --volume-stats-agg-period="1m0s" Jul 10 18:02:51.203142 ip-10-0-128-39 kubenswrapper[2558]: W0710 18:02:50.841078 2558 feature_gate.go:328] unrecognized feature gate: NewOLMPreflightPermissionChecks Jul 10 18:02:51.203142 ip-10-0-128-39 kubenswrapper[2558]: W0710 18:02:50.841082 2558 feature_gate.go:328] unrecognized feature gate: NewOLMCatalogdAPIV1Metas Jul 10 18:02:51.206982 ip-10-0-128-39 kubenswrapper[2558]: W0710 18:02:50.841086 2558 feature_gate.go:328] unrecognized feature gate: OVNObservability Jul 10 18:02:51.206982 ip-10-0-128-39 kubenswrapper[2558]: W0710 18:02:50.841092 2558 feature_gate.go:351] Setting GA feature gate VolumeAttributesClass=true. It will be removed in a future release. Jul 10 18:02:51.206982 ip-10-0-128-39 kubenswrapper[2558]: W0710 18:02:50.841096 2558 feature_gate.go:328] unrecognized feature gate: UpgradeStatus Jul 10 18:02:51.206982 ip-10-0-128-39 kubenswrapper[2558]: W0710 18:02:50.841098 2558 feature_gate.go:328] unrecognized feature gate: NutanixMultiSubnets Jul 10 18:02:51.206982 ip-10-0-128-39 kubenswrapper[2558]: W0710 18:02:50.841101 2558 feature_gate.go:328] unrecognized feature gate: InsightsConfig Jul 10 18:02:51.206982 ip-10-0-128-39 kubenswrapper[2558]: W0710 18:02:50.841104 2558 feature_gate.go:328] unrecognized feature gate: AutomatedEtcdBackup Jul 10 18:02:51.206982 ip-10-0-128-39 kubenswrapper[2558]: W0710 18:02:50.841107 2558 feature_gate.go:328] unrecognized feature gate: IrreconcilableMachineConfig Jul 10 18:02:51.206982 ip-10-0-128-39 kubenswrapper[2558]: W0710 18:02:50.841110 2558 feature_gate.go:328] unrecognized feature gate: DyanmicServiceEndpointIBMCloud Jul 10 18:02:51.206982 ip-10-0-128-39 kubenswrapper[2558]: W0710 18:02:50.841113 2558 feature_gate.go:328] unrecognized feature gate: HighlyAvailableArbiter Jul 10 18:02:51.206982 ip-10-0-128-39 kubenswrapper[2558]: W0710 18:02:50.841116 2558 feature_gate.go:328] unrecognized feature gate: ManagedBootImages Jul 10 18:02:51.206982 ip-10-0-128-39 kubenswrapper[2558]: W0710 18:02:50.841118 2558 feature_gate.go:328] unrecognized feature gate: VSphereMultiNetworks Jul 10 18:02:51.206982 ip-10-0-128-39 kubenswrapper[2558]: W0710 18:02:50.841121 2558 feature_gate.go:328] unrecognized feature gate: NewOLMBoxCutterRuntime Jul 10 18:02:51.206982 ip-10-0-128-39 kubenswrapper[2558]: W0710 18:02:50.841124 2558 feature_gate.go:328] unrecognized feature gate: InsightsOnDemandDataGather Jul 10 18:02:51.206982 ip-10-0-128-39 kubenswrapper[2558]: W0710 18:02:50.841126 2558 feature_gate.go:328] unrecognized feature gate: ExternalOIDCWithUIDAndExtraClaimMappings Jul 10 18:02:51.206982 ip-10-0-128-39 kubenswrapper[2558]: W0710 18:02:50.841129 2558 feature_gate.go:328] unrecognized feature gate: ClusterVersionOperatorConfiguration Jul 10 18:02:51.206982 ip-10-0-128-39 kubenswrapper[2558]: W0710 18:02:50.841132 2558 feature_gate.go:328] unrecognized feature gate: AzureMultiDisk Jul 10 18:02:51.206982 ip-10-0-128-39 kubenswrapper[2558]: W0710 18:02:50.841134 2558 feature_gate.go:328] unrecognized feature gate: MultiArchInstallAzure Jul 10 18:02:51.206982 ip-10-0-128-39 kubenswrapper[2558]: W0710 18:02:50.841137 2558 feature_gate.go:328] unrecognized feature gate: AWSClusterHostedDNSInstall Jul 10 18:02:51.206982 ip-10-0-128-39 kubenswrapper[2558]: W0710 18:02:50.841140 2558 feature_gate.go:328] unrecognized feature gate: AzureClusterHostedDNSInstall Jul 10 18:02:51.207543 ip-10-0-128-39 kubenswrapper[2558]: W0710 18:02:50.841144 2558 feature_gate.go:349] Setting deprecated feature gate KMSv1=true. It will be removed in a future release. Jul 10 18:02:51.207543 ip-10-0-128-39 kubenswrapper[2558]: W0710 18:02:50.841149 2558 feature_gate.go:328] unrecognized feature gate: HyperShiftOnlyDynamicResourceAllocation Jul 10 18:02:51.207543 ip-10-0-128-39 kubenswrapper[2558]: W0710 18:02:50.841152 2558 feature_gate.go:328] unrecognized feature gate: BuildCSIVolumes Jul 10 18:02:51.207543 ip-10-0-128-39 kubenswrapper[2558]: W0710 18:02:50.841156 2558 feature_gate.go:351] Setting GA feature gate ServiceAccountTokenNodeBinding=true. It will be removed in a future release. Jul 10 18:02:51.207543 ip-10-0-128-39 kubenswrapper[2558]: W0710 18:02:50.841159 2558 feature_gate.go:328] unrecognized feature gate: NewOLMWebhookProviderOpenshiftServiceCA Jul 10 18:02:51.207543 ip-10-0-128-39 kubenswrapper[2558]: W0710 18:02:50.841162 2558 feature_gate.go:328] unrecognized feature gate: Example Jul 10 18:02:51.207543 ip-10-0-128-39 kubenswrapper[2558]: W0710 18:02:50.841165 2558 feature_gate.go:328] unrecognized feature gate: AlibabaPlatform Jul 10 18:02:51.207543 ip-10-0-128-39 kubenswrapper[2558]: W0710 18:02:50.841168 2558 feature_gate.go:328] unrecognized feature gate: CPMSMachineNamePrefix Jul 10 18:02:51.207543 ip-10-0-128-39 kubenswrapper[2558]: W0710 18:02:50.841184 2558 feature_gate.go:328] unrecognized feature gate: IngressControllerDynamicConfigurationManager Jul 10 18:02:51.207543 ip-10-0-128-39 kubenswrapper[2558]: W0710 18:02:50.841187 2558 feature_gate.go:328] unrecognized feature gate: ImageModeStatusReporting Jul 10 18:02:51.207543 ip-10-0-128-39 kubenswrapper[2558]: W0710 18:02:50.841189 2558 feature_gate.go:328] unrecognized feature gate: AWSDualStackInstall Jul 10 18:02:51.207543 ip-10-0-128-39 kubenswrapper[2558]: W0710 18:02:50.841192 2558 feature_gate.go:328] unrecognized feature gate: MachineAPIOperatorDisableMachineHealthCheckController Jul 10 18:02:51.207543 ip-10-0-128-39 kubenswrapper[2558]: W0710 18:02:50.841195 2558 feature_gate.go:328] unrecognized feature gate: ExternalOIDC Jul 10 18:02:51.207543 ip-10-0-128-39 kubenswrapper[2558]: W0710 18:02:50.841198 2558 feature_gate.go:328] unrecognized feature gate: GCPCustomAPIEndpoints Jul 10 18:02:51.207543 ip-10-0-128-39 kubenswrapper[2558]: W0710 18:02:50.841202 2558 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesCPMS Jul 10 18:02:51.207543 ip-10-0-128-39 kubenswrapper[2558]: W0710 18:02:50.841205 2558 feature_gate.go:328] unrecognized feature gate: GatewayAPI Jul 10 18:02:51.207543 ip-10-0-128-39 kubenswrapper[2558]: W0710 18:02:50.841207 2558 feature_gate.go:328] unrecognized feature gate: VolumeGroupSnapshot Jul 10 18:02:51.207543 ip-10-0-128-39 kubenswrapper[2558]: W0710 18:02:50.841210 2558 feature_gate.go:328] unrecognized feature gate: OpenShiftPodSecurityAdmission Jul 10 18:02:51.207543 ip-10-0-128-39 kubenswrapper[2558]: W0710 18:02:50.841213 2558 feature_gate.go:328] unrecognized feature gate: DualReplica Jul 10 18:02:51.208048 ip-10-0-128-39 kubenswrapper[2558]: W0710 18:02:50.841215 2558 feature_gate.go:328] unrecognized feature gate: AzureWorkloadIdentity Jul 10 18:02:51.208048 ip-10-0-128-39 kubenswrapper[2558]: W0710 18:02:50.841218 2558 feature_gate.go:328] unrecognized feature gate: SignatureStores Jul 10 18:02:51.208048 ip-10-0-128-39 kubenswrapper[2558]: W0710 18:02:50.841221 2558 feature_gate.go:328] unrecognized feature gate: NoRegistryClusterInstall Jul 10 18:02:51.208048 ip-10-0-128-39 kubenswrapper[2558]: W0710 18:02:50.841223 2558 feature_gate.go:328] unrecognized feature gate: VSphereMixedNodeEnv Jul 10 18:02:51.208048 ip-10-0-128-39 kubenswrapper[2558]: W0710 18:02:50.841226 2558 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesAzure Jul 10 18:02:51.208048 ip-10-0-128-39 kubenswrapper[2558]: W0710 18:02:50.841229 2558 feature_gate.go:328] unrecognized feature gate: VSphereMultiDisk Jul 10 18:02:51.208048 ip-10-0-128-39 kubenswrapper[2558]: W0710 18:02:50.841231 2558 feature_gate.go:328] unrecognized feature gate: AWSServiceLBNetworkSecurityGroup Jul 10 18:02:51.208048 ip-10-0-128-39 kubenswrapper[2558]: W0710 18:02:50.841234 2558 feature_gate.go:328] unrecognized feature gate: AzureDualStackInstall Jul 10 18:02:51.208048 ip-10-0-128-39 kubenswrapper[2558]: W0710 18:02:50.841236 2558 feature_gate.go:328] unrecognized feature gate: OnPremDNSRecords Jul 10 18:02:51.208048 ip-10-0-128-39 kubenswrapper[2558]: W0710 18:02:50.841239 2558 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesvSphere Jul 10 18:02:51.208048 ip-10-0-128-39 kubenswrapper[2558]: W0710 18:02:50.841242 2558 feature_gate.go:328] unrecognized feature gate: CRDCompatibilityRequirementOperator Jul 10 18:02:51.208048 ip-10-0-128-39 kubenswrapper[2558]: W0710 18:02:50.841244 2558 feature_gate.go:328] unrecognized feature gate: MetricsCollectionProfiles Jul 10 18:02:51.208048 ip-10-0-128-39 kubenswrapper[2558]: W0710 18:02:50.841247 2558 feature_gate.go:328] unrecognized feature gate: ClusterAPIMachineManagementVSphere Jul 10 18:02:51.208048 ip-10-0-128-39 kubenswrapper[2558]: W0710 18:02:50.841250 2558 feature_gate.go:328] unrecognized feature gate: SigstoreImageVerificationPKI Jul 10 18:02:51.208048 ip-10-0-128-39 kubenswrapper[2558]: W0710 18:02:50.841254 2558 feature_gate.go:328] unrecognized feature gate: AdminNetworkPolicy Jul 10 18:02:51.208048 ip-10-0-128-39 kubenswrapper[2558]: W0710 18:02:50.841257 2558 feature_gate.go:328] unrecognized feature gate: SigstoreImageVerification Jul 10 18:02:51.208048 ip-10-0-128-39 kubenswrapper[2558]: W0710 18:02:50.841260 2558 feature_gate.go:328] unrecognized feature gate: GatewayAPIController Jul 10 18:02:51.208048 ip-10-0-128-39 kubenswrapper[2558]: W0710 18:02:50.841262 2558 feature_gate.go:328] unrecognized feature gate: PreconfiguredUDNAddresses Jul 10 18:02:51.208048 ip-10-0-128-39 kubenswrapper[2558]: W0710 18:02:50.841265 2558 feature_gate.go:328] unrecognized feature gate: ClusterAPIInstall Jul 10 18:02:51.208645 ip-10-0-128-39 kubenswrapper[2558]: W0710 18:02:50.841267 2558 feature_gate.go:328] unrecognized feature gate: VSphereConfigurableMaxAllowedBlockVolumesPerNode Jul 10 18:02:51.208645 ip-10-0-128-39 kubenswrapper[2558]: W0710 18:02:50.841269 2558 feature_gate.go:328] unrecognized feature gate: ClusterMonitoringConfig Jul 10 18:02:51.208645 ip-10-0-128-39 kubenswrapper[2558]: W0710 18:02:50.841272 2558 feature_gate.go:328] unrecognized feature gate: RouteAdvertisements Jul 10 18:02:51.208645 ip-10-0-128-39 kubenswrapper[2558]: W0710 18:02:50.841274 2558 feature_gate.go:328] unrecognized feature gate: ClusterAPIMachineManagement Jul 10 18:02:51.208645 ip-10-0-128-39 kubenswrapper[2558]: W0710 18:02:50.841277 2558 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesAWS Jul 10 18:02:51.208645 ip-10-0-128-39 kubenswrapper[2558]: W0710 18:02:50.841279 2558 feature_gate.go:328] unrecognized feature gate: NetworkSegmentation Jul 10 18:02:51.208645 ip-10-0-128-39 kubenswrapper[2558]: W0710 18:02:50.841282 2558 feature_gate.go:328] unrecognized feature gate: Example2 Jul 10 18:02:51.208645 ip-10-0-128-39 kubenswrapper[2558]: W0710 18:02:50.841285 2558 feature_gate.go:328] unrecognized feature gate: VSphereHostVMGroupZonal Jul 10 18:02:51.208645 ip-10-0-128-39 kubenswrapper[2558]: W0710 18:02:50.841288 2558 feature_gate.go:328] unrecognized feature gate: GCPClusterHostedDNSInstall Jul 10 18:02:51.208645 ip-10-0-128-39 kubenswrapper[2558]: W0710 18:02:50.841291 2558 feature_gate.go:328] unrecognized feature gate: NetworkDiagnosticsConfig Jul 10 18:02:51.208645 ip-10-0-128-39 kubenswrapper[2558]: W0710 18:02:50.841294 2558 feature_gate.go:328] unrecognized feature gate: ShortCertRotation Jul 10 18:02:51.208645 ip-10-0-128-39 kubenswrapper[2558]: W0710 18:02:50.841296 2558 feature_gate.go:328] unrecognized feature gate: AzureDedicatedHosts Jul 10 18:02:51.208645 ip-10-0-128-39 kubenswrapper[2558]: W0710 18:02:50.841299 2558 feature_gate.go:328] unrecognized feature gate: EtcdBackendQuota Jul 10 18:02:51.208645 ip-10-0-128-39 kubenswrapper[2558]: W0710 18:02:50.841302 2558 feature_gate.go:328] unrecognized feature gate: GCPCustomAPIEndpointsInstall Jul 10 18:02:51.208645 ip-10-0-128-39 kubenswrapper[2558]: W0710 18:02:50.841304 2558 feature_gate.go:328] unrecognized feature gate: AWSDedicatedHosts Jul 10 18:02:51.208645 ip-10-0-128-39 kubenswrapper[2558]: W0710 18:02:50.841307 2558 feature_gate.go:328] unrecognized feature gate: ProvisioningRequestAvailable Jul 10 18:02:51.208645 ip-10-0-128-39 kubenswrapper[2558]: W0710 18:02:50.841309 2558 feature_gate.go:328] unrecognized feature gate: BootcNodeManagement Jul 10 18:02:51.208645 ip-10-0-128-39 kubenswrapper[2558]: W0710 18:02:50.841312 2558 feature_gate.go:328] unrecognized feature gate: ConsolePluginContentSecurityPolicy Jul 10 18:02:51.208645 ip-10-0-128-39 kubenswrapper[2558]: W0710 18:02:50.841315 2558 feature_gate.go:328] unrecognized feature gate: GCPClusterHostedDNS Jul 10 18:02:51.208645 ip-10-0-128-39 kubenswrapper[2558]: W0710 18:02:50.841317 2558 feature_gate.go:328] unrecognized feature gate: ImageStreamImportMode Jul 10 18:02:51.454930 ip-10-0-128-39 kubenswrapper[2558]: W0710 18:02:50.841320 2558 feature_gate.go:328] unrecognized feature gate: AdditionalRoutingCapabilities Jul 10 18:02:51.454930 ip-10-0-128-39 kubenswrapper[2558]: W0710 18:02:50.841322 2558 feature_gate.go:328] unrecognized feature gate: ClusterAPIInstallIBMCloud Jul 10 18:02:51.454930 ip-10-0-128-39 kubenswrapper[2558]: W0710 18:02:50.841324 2558 feature_gate.go:328] unrecognized feature gate: GCPDualStackInstall Jul 10 18:02:51.454930 ip-10-0-128-39 kubenswrapper[2558]: W0710 18:02:50.841327 2558 feature_gate.go:328] unrecognized feature gate: PinnedImages Jul 10 18:02:51.454930 ip-10-0-128-39 kubenswrapper[2558]: W0710 18:02:50.841329 2558 feature_gate.go:328] unrecognized feature gate: ExternalSnapshotMetadata Jul 10 18:02:51.454930 ip-10-0-128-39 kubenswrapper[2558]: W0710 18:02:50.841332 2558 feature_gate.go:328] unrecognized feature gate: BootImageSkewEnforcement Jul 10 18:02:51.454930 ip-10-0-128-39 kubenswrapper[2558]: W0710 18:02:50.841335 2558 feature_gate.go:328] unrecognized feature gate: MachineAPIMigration Jul 10 18:02:51.454930 ip-10-0-128-39 kubenswrapper[2558]: W0710 18:02:50.841339 2558 feature_gate.go:328] unrecognized feature gate: DNSNameResolver Jul 10 18:02:51.454930 ip-10-0-128-39 kubenswrapper[2558]: W0710 18:02:50.841342 2558 feature_gate.go:328] unrecognized feature gate: OSStreams Jul 10 18:02:51.454930 ip-10-0-128-39 kubenswrapper[2558]: W0710 18:02:50.841344 2558 feature_gate.go:328] unrecognized feature gate: KMSEncryptionProvider Jul 10 18:02:51.454930 ip-10-0-128-39 kubenswrapper[2558]: W0710 18:02:50.841346 2558 feature_gate.go:328] unrecognized feature gate: NewOLM Jul 10 18:02:51.454930 ip-10-0-128-39 kubenswrapper[2558]: W0710 18:02:50.841349 2558 feature_gate.go:328] unrecognized feature gate: MachineConfigNodes Jul 10 18:02:51.454930 ip-10-0-128-39 kubenswrapper[2558]: W0710 18:02:50.841352 2558 feature_gate.go:328] unrecognized feature gate: NewOLMOwnSingleNamespace Jul 10 18:02:51.454930 ip-10-0-128-39 kubenswrapper[2558]: W0710 18:02:50.841354 2558 feature_gate.go:328] unrecognized feature gate: MultiDiskSetup Jul 10 18:02:51.454930 ip-10-0-128-39 kubenswrapper[2558]: W0710 18:02:50.841356 2558 feature_gate.go:328] unrecognized feature gate: EventTTL Jul 10 18:02:51.454930 ip-10-0-128-39 kubenswrapper[2558]: W0710 18:02:50.841359 2558 feature_gate.go:328] unrecognized feature gate: MixedCPUsAllocation Jul 10 18:02:51.454930 ip-10-0-128-39 kubenswrapper[2558]: W0710 18:02:50.841362 2558 feature_gate.go:328] unrecognized feature gate: NetworkLiveMigration Jul 10 18:02:51.454930 ip-10-0-128-39 kubenswrapper[2558]: W0710 18:02:50.841364 2558 feature_gate.go:328] unrecognized feature gate: GatewayAPIWithoutOLM Jul 10 18:02:51.454930 ip-10-0-128-39 kubenswrapper[2558]: W0710 18:02:50.841367 2558 feature_gate.go:328] unrecognized feature gate: KMSEncryption Jul 10 18:02:51.454930 ip-10-0-128-39 kubenswrapper[2558]: W0710 18:02:50.841369 2558 feature_gate.go:328] unrecognized feature gate: AWSClusterHostedDNS Jul 10 18:02:51.456435 ip-10-0-128-39 kubenswrapper[2558]: I0710 18:02:50.841376 2558 feature_gate.go:384] feature gates: {map[CBORServingAndStorage:false ClientsAllowCBOR:false ClientsPreferCBOR:false EventedPLEG:false ImageVolume:true KMSv1:true MaxUnavailableStatefulSet:false MinimumKubeletVersion:false MutableCSINodeAllocatableCount:false MutatingAdmissionPolicy:false ProcMountType:true RouteExternalCertificate:true SELinuxMount:false ServiceAccountTokenNodeBinding:true StoragePerformantSecurityPolicy:true TranslateStreamCloseWebsocketRequests:false UserNamespacesPodSecurityStandards:true UserNamespacesSupport:true VolumeAttributesClass:true]} Jul 10 18:02:51.456435 ip-10-0-128-39 kubenswrapper[2558]: I0710 18:02:50.848505 2558 server.go:529] "Kubelet version" kubeletVersion="v1.34.9" Jul 10 18:02:51.456435 ip-10-0-128-39 kubenswrapper[2558]: I0710 18:02:50.848522 2558 server.go:531] "Golang settings" GOGC="" GOMAXPROCS="" GOTRACEBACK="" Jul 10 18:02:51.456435 ip-10-0-128-39 kubenswrapper[2558]: I0710 18:02:50.848542 2558 watchdog_linux.go:95] "Systemd watchdog is not enabled" Jul 10 18:02:51.456435 ip-10-0-128-39 kubenswrapper[2558]: I0710 18:02:50.848552 2558 watchdog_linux.go:137] "Systemd watchdog is not enabled or the interval is invalid, so health checking will not be started." Jul 10 18:02:51.456435 ip-10-0-128-39 kubenswrapper[2558]: W0710 18:02:50.848610 2558 feature_gate.go:328] unrecognized feature gate: ExternalOIDC Jul 10 18:02:51.456435 ip-10-0-128-39 kubenswrapper[2558]: W0710 18:02:50.848615 2558 feature_gate.go:328] unrecognized feature gate: InsightsConfig Jul 10 18:02:51.456435 ip-10-0-128-39 kubenswrapper[2558]: W0710 18:02:50.848619 2558 feature_gate.go:328] unrecognized feature gate: ImageModeStatusReporting Jul 10 18:02:51.456435 ip-10-0-128-39 kubenswrapper[2558]: W0710 18:02:50.848622 2558 feature_gate.go:328] unrecognized feature gate: ShortCertRotation Jul 10 18:02:51.456435 ip-10-0-128-39 kubenswrapper[2558]: W0710 18:02:50.848625 2558 feature_gate.go:328] unrecognized feature gate: DNSNameResolver Jul 10 18:02:51.456435 ip-10-0-128-39 kubenswrapper[2558]: W0710 18:02:50.848628 2558 feature_gate.go:328] unrecognized feature gate: ClusterAPIMachineManagementVSphere Jul 10 18:02:51.456435 ip-10-0-128-39 kubenswrapper[2558]: W0710 18:02:50.848630 2558 feature_gate.go:328] unrecognized feature gate: GatewayAPIController Jul 10 18:02:51.456435 ip-10-0-128-39 kubenswrapper[2558]: W0710 18:02:50.848633 2558 feature_gate.go:328] unrecognized feature gate: ClusterMonitoringConfig Jul 10 18:02:51.456435 ip-10-0-128-39 kubenswrapper[2558]: W0710 18:02:50.848636 2558 feature_gate.go:328] unrecognized feature gate: VSphereMixedNodeEnv Jul 10 18:02:51.456435 ip-10-0-128-39 kubenswrapper[2558]: W0710 18:02:50.848639 2558 feature_gate.go:328] unrecognized feature gate: ImageStreamImportMode Jul 10 18:02:51.457092 ip-10-0-128-39 kubenswrapper[2558]: W0710 18:02:50.848641 2558 feature_gate.go:328] unrecognized feature gate: SigstoreImageVerification Jul 10 18:02:51.457092 ip-10-0-128-39 kubenswrapper[2558]: W0710 18:02:50.848644 2558 feature_gate.go:328] unrecognized feature gate: MachineAPIOperatorDisableMachineHealthCheckController Jul 10 18:02:51.457092 ip-10-0-128-39 kubenswrapper[2558]: W0710 18:02:50.848647 2558 feature_gate.go:328] unrecognized feature gate: MetricsCollectionProfiles Jul 10 18:02:51.457092 ip-10-0-128-39 kubenswrapper[2558]: W0710 18:02:50.848651 2558 feature_gate.go:349] Setting deprecated feature gate KMSv1=true. It will be removed in a future release. Jul 10 18:02:51.457092 ip-10-0-128-39 kubenswrapper[2558]: W0710 18:02:50.848656 2558 feature_gate.go:351] Setting GA feature gate VolumeAttributesClass=true. It will be removed in a future release. Jul 10 18:02:51.457092 ip-10-0-128-39 kubenswrapper[2558]: W0710 18:02:50.848660 2558 feature_gate.go:328] unrecognized feature gate: VSphereHostVMGroupZonal Jul 10 18:02:51.457092 ip-10-0-128-39 kubenswrapper[2558]: W0710 18:02:50.848664 2558 feature_gate.go:328] unrecognized feature gate: IngressControllerDynamicConfigurationManager Jul 10 18:02:51.457092 ip-10-0-128-39 kubenswrapper[2558]: W0710 18:02:50.848667 2558 feature_gate.go:328] unrecognized feature gate: AzureDualStackInstall Jul 10 18:02:51.457092 ip-10-0-128-39 kubenswrapper[2558]: W0710 18:02:50.848670 2558 feature_gate.go:328] unrecognized feature gate: AlibabaPlatform Jul 10 18:02:51.457092 ip-10-0-128-39 kubenswrapper[2558]: W0710 18:02:50.848672 2558 feature_gate.go:328] unrecognized feature gate: EventTTL Jul 10 18:02:51.457092 ip-10-0-128-39 kubenswrapper[2558]: W0710 18:02:50.848675 2558 feature_gate.go:328] unrecognized feature gate: MixedCPUsAllocation Jul 10 18:02:51.457092 ip-10-0-128-39 kubenswrapper[2558]: W0710 18:02:50.848678 2558 feature_gate.go:328] unrecognized feature gate: ClusterVersionOperatorConfiguration Jul 10 18:02:51.457092 ip-10-0-128-39 kubenswrapper[2558]: W0710 18:02:50.848683 2558 feature_gate.go:328] unrecognized feature gate: GCPCustomAPIEndpoints Jul 10 18:02:51.457092 ip-10-0-128-39 kubenswrapper[2558]: W0710 18:02:50.848686 2558 feature_gate.go:328] unrecognized feature gate: KMSEncryptionProvider Jul 10 18:02:51.457092 ip-10-0-128-39 kubenswrapper[2558]: W0710 18:02:50.848689 2558 feature_gate.go:328] unrecognized feature gate: IrreconcilableMachineConfig Jul 10 18:02:51.457092 ip-10-0-128-39 kubenswrapper[2558]: W0710 18:02:50.848691 2558 feature_gate.go:328] unrecognized feature gate: MultiArchInstallAzure Jul 10 18:02:51.457092 ip-10-0-128-39 kubenswrapper[2558]: W0710 18:02:50.848694 2558 feature_gate.go:328] unrecognized feature gate: NoRegistryClusterInstall Jul 10 18:02:51.457092 ip-10-0-128-39 kubenswrapper[2558]: W0710 18:02:50.848696 2558 feature_gate.go:328] unrecognized feature gate: VolumeGroupSnapshot Jul 10 18:02:51.457092 ip-10-0-128-39 kubenswrapper[2558]: W0710 18:02:50.848699 2558 feature_gate.go:328] unrecognized feature gate: OnPremDNSRecords Jul 10 18:02:51.457971 ip-10-0-128-39 kubenswrapper[2558]: W0710 18:02:50.848702 2558 feature_gate.go:328] unrecognized feature gate: PinnedImages Jul 10 18:02:51.457971 ip-10-0-128-39 kubenswrapper[2558]: W0710 18:02:50.848707 2558 feature_gate.go:351] Setting GA feature gate ServiceAccountTokenNodeBinding=true. It will be removed in a future release. Jul 10 18:02:51.457971 ip-10-0-128-39 kubenswrapper[2558]: W0710 18:02:50.848711 2558 feature_gate.go:328] unrecognized feature gate: ManagedBootImages Jul 10 18:02:51.457971 ip-10-0-128-39 kubenswrapper[2558]: W0710 18:02:50.848714 2558 feature_gate.go:328] unrecognized feature gate: GCPDualStackInstall Jul 10 18:02:51.457971 ip-10-0-128-39 kubenswrapper[2558]: W0710 18:02:50.848717 2558 feature_gate.go:328] unrecognized feature gate: Example2 Jul 10 18:02:51.457971 ip-10-0-128-39 kubenswrapper[2558]: W0710 18:02:50.848720 2558 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesAWS Jul 10 18:02:51.457971 ip-10-0-128-39 kubenswrapper[2558]: W0710 18:02:50.848723 2558 feature_gate.go:328] unrecognized feature gate: ExternalOIDCWithUIDAndExtraClaimMappings Jul 10 18:02:51.457971 ip-10-0-128-39 kubenswrapper[2558]: W0710 18:02:50.848725 2558 feature_gate.go:328] unrecognized feature gate: VSphereMultiNetworks Jul 10 18:02:51.457971 ip-10-0-128-39 kubenswrapper[2558]: W0710 18:02:50.848728 2558 feature_gate.go:328] unrecognized feature gate: BootcNodeManagement Jul 10 18:02:51.457971 ip-10-0-128-39 kubenswrapper[2558]: W0710 18:02:50.848731 2558 feature_gate.go:328] unrecognized feature gate: NewOLM Jul 10 18:02:51.457971 ip-10-0-128-39 kubenswrapper[2558]: W0710 18:02:50.848734 2558 feature_gate.go:328] unrecognized feature gate: AWSServiceLBNetworkSecurityGroup Jul 10 18:02:51.457971 ip-10-0-128-39 kubenswrapper[2558]: W0710 18:02:50.848737 2558 feature_gate.go:328] unrecognized feature gate: NewOLMWebhookProviderOpenshiftServiceCA Jul 10 18:02:51.457971 ip-10-0-128-39 kubenswrapper[2558]: W0710 18:02:50.848740 2558 feature_gate.go:328] unrecognized feature gate: AzureClusterHostedDNSInstall Jul 10 18:02:51.457971 ip-10-0-128-39 kubenswrapper[2558]: W0710 18:02:50.848742 2558 feature_gate.go:328] unrecognized feature gate: GCPClusterHostedDNSInstall Jul 10 18:02:51.457971 ip-10-0-128-39 kubenswrapper[2558]: W0710 18:02:50.848745 2558 feature_gate.go:328] unrecognized feature gate: EtcdBackendQuota Jul 10 18:02:51.457971 ip-10-0-128-39 kubenswrapper[2558]: W0710 18:02:50.848748 2558 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesAzure Jul 10 18:02:51.457971 ip-10-0-128-39 kubenswrapper[2558]: W0710 18:02:50.848751 2558 feature_gate.go:328] unrecognized feature gate: DyanmicServiceEndpointIBMCloud Jul 10 18:02:51.457971 ip-10-0-128-39 kubenswrapper[2558]: W0710 18:02:50.848753 2558 feature_gate.go:328] unrecognized feature gate: OSStreams Jul 10 18:02:51.457971 ip-10-0-128-39 kubenswrapper[2558]: W0710 18:02:50.848756 2558 feature_gate.go:328] unrecognized feature gate: ClusterAPIInstallIBMCloud Jul 10 18:02:51.458837 ip-10-0-128-39 kubenswrapper[2558]: W0710 18:02:50.848758 2558 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesvSphere Jul 10 18:02:51.458837 ip-10-0-128-39 kubenswrapper[2558]: W0710 18:02:50.848761 2558 feature_gate.go:328] unrecognized feature gate: AzureDedicatedHosts Jul 10 18:02:51.458837 ip-10-0-128-39 kubenswrapper[2558]: W0710 18:02:50.848763 2558 feature_gate.go:328] unrecognized feature gate: AWSClusterHostedDNS Jul 10 18:02:51.458837 ip-10-0-128-39 kubenswrapper[2558]: W0710 18:02:50.848766 2558 feature_gate.go:328] unrecognized feature gate: ConsolePluginContentSecurityPolicy Jul 10 18:02:51.458837 ip-10-0-128-39 kubenswrapper[2558]: W0710 18:02:50.848769 2558 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesCPMS Jul 10 18:02:51.458837 ip-10-0-128-39 kubenswrapper[2558]: W0710 18:02:50.848771 2558 feature_gate.go:328] unrecognized feature gate: UpgradeStatus Jul 10 18:02:51.458837 ip-10-0-128-39 kubenswrapper[2558]: W0710 18:02:50.848775 2558 feature_gate.go:328] unrecognized feature gate: AzureMultiDisk Jul 10 18:02:51.458837 ip-10-0-128-39 kubenswrapper[2558]: W0710 18:02:50.848778 2558 feature_gate.go:328] unrecognized feature gate: CPMSMachineNamePrefix Jul 10 18:02:51.458837 ip-10-0-128-39 kubenswrapper[2558]: W0710 18:02:50.848781 2558 feature_gate.go:328] unrecognized feature gate: PreconfiguredUDNAddresses Jul 10 18:02:51.458837 ip-10-0-128-39 kubenswrapper[2558]: W0710 18:02:50.848783 2558 feature_gate.go:328] unrecognized feature gate: OVNObservability Jul 10 18:02:51.458837 ip-10-0-128-39 kubenswrapper[2558]: W0710 18:02:50.848786 2558 feature_gate.go:328] unrecognized feature gate: GatewayAPI Jul 10 18:02:51.458837 ip-10-0-128-39 kubenswrapper[2558]: W0710 18:02:50.848788 2558 feature_gate.go:328] unrecognized feature gate: AWSClusterHostedDNSInstall Jul 10 18:02:51.458837 ip-10-0-128-39 kubenswrapper[2558]: W0710 18:02:50.848791 2558 feature_gate.go:328] unrecognized feature gate: NetworkSegmentation Jul 10 18:02:51.458837 ip-10-0-128-39 kubenswrapper[2558]: W0710 18:02:50.848793 2558 feature_gate.go:328] unrecognized feature gate: MachineConfigNodes Jul 10 18:02:51.458837 ip-10-0-128-39 kubenswrapper[2558]: W0710 18:02:50.848796 2558 feature_gate.go:328] unrecognized feature gate: RouteAdvertisements Jul 10 18:02:51.458837 ip-10-0-128-39 kubenswrapper[2558]: W0710 18:02:50.848799 2558 feature_gate.go:328] unrecognized feature gate: MultiDiskSetup Jul 10 18:02:51.458837 ip-10-0-128-39 kubenswrapper[2558]: W0710 18:02:50.848802 2558 feature_gate.go:328] unrecognized feature gate: GCPCustomAPIEndpointsInstall Jul 10 18:02:51.458837 ip-10-0-128-39 kubenswrapper[2558]: W0710 18:02:50.848804 2558 feature_gate.go:328] unrecognized feature gate: ClusterAPIMachineManagement Jul 10 18:02:51.458837 ip-10-0-128-39 kubenswrapper[2558]: W0710 18:02:50.848807 2558 feature_gate.go:328] unrecognized feature gate: NetworkLiveMigration Jul 10 18:02:51.458837 ip-10-0-128-39 kubenswrapper[2558]: W0710 18:02:50.848809 2558 feature_gate.go:328] unrecognized feature gate: NewOLMBoxCutterRuntime Jul 10 18:02:51.459704 ip-10-0-128-39 kubenswrapper[2558]: W0710 18:02:50.848812 2558 feature_gate.go:328] unrecognized feature gate: BuildCSIVolumes Jul 10 18:02:51.459704 ip-10-0-128-39 kubenswrapper[2558]: W0710 18:02:50.848815 2558 feature_gate.go:328] unrecognized feature gate: AWSDedicatedHosts Jul 10 18:02:51.459704 ip-10-0-128-39 kubenswrapper[2558]: W0710 18:02:50.848817 2558 feature_gate.go:328] unrecognized feature gate: SigstoreImageVerificationPKI Jul 10 18:02:51.459704 ip-10-0-128-39 kubenswrapper[2558]: W0710 18:02:50.848820 2558 feature_gate.go:328] unrecognized feature gate: DualReplica Jul 10 18:02:51.459704 ip-10-0-128-39 kubenswrapper[2558]: W0710 18:02:50.848823 2558 feature_gate.go:328] unrecognized feature gate: GCPClusterHostedDNS Jul 10 18:02:51.459704 ip-10-0-128-39 kubenswrapper[2558]: W0710 18:02:50.848825 2558 feature_gate.go:328] unrecognized feature gate: AdditionalRoutingCapabilities Jul 10 18:02:51.459704 ip-10-0-128-39 kubenswrapper[2558]: W0710 18:02:50.848828 2558 feature_gate.go:328] unrecognized feature gate: AutomatedEtcdBackup Jul 10 18:02:51.459704 ip-10-0-128-39 kubenswrapper[2558]: W0710 18:02:50.848830 2558 feature_gate.go:328] unrecognized feature gate: AzureWorkloadIdentity Jul 10 18:02:51.459704 ip-10-0-128-39 kubenswrapper[2558]: W0710 18:02:50.848833 2558 feature_gate.go:328] unrecognized feature gate: VSphereMultiDisk Jul 10 18:02:51.459704 ip-10-0-128-39 kubenswrapper[2558]: W0710 18:02:50.848835 2558 feature_gate.go:328] unrecognized feature gate: NewOLMPreflightPermissionChecks Jul 10 18:02:51.459704 ip-10-0-128-39 kubenswrapper[2558]: W0710 18:02:50.848839 2558 feature_gate.go:328] unrecognized feature gate: CRDCompatibilityRequirementOperator Jul 10 18:02:51.459704 ip-10-0-128-39 kubenswrapper[2558]: W0710 18:02:50.848842 2558 feature_gate.go:328] unrecognized feature gate: BootImageSkewEnforcement Jul 10 18:02:51.459704 ip-10-0-128-39 kubenswrapper[2558]: W0710 18:02:50.848844 2558 feature_gate.go:328] unrecognized feature gate: VSphereConfigurableMaxAllowedBlockVolumesPerNode Jul 10 18:02:51.459704 ip-10-0-128-39 kubenswrapper[2558]: W0710 18:02:50.848847 2558 feature_gate.go:328] unrecognized feature gate: HighlyAvailableArbiter Jul 10 18:02:51.459704 ip-10-0-128-39 kubenswrapper[2558]: W0710 18:02:50.848849 2558 feature_gate.go:328] unrecognized feature gate: HyperShiftOnlyDynamicResourceAllocation Jul 10 18:02:51.459704 ip-10-0-128-39 kubenswrapper[2558]: W0710 18:02:50.848852 2558 feature_gate.go:328] unrecognized feature gate: SignatureStores Jul 10 18:02:51.459704 ip-10-0-128-39 kubenswrapper[2558]: W0710 18:02:50.848855 2558 feature_gate.go:328] unrecognized feature gate: NetworkDiagnosticsConfig Jul 10 18:02:51.459704 ip-10-0-128-39 kubenswrapper[2558]: W0710 18:02:50.848858 2558 feature_gate.go:328] unrecognized feature gate: ProvisioningRequestAvailable Jul 10 18:02:51.459704 ip-10-0-128-39 kubenswrapper[2558]: W0710 18:02:50.848862 2558 feature_gate.go:328] unrecognized feature gate: NewOLMOwnSingleNamespace Jul 10 18:02:51.460583 ip-10-0-128-39 kubenswrapper[2558]: W0710 18:02:50.848864 2558 feature_gate.go:328] unrecognized feature gate: GatewayAPIWithoutOLM Jul 10 18:02:51.460583 ip-10-0-128-39 kubenswrapper[2558]: W0710 18:02:50.848867 2558 feature_gate.go:328] unrecognized feature gate: OpenShiftPodSecurityAdmission Jul 10 18:02:51.460583 ip-10-0-128-39 kubenswrapper[2558]: W0710 18:02:50.848869 2558 feature_gate.go:328] unrecognized feature gate: ExternalSnapshotMetadata Jul 10 18:02:51.460583 ip-10-0-128-39 kubenswrapper[2558]: W0710 18:02:50.848872 2558 feature_gate.go:328] unrecognized feature gate: Example Jul 10 18:02:51.460583 ip-10-0-128-39 kubenswrapper[2558]: W0710 18:02:50.848874 2558 feature_gate.go:328] unrecognized feature gate: InsightsOnDemandDataGather Jul 10 18:02:51.460583 ip-10-0-128-39 kubenswrapper[2558]: W0710 18:02:50.848877 2558 feature_gate.go:328] unrecognized feature gate: ClusterAPIInstall Jul 10 18:02:51.460583 ip-10-0-128-39 kubenswrapper[2558]: W0710 18:02:50.848879 2558 feature_gate.go:328] unrecognized feature gate: MachineAPIMigration Jul 10 18:02:51.460583 ip-10-0-128-39 kubenswrapper[2558]: W0710 18:02:50.848882 2558 feature_gate.go:328] unrecognized feature gate: KMSEncryption Jul 10 18:02:51.460583 ip-10-0-128-39 kubenswrapper[2558]: W0710 18:02:50.848884 2558 feature_gate.go:328] unrecognized feature gate: AWSDualStackInstall Jul 10 18:02:51.460583 ip-10-0-128-39 kubenswrapper[2558]: W0710 18:02:50.848887 2558 feature_gate.go:328] unrecognized feature gate: NewOLMCatalogdAPIV1Metas Jul 10 18:02:51.460583 ip-10-0-128-39 kubenswrapper[2558]: W0710 18:02:50.848889 2558 feature_gate.go:328] unrecognized feature gate: NutanixMultiSubnets Jul 10 18:02:51.460583 ip-10-0-128-39 kubenswrapper[2558]: W0710 18:02:50.848892 2558 feature_gate.go:328] unrecognized feature gate: AdminNetworkPolicy Jul 10 18:02:51.460583 ip-10-0-128-39 kubenswrapper[2558]: I0710 18:02:50.848897 2558 feature_gate.go:384] feature gates: {map[CBORServingAndStorage:false ClientsAllowCBOR:false ClientsPreferCBOR:false EventedPLEG:false ImageVolume:true KMSv1:true MaxUnavailableStatefulSet:false MinimumKubeletVersion:false MutableCSINodeAllocatableCount:false MutatingAdmissionPolicy:false ProcMountType:true RouteExternalCertificate:true SELinuxMount:false ServiceAccountTokenNodeBinding:true StoragePerformantSecurityPolicy:true TranslateStreamCloseWebsocketRequests:false UserNamespacesPodSecurityStandards:true UserNamespacesSupport:true VolumeAttributesClass:true]} Jul 10 18:02:51.460583 ip-10-0-128-39 kubenswrapper[2558]: W0710 18:02:50.849009 2558 feature_gate.go:328] unrecognized feature gate: BuildCSIVolumes Jul 10 18:02:51.460583 ip-10-0-128-39 kubenswrapper[2558]: W0710 18:02:50.849014 2558 feature_gate.go:328] unrecognized feature gate: VSphereMixedNodeEnv Jul 10 18:02:51.461235 ip-10-0-128-39 kubenswrapper[2558]: W0710 18:02:50.849017 2558 feature_gate.go:328] unrecognized feature gate: NutanixMultiSubnets Jul 10 18:02:51.461235 ip-10-0-128-39 kubenswrapper[2558]: W0710 18:02:50.849019 2558 feature_gate.go:328] unrecognized feature gate: ShortCertRotation Jul 10 18:02:51.461235 ip-10-0-128-39 kubenswrapper[2558]: W0710 18:02:50.849023 2558 feature_gate.go:349] Setting deprecated feature gate KMSv1=true. It will be removed in a future release. Jul 10 18:02:51.461235 ip-10-0-128-39 kubenswrapper[2558]: W0710 18:02:50.849027 2558 feature_gate.go:328] unrecognized feature gate: AWSDedicatedHosts Jul 10 18:02:51.461235 ip-10-0-128-39 kubenswrapper[2558]: W0710 18:02:50.849030 2558 feature_gate.go:328] unrecognized feature gate: MultiArchInstallAzure Jul 10 18:02:51.461235 ip-10-0-128-39 kubenswrapper[2558]: W0710 18:02:50.849033 2558 feature_gate.go:328] unrecognized feature gate: RouteAdvertisements Jul 10 18:02:51.461235 ip-10-0-128-39 kubenswrapper[2558]: W0710 18:02:50.849036 2558 feature_gate.go:328] unrecognized feature gate: PinnedImages Jul 10 18:02:51.461235 ip-10-0-128-39 kubenswrapper[2558]: W0710 18:02:50.849038 2558 feature_gate.go:328] unrecognized feature gate: HyperShiftOnlyDynamicResourceAllocation Jul 10 18:02:51.461235 ip-10-0-128-39 kubenswrapper[2558]: W0710 18:02:50.849042 2558 feature_gate.go:328] unrecognized feature gate: VSphereHostVMGroupZonal Jul 10 18:02:51.461235 ip-10-0-128-39 kubenswrapper[2558]: W0710 18:02:50.849045 2558 feature_gate.go:328] unrecognized feature gate: AWSClusterHostedDNS Jul 10 18:02:51.461235 ip-10-0-128-39 kubenswrapper[2558]: W0710 18:02:50.849047 2558 feature_gate.go:328] unrecognized feature gate: AdminNetworkPolicy Jul 10 18:02:51.461235 ip-10-0-128-39 kubenswrapper[2558]: W0710 18:02:50.849049 2558 feature_gate.go:328] unrecognized feature gate: MachineAPIOperatorDisableMachineHealthCheckController Jul 10 18:02:51.461235 ip-10-0-128-39 kubenswrapper[2558]: W0710 18:02:50.849052 2558 feature_gate.go:328] unrecognized feature gate: NewOLMOwnSingleNamespace Jul 10 18:02:51.461235 ip-10-0-128-39 kubenswrapper[2558]: W0710 18:02:50.849054 2558 feature_gate.go:328] unrecognized feature gate: GatewayAPIWithoutOLM Jul 10 18:02:51.461235 ip-10-0-128-39 kubenswrapper[2558]: W0710 18:02:50.849057 2558 feature_gate.go:328] unrecognized feature gate: ImageModeStatusReporting Jul 10 18:02:51.461235 ip-10-0-128-39 kubenswrapper[2558]: W0710 18:02:50.849060 2558 feature_gate.go:328] unrecognized feature gate: OSStreams Jul 10 18:02:51.461235 ip-10-0-128-39 kubenswrapper[2558]: W0710 18:02:50.849062 2558 feature_gate.go:328] unrecognized feature gate: NewOLMBoxCutterRuntime Jul 10 18:02:51.461235 ip-10-0-128-39 kubenswrapper[2558]: W0710 18:02:50.849067 2558 feature_gate.go:328] unrecognized feature gate: AWSServiceLBNetworkSecurityGroup Jul 10 18:02:51.461235 ip-10-0-128-39 kubenswrapper[2558]: W0710 18:02:50.849069 2558 feature_gate.go:328] unrecognized feature gate: MachineAPIMigration Jul 10 18:02:51.462103 ip-10-0-128-39 kubenswrapper[2558]: W0710 18:02:50.849072 2558 feature_gate.go:328] unrecognized feature gate: ClusterAPIMachineManagementVSphere Jul 10 18:02:51.462103 ip-10-0-128-39 kubenswrapper[2558]: W0710 18:02:50.849074 2558 feature_gate.go:328] unrecognized feature gate: AdditionalRoutingCapabilities Jul 10 18:02:51.462103 ip-10-0-128-39 kubenswrapper[2558]: W0710 18:02:50.849077 2558 feature_gate.go:328] unrecognized feature gate: ClusterAPIMachineManagement Jul 10 18:02:51.462103 ip-10-0-128-39 kubenswrapper[2558]: W0710 18:02:50.849079 2558 feature_gate.go:328] unrecognized feature gate: EventTTL Jul 10 18:02:51.462103 ip-10-0-128-39 kubenswrapper[2558]: W0710 18:02:50.849082 2558 feature_gate.go:328] unrecognized feature gate: ClusterVersionOperatorConfiguration Jul 10 18:02:51.462103 ip-10-0-128-39 kubenswrapper[2558]: W0710 18:02:50.849085 2558 feature_gate.go:328] unrecognized feature gate: DyanmicServiceEndpointIBMCloud Jul 10 18:02:51.462103 ip-10-0-128-39 kubenswrapper[2558]: W0710 18:02:50.849087 2558 feature_gate.go:328] unrecognized feature gate: AzureMultiDisk Jul 10 18:02:51.462103 ip-10-0-128-39 kubenswrapper[2558]: W0710 18:02:50.849091 2558 feature_gate.go:328] unrecognized feature gate: Example Jul 10 18:02:51.462103 ip-10-0-128-39 kubenswrapper[2558]: W0710 18:02:50.849093 2558 feature_gate.go:328] unrecognized feature gate: NewOLMPreflightPermissionChecks Jul 10 18:02:51.462103 ip-10-0-128-39 kubenswrapper[2558]: W0710 18:02:50.849095 2558 feature_gate.go:328] unrecognized feature gate: ClusterMonitoringConfig Jul 10 18:02:51.462103 ip-10-0-128-39 kubenswrapper[2558]: W0710 18:02:50.849098 2558 feature_gate.go:328] unrecognized feature gate: NetworkDiagnosticsConfig Jul 10 18:02:51.462103 ip-10-0-128-39 kubenswrapper[2558]: W0710 18:02:50.849101 2558 feature_gate.go:328] unrecognized feature gate: ExternalSnapshotMetadata Jul 10 18:02:51.462103 ip-10-0-128-39 kubenswrapper[2558]: W0710 18:02:50.849103 2558 feature_gate.go:328] unrecognized feature gate: SigstoreImageVerificationPKI Jul 10 18:02:51.462103 ip-10-0-128-39 kubenswrapper[2558]: W0710 18:02:50.849106 2558 feature_gate.go:328] unrecognized feature gate: AzureWorkloadIdentity Jul 10 18:02:51.462103 ip-10-0-128-39 kubenswrapper[2558]: W0710 18:02:50.849108 2558 feature_gate.go:328] unrecognized feature gate: InsightsOnDemandDataGather Jul 10 18:02:51.462103 ip-10-0-128-39 kubenswrapper[2558]: W0710 18:02:50.849111 2558 feature_gate.go:328] unrecognized feature gate: SigstoreImageVerification Jul 10 18:02:51.462103 ip-10-0-128-39 kubenswrapper[2558]: W0710 18:02:50.849114 2558 feature_gate.go:328] unrecognized feature gate: AzureClusterHostedDNSInstall Jul 10 18:02:51.462103 ip-10-0-128-39 kubenswrapper[2558]: W0710 18:02:50.849116 2558 feature_gate.go:328] unrecognized feature gate: VSphereMultiDisk Jul 10 18:02:51.462103 ip-10-0-128-39 kubenswrapper[2558]: W0710 18:02:50.849118 2558 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesCPMS Jul 10 18:02:51.462103 ip-10-0-128-39 kubenswrapper[2558]: W0710 18:02:50.849121 2558 feature_gate.go:328] unrecognized feature gate: ManagedBootImages Jul 10 18:02:51.462967 ip-10-0-128-39 kubenswrapper[2558]: W0710 18:02:50.849123 2558 feature_gate.go:328] unrecognized feature gate: NetworkSegmentation Jul 10 18:02:51.462967 ip-10-0-128-39 kubenswrapper[2558]: W0710 18:02:50.849128 2558 feature_gate.go:328] unrecognized feature gate: NewOLMCatalogdAPIV1Metas Jul 10 18:02:51.462967 ip-10-0-128-39 kubenswrapper[2558]: W0710 18:02:50.849130 2558 feature_gate.go:328] unrecognized feature gate: HighlyAvailableArbiter Jul 10 18:02:51.462967 ip-10-0-128-39 kubenswrapper[2558]: W0710 18:02:50.849133 2558 feature_gate.go:328] unrecognized feature gate: KMSEncryption Jul 10 18:02:51.462967 ip-10-0-128-39 kubenswrapper[2558]: W0710 18:02:50.849135 2558 feature_gate.go:328] unrecognized feature gate: MachineConfigNodes Jul 10 18:02:51.462967 ip-10-0-128-39 kubenswrapper[2558]: W0710 18:02:50.849138 2558 feature_gate.go:328] unrecognized feature gate: AWSDualStackInstall Jul 10 18:02:51.462967 ip-10-0-128-39 kubenswrapper[2558]: W0710 18:02:50.849142 2558 feature_gate.go:351] Setting GA feature gate VolumeAttributesClass=true. It will be removed in a future release. Jul 10 18:02:51.462967 ip-10-0-128-39 kubenswrapper[2558]: W0710 18:02:50.849145 2558 feature_gate.go:328] unrecognized feature gate: OnPremDNSRecords Jul 10 18:02:51.462967 ip-10-0-128-39 kubenswrapper[2558]: W0710 18:02:50.849148 2558 feature_gate.go:328] unrecognized feature gate: MixedCPUsAllocation Jul 10 18:02:51.462967 ip-10-0-128-39 kubenswrapper[2558]: W0710 18:02:50.849151 2558 feature_gate.go:328] unrecognized feature gate: NoRegistryClusterInstall Jul 10 18:02:51.462967 ip-10-0-128-39 kubenswrapper[2558]: W0710 18:02:50.849155 2558 feature_gate.go:328] unrecognized feature gate: CRDCompatibilityRequirementOperator Jul 10 18:02:51.462967 ip-10-0-128-39 kubenswrapper[2558]: W0710 18:02:50.849159 2558 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesAzure Jul 10 18:02:51.462967 ip-10-0-128-39 kubenswrapper[2558]: W0710 18:02:50.849162 2558 feature_gate.go:328] unrecognized feature gate: CPMSMachineNamePrefix Jul 10 18:02:51.462967 ip-10-0-128-39 kubenswrapper[2558]: W0710 18:02:50.849164 2558 feature_gate.go:328] unrecognized feature gate: KMSEncryptionProvider Jul 10 18:02:51.462967 ip-10-0-128-39 kubenswrapper[2558]: W0710 18:02:50.849167 2558 feature_gate.go:328] unrecognized feature gate: IrreconcilableMachineConfig Jul 10 18:02:51.462967 ip-10-0-128-39 kubenswrapper[2558]: W0710 18:02:50.849183 2558 feature_gate.go:328] unrecognized feature gate: GatewayAPI Jul 10 18:02:51.462967 ip-10-0-128-39 kubenswrapper[2558]: W0710 18:02:50.849186 2558 feature_gate.go:328] unrecognized feature gate: PreconfiguredUDNAddresses Jul 10 18:02:51.462967 ip-10-0-128-39 kubenswrapper[2558]: W0710 18:02:50.849189 2558 feature_gate.go:328] unrecognized feature gate: NewOLMWebhookProviderOpenshiftServiceCA Jul 10 18:02:51.462967 ip-10-0-128-39 kubenswrapper[2558]: W0710 18:02:50.849191 2558 feature_gate.go:328] unrecognized feature gate: OVNObservability Jul 10 18:02:51.462967 ip-10-0-128-39 kubenswrapper[2558]: W0710 18:02:50.849194 2558 feature_gate.go:328] unrecognized feature gate: NetworkLiveMigration Jul 10 18:02:51.464060 ip-10-0-128-39 kubenswrapper[2558]: W0710 18:02:50.849197 2558 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesvSphere Jul 10 18:02:51.464060 ip-10-0-128-39 kubenswrapper[2558]: W0710 18:02:50.849200 2558 feature_gate.go:328] unrecognized feature gate: ExternalOIDCWithUIDAndExtraClaimMappings Jul 10 18:02:51.464060 ip-10-0-128-39 kubenswrapper[2558]: W0710 18:02:50.849203 2558 feature_gate.go:328] unrecognized feature gate: GCPDualStackInstall Jul 10 18:02:51.464060 ip-10-0-128-39 kubenswrapper[2558]: W0710 18:02:50.849205 2558 feature_gate.go:328] unrecognized feature gate: DualReplica Jul 10 18:02:51.464060 ip-10-0-128-39 kubenswrapper[2558]: W0710 18:02:50.849208 2558 feature_gate.go:328] unrecognized feature gate: Example2 Jul 10 18:02:51.464060 ip-10-0-128-39 kubenswrapper[2558]: W0710 18:02:50.849211 2558 feature_gate.go:328] unrecognized feature gate: ProvisioningRequestAvailable Jul 10 18:02:51.464060 ip-10-0-128-39 kubenswrapper[2558]: W0710 18:02:50.849213 2558 feature_gate.go:328] unrecognized feature gate: InsightsConfig Jul 10 18:02:51.464060 ip-10-0-128-39 kubenswrapper[2558]: W0710 18:02:50.849216 2558 feature_gate.go:328] unrecognized feature gate: OpenShiftPodSecurityAdmission Jul 10 18:02:51.464060 ip-10-0-128-39 kubenswrapper[2558]: W0710 18:02:50.849218 2558 feature_gate.go:328] unrecognized feature gate: AutomatedEtcdBackup Jul 10 18:02:51.464060 ip-10-0-128-39 kubenswrapper[2558]: W0710 18:02:50.849221 2558 feature_gate.go:328] unrecognized feature gate: ExternalOIDC Jul 10 18:02:51.464060 ip-10-0-128-39 kubenswrapper[2558]: W0710 18:02:50.849223 2558 feature_gate.go:328] unrecognized feature gate: MetricsCollectionProfiles Jul 10 18:02:51.464060 ip-10-0-128-39 kubenswrapper[2558]: W0710 18:02:50.849226 2558 feature_gate.go:328] unrecognized feature gate: IngressControllerDynamicConfigurationManager Jul 10 18:02:51.464060 ip-10-0-128-39 kubenswrapper[2558]: W0710 18:02:50.849229 2558 feature_gate.go:328] unrecognized feature gate: VolumeGroupSnapshot Jul 10 18:02:51.464060 ip-10-0-128-39 kubenswrapper[2558]: W0710 18:02:50.849234 2558 feature_gate.go:351] Setting GA feature gate ServiceAccountTokenNodeBinding=true. It will be removed in a future release. Jul 10 18:02:51.464060 ip-10-0-128-39 kubenswrapper[2558]: W0710 18:02:50.849237 2558 feature_gate.go:328] unrecognized feature gate: VSphereMultiNetworks Jul 10 18:02:51.464060 ip-10-0-128-39 kubenswrapper[2558]: W0710 18:02:50.849240 2558 feature_gate.go:328] unrecognized feature gate: NewOLM Jul 10 18:02:51.464060 ip-10-0-128-39 kubenswrapper[2558]: W0710 18:02:50.849243 2558 feature_gate.go:328] unrecognized feature gate: GCPCustomAPIEndpoints Jul 10 18:02:51.464060 ip-10-0-128-39 kubenswrapper[2558]: W0710 18:02:50.849246 2558 feature_gate.go:328] unrecognized feature gate: ConsolePluginContentSecurityPolicy Jul 10 18:02:51.464060 ip-10-0-128-39 kubenswrapper[2558]: W0710 18:02:50.849249 2558 feature_gate.go:328] unrecognized feature gate: GatewayAPIController Jul 10 18:02:51.464917 ip-10-0-128-39 kubenswrapper[2558]: W0710 18:02:50.849251 2558 feature_gate.go:328] unrecognized feature gate: ImageStreamImportMode Jul 10 18:02:51.464917 ip-10-0-128-39 kubenswrapper[2558]: W0710 18:02:50.849254 2558 feature_gate.go:328] unrecognized feature gate: AlibabaPlatform Jul 10 18:02:51.464917 ip-10-0-128-39 kubenswrapper[2558]: W0710 18:02:50.849256 2558 feature_gate.go:328] unrecognized feature gate: ClusterAPIInstall Jul 10 18:02:51.464917 ip-10-0-128-39 kubenswrapper[2558]: W0710 18:02:50.849261 2558 feature_gate.go:328] unrecognized feature gate: AzureDedicatedHosts Jul 10 18:02:51.464917 ip-10-0-128-39 kubenswrapper[2558]: W0710 18:02:50.849263 2558 feature_gate.go:328] unrecognized feature gate: AzureDualStackInstall Jul 10 18:02:51.464917 ip-10-0-128-39 kubenswrapper[2558]: W0710 18:02:50.849266 2558 feature_gate.go:328] unrecognized feature gate: UpgradeStatus Jul 10 18:02:51.464917 ip-10-0-128-39 kubenswrapper[2558]: W0710 18:02:50.849268 2558 feature_gate.go:328] unrecognized feature gate: SignatureStores Jul 10 18:02:51.464917 ip-10-0-128-39 kubenswrapper[2558]: W0710 18:02:50.849271 2558 feature_gate.go:328] unrecognized feature gate: ClusterAPIInstallIBMCloud Jul 10 18:02:51.464917 ip-10-0-128-39 kubenswrapper[2558]: W0710 18:02:50.849273 2558 feature_gate.go:328] unrecognized feature gate: GCPCustomAPIEndpointsInstall Jul 10 18:02:51.464917 ip-10-0-128-39 kubenswrapper[2558]: W0710 18:02:50.849276 2558 feature_gate.go:328] unrecognized feature gate: EtcdBackendQuota Jul 10 18:02:51.464917 ip-10-0-128-39 kubenswrapper[2558]: W0710 18:02:50.849279 2558 feature_gate.go:328] unrecognized feature gate: AWSClusterHostedDNSInstall Jul 10 18:02:51.464917 ip-10-0-128-39 kubenswrapper[2558]: W0710 18:02:50.849281 2558 feature_gate.go:328] unrecognized feature gate: GCPClusterHostedDNS Jul 10 18:02:51.464917 ip-10-0-128-39 kubenswrapper[2558]: W0710 18:02:50.849284 2558 feature_gate.go:328] unrecognized feature gate: BootImageSkewEnforcement Jul 10 18:02:51.464917 ip-10-0-128-39 kubenswrapper[2558]: W0710 18:02:50.849286 2558 feature_gate.go:328] unrecognized feature gate: BootcNodeManagement Jul 10 18:02:51.464917 ip-10-0-128-39 kubenswrapper[2558]: W0710 18:02:50.849289 2558 feature_gate.go:328] unrecognized feature gate: VSphereConfigurableMaxAllowedBlockVolumesPerNode Jul 10 18:02:51.464917 ip-10-0-128-39 kubenswrapper[2558]: W0710 18:02:50.849292 2558 feature_gate.go:328] unrecognized feature gate: DNSNameResolver Jul 10 18:02:51.464917 ip-10-0-128-39 kubenswrapper[2558]: W0710 18:02:50.849294 2558 feature_gate.go:328] unrecognized feature gate: GCPClusterHostedDNSInstall Jul 10 18:02:51.464917 ip-10-0-128-39 kubenswrapper[2558]: W0710 18:02:50.849297 2558 feature_gate.go:328] unrecognized feature gate: MultiDiskSetup Jul 10 18:02:51.464917 ip-10-0-128-39 kubenswrapper[2558]: W0710 18:02:50.849300 2558 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesAWS Jul 10 18:02:51.465795 ip-10-0-128-39 kubenswrapper[2558]: I0710 18:02:50.849305 2558 feature_gate.go:384] feature gates: {map[CBORServingAndStorage:false ClientsAllowCBOR:false ClientsPreferCBOR:false EventedPLEG:false ImageVolume:true KMSv1:true MaxUnavailableStatefulSet:false MinimumKubeletVersion:false MutableCSINodeAllocatableCount:false MutatingAdmissionPolicy:false ProcMountType:true RouteExternalCertificate:true SELinuxMount:false ServiceAccountTokenNodeBinding:true StoragePerformantSecurityPolicy:true TranslateStreamCloseWebsocketRequests:false UserNamespacesPodSecurityStandards:true UserNamespacesSupport:true VolumeAttributesClass:true]} Jul 10 18:02:51.465795 ip-10-0-128-39 kubenswrapper[2558]: I0710 18:02:50.849416 2558 server.go:962] "Client rotation is on, will bootstrap in background" Jul 10 18:02:51.465795 ip-10-0-128-39 kubenswrapper[2558]: I0710 18:02:50.853905 2558 bootstrap.go:101] "Use the bootstrap credentials to request a cert, and set kubeconfig to point to the certificate dir" Jul 10 18:02:51.465795 ip-10-0-128-39 kubenswrapper[2558]: I0710 18:02:50.855339 2558 server.go:1019] "Starting client certificate rotation" Jul 10 18:02:51.465795 ip-10-0-128-39 kubenswrapper[2558]: I0710 18:02:50.855447 2558 certificate_manager.go:422] "Certificate rotation is enabled" logger="kubernetes.io/kube-apiserver-client-kubelet" Jul 10 18:02:51.465795 ip-10-0-128-39 kubenswrapper[2558]: I0710 18:02:50.855484 2558 certificate_manager.go:566] "Rotating certificates" logger="kubernetes.io/kube-apiserver-client-kubelet" Jul 10 18:02:51.465795 ip-10-0-128-39 kubenswrapper[2558]: I0710 18:02:50.857436 2558 dynamic_cafile_content.go:123] "Loaded a new CA Bundle and Verifier" name="client-ca-bundle::/etc/kubernetes/kubelet-ca.crt" Jul 10 18:02:51.465795 ip-10-0-128-39 kubenswrapper[2558]: I0710 18:02:50.858623 2558 dynamic_cafile_content.go:161] "Starting controller" name="client-ca-bundle::/etc/kubernetes/kubelet-ca.crt" Jul 10 18:02:51.465795 ip-10-0-128-39 kubenswrapper[2558]: I0710 18:02:50.866022 2558 log.go:25] "Validated CRI v1 runtime API" Jul 10 18:02:51.465795 ip-10-0-128-39 kubenswrapper[2558]: I0710 18:02:50.868494 2558 log.go:25] "Validated CRI v1 image API" Jul 10 18:02:51.465795 ip-10-0-128-39 kubenswrapper[2558]: I0710 18:02:50.870909 2558 server.go:1429] "Using cgroup driver setting received from the CRI runtime" cgroupDriver="systemd" Jul 10 18:02:51.465795 ip-10-0-128-39 kubenswrapper[2558]: I0710 18:02:50.875732 2558 fs.go:135] Filesystem UUIDs: map[091966b0-73a1-4113-bffd-7e4ff4c46912:/dev/nvme0n1p4 7B77-95E7:/dev/nvme0n1p2 bc97a5d9-2fc8-4c0a-97e3-9f7d39d06dbd:/dev/nvme0n1p3] Jul 10 18:02:51.466428 ip-10-0-128-39 kubenswrapper[2558]: I0710 18:02:50.875756 2558 fs.go:136] Filesystem partitions: map[/dev/nvme0n1p3:{mountpoint:/boot major:259 minor:3 fsType:ext4 blockSize:0} /dev/nvme0n1p4:{mountpoint:/var major:259 minor:4 fsType:xfs blockSize:0} /dev/shm:{mountpoint:/dev/shm major:0 minor:22 fsType:tmpfs blockSize:0} /run:{mountpoint:/run major:0 minor:24 fsType:tmpfs blockSize:0} /tmp:{mountpoint:/tmp major:0 minor:31 fsType:tmpfs blockSize:0} composefs_0-33:{mountpoint:/ major:0 minor:33 fsType:overlay blockSize:0}] Jul 10 18:02:51.466642 ip-10-0-128-39 kubenswrapper[2558]: I0710 18:02:50.881920 2558 manager.go:216] Machine: {Timestamp:2026-07-10 18:02:50.879641315 +0000 UTC m=+0.399769252 CPUVendorID:GenuineIntel NumCores:8 NumPhysicalCores:4 NumSockets:1 CpuFrequency:3100755 MemoryCapacity:33164492800 SwapCapacity:0 MemoryByType:map[] NVMInfo:{MemoryModeCapacity:0 AppDirectModeCapacity:0 AvgPowerBudget:0} HugePages:[{PageSize:1048576 NumPages:0} {PageSize:2048 NumPages:0}] MachineID:ec2f067c5999ea9c2cf212aa325e3ad3 SystemUUID:ec2f067c-5999-ea9c-2cf2-12aa325e3ad3 BootID:013a744b-f519-4048-aca4-9d2ed26f741a Filesystems:[{Device:composefs_0-33 DeviceMajor:0 DeviceMinor:33 Capacity:6107136 Type:vfs Inodes:18446744073709551615 HasInodes:true} {Device:/tmp DeviceMajor:0 DeviceMinor:31 Capacity:16582246400 Type:vfs Inodes:1048576 HasInodes:true} {Device:/dev/nvme0n1p3 DeviceMajor:259 DeviceMinor:3 Capacity:366869504 Type:vfs Inodes:98304 HasInodes:true} {Device:/dev/shm DeviceMajor:0 DeviceMinor:22 Capacity:16582246400 Type:vfs Inodes:4048400 HasInodes:true} {Device:/run DeviceMajor:0 DeviceMinor:24 Capacity:6632898560 Type:vfs Inodes:819200 HasInodes:true} {Device:/dev/nvme0n1p4 DeviceMajor:259 DeviceMinor:4 Capacity:128243970048 Type:vfs Inodes:62651840 HasInodes:true}] DiskMap:map[259:0:{Name:nvme0n1 Major:259 Minor:0 Size:128849018880 Scheduler:none}] NetworkDevices:[{Name:br-ex MacAddress:02:a7:90:dd:ff:6d Speed:0 Mtu:9001} {Name:ens5 MacAddress:02:a7:90:dd:ff:6d Speed:0 Mtu:9001} {Name:ovs-system MacAddress:7a:f0:5a:df:5b:ea Speed:0 Mtu:1500}] Topology:[{Id:0 Memory:33164492800 HugePages:[{PageSize:1048576 NumPages:0} {PageSize:2048 NumPages:0}] Cores:[{Id:0 Threads:[0 4] Caches:[{Id:0 Size:32768 Type:Data Level:1} {Id:0 Size:32768 Type:Instruction Level:1} {Id:0 Size:1048576 Type:Unified Level:2}] UncoreCaches:[] SocketID:0 BookID: DrawerID:} {Id:1 Threads:[1 5] Caches:[{Id:1 Size:32768 Type:Data Level:1} {Id:1 Size:32768 Type:Instruction Level:1} {Id:1 Size:1048576 Type:Unified Level:2}] UncoreCaches:[] SocketID:0 BookID: DrawerID:} {Id:2 Threads:[2 6] Caches:[{Id:2 Size:32768 Type:Data Level:1} {Id:2 Size:32768 Type:Instruction Level:1} {Id:2 Size:1048576 Type:Unified Level:2}] UncoreCaches:[] SocketID:0 BookID: DrawerID:} {Id:3 Threads:[3 7] Caches:[{Id:3 Size:32768 Type:Data Level:1} {Id:3 Size:32768 Type:Instruction Level:1} {Id:3 Size:1048576 Type:Unified Level:2}] UncoreCaches:[] SocketID:0 BookID: DrawerID:}] Caches:[{Id:0 Size:37486592 Type:Unified Level:3}] Distances:[10]}] CloudProvider:Unknown InstanceType:Unknown InstanceID:None} Jul 10 18:02:51.466642 ip-10-0-128-39 kubenswrapper[2558]: I0710 18:02:50.882023 2558 manager_no_libpfm.go:29] cAdvisor is build without cgo and/or libpfm support. Perf event counters are not available. Jul 10 18:02:51.466642 ip-10-0-128-39 kubenswrapper[2558]: I0710 18:02:50.882157 2558 manager.go:232] Version: {KernelVersion:5.14.0-570.126.1.el9_6.x86_64 ContainerOsVersion:Red Hat Enterprise Linux CoreOS 9.6.20260702-0 (Plow) DockerVersion: DockerAPIVersion: CadvisorVersion: CadvisorRevision:} Jul 10 18:02:51.466642 ip-10-0-128-39 kubenswrapper[2558]: I0710 18:02:50.885154 2558 container_manager_linux.go:273] "Container manager verified user specified cgroup-root exists" cgroupRoot=[] Jul 10 18:02:51.466642 ip-10-0-128-39 kubenswrapper[2558]: I0710 18:02:50.885191 2558 container_manager_linux.go:278] "Creating Container Manager object based on Node Config" nodeConfig={"NodeName":"ip-10-0-128-39.ec2.internal","RuntimeCgroupsName":"/system.slice/crio.service","SystemCgroupsName":"/system.slice","KubeletCgroupsName":"","KubeletOOMScoreAdj":-999,"ContainerRuntime":"","CgroupsPerQOS":true,"CgroupRoot":"/","CgroupDriver":"systemd","KubeletRootDir":"/var/lib/kubelet","ProtectKernelDefaults":true,"KubeReservedCgroupName":"","SystemReservedCgroupName":"","ReservedSystemCPUs":{},"EnforceNodeAllocatable":{"pods":{}},"KubeReserved":null,"SystemReserved":{"cpu":"500m","ephemeral-storage":"1Gi","memory":"1Gi"},"HardEvictionThresholds":[{"Signal":"imagefs.available","Operator":"LessThan","Value":{"Quantity":null,"Percentage":0.15},"GracePeriod":0,"MinReclaim":null},{"Signal":"imagefs.inodesFree","Operator":"LessThan","Value":{"Quantity":null,"Percentage":0.05},"GracePeriod":0,"MinReclaim":null},{"Signal":"memory.available","Operator":"LessThan","Value":{"Quantity":"100Mi","Percentage":0},"GracePeriod":0,"MinReclaim":null},{"Signal":"nodefs.available","Operator":"LessThan","Value":{"Quantity":null,"Percentage":0.1},"GracePeriod":0,"MinReclaim":null},{"Signal":"nodefs.inodesFree","Operator":"LessThan","Value":{"Quantity":null,"Percentage":0.05},"GracePeriod":0,"MinReclaim":null}],"QOSReserved":{},"CPUManagerPolicy":"none","CPUManagerPolicyOptions":null,"TopologyManagerScope":"container","CPUManagerReconcilePeriod":10000000000,"MemoryManagerPolicy":"None","MemoryManagerReservedMemory":null,"PodPidsLimit":4096,"EnforceCPULimits":true,"CPUCFSQuotaPeriod":100000000,"TopologyManagerPolicy":"none","TopologyManagerPolicyOptions":null,"CgroupVersion":2} Jul 10 18:02:51.466642 ip-10-0-128-39 kubenswrapper[2558]: I0710 18:02:50.886137 2558 topology_manager.go:138] "Creating topology manager with none policy" Jul 10 18:02:51.466642 ip-10-0-128-39 kubenswrapper[2558]: I0710 18:02:50.886149 2558 container_manager_linux.go:309] "Creating device plugin manager" Jul 10 18:02:51.466642 ip-10-0-128-39 kubenswrapper[2558]: I0710 18:02:50.886156 2558 manager.go:141] "Creating Device Plugin manager" path="/var/lib/kubelet/device-plugins/kubelet.sock" Jul 10 18:02:51.467468 ip-10-0-128-39 kubenswrapper[2558]: I0710 18:02:50.886842 2558 server.go:74] "Creating device plugin registration server" version="v1beta1" socket="/var/lib/kubelet/device-plugins/kubelet.sock" Jul 10 18:02:51.467468 ip-10-0-128-39 kubenswrapper[2558]: I0710 18:02:50.887887 2558 reflector.go:439] "Caches populated" logger="kubernetes.io/kube-apiserver-client-kubelet" type="*v1.CertificateSigningRequest" reflector="k8s.io/client-go/tools/watch/informerwatcher.go:162" Jul 10 18:02:51.467468 ip-10-0-128-39 kubenswrapper[2558]: I0710 18:02:50.888701 2558 container_manager_linux.go:318] "Creating Dynamic Resource Allocation (DRA) manager" Jul 10 18:02:51.467468 ip-10-0-128-39 kubenswrapper[2558]: I0710 18:02:50.892382 2558 state_mem.go:36] "Initialized new in-memory state store" Jul 10 18:02:51.467468 ip-10-0-128-39 kubenswrapper[2558]: I0710 18:02:50.892499 2558 server.go:1240] "Using root directory" path="/var/lib/kubelet" Jul 10 18:02:51.467468 ip-10-0-128-39 kubenswrapper[2558]: I0710 18:02:50.892530 2558 kubelet.go:486] "Attempting to sync node with API server" Jul 10 18:02:51.467468 ip-10-0-128-39 kubenswrapper[2558]: I0710 18:02:50.892543 2558 kubelet.go:387] "Adding static pod path" path="/etc/kubernetes/manifests" Jul 10 18:02:51.467468 ip-10-0-128-39 kubenswrapper[2558]: I0710 18:02:50.892563 2558 file.go:69] "Watching path" path="/etc/kubernetes/manifests" Jul 10 18:02:51.467468 ip-10-0-128-39 kubenswrapper[2558]: I0710 18:02:50.892576 2558 kubelet.go:398] "Adding apiserver pod source" Jul 10 18:02:51.467468 ip-10-0-128-39 kubenswrapper[2558]: I0710 18:02:50.892587 2558 apiserver.go:42] "Waiting for node sync before watching apiserver pods" Jul 10 18:02:51.467468 ip-10-0-128-39 kubenswrapper[2558]: I0710 18:02:50.893007 2558 state_mem.go:40] "Initialized new in-memory state store for pod resource information tracking" Jul 10 18:02:51.467468 ip-10-0-128-39 kubenswrapper[2558]: I0710 18:02:50.893023 2558 state_mem.go:40] "Initialized new in-memory state store for pod resource information tracking" Jul 10 18:02:51.467468 ip-10-0-128-39 kubenswrapper[2558]: I0710 18:02:50.895087 2558 kuberuntime_manager.go:301] "Container runtime initialized" containerRuntime="cri-o" version="1.34.10-2.rhaos4.21.git83ee25b.el9" apiVersion="v1" Jul 10 18:02:51.467468 ip-10-0-128-39 kubenswrapper[2558]: I0710 18:02:50.897310 2558 kubelet.go:960] "Not starting ClusterTrustBundle informer because we are in static kubelet mode or the ClusterTrustBundleProjection featuregate is disabled" Jul 10 18:02:51.467468 ip-10-0-128-39 kubenswrapper[2558]: I0710 18:02:50.897338 2558 kubelet.go:984] "Not starting PodCertificateRequest manager because we are in static kubelet mode or the PodCertificateProjection feature gate is disabled" Jul 10 18:02:51.467468 ip-10-0-128-39 kubenswrapper[2558]: I0710 18:02:50.897477 2558 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/portworx-volume" Jul 10 18:02:51.467468 ip-10-0-128-39 kubenswrapper[2558]: I0710 18:02:50.897489 2558 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/empty-dir" Jul 10 18:02:51.467468 ip-10-0-128-39 kubenswrapper[2558]: I0710 18:02:50.897496 2558 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/git-repo" Jul 10 18:02:51.467468 ip-10-0-128-39 kubenswrapper[2558]: I0710 18:02:50.897501 2558 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/host-path" Jul 10 18:02:51.467468 ip-10-0-128-39 kubenswrapper[2558]: I0710 18:02:50.897509 2558 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/nfs" Jul 10 18:02:51.467468 ip-10-0-128-39 kubenswrapper[2558]: I0710 18:02:50.897514 2558 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/secret" Jul 10 18:02:51.467468 ip-10-0-128-39 kubenswrapper[2558]: I0710 18:02:50.898194 2558 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/iscsi" Jul 10 18:02:51.467468 ip-10-0-128-39 kubenswrapper[2558]: I0710 18:02:50.898201 2558 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/downward-api" Jul 10 18:02:51.467468 ip-10-0-128-39 kubenswrapper[2558]: I0710 18:02:50.898207 2558 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/fc" Jul 10 18:02:51.467468 ip-10-0-128-39 kubenswrapper[2558]: I0710 18:02:50.898213 2558 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/configmap" Jul 10 18:02:51.467468 ip-10-0-128-39 kubenswrapper[2558]: I0710 18:02:50.898225 2558 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/projected" Jul 10 18:02:51.467468 ip-10-0-128-39 kubenswrapper[2558]: I0710 18:02:50.898233 2558 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/local-volume" Jul 10 18:02:51.467468 ip-10-0-128-39 kubenswrapper[2558]: I0710 18:02:50.898966 2558 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/csi" Jul 10 18:02:51.467468 ip-10-0-128-39 kubenswrapper[2558]: I0710 18:02:50.898972 2558 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/image" Jul 10 18:02:51.467468 ip-10-0-128-39 kubenswrapper[2558]: E0710 18:02:50.903028 2558 reflector.go:205] "Failed to watch" err="failed to list *v1.Node: nodes \"ip-10-0-128-39.ec2.internal\" is forbidden: User \"system:anonymous\" cannot list resource \"nodes\" in API group \"\" at the cluster scope" logger="UnhandledError" reflector="k8s.io/client-go/informers/factory.go:160" type="*v1.Node" Jul 10 18:02:51.467468 ip-10-0-128-39 kubenswrapper[2558]: I0710 18:02:50.903202 2558 csi_plugin.go:990] Failed to contact API server when waiting for CSINode publishing: csinodes.storage.k8s.io "ip-10-0-128-39.ec2.internal" is forbidden: User "system:anonymous" cannot get resource "csinodes" in API group "storage.k8s.io" at the cluster scope Jul 10 18:02:51.467468 ip-10-0-128-39 kubenswrapper[2558]: E0710 18:02:50.903262 2558 reflector.go:205] "Failed to watch" err="failed to list *v1.Service: services is forbidden: User \"system:anonymous\" cannot list resource \"services\" in API group \"\" at the cluster scope" logger="UnhandledError" reflector="k8s.io/client-go/informers/factory.go:160" type="*v1.Service" Jul 10 18:02:51.467468 ip-10-0-128-39 kubenswrapper[2558]: I0710 18:02:50.904100 2558 server.go:1268] "Started kubelet" Jul 10 18:02:51.467468 ip-10-0-128-39 kubenswrapper[2558]: I0710 18:02:50.904220 2558 ratelimit.go:56] "Setting rate limiting for endpoint" service="podresources" qps=100 burstTokens=10 Jul 10 18:02:51.467468 ip-10-0-128-39 kubenswrapper[2558]: I0710 18:02:50.904328 2558 server_v1.go:49] "podresources" method="list" useActivePods=true Jul 10 18:02:51.467468 ip-10-0-128-39 kubenswrapper[2558]: I0710 18:02:50.904882 2558 server.go:180] "Starting to listen" address="0.0.0.0" port=10250 Jul 10 18:02:51.467468 ip-10-0-128-39 kubenswrapper[2558]: I0710 18:02:50.905831 2558 server.go:249] "Starting to serve the podresources API" endpoint="unix:/var/lib/kubelet/pod-resources/kubelet.sock" Jul 10 18:02:51.467468 ip-10-0-128-39 kubenswrapper[2558]: I0710 18:02:50.906429 2558 server.go:310] "Adding debug handlers to kubelet server" Jul 10 18:02:51.467468 ip-10-0-128-39 kubenswrapper[2558]: E0710 18:02:50.909766 2558 kubelet.go:1638] "Image garbage collection failed once. Stats initialization may not have completed yet" err="failed to get imageFs info: unable to find data in memory cache" Jul 10 18:02:51.471057 ip-10-0-128-39 kubenswrapper[2558]: E0710 18:02:50.909889 2558 event.go:359] "Server rejected event (will not retry!)" err="events is forbidden: User \"system:anonymous\" cannot create resource \"events\" in API group \"\" in the namespace \"default\"" event="&Event{ObjectMeta:{ip-10-0-128-39.ec2.internal.18c0ffacfe447dc1 default 0 0001-01-01 00:00:00 +0000 UTC map[] map[] [] [] []},InvolvedObject:ObjectReference{Kind:Node,Namespace:,Name:ip-10-0-128-39.ec2.internal,UID:ip-10-0-128-39.ec2.internal,APIVersion:,ResourceVersion:,FieldPath:,},Reason:Starting,Message:Starting kubelet.,Source:EventSource{Component:kubelet,Host:ip-10-0-128-39.ec2.internal,},FirstTimestamp:2026-07-10 18:02:50.904075713 +0000 UTC m=+0.424203650,LastTimestamp:2026-07-10 18:02:50.904075713 +0000 UTC m=+0.424203650,Count:1,Type:Normal,EventTime:0001-01-01 00:00:00 +0000 UTC,Series:nil,Action:,Related:nil,ReportingController:kubelet,ReportingInstance:ip-10-0-128-39.ec2.internal,}" Jul 10 18:02:51.471057 ip-10-0-128-39 kubenswrapper[2558]: I0710 18:02:50.914752 2558 fs_resource_analyzer.go:67] "Starting FS ResourceAnalyzer" Jul 10 18:02:51.471057 ip-10-0-128-39 kubenswrapper[2558]: I0710 18:02:50.914800 2558 certificate_manager.go:422] "Certificate rotation is enabled" logger="kubernetes.io/kubelet-serving" Jul 10 18:02:51.471057 ip-10-0-128-39 kubenswrapper[2558]: E0710 18:02:50.914841 2558 kubelet_node_status.go:454] "Error getting the current node from lister" err="node \"ip-10-0-128-39.ec2.internal\" not found" Jul 10 18:02:51.471057 ip-10-0-128-39 kubenswrapper[2558]: I0710 18:02:50.915432 2558 volume_manager.go:311] "The desired_state_of_world populator starts" Jul 10 18:02:51.471057 ip-10-0-128-39 kubenswrapper[2558]: I0710 18:02:50.915441 2558 desired_state_of_world_populator.go:146] "Desired state populator starts to run" Jul 10 18:02:51.471057 ip-10-0-128-39 kubenswrapper[2558]: I0710 18:02:50.915448 2558 volume_manager.go:313] "Starting Kubelet Volume Manager" Jul 10 18:02:51.471057 ip-10-0-128-39 kubenswrapper[2558]: I0710 18:02:50.915496 2558 reconstruct.go:97] "Volume reconstruction finished" Jul 10 18:02:51.471057 ip-10-0-128-39 kubenswrapper[2558]: I0710 18:02:50.915503 2558 reconciler.go:29] "Reconciler: start to sync state" Jul 10 18:02:51.471057 ip-10-0-128-39 kubenswrapper[2558]: I0710 18:02:50.915590 2558 factory.go:55] Registering systemd factory Jul 10 18:02:51.471057 ip-10-0-128-39 kubenswrapper[2558]: I0710 18:02:50.915605 2558 factory.go:223] Registration of the systemd container factory successfully Jul 10 18:02:51.471057 ip-10-0-128-39 kubenswrapper[2558]: I0710 18:02:50.915777 2558 factory.go:153] Registering CRI-O factory Jul 10 18:02:51.471057 ip-10-0-128-39 kubenswrapper[2558]: I0710 18:02:50.915791 2558 factory.go:223] Registration of the crio container factory successfully Jul 10 18:02:51.471057 ip-10-0-128-39 kubenswrapper[2558]: I0710 18:02:50.915857 2558 factory.go:221] Registration of the containerd container factory failed: unable to create containerd client: containerd: cannot unix dial containerd api service: dial unix /run/containerd/containerd.sock: connect: no such file or directory Jul 10 18:02:51.471057 ip-10-0-128-39 kubenswrapper[2558]: I0710 18:02:50.915880 2558 factory.go:103] Registering Raw factory Jul 10 18:02:51.471057 ip-10-0-128-39 kubenswrapper[2558]: I0710 18:02:50.915894 2558 manager.go:1196] Started watching for new ooms in manager Jul 10 18:02:51.471057 ip-10-0-128-39 kubenswrapper[2558]: I0710 18:02:50.916482 2558 manager.go:349] Starting recovery of all containers Jul 10 18:02:51.471057 ip-10-0-128-39 kubenswrapper[2558]: E0710 18:02:50.917070 2558 reflector.go:205] "Failed to watch" err="failed to list *v1.CSIDriver: csidrivers.storage.k8s.io is forbidden: User \"system:anonymous\" cannot list resource \"csidrivers\" in API group \"storage.k8s.io\" at the cluster scope" logger="UnhandledError" reflector="k8s.io/client-go/informers/factory.go:160" type="*v1.CSIDriver" Jul 10 18:02:51.471057 ip-10-0-128-39 kubenswrapper[2558]: I0710 18:02:50.927673 2558 manager.go:354] Recovery completed Jul 10 18:02:51.471057 ip-10-0-128-39 kubenswrapper[2558]: E0710 18:02:50.927765 2558 controller.go:145] "Failed to ensure lease exists, will retry" err="leases.coordination.k8s.io \"ip-10-0-128-39.ec2.internal\" is forbidden: User \"system:anonymous\" cannot get resource \"leases\" in API group \"coordination.k8s.io\" in the namespace \"kube-node-lease\"" interval="200ms" Jul 10 18:02:51.471057 ip-10-0-128-39 kubenswrapper[2558]: I0710 18:02:50.928384 2558 kubelet_network_linux.go:54] "Initialized iptables rules." protocol="IPv6" Jul 10 18:02:51.471057 ip-10-0-128-39 kubenswrapper[2558]: I0710 18:02:50.931832 2558 kubelet_node_status.go:400] "Setting node annotation to enable volume controller attach/detach" Jul 10 18:02:51.471057 ip-10-0-128-39 kubenswrapper[2558]: I0710 18:02:50.933598 2558 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-128-39.ec2.internal" event="NodeHasSufficientMemory" Jul 10 18:02:51.471057 ip-10-0-128-39 kubenswrapper[2558]: I0710 18:02:50.933634 2558 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-128-39.ec2.internal" event="NodeHasNoDiskPressure" Jul 10 18:02:51.471057 ip-10-0-128-39 kubenswrapper[2558]: I0710 18:02:50.933649 2558 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-128-39.ec2.internal" event="NodeHasSufficientPID" Jul 10 18:02:51.471057 ip-10-0-128-39 kubenswrapper[2558]: I0710 18:02:50.934232 2558 cpu_manager.go:222] "Starting CPU manager" policy="none" Jul 10 18:02:51.471057 ip-10-0-128-39 kubenswrapper[2558]: I0710 18:02:50.934241 2558 cpu_manager.go:223] "Reconciling" reconcilePeriod="10s" Jul 10 18:02:51.471057 ip-10-0-128-39 kubenswrapper[2558]: I0710 18:02:50.934257 2558 state_mem.go:36] "Initialized new in-memory state store" Jul 10 18:02:51.471057 ip-10-0-128-39 kubenswrapper[2558]: I0710 18:02:50.934258 2558 kubelet_node_status.go:400] "Setting node annotation to enable volume controller attach/detach" Jul 10 18:02:51.471057 ip-10-0-128-39 kubenswrapper[2558]: I0710 18:02:50.935025 2558 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-128-39.ec2.internal" event="NodeHasSufficientMemory" Jul 10 18:02:51.471057 ip-10-0-128-39 kubenswrapper[2558]: I0710 18:02:50.935057 2558 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-128-39.ec2.internal" event="NodeHasNoDiskPressure" Jul 10 18:02:51.471057 ip-10-0-128-39 kubenswrapper[2558]: I0710 18:02:50.935069 2558 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-128-39.ec2.internal" event="NodeHasSufficientPID" Jul 10 18:02:51.471057 ip-10-0-128-39 kubenswrapper[2558]: I0710 18:02:50.935755 2558 policy_none.go:49] "None policy: Start" Jul 10 18:02:51.471057 ip-10-0-128-39 kubenswrapper[2558]: I0710 18:02:50.935768 2558 memory_manager.go:187] "Starting memorymanager" policy="None" Jul 10 18:02:51.472570 ip-10-0-128-39 kubenswrapper[2558]: I0710 18:02:50.935777 2558 state_mem.go:36] "Initializing new in-memory state store" logger="Memory Manager state checkpoint" Jul 10 18:02:51.472570 ip-10-0-128-39 kubenswrapper[2558]: I0710 18:02:50.936733 2558 policy_none.go:47] "Start" Jul 10 18:02:51.472570 ip-10-0-128-39 kubenswrapper[2558]: I0710 18:02:50.937139 2558 csr.go:274] "Certificate signing request is approved, waiting to be issued" logger="kubernetes.io/kube-apiserver-client-kubelet" csr="csr-ztbtl" Jul 10 18:02:51.472570 ip-10-0-128-39 kubenswrapper[2558]: I0710 18:02:50.944355 2558 csr.go:270] "Certificate signing request is issued" logger="kubernetes.io/kube-apiserver-client-kubelet" csr="csr-ztbtl" Jul 10 18:02:51.472570 ip-10-0-128-39 kubenswrapper[2558]: E0710 18:02:50.944519 2558 event.go:359] "Server rejected event (will not retry!)" err="events is forbidden: User \"system:anonymous\" cannot create resource \"events\" in API group \"\" in the namespace \"default\"" event="&Event{ObjectMeta:{ip-10-0-128-39.ec2.internal.18c0ffad00073e19 default 0 0001-01-01 00:00:00 +0000 UTC map[] map[] [] [] []},InvolvedObject:ObjectReference{Kind:Node,Namespace:,Name:ip-10-0-128-39.ec2.internal,UID:ip-10-0-128-39.ec2.internal,APIVersion:,ResourceVersion:,FieldPath:,},Reason:NodeHasSufficientMemory,Message:Node ip-10-0-128-39.ec2.internal status is now: NodeHasSufficientMemory,Source:EventSource{Component:kubelet,Host:ip-10-0-128-39.ec2.internal,},FirstTimestamp:2026-07-10 18:02:50.933616153 +0000 UTC m=+0.453744098,LastTimestamp:2026-07-10 18:02:50.933616153 +0000 UTC m=+0.453744098,Count:1,Type:Normal,EventTime:0001-01-01 00:00:00 +0000 UTC,Series:nil,Action:,Related:nil,ReportingController:kubelet,ReportingInstance:ip-10-0-128-39.ec2.internal,}" Jul 10 18:02:51.472570 ip-10-0-128-39 kubenswrapper[2558]: E0710 18:02:51.015654 2558 kubelet_node_status.go:454] "Error getting the current node from lister" err="node \"ip-10-0-128-39.ec2.internal\" not found" Jul 10 18:02:51.472570 ip-10-0-128-39 kubenswrapper[2558]: I0710 18:02:51.025266 2558 manager.go:337] "Starting Device Plugin manager" Jul 10 18:02:51.472570 ip-10-0-128-39 kubenswrapper[2558]: E0710 18:02:51.025316 2558 manager.go:513] "Failed to read data from checkpoint" err="checkpoint is not found" checkpoint="kubelet_internal_checkpoint" Jul 10 18:02:51.472570 ip-10-0-128-39 kubenswrapper[2558]: I0710 18:02:51.025326 2558 server.go:87] "Starting device plugin registration server" Jul 10 18:02:51.472570 ip-10-0-128-39 kubenswrapper[2558]: I0710 18:02:51.025643 2558 eviction_manager.go:189] "Eviction manager: starting control loop" Jul 10 18:02:51.472570 ip-10-0-128-39 kubenswrapper[2558]: I0710 18:02:51.025866 2558 container_log_manager.go:146] "Initializing container log rotate workers" workers=1 monitorPeriod="10s" Jul 10 18:02:51.472570 ip-10-0-128-39 kubenswrapper[2558]: I0710 18:02:51.025964 2558 plugin_watcher.go:51] "Plugin Watcher Start" path="/var/lib/kubelet/plugins_registry" Jul 10 18:02:51.472570 ip-10-0-128-39 kubenswrapper[2558]: I0710 18:02:51.026062 2558 plugin_manager.go:116] "The desired_state_of_world populator (plugin watcher) starts" Jul 10 18:02:51.472570 ip-10-0-128-39 kubenswrapper[2558]: I0710 18:02:51.026071 2558 plugin_manager.go:118] "Starting Kubelet Plugin Manager" Jul 10 18:02:51.472570 ip-10-0-128-39 kubenswrapper[2558]: E0710 18:02:51.026752 2558 eviction_manager.go:267] "eviction manager: failed to check if we have separate container filesystem. Ignoring." err="non-existent label \"crio-containers\"" Jul 10 18:02:51.472570 ip-10-0-128-39 kubenswrapper[2558]: E0710 18:02:51.026802 2558 eviction_manager.go:292] "Eviction manager: failed to get summary stats" err="failed to get node info: node \"ip-10-0-128-39.ec2.internal\" not found" Jul 10 18:02:51.472570 ip-10-0-128-39 kubenswrapper[2558]: I0710 18:02:51.077934 2558 kubelet_network_linux.go:54] "Initialized iptables rules." protocol="IPv4" Jul 10 18:02:51.472570 ip-10-0-128-39 kubenswrapper[2558]: I0710 18:02:51.077975 2558 status_manager.go:244] "Starting to sync pod status with apiserver" Jul 10 18:02:51.472570 ip-10-0-128-39 kubenswrapper[2558]: I0710 18:02:51.077997 2558 kubelet.go:2448] "Starting kubelet main sync loop" Jul 10 18:02:51.472570 ip-10-0-128-39 kubenswrapper[2558]: E0710 18:02:51.078027 2558 kubelet.go:2472] "Skipping pod synchronization" err="PLEG is not healthy: pleg has yet to be successful" Jul 10 18:02:51.472570 ip-10-0-128-39 kubenswrapper[2558]: I0710 18:02:51.080515 2558 reflector.go:439] "Caches populated" type="*v1.RuntimeClass" reflector="k8s.io/client-go/informers/factory.go:160" Jul 10 18:02:51.472570 ip-10-0-128-39 kubenswrapper[2558]: I0710 18:02:51.126893 2558 kubelet_node_status.go:400] "Setting node annotation to enable volume controller attach/detach" Jul 10 18:02:51.472570 ip-10-0-128-39 kubenswrapper[2558]: I0710 18:02:51.127846 2558 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-128-39.ec2.internal" event="NodeHasSufficientMemory" Jul 10 18:02:51.472570 ip-10-0-128-39 kubenswrapper[2558]: I0710 18:02:51.127872 2558 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-128-39.ec2.internal" event="NodeHasNoDiskPressure" Jul 10 18:02:51.472570 ip-10-0-128-39 kubenswrapper[2558]: I0710 18:02:51.127883 2558 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-128-39.ec2.internal" event="NodeHasSufficientPID" Jul 10 18:02:51.472570 ip-10-0-128-39 kubenswrapper[2558]: I0710 18:02:51.127905 2558 kubelet_node_status.go:78] "Attempting to register node" node="ip-10-0-128-39.ec2.internal" Jul 10 18:02:51.472570 ip-10-0-128-39 kubenswrapper[2558]: E0710 18:02:51.136649 2558 nodelease.go:49] "Failed to get node when trying to set owner ref to the node lease" err="nodes \"ip-10-0-128-39.ec2.internal\" not found" node="ip-10-0-128-39.ec2.internal" Jul 10 18:02:51.472570 ip-10-0-128-39 kubenswrapper[2558]: I0710 18:02:51.137813 2558 kubelet_node_status.go:81] "Successfully registered node" node="ip-10-0-128-39.ec2.internal" Jul 10 18:02:51.472570 ip-10-0-128-39 kubenswrapper[2558]: E0710 18:02:51.137832 2558 kubelet_node_status.go:536] "Error updating node status, will retry" err="error getting node \"ip-10-0-128-39.ec2.internal\": node \"ip-10-0-128-39.ec2.internal\" not found" Jul 10 18:02:51.472570 ip-10-0-128-39 kubenswrapper[2558]: E0710 18:02:51.167761 2558 kubelet_node_status.go:454] "Error getting the current node from lister" err="node \"ip-10-0-128-39.ec2.internal\" not found" Jul 10 18:02:51.472570 ip-10-0-128-39 kubenswrapper[2558]: I0710 18:02:51.178759 2558 kubelet.go:2535] "SyncLoop ADD" source="file" pods=["openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-128-39.ec2.internal","kube-system/kube-apiserver-proxy-ip-10-0-128-39.ec2.internal"] Jul 10 18:02:51.477491 ip-10-0-128-39 kubenswrapper[2558]: I0710 18:02:51.178838 2558 kubelet_node_status.go:400] "Setting node annotation to enable volume controller attach/detach" Jul 10 18:02:51.477491 ip-10-0-128-39 kubenswrapper[2558]: I0710 18:02:51.180013 2558 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-128-39.ec2.internal" event="NodeHasSufficientMemory" Jul 10 18:02:51.477491 ip-10-0-128-39 kubenswrapper[2558]: I0710 18:02:51.180041 2558 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-128-39.ec2.internal" event="NodeHasNoDiskPressure" Jul 10 18:02:51.477491 ip-10-0-128-39 kubenswrapper[2558]: I0710 18:02:51.180054 2558 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-128-39.ec2.internal" event="NodeHasSufficientPID" Jul 10 18:02:51.477491 ip-10-0-128-39 kubenswrapper[2558]: I0710 18:02:51.181603 2558 kubelet_node_status.go:400] "Setting node annotation to enable volume controller attach/detach" Jul 10 18:02:51.477491 ip-10-0-128-39 kubenswrapper[2558]: I0710 18:02:51.181717 2558 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-128-39.ec2.internal" Jul 10 18:02:51.477491 ip-10-0-128-39 kubenswrapper[2558]: I0710 18:02:51.181743 2558 kubelet_node_status.go:400] "Setting node annotation to enable volume controller attach/detach" Jul 10 18:02:51.477491 ip-10-0-128-39 kubenswrapper[2558]: I0710 18:02:51.182303 2558 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-128-39.ec2.internal" event="NodeHasSufficientMemory" Jul 10 18:02:51.477491 ip-10-0-128-39 kubenswrapper[2558]: I0710 18:02:51.182323 2558 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-128-39.ec2.internal" event="NodeHasNoDiskPressure" Jul 10 18:02:51.477491 ip-10-0-128-39 kubenswrapper[2558]: I0710 18:02:51.182333 2558 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-128-39.ec2.internal" event="NodeHasSufficientPID" Jul 10 18:02:51.477491 ip-10-0-128-39 kubenswrapper[2558]: I0710 18:02:51.183874 2558 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-128-39.ec2.internal" event="NodeHasSufficientMemory" Jul 10 18:02:51.477491 ip-10-0-128-39 kubenswrapper[2558]: I0710 18:02:51.183906 2558 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-128-39.ec2.internal" event="NodeHasNoDiskPressure" Jul 10 18:02:51.477491 ip-10-0-128-39 kubenswrapper[2558]: I0710 18:02:51.183920 2558 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-128-39.ec2.internal" event="NodeHasSufficientPID" Jul 10 18:02:51.477491 ip-10-0-128-39 kubenswrapper[2558]: I0710 18:02:51.185780 2558 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/kube-apiserver-proxy-ip-10-0-128-39.ec2.internal" Jul 10 18:02:51.477491 ip-10-0-128-39 kubenswrapper[2558]: I0710 18:02:51.185802 2558 kubelet_node_status.go:400] "Setting node annotation to enable volume controller attach/detach" Jul 10 18:02:51.477491 ip-10-0-128-39 kubenswrapper[2558]: I0710 18:02:51.186626 2558 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-128-39.ec2.internal" event="NodeHasSufficientMemory" Jul 10 18:02:51.477491 ip-10-0-128-39 kubenswrapper[2558]: I0710 18:02:51.186655 2558 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-128-39.ec2.internal" event="NodeHasNoDiskPressure" Jul 10 18:02:51.477491 ip-10-0-128-39 kubenswrapper[2558]: I0710 18:02:51.186672 2558 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-128-39.ec2.internal" event="NodeHasSufficientPID" Jul 10 18:02:51.477491 ip-10-0-128-39 kubenswrapper[2558]: I0710 18:02:51.215773 2558 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"config\" (UniqueName: \"kubernetes.io/host-path/91839b707975206c1eb988a2e4de26d3-config\") pod \"kube-apiserver-proxy-ip-10-0-128-39.ec2.internal\" (UID: \"91839b707975206c1eb988a2e4de26d3\") " pod="kube-system/kube-apiserver-proxy-ip-10-0-128-39.ec2.internal" Jul 10 18:02:51.477491 ip-10-0-128-39 kubenswrapper[2558]: I0710 18:02:51.215794 2558 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"etc-kube\" (UniqueName: \"kubernetes.io/host-path/a549cd29ca3f3dc6e1085c2e86957fb9-etc-kube\") pod \"kube-rbac-proxy-crio-ip-10-0-128-39.ec2.internal\" (UID: \"a549cd29ca3f3dc6e1085c2e86957fb9\") " pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-128-39.ec2.internal" Jul 10 18:02:51.477491 ip-10-0-128-39 kubenswrapper[2558]: I0710 18:02:51.215827 2558 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"var-lib-kubelet\" (UniqueName: \"kubernetes.io/host-path/a549cd29ca3f3dc6e1085c2e86957fb9-var-lib-kubelet\") pod \"kube-rbac-proxy-crio-ip-10-0-128-39.ec2.internal\" (UID: \"a549cd29ca3f3dc6e1085c2e86957fb9\") " pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-128-39.ec2.internal" Jul 10 18:02:51.477491 ip-10-0-128-39 kubenswrapper[2558]: E0710 18:02:51.225931 2558 kubelet.go:3244] "No need to create a mirror pod, since failed to get node info from the cluster" err="node \"ip-10-0-128-39.ec2.internal\" not found" node="ip-10-0-128-39.ec2.internal" Jul 10 18:02:51.477491 ip-10-0-128-39 kubenswrapper[2558]: E0710 18:02:51.230776 2558 kubelet.go:3244] "No need to create a mirror pod, since failed to get node info from the cluster" err="node \"ip-10-0-128-39.ec2.internal\" not found" node="ip-10-0-128-39.ec2.internal" Jul 10 18:02:51.477491 ip-10-0-128-39 kubenswrapper[2558]: E0710 18:02:51.268052 2558 kubelet_node_status.go:454] "Error getting the current node from lister" err="node \"ip-10-0-128-39.ec2.internal\" not found" Jul 10 18:02:51.477491 ip-10-0-128-39 kubenswrapper[2558]: I0710 18:02:51.316075 2558 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"etc-kube\" (UniqueName: \"kubernetes.io/host-path/a549cd29ca3f3dc6e1085c2e86957fb9-etc-kube\") pod \"kube-rbac-proxy-crio-ip-10-0-128-39.ec2.internal\" (UID: \"a549cd29ca3f3dc6e1085c2e86957fb9\") " pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-128-39.ec2.internal" Jul 10 18:02:52.713318 ip-10-0-128-39 kubenswrapper[2558]: I0710 18:02:51.316103 2558 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"var-lib-kubelet\" (UniqueName: \"kubernetes.io/host-path/a549cd29ca3f3dc6e1085c2e86957fb9-var-lib-kubelet\") pod \"kube-rbac-proxy-crio-ip-10-0-128-39.ec2.internal\" (UID: \"a549cd29ca3f3dc6e1085c2e86957fb9\") " pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-128-39.ec2.internal" Jul 10 18:02:52.713318 ip-10-0-128-39 kubenswrapper[2558]: I0710 18:02:51.316125 2558 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"config\" (UniqueName: \"kubernetes.io/host-path/91839b707975206c1eb988a2e4de26d3-config\") pod \"kube-apiserver-proxy-ip-10-0-128-39.ec2.internal\" (UID: \"91839b707975206c1eb988a2e4de26d3\") " pod="kube-system/kube-apiserver-proxy-ip-10-0-128-39.ec2.internal" Jul 10 18:02:52.713318 ip-10-0-128-39 kubenswrapper[2558]: I0710 18:02:51.316163 2558 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"config\" (UniqueName: \"kubernetes.io/host-path/91839b707975206c1eb988a2e4de26d3-config\") pod \"kube-apiserver-proxy-ip-10-0-128-39.ec2.internal\" (UID: \"91839b707975206c1eb988a2e4de26d3\") " pod="kube-system/kube-apiserver-proxy-ip-10-0-128-39.ec2.internal" Jul 10 18:02:52.713318 ip-10-0-128-39 kubenswrapper[2558]: I0710 18:02:51.316186 2558 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"etc-kube\" (UniqueName: \"kubernetes.io/host-path/a549cd29ca3f3dc6e1085c2e86957fb9-etc-kube\") pod \"kube-rbac-proxy-crio-ip-10-0-128-39.ec2.internal\" (UID: \"a549cd29ca3f3dc6e1085c2e86957fb9\") " pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-128-39.ec2.internal" Jul 10 18:02:52.713318 ip-10-0-128-39 kubenswrapper[2558]: I0710 18:02:51.316196 2558 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"var-lib-kubelet\" (UniqueName: \"kubernetes.io/host-path/a549cd29ca3f3dc6e1085c2e86957fb9-var-lib-kubelet\") pod \"kube-rbac-proxy-crio-ip-10-0-128-39.ec2.internal\" (UID: \"a549cd29ca3f3dc6e1085c2e86957fb9\") " pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-128-39.ec2.internal" Jul 10 18:02:52.713318 ip-10-0-128-39 kubenswrapper[2558]: E0710 18:02:51.368363 2558 kubelet_node_status.go:454] "Error getting the current node from lister" err="node \"ip-10-0-128-39.ec2.internal\" not found" Jul 10 18:02:52.713318 ip-10-0-128-39 kubenswrapper[2558]: E0710 18:02:51.468967 2558 kubelet_node_status.go:454] "Error getting the current node from lister" err="node \"ip-10-0-128-39.ec2.internal\" not found" Jul 10 18:02:52.713318 ip-10-0-128-39 kubenswrapper[2558]: I0710 18:02:51.528475 2558 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-128-39.ec2.internal" Jul 10 18:02:52.713318 ip-10-0-128-39 kubenswrapper[2558]: I0710 18:02:51.534780 2558 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/kube-apiserver-proxy-ip-10-0-128-39.ec2.internal" Jul 10 18:02:52.713318 ip-10-0-128-39 kubenswrapper[2558]: E0710 18:02:51.569592 2558 kubelet_node_status.go:454] "Error getting the current node from lister" err="node \"ip-10-0-128-39.ec2.internal\" not found" Jul 10 18:02:52.713318 ip-10-0-128-39 kubenswrapper[2558]: E0710 18:02:51.670083 2558 kubelet_node_status.go:454] "Error getting the current node from lister" err="node \"ip-10-0-128-39.ec2.internal\" not found" Jul 10 18:02:52.954182 ip-10-0-128-39 kubenswrapper[2558]: E0710 18:02:52.787268 2558 kubelet_node_status.go:454] "Error getting the current node from lister" err="node \"ip-10-0-128-39.ec2.internal\" not found" Jul 10 18:02:52.954182 ip-10-0-128-39 kubenswrapper[2558]: I0710 18:02:52.801019 2558 reflector.go:439] "Caches populated" type="*v1.Service" reflector="k8s.io/client-go/informers/factory.go:160" Jul 10 18:02:52.954182 ip-10-0-128-39 kubenswrapper[2558]: I0710 18:02:52.871610 2558 transport.go:147] "Certificate rotation detected, shutting down client connections to start using new credentials" Jul 10 18:02:52.954182 ip-10-0-128-39 kubenswrapper[2558]: I0710 18:02:52.871739 2558 reflector.go:571] "Warning: watch ended with error" reflector="k8s.io/client-go/informers/factory.go:160" type="*v1.RuntimeClass" err="very short watch: k8s.io/client-go/informers/factory.go:160: Unexpected watch close - watch lasted less than a second and no items received" Jul 10 18:02:52.954182 ip-10-0-128-39 kubenswrapper[2558]: I0710 18:02:52.871769 2558 reflector.go:571] "Warning: watch ended with error" reflector="k8s.io/client-go/informers/factory.go:160" type="*v1.Service" err="very short watch: k8s.io/client-go/informers/factory.go:160: Unexpected watch close - watch lasted less than a second and no items received" Jul 10 18:02:52.954182 ip-10-0-128-39 kubenswrapper[2558]: E0710 18:02:52.887467 2558 kubelet_node_status.go:454] "Error getting the current node from lister" err="node \"ip-10-0-128-39.ec2.internal\" not found" Jul 10 18:02:53.162630 ip-10-0-128-39 kubenswrapper[2558]: I0710 18:02:52.964477 2558 certificate_manager.go:715] "Certificate rotation deadline determined" logger="kubernetes.io/kube-apiserver-client-kubelet" expiration="2028-07-09 17:57:51 +0000 UTC" deadline="2028-04-08 06:40:32.434421096 +0000 UTC" Jul 10 18:02:53.162630 ip-10-0-128-39 kubenswrapper[2558]: I0710 18:02:52.964502 2558 certificate_manager.go:431] "Waiting for next certificate rotation" logger="kubernetes.io/kube-apiserver-client-kubelet" sleep="15300h37m39.469922215s" Jul 10 18:02:53.162630 ip-10-0-128-39 kubenswrapper[2558]: E0710 18:02:52.987557 2558 kubelet_node_status.go:454] "Error getting the current node from lister" err="node \"ip-10-0-128-39.ec2.internal\" not found" Jul 10 18:02:53.162630 ip-10-0-128-39 kubenswrapper[2558]: I0710 18:02:53.051287 2558 kubelet_node_status.go:400] "Setting node annotation to enable volume controller attach/detach" Jul 10 18:02:53.162630 ip-10-0-128-39 kubenswrapper[2558]: I0710 18:02:53.052296 2558 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-128-39.ec2.internal" event="NodeHasSufficientMemory" Jul 10 18:02:53.162630 ip-10-0-128-39 kubenswrapper[2558]: I0710 18:02:53.052325 2558 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-128-39.ec2.internal" event="NodeHasNoDiskPressure" Jul 10 18:02:53.162630 ip-10-0-128-39 kubenswrapper[2558]: I0710 18:02:53.052337 2558 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-128-39.ec2.internal" event="NodeHasSufficientPID" Jul 10 18:02:53.162630 ip-10-0-128-39 kubenswrapper[2558]: E0710 18:02:53.088234 2558 kubelet_node_status.go:454] "Error getting the current node from lister" err="node \"ip-10-0-128-39.ec2.internal\" not found" Jul 10 18:02:53.162630 ip-10-0-128-39 kubenswrapper[2558]: I0710 18:02:53.090240 2558 reflector.go:439] "Caches populated" type="*v1.CSIDriver" reflector="k8s.io/client-go/informers/factory.go:160" Jul 10 18:02:53.390189 ip-10-0-128-39 kubenswrapper[2558]: E0710 18:02:53.188875 2558 kubelet_node_status.go:454] "Error getting the current node from lister" err="node \"ip-10-0-128-39.ec2.internal\" not found" Jul 10 18:02:53.390189 ip-10-0-128-39 kubenswrapper[2558]: W0710 18:02:53.251882 2558 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod91839b707975206c1eb988a2e4de26d3.slice/crio-aab32dbfbc655c2795418fb80e0364d09576930551a70fba0f25ee7836f6c303 WatchSource:0}: Error finding container aab32dbfbc655c2795418fb80e0364d09576930551a70fba0f25ee7836f6c303: Status 404 returned error can't find the container with id aab32dbfbc655c2795418fb80e0364d09576930551a70fba0f25ee7836f6c303 Jul 10 18:02:53.390189 ip-10-0-128-39 kubenswrapper[2558]: W0710 18:02:53.252239 2558 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-poda549cd29ca3f3dc6e1085c2e86957fb9.slice/crio-1e6740d5f5b98de32e68e870d5f4d0922a9df9a92fb837c73b33c56feaa34f9c WatchSource:0}: Error finding container 1e6740d5f5b98de32e68e870d5f4d0922a9df9a92fb837c73b33c56feaa34f9c: Status 404 returned error can't find the container with id 1e6740d5f5b98de32e68e870d5f4d0922a9df9a92fb837c73b33c56feaa34f9c Jul 10 18:02:53.390189 ip-10-0-128-39 kubenswrapper[2558]: I0710 18:02:53.260280 2558 provider.go:93] Refreshing cache for provider: *credentialprovider.defaultDockerConfigProvider Jul 10 18:02:53.390189 ip-10-0-128-39 kubenswrapper[2558]: I0710 18:02:53.269872 2558 reflector.go:439] "Caches populated" type="*v1.Node" reflector="k8s.io/client-go/informers/factory.go:160" Jul 10 18:02:53.390189 ip-10-0-128-39 kubenswrapper[2558]: I0710 18:02:53.333670 2558 kubelet.go:3248] "Creating a mirror pod for static pod" pod="kube-system/kube-apiserver-proxy-ip-10-0-128-39.ec2.internal" Jul 10 18:02:53.390189 ip-10-0-128-39 kubenswrapper[2558]: I0710 18:02:53.341717 2558 warnings.go:110] "Warning: metadata.name: this is used in the Pod's hostname, which can result in surprising behavior; a DNS label is recommended: [must not contain dots]" Jul 10 18:02:53.390189 ip-10-0-128-39 kubenswrapper[2558]: I0710 18:02:53.342723 2558 kubelet.go:3248] "Creating a mirror pod for static pod" pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-128-39.ec2.internal" Jul 10 18:02:53.390189 ip-10-0-128-39 kubenswrapper[2558]: I0710 18:02:53.360273 2558 warnings.go:110] "Warning: metadata.name: this is used in the Pod's hostname, which can result in surprising behavior; a DNS label is recommended: [must not contain dots]" Jul 10 18:02:53.992681 ip-10-0-128-39 kubenswrapper[2558]: I0710 18:02:53.909685 2558 apiserver.go:52] "Watching apiserver" Jul 10 18:02:53.992681 ip-10-0-128-39 kubenswrapper[2558]: I0710 18:02:53.919199 2558 reflector.go:439] "Caches populated" type="*v1.Pod" reflector="pkg/kubelet/config/apiserver.go:66" Jul 10 18:02:53.992681 ip-10-0-128-39 kubenswrapper[2558]: I0710 18:02:53.919560 2558 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["kube-system/global-pull-secret-syncer-vsbdm","kube-system/konnectivity-agent-dvp86","kube-system/kube-apiserver-proxy-ip-10-0-128-39.ec2.internal","openshift-dns/node-resolver-jqb25","openshift-image-registry/node-ca-r7qh2","openshift-multus/multus-kvqn7","openshift-network-operator/iptables-alerter-zmg24","openshift-ovn-kubernetes/ovnkube-node-ll44q","openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-hps6m","openshift-cluster-node-tuning-operator/tuned-kbrll","openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-128-39.ec2.internal","openshift-multus/multus-additional-cni-plugins-v22pb","openshift-multus/network-metrics-daemon-jkr77","openshift-network-diagnostics/network-check-target-rzfgn"] Jul 10 18:02:53.992681 ip-10-0-128-39 kubenswrapper[2558]: I0710 18:02:53.921427 2558 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-operator/iptables-alerter-zmg24" Jul 10 18:02:53.992681 ip-10-0-128-39 kubenswrapper[2558]: I0710 18:02:53.923062 2558 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-jkr77" Jul 10 18:02:53.992681 ip-10-0-128-39 kubenswrapper[2558]: E0710 18:02:53.923130 2558 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-jkr77" podUID="7fa20252-06b7-48e4-8620-3e52dc980486" Jul 10 18:02:53.992681 ip-10-0-128-39 kubenswrapper[2558]: I0710 18:02:53.923726 2558 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-network-operator\"/\"iptables-alerter-dockercfg-n8p2z\"" Jul 10 18:02:53.992681 ip-10-0-128-39 kubenswrapper[2558]: I0710 18:02:53.924210 2558 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-network-operator\"/\"kube-root-ca.crt\"" Jul 10 18:02:53.992681 ip-10-0-128-39 kubenswrapper[2558]: I0710 18:02:53.924236 2558 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-network-operator\"/\"iptables-alerter-script\"" Jul 10 18:02:53.992681 ip-10-0-128-39 kubenswrapper[2558]: I0710 18:02:53.924265 2558 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-network-operator\"/\"openshift-service-ca.crt\"" Jul 10 18:02:53.992681 ip-10-0-128-39 kubenswrapper[2558]: I0710 18:02:53.924589 2558 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-rzfgn" Jul 10 18:02:53.992681 ip-10-0-128-39 kubenswrapper[2558]: E0710 18:02:53.924648 2558 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-rzfgn" podUID="06aa905d-9b4b-4d36-bafc-8fa0c11b3057" Jul 10 18:02:53.992681 ip-10-0-128-39 kubenswrapper[2558]: I0710 18:02:53.926364 2558 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-dns/node-resolver-jqb25" Jul 10 18:02:53.992681 ip-10-0-128-39 kubenswrapper[2558]: I0710 18:02:53.928541 2558 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-image-registry/node-ca-r7qh2" Jul 10 18:02:53.992681 ip-10-0-128-39 kubenswrapper[2558]: I0710 18:02:53.928870 2558 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-dns\"/\"openshift-service-ca.crt\"" Jul 10 18:02:53.992681 ip-10-0-128-39 kubenswrapper[2558]: I0710 18:02:53.929135 2558 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-dns\"/\"node-resolver-dockercfg-pn24c\"" Jul 10 18:02:53.992681 ip-10-0-128-39 kubenswrapper[2558]: I0710 18:02:53.929219 2558 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-dns\"/\"kube-root-ca.crt\"" Jul 10 18:02:53.992681 ip-10-0-128-39 kubenswrapper[2558]: I0710 18:02:53.929824 2558 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/multus-kvqn7" Jul 10 18:02:53.992681 ip-10-0-128-39 kubenswrapper[2558]: I0710 18:02:53.930954 2558 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-image-registry\"/\"openshift-service-ca.crt\"" Jul 10 18:02:53.992681 ip-10-0-128-39 kubenswrapper[2558]: I0710 18:02:53.931003 2558 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-image-registry\"/\"image-registry-certificates\"" Jul 10 18:02:53.992681 ip-10-0-128-39 kubenswrapper[2558]: I0710 18:02:53.931054 2558 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-image-registry\"/\"kube-root-ca.crt\"" Jul 10 18:02:53.992681 ip-10-0-128-39 kubenswrapper[2558]: I0710 18:02:53.931057 2558 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-image-registry\"/\"node-ca-dockercfg-68kwh\"" Jul 10 18:02:53.992681 ip-10-0-128-39 kubenswrapper[2558]: I0710 18:02:53.931798 2558 certificate_manager.go:566] "Rotating certificates" logger="kubernetes.io/kubelet-serving" Jul 10 18:02:53.992681 ip-10-0-128-39 kubenswrapper[2558]: I0710 18:02:53.931915 2558 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-multus\"/\"multus-daemon-config\"" Jul 10 18:02:53.992681 ip-10-0-128-39 kubenswrapper[2558]: I0710 18:02:53.932008 2558 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-hps6m" Jul 10 18:02:53.992681 ip-10-0-128-39 kubenswrapper[2558]: I0710 18:02:53.932767 2558 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-multus\"/\"cni-copy-resources\"" Jul 10 18:02:53.994161 ip-10-0-128-39 kubenswrapper[2558]: I0710 18:02:53.932780 2558 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-multus\"/\"kube-root-ca.crt\"" Jul 10 18:02:53.994161 ip-10-0-128-39 kubenswrapper[2558]: I0710 18:02:53.932913 2558 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-multus\"/\"openshift-service-ca.crt\"" Jul 10 18:02:53.994161 ip-10-0-128-39 kubenswrapper[2558]: I0710 18:02:53.932974 2558 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-multus\"/\"default-dockercfg-s45xp\"" Jul 10 18:02:53.994161 ip-10-0-128-39 kubenswrapper[2558]: I0710 18:02:53.933660 2558 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-vsbdm" Jul 10 18:02:53.994161 ip-10-0-128-39 kubenswrapper[2558]: E0710 18:02:53.933744 2558 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-vsbdm" podUID="3b27429f-90aa-4170-9078-0aa4ea0d0ab8" Jul 10 18:02:53.994161 ip-10-0-128-39 kubenswrapper[2558]: I0710 18:02:53.934428 2558 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-cluster-csi-drivers\"/\"aws-ebs-csi-driver-node-sa-dockercfg-dwl5b\"" Jul 10 18:02:53.994161 ip-10-0-128-39 kubenswrapper[2558]: I0710 18:02:53.934493 2558 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-cluster-csi-drivers\"/\"aws-ebs-csi-driver-node-metrics-serving-cert\"" Jul 10 18:02:53.994161 ip-10-0-128-39 kubenswrapper[2558]: I0710 18:02:53.934531 2558 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-cluster-csi-drivers\"/\"openshift-service-ca.crt\"" Jul 10 18:02:53.994161 ip-10-0-128-39 kubenswrapper[2558]: I0710 18:02:53.934567 2558 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-cluster-csi-drivers\"/\"kube-root-ca.crt\"" Jul 10 18:02:53.994161 ip-10-0-128-39 kubenswrapper[2558]: I0710 18:02:53.935101 2558 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/konnectivity-agent-dvp86" Jul 10 18:02:53.994161 ip-10-0-128-39 kubenswrapper[2558]: I0710 18:02:53.936701 2558 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-cluster-node-tuning-operator/tuned-kbrll" Jul 10 18:02:53.994161 ip-10-0-128-39 kubenswrapper[2558]: I0710 18:02:53.937347 2558 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"kube-system\"/\"konnectivity-ca-bundle\"" Jul 10 18:02:53.994161 ip-10-0-128-39 kubenswrapper[2558]: I0710 18:02:53.937512 2558 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"kube-system\"/\"default-dockercfg-sj2zt\"" Jul 10 18:02:53.994161 ip-10-0-128-39 kubenswrapper[2558]: I0710 18:02:53.937534 2558 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"kube-system\"/\"konnectivity-agent\"" Jul 10 18:02:53.994161 ip-10-0-128-39 kubenswrapper[2558]: I0710 18:02:53.939037 2558 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-cluster-node-tuning-operator\"/\"kube-root-ca.crt\"" Jul 10 18:02:53.994161 ip-10-0-128-39 kubenswrapper[2558]: I0710 18:02:53.939134 2558 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-cluster-node-tuning-operator\"/\"openshift-service-ca.crt\"" Jul 10 18:02:53.994161 ip-10-0-128-39 kubenswrapper[2558]: I0710 18:02:53.939540 2558 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/multus-additional-cni-plugins-v22pb" Jul 10 18:02:53.994161 ip-10-0-128-39 kubenswrapper[2558]: I0710 18:02:53.939582 2558 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-cluster-node-tuning-operator\"/\"tuned-dockercfg-kbqb8\"" Jul 10 18:02:53.994161 ip-10-0-128-39 kubenswrapper[2558]: I0710 18:02:53.940527 2558 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-run-multus-certs\" (UniqueName: \"kubernetes.io/host-path/11be9079-5d81-46c9-8f4c-c2e305bf2144-host-run-multus-certs\") pod \"multus-kvqn7\" (UID: \"11be9079-5d81-46c9-8f4c-c2e305bf2144\") " pod="openshift-multus/multus-kvqn7" Jul 10 18:02:53.994161 ip-10-0-128-39 kubenswrapper[2558]: I0710 18:02:53.940553 2558 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"dbus\" (UniqueName: \"kubernetes.io/host-path/3b27429f-90aa-4170-9078-0aa4ea0d0ab8-dbus\") pod \"global-pull-secret-syncer-vsbdm\" (UID: \"3b27429f-90aa-4170-9078-0aa4ea0d0ab8\") " pod="kube-system/global-pull-secret-syncer-vsbdm" Jul 10 18:02:53.994161 ip-10-0-128-39 kubenswrapper[2558]: I0710 18:02:53.940574 2558 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/12ad9a8e-4768-4362-818f-d2fb73c8a236-tmp-dir\") pod \"node-resolver-jqb25\" (UID: \"12ad9a8e-4768-4362-818f-d2fb73c8a236\") " pod="openshift-dns/node-resolver-jqb25" Jul 10 18:02:53.994161 ip-10-0-128-39 kubenswrapper[2558]: I0710 18:02:53.940597 2558 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"sys-fs\" (UniqueName: \"kubernetes.io/host-path/cdee596a-a663-4627-8c46-d7066d21978b-sys-fs\") pod \"aws-ebs-csi-driver-node-hps6m\" (UID: \"cdee596a-a663-4627-8c46-d7066d21978b\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-hps6m" Jul 10 18:02:53.994161 ip-10-0-128-39 kubenswrapper[2558]: I0710 18:02:53.940624 2558 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"system-cni-dir\" (UniqueName: \"kubernetes.io/host-path/11be9079-5d81-46c9-8f4c-c2e305bf2144-system-cni-dir\") pod \"multus-kvqn7\" (UID: \"11be9079-5d81-46c9-8f4c-c2e305bf2144\") " pod="openshift-multus/multus-kvqn7" Jul 10 18:02:53.994161 ip-10-0-128-39 kubenswrapper[2558]: I0710 18:02:53.940649 2558 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ovn-kubernetes/ovnkube-node-ll44q" Jul 10 18:02:53.995482 ip-10-0-128-39 kubenswrapper[2558]: I0710 18:02:53.940648 2558 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"multus-socket-dir-parent\" (UniqueName: \"kubernetes.io/host-path/11be9079-5d81-46c9-8f4c-c2e305bf2144-multus-socket-dir-parent\") pod \"multus-kvqn7\" (UID: \"11be9079-5d81-46c9-8f4c-c2e305bf2144\") " pod="openshift-multus/multus-kvqn7" Jul 10 18:02:53.995482 ip-10-0-128-39 kubenswrapper[2558]: I0710 18:02:53.940856 2558 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"multus-conf-dir\" (UniqueName: \"kubernetes.io/host-path/11be9079-5d81-46c9-8f4c-c2e305bf2144-multus-conf-dir\") pod \"multus-kvqn7\" (UID: \"11be9079-5d81-46c9-8f4c-c2e305bf2144\") " pod="openshift-multus/multus-kvqn7" Jul 10 18:02:53.995482 ip-10-0-128-39 kubenswrapper[2558]: I0710 18:02:53.940884 2558 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"original-pull-secret\" (UniqueName: \"kubernetes.io/secret/3b27429f-90aa-4170-9078-0aa4ea0d0ab8-original-pull-secret\") pod \"global-pull-secret-syncer-vsbdm\" (UID: \"3b27429f-90aa-4170-9078-0aa4ea0d0ab8\") " pod="kube-system/global-pull-secret-syncer-vsbdm" Jul 10 18:02:53.995482 ip-10-0-128-39 kubenswrapper[2558]: I0710 18:02:53.940909 2558 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kubelet-dir\" (UniqueName: \"kubernetes.io/host-path/cdee596a-a663-4627-8c46-d7066d21978b-kubelet-dir\") pod \"aws-ebs-csi-driver-node-hps6m\" (UID: \"cdee596a-a663-4627-8c46-d7066d21978b\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-hps6m" Jul 10 18:02:53.995482 ip-10-0-128-39 kubenswrapper[2558]: I0710 18:02:53.940931 2558 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-92748\" (UniqueName: \"kubernetes.io/projected/7fa20252-06b7-48e4-8620-3e52dc980486-kube-api-access-92748\") pod \"network-metrics-daemon-jkr77\" (UID: \"7fa20252-06b7-48e4-8620-3e52dc980486\") " pod="openshift-multus/network-metrics-daemon-jkr77" Jul 10 18:02:53.995482 ip-10-0-128-39 kubenswrapper[2558]: I0710 18:02:53.940952 2558 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-h74dt\" (UniqueName: \"kubernetes.io/projected/06aa905d-9b4b-4d36-bafc-8fa0c11b3057-kube-api-access-h74dt\") pod \"network-check-target-rzfgn\" (UID: \"06aa905d-9b4b-4d36-bafc-8fa0c11b3057\") " pod="openshift-network-diagnostics/network-check-target-rzfgn" Jul 10 18:02:53.995482 ip-10-0-128-39 kubenswrapper[2558]: I0710 18:02:53.940976 2558 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"multus-cni-dir\" (UniqueName: \"kubernetes.io/host-path/11be9079-5d81-46c9-8f4c-c2e305bf2144-multus-cni-dir\") pod \"multus-kvqn7\" (UID: \"11be9079-5d81-46c9-8f4c-c2e305bf2144\") " pod="openshift-multus/multus-kvqn7" Jul 10 18:02:53.995482 ip-10-0-128-39 kubenswrapper[2558]: I0710 18:02:53.940997 2558 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"iptables-alerter-script\" (UniqueName: \"kubernetes.io/configmap/1b18a733-4a78-4cf0-9377-3a26c9ae4a2f-iptables-alerter-script\") pod \"iptables-alerter-zmg24\" (UID: \"1b18a733-4a78-4cf0-9377-3a26c9ae4a2f\") " pod="openshift-network-operator/iptables-alerter-zmg24" Jul 10 18:02:53.995482 ip-10-0-128-39 kubenswrapper[2558]: I0710 18:02:53.941018 2558 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"cnibin\" (UniqueName: \"kubernetes.io/host-path/11be9079-5d81-46c9-8f4c-c2e305bf2144-cnibin\") pod \"multus-kvqn7\" (UID: \"11be9079-5d81-46c9-8f4c-c2e305bf2144\") " pod="openshift-multus/multus-kvqn7" Jul 10 18:02:53.995482 ip-10-0-128-39 kubenswrapper[2558]: I0710 18:02:53.941041 2558 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-run-netns\" (UniqueName: \"kubernetes.io/host-path/11be9079-5d81-46c9-8f4c-c2e305bf2144-host-run-netns\") pod \"multus-kvqn7\" (UID: \"11be9079-5d81-46c9-8f4c-c2e305bf2144\") " pod="openshift-multus/multus-kvqn7" Jul 10 18:02:53.995482 ip-10-0-128-39 kubenswrapper[2558]: I0710 18:02:53.941062 2558 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-var-lib-kubelet\" (UniqueName: \"kubernetes.io/host-path/11be9079-5d81-46c9-8f4c-c2e305bf2144-host-var-lib-kubelet\") pod \"multus-kvqn7\" (UID: \"11be9079-5d81-46c9-8f4c-c2e305bf2144\") " pod="openshift-multus/multus-kvqn7" Jul 10 18:02:53.995482 ip-10-0-128-39 kubenswrapper[2558]: I0710 18:02:53.941083 2558 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"etc-kubernetes\" (UniqueName: \"kubernetes.io/host-path/11be9079-5d81-46c9-8f4c-c2e305bf2144-etc-kubernetes\") pod \"multus-kvqn7\" (UID: \"11be9079-5d81-46c9-8f4c-c2e305bf2144\") " pod="openshift-multus/multus-kvqn7" Jul 10 18:02:53.995482 ip-10-0-128-39 kubenswrapper[2558]: I0710 18:02:53.941102 2558 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"serviceca\" (UniqueName: \"kubernetes.io/configmap/fe05c85a-2eb8-429b-9776-a3408a38903f-serviceca\") pod \"node-ca-r7qh2\" (UID: \"fe05c85a-2eb8-429b-9776-a3408a38903f\") " pod="openshift-image-registry/node-ca-r7qh2" Jul 10 18:02:53.995482 ip-10-0-128-39 kubenswrapper[2558]: I0710 18:02:53.941121 2558 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"hostroot\" (UniqueName: \"kubernetes.io/host-path/11be9079-5d81-46c9-8f4c-c2e305bf2144-hostroot\") pod \"multus-kvqn7\" (UID: \"11be9079-5d81-46c9-8f4c-c2e305bf2144\") " pod="openshift-multus/multus-kvqn7" Jul 10 18:02:53.997175 ip-10-0-128-39 kubenswrapper[2558]: I0710 18:02:53.941150 2558 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kubelet-config\" (UniqueName: \"kubernetes.io/host-path/3b27429f-90aa-4170-9078-0aa4ea0d0ab8-kubelet-config\") pod \"global-pull-secret-syncer-vsbdm\" (UID: \"3b27429f-90aa-4170-9078-0aa4ea0d0ab8\") " pod="kube-system/global-pull-secret-syncer-vsbdm" Jul 10 18:02:53.997175 ip-10-0-128-39 kubenswrapper[2558]: I0710 18:02:53.941170 2558 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"registration-dir\" (UniqueName: \"kubernetes.io/host-path/cdee596a-a663-4627-8c46-d7066d21978b-registration-dir\") pod \"aws-ebs-csi-driver-node-hps6m\" (UID: \"cdee596a-a663-4627-8c46-d7066d21978b\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-hps6m" Jul 10 18:02:53.997175 ip-10-0-128-39 kubenswrapper[2558]: I0710 18:02:53.941224 2558 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-t96dv\" (UniqueName: \"kubernetes.io/projected/cdee596a-a663-4627-8c46-d7066d21978b-kube-api-access-t96dv\") pod \"aws-ebs-csi-driver-node-hps6m\" (UID: \"cdee596a-a663-4627-8c46-d7066d21978b\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-hps6m" Jul 10 18:02:53.997175 ip-10-0-128-39 kubenswrapper[2558]: I0710 18:02:53.941246 2558 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/7fa20252-06b7-48e4-8620-3e52dc980486-metrics-certs\") pod \"network-metrics-daemon-jkr77\" (UID: \"7fa20252-06b7-48e4-8620-3e52dc980486\") " pod="openshift-multus/network-metrics-daemon-jkr77" Jul 10 18:02:53.997175 ip-10-0-128-39 kubenswrapper[2558]: I0710 18:02:53.941282 2558 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host\" (UniqueName: \"kubernetes.io/host-path/fe05c85a-2eb8-429b-9776-a3408a38903f-host\") pod \"node-ca-r7qh2\" (UID: \"fe05c85a-2eb8-429b-9776-a3408a38903f\") " pod="openshift-image-registry/node-ca-r7qh2" Jul 10 18:02:53.997175 ip-10-0-128-39 kubenswrapper[2558]: I0710 18:02:53.941346 2558 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"multus-daemon-config\" (UniqueName: \"kubernetes.io/configmap/11be9079-5d81-46c9-8f4c-c2e305bf2144-multus-daemon-config\") pod \"multus-kvqn7\" (UID: \"11be9079-5d81-46c9-8f4c-c2e305bf2144\") " pod="openshift-multus/multus-kvqn7" Jul 10 18:02:53.997175 ip-10-0-128-39 kubenswrapper[2558]: I0710 18:02:53.941374 2558 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"hosts-file\" (UniqueName: \"kubernetes.io/host-path/12ad9a8e-4768-4362-818f-d2fb73c8a236-hosts-file\") pod \"node-resolver-jqb25\" (UID: \"12ad9a8e-4768-4362-818f-d2fb73c8a236\") " pod="openshift-dns/node-resolver-jqb25" Jul 10 18:02:53.997175 ip-10-0-128-39 kubenswrapper[2558]: I0710 18:02:53.941397 2558 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"etc-selinux\" (UniqueName: \"kubernetes.io/host-path/cdee596a-a663-4627-8c46-d7066d21978b-etc-selinux\") pod \"aws-ebs-csi-driver-node-hps6m\" (UID: \"cdee596a-a663-4627-8c46-d7066d21978b\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-hps6m" Jul 10 18:02:53.997175 ip-10-0-128-39 kubenswrapper[2558]: I0710 18:02:53.941420 2558 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"os-release\" (UniqueName: \"kubernetes.io/host-path/11be9079-5d81-46c9-8f4c-c2e305bf2144-os-release\") pod \"multus-kvqn7\" (UID: \"11be9079-5d81-46c9-8f4c-c2e305bf2144\") " pod="openshift-multus/multus-kvqn7" Jul 10 18:02:53.997175 ip-10-0-128-39 kubenswrapper[2558]: I0710 18:02:53.941451 2558 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"cni-binary-copy\" (UniqueName: \"kubernetes.io/configmap/11be9079-5d81-46c9-8f4c-c2e305bf2144-cni-binary-copy\") pod \"multus-kvqn7\" (UID: \"11be9079-5d81-46c9-8f4c-c2e305bf2144\") " pod="openshift-multus/multus-kvqn7" Jul 10 18:02:53.997175 ip-10-0-128-39 kubenswrapper[2558]: I0710 18:02:53.941473 2558 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-var-lib-cni-bin\" (UniqueName: \"kubernetes.io/host-path/11be9079-5d81-46c9-8f4c-c2e305bf2144-host-var-lib-cni-bin\") pod \"multus-kvqn7\" (UID: \"11be9079-5d81-46c9-8f4c-c2e305bf2144\") " pod="openshift-multus/multus-kvqn7" Jul 10 18:02:53.997175 ip-10-0-128-39 kubenswrapper[2558]: I0710 18:02:53.941510 2558 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-var-lib-cni-multus\" (UniqueName: \"kubernetes.io/host-path/11be9079-5d81-46c9-8f4c-c2e305bf2144-host-var-lib-cni-multus\") pod \"multus-kvqn7\" (UID: \"11be9079-5d81-46c9-8f4c-c2e305bf2144\") " pod="openshift-multus/multus-kvqn7" Jul 10 18:02:53.997175 ip-10-0-128-39 kubenswrapper[2558]: I0710 18:02:53.941532 2558 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-hczxg\" (UniqueName: \"kubernetes.io/projected/11be9079-5d81-46c9-8f4c-c2e305bf2144-kube-api-access-hczxg\") pod \"multus-kvqn7\" (UID: \"11be9079-5d81-46c9-8f4c-c2e305bf2144\") " pod="openshift-multus/multus-kvqn7" Jul 10 18:02:53.997175 ip-10-0-128-39 kubenswrapper[2558]: I0710 18:02:53.941558 2558 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"device-dir\" (UniqueName: \"kubernetes.io/host-path/cdee596a-a663-4627-8c46-d7066d21978b-device-dir\") pod \"aws-ebs-csi-driver-node-hps6m\" (UID: \"cdee596a-a663-4627-8c46-d7066d21978b\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-hps6m" Jul 10 18:02:53.998046 ip-10-0-128-39 kubenswrapper[2558]: I0710 18:02:53.941587 2558 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-bdplb\" (UniqueName: \"kubernetes.io/projected/12ad9a8e-4768-4362-818f-d2fb73c8a236-kube-api-access-bdplb\") pod \"node-resolver-jqb25\" (UID: \"12ad9a8e-4768-4362-818f-d2fb73c8a236\") " pod="openshift-dns/node-resolver-jqb25" Jul 10 18:02:53.998046 ip-10-0-128-39 kubenswrapper[2558]: I0710 18:02:53.941610 2558 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"socket-dir\" (UniqueName: \"kubernetes.io/host-path/cdee596a-a663-4627-8c46-d7066d21978b-socket-dir\") pod \"aws-ebs-csi-driver-node-hps6m\" (UID: \"cdee596a-a663-4627-8c46-d7066d21978b\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-hps6m" Jul 10 18:02:53.998046 ip-10-0-128-39 kubenswrapper[2558]: I0710 18:02:53.941630 2558 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-slash\" (UniqueName: \"kubernetes.io/host-path/1b18a733-4a78-4cf0-9377-3a26c9ae4a2f-host-slash\") pod \"iptables-alerter-zmg24\" (UID: \"1b18a733-4a78-4cf0-9377-3a26c9ae4a2f\") " pod="openshift-network-operator/iptables-alerter-zmg24" Jul 10 18:02:53.998046 ip-10-0-128-39 kubenswrapper[2558]: I0710 18:02:53.941650 2558 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-z5tp5\" (UniqueName: \"kubernetes.io/projected/1b18a733-4a78-4cf0-9377-3a26c9ae4a2f-kube-api-access-z5tp5\") pod \"iptables-alerter-zmg24\" (UID: \"1b18a733-4a78-4cf0-9377-3a26c9ae4a2f\") " pod="openshift-network-operator/iptables-alerter-zmg24" Jul 10 18:02:53.998046 ip-10-0-128-39 kubenswrapper[2558]: I0710 18:02:53.941671 2558 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-xvx9r\" (UniqueName: \"kubernetes.io/projected/fe05c85a-2eb8-429b-9776-a3408a38903f-kube-api-access-xvx9r\") pod \"node-ca-r7qh2\" (UID: \"fe05c85a-2eb8-429b-9776-a3408a38903f\") " pod="openshift-image-registry/node-ca-r7qh2" Jul 10 18:02:53.998046 ip-10-0-128-39 kubenswrapper[2558]: I0710 18:02:53.941693 2558 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-run-k8s-cni-cncf-io\" (UniqueName: \"kubernetes.io/host-path/11be9079-5d81-46c9-8f4c-c2e305bf2144-host-run-k8s-cni-cncf-io\") pod \"multus-kvqn7\" (UID: \"11be9079-5d81-46c9-8f4c-c2e305bf2144\") " pod="openshift-multus/multus-kvqn7" Jul 10 18:02:53.998046 ip-10-0-128-39 kubenswrapper[2558]: I0710 18:02:53.941980 2558 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-multus\"/\"whereabouts-flatfile-config\"" Jul 10 18:02:53.998046 ip-10-0-128-39 kubenswrapper[2558]: I0710 18:02:53.941990 2558 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-multus\"/\"default-cni-sysctl-allowlist\"" Jul 10 18:02:53.998046 ip-10-0-128-39 kubenswrapper[2558]: I0710 18:02:53.942009 2558 reflector.go:439] "Caches populated" logger="kubernetes.io/kubelet-serving" type="*v1.CertificateSigningRequest" reflector="k8s.io/client-go/tools/watch/informerwatcher.go:162" Jul 10 18:02:53.998046 ip-10-0-128-39 kubenswrapper[2558]: I0710 18:02:53.942017 2558 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-multus\"/\"multus-ancillary-tools-dockercfg-dz8t5\"" Jul 10 18:02:53.998046 ip-10-0-128-39 kubenswrapper[2558]: I0710 18:02:53.942915 2558 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ovn-kubernetes\"/\"openshift-service-ca.crt\"" Jul 10 18:02:53.998046 ip-10-0-128-39 kubenswrapper[2558]: I0710 18:02:53.943153 2558 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ovn-kubernetes\"/\"env-overrides\"" Jul 10 18:02:53.998046 ip-10-0-128-39 kubenswrapper[2558]: I0710 18:02:53.943173 2558 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ovn-kubernetes\"/\"ovnkube-config\"" Jul 10 18:02:53.998046 ip-10-0-128-39 kubenswrapper[2558]: I0710 18:02:53.943177 2558 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ovn-kubernetes\"/\"ovn-node-metrics-cert\"" Jul 10 18:02:53.998046 ip-10-0-128-39 kubenswrapper[2558]: I0710 18:02:53.943628 2558 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ovn-kubernetes\"/\"ovn-kubernetes-node-dockercfg-jm7pt\"" Jul 10 18:02:53.998046 ip-10-0-128-39 kubenswrapper[2558]: I0710 18:02:53.944402 2558 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ovn-kubernetes\"/\"kube-root-ca.crt\"" Jul 10 18:02:53.998046 ip-10-0-128-39 kubenswrapper[2558]: I0710 18:02:53.944443 2558 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ovn-kubernetes\"/\"ovnkube-script-lib\"" Jul 10 18:02:53.998046 ip-10-0-128-39 kubenswrapper[2558]: I0710 18:02:53.967166 2558 csr.go:274] "Certificate signing request is approved, waiting to be issued" logger="kubernetes.io/kubelet-serving" csr="csr-zmmv9" Jul 10 18:02:53.998046 ip-10-0-128-39 kubenswrapper[2558]: I0710 18:02:53.972450 2558 reflector.go:439] "Caches populated" type="*v1.RuntimeClass" reflector="k8s.io/client-go/informers/factory.go:160" Jul 10 18:02:53.998046 ip-10-0-128-39 kubenswrapper[2558]: I0710 18:02:53.977667 2558 csr.go:270] "Certificate signing request is issued" logger="kubernetes.io/kubelet-serving" csr="csr-zmmv9" Jul 10 18:02:54.065916 ip-10-0-128-39 kubenswrapper[2558]: I0710 18:02:54.033759 2558 desired_state_of_world_populator.go:154] "Finished populating initial desired state of world" Jul 10 18:02:54.065916 ip-10-0-128-39 kubenswrapper[2558]: I0710 18:02:54.042168 2558 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-bdplb\" (UniqueName: \"kubernetes.io/projected/12ad9a8e-4768-4362-818f-d2fb73c8a236-kube-api-access-bdplb\") pod \"node-resolver-jqb25\" (UID: \"12ad9a8e-4768-4362-818f-d2fb73c8a236\") " pod="openshift-dns/node-resolver-jqb25" Jul 10 18:02:54.065916 ip-10-0-128-39 kubenswrapper[2558]: I0710 18:02:54.042206 2558 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"socket-dir\" (UniqueName: \"kubernetes.io/host-path/cdee596a-a663-4627-8c46-d7066d21978b-socket-dir\") pod \"aws-ebs-csi-driver-node-hps6m\" (UID: \"cdee596a-a663-4627-8c46-d7066d21978b\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-hps6m" Jul 10 18:02:54.065916 ip-10-0-128-39 kubenswrapper[2558]: I0710 18:02:54.042236 2558 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-slash\" (UniqueName: \"kubernetes.io/host-path/1b18a733-4a78-4cf0-9377-3a26c9ae4a2f-host-slash\") pod \"iptables-alerter-zmg24\" (UID: \"1b18a733-4a78-4cf0-9377-3a26c9ae4a2f\") " pod="openshift-network-operator/iptables-alerter-zmg24" Jul 10 18:02:54.065916 ip-10-0-128-39 kubenswrapper[2558]: I0710 18:02:54.042260 2558 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-xvx9r\" (UniqueName: \"kubernetes.io/projected/fe05c85a-2eb8-429b-9776-a3408a38903f-kube-api-access-xvx9r\") pod \"node-ca-r7qh2\" (UID: \"fe05c85a-2eb8-429b-9776-a3408a38903f\") " pod="openshift-image-registry/node-ca-r7qh2" Jul 10 18:02:54.065916 ip-10-0-128-39 kubenswrapper[2558]: I0710 18:02:54.042294 2558 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"etc-sysctl-conf\" (UniqueName: \"kubernetes.io/host-path/df37e213-59f3-4309-9bec-a41b2238ab98-etc-sysctl-conf\") pod \"tuned-kbrll\" (UID: \"df37e213-59f3-4309-9bec-a41b2238ab98\") " pod="openshift-cluster-node-tuning-operator/tuned-kbrll" Jul 10 18:02:54.065916 ip-10-0-128-39 kubenswrapper[2558]: I0710 18:02:54.042319 2558 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-kubelet\" (UniqueName: \"kubernetes.io/host-path/eabc6c69-b1e8-40d5-b41f-f49778a5248c-host-kubelet\") pod \"ovnkube-node-ll44q\" (UID: \"eabc6c69-b1e8-40d5-b41f-f49778a5248c\") " pod="openshift-ovn-kubernetes/ovnkube-node-ll44q" Jul 10 18:02:54.065916 ip-10-0-128-39 kubenswrapper[2558]: I0710 18:02:54.042345 2558 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"var-lib-openvswitch\" (UniqueName: \"kubernetes.io/host-path/eabc6c69-b1e8-40d5-b41f-f49778a5248c-var-lib-openvswitch\") pod \"ovnkube-node-ll44q\" (UID: \"eabc6c69-b1e8-40d5-b41f-f49778a5248c\") " pod="openshift-ovn-kubernetes/ovnkube-node-ll44q" Jul 10 18:02:54.065916 ip-10-0-128-39 kubenswrapper[2558]: I0710 18:02:54.042371 2558 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"dbus\" (UniqueName: \"kubernetes.io/host-path/3b27429f-90aa-4170-9078-0aa4ea0d0ab8-dbus\") pod \"global-pull-secret-syncer-vsbdm\" (UID: \"3b27429f-90aa-4170-9078-0aa4ea0d0ab8\") " pod="kube-system/global-pull-secret-syncer-vsbdm" Jul 10 18:02:54.065916 ip-10-0-128-39 kubenswrapper[2558]: I0710 18:02:54.042398 2558 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/12ad9a8e-4768-4362-818f-d2fb73c8a236-tmp-dir\") pod \"node-resolver-jqb25\" (UID: \"12ad9a8e-4768-4362-818f-d2fb73c8a236\") " pod="openshift-dns/node-resolver-jqb25" Jul 10 18:02:54.065916 ip-10-0-128-39 kubenswrapper[2558]: I0710 18:02:54.042420 2558 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"sys-fs\" (UniqueName: \"kubernetes.io/host-path/cdee596a-a663-4627-8c46-d7066d21978b-sys-fs\") pod \"aws-ebs-csi-driver-node-hps6m\" (UID: \"cdee596a-a663-4627-8c46-d7066d21978b\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-hps6m" Jul 10 18:02:54.065916 ip-10-0-128-39 kubenswrapper[2558]: I0710 18:02:54.042443 2558 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"etc-openvswitch\" (UniqueName: \"kubernetes.io/host-path/eabc6c69-b1e8-40d5-b41f-f49778a5248c-etc-openvswitch\") pod \"ovnkube-node-ll44q\" (UID: \"eabc6c69-b1e8-40d5-b41f-f49778a5248c\") " pod="openshift-ovn-kubernetes/ovnkube-node-ll44q" Jul 10 18:02:54.065916 ip-10-0-128-39 kubenswrapper[2558]: I0710 18:02:54.042466 2558 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"run-openvswitch\" (UniqueName: \"kubernetes.io/host-path/eabc6c69-b1e8-40d5-b41f-f49778a5248c-run-openvswitch\") pod \"ovnkube-node-ll44q\" (UID: \"eabc6c69-b1e8-40d5-b41f-f49778a5248c\") " pod="openshift-ovn-kubernetes/ovnkube-node-ll44q" Jul 10 18:02:54.065916 ip-10-0-128-39 kubenswrapper[2558]: I0710 18:02:54.042490 2558 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-275xj\" (UniqueName: \"kubernetes.io/projected/eabc6c69-b1e8-40d5-b41f-f49778a5248c-kube-api-access-275xj\") pod \"ovnkube-node-ll44q\" (UID: \"eabc6c69-b1e8-40d5-b41f-f49778a5248c\") " pod="openshift-ovn-kubernetes/ovnkube-node-ll44q" Jul 10 18:02:54.065916 ip-10-0-128-39 kubenswrapper[2558]: I0710 18:02:54.042514 2558 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"multus-socket-dir-parent\" (UniqueName: \"kubernetes.io/host-path/11be9079-5d81-46c9-8f4c-c2e305bf2144-multus-socket-dir-parent\") pod \"multus-kvqn7\" (UID: \"11be9079-5d81-46c9-8f4c-c2e305bf2144\") " pod="openshift-multus/multus-kvqn7" Jul 10 18:02:54.067010 ip-10-0-128-39 kubenswrapper[2558]: I0710 18:02:54.042536 2558 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"multus-conf-dir\" (UniqueName: \"kubernetes.io/host-path/11be9079-5d81-46c9-8f4c-c2e305bf2144-multus-conf-dir\") pod \"multus-kvqn7\" (UID: \"11be9079-5d81-46c9-8f4c-c2e305bf2144\") " pod="openshift-multus/multus-kvqn7" Jul 10 18:02:54.067010 ip-10-0-128-39 kubenswrapper[2558]: I0710 18:02:54.042560 2558 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"original-pull-secret\" (UniqueName: \"kubernetes.io/secret/3b27429f-90aa-4170-9078-0aa4ea0d0ab8-original-pull-secret\") pod \"global-pull-secret-syncer-vsbdm\" (UID: \"3b27429f-90aa-4170-9078-0aa4ea0d0ab8\") " pod="kube-system/global-pull-secret-syncer-vsbdm" Jul 10 18:02:54.067010 ip-10-0-128-39 kubenswrapper[2558]: I0710 18:02:54.042585 2558 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kubelet-dir\" (UniqueName: \"kubernetes.io/host-path/cdee596a-a663-4627-8c46-d7066d21978b-kubelet-dir\") pod \"aws-ebs-csi-driver-node-hps6m\" (UID: \"cdee596a-a663-4627-8c46-d7066d21978b\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-hps6m" Jul 10 18:02:54.067010 ip-10-0-128-39 kubenswrapper[2558]: I0710 18:02:54.042608 2558 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tmp\" (UniqueName: \"kubernetes.io/empty-dir/df37e213-59f3-4309-9bec-a41b2238ab98-tmp\") pod \"tuned-kbrll\" (UID: \"df37e213-59f3-4309-9bec-a41b2238ab98\") " pod="openshift-cluster-node-tuning-operator/tuned-kbrll" Jul 10 18:02:54.067010 ip-10-0-128-39 kubenswrapper[2558]: I0710 18:02:54.042630 2558 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"systemd-units\" (UniqueName: \"kubernetes.io/host-path/eabc6c69-b1e8-40d5-b41f-f49778a5248c-systemd-units\") pod \"ovnkube-node-ll44q\" (UID: \"eabc6c69-b1e8-40d5-b41f-f49778a5248c\") " pod="openshift-ovn-kubernetes/ovnkube-node-ll44q" Jul 10 18:02:54.067010 ip-10-0-128-39 kubenswrapper[2558]: I0710 18:02:54.042652 2558 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-cni-netd\" (UniqueName: \"kubernetes.io/host-path/eabc6c69-b1e8-40d5-b41f-f49778a5248c-host-cni-netd\") pod \"ovnkube-node-ll44q\" (UID: \"eabc6c69-b1e8-40d5-b41f-f49778a5248c\") " pod="openshift-ovn-kubernetes/ovnkube-node-ll44q" Jul 10 18:02:54.067010 ip-10-0-128-39 kubenswrapper[2558]: I0710 18:02:54.042675 2558 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"env-overrides\" (UniqueName: \"kubernetes.io/configmap/eabc6c69-b1e8-40d5-b41f-f49778a5248c-env-overrides\") pod \"ovnkube-node-ll44q\" (UID: \"eabc6c69-b1e8-40d5-b41f-f49778a5248c\") " pod="openshift-ovn-kubernetes/ovnkube-node-ll44q" Jul 10 18:02:54.067010 ip-10-0-128-39 kubenswrapper[2558]: I0710 18:02:54.042700 2558 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"multus-cni-dir\" (UniqueName: \"kubernetes.io/host-path/11be9079-5d81-46c9-8f4c-c2e305bf2144-multus-cni-dir\") pod \"multus-kvqn7\" (UID: \"11be9079-5d81-46c9-8f4c-c2e305bf2144\") " pod="openshift-multus/multus-kvqn7" Jul 10 18:02:54.067010 ip-10-0-128-39 kubenswrapper[2558]: I0710 18:02:54.042726 2558 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"iptables-alerter-script\" (UniqueName: \"kubernetes.io/configmap/1b18a733-4a78-4cf0-9377-3a26c9ae4a2f-iptables-alerter-script\") pod \"iptables-alerter-zmg24\" (UID: \"1b18a733-4a78-4cf0-9377-3a26c9ae4a2f\") " pod="openshift-network-operator/iptables-alerter-zmg24" Jul 10 18:02:54.067010 ip-10-0-128-39 kubenswrapper[2558]: I0710 18:02:54.042748 2558 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"etc-modprobe-d\" (UniqueName: \"kubernetes.io/host-path/df37e213-59f3-4309-9bec-a41b2238ab98-etc-modprobe-d\") pod \"tuned-kbrll\" (UID: \"df37e213-59f3-4309-9bec-a41b2238ab98\") " pod="openshift-cluster-node-tuning-operator/tuned-kbrll" Jul 10 18:02:54.067010 ip-10-0-128-39 kubenswrapper[2558]: I0710 18:02:54.042771 2558 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host\" (UniqueName: \"kubernetes.io/host-path/df37e213-59f3-4309-9bec-a41b2238ab98-host\") pod \"tuned-kbrll\" (UID: \"df37e213-59f3-4309-9bec-a41b2238ab98\") " pod="openshift-cluster-node-tuning-operator/tuned-kbrll" Jul 10 18:02:54.067010 ip-10-0-128-39 kubenswrapper[2558]: I0710 18:02:54.043737 2558 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"host-slash\" (UniqueName: \"kubernetes.io/host-path/1b18a733-4a78-4cf0-9377-3a26c9ae4a2f-host-slash\") pod \"iptables-alerter-zmg24\" (UID: \"1b18a733-4a78-4cf0-9377-3a26c9ae4a2f\") " pod="openshift-network-operator/iptables-alerter-zmg24" Jul 10 18:02:54.067010 ip-10-0-128-39 kubenswrapper[2558]: I0710 18:02:54.043880 2558 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"sys-fs\" (UniqueName: \"kubernetes.io/host-path/cdee596a-a663-4627-8c46-d7066d21978b-sys-fs\") pod \"aws-ebs-csi-driver-node-hps6m\" (UID: \"cdee596a-a663-4627-8c46-d7066d21978b\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-hps6m" Jul 10 18:02:54.067010 ip-10-0-128-39 kubenswrapper[2558]: I0710 18:02:54.043904 2558 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"dbus\" (UniqueName: \"kubernetes.io/host-path/3b27429f-90aa-4170-9078-0aa4ea0d0ab8-dbus\") pod \"global-pull-secret-syncer-vsbdm\" (UID: \"3b27429f-90aa-4170-9078-0aa4ea0d0ab8\") " pod="kube-system/global-pull-secret-syncer-vsbdm" Jul 10 18:02:54.067893 ip-10-0-128-39 kubenswrapper[2558]: I0710 18:02:54.044029 2558 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"socket-dir\" (UniqueName: \"kubernetes.io/host-path/cdee596a-a663-4627-8c46-d7066d21978b-socket-dir\") pod \"aws-ebs-csi-driver-node-hps6m\" (UID: \"cdee596a-a663-4627-8c46-d7066d21978b\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-hps6m" Jul 10 18:02:54.067893 ip-10-0-128-39 kubenswrapper[2558]: I0710 18:02:54.044099 2558 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"multus-socket-dir-parent\" (UniqueName: \"kubernetes.io/host-path/11be9079-5d81-46c9-8f4c-c2e305bf2144-multus-socket-dir-parent\") pod \"multus-kvqn7\" (UID: \"11be9079-5d81-46c9-8f4c-c2e305bf2144\") " pod="openshift-multus/multus-kvqn7" Jul 10 18:02:54.067893 ip-10-0-128-39 kubenswrapper[2558]: I0710 18:02:54.044134 2558 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"multus-conf-dir\" (UniqueName: \"kubernetes.io/host-path/11be9079-5d81-46c9-8f4c-c2e305bf2144-multus-conf-dir\") pod \"multus-kvqn7\" (UID: \"11be9079-5d81-46c9-8f4c-c2e305bf2144\") " pod="openshift-multus/multus-kvqn7" Jul 10 18:02:54.067893 ip-10-0-128-39 kubenswrapper[2558]: I0710 18:02:54.044189 2558 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/12ad9a8e-4768-4362-818f-d2fb73c8a236-tmp-dir\") pod \"node-resolver-jqb25\" (UID: \"12ad9a8e-4768-4362-818f-d2fb73c8a236\") " pod="openshift-dns/node-resolver-jqb25" Jul 10 18:02:54.067893 ip-10-0-128-39 kubenswrapper[2558]: E0710 18:02:54.044212 2558 secret.go:189] Couldn't get secret kube-system/original-pull-secret: object "kube-system"/"original-pull-secret" not registered Jul 10 18:02:54.067893 ip-10-0-128-39 kubenswrapper[2558]: I0710 18:02:54.045681 2558 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"multus-cni-dir\" (UniqueName: \"kubernetes.io/host-path/11be9079-5d81-46c9-8f4c-c2e305bf2144-multus-cni-dir\") pod \"multus-kvqn7\" (UID: \"11be9079-5d81-46c9-8f4c-c2e305bf2144\") " pod="openshift-multus/multus-kvqn7" Jul 10 18:02:54.067893 ip-10-0-128-39 kubenswrapper[2558]: I0710 18:02:54.045762 2558 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kubelet-dir\" (UniqueName: \"kubernetes.io/host-path/cdee596a-a663-4627-8c46-d7066d21978b-kubelet-dir\") pod \"aws-ebs-csi-driver-node-hps6m\" (UID: \"cdee596a-a663-4627-8c46-d7066d21978b\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-hps6m" Jul 10 18:02:54.067893 ip-10-0-128-39 kubenswrapper[2558]: I0710 18:02:54.046210 2558 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"iptables-alerter-script\" (UniqueName: \"kubernetes.io/configmap/1b18a733-4a78-4cf0-9377-3a26c9ae4a2f-iptables-alerter-script\") pod \"iptables-alerter-zmg24\" (UID: \"1b18a733-4a78-4cf0-9377-3a26c9ae4a2f\") " pod="openshift-network-operator/iptables-alerter-zmg24" Jul 10 18:02:54.067893 ip-10-0-128-39 kubenswrapper[2558]: E0710 18:02:54.065493 2558 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/3b27429f-90aa-4170-9078-0aa4ea0d0ab8-original-pull-secret podName:3b27429f-90aa-4170-9078-0aa4ea0d0ab8 nodeName:}" failed. No retries permitted until 2026-07-10 18:02:54.544270274 +0000 UTC m=+3.047762764 (durationBeforeRetry 500ms). Error: MountVolume.SetUp failed for volume "original-pull-secret" (UniqueName: "kubernetes.io/secret/3b27429f-90aa-4170-9078-0aa4ea0d0ab8-original-pull-secret") pod "global-pull-secret-syncer-vsbdm" (UID: "3b27429f-90aa-4170-9078-0aa4ea0d0ab8") : object "kube-system"/"original-pull-secret" not registered Jul 10 18:02:54.067893 ip-10-0-128-39 kubenswrapper[2558]: I0710 18:02:54.065545 2558 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"cnibin\" (UniqueName: \"kubernetes.io/host-path/11be9079-5d81-46c9-8f4c-c2e305bf2144-cnibin\") pod \"multus-kvqn7\" (UID: \"11be9079-5d81-46c9-8f4c-c2e305bf2144\") " pod="openshift-multus/multus-kvqn7" Jul 10 18:02:54.067893 ip-10-0-128-39 kubenswrapper[2558]: I0710 18:02:54.065575 2558 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-run-netns\" (UniqueName: \"kubernetes.io/host-path/11be9079-5d81-46c9-8f4c-c2e305bf2144-host-run-netns\") pod \"multus-kvqn7\" (UID: \"11be9079-5d81-46c9-8f4c-c2e305bf2144\") " pod="openshift-multus/multus-kvqn7" Jul 10 18:02:54.067893 ip-10-0-128-39 kubenswrapper[2558]: I0710 18:02:54.065598 2558 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"etc-kubernetes\" (UniqueName: \"kubernetes.io/host-path/11be9079-5d81-46c9-8f4c-c2e305bf2144-etc-kubernetes\") pod \"multus-kvqn7\" (UID: \"11be9079-5d81-46c9-8f4c-c2e305bf2144\") " pod="openshift-multus/multus-kvqn7" Jul 10 18:02:54.067893 ip-10-0-128-39 kubenswrapper[2558]: I0710 18:02:54.065623 2558 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"lib-modules\" (UniqueName: \"kubernetes.io/host-path/df37e213-59f3-4309-9bec-a41b2238ab98-lib-modules\") pod \"tuned-kbrll\" (UID: \"df37e213-59f3-4309-9bec-a41b2238ab98\") " pod="openshift-cluster-node-tuning-operator/tuned-kbrll" Jul 10 18:02:54.067893 ip-10-0-128-39 kubenswrapper[2558]: I0710 18:02:54.065646 2558 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"hostroot\" (UniqueName: \"kubernetes.io/host-path/11be9079-5d81-46c9-8f4c-c2e305bf2144-hostroot\") pod \"multus-kvqn7\" (UID: \"11be9079-5d81-46c9-8f4c-c2e305bf2144\") " pod="openshift-multus/multus-kvqn7" Jul 10 18:02:54.067893 ip-10-0-128-39 kubenswrapper[2558]: I0710 18:02:54.065669 2558 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kubelet-config\" (UniqueName: \"kubernetes.io/host-path/3b27429f-90aa-4170-9078-0aa4ea0d0ab8-kubelet-config\") pod \"global-pull-secret-syncer-vsbdm\" (UID: \"3b27429f-90aa-4170-9078-0aa4ea0d0ab8\") " pod="kube-system/global-pull-secret-syncer-vsbdm" Jul 10 18:02:54.068728 ip-10-0-128-39 kubenswrapper[2558]: I0710 18:02:54.065676 2558 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"host-run-netns\" (UniqueName: \"kubernetes.io/host-path/11be9079-5d81-46c9-8f4c-c2e305bf2144-host-run-netns\") pod \"multus-kvqn7\" (UID: \"11be9079-5d81-46c9-8f4c-c2e305bf2144\") " pod="openshift-multus/multus-kvqn7" Jul 10 18:02:54.068728 ip-10-0-128-39 kubenswrapper[2558]: I0710 18:02:54.065684 2558 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"cnibin\" (UniqueName: \"kubernetes.io/host-path/11be9079-5d81-46c9-8f4c-c2e305bf2144-cnibin\") pod \"multus-kvqn7\" (UID: \"11be9079-5d81-46c9-8f4c-c2e305bf2144\") " pod="openshift-multus/multus-kvqn7" Jul 10 18:02:54.068728 ip-10-0-128-39 kubenswrapper[2558]: I0710 18:02:54.065694 2558 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"registration-dir\" (UniqueName: \"kubernetes.io/host-path/cdee596a-a663-4627-8c46-d7066d21978b-registration-dir\") pod \"aws-ebs-csi-driver-node-hps6m\" (UID: \"cdee596a-a663-4627-8c46-d7066d21978b\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-hps6m" Jul 10 18:02:54.068728 ip-10-0-128-39 kubenswrapper[2558]: I0710 18:02:54.065718 2558 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"etc-kubernetes\" (UniqueName: \"kubernetes.io/host-path/11be9079-5d81-46c9-8f4c-c2e305bf2144-etc-kubernetes\") pod \"multus-kvqn7\" (UID: \"11be9079-5d81-46c9-8f4c-c2e305bf2144\") " pod="openshift-multus/multus-kvqn7" Jul 10 18:02:54.068728 ip-10-0-128-39 kubenswrapper[2558]: I0710 18:02:54.065736 2558 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"hostroot\" (UniqueName: \"kubernetes.io/host-path/11be9079-5d81-46c9-8f4c-c2e305bf2144-hostroot\") pod \"multus-kvqn7\" (UID: \"11be9079-5d81-46c9-8f4c-c2e305bf2144\") " pod="openshift-multus/multus-kvqn7" Jul 10 18:02:54.068728 ip-10-0-128-39 kubenswrapper[2558]: I0710 18:02:54.065753 2558 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-t96dv\" (UniqueName: \"kubernetes.io/projected/cdee596a-a663-4627-8c46-d7066d21978b-kube-api-access-t96dv\") pod \"aws-ebs-csi-driver-node-hps6m\" (UID: \"cdee596a-a663-4627-8c46-d7066d21978b\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-hps6m" Jul 10 18:02:54.068728 ip-10-0-128-39 kubenswrapper[2558]: I0710 18:02:54.065767 2558 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kubelet-config\" (UniqueName: \"kubernetes.io/host-path/3b27429f-90aa-4170-9078-0aa4ea0d0ab8-kubelet-config\") pod \"global-pull-secret-syncer-vsbdm\" (UID: \"3b27429f-90aa-4170-9078-0aa4ea0d0ab8\") " pod="kube-system/global-pull-secret-syncer-vsbdm" Jul 10 18:02:54.068728 ip-10-0-128-39 kubenswrapper[2558]: I0710 18:02:54.065791 2558 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/7fa20252-06b7-48e4-8620-3e52dc980486-metrics-certs\") pod \"network-metrics-daemon-jkr77\" (UID: \"7fa20252-06b7-48e4-8620-3e52dc980486\") " pod="openshift-multus/network-metrics-daemon-jkr77" Jul 10 18:02:54.068728 ip-10-0-128-39 kubenswrapper[2558]: I0710 18:02:54.065742 2558 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"registration-dir\" (UniqueName: \"kubernetes.io/host-path/cdee596a-a663-4627-8c46-d7066d21978b-registration-dir\") pod \"aws-ebs-csi-driver-node-hps6m\" (UID: \"cdee596a-a663-4627-8c46-d7066d21978b\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-hps6m" Jul 10 18:02:54.068728 ip-10-0-128-39 kubenswrapper[2558]: I0710 18:02:54.065934 2558 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host\" (UniqueName: \"kubernetes.io/host-path/fe05c85a-2eb8-429b-9776-a3408a38903f-host\") pod \"node-ca-r7qh2\" (UID: \"fe05c85a-2eb8-429b-9776-a3408a38903f\") " pod="openshift-image-registry/node-ca-r7qh2" Jul 10 18:02:54.068728 ip-10-0-128-39 kubenswrapper[2558]: I0710 18:02:54.065991 2558 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"host\" (UniqueName: \"kubernetes.io/host-path/fe05c85a-2eb8-429b-9776-a3408a38903f-host\") pod \"node-ca-r7qh2\" (UID: \"fe05c85a-2eb8-429b-9776-a3408a38903f\") " pod="openshift-image-registry/node-ca-r7qh2" Jul 10 18:02:54.068728 ip-10-0-128-39 kubenswrapper[2558]: E0710 18:02:54.066047 2558 secret.go:189] Couldn't get secret openshift-multus/metrics-daemon-secret: object "openshift-multus"/"metrics-daemon-secret" not registered Jul 10 18:02:54.068728 ip-10-0-128-39 kubenswrapper[2558]: E0710 18:02:54.067199 2558 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/7fa20252-06b7-48e4-8620-3e52dc980486-metrics-certs podName:7fa20252-06b7-48e4-8620-3e52dc980486 nodeName:}" failed. No retries permitted until 2026-07-10 18:02:54.567179424 +0000 UTC m=+3.070671923 (durationBeforeRetry 500ms). Error: MountVolume.SetUp failed for volume "metrics-certs" (UniqueName: "kubernetes.io/secret/7fa20252-06b7-48e4-8620-3e52dc980486-metrics-certs") pod "network-metrics-daemon-jkr77" (UID: "7fa20252-06b7-48e4-8620-3e52dc980486") : object "openshift-multus"/"metrics-daemon-secret" not registered Jul 10 18:02:54.068728 ip-10-0-128-39 kubenswrapper[2558]: I0710 18:02:54.067393 2558 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-run-netns\" (UniqueName: \"kubernetes.io/host-path/eabc6c69-b1e8-40d5-b41f-f49778a5248c-host-run-netns\") pod \"ovnkube-node-ll44q\" (UID: \"eabc6c69-b1e8-40d5-b41f-f49778a5248c\") " pod="openshift-ovn-kubernetes/ovnkube-node-ll44q" Jul 10 18:02:54.069523 ip-10-0-128-39 kubenswrapper[2558]: I0710 18:02:54.067448 2558 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"ovnkube-script-lib\" (UniqueName: \"kubernetes.io/configmap/eabc6c69-b1e8-40d5-b41f-f49778a5248c-ovnkube-script-lib\") pod \"ovnkube-node-ll44q\" (UID: \"eabc6c69-b1e8-40d5-b41f-f49778a5248c\") " pod="openshift-ovn-kubernetes/ovnkube-node-ll44q" Jul 10 18:02:54.069523 ip-10-0-128-39 kubenswrapper[2558]: I0710 18:02:54.067478 2558 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-slash\" (UniqueName: \"kubernetes.io/host-path/eabc6c69-b1e8-40d5-b41f-f49778a5248c-host-slash\") pod \"ovnkube-node-ll44q\" (UID: \"eabc6c69-b1e8-40d5-b41f-f49778a5248c\") " pod="openshift-ovn-kubernetes/ovnkube-node-ll44q" Jul 10 18:02:54.069523 ip-10-0-128-39 kubenswrapper[2558]: I0710 18:02:54.067503 2558 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"cni-binary-copy\" (UniqueName: \"kubernetes.io/configmap/53492c2f-2433-4658-af05-d58878aed082-cni-binary-copy\") pod \"multus-additional-cni-plugins-v22pb\" (UID: \"53492c2f-2433-4658-af05-d58878aed082\") " pod="openshift-multus/multus-additional-cni-plugins-v22pb" Jul 10 18:02:54.069523 ip-10-0-128-39 kubenswrapper[2558]: I0710 18:02:54.067561 2558 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tuning-conf-dir\" (UniqueName: \"kubernetes.io/host-path/53492c2f-2433-4658-af05-d58878aed082-tuning-conf-dir\") pod \"multus-additional-cni-plugins-v22pb\" (UID: \"53492c2f-2433-4658-af05-d58878aed082\") " pod="openshift-multus/multus-additional-cni-plugins-v22pb" Jul 10 18:02:54.069523 ip-10-0-128-39 kubenswrapper[2558]: I0710 18:02:54.067592 2558 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"os-release\" (UniqueName: \"kubernetes.io/host-path/11be9079-5d81-46c9-8f4c-c2e305bf2144-os-release\") pod \"multus-kvqn7\" (UID: \"11be9079-5d81-46c9-8f4c-c2e305bf2144\") " pod="openshift-multus/multus-kvqn7" Jul 10 18:02:54.069523 ip-10-0-128-39 kubenswrapper[2558]: I0710 18:02:54.067634 2558 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-var-lib-cni-multus\" (UniqueName: \"kubernetes.io/host-path/11be9079-5d81-46c9-8f4c-c2e305bf2144-host-var-lib-cni-multus\") pod \"multus-kvqn7\" (UID: \"11be9079-5d81-46c9-8f4c-c2e305bf2144\") " pod="openshift-multus/multus-kvqn7" Jul 10 18:02:54.069523 ip-10-0-128-39 kubenswrapper[2558]: I0710 18:02:54.067707 2558 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"host-var-lib-cni-multus\" (UniqueName: \"kubernetes.io/host-path/11be9079-5d81-46c9-8f4c-c2e305bf2144-host-var-lib-cni-multus\") pod \"multus-kvqn7\" (UID: \"11be9079-5d81-46c9-8f4c-c2e305bf2144\") " pod="openshift-multus/multus-kvqn7" Jul 10 18:02:54.069523 ip-10-0-128-39 kubenswrapper[2558]: I0710 18:02:54.067735 2558 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"os-release\" (UniqueName: \"kubernetes.io/host-path/11be9079-5d81-46c9-8f4c-c2e305bf2144-os-release\") pod \"multus-kvqn7\" (UID: \"11be9079-5d81-46c9-8f4c-c2e305bf2144\") " pod="openshift-multus/multus-kvqn7" Jul 10 18:02:54.069523 ip-10-0-128-39 kubenswrapper[2558]: I0710 18:02:54.067745 2558 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-hczxg\" (UniqueName: \"kubernetes.io/projected/11be9079-5d81-46c9-8f4c-c2e305bf2144-kube-api-access-hczxg\") pod \"multus-kvqn7\" (UID: \"11be9079-5d81-46c9-8f4c-c2e305bf2144\") " pod="openshift-multus/multus-kvqn7" Jul 10 18:02:54.069523 ip-10-0-128-39 kubenswrapper[2558]: I0710 18:02:54.067780 2558 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"konnectivity-ca\" (UniqueName: \"kubernetes.io/configmap/2da19865-b021-4789-b1e0-694d4f6c543d-konnectivity-ca\") pod \"konnectivity-agent-dvp86\" (UID: \"2da19865-b021-4789-b1e0-694d4f6c543d\") " pod="kube-system/konnectivity-agent-dvp86" Jul 10 18:02:54.069523 ip-10-0-128-39 kubenswrapper[2558]: I0710 18:02:54.067832 2558 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-cni-bin\" (UniqueName: \"kubernetes.io/host-path/eabc6c69-b1e8-40d5-b41f-f49778a5248c-host-cni-bin\") pod \"ovnkube-node-ll44q\" (UID: \"eabc6c69-b1e8-40d5-b41f-f49778a5248c\") " pod="openshift-ovn-kubernetes/ovnkube-node-ll44q" Jul 10 18:02:54.069523 ip-10-0-128-39 kubenswrapper[2558]: I0710 18:02:54.067862 2558 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-z5tp5\" (UniqueName: \"kubernetes.io/projected/1b18a733-4a78-4cf0-9377-3a26c9ae4a2f-kube-api-access-z5tp5\") pod \"iptables-alerter-zmg24\" (UID: \"1b18a733-4a78-4cf0-9377-3a26c9ae4a2f\") " pod="openshift-network-operator/iptables-alerter-zmg24" Jul 10 18:02:54.069523 ip-10-0-128-39 kubenswrapper[2558]: I0710 18:02:54.067887 2558 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"node-log\" (UniqueName: \"kubernetes.io/host-path/eabc6c69-b1e8-40d5-b41f-f49778a5248c-node-log\") pod \"ovnkube-node-ll44q\" (UID: \"eabc6c69-b1e8-40d5-b41f-f49778a5248c\") " pod="openshift-ovn-kubernetes/ovnkube-node-ll44q" Jul 10 18:02:54.069523 ip-10-0-128-39 kubenswrapper[2558]: I0710 18:02:54.067910 2558 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-run-ovn-kubernetes\" (UniqueName: \"kubernetes.io/host-path/eabc6c69-b1e8-40d5-b41f-f49778a5248c-host-run-ovn-kubernetes\") pod \"ovnkube-node-ll44q\" (UID: \"eabc6c69-b1e8-40d5-b41f-f49778a5248c\") " pod="openshift-ovn-kubernetes/ovnkube-node-ll44q" Jul 10 18:02:54.070251 ip-10-0-128-39 kubenswrapper[2558]: I0710 18:02:54.067938 2558 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-run-k8s-cni-cncf-io\" (UniqueName: \"kubernetes.io/host-path/11be9079-5d81-46c9-8f4c-c2e305bf2144-host-run-k8s-cni-cncf-io\") pod \"multus-kvqn7\" (UID: \"11be9079-5d81-46c9-8f4c-c2e305bf2144\") " pod="openshift-multus/multus-kvqn7" Jul 10 18:02:54.070251 ip-10-0-128-39 kubenswrapper[2558]: I0710 18:02:54.067964 2558 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-run-multus-certs\" (UniqueName: \"kubernetes.io/host-path/11be9079-5d81-46c9-8f4c-c2e305bf2144-host-run-multus-certs\") pod \"multus-kvqn7\" (UID: \"11be9079-5d81-46c9-8f4c-c2e305bf2144\") " pod="openshift-multus/multus-kvqn7" Jul 10 18:02:54.070251 ip-10-0-128-39 kubenswrapper[2558]: I0710 18:02:54.067989 2558 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"etc-sysconfig\" (UniqueName: \"kubernetes.io/host-path/df37e213-59f3-4309-9bec-a41b2238ab98-etc-sysconfig\") pod \"tuned-kbrll\" (UID: \"df37e213-59f3-4309-9bec-a41b2238ab98\") " pod="openshift-cluster-node-tuning-operator/tuned-kbrll" Jul 10 18:02:54.070251 ip-10-0-128-39 kubenswrapper[2558]: I0710 18:02:54.068012 2558 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"sys\" (UniqueName: \"kubernetes.io/host-path/df37e213-59f3-4309-9bec-a41b2238ab98-sys\") pod \"tuned-kbrll\" (UID: \"df37e213-59f3-4309-9bec-a41b2238ab98\") " pod="openshift-cluster-node-tuning-operator/tuned-kbrll" Jul 10 18:02:54.070251 ip-10-0-128-39 kubenswrapper[2558]: I0710 18:02:54.068034 2558 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-t7npn\" (UniqueName: \"kubernetes.io/projected/df37e213-59f3-4309-9bec-a41b2238ab98-kube-api-access-t7npn\") pod \"tuned-kbrll\" (UID: \"df37e213-59f3-4309-9bec-a41b2238ab98\") " pod="openshift-cluster-node-tuning-operator/tuned-kbrll" Jul 10 18:02:54.070251 ip-10-0-128-39 kubenswrapper[2558]: I0710 18:02:54.068059 2558 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"run-ovn\" (UniqueName: \"kubernetes.io/host-path/eabc6c69-b1e8-40d5-b41f-f49778a5248c-run-ovn\") pod \"ovnkube-node-ll44q\" (UID: \"eabc6c69-b1e8-40d5-b41f-f49778a5248c\") " pod="openshift-ovn-kubernetes/ovnkube-node-ll44q" Jul 10 18:02:54.070251 ip-10-0-128-39 kubenswrapper[2558]: I0710 18:02:54.068083 2558 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"cni-sysctl-allowlist\" (UniqueName: \"kubernetes.io/configmap/53492c2f-2433-4658-af05-d58878aed082-cni-sysctl-allowlist\") pod \"multus-additional-cni-plugins-v22pb\" (UID: \"53492c2f-2433-4658-af05-d58878aed082\") " pod="openshift-multus/multus-additional-cni-plugins-v22pb" Jul 10 18:02:54.070251 ip-10-0-128-39 kubenswrapper[2558]: I0710 18:02:54.068107 2558 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"system-cni-dir\" (UniqueName: \"kubernetes.io/host-path/11be9079-5d81-46c9-8f4c-c2e305bf2144-system-cni-dir\") pod \"multus-kvqn7\" (UID: \"11be9079-5d81-46c9-8f4c-c2e305bf2144\") " pod="openshift-multus/multus-kvqn7" Jul 10 18:02:54.070251 ip-10-0-128-39 kubenswrapper[2558]: I0710 18:02:54.068120 2558 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"host-run-multus-certs\" (UniqueName: \"kubernetes.io/host-path/11be9079-5d81-46c9-8f4c-c2e305bf2144-host-run-multus-certs\") pod \"multus-kvqn7\" (UID: \"11be9079-5d81-46c9-8f4c-c2e305bf2144\") " pod="openshift-multus/multus-kvqn7" Jul 10 18:02:54.070251 ip-10-0-128-39 kubenswrapper[2558]: I0710 18:02:54.068130 2558 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-92748\" (UniqueName: \"kubernetes.io/projected/7fa20252-06b7-48e4-8620-3e52dc980486-kube-api-access-92748\") pod \"network-metrics-daemon-jkr77\" (UID: \"7fa20252-06b7-48e4-8620-3e52dc980486\") " pod="openshift-multus/network-metrics-daemon-jkr77" Jul 10 18:02:54.070251 ip-10-0-128-39 kubenswrapper[2558]: I0710 18:02:54.068154 2558 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-h74dt\" (UniqueName: \"kubernetes.io/projected/06aa905d-9b4b-4d36-bafc-8fa0c11b3057-kube-api-access-h74dt\") pod \"network-check-target-rzfgn\" (UID: \"06aa905d-9b4b-4d36-bafc-8fa0c11b3057\") " pod="openshift-network-diagnostics/network-check-target-rzfgn" Jul 10 18:02:54.070251 ip-10-0-128-39 kubenswrapper[2558]: I0710 18:02:54.068191 2558 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"host-run-k8s-cni-cncf-io\" (UniqueName: \"kubernetes.io/host-path/11be9079-5d81-46c9-8f4c-c2e305bf2144-host-run-k8s-cni-cncf-io\") pod \"multus-kvqn7\" (UID: \"11be9079-5d81-46c9-8f4c-c2e305bf2144\") " pod="openshift-multus/multus-kvqn7" Jul 10 18:02:54.070251 ip-10-0-128-39 kubenswrapper[2558]: I0710 18:02:54.068221 2558 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"log-socket\" (UniqueName: \"kubernetes.io/host-path/eabc6c69-b1e8-40d5-b41f-f49778a5248c-log-socket\") pod \"ovnkube-node-ll44q\" (UID: \"eabc6c69-b1e8-40d5-b41f-f49778a5248c\") " pod="openshift-ovn-kubernetes/ovnkube-node-ll44q" Jul 10 18:02:54.070251 ip-10-0-128-39 kubenswrapper[2558]: I0710 18:02:54.068248 2558 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"os-release\" (UniqueName: \"kubernetes.io/host-path/53492c2f-2433-4658-af05-d58878aed082-os-release\") pod \"multus-additional-cni-plugins-v22pb\" (UID: \"53492c2f-2433-4658-af05-d58878aed082\") " pod="openshift-multus/multus-additional-cni-plugins-v22pb" Jul 10 18:02:54.071060 ip-10-0-128-39 kubenswrapper[2558]: I0710 18:02:54.068277 2558 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-var-lib-cni-networks-ovn-kubernetes\" (UniqueName: \"kubernetes.io/host-path/eabc6c69-b1e8-40d5-b41f-f49778a5248c-host-var-lib-cni-networks-ovn-kubernetes\") pod \"ovnkube-node-ll44q\" (UID: \"eabc6c69-b1e8-40d5-b41f-f49778a5248c\") " pod="openshift-ovn-kubernetes/ovnkube-node-ll44q" Jul 10 18:02:54.071060 ip-10-0-128-39 kubenswrapper[2558]: I0710 18:02:54.068303 2558 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"ovnkube-config\" (UniqueName: \"kubernetes.io/configmap/eabc6c69-b1e8-40d5-b41f-f49778a5248c-ovnkube-config\") pod \"ovnkube-node-ll44q\" (UID: \"eabc6c69-b1e8-40d5-b41f-f49778a5248c\") " pod="openshift-ovn-kubernetes/ovnkube-node-ll44q" Jul 10 18:02:54.071060 ip-10-0-128-39 kubenswrapper[2558]: I0710 18:02:54.068333 2558 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-var-lib-kubelet\" (UniqueName: \"kubernetes.io/host-path/11be9079-5d81-46c9-8f4c-c2e305bf2144-host-var-lib-kubelet\") pod \"multus-kvqn7\" (UID: \"11be9079-5d81-46c9-8f4c-c2e305bf2144\") " pod="openshift-multus/multus-kvqn7" Jul 10 18:02:54.071060 ip-10-0-128-39 kubenswrapper[2558]: I0710 18:02:54.068389 2558 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"serviceca\" (UniqueName: \"kubernetes.io/configmap/fe05c85a-2eb8-429b-9776-a3408a38903f-serviceca\") pod \"node-ca-r7qh2\" (UID: \"fe05c85a-2eb8-429b-9776-a3408a38903f\") " pod="openshift-image-registry/node-ca-r7qh2" Jul 10 18:02:54.071060 ip-10-0-128-39 kubenswrapper[2558]: I0710 18:02:54.068446 2558 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"system-cni-dir\" (UniqueName: \"kubernetes.io/host-path/11be9079-5d81-46c9-8f4c-c2e305bf2144-system-cni-dir\") pod \"multus-kvqn7\" (UID: \"11be9079-5d81-46c9-8f4c-c2e305bf2144\") " pod="openshift-multus/multus-kvqn7" Jul 10 18:02:54.071060 ip-10-0-128-39 kubenswrapper[2558]: I0710 18:02:54.068504 2558 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"host-var-lib-kubelet\" (UniqueName: \"kubernetes.io/host-path/11be9079-5d81-46c9-8f4c-c2e305bf2144-host-var-lib-kubelet\") pod \"multus-kvqn7\" (UID: \"11be9079-5d81-46c9-8f4c-c2e305bf2144\") " pod="openshift-multus/multus-kvqn7" Jul 10 18:02:54.071060 ip-10-0-128-39 kubenswrapper[2558]: I0710 18:02:54.068575 2558 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"system-cni-dir\" (UniqueName: \"kubernetes.io/host-path/53492c2f-2433-4658-af05-d58878aed082-system-cni-dir\") pod \"multus-additional-cni-plugins-v22pb\" (UID: \"53492c2f-2433-4658-af05-d58878aed082\") " pod="openshift-multus/multus-additional-cni-plugins-v22pb" Jul 10 18:02:54.071060 ip-10-0-128-39 kubenswrapper[2558]: I0710 18:02:54.068604 2558 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"cnibin\" (UniqueName: \"kubernetes.io/host-path/53492c2f-2433-4658-af05-d58878aed082-cnibin\") pod \"multus-additional-cni-plugins-v22pb\" (UID: \"53492c2f-2433-4658-af05-d58878aed082\") " pod="openshift-multus/multus-additional-cni-plugins-v22pb" Jul 10 18:02:54.071060 ip-10-0-128-39 kubenswrapper[2558]: I0710 18:02:54.068630 2558 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"etc-sysctl-d\" (UniqueName: \"kubernetes.io/host-path/df37e213-59f3-4309-9bec-a41b2238ab98-etc-sysctl-d\") pod \"tuned-kbrll\" (UID: \"df37e213-59f3-4309-9bec-a41b2238ab98\") " pod="openshift-cluster-node-tuning-operator/tuned-kbrll" Jul 10 18:02:54.071060 ip-10-0-128-39 kubenswrapper[2558]: I0710 18:02:54.068654 2558 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"multus-daemon-config\" (UniqueName: \"kubernetes.io/configmap/11be9079-5d81-46c9-8f4c-c2e305bf2144-multus-daemon-config\") pod \"multus-kvqn7\" (UID: \"11be9079-5d81-46c9-8f4c-c2e305bf2144\") " pod="openshift-multus/multus-kvqn7" Jul 10 18:02:54.071060 ip-10-0-128-39 kubenswrapper[2558]: I0710 18:02:54.068681 2558 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"hosts-file\" (UniqueName: \"kubernetes.io/host-path/12ad9a8e-4768-4362-818f-d2fb73c8a236-hosts-file\") pod \"node-resolver-jqb25\" (UID: \"12ad9a8e-4768-4362-818f-d2fb73c8a236\") " pod="openshift-dns/node-resolver-jqb25" Jul 10 18:02:54.071060 ip-10-0-128-39 kubenswrapper[2558]: I0710 18:02:54.068707 2558 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"etc-selinux\" (UniqueName: \"kubernetes.io/host-path/cdee596a-a663-4627-8c46-d7066d21978b-etc-selinux\") pod \"aws-ebs-csi-driver-node-hps6m\" (UID: \"cdee596a-a663-4627-8c46-d7066d21978b\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-hps6m" Jul 10 18:02:54.071060 ip-10-0-128-39 kubenswrapper[2558]: I0710 18:02:54.068731 2558 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"etc-tuned\" (UniqueName: \"kubernetes.io/empty-dir/df37e213-59f3-4309-9bec-a41b2238ab98-etc-tuned\") pod \"tuned-kbrll\" (UID: \"df37e213-59f3-4309-9bec-a41b2238ab98\") " pod="openshift-cluster-node-tuning-operator/tuned-kbrll" Jul 10 18:02:54.071060 ip-10-0-128-39 kubenswrapper[2558]: I0710 18:02:54.068754 2558 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"run-systemd\" (UniqueName: \"kubernetes.io/host-path/eabc6c69-b1e8-40d5-b41f-f49778a5248c-run-systemd\") pod \"ovnkube-node-ll44q\" (UID: \"eabc6c69-b1e8-40d5-b41f-f49778a5248c\") " pod="openshift-ovn-kubernetes/ovnkube-node-ll44q" Jul 10 18:02:54.072766 ip-10-0-128-39 kubenswrapper[2558]: I0710 18:02:54.068777 2558 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"ovn-node-metrics-cert\" (UniqueName: \"kubernetes.io/secret/eabc6c69-b1e8-40d5-b41f-f49778a5248c-ovn-node-metrics-cert\") pod \"ovnkube-node-ll44q\" (UID: \"eabc6c69-b1e8-40d5-b41f-f49778a5248c\") " pod="openshift-ovn-kubernetes/ovnkube-node-ll44q" Jul 10 18:02:54.072766 ip-10-0-128-39 kubenswrapper[2558]: I0710 18:02:54.068801 2558 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"whereabouts-flatfile-configmap\" (UniqueName: \"kubernetes.io/configmap/53492c2f-2433-4658-af05-d58878aed082-whereabouts-flatfile-configmap\") pod \"multus-additional-cni-plugins-v22pb\" (UID: \"53492c2f-2433-4658-af05-d58878aed082\") " pod="openshift-multus/multus-additional-cni-plugins-v22pb" Jul 10 18:02:54.072766 ip-10-0-128-39 kubenswrapper[2558]: I0710 18:02:54.068849 2558 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-znqb9\" (UniqueName: \"kubernetes.io/projected/53492c2f-2433-4658-af05-d58878aed082-kube-api-access-znqb9\") pod \"multus-additional-cni-plugins-v22pb\" (UID: \"53492c2f-2433-4658-af05-d58878aed082\") " pod="openshift-multus/multus-additional-cni-plugins-v22pb" Jul 10 18:02:54.072766 ip-10-0-128-39 kubenswrapper[2558]: I0710 18:02:54.068871 2558 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"run\" (UniqueName: \"kubernetes.io/host-path/df37e213-59f3-4309-9bec-a41b2238ab98-run\") pod \"tuned-kbrll\" (UID: \"df37e213-59f3-4309-9bec-a41b2238ab98\") " pod="openshift-cluster-node-tuning-operator/tuned-kbrll" Jul 10 18:02:54.072766 ip-10-0-128-39 kubenswrapper[2558]: I0710 18:02:54.068894 2558 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"var-lib-kubelet\" (UniqueName: \"kubernetes.io/host-path/df37e213-59f3-4309-9bec-a41b2238ab98-var-lib-kubelet\") pod \"tuned-kbrll\" (UID: \"df37e213-59f3-4309-9bec-a41b2238ab98\") " pod="openshift-cluster-node-tuning-operator/tuned-kbrll" Jul 10 18:02:54.072766 ip-10-0-128-39 kubenswrapper[2558]: I0710 18:02:54.068919 2558 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"cni-binary-copy\" (UniqueName: \"kubernetes.io/configmap/11be9079-5d81-46c9-8f4c-c2e305bf2144-cni-binary-copy\") pod \"multus-kvqn7\" (UID: \"11be9079-5d81-46c9-8f4c-c2e305bf2144\") " pod="openshift-multus/multus-kvqn7" Jul 10 18:02:54.072766 ip-10-0-128-39 kubenswrapper[2558]: I0710 18:02:54.068944 2558 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-var-lib-cni-bin\" (UniqueName: \"kubernetes.io/host-path/11be9079-5d81-46c9-8f4c-c2e305bf2144-host-var-lib-cni-bin\") pod \"multus-kvqn7\" (UID: \"11be9079-5d81-46c9-8f4c-c2e305bf2144\") " pod="openshift-multus/multus-kvqn7" Jul 10 18:02:54.072766 ip-10-0-128-39 kubenswrapper[2558]: I0710 18:02:54.068967 2558 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"device-dir\" (UniqueName: \"kubernetes.io/host-path/cdee596a-a663-4627-8c46-d7066d21978b-device-dir\") pod \"aws-ebs-csi-driver-node-hps6m\" (UID: \"cdee596a-a663-4627-8c46-d7066d21978b\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-hps6m" Jul 10 18:02:54.072766 ip-10-0-128-39 kubenswrapper[2558]: I0710 18:02:54.068991 2558 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"serviceca\" (UniqueName: \"kubernetes.io/configmap/fe05c85a-2eb8-429b-9776-a3408a38903f-serviceca\") pod \"node-ca-r7qh2\" (UID: \"fe05c85a-2eb8-429b-9776-a3408a38903f\") " pod="openshift-image-registry/node-ca-r7qh2" Jul 10 18:02:54.072766 ip-10-0-128-39 kubenswrapper[2558]: I0710 18:02:54.068994 2558 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"agent-certs\" (UniqueName: \"kubernetes.io/secret/2da19865-b021-4789-b1e0-694d4f6c543d-agent-certs\") pod \"konnectivity-agent-dvp86\" (UID: \"2da19865-b021-4789-b1e0-694d4f6c543d\") " pod="kube-system/konnectivity-agent-dvp86" Jul 10 18:02:54.072766 ip-10-0-128-39 kubenswrapper[2558]: I0710 18:02:54.069155 2558 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"etc-kubernetes\" (UniqueName: \"kubernetes.io/host-path/df37e213-59f3-4309-9bec-a41b2238ab98-etc-kubernetes\") pod \"tuned-kbrll\" (UID: \"df37e213-59f3-4309-9bec-a41b2238ab98\") " pod="openshift-cluster-node-tuning-operator/tuned-kbrll" Jul 10 18:02:54.072766 ip-10-0-128-39 kubenswrapper[2558]: I0710 18:02:54.069180 2558 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"etc-systemd\" (UniqueName: \"kubernetes.io/host-path/df37e213-59f3-4309-9bec-a41b2238ab98-etc-systemd\") pod \"tuned-kbrll\" (UID: \"df37e213-59f3-4309-9bec-a41b2238ab98\") " pod="openshift-cluster-node-tuning-operator/tuned-kbrll" Jul 10 18:02:54.072766 ip-10-0-128-39 kubenswrapper[2558]: I0710 18:02:54.069277 2558 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"hosts-file\" (UniqueName: \"kubernetes.io/host-path/12ad9a8e-4768-4362-818f-d2fb73c8a236-hosts-file\") pod \"node-resolver-jqb25\" (UID: \"12ad9a8e-4768-4362-818f-d2fb73c8a236\") " pod="openshift-dns/node-resolver-jqb25" Jul 10 18:02:54.072766 ip-10-0-128-39 kubenswrapper[2558]: I0710 18:02:54.069379 2558 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"etc-selinux\" (UniqueName: \"kubernetes.io/host-path/cdee596a-a663-4627-8c46-d7066d21978b-etc-selinux\") pod \"aws-ebs-csi-driver-node-hps6m\" (UID: \"cdee596a-a663-4627-8c46-d7066d21978b\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-hps6m" Jul 10 18:02:54.073622 ip-10-0-128-39 kubenswrapper[2558]: I0710 18:02:54.069452 2558 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"host-var-lib-cni-bin\" (UniqueName: \"kubernetes.io/host-path/11be9079-5d81-46c9-8f4c-c2e305bf2144-host-var-lib-cni-bin\") pod \"multus-kvqn7\" (UID: \"11be9079-5d81-46c9-8f4c-c2e305bf2144\") " pod="openshift-multus/multus-kvqn7" Jul 10 18:02:54.073622 ip-10-0-128-39 kubenswrapper[2558]: I0710 18:02:54.069511 2558 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"multus-daemon-config\" (UniqueName: \"kubernetes.io/configmap/11be9079-5d81-46c9-8f4c-c2e305bf2144-multus-daemon-config\") pod \"multus-kvqn7\" (UID: \"11be9079-5d81-46c9-8f4c-c2e305bf2144\") " pod="openshift-multus/multus-kvqn7" Jul 10 18:02:54.073622 ip-10-0-128-39 kubenswrapper[2558]: I0710 18:02:54.069580 2558 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"device-dir\" (UniqueName: \"kubernetes.io/host-path/cdee596a-a663-4627-8c46-d7066d21978b-device-dir\") pod \"aws-ebs-csi-driver-node-hps6m\" (UID: \"cdee596a-a663-4627-8c46-d7066d21978b\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-hps6m" Jul 10 18:02:54.073622 ip-10-0-128-39 kubenswrapper[2558]: I0710 18:02:54.069885 2558 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"cni-binary-copy\" (UniqueName: \"kubernetes.io/configmap/11be9079-5d81-46c9-8f4c-c2e305bf2144-cni-binary-copy\") pod \"multus-kvqn7\" (UID: \"11be9079-5d81-46c9-8f4c-c2e305bf2144\") " pod="openshift-multus/multus-kvqn7" Jul 10 18:02:54.073622 ip-10-0-128-39 kubenswrapper[2558]: I0710 18:02:54.071793 2558 swap_util.go:74] "error creating dir to test if tmpfs noswap is enabled. Assuming not supported" mount path="" error="stat /var/lib/kubelet/plugins/kubernetes.io/empty-dir: no such file or directory" Jul 10 18:02:54.078213 ip-10-0-128-39 kubenswrapper[2558]: I0710 18:02:54.077950 2558 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-xvx9r\" (UniqueName: \"kubernetes.io/projected/fe05c85a-2eb8-429b-9776-a3408a38903f-kube-api-access-xvx9r\") pod \"node-ca-r7qh2\" (UID: \"fe05c85a-2eb8-429b-9776-a3408a38903f\") " pod="openshift-image-registry/node-ca-r7qh2" Jul 10 18:02:54.176229 ip-10-0-128-39 kubenswrapper[2558]: E0710 18:02:54.080460 2558 projected.go:291] Couldn't get configMap openshift-network-diagnostics/kube-root-ca.crt: object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered Jul 10 18:02:54.176229 ip-10-0-128-39 kubenswrapper[2558]: E0710 18:02:54.080481 2558 projected.go:291] Couldn't get configMap openshift-network-diagnostics/openshift-service-ca.crt: object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered Jul 10 18:02:54.176229 ip-10-0-128-39 kubenswrapper[2558]: E0710 18:02:54.080495 2558 projected.go:196] Error preparing data for projected volume kube-api-access-h74dt for pod openshift-network-diagnostics/network-check-target-rzfgn: [object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered, object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered] Jul 10 18:02:54.176229 ip-10-0-128-39 kubenswrapper[2558]: E0710 18:02:54.080559 2558 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/projected/06aa905d-9b4b-4d36-bafc-8fa0c11b3057-kube-api-access-h74dt podName:06aa905d-9b4b-4d36-bafc-8fa0c11b3057 nodeName:}" failed. No retries permitted until 2026-07-10 18:02:54.580540624 +0000 UTC m=+3.084033111 (durationBeforeRetry 500ms). Error: MountVolume.SetUp failed for volume "kube-api-access-h74dt" (UniqueName: "kubernetes.io/projected/06aa905d-9b4b-4d36-bafc-8fa0c11b3057-kube-api-access-h74dt") pod "network-check-target-rzfgn" (UID: "06aa905d-9b4b-4d36-bafc-8fa0c11b3057") : [object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered, object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered] Jul 10 18:02:54.176229 ip-10-0-128-39 kubenswrapper[2558]: I0710 18:02:54.081571 2558 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-t96dv\" (UniqueName: \"kubernetes.io/projected/cdee596a-a663-4627-8c46-d7066d21978b-kube-api-access-t96dv\") pod \"aws-ebs-csi-driver-node-hps6m\" (UID: \"cdee596a-a663-4627-8c46-d7066d21978b\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-hps6m" Jul 10 18:02:54.176229 ip-10-0-128-39 kubenswrapper[2558]: I0710 18:02:54.084151 2558 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-bdplb\" (UniqueName: \"kubernetes.io/projected/12ad9a8e-4768-4362-818f-d2fb73c8a236-kube-api-access-bdplb\") pod \"node-resolver-jqb25\" (UID: \"12ad9a8e-4768-4362-818f-d2fb73c8a236\") " pod="openshift-dns/node-resolver-jqb25" Jul 10 18:02:54.176229 ip-10-0-128-39 kubenswrapper[2558]: I0710 18:02:54.084153 2558 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-z5tp5\" (UniqueName: \"kubernetes.io/projected/1b18a733-4a78-4cf0-9377-3a26c9ae4a2f-kube-api-access-z5tp5\") pod \"iptables-alerter-zmg24\" (UID: \"1b18a733-4a78-4cf0-9377-3a26c9ae4a2f\") " pod="openshift-network-operator/iptables-alerter-zmg24" Jul 10 18:02:54.176229 ip-10-0-128-39 kubenswrapper[2558]: I0710 18:02:54.084698 2558 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-hczxg\" (UniqueName: \"kubernetes.io/projected/11be9079-5d81-46c9-8f4c-c2e305bf2144-kube-api-access-hczxg\") pod \"multus-kvqn7\" (UID: \"11be9079-5d81-46c9-8f4c-c2e305bf2144\") " pod="openshift-multus/multus-kvqn7" Jul 10 18:02:54.176229 ip-10-0-128-39 kubenswrapper[2558]: I0710 18:02:54.086005 2558 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-92748\" (UniqueName: \"kubernetes.io/projected/7fa20252-06b7-48e4-8620-3e52dc980486-kube-api-access-92748\") pod \"network-metrics-daemon-jkr77\" (UID: \"7fa20252-06b7-48e4-8620-3e52dc980486\") " pod="openshift-multus/network-metrics-daemon-jkr77" Jul 10 18:02:54.176229 ip-10-0-128-39 kubenswrapper[2558]: I0710 18:02:54.102128 2558 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kube-system/kube-apiserver-proxy-ip-10-0-128-39.ec2.internal" event={"ID":"91839b707975206c1eb988a2e4de26d3","Type":"ContainerStarted","Data":"aab32dbfbc655c2795418fb80e0364d09576930551a70fba0f25ee7836f6c303"} Jul 10 18:02:54.176229 ip-10-0-128-39 kubenswrapper[2558]: I0710 18:02:54.103634 2558 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-128-39.ec2.internal" event={"ID":"a549cd29ca3f3dc6e1085c2e86957fb9","Type":"ContainerStarted","Data":"1e6740d5f5b98de32e68e870d5f4d0922a9df9a92fb837c73b33c56feaa34f9c"} Jul 10 18:02:54.176229 ip-10-0-128-39 kubenswrapper[2558]: I0710 18:02:54.170085 2558 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"etc-sysconfig\" (UniqueName: \"kubernetes.io/host-path/df37e213-59f3-4309-9bec-a41b2238ab98-etc-sysconfig\") pod \"tuned-kbrll\" (UID: \"df37e213-59f3-4309-9bec-a41b2238ab98\") " pod="openshift-cluster-node-tuning-operator/tuned-kbrll" Jul 10 18:02:54.176229 ip-10-0-128-39 kubenswrapper[2558]: I0710 18:02:54.170118 2558 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"sys\" (UniqueName: \"kubernetes.io/host-path/df37e213-59f3-4309-9bec-a41b2238ab98-sys\") pod \"tuned-kbrll\" (UID: \"df37e213-59f3-4309-9bec-a41b2238ab98\") " pod="openshift-cluster-node-tuning-operator/tuned-kbrll" Jul 10 18:02:54.176229 ip-10-0-128-39 kubenswrapper[2558]: I0710 18:02:54.170139 2558 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-t7npn\" (UniqueName: \"kubernetes.io/projected/df37e213-59f3-4309-9bec-a41b2238ab98-kube-api-access-t7npn\") pod \"tuned-kbrll\" (UID: \"df37e213-59f3-4309-9bec-a41b2238ab98\") " pod="openshift-cluster-node-tuning-operator/tuned-kbrll" Jul 10 18:02:54.176923 ip-10-0-128-39 kubenswrapper[2558]: I0710 18:02:54.170163 2558 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"run-ovn\" (UniqueName: \"kubernetes.io/host-path/eabc6c69-b1e8-40d5-b41f-f49778a5248c-run-ovn\") pod \"ovnkube-node-ll44q\" (UID: \"eabc6c69-b1e8-40d5-b41f-f49778a5248c\") " pod="openshift-ovn-kubernetes/ovnkube-node-ll44q" Jul 10 18:02:54.176923 ip-10-0-128-39 kubenswrapper[2558]: I0710 18:02:54.170186 2558 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"cni-sysctl-allowlist\" (UniqueName: \"kubernetes.io/configmap/53492c2f-2433-4658-af05-d58878aed082-cni-sysctl-allowlist\") pod \"multus-additional-cni-plugins-v22pb\" (UID: \"53492c2f-2433-4658-af05-d58878aed082\") " pod="openshift-multus/multus-additional-cni-plugins-v22pb" Jul 10 18:02:54.176923 ip-10-0-128-39 kubenswrapper[2558]: I0710 18:02:54.170214 2558 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"etc-sysconfig\" (UniqueName: \"kubernetes.io/host-path/df37e213-59f3-4309-9bec-a41b2238ab98-etc-sysconfig\") pod \"tuned-kbrll\" (UID: \"df37e213-59f3-4309-9bec-a41b2238ab98\") " pod="openshift-cluster-node-tuning-operator/tuned-kbrll" Jul 10 18:02:54.176923 ip-10-0-128-39 kubenswrapper[2558]: I0710 18:02:54.170227 2558 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"log-socket\" (UniqueName: \"kubernetes.io/host-path/eabc6c69-b1e8-40d5-b41f-f49778a5248c-log-socket\") pod \"ovnkube-node-ll44q\" (UID: \"eabc6c69-b1e8-40d5-b41f-f49778a5248c\") " pod="openshift-ovn-kubernetes/ovnkube-node-ll44q" Jul 10 18:02:54.176923 ip-10-0-128-39 kubenswrapper[2558]: I0710 18:02:54.170237 2558 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"sys\" (UniqueName: \"kubernetes.io/host-path/df37e213-59f3-4309-9bec-a41b2238ab98-sys\") pod \"tuned-kbrll\" (UID: \"df37e213-59f3-4309-9bec-a41b2238ab98\") " pod="openshift-cluster-node-tuning-operator/tuned-kbrll" Jul 10 18:02:54.176923 ip-10-0-128-39 kubenswrapper[2558]: I0710 18:02:54.170260 2558 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"log-socket\" (UniqueName: \"kubernetes.io/host-path/eabc6c69-b1e8-40d5-b41f-f49778a5248c-log-socket\") pod \"ovnkube-node-ll44q\" (UID: \"eabc6c69-b1e8-40d5-b41f-f49778a5248c\") " pod="openshift-ovn-kubernetes/ovnkube-node-ll44q" Jul 10 18:02:54.176923 ip-10-0-128-39 kubenswrapper[2558]: I0710 18:02:54.170262 2558 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"os-release\" (UniqueName: \"kubernetes.io/host-path/53492c2f-2433-4658-af05-d58878aed082-os-release\") pod \"multus-additional-cni-plugins-v22pb\" (UID: \"53492c2f-2433-4658-af05-d58878aed082\") " pod="openshift-multus/multus-additional-cni-plugins-v22pb" Jul 10 18:02:54.176923 ip-10-0-128-39 kubenswrapper[2558]: I0710 18:02:54.170299 2558 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-var-lib-cni-networks-ovn-kubernetes\" (UniqueName: \"kubernetes.io/host-path/eabc6c69-b1e8-40d5-b41f-f49778a5248c-host-var-lib-cni-networks-ovn-kubernetes\") pod \"ovnkube-node-ll44q\" (UID: \"eabc6c69-b1e8-40d5-b41f-f49778a5248c\") " pod="openshift-ovn-kubernetes/ovnkube-node-ll44q" Jul 10 18:02:54.176923 ip-10-0-128-39 kubenswrapper[2558]: I0710 18:02:54.170324 2558 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"ovnkube-config\" (UniqueName: \"kubernetes.io/configmap/eabc6c69-b1e8-40d5-b41f-f49778a5248c-ovnkube-config\") pod \"ovnkube-node-ll44q\" (UID: \"eabc6c69-b1e8-40d5-b41f-f49778a5248c\") " pod="openshift-ovn-kubernetes/ovnkube-node-ll44q" Jul 10 18:02:54.176923 ip-10-0-128-39 kubenswrapper[2558]: I0710 18:02:54.170349 2558 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"system-cni-dir\" (UniqueName: \"kubernetes.io/host-path/53492c2f-2433-4658-af05-d58878aed082-system-cni-dir\") pod \"multus-additional-cni-plugins-v22pb\" (UID: \"53492c2f-2433-4658-af05-d58878aed082\") " pod="openshift-multus/multus-additional-cni-plugins-v22pb" Jul 10 18:02:54.176923 ip-10-0-128-39 kubenswrapper[2558]: I0710 18:02:54.170373 2558 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"cnibin\" (UniqueName: \"kubernetes.io/host-path/53492c2f-2433-4658-af05-d58878aed082-cnibin\") pod \"multus-additional-cni-plugins-v22pb\" (UID: \"53492c2f-2433-4658-af05-d58878aed082\") " pod="openshift-multus/multus-additional-cni-plugins-v22pb" Jul 10 18:02:54.176923 ip-10-0-128-39 kubenswrapper[2558]: I0710 18:02:54.170393 2558 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"etc-sysctl-d\" (UniqueName: \"kubernetes.io/host-path/df37e213-59f3-4309-9bec-a41b2238ab98-etc-sysctl-d\") pod \"tuned-kbrll\" (UID: \"df37e213-59f3-4309-9bec-a41b2238ab98\") " pod="openshift-cluster-node-tuning-operator/tuned-kbrll" Jul 10 18:02:54.176923 ip-10-0-128-39 kubenswrapper[2558]: I0710 18:02:54.170417 2558 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"etc-tuned\" (UniqueName: \"kubernetes.io/empty-dir/df37e213-59f3-4309-9bec-a41b2238ab98-etc-tuned\") pod \"tuned-kbrll\" (UID: \"df37e213-59f3-4309-9bec-a41b2238ab98\") " pod="openshift-cluster-node-tuning-operator/tuned-kbrll" Jul 10 18:02:54.176923 ip-10-0-128-39 kubenswrapper[2558]: I0710 18:02:54.170438 2558 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"run-systemd\" (UniqueName: \"kubernetes.io/host-path/eabc6c69-b1e8-40d5-b41f-f49778a5248c-run-systemd\") pod \"ovnkube-node-ll44q\" (UID: \"eabc6c69-b1e8-40d5-b41f-f49778a5248c\") " pod="openshift-ovn-kubernetes/ovnkube-node-ll44q" Jul 10 18:02:54.177772 ip-10-0-128-39 kubenswrapper[2558]: I0710 18:02:54.170527 2558 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"ovn-node-metrics-cert\" (UniqueName: \"kubernetes.io/secret/eabc6c69-b1e8-40d5-b41f-f49778a5248c-ovn-node-metrics-cert\") pod \"ovnkube-node-ll44q\" (UID: \"eabc6c69-b1e8-40d5-b41f-f49778a5248c\") " pod="openshift-ovn-kubernetes/ovnkube-node-ll44q" Jul 10 18:02:54.177772 ip-10-0-128-39 kubenswrapper[2558]: I0710 18:02:54.170553 2558 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"whereabouts-flatfile-configmap\" (UniqueName: \"kubernetes.io/configmap/53492c2f-2433-4658-af05-d58878aed082-whereabouts-flatfile-configmap\") pod \"multus-additional-cni-plugins-v22pb\" (UID: \"53492c2f-2433-4658-af05-d58878aed082\") " pod="openshift-multus/multus-additional-cni-plugins-v22pb" Jul 10 18:02:54.177772 ip-10-0-128-39 kubenswrapper[2558]: I0710 18:02:54.170581 2558 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-znqb9\" (UniqueName: \"kubernetes.io/projected/53492c2f-2433-4658-af05-d58878aed082-kube-api-access-znqb9\") pod \"multus-additional-cni-plugins-v22pb\" (UID: \"53492c2f-2433-4658-af05-d58878aed082\") " pod="openshift-multus/multus-additional-cni-plugins-v22pb" Jul 10 18:02:54.177772 ip-10-0-128-39 kubenswrapper[2558]: I0710 18:02:54.170603 2558 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"run\" (UniqueName: \"kubernetes.io/host-path/df37e213-59f3-4309-9bec-a41b2238ab98-run\") pod \"tuned-kbrll\" (UID: \"df37e213-59f3-4309-9bec-a41b2238ab98\") " pod="openshift-cluster-node-tuning-operator/tuned-kbrll" Jul 10 18:02:54.177772 ip-10-0-128-39 kubenswrapper[2558]: I0710 18:02:54.170625 2558 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"var-lib-kubelet\" (UniqueName: \"kubernetes.io/host-path/df37e213-59f3-4309-9bec-a41b2238ab98-var-lib-kubelet\") pod \"tuned-kbrll\" (UID: \"df37e213-59f3-4309-9bec-a41b2238ab98\") " pod="openshift-cluster-node-tuning-operator/tuned-kbrll" Jul 10 18:02:54.177772 ip-10-0-128-39 kubenswrapper[2558]: I0710 18:02:54.170654 2558 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"agent-certs\" (UniqueName: \"kubernetes.io/secret/2da19865-b021-4789-b1e0-694d4f6c543d-agent-certs\") pod \"konnectivity-agent-dvp86\" (UID: \"2da19865-b021-4789-b1e0-694d4f6c543d\") " pod="kube-system/konnectivity-agent-dvp86" Jul 10 18:02:54.177772 ip-10-0-128-39 kubenswrapper[2558]: I0710 18:02:54.170678 2558 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"etc-kubernetes\" (UniqueName: \"kubernetes.io/host-path/df37e213-59f3-4309-9bec-a41b2238ab98-etc-kubernetes\") pod \"tuned-kbrll\" (UID: \"df37e213-59f3-4309-9bec-a41b2238ab98\") " pod="openshift-cluster-node-tuning-operator/tuned-kbrll" Jul 10 18:02:54.177772 ip-10-0-128-39 kubenswrapper[2558]: I0710 18:02:54.170701 2558 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"etc-systemd\" (UniqueName: \"kubernetes.io/host-path/df37e213-59f3-4309-9bec-a41b2238ab98-etc-systemd\") pod \"tuned-kbrll\" (UID: \"df37e213-59f3-4309-9bec-a41b2238ab98\") " pod="openshift-cluster-node-tuning-operator/tuned-kbrll" Jul 10 18:02:54.177772 ip-10-0-128-39 kubenswrapper[2558]: I0710 18:02:54.170733 2558 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"etc-sysctl-conf\" (UniqueName: \"kubernetes.io/host-path/df37e213-59f3-4309-9bec-a41b2238ab98-etc-sysctl-conf\") pod \"tuned-kbrll\" (UID: \"df37e213-59f3-4309-9bec-a41b2238ab98\") " pod="openshift-cluster-node-tuning-operator/tuned-kbrll" Jul 10 18:02:54.177772 ip-10-0-128-39 kubenswrapper[2558]: I0710 18:02:54.170758 2558 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-kubelet\" (UniqueName: \"kubernetes.io/host-path/eabc6c69-b1e8-40d5-b41f-f49778a5248c-host-kubelet\") pod \"ovnkube-node-ll44q\" (UID: \"eabc6c69-b1e8-40d5-b41f-f49778a5248c\") " pod="openshift-ovn-kubernetes/ovnkube-node-ll44q" Jul 10 18:02:54.177772 ip-10-0-128-39 kubenswrapper[2558]: I0710 18:02:54.170785 2558 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"var-lib-openvswitch\" (UniqueName: \"kubernetes.io/host-path/eabc6c69-b1e8-40d5-b41f-f49778a5248c-var-lib-openvswitch\") pod \"ovnkube-node-ll44q\" (UID: \"eabc6c69-b1e8-40d5-b41f-f49778a5248c\") " pod="openshift-ovn-kubernetes/ovnkube-node-ll44q" Jul 10 18:02:54.177772 ip-10-0-128-39 kubenswrapper[2558]: I0710 18:02:54.170831 2558 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"etc-openvswitch\" (UniqueName: \"kubernetes.io/host-path/eabc6c69-b1e8-40d5-b41f-f49778a5248c-etc-openvswitch\") pod \"ovnkube-node-ll44q\" (UID: \"eabc6c69-b1e8-40d5-b41f-f49778a5248c\") " pod="openshift-ovn-kubernetes/ovnkube-node-ll44q" Jul 10 18:02:54.177772 ip-10-0-128-39 kubenswrapper[2558]: I0710 18:02:54.170856 2558 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"run-openvswitch\" (UniqueName: \"kubernetes.io/host-path/eabc6c69-b1e8-40d5-b41f-f49778a5248c-run-openvswitch\") pod \"ovnkube-node-ll44q\" (UID: \"eabc6c69-b1e8-40d5-b41f-f49778a5248c\") " pod="openshift-ovn-kubernetes/ovnkube-node-ll44q" Jul 10 18:02:54.177772 ip-10-0-128-39 kubenswrapper[2558]: I0710 18:02:54.170865 2558 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"cni-sysctl-allowlist\" (UniqueName: \"kubernetes.io/configmap/53492c2f-2433-4658-af05-d58878aed082-cni-sysctl-allowlist\") pod \"multus-additional-cni-plugins-v22pb\" (UID: \"53492c2f-2433-4658-af05-d58878aed082\") " pod="openshift-multus/multus-additional-cni-plugins-v22pb" Jul 10 18:02:54.179051 ip-10-0-128-39 kubenswrapper[2558]: I0710 18:02:54.170880 2558 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-275xj\" (UniqueName: \"kubernetes.io/projected/eabc6c69-b1e8-40d5-b41f-f49778a5248c-kube-api-access-275xj\") pod \"ovnkube-node-ll44q\" (UID: \"eabc6c69-b1e8-40d5-b41f-f49778a5248c\") " pod="openshift-ovn-kubernetes/ovnkube-node-ll44q" Jul 10 18:02:54.179051 ip-10-0-128-39 kubenswrapper[2558]: I0710 18:02:54.170908 2558 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"run-ovn\" (UniqueName: \"kubernetes.io/host-path/eabc6c69-b1e8-40d5-b41f-f49778a5248c-run-ovn\") pod \"ovnkube-node-ll44q\" (UID: \"eabc6c69-b1e8-40d5-b41f-f49778a5248c\") " pod="openshift-ovn-kubernetes/ovnkube-node-ll44q" Jul 10 18:02:54.179051 ip-10-0-128-39 kubenswrapper[2558]: I0710 18:02:54.170922 2558 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tmp\" (UniqueName: \"kubernetes.io/empty-dir/df37e213-59f3-4309-9bec-a41b2238ab98-tmp\") pod \"tuned-kbrll\" (UID: \"df37e213-59f3-4309-9bec-a41b2238ab98\") " pod="openshift-cluster-node-tuning-operator/tuned-kbrll" Jul 10 18:02:54.179051 ip-10-0-128-39 kubenswrapper[2558]: I0710 18:02:54.170948 2558 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"systemd-units\" (UniqueName: \"kubernetes.io/host-path/eabc6c69-b1e8-40d5-b41f-f49778a5248c-systemd-units\") pod \"ovnkube-node-ll44q\" (UID: \"eabc6c69-b1e8-40d5-b41f-f49778a5248c\") " pod="openshift-ovn-kubernetes/ovnkube-node-ll44q" Jul 10 18:02:54.179051 ip-10-0-128-39 kubenswrapper[2558]: I0710 18:02:54.170966 2558 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"host-var-lib-cni-networks-ovn-kubernetes\" (UniqueName: \"kubernetes.io/host-path/eabc6c69-b1e8-40d5-b41f-f49778a5248c-host-var-lib-cni-networks-ovn-kubernetes\") pod \"ovnkube-node-ll44q\" (UID: \"eabc6c69-b1e8-40d5-b41f-f49778a5248c\") " pod="openshift-ovn-kubernetes/ovnkube-node-ll44q" Jul 10 18:02:54.179051 ip-10-0-128-39 kubenswrapper[2558]: I0710 18:02:54.170973 2558 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-cni-netd\" (UniqueName: \"kubernetes.io/host-path/eabc6c69-b1e8-40d5-b41f-f49778a5248c-host-cni-netd\") pod \"ovnkube-node-ll44q\" (UID: \"eabc6c69-b1e8-40d5-b41f-f49778a5248c\") " pod="openshift-ovn-kubernetes/ovnkube-node-ll44q" Jul 10 18:02:54.179051 ip-10-0-128-39 kubenswrapper[2558]: I0710 18:02:54.170999 2558 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"env-overrides\" (UniqueName: \"kubernetes.io/configmap/eabc6c69-b1e8-40d5-b41f-f49778a5248c-env-overrides\") pod \"ovnkube-node-ll44q\" (UID: \"eabc6c69-b1e8-40d5-b41f-f49778a5248c\") " pod="openshift-ovn-kubernetes/ovnkube-node-ll44q" Jul 10 18:02:54.179051 ip-10-0-128-39 kubenswrapper[2558]: I0710 18:02:54.171014 2558 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"system-cni-dir\" (UniqueName: \"kubernetes.io/host-path/53492c2f-2433-4658-af05-d58878aed082-system-cni-dir\") pod \"multus-additional-cni-plugins-v22pb\" (UID: \"53492c2f-2433-4658-af05-d58878aed082\") " pod="openshift-multus/multus-additional-cni-plugins-v22pb" Jul 10 18:02:54.179051 ip-10-0-128-39 kubenswrapper[2558]: I0710 18:02:54.171027 2558 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"etc-modprobe-d\" (UniqueName: \"kubernetes.io/host-path/df37e213-59f3-4309-9bec-a41b2238ab98-etc-modprobe-d\") pod \"tuned-kbrll\" (UID: \"df37e213-59f3-4309-9bec-a41b2238ab98\") " pod="openshift-cluster-node-tuning-operator/tuned-kbrll" Jul 10 18:02:54.179051 ip-10-0-128-39 kubenswrapper[2558]: I0710 18:02:54.171051 2558 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host\" (UniqueName: \"kubernetes.io/host-path/df37e213-59f3-4309-9bec-a41b2238ab98-host\") pod \"tuned-kbrll\" (UID: \"df37e213-59f3-4309-9bec-a41b2238ab98\") " pod="openshift-cluster-node-tuning-operator/tuned-kbrll" Jul 10 18:02:54.179051 ip-10-0-128-39 kubenswrapper[2558]: I0710 18:02:54.171078 2558 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"lib-modules\" (UniqueName: \"kubernetes.io/host-path/df37e213-59f3-4309-9bec-a41b2238ab98-lib-modules\") pod \"tuned-kbrll\" (UID: \"df37e213-59f3-4309-9bec-a41b2238ab98\") " pod="openshift-cluster-node-tuning-operator/tuned-kbrll" Jul 10 18:02:54.179051 ip-10-0-128-39 kubenswrapper[2558]: I0710 18:02:54.171113 2558 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-run-netns\" (UniqueName: \"kubernetes.io/host-path/eabc6c69-b1e8-40d5-b41f-f49778a5248c-host-run-netns\") pod \"ovnkube-node-ll44q\" (UID: \"eabc6c69-b1e8-40d5-b41f-f49778a5248c\") " pod="openshift-ovn-kubernetes/ovnkube-node-ll44q" Jul 10 18:02:54.179051 ip-10-0-128-39 kubenswrapper[2558]: I0710 18:02:54.171136 2558 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"ovnkube-script-lib\" (UniqueName: \"kubernetes.io/configmap/eabc6c69-b1e8-40d5-b41f-f49778a5248c-ovnkube-script-lib\") pod \"ovnkube-node-ll44q\" (UID: \"eabc6c69-b1e8-40d5-b41f-f49778a5248c\") " pod="openshift-ovn-kubernetes/ovnkube-node-ll44q" Jul 10 18:02:54.179051 ip-10-0-128-39 kubenswrapper[2558]: I0710 18:02:54.171149 2558 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"run-systemd\" (UniqueName: \"kubernetes.io/host-path/eabc6c69-b1e8-40d5-b41f-f49778a5248c-run-systemd\") pod \"ovnkube-node-ll44q\" (UID: \"eabc6c69-b1e8-40d5-b41f-f49778a5248c\") " pod="openshift-ovn-kubernetes/ovnkube-node-ll44q" Jul 10 18:02:54.180121 ip-10-0-128-39 kubenswrapper[2558]: I0710 18:02:54.171157 2558 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-slash\" (UniqueName: \"kubernetes.io/host-path/eabc6c69-b1e8-40d5-b41f-f49778a5248c-host-slash\") pod \"ovnkube-node-ll44q\" (UID: \"eabc6c69-b1e8-40d5-b41f-f49778a5248c\") " pod="openshift-ovn-kubernetes/ovnkube-node-ll44q" Jul 10 18:02:54.180121 ip-10-0-128-39 kubenswrapper[2558]: I0710 18:02:54.171174 2558 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"etc-sysctl-d\" (UniqueName: \"kubernetes.io/host-path/df37e213-59f3-4309-9bec-a41b2238ab98-etc-sysctl-d\") pod \"tuned-kbrll\" (UID: \"df37e213-59f3-4309-9bec-a41b2238ab98\") " pod="openshift-cluster-node-tuning-operator/tuned-kbrll" Jul 10 18:02:54.180121 ip-10-0-128-39 kubenswrapper[2558]: I0710 18:02:54.171191 2558 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"host-slash\" (UniqueName: \"kubernetes.io/host-path/eabc6c69-b1e8-40d5-b41f-f49778a5248c-host-slash\") pod \"ovnkube-node-ll44q\" (UID: \"eabc6c69-b1e8-40d5-b41f-f49778a5248c\") " pod="openshift-ovn-kubernetes/ovnkube-node-ll44q" Jul 10 18:02:54.180121 ip-10-0-128-39 kubenswrapper[2558]: I0710 18:02:54.171204 2558 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"cni-binary-copy\" (UniqueName: \"kubernetes.io/configmap/53492c2f-2433-4658-af05-d58878aed082-cni-binary-copy\") pod \"multus-additional-cni-plugins-v22pb\" (UID: \"53492c2f-2433-4658-af05-d58878aed082\") " pod="openshift-multus/multus-additional-cni-plugins-v22pb" Jul 10 18:02:54.180121 ip-10-0-128-39 kubenswrapper[2558]: I0710 18:02:54.171231 2558 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tuning-conf-dir\" (UniqueName: \"kubernetes.io/host-path/53492c2f-2433-4658-af05-d58878aed082-tuning-conf-dir\") pod \"multus-additional-cni-plugins-v22pb\" (UID: \"53492c2f-2433-4658-af05-d58878aed082\") " pod="openshift-multus/multus-additional-cni-plugins-v22pb" Jul 10 18:02:54.180121 ip-10-0-128-39 kubenswrapper[2558]: I0710 18:02:54.171242 2558 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"run\" (UniqueName: \"kubernetes.io/host-path/df37e213-59f3-4309-9bec-a41b2238ab98-run\") pod \"tuned-kbrll\" (UID: \"df37e213-59f3-4309-9bec-a41b2238ab98\") " pod="openshift-cluster-node-tuning-operator/tuned-kbrll" Jul 10 18:02:54.180121 ip-10-0-128-39 kubenswrapper[2558]: I0710 18:02:54.171267 2558 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"konnectivity-ca\" (UniqueName: \"kubernetes.io/configmap/2da19865-b021-4789-b1e0-694d4f6c543d-konnectivity-ca\") pod \"konnectivity-agent-dvp86\" (UID: \"2da19865-b021-4789-b1e0-694d4f6c543d\") " pod="kube-system/konnectivity-agent-dvp86" Jul 10 18:02:54.180121 ip-10-0-128-39 kubenswrapper[2558]: I0710 18:02:54.171292 2558 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-cni-bin\" (UniqueName: \"kubernetes.io/host-path/eabc6c69-b1e8-40d5-b41f-f49778a5248c-host-cni-bin\") pod \"ovnkube-node-ll44q\" (UID: \"eabc6c69-b1e8-40d5-b41f-f49778a5248c\") " pod="openshift-ovn-kubernetes/ovnkube-node-ll44q" Jul 10 18:02:54.181876 ip-10-0-128-39 kubenswrapper[2558]: I0710 18:02:54.181658 2558 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"node-log\" (UniqueName: \"kubernetes.io/host-path/eabc6c69-b1e8-40d5-b41f-f49778a5248c-node-log\") pod \"ovnkube-node-ll44q\" (UID: \"eabc6c69-b1e8-40d5-b41f-f49778a5248c\") " pod="openshift-ovn-kubernetes/ovnkube-node-ll44q" Jul 10 18:02:54.181876 ip-10-0-128-39 kubenswrapper[2558]: I0710 18:02:54.181712 2558 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-run-ovn-kubernetes\" (UniqueName: \"kubernetes.io/host-path/eabc6c69-b1e8-40d5-b41f-f49778a5248c-host-run-ovn-kubernetes\") pod \"ovnkube-node-ll44q\" (UID: \"eabc6c69-b1e8-40d5-b41f-f49778a5248c\") " pod="openshift-ovn-kubernetes/ovnkube-node-ll44q" Jul 10 18:02:54.181876 ip-10-0-128-39 kubenswrapper[2558]: I0710 18:02:54.181794 2558 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"host-run-ovn-kubernetes\" (UniqueName: \"kubernetes.io/host-path/eabc6c69-b1e8-40d5-b41f-f49778a5248c-host-run-ovn-kubernetes\") pod \"ovnkube-node-ll44q\" (UID: \"eabc6c69-b1e8-40d5-b41f-f49778a5248c\") " pod="openshift-ovn-kubernetes/ovnkube-node-ll44q" Jul 10 18:02:54.181876 ip-10-0-128-39 kubenswrapper[2558]: I0710 18:02:54.181873 2558 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"host-cni-netd\" (UniqueName: \"kubernetes.io/host-path/eabc6c69-b1e8-40d5-b41f-f49778a5248c-host-cni-netd\") pod \"ovnkube-node-ll44q\" (UID: \"eabc6c69-b1e8-40d5-b41f-f49778a5248c\") " pod="openshift-ovn-kubernetes/ovnkube-node-ll44q" Jul 10 18:02:54.182379 ip-10-0-128-39 kubenswrapper[2558]: I0710 18:02:54.182161 2558 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"tuning-conf-dir\" (UniqueName: \"kubernetes.io/host-path/53492c2f-2433-4658-af05-d58878aed082-tuning-conf-dir\") pod \"multus-additional-cni-plugins-v22pb\" (UID: \"53492c2f-2433-4658-af05-d58878aed082\") " pod="openshift-multus/multus-additional-cni-plugins-v22pb" Jul 10 18:02:54.182647 ip-10-0-128-39 kubenswrapper[2558]: I0710 18:02:54.182485 2558 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"ovnkube-config\" (UniqueName: \"kubernetes.io/configmap/eabc6c69-b1e8-40d5-b41f-f49778a5248c-ovnkube-config\") pod \"ovnkube-node-ll44q\" (UID: \"eabc6c69-b1e8-40d5-b41f-f49778a5248c\") " pod="openshift-ovn-kubernetes/ovnkube-node-ll44q" Jul 10 18:02:54.182888 ip-10-0-128-39 kubenswrapper[2558]: I0710 18:02:54.182691 2558 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"os-release\" (UniqueName: \"kubernetes.io/host-path/53492c2f-2433-4658-af05-d58878aed082-os-release\") pod \"multus-additional-cni-plugins-v22pb\" (UID: \"53492c2f-2433-4658-af05-d58878aed082\") " pod="openshift-multus/multus-additional-cni-plugins-v22pb" Jul 10 18:02:54.182888 ip-10-0-128-39 kubenswrapper[2558]: I0710 18:02:54.171052 2558 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"cnibin\" (UniqueName: \"kubernetes.io/host-path/53492c2f-2433-4658-af05-d58878aed082-cnibin\") pod \"multus-additional-cni-plugins-v22pb\" (UID: \"53492c2f-2433-4658-af05-d58878aed082\") " pod="openshift-multus/multus-additional-cni-plugins-v22pb" Jul 10 18:02:54.265611 ip-10-0-128-39 kubenswrapper[2558]: I0710 18:02:54.184160 2558 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"systemd-units\" (UniqueName: \"kubernetes.io/host-path/eabc6c69-b1e8-40d5-b41f-f49778a5248c-systemd-units\") pod \"ovnkube-node-ll44q\" (UID: \"eabc6c69-b1e8-40d5-b41f-f49778a5248c\") " pod="openshift-ovn-kubernetes/ovnkube-node-ll44q" Jul 10 18:02:54.265611 ip-10-0-128-39 kubenswrapper[2558]: I0710 18:02:54.184240 2558 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"host-kubelet\" (UniqueName: \"kubernetes.io/host-path/eabc6c69-b1e8-40d5-b41f-f49778a5248c-host-kubelet\") pod \"ovnkube-node-ll44q\" (UID: \"eabc6c69-b1e8-40d5-b41f-f49778a5248c\") " pod="openshift-ovn-kubernetes/ovnkube-node-ll44q" Jul 10 18:02:54.265611 ip-10-0-128-39 kubenswrapper[2558]: I0710 18:02:54.184416 2558 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"var-lib-openvswitch\" (UniqueName: \"kubernetes.io/host-path/eabc6c69-b1e8-40d5-b41f-f49778a5248c-var-lib-openvswitch\") pod \"ovnkube-node-ll44q\" (UID: \"eabc6c69-b1e8-40d5-b41f-f49778a5248c\") " pod="openshift-ovn-kubernetes/ovnkube-node-ll44q" Jul 10 18:02:54.265611 ip-10-0-128-39 kubenswrapper[2558]: I0710 18:02:54.184477 2558 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"etc-openvswitch\" (UniqueName: \"kubernetes.io/host-path/eabc6c69-b1e8-40d5-b41f-f49778a5248c-etc-openvswitch\") pod \"ovnkube-node-ll44q\" (UID: \"eabc6c69-b1e8-40d5-b41f-f49778a5248c\") " pod="openshift-ovn-kubernetes/ovnkube-node-ll44q" Jul 10 18:02:54.265611 ip-10-0-128-39 kubenswrapper[2558]: I0710 18:02:54.184553 2558 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"env-overrides\" (UniqueName: \"kubernetes.io/configmap/eabc6c69-b1e8-40d5-b41f-f49778a5248c-env-overrides\") pod \"ovnkube-node-ll44q\" (UID: \"eabc6c69-b1e8-40d5-b41f-f49778a5248c\") " pod="openshift-ovn-kubernetes/ovnkube-node-ll44q" Jul 10 18:02:54.265611 ip-10-0-128-39 kubenswrapper[2558]: I0710 18:02:54.184608 2558 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"run-openvswitch\" (UniqueName: \"kubernetes.io/host-path/eabc6c69-b1e8-40d5-b41f-f49778a5248c-run-openvswitch\") pod \"ovnkube-node-ll44q\" (UID: \"eabc6c69-b1e8-40d5-b41f-f49778a5248c\") " pod="openshift-ovn-kubernetes/ovnkube-node-ll44q" Jul 10 18:02:54.265611 ip-10-0-128-39 kubenswrapper[2558]: I0710 18:02:54.184671 2558 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"etc-modprobe-d\" (UniqueName: \"kubernetes.io/host-path/df37e213-59f3-4309-9bec-a41b2238ab98-etc-modprobe-d\") pod \"tuned-kbrll\" (UID: \"df37e213-59f3-4309-9bec-a41b2238ab98\") " pod="openshift-cluster-node-tuning-operator/tuned-kbrll" Jul 10 18:02:54.265611 ip-10-0-128-39 kubenswrapper[2558]: I0710 18:02:54.184717 2558 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"host\" (UniqueName: \"kubernetes.io/host-path/df37e213-59f3-4309-9bec-a41b2238ab98-host\") pod \"tuned-kbrll\" (UID: \"df37e213-59f3-4309-9bec-a41b2238ab98\") " pod="openshift-cluster-node-tuning-operator/tuned-kbrll" Jul 10 18:02:54.265611 ip-10-0-128-39 kubenswrapper[2558]: I0710 18:02:54.184825 2558 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"node-log\" (UniqueName: \"kubernetes.io/host-path/eabc6c69-b1e8-40d5-b41f-f49778a5248c-node-log\") pod \"ovnkube-node-ll44q\" (UID: \"eabc6c69-b1e8-40d5-b41f-f49778a5248c\") " pod="openshift-ovn-kubernetes/ovnkube-node-ll44q" Jul 10 18:02:54.265611 ip-10-0-128-39 kubenswrapper[2558]: I0710 18:02:54.184967 2558 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"lib-modules\" (UniqueName: \"kubernetes.io/host-path/df37e213-59f3-4309-9bec-a41b2238ab98-lib-modules\") pod \"tuned-kbrll\" (UID: \"df37e213-59f3-4309-9bec-a41b2238ab98\") " pod="openshift-cluster-node-tuning-operator/tuned-kbrll" Jul 10 18:02:54.265611 ip-10-0-128-39 kubenswrapper[2558]: I0710 18:02:54.185098 2558 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"host-run-netns\" (UniqueName: \"kubernetes.io/host-path/eabc6c69-b1e8-40d5-b41f-f49778a5248c-host-run-netns\") pod \"ovnkube-node-ll44q\" (UID: \"eabc6c69-b1e8-40d5-b41f-f49778a5248c\") " pod="openshift-ovn-kubernetes/ovnkube-node-ll44q" Jul 10 18:02:54.265611 ip-10-0-128-39 kubenswrapper[2558]: I0710 18:02:54.185106 2558 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"agent-certs\" (UniqueName: \"kubernetes.io/secret/2da19865-b021-4789-b1e0-694d4f6c543d-agent-certs\") pod \"konnectivity-agent-dvp86\" (UID: \"2da19865-b021-4789-b1e0-694d4f6c543d\") " pod="kube-system/konnectivity-agent-dvp86" Jul 10 18:02:54.265611 ip-10-0-128-39 kubenswrapper[2558]: I0710 18:02:54.185328 2558 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"var-lib-kubelet\" (UniqueName: \"kubernetes.io/host-path/df37e213-59f3-4309-9bec-a41b2238ab98-var-lib-kubelet\") pod \"tuned-kbrll\" (UID: \"df37e213-59f3-4309-9bec-a41b2238ab98\") " pod="openshift-cluster-node-tuning-operator/tuned-kbrll" Jul 10 18:02:54.265611 ip-10-0-128-39 kubenswrapper[2558]: I0710 18:02:54.185678 2558 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"ovnkube-script-lib\" (UniqueName: \"kubernetes.io/configmap/eabc6c69-b1e8-40d5-b41f-f49778a5248c-ovnkube-script-lib\") pod \"ovnkube-node-ll44q\" (UID: \"eabc6c69-b1e8-40d5-b41f-f49778a5248c\") " pod="openshift-ovn-kubernetes/ovnkube-node-ll44q" Jul 10 18:02:54.265611 ip-10-0-128-39 kubenswrapper[2558]: I0710 18:02:54.186350 2558 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"etc-kubernetes\" (UniqueName: \"kubernetes.io/host-path/df37e213-59f3-4309-9bec-a41b2238ab98-etc-kubernetes\") pod \"tuned-kbrll\" (UID: \"df37e213-59f3-4309-9bec-a41b2238ab98\") " pod="openshift-cluster-node-tuning-operator/tuned-kbrll" Jul 10 18:02:54.267587 ip-10-0-128-39 kubenswrapper[2558]: I0710 18:02:54.186885 2558 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"whereabouts-flatfile-configmap\" (UniqueName: \"kubernetes.io/configmap/53492c2f-2433-4658-af05-d58878aed082-whereabouts-flatfile-configmap\") pod \"multus-additional-cni-plugins-v22pb\" (UID: \"53492c2f-2433-4658-af05-d58878aed082\") " pod="openshift-multus/multus-additional-cni-plugins-v22pb" Jul 10 18:02:54.267587 ip-10-0-128-39 kubenswrapper[2558]: I0710 18:02:54.186967 2558 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"etc-systemd\" (UniqueName: \"kubernetes.io/host-path/df37e213-59f3-4309-9bec-a41b2238ab98-etc-systemd\") pod \"tuned-kbrll\" (UID: \"df37e213-59f3-4309-9bec-a41b2238ab98\") " pod="openshift-cluster-node-tuning-operator/tuned-kbrll" Jul 10 18:02:54.267587 ip-10-0-128-39 kubenswrapper[2558]: I0710 18:02:54.187339 2558 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"host-cni-bin\" (UniqueName: \"kubernetes.io/host-path/eabc6c69-b1e8-40d5-b41f-f49778a5248c-host-cni-bin\") pod \"ovnkube-node-ll44q\" (UID: \"eabc6c69-b1e8-40d5-b41f-f49778a5248c\") " pod="openshift-ovn-kubernetes/ovnkube-node-ll44q" Jul 10 18:02:54.267587 ip-10-0-128-39 kubenswrapper[2558]: I0710 18:02:54.187416 2558 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"konnectivity-ca\" (UniqueName: \"kubernetes.io/configmap/2da19865-b021-4789-b1e0-694d4f6c543d-konnectivity-ca\") pod \"konnectivity-agent-dvp86\" (UID: \"2da19865-b021-4789-b1e0-694d4f6c543d\") " pod="kube-system/konnectivity-agent-dvp86" Jul 10 18:02:54.267587 ip-10-0-128-39 kubenswrapper[2558]: I0710 18:02:54.187621 2558 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"etc-sysctl-conf\" (UniqueName: \"kubernetes.io/host-path/df37e213-59f3-4309-9bec-a41b2238ab98-etc-sysctl-conf\") pod \"tuned-kbrll\" (UID: \"df37e213-59f3-4309-9bec-a41b2238ab98\") " pod="openshift-cluster-node-tuning-operator/tuned-kbrll" Jul 10 18:02:54.267587 ip-10-0-128-39 kubenswrapper[2558]: I0710 18:02:54.188230 2558 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"cni-binary-copy\" (UniqueName: \"kubernetes.io/configmap/53492c2f-2433-4658-af05-d58878aed082-cni-binary-copy\") pod \"multus-additional-cni-plugins-v22pb\" (UID: \"53492c2f-2433-4658-af05-d58878aed082\") " pod="openshift-multus/multus-additional-cni-plugins-v22pb" Jul 10 18:02:54.267587 ip-10-0-128-39 kubenswrapper[2558]: I0710 18:02:54.189000 2558 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"tmp\" (UniqueName: \"kubernetes.io/empty-dir/df37e213-59f3-4309-9bec-a41b2238ab98-tmp\") pod \"tuned-kbrll\" (UID: \"df37e213-59f3-4309-9bec-a41b2238ab98\") " pod="openshift-cluster-node-tuning-operator/tuned-kbrll" Jul 10 18:02:54.267587 ip-10-0-128-39 kubenswrapper[2558]: I0710 18:02:54.190120 2558 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"ovn-node-metrics-cert\" (UniqueName: \"kubernetes.io/secret/eabc6c69-b1e8-40d5-b41f-f49778a5248c-ovn-node-metrics-cert\") pod \"ovnkube-node-ll44q\" (UID: \"eabc6c69-b1e8-40d5-b41f-f49778a5248c\") " pod="openshift-ovn-kubernetes/ovnkube-node-ll44q" Jul 10 18:02:54.267587 ip-10-0-128-39 kubenswrapper[2558]: I0710 18:02:54.190294 2558 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"etc-tuned\" (UniqueName: \"kubernetes.io/empty-dir/df37e213-59f3-4309-9bec-a41b2238ab98-etc-tuned\") pod \"tuned-kbrll\" (UID: \"df37e213-59f3-4309-9bec-a41b2238ab98\") " pod="openshift-cluster-node-tuning-operator/tuned-kbrll" Jul 10 18:02:54.267587 ip-10-0-128-39 kubenswrapper[2558]: I0710 18:02:54.190454 2558 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-t7npn\" (UniqueName: \"kubernetes.io/projected/df37e213-59f3-4309-9bec-a41b2238ab98-kube-api-access-t7npn\") pod \"tuned-kbrll\" (UID: \"df37e213-59f3-4309-9bec-a41b2238ab98\") " pod="openshift-cluster-node-tuning-operator/tuned-kbrll" Jul 10 18:02:54.267587 ip-10-0-128-39 kubenswrapper[2558]: I0710 18:02:54.192365 2558 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-znqb9\" (UniqueName: \"kubernetes.io/projected/53492c2f-2433-4658-af05-d58878aed082-kube-api-access-znqb9\") pod \"multus-additional-cni-plugins-v22pb\" (UID: \"53492c2f-2433-4658-af05-d58878aed082\") " pod="openshift-multus/multus-additional-cni-plugins-v22pb" Jul 10 18:02:54.267587 ip-10-0-128-39 kubenswrapper[2558]: I0710 18:02:54.192764 2558 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-275xj\" (UniqueName: \"kubernetes.io/projected/eabc6c69-b1e8-40d5-b41f-f49778a5248c-kube-api-access-275xj\") pod \"ovnkube-node-ll44q\" (UID: \"eabc6c69-b1e8-40d5-b41f-f49778a5248c\") " pod="openshift-ovn-kubernetes/ovnkube-node-ll44q" Jul 10 18:02:54.267587 ip-10-0-128-39 kubenswrapper[2558]: I0710 18:02:54.236089 2558 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-operator/iptables-alerter-zmg24" Jul 10 18:02:54.267587 ip-10-0-128-39 kubenswrapper[2558]: I0710 18:02:54.243526 2558 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-dns/node-resolver-jqb25" Jul 10 18:02:54.267587 ip-10-0-128-39 kubenswrapper[2558]: I0710 18:02:54.253789 2558 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-image-registry/node-ca-r7qh2" Jul 10 18:02:54.267587 ip-10-0-128-39 kubenswrapper[2558]: W0710 18:02:54.254168 2558 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod1b18a733_4a78_4cf0_9377_3a26c9ae4a2f.slice/crio-1ea2b41dff24c7d3966928cbadec693751b60d1d56e884199d7efc9020535b70 WatchSource:0}: Error finding container 1ea2b41dff24c7d3966928cbadec693751b60d1d56e884199d7efc9020535b70: Status 404 returned error can't find the container with id 1ea2b41dff24c7d3966928cbadec693751b60d1d56e884199d7efc9020535b70 Jul 10 18:02:54.268633 ip-10-0-128-39 kubenswrapper[2558]: W0710 18:02:54.254591 2558 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod12ad9a8e_4768_4362_818f_d2fb73c8a236.slice/crio-82ab7bb984eef5784a38965415cec66127c12463052a396a4ec4abfbd4725678 WatchSource:0}: Error finding container 82ab7bb984eef5784a38965415cec66127c12463052a396a4ec4abfbd4725678: Status 404 returned error can't find the container with id 82ab7bb984eef5784a38965415cec66127c12463052a396a4ec4abfbd4725678 Jul 10 18:02:54.268633 ip-10-0-128-39 kubenswrapper[2558]: I0710 18:02:54.257675 2558 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/multus-kvqn7" Jul 10 18:02:54.268633 ip-10-0-128-39 kubenswrapper[2558]: I0710 18:02:54.265904 2558 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-hps6m" Jul 10 18:02:54.276741 ip-10-0-128-39 kubenswrapper[2558]: I0710 18:02:54.276362 2558 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/konnectivity-agent-dvp86" Jul 10 18:02:54.360716 ip-10-0-128-39 kubenswrapper[2558]: W0710 18:02:54.281520 2558 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podfe05c85a_2eb8_429b_9776_a3408a38903f.slice/crio-a0ba9d567f0514ea371328155bcf864107d116dc8db0681b68bc20536e9c7a53 WatchSource:0}: Error finding container a0ba9d567f0514ea371328155bcf864107d116dc8db0681b68bc20536e9c7a53: Status 404 returned error can't find the container with id a0ba9d567f0514ea371328155bcf864107d116dc8db0681b68bc20536e9c7a53 Jul 10 18:02:54.360716 ip-10-0-128-39 kubenswrapper[2558]: I0710 18:02:54.283846 2558 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-cluster-node-tuning-operator/tuned-kbrll" Jul 10 18:02:54.360716 ip-10-0-128-39 kubenswrapper[2558]: W0710 18:02:54.291108 2558 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod11be9079_5d81_46c9_8f4c_c2e305bf2144.slice/crio-c8930a8d040aecc1f89a663713fcfc1c1b5681264fcc62ac5ca06ad63168ac11 WatchSource:0}: Error finding container c8930a8d040aecc1f89a663713fcfc1c1b5681264fcc62ac5ca06ad63168ac11: Status 404 returned error can't find the container with id c8930a8d040aecc1f89a663713fcfc1c1b5681264fcc62ac5ca06ad63168ac11 Jul 10 18:02:54.360716 ip-10-0-128-39 kubenswrapper[2558]: I0710 18:02:54.293306 2558 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/multus-additional-cni-plugins-v22pb" Jul 10 18:02:54.360716 ip-10-0-128-39 kubenswrapper[2558]: I0710 18:02:54.318403 2558 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ovn-kubernetes/ovnkube-node-ll44q" Jul 10 18:02:54.360716 ip-10-0-128-39 kubenswrapper[2558]: W0710 18:02:54.323552 2558 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-poddf37e213_59f3_4309_9bec_a41b2238ab98.slice/crio-9a5f390faa9eea17e412cd15e1c1821d8c03dc5f6abd77a221807068be8805a9 WatchSource:0}: Error finding container 9a5f390faa9eea17e412cd15e1c1821d8c03dc5f6abd77a221807068be8805a9: Status 404 returned error can't find the container with id 9a5f390faa9eea17e412cd15e1c1821d8c03dc5f6abd77a221807068be8805a9 Jul 10 18:02:54.360716 ip-10-0-128-39 kubenswrapper[2558]: W0710 18:02:54.325917 2558 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod2da19865_b021_4789_b1e0_694d4f6c543d.slice/crio-d3148b1dd08f6ee7f40fc658fcb8e64cc68c323edcc06dc3cfa300705c9e5080 WatchSource:0}: Error finding container d3148b1dd08f6ee7f40fc658fcb8e64cc68c323edcc06dc3cfa300705c9e5080: Status 404 returned error can't find the container with id d3148b1dd08f6ee7f40fc658fcb8e64cc68c323edcc06dc3cfa300705c9e5080 Jul 10 18:02:54.360716 ip-10-0-128-39 kubenswrapper[2558]: W0710 18:02:54.331369 2558 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod53492c2f_2433_4658_af05_d58878aed082.slice/crio-4c52c01b68160ca3672fd7472c9019b6840db46e5ac6cfb575c6d403df99c82a WatchSource:0}: Error finding container 4c52c01b68160ca3672fd7472c9019b6840db46e5ac6cfb575c6d403df99c82a: Status 404 returned error can't find the container with id 4c52c01b68160ca3672fd7472c9019b6840db46e5ac6cfb575c6d403df99c82a Jul 10 18:02:54.711620 ip-10-0-128-39 kubenswrapper[2558]: I0710 18:02:54.596369 2558 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-h74dt\" (UniqueName: \"kubernetes.io/projected/06aa905d-9b4b-4d36-bafc-8fa0c11b3057-kube-api-access-h74dt\") pod \"network-check-target-rzfgn\" (UID: \"06aa905d-9b4b-4d36-bafc-8fa0c11b3057\") " pod="openshift-network-diagnostics/network-check-target-rzfgn" Jul 10 18:02:54.711620 ip-10-0-128-39 kubenswrapper[2558]: I0710 18:02:54.596421 2558 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"original-pull-secret\" (UniqueName: \"kubernetes.io/secret/3b27429f-90aa-4170-9078-0aa4ea0d0ab8-original-pull-secret\") pod \"global-pull-secret-syncer-vsbdm\" (UID: \"3b27429f-90aa-4170-9078-0aa4ea0d0ab8\") " pod="kube-system/global-pull-secret-syncer-vsbdm" Jul 10 18:02:54.711620 ip-10-0-128-39 kubenswrapper[2558]: E0710 18:02:54.596508 2558 secret.go:189] Couldn't get secret kube-system/original-pull-secret: object "kube-system"/"original-pull-secret" not registered Jul 10 18:02:54.711620 ip-10-0-128-39 kubenswrapper[2558]: E0710 18:02:54.596544 2558 projected.go:291] Couldn't get configMap openshift-network-diagnostics/kube-root-ca.crt: object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered Jul 10 18:02:54.711620 ip-10-0-128-39 kubenswrapper[2558]: E0710 18:02:54.596566 2558 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/3b27429f-90aa-4170-9078-0aa4ea0d0ab8-original-pull-secret podName:3b27429f-90aa-4170-9078-0aa4ea0d0ab8 nodeName:}" failed. No retries permitted until 2026-07-10 18:02:55.596547624 +0000 UTC m=+4.100040108 (durationBeforeRetry 1s). Error: MountVolume.SetUp failed for volume "original-pull-secret" (UniqueName: "kubernetes.io/secret/3b27429f-90aa-4170-9078-0aa4ea0d0ab8-original-pull-secret") pod "global-pull-secret-syncer-vsbdm" (UID: "3b27429f-90aa-4170-9078-0aa4ea0d0ab8") : object "kube-system"/"original-pull-secret" not registered Jul 10 18:02:54.711620 ip-10-0-128-39 kubenswrapper[2558]: I0710 18:02:54.596561 2558 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/7fa20252-06b7-48e4-8620-3e52dc980486-metrics-certs\") pod \"network-metrics-daemon-jkr77\" (UID: \"7fa20252-06b7-48e4-8620-3e52dc980486\") " pod="openshift-multus/network-metrics-daemon-jkr77" Jul 10 18:02:54.711620 ip-10-0-128-39 kubenswrapper[2558]: E0710 18:02:54.596574 2558 projected.go:291] Couldn't get configMap openshift-network-diagnostics/openshift-service-ca.crt: object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered Jul 10 18:02:54.711620 ip-10-0-128-39 kubenswrapper[2558]: E0710 18:02:54.596621 2558 secret.go:189] Couldn't get secret openshift-multus/metrics-daemon-secret: object "openshift-multus"/"metrics-daemon-secret" not registered Jul 10 18:02:54.711620 ip-10-0-128-39 kubenswrapper[2558]: E0710 18:02:54.596671 2558 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/7fa20252-06b7-48e4-8620-3e52dc980486-metrics-certs podName:7fa20252-06b7-48e4-8620-3e52dc980486 nodeName:}" failed. No retries permitted until 2026-07-10 18:02:55.596655987 +0000 UTC m=+4.100148481 (durationBeforeRetry 1s). Error: MountVolume.SetUp failed for volume "metrics-certs" (UniqueName: "kubernetes.io/secret/7fa20252-06b7-48e4-8620-3e52dc980486-metrics-certs") pod "network-metrics-daemon-jkr77" (UID: "7fa20252-06b7-48e4-8620-3e52dc980486") : object "openshift-multus"/"metrics-daemon-secret" not registered Jul 10 18:02:54.711620 ip-10-0-128-39 kubenswrapper[2558]: E0710 18:02:54.596623 2558 projected.go:196] Error preparing data for projected volume kube-api-access-h74dt for pod openshift-network-diagnostics/network-check-target-rzfgn: [object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered, object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered] Jul 10 18:02:54.711620 ip-10-0-128-39 kubenswrapper[2558]: E0710 18:02:54.596743 2558 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/projected/06aa905d-9b4b-4d36-bafc-8fa0c11b3057-kube-api-access-h74dt podName:06aa905d-9b4b-4d36-bafc-8fa0c11b3057 nodeName:}" failed. No retries permitted until 2026-07-10 18:02:55.596735884 +0000 UTC m=+4.100228368 (durationBeforeRetry 1s). Error: MountVolume.SetUp failed for volume "kube-api-access-h74dt" (UniqueName: "kubernetes.io/projected/06aa905d-9b4b-4d36-bafc-8fa0c11b3057-kube-api-access-h74dt") pod "network-check-target-rzfgn" (UID: "06aa905d-9b4b-4d36-bafc-8fa0c11b3057") : [object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered, object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered] Jul 10 18:02:54.711620 ip-10-0-128-39 kubenswrapper[2558]: I0710 18:02:54.630495 2558 reflector.go:439] "Caches populated" type="*v1.Service" reflector="k8s.io/client-go/informers/factory.go:160" Jul 10 18:02:54.980250 ip-10-0-128-39 kubenswrapper[2558]: I0710 18:02:54.980152 2558 certificate_manager.go:715] "Certificate rotation deadline determined" logger="kubernetes.io/kubelet-serving" expiration="2028-07-09 17:57:53 +0000 UTC" deadline="2028-03-31 13:32:52.001613128 +0000 UTC" Jul 10 18:02:54.980250 ip-10-0-128-39 kubenswrapper[2558]: I0710 18:02:54.980229 2558 certificate_manager.go:431] "Waiting for next certificate rotation" logger="kubernetes.io/kubelet-serving" sleep="15115h29m57.021414729s" Jul 10 18:02:55.095031 ip-10-0-128-39 kubenswrapper[2558]: I0710 18:02:55.095001 2558 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-vsbdm" Jul 10 18:02:55.095185 ip-10-0-128-39 kubenswrapper[2558]: E0710 18:02:55.095110 2558 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-vsbdm" podUID="3b27429f-90aa-4170-9078-0aa4ea0d0ab8" Jul 10 18:02:55.106785 ip-10-0-128-39 kubenswrapper[2558]: I0710 18:02:55.106750 2558 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ovn-kubernetes/ovnkube-node-ll44q" event={"ID":"eabc6c69-b1e8-40d5-b41f-f49778a5248c","Type":"ContainerStarted","Data":"9b3e00092047116ad5a8572117627607ac6978a6457117c37ee257420d2d25c2"} Jul 10 18:02:55.107871 ip-10-0-128-39 kubenswrapper[2558]: I0710 18:02:55.107839 2558 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-multus/multus-additional-cni-plugins-v22pb" event={"ID":"53492c2f-2433-4658-af05-d58878aed082","Type":"ContainerStarted","Data":"4c52c01b68160ca3672fd7472c9019b6840db46e5ac6cfb575c6d403df99c82a"} Jul 10 18:02:55.109132 ip-10-0-128-39 kubenswrapper[2558]: I0710 18:02:55.109100 2558 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-cluster-node-tuning-operator/tuned-kbrll" event={"ID":"df37e213-59f3-4309-9bec-a41b2238ab98","Type":"ContainerStarted","Data":"9a5f390faa9eea17e412cd15e1c1821d8c03dc5f6abd77a221807068be8805a9"} Jul 10 18:02:55.110412 ip-10-0-128-39 kubenswrapper[2558]: I0710 18:02:55.110352 2558 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-image-registry/node-ca-r7qh2" event={"ID":"fe05c85a-2eb8-429b-9776-a3408a38903f","Type":"ContainerStarted","Data":"a0ba9d567f0514ea371328155bcf864107d116dc8db0681b68bc20536e9c7a53"} Jul 10 18:02:55.111485 ip-10-0-128-39 kubenswrapper[2558]: I0710 18:02:55.111455 2558 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-dns/node-resolver-jqb25" event={"ID":"12ad9a8e-4768-4362-818f-d2fb73c8a236","Type":"ContainerStarted","Data":"82ab7bb984eef5784a38965415cec66127c12463052a396a4ec4abfbd4725678"} Jul 10 18:02:55.112503 ip-10-0-128-39 kubenswrapper[2558]: I0710 18:02:55.112477 2558 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kube-system/konnectivity-agent-dvp86" event={"ID":"2da19865-b021-4789-b1e0-694d4f6c543d","Type":"ContainerStarted","Data":"d3148b1dd08f6ee7f40fc658fcb8e64cc68c323edcc06dc3cfa300705c9e5080"} Jul 10 18:02:55.113550 ip-10-0-128-39 kubenswrapper[2558]: I0710 18:02:55.113532 2558 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-hps6m" event={"ID":"cdee596a-a663-4627-8c46-d7066d21978b","Type":"ContainerStarted","Data":"69abc14bd0b18ff75febc232da12a18ab37a68c0ba9ba67b0d464e79ee33b763"} Jul 10 18:02:55.114696 ip-10-0-128-39 kubenswrapper[2558]: I0710 18:02:55.114646 2558 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-multus/multus-kvqn7" event={"ID":"11be9079-5d81-46c9-8f4c-c2e305bf2144","Type":"ContainerStarted","Data":"c8930a8d040aecc1f89a663713fcfc1c1b5681264fcc62ac5ca06ad63168ac11"} Jul 10 18:02:55.116254 ip-10-0-128-39 kubenswrapper[2558]: I0710 18:02:55.116221 2558 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-network-operator/iptables-alerter-zmg24" event={"ID":"1b18a733-4a78-4cf0-9377-3a26c9ae4a2f","Type":"ContainerStarted","Data":"1ea2b41dff24c7d3966928cbadec693751b60d1d56e884199d7efc9020535b70"} Jul 10 18:02:55.605827 ip-10-0-128-39 kubenswrapper[2558]: I0710 18:02:55.605729 2558 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-h74dt\" (UniqueName: \"kubernetes.io/projected/06aa905d-9b4b-4d36-bafc-8fa0c11b3057-kube-api-access-h74dt\") pod \"network-check-target-rzfgn\" (UID: \"06aa905d-9b4b-4d36-bafc-8fa0c11b3057\") " pod="openshift-network-diagnostics/network-check-target-rzfgn" Jul 10 18:02:55.605992 ip-10-0-128-39 kubenswrapper[2558]: E0710 18:02:55.605967 2558 projected.go:291] Couldn't get configMap openshift-network-diagnostics/kube-root-ca.crt: object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered Jul 10 18:02:55.605992 ip-10-0-128-39 kubenswrapper[2558]: E0710 18:02:55.605993 2558 projected.go:291] Couldn't get configMap openshift-network-diagnostics/openshift-service-ca.crt: object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered Jul 10 18:02:55.606107 ip-10-0-128-39 kubenswrapper[2558]: E0710 18:02:55.606005 2558 projected.go:196] Error preparing data for projected volume kube-api-access-h74dt for pod openshift-network-diagnostics/network-check-target-rzfgn: [object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered, object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered] Jul 10 18:02:55.606107 ip-10-0-128-39 kubenswrapper[2558]: E0710 18:02:55.606014 2558 secret.go:189] Couldn't get secret kube-system/original-pull-secret: object "kube-system"/"original-pull-secret" not registered Jul 10 18:02:55.606107 ip-10-0-128-39 kubenswrapper[2558]: E0710 18:02:55.606070 2558 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/projected/06aa905d-9b4b-4d36-bafc-8fa0c11b3057-kube-api-access-h74dt podName:06aa905d-9b4b-4d36-bafc-8fa0c11b3057 nodeName:}" failed. No retries permitted until 2026-07-10 18:02:57.606051574 +0000 UTC m=+6.109544081 (durationBeforeRetry 2s). Error: MountVolume.SetUp failed for volume "kube-api-access-h74dt" (UniqueName: "kubernetes.io/projected/06aa905d-9b4b-4d36-bafc-8fa0c11b3057-kube-api-access-h74dt") pod "network-check-target-rzfgn" (UID: "06aa905d-9b4b-4d36-bafc-8fa0c11b3057") : [object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered, object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered] Jul 10 18:02:55.606107 ip-10-0-128-39 kubenswrapper[2558]: E0710 18:02:55.606090 2558 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/3b27429f-90aa-4170-9078-0aa4ea0d0ab8-original-pull-secret podName:3b27429f-90aa-4170-9078-0aa4ea0d0ab8 nodeName:}" failed. No retries permitted until 2026-07-10 18:02:57.606081251 +0000 UTC m=+6.109573738 (durationBeforeRetry 2s). Error: MountVolume.SetUp failed for volume "original-pull-secret" (UniqueName: "kubernetes.io/secret/3b27429f-90aa-4170-9078-0aa4ea0d0ab8-original-pull-secret") pod "global-pull-secret-syncer-vsbdm" (UID: "3b27429f-90aa-4170-9078-0aa4ea0d0ab8") : object "kube-system"/"original-pull-secret" not registered Jul 10 18:02:55.606448 ip-10-0-128-39 kubenswrapper[2558]: I0710 18:02:55.605803 2558 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"original-pull-secret\" (UniqueName: \"kubernetes.io/secret/3b27429f-90aa-4170-9078-0aa4ea0d0ab8-original-pull-secret\") pod \"global-pull-secret-syncer-vsbdm\" (UID: \"3b27429f-90aa-4170-9078-0aa4ea0d0ab8\") " pod="kube-system/global-pull-secret-syncer-vsbdm" Jul 10 18:02:55.606516 ip-10-0-128-39 kubenswrapper[2558]: I0710 18:02:55.606495 2558 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/7fa20252-06b7-48e4-8620-3e52dc980486-metrics-certs\") pod \"network-metrics-daemon-jkr77\" (UID: \"7fa20252-06b7-48e4-8620-3e52dc980486\") " pod="openshift-multus/network-metrics-daemon-jkr77" Jul 10 18:02:55.606636 ip-10-0-128-39 kubenswrapper[2558]: E0710 18:02:55.606607 2558 secret.go:189] Couldn't get secret openshift-multus/metrics-daemon-secret: object "openshift-multus"/"metrics-daemon-secret" not registered Jul 10 18:02:55.606703 ip-10-0-128-39 kubenswrapper[2558]: E0710 18:02:55.606650 2558 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/7fa20252-06b7-48e4-8620-3e52dc980486-metrics-certs podName:7fa20252-06b7-48e4-8620-3e52dc980486 nodeName:}" failed. No retries permitted until 2026-07-10 18:02:57.606636723 +0000 UTC m=+6.110129221 (durationBeforeRetry 2s). Error: MountVolume.SetUp failed for volume "metrics-certs" (UniqueName: "kubernetes.io/secret/7fa20252-06b7-48e4-8620-3e52dc980486-metrics-certs") pod "network-metrics-daemon-jkr77" (UID: "7fa20252-06b7-48e4-8620-3e52dc980486") : object "openshift-multus"/"metrics-daemon-secret" not registered Jul 10 18:02:55.980675 ip-10-0-128-39 kubenswrapper[2558]: I0710 18:02:55.980538 2558 certificate_manager.go:715] "Certificate rotation deadline determined" logger="kubernetes.io/kubelet-serving" expiration="2028-07-09 17:57:53 +0000 UTC" deadline="2028-04-21 20:01:22.863828208 +0000 UTC" Jul 10 18:02:55.980675 ip-10-0-128-39 kubenswrapper[2558]: I0710 18:02:55.980576 2558 certificate_manager.go:431] "Waiting for next certificate rotation" logger="kubernetes.io/kubelet-serving" sleep="15625h58m26.883257469s" Jul 10 18:02:56.095839 ip-10-0-128-39 kubenswrapper[2558]: I0710 18:02:56.095401 2558 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-jkr77" Jul 10 18:02:56.095839 ip-10-0-128-39 kubenswrapper[2558]: E0710 18:02:56.095542 2558 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-jkr77" podUID="7fa20252-06b7-48e4-8620-3e52dc980486" Jul 10 18:02:56.096077 ip-10-0-128-39 kubenswrapper[2558]: I0710 18:02:56.096053 2558 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-rzfgn" Jul 10 18:02:56.097967 ip-10-0-128-39 kubenswrapper[2558]: E0710 18:02:56.096148 2558 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-rzfgn" podUID="06aa905d-9b4b-4d36-bafc-8fa0c11b3057" Jul 10 18:02:56.133796 ip-10-0-128-39 kubenswrapper[2558]: I0710 18:02:56.133753 2558 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kube-system/kube-apiserver-proxy-ip-10-0-128-39.ec2.internal" event={"ID":"91839b707975206c1eb988a2e4de26d3","Type":"ContainerStarted","Data":"1c778a1aa83570772650f731359a3548a6aaaf287b5d79e13f6ce40690ac0e9c"} Jul 10 18:02:57.096118 ip-10-0-128-39 kubenswrapper[2558]: I0710 18:02:57.095733 2558 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-vsbdm" Jul 10 18:02:57.096118 ip-10-0-128-39 kubenswrapper[2558]: E0710 18:02:57.095886 2558 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-vsbdm" podUID="3b27429f-90aa-4170-9078-0aa4ea0d0ab8" Jul 10 18:02:57.623552 ip-10-0-128-39 kubenswrapper[2558]: I0710 18:02:57.623463 2558 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-h74dt\" (UniqueName: \"kubernetes.io/projected/06aa905d-9b4b-4d36-bafc-8fa0c11b3057-kube-api-access-h74dt\") pod \"network-check-target-rzfgn\" (UID: \"06aa905d-9b4b-4d36-bafc-8fa0c11b3057\") " pod="openshift-network-diagnostics/network-check-target-rzfgn" Jul 10 18:02:57.623552 ip-10-0-128-39 kubenswrapper[2558]: I0710 18:02:57.623531 2558 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"original-pull-secret\" (UniqueName: \"kubernetes.io/secret/3b27429f-90aa-4170-9078-0aa4ea0d0ab8-original-pull-secret\") pod \"global-pull-secret-syncer-vsbdm\" (UID: \"3b27429f-90aa-4170-9078-0aa4ea0d0ab8\") " pod="kube-system/global-pull-secret-syncer-vsbdm" Jul 10 18:02:57.623775 ip-10-0-128-39 kubenswrapper[2558]: I0710 18:02:57.623565 2558 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/7fa20252-06b7-48e4-8620-3e52dc980486-metrics-certs\") pod \"network-metrics-daemon-jkr77\" (UID: \"7fa20252-06b7-48e4-8620-3e52dc980486\") " pod="openshift-multus/network-metrics-daemon-jkr77" Jul 10 18:02:57.623775 ip-10-0-128-39 kubenswrapper[2558]: E0710 18:02:57.623687 2558 secret.go:189] Couldn't get secret openshift-multus/metrics-daemon-secret: object "openshift-multus"/"metrics-daemon-secret" not registered Jul 10 18:02:57.623775 ip-10-0-128-39 kubenswrapper[2558]: E0710 18:02:57.623747 2558 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/7fa20252-06b7-48e4-8620-3e52dc980486-metrics-certs podName:7fa20252-06b7-48e4-8620-3e52dc980486 nodeName:}" failed. No retries permitted until 2026-07-10 18:03:01.623728635 +0000 UTC m=+10.127221141 (durationBeforeRetry 4s). Error: MountVolume.SetUp failed for volume "metrics-certs" (UniqueName: "kubernetes.io/secret/7fa20252-06b7-48e4-8620-3e52dc980486-metrics-certs") pod "network-metrics-daemon-jkr77" (UID: "7fa20252-06b7-48e4-8620-3e52dc980486") : object "openshift-multus"/"metrics-daemon-secret" not registered Jul 10 18:02:57.624211 ip-10-0-128-39 kubenswrapper[2558]: E0710 18:02:57.624191 2558 projected.go:291] Couldn't get configMap openshift-network-diagnostics/kube-root-ca.crt: object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered Jul 10 18:02:57.624211 ip-10-0-128-39 kubenswrapper[2558]: E0710 18:02:57.624215 2558 projected.go:291] Couldn't get configMap openshift-network-diagnostics/openshift-service-ca.crt: object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered Jul 10 18:02:57.624356 ip-10-0-128-39 kubenswrapper[2558]: E0710 18:02:57.624226 2558 projected.go:196] Error preparing data for projected volume kube-api-access-h74dt for pod openshift-network-diagnostics/network-check-target-rzfgn: [object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered, object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered] Jul 10 18:02:57.624356 ip-10-0-128-39 kubenswrapper[2558]: E0710 18:02:57.624271 2558 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/projected/06aa905d-9b4b-4d36-bafc-8fa0c11b3057-kube-api-access-h74dt podName:06aa905d-9b4b-4d36-bafc-8fa0c11b3057 nodeName:}" failed. No retries permitted until 2026-07-10 18:03:01.624255178 +0000 UTC m=+10.127747681 (durationBeforeRetry 4s). Error: MountVolume.SetUp failed for volume "kube-api-access-h74dt" (UniqueName: "kubernetes.io/projected/06aa905d-9b4b-4d36-bafc-8fa0c11b3057-kube-api-access-h74dt") pod "network-check-target-rzfgn" (UID: "06aa905d-9b4b-4d36-bafc-8fa0c11b3057") : [object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered, object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered] Jul 10 18:02:57.624356 ip-10-0-128-39 kubenswrapper[2558]: E0710 18:02:57.624331 2558 secret.go:189] Couldn't get secret kube-system/original-pull-secret: object "kube-system"/"original-pull-secret" not registered Jul 10 18:02:57.624505 ip-10-0-128-39 kubenswrapper[2558]: E0710 18:02:57.624365 2558 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/3b27429f-90aa-4170-9078-0aa4ea0d0ab8-original-pull-secret podName:3b27429f-90aa-4170-9078-0aa4ea0d0ab8 nodeName:}" failed. No retries permitted until 2026-07-10 18:03:01.624354293 +0000 UTC m=+10.127846784 (durationBeforeRetry 4s). Error: MountVolume.SetUp failed for volume "original-pull-secret" (UniqueName: "kubernetes.io/secret/3b27429f-90aa-4170-9078-0aa4ea0d0ab8-original-pull-secret") pod "global-pull-secret-syncer-vsbdm" (UID: "3b27429f-90aa-4170-9078-0aa4ea0d0ab8") : object "kube-system"/"original-pull-secret" not registered Jul 10 18:02:58.097003 ip-10-0-128-39 kubenswrapper[2558]: I0710 18:02:58.096909 2558 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-jkr77" Jul 10 18:02:58.097378 ip-10-0-128-39 kubenswrapper[2558]: E0710 18:02:58.097049 2558 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-jkr77" podUID="7fa20252-06b7-48e4-8620-3e52dc980486" Jul 10 18:02:58.097378 ip-10-0-128-39 kubenswrapper[2558]: I0710 18:02:58.097228 2558 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-rzfgn" Jul 10 18:02:58.097378 ip-10-0-128-39 kubenswrapper[2558]: E0710 18:02:58.097341 2558 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-rzfgn" podUID="06aa905d-9b4b-4d36-bafc-8fa0c11b3057" Jul 10 18:02:58.150215 ip-10-0-128-39 kubenswrapper[2558]: I0710 18:02:58.150158 2558 generic.go:356] "Generic (PLEG): container finished" podID="a549cd29ca3f3dc6e1085c2e86957fb9" containerID="2c06fdb06db33bf93dfc12ce9ac5bd9063cbb698ecf2d39b8e14b79cc91fa66d" exitCode=0 Jul 10 18:02:58.150215 ip-10-0-128-39 kubenswrapper[2558]: I0710 18:02:58.150205 2558 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-128-39.ec2.internal" event={"ID":"a549cd29ca3f3dc6e1085c2e86957fb9","Type":"ContainerDied","Data":"2c06fdb06db33bf93dfc12ce9ac5bd9063cbb698ecf2d39b8e14b79cc91fa66d"} Jul 10 18:02:58.167417 ip-10-0-128-39 kubenswrapper[2558]: I0710 18:02:58.167235 2558 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="kube-system/kube-apiserver-proxy-ip-10-0-128-39.ec2.internal" podStartSLOduration=5.167215692 podStartE2EDuration="5.167215692s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=true podCreationTimestamp="2026-07-10 18:02:53 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-10 18:02:56.149867906 +0000 UTC m=+4.653360416" watchObservedRunningTime="2026-07-10 18:02:58.167215692 +0000 UTC m=+6.670708199" Jul 10 18:02:59.095109 ip-10-0-128-39 kubenswrapper[2558]: I0710 18:02:59.095063 2558 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-vsbdm" Jul 10 18:02:59.095377 ip-10-0-128-39 kubenswrapper[2558]: E0710 18:02:59.095215 2558 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-vsbdm" podUID="3b27429f-90aa-4170-9078-0aa4ea0d0ab8" Jul 10 18:03:00.095090 ip-10-0-128-39 kubenswrapper[2558]: I0710 18:03:00.095054 2558 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-jkr77" Jul 10 18:03:00.095563 ip-10-0-128-39 kubenswrapper[2558]: I0710 18:03:00.095201 2558 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-rzfgn" Jul 10 18:03:00.095563 ip-10-0-128-39 kubenswrapper[2558]: E0710 18:03:00.095273 2558 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-rzfgn" podUID="06aa905d-9b4b-4d36-bafc-8fa0c11b3057" Jul 10 18:03:00.095563 ip-10-0-128-39 kubenswrapper[2558]: E0710 18:03:00.095195 2558 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-jkr77" podUID="7fa20252-06b7-48e4-8620-3e52dc980486" Jul 10 18:03:01.096055 ip-10-0-128-39 kubenswrapper[2558]: I0710 18:03:01.095966 2558 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-vsbdm" Jul 10 18:03:01.096535 ip-10-0-128-39 kubenswrapper[2558]: E0710 18:03:01.096196 2558 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-vsbdm" podUID="3b27429f-90aa-4170-9078-0aa4ea0d0ab8" Jul 10 18:03:01.659300 ip-10-0-128-39 kubenswrapper[2558]: I0710 18:03:01.659259 2558 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-h74dt\" (UniqueName: \"kubernetes.io/projected/06aa905d-9b4b-4d36-bafc-8fa0c11b3057-kube-api-access-h74dt\") pod \"network-check-target-rzfgn\" (UID: \"06aa905d-9b4b-4d36-bafc-8fa0c11b3057\") " pod="openshift-network-diagnostics/network-check-target-rzfgn" Jul 10 18:03:01.659498 ip-10-0-128-39 kubenswrapper[2558]: I0710 18:03:01.659338 2558 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"original-pull-secret\" (UniqueName: \"kubernetes.io/secret/3b27429f-90aa-4170-9078-0aa4ea0d0ab8-original-pull-secret\") pod \"global-pull-secret-syncer-vsbdm\" (UID: \"3b27429f-90aa-4170-9078-0aa4ea0d0ab8\") " pod="kube-system/global-pull-secret-syncer-vsbdm" Jul 10 18:03:01.659498 ip-10-0-128-39 kubenswrapper[2558]: I0710 18:03:01.659375 2558 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/7fa20252-06b7-48e4-8620-3e52dc980486-metrics-certs\") pod \"network-metrics-daemon-jkr77\" (UID: \"7fa20252-06b7-48e4-8620-3e52dc980486\") " pod="openshift-multus/network-metrics-daemon-jkr77" Jul 10 18:03:01.659498 ip-10-0-128-39 kubenswrapper[2558]: E0710 18:03:01.659452 2558 secret.go:189] Couldn't get secret kube-system/original-pull-secret: object "kube-system"/"original-pull-secret" not registered Jul 10 18:03:01.659498 ip-10-0-128-39 kubenswrapper[2558]: E0710 18:03:01.659470 2558 secret.go:189] Couldn't get secret openshift-multus/metrics-daemon-secret: object "openshift-multus"/"metrics-daemon-secret" not registered Jul 10 18:03:01.659687 ip-10-0-128-39 kubenswrapper[2558]: E0710 18:03:01.659523 2558 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/7fa20252-06b7-48e4-8620-3e52dc980486-metrics-certs podName:7fa20252-06b7-48e4-8620-3e52dc980486 nodeName:}" failed. No retries permitted until 2026-07-10 18:03:09.659506089 +0000 UTC m=+18.162998573 (durationBeforeRetry 8s). Error: MountVolume.SetUp failed for volume "metrics-certs" (UniqueName: "kubernetes.io/secret/7fa20252-06b7-48e4-8620-3e52dc980486-metrics-certs") pod "network-metrics-daemon-jkr77" (UID: "7fa20252-06b7-48e4-8620-3e52dc980486") : object "openshift-multus"/"metrics-daemon-secret" not registered Jul 10 18:03:01.659687 ip-10-0-128-39 kubenswrapper[2558]: E0710 18:03:01.659542 2558 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/3b27429f-90aa-4170-9078-0aa4ea0d0ab8-original-pull-secret podName:3b27429f-90aa-4170-9078-0aa4ea0d0ab8 nodeName:}" failed. No retries permitted until 2026-07-10 18:03:09.659530618 +0000 UTC m=+18.163023105 (durationBeforeRetry 8s). Error: MountVolume.SetUp failed for volume "original-pull-secret" (UniqueName: "kubernetes.io/secret/3b27429f-90aa-4170-9078-0aa4ea0d0ab8-original-pull-secret") pod "global-pull-secret-syncer-vsbdm" (UID: "3b27429f-90aa-4170-9078-0aa4ea0d0ab8") : object "kube-system"/"original-pull-secret" not registered Jul 10 18:03:01.659687 ip-10-0-128-39 kubenswrapper[2558]: E0710 18:03:01.659454 2558 projected.go:291] Couldn't get configMap openshift-network-diagnostics/kube-root-ca.crt: object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered Jul 10 18:03:01.659687 ip-10-0-128-39 kubenswrapper[2558]: E0710 18:03:01.659579 2558 projected.go:291] Couldn't get configMap openshift-network-diagnostics/openshift-service-ca.crt: object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered Jul 10 18:03:01.659687 ip-10-0-128-39 kubenswrapper[2558]: E0710 18:03:01.659593 2558 projected.go:196] Error preparing data for projected volume kube-api-access-h74dt for pod openshift-network-diagnostics/network-check-target-rzfgn: [object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered, object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered] Jul 10 18:03:01.659687 ip-10-0-128-39 kubenswrapper[2558]: E0710 18:03:01.659627 2558 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/projected/06aa905d-9b4b-4d36-bafc-8fa0c11b3057-kube-api-access-h74dt podName:06aa905d-9b4b-4d36-bafc-8fa0c11b3057 nodeName:}" failed. No retries permitted until 2026-07-10 18:03:09.659616578 +0000 UTC m=+18.163109093 (durationBeforeRetry 8s). Error: MountVolume.SetUp failed for volume "kube-api-access-h74dt" (UniqueName: "kubernetes.io/projected/06aa905d-9b4b-4d36-bafc-8fa0c11b3057-kube-api-access-h74dt") pod "network-check-target-rzfgn" (UID: "06aa905d-9b4b-4d36-bafc-8fa0c11b3057") : [object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered, object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered] Jul 10 18:03:02.096734 ip-10-0-128-39 kubenswrapper[2558]: I0710 18:03:02.096117 2558 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-rzfgn" Jul 10 18:03:02.096734 ip-10-0-128-39 kubenswrapper[2558]: E0710 18:03:02.096229 2558 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-rzfgn" podUID="06aa905d-9b4b-4d36-bafc-8fa0c11b3057" Jul 10 18:03:02.096734 ip-10-0-128-39 kubenswrapper[2558]: I0710 18:03:02.096594 2558 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-jkr77" Jul 10 18:03:02.096734 ip-10-0-128-39 kubenswrapper[2558]: E0710 18:03:02.096694 2558 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-jkr77" podUID="7fa20252-06b7-48e4-8620-3e52dc980486" Jul 10 18:03:03.095032 ip-10-0-128-39 kubenswrapper[2558]: I0710 18:03:03.095001 2558 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-vsbdm" Jul 10 18:03:03.095195 ip-10-0-128-39 kubenswrapper[2558]: E0710 18:03:03.095134 2558 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-vsbdm" podUID="3b27429f-90aa-4170-9078-0aa4ea0d0ab8" Jul 10 18:03:04.095208 ip-10-0-128-39 kubenswrapper[2558]: I0710 18:03:04.095176 2558 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-jkr77" Jul 10 18:03:04.095658 ip-10-0-128-39 kubenswrapper[2558]: E0710 18:03:04.095317 2558 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-jkr77" podUID="7fa20252-06b7-48e4-8620-3e52dc980486" Jul 10 18:03:04.095790 ip-10-0-128-39 kubenswrapper[2558]: I0710 18:03:04.095770 2558 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-rzfgn" Jul 10 18:03:04.095920 ip-10-0-128-39 kubenswrapper[2558]: E0710 18:03:04.095896 2558 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-rzfgn" podUID="06aa905d-9b4b-4d36-bafc-8fa0c11b3057" Jul 10 18:03:05.094889 ip-10-0-128-39 kubenswrapper[2558]: I0710 18:03:05.094845 2558 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-vsbdm" Jul 10 18:03:05.095075 ip-10-0-128-39 kubenswrapper[2558]: E0710 18:03:05.094974 2558 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-vsbdm" podUID="3b27429f-90aa-4170-9078-0aa4ea0d0ab8" Jul 10 18:03:06.095166 ip-10-0-128-39 kubenswrapper[2558]: I0710 18:03:06.095099 2558 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-jkr77" Jul 10 18:03:06.095166 ip-10-0-128-39 kubenswrapper[2558]: I0710 18:03:06.095146 2558 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-rzfgn" Jul 10 18:03:06.095667 ip-10-0-128-39 kubenswrapper[2558]: E0710 18:03:06.095254 2558 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-jkr77" podUID="7fa20252-06b7-48e4-8620-3e52dc980486" Jul 10 18:03:06.095667 ip-10-0-128-39 kubenswrapper[2558]: E0710 18:03:06.095380 2558 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-rzfgn" podUID="06aa905d-9b4b-4d36-bafc-8fa0c11b3057" Jul 10 18:03:07.095689 ip-10-0-128-39 kubenswrapper[2558]: I0710 18:03:07.095655 2558 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-vsbdm" Jul 10 18:03:07.096224 ip-10-0-128-39 kubenswrapper[2558]: E0710 18:03:07.095794 2558 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-vsbdm" podUID="3b27429f-90aa-4170-9078-0aa4ea0d0ab8" Jul 10 18:03:08.095387 ip-10-0-128-39 kubenswrapper[2558]: I0710 18:03:08.095349 2558 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-jkr77" Jul 10 18:03:08.095540 ip-10-0-128-39 kubenswrapper[2558]: I0710 18:03:08.095355 2558 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-rzfgn" Jul 10 18:03:08.095540 ip-10-0-128-39 kubenswrapper[2558]: E0710 18:03:08.095476 2558 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-jkr77" podUID="7fa20252-06b7-48e4-8620-3e52dc980486" Jul 10 18:03:08.095540 ip-10-0-128-39 kubenswrapper[2558]: E0710 18:03:08.095530 2558 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-rzfgn" podUID="06aa905d-9b4b-4d36-bafc-8fa0c11b3057" Jul 10 18:03:09.095507 ip-10-0-128-39 kubenswrapper[2558]: I0710 18:03:09.095474 2558 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-vsbdm" Jul 10 18:03:09.095999 ip-10-0-128-39 kubenswrapper[2558]: E0710 18:03:09.095607 2558 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-vsbdm" podUID="3b27429f-90aa-4170-9078-0aa4ea0d0ab8" Jul 10 18:03:09.723193 ip-10-0-128-39 kubenswrapper[2558]: I0710 18:03:09.723103 2558 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"original-pull-secret\" (UniqueName: \"kubernetes.io/secret/3b27429f-90aa-4170-9078-0aa4ea0d0ab8-original-pull-secret\") pod \"global-pull-secret-syncer-vsbdm\" (UID: \"3b27429f-90aa-4170-9078-0aa4ea0d0ab8\") " pod="kube-system/global-pull-secret-syncer-vsbdm" Jul 10 18:03:09.723193 ip-10-0-128-39 kubenswrapper[2558]: I0710 18:03:09.723165 2558 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/7fa20252-06b7-48e4-8620-3e52dc980486-metrics-certs\") pod \"network-metrics-daemon-jkr77\" (UID: \"7fa20252-06b7-48e4-8620-3e52dc980486\") " pod="openshift-multus/network-metrics-daemon-jkr77" Jul 10 18:03:09.723193 ip-10-0-128-39 kubenswrapper[2558]: I0710 18:03:09.723192 2558 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-h74dt\" (UniqueName: \"kubernetes.io/projected/06aa905d-9b4b-4d36-bafc-8fa0c11b3057-kube-api-access-h74dt\") pod \"network-check-target-rzfgn\" (UID: \"06aa905d-9b4b-4d36-bafc-8fa0c11b3057\") " pod="openshift-network-diagnostics/network-check-target-rzfgn" Jul 10 18:03:09.723519 ip-10-0-128-39 kubenswrapper[2558]: E0710 18:03:09.723235 2558 secret.go:189] Couldn't get secret kube-system/original-pull-secret: object "kube-system"/"original-pull-secret" not registered Jul 10 18:03:09.723519 ip-10-0-128-39 kubenswrapper[2558]: E0710 18:03:09.723311 2558 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/3b27429f-90aa-4170-9078-0aa4ea0d0ab8-original-pull-secret podName:3b27429f-90aa-4170-9078-0aa4ea0d0ab8 nodeName:}" failed. No retries permitted until 2026-07-10 18:03:25.723289099 +0000 UTC m=+34.226781595 (durationBeforeRetry 16s). Error: MountVolume.SetUp failed for volume "original-pull-secret" (UniqueName: "kubernetes.io/secret/3b27429f-90aa-4170-9078-0aa4ea0d0ab8-original-pull-secret") pod "global-pull-secret-syncer-vsbdm" (UID: "3b27429f-90aa-4170-9078-0aa4ea0d0ab8") : object "kube-system"/"original-pull-secret" not registered Jul 10 18:03:09.723519 ip-10-0-128-39 kubenswrapper[2558]: E0710 18:03:09.723326 2558 projected.go:291] Couldn't get configMap openshift-network-diagnostics/kube-root-ca.crt: object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered Jul 10 18:03:09.723519 ip-10-0-128-39 kubenswrapper[2558]: E0710 18:03:09.723343 2558 projected.go:291] Couldn't get configMap openshift-network-diagnostics/openshift-service-ca.crt: object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered Jul 10 18:03:09.723519 ip-10-0-128-39 kubenswrapper[2558]: E0710 18:03:09.723342 2558 secret.go:189] Couldn't get secret openshift-multus/metrics-daemon-secret: object "openshift-multus"/"metrics-daemon-secret" not registered Jul 10 18:03:09.723519 ip-10-0-128-39 kubenswrapper[2558]: E0710 18:03:09.723394 2558 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/7fa20252-06b7-48e4-8620-3e52dc980486-metrics-certs podName:7fa20252-06b7-48e4-8620-3e52dc980486 nodeName:}" failed. No retries permitted until 2026-07-10 18:03:25.72338081 +0000 UTC m=+34.226873295 (durationBeforeRetry 16s). Error: MountVolume.SetUp failed for volume "metrics-certs" (UniqueName: "kubernetes.io/secret/7fa20252-06b7-48e4-8620-3e52dc980486-metrics-certs") pod "network-metrics-daemon-jkr77" (UID: "7fa20252-06b7-48e4-8620-3e52dc980486") : object "openshift-multus"/"metrics-daemon-secret" not registered Jul 10 18:03:09.723519 ip-10-0-128-39 kubenswrapper[2558]: E0710 18:03:09.723352 2558 projected.go:196] Error preparing data for projected volume kube-api-access-h74dt for pod openshift-network-diagnostics/network-check-target-rzfgn: [object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered, object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered] Jul 10 18:03:09.723519 ip-10-0-128-39 kubenswrapper[2558]: E0710 18:03:09.723432 2558 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/projected/06aa905d-9b4b-4d36-bafc-8fa0c11b3057-kube-api-access-h74dt podName:06aa905d-9b4b-4d36-bafc-8fa0c11b3057 nodeName:}" failed. No retries permitted until 2026-07-10 18:03:25.723425587 +0000 UTC m=+34.226918071 (durationBeforeRetry 16s). Error: MountVolume.SetUp failed for volume "kube-api-access-h74dt" (UniqueName: "kubernetes.io/projected/06aa905d-9b4b-4d36-bafc-8fa0c11b3057-kube-api-access-h74dt") pod "network-check-target-rzfgn" (UID: "06aa905d-9b4b-4d36-bafc-8fa0c11b3057") : [object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered, object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered] Jul 10 18:03:10.095711 ip-10-0-128-39 kubenswrapper[2558]: I0710 18:03:10.095626 2558 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-rzfgn" Jul 10 18:03:10.096140 ip-10-0-128-39 kubenswrapper[2558]: I0710 18:03:10.095629 2558 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-jkr77" Jul 10 18:03:10.096140 ip-10-0-128-39 kubenswrapper[2558]: E0710 18:03:10.095755 2558 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-rzfgn" podUID="06aa905d-9b4b-4d36-bafc-8fa0c11b3057" Jul 10 18:03:10.096140 ip-10-0-128-39 kubenswrapper[2558]: E0710 18:03:10.095864 2558 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-jkr77" podUID="7fa20252-06b7-48e4-8620-3e52dc980486" Jul 10 18:03:11.095490 ip-10-0-128-39 kubenswrapper[2558]: I0710 18:03:11.095457 2558 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-vsbdm" Jul 10 18:03:11.095667 ip-10-0-128-39 kubenswrapper[2558]: E0710 18:03:11.095566 2558 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-vsbdm" podUID="3b27429f-90aa-4170-9078-0aa4ea0d0ab8" Jul 10 18:03:12.096149 ip-10-0-128-39 kubenswrapper[2558]: I0710 18:03:12.096105 2558 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-jkr77" Jul 10 18:03:12.096731 ip-10-0-128-39 kubenswrapper[2558]: E0710 18:03:12.096702 2558 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-jkr77" podUID="7fa20252-06b7-48e4-8620-3e52dc980486" Jul 10 18:03:12.096858 ip-10-0-128-39 kubenswrapper[2558]: I0710 18:03:12.096788 2558 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-rzfgn" Jul 10 18:03:12.097227 ip-10-0-128-39 kubenswrapper[2558]: E0710 18:03:12.097180 2558 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-rzfgn" podUID="06aa905d-9b4b-4d36-bafc-8fa0c11b3057" Jul 10 18:03:13.095068 ip-10-0-128-39 kubenswrapper[2558]: I0710 18:03:13.095023 2558 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-vsbdm" Jul 10 18:03:13.095248 ip-10-0-128-39 kubenswrapper[2558]: E0710 18:03:13.095138 2558 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-vsbdm" podUID="3b27429f-90aa-4170-9078-0aa4ea0d0ab8" Jul 10 18:03:14.095475 ip-10-0-128-39 kubenswrapper[2558]: I0710 18:03:14.095286 2558 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-jkr77" Jul 10 18:03:14.096252 ip-10-0-128-39 kubenswrapper[2558]: I0710 18:03:14.095328 2558 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-rzfgn" Jul 10 18:03:14.096252 ip-10-0-128-39 kubenswrapper[2558]: E0710 18:03:14.095565 2558 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-jkr77" podUID="7fa20252-06b7-48e4-8620-3e52dc980486" Jul 10 18:03:14.096252 ip-10-0-128-39 kubenswrapper[2558]: E0710 18:03:14.095643 2558 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-rzfgn" podUID="06aa905d-9b4b-4d36-bafc-8fa0c11b3057" Jul 10 18:03:14.188217 ip-10-0-128-39 kubenswrapper[2558]: I0710 18:03:14.188174 2558 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-128-39.ec2.internal" event={"ID":"a549cd29ca3f3dc6e1085c2e86957fb9","Type":"ContainerStarted","Data":"63237e877e62207306a133fe9abd414e92d06adf4ff763e9e345cabfbf3f1065"} Jul 10 18:03:14.190034 ip-10-0-128-39 kubenswrapper[2558]: I0710 18:03:14.190008 2558 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ovn-kubernetes/ovnkube-node-ll44q" event={"ID":"eabc6c69-b1e8-40d5-b41f-f49778a5248c","Type":"ContainerStarted","Data":"1a956f3831339136536286c61a717aa148ef39246aa0031110ff6f337b8fa9ee"} Jul 10 18:03:14.190145 ip-10-0-128-39 kubenswrapper[2558]: I0710 18:03:14.190093 2558 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ovn-kubernetes/ovnkube-node-ll44q" event={"ID":"eabc6c69-b1e8-40d5-b41f-f49778a5248c","Type":"ContainerStarted","Data":"db988fa2cb5f7886ab42897aa0d60d648d254d601306785e3a98217d76aed1bf"} Jul 10 18:03:14.191613 ip-10-0-128-39 kubenswrapper[2558]: I0710 18:03:14.191587 2558 generic.go:356] "Generic (PLEG): container finished" podID="53492c2f-2433-4658-af05-d58878aed082" containerID="83b06db9ab42de9d2716a7b196499b677b4c2a15cdd3a2c67be858707b8f4d36" exitCode=0 Jul 10 18:03:14.191708 ip-10-0-128-39 kubenswrapper[2558]: I0710 18:03:14.191658 2558 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-multus/multus-additional-cni-plugins-v22pb" event={"ID":"53492c2f-2433-4658-af05-d58878aed082","Type":"ContainerDied","Data":"83b06db9ab42de9d2716a7b196499b677b4c2a15cdd3a2c67be858707b8f4d36"} Jul 10 18:03:14.193410 ip-10-0-128-39 kubenswrapper[2558]: I0710 18:03:14.193369 2558 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-cluster-node-tuning-operator/tuned-kbrll" event={"ID":"df37e213-59f3-4309-9bec-a41b2238ab98","Type":"ContainerStarted","Data":"503b865f4219ba7525fe69da3eac83486208d7a19023e045e8cb6fa4068ac9ff"} Jul 10 18:03:14.194750 ip-10-0-128-39 kubenswrapper[2558]: I0710 18:03:14.194718 2558 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-image-registry/node-ca-r7qh2" event={"ID":"fe05c85a-2eb8-429b-9776-a3408a38903f","Type":"ContainerStarted","Data":"557eb0811d6831f065b754bb642011353d0dde6173914a7a83b07781a97dae28"} Jul 10 18:03:14.196273 ip-10-0-128-39 kubenswrapper[2558]: I0710 18:03:14.196247 2558 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-dns/node-resolver-jqb25" event={"ID":"12ad9a8e-4768-4362-818f-d2fb73c8a236","Type":"ContainerStarted","Data":"8ffddf73e2bb9d66f9d8ca54aafa5ad05db16d998ba0e9b67fdd7b4cac8c09aa"} Jul 10 18:03:14.197677 ip-10-0-128-39 kubenswrapper[2558]: I0710 18:03:14.197646 2558 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kube-system/konnectivity-agent-dvp86" event={"ID":"2da19865-b021-4789-b1e0-694d4f6c543d","Type":"ContainerStarted","Data":"12a4489ba61d704a86f9527f271ff636ba68e82654c7b7cd6d1029c94fdd858c"} Jul 10 18:03:14.199716 ip-10-0-128-39 kubenswrapper[2558]: I0710 18:03:14.199679 2558 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-hps6m" event={"ID":"cdee596a-a663-4627-8c46-d7066d21978b","Type":"ContainerStarted","Data":"028c55453c48779487912f11babdd3ef5988dd397870f9231777227302759dd5"} Jul 10 18:03:14.201306 ip-10-0-128-39 kubenswrapper[2558]: I0710 18:03:14.201285 2558 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-multus/multus-kvqn7" event={"ID":"11be9079-5d81-46c9-8f4c-c2e305bf2144","Type":"ContainerStarted","Data":"347d4121798c0f9378db6d980a35546f5f3dd5bbd0186e3c42604e3e9521f0da"} Jul 10 18:03:14.239745 ip-10-0-128-39 kubenswrapper[2558]: I0710 18:03:14.239684 2558 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-image-registry/node-ca-r7qh2" podStartSLOduration=2.692029924 podStartE2EDuration="22.23966063s" totalImagesPullingTime="19.547630706s" totalInitContainerRuntime="0s" isStatefulPod=true podCreationTimestamp="2026-07-10 18:02:52 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-10 18:03:14.238507177 +0000 UTC m=+22.741999683" watchObservedRunningTime="2026-07-10 18:03:14.23966063 +0000 UTC m=+22.743153137" Jul 10 18:03:14.256388 ip-10-0-128-39 kubenswrapper[2558]: I0710 18:03:14.256344 2558 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="kube-system/konnectivity-agent-dvp86" podStartSLOduration=11.065312738 podStartE2EDuration="22.256329755s" totalImagesPullingTime="11.191017017s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-10 18:02:52 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-10 18:03:14.25616171 +0000 UTC m=+22.759654216" watchObservedRunningTime="2026-07-10 18:03:14.256329755 +0000 UTC m=+22.759822261" Jul 10 18:03:14.311343 ip-10-0-128-39 kubenswrapper[2558]: I0710 18:03:14.311295 2558 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-cluster-node-tuning-operator/tuned-kbrll" podStartSLOduration=2.801515416 podStartE2EDuration="22.311279424s" totalImagesPullingTime="19.509764008s" totalInitContainerRuntime="0s" isStatefulPod=true podCreationTimestamp="2026-07-10 18:02:52 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-10 18:03:14.285482763 +0000 UTC m=+22.788975269" watchObservedRunningTime="2026-07-10 18:03:14.311279424 +0000 UTC m=+22.814771969" Jul 10 18:03:14.331719 ip-10-0-128-39 kubenswrapper[2558]: I0710 18:03:14.331676 2558 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-128-39.ec2.internal" podStartSLOduration=21.331659303 podStartE2EDuration="21.331659303s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=true podCreationTimestamp="2026-07-10 18:02:53 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-10 18:03:14.311364862 +0000 UTC m=+22.814857368" watchObservedRunningTime="2026-07-10 18:03:14.331659303 +0000 UTC m=+22.835151808" Jul 10 18:03:14.331977 ip-10-0-128-39 kubenswrapper[2558]: I0710 18:03:14.331951 2558 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-dns/node-resolver-jqb25" podStartSLOduration=2.753219788 podStartE2EDuration="22.331945026s" totalImagesPullingTime="19.578725238s" totalInitContainerRuntime="0s" isStatefulPod=true podCreationTimestamp="2026-07-10 18:02:52 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-10 18:03:14.331500801 +0000 UTC m=+22.834993308" watchObservedRunningTime="2026-07-10 18:03:14.331945026 +0000 UTC m=+22.835437532" Jul 10 18:03:14.365733 ip-10-0-128-39 kubenswrapper[2558]: I0710 18:03:14.365688 2558 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-multus/multus-kvqn7" podStartSLOduration=2.790366417 podStartE2EDuration="22.365672931s" totalImagesPullingTime="19.575306514s" totalInitContainerRuntime="0s" isStatefulPod=true podCreationTimestamp="2026-07-10 18:02:52 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-10 18:03:14.365654016 +0000 UTC m=+22.869146523" watchObservedRunningTime="2026-07-10 18:03:14.365672931 +0000 UTC m=+22.869165437" Jul 10 18:03:14.963501 ip-10-0-128-39 kubenswrapper[2558]: I0710 18:03:14.963175 2558 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="kube-system/konnectivity-agent-dvp86" Jul 10 18:03:14.964034 ip-10-0-128-39 kubenswrapper[2558]: I0710 18:03:14.964011 2558 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="kube-system/konnectivity-agent-dvp86" Jul 10 18:03:15.095010 ip-10-0-128-39 kubenswrapper[2558]: I0710 18:03:15.094981 2558 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-vsbdm" Jul 10 18:03:15.095217 ip-10-0-128-39 kubenswrapper[2558]: E0710 18:03:15.095074 2558 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-vsbdm" podUID="3b27429f-90aa-4170-9078-0aa4ea0d0ab8" Jul 10 18:03:15.205098 ip-10-0-128-39 kubenswrapper[2558]: I0710 18:03:15.205066 2558 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-network-operator/iptables-alerter-zmg24" event={"ID":"1b18a733-4a78-4cf0-9377-3a26c9ae4a2f","Type":"ContainerStarted","Data":"f719f07d066d24b96cd2c1edffc1ea6d5be66ea6f0a114a70a08609022c0513d"} Jul 10 18:03:15.208581 ip-10-0-128-39 kubenswrapper[2558]: I0710 18:03:15.208555 2558 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ovn-kubernetes_ovnkube-node-ll44q_eabc6c69-b1e8-40d5-b41f-f49778a5248c/ovn-acl-logging/0.log" Jul 10 18:03:15.208961 ip-10-0-128-39 kubenswrapper[2558]: I0710 18:03:15.208939 2558 generic.go:356] "Generic (PLEG): container finished" podID="eabc6c69-b1e8-40d5-b41f-f49778a5248c" containerID="1a956f3831339136536286c61a717aa148ef39246aa0031110ff6f337b8fa9ee" exitCode=1 Jul 10 18:03:15.209076 ip-10-0-128-39 kubenswrapper[2558]: I0710 18:03:15.209034 2558 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ovn-kubernetes/ovnkube-node-ll44q" event={"ID":"eabc6c69-b1e8-40d5-b41f-f49778a5248c","Type":"ContainerDied","Data":"1a956f3831339136536286c61a717aa148ef39246aa0031110ff6f337b8fa9ee"} Jul 10 18:03:15.209154 ip-10-0-128-39 kubenswrapper[2558]: I0710 18:03:15.209135 2558 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ovn-kubernetes/ovnkube-node-ll44q" event={"ID":"eabc6c69-b1e8-40d5-b41f-f49778a5248c","Type":"ContainerStarted","Data":"e51171012cc0cd07767fbfa4db1f3dc756bafa041703cd322ca53118cc1ec502"} Jul 10 18:03:15.209248 ip-10-0-128-39 kubenswrapper[2558]: I0710 18:03:15.209178 2558 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ovn-kubernetes/ovnkube-node-ll44q" event={"ID":"eabc6c69-b1e8-40d5-b41f-f49778a5248c","Type":"ContainerStarted","Data":"f1e88148c17470784af23b83120102a4f956b7d46967109d810414fcdf06a877"} Jul 10 18:03:15.209248 ip-10-0-128-39 kubenswrapper[2558]: I0710 18:03:15.209212 2558 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ovn-kubernetes/ovnkube-node-ll44q" event={"ID":"eabc6c69-b1e8-40d5-b41f-f49778a5248c","Type":"ContainerStarted","Data":"48185790c252102739a342124b7386030d211195de5bd285cefac6d23b1c8611"} Jul 10 18:03:15.209360 ip-10-0-128-39 kubenswrapper[2558]: I0710 18:03:15.209256 2558 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ovn-kubernetes/ovnkube-node-ll44q" event={"ID":"eabc6c69-b1e8-40d5-b41f-f49778a5248c","Type":"ContainerStarted","Data":"28e16213a70a38e7e8f9e6115db13820c44b4bd661257bc7620df155e0fbf770"} Jul 10 18:03:15.212738 ip-10-0-128-39 kubenswrapper[2558]: I0710 18:03:15.212712 2558 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="kube-system/konnectivity-agent-dvp86" Jul 10 18:03:15.212974 ip-10-0-128-39 kubenswrapper[2558]: I0710 18:03:15.212956 2558 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="kube-system/konnectivity-agent-dvp86" Jul 10 18:03:15.221114 ip-10-0-128-39 kubenswrapper[2558]: I0710 18:03:15.221077 2558 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-network-operator/iptables-alerter-zmg24" podStartSLOduration=3.640285454 podStartE2EDuration="23.221066656s" totalImagesPullingTime="19.580781202s" totalInitContainerRuntime="0s" isStatefulPod=true podCreationTimestamp="2026-07-10 18:02:52 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-10 18:03:15.220774171 +0000 UTC m=+23.724266677" watchObservedRunningTime="2026-07-10 18:03:15.221066656 +0000 UTC m=+23.724559162" Jul 10 18:03:15.395932 ip-10-0-128-39 kubenswrapper[2558]: I0710 18:03:15.395882 2558 plugin_watcher.go:194] "Adding socket path or updating timestamp to desired state cache" path="/var/lib/kubelet/plugins_registry/ebs.csi.aws.com-reg.sock" Jul 10 18:03:16.056602 ip-10-0-128-39 kubenswrapper[2558]: I0710 18:03:16.056491 2558 reconciler.go:161] "OperationExecutor.RegisterPlugin started" plugin={"SocketPath":"/var/lib/kubelet/plugins_registry/ebs.csi.aws.com-reg.sock","Timestamp":"2026-07-10T18:03:15.395903443Z","UUID":"7f438dff-3623-4b46-9789-428bd15544c3","Handler":null,"Name":"","Endpoint":""} Jul 10 18:03:16.058719 ip-10-0-128-39 kubenswrapper[2558]: I0710 18:03:16.058693 2558 csi_plugin.go:106] kubernetes.io/csi: Trying to validate a new CSI Driver with name: ebs.csi.aws.com endpoint: /var/lib/kubelet/plugins/ebs.csi.aws.com/csi.sock versions: 1.0.0 Jul 10 18:03:16.058887 ip-10-0-128-39 kubenswrapper[2558]: I0710 18:03:16.058743 2558 csi_plugin.go:119] kubernetes.io/csi: Register new plugin with name: ebs.csi.aws.com at endpoint: /var/lib/kubelet/plugins/ebs.csi.aws.com/csi.sock Jul 10 18:03:16.095042 ip-10-0-128-39 kubenswrapper[2558]: I0710 18:03:16.094901 2558 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-rzfgn" Jul 10 18:03:16.095042 ip-10-0-128-39 kubenswrapper[2558]: I0710 18:03:16.094948 2558 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-jkr77" Jul 10 18:03:16.095042 ip-10-0-128-39 kubenswrapper[2558]: E0710 18:03:16.095018 2558 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-rzfgn" podUID="06aa905d-9b4b-4d36-bafc-8fa0c11b3057" Jul 10 18:03:16.095352 ip-10-0-128-39 kubenswrapper[2558]: E0710 18:03:16.095172 2558 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-jkr77" podUID="7fa20252-06b7-48e4-8620-3e52dc980486" Jul 10 18:03:16.212944 ip-10-0-128-39 kubenswrapper[2558]: I0710 18:03:16.212880 2558 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-hps6m" event={"ID":"cdee596a-a663-4627-8c46-d7066d21978b","Type":"ContainerStarted","Data":"ecd09ed705542831172a088ab0b7a833b6546a8cfa7d784c0c33d39406e7fbea"} Jul 10 18:03:17.095214 ip-10-0-128-39 kubenswrapper[2558]: I0710 18:03:17.094978 2558 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-vsbdm" Jul 10 18:03:17.095356 ip-10-0-128-39 kubenswrapper[2558]: E0710 18:03:17.095259 2558 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-vsbdm" podUID="3b27429f-90aa-4170-9078-0aa4ea0d0ab8" Jul 10 18:03:17.219912 ip-10-0-128-39 kubenswrapper[2558]: I0710 18:03:17.219884 2558 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ovn-kubernetes_ovnkube-node-ll44q_eabc6c69-b1e8-40d5-b41f-f49778a5248c/ovn-acl-logging/0.log" Jul 10 18:03:17.220407 ip-10-0-128-39 kubenswrapper[2558]: I0710 18:03:17.220376 2558 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ovn-kubernetes/ovnkube-node-ll44q" event={"ID":"eabc6c69-b1e8-40d5-b41f-f49778a5248c","Type":"ContainerStarted","Data":"7ac8a8c1dcf280a8b74ab31697d1be74409b89f18afa439ae969751cec791105"} Jul 10 18:03:17.222828 ip-10-0-128-39 kubenswrapper[2558]: I0710 18:03:17.222783 2558 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-hps6m" event={"ID":"cdee596a-a663-4627-8c46-d7066d21978b","Type":"ContainerStarted","Data":"953c8ffba7cd015d2eab36063551f3a2f6ede15012283cc9641f7d2452b3b135"} Jul 10 18:03:17.242291 ip-10-0-128-39 kubenswrapper[2558]: I0710 18:03:17.242238 2558 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-hps6m" podStartSLOduration=3.463368022 podStartE2EDuration="25.242223679s" totalImagesPullingTime="21.778855657s" totalInitContainerRuntime="0s" isStatefulPod=true podCreationTimestamp="2026-07-10 18:02:52 +0000 UTC" imagePullSessionsCount=3 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-10 18:03:17.240973605 +0000 UTC m=+25.744466111" watchObservedRunningTime="2026-07-10 18:03:17.242223679 +0000 UTC m=+25.745716185" Jul 10 18:03:18.095314 ip-10-0-128-39 kubenswrapper[2558]: I0710 18:03:18.095276 2558 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-jkr77" Jul 10 18:03:18.095540 ip-10-0-128-39 kubenswrapper[2558]: I0710 18:03:18.095276 2558 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-rzfgn" Jul 10 18:03:18.095540 ip-10-0-128-39 kubenswrapper[2558]: E0710 18:03:18.095442 2558 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-jkr77" podUID="7fa20252-06b7-48e4-8620-3e52dc980486" Jul 10 18:03:18.095540 ip-10-0-128-39 kubenswrapper[2558]: E0710 18:03:18.095463 2558 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-rzfgn" podUID="06aa905d-9b4b-4d36-bafc-8fa0c11b3057" Jul 10 18:03:19.095365 ip-10-0-128-39 kubenswrapper[2558]: I0710 18:03:19.095326 2558 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-vsbdm" Jul 10 18:03:19.096092 ip-10-0-128-39 kubenswrapper[2558]: E0710 18:03:19.095455 2558 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-vsbdm" podUID="3b27429f-90aa-4170-9078-0aa4ea0d0ab8" Jul 10 18:03:20.095908 ip-10-0-128-39 kubenswrapper[2558]: I0710 18:03:20.095713 2558 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-jkr77" Jul 10 18:03:20.096548 ip-10-0-128-39 kubenswrapper[2558]: I0710 18:03:20.095764 2558 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-rzfgn" Jul 10 18:03:20.096548 ip-10-0-128-39 kubenswrapper[2558]: E0710 18:03:20.095991 2558 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-jkr77" podUID="7fa20252-06b7-48e4-8620-3e52dc980486" Jul 10 18:03:20.096548 ip-10-0-128-39 kubenswrapper[2558]: E0710 18:03:20.096048 2558 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-rzfgn" podUID="06aa905d-9b4b-4d36-bafc-8fa0c11b3057" Jul 10 18:03:20.233383 ip-10-0-128-39 kubenswrapper[2558]: I0710 18:03:20.233357 2558 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ovn-kubernetes_ovnkube-node-ll44q_eabc6c69-b1e8-40d5-b41f-f49778a5248c/ovn-acl-logging/0.log" Jul 10 18:03:20.233749 ip-10-0-128-39 kubenswrapper[2558]: I0710 18:03:20.233721 2558 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ovn-kubernetes/ovnkube-node-ll44q" event={"ID":"eabc6c69-b1e8-40d5-b41f-f49778a5248c","Type":"ContainerStarted","Data":"43175d8a12551d9f0d216a4d8f7d8dcf94721648f3d850ccc16cf6c1f01cd554"} Jul 10 18:03:20.234004 ip-10-0-128-39 kubenswrapper[2558]: I0710 18:03:20.233983 2558 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ovn-kubernetes/ovnkube-node-ll44q" Jul 10 18:03:20.234195 ip-10-0-128-39 kubenswrapper[2558]: I0710 18:03:20.234179 2558 scope.go:117] "RemoveContainer" containerID="1a956f3831339136536286c61a717aa148ef39246aa0031110ff6f337b8fa9ee" Jul 10 18:03:20.235684 ip-10-0-128-39 kubenswrapper[2558]: I0710 18:03:20.235659 2558 generic.go:356] "Generic (PLEG): container finished" podID="53492c2f-2433-4658-af05-d58878aed082" containerID="3ce629d860b0609d9da48ba9d49cc09877efdf4f748503a0b4c38f614002b4b0" exitCode=0 Jul 10 18:03:20.235759 ip-10-0-128-39 kubenswrapper[2558]: I0710 18:03:20.235697 2558 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-multus/multus-additional-cni-plugins-v22pb" event={"ID":"53492c2f-2433-4658-af05-d58878aed082","Type":"ContainerDied","Data":"3ce629d860b0609d9da48ba9d49cc09877efdf4f748503a0b4c38f614002b4b0"} Jul 10 18:03:20.250268 ip-10-0-128-39 kubenswrapper[2558]: I0710 18:03:20.250239 2558 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ovn-kubernetes/ovnkube-node-ll44q" Jul 10 18:03:21.095338 ip-10-0-128-39 kubenswrapper[2558]: I0710 18:03:21.095300 2558 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-vsbdm" Jul 10 18:03:21.095669 ip-10-0-128-39 kubenswrapper[2558]: E0710 18:03:21.095438 2558 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-vsbdm" podUID="3b27429f-90aa-4170-9078-0aa4ea0d0ab8" Jul 10 18:03:21.241703 ip-10-0-128-39 kubenswrapper[2558]: I0710 18:03:21.241679 2558 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ovn-kubernetes_ovnkube-node-ll44q_eabc6c69-b1e8-40d5-b41f-f49778a5248c/ovn-acl-logging/0.log" Jul 10 18:03:21.242193 ip-10-0-128-39 kubenswrapper[2558]: I0710 18:03:21.242030 2558 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ovn-kubernetes/ovnkube-node-ll44q" event={"ID":"eabc6c69-b1e8-40d5-b41f-f49778a5248c","Type":"ContainerStarted","Data":"374ee226e0a7c881ac3da6f346b094a8c25099b35489dc58cb10416d6dd8d26d"} Jul 10 18:03:21.242408 ip-10-0-128-39 kubenswrapper[2558]: I0710 18:03:21.242382 2558 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ovn-kubernetes/ovnkube-node-ll44q" Jul 10 18:03:21.242559 ip-10-0-128-39 kubenswrapper[2558]: I0710 18:03:21.242539 2558 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ovn-kubernetes/ovnkube-node-ll44q" Jul 10 18:03:21.257311 ip-10-0-128-39 kubenswrapper[2558]: I0710 18:03:21.257286 2558 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ovn-kubernetes/ovnkube-node-ll44q" Jul 10 18:03:21.268796 ip-10-0-128-39 kubenswrapper[2558]: I0710 18:03:21.268749 2558 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-ovn-kubernetes/ovnkube-node-ll44q" podStartSLOduration=9.727742183 podStartE2EDuration="29.268734459s" totalImagesPullingTime="19.540992276s" totalInitContainerRuntime="0s" isStatefulPod=true podCreationTimestamp="2026-07-10 18:02:52 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-10 18:03:21.267322336 +0000 UTC m=+29.770814841" watchObservedRunningTime="2026-07-10 18:03:21.268734459 +0000 UTC m=+29.772226964" Jul 10 18:03:21.783192 ip-10-0-128-39 kubenswrapper[2558]: I0710 18:03:21.783107 2558 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kube-system/global-pull-secret-syncer-vsbdm"] Jul 10 18:03:21.783350 ip-10-0-128-39 kubenswrapper[2558]: I0710 18:03:21.783217 2558 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-vsbdm" Jul 10 18:03:21.783350 ip-10-0-128-39 kubenswrapper[2558]: E0710 18:03:21.783307 2558 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-vsbdm" podUID="3b27429f-90aa-4170-9078-0aa4ea0d0ab8" Jul 10 18:03:21.787027 ip-10-0-128-39 kubenswrapper[2558]: I0710 18:03:21.787001 2558 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-multus/network-metrics-daemon-jkr77"] Jul 10 18:03:21.787159 ip-10-0-128-39 kubenswrapper[2558]: I0710 18:03:21.787123 2558 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-jkr77" Jul 10 18:03:21.787262 ip-10-0-128-39 kubenswrapper[2558]: E0710 18:03:21.787237 2558 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-jkr77" podUID="7fa20252-06b7-48e4-8620-3e52dc980486" Jul 10 18:03:21.787590 ip-10-0-128-39 kubenswrapper[2558]: I0710 18:03:21.787570 2558 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-network-diagnostics/network-check-target-rzfgn"] Jul 10 18:03:21.787685 ip-10-0-128-39 kubenswrapper[2558]: I0710 18:03:21.787652 2558 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-rzfgn" Jul 10 18:03:21.787744 ip-10-0-128-39 kubenswrapper[2558]: E0710 18:03:21.787726 2558 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-rzfgn" podUID="06aa905d-9b4b-4d36-bafc-8fa0c11b3057" Jul 10 18:03:22.246587 ip-10-0-128-39 kubenswrapper[2558]: I0710 18:03:22.246397 2558 generic.go:356] "Generic (PLEG): container finished" podID="53492c2f-2433-4658-af05-d58878aed082" containerID="8d2010edf24023385329503943a76820e0231d438fe2a27c0b61faff6eaf1d6a" exitCode=0 Jul 10 18:03:22.247046 ip-10-0-128-39 kubenswrapper[2558]: I0710 18:03:22.246477 2558 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-multus/multus-additional-cni-plugins-v22pb" event={"ID":"53492c2f-2433-4658-af05-d58878aed082","Type":"ContainerDied","Data":"8d2010edf24023385329503943a76820e0231d438fe2a27c0b61faff6eaf1d6a"} Jul 10 18:03:23.094898 ip-10-0-128-39 kubenswrapper[2558]: I0710 18:03:23.094870 2558 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-rzfgn" Jul 10 18:03:23.095035 ip-10-0-128-39 kubenswrapper[2558]: I0710 18:03:23.094870 2558 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-vsbdm" Jul 10 18:03:23.095035 ip-10-0-128-39 kubenswrapper[2558]: E0710 18:03:23.094990 2558 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-rzfgn" podUID="06aa905d-9b4b-4d36-bafc-8fa0c11b3057" Jul 10 18:03:23.095099 ip-10-0-128-39 kubenswrapper[2558]: E0710 18:03:23.095033 2558 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-vsbdm" podUID="3b27429f-90aa-4170-9078-0aa4ea0d0ab8" Jul 10 18:03:24.095311 ip-10-0-128-39 kubenswrapper[2558]: I0710 18:03:24.095280 2558 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-jkr77" Jul 10 18:03:24.095929 ip-10-0-128-39 kubenswrapper[2558]: E0710 18:03:24.095392 2558 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-jkr77" podUID="7fa20252-06b7-48e4-8620-3e52dc980486" Jul 10 18:03:24.252536 ip-10-0-128-39 kubenswrapper[2558]: I0710 18:03:24.252506 2558 generic.go:356] "Generic (PLEG): container finished" podID="53492c2f-2433-4658-af05-d58878aed082" containerID="b7af2c0250fe22faf7abf756298acd7cd6f5a5a478f9d8b1a8b49bc9ab87ebad" exitCode=0 Jul 10 18:03:24.252685 ip-10-0-128-39 kubenswrapper[2558]: I0710 18:03:24.252549 2558 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-multus/multus-additional-cni-plugins-v22pb" event={"ID":"53492c2f-2433-4658-af05-d58878aed082","Type":"ContainerDied","Data":"b7af2c0250fe22faf7abf756298acd7cd6f5a5a478f9d8b1a8b49bc9ab87ebad"} Jul 10 18:03:25.095352 ip-10-0-128-39 kubenswrapper[2558]: I0710 18:03:25.095317 2558 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-vsbdm" Jul 10 18:03:25.095801 ip-10-0-128-39 kubenswrapper[2558]: I0710 18:03:25.095324 2558 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-rzfgn" Jul 10 18:03:25.095801 ip-10-0-128-39 kubenswrapper[2558]: E0710 18:03:25.095434 2558 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-vsbdm" podUID="3b27429f-90aa-4170-9078-0aa4ea0d0ab8" Jul 10 18:03:25.095801 ip-10-0-128-39 kubenswrapper[2558]: E0710 18:03:25.095533 2558 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-rzfgn" podUID="06aa905d-9b4b-4d36-bafc-8fa0c11b3057" Jul 10 18:03:25.742669 ip-10-0-128-39 kubenswrapper[2558]: I0710 18:03:25.742635 2558 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-h74dt\" (UniqueName: \"kubernetes.io/projected/06aa905d-9b4b-4d36-bafc-8fa0c11b3057-kube-api-access-h74dt\") pod \"network-check-target-rzfgn\" (UID: \"06aa905d-9b4b-4d36-bafc-8fa0c11b3057\") " pod="openshift-network-diagnostics/network-check-target-rzfgn" Jul 10 18:03:25.742874 ip-10-0-128-39 kubenswrapper[2558]: I0710 18:03:25.742688 2558 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"original-pull-secret\" (UniqueName: \"kubernetes.io/secret/3b27429f-90aa-4170-9078-0aa4ea0d0ab8-original-pull-secret\") pod \"global-pull-secret-syncer-vsbdm\" (UID: \"3b27429f-90aa-4170-9078-0aa4ea0d0ab8\") " pod="kube-system/global-pull-secret-syncer-vsbdm" Jul 10 18:03:25.742874 ip-10-0-128-39 kubenswrapper[2558]: I0710 18:03:25.742723 2558 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/7fa20252-06b7-48e4-8620-3e52dc980486-metrics-certs\") pod \"network-metrics-daemon-jkr77\" (UID: \"7fa20252-06b7-48e4-8620-3e52dc980486\") " pod="openshift-multus/network-metrics-daemon-jkr77" Jul 10 18:03:25.742874 ip-10-0-128-39 kubenswrapper[2558]: E0710 18:03:25.742832 2558 projected.go:291] Couldn't get configMap openshift-network-diagnostics/kube-root-ca.crt: object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered Jul 10 18:03:25.742874 ip-10-0-128-39 kubenswrapper[2558]: E0710 18:03:25.742854 2558 secret.go:189] Couldn't get secret openshift-multus/metrics-daemon-secret: object "openshift-multus"/"metrics-daemon-secret" not registered Jul 10 18:03:25.743073 ip-10-0-128-39 kubenswrapper[2558]: E0710 18:03:25.742888 2558 secret.go:189] Couldn't get secret kube-system/original-pull-secret: object "kube-system"/"original-pull-secret" not registered Jul 10 18:03:25.743073 ip-10-0-128-39 kubenswrapper[2558]: E0710 18:03:25.742904 2558 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/7fa20252-06b7-48e4-8620-3e52dc980486-metrics-certs podName:7fa20252-06b7-48e4-8620-3e52dc980486 nodeName:}" failed. No retries permitted until 2026-07-10 18:03:57.74289066 +0000 UTC m=+66.246383144 (durationBeforeRetry 32s). Error: MountVolume.SetUp failed for volume "metrics-certs" (UniqueName: "kubernetes.io/secret/7fa20252-06b7-48e4-8620-3e52dc980486-metrics-certs") pod "network-metrics-daemon-jkr77" (UID: "7fa20252-06b7-48e4-8620-3e52dc980486") : object "openshift-multus"/"metrics-daemon-secret" not registered Jul 10 18:03:25.743073 ip-10-0-128-39 kubenswrapper[2558]: E0710 18:03:25.742856 2558 projected.go:291] Couldn't get configMap openshift-network-diagnostics/openshift-service-ca.crt: object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered Jul 10 18:03:25.743073 ip-10-0-128-39 kubenswrapper[2558]: E0710 18:03:25.742946 2558 projected.go:196] Error preparing data for projected volume kube-api-access-h74dt for pod openshift-network-diagnostics/network-check-target-rzfgn: [object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered, object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered] Jul 10 18:03:25.743073 ip-10-0-128-39 kubenswrapper[2558]: E0710 18:03:25.742950 2558 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/3b27429f-90aa-4170-9078-0aa4ea0d0ab8-original-pull-secret podName:3b27429f-90aa-4170-9078-0aa4ea0d0ab8 nodeName:}" failed. No retries permitted until 2026-07-10 18:03:57.742932543 +0000 UTC m=+66.246425033 (durationBeforeRetry 32s). Error: MountVolume.SetUp failed for volume "original-pull-secret" (UniqueName: "kubernetes.io/secret/3b27429f-90aa-4170-9078-0aa4ea0d0ab8-original-pull-secret") pod "global-pull-secret-syncer-vsbdm" (UID: "3b27429f-90aa-4170-9078-0aa4ea0d0ab8") : object "kube-system"/"original-pull-secret" not registered Jul 10 18:03:25.743073 ip-10-0-128-39 kubenswrapper[2558]: E0710 18:03:25.742983 2558 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/projected/06aa905d-9b4b-4d36-bafc-8fa0c11b3057-kube-api-access-h74dt podName:06aa905d-9b4b-4d36-bafc-8fa0c11b3057 nodeName:}" failed. No retries permitted until 2026-07-10 18:03:57.742971284 +0000 UTC m=+66.246463779 (durationBeforeRetry 32s). Error: MountVolume.SetUp failed for volume "kube-api-access-h74dt" (UniqueName: "kubernetes.io/projected/06aa905d-9b4b-4d36-bafc-8fa0c11b3057-kube-api-access-h74dt") pod "network-check-target-rzfgn" (UID: "06aa905d-9b4b-4d36-bafc-8fa0c11b3057") : [object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered, object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered] Jul 10 18:03:26.095856 ip-10-0-128-39 kubenswrapper[2558]: I0710 18:03:26.095760 2558 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-jkr77" Jul 10 18:03:26.096274 ip-10-0-128-39 kubenswrapper[2558]: E0710 18:03:26.095924 2558 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-jkr77" podUID="7fa20252-06b7-48e4-8620-3e52dc980486" Jul 10 18:03:26.776051 ip-10-0-128-39 kubenswrapper[2558]: I0710 18:03:26.776025 2558 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-128-39.ec2.internal" event="NodeReady" Jul 10 18:03:26.776231 ip-10-0-128-39 kubenswrapper[2558]: I0710 18:03:26.776165 2558 kubelet_node_status.go:489] "Fast updating node status as it just became ready" Jul 10 18:03:26.818372 ip-10-0-128-39 kubenswrapper[2558]: I0710 18:03:26.818339 2558 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-image-registry/image-registry-cf9777cd6-d9zpc"] Jul 10 18:03:26.820512 ip-10-0-128-39 kubenswrapper[2558]: I0710 18:03:26.820462 2558 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-image-registry/image-registry-cf9777cd6-d9zpc" Jul 10 18:03:26.822984 ip-10-0-128-39 kubenswrapper[2558]: I0710 18:03:26.822931 2558 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-image-registry\"/\"image-registry-private-configuration\"" Jul 10 18:03:26.822984 ip-10-0-128-39 kubenswrapper[2558]: I0710 18:03:26.822955 2558 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-image-registry\"/\"installation-pull-secrets\"" Jul 10 18:03:26.823227 ip-10-0-128-39 kubenswrapper[2558]: I0710 18:03:26.823204 2558 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-image-registry\"/\"registry-dockercfg-g4gf6\"" Jul 10 18:03:26.823345 ip-10-0-128-39 kubenswrapper[2558]: I0710 18:03:26.823328 2558 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-image-registry\"/\"image-registry-tls\"" Jul 10 18:03:26.830722 ip-10-0-128-39 kubenswrapper[2558]: I0710 18:03:26.830627 2558 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-image-registry\"/\"trusted-ca\"" Jul 10 18:03:26.834199 ip-10-0-128-39 kubenswrapper[2558]: I0710 18:03:26.834172 2558 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-ingress-canary/ingress-canary-bpwbf"] Jul 10 18:03:26.836117 ip-10-0-128-39 kubenswrapper[2558]: I0710 18:03:26.836098 2558 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-dns/dns-default-22khc"] Jul 10 18:03:26.836267 ip-10-0-128-39 kubenswrapper[2558]: I0710 18:03:26.836229 2558 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress-canary/ingress-canary-bpwbf" Jul 10 18:03:26.838646 ip-10-0-128-39 kubenswrapper[2558]: I0710 18:03:26.838626 2558 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-image-registry/image-registry-cf9777cd6-d9zpc"] Jul 10 18:03:26.838746 ip-10-0-128-39 kubenswrapper[2558]: I0710 18:03:26.838732 2558 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-dns/dns-default-22khc" Jul 10 18:03:26.839881 ip-10-0-128-39 kubenswrapper[2558]: I0710 18:03:26.839860 2558 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress-canary\"/\"canary-serving-cert\"" Jul 10 18:03:26.839881 ip-10-0-128-39 kubenswrapper[2558]: I0710 18:03:26.839874 2558 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ingress-canary\"/\"openshift-service-ca.crt\"" Jul 10 18:03:26.840050 ip-10-0-128-39 kubenswrapper[2558]: I0710 18:03:26.839926 2558 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress-canary\"/\"default-dockercfg-57cv4\"" Jul 10 18:03:26.840831 ip-10-0-128-39 kubenswrapper[2558]: I0710 18:03:26.840186 2558 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ingress-canary\"/\"kube-root-ca.crt\"" Jul 10 18:03:26.841136 ip-10-0-128-39 kubenswrapper[2558]: I0710 18:03:26.841117 2558 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-dns\"/\"dns-default-metrics-tls\"" Jul 10 18:03:26.841136 ip-10-0-128-39 kubenswrapper[2558]: I0710 18:03:26.841127 2558 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-dns\"/\"dns-dockercfg-4kztg\"" Jul 10 18:03:26.841438 ip-10-0-128-39 kubenswrapper[2558]: I0710 18:03:26.841421 2558 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-dns\"/\"dns-default\"" Jul 10 18:03:26.846112 ip-10-0-128-39 kubenswrapper[2558]: I0710 18:03:26.846089 2558 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress-canary/ingress-canary-bpwbf"] Jul 10 18:03:26.854008 ip-10-0-128-39 kubenswrapper[2558]: I0710 18:03:26.853988 2558 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-dns/dns-default-22khc"] Jul 10 18:03:26.951989 ip-10-0-128-39 kubenswrapper[2558]: I0710 18:03:26.951948 2558 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-fmt6b\" (UniqueName: \"kubernetes.io/projected/15c4360f-6bc7-4948-b0e6-ebc018174856-kube-api-access-fmt6b\") pod \"dns-default-22khc\" (UID: \"15c4360f-6bc7-4948-b0e6-ebc018174856\") " pod="openshift-dns/dns-default-22khc" Jul 10 18:03:26.951989 ip-10-0-128-39 kubenswrapper[2558]: I0710 18:03:26.951986 2558 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"registry-tls\" (UniqueName: \"kubernetes.io/projected/4b133a69-01d6-42b5-b12f-d063856c87cb-registry-tls\") pod \"image-registry-cf9777cd6-d9zpc\" (UID: \"4b133a69-01d6-42b5-b12f-d063856c87cb\") " pod="openshift-image-registry/image-registry-cf9777cd6-d9zpc" Jul 10 18:03:26.952193 ip-10-0-128-39 kubenswrapper[2558]: I0710 18:03:26.952007 2558 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"installation-pull-secrets\" (UniqueName: \"kubernetes.io/secret/4b133a69-01d6-42b5-b12f-d063856c87cb-installation-pull-secrets\") pod \"image-registry-cf9777cd6-d9zpc\" (UID: \"4b133a69-01d6-42b5-b12f-d063856c87cb\") " pod="openshift-image-registry/image-registry-cf9777cd6-d9zpc" Jul 10 18:03:26.952193 ip-10-0-128-39 kubenswrapper[2558]: I0710 18:03:26.952031 2558 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"bound-sa-token\" (UniqueName: \"kubernetes.io/projected/4b133a69-01d6-42b5-b12f-d063856c87cb-bound-sa-token\") pod \"image-registry-cf9777cd6-d9zpc\" (UID: \"4b133a69-01d6-42b5-b12f-d063856c87cb\") " pod="openshift-image-registry/image-registry-cf9777cd6-d9zpc" Jul 10 18:03:26.952193 ip-10-0-128-39 kubenswrapper[2558]: I0710 18:03:26.952135 2558 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-w6rfg\" (UniqueName: \"kubernetes.io/projected/f754e509-abbe-4451-833d-abb0e28c7e19-kube-api-access-w6rfg\") pod \"ingress-canary-bpwbf\" (UID: \"f754e509-abbe-4451-833d-abb0e28c7e19\") " pod="openshift-ingress-canary/ingress-canary-bpwbf" Jul 10 18:03:26.952193 ip-10-0-128-39 kubenswrapper[2558]: I0710 18:03:26.952165 2558 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"ca-trust-extracted\" (UniqueName: \"kubernetes.io/empty-dir/4b133a69-01d6-42b5-b12f-d063856c87cb-ca-trust-extracted\") pod \"image-registry-cf9777cd6-d9zpc\" (UID: \"4b133a69-01d6-42b5-b12f-d063856c87cb\") " pod="openshift-image-registry/image-registry-cf9777cd6-d9zpc" Jul 10 18:03:26.952193 ip-10-0-128-39 kubenswrapper[2558]: I0710 18:03:26.952183 2558 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-9kb59\" (UniqueName: \"kubernetes.io/projected/4b133a69-01d6-42b5-b12f-d063856c87cb-kube-api-access-9kb59\") pod \"image-registry-cf9777cd6-d9zpc\" (UID: \"4b133a69-01d6-42b5-b12f-d063856c87cb\") " pod="openshift-image-registry/image-registry-cf9777cd6-d9zpc" Jul 10 18:03:26.952419 ip-10-0-128-39 kubenswrapper[2558]: I0710 18:03:26.952212 2558 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"trusted-ca\" (UniqueName: \"kubernetes.io/configmap/4b133a69-01d6-42b5-b12f-d063856c87cb-trusted-ca\") pod \"image-registry-cf9777cd6-d9zpc\" (UID: \"4b133a69-01d6-42b5-b12f-d063856c87cb\") " pod="openshift-image-registry/image-registry-cf9777cd6-d9zpc" Jul 10 18:03:26.952419 ip-10-0-128-39 kubenswrapper[2558]: I0710 18:03:26.952262 2558 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"cert\" (UniqueName: \"kubernetes.io/secret/f754e509-abbe-4451-833d-abb0e28c7e19-cert\") pod \"ingress-canary-bpwbf\" (UID: \"f754e509-abbe-4451-833d-abb0e28c7e19\") " pod="openshift-ingress-canary/ingress-canary-bpwbf" Jul 10 18:03:26.952419 ip-10-0-128-39 kubenswrapper[2558]: I0710 18:03:26.952302 2558 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"metrics-tls\" (UniqueName: \"kubernetes.io/secret/15c4360f-6bc7-4948-b0e6-ebc018174856-metrics-tls\") pod \"dns-default-22khc\" (UID: \"15c4360f-6bc7-4948-b0e6-ebc018174856\") " pod="openshift-dns/dns-default-22khc" Jul 10 18:03:26.952419 ip-10-0-128-39 kubenswrapper[2558]: I0710 18:03:26.952316 2558 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/15c4360f-6bc7-4948-b0e6-ebc018174856-tmp-dir\") pod \"dns-default-22khc\" (UID: \"15c4360f-6bc7-4948-b0e6-ebc018174856\") " pod="openshift-dns/dns-default-22khc" Jul 10 18:03:26.952419 ip-10-0-128-39 kubenswrapper[2558]: I0710 18:03:26.952379 2558 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"config-volume\" (UniqueName: \"kubernetes.io/configmap/15c4360f-6bc7-4948-b0e6-ebc018174856-config-volume\") pod \"dns-default-22khc\" (UID: \"15c4360f-6bc7-4948-b0e6-ebc018174856\") " pod="openshift-dns/dns-default-22khc" Jul 10 18:03:26.952601 ip-10-0-128-39 kubenswrapper[2558]: I0710 18:03:26.952420 2558 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"image-registry-private-configuration\" (UniqueName: \"kubernetes.io/secret/4b133a69-01d6-42b5-b12f-d063856c87cb-image-registry-private-configuration\") pod \"image-registry-cf9777cd6-d9zpc\" (UID: \"4b133a69-01d6-42b5-b12f-d063856c87cb\") " pod="openshift-image-registry/image-registry-cf9777cd6-d9zpc" Jul 10 18:03:26.952601 ip-10-0-128-39 kubenswrapper[2558]: I0710 18:03:26.952450 2558 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"registry-certificates\" (UniqueName: \"kubernetes.io/configmap/4b133a69-01d6-42b5-b12f-d063856c87cb-registry-certificates\") pod \"image-registry-cf9777cd6-d9zpc\" (UID: \"4b133a69-01d6-42b5-b12f-d063856c87cb\") " pod="openshift-image-registry/image-registry-cf9777cd6-d9zpc" Jul 10 18:03:26.978734 ip-10-0-128-39 kubenswrapper[2558]: I0710 18:03:26.978681 2558 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-dvp86_2da19865-b021-4789-b1e0-694d4f6c543d/konnectivity-agent/0.log" limit=1024 Jul 10 18:03:27.053769 ip-10-0-128-39 kubenswrapper[2558]: I0710 18:03:27.053682 2558 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-tls\" (UniqueName: \"kubernetes.io/secret/15c4360f-6bc7-4948-b0e6-ebc018174856-metrics-tls\") pod \"dns-default-22khc\" (UID: \"15c4360f-6bc7-4948-b0e6-ebc018174856\") " pod="openshift-dns/dns-default-22khc" Jul 10 18:03:27.053769 ip-10-0-128-39 kubenswrapper[2558]: I0710 18:03:27.053727 2558 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/15c4360f-6bc7-4948-b0e6-ebc018174856-tmp-dir\") pod \"dns-default-22khc\" (UID: \"15c4360f-6bc7-4948-b0e6-ebc018174856\") " pod="openshift-dns/dns-default-22khc" Jul 10 18:03:27.053769 ip-10-0-128-39 kubenswrapper[2558]: I0710 18:03:27.053764 2558 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"config-volume\" (UniqueName: \"kubernetes.io/configmap/15c4360f-6bc7-4948-b0e6-ebc018174856-config-volume\") pod \"dns-default-22khc\" (UID: \"15c4360f-6bc7-4948-b0e6-ebc018174856\") " pod="openshift-dns/dns-default-22khc" Jul 10 18:03:27.054050 ip-10-0-128-39 kubenswrapper[2558]: I0710 18:03:27.053785 2558 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"image-registry-private-configuration\" (UniqueName: \"kubernetes.io/secret/4b133a69-01d6-42b5-b12f-d063856c87cb-image-registry-private-configuration\") pod \"image-registry-cf9777cd6-d9zpc\" (UID: \"4b133a69-01d6-42b5-b12f-d063856c87cb\") " pod="openshift-image-registry/image-registry-cf9777cd6-d9zpc" Jul 10 18:03:27.054050 ip-10-0-128-39 kubenswrapper[2558]: I0710 18:03:27.053827 2558 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"registry-certificates\" (UniqueName: \"kubernetes.io/configmap/4b133a69-01d6-42b5-b12f-d063856c87cb-registry-certificates\") pod \"image-registry-cf9777cd6-d9zpc\" (UID: \"4b133a69-01d6-42b5-b12f-d063856c87cb\") " pod="openshift-image-registry/image-registry-cf9777cd6-d9zpc" Jul 10 18:03:27.054050 ip-10-0-128-39 kubenswrapper[2558]: I0710 18:03:27.053856 2558 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-fmt6b\" (UniqueName: \"kubernetes.io/projected/15c4360f-6bc7-4948-b0e6-ebc018174856-kube-api-access-fmt6b\") pod \"dns-default-22khc\" (UID: \"15c4360f-6bc7-4948-b0e6-ebc018174856\") " pod="openshift-dns/dns-default-22khc" Jul 10 18:03:27.054050 ip-10-0-128-39 kubenswrapper[2558]: E0710 18:03:27.053856 2558 secret.go:189] Couldn't get secret openshift-dns/dns-default-metrics-tls: secret "dns-default-metrics-tls" not found Jul 10 18:03:27.054050 ip-10-0-128-39 kubenswrapper[2558]: I0710 18:03:27.053882 2558 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"registry-tls\" (UniqueName: \"kubernetes.io/projected/4b133a69-01d6-42b5-b12f-d063856c87cb-registry-tls\") pod \"image-registry-cf9777cd6-d9zpc\" (UID: \"4b133a69-01d6-42b5-b12f-d063856c87cb\") " pod="openshift-image-registry/image-registry-cf9777cd6-d9zpc" Jul 10 18:03:27.054050 ip-10-0-128-39 kubenswrapper[2558]: I0710 18:03:27.053905 2558 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"installation-pull-secrets\" (UniqueName: \"kubernetes.io/secret/4b133a69-01d6-42b5-b12f-d063856c87cb-installation-pull-secrets\") pod \"image-registry-cf9777cd6-d9zpc\" (UID: \"4b133a69-01d6-42b5-b12f-d063856c87cb\") " pod="openshift-image-registry/image-registry-cf9777cd6-d9zpc" Jul 10 18:03:27.054050 ip-10-0-128-39 kubenswrapper[2558]: E0710 18:03:27.053942 2558 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/15c4360f-6bc7-4948-b0e6-ebc018174856-metrics-tls podName:15c4360f-6bc7-4948-b0e6-ebc018174856 nodeName:}" failed. No retries permitted until 2026-07-10 18:03:27.553921982 +0000 UTC m=+36.057414483 (durationBeforeRetry 500ms). Error: MountVolume.SetUp failed for volume "metrics-tls" (UniqueName: "kubernetes.io/secret/15c4360f-6bc7-4948-b0e6-ebc018174856-metrics-tls") pod "dns-default-22khc" (UID: "15c4360f-6bc7-4948-b0e6-ebc018174856") : secret "dns-default-metrics-tls" not found Jul 10 18:03:27.054050 ip-10-0-128-39 kubenswrapper[2558]: I0710 18:03:27.053966 2558 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"bound-sa-token\" (UniqueName: \"kubernetes.io/projected/4b133a69-01d6-42b5-b12f-d063856c87cb-bound-sa-token\") pod \"image-registry-cf9777cd6-d9zpc\" (UID: \"4b133a69-01d6-42b5-b12f-d063856c87cb\") " pod="openshift-image-registry/image-registry-cf9777cd6-d9zpc" Jul 10 18:03:27.054050 ip-10-0-128-39 kubenswrapper[2558]: I0710 18:03:27.054016 2558 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-w6rfg\" (UniqueName: \"kubernetes.io/projected/f754e509-abbe-4451-833d-abb0e28c7e19-kube-api-access-w6rfg\") pod \"ingress-canary-bpwbf\" (UID: \"f754e509-abbe-4451-833d-abb0e28c7e19\") " pod="openshift-ingress-canary/ingress-canary-bpwbf" Jul 10 18:03:27.054050 ip-10-0-128-39 kubenswrapper[2558]: I0710 18:03:27.054040 2558 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"ca-trust-extracted\" (UniqueName: \"kubernetes.io/empty-dir/4b133a69-01d6-42b5-b12f-d063856c87cb-ca-trust-extracted\") pod \"image-registry-cf9777cd6-d9zpc\" (UID: \"4b133a69-01d6-42b5-b12f-d063856c87cb\") " pod="openshift-image-registry/image-registry-cf9777cd6-d9zpc" Jul 10 18:03:27.054603 ip-10-0-128-39 kubenswrapper[2558]: I0710 18:03:27.054065 2558 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-9kb59\" (UniqueName: \"kubernetes.io/projected/4b133a69-01d6-42b5-b12f-d063856c87cb-kube-api-access-9kb59\") pod \"image-registry-cf9777cd6-d9zpc\" (UID: \"4b133a69-01d6-42b5-b12f-d063856c87cb\") " pod="openshift-image-registry/image-registry-cf9777cd6-d9zpc" Jul 10 18:03:27.054603 ip-10-0-128-39 kubenswrapper[2558]: I0710 18:03:27.054104 2558 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"trusted-ca\" (UniqueName: \"kubernetes.io/configmap/4b133a69-01d6-42b5-b12f-d063856c87cb-trusted-ca\") pod \"image-registry-cf9777cd6-d9zpc\" (UID: \"4b133a69-01d6-42b5-b12f-d063856c87cb\") " pod="openshift-image-registry/image-registry-cf9777cd6-d9zpc" Jul 10 18:03:27.054603 ip-10-0-128-39 kubenswrapper[2558]: I0710 18:03:27.054152 2558 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"cert\" (UniqueName: \"kubernetes.io/secret/f754e509-abbe-4451-833d-abb0e28c7e19-cert\") pod \"ingress-canary-bpwbf\" (UID: \"f754e509-abbe-4451-833d-abb0e28c7e19\") " pod="openshift-ingress-canary/ingress-canary-bpwbf" Jul 10 18:03:27.054603 ip-10-0-128-39 kubenswrapper[2558]: I0710 18:03:27.054201 2558 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/15c4360f-6bc7-4948-b0e6-ebc018174856-tmp-dir\") pod \"dns-default-22khc\" (UID: \"15c4360f-6bc7-4948-b0e6-ebc018174856\") " pod="openshift-dns/dns-default-22khc" Jul 10 18:03:27.054603 ip-10-0-128-39 kubenswrapper[2558]: I0710 18:03:27.054422 2558 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"registry-certificates\" (UniqueName: \"kubernetes.io/configmap/4b133a69-01d6-42b5-b12f-d063856c87cb-registry-certificates\") pod \"image-registry-cf9777cd6-d9zpc\" (UID: \"4b133a69-01d6-42b5-b12f-d063856c87cb\") " pod="openshift-image-registry/image-registry-cf9777cd6-d9zpc" Jul 10 18:03:27.054603 ip-10-0-128-39 kubenswrapper[2558]: I0710 18:03:27.054460 2558 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"config-volume\" (UniqueName: \"kubernetes.io/configmap/15c4360f-6bc7-4948-b0e6-ebc018174856-config-volume\") pod \"dns-default-22khc\" (UID: \"15c4360f-6bc7-4948-b0e6-ebc018174856\") " pod="openshift-dns/dns-default-22khc" Jul 10 18:03:27.054603 ip-10-0-128-39 kubenswrapper[2558]: E0710 18:03:27.054552 2558 projected.go:266] Couldn't get secret openshift-image-registry/image-registry-tls: secret "image-registry-tls" not found Jul 10 18:03:27.054603 ip-10-0-128-39 kubenswrapper[2558]: E0710 18:03:27.054564 2558 projected.go:196] Error preparing data for projected volume registry-tls for pod openshift-image-registry/image-registry-cf9777cd6-d9zpc: secret "image-registry-tls" not found Jul 10 18:03:27.054603 ip-10-0-128-39 kubenswrapper[2558]: E0710 18:03:27.054601 2558 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/projected/4b133a69-01d6-42b5-b12f-d063856c87cb-registry-tls podName:4b133a69-01d6-42b5-b12f-d063856c87cb nodeName:}" failed. No retries permitted until 2026-07-10 18:03:27.554585971 +0000 UTC m=+36.058078457 (durationBeforeRetry 500ms). Error: MountVolume.SetUp failed for volume "registry-tls" (UniqueName: "kubernetes.io/projected/4b133a69-01d6-42b5-b12f-d063856c87cb-registry-tls") pod "image-registry-cf9777cd6-d9zpc" (UID: "4b133a69-01d6-42b5-b12f-d063856c87cb") : secret "image-registry-tls" not found Jul 10 18:03:27.055053 ip-10-0-128-39 kubenswrapper[2558]: I0710 18:03:27.054980 2558 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"ca-trust-extracted\" (UniqueName: \"kubernetes.io/empty-dir/4b133a69-01d6-42b5-b12f-d063856c87cb-ca-trust-extracted\") pod \"image-registry-cf9777cd6-d9zpc\" (UID: \"4b133a69-01d6-42b5-b12f-d063856c87cb\") " pod="openshift-image-registry/image-registry-cf9777cd6-d9zpc" Jul 10 18:03:27.055433 ip-10-0-128-39 kubenswrapper[2558]: I0710 18:03:27.055408 2558 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"trusted-ca\" (UniqueName: \"kubernetes.io/configmap/4b133a69-01d6-42b5-b12f-d063856c87cb-trusted-ca\") pod \"image-registry-cf9777cd6-d9zpc\" (UID: \"4b133a69-01d6-42b5-b12f-d063856c87cb\") " pod="openshift-image-registry/image-registry-cf9777cd6-d9zpc" Jul 10 18:03:27.058794 ip-10-0-128-39 kubenswrapper[2558]: I0710 18:03:27.058724 2558 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"image-registry-private-configuration\" (UniqueName: \"kubernetes.io/secret/4b133a69-01d6-42b5-b12f-d063856c87cb-image-registry-private-configuration\") pod \"image-registry-cf9777cd6-d9zpc\" (UID: \"4b133a69-01d6-42b5-b12f-d063856c87cb\") " pod="openshift-image-registry/image-registry-cf9777cd6-d9zpc" Jul 10 18:03:27.058794 ip-10-0-128-39 kubenswrapper[2558]: I0710 18:03:27.058726 2558 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"installation-pull-secrets\" (UniqueName: \"kubernetes.io/secret/4b133a69-01d6-42b5-b12f-d063856c87cb-installation-pull-secrets\") pod \"image-registry-cf9777cd6-d9zpc\" (UID: \"4b133a69-01d6-42b5-b12f-d063856c87cb\") " pod="openshift-image-registry/image-registry-cf9777cd6-d9zpc" Jul 10 18:03:27.058794 ip-10-0-128-39 kubenswrapper[2558]: I0710 18:03:27.058732 2558 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"cert\" (UniqueName: \"kubernetes.io/secret/f754e509-abbe-4451-833d-abb0e28c7e19-cert\") pod \"ingress-canary-bpwbf\" (UID: \"f754e509-abbe-4451-833d-abb0e28c7e19\") " pod="openshift-ingress-canary/ingress-canary-bpwbf" Jul 10 18:03:27.063682 ip-10-0-128-39 kubenswrapper[2558]: I0710 18:03:27.063655 2558 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-fmt6b\" (UniqueName: \"kubernetes.io/projected/15c4360f-6bc7-4948-b0e6-ebc018174856-kube-api-access-fmt6b\") pod \"dns-default-22khc\" (UID: \"15c4360f-6bc7-4948-b0e6-ebc018174856\") " pod="openshift-dns/dns-default-22khc" Jul 10 18:03:27.063789 ip-10-0-128-39 kubenswrapper[2558]: I0710 18:03:27.063667 2558 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-9kb59\" (UniqueName: \"kubernetes.io/projected/4b133a69-01d6-42b5-b12f-d063856c87cb-kube-api-access-9kb59\") pod \"image-registry-cf9777cd6-d9zpc\" (UID: \"4b133a69-01d6-42b5-b12f-d063856c87cb\") " pod="openshift-image-registry/image-registry-cf9777cd6-d9zpc" Jul 10 18:03:27.063901 ip-10-0-128-39 kubenswrapper[2558]: I0710 18:03:27.063880 2558 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-w6rfg\" (UniqueName: \"kubernetes.io/projected/f754e509-abbe-4451-833d-abb0e28c7e19-kube-api-access-w6rfg\") pod \"ingress-canary-bpwbf\" (UID: \"f754e509-abbe-4451-833d-abb0e28c7e19\") " pod="openshift-ingress-canary/ingress-canary-bpwbf" Jul 10 18:03:27.064269 ip-10-0-128-39 kubenswrapper[2558]: I0710 18:03:27.064249 2558 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"bound-sa-token\" (UniqueName: \"kubernetes.io/projected/4b133a69-01d6-42b5-b12f-d063856c87cb-bound-sa-token\") pod \"image-registry-cf9777cd6-d9zpc\" (UID: \"4b133a69-01d6-42b5-b12f-d063856c87cb\") " pod="openshift-image-registry/image-registry-cf9777cd6-d9zpc" Jul 10 18:03:27.068318 ip-10-0-128-39 kubenswrapper[2558]: I0710 18:03:27.068204 2558 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-dvp86_2da19865-b021-4789-b1e0-694d4f6c543d/konnectivity-agent/0.log" limit=1024 Jul 10 18:03:27.095147 ip-10-0-128-39 kubenswrapper[2558]: I0710 18:03:27.095120 2558 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-rzfgn" Jul 10 18:03:27.095293 ip-10-0-128-39 kubenswrapper[2558]: I0710 18:03:27.095161 2558 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-vsbdm" Jul 10 18:03:27.098007 ip-10-0-128-39 kubenswrapper[2558]: I0710 18:03:27.097981 2558 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-network-diagnostics\"/\"openshift-service-ca.crt\"" Jul 10 18:03:27.098530 ip-10-0-128-39 kubenswrapper[2558]: I0710 18:03:27.098176 2558 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"kube-system\"/\"original-pull-secret\"" Jul 10 18:03:27.098530 ip-10-0-128-39 kubenswrapper[2558]: I0710 18:03:27.098223 2558 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-network-diagnostics\"/\"kube-root-ca.crt\"" Jul 10 18:03:27.098530 ip-10-0-128-39 kubenswrapper[2558]: I0710 18:03:27.098278 2558 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-network-diagnostics\"/\"default-dockercfg-7bsz9\"" Jul 10 18:03:27.149651 ip-10-0-128-39 kubenswrapper[2558]: I0710 18:03:27.149613 2558 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress-canary/ingress-canary-bpwbf" Jul 10 18:03:27.558536 ip-10-0-128-39 kubenswrapper[2558]: I0710 18:03:27.558337 2558 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-tls\" (UniqueName: \"kubernetes.io/secret/15c4360f-6bc7-4948-b0e6-ebc018174856-metrics-tls\") pod \"dns-default-22khc\" (UID: \"15c4360f-6bc7-4948-b0e6-ebc018174856\") " pod="openshift-dns/dns-default-22khc" Jul 10 18:03:27.558700 ip-10-0-128-39 kubenswrapper[2558]: I0710 18:03:27.558573 2558 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"registry-tls\" (UniqueName: \"kubernetes.io/projected/4b133a69-01d6-42b5-b12f-d063856c87cb-registry-tls\") pod \"image-registry-cf9777cd6-d9zpc\" (UID: \"4b133a69-01d6-42b5-b12f-d063856c87cb\") " pod="openshift-image-registry/image-registry-cf9777cd6-d9zpc" Jul 10 18:03:27.558700 ip-10-0-128-39 kubenswrapper[2558]: E0710 18:03:27.558501 2558 secret.go:189] Couldn't get secret openshift-dns/dns-default-metrics-tls: secret "dns-default-metrics-tls" not found Jul 10 18:03:27.558700 ip-10-0-128-39 kubenswrapper[2558]: E0710 18:03:27.558684 2558 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/15c4360f-6bc7-4948-b0e6-ebc018174856-metrics-tls podName:15c4360f-6bc7-4948-b0e6-ebc018174856 nodeName:}" failed. No retries permitted until 2026-07-10 18:03:28.558661046 +0000 UTC m=+37.062153538 (durationBeforeRetry 1s). Error: MountVolume.SetUp failed for volume "metrics-tls" (UniqueName: "kubernetes.io/secret/15c4360f-6bc7-4948-b0e6-ebc018174856-metrics-tls") pod "dns-default-22khc" (UID: "15c4360f-6bc7-4948-b0e6-ebc018174856") : secret "dns-default-metrics-tls" not found Jul 10 18:03:27.558878 ip-10-0-128-39 kubenswrapper[2558]: E0710 18:03:27.558729 2558 projected.go:266] Couldn't get secret openshift-image-registry/image-registry-tls: secret "image-registry-tls" not found Jul 10 18:03:27.558878 ip-10-0-128-39 kubenswrapper[2558]: E0710 18:03:27.558745 2558 projected.go:196] Error preparing data for projected volume registry-tls for pod openshift-image-registry/image-registry-cf9777cd6-d9zpc: secret "image-registry-tls" not found Jul 10 18:03:27.558878 ip-10-0-128-39 kubenswrapper[2558]: E0710 18:03:27.558789 2558 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/projected/4b133a69-01d6-42b5-b12f-d063856c87cb-registry-tls podName:4b133a69-01d6-42b5-b12f-d063856c87cb nodeName:}" failed. No retries permitted until 2026-07-10 18:03:28.558774456 +0000 UTC m=+37.062266957 (durationBeforeRetry 1s). Error: MountVolume.SetUp failed for volume "registry-tls" (UniqueName: "kubernetes.io/projected/4b133a69-01d6-42b5-b12f-d063856c87cb-registry-tls") pod "image-registry-cf9777cd6-d9zpc" (UID: "4b133a69-01d6-42b5-b12f-d063856c87cb") : secret "image-registry-tls" not found Jul 10 18:03:28.095114 ip-10-0-128-39 kubenswrapper[2558]: I0710 18:03:28.095077 2558 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-jkr77" Jul 10 18:03:28.098140 ip-10-0-128-39 kubenswrapper[2558]: I0710 18:03:28.097996 2558 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-multus\"/\"metrics-daemon-secret\"" Jul 10 18:03:28.098140 ip-10-0-128-39 kubenswrapper[2558]: I0710 18:03:28.098018 2558 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-multus\"/\"metrics-daemon-sa-dockercfg-fb6vc\"" Jul 10 18:03:28.234691 ip-10-0-128-39 kubenswrapper[2558]: I0710 18:03:28.234656 2558 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-network-diagnostics/network-check-source-869dcfcdcd-7n55h"] Jul 10 18:03:28.236882 ip-10-0-128-39 kubenswrapper[2558]: I0710 18:03:28.236857 2558 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-source-869dcfcdcd-7n55h" Jul 10 18:03:28.239185 ip-10-0-128-39 kubenswrapper[2558]: I0710 18:03:28.239163 2558 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-network-diagnostics\"/\"network-diagnostics-dockercfg-msj57\"" Jul 10 18:03:28.246601 ip-10-0-128-39 kubenswrapper[2558]: I0710 18:03:28.246575 2558 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-cluster-storage-operator/volume-data-source-validator-78495597f6-v7ckr"] Jul 10 18:03:28.249120 ip-10-0-128-39 kubenswrapper[2558]: I0710 18:03:28.249098 2558 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-insights/insights-operator-68d9dd94cf-q5s5m"] Jul 10 18:03:28.249476 ip-10-0-128-39 kubenswrapper[2558]: I0710 18:03:28.249433 2558 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-cluster-storage-operator/volume-data-source-validator-78495597f6-v7ckr" Jul 10 18:03:28.252737 ip-10-0-128-39 kubenswrapper[2558]: I0710 18:03:28.252711 2558 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-cluster-storage-operator\"/\"openshift-service-ca.crt\"" Jul 10 18:03:28.252737 ip-10-0-128-39 kubenswrapper[2558]: I0710 18:03:28.252731 2558 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-cluster-storage-operator\"/\"kube-root-ca.crt\"" Jul 10 18:03:28.252936 ip-10-0-128-39 kubenswrapper[2558]: I0710 18:03:28.252737 2558 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-cluster-storage-operator\"/\"volume-data-source-validator-dockercfg-mspp7\"" Jul 10 18:03:28.253070 ip-10-0-128-39 kubenswrapper[2558]: I0710 18:03:28.253051 2558 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-monitoring/cluster-monitoring-operator-5f7f494dd4-46fc9"] Jul 10 18:03:28.253175 ip-10-0-128-39 kubenswrapper[2558]: I0710 18:03:28.253159 2558 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-insights/insights-operator-68d9dd94cf-q5s5m" Jul 10 18:03:28.254969 ip-10-0-128-39 kubenswrapper[2558]: I0710 18:03:28.254952 2558 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-network-diagnostics/network-check-source-869dcfcdcd-7n55h"] Jul 10 18:03:28.255082 ip-10-0-128-39 kubenswrapper[2558]: I0710 18:03:28.255066 2558 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-monitoring/cluster-monitoring-operator-5f7f494dd4-46fc9" Jul 10 18:03:28.255441 ip-10-0-128-39 kubenswrapper[2558]: I0710 18:03:28.255408 2558 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-insights\"/\"openshift-service-ca.crt\"" Jul 10 18:03:28.255731 ip-10-0-128-39 kubenswrapper[2558]: I0710 18:03:28.255714 2558 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-insights\"/\"operator-dockercfg-px4hr\"" Jul 10 18:03:28.255892 ip-10-0-128-39 kubenswrapper[2558]: I0710 18:03:28.255871 2558 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-insights\"/\"openshift-insights-serving-cert\"" Jul 10 18:03:28.255959 ip-10-0-128-39 kubenswrapper[2558]: I0710 18:03:28.255906 2558 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-insights\"/\"service-ca-bundle\"" Jul 10 18:03:28.256024 ip-10-0-128-39 kubenswrapper[2558]: I0710 18:03:28.255977 2558 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-insights\"/\"kube-root-ca.crt\"" Jul 10 18:03:28.257374 ip-10-0-128-39 kubenswrapper[2558]: I0710 18:03:28.257356 2558 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-monitoring\"/\"openshift-service-ca.crt\"" Jul 10 18:03:28.257589 ip-10-0-128-39 kubenswrapper[2558]: I0710 18:03:28.257571 2558 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"cluster-monitoring-operator-dockercfg-bx2s5\"" Jul 10 18:03:28.258669 ip-10-0-128-39 kubenswrapper[2558]: I0710 18:03:28.258646 2558 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"cluster-monitoring-operator-tls\"" Jul 10 18:03:28.259101 ip-10-0-128-39 kubenswrapper[2558]: I0710 18:03:28.259083 2558 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-monitoring\"/\"kube-root-ca.crt\"" Jul 10 18:03:28.259898 ip-10-0-128-39 kubenswrapper[2558]: I0710 18:03:28.259878 2558 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-monitoring\"/\"telemetry-config\"" Jul 10 18:03:28.262722 ip-10-0-128-39 kubenswrapper[2558]: I0710 18:03:28.262703 2558 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-cluster-storage-operator/volume-data-source-validator-78495597f6-v7ckr"] Jul 10 18:03:28.263062 ip-10-0-128-39 kubenswrapper[2558]: I0710 18:03:28.263027 2558 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-monitoring/cluster-monitoring-operator-5f7f494dd4-46fc9"] Jul 10 18:03:28.263488 ip-10-0-128-39 kubenswrapper[2558]: I0710 18:03:28.263462 2558 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-insights\"/\"trusted-ca-bundle\"" Jul 10 18:03:28.263922 ip-10-0-128-39 kubenswrapper[2558]: I0710 18:03:28.263899 2558 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-97j66\" (UniqueName: \"kubernetes.io/projected/5997642b-76cf-44e6-8b06-138718112a30-kube-api-access-97j66\") pod \"network-check-source-869dcfcdcd-7n55h\" (UID: \"5997642b-76cf-44e6-8b06-138718112a30\") " pod="openshift-network-diagnostics/network-check-source-869dcfcdcd-7n55h" Jul 10 18:03:28.264083 ip-10-0-128-39 kubenswrapper[2558]: I0710 18:03:28.264047 2558 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"telemetry-config\" (UniqueName: \"kubernetes.io/configmap/19b8d048-b0b5-4b42-9f4f-ad9f47a105cd-telemetry-config\") pod \"cluster-monitoring-operator-5f7f494dd4-46fc9\" (UID: \"19b8d048-b0b5-4b42-9f4f-ad9f47a105cd\") " pod="openshift-monitoring/cluster-monitoring-operator-5f7f494dd4-46fc9" Jul 10 18:03:28.264083 ip-10-0-128-39 kubenswrapper[2558]: I0710 18:03:28.264009 2558 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-insights/insights-operator-68d9dd94cf-q5s5m"] Jul 10 18:03:28.264230 ip-10-0-128-39 kubenswrapper[2558]: I0710 18:03:28.264122 2558 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"cluster-monitoring-operator-tls\" (UniqueName: \"kubernetes.io/secret/19b8d048-b0b5-4b42-9f4f-ad9f47a105cd-cluster-monitoring-operator-tls\") pod \"cluster-monitoring-operator-5f7f494dd4-46fc9\" (UID: \"19b8d048-b0b5-4b42-9f4f-ad9f47a105cd\") " pod="openshift-monitoring/cluster-monitoring-operator-5f7f494dd4-46fc9" Jul 10 18:03:28.264230 ip-10-0-128-39 kubenswrapper[2558]: I0710 18:03:28.264151 2558 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-6cxkp\" (UniqueName: \"kubernetes.io/projected/19b8d048-b0b5-4b42-9f4f-ad9f47a105cd-kube-api-access-6cxkp\") pod \"cluster-monitoring-operator-5f7f494dd4-46fc9\" (UID: \"19b8d048-b0b5-4b42-9f4f-ad9f47a105cd\") " pod="openshift-monitoring/cluster-monitoring-operator-5f7f494dd4-46fc9" Jul 10 18:03:28.339294 ip-10-0-128-39 kubenswrapper[2558]: I0710 18:03:28.339258 2558 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-service-ca-operator/service-ca-operator-7695fc6c8c-hfzjm"] Jul 10 18:03:28.341610 ip-10-0-128-39 kubenswrapper[2558]: I0710 18:03:28.341582 2558 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-service-ca-operator/service-ca-operator-7695fc6c8c-hfzjm" Jul 10 18:03:28.342559 ip-10-0-128-39 kubenswrapper[2558]: I0710 18:03:28.342541 2558 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-network-console/networking-console-plugin-6cf856758c-qg5vl"] Jul 10 18:03:28.344093 ip-10-0-128-39 kubenswrapper[2558]: I0710 18:03:28.344072 2558 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-service-ca-operator\"/\"serving-cert\"" Jul 10 18:03:28.344503 ip-10-0-128-39 kubenswrapper[2558]: I0710 18:03:28.344484 2558 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-kube-storage-version-migrator-operator/kube-storage-version-migrator-operator-799cb7f958-vggpz"] Jul 10 18:03:28.344665 ip-10-0-128-39 kubenswrapper[2558]: I0710 18:03:28.344648 2558 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-console/networking-console-plugin-6cf856758c-qg5vl" Jul 10 18:03:28.344945 ip-10-0-128-39 kubenswrapper[2558]: I0710 18:03:28.344924 2558 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-service-ca-operator\"/\"service-ca-operator-dockercfg-dml7p\"" Jul 10 18:03:28.345243 ip-10-0-128-39 kubenswrapper[2558]: I0710 18:03:28.345146 2558 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-service-ca-operator\"/\"kube-root-ca.crt\"" Jul 10 18:03:28.345456 ip-10-0-128-39 kubenswrapper[2558]: I0710 18:03:28.345435 2558 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-service-ca-operator\"/\"openshift-service-ca.crt\"" Jul 10 18:03:28.345456 ip-10-0-128-39 kubenswrapper[2558]: I0710 18:03:28.345452 2558 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-service-ca-operator\"/\"service-ca-operator-config\"" Jul 10 18:03:28.346336 ip-10-0-128-39 kubenswrapper[2558]: I0710 18:03:28.346321 2558 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-cluster-samples-operator/cluster-samples-operator-65db858894-2b2km"] Jul 10 18:03:28.346500 ip-10-0-128-39 kubenswrapper[2558]: I0710 18:03:28.346486 2558 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-kube-storage-version-migrator-operator/kube-storage-version-migrator-operator-799cb7f958-vggpz" Jul 10 18:03:28.346992 ip-10-0-128-39 kubenswrapper[2558]: I0710 18:03:28.346973 2558 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-network-console\"/\"networking-console-plugin\"" Jul 10 18:03:28.348071 ip-10-0-128-39 kubenswrapper[2558]: I0710 18:03:28.348052 2558 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-network-console\"/\"networking-console-plugin-cert\"" Jul 10 18:03:28.348330 ip-10-0-128-39 kubenswrapper[2558]: I0710 18:03:28.348134 2558 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-network-console\"/\"default-dockercfg-cwkwl\"" Jul 10 18:03:28.349089 ip-10-0-128-39 kubenswrapper[2558]: I0710 18:03:28.349066 2558 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-kube-storage-version-migrator-operator\"/\"config\"" Jul 10 18:03:28.349186 ip-10-0-128-39 kubenswrapper[2558]: I0710 18:03:28.349085 2558 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-console-operator/console-operator-7866497494-26542"] Jul 10 18:03:28.349241 ip-10-0-128-39 kubenswrapper[2558]: I0710 18:03:28.349180 2558 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-kube-storage-version-migrator-operator\"/\"serving-cert\"" Jul 10 18:03:28.349832 ip-10-0-128-39 kubenswrapper[2558]: I0710 18:03:28.349430 2558 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-kube-storage-version-migrator-operator\"/\"openshift-service-ca.crt\"" Jul 10 18:03:28.349832 ip-10-0-128-39 kubenswrapper[2558]: I0710 18:03:28.349492 2558 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-cluster-samples-operator/cluster-samples-operator-65db858894-2b2km" Jul 10 18:03:28.350558 ip-10-0-128-39 kubenswrapper[2558]: I0710 18:03:28.350366 2558 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-kube-storage-version-migrator-operator\"/\"kube-storage-version-migrator-operator-dockercfg-6pmhk\"" Jul 10 18:03:28.350558 ip-10-0-128-39 kubenswrapper[2558]: I0710 18:03:28.350464 2558 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-kube-storage-version-migrator-operator\"/\"kube-root-ca.crt\"" Jul 10 18:03:28.352286 ip-10-0-128-39 kubenswrapper[2558]: I0710 18:03:28.352268 2558 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-cluster-samples-operator\"/\"samples-operator-tls\"" Jul 10 18:03:28.352594 ip-10-0-128-39 kubenswrapper[2558]: I0710 18:03:28.352497 2558 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-cluster-samples-operator\"/\"openshift-service-ca.crt\"" Jul 10 18:03:28.352697 ip-10-0-128-39 kubenswrapper[2558]: I0710 18:03:28.352438 2558 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-cluster-samples-operator\"/\"cluster-samples-operator-dockercfg-rvj2g\"" Jul 10 18:03:28.352938 ip-10-0-128-39 kubenswrapper[2558]: I0710 18:03:28.352376 2558 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-cluster-samples-operator\"/\"kube-root-ca.crt\"" Jul 10 18:03:28.353500 ip-10-0-128-39 kubenswrapper[2558]: I0710 18:03:28.353476 2558 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-service-ca-operator/service-ca-operator-7695fc6c8c-hfzjm"] Jul 10 18:03:28.353616 ip-10-0-128-39 kubenswrapper[2558]: I0710 18:03:28.353599 2558 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-console-operator/console-operator-7866497494-26542" Jul 10 18:03:28.355706 ip-10-0-128-39 kubenswrapper[2558]: I0710 18:03:28.355687 2558 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-console-operator\"/\"console-operator-config\"" Jul 10 18:03:28.356126 ip-10-0-128-39 kubenswrapper[2558]: I0710 18:03:28.356015 2558 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-console-operator\"/\"openshift-service-ca.crt\"" Jul 10 18:03:28.356234 ip-10-0-128-39 kubenswrapper[2558]: I0710 18:03:28.356191 2558 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-console-operator\"/\"console-operator-dockercfg-89x4l\"" Jul 10 18:03:28.356234 ip-10-0-128-39 kubenswrapper[2558]: I0710 18:03:28.356216 2558 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-console-operator\"/\"serving-cert\"" Jul 10 18:03:28.356936 ip-10-0-128-39 kubenswrapper[2558]: I0710 18:03:28.356905 2558 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-console-operator\"/\"kube-root-ca.crt\"" Jul 10 18:03:28.360796 ip-10-0-128-39 kubenswrapper[2558]: I0710 18:03:28.360559 2558 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-kube-storage-version-migrator-operator/kube-storage-version-migrator-operator-799cb7f958-vggpz"] Jul 10 18:03:28.362094 ip-10-0-128-39 kubenswrapper[2558]: I0710 18:03:28.362072 2558 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-network-console/networking-console-plugin-6cf856758c-qg5vl"] Jul 10 18:03:28.363374 ip-10-0-128-39 kubenswrapper[2558]: I0710 18:03:28.363352 2558 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-cluster-samples-operator/cluster-samples-operator-65db858894-2b2km"] Jul 10 18:03:28.364676 ip-10-0-128-39 kubenswrapper[2558]: I0710 18:03:28.364654 2558 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-console-operator/console-operator-7866497494-26542"] Jul 10 18:03:28.365014 ip-10-0-128-39 kubenswrapper[2558]: I0710 18:03:28.364991 2558 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"snapshots\" (UniqueName: \"kubernetes.io/empty-dir/748dca80-632f-44ef-b3a2-5292df366af1-snapshots\") pod \"insights-operator-68d9dd94cf-q5s5m\" (UID: \"748dca80-632f-44ef-b3a2-5292df366af1\") " pod="openshift-insights/insights-operator-68d9dd94cf-q5s5m" Jul 10 18:03:28.365089 ip-10-0-128-39 kubenswrapper[2558]: I0710 18:03:28.365052 2558 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-v8h6d\" (UniqueName: \"kubernetes.io/projected/748dca80-632f-44ef-b3a2-5292df366af1-kube-api-access-v8h6d\") pod \"insights-operator-68d9dd94cf-q5s5m\" (UID: \"748dca80-632f-44ef-b3a2-5292df366af1\") " pod="openshift-insights/insights-operator-68d9dd94cf-q5s5m" Jul 10 18:03:28.365151 ip-10-0-128-39 kubenswrapper[2558]: I0710 18:03:28.365088 2558 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-6cxkp\" (UniqueName: \"kubernetes.io/projected/19b8d048-b0b5-4b42-9f4f-ad9f47a105cd-kube-api-access-6cxkp\") pod \"cluster-monitoring-operator-5f7f494dd4-46fc9\" (UID: \"19b8d048-b0b5-4b42-9f4f-ad9f47a105cd\") " pod="openshift-monitoring/cluster-monitoring-operator-5f7f494dd4-46fc9" Jul 10 18:03:28.365151 ip-10-0-128-39 kubenswrapper[2558]: I0710 18:03:28.365114 2558 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-qtzrd\" (UniqueName: \"kubernetes.io/projected/5c7b84a3-17f6-4ed7-bb37-5e5b19a52916-kube-api-access-qtzrd\") pod \"cluster-samples-operator-65db858894-2b2km\" (UID: \"5c7b84a3-17f6-4ed7-bb37-5e5b19a52916\") " pod="openshift-cluster-samples-operator/cluster-samples-operator-65db858894-2b2km" Jul 10 18:03:28.365301 ip-10-0-128-39 kubenswrapper[2558]: I0710 18:03:28.365284 2558 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"config\" (UniqueName: \"kubernetes.io/configmap/2b29fb39-89b3-42c3-b334-3a195242248c-config\") pod \"console-operator-7866497494-26542\" (UID: \"2b29fb39-89b3-42c3-b334-3a195242248c\") " pod="openshift-console-operator/console-operator-7866497494-26542" Jul 10 18:03:28.365373 ip-10-0-128-39 kubenswrapper[2558]: I0710 18:03:28.365319 2558 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"config\" (UniqueName: \"kubernetes.io/configmap/893dc3d5-4a12-4832-9a96-898388c12ae4-config\") pod \"service-ca-operator-7695fc6c8c-hfzjm\" (UID: \"893dc3d5-4a12-4832-9a96-898388c12ae4\") " pod="openshift-service-ca-operator/service-ca-operator-7695fc6c8c-hfzjm" Jul 10 18:03:28.365373 ip-10-0-128-39 kubenswrapper[2558]: I0710 18:03:28.365341 2558 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tmp\" (UniqueName: \"kubernetes.io/empty-dir/748dca80-632f-44ef-b3a2-5292df366af1-tmp\") pod \"insights-operator-68d9dd94cf-q5s5m\" (UID: \"748dca80-632f-44ef-b3a2-5292df366af1\") " pod="openshift-insights/insights-operator-68d9dd94cf-q5s5m" Jul 10 18:03:28.365373 ip-10-0-128-39 kubenswrapper[2558]: I0710 18:03:28.365361 2558 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"serving-cert\" (UniqueName: \"kubernetes.io/secret/2b29fb39-89b3-42c3-b334-3a195242248c-serving-cert\") pod \"console-operator-7866497494-26542\" (UID: \"2b29fb39-89b3-42c3-b334-3a195242248c\") " pod="openshift-console-operator/console-operator-7866497494-26542" Jul 10 18:03:28.365526 ip-10-0-128-39 kubenswrapper[2558]: I0710 18:03:28.365383 2558 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-7xcrn\" (UniqueName: \"kubernetes.io/projected/2b29fb39-89b3-42c3-b334-3a195242248c-kube-api-access-7xcrn\") pod \"console-operator-7866497494-26542\" (UID: \"2b29fb39-89b3-42c3-b334-3a195242248c\") " pod="openshift-console-operator/console-operator-7866497494-26542" Jul 10 18:03:28.365526 ip-10-0-128-39 kubenswrapper[2558]: I0710 18:03:28.365412 2558 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"serving-cert\" (UniqueName: \"kubernetes.io/secret/eec35344-a5cb-4a6e-9e2e-5adca5a0fa1a-serving-cert\") pod \"kube-storage-version-migrator-operator-799cb7f958-vggpz\" (UID: \"eec35344-a5cb-4a6e-9e2e-5adca5a0fa1a\") " pod="openshift-kube-storage-version-migrator-operator/kube-storage-version-migrator-operator-799cb7f958-vggpz" Jul 10 18:03:28.365526 ip-10-0-128-39 kubenswrapper[2558]: I0710 18:03:28.365453 2558 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"config\" (UniqueName: \"kubernetes.io/configmap/eec35344-a5cb-4a6e-9e2e-5adca5a0fa1a-config\") pod \"kube-storage-version-migrator-operator-799cb7f958-vggpz\" (UID: \"eec35344-a5cb-4a6e-9e2e-5adca5a0fa1a\") " pod="openshift-kube-storage-version-migrator-operator/kube-storage-version-migrator-operator-799cb7f958-vggpz" Jul 10 18:03:28.365526 ip-10-0-128-39 kubenswrapper[2558]: I0710 18:03:28.365490 2558 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/748dca80-632f-44ef-b3a2-5292df366af1-trusted-ca-bundle\") pod \"insights-operator-68d9dd94cf-q5s5m\" (UID: \"748dca80-632f-44ef-b3a2-5292df366af1\") " pod="openshift-insights/insights-operator-68d9dd94cf-q5s5m" Jul 10 18:03:28.365526 ip-10-0-128-39 kubenswrapper[2558]: I0710 18:03:28.365507 2558 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-mlxgd\" (UniqueName: \"kubernetes.io/projected/893dc3d5-4a12-4832-9a96-898388c12ae4-kube-api-access-mlxgd\") pod \"service-ca-operator-7695fc6c8c-hfzjm\" (UID: \"893dc3d5-4a12-4832-9a96-898388c12ae4\") " pod="openshift-service-ca-operator/service-ca-operator-7695fc6c8c-hfzjm" Jul 10 18:03:28.365748 ip-10-0-128-39 kubenswrapper[2558]: I0710 18:03:28.365524 2558 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"trusted-ca\" (UniqueName: \"kubernetes.io/configmap/2b29fb39-89b3-42c3-b334-3a195242248c-trusted-ca\") pod \"console-operator-7866497494-26542\" (UID: \"2b29fb39-89b3-42c3-b334-3a195242248c\") " pod="openshift-console-operator/console-operator-7866497494-26542" Jul 10 18:03:28.365748 ip-10-0-128-39 kubenswrapper[2558]: I0710 18:03:28.365553 2558 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-2ztqb\" (UniqueName: \"kubernetes.io/projected/5df2ceaf-9a85-4d1a-ad19-76df6f2b955f-kube-api-access-2ztqb\") pod \"volume-data-source-validator-78495597f6-v7ckr\" (UID: \"5df2ceaf-9a85-4d1a-ad19-76df6f2b955f\") " pod="openshift-cluster-storage-operator/volume-data-source-validator-78495597f6-v7ckr" Jul 10 18:03:28.365748 ip-10-0-128-39 kubenswrapper[2558]: I0710 18:03:28.365579 2558 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-vqqcp\" (UniqueName: \"kubernetes.io/projected/eec35344-a5cb-4a6e-9e2e-5adca5a0fa1a-kube-api-access-vqqcp\") pod \"kube-storage-version-migrator-operator-799cb7f958-vggpz\" (UID: \"eec35344-a5cb-4a6e-9e2e-5adca5a0fa1a\") " pod="openshift-kube-storage-version-migrator-operator/kube-storage-version-migrator-operator-799cb7f958-vggpz" Jul 10 18:03:28.365748 ip-10-0-128-39 kubenswrapper[2558]: I0710 18:03:28.365613 2558 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-97j66\" (UniqueName: \"kubernetes.io/projected/5997642b-76cf-44e6-8b06-138718112a30-kube-api-access-97j66\") pod \"network-check-source-869dcfcdcd-7n55h\" (UID: \"5997642b-76cf-44e6-8b06-138718112a30\") " pod="openshift-network-diagnostics/network-check-source-869dcfcdcd-7n55h" Jul 10 18:03:28.365748 ip-10-0-128-39 kubenswrapper[2558]: I0710 18:03:28.365636 2558 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"networking-console-plugin-cert\" (UniqueName: \"kubernetes.io/secret/b95b59af-225c-4907-9643-5d916a325e56-networking-console-plugin-cert\") pod \"networking-console-plugin-6cf856758c-qg5vl\" (UID: \"b95b59af-225c-4907-9643-5d916a325e56\") " pod="openshift-network-console/networking-console-plugin-6cf856758c-qg5vl" Jul 10 18:03:28.365748 ip-10-0-128-39 kubenswrapper[2558]: I0710 18:03:28.365656 2558 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"samples-operator-tls\" (UniqueName: \"kubernetes.io/secret/5c7b84a3-17f6-4ed7-bb37-5e5b19a52916-samples-operator-tls\") pod \"cluster-samples-operator-65db858894-2b2km\" (UID: \"5c7b84a3-17f6-4ed7-bb37-5e5b19a52916\") " pod="openshift-cluster-samples-operator/cluster-samples-operator-65db858894-2b2km" Jul 10 18:03:28.365748 ip-10-0-128-39 kubenswrapper[2558]: I0710 18:03:28.365679 2558 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"cluster-monitoring-operator-tls\" (UniqueName: \"kubernetes.io/secret/19b8d048-b0b5-4b42-9f4f-ad9f47a105cd-cluster-monitoring-operator-tls\") pod \"cluster-monitoring-operator-5f7f494dd4-46fc9\" (UID: \"19b8d048-b0b5-4b42-9f4f-ad9f47a105cd\") " pod="openshift-monitoring/cluster-monitoring-operator-5f7f494dd4-46fc9" Jul 10 18:03:28.365748 ip-10-0-128-39 kubenswrapper[2558]: I0710 18:03:28.365697 2558 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"nginx-conf\" (UniqueName: \"kubernetes.io/configmap/b95b59af-225c-4907-9643-5d916a325e56-nginx-conf\") pod \"networking-console-plugin-6cf856758c-qg5vl\" (UID: \"b95b59af-225c-4907-9643-5d916a325e56\") " pod="openshift-network-console/networking-console-plugin-6cf856758c-qg5vl" Jul 10 18:03:28.365748 ip-10-0-128-39 kubenswrapper[2558]: I0710 18:03:28.365716 2558 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"serving-cert\" (UniqueName: \"kubernetes.io/secret/748dca80-632f-44ef-b3a2-5292df366af1-serving-cert\") pod \"insights-operator-68d9dd94cf-q5s5m\" (UID: \"748dca80-632f-44ef-b3a2-5292df366af1\") " pod="openshift-insights/insights-operator-68d9dd94cf-q5s5m" Jul 10 18:03:28.365748 ip-10-0-128-39 kubenswrapper[2558]: I0710 18:03:28.365749 2558 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"service-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/748dca80-632f-44ef-b3a2-5292df366af1-service-ca-bundle\") pod \"insights-operator-68d9dd94cf-q5s5m\" (UID: \"748dca80-632f-44ef-b3a2-5292df366af1\") " pod="openshift-insights/insights-operator-68d9dd94cf-q5s5m" Jul 10 18:03:28.366253 ip-10-0-128-39 kubenswrapper[2558]: I0710 18:03:28.365791 2558 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"telemetry-config\" (UniqueName: \"kubernetes.io/configmap/19b8d048-b0b5-4b42-9f4f-ad9f47a105cd-telemetry-config\") pod \"cluster-monitoring-operator-5f7f494dd4-46fc9\" (UID: \"19b8d048-b0b5-4b42-9f4f-ad9f47a105cd\") " pod="openshift-monitoring/cluster-monitoring-operator-5f7f494dd4-46fc9" Jul 10 18:03:28.366253 ip-10-0-128-39 kubenswrapper[2558]: I0710 18:03:28.365804 2558 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-console-operator\"/\"trusted-ca\"" Jul 10 18:03:28.366253 ip-10-0-128-39 kubenswrapper[2558]: I0710 18:03:28.365854 2558 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"serving-cert\" (UniqueName: \"kubernetes.io/secret/893dc3d5-4a12-4832-9a96-898388c12ae4-serving-cert\") pod \"service-ca-operator-7695fc6c8c-hfzjm\" (UID: \"893dc3d5-4a12-4832-9a96-898388c12ae4\") " pod="openshift-service-ca-operator/service-ca-operator-7695fc6c8c-hfzjm" Jul 10 18:03:28.366390 ip-10-0-128-39 kubenswrapper[2558]: E0710 18:03:28.366267 2558 secret.go:189] Couldn't get secret openshift-monitoring/cluster-monitoring-operator-tls: secret "cluster-monitoring-operator-tls" not found Jul 10 18:03:28.366390 ip-10-0-128-39 kubenswrapper[2558]: E0710 18:03:28.366340 2558 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/19b8d048-b0b5-4b42-9f4f-ad9f47a105cd-cluster-monitoring-operator-tls podName:19b8d048-b0b5-4b42-9f4f-ad9f47a105cd nodeName:}" failed. No retries permitted until 2026-07-10 18:03:28.866322109 +0000 UTC m=+37.369814615 (durationBeforeRetry 500ms). Error: MountVolume.SetUp failed for volume "cluster-monitoring-operator-tls" (UniqueName: "kubernetes.io/secret/19b8d048-b0b5-4b42-9f4f-ad9f47a105cd-cluster-monitoring-operator-tls") pod "cluster-monitoring-operator-5f7f494dd4-46fc9" (UID: "19b8d048-b0b5-4b42-9f4f-ad9f47a105cd") : secret "cluster-monitoring-operator-tls" not found Jul 10 18:03:28.367010 ip-10-0-128-39 kubenswrapper[2558]: I0710 18:03:28.366986 2558 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"telemetry-config\" (UniqueName: \"kubernetes.io/configmap/19b8d048-b0b5-4b42-9f4f-ad9f47a105cd-telemetry-config\") pod \"cluster-monitoring-operator-5f7f494dd4-46fc9\" (UID: \"19b8d048-b0b5-4b42-9f4f-ad9f47a105cd\") " pod="openshift-monitoring/cluster-monitoring-operator-5f7f494dd4-46fc9" Jul 10 18:03:28.380995 ip-10-0-128-39 kubenswrapper[2558]: I0710 18:03:28.380884 2558 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-97j66\" (UniqueName: \"kubernetes.io/projected/5997642b-76cf-44e6-8b06-138718112a30-kube-api-access-97j66\") pod \"network-check-source-869dcfcdcd-7n55h\" (UID: \"5997642b-76cf-44e6-8b06-138718112a30\") " pod="openshift-network-diagnostics/network-check-source-869dcfcdcd-7n55h" Jul 10 18:03:28.381142 ip-10-0-128-39 kubenswrapper[2558]: I0710 18:03:28.381123 2558 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-6cxkp\" (UniqueName: \"kubernetes.io/projected/19b8d048-b0b5-4b42-9f4f-ad9f47a105cd-kube-api-access-6cxkp\") pod \"cluster-monitoring-operator-5f7f494dd4-46fc9\" (UID: \"19b8d048-b0b5-4b42-9f4f-ad9f47a105cd\") " pod="openshift-monitoring/cluster-monitoring-operator-5f7f494dd4-46fc9" Jul 10 18:03:28.467242 ip-10-0-128-39 kubenswrapper[2558]: I0710 18:03:28.467204 2558 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"nginx-conf\" (UniqueName: \"kubernetes.io/configmap/b95b59af-225c-4907-9643-5d916a325e56-nginx-conf\") pod \"networking-console-plugin-6cf856758c-qg5vl\" (UID: \"b95b59af-225c-4907-9643-5d916a325e56\") " pod="openshift-network-console/networking-console-plugin-6cf856758c-qg5vl" Jul 10 18:03:28.467242 ip-10-0-128-39 kubenswrapper[2558]: I0710 18:03:28.467251 2558 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"serving-cert\" (UniqueName: \"kubernetes.io/secret/748dca80-632f-44ef-b3a2-5292df366af1-serving-cert\") pod \"insights-operator-68d9dd94cf-q5s5m\" (UID: \"748dca80-632f-44ef-b3a2-5292df366af1\") " pod="openshift-insights/insights-operator-68d9dd94cf-q5s5m" Jul 10 18:03:28.467490 ip-10-0-128-39 kubenswrapper[2558]: I0710 18:03:28.467441 2558 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"service-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/748dca80-632f-44ef-b3a2-5292df366af1-service-ca-bundle\") pod \"insights-operator-68d9dd94cf-q5s5m\" (UID: \"748dca80-632f-44ef-b3a2-5292df366af1\") " pod="openshift-insights/insights-operator-68d9dd94cf-q5s5m" Jul 10 18:03:28.467555 ip-10-0-128-39 kubenswrapper[2558]: I0710 18:03:28.467519 2558 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"serving-cert\" (UniqueName: \"kubernetes.io/secret/893dc3d5-4a12-4832-9a96-898388c12ae4-serving-cert\") pod \"service-ca-operator-7695fc6c8c-hfzjm\" (UID: \"893dc3d5-4a12-4832-9a96-898388c12ae4\") " pod="openshift-service-ca-operator/service-ca-operator-7695fc6c8c-hfzjm" Jul 10 18:03:28.467612 ip-10-0-128-39 kubenswrapper[2558]: I0710 18:03:28.467558 2558 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"snapshots\" (UniqueName: \"kubernetes.io/empty-dir/748dca80-632f-44ef-b3a2-5292df366af1-snapshots\") pod \"insights-operator-68d9dd94cf-q5s5m\" (UID: \"748dca80-632f-44ef-b3a2-5292df366af1\") " pod="openshift-insights/insights-operator-68d9dd94cf-q5s5m" Jul 10 18:03:28.467612 ip-10-0-128-39 kubenswrapper[2558]: I0710 18:03:28.467593 2558 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-v8h6d\" (UniqueName: \"kubernetes.io/projected/748dca80-632f-44ef-b3a2-5292df366af1-kube-api-access-v8h6d\") pod \"insights-operator-68d9dd94cf-q5s5m\" (UID: \"748dca80-632f-44ef-b3a2-5292df366af1\") " pod="openshift-insights/insights-operator-68d9dd94cf-q5s5m" Jul 10 18:03:28.467703 ip-10-0-128-39 kubenswrapper[2558]: I0710 18:03:28.467631 2558 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-qtzrd\" (UniqueName: \"kubernetes.io/projected/5c7b84a3-17f6-4ed7-bb37-5e5b19a52916-kube-api-access-qtzrd\") pod \"cluster-samples-operator-65db858894-2b2km\" (UID: \"5c7b84a3-17f6-4ed7-bb37-5e5b19a52916\") " pod="openshift-cluster-samples-operator/cluster-samples-operator-65db858894-2b2km" Jul 10 18:03:28.467703 ip-10-0-128-39 kubenswrapper[2558]: I0710 18:03:28.467669 2558 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"config\" (UniqueName: \"kubernetes.io/configmap/2b29fb39-89b3-42c3-b334-3a195242248c-config\") pod \"console-operator-7866497494-26542\" (UID: \"2b29fb39-89b3-42c3-b334-3a195242248c\") " pod="openshift-console-operator/console-operator-7866497494-26542" Jul 10 18:03:28.467703 ip-10-0-128-39 kubenswrapper[2558]: I0710 18:03:28.467686 2558 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"config\" (UniqueName: \"kubernetes.io/configmap/893dc3d5-4a12-4832-9a96-898388c12ae4-config\") pod \"service-ca-operator-7695fc6c8c-hfzjm\" (UID: \"893dc3d5-4a12-4832-9a96-898388c12ae4\") " pod="openshift-service-ca-operator/service-ca-operator-7695fc6c8c-hfzjm" Jul 10 18:03:28.468108 ip-10-0-128-39 kubenswrapper[2558]: I0710 18:03:28.467708 2558 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tmp\" (UniqueName: \"kubernetes.io/empty-dir/748dca80-632f-44ef-b3a2-5292df366af1-tmp\") pod \"insights-operator-68d9dd94cf-q5s5m\" (UID: \"748dca80-632f-44ef-b3a2-5292df366af1\") " pod="openshift-insights/insights-operator-68d9dd94cf-q5s5m" Jul 10 18:03:28.468108 ip-10-0-128-39 kubenswrapper[2558]: I0710 18:03:28.468090 2558 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"tmp\" (UniqueName: \"kubernetes.io/empty-dir/748dca80-632f-44ef-b3a2-5292df366af1-tmp\") pod \"insights-operator-68d9dd94cf-q5s5m\" (UID: \"748dca80-632f-44ef-b3a2-5292df366af1\") " pod="openshift-insights/insights-operator-68d9dd94cf-q5s5m" Jul 10 18:03:28.468254 ip-10-0-128-39 kubenswrapper[2558]: I0710 18:03:28.468227 2558 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"serving-cert\" (UniqueName: \"kubernetes.io/secret/2b29fb39-89b3-42c3-b334-3a195242248c-serving-cert\") pod \"console-operator-7866497494-26542\" (UID: \"2b29fb39-89b3-42c3-b334-3a195242248c\") " pod="openshift-console-operator/console-operator-7866497494-26542" Jul 10 18:03:28.468304 ip-10-0-128-39 kubenswrapper[2558]: I0710 18:03:28.468266 2558 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"nginx-conf\" (UniqueName: \"kubernetes.io/configmap/b95b59af-225c-4907-9643-5d916a325e56-nginx-conf\") pod \"networking-console-plugin-6cf856758c-qg5vl\" (UID: \"b95b59af-225c-4907-9643-5d916a325e56\") " pod="openshift-network-console/networking-console-plugin-6cf856758c-qg5vl" Jul 10 18:03:28.468304 ip-10-0-128-39 kubenswrapper[2558]: I0710 18:03:28.468272 2558 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-7xcrn\" (UniqueName: \"kubernetes.io/projected/2b29fb39-89b3-42c3-b334-3a195242248c-kube-api-access-7xcrn\") pod \"console-operator-7866497494-26542\" (UID: \"2b29fb39-89b3-42c3-b334-3a195242248c\") " pod="openshift-console-operator/console-operator-7866497494-26542" Jul 10 18:03:28.468377 ip-10-0-128-39 kubenswrapper[2558]: I0710 18:03:28.468324 2558 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"serving-cert\" (UniqueName: \"kubernetes.io/secret/eec35344-a5cb-4a6e-9e2e-5adca5a0fa1a-serving-cert\") pod \"kube-storage-version-migrator-operator-799cb7f958-vggpz\" (UID: \"eec35344-a5cb-4a6e-9e2e-5adca5a0fa1a\") " pod="openshift-kube-storage-version-migrator-operator/kube-storage-version-migrator-operator-799cb7f958-vggpz" Jul 10 18:03:28.468377 ip-10-0-128-39 kubenswrapper[2558]: I0710 18:03:28.468352 2558 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"config\" (UniqueName: \"kubernetes.io/configmap/eec35344-a5cb-4a6e-9e2e-5adca5a0fa1a-config\") pod \"kube-storage-version-migrator-operator-799cb7f958-vggpz\" (UID: \"eec35344-a5cb-4a6e-9e2e-5adca5a0fa1a\") " pod="openshift-kube-storage-version-migrator-operator/kube-storage-version-migrator-operator-799cb7f958-vggpz" Jul 10 18:03:28.468460 ip-10-0-128-39 kubenswrapper[2558]: I0710 18:03:28.468412 2558 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/748dca80-632f-44ef-b3a2-5292df366af1-trusted-ca-bundle\") pod \"insights-operator-68d9dd94cf-q5s5m\" (UID: \"748dca80-632f-44ef-b3a2-5292df366af1\") " pod="openshift-insights/insights-operator-68d9dd94cf-q5s5m" Jul 10 18:03:28.468460 ip-10-0-128-39 kubenswrapper[2558]: I0710 18:03:28.468438 2558 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-mlxgd\" (UniqueName: \"kubernetes.io/projected/893dc3d5-4a12-4832-9a96-898388c12ae4-kube-api-access-mlxgd\") pod \"service-ca-operator-7695fc6c8c-hfzjm\" (UID: \"893dc3d5-4a12-4832-9a96-898388c12ae4\") " pod="openshift-service-ca-operator/service-ca-operator-7695fc6c8c-hfzjm" Jul 10 18:03:28.468533 ip-10-0-128-39 kubenswrapper[2558]: I0710 18:03:28.468465 2558 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"trusted-ca\" (UniqueName: \"kubernetes.io/configmap/2b29fb39-89b3-42c3-b334-3a195242248c-trusted-ca\") pod \"console-operator-7866497494-26542\" (UID: \"2b29fb39-89b3-42c3-b334-3a195242248c\") " pod="openshift-console-operator/console-operator-7866497494-26542" Jul 10 18:03:28.468533 ip-10-0-128-39 kubenswrapper[2558]: I0710 18:03:28.468497 2558 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-2ztqb\" (UniqueName: \"kubernetes.io/projected/5df2ceaf-9a85-4d1a-ad19-76df6f2b955f-kube-api-access-2ztqb\") pod \"volume-data-source-validator-78495597f6-v7ckr\" (UID: \"5df2ceaf-9a85-4d1a-ad19-76df6f2b955f\") " pod="openshift-cluster-storage-operator/volume-data-source-validator-78495597f6-v7ckr" Jul 10 18:03:28.468533 ip-10-0-128-39 kubenswrapper[2558]: I0710 18:03:28.468525 2558 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-vqqcp\" (UniqueName: \"kubernetes.io/projected/eec35344-a5cb-4a6e-9e2e-5adca5a0fa1a-kube-api-access-vqqcp\") pod \"kube-storage-version-migrator-operator-799cb7f958-vggpz\" (UID: \"eec35344-a5cb-4a6e-9e2e-5adca5a0fa1a\") " pod="openshift-kube-storage-version-migrator-operator/kube-storage-version-migrator-operator-799cb7f958-vggpz" Jul 10 18:03:28.468655 ip-10-0-128-39 kubenswrapper[2558]: I0710 18:03:28.468554 2558 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"networking-console-plugin-cert\" (UniqueName: \"kubernetes.io/secret/b95b59af-225c-4907-9643-5d916a325e56-networking-console-plugin-cert\") pod \"networking-console-plugin-6cf856758c-qg5vl\" (UID: \"b95b59af-225c-4907-9643-5d916a325e56\") " pod="openshift-network-console/networking-console-plugin-6cf856758c-qg5vl" Jul 10 18:03:28.468655 ip-10-0-128-39 kubenswrapper[2558]: I0710 18:03:28.468580 2558 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"samples-operator-tls\" (UniqueName: \"kubernetes.io/secret/5c7b84a3-17f6-4ed7-bb37-5e5b19a52916-samples-operator-tls\") pod \"cluster-samples-operator-65db858894-2b2km\" (UID: \"5c7b84a3-17f6-4ed7-bb37-5e5b19a52916\") " pod="openshift-cluster-samples-operator/cluster-samples-operator-65db858894-2b2km" Jul 10 18:03:28.468770 ip-10-0-128-39 kubenswrapper[2558]: I0710 18:03:28.468747 2558 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"service-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/748dca80-632f-44ef-b3a2-5292df366af1-service-ca-bundle\") pod \"insights-operator-68d9dd94cf-q5s5m\" (UID: \"748dca80-632f-44ef-b3a2-5292df366af1\") " pod="openshift-insights/insights-operator-68d9dd94cf-q5s5m" Jul 10 18:03:28.469048 ip-10-0-128-39 kubenswrapper[2558]: I0710 18:03:28.469027 2558 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"config\" (UniqueName: \"kubernetes.io/configmap/2b29fb39-89b3-42c3-b334-3a195242248c-config\") pod \"console-operator-7866497494-26542\" (UID: \"2b29fb39-89b3-42c3-b334-3a195242248c\") " pod="openshift-console-operator/console-operator-7866497494-26542" Jul 10 18:03:28.469138 ip-10-0-128-39 kubenswrapper[2558]: E0710 18:03:28.469119 2558 secret.go:189] Couldn't get secret openshift-network-console/networking-console-plugin-cert: secret "networking-console-plugin-cert" not found Jul 10 18:03:28.469216 ip-10-0-128-39 kubenswrapper[2558]: E0710 18:03:28.469190 2558 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/b95b59af-225c-4907-9643-5d916a325e56-networking-console-plugin-cert podName:b95b59af-225c-4907-9643-5d916a325e56 nodeName:}" failed. No retries permitted until 2026-07-10 18:03:28.9691722 +0000 UTC m=+37.472664687 (durationBeforeRetry 500ms). Error: MountVolume.SetUp failed for volume "networking-console-plugin-cert" (UniqueName: "kubernetes.io/secret/b95b59af-225c-4907-9643-5d916a325e56-networking-console-plugin-cert") pod "networking-console-plugin-6cf856758c-qg5vl" (UID: "b95b59af-225c-4907-9643-5d916a325e56") : secret "networking-console-plugin-cert" not found Jul 10 18:03:28.469281 ip-10-0-128-39 kubenswrapper[2558]: I0710 18:03:28.469249 2558 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"config\" (UniqueName: \"kubernetes.io/configmap/893dc3d5-4a12-4832-9a96-898388c12ae4-config\") pod \"service-ca-operator-7695fc6c8c-hfzjm\" (UID: \"893dc3d5-4a12-4832-9a96-898388c12ae4\") " pod="openshift-service-ca-operator/service-ca-operator-7695fc6c8c-hfzjm" Jul 10 18:03:28.469351 ip-10-0-128-39 kubenswrapper[2558]: E0710 18:03:28.469333 2558 secret.go:189] Couldn't get secret openshift-cluster-samples-operator/samples-operator-tls: secret "samples-operator-tls" not found Jul 10 18:03:28.469423 ip-10-0-128-39 kubenswrapper[2558]: E0710 18:03:28.469407 2558 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/5c7b84a3-17f6-4ed7-bb37-5e5b19a52916-samples-operator-tls podName:5c7b84a3-17f6-4ed7-bb37-5e5b19a52916 nodeName:}" failed. No retries permitted until 2026-07-10 18:03:28.969391055 +0000 UTC m=+37.472883541 (durationBeforeRetry 500ms). Error: MountVolume.SetUp failed for volume "samples-operator-tls" (UniqueName: "kubernetes.io/secret/5c7b84a3-17f6-4ed7-bb37-5e5b19a52916-samples-operator-tls") pod "cluster-samples-operator-65db858894-2b2km" (UID: "5c7b84a3-17f6-4ed7-bb37-5e5b19a52916") : secret "samples-operator-tls" not found Jul 10 18:03:28.469663 ip-10-0-128-39 kubenswrapper[2558]: I0710 18:03:28.469643 2558 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"trusted-ca\" (UniqueName: \"kubernetes.io/configmap/2b29fb39-89b3-42c3-b334-3a195242248c-trusted-ca\") pod \"console-operator-7866497494-26542\" (UID: \"2b29fb39-89b3-42c3-b334-3a195242248c\") " pod="openshift-console-operator/console-operator-7866497494-26542" Jul 10 18:03:28.469767 ip-10-0-128-39 kubenswrapper[2558]: I0710 18:03:28.469743 2558 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"config\" (UniqueName: \"kubernetes.io/configmap/eec35344-a5cb-4a6e-9e2e-5adca5a0fa1a-config\") pod \"kube-storage-version-migrator-operator-799cb7f958-vggpz\" (UID: \"eec35344-a5cb-4a6e-9e2e-5adca5a0fa1a\") " pod="openshift-kube-storage-version-migrator-operator/kube-storage-version-migrator-operator-799cb7f958-vggpz" Jul 10 18:03:28.470043 ip-10-0-128-39 kubenswrapper[2558]: I0710 18:03:28.470024 2558 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/748dca80-632f-44ef-b3a2-5292df366af1-trusted-ca-bundle\") pod \"insights-operator-68d9dd94cf-q5s5m\" (UID: \"748dca80-632f-44ef-b3a2-5292df366af1\") " pod="openshift-insights/insights-operator-68d9dd94cf-q5s5m" Jul 10 18:03:28.470165 ip-10-0-128-39 kubenswrapper[2558]: I0710 18:03:28.470144 2558 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"snapshots\" (UniqueName: \"kubernetes.io/empty-dir/748dca80-632f-44ef-b3a2-5292df366af1-snapshots\") pod \"insights-operator-68d9dd94cf-q5s5m\" (UID: \"748dca80-632f-44ef-b3a2-5292df366af1\") " pod="openshift-insights/insights-operator-68d9dd94cf-q5s5m" Jul 10 18:03:28.470651 ip-10-0-128-39 kubenswrapper[2558]: I0710 18:03:28.470627 2558 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"serving-cert\" (UniqueName: \"kubernetes.io/secret/893dc3d5-4a12-4832-9a96-898388c12ae4-serving-cert\") pod \"service-ca-operator-7695fc6c8c-hfzjm\" (UID: \"893dc3d5-4a12-4832-9a96-898388c12ae4\") " pod="openshift-service-ca-operator/service-ca-operator-7695fc6c8c-hfzjm" Jul 10 18:03:28.470988 ip-10-0-128-39 kubenswrapper[2558]: I0710 18:03:28.470952 2558 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"serving-cert\" (UniqueName: \"kubernetes.io/secret/748dca80-632f-44ef-b3a2-5292df366af1-serving-cert\") pod \"insights-operator-68d9dd94cf-q5s5m\" (UID: \"748dca80-632f-44ef-b3a2-5292df366af1\") " pod="openshift-insights/insights-operator-68d9dd94cf-q5s5m" Jul 10 18:03:28.471578 ip-10-0-128-39 kubenswrapper[2558]: I0710 18:03:28.471545 2558 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"serving-cert\" (UniqueName: \"kubernetes.io/secret/2b29fb39-89b3-42c3-b334-3a195242248c-serving-cert\") pod \"console-operator-7866497494-26542\" (UID: \"2b29fb39-89b3-42c3-b334-3a195242248c\") " pod="openshift-console-operator/console-operator-7866497494-26542" Jul 10 18:03:28.471578 ip-10-0-128-39 kubenswrapper[2558]: I0710 18:03:28.471570 2558 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"serving-cert\" (UniqueName: \"kubernetes.io/secret/eec35344-a5cb-4a6e-9e2e-5adca5a0fa1a-serving-cert\") pod \"kube-storage-version-migrator-operator-799cb7f958-vggpz\" (UID: \"eec35344-a5cb-4a6e-9e2e-5adca5a0fa1a\") " pod="openshift-kube-storage-version-migrator-operator/kube-storage-version-migrator-operator-799cb7f958-vggpz" Jul 10 18:03:28.479117 ip-10-0-128-39 kubenswrapper[2558]: I0710 18:03:28.479088 2558 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-qtzrd\" (UniqueName: \"kubernetes.io/projected/5c7b84a3-17f6-4ed7-bb37-5e5b19a52916-kube-api-access-qtzrd\") pod \"cluster-samples-operator-65db858894-2b2km\" (UID: \"5c7b84a3-17f6-4ed7-bb37-5e5b19a52916\") " pod="openshift-cluster-samples-operator/cluster-samples-operator-65db858894-2b2km" Jul 10 18:03:28.479244 ip-10-0-128-39 kubenswrapper[2558]: I0710 18:03:28.479221 2558 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-vqqcp\" (UniqueName: \"kubernetes.io/projected/eec35344-a5cb-4a6e-9e2e-5adca5a0fa1a-kube-api-access-vqqcp\") pod \"kube-storage-version-migrator-operator-799cb7f958-vggpz\" (UID: \"eec35344-a5cb-4a6e-9e2e-5adca5a0fa1a\") " pod="openshift-kube-storage-version-migrator-operator/kube-storage-version-migrator-operator-799cb7f958-vggpz" Jul 10 18:03:28.485005 ip-10-0-128-39 kubenswrapper[2558]: I0710 18:03:28.484142 2558 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-ingress/router-default-cd6bb8744-wq24m"] Jul 10 18:03:28.486023 ip-10-0-128-39 kubenswrapper[2558]: I0710 18:03:28.485962 2558 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-mlxgd\" (UniqueName: \"kubernetes.io/projected/893dc3d5-4a12-4832-9a96-898388c12ae4-kube-api-access-mlxgd\") pod \"service-ca-operator-7695fc6c8c-hfzjm\" (UID: \"893dc3d5-4a12-4832-9a96-898388c12ae4\") " pod="openshift-service-ca-operator/service-ca-operator-7695fc6c8c-hfzjm" Jul 10 18:03:28.487418 ip-10-0-128-39 kubenswrapper[2558]: I0710 18:03:28.487397 2558 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-2ztqb\" (UniqueName: \"kubernetes.io/projected/5df2ceaf-9a85-4d1a-ad19-76df6f2b955f-kube-api-access-2ztqb\") pod \"volume-data-source-validator-78495597f6-v7ckr\" (UID: \"5df2ceaf-9a85-4d1a-ad19-76df6f2b955f\") " pod="openshift-cluster-storage-operator/volume-data-source-validator-78495597f6-v7ckr" Jul 10 18:03:28.488450 ip-10-0-128-39 kubenswrapper[2558]: I0710 18:03:28.488429 2558 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/router-default-cd6bb8744-wq24m" Jul 10 18:03:28.490516 ip-10-0-128-39 kubenswrapper[2558]: I0710 18:03:28.490493 2558 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-v8h6d\" (UniqueName: \"kubernetes.io/projected/748dca80-632f-44ef-b3a2-5292df366af1-kube-api-access-v8h6d\") pod \"insights-operator-68d9dd94cf-q5s5m\" (UID: \"748dca80-632f-44ef-b3a2-5292df366af1\") " pod="openshift-insights/insights-operator-68d9dd94cf-q5s5m" Jul 10 18:03:28.497329 ip-10-0-128-39 kubenswrapper[2558]: I0710 18:03:28.497298 2558 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress\"/\"router-metrics-certs-default\"" Jul 10 18:03:28.499089 ip-10-0-128-39 kubenswrapper[2558]: I0710 18:03:28.499069 2558 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ingress\"/\"service-ca-bundle\"" Jul 10 18:03:28.500304 ip-10-0-128-39 kubenswrapper[2558]: I0710 18:03:28.500280 2558 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-7xcrn\" (UniqueName: \"kubernetes.io/projected/2b29fb39-89b3-42c3-b334-3a195242248c-kube-api-access-7xcrn\") pod \"console-operator-7866497494-26542\" (UID: \"2b29fb39-89b3-42c3-b334-3a195242248c\") " pod="openshift-console-operator/console-operator-7866497494-26542" Jul 10 18:03:28.500752 ip-10-0-128-39 kubenswrapper[2558]: I0710 18:03:28.500734 2558 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ingress\"/\"kube-root-ca.crt\"" Jul 10 18:03:28.500920 ip-10-0-128-39 kubenswrapper[2558]: I0710 18:03:28.500905 2558 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress\"/\"router-dockercfg-w6gqc\"" Jul 10 18:03:28.504688 ip-10-0-128-39 kubenswrapper[2558]: I0710 18:03:28.504667 2558 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress\"/\"router-stats-default\"" Jul 10 18:03:28.504797 ip-10-0-128-39 kubenswrapper[2558]: I0710 18:03:28.504780 2558 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress\"/\"default-ingress-cert\"" Jul 10 18:03:28.505320 ip-10-0-128-39 kubenswrapper[2558]: I0710 18:03:28.505298 2558 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ingress\"/\"openshift-service-ca.crt\"" Jul 10 18:03:28.508992 ip-10-0-128-39 kubenswrapper[2558]: I0710 18:03:28.508971 2558 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/router-default-cd6bb8744-wq24m"] Jul 10 18:03:28.509581 ip-10-0-128-39 kubenswrapper[2558]: I0710 18:03:28.509555 2558 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-dvp86_2da19865-b021-4789-b1e0-694d4f6c543d/konnectivity-agent/0.log" limit=1024 Jul 10 18:03:28.546965 ip-10-0-128-39 kubenswrapper[2558]: I0710 18:03:28.546940 2558 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-source-869dcfcdcd-7n55h" Jul 10 18:03:28.563988 ip-10-0-128-39 kubenswrapper[2558]: I0710 18:03:28.563961 2558 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-cluster-storage-operator/volume-data-source-validator-78495597f6-v7ckr" Jul 10 18:03:28.568961 ip-10-0-128-39 kubenswrapper[2558]: I0710 18:03:28.568942 2558 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"default-certificate\" (UniqueName: \"kubernetes.io/secret/5fab65cc-f578-4cae-919e-1e630186e6d3-default-certificate\") pod \"router-default-cd6bb8744-wq24m\" (UID: \"5fab65cc-f578-4cae-919e-1e630186e6d3\") " pod="openshift-ingress/router-default-cd6bb8744-wq24m" Jul 10 18:03:28.569070 ip-10-0-128-39 kubenswrapper[2558]: I0710 18:03:28.569052 2558 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"stats-auth\" (UniqueName: \"kubernetes.io/secret/5fab65cc-f578-4cae-919e-1e630186e6d3-stats-auth\") pod \"router-default-cd6bb8744-wq24m\" (UID: \"5fab65cc-f578-4cae-919e-1e630186e6d3\") " pod="openshift-ingress/router-default-cd6bb8744-wq24m" Jul 10 18:03:28.569125 ip-10-0-128-39 kubenswrapper[2558]: I0710 18:03:28.569086 2558 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-rqq89\" (UniqueName: \"kubernetes.io/projected/5fab65cc-f578-4cae-919e-1e630186e6d3-kube-api-access-rqq89\") pod \"router-default-cd6bb8744-wq24m\" (UID: \"5fab65cc-f578-4cae-919e-1e630186e6d3\") " pod="openshift-ingress/router-default-cd6bb8744-wq24m" Jul 10 18:03:28.569125 ip-10-0-128-39 kubenswrapper[2558]: I0710 18:03:28.569117 2558 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"service-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/5fab65cc-f578-4cae-919e-1e630186e6d3-service-ca-bundle\") pod \"router-default-cd6bb8744-wq24m\" (UID: \"5fab65cc-f578-4cae-919e-1e630186e6d3\") " pod="openshift-ingress/router-default-cd6bb8744-wq24m" Jul 10 18:03:28.569241 ip-10-0-128-39 kubenswrapper[2558]: I0710 18:03:28.569145 2558 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"registry-tls\" (UniqueName: \"kubernetes.io/projected/4b133a69-01d6-42b5-b12f-d063856c87cb-registry-tls\") pod \"image-registry-cf9777cd6-d9zpc\" (UID: \"4b133a69-01d6-42b5-b12f-d063856c87cb\") " pod="openshift-image-registry/image-registry-cf9777cd6-d9zpc" Jul 10 18:03:28.569293 ip-10-0-128-39 kubenswrapper[2558]: I0710 18:03:28.569234 2558 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-tls\" (UniqueName: \"kubernetes.io/secret/15c4360f-6bc7-4948-b0e6-ebc018174856-metrics-tls\") pod \"dns-default-22khc\" (UID: \"15c4360f-6bc7-4948-b0e6-ebc018174856\") " pod="openshift-dns/dns-default-22khc" Jul 10 18:03:28.569293 ip-10-0-128-39 kubenswrapper[2558]: E0710 18:03:28.569274 2558 projected.go:266] Couldn't get secret openshift-image-registry/image-registry-tls: secret "image-registry-tls" not found Jul 10 18:03:28.569293 ip-10-0-128-39 kubenswrapper[2558]: E0710 18:03:28.569292 2558 projected.go:196] Error preparing data for projected volume registry-tls for pod openshift-image-registry/image-registry-cf9777cd6-d9zpc: secret "image-registry-tls" not found Jul 10 18:03:28.569438 ip-10-0-128-39 kubenswrapper[2558]: E0710 18:03:28.569337 2558 secret.go:189] Couldn't get secret openshift-dns/dns-default-metrics-tls: secret "dns-default-metrics-tls" not found Jul 10 18:03:28.569438 ip-10-0-128-39 kubenswrapper[2558]: E0710 18:03:28.569342 2558 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/projected/4b133a69-01d6-42b5-b12f-d063856c87cb-registry-tls podName:4b133a69-01d6-42b5-b12f-d063856c87cb nodeName:}" failed. No retries permitted until 2026-07-10 18:03:30.56932277 +0000 UTC m=+39.072815269 (durationBeforeRetry 2s). Error: MountVolume.SetUp failed for volume "registry-tls" (UniqueName: "kubernetes.io/projected/4b133a69-01d6-42b5-b12f-d063856c87cb-registry-tls") pod "image-registry-cf9777cd6-d9zpc" (UID: "4b133a69-01d6-42b5-b12f-d063856c87cb") : secret "image-registry-tls" not found Jul 10 18:03:28.569438 ip-10-0-128-39 kubenswrapper[2558]: I0710 18:03:28.569273 2558 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/5fab65cc-f578-4cae-919e-1e630186e6d3-metrics-certs\") pod \"router-default-cd6bb8744-wq24m\" (UID: \"5fab65cc-f578-4cae-919e-1e630186e6d3\") " pod="openshift-ingress/router-default-cd6bb8744-wq24m" Jul 10 18:03:28.569438 ip-10-0-128-39 kubenswrapper[2558]: E0710 18:03:28.569380 2558 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/15c4360f-6bc7-4948-b0e6-ebc018174856-metrics-tls podName:15c4360f-6bc7-4948-b0e6-ebc018174856 nodeName:}" failed. No retries permitted until 2026-07-10 18:03:30.569369877 +0000 UTC m=+39.072862361 (durationBeforeRetry 2s). Error: MountVolume.SetUp failed for volume "metrics-tls" (UniqueName: "kubernetes.io/secret/15c4360f-6bc7-4948-b0e6-ebc018174856-metrics-tls") pod "dns-default-22khc" (UID: "15c4360f-6bc7-4948-b0e6-ebc018174856") : secret "dns-default-metrics-tls" not found Jul 10 18:03:28.570911 ip-10-0-128-39 kubenswrapper[2558]: I0710 18:03:28.570895 2558 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-insights/insights-operator-68d9dd94cf-q5s5m" Jul 10 18:03:28.642533 ip-10-0-128-39 kubenswrapper[2558]: I0710 18:03:28.642441 2558 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-dvp86_2da19865-b021-4789-b1e0-694d4f6c543d/konnectivity-agent/0.log" limit=1024 Jul 10 18:03:28.656553 ip-10-0-128-39 kubenswrapper[2558]: I0710 18:03:28.656521 2558 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-service-ca-operator/service-ca-operator-7695fc6c8c-hfzjm" Jul 10 18:03:28.671198 ip-10-0-128-39 kubenswrapper[2558]: I0710 18:03:28.671168 2558 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"default-certificate\" (UniqueName: \"kubernetes.io/secret/5fab65cc-f578-4cae-919e-1e630186e6d3-default-certificate\") pod \"router-default-cd6bb8744-wq24m\" (UID: \"5fab65cc-f578-4cae-919e-1e630186e6d3\") " pod="openshift-ingress/router-default-cd6bb8744-wq24m" Jul 10 18:03:28.671336 ip-10-0-128-39 kubenswrapper[2558]: I0710 18:03:28.671250 2558 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"stats-auth\" (UniqueName: \"kubernetes.io/secret/5fab65cc-f578-4cae-919e-1e630186e6d3-stats-auth\") pod \"router-default-cd6bb8744-wq24m\" (UID: \"5fab65cc-f578-4cae-919e-1e630186e6d3\") " pod="openshift-ingress/router-default-cd6bb8744-wq24m" Jul 10 18:03:28.671336 ip-10-0-128-39 kubenswrapper[2558]: I0710 18:03:28.671268 2558 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-rqq89\" (UniqueName: \"kubernetes.io/projected/5fab65cc-f578-4cae-919e-1e630186e6d3-kube-api-access-rqq89\") pod \"router-default-cd6bb8744-wq24m\" (UID: \"5fab65cc-f578-4cae-919e-1e630186e6d3\") " pod="openshift-ingress/router-default-cd6bb8744-wq24m" Jul 10 18:03:28.671336 ip-10-0-128-39 kubenswrapper[2558]: I0710 18:03:28.671292 2558 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"service-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/5fab65cc-f578-4cae-919e-1e630186e6d3-service-ca-bundle\") pod \"router-default-cd6bb8744-wq24m\" (UID: \"5fab65cc-f578-4cae-919e-1e630186e6d3\") " pod="openshift-ingress/router-default-cd6bb8744-wq24m" Jul 10 18:03:28.671489 ip-10-0-128-39 kubenswrapper[2558]: I0710 18:03:28.671390 2558 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/5fab65cc-f578-4cae-919e-1e630186e6d3-metrics-certs\") pod \"router-default-cd6bb8744-wq24m\" (UID: \"5fab65cc-f578-4cae-919e-1e630186e6d3\") " pod="openshift-ingress/router-default-cd6bb8744-wq24m" Jul 10 18:03:28.671489 ip-10-0-128-39 kubenswrapper[2558]: E0710 18:03:28.671416 2558 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/configmap/5fab65cc-f578-4cae-919e-1e630186e6d3-service-ca-bundle podName:5fab65cc-f578-4cae-919e-1e630186e6d3 nodeName:}" failed. No retries permitted until 2026-07-10 18:03:29.171397716 +0000 UTC m=+37.674890234 (durationBeforeRetry 500ms). Error: MountVolume.SetUp failed for volume "service-ca-bundle" (UniqueName: "kubernetes.io/configmap/5fab65cc-f578-4cae-919e-1e630186e6d3-service-ca-bundle") pod "router-default-cd6bb8744-wq24m" (UID: "5fab65cc-f578-4cae-919e-1e630186e6d3") : configmap references non-existent config key: service-ca.crt Jul 10 18:03:28.671489 ip-10-0-128-39 kubenswrapper[2558]: E0710 18:03:28.671456 2558 secret.go:189] Couldn't get secret openshift-ingress/router-metrics-certs-default: secret "router-metrics-certs-default" not found Jul 10 18:03:28.671661 ip-10-0-128-39 kubenswrapper[2558]: E0710 18:03:28.671502 2558 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/5fab65cc-f578-4cae-919e-1e630186e6d3-metrics-certs podName:5fab65cc-f578-4cae-919e-1e630186e6d3 nodeName:}" failed. No retries permitted until 2026-07-10 18:03:29.171486953 +0000 UTC m=+37.674979438 (durationBeforeRetry 500ms). Error: MountVolume.SetUp failed for volume "metrics-certs" (UniqueName: "kubernetes.io/secret/5fab65cc-f578-4cae-919e-1e630186e6d3-metrics-certs") pod "router-default-cd6bb8744-wq24m" (UID: "5fab65cc-f578-4cae-919e-1e630186e6d3") : secret "router-metrics-certs-default" not found Jul 10 18:03:28.673554 ip-10-0-128-39 kubenswrapper[2558]: I0710 18:03:28.673530 2558 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"stats-auth\" (UniqueName: \"kubernetes.io/secret/5fab65cc-f578-4cae-919e-1e630186e6d3-stats-auth\") pod \"router-default-cd6bb8744-wq24m\" (UID: \"5fab65cc-f578-4cae-919e-1e630186e6d3\") " pod="openshift-ingress/router-default-cd6bb8744-wq24m" Jul 10 18:03:28.673736 ip-10-0-128-39 kubenswrapper[2558]: I0710 18:03:28.673718 2558 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"default-certificate\" (UniqueName: \"kubernetes.io/secret/5fab65cc-f578-4cae-919e-1e630186e6d3-default-certificate\") pod \"router-default-cd6bb8744-wq24m\" (UID: \"5fab65cc-f578-4cae-919e-1e630186e6d3\") " pod="openshift-ingress/router-default-cd6bb8744-wq24m" Jul 10 18:03:28.676484 ip-10-0-128-39 kubenswrapper[2558]: I0710 18:03:28.676463 2558 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-kube-storage-version-migrator-operator/kube-storage-version-migrator-operator-799cb7f958-vggpz" Jul 10 18:03:28.679593 ip-10-0-128-39 kubenswrapper[2558]: I0710 18:03:28.679572 2558 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-rqq89\" (UniqueName: \"kubernetes.io/projected/5fab65cc-f578-4cae-919e-1e630186e6d3-kube-api-access-rqq89\") pod \"router-default-cd6bb8744-wq24m\" (UID: \"5fab65cc-f578-4cae-919e-1e630186e6d3\") " pod="openshift-ingress/router-default-cd6bb8744-wq24m" Jul 10 18:03:28.695464 ip-10-0-128-39 kubenswrapper[2558]: I0710 18:03:28.695435 2558 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-console-operator/console-operator-7866497494-26542" Jul 10 18:03:28.746591 ip-10-0-128-39 kubenswrapper[2558]: I0710 18:03:28.746517 2558 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-dvp86_2da19865-b021-4789-b1e0-694d4f6c543d/konnectivity-agent/0.log" limit=1024 Jul 10 18:03:28.842390 ip-10-0-128-39 kubenswrapper[2558]: I0710 18:03:28.842352 2558 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-dvp86_2da19865-b021-4789-b1e0-694d4f6c543d/konnectivity-agent/0.log" limit=1024 Jul 10 18:03:28.876902 ip-10-0-128-39 kubenswrapper[2558]: I0710 18:03:28.875332 2558 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"cluster-monitoring-operator-tls\" (UniqueName: \"kubernetes.io/secret/19b8d048-b0b5-4b42-9f4f-ad9f47a105cd-cluster-monitoring-operator-tls\") pod \"cluster-monitoring-operator-5f7f494dd4-46fc9\" (UID: \"19b8d048-b0b5-4b42-9f4f-ad9f47a105cd\") " pod="openshift-monitoring/cluster-monitoring-operator-5f7f494dd4-46fc9" Jul 10 18:03:28.876902 ip-10-0-128-39 kubenswrapper[2558]: E0710 18:03:28.875484 2558 secret.go:189] Couldn't get secret openshift-monitoring/cluster-monitoring-operator-tls: secret "cluster-monitoring-operator-tls" not found Jul 10 18:03:28.876902 ip-10-0-128-39 kubenswrapper[2558]: E0710 18:03:28.875553 2558 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/19b8d048-b0b5-4b42-9f4f-ad9f47a105cd-cluster-monitoring-operator-tls podName:19b8d048-b0b5-4b42-9f4f-ad9f47a105cd nodeName:}" failed. No retries permitted until 2026-07-10 18:03:29.875532465 +0000 UTC m=+38.379024948 (durationBeforeRetry 1s). Error: MountVolume.SetUp failed for volume "cluster-monitoring-operator-tls" (UniqueName: "kubernetes.io/secret/19b8d048-b0b5-4b42-9f4f-ad9f47a105cd-cluster-monitoring-operator-tls") pod "cluster-monitoring-operator-5f7f494dd4-46fc9" (UID: "19b8d048-b0b5-4b42-9f4f-ad9f47a105cd") : secret "cluster-monitoring-operator-tls" not found Jul 10 18:03:28.946269 ip-10-0-128-39 kubenswrapper[2558]: I0710 18:03:28.946121 2558 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-cluster-storage-operator/volume-data-source-validator-78495597f6-v7ckr"] Jul 10 18:03:28.949917 ip-10-0-128-39 kubenswrapper[2558]: I0710 18:03:28.949848 2558 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-insights/insights-operator-68d9dd94cf-q5s5m"] Jul 10 18:03:28.956310 ip-10-0-128-39 kubenswrapper[2558]: I0710 18:03:28.955981 2558 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-network-diagnostics/network-check-source-869dcfcdcd-7n55h"] Jul 10 18:03:28.960066 ip-10-0-128-39 kubenswrapper[2558]: W0710 18:03:28.960007 2558 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod5df2ceaf_9a85_4d1a_ad19_76df6f2b955f.slice/crio-e9ac5e3286392eb8b767c6513eaa070c8d0a9a3ebef55d93ed2b48645b2559bf WatchSource:0}: Error finding container e9ac5e3286392eb8b767c6513eaa070c8d0a9a3ebef55d93ed2b48645b2559bf: Status 404 returned error can't find the container with id e9ac5e3286392eb8b767c6513eaa070c8d0a9a3ebef55d93ed2b48645b2559bf Jul 10 18:03:28.977064 ip-10-0-128-39 kubenswrapper[2558]: I0710 18:03:28.977030 2558 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"networking-console-plugin-cert\" (UniqueName: \"kubernetes.io/secret/b95b59af-225c-4907-9643-5d916a325e56-networking-console-plugin-cert\") pod \"networking-console-plugin-6cf856758c-qg5vl\" (UID: \"b95b59af-225c-4907-9643-5d916a325e56\") " pod="openshift-network-console/networking-console-plugin-6cf856758c-qg5vl" Jul 10 18:03:28.977225 ip-10-0-128-39 kubenswrapper[2558]: I0710 18:03:28.977071 2558 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"samples-operator-tls\" (UniqueName: \"kubernetes.io/secret/5c7b84a3-17f6-4ed7-bb37-5e5b19a52916-samples-operator-tls\") pod \"cluster-samples-operator-65db858894-2b2km\" (UID: \"5c7b84a3-17f6-4ed7-bb37-5e5b19a52916\") " pod="openshift-cluster-samples-operator/cluster-samples-operator-65db858894-2b2km" Jul 10 18:03:28.977225 ip-10-0-128-39 kubenswrapper[2558]: E0710 18:03:28.977158 2558 secret.go:189] Couldn't get secret openshift-network-console/networking-console-plugin-cert: secret "networking-console-plugin-cert" not found Jul 10 18:03:28.977299 ip-10-0-128-39 kubenswrapper[2558]: E0710 18:03:28.977229 2558 secret.go:189] Couldn't get secret openshift-cluster-samples-operator/samples-operator-tls: secret "samples-operator-tls" not found Jul 10 18:03:28.977299 ip-10-0-128-39 kubenswrapper[2558]: E0710 18:03:28.977230 2558 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/b95b59af-225c-4907-9643-5d916a325e56-networking-console-plugin-cert podName:b95b59af-225c-4907-9643-5d916a325e56 nodeName:}" failed. No retries permitted until 2026-07-10 18:03:29.977210224 +0000 UTC m=+38.480702729 (durationBeforeRetry 1s). Error: MountVolume.SetUp failed for volume "networking-console-plugin-cert" (UniqueName: "kubernetes.io/secret/b95b59af-225c-4907-9643-5d916a325e56-networking-console-plugin-cert") pod "networking-console-plugin-6cf856758c-qg5vl" (UID: "b95b59af-225c-4907-9643-5d916a325e56") : secret "networking-console-plugin-cert" not found Jul 10 18:03:28.977299 ip-10-0-128-39 kubenswrapper[2558]: E0710 18:03:28.977274 2558 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/5c7b84a3-17f6-4ed7-bb37-5e5b19a52916-samples-operator-tls podName:5c7b84a3-17f6-4ed7-bb37-5e5b19a52916 nodeName:}" failed. No retries permitted until 2026-07-10 18:03:29.977263464 +0000 UTC m=+38.480755963 (durationBeforeRetry 1s). Error: MountVolume.SetUp failed for volume "samples-operator-tls" (UniqueName: "kubernetes.io/secret/5c7b84a3-17f6-4ed7-bb37-5e5b19a52916-samples-operator-tls") pod "cluster-samples-operator-65db858894-2b2km" (UID: "5c7b84a3-17f6-4ed7-bb37-5e5b19a52916") : secret "samples-operator-tls" not found Jul 10 18:03:28.986564 ip-10-0-128-39 kubenswrapper[2558]: I0710 18:03:28.986533 2558 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-service-ca-operator/service-ca-operator-7695fc6c8c-hfzjm"] Jul 10 18:03:28.992038 ip-10-0-128-39 kubenswrapper[2558]: I0710 18:03:28.992010 2558 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-kube-storage-version-migrator-operator/kube-storage-version-migrator-operator-799cb7f958-vggpz"] Jul 10 18:03:29.000298 ip-10-0-128-39 kubenswrapper[2558]: I0710 18:03:29.000275 2558 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress-canary/ingress-canary-bpwbf"] Jul 10 18:03:29.006690 ip-10-0-128-39 kubenswrapper[2558]: I0710 18:03:29.006656 2558 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-console-operator/console-operator-7866497494-26542"] Jul 10 18:03:29.010025 ip-10-0-128-39 kubenswrapper[2558]: W0710 18:03:29.009998 2558 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podf754e509_abbe_4451_833d_abb0e28c7e19.slice/crio-81d667269e057c5b843b940ddaac7c3c7dd0a961fe39fadc368890e6617a50f8 WatchSource:0}: Error finding container 81d667269e057c5b843b940ddaac7c3c7dd0a961fe39fadc368890e6617a50f8: Status 404 returned error can't find the container with id 81d667269e057c5b843b940ddaac7c3c7dd0a961fe39fadc368890e6617a50f8 Jul 10 18:03:29.010938 ip-10-0-128-39 kubenswrapper[2558]: W0710 18:03:29.010911 2558 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod5997642b_76cf_44e6_8b06_138718112a30.slice/crio-211d76c8efc5c3cbdf1533742c557d03bcac3399caf3b0ddbb8622c187183835 WatchSource:0}: Error finding container 211d76c8efc5c3cbdf1533742c557d03bcac3399caf3b0ddbb8622c187183835: Status 404 returned error can't find the container with id 211d76c8efc5c3cbdf1533742c557d03bcac3399caf3b0ddbb8622c187183835 Jul 10 18:03:29.011641 ip-10-0-128-39 kubenswrapper[2558]: W0710 18:03:29.011616 2558 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod2b29fb39_89b3_42c3_b334_3a195242248c.slice/crio-98046f121db629519aa8172c683f827a5b84f8169b3b9d89c27f82172e487d76 WatchSource:0}: Error finding container 98046f121db629519aa8172c683f827a5b84f8169b3b9d89c27f82172e487d76: Status 404 returned error can't find the container with id 98046f121db629519aa8172c683f827a5b84f8169b3b9d89c27f82172e487d76 Jul 10 18:03:29.179116 ip-10-0-128-39 kubenswrapper[2558]: I0710 18:03:29.179027 2558 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"service-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/5fab65cc-f578-4cae-919e-1e630186e6d3-service-ca-bundle\") pod \"router-default-cd6bb8744-wq24m\" (UID: \"5fab65cc-f578-4cae-919e-1e630186e6d3\") " pod="openshift-ingress/router-default-cd6bb8744-wq24m" Jul 10 18:03:29.179116 ip-10-0-128-39 kubenswrapper[2558]: I0710 18:03:29.179086 2558 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/5fab65cc-f578-4cae-919e-1e630186e6d3-metrics-certs\") pod \"router-default-cd6bb8744-wq24m\" (UID: \"5fab65cc-f578-4cae-919e-1e630186e6d3\") " pod="openshift-ingress/router-default-cd6bb8744-wq24m" Jul 10 18:03:29.179530 ip-10-0-128-39 kubenswrapper[2558]: E0710 18:03:29.179170 2558 secret.go:189] Couldn't get secret openshift-ingress/router-metrics-certs-default: secret "router-metrics-certs-default" not found Jul 10 18:03:29.179530 ip-10-0-128-39 kubenswrapper[2558]: E0710 18:03:29.179194 2558 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/configmap/5fab65cc-f578-4cae-919e-1e630186e6d3-service-ca-bundle podName:5fab65cc-f578-4cae-919e-1e630186e6d3 nodeName:}" failed. No retries permitted until 2026-07-10 18:03:30.179177575 +0000 UTC m=+38.682670058 (durationBeforeRetry 1s). Error: MountVolume.SetUp failed for volume "service-ca-bundle" (UniqueName: "kubernetes.io/configmap/5fab65cc-f578-4cae-919e-1e630186e6d3-service-ca-bundle") pod "router-default-cd6bb8744-wq24m" (UID: "5fab65cc-f578-4cae-919e-1e630186e6d3") : configmap references non-existent config key: service-ca.crt Jul 10 18:03:29.179530 ip-10-0-128-39 kubenswrapper[2558]: E0710 18:03:29.179279 2558 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/5fab65cc-f578-4cae-919e-1e630186e6d3-metrics-certs podName:5fab65cc-f578-4cae-919e-1e630186e6d3 nodeName:}" failed. No retries permitted until 2026-07-10 18:03:30.179270254 +0000 UTC m=+38.682762737 (durationBeforeRetry 1s). Error: MountVolume.SetUp failed for volume "metrics-certs" (UniqueName: "kubernetes.io/secret/5fab65cc-f578-4cae-919e-1e630186e6d3-metrics-certs") pod "router-default-cd6bb8744-wq24m" (UID: "5fab65cc-f578-4cae-919e-1e630186e6d3") : secret "router-metrics-certs-default" not found Jul 10 18:03:29.265948 ip-10-0-128-39 kubenswrapper[2558]: I0710 18:03:29.265913 2558 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-kube-storage-version-migrator-operator/kube-storage-version-migrator-operator-799cb7f958-vggpz" event={"ID":"eec35344-a5cb-4a6e-9e2e-5adca5a0fa1a","Type":"ContainerStarted","Data":"d35e71bd9a9130975a409166c0850a982dd3aad79fea77a51a4e60b00cb97296"} Jul 10 18:03:29.267044 ip-10-0-128-39 kubenswrapper[2558]: I0710 18:03:29.267015 2558 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-service-ca-operator/service-ca-operator-7695fc6c8c-hfzjm" event={"ID":"893dc3d5-4a12-4832-9a96-898388c12ae4","Type":"ContainerStarted","Data":"f686c32564fcf0a3262e0493f2a51864fd8f2d552046e0e06bfc4b7da0abcc63"} Jul 10 18:03:29.268005 ip-10-0-128-39 kubenswrapper[2558]: I0710 18:03:29.267972 2558 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-console-operator/console-operator-7866497494-26542" event={"ID":"2b29fb39-89b3-42c3-b334-3a195242248c","Type":"ContainerStarted","Data":"98046f121db629519aa8172c683f827a5b84f8169b3b9d89c27f82172e487d76"} Jul 10 18:03:29.269046 ip-10-0-128-39 kubenswrapper[2558]: I0710 18:03:29.269023 2558 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress-canary/ingress-canary-bpwbf" event={"ID":"f754e509-abbe-4451-833d-abb0e28c7e19","Type":"ContainerStarted","Data":"81d667269e057c5b843b940ddaac7c3c7dd0a961fe39fadc368890e6617a50f8"} Jul 10 18:03:29.270034 ip-10-0-128-39 kubenswrapper[2558]: I0710 18:03:29.270014 2558 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-cluster-storage-operator/volume-data-source-validator-78495597f6-v7ckr" event={"ID":"5df2ceaf-9a85-4d1a-ad19-76df6f2b955f","Type":"ContainerStarted","Data":"e9ac5e3286392eb8b767c6513eaa070c8d0a9a3ebef55d93ed2b48645b2559bf"} Jul 10 18:03:29.272572 ip-10-0-128-39 kubenswrapper[2558]: I0710 18:03:29.272553 2558 generic.go:356] "Generic (PLEG): container finished" podID="53492c2f-2433-4658-af05-d58878aed082" containerID="cbf06aaa4210931377684a36a48625568aeeb372ca8b3fa022e12ab90ec97139" exitCode=0 Jul 10 18:03:29.272652 ip-10-0-128-39 kubenswrapper[2558]: I0710 18:03:29.272613 2558 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-multus/multus-additional-cni-plugins-v22pb" event={"ID":"53492c2f-2433-4658-af05-d58878aed082","Type":"ContainerDied","Data":"cbf06aaa4210931377684a36a48625568aeeb372ca8b3fa022e12ab90ec97139"} Jul 10 18:03:29.273651 ip-10-0-128-39 kubenswrapper[2558]: I0710 18:03:29.273633 2558 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-insights/insights-operator-68d9dd94cf-q5s5m" event={"ID":"748dca80-632f-44ef-b3a2-5292df366af1","Type":"ContainerStarted","Data":"bfc2dd062b43f91b845519d0848455c89c6fc12d97bb19dd224e876f7c3e641f"} Jul 10 18:03:29.274832 ip-10-0-128-39 kubenswrapper[2558]: I0710 18:03:29.274796 2558 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-network-diagnostics/network-check-source-869dcfcdcd-7n55h" event={"ID":"5997642b-76cf-44e6-8b06-138718112a30","Type":"ContainerStarted","Data":"211d76c8efc5c3cbdf1533742c557d03bcac3399caf3b0ddbb8622c187183835"} Jul 10 18:03:29.887318 ip-10-0-128-39 kubenswrapper[2558]: I0710 18:03:29.886953 2558 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"cluster-monitoring-operator-tls\" (UniqueName: \"kubernetes.io/secret/19b8d048-b0b5-4b42-9f4f-ad9f47a105cd-cluster-monitoring-operator-tls\") pod \"cluster-monitoring-operator-5f7f494dd4-46fc9\" (UID: \"19b8d048-b0b5-4b42-9f4f-ad9f47a105cd\") " pod="openshift-monitoring/cluster-monitoring-operator-5f7f494dd4-46fc9" Jul 10 18:03:29.887318 ip-10-0-128-39 kubenswrapper[2558]: E0710 18:03:29.887230 2558 secret.go:189] Couldn't get secret openshift-monitoring/cluster-monitoring-operator-tls: secret "cluster-monitoring-operator-tls" not found Jul 10 18:03:29.887318 ip-10-0-128-39 kubenswrapper[2558]: E0710 18:03:29.887308 2558 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/19b8d048-b0b5-4b42-9f4f-ad9f47a105cd-cluster-monitoring-operator-tls podName:19b8d048-b0b5-4b42-9f4f-ad9f47a105cd nodeName:}" failed. No retries permitted until 2026-07-10 18:03:31.887286307 +0000 UTC m=+40.390778797 (durationBeforeRetry 2s). Error: MountVolume.SetUp failed for volume "cluster-monitoring-operator-tls" (UniqueName: "kubernetes.io/secret/19b8d048-b0b5-4b42-9f4f-ad9f47a105cd-cluster-monitoring-operator-tls") pod "cluster-monitoring-operator-5f7f494dd4-46fc9" (UID: "19b8d048-b0b5-4b42-9f4f-ad9f47a105cd") : secret "cluster-monitoring-operator-tls" not found Jul 10 18:03:29.989474 ip-10-0-128-39 kubenswrapper[2558]: I0710 18:03:29.988566 2558 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"networking-console-plugin-cert\" (UniqueName: \"kubernetes.io/secret/b95b59af-225c-4907-9643-5d916a325e56-networking-console-plugin-cert\") pod \"networking-console-plugin-6cf856758c-qg5vl\" (UID: \"b95b59af-225c-4907-9643-5d916a325e56\") " pod="openshift-network-console/networking-console-plugin-6cf856758c-qg5vl" Jul 10 18:03:29.989474 ip-10-0-128-39 kubenswrapper[2558]: I0710 18:03:29.988622 2558 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"samples-operator-tls\" (UniqueName: \"kubernetes.io/secret/5c7b84a3-17f6-4ed7-bb37-5e5b19a52916-samples-operator-tls\") pod \"cluster-samples-operator-65db858894-2b2km\" (UID: \"5c7b84a3-17f6-4ed7-bb37-5e5b19a52916\") " pod="openshift-cluster-samples-operator/cluster-samples-operator-65db858894-2b2km" Jul 10 18:03:29.989474 ip-10-0-128-39 kubenswrapper[2558]: E0710 18:03:29.988907 2558 secret.go:189] Couldn't get secret openshift-cluster-samples-operator/samples-operator-tls: secret "samples-operator-tls" not found Jul 10 18:03:29.989474 ip-10-0-128-39 kubenswrapper[2558]: E0710 18:03:29.988977 2558 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/5c7b84a3-17f6-4ed7-bb37-5e5b19a52916-samples-operator-tls podName:5c7b84a3-17f6-4ed7-bb37-5e5b19a52916 nodeName:}" failed. No retries permitted until 2026-07-10 18:03:31.988956416 +0000 UTC m=+40.492448902 (durationBeforeRetry 2s). Error: MountVolume.SetUp failed for volume "samples-operator-tls" (UniqueName: "kubernetes.io/secret/5c7b84a3-17f6-4ed7-bb37-5e5b19a52916-samples-operator-tls") pod "cluster-samples-operator-65db858894-2b2km" (UID: "5c7b84a3-17f6-4ed7-bb37-5e5b19a52916") : secret "samples-operator-tls" not found Jul 10 18:03:29.989474 ip-10-0-128-39 kubenswrapper[2558]: E0710 18:03:29.989379 2558 secret.go:189] Couldn't get secret openshift-network-console/networking-console-plugin-cert: secret "networking-console-plugin-cert" not found Jul 10 18:03:29.989474 ip-10-0-128-39 kubenswrapper[2558]: E0710 18:03:29.989431 2558 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/b95b59af-225c-4907-9643-5d916a325e56-networking-console-plugin-cert podName:b95b59af-225c-4907-9643-5d916a325e56 nodeName:}" failed. No retries permitted until 2026-07-10 18:03:31.989416075 +0000 UTC m=+40.492908564 (durationBeforeRetry 2s). Error: MountVolume.SetUp failed for volume "networking-console-plugin-cert" (UniqueName: "kubernetes.io/secret/b95b59af-225c-4907-9643-5d916a325e56-networking-console-plugin-cert") pod "networking-console-plugin-6cf856758c-qg5vl" (UID: "b95b59af-225c-4907-9643-5d916a325e56") : secret "networking-console-plugin-cert" not found Jul 10 18:03:30.191986 ip-10-0-128-39 kubenswrapper[2558]: I0710 18:03:30.190990 2558 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"service-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/5fab65cc-f578-4cae-919e-1e630186e6d3-service-ca-bundle\") pod \"router-default-cd6bb8744-wq24m\" (UID: \"5fab65cc-f578-4cae-919e-1e630186e6d3\") " pod="openshift-ingress/router-default-cd6bb8744-wq24m" Jul 10 18:03:30.191986 ip-10-0-128-39 kubenswrapper[2558]: I0710 18:03:30.191076 2558 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/5fab65cc-f578-4cae-919e-1e630186e6d3-metrics-certs\") pod \"router-default-cd6bb8744-wq24m\" (UID: \"5fab65cc-f578-4cae-919e-1e630186e6d3\") " pod="openshift-ingress/router-default-cd6bb8744-wq24m" Jul 10 18:03:30.191986 ip-10-0-128-39 kubenswrapper[2558]: E0710 18:03:30.191273 2558 secret.go:189] Couldn't get secret openshift-ingress/router-metrics-certs-default: secret "router-metrics-certs-default" not found Jul 10 18:03:30.191986 ip-10-0-128-39 kubenswrapper[2558]: E0710 18:03:30.191338 2558 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/5fab65cc-f578-4cae-919e-1e630186e6d3-metrics-certs podName:5fab65cc-f578-4cae-919e-1e630186e6d3 nodeName:}" failed. No retries permitted until 2026-07-10 18:03:32.19131646 +0000 UTC m=+40.694808967 (durationBeforeRetry 2s). Error: MountVolume.SetUp failed for volume "metrics-certs" (UniqueName: "kubernetes.io/secret/5fab65cc-f578-4cae-919e-1e630186e6d3-metrics-certs") pod "router-default-cd6bb8744-wq24m" (UID: "5fab65cc-f578-4cae-919e-1e630186e6d3") : secret "router-metrics-certs-default" not found Jul 10 18:03:30.191986 ip-10-0-128-39 kubenswrapper[2558]: E0710 18:03:30.191821 2558 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/configmap/5fab65cc-f578-4cae-919e-1e630186e6d3-service-ca-bundle podName:5fab65cc-f578-4cae-919e-1e630186e6d3 nodeName:}" failed. No retries permitted until 2026-07-10 18:03:32.191789659 +0000 UTC m=+40.695282160 (durationBeforeRetry 2s). Error: MountVolume.SetUp failed for volume "service-ca-bundle" (UniqueName: "kubernetes.io/configmap/5fab65cc-f578-4cae-919e-1e630186e6d3-service-ca-bundle") pod "router-default-cd6bb8744-wq24m" (UID: "5fab65cc-f578-4cae-919e-1e630186e6d3") : configmap references non-existent config key: service-ca.crt Jul 10 18:03:30.294462 ip-10-0-128-39 kubenswrapper[2558]: I0710 18:03:30.293534 2558 generic.go:356] "Generic (PLEG): container finished" podID="53492c2f-2433-4658-af05-d58878aed082" containerID="78ccfd7c84b5b8583cacd5f2c8f8ed92729de937304c3be79e5d3d870987a02b" exitCode=0 Jul 10 18:03:30.294462 ip-10-0-128-39 kubenswrapper[2558]: I0710 18:03:30.293587 2558 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-multus/multus-additional-cni-plugins-v22pb" event={"ID":"53492c2f-2433-4658-af05-d58878aed082","Type":"ContainerDied","Data":"78ccfd7c84b5b8583cacd5f2c8f8ed92729de937304c3be79e5d3d870987a02b"} Jul 10 18:03:30.596059 ip-10-0-128-39 kubenswrapper[2558]: I0710 18:03:30.595258 2558 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"registry-tls\" (UniqueName: \"kubernetes.io/projected/4b133a69-01d6-42b5-b12f-d063856c87cb-registry-tls\") pod \"image-registry-cf9777cd6-d9zpc\" (UID: \"4b133a69-01d6-42b5-b12f-d063856c87cb\") " pod="openshift-image-registry/image-registry-cf9777cd6-d9zpc" Jul 10 18:03:30.596059 ip-10-0-128-39 kubenswrapper[2558]: I0710 18:03:30.595309 2558 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-tls\" (UniqueName: \"kubernetes.io/secret/15c4360f-6bc7-4948-b0e6-ebc018174856-metrics-tls\") pod \"dns-default-22khc\" (UID: \"15c4360f-6bc7-4948-b0e6-ebc018174856\") " pod="openshift-dns/dns-default-22khc" Jul 10 18:03:30.596059 ip-10-0-128-39 kubenswrapper[2558]: E0710 18:03:30.595466 2558 secret.go:189] Couldn't get secret openshift-dns/dns-default-metrics-tls: secret "dns-default-metrics-tls" not found Jul 10 18:03:30.596059 ip-10-0-128-39 kubenswrapper[2558]: E0710 18:03:30.595533 2558 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/15c4360f-6bc7-4948-b0e6-ebc018174856-metrics-tls podName:15c4360f-6bc7-4948-b0e6-ebc018174856 nodeName:}" failed. No retries permitted until 2026-07-10 18:03:34.595513096 +0000 UTC m=+43.099005605 (durationBeforeRetry 4s). Error: MountVolume.SetUp failed for volume "metrics-tls" (UniqueName: "kubernetes.io/secret/15c4360f-6bc7-4948-b0e6-ebc018174856-metrics-tls") pod "dns-default-22khc" (UID: "15c4360f-6bc7-4948-b0e6-ebc018174856") : secret "dns-default-metrics-tls" not found Jul 10 18:03:30.596059 ip-10-0-128-39 kubenswrapper[2558]: E0710 18:03:30.595958 2558 projected.go:266] Couldn't get secret openshift-image-registry/image-registry-tls: secret "image-registry-tls" not found Jul 10 18:03:30.596059 ip-10-0-128-39 kubenswrapper[2558]: E0710 18:03:30.595976 2558 projected.go:196] Error preparing data for projected volume registry-tls for pod openshift-image-registry/image-registry-cf9777cd6-d9zpc: secret "image-registry-tls" not found Jul 10 18:03:30.596059 ip-10-0-128-39 kubenswrapper[2558]: E0710 18:03:30.596026 2558 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/projected/4b133a69-01d6-42b5-b12f-d063856c87cb-registry-tls podName:4b133a69-01d6-42b5-b12f-d063856c87cb nodeName:}" failed. No retries permitted until 2026-07-10 18:03:34.596009585 +0000 UTC m=+43.099502090 (durationBeforeRetry 4s). Error: MountVolume.SetUp failed for volume "registry-tls" (UniqueName: "kubernetes.io/projected/4b133a69-01d6-42b5-b12f-d063856c87cb-registry-tls") pod "image-registry-cf9777cd6-d9zpc" (UID: "4b133a69-01d6-42b5-b12f-d063856c87cb") : secret "image-registry-tls" not found Jul 10 18:03:31.305665 ip-10-0-128-39 kubenswrapper[2558]: I0710 18:03:31.304994 2558 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-multus/multus-additional-cni-plugins-v22pb" event={"ID":"53492c2f-2433-4658-af05-d58878aed082","Type":"ContainerStarted","Data":"59a22a20b6ee35b88513b5c241c977c0fa1597c49915957067c02776133cb72a"} Jul 10 18:03:31.330731 ip-10-0-128-39 kubenswrapper[2558]: I0710 18:03:31.329958 2558 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-multus/multus-additional-cni-plugins-v22pb" podStartSLOduration=7.4917357110000005 podStartE2EDuration="39.329913026s" totalImagesPullingTime="31.838177315s" totalInitContainerRuntime="0s" isStatefulPod=true podCreationTimestamp="2026-07-10 18:02:52 +0000 UTC" imagePullSessionsCount=5 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-10 18:03:31.328224169 +0000 UTC m=+39.831716678" watchObservedRunningTime="2026-07-10 18:03:31.329913026 +0000 UTC m=+39.833405533" Jul 10 18:03:31.453669 ip-10-0-128-39 kubenswrapper[2558]: I0710 18:03:31.453632 2558 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-dvp86_2da19865-b021-4789-b1e0-694d4f6c543d/konnectivity-agent/0.log" limit=1024 Jul 10 18:03:31.553265 ip-10-0-128-39 kubenswrapper[2558]: I0710 18:03:31.553224 2558 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-dvp86_2da19865-b021-4789-b1e0-694d4f6c543d/konnectivity-agent/0.log" limit=1024 Jul 10 18:03:31.733215 ip-10-0-128-39 kubenswrapper[2558]: I0710 18:03:31.733098 2558 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-dvp86_2da19865-b021-4789-b1e0-694d4f6c543d/konnectivity-agent/0.log" limit=1024 Jul 10 18:03:31.834839 ip-10-0-128-39 kubenswrapper[2558]: I0710 18:03:31.834745 2558 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-dvp86_2da19865-b021-4789-b1e0-694d4f6c543d/konnectivity-agent/0.log" limit=1024 Jul 10 18:03:31.902347 ip-10-0-128-39 kubenswrapper[2558]: I0710 18:03:31.902311 2558 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-dvp86_2da19865-b021-4789-b1e0-694d4f6c543d/konnectivity-agent/0.log" limit=1024 Jul 10 18:03:31.911052 ip-10-0-128-39 kubenswrapper[2558]: I0710 18:03:31.910977 2558 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"cluster-monitoring-operator-tls\" (UniqueName: \"kubernetes.io/secret/19b8d048-b0b5-4b42-9f4f-ad9f47a105cd-cluster-monitoring-operator-tls\") pod \"cluster-monitoring-operator-5f7f494dd4-46fc9\" (UID: \"19b8d048-b0b5-4b42-9f4f-ad9f47a105cd\") " pod="openshift-monitoring/cluster-monitoring-operator-5f7f494dd4-46fc9" Jul 10 18:03:31.911186 ip-10-0-128-39 kubenswrapper[2558]: E0710 18:03:31.911135 2558 secret.go:189] Couldn't get secret openshift-monitoring/cluster-monitoring-operator-tls: secret "cluster-monitoring-operator-tls" not found Jul 10 18:03:31.911250 ip-10-0-128-39 kubenswrapper[2558]: E0710 18:03:31.911194 2558 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/19b8d048-b0b5-4b42-9f4f-ad9f47a105cd-cluster-monitoring-operator-tls podName:19b8d048-b0b5-4b42-9f4f-ad9f47a105cd nodeName:}" failed. No retries permitted until 2026-07-10 18:03:35.911176022 +0000 UTC m=+44.414668520 (durationBeforeRetry 4s). Error: MountVolume.SetUp failed for volume "cluster-monitoring-operator-tls" (UniqueName: "kubernetes.io/secret/19b8d048-b0b5-4b42-9f4f-ad9f47a105cd-cluster-monitoring-operator-tls") pod "cluster-monitoring-operator-5f7f494dd4-46fc9" (UID: "19b8d048-b0b5-4b42-9f4f-ad9f47a105cd") : secret "cluster-monitoring-operator-tls" not found Jul 10 18:03:32.012601 ip-10-0-128-39 kubenswrapper[2558]: I0710 18:03:32.012317 2558 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"networking-console-plugin-cert\" (UniqueName: \"kubernetes.io/secret/b95b59af-225c-4907-9643-5d916a325e56-networking-console-plugin-cert\") pod \"networking-console-plugin-6cf856758c-qg5vl\" (UID: \"b95b59af-225c-4907-9643-5d916a325e56\") " pod="openshift-network-console/networking-console-plugin-6cf856758c-qg5vl" Jul 10 18:03:32.012601 ip-10-0-128-39 kubenswrapper[2558]: I0710 18:03:32.012369 2558 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"samples-operator-tls\" (UniqueName: \"kubernetes.io/secret/5c7b84a3-17f6-4ed7-bb37-5e5b19a52916-samples-operator-tls\") pod \"cluster-samples-operator-65db858894-2b2km\" (UID: \"5c7b84a3-17f6-4ed7-bb37-5e5b19a52916\") " pod="openshift-cluster-samples-operator/cluster-samples-operator-65db858894-2b2km" Jul 10 18:03:32.012601 ip-10-0-128-39 kubenswrapper[2558]: E0710 18:03:32.012483 2558 secret.go:189] Couldn't get secret openshift-cluster-samples-operator/samples-operator-tls: secret "samples-operator-tls" not found Jul 10 18:03:32.012601 ip-10-0-128-39 kubenswrapper[2558]: E0710 18:03:32.012543 2558 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/5c7b84a3-17f6-4ed7-bb37-5e5b19a52916-samples-operator-tls podName:5c7b84a3-17f6-4ed7-bb37-5e5b19a52916 nodeName:}" failed. No retries permitted until 2026-07-10 18:03:36.012523223 +0000 UTC m=+44.516015711 (durationBeforeRetry 4s). Error: MountVolume.SetUp failed for volume "samples-operator-tls" (UniqueName: "kubernetes.io/secret/5c7b84a3-17f6-4ed7-bb37-5e5b19a52916-samples-operator-tls") pod "cluster-samples-operator-65db858894-2b2km" (UID: "5c7b84a3-17f6-4ed7-bb37-5e5b19a52916") : secret "samples-operator-tls" not found Jul 10 18:03:32.012601 ip-10-0-128-39 kubenswrapper[2558]: E0710 18:03:32.012564 2558 secret.go:189] Couldn't get secret openshift-network-console/networking-console-plugin-cert: secret "networking-console-plugin-cert" not found Jul 10 18:03:32.012978 ip-10-0-128-39 kubenswrapper[2558]: E0710 18:03:32.012622 2558 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/b95b59af-225c-4907-9643-5d916a325e56-networking-console-plugin-cert podName:b95b59af-225c-4907-9643-5d916a325e56 nodeName:}" failed. No retries permitted until 2026-07-10 18:03:36.012603347 +0000 UTC m=+44.516095844 (durationBeforeRetry 4s). Error: MountVolume.SetUp failed for volume "networking-console-plugin-cert" (UniqueName: "kubernetes.io/secret/b95b59af-225c-4907-9643-5d916a325e56-networking-console-plugin-cert") pod "networking-console-plugin-6cf856758c-qg5vl" (UID: "b95b59af-225c-4907-9643-5d916a325e56") : secret "networking-console-plugin-cert" not found Jul 10 18:03:32.213866 ip-10-0-128-39 kubenswrapper[2558]: I0710 18:03:32.213833 2558 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"service-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/5fab65cc-f578-4cae-919e-1e630186e6d3-service-ca-bundle\") pod \"router-default-cd6bb8744-wq24m\" (UID: \"5fab65cc-f578-4cae-919e-1e630186e6d3\") " pod="openshift-ingress/router-default-cd6bb8744-wq24m" Jul 10 18:03:32.214019 ip-10-0-128-39 kubenswrapper[2558]: I0710 18:03:32.213914 2558 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/5fab65cc-f578-4cae-919e-1e630186e6d3-metrics-certs\") pod \"router-default-cd6bb8744-wq24m\" (UID: \"5fab65cc-f578-4cae-919e-1e630186e6d3\") " pod="openshift-ingress/router-default-cd6bb8744-wq24m" Jul 10 18:03:32.214079 ip-10-0-128-39 kubenswrapper[2558]: E0710 18:03:32.214038 2558 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/configmap/5fab65cc-f578-4cae-919e-1e630186e6d3-service-ca-bundle podName:5fab65cc-f578-4cae-919e-1e630186e6d3 nodeName:}" failed. No retries permitted until 2026-07-10 18:03:36.214013476 +0000 UTC m=+44.717505963 (durationBeforeRetry 4s). Error: MountVolume.SetUp failed for volume "service-ca-bundle" (UniqueName: "kubernetes.io/configmap/5fab65cc-f578-4cae-919e-1e630186e6d3-service-ca-bundle") pod "router-default-cd6bb8744-wq24m" (UID: "5fab65cc-f578-4cae-919e-1e630186e6d3") : configmap references non-existent config key: service-ca.crt Jul 10 18:03:32.214079 ip-10-0-128-39 kubenswrapper[2558]: E0710 18:03:32.214065 2558 secret.go:189] Couldn't get secret openshift-ingress/router-metrics-certs-default: secret "router-metrics-certs-default" not found Jul 10 18:03:32.214180 ip-10-0-128-39 kubenswrapper[2558]: E0710 18:03:32.214116 2558 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/5fab65cc-f578-4cae-919e-1e630186e6d3-metrics-certs podName:5fab65cc-f578-4cae-919e-1e630186e6d3 nodeName:}" failed. No retries permitted until 2026-07-10 18:03:36.21410296 +0000 UTC m=+44.717595450 (durationBeforeRetry 4s). Error: MountVolume.SetUp failed for volume "metrics-certs" (UniqueName: "kubernetes.io/secret/5fab65cc-f578-4cae-919e-1e630186e6d3-metrics-certs") pod "router-default-cd6bb8744-wq24m" (UID: "5fab65cc-f578-4cae-919e-1e630186e6d3") : secret "router-metrics-certs-default" not found Jul 10 18:03:34.637211 ip-10-0-128-39 kubenswrapper[2558]: I0710 18:03:34.637170 2558 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"registry-tls\" (UniqueName: \"kubernetes.io/projected/4b133a69-01d6-42b5-b12f-d063856c87cb-registry-tls\") pod \"image-registry-cf9777cd6-d9zpc\" (UID: \"4b133a69-01d6-42b5-b12f-d063856c87cb\") " pod="openshift-image-registry/image-registry-cf9777cd6-d9zpc" Jul 10 18:03:34.637629 ip-10-0-128-39 kubenswrapper[2558]: I0710 18:03:34.637221 2558 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-tls\" (UniqueName: \"kubernetes.io/secret/15c4360f-6bc7-4948-b0e6-ebc018174856-metrics-tls\") pod \"dns-default-22khc\" (UID: \"15c4360f-6bc7-4948-b0e6-ebc018174856\") " pod="openshift-dns/dns-default-22khc" Jul 10 18:03:34.637629 ip-10-0-128-39 kubenswrapper[2558]: E0710 18:03:34.637327 2558 secret.go:189] Couldn't get secret openshift-dns/dns-default-metrics-tls: secret "dns-default-metrics-tls" not found Jul 10 18:03:34.637629 ip-10-0-128-39 kubenswrapper[2558]: E0710 18:03:34.637332 2558 projected.go:266] Couldn't get secret openshift-image-registry/image-registry-tls: secret "image-registry-tls" not found Jul 10 18:03:34.637629 ip-10-0-128-39 kubenswrapper[2558]: E0710 18:03:34.637349 2558 projected.go:196] Error preparing data for projected volume registry-tls for pod openshift-image-registry/image-registry-cf9777cd6-d9zpc: secret "image-registry-tls" not found Jul 10 18:03:34.637629 ip-10-0-128-39 kubenswrapper[2558]: E0710 18:03:34.637404 2558 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/15c4360f-6bc7-4948-b0e6-ebc018174856-metrics-tls podName:15c4360f-6bc7-4948-b0e6-ebc018174856 nodeName:}" failed. No retries permitted until 2026-07-10 18:03:42.63738965 +0000 UTC m=+51.140882134 (durationBeforeRetry 8s). Error: MountVolume.SetUp failed for volume "metrics-tls" (UniqueName: "kubernetes.io/secret/15c4360f-6bc7-4948-b0e6-ebc018174856-metrics-tls") pod "dns-default-22khc" (UID: "15c4360f-6bc7-4948-b0e6-ebc018174856") : secret "dns-default-metrics-tls" not found Jul 10 18:03:34.637629 ip-10-0-128-39 kubenswrapper[2558]: E0710 18:03:34.637418 2558 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/projected/4b133a69-01d6-42b5-b12f-d063856c87cb-registry-tls podName:4b133a69-01d6-42b5-b12f-d063856c87cb nodeName:}" failed. No retries permitted until 2026-07-10 18:03:42.637412864 +0000 UTC m=+51.140905347 (durationBeforeRetry 8s). Error: MountVolume.SetUp failed for volume "registry-tls" (UniqueName: "kubernetes.io/projected/4b133a69-01d6-42b5-b12f-d063856c87cb-registry-tls") pod "image-registry-cf9777cd6-d9zpc" (UID: "4b133a69-01d6-42b5-b12f-d063856c87cb") : secret "image-registry-tls" not found Jul 10 18:03:34.904064 ip-10-0-128-39 kubenswrapper[2558]: I0710 18:03:34.903975 2558 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-dvp86_2da19865-b021-4789-b1e0-694d4f6c543d/konnectivity-agent/0.log" limit=1024 Jul 10 18:03:35.004332 ip-10-0-128-39 kubenswrapper[2558]: I0710 18:03:35.004284 2558 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-dvp86_2da19865-b021-4789-b1e0-694d4f6c543d/konnectivity-agent/0.log" limit=1024 Jul 10 18:03:35.915878 ip-10-0-128-39 kubenswrapper[2558]: I0710 18:03:35.915837 2558 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-dvp86_2da19865-b021-4789-b1e0-694d4f6c543d/konnectivity-agent/0.log" limit=1024 Jul 10 18:03:35.950520 ip-10-0-128-39 kubenswrapper[2558]: I0710 18:03:35.950478 2558 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"cluster-monitoring-operator-tls\" (UniqueName: \"kubernetes.io/secret/19b8d048-b0b5-4b42-9f4f-ad9f47a105cd-cluster-monitoring-operator-tls\") pod \"cluster-monitoring-operator-5f7f494dd4-46fc9\" (UID: \"19b8d048-b0b5-4b42-9f4f-ad9f47a105cd\") " pod="openshift-monitoring/cluster-monitoring-operator-5f7f494dd4-46fc9" Jul 10 18:03:35.950695 ip-10-0-128-39 kubenswrapper[2558]: E0710 18:03:35.950603 2558 secret.go:189] Couldn't get secret openshift-monitoring/cluster-monitoring-operator-tls: secret "cluster-monitoring-operator-tls" not found Jul 10 18:03:35.950695 ip-10-0-128-39 kubenswrapper[2558]: E0710 18:03:35.950675 2558 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/19b8d048-b0b5-4b42-9f4f-ad9f47a105cd-cluster-monitoring-operator-tls podName:19b8d048-b0b5-4b42-9f4f-ad9f47a105cd nodeName:}" failed. No retries permitted until 2026-07-10 18:03:43.950659201 +0000 UTC m=+52.454151699 (durationBeforeRetry 8s). Error: MountVolume.SetUp failed for volume "cluster-monitoring-operator-tls" (UniqueName: "kubernetes.io/secret/19b8d048-b0b5-4b42-9f4f-ad9f47a105cd-cluster-monitoring-operator-tls") pod "cluster-monitoring-operator-5f7f494dd4-46fc9" (UID: "19b8d048-b0b5-4b42-9f4f-ad9f47a105cd") : secret "cluster-monitoring-operator-tls" not found Jul 10 18:03:36.016010 ip-10-0-128-39 kubenswrapper[2558]: I0710 18:03:36.015974 2558 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-dvp86_2da19865-b021-4789-b1e0-694d4f6c543d/konnectivity-agent/0.log" limit=1024 Jul 10 18:03:36.052028 ip-10-0-128-39 kubenswrapper[2558]: I0710 18:03:36.051990 2558 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"networking-console-plugin-cert\" (UniqueName: \"kubernetes.io/secret/b95b59af-225c-4907-9643-5d916a325e56-networking-console-plugin-cert\") pod \"networking-console-plugin-6cf856758c-qg5vl\" (UID: \"b95b59af-225c-4907-9643-5d916a325e56\") " pod="openshift-network-console/networking-console-plugin-6cf856758c-qg5vl" Jul 10 18:03:36.052222 ip-10-0-128-39 kubenswrapper[2558]: I0710 18:03:36.052042 2558 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"samples-operator-tls\" (UniqueName: \"kubernetes.io/secret/5c7b84a3-17f6-4ed7-bb37-5e5b19a52916-samples-operator-tls\") pod \"cluster-samples-operator-65db858894-2b2km\" (UID: \"5c7b84a3-17f6-4ed7-bb37-5e5b19a52916\") " pod="openshift-cluster-samples-operator/cluster-samples-operator-65db858894-2b2km" Jul 10 18:03:36.052222 ip-10-0-128-39 kubenswrapper[2558]: E0710 18:03:36.052155 2558 secret.go:189] Couldn't get secret openshift-cluster-samples-operator/samples-operator-tls: secret "samples-operator-tls" not found Jul 10 18:03:36.052222 ip-10-0-128-39 kubenswrapper[2558]: E0710 18:03:36.052159 2558 secret.go:189] Couldn't get secret openshift-network-console/networking-console-plugin-cert: secret "networking-console-plugin-cert" not found Jul 10 18:03:36.052376 ip-10-0-128-39 kubenswrapper[2558]: E0710 18:03:36.052227 2558 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/b95b59af-225c-4907-9643-5d916a325e56-networking-console-plugin-cert podName:b95b59af-225c-4907-9643-5d916a325e56 nodeName:}" failed. No retries permitted until 2026-07-10 18:03:44.052207995 +0000 UTC m=+52.555700494 (durationBeforeRetry 8s). Error: MountVolume.SetUp failed for volume "networking-console-plugin-cert" (UniqueName: "kubernetes.io/secret/b95b59af-225c-4907-9643-5d916a325e56-networking-console-plugin-cert") pod "networking-console-plugin-6cf856758c-qg5vl" (UID: "b95b59af-225c-4907-9643-5d916a325e56") : secret "networking-console-plugin-cert" not found Jul 10 18:03:36.052376 ip-10-0-128-39 kubenswrapper[2558]: E0710 18:03:36.052346 2558 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/5c7b84a3-17f6-4ed7-bb37-5e5b19a52916-samples-operator-tls podName:5c7b84a3-17f6-4ed7-bb37-5e5b19a52916 nodeName:}" failed. No retries permitted until 2026-07-10 18:03:44.052318903 +0000 UTC m=+52.555811398 (durationBeforeRetry 8s). Error: MountVolume.SetUp failed for volume "samples-operator-tls" (UniqueName: "kubernetes.io/secret/5c7b84a3-17f6-4ed7-bb37-5e5b19a52916-samples-operator-tls") pod "cluster-samples-operator-65db858894-2b2km" (UID: "5c7b84a3-17f6-4ed7-bb37-5e5b19a52916") : secret "samples-operator-tls" not found Jul 10 18:03:36.254136 ip-10-0-128-39 kubenswrapper[2558]: I0710 18:03:36.254102 2558 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/5fab65cc-f578-4cae-919e-1e630186e6d3-metrics-certs\") pod \"router-default-cd6bb8744-wq24m\" (UID: \"5fab65cc-f578-4cae-919e-1e630186e6d3\") " pod="openshift-ingress/router-default-cd6bb8744-wq24m" Jul 10 18:03:36.254311 ip-10-0-128-39 kubenswrapper[2558]: E0710 18:03:36.254254 2558 secret.go:189] Couldn't get secret openshift-ingress/router-metrics-certs-default: secret "router-metrics-certs-default" not found Jul 10 18:03:36.254311 ip-10-0-128-39 kubenswrapper[2558]: I0710 18:03:36.254268 2558 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"service-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/5fab65cc-f578-4cae-919e-1e630186e6d3-service-ca-bundle\") pod \"router-default-cd6bb8744-wq24m\" (UID: \"5fab65cc-f578-4cae-919e-1e630186e6d3\") " pod="openshift-ingress/router-default-cd6bb8744-wq24m" Jul 10 18:03:36.254387 ip-10-0-128-39 kubenswrapper[2558]: E0710 18:03:36.254314 2558 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/5fab65cc-f578-4cae-919e-1e630186e6d3-metrics-certs podName:5fab65cc-f578-4cae-919e-1e630186e6d3 nodeName:}" failed. No retries permitted until 2026-07-10 18:03:44.254299647 +0000 UTC m=+52.757792134 (durationBeforeRetry 8s). Error: MountVolume.SetUp failed for volume "metrics-certs" (UniqueName: "kubernetes.io/secret/5fab65cc-f578-4cae-919e-1e630186e6d3-metrics-certs") pod "router-default-cd6bb8744-wq24m" (UID: "5fab65cc-f578-4cae-919e-1e630186e6d3") : secret "router-metrics-certs-default" not found Jul 10 18:03:36.254438 ip-10-0-128-39 kubenswrapper[2558]: E0710 18:03:36.254417 2558 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/configmap/5fab65cc-f578-4cae-919e-1e630186e6d3-service-ca-bundle podName:5fab65cc-f578-4cae-919e-1e630186e6d3 nodeName:}" failed. No retries permitted until 2026-07-10 18:03:44.254402424 +0000 UTC m=+52.757894908 (durationBeforeRetry 8s). Error: MountVolume.SetUp failed for volume "service-ca-bundle" (UniqueName: "kubernetes.io/configmap/5fab65cc-f578-4cae-919e-1e630186e6d3-service-ca-bundle") pod "router-default-cd6bb8744-wq24m" (UID: "5fab65cc-f578-4cae-919e-1e630186e6d3") : configmap references non-existent config key: service-ca.crt Jul 10 18:03:37.324034 ip-10-0-128-39 kubenswrapper[2558]: I0710 18:03:37.323995 2558 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-kube-storage-version-migrator-operator/kube-storage-version-migrator-operator-799cb7f958-vggpz" event={"ID":"eec35344-a5cb-4a6e-9e2e-5adca5a0fa1a","Type":"ContainerStarted","Data":"a3c44a99161d39c31cabed3cb7cc5b67842e3079249e858f1388e6918e7ce078"} Jul 10 18:03:37.325944 ip-10-0-128-39 kubenswrapper[2558]: I0710 18:03:37.325913 2558 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-service-ca-operator/service-ca-operator-7695fc6c8c-hfzjm" event={"ID":"893dc3d5-4a12-4832-9a96-898388c12ae4","Type":"ContainerStarted","Data":"3e176c50cd8670cbb5c7023435f89cc28f2b4ce1b5249daeba7a0b06b67d76c3"} Jul 10 18:03:37.327782 ip-10-0-128-39 kubenswrapper[2558]: I0710 18:03:37.327756 2558 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-console-operator_console-operator-7866497494-26542_2b29fb39-89b3-42c3-b334-3a195242248c/console-operator/0.log" Jul 10 18:03:37.327934 ip-10-0-128-39 kubenswrapper[2558]: I0710 18:03:37.327800 2558 generic.go:356] "Generic (PLEG): container finished" podID="2b29fb39-89b3-42c3-b334-3a195242248c" containerID="2c5fc3b76ab13f78d1f667171aadfaf11ed235fd5ece69100df86c6fbe938256" exitCode=255 Jul 10 18:03:37.327934 ip-10-0-128-39 kubenswrapper[2558]: I0710 18:03:37.327891 2558 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-console-operator/console-operator-7866497494-26542" event={"ID":"2b29fb39-89b3-42c3-b334-3a195242248c","Type":"ContainerDied","Data":"2c5fc3b76ab13f78d1f667171aadfaf11ed235fd5ece69100df86c6fbe938256"} Jul 10 18:03:37.328124 ip-10-0-128-39 kubenswrapper[2558]: I0710 18:03:37.328103 2558 scope.go:117] "RemoveContainer" containerID="2c5fc3b76ab13f78d1f667171aadfaf11ed235fd5ece69100df86c6fbe938256" Jul 10 18:03:37.330089 ip-10-0-128-39 kubenswrapper[2558]: I0710 18:03:37.330063 2558 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress-canary/ingress-canary-bpwbf" event={"ID":"f754e509-abbe-4451-833d-abb0e28c7e19","Type":"ContainerStarted","Data":"306d307fcefafb738eaba442f668f01cd25fa29e3f52a5e0023d09c5f1008fe6"} Jul 10 18:03:37.331936 ip-10-0-128-39 kubenswrapper[2558]: I0710 18:03:37.331915 2558 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-cluster-storage-operator/volume-data-source-validator-78495597f6-v7ckr" event={"ID":"5df2ceaf-9a85-4d1a-ad19-76df6f2b955f","Type":"ContainerStarted","Data":"c26f524d5d7c99f3fc766f4b03ee3acfd8ec680269a493cc288e19fa53417a0f"} Jul 10 18:03:37.333662 ip-10-0-128-39 kubenswrapper[2558]: I0710 18:03:37.333638 2558 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-insights/insights-operator-68d9dd94cf-q5s5m" event={"ID":"748dca80-632f-44ef-b3a2-5292df366af1","Type":"ContainerStarted","Data":"d3b4b1905749b7a33cf2188b7a1718c44f18c3f6d613a9e2183b17d13115dc27"} Jul 10 18:03:37.335193 ip-10-0-128-39 kubenswrapper[2558]: I0710 18:03:37.335172 2558 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-network-diagnostics/network-check-source-869dcfcdcd-7n55h" event={"ID":"5997642b-76cf-44e6-8b06-138718112a30","Type":"ContainerStarted","Data":"e06cea2f6e91dfce8e3bb0940aee24f546a33fe6d61a7e08602afa529e160859"} Jul 10 18:03:37.339709 ip-10-0-128-39 kubenswrapper[2558]: I0710 18:03:37.339654 2558 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-kube-storage-version-migrator-operator/kube-storage-version-migrator-operator-799cb7f958-vggpz" podStartSLOduration=1.662919517 podStartE2EDuration="9.339637557s" totalImagesPullingTime="7.67671804s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-10 18:03:28 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-10 18:03:37.33826504 +0000 UTC m=+45.841757544" watchObservedRunningTime="2026-07-10 18:03:37.339637557 +0000 UTC m=+45.843130064" Jul 10 18:03:37.353156 ip-10-0-128-39 kubenswrapper[2558]: I0710 18:03:37.353112 2558 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-cluster-storage-operator/volume-data-source-validator-78495597f6-v7ckr" podStartSLOduration=1.6744670369999999 podStartE2EDuration="9.353099101s" totalImagesPullingTime="7.678632064s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-10 18:03:28 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-10 18:03:37.352172658 +0000 UTC m=+45.855665164" watchObservedRunningTime="2026-07-10 18:03:37.353099101 +0000 UTC m=+45.856591598" Jul 10 18:03:37.369377 ip-10-0-128-39 kubenswrapper[2558]: I0710 18:03:37.369319 2558 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-insights/insights-operator-68d9dd94cf-q5s5m" podStartSLOduration=1.6343287960000001 podStartE2EDuration="9.369299188s" totalImagesPullingTime="7.734970392s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-10 18:03:28 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-10 18:03:37.368944413 +0000 UTC m=+45.872436913" watchObservedRunningTime="2026-07-10 18:03:37.369299188 +0000 UTC m=+45.872791695" Jul 10 18:03:37.387687 ip-10-0-128-39 kubenswrapper[2558]: I0710 18:03:37.387621 2558 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-service-ca-operator/service-ca-operator-7695fc6c8c-hfzjm" podStartSLOduration=1.7091832500000002 podStartE2EDuration="9.387605542s" totalImagesPullingTime="7.678422292s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-10 18:03:28 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-10 18:03:37.387082367 +0000 UTC m=+45.890574872" watchObservedRunningTime="2026-07-10 18:03:37.387605542 +0000 UTC m=+45.891098043" Jul 10 18:03:37.429669 ip-10-0-128-39 kubenswrapper[2558]: I0710 18:03:37.428259 2558 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-ingress-canary/ingress-canary-bpwbf" podStartSLOduration=3.748545422 podStartE2EDuration="11.428241317s" totalImagesPullingTime="7.679695895s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-10 18:03:26 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-10 18:03:37.427565864 +0000 UTC m=+45.931058371" watchObservedRunningTime="2026-07-10 18:03:37.428241317 +0000 UTC m=+45.931733824" Jul 10 18:03:37.444798 ip-10-0-128-39 kubenswrapper[2558]: I0710 18:03:37.443962 2558 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-network-diagnostics/network-check-source-869dcfcdcd-7n55h" podStartSLOduration=1.75513649 podStartE2EDuration="9.4439421s" totalImagesPullingTime="7.68880561s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-10 18:03:28 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-10 18:03:37.443713562 +0000 UTC m=+45.947206058" watchObservedRunningTime="2026-07-10 18:03:37.4439421 +0000 UTC m=+45.947434608" Jul 10 18:03:37.491894 ip-10-0-128-39 kubenswrapper[2558]: I0710 18:03:37.491678 2558 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-dvp86_2da19865-b021-4789-b1e0-694d4f6c543d/konnectivity-agent/0.log" limit=1024 Jul 10 18:03:37.595639 ip-10-0-128-39 kubenswrapper[2558]: I0710 18:03:37.595539 2558 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-dvp86_2da19865-b021-4789-b1e0-694d4f6c543d/konnectivity-agent/0.log" limit=1024 Jul 10 18:03:38.070353 ip-10-0-128-39 kubenswrapper[2558]: I0710 18:03:38.069160 2558 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-insights/insights-runtime-extractor-lbs9p"] Jul 10 18:03:38.096594 ip-10-0-128-39 kubenswrapper[2558]: I0710 18:03:38.096555 2558 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-insights/insights-runtime-extractor-lbs9p" Jul 10 18:03:38.099103 ip-10-0-128-39 kubenswrapper[2558]: I0710 18:03:38.098996 2558 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-insights\"/\"kube-rbac-proxy\"" Jul 10 18:03:38.099252 ip-10-0-128-39 kubenswrapper[2558]: I0710 18:03:38.099123 2558 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-insights\"/\"insights-runtime-extractor-tls\"" Jul 10 18:03:38.099252 ip-10-0-128-39 kubenswrapper[2558]: I0710 18:03:38.099236 2558 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-insights\"/\"insights-runtime-extractor-sa-dockercfg-cpkz8\"" Jul 10 18:03:38.105957 ip-10-0-128-39 kubenswrapper[2558]: I0710 18:03:38.102392 2558 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-insights/insights-runtime-extractor-lbs9p"] Jul 10 18:03:38.176508 ip-10-0-128-39 kubenswrapper[2558]: I0710 18:03:38.176475 2558 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"data-volume\" (UniqueName: \"kubernetes.io/empty-dir/cf337123-3a94-4667-a27e-7fc30a589451-data-volume\") pod \"insights-runtime-extractor-lbs9p\" (UID: \"cf337123-3a94-4667-a27e-7fc30a589451\") " pod="openshift-insights/insights-runtime-extractor-lbs9p" Jul 10 18:03:38.176675 ip-10-0-128-39 kubenswrapper[2558]: I0710 18:03:38.176556 2558 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"crio-socket\" (UniqueName: \"kubernetes.io/host-path/cf337123-3a94-4667-a27e-7fc30a589451-crio-socket\") pod \"insights-runtime-extractor-lbs9p\" (UID: \"cf337123-3a94-4667-a27e-7fc30a589451\") " pod="openshift-insights/insights-runtime-extractor-lbs9p" Jul 10 18:03:38.176675 ip-10-0-128-39 kubenswrapper[2558]: I0710 18:03:38.176607 2558 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"insights-runtime-extractor-tls\" (UniqueName: \"kubernetes.io/secret/cf337123-3a94-4667-a27e-7fc30a589451-insights-runtime-extractor-tls\") pod \"insights-runtime-extractor-lbs9p\" (UID: \"cf337123-3a94-4667-a27e-7fc30a589451\") " pod="openshift-insights/insights-runtime-extractor-lbs9p" Jul 10 18:03:38.176759 ip-10-0-128-39 kubenswrapper[2558]: I0710 18:03:38.176705 2558 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-rbac-proxy-cm\" (UniqueName: \"kubernetes.io/configmap/cf337123-3a94-4667-a27e-7fc30a589451-kube-rbac-proxy-cm\") pod \"insights-runtime-extractor-lbs9p\" (UID: \"cf337123-3a94-4667-a27e-7fc30a589451\") " pod="openshift-insights/insights-runtime-extractor-lbs9p" Jul 10 18:03:38.176850 ip-10-0-128-39 kubenswrapper[2558]: I0710 18:03:38.176834 2558 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-5q5t7\" (UniqueName: \"kubernetes.io/projected/cf337123-3a94-4667-a27e-7fc30a589451-kube-api-access-5q5t7\") pod \"insights-runtime-extractor-lbs9p\" (UID: \"cf337123-3a94-4667-a27e-7fc30a589451\") " pod="openshift-insights/insights-runtime-extractor-lbs9p" Jul 10 18:03:38.277574 ip-10-0-128-39 kubenswrapper[2558]: I0710 18:03:38.277529 2558 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-5q5t7\" (UniqueName: \"kubernetes.io/projected/cf337123-3a94-4667-a27e-7fc30a589451-kube-api-access-5q5t7\") pod \"insights-runtime-extractor-lbs9p\" (UID: \"cf337123-3a94-4667-a27e-7fc30a589451\") " pod="openshift-insights/insights-runtime-extractor-lbs9p" Jul 10 18:03:38.277751 ip-10-0-128-39 kubenswrapper[2558]: I0710 18:03:38.277723 2558 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"data-volume\" (UniqueName: \"kubernetes.io/empty-dir/cf337123-3a94-4667-a27e-7fc30a589451-data-volume\") pod \"insights-runtime-extractor-lbs9p\" (UID: \"cf337123-3a94-4667-a27e-7fc30a589451\") " pod="openshift-insights/insights-runtime-extractor-lbs9p" Jul 10 18:03:38.277845 ip-10-0-128-39 kubenswrapper[2558]: I0710 18:03:38.277820 2558 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"crio-socket\" (UniqueName: \"kubernetes.io/host-path/cf337123-3a94-4667-a27e-7fc30a589451-crio-socket\") pod \"insights-runtime-extractor-lbs9p\" (UID: \"cf337123-3a94-4667-a27e-7fc30a589451\") " pod="openshift-insights/insights-runtime-extractor-lbs9p" Jul 10 18:03:38.277921 ip-10-0-128-39 kubenswrapper[2558]: I0710 18:03:38.277870 2558 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"insights-runtime-extractor-tls\" (UniqueName: \"kubernetes.io/secret/cf337123-3a94-4667-a27e-7fc30a589451-insights-runtime-extractor-tls\") pod \"insights-runtime-extractor-lbs9p\" (UID: \"cf337123-3a94-4667-a27e-7fc30a589451\") " pod="openshift-insights/insights-runtime-extractor-lbs9p" Jul 10 18:03:38.277961 ip-10-0-128-39 kubenswrapper[2558]: I0710 18:03:38.277923 2558 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-rbac-proxy-cm\" (UniqueName: \"kubernetes.io/configmap/cf337123-3a94-4667-a27e-7fc30a589451-kube-rbac-proxy-cm\") pod \"insights-runtime-extractor-lbs9p\" (UID: \"cf337123-3a94-4667-a27e-7fc30a589451\") " pod="openshift-insights/insights-runtime-extractor-lbs9p" Jul 10 18:03:38.278011 ip-10-0-128-39 kubenswrapper[2558]: I0710 18:03:38.277975 2558 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"crio-socket\" (UniqueName: \"kubernetes.io/host-path/cf337123-3a94-4667-a27e-7fc30a589451-crio-socket\") pod \"insights-runtime-extractor-lbs9p\" (UID: \"cf337123-3a94-4667-a27e-7fc30a589451\") " pod="openshift-insights/insights-runtime-extractor-lbs9p" Jul 10 18:03:38.278068 ip-10-0-128-39 kubenswrapper[2558]: E0710 18:03:38.278055 2558 configmap.go:193] Couldn't get configMap openshift-insights/kube-rbac-proxy: configmap "kube-rbac-proxy" not found Jul 10 18:03:38.278101 ip-10-0-128-39 kubenswrapper[2558]: E0710 18:03:38.278093 2558 secret.go:189] Couldn't get secret openshift-insights/insights-runtime-extractor-tls: secret "insights-runtime-extractor-tls" not found Jul 10 18:03:38.278135 ip-10-0-128-39 kubenswrapper[2558]: E0710 18:03:38.278119 2558 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/configmap/cf337123-3a94-4667-a27e-7fc30a589451-kube-rbac-proxy-cm podName:cf337123-3a94-4667-a27e-7fc30a589451 nodeName:}" failed. No retries permitted until 2026-07-10 18:03:38.778100991 +0000 UTC m=+47.281593479 (durationBeforeRetry 500ms). Error: MountVolume.SetUp failed for volume "kube-rbac-proxy-cm" (UniqueName: "kubernetes.io/configmap/cf337123-3a94-4667-a27e-7fc30a589451-kube-rbac-proxy-cm") pod "insights-runtime-extractor-lbs9p" (UID: "cf337123-3a94-4667-a27e-7fc30a589451") : configmap "kube-rbac-proxy" not found Jul 10 18:03:38.278176 ip-10-0-128-39 kubenswrapper[2558]: E0710 18:03:38.278141 2558 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/cf337123-3a94-4667-a27e-7fc30a589451-insights-runtime-extractor-tls podName:cf337123-3a94-4667-a27e-7fc30a589451 nodeName:}" failed. No retries permitted until 2026-07-10 18:03:38.778125759 +0000 UTC m=+47.281618244 (durationBeforeRetry 500ms). Error: MountVolume.SetUp failed for volume "insights-runtime-extractor-tls" (UniqueName: "kubernetes.io/secret/cf337123-3a94-4667-a27e-7fc30a589451-insights-runtime-extractor-tls") pod "insights-runtime-extractor-lbs9p" (UID: "cf337123-3a94-4667-a27e-7fc30a589451") : secret "insights-runtime-extractor-tls" not found Jul 10 18:03:38.278176 ip-10-0-128-39 kubenswrapper[2558]: I0710 18:03:38.278150 2558 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"data-volume\" (UniqueName: \"kubernetes.io/empty-dir/cf337123-3a94-4667-a27e-7fc30a589451-data-volume\") pod \"insights-runtime-extractor-lbs9p\" (UID: \"cf337123-3a94-4667-a27e-7fc30a589451\") " pod="openshift-insights/insights-runtime-extractor-lbs9p" Jul 10 18:03:38.290139 ip-10-0-128-39 kubenswrapper[2558]: I0710 18:03:38.290110 2558 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-5q5t7\" (UniqueName: \"kubernetes.io/projected/cf337123-3a94-4667-a27e-7fc30a589451-kube-api-access-5q5t7\") pod \"insights-runtime-extractor-lbs9p\" (UID: \"cf337123-3a94-4667-a27e-7fc30a589451\") " pod="openshift-insights/insights-runtime-extractor-lbs9p" Jul 10 18:03:38.340762 ip-10-0-128-39 kubenswrapper[2558]: I0710 18:03:38.340681 2558 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-console-operator_console-operator-7866497494-26542_2b29fb39-89b3-42c3-b334-3a195242248c/console-operator/1.log" Jul 10 18:03:38.341226 ip-10-0-128-39 kubenswrapper[2558]: I0710 18:03:38.341128 2558 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-console-operator_console-operator-7866497494-26542_2b29fb39-89b3-42c3-b334-3a195242248c/console-operator/0.log" Jul 10 18:03:38.341226 ip-10-0-128-39 kubenswrapper[2558]: I0710 18:03:38.341160 2558 generic.go:356] "Generic (PLEG): container finished" podID="2b29fb39-89b3-42c3-b334-3a195242248c" containerID="4e0c837200fa48d93da2f5ad906f1315f35e0e717200ab41550fef8a1543726b" exitCode=255 Jul 10 18:03:38.341226 ip-10-0-128-39 kubenswrapper[2558]: I0710 18:03:38.341195 2558 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-console-operator/console-operator-7866497494-26542" event={"ID":"2b29fb39-89b3-42c3-b334-3a195242248c","Type":"ContainerDied","Data":"4e0c837200fa48d93da2f5ad906f1315f35e0e717200ab41550fef8a1543726b"} Jul 10 18:03:38.341379 ip-10-0-128-39 kubenswrapper[2558]: I0710 18:03:38.341252 2558 scope.go:117] "RemoveContainer" containerID="2c5fc3b76ab13f78d1f667171aadfaf11ed235fd5ece69100df86c6fbe938256" Jul 10 18:03:38.341533 ip-10-0-128-39 kubenswrapper[2558]: I0710 18:03:38.341514 2558 scope.go:117] "RemoveContainer" containerID="4e0c837200fa48d93da2f5ad906f1315f35e0e717200ab41550fef8a1543726b" Jul 10 18:03:38.341747 ip-10-0-128-39 kubenswrapper[2558]: E0710 18:03:38.341719 2558 pod_workers.go:1324] "Error syncing pod, skipping" err="failed to \"StartContainer\" for \"console-operator\" with CrashLoopBackOff: \"back-off 10s restarting failed container=console-operator pod=console-operator-7866497494-26542_openshift-console-operator(2b29fb39-89b3-42c3-b334-3a195242248c)\"" pod="openshift-console-operator/console-operator-7866497494-26542" podUID="2b29fb39-89b3-42c3-b334-3a195242248c" Jul 10 18:03:38.696418 ip-10-0-128-39 kubenswrapper[2558]: I0710 18:03:38.696333 2558 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-console-operator/console-operator-7866497494-26542" Jul 10 18:03:38.696418 ip-10-0-128-39 kubenswrapper[2558]: I0710 18:03:38.696392 2558 kubelet.go:2654] "SyncLoop (probe)" probe="liveness" status="unhealthy" pod="openshift-console-operator/console-operator-7866497494-26542" Jul 10 18:03:38.783461 ip-10-0-128-39 kubenswrapper[2558]: I0710 18:03:38.783419 2558 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"insights-runtime-extractor-tls\" (UniqueName: \"kubernetes.io/secret/cf337123-3a94-4667-a27e-7fc30a589451-insights-runtime-extractor-tls\") pod \"insights-runtime-extractor-lbs9p\" (UID: \"cf337123-3a94-4667-a27e-7fc30a589451\") " pod="openshift-insights/insights-runtime-extractor-lbs9p" Jul 10 18:03:38.783631 ip-10-0-128-39 kubenswrapper[2558]: I0710 18:03:38.783484 2558 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-rbac-proxy-cm\" (UniqueName: \"kubernetes.io/configmap/cf337123-3a94-4667-a27e-7fc30a589451-kube-rbac-proxy-cm\") pod \"insights-runtime-extractor-lbs9p\" (UID: \"cf337123-3a94-4667-a27e-7fc30a589451\") " pod="openshift-insights/insights-runtime-extractor-lbs9p" Jul 10 18:03:38.783631 ip-10-0-128-39 kubenswrapper[2558]: E0710 18:03:38.783574 2558 configmap.go:193] Couldn't get configMap openshift-insights/kube-rbac-proxy: configmap "kube-rbac-proxy" not found Jul 10 18:03:38.783631 ip-10-0-128-39 kubenswrapper[2558]: E0710 18:03:38.783585 2558 secret.go:189] Couldn't get secret openshift-insights/insights-runtime-extractor-tls: secret "insights-runtime-extractor-tls" not found Jul 10 18:03:38.783723 ip-10-0-128-39 kubenswrapper[2558]: E0710 18:03:38.783640 2558 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/configmap/cf337123-3a94-4667-a27e-7fc30a589451-kube-rbac-proxy-cm podName:cf337123-3a94-4667-a27e-7fc30a589451 nodeName:}" failed. No retries permitted until 2026-07-10 18:03:39.783620805 +0000 UTC m=+48.287113293 (durationBeforeRetry 1s). Error: MountVolume.SetUp failed for volume "kube-rbac-proxy-cm" (UniqueName: "kubernetes.io/configmap/cf337123-3a94-4667-a27e-7fc30a589451-kube-rbac-proxy-cm") pod "insights-runtime-extractor-lbs9p" (UID: "cf337123-3a94-4667-a27e-7fc30a589451") : configmap "kube-rbac-proxy" not found Jul 10 18:03:38.783723 ip-10-0-128-39 kubenswrapper[2558]: E0710 18:03:38.783666 2558 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/cf337123-3a94-4667-a27e-7fc30a589451-insights-runtime-extractor-tls podName:cf337123-3a94-4667-a27e-7fc30a589451 nodeName:}" failed. No retries permitted until 2026-07-10 18:03:39.783647047 +0000 UTC m=+48.287139549 (durationBeforeRetry 1s). Error: MountVolume.SetUp failed for volume "insights-runtime-extractor-tls" (UniqueName: "kubernetes.io/secret/cf337123-3a94-4667-a27e-7fc30a589451-insights-runtime-extractor-tls") pod "insights-runtime-extractor-lbs9p" (UID: "cf337123-3a94-4667-a27e-7fc30a589451") : secret "insights-runtime-extractor-tls" not found Jul 10 18:03:39.252384 ip-10-0-128-39 kubenswrapper[2558]: I0710 18:03:39.252356 2558 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-dns_node-resolver-jqb25_12ad9a8e-4768-4362-818f-d2fb73c8a236/dns-node-resolver/0.log" Jul 10 18:03:39.345247 ip-10-0-128-39 kubenswrapper[2558]: I0710 18:03:39.345219 2558 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-console-operator_console-operator-7866497494-26542_2b29fb39-89b3-42c3-b334-3a195242248c/console-operator/1.log" Jul 10 18:03:39.345633 ip-10-0-128-39 kubenswrapper[2558]: I0710 18:03:39.345562 2558 scope.go:117] "RemoveContainer" containerID="4e0c837200fa48d93da2f5ad906f1315f35e0e717200ab41550fef8a1543726b" Jul 10 18:03:39.345726 ip-10-0-128-39 kubenswrapper[2558]: E0710 18:03:39.345706 2558 pod_workers.go:1324] "Error syncing pod, skipping" err="failed to \"StartContainer\" for \"console-operator\" with CrashLoopBackOff: \"back-off 10s restarting failed container=console-operator pod=console-operator-7866497494-26542_openshift-console-operator(2b29fb39-89b3-42c3-b334-3a195242248c)\"" pod="openshift-console-operator/console-operator-7866497494-26542" podUID="2b29fb39-89b3-42c3-b334-3a195242248c" Jul 10 18:03:39.791866 ip-10-0-128-39 kubenswrapper[2558]: I0710 18:03:39.791825 2558 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"insights-runtime-extractor-tls\" (UniqueName: \"kubernetes.io/secret/cf337123-3a94-4667-a27e-7fc30a589451-insights-runtime-extractor-tls\") pod \"insights-runtime-extractor-lbs9p\" (UID: \"cf337123-3a94-4667-a27e-7fc30a589451\") " pod="openshift-insights/insights-runtime-extractor-lbs9p" Jul 10 18:03:39.792069 ip-10-0-128-39 kubenswrapper[2558]: I0710 18:03:39.791878 2558 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-rbac-proxy-cm\" (UniqueName: \"kubernetes.io/configmap/cf337123-3a94-4667-a27e-7fc30a589451-kube-rbac-proxy-cm\") pod \"insights-runtime-extractor-lbs9p\" (UID: \"cf337123-3a94-4667-a27e-7fc30a589451\") " pod="openshift-insights/insights-runtime-extractor-lbs9p" Jul 10 18:03:39.792069 ip-10-0-128-39 kubenswrapper[2558]: E0710 18:03:39.791978 2558 secret.go:189] Couldn't get secret openshift-insights/insights-runtime-extractor-tls: secret "insights-runtime-extractor-tls" not found Jul 10 18:03:39.792069 ip-10-0-128-39 kubenswrapper[2558]: E0710 18:03:39.792014 2558 configmap.go:193] Couldn't get configMap openshift-insights/kube-rbac-proxy: configmap "kube-rbac-proxy" not found Jul 10 18:03:39.792069 ip-10-0-128-39 kubenswrapper[2558]: E0710 18:03:39.792048 2558 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/cf337123-3a94-4667-a27e-7fc30a589451-insights-runtime-extractor-tls podName:cf337123-3a94-4667-a27e-7fc30a589451 nodeName:}" failed. No retries permitted until 2026-07-10 18:03:41.792031766 +0000 UTC m=+50.295524250 (durationBeforeRetry 2s). Error: MountVolume.SetUp failed for volume "insights-runtime-extractor-tls" (UniqueName: "kubernetes.io/secret/cf337123-3a94-4667-a27e-7fc30a589451-insights-runtime-extractor-tls") pod "insights-runtime-extractor-lbs9p" (UID: "cf337123-3a94-4667-a27e-7fc30a589451") : secret "insights-runtime-extractor-tls" not found Jul 10 18:03:39.792069 ip-10-0-128-39 kubenswrapper[2558]: E0710 18:03:39.792063 2558 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/configmap/cf337123-3a94-4667-a27e-7fc30a589451-kube-rbac-proxy-cm podName:cf337123-3a94-4667-a27e-7fc30a589451 nodeName:}" failed. No retries permitted until 2026-07-10 18:03:41.792057059 +0000 UTC m=+50.295549542 (durationBeforeRetry 2s). Error: MountVolume.SetUp failed for volume "kube-rbac-proxy-cm" (UniqueName: "kubernetes.io/configmap/cf337123-3a94-4667-a27e-7fc30a589451-kube-rbac-proxy-cm") pod "insights-runtime-extractor-lbs9p" (UID: "cf337123-3a94-4667-a27e-7fc30a589451") : configmap "kube-rbac-proxy" not found Jul 10 18:03:40.171072 ip-10-0-128-39 kubenswrapper[2558]: I0710 18:03:40.170997 2558 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-kube-storage-version-migrator/migrator-7f59b54f6c-5nctn"] Jul 10 18:03:40.211305 ip-10-0-128-39 kubenswrapper[2558]: I0710 18:03:40.211271 2558 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-kube-storage-version-migrator/migrator-7f59b54f6c-5nctn"] Jul 10 18:03:40.211451 ip-10-0-128-39 kubenswrapper[2558]: I0710 18:03:40.211372 2558 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-kube-storage-version-migrator/migrator-7f59b54f6c-5nctn" Jul 10 18:03:40.214606 ip-10-0-128-39 kubenswrapper[2558]: I0710 18:03:40.214584 2558 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-kube-storage-version-migrator\"/\"openshift-service-ca.crt\"" Jul 10 18:03:40.214726 ip-10-0-128-39 kubenswrapper[2558]: I0710 18:03:40.214640 2558 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-kube-storage-version-migrator\"/\"kube-storage-version-migrator-sa-dockercfg-gc27b\"" Jul 10 18:03:40.214726 ip-10-0-128-39 kubenswrapper[2558]: I0710 18:03:40.214654 2558 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-kube-storage-version-migrator\"/\"kube-root-ca.crt\"" Jul 10 18:03:40.295048 ip-10-0-128-39 kubenswrapper[2558]: I0710 18:03:40.295017 2558 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-jfxd9\" (UniqueName: \"kubernetes.io/projected/fb2c6bad-c788-4a76-beab-e424603069c4-kube-api-access-jfxd9\") pod \"migrator-7f59b54f6c-5nctn\" (UID: \"fb2c6bad-c788-4a76-beab-e424603069c4\") " pod="openshift-kube-storage-version-migrator/migrator-7f59b54f6c-5nctn" Jul 10 18:03:40.349603 ip-10-0-128-39 kubenswrapper[2558]: I0710 18:03:40.349578 2558 scope.go:117] "RemoveContainer" containerID="4e0c837200fa48d93da2f5ad906f1315f35e0e717200ab41550fef8a1543726b" Jul 10 18:03:40.349972 ip-10-0-128-39 kubenswrapper[2558]: E0710 18:03:40.349735 2558 pod_workers.go:1324] "Error syncing pod, skipping" err="failed to \"StartContainer\" for \"console-operator\" with CrashLoopBackOff: \"back-off 10s restarting failed container=console-operator pod=console-operator-7866497494-26542_openshift-console-operator(2b29fb39-89b3-42c3-b334-3a195242248c)\"" pod="openshift-console-operator/console-operator-7866497494-26542" podUID="2b29fb39-89b3-42c3-b334-3a195242248c" Jul 10 18:03:40.396081 ip-10-0-128-39 kubenswrapper[2558]: I0710 18:03:40.396054 2558 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-jfxd9\" (UniqueName: \"kubernetes.io/projected/fb2c6bad-c788-4a76-beab-e424603069c4-kube-api-access-jfxd9\") pod \"migrator-7f59b54f6c-5nctn\" (UID: \"fb2c6bad-c788-4a76-beab-e424603069c4\") " pod="openshift-kube-storage-version-migrator/migrator-7f59b54f6c-5nctn" Jul 10 18:03:40.404858 ip-10-0-128-39 kubenswrapper[2558]: I0710 18:03:40.404834 2558 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-jfxd9\" (UniqueName: \"kubernetes.io/projected/fb2c6bad-c788-4a76-beab-e424603069c4-kube-api-access-jfxd9\") pod \"migrator-7f59b54f6c-5nctn\" (UID: \"fb2c6bad-c788-4a76-beab-e424603069c4\") " pod="openshift-kube-storage-version-migrator/migrator-7f59b54f6c-5nctn" Jul 10 18:03:40.453952 ip-10-0-128-39 kubenswrapper[2558]: I0710 18:03:40.453885 2558 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-image-registry_node-ca-r7qh2_fe05c85a-2eb8-429b-9776-a3408a38903f/node-ca/0.log" Jul 10 18:03:40.520448 ip-10-0-128-39 kubenswrapper[2558]: I0710 18:03:40.520407 2558 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-kube-storage-version-migrator/migrator-7f59b54f6c-5nctn" Jul 10 18:03:40.681109 ip-10-0-128-39 kubenswrapper[2558]: I0710 18:03:40.681078 2558 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-kube-storage-version-migrator/migrator-7f59b54f6c-5nctn"] Jul 10 18:03:40.684019 ip-10-0-128-39 kubenswrapper[2558]: W0710 18:03:40.683988 2558 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podfb2c6bad_c788_4a76_beab_e424603069c4.slice/crio-9dbf8476125a44d01e7eaa16cb9b6a7be47763ba7a44b9837418aa8ec0b1e8bf WatchSource:0}: Error finding container 9dbf8476125a44d01e7eaa16cb9b6a7be47763ba7a44b9837418aa8ec0b1e8bf: Status 404 returned error can't find the container with id 9dbf8476125a44d01e7eaa16cb9b6a7be47763ba7a44b9837418aa8ec0b1e8bf Jul 10 18:03:40.779527 ip-10-0-128-39 kubenswrapper[2558]: I0710 18:03:40.779500 2558 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-service-ca/service-ca-79f68b9f4f-qjrsh"] Jul 10 18:03:40.808453 ip-10-0-128-39 kubenswrapper[2558]: I0710 18:03:40.808426 2558 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-service-ca/service-ca-79f68b9f4f-qjrsh"] Jul 10 18:03:40.808593 ip-10-0-128-39 kubenswrapper[2558]: I0710 18:03:40.808513 2558 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-service-ca/service-ca-79f68b9f4f-qjrsh" Jul 10 18:03:40.811158 ip-10-0-128-39 kubenswrapper[2558]: I0710 18:03:40.811135 2558 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-service-ca\"/\"openshift-service-ca.crt\"" Jul 10 18:03:40.811285 ip-10-0-128-39 kubenswrapper[2558]: I0710 18:03:40.811135 2558 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-service-ca\"/\"signing-key\"" Jul 10 18:03:40.811285 ip-10-0-128-39 kubenswrapper[2558]: I0710 18:03:40.811136 2558 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-service-ca\"/\"signing-cabundle\"" Jul 10 18:03:40.811977 ip-10-0-128-39 kubenswrapper[2558]: I0710 18:03:40.811961 2558 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-service-ca\"/\"service-ca-dockercfg-scwhj\"" Jul 10 18:03:40.812034 ip-10-0-128-39 kubenswrapper[2558]: I0710 18:03:40.811983 2558 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-service-ca\"/\"kube-root-ca.crt\"" Jul 10 18:03:40.852894 ip-10-0-128-39 kubenswrapper[2558]: I0710 18:03:40.852873 2558 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ingress-canary_ingress-canary-bpwbf_f754e509-abbe-4451-833d-abb0e28c7e19/serve-healthcheck-canary/0.log" Jul 10 18:03:40.905632 ip-10-0-128-39 kubenswrapper[2558]: I0710 18:03:40.905595 2558 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"signing-cabundle\" (UniqueName: \"kubernetes.io/configmap/7854117d-b5d9-4b7b-95ef-2e06f6387162-signing-cabundle\") pod \"service-ca-79f68b9f4f-qjrsh\" (UID: \"7854117d-b5d9-4b7b-95ef-2e06f6387162\") " pod="openshift-service-ca/service-ca-79f68b9f4f-qjrsh" Jul 10 18:03:40.905786 ip-10-0-128-39 kubenswrapper[2558]: I0710 18:03:40.905678 2558 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-nkxrp\" (UniqueName: \"kubernetes.io/projected/7854117d-b5d9-4b7b-95ef-2e06f6387162-kube-api-access-nkxrp\") pod \"service-ca-79f68b9f4f-qjrsh\" (UID: \"7854117d-b5d9-4b7b-95ef-2e06f6387162\") " pod="openshift-service-ca/service-ca-79f68b9f4f-qjrsh" Jul 10 18:03:40.905786 ip-10-0-128-39 kubenswrapper[2558]: I0710 18:03:40.905720 2558 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"signing-key\" (UniqueName: \"kubernetes.io/secret/7854117d-b5d9-4b7b-95ef-2e06f6387162-signing-key\") pod \"service-ca-79f68b9f4f-qjrsh\" (UID: \"7854117d-b5d9-4b7b-95ef-2e06f6387162\") " pod="openshift-service-ca/service-ca-79f68b9f4f-qjrsh" Jul 10 18:03:41.006618 ip-10-0-128-39 kubenswrapper[2558]: I0710 18:03:41.006574 2558 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-nkxrp\" (UniqueName: \"kubernetes.io/projected/7854117d-b5d9-4b7b-95ef-2e06f6387162-kube-api-access-nkxrp\") pod \"service-ca-79f68b9f4f-qjrsh\" (UID: \"7854117d-b5d9-4b7b-95ef-2e06f6387162\") " pod="openshift-service-ca/service-ca-79f68b9f4f-qjrsh" Jul 10 18:03:41.006792 ip-10-0-128-39 kubenswrapper[2558]: I0710 18:03:41.006643 2558 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"signing-key\" (UniqueName: \"kubernetes.io/secret/7854117d-b5d9-4b7b-95ef-2e06f6387162-signing-key\") pod \"service-ca-79f68b9f4f-qjrsh\" (UID: \"7854117d-b5d9-4b7b-95ef-2e06f6387162\") " pod="openshift-service-ca/service-ca-79f68b9f4f-qjrsh" Jul 10 18:03:41.006945 ip-10-0-128-39 kubenswrapper[2558]: I0710 18:03:41.006911 2558 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"signing-cabundle\" (UniqueName: \"kubernetes.io/configmap/7854117d-b5d9-4b7b-95ef-2e06f6387162-signing-cabundle\") pod \"service-ca-79f68b9f4f-qjrsh\" (UID: \"7854117d-b5d9-4b7b-95ef-2e06f6387162\") " pod="openshift-service-ca/service-ca-79f68b9f4f-qjrsh" Jul 10 18:03:41.007503 ip-10-0-128-39 kubenswrapper[2558]: I0710 18:03:41.007479 2558 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"signing-cabundle\" (UniqueName: \"kubernetes.io/configmap/7854117d-b5d9-4b7b-95ef-2e06f6387162-signing-cabundle\") pod \"service-ca-79f68b9f4f-qjrsh\" (UID: \"7854117d-b5d9-4b7b-95ef-2e06f6387162\") " pod="openshift-service-ca/service-ca-79f68b9f4f-qjrsh" Jul 10 18:03:41.009105 ip-10-0-128-39 kubenswrapper[2558]: I0710 18:03:41.009084 2558 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"signing-key\" (UniqueName: \"kubernetes.io/secret/7854117d-b5d9-4b7b-95ef-2e06f6387162-signing-key\") pod \"service-ca-79f68b9f4f-qjrsh\" (UID: \"7854117d-b5d9-4b7b-95ef-2e06f6387162\") " pod="openshift-service-ca/service-ca-79f68b9f4f-qjrsh" Jul 10 18:03:41.015371 ip-10-0-128-39 kubenswrapper[2558]: I0710 18:03:41.015349 2558 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-nkxrp\" (UniqueName: \"kubernetes.io/projected/7854117d-b5d9-4b7b-95ef-2e06f6387162-kube-api-access-nkxrp\") pod \"service-ca-79f68b9f4f-qjrsh\" (UID: \"7854117d-b5d9-4b7b-95ef-2e06f6387162\") " pod="openshift-service-ca/service-ca-79f68b9f4f-qjrsh" Jul 10 18:03:41.118647 ip-10-0-128-39 kubenswrapper[2558]: I0710 18:03:41.118565 2558 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-service-ca/service-ca-79f68b9f4f-qjrsh" Jul 10 18:03:41.264029 ip-10-0-128-39 kubenswrapper[2558]: I0710 18:03:41.263998 2558 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-service-ca/service-ca-79f68b9f4f-qjrsh"] Jul 10 18:03:41.267775 ip-10-0-128-39 kubenswrapper[2558]: W0710 18:03:41.267739 2558 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod7854117d_b5d9_4b7b_95ef_2e06f6387162.slice/crio-00a936f185a160a9ca6e1dbe8abad0219f5f4caf31eff80819aa6c448fd4e5ee WatchSource:0}: Error finding container 00a936f185a160a9ca6e1dbe8abad0219f5f4caf31eff80819aa6c448fd4e5ee: Status 404 returned error can't find the container with id 00a936f185a160a9ca6e1dbe8abad0219f5f4caf31eff80819aa6c448fd4e5ee Jul 10 18:03:41.354079 ip-10-0-128-39 kubenswrapper[2558]: I0710 18:03:41.354055 2558 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-service-ca/service-ca-79f68b9f4f-qjrsh" event={"ID":"7854117d-b5d9-4b7b-95ef-2e06f6387162","Type":"ContainerStarted","Data":"00a936f185a160a9ca6e1dbe8abad0219f5f4caf31eff80819aa6c448fd4e5ee"} Jul 10 18:03:41.355148 ip-10-0-128-39 kubenswrapper[2558]: I0710 18:03:41.355124 2558 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-kube-storage-version-migrator/migrator-7f59b54f6c-5nctn" event={"ID":"fb2c6bad-c788-4a76-beab-e424603069c4","Type":"ContainerStarted","Data":"9dbf8476125a44d01e7eaa16cb9b6a7be47763ba7a44b9837418aa8ec0b1e8bf"} Jul 10 18:03:41.814070 ip-10-0-128-39 kubenswrapper[2558]: I0710 18:03:41.814029 2558 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"insights-runtime-extractor-tls\" (UniqueName: \"kubernetes.io/secret/cf337123-3a94-4667-a27e-7fc30a589451-insights-runtime-extractor-tls\") pod \"insights-runtime-extractor-lbs9p\" (UID: \"cf337123-3a94-4667-a27e-7fc30a589451\") " pod="openshift-insights/insights-runtime-extractor-lbs9p" Jul 10 18:03:41.814070 ip-10-0-128-39 kubenswrapper[2558]: I0710 18:03:41.814078 2558 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-rbac-proxy-cm\" (UniqueName: \"kubernetes.io/configmap/cf337123-3a94-4667-a27e-7fc30a589451-kube-rbac-proxy-cm\") pod \"insights-runtime-extractor-lbs9p\" (UID: \"cf337123-3a94-4667-a27e-7fc30a589451\") " pod="openshift-insights/insights-runtime-extractor-lbs9p" Jul 10 18:03:41.814320 ip-10-0-128-39 kubenswrapper[2558]: E0710 18:03:41.814149 2558 configmap.go:193] Couldn't get configMap openshift-insights/kube-rbac-proxy: configmap "kube-rbac-proxy" not found Jul 10 18:03:41.814320 ip-10-0-128-39 kubenswrapper[2558]: E0710 18:03:41.814174 2558 secret.go:189] Couldn't get secret openshift-insights/insights-runtime-extractor-tls: secret "insights-runtime-extractor-tls" not found Jul 10 18:03:41.814320 ip-10-0-128-39 kubenswrapper[2558]: E0710 18:03:41.814199 2558 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/configmap/cf337123-3a94-4667-a27e-7fc30a589451-kube-rbac-proxy-cm podName:cf337123-3a94-4667-a27e-7fc30a589451 nodeName:}" failed. No retries permitted until 2026-07-10 18:03:45.814184437 +0000 UTC m=+54.317676921 (durationBeforeRetry 4s). Error: MountVolume.SetUp failed for volume "kube-rbac-proxy-cm" (UniqueName: "kubernetes.io/configmap/cf337123-3a94-4667-a27e-7fc30a589451-kube-rbac-proxy-cm") pod "insights-runtime-extractor-lbs9p" (UID: "cf337123-3a94-4667-a27e-7fc30a589451") : configmap "kube-rbac-proxy" not found Jul 10 18:03:41.814320 ip-10-0-128-39 kubenswrapper[2558]: E0710 18:03:41.814227 2558 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/cf337123-3a94-4667-a27e-7fc30a589451-insights-runtime-extractor-tls podName:cf337123-3a94-4667-a27e-7fc30a589451 nodeName:}" failed. No retries permitted until 2026-07-10 18:03:45.81421216 +0000 UTC m=+54.317704647 (durationBeforeRetry 4s). Error: MountVolume.SetUp failed for volume "insights-runtime-extractor-tls" (UniqueName: "kubernetes.io/secret/cf337123-3a94-4667-a27e-7fc30a589451-insights-runtime-extractor-tls") pod "insights-runtime-extractor-lbs9p" (UID: "cf337123-3a94-4667-a27e-7fc30a589451") : secret "insights-runtime-extractor-tls" not found Jul 10 18:03:42.361822 ip-10-0-128-39 kubenswrapper[2558]: I0710 18:03:42.361770 2558 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-service-ca/service-ca-79f68b9f4f-qjrsh" event={"ID":"7854117d-b5d9-4b7b-95ef-2e06f6387162","Type":"ContainerStarted","Data":"567e55ea5bbe02b03f408150802fd2553408833d52917e7fd2dc9eb372cf5cbd"} Jul 10 18:03:42.381466 ip-10-0-128-39 kubenswrapper[2558]: I0710 18:03:42.381303 2558 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-service-ca/service-ca-79f68b9f4f-qjrsh" podStartSLOduration=2.381282833 podStartE2EDuration="2.381282833s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-10 18:03:40 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-10 18:03:42.379981282 +0000 UTC m=+50.883473789" watchObservedRunningTime="2026-07-10 18:03:42.381282833 +0000 UTC m=+50.884775340" Jul 10 18:03:42.725004 ip-10-0-128-39 kubenswrapper[2558]: I0710 18:03:42.724970 2558 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"registry-tls\" (UniqueName: \"kubernetes.io/projected/4b133a69-01d6-42b5-b12f-d063856c87cb-registry-tls\") pod \"image-registry-cf9777cd6-d9zpc\" (UID: \"4b133a69-01d6-42b5-b12f-d063856c87cb\") " pod="openshift-image-registry/image-registry-cf9777cd6-d9zpc" Jul 10 18:03:42.725004 ip-10-0-128-39 kubenswrapper[2558]: I0710 18:03:42.725009 2558 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-tls\" (UniqueName: \"kubernetes.io/secret/15c4360f-6bc7-4948-b0e6-ebc018174856-metrics-tls\") pod \"dns-default-22khc\" (UID: \"15c4360f-6bc7-4948-b0e6-ebc018174856\") " pod="openshift-dns/dns-default-22khc" Jul 10 18:03:42.725262 ip-10-0-128-39 kubenswrapper[2558]: E0710 18:03:42.725124 2558 secret.go:189] Couldn't get secret openshift-dns/dns-default-metrics-tls: secret "dns-default-metrics-tls" not found Jul 10 18:03:42.725262 ip-10-0-128-39 kubenswrapper[2558]: E0710 18:03:42.725135 2558 projected.go:266] Couldn't get secret openshift-image-registry/image-registry-tls: secret "image-registry-tls" not found Jul 10 18:03:42.725262 ip-10-0-128-39 kubenswrapper[2558]: E0710 18:03:42.725158 2558 projected.go:196] Error preparing data for projected volume registry-tls for pod openshift-image-registry/image-registry-cf9777cd6-d9zpc: secret "image-registry-tls" not found Jul 10 18:03:42.725262 ip-10-0-128-39 kubenswrapper[2558]: E0710 18:03:42.725175 2558 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/15c4360f-6bc7-4948-b0e6-ebc018174856-metrics-tls podName:15c4360f-6bc7-4948-b0e6-ebc018174856 nodeName:}" failed. No retries permitted until 2026-07-10 18:03:58.725161039 +0000 UTC m=+67.228653523 (durationBeforeRetry 16s). Error: MountVolume.SetUp failed for volume "metrics-tls" (UniqueName: "kubernetes.io/secret/15c4360f-6bc7-4948-b0e6-ebc018174856-metrics-tls") pod "dns-default-22khc" (UID: "15c4360f-6bc7-4948-b0e6-ebc018174856") : secret "dns-default-metrics-tls" not found Jul 10 18:03:42.725262 ip-10-0-128-39 kubenswrapper[2558]: E0710 18:03:42.725209 2558 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/projected/4b133a69-01d6-42b5-b12f-d063856c87cb-registry-tls podName:4b133a69-01d6-42b5-b12f-d063856c87cb nodeName:}" failed. No retries permitted until 2026-07-10 18:03:58.725193555 +0000 UTC m=+67.228686039 (durationBeforeRetry 16s). Error: MountVolume.SetUp failed for volume "registry-tls" (UniqueName: "kubernetes.io/projected/4b133a69-01d6-42b5-b12f-d063856c87cb-registry-tls") pod "image-registry-cf9777cd6-d9zpc" (UID: "4b133a69-01d6-42b5-b12f-d063856c87cb") : secret "image-registry-tls" not found Jul 10 18:03:43.367053 ip-10-0-128-39 kubenswrapper[2558]: I0710 18:03:43.367018 2558 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-kube-storage-version-migrator/migrator-7f59b54f6c-5nctn" event={"ID":"fb2c6bad-c788-4a76-beab-e424603069c4","Type":"ContainerStarted","Data":"73a02f844c64c1e63605d71dc54f3e9ad3d495fba1ce1583945df404cf066c1b"} Jul 10 18:03:43.367438 ip-10-0-128-39 kubenswrapper[2558]: I0710 18:03:43.367105 2558 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-kube-storage-version-migrator/migrator-7f59b54f6c-5nctn" event={"ID":"fb2c6bad-c788-4a76-beab-e424603069c4","Type":"ContainerStarted","Data":"8609a02154baa3da035ccfd084f0ea4861e681d910c54d882fa9c0824ae6c102"} Jul 10 18:03:43.385395 ip-10-0-128-39 kubenswrapper[2558]: I0710 18:03:43.385347 2558 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-kube-storage-version-migrator/migrator-7f59b54f6c-5nctn" podStartSLOduration=1.3135100450000001 podStartE2EDuration="3.385331493s" totalImagesPullingTime="2.071821448s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-10 18:03:40 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-10 18:03:43.384155283 +0000 UTC m=+51.887647789" watchObservedRunningTime="2026-07-10 18:03:43.385331493 +0000 UTC m=+51.888824000" Jul 10 18:03:44.036428 ip-10-0-128-39 kubenswrapper[2558]: I0710 18:03:44.036394 2558 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"cluster-monitoring-operator-tls\" (UniqueName: \"kubernetes.io/secret/19b8d048-b0b5-4b42-9f4f-ad9f47a105cd-cluster-monitoring-operator-tls\") pod \"cluster-monitoring-operator-5f7f494dd4-46fc9\" (UID: \"19b8d048-b0b5-4b42-9f4f-ad9f47a105cd\") " pod="openshift-monitoring/cluster-monitoring-operator-5f7f494dd4-46fc9" Jul 10 18:03:44.036612 ip-10-0-128-39 kubenswrapper[2558]: E0710 18:03:44.036544 2558 secret.go:189] Couldn't get secret openshift-monitoring/cluster-monitoring-operator-tls: secret "cluster-monitoring-operator-tls" not found Jul 10 18:03:44.036665 ip-10-0-128-39 kubenswrapper[2558]: E0710 18:03:44.036620 2558 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/19b8d048-b0b5-4b42-9f4f-ad9f47a105cd-cluster-monitoring-operator-tls podName:19b8d048-b0b5-4b42-9f4f-ad9f47a105cd nodeName:}" failed. No retries permitted until 2026-07-10 18:04:00.036603712 +0000 UTC m=+68.540096196 (durationBeforeRetry 16s). Error: MountVolume.SetUp failed for volume "cluster-monitoring-operator-tls" (UniqueName: "kubernetes.io/secret/19b8d048-b0b5-4b42-9f4f-ad9f47a105cd-cluster-monitoring-operator-tls") pod "cluster-monitoring-operator-5f7f494dd4-46fc9" (UID: "19b8d048-b0b5-4b42-9f4f-ad9f47a105cd") : secret "cluster-monitoring-operator-tls" not found Jul 10 18:03:44.137502 ip-10-0-128-39 kubenswrapper[2558]: I0710 18:03:44.137462 2558 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"networking-console-plugin-cert\" (UniqueName: \"kubernetes.io/secret/b95b59af-225c-4907-9643-5d916a325e56-networking-console-plugin-cert\") pod \"networking-console-plugin-6cf856758c-qg5vl\" (UID: \"b95b59af-225c-4907-9643-5d916a325e56\") " pod="openshift-network-console/networking-console-plugin-6cf856758c-qg5vl" Jul 10 18:03:44.137502 ip-10-0-128-39 kubenswrapper[2558]: I0710 18:03:44.137502 2558 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"samples-operator-tls\" (UniqueName: \"kubernetes.io/secret/5c7b84a3-17f6-4ed7-bb37-5e5b19a52916-samples-operator-tls\") pod \"cluster-samples-operator-65db858894-2b2km\" (UID: \"5c7b84a3-17f6-4ed7-bb37-5e5b19a52916\") " pod="openshift-cluster-samples-operator/cluster-samples-operator-65db858894-2b2km" Jul 10 18:03:44.137750 ip-10-0-128-39 kubenswrapper[2558]: E0710 18:03:44.137611 2558 secret.go:189] Couldn't get secret openshift-network-console/networking-console-plugin-cert: secret "networking-console-plugin-cert" not found Jul 10 18:03:44.137750 ip-10-0-128-39 kubenswrapper[2558]: E0710 18:03:44.137701 2558 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/b95b59af-225c-4907-9643-5d916a325e56-networking-console-plugin-cert podName:b95b59af-225c-4907-9643-5d916a325e56 nodeName:}" failed. No retries permitted until 2026-07-10 18:04:00.137682417 +0000 UTC m=+68.641174908 (durationBeforeRetry 16s). Error: MountVolume.SetUp failed for volume "networking-console-plugin-cert" (UniqueName: "kubernetes.io/secret/b95b59af-225c-4907-9643-5d916a325e56-networking-console-plugin-cert") pod "networking-console-plugin-6cf856758c-qg5vl" (UID: "b95b59af-225c-4907-9643-5d916a325e56") : secret "networking-console-plugin-cert" not found Jul 10 18:03:44.140039 ip-10-0-128-39 kubenswrapper[2558]: I0710 18:03:44.140013 2558 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"samples-operator-tls\" (UniqueName: \"kubernetes.io/secret/5c7b84a3-17f6-4ed7-bb37-5e5b19a52916-samples-operator-tls\") pod \"cluster-samples-operator-65db858894-2b2km\" (UID: \"5c7b84a3-17f6-4ed7-bb37-5e5b19a52916\") " pod="openshift-cluster-samples-operator/cluster-samples-operator-65db858894-2b2km" Jul 10 18:03:44.290148 ip-10-0-128-39 kubenswrapper[2558]: I0710 18:03:44.290061 2558 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-cluster-samples-operator/cluster-samples-operator-65db858894-2b2km" Jul 10 18:03:44.339748 ip-10-0-128-39 kubenswrapper[2558]: I0710 18:03:44.339716 2558 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"service-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/5fab65cc-f578-4cae-919e-1e630186e6d3-service-ca-bundle\") pod \"router-default-cd6bb8744-wq24m\" (UID: \"5fab65cc-f578-4cae-919e-1e630186e6d3\") " pod="openshift-ingress/router-default-cd6bb8744-wq24m" Jul 10 18:03:44.339937 ip-10-0-128-39 kubenswrapper[2558]: I0710 18:03:44.339787 2558 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/5fab65cc-f578-4cae-919e-1e630186e6d3-metrics-certs\") pod \"router-default-cd6bb8744-wq24m\" (UID: \"5fab65cc-f578-4cae-919e-1e630186e6d3\") " pod="openshift-ingress/router-default-cd6bb8744-wq24m" Jul 10 18:03:44.339937 ip-10-0-128-39 kubenswrapper[2558]: E0710 18:03:44.339906 2558 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/configmap/5fab65cc-f578-4cae-919e-1e630186e6d3-service-ca-bundle podName:5fab65cc-f578-4cae-919e-1e630186e6d3 nodeName:}" failed. No retries permitted until 2026-07-10 18:04:00.339886408 +0000 UTC m=+68.843378896 (durationBeforeRetry 16s). Error: MountVolume.SetUp failed for volume "service-ca-bundle" (UniqueName: "kubernetes.io/configmap/5fab65cc-f578-4cae-919e-1e630186e6d3-service-ca-bundle") pod "router-default-cd6bb8744-wq24m" (UID: "5fab65cc-f578-4cae-919e-1e630186e6d3") : configmap references non-existent config key: service-ca.crt Jul 10 18:03:44.339937 ip-10-0-128-39 kubenswrapper[2558]: E0710 18:03:44.339915 2558 secret.go:189] Couldn't get secret openshift-ingress/router-metrics-certs-default: secret "router-metrics-certs-default" not found Jul 10 18:03:44.340069 ip-10-0-128-39 kubenswrapper[2558]: E0710 18:03:44.339967 2558 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/5fab65cc-f578-4cae-919e-1e630186e6d3-metrics-certs podName:5fab65cc-f578-4cae-919e-1e630186e6d3 nodeName:}" failed. No retries permitted until 2026-07-10 18:04:00.339951104 +0000 UTC m=+68.843443588 (durationBeforeRetry 16s). Error: MountVolume.SetUp failed for volume "metrics-certs" (UniqueName: "kubernetes.io/secret/5fab65cc-f578-4cae-919e-1e630186e6d3-metrics-certs") pod "router-default-cd6bb8744-wq24m" (UID: "5fab65cc-f578-4cae-919e-1e630186e6d3") : secret "router-metrics-certs-default" not found Jul 10 18:03:44.450904 ip-10-0-128-39 kubenswrapper[2558]: I0710 18:03:44.450865 2558 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-cluster-samples-operator/cluster-samples-operator-65db858894-2b2km"] Jul 10 18:03:45.376563 ip-10-0-128-39 kubenswrapper[2558]: I0710 18:03:45.376532 2558 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-cluster-samples-operator/cluster-samples-operator-65db858894-2b2km" event={"ID":"5c7b84a3-17f6-4ed7-bb37-5e5b19a52916","Type":"ContainerStarted","Data":"de7ee4d2cd1de8326c9eb4266ba08bfe79de939d044493611d9cf6ccf055d664"} Jul 10 18:03:45.855598 ip-10-0-128-39 kubenswrapper[2558]: I0710 18:03:45.855564 2558 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"insights-runtime-extractor-tls\" (UniqueName: \"kubernetes.io/secret/cf337123-3a94-4667-a27e-7fc30a589451-insights-runtime-extractor-tls\") pod \"insights-runtime-extractor-lbs9p\" (UID: \"cf337123-3a94-4667-a27e-7fc30a589451\") " pod="openshift-insights/insights-runtime-extractor-lbs9p" Jul 10 18:03:45.856099 ip-10-0-128-39 kubenswrapper[2558]: I0710 18:03:45.855621 2558 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-rbac-proxy-cm\" (UniqueName: \"kubernetes.io/configmap/cf337123-3a94-4667-a27e-7fc30a589451-kube-rbac-proxy-cm\") pod \"insights-runtime-extractor-lbs9p\" (UID: \"cf337123-3a94-4667-a27e-7fc30a589451\") " pod="openshift-insights/insights-runtime-extractor-lbs9p" Jul 10 18:03:45.856099 ip-10-0-128-39 kubenswrapper[2558]: E0710 18:03:45.855732 2558 secret.go:189] Couldn't get secret openshift-insights/insights-runtime-extractor-tls: secret "insights-runtime-extractor-tls" not found Jul 10 18:03:45.856099 ip-10-0-128-39 kubenswrapper[2558]: E0710 18:03:45.855818 2558 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/cf337123-3a94-4667-a27e-7fc30a589451-insights-runtime-extractor-tls podName:cf337123-3a94-4667-a27e-7fc30a589451 nodeName:}" failed. No retries permitted until 2026-07-10 18:03:53.855784161 +0000 UTC m=+62.359276644 (durationBeforeRetry 8s). Error: MountVolume.SetUp failed for volume "insights-runtime-extractor-tls" (UniqueName: "kubernetes.io/secret/cf337123-3a94-4667-a27e-7fc30a589451-insights-runtime-extractor-tls") pod "insights-runtime-extractor-lbs9p" (UID: "cf337123-3a94-4667-a27e-7fc30a589451") : secret "insights-runtime-extractor-tls" not found Jul 10 18:03:45.856304 ip-10-0-128-39 kubenswrapper[2558]: I0710 18:03:45.856281 2558 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-rbac-proxy-cm\" (UniqueName: \"kubernetes.io/configmap/cf337123-3a94-4667-a27e-7fc30a589451-kube-rbac-proxy-cm\") pod \"insights-runtime-extractor-lbs9p\" (UID: \"cf337123-3a94-4667-a27e-7fc30a589451\") " pod="openshift-insights/insights-runtime-extractor-lbs9p" Jul 10 18:03:47.385089 ip-10-0-128-39 kubenswrapper[2558]: I0710 18:03:47.385053 2558 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-cluster-samples-operator/cluster-samples-operator-65db858894-2b2km" event={"ID":"5c7b84a3-17f6-4ed7-bb37-5e5b19a52916","Type":"ContainerStarted","Data":"101f15ee3f77e129fe0d81781e2fbcda3695f3df9a727cf331edea97dad742b4"} Jul 10 18:03:47.385563 ip-10-0-128-39 kubenswrapper[2558]: I0710 18:03:47.385102 2558 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-cluster-samples-operator/cluster-samples-operator-65db858894-2b2km" event={"ID":"5c7b84a3-17f6-4ed7-bb37-5e5b19a52916","Type":"ContainerStarted","Data":"85548e06793e16b2ac0ce39b5670f25a85309307e337d165f5911fdbd098b829"} Jul 10 18:03:47.401899 ip-10-0-128-39 kubenswrapper[2558]: I0710 18:03:47.401849 2558 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-cluster-samples-operator/cluster-samples-operator-65db858894-2b2km" podStartSLOduration=17.007994809 podStartE2EDuration="19.401832248s" totalImagesPullingTime="2.393837439s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-10 18:03:28 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-10 18:03:47.401304495 +0000 UTC m=+55.904797002" watchObservedRunningTime="2026-07-10 18:03:47.401832248 +0000 UTC m=+55.905324750" Jul 10 18:03:53.095600 ip-10-0-128-39 kubenswrapper[2558]: I0710 18:03:53.095570 2558 scope.go:117] "RemoveContainer" containerID="4e0c837200fa48d93da2f5ad906f1315f35e0e717200ab41550fef8a1543726b" Jul 10 18:03:53.267613 ip-10-0-128-39 kubenswrapper[2558]: I0710 18:03:53.267583 2558 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ovn-kubernetes/ovnkube-node-ll44q" Jul 10 18:03:53.410103 ip-10-0-128-39 kubenswrapper[2558]: I0710 18:03:53.410026 2558 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-console-operator_console-operator-7866497494-26542_2b29fb39-89b3-42c3-b334-3a195242248c/console-operator/1.log" Jul 10 18:03:53.410103 ip-10-0-128-39 kubenswrapper[2558]: I0710 18:03:53.410098 2558 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-console-operator/console-operator-7866497494-26542" event={"ID":"2b29fb39-89b3-42c3-b334-3a195242248c","Type":"ContainerStarted","Data":"89143790339ded47b31edfa274e61fc81fb2e89796aaa9b4cbd547bada42747b"} Jul 10 18:03:53.410391 ip-10-0-128-39 kubenswrapper[2558]: I0710 18:03:53.410371 2558 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-console-operator/console-operator-7866497494-26542" Jul 10 18:03:53.428929 ip-10-0-128-39 kubenswrapper[2558]: I0710 18:03:53.428879 2558 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-console-operator/console-operator-7866497494-26542" podStartSLOduration=17.751169945 podStartE2EDuration="25.42886227s" totalImagesPullingTime="7.677692325s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-10 18:03:28 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-10 18:03:53.42748486 +0000 UTC m=+61.930977404" watchObservedRunningTime="2026-07-10 18:03:53.42886227 +0000 UTC m=+61.932354776" Jul 10 18:03:53.483610 ip-10-0-128-39 kubenswrapper[2558]: I0710 18:03:53.483579 2558 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-console-operator/console-operator-7866497494-26542" Jul 10 18:03:53.930639 ip-10-0-128-39 kubenswrapper[2558]: I0710 18:03:53.930601 2558 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"insights-runtime-extractor-tls\" (UniqueName: \"kubernetes.io/secret/cf337123-3a94-4667-a27e-7fc30a589451-insights-runtime-extractor-tls\") pod \"insights-runtime-extractor-lbs9p\" (UID: \"cf337123-3a94-4667-a27e-7fc30a589451\") " pod="openshift-insights/insights-runtime-extractor-lbs9p" Jul 10 18:03:53.932883 ip-10-0-128-39 kubenswrapper[2558]: I0710 18:03:53.932862 2558 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"insights-runtime-extractor-tls\" (UniqueName: \"kubernetes.io/secret/cf337123-3a94-4667-a27e-7fc30a589451-insights-runtime-extractor-tls\") pod \"insights-runtime-extractor-lbs9p\" (UID: \"cf337123-3a94-4667-a27e-7fc30a589451\") " pod="openshift-insights/insights-runtime-extractor-lbs9p" Jul 10 18:03:54.014636 ip-10-0-128-39 kubenswrapper[2558]: I0710 18:03:54.014609 2558 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-insights\"/\"insights-runtime-extractor-sa-dockercfg-cpkz8\"" Jul 10 18:03:54.022871 ip-10-0-128-39 kubenswrapper[2558]: I0710 18:03:54.022846 2558 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-insights/insights-runtime-extractor-lbs9p" Jul 10 18:03:54.176355 ip-10-0-128-39 kubenswrapper[2558]: I0710 18:03:54.176330 2558 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-insights/insights-runtime-extractor-lbs9p"] Jul 10 18:03:54.178688 ip-10-0-128-39 kubenswrapper[2558]: W0710 18:03:54.178658 2558 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podcf337123_3a94_4667_a27e_7fc30a589451.slice/crio-4036e28815f77ea6eaada52abd6d16ba656c3d34c54654d94ac1aa977820c96f WatchSource:0}: Error finding container 4036e28815f77ea6eaada52abd6d16ba656c3d34c54654d94ac1aa977820c96f: Status 404 returned error can't find the container with id 4036e28815f77ea6eaada52abd6d16ba656c3d34c54654d94ac1aa977820c96f Jul 10 18:03:54.415040 ip-10-0-128-39 kubenswrapper[2558]: I0710 18:03:54.415005 2558 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-insights/insights-runtime-extractor-lbs9p" event={"ID":"cf337123-3a94-4667-a27e-7fc30a589451","Type":"ContainerStarted","Data":"4036e28815f77ea6eaada52abd6d16ba656c3d34c54654d94ac1aa977820c96f"} Jul 10 18:03:57.766277 ip-10-0-128-39 kubenswrapper[2558]: I0710 18:03:57.766221 2558 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"original-pull-secret\" (UniqueName: \"kubernetes.io/secret/3b27429f-90aa-4170-9078-0aa4ea0d0ab8-original-pull-secret\") pod \"global-pull-secret-syncer-vsbdm\" (UID: \"3b27429f-90aa-4170-9078-0aa4ea0d0ab8\") " pod="kube-system/global-pull-secret-syncer-vsbdm" Jul 10 18:03:57.766277 ip-10-0-128-39 kubenswrapper[2558]: I0710 18:03:57.766283 2558 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/7fa20252-06b7-48e4-8620-3e52dc980486-metrics-certs\") pod \"network-metrics-daemon-jkr77\" (UID: \"7fa20252-06b7-48e4-8620-3e52dc980486\") " pod="openshift-multus/network-metrics-daemon-jkr77" Jul 10 18:03:57.766723 ip-10-0-128-39 kubenswrapper[2558]: I0710 18:03:57.766331 2558 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-h74dt\" (UniqueName: \"kubernetes.io/projected/06aa905d-9b4b-4d36-bafc-8fa0c11b3057-kube-api-access-h74dt\") pod \"network-check-target-rzfgn\" (UID: \"06aa905d-9b4b-4d36-bafc-8fa0c11b3057\") " pod="openshift-network-diagnostics/network-check-target-rzfgn" Jul 10 18:03:57.769259 ip-10-0-128-39 kubenswrapper[2558]: I0710 18:03:57.769231 2558 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-h74dt\" (UniqueName: \"kubernetes.io/projected/06aa905d-9b4b-4d36-bafc-8fa0c11b3057-kube-api-access-h74dt\") pod \"network-check-target-rzfgn\" (UID: \"06aa905d-9b4b-4d36-bafc-8fa0c11b3057\") " pod="openshift-network-diagnostics/network-check-target-rzfgn" Jul 10 18:03:57.769399 ip-10-0-128-39 kubenswrapper[2558]: I0710 18:03:57.769286 2558 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"kube-system\"/\"original-pull-secret\"" Jul 10 18:03:57.769939 ip-10-0-128-39 kubenswrapper[2558]: I0710 18:03:57.769918 2558 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-multus\"/\"metrics-daemon-secret\"" Jul 10 18:03:57.780028 ip-10-0-128-39 kubenswrapper[2558]: I0710 18:03:57.779999 2558 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"original-pull-secret\" (UniqueName: \"kubernetes.io/secret/3b27429f-90aa-4170-9078-0aa4ea0d0ab8-original-pull-secret\") pod \"global-pull-secret-syncer-vsbdm\" (UID: \"3b27429f-90aa-4170-9078-0aa4ea0d0ab8\") " pod="kube-system/global-pull-secret-syncer-vsbdm" Jul 10 18:03:57.780028 ip-10-0-128-39 kubenswrapper[2558]: I0710 18:03:57.780025 2558 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/7fa20252-06b7-48e4-8620-3e52dc980486-metrics-certs\") pod \"network-metrics-daemon-jkr77\" (UID: \"7fa20252-06b7-48e4-8620-3e52dc980486\") " pod="openshift-multus/network-metrics-daemon-jkr77" Jul 10 18:03:57.807431 ip-10-0-128-39 kubenswrapper[2558]: I0710 18:03:57.807400 2558 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-multus\"/\"metrics-daemon-sa-dockercfg-fb6vc\"" Jul 10 18:03:57.815569 ip-10-0-128-39 kubenswrapper[2558]: I0710 18:03:57.815546 2558 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-jkr77" Jul 10 18:03:58.009310 ip-10-0-128-39 kubenswrapper[2558]: I0710 18:03:58.009279 2558 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-network-diagnostics\"/\"default-dockercfg-7bsz9\"" Jul 10 18:03:58.012421 ip-10-0-128-39 kubenswrapper[2558]: I0710 18:03:58.012398 2558 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-vsbdm" Jul 10 18:03:58.017180 ip-10-0-128-39 kubenswrapper[2558]: I0710 18:03:58.017137 2558 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-rzfgn" Jul 10 18:03:58.156124 ip-10-0-128-39 kubenswrapper[2558]: I0710 18:03:58.156044 2558 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-multus/network-metrics-daemon-jkr77"] Jul 10 18:03:58.160044 ip-10-0-128-39 kubenswrapper[2558]: W0710 18:03:58.159998 2558 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod7fa20252_06b7_48e4_8620_3e52dc980486.slice/crio-a230ab8a3043de946b75a8bf5b63872064f8a893465e351f3c7f44d9dea6bb64 WatchSource:0}: Error finding container a230ab8a3043de946b75a8bf5b63872064f8a893465e351f3c7f44d9dea6bb64: Status 404 returned error can't find the container with id a230ab8a3043de946b75a8bf5b63872064f8a893465e351f3c7f44d9dea6bb64 Jul 10 18:03:58.195975 ip-10-0-128-39 kubenswrapper[2558]: I0710 18:03:58.195946 2558 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-network-diagnostics/network-check-target-rzfgn"] Jul 10 18:03:58.199419 ip-10-0-128-39 kubenswrapper[2558]: W0710 18:03:58.199388 2558 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod06aa905d_9b4b_4d36_bafc_8fa0c11b3057.slice/crio-0b70e6e17ae9d44056c1a1b2ca823c612d7731118373733752bbe8d479e1a1d2 WatchSource:0}: Error finding container 0b70e6e17ae9d44056c1a1b2ca823c612d7731118373733752bbe8d479e1a1d2: Status 404 returned error can't find the container with id 0b70e6e17ae9d44056c1a1b2ca823c612d7731118373733752bbe8d479e1a1d2 Jul 10 18:03:58.215428 ip-10-0-128-39 kubenswrapper[2558]: I0710 18:03:58.215403 2558 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kube-system/global-pull-secret-syncer-vsbdm"] Jul 10 18:03:58.230772 ip-10-0-128-39 kubenswrapper[2558]: W0710 18:03:58.230744 2558 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod3b27429f_90aa_4170_9078_0aa4ea0d0ab8.slice/crio-ce515f678e5ce3e3e2be3ec05cbfb5d8f20ca8f366aef272fa549bf71d6bbbf6 WatchSource:0}: Error finding container ce515f678e5ce3e3e2be3ec05cbfb5d8f20ca8f366aef272fa549bf71d6bbbf6: Status 404 returned error can't find the container with id ce515f678e5ce3e3e2be3ec05cbfb5d8f20ca8f366aef272fa549bf71d6bbbf6 Jul 10 18:03:58.436876 ip-10-0-128-39 kubenswrapper[2558]: I0710 18:03:58.436768 2558 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kube-system/global-pull-secret-syncer-vsbdm" event={"ID":"3b27429f-90aa-4170-9078-0aa4ea0d0ab8","Type":"ContainerStarted","Data":"ce515f678e5ce3e3e2be3ec05cbfb5d8f20ca8f366aef272fa549bf71d6bbbf6"} Jul 10 18:03:58.438300 ip-10-0-128-39 kubenswrapper[2558]: I0710 18:03:58.438275 2558 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-insights/insights-runtime-extractor-lbs9p" event={"ID":"cf337123-3a94-4667-a27e-7fc30a589451","Type":"ContainerStarted","Data":"8f9cece662bf9d47e6fe15799dbb500145f850fce1c1a8d493f4d0ace8314515"} Jul 10 18:03:58.439714 ip-10-0-128-39 kubenswrapper[2558]: I0710 18:03:58.439693 2558 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-network-diagnostics/network-check-target-rzfgn" event={"ID":"06aa905d-9b4b-4d36-bafc-8fa0c11b3057","Type":"ContainerStarted","Data":"b8ac20adbc660c39988d90d651ff091dfd45be12643b36b04c93969f91b60182"} Jul 10 18:03:58.439776 ip-10-0-128-39 kubenswrapper[2558]: I0710 18:03:58.439730 2558 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-network-diagnostics/network-check-target-rzfgn" event={"ID":"06aa905d-9b4b-4d36-bafc-8fa0c11b3057","Type":"ContainerStarted","Data":"0b70e6e17ae9d44056c1a1b2ca823c612d7731118373733752bbe8d479e1a1d2"} Jul 10 18:03:58.439856 ip-10-0-128-39 kubenswrapper[2558]: I0710 18:03:58.439769 2558 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-network-diagnostics/network-check-target-rzfgn" Jul 10 18:03:58.440848 ip-10-0-128-39 kubenswrapper[2558]: I0710 18:03:58.440827 2558 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-multus/network-metrics-daemon-jkr77" event={"ID":"7fa20252-06b7-48e4-8620-3e52dc980486","Type":"ContainerStarted","Data":"a230ab8a3043de946b75a8bf5b63872064f8a893465e351f3c7f44d9dea6bb64"} Jul 10 18:03:58.456721 ip-10-0-128-39 kubenswrapper[2558]: I0710 18:03:58.456676 2558 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-network-diagnostics/network-check-target-rzfgn" podStartSLOduration=66.456663255 podStartE2EDuration="1m6.456663255s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-10 18:02:52 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-10 18:03:58.456424038 +0000 UTC m=+66.959916545" watchObservedRunningTime="2026-07-10 18:03:58.456663255 +0000 UTC m=+66.960155762" Jul 10 18:03:58.774794 ip-10-0-128-39 kubenswrapper[2558]: I0710 18:03:58.774755 2558 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"registry-tls\" (UniqueName: \"kubernetes.io/projected/4b133a69-01d6-42b5-b12f-d063856c87cb-registry-tls\") pod \"image-registry-cf9777cd6-d9zpc\" (UID: \"4b133a69-01d6-42b5-b12f-d063856c87cb\") " pod="openshift-image-registry/image-registry-cf9777cd6-d9zpc" Jul 10 18:03:58.775323 ip-10-0-128-39 kubenswrapper[2558]: I0710 18:03:58.774802 2558 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-tls\" (UniqueName: \"kubernetes.io/secret/15c4360f-6bc7-4948-b0e6-ebc018174856-metrics-tls\") pod \"dns-default-22khc\" (UID: \"15c4360f-6bc7-4948-b0e6-ebc018174856\") " pod="openshift-dns/dns-default-22khc" Jul 10 18:03:58.777488 ip-10-0-128-39 kubenswrapper[2558]: I0710 18:03:58.777464 2558 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"registry-tls\" (UniqueName: \"kubernetes.io/projected/4b133a69-01d6-42b5-b12f-d063856c87cb-registry-tls\") pod \"image-registry-cf9777cd6-d9zpc\" (UID: \"4b133a69-01d6-42b5-b12f-d063856c87cb\") " pod="openshift-image-registry/image-registry-cf9777cd6-d9zpc" Jul 10 18:03:58.777615 ip-10-0-128-39 kubenswrapper[2558]: I0710 18:03:58.777595 2558 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"metrics-tls\" (UniqueName: \"kubernetes.io/secret/15c4360f-6bc7-4948-b0e6-ebc018174856-metrics-tls\") pod \"dns-default-22khc\" (UID: \"15c4360f-6bc7-4948-b0e6-ebc018174856\") " pod="openshift-dns/dns-default-22khc" Jul 10 18:03:58.936722 ip-10-0-128-39 kubenswrapper[2558]: I0710 18:03:58.936483 2558 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-image-registry\"/\"registry-dockercfg-g4gf6\"" Jul 10 18:03:58.944567 ip-10-0-128-39 kubenswrapper[2558]: I0710 18:03:58.944534 2558 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-image-registry/image-registry-cf9777cd6-d9zpc" Jul 10 18:03:58.958458 ip-10-0-128-39 kubenswrapper[2558]: I0710 18:03:58.958436 2558 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-dns\"/\"dns-dockercfg-4kztg\"" Jul 10 18:03:58.966592 ip-10-0-128-39 kubenswrapper[2558]: I0710 18:03:58.966518 2558 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-dns/dns-default-22khc" Jul 10 18:03:59.099902 ip-10-0-128-39 kubenswrapper[2558]: I0710 18:03:59.099869 2558 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-image-registry/image-registry-cf9777cd6-d9zpc"] Jul 10 18:03:59.342924 ip-10-0-128-39 kubenswrapper[2558]: W0710 18:03:59.342847 2558 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod4b133a69_01d6_42b5_b12f_d063856c87cb.slice/crio-772572baec4bfb2b4d2a54152143108d673793a6ad178a51ed113d60dcf31380 WatchSource:0}: Error finding container 772572baec4bfb2b4d2a54152143108d673793a6ad178a51ed113d60dcf31380: Status 404 returned error can't find the container with id 772572baec4bfb2b4d2a54152143108d673793a6ad178a51ed113d60dcf31380 Jul 10 18:03:59.449457 ip-10-0-128-39 kubenswrapper[2558]: I0710 18:03:59.449354 2558 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-insights/insights-runtime-extractor-lbs9p" event={"ID":"cf337123-3a94-4667-a27e-7fc30a589451","Type":"ContainerStarted","Data":"a6f09a1e06b6333f21912a0bab47a4400d99a267c89eed87499b7328a93541ad"} Jul 10 18:03:59.452999 ip-10-0-128-39 kubenswrapper[2558]: I0710 18:03:59.452864 2558 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-image-registry/image-registry-cf9777cd6-d9zpc" event={"ID":"4b133a69-01d6-42b5-b12f-d063856c87cb","Type":"ContainerStarted","Data":"3eadfc79825d6da28ca7f0788e774cc4c167cf06e2a2d9b85dd22e439b1c49ec"} Jul 10 18:03:59.452999 ip-10-0-128-39 kubenswrapper[2558]: I0710 18:03:59.452910 2558 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-image-registry/image-registry-cf9777cd6-d9zpc" event={"ID":"4b133a69-01d6-42b5-b12f-d063856c87cb","Type":"ContainerStarted","Data":"772572baec4bfb2b4d2a54152143108d673793a6ad178a51ed113d60dcf31380"} Jul 10 18:03:59.452999 ip-10-0-128-39 kubenswrapper[2558]: I0710 18:03:59.452956 2558 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-image-registry/image-registry-cf9777cd6-d9zpc" Jul 10 18:03:59.476136 ip-10-0-128-39 kubenswrapper[2558]: I0710 18:03:59.475916 2558 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-image-registry/image-registry-cf9777cd6-d9zpc" podStartSLOduration=67.47589463 podStartE2EDuration="1m7.47589463s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-10 18:02:52 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-10 18:03:59.47384601 +0000 UTC m=+67.977338513" watchObservedRunningTime="2026-07-10 18:03:59.47589463 +0000 UTC m=+67.979387140" Jul 10 18:03:59.514631 ip-10-0-128-39 kubenswrapper[2558]: I0710 18:03:59.514604 2558 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-dns/dns-default-22khc"] Jul 10 18:03:59.518211 ip-10-0-128-39 kubenswrapper[2558]: W0710 18:03:59.518179 2558 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod15c4360f_6bc7_4948_b0e6_ebc018174856.slice/crio-816722e98cb5ad7f30580c97b8cd50c5411665983fb86ef3af5448a67d2b7018 WatchSource:0}: Error finding container 816722e98cb5ad7f30580c97b8cd50c5411665983fb86ef3af5448a67d2b7018: Status 404 returned error can't find the container with id 816722e98cb5ad7f30580c97b8cd50c5411665983fb86ef3af5448a67d2b7018 Jul 10 18:04:00.085443 ip-10-0-128-39 kubenswrapper[2558]: I0710 18:04:00.085300 2558 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"cluster-monitoring-operator-tls\" (UniqueName: \"kubernetes.io/secret/19b8d048-b0b5-4b42-9f4f-ad9f47a105cd-cluster-monitoring-operator-tls\") pod \"cluster-monitoring-operator-5f7f494dd4-46fc9\" (UID: \"19b8d048-b0b5-4b42-9f4f-ad9f47a105cd\") " pod="openshift-monitoring/cluster-monitoring-operator-5f7f494dd4-46fc9" Jul 10 18:04:00.089778 ip-10-0-128-39 kubenswrapper[2558]: I0710 18:04:00.089683 2558 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"cluster-monitoring-operator-tls\" (UniqueName: \"kubernetes.io/secret/19b8d048-b0b5-4b42-9f4f-ad9f47a105cd-cluster-monitoring-operator-tls\") pod \"cluster-monitoring-operator-5f7f494dd4-46fc9\" (UID: \"19b8d048-b0b5-4b42-9f4f-ad9f47a105cd\") " pod="openshift-monitoring/cluster-monitoring-operator-5f7f494dd4-46fc9" Jul 10 18:04:00.186804 ip-10-0-128-39 kubenswrapper[2558]: I0710 18:04:00.186762 2558 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"networking-console-plugin-cert\" (UniqueName: \"kubernetes.io/secret/b95b59af-225c-4907-9643-5d916a325e56-networking-console-plugin-cert\") pod \"networking-console-plugin-6cf856758c-qg5vl\" (UID: \"b95b59af-225c-4907-9643-5d916a325e56\") " pod="openshift-network-console/networking-console-plugin-6cf856758c-qg5vl" Jul 10 18:04:00.189988 ip-10-0-128-39 kubenswrapper[2558]: I0710 18:04:00.189959 2558 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"networking-console-plugin-cert\" (UniqueName: \"kubernetes.io/secret/b95b59af-225c-4907-9643-5d916a325e56-networking-console-plugin-cert\") pod \"networking-console-plugin-6cf856758c-qg5vl\" (UID: \"b95b59af-225c-4907-9643-5d916a325e56\") " pod="openshift-network-console/networking-console-plugin-6cf856758c-qg5vl" Jul 10 18:04:00.380196 ip-10-0-128-39 kubenswrapper[2558]: I0710 18:04:00.380096 2558 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"cluster-monitoring-operator-dockercfg-bx2s5\"" Jul 10 18:04:00.387683 ip-10-0-128-39 kubenswrapper[2558]: I0710 18:04:00.387650 2558 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-monitoring/cluster-monitoring-operator-5f7f494dd4-46fc9" Jul 10 18:04:00.388058 ip-10-0-128-39 kubenswrapper[2558]: I0710 18:04:00.388033 2558 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/5fab65cc-f578-4cae-919e-1e630186e6d3-metrics-certs\") pod \"router-default-cd6bb8744-wq24m\" (UID: \"5fab65cc-f578-4cae-919e-1e630186e6d3\") " pod="openshift-ingress/router-default-cd6bb8744-wq24m" Jul 10 18:04:00.388165 ip-10-0-128-39 kubenswrapper[2558]: I0710 18:04:00.388132 2558 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"service-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/5fab65cc-f578-4cae-919e-1e630186e6d3-service-ca-bundle\") pod \"router-default-cd6bb8744-wq24m\" (UID: \"5fab65cc-f578-4cae-919e-1e630186e6d3\") " pod="openshift-ingress/router-default-cd6bb8744-wq24m" Jul 10 18:04:00.388867 ip-10-0-128-39 kubenswrapper[2558]: I0710 18:04:00.388820 2558 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"service-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/5fab65cc-f578-4cae-919e-1e630186e6d3-service-ca-bundle\") pod \"router-default-cd6bb8744-wq24m\" (UID: \"5fab65cc-f578-4cae-919e-1e630186e6d3\") " pod="openshift-ingress/router-default-cd6bb8744-wq24m" Jul 10 18:04:00.391203 ip-10-0-128-39 kubenswrapper[2558]: I0710 18:04:00.391177 2558 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/5fab65cc-f578-4cae-919e-1e630186e6d3-metrics-certs\") pod \"router-default-cd6bb8744-wq24m\" (UID: \"5fab65cc-f578-4cae-919e-1e630186e6d3\") " pod="openshift-ingress/router-default-cd6bb8744-wq24m" Jul 10 18:04:00.460004 ip-10-0-128-39 kubenswrapper[2558]: I0710 18:04:00.459918 2558 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-dns/dns-default-22khc" event={"ID":"15c4360f-6bc7-4948-b0e6-ebc018174856","Type":"ContainerStarted","Data":"816722e98cb5ad7f30580c97b8cd50c5411665983fb86ef3af5448a67d2b7018"} Jul 10 18:04:00.463386 ip-10-0-128-39 kubenswrapper[2558]: I0710 18:04:00.463338 2558 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-multus/network-metrics-daemon-jkr77" event={"ID":"7fa20252-06b7-48e4-8620-3e52dc980486","Type":"ContainerStarted","Data":"13f0c309464f36f63f6830e2579231ba5c64b4132a73d0c0ad3ad5b9b3a78de3"} Jul 10 18:04:00.463682 ip-10-0-128-39 kubenswrapper[2558]: I0710 18:04:00.463590 2558 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-multus/network-metrics-daemon-jkr77" event={"ID":"7fa20252-06b7-48e4-8620-3e52dc980486","Type":"ContainerStarted","Data":"a554eb5eed7ce0ab713a4b450047162e50c84ff78db62de9ad1ee68e3bc1c844"} Jul 10 18:04:00.473971 ip-10-0-128-39 kubenswrapper[2558]: I0710 18:04:00.473948 2558 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-network-console\"/\"default-dockercfg-cwkwl\"" Jul 10 18:04:00.482397 ip-10-0-128-39 kubenswrapper[2558]: I0710 18:04:00.482167 2558 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-multus/network-metrics-daemon-jkr77" podStartSLOduration=67.279967275 podStartE2EDuration="1m8.482146074s" totalImagesPullingTime="1.202178799s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-10 18:02:52 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-10 18:04:00.479767495 +0000 UTC m=+68.983260005" watchObservedRunningTime="2026-07-10 18:04:00.482146074 +0000 UTC m=+68.985638583" Jul 10 18:04:00.482397 ip-10-0-128-39 kubenswrapper[2558]: I0710 18:04:00.482381 2558 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-console/networking-console-plugin-6cf856758c-qg5vl" Jul 10 18:04:00.576318 ip-10-0-128-39 kubenswrapper[2558]: I0710 18:04:00.576070 2558 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-monitoring/cluster-monitoring-operator-5f7f494dd4-46fc9"] Jul 10 18:04:00.579361 ip-10-0-128-39 kubenswrapper[2558]: W0710 18:04:00.579331 2558 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod19b8d048_b0b5_4b42_9f4f_ad9f47a105cd.slice/crio-48dcf3f6769ecc61cb84f2c0e1ac6b05f2cc565eba09f44f74f4448de9a10700 WatchSource:0}: Error finding container 48dcf3f6769ecc61cb84f2c0e1ac6b05f2cc565eba09f44f74f4448de9a10700: Status 404 returned error can't find the container with id 48dcf3f6769ecc61cb84f2c0e1ac6b05f2cc565eba09f44f74f4448de9a10700 Jul 10 18:04:00.610087 ip-10-0-128-39 kubenswrapper[2558]: I0710 18:04:00.609879 2558 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress\"/\"router-dockercfg-w6gqc\"" Jul 10 18:04:00.618434 ip-10-0-128-39 kubenswrapper[2558]: I0710 18:04:00.618128 2558 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/router-default-cd6bb8744-wq24m" Jul 10 18:04:00.673196 ip-10-0-128-39 kubenswrapper[2558]: I0710 18:04:00.672864 2558 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-network-console/networking-console-plugin-6cf856758c-qg5vl"] Jul 10 18:04:00.675400 ip-10-0-128-39 kubenswrapper[2558]: W0710 18:04:00.675341 2558 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podb95b59af_225c_4907_9643_5d916a325e56.slice/crio-99366866947cbfda8381df47d5eeb256a5a4b87d305dfe19944933de3502f9be WatchSource:0}: Error finding container 99366866947cbfda8381df47d5eeb256a5a4b87d305dfe19944933de3502f9be: Status 404 returned error can't find the container with id 99366866947cbfda8381df47d5eeb256a5a4b87d305dfe19944933de3502f9be Jul 10 18:04:00.843610 ip-10-0-128-39 kubenswrapper[2558]: I0710 18:04:00.843574 2558 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/router-default-cd6bb8744-wq24m"] Jul 10 18:04:00.848190 ip-10-0-128-39 kubenswrapper[2558]: W0710 18:04:00.848160 2558 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod5fab65cc_f578_4cae_919e_1e630186e6d3.slice/crio-7d7594672e4254e96ddebd64bc4d0a339a512e043a7677e0609385da02fadc57 WatchSource:0}: Error finding container 7d7594672e4254e96ddebd64bc4d0a339a512e043a7677e0609385da02fadc57: Status 404 returned error can't find the container with id 7d7594672e4254e96ddebd64bc4d0a339a512e043a7677e0609385da02fadc57 Jul 10 18:04:01.471729 ip-10-0-128-39 kubenswrapper[2558]: I0710 18:04:01.471689 2558 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/router-default-cd6bb8744-wq24m" event={"ID":"5fab65cc-f578-4cae-919e-1e630186e6d3","Type":"ContainerStarted","Data":"ec24a6380228fed7edff176b806084730dfe023663a25fecd3ba242e44a229b6"} Jul 10 18:04:01.472160 ip-10-0-128-39 kubenswrapper[2558]: I0710 18:04:01.471755 2558 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/router-default-cd6bb8744-wq24m" event={"ID":"5fab65cc-f578-4cae-919e-1e630186e6d3","Type":"ContainerStarted","Data":"7d7594672e4254e96ddebd64bc4d0a339a512e043a7677e0609385da02fadc57"} Jul 10 18:04:01.475026 ip-10-0-128-39 kubenswrapper[2558]: I0710 18:04:01.474961 2558 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-network-console/networking-console-plugin-6cf856758c-qg5vl" event={"ID":"b95b59af-225c-4907-9643-5d916a325e56","Type":"ContainerStarted","Data":"99366866947cbfda8381df47d5eeb256a5a4b87d305dfe19944933de3502f9be"} Jul 10 18:04:01.476679 ip-10-0-128-39 kubenswrapper[2558]: I0710 18:04:01.476586 2558 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/cluster-monitoring-operator-5f7f494dd4-46fc9" event={"ID":"19b8d048-b0b5-4b42-9f4f-ad9f47a105cd","Type":"ContainerStarted","Data":"48dcf3f6769ecc61cb84f2c0e1ac6b05f2cc565eba09f44f74f4448de9a10700"} Jul 10 18:04:01.493693 ip-10-0-128-39 kubenswrapper[2558]: I0710 18:04:01.492626 2558 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-ingress/router-default-cd6bb8744-wq24m" podStartSLOduration=33.492605535 podStartE2EDuration="33.492605535s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-10 18:03:28 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-10 18:04:01.492587455 +0000 UTC m=+69.996079969" watchObservedRunningTime="2026-07-10 18:04:01.492605535 +0000 UTC m=+69.996098033" Jul 10 18:04:01.619304 ip-10-0-128-39 kubenswrapper[2558]: I0710 18:04:01.619267 2558 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="openshift-ingress/router-default-cd6bb8744-wq24m" Jul 10 18:04:01.622212 ip-10-0-128-39 kubenswrapper[2558]: I0710 18:04:01.622181 2558 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="openshift-ingress/router-default-cd6bb8744-wq24m" Jul 10 18:04:02.155028 ip-10-0-128-39 kubenswrapper[2558]: I0710 18:04:02.154996 2558 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["open-cluster-management-agent-addon/klusterlet-addon-workmgr-b54577f8c-dvxxx"] Jul 10 18:04:02.179115 ip-10-0-128-39 kubenswrapper[2558]: I0710 18:04:02.176891 2558 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["open-cluster-management-agent-addon/managed-serviceaccount-addon-agent-5bd6f88fbd-x2c6c"] Jul 10 18:04:02.179652 ip-10-0-128-39 kubenswrapper[2558]: I0710 18:04:02.179631 2558 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="open-cluster-management-agent-addon/klusterlet-addon-workmgr-b54577f8c-dvxxx" Jul 10 18:04:02.195183 ip-10-0-128-39 kubenswrapper[2558]: I0710 18:04:02.194877 2558 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"open-cluster-management-agent-addon\"/\"openshift-service-ca.crt\"" Jul 10 18:04:02.195183 ip-10-0-128-39 kubenswrapper[2558]: I0710 18:04:02.194945 2558 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"open-cluster-management-agent-addon\"/\"open-cluster-management-image-pull-credentials\"" Jul 10 18:04:02.195183 ip-10-0-128-39 kubenswrapper[2558]: I0710 18:04:02.194974 2558 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"open-cluster-management-agent-addon\"/\"kube-root-ca.crt\"" Jul 10 18:04:02.195820 ip-10-0-128-39 kubenswrapper[2558]: I0710 18:04:02.195615 2558 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"open-cluster-management-agent-addon\"/\"work-manager-hub-kubeconfig\"" Jul 10 18:04:02.204573 ip-10-0-128-39 kubenswrapper[2558]: I0710 18:04:02.204256 2558 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tmp\" (UniqueName: \"kubernetes.io/empty-dir/7fc55090-8676-4747-9164-db67fa5c7f7c-tmp\") pod \"klusterlet-addon-workmgr-b54577f8c-dvxxx\" (UID: \"7fc55090-8676-4747-9164-db67fa5c7f7c\") " pod="open-cluster-management-agent-addon/klusterlet-addon-workmgr-b54577f8c-dvxxx" Jul 10 18:04:02.204573 ip-10-0-128-39 kubenswrapper[2558]: I0710 18:04:02.204349 2558 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"klusterlet-config\" (UniqueName: \"kubernetes.io/secret/7fc55090-8676-4747-9164-db67fa5c7f7c-klusterlet-config\") pod \"klusterlet-addon-workmgr-b54577f8c-dvxxx\" (UID: \"7fc55090-8676-4747-9164-db67fa5c7f7c\") " pod="open-cluster-management-agent-addon/klusterlet-addon-workmgr-b54577f8c-dvxxx" Jul 10 18:04:02.204573 ip-10-0-128-39 kubenswrapper[2558]: I0710 18:04:02.204422 2558 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-bjqcq\" (UniqueName: \"kubernetes.io/projected/7fc55090-8676-4747-9164-db67fa5c7f7c-kube-api-access-bjqcq\") pod \"klusterlet-addon-workmgr-b54577f8c-dvxxx\" (UID: \"7fc55090-8676-4747-9164-db67fa5c7f7c\") " pod="open-cluster-management-agent-addon/klusterlet-addon-workmgr-b54577f8c-dvxxx" Jul 10 18:04:02.207085 ip-10-0-128-39 kubenswrapper[2558]: I0710 18:04:02.207057 2558 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["open-cluster-management-agent-addon/cluster-proxy-proxy-agent-5fc9697bb5-lcw4g"] Jul 10 18:04:02.207207 ip-10-0-128-39 kubenswrapper[2558]: I0710 18:04:02.207184 2558 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="open-cluster-management-agent-addon/managed-serviceaccount-addon-agent-5bd6f88fbd-x2c6c" Jul 10 18:04:02.209850 ip-10-0-128-39 kubenswrapper[2558]: I0710 18:04:02.209831 2558 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"open-cluster-management-agent-addon\"/\"managed-serviceaccount-dockercfg-h5cs2\"" Jul 10 18:04:02.210044 ip-10-0-128-39 kubenswrapper[2558]: I0710 18:04:02.210020 2558 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"open-cluster-management-agent-addon\"/\"managed-serviceaccount-hub-kubeconfig\"" Jul 10 18:04:02.223309 ip-10-0-128-39 kubenswrapper[2558]: I0710 18:04:02.223278 2558 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["open-cluster-management-agent-addon/klusterlet-addon-workmgr-b54577f8c-dvxxx"] Jul 10 18:04:02.223441 ip-10-0-128-39 kubenswrapper[2558]: I0710 18:04:02.223324 2558 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["open-cluster-management-agent-addon/managed-serviceaccount-addon-agent-5bd6f88fbd-x2c6c"] Jul 10 18:04:02.223441 ip-10-0-128-39 kubenswrapper[2558]: I0710 18:04:02.223340 2558 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["open-cluster-management-agent-addon/cluster-proxy-proxy-agent-5fc9697bb5-lcw4g"] Jul 10 18:04:02.223441 ip-10-0-128-39 kubenswrapper[2558]: I0710 18:04:02.223361 2558 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-console/downloads-68c9cbb497-64qk8"] Jul 10 18:04:02.223441 ip-10-0-128-39 kubenswrapper[2558]: I0710 18:04:02.223415 2558 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="open-cluster-management-agent-addon/cluster-proxy-proxy-agent-5fc9697bb5-lcw4g" Jul 10 18:04:02.225914 ip-10-0-128-39 kubenswrapper[2558]: I0710 18:04:02.225893 2558 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"open-cluster-management-agent-addon\"/\"cluster-proxy-hub-kubeconfig\"" Jul 10 18:04:02.226049 ip-10-0-128-39 kubenswrapper[2558]: I0710 18:04:02.225996 2558 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"open-cluster-management-agent-addon\"/\"cluster-proxy-ca\"" Jul 10 18:04:02.226287 ip-10-0-128-39 kubenswrapper[2558]: I0710 18:04:02.226254 2558 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"open-cluster-management-agent-addon\"/\"cluster-proxy-service-proxy-server-certificates\"" Jul 10 18:04:02.226648 ip-10-0-128-39 kubenswrapper[2558]: I0710 18:04:02.226600 2558 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"open-cluster-management-agent-addon\"/\"cluster-proxy-open-cluster-management.io-proxy-agent-signer-client-cert\"" Jul 10 18:04:02.243470 ip-10-0-128-39 kubenswrapper[2558]: I0710 18:04:02.243448 2558 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-console/downloads-68c9cbb497-64qk8"] Jul 10 18:04:02.243558 ip-10-0-128-39 kubenswrapper[2558]: I0710 18:04:02.243545 2558 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-console/downloads-68c9cbb497-64qk8" Jul 10 18:04:02.246836 ip-10-0-128-39 kubenswrapper[2558]: I0710 18:04:02.246802 2558 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-console\"/\"default-dockercfg-pm2jz\"" Jul 10 18:04:02.246952 ip-10-0-128-39 kubenswrapper[2558]: I0710 18:04:02.246849 2558 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-console\"/\"kube-root-ca.crt\"" Jul 10 18:04:02.247229 ip-10-0-128-39 kubenswrapper[2558]: I0710 18:04:02.247210 2558 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-console\"/\"openshift-service-ca.crt\"" Jul 10 18:04:02.305871 ip-10-0-128-39 kubenswrapper[2558]: I0710 18:04:02.305837 2558 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"service-proxy-server-cert\" (UniqueName: \"kubernetes.io/secret/3662ac8b-ea79-4b39-ad34-556580830af9-service-proxy-server-cert\") pod \"cluster-proxy-proxy-agent-5fc9697bb5-lcw4g\" (UID: \"3662ac8b-ea79-4b39-ad34-556580830af9\") " pod="open-cluster-management-agent-addon/cluster-proxy-proxy-agent-5fc9697bb5-lcw4g" Jul 10 18:04:02.305871 ip-10-0-128-39 kubenswrapper[2558]: I0710 18:04:02.305880 2558 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"hub\" (UniqueName: \"kubernetes.io/secret/3662ac8b-ea79-4b39-ad34-556580830af9-hub\") pod \"cluster-proxy-proxy-agent-5fc9697bb5-lcw4g\" (UID: \"3662ac8b-ea79-4b39-ad34-556580830af9\") " pod="open-cluster-management-agent-addon/cluster-proxy-proxy-agent-5fc9697bb5-lcw4g" Jul 10 18:04:02.306149 ip-10-0-128-39 kubenswrapper[2558]: I0710 18:04:02.305919 2558 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-2crsv\" (UniqueName: \"kubernetes.io/projected/660e93ff-f281-43ef-afd5-45d845669b41-kube-api-access-2crsv\") pod \"managed-serviceaccount-addon-agent-5bd6f88fbd-x2c6c\" (UID: \"660e93ff-f281-43ef-afd5-45d845669b41\") " pod="open-cluster-management-agent-addon/managed-serviceaccount-addon-agent-5bd6f88fbd-x2c6c" Jul 10 18:04:02.306149 ip-10-0-128-39 kubenswrapper[2558]: I0710 18:04:02.305958 2558 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-66hx5\" (UniqueName: \"kubernetes.io/projected/3662ac8b-ea79-4b39-ad34-556580830af9-kube-api-access-66hx5\") pod \"cluster-proxy-proxy-agent-5fc9697bb5-lcw4g\" (UID: \"3662ac8b-ea79-4b39-ad34-556580830af9\") " pod="open-cluster-management-agent-addon/cluster-proxy-proxy-agent-5fc9697bb5-lcw4g" Jul 10 18:04:02.306149 ip-10-0-128-39 kubenswrapper[2558]: I0710 18:04:02.305994 2558 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"hub-kubeconfig\" (UniqueName: \"kubernetes.io/secret/3662ac8b-ea79-4b39-ad34-556580830af9-hub-kubeconfig\") pod \"cluster-proxy-proxy-agent-5fc9697bb5-lcw4g\" (UID: \"3662ac8b-ea79-4b39-ad34-556580830af9\") " pod="open-cluster-management-agent-addon/cluster-proxy-proxy-agent-5fc9697bb5-lcw4g" Jul 10 18:04:02.306149 ip-10-0-128-39 kubenswrapper[2558]: I0710 18:04:02.306019 2558 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tmp\" (UniqueName: \"kubernetes.io/empty-dir/7fc55090-8676-4747-9164-db67fa5c7f7c-tmp\") pod \"klusterlet-addon-workmgr-b54577f8c-dvxxx\" (UID: \"7fc55090-8676-4747-9164-db67fa5c7f7c\") " pod="open-cluster-management-agent-addon/klusterlet-addon-workmgr-b54577f8c-dvxxx" Jul 10 18:04:02.306149 ip-10-0-128-39 kubenswrapper[2558]: I0710 18:04:02.306043 2558 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-s456h\" (UniqueName: \"kubernetes.io/projected/dfc9aa8c-b41d-48fb-b346-ad46b6682c5f-kube-api-access-s456h\") pod \"downloads-68c9cbb497-64qk8\" (UID: \"dfc9aa8c-b41d-48fb-b346-ad46b6682c5f\") " pod="openshift-console/downloads-68c9cbb497-64qk8" Jul 10 18:04:02.306149 ip-10-0-128-39 kubenswrapper[2558]: I0710 18:04:02.306062 2558 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"hub-kubeconfig\" (UniqueName: \"kubernetes.io/secret/660e93ff-f281-43ef-afd5-45d845669b41-hub-kubeconfig\") pod \"managed-serviceaccount-addon-agent-5bd6f88fbd-x2c6c\" (UID: \"660e93ff-f281-43ef-afd5-45d845669b41\") " pod="open-cluster-management-agent-addon/managed-serviceaccount-addon-agent-5bd6f88fbd-x2c6c" Jul 10 18:04:02.306149 ip-10-0-128-39 kubenswrapper[2558]: I0710 18:04:02.306080 2558 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"additional-service-ca\" (UniqueName: \"kubernetes.io/configmap/3662ac8b-ea79-4b39-ad34-556580830af9-additional-service-ca\") pod \"cluster-proxy-proxy-agent-5fc9697bb5-lcw4g\" (UID: \"3662ac8b-ea79-4b39-ad34-556580830af9\") " pod="open-cluster-management-agent-addon/cluster-proxy-proxy-agent-5fc9697bb5-lcw4g" Jul 10 18:04:02.306149 ip-10-0-128-39 kubenswrapper[2558]: I0710 18:04:02.306121 2558 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"klusterlet-config\" (UniqueName: \"kubernetes.io/secret/7fc55090-8676-4747-9164-db67fa5c7f7c-klusterlet-config\") pod \"klusterlet-addon-workmgr-b54577f8c-dvxxx\" (UID: \"7fc55090-8676-4747-9164-db67fa5c7f7c\") " pod="open-cluster-management-agent-addon/klusterlet-addon-workmgr-b54577f8c-dvxxx" Jul 10 18:04:02.306553 ip-10-0-128-39 kubenswrapper[2558]: I0710 18:04:02.306237 2558 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"ca\" (UniqueName: \"kubernetes.io/secret/3662ac8b-ea79-4b39-ad34-556580830af9-ca\") pod \"cluster-proxy-proxy-agent-5fc9697bb5-lcw4g\" (UID: \"3662ac8b-ea79-4b39-ad34-556580830af9\") " pod="open-cluster-management-agent-addon/cluster-proxy-proxy-agent-5fc9697bb5-lcw4g" Jul 10 18:04:02.306553 ip-10-0-128-39 kubenswrapper[2558]: I0710 18:04:02.306265 2558 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-bjqcq\" (UniqueName: \"kubernetes.io/projected/7fc55090-8676-4747-9164-db67fa5c7f7c-kube-api-access-bjqcq\") pod \"klusterlet-addon-workmgr-b54577f8c-dvxxx\" (UID: \"7fc55090-8676-4747-9164-db67fa5c7f7c\") " pod="open-cluster-management-agent-addon/klusterlet-addon-workmgr-b54577f8c-dvxxx" Jul 10 18:04:02.306553 ip-10-0-128-39 kubenswrapper[2558]: I0710 18:04:02.306457 2558 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"tmp\" (UniqueName: \"kubernetes.io/empty-dir/7fc55090-8676-4747-9164-db67fa5c7f7c-tmp\") pod \"klusterlet-addon-workmgr-b54577f8c-dvxxx\" (UID: \"7fc55090-8676-4747-9164-db67fa5c7f7c\") " pod="open-cluster-management-agent-addon/klusterlet-addon-workmgr-b54577f8c-dvxxx" Jul 10 18:04:02.309066 ip-10-0-128-39 kubenswrapper[2558]: I0710 18:04:02.309042 2558 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"klusterlet-config\" (UniqueName: \"kubernetes.io/secret/7fc55090-8676-4747-9164-db67fa5c7f7c-klusterlet-config\") pod \"klusterlet-addon-workmgr-b54577f8c-dvxxx\" (UID: \"7fc55090-8676-4747-9164-db67fa5c7f7c\") " pod="open-cluster-management-agent-addon/klusterlet-addon-workmgr-b54577f8c-dvxxx" Jul 10 18:04:02.324553 ip-10-0-128-39 kubenswrapper[2558]: I0710 18:04:02.324526 2558 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-bjqcq\" (UniqueName: \"kubernetes.io/projected/7fc55090-8676-4747-9164-db67fa5c7f7c-kube-api-access-bjqcq\") pod \"klusterlet-addon-workmgr-b54577f8c-dvxxx\" (UID: \"7fc55090-8676-4747-9164-db67fa5c7f7c\") " pod="open-cluster-management-agent-addon/klusterlet-addon-workmgr-b54577f8c-dvxxx" Jul 10 18:04:02.406844 ip-10-0-128-39 kubenswrapper[2558]: I0710 18:04:02.406737 2558 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-2crsv\" (UniqueName: \"kubernetes.io/projected/660e93ff-f281-43ef-afd5-45d845669b41-kube-api-access-2crsv\") pod \"managed-serviceaccount-addon-agent-5bd6f88fbd-x2c6c\" (UID: \"660e93ff-f281-43ef-afd5-45d845669b41\") " pod="open-cluster-management-agent-addon/managed-serviceaccount-addon-agent-5bd6f88fbd-x2c6c" Jul 10 18:04:02.406844 ip-10-0-128-39 kubenswrapper[2558]: I0710 18:04:02.406787 2558 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-66hx5\" (UniqueName: \"kubernetes.io/projected/3662ac8b-ea79-4b39-ad34-556580830af9-kube-api-access-66hx5\") pod \"cluster-proxy-proxy-agent-5fc9697bb5-lcw4g\" (UID: \"3662ac8b-ea79-4b39-ad34-556580830af9\") " pod="open-cluster-management-agent-addon/cluster-proxy-proxy-agent-5fc9697bb5-lcw4g" Jul 10 18:04:02.406844 ip-10-0-128-39 kubenswrapper[2558]: I0710 18:04:02.406837 2558 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"hub-kubeconfig\" (UniqueName: \"kubernetes.io/secret/3662ac8b-ea79-4b39-ad34-556580830af9-hub-kubeconfig\") pod \"cluster-proxy-proxy-agent-5fc9697bb5-lcw4g\" (UID: \"3662ac8b-ea79-4b39-ad34-556580830af9\") " pod="open-cluster-management-agent-addon/cluster-proxy-proxy-agent-5fc9697bb5-lcw4g" Jul 10 18:04:02.407133 ip-10-0-128-39 kubenswrapper[2558]: I0710 18:04:02.406889 2558 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-s456h\" (UniqueName: \"kubernetes.io/projected/dfc9aa8c-b41d-48fb-b346-ad46b6682c5f-kube-api-access-s456h\") pod \"downloads-68c9cbb497-64qk8\" (UID: \"dfc9aa8c-b41d-48fb-b346-ad46b6682c5f\") " pod="openshift-console/downloads-68c9cbb497-64qk8" Jul 10 18:04:02.407133 ip-10-0-128-39 kubenswrapper[2558]: I0710 18:04:02.406916 2558 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"hub-kubeconfig\" (UniqueName: \"kubernetes.io/secret/660e93ff-f281-43ef-afd5-45d845669b41-hub-kubeconfig\") pod \"managed-serviceaccount-addon-agent-5bd6f88fbd-x2c6c\" (UID: \"660e93ff-f281-43ef-afd5-45d845669b41\") " pod="open-cluster-management-agent-addon/managed-serviceaccount-addon-agent-5bd6f88fbd-x2c6c" Jul 10 18:04:02.407133 ip-10-0-128-39 kubenswrapper[2558]: I0710 18:04:02.406939 2558 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"additional-service-ca\" (UniqueName: \"kubernetes.io/configmap/3662ac8b-ea79-4b39-ad34-556580830af9-additional-service-ca\") pod \"cluster-proxy-proxy-agent-5fc9697bb5-lcw4g\" (UID: \"3662ac8b-ea79-4b39-ad34-556580830af9\") " pod="open-cluster-management-agent-addon/cluster-proxy-proxy-agent-5fc9697bb5-lcw4g" Jul 10 18:04:02.407133 ip-10-0-128-39 kubenswrapper[2558]: I0710 18:04:02.407009 2558 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"ca\" (UniqueName: \"kubernetes.io/secret/3662ac8b-ea79-4b39-ad34-556580830af9-ca\") pod \"cluster-proxy-proxy-agent-5fc9697bb5-lcw4g\" (UID: \"3662ac8b-ea79-4b39-ad34-556580830af9\") " pod="open-cluster-management-agent-addon/cluster-proxy-proxy-agent-5fc9697bb5-lcw4g" Jul 10 18:04:02.407133 ip-10-0-128-39 kubenswrapper[2558]: I0710 18:04:02.407036 2558 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"service-proxy-server-cert\" (UniqueName: \"kubernetes.io/secret/3662ac8b-ea79-4b39-ad34-556580830af9-service-proxy-server-cert\") pod \"cluster-proxy-proxy-agent-5fc9697bb5-lcw4g\" (UID: \"3662ac8b-ea79-4b39-ad34-556580830af9\") " pod="open-cluster-management-agent-addon/cluster-proxy-proxy-agent-5fc9697bb5-lcw4g" Jul 10 18:04:02.407133 ip-10-0-128-39 kubenswrapper[2558]: I0710 18:04:02.407065 2558 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"hub\" (UniqueName: \"kubernetes.io/secret/3662ac8b-ea79-4b39-ad34-556580830af9-hub\") pod \"cluster-proxy-proxy-agent-5fc9697bb5-lcw4g\" (UID: \"3662ac8b-ea79-4b39-ad34-556580830af9\") " pod="open-cluster-management-agent-addon/cluster-proxy-proxy-agent-5fc9697bb5-lcw4g" Jul 10 18:04:02.408126 ip-10-0-128-39 kubenswrapper[2558]: I0710 18:04:02.408084 2558 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"additional-service-ca\" (UniqueName: \"kubernetes.io/configmap/3662ac8b-ea79-4b39-ad34-556580830af9-additional-service-ca\") pod \"cluster-proxy-proxy-agent-5fc9697bb5-lcw4g\" (UID: \"3662ac8b-ea79-4b39-ad34-556580830af9\") " pod="open-cluster-management-agent-addon/cluster-proxy-proxy-agent-5fc9697bb5-lcw4g" Jul 10 18:04:02.410330 ip-10-0-128-39 kubenswrapper[2558]: I0710 18:04:02.410269 2558 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"hub\" (UniqueName: \"kubernetes.io/secret/3662ac8b-ea79-4b39-ad34-556580830af9-hub\") pod \"cluster-proxy-proxy-agent-5fc9697bb5-lcw4g\" (UID: \"3662ac8b-ea79-4b39-ad34-556580830af9\") " pod="open-cluster-management-agent-addon/cluster-proxy-proxy-agent-5fc9697bb5-lcw4g" Jul 10 18:04:02.410425 ip-10-0-128-39 kubenswrapper[2558]: I0710 18:04:02.410394 2558 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"ca\" (UniqueName: \"kubernetes.io/secret/3662ac8b-ea79-4b39-ad34-556580830af9-ca\") pod \"cluster-proxy-proxy-agent-5fc9697bb5-lcw4g\" (UID: \"3662ac8b-ea79-4b39-ad34-556580830af9\") " pod="open-cluster-management-agent-addon/cluster-proxy-proxy-agent-5fc9697bb5-lcw4g" Jul 10 18:04:02.410490 ip-10-0-128-39 kubenswrapper[2558]: I0710 18:04:02.410444 2558 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"hub-kubeconfig\" (UniqueName: \"kubernetes.io/secret/3662ac8b-ea79-4b39-ad34-556580830af9-hub-kubeconfig\") pod \"cluster-proxy-proxy-agent-5fc9697bb5-lcw4g\" (UID: \"3662ac8b-ea79-4b39-ad34-556580830af9\") " pod="open-cluster-management-agent-addon/cluster-proxy-proxy-agent-5fc9697bb5-lcw4g" Jul 10 18:04:02.410950 ip-10-0-128-39 kubenswrapper[2558]: I0710 18:04:02.410927 2558 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"service-proxy-server-cert\" (UniqueName: \"kubernetes.io/secret/3662ac8b-ea79-4b39-ad34-556580830af9-service-proxy-server-cert\") pod \"cluster-proxy-proxy-agent-5fc9697bb5-lcw4g\" (UID: \"3662ac8b-ea79-4b39-ad34-556580830af9\") " pod="open-cluster-management-agent-addon/cluster-proxy-proxy-agent-5fc9697bb5-lcw4g" Jul 10 18:04:02.411124 ip-10-0-128-39 kubenswrapper[2558]: I0710 18:04:02.411107 2558 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"hub-kubeconfig\" (UniqueName: \"kubernetes.io/secret/660e93ff-f281-43ef-afd5-45d845669b41-hub-kubeconfig\") pod \"managed-serviceaccount-addon-agent-5bd6f88fbd-x2c6c\" (UID: \"660e93ff-f281-43ef-afd5-45d845669b41\") " pod="open-cluster-management-agent-addon/managed-serviceaccount-addon-agent-5bd6f88fbd-x2c6c" Jul 10 18:04:02.421657 ip-10-0-128-39 kubenswrapper[2558]: I0710 18:04:02.421634 2558 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-2crsv\" (UniqueName: \"kubernetes.io/projected/660e93ff-f281-43ef-afd5-45d845669b41-kube-api-access-2crsv\") pod \"managed-serviceaccount-addon-agent-5bd6f88fbd-x2c6c\" (UID: \"660e93ff-f281-43ef-afd5-45d845669b41\") " pod="open-cluster-management-agent-addon/managed-serviceaccount-addon-agent-5bd6f88fbd-x2c6c" Jul 10 18:04:02.421907 ip-10-0-128-39 kubenswrapper[2558]: I0710 18:04:02.421885 2558 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-66hx5\" (UniqueName: \"kubernetes.io/projected/3662ac8b-ea79-4b39-ad34-556580830af9-kube-api-access-66hx5\") pod \"cluster-proxy-proxy-agent-5fc9697bb5-lcw4g\" (UID: \"3662ac8b-ea79-4b39-ad34-556580830af9\") " pod="open-cluster-management-agent-addon/cluster-proxy-proxy-agent-5fc9697bb5-lcw4g" Jul 10 18:04:02.422569 ip-10-0-128-39 kubenswrapper[2558]: I0710 18:04:02.422540 2558 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-s456h\" (UniqueName: \"kubernetes.io/projected/dfc9aa8c-b41d-48fb-b346-ad46b6682c5f-kube-api-access-s456h\") pod \"downloads-68c9cbb497-64qk8\" (UID: \"dfc9aa8c-b41d-48fb-b346-ad46b6682c5f\") " pod="openshift-console/downloads-68c9cbb497-64qk8" Jul 10 18:04:02.480693 ip-10-0-128-39 kubenswrapper[2558]: I0710 18:04:02.480660 2558 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/router-default-cd6bb8744-wq24m" Jul 10 18:04:02.482046 ip-10-0-128-39 kubenswrapper[2558]: I0710 18:04:02.482024 2558 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/router-default-cd6bb8744-wq24m" Jul 10 18:04:02.507839 ip-10-0-128-39 kubenswrapper[2558]: I0710 18:04:02.507791 2558 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="open-cluster-management-agent-addon/klusterlet-addon-workmgr-b54577f8c-dvxxx" Jul 10 18:04:02.528967 ip-10-0-128-39 kubenswrapper[2558]: I0710 18:04:02.528911 2558 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="open-cluster-management-agent-addon/managed-serviceaccount-addon-agent-5bd6f88fbd-x2c6c" Jul 10 18:04:02.540467 ip-10-0-128-39 kubenswrapper[2558]: I0710 18:04:02.540436 2558 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="open-cluster-management-agent-addon/cluster-proxy-proxy-agent-5fc9697bb5-lcw4g" Jul 10 18:04:02.570535 ip-10-0-128-39 kubenswrapper[2558]: I0710 18:04:02.570500 2558 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-console/downloads-68c9cbb497-64qk8" Jul 10 18:04:06.654384 ip-10-0-128-39 kubenswrapper[2558]: I0710 18:04:06.654293 2558 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["open-cluster-management-agent-addon/klusterlet-addon-workmgr-b54577f8c-dvxxx"] Jul 10 18:04:06.897591 ip-10-0-128-39 kubenswrapper[2558]: I0710 18:04:06.897547 2558 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["open-cluster-management-agent-addon/managed-serviceaccount-addon-agent-5bd6f88fbd-x2c6c"] Jul 10 18:04:06.901212 ip-10-0-128-39 kubenswrapper[2558]: W0710 18:04:06.901179 2558 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-besteffort.slice/kubepods-besteffort-pod660e93ff_f281_43ef_afd5_45d845669b41.slice/crio-f83974276e90db7335e23f272710018114c5bfd4c197bf6845663518875513b0 WatchSource:0}: Error finding container f83974276e90db7335e23f272710018114c5bfd4c197bf6845663518875513b0: Status 404 returned error can't find the container with id f83974276e90db7335e23f272710018114c5bfd4c197bf6845663518875513b0 Jul 10 18:04:06.902061 ip-10-0-128-39 kubenswrapper[2558]: I0710 18:04:06.901993 2558 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-console/downloads-68c9cbb497-64qk8"] Jul 10 18:04:06.906024 ip-10-0-128-39 kubenswrapper[2558]: W0710 18:04:06.905191 2558 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-poddfc9aa8c_b41d_48fb_b346_ad46b6682c5f.slice/crio-b26550835786178c872c7dbf65f798b8d20b6653f227b248eb0386b2ba1aebee WatchSource:0}: Error finding container b26550835786178c872c7dbf65f798b8d20b6653f227b248eb0386b2ba1aebee: Status 404 returned error can't find the container with id b26550835786178c872c7dbf65f798b8d20b6653f227b248eb0386b2ba1aebee Jul 10 18:04:06.907552 ip-10-0-128-39 kubenswrapper[2558]: I0710 18:04:06.907523 2558 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["open-cluster-management-agent-addon/cluster-proxy-proxy-agent-5fc9697bb5-lcw4g"] Jul 10 18:04:06.911054 ip-10-0-128-39 kubenswrapper[2558]: W0710 18:04:06.911031 2558 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-besteffort.slice/kubepods-besteffort-pod3662ac8b_ea79_4b39_ad34_556580830af9.slice/crio-9a96c2d3d4801e8b3760d185ad2b293f3fd1ced76bc76bf7de1d3782d7e15fae WatchSource:0}: Error finding container 9a96c2d3d4801e8b3760d185ad2b293f3fd1ced76bc76bf7de1d3782d7e15fae: Status 404 returned error can't find the container with id 9a96c2d3d4801e8b3760d185ad2b293f3fd1ced76bc76bf7de1d3782d7e15fae Jul 10 18:04:07.509577 ip-10-0-128-39 kubenswrapper[2558]: I0710 18:04:07.509536 2558 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="open-cluster-management-agent-addon/managed-serviceaccount-addon-agent-5bd6f88fbd-x2c6c" event={"ID":"660e93ff-f281-43ef-afd5-45d845669b41","Type":"ContainerStarted","Data":"f83974276e90db7335e23f272710018114c5bfd4c197bf6845663518875513b0"} Jul 10 18:04:07.511693 ip-10-0-128-39 kubenswrapper[2558]: I0710 18:04:07.511661 2558 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="open-cluster-management-agent-addon/cluster-proxy-proxy-agent-5fc9697bb5-lcw4g" event={"ID":"3662ac8b-ea79-4b39-ad34-556580830af9","Type":"ContainerStarted","Data":"9a96c2d3d4801e8b3760d185ad2b293f3fd1ced76bc76bf7de1d3782d7e15fae"} Jul 10 18:04:07.514146 ip-10-0-128-39 kubenswrapper[2558]: I0710 18:04:07.514114 2558 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kube-system/global-pull-secret-syncer-vsbdm" event={"ID":"3b27429f-90aa-4170-9078-0aa4ea0d0ab8","Type":"ContainerStarted","Data":"dda62db6f6c1f28bc031905a36707966ef7760cbfc0b5c5897ca82558d1d7426"} Jul 10 18:04:07.517622 ip-10-0-128-39 kubenswrapper[2558]: I0710 18:04:07.517532 2558 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-dns/dns-default-22khc" event={"ID":"15c4360f-6bc7-4948-b0e6-ebc018174856","Type":"ContainerStarted","Data":"e8c4fca5829606d84b86946567ad824c0ff2073583c1a4514f73ec9315467b78"} Jul 10 18:04:07.517622 ip-10-0-128-39 kubenswrapper[2558]: I0710 18:04:07.517587 2558 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-dns/dns-default-22khc" event={"ID":"15c4360f-6bc7-4948-b0e6-ebc018174856","Type":"ContainerStarted","Data":"cea60cf40a847e6e30e62413ca7f4990279e57f51df88e531e6d7817aa8302a8"} Jul 10 18:04:07.518130 ip-10-0-128-39 kubenswrapper[2558]: I0710 18:04:07.517989 2558 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-dns/dns-default-22khc" Jul 10 18:04:07.519764 ip-10-0-128-39 kubenswrapper[2558]: I0710 18:04:07.519710 2558 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="open-cluster-management-agent-addon/klusterlet-addon-workmgr-b54577f8c-dvxxx" event={"ID":"7fc55090-8676-4747-9164-db67fa5c7f7c","Type":"ContainerStarted","Data":"a87bd36168389df3f359c18b5cd1a5f9d746d669d6638a703983c4897c6847b0"} Jul 10 18:04:07.522837 ip-10-0-128-39 kubenswrapper[2558]: I0710 18:04:07.522492 2558 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-insights/insights-runtime-extractor-lbs9p" event={"ID":"cf337123-3a94-4667-a27e-7fc30a589451","Type":"ContainerStarted","Data":"8ea77fd12855614780bc2bcc6bc26186e9b438819ad5054c70fdf97ec46670dd"} Jul 10 18:04:07.525678 ip-10-0-128-39 kubenswrapper[2558]: I0710 18:04:07.525464 2558 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-network-console/networking-console-plugin-6cf856758c-qg5vl" event={"ID":"b95b59af-225c-4907-9643-5d916a325e56","Type":"ContainerStarted","Data":"bb8451761e17decb249dbf01efe5d3f0700ad7ffa0aec8b867b349ce4aa8e0f5"} Jul 10 18:04:07.530059 ip-10-0-128-39 kubenswrapper[2558]: I0710 18:04:07.529293 2558 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/cluster-monitoring-operator-5f7f494dd4-46fc9" event={"ID":"19b8d048-b0b5-4b42-9f4f-ad9f47a105cd","Type":"ContainerStarted","Data":"703385891df7e690c6ac1e6b0de6a3f7672d03633d45623c089465cd7ef8f159"} Jul 10 18:04:07.531766 ip-10-0-128-39 kubenswrapper[2558]: I0710 18:04:07.531229 2558 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="kube-system/global-pull-secret-syncer-vsbdm" podStartSLOduration=67.299812566 podStartE2EDuration="1m15.531213801s" totalImagesPullingTime="8.231401235s" totalInitContainerRuntime="0s" isStatefulPod=true podCreationTimestamp="2026-07-10 18:02:52 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-10 18:04:07.530960532 +0000 UTC m=+76.034453041" watchObservedRunningTime="2026-07-10 18:04:07.531213801 +0000 UTC m=+76.034706309" Jul 10 18:04:07.532993 ip-10-0-128-39 kubenswrapper[2558]: I0710 18:04:07.532861 2558 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-console/downloads-68c9cbb497-64qk8" event={"ID":"dfc9aa8c-b41d-48fb-b346-ad46b6682c5f","Type":"ContainerStarted","Data":"b26550835786178c872c7dbf65f798b8d20b6653f227b248eb0386b2ba1aebee"} Jul 10 18:04:07.547766 ip-10-0-128-39 kubenswrapper[2558]: I0710 18:04:07.547104 2558 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-monitoring/cluster-monitoring-operator-5f7f494dd4-46fc9" podStartSLOduration=33.660477014 podStartE2EDuration="39.547085711s" totalImagesPullingTime="5.886608697s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-10 18:03:28 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-10 18:04:07.545855849 +0000 UTC m=+76.049348356" watchObservedRunningTime="2026-07-10 18:04:07.547085711 +0000 UTC m=+76.050578221" Jul 10 18:04:07.560156 ip-10-0-128-39 kubenswrapper[2558]: I0710 18:04:07.560104 2558 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-network-console/networking-console-plugin-6cf856758c-qg5vl" podStartSLOduration=33.773251262 podStartE2EDuration="39.560086142s" totalImagesPullingTime="5.78683488s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-10 18:03:28 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-10 18:04:07.560048767 +0000 UTC m=+76.063541274" watchObservedRunningTime="2026-07-10 18:04:07.560086142 +0000 UTC m=+76.063578648" Jul 10 18:04:07.583611 ip-10-0-128-39 kubenswrapper[2558]: I0710 18:04:07.582751 2558 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-dns/dns-default-22khc" podStartSLOduration=34.679413909 podStartE2EDuration="41.582731046s" totalImagesPullingTime="6.903317137s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-10 18:03:26 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-10 18:04:07.581473669 +0000 UTC m=+76.084966177" watchObservedRunningTime="2026-07-10 18:04:07.582731046 +0000 UTC m=+76.086223553" Jul 10 18:04:07.603003 ip-10-0-128-39 kubenswrapper[2558]: I0710 18:04:07.601564 2558 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-insights/insights-runtime-extractor-lbs9p" podStartSLOduration=17.503525840000002 podStartE2EDuration="29.601546463s" totalImagesPullingTime="12.098020623s" totalInitContainerRuntime="0s" isStatefulPod=true podCreationTimestamp="2026-07-10 18:03:38 +0000 UTC" imagePullSessionsCount=3 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-10 18:04:07.600804571 +0000 UTC m=+76.104297080" watchObservedRunningTime="2026-07-10 18:04:07.601546463 +0000 UTC m=+76.105038969" Jul 10 18:04:13.574934 ip-10-0-128-39 kubenswrapper[2558]: I0710 18:04:13.574893 2558 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="open-cluster-management-agent-addon/managed-serviceaccount-addon-agent-5bd6f88fbd-x2c6c" event={"ID":"660e93ff-f281-43ef-afd5-45d845669b41","Type":"ContainerStarted","Data":"9dd50dec111f5a632a80bcb6eb478b4074d663ade8616c990d61bdb4a66071f3"} Jul 10 18:04:13.577313 ip-10-0-128-39 kubenswrapper[2558]: I0710 18:04:13.577233 2558 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="open-cluster-management-agent-addon/cluster-proxy-proxy-agent-5fc9697bb5-lcw4g" event={"ID":"3662ac8b-ea79-4b39-ad34-556580830af9","Type":"ContainerStarted","Data":"e92d131bb6dc59468fce0dc905ce63e6329a87c5c9f5cf31e7fa30d0d6c42fab"} Jul 10 18:04:13.577313 ip-10-0-128-39 kubenswrapper[2558]: I0710 18:04:13.577277 2558 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="open-cluster-management-agent-addon/cluster-proxy-proxy-agent-5fc9697bb5-lcw4g" event={"ID":"3662ac8b-ea79-4b39-ad34-556580830af9","Type":"ContainerStarted","Data":"6f15a200f392370f45e6a77fc7ba9d92bd138be4138483ef9f05dbff63de880b"} Jul 10 18:04:13.577313 ip-10-0-128-39 kubenswrapper[2558]: I0710 18:04:13.577293 2558 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="open-cluster-management-agent-addon/cluster-proxy-proxy-agent-5fc9697bb5-lcw4g" event={"ID":"3662ac8b-ea79-4b39-ad34-556580830af9","Type":"ContainerStarted","Data":"a81e14e9ca669ab2b13d96cf1603ae5dca18c3d146d16a7a67b6ab59e5979155"} Jul 10 18:04:13.579170 ip-10-0-128-39 kubenswrapper[2558]: I0710 18:04:13.579140 2558 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="open-cluster-management-agent-addon/klusterlet-addon-workmgr-b54577f8c-dvxxx" event={"ID":"7fc55090-8676-4747-9164-db67fa5c7f7c","Type":"ContainerStarted","Data":"6f64bfe0346bad8df4bf39b2afc0bf1bfcda22ea9f968b3f660611f9c3de2edf"} Jul 10 18:04:13.579347 ip-10-0-128-39 kubenswrapper[2558]: I0710 18:04:13.579230 2558 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="open-cluster-management-agent-addon/klusterlet-addon-workmgr-b54577f8c-dvxxx" Jul 10 18:04:13.581329 ip-10-0-128-39 kubenswrapper[2558]: I0710 18:04:13.581299 2558 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="open-cluster-management-agent-addon/klusterlet-addon-workmgr-b54577f8c-dvxxx" Jul 10 18:04:13.594641 ip-10-0-128-39 kubenswrapper[2558]: I0710 18:04:13.594598 2558 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="open-cluster-management-agent-addon/managed-serviceaccount-addon-agent-5bd6f88fbd-x2c6c" podStartSLOduration=5.311092928 podStartE2EDuration="11.594583261s" totalImagesPullingTime="6.283490333s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-10 18:04:02 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-10 18:04:13.59349249 +0000 UTC m=+82.096984999" watchObservedRunningTime="2026-07-10 18:04:13.594583261 +0000 UTC m=+82.098075746" Jul 10 18:04:13.616661 ip-10-0-128-39 kubenswrapper[2558]: I0710 18:04:13.616613 2558 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="open-cluster-management-agent-addon/klusterlet-addon-workmgr-b54577f8c-dvxxx" podStartSLOduration=5.073859857 podStartE2EDuration="11.616597437s" totalImagesPullingTime="6.54273758s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-10 18:04:02 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-10 18:04:13.615428365 +0000 UTC m=+82.118920871" watchObservedRunningTime="2026-07-10 18:04:13.616597437 +0000 UTC m=+82.120089942" Jul 10 18:04:13.634977 ip-10-0-128-39 kubenswrapper[2558]: I0710 18:04:13.634872 2558 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="open-cluster-management-agent-addon/cluster-proxy-proxy-agent-5fc9697bb5-lcw4g" podStartSLOduration=5.344605559 podStartE2EDuration="11.634851685s" totalImagesPullingTime="6.290246126s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-10 18:04:02 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-10 18:04:13.634510882 +0000 UTC m=+82.138003387" watchObservedRunningTime="2026-07-10 18:04:13.634851685 +0000 UTC m=+82.138344191" Jul 10 18:04:17.545893 ip-10-0-128-39 kubenswrapper[2558]: I0710 18:04:17.545736 2558 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-dns/dns-default-22khc" Jul 10 18:04:18.949830 ip-10-0-128-39 kubenswrapper[2558]: I0710 18:04:18.949764 2558 patch_prober.go:28] interesting pod/image-registry-cf9777cd6-d9zpc container/registry namespace/openshift-image-registry: Liveness probe status=failure output="HTTP probe failed with statuscode: 503" start-of-body={"errors":[{"code":"UNAVAILABLE","message":"service unavailable","detail":"health check failed: please see /debug/health"}]} Jul 10 18:04:18.950266 ip-10-0-128-39 kubenswrapper[2558]: I0710 18:04:18.949839 2558 prober.go:120] "Probe failed" probeType="Liveness" pod="openshift-image-registry/image-registry-cf9777cd6-d9zpc" podUID="4b133a69-01d6-42b5-b12f-d063856c87cb" containerName="registry" probeResult="failure" output="HTTP probe failed with statuscode: 503" Jul 10 18:04:19.699054 ip-10-0-128-39 kubenswrapper[2558]: I0710 18:04:19.698773 2558 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-monitoring/node-exporter-cj5n2"] Jul 10 18:04:19.732827 ip-10-0-128-39 kubenswrapper[2558]: I0710 18:04:19.732771 2558 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-monitoring/node-exporter-cj5n2" Jul 10 18:04:19.735399 ip-10-0-128-39 kubenswrapper[2558]: I0710 18:04:19.735367 2558 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"node-exporter-dockercfg-vwlhd\"" Jul 10 18:04:19.735546 ip-10-0-128-39 kubenswrapper[2558]: I0710 18:04:19.735501 2558 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-monitoring\"/\"metrics-client-ca\"" Jul 10 18:04:19.735735 ip-10-0-128-39 kubenswrapper[2558]: I0710 18:04:19.735626 2558 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-monitoring\"/\"node-exporter-accelerators-collector-config\"" Jul 10 18:04:19.736367 ip-10-0-128-39 kubenswrapper[2558]: I0710 18:04:19.736344 2558 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"node-exporter-tls\"" Jul 10 18:04:19.736367 ip-10-0-128-39 kubenswrapper[2558]: I0710 18:04:19.736344 2558 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"node-exporter-kube-rbac-proxy-config\"" Jul 10 18:04:19.866027 ip-10-0-128-39 kubenswrapper[2558]: I0710 18:04:19.865989 2558 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"node-exporter-accelerators-collector-config\" (UniqueName: \"kubernetes.io/configmap/0f113ed9-f2c4-44d5-9848-97e6250dbc22-node-exporter-accelerators-collector-config\") pod \"node-exporter-cj5n2\" (UID: \"0f113ed9-f2c4-44d5-9848-97e6250dbc22\") " pod="openshift-monitoring/node-exporter-cj5n2" Jul 10 18:04:19.866214 ip-10-0-128-39 kubenswrapper[2558]: I0710 18:04:19.866052 2558 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"node-exporter-wtmp\" (UniqueName: \"kubernetes.io/host-path/0f113ed9-f2c4-44d5-9848-97e6250dbc22-node-exporter-wtmp\") pod \"node-exporter-cj5n2\" (UID: \"0f113ed9-f2c4-44d5-9848-97e6250dbc22\") " pod="openshift-monitoring/node-exporter-cj5n2" Jul 10 18:04:19.866214 ip-10-0-128-39 kubenswrapper[2558]: I0710 18:04:19.866112 2558 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"node-exporter-textfile\" (UniqueName: \"kubernetes.io/empty-dir/0f113ed9-f2c4-44d5-9848-97e6250dbc22-node-exporter-textfile\") pod \"node-exporter-cj5n2\" (UID: \"0f113ed9-f2c4-44d5-9848-97e6250dbc22\") " pod="openshift-monitoring/node-exporter-cj5n2" Jul 10 18:04:19.866214 ip-10-0-128-39 kubenswrapper[2558]: I0710 18:04:19.866136 2558 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"sys\" (UniqueName: \"kubernetes.io/host-path/0f113ed9-f2c4-44d5-9848-97e6250dbc22-sys\") pod \"node-exporter-cj5n2\" (UID: \"0f113ed9-f2c4-44d5-9848-97e6250dbc22\") " pod="openshift-monitoring/node-exporter-cj5n2" Jul 10 18:04:19.866214 ip-10-0-128-39 kubenswrapper[2558]: I0710 18:04:19.866179 2558 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-52sn2\" (UniqueName: \"kubernetes.io/projected/0f113ed9-f2c4-44d5-9848-97e6250dbc22-kube-api-access-52sn2\") pod \"node-exporter-cj5n2\" (UID: \"0f113ed9-f2c4-44d5-9848-97e6250dbc22\") " pod="openshift-monitoring/node-exporter-cj5n2" Jul 10 18:04:19.866448 ip-10-0-128-39 kubenswrapper[2558]: I0710 18:04:19.866217 2558 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"node-exporter-kube-rbac-proxy-config\" (UniqueName: \"kubernetes.io/secret/0f113ed9-f2c4-44d5-9848-97e6250dbc22-node-exporter-kube-rbac-proxy-config\") pod \"node-exporter-cj5n2\" (UID: \"0f113ed9-f2c4-44d5-9848-97e6250dbc22\") " pod="openshift-monitoring/node-exporter-cj5n2" Jul 10 18:04:19.866448 ip-10-0-128-39 kubenswrapper[2558]: I0710 18:04:19.866268 2558 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"metrics-client-ca\" (UniqueName: \"kubernetes.io/configmap/0f113ed9-f2c4-44d5-9848-97e6250dbc22-metrics-client-ca\") pod \"node-exporter-cj5n2\" (UID: \"0f113ed9-f2c4-44d5-9848-97e6250dbc22\") " pod="openshift-monitoring/node-exporter-cj5n2" Jul 10 18:04:19.866448 ip-10-0-128-39 kubenswrapper[2558]: I0710 18:04:19.866294 2558 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"node-exporter-tls\" (UniqueName: \"kubernetes.io/secret/0f113ed9-f2c4-44d5-9848-97e6250dbc22-node-exporter-tls\") pod \"node-exporter-cj5n2\" (UID: \"0f113ed9-f2c4-44d5-9848-97e6250dbc22\") " pod="openshift-monitoring/node-exporter-cj5n2" Jul 10 18:04:19.866448 ip-10-0-128-39 kubenswrapper[2558]: I0710 18:04:19.866339 2558 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"root\" (UniqueName: \"kubernetes.io/host-path/0f113ed9-f2c4-44d5-9848-97e6250dbc22-root\") pod \"node-exporter-cj5n2\" (UID: \"0f113ed9-f2c4-44d5-9848-97e6250dbc22\") " pod="openshift-monitoring/node-exporter-cj5n2" Jul 10 18:04:19.967258 ip-10-0-128-39 kubenswrapper[2558]: I0710 18:04:19.967218 2558 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"node-exporter-accelerators-collector-config\" (UniqueName: \"kubernetes.io/configmap/0f113ed9-f2c4-44d5-9848-97e6250dbc22-node-exporter-accelerators-collector-config\") pod \"node-exporter-cj5n2\" (UID: \"0f113ed9-f2c4-44d5-9848-97e6250dbc22\") " pod="openshift-monitoring/node-exporter-cj5n2" Jul 10 18:04:19.967755 ip-10-0-128-39 kubenswrapper[2558]: I0710 18:04:19.967281 2558 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"node-exporter-wtmp\" (UniqueName: \"kubernetes.io/host-path/0f113ed9-f2c4-44d5-9848-97e6250dbc22-node-exporter-wtmp\") pod \"node-exporter-cj5n2\" (UID: \"0f113ed9-f2c4-44d5-9848-97e6250dbc22\") " pod="openshift-monitoring/node-exporter-cj5n2" Jul 10 18:04:19.967755 ip-10-0-128-39 kubenswrapper[2558]: I0710 18:04:19.967313 2558 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"node-exporter-textfile\" (UniqueName: \"kubernetes.io/empty-dir/0f113ed9-f2c4-44d5-9848-97e6250dbc22-node-exporter-textfile\") pod \"node-exporter-cj5n2\" (UID: \"0f113ed9-f2c4-44d5-9848-97e6250dbc22\") " pod="openshift-monitoring/node-exporter-cj5n2" Jul 10 18:04:19.967755 ip-10-0-128-39 kubenswrapper[2558]: I0710 18:04:19.967336 2558 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"sys\" (UniqueName: \"kubernetes.io/host-path/0f113ed9-f2c4-44d5-9848-97e6250dbc22-sys\") pod \"node-exporter-cj5n2\" (UID: \"0f113ed9-f2c4-44d5-9848-97e6250dbc22\") " pod="openshift-monitoring/node-exporter-cj5n2" Jul 10 18:04:19.967755 ip-10-0-128-39 kubenswrapper[2558]: I0710 18:04:19.967378 2558 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-52sn2\" (UniqueName: \"kubernetes.io/projected/0f113ed9-f2c4-44d5-9848-97e6250dbc22-kube-api-access-52sn2\") pod \"node-exporter-cj5n2\" (UID: \"0f113ed9-f2c4-44d5-9848-97e6250dbc22\") " pod="openshift-monitoring/node-exporter-cj5n2" Jul 10 18:04:19.967755 ip-10-0-128-39 kubenswrapper[2558]: I0710 18:04:19.967503 2558 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"sys\" (UniqueName: \"kubernetes.io/host-path/0f113ed9-f2c4-44d5-9848-97e6250dbc22-sys\") pod \"node-exporter-cj5n2\" (UID: \"0f113ed9-f2c4-44d5-9848-97e6250dbc22\") " pod="openshift-monitoring/node-exporter-cj5n2" Jul 10 18:04:19.967755 ip-10-0-128-39 kubenswrapper[2558]: I0710 18:04:19.967522 2558 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"node-exporter-wtmp\" (UniqueName: \"kubernetes.io/host-path/0f113ed9-f2c4-44d5-9848-97e6250dbc22-node-exporter-wtmp\") pod \"node-exporter-cj5n2\" (UID: \"0f113ed9-f2c4-44d5-9848-97e6250dbc22\") " pod="openshift-monitoring/node-exporter-cj5n2" Jul 10 18:04:19.967755 ip-10-0-128-39 kubenswrapper[2558]: I0710 18:04:19.967562 2558 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"node-exporter-kube-rbac-proxy-config\" (UniqueName: \"kubernetes.io/secret/0f113ed9-f2c4-44d5-9848-97e6250dbc22-node-exporter-kube-rbac-proxy-config\") pod \"node-exporter-cj5n2\" (UID: \"0f113ed9-f2c4-44d5-9848-97e6250dbc22\") " pod="openshift-monitoring/node-exporter-cj5n2" Jul 10 18:04:19.967755 ip-10-0-128-39 kubenswrapper[2558]: I0710 18:04:19.967625 2558 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-client-ca\" (UniqueName: \"kubernetes.io/configmap/0f113ed9-f2c4-44d5-9848-97e6250dbc22-metrics-client-ca\") pod \"node-exporter-cj5n2\" (UID: \"0f113ed9-f2c4-44d5-9848-97e6250dbc22\") " pod="openshift-monitoring/node-exporter-cj5n2" Jul 10 18:04:19.967755 ip-10-0-128-39 kubenswrapper[2558]: I0710 18:04:19.967653 2558 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"node-exporter-tls\" (UniqueName: \"kubernetes.io/secret/0f113ed9-f2c4-44d5-9848-97e6250dbc22-node-exporter-tls\") pod \"node-exporter-cj5n2\" (UID: \"0f113ed9-f2c4-44d5-9848-97e6250dbc22\") " pod="openshift-monitoring/node-exporter-cj5n2" Jul 10 18:04:19.967755 ip-10-0-128-39 kubenswrapper[2558]: I0710 18:04:19.967701 2558 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"root\" (UniqueName: \"kubernetes.io/host-path/0f113ed9-f2c4-44d5-9848-97e6250dbc22-root\") pod \"node-exporter-cj5n2\" (UID: \"0f113ed9-f2c4-44d5-9848-97e6250dbc22\") " pod="openshift-monitoring/node-exporter-cj5n2" Jul 10 18:04:19.968289 ip-10-0-128-39 kubenswrapper[2558]: I0710 18:04:19.967773 2558 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"root\" (UniqueName: \"kubernetes.io/host-path/0f113ed9-f2c4-44d5-9848-97e6250dbc22-root\") pod \"node-exporter-cj5n2\" (UID: \"0f113ed9-f2c4-44d5-9848-97e6250dbc22\") " pod="openshift-monitoring/node-exporter-cj5n2" Jul 10 18:04:19.968289 ip-10-0-128-39 kubenswrapper[2558]: E0710 18:04:19.967884 2558 secret.go:189] Couldn't get secret openshift-monitoring/node-exporter-tls: secret "node-exporter-tls" not found Jul 10 18:04:19.968289 ip-10-0-128-39 kubenswrapper[2558]: E0710 18:04:19.967937 2558 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/0f113ed9-f2c4-44d5-9848-97e6250dbc22-node-exporter-tls podName:0f113ed9-f2c4-44d5-9848-97e6250dbc22 nodeName:}" failed. No retries permitted until 2026-07-10 18:04:20.467918734 +0000 UTC m=+88.971411229 (durationBeforeRetry 500ms). Error: MountVolume.SetUp failed for volume "node-exporter-tls" (UniqueName: "kubernetes.io/secret/0f113ed9-f2c4-44d5-9848-97e6250dbc22-node-exporter-tls") pod "node-exporter-cj5n2" (UID: "0f113ed9-f2c4-44d5-9848-97e6250dbc22") : secret "node-exporter-tls" not found Jul 10 18:04:19.968289 ip-10-0-128-39 kubenswrapper[2558]: I0710 18:04:19.967989 2558 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"node-exporter-accelerators-collector-config\" (UniqueName: \"kubernetes.io/configmap/0f113ed9-f2c4-44d5-9848-97e6250dbc22-node-exporter-accelerators-collector-config\") pod \"node-exporter-cj5n2\" (UID: \"0f113ed9-f2c4-44d5-9848-97e6250dbc22\") " pod="openshift-monitoring/node-exporter-cj5n2" Jul 10 18:04:19.968289 ip-10-0-128-39 kubenswrapper[2558]: I0710 18:04:19.968238 2558 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"metrics-client-ca\" (UniqueName: \"kubernetes.io/configmap/0f113ed9-f2c4-44d5-9848-97e6250dbc22-metrics-client-ca\") pod \"node-exporter-cj5n2\" (UID: \"0f113ed9-f2c4-44d5-9848-97e6250dbc22\") " pod="openshift-monitoring/node-exporter-cj5n2" Jul 10 18:04:19.968531 ip-10-0-128-39 kubenswrapper[2558]: I0710 18:04:19.968321 2558 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"node-exporter-textfile\" (UniqueName: \"kubernetes.io/empty-dir/0f113ed9-f2c4-44d5-9848-97e6250dbc22-node-exporter-textfile\") pod \"node-exporter-cj5n2\" (UID: \"0f113ed9-f2c4-44d5-9848-97e6250dbc22\") " pod="openshift-monitoring/node-exporter-cj5n2" Jul 10 18:04:19.970678 ip-10-0-128-39 kubenswrapper[2558]: I0710 18:04:19.970633 2558 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"node-exporter-kube-rbac-proxy-config\" (UniqueName: \"kubernetes.io/secret/0f113ed9-f2c4-44d5-9848-97e6250dbc22-node-exporter-kube-rbac-proxy-config\") pod \"node-exporter-cj5n2\" (UID: \"0f113ed9-f2c4-44d5-9848-97e6250dbc22\") " pod="openshift-monitoring/node-exporter-cj5n2" Jul 10 18:04:19.982867 ip-10-0-128-39 kubenswrapper[2558]: I0710 18:04:19.982836 2558 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-52sn2\" (UniqueName: \"kubernetes.io/projected/0f113ed9-f2c4-44d5-9848-97e6250dbc22-kube-api-access-52sn2\") pod \"node-exporter-cj5n2\" (UID: \"0f113ed9-f2c4-44d5-9848-97e6250dbc22\") " pod="openshift-monitoring/node-exporter-cj5n2" Jul 10 18:04:20.472678 ip-10-0-128-39 kubenswrapper[2558]: I0710 18:04:20.472640 2558 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"node-exporter-tls\" (UniqueName: \"kubernetes.io/secret/0f113ed9-f2c4-44d5-9848-97e6250dbc22-node-exporter-tls\") pod \"node-exporter-cj5n2\" (UID: \"0f113ed9-f2c4-44d5-9848-97e6250dbc22\") " pod="openshift-monitoring/node-exporter-cj5n2" Jul 10 18:04:20.475478 ip-10-0-128-39 kubenswrapper[2558]: I0710 18:04:20.475434 2558 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"node-exporter-tls\" (UniqueName: \"kubernetes.io/secret/0f113ed9-f2c4-44d5-9848-97e6250dbc22-node-exporter-tls\") pod \"node-exporter-cj5n2\" (UID: \"0f113ed9-f2c4-44d5-9848-97e6250dbc22\") " pod="openshift-monitoring/node-exporter-cj5n2" Jul 10 18:04:20.647319 ip-10-0-128-39 kubenswrapper[2558]: I0710 18:04:20.647285 2558 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-monitoring/node-exporter-cj5n2" Jul 10 18:04:21.483403 ip-10-0-128-39 kubenswrapper[2558]: I0710 18:04:21.483370 2558 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-image-registry/image-registry-cf9777cd6-d9zpc" Jul 10 18:04:24.952672 ip-10-0-128-39 kubenswrapper[2558]: I0710 18:04:24.952633 2558 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-image-registry/image-registry-cf9777cd6-d9zpc"] Jul 10 18:04:26.289754 ip-10-0-128-39 kubenswrapper[2558]: W0710 18:04:26.288687 2558 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod0f113ed9_f2c4_44d5_9848_97e6250dbc22.slice/crio-c6d09af5bcfa6e25f7b2b74cfb86e6c67085d538222ac6944fb4ee883c7efdf9 WatchSource:0}: Error finding container c6d09af5bcfa6e25f7b2b74cfb86e6c67085d538222ac6944fb4ee883c7efdf9: Status 404 returned error can't find the container with id c6d09af5bcfa6e25f7b2b74cfb86e6c67085d538222ac6944fb4ee883c7efdf9 Jul 10 18:04:26.642069 ip-10-0-128-39 kubenswrapper[2558]: I0710 18:04:26.641980 2558 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/node-exporter-cj5n2" event={"ID":"0f113ed9-f2c4-44d5-9848-97e6250dbc22","Type":"ContainerStarted","Data":"c6d09af5bcfa6e25f7b2b74cfb86e6c67085d538222ac6944fb4ee883c7efdf9"} Jul 10 18:04:27.648383 ip-10-0-128-39 kubenswrapper[2558]: I0710 18:04:27.648337 2558 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-console/downloads-68c9cbb497-64qk8" event={"ID":"dfc9aa8c-b41d-48fb-b346-ad46b6682c5f","Type":"ContainerStarted","Data":"e5940bc34cc4ff241037c05804935e6e7d1c3574e9cfafef0d3da9bd3e2daace"} Jul 10 18:04:27.648868 ip-10-0-128-39 kubenswrapper[2558]: I0710 18:04:27.648569 2558 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-console/downloads-68c9cbb497-64qk8" Jul 10 18:04:27.667309 ip-10-0-128-39 kubenswrapper[2558]: I0710 18:04:27.667253 2558 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-console/downloads-68c9cbb497-64qk8" podStartSLOduration=5.507344384 podStartE2EDuration="25.667235837s" totalImagesPullingTime="20.159891453s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-10 18:04:02 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-10 18:04:27.665757717 +0000 UTC m=+96.169250225" watchObservedRunningTime="2026-07-10 18:04:27.667235837 +0000 UTC m=+96.170728344" Jul 10 18:04:27.672332 ip-10-0-128-39 kubenswrapper[2558]: I0710 18:04:27.672302 2558 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-console/downloads-68c9cbb497-64qk8" Jul 10 18:04:28.656017 ip-10-0-128-39 kubenswrapper[2558]: I0710 18:04:28.655977 2558 generic.go:356] "Generic (PLEG): container finished" podID="0f113ed9-f2c4-44d5-9848-97e6250dbc22" containerID="e5158fa1c61514117e22a885748a60cc0f51191c199cf71d2d635823801c2719" exitCode=0 Jul 10 18:04:28.656593 ip-10-0-128-39 kubenswrapper[2558]: I0710 18:04:28.656070 2558 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/node-exporter-cj5n2" event={"ID":"0f113ed9-f2c4-44d5-9848-97e6250dbc22","Type":"ContainerDied","Data":"e5158fa1c61514117e22a885748a60cc0f51191c199cf71d2d635823801c2719"} Jul 10 18:04:29.455456 ip-10-0-128-39 kubenswrapper[2558]: I0710 18:04:29.455418 2558 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-network-diagnostics/network-check-target-rzfgn" Jul 10 18:04:29.664254 ip-10-0-128-39 kubenswrapper[2558]: I0710 18:04:29.664214 2558 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/node-exporter-cj5n2" event={"ID":"0f113ed9-f2c4-44d5-9848-97e6250dbc22","Type":"ContainerStarted","Data":"89680b18666aaf19e91e1d56530aaa045bf2130232ad345045c92dcbeb527825"} Jul 10 18:04:29.664729 ip-10-0-128-39 kubenswrapper[2558]: I0710 18:04:29.664285 2558 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/node-exporter-cj5n2" event={"ID":"0f113ed9-f2c4-44d5-9848-97e6250dbc22","Type":"ContainerStarted","Data":"205fa5258df93eec03d329a3da7b776157f250e5dc95aa8abc318ac9721bd08e"} Jul 10 18:04:29.686523 ip-10-0-128-39 kubenswrapper[2558]: I0710 18:04:29.686463 2558 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-monitoring/node-exporter-cj5n2" podStartSLOduration=9.028869719 podStartE2EDuration="10.686442886s" totalImagesPullingTime="1.563844833s" totalInitContainerRuntime="93.728334ms" isStatefulPod=true podCreationTimestamp="2026-07-10 18:04:19 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-10 18:04:29.684566083 +0000 UTC m=+98.188058584" watchObservedRunningTime="2026-07-10 18:04:29.686442886 +0000 UTC m=+98.189935392" Jul 10 18:04:42.720785 ip-10-0-128-39 kubenswrapper[2558]: I0710 18:04:42.720747 2558 generic.go:356] "Generic (PLEG): container finished" podID="748dca80-632f-44ef-b3a2-5292df366af1" containerID="d3b4b1905749b7a33cf2188b7a1718c44f18c3f6d613a9e2183b17d13115dc27" exitCode=0 Jul 10 18:04:42.721212 ip-10-0-128-39 kubenswrapper[2558]: I0710 18:04:42.720837 2558 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-insights/insights-operator-68d9dd94cf-q5s5m" event={"ID":"748dca80-632f-44ef-b3a2-5292df366af1","Type":"ContainerDied","Data":"d3b4b1905749b7a33cf2188b7a1718c44f18c3f6d613a9e2183b17d13115dc27"} Jul 10 18:04:42.721212 ip-10-0-128-39 kubenswrapper[2558]: I0710 18:04:42.721159 2558 scope.go:117] "RemoveContainer" containerID="d3b4b1905749b7a33cf2188b7a1718c44f18c3f6d613a9e2183b17d13115dc27" Jul 10 18:04:43.728107 ip-10-0-128-39 kubenswrapper[2558]: I0710 18:04:43.728071 2558 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-insights/insights-operator-68d9dd94cf-q5s5m" event={"ID":"748dca80-632f-44ef-b3a2-5292df366af1","Type":"ContainerStarted","Data":"e539fac389083de53a2b2ca6b515d6005f53d2fe67668395c41f88fa2782ebf6"} Jul 10 18:04:43.757708 ip-10-0-128-39 kubenswrapper[2558]: I0710 18:04:43.757675 2558 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ingress_router-default-cd6bb8744-wq24m_5fab65cc-f578-4cae-919e-1e630186e6d3/router/0.log" Jul 10 18:04:43.770385 ip-10-0-128-39 kubenswrapper[2558]: I0710 18:04:43.770359 2558 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ingress-canary_ingress-canary-bpwbf_f754e509-abbe-4451-833d-abb0e28c7e19/serve-healthcheck-canary/0.log" Jul 10 18:04:49.982296 ip-10-0-128-39 kubenswrapper[2558]: I0710 18:04:49.982240 2558 kuberuntime_container.go:873] "Killing container with a grace period" pod="openshift-image-registry/image-registry-cf9777cd6-d9zpc" podUID="4b133a69-01d6-42b5-b12f-d063856c87cb" containerName="registry" containerID="cri-o://3eadfc79825d6da28ca7f0788e774cc4c167cf06e2a2d9b85dd22e439b1c49ec" gracePeriod=30 Jul 10 18:04:50.252445 ip-10-0-128-39 kubenswrapper[2558]: I0710 18:04:50.252379 2558 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-image-registry/image-registry-cf9777cd6-d9zpc" Jul 10 18:04:50.334653 ip-10-0-128-39 kubenswrapper[2558]: I0710 18:04:50.334620 2558 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"image-registry-private-configuration\" (UniqueName: \"kubernetes.io/secret/4b133a69-01d6-42b5-b12f-d063856c87cb-image-registry-private-configuration\") pod \"4b133a69-01d6-42b5-b12f-d063856c87cb\" (UID: \"4b133a69-01d6-42b5-b12f-d063856c87cb\") " Jul 10 18:04:50.334876 ip-10-0-128-39 kubenswrapper[2558]: I0710 18:04:50.334663 2558 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"registry-tls\" (UniqueName: \"kubernetes.io/projected/4b133a69-01d6-42b5-b12f-d063856c87cb-registry-tls\") pod \"4b133a69-01d6-42b5-b12f-d063856c87cb\" (UID: \"4b133a69-01d6-42b5-b12f-d063856c87cb\") " Jul 10 18:04:50.334876 ip-10-0-128-39 kubenswrapper[2558]: I0710 18:04:50.334691 2558 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"trusted-ca\" (UniqueName: \"kubernetes.io/configmap/4b133a69-01d6-42b5-b12f-d063856c87cb-trusted-ca\") pod \"4b133a69-01d6-42b5-b12f-d063856c87cb\" (UID: \"4b133a69-01d6-42b5-b12f-d063856c87cb\") " Jul 10 18:04:50.334876 ip-10-0-128-39 kubenswrapper[2558]: I0710 18:04:50.334714 2558 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"ca-trust-extracted\" (UniqueName: \"kubernetes.io/empty-dir/4b133a69-01d6-42b5-b12f-d063856c87cb-ca-trust-extracted\") pod \"4b133a69-01d6-42b5-b12f-d063856c87cb\" (UID: \"4b133a69-01d6-42b5-b12f-d063856c87cb\") " Jul 10 18:04:50.334876 ip-10-0-128-39 kubenswrapper[2558]: I0710 18:04:50.334741 2558 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"bound-sa-token\" (UniqueName: \"kubernetes.io/projected/4b133a69-01d6-42b5-b12f-d063856c87cb-bound-sa-token\") pod \"4b133a69-01d6-42b5-b12f-d063856c87cb\" (UID: \"4b133a69-01d6-42b5-b12f-d063856c87cb\") " Jul 10 18:04:50.334876 ip-10-0-128-39 kubenswrapper[2558]: I0710 18:04:50.334757 2558 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-9kb59\" (UniqueName: \"kubernetes.io/projected/4b133a69-01d6-42b5-b12f-d063856c87cb-kube-api-access-9kb59\") pod \"4b133a69-01d6-42b5-b12f-d063856c87cb\" (UID: \"4b133a69-01d6-42b5-b12f-d063856c87cb\") " Jul 10 18:04:50.334876 ip-10-0-128-39 kubenswrapper[2558]: I0710 18:04:50.334774 2558 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"installation-pull-secrets\" (UniqueName: \"kubernetes.io/secret/4b133a69-01d6-42b5-b12f-d063856c87cb-installation-pull-secrets\") pod \"4b133a69-01d6-42b5-b12f-d063856c87cb\" (UID: \"4b133a69-01d6-42b5-b12f-d063856c87cb\") " Jul 10 18:04:50.334876 ip-10-0-128-39 kubenswrapper[2558]: I0710 18:04:50.334804 2558 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"registry-certificates\" (UniqueName: \"kubernetes.io/configmap/4b133a69-01d6-42b5-b12f-d063856c87cb-registry-certificates\") pod \"4b133a69-01d6-42b5-b12f-d063856c87cb\" (UID: \"4b133a69-01d6-42b5-b12f-d063856c87cb\") " Jul 10 18:04:50.335533 ip-10-0-128-39 kubenswrapper[2558]: I0710 18:04:50.335376 2558 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/4b133a69-01d6-42b5-b12f-d063856c87cb-registry-certificates" (OuterVolumeSpecName: "registry-certificates") pod "4b133a69-01d6-42b5-b12f-d063856c87cb" (UID: "4b133a69-01d6-42b5-b12f-d063856c87cb"). InnerVolumeSpecName "registry-certificates". PluginName "kubernetes.io/configmap", VolumeGIDValue "" Jul 10 18:04:50.335687 ip-10-0-128-39 kubenswrapper[2558]: I0710 18:04:50.335646 2558 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/4b133a69-01d6-42b5-b12f-d063856c87cb-trusted-ca" (OuterVolumeSpecName: "trusted-ca") pod "4b133a69-01d6-42b5-b12f-d063856c87cb" (UID: "4b133a69-01d6-42b5-b12f-d063856c87cb"). InnerVolumeSpecName "trusted-ca". PluginName "kubernetes.io/configmap", VolumeGIDValue "" Jul 10 18:04:50.337387 ip-10-0-128-39 kubenswrapper[2558]: I0710 18:04:50.337356 2558 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/4b133a69-01d6-42b5-b12f-d063856c87cb-installation-pull-secrets" (OuterVolumeSpecName: "installation-pull-secrets") pod "4b133a69-01d6-42b5-b12f-d063856c87cb" (UID: "4b133a69-01d6-42b5-b12f-d063856c87cb"). InnerVolumeSpecName "installation-pull-secrets". PluginName "kubernetes.io/secret", VolumeGIDValue "" Jul 10 18:04:50.337544 ip-10-0-128-39 kubenswrapper[2558]: I0710 18:04:50.337523 2558 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/4b133a69-01d6-42b5-b12f-d063856c87cb-image-registry-private-configuration" (OuterVolumeSpecName: "image-registry-private-configuration") pod "4b133a69-01d6-42b5-b12f-d063856c87cb" (UID: "4b133a69-01d6-42b5-b12f-d063856c87cb"). InnerVolumeSpecName "image-registry-private-configuration". PluginName "kubernetes.io/secret", VolumeGIDValue "" Jul 10 18:04:50.337610 ip-10-0-128-39 kubenswrapper[2558]: I0710 18:04:50.337581 2558 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/4b133a69-01d6-42b5-b12f-d063856c87cb-kube-api-access-9kb59" (OuterVolumeSpecName: "kube-api-access-9kb59") pod "4b133a69-01d6-42b5-b12f-d063856c87cb" (UID: "4b133a69-01d6-42b5-b12f-d063856c87cb"). InnerVolumeSpecName "kube-api-access-9kb59". PluginName "kubernetes.io/projected", VolumeGIDValue "" Jul 10 18:04:50.337659 ip-10-0-128-39 kubenswrapper[2558]: I0710 18:04:50.337619 2558 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/4b133a69-01d6-42b5-b12f-d063856c87cb-bound-sa-token" (OuterVolumeSpecName: "bound-sa-token") pod "4b133a69-01d6-42b5-b12f-d063856c87cb" (UID: "4b133a69-01d6-42b5-b12f-d063856c87cb"). InnerVolumeSpecName "bound-sa-token". PluginName "kubernetes.io/projected", VolumeGIDValue "" Jul 10 18:04:50.337716 ip-10-0-128-39 kubenswrapper[2558]: I0710 18:04:50.337688 2558 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/4b133a69-01d6-42b5-b12f-d063856c87cb-registry-tls" (OuterVolumeSpecName: "registry-tls") pod "4b133a69-01d6-42b5-b12f-d063856c87cb" (UID: "4b133a69-01d6-42b5-b12f-d063856c87cb"). InnerVolumeSpecName "registry-tls". PluginName "kubernetes.io/projected", VolumeGIDValue "" Jul 10 18:04:50.343302 ip-10-0-128-39 kubenswrapper[2558]: I0710 18:04:50.343281 2558 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/4b133a69-01d6-42b5-b12f-d063856c87cb-ca-trust-extracted" (OuterVolumeSpecName: "ca-trust-extracted") pod "4b133a69-01d6-42b5-b12f-d063856c87cb" (UID: "4b133a69-01d6-42b5-b12f-d063856c87cb"). InnerVolumeSpecName "ca-trust-extracted". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Jul 10 18:04:50.435443 ip-10-0-128-39 kubenswrapper[2558]: I0710 18:04:50.435398 2558 reconciler_common.go:299] "Volume detached for volume \"ca-trust-extracted\" (UniqueName: \"kubernetes.io/empty-dir/4b133a69-01d6-42b5-b12f-d063856c87cb-ca-trust-extracted\") on node \"ip-10-0-128-39.ec2.internal\" DevicePath \"\"" Jul 10 18:04:50.435443 ip-10-0-128-39 kubenswrapper[2558]: I0710 18:04:50.435434 2558 reconciler_common.go:299] "Volume detached for volume \"bound-sa-token\" (UniqueName: \"kubernetes.io/projected/4b133a69-01d6-42b5-b12f-d063856c87cb-bound-sa-token\") on node \"ip-10-0-128-39.ec2.internal\" DevicePath \"\"" Jul 10 18:04:50.435443 ip-10-0-128-39 kubenswrapper[2558]: I0710 18:04:50.435448 2558 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-9kb59\" (UniqueName: \"kubernetes.io/projected/4b133a69-01d6-42b5-b12f-d063856c87cb-kube-api-access-9kb59\") on node \"ip-10-0-128-39.ec2.internal\" DevicePath \"\"" Jul 10 18:04:50.435677 ip-10-0-128-39 kubenswrapper[2558]: I0710 18:04:50.435460 2558 reconciler_common.go:299] "Volume detached for volume \"installation-pull-secrets\" (UniqueName: \"kubernetes.io/secret/4b133a69-01d6-42b5-b12f-d063856c87cb-installation-pull-secrets\") on node \"ip-10-0-128-39.ec2.internal\" DevicePath \"\"" Jul 10 18:04:50.435677 ip-10-0-128-39 kubenswrapper[2558]: I0710 18:04:50.435472 2558 reconciler_common.go:299] "Volume detached for volume \"registry-certificates\" (UniqueName: \"kubernetes.io/configmap/4b133a69-01d6-42b5-b12f-d063856c87cb-registry-certificates\") on node \"ip-10-0-128-39.ec2.internal\" DevicePath \"\"" Jul 10 18:04:50.435677 ip-10-0-128-39 kubenswrapper[2558]: I0710 18:04:50.435484 2558 reconciler_common.go:299] "Volume detached for volume \"image-registry-private-configuration\" (UniqueName: \"kubernetes.io/secret/4b133a69-01d6-42b5-b12f-d063856c87cb-image-registry-private-configuration\") on node \"ip-10-0-128-39.ec2.internal\" DevicePath \"\"" Jul 10 18:04:50.435677 ip-10-0-128-39 kubenswrapper[2558]: I0710 18:04:50.435497 2558 reconciler_common.go:299] "Volume detached for volume \"registry-tls\" (UniqueName: \"kubernetes.io/projected/4b133a69-01d6-42b5-b12f-d063856c87cb-registry-tls\") on node \"ip-10-0-128-39.ec2.internal\" DevicePath \"\"" Jul 10 18:04:50.435677 ip-10-0-128-39 kubenswrapper[2558]: I0710 18:04:50.435508 2558 reconciler_common.go:299] "Volume detached for volume \"trusted-ca\" (UniqueName: \"kubernetes.io/configmap/4b133a69-01d6-42b5-b12f-d063856c87cb-trusted-ca\") on node \"ip-10-0-128-39.ec2.internal\" DevicePath \"\"" Jul 10 18:04:50.760292 ip-10-0-128-39 kubenswrapper[2558]: I0710 18:04:50.760256 2558 generic.go:356] "Generic (PLEG): container finished" podID="4b133a69-01d6-42b5-b12f-d063856c87cb" containerID="3eadfc79825d6da28ca7f0788e774cc4c167cf06e2a2d9b85dd22e439b1c49ec" exitCode=0 Jul 10 18:04:50.760486 ip-10-0-128-39 kubenswrapper[2558]: I0710 18:04:50.760303 2558 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-image-registry/image-registry-cf9777cd6-d9zpc" Jul 10 18:04:50.760486 ip-10-0-128-39 kubenswrapper[2558]: I0710 18:04:50.760347 2558 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-image-registry/image-registry-cf9777cd6-d9zpc" event={"ID":"4b133a69-01d6-42b5-b12f-d063856c87cb","Type":"ContainerDied","Data":"3eadfc79825d6da28ca7f0788e774cc4c167cf06e2a2d9b85dd22e439b1c49ec"} Jul 10 18:04:50.760486 ip-10-0-128-39 kubenswrapper[2558]: I0710 18:04:50.760418 2558 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-image-registry/image-registry-cf9777cd6-d9zpc" event={"ID":"4b133a69-01d6-42b5-b12f-d063856c87cb","Type":"ContainerDied","Data":"772572baec4bfb2b4d2a54152143108d673793a6ad178a51ed113d60dcf31380"} Jul 10 18:04:50.760486 ip-10-0-128-39 kubenswrapper[2558]: I0710 18:04:50.760447 2558 scope.go:117] "RemoveContainer" containerID="3eadfc79825d6da28ca7f0788e774cc4c167cf06e2a2d9b85dd22e439b1c49ec" Jul 10 18:04:50.771743 ip-10-0-128-39 kubenswrapper[2558]: I0710 18:04:50.771720 2558 scope.go:117] "RemoveContainer" containerID="3eadfc79825d6da28ca7f0788e774cc4c167cf06e2a2d9b85dd22e439b1c49ec" Jul 10 18:04:50.772127 ip-10-0-128-39 kubenswrapper[2558]: E0710 18:04:50.772097 2558 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"3eadfc79825d6da28ca7f0788e774cc4c167cf06e2a2d9b85dd22e439b1c49ec\": container with ID starting with 3eadfc79825d6da28ca7f0788e774cc4c167cf06e2a2d9b85dd22e439b1c49ec not found: ID does not exist" containerID="3eadfc79825d6da28ca7f0788e774cc4c167cf06e2a2d9b85dd22e439b1c49ec" Jul 10 18:04:50.772219 ip-10-0-128-39 kubenswrapper[2558]: I0710 18:04:50.772136 2558 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"3eadfc79825d6da28ca7f0788e774cc4c167cf06e2a2d9b85dd22e439b1c49ec"} err="failed to get container status \"3eadfc79825d6da28ca7f0788e774cc4c167cf06e2a2d9b85dd22e439b1c49ec\": rpc error: code = NotFound desc = could not find container \"3eadfc79825d6da28ca7f0788e774cc4c167cf06e2a2d9b85dd22e439b1c49ec\": container with ID starting with 3eadfc79825d6da28ca7f0788e774cc4c167cf06e2a2d9b85dd22e439b1c49ec not found: ID does not exist" Jul 10 18:04:50.783723 ip-10-0-128-39 kubenswrapper[2558]: I0710 18:04:50.783699 2558 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-image-registry/image-registry-cf9777cd6-d9zpc"] Jul 10 18:04:50.787253 ip-10-0-128-39 kubenswrapper[2558]: I0710 18:04:50.787232 2558 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["openshift-image-registry/image-registry-cf9777cd6-d9zpc"] Jul 10 18:04:52.100984 ip-10-0-128-39 kubenswrapper[2558]: I0710 18:04:52.100952 2558 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="4b133a69-01d6-42b5-b12f-d063856c87cb" path="/var/lib/kubelet/pods/4b133a69-01d6-42b5-b12f-d063856c87cb/volumes" Jul 10 18:05:02.816231 ip-10-0-128-39 kubenswrapper[2558]: I0710 18:05:02.816190 2558 generic.go:356] "Generic (PLEG): container finished" podID="eec35344-a5cb-4a6e-9e2e-5adca5a0fa1a" containerID="a3c44a99161d39c31cabed3cb7cc5b67842e3079249e858f1388e6918e7ce078" exitCode=0 Jul 10 18:05:02.816735 ip-10-0-128-39 kubenswrapper[2558]: I0710 18:05:02.816273 2558 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-kube-storage-version-migrator-operator/kube-storage-version-migrator-operator-799cb7f958-vggpz" event={"ID":"eec35344-a5cb-4a6e-9e2e-5adca5a0fa1a","Type":"ContainerDied","Data":"a3c44a99161d39c31cabed3cb7cc5b67842e3079249e858f1388e6918e7ce078"} Jul 10 18:05:02.816735 ip-10-0-128-39 kubenswrapper[2558]: I0710 18:05:02.816704 2558 scope.go:117] "RemoveContainer" containerID="a3c44a99161d39c31cabed3cb7cc5b67842e3079249e858f1388e6918e7ce078" Jul 10 18:05:03.822487 ip-10-0-128-39 kubenswrapper[2558]: I0710 18:05:03.822449 2558 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-kube-storage-version-migrator-operator/kube-storage-version-migrator-operator-799cb7f958-vggpz" event={"ID":"eec35344-a5cb-4a6e-9e2e-5adca5a0fa1a","Type":"ContainerStarted","Data":"67aa2b058b2131bb3932c0d2832dab3ea548ac253a4f031a82f5936dd254c331"} Jul 10 18:05:07.839151 ip-10-0-128-39 kubenswrapper[2558]: I0710 18:05:07.839118 2558 generic.go:356] "Generic (PLEG): container finished" podID="893dc3d5-4a12-4832-9a96-898388c12ae4" containerID="3e176c50cd8670cbb5c7023435f89cc28f2b4ce1b5249daeba7a0b06b67d76c3" exitCode=0 Jul 10 18:05:07.839639 ip-10-0-128-39 kubenswrapper[2558]: I0710 18:05:07.839193 2558 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-service-ca-operator/service-ca-operator-7695fc6c8c-hfzjm" event={"ID":"893dc3d5-4a12-4832-9a96-898388c12ae4","Type":"ContainerDied","Data":"3e176c50cd8670cbb5c7023435f89cc28f2b4ce1b5249daeba7a0b06b67d76c3"} Jul 10 18:05:07.839639 ip-10-0-128-39 kubenswrapper[2558]: I0710 18:05:07.839562 2558 scope.go:117] "RemoveContainer" containerID="3e176c50cd8670cbb5c7023435f89cc28f2b4ce1b5249daeba7a0b06b67d76c3" Jul 10 18:05:08.844480 ip-10-0-128-39 kubenswrapper[2558]: I0710 18:05:08.844445 2558 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-service-ca-operator/service-ca-operator-7695fc6c8c-hfzjm" event={"ID":"893dc3d5-4a12-4832-9a96-898388c12ae4","Type":"ContainerStarted","Data":"e663bbf7eba8c300a2219dfba7fd060780c417e7beeb209343bec308d8a2feca"} Jul 10 18:07:51.940096 ip-10-0-128-39 kubenswrapper[2558]: I0710 18:07:51.940068 2558 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-console-operator_console-operator-7866497494-26542_2b29fb39-89b3-42c3-b334-3a195242248c/console-operator/1.log" Jul 10 18:07:51.944777 ip-10-0-128-39 kubenswrapper[2558]: I0710 18:07:51.944752 2558 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ovn-kubernetes_ovnkube-node-ll44q_eabc6c69-b1e8-40d5-b41f-f49778a5248c/ovn-acl-logging/0.log" Jul 10 18:07:51.946484 ip-10-0-128-39 kubenswrapper[2558]: I0710 18:07:51.946461 2558 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-console-operator_console-operator-7866497494-26542_2b29fb39-89b3-42c3-b334-3a195242248c/console-operator/1.log" Jul 10 18:07:51.951902 ip-10-0-128-39 kubenswrapper[2558]: I0710 18:07:51.951873 2558 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ovn-kubernetes_ovnkube-node-ll44q_eabc6c69-b1e8-40d5-b41f-f49778a5248c/ovn-acl-logging/0.log" Jul 10 18:07:51.953511 ip-10-0-128-39 kubenswrapper[2558]: I0710 18:07:51.953491 2558 kubelet.go:1648] "Image garbage collection succeeded" Jul 10 18:08:06.029068 ip-10-0-128-39 kubenswrapper[2558]: I0710 18:08:06.029032 2558 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["opendatahub/opendatahub-operator-controller-manager-795b486df7-vfgpp"] Jul 10 18:08:06.030642 ip-10-0-128-39 kubenswrapper[2558]: I0710 18:08:06.029472 2558 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="4b133a69-01d6-42b5-b12f-d063856c87cb" containerName="registry" Jul 10 18:08:06.030642 ip-10-0-128-39 kubenswrapper[2558]: I0710 18:08:06.029488 2558 state_mem.go:107] "Deleted CPUSet assignment" podUID="4b133a69-01d6-42b5-b12f-d063856c87cb" containerName="registry" Jul 10 18:08:06.030642 ip-10-0-128-39 kubenswrapper[2558]: I0710 18:08:06.029563 2558 memory_manager.go:368] "RemoveStaleState removing state" podUID="4b133a69-01d6-42b5-b12f-d063856c87cb" containerName="registry" Jul 10 18:08:06.031732 ip-10-0-128-39 kubenswrapper[2558]: I0710 18:08:06.031712 2558 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="opendatahub/opendatahub-operator-controller-manager-795b486df7-vfgpp" Jul 10 18:08:06.034473 ip-10-0-128-39 kubenswrapper[2558]: I0710 18:08:06.034456 2558 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"opendatahub\"/\"opendatahub-operator-controller-manager-dockercfg-gvddb\"" Jul 10 18:08:06.035302 ip-10-0-128-39 kubenswrapper[2558]: I0710 18:08:06.035269 2558 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"opendatahub\"/\"openshift-service-ca.crt\"" Jul 10 18:08:06.035417 ip-10-0-128-39 kubenswrapper[2558]: I0710 18:08:06.035339 2558 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"opendatahub\"/\"opendatahub-operator-controller-manager-service-cert\"" Jul 10 18:08:06.035417 ip-10-0-128-39 kubenswrapper[2558]: I0710 18:08:06.035280 2558 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"opendatahub\"/\"opendatahub-operator-controller-webhook-cert\"" Jul 10 18:08:06.038362 ip-10-0-128-39 kubenswrapper[2558]: I0710 18:08:06.038019 2558 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"opendatahub\"/\"kube-root-ca.crt\"" Jul 10 18:08:06.053472 ip-10-0-128-39 kubenswrapper[2558]: I0710 18:08:06.053443 2558 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["opendatahub/opendatahub-operator-controller-manager-795b486df7-vfgpp"] Jul 10 18:08:06.106106 ip-10-0-128-39 kubenswrapper[2558]: I0710 18:08:06.106068 2558 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tmp\" (UniqueName: \"kubernetes.io/empty-dir/5a66be4d-6d90-47b6-a1c2-30a7734df1e8-tmp\") pod \"opendatahub-operator-controller-manager-795b486df7-vfgpp\" (UID: \"5a66be4d-6d90-47b6-a1c2-30a7734df1e8\") " pod="opendatahub/opendatahub-operator-controller-manager-795b486df7-vfgpp" Jul 10 18:08:06.106262 ip-10-0-128-39 kubenswrapper[2558]: I0710 18:08:06.106115 2558 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"manifests\" (UniqueName: \"kubernetes.io/empty-dir/5a66be4d-6d90-47b6-a1c2-30a7734df1e8-manifests\") pod \"opendatahub-operator-controller-manager-795b486df7-vfgpp\" (UID: \"5a66be4d-6d90-47b6-a1c2-30a7734df1e8\") " pod="opendatahub/opendatahub-operator-controller-manager-795b486df7-vfgpp" Jul 10 18:08:06.106262 ip-10-0-128-39 kubenswrapper[2558]: I0710 18:08:06.106133 2558 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"webhook-cert\" (UniqueName: \"kubernetes.io/secret/5a66be4d-6d90-47b6-a1c2-30a7734df1e8-webhook-cert\") pod \"opendatahub-operator-controller-manager-795b486df7-vfgpp\" (UID: \"5a66be4d-6d90-47b6-a1c2-30a7734df1e8\") " pod="opendatahub/opendatahub-operator-controller-manager-795b486df7-vfgpp" Jul 10 18:08:06.106262 ip-10-0-128-39 kubenswrapper[2558]: I0710 18:08:06.106200 2558 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-4c4sd\" (UniqueName: \"kubernetes.io/projected/5a66be4d-6d90-47b6-a1c2-30a7734df1e8-kube-api-access-4c4sd\") pod \"opendatahub-operator-controller-manager-795b486df7-vfgpp\" (UID: \"5a66be4d-6d90-47b6-a1c2-30a7734df1e8\") " pod="opendatahub/opendatahub-operator-controller-manager-795b486df7-vfgpp" Jul 10 18:08:06.106262 ip-10-0-128-39 kubenswrapper[2558]: I0710 18:08:06.106255 2558 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"apiservice-cert\" (UniqueName: \"kubernetes.io/secret/5a66be4d-6d90-47b6-a1c2-30a7734df1e8-apiservice-cert\") pod \"opendatahub-operator-controller-manager-795b486df7-vfgpp\" (UID: \"5a66be4d-6d90-47b6-a1c2-30a7734df1e8\") " pod="opendatahub/opendatahub-operator-controller-manager-795b486df7-vfgpp" Jul 10 18:08:06.207576 ip-10-0-128-39 kubenswrapper[2558]: I0710 18:08:06.207533 2558 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"manifests\" (UniqueName: \"kubernetes.io/empty-dir/5a66be4d-6d90-47b6-a1c2-30a7734df1e8-manifests\") pod \"opendatahub-operator-controller-manager-795b486df7-vfgpp\" (UID: \"5a66be4d-6d90-47b6-a1c2-30a7734df1e8\") " pod="opendatahub/opendatahub-operator-controller-manager-795b486df7-vfgpp" Jul 10 18:08:06.207576 ip-10-0-128-39 kubenswrapper[2558]: I0710 18:08:06.207573 2558 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"webhook-cert\" (UniqueName: \"kubernetes.io/secret/5a66be4d-6d90-47b6-a1c2-30a7734df1e8-webhook-cert\") pod \"opendatahub-operator-controller-manager-795b486df7-vfgpp\" (UID: \"5a66be4d-6d90-47b6-a1c2-30a7734df1e8\") " pod="opendatahub/opendatahub-operator-controller-manager-795b486df7-vfgpp" Jul 10 18:08:06.207844 ip-10-0-128-39 kubenswrapper[2558]: I0710 18:08:06.207616 2558 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-4c4sd\" (UniqueName: \"kubernetes.io/projected/5a66be4d-6d90-47b6-a1c2-30a7734df1e8-kube-api-access-4c4sd\") pod \"opendatahub-operator-controller-manager-795b486df7-vfgpp\" (UID: \"5a66be4d-6d90-47b6-a1c2-30a7734df1e8\") " pod="opendatahub/opendatahub-operator-controller-manager-795b486df7-vfgpp" Jul 10 18:08:06.207844 ip-10-0-128-39 kubenswrapper[2558]: I0710 18:08:06.207663 2558 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"apiservice-cert\" (UniqueName: \"kubernetes.io/secret/5a66be4d-6d90-47b6-a1c2-30a7734df1e8-apiservice-cert\") pod \"opendatahub-operator-controller-manager-795b486df7-vfgpp\" (UID: \"5a66be4d-6d90-47b6-a1c2-30a7734df1e8\") " pod="opendatahub/opendatahub-operator-controller-manager-795b486df7-vfgpp" Jul 10 18:08:06.207844 ip-10-0-128-39 kubenswrapper[2558]: I0710 18:08:06.207708 2558 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tmp\" (UniqueName: \"kubernetes.io/empty-dir/5a66be4d-6d90-47b6-a1c2-30a7734df1e8-tmp\") pod \"opendatahub-operator-controller-manager-795b486df7-vfgpp\" (UID: \"5a66be4d-6d90-47b6-a1c2-30a7734df1e8\") " pod="opendatahub/opendatahub-operator-controller-manager-795b486df7-vfgpp" Jul 10 18:08:06.207989 ip-10-0-128-39 kubenswrapper[2558]: I0710 18:08:06.207971 2558 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"manifests\" (UniqueName: \"kubernetes.io/empty-dir/5a66be4d-6d90-47b6-a1c2-30a7734df1e8-manifests\") pod \"opendatahub-operator-controller-manager-795b486df7-vfgpp\" (UID: \"5a66be4d-6d90-47b6-a1c2-30a7734df1e8\") " pod="opendatahub/opendatahub-operator-controller-manager-795b486df7-vfgpp" Jul 10 18:08:06.209967 ip-10-0-128-39 kubenswrapper[2558]: I0710 18:08:06.209942 2558 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"tmp\" (UniqueName: \"kubernetes.io/empty-dir/5a66be4d-6d90-47b6-a1c2-30a7734df1e8-tmp\") pod \"opendatahub-operator-controller-manager-795b486df7-vfgpp\" (UID: \"5a66be4d-6d90-47b6-a1c2-30a7734df1e8\") " pod="opendatahub/opendatahub-operator-controller-manager-795b486df7-vfgpp" Jul 10 18:08:06.210295 ip-10-0-128-39 kubenswrapper[2558]: I0710 18:08:06.210273 2558 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"webhook-cert\" (UniqueName: \"kubernetes.io/secret/5a66be4d-6d90-47b6-a1c2-30a7734df1e8-webhook-cert\") pod \"opendatahub-operator-controller-manager-795b486df7-vfgpp\" (UID: \"5a66be4d-6d90-47b6-a1c2-30a7734df1e8\") " pod="opendatahub/opendatahub-operator-controller-manager-795b486df7-vfgpp" Jul 10 18:08:06.210367 ip-10-0-128-39 kubenswrapper[2558]: I0710 18:08:06.210290 2558 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"apiservice-cert\" (UniqueName: \"kubernetes.io/secret/5a66be4d-6d90-47b6-a1c2-30a7734df1e8-apiservice-cert\") pod \"opendatahub-operator-controller-manager-795b486df7-vfgpp\" (UID: \"5a66be4d-6d90-47b6-a1c2-30a7734df1e8\") " pod="opendatahub/opendatahub-operator-controller-manager-795b486df7-vfgpp" Jul 10 18:08:06.221343 ip-10-0-128-39 kubenswrapper[2558]: I0710 18:08:06.221323 2558 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-4c4sd\" (UniqueName: \"kubernetes.io/projected/5a66be4d-6d90-47b6-a1c2-30a7734df1e8-kube-api-access-4c4sd\") pod \"opendatahub-operator-controller-manager-795b486df7-vfgpp\" (UID: \"5a66be4d-6d90-47b6-a1c2-30a7734df1e8\") " pod="opendatahub/opendatahub-operator-controller-manager-795b486df7-vfgpp" Jul 10 18:08:06.341250 ip-10-0-128-39 kubenswrapper[2558]: I0710 18:08:06.341166 2558 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="opendatahub/opendatahub-operator-controller-manager-795b486df7-vfgpp" Jul 10 18:08:06.488442 ip-10-0-128-39 kubenswrapper[2558]: I0710 18:08:06.488405 2558 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["opendatahub/opendatahub-operator-controller-manager-795b486df7-vfgpp"] Jul 10 18:08:06.491719 ip-10-0-128-39 kubenswrapper[2558]: W0710 18:08:06.491691 2558 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod5a66be4d_6d90_47b6_a1c2_30a7734df1e8.slice/crio-2f663aadbb9735cb1dd38f1ee2a370b4c25d014677e255ce102094a484a69179 WatchSource:0}: Error finding container 2f663aadbb9735cb1dd38f1ee2a370b4c25d014677e255ce102094a484a69179: Status 404 returned error can't find the container with id 2f663aadbb9735cb1dd38f1ee2a370b4c25d014677e255ce102094a484a69179 Jul 10 18:08:06.493441 ip-10-0-128-39 kubenswrapper[2558]: I0710 18:08:06.493426 2558 provider.go:93] Refreshing cache for provider: *credentialprovider.defaultDockerConfigProvider Jul 10 18:08:06.529485 ip-10-0-128-39 kubenswrapper[2558]: I0710 18:08:06.529454 2558 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="opendatahub/opendatahub-operator-controller-manager-795b486df7-vfgpp" event={"ID":"5a66be4d-6d90-47b6-a1c2-30a7734df1e8","Type":"ContainerStarted","Data":"2f663aadbb9735cb1dd38f1ee2a370b4c25d014677e255ce102094a484a69179"} Jul 10 18:08:10.548274 ip-10-0-128-39 kubenswrapper[2558]: I0710 18:08:10.548242 2558 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="opendatahub/opendatahub-operator-controller-manager-795b486df7-vfgpp" event={"ID":"5a66be4d-6d90-47b6-a1c2-30a7734df1e8","Type":"ContainerStarted","Data":"17ff5e01f4a0cc4040bb5e28a7988b560645909c51abd733419d9266f51950bf"} Jul 10 18:08:11.554342 ip-10-0-128-39 kubenswrapper[2558]: I0710 18:08:11.554311 2558 generic.go:356] "Generic (PLEG): container finished" podID="5a66be4d-6d90-47b6-a1c2-30a7734df1e8" containerID="17ff5e01f4a0cc4040bb5e28a7988b560645909c51abd733419d9266f51950bf" exitCode=0 Jul 10 18:08:11.554738 ip-10-0-128-39 kubenswrapper[2558]: I0710 18:08:11.554394 2558 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="opendatahub/opendatahub-operator-controller-manager-795b486df7-vfgpp" event={"ID":"5a66be4d-6d90-47b6-a1c2-30a7734df1e8","Type":"ContainerDied","Data":"17ff5e01f4a0cc4040bb5e28a7988b560645909c51abd733419d9266f51950bf"} Jul 10 18:08:12.560769 ip-10-0-128-39 kubenswrapper[2558]: I0710 18:08:12.560734 2558 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="opendatahub/opendatahub-operator-controller-manager-795b486df7-vfgpp" event={"ID":"5a66be4d-6d90-47b6-a1c2-30a7734df1e8","Type":"ContainerStarted","Data":"f61aaf7e854d2a0c729ad8f9135f165ead8797f1eb26c68a72600dd461f235a5"} Jul 10 18:08:12.561257 ip-10-0-128-39 kubenswrapper[2558]: I0710 18:08:12.560842 2558 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="opendatahub/opendatahub-operator-controller-manager-795b486df7-vfgpp" Jul 10 18:08:12.595160 ip-10-0-128-39 kubenswrapper[2558]: I0710 18:08:12.595110 2558 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="opendatahub/opendatahub-operator-controller-manager-795b486df7-vfgpp" podStartSLOduration=2.94835045 podStartE2EDuration="7.595095051s" totalImagesPullingTime="3.552361016s" totalInitContainerRuntime="1.094383585s" isStatefulPod=false podCreationTimestamp="2026-07-10 18:08:05 +0000 UTC" imagePullSessionsCount=2 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-10 18:08:12.594036316 +0000 UTC m=+321.097528821" watchObservedRunningTime="2026-07-10 18:08:12.595095051 +0000 UTC m=+321.098587556" Jul 10 18:08:23.568215 ip-10-0-128-39 kubenswrapper[2558]: I0710 18:08:23.568187 2558 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="opendatahub/opendatahub-operator-controller-manager-795b486df7-vfgpp" Jul 10 18:08:37.046654 ip-10-0-128-39 kubenswrapper[2558]: I0710 18:08:37.046614 2558 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-lws-operator/lws-controller-manager-59dbbcc5d9-4l2s9"] Jul 10 18:08:37.056610 ip-10-0-128-39 kubenswrapper[2558]: I0710 18:08:37.056589 2558 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-lws-operator/lws-controller-manager-59dbbcc5d9-4l2s9" Jul 10 18:08:37.060540 ip-10-0-128-39 kubenswrapper[2558]: I0710 18:08:37.060509 2558 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-lws-operator\"/\"kube-root-ca.crt\"" Jul 10 18:08:37.060540 ip-10-0-128-39 kubenswrapper[2558]: I0710 18:08:37.060511 2558 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-lws-operator\"/\"webhook-server-cert\"" Jul 10 18:08:37.060694 ip-10-0-128-39 kubenswrapper[2558]: I0710 18:08:37.060511 2558 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-lws-operator\"/\"lws-controller-manager-dockercfg-dgtkd\"" Jul 10 18:08:37.060694 ip-10-0-128-39 kubenswrapper[2558]: I0710 18:08:37.060625 2558 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-lws-operator\"/\"lws-manager-config\"" Jul 10 18:08:37.060694 ip-10-0-128-39 kubenswrapper[2558]: I0710 18:08:37.060640 2558 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-lws-operator\"/\"metrics-server-cert\"" Jul 10 18:08:37.060892 ip-10-0-128-39 kubenswrapper[2558]: I0710 18:08:37.060879 2558 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-lws-operator\"/\"openshift-service-ca.crt\"" Jul 10 18:08:37.066208 ip-10-0-128-39 kubenswrapper[2558]: I0710 18:08:37.066183 2558 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-lws-operator/lws-controller-manager-59dbbcc5d9-4l2s9"] Jul 10 18:08:37.171884 ip-10-0-128-39 kubenswrapper[2558]: I0710 18:08:37.171847 2558 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"cert\" (UniqueName: \"kubernetes.io/secret/48e923db-401e-40a8-acc6-7fb572b98f7f-cert\") pod \"lws-controller-manager-59dbbcc5d9-4l2s9\" (UID: \"48e923db-401e-40a8-acc6-7fb572b98f7f\") " pod="openshift-lws-operator/lws-controller-manager-59dbbcc5d9-4l2s9" Jul 10 18:08:37.171884 ip-10-0-128-39 kubenswrapper[2558]: I0710 18:08:37.171896 2558 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-tjrg7\" (UniqueName: \"kubernetes.io/projected/48e923db-401e-40a8-acc6-7fb572b98f7f-kube-api-access-tjrg7\") pod \"lws-controller-manager-59dbbcc5d9-4l2s9\" (UID: \"48e923db-401e-40a8-acc6-7fb572b98f7f\") " pod="openshift-lws-operator/lws-controller-manager-59dbbcc5d9-4l2s9" Jul 10 18:08:37.172122 ip-10-0-128-39 kubenswrapper[2558]: I0710 18:08:37.171998 2558 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"metrics-cert\" (UniqueName: \"kubernetes.io/secret/48e923db-401e-40a8-acc6-7fb572b98f7f-metrics-cert\") pod \"lws-controller-manager-59dbbcc5d9-4l2s9\" (UID: \"48e923db-401e-40a8-acc6-7fb572b98f7f\") " pod="openshift-lws-operator/lws-controller-manager-59dbbcc5d9-4l2s9" Jul 10 18:08:37.172122 ip-10-0-128-39 kubenswrapper[2558]: I0710 18:08:37.172037 2558 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"manager-config\" (UniqueName: \"kubernetes.io/configmap/48e923db-401e-40a8-acc6-7fb572b98f7f-manager-config\") pod \"lws-controller-manager-59dbbcc5d9-4l2s9\" (UID: \"48e923db-401e-40a8-acc6-7fb572b98f7f\") " pod="openshift-lws-operator/lws-controller-manager-59dbbcc5d9-4l2s9" Jul 10 18:08:37.272685 ip-10-0-128-39 kubenswrapper[2558]: I0710 18:08:37.272650 2558 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-cert\" (UniqueName: \"kubernetes.io/secret/48e923db-401e-40a8-acc6-7fb572b98f7f-metrics-cert\") pod \"lws-controller-manager-59dbbcc5d9-4l2s9\" (UID: \"48e923db-401e-40a8-acc6-7fb572b98f7f\") " pod="openshift-lws-operator/lws-controller-manager-59dbbcc5d9-4l2s9" Jul 10 18:08:37.272909 ip-10-0-128-39 kubenswrapper[2558]: I0710 18:08:37.272704 2558 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"manager-config\" (UniqueName: \"kubernetes.io/configmap/48e923db-401e-40a8-acc6-7fb572b98f7f-manager-config\") pod \"lws-controller-manager-59dbbcc5d9-4l2s9\" (UID: \"48e923db-401e-40a8-acc6-7fb572b98f7f\") " pod="openshift-lws-operator/lws-controller-manager-59dbbcc5d9-4l2s9" Jul 10 18:08:37.272909 ip-10-0-128-39 kubenswrapper[2558]: I0710 18:08:37.272751 2558 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"cert\" (UniqueName: \"kubernetes.io/secret/48e923db-401e-40a8-acc6-7fb572b98f7f-cert\") pod \"lws-controller-manager-59dbbcc5d9-4l2s9\" (UID: \"48e923db-401e-40a8-acc6-7fb572b98f7f\") " pod="openshift-lws-operator/lws-controller-manager-59dbbcc5d9-4l2s9" Jul 10 18:08:37.272909 ip-10-0-128-39 kubenswrapper[2558]: I0710 18:08:37.272779 2558 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-tjrg7\" (UniqueName: \"kubernetes.io/projected/48e923db-401e-40a8-acc6-7fb572b98f7f-kube-api-access-tjrg7\") pod \"lws-controller-manager-59dbbcc5d9-4l2s9\" (UID: \"48e923db-401e-40a8-acc6-7fb572b98f7f\") " pod="openshift-lws-operator/lws-controller-manager-59dbbcc5d9-4l2s9" Jul 10 18:08:37.273483 ip-10-0-128-39 kubenswrapper[2558]: I0710 18:08:37.273460 2558 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"manager-config\" (UniqueName: \"kubernetes.io/configmap/48e923db-401e-40a8-acc6-7fb572b98f7f-manager-config\") pod \"lws-controller-manager-59dbbcc5d9-4l2s9\" (UID: \"48e923db-401e-40a8-acc6-7fb572b98f7f\") " pod="openshift-lws-operator/lws-controller-manager-59dbbcc5d9-4l2s9" Jul 10 18:08:37.275248 ip-10-0-128-39 kubenswrapper[2558]: I0710 18:08:37.275226 2558 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"cert\" (UniqueName: \"kubernetes.io/secret/48e923db-401e-40a8-acc6-7fb572b98f7f-cert\") pod \"lws-controller-manager-59dbbcc5d9-4l2s9\" (UID: \"48e923db-401e-40a8-acc6-7fb572b98f7f\") " pod="openshift-lws-operator/lws-controller-manager-59dbbcc5d9-4l2s9" Jul 10 18:08:37.275363 ip-10-0-128-39 kubenswrapper[2558]: I0710 18:08:37.275339 2558 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"metrics-cert\" (UniqueName: \"kubernetes.io/secret/48e923db-401e-40a8-acc6-7fb572b98f7f-metrics-cert\") pod \"lws-controller-manager-59dbbcc5d9-4l2s9\" (UID: \"48e923db-401e-40a8-acc6-7fb572b98f7f\") " pod="openshift-lws-operator/lws-controller-manager-59dbbcc5d9-4l2s9" Jul 10 18:08:37.288263 ip-10-0-128-39 kubenswrapper[2558]: I0710 18:08:37.288240 2558 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-tjrg7\" (UniqueName: \"kubernetes.io/projected/48e923db-401e-40a8-acc6-7fb572b98f7f-kube-api-access-tjrg7\") pod \"lws-controller-manager-59dbbcc5d9-4l2s9\" (UID: \"48e923db-401e-40a8-acc6-7fb572b98f7f\") " pod="openshift-lws-operator/lws-controller-manager-59dbbcc5d9-4l2s9" Jul 10 18:08:37.366406 ip-10-0-128-39 kubenswrapper[2558]: I0710 18:08:37.366305 2558 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-lws-operator/lws-controller-manager-59dbbcc5d9-4l2s9" Jul 10 18:08:37.499966 ip-10-0-128-39 kubenswrapper[2558]: I0710 18:08:37.499941 2558 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-lws-operator/lws-controller-manager-59dbbcc5d9-4l2s9"] Jul 10 18:08:37.502670 ip-10-0-128-39 kubenswrapper[2558]: W0710 18:08:37.502642 2558 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod48e923db_401e_40a8_acc6_7fb572b98f7f.slice/crio-e1cd5345f4695e030f5df66005cc3cac541e66f5f5a5aa760e00243d2af2519f WatchSource:0}: Error finding container e1cd5345f4695e030f5df66005cc3cac541e66f5f5a5aa760e00243d2af2519f: Status 404 returned error can't find the container with id e1cd5345f4695e030f5df66005cc3cac541e66f5f5a5aa760e00243d2af2519f Jul 10 18:08:37.667069 ip-10-0-128-39 kubenswrapper[2558]: I0710 18:08:37.666981 2558 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-lws-operator/lws-controller-manager-59dbbcc5d9-4l2s9" event={"ID":"48e923db-401e-40a8-acc6-7fb572b98f7f","Type":"ContainerStarted","Data":"e1cd5345f4695e030f5df66005cc3cac541e66f5f5a5aa760e00243d2af2519f"} Jul 10 18:08:38.234175 ip-10-0-128-39 kubenswrapper[2558]: I0710 18:08:38.234139 2558 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["opendatahub/kserve-controller-manager-d6f4b65b8-s4grg"] Jul 10 18:08:38.236756 ip-10-0-128-39 kubenswrapper[2558]: I0710 18:08:38.236619 2558 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="opendatahub/kserve-controller-manager-d6f4b65b8-s4grg" Jul 10 18:08:38.243499 ip-10-0-128-39 kubenswrapper[2558]: I0710 18:08:38.243457 2558 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"opendatahub\"/\"kserve-webhook-server-cert\"" Jul 10 18:08:38.243620 ip-10-0-128-39 kubenswrapper[2558]: I0710 18:08:38.243588 2558 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"opendatahub\"/\"kserve-controller-manager-dockercfg-nhpxl\"" Jul 10 18:08:38.262248 ip-10-0-128-39 kubenswrapper[2558]: I0710 18:08:38.262220 2558 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["opendatahub/kserve-controller-manager-d6f4b65b8-s4grg"] Jul 10 18:08:38.385402 ip-10-0-128-39 kubenswrapper[2558]: I0710 18:08:38.385361 2558 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-6bnpr\" (UniqueName: \"kubernetes.io/projected/e28e2570-6896-422b-bfa9-30402ea86e77-kube-api-access-6bnpr\") pod \"kserve-controller-manager-d6f4b65b8-s4grg\" (UID: \"e28e2570-6896-422b-bfa9-30402ea86e77\") " pod="opendatahub/kserve-controller-manager-d6f4b65b8-s4grg" Jul 10 18:08:38.385568 ip-10-0-128-39 kubenswrapper[2558]: I0710 18:08:38.385419 2558 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"cert\" (UniqueName: \"kubernetes.io/secret/e28e2570-6896-422b-bfa9-30402ea86e77-cert\") pod \"kserve-controller-manager-d6f4b65b8-s4grg\" (UID: \"e28e2570-6896-422b-bfa9-30402ea86e77\") " pod="opendatahub/kserve-controller-manager-d6f4b65b8-s4grg" Jul 10 18:08:38.486941 ip-10-0-128-39 kubenswrapper[2558]: I0710 18:08:38.486856 2558 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-6bnpr\" (UniqueName: \"kubernetes.io/projected/e28e2570-6896-422b-bfa9-30402ea86e77-kube-api-access-6bnpr\") pod \"kserve-controller-manager-d6f4b65b8-s4grg\" (UID: \"e28e2570-6896-422b-bfa9-30402ea86e77\") " pod="opendatahub/kserve-controller-manager-d6f4b65b8-s4grg" Jul 10 18:08:38.486941 ip-10-0-128-39 kubenswrapper[2558]: I0710 18:08:38.486925 2558 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"cert\" (UniqueName: \"kubernetes.io/secret/e28e2570-6896-422b-bfa9-30402ea86e77-cert\") pod \"kserve-controller-manager-d6f4b65b8-s4grg\" (UID: \"e28e2570-6896-422b-bfa9-30402ea86e77\") " pod="opendatahub/kserve-controller-manager-d6f4b65b8-s4grg" Jul 10 18:08:38.487171 ip-10-0-128-39 kubenswrapper[2558]: E0710 18:08:38.487138 2558 secret.go:189] Couldn't get secret opendatahub/kserve-webhook-server-cert: secret "kserve-webhook-server-cert" not found Jul 10 18:08:38.487235 ip-10-0-128-39 kubenswrapper[2558]: E0710 18:08:38.487201 2558 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/e28e2570-6896-422b-bfa9-30402ea86e77-cert podName:e28e2570-6896-422b-bfa9-30402ea86e77 nodeName:}" failed. No retries permitted until 2026-07-10 18:08:38.98717815 +0000 UTC m=+347.490670640 (durationBeforeRetry 500ms). Error: MountVolume.SetUp failed for volume "cert" (UniqueName: "kubernetes.io/secret/e28e2570-6896-422b-bfa9-30402ea86e77-cert") pod "kserve-controller-manager-d6f4b65b8-s4grg" (UID: "e28e2570-6896-422b-bfa9-30402ea86e77") : secret "kserve-webhook-server-cert" not found Jul 10 18:08:38.510163 ip-10-0-128-39 kubenswrapper[2558]: I0710 18:08:38.510126 2558 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-6bnpr\" (UniqueName: \"kubernetes.io/projected/e28e2570-6896-422b-bfa9-30402ea86e77-kube-api-access-6bnpr\") pod \"kserve-controller-manager-d6f4b65b8-s4grg\" (UID: \"e28e2570-6896-422b-bfa9-30402ea86e77\") " pod="opendatahub/kserve-controller-manager-d6f4b65b8-s4grg" Jul 10 18:08:38.991081 ip-10-0-128-39 kubenswrapper[2558]: I0710 18:08:38.991049 2558 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"cert\" (UniqueName: \"kubernetes.io/secret/e28e2570-6896-422b-bfa9-30402ea86e77-cert\") pod \"kserve-controller-manager-d6f4b65b8-s4grg\" (UID: \"e28e2570-6896-422b-bfa9-30402ea86e77\") " pod="opendatahub/kserve-controller-manager-d6f4b65b8-s4grg" Jul 10 18:08:38.993652 ip-10-0-128-39 kubenswrapper[2558]: I0710 18:08:38.993629 2558 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"cert\" (UniqueName: \"kubernetes.io/secret/e28e2570-6896-422b-bfa9-30402ea86e77-cert\") pod \"kserve-controller-manager-d6f4b65b8-s4grg\" (UID: \"e28e2570-6896-422b-bfa9-30402ea86e77\") " pod="opendatahub/kserve-controller-manager-d6f4b65b8-s4grg" Jul 10 18:08:39.151827 ip-10-0-128-39 kubenswrapper[2558]: I0710 18:08:39.151777 2558 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="opendatahub/kserve-controller-manager-d6f4b65b8-s4grg" Jul 10 18:08:40.060250 ip-10-0-128-39 kubenswrapper[2558]: I0710 18:08:40.060221 2558 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["opendatahub/kserve-controller-manager-d6f4b65b8-s4grg"] Jul 10 18:08:40.062413 ip-10-0-128-39 kubenswrapper[2558]: W0710 18:08:40.062374 2558 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pode28e2570_6896_422b_bfa9_30402ea86e77.slice/crio-6d04bb9d441db2a327598b067446eb7edce765ed738bd71f556f0f4e336ace5d WatchSource:0}: Error finding container 6d04bb9d441db2a327598b067446eb7edce765ed738bd71f556f0f4e336ace5d: Status 404 returned error can't find the container with id 6d04bb9d441db2a327598b067446eb7edce765ed738bd71f556f0f4e336ace5d Jul 10 18:08:40.685483 ip-10-0-128-39 kubenswrapper[2558]: I0710 18:08:40.685445 2558 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-lws-operator/lws-controller-manager-59dbbcc5d9-4l2s9" event={"ID":"48e923db-401e-40a8-acc6-7fb572b98f7f","Type":"ContainerStarted","Data":"be5bf44fdd88ce1709fcdc1577e1ab304c59373a93e8f6164f2bdc457a5741d9"} Jul 10 18:08:40.685678 ip-10-0-128-39 kubenswrapper[2558]: I0710 18:08:40.685575 2558 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-lws-operator/lws-controller-manager-59dbbcc5d9-4l2s9" Jul 10 18:08:40.686848 ip-10-0-128-39 kubenswrapper[2558]: I0710 18:08:40.686789 2558 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="opendatahub/kserve-controller-manager-d6f4b65b8-s4grg" event={"ID":"e28e2570-6896-422b-bfa9-30402ea86e77","Type":"ContainerStarted","Data":"6d04bb9d441db2a327598b067446eb7edce765ed738bd71f556f0f4e336ace5d"} Jul 10 18:08:40.712871 ip-10-0-128-39 kubenswrapper[2558]: I0710 18:08:40.712803 2558 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-lws-operator/lws-controller-manager-59dbbcc5d9-4l2s9" podStartSLOduration=1.256809934 podStartE2EDuration="3.712784134s" totalImagesPullingTime="2.4559742s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-10 18:08:37 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-10 18:08:40.712042009 +0000 UTC m=+349.215534515" watchObservedRunningTime="2026-07-10 18:08:40.712784134 +0000 UTC m=+349.216276645" Jul 10 18:08:43.707626 ip-10-0-128-39 kubenswrapper[2558]: I0710 18:08:43.707587 2558 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="opendatahub/kserve-controller-manager-d6f4b65b8-s4grg" event={"ID":"e28e2570-6896-422b-bfa9-30402ea86e77","Type":"ContainerStarted","Data":"5523afb87baad311e01f371cd579e6e98aecff939f40311fd2ec7ff4d61d2e79"} Jul 10 18:08:43.708132 ip-10-0-128-39 kubenswrapper[2558]: I0710 18:08:43.707713 2558 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="opendatahub/kserve-controller-manager-d6f4b65b8-s4grg" Jul 10 18:08:43.742887 ip-10-0-128-39 kubenswrapper[2558]: I0710 18:08:43.742838 2558 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="opendatahub/kserve-controller-manager-d6f4b65b8-s4grg" podStartSLOduration=2.754508699 podStartE2EDuration="5.742824088s" totalImagesPullingTime="2.988315389s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-10 18:08:38 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-10 18:08:43.741411567 +0000 UTC m=+352.244904072" watchObservedRunningTime="2026-07-10 18:08:43.742824088 +0000 UTC m=+352.246316585" Jul 10 18:08:51.699023 ip-10-0-128-39 kubenswrapper[2558]: I0710 18:08:51.698991 2558 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-lws-operator/lws-controller-manager-59dbbcc5d9-4l2s9" Jul 10 18:09:14.718244 ip-10-0-128-39 kubenswrapper[2558]: I0710 18:09:14.718207 2558 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="opendatahub/kserve-controller-manager-d6f4b65b8-s4grg" Jul 10 18:10:01.195948 ip-10-0-128-39 kubenswrapper[2558]: I0710 18:10:01.195914 2558 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["kuadrant-system/dns-operator-controller-manager-864dd78c4c-6x5s9"] Jul 10 18:10:01.198332 ip-10-0-128-39 kubenswrapper[2558]: I0710 18:10:01.198305 2558 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/dns-operator-controller-manager-864dd78c4c-6x5s9" Jul 10 18:10:01.204942 ip-10-0-128-39 kubenswrapper[2558]: I0710 18:10:01.204918 2558 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"kuadrant-system\"/\"dns-operator-controller-manager-dockercfg-bgg7c\"" Jul 10 18:10:01.205051 ip-10-0-128-39 kubenswrapper[2558]: I0710 18:10:01.204930 2558 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"kuadrant-system\"/\"dns-operator-controller-env\"" Jul 10 18:10:01.205566 ip-10-0-128-39 kubenswrapper[2558]: I0710 18:10:01.205551 2558 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"kuadrant-system\"/\"openshift-service-ca.crt\"" Jul 10 18:10:01.206828 ip-10-0-128-39 kubenswrapper[2558]: I0710 18:10:01.206799 2558 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"kuadrant-system\"/\"kube-root-ca.crt\"" Jul 10 18:10:01.221025 ip-10-0-128-39 kubenswrapper[2558]: I0710 18:10:01.220994 2558 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/dns-operator-controller-manager-864dd78c4c-6x5s9"] Jul 10 18:10:01.256909 ip-10-0-128-39 kubenswrapper[2558]: I0710 18:10:01.256876 2558 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-fxgx4\" (UniqueName: \"kubernetes.io/projected/b6eb1bbd-47d9-4504-b855-e0bd01f5a514-kube-api-access-fxgx4\") pod \"dns-operator-controller-manager-864dd78c4c-6x5s9\" (UID: \"b6eb1bbd-47d9-4504-b855-e0bd01f5a514\") " pod="kuadrant-system/dns-operator-controller-manager-864dd78c4c-6x5s9" Jul 10 18:10:01.357360 ip-10-0-128-39 kubenswrapper[2558]: I0710 18:10:01.357322 2558 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-fxgx4\" (UniqueName: \"kubernetes.io/projected/b6eb1bbd-47d9-4504-b855-e0bd01f5a514-kube-api-access-fxgx4\") pod \"dns-operator-controller-manager-864dd78c4c-6x5s9\" (UID: \"b6eb1bbd-47d9-4504-b855-e0bd01f5a514\") " pod="kuadrant-system/dns-operator-controller-manager-864dd78c4c-6x5s9" Jul 10 18:10:01.370418 ip-10-0-128-39 kubenswrapper[2558]: I0710 18:10:01.370394 2558 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-fxgx4\" (UniqueName: \"kubernetes.io/projected/b6eb1bbd-47d9-4504-b855-e0bd01f5a514-kube-api-access-fxgx4\") pod \"dns-operator-controller-manager-864dd78c4c-6x5s9\" (UID: \"b6eb1bbd-47d9-4504-b855-e0bd01f5a514\") " pod="kuadrant-system/dns-operator-controller-manager-864dd78c4c-6x5s9" Jul 10 18:10:01.508906 ip-10-0-128-39 kubenswrapper[2558]: I0710 18:10:01.508871 2558 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/dns-operator-controller-manager-864dd78c4c-6x5s9" Jul 10 18:10:01.661907 ip-10-0-128-39 kubenswrapper[2558]: I0710 18:10:01.661856 2558 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/dns-operator-controller-manager-864dd78c4c-6x5s9"] Jul 10 18:10:01.664780 ip-10-0-128-39 kubenswrapper[2558]: W0710 18:10:01.664749 2558 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podb6eb1bbd_47d9_4504_b855_e0bd01f5a514.slice/crio-318f4d421c6321ecd3d2dee9f53433668b2dc94bf314f13ba51d98c0812439be WatchSource:0}: Error finding container 318f4d421c6321ecd3d2dee9f53433668b2dc94bf314f13ba51d98c0812439be: Status 404 returned error can't find the container with id 318f4d421c6321ecd3d2dee9f53433668b2dc94bf314f13ba51d98c0812439be Jul 10 18:10:02.055276 ip-10-0-128-39 kubenswrapper[2558]: I0710 18:10:02.055233 2558 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/dns-operator-controller-manager-864dd78c4c-6x5s9" event={"ID":"b6eb1bbd-47d9-4504-b855-e0bd01f5a514","Type":"ContainerStarted","Data":"318f4d421c6321ecd3d2dee9f53433668b2dc94bf314f13ba51d98c0812439be"} Jul 10 18:10:03.967911 ip-10-0-128-39 kubenswrapper[2558]: I0710 18:10:03.967880 2558 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["kuadrant-system/limitador-operator-controller-manager-859d7b78bd-j7zn9"] Jul 10 18:10:03.970037 ip-10-0-128-39 kubenswrapper[2558]: I0710 18:10:03.970020 2558 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/limitador-operator-controller-manager-859d7b78bd-j7zn9" Jul 10 18:10:03.974216 ip-10-0-128-39 kubenswrapper[2558]: I0710 18:10:03.974194 2558 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"kuadrant-system\"/\"limitador-operator-controller-manager-dockercfg-c9p7d\"" Jul 10 18:10:03.978775 ip-10-0-128-39 kubenswrapper[2558]: I0710 18:10:03.978751 2558 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-hz6gj\" (UniqueName: \"kubernetes.io/projected/103d246f-45c1-49e6-80e8-19314ac0aa00-kube-api-access-hz6gj\") pod \"limitador-operator-controller-manager-859d7b78bd-j7zn9\" (UID: \"103d246f-45c1-49e6-80e8-19314ac0aa00\") " pod="kuadrant-system/limitador-operator-controller-manager-859d7b78bd-j7zn9" Jul 10 18:10:03.997324 ip-10-0-128-39 kubenswrapper[2558]: I0710 18:10:03.997297 2558 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-operator-controller-manager-859d7b78bd-j7zn9"] Jul 10 18:10:04.079752 ip-10-0-128-39 kubenswrapper[2558]: I0710 18:10:04.079713 2558 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-hz6gj\" (UniqueName: \"kubernetes.io/projected/103d246f-45c1-49e6-80e8-19314ac0aa00-kube-api-access-hz6gj\") pod \"limitador-operator-controller-manager-859d7b78bd-j7zn9\" (UID: \"103d246f-45c1-49e6-80e8-19314ac0aa00\") " pod="kuadrant-system/limitador-operator-controller-manager-859d7b78bd-j7zn9" Jul 10 18:10:04.099073 ip-10-0-128-39 kubenswrapper[2558]: I0710 18:10:04.099047 2558 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-hz6gj\" (UniqueName: \"kubernetes.io/projected/103d246f-45c1-49e6-80e8-19314ac0aa00-kube-api-access-hz6gj\") pod \"limitador-operator-controller-manager-859d7b78bd-j7zn9\" (UID: \"103d246f-45c1-49e6-80e8-19314ac0aa00\") " pod="kuadrant-system/limitador-operator-controller-manager-859d7b78bd-j7zn9" Jul 10 18:10:04.280487 ip-10-0-128-39 kubenswrapper[2558]: I0710 18:10:04.280409 2558 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/limitador-operator-controller-manager-859d7b78bd-j7zn9" Jul 10 18:10:04.432372 ip-10-0-128-39 kubenswrapper[2558]: I0710 18:10:04.432343 2558 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-operator-controller-manager-859d7b78bd-j7zn9"] Jul 10 18:10:04.435022 ip-10-0-128-39 kubenswrapper[2558]: W0710 18:10:04.434991 2558 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod103d246f_45c1_49e6_80e8_19314ac0aa00.slice/crio-06bad98fc58d900054c08c1a31101794885474f26353ce1cc2c341a3a5438656 WatchSource:0}: Error finding container 06bad98fc58d900054c08c1a31101794885474f26353ce1cc2c341a3a5438656: Status 404 returned error can't find the container with id 06bad98fc58d900054c08c1a31101794885474f26353ce1cc2c341a3a5438656 Jul 10 18:10:05.069233 ip-10-0-128-39 kubenswrapper[2558]: I0710 18:10:05.069196 2558 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/limitador-operator-controller-manager-859d7b78bd-j7zn9" event={"ID":"103d246f-45c1-49e6-80e8-19314ac0aa00","Type":"ContainerStarted","Data":"06bad98fc58d900054c08c1a31101794885474f26353ce1cc2c341a3a5438656"} Jul 10 18:10:05.071046 ip-10-0-128-39 kubenswrapper[2558]: I0710 18:10:05.071013 2558 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/dns-operator-controller-manager-864dd78c4c-6x5s9" event={"ID":"b6eb1bbd-47d9-4504-b855-e0bd01f5a514","Type":"ContainerStarted","Data":"ce38b219d666237bb8ad6a886501c48c409b17e0bdaa080ccfbbebbc6f2c80a9"} Jul 10 18:10:05.071199 ip-10-0-128-39 kubenswrapper[2558]: I0710 18:10:05.071180 2558 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="kuadrant-system/dns-operator-controller-manager-864dd78c4c-6x5s9" Jul 10 18:10:05.099229 ip-10-0-128-39 kubenswrapper[2558]: I0710 18:10:05.099169 2558 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="kuadrant-system/dns-operator-controller-manager-864dd78c4c-6x5s9" podStartSLOduration=0.825723631 podStartE2EDuration="4.099150635s" totalImagesPullingTime="3.273427004s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-10 18:10:01 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-10 18:10:05.098249717 +0000 UTC m=+433.601742225" watchObservedRunningTime="2026-07-10 18:10:05.099150635 +0000 UTC m=+433.602643140" Jul 10 18:10:07.081763 ip-10-0-128-39 kubenswrapper[2558]: I0710 18:10:07.081726 2558 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/limitador-operator-controller-manager-859d7b78bd-j7zn9" event={"ID":"103d246f-45c1-49e6-80e8-19314ac0aa00","Type":"ContainerStarted","Data":"f7657fa920217f734d6a7cdce17b2164561581e9f80eaf9ce790a1746235e68e"} Jul 10 18:10:07.082301 ip-10-0-128-39 kubenswrapper[2558]: I0710 18:10:07.081850 2558 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="kuadrant-system/limitador-operator-controller-manager-859d7b78bd-j7zn9" Jul 10 18:10:07.105754 ip-10-0-128-39 kubenswrapper[2558]: I0710 18:10:07.105706 2558 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="kuadrant-system/limitador-operator-controller-manager-859d7b78bd-j7zn9" podStartSLOduration=1.964311632 podStartE2EDuration="4.105690325s" totalImagesPullingTime="2.141378693s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-10 18:10:03 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-10 18:10:07.103466783 +0000 UTC m=+435.606959302" watchObservedRunningTime="2026-07-10 18:10:07.105690325 +0000 UTC m=+435.609182831" Jul 10 18:10:16.077879 ip-10-0-128-39 kubenswrapper[2558]: I0710 18:10:16.077778 2558 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="kuadrant-system/dns-operator-controller-manager-864dd78c4c-6x5s9" Jul 10 18:10:18.088300 ip-10-0-128-39 kubenswrapper[2558]: I0710 18:10:18.088267 2558 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="kuadrant-system/limitador-operator-controller-manager-859d7b78bd-j7zn9" Jul 10 18:10:20.356788 ip-10-0-128-39 kubenswrapper[2558]: I0710 18:10:20.356749 2558 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["kuadrant-system/limitador-operator-controller-manager-859d7b78bd-j7zn9"] Jul 10 18:10:20.357241 ip-10-0-128-39 kubenswrapper[2558]: I0710 18:10:20.357025 2558 kuberuntime_container.go:873] "Killing container with a grace period" pod="kuadrant-system/limitador-operator-controller-manager-859d7b78bd-j7zn9" podUID="103d246f-45c1-49e6-80e8-19314ac0aa00" containerName="manager" containerID="cri-o://f7657fa920217f734d6a7cdce17b2164561581e9f80eaf9ce790a1746235e68e" gracePeriod=2 Jul 10 18:10:20.379837 ip-10-0-128-39 kubenswrapper[2558]: I0710 18:10:20.379777 2558 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["kuadrant-system/limitador-operator-controller-manager-859d7b78bd-j7zn9"] Jul 10 18:10:20.392723 ip-10-0-128-39 kubenswrapper[2558]: E0710 18:10:20.392685 2558 status_manager.go:1041] "Failed to update status for pod" err="failed to patch status \"{\\\"metadata\\\":{\\\"uid\\\":\\\"103d246f-45c1-49e6-80e8-19314ac0aa00\\\"},\\\"status\\\":{\\\"$setElementOrder/conditions\\\":[{\\\"type\\\":\\\"PodReadyToStartContainers\\\"},{\\\"type\\\":\\\"Initialized\\\"},{\\\"type\\\":\\\"Ready\\\"},{\\\"type\\\":\\\"ContainersReady\\\"},{\\\"type\\\":\\\"PodScheduled\\\"}],\\\"conditions\\\":[{\\\"observedGeneration\\\":2,\\\"type\\\":\\\"PodReadyToStartContainers\\\"},{\\\"observedGeneration\\\":2,\\\"type\\\":\\\"Initialized\\\"},{\\\"observedGeneration\\\":2,\\\"type\\\":\\\"Ready\\\"},{\\\"observedGeneration\\\":2,\\\"type\\\":\\\"ContainersReady\\\"},{\\\"observedGeneration\\\":2,\\\"type\\\":\\\"PodScheduled\\\"}],\\\"observedGeneration\\\":2}}\" for pod \"kuadrant-system\"/\"limitador-operator-controller-manager-859d7b78bd-j7zn9\": pods \"limitador-operator-controller-manager-859d7b78bd-j7zn9\" not found" pod="kuadrant-system/limitador-operator-controller-manager-859d7b78bd-j7zn9" Jul 10 18:10:20.410339 ip-10-0-128-39 kubenswrapper[2558]: I0710 18:10:20.410298 2558 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["kuadrant-system/limitador-operator-controller-manager-859d7b78bd-6nzmh"] Jul 10 18:10:20.410645 ip-10-0-128-39 kubenswrapper[2558]: I0710 18:10:20.410632 2558 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="103d246f-45c1-49e6-80e8-19314ac0aa00" containerName="manager" Jul 10 18:10:20.410698 ip-10-0-128-39 kubenswrapper[2558]: I0710 18:10:20.410647 2558 state_mem.go:107] "Deleted CPUSet assignment" podUID="103d246f-45c1-49e6-80e8-19314ac0aa00" containerName="manager" Jul 10 18:10:20.410742 ip-10-0-128-39 kubenswrapper[2558]: I0710 18:10:20.410715 2558 memory_manager.go:368] "RemoveStaleState removing state" podUID="103d246f-45c1-49e6-80e8-19314ac0aa00" containerName="manager" Jul 10 18:10:20.413470 ip-10-0-128-39 kubenswrapper[2558]: I0710 18:10:20.413450 2558 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/limitador-operator-controller-manager-859d7b78bd-6nzmh" Jul 10 18:10:20.416762 ip-10-0-128-39 kubenswrapper[2558]: E0710 18:10:20.416737 2558 status_manager.go:1018] "Failed to get status for pod" err="pods \"limitador-operator-controller-manager-859d7b78bd-j7zn9\" is forbidden: User \"system:node:ip-10-0-128-39.ec2.internal\" cannot get resource \"pods\" in API group \"\" in the namespace \"kuadrant-system\": no relationship found between node 'ip-10-0-128-39.ec2.internal' and this object" podUID="103d246f-45c1-49e6-80e8-19314ac0aa00" pod="kuadrant-system/limitador-operator-controller-manager-859d7b78bd-j7zn9" Jul 10 18:10:20.426514 ip-10-0-128-39 kubenswrapper[2558]: I0710 18:10:20.426485 2558 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-operator-controller-manager-859d7b78bd-6nzmh"] Jul 10 18:10:20.514899 ip-10-0-128-39 kubenswrapper[2558]: I0710 18:10:20.514868 2558 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-mj5bg\" (UniqueName: \"kubernetes.io/projected/2abaa1e1-c899-43bd-930a-7f42e12adee0-kube-api-access-mj5bg\") pod \"limitador-operator-controller-manager-859d7b78bd-6nzmh\" (UID: \"2abaa1e1-c899-43bd-930a-7f42e12adee0\") " pod="kuadrant-system/limitador-operator-controller-manager-859d7b78bd-6nzmh" Jul 10 18:10:20.614617 ip-10-0-128-39 kubenswrapper[2558]: I0710 18:10:20.614552 2558 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/limitador-operator-controller-manager-859d7b78bd-j7zn9" Jul 10 18:10:20.615768 ip-10-0-128-39 kubenswrapper[2558]: I0710 18:10:20.615745 2558 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-mj5bg\" (UniqueName: \"kubernetes.io/projected/2abaa1e1-c899-43bd-930a-7f42e12adee0-kube-api-access-mj5bg\") pod \"limitador-operator-controller-manager-859d7b78bd-6nzmh\" (UID: \"2abaa1e1-c899-43bd-930a-7f42e12adee0\") " pod="kuadrant-system/limitador-operator-controller-manager-859d7b78bd-6nzmh" Jul 10 18:10:20.616988 ip-10-0-128-39 kubenswrapper[2558]: E0710 18:10:20.616964 2558 status_manager.go:1018] "Failed to get status for pod" err="pods \"limitador-operator-controller-manager-859d7b78bd-j7zn9\" is forbidden: User \"system:node:ip-10-0-128-39.ec2.internal\" cannot get resource \"pods\" in API group \"\" in the namespace \"kuadrant-system\": no relationship found between node 'ip-10-0-128-39.ec2.internal' and this object" podUID="103d246f-45c1-49e6-80e8-19314ac0aa00" pod="kuadrant-system/limitador-operator-controller-manager-859d7b78bd-j7zn9" Jul 10 18:10:20.627251 ip-10-0-128-39 kubenswrapper[2558]: I0710 18:10:20.627221 2558 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-mj5bg\" (UniqueName: \"kubernetes.io/projected/2abaa1e1-c899-43bd-930a-7f42e12adee0-kube-api-access-mj5bg\") pod \"limitador-operator-controller-manager-859d7b78bd-6nzmh\" (UID: \"2abaa1e1-c899-43bd-930a-7f42e12adee0\") " pod="kuadrant-system/limitador-operator-controller-manager-859d7b78bd-6nzmh" Jul 10 18:10:20.716302 ip-10-0-128-39 kubenswrapper[2558]: I0710 18:10:20.716268 2558 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-hz6gj\" (UniqueName: \"kubernetes.io/projected/103d246f-45c1-49e6-80e8-19314ac0aa00-kube-api-access-hz6gj\") pod \"103d246f-45c1-49e6-80e8-19314ac0aa00\" (UID: \"103d246f-45c1-49e6-80e8-19314ac0aa00\") " Jul 10 18:10:20.718522 ip-10-0-128-39 kubenswrapper[2558]: I0710 18:10:20.718493 2558 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/103d246f-45c1-49e6-80e8-19314ac0aa00-kube-api-access-hz6gj" (OuterVolumeSpecName: "kube-api-access-hz6gj") pod "103d246f-45c1-49e6-80e8-19314ac0aa00" (UID: "103d246f-45c1-49e6-80e8-19314ac0aa00"). InnerVolumeSpecName "kube-api-access-hz6gj". PluginName "kubernetes.io/projected", VolumeGIDValue "" Jul 10 18:10:20.735706 ip-10-0-128-39 kubenswrapper[2558]: I0710 18:10:20.735674 2558 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/limitador-operator-controller-manager-859d7b78bd-6nzmh" Jul 10 18:10:20.817714 ip-10-0-128-39 kubenswrapper[2558]: I0710 18:10:20.817683 2558 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-hz6gj\" (UniqueName: \"kubernetes.io/projected/103d246f-45c1-49e6-80e8-19314ac0aa00-kube-api-access-hz6gj\") on node \"ip-10-0-128-39.ec2.internal\" DevicePath \"\"" Jul 10 18:10:20.883513 ip-10-0-128-39 kubenswrapper[2558]: I0710 18:10:20.883487 2558 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-operator-controller-manager-859d7b78bd-6nzmh"] Jul 10 18:10:20.886117 ip-10-0-128-39 kubenswrapper[2558]: W0710 18:10:20.886085 2558 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod2abaa1e1_c899_43bd_930a_7f42e12adee0.slice/crio-74a94012d1c9b53842dbd4f9d959e2fbd9220365c29047c98750d9b949c9fd92 WatchSource:0}: Error finding container 74a94012d1c9b53842dbd4f9d959e2fbd9220365c29047c98750d9b949c9fd92: Status 404 returned error can't find the container with id 74a94012d1c9b53842dbd4f9d959e2fbd9220365c29047c98750d9b949c9fd92 Jul 10 18:10:21.145275 ip-10-0-128-39 kubenswrapper[2558]: I0710 18:10:21.145170 2558 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/limitador-operator-controller-manager-859d7b78bd-6nzmh" event={"ID":"2abaa1e1-c899-43bd-930a-7f42e12adee0","Type":"ContainerStarted","Data":"514a93cb9beb8affa6c863f6505ae683f0d6ba34c73938dd03ab79ef49abddfc"} Jul 10 18:10:21.145275 ip-10-0-128-39 kubenswrapper[2558]: I0710 18:10:21.145240 2558 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="kuadrant-system/limitador-operator-controller-manager-859d7b78bd-6nzmh" Jul 10 18:10:21.145275 ip-10-0-128-39 kubenswrapper[2558]: I0710 18:10:21.145267 2558 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/limitador-operator-controller-manager-859d7b78bd-6nzmh" event={"ID":"2abaa1e1-c899-43bd-930a-7f42e12adee0","Type":"ContainerStarted","Data":"74a94012d1c9b53842dbd4f9d959e2fbd9220365c29047c98750d9b949c9fd92"} Jul 10 18:10:21.146417 ip-10-0-128-39 kubenswrapper[2558]: I0710 18:10:21.146386 2558 generic.go:356] "Generic (PLEG): container finished" podID="103d246f-45c1-49e6-80e8-19314ac0aa00" containerID="f7657fa920217f734d6a7cdce17b2164561581e9f80eaf9ce790a1746235e68e" exitCode=0 Jul 10 18:10:21.146508 ip-10-0-128-39 kubenswrapper[2558]: I0710 18:10:21.146424 2558 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/limitador-operator-controller-manager-859d7b78bd-j7zn9" Jul 10 18:10:21.146508 ip-10-0-128-39 kubenswrapper[2558]: I0710 18:10:21.146460 2558 scope.go:117] "RemoveContainer" containerID="f7657fa920217f734d6a7cdce17b2164561581e9f80eaf9ce790a1746235e68e" Jul 10 18:10:21.155265 ip-10-0-128-39 kubenswrapper[2558]: I0710 18:10:21.155242 2558 scope.go:117] "RemoveContainer" containerID="f7657fa920217f734d6a7cdce17b2164561581e9f80eaf9ce790a1746235e68e" Jul 10 18:10:21.155546 ip-10-0-128-39 kubenswrapper[2558]: E0710 18:10:21.155521 2558 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"f7657fa920217f734d6a7cdce17b2164561581e9f80eaf9ce790a1746235e68e\": container with ID starting with f7657fa920217f734d6a7cdce17b2164561581e9f80eaf9ce790a1746235e68e not found: ID does not exist" containerID="f7657fa920217f734d6a7cdce17b2164561581e9f80eaf9ce790a1746235e68e" Jul 10 18:10:21.155635 ip-10-0-128-39 kubenswrapper[2558]: I0710 18:10:21.155550 2558 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"f7657fa920217f734d6a7cdce17b2164561581e9f80eaf9ce790a1746235e68e"} err="failed to get container status \"f7657fa920217f734d6a7cdce17b2164561581e9f80eaf9ce790a1746235e68e\": rpc error: code = NotFound desc = could not find container \"f7657fa920217f734d6a7cdce17b2164561581e9f80eaf9ce790a1746235e68e\": container with ID starting with f7657fa920217f734d6a7cdce17b2164561581e9f80eaf9ce790a1746235e68e not found: ID does not exist" Jul 10 18:10:21.165776 ip-10-0-128-39 kubenswrapper[2558]: I0710 18:10:21.165734 2558 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="kuadrant-system/limitador-operator-controller-manager-859d7b78bd-6nzmh" podStartSLOduration=1.165720483 podStartE2EDuration="1.165720483s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-10 18:10:20 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-10 18:10:21.164487747 +0000 UTC m=+449.667980252" watchObservedRunningTime="2026-07-10 18:10:21.165720483 +0000 UTC m=+449.669212988" Jul 10 18:10:21.166736 ip-10-0-128-39 kubenswrapper[2558]: E0710 18:10:21.166709 2558 status_manager.go:1018] "Failed to get status for pod" err="pods \"limitador-operator-controller-manager-859d7b78bd-j7zn9\" is forbidden: User \"system:node:ip-10-0-128-39.ec2.internal\" cannot get resource \"pods\" in API group \"\" in the namespace \"kuadrant-system\": no relationship found between node 'ip-10-0-128-39.ec2.internal' and this object" podUID="103d246f-45c1-49e6-80e8-19314ac0aa00" pod="kuadrant-system/limitador-operator-controller-manager-859d7b78bd-j7zn9" Jul 10 18:10:21.168515 ip-10-0-128-39 kubenswrapper[2558]: E0710 18:10:21.168486 2558 status_manager.go:1018] "Failed to get status for pod" err="pods \"limitador-operator-controller-manager-859d7b78bd-j7zn9\" is forbidden: User \"system:node:ip-10-0-128-39.ec2.internal\" cannot get resource \"pods\" in API group \"\" in the namespace \"kuadrant-system\": no relationship found between node 'ip-10-0-128-39.ec2.internal' and this object" podUID="103d246f-45c1-49e6-80e8-19314ac0aa00" pod="kuadrant-system/limitador-operator-controller-manager-859d7b78bd-j7zn9" Jul 10 18:10:22.099557 ip-10-0-128-39 kubenswrapper[2558]: E0710 18:10:22.099521 2558 status_manager.go:1018] "Failed to get status for pod" err="pods \"limitador-operator-controller-manager-859d7b78bd-j7zn9\" is forbidden: User \"system:node:ip-10-0-128-39.ec2.internal\" cannot get resource \"pods\" in API group \"\" in the namespace \"kuadrant-system\": no relationship found between node 'ip-10-0-128-39.ec2.internal' and this object" podUID="103d246f-45c1-49e6-80e8-19314ac0aa00" pod="kuadrant-system/limitador-operator-controller-manager-859d7b78bd-j7zn9" Jul 10 18:10:22.100913 ip-10-0-128-39 kubenswrapper[2558]: I0710 18:10:22.100887 2558 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="103d246f-45c1-49e6-80e8-19314ac0aa00" path="/var/lib/kubelet/pods/103d246f-45c1-49e6-80e8-19314ac0aa00/volumes" Jul 10 18:10:32.154825 ip-10-0-128-39 kubenswrapper[2558]: I0710 18:10:32.154776 2558 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="kuadrant-system/limitador-operator-controller-manager-859d7b78bd-6nzmh" Jul 10 18:11:01.326383 ip-10-0-128-39 kubenswrapper[2558]: I0710 18:11:01.326347 2558 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-hjf5z"] Jul 10 18:11:01.329437 ip-10-0-128-39 kubenswrapper[2558]: I0710 18:11:01.329417 2558 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/limitador-limitador-54d6d988d5-hjf5z" Jul 10 18:11:01.332013 ip-10-0-128-39 kubenswrapper[2558]: I0710 18:11:01.331991 2558 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"kuadrant-system\"/\"limitador-limits-config-limitador\"" Jul 10 18:11:01.332121 ip-10-0-128-39 kubenswrapper[2558]: I0710 18:11:01.332049 2558 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"kuadrant-system\"/\"default-dockercfg-kdj45\"" Jul 10 18:11:01.342850 ip-10-0-128-39 kubenswrapper[2558]: I0710 18:11:01.342825 2558 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-hjf5z"] Jul 10 18:11:01.358843 ip-10-0-128-39 kubenswrapper[2558]: I0710 18:11:01.358794 2558 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-hjf5z"] Jul 10 18:11:01.369169 ip-10-0-128-39 kubenswrapper[2558]: I0710 18:11:01.369145 2558 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"config-file\" (UniqueName: \"kubernetes.io/configmap/55873f81-1144-4875-b3aa-88b7c1c3da29-config-file\") pod \"limitador-limitador-54d6d988d5-hjf5z\" (UID: \"55873f81-1144-4875-b3aa-88b7c1c3da29\") " pod="kuadrant-system/limitador-limitador-54d6d988d5-hjf5z" Jul 10 18:11:01.369305 ip-10-0-128-39 kubenswrapper[2558]: I0710 18:11:01.369182 2558 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-kwxz8\" (UniqueName: \"kubernetes.io/projected/55873f81-1144-4875-b3aa-88b7c1c3da29-kube-api-access-kwxz8\") pod \"limitador-limitador-54d6d988d5-hjf5z\" (UID: \"55873f81-1144-4875-b3aa-88b7c1c3da29\") " pod="kuadrant-system/limitador-limitador-54d6d988d5-hjf5z" Jul 10 18:11:01.470219 ip-10-0-128-39 kubenswrapper[2558]: I0710 18:11:01.470180 2558 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"config-file\" (UniqueName: \"kubernetes.io/configmap/55873f81-1144-4875-b3aa-88b7c1c3da29-config-file\") pod \"limitador-limitador-54d6d988d5-hjf5z\" (UID: \"55873f81-1144-4875-b3aa-88b7c1c3da29\") " pod="kuadrant-system/limitador-limitador-54d6d988d5-hjf5z" Jul 10 18:11:01.470415 ip-10-0-128-39 kubenswrapper[2558]: I0710 18:11:01.470226 2558 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-kwxz8\" (UniqueName: \"kubernetes.io/projected/55873f81-1144-4875-b3aa-88b7c1c3da29-kube-api-access-kwxz8\") pod \"limitador-limitador-54d6d988d5-hjf5z\" (UID: \"55873f81-1144-4875-b3aa-88b7c1c3da29\") " pod="kuadrant-system/limitador-limitador-54d6d988d5-hjf5z" Jul 10 18:11:01.470925 ip-10-0-128-39 kubenswrapper[2558]: I0710 18:11:01.470902 2558 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"config-file\" (UniqueName: \"kubernetes.io/configmap/55873f81-1144-4875-b3aa-88b7c1c3da29-config-file\") pod \"limitador-limitador-54d6d988d5-hjf5z\" (UID: \"55873f81-1144-4875-b3aa-88b7c1c3da29\") " pod="kuadrant-system/limitador-limitador-54d6d988d5-hjf5z" Jul 10 18:11:01.479045 ip-10-0-128-39 kubenswrapper[2558]: I0710 18:11:01.479023 2558 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-kwxz8\" (UniqueName: \"kubernetes.io/projected/55873f81-1144-4875-b3aa-88b7c1c3da29-kube-api-access-kwxz8\") pod \"limitador-limitador-54d6d988d5-hjf5z\" (UID: \"55873f81-1144-4875-b3aa-88b7c1c3da29\") " pod="kuadrant-system/limitador-limitador-54d6d988d5-hjf5z" Jul 10 18:11:01.640182 ip-10-0-128-39 kubenswrapper[2558]: I0710 18:11:01.640079 2558 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/limitador-limitador-54d6d988d5-hjf5z" Jul 10 18:11:01.779876 ip-10-0-128-39 kubenswrapper[2558]: I0710 18:11:01.779851 2558 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-hjf5z"] Jul 10 18:11:01.782296 ip-10-0-128-39 kubenswrapper[2558]: W0710 18:11:01.782263 2558 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod55873f81_1144_4875_b3aa_88b7c1c3da29.slice/crio-cc4f0497e72e7452f800d169cbba022807ff0eb2fe7d6a4e120f1ecbf813bffc WatchSource:0}: Error finding container cc4f0497e72e7452f800d169cbba022807ff0eb2fe7d6a4e120f1ecbf813bffc: Status 404 returned error can't find the container with id cc4f0497e72e7452f800d169cbba022807ff0eb2fe7d6a4e120f1ecbf813bffc Jul 10 18:11:02.345401 ip-10-0-128-39 kubenswrapper[2558]: I0710 18:11:02.345359 2558 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/limitador-limitador-54d6d988d5-hjf5z" event={"ID":"55873f81-1144-4875-b3aa-88b7c1c3da29","Type":"ContainerStarted","Data":"cc4f0497e72e7452f800d169cbba022807ff0eb2fe7d6a4e120f1ecbf813bffc"} Jul 10 18:11:04.358724 ip-10-0-128-39 kubenswrapper[2558]: I0710 18:11:04.358674 2558 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/limitador-limitador-54d6d988d5-hjf5z" event={"ID":"55873f81-1144-4875-b3aa-88b7c1c3da29","Type":"ContainerStarted","Data":"f723e4759c4e3b68690019363879283252f3052582ed39affe40dd9dfefd13a3"} Jul 10 18:11:04.359110 ip-10-0-128-39 kubenswrapper[2558]: I0710 18:11:04.358754 2558 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="kuadrant-system/limitador-limitador-54d6d988d5-hjf5z" Jul 10 18:11:04.385876 ip-10-0-128-39 kubenswrapper[2558]: I0710 18:11:04.385802 2558 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="kuadrant-system/limitador-limitador-54d6d988d5-hjf5z" podStartSLOduration=1.023097024 podStartE2EDuration="3.38578482s" totalImagesPullingTime="2.362687796s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-10 18:11:01 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-10 18:11:04.384132282 +0000 UTC m=+492.887624789" watchObservedRunningTime="2026-07-10 18:11:04.38578482 +0000 UTC m=+492.889277325" Jul 10 18:11:15.363627 ip-10-0-128-39 kubenswrapper[2558]: I0710 18:11:15.363597 2558 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="kuadrant-system/limitador-limitador-54d6d988d5-hjf5z" Jul 10 18:12:14.905849 ip-10-0-128-39 kubenswrapper[2558]: I0710 18:12:14.905742 2558 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-hjf5z"] Jul 10 18:12:49.493326 ip-10-0-128-39 kubenswrapper[2558]: I0710 18:12:49.493291 2558 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-hjf5z"] Jul 10 18:12:51.970335 ip-10-0-128-39 kubenswrapper[2558]: I0710 18:12:51.970304 2558 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-console-operator_console-operator-7866497494-26542_2b29fb39-89b3-42c3-b334-3a195242248c/console-operator/1.log" Jul 10 18:12:51.974297 ip-10-0-128-39 kubenswrapper[2558]: I0710 18:12:51.974272 2558 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-console-operator_console-operator-7866497494-26542_2b29fb39-89b3-42c3-b334-3a195242248c/console-operator/1.log" Jul 10 18:12:51.974875 ip-10-0-128-39 kubenswrapper[2558]: I0710 18:12:51.974857 2558 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ovn-kubernetes_ovnkube-node-ll44q_eabc6c69-b1e8-40d5-b41f-f49778a5248c/ovn-acl-logging/0.log" Jul 10 18:12:51.978461 ip-10-0-128-39 kubenswrapper[2558]: I0710 18:12:51.978443 2558 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ovn-kubernetes_ovnkube-node-ll44q_eabc6c69-b1e8-40d5-b41f-f49778a5248c/ovn-acl-logging/0.log" Jul 10 18:12:59.995772 ip-10-0-128-39 kubenswrapper[2558]: I0710 18:12:59.995736 2558 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-hjf5z"] Jul 10 18:13:05.194934 ip-10-0-128-39 kubenswrapper[2558]: I0710 18:13:05.194897 2558 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-hjf5z"] Jul 10 18:13:17.594998 ip-10-0-128-39 kubenswrapper[2558]: I0710 18:13:17.594909 2558 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-hjf5z"] Jul 10 18:13:25.585649 ip-10-0-128-39 kubenswrapper[2558]: I0710 18:13:25.585615 2558 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["llm/facebook-opt-125m-simulated-kserve-74b68447df-mpr5p"] Jul 10 18:13:25.588480 ip-10-0-128-39 kubenswrapper[2558]: I0710 18:13:25.588457 2558 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="llm/facebook-opt-125m-simulated-kserve-74b68447df-mpr5p" Jul 10 18:13:25.592297 ip-10-0-128-39 kubenswrapper[2558]: I0710 18:13:25.592271 2558 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"llm\"/\"default-dockercfg-t2lwp\"" Jul 10 18:13:25.592425 ip-10-0-128-39 kubenswrapper[2558]: I0710 18:13:25.592300 2558 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"llm\"/\"kube-root-ca.crt\"" Jul 10 18:13:25.592425 ip-10-0-128-39 kubenswrapper[2558]: I0710 18:13:25.592339 2558 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"llm\"/\"facebook-opt-125m-simulated-kserve-self-signed-certs\"" Jul 10 18:13:25.592425 ip-10-0-128-39 kubenswrapper[2558]: I0710 18:13:25.592344 2558 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"llm\"/\"openshift-service-ca.crt\"" Jul 10 18:13:25.606178 ip-10-0-128-39 kubenswrapper[2558]: I0710 18:13:25.606156 2558 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["llm/facebook-opt-125m-simulated-kserve-74b68447df-mpr5p"] Jul 10 18:13:25.699762 ip-10-0-128-39 kubenswrapper[2558]: I0710 18:13:25.699726 2558 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-l6ft4\" (UniqueName: \"kubernetes.io/projected/c4e8ab9b-bb52-4775-b742-eb3fc91269fd-kube-api-access-l6ft4\") pod \"facebook-opt-125m-simulated-kserve-74b68447df-mpr5p\" (UID: \"c4e8ab9b-bb52-4775-b742-eb3fc91269fd\") " pod="llm/facebook-opt-125m-simulated-kserve-74b68447df-mpr5p" Jul 10 18:13:25.699959 ip-10-0-128-39 kubenswrapper[2558]: I0710 18:13:25.699770 2558 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/c4e8ab9b-bb52-4775-b742-eb3fc91269fd-dshm\") pod \"facebook-opt-125m-simulated-kserve-74b68447df-mpr5p\" (UID: \"c4e8ab9b-bb52-4775-b742-eb3fc91269fd\") " pod="llm/facebook-opt-125m-simulated-kserve-74b68447df-mpr5p" Jul 10 18:13:25.699959 ip-10-0-128-39 kubenswrapper[2558]: I0710 18:13:25.699831 2558 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/c4e8ab9b-bb52-4775-b742-eb3fc91269fd-tls-certs\") pod \"facebook-opt-125m-simulated-kserve-74b68447df-mpr5p\" (UID: \"c4e8ab9b-bb52-4775-b742-eb3fc91269fd\") " pod="llm/facebook-opt-125m-simulated-kserve-74b68447df-mpr5p" Jul 10 18:13:25.699959 ip-10-0-128-39 kubenswrapper[2558]: I0710 18:13:25.699864 2558 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/c4e8ab9b-bb52-4775-b742-eb3fc91269fd-model-cache\") pod \"facebook-opt-125m-simulated-kserve-74b68447df-mpr5p\" (UID: \"c4e8ab9b-bb52-4775-b742-eb3fc91269fd\") " pod="llm/facebook-opt-125m-simulated-kserve-74b68447df-mpr5p" Jul 10 18:13:25.699959 ip-10-0-128-39 kubenswrapper[2558]: I0710 18:13:25.699891 2558 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/c4e8ab9b-bb52-4775-b742-eb3fc91269fd-home\") pod \"facebook-opt-125m-simulated-kserve-74b68447df-mpr5p\" (UID: \"c4e8ab9b-bb52-4775-b742-eb3fc91269fd\") " pod="llm/facebook-opt-125m-simulated-kserve-74b68447df-mpr5p" Jul 10 18:13:25.700093 ip-10-0-128-39 kubenswrapper[2558]: I0710 18:13:25.699955 2558 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/c4e8ab9b-bb52-4775-b742-eb3fc91269fd-tmp-dir\") pod \"facebook-opt-125m-simulated-kserve-74b68447df-mpr5p\" (UID: \"c4e8ab9b-bb52-4775-b742-eb3fc91269fd\") " pod="llm/facebook-opt-125m-simulated-kserve-74b68447df-mpr5p" Jul 10 18:13:25.801373 ip-10-0-128-39 kubenswrapper[2558]: I0710 18:13:25.801332 2558 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/c4e8ab9b-bb52-4775-b742-eb3fc91269fd-tmp-dir\") pod \"facebook-opt-125m-simulated-kserve-74b68447df-mpr5p\" (UID: \"c4e8ab9b-bb52-4775-b742-eb3fc91269fd\") " pod="llm/facebook-opt-125m-simulated-kserve-74b68447df-mpr5p" Jul 10 18:13:25.801562 ip-10-0-128-39 kubenswrapper[2558]: I0710 18:13:25.801393 2558 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-l6ft4\" (UniqueName: \"kubernetes.io/projected/c4e8ab9b-bb52-4775-b742-eb3fc91269fd-kube-api-access-l6ft4\") pod \"facebook-opt-125m-simulated-kserve-74b68447df-mpr5p\" (UID: \"c4e8ab9b-bb52-4775-b742-eb3fc91269fd\") " pod="llm/facebook-opt-125m-simulated-kserve-74b68447df-mpr5p" Jul 10 18:13:25.801562 ip-10-0-128-39 kubenswrapper[2558]: I0710 18:13:25.801429 2558 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/c4e8ab9b-bb52-4775-b742-eb3fc91269fd-dshm\") pod \"facebook-opt-125m-simulated-kserve-74b68447df-mpr5p\" (UID: \"c4e8ab9b-bb52-4775-b742-eb3fc91269fd\") " pod="llm/facebook-opt-125m-simulated-kserve-74b68447df-mpr5p" Jul 10 18:13:25.801562 ip-10-0-128-39 kubenswrapper[2558]: I0710 18:13:25.801455 2558 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/c4e8ab9b-bb52-4775-b742-eb3fc91269fd-tls-certs\") pod \"facebook-opt-125m-simulated-kserve-74b68447df-mpr5p\" (UID: \"c4e8ab9b-bb52-4775-b742-eb3fc91269fd\") " pod="llm/facebook-opt-125m-simulated-kserve-74b68447df-mpr5p" Jul 10 18:13:25.801562 ip-10-0-128-39 kubenswrapper[2558]: I0710 18:13:25.801485 2558 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/c4e8ab9b-bb52-4775-b742-eb3fc91269fd-model-cache\") pod \"facebook-opt-125m-simulated-kserve-74b68447df-mpr5p\" (UID: \"c4e8ab9b-bb52-4775-b742-eb3fc91269fd\") " pod="llm/facebook-opt-125m-simulated-kserve-74b68447df-mpr5p" Jul 10 18:13:25.801562 ip-10-0-128-39 kubenswrapper[2558]: I0710 18:13:25.801512 2558 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/c4e8ab9b-bb52-4775-b742-eb3fc91269fd-home\") pod \"facebook-opt-125m-simulated-kserve-74b68447df-mpr5p\" (UID: \"c4e8ab9b-bb52-4775-b742-eb3fc91269fd\") " pod="llm/facebook-opt-125m-simulated-kserve-74b68447df-mpr5p" Jul 10 18:13:25.801852 ip-10-0-128-39 kubenswrapper[2558]: I0710 18:13:25.801798 2558 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/c4e8ab9b-bb52-4775-b742-eb3fc91269fd-tmp-dir\") pod \"facebook-opt-125m-simulated-kserve-74b68447df-mpr5p\" (UID: \"c4e8ab9b-bb52-4775-b742-eb3fc91269fd\") " pod="llm/facebook-opt-125m-simulated-kserve-74b68447df-mpr5p" Jul 10 18:13:25.801983 ip-10-0-128-39 kubenswrapper[2558]: I0710 18:13:25.801877 2558 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/c4e8ab9b-bb52-4775-b742-eb3fc91269fd-home\") pod \"facebook-opt-125m-simulated-kserve-74b68447df-mpr5p\" (UID: \"c4e8ab9b-bb52-4775-b742-eb3fc91269fd\") " pod="llm/facebook-opt-125m-simulated-kserve-74b68447df-mpr5p" Jul 10 18:13:25.801983 ip-10-0-128-39 kubenswrapper[2558]: I0710 18:13:25.801925 2558 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/c4e8ab9b-bb52-4775-b742-eb3fc91269fd-model-cache\") pod \"facebook-opt-125m-simulated-kserve-74b68447df-mpr5p\" (UID: \"c4e8ab9b-bb52-4775-b742-eb3fc91269fd\") " pod="llm/facebook-opt-125m-simulated-kserve-74b68447df-mpr5p" Jul 10 18:13:25.803713 ip-10-0-128-39 kubenswrapper[2558]: I0710 18:13:25.803691 2558 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/c4e8ab9b-bb52-4775-b742-eb3fc91269fd-dshm\") pod \"facebook-opt-125m-simulated-kserve-74b68447df-mpr5p\" (UID: \"c4e8ab9b-bb52-4775-b742-eb3fc91269fd\") " pod="llm/facebook-opt-125m-simulated-kserve-74b68447df-mpr5p" Jul 10 18:13:25.803940 ip-10-0-128-39 kubenswrapper[2558]: I0710 18:13:25.803920 2558 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/c4e8ab9b-bb52-4775-b742-eb3fc91269fd-tls-certs\") pod \"facebook-opt-125m-simulated-kserve-74b68447df-mpr5p\" (UID: \"c4e8ab9b-bb52-4775-b742-eb3fc91269fd\") " pod="llm/facebook-opt-125m-simulated-kserve-74b68447df-mpr5p" Jul 10 18:13:25.810253 ip-10-0-128-39 kubenswrapper[2558]: I0710 18:13:25.810228 2558 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-l6ft4\" (UniqueName: \"kubernetes.io/projected/c4e8ab9b-bb52-4775-b742-eb3fc91269fd-kube-api-access-l6ft4\") pod \"facebook-opt-125m-simulated-kserve-74b68447df-mpr5p\" (UID: \"c4e8ab9b-bb52-4775-b742-eb3fc91269fd\") " pod="llm/facebook-opt-125m-simulated-kserve-74b68447df-mpr5p" Jul 10 18:13:25.898564 ip-10-0-128-39 kubenswrapper[2558]: I0710 18:13:25.898470 2558 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="llm/facebook-opt-125m-simulated-kserve-74b68447df-mpr5p" Jul 10 18:13:26.037142 ip-10-0-128-39 kubenswrapper[2558]: I0710 18:13:26.037105 2558 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["llm/facebook-opt-125m-simulated-kserve-74b68447df-mpr5p"] Jul 10 18:13:26.040557 ip-10-0-128-39 kubenswrapper[2558]: W0710 18:13:26.040529 2558 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podc4e8ab9b_bb52_4775_b742_eb3fc91269fd.slice/crio-2ab1fc54027b4e61fd8c046b18a17af898693eb34272c2c11846bab308e2381e WatchSource:0}: Error finding container 2ab1fc54027b4e61fd8c046b18a17af898693eb34272c2c11846bab308e2381e: Status 404 returned error can't find the container with id 2ab1fc54027b4e61fd8c046b18a17af898693eb34272c2c11846bab308e2381e Jul 10 18:13:26.041838 ip-10-0-128-39 kubenswrapper[2558]: I0710 18:13:26.041797 2558 provider.go:93] Refreshing cache for provider: *credentialprovider.defaultDockerConfigProvider Jul 10 18:13:27.002577 ip-10-0-128-39 kubenswrapper[2558]: I0710 18:13:27.002539 2558 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="llm/facebook-opt-125m-simulated-kserve-74b68447df-mpr5p" event={"ID":"c4e8ab9b-bb52-4775-b742-eb3fc91269fd","Type":"ContainerStarted","Data":"2ab1fc54027b4e61fd8c046b18a17af898693eb34272c2c11846bab308e2381e"} Jul 10 18:13:27.788421 ip-10-0-128-39 kubenswrapper[2558]: I0710 18:13:27.788390 2558 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-hjf5z"] Jul 10 18:13:28.008167 ip-10-0-128-39 kubenswrapper[2558]: I0710 18:13:28.008135 2558 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="llm/facebook-opt-125m-simulated-kserve-74b68447df-mpr5p" event={"ID":"c4e8ab9b-bb52-4775-b742-eb3fc91269fd","Type":"ContainerStarted","Data":"197a5210be25c3c5bcb0e697436cc06d185b040c150a755e166726ae4badd7f0"} Jul 10 18:13:28.026849 ip-10-0-128-39 kubenswrapper[2558]: I0710 18:13:28.026777 2558 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="llm/facebook-opt-125m-simulated-kserve-74b68447df-mpr5p" podStartSLOduration=2.035612489 podStartE2EDuration="3.026762407s" totalImagesPullingTime="991.149918ms" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-10 18:13:25 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-10 18:13:28.025995228 +0000 UTC m=+636.529487736" watchObservedRunningTime="2026-07-10 18:13:28.026762407 +0000 UTC m=+636.530254911" Jul 10 18:13:35.899058 ip-10-0-128-39 kubenswrapper[2558]: I0710 18:13:35.899012 2558 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="llm/facebook-opt-125m-simulated-kserve-74b68447df-mpr5p" Jul 10 18:13:35.899470 ip-10-0-128-39 kubenswrapper[2558]: I0710 18:13:35.899082 2558 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="llm/facebook-opt-125m-simulated-kserve-74b68447df-mpr5p" Jul 10 18:13:35.911252 ip-10-0-128-39 kubenswrapper[2558]: I0710 18:13:35.911221 2558 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="llm/facebook-opt-125m-simulated-kserve-74b68447df-mpr5p" Jul 10 18:13:36.056091 ip-10-0-128-39 kubenswrapper[2558]: I0710 18:13:36.056064 2558 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="llm/facebook-opt-125m-simulated-kserve-74b68447df-mpr5p" Jul 10 18:13:36.195109 ip-10-0-128-39 kubenswrapper[2558]: I0710 18:13:36.195026 2558 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-hjf5z"] Jul 10 18:15:00.152395 ip-10-0-128-39 kubenswrapper[2558]: I0710 18:15:00.152313 2558 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["odh-ai-gateway-infra/maas-api-key-cleanup-29728455-bsrzq"] Jul 10 18:15:00.156885 ip-10-0-128-39 kubenswrapper[2558]: I0710 18:15:00.155585 2558 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-key-cleanup-29728455-bsrzq" Jul 10 18:15:00.160407 ip-10-0-128-39 kubenswrapper[2558]: I0710 18:15:00.160380 2558 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"odh-ai-gateway-infra\"/\"maas-api-cleanup-dockercfg-fhgdj\"" Jul 10 18:15:00.162508 ip-10-0-128-39 kubenswrapper[2558]: I0710 18:15:00.162480 2558 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-key-cleanup-29728455-bsrzq"] Jul 10 18:15:00.168987 ip-10-0-128-39 kubenswrapper[2558]: I0710 18:15:00.168966 2558 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-key-cleanup-29728455-bsrzq" Jul 10 18:15:00.300744 ip-10-0-128-39 kubenswrapper[2558]: I0710 18:15:00.300700 2558 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-hjf5z"] Jul 10 18:15:00.304053 ip-10-0-128-39 kubenswrapper[2558]: I0710 18:15:00.304031 2558 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-key-cleanup-29728455-bsrzq"] Jul 10 18:15:00.306825 ip-10-0-128-39 kubenswrapper[2558]: W0710 18:15:00.306785 2558 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podc69a5e97_ddc4_4e40_8c44_143a97f7e699.slice/crio-53652c5fd5b2559fda205aad1876455d05d8f3df685e568911b9a11adc01f43d WatchSource:0}: Error finding container 53652c5fd5b2559fda205aad1876455d05d8f3df685e568911b9a11adc01f43d: Status 404 returned error can't find the container with id 53652c5fd5b2559fda205aad1876455d05d8f3df685e568911b9a11adc01f43d Jul 10 18:15:00.432209 ip-10-0-128-39 kubenswrapper[2558]: I0710 18:15:00.432121 2558 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-key-cleanup-29728455-bsrzq" event={"ID":"c69a5e97-ddc4-4e40-8c44-143a97f7e699","Type":"ContainerStarted","Data":"53652c5fd5b2559fda205aad1876455d05d8f3df685e568911b9a11adc01f43d"} Jul 10 18:15:03.448168 ip-10-0-128-39 kubenswrapper[2558]: I0710 18:15:03.448128 2558 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-key-cleanup-29728455-bsrzq" event={"ID":"c69a5e97-ddc4-4e40-8c44-143a97f7e699","Type":"ContainerStarted","Data":"bd197989a20e99996d56c3f0799cee6ac36ed116cf439fcfb26fffbde41f49e6"} Jul 10 18:15:03.466755 ip-10-0-128-39 kubenswrapper[2558]: I0710 18:15:03.466708 2558 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="odh-ai-gateway-infra/maas-api-key-cleanup-29728455-bsrzq" podStartSLOduration=1.5513919409999999 podStartE2EDuration="3.466691232s" totalImagesPullingTime="1.915299291s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-10 18:15:00 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-10 18:15:03.465418079 +0000 UTC m=+731.968910582" watchObservedRunningTime="2026-07-10 18:15:03.466691232 +0000 UTC m=+731.970183740" Jul 10 18:15:10.306304 ip-10-0-128-39 kubenswrapper[2558]: I0710 18:15:10.306267 2558 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-hjf5z"] Jul 10 18:15:19.995497 ip-10-0-128-39 kubenswrapper[2558]: I0710 18:15:19.995459 2558 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-hjf5z"] Jul 10 18:15:30.998953 ip-10-0-128-39 kubenswrapper[2558]: I0710 18:15:30.998917 2558 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-hjf5z"] Jul 10 18:15:41.291919 ip-10-0-128-39 kubenswrapper[2558]: I0710 18:15:41.291882 2558 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-hjf5z"] Jul 10 18:16:00.013200 ip-10-0-128-39 kubenswrapper[2558]: I0710 18:16:00.013165 2558 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["odh-ai-gateway-infra/maas-api-key-cleanup-29728455-bsrzq"] Jul 10 18:16:00.013650 ip-10-0-128-39 kubenswrapper[2558]: I0710 18:16:00.013373 2558 kuberuntime_container.go:873] "Killing container with a grace period" pod="odh-ai-gateway-infra/maas-api-key-cleanup-29728455-bsrzq" podUID="c69a5e97-ddc4-4e40-8c44-143a97f7e699" containerName="cleanup" containerID="cri-o://bd197989a20e99996d56c3f0799cee6ac36ed116cf439fcfb26fffbde41f49e6" gracePeriod=30 Jul 10 18:16:07.392200 ip-10-0-128-39 kubenswrapper[2558]: I0710 18:16:07.392164 2558 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-hjf5z"] Jul 10 18:16:21.788930 ip-10-0-128-39 kubenswrapper[2558]: I0710 18:16:21.788845 2558 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-hjf5z"] Jul 10 18:16:30.194412 ip-10-0-128-39 kubenswrapper[2558]: I0710 18:16:30.194386 2558 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-key-cleanup-29728455-bsrzq" Jul 10 18:16:30.861427 ip-10-0-128-39 kubenswrapper[2558]: I0710 18:16:30.861390 2558 generic.go:356] "Generic (PLEG): container finished" podID="c69a5e97-ddc4-4e40-8c44-143a97f7e699" containerID="bd197989a20e99996d56c3f0799cee6ac36ed116cf439fcfb26fffbde41f49e6" exitCode=137 Jul 10 18:16:30.861427 ip-10-0-128-39 kubenswrapper[2558]: I0710 18:16:30.861431 2558 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-key-cleanup-29728455-bsrzq" Jul 10 18:16:30.861657 ip-10-0-128-39 kubenswrapper[2558]: I0710 18:16:30.861449 2558 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-key-cleanup-29728455-bsrzq" event={"ID":"c69a5e97-ddc4-4e40-8c44-143a97f7e699","Type":"ContainerDied","Data":"bd197989a20e99996d56c3f0799cee6ac36ed116cf439fcfb26fffbde41f49e6"} Jul 10 18:16:30.861657 ip-10-0-128-39 kubenswrapper[2558]: I0710 18:16:30.861502 2558 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-key-cleanup-29728455-bsrzq" event={"ID":"c69a5e97-ddc4-4e40-8c44-143a97f7e699","Type":"ContainerDied","Data":"53652c5fd5b2559fda205aad1876455d05d8f3df685e568911b9a11adc01f43d"} Jul 10 18:16:30.861657 ip-10-0-128-39 kubenswrapper[2558]: I0710 18:16:30.861526 2558 scope.go:117] "RemoveContainer" containerID="bd197989a20e99996d56c3f0799cee6ac36ed116cf439fcfb26fffbde41f49e6" Jul 10 18:16:30.870306 ip-10-0-128-39 kubenswrapper[2558]: I0710 18:16:30.870290 2558 scope.go:117] "RemoveContainer" containerID="bd197989a20e99996d56c3f0799cee6ac36ed116cf439fcfb26fffbde41f49e6" Jul 10 18:16:30.870589 ip-10-0-128-39 kubenswrapper[2558]: E0710 18:16:30.870571 2558 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"bd197989a20e99996d56c3f0799cee6ac36ed116cf439fcfb26fffbde41f49e6\": container with ID starting with bd197989a20e99996d56c3f0799cee6ac36ed116cf439fcfb26fffbde41f49e6 not found: ID does not exist" containerID="bd197989a20e99996d56c3f0799cee6ac36ed116cf439fcfb26fffbde41f49e6" Jul 10 18:16:30.870650 ip-10-0-128-39 kubenswrapper[2558]: I0710 18:16:30.870598 2558 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"bd197989a20e99996d56c3f0799cee6ac36ed116cf439fcfb26fffbde41f49e6"} err="failed to get container status \"bd197989a20e99996d56c3f0799cee6ac36ed116cf439fcfb26fffbde41f49e6\": rpc error: code = NotFound desc = could not find container \"bd197989a20e99996d56c3f0799cee6ac36ed116cf439fcfb26fffbde41f49e6\": container with ID starting with bd197989a20e99996d56c3f0799cee6ac36ed116cf439fcfb26fffbde41f49e6 not found: ID does not exist" Jul 10 18:16:30.882190 ip-10-0-128-39 kubenswrapper[2558]: I0710 18:16:30.882166 2558 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["odh-ai-gateway-infra/maas-api-key-cleanup-29728455-bsrzq"] Jul 10 18:16:30.886696 ip-10-0-128-39 kubenswrapper[2558]: I0710 18:16:30.886678 2558 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["odh-ai-gateway-infra/maas-api-key-cleanup-29728455-bsrzq"] Jul 10 18:16:32.101296 ip-10-0-128-39 kubenswrapper[2558]: I0710 18:16:32.101263 2558 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="c69a5e97-ddc4-4e40-8c44-143a97f7e699" path="/var/lib/kubelet/pods/c69a5e97-ddc4-4e40-8c44-143a97f7e699/volumes" Jul 10 18:16:55.679113 ip-10-0-128-39 kubenswrapper[2558]: I0710 18:16:55.679074 2558 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-hjf5z"] Jul 10 18:17:06.078683 ip-10-0-128-39 kubenswrapper[2558]: I0710 18:17:06.078649 2558 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-hjf5z"] Jul 10 18:17:13.980230 ip-10-0-128-39 kubenswrapper[2558]: I0710 18:17:13.980192 2558 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-hjf5z"] Jul 10 18:17:23.379185 ip-10-0-128-39 kubenswrapper[2558]: I0710 18:17:23.379153 2558 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-hjf5z"] Jul 10 18:17:35.385042 ip-10-0-128-39 kubenswrapper[2558]: I0710 18:17:35.385003 2558 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-hjf5z"] Jul 10 18:17:45.386200 ip-10-0-128-39 kubenswrapper[2558]: I0710 18:17:45.386119 2558 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-hjf5z"] Jul 10 18:17:52.001612 ip-10-0-128-39 kubenswrapper[2558]: I0710 18:17:52.001582 2558 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-console-operator_console-operator-7866497494-26542_2b29fb39-89b3-42c3-b334-3a195242248c/console-operator/1.log" Jul 10 18:17:52.002071 ip-10-0-128-39 kubenswrapper[2558]: I0710 18:17:52.001621 2558 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-console-operator_console-operator-7866497494-26542_2b29fb39-89b3-42c3-b334-3a195242248c/console-operator/1.log" Jul 10 18:17:52.006407 ip-10-0-128-39 kubenswrapper[2558]: I0710 18:17:52.006385 2558 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ovn-kubernetes_ovnkube-node-ll44q_eabc6c69-b1e8-40d5-b41f-f49778a5248c/ovn-acl-logging/0.log" Jul 10 18:17:52.006407 ip-10-0-128-39 kubenswrapper[2558]: I0710 18:17:52.006400 2558 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ovn-kubernetes_ovnkube-node-ll44q_eabc6c69-b1e8-40d5-b41f-f49778a5248c/ovn-acl-logging/0.log" Jul 10 18:17:56.573818 ip-10-0-128-39 kubenswrapper[2558]: I0710 18:17:56.573781 2558 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-hjf5z"] Jul 10 18:18:04.576647 ip-10-0-128-39 kubenswrapper[2558]: I0710 18:18:04.576611 2558 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-hjf5z"] Jul 10 18:18:27.681650 ip-10-0-128-39 kubenswrapper[2558]: I0710 18:18:27.681615 2558 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-hjf5z"] Jul 10 18:18:40.281839 ip-10-0-128-39 kubenswrapper[2558]: I0710 18:18:40.281791 2558 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-hjf5z"] Jul 10 18:19:01.684729 ip-10-0-128-39 kubenswrapper[2558]: I0710 18:19:01.684697 2558 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-hjf5z"] Jul 10 18:19:14.692701 ip-10-0-128-39 kubenswrapper[2558]: I0710 18:19:14.692617 2558 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-hjf5z"] Jul 10 18:19:30.886536 ip-10-0-128-39 kubenswrapper[2558]: I0710 18:19:30.886495 2558 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-hjf5z"] Jul 10 18:19:40.878554 ip-10-0-128-39 kubenswrapper[2558]: I0710 18:19:40.878521 2558 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-hjf5z"] Jul 10 18:19:51.400346 ip-10-0-128-39 kubenswrapper[2558]: I0710 18:19:51.400311 2558 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-hjf5z"] Jul 10 18:20:04.279659 ip-10-0-128-39 kubenswrapper[2558]: I0710 18:20:04.279627 2558 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-hjf5z"] Jul 10 18:21:58.983326 ip-10-0-128-39 kubenswrapper[2558]: I0710 18:21:58.983292 2558 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-hjf5z"] Jul 10 18:22:08.183565 ip-10-0-128-39 kubenswrapper[2558]: I0710 18:22:08.183531 2558 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-hjf5z"] Jul 10 18:22:22.382744 ip-10-0-128-39 kubenswrapper[2558]: I0710 18:22:22.382701 2558 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-hjf5z"] Jul 10 18:22:31.690400 ip-10-0-128-39 kubenswrapper[2558]: I0710 18:22:31.690348 2558 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-hjf5z"] Jul 10 18:22:41.482523 ip-10-0-128-39 kubenswrapper[2558]: I0710 18:22:41.482475 2558 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-hjf5z"] Jul 10 18:22:50.384513 ip-10-0-128-39 kubenswrapper[2558]: I0710 18:22:50.384472 2558 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-hjf5z"] Jul 10 18:22:52.036847 ip-10-0-128-39 kubenswrapper[2558]: I0710 18:22:52.036784 2558 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-console-operator_console-operator-7866497494-26542_2b29fb39-89b3-42c3-b334-3a195242248c/console-operator/1.log" Jul 10 18:22:52.037376 ip-10-0-128-39 kubenswrapper[2558]: I0710 18:22:52.037355 2558 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-console-operator_console-operator-7866497494-26542_2b29fb39-89b3-42c3-b334-3a195242248c/console-operator/1.log" Jul 10 18:22:52.041353 ip-10-0-128-39 kubenswrapper[2558]: I0710 18:22:52.041327 2558 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ovn-kubernetes_ovnkube-node-ll44q_eabc6c69-b1e8-40d5-b41f-f49778a5248c/ovn-acl-logging/0.log" Jul 10 18:22:52.041798 ip-10-0-128-39 kubenswrapper[2558]: I0710 18:22:52.041775 2558 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ovn-kubernetes_ovnkube-node-ll44q_eabc6c69-b1e8-40d5-b41f-f49778a5248c/ovn-acl-logging/0.log" Jul 10 18:23:01.087982 ip-10-0-128-39 kubenswrapper[2558]: I0710 18:23:01.087933 2558 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-hjf5z"] Jul 10 18:23:12.386172 ip-10-0-128-39 kubenswrapper[2558]: I0710 18:23:12.386137 2558 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-hjf5z"] Jul 10 18:23:21.382663 ip-10-0-128-39 kubenswrapper[2558]: I0710 18:23:21.382629 2558 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-hjf5z"] Jul 10 18:23:32.385454 ip-10-0-128-39 kubenswrapper[2558]: I0710 18:23:32.385421 2558 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-hjf5z"] Jul 10 18:23:43.460935 ip-10-0-128-39 kubenswrapper[2558]: I0710 18:23:43.460902 2558 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-hjf5z"] Jul 10 18:23:51.485055 ip-10-0-128-39 kubenswrapper[2558]: I0710 18:23:51.484963 2558 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-hjf5z"] Jul 10 18:24:09.390853 ip-10-0-128-39 kubenswrapper[2558]: I0710 18:24:09.390821 2558 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-hjf5z"] Jul 10 18:24:19.088634 ip-10-0-128-39 kubenswrapper[2558]: I0710 18:24:19.088601 2558 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-hjf5z"] Jul 10 18:24:29.278540 ip-10-0-128-39 kubenswrapper[2558]: I0710 18:24:29.278503 2558 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-hjf5z"] Jul 10 18:24:42.680497 ip-10-0-128-39 kubenswrapper[2558]: I0710 18:24:42.680463 2558 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-hjf5z"] Jul 10 18:24:59.283980 ip-10-0-128-39 kubenswrapper[2558]: I0710 18:24:59.283945 2558 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-hjf5z"] Jul 10 18:25:08.875243 ip-10-0-128-39 kubenswrapper[2558]: I0710 18:25:08.875206 2558 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-hjf5z"] Jul 10 18:25:18.485986 ip-10-0-128-39 kubenswrapper[2558]: I0710 18:25:18.485904 2558 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-hjf5z"] Jul 10 18:25:26.990841 ip-10-0-128-39 kubenswrapper[2558]: I0710 18:25:26.990792 2558 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-hjf5z"] Jul 10 18:25:36.786394 ip-10-0-128-39 kubenswrapper[2558]: I0710 18:25:36.786360 2558 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-hjf5z"] Jul 10 18:25:46.683489 ip-10-0-128-39 kubenswrapper[2558]: I0710 18:25:46.683429 2558 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-hjf5z"] Jul 10 18:26:00.176095 ip-10-0-128-39 kubenswrapper[2558]: I0710 18:26:00.176061 2558 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-hjf5z"] Jul 10 18:26:13.480590 ip-10-0-128-39 kubenswrapper[2558]: I0710 18:26:13.480554 2558 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-hjf5z"] Jul 10 18:26:23.282494 ip-10-0-128-39 kubenswrapper[2558]: I0710 18:26:23.282462 2558 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-hjf5z"] Jul 10 18:26:32.885736 ip-10-0-128-39 kubenswrapper[2558]: I0710 18:26:32.885699 2558 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-hjf5z"] Jul 10 18:26:45.776402 ip-10-0-128-39 kubenswrapper[2558]: I0710 18:26:45.776320 2558 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-hjf5z"] Jul 10 18:27:02.685507 ip-10-0-128-39 kubenswrapper[2558]: I0710 18:27:02.685470 2558 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-hjf5z"] Jul 10 18:27:15.187781 ip-10-0-128-39 kubenswrapper[2558]: I0710 18:27:15.187743 2558 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-hjf5z"] Jul 10 18:27:26.778667 ip-10-0-128-39 kubenswrapper[2558]: I0710 18:27:26.778623 2558 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-hjf5z"] Jul 10 18:27:35.782591 ip-10-0-128-39 kubenswrapper[2558]: I0710 18:27:35.782553 2558 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-hjf5z"] Jul 10 18:27:48.485625 ip-10-0-128-39 kubenswrapper[2558]: I0710 18:27:48.485589 2558 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-hjf5z"] Jul 10 18:27:52.062002 ip-10-0-128-39 kubenswrapper[2558]: I0710 18:27:52.061972 2558 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-console-operator_console-operator-7866497494-26542_2b29fb39-89b3-42c3-b334-3a195242248c/console-operator/1.log" Jul 10 18:27:52.064281 ip-10-0-128-39 kubenswrapper[2558]: I0710 18:27:52.064259 2558 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-console-operator_console-operator-7866497494-26542_2b29fb39-89b3-42c3-b334-3a195242248c/console-operator/1.log" Jul 10 18:27:52.066532 ip-10-0-128-39 kubenswrapper[2558]: I0710 18:27:52.066506 2558 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ovn-kubernetes_ovnkube-node-ll44q_eabc6c69-b1e8-40d5-b41f-f49778a5248c/ovn-acl-logging/0.log" Jul 10 18:27:52.068544 ip-10-0-128-39 kubenswrapper[2558]: I0710 18:27:52.068521 2558 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ovn-kubernetes_ovnkube-node-ll44q_eabc6c69-b1e8-40d5-b41f-f49778a5248c/ovn-acl-logging/0.log" Jul 10 18:27:58.285681 ip-10-0-128-39 kubenswrapper[2558]: I0710 18:27:58.285635 2558 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-hjf5z"] Jul 10 18:28:20.984881 ip-10-0-128-39 kubenswrapper[2558]: I0710 18:28:20.984837 2558 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-hjf5z"] Jul 10 18:28:31.086777 ip-10-0-128-39 kubenswrapper[2558]: I0710 18:28:31.086741 2558 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-hjf5z"] Jul 10 18:30:00.152844 ip-10-0-128-39 kubenswrapper[2558]: I0710 18:30:00.152606 2558 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["odh-ai-gateway-infra/maas-api-key-cleanup-29728470-dwvsp"] Jul 10 18:30:00.153377 ip-10-0-128-39 kubenswrapper[2558]: I0710 18:30:00.153121 2558 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="c69a5e97-ddc4-4e40-8c44-143a97f7e699" containerName="cleanup" Jul 10 18:30:00.153377 ip-10-0-128-39 kubenswrapper[2558]: I0710 18:30:00.153138 2558 state_mem.go:107] "Deleted CPUSet assignment" podUID="c69a5e97-ddc4-4e40-8c44-143a97f7e699" containerName="cleanup" Jul 10 18:30:00.153377 ip-10-0-128-39 kubenswrapper[2558]: I0710 18:30:00.153234 2558 memory_manager.go:368] "RemoveStaleState removing state" podUID="c69a5e97-ddc4-4e40-8c44-143a97f7e699" containerName="cleanup" Jul 10 18:30:00.156341 ip-10-0-128-39 kubenswrapper[2558]: I0710 18:30:00.156317 2558 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-key-cleanup-29728470-dwvsp" Jul 10 18:30:00.158916 ip-10-0-128-39 kubenswrapper[2558]: I0710 18:30:00.158894 2558 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"odh-ai-gateway-infra\"/\"maas-api-cleanup-dockercfg-fhgdj\"" Jul 10 18:30:00.159748 ip-10-0-128-39 kubenswrapper[2558]: I0710 18:30:00.159723 2558 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-key-cleanup-29728470-dwvsp"] Jul 10 18:30:00.166608 ip-10-0-128-39 kubenswrapper[2558]: I0710 18:30:00.166589 2558 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-key-cleanup-29728470-dwvsp" Jul 10 18:30:00.301497 ip-10-0-128-39 kubenswrapper[2558]: I0710 18:30:00.299723 2558 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-key-cleanup-29728470-dwvsp"] Jul 10 18:30:00.305232 ip-10-0-128-39 kubenswrapper[2558]: W0710 18:30:00.305206 2558 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod0b35af7b_bbbf_429c_94c4_4fdce6c75d93.slice/crio-6468a745c8aa09a8b5bb2987d68ae6d8f1bf72247753057ab259ad2c5087851f WatchSource:0}: Error finding container 6468a745c8aa09a8b5bb2987d68ae6d8f1bf72247753057ab259ad2c5087851f: Status 404 returned error can't find the container with id 6468a745c8aa09a8b5bb2987d68ae6d8f1bf72247753057ab259ad2c5087851f Jul 10 18:30:00.593733 ip-10-0-128-39 kubenswrapper[2558]: I0710 18:30:00.593687 2558 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-key-cleanup-29728470-dwvsp" event={"ID":"0b35af7b-bbbf-429c-94c4-4fdce6c75d93","Type":"ContainerStarted","Data":"e600656772e124451ec781ae78c1114f82873b35f809b2a0ec1b7abfc85f4604"} Jul 10 18:30:00.593899 ip-10-0-128-39 kubenswrapper[2558]: I0710 18:30:00.593749 2558 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-key-cleanup-29728470-dwvsp" event={"ID":"0b35af7b-bbbf-429c-94c4-4fdce6c75d93","Type":"ContainerStarted","Data":"6468a745c8aa09a8b5bb2987d68ae6d8f1bf72247753057ab259ad2c5087851f"} Jul 10 18:30:00.608930 ip-10-0-128-39 kubenswrapper[2558]: I0710 18:30:00.608880 2558 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="odh-ai-gateway-infra/maas-api-key-cleanup-29728470-dwvsp" podStartSLOduration=0.608867706 podStartE2EDuration="608.867706ms" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-10 18:30:00 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-10 18:30:00.608291048 +0000 UTC m=+1629.111783552" watchObservedRunningTime="2026-07-10 18:30:00.608867706 +0000 UTC m=+1629.112360202" Jul 10 18:30:08.187991 ip-10-0-128-39 kubenswrapper[2558]: I0710 18:30:08.187948 2558 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-hjf5z"] Jul 10 18:30:22.584572 ip-10-0-128-39 kubenswrapper[2558]: I0710 18:30:22.584437 2558 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-hjf5z"] Jul 10 18:30:34.696313 ip-10-0-128-39 kubenswrapper[2558]: I0710 18:30:34.696277 2558 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-hjf5z"] Jul 10 18:30:45.376896 ip-10-0-128-39 kubenswrapper[2558]: I0710 18:30:45.376864 2558 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-hjf5z"] Jul 10 18:31:00.011170 ip-10-0-128-39 kubenswrapper[2558]: I0710 18:31:00.011132 2558 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["odh-ai-gateway-infra/maas-api-key-cleanup-29728470-dwvsp"] Jul 10 18:31:00.011561 ip-10-0-128-39 kubenswrapper[2558]: I0710 18:31:00.011355 2558 kuberuntime_container.go:873] "Killing container with a grace period" pod="odh-ai-gateway-infra/maas-api-key-cleanup-29728470-dwvsp" podUID="0b35af7b-bbbf-429c-94c4-4fdce6c75d93" containerName="cleanup" containerID="cri-o://e600656772e124451ec781ae78c1114f82873b35f809b2a0ec1b7abfc85f4604" gracePeriod=30 Jul 10 18:31:07.697442 ip-10-0-128-39 kubenswrapper[2558]: I0710 18:31:07.697406 2558 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-hjf5z"] Jul 10 18:31:29.790184 ip-10-0-128-39 kubenswrapper[2558]: I0710 18:31:29.790149 2558 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["ai-tenant-e2e-mt-d047b574/e2e-lifecycle-model-d047b574-kserve-767d988d8c-lcncx"] Jul 10 18:31:29.793419 ip-10-0-128-39 kubenswrapper[2558]: I0710 18:31:29.793402 2558 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="ai-tenant-e2e-mt-d047b574/e2e-lifecycle-model-d047b574-kserve-767d988d8c-lcncx" Jul 10 18:31:29.796761 ip-10-0-128-39 kubenswrapper[2558]: I0710 18:31:29.796728 2558 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"ai-tenant-e2e-mt-d047b574\"/\"openshift-service-ca.crt\"" Jul 10 18:31:29.796923 ip-10-0-128-39 kubenswrapper[2558]: I0710 18:31:29.796777 2558 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"ai-tenant-e2e-mt-d047b574\"/\"e2e-lifecycle-model-d047b574-kserve-self-signed-certs\"" Jul 10 18:31:29.796923 ip-10-0-128-39 kubenswrapper[2558]: I0710 18:31:29.796784 2558 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"ai-tenant-e2e-mt-d047b574\"/\"kube-root-ca.crt\"" Jul 10 18:31:29.796923 ip-10-0-128-39 kubenswrapper[2558]: I0710 18:31:29.796734 2558 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"ai-tenant-e2e-mt-d047b574\"/\"default-dockercfg-7nwjv\"" Jul 10 18:31:29.803740 ip-10-0-128-39 kubenswrapper[2558]: I0710 18:31:29.803720 2558 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["ai-tenant-e2e-mt-d047b574/e2e-lifecycle-model-d047b574-kserve-767d988d8c-lcncx"] Jul 10 18:31:29.925156 ip-10-0-128-39 kubenswrapper[2558]: I0710 18:31:29.925119 2558 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/c5c86441-0123-43af-a827-2d7369535cbd-tmp-dir\") pod \"e2e-lifecycle-model-d047b574-kserve-767d988d8c-lcncx\" (UID: \"c5c86441-0123-43af-a827-2d7369535cbd\") " pod="ai-tenant-e2e-mt-d047b574/e2e-lifecycle-model-d047b574-kserve-767d988d8c-lcncx" Jul 10 18:31:29.925156 ip-10-0-128-39 kubenswrapper[2558]: I0710 18:31:29.925155 2558 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/c5c86441-0123-43af-a827-2d7369535cbd-tls-certs\") pod \"e2e-lifecycle-model-d047b574-kserve-767d988d8c-lcncx\" (UID: \"c5c86441-0123-43af-a827-2d7369535cbd\") " pod="ai-tenant-e2e-mt-d047b574/e2e-lifecycle-model-d047b574-kserve-767d988d8c-lcncx" Jul 10 18:31:29.925368 ip-10-0-128-39 kubenswrapper[2558]: I0710 18:31:29.925176 2558 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-ghppr\" (UniqueName: \"kubernetes.io/projected/c5c86441-0123-43af-a827-2d7369535cbd-kube-api-access-ghppr\") pod \"e2e-lifecycle-model-d047b574-kserve-767d988d8c-lcncx\" (UID: \"c5c86441-0123-43af-a827-2d7369535cbd\") " pod="ai-tenant-e2e-mt-d047b574/e2e-lifecycle-model-d047b574-kserve-767d988d8c-lcncx" Jul 10 18:31:29.925368 ip-10-0-128-39 kubenswrapper[2558]: I0710 18:31:29.925247 2558 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/c5c86441-0123-43af-a827-2d7369535cbd-model-cache\") pod \"e2e-lifecycle-model-d047b574-kserve-767d988d8c-lcncx\" (UID: \"c5c86441-0123-43af-a827-2d7369535cbd\") " pod="ai-tenant-e2e-mt-d047b574/e2e-lifecycle-model-d047b574-kserve-767d988d8c-lcncx" Jul 10 18:31:29.925368 ip-10-0-128-39 kubenswrapper[2558]: I0710 18:31:29.925311 2558 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/c5c86441-0123-43af-a827-2d7369535cbd-dshm\") pod \"e2e-lifecycle-model-d047b574-kserve-767d988d8c-lcncx\" (UID: \"c5c86441-0123-43af-a827-2d7369535cbd\") " pod="ai-tenant-e2e-mt-d047b574/e2e-lifecycle-model-d047b574-kserve-767d988d8c-lcncx" Jul 10 18:31:29.925368 ip-10-0-128-39 kubenswrapper[2558]: I0710 18:31:29.925338 2558 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/c5c86441-0123-43af-a827-2d7369535cbd-home\") pod \"e2e-lifecycle-model-d047b574-kserve-767d988d8c-lcncx\" (UID: \"c5c86441-0123-43af-a827-2d7369535cbd\") " pod="ai-tenant-e2e-mt-d047b574/e2e-lifecycle-model-d047b574-kserve-767d988d8c-lcncx" Jul 10 18:31:30.026564 ip-10-0-128-39 kubenswrapper[2558]: I0710 18:31:30.026540 2558 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/c5c86441-0123-43af-a827-2d7369535cbd-dshm\") pod \"e2e-lifecycle-model-d047b574-kserve-767d988d8c-lcncx\" (UID: \"c5c86441-0123-43af-a827-2d7369535cbd\") " pod="ai-tenant-e2e-mt-d047b574/e2e-lifecycle-model-d047b574-kserve-767d988d8c-lcncx" Jul 10 18:31:30.026665 ip-10-0-128-39 kubenswrapper[2558]: I0710 18:31:30.026573 2558 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/c5c86441-0123-43af-a827-2d7369535cbd-home\") pod \"e2e-lifecycle-model-d047b574-kserve-767d988d8c-lcncx\" (UID: \"c5c86441-0123-43af-a827-2d7369535cbd\") " pod="ai-tenant-e2e-mt-d047b574/e2e-lifecycle-model-d047b574-kserve-767d988d8c-lcncx" Jul 10 18:31:30.026665 ip-10-0-128-39 kubenswrapper[2558]: I0710 18:31:30.026611 2558 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/c5c86441-0123-43af-a827-2d7369535cbd-tmp-dir\") pod \"e2e-lifecycle-model-d047b574-kserve-767d988d8c-lcncx\" (UID: \"c5c86441-0123-43af-a827-2d7369535cbd\") " pod="ai-tenant-e2e-mt-d047b574/e2e-lifecycle-model-d047b574-kserve-767d988d8c-lcncx" Jul 10 18:31:30.026764 ip-10-0-128-39 kubenswrapper[2558]: I0710 18:31:30.026718 2558 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/c5c86441-0123-43af-a827-2d7369535cbd-tls-certs\") pod \"e2e-lifecycle-model-d047b574-kserve-767d988d8c-lcncx\" (UID: \"c5c86441-0123-43af-a827-2d7369535cbd\") " pod="ai-tenant-e2e-mt-d047b574/e2e-lifecycle-model-d047b574-kserve-767d988d8c-lcncx" Jul 10 18:31:30.026843 ip-10-0-128-39 kubenswrapper[2558]: I0710 18:31:30.026765 2558 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-ghppr\" (UniqueName: \"kubernetes.io/projected/c5c86441-0123-43af-a827-2d7369535cbd-kube-api-access-ghppr\") pod \"e2e-lifecycle-model-d047b574-kserve-767d988d8c-lcncx\" (UID: \"c5c86441-0123-43af-a827-2d7369535cbd\") " pod="ai-tenant-e2e-mt-d047b574/e2e-lifecycle-model-d047b574-kserve-767d988d8c-lcncx" Jul 10 18:31:30.026843 ip-10-0-128-39 kubenswrapper[2558]: I0710 18:31:30.026822 2558 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/c5c86441-0123-43af-a827-2d7369535cbd-model-cache\") pod \"e2e-lifecycle-model-d047b574-kserve-767d988d8c-lcncx\" (UID: \"c5c86441-0123-43af-a827-2d7369535cbd\") " pod="ai-tenant-e2e-mt-d047b574/e2e-lifecycle-model-d047b574-kserve-767d988d8c-lcncx" Jul 10 18:31:30.026955 ip-10-0-128-39 kubenswrapper[2558]: I0710 18:31:30.026920 2558 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/c5c86441-0123-43af-a827-2d7369535cbd-tmp-dir\") pod \"e2e-lifecycle-model-d047b574-kserve-767d988d8c-lcncx\" (UID: \"c5c86441-0123-43af-a827-2d7369535cbd\") " pod="ai-tenant-e2e-mt-d047b574/e2e-lifecycle-model-d047b574-kserve-767d988d8c-lcncx" Jul 10 18:31:30.027008 ip-10-0-128-39 kubenswrapper[2558]: I0710 18:31:30.026970 2558 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/c5c86441-0123-43af-a827-2d7369535cbd-home\") pod \"e2e-lifecycle-model-d047b574-kserve-767d988d8c-lcncx\" (UID: \"c5c86441-0123-43af-a827-2d7369535cbd\") " pod="ai-tenant-e2e-mt-d047b574/e2e-lifecycle-model-d047b574-kserve-767d988d8c-lcncx" Jul 10 18:31:30.027138 ip-10-0-128-39 kubenswrapper[2558]: I0710 18:31:30.027120 2558 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/c5c86441-0123-43af-a827-2d7369535cbd-model-cache\") pod \"e2e-lifecycle-model-d047b574-kserve-767d988d8c-lcncx\" (UID: \"c5c86441-0123-43af-a827-2d7369535cbd\") " pod="ai-tenant-e2e-mt-d047b574/e2e-lifecycle-model-d047b574-kserve-767d988d8c-lcncx" Jul 10 18:31:30.028894 ip-10-0-128-39 kubenswrapper[2558]: I0710 18:31:30.028874 2558 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/c5c86441-0123-43af-a827-2d7369535cbd-dshm\") pod \"e2e-lifecycle-model-d047b574-kserve-767d988d8c-lcncx\" (UID: \"c5c86441-0123-43af-a827-2d7369535cbd\") " pod="ai-tenant-e2e-mt-d047b574/e2e-lifecycle-model-d047b574-kserve-767d988d8c-lcncx" Jul 10 18:31:30.029044 ip-10-0-128-39 kubenswrapper[2558]: I0710 18:31:30.029027 2558 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/c5c86441-0123-43af-a827-2d7369535cbd-tls-certs\") pod \"e2e-lifecycle-model-d047b574-kserve-767d988d8c-lcncx\" (UID: \"c5c86441-0123-43af-a827-2d7369535cbd\") " pod="ai-tenant-e2e-mt-d047b574/e2e-lifecycle-model-d047b574-kserve-767d988d8c-lcncx" Jul 10 18:31:30.035379 ip-10-0-128-39 kubenswrapper[2558]: I0710 18:31:30.035359 2558 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-ghppr\" (UniqueName: \"kubernetes.io/projected/c5c86441-0123-43af-a827-2d7369535cbd-kube-api-access-ghppr\") pod \"e2e-lifecycle-model-d047b574-kserve-767d988d8c-lcncx\" (UID: \"c5c86441-0123-43af-a827-2d7369535cbd\") " pod="ai-tenant-e2e-mt-d047b574/e2e-lifecycle-model-d047b574-kserve-767d988d8c-lcncx" Jul 10 18:31:30.103008 ip-10-0-128-39 kubenswrapper[2558]: I0710 18:31:30.102960 2558 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="ai-tenant-e2e-mt-d047b574/e2e-lifecycle-model-d047b574-kserve-767d988d8c-lcncx" Jul 10 18:31:30.215363 ip-10-0-128-39 kubenswrapper[2558]: I0710 18:31:30.215330 2558 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-key-cleanup-29728470-dwvsp" Jul 10 18:31:30.254488 ip-10-0-128-39 kubenswrapper[2558]: I0710 18:31:30.254461 2558 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["ai-tenant-e2e-mt-d047b574/e2e-lifecycle-model-d047b574-kserve-767d988d8c-lcncx"] Jul 10 18:31:30.255751 ip-10-0-128-39 kubenswrapper[2558]: W0710 18:31:30.255716 2558 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-besteffort.slice/kubepods-besteffort-podc5c86441_0123_43af_a827_2d7369535cbd.slice/crio-81fa2e9ee3f06c5e4ae38ae7f868cc0b099284433b83cc834c3592cef8af5479 WatchSource:0}: Error finding container 81fa2e9ee3f06c5e4ae38ae7f868cc0b099284433b83cc834c3592cef8af5479: Status 404 returned error can't find the container with id 81fa2e9ee3f06c5e4ae38ae7f868cc0b099284433b83cc834c3592cef8af5479 Jul 10 18:31:31.030229 ip-10-0-128-39 kubenswrapper[2558]: I0710 18:31:31.030198 2558 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="ai-tenant-e2e-mt-d047b574/e2e-lifecycle-model-d047b574-kserve-767d988d8c-lcncx" event={"ID":"c5c86441-0123-43af-a827-2d7369535cbd","Type":"ContainerStarted","Data":"eab5a6e6b4dca0518c158fbc9cd66b129861bdb86ddfce0a74aee8b2d0392652"} Jul 10 18:31:31.030647 ip-10-0-128-39 kubenswrapper[2558]: I0710 18:31:31.030243 2558 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="ai-tenant-e2e-mt-d047b574/e2e-lifecycle-model-d047b574-kserve-767d988d8c-lcncx" event={"ID":"c5c86441-0123-43af-a827-2d7369535cbd","Type":"ContainerStarted","Data":"81fa2e9ee3f06c5e4ae38ae7f868cc0b099284433b83cc834c3592cef8af5479"} Jul 10 18:31:31.031457 ip-10-0-128-39 kubenswrapper[2558]: I0710 18:31:31.031432 2558 generic.go:356] "Generic (PLEG): container finished" podID="0b35af7b-bbbf-429c-94c4-4fdce6c75d93" containerID="e600656772e124451ec781ae78c1114f82873b35f809b2a0ec1b7abfc85f4604" exitCode=137 Jul 10 18:31:31.031580 ip-10-0-128-39 kubenswrapper[2558]: I0710 18:31:31.031471 2558 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-key-cleanup-29728470-dwvsp" Jul 10 18:31:31.031580 ip-10-0-128-39 kubenswrapper[2558]: I0710 18:31:31.031481 2558 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-key-cleanup-29728470-dwvsp" event={"ID":"0b35af7b-bbbf-429c-94c4-4fdce6c75d93","Type":"ContainerDied","Data":"e600656772e124451ec781ae78c1114f82873b35f809b2a0ec1b7abfc85f4604"} Jul 10 18:31:31.031580 ip-10-0-128-39 kubenswrapper[2558]: I0710 18:31:31.031518 2558 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-key-cleanup-29728470-dwvsp" event={"ID":"0b35af7b-bbbf-429c-94c4-4fdce6c75d93","Type":"ContainerDied","Data":"6468a745c8aa09a8b5bb2987d68ae6d8f1bf72247753057ab259ad2c5087851f"} Jul 10 18:31:31.031580 ip-10-0-128-39 kubenswrapper[2558]: I0710 18:31:31.031540 2558 scope.go:117] "RemoveContainer" containerID="e600656772e124451ec781ae78c1114f82873b35f809b2a0ec1b7abfc85f4604" Jul 10 18:31:31.040607 ip-10-0-128-39 kubenswrapper[2558]: I0710 18:31:31.040582 2558 scope.go:117] "RemoveContainer" containerID="e600656772e124451ec781ae78c1114f82873b35f809b2a0ec1b7abfc85f4604" Jul 10 18:31:31.040901 ip-10-0-128-39 kubenswrapper[2558]: E0710 18:31:31.040879 2558 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"e600656772e124451ec781ae78c1114f82873b35f809b2a0ec1b7abfc85f4604\": container with ID starting with e600656772e124451ec781ae78c1114f82873b35f809b2a0ec1b7abfc85f4604 not found: ID does not exist" containerID="e600656772e124451ec781ae78c1114f82873b35f809b2a0ec1b7abfc85f4604" Jul 10 18:31:31.040980 ip-10-0-128-39 kubenswrapper[2558]: I0710 18:31:31.040913 2558 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"e600656772e124451ec781ae78c1114f82873b35f809b2a0ec1b7abfc85f4604"} err="failed to get container status \"e600656772e124451ec781ae78c1114f82873b35f809b2a0ec1b7abfc85f4604\": rpc error: code = NotFound desc = could not find container \"e600656772e124451ec781ae78c1114f82873b35f809b2a0ec1b7abfc85f4604\": container with ID starting with e600656772e124451ec781ae78c1114f82873b35f809b2a0ec1b7abfc85f4604 not found: ID does not exist" Jul 10 18:31:31.051862 ip-10-0-128-39 kubenswrapper[2558]: I0710 18:31:31.051800 2558 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="ai-tenant-e2e-mt-d047b574/e2e-lifecycle-model-d047b574-kserve-767d988d8c-lcncx" podStartSLOduration=2.051784094 podStartE2EDuration="2.051784094s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-10 18:31:29 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-10 18:31:31.050215489 +0000 UTC m=+1719.553707995" watchObservedRunningTime="2026-07-10 18:31:31.051784094 +0000 UTC m=+1719.555276601" Jul 10 18:31:31.064508 ip-10-0-128-39 kubenswrapper[2558]: I0710 18:31:31.064477 2558 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["odh-ai-gateway-infra/maas-api-key-cleanup-29728470-dwvsp"] Jul 10 18:31:31.069759 ip-10-0-128-39 kubenswrapper[2558]: I0710 18:31:31.069736 2558 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["odh-ai-gateway-infra/maas-api-key-cleanup-29728470-dwvsp"] Jul 10 18:31:32.101870 ip-10-0-128-39 kubenswrapper[2558]: I0710 18:31:32.101840 2558 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="0b35af7b-bbbf-429c-94c4-4fdce6c75d93" path="/var/lib/kubelet/pods/0b35af7b-bbbf-429c-94c4-4fdce6c75d93/volumes" Jul 10 18:31:33.480616 ip-10-0-128-39 kubenswrapper[2558]: I0710 18:31:33.480573 2558 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-hjf5z"] Jul 10 18:31:40.103735 ip-10-0-128-39 kubenswrapper[2558]: I0710 18:31:40.103701 2558 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="ai-tenant-e2e-mt-d047b574/e2e-lifecycle-model-d047b574-kserve-767d988d8c-lcncx" Jul 10 18:31:40.104352 ip-10-0-128-39 kubenswrapper[2558]: I0710 18:31:40.103776 2558 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="ai-tenant-e2e-mt-d047b574/e2e-lifecycle-model-d047b574-kserve-767d988d8c-lcncx" Jul 10 18:31:40.116182 ip-10-0-128-39 kubenswrapper[2558]: I0710 18:31:40.116160 2558 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="ai-tenant-e2e-mt-d047b574/e2e-lifecycle-model-d047b574-kserve-767d988d8c-lcncx" Jul 10 18:31:41.090616 ip-10-0-128-39 kubenswrapper[2558]: I0710 18:31:41.090582 2558 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="ai-tenant-e2e-mt-d047b574/e2e-lifecycle-model-d047b574-kserve-767d988d8c-lcncx" Jul 10 18:31:46.783103 ip-10-0-128-39 kubenswrapper[2558]: I0710 18:31:46.783066 2558 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-hjf5z"] Jul 10 18:31:57.890250 ip-10-0-128-39 kubenswrapper[2558]: I0710 18:31:57.890212 2558 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-hjf5z"] Jul 10 18:32:00.223630 ip-10-0-128-39 kubenswrapper[2558]: I0710 18:32:00.223587 2558 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["ai-tenant-e2e-mt-d047b574/e2e-lifecycle-model-d047b574-kserve-767d988d8c-lcncx"] Jul 10 18:32:15.244345 ip-10-0-128-39 kubenswrapper[2558]: I0710 18:32:15.244303 2558 kuberuntime_container.go:873] "Killing container with a grace period" pod="ai-tenant-e2e-mt-d047b574/e2e-lifecycle-model-d047b574-kserve-767d988d8c-lcncx" podUID="c5c86441-0123-43af-a827-2d7369535cbd" containerName="main" containerID="cri-o://eab5a6e6b4dca0518c158fbc9cd66b129861bdb86ddfce0a74aee8b2d0392652" gracePeriod=45 Jul 10 18:32:15.497009 ip-10-0-128-39 kubenswrapper[2558]: I0710 18:32:15.496954 2558 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="ai-tenant-e2e-mt-d047b574/e2e-lifecycle-model-d047b574-kserve-767d988d8c-lcncx" Jul 10 18:32:15.595182 ip-10-0-128-39 kubenswrapper[2558]: I0710 18:32:15.595147 2558 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/c5c86441-0123-43af-a827-2d7369535cbd-tls-certs\") pod \"c5c86441-0123-43af-a827-2d7369535cbd\" (UID: \"c5c86441-0123-43af-a827-2d7369535cbd\") " Jul 10 18:32:15.595344 ip-10-0-128-39 kubenswrapper[2558]: I0710 18:32:15.595211 2558 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/c5c86441-0123-43af-a827-2d7369535cbd-tmp-dir\") pod \"c5c86441-0123-43af-a827-2d7369535cbd\" (UID: \"c5c86441-0123-43af-a827-2d7369535cbd\") " Jul 10 18:32:15.595344 ip-10-0-128-39 kubenswrapper[2558]: I0710 18:32:15.595245 2558 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/c5c86441-0123-43af-a827-2d7369535cbd-home\") pod \"c5c86441-0123-43af-a827-2d7369535cbd\" (UID: \"c5c86441-0123-43af-a827-2d7369535cbd\") " Jul 10 18:32:15.595344 ip-10-0-128-39 kubenswrapper[2558]: I0710 18:32:15.595276 2558 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/c5c86441-0123-43af-a827-2d7369535cbd-dshm\") pod \"c5c86441-0123-43af-a827-2d7369535cbd\" (UID: \"c5c86441-0123-43af-a827-2d7369535cbd\") " Jul 10 18:32:15.595344 ip-10-0-128-39 kubenswrapper[2558]: I0710 18:32:15.595294 2558 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/c5c86441-0123-43af-a827-2d7369535cbd-model-cache\") pod \"c5c86441-0123-43af-a827-2d7369535cbd\" (UID: \"c5c86441-0123-43af-a827-2d7369535cbd\") " Jul 10 18:32:15.595344 ip-10-0-128-39 kubenswrapper[2558]: I0710 18:32:15.595330 2558 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-ghppr\" (UniqueName: \"kubernetes.io/projected/c5c86441-0123-43af-a827-2d7369535cbd-kube-api-access-ghppr\") pod \"c5c86441-0123-43af-a827-2d7369535cbd\" (UID: \"c5c86441-0123-43af-a827-2d7369535cbd\") " Jul 10 18:32:15.595591 ip-10-0-128-39 kubenswrapper[2558]: I0710 18:32:15.595537 2558 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/c5c86441-0123-43af-a827-2d7369535cbd-home" (OuterVolumeSpecName: "home") pod "c5c86441-0123-43af-a827-2d7369535cbd" (UID: "c5c86441-0123-43af-a827-2d7369535cbd"). InnerVolumeSpecName "home". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Jul 10 18:32:15.595591 ip-10-0-128-39 kubenswrapper[2558]: I0710 18:32:15.595548 2558 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/c5c86441-0123-43af-a827-2d7369535cbd-tmp-dir" (OuterVolumeSpecName: "tmp-dir") pod "c5c86441-0123-43af-a827-2d7369535cbd" (UID: "c5c86441-0123-43af-a827-2d7369535cbd"). InnerVolumeSpecName "tmp-dir". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Jul 10 18:32:15.595591 ip-10-0-128-39 kubenswrapper[2558]: I0710 18:32:15.595564 2558 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/c5c86441-0123-43af-a827-2d7369535cbd-model-cache" (OuterVolumeSpecName: "model-cache") pod "c5c86441-0123-43af-a827-2d7369535cbd" (UID: "c5c86441-0123-43af-a827-2d7369535cbd"). InnerVolumeSpecName "model-cache". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Jul 10 18:32:15.595728 ip-10-0-128-39 kubenswrapper[2558]: I0710 18:32:15.595646 2558 reconciler_common.go:299] "Volume detached for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/c5c86441-0123-43af-a827-2d7369535cbd-model-cache\") on node \"ip-10-0-128-39.ec2.internal\" DevicePath \"\"" Jul 10 18:32:15.595728 ip-10-0-128-39 kubenswrapper[2558]: I0710 18:32:15.595664 2558 reconciler_common.go:299] "Volume detached for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/c5c86441-0123-43af-a827-2d7369535cbd-tmp-dir\") on node \"ip-10-0-128-39.ec2.internal\" DevicePath \"\"" Jul 10 18:32:15.595728 ip-10-0-128-39 kubenswrapper[2558]: I0710 18:32:15.595677 2558 reconciler_common.go:299] "Volume detached for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/c5c86441-0123-43af-a827-2d7369535cbd-home\") on node \"ip-10-0-128-39.ec2.internal\" DevicePath \"\"" Jul 10 18:32:15.597380 ip-10-0-128-39 kubenswrapper[2558]: I0710 18:32:15.597355 2558 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/c5c86441-0123-43af-a827-2d7369535cbd-dshm" (OuterVolumeSpecName: "dshm") pod "c5c86441-0123-43af-a827-2d7369535cbd" (UID: "c5c86441-0123-43af-a827-2d7369535cbd"). InnerVolumeSpecName "dshm". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Jul 10 18:32:15.597380 ip-10-0-128-39 kubenswrapper[2558]: I0710 18:32:15.597369 2558 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/c5c86441-0123-43af-a827-2d7369535cbd-tls-certs" (OuterVolumeSpecName: "tls-certs") pod "c5c86441-0123-43af-a827-2d7369535cbd" (UID: "c5c86441-0123-43af-a827-2d7369535cbd"). InnerVolumeSpecName "tls-certs". PluginName "kubernetes.io/secret", VolumeGIDValue "" Jul 10 18:32:15.597537 ip-10-0-128-39 kubenswrapper[2558]: I0710 18:32:15.597410 2558 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/c5c86441-0123-43af-a827-2d7369535cbd-kube-api-access-ghppr" (OuterVolumeSpecName: "kube-api-access-ghppr") pod "c5c86441-0123-43af-a827-2d7369535cbd" (UID: "c5c86441-0123-43af-a827-2d7369535cbd"). InnerVolumeSpecName "kube-api-access-ghppr". PluginName "kubernetes.io/projected", VolumeGIDValue "" Jul 10 18:32:15.696136 ip-10-0-128-39 kubenswrapper[2558]: I0710 18:32:15.696109 2558 reconciler_common.go:299] "Volume detached for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/c5c86441-0123-43af-a827-2d7369535cbd-dshm\") on node \"ip-10-0-128-39.ec2.internal\" DevicePath \"\"" Jul 10 18:32:15.696136 ip-10-0-128-39 kubenswrapper[2558]: I0710 18:32:15.696132 2558 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-ghppr\" (UniqueName: \"kubernetes.io/projected/c5c86441-0123-43af-a827-2d7369535cbd-kube-api-access-ghppr\") on node \"ip-10-0-128-39.ec2.internal\" DevicePath \"\"" Jul 10 18:32:15.696136 ip-10-0-128-39 kubenswrapper[2558]: I0710 18:32:15.696143 2558 reconciler_common.go:299] "Volume detached for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/c5c86441-0123-43af-a827-2d7369535cbd-tls-certs\") on node \"ip-10-0-128-39.ec2.internal\" DevicePath \"\"" Jul 10 18:32:16.237672 ip-10-0-128-39 kubenswrapper[2558]: I0710 18:32:16.237640 2558 generic.go:356] "Generic (PLEG): container finished" podID="c5c86441-0123-43af-a827-2d7369535cbd" containerID="eab5a6e6b4dca0518c158fbc9cd66b129861bdb86ddfce0a74aee8b2d0392652" exitCode=0 Jul 10 18:32:16.237847 ip-10-0-128-39 kubenswrapper[2558]: I0710 18:32:16.237681 2558 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="ai-tenant-e2e-mt-d047b574/e2e-lifecycle-model-d047b574-kserve-767d988d8c-lcncx" Jul 10 18:32:16.237847 ip-10-0-128-39 kubenswrapper[2558]: I0710 18:32:16.237693 2558 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="ai-tenant-e2e-mt-d047b574/e2e-lifecycle-model-d047b574-kserve-767d988d8c-lcncx" event={"ID":"c5c86441-0123-43af-a827-2d7369535cbd","Type":"ContainerDied","Data":"eab5a6e6b4dca0518c158fbc9cd66b129861bdb86ddfce0a74aee8b2d0392652"} Jul 10 18:32:16.237847 ip-10-0-128-39 kubenswrapper[2558]: I0710 18:32:16.237726 2558 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="ai-tenant-e2e-mt-d047b574/e2e-lifecycle-model-d047b574-kserve-767d988d8c-lcncx" event={"ID":"c5c86441-0123-43af-a827-2d7369535cbd","Type":"ContainerDied","Data":"81fa2e9ee3f06c5e4ae38ae7f868cc0b099284433b83cc834c3592cef8af5479"} Jul 10 18:32:16.237847 ip-10-0-128-39 kubenswrapper[2558]: I0710 18:32:16.237747 2558 scope.go:117] "RemoveContainer" containerID="eab5a6e6b4dca0518c158fbc9cd66b129861bdb86ddfce0a74aee8b2d0392652" Jul 10 18:32:16.246338 ip-10-0-128-39 kubenswrapper[2558]: I0710 18:32:16.246159 2558 scope.go:117] "RemoveContainer" containerID="eab5a6e6b4dca0518c158fbc9cd66b129861bdb86ddfce0a74aee8b2d0392652" Jul 10 18:32:16.246584 ip-10-0-128-39 kubenswrapper[2558]: E0710 18:32:16.246460 2558 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"eab5a6e6b4dca0518c158fbc9cd66b129861bdb86ddfce0a74aee8b2d0392652\": container with ID starting with eab5a6e6b4dca0518c158fbc9cd66b129861bdb86ddfce0a74aee8b2d0392652 not found: ID does not exist" containerID="eab5a6e6b4dca0518c158fbc9cd66b129861bdb86ddfce0a74aee8b2d0392652" Jul 10 18:32:16.246584 ip-10-0-128-39 kubenswrapper[2558]: I0710 18:32:16.246487 2558 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"eab5a6e6b4dca0518c158fbc9cd66b129861bdb86ddfce0a74aee8b2d0392652"} err="failed to get container status \"eab5a6e6b4dca0518c158fbc9cd66b129861bdb86ddfce0a74aee8b2d0392652\": rpc error: code = NotFound desc = could not find container \"eab5a6e6b4dca0518c158fbc9cd66b129861bdb86ddfce0a74aee8b2d0392652\": container with ID starting with eab5a6e6b4dca0518c158fbc9cd66b129861bdb86ddfce0a74aee8b2d0392652 not found: ID does not exist" Jul 10 18:32:16.257093 ip-10-0-128-39 kubenswrapper[2558]: I0710 18:32:16.257068 2558 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["ai-tenant-e2e-mt-d047b574/e2e-lifecycle-model-d047b574-kserve-767d988d8c-lcncx"] Jul 10 18:32:16.261291 ip-10-0-128-39 kubenswrapper[2558]: I0710 18:32:16.261267 2558 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["ai-tenant-e2e-mt-d047b574/e2e-lifecycle-model-d047b574-kserve-767d988d8c-lcncx"] Jul 10 18:32:18.100912 ip-10-0-128-39 kubenswrapper[2558]: I0710 18:32:18.100876 2558 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="c5c86441-0123-43af-a827-2d7369535cbd" path="/var/lib/kubelet/pods/c5c86441-0123-43af-a827-2d7369535cbd/volumes" Jul 10 18:32:39.892948 ip-10-0-128-39 kubenswrapper[2558]: I0710 18:32:39.892916 2558 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-hjf5z"] Jul 10 18:32:52.088605 ip-10-0-128-39 kubenswrapper[2558]: I0710 18:32:52.088579 2558 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-console-operator_console-operator-7866497494-26542_2b29fb39-89b3-42c3-b334-3a195242248c/console-operator/1.log" Jul 10 18:32:52.098423 ip-10-0-128-39 kubenswrapper[2558]: I0710 18:32:52.098397 2558 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-console-operator_console-operator-7866497494-26542_2b29fb39-89b3-42c3-b334-3a195242248c/console-operator/1.log" Jul 10 18:32:52.098856 ip-10-0-128-39 kubenswrapper[2558]: I0710 18:32:52.098829 2558 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ovn-kubernetes_ovnkube-node-ll44q_eabc6c69-b1e8-40d5-b41f-f49778a5248c/ovn-acl-logging/0.log" Jul 10 18:32:52.103560 ip-10-0-128-39 kubenswrapper[2558]: I0710 18:32:52.103536 2558 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ovn-kubernetes_ovnkube-node-ll44q_eabc6c69-b1e8-40d5-b41f-f49778a5248c/ovn-acl-logging/0.log" Jul 10 18:33:12.644629 ip-10-0-128-39 kubenswrapper[2558]: I0710 18:33:12.644586 2558 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-hjf5z"] Jul 10 18:33:28.350749 ip-10-0-128-39 kubenswrapper[2558]: I0710 18:33:28.350711 2558 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-hjf5z"] Jul 10 18:34:17.653880 ip-10-0-128-39 kubenswrapper[2558]: I0710 18:34:17.653784 2558 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-ingress/e2e-inf-b-0dfdd8-openshift-default-9d98c65c5-hfvcb"] Jul 10 18:34:17.654313 ip-10-0-128-39 kubenswrapper[2558]: I0710 18:34:17.654297 2558 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="0b35af7b-bbbf-429c-94c4-4fdce6c75d93" containerName="cleanup" Jul 10 18:34:17.654368 ip-10-0-128-39 kubenswrapper[2558]: I0710 18:34:17.654316 2558 state_mem.go:107] "Deleted CPUSet assignment" podUID="0b35af7b-bbbf-429c-94c4-4fdce6c75d93" containerName="cleanup" Jul 10 18:34:17.654368 ip-10-0-128-39 kubenswrapper[2558]: I0710 18:34:17.654331 2558 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="c5c86441-0123-43af-a827-2d7369535cbd" containerName="main" Jul 10 18:34:17.654368 ip-10-0-128-39 kubenswrapper[2558]: I0710 18:34:17.654340 2558 state_mem.go:107] "Deleted CPUSet assignment" podUID="c5c86441-0123-43af-a827-2d7369535cbd" containerName="main" Jul 10 18:34:17.654462 ip-10-0-128-39 kubenswrapper[2558]: I0710 18:34:17.654432 2558 memory_manager.go:368] "RemoveStaleState removing state" podUID="0b35af7b-bbbf-429c-94c4-4fdce6c75d93" containerName="cleanup" Jul 10 18:34:17.654462 ip-10-0-128-39 kubenswrapper[2558]: I0710 18:34:17.654445 2558 memory_manager.go:368] "RemoveStaleState removing state" podUID="c5c86441-0123-43af-a827-2d7369535cbd" containerName="main" Jul 10 18:34:17.657746 ip-10-0-128-39 kubenswrapper[2558]: I0710 18:34:17.657728 2558 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-inf-b-0dfdd8-openshift-default-9d98c65c5-hfvcb" Jul 10 18:34:17.662906 ip-10-0-128-39 kubenswrapper[2558]: I0710 18:34:17.662887 2558 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress\"/\"e2e-inf-b-0dfdd8-openshift-default-dockercfg-ch5mf\"" Jul 10 18:34:17.663256 ip-10-0-128-39 kubenswrapper[2558]: I0710 18:34:17.663236 2558 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ingress\"/\"openshift-gw-ca-root-cert\"" Jul 10 18:34:17.686544 ip-10-0-128-39 kubenswrapper[2558]: I0710 18:34:17.686515 2558 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/e2e-inf-b-0dfdd8-openshift-default-9d98c65c5-hfvcb"] Jul 10 18:34:17.733887 ip-10-0-128-39 kubenswrapper[2558]: I0710 18:34:17.733856 2558 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/ff127468-61cc-4243-b114-9374e2c9e52a-istio-podinfo\") pod \"e2e-inf-b-0dfdd8-openshift-default-9d98c65c5-hfvcb\" (UID: \"ff127468-61cc-4243-b114-9374e2c9e52a\") " pod="openshift-ingress/e2e-inf-b-0dfdd8-openshift-default-9d98c65c5-hfvcb" Jul 10 18:34:17.733887 ip-10-0-128-39 kubenswrapper[2558]: I0710 18:34:17.733887 2558 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-xc4z5\" (UniqueName: \"kubernetes.io/projected/ff127468-61cc-4243-b114-9374e2c9e52a-kube-api-access-xc4z5\") pod \"e2e-inf-b-0dfdd8-openshift-default-9d98c65c5-hfvcb\" (UID: \"ff127468-61cc-4243-b114-9374e2c9e52a\") " pod="openshift-ingress/e2e-inf-b-0dfdd8-openshift-default-9d98c65c5-hfvcb" Jul 10 18:34:17.734089 ip-10-0-128-39 kubenswrapper[2558]: I0710 18:34:17.733914 2558 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/ff127468-61cc-4243-b114-9374e2c9e52a-istio-data\") pod \"e2e-inf-b-0dfdd8-openshift-default-9d98c65c5-hfvcb\" (UID: \"ff127468-61cc-4243-b114-9374e2c9e52a\") " pod="openshift-ingress/e2e-inf-b-0dfdd8-openshift-default-9d98c65c5-hfvcb" Jul 10 18:34:17.734089 ip-10-0-128-39 kubenswrapper[2558]: I0710 18:34:17.734027 2558 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/ff127468-61cc-4243-b114-9374e2c9e52a-workload-certs\") pod \"e2e-inf-b-0dfdd8-openshift-default-9d98c65c5-hfvcb\" (UID: \"ff127468-61cc-4243-b114-9374e2c9e52a\") " pod="openshift-ingress/e2e-inf-b-0dfdd8-openshift-default-9d98c65c5-hfvcb" Jul 10 18:34:17.734089 ip-10-0-128-39 kubenswrapper[2558]: I0710 18:34:17.734068 2558 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/ff127468-61cc-4243-b114-9374e2c9e52a-istio-envoy\") pod \"e2e-inf-b-0dfdd8-openshift-default-9d98c65c5-hfvcb\" (UID: \"ff127468-61cc-4243-b114-9374e2c9e52a\") " pod="openshift-ingress/e2e-inf-b-0dfdd8-openshift-default-9d98c65c5-hfvcb" Jul 10 18:34:17.734182 ip-10-0-128-39 kubenswrapper[2558]: I0710 18:34:17.734102 2558 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/ff127468-61cc-4243-b114-9374e2c9e52a-workload-socket\") pod \"e2e-inf-b-0dfdd8-openshift-default-9d98c65c5-hfvcb\" (UID: \"ff127468-61cc-4243-b114-9374e2c9e52a\") " pod="openshift-ingress/e2e-inf-b-0dfdd8-openshift-default-9d98c65c5-hfvcb" Jul 10 18:34:17.734182 ip-10-0-128-39 kubenswrapper[2558]: I0710 18:34:17.734122 2558 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/ff127468-61cc-4243-b114-9374e2c9e52a-istiod-ca-cert\") pod \"e2e-inf-b-0dfdd8-openshift-default-9d98c65c5-hfvcb\" (UID: \"ff127468-61cc-4243-b114-9374e2c9e52a\") " pod="openshift-ingress/e2e-inf-b-0dfdd8-openshift-default-9d98c65c5-hfvcb" Jul 10 18:34:17.734182 ip-10-0-128-39 kubenswrapper[2558]: I0710 18:34:17.734155 2558 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/ff127468-61cc-4243-b114-9374e2c9e52a-credential-socket\") pod \"e2e-inf-b-0dfdd8-openshift-default-9d98c65c5-hfvcb\" (UID: \"ff127468-61cc-4243-b114-9374e2c9e52a\") " pod="openshift-ingress/e2e-inf-b-0dfdd8-openshift-default-9d98c65c5-hfvcb" Jul 10 18:34:17.734272 ip-10-0-128-39 kubenswrapper[2558]: I0710 18:34:17.734181 2558 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/ff127468-61cc-4243-b114-9374e2c9e52a-istio-token\") pod \"e2e-inf-b-0dfdd8-openshift-default-9d98c65c5-hfvcb\" (UID: \"ff127468-61cc-4243-b114-9374e2c9e52a\") " pod="openshift-ingress/e2e-inf-b-0dfdd8-openshift-default-9d98c65c5-hfvcb" Jul 10 18:34:17.834573 ip-10-0-128-39 kubenswrapper[2558]: I0710 18:34:17.834534 2558 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/ff127468-61cc-4243-b114-9374e2c9e52a-workload-certs\") pod \"e2e-inf-b-0dfdd8-openshift-default-9d98c65c5-hfvcb\" (UID: \"ff127468-61cc-4243-b114-9374e2c9e52a\") " pod="openshift-ingress/e2e-inf-b-0dfdd8-openshift-default-9d98c65c5-hfvcb" Jul 10 18:34:17.834763 ip-10-0-128-39 kubenswrapper[2558]: I0710 18:34:17.834584 2558 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/ff127468-61cc-4243-b114-9374e2c9e52a-istio-envoy\") pod \"e2e-inf-b-0dfdd8-openshift-default-9d98c65c5-hfvcb\" (UID: \"ff127468-61cc-4243-b114-9374e2c9e52a\") " pod="openshift-ingress/e2e-inf-b-0dfdd8-openshift-default-9d98c65c5-hfvcb" Jul 10 18:34:17.834763 ip-10-0-128-39 kubenswrapper[2558]: I0710 18:34:17.834615 2558 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/ff127468-61cc-4243-b114-9374e2c9e52a-workload-socket\") pod \"e2e-inf-b-0dfdd8-openshift-default-9d98c65c5-hfvcb\" (UID: \"ff127468-61cc-4243-b114-9374e2c9e52a\") " pod="openshift-ingress/e2e-inf-b-0dfdd8-openshift-default-9d98c65c5-hfvcb" Jul 10 18:34:17.834763 ip-10-0-128-39 kubenswrapper[2558]: I0710 18:34:17.834642 2558 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/ff127468-61cc-4243-b114-9374e2c9e52a-istiod-ca-cert\") pod \"e2e-inf-b-0dfdd8-openshift-default-9d98c65c5-hfvcb\" (UID: \"ff127468-61cc-4243-b114-9374e2c9e52a\") " pod="openshift-ingress/e2e-inf-b-0dfdd8-openshift-default-9d98c65c5-hfvcb" Jul 10 18:34:17.834763 ip-10-0-128-39 kubenswrapper[2558]: I0710 18:34:17.834690 2558 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/ff127468-61cc-4243-b114-9374e2c9e52a-credential-socket\") pod \"e2e-inf-b-0dfdd8-openshift-default-9d98c65c5-hfvcb\" (UID: \"ff127468-61cc-4243-b114-9374e2c9e52a\") " pod="openshift-ingress/e2e-inf-b-0dfdd8-openshift-default-9d98c65c5-hfvcb" Jul 10 18:34:17.834763 ip-10-0-128-39 kubenswrapper[2558]: I0710 18:34:17.834712 2558 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/ff127468-61cc-4243-b114-9374e2c9e52a-istio-token\") pod \"e2e-inf-b-0dfdd8-openshift-default-9d98c65c5-hfvcb\" (UID: \"ff127468-61cc-4243-b114-9374e2c9e52a\") " pod="openshift-ingress/e2e-inf-b-0dfdd8-openshift-default-9d98c65c5-hfvcb" Jul 10 18:34:17.834763 ip-10-0-128-39 kubenswrapper[2558]: I0710 18:34:17.834761 2558 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/ff127468-61cc-4243-b114-9374e2c9e52a-istio-podinfo\") pod \"e2e-inf-b-0dfdd8-openshift-default-9d98c65c5-hfvcb\" (UID: \"ff127468-61cc-4243-b114-9374e2c9e52a\") " pod="openshift-ingress/e2e-inf-b-0dfdd8-openshift-default-9d98c65c5-hfvcb" Jul 10 18:34:17.835159 ip-10-0-128-39 kubenswrapper[2558]: I0710 18:34:17.834784 2558 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-xc4z5\" (UniqueName: \"kubernetes.io/projected/ff127468-61cc-4243-b114-9374e2c9e52a-kube-api-access-xc4z5\") pod \"e2e-inf-b-0dfdd8-openshift-default-9d98c65c5-hfvcb\" (UID: \"ff127468-61cc-4243-b114-9374e2c9e52a\") " pod="openshift-ingress/e2e-inf-b-0dfdd8-openshift-default-9d98c65c5-hfvcb" Jul 10 18:34:17.835159 ip-10-0-128-39 kubenswrapper[2558]: I0710 18:34:17.834847 2558 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/ff127468-61cc-4243-b114-9374e2c9e52a-istio-data\") pod \"e2e-inf-b-0dfdd8-openshift-default-9d98c65c5-hfvcb\" (UID: \"ff127468-61cc-4243-b114-9374e2c9e52a\") " pod="openshift-ingress/e2e-inf-b-0dfdd8-openshift-default-9d98c65c5-hfvcb" Jul 10 18:34:17.835159 ip-10-0-128-39 kubenswrapper[2558]: I0710 18:34:17.835078 2558 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/ff127468-61cc-4243-b114-9374e2c9e52a-workload-certs\") pod \"e2e-inf-b-0dfdd8-openshift-default-9d98c65c5-hfvcb\" (UID: \"ff127468-61cc-4243-b114-9374e2c9e52a\") " pod="openshift-ingress/e2e-inf-b-0dfdd8-openshift-default-9d98c65c5-hfvcb" Jul 10 18:34:17.835159 ip-10-0-128-39 kubenswrapper[2558]: I0710 18:34:17.835150 2558 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/ff127468-61cc-4243-b114-9374e2c9e52a-workload-socket\") pod \"e2e-inf-b-0dfdd8-openshift-default-9d98c65c5-hfvcb\" (UID: \"ff127468-61cc-4243-b114-9374e2c9e52a\") " pod="openshift-ingress/e2e-inf-b-0dfdd8-openshift-default-9d98c65c5-hfvcb" Jul 10 18:34:17.835366 ip-10-0-128-39 kubenswrapper[2558]: I0710 18:34:17.835185 2558 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/ff127468-61cc-4243-b114-9374e2c9e52a-credential-socket\") pod \"e2e-inf-b-0dfdd8-openshift-default-9d98c65c5-hfvcb\" (UID: \"ff127468-61cc-4243-b114-9374e2c9e52a\") " pod="openshift-ingress/e2e-inf-b-0dfdd8-openshift-default-9d98c65c5-hfvcb" Jul 10 18:34:17.835453 ip-10-0-128-39 kubenswrapper[2558]: I0710 18:34:17.835429 2558 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/ff127468-61cc-4243-b114-9374e2c9e52a-istiod-ca-cert\") pod \"e2e-inf-b-0dfdd8-openshift-default-9d98c65c5-hfvcb\" (UID: \"ff127468-61cc-4243-b114-9374e2c9e52a\") " pod="openshift-ingress/e2e-inf-b-0dfdd8-openshift-default-9d98c65c5-hfvcb" Jul 10 18:34:17.835635 ip-10-0-128-39 kubenswrapper[2558]: I0710 18:34:17.835617 2558 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/ff127468-61cc-4243-b114-9374e2c9e52a-istio-data\") pod \"e2e-inf-b-0dfdd8-openshift-default-9d98c65c5-hfvcb\" (UID: \"ff127468-61cc-4243-b114-9374e2c9e52a\") " pod="openshift-ingress/e2e-inf-b-0dfdd8-openshift-default-9d98c65c5-hfvcb" Jul 10 18:34:17.837168 ip-10-0-128-39 kubenswrapper[2558]: I0710 18:34:17.837143 2558 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/ff127468-61cc-4243-b114-9374e2c9e52a-istio-envoy\") pod \"e2e-inf-b-0dfdd8-openshift-default-9d98c65c5-hfvcb\" (UID: \"ff127468-61cc-4243-b114-9374e2c9e52a\") " pod="openshift-ingress/e2e-inf-b-0dfdd8-openshift-default-9d98c65c5-hfvcb" Jul 10 18:34:17.837394 ip-10-0-128-39 kubenswrapper[2558]: I0710 18:34:17.837374 2558 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/ff127468-61cc-4243-b114-9374e2c9e52a-istio-podinfo\") pod \"e2e-inf-b-0dfdd8-openshift-default-9d98c65c5-hfvcb\" (UID: \"ff127468-61cc-4243-b114-9374e2c9e52a\") " pod="openshift-ingress/e2e-inf-b-0dfdd8-openshift-default-9d98c65c5-hfvcb" Jul 10 18:34:17.848992 ip-10-0-128-39 kubenswrapper[2558]: I0710 18:34:17.848973 2558 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/ff127468-61cc-4243-b114-9374e2c9e52a-istio-token\") pod \"e2e-inf-b-0dfdd8-openshift-default-9d98c65c5-hfvcb\" (UID: \"ff127468-61cc-4243-b114-9374e2c9e52a\") " pod="openshift-ingress/e2e-inf-b-0dfdd8-openshift-default-9d98c65c5-hfvcb" Jul 10 18:34:17.849198 ip-10-0-128-39 kubenswrapper[2558]: I0710 18:34:17.849176 2558 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-xc4z5\" (UniqueName: \"kubernetes.io/projected/ff127468-61cc-4243-b114-9374e2c9e52a-kube-api-access-xc4z5\") pod \"e2e-inf-b-0dfdd8-openshift-default-9d98c65c5-hfvcb\" (UID: \"ff127468-61cc-4243-b114-9374e2c9e52a\") " pod="openshift-ingress/e2e-inf-b-0dfdd8-openshift-default-9d98c65c5-hfvcb" Jul 10 18:34:17.968094 ip-10-0-128-39 kubenswrapper[2558]: I0710 18:34:17.968056 2558 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-inf-b-0dfdd8-openshift-default-9d98c65c5-hfvcb" Jul 10 18:34:18.113691 ip-10-0-128-39 kubenswrapper[2558]: I0710 18:34:18.113658 2558 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/e2e-inf-b-0dfdd8-openshift-default-9d98c65c5-hfvcb"] Jul 10 18:34:18.117517 ip-10-0-128-39 kubenswrapper[2558]: W0710 18:34:18.117486 2558 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podff127468_61cc_4243_b114_9374e2c9e52a.slice/crio-c6dc97d88ded262dc0c14ffd9ac719627cbae5f2d479399681580f647cc65839 WatchSource:0}: Error finding container c6dc97d88ded262dc0c14ffd9ac719627cbae5f2d479399681580f647cc65839: Status 404 returned error can't find the container with id c6dc97d88ded262dc0c14ffd9ac719627cbae5f2d479399681580f647cc65839 Jul 10 18:34:18.119395 ip-10-0-128-39 kubenswrapper[2558]: I0710 18:34:18.119378 2558 provider.go:93] Refreshing cache for provider: *credentialprovider.defaultDockerConfigProvider Jul 10 18:34:18.800991 ip-10-0-128-39 kubenswrapper[2558]: I0710 18:34:18.800957 2558 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-inf-b-0dfdd8-openshift-default-9d98c65c5-hfvcb" event={"ID":"ff127468-61cc-4243-b114-9374e2c9e52a","Type":"ContainerStarted","Data":"c6dc97d88ded262dc0c14ffd9ac719627cbae5f2d479399681580f647cc65839"} Jul 10 18:34:20.735002 ip-10-0-128-39 kubenswrapper[2558]: I0710 18:34:20.734956 2558 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-hjf5z"] Jul 10 18:34:20.815592 ip-10-0-128-39 kubenswrapper[2558]: I0710 18:34:20.815553 2558 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236224Ki","pods":"250"} Jul 10 18:34:20.815667 ip-10-0-128-39 kubenswrapper[2558]: I0710 18:34:20.815631 2558 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236224Ki","pods":"250"} Jul 10 18:34:20.815667 ip-10-0-128-39 kubenswrapper[2558]: I0710 18:34:20.815660 2558 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236224Ki","pods":"250"} Jul 10 18:34:21.817172 ip-10-0-128-39 kubenswrapper[2558]: I0710 18:34:21.817132 2558 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-inf-b-0dfdd8-openshift-default-9d98c65c5-hfvcb" event={"ID":"ff127468-61cc-4243-b114-9374e2c9e52a","Type":"ContainerStarted","Data":"947a89a261812b3767a71a1f8d1385a0c07466f5660074c46a939ab4102ae1a4"} Jul 10 18:34:21.857983 ip-10-0-128-39 kubenswrapper[2558]: I0710 18:34:21.857927 2558 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-ingress/e2e-inf-b-0dfdd8-openshift-default-9d98c65c5-hfvcb" podStartSLOduration=2.162239459 podStartE2EDuration="4.857910894s" totalImagesPullingTime="2.695671435s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-10 18:34:17 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-10 18:34:21.857822076 +0000 UTC m=+1890.361314576" watchObservedRunningTime="2026-07-10 18:34:21.857910894 +0000 UTC m=+1890.361403399" Jul 10 18:34:21.969051 ip-10-0-128-39 kubenswrapper[2558]: I0710 18:34:21.969012 2558 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="openshift-ingress/e2e-inf-b-0dfdd8-openshift-default-9d98c65c5-hfvcb" Jul 10 18:34:21.974175 ip-10-0-128-39 kubenswrapper[2558]: I0710 18:34:21.974151 2558 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="openshift-ingress/e2e-inf-b-0dfdd8-openshift-default-9d98c65c5-hfvcb" Jul 10 18:34:22.821690 ip-10-0-128-39 kubenswrapper[2558]: I0710 18:34:22.821657 2558 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/e2e-inf-b-0dfdd8-openshift-default-9d98c65c5-hfvcb" Jul 10 18:34:22.822649 ip-10-0-128-39 kubenswrapper[2558]: I0710 18:34:22.822631 2558 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/e2e-inf-b-0dfdd8-openshift-default-9d98c65c5-hfvcb" Jul 10 18:34:38.862999 ip-10-0-128-39 kubenswrapper[2558]: I0710 18:34:38.862964 2558 patch_prober.go:28] interesting pod/e2e-inf-b-0dfdd8-openshift-default-9d98c65c5-hfvcb container/istio-proxy namespace/openshift-ingress: Readiness probe status=failure output="Get \"http://10.132.0.37:15021/healthz/ready\": context deadline exceeded (Client.Timeout exceeded while awaiting headers)" start-of-body= Jul 10 18:34:38.863387 ip-10-0-128-39 kubenswrapper[2558]: I0710 18:34:38.863023 2558 prober.go:120] "Probe failed" probeType="Readiness" pod="openshift-ingress/e2e-inf-b-0dfdd8-openshift-default-9d98c65c5-hfvcb" podUID="ff127468-61cc-4243-b114-9374e2c9e52a" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.132.0.37:15021/healthz/ready\": context deadline exceeded (Client.Timeout exceeded while awaiting headers)" Jul 10 18:34:39.472463 ip-10-0-128-39 kubenswrapper[2558]: I0710 18:34:39.472419 2558 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-hjf5z"] Jul 10 18:34:39.909482 ip-10-0-128-39 kubenswrapper[2558]: I0710 18:34:39.909393 2558 generic.go:356] "Generic (PLEG): container finished" podID="ff127468-61cc-4243-b114-9374e2c9e52a" containerID="947a89a261812b3767a71a1f8d1385a0c07466f5660074c46a939ab4102ae1a4" exitCode=0 Jul 10 18:34:39.909967 ip-10-0-128-39 kubenswrapper[2558]: I0710 18:34:39.909471 2558 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-inf-b-0dfdd8-openshift-default-9d98c65c5-hfvcb" event={"ID":"ff127468-61cc-4243-b114-9374e2c9e52a","Type":"ContainerDied","Data":"947a89a261812b3767a71a1f8d1385a0c07466f5660074c46a939ab4102ae1a4"} Jul 10 18:34:39.910030 ip-10-0-128-39 kubenswrapper[2558]: I0710 18:34:39.910019 2558 scope.go:117] "RemoveContainer" containerID="947a89a261812b3767a71a1f8d1385a0c07466f5660074c46a939ab4102ae1a4" Jul 10 18:34:39.911193 ip-10-0-128-39 kubenswrapper[2558]: I0710 18:34:39.911156 2558 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236224Ki","pods":"250"} Jul 10 18:34:39.911308 ip-10-0-128-39 kubenswrapper[2558]: I0710 18:34:39.911277 2558 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236224Ki","pods":"250"} Jul 10 18:34:39.911371 ip-10-0-128-39 kubenswrapper[2558]: I0710 18:34:39.911322 2558 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236224Ki","pods":"250"} Jul 10 18:34:40.916208 ip-10-0-128-39 kubenswrapper[2558]: I0710 18:34:40.916130 2558 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-inf-b-0dfdd8-openshift-default-9d98c65c5-hfvcb" event={"ID":"ff127468-61cc-4243-b114-9374e2c9e52a","Type":"ContainerStarted","Data":"cb8ae0cf4428365fdbfac50870a5203cd0471fb1dcdb0b3799a0a9b40eca138e"} Jul 10 18:34:40.969584 ip-10-0-128-39 kubenswrapper[2558]: I0710 18:34:40.969536 2558 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="openshift-ingress/e2e-inf-b-0dfdd8-openshift-default-9d98c65c5-hfvcb" Jul 10 18:34:40.969788 ip-10-0-128-39 kubenswrapper[2558]: I0710 18:34:40.969679 2558 patch_prober.go:28] interesting pod/e2e-inf-b-0dfdd8-openshift-default-9d98c65c5-hfvcb container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.132.0.37:15021/healthz/ready\": dial tcp 10.132.0.37:15021: connect: connection refused" start-of-body= Jul 10 18:34:40.969788 ip-10-0-128-39 kubenswrapper[2558]: I0710 18:34:40.969737 2558 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-inf-b-0dfdd8-openshift-default-9d98c65c5-hfvcb" podUID="ff127468-61cc-4243-b114-9374e2c9e52a" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.132.0.37:15021/healthz/ready\": dial tcp 10.132.0.37:15021: connect: connection refused" Jul 10 18:34:41.969461 ip-10-0-128-39 kubenswrapper[2558]: I0710 18:34:41.968961 2558 patch_prober.go:28] interesting pod/e2e-inf-b-0dfdd8-openshift-default-9d98c65c5-hfvcb container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.132.0.37:15021/healthz/ready\": dial tcp 10.132.0.37:15021: connect: connection refused" start-of-body= Jul 10 18:34:41.969461 ip-10-0-128-39 kubenswrapper[2558]: I0710 18:34:41.969066 2558 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-inf-b-0dfdd8-openshift-default-9d98c65c5-hfvcb" podUID="ff127468-61cc-4243-b114-9374e2c9e52a" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.132.0.37:15021/healthz/ready\": dial tcp 10.132.0.37:15021: connect: connection refused" Jul 10 18:34:42.969483 ip-10-0-128-39 kubenswrapper[2558]: I0710 18:34:42.969439 2558 patch_prober.go:28] interesting pod/e2e-inf-b-0dfdd8-openshift-default-9d98c65c5-hfvcb container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.132.0.37:15021/healthz/ready\": dial tcp 10.132.0.37:15021: connect: connection refused" start-of-body= Jul 10 18:34:42.970457 ip-10-0-128-39 kubenswrapper[2558]: I0710 18:34:42.970418 2558 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-inf-b-0dfdd8-openshift-default-9d98c65c5-hfvcb" podUID="ff127468-61cc-4243-b114-9374e2c9e52a" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.132.0.37:15021/healthz/ready\": dial tcp 10.132.0.37:15021: connect: connection refused" Jul 10 18:34:43.969427 ip-10-0-128-39 kubenswrapper[2558]: I0710 18:34:43.969377 2558 patch_prober.go:28] interesting pod/e2e-inf-b-0dfdd8-openshift-default-9d98c65c5-hfvcb container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.132.0.37:15021/healthz/ready\": dial tcp 10.132.0.37:15021: connect: connection refused" start-of-body= Jul 10 18:34:43.969640 ip-10-0-128-39 kubenswrapper[2558]: I0710 18:34:43.969451 2558 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-inf-b-0dfdd8-openshift-default-9d98c65c5-hfvcb" podUID="ff127468-61cc-4243-b114-9374e2c9e52a" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.132.0.37:15021/healthz/ready\": dial tcp 10.132.0.37:15021: connect: connection refused" Jul 10 18:34:44.972652 ip-10-0-128-39 kubenswrapper[2558]: I0710 18:34:44.972622 2558 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="openshift-ingress/e2e-inf-b-0dfdd8-openshift-default-9d98c65c5-hfvcb" Jul 10 18:34:44.973129 ip-10-0-128-39 kubenswrapper[2558]: I0710 18:34:44.973007 2558 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/e2e-inf-b-0dfdd8-openshift-default-9d98c65c5-hfvcb" Jul 10 18:34:44.979571 ip-10-0-128-39 kubenswrapper[2558]: I0710 18:34:44.979541 2558 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/e2e-inf-b-0dfdd8-openshift-default-9d98c65c5-hfvcb" Jul 10 18:34:50.126599 ip-10-0-128-39 kubenswrapper[2558]: I0710 18:34:50.126561 2558 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-hjf5z"] Jul 10 18:35:20.597620 ip-10-0-128-39 kubenswrapper[2558]: I0710 18:35:20.597586 2558 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["ai-tenant-e2e-inf-b-0dfdd8/test-model-0dfdd8-kserve-76646876b5-hm2kn"] Jul 10 18:35:20.601172 ip-10-0-128-39 kubenswrapper[2558]: I0710 18:35:20.601151 2558 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="ai-tenant-e2e-inf-b-0dfdd8/test-model-0dfdd8-kserve-76646876b5-hm2kn" Jul 10 18:35:20.606316 ip-10-0-128-39 kubenswrapper[2558]: I0710 18:35:20.606292 2558 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"ai-tenant-e2e-inf-b-0dfdd8\"/\"default-dockercfg-ft6pq\"" Jul 10 18:35:20.609453 ip-10-0-128-39 kubenswrapper[2558]: I0710 18:35:20.609436 2558 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"ai-tenant-e2e-inf-b-0dfdd8\"/\"test-model-0dfdd8-kserve-self-signed-certs\"" Jul 10 18:35:20.609629 ip-10-0-128-39 kubenswrapper[2558]: I0710 18:35:20.609612 2558 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"ai-tenant-e2e-inf-b-0dfdd8\"/\"openshift-service-ca.crt\"" Jul 10 18:35:20.609932 ip-10-0-128-39 kubenswrapper[2558]: I0710 18:35:20.609919 2558 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"ai-tenant-e2e-inf-b-0dfdd8\"/\"kube-root-ca.crt\"" Jul 10 18:35:20.639284 ip-10-0-128-39 kubenswrapper[2558]: I0710 18:35:20.639259 2558 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["ai-tenant-e2e-inf-b-0dfdd8/test-model-0dfdd8-kserve-76646876b5-hm2kn"] Jul 10 18:35:20.652918 ip-10-0-128-39 kubenswrapper[2558]: I0710 18:35:20.652887 2558 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/19f5aecc-2f28-4251-a926-89d1f8edcc6e-tls-certs\") pod \"test-model-0dfdd8-kserve-76646876b5-hm2kn\" (UID: \"19f5aecc-2f28-4251-a926-89d1f8edcc6e\") " pod="ai-tenant-e2e-inf-b-0dfdd8/test-model-0dfdd8-kserve-76646876b5-hm2kn" Jul 10 18:35:20.652918 ip-10-0-128-39 kubenswrapper[2558]: I0710 18:35:20.652921 2558 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/19f5aecc-2f28-4251-a926-89d1f8edcc6e-tmp-dir\") pod \"test-model-0dfdd8-kserve-76646876b5-hm2kn\" (UID: \"19f5aecc-2f28-4251-a926-89d1f8edcc6e\") " pod="ai-tenant-e2e-inf-b-0dfdd8/test-model-0dfdd8-kserve-76646876b5-hm2kn" Jul 10 18:35:20.653108 ip-10-0-128-39 kubenswrapper[2558]: I0710 18:35:20.652950 2558 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/19f5aecc-2f28-4251-a926-89d1f8edcc6e-model-cache\") pod \"test-model-0dfdd8-kserve-76646876b5-hm2kn\" (UID: \"19f5aecc-2f28-4251-a926-89d1f8edcc6e\") " pod="ai-tenant-e2e-inf-b-0dfdd8/test-model-0dfdd8-kserve-76646876b5-hm2kn" Jul 10 18:35:20.653108 ip-10-0-128-39 kubenswrapper[2558]: I0710 18:35:20.653032 2558 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-82wfm\" (UniqueName: \"kubernetes.io/projected/19f5aecc-2f28-4251-a926-89d1f8edcc6e-kube-api-access-82wfm\") pod \"test-model-0dfdd8-kserve-76646876b5-hm2kn\" (UID: \"19f5aecc-2f28-4251-a926-89d1f8edcc6e\") " pod="ai-tenant-e2e-inf-b-0dfdd8/test-model-0dfdd8-kserve-76646876b5-hm2kn" Jul 10 18:35:20.653108 ip-10-0-128-39 kubenswrapper[2558]: I0710 18:35:20.653066 2558 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/19f5aecc-2f28-4251-a926-89d1f8edcc6e-dshm\") pod \"test-model-0dfdd8-kserve-76646876b5-hm2kn\" (UID: \"19f5aecc-2f28-4251-a926-89d1f8edcc6e\") " pod="ai-tenant-e2e-inf-b-0dfdd8/test-model-0dfdd8-kserve-76646876b5-hm2kn" Jul 10 18:35:20.653209 ip-10-0-128-39 kubenswrapper[2558]: I0710 18:35:20.653119 2558 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/19f5aecc-2f28-4251-a926-89d1f8edcc6e-home\") pod \"test-model-0dfdd8-kserve-76646876b5-hm2kn\" (UID: \"19f5aecc-2f28-4251-a926-89d1f8edcc6e\") " pod="ai-tenant-e2e-inf-b-0dfdd8/test-model-0dfdd8-kserve-76646876b5-hm2kn" Jul 10 18:35:20.754165 ip-10-0-128-39 kubenswrapper[2558]: I0710 18:35:20.754128 2558 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/19f5aecc-2f28-4251-a926-89d1f8edcc6e-model-cache\") pod \"test-model-0dfdd8-kserve-76646876b5-hm2kn\" (UID: \"19f5aecc-2f28-4251-a926-89d1f8edcc6e\") " pod="ai-tenant-e2e-inf-b-0dfdd8/test-model-0dfdd8-kserve-76646876b5-hm2kn" Jul 10 18:35:20.754364 ip-10-0-128-39 kubenswrapper[2558]: I0710 18:35:20.754172 2558 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-82wfm\" (UniqueName: \"kubernetes.io/projected/19f5aecc-2f28-4251-a926-89d1f8edcc6e-kube-api-access-82wfm\") pod \"test-model-0dfdd8-kserve-76646876b5-hm2kn\" (UID: \"19f5aecc-2f28-4251-a926-89d1f8edcc6e\") " pod="ai-tenant-e2e-inf-b-0dfdd8/test-model-0dfdd8-kserve-76646876b5-hm2kn" Jul 10 18:35:20.754364 ip-10-0-128-39 kubenswrapper[2558]: I0710 18:35:20.754200 2558 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/19f5aecc-2f28-4251-a926-89d1f8edcc6e-dshm\") pod \"test-model-0dfdd8-kserve-76646876b5-hm2kn\" (UID: \"19f5aecc-2f28-4251-a926-89d1f8edcc6e\") " pod="ai-tenant-e2e-inf-b-0dfdd8/test-model-0dfdd8-kserve-76646876b5-hm2kn" Jul 10 18:35:20.754364 ip-10-0-128-39 kubenswrapper[2558]: I0710 18:35:20.754241 2558 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/19f5aecc-2f28-4251-a926-89d1f8edcc6e-home\") pod \"test-model-0dfdd8-kserve-76646876b5-hm2kn\" (UID: \"19f5aecc-2f28-4251-a926-89d1f8edcc6e\") " pod="ai-tenant-e2e-inf-b-0dfdd8/test-model-0dfdd8-kserve-76646876b5-hm2kn" Jul 10 18:35:20.754364 ip-10-0-128-39 kubenswrapper[2558]: I0710 18:35:20.754280 2558 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/19f5aecc-2f28-4251-a926-89d1f8edcc6e-tls-certs\") pod \"test-model-0dfdd8-kserve-76646876b5-hm2kn\" (UID: \"19f5aecc-2f28-4251-a926-89d1f8edcc6e\") " pod="ai-tenant-e2e-inf-b-0dfdd8/test-model-0dfdd8-kserve-76646876b5-hm2kn" Jul 10 18:35:20.754364 ip-10-0-128-39 kubenswrapper[2558]: I0710 18:35:20.754300 2558 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/19f5aecc-2f28-4251-a926-89d1f8edcc6e-tmp-dir\") pod \"test-model-0dfdd8-kserve-76646876b5-hm2kn\" (UID: \"19f5aecc-2f28-4251-a926-89d1f8edcc6e\") " pod="ai-tenant-e2e-inf-b-0dfdd8/test-model-0dfdd8-kserve-76646876b5-hm2kn" Jul 10 18:35:20.754621 ip-10-0-128-39 kubenswrapper[2558]: I0710 18:35:20.754598 2558 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/19f5aecc-2f28-4251-a926-89d1f8edcc6e-model-cache\") pod \"test-model-0dfdd8-kserve-76646876b5-hm2kn\" (UID: \"19f5aecc-2f28-4251-a926-89d1f8edcc6e\") " pod="ai-tenant-e2e-inf-b-0dfdd8/test-model-0dfdd8-kserve-76646876b5-hm2kn" Jul 10 18:35:20.754677 ip-10-0-128-39 kubenswrapper[2558]: I0710 18:35:20.754659 2558 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/19f5aecc-2f28-4251-a926-89d1f8edcc6e-home\") pod \"test-model-0dfdd8-kserve-76646876b5-hm2kn\" (UID: \"19f5aecc-2f28-4251-a926-89d1f8edcc6e\") " pod="ai-tenant-e2e-inf-b-0dfdd8/test-model-0dfdd8-kserve-76646876b5-hm2kn" Jul 10 18:35:20.754717 ip-10-0-128-39 kubenswrapper[2558]: I0710 18:35:20.754701 2558 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/19f5aecc-2f28-4251-a926-89d1f8edcc6e-tmp-dir\") pod \"test-model-0dfdd8-kserve-76646876b5-hm2kn\" (UID: \"19f5aecc-2f28-4251-a926-89d1f8edcc6e\") " pod="ai-tenant-e2e-inf-b-0dfdd8/test-model-0dfdd8-kserve-76646876b5-hm2kn" Jul 10 18:35:20.756427 ip-10-0-128-39 kubenswrapper[2558]: I0710 18:35:20.756402 2558 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/19f5aecc-2f28-4251-a926-89d1f8edcc6e-dshm\") pod \"test-model-0dfdd8-kserve-76646876b5-hm2kn\" (UID: \"19f5aecc-2f28-4251-a926-89d1f8edcc6e\") " pod="ai-tenant-e2e-inf-b-0dfdd8/test-model-0dfdd8-kserve-76646876b5-hm2kn" Jul 10 18:35:20.756740 ip-10-0-128-39 kubenswrapper[2558]: I0710 18:35:20.756724 2558 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/19f5aecc-2f28-4251-a926-89d1f8edcc6e-tls-certs\") pod \"test-model-0dfdd8-kserve-76646876b5-hm2kn\" (UID: \"19f5aecc-2f28-4251-a926-89d1f8edcc6e\") " pod="ai-tenant-e2e-inf-b-0dfdd8/test-model-0dfdd8-kserve-76646876b5-hm2kn" Jul 10 18:35:20.765170 ip-10-0-128-39 kubenswrapper[2558]: I0710 18:35:20.765145 2558 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-82wfm\" (UniqueName: \"kubernetes.io/projected/19f5aecc-2f28-4251-a926-89d1f8edcc6e-kube-api-access-82wfm\") pod \"test-model-0dfdd8-kserve-76646876b5-hm2kn\" (UID: \"19f5aecc-2f28-4251-a926-89d1f8edcc6e\") " pod="ai-tenant-e2e-inf-b-0dfdd8/test-model-0dfdd8-kserve-76646876b5-hm2kn" Jul 10 18:35:20.910199 ip-10-0-128-39 kubenswrapper[2558]: I0710 18:35:20.910107 2558 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="ai-tenant-e2e-inf-b-0dfdd8/test-model-0dfdd8-kserve-76646876b5-hm2kn" Jul 10 18:35:21.061135 ip-10-0-128-39 kubenswrapper[2558]: I0710 18:35:21.061106 2558 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["ai-tenant-e2e-inf-b-0dfdd8/test-model-0dfdd8-kserve-76646876b5-hm2kn"] Jul 10 18:35:21.063774 ip-10-0-128-39 kubenswrapper[2558]: W0710 18:35:21.063747 2558 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-besteffort.slice/kubepods-besteffort-pod19f5aecc_2f28_4251_a926_89d1f8edcc6e.slice/crio-aabebd4659cbcd30d95f34214b1d81b12da2dd089968a482564442edaa5e16f8 WatchSource:0}: Error finding container aabebd4659cbcd30d95f34214b1d81b12da2dd089968a482564442edaa5e16f8: Status 404 returned error can't find the container with id aabebd4659cbcd30d95f34214b1d81b12da2dd089968a482564442edaa5e16f8 Jul 10 18:35:21.106989 ip-10-0-128-39 kubenswrapper[2558]: I0710 18:35:21.106962 2558 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="ai-tenant-e2e-inf-b-0dfdd8/test-model-0dfdd8-kserve-76646876b5-hm2kn" event={"ID":"19f5aecc-2f28-4251-a926-89d1f8edcc6e","Type":"ContainerStarted","Data":"aabebd4659cbcd30d95f34214b1d81b12da2dd089968a482564442edaa5e16f8"} Jul 10 18:35:22.112006 ip-10-0-128-39 kubenswrapper[2558]: I0710 18:35:22.111967 2558 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="ai-tenant-e2e-inf-b-0dfdd8/test-model-0dfdd8-kserve-76646876b5-hm2kn" event={"ID":"19f5aecc-2f28-4251-a926-89d1f8edcc6e","Type":"ContainerStarted","Data":"f47ef5bbfba42d52d518a5b718952e137905e0be66df8cfa10e56523c44289e3"} Jul 10 18:35:22.151736 ip-10-0-128-39 kubenswrapper[2558]: I0710 18:35:22.151674 2558 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="ai-tenant-e2e-inf-b-0dfdd8/test-model-0dfdd8-kserve-76646876b5-hm2kn" podStartSLOduration=2.1516554 podStartE2EDuration="2.1516554s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-10 18:35:20 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-10 18:35:22.15159781 +0000 UTC m=+1950.655090316" watchObservedRunningTime="2026-07-10 18:35:22.1516554 +0000 UTC m=+1950.655147906" Jul 10 18:35:26.769874 ip-10-0-128-39 kubenswrapper[2558]: I0710 18:35:26.769784 2558 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-hjf5z"] Jul 10 18:35:30.910964 ip-10-0-128-39 kubenswrapper[2558]: I0710 18:35:30.910922 2558 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="ai-tenant-e2e-inf-b-0dfdd8/test-model-0dfdd8-kserve-76646876b5-hm2kn" Jul 10 18:35:30.911458 ip-10-0-128-39 kubenswrapper[2558]: I0710 18:35:30.910979 2558 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="ai-tenant-e2e-inf-b-0dfdd8/test-model-0dfdd8-kserve-76646876b5-hm2kn" Jul 10 18:35:30.923527 ip-10-0-128-39 kubenswrapper[2558]: I0710 18:35:30.923498 2558 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="ai-tenant-e2e-inf-b-0dfdd8/test-model-0dfdd8-kserve-76646876b5-hm2kn" Jul 10 18:35:31.166392 ip-10-0-128-39 kubenswrapper[2558]: I0710 18:35:31.166288 2558 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="ai-tenant-e2e-inf-b-0dfdd8/test-model-0dfdd8-kserve-76646876b5-hm2kn" Jul 10 18:35:49.089097 ip-10-0-128-39 kubenswrapper[2558]: I0710 18:35:49.089010 2558 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["ai-tenant-e2e-inf-b-0dfdd8/test-model-0dfdd8-kserve-76646876b5-hm2kn"] Jul 10 18:35:53.104006 ip-10-0-128-39 kubenswrapper[2558]: I0710 18:35:53.103972 2558 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-hjf5z"] Jul 10 18:36:04.109980 ip-10-0-128-39 kubenswrapper[2558]: I0710 18:36:04.109921 2558 kuberuntime_container.go:873] "Killing container with a grace period" pod="ai-tenant-e2e-inf-b-0dfdd8/test-model-0dfdd8-kserve-76646876b5-hm2kn" podUID="19f5aecc-2f28-4251-a926-89d1f8edcc6e" containerName="main" containerID="cri-o://f47ef5bbfba42d52d518a5b718952e137905e0be66df8cfa10e56523c44289e3" gracePeriod=45 Jul 10 18:36:04.323405 ip-10-0-128-39 kubenswrapper[2558]: I0710 18:36:04.323373 2558 generic.go:356] "Generic (PLEG): container finished" podID="19f5aecc-2f28-4251-a926-89d1f8edcc6e" containerID="f47ef5bbfba42d52d518a5b718952e137905e0be66df8cfa10e56523c44289e3" exitCode=0 Jul 10 18:36:04.323554 ip-10-0-128-39 kubenswrapper[2558]: I0710 18:36:04.323443 2558 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="ai-tenant-e2e-inf-b-0dfdd8/test-model-0dfdd8-kserve-76646876b5-hm2kn" event={"ID":"19f5aecc-2f28-4251-a926-89d1f8edcc6e","Type":"ContainerDied","Data":"f47ef5bbfba42d52d518a5b718952e137905e0be66df8cfa10e56523c44289e3"} Jul 10 18:36:04.358906 ip-10-0-128-39 kubenswrapper[2558]: I0710 18:36:04.358882 2558 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="ai-tenant-e2e-inf-b-0dfdd8/test-model-0dfdd8-kserve-76646876b5-hm2kn" Jul 10 18:36:04.455877 ip-10-0-128-39 kubenswrapper[2558]: I0710 18:36:04.455780 2558 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/19f5aecc-2f28-4251-a926-89d1f8edcc6e-model-cache\") pod \"19f5aecc-2f28-4251-a926-89d1f8edcc6e\" (UID: \"19f5aecc-2f28-4251-a926-89d1f8edcc6e\") " Jul 10 18:36:04.455877 ip-10-0-128-39 kubenswrapper[2558]: I0710 18:36:04.455851 2558 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/19f5aecc-2f28-4251-a926-89d1f8edcc6e-tmp-dir\") pod \"19f5aecc-2f28-4251-a926-89d1f8edcc6e\" (UID: \"19f5aecc-2f28-4251-a926-89d1f8edcc6e\") " Jul 10 18:36:04.456110 ip-10-0-128-39 kubenswrapper[2558]: I0710 18:36:04.455903 2558 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/19f5aecc-2f28-4251-a926-89d1f8edcc6e-tls-certs\") pod \"19f5aecc-2f28-4251-a926-89d1f8edcc6e\" (UID: \"19f5aecc-2f28-4251-a926-89d1f8edcc6e\") " Jul 10 18:36:04.456110 ip-10-0-128-39 kubenswrapper[2558]: I0710 18:36:04.455942 2558 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-82wfm\" (UniqueName: \"kubernetes.io/projected/19f5aecc-2f28-4251-a926-89d1f8edcc6e-kube-api-access-82wfm\") pod \"19f5aecc-2f28-4251-a926-89d1f8edcc6e\" (UID: \"19f5aecc-2f28-4251-a926-89d1f8edcc6e\") " Jul 10 18:36:04.456110 ip-10-0-128-39 kubenswrapper[2558]: I0710 18:36:04.455981 2558 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/19f5aecc-2f28-4251-a926-89d1f8edcc6e-dshm\") pod \"19f5aecc-2f28-4251-a926-89d1f8edcc6e\" (UID: \"19f5aecc-2f28-4251-a926-89d1f8edcc6e\") " Jul 10 18:36:04.456110 ip-10-0-128-39 kubenswrapper[2558]: I0710 18:36:04.455999 2558 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/19f5aecc-2f28-4251-a926-89d1f8edcc6e-home\") pod \"19f5aecc-2f28-4251-a926-89d1f8edcc6e\" (UID: \"19f5aecc-2f28-4251-a926-89d1f8edcc6e\") " Jul 10 18:36:04.456110 ip-10-0-128-39 kubenswrapper[2558]: I0710 18:36:04.456097 2558 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/19f5aecc-2f28-4251-a926-89d1f8edcc6e-model-cache" (OuterVolumeSpecName: "model-cache") pod "19f5aecc-2f28-4251-a926-89d1f8edcc6e" (UID: "19f5aecc-2f28-4251-a926-89d1f8edcc6e"). InnerVolumeSpecName "model-cache". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Jul 10 18:36:04.456370 ip-10-0-128-39 kubenswrapper[2558]: I0710 18:36:04.456107 2558 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/19f5aecc-2f28-4251-a926-89d1f8edcc6e-tmp-dir" (OuterVolumeSpecName: "tmp-dir") pod "19f5aecc-2f28-4251-a926-89d1f8edcc6e" (UID: "19f5aecc-2f28-4251-a926-89d1f8edcc6e"). InnerVolumeSpecName "tmp-dir". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Jul 10 18:36:04.456370 ip-10-0-128-39 kubenswrapper[2558]: I0710 18:36:04.456210 2558 reconciler_common.go:299] "Volume detached for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/19f5aecc-2f28-4251-a926-89d1f8edcc6e-model-cache\") on node \"ip-10-0-128-39.ec2.internal\" DevicePath \"\"" Jul 10 18:36:04.456370 ip-10-0-128-39 kubenswrapper[2558]: I0710 18:36:04.456224 2558 reconciler_common.go:299] "Volume detached for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/19f5aecc-2f28-4251-a926-89d1f8edcc6e-tmp-dir\") on node \"ip-10-0-128-39.ec2.internal\" DevicePath \"\"" Jul 10 18:36:04.456370 ip-10-0-128-39 kubenswrapper[2558]: I0710 18:36:04.456352 2558 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/19f5aecc-2f28-4251-a926-89d1f8edcc6e-home" (OuterVolumeSpecName: "home") pod "19f5aecc-2f28-4251-a926-89d1f8edcc6e" (UID: "19f5aecc-2f28-4251-a926-89d1f8edcc6e"). InnerVolumeSpecName "home". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Jul 10 18:36:04.458340 ip-10-0-128-39 kubenswrapper[2558]: I0710 18:36:04.458310 2558 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/19f5aecc-2f28-4251-a926-89d1f8edcc6e-tls-certs" (OuterVolumeSpecName: "tls-certs") pod "19f5aecc-2f28-4251-a926-89d1f8edcc6e" (UID: "19f5aecc-2f28-4251-a926-89d1f8edcc6e"). InnerVolumeSpecName "tls-certs". PluginName "kubernetes.io/secret", VolumeGIDValue "" Jul 10 18:36:04.458445 ip-10-0-128-39 kubenswrapper[2558]: I0710 18:36:04.458379 2558 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/19f5aecc-2f28-4251-a926-89d1f8edcc6e-dshm" (OuterVolumeSpecName: "dshm") pod "19f5aecc-2f28-4251-a926-89d1f8edcc6e" (UID: "19f5aecc-2f28-4251-a926-89d1f8edcc6e"). InnerVolumeSpecName "dshm". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Jul 10 18:36:04.458445 ip-10-0-128-39 kubenswrapper[2558]: I0710 18:36:04.458392 2558 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/19f5aecc-2f28-4251-a926-89d1f8edcc6e-kube-api-access-82wfm" (OuterVolumeSpecName: "kube-api-access-82wfm") pod "19f5aecc-2f28-4251-a926-89d1f8edcc6e" (UID: "19f5aecc-2f28-4251-a926-89d1f8edcc6e"). InnerVolumeSpecName "kube-api-access-82wfm". PluginName "kubernetes.io/projected", VolumeGIDValue "" Jul 10 18:36:04.557106 ip-10-0-128-39 kubenswrapper[2558]: I0710 18:36:04.557068 2558 reconciler_common.go:299] "Volume detached for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/19f5aecc-2f28-4251-a926-89d1f8edcc6e-tls-certs\") on node \"ip-10-0-128-39.ec2.internal\" DevicePath \"\"" Jul 10 18:36:04.557106 ip-10-0-128-39 kubenswrapper[2558]: I0710 18:36:04.557096 2558 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-82wfm\" (UniqueName: \"kubernetes.io/projected/19f5aecc-2f28-4251-a926-89d1f8edcc6e-kube-api-access-82wfm\") on node \"ip-10-0-128-39.ec2.internal\" DevicePath \"\"" Jul 10 18:36:04.557106 ip-10-0-128-39 kubenswrapper[2558]: I0710 18:36:04.557106 2558 reconciler_common.go:299] "Volume detached for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/19f5aecc-2f28-4251-a926-89d1f8edcc6e-dshm\") on node \"ip-10-0-128-39.ec2.internal\" DevicePath \"\"" Jul 10 18:36:04.557106 ip-10-0-128-39 kubenswrapper[2558]: I0710 18:36:04.557115 2558 reconciler_common.go:299] "Volume detached for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/19f5aecc-2f28-4251-a926-89d1f8edcc6e-home\") on node \"ip-10-0-128-39.ec2.internal\" DevicePath \"\"" Jul 10 18:36:05.329977 ip-10-0-128-39 kubenswrapper[2558]: I0710 18:36:05.329947 2558 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="ai-tenant-e2e-inf-b-0dfdd8/test-model-0dfdd8-kserve-76646876b5-hm2kn" Jul 10 18:36:05.330459 ip-10-0-128-39 kubenswrapper[2558]: I0710 18:36:05.329978 2558 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="ai-tenant-e2e-inf-b-0dfdd8/test-model-0dfdd8-kserve-76646876b5-hm2kn" event={"ID":"19f5aecc-2f28-4251-a926-89d1f8edcc6e","Type":"ContainerDied","Data":"aabebd4659cbcd30d95f34214b1d81b12da2dd089968a482564442edaa5e16f8"} Jul 10 18:36:05.330459 ip-10-0-128-39 kubenswrapper[2558]: I0710 18:36:05.330029 2558 scope.go:117] "RemoveContainer" containerID="f47ef5bbfba42d52d518a5b718952e137905e0be66df8cfa10e56523c44289e3" Jul 10 18:36:05.355727 ip-10-0-128-39 kubenswrapper[2558]: I0710 18:36:05.355701 2558 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["ai-tenant-e2e-inf-b-0dfdd8/test-model-0dfdd8-kserve-76646876b5-hm2kn"] Jul 10 18:36:05.362762 ip-10-0-128-39 kubenswrapper[2558]: I0710 18:36:05.362735 2558 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["ai-tenant-e2e-inf-b-0dfdd8/test-model-0dfdd8-kserve-76646876b5-hm2kn"] Jul 10 18:36:06.100769 ip-10-0-128-39 kubenswrapper[2558]: I0710 18:36:06.100735 2558 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="19f5aecc-2f28-4251-a926-89d1f8edcc6e" path="/var/lib/kubelet/pods/19f5aecc-2f28-4251-a926-89d1f8edcc6e/volumes" Jul 10 18:36:36.714254 ip-10-0-128-39 kubenswrapper[2558]: I0710 18:36:36.707961 2558 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-hjf5z"] Jul 10 18:36:39.600418 ip-10-0-128-39 kubenswrapper[2558]: I0710 18:36:39.600378 2558 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-ingress/e2e-inf-b-0dfdd8-openshift-default-9d98c65c5-hfvcb"] Jul 10 18:36:39.600830 ip-10-0-128-39 kubenswrapper[2558]: I0710 18:36:39.600638 2558 kuberuntime_container.go:873] "Killing container with a grace period" pod="openshift-ingress/e2e-inf-b-0dfdd8-openshift-default-9d98c65c5-hfvcb" podUID="ff127468-61cc-4243-b114-9374e2c9e52a" containerName="istio-proxy" containerID="cri-o://cb8ae0cf4428365fdbfac50870a5203cd0471fb1dcdb0b3799a0a9b40eca138e" gracePeriod=30 Jul 10 18:36:44.861434 ip-10-0-128-39 kubenswrapper[2558]: I0710 18:36:44.861410 2558 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-inf-b-0dfdd8-openshift-default-9d98c65c5-hfvcb" Jul 10 18:36:44.916553 ip-10-0-128-39 kubenswrapper[2558]: I0710 18:36:44.916463 2558 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/ff127468-61cc-4243-b114-9374e2c9e52a-istio-podinfo\") pod \"ff127468-61cc-4243-b114-9374e2c9e52a\" (UID: \"ff127468-61cc-4243-b114-9374e2c9e52a\") " Jul 10 18:36:44.916553 ip-10-0-128-39 kubenswrapper[2558]: I0710 18:36:44.916499 2558 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/ff127468-61cc-4243-b114-9374e2c9e52a-istio-data\") pod \"ff127468-61cc-4243-b114-9374e2c9e52a\" (UID: \"ff127468-61cc-4243-b114-9374e2c9e52a\") " Jul 10 18:36:44.916553 ip-10-0-128-39 kubenswrapper[2558]: I0710 18:36:44.916519 2558 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/ff127468-61cc-4243-b114-9374e2c9e52a-credential-socket\") pod \"ff127468-61cc-4243-b114-9374e2c9e52a\" (UID: \"ff127468-61cc-4243-b114-9374e2c9e52a\") " Jul 10 18:36:44.916553 ip-10-0-128-39 kubenswrapper[2558]: I0710 18:36:44.916545 2558 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/ff127468-61cc-4243-b114-9374e2c9e52a-workload-certs\") pod \"ff127468-61cc-4243-b114-9374e2c9e52a\" (UID: \"ff127468-61cc-4243-b114-9374e2c9e52a\") " Jul 10 18:36:44.916902 ip-10-0-128-39 kubenswrapper[2558]: I0710 18:36:44.916685 2558 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/ff127468-61cc-4243-b114-9374e2c9e52a-istiod-ca-cert\") pod \"ff127468-61cc-4243-b114-9374e2c9e52a\" (UID: \"ff127468-61cc-4243-b114-9374e2c9e52a\") " Jul 10 18:36:44.916902 ip-10-0-128-39 kubenswrapper[2558]: I0710 18:36:44.916717 2558 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/ff127468-61cc-4243-b114-9374e2c9e52a-istio-token\") pod \"ff127468-61cc-4243-b114-9374e2c9e52a\" (UID: \"ff127468-61cc-4243-b114-9374e2c9e52a\") " Jul 10 18:36:44.916902 ip-10-0-128-39 kubenswrapper[2558]: I0710 18:36:44.916801 2558 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-xc4z5\" (UniqueName: \"kubernetes.io/projected/ff127468-61cc-4243-b114-9374e2c9e52a-kube-api-access-xc4z5\") pod \"ff127468-61cc-4243-b114-9374e2c9e52a\" (UID: \"ff127468-61cc-4243-b114-9374e2c9e52a\") " Jul 10 18:36:44.916902 ip-10-0-128-39 kubenswrapper[2558]: I0710 18:36:44.916859 2558 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/ff127468-61cc-4243-b114-9374e2c9e52a-workload-certs" (OuterVolumeSpecName: "workload-certs") pod "ff127468-61cc-4243-b114-9374e2c9e52a" (UID: "ff127468-61cc-4243-b114-9374e2c9e52a"). InnerVolumeSpecName "workload-certs". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Jul 10 18:36:44.916902 ip-10-0-128-39 kubenswrapper[2558]: I0710 18:36:44.916865 2558 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/ff127468-61cc-4243-b114-9374e2c9e52a-istio-envoy\") pod \"ff127468-61cc-4243-b114-9374e2c9e52a\" (UID: \"ff127468-61cc-4243-b114-9374e2c9e52a\") " Jul 10 18:36:44.917154 ip-10-0-128-39 kubenswrapper[2558]: I0710 18:36:44.916949 2558 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/ff127468-61cc-4243-b114-9374e2c9e52a-workload-socket\") pod \"ff127468-61cc-4243-b114-9374e2c9e52a\" (UID: \"ff127468-61cc-4243-b114-9374e2c9e52a\") " Jul 10 18:36:44.917154 ip-10-0-128-39 kubenswrapper[2558]: I0710 18:36:44.917072 2558 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/ff127468-61cc-4243-b114-9374e2c9e52a-istiod-ca-cert" (OuterVolumeSpecName: "istiod-ca-cert") pod "ff127468-61cc-4243-b114-9374e2c9e52a" (UID: "ff127468-61cc-4243-b114-9374e2c9e52a"). InnerVolumeSpecName "istiod-ca-cert". PluginName "kubernetes.io/configmap", VolumeGIDValue "" Jul 10 18:36:44.917154 ip-10-0-128-39 kubenswrapper[2558]: I0710 18:36:44.917141 2558 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/ff127468-61cc-4243-b114-9374e2c9e52a-credential-socket" (OuterVolumeSpecName: "credential-socket") pod "ff127468-61cc-4243-b114-9374e2c9e52a" (UID: "ff127468-61cc-4243-b114-9374e2c9e52a"). InnerVolumeSpecName "credential-socket". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Jul 10 18:36:44.917308 ip-10-0-128-39 kubenswrapper[2558]: I0710 18:36:44.917291 2558 reconciler_common.go:299] "Volume detached for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/ff127468-61cc-4243-b114-9374e2c9e52a-credential-socket\") on node \"ip-10-0-128-39.ec2.internal\" DevicePath \"\"" Jul 10 18:36:44.917308 ip-10-0-128-39 kubenswrapper[2558]: I0710 18:36:44.917297 2558 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/ff127468-61cc-4243-b114-9374e2c9e52a-workload-socket" (OuterVolumeSpecName: "workload-socket") pod "ff127468-61cc-4243-b114-9374e2c9e52a" (UID: "ff127468-61cc-4243-b114-9374e2c9e52a"). InnerVolumeSpecName "workload-socket". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Jul 10 18:36:44.917403 ip-10-0-128-39 kubenswrapper[2558]: I0710 18:36:44.917308 2558 reconciler_common.go:299] "Volume detached for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/ff127468-61cc-4243-b114-9374e2c9e52a-workload-certs\") on node \"ip-10-0-128-39.ec2.internal\" DevicePath \"\"" Jul 10 18:36:44.917403 ip-10-0-128-39 kubenswrapper[2558]: I0710 18:36:44.917324 2558 reconciler_common.go:299] "Volume detached for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/ff127468-61cc-4243-b114-9374e2c9e52a-istiod-ca-cert\") on node \"ip-10-0-128-39.ec2.internal\" DevicePath \"\"" Jul 10 18:36:44.917505 ip-10-0-128-39 kubenswrapper[2558]: I0710 18:36:44.917434 2558 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/ff127468-61cc-4243-b114-9374e2c9e52a-istio-data" (OuterVolumeSpecName: "istio-data") pod "ff127468-61cc-4243-b114-9374e2c9e52a" (UID: "ff127468-61cc-4243-b114-9374e2c9e52a"). InnerVolumeSpecName "istio-data". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Jul 10 18:36:44.919088 ip-10-0-128-39 kubenswrapper[2558]: I0710 18:36:44.919060 2558 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/ff127468-61cc-4243-b114-9374e2c9e52a-kube-api-access-xc4z5" (OuterVolumeSpecName: "kube-api-access-xc4z5") pod "ff127468-61cc-4243-b114-9374e2c9e52a" (UID: "ff127468-61cc-4243-b114-9374e2c9e52a"). InnerVolumeSpecName "kube-api-access-xc4z5". PluginName "kubernetes.io/projected", VolumeGIDValue "" Jul 10 18:36:44.919205 ip-10-0-128-39 kubenswrapper[2558]: I0710 18:36:44.919085 2558 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/ff127468-61cc-4243-b114-9374e2c9e52a-istio-token" (OuterVolumeSpecName: "istio-token") pod "ff127468-61cc-4243-b114-9374e2c9e52a" (UID: "ff127468-61cc-4243-b114-9374e2c9e52a"). InnerVolumeSpecName "istio-token". PluginName "kubernetes.io/projected", VolumeGIDValue "" Jul 10 18:36:44.919205 ip-10-0-128-39 kubenswrapper[2558]: I0710 18:36:44.919104 2558 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/downward-api/ff127468-61cc-4243-b114-9374e2c9e52a-istio-podinfo" (OuterVolumeSpecName: "istio-podinfo") pod "ff127468-61cc-4243-b114-9374e2c9e52a" (UID: "ff127468-61cc-4243-b114-9374e2c9e52a"). InnerVolumeSpecName "istio-podinfo". PluginName "kubernetes.io/downward-api", VolumeGIDValue "" Jul 10 18:36:44.919205 ip-10-0-128-39 kubenswrapper[2558]: I0710 18:36:44.919117 2558 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/ff127468-61cc-4243-b114-9374e2c9e52a-istio-envoy" (OuterVolumeSpecName: "istio-envoy") pod "ff127468-61cc-4243-b114-9374e2c9e52a" (UID: "ff127468-61cc-4243-b114-9374e2c9e52a"). InnerVolumeSpecName "istio-envoy". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Jul 10 18:36:45.017789 ip-10-0-128-39 kubenswrapper[2558]: I0710 18:36:45.017756 2558 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-xc4z5\" (UniqueName: \"kubernetes.io/projected/ff127468-61cc-4243-b114-9374e2c9e52a-kube-api-access-xc4z5\") on node \"ip-10-0-128-39.ec2.internal\" DevicePath \"\"" Jul 10 18:36:45.017789 ip-10-0-128-39 kubenswrapper[2558]: I0710 18:36:45.017785 2558 reconciler_common.go:299] "Volume detached for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/ff127468-61cc-4243-b114-9374e2c9e52a-istio-envoy\") on node \"ip-10-0-128-39.ec2.internal\" DevicePath \"\"" Jul 10 18:36:45.017789 ip-10-0-128-39 kubenswrapper[2558]: I0710 18:36:45.017794 2558 reconciler_common.go:299] "Volume detached for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/ff127468-61cc-4243-b114-9374e2c9e52a-workload-socket\") on node \"ip-10-0-128-39.ec2.internal\" DevicePath \"\"" Jul 10 18:36:45.018038 ip-10-0-128-39 kubenswrapper[2558]: I0710 18:36:45.017802 2558 reconciler_common.go:299] "Volume detached for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/ff127468-61cc-4243-b114-9374e2c9e52a-istio-podinfo\") on node \"ip-10-0-128-39.ec2.internal\" DevicePath \"\"" Jul 10 18:36:45.018038 ip-10-0-128-39 kubenswrapper[2558]: I0710 18:36:45.017833 2558 reconciler_common.go:299] "Volume detached for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/ff127468-61cc-4243-b114-9374e2c9e52a-istio-data\") on node \"ip-10-0-128-39.ec2.internal\" DevicePath \"\"" Jul 10 18:36:45.018038 ip-10-0-128-39 kubenswrapper[2558]: I0710 18:36:45.017841 2558 reconciler_common.go:299] "Volume detached for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/ff127468-61cc-4243-b114-9374e2c9e52a-istio-token\") on node \"ip-10-0-128-39.ec2.internal\" DevicePath \"\"" Jul 10 18:36:45.517869 ip-10-0-128-39 kubenswrapper[2558]: I0710 18:36:45.517831 2558 generic.go:356] "Generic (PLEG): container finished" podID="ff127468-61cc-4243-b114-9374e2c9e52a" containerID="cb8ae0cf4428365fdbfac50870a5203cd0471fb1dcdb0b3799a0a9b40eca138e" exitCode=0 Jul 10 18:36:45.518034 ip-10-0-128-39 kubenswrapper[2558]: I0710 18:36:45.517886 2558 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-inf-b-0dfdd8-openshift-default-9d98c65c5-hfvcb" Jul 10 18:36:45.518034 ip-10-0-128-39 kubenswrapper[2558]: I0710 18:36:45.517905 2558 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-inf-b-0dfdd8-openshift-default-9d98c65c5-hfvcb" event={"ID":"ff127468-61cc-4243-b114-9374e2c9e52a","Type":"ContainerDied","Data":"cb8ae0cf4428365fdbfac50870a5203cd0471fb1dcdb0b3799a0a9b40eca138e"} Jul 10 18:36:45.518034 ip-10-0-128-39 kubenswrapper[2558]: I0710 18:36:45.517954 2558 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-inf-b-0dfdd8-openshift-default-9d98c65c5-hfvcb" event={"ID":"ff127468-61cc-4243-b114-9374e2c9e52a","Type":"ContainerDied","Data":"c6dc97d88ded262dc0c14ffd9ac719627cbae5f2d479399681580f647cc65839"} Jul 10 18:36:45.518034 ip-10-0-128-39 kubenswrapper[2558]: I0710 18:36:45.517975 2558 scope.go:117] "RemoveContainer" containerID="cb8ae0cf4428365fdbfac50870a5203cd0471fb1dcdb0b3799a0a9b40eca138e" Jul 10 18:36:45.529384 ip-10-0-128-39 kubenswrapper[2558]: I0710 18:36:45.529367 2558 scope.go:117] "RemoveContainer" containerID="947a89a261812b3767a71a1f8d1385a0c07466f5660074c46a939ab4102ae1a4" Jul 10 18:36:45.537910 ip-10-0-128-39 kubenswrapper[2558]: I0710 18:36:45.537891 2558 scope.go:117] "RemoveContainer" containerID="cb8ae0cf4428365fdbfac50870a5203cd0471fb1dcdb0b3799a0a9b40eca138e" Jul 10 18:36:45.538190 ip-10-0-128-39 kubenswrapper[2558]: E0710 18:36:45.538170 2558 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"cb8ae0cf4428365fdbfac50870a5203cd0471fb1dcdb0b3799a0a9b40eca138e\": container with ID starting with cb8ae0cf4428365fdbfac50870a5203cd0471fb1dcdb0b3799a0a9b40eca138e not found: ID does not exist" containerID="cb8ae0cf4428365fdbfac50870a5203cd0471fb1dcdb0b3799a0a9b40eca138e" Jul 10 18:36:45.538235 ip-10-0-128-39 kubenswrapper[2558]: I0710 18:36:45.538201 2558 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"cb8ae0cf4428365fdbfac50870a5203cd0471fb1dcdb0b3799a0a9b40eca138e"} err="failed to get container status \"cb8ae0cf4428365fdbfac50870a5203cd0471fb1dcdb0b3799a0a9b40eca138e\": rpc error: code = NotFound desc = could not find container \"cb8ae0cf4428365fdbfac50870a5203cd0471fb1dcdb0b3799a0a9b40eca138e\": container with ID starting with cb8ae0cf4428365fdbfac50870a5203cd0471fb1dcdb0b3799a0a9b40eca138e not found: ID does not exist" Jul 10 18:36:45.538235 ip-10-0-128-39 kubenswrapper[2558]: I0710 18:36:45.538221 2558 scope.go:117] "RemoveContainer" containerID="947a89a261812b3767a71a1f8d1385a0c07466f5660074c46a939ab4102ae1a4" Jul 10 18:36:45.538442 ip-10-0-128-39 kubenswrapper[2558]: E0710 18:36:45.538424 2558 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"947a89a261812b3767a71a1f8d1385a0c07466f5660074c46a939ab4102ae1a4\": container with ID starting with 947a89a261812b3767a71a1f8d1385a0c07466f5660074c46a939ab4102ae1a4 not found: ID does not exist" containerID="947a89a261812b3767a71a1f8d1385a0c07466f5660074c46a939ab4102ae1a4" Jul 10 18:36:45.538481 ip-10-0-128-39 kubenswrapper[2558]: I0710 18:36:45.538447 2558 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"947a89a261812b3767a71a1f8d1385a0c07466f5660074c46a939ab4102ae1a4"} err="failed to get container status \"947a89a261812b3767a71a1f8d1385a0c07466f5660074c46a939ab4102ae1a4\": rpc error: code = NotFound desc = could not find container \"947a89a261812b3767a71a1f8d1385a0c07466f5660074c46a939ab4102ae1a4\": container with ID starting with 947a89a261812b3767a71a1f8d1385a0c07466f5660074c46a939ab4102ae1a4 not found: ID does not exist" Jul 10 18:36:45.553572 ip-10-0-128-39 kubenswrapper[2558]: I0710 18:36:45.553546 2558 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-ingress/e2e-inf-b-0dfdd8-openshift-default-9d98c65c5-hfvcb"] Jul 10 18:36:45.555084 ip-10-0-128-39 kubenswrapper[2558]: I0710 18:36:45.555064 2558 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["openshift-ingress/e2e-inf-b-0dfdd8-openshift-default-9d98c65c5-hfvcb"] Jul 10 18:36:46.100995 ip-10-0-128-39 kubenswrapper[2558]: I0710 18:36:46.100960 2558 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="ff127468-61cc-4243-b114-9374e2c9e52a" path="/var/lib/kubelet/pods/ff127468-61cc-4243-b114-9374e2c9e52a/volumes" Jul 10 18:36:48.289026 ip-10-0-128-39 kubenswrapper[2558]: I0710 18:36:48.288993 2558 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-ingress/e2e-api-b-171b27-openshift-default-6c8f584f77-rccmt"] Jul 10 18:36:48.289414 ip-10-0-128-39 kubenswrapper[2558]: I0710 18:36:48.289333 2558 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="19f5aecc-2f28-4251-a926-89d1f8edcc6e" containerName="main" Jul 10 18:36:48.289414 ip-10-0-128-39 kubenswrapper[2558]: I0710 18:36:48.289343 2558 state_mem.go:107] "Deleted CPUSet assignment" podUID="19f5aecc-2f28-4251-a926-89d1f8edcc6e" containerName="main" Jul 10 18:36:48.289414 ip-10-0-128-39 kubenswrapper[2558]: I0710 18:36:48.289365 2558 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="ff127468-61cc-4243-b114-9374e2c9e52a" containerName="istio-proxy" Jul 10 18:36:48.289414 ip-10-0-128-39 kubenswrapper[2558]: I0710 18:36:48.289370 2558 state_mem.go:107] "Deleted CPUSet assignment" podUID="ff127468-61cc-4243-b114-9374e2c9e52a" containerName="istio-proxy" Jul 10 18:36:48.289414 ip-10-0-128-39 kubenswrapper[2558]: I0710 18:36:48.289378 2558 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="ff127468-61cc-4243-b114-9374e2c9e52a" containerName="istio-proxy" Jul 10 18:36:48.289414 ip-10-0-128-39 kubenswrapper[2558]: I0710 18:36:48.289385 2558 state_mem.go:107] "Deleted CPUSet assignment" podUID="ff127468-61cc-4243-b114-9374e2c9e52a" containerName="istio-proxy" Jul 10 18:36:48.289627 ip-10-0-128-39 kubenswrapper[2558]: I0710 18:36:48.289444 2558 memory_manager.go:368] "RemoveStaleState removing state" podUID="ff127468-61cc-4243-b114-9374e2c9e52a" containerName="istio-proxy" Jul 10 18:36:48.289627 ip-10-0-128-39 kubenswrapper[2558]: I0710 18:36:48.289453 2558 memory_manager.go:368] "RemoveStaleState removing state" podUID="19f5aecc-2f28-4251-a926-89d1f8edcc6e" containerName="main" Jul 10 18:36:48.293685 ip-10-0-128-39 kubenswrapper[2558]: I0710 18:36:48.293668 2558 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-api-b-171b27-openshift-default-6c8f584f77-rccmt" Jul 10 18:36:48.296196 ip-10-0-128-39 kubenswrapper[2558]: I0710 18:36:48.296171 2558 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ingress\"/\"openshift-gw-ca-root-cert\"" Jul 10 18:36:48.298777 ip-10-0-128-39 kubenswrapper[2558]: I0710 18:36:48.298761 2558 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress\"/\"e2e-api-b-171b27-openshift-default-dockercfg-6clsq\"" Jul 10 18:36:48.322645 ip-10-0-128-39 kubenswrapper[2558]: I0710 18:36:48.322617 2558 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/e2e-api-b-171b27-openshift-default-6c8f584f77-rccmt"] Jul 10 18:36:48.343856 ip-10-0-128-39 kubenswrapper[2558]: I0710 18:36:48.343797 2558 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/1ec52f5a-3489-498e-9265-18fb1c32b6de-workload-certs\") pod \"e2e-api-b-171b27-openshift-default-6c8f584f77-rccmt\" (UID: \"1ec52f5a-3489-498e-9265-18fb1c32b6de\") " pod="openshift-ingress/e2e-api-b-171b27-openshift-default-6c8f584f77-rccmt" Jul 10 18:36:48.343961 ip-10-0-128-39 kubenswrapper[2558]: I0710 18:36:48.343864 2558 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/1ec52f5a-3489-498e-9265-18fb1c32b6de-istio-data\") pod \"e2e-api-b-171b27-openshift-default-6c8f584f77-rccmt\" (UID: \"1ec52f5a-3489-498e-9265-18fb1c32b6de\") " pod="openshift-ingress/e2e-api-b-171b27-openshift-default-6c8f584f77-rccmt" Jul 10 18:36:48.343961 ip-10-0-128-39 kubenswrapper[2558]: I0710 18:36:48.343889 2558 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/1ec52f5a-3489-498e-9265-18fb1c32b6de-istio-podinfo\") pod \"e2e-api-b-171b27-openshift-default-6c8f584f77-rccmt\" (UID: \"1ec52f5a-3489-498e-9265-18fb1c32b6de\") " pod="openshift-ingress/e2e-api-b-171b27-openshift-default-6c8f584f77-rccmt" Jul 10 18:36:48.343961 ip-10-0-128-39 kubenswrapper[2558]: I0710 18:36:48.343934 2558 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/1ec52f5a-3489-498e-9265-18fb1c32b6de-istiod-ca-cert\") pod \"e2e-api-b-171b27-openshift-default-6c8f584f77-rccmt\" (UID: \"1ec52f5a-3489-498e-9265-18fb1c32b6de\") " pod="openshift-ingress/e2e-api-b-171b27-openshift-default-6c8f584f77-rccmt" Jul 10 18:36:48.343961 ip-10-0-128-39 kubenswrapper[2558]: I0710 18:36:48.343958 2558 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/1ec52f5a-3489-498e-9265-18fb1c32b6de-credential-socket\") pod \"e2e-api-b-171b27-openshift-default-6c8f584f77-rccmt\" (UID: \"1ec52f5a-3489-498e-9265-18fb1c32b6de\") " pod="openshift-ingress/e2e-api-b-171b27-openshift-default-6c8f584f77-rccmt" Jul 10 18:36:48.344170 ip-10-0-128-39 kubenswrapper[2558]: I0710 18:36:48.343980 2558 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/1ec52f5a-3489-498e-9265-18fb1c32b6de-istio-token\") pod \"e2e-api-b-171b27-openshift-default-6c8f584f77-rccmt\" (UID: \"1ec52f5a-3489-498e-9265-18fb1c32b6de\") " pod="openshift-ingress/e2e-api-b-171b27-openshift-default-6c8f584f77-rccmt" Jul 10 18:36:48.344170 ip-10-0-128-39 kubenswrapper[2558]: I0710 18:36:48.344050 2558 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/1ec52f5a-3489-498e-9265-18fb1c32b6de-workload-socket\") pod \"e2e-api-b-171b27-openshift-default-6c8f584f77-rccmt\" (UID: \"1ec52f5a-3489-498e-9265-18fb1c32b6de\") " pod="openshift-ingress/e2e-api-b-171b27-openshift-default-6c8f584f77-rccmt" Jul 10 18:36:48.344170 ip-10-0-128-39 kubenswrapper[2558]: I0710 18:36:48.344075 2558 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/1ec52f5a-3489-498e-9265-18fb1c32b6de-istio-envoy\") pod \"e2e-api-b-171b27-openshift-default-6c8f584f77-rccmt\" (UID: \"1ec52f5a-3489-498e-9265-18fb1c32b6de\") " pod="openshift-ingress/e2e-api-b-171b27-openshift-default-6c8f584f77-rccmt" Jul 10 18:36:48.344170 ip-10-0-128-39 kubenswrapper[2558]: I0710 18:36:48.344095 2558 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-7crcl\" (UniqueName: \"kubernetes.io/projected/1ec52f5a-3489-498e-9265-18fb1c32b6de-kube-api-access-7crcl\") pod \"e2e-api-b-171b27-openshift-default-6c8f584f77-rccmt\" (UID: \"1ec52f5a-3489-498e-9265-18fb1c32b6de\") " pod="openshift-ingress/e2e-api-b-171b27-openshift-default-6c8f584f77-rccmt" Jul 10 18:36:48.445223 ip-10-0-128-39 kubenswrapper[2558]: I0710 18:36:48.445180 2558 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/1ec52f5a-3489-498e-9265-18fb1c32b6de-workload-certs\") pod \"e2e-api-b-171b27-openshift-default-6c8f584f77-rccmt\" (UID: \"1ec52f5a-3489-498e-9265-18fb1c32b6de\") " pod="openshift-ingress/e2e-api-b-171b27-openshift-default-6c8f584f77-rccmt" Jul 10 18:36:48.445223 ip-10-0-128-39 kubenswrapper[2558]: I0710 18:36:48.445223 2558 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/1ec52f5a-3489-498e-9265-18fb1c32b6de-istio-data\") pod \"e2e-api-b-171b27-openshift-default-6c8f584f77-rccmt\" (UID: \"1ec52f5a-3489-498e-9265-18fb1c32b6de\") " pod="openshift-ingress/e2e-api-b-171b27-openshift-default-6c8f584f77-rccmt" Jul 10 18:36:48.445474 ip-10-0-128-39 kubenswrapper[2558]: I0710 18:36:48.445249 2558 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/1ec52f5a-3489-498e-9265-18fb1c32b6de-istio-podinfo\") pod \"e2e-api-b-171b27-openshift-default-6c8f584f77-rccmt\" (UID: \"1ec52f5a-3489-498e-9265-18fb1c32b6de\") " pod="openshift-ingress/e2e-api-b-171b27-openshift-default-6c8f584f77-rccmt" Jul 10 18:36:48.445474 ip-10-0-128-39 kubenswrapper[2558]: I0710 18:36:48.445287 2558 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/1ec52f5a-3489-498e-9265-18fb1c32b6de-istiod-ca-cert\") pod \"e2e-api-b-171b27-openshift-default-6c8f584f77-rccmt\" (UID: \"1ec52f5a-3489-498e-9265-18fb1c32b6de\") " pod="openshift-ingress/e2e-api-b-171b27-openshift-default-6c8f584f77-rccmt" Jul 10 18:36:48.445474 ip-10-0-128-39 kubenswrapper[2558]: I0710 18:36:48.445313 2558 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/1ec52f5a-3489-498e-9265-18fb1c32b6de-credential-socket\") pod \"e2e-api-b-171b27-openshift-default-6c8f584f77-rccmt\" (UID: \"1ec52f5a-3489-498e-9265-18fb1c32b6de\") " pod="openshift-ingress/e2e-api-b-171b27-openshift-default-6c8f584f77-rccmt" Jul 10 18:36:48.445474 ip-10-0-128-39 kubenswrapper[2558]: I0710 18:36:48.445345 2558 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/1ec52f5a-3489-498e-9265-18fb1c32b6de-istio-token\") pod \"e2e-api-b-171b27-openshift-default-6c8f584f77-rccmt\" (UID: \"1ec52f5a-3489-498e-9265-18fb1c32b6de\") " pod="openshift-ingress/e2e-api-b-171b27-openshift-default-6c8f584f77-rccmt" Jul 10 18:36:48.445474 ip-10-0-128-39 kubenswrapper[2558]: I0710 18:36:48.445392 2558 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/1ec52f5a-3489-498e-9265-18fb1c32b6de-workload-socket\") pod \"e2e-api-b-171b27-openshift-default-6c8f584f77-rccmt\" (UID: \"1ec52f5a-3489-498e-9265-18fb1c32b6de\") " pod="openshift-ingress/e2e-api-b-171b27-openshift-default-6c8f584f77-rccmt" Jul 10 18:36:48.445474 ip-10-0-128-39 kubenswrapper[2558]: I0710 18:36:48.445426 2558 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/1ec52f5a-3489-498e-9265-18fb1c32b6de-istio-envoy\") pod \"e2e-api-b-171b27-openshift-default-6c8f584f77-rccmt\" (UID: \"1ec52f5a-3489-498e-9265-18fb1c32b6de\") " pod="openshift-ingress/e2e-api-b-171b27-openshift-default-6c8f584f77-rccmt" Jul 10 18:36:48.445474 ip-10-0-128-39 kubenswrapper[2558]: I0710 18:36:48.445457 2558 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-7crcl\" (UniqueName: \"kubernetes.io/projected/1ec52f5a-3489-498e-9265-18fb1c32b6de-kube-api-access-7crcl\") pod \"e2e-api-b-171b27-openshift-default-6c8f584f77-rccmt\" (UID: \"1ec52f5a-3489-498e-9265-18fb1c32b6de\") " pod="openshift-ingress/e2e-api-b-171b27-openshift-default-6c8f584f77-rccmt" Jul 10 18:36:48.445909 ip-10-0-128-39 kubenswrapper[2558]: I0710 18:36:48.445674 2558 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/1ec52f5a-3489-498e-9265-18fb1c32b6de-istio-data\") pod \"e2e-api-b-171b27-openshift-default-6c8f584f77-rccmt\" (UID: \"1ec52f5a-3489-498e-9265-18fb1c32b6de\") " pod="openshift-ingress/e2e-api-b-171b27-openshift-default-6c8f584f77-rccmt" Jul 10 18:36:48.445909 ip-10-0-128-39 kubenswrapper[2558]: I0710 18:36:48.445713 2558 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/1ec52f5a-3489-498e-9265-18fb1c32b6de-credential-socket\") pod \"e2e-api-b-171b27-openshift-default-6c8f584f77-rccmt\" (UID: \"1ec52f5a-3489-498e-9265-18fb1c32b6de\") " pod="openshift-ingress/e2e-api-b-171b27-openshift-default-6c8f584f77-rccmt" Jul 10 18:36:48.446026 ip-10-0-128-39 kubenswrapper[2558]: I0710 18:36:48.446001 2558 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/1ec52f5a-3489-498e-9265-18fb1c32b6de-workload-socket\") pod \"e2e-api-b-171b27-openshift-default-6c8f584f77-rccmt\" (UID: \"1ec52f5a-3489-498e-9265-18fb1c32b6de\") " pod="openshift-ingress/e2e-api-b-171b27-openshift-default-6c8f584f77-rccmt" Jul 10 18:36:48.446084 ip-10-0-128-39 kubenswrapper[2558]: I0710 18:36:48.446035 2558 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/1ec52f5a-3489-498e-9265-18fb1c32b6de-workload-certs\") pod \"e2e-api-b-171b27-openshift-default-6c8f584f77-rccmt\" (UID: \"1ec52f5a-3489-498e-9265-18fb1c32b6de\") " pod="openshift-ingress/e2e-api-b-171b27-openshift-default-6c8f584f77-rccmt" Jul 10 18:36:48.446139 ip-10-0-128-39 kubenswrapper[2558]: I0710 18:36:48.446117 2558 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/1ec52f5a-3489-498e-9265-18fb1c32b6de-istiod-ca-cert\") pod \"e2e-api-b-171b27-openshift-default-6c8f584f77-rccmt\" (UID: \"1ec52f5a-3489-498e-9265-18fb1c32b6de\") " pod="openshift-ingress/e2e-api-b-171b27-openshift-default-6c8f584f77-rccmt" Jul 10 18:36:48.447793 ip-10-0-128-39 kubenswrapper[2558]: I0710 18:36:48.447763 2558 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/1ec52f5a-3489-498e-9265-18fb1c32b6de-istio-envoy\") pod \"e2e-api-b-171b27-openshift-default-6c8f584f77-rccmt\" (UID: \"1ec52f5a-3489-498e-9265-18fb1c32b6de\") " pod="openshift-ingress/e2e-api-b-171b27-openshift-default-6c8f584f77-rccmt" Jul 10 18:36:48.447972 ip-10-0-128-39 kubenswrapper[2558]: I0710 18:36:48.447948 2558 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/1ec52f5a-3489-498e-9265-18fb1c32b6de-istio-podinfo\") pod \"e2e-api-b-171b27-openshift-default-6c8f584f77-rccmt\" (UID: \"1ec52f5a-3489-498e-9265-18fb1c32b6de\") " pod="openshift-ingress/e2e-api-b-171b27-openshift-default-6c8f584f77-rccmt" Jul 10 18:36:48.455374 ip-10-0-128-39 kubenswrapper[2558]: I0710 18:36:48.455346 2558 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-7crcl\" (UniqueName: \"kubernetes.io/projected/1ec52f5a-3489-498e-9265-18fb1c32b6de-kube-api-access-7crcl\") pod \"e2e-api-b-171b27-openshift-default-6c8f584f77-rccmt\" (UID: \"1ec52f5a-3489-498e-9265-18fb1c32b6de\") " pod="openshift-ingress/e2e-api-b-171b27-openshift-default-6c8f584f77-rccmt" Jul 10 18:36:48.455881 ip-10-0-128-39 kubenswrapper[2558]: I0710 18:36:48.455860 2558 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/1ec52f5a-3489-498e-9265-18fb1c32b6de-istio-token\") pod \"e2e-api-b-171b27-openshift-default-6c8f584f77-rccmt\" (UID: \"1ec52f5a-3489-498e-9265-18fb1c32b6de\") " pod="openshift-ingress/e2e-api-b-171b27-openshift-default-6c8f584f77-rccmt" Jul 10 18:36:48.603993 ip-10-0-128-39 kubenswrapper[2558]: I0710 18:36:48.603904 2558 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-api-b-171b27-openshift-default-6c8f584f77-rccmt" Jul 10 18:36:48.755534 ip-10-0-128-39 kubenswrapper[2558]: I0710 18:36:48.755509 2558 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/e2e-api-b-171b27-openshift-default-6c8f584f77-rccmt"] Jul 10 18:36:48.757983 ip-10-0-128-39 kubenswrapper[2558]: W0710 18:36:48.757953 2558 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod1ec52f5a_3489_498e_9265_18fb1c32b6de.slice/crio-2b88062ec7a2f0bcf436c1a2afd451707c2b0a28e71fae5be1065f851d859642 WatchSource:0}: Error finding container 2b88062ec7a2f0bcf436c1a2afd451707c2b0a28e71fae5be1065f851d859642: Status 404 returned error can't find the container with id 2b88062ec7a2f0bcf436c1a2afd451707c2b0a28e71fae5be1065f851d859642 Jul 10 18:36:48.760526 ip-10-0-128-39 kubenswrapper[2558]: I0710 18:36:48.760475 2558 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236224Ki","pods":"250"} Jul 10 18:36:48.760610 ip-10-0-128-39 kubenswrapper[2558]: I0710 18:36:48.760546 2558 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236224Ki","pods":"250"} Jul 10 18:36:48.760610 ip-10-0-128-39 kubenswrapper[2558]: I0710 18:36:48.760574 2558 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236224Ki","pods":"250"} Jul 10 18:36:49.539470 ip-10-0-128-39 kubenswrapper[2558]: I0710 18:36:49.539432 2558 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-api-b-171b27-openshift-default-6c8f584f77-rccmt" event={"ID":"1ec52f5a-3489-498e-9265-18fb1c32b6de","Type":"ContainerStarted","Data":"e2844545fd6a0ff57adfd5d640e83db2b62cad2044c70271fc1dc3f85ef69d0f"} Jul 10 18:36:49.539898 ip-10-0-128-39 kubenswrapper[2558]: I0710 18:36:49.539486 2558 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-api-b-171b27-openshift-default-6c8f584f77-rccmt" event={"ID":"1ec52f5a-3489-498e-9265-18fb1c32b6de","Type":"ContainerStarted","Data":"2b88062ec7a2f0bcf436c1a2afd451707c2b0a28e71fae5be1065f851d859642"} Jul 10 18:36:49.604568 ip-10-0-128-39 kubenswrapper[2558]: I0710 18:36:49.604535 2558 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="openshift-ingress/e2e-api-b-171b27-openshift-default-6c8f584f77-rccmt" Jul 10 18:36:49.609524 ip-10-0-128-39 kubenswrapper[2558]: I0710 18:36:49.609498 2558 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="openshift-ingress/e2e-api-b-171b27-openshift-default-6c8f584f77-rccmt" Jul 10 18:36:49.617488 ip-10-0-128-39 kubenswrapper[2558]: I0710 18:36:49.617439 2558 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-ingress/e2e-api-b-171b27-openshift-default-6c8f584f77-rccmt" podStartSLOduration=1.61742266 podStartE2EDuration="1.61742266s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-10 18:36:48 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-10 18:36:49.616672554 +0000 UTC m=+2038.120165060" watchObservedRunningTime="2026-07-10 18:36:49.61742266 +0000 UTC m=+2038.120915217" Jul 10 18:36:50.544113 ip-10-0-128-39 kubenswrapper[2558]: I0710 18:36:50.544076 2558 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/e2e-api-b-171b27-openshift-default-6c8f584f77-rccmt" Jul 10 18:36:50.544935 ip-10-0-128-39 kubenswrapper[2558]: I0710 18:36:50.544915 2558 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/e2e-api-b-171b27-openshift-default-6c8f584f77-rccmt" Jul 10 18:36:52.313583 ip-10-0-128-39 kubenswrapper[2558]: I0710 18:36:52.313546 2558 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-hjf5z"] Jul 10 18:37:07.996601 ip-10-0-128-39 kubenswrapper[2558]: I0710 18:37:07.996562 2558 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-hjf5z"] Jul 10 18:37:09.637408 ip-10-0-128-39 kubenswrapper[2558]: I0710 18:37:09.637322 2558 generic.go:356] "Generic (PLEG): container finished" podID="1ec52f5a-3489-498e-9265-18fb1c32b6de" containerID="e2844545fd6a0ff57adfd5d640e83db2b62cad2044c70271fc1dc3f85ef69d0f" exitCode=0 Jul 10 18:37:09.637850 ip-10-0-128-39 kubenswrapper[2558]: I0710 18:37:09.637397 2558 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-api-b-171b27-openshift-default-6c8f584f77-rccmt" event={"ID":"1ec52f5a-3489-498e-9265-18fb1c32b6de","Type":"ContainerDied","Data":"e2844545fd6a0ff57adfd5d640e83db2b62cad2044c70271fc1dc3f85ef69d0f"} Jul 10 18:37:09.637850 ip-10-0-128-39 kubenswrapper[2558]: I0710 18:37:09.637785 2558 scope.go:117] "RemoveContainer" containerID="e2844545fd6a0ff57adfd5d640e83db2b62cad2044c70271fc1dc3f85ef69d0f" Jul 10 18:37:09.638851 ip-10-0-128-39 kubenswrapper[2558]: I0710 18:37:09.638791 2558 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236224Ki","pods":"250"} Jul 10 18:37:09.638965 ip-10-0-128-39 kubenswrapper[2558]: I0710 18:37:09.638893 2558 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236224Ki","pods":"250"} Jul 10 18:37:09.638965 ip-10-0-128-39 kubenswrapper[2558]: I0710 18:37:09.638928 2558 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236224Ki","pods":"250"} Jul 10 18:37:10.644136 ip-10-0-128-39 kubenswrapper[2558]: I0710 18:37:10.644096 2558 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-api-b-171b27-openshift-default-6c8f584f77-rccmt" event={"ID":"1ec52f5a-3489-498e-9265-18fb1c32b6de","Type":"ContainerStarted","Data":"51747b9a21afa18cf5c3e1b65a6f440051a81a38070291d50ac1b0b0a8260aa2"} Jul 10 18:37:11.604734 ip-10-0-128-39 kubenswrapper[2558]: I0710 18:37:11.604692 2558 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="openshift-ingress/e2e-api-b-171b27-openshift-default-6c8f584f77-rccmt" Jul 10 18:37:11.605035 ip-10-0-128-39 kubenswrapper[2558]: I0710 18:37:11.605008 2558 patch_prober.go:28] interesting pod/e2e-api-b-171b27-openshift-default-6c8f584f77-rccmt container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.132.0.39:15021/healthz/ready\": dial tcp 10.132.0.39:15021: connect: connection refused" start-of-body= Jul 10 18:37:11.605120 ip-10-0-128-39 kubenswrapper[2558]: I0710 18:37:11.605071 2558 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-api-b-171b27-openshift-default-6c8f584f77-rccmt" podUID="1ec52f5a-3489-498e-9265-18fb1c32b6de" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.132.0.39:15021/healthz/ready\": dial tcp 10.132.0.39:15021: connect: connection refused" Jul 10 18:37:12.604354 ip-10-0-128-39 kubenswrapper[2558]: I0710 18:37:12.604315 2558 patch_prober.go:28] interesting pod/e2e-api-b-171b27-openshift-default-6c8f584f77-rccmt container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.132.0.39:15021/healthz/ready\": dial tcp 10.132.0.39:15021: connect: connection refused" start-of-body= Jul 10 18:37:12.604846 ip-10-0-128-39 kubenswrapper[2558]: I0710 18:37:12.604378 2558 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-api-b-171b27-openshift-default-6c8f584f77-rccmt" podUID="1ec52f5a-3489-498e-9265-18fb1c32b6de" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.132.0.39:15021/healthz/ready\": dial tcp 10.132.0.39:15021: connect: connection refused" Jul 10 18:37:13.605267 ip-10-0-128-39 kubenswrapper[2558]: I0710 18:37:13.605221 2558 patch_prober.go:28] interesting pod/e2e-api-b-171b27-openshift-default-6c8f584f77-rccmt container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.132.0.39:15021/healthz/ready\": dial tcp 10.132.0.39:15021: connect: connection refused" start-of-body= Jul 10 18:37:13.605681 ip-10-0-128-39 kubenswrapper[2558]: I0710 18:37:13.605280 2558 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-api-b-171b27-openshift-default-6c8f584f77-rccmt" podUID="1ec52f5a-3489-498e-9265-18fb1c32b6de" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.132.0.39:15021/healthz/ready\": dial tcp 10.132.0.39:15021: connect: connection refused" Jul 10 18:37:14.671873 ip-10-0-128-39 kubenswrapper[2558]: I0710 18:37:14.671767 2558 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="openshift-ingress/e2e-api-b-171b27-openshift-default-6c8f584f77-rccmt" Jul 10 18:37:14.672367 ip-10-0-128-39 kubenswrapper[2558]: I0710 18:37:14.672265 2558 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/e2e-api-b-171b27-openshift-default-6c8f584f77-rccmt" Jul 10 18:37:14.673670 ip-10-0-128-39 kubenswrapper[2558]: I0710 18:37:14.673642 2558 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/e2e-api-b-171b27-openshift-default-6c8f584f77-rccmt" Jul 10 18:37:17.916463 ip-10-0-128-39 kubenswrapper[2558]: I0710 18:37:17.916427 2558 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-ingress/e2e-api-b-171b27-openshift-default-6c8f584f77-rccmt"] Jul 10 18:37:17.917095 ip-10-0-128-39 kubenswrapper[2558]: I0710 18:37:17.916635 2558 kuberuntime_container.go:873] "Killing container with a grace period" pod="openshift-ingress/e2e-api-b-171b27-openshift-default-6c8f584f77-rccmt" podUID="1ec52f5a-3489-498e-9265-18fb1c32b6de" containerName="istio-proxy" containerID="cri-o://51747b9a21afa18cf5c3e1b65a6f440051a81a38070291d50ac1b0b0a8260aa2" gracePeriod=30 Jul 10 18:37:21.098859 ip-10-0-128-39 kubenswrapper[2558]: I0710 18:37:21.098799 2558 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-hjf5z"] Jul 10 18:37:23.187252 ip-10-0-128-39 kubenswrapper[2558]: I0710 18:37:23.187175 2558 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-api-b-171b27-openshift-default-6c8f584f77-rccmt" Jul 10 18:37:23.281027 ip-10-0-128-39 kubenswrapper[2558]: I0710 18:37:23.280987 2558 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/1ec52f5a-3489-498e-9265-18fb1c32b6de-workload-certs\") pod \"1ec52f5a-3489-498e-9265-18fb1c32b6de\" (UID: \"1ec52f5a-3489-498e-9265-18fb1c32b6de\") " Jul 10 18:37:23.281027 ip-10-0-128-39 kubenswrapper[2558]: I0710 18:37:23.281028 2558 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/1ec52f5a-3489-498e-9265-18fb1c32b6de-credential-socket\") pod \"1ec52f5a-3489-498e-9265-18fb1c32b6de\" (UID: \"1ec52f5a-3489-498e-9265-18fb1c32b6de\") " Jul 10 18:37:23.281251 ip-10-0-128-39 kubenswrapper[2558]: I0710 18:37:23.281061 2558 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/1ec52f5a-3489-498e-9265-18fb1c32b6de-istiod-ca-cert\") pod \"1ec52f5a-3489-498e-9265-18fb1c32b6de\" (UID: \"1ec52f5a-3489-498e-9265-18fb1c32b6de\") " Jul 10 18:37:23.281251 ip-10-0-128-39 kubenswrapper[2558]: I0710 18:37:23.281129 2558 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/1ec52f5a-3489-498e-9265-18fb1c32b6de-istio-envoy\") pod \"1ec52f5a-3489-498e-9265-18fb1c32b6de\" (UID: \"1ec52f5a-3489-498e-9265-18fb1c32b6de\") " Jul 10 18:37:23.281251 ip-10-0-128-39 kubenswrapper[2558]: I0710 18:37:23.281168 2558 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/1ec52f5a-3489-498e-9265-18fb1c32b6de-istio-podinfo\") pod \"1ec52f5a-3489-498e-9265-18fb1c32b6de\" (UID: \"1ec52f5a-3489-498e-9265-18fb1c32b6de\") " Jul 10 18:37:23.281251 ip-10-0-128-39 kubenswrapper[2558]: I0710 18:37:23.281195 2558 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-7crcl\" (UniqueName: \"kubernetes.io/projected/1ec52f5a-3489-498e-9265-18fb1c32b6de-kube-api-access-7crcl\") pod \"1ec52f5a-3489-498e-9265-18fb1c32b6de\" (UID: \"1ec52f5a-3489-498e-9265-18fb1c32b6de\") " Jul 10 18:37:23.281251 ip-10-0-128-39 kubenswrapper[2558]: I0710 18:37:23.281236 2558 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/1ec52f5a-3489-498e-9265-18fb1c32b6de-istio-data\") pod \"1ec52f5a-3489-498e-9265-18fb1c32b6de\" (UID: \"1ec52f5a-3489-498e-9265-18fb1c32b6de\") " Jul 10 18:37:23.281495 ip-10-0-128-39 kubenswrapper[2558]: I0710 18:37:23.281260 2558 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/1ec52f5a-3489-498e-9265-18fb1c32b6de-istio-token\") pod \"1ec52f5a-3489-498e-9265-18fb1c32b6de\" (UID: \"1ec52f5a-3489-498e-9265-18fb1c32b6de\") " Jul 10 18:37:23.281495 ip-10-0-128-39 kubenswrapper[2558]: I0710 18:37:23.281340 2558 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/1ec52f5a-3489-498e-9265-18fb1c32b6de-workload-certs" (OuterVolumeSpecName: "workload-certs") pod "1ec52f5a-3489-498e-9265-18fb1c32b6de" (UID: "1ec52f5a-3489-498e-9265-18fb1c32b6de"). InnerVolumeSpecName "workload-certs". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Jul 10 18:37:23.281495 ip-10-0-128-39 kubenswrapper[2558]: I0710 18:37:23.281411 2558 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/1ec52f5a-3489-498e-9265-18fb1c32b6de-credential-socket" (OuterVolumeSpecName: "credential-socket") pod "1ec52f5a-3489-498e-9265-18fb1c32b6de" (UID: "1ec52f5a-3489-498e-9265-18fb1c32b6de"). InnerVolumeSpecName "credential-socket". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Jul 10 18:37:23.281495 ip-10-0-128-39 kubenswrapper[2558]: I0710 18:37:23.281418 2558 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/1ec52f5a-3489-498e-9265-18fb1c32b6de-workload-socket\") pod \"1ec52f5a-3489-498e-9265-18fb1c32b6de\" (UID: \"1ec52f5a-3489-498e-9265-18fb1c32b6de\") " Jul 10 18:37:23.281683 ip-10-0-128-39 kubenswrapper[2558]: I0710 18:37:23.281575 2558 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/1ec52f5a-3489-498e-9265-18fb1c32b6de-workload-socket" (OuterVolumeSpecName: "workload-socket") pod "1ec52f5a-3489-498e-9265-18fb1c32b6de" (UID: "1ec52f5a-3489-498e-9265-18fb1c32b6de"). InnerVolumeSpecName "workload-socket". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Jul 10 18:37:23.281885 ip-10-0-128-39 kubenswrapper[2558]: I0710 18:37:23.281801 2558 reconciler_common.go:299] "Volume detached for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/1ec52f5a-3489-498e-9265-18fb1c32b6de-workload-certs\") on node \"ip-10-0-128-39.ec2.internal\" DevicePath \"\"" Jul 10 18:37:23.281885 ip-10-0-128-39 kubenswrapper[2558]: I0710 18:37:23.281846 2558 reconciler_common.go:299] "Volume detached for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/1ec52f5a-3489-498e-9265-18fb1c32b6de-credential-socket\") on node \"ip-10-0-128-39.ec2.internal\" DevicePath \"\"" Jul 10 18:37:23.281885 ip-10-0-128-39 kubenswrapper[2558]: I0710 18:37:23.281847 2558 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/1ec52f5a-3489-498e-9265-18fb1c32b6de-istiod-ca-cert" (OuterVolumeSpecName: "istiod-ca-cert") pod "1ec52f5a-3489-498e-9265-18fb1c32b6de" (UID: "1ec52f5a-3489-498e-9265-18fb1c32b6de"). InnerVolumeSpecName "istiod-ca-cert". PluginName "kubernetes.io/configmap", VolumeGIDValue "" Jul 10 18:37:23.281885 ip-10-0-128-39 kubenswrapper[2558]: I0710 18:37:23.281862 2558 reconciler_common.go:299] "Volume detached for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/1ec52f5a-3489-498e-9265-18fb1c32b6de-workload-socket\") on node \"ip-10-0-128-39.ec2.internal\" DevicePath \"\"" Jul 10 18:37:23.282344 ip-10-0-128-39 kubenswrapper[2558]: I0710 18:37:23.282318 2558 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/1ec52f5a-3489-498e-9265-18fb1c32b6de-istio-data" (OuterVolumeSpecName: "istio-data") pod "1ec52f5a-3489-498e-9265-18fb1c32b6de" (UID: "1ec52f5a-3489-498e-9265-18fb1c32b6de"). InnerVolumeSpecName "istio-data". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Jul 10 18:37:23.283661 ip-10-0-128-39 kubenswrapper[2558]: I0710 18:37:23.283637 2558 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/1ec52f5a-3489-498e-9265-18fb1c32b6de-istio-envoy" (OuterVolumeSpecName: "istio-envoy") pod "1ec52f5a-3489-498e-9265-18fb1c32b6de" (UID: "1ec52f5a-3489-498e-9265-18fb1c32b6de"). InnerVolumeSpecName "istio-envoy". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Jul 10 18:37:23.283775 ip-10-0-128-39 kubenswrapper[2558]: I0710 18:37:23.283735 2558 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/downward-api/1ec52f5a-3489-498e-9265-18fb1c32b6de-istio-podinfo" (OuterVolumeSpecName: "istio-podinfo") pod "1ec52f5a-3489-498e-9265-18fb1c32b6de" (UID: "1ec52f5a-3489-498e-9265-18fb1c32b6de"). InnerVolumeSpecName "istio-podinfo". PluginName "kubernetes.io/downward-api", VolumeGIDValue "" Jul 10 18:37:23.283953 ip-10-0-128-39 kubenswrapper[2558]: I0710 18:37:23.283935 2558 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/1ec52f5a-3489-498e-9265-18fb1c32b6de-kube-api-access-7crcl" (OuterVolumeSpecName: "kube-api-access-7crcl") pod "1ec52f5a-3489-498e-9265-18fb1c32b6de" (UID: "1ec52f5a-3489-498e-9265-18fb1c32b6de"). InnerVolumeSpecName "kube-api-access-7crcl". PluginName "kubernetes.io/projected", VolumeGIDValue "" Jul 10 18:37:23.284018 ip-10-0-128-39 kubenswrapper[2558]: I0710 18:37:23.283952 2558 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/1ec52f5a-3489-498e-9265-18fb1c32b6de-istio-token" (OuterVolumeSpecName: "istio-token") pod "1ec52f5a-3489-498e-9265-18fb1c32b6de" (UID: "1ec52f5a-3489-498e-9265-18fb1c32b6de"). InnerVolumeSpecName "istio-token". PluginName "kubernetes.io/projected", VolumeGIDValue "" Jul 10 18:37:23.382558 ip-10-0-128-39 kubenswrapper[2558]: I0710 18:37:23.382518 2558 reconciler_common.go:299] "Volume detached for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/1ec52f5a-3489-498e-9265-18fb1c32b6de-istio-envoy\") on node \"ip-10-0-128-39.ec2.internal\" DevicePath \"\"" Jul 10 18:37:23.382558 ip-10-0-128-39 kubenswrapper[2558]: I0710 18:37:23.382551 2558 reconciler_common.go:299] "Volume detached for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/1ec52f5a-3489-498e-9265-18fb1c32b6de-istio-podinfo\") on node \"ip-10-0-128-39.ec2.internal\" DevicePath \"\"" Jul 10 18:37:23.382558 ip-10-0-128-39 kubenswrapper[2558]: I0710 18:37:23.382560 2558 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-7crcl\" (UniqueName: \"kubernetes.io/projected/1ec52f5a-3489-498e-9265-18fb1c32b6de-kube-api-access-7crcl\") on node \"ip-10-0-128-39.ec2.internal\" DevicePath \"\"" Jul 10 18:37:23.382558 ip-10-0-128-39 kubenswrapper[2558]: I0710 18:37:23.382569 2558 reconciler_common.go:299] "Volume detached for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/1ec52f5a-3489-498e-9265-18fb1c32b6de-istio-data\") on node \"ip-10-0-128-39.ec2.internal\" DevicePath \"\"" Jul 10 18:37:23.382881 ip-10-0-128-39 kubenswrapper[2558]: I0710 18:37:23.382578 2558 reconciler_common.go:299] "Volume detached for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/1ec52f5a-3489-498e-9265-18fb1c32b6de-istio-token\") on node \"ip-10-0-128-39.ec2.internal\" DevicePath \"\"" Jul 10 18:37:23.382881 ip-10-0-128-39 kubenswrapper[2558]: I0710 18:37:23.382586 2558 reconciler_common.go:299] "Volume detached for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/1ec52f5a-3489-498e-9265-18fb1c32b6de-istiod-ca-cert\") on node \"ip-10-0-128-39.ec2.internal\" DevicePath \"\"" Jul 10 18:37:23.726485 ip-10-0-128-39 kubenswrapper[2558]: I0710 18:37:23.726451 2558 generic.go:356] "Generic (PLEG): container finished" podID="1ec52f5a-3489-498e-9265-18fb1c32b6de" containerID="51747b9a21afa18cf5c3e1b65a6f440051a81a38070291d50ac1b0b0a8260aa2" exitCode=0 Jul 10 18:37:23.726661 ip-10-0-128-39 kubenswrapper[2558]: I0710 18:37:23.726505 2558 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-api-b-171b27-openshift-default-6c8f584f77-rccmt" Jul 10 18:37:23.726661 ip-10-0-128-39 kubenswrapper[2558]: I0710 18:37:23.726534 2558 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-api-b-171b27-openshift-default-6c8f584f77-rccmt" event={"ID":"1ec52f5a-3489-498e-9265-18fb1c32b6de","Type":"ContainerDied","Data":"51747b9a21afa18cf5c3e1b65a6f440051a81a38070291d50ac1b0b0a8260aa2"} Jul 10 18:37:23.726661 ip-10-0-128-39 kubenswrapper[2558]: I0710 18:37:23.726597 2558 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-api-b-171b27-openshift-default-6c8f584f77-rccmt" event={"ID":"1ec52f5a-3489-498e-9265-18fb1c32b6de","Type":"ContainerDied","Data":"2b88062ec7a2f0bcf436c1a2afd451707c2b0a28e71fae5be1065f851d859642"} Jul 10 18:37:23.726661 ip-10-0-128-39 kubenswrapper[2558]: I0710 18:37:23.726630 2558 scope.go:117] "RemoveContainer" containerID="51747b9a21afa18cf5c3e1b65a6f440051a81a38070291d50ac1b0b0a8260aa2" Jul 10 18:37:23.736600 ip-10-0-128-39 kubenswrapper[2558]: I0710 18:37:23.736582 2558 scope.go:117] "RemoveContainer" containerID="e2844545fd6a0ff57adfd5d640e83db2b62cad2044c70271fc1dc3f85ef69d0f" Jul 10 18:37:23.750571 ip-10-0-128-39 kubenswrapper[2558]: I0710 18:37:23.750543 2558 scope.go:117] "RemoveContainer" containerID="51747b9a21afa18cf5c3e1b65a6f440051a81a38070291d50ac1b0b0a8260aa2" Jul 10 18:37:23.750970 ip-10-0-128-39 kubenswrapper[2558]: E0710 18:37:23.750947 2558 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"51747b9a21afa18cf5c3e1b65a6f440051a81a38070291d50ac1b0b0a8260aa2\": container with ID starting with 51747b9a21afa18cf5c3e1b65a6f440051a81a38070291d50ac1b0b0a8260aa2 not found: ID does not exist" containerID="51747b9a21afa18cf5c3e1b65a6f440051a81a38070291d50ac1b0b0a8260aa2" Jul 10 18:37:23.751042 ip-10-0-128-39 kubenswrapper[2558]: I0710 18:37:23.750980 2558 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"51747b9a21afa18cf5c3e1b65a6f440051a81a38070291d50ac1b0b0a8260aa2"} err="failed to get container status \"51747b9a21afa18cf5c3e1b65a6f440051a81a38070291d50ac1b0b0a8260aa2\": rpc error: code = NotFound desc = could not find container \"51747b9a21afa18cf5c3e1b65a6f440051a81a38070291d50ac1b0b0a8260aa2\": container with ID starting with 51747b9a21afa18cf5c3e1b65a6f440051a81a38070291d50ac1b0b0a8260aa2 not found: ID does not exist" Jul 10 18:37:23.751042 ip-10-0-128-39 kubenswrapper[2558]: I0710 18:37:23.751000 2558 scope.go:117] "RemoveContainer" containerID="e2844545fd6a0ff57adfd5d640e83db2b62cad2044c70271fc1dc3f85ef69d0f" Jul 10 18:37:23.751381 ip-10-0-128-39 kubenswrapper[2558]: E0710 18:37:23.751357 2558 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"e2844545fd6a0ff57adfd5d640e83db2b62cad2044c70271fc1dc3f85ef69d0f\": container with ID starting with e2844545fd6a0ff57adfd5d640e83db2b62cad2044c70271fc1dc3f85ef69d0f not found: ID does not exist" containerID="e2844545fd6a0ff57adfd5d640e83db2b62cad2044c70271fc1dc3f85ef69d0f" Jul 10 18:37:23.751440 ip-10-0-128-39 kubenswrapper[2558]: I0710 18:37:23.751391 2558 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"e2844545fd6a0ff57adfd5d640e83db2b62cad2044c70271fc1dc3f85ef69d0f"} err="failed to get container status \"e2844545fd6a0ff57adfd5d640e83db2b62cad2044c70271fc1dc3f85ef69d0f\": rpc error: code = NotFound desc = could not find container \"e2844545fd6a0ff57adfd5d640e83db2b62cad2044c70271fc1dc3f85ef69d0f\": container with ID starting with e2844545fd6a0ff57adfd5d640e83db2b62cad2044c70271fc1dc3f85ef69d0f not found: ID does not exist" Jul 10 18:37:23.752169 ip-10-0-128-39 kubenswrapper[2558]: I0710 18:37:23.752149 2558 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-ingress/e2e-api-b-171b27-openshift-default-6c8f584f77-rccmt"] Jul 10 18:37:23.757541 ip-10-0-128-39 kubenswrapper[2558]: I0710 18:37:23.757517 2558 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["openshift-ingress/e2e-api-b-171b27-openshift-default-6c8f584f77-rccmt"] Jul 10 18:37:24.101751 ip-10-0-128-39 kubenswrapper[2558]: I0710 18:37:24.101648 2558 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="1ec52f5a-3489-498e-9265-18fb1c32b6de" path="/var/lib/kubelet/pods/1ec52f5a-3489-498e-9265-18fb1c32b6de/volumes" Jul 10 18:37:26.405120 ip-10-0-128-39 kubenswrapper[2558]: I0710 18:37:26.405076 2558 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-ingress/e2e-shared-b-4cfdb5-openshift-default-85569b84ff-swg7c"] Jul 10 18:37:26.405669 ip-10-0-128-39 kubenswrapper[2558]: I0710 18:37:26.405647 2558 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="1ec52f5a-3489-498e-9265-18fb1c32b6de" containerName="istio-proxy" Jul 10 18:37:26.405745 ip-10-0-128-39 kubenswrapper[2558]: I0710 18:37:26.405673 2558 state_mem.go:107] "Deleted CPUSet assignment" podUID="1ec52f5a-3489-498e-9265-18fb1c32b6de" containerName="istio-proxy" Jul 10 18:37:26.405745 ip-10-0-128-39 kubenswrapper[2558]: I0710 18:37:26.405686 2558 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="1ec52f5a-3489-498e-9265-18fb1c32b6de" containerName="istio-proxy" Jul 10 18:37:26.405745 ip-10-0-128-39 kubenswrapper[2558]: I0710 18:37:26.405694 2558 state_mem.go:107] "Deleted CPUSet assignment" podUID="1ec52f5a-3489-498e-9265-18fb1c32b6de" containerName="istio-proxy" Jul 10 18:37:26.405972 ip-10-0-128-39 kubenswrapper[2558]: I0710 18:37:26.405791 2558 memory_manager.go:368] "RemoveStaleState removing state" podUID="ff127468-61cc-4243-b114-9374e2c9e52a" containerName="istio-proxy" Jul 10 18:37:26.405972 ip-10-0-128-39 kubenswrapper[2558]: I0710 18:37:26.405822 2558 memory_manager.go:368] "RemoveStaleState removing state" podUID="1ec52f5a-3489-498e-9265-18fb1c32b6de" containerName="istio-proxy" Jul 10 18:37:26.405972 ip-10-0-128-39 kubenswrapper[2558]: I0710 18:37:26.405838 2558 memory_manager.go:368] "RemoveStaleState removing state" podUID="1ec52f5a-3489-498e-9265-18fb1c32b6de" containerName="istio-proxy" Jul 10 18:37:26.408996 ip-10-0-128-39 kubenswrapper[2558]: I0710 18:37:26.408972 2558 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-shared-b-4cfdb5-openshift-default-85569b84ff-swg7c" Jul 10 18:37:26.417672 ip-10-0-128-39 kubenswrapper[2558]: I0710 18:37:26.417647 2558 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress\"/\"e2e-shared-b-4cfdb5-openshift-default-dockercfg-7d9lw\"" Jul 10 18:37:26.417920 ip-10-0-128-39 kubenswrapper[2558]: I0710 18:37:26.417902 2558 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ingress\"/\"openshift-gw-ca-root-cert\"" Jul 10 18:37:26.432146 ip-10-0-128-39 kubenswrapper[2558]: I0710 18:37:26.432115 2558 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/e2e-shared-b-4cfdb5-openshift-default-85569b84ff-swg7c"] Jul 10 18:37:26.509874 ip-10-0-128-39 kubenswrapper[2558]: I0710 18:37:26.509803 2558 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-kjlnx\" (UniqueName: \"kubernetes.io/projected/6735c6d1-6741-40ee-901f-e6cfd26d62f2-kube-api-access-kjlnx\") pod \"e2e-shared-b-4cfdb5-openshift-default-85569b84ff-swg7c\" (UID: \"6735c6d1-6741-40ee-901f-e6cfd26d62f2\") " pod="openshift-ingress/e2e-shared-b-4cfdb5-openshift-default-85569b84ff-swg7c" Jul 10 18:37:26.510099 ip-10-0-128-39 kubenswrapper[2558]: I0710 18:37:26.509913 2558 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/6735c6d1-6741-40ee-901f-e6cfd26d62f2-istio-token\") pod \"e2e-shared-b-4cfdb5-openshift-default-85569b84ff-swg7c\" (UID: \"6735c6d1-6741-40ee-901f-e6cfd26d62f2\") " pod="openshift-ingress/e2e-shared-b-4cfdb5-openshift-default-85569b84ff-swg7c" Jul 10 18:37:26.510099 ip-10-0-128-39 kubenswrapper[2558]: I0710 18:37:26.509960 2558 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/6735c6d1-6741-40ee-901f-e6cfd26d62f2-istio-data\") pod \"e2e-shared-b-4cfdb5-openshift-default-85569b84ff-swg7c\" (UID: \"6735c6d1-6741-40ee-901f-e6cfd26d62f2\") " pod="openshift-ingress/e2e-shared-b-4cfdb5-openshift-default-85569b84ff-swg7c" Jul 10 18:37:26.510099 ip-10-0-128-39 kubenswrapper[2558]: I0710 18:37:26.509991 2558 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/6735c6d1-6741-40ee-901f-e6cfd26d62f2-workload-certs\") pod \"e2e-shared-b-4cfdb5-openshift-default-85569b84ff-swg7c\" (UID: \"6735c6d1-6741-40ee-901f-e6cfd26d62f2\") " pod="openshift-ingress/e2e-shared-b-4cfdb5-openshift-default-85569b84ff-swg7c" Jul 10 18:37:26.510099 ip-10-0-128-39 kubenswrapper[2558]: I0710 18:37:26.510016 2558 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/6735c6d1-6741-40ee-901f-e6cfd26d62f2-istiod-ca-cert\") pod \"e2e-shared-b-4cfdb5-openshift-default-85569b84ff-swg7c\" (UID: \"6735c6d1-6741-40ee-901f-e6cfd26d62f2\") " pod="openshift-ingress/e2e-shared-b-4cfdb5-openshift-default-85569b84ff-swg7c" Jul 10 18:37:26.510258 ip-10-0-128-39 kubenswrapper[2558]: I0710 18:37:26.510115 2558 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/6735c6d1-6741-40ee-901f-e6cfd26d62f2-istio-podinfo\") pod \"e2e-shared-b-4cfdb5-openshift-default-85569b84ff-swg7c\" (UID: \"6735c6d1-6741-40ee-901f-e6cfd26d62f2\") " pod="openshift-ingress/e2e-shared-b-4cfdb5-openshift-default-85569b84ff-swg7c" Jul 10 18:37:26.510258 ip-10-0-128-39 kubenswrapper[2558]: I0710 18:37:26.510165 2558 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/6735c6d1-6741-40ee-901f-e6cfd26d62f2-credential-socket\") pod \"e2e-shared-b-4cfdb5-openshift-default-85569b84ff-swg7c\" (UID: \"6735c6d1-6741-40ee-901f-e6cfd26d62f2\") " pod="openshift-ingress/e2e-shared-b-4cfdb5-openshift-default-85569b84ff-swg7c" Jul 10 18:37:26.510258 ip-10-0-128-39 kubenswrapper[2558]: I0710 18:37:26.510221 2558 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/6735c6d1-6741-40ee-901f-e6cfd26d62f2-workload-socket\") pod \"e2e-shared-b-4cfdb5-openshift-default-85569b84ff-swg7c\" (UID: \"6735c6d1-6741-40ee-901f-e6cfd26d62f2\") " pod="openshift-ingress/e2e-shared-b-4cfdb5-openshift-default-85569b84ff-swg7c" Jul 10 18:37:26.510258 ip-10-0-128-39 kubenswrapper[2558]: I0710 18:37:26.510248 2558 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/6735c6d1-6741-40ee-901f-e6cfd26d62f2-istio-envoy\") pod \"e2e-shared-b-4cfdb5-openshift-default-85569b84ff-swg7c\" (UID: \"6735c6d1-6741-40ee-901f-e6cfd26d62f2\") " pod="openshift-ingress/e2e-shared-b-4cfdb5-openshift-default-85569b84ff-swg7c" Jul 10 18:37:26.611223 ip-10-0-128-39 kubenswrapper[2558]: I0710 18:37:26.611168 2558 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-kjlnx\" (UniqueName: \"kubernetes.io/projected/6735c6d1-6741-40ee-901f-e6cfd26d62f2-kube-api-access-kjlnx\") pod \"e2e-shared-b-4cfdb5-openshift-default-85569b84ff-swg7c\" (UID: \"6735c6d1-6741-40ee-901f-e6cfd26d62f2\") " pod="openshift-ingress/e2e-shared-b-4cfdb5-openshift-default-85569b84ff-swg7c" Jul 10 18:37:26.611223 ip-10-0-128-39 kubenswrapper[2558]: I0710 18:37:26.611230 2558 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/6735c6d1-6741-40ee-901f-e6cfd26d62f2-istio-token\") pod \"e2e-shared-b-4cfdb5-openshift-default-85569b84ff-swg7c\" (UID: \"6735c6d1-6741-40ee-901f-e6cfd26d62f2\") " pod="openshift-ingress/e2e-shared-b-4cfdb5-openshift-default-85569b84ff-swg7c" Jul 10 18:37:26.611488 ip-10-0-128-39 kubenswrapper[2558]: I0710 18:37:26.611271 2558 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/6735c6d1-6741-40ee-901f-e6cfd26d62f2-istio-data\") pod \"e2e-shared-b-4cfdb5-openshift-default-85569b84ff-swg7c\" (UID: \"6735c6d1-6741-40ee-901f-e6cfd26d62f2\") " pod="openshift-ingress/e2e-shared-b-4cfdb5-openshift-default-85569b84ff-swg7c" Jul 10 18:37:26.611488 ip-10-0-128-39 kubenswrapper[2558]: I0710 18:37:26.611303 2558 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/6735c6d1-6741-40ee-901f-e6cfd26d62f2-workload-certs\") pod \"e2e-shared-b-4cfdb5-openshift-default-85569b84ff-swg7c\" (UID: \"6735c6d1-6741-40ee-901f-e6cfd26d62f2\") " pod="openshift-ingress/e2e-shared-b-4cfdb5-openshift-default-85569b84ff-swg7c" Jul 10 18:37:26.611488 ip-10-0-128-39 kubenswrapper[2558]: I0710 18:37:26.611327 2558 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/6735c6d1-6741-40ee-901f-e6cfd26d62f2-istiod-ca-cert\") pod \"e2e-shared-b-4cfdb5-openshift-default-85569b84ff-swg7c\" (UID: \"6735c6d1-6741-40ee-901f-e6cfd26d62f2\") " pod="openshift-ingress/e2e-shared-b-4cfdb5-openshift-default-85569b84ff-swg7c" Jul 10 18:37:26.611488 ip-10-0-128-39 kubenswrapper[2558]: I0710 18:37:26.611351 2558 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/6735c6d1-6741-40ee-901f-e6cfd26d62f2-istio-podinfo\") pod \"e2e-shared-b-4cfdb5-openshift-default-85569b84ff-swg7c\" (UID: \"6735c6d1-6741-40ee-901f-e6cfd26d62f2\") " pod="openshift-ingress/e2e-shared-b-4cfdb5-openshift-default-85569b84ff-swg7c" Jul 10 18:37:26.611488 ip-10-0-128-39 kubenswrapper[2558]: I0710 18:37:26.611383 2558 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/6735c6d1-6741-40ee-901f-e6cfd26d62f2-credential-socket\") pod \"e2e-shared-b-4cfdb5-openshift-default-85569b84ff-swg7c\" (UID: \"6735c6d1-6741-40ee-901f-e6cfd26d62f2\") " pod="openshift-ingress/e2e-shared-b-4cfdb5-openshift-default-85569b84ff-swg7c" Jul 10 18:37:26.611731 ip-10-0-128-39 kubenswrapper[2558]: I0710 18:37:26.611524 2558 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/6735c6d1-6741-40ee-901f-e6cfd26d62f2-workload-socket\") pod \"e2e-shared-b-4cfdb5-openshift-default-85569b84ff-swg7c\" (UID: \"6735c6d1-6741-40ee-901f-e6cfd26d62f2\") " pod="openshift-ingress/e2e-shared-b-4cfdb5-openshift-default-85569b84ff-swg7c" Jul 10 18:37:26.611731 ip-10-0-128-39 kubenswrapper[2558]: I0710 18:37:26.611590 2558 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/6735c6d1-6741-40ee-901f-e6cfd26d62f2-istio-envoy\") pod \"e2e-shared-b-4cfdb5-openshift-default-85569b84ff-swg7c\" (UID: \"6735c6d1-6741-40ee-901f-e6cfd26d62f2\") " pod="openshift-ingress/e2e-shared-b-4cfdb5-openshift-default-85569b84ff-swg7c" Jul 10 18:37:26.611857 ip-10-0-128-39 kubenswrapper[2558]: I0710 18:37:26.611737 2558 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/6735c6d1-6741-40ee-901f-e6cfd26d62f2-istio-data\") pod \"e2e-shared-b-4cfdb5-openshift-default-85569b84ff-swg7c\" (UID: \"6735c6d1-6741-40ee-901f-e6cfd26d62f2\") " pod="openshift-ingress/e2e-shared-b-4cfdb5-openshift-default-85569b84ff-swg7c" Jul 10 18:37:26.611857 ip-10-0-128-39 kubenswrapper[2558]: I0710 18:37:26.611802 2558 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/6735c6d1-6741-40ee-901f-e6cfd26d62f2-workload-certs\") pod \"e2e-shared-b-4cfdb5-openshift-default-85569b84ff-swg7c\" (UID: \"6735c6d1-6741-40ee-901f-e6cfd26d62f2\") " pod="openshift-ingress/e2e-shared-b-4cfdb5-openshift-default-85569b84ff-swg7c" Jul 10 18:37:26.611973 ip-10-0-128-39 kubenswrapper[2558]: I0710 18:37:26.611919 2558 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/6735c6d1-6741-40ee-901f-e6cfd26d62f2-credential-socket\") pod \"e2e-shared-b-4cfdb5-openshift-default-85569b84ff-swg7c\" (UID: \"6735c6d1-6741-40ee-901f-e6cfd26d62f2\") " pod="openshift-ingress/e2e-shared-b-4cfdb5-openshift-default-85569b84ff-swg7c" Jul 10 18:37:26.612041 ip-10-0-128-39 kubenswrapper[2558]: I0710 18:37:26.612020 2558 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/6735c6d1-6741-40ee-901f-e6cfd26d62f2-workload-socket\") pod \"e2e-shared-b-4cfdb5-openshift-default-85569b84ff-swg7c\" (UID: \"6735c6d1-6741-40ee-901f-e6cfd26d62f2\") " pod="openshift-ingress/e2e-shared-b-4cfdb5-openshift-default-85569b84ff-swg7c" Jul 10 18:37:26.612177 ip-10-0-128-39 kubenswrapper[2558]: I0710 18:37:26.612157 2558 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/6735c6d1-6741-40ee-901f-e6cfd26d62f2-istiod-ca-cert\") pod \"e2e-shared-b-4cfdb5-openshift-default-85569b84ff-swg7c\" (UID: \"6735c6d1-6741-40ee-901f-e6cfd26d62f2\") " pod="openshift-ingress/e2e-shared-b-4cfdb5-openshift-default-85569b84ff-swg7c" Jul 10 18:37:26.613846 ip-10-0-128-39 kubenswrapper[2558]: I0710 18:37:26.613801 2558 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/6735c6d1-6741-40ee-901f-e6cfd26d62f2-istio-envoy\") pod \"e2e-shared-b-4cfdb5-openshift-default-85569b84ff-swg7c\" (UID: \"6735c6d1-6741-40ee-901f-e6cfd26d62f2\") " pod="openshift-ingress/e2e-shared-b-4cfdb5-openshift-default-85569b84ff-swg7c" Jul 10 18:37:26.613995 ip-10-0-128-39 kubenswrapper[2558]: I0710 18:37:26.613978 2558 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/6735c6d1-6741-40ee-901f-e6cfd26d62f2-istio-podinfo\") pod \"e2e-shared-b-4cfdb5-openshift-default-85569b84ff-swg7c\" (UID: \"6735c6d1-6741-40ee-901f-e6cfd26d62f2\") " pod="openshift-ingress/e2e-shared-b-4cfdb5-openshift-default-85569b84ff-swg7c" Jul 10 18:37:26.620710 ip-10-0-128-39 kubenswrapper[2558]: I0710 18:37:26.620675 2558 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/6735c6d1-6741-40ee-901f-e6cfd26d62f2-istio-token\") pod \"e2e-shared-b-4cfdb5-openshift-default-85569b84ff-swg7c\" (UID: \"6735c6d1-6741-40ee-901f-e6cfd26d62f2\") " pod="openshift-ingress/e2e-shared-b-4cfdb5-openshift-default-85569b84ff-swg7c" Jul 10 18:37:26.620853 ip-10-0-128-39 kubenswrapper[2558]: I0710 18:37:26.620837 2558 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-kjlnx\" (UniqueName: \"kubernetes.io/projected/6735c6d1-6741-40ee-901f-e6cfd26d62f2-kube-api-access-kjlnx\") pod \"e2e-shared-b-4cfdb5-openshift-default-85569b84ff-swg7c\" (UID: \"6735c6d1-6741-40ee-901f-e6cfd26d62f2\") " pod="openshift-ingress/e2e-shared-b-4cfdb5-openshift-default-85569b84ff-swg7c" Jul 10 18:37:26.720266 ip-10-0-128-39 kubenswrapper[2558]: I0710 18:37:26.720221 2558 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-shared-b-4cfdb5-openshift-default-85569b84ff-swg7c" Jul 10 18:37:26.876245 ip-10-0-128-39 kubenswrapper[2558]: I0710 18:37:26.876195 2558 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/e2e-shared-b-4cfdb5-openshift-default-85569b84ff-swg7c"] Jul 10 18:37:26.879296 ip-10-0-128-39 kubenswrapper[2558]: W0710 18:37:26.879239 2558 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod6735c6d1_6741_40ee_901f_e6cfd26d62f2.slice/crio-e79d2104abbadcd8920be66577992fcb302d2f6a074924ff98fd95299920ecdf WatchSource:0}: Error finding container e79d2104abbadcd8920be66577992fcb302d2f6a074924ff98fd95299920ecdf: Status 404 returned error can't find the container with id e79d2104abbadcd8920be66577992fcb302d2f6a074924ff98fd95299920ecdf Jul 10 18:37:26.881636 ip-10-0-128-39 kubenswrapper[2558]: I0710 18:37:26.881589 2558 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236224Ki","pods":"250"} Jul 10 18:37:26.881735 ip-10-0-128-39 kubenswrapper[2558]: I0710 18:37:26.881691 2558 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236224Ki","pods":"250"} Jul 10 18:37:26.881774 ip-10-0-128-39 kubenswrapper[2558]: I0710 18:37:26.881741 2558 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236224Ki","pods":"250"} Jul 10 18:37:27.750543 ip-10-0-128-39 kubenswrapper[2558]: I0710 18:37:27.750504 2558 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-shared-b-4cfdb5-openshift-default-85569b84ff-swg7c" event={"ID":"6735c6d1-6741-40ee-901f-e6cfd26d62f2","Type":"ContainerStarted","Data":"4af28f235ee2d04110c372522a0ed97abb1f9e170b1ffe510762c3d09962523e"} Jul 10 18:37:27.750977 ip-10-0-128-39 kubenswrapper[2558]: I0710 18:37:27.750567 2558 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-shared-b-4cfdb5-openshift-default-85569b84ff-swg7c" event={"ID":"6735c6d1-6741-40ee-901f-e6cfd26d62f2","Type":"ContainerStarted","Data":"e79d2104abbadcd8920be66577992fcb302d2f6a074924ff98fd95299920ecdf"} Jul 10 18:37:27.800685 ip-10-0-128-39 kubenswrapper[2558]: I0710 18:37:27.800628 2558 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-ingress/e2e-shared-b-4cfdb5-openshift-default-85569b84ff-swg7c" podStartSLOduration=1.80060939 podStartE2EDuration="1.80060939s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-10 18:37:26 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-10 18:37:27.799348234 +0000 UTC m=+2076.302840739" watchObservedRunningTime="2026-07-10 18:37:27.80060939 +0000 UTC m=+2076.304101873" Jul 10 18:37:28.721088 ip-10-0-128-39 kubenswrapper[2558]: I0710 18:37:28.721039 2558 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="openshift-ingress/e2e-shared-b-4cfdb5-openshift-default-85569b84ff-swg7c" Jul 10 18:37:28.726724 ip-10-0-128-39 kubenswrapper[2558]: I0710 18:37:28.726694 2558 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="openshift-ingress/e2e-shared-b-4cfdb5-openshift-default-85569b84ff-swg7c" Jul 10 18:37:28.755684 ip-10-0-128-39 kubenswrapper[2558]: I0710 18:37:28.755650 2558 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/e2e-shared-b-4cfdb5-openshift-default-85569b84ff-swg7c" Jul 10 18:37:28.756752 ip-10-0-128-39 kubenswrapper[2558]: I0710 18:37:28.756726 2558 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/e2e-shared-b-4cfdb5-openshift-default-85569b84ff-swg7c" Jul 10 18:37:34.529944 ip-10-0-128-39 kubenswrapper[2558]: I0710 18:37:34.529905 2558 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-hjf5z"] Jul 10 18:37:42.826913 ip-10-0-128-39 kubenswrapper[2558]: I0710 18:37:42.826877 2558 generic.go:356] "Generic (PLEG): container finished" podID="6735c6d1-6741-40ee-901f-e6cfd26d62f2" containerID="4af28f235ee2d04110c372522a0ed97abb1f9e170b1ffe510762c3d09962523e" exitCode=0 Jul 10 18:37:42.827321 ip-10-0-128-39 kubenswrapper[2558]: I0710 18:37:42.826951 2558 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-shared-b-4cfdb5-openshift-default-85569b84ff-swg7c" event={"ID":"6735c6d1-6741-40ee-901f-e6cfd26d62f2","Type":"ContainerDied","Data":"4af28f235ee2d04110c372522a0ed97abb1f9e170b1ffe510762c3d09962523e"} Jul 10 18:37:42.827364 ip-10-0-128-39 kubenswrapper[2558]: I0710 18:37:42.827354 2558 scope.go:117] "RemoveContainer" containerID="4af28f235ee2d04110c372522a0ed97abb1f9e170b1ffe510762c3d09962523e" Jul 10 18:37:42.828387 ip-10-0-128-39 kubenswrapper[2558]: I0710 18:37:42.828362 2558 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236224Ki","pods":"250"} Jul 10 18:37:42.828449 ip-10-0-128-39 kubenswrapper[2558]: I0710 18:37:42.828427 2558 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236224Ki","pods":"250"} Jul 10 18:37:42.828483 ip-10-0-128-39 kubenswrapper[2558]: I0710 18:37:42.828460 2558 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236224Ki","pods":"250"} Jul 10 18:37:43.756385 ip-10-0-128-39 kubenswrapper[2558]: I0710 18:37:43.756327 2558 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/e2e-shared-b-4cfdb5-openshift-default-85569b84ff-swg7c" Jul 10 18:37:43.833837 ip-10-0-128-39 kubenswrapper[2558]: I0710 18:37:43.833768 2558 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-shared-b-4cfdb5-openshift-default-85569b84ff-swg7c" event={"ID":"6735c6d1-6741-40ee-901f-e6cfd26d62f2","Type":"ContainerStarted","Data":"022cd5e08a211713dc2754d5b11aabb9e2fd4dea4d054a4d4da35b24a6185058"} Jul 10 18:37:44.720843 ip-10-0-128-39 kubenswrapper[2558]: I0710 18:37:44.720795 2558 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="openshift-ingress/e2e-shared-b-4cfdb5-openshift-default-85569b84ff-swg7c" Jul 10 18:37:44.721979 ip-10-0-128-39 kubenswrapper[2558]: I0710 18:37:44.721954 2558 patch_prober.go:28] interesting pod/e2e-shared-b-4cfdb5-openshift-default-85569b84ff-swg7c container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.132.0.40:15021/healthz/ready\": dial tcp 10.132.0.40:15021: connect: connection refused" start-of-body= Jul 10 18:37:44.722078 ip-10-0-128-39 kubenswrapper[2558]: I0710 18:37:44.722001 2558 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-shared-b-4cfdb5-openshift-default-85569b84ff-swg7c" podUID="6735c6d1-6741-40ee-901f-e6cfd26d62f2" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.132.0.40:15021/healthz/ready\": dial tcp 10.132.0.40:15021: connect: connection refused" Jul 10 18:37:45.721212 ip-10-0-128-39 kubenswrapper[2558]: I0710 18:37:45.721170 2558 patch_prober.go:28] interesting pod/e2e-shared-b-4cfdb5-openshift-default-85569b84ff-swg7c container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.132.0.40:15021/healthz/ready\": dial tcp 10.132.0.40:15021: connect: connection refused" start-of-body= Jul 10 18:37:45.721612 ip-10-0-128-39 kubenswrapper[2558]: I0710 18:37:45.721235 2558 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-shared-b-4cfdb5-openshift-default-85569b84ff-swg7c" podUID="6735c6d1-6741-40ee-901f-e6cfd26d62f2" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.132.0.40:15021/healthz/ready\": dial tcp 10.132.0.40:15021: connect: connection refused" Jul 10 18:37:46.721585 ip-10-0-128-39 kubenswrapper[2558]: I0710 18:37:46.721548 2558 patch_prober.go:28] interesting pod/e2e-shared-b-4cfdb5-openshift-default-85569b84ff-swg7c container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.132.0.40:15021/healthz/ready\": dial tcp 10.132.0.40:15021: connect: connection refused" start-of-body= Jul 10 18:37:46.721976 ip-10-0-128-39 kubenswrapper[2558]: I0710 18:37:46.721604 2558 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-shared-b-4cfdb5-openshift-default-85569b84ff-swg7c" podUID="6735c6d1-6741-40ee-901f-e6cfd26d62f2" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.132.0.40:15021/healthz/ready\": dial tcp 10.132.0.40:15021: connect: connection refused" Jul 10 18:37:47.720904 ip-10-0-128-39 kubenswrapper[2558]: I0710 18:37:47.720863 2558 patch_prober.go:28] interesting pod/e2e-shared-b-4cfdb5-openshift-default-85569b84ff-swg7c container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.132.0.40:15021/healthz/ready\": dial tcp 10.132.0.40:15021: connect: connection refused" start-of-body= Jul 10 18:37:47.721108 ip-10-0-128-39 kubenswrapper[2558]: I0710 18:37:47.720924 2558 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-shared-b-4cfdb5-openshift-default-85569b84ff-swg7c" podUID="6735c6d1-6741-40ee-901f-e6cfd26d62f2" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.132.0.40:15021/healthz/ready\": dial tcp 10.132.0.40:15021: connect: connection refused" Jul 10 18:37:48.724882 ip-10-0-128-39 kubenswrapper[2558]: I0710 18:37:48.724849 2558 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="openshift-ingress/e2e-shared-b-4cfdb5-openshift-default-85569b84ff-swg7c" Jul 10 18:37:48.725358 ip-10-0-128-39 kubenswrapper[2558]: I0710 18:37:48.725251 2558 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/e2e-shared-b-4cfdb5-openshift-default-85569b84ff-swg7c" Jul 10 18:37:48.725987 ip-10-0-128-39 kubenswrapper[2558]: I0710 18:37:48.725965 2558 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/e2e-shared-b-4cfdb5-openshift-default-85569b84ff-swg7c" Jul 10 18:37:52.121482 ip-10-0-128-39 kubenswrapper[2558]: I0710 18:37:52.121456 2558 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-console-operator_console-operator-7866497494-26542_2b29fb39-89b3-42c3-b334-3a195242248c/console-operator/1.log" Jul 10 18:37:52.126269 ip-10-0-128-39 kubenswrapper[2558]: I0710 18:37:52.126246 2558 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ovn-kubernetes_ovnkube-node-ll44q_eabc6c69-b1e8-40d5-b41f-f49778a5248c/ovn-acl-logging/0.log" Jul 10 18:37:52.128074 ip-10-0-128-39 kubenswrapper[2558]: I0710 18:37:52.128052 2558 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-console-operator_console-operator-7866497494-26542_2b29fb39-89b3-42c3-b334-3a195242248c/console-operator/1.log" Jul 10 18:37:52.132622 ip-10-0-128-39 kubenswrapper[2558]: I0710 18:37:52.132603 2558 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ovn-kubernetes_ovnkube-node-ll44q_eabc6c69-b1e8-40d5-b41f-f49778a5248c/ovn-acl-logging/0.log" Jul 10 18:38:21.094326 ip-10-0-128-39 kubenswrapper[2558]: I0710 18:38:21.094287 2558 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-hjf5z"] Jul 10 18:38:43.287827 ip-10-0-128-39 kubenswrapper[2558]: I0710 18:38:43.287778 2558 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-hjf5z"] Jul 10 18:38:57.391079 ip-10-0-128-39 kubenswrapper[2558]: I0710 18:38:57.390994 2558 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-hjf5z"] Jul 10 18:39:12.589851 ip-10-0-128-39 kubenswrapper[2558]: I0710 18:39:12.589798 2558 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-hjf5z"] Jul 10 18:39:26.783434 ip-10-0-128-39 kubenswrapper[2558]: I0710 18:39:26.783399 2558 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-hjf5z"] Jul 10 18:39:40.886905 ip-10-0-128-39 kubenswrapper[2558]: I0710 18:39:40.886866 2558 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-hjf5z"] Jul 10 18:39:59.679712 ip-10-0-128-39 kubenswrapper[2558]: I0710 18:39:59.679672 2558 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-hjf5z"] Jul 10 18:40:18.891718 ip-10-0-128-39 kubenswrapper[2558]: I0710 18:40:18.891630 2558 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-hjf5z"] Jul 10 18:40:38.689497 ip-10-0-128-39 kubenswrapper[2558]: I0710 18:40:38.689457 2558 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-hjf5z"] Jul 10 18:40:55.086034 ip-10-0-128-39 kubenswrapper[2558]: I0710 18:40:55.085999 2558 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-hjf5z"] Jul 10 18:41:34.787891 ip-10-0-128-39 kubenswrapper[2558]: I0710 18:41:34.787852 2558 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-hjf5z"] Jul 10 18:41:50.983287 ip-10-0-128-39 kubenswrapper[2558]: I0710 18:41:50.983249 2558 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-hjf5z"] Jul 10 18:42:12.780981 ip-10-0-128-39 kubenswrapper[2558]: I0710 18:42:12.780947 2558 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-hjf5z"] Jul 10 18:42:43.185166 ip-10-0-128-39 kubenswrapper[2558]: I0710 18:42:43.185132 2558 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-hjf5z"] Jul 10 18:42:52.148830 ip-10-0-128-39 kubenswrapper[2558]: I0710 18:42:52.148777 2558 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-console-operator_console-operator-7866497494-26542_2b29fb39-89b3-42c3-b334-3a195242248c/console-operator/1.log" Jul 10 18:42:52.153779 ip-10-0-128-39 kubenswrapper[2558]: I0710 18:42:52.153756 2558 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ovn-kubernetes_ovnkube-node-ll44q_eabc6c69-b1e8-40d5-b41f-f49778a5248c/ovn-acl-logging/0.log" Jul 10 18:42:52.157380 ip-10-0-128-39 kubenswrapper[2558]: I0710 18:42:52.157359 2558 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-console-operator_console-operator-7866497494-26542_2b29fb39-89b3-42c3-b334-3a195242248c/console-operator/1.log" Jul 10 18:42:52.161746 ip-10-0-128-39 kubenswrapper[2558]: I0710 18:42:52.161728 2558 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ovn-kubernetes_ovnkube-node-ll44q_eabc6c69-b1e8-40d5-b41f-f49778a5248c/ovn-acl-logging/0.log" Jul 10 18:43:04.812826 ip-10-0-128-39 kubenswrapper[2558]: I0710 18:43:04.812778 2558 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-hjf5z"] Jul 10 18:43:26.982123 ip-10-0-128-39 kubenswrapper[2558]: I0710 18:43:26.982088 2558 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-hjf5z"] Jul 10 18:43:44.507370 ip-10-0-128-39 kubenswrapper[2558]: I0710 18:43:44.507335 2558 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-hjf5z"] Jul 10 18:43:53.348762 ip-10-0-128-39 kubenswrapper[2558]: I0710 18:43:53.348728 2558 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-ingress/test-curl-7380-139772316421808"] Jul 10 18:43:53.352165 ip-10-0-128-39 kubenswrapper[2558]: I0710 18:43:53.352149 2558 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/test-curl-7380-139772316421808" Jul 10 18:43:53.354761 ip-10-0-128-39 kubenswrapper[2558]: I0710 18:43:53.354742 2558 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress\"/\"default-dockercfg-kjqpt\"" Jul 10 18:43:53.357368 ip-10-0-128-39 kubenswrapper[2558]: I0710 18:43:53.357344 2558 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/test-curl-7380-139772316421808"] Jul 10 18:43:53.410418 ip-10-0-128-39 kubenswrapper[2558]: I0710 18:43:53.410386 2558 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-lrtg6\" (UniqueName: \"kubernetes.io/projected/c278f27d-6604-42d8-8fb5-e049537b3705-kube-api-access-lrtg6\") pod \"test-curl-7380-139772316421808\" (UID: \"c278f27d-6604-42d8-8fb5-e049537b3705\") " pod="openshift-ingress/test-curl-7380-139772316421808" Jul 10 18:43:53.511732 ip-10-0-128-39 kubenswrapper[2558]: I0710 18:43:53.511694 2558 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-lrtg6\" (UniqueName: \"kubernetes.io/projected/c278f27d-6604-42d8-8fb5-e049537b3705-kube-api-access-lrtg6\") pod \"test-curl-7380-139772316421808\" (UID: \"c278f27d-6604-42d8-8fb5-e049537b3705\") " pod="openshift-ingress/test-curl-7380-139772316421808" Jul 10 18:43:53.521369 ip-10-0-128-39 kubenswrapper[2558]: I0710 18:43:53.521331 2558 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-lrtg6\" (UniqueName: \"kubernetes.io/projected/c278f27d-6604-42d8-8fb5-e049537b3705-kube-api-access-lrtg6\") pod \"test-curl-7380-139772316421808\" (UID: \"c278f27d-6604-42d8-8fb5-e049537b3705\") " pod="openshift-ingress/test-curl-7380-139772316421808" Jul 10 18:43:53.662194 ip-10-0-128-39 kubenswrapper[2558]: I0710 18:43:53.662100 2558 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/test-curl-7380-139772316421808" Jul 10 18:43:53.791578 ip-10-0-128-39 kubenswrapper[2558]: I0710 18:43:53.791531 2558 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/test-curl-7380-139772316421808"] Jul 10 18:43:53.793795 ip-10-0-128-39 kubenswrapper[2558]: W0710 18:43:53.793768 2558 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-besteffort.slice/kubepods-besteffort-podc278f27d_6604_42d8_8fb5_e049537b3705.slice/crio-14575a84a6d0c248a79fad65336d1d001c352eb2c878def460917c8020d221e8 WatchSource:0}: Error finding container 14575a84a6d0c248a79fad65336d1d001c352eb2c878def460917c8020d221e8: Status 404 returned error can't find the container with id 14575a84a6d0c248a79fad65336d1d001c352eb2c878def460917c8020d221e8 Jul 10 18:43:53.795172 ip-10-0-128-39 kubenswrapper[2558]: I0710 18:43:53.795152 2558 provider.go:93] Refreshing cache for provider: *credentialprovider.defaultDockerConfigProvider Jul 10 18:43:54.558644 ip-10-0-128-39 kubenswrapper[2558]: I0710 18:43:54.558601 2558 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/test-curl-7380-139772316421808" event={"ID":"c278f27d-6604-42d8-8fb5-e049537b3705","Type":"ContainerStarted","Data":"c23466df766d69c300f5c6562a2701e11c5b07fb025a2f1790df21be819ab640"} Jul 10 18:43:54.559002 ip-10-0-128-39 kubenswrapper[2558]: I0710 18:43:54.558653 2558 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/test-curl-7380-139772316421808" event={"ID":"c278f27d-6604-42d8-8fb5-e049537b3705","Type":"ContainerStarted","Data":"14575a84a6d0c248a79fad65336d1d001c352eb2c878def460917c8020d221e8"} Jul 10 18:43:54.578660 ip-10-0-128-39 kubenswrapper[2558]: I0710 18:43:54.578616 2558 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-ingress/test-curl-7380-139772316421808" podStartSLOduration=0.892591627 podStartE2EDuration="1.578595951s" totalImagesPullingTime="686.004324ms" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-10 18:43:53 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-10 18:43:54.576895968 +0000 UTC m=+2463.080388474" watchObservedRunningTime="2026-07-10 18:43:54.578595951 +0000 UTC m=+2463.082088456" Jul 10 18:43:54.695249 ip-10-0-128-39 kubenswrapper[2558]: I0710 18:43:54.695218 2558 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ingress_test-curl-7380-139772316421808_c278f27d-6604-42d8-8fb5-e049537b3705/test-curl-7380-139772316421808/0.log" Jul 10 18:43:55.564713 ip-10-0-128-39 kubenswrapper[2558]: I0710 18:43:55.564672 2558 generic.go:356] "Generic (PLEG): container finished" podID="c278f27d-6604-42d8-8fb5-e049537b3705" containerID="c23466df766d69c300f5c6562a2701e11c5b07fb025a2f1790df21be819ab640" exitCode=0 Jul 10 18:43:55.565106 ip-10-0-128-39 kubenswrapper[2558]: I0710 18:43:55.564758 2558 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/test-curl-7380-139772316421808" event={"ID":"c278f27d-6604-42d8-8fb5-e049537b3705","Type":"ContainerDied","Data":"c23466df766d69c300f5c6562a2701e11c5b07fb025a2f1790df21be819ab640"} Jul 10 18:43:56.710579 ip-10-0-128-39 kubenswrapper[2558]: I0710 18:43:56.710551 2558 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/test-curl-7380-139772316421808" Jul 10 18:43:56.742750 ip-10-0-128-39 kubenswrapper[2558]: I0710 18:43:56.742722 2558 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-ingress/test-curl-7380-139772316421808"] Jul 10 18:43:56.746699 ip-10-0-128-39 kubenswrapper[2558]: I0710 18:43:56.746673 2558 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["openshift-ingress/test-curl-7380-139772316421808"] Jul 10 18:43:56.841649 ip-10-0-128-39 kubenswrapper[2558]: I0710 18:43:56.841567 2558 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-lrtg6\" (UniqueName: \"kubernetes.io/projected/c278f27d-6604-42d8-8fb5-e049537b3705-kube-api-access-lrtg6\") pod \"c278f27d-6604-42d8-8fb5-e049537b3705\" (UID: \"c278f27d-6604-42d8-8fb5-e049537b3705\") " Jul 10 18:43:56.843617 ip-10-0-128-39 kubenswrapper[2558]: I0710 18:43:56.843584 2558 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/c278f27d-6604-42d8-8fb5-e049537b3705-kube-api-access-lrtg6" (OuterVolumeSpecName: "kube-api-access-lrtg6") pod "c278f27d-6604-42d8-8fb5-e049537b3705" (UID: "c278f27d-6604-42d8-8fb5-e049537b3705"). InnerVolumeSpecName "kube-api-access-lrtg6". PluginName "kubernetes.io/projected", VolumeGIDValue "" Jul 10 18:43:56.864105 ip-10-0-128-39 kubenswrapper[2558]: I0710 18:43:56.864075 2558 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-ingress/test-curl-7380-139772316419504"] Jul 10 18:43:56.864457 ip-10-0-128-39 kubenswrapper[2558]: I0710 18:43:56.864445 2558 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="c278f27d-6604-42d8-8fb5-e049537b3705" containerName="test-curl-7380-139772316421808" Jul 10 18:43:56.864519 ip-10-0-128-39 kubenswrapper[2558]: I0710 18:43:56.864459 2558 state_mem.go:107] "Deleted CPUSet assignment" podUID="c278f27d-6604-42d8-8fb5-e049537b3705" containerName="test-curl-7380-139772316421808" Jul 10 18:43:56.864519 ip-10-0-128-39 kubenswrapper[2558]: I0710 18:43:56.864515 2558 memory_manager.go:368] "RemoveStaleState removing state" podUID="c278f27d-6604-42d8-8fb5-e049537b3705" containerName="test-curl-7380-139772316421808" Jul 10 18:43:56.867351 ip-10-0-128-39 kubenswrapper[2558]: I0710 18:43:56.867337 2558 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/test-curl-7380-139772316419504" Jul 10 18:43:56.873723 ip-10-0-128-39 kubenswrapper[2558]: I0710 18:43:56.873688 2558 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/test-curl-7380-139772316419504"] Jul 10 18:43:56.942229 ip-10-0-128-39 kubenswrapper[2558]: I0710 18:43:56.942197 2558 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-6btc8\" (UniqueName: \"kubernetes.io/projected/84d2f9a1-945d-459e-a113-f04a428bd7e2-kube-api-access-6btc8\") pod \"test-curl-7380-139772316419504\" (UID: \"84d2f9a1-945d-459e-a113-f04a428bd7e2\") " pod="openshift-ingress/test-curl-7380-139772316419504" Jul 10 18:43:56.942393 ip-10-0-128-39 kubenswrapper[2558]: I0710 18:43:56.942265 2558 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-lrtg6\" (UniqueName: \"kubernetes.io/projected/c278f27d-6604-42d8-8fb5-e049537b3705-kube-api-access-lrtg6\") on node \"ip-10-0-128-39.ec2.internal\" DevicePath \"\"" Jul 10 18:43:57.043710 ip-10-0-128-39 kubenswrapper[2558]: I0710 18:43:57.043668 2558 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-6btc8\" (UniqueName: \"kubernetes.io/projected/84d2f9a1-945d-459e-a113-f04a428bd7e2-kube-api-access-6btc8\") pod \"test-curl-7380-139772316419504\" (UID: \"84d2f9a1-945d-459e-a113-f04a428bd7e2\") " pod="openshift-ingress/test-curl-7380-139772316419504" Jul 10 18:43:57.051956 ip-10-0-128-39 kubenswrapper[2558]: I0710 18:43:57.051931 2558 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-6btc8\" (UniqueName: \"kubernetes.io/projected/84d2f9a1-945d-459e-a113-f04a428bd7e2-kube-api-access-6btc8\") pod \"test-curl-7380-139772316419504\" (UID: \"84d2f9a1-945d-459e-a113-f04a428bd7e2\") " pod="openshift-ingress/test-curl-7380-139772316419504" Jul 10 18:43:57.177841 ip-10-0-128-39 kubenswrapper[2558]: I0710 18:43:57.177721 2558 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/test-curl-7380-139772316419504" Jul 10 18:43:57.308948 ip-10-0-128-39 kubenswrapper[2558]: I0710 18:43:57.308924 2558 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/test-curl-7380-139772316419504"] Jul 10 18:43:57.310589 ip-10-0-128-39 kubenswrapper[2558]: W0710 18:43:57.310552 2558 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-besteffort.slice/kubepods-besteffort-pod84d2f9a1_945d_459e_a113_f04a428bd7e2.slice/crio-ce1a9995f91ef487b4624794a74fda85e2c30b9fddb68758abcf3a24f2ec2943 WatchSource:0}: Error finding container ce1a9995f91ef487b4624794a74fda85e2c30b9fddb68758abcf3a24f2ec2943: Status 404 returned error can't find the container with id ce1a9995f91ef487b4624794a74fda85e2c30b9fddb68758abcf3a24f2ec2943 Jul 10 18:43:57.574606 ip-10-0-128-39 kubenswrapper[2558]: I0710 18:43:57.574570 2558 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/test-curl-7380-139772316419504" event={"ID":"84d2f9a1-945d-459e-a113-f04a428bd7e2","Type":"ContainerStarted","Data":"ce1a9995f91ef487b4624794a74fda85e2c30b9fddb68758abcf3a24f2ec2943"} Jul 10 18:43:57.575910 ip-10-0-128-39 kubenswrapper[2558]: I0710 18:43:57.575892 2558 pod_container_deletor.go:80] "Container not found in pod's containers" containerID="14575a84a6d0c248a79fad65336d1d001c352eb2c878def460917c8020d221e8" Jul 10 18:43:57.576019 ip-10-0-128-39 kubenswrapper[2558]: I0710 18:43:57.575927 2558 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/test-curl-7380-139772316421808" Jul 10 18:43:57.585955 ip-10-0-128-39 kubenswrapper[2558]: E0710 18:43:57.585927 2558 status_manager.go:1018] "Failed to get status for pod" err="pods \"test-curl-7380-139772316421808\" is forbidden: User \"system:node:ip-10-0-128-39.ec2.internal\" cannot get resource \"pods\" in API group \"\" in the namespace \"openshift-ingress\": no relationship found between node 'ip-10-0-128-39.ec2.internal' and this object" podUID="c278f27d-6604-42d8-8fb5-e049537b3705" pod="openshift-ingress/test-curl-7380-139772316421808" Jul 10 18:43:58.101147 ip-10-0-128-39 kubenswrapper[2558]: I0710 18:43:58.101114 2558 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="c278f27d-6604-42d8-8fb5-e049537b3705" path="/var/lib/kubelet/pods/c278f27d-6604-42d8-8fb5-e049537b3705/volumes" Jul 10 18:43:58.582287 ip-10-0-128-39 kubenswrapper[2558]: I0710 18:43:58.582255 2558 generic.go:356] "Generic (PLEG): container finished" podID="84d2f9a1-945d-459e-a113-f04a428bd7e2" containerID="24c44bf2a7dfa0d96cab69d5ac45c659ba4c49f532fed606df155f729a7b5377" exitCode=0 Jul 10 18:43:58.582471 ip-10-0-128-39 kubenswrapper[2558]: I0710 18:43:58.582298 2558 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/test-curl-7380-139772316419504" event={"ID":"84d2f9a1-945d-459e-a113-f04a428bd7e2","Type":"ContainerDied","Data":"24c44bf2a7dfa0d96cab69d5ac45c659ba4c49f532fed606df155f729a7b5377"} Jul 10 18:43:59.730093 ip-10-0-128-39 kubenswrapper[2558]: I0710 18:43:59.730068 2558 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/test-curl-7380-139772316419504" Jul 10 18:43:59.750111 ip-10-0-128-39 kubenswrapper[2558]: I0710 18:43:59.750087 2558 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ingress_test-curl-7380-139772316419504_84d2f9a1-945d-459e-a113-f04a428bd7e2/test-curl-7380-139772316419504/0.log" Jul 10 18:43:59.772458 ip-10-0-128-39 kubenswrapper[2558]: I0710 18:43:59.772422 2558 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-ingress/test-curl-7380-139772316419504"] Jul 10 18:43:59.777458 ip-10-0-128-39 kubenswrapper[2558]: I0710 18:43:59.777434 2558 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["openshift-ingress/test-curl-7380-139772316419504"] Jul 10 18:43:59.868780 ip-10-0-128-39 kubenswrapper[2558]: I0710 18:43:59.868681 2558 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-6btc8\" (UniqueName: \"kubernetes.io/projected/84d2f9a1-945d-459e-a113-f04a428bd7e2-kube-api-access-6btc8\") pod \"84d2f9a1-945d-459e-a113-f04a428bd7e2\" (UID: \"84d2f9a1-945d-459e-a113-f04a428bd7e2\") " Jul 10 18:43:59.870734 ip-10-0-128-39 kubenswrapper[2558]: I0710 18:43:59.870705 2558 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/84d2f9a1-945d-459e-a113-f04a428bd7e2-kube-api-access-6btc8" (OuterVolumeSpecName: "kube-api-access-6btc8") pod "84d2f9a1-945d-459e-a113-f04a428bd7e2" (UID: "84d2f9a1-945d-459e-a113-f04a428bd7e2"). InnerVolumeSpecName "kube-api-access-6btc8". PluginName "kubernetes.io/projected", VolumeGIDValue "" Jul 10 18:43:59.928458 ip-10-0-128-39 kubenswrapper[2558]: I0710 18:43:59.928428 2558 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-ingress/test-curl-7380-139772321404800"] Jul 10 18:43:59.928790 ip-10-0-128-39 kubenswrapper[2558]: I0710 18:43:59.928776 2558 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="84d2f9a1-945d-459e-a113-f04a428bd7e2" containerName="test-curl-7380-139772316419504" Jul 10 18:43:59.928873 ip-10-0-128-39 kubenswrapper[2558]: I0710 18:43:59.928792 2558 state_mem.go:107] "Deleted CPUSet assignment" podUID="84d2f9a1-945d-459e-a113-f04a428bd7e2" containerName="test-curl-7380-139772316419504" Jul 10 18:43:59.928913 ip-10-0-128-39 kubenswrapper[2558]: I0710 18:43:59.928893 2558 memory_manager.go:368] "RemoveStaleState removing state" podUID="84d2f9a1-945d-459e-a113-f04a428bd7e2" containerName="test-curl-7380-139772316419504" Jul 10 18:43:59.931694 ip-10-0-128-39 kubenswrapper[2558]: I0710 18:43:59.931680 2558 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/test-curl-7380-139772321404800" Jul 10 18:43:59.936709 ip-10-0-128-39 kubenswrapper[2558]: I0710 18:43:59.936686 2558 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/test-curl-7380-139772321404800"] Jul 10 18:43:59.969524 ip-10-0-128-39 kubenswrapper[2558]: I0710 18:43:59.969497 2558 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-6btc8\" (UniqueName: \"kubernetes.io/projected/84d2f9a1-945d-459e-a113-f04a428bd7e2-kube-api-access-6btc8\") on node \"ip-10-0-128-39.ec2.internal\" DevicePath \"\"" Jul 10 18:44:00.070579 ip-10-0-128-39 kubenswrapper[2558]: I0710 18:44:00.070539 2558 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-sq9vl\" (UniqueName: \"kubernetes.io/projected/15ab0755-d1aa-4a2d-9646-a157c213d26b-kube-api-access-sq9vl\") pod \"test-curl-7380-139772321404800\" (UID: \"15ab0755-d1aa-4a2d-9646-a157c213d26b\") " pod="openshift-ingress/test-curl-7380-139772321404800" Jul 10 18:44:00.101944 ip-10-0-128-39 kubenswrapper[2558]: I0710 18:44:00.101913 2558 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="84d2f9a1-945d-459e-a113-f04a428bd7e2" path="/var/lib/kubelet/pods/84d2f9a1-945d-459e-a113-f04a428bd7e2/volumes" Jul 10 18:44:00.171937 ip-10-0-128-39 kubenswrapper[2558]: I0710 18:44:00.171842 2558 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-sq9vl\" (UniqueName: \"kubernetes.io/projected/15ab0755-d1aa-4a2d-9646-a157c213d26b-kube-api-access-sq9vl\") pod \"test-curl-7380-139772321404800\" (UID: \"15ab0755-d1aa-4a2d-9646-a157c213d26b\") " pod="openshift-ingress/test-curl-7380-139772321404800" Jul 10 18:44:00.180794 ip-10-0-128-39 kubenswrapper[2558]: I0710 18:44:00.180764 2558 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-sq9vl\" (UniqueName: \"kubernetes.io/projected/15ab0755-d1aa-4a2d-9646-a157c213d26b-kube-api-access-sq9vl\") pod \"test-curl-7380-139772321404800\" (UID: \"15ab0755-d1aa-4a2d-9646-a157c213d26b\") " pod="openshift-ingress/test-curl-7380-139772321404800" Jul 10 18:44:00.241383 ip-10-0-128-39 kubenswrapper[2558]: I0710 18:44:00.241356 2558 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/test-curl-7380-139772321404800" Jul 10 18:44:00.377793 ip-10-0-128-39 kubenswrapper[2558]: I0710 18:44:00.377771 2558 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/test-curl-7380-139772321404800"] Jul 10 18:44:00.380327 ip-10-0-128-39 kubenswrapper[2558]: W0710 18:44:00.380301 2558 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-besteffort.slice/kubepods-besteffort-pod15ab0755_d1aa_4a2d_9646_a157c213d26b.slice/crio-65660fcec010dd4942f1932a23ff33cccda6a94402b2e22068c5b3f4b27ff5cd WatchSource:0}: Error finding container 65660fcec010dd4942f1932a23ff33cccda6a94402b2e22068c5b3f4b27ff5cd: Status 404 returned error can't find the container with id 65660fcec010dd4942f1932a23ff33cccda6a94402b2e22068c5b3f4b27ff5cd Jul 10 18:44:00.483526 ip-10-0-128-39 kubenswrapper[2558]: I0710 18:44:00.483487 2558 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-hjf5z"] Jul 10 18:44:00.598642 ip-10-0-128-39 kubenswrapper[2558]: I0710 18:44:00.598600 2558 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/test-curl-7380-139772321404800" event={"ID":"15ab0755-d1aa-4a2d-9646-a157c213d26b","Type":"ContainerStarted","Data":"65660fcec010dd4942f1932a23ff33cccda6a94402b2e22068c5b3f4b27ff5cd"} Jul 10 18:44:00.599789 ip-10-0-128-39 kubenswrapper[2558]: I0710 18:44:00.599763 2558 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/test-curl-7380-139772316419504" Jul 10 18:44:00.599789 ip-10-0-128-39 kubenswrapper[2558]: I0710 18:44:00.599775 2558 scope.go:117] "RemoveContainer" containerID="24c44bf2a7dfa0d96cab69d5ac45c659ba4c49f532fed606df155f729a7b5377" Jul 10 18:44:01.606564 ip-10-0-128-39 kubenswrapper[2558]: I0710 18:44:01.606530 2558 generic.go:356] "Generic (PLEG): container finished" podID="15ab0755-d1aa-4a2d-9646-a157c213d26b" containerID="8afc7e591f743793db1ec323873366bddc32ca0fdcccb8868f149a16c979efd7" exitCode=0 Jul 10 18:44:01.607051 ip-10-0-128-39 kubenswrapper[2558]: I0710 18:44:01.606613 2558 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/test-curl-7380-139772321404800" event={"ID":"15ab0755-d1aa-4a2d-9646-a157c213d26b","Type":"ContainerDied","Data":"8afc7e591f743793db1ec323873366bddc32ca0fdcccb8868f149a16c979efd7"} Jul 10 18:44:02.744371 ip-10-0-128-39 kubenswrapper[2558]: I0710 18:44:02.744350 2558 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/test-curl-7380-139772321404800" Jul 10 18:44:02.765357 ip-10-0-128-39 kubenswrapper[2558]: I0710 18:44:02.765325 2558 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ingress_test-curl-7380-139772321404800_15ab0755-d1aa-4a2d-9646-a157c213d26b/test-curl-7380-139772321404800/0.log" Jul 10 18:44:02.794596 ip-10-0-128-39 kubenswrapper[2558]: I0710 18:44:02.794567 2558 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-ingress/test-curl-7380-139772321404800"] Jul 10 18:44:02.797787 ip-10-0-128-39 kubenswrapper[2558]: I0710 18:44:02.797762 2558 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["openshift-ingress/test-curl-7380-139772321404800"] Jul 10 18:44:02.894613 ip-10-0-128-39 kubenswrapper[2558]: I0710 18:44:02.894522 2558 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-sq9vl\" (UniqueName: \"kubernetes.io/projected/15ab0755-d1aa-4a2d-9646-a157c213d26b-kube-api-access-sq9vl\") pod \"15ab0755-d1aa-4a2d-9646-a157c213d26b\" (UID: \"15ab0755-d1aa-4a2d-9646-a157c213d26b\") " Jul 10 18:44:02.896618 ip-10-0-128-39 kubenswrapper[2558]: I0710 18:44:02.896592 2558 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/15ab0755-d1aa-4a2d-9646-a157c213d26b-kube-api-access-sq9vl" (OuterVolumeSpecName: "kube-api-access-sq9vl") pod "15ab0755-d1aa-4a2d-9646-a157c213d26b" (UID: "15ab0755-d1aa-4a2d-9646-a157c213d26b"). InnerVolumeSpecName "kube-api-access-sq9vl". PluginName "kubernetes.io/projected", VolumeGIDValue "" Jul 10 18:44:02.917636 ip-10-0-128-39 kubenswrapper[2558]: I0710 18:44:02.917603 2558 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-ingress/test-curl-7380-139772321404800"] Jul 10 18:44:02.918003 ip-10-0-128-39 kubenswrapper[2558]: I0710 18:44:02.917990 2558 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="15ab0755-d1aa-4a2d-9646-a157c213d26b" containerName="test-curl-7380-139772321404800" Jul 10 18:44:02.918055 ip-10-0-128-39 kubenswrapper[2558]: I0710 18:44:02.918004 2558 state_mem.go:107] "Deleted CPUSet assignment" podUID="15ab0755-d1aa-4a2d-9646-a157c213d26b" containerName="test-curl-7380-139772321404800" Jul 10 18:44:02.918114 ip-10-0-128-39 kubenswrapper[2558]: I0710 18:44:02.918074 2558 memory_manager.go:368] "RemoveStaleState removing state" podUID="15ab0755-d1aa-4a2d-9646-a157c213d26b" containerName="test-curl-7380-139772321404800" Jul 10 18:44:02.920892 ip-10-0-128-39 kubenswrapper[2558]: I0710 18:44:02.920877 2558 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/test-curl-7380-139772321404800" Jul 10 18:44:02.930123 ip-10-0-128-39 kubenswrapper[2558]: I0710 18:44:02.930101 2558 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/test-curl-7380-139772321404800"] Jul 10 18:44:02.995348 ip-10-0-128-39 kubenswrapper[2558]: I0710 18:44:02.995315 2558 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-wdn9l\" (UniqueName: \"kubernetes.io/projected/558b4dba-812d-4870-b4d3-243e08f07d95-kube-api-access-wdn9l\") pod \"test-curl-7380-139772321404800\" (UID: \"558b4dba-812d-4870-b4d3-243e08f07d95\") " pod="openshift-ingress/test-curl-7380-139772321404800" Jul 10 18:44:02.995506 ip-10-0-128-39 kubenswrapper[2558]: I0710 18:44:02.995421 2558 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-sq9vl\" (UniqueName: \"kubernetes.io/projected/15ab0755-d1aa-4a2d-9646-a157c213d26b-kube-api-access-sq9vl\") on node \"ip-10-0-128-39.ec2.internal\" DevicePath \"\"" Jul 10 18:44:03.096774 ip-10-0-128-39 kubenswrapper[2558]: I0710 18:44:03.096737 2558 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-wdn9l\" (UniqueName: \"kubernetes.io/projected/558b4dba-812d-4870-b4d3-243e08f07d95-kube-api-access-wdn9l\") pod \"test-curl-7380-139772321404800\" (UID: \"558b4dba-812d-4870-b4d3-243e08f07d95\") " pod="openshift-ingress/test-curl-7380-139772321404800" Jul 10 18:44:03.105896 ip-10-0-128-39 kubenswrapper[2558]: I0710 18:44:03.105863 2558 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-wdn9l\" (UniqueName: \"kubernetes.io/projected/558b4dba-812d-4870-b4d3-243e08f07d95-kube-api-access-wdn9l\") pod \"test-curl-7380-139772321404800\" (UID: \"558b4dba-812d-4870-b4d3-243e08f07d95\") " pod="openshift-ingress/test-curl-7380-139772321404800" Jul 10 18:44:03.229996 ip-10-0-128-39 kubenswrapper[2558]: I0710 18:44:03.229953 2558 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/test-curl-7380-139772321404800" Jul 10 18:44:03.362361 ip-10-0-128-39 kubenswrapper[2558]: I0710 18:44:03.362337 2558 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/test-curl-7380-139772321404800"] Jul 10 18:44:03.364325 ip-10-0-128-39 kubenswrapper[2558]: W0710 18:44:03.364292 2558 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-besteffort.slice/kubepods-besteffort-pod558b4dba_812d_4870_b4d3_243e08f07d95.slice/crio-d9220f5dc048d0f78412fd55b0baa8d82e2bc9ff181107e790fc1c0ee49247b7 WatchSource:0}: Error finding container d9220f5dc048d0f78412fd55b0baa8d82e2bc9ff181107e790fc1c0ee49247b7: Status 404 returned error can't find the container with id d9220f5dc048d0f78412fd55b0baa8d82e2bc9ff181107e790fc1c0ee49247b7 Jul 10 18:44:03.618666 ip-10-0-128-39 kubenswrapper[2558]: I0710 18:44:03.618581 2558 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/test-curl-7380-139772321404800" Jul 10 18:44:03.618666 ip-10-0-128-39 kubenswrapper[2558]: I0710 18:44:03.618600 2558 pod_container_deletor.go:80] "Container not found in pod's containers" containerID="65660fcec010dd4942f1932a23ff33cccda6a94402b2e22068c5b3f4b27ff5cd" Jul 10 18:44:03.619961 ip-10-0-128-39 kubenswrapper[2558]: I0710 18:44:03.619918 2558 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/test-curl-7380-139772321404800" event={"ID":"558b4dba-812d-4870-b4d3-243e08f07d95","Type":"ContainerStarted","Data":"d9220f5dc048d0f78412fd55b0baa8d82e2bc9ff181107e790fc1c0ee49247b7"} Jul 10 18:44:03.631359 ip-10-0-128-39 kubenswrapper[2558]: I0710 18:44:03.631328 2558 status_manager.go:1027] "Pod was deleted and then recreated, skipping status update" pod="openshift-ingress/test-curl-7380-139772321404800" oldPodUID="15ab0755-d1aa-4a2d-9646-a157c213d26b" podUID="558b4dba-812d-4870-b4d3-243e08f07d95" Jul 10 18:44:04.103723 ip-10-0-128-39 kubenswrapper[2558]: I0710 18:44:04.103689 2558 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="15ab0755-d1aa-4a2d-9646-a157c213d26b" path="/var/lib/kubelet/pods/15ab0755-d1aa-4a2d-9646-a157c213d26b/volumes" Jul 10 18:44:04.626338 ip-10-0-128-39 kubenswrapper[2558]: I0710 18:44:04.626302 2558 generic.go:356] "Generic (PLEG): container finished" podID="558b4dba-812d-4870-b4d3-243e08f07d95" containerID="b9db396386786dfd94d6d828d44784abf691d1bdfdafdef7778b60b75837f27f" exitCode=0 Jul 10 18:44:04.626512 ip-10-0-128-39 kubenswrapper[2558]: I0710 18:44:04.626394 2558 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/test-curl-7380-139772321404800" event={"ID":"558b4dba-812d-4870-b4d3-243e08f07d95","Type":"ContainerDied","Data":"b9db396386786dfd94d6d828d44784abf691d1bdfdafdef7778b60b75837f27f"} Jul 10 18:44:05.774283 ip-10-0-128-39 kubenswrapper[2558]: I0710 18:44:05.774257 2558 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/test-curl-7380-139772321404800" Jul 10 18:44:05.808872 ip-10-0-128-39 kubenswrapper[2558]: I0710 18:44:05.808842 2558 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ingress_test-curl-7380-139772321404800_558b4dba-812d-4870-b4d3-243e08f07d95/test-curl-7380-139772321404800/0.log" Jul 10 18:44:05.845790 ip-10-0-128-39 kubenswrapper[2558]: I0710 18:44:05.845761 2558 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-ingress/test-curl-7380-139772321404800"] Jul 10 18:44:05.850361 ip-10-0-128-39 kubenswrapper[2558]: I0710 18:44:05.850336 2558 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["openshift-ingress/test-curl-7380-139772321404800"] Jul 10 18:44:05.923001 ip-10-0-128-39 kubenswrapper[2558]: I0710 18:44:05.922924 2558 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-wdn9l\" (UniqueName: \"kubernetes.io/projected/558b4dba-812d-4870-b4d3-243e08f07d95-kube-api-access-wdn9l\") pod \"558b4dba-812d-4870-b4d3-243e08f07d95\" (UID: \"558b4dba-812d-4870-b4d3-243e08f07d95\") " Jul 10 18:44:05.924898 ip-10-0-128-39 kubenswrapper[2558]: I0710 18:44:05.924869 2558 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/558b4dba-812d-4870-b4d3-243e08f07d95-kube-api-access-wdn9l" (OuterVolumeSpecName: "kube-api-access-wdn9l") pod "558b4dba-812d-4870-b4d3-243e08f07d95" (UID: "558b4dba-812d-4870-b4d3-243e08f07d95"). InnerVolumeSpecName "kube-api-access-wdn9l". PluginName "kubernetes.io/projected", VolumeGIDValue "" Jul 10 18:44:05.989145 ip-10-0-128-39 kubenswrapper[2558]: I0710 18:44:05.989110 2558 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-ingress/test-curl-7380-139772321405808"] Jul 10 18:44:05.989465 ip-10-0-128-39 kubenswrapper[2558]: I0710 18:44:05.989454 2558 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="558b4dba-812d-4870-b4d3-243e08f07d95" containerName="test-curl-7380-139772321404800" Jul 10 18:44:05.989521 ip-10-0-128-39 kubenswrapper[2558]: I0710 18:44:05.989467 2558 state_mem.go:107] "Deleted CPUSet assignment" podUID="558b4dba-812d-4870-b4d3-243e08f07d95" containerName="test-curl-7380-139772321404800" Jul 10 18:44:05.989521 ip-10-0-128-39 kubenswrapper[2558]: I0710 18:44:05.989518 2558 memory_manager.go:368] "RemoveStaleState removing state" podUID="558b4dba-812d-4870-b4d3-243e08f07d95" containerName="test-curl-7380-139772321404800" Jul 10 18:44:05.992321 ip-10-0-128-39 kubenswrapper[2558]: I0710 18:44:05.992303 2558 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/test-curl-7380-139772321405808" Jul 10 18:44:05.999666 ip-10-0-128-39 kubenswrapper[2558]: I0710 18:44:05.999643 2558 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/test-curl-7380-139772321405808"] Jul 10 18:44:06.024371 ip-10-0-128-39 kubenswrapper[2558]: I0710 18:44:06.024339 2558 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-wdn9l\" (UniqueName: \"kubernetes.io/projected/558b4dba-812d-4870-b4d3-243e08f07d95-kube-api-access-wdn9l\") on node \"ip-10-0-128-39.ec2.internal\" DevicePath \"\"" Jul 10 18:44:06.101328 ip-10-0-128-39 kubenswrapper[2558]: I0710 18:44:06.101289 2558 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="558b4dba-812d-4870-b4d3-243e08f07d95" path="/var/lib/kubelet/pods/558b4dba-812d-4870-b4d3-243e08f07d95/volumes" Jul 10 18:44:06.125347 ip-10-0-128-39 kubenswrapper[2558]: I0710 18:44:06.125321 2558 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-4vh27\" (UniqueName: \"kubernetes.io/projected/eccf53c5-36a1-45d7-af34-23e96c86f9e3-kube-api-access-4vh27\") pod \"test-curl-7380-139772321405808\" (UID: \"eccf53c5-36a1-45d7-af34-23e96c86f9e3\") " pod="openshift-ingress/test-curl-7380-139772321405808" Jul 10 18:44:06.226251 ip-10-0-128-39 kubenswrapper[2558]: I0710 18:44:06.226219 2558 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-4vh27\" (UniqueName: \"kubernetes.io/projected/eccf53c5-36a1-45d7-af34-23e96c86f9e3-kube-api-access-4vh27\") pod \"test-curl-7380-139772321405808\" (UID: \"eccf53c5-36a1-45d7-af34-23e96c86f9e3\") " pod="openshift-ingress/test-curl-7380-139772321405808" Jul 10 18:44:06.235223 ip-10-0-128-39 kubenswrapper[2558]: I0710 18:44:06.235198 2558 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-4vh27\" (UniqueName: \"kubernetes.io/projected/eccf53c5-36a1-45d7-af34-23e96c86f9e3-kube-api-access-4vh27\") pod \"test-curl-7380-139772321405808\" (UID: \"eccf53c5-36a1-45d7-af34-23e96c86f9e3\") " pod="openshift-ingress/test-curl-7380-139772321405808" Jul 10 18:44:06.302388 ip-10-0-128-39 kubenswrapper[2558]: I0710 18:44:06.302342 2558 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/test-curl-7380-139772321405808" Jul 10 18:44:06.433576 ip-10-0-128-39 kubenswrapper[2558]: I0710 18:44:06.433551 2558 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/test-curl-7380-139772321405808"] Jul 10 18:44:06.435541 ip-10-0-128-39 kubenswrapper[2558]: W0710 18:44:06.435516 2558 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-besteffort.slice/kubepods-besteffort-podeccf53c5_36a1_45d7_af34_23e96c86f9e3.slice/crio-33301e0fb0a5030137ae6e87a83602abad9f5e62890e53e2b830b3c3bf4d6cc8 WatchSource:0}: Error finding container 33301e0fb0a5030137ae6e87a83602abad9f5e62890e53e2b830b3c3bf4d6cc8: Status 404 returned error can't find the container with id 33301e0fb0a5030137ae6e87a83602abad9f5e62890e53e2b830b3c3bf4d6cc8 Jul 10 18:44:06.637628 ip-10-0-128-39 kubenswrapper[2558]: I0710 18:44:06.637537 2558 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/test-curl-7380-139772321405808" event={"ID":"eccf53c5-36a1-45d7-af34-23e96c86f9e3","Type":"ContainerStarted","Data":"33301e0fb0a5030137ae6e87a83602abad9f5e62890e53e2b830b3c3bf4d6cc8"} Jul 10 18:44:06.638764 ip-10-0-128-39 kubenswrapper[2558]: I0710 18:44:06.638743 2558 scope.go:117] "RemoveContainer" containerID="b9db396386786dfd94d6d828d44784abf691d1bdfdafdef7778b60b75837f27f" Jul 10 18:44:06.638840 ip-10-0-128-39 kubenswrapper[2558]: I0710 18:44:06.638777 2558 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/test-curl-7380-139772321404800" Jul 10 18:44:07.645703 ip-10-0-128-39 kubenswrapper[2558]: I0710 18:44:07.645671 2558 generic.go:356] "Generic (PLEG): container finished" podID="eccf53c5-36a1-45d7-af34-23e96c86f9e3" containerID="3e14ec2cea9075c54be709af1d9d9e1b43260f79c367babdf5915c60ba2b8a7f" exitCode=0 Jul 10 18:44:07.646105 ip-10-0-128-39 kubenswrapper[2558]: I0710 18:44:07.645738 2558 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/test-curl-7380-139772321405808" event={"ID":"eccf53c5-36a1-45d7-af34-23e96c86f9e3","Type":"ContainerDied","Data":"3e14ec2cea9075c54be709af1d9d9e1b43260f79c367babdf5915c60ba2b8a7f"} Jul 10 18:44:08.788086 ip-10-0-128-39 kubenswrapper[2558]: I0710 18:44:08.788058 2558 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/test-curl-7380-139772321405808" Jul 10 18:44:08.810230 ip-10-0-128-39 kubenswrapper[2558]: I0710 18:44:08.810200 2558 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ingress_test-curl-7380-139772321405808_eccf53c5-36a1-45d7-af34-23e96c86f9e3/test-curl-7380-139772321405808/0.log" Jul 10 18:44:08.833020 ip-10-0-128-39 kubenswrapper[2558]: I0710 18:44:08.832985 2558 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-ingress/test-curl-7380-139772321405808"] Jul 10 18:44:08.839496 ip-10-0-128-39 kubenswrapper[2558]: I0710 18:44:08.839459 2558 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["openshift-ingress/test-curl-7380-139772321405808"] Jul 10 18:44:08.948631 ip-10-0-128-39 kubenswrapper[2558]: I0710 18:44:08.948534 2558 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-4vh27\" (UniqueName: \"kubernetes.io/projected/eccf53c5-36a1-45d7-af34-23e96c86f9e3-kube-api-access-4vh27\") pod \"eccf53c5-36a1-45d7-af34-23e96c86f9e3\" (UID: \"eccf53c5-36a1-45d7-af34-23e96c86f9e3\") " Jul 10 18:44:08.950742 ip-10-0-128-39 kubenswrapper[2558]: I0710 18:44:08.950704 2558 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/eccf53c5-36a1-45d7-af34-23e96c86f9e3-kube-api-access-4vh27" (OuterVolumeSpecName: "kube-api-access-4vh27") pod "eccf53c5-36a1-45d7-af34-23e96c86f9e3" (UID: "eccf53c5-36a1-45d7-af34-23e96c86f9e3"). InnerVolumeSpecName "kube-api-access-4vh27". PluginName "kubernetes.io/projected", VolumeGIDValue "" Jul 10 18:44:09.049158 ip-10-0-128-39 kubenswrapper[2558]: I0710 18:44:09.049112 2558 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-4vh27\" (UniqueName: \"kubernetes.io/projected/eccf53c5-36a1-45d7-af34-23e96c86f9e3-kube-api-access-4vh27\") on node \"ip-10-0-128-39.ec2.internal\" DevicePath \"\"" Jul 10 18:44:09.656217 ip-10-0-128-39 kubenswrapper[2558]: I0710 18:44:09.656194 2558 pod_container_deletor.go:80] "Container not found in pod's containers" containerID="33301e0fb0a5030137ae6e87a83602abad9f5e62890e53e2b830b3c3bf4d6cc8" Jul 10 18:44:09.656217 ip-10-0-128-39 kubenswrapper[2558]: I0710 18:44:09.656205 2558 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/test-curl-7380-139772321405808" Jul 10 18:44:09.665891 ip-10-0-128-39 kubenswrapper[2558]: E0710 18:44:09.665862 2558 status_manager.go:1018] "Failed to get status for pod" err="pods \"test-curl-7380-139772321405808\" is forbidden: User \"system:node:ip-10-0-128-39.ec2.internal\" cannot get resource \"pods\" in API group \"\" in the namespace \"openshift-ingress\": no relationship found between node 'ip-10-0-128-39.ec2.internal' and this object" podUID="eccf53c5-36a1-45d7-af34-23e96c86f9e3" pod="openshift-ingress/test-curl-7380-139772321405808" Jul 10 18:44:10.101559 ip-10-0-128-39 kubenswrapper[2558]: I0710 18:44:10.101525 2558 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="eccf53c5-36a1-45d7-af34-23e96c86f9e3" path="/var/lib/kubelet/pods/eccf53c5-36a1-45d7-af34-23e96c86f9e3/volumes" Jul 10 18:44:16.689232 ip-10-0-128-39 kubenswrapper[2558]: I0710 18:44:16.689203 2558 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-hjf5z"] Jul 10 18:44:26.165617 ip-10-0-128-39 kubenswrapper[2558]: I0710 18:44:26.165584 2558 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-ingress/e2e-shared-b-4cfdb5-openshift-default-85569b84ff-swg7c"] Jul 10 18:44:26.166125 ip-10-0-128-39 kubenswrapper[2558]: I0710 18:44:26.165784 2558 kuberuntime_container.go:873] "Killing container with a grace period" pod="openshift-ingress/e2e-shared-b-4cfdb5-openshift-default-85569b84ff-swg7c" podUID="6735c6d1-6741-40ee-901f-e6cfd26d62f2" containerName="istio-proxy" containerID="cri-o://022cd5e08a211713dc2754d5b11aabb9e2fd4dea4d054a4d4da35b24a6185058" gracePeriod=30 Jul 10 18:44:30.691526 ip-10-0-128-39 kubenswrapper[2558]: I0710 18:44:30.691490 2558 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-hjf5z"] Jul 10 18:44:31.420602 ip-10-0-128-39 kubenswrapper[2558]: I0710 18:44:31.420580 2558 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-shared-b-4cfdb5-openshift-default-85569b84ff-swg7c" Jul 10 18:44:31.453525 ip-10-0-128-39 kubenswrapper[2558]: I0710 18:44:31.453429 2558 log.go:25] "Finished parsing log file" path="/var/log/pods/opendatahub_kserve-controller-manager-d6f4b65b8-s4grg_e28e2570-6896-422b-bfa9-30402ea86e77/manager/0.log" Jul 10 18:44:31.545978 ip-10-0-128-39 kubenswrapper[2558]: I0710 18:44:31.545947 2558 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/6735c6d1-6741-40ee-901f-e6cfd26d62f2-istio-envoy\") pod \"6735c6d1-6741-40ee-901f-e6cfd26d62f2\" (UID: \"6735c6d1-6741-40ee-901f-e6cfd26d62f2\") " Jul 10 18:44:31.545978 ip-10-0-128-39 kubenswrapper[2558]: I0710 18:44:31.545986 2558 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/6735c6d1-6741-40ee-901f-e6cfd26d62f2-credential-socket\") pod \"6735c6d1-6741-40ee-901f-e6cfd26d62f2\" (UID: \"6735c6d1-6741-40ee-901f-e6cfd26d62f2\") " Jul 10 18:44:31.546232 ip-10-0-128-39 kubenswrapper[2558]: I0710 18:44:31.546017 2558 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/6735c6d1-6741-40ee-901f-e6cfd26d62f2-istiod-ca-cert\") pod \"6735c6d1-6741-40ee-901f-e6cfd26d62f2\" (UID: \"6735c6d1-6741-40ee-901f-e6cfd26d62f2\") " Jul 10 18:44:31.546232 ip-10-0-128-39 kubenswrapper[2558]: I0710 18:44:31.546060 2558 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/6735c6d1-6741-40ee-901f-e6cfd26d62f2-istio-token\") pod \"6735c6d1-6741-40ee-901f-e6cfd26d62f2\" (UID: \"6735c6d1-6741-40ee-901f-e6cfd26d62f2\") " Jul 10 18:44:31.546232 ip-10-0-128-39 kubenswrapper[2558]: I0710 18:44:31.546098 2558 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/6735c6d1-6741-40ee-901f-e6cfd26d62f2-workload-certs\") pod \"6735c6d1-6741-40ee-901f-e6cfd26d62f2\" (UID: \"6735c6d1-6741-40ee-901f-e6cfd26d62f2\") " Jul 10 18:44:31.546232 ip-10-0-128-39 kubenswrapper[2558]: I0710 18:44:31.546131 2558 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/6735c6d1-6741-40ee-901f-e6cfd26d62f2-istio-podinfo\") pod \"6735c6d1-6741-40ee-901f-e6cfd26d62f2\" (UID: \"6735c6d1-6741-40ee-901f-e6cfd26d62f2\") " Jul 10 18:44:31.546232 ip-10-0-128-39 kubenswrapper[2558]: I0710 18:44:31.546173 2558 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/6735c6d1-6741-40ee-901f-e6cfd26d62f2-workload-socket\") pod \"6735c6d1-6741-40ee-901f-e6cfd26d62f2\" (UID: \"6735c6d1-6741-40ee-901f-e6cfd26d62f2\") " Jul 10 18:44:31.546232 ip-10-0-128-39 kubenswrapper[2558]: I0710 18:44:31.546215 2558 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-kjlnx\" (UniqueName: \"kubernetes.io/projected/6735c6d1-6741-40ee-901f-e6cfd26d62f2-kube-api-access-kjlnx\") pod \"6735c6d1-6741-40ee-901f-e6cfd26d62f2\" (UID: \"6735c6d1-6741-40ee-901f-e6cfd26d62f2\") " Jul 10 18:44:31.546596 ip-10-0-128-39 kubenswrapper[2558]: I0710 18:44:31.546237 2558 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/6735c6d1-6741-40ee-901f-e6cfd26d62f2-istio-data\") pod \"6735c6d1-6741-40ee-901f-e6cfd26d62f2\" (UID: \"6735c6d1-6741-40ee-901f-e6cfd26d62f2\") " Jul 10 18:44:31.546596 ip-10-0-128-39 kubenswrapper[2558]: I0710 18:44:31.546313 2558 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/6735c6d1-6741-40ee-901f-e6cfd26d62f2-credential-socket" (OuterVolumeSpecName: "credential-socket") pod "6735c6d1-6741-40ee-901f-e6cfd26d62f2" (UID: "6735c6d1-6741-40ee-901f-e6cfd26d62f2"). InnerVolumeSpecName "credential-socket". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Jul 10 18:44:31.546596 ip-10-0-128-39 kubenswrapper[2558]: I0710 18:44:31.546399 2558 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/6735c6d1-6741-40ee-901f-e6cfd26d62f2-workload-certs" (OuterVolumeSpecName: "workload-certs") pod "6735c6d1-6741-40ee-901f-e6cfd26d62f2" (UID: "6735c6d1-6741-40ee-901f-e6cfd26d62f2"). InnerVolumeSpecName "workload-certs". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Jul 10 18:44:31.546596 ip-10-0-128-39 kubenswrapper[2558]: I0710 18:44:31.546553 2558 reconciler_common.go:299] "Volume detached for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/6735c6d1-6741-40ee-901f-e6cfd26d62f2-workload-certs\") on node \"ip-10-0-128-39.ec2.internal\" DevicePath \"\"" Jul 10 18:44:31.546596 ip-10-0-128-39 kubenswrapper[2558]: I0710 18:44:31.546491 2558 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/6735c6d1-6741-40ee-901f-e6cfd26d62f2-istiod-ca-cert" (OuterVolumeSpecName: "istiod-ca-cert") pod "6735c6d1-6741-40ee-901f-e6cfd26d62f2" (UID: "6735c6d1-6741-40ee-901f-e6cfd26d62f2"). InnerVolumeSpecName "istiod-ca-cert". PluginName "kubernetes.io/configmap", VolumeGIDValue "" Jul 10 18:44:31.546596 ip-10-0-128-39 kubenswrapper[2558]: I0710 18:44:31.546582 2558 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/6735c6d1-6741-40ee-901f-e6cfd26d62f2-workload-socket" (OuterVolumeSpecName: "workload-socket") pod "6735c6d1-6741-40ee-901f-e6cfd26d62f2" (UID: "6735c6d1-6741-40ee-901f-e6cfd26d62f2"). InnerVolumeSpecName "workload-socket". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Jul 10 18:44:31.546596 ip-10-0-128-39 kubenswrapper[2558]: I0710 18:44:31.546572 2558 reconciler_common.go:299] "Volume detached for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/6735c6d1-6741-40ee-901f-e6cfd26d62f2-credential-socket\") on node \"ip-10-0-128-39.ec2.internal\" DevicePath \"\"" Jul 10 18:44:31.547216 ip-10-0-128-39 kubenswrapper[2558]: I0710 18:44:31.547195 2558 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/6735c6d1-6741-40ee-901f-e6cfd26d62f2-istio-data" (OuterVolumeSpecName: "istio-data") pod "6735c6d1-6741-40ee-901f-e6cfd26d62f2" (UID: "6735c6d1-6741-40ee-901f-e6cfd26d62f2"). InnerVolumeSpecName "istio-data". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Jul 10 18:44:31.548314 ip-10-0-128-39 kubenswrapper[2558]: I0710 18:44:31.548282 2558 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/6735c6d1-6741-40ee-901f-e6cfd26d62f2-istio-envoy" (OuterVolumeSpecName: "istio-envoy") pod "6735c6d1-6741-40ee-901f-e6cfd26d62f2" (UID: "6735c6d1-6741-40ee-901f-e6cfd26d62f2"). InnerVolumeSpecName "istio-envoy". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Jul 10 18:44:31.548415 ip-10-0-128-39 kubenswrapper[2558]: I0710 18:44:31.548353 2558 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/6735c6d1-6741-40ee-901f-e6cfd26d62f2-kube-api-access-kjlnx" (OuterVolumeSpecName: "kube-api-access-kjlnx") pod "6735c6d1-6741-40ee-901f-e6cfd26d62f2" (UID: "6735c6d1-6741-40ee-901f-e6cfd26d62f2"). InnerVolumeSpecName "kube-api-access-kjlnx". PluginName "kubernetes.io/projected", VolumeGIDValue "" Jul 10 18:44:31.548651 ip-10-0-128-39 kubenswrapper[2558]: I0710 18:44:31.548625 2558 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/6735c6d1-6741-40ee-901f-e6cfd26d62f2-istio-token" (OuterVolumeSpecName: "istio-token") pod "6735c6d1-6741-40ee-901f-e6cfd26d62f2" (UID: "6735c6d1-6741-40ee-901f-e6cfd26d62f2"). InnerVolumeSpecName "istio-token". PluginName "kubernetes.io/projected", VolumeGIDValue "" Jul 10 18:44:31.548735 ip-10-0-128-39 kubenswrapper[2558]: I0710 18:44:31.548669 2558 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/downward-api/6735c6d1-6741-40ee-901f-e6cfd26d62f2-istio-podinfo" (OuterVolumeSpecName: "istio-podinfo") pod "6735c6d1-6741-40ee-901f-e6cfd26d62f2" (UID: "6735c6d1-6741-40ee-901f-e6cfd26d62f2"). InnerVolumeSpecName "istio-podinfo". PluginName "kubernetes.io/downward-api", VolumeGIDValue "" Jul 10 18:44:31.647648 ip-10-0-128-39 kubenswrapper[2558]: I0710 18:44:31.647619 2558 reconciler_common.go:299] "Volume detached for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/6735c6d1-6741-40ee-901f-e6cfd26d62f2-istio-token\") on node \"ip-10-0-128-39.ec2.internal\" DevicePath \"\"" Jul 10 18:44:31.647648 ip-10-0-128-39 kubenswrapper[2558]: I0710 18:44:31.647643 2558 reconciler_common.go:299] "Volume detached for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/6735c6d1-6741-40ee-901f-e6cfd26d62f2-istio-podinfo\") on node \"ip-10-0-128-39.ec2.internal\" DevicePath \"\"" Jul 10 18:44:31.647648 ip-10-0-128-39 kubenswrapper[2558]: I0710 18:44:31.647652 2558 reconciler_common.go:299] "Volume detached for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/6735c6d1-6741-40ee-901f-e6cfd26d62f2-workload-socket\") on node \"ip-10-0-128-39.ec2.internal\" DevicePath \"\"" Jul 10 18:44:31.647900 ip-10-0-128-39 kubenswrapper[2558]: I0710 18:44:31.647659 2558 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-kjlnx\" (UniqueName: \"kubernetes.io/projected/6735c6d1-6741-40ee-901f-e6cfd26d62f2-kube-api-access-kjlnx\") on node \"ip-10-0-128-39.ec2.internal\" DevicePath \"\"" Jul 10 18:44:31.647900 ip-10-0-128-39 kubenswrapper[2558]: I0710 18:44:31.647668 2558 reconciler_common.go:299] "Volume detached for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/6735c6d1-6741-40ee-901f-e6cfd26d62f2-istio-data\") on node \"ip-10-0-128-39.ec2.internal\" DevicePath \"\"" Jul 10 18:44:31.647900 ip-10-0-128-39 kubenswrapper[2558]: I0710 18:44:31.647675 2558 reconciler_common.go:299] "Volume detached for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/6735c6d1-6741-40ee-901f-e6cfd26d62f2-istio-envoy\") on node \"ip-10-0-128-39.ec2.internal\" DevicePath \"\"" Jul 10 18:44:31.647900 ip-10-0-128-39 kubenswrapper[2558]: I0710 18:44:31.647683 2558 reconciler_common.go:299] "Volume detached for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/6735c6d1-6741-40ee-901f-e6cfd26d62f2-istiod-ca-cert\") on node \"ip-10-0-128-39.ec2.internal\" DevicePath \"\"" Jul 10 18:44:31.762421 ip-10-0-128-39 kubenswrapper[2558]: I0710 18:44:31.762385 2558 generic.go:356] "Generic (PLEG): container finished" podID="6735c6d1-6741-40ee-901f-e6cfd26d62f2" containerID="022cd5e08a211713dc2754d5b11aabb9e2fd4dea4d054a4d4da35b24a6185058" exitCode=0 Jul 10 18:44:31.762838 ip-10-0-128-39 kubenswrapper[2558]: I0710 18:44:31.762433 2558 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-shared-b-4cfdb5-openshift-default-85569b84ff-swg7c" Jul 10 18:44:31.762838 ip-10-0-128-39 kubenswrapper[2558]: I0710 18:44:31.762435 2558 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-shared-b-4cfdb5-openshift-default-85569b84ff-swg7c" event={"ID":"6735c6d1-6741-40ee-901f-e6cfd26d62f2","Type":"ContainerDied","Data":"022cd5e08a211713dc2754d5b11aabb9e2fd4dea4d054a4d4da35b24a6185058"} Jul 10 18:44:31.762838 ip-10-0-128-39 kubenswrapper[2558]: I0710 18:44:31.762562 2558 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-shared-b-4cfdb5-openshift-default-85569b84ff-swg7c" event={"ID":"6735c6d1-6741-40ee-901f-e6cfd26d62f2","Type":"ContainerDied","Data":"e79d2104abbadcd8920be66577992fcb302d2f6a074924ff98fd95299920ecdf"} Jul 10 18:44:31.762838 ip-10-0-128-39 kubenswrapper[2558]: I0710 18:44:31.762596 2558 scope.go:117] "RemoveContainer" containerID="022cd5e08a211713dc2754d5b11aabb9e2fd4dea4d054a4d4da35b24a6185058" Jul 10 18:44:31.772458 ip-10-0-128-39 kubenswrapper[2558]: I0710 18:44:31.772440 2558 scope.go:117] "RemoveContainer" containerID="4af28f235ee2d04110c372522a0ed97abb1f9e170b1ffe510762c3d09962523e" Jul 10 18:44:31.780438 ip-10-0-128-39 kubenswrapper[2558]: I0710 18:44:31.780421 2558 scope.go:117] "RemoveContainer" containerID="022cd5e08a211713dc2754d5b11aabb9e2fd4dea4d054a4d4da35b24a6185058" Jul 10 18:44:31.780695 ip-10-0-128-39 kubenswrapper[2558]: E0710 18:44:31.780677 2558 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"022cd5e08a211713dc2754d5b11aabb9e2fd4dea4d054a4d4da35b24a6185058\": container with ID starting with 022cd5e08a211713dc2754d5b11aabb9e2fd4dea4d054a4d4da35b24a6185058 not found: ID does not exist" containerID="022cd5e08a211713dc2754d5b11aabb9e2fd4dea4d054a4d4da35b24a6185058" Jul 10 18:44:31.780741 ip-10-0-128-39 kubenswrapper[2558]: I0710 18:44:31.780705 2558 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"022cd5e08a211713dc2754d5b11aabb9e2fd4dea4d054a4d4da35b24a6185058"} err="failed to get container status \"022cd5e08a211713dc2754d5b11aabb9e2fd4dea4d054a4d4da35b24a6185058\": rpc error: code = NotFound desc = could not find container \"022cd5e08a211713dc2754d5b11aabb9e2fd4dea4d054a4d4da35b24a6185058\": container with ID starting with 022cd5e08a211713dc2754d5b11aabb9e2fd4dea4d054a4d4da35b24a6185058 not found: ID does not exist" Jul 10 18:44:31.780741 ip-10-0-128-39 kubenswrapper[2558]: I0710 18:44:31.780723 2558 scope.go:117] "RemoveContainer" containerID="4af28f235ee2d04110c372522a0ed97abb1f9e170b1ffe510762c3d09962523e" Jul 10 18:44:31.781082 ip-10-0-128-39 kubenswrapper[2558]: E0710 18:44:31.781056 2558 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"4af28f235ee2d04110c372522a0ed97abb1f9e170b1ffe510762c3d09962523e\": container with ID starting with 4af28f235ee2d04110c372522a0ed97abb1f9e170b1ffe510762c3d09962523e not found: ID does not exist" containerID="4af28f235ee2d04110c372522a0ed97abb1f9e170b1ffe510762c3d09962523e" Jul 10 18:44:31.781172 ip-10-0-128-39 kubenswrapper[2558]: I0710 18:44:31.781084 2558 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"4af28f235ee2d04110c372522a0ed97abb1f9e170b1ffe510762c3d09962523e"} err="failed to get container status \"4af28f235ee2d04110c372522a0ed97abb1f9e170b1ffe510762c3d09962523e\": rpc error: code = NotFound desc = could not find container \"4af28f235ee2d04110c372522a0ed97abb1f9e170b1ffe510762c3d09962523e\": container with ID starting with 4af28f235ee2d04110c372522a0ed97abb1f9e170b1ffe510762c3d09962523e not found: ID does not exist" Jul 10 18:44:31.788515 ip-10-0-128-39 kubenswrapper[2558]: I0710 18:44:31.786935 2558 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-ingress/e2e-shared-b-4cfdb5-openshift-default-85569b84ff-swg7c"] Jul 10 18:44:31.790319 ip-10-0-128-39 kubenswrapper[2558]: I0710 18:44:31.790293 2558 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["openshift-ingress/e2e-shared-b-4cfdb5-openshift-default-85569b84ff-swg7c"] Jul 10 18:44:32.101466 ip-10-0-128-39 kubenswrapper[2558]: I0710 18:44:32.101385 2558 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="6735c6d1-6741-40ee-901f-e6cfd26d62f2" path="/var/lib/kubelet/pods/6735c6d1-6741-40ee-901f-e6cfd26d62f2/volumes" Jul 10 18:44:32.182157 ip-10-0-128-39 kubenswrapper[2558]: I0710 18:44:32.182133 2558 log.go:25] "Finished parsing log file" path="/var/log/pods/opendatahub_opendatahub-operator-controller-manager-795b486df7-vfgpp_5a66be4d-6d90-47b6-a1c2-30a7734df1e8/copy-manifests/0.log" Jul 10 18:44:32.189592 ip-10-0-128-39 kubenswrapper[2558]: I0710 18:44:32.189568 2558 log.go:25] "Finished parsing log file" path="/var/log/pods/opendatahub_opendatahub-operator-controller-manager-795b486df7-vfgpp_5a66be4d-6d90-47b6-a1c2-30a7734df1e8/manager/0.log" Jul 10 18:44:34.208981 ip-10-0-128-39 kubenswrapper[2558]: I0710 18:44:34.208947 2558 log.go:25] "Finished parsing log file" path="/var/log/pods/kuadrant-system_dns-operator-controller-manager-864dd78c4c-6x5s9_b6eb1bbd-47d9-4504-b855-e0bd01f5a514/manager/0.log" Jul 10 18:44:34.667025 ip-10-0-128-39 kubenswrapper[2558]: I0710 18:44:34.666937 2558 log.go:25] "Finished parsing log file" path="/var/log/pods/kuadrant-system_limitador-limitador-54d6d988d5-hjf5z_55873f81-1144-4875-b3aa-88b7c1c3da29/limitador/0.log" Jul 10 18:44:34.784830 ip-10-0-128-39 kubenswrapper[2558]: I0710 18:44:34.784785 2558 log.go:25] "Finished parsing log file" path="/var/log/pods/kuadrant-system_limitador-operator-controller-manager-859d7b78bd-6nzmh_2abaa1e1-c899-43bd-930a-7f42e12adee0/manager/0.log" Jul 10 18:44:35.912769 ip-10-0-128-39 kubenswrapper[2558]: I0710 18:44:35.912745 2558 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ingress_router-default-cd6bb8744-wq24m_5fab65cc-f578-4cae-919e-1e630186e6d3/router/0.log" Jul 10 18:44:36.671128 ip-10-0-128-39 kubenswrapper[2558]: I0710 18:44:36.671099 2558 log.go:25] "Finished parsing log file" path="/var/log/pods/llm_facebook-opt-125m-simulated-kserve-74b68447df-mpr5p_c4e8ab9b-bb52-4775-b742-eb3fc91269fd/main/0.log" Jul 10 18:44:43.903370 ip-10-0-128-39 kubenswrapper[2558]: I0710 18:44:43.903340 2558 log.go:25] "Finished parsing log file" path="/var/log/pods/kube-system_global-pull-secret-syncer-vsbdm_3b27429f-90aa-4170-9078-0aa4ea0d0ab8/global-pull-secret-syncer/0.log" Jul 10 18:44:44.003442 ip-10-0-128-39 kubenswrapper[2558]: I0710 18:44:44.003411 2558 log.go:25] "Finished parsing log file" path="/var/log/pods/kube-system_konnectivity-agent-dvp86_2da19865-b021-4789-b1e0-694d4f6c543d/konnectivity-agent/0.log" Jul 10 18:44:44.069955 ip-10-0-128-39 kubenswrapper[2558]: I0710 18:44:44.069928 2558 log.go:25] "Finished parsing log file" path="/var/log/pods/kube-system_kube-apiserver-proxy-ip-10-0-128-39.ec2.internal_91839b707975206c1eb988a2e4de26d3/haproxy/0.log" Jul 10 18:44:48.947371 ip-10-0-128-39 kubenswrapper[2558]: I0710 18:44:48.947284 2558 log.go:25] "Finished parsing log file" path="/var/log/pods/kuadrant-system_dns-operator-controller-manager-864dd78c4c-6x5s9_b6eb1bbd-47d9-4504-b855-e0bd01f5a514/manager/0.log" Jul 10 18:44:49.155414 ip-10-0-128-39 kubenswrapper[2558]: I0710 18:44:49.155376 2558 log.go:25] "Finished parsing log file" path="/var/log/pods/kuadrant-system_limitador-limitador-54d6d988d5-hjf5z_55873f81-1144-4875-b3aa-88b7c1c3da29/limitador/0.log" Jul 10 18:44:49.247058 ip-10-0-128-39 kubenswrapper[2558]: I0710 18:44:49.247027 2558 log.go:25] "Finished parsing log file" path="/var/log/pods/kuadrant-system_limitador-operator-controller-manager-859d7b78bd-6nzmh_2abaa1e1-c899-43bd-930a-7f42e12adee0/manager/0.log" Jul 10 18:44:51.003589 ip-10-0-128-39 kubenswrapper[2558]: I0710 18:44:51.003557 2558 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_cluster-monitoring-operator-5f7f494dd4-46fc9_19b8d048-b0b5-4b42-9f4f-ad9f47a105cd/cluster-monitoring-operator/0.log" Jul 10 18:44:51.254132 ip-10-0-128-39 kubenswrapper[2558]: I0710 18:44:51.254058 2558 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_node-exporter-cj5n2_0f113ed9-f2c4-44d5-9848-97e6250dbc22/node-exporter/0.log" Jul 10 18:44:51.275782 ip-10-0-128-39 kubenswrapper[2558]: I0710 18:44:51.275755 2558 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_node-exporter-cj5n2_0f113ed9-f2c4-44d5-9848-97e6250dbc22/kube-rbac-proxy/0.log" Jul 10 18:44:51.299389 ip-10-0-128-39 kubenswrapper[2558]: I0710 18:44:51.299362 2558 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_node-exporter-cj5n2_0f113ed9-f2c4-44d5-9848-97e6250dbc22/init-textfile/0.log" Jul 10 18:44:53.401056 ip-10-0-128-39 kubenswrapper[2558]: I0710 18:44:53.401016 2558 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-network-console_networking-console-plugin-6cf856758c-qg5vl_b95b59af-225c-4907-9643-5d916a325e56/networking-console-plugin/0.log" Jul 10 18:44:54.051966 ip-10-0-128-39 kubenswrapper[2558]: I0710 18:44:54.051941 2558 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-console-operator_console-operator-7866497494-26542_2b29fb39-89b3-42c3-b334-3a195242248c/console-operator/1.log" Jul 10 18:44:54.061879 ip-10-0-128-39 kubenswrapper[2558]: I0710 18:44:54.061853 2558 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-console-operator_console-operator-7866497494-26542_2b29fb39-89b3-42c3-b334-3a195242248c/console-operator/2.log" Jul 10 18:44:54.445070 ip-10-0-128-39 kubenswrapper[2558]: I0710 18:44:54.444992 2558 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-must-gather-rjmm9/perf-node-gather-daemonset-r8spw"] Jul 10 18:44:54.445417 ip-10-0-128-39 kubenswrapper[2558]: I0710 18:44:54.445329 2558 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="6735c6d1-6741-40ee-901f-e6cfd26d62f2" containerName="istio-proxy" Jul 10 18:44:54.445417 ip-10-0-128-39 kubenswrapper[2558]: I0710 18:44:54.445340 2558 state_mem.go:107] "Deleted CPUSet assignment" podUID="6735c6d1-6741-40ee-901f-e6cfd26d62f2" containerName="istio-proxy" Jul 10 18:44:54.445417 ip-10-0-128-39 kubenswrapper[2558]: I0710 18:44:54.445357 2558 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="eccf53c5-36a1-45d7-af34-23e96c86f9e3" containerName="test-curl-7380-139772321405808" Jul 10 18:44:54.445417 ip-10-0-128-39 kubenswrapper[2558]: I0710 18:44:54.445363 2558 state_mem.go:107] "Deleted CPUSet assignment" podUID="eccf53c5-36a1-45d7-af34-23e96c86f9e3" containerName="test-curl-7380-139772321405808" Jul 10 18:44:54.445539 ip-10-0-128-39 kubenswrapper[2558]: I0710 18:44:54.445437 2558 memory_manager.go:368] "RemoveStaleState removing state" podUID="6735c6d1-6741-40ee-901f-e6cfd26d62f2" containerName="istio-proxy" Jul 10 18:44:54.445539 ip-10-0-128-39 kubenswrapper[2558]: I0710 18:44:54.445447 2558 memory_manager.go:368] "RemoveStaleState removing state" podUID="6735c6d1-6741-40ee-901f-e6cfd26d62f2" containerName="istio-proxy" Jul 10 18:44:54.445539 ip-10-0-128-39 kubenswrapper[2558]: I0710 18:44:54.445455 2558 memory_manager.go:368] "RemoveStaleState removing state" podUID="eccf53c5-36a1-45d7-af34-23e96c86f9e3" containerName="test-curl-7380-139772321405808" Jul 10 18:44:54.450141 ip-10-0-128-39 kubenswrapper[2558]: I0710 18:44:54.450119 2558 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-must-gather-rjmm9/perf-node-gather-daemonset-r8spw" Jul 10 18:44:54.452757 ip-10-0-128-39 kubenswrapper[2558]: I0710 18:44:54.452733 2558 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-must-gather-rjmm9\"/\"openshift-service-ca.crt\"" Jul 10 18:44:54.452757 ip-10-0-128-39 kubenswrapper[2558]: I0710 18:44:54.452746 2558 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-must-gather-rjmm9\"/\"kube-root-ca.crt\"" Jul 10 18:44:54.452757 ip-10-0-128-39 kubenswrapper[2558]: I0710 18:44:54.452756 2558 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-must-gather-rjmm9\"/\"default-dockercfg-r9c95\"" Jul 10 18:44:54.459074 ip-10-0-128-39 kubenswrapper[2558]: I0710 18:44:54.459051 2558 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-must-gather-rjmm9/perf-node-gather-daemonset-r8spw"] Jul 10 18:44:54.548266 ip-10-0-128-39 kubenswrapper[2558]: I0710 18:44:54.548230 2558 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-lhh82\" (UniqueName: \"kubernetes.io/projected/80c688a2-718f-4106-9b89-86a208c31a21-kube-api-access-lhh82\") pod \"perf-node-gather-daemonset-r8spw\" (UID: \"80c688a2-718f-4106-9b89-86a208c31a21\") " pod="openshift-must-gather-rjmm9/perf-node-gather-daemonset-r8spw" Jul 10 18:44:54.548266 ip-10-0-128-39 kubenswrapper[2558]: I0710 18:44:54.548272 2558 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"proc\" (UniqueName: \"kubernetes.io/host-path/80c688a2-718f-4106-9b89-86a208c31a21-proc\") pod \"perf-node-gather-daemonset-r8spw\" (UID: \"80c688a2-718f-4106-9b89-86a208c31a21\") " pod="openshift-must-gather-rjmm9/perf-node-gather-daemonset-r8spw" Jul 10 18:44:54.548487 ip-10-0-128-39 kubenswrapper[2558]: I0710 18:44:54.548288 2558 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"podres\" (UniqueName: \"kubernetes.io/host-path/80c688a2-718f-4106-9b89-86a208c31a21-podres\") pod \"perf-node-gather-daemonset-r8spw\" (UID: \"80c688a2-718f-4106-9b89-86a208c31a21\") " pod="openshift-must-gather-rjmm9/perf-node-gather-daemonset-r8spw" Jul 10 18:44:54.548487 ip-10-0-128-39 kubenswrapper[2558]: I0710 18:44:54.548312 2558 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"lib-modules\" (UniqueName: \"kubernetes.io/host-path/80c688a2-718f-4106-9b89-86a208c31a21-lib-modules\") pod \"perf-node-gather-daemonset-r8spw\" (UID: \"80c688a2-718f-4106-9b89-86a208c31a21\") " pod="openshift-must-gather-rjmm9/perf-node-gather-daemonset-r8spw" Jul 10 18:44:54.548487 ip-10-0-128-39 kubenswrapper[2558]: I0710 18:44:54.548354 2558 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"sys\" (UniqueName: \"kubernetes.io/host-path/80c688a2-718f-4106-9b89-86a208c31a21-sys\") pod \"perf-node-gather-daemonset-r8spw\" (UID: \"80c688a2-718f-4106-9b89-86a208c31a21\") " pod="openshift-must-gather-rjmm9/perf-node-gather-daemonset-r8spw" Jul 10 18:44:54.649300 ip-10-0-128-39 kubenswrapper[2558]: I0710 18:44:54.649265 2558 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"sys\" (UniqueName: \"kubernetes.io/host-path/80c688a2-718f-4106-9b89-86a208c31a21-sys\") pod \"perf-node-gather-daemonset-r8spw\" (UID: \"80c688a2-718f-4106-9b89-86a208c31a21\") " pod="openshift-must-gather-rjmm9/perf-node-gather-daemonset-r8spw" Jul 10 18:44:54.649466 ip-10-0-128-39 kubenswrapper[2558]: I0710 18:44:54.649404 2558 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"sys\" (UniqueName: \"kubernetes.io/host-path/80c688a2-718f-4106-9b89-86a208c31a21-sys\") pod \"perf-node-gather-daemonset-r8spw\" (UID: \"80c688a2-718f-4106-9b89-86a208c31a21\") " pod="openshift-must-gather-rjmm9/perf-node-gather-daemonset-r8spw" Jul 10 18:44:54.649466 ip-10-0-128-39 kubenswrapper[2558]: I0710 18:44:54.649431 2558 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-lhh82\" (UniqueName: \"kubernetes.io/projected/80c688a2-718f-4106-9b89-86a208c31a21-kube-api-access-lhh82\") pod \"perf-node-gather-daemonset-r8spw\" (UID: \"80c688a2-718f-4106-9b89-86a208c31a21\") " pod="openshift-must-gather-rjmm9/perf-node-gather-daemonset-r8spw" Jul 10 18:44:54.649542 ip-10-0-128-39 kubenswrapper[2558]: I0710 18:44:54.649464 2558 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"proc\" (UniqueName: \"kubernetes.io/host-path/80c688a2-718f-4106-9b89-86a208c31a21-proc\") pod \"perf-node-gather-daemonset-r8spw\" (UID: \"80c688a2-718f-4106-9b89-86a208c31a21\") " pod="openshift-must-gather-rjmm9/perf-node-gather-daemonset-r8spw" Jul 10 18:44:54.649542 ip-10-0-128-39 kubenswrapper[2558]: I0710 18:44:54.649484 2558 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"podres\" (UniqueName: \"kubernetes.io/host-path/80c688a2-718f-4106-9b89-86a208c31a21-podres\") pod \"perf-node-gather-daemonset-r8spw\" (UID: \"80c688a2-718f-4106-9b89-86a208c31a21\") " pod="openshift-must-gather-rjmm9/perf-node-gather-daemonset-r8spw" Jul 10 18:44:54.649542 ip-10-0-128-39 kubenswrapper[2558]: I0710 18:44:54.649515 2558 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"lib-modules\" (UniqueName: \"kubernetes.io/host-path/80c688a2-718f-4106-9b89-86a208c31a21-lib-modules\") pod \"perf-node-gather-daemonset-r8spw\" (UID: \"80c688a2-718f-4106-9b89-86a208c31a21\") " pod="openshift-must-gather-rjmm9/perf-node-gather-daemonset-r8spw" Jul 10 18:44:54.649542 ip-10-0-128-39 kubenswrapper[2558]: I0710 18:44:54.649528 2558 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"proc\" (UniqueName: \"kubernetes.io/host-path/80c688a2-718f-4106-9b89-86a208c31a21-proc\") pod \"perf-node-gather-daemonset-r8spw\" (UID: \"80c688a2-718f-4106-9b89-86a208c31a21\") " pod="openshift-must-gather-rjmm9/perf-node-gather-daemonset-r8spw" Jul 10 18:44:54.649690 ip-10-0-128-39 kubenswrapper[2558]: I0710 18:44:54.649674 2558 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"podres\" (UniqueName: \"kubernetes.io/host-path/80c688a2-718f-4106-9b89-86a208c31a21-podres\") pod \"perf-node-gather-daemonset-r8spw\" (UID: \"80c688a2-718f-4106-9b89-86a208c31a21\") " pod="openshift-must-gather-rjmm9/perf-node-gather-daemonset-r8spw" Jul 10 18:44:54.649724 ip-10-0-128-39 kubenswrapper[2558]: I0710 18:44:54.649691 2558 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"lib-modules\" (UniqueName: \"kubernetes.io/host-path/80c688a2-718f-4106-9b89-86a208c31a21-lib-modules\") pod \"perf-node-gather-daemonset-r8spw\" (UID: \"80c688a2-718f-4106-9b89-86a208c31a21\") " pod="openshift-must-gather-rjmm9/perf-node-gather-daemonset-r8spw" Jul 10 18:44:54.658241 ip-10-0-128-39 kubenswrapper[2558]: I0710 18:44:54.658223 2558 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-lhh82\" (UniqueName: \"kubernetes.io/projected/80c688a2-718f-4106-9b89-86a208c31a21-kube-api-access-lhh82\") pod \"perf-node-gather-daemonset-r8spw\" (UID: \"80c688a2-718f-4106-9b89-86a208c31a21\") " pod="openshift-must-gather-rjmm9/perf-node-gather-daemonset-r8spw" Jul 10 18:44:54.706552 ip-10-0-128-39 kubenswrapper[2558]: I0710 18:44:54.706522 2558 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-console_downloads-68c9cbb497-64qk8_dfc9aa8c-b41d-48fb-b346-ad46b6682c5f/download-server/0.log" Jul 10 18:44:54.761165 ip-10-0-128-39 kubenswrapper[2558]: I0710 18:44:54.761138 2558 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-must-gather-rjmm9/perf-node-gather-daemonset-r8spw" Jul 10 18:44:54.890961 ip-10-0-128-39 kubenswrapper[2558]: I0710 18:44:54.890939 2558 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-must-gather-rjmm9/perf-node-gather-daemonset-r8spw"] Jul 10 18:44:54.892835 ip-10-0-128-39 kubenswrapper[2558]: W0710 18:44:54.892790 2558 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-pod80c688a2_718f_4106_9b89_86a208c31a21.slice/crio-28ac28080072a4f1e0f5204187609babaedeb735f8071d579b1c48086332aeae WatchSource:0}: Error finding container 28ac28080072a4f1e0f5204187609babaedeb735f8071d579b1c48086332aeae: Status 404 returned error can't find the container with id 28ac28080072a4f1e0f5204187609babaedeb735f8071d579b1c48086332aeae Jul 10 18:44:55.343425 ip-10-0-128-39 kubenswrapper[2558]: I0710 18:44:55.343389 2558 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-cluster-storage-operator_volume-data-source-validator-78495597f6-v7ckr_5df2ceaf-9a85-4d1a-ad19-76df6f2b955f/volume-data-source-validator/0.log" Jul 10 18:44:55.879793 ip-10-0-128-39 kubenswrapper[2558]: I0710 18:44:55.879762 2558 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-must-gather-rjmm9/perf-node-gather-daemonset-r8spw" event={"ID":"80c688a2-718f-4106-9b89-86a208c31a21","Type":"ContainerStarted","Data":"32c9a869f47beeeffc584b82fc492000b1645d22f5e3a09aac1eba97d0f8a941"} Jul 10 18:44:55.880203 ip-10-0-128-39 kubenswrapper[2558]: I0710 18:44:55.879828 2558 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-must-gather-rjmm9/perf-node-gather-daemonset-r8spw" event={"ID":"80c688a2-718f-4106-9b89-86a208c31a21","Type":"ContainerStarted","Data":"28ac28080072a4f1e0f5204187609babaedeb735f8071d579b1c48086332aeae"} Jul 10 18:44:55.880203 ip-10-0-128-39 kubenswrapper[2558]: I0710 18:44:55.879914 2558 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-must-gather-rjmm9/perf-node-gather-daemonset-r8spw" Jul 10 18:44:55.895420 ip-10-0-128-39 kubenswrapper[2558]: I0710 18:44:55.895374 2558 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-must-gather-rjmm9/perf-node-gather-daemonset-r8spw" podStartSLOduration=1.895358113 podStartE2EDuration="1.895358113s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=true podCreationTimestamp="2026-07-10 18:44:54 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-10 18:44:55.894453518 +0000 UTC m=+2524.397946022" watchObservedRunningTime="2026-07-10 18:44:55.895358113 +0000 UTC m=+2524.398850617" Jul 10 18:44:56.379603 ip-10-0-128-39 kubenswrapper[2558]: I0710 18:44:56.379579 2558 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-dns_dns-default-22khc_15c4360f-6bc7-4948-b0e6-ebc018174856/dns/0.log" Jul 10 18:44:56.418800 ip-10-0-128-39 kubenswrapper[2558]: I0710 18:44:56.418767 2558 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-dns_dns-default-22khc_15c4360f-6bc7-4948-b0e6-ebc018174856/kube-rbac-proxy/0.log" Jul 10 18:44:56.670937 ip-10-0-128-39 kubenswrapper[2558]: I0710 18:44:56.670852 2558 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-dns_node-resolver-jqb25_12ad9a8e-4768-4362-818f-d2fb73c8a236/dns-node-resolver/0.log" Jul 10 18:44:57.421286 ip-10-0-128-39 kubenswrapper[2558]: I0710 18:44:57.421257 2558 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-image-registry_node-ca-r7qh2_fe05c85a-2eb8-429b-9776-a3408a38903f/node-ca/0.log" Jul 10 18:44:58.703106 ip-10-0-128-39 kubenswrapper[2558]: I0710 18:44:58.703081 2558 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ingress_router-default-cd6bb8744-wq24m_5fab65cc-f578-4cae-919e-1e630186e6d3/router/0.log" Jul 10 18:44:59.452645 ip-10-0-128-39 kubenswrapper[2558]: I0710 18:44:59.452612 2558 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ingress-canary_ingress-canary-bpwbf_f754e509-abbe-4451-833d-abb0e28c7e19/serve-healthcheck-canary/0.log" Jul 10 18:44:59.962335 ip-10-0-128-39 kubenswrapper[2558]: I0710 18:44:59.962288 2558 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-insights_insights-operator-68d9dd94cf-q5s5m_748dca80-632f-44ef-b3a2-5292df366af1/insights-operator/0.log" Jul 10 18:44:59.963965 ip-10-0-128-39 kubenswrapper[2558]: I0710 18:44:59.963938 2558 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-insights_insights-operator-68d9dd94cf-q5s5m_748dca80-632f-44ef-b3a2-5292df366af1/insights-operator/1.log" Jul 10 18:45:00.138133 ip-10-0-128-39 kubenswrapper[2558]: I0710 18:45:00.138101 2558 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["odh-ai-gateway-infra/maas-api-key-cleanup-29728485-s5xn2"] Jul 10 18:45:00.138455 ip-10-0-128-39 kubenswrapper[2558]: I0710 18:45:00.138442 2558 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="6735c6d1-6741-40ee-901f-e6cfd26d62f2" containerName="istio-proxy" Jul 10 18:45:00.138496 ip-10-0-128-39 kubenswrapper[2558]: I0710 18:45:00.138456 2558 state_mem.go:107] "Deleted CPUSet assignment" podUID="6735c6d1-6741-40ee-901f-e6cfd26d62f2" containerName="istio-proxy" Jul 10 18:45:00.141391 ip-10-0-128-39 kubenswrapper[2558]: I0710 18:45:00.141375 2558 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-key-cleanup-29728485-s5xn2" Jul 10 18:45:00.143691 ip-10-0-128-39 kubenswrapper[2558]: I0710 18:45:00.143673 2558 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"odh-ai-gateway-infra\"/\"maas-api-cleanup-dockercfg-fhgdj\"" Jul 10 18:45:00.147268 ip-10-0-128-39 kubenswrapper[2558]: I0710 18:45:00.147243 2558 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-key-cleanup-29728485-s5xn2"] Jul 10 18:45:00.151826 ip-10-0-128-39 kubenswrapper[2558]: I0710 18:45:00.151786 2558 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-key-cleanup-29728485-s5xn2" Jul 10 18:45:00.157342 ip-10-0-128-39 kubenswrapper[2558]: I0710 18:45:00.157321 2558 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-insights_insights-runtime-extractor-lbs9p_cf337123-3a94-4667-a27e-7fc30a589451/kube-rbac-proxy/0.log" Jul 10 18:45:00.182307 ip-10-0-128-39 kubenswrapper[2558]: I0710 18:45:00.182278 2558 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-insights_insights-runtime-extractor-lbs9p_cf337123-3a94-4667-a27e-7fc30a589451/exporter/0.log" Jul 10 18:45:00.211137 ip-10-0-128-39 kubenswrapper[2558]: I0710 18:45:00.211110 2558 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-insights_insights-runtime-extractor-lbs9p_cf337123-3a94-4667-a27e-7fc30a589451/extractor/0.log" Jul 10 18:45:00.287733 ip-10-0-128-39 kubenswrapper[2558]: I0710 18:45:00.287695 2558 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-key-cleanup-29728485-s5xn2"] Jul 10 18:45:00.288889 ip-10-0-128-39 kubenswrapper[2558]: W0710 18:45:00.288864 2558 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod67317bfe_d0ac_45b5_bc22_2a62e4a80aca.slice/crio-16b4167908ba27e111d9258703d79d4de128012157e03da71b19d9f0ab57b52e WatchSource:0}: Error finding container 16b4167908ba27e111d9258703d79d4de128012157e03da71b19d9f0ab57b52e: Status 404 returned error can't find the container with id 16b4167908ba27e111d9258703d79d4de128012157e03da71b19d9f0ab57b52e Jul 10 18:45:00.904928 ip-10-0-128-39 kubenswrapper[2558]: I0710 18:45:00.904834 2558 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-key-cleanup-29728485-s5xn2" event={"ID":"67317bfe-d0ac-45b5-bc22-2a62e4a80aca","Type":"ContainerStarted","Data":"c9018bf23770a45b0e9ff3ebef620ecbf5989e4262bd4d177e6dbb9137ee5499"} Jul 10 18:45:00.904928 ip-10-0-128-39 kubenswrapper[2558]: I0710 18:45:00.904889 2558 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-key-cleanup-29728485-s5xn2" event={"ID":"67317bfe-d0ac-45b5-bc22-2a62e4a80aca","Type":"ContainerStarted","Data":"16b4167908ba27e111d9258703d79d4de128012157e03da71b19d9f0ab57b52e"} Jul 10 18:45:00.921417 ip-10-0-128-39 kubenswrapper[2558]: I0710 18:45:00.921372 2558 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="odh-ai-gateway-infra/maas-api-key-cleanup-29728485-s5xn2" podStartSLOduration=0.921358272 podStartE2EDuration="921.358272ms" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-10 18:45:00 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-10 18:45:00.920066121 +0000 UTC m=+2529.423558628" watchObservedRunningTime="2026-07-10 18:45:00.921358272 +0000 UTC m=+2529.424850776" Jul 10 18:45:01.895635 ip-10-0-128-39 kubenswrapper[2558]: I0710 18:45:01.895610 2558 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-must-gather-rjmm9/perf-node-gather-daemonset-r8spw" Jul 10 18:45:02.315830 ip-10-0-128-39 kubenswrapper[2558]: I0710 18:45:02.315788 2558 log.go:25] "Finished parsing log file" path="/var/log/pods/opendatahub_kserve-controller-manager-d6f4b65b8-s4grg_e28e2570-6896-422b-bfa9-30402ea86e77/manager/0.log" Jul 10 18:45:02.884082 ip-10-0-128-39 kubenswrapper[2558]: I0710 18:45:02.884052 2558 log.go:25] "Finished parsing log file" path="/var/log/pods/opendatahub_opendatahub-operator-controller-manager-795b486df7-vfgpp_5a66be4d-6d90-47b6-a1c2-30a7734df1e8/manager/0.log" Jul 10 18:45:02.905837 ip-10-0-128-39 kubenswrapper[2558]: I0710 18:45:02.905796 2558 log.go:25] "Finished parsing log file" path="/var/log/pods/opendatahub_opendatahub-operator-controller-manager-795b486df7-vfgpp_5a66be4d-6d90-47b6-a1c2-30a7734df1e8/copy-manifests/0.log" Jul 10 18:45:04.110600 ip-10-0-128-39 kubenswrapper[2558]: I0710 18:45:04.110563 2558 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-lws-operator_lws-controller-manager-59dbbcc5d9-4l2s9_48e923db-401e-40a8-acc6-7fb572b98f7f/manager/0.log" Jul 10 18:45:04.874021 ip-10-0-128-39 kubenswrapper[2558]: I0710 18:45:04.873936 2558 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ingress_router-default-cd6bb8744-wq24m_5fab65cc-f578-4cae-919e-1e630186e6d3/router/0.log" Jul 10 18:45:06.496528 ip-10-0-128-39 kubenswrapper[2558]: I0710 18:45:06.496480 2558 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ingress_router-default-cd6bb8744-wq24m_5fab65cc-f578-4cae-919e-1e630186e6d3/router/0.log" Jul 10 18:45:09.500535 ip-10-0-128-39 kubenswrapper[2558]: I0710 18:45:09.500507 2558 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-kube-storage-version-migrator_migrator-7f59b54f6c-5nctn_fb2c6bad-c788-4a76-beab-e424603069c4/migrator/0.log" Jul 10 18:45:09.521291 ip-10-0-128-39 kubenswrapper[2558]: I0710 18:45:09.521263 2558 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-kube-storage-version-migrator_migrator-7f59b54f6c-5nctn_fb2c6bad-c788-4a76-beab-e424603069c4/graceful-termination/0.log" Jul 10 18:45:09.891509 ip-10-0-128-39 kubenswrapper[2558]: I0710 18:45:09.891429 2558 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-kube-storage-version-migrator-operator_kube-storage-version-migrator-operator-799cb7f958-vggpz_eec35344-a5cb-4a6e-9e2e-5adca5a0fa1a/kube-storage-version-migrator-operator/1.log" Jul 10 18:45:09.892991 ip-10-0-128-39 kubenswrapper[2558]: I0710 18:45:09.892963 2558 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-kube-storage-version-migrator-operator_kube-storage-version-migrator-operator-799cb7f958-vggpz_eec35344-a5cb-4a6e-9e2e-5adca5a0fa1a/kube-storage-version-migrator-operator/0.log" Jul 10 18:45:11.160177 ip-10-0-128-39 kubenswrapper[2558]: I0710 18:45:11.160146 2558 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-multus_multus-additional-cni-plugins-v22pb_53492c2f-2433-4658-af05-d58878aed082/kube-multus-additional-cni-plugins/0.log" Jul 10 18:45:11.186429 ip-10-0-128-39 kubenswrapper[2558]: I0710 18:45:11.186403 2558 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-multus_multus-additional-cni-plugins-v22pb_53492c2f-2433-4658-af05-d58878aed082/egress-router-binary-copy/0.log" Jul 10 18:45:11.208952 ip-10-0-128-39 kubenswrapper[2558]: I0710 18:45:11.208926 2558 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-multus_multus-additional-cni-plugins-v22pb_53492c2f-2433-4658-af05-d58878aed082/cni-plugins/0.log" Jul 10 18:45:11.230578 ip-10-0-128-39 kubenswrapper[2558]: I0710 18:45:11.230550 2558 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-multus_multus-additional-cni-plugins-v22pb_53492c2f-2433-4658-af05-d58878aed082/bond-cni-plugin/0.log" Jul 10 18:45:11.251718 ip-10-0-128-39 kubenswrapper[2558]: I0710 18:45:11.251689 2558 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-multus_multus-additional-cni-plugins-v22pb_53492c2f-2433-4658-af05-d58878aed082/routeoverride-cni/0.log" Jul 10 18:45:11.273554 ip-10-0-128-39 kubenswrapper[2558]: I0710 18:45:11.273526 2558 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-multus_multus-additional-cni-plugins-v22pb_53492c2f-2433-4658-af05-d58878aed082/whereabouts-cni-bincopy/0.log" Jul 10 18:45:11.295936 ip-10-0-128-39 kubenswrapper[2558]: I0710 18:45:11.295902 2558 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-multus_multus-additional-cni-plugins-v22pb_53492c2f-2433-4658-af05-d58878aed082/whereabouts-cni/0.log" Jul 10 18:45:11.516257 ip-10-0-128-39 kubenswrapper[2558]: I0710 18:45:11.516229 2558 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-multus_multus-kvqn7_11be9079-5d81-46c9-8f4c-c2e305bf2144/kube-multus/0.log" Jul 10 18:45:11.667832 ip-10-0-128-39 kubenswrapper[2558]: I0710 18:45:11.667784 2558 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-multus_network-metrics-daemon-jkr77_7fa20252-06b7-48e4-8620-3e52dc980486/network-metrics-daemon/0.log" Jul 10 18:45:11.686690 ip-10-0-128-39 kubenswrapper[2558]: I0710 18:45:11.686668 2558 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-multus_network-metrics-daemon-jkr77_7fa20252-06b7-48e4-8620-3e52dc980486/kube-rbac-proxy/0.log"