Jul 12 17:51:30.005067 ip-10-0-129-128 systemd[1]: kubelet.service: Failed to load environment files: No such file or directory Jul 12 17:51:30.005079 ip-10-0-129-128 systemd[1]: kubelet.service: Failed to run 'start-pre' task: No such file or directory Jul 12 17:51:30.005085 ip-10-0-129-128 systemd[1]: kubelet.service: Failed with result 'resources'. Jul 12 17:51:30.005335 ip-10-0-129-128 systemd[1]: Failed to start Kubernetes Kubelet. Jul 12 17:51:40.210104 ip-10-0-129-128 systemd[1]: kubelet.service: Failed to schedule restart job: Unit crio.service not found. Jul 12 17:51:40.210123 ip-10-0-129-128 systemd[1]: kubelet.service: Failed with result 'resources'. -- Boot f36a43ffee1d4b0799439fe618be532b -- Jul 12 17:53:11.089426 ip-10-0-129-128 systemd[1]: Starting Kubernetes Kubelet... Jul 12 17:53:11.643333 ip-10-0-129-128 kubenswrapper[2559]: Flag --container-runtime-endpoint has been deprecated, This parameter should be set via the config file specified by the Kubelet's --config flag. See https://kubernetes.io/docs/tasks/administer-cluster/kubelet-config-file/ for more information. Jul 12 17:53:11.643333 ip-10-0-129-128 kubenswrapper[2559]: Flag --minimum-container-ttl-duration has been deprecated, Use --eviction-hard or --eviction-soft instead. Will be removed in a future version. Jul 12 17:53:11.643333 ip-10-0-129-128 kubenswrapper[2559]: Flag --volume-plugin-dir has been deprecated, This parameter should be set via the config file specified by the Kubelet's --config flag. See https://kubernetes.io/docs/tasks/administer-cluster/kubelet-config-file/ for more information. Jul 12 17:53:11.643333 ip-10-0-129-128 kubenswrapper[2559]: Flag --pod-infra-container-image has been deprecated, will be removed in 1.35. Image garbage collector will get sandbox image information from CRI. Jul 12 17:53:11.643333 ip-10-0-129-128 kubenswrapper[2559]: Flag --system-reserved has been deprecated, This parameter should be set via the config file specified by the Kubelet's --config flag. See https://kubernetes.io/docs/tasks/administer-cluster/kubelet-config-file/ for more information. Jul 12 17:53:11.643333 ip-10-0-129-128 kubenswrapper[2559]: I0712 17:53:11.532000 2559 server.go:213] "--pod-infra-container-image will not be pruned by the image garbage collector in kubelet and should also be set in the remote runtime" Jul 12 17:53:11.643333 ip-10-0-129-128 kubenswrapper[2559]: W0712 17:53:11.536891 2559 feature_gate.go:328] unrecognized feature gate: DualReplica Jul 12 17:53:11.643333 ip-10-0-129-128 kubenswrapper[2559]: W0712 17:53:11.536911 2559 feature_gate.go:328] unrecognized feature gate: NewOLMCatalogdAPIV1Metas Jul 12 17:53:11.643333 ip-10-0-129-128 kubenswrapper[2559]: W0712 17:53:11.536915 2559 feature_gate.go:328] unrecognized feature gate: NetworkSegmentation Jul 12 17:53:11.643333 ip-10-0-129-128 kubenswrapper[2559]: W0712 17:53:11.536918 2559 feature_gate.go:328] unrecognized feature gate: ClusterVersionOperatorConfiguration Jul 12 17:53:11.643333 ip-10-0-129-128 kubenswrapper[2559]: W0712 17:53:11.536921 2559 feature_gate.go:328] unrecognized feature gate: Example2 Jul 12 17:53:11.643333 ip-10-0-129-128 kubenswrapper[2559]: W0712 17:53:11.536925 2559 feature_gate.go:328] unrecognized feature gate: GCPCustomAPIEndpointsInstall Jul 12 17:53:11.643333 ip-10-0-129-128 kubenswrapper[2559]: W0712 17:53:11.536927 2559 feature_gate.go:328] unrecognized feature gate: SignatureStores Jul 12 17:53:11.643333 ip-10-0-129-128 kubenswrapper[2559]: W0712 17:53:11.536930 2559 feature_gate.go:328] unrecognized feature gate: VolumeGroupSnapshot Jul 12 17:53:11.610080 ip-10-0-129-128 systemd[1]: Started Kubernetes Kubelet. Jul 12 17:53:11.658413 ip-10-0-129-128 kubenswrapper[2559]: W0712 17:53:11.536933 2559 feature_gate.go:328] unrecognized feature gate: ClusterAPIMachineManagementVSphere Jul 12 17:53:11.658413 ip-10-0-129-128 kubenswrapper[2559]: W0712 17:53:11.536935 2559 feature_gate.go:328] unrecognized feature gate: AdditionalRoutingCapabilities Jul 12 17:53:11.658413 ip-10-0-129-128 kubenswrapper[2559]: W0712 17:53:11.536938 2559 feature_gate.go:328] unrecognized feature gate: AutomatedEtcdBackup Jul 12 17:53:11.658413 ip-10-0-129-128 kubenswrapper[2559]: W0712 17:53:11.536940 2559 feature_gate.go:328] unrecognized feature gate: AzureClusterHostedDNSInstall Jul 12 17:53:11.658413 ip-10-0-129-128 kubenswrapper[2559]: W0712 17:53:11.536943 2559 feature_gate.go:328] unrecognized feature gate: NewOLMOwnSingleNamespace Jul 12 17:53:11.658413 ip-10-0-129-128 kubenswrapper[2559]: W0712 17:53:11.536946 2559 feature_gate.go:328] unrecognized feature gate: NewOLM Jul 12 17:53:11.658413 ip-10-0-129-128 kubenswrapper[2559]: W0712 17:53:11.536948 2559 feature_gate.go:328] unrecognized feature gate: DNSNameResolver Jul 12 17:53:11.658413 ip-10-0-129-128 kubenswrapper[2559]: W0712 17:53:11.536960 2559 feature_gate.go:328] unrecognized feature gate: AWSDualStackInstall Jul 12 17:53:11.658413 ip-10-0-129-128 kubenswrapper[2559]: W0712 17:53:11.536963 2559 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesAzure Jul 12 17:53:11.658413 ip-10-0-129-128 kubenswrapper[2559]: W0712 17:53:11.536966 2559 feature_gate.go:328] unrecognized feature gate: HyperShiftOnlyDynamicResourceAllocation Jul 12 17:53:11.658413 ip-10-0-129-128 kubenswrapper[2559]: W0712 17:53:11.536971 2559 feature_gate.go:351] Setting GA feature gate VolumeAttributesClass=true. It will be removed in a future release. Jul 12 17:53:11.658413 ip-10-0-129-128 kubenswrapper[2559]: W0712 17:53:11.536974 2559 feature_gate.go:328] unrecognized feature gate: OVNObservability Jul 12 17:53:11.658413 ip-10-0-129-128 kubenswrapper[2559]: W0712 17:53:11.536978 2559 feature_gate.go:328] unrecognized feature gate: GCPClusterHostedDNSInstall Jul 12 17:53:11.658413 ip-10-0-129-128 kubenswrapper[2559]: W0712 17:53:11.536981 2559 feature_gate.go:328] unrecognized feature gate: AWSServiceLBNetworkSecurityGroup Jul 12 17:53:11.658413 ip-10-0-129-128 kubenswrapper[2559]: W0712 17:53:11.536984 2559 feature_gate.go:328] unrecognized feature gate: BootcNodeManagement Jul 12 17:53:11.658413 ip-10-0-129-128 kubenswrapper[2559]: W0712 17:53:11.536986 2559 feature_gate.go:328] unrecognized feature gate: ExternalSnapshotMetadata Jul 12 17:53:11.658413 ip-10-0-129-128 kubenswrapper[2559]: W0712 17:53:11.536989 2559 feature_gate.go:328] unrecognized feature gate: CRDCompatibilityRequirementOperator Jul 12 17:53:11.658413 ip-10-0-129-128 kubenswrapper[2559]: W0712 17:53:11.536992 2559 feature_gate.go:328] unrecognized feature gate: NetworkLiveMigration Jul 12 17:53:11.658413 ip-10-0-129-128 kubenswrapper[2559]: W0712 17:53:11.536995 2559 feature_gate.go:328] unrecognized feature gate: ExternalOIDC Jul 12 17:53:11.660218 ip-10-0-129-128 kubenswrapper[2559]: W0712 17:53:11.536997 2559 feature_gate.go:328] unrecognized feature gate: SigstoreImageVerificationPKI Jul 12 17:53:11.660218 ip-10-0-129-128 kubenswrapper[2559]: W0712 17:53:11.537000 2559 feature_gate.go:328] unrecognized feature gate: BootImageSkewEnforcement Jul 12 17:53:11.660218 ip-10-0-129-128 kubenswrapper[2559]: W0712 17:53:11.537003 2559 feature_gate.go:328] unrecognized feature gate: ManagedBootImages Jul 12 17:53:11.660218 ip-10-0-129-128 kubenswrapper[2559]: W0712 17:53:11.537007 2559 feature_gate.go:349] Setting deprecated feature gate KMSv1=true. It will be removed in a future release. Jul 12 17:53:11.660218 ip-10-0-129-128 kubenswrapper[2559]: W0712 17:53:11.537010 2559 feature_gate.go:328] unrecognized feature gate: NoRegistryClusterInstall Jul 12 17:53:11.660218 ip-10-0-129-128 kubenswrapper[2559]: W0712 17:53:11.537013 2559 feature_gate.go:328] unrecognized feature gate: NewOLMPreflightPermissionChecks Jul 12 17:53:11.660218 ip-10-0-129-128 kubenswrapper[2559]: W0712 17:53:11.537016 2559 feature_gate.go:328] unrecognized feature gate: IngressControllerDynamicConfigurationManager Jul 12 17:53:11.660218 ip-10-0-129-128 kubenswrapper[2559]: W0712 17:53:11.537019 2559 feature_gate.go:328] unrecognized feature gate: CPMSMachineNamePrefix Jul 12 17:53:11.660218 ip-10-0-129-128 kubenswrapper[2559]: W0712 17:53:11.537022 2559 feature_gate.go:328] unrecognized feature gate: HighlyAvailableArbiter Jul 12 17:53:11.660218 ip-10-0-129-128 kubenswrapper[2559]: W0712 17:53:11.537024 2559 feature_gate.go:328] unrecognized feature gate: ExternalOIDCWithUIDAndExtraClaimMappings Jul 12 17:53:11.660218 ip-10-0-129-128 kubenswrapper[2559]: W0712 17:53:11.537028 2559 feature_gate.go:328] unrecognized feature gate: AlibabaPlatform Jul 12 17:53:11.660218 ip-10-0-129-128 kubenswrapper[2559]: W0712 17:53:11.537038 2559 feature_gate.go:328] unrecognized feature gate: InsightsOnDemandDataGather Jul 12 17:53:11.660218 ip-10-0-129-128 kubenswrapper[2559]: W0712 17:53:11.537041 2559 feature_gate.go:328] unrecognized feature gate: EventTTL Jul 12 17:53:11.660218 ip-10-0-129-128 kubenswrapper[2559]: W0712 17:53:11.537046 2559 feature_gate.go:351] Setting GA feature gate ServiceAccountTokenNodeBinding=true. It will be removed in a future release. Jul 12 17:53:11.660218 ip-10-0-129-128 kubenswrapper[2559]: W0712 17:53:11.537050 2559 feature_gate.go:328] unrecognized feature gate: MultiDiskSetup Jul 12 17:53:11.660218 ip-10-0-129-128 kubenswrapper[2559]: W0712 17:53:11.537053 2559 feature_gate.go:328] unrecognized feature gate: OnPremDNSRecords Jul 12 17:53:11.660218 ip-10-0-129-128 kubenswrapper[2559]: W0712 17:53:11.537056 2559 feature_gate.go:328] unrecognized feature gate: PinnedImages Jul 12 17:53:11.660218 ip-10-0-129-128 kubenswrapper[2559]: W0712 17:53:11.537058 2559 feature_gate.go:328] unrecognized feature gate: NewOLMWebhookProviderOpenshiftServiceCA Jul 12 17:53:11.660218 ip-10-0-129-128 kubenswrapper[2559]: W0712 17:53:11.537061 2559 feature_gate.go:328] unrecognized feature gate: ClusterMonitoringConfig Jul 12 17:53:11.662520 ip-10-0-129-128 kubenswrapper[2559]: W0712 17:53:11.537064 2559 feature_gate.go:328] unrecognized feature gate: DyanmicServiceEndpointIBMCloud Jul 12 17:53:11.662520 ip-10-0-129-128 kubenswrapper[2559]: W0712 17:53:11.537067 2559 feature_gate.go:328] unrecognized feature gate: MachineConfigNodes Jul 12 17:53:11.662520 ip-10-0-129-128 kubenswrapper[2559]: W0712 17:53:11.537070 2559 feature_gate.go:328] unrecognized feature gate: UpgradeStatus Jul 12 17:53:11.662520 ip-10-0-129-128 kubenswrapper[2559]: W0712 17:53:11.537072 2559 feature_gate.go:328] unrecognized feature gate: SigstoreImageVerification Jul 12 17:53:11.662520 ip-10-0-129-128 kubenswrapper[2559]: W0712 17:53:11.537075 2559 feature_gate.go:328] unrecognized feature gate: BuildCSIVolumes Jul 12 17:53:11.662520 ip-10-0-129-128 kubenswrapper[2559]: W0712 17:53:11.537078 2559 feature_gate.go:328] unrecognized feature gate: KMSEncryption Jul 12 17:53:11.662520 ip-10-0-129-128 kubenswrapper[2559]: W0712 17:53:11.537080 2559 feature_gate.go:328] unrecognized feature gate: AzureWorkloadIdentity Jul 12 17:53:11.662520 ip-10-0-129-128 kubenswrapper[2559]: W0712 17:53:11.537083 2559 feature_gate.go:328] unrecognized feature gate: VSphereMultiNetworks Jul 12 17:53:11.662520 ip-10-0-129-128 kubenswrapper[2559]: W0712 17:53:11.537085 2559 feature_gate.go:328] unrecognized feature gate: IrreconcilableMachineConfig Jul 12 17:53:11.662520 ip-10-0-129-128 kubenswrapper[2559]: W0712 17:53:11.537088 2559 feature_gate.go:328] unrecognized feature gate: NetworkDiagnosticsConfig Jul 12 17:53:11.662520 ip-10-0-129-128 kubenswrapper[2559]: W0712 17:53:11.537091 2559 feature_gate.go:328] unrecognized feature gate: PreconfiguredUDNAddresses Jul 12 17:53:11.662520 ip-10-0-129-128 kubenswrapper[2559]: W0712 17:53:11.537093 2559 feature_gate.go:328] unrecognized feature gate: ShortCertRotation Jul 12 17:53:11.662520 ip-10-0-129-128 kubenswrapper[2559]: W0712 17:53:11.537096 2559 feature_gate.go:328] unrecognized feature gate: MixedCPUsAllocation Jul 12 17:53:11.662520 ip-10-0-129-128 kubenswrapper[2559]: W0712 17:53:11.537099 2559 feature_gate.go:328] unrecognized feature gate: RouteAdvertisements Jul 12 17:53:11.662520 ip-10-0-129-128 kubenswrapper[2559]: W0712 17:53:11.537101 2559 feature_gate.go:328] unrecognized feature gate: AdminNetworkPolicy Jul 12 17:53:11.662520 ip-10-0-129-128 kubenswrapper[2559]: W0712 17:53:11.537104 2559 feature_gate.go:328] unrecognized feature gate: ConsolePluginContentSecurityPolicy Jul 12 17:53:11.662520 ip-10-0-129-128 kubenswrapper[2559]: W0712 17:53:11.537106 2559 feature_gate.go:328] unrecognized feature gate: ClusterAPIMachineManagement Jul 12 17:53:11.662520 ip-10-0-129-128 kubenswrapper[2559]: W0712 17:53:11.537109 2559 feature_gate.go:328] unrecognized feature gate: OpenShiftPodSecurityAdmission Jul 12 17:53:11.662520 ip-10-0-129-128 kubenswrapper[2559]: W0712 17:53:11.537111 2559 feature_gate.go:328] unrecognized feature gate: EtcdBackendQuota Jul 12 17:53:11.662520 ip-10-0-129-128 kubenswrapper[2559]: W0712 17:53:11.537115 2559 feature_gate.go:328] unrecognized feature gate: AzureMultiDisk Jul 12 17:53:11.675221 ip-10-0-129-128 kubenswrapper[2559]: W0712 17:53:11.537117 2559 feature_gate.go:328] unrecognized feature gate: ClusterAPIInstallIBMCloud Jul 12 17:53:11.675221 ip-10-0-129-128 kubenswrapper[2559]: W0712 17:53:11.537120 2559 feature_gate.go:328] unrecognized feature gate: OSStreams Jul 12 17:53:11.675221 ip-10-0-129-128 kubenswrapper[2559]: W0712 17:53:11.537122 2559 feature_gate.go:328] unrecognized feature gate: AWSClusterHostedDNSInstall Jul 12 17:53:11.675221 ip-10-0-129-128 kubenswrapper[2559]: W0712 17:53:11.537125 2559 feature_gate.go:328] unrecognized feature gate: MachineAPIMigration Jul 12 17:53:11.675221 ip-10-0-129-128 kubenswrapper[2559]: W0712 17:53:11.537127 2559 feature_gate.go:328] unrecognized feature gate: VSphereHostVMGroupZonal Jul 12 17:53:11.675221 ip-10-0-129-128 kubenswrapper[2559]: W0712 17:53:11.537135 2559 feature_gate.go:328] unrecognized feature gate: InsightsConfig Jul 12 17:53:11.675221 ip-10-0-129-128 kubenswrapper[2559]: W0712 17:53:11.537139 2559 feature_gate.go:328] unrecognized feature gate: MetricsCollectionProfiles Jul 12 17:53:11.675221 ip-10-0-129-128 kubenswrapper[2559]: W0712 17:53:11.537141 2559 feature_gate.go:328] unrecognized feature gate: MultiArchInstallAzure Jul 12 17:53:11.675221 ip-10-0-129-128 kubenswrapper[2559]: W0712 17:53:11.537162 2559 feature_gate.go:328] unrecognized feature gate: Example Jul 12 17:53:11.675221 ip-10-0-129-128 kubenswrapper[2559]: W0712 17:53:11.537165 2559 feature_gate.go:328] unrecognized feature gate: KMSEncryptionProvider Jul 12 17:53:11.675221 ip-10-0-129-128 kubenswrapper[2559]: W0712 17:53:11.537168 2559 feature_gate.go:328] unrecognized feature gate: GCPClusterHostedDNS Jul 12 17:53:11.675221 ip-10-0-129-128 kubenswrapper[2559]: W0712 17:53:11.537170 2559 feature_gate.go:328] unrecognized feature gate: ImageStreamImportMode Jul 12 17:53:11.675221 ip-10-0-129-128 kubenswrapper[2559]: W0712 17:53:11.537173 2559 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesAWS Jul 12 17:53:11.675221 ip-10-0-129-128 kubenswrapper[2559]: W0712 17:53:11.537176 2559 feature_gate.go:328] unrecognized feature gate: GatewayAPIController Jul 12 17:53:11.675221 ip-10-0-129-128 kubenswrapper[2559]: W0712 17:53:11.537179 2559 feature_gate.go:328] unrecognized feature gate: AWSClusterHostedDNS Jul 12 17:53:11.675221 ip-10-0-129-128 kubenswrapper[2559]: W0712 17:53:11.537182 2559 feature_gate.go:328] unrecognized feature gate: GCPCustomAPIEndpoints Jul 12 17:53:11.675221 ip-10-0-129-128 kubenswrapper[2559]: W0712 17:53:11.537184 2559 feature_gate.go:328] unrecognized feature gate: VSphereMultiDisk Jul 12 17:53:11.675221 ip-10-0-129-128 kubenswrapper[2559]: W0712 17:53:11.537187 2559 feature_gate.go:328] unrecognized feature gate: VSphereMixedNodeEnv Jul 12 17:53:11.675221 ip-10-0-129-128 kubenswrapper[2559]: W0712 17:53:11.537189 2559 feature_gate.go:328] unrecognized feature gate: ProvisioningRequestAvailable Jul 12 17:53:11.675221 ip-10-0-129-128 kubenswrapper[2559]: W0712 17:53:11.537192 2559 feature_gate.go:328] unrecognized feature gate: GatewayAPI Jul 12 17:53:11.677577 ip-10-0-129-128 kubenswrapper[2559]: W0712 17:53:11.537195 2559 feature_gate.go:328] unrecognized feature gate: ImageModeStatusReporting Jul 12 17:53:11.677577 ip-10-0-129-128 kubenswrapper[2559]: W0712 17:53:11.537198 2559 feature_gate.go:328] unrecognized feature gate: MachineAPIOperatorDisableMachineHealthCheckController Jul 12 17:53:11.677577 ip-10-0-129-128 kubenswrapper[2559]: W0712 17:53:11.537201 2559 feature_gate.go:328] unrecognized feature gate: NewOLMBoxCutterRuntime Jul 12 17:53:11.677577 ip-10-0-129-128 kubenswrapper[2559]: W0712 17:53:11.537203 2559 feature_gate.go:328] unrecognized feature gate: GatewayAPIWithoutOLM Jul 12 17:53:11.677577 ip-10-0-129-128 kubenswrapper[2559]: W0712 17:53:11.537206 2559 feature_gate.go:328] unrecognized feature gate: AzureDualStackInstall Jul 12 17:53:11.677577 ip-10-0-129-128 kubenswrapper[2559]: W0712 17:53:11.537209 2559 feature_gate.go:328] unrecognized feature gate: GCPDualStackInstall Jul 12 17:53:11.677577 ip-10-0-129-128 kubenswrapper[2559]: W0712 17:53:11.537211 2559 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesvSphere Jul 12 17:53:11.677577 ip-10-0-129-128 kubenswrapper[2559]: W0712 17:53:11.537214 2559 feature_gate.go:328] unrecognized feature gate: AzureDedicatedHosts Jul 12 17:53:11.677577 ip-10-0-129-128 kubenswrapper[2559]: W0712 17:53:11.537217 2559 feature_gate.go:328] unrecognized feature gate: AWSDedicatedHosts Jul 12 17:53:11.677577 ip-10-0-129-128 kubenswrapper[2559]: W0712 17:53:11.537220 2559 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesCPMS Jul 12 17:53:11.677577 ip-10-0-129-128 kubenswrapper[2559]: W0712 17:53:11.537223 2559 feature_gate.go:328] unrecognized feature gate: NutanixMultiSubnets Jul 12 17:53:11.677577 ip-10-0-129-128 kubenswrapper[2559]: W0712 17:53:11.537226 2559 feature_gate.go:328] unrecognized feature gate: VSphereConfigurableMaxAllowedBlockVolumesPerNode Jul 12 17:53:11.677577 ip-10-0-129-128 kubenswrapper[2559]: W0712 17:53:11.537228 2559 feature_gate.go:328] unrecognized feature gate: ClusterAPIInstall Jul 12 17:53:11.677577 ip-10-0-129-128 kubenswrapper[2559]: W0712 17:53:11.540729 2559 feature_gate.go:328] unrecognized feature gate: KMSEncryption Jul 12 17:53:11.677577 ip-10-0-129-128 kubenswrapper[2559]: W0712 17:53:11.540737 2559 feature_gate.go:328] unrecognized feature gate: ExternalOIDCWithUIDAndExtraClaimMappings Jul 12 17:53:11.677577 ip-10-0-129-128 kubenswrapper[2559]: W0712 17:53:11.540740 2559 feature_gate.go:328] unrecognized feature gate: ClusterVersionOperatorConfiguration Jul 12 17:53:11.677577 ip-10-0-129-128 kubenswrapper[2559]: W0712 17:53:11.540743 2559 feature_gate.go:328] unrecognized feature gate: MultiDiskSetup Jul 12 17:53:11.677577 ip-10-0-129-128 kubenswrapper[2559]: W0712 17:53:11.540747 2559 feature_gate.go:328] unrecognized feature gate: AWSDedicatedHosts Jul 12 17:53:11.677577 ip-10-0-129-128 kubenswrapper[2559]: W0712 17:53:11.540751 2559 feature_gate.go:328] unrecognized feature gate: HyperShiftOnlyDynamicResourceAllocation Jul 12 17:53:11.690313 ip-10-0-129-128 kubenswrapper[2559]: W0712 17:53:11.540753 2559 feature_gate.go:328] unrecognized feature gate: ClusterAPIMachineManagementVSphere Jul 12 17:53:11.690313 ip-10-0-129-128 kubenswrapper[2559]: W0712 17:53:11.540756 2559 feature_gate.go:328] unrecognized feature gate: HighlyAvailableArbiter Jul 12 17:53:11.690313 ip-10-0-129-128 kubenswrapper[2559]: W0712 17:53:11.540759 2559 feature_gate.go:328] unrecognized feature gate: ShortCertRotation Jul 12 17:53:11.690313 ip-10-0-129-128 kubenswrapper[2559]: W0712 17:53:11.540762 2559 feature_gate.go:328] unrecognized feature gate: ManagedBootImages Jul 12 17:53:11.690313 ip-10-0-129-128 kubenswrapper[2559]: W0712 17:53:11.540765 2559 feature_gate.go:328] unrecognized feature gate: MultiArchInstallAzure Jul 12 17:53:11.690313 ip-10-0-129-128 kubenswrapper[2559]: W0712 17:53:11.540767 2559 feature_gate.go:328] unrecognized feature gate: IrreconcilableMachineConfig Jul 12 17:53:11.690313 ip-10-0-129-128 kubenswrapper[2559]: W0712 17:53:11.540770 2559 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesCPMS Jul 12 17:53:11.690313 ip-10-0-129-128 kubenswrapper[2559]: W0712 17:53:11.540773 2559 feature_gate.go:328] unrecognized feature gate: AutomatedEtcdBackup Jul 12 17:53:11.690313 ip-10-0-129-128 kubenswrapper[2559]: W0712 17:53:11.540775 2559 feature_gate.go:328] unrecognized feature gate: NewOLMOwnSingleNamespace Jul 12 17:53:11.690313 ip-10-0-129-128 kubenswrapper[2559]: W0712 17:53:11.540778 2559 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesAWS Jul 12 17:53:11.690313 ip-10-0-129-128 kubenswrapper[2559]: W0712 17:53:11.540782 2559 feature_gate.go:351] Setting GA feature gate VolumeAttributesClass=true. It will be removed in a future release. Jul 12 17:53:11.690313 ip-10-0-129-128 kubenswrapper[2559]: W0712 17:53:11.540785 2559 feature_gate.go:328] unrecognized feature gate: UpgradeStatus Jul 12 17:53:11.690313 ip-10-0-129-128 kubenswrapper[2559]: W0712 17:53:11.540788 2559 feature_gate.go:328] unrecognized feature gate: GatewayAPIWithoutOLM Jul 12 17:53:11.690313 ip-10-0-129-128 kubenswrapper[2559]: W0712 17:53:11.540792 2559 feature_gate.go:349] Setting deprecated feature gate KMSv1=true. It will be removed in a future release. Jul 12 17:53:11.690313 ip-10-0-129-128 kubenswrapper[2559]: W0712 17:53:11.540797 2559 feature_gate.go:328] unrecognized feature gate: OVNObservability Jul 12 17:53:11.690313 ip-10-0-129-128 kubenswrapper[2559]: W0712 17:53:11.540800 2559 feature_gate.go:328] unrecognized feature gate: PreconfiguredUDNAddresses Jul 12 17:53:11.690313 ip-10-0-129-128 kubenswrapper[2559]: W0712 17:53:11.540803 2559 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesvSphere Jul 12 17:53:11.690313 ip-10-0-129-128 kubenswrapper[2559]: W0712 17:53:11.540807 2559 feature_gate.go:351] Setting GA feature gate ServiceAccountTokenNodeBinding=true. It will be removed in a future release. Jul 12 17:53:11.690313 ip-10-0-129-128 kubenswrapper[2559]: W0712 17:53:11.540810 2559 feature_gate.go:328] unrecognized feature gate: SignatureStores Jul 12 17:53:11.706697 ip-10-0-129-128 kubenswrapper[2559]: W0712 17:53:11.540813 2559 feature_gate.go:328] unrecognized feature gate: MachineAPIMigration Jul 12 17:53:11.706697 ip-10-0-129-128 kubenswrapper[2559]: W0712 17:53:11.540816 2559 feature_gate.go:328] unrecognized feature gate: PinnedImages Jul 12 17:53:11.706697 ip-10-0-129-128 kubenswrapper[2559]: W0712 17:53:11.540819 2559 feature_gate.go:328] unrecognized feature gate: NewOLMBoxCutterRuntime Jul 12 17:53:11.706697 ip-10-0-129-128 kubenswrapper[2559]: W0712 17:53:11.540821 2559 feature_gate.go:328] unrecognized feature gate: VSphereMultiDisk Jul 12 17:53:11.706697 ip-10-0-129-128 kubenswrapper[2559]: W0712 17:53:11.540824 2559 feature_gate.go:328] unrecognized feature gate: AzureClusterHostedDNSInstall Jul 12 17:53:11.706697 ip-10-0-129-128 kubenswrapper[2559]: W0712 17:53:11.540826 2559 feature_gate.go:328] unrecognized feature gate: AdditionalRoutingCapabilities Jul 12 17:53:11.706697 ip-10-0-129-128 kubenswrapper[2559]: W0712 17:53:11.540829 2559 feature_gate.go:328] unrecognized feature gate: NetworkLiveMigration Jul 12 17:53:11.706697 ip-10-0-129-128 kubenswrapper[2559]: W0712 17:53:11.540832 2559 feature_gate.go:328] unrecognized feature gate: VSphereConfigurableMaxAllowedBlockVolumesPerNode Jul 12 17:53:11.706697 ip-10-0-129-128 kubenswrapper[2559]: W0712 17:53:11.540835 2559 feature_gate.go:328] unrecognized feature gate: Example2 Jul 12 17:53:11.706697 ip-10-0-129-128 kubenswrapper[2559]: W0712 17:53:11.540838 2559 feature_gate.go:328] unrecognized feature gate: VolumeGroupSnapshot Jul 12 17:53:11.706697 ip-10-0-129-128 kubenswrapper[2559]: W0712 17:53:11.540841 2559 feature_gate.go:328] unrecognized feature gate: ImageModeStatusReporting Jul 12 17:53:11.706697 ip-10-0-129-128 kubenswrapper[2559]: W0712 17:53:11.540845 2559 feature_gate.go:328] unrecognized feature gate: ProvisioningRequestAvailable Jul 12 17:53:11.706697 ip-10-0-129-128 kubenswrapper[2559]: W0712 17:53:11.540847 2559 feature_gate.go:328] unrecognized feature gate: BootImageSkewEnforcement Jul 12 17:53:11.706697 ip-10-0-129-128 kubenswrapper[2559]: W0712 17:53:11.540856 2559 feature_gate.go:328] unrecognized feature gate: NewOLMWebhookProviderOpenshiftServiceCA Jul 12 17:53:11.706697 ip-10-0-129-128 kubenswrapper[2559]: W0712 17:53:11.540859 2559 feature_gate.go:328] unrecognized feature gate: DNSNameResolver Jul 12 17:53:11.706697 ip-10-0-129-128 kubenswrapper[2559]: W0712 17:53:11.540861 2559 feature_gate.go:328] unrecognized feature gate: DualReplica Jul 12 17:53:11.706697 ip-10-0-129-128 kubenswrapper[2559]: W0712 17:53:11.540864 2559 feature_gate.go:328] unrecognized feature gate: AWSClusterHostedDNSInstall Jul 12 17:53:11.706697 ip-10-0-129-128 kubenswrapper[2559]: W0712 17:53:11.540867 2559 feature_gate.go:328] unrecognized feature gate: VSphereMixedNodeEnv Jul 12 17:53:11.706697 ip-10-0-129-128 kubenswrapper[2559]: W0712 17:53:11.540869 2559 feature_gate.go:328] unrecognized feature gate: InsightsOnDemandDataGather Jul 12 17:53:11.706697 ip-10-0-129-128 kubenswrapper[2559]: W0712 17:53:11.540872 2559 feature_gate.go:328] unrecognized feature gate: ClusterAPIMachineManagement Jul 12 17:53:11.722573 ip-10-0-129-128 kubenswrapper[2559]: W0712 17:53:11.540874 2559 feature_gate.go:328] unrecognized feature gate: RouteAdvertisements Jul 12 17:53:11.722573 ip-10-0-129-128 kubenswrapper[2559]: W0712 17:53:11.540877 2559 feature_gate.go:328] unrecognized feature gate: NetworkSegmentation Jul 12 17:53:11.722573 ip-10-0-129-128 kubenswrapper[2559]: W0712 17:53:11.540879 2559 feature_gate.go:328] unrecognized feature gate: AzureMultiDisk Jul 12 17:53:11.722573 ip-10-0-129-128 kubenswrapper[2559]: W0712 17:53:11.540882 2559 feature_gate.go:328] unrecognized feature gate: CPMSMachineNamePrefix Jul 12 17:53:11.722573 ip-10-0-129-128 kubenswrapper[2559]: W0712 17:53:11.540885 2559 feature_gate.go:328] unrecognized feature gate: Example Jul 12 17:53:11.722573 ip-10-0-129-128 kubenswrapper[2559]: W0712 17:53:11.540887 2559 feature_gate.go:328] unrecognized feature gate: AzureDualStackInstall Jul 12 17:53:11.722573 ip-10-0-129-128 kubenswrapper[2559]: W0712 17:53:11.540889 2559 feature_gate.go:328] unrecognized feature gate: ExternalSnapshotMetadata Jul 12 17:53:11.722573 ip-10-0-129-128 kubenswrapper[2559]: W0712 17:53:11.540892 2559 feature_gate.go:328] unrecognized feature gate: ClusterAPIInstallIBMCloud Jul 12 17:53:11.722573 ip-10-0-129-128 kubenswrapper[2559]: W0712 17:53:11.540894 2559 feature_gate.go:328] unrecognized feature gate: IngressControllerDynamicConfigurationManager Jul 12 17:53:11.722573 ip-10-0-129-128 kubenswrapper[2559]: W0712 17:53:11.540907 2559 feature_gate.go:328] unrecognized feature gate: OpenShiftPodSecurityAdmission Jul 12 17:53:11.722573 ip-10-0-129-128 kubenswrapper[2559]: W0712 17:53:11.540910 2559 feature_gate.go:328] unrecognized feature gate: GCPCustomAPIEndpointsInstall Jul 12 17:53:11.722573 ip-10-0-129-128 kubenswrapper[2559]: W0712 17:53:11.540913 2559 feature_gate.go:328] unrecognized feature gate: ClusterAPIInstall Jul 12 17:53:11.722573 ip-10-0-129-128 kubenswrapper[2559]: W0712 17:53:11.540915 2559 feature_gate.go:328] unrecognized feature gate: MachineAPIOperatorDisableMachineHealthCheckController Jul 12 17:53:11.722573 ip-10-0-129-128 kubenswrapper[2559]: W0712 17:53:11.540918 2559 feature_gate.go:328] unrecognized feature gate: VSphereHostVMGroupZonal Jul 12 17:53:11.722573 ip-10-0-129-128 kubenswrapper[2559]: W0712 17:53:11.540921 2559 feature_gate.go:328] unrecognized feature gate: GCPDualStackInstall Jul 12 17:53:11.722573 ip-10-0-129-128 kubenswrapper[2559]: W0712 17:53:11.540923 2559 feature_gate.go:328] unrecognized feature gate: NutanixMultiSubnets Jul 12 17:53:11.722573 ip-10-0-129-128 kubenswrapper[2559]: W0712 17:53:11.540926 2559 feature_gate.go:328] unrecognized feature gate: KMSEncryptionProvider Jul 12 17:53:11.722573 ip-10-0-129-128 kubenswrapper[2559]: W0712 17:53:11.540934 2559 feature_gate.go:328] unrecognized feature gate: InsightsConfig Jul 12 17:53:11.722573 ip-10-0-129-128 kubenswrapper[2559]: W0712 17:53:11.540937 2559 feature_gate.go:328] unrecognized feature gate: AWSServiceLBNetworkSecurityGroup Jul 12 17:53:11.722573 ip-10-0-129-128 kubenswrapper[2559]: W0712 17:53:11.540940 2559 feature_gate.go:328] unrecognized feature gate: AWSClusterHostedDNS Jul 12 17:53:11.864632 ip-10-0-129-128 kubenswrapper[2559]: W0712 17:53:11.540942 2559 feature_gate.go:328] unrecognized feature gate: MachineConfigNodes Jul 12 17:53:11.864632 ip-10-0-129-128 kubenswrapper[2559]: W0712 17:53:11.540945 2559 feature_gate.go:328] unrecognized feature gate: AlibabaPlatform Jul 12 17:53:11.864632 ip-10-0-129-128 kubenswrapper[2559]: W0712 17:53:11.540951 2559 feature_gate.go:328] unrecognized feature gate: BuildCSIVolumes Jul 12 17:53:11.864632 ip-10-0-129-128 kubenswrapper[2559]: W0712 17:53:11.540957 2559 feature_gate.go:328] unrecognized feature gate: NewOLMPreflightPermissionChecks Jul 12 17:53:11.864632 ip-10-0-129-128 kubenswrapper[2559]: W0712 17:53:11.540960 2559 feature_gate.go:328] unrecognized feature gate: GatewayAPI Jul 12 17:53:11.864632 ip-10-0-129-128 kubenswrapper[2559]: W0712 17:53:11.540962 2559 feature_gate.go:328] unrecognized feature gate: AzureWorkloadIdentity Jul 12 17:53:11.864632 ip-10-0-129-128 kubenswrapper[2559]: W0712 17:53:11.540967 2559 feature_gate.go:328] unrecognized feature gate: BootcNodeManagement Jul 12 17:53:11.864632 ip-10-0-129-128 kubenswrapper[2559]: W0712 17:53:11.540970 2559 feature_gate.go:328] unrecognized feature gate: SigstoreImageVerificationPKI Jul 12 17:53:11.864632 ip-10-0-129-128 kubenswrapper[2559]: W0712 17:53:11.540972 2559 feature_gate.go:328] unrecognized feature gate: GCPClusterHostedDNSInstall Jul 12 17:53:11.864632 ip-10-0-129-128 kubenswrapper[2559]: W0712 17:53:11.540975 2559 feature_gate.go:328] unrecognized feature gate: OnPremDNSRecords Jul 12 17:53:11.864632 ip-10-0-129-128 kubenswrapper[2559]: W0712 17:53:11.540980 2559 feature_gate.go:328] unrecognized feature gate: SigstoreImageVerification Jul 12 17:53:11.864632 ip-10-0-129-128 kubenswrapper[2559]: W0712 17:53:11.540982 2559 feature_gate.go:328] unrecognized feature gate: MetricsCollectionProfiles Jul 12 17:53:11.864632 ip-10-0-129-128 kubenswrapper[2559]: W0712 17:53:11.540985 2559 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesAzure Jul 12 17:53:11.864632 ip-10-0-129-128 kubenswrapper[2559]: W0712 17:53:11.540987 2559 feature_gate.go:328] unrecognized feature gate: AzureDedicatedHosts Jul 12 17:53:11.864632 ip-10-0-129-128 kubenswrapper[2559]: W0712 17:53:11.540992 2559 feature_gate.go:328] unrecognized feature gate: GCPClusterHostedDNS Jul 12 17:53:11.864632 ip-10-0-129-128 kubenswrapper[2559]: W0712 17:53:11.540994 2559 feature_gate.go:328] unrecognized feature gate: OSStreams Jul 12 17:53:11.864632 ip-10-0-129-128 kubenswrapper[2559]: W0712 17:53:11.540998 2559 feature_gate.go:328] unrecognized feature gate: MixedCPUsAllocation Jul 12 17:53:11.864632 ip-10-0-129-128 kubenswrapper[2559]: W0712 17:53:11.541003 2559 feature_gate.go:328] unrecognized feature gate: ConsolePluginContentSecurityPolicy Jul 12 17:53:11.864632 ip-10-0-129-128 kubenswrapper[2559]: W0712 17:53:11.541005 2559 feature_gate.go:328] unrecognized feature gate: GatewayAPIController Jul 12 17:53:11.864632 ip-10-0-129-128 kubenswrapper[2559]: W0712 17:53:11.541008 2559 feature_gate.go:328] unrecognized feature gate: DyanmicServiceEndpointIBMCloud Jul 12 17:53:11.866511 ip-10-0-129-128 kubenswrapper[2559]: W0712 17:53:11.541010 2559 feature_gate.go:328] unrecognized feature gate: AdminNetworkPolicy Jul 12 17:53:11.866511 ip-10-0-129-128 kubenswrapper[2559]: W0712 17:53:11.541015 2559 feature_gate.go:328] unrecognized feature gate: NewOLM Jul 12 17:53:11.866511 ip-10-0-129-128 kubenswrapper[2559]: W0712 17:53:11.541018 2559 feature_gate.go:328] unrecognized feature gate: NetworkDiagnosticsConfig Jul 12 17:53:11.866511 ip-10-0-129-128 kubenswrapper[2559]: W0712 17:53:11.541020 2559 feature_gate.go:328] unrecognized feature gate: EventTTL Jul 12 17:53:11.866511 ip-10-0-129-128 kubenswrapper[2559]: W0712 17:53:11.541023 2559 feature_gate.go:328] unrecognized feature gate: AWSDualStackInstall Jul 12 17:53:11.866511 ip-10-0-129-128 kubenswrapper[2559]: W0712 17:53:11.541028 2559 feature_gate.go:328] unrecognized feature gate: NoRegistryClusterInstall Jul 12 17:53:11.866511 ip-10-0-129-128 kubenswrapper[2559]: W0712 17:53:11.541030 2559 feature_gate.go:328] unrecognized feature gate: CRDCompatibilityRequirementOperator Jul 12 17:53:11.866511 ip-10-0-129-128 kubenswrapper[2559]: W0712 17:53:11.541033 2559 feature_gate.go:328] unrecognized feature gate: GCPCustomAPIEndpoints Jul 12 17:53:11.866511 ip-10-0-129-128 kubenswrapper[2559]: W0712 17:53:11.541035 2559 feature_gate.go:328] unrecognized feature gate: ClusterMonitoringConfig Jul 12 17:53:11.866511 ip-10-0-129-128 kubenswrapper[2559]: W0712 17:53:11.541040 2559 feature_gate.go:328] unrecognized feature gate: VSphereMultiNetworks Jul 12 17:53:11.866511 ip-10-0-129-128 kubenswrapper[2559]: W0712 17:53:11.541043 2559 feature_gate.go:328] unrecognized feature gate: ExternalOIDC Jul 12 17:53:11.866511 ip-10-0-129-128 kubenswrapper[2559]: W0712 17:53:11.541045 2559 feature_gate.go:328] unrecognized feature gate: EtcdBackendQuota Jul 12 17:53:11.866511 ip-10-0-129-128 kubenswrapper[2559]: W0712 17:53:11.541050 2559 feature_gate.go:328] unrecognized feature gate: ImageStreamImportMode Jul 12 17:53:11.866511 ip-10-0-129-128 kubenswrapper[2559]: W0712 17:53:11.541053 2559 feature_gate.go:328] unrecognized feature gate: NewOLMCatalogdAPIV1Metas Jul 12 17:53:11.866511 ip-10-0-129-128 kubenswrapper[2559]: W0712 17:53:11.541554 2559 feature_gate.go:328] unrecognized feature gate: Example Jul 12 17:53:11.866511 ip-10-0-129-128 kubenswrapper[2559]: W0712 17:53:11.541559 2559 feature_gate.go:328] unrecognized feature gate: MultiDiskSetup Jul 12 17:53:11.866511 ip-10-0-129-128 kubenswrapper[2559]: W0712 17:53:11.541562 2559 feature_gate.go:328] unrecognized feature gate: KMSEncryption Jul 12 17:53:11.866511 ip-10-0-129-128 kubenswrapper[2559]: W0712 17:53:11.541566 2559 feature_gate.go:328] unrecognized feature gate: AzureClusterHostedDNSInstall Jul 12 17:53:11.866511 ip-10-0-129-128 kubenswrapper[2559]: W0712 17:53:11.541570 2559 feature_gate.go:328] unrecognized feature gate: ImageModeStatusReporting Jul 12 17:53:11.866511 ip-10-0-129-128 kubenswrapper[2559]: W0712 17:53:11.541572 2559 feature_gate.go:328] unrecognized feature gate: GCPDualStackInstall Jul 12 17:53:11.868925 ip-10-0-129-128 kubenswrapper[2559]: W0712 17:53:11.541575 2559 feature_gate.go:328] unrecognized feature gate: ClusterAPIMachineManagementVSphere Jul 12 17:53:11.868925 ip-10-0-129-128 kubenswrapper[2559]: W0712 17:53:11.541582 2559 feature_gate.go:328] unrecognized feature gate: AWSClusterHostedDNS Jul 12 17:53:11.868925 ip-10-0-129-128 kubenswrapper[2559]: W0712 17:53:11.541585 2559 feature_gate.go:328] unrecognized feature gate: RouteAdvertisements Jul 12 17:53:11.868925 ip-10-0-129-128 kubenswrapper[2559]: W0712 17:53:11.541588 2559 feature_gate.go:328] unrecognized feature gate: BootcNodeManagement Jul 12 17:53:11.868925 ip-10-0-129-128 kubenswrapper[2559]: W0712 17:53:11.541590 2559 feature_gate.go:328] unrecognized feature gate: HighlyAvailableArbiter Jul 12 17:53:11.868925 ip-10-0-129-128 kubenswrapper[2559]: W0712 17:53:11.541593 2559 feature_gate.go:328] unrecognized feature gate: NetworkSegmentation Jul 12 17:53:11.868925 ip-10-0-129-128 kubenswrapper[2559]: W0712 17:53:11.541598 2559 feature_gate.go:351] Setting GA feature gate VolumeAttributesClass=true. It will be removed in a future release. Jul 12 17:53:11.868925 ip-10-0-129-128 kubenswrapper[2559]: W0712 17:53:11.541605 2559 feature_gate.go:328] unrecognized feature gate: DNSNameResolver Jul 12 17:53:11.868925 ip-10-0-129-128 kubenswrapper[2559]: W0712 17:53:11.541608 2559 feature_gate.go:328] unrecognized feature gate: ManagedBootImages Jul 12 17:53:11.868925 ip-10-0-129-128 kubenswrapper[2559]: W0712 17:53:11.541613 2559 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesAzure Jul 12 17:53:11.868925 ip-10-0-129-128 kubenswrapper[2559]: W0712 17:53:11.541616 2559 feature_gate.go:328] unrecognized feature gate: PreconfiguredUDNAddresses Jul 12 17:53:11.868925 ip-10-0-129-128 kubenswrapper[2559]: W0712 17:53:11.541619 2559 feature_gate.go:328] unrecognized feature gate: ClusterAPIMachineManagement Jul 12 17:53:11.868925 ip-10-0-129-128 kubenswrapper[2559]: W0712 17:53:11.541621 2559 feature_gate.go:328] unrecognized feature gate: AutomatedEtcdBackup Jul 12 17:53:11.868925 ip-10-0-129-128 kubenswrapper[2559]: W0712 17:53:11.541627 2559 feature_gate.go:328] unrecognized feature gate: SigstoreImageVerification Jul 12 17:53:11.868925 ip-10-0-129-128 kubenswrapper[2559]: W0712 17:53:11.541629 2559 feature_gate.go:328] unrecognized feature gate: ImageStreamImportMode Jul 12 17:53:11.868925 ip-10-0-129-128 kubenswrapper[2559]: W0712 17:53:11.541633 2559 feature_gate.go:349] Setting deprecated feature gate KMSv1=true. It will be removed in a future release. Jul 12 17:53:11.868925 ip-10-0-129-128 kubenswrapper[2559]: W0712 17:53:11.541640 2559 feature_gate.go:328] unrecognized feature gate: NewOLMBoxCutterRuntime Jul 12 17:53:11.868925 ip-10-0-129-128 kubenswrapper[2559]: W0712 17:53:11.541643 2559 feature_gate.go:328] unrecognized feature gate: ClusterAPIInstall Jul 12 17:53:11.868925 ip-10-0-129-128 kubenswrapper[2559]: W0712 17:53:11.541646 2559 feature_gate.go:328] unrecognized feature gate: AWSClusterHostedDNSInstall Jul 12 17:53:11.870730 ip-10-0-129-128 kubenswrapper[2559]: W0712 17:53:11.541651 2559 feature_gate.go:328] unrecognized feature gate: ClusterMonitoringConfig Jul 12 17:53:11.870730 ip-10-0-129-128 kubenswrapper[2559]: W0712 17:53:11.541654 2559 feature_gate.go:328] unrecognized feature gate: PinnedImages Jul 12 17:53:11.870730 ip-10-0-129-128 kubenswrapper[2559]: W0712 17:53:11.541657 2559 feature_gate.go:328] unrecognized feature gate: ProvisioningRequestAvailable Jul 12 17:53:11.870730 ip-10-0-129-128 kubenswrapper[2559]: W0712 17:53:11.541662 2559 feature_gate.go:328] unrecognized feature gate: NutanixMultiSubnets Jul 12 17:53:11.870730 ip-10-0-129-128 kubenswrapper[2559]: W0712 17:53:11.541665 2559 feature_gate.go:328] unrecognized feature gate: AdminNetworkPolicy Jul 12 17:53:11.870730 ip-10-0-129-128 kubenswrapper[2559]: W0712 17:53:11.541667 2559 feature_gate.go:328] unrecognized feature gate: GCPClusterHostedDNSInstall Jul 12 17:53:11.870730 ip-10-0-129-128 kubenswrapper[2559]: W0712 17:53:11.541670 2559 feature_gate.go:328] unrecognized feature gate: VSphereMultiDisk Jul 12 17:53:11.870730 ip-10-0-129-128 kubenswrapper[2559]: W0712 17:53:11.541676 2559 feature_gate.go:328] unrecognized feature gate: NewOLMWebhookProviderOpenshiftServiceCA Jul 12 17:53:11.870730 ip-10-0-129-128 kubenswrapper[2559]: W0712 17:53:11.541679 2559 feature_gate.go:328] unrecognized feature gate: MachineAPIOperatorDisableMachineHealthCheckController Jul 12 17:53:11.870730 ip-10-0-129-128 kubenswrapper[2559]: W0712 17:53:11.541682 2559 feature_gate.go:328] unrecognized feature gate: VSphereConfigurableMaxAllowedBlockVolumesPerNode Jul 12 17:53:11.870730 ip-10-0-129-128 kubenswrapper[2559]: W0712 17:53:11.541686 2559 feature_gate.go:328] unrecognized feature gate: ExternalOIDC Jul 12 17:53:11.870730 ip-10-0-129-128 kubenswrapper[2559]: W0712 17:53:11.541689 2559 feature_gate.go:328] unrecognized feature gate: ClusterVersionOperatorConfiguration Jul 12 17:53:11.870730 ip-10-0-129-128 kubenswrapper[2559]: W0712 17:53:11.541692 2559 feature_gate.go:328] unrecognized feature gate: GatewayAPI Jul 12 17:53:11.870730 ip-10-0-129-128 kubenswrapper[2559]: W0712 17:53:11.541697 2559 feature_gate.go:328] unrecognized feature gate: BuildCSIVolumes Jul 12 17:53:11.870730 ip-10-0-129-128 kubenswrapper[2559]: W0712 17:53:11.541700 2559 feature_gate.go:328] unrecognized feature gate: NewOLMPreflightPermissionChecks Jul 12 17:53:11.870730 ip-10-0-129-128 kubenswrapper[2559]: W0712 17:53:11.541702 2559 feature_gate.go:328] unrecognized feature gate: CPMSMachineNamePrefix Jul 12 17:53:11.870730 ip-10-0-129-128 kubenswrapper[2559]: W0712 17:53:11.541709 2559 feature_gate.go:351] Setting GA feature gate ServiceAccountTokenNodeBinding=true. It will be removed in a future release. Jul 12 17:53:11.870730 ip-10-0-129-128 kubenswrapper[2559]: W0712 17:53:11.541712 2559 feature_gate.go:328] unrecognized feature gate: UpgradeStatus Jul 12 17:53:11.870730 ip-10-0-129-128 kubenswrapper[2559]: W0712 17:53:11.541715 2559 feature_gate.go:328] unrecognized feature gate: NewOLMOwnSingleNamespace Jul 12 17:53:11.872881 ip-10-0-129-128 kubenswrapper[2559]: W0712 17:53:11.541721 2559 feature_gate.go:328] unrecognized feature gate: OnPremDNSRecords Jul 12 17:53:11.872881 ip-10-0-129-128 kubenswrapper[2559]: W0712 17:53:11.541725 2559 feature_gate.go:328] unrecognized feature gate: GatewayAPIWithoutOLM Jul 12 17:53:11.872881 ip-10-0-129-128 kubenswrapper[2559]: W0712 17:53:11.541728 2559 feature_gate.go:328] unrecognized feature gate: DualReplica Jul 12 17:53:11.872881 ip-10-0-129-128 kubenswrapper[2559]: W0712 17:53:11.541733 2559 feature_gate.go:328] unrecognized feature gate: DyanmicServiceEndpointIBMCloud Jul 12 17:53:11.872881 ip-10-0-129-128 kubenswrapper[2559]: W0712 17:53:11.541736 2559 feature_gate.go:328] unrecognized feature gate: GCPCustomAPIEndpointsInstall Jul 12 17:53:11.872881 ip-10-0-129-128 kubenswrapper[2559]: W0712 17:53:11.541739 2559 feature_gate.go:328] unrecognized feature gate: VSphereHostVMGroupZonal Jul 12 17:53:11.872881 ip-10-0-129-128 kubenswrapper[2559]: W0712 17:53:11.541744 2559 feature_gate.go:328] unrecognized feature gate: KMSEncryptionProvider Jul 12 17:53:11.872881 ip-10-0-129-128 kubenswrapper[2559]: W0712 17:53:11.541747 2559 feature_gate.go:328] unrecognized feature gate: MixedCPUsAllocation Jul 12 17:53:11.872881 ip-10-0-129-128 kubenswrapper[2559]: W0712 17:53:11.541750 2559 feature_gate.go:328] unrecognized feature gate: NetworkDiagnosticsConfig Jul 12 17:53:11.872881 ip-10-0-129-128 kubenswrapper[2559]: W0712 17:53:11.541753 2559 feature_gate.go:328] unrecognized feature gate: IrreconcilableMachineConfig Jul 12 17:53:11.872881 ip-10-0-129-128 kubenswrapper[2559]: W0712 17:53:11.541757 2559 feature_gate.go:328] unrecognized feature gate: VSphereMultiNetworks Jul 12 17:53:11.872881 ip-10-0-129-128 kubenswrapper[2559]: W0712 17:53:11.541760 2559 feature_gate.go:328] unrecognized feature gate: AWSServiceLBNetworkSecurityGroup Jul 12 17:53:11.872881 ip-10-0-129-128 kubenswrapper[2559]: W0712 17:53:11.541762 2559 feature_gate.go:328] unrecognized feature gate: InsightsOnDemandDataGather Jul 12 17:53:11.872881 ip-10-0-129-128 kubenswrapper[2559]: W0712 17:53:11.541765 2559 feature_gate.go:328] unrecognized feature gate: ClusterAPIInstallIBMCloud Jul 12 17:53:11.872881 ip-10-0-129-128 kubenswrapper[2559]: W0712 17:53:11.541770 2559 feature_gate.go:328] unrecognized feature gate: AdditionalRoutingCapabilities Jul 12 17:53:11.872881 ip-10-0-129-128 kubenswrapper[2559]: W0712 17:53:11.541772 2559 feature_gate.go:328] unrecognized feature gate: OVNObservability Jul 12 17:53:11.872881 ip-10-0-129-128 kubenswrapper[2559]: W0712 17:53:11.541775 2559 feature_gate.go:328] unrecognized feature gate: NetworkLiveMigration Jul 12 17:53:11.872881 ip-10-0-129-128 kubenswrapper[2559]: W0712 17:53:11.541780 2559 feature_gate.go:328] unrecognized feature gate: MetricsCollectionProfiles Jul 12 17:53:11.872881 ip-10-0-129-128 kubenswrapper[2559]: W0712 17:53:11.541782 2559 feature_gate.go:328] unrecognized feature gate: SignatureStores Jul 12 17:53:11.872881 ip-10-0-129-128 kubenswrapper[2559]: W0712 17:53:11.541785 2559 feature_gate.go:328] unrecognized feature gate: AzureDualStackInstall Jul 12 17:53:11.874729 ip-10-0-129-128 kubenswrapper[2559]: W0712 17:53:11.541790 2559 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesAWS Jul 12 17:53:11.874729 ip-10-0-129-128 kubenswrapper[2559]: W0712 17:53:11.541793 2559 feature_gate.go:328] unrecognized feature gate: SigstoreImageVerificationPKI Jul 12 17:53:11.874729 ip-10-0-129-128 kubenswrapper[2559]: W0712 17:53:11.541795 2559 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesCPMS Jul 12 17:53:11.874729 ip-10-0-129-128 kubenswrapper[2559]: W0712 17:53:11.541798 2559 feature_gate.go:328] unrecognized feature gate: AlibabaPlatform Jul 12 17:53:11.874729 ip-10-0-129-128 kubenswrapper[2559]: W0712 17:53:11.541801 2559 feature_gate.go:328] unrecognized feature gate: GCPCustomAPIEndpoints Jul 12 17:53:11.874729 ip-10-0-129-128 kubenswrapper[2559]: W0712 17:53:11.541803 2559 feature_gate.go:328] unrecognized feature gate: EtcdBackendQuota Jul 12 17:53:11.874729 ip-10-0-129-128 kubenswrapper[2559]: W0712 17:53:11.541806 2559 feature_gate.go:328] unrecognized feature gate: MachineConfigNodes Jul 12 17:53:11.874729 ip-10-0-129-128 kubenswrapper[2559]: W0712 17:53:11.541809 2559 feature_gate.go:328] unrecognized feature gate: OpenShiftPodSecurityAdmission Jul 12 17:53:11.874729 ip-10-0-129-128 kubenswrapper[2559]: W0712 17:53:11.541816 2559 feature_gate.go:328] unrecognized feature gate: GCPClusterHostedDNS Jul 12 17:53:11.874729 ip-10-0-129-128 kubenswrapper[2559]: W0712 17:53:11.541819 2559 feature_gate.go:328] unrecognized feature gate: VSphereMixedNodeEnv Jul 12 17:53:11.874729 ip-10-0-129-128 kubenswrapper[2559]: W0712 17:53:11.541823 2559 feature_gate.go:328] unrecognized feature gate: ShortCertRotation Jul 12 17:53:11.874729 ip-10-0-129-128 kubenswrapper[2559]: W0712 17:53:11.541826 2559 feature_gate.go:328] unrecognized feature gate: BootImageSkewEnforcement Jul 12 17:53:11.874729 ip-10-0-129-128 kubenswrapper[2559]: W0712 17:53:11.541829 2559 feature_gate.go:328] unrecognized feature gate: OSStreams Jul 12 17:53:11.874729 ip-10-0-129-128 kubenswrapper[2559]: W0712 17:53:11.541835 2559 feature_gate.go:328] unrecognized feature gate: ConsolePluginContentSecurityPolicy Jul 12 17:53:11.874729 ip-10-0-129-128 kubenswrapper[2559]: W0712 17:53:11.541837 2559 feature_gate.go:328] unrecognized feature gate: AzureMultiDisk Jul 12 17:53:11.874729 ip-10-0-129-128 kubenswrapper[2559]: W0712 17:53:11.541840 2559 feature_gate.go:328] unrecognized feature gate: GatewayAPIController Jul 12 17:53:11.874729 ip-10-0-129-128 kubenswrapper[2559]: W0712 17:53:11.541842 2559 feature_gate.go:328] unrecognized feature gate: IngressControllerDynamicConfigurationManager Jul 12 17:53:11.874729 ip-10-0-129-128 kubenswrapper[2559]: W0712 17:53:11.541847 2559 feature_gate.go:328] unrecognized feature gate: MultiArchInstallAzure Jul 12 17:53:11.874729 ip-10-0-129-128 kubenswrapper[2559]: W0712 17:53:11.541849 2559 feature_gate.go:328] unrecognized feature gate: AWSDualStackInstall Jul 12 17:53:11.874729 ip-10-0-129-128 kubenswrapper[2559]: W0712 17:53:11.541852 2559 feature_gate.go:328] unrecognized feature gate: InsightsConfig Jul 12 17:53:11.877020 ip-10-0-129-128 kubenswrapper[2559]: W0712 17:53:11.541855 2559 feature_gate.go:328] unrecognized feature gate: VolumeGroupSnapshot Jul 12 17:53:11.877020 ip-10-0-129-128 kubenswrapper[2559]: W0712 17:53:11.541860 2559 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesvSphere Jul 12 17:53:11.877020 ip-10-0-129-128 kubenswrapper[2559]: W0712 17:53:11.541862 2559 feature_gate.go:328] unrecognized feature gate: MachineAPIMigration Jul 12 17:53:11.877020 ip-10-0-129-128 kubenswrapper[2559]: W0712 17:53:11.541864 2559 feature_gate.go:328] unrecognized feature gate: Example2 Jul 12 17:53:11.877020 ip-10-0-129-128 kubenswrapper[2559]: W0712 17:53:11.541867 2559 feature_gate.go:328] unrecognized feature gate: NewOLMCatalogdAPIV1Metas Jul 12 17:53:11.877020 ip-10-0-129-128 kubenswrapper[2559]: W0712 17:53:11.541871 2559 feature_gate.go:328] unrecognized feature gate: EventTTL Jul 12 17:53:11.877020 ip-10-0-129-128 kubenswrapper[2559]: W0712 17:53:11.541874 2559 feature_gate.go:328] unrecognized feature gate: ExternalOIDCWithUIDAndExtraClaimMappings Jul 12 17:53:11.877020 ip-10-0-129-128 kubenswrapper[2559]: W0712 17:53:11.541876 2559 feature_gate.go:328] unrecognized feature gate: HyperShiftOnlyDynamicResourceAllocation Jul 12 17:53:11.877020 ip-10-0-129-128 kubenswrapper[2559]: W0712 17:53:11.541879 2559 feature_gate.go:328] unrecognized feature gate: AzureDedicatedHosts Jul 12 17:53:11.877020 ip-10-0-129-128 kubenswrapper[2559]: W0712 17:53:11.541884 2559 feature_gate.go:328] unrecognized feature gate: AWSDedicatedHosts Jul 12 17:53:11.877020 ip-10-0-129-128 kubenswrapper[2559]: W0712 17:53:11.541886 2559 feature_gate.go:328] unrecognized feature gate: AzureWorkloadIdentity Jul 12 17:53:11.877020 ip-10-0-129-128 kubenswrapper[2559]: W0712 17:53:11.541889 2559 feature_gate.go:328] unrecognized feature gate: CRDCompatibilityRequirementOperator Jul 12 17:53:11.877020 ip-10-0-129-128 kubenswrapper[2559]: W0712 17:53:11.541895 2559 feature_gate.go:328] unrecognized feature gate: ExternalSnapshotMetadata Jul 12 17:53:11.877020 ip-10-0-129-128 kubenswrapper[2559]: W0712 17:53:11.541898 2559 feature_gate.go:328] unrecognized feature gate: NewOLM Jul 12 17:53:11.877020 ip-10-0-129-128 kubenswrapper[2559]: W0712 17:53:11.541900 2559 feature_gate.go:328] unrecognized feature gate: NoRegistryClusterInstall Jul 12 17:53:11.877020 ip-10-0-129-128 kubenswrapper[2559]: I0712 17:53:11.541984 2559 flags.go:64] FLAG: --address="0.0.0.0" Jul 12 17:53:11.877020 ip-10-0-129-128 kubenswrapper[2559]: I0712 17:53:11.541993 2559 flags.go:64] FLAG: --allowed-unsafe-sysctls="[]" Jul 12 17:53:11.877020 ip-10-0-129-128 kubenswrapper[2559]: I0712 17:53:11.541999 2559 flags.go:64] FLAG: --anonymous-auth="true" Jul 12 17:53:11.877020 ip-10-0-129-128 kubenswrapper[2559]: I0712 17:53:11.542007 2559 flags.go:64] FLAG: --application-metrics-count-limit="100" Jul 12 17:53:11.877020 ip-10-0-129-128 kubenswrapper[2559]: I0712 17:53:11.542012 2559 flags.go:64] FLAG: --authentication-token-webhook="false" Jul 12 17:53:11.877020 ip-10-0-129-128 kubenswrapper[2559]: I0712 17:53:11.542015 2559 flags.go:64] FLAG: --authentication-token-webhook-cache-ttl="2m0s" Jul 12 17:53:11.878775 ip-10-0-129-128 kubenswrapper[2559]: I0712 17:53:11.542019 2559 flags.go:64] FLAG: --authorization-mode="AlwaysAllow" Jul 12 17:53:11.878775 ip-10-0-129-128 kubenswrapper[2559]: I0712 17:53:11.542026 2559 flags.go:64] FLAG: --authorization-webhook-cache-authorized-ttl="5m0s" Jul 12 17:53:11.878775 ip-10-0-129-128 kubenswrapper[2559]: I0712 17:53:11.542030 2559 flags.go:64] FLAG: --authorization-webhook-cache-unauthorized-ttl="30s" Jul 12 17:53:11.878775 ip-10-0-129-128 kubenswrapper[2559]: I0712 17:53:11.542033 2559 flags.go:64] FLAG: --boot-id-file="/proc/sys/kernel/random/boot_id" Jul 12 17:53:11.878775 ip-10-0-129-128 kubenswrapper[2559]: I0712 17:53:11.542040 2559 flags.go:64] FLAG: --bootstrap-kubeconfig="/etc/kubernetes/kubeconfig" Jul 12 17:53:11.878775 ip-10-0-129-128 kubenswrapper[2559]: I0712 17:53:11.542043 2559 flags.go:64] FLAG: --cert-dir="/var/lib/kubelet/pki" Jul 12 17:53:11.878775 ip-10-0-129-128 kubenswrapper[2559]: I0712 17:53:11.542048 2559 flags.go:64] FLAG: --cgroup-driver="cgroupfs" Jul 12 17:53:11.878775 ip-10-0-129-128 kubenswrapper[2559]: I0712 17:53:11.542051 2559 flags.go:64] FLAG: --cgroup-root="" Jul 12 17:53:11.878775 ip-10-0-129-128 kubenswrapper[2559]: I0712 17:53:11.542054 2559 flags.go:64] FLAG: --cgroups-per-qos="true" Jul 12 17:53:11.878775 ip-10-0-129-128 kubenswrapper[2559]: I0712 17:53:11.542057 2559 flags.go:64] FLAG: --client-ca-file="" Jul 12 17:53:11.878775 ip-10-0-129-128 kubenswrapper[2559]: I0712 17:53:11.542062 2559 flags.go:64] FLAG: --cloud-provider="external" Jul 12 17:53:11.878775 ip-10-0-129-128 kubenswrapper[2559]: I0712 17:53:11.542065 2559 flags.go:64] FLAG: --cluster-dns="[]" Jul 12 17:53:11.878775 ip-10-0-129-128 kubenswrapper[2559]: I0712 17:53:11.542072 2559 flags.go:64] FLAG: --cluster-domain="" Jul 12 17:53:11.878775 ip-10-0-129-128 kubenswrapper[2559]: I0712 17:53:11.542075 2559 flags.go:64] FLAG: --config="/etc/kubernetes/kubelet.conf" Jul 12 17:53:11.878775 ip-10-0-129-128 kubenswrapper[2559]: I0712 17:53:11.542078 2559 flags.go:64] FLAG: --config-dir="/etc/openshift/kubelet.conf.d" Jul 12 17:53:11.878775 ip-10-0-129-128 kubenswrapper[2559]: I0712 17:53:11.542081 2559 flags.go:64] FLAG: --container-hints="/etc/cadvisor/container_hints.json" Jul 12 17:53:11.878775 ip-10-0-129-128 kubenswrapper[2559]: I0712 17:53:11.542087 2559 flags.go:64] FLAG: --container-log-max-files="5" Jul 12 17:53:11.878775 ip-10-0-129-128 kubenswrapper[2559]: I0712 17:53:11.542090 2559 flags.go:64] FLAG: --container-log-max-size="10Mi" Jul 12 17:53:11.878775 ip-10-0-129-128 kubenswrapper[2559]: I0712 17:53:11.542095 2559 flags.go:64] FLAG: --container-runtime-endpoint="/var/run/crio/crio.sock" Jul 12 17:53:11.878775 ip-10-0-129-128 kubenswrapper[2559]: I0712 17:53:11.542099 2559 flags.go:64] FLAG: --containerd="/run/containerd/containerd.sock" Jul 12 17:53:11.878775 ip-10-0-129-128 kubenswrapper[2559]: I0712 17:53:11.542103 2559 flags.go:64] FLAG: --containerd-namespace="k8s.io" Jul 12 17:53:11.878775 ip-10-0-129-128 kubenswrapper[2559]: I0712 17:53:11.542108 2559 flags.go:64] FLAG: --contention-profiling="false" Jul 12 17:53:11.878775 ip-10-0-129-128 kubenswrapper[2559]: I0712 17:53:11.542111 2559 flags.go:64] FLAG: --cpu-cfs-quota="true" Jul 12 17:53:11.878775 ip-10-0-129-128 kubenswrapper[2559]: I0712 17:53:11.542115 2559 flags.go:64] FLAG: --cpu-cfs-quota-period="100ms" Jul 12 17:53:11.879482 ip-10-0-129-128 kubenswrapper[2559]: I0712 17:53:11.542120 2559 flags.go:64] FLAG: --cpu-manager-policy="none" Jul 12 17:53:11.879482 ip-10-0-129-128 kubenswrapper[2559]: I0712 17:53:11.542123 2559 flags.go:64] FLAG: --cpu-manager-policy-options="" Jul 12 17:53:11.879482 ip-10-0-129-128 kubenswrapper[2559]: I0712 17:53:11.542127 2559 flags.go:64] FLAG: --cpu-manager-reconcile-period="10s" Jul 12 17:53:11.879482 ip-10-0-129-128 kubenswrapper[2559]: I0712 17:53:11.542132 2559 flags.go:64] FLAG: --enable-controller-attach-detach="true" Jul 12 17:53:11.879482 ip-10-0-129-128 kubenswrapper[2559]: I0712 17:53:11.542135 2559 flags.go:64] FLAG: --enable-debugging-handlers="true" Jul 12 17:53:11.879482 ip-10-0-129-128 kubenswrapper[2559]: I0712 17:53:11.542138 2559 flags.go:64] FLAG: --enable-load-reader="false" Jul 12 17:53:11.879482 ip-10-0-129-128 kubenswrapper[2559]: I0712 17:53:11.542144 2559 flags.go:64] FLAG: --enable-server="true" Jul 12 17:53:11.879482 ip-10-0-129-128 kubenswrapper[2559]: I0712 17:53:11.542163 2559 flags.go:64] FLAG: --enforce-node-allocatable="[pods]" Jul 12 17:53:11.879482 ip-10-0-129-128 kubenswrapper[2559]: I0712 17:53:11.542170 2559 flags.go:64] FLAG: --event-burst="100" Jul 12 17:53:11.879482 ip-10-0-129-128 kubenswrapper[2559]: I0712 17:53:11.542173 2559 flags.go:64] FLAG: --event-qps="50" Jul 12 17:53:11.879482 ip-10-0-129-128 kubenswrapper[2559]: I0712 17:53:11.542179 2559 flags.go:64] FLAG: --event-storage-age-limit="default=0" Jul 12 17:53:11.879482 ip-10-0-129-128 kubenswrapper[2559]: I0712 17:53:11.542182 2559 flags.go:64] FLAG: --event-storage-event-limit="default=0" Jul 12 17:53:11.879482 ip-10-0-129-128 kubenswrapper[2559]: I0712 17:53:11.542186 2559 flags.go:64] FLAG: --eviction-hard="" Jul 12 17:53:11.879482 ip-10-0-129-128 kubenswrapper[2559]: I0712 17:53:11.542196 2559 flags.go:64] FLAG: --eviction-max-pod-grace-period="0" Jul 12 17:53:11.879482 ip-10-0-129-128 kubenswrapper[2559]: I0712 17:53:11.542199 2559 flags.go:64] FLAG: --eviction-minimum-reclaim="" Jul 12 17:53:11.879482 ip-10-0-129-128 kubenswrapper[2559]: I0712 17:53:11.542204 2559 flags.go:64] FLAG: --eviction-pressure-transition-period="5m0s" Jul 12 17:53:11.879482 ip-10-0-129-128 kubenswrapper[2559]: I0712 17:53:11.542207 2559 flags.go:64] FLAG: --eviction-soft="" Jul 12 17:53:11.879482 ip-10-0-129-128 kubenswrapper[2559]: I0712 17:53:11.542210 2559 flags.go:64] FLAG: --eviction-soft-grace-period="" Jul 12 17:53:11.879482 ip-10-0-129-128 kubenswrapper[2559]: I0712 17:53:11.542215 2559 flags.go:64] FLAG: --exit-on-lock-contention="false" Jul 12 17:53:11.879482 ip-10-0-129-128 kubenswrapper[2559]: I0712 17:53:11.542218 2559 flags.go:64] FLAG: --experimental-allocatable-ignore-eviction="false" Jul 12 17:53:11.879482 ip-10-0-129-128 kubenswrapper[2559]: I0712 17:53:11.542221 2559 flags.go:64] FLAG: --experimental-mounter-path="" Jul 12 17:53:11.879482 ip-10-0-129-128 kubenswrapper[2559]: I0712 17:53:11.542224 2559 flags.go:64] FLAG: --fail-cgroupv1="false" Jul 12 17:53:11.879482 ip-10-0-129-128 kubenswrapper[2559]: I0712 17:53:11.542226 2559 flags.go:64] FLAG: --fail-swap-on="true" Jul 12 17:53:11.879482 ip-10-0-129-128 kubenswrapper[2559]: I0712 17:53:11.542229 2559 flags.go:64] FLAG: --feature-gates="" Jul 12 17:53:11.879482 ip-10-0-129-128 kubenswrapper[2559]: I0712 17:53:11.542233 2559 flags.go:64] FLAG: --file-check-frequency="20s" Jul 12 17:53:11.880563 ip-10-0-129-128 kubenswrapper[2559]: I0712 17:53:11.542239 2559 flags.go:64] FLAG: --global-housekeeping-interval="1m0s" Jul 12 17:53:11.880563 ip-10-0-129-128 kubenswrapper[2559]: I0712 17:53:11.542243 2559 flags.go:64] FLAG: --hairpin-mode="promiscuous-bridge" Jul 12 17:53:11.880563 ip-10-0-129-128 kubenswrapper[2559]: I0712 17:53:11.542248 2559 flags.go:64] FLAG: --healthz-bind-address="127.0.0.1" Jul 12 17:53:11.880563 ip-10-0-129-128 kubenswrapper[2559]: I0712 17:53:11.542252 2559 flags.go:64] FLAG: --healthz-port="10248" Jul 12 17:53:11.880563 ip-10-0-129-128 kubenswrapper[2559]: I0712 17:53:11.542255 2559 flags.go:64] FLAG: --help="false" Jul 12 17:53:11.880563 ip-10-0-129-128 kubenswrapper[2559]: I0712 17:53:11.542260 2559 flags.go:64] FLAG: --hostname-override="ip-10-0-129-128.ec2.internal" Jul 12 17:53:11.880563 ip-10-0-129-128 kubenswrapper[2559]: I0712 17:53:11.542265 2559 flags.go:64] FLAG: --housekeeping-interval="10s" Jul 12 17:53:11.880563 ip-10-0-129-128 kubenswrapper[2559]: I0712 17:53:11.542268 2559 flags.go:64] FLAG: --http-check-frequency="20s" Jul 12 17:53:11.880563 ip-10-0-129-128 kubenswrapper[2559]: I0712 17:53:11.542274 2559 flags.go:64] FLAG: --image-credential-provider-bin-dir="/usr/libexec/kubelet-image-credential-provider-plugins" Jul 12 17:53:11.880563 ip-10-0-129-128 kubenswrapper[2559]: I0712 17:53:11.542277 2559 flags.go:64] FLAG: --image-credential-provider-config="/etc/kubernetes/credential-providers/ecr-credential-provider.yaml" Jul 12 17:53:11.880563 ip-10-0-129-128 kubenswrapper[2559]: I0712 17:53:11.542280 2559 flags.go:64] FLAG: --image-gc-high-threshold="85" Jul 12 17:53:11.880563 ip-10-0-129-128 kubenswrapper[2559]: I0712 17:53:11.542286 2559 flags.go:64] FLAG: --image-gc-low-threshold="80" Jul 12 17:53:11.880563 ip-10-0-129-128 kubenswrapper[2559]: I0712 17:53:11.542289 2559 flags.go:64] FLAG: --image-service-endpoint="" Jul 12 17:53:11.880563 ip-10-0-129-128 kubenswrapper[2559]: I0712 17:53:11.542292 2559 flags.go:64] FLAG: --kernel-memcg-notification="false" Jul 12 17:53:11.880563 ip-10-0-129-128 kubenswrapper[2559]: I0712 17:53:11.542296 2559 flags.go:64] FLAG: --kube-api-burst="100" Jul 12 17:53:11.880563 ip-10-0-129-128 kubenswrapper[2559]: I0712 17:53:11.542300 2559 flags.go:64] FLAG: --kube-api-content-type="application/vnd.kubernetes.protobuf" Jul 12 17:53:11.880563 ip-10-0-129-128 kubenswrapper[2559]: I0712 17:53:11.542303 2559 flags.go:64] FLAG: --kube-api-qps="50" Jul 12 17:53:11.880563 ip-10-0-129-128 kubenswrapper[2559]: I0712 17:53:11.542306 2559 flags.go:64] FLAG: --kube-reserved="" Jul 12 17:53:11.880563 ip-10-0-129-128 kubenswrapper[2559]: I0712 17:53:11.542313 2559 flags.go:64] FLAG: --kube-reserved-cgroup="" Jul 12 17:53:11.880563 ip-10-0-129-128 kubenswrapper[2559]: I0712 17:53:11.542317 2559 flags.go:64] FLAG: --kubeconfig="/var/lib/kubelet/kubeconfig" Jul 12 17:53:11.880563 ip-10-0-129-128 kubenswrapper[2559]: I0712 17:53:11.542323 2559 flags.go:64] FLAG: --kubelet-cgroups="" Jul 12 17:53:11.880563 ip-10-0-129-128 kubenswrapper[2559]: I0712 17:53:11.542325 2559 flags.go:64] FLAG: --local-storage-capacity-isolation="true" Jul 12 17:53:11.880563 ip-10-0-129-128 kubenswrapper[2559]: I0712 17:53:11.542328 2559 flags.go:64] FLAG: --lock-file="" Jul 12 17:53:11.880563 ip-10-0-129-128 kubenswrapper[2559]: I0712 17:53:11.542333 2559 flags.go:64] FLAG: --log-cadvisor-usage="false" Jul 12 17:53:11.881390 ip-10-0-129-128 kubenswrapper[2559]: I0712 17:53:11.542336 2559 flags.go:64] FLAG: --log-flush-frequency="5s" Jul 12 17:53:11.881390 ip-10-0-129-128 kubenswrapper[2559]: I0712 17:53:11.542339 2559 flags.go:64] FLAG: --log-json-info-buffer-size="0" Jul 12 17:53:11.881390 ip-10-0-129-128 kubenswrapper[2559]: I0712 17:53:11.542346 2559 flags.go:64] FLAG: --log-json-split-stream="false" Jul 12 17:53:11.881390 ip-10-0-129-128 kubenswrapper[2559]: I0712 17:53:11.542349 2559 flags.go:64] FLAG: --log-text-info-buffer-size="0" Jul 12 17:53:11.881390 ip-10-0-129-128 kubenswrapper[2559]: I0712 17:53:11.542352 2559 flags.go:64] FLAG: --log-text-split-stream="false" Jul 12 17:53:11.881390 ip-10-0-129-128 kubenswrapper[2559]: I0712 17:53:11.542357 2559 flags.go:64] FLAG: --logging-format="text" Jul 12 17:53:11.881390 ip-10-0-129-128 kubenswrapper[2559]: I0712 17:53:11.542360 2559 flags.go:64] FLAG: --machine-id-file="/etc/machine-id,/var/lib/dbus/machine-id" Jul 12 17:53:11.881390 ip-10-0-129-128 kubenswrapper[2559]: I0712 17:53:11.542363 2559 flags.go:64] FLAG: --make-iptables-util-chains="true" Jul 12 17:53:11.881390 ip-10-0-129-128 kubenswrapper[2559]: I0712 17:53:11.542366 2559 flags.go:64] FLAG: --manifest-url="" Jul 12 17:53:11.881390 ip-10-0-129-128 kubenswrapper[2559]: I0712 17:53:11.542371 2559 flags.go:64] FLAG: --manifest-url-header="" Jul 12 17:53:11.881390 ip-10-0-129-128 kubenswrapper[2559]: I0712 17:53:11.542375 2559 flags.go:64] FLAG: --max-housekeeping-interval="15s" Jul 12 17:53:11.881390 ip-10-0-129-128 kubenswrapper[2559]: I0712 17:53:11.542378 2559 flags.go:64] FLAG: --max-open-files="1000000" Jul 12 17:53:11.881390 ip-10-0-129-128 kubenswrapper[2559]: I0712 17:53:11.542384 2559 flags.go:64] FLAG: --max-pods="110" Jul 12 17:53:11.881390 ip-10-0-129-128 kubenswrapper[2559]: I0712 17:53:11.542387 2559 flags.go:64] FLAG: --maximum-dead-containers="-1" Jul 12 17:53:11.881390 ip-10-0-129-128 kubenswrapper[2559]: I0712 17:53:11.542393 2559 flags.go:64] FLAG: --maximum-dead-containers-per-container="1" Jul 12 17:53:11.881390 ip-10-0-129-128 kubenswrapper[2559]: I0712 17:53:11.542396 2559 flags.go:64] FLAG: --memory-manager-policy="None" Jul 12 17:53:11.881390 ip-10-0-129-128 kubenswrapper[2559]: I0712 17:53:11.542399 2559 flags.go:64] FLAG: --minimum-container-ttl-duration="6m0s" Jul 12 17:53:11.881390 ip-10-0-129-128 kubenswrapper[2559]: I0712 17:53:11.542405 2559 flags.go:64] FLAG: --minimum-image-ttl-duration="2m0s" Jul 12 17:53:11.881390 ip-10-0-129-128 kubenswrapper[2559]: I0712 17:53:11.542408 2559 flags.go:64] FLAG: --node-ip="0.0.0.0" Jul 12 17:53:11.881390 ip-10-0-129-128 kubenswrapper[2559]: I0712 17:53:11.542411 2559 flags.go:64] FLAG: --node-labels="node-role.kubernetes.io/worker=,node.openshift.io/os_id=rhel" Jul 12 17:53:11.881390 ip-10-0-129-128 kubenswrapper[2559]: I0712 17:53:11.542420 2559 flags.go:64] FLAG: --node-status-max-images="50" Jul 12 17:53:11.881390 ip-10-0-129-128 kubenswrapper[2559]: I0712 17:53:11.542423 2559 flags.go:64] FLAG: --node-status-update-frequency="10s" Jul 12 17:53:11.881390 ip-10-0-129-128 kubenswrapper[2559]: I0712 17:53:11.542428 2559 flags.go:64] FLAG: --oom-score-adj="-999" Jul 12 17:53:11.881390 ip-10-0-129-128 kubenswrapper[2559]: I0712 17:53:11.542431 2559 flags.go:64] FLAG: --pod-cidr="" Jul 12 17:53:11.884545 ip-10-0-129-128 kubenswrapper[2559]: I0712 17:53:11.542434 2559 flags.go:64] FLAG: --pod-infra-container-image="quay.io/openshift-release-dev/ocp-v4.0-art-dev@sha256:b04c0b1f73af4d021dd8f0b51bfeb302c24482c093de5f2c43654ab56c506926" Jul 12 17:53:11.884545 ip-10-0-129-128 kubenswrapper[2559]: I0712 17:53:11.542439 2559 flags.go:64] FLAG: --pod-manifest-path="" Jul 12 17:53:11.884545 ip-10-0-129-128 kubenswrapper[2559]: I0712 17:53:11.542442 2559 flags.go:64] FLAG: --pod-max-pids="-1" Jul 12 17:53:11.884545 ip-10-0-129-128 kubenswrapper[2559]: I0712 17:53:11.542447 2559 flags.go:64] FLAG: --pods-per-core="0" Jul 12 17:53:11.884545 ip-10-0-129-128 kubenswrapper[2559]: I0712 17:53:11.542453 2559 flags.go:64] FLAG: --port="10250" Jul 12 17:53:11.884545 ip-10-0-129-128 kubenswrapper[2559]: I0712 17:53:11.542456 2559 flags.go:64] FLAG: --protect-kernel-defaults="false" Jul 12 17:53:11.884545 ip-10-0-129-128 kubenswrapper[2559]: I0712 17:53:11.542459 2559 flags.go:64] FLAG: --provider-id="aws:///us-east-1a/i-0d3e188f10b10aef4" Jul 12 17:53:11.884545 ip-10-0-129-128 kubenswrapper[2559]: I0712 17:53:11.542464 2559 flags.go:64] FLAG: --qos-reserved="" Jul 12 17:53:11.884545 ip-10-0-129-128 kubenswrapper[2559]: I0712 17:53:11.542468 2559 flags.go:64] FLAG: --read-only-port="10255" Jul 12 17:53:11.884545 ip-10-0-129-128 kubenswrapper[2559]: I0712 17:53:11.542471 2559 flags.go:64] FLAG: --register-node="true" Jul 12 17:53:11.884545 ip-10-0-129-128 kubenswrapper[2559]: I0712 17:53:11.542476 2559 flags.go:64] FLAG: --register-with-taints="" Jul 12 17:53:11.884545 ip-10-0-129-128 kubenswrapper[2559]: I0712 17:53:11.542479 2559 flags.go:64] FLAG: --registry-burst="10" Jul 12 17:53:11.884545 ip-10-0-129-128 kubenswrapper[2559]: I0712 17:53:11.542482 2559 flags.go:64] FLAG: --registry-qps="5" Jul 12 17:53:11.884545 ip-10-0-129-128 kubenswrapper[2559]: I0712 17:53:11.542487 2559 flags.go:64] FLAG: --reserved-cpus="" Jul 12 17:53:11.884545 ip-10-0-129-128 kubenswrapper[2559]: I0712 17:53:11.542490 2559 flags.go:64] FLAG: --reserved-memory="" Jul 12 17:53:11.884545 ip-10-0-129-128 kubenswrapper[2559]: I0712 17:53:11.542494 2559 flags.go:64] FLAG: --resolv-conf="/etc/resolv.conf" Jul 12 17:53:11.884545 ip-10-0-129-128 kubenswrapper[2559]: I0712 17:53:11.542497 2559 flags.go:64] FLAG: --root-dir="/var/lib/kubelet" Jul 12 17:53:11.884545 ip-10-0-129-128 kubenswrapper[2559]: I0712 17:53:11.542502 2559 flags.go:64] FLAG: --rotate-certificates="false" Jul 12 17:53:11.884545 ip-10-0-129-128 kubenswrapper[2559]: I0712 17:53:11.542505 2559 flags.go:64] FLAG: --rotate-server-certificates="false" Jul 12 17:53:11.884545 ip-10-0-129-128 kubenswrapper[2559]: I0712 17:53:11.542507 2559 flags.go:64] FLAG: --runonce="false" Jul 12 17:53:11.884545 ip-10-0-129-128 kubenswrapper[2559]: I0712 17:53:11.542512 2559 flags.go:64] FLAG: --runtime-cgroups="/system.slice/crio.service" Jul 12 17:53:11.884545 ip-10-0-129-128 kubenswrapper[2559]: I0712 17:53:11.542515 2559 flags.go:64] FLAG: --runtime-request-timeout="2m0s" Jul 12 17:53:11.884545 ip-10-0-129-128 kubenswrapper[2559]: I0712 17:53:11.542518 2559 flags.go:64] FLAG: --seccomp-default="false" Jul 12 17:53:11.884545 ip-10-0-129-128 kubenswrapper[2559]: I0712 17:53:11.542524 2559 flags.go:64] FLAG: --serialize-image-pulls="true" Jul 12 17:53:11.884545 ip-10-0-129-128 kubenswrapper[2559]: I0712 17:53:11.542527 2559 flags.go:64] FLAG: --storage-driver-buffer-duration="1m0s" Jul 12 17:53:11.885389 ip-10-0-129-128 kubenswrapper[2559]: I0712 17:53:11.542531 2559 flags.go:64] FLAG: --storage-driver-db="cadvisor" Jul 12 17:53:11.885389 ip-10-0-129-128 kubenswrapper[2559]: I0712 17:53:11.542533 2559 flags.go:64] FLAG: --storage-driver-host="localhost:8086" Jul 12 17:53:11.885389 ip-10-0-129-128 kubenswrapper[2559]: I0712 17:53:11.542536 2559 flags.go:64] FLAG: --storage-driver-password="root" Jul 12 17:53:11.885389 ip-10-0-129-128 kubenswrapper[2559]: I0712 17:53:11.542539 2559 flags.go:64] FLAG: --storage-driver-secure="false" Jul 12 17:53:11.885389 ip-10-0-129-128 kubenswrapper[2559]: I0712 17:53:11.542542 2559 flags.go:64] FLAG: --storage-driver-table="stats" Jul 12 17:53:11.885389 ip-10-0-129-128 kubenswrapper[2559]: I0712 17:53:11.542545 2559 flags.go:64] FLAG: --storage-driver-user="root" Jul 12 17:53:11.885389 ip-10-0-129-128 kubenswrapper[2559]: I0712 17:53:11.542548 2559 flags.go:64] FLAG: --streaming-connection-idle-timeout="4h0m0s" Jul 12 17:53:11.885389 ip-10-0-129-128 kubenswrapper[2559]: I0712 17:53:11.542551 2559 flags.go:64] FLAG: --sync-frequency="1m0s" Jul 12 17:53:11.885389 ip-10-0-129-128 kubenswrapper[2559]: I0712 17:53:11.542554 2559 flags.go:64] FLAG: --system-cgroups="" Jul 12 17:53:11.885389 ip-10-0-129-128 kubenswrapper[2559]: I0712 17:53:11.542557 2559 flags.go:64] FLAG: --system-reserved="cpu=500m,ephemeral-storage=1Gi,memory=1Gi" Jul 12 17:53:11.885389 ip-10-0-129-128 kubenswrapper[2559]: I0712 17:53:11.542564 2559 flags.go:64] FLAG: --system-reserved-cgroup="" Jul 12 17:53:11.885389 ip-10-0-129-128 kubenswrapper[2559]: I0712 17:53:11.542567 2559 flags.go:64] FLAG: --tls-cert-file="" Jul 12 17:53:11.885389 ip-10-0-129-128 kubenswrapper[2559]: I0712 17:53:11.542570 2559 flags.go:64] FLAG: --tls-cipher-suites="[]" Jul 12 17:53:11.885389 ip-10-0-129-128 kubenswrapper[2559]: I0712 17:53:11.542574 2559 flags.go:64] FLAG: --tls-min-version="" Jul 12 17:53:11.885389 ip-10-0-129-128 kubenswrapper[2559]: I0712 17:53:11.542577 2559 flags.go:64] FLAG: --tls-private-key-file="" Jul 12 17:53:11.885389 ip-10-0-129-128 kubenswrapper[2559]: I0712 17:53:11.542580 2559 flags.go:64] FLAG: --topology-manager-policy="none" Jul 12 17:53:11.885389 ip-10-0-129-128 kubenswrapper[2559]: I0712 17:53:11.542583 2559 flags.go:64] FLAG: --topology-manager-policy-options="" Jul 12 17:53:11.885389 ip-10-0-129-128 kubenswrapper[2559]: I0712 17:53:11.542586 2559 flags.go:64] FLAG: --topology-manager-scope="container" Jul 12 17:53:11.885389 ip-10-0-129-128 kubenswrapper[2559]: I0712 17:53:11.542589 2559 flags.go:64] FLAG: --v="2" Jul 12 17:53:11.885389 ip-10-0-129-128 kubenswrapper[2559]: I0712 17:53:11.542592 2559 flags.go:64] FLAG: --version="false" Jul 12 17:53:11.885389 ip-10-0-129-128 kubenswrapper[2559]: I0712 17:53:11.542596 2559 flags.go:64] FLAG: --vmodule="" Jul 12 17:53:11.885389 ip-10-0-129-128 kubenswrapper[2559]: I0712 17:53:11.542600 2559 flags.go:64] FLAG: --volume-plugin-dir="/etc/kubernetes/kubelet-plugins/volume/exec" Jul 12 17:53:11.885389 ip-10-0-129-128 kubenswrapper[2559]: I0712 17:53:11.542604 2559 flags.go:64] FLAG: --volume-stats-agg-period="1m0s" Jul 12 17:53:11.885389 ip-10-0-129-128 kubenswrapper[2559]: W0712 17:53:11.542726 2559 feature_gate.go:328] unrecognized feature gate: MachineAPIOperatorDisableMachineHealthCheckController Jul 12 17:53:11.893944 ip-10-0-129-128 kubenswrapper[2559]: W0712 17:53:11.542734 2559 feature_gate.go:328] unrecognized feature gate: NewOLMCatalogdAPIV1Metas Jul 12 17:53:11.893944 ip-10-0-129-128 kubenswrapper[2559]: W0712 17:53:11.542737 2559 feature_gate.go:328] unrecognized feature gate: OpenShiftPodSecurityAdmission Jul 12 17:53:11.893944 ip-10-0-129-128 kubenswrapper[2559]: W0712 17:53:11.542740 2559 feature_gate.go:328] unrecognized feature gate: MultiDiskSetup Jul 12 17:53:11.893944 ip-10-0-129-128 kubenswrapper[2559]: W0712 17:53:11.542745 2559 feature_gate.go:328] unrecognized feature gate: ManagedBootImages Jul 12 17:53:11.893944 ip-10-0-129-128 kubenswrapper[2559]: W0712 17:53:11.542748 2559 feature_gate.go:328] unrecognized feature gate: AWSServiceLBNetworkSecurityGroup Jul 12 17:53:11.893944 ip-10-0-129-128 kubenswrapper[2559]: W0712 17:53:11.542750 2559 feature_gate.go:328] unrecognized feature gate: ExternalSnapshotMetadata Jul 12 17:53:11.893944 ip-10-0-129-128 kubenswrapper[2559]: W0712 17:53:11.542757 2559 feature_gate.go:328] unrecognized feature gate: NetworkSegmentation Jul 12 17:53:11.893944 ip-10-0-129-128 kubenswrapper[2559]: W0712 17:53:11.542759 2559 feature_gate.go:328] unrecognized feature gate: HighlyAvailableArbiter Jul 12 17:53:11.893944 ip-10-0-129-128 kubenswrapper[2559]: W0712 17:53:11.542762 2559 feature_gate.go:328] unrecognized feature gate: KMSEncryption Jul 12 17:53:11.893944 ip-10-0-129-128 kubenswrapper[2559]: W0712 17:53:11.542765 2559 feature_gate.go:328] unrecognized feature gate: NewOLMWebhookProviderOpenshiftServiceCA Jul 12 17:53:11.893944 ip-10-0-129-128 kubenswrapper[2559]: W0712 17:53:11.542771 2559 feature_gate.go:328] unrecognized feature gate: RouteAdvertisements Jul 12 17:53:11.893944 ip-10-0-129-128 kubenswrapper[2559]: W0712 17:53:11.542774 2559 feature_gate.go:328] unrecognized feature gate: HyperShiftOnlyDynamicResourceAllocation Jul 12 17:53:11.893944 ip-10-0-129-128 kubenswrapper[2559]: W0712 17:53:11.542778 2559 feature_gate.go:351] Setting GA feature gate ServiceAccountTokenNodeBinding=true. It will be removed in a future release. Jul 12 17:53:11.893944 ip-10-0-129-128 kubenswrapper[2559]: W0712 17:53:11.542784 2559 feature_gate.go:328] unrecognized feature gate: BuildCSIVolumes Jul 12 17:53:11.893944 ip-10-0-129-128 kubenswrapper[2559]: W0712 17:53:11.542787 2559 feature_gate.go:328] unrecognized feature gate: GatewayAPIWithoutOLM Jul 12 17:53:11.893944 ip-10-0-129-128 kubenswrapper[2559]: W0712 17:53:11.542790 2559 feature_gate.go:328] unrecognized feature gate: GCPCustomAPIEndpointsInstall Jul 12 17:53:11.893944 ip-10-0-129-128 kubenswrapper[2559]: W0712 17:53:11.542795 2559 feature_gate.go:328] unrecognized feature gate: AlibabaPlatform Jul 12 17:53:11.893944 ip-10-0-129-128 kubenswrapper[2559]: W0712 17:53:11.542798 2559 feature_gate.go:328] unrecognized feature gate: ClusterAPIMachineManagement Jul 12 17:53:11.893944 ip-10-0-129-128 kubenswrapper[2559]: W0712 17:53:11.542803 2559 feature_gate.go:328] unrecognized feature gate: BootcNodeManagement Jul 12 17:53:11.895699 ip-10-0-129-128 kubenswrapper[2559]: W0712 17:53:11.542808 2559 feature_gate.go:328] unrecognized feature gate: MetricsCollectionProfiles Jul 12 17:53:11.895699 ip-10-0-129-128 kubenswrapper[2559]: W0712 17:53:11.542811 2559 feature_gate.go:328] unrecognized feature gate: DualReplica Jul 12 17:53:11.895699 ip-10-0-129-128 kubenswrapper[2559]: W0712 17:53:11.542814 2559 feature_gate.go:328] unrecognized feature gate: MultiArchInstallAzure Jul 12 17:53:11.895699 ip-10-0-129-128 kubenswrapper[2559]: W0712 17:53:11.542817 2559 feature_gate.go:328] unrecognized feature gate: AzureDedicatedHosts Jul 12 17:53:11.895699 ip-10-0-129-128 kubenswrapper[2559]: W0712 17:53:11.542822 2559 feature_gate.go:328] unrecognized feature gate: ExternalOIDCWithUIDAndExtraClaimMappings Jul 12 17:53:11.895699 ip-10-0-129-128 kubenswrapper[2559]: W0712 17:53:11.542825 2559 feature_gate.go:328] unrecognized feature gate: VSphereMultiNetworks Jul 12 17:53:11.895699 ip-10-0-129-128 kubenswrapper[2559]: W0712 17:53:11.542828 2559 feature_gate.go:328] unrecognized feature gate: InsightsConfig Jul 12 17:53:11.895699 ip-10-0-129-128 kubenswrapper[2559]: W0712 17:53:11.542833 2559 feature_gate.go:328] unrecognized feature gate: PinnedImages Jul 12 17:53:11.895699 ip-10-0-129-128 kubenswrapper[2559]: W0712 17:53:11.542835 2559 feature_gate.go:328] unrecognized feature gate: SignatureStores Jul 12 17:53:11.895699 ip-10-0-129-128 kubenswrapper[2559]: W0712 17:53:11.542838 2559 feature_gate.go:328] unrecognized feature gate: NewOLMBoxCutterRuntime Jul 12 17:53:11.895699 ip-10-0-129-128 kubenswrapper[2559]: W0712 17:53:11.542840 2559 feature_gate.go:328] unrecognized feature gate: NutanixMultiSubnets Jul 12 17:53:11.895699 ip-10-0-129-128 kubenswrapper[2559]: W0712 17:53:11.542845 2559 feature_gate.go:328] unrecognized feature gate: ExternalOIDC Jul 12 17:53:11.895699 ip-10-0-129-128 kubenswrapper[2559]: W0712 17:53:11.542848 2559 feature_gate.go:328] unrecognized feature gate: Example2 Jul 12 17:53:11.895699 ip-10-0-129-128 kubenswrapper[2559]: W0712 17:53:11.542851 2559 feature_gate.go:328] unrecognized feature gate: ClusterMonitoringConfig Jul 12 17:53:11.895699 ip-10-0-129-128 kubenswrapper[2559]: W0712 17:53:11.542853 2559 feature_gate.go:328] unrecognized feature gate: VSphereHostVMGroupZonal Jul 12 17:53:11.895699 ip-10-0-129-128 kubenswrapper[2559]: W0712 17:53:11.542858 2559 feature_gate.go:328] unrecognized feature gate: DNSNameResolver Jul 12 17:53:11.895699 ip-10-0-129-128 kubenswrapper[2559]: W0712 17:53:11.542860 2559 feature_gate.go:328] unrecognized feature gate: EtcdBackendQuota Jul 12 17:53:11.895699 ip-10-0-129-128 kubenswrapper[2559]: W0712 17:53:11.542863 2559 feature_gate.go:328] unrecognized feature gate: NewOLM Jul 12 17:53:11.895699 ip-10-0-129-128 kubenswrapper[2559]: W0712 17:53:11.542867 2559 feature_gate.go:349] Setting deprecated feature gate KMSv1=true. It will be removed in a future release. Jul 12 17:53:11.895699 ip-10-0-129-128 kubenswrapper[2559]: W0712 17:53:11.542875 2559 feature_gate.go:328] unrecognized feature gate: InsightsOnDemandDataGather Jul 12 17:53:11.898176 ip-10-0-129-128 kubenswrapper[2559]: W0712 17:53:11.542878 2559 feature_gate.go:328] unrecognized feature gate: Example Jul 12 17:53:11.898176 ip-10-0-129-128 kubenswrapper[2559]: W0712 17:53:11.542882 2559 feature_gate.go:328] unrecognized feature gate: MachineAPIMigration Jul 12 17:53:11.898176 ip-10-0-129-128 kubenswrapper[2559]: W0712 17:53:11.542885 2559 feature_gate.go:328] unrecognized feature gate: IrreconcilableMachineConfig Jul 12 17:53:11.898176 ip-10-0-129-128 kubenswrapper[2559]: W0712 17:53:11.542888 2559 feature_gate.go:328] unrecognized feature gate: IngressControllerDynamicConfigurationManager Jul 12 17:53:11.898176 ip-10-0-129-128 kubenswrapper[2559]: W0712 17:53:11.542890 2559 feature_gate.go:328] unrecognized feature gate: VSphereMultiDisk Jul 12 17:53:11.898176 ip-10-0-129-128 kubenswrapper[2559]: W0712 17:53:11.542895 2559 feature_gate.go:328] unrecognized feature gate: MachineConfigNodes Jul 12 17:53:11.898176 ip-10-0-129-128 kubenswrapper[2559]: W0712 17:53:11.542898 2559 feature_gate.go:328] unrecognized feature gate: KMSEncryptionProvider Jul 12 17:53:11.898176 ip-10-0-129-128 kubenswrapper[2559]: W0712 17:53:11.542900 2559 feature_gate.go:328] unrecognized feature gate: AWSClusterHostedDNS Jul 12 17:53:11.898176 ip-10-0-129-128 kubenswrapper[2559]: W0712 17:53:11.542903 2559 feature_gate.go:328] unrecognized feature gate: ProvisioningRequestAvailable Jul 12 17:53:11.898176 ip-10-0-129-128 kubenswrapper[2559]: W0712 17:53:11.542907 2559 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesAWS Jul 12 17:53:11.898176 ip-10-0-129-128 kubenswrapper[2559]: W0712 17:53:11.542910 2559 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesAzure Jul 12 17:53:11.898176 ip-10-0-129-128 kubenswrapper[2559]: W0712 17:53:11.542915 2559 feature_gate.go:328] unrecognized feature gate: ClusterVersionOperatorConfiguration Jul 12 17:53:11.898176 ip-10-0-129-128 kubenswrapper[2559]: W0712 17:53:11.542919 2559 feature_gate.go:328] unrecognized feature gate: AdminNetworkPolicy Jul 12 17:53:11.898176 ip-10-0-129-128 kubenswrapper[2559]: W0712 17:53:11.542922 2559 feature_gate.go:328] unrecognized feature gate: GatewayAPIController Jul 12 17:53:11.898176 ip-10-0-129-128 kubenswrapper[2559]: W0712 17:53:11.542925 2559 feature_gate.go:328] unrecognized feature gate: AzureMultiDisk Jul 12 17:53:11.898176 ip-10-0-129-128 kubenswrapper[2559]: W0712 17:53:11.542929 2559 feature_gate.go:328] unrecognized feature gate: ShortCertRotation Jul 12 17:53:11.898176 ip-10-0-129-128 kubenswrapper[2559]: W0712 17:53:11.542931 2559 feature_gate.go:328] unrecognized feature gate: NetworkDiagnosticsConfig Jul 12 17:53:11.898176 ip-10-0-129-128 kubenswrapper[2559]: W0712 17:53:11.542934 2559 feature_gate.go:328] unrecognized feature gate: ImageModeStatusReporting Jul 12 17:53:11.898176 ip-10-0-129-128 kubenswrapper[2559]: W0712 17:53:11.542936 2559 feature_gate.go:328] unrecognized feature gate: GCPCustomAPIEndpoints Jul 12 17:53:11.898176 ip-10-0-129-128 kubenswrapper[2559]: W0712 17:53:11.542939 2559 feature_gate.go:328] unrecognized feature gate: GatewayAPI Jul 12 17:53:11.899569 ip-10-0-129-128 kubenswrapper[2559]: W0712 17:53:11.542941 2559 feature_gate.go:328] unrecognized feature gate: EventTTL Jul 12 17:53:11.899569 ip-10-0-129-128 kubenswrapper[2559]: W0712 17:53:11.542944 2559 feature_gate.go:328] unrecognized feature gate: AdditionalRoutingCapabilities Jul 12 17:53:11.899569 ip-10-0-129-128 kubenswrapper[2559]: W0712 17:53:11.542946 2559 feature_gate.go:328] unrecognized feature gate: NewOLMOwnSingleNamespace Jul 12 17:53:11.899569 ip-10-0-129-128 kubenswrapper[2559]: W0712 17:53:11.542949 2559 feature_gate.go:328] unrecognized feature gate: NewOLMPreflightPermissionChecks Jul 12 17:53:11.899569 ip-10-0-129-128 kubenswrapper[2559]: W0712 17:53:11.542955 2559 feature_gate.go:328] unrecognized feature gate: UpgradeStatus Jul 12 17:53:11.899569 ip-10-0-129-128 kubenswrapper[2559]: W0712 17:53:11.542957 2559 feature_gate.go:328] unrecognized feature gate: CPMSMachineNamePrefix Jul 12 17:53:11.899569 ip-10-0-129-128 kubenswrapper[2559]: W0712 17:53:11.542959 2559 feature_gate.go:328] unrecognized feature gate: NetworkLiveMigration Jul 12 17:53:11.899569 ip-10-0-129-128 kubenswrapper[2559]: W0712 17:53:11.542962 2559 feature_gate.go:328] unrecognized feature gate: OnPremDNSRecords Jul 12 17:53:11.899569 ip-10-0-129-128 kubenswrapper[2559]: W0712 17:53:11.542967 2559 feature_gate.go:328] unrecognized feature gate: ClusterAPIInstall Jul 12 17:53:11.899569 ip-10-0-129-128 kubenswrapper[2559]: W0712 17:53:11.542969 2559 feature_gate.go:328] unrecognized feature gate: PreconfiguredUDNAddresses Jul 12 17:53:11.899569 ip-10-0-129-128 kubenswrapper[2559]: W0712 17:53:11.542972 2559 feature_gate.go:328] unrecognized feature gate: ImageStreamImportMode Jul 12 17:53:11.899569 ip-10-0-129-128 kubenswrapper[2559]: W0712 17:53:11.542978 2559 feature_gate.go:328] unrecognized feature gate: DyanmicServiceEndpointIBMCloud Jul 12 17:53:11.899569 ip-10-0-129-128 kubenswrapper[2559]: W0712 17:53:11.542981 2559 feature_gate.go:328] unrecognized feature gate: AWSClusterHostedDNSInstall Jul 12 17:53:11.899569 ip-10-0-129-128 kubenswrapper[2559]: W0712 17:53:11.542983 2559 feature_gate.go:328] unrecognized feature gate: BootImageSkewEnforcement Jul 12 17:53:11.899569 ip-10-0-129-128 kubenswrapper[2559]: W0712 17:53:11.542986 2559 feature_gate.go:328] unrecognized feature gate: GCPClusterHostedDNS Jul 12 17:53:11.899569 ip-10-0-129-128 kubenswrapper[2559]: W0712 17:53:11.542991 2559 feature_gate.go:328] unrecognized feature gate: SigstoreImageVerificationPKI Jul 12 17:53:11.899569 ip-10-0-129-128 kubenswrapper[2559]: W0712 17:53:11.542994 2559 feature_gate.go:328] unrecognized feature gate: VSphereMixedNodeEnv Jul 12 17:53:11.899569 ip-10-0-129-128 kubenswrapper[2559]: W0712 17:53:11.542996 2559 feature_gate.go:328] unrecognized feature gate: AzureClusterHostedDNSInstall Jul 12 17:53:11.899569 ip-10-0-129-128 kubenswrapper[2559]: W0712 17:53:11.543001 2559 feature_gate.go:328] unrecognized feature gate: MixedCPUsAllocation Jul 12 17:53:11.899569 ip-10-0-129-128 kubenswrapper[2559]: W0712 17:53:11.543004 2559 feature_gate.go:328] unrecognized feature gate: AWSDedicatedHosts Jul 12 17:53:11.900703 ip-10-0-129-128 kubenswrapper[2559]: W0712 17:53:11.543006 2559 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesvSphere Jul 12 17:53:11.900703 ip-10-0-129-128 kubenswrapper[2559]: W0712 17:53:11.543009 2559 feature_gate.go:328] unrecognized feature gate: OSStreams Jul 12 17:53:11.900703 ip-10-0-129-128 kubenswrapper[2559]: W0712 17:53:11.543014 2559 feature_gate.go:328] unrecognized feature gate: ConsolePluginContentSecurityPolicy Jul 12 17:53:11.900703 ip-10-0-129-128 kubenswrapper[2559]: W0712 17:53:11.543018 2559 feature_gate.go:328] unrecognized feature gate: ClusterAPIInstallIBMCloud Jul 12 17:53:11.900703 ip-10-0-129-128 kubenswrapper[2559]: W0712 17:53:11.543021 2559 feature_gate.go:328] unrecognized feature gate: AzureDualStackInstall Jul 12 17:53:11.900703 ip-10-0-129-128 kubenswrapper[2559]: W0712 17:53:11.543026 2559 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesCPMS Jul 12 17:53:11.900703 ip-10-0-129-128 kubenswrapper[2559]: W0712 17:53:11.543028 2559 feature_gate.go:328] unrecognized feature gate: AzureWorkloadIdentity Jul 12 17:53:11.900703 ip-10-0-129-128 kubenswrapper[2559]: W0712 17:53:11.543031 2559 feature_gate.go:328] unrecognized feature gate: CRDCompatibilityRequirementOperator Jul 12 17:53:11.900703 ip-10-0-129-128 kubenswrapper[2559]: W0712 17:53:11.543036 2559 feature_gate.go:328] unrecognized feature gate: OVNObservability Jul 12 17:53:11.900703 ip-10-0-129-128 kubenswrapper[2559]: W0712 17:53:11.543039 2559 feature_gate.go:328] unrecognized feature gate: VSphereConfigurableMaxAllowedBlockVolumesPerNode Jul 12 17:53:11.900703 ip-10-0-129-128 kubenswrapper[2559]: W0712 17:53:11.543041 2559 feature_gate.go:328] unrecognized feature gate: NoRegistryClusterInstall Jul 12 17:53:11.900703 ip-10-0-129-128 kubenswrapper[2559]: W0712 17:53:11.543044 2559 feature_gate.go:328] unrecognized feature gate: GCPDualStackInstall Jul 12 17:53:11.900703 ip-10-0-129-128 kubenswrapper[2559]: W0712 17:53:11.543048 2559 feature_gate.go:328] unrecognized feature gate: GCPClusterHostedDNSInstall Jul 12 17:53:11.900703 ip-10-0-129-128 kubenswrapper[2559]: W0712 17:53:11.543051 2559 feature_gate.go:328] unrecognized feature gate: AutomatedEtcdBackup Jul 12 17:53:11.900703 ip-10-0-129-128 kubenswrapper[2559]: W0712 17:53:11.543053 2559 feature_gate.go:328] unrecognized feature gate: SigstoreImageVerification Jul 12 17:53:11.900703 ip-10-0-129-128 kubenswrapper[2559]: W0712 17:53:11.543057 2559 feature_gate.go:351] Setting GA feature gate VolumeAttributesClass=true. It will be removed in a future release. Jul 12 17:53:11.900703 ip-10-0-129-128 kubenswrapper[2559]: W0712 17:53:11.543062 2559 feature_gate.go:328] unrecognized feature gate: AWSDualStackInstall Jul 12 17:53:11.900703 ip-10-0-129-128 kubenswrapper[2559]: W0712 17:53:11.543065 2559 feature_gate.go:328] unrecognized feature gate: VolumeGroupSnapshot Jul 12 17:53:11.900703 ip-10-0-129-128 kubenswrapper[2559]: W0712 17:53:11.543068 2559 feature_gate.go:328] unrecognized feature gate: ClusterAPIMachineManagementVSphere Jul 12 17:53:11.901538 ip-10-0-129-128 kubenswrapper[2559]: I0712 17:53:11.543822 2559 feature_gate.go:384] feature gates: {map[CBORServingAndStorage:false ClientsAllowCBOR:false ClientsPreferCBOR:false EventedPLEG:false ImageVolume:true KMSv1:true MaxUnavailableStatefulSet:false MinimumKubeletVersion:false MutableCSINodeAllocatableCount:false MutatingAdmissionPolicy:false ProcMountType:true RouteExternalCertificate:true SELinuxMount:false ServiceAccountTokenNodeBinding:true StoragePerformantSecurityPolicy:true TranslateStreamCloseWebsocketRequests:false UserNamespacesPodSecurityStandards:true UserNamespacesSupport:true VolumeAttributesClass:true]} Jul 12 17:53:11.901538 ip-10-0-129-128 kubenswrapper[2559]: I0712 17:53:11.550861 2559 server.go:529] "Kubelet version" kubeletVersion="v1.34.9" Jul 12 17:53:11.901538 ip-10-0-129-128 kubenswrapper[2559]: I0712 17:53:11.550876 2559 server.go:531] "Golang settings" GOGC="" GOMAXPROCS="" GOTRACEBACK="" Jul 12 17:53:11.901538 ip-10-0-129-128 kubenswrapper[2559]: I0712 17:53:11.550889 2559 watchdog_linux.go:95] "Systemd watchdog is not enabled" Jul 12 17:53:11.901538 ip-10-0-129-128 kubenswrapper[2559]: I0712 17:53:11.550895 2559 watchdog_linux.go:137] "Systemd watchdog is not enabled or the interval is invalid, so health checking will not be started." Jul 12 17:53:11.901538 ip-10-0-129-128 kubenswrapper[2559]: W0712 17:53:11.550949 2559 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesvSphere Jul 12 17:53:11.901538 ip-10-0-129-128 kubenswrapper[2559]: W0712 17:53:11.550954 2559 feature_gate.go:328] unrecognized feature gate: VSphereConfigurableMaxAllowedBlockVolumesPerNode Jul 12 17:53:11.901538 ip-10-0-129-128 kubenswrapper[2559]: W0712 17:53:11.550959 2559 feature_gate.go:351] Setting GA feature gate ServiceAccountTokenNodeBinding=true. It will be removed in a future release. Jul 12 17:53:11.901538 ip-10-0-129-128 kubenswrapper[2559]: W0712 17:53:11.550963 2559 feature_gate.go:328] unrecognized feature gate: AWSServiceLBNetworkSecurityGroup Jul 12 17:53:11.901538 ip-10-0-129-128 kubenswrapper[2559]: W0712 17:53:11.550967 2559 feature_gate.go:328] unrecognized feature gate: HyperShiftOnlyDynamicResourceAllocation Jul 12 17:53:11.901538 ip-10-0-129-128 kubenswrapper[2559]: W0712 17:53:11.550970 2559 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesCPMS Jul 12 17:53:11.901538 ip-10-0-129-128 kubenswrapper[2559]: W0712 17:53:11.550973 2559 feature_gate.go:328] unrecognized feature gate: EventTTL Jul 12 17:53:11.901538 ip-10-0-129-128 kubenswrapper[2559]: W0712 17:53:11.550976 2559 feature_gate.go:328] unrecognized feature gate: GCPDualStackInstall Jul 12 17:53:11.901538 ip-10-0-129-128 kubenswrapper[2559]: W0712 17:53:11.550978 2559 feature_gate.go:328] unrecognized feature gate: UpgradeStatus Jul 12 17:53:11.902412 ip-10-0-129-128 kubenswrapper[2559]: W0712 17:53:11.550981 2559 feature_gate.go:328] unrecognized feature gate: GCPCustomAPIEndpointsInstall Jul 12 17:53:11.902412 ip-10-0-129-128 kubenswrapper[2559]: W0712 17:53:11.550984 2559 feature_gate.go:328] unrecognized feature gate: MultiDiskSetup Jul 12 17:53:11.902412 ip-10-0-129-128 kubenswrapper[2559]: W0712 17:53:11.550987 2559 feature_gate.go:328] unrecognized feature gate: SigstoreImageVerificationPKI Jul 12 17:53:11.902412 ip-10-0-129-128 kubenswrapper[2559]: W0712 17:53:11.550990 2559 feature_gate.go:328] unrecognized feature gate: BootcNodeManagement Jul 12 17:53:11.902412 ip-10-0-129-128 kubenswrapper[2559]: W0712 17:53:11.550993 2559 feature_gate.go:328] unrecognized feature gate: ManagedBootImages Jul 12 17:53:11.902412 ip-10-0-129-128 kubenswrapper[2559]: W0712 17:53:11.550996 2559 feature_gate.go:328] unrecognized feature gate: GatewayAPI Jul 12 17:53:11.902412 ip-10-0-129-128 kubenswrapper[2559]: W0712 17:53:11.550998 2559 feature_gate.go:328] unrecognized feature gate: HighlyAvailableArbiter Jul 12 17:53:11.902412 ip-10-0-129-128 kubenswrapper[2559]: W0712 17:53:11.551001 2559 feature_gate.go:328] unrecognized feature gate: OpenShiftPodSecurityAdmission Jul 12 17:53:11.902412 ip-10-0-129-128 kubenswrapper[2559]: W0712 17:53:11.551004 2559 feature_gate.go:328] unrecognized feature gate: IrreconcilableMachineConfig Jul 12 17:53:11.902412 ip-10-0-129-128 kubenswrapper[2559]: W0712 17:53:11.551007 2559 feature_gate.go:328] unrecognized feature gate: VSphereMultiNetworks Jul 12 17:53:11.902412 ip-10-0-129-128 kubenswrapper[2559]: W0712 17:53:11.551009 2559 feature_gate.go:328] unrecognized feature gate: NetworkLiveMigration Jul 12 17:53:11.902412 ip-10-0-129-128 kubenswrapper[2559]: W0712 17:53:11.551012 2559 feature_gate.go:328] unrecognized feature gate: GCPCustomAPIEndpoints Jul 12 17:53:11.902412 ip-10-0-129-128 kubenswrapper[2559]: W0712 17:53:11.551014 2559 feature_gate.go:328] unrecognized feature gate: GCPClusterHostedDNS Jul 12 17:53:11.902412 ip-10-0-129-128 kubenswrapper[2559]: W0712 17:53:11.551017 2559 feature_gate.go:328] unrecognized feature gate: VolumeGroupSnapshot Jul 12 17:53:11.902412 ip-10-0-129-128 kubenswrapper[2559]: W0712 17:53:11.551019 2559 feature_gate.go:328] unrecognized feature gate: OnPremDNSRecords Jul 12 17:53:11.902412 ip-10-0-129-128 kubenswrapper[2559]: W0712 17:53:11.551022 2559 feature_gate.go:328] unrecognized feature gate: ClusterAPIInstallIBMCloud Jul 12 17:53:11.902412 ip-10-0-129-128 kubenswrapper[2559]: W0712 17:53:11.551025 2559 feature_gate.go:328] unrecognized feature gate: MachineConfigNodes Jul 12 17:53:11.902412 ip-10-0-129-128 kubenswrapper[2559]: W0712 17:53:11.551027 2559 feature_gate.go:328] unrecognized feature gate: NewOLMCatalogdAPIV1Metas Jul 12 17:53:11.902412 ip-10-0-129-128 kubenswrapper[2559]: W0712 17:53:11.551030 2559 feature_gate.go:328] unrecognized feature gate: AutomatedEtcdBackup Jul 12 17:53:11.902412 ip-10-0-129-128 kubenswrapper[2559]: W0712 17:53:11.551032 2559 feature_gate.go:328] unrecognized feature gate: GCPClusterHostedDNSInstall Jul 12 17:53:11.903293 ip-10-0-129-128 kubenswrapper[2559]: W0712 17:53:11.551035 2559 feature_gate.go:328] unrecognized feature gate: InsightsOnDemandDataGather Jul 12 17:53:11.903293 ip-10-0-129-128 kubenswrapper[2559]: W0712 17:53:11.551037 2559 feature_gate.go:328] unrecognized feature gate: EtcdBackendQuota Jul 12 17:53:11.903293 ip-10-0-129-128 kubenswrapper[2559]: W0712 17:53:11.551040 2559 feature_gate.go:328] unrecognized feature gate: ClusterAPIMachineManagementVSphere Jul 12 17:53:11.903293 ip-10-0-129-128 kubenswrapper[2559]: W0712 17:53:11.551042 2559 feature_gate.go:328] unrecognized feature gate: AdditionalRoutingCapabilities Jul 12 17:53:11.903293 ip-10-0-129-128 kubenswrapper[2559]: W0712 17:53:11.551044 2559 feature_gate.go:328] unrecognized feature gate: AzureDualStackInstall Jul 12 17:53:11.903293 ip-10-0-129-128 kubenswrapper[2559]: W0712 17:53:11.551047 2559 feature_gate.go:328] unrecognized feature gate: KMSEncryptionProvider Jul 12 17:53:11.903293 ip-10-0-129-128 kubenswrapper[2559]: W0712 17:53:11.551050 2559 feature_gate.go:328] unrecognized feature gate: SignatureStores Jul 12 17:53:11.903293 ip-10-0-129-128 kubenswrapper[2559]: W0712 17:53:11.551053 2559 feature_gate.go:328] unrecognized feature gate: ConsolePluginContentSecurityPolicy Jul 12 17:53:11.903293 ip-10-0-129-128 kubenswrapper[2559]: W0712 17:53:11.551056 2559 feature_gate.go:328] unrecognized feature gate: GatewayAPIController Jul 12 17:53:11.903293 ip-10-0-129-128 kubenswrapper[2559]: W0712 17:53:11.551058 2559 feature_gate.go:328] unrecognized feature gate: MachineAPIMigration Jul 12 17:53:11.903293 ip-10-0-129-128 kubenswrapper[2559]: W0712 17:53:11.551061 2559 feature_gate.go:328] unrecognized feature gate: NetworkDiagnosticsConfig Jul 12 17:53:11.903293 ip-10-0-129-128 kubenswrapper[2559]: W0712 17:53:11.551063 2559 feature_gate.go:328] unrecognized feature gate: AWSClusterHostedDNS Jul 12 17:53:11.903293 ip-10-0-129-128 kubenswrapper[2559]: W0712 17:53:11.551066 2559 feature_gate.go:328] unrecognized feature gate: VSphereMultiDisk Jul 12 17:53:11.903293 ip-10-0-129-128 kubenswrapper[2559]: W0712 17:53:11.551068 2559 feature_gate.go:328] unrecognized feature gate: NewOLMWebhookProviderOpenshiftServiceCA Jul 12 17:53:11.903293 ip-10-0-129-128 kubenswrapper[2559]: W0712 17:53:11.551071 2559 feature_gate.go:328] unrecognized feature gate: MixedCPUsAllocation Jul 12 17:53:11.903293 ip-10-0-129-128 kubenswrapper[2559]: W0712 17:53:11.551073 2559 feature_gate.go:328] unrecognized feature gate: BootImageSkewEnforcement Jul 12 17:53:11.903293 ip-10-0-129-128 kubenswrapper[2559]: W0712 17:53:11.551076 2559 feature_gate.go:328] unrecognized feature gate: AzureWorkloadIdentity Jul 12 17:53:11.903293 ip-10-0-129-128 kubenswrapper[2559]: W0712 17:53:11.551079 2559 feature_gate.go:328] unrecognized feature gate: DNSNameResolver Jul 12 17:53:11.903293 ip-10-0-129-128 kubenswrapper[2559]: W0712 17:53:11.551082 2559 feature_gate.go:328] unrecognized feature gate: NutanixMultiSubnets Jul 12 17:53:11.903293 ip-10-0-129-128 kubenswrapper[2559]: W0712 17:53:11.551084 2559 feature_gate.go:328] unrecognized feature gate: RouteAdvertisements Jul 12 17:53:11.904194 ip-10-0-129-128 kubenswrapper[2559]: W0712 17:53:11.551087 2559 feature_gate.go:328] unrecognized feature gate: NetworkSegmentation Jul 12 17:53:11.904194 ip-10-0-129-128 kubenswrapper[2559]: W0712 17:53:11.551090 2559 feature_gate.go:328] unrecognized feature gate: OVNObservability Jul 12 17:53:11.904194 ip-10-0-129-128 kubenswrapper[2559]: W0712 17:53:11.551093 2559 feature_gate.go:328] unrecognized feature gate: OSStreams Jul 12 17:53:11.904194 ip-10-0-129-128 kubenswrapper[2559]: W0712 17:53:11.551095 2559 feature_gate.go:328] unrecognized feature gate: AWSDualStackInstall Jul 12 17:53:11.904194 ip-10-0-129-128 kubenswrapper[2559]: W0712 17:53:11.551098 2559 feature_gate.go:328] unrecognized feature gate: CRDCompatibilityRequirementOperator Jul 12 17:53:11.904194 ip-10-0-129-128 kubenswrapper[2559]: W0712 17:53:11.551101 2559 feature_gate.go:349] Setting deprecated feature gate KMSv1=true. It will be removed in a future release. Jul 12 17:53:11.904194 ip-10-0-129-128 kubenswrapper[2559]: W0712 17:53:11.551107 2559 feature_gate.go:328] unrecognized feature gate: GatewayAPIWithoutOLM Jul 12 17:53:11.904194 ip-10-0-129-128 kubenswrapper[2559]: W0712 17:53:11.551110 2559 feature_gate.go:328] unrecognized feature gate: Example Jul 12 17:53:11.904194 ip-10-0-129-128 kubenswrapper[2559]: W0712 17:53:11.551114 2559 feature_gate.go:351] Setting GA feature gate VolumeAttributesClass=true. It will be removed in a future release. Jul 12 17:53:11.904194 ip-10-0-129-128 kubenswrapper[2559]: W0712 17:53:11.551117 2559 feature_gate.go:328] unrecognized feature gate: AWSDedicatedHosts Jul 12 17:53:11.904194 ip-10-0-129-128 kubenswrapper[2559]: W0712 17:53:11.551120 2559 feature_gate.go:328] unrecognized feature gate: KMSEncryption Jul 12 17:53:11.904194 ip-10-0-129-128 kubenswrapper[2559]: W0712 17:53:11.551123 2559 feature_gate.go:328] unrecognized feature gate: AWSClusterHostedDNSInstall Jul 12 17:53:11.904194 ip-10-0-129-128 kubenswrapper[2559]: W0712 17:53:11.551126 2559 feature_gate.go:328] unrecognized feature gate: BuildCSIVolumes Jul 12 17:53:11.904194 ip-10-0-129-128 kubenswrapper[2559]: W0712 17:53:11.551129 2559 feature_gate.go:328] unrecognized feature gate: NewOLMPreflightPermissionChecks Jul 12 17:53:11.904194 ip-10-0-129-128 kubenswrapper[2559]: W0712 17:53:11.551131 2559 feature_gate.go:328] unrecognized feature gate: VSphereHostVMGroupZonal Jul 12 17:53:11.904194 ip-10-0-129-128 kubenswrapper[2559]: W0712 17:53:11.551134 2559 feature_gate.go:328] unrecognized feature gate: AdminNetworkPolicy Jul 12 17:53:11.904194 ip-10-0-129-128 kubenswrapper[2559]: W0712 17:53:11.551136 2559 feature_gate.go:328] unrecognized feature gate: AzureMultiDisk Jul 12 17:53:11.904194 ip-10-0-129-128 kubenswrapper[2559]: W0712 17:53:11.551139 2559 feature_gate.go:328] unrecognized feature gate: AzureDedicatedHosts Jul 12 17:53:11.904194 ip-10-0-129-128 kubenswrapper[2559]: W0712 17:53:11.551142 2559 feature_gate.go:328] unrecognized feature gate: NewOLMBoxCutterRuntime Jul 12 17:53:11.904194 ip-10-0-129-128 kubenswrapper[2559]: W0712 17:53:11.551161 2559 feature_gate.go:328] unrecognized feature gate: DualReplica Jul 12 17:53:11.905054 ip-10-0-129-128 kubenswrapper[2559]: W0712 17:53:11.551164 2559 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesAWS Jul 12 17:53:11.905054 ip-10-0-129-128 kubenswrapper[2559]: W0712 17:53:11.551167 2559 feature_gate.go:328] unrecognized feature gate: MultiArchInstallAzure Jul 12 17:53:11.905054 ip-10-0-129-128 kubenswrapper[2559]: W0712 17:53:11.551169 2559 feature_gate.go:328] unrecognized feature gate: Example2 Jul 12 17:53:11.905054 ip-10-0-129-128 kubenswrapper[2559]: W0712 17:53:11.551172 2559 feature_gate.go:328] unrecognized feature gate: ClusterVersionOperatorConfiguration Jul 12 17:53:11.905054 ip-10-0-129-128 kubenswrapper[2559]: W0712 17:53:11.551175 2559 feature_gate.go:328] unrecognized feature gate: AzureClusterHostedDNSInstall Jul 12 17:53:11.905054 ip-10-0-129-128 kubenswrapper[2559]: W0712 17:53:11.551178 2559 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesAzure Jul 12 17:53:11.905054 ip-10-0-129-128 kubenswrapper[2559]: W0712 17:53:11.551181 2559 feature_gate.go:328] unrecognized feature gate: ExternalOIDC Jul 12 17:53:11.905054 ip-10-0-129-128 kubenswrapper[2559]: W0712 17:53:11.551184 2559 feature_gate.go:328] unrecognized feature gate: ExternalOIDCWithUIDAndExtraClaimMappings Jul 12 17:53:11.905054 ip-10-0-129-128 kubenswrapper[2559]: W0712 17:53:11.551186 2559 feature_gate.go:328] unrecognized feature gate: NoRegistryClusterInstall Jul 12 17:53:11.905054 ip-10-0-129-128 kubenswrapper[2559]: W0712 17:53:11.551190 2559 feature_gate.go:328] unrecognized feature gate: ProvisioningRequestAvailable Jul 12 17:53:11.905054 ip-10-0-129-128 kubenswrapper[2559]: W0712 17:53:11.551193 2559 feature_gate.go:328] unrecognized feature gate: InsightsConfig Jul 12 17:53:11.905054 ip-10-0-129-128 kubenswrapper[2559]: W0712 17:53:11.551196 2559 feature_gate.go:328] unrecognized feature gate: NewOLM Jul 12 17:53:11.905054 ip-10-0-129-128 kubenswrapper[2559]: W0712 17:53:11.551198 2559 feature_gate.go:328] unrecognized feature gate: ImageStreamImportMode Jul 12 17:53:11.905054 ip-10-0-129-128 kubenswrapper[2559]: W0712 17:53:11.551201 2559 feature_gate.go:328] unrecognized feature gate: ClusterMonitoringConfig Jul 12 17:53:11.905054 ip-10-0-129-128 kubenswrapper[2559]: W0712 17:53:11.551204 2559 feature_gate.go:328] unrecognized feature gate: ImageModeStatusReporting Jul 12 17:53:11.905054 ip-10-0-129-128 kubenswrapper[2559]: W0712 17:53:11.551206 2559 feature_gate.go:328] unrecognized feature gate: MachineAPIOperatorDisableMachineHealthCheckController Jul 12 17:53:11.905054 ip-10-0-129-128 kubenswrapper[2559]: W0712 17:53:11.551209 2559 feature_gate.go:328] unrecognized feature gate: DyanmicServiceEndpointIBMCloud Jul 12 17:53:11.905054 ip-10-0-129-128 kubenswrapper[2559]: W0712 17:53:11.551211 2559 feature_gate.go:328] unrecognized feature gate: MetricsCollectionProfiles Jul 12 17:53:11.905054 ip-10-0-129-128 kubenswrapper[2559]: W0712 17:53:11.551215 2559 feature_gate.go:328] unrecognized feature gate: ShortCertRotation Jul 12 17:53:11.905941 ip-10-0-129-128 kubenswrapper[2559]: W0712 17:53:11.551218 2559 feature_gate.go:328] unrecognized feature gate: IngressControllerDynamicConfigurationManager Jul 12 17:53:11.905941 ip-10-0-129-128 kubenswrapper[2559]: W0712 17:53:11.551221 2559 feature_gate.go:328] unrecognized feature gate: ClusterAPIMachineManagement Jul 12 17:53:11.905941 ip-10-0-129-128 kubenswrapper[2559]: W0712 17:53:11.551223 2559 feature_gate.go:328] unrecognized feature gate: PinnedImages Jul 12 17:53:11.905941 ip-10-0-129-128 kubenswrapper[2559]: W0712 17:53:11.551226 2559 feature_gate.go:328] unrecognized feature gate: VSphereMixedNodeEnv Jul 12 17:53:11.905941 ip-10-0-129-128 kubenswrapper[2559]: W0712 17:53:11.551228 2559 feature_gate.go:328] unrecognized feature gate: ExternalSnapshotMetadata Jul 12 17:53:11.905941 ip-10-0-129-128 kubenswrapper[2559]: W0712 17:53:11.551231 2559 feature_gate.go:328] unrecognized feature gate: ClusterAPIInstall Jul 12 17:53:11.905941 ip-10-0-129-128 kubenswrapper[2559]: W0712 17:53:11.551233 2559 feature_gate.go:328] unrecognized feature gate: NewOLMOwnSingleNamespace Jul 12 17:53:11.905941 ip-10-0-129-128 kubenswrapper[2559]: W0712 17:53:11.551236 2559 feature_gate.go:328] unrecognized feature gate: AlibabaPlatform Jul 12 17:53:11.905941 ip-10-0-129-128 kubenswrapper[2559]: W0712 17:53:11.551239 2559 feature_gate.go:328] unrecognized feature gate: PreconfiguredUDNAddresses Jul 12 17:53:11.905941 ip-10-0-129-128 kubenswrapper[2559]: W0712 17:53:11.551241 2559 feature_gate.go:328] unrecognized feature gate: SigstoreImageVerification Jul 12 17:53:11.905941 ip-10-0-129-128 kubenswrapper[2559]: W0712 17:53:11.551244 2559 feature_gate.go:328] unrecognized feature gate: CPMSMachineNamePrefix Jul 12 17:53:11.905941 ip-10-0-129-128 kubenswrapper[2559]: I0712 17:53:11.551249 2559 feature_gate.go:384] feature gates: {map[CBORServingAndStorage:false ClientsAllowCBOR:false ClientsPreferCBOR:false EventedPLEG:false ImageVolume:true KMSv1:true MaxUnavailableStatefulSet:false MinimumKubeletVersion:false MutableCSINodeAllocatableCount:false MutatingAdmissionPolicy:false ProcMountType:true RouteExternalCertificate:true SELinuxMount:false ServiceAccountTokenNodeBinding:true StoragePerformantSecurityPolicy:true TranslateStreamCloseWebsocketRequests:false UserNamespacesPodSecurityStandards:true UserNamespacesSupport:true VolumeAttributesClass:true]} Jul 12 17:53:11.905941 ip-10-0-129-128 kubenswrapper[2559]: W0712 17:53:11.551357 2559 feature_gate.go:328] unrecognized feature gate: ConsolePluginContentSecurityPolicy Jul 12 17:53:11.905941 ip-10-0-129-128 kubenswrapper[2559]: W0712 17:53:11.551362 2559 feature_gate.go:328] unrecognized feature gate: HighlyAvailableArbiter Jul 12 17:53:11.905941 ip-10-0-129-128 kubenswrapper[2559]: W0712 17:53:11.551365 2559 feature_gate.go:328] unrecognized feature gate: ShortCertRotation Jul 12 17:53:11.906599 ip-10-0-129-128 kubenswrapper[2559]: W0712 17:53:11.551369 2559 feature_gate.go:351] Setting GA feature gate VolumeAttributesClass=true. It will be removed in a future release. Jul 12 17:53:11.906599 ip-10-0-129-128 kubenswrapper[2559]: W0712 17:53:11.551373 2559 feature_gate.go:328] unrecognized feature gate: AWSServiceLBNetworkSecurityGroup Jul 12 17:53:11.906599 ip-10-0-129-128 kubenswrapper[2559]: W0712 17:53:11.551376 2559 feature_gate.go:328] unrecognized feature gate: OnPremDNSRecords Jul 12 17:53:11.906599 ip-10-0-129-128 kubenswrapper[2559]: W0712 17:53:11.551379 2559 feature_gate.go:328] unrecognized feature gate: BuildCSIVolumes Jul 12 17:53:11.906599 ip-10-0-129-128 kubenswrapper[2559]: W0712 17:53:11.551382 2559 feature_gate.go:328] unrecognized feature gate: MixedCPUsAllocation Jul 12 17:53:11.906599 ip-10-0-129-128 kubenswrapper[2559]: W0712 17:53:11.551385 2559 feature_gate.go:328] unrecognized feature gate: OpenShiftPodSecurityAdmission Jul 12 17:53:11.906599 ip-10-0-129-128 kubenswrapper[2559]: W0712 17:53:11.551387 2559 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesAzure Jul 12 17:53:11.906599 ip-10-0-129-128 kubenswrapper[2559]: W0712 17:53:11.551390 2559 feature_gate.go:328] unrecognized feature gate: AdditionalRoutingCapabilities Jul 12 17:53:11.906599 ip-10-0-129-128 kubenswrapper[2559]: W0712 17:53:11.551394 2559 feature_gate.go:328] unrecognized feature gate: KMSEncryption Jul 12 17:53:11.906599 ip-10-0-129-128 kubenswrapper[2559]: W0712 17:53:11.551397 2559 feature_gate.go:328] unrecognized feature gate: NutanixMultiSubnets Jul 12 17:53:11.906599 ip-10-0-129-128 kubenswrapper[2559]: W0712 17:53:11.551400 2559 feature_gate.go:328] unrecognized feature gate: NewOLMCatalogdAPIV1Metas Jul 12 17:53:11.906599 ip-10-0-129-128 kubenswrapper[2559]: W0712 17:53:11.551403 2559 feature_gate.go:328] unrecognized feature gate: NewOLM Jul 12 17:53:11.906599 ip-10-0-129-128 kubenswrapper[2559]: W0712 17:53:11.551406 2559 feature_gate.go:328] unrecognized feature gate: UpgradeStatus Jul 12 17:53:11.906599 ip-10-0-129-128 kubenswrapper[2559]: W0712 17:53:11.551408 2559 feature_gate.go:328] unrecognized feature gate: Example2 Jul 12 17:53:11.906599 ip-10-0-129-128 kubenswrapper[2559]: W0712 17:53:11.551411 2559 feature_gate.go:328] unrecognized feature gate: DNSNameResolver Jul 12 17:53:11.906599 ip-10-0-129-128 kubenswrapper[2559]: W0712 17:53:11.551413 2559 feature_gate.go:328] unrecognized feature gate: VSphereConfigurableMaxAllowedBlockVolumesPerNode Jul 12 17:53:11.906599 ip-10-0-129-128 kubenswrapper[2559]: W0712 17:53:11.551416 2559 feature_gate.go:328] unrecognized feature gate: CPMSMachineNamePrefix Jul 12 17:53:11.906599 ip-10-0-129-128 kubenswrapper[2559]: W0712 17:53:11.551420 2559 feature_gate.go:328] unrecognized feature gate: ExternalOIDC Jul 12 17:53:11.906599 ip-10-0-129-128 kubenswrapper[2559]: W0712 17:53:11.551422 2559 feature_gate.go:328] unrecognized feature gate: AzureDualStackInstall Jul 12 17:53:11.906599 ip-10-0-129-128 kubenswrapper[2559]: W0712 17:53:11.551425 2559 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesAWS Jul 12 17:53:11.907472 ip-10-0-129-128 kubenswrapper[2559]: W0712 17:53:11.551428 2559 feature_gate.go:328] unrecognized feature gate: CRDCompatibilityRequirementOperator Jul 12 17:53:11.907472 ip-10-0-129-128 kubenswrapper[2559]: W0712 17:53:11.551430 2559 feature_gate.go:328] unrecognized feature gate: NetworkSegmentation Jul 12 17:53:11.907472 ip-10-0-129-128 kubenswrapper[2559]: W0712 17:53:11.551433 2559 feature_gate.go:328] unrecognized feature gate: AWSClusterHostedDNSInstall Jul 12 17:53:11.907472 ip-10-0-129-128 kubenswrapper[2559]: W0712 17:53:11.551435 2559 feature_gate.go:328] unrecognized feature gate: AdminNetworkPolicy Jul 12 17:53:11.907472 ip-10-0-129-128 kubenswrapper[2559]: W0712 17:53:11.551438 2559 feature_gate.go:328] unrecognized feature gate: HyperShiftOnlyDynamicResourceAllocation Jul 12 17:53:11.907472 ip-10-0-129-128 kubenswrapper[2559]: W0712 17:53:11.551440 2559 feature_gate.go:328] unrecognized feature gate: ClusterAPIInstallIBMCloud Jul 12 17:53:11.907472 ip-10-0-129-128 kubenswrapper[2559]: W0712 17:53:11.551443 2559 feature_gate.go:328] unrecognized feature gate: MultiArchInstallAzure Jul 12 17:53:11.907472 ip-10-0-129-128 kubenswrapper[2559]: W0712 17:53:11.551446 2559 feature_gate.go:328] unrecognized feature gate: VSphereMultiNetworks Jul 12 17:53:11.907472 ip-10-0-129-128 kubenswrapper[2559]: W0712 17:53:11.551450 2559 feature_gate.go:328] unrecognized feature gate: InsightsConfig Jul 12 17:53:11.907472 ip-10-0-129-128 kubenswrapper[2559]: W0712 17:53:11.551452 2559 feature_gate.go:328] unrecognized feature gate: OSStreams Jul 12 17:53:11.907472 ip-10-0-129-128 kubenswrapper[2559]: W0712 17:53:11.551455 2559 feature_gate.go:328] unrecognized feature gate: GCPCustomAPIEndpointsInstall Jul 12 17:53:11.907472 ip-10-0-129-128 kubenswrapper[2559]: W0712 17:53:11.551457 2559 feature_gate.go:328] unrecognized feature gate: NewOLMOwnSingleNamespace Jul 12 17:53:11.907472 ip-10-0-129-128 kubenswrapper[2559]: W0712 17:53:11.551460 2559 feature_gate.go:328] unrecognized feature gate: IrreconcilableMachineConfig Jul 12 17:53:11.907472 ip-10-0-129-128 kubenswrapper[2559]: W0712 17:53:11.551462 2559 feature_gate.go:328] unrecognized feature gate: PreconfiguredUDNAddresses Jul 12 17:53:11.907472 ip-10-0-129-128 kubenswrapper[2559]: W0712 17:53:11.551465 2559 feature_gate.go:328] unrecognized feature gate: VolumeGroupSnapshot Jul 12 17:53:11.907472 ip-10-0-129-128 kubenswrapper[2559]: W0712 17:53:11.551467 2559 feature_gate.go:328] unrecognized feature gate: MultiDiskSetup Jul 12 17:53:11.907472 ip-10-0-129-128 kubenswrapper[2559]: W0712 17:53:11.551470 2559 feature_gate.go:328] unrecognized feature gate: GCPClusterHostedDNSInstall Jul 12 17:53:11.907472 ip-10-0-129-128 kubenswrapper[2559]: W0712 17:53:11.551472 2559 feature_gate.go:328] unrecognized feature gate: ClusterVersionOperatorConfiguration Jul 12 17:53:11.907472 ip-10-0-129-128 kubenswrapper[2559]: W0712 17:53:11.551475 2559 feature_gate.go:328] unrecognized feature gate: DyanmicServiceEndpointIBMCloud Jul 12 17:53:11.907472 ip-10-0-129-128 kubenswrapper[2559]: W0712 17:53:11.551477 2559 feature_gate.go:328] unrecognized feature gate: AutomatedEtcdBackup Jul 12 17:53:11.908333 ip-10-0-129-128 kubenswrapper[2559]: W0712 17:53:11.551480 2559 feature_gate.go:328] unrecognized feature gate: PinnedImages Jul 12 17:53:11.908333 ip-10-0-129-128 kubenswrapper[2559]: W0712 17:53:11.551483 2559 feature_gate.go:328] unrecognized feature gate: AWSClusterHostedDNS Jul 12 17:53:11.908333 ip-10-0-129-128 kubenswrapper[2559]: W0712 17:53:11.551486 2559 feature_gate.go:328] unrecognized feature gate: AzureMultiDisk Jul 12 17:53:11.908333 ip-10-0-129-128 kubenswrapper[2559]: W0712 17:53:11.551489 2559 feature_gate.go:328] unrecognized feature gate: GatewayAPI Jul 12 17:53:11.908333 ip-10-0-129-128 kubenswrapper[2559]: W0712 17:53:11.551491 2559 feature_gate.go:328] unrecognized feature gate: MachineConfigNodes Jul 12 17:53:11.908333 ip-10-0-129-128 kubenswrapper[2559]: W0712 17:53:11.551494 2559 feature_gate.go:328] unrecognized feature gate: GCPCustomAPIEndpoints Jul 12 17:53:11.908333 ip-10-0-129-128 kubenswrapper[2559]: W0712 17:53:11.551496 2559 feature_gate.go:328] unrecognized feature gate: AWSDedicatedHosts Jul 12 17:53:11.908333 ip-10-0-129-128 kubenswrapper[2559]: W0712 17:53:11.551499 2559 feature_gate.go:328] unrecognized feature gate: NoRegistryClusterInstall Jul 12 17:53:11.908333 ip-10-0-129-128 kubenswrapper[2559]: W0712 17:53:11.551501 2559 feature_gate.go:328] unrecognized feature gate: EventTTL Jul 12 17:53:11.908333 ip-10-0-129-128 kubenswrapper[2559]: W0712 17:53:11.551506 2559 feature_gate.go:328] unrecognized feature gate: AzureWorkloadIdentity Jul 12 17:53:11.908333 ip-10-0-129-128 kubenswrapper[2559]: W0712 17:53:11.551509 2559 feature_gate.go:328] unrecognized feature gate: IngressControllerDynamicConfigurationManager Jul 12 17:53:11.908333 ip-10-0-129-128 kubenswrapper[2559]: W0712 17:53:11.551512 2559 feature_gate.go:349] Setting deprecated feature gate KMSv1=true. It will be removed in a future release. Jul 12 17:53:11.908333 ip-10-0-129-128 kubenswrapper[2559]: W0712 17:53:11.551515 2559 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesvSphere Jul 12 17:53:11.908333 ip-10-0-129-128 kubenswrapper[2559]: W0712 17:53:11.551517 2559 feature_gate.go:328] unrecognized feature gate: ClusterAPIMachineManagement Jul 12 17:53:11.908333 ip-10-0-129-128 kubenswrapper[2559]: W0712 17:53:11.551520 2559 feature_gate.go:328] unrecognized feature gate: AlibabaPlatform Jul 12 17:53:11.908333 ip-10-0-129-128 kubenswrapper[2559]: W0712 17:53:11.551522 2559 feature_gate.go:328] unrecognized feature gate: GatewayAPIWithoutOLM Jul 12 17:53:11.908333 ip-10-0-129-128 kubenswrapper[2559]: W0712 17:53:11.551525 2559 feature_gate.go:328] unrecognized feature gate: NewOLMWebhookProviderOpenshiftServiceCA Jul 12 17:53:11.908333 ip-10-0-129-128 kubenswrapper[2559]: W0712 17:53:11.551527 2559 feature_gate.go:328] unrecognized feature gate: InsightsOnDemandDataGather Jul 12 17:53:11.908333 ip-10-0-129-128 kubenswrapper[2559]: W0712 17:53:11.551530 2559 feature_gate.go:328] unrecognized feature gate: EtcdBackendQuota Jul 12 17:53:11.908333 ip-10-0-129-128 kubenswrapper[2559]: W0712 17:53:11.551532 2559 feature_gate.go:328] unrecognized feature gate: DualReplica Jul 12 17:53:11.909420 ip-10-0-129-128 kubenswrapper[2559]: W0712 17:53:11.551536 2559 feature_gate.go:328] unrecognized feature gate: ImageStreamImportMode Jul 12 17:53:11.909420 ip-10-0-129-128 kubenswrapper[2559]: W0712 17:53:11.551539 2559 feature_gate.go:328] unrecognized feature gate: VSphereHostVMGroupZonal Jul 12 17:53:11.909420 ip-10-0-129-128 kubenswrapper[2559]: W0712 17:53:11.551541 2559 feature_gate.go:328] unrecognized feature gate: SigstoreImageVerification Jul 12 17:53:11.909420 ip-10-0-129-128 kubenswrapper[2559]: W0712 17:53:11.551544 2559 feature_gate.go:328] unrecognized feature gate: GCPDualStackInstall Jul 12 17:53:11.909420 ip-10-0-129-128 kubenswrapper[2559]: W0712 17:53:11.551546 2559 feature_gate.go:328] unrecognized feature gate: GatewayAPIController Jul 12 17:53:11.909420 ip-10-0-129-128 kubenswrapper[2559]: W0712 17:53:11.551549 2559 feature_gate.go:328] unrecognized feature gate: ClusterMonitoringConfig Jul 12 17:53:11.909420 ip-10-0-129-128 kubenswrapper[2559]: W0712 17:53:11.551551 2559 feature_gate.go:328] unrecognized feature gate: MachineAPIOperatorDisableMachineHealthCheckController Jul 12 17:53:11.909420 ip-10-0-129-128 kubenswrapper[2559]: W0712 17:53:11.551555 2559 feature_gate.go:351] Setting GA feature gate ServiceAccountTokenNodeBinding=true. It will be removed in a future release. Jul 12 17:53:11.909420 ip-10-0-129-128 kubenswrapper[2559]: W0712 17:53:11.551558 2559 feature_gate.go:328] unrecognized feature gate: GCPClusterHostedDNS Jul 12 17:53:11.909420 ip-10-0-129-128 kubenswrapper[2559]: W0712 17:53:11.551561 2559 feature_gate.go:328] unrecognized feature gate: MetricsCollectionProfiles Jul 12 17:53:11.909420 ip-10-0-129-128 kubenswrapper[2559]: W0712 17:53:11.551564 2559 feature_gate.go:328] unrecognized feature gate: MachineAPIMigration Jul 12 17:53:11.909420 ip-10-0-129-128 kubenswrapper[2559]: W0712 17:53:11.551567 2559 feature_gate.go:328] unrecognized feature gate: VSphereMultiDisk Jul 12 17:53:11.909420 ip-10-0-129-128 kubenswrapper[2559]: W0712 17:53:11.551570 2559 feature_gate.go:328] unrecognized feature gate: ManagedBootImages Jul 12 17:53:11.909420 ip-10-0-129-128 kubenswrapper[2559]: W0712 17:53:11.551572 2559 feature_gate.go:328] unrecognized feature gate: OVNObservability Jul 12 17:53:11.909420 ip-10-0-129-128 kubenswrapper[2559]: W0712 17:53:11.551575 2559 feature_gate.go:328] unrecognized feature gate: AzureDedicatedHosts Jul 12 17:53:11.909420 ip-10-0-129-128 kubenswrapper[2559]: W0712 17:53:11.551578 2559 feature_gate.go:328] unrecognized feature gate: ExternalSnapshotMetadata Jul 12 17:53:11.909420 ip-10-0-129-128 kubenswrapper[2559]: W0712 17:53:11.551580 2559 feature_gate.go:328] unrecognized feature gate: ImageModeStatusReporting Jul 12 17:53:11.909420 ip-10-0-129-128 kubenswrapper[2559]: W0712 17:53:11.551583 2559 feature_gate.go:328] unrecognized feature gate: SignatureStores Jul 12 17:53:11.909420 ip-10-0-129-128 kubenswrapper[2559]: W0712 17:53:11.551586 2559 feature_gate.go:328] unrecognized feature gate: NetworkDiagnosticsConfig Jul 12 17:53:11.910282 ip-10-0-129-128 kubenswrapper[2559]: W0712 17:53:11.551588 2559 feature_gate.go:328] unrecognized feature gate: NetworkLiveMigration Jul 12 17:53:11.910282 ip-10-0-129-128 kubenswrapper[2559]: W0712 17:53:11.551590 2559 feature_gate.go:328] unrecognized feature gate: AzureClusterHostedDNSInstall Jul 12 17:53:11.910282 ip-10-0-129-128 kubenswrapper[2559]: W0712 17:53:11.551594 2559 feature_gate.go:328] unrecognized feature gate: VSphereMixedNodeEnv Jul 12 17:53:11.910282 ip-10-0-129-128 kubenswrapper[2559]: W0712 17:53:11.551597 2559 feature_gate.go:328] unrecognized feature gate: ClusterAPIMachineManagementVSphere Jul 12 17:53:11.910282 ip-10-0-129-128 kubenswrapper[2559]: W0712 17:53:11.551599 2559 feature_gate.go:328] unrecognized feature gate: NewOLMBoxCutterRuntime Jul 12 17:53:11.910282 ip-10-0-129-128 kubenswrapper[2559]: W0712 17:53:11.551602 2559 feature_gate.go:328] unrecognized feature gate: NewOLMPreflightPermissionChecks Jul 12 17:53:11.910282 ip-10-0-129-128 kubenswrapper[2559]: W0712 17:53:11.551604 2559 feature_gate.go:328] unrecognized feature gate: KMSEncryptionProvider Jul 12 17:53:11.910282 ip-10-0-129-128 kubenswrapper[2559]: W0712 17:53:11.551607 2559 feature_gate.go:328] unrecognized feature gate: Example Jul 12 17:53:11.910282 ip-10-0-129-128 kubenswrapper[2559]: W0712 17:53:11.551610 2559 feature_gate.go:328] unrecognized feature gate: ProvisioningRequestAvailable Jul 12 17:53:11.910282 ip-10-0-129-128 kubenswrapper[2559]: W0712 17:53:11.551612 2559 feature_gate.go:328] unrecognized feature gate: AWSDualStackInstall Jul 12 17:53:11.910282 ip-10-0-129-128 kubenswrapper[2559]: W0712 17:53:11.551615 2559 feature_gate.go:328] unrecognized feature gate: BootImageSkewEnforcement Jul 12 17:53:11.910282 ip-10-0-129-128 kubenswrapper[2559]: W0712 17:53:11.551617 2559 feature_gate.go:328] unrecognized feature gate: BootcNodeManagement Jul 12 17:53:11.910282 ip-10-0-129-128 kubenswrapper[2559]: W0712 17:53:11.551620 2559 feature_gate.go:328] unrecognized feature gate: ClusterAPIInstall Jul 12 17:53:11.910282 ip-10-0-129-128 kubenswrapper[2559]: W0712 17:53:11.551624 2559 feature_gate.go:328] unrecognized feature gate: ExternalOIDCWithUIDAndExtraClaimMappings Jul 12 17:53:11.910282 ip-10-0-129-128 kubenswrapper[2559]: W0712 17:53:11.551626 2559 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesCPMS Jul 12 17:53:11.910282 ip-10-0-129-128 kubenswrapper[2559]: W0712 17:53:11.551629 2559 feature_gate.go:328] unrecognized feature gate: RouteAdvertisements Jul 12 17:53:11.910282 ip-10-0-129-128 kubenswrapper[2559]: W0712 17:53:11.551631 2559 feature_gate.go:328] unrecognized feature gate: SigstoreImageVerificationPKI Jul 12 17:53:11.911140 ip-10-0-129-128 kubenswrapper[2559]: I0712 17:53:11.551636 2559 feature_gate.go:384] feature gates: {map[CBORServingAndStorage:false ClientsAllowCBOR:false ClientsPreferCBOR:false EventedPLEG:false ImageVolume:true KMSv1:true MaxUnavailableStatefulSet:false MinimumKubeletVersion:false MutableCSINodeAllocatableCount:false MutatingAdmissionPolicy:false ProcMountType:true RouteExternalCertificate:true SELinuxMount:false ServiceAccountTokenNodeBinding:true StoragePerformantSecurityPolicy:true TranslateStreamCloseWebsocketRequests:false UserNamespacesPodSecurityStandards:true UserNamespacesSupport:true VolumeAttributesClass:true]} Jul 12 17:53:11.911140 ip-10-0-129-128 kubenswrapper[2559]: I0712 17:53:11.551726 2559 server.go:962] "Client rotation is on, will bootstrap in background" Jul 12 17:53:11.911140 ip-10-0-129-128 kubenswrapper[2559]: I0712 17:53:11.555576 2559 bootstrap.go:101] "Use the bootstrap credentials to request a cert, and set kubeconfig to point to the certificate dir" Jul 12 17:53:11.911140 ip-10-0-129-128 kubenswrapper[2559]: I0712 17:53:11.558394 2559 server.go:1019] "Starting client certificate rotation" Jul 12 17:53:11.911140 ip-10-0-129-128 kubenswrapper[2559]: I0712 17:53:11.558506 2559 certificate_manager.go:422] "Certificate rotation is enabled" logger="kubernetes.io/kube-apiserver-client-kubelet" Jul 12 17:53:11.911140 ip-10-0-129-128 kubenswrapper[2559]: I0712 17:53:11.558649 2559 certificate_manager.go:566] "Rotating certificates" logger="kubernetes.io/kube-apiserver-client-kubelet" Jul 12 17:53:11.911140 ip-10-0-129-128 kubenswrapper[2559]: I0712 17:53:11.562523 2559 dynamic_cafile_content.go:123] "Loaded a new CA Bundle and Verifier" name="client-ca-bundle::/etc/kubernetes/kubelet-ca.crt" Jul 12 17:53:11.911140 ip-10-0-129-128 kubenswrapper[2559]: I0712 17:53:11.563663 2559 dynamic_cafile_content.go:161] "Starting controller" name="client-ca-bundle::/etc/kubernetes/kubelet-ca.crt" Jul 12 17:53:11.911140 ip-10-0-129-128 kubenswrapper[2559]: I0712 17:53:11.573540 2559 log.go:25] "Validated CRI v1 runtime API" Jul 12 17:53:11.911140 ip-10-0-129-128 kubenswrapper[2559]: I0712 17:53:11.575942 2559 log.go:25] "Validated CRI v1 image API" Jul 12 17:53:11.911140 ip-10-0-129-128 kubenswrapper[2559]: I0712 17:53:11.577574 2559 server.go:1429] "Using cgroup driver setting received from the CRI runtime" cgroupDriver="systemd" Jul 12 17:53:11.911140 ip-10-0-129-128 kubenswrapper[2559]: I0712 17:53:11.583198 2559 fs.go:135] Filesystem UUIDs: map[4306c0fb-8a85-41af-92c8-9fbde88e371c:/dev/nvme0n1p3 4bd0721e-450c-4ab3-a62e-14af5ce7114b:/dev/nvme0n1p4 7B77-95E7:/dev/nvme0n1p2] Jul 12 17:53:11.911783 ip-10-0-129-128 kubenswrapper[2559]: I0712 17:53:11.583218 2559 fs.go:136] Filesystem partitions: map[/dev/nvme0n1p3:{mountpoint:/boot major:259 minor:3 fsType:ext4 blockSize:0} /dev/nvme0n1p4:{mountpoint:/var major:259 minor:4 fsType:xfs blockSize:0} /dev/shm:{mountpoint:/dev/shm major:0 minor:22 fsType:tmpfs blockSize:0} /run:{mountpoint:/run major:0 minor:24 fsType:tmpfs blockSize:0} /tmp:{mountpoint:/tmp major:0 minor:31 fsType:tmpfs blockSize:0} composefs_0-33:{mountpoint:/ major:0 minor:33 fsType:overlay blockSize:0}] Jul 12 17:53:11.911996 ip-10-0-129-128 kubenswrapper[2559]: I0712 17:53:11.588987 2559 manager.go:216] Machine: {Timestamp:2026-07-12 17:53:11.58762315 +0000 UTC m=+0.367182557 CPUVendorID:GenuineIntel NumCores:8 NumPhysicalCores:4 NumSockets:1 CpuFrequency:3098078 MemoryCapacity:33164496896 SwapCapacity:0 MemoryByType:map[] NVMInfo:{MemoryModeCapacity:0 AppDirectModeCapacity:0 AvgPowerBudget:0} HugePages:[{PageSize:1048576 NumPages:0} {PageSize:2048 NumPages:0}] MachineID:ec2bf41a4376301643e181db5841778e SystemUUID:ec2bf41a-4376-3016-43e1-81db5841778e BootID:f36a43ff-ee1d-4b07-9943-9fe618be532b Filesystems:[{Device:composefs_0-33 DeviceMajor:0 DeviceMinor:33 Capacity:6107136 Type:vfs Inodes:18446744073709551615 HasInodes:true} {Device:/tmp DeviceMajor:0 DeviceMinor:31 Capacity:16582250496 Type:vfs Inodes:1048576 HasInodes:true} {Device:/dev/nvme0n1p3 DeviceMajor:259 DeviceMinor:3 Capacity:366869504 Type:vfs Inodes:98304 HasInodes:true} {Device:/dev/shm DeviceMajor:0 DeviceMinor:22 Capacity:16582246400 Type:vfs Inodes:4048400 HasInodes:true} {Device:/run DeviceMajor:0 DeviceMinor:24 Capacity:6632902656 Type:vfs Inodes:819200 HasInodes:true} {Device:/dev/nvme0n1p4 DeviceMajor:259 DeviceMinor:4 Capacity:128243970048 Type:vfs Inodes:62651840 HasInodes:true}] DiskMap:map[259:0:{Name:nvme0n1 Major:259 Minor:0 Size:128849018880 Scheduler:none}] NetworkDevices:[{Name:br-ex MacAddress:02:5d:42:cc:ce:93 Speed:0 Mtu:9001} {Name:ens5 MacAddress:02:5d:42:cc:ce:93 Speed:0 Mtu:9001} {Name:ovs-system MacAddress:f6:94:7c:da:3c:f2 Speed:0 Mtu:1500}] Topology:[{Id:0 Memory:33164496896 HugePages:[{PageSize:1048576 NumPages:0} {PageSize:2048 NumPages:0}] Cores:[{Id:0 Threads:[0 4] Caches:[{Id:0 Size:32768 Type:Data Level:1} {Id:0 Size:32768 Type:Instruction Level:1} {Id:0 Size:1048576 Type:Unified Level:2}] UncoreCaches:[] SocketID:0 BookID: DrawerID:} {Id:1 Threads:[1 5] Caches:[{Id:1 Size:32768 Type:Data Level:1} {Id:1 Size:32768 Type:Instruction Level:1} {Id:1 Size:1048576 Type:Unified Level:2}] UncoreCaches:[] SocketID:0 BookID: DrawerID:} {Id:2 Threads:[2 6] Caches:[{Id:2 Size:32768 Type:Data Level:1} {Id:2 Size:32768 Type:Instruction Level:1} {Id:2 Size:1048576 Type:Unified Level:2}] UncoreCaches:[] SocketID:0 BookID: DrawerID:} {Id:3 Threads:[3 7] Caches:[{Id:3 Size:32768 Type:Data Level:1} {Id:3 Size:32768 Type:Instruction Level:1} {Id:3 Size:1048576 Type:Unified Level:2}] UncoreCaches:[] SocketID:0 BookID: DrawerID:}] Caches:[{Id:0 Size:37486592 Type:Unified Level:3}] Distances:[10]}] CloudProvider:Unknown InstanceType:Unknown InstanceID:None} Jul 12 17:53:11.911996 ip-10-0-129-128 kubenswrapper[2559]: I0712 17:53:11.589089 2559 manager_no_libpfm.go:29] cAdvisor is build without cgo and/or libpfm support. Perf event counters are not available. Jul 12 17:53:11.911996 ip-10-0-129-128 kubenswrapper[2559]: I0712 17:53:11.589193 2559 manager.go:232] Version: {KernelVersion:5.14.0-570.126.1.el9_6.x86_64 ContainerOsVersion:Red Hat Enterprise Linux CoreOS 9.6.20260702-0 (Plow) DockerVersion: DockerAPIVersion: CadvisorVersion: CadvisorRevision:} Jul 12 17:53:11.911996 ip-10-0-129-128 kubenswrapper[2559]: I0712 17:53:11.591230 2559 container_manager_linux.go:273] "Container manager verified user specified cgroup-root exists" cgroupRoot=[] Jul 12 17:53:11.911996 ip-10-0-129-128 kubenswrapper[2559]: I0712 17:53:11.591251 2559 container_manager_linux.go:278] "Creating Container Manager object based on Node Config" nodeConfig={"NodeName":"ip-10-0-129-128.ec2.internal","RuntimeCgroupsName":"/system.slice/crio.service","SystemCgroupsName":"/system.slice","KubeletCgroupsName":"","KubeletOOMScoreAdj":-999,"ContainerRuntime":"","CgroupsPerQOS":true,"CgroupRoot":"/","CgroupDriver":"systemd","KubeletRootDir":"/var/lib/kubelet","ProtectKernelDefaults":true,"KubeReservedCgroupName":"","SystemReservedCgroupName":"","ReservedSystemCPUs":{},"EnforceNodeAllocatable":{"pods":{}},"KubeReserved":null,"SystemReserved":{"cpu":"500m","ephemeral-storage":"1Gi","memory":"1Gi"},"HardEvictionThresholds":[{"Signal":"imagefs.inodesFree","Operator":"LessThan","Value":{"Quantity":null,"Percentage":0.05},"GracePeriod":0,"MinReclaim":null},{"Signal":"memory.available","Operator":"LessThan","Value":{"Quantity":"100Mi","Percentage":0},"GracePeriod":0,"MinReclaim":null},{"Signal":"nodefs.available","Operator":"LessThan","Value":{"Quantity":null,"Percentage":0.1},"GracePeriod":0,"MinReclaim":null},{"Signal":"nodefs.inodesFree","Operator":"LessThan","Value":{"Quantity":null,"Percentage":0.05},"GracePeriod":0,"MinReclaim":null},{"Signal":"imagefs.available","Operator":"LessThan","Value":{"Quantity":null,"Percentage":0.15},"GracePeriod":0,"MinReclaim":null}],"QOSReserved":{},"CPUManagerPolicy":"none","CPUManagerPolicyOptions":null,"TopologyManagerScope":"container","CPUManagerReconcilePeriod":10000000000,"MemoryManagerPolicy":"None","MemoryManagerReservedMemory":null,"PodPidsLimit":4096,"EnforceCPULimits":true,"CPUCFSQuotaPeriod":100000000,"TopologyManagerPolicy":"none","TopologyManagerPolicyOptions":null,"CgroupVersion":2} Jul 12 17:53:11.911996 ip-10-0-129-128 kubenswrapper[2559]: I0712 17:53:11.592068 2559 topology_manager.go:138] "Creating topology manager with none policy" Jul 12 17:53:11.911996 ip-10-0-129-128 kubenswrapper[2559]: I0712 17:53:11.592077 2559 container_manager_linux.go:309] "Creating device plugin manager" Jul 12 17:53:11.911996 ip-10-0-129-128 kubenswrapper[2559]: I0712 17:53:11.592084 2559 manager.go:141] "Creating Device Plugin manager" path="/var/lib/kubelet/device-plugins/kubelet.sock" Jul 12 17:53:11.912632 ip-10-0-129-128 kubenswrapper[2559]: I0712 17:53:11.592859 2559 server.go:74] "Creating device plugin registration server" version="v1beta1" socket="/var/lib/kubelet/device-plugins/kubelet.sock" Jul 12 17:53:11.912632 ip-10-0-129-128 kubenswrapper[2559]: I0712 17:53:11.594108 2559 container_manager_linux.go:318] "Creating Dynamic Resource Allocation (DRA) manager" Jul 12 17:53:11.912632 ip-10-0-129-128 kubenswrapper[2559]: I0712 17:53:11.597621 2559 state_mem.go:36] "Initialized new in-memory state store" Jul 12 17:53:11.912632 ip-10-0-129-128 kubenswrapper[2559]: I0712 17:53:11.597906 2559 server.go:1240] "Using root directory" path="/var/lib/kubelet" Jul 12 17:53:11.912632 ip-10-0-129-128 kubenswrapper[2559]: I0712 17:53:11.598013 2559 kubelet.go:486] "Attempting to sync node with API server" Jul 12 17:53:11.912632 ip-10-0-129-128 kubenswrapper[2559]: I0712 17:53:11.598031 2559 kubelet.go:387] "Adding static pod path" path="/etc/kubernetes/manifests" Jul 12 17:53:11.912632 ip-10-0-129-128 kubenswrapper[2559]: I0712 17:53:11.598052 2559 file.go:69] "Watching path" path="/etc/kubernetes/manifests" Jul 12 17:53:11.912632 ip-10-0-129-128 kubenswrapper[2559]: I0712 17:53:11.598067 2559 kubelet.go:398] "Adding apiserver pod source" Jul 12 17:53:11.912632 ip-10-0-129-128 kubenswrapper[2559]: I0712 17:53:11.598079 2559 apiserver.go:42] "Waiting for node sync before watching apiserver pods" Jul 12 17:53:11.912632 ip-10-0-129-128 kubenswrapper[2559]: I0712 17:53:11.598554 2559 state_mem.go:40] "Initialized new in-memory state store for pod resource information tracking" Jul 12 17:53:11.912632 ip-10-0-129-128 kubenswrapper[2559]: I0712 17:53:11.598579 2559 state_mem.go:40] "Initialized new in-memory state store for pod resource information tracking" Jul 12 17:53:11.912632 ip-10-0-129-128 kubenswrapper[2559]: I0712 17:53:11.599632 2559 reflector.go:439] "Caches populated" logger="kubernetes.io/kube-apiserver-client-kubelet" type="*v1.CertificateSigningRequest" reflector="k8s.io/client-go/tools/watch/informerwatcher.go:162" Jul 12 17:53:11.912632 ip-10-0-129-128 kubenswrapper[2559]: I0712 17:53:11.601514 2559 kuberuntime_manager.go:301] "Container runtime initialized" containerRuntime="cri-o" version="1.34.10-2.rhaos4.21.git83ee25b.el9" apiVersion="v1" Jul 12 17:53:11.912632 ip-10-0-129-128 kubenswrapper[2559]: I0712 17:53:11.603407 2559 kubelet.go:960] "Not starting ClusterTrustBundle informer because we are in static kubelet mode or the ClusterTrustBundleProjection featuregate is disabled" Jul 12 17:53:11.912632 ip-10-0-129-128 kubenswrapper[2559]: I0712 17:53:11.603450 2559 kubelet.go:984] "Not starting PodCertificateRequest manager because we are in static kubelet mode or the PodCertificateProjection feature gate is disabled" Jul 12 17:53:11.912632 ip-10-0-129-128 kubenswrapper[2559]: I0712 17:53:11.604234 2559 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/portworx-volume" Jul 12 17:53:11.912632 ip-10-0-129-128 kubenswrapper[2559]: I0712 17:53:11.604247 2559 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/empty-dir" Jul 12 17:53:11.912632 ip-10-0-129-128 kubenswrapper[2559]: I0712 17:53:11.604254 2559 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/git-repo" Jul 12 17:53:11.912632 ip-10-0-129-128 kubenswrapper[2559]: I0712 17:53:11.604260 2559 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/host-path" Jul 12 17:53:11.912632 ip-10-0-129-128 kubenswrapper[2559]: I0712 17:53:11.604269 2559 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/nfs" Jul 12 17:53:11.912632 ip-10-0-129-128 kubenswrapper[2559]: I0712 17:53:11.604275 2559 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/secret" Jul 12 17:53:11.912632 ip-10-0-129-128 kubenswrapper[2559]: I0712 17:53:11.604282 2559 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/iscsi" Jul 12 17:53:11.912632 ip-10-0-129-128 kubenswrapper[2559]: I0712 17:53:11.604288 2559 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/downward-api" Jul 12 17:53:11.912632 ip-10-0-129-128 kubenswrapper[2559]: I0712 17:53:11.604294 2559 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/fc" Jul 12 17:53:11.912632 ip-10-0-129-128 kubenswrapper[2559]: I0712 17:53:11.604299 2559 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/configmap" Jul 12 17:53:11.912632 ip-10-0-129-128 kubenswrapper[2559]: I0712 17:53:11.604307 2559 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/projected" Jul 12 17:53:11.912632 ip-10-0-129-128 kubenswrapper[2559]: I0712 17:53:11.604313 2559 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/local-volume" Jul 12 17:53:11.912632 ip-10-0-129-128 kubenswrapper[2559]: I0712 17:53:11.606030 2559 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/csi" Jul 12 17:53:11.912632 ip-10-0-129-128 kubenswrapper[2559]: I0712 17:53:11.606041 2559 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/image" Jul 12 17:53:11.912632 ip-10-0-129-128 kubenswrapper[2559]: I0712 17:53:11.609187 2559 server.go:1268] "Started kubelet" Jul 12 17:53:11.912632 ip-10-0-129-128 kubenswrapper[2559]: I0712 17:53:11.609258 2559 server.go:180] "Starting to listen" address="0.0.0.0" port=10250 Jul 12 17:53:11.912632 ip-10-0-129-128 kubenswrapper[2559]: I0712 17:53:11.610102 2559 ratelimit.go:56] "Setting rate limiting for endpoint" service="podresources" qps=100 burstTokens=10 Jul 12 17:53:11.912632 ip-10-0-129-128 kubenswrapper[2559]: I0712 17:53:11.610564 2559 server_v1.go:49] "podresources" method="list" useActivePods=true Jul 12 17:53:11.912632 ip-10-0-129-128 kubenswrapper[2559]: E0712 17:53:11.610574 2559 reflector.go:205] "Failed to watch" err="failed to list *v1.Service: services is forbidden: User \"system:anonymous\" cannot list resource \"services\" in API group \"\" at the cluster scope" logger="UnhandledError" reflector="k8s.io/client-go/informers/factory.go:160" type="*v1.Service" Jul 12 17:53:11.912632 ip-10-0-129-128 kubenswrapper[2559]: I0712 17:53:11.610643 2559 csi_plugin.go:990] Failed to contact API server when waiting for CSINode publishing: csinodes.storage.k8s.io "ip-10-0-129-128.ec2.internal" is forbidden: User "system:anonymous" cannot get resource "csinodes" in API group "storage.k8s.io" at the cluster scope Jul 12 17:53:11.912632 ip-10-0-129-128 kubenswrapper[2559]: E0712 17:53:11.610749 2559 reflector.go:205] "Failed to watch" err="failed to list *v1.Node: nodes \"ip-10-0-129-128.ec2.internal\" is forbidden: User \"system:anonymous\" cannot list resource \"nodes\" in API group \"\" at the cluster scope" logger="UnhandledError" reflector="k8s.io/client-go/informers/factory.go:160" type="*v1.Node" Jul 12 17:53:11.912632 ip-10-0-129-128 kubenswrapper[2559]: I0712 17:53:11.611028 2559 server.go:310] "Adding debug handlers to kubelet server" Jul 12 17:53:11.912632 ip-10-0-129-128 kubenswrapper[2559]: I0712 17:53:11.612437 2559 server.go:249] "Starting to serve the podresources API" endpoint="unix:/var/lib/kubelet/pod-resources/kubelet.sock" Jul 12 17:53:11.912632 ip-10-0-129-128 kubenswrapper[2559]: E0712 17:53:11.614570 2559 kubelet.go:1638] "Image garbage collection failed once. Stats initialization may not have completed yet" err="failed to get imageFs info: unable to find data in memory cache" Jul 12 17:53:11.916726 ip-10-0-129-128 kubenswrapper[2559]: E0712 17:53:11.614081 2559 event.go:359] "Server rejected event (will not retry!)" err="events is forbidden: User \"system:anonymous\" cannot create resource \"events\" in API group \"\" in the namespace \"default\"" event="&Event{ObjectMeta:{ip-10-0-129-128.ec2.internal.18c19c4f40374e53 default 0 0001-01-01 00:00:00 +0000 UTC map[] map[] [] [] []},InvolvedObject:ObjectReference{Kind:Node,Namespace:,Name:ip-10-0-129-128.ec2.internal,UID:ip-10-0-129-128.ec2.internal,APIVersion:,ResourceVersion:,FieldPath:,},Reason:Starting,Message:Starting kubelet.,Source:EventSource{Component:kubelet,Host:ip-10-0-129-128.ec2.internal,},FirstTimestamp:2026-07-12 17:53:11.609142867 +0000 UTC m=+0.388702275,LastTimestamp:2026-07-12 17:53:11.609142867 +0000 UTC m=+0.388702275,Count:1,Type:Normal,EventTime:0001-01-01 00:00:00 +0000 UTC,Series:nil,Action:,Related:nil,ReportingController:kubelet,ReportingInstance:ip-10-0-129-128.ec2.internal,}" Jul 12 17:53:11.916726 ip-10-0-129-128 kubenswrapper[2559]: I0712 17:53:11.617407 2559 fs_resource_analyzer.go:67] "Starting FS ResourceAnalyzer" Jul 12 17:53:11.916726 ip-10-0-129-128 kubenswrapper[2559]: I0712 17:53:11.617417 2559 certificate_manager.go:422] "Certificate rotation is enabled" logger="kubernetes.io/kubelet-serving" Jul 12 17:53:11.916726 ip-10-0-129-128 kubenswrapper[2559]: E0712 17:53:11.617484 2559 kubelet_node_status.go:454] "Error getting the current node from lister" err="node \"ip-10-0-129-128.ec2.internal\" not found" Jul 12 17:53:11.916726 ip-10-0-129-128 kubenswrapper[2559]: I0712 17:53:11.618114 2559 factory.go:55] Registering systemd factory Jul 12 17:53:11.916726 ip-10-0-129-128 kubenswrapper[2559]: I0712 17:53:11.618130 2559 factory.go:223] Registration of the systemd container factory successfully Jul 12 17:53:11.916726 ip-10-0-129-128 kubenswrapper[2559]: I0712 17:53:11.618134 2559 volume_manager.go:311] "The desired_state_of_world populator starts" Jul 12 17:53:11.916726 ip-10-0-129-128 kubenswrapper[2559]: I0712 17:53:11.618141 2559 desired_state_of_world_populator.go:146] "Desired state populator starts to run" Jul 12 17:53:11.916726 ip-10-0-129-128 kubenswrapper[2559]: I0712 17:53:11.618166 2559 volume_manager.go:313] "Starting Kubelet Volume Manager" Jul 12 17:53:11.916726 ip-10-0-129-128 kubenswrapper[2559]: I0712 17:53:11.618303 2559 reconstruct.go:97] "Volume reconstruction finished" Jul 12 17:53:11.916726 ip-10-0-129-128 kubenswrapper[2559]: I0712 17:53:11.618309 2559 reconciler.go:29] "Reconciler: start to sync state" Jul 12 17:53:11.916726 ip-10-0-129-128 kubenswrapper[2559]: I0712 17:53:11.618367 2559 factory.go:153] Registering CRI-O factory Jul 12 17:53:11.916726 ip-10-0-129-128 kubenswrapper[2559]: I0712 17:53:11.618378 2559 factory.go:223] Registration of the crio container factory successfully Jul 12 17:53:11.916726 ip-10-0-129-128 kubenswrapper[2559]: I0712 17:53:11.618423 2559 factory.go:221] Registration of the containerd container factory failed: unable to create containerd client: containerd: cannot unix dial containerd api service: dial unix /run/containerd/containerd.sock: connect: no such file or directory Jul 12 17:53:11.916726 ip-10-0-129-128 kubenswrapper[2559]: I0712 17:53:11.618447 2559 factory.go:103] Registering Raw factory Jul 12 17:53:11.916726 ip-10-0-129-128 kubenswrapper[2559]: I0712 17:53:11.618460 2559 manager.go:1196] Started watching for new ooms in manager Jul 12 17:53:11.916726 ip-10-0-129-128 kubenswrapper[2559]: I0712 17:53:11.618970 2559 manager.go:349] Starting recovery of all containers Jul 12 17:53:11.916726 ip-10-0-129-128 kubenswrapper[2559]: I0712 17:53:11.626516 2559 kubelet_network_linux.go:54] "Initialized iptables rules." protocol="IPv6" Jul 12 17:53:11.916726 ip-10-0-129-128 kubenswrapper[2559]: I0712 17:53:11.627370 2559 manager.go:354] Recovery completed Jul 12 17:53:11.916726 ip-10-0-129-128 kubenswrapper[2559]: E0712 17:53:11.628685 2559 reflector.go:205] "Failed to watch" err="failed to list *v1.CSIDriver: csidrivers.storage.k8s.io is forbidden: User \"system:anonymous\" cannot list resource \"csidrivers\" in API group \"storage.k8s.io\" at the cluster scope" logger="UnhandledError" reflector="k8s.io/client-go/informers/factory.go:160" type="*v1.CSIDriver" Jul 12 17:53:11.916726 ip-10-0-129-128 kubenswrapper[2559]: E0712 17:53:11.628743 2559 controller.go:145] "Failed to ensure lease exists, will retry" err="leases.coordination.k8s.io \"ip-10-0-129-128.ec2.internal\" is forbidden: User \"system:anonymous\" cannot get resource \"leases\" in API group \"coordination.k8s.io\" in the namespace \"kube-node-lease\"" interval="200ms" Jul 12 17:53:11.916726 ip-10-0-129-128 kubenswrapper[2559]: I0712 17:53:11.633084 2559 kubelet_node_status.go:400] "Setting node annotation to enable volume controller attach/detach" Jul 12 17:53:11.916726 ip-10-0-129-128 kubenswrapper[2559]: I0712 17:53:11.634940 2559 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-129-128.ec2.internal" event="NodeHasSufficientMemory" Jul 12 17:53:11.916726 ip-10-0-129-128 kubenswrapper[2559]: I0712 17:53:11.634965 2559 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-129-128.ec2.internal" event="NodeHasNoDiskPressure" Jul 12 17:53:11.916726 ip-10-0-129-128 kubenswrapper[2559]: I0712 17:53:11.634976 2559 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-129-128.ec2.internal" event="NodeHasSufficientPID" Jul 12 17:53:11.916726 ip-10-0-129-128 kubenswrapper[2559]: I0712 17:53:11.635517 2559 cpu_manager.go:222] "Starting CPU manager" policy="none" Jul 12 17:53:11.916726 ip-10-0-129-128 kubenswrapper[2559]: I0712 17:53:11.635532 2559 cpu_manager.go:223] "Reconciling" reconcilePeriod="10s" Jul 12 17:53:11.916726 ip-10-0-129-128 kubenswrapper[2559]: I0712 17:53:11.635551 2559 state_mem.go:36] "Initialized new in-memory state store" Jul 12 17:53:11.916726 ip-10-0-129-128 kubenswrapper[2559]: I0712 17:53:11.635522 2559 kubelet_node_status.go:400] "Setting node annotation to enable volume controller attach/detach" Jul 12 17:53:11.916726 ip-10-0-129-128 kubenswrapper[2559]: I0712 17:53:11.636172 2559 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-129-128.ec2.internal" event="NodeHasSufficientMemory" Jul 12 17:53:11.916726 ip-10-0-129-128 kubenswrapper[2559]: I0712 17:53:11.636194 2559 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-129-128.ec2.internal" event="NodeHasNoDiskPressure" Jul 12 17:53:11.916726 ip-10-0-129-128 kubenswrapper[2559]: I0712 17:53:11.636203 2559 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-129-128.ec2.internal" event="NodeHasSufficientPID" Jul 12 17:53:11.916726 ip-10-0-129-128 kubenswrapper[2559]: I0712 17:53:11.636602 2559 policy_none.go:49] "None policy: Start" Jul 12 17:53:11.916726 ip-10-0-129-128 kubenswrapper[2559]: I0712 17:53:11.636614 2559 memory_manager.go:187] "Starting memorymanager" policy="None" Jul 12 17:53:12.031223 ip-10-0-129-128 kubenswrapper[2559]: I0712 17:53:11.636625 2559 state_mem.go:36] "Initializing new in-memory state store" logger="Memory Manager state checkpoint" Jul 12 17:53:12.031223 ip-10-0-129-128 kubenswrapper[2559]: I0712 17:53:11.637373 2559 policy_none.go:47] "Start" Jul 12 17:53:12.031223 ip-10-0-129-128 kubenswrapper[2559]: I0712 17:53:11.644969 2559 csr.go:274] "Certificate signing request is approved, waiting to be issued" logger="kubernetes.io/kube-apiserver-client-kubelet" csr="csr-9qwfc" Jul 12 17:53:12.031223 ip-10-0-129-128 kubenswrapper[2559]: E0712 17:53:11.646859 2559 event.go:359] "Server rejected event (will not retry!)" err="events is forbidden: User \"system:anonymous\" cannot create resource \"events\" in API group \"\" in the namespace \"default\"" event="&Event{ObjectMeta:{ip-10-0-129-128.ec2.internal.18c19c4f41c11dde default 0 0001-01-01 00:00:00 +0000 UTC map[] map[] [] [] []},InvolvedObject:ObjectReference{Kind:Node,Namespace:,Name:ip-10-0-129-128.ec2.internal,UID:ip-10-0-129-128.ec2.internal,APIVersion:,ResourceVersion:,FieldPath:,},Reason:NodeHasSufficientMemory,Message:Node ip-10-0-129-128.ec2.internal status is now: NodeHasSufficientMemory,Source:EventSource{Component:kubelet,Host:ip-10-0-129-128.ec2.internal,},FirstTimestamp:2026-07-12 17:53:11.634951646 +0000 UTC m=+0.414511052,LastTimestamp:2026-07-12 17:53:11.634951646 +0000 UTC m=+0.414511052,Count:1,Type:Normal,EventTime:0001-01-01 00:00:00 +0000 UTC,Series:nil,Action:,Related:nil,ReportingController:kubelet,ReportingInstance:ip-10-0-129-128.ec2.internal,}" Jul 12 17:53:12.031223 ip-10-0-129-128 kubenswrapper[2559]: I0712 17:53:11.651893 2559 csr.go:270] "Certificate signing request is issued" logger="kubernetes.io/kube-apiserver-client-kubelet" csr="csr-9qwfc" Jul 12 17:53:12.031223 ip-10-0-129-128 kubenswrapper[2559]: E0712 17:53:11.655231 2559 event.go:359] "Server rejected event (will not retry!)" err="events is forbidden: User \"system:anonymous\" cannot create resource \"events\" in API group \"\" in the namespace \"default\"" event="&Event{ObjectMeta:{ip-10-0-129-128.ec2.internal.18c19c4f41c1661f default 0 0001-01-01 00:00:00 +0000 UTC map[] map[] [] [] []},InvolvedObject:ObjectReference{Kind:Node,Namespace:,Name:ip-10-0-129-128.ec2.internal,UID:ip-10-0-129-128.ec2.internal,APIVersion:,ResourceVersion:,FieldPath:,},Reason:NodeHasNoDiskPressure,Message:Node ip-10-0-129-128.ec2.internal status is now: NodeHasNoDiskPressure,Source:EventSource{Component:kubelet,Host:ip-10-0-129-128.ec2.internal,},FirstTimestamp:2026-07-12 17:53:11.634970143 +0000 UTC m=+0.414529549,LastTimestamp:2026-07-12 17:53:11.634970143 +0000 UTC m=+0.414529549,Count:1,Type:Normal,EventTime:0001-01-01 00:00:00 +0000 UTC,Series:nil,Action:,Related:nil,ReportingController:kubelet,ReportingInstance:ip-10-0-129-128.ec2.internal,}" Jul 12 17:53:12.031223 ip-10-0-129-128 kubenswrapper[2559]: E0712 17:53:11.717536 2559 kubelet_node_status.go:454] "Error getting the current node from lister" err="node \"ip-10-0-129-128.ec2.internal\" not found" Jul 12 17:53:12.031223 ip-10-0-129-128 kubenswrapper[2559]: I0712 17:53:11.736355 2559 manager.go:337] "Starting Device Plugin manager" Jul 12 17:53:12.031223 ip-10-0-129-128 kubenswrapper[2559]: E0712 17:53:11.736385 2559 manager.go:513] "Failed to read data from checkpoint" err="checkpoint is not found" checkpoint="kubelet_internal_checkpoint" Jul 12 17:53:12.031223 ip-10-0-129-128 kubenswrapper[2559]: I0712 17:53:11.736395 2559 server.go:87] "Starting device plugin registration server" Jul 12 17:53:12.031223 ip-10-0-129-128 kubenswrapper[2559]: I0712 17:53:11.736677 2559 eviction_manager.go:189] "Eviction manager: starting control loop" Jul 12 17:53:12.031223 ip-10-0-129-128 kubenswrapper[2559]: I0712 17:53:11.736688 2559 container_log_manager.go:146] "Initializing container log rotate workers" workers=1 monitorPeriod="10s" Jul 12 17:53:12.031223 ip-10-0-129-128 kubenswrapper[2559]: I0712 17:53:11.736789 2559 plugin_watcher.go:51] "Plugin Watcher Start" path="/var/lib/kubelet/plugins_registry" Jul 12 17:53:12.031223 ip-10-0-129-128 kubenswrapper[2559]: I0712 17:53:11.736871 2559 plugin_manager.go:116] "The desired_state_of_world populator (plugin watcher) starts" Jul 12 17:53:12.031223 ip-10-0-129-128 kubenswrapper[2559]: I0712 17:53:11.736933 2559 plugin_manager.go:118] "Starting Kubelet Plugin Manager" Jul 12 17:53:12.031223 ip-10-0-129-128 kubenswrapper[2559]: E0712 17:53:11.737435 2559 eviction_manager.go:267] "eviction manager: failed to check if we have separate container filesystem. Ignoring." err="non-existent label \"crio-containers\"" Jul 12 17:53:12.031223 ip-10-0-129-128 kubenswrapper[2559]: E0712 17:53:11.737476 2559 eviction_manager.go:292] "Eviction manager: failed to get summary stats" err="failed to get node info: node \"ip-10-0-129-128.ec2.internal\" not found" Jul 12 17:53:12.031223 ip-10-0-129-128 kubenswrapper[2559]: I0712 17:53:11.742248 2559 kubelet_network_linux.go:54] "Initialized iptables rules." protocol="IPv4" Jul 12 17:53:12.031223 ip-10-0-129-128 kubenswrapper[2559]: I0712 17:53:11.742281 2559 status_manager.go:244] "Starting to sync pod status with apiserver" Jul 12 17:53:12.031223 ip-10-0-129-128 kubenswrapper[2559]: I0712 17:53:11.742300 2559 kubelet.go:2448] "Starting kubelet main sync loop" Jul 12 17:53:12.031223 ip-10-0-129-128 kubenswrapper[2559]: E0712 17:53:11.742327 2559 kubelet.go:2472] "Skipping pod synchronization" err="PLEG is not healthy: pleg has yet to be successful" Jul 12 17:53:12.031223 ip-10-0-129-128 kubenswrapper[2559]: I0712 17:53:11.745022 2559 reflector.go:439] "Caches populated" type="*v1.RuntimeClass" reflector="k8s.io/client-go/informers/factory.go:160" Jul 12 17:53:12.031223 ip-10-0-129-128 kubenswrapper[2559]: E0712 17:53:11.835031 2559 nodelease.go:49] "Failed to get node when trying to set owner ref to the node lease" err="nodes \"ip-10-0-129-128.ec2.internal\" not found" node="ip-10-0-129-128.ec2.internal" Jul 12 17:53:12.031223 ip-10-0-129-128 kubenswrapper[2559]: I0712 17:53:11.837107 2559 kubelet_node_status.go:400] "Setting node annotation to enable volume controller attach/detach" Jul 12 17:53:12.031223 ip-10-0-129-128 kubenswrapper[2559]: I0712 17:53:11.838017 2559 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-129-128.ec2.internal" event="NodeHasSufficientMemory" Jul 12 17:53:12.031223 ip-10-0-129-128 kubenswrapper[2559]: I0712 17:53:11.838043 2559 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-129-128.ec2.internal" event="NodeHasNoDiskPressure" Jul 12 17:53:12.038653 ip-10-0-129-128 kubenswrapper[2559]: I0712 17:53:11.838055 2559 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-129-128.ec2.internal" event="NodeHasSufficientPID" Jul 12 17:53:12.038653 ip-10-0-129-128 kubenswrapper[2559]: I0712 17:53:11.838084 2559 kubelet_node_status.go:78] "Attempting to register node" node="ip-10-0-129-128.ec2.internal" Jul 12 17:53:12.038653 ip-10-0-129-128 kubenswrapper[2559]: I0712 17:53:11.843169 2559 kubelet.go:2535] "SyncLoop ADD" source="file" pods=["openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-129-128.ec2.internal","kube-system/kube-apiserver-proxy-ip-10-0-129-128.ec2.internal"] Jul 12 17:53:12.038653 ip-10-0-129-128 kubenswrapper[2559]: I0712 17:53:11.843242 2559 kubelet_node_status.go:400] "Setting node annotation to enable volume controller attach/detach" Jul 12 17:53:12.038653 ip-10-0-129-128 kubenswrapper[2559]: I0712 17:53:11.846624 2559 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-129-128.ec2.internal" event="NodeHasSufficientMemory" Jul 12 17:53:12.038653 ip-10-0-129-128 kubenswrapper[2559]: I0712 17:53:11.846647 2559 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-129-128.ec2.internal" event="NodeHasNoDiskPressure" Jul 12 17:53:12.038653 ip-10-0-129-128 kubenswrapper[2559]: I0712 17:53:11.846661 2559 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-129-128.ec2.internal" event="NodeHasSufficientPID" Jul 12 17:53:12.038653 ip-10-0-129-128 kubenswrapper[2559]: I0712 17:53:11.847919 2559 kubelet_node_status.go:400] "Setting node annotation to enable volume controller attach/detach" Jul 12 17:53:12.038653 ip-10-0-129-128 kubenswrapper[2559]: I0712 17:53:11.848040 2559 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-129-128.ec2.internal" Jul 12 17:53:12.038653 ip-10-0-129-128 kubenswrapper[2559]: I0712 17:53:11.848069 2559 kubelet_node_status.go:400] "Setting node annotation to enable volume controller attach/detach" Jul 12 17:53:12.038653 ip-10-0-129-128 kubenswrapper[2559]: I0712 17:53:11.848632 2559 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-129-128.ec2.internal" event="NodeHasSufficientMemory" Jul 12 17:53:12.038653 ip-10-0-129-128 kubenswrapper[2559]: I0712 17:53:11.848655 2559 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-129-128.ec2.internal" event="NodeHasNoDiskPressure" Jul 12 17:53:12.038653 ip-10-0-129-128 kubenswrapper[2559]: I0712 17:53:11.848665 2559 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-129-128.ec2.internal" event="NodeHasSufficientPID" Jul 12 17:53:12.038653 ip-10-0-129-128 kubenswrapper[2559]: I0712 17:53:11.849222 2559 kubelet_node_status.go:81] "Successfully registered node" node="ip-10-0-129-128.ec2.internal" Jul 12 17:53:12.038653 ip-10-0-129-128 kubenswrapper[2559]: E0712 17:53:11.849243 2559 kubelet_node_status.go:536] "Error updating node status, will retry" err="error getting node \"ip-10-0-129-128.ec2.internal\": node \"ip-10-0-129-128.ec2.internal\" not found" Jul 12 17:53:12.038653 ip-10-0-129-128 kubenswrapper[2559]: I0712 17:53:11.849273 2559 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-129-128.ec2.internal" event="NodeHasSufficientMemory" Jul 12 17:53:12.038653 ip-10-0-129-128 kubenswrapper[2559]: I0712 17:53:11.849296 2559 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-129-128.ec2.internal" event="NodeHasNoDiskPressure" Jul 12 17:53:12.038653 ip-10-0-129-128 kubenswrapper[2559]: I0712 17:53:11.849305 2559 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-129-128.ec2.internal" event="NodeHasSufficientPID" Jul 12 17:53:12.038653 ip-10-0-129-128 kubenswrapper[2559]: I0712 17:53:11.850276 2559 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/kube-apiserver-proxy-ip-10-0-129-128.ec2.internal" Jul 12 17:53:12.038653 ip-10-0-129-128 kubenswrapper[2559]: I0712 17:53:11.850297 2559 kubelet_node_status.go:400] "Setting node annotation to enable volume controller attach/detach" Jul 12 17:53:12.038653 ip-10-0-129-128 kubenswrapper[2559]: I0712 17:53:11.850909 2559 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-129-128.ec2.internal" event="NodeHasSufficientMemory" Jul 12 17:53:12.038653 ip-10-0-129-128 kubenswrapper[2559]: I0712 17:53:11.850931 2559 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-129-128.ec2.internal" event="NodeHasNoDiskPressure" Jul 12 17:53:12.038653 ip-10-0-129-128 kubenswrapper[2559]: I0712 17:53:11.850943 2559 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-129-128.ec2.internal" event="NodeHasSufficientPID" Jul 12 17:53:12.038653 ip-10-0-129-128 kubenswrapper[2559]: E0712 17:53:11.868851 2559 kubelet_node_status.go:454] "Error getting the current node from lister" err="node \"ip-10-0-129-128.ec2.internal\" not found" Jul 12 17:53:12.038653 ip-10-0-129-128 kubenswrapper[2559]: E0712 17:53:11.891939 2559 kubelet.go:3244] "No need to create a mirror pod, since failed to get node info from the cluster" err="node \"ip-10-0-129-128.ec2.internal\" not found" node="ip-10-0-129-128.ec2.internal" Jul 12 17:53:12.038653 ip-10-0-129-128 kubenswrapper[2559]: E0712 17:53:11.896607 2559 kubelet.go:3244] "No need to create a mirror pod, since failed to get node info from the cluster" err="node \"ip-10-0-129-128.ec2.internal\" not found" node="ip-10-0-129-128.ec2.internal" Jul 12 17:53:12.038653 ip-10-0-129-128 kubenswrapper[2559]: I0712 17:53:11.919865 2559 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"etc-kube\" (UniqueName: \"kubernetes.io/host-path/862cce2888b36ce6c9c6601a2547ab41-etc-kube\") pod \"kube-rbac-proxy-crio-ip-10-0-129-128.ec2.internal\" (UID: \"862cce2888b36ce6c9c6601a2547ab41\") " pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-129-128.ec2.internal" Jul 12 17:53:12.038653 ip-10-0-129-128 kubenswrapper[2559]: I0712 17:53:11.919888 2559 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"var-lib-kubelet\" (UniqueName: \"kubernetes.io/host-path/862cce2888b36ce6c9c6601a2547ab41-var-lib-kubelet\") pod \"kube-rbac-proxy-crio-ip-10-0-129-128.ec2.internal\" (UID: \"862cce2888b36ce6c9c6601a2547ab41\") " pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-129-128.ec2.internal" Jul 12 17:53:12.039986 ip-10-0-129-128 kubenswrapper[2559]: I0712 17:53:11.919905 2559 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"config\" (UniqueName: \"kubernetes.io/host-path/89625644b4c452349901306bb1be9b95-config\") pod \"kube-apiserver-proxy-ip-10-0-129-128.ec2.internal\" (UID: \"89625644b4c452349901306bb1be9b95\") " pod="kube-system/kube-apiserver-proxy-ip-10-0-129-128.ec2.internal" Jul 12 17:53:12.039986 ip-10-0-129-128 kubenswrapper[2559]: E0712 17:53:11.969230 2559 kubelet_node_status.go:454] "Error getting the current node from lister" err="node \"ip-10-0-129-128.ec2.internal\" not found" Jul 12 17:53:12.039986 ip-10-0-129-128 kubenswrapper[2559]: I0712 17:53:12.020032 2559 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"etc-kube\" (UniqueName: \"kubernetes.io/host-path/862cce2888b36ce6c9c6601a2547ab41-etc-kube\") pod \"kube-rbac-proxy-crio-ip-10-0-129-128.ec2.internal\" (UID: \"862cce2888b36ce6c9c6601a2547ab41\") " pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-129-128.ec2.internal" Jul 12 17:53:12.039986 ip-10-0-129-128 kubenswrapper[2559]: I0712 17:53:12.020076 2559 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"var-lib-kubelet\" (UniqueName: \"kubernetes.io/host-path/862cce2888b36ce6c9c6601a2547ab41-var-lib-kubelet\") pod \"kube-rbac-proxy-crio-ip-10-0-129-128.ec2.internal\" (UID: \"862cce2888b36ce6c9c6601a2547ab41\") " pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-129-128.ec2.internal" Jul 12 17:53:12.039986 ip-10-0-129-128 kubenswrapper[2559]: I0712 17:53:12.020126 2559 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"etc-kube\" (UniqueName: \"kubernetes.io/host-path/862cce2888b36ce6c9c6601a2547ab41-etc-kube\") pod \"kube-rbac-proxy-crio-ip-10-0-129-128.ec2.internal\" (UID: \"862cce2888b36ce6c9c6601a2547ab41\") " pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-129-128.ec2.internal" Jul 12 17:53:12.039986 ip-10-0-129-128 kubenswrapper[2559]: I0712 17:53:12.020195 2559 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"var-lib-kubelet\" (UniqueName: \"kubernetes.io/host-path/862cce2888b36ce6c9c6601a2547ab41-var-lib-kubelet\") pod \"kube-rbac-proxy-crio-ip-10-0-129-128.ec2.internal\" (UID: \"862cce2888b36ce6c9c6601a2547ab41\") " pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-129-128.ec2.internal" Jul 12 17:53:12.039986 ip-10-0-129-128 kubenswrapper[2559]: I0712 17:53:12.020228 2559 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"config\" (UniqueName: \"kubernetes.io/host-path/89625644b4c452349901306bb1be9b95-config\") pod \"kube-apiserver-proxy-ip-10-0-129-128.ec2.internal\" (UID: \"89625644b4c452349901306bb1be9b95\") " pod="kube-system/kube-apiserver-proxy-ip-10-0-129-128.ec2.internal" Jul 12 17:53:12.039986 ip-10-0-129-128 kubenswrapper[2559]: I0712 17:53:12.020253 2559 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"config\" (UniqueName: \"kubernetes.io/host-path/89625644b4c452349901306bb1be9b95-config\") pod \"kube-apiserver-proxy-ip-10-0-129-128.ec2.internal\" (UID: \"89625644b4c452349901306bb1be9b95\") " pod="kube-system/kube-apiserver-proxy-ip-10-0-129-128.ec2.internal" Jul 12 17:53:12.136192 ip-10-0-129-128 kubenswrapper[2559]: E0712 17:53:12.070255 2559 kubelet_node_status.go:454] "Error getting the current node from lister" err="node \"ip-10-0-129-128.ec2.internal\" not found" Jul 12 17:53:12.279030 ip-10-0-129-128 kubenswrapper[2559]: E0712 17:53:12.170864 2559 kubelet_node_status.go:454] "Error getting the current node from lister" err="node \"ip-10-0-129-128.ec2.internal\" not found" Jul 12 17:53:12.279030 ip-10-0-129-128 kubenswrapper[2559]: I0712 17:53:12.193971 2559 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-129-128.ec2.internal" Jul 12 17:53:12.279030 ip-10-0-129-128 kubenswrapper[2559]: I0712 17:53:12.199726 2559 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/kube-apiserver-proxy-ip-10-0-129-128.ec2.internal" Jul 12 17:53:12.279030 ip-10-0-129-128 kubenswrapper[2559]: E0712 17:53:12.271469 2559 kubelet_node_status.go:454] "Error getting the current node from lister" err="node \"ip-10-0-129-128.ec2.internal\" not found" Jul 12 17:53:12.379171 ip-10-0-129-128 kubenswrapper[2559]: E0712 17:53:12.371964 2559 kubelet_node_status.go:454] "Error getting the current node from lister" err="node \"ip-10-0-129-128.ec2.internal\" not found" Jul 12 17:53:13.797535 ip-10-0-129-128 kubenswrapper[2559]: E0712 17:53:12.472505 2559 kubelet_node_status.go:454] "Error getting the current node from lister" err="node \"ip-10-0-129-128.ec2.internal\" not found" Jul 12 17:53:13.910106 ip-10-0-129-128 kubenswrapper[2559]: I0712 17:53:13.842653 2559 transport.go:147] "Certificate rotation detected, shutting down client connections to start using new credentials" Jul 12 17:53:13.910106 ip-10-0-129-128 kubenswrapper[2559]: I0712 17:53:13.842812 2559 reflector.go:571] "Warning: watch ended with error" reflector="k8s.io/client-go/informers/factory.go:160" type="*v1.RuntimeClass" err="very short watch: k8s.io/client-go/informers/factory.go:160: Unexpected watch close - watch lasted less than a second and no items received" Jul 12 17:53:13.910106 ip-10-0-129-128 kubenswrapper[2559]: E0712 17:53:13.858929 2559 kubelet_node_status.go:454] "Error getting the current node from lister" err="node \"ip-10-0-129-128.ec2.internal\" not found" Jul 12 17:53:13.910106 ip-10-0-129-128 kubenswrapper[2559]: I0712 17:53:13.901538 2559 reflector.go:439] "Caches populated" type="*v1.CSIDriver" reflector="k8s.io/client-go/informers/factory.go:160" Jul 12 17:53:14.007702 ip-10-0-129-128 kubenswrapper[2559]: I0712 17:53:13.941768 2559 certificate_manager.go:715] "Certificate rotation deadline determined" logger="kubernetes.io/kube-apiserver-client-kubelet" expiration="2028-07-11 17:48:12 +0000 UTC" deadline="2028-04-28 07:52:14.17780338 +0000 UTC" Jul 12 17:53:14.007702 ip-10-0-129-128 kubenswrapper[2559]: I0712 17:53:13.941798 2559 certificate_manager.go:431] "Waiting for next certificate rotation" logger="kubernetes.io/kube-apiserver-client-kubelet" sleep="15733h59m0.236008712s" Jul 12 17:53:14.007702 ip-10-0-129-128 kubenswrapper[2559]: I0712 17:53:13.943056 2559 reflector.go:439] "Caches populated" type="*v1.Node" reflector="k8s.io/client-go/informers/factory.go:160" Jul 12 17:53:14.007702 ip-10-0-129-128 kubenswrapper[2559]: I0712 17:53:14.003531 2559 kubelet.go:3248] "Creating a mirror pod for static pod" pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-129-128.ec2.internal" Jul 12 17:53:14.017145 ip-10-0-129-128 kubenswrapper[2559]: I0712 17:53:14.016722 2559 warnings.go:110] "Warning: metadata.name: this is used in the Pod's hostname, which can result in surprising behavior; a DNS label is recommended: [must not contain dots]" Jul 12 17:53:14.102156 ip-10-0-129-128 kubenswrapper[2559]: I0712 17:53:14.018293 2559 kubelet.go:3248] "Creating a mirror pod for static pod" pod="kube-system/kube-apiserver-proxy-ip-10-0-129-128.ec2.internal" Jul 12 17:53:14.102156 ip-10-0-129-128 kubenswrapper[2559]: I0712 17:53:14.026910 2559 warnings.go:110] "Warning: metadata.name: this is used in the Pod's hostname, which can result in surprising behavior; a DNS label is recommended: [must not contain dots]" Jul 12 17:53:14.201420 ip-10-0-129-128 kubenswrapper[2559]: W0712 17:53:14.156089 2559 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod89625644b4c452349901306bb1be9b95.slice/crio-b21ddbce83ff6ecccf906e84f76462a21d3f412e75be1f0fb292e1cd9101fdbc WatchSource:0}: Error finding container b21ddbce83ff6ecccf906e84f76462a21d3f412e75be1f0fb292e1cd9101fdbc: Status 404 returned error can't find the container with id b21ddbce83ff6ecccf906e84f76462a21d3f412e75be1f0fb292e1cd9101fdbc Jul 12 17:53:14.201420 ip-10-0-129-128 kubenswrapper[2559]: W0712 17:53:14.156527 2559 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod862cce2888b36ce6c9c6601a2547ab41.slice/crio-60ac41c3b6c2ebc2c09802f521dae9426aaab25d711049f38d311b23926f97b8 WatchSource:0}: Error finding container 60ac41c3b6c2ebc2c09802f521dae9426aaab25d711049f38d311b23926f97b8: Status 404 returned error can't find the container with id 60ac41c3b6c2ebc2c09802f521dae9426aaab25d711049f38d311b23926f97b8 Jul 12 17:53:14.201420 ip-10-0-129-128 kubenswrapper[2559]: I0712 17:53:14.160585 2559 provider.go:93] Refreshing cache for provider: *credentialprovider.defaultDockerConfigProvider Jul 12 17:53:14.211395 ip-10-0-129-128 kubenswrapper[2559]: I0712 17:53:14.210929 2559 reflector.go:439] "Caches populated" type="*v1.Service" reflector="k8s.io/client-go/informers/factory.go:160" Jul 12 17:53:14.725599 ip-10-0-129-128 kubenswrapper[2559]: I0712 17:53:14.669544 2559 reflector.go:439] "Caches populated" type="*v1.RuntimeClass" reflector="k8s.io/client-go/informers/factory.go:160" Jul 12 17:53:14.886593 ip-10-0-129-128 kubenswrapper[2559]: I0712 17:53:14.886560 2559 apiserver.go:52] "Watching apiserver" Jul 12 17:53:14.897041 ip-10-0-129-128 kubenswrapper[2559]: I0712 17:53:14.897009 2559 reflector.go:439] "Caches populated" type="*v1.Pod" reflector="pkg/kubelet/config/apiserver.go:66" Jul 12 17:53:14.899071 ip-10-0-129-128 kubenswrapper[2559]: I0712 17:53:14.899045 2559 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["kube-system/global-pull-secret-syncer-mpqvv","openshift-image-registry/node-ca-vxx7w","openshift-multus/multus-additional-cni-plugins-9dh8l","openshift-multus/multus-c2jnt","openshift-multus/network-metrics-daemon-qtkvt","openshift-network-diagnostics/network-check-target-h4bgw","openshift-network-operator/iptables-alerter-fql27","openshift-ovn-kubernetes/ovnkube-node-q5dw8","kube-system/konnectivity-agent-g6rx5","kube-system/kube-apiserver-proxy-ip-10-0-129-128.ec2.internal","openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-m2r7q","openshift-cluster-node-tuning-operator/tuned-66xkf","openshift-dns/node-resolver-fqnvr","openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-129-128.ec2.internal"] Jul 12 17:53:14.901869 ip-10-0-129-128 kubenswrapper[2559]: I0712 17:53:14.901810 2559 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-operator/iptables-alerter-fql27" Jul 12 17:53:14.903336 ip-10-0-129-128 kubenswrapper[2559]: I0712 17:53:14.903316 2559 certificate_manager.go:566] "Rotating certificates" logger="kubernetes.io/kubelet-serving" Jul 12 17:53:14.904070 ip-10-0-129-128 kubenswrapper[2559]: I0712 17:53:14.903888 2559 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-dns/node-resolver-fqnvr" Jul 12 17:53:14.904844 ip-10-0-129-128 kubenswrapper[2559]: I0712 17:53:14.904813 2559 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-network-operator\"/\"openshift-service-ca.crt\"" Jul 12 17:53:14.905094 ip-10-0-129-128 kubenswrapper[2559]: I0712 17:53:14.905067 2559 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-network-operator\"/\"kube-root-ca.crt\"" Jul 12 17:53:14.905191 ip-10-0-129-128 kubenswrapper[2559]: I0712 17:53:14.905097 2559 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-network-operator\"/\"iptables-alerter-script\"" Jul 12 17:53:14.905191 ip-10-0-129-128 kubenswrapper[2559]: I0712 17:53:14.905109 2559 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-network-operator\"/\"iptables-alerter-dockercfg-lcbzt\"" Jul 12 17:53:14.906149 ip-10-0-129-128 kubenswrapper[2559]: I0712 17:53:14.906066 2559 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/multus-additional-cni-plugins-9dh8l" Jul 12 17:53:14.906752 ip-10-0-129-128 kubenswrapper[2559]: I0712 17:53:14.906518 2559 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-dns\"/\"kube-root-ca.crt\"" Jul 12 17:53:14.906752 ip-10-0-129-128 kubenswrapper[2559]: I0712 17:53:14.906585 2559 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-dns\"/\"node-resolver-dockercfg-58ljb\"" Jul 12 17:53:14.906752 ip-10-0-129-128 kubenswrapper[2559]: I0712 17:53:14.906662 2559 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-dns\"/\"openshift-service-ca.crt\"" Jul 12 17:53:14.909481 ip-10-0-129-128 kubenswrapper[2559]: I0712 17:53:14.908713 2559 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/multus-c2jnt" Jul 12 17:53:14.909481 ip-10-0-129-128 kubenswrapper[2559]: I0712 17:53:14.908910 2559 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-multus\"/\"cni-copy-resources\"" Jul 12 17:53:14.909481 ip-10-0-129-128 kubenswrapper[2559]: I0712 17:53:14.909160 2559 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-multus\"/\"kube-root-ca.crt\"" Jul 12 17:53:14.909481 ip-10-0-129-128 kubenswrapper[2559]: I0712 17:53:14.909174 2559 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-multus\"/\"multus-ancillary-tools-dockercfg-qw8kl\"" Jul 12 17:53:14.909481 ip-10-0-129-128 kubenswrapper[2559]: I0712 17:53:14.909340 2559 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-multus\"/\"openshift-service-ca.crt\"" Jul 12 17:53:14.909481 ip-10-0-129-128 kubenswrapper[2559]: I0712 17:53:14.909392 2559 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-multus\"/\"default-cni-sysctl-allowlist\"" Jul 12 17:53:14.909855 ip-10-0-129-128 kubenswrapper[2559]: I0712 17:53:14.909571 2559 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-multus\"/\"whereabouts-flatfile-config\"" Jul 12 17:53:14.910912 ip-10-0-129-128 kubenswrapper[2559]: I0712 17:53:14.910894 2559 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-qtkvt" Jul 12 17:53:14.910996 ip-10-0-129-128 kubenswrapper[2559]: E0712 17:53:14.910958 2559 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-qtkvt" podUID="6f65bad8-9ec7-4a51-8557-74b3f79ef76c" Jul 12 17:53:14.911313 ip-10-0-129-128 kubenswrapper[2559]: I0712 17:53:14.911295 2559 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-multus\"/\"default-dockercfg-9fd42\"" Jul 12 17:53:14.912622 ip-10-0-129-128 kubenswrapper[2559]: I0712 17:53:14.911910 2559 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-multus\"/\"multus-daemon-config\"" Jul 12 17:53:14.913833 ip-10-0-129-128 kubenswrapper[2559]: I0712 17:53:14.913814 2559 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-h4bgw" Jul 12 17:53:14.913923 ip-10-0-129-128 kubenswrapper[2559]: E0712 17:53:14.913882 2559 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-h4bgw" podUID="e2c3a206-19c0-4d4d-bb98-a99190023ad4" Jul 12 17:53:14.915407 ip-10-0-129-128 kubenswrapper[2559]: I0712 17:53:14.915383 2559 reflector.go:439] "Caches populated" logger="kubernetes.io/kubelet-serving" type="*v1.CertificateSigningRequest" reflector="k8s.io/client-go/tools/watch/informerwatcher.go:162" Jul 12 17:53:14.916873 ip-10-0-129-128 kubenswrapper[2559]: I0712 17:53:14.916330 2559 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/konnectivity-agent-g6rx5" Jul 12 17:53:14.918933 ip-10-0-129-128 kubenswrapper[2559]: I0712 17:53:14.918915 2559 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"kube-system\"/\"konnectivity-ca-bundle\"" Jul 12 17:53:14.919047 ip-10-0-129-128 kubenswrapper[2559]: I0712 17:53:14.918948 2559 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-mpqvv" Jul 12 17:53:14.919047 ip-10-0-129-128 kubenswrapper[2559]: E0712 17:53:14.919009 2559 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-mpqvv" podUID="6ab3daa1-3376-4f53-8106-d6d7417a327e" Jul 12 17:53:14.919211 ip-10-0-129-128 kubenswrapper[2559]: I0712 17:53:14.919167 2559 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"kube-system\"/\"default-dockercfg-rf5jb\"" Jul 12 17:53:14.919211 ip-10-0-129-128 kubenswrapper[2559]: I0712 17:53:14.919169 2559 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-slash\" (UniqueName: \"kubernetes.io/host-path/5841758e-4587-4d16-807c-b4a40b824c1b-host-slash\") pod \"iptables-alerter-fql27\" (UID: \"5841758e-4587-4d16-807c-b4a40b824c1b\") " pod="openshift-network-operator/iptables-alerter-fql27" Jul 12 17:53:14.919393 ip-10-0-129-128 kubenswrapper[2559]: I0712 17:53:14.919211 2559 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-l54gm\" (UniqueName: \"kubernetes.io/projected/41eafd0c-4a0b-48f6-8e30-8e408412d389-kube-api-access-l54gm\") pod \"node-resolver-fqnvr\" (UID: \"41eafd0c-4a0b-48f6-8e30-8e408412d389\") " pod="openshift-dns/node-resolver-fqnvr" Jul 12 17:53:14.919393 ip-10-0-129-128 kubenswrapper[2559]: I0712 17:53:14.919238 2559 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"os-release\" (UniqueName: \"kubernetes.io/host-path/370d4e7b-6713-48b2-81ee-f055fe1d5490-os-release\") pod \"multus-additional-cni-plugins-9dh8l\" (UID: \"370d4e7b-6713-48b2-81ee-f055fe1d5490\") " pod="openshift-multus/multus-additional-cni-plugins-9dh8l" Jul 12 17:53:14.919393 ip-10-0-129-128 kubenswrapper[2559]: I0712 17:53:14.919275 2559 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"cni-sysctl-allowlist\" (UniqueName: \"kubernetes.io/configmap/370d4e7b-6713-48b2-81ee-f055fe1d5490-cni-sysctl-allowlist\") pod \"multus-additional-cni-plugins-9dh8l\" (UID: \"370d4e7b-6713-48b2-81ee-f055fe1d5490\") " pod="openshift-multus/multus-additional-cni-plugins-9dh8l" Jul 12 17:53:14.919393 ip-10-0-129-128 kubenswrapper[2559]: I0712 17:53:14.919299 2559 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-zz8nf\" (UniqueName: \"kubernetes.io/projected/370d4e7b-6713-48b2-81ee-f055fe1d5490-kube-api-access-zz8nf\") pod \"multus-additional-cni-plugins-9dh8l\" (UID: \"370d4e7b-6713-48b2-81ee-f055fe1d5490\") " pod="openshift-multus/multus-additional-cni-plugins-9dh8l" Jul 12 17:53:14.919393 ip-10-0-129-128 kubenswrapper[2559]: I0712 17:53:14.919329 2559 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"hosts-file\" (UniqueName: \"kubernetes.io/host-path/41eafd0c-4a0b-48f6-8e30-8e408412d389-hosts-file\") pod \"node-resolver-fqnvr\" (UID: \"41eafd0c-4a0b-48f6-8e30-8e408412d389\") " pod="openshift-dns/node-resolver-fqnvr" Jul 12 17:53:14.919393 ip-10-0-129-128 kubenswrapper[2559]: I0712 17:53:14.919355 2559 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"cni-binary-copy\" (UniqueName: \"kubernetes.io/configmap/370d4e7b-6713-48b2-81ee-f055fe1d5490-cni-binary-copy\") pod \"multus-additional-cni-plugins-9dh8l\" (UID: \"370d4e7b-6713-48b2-81ee-f055fe1d5490\") " pod="openshift-multus/multus-additional-cni-plugins-9dh8l" Jul 12 17:53:14.919393 ip-10-0-129-128 kubenswrapper[2559]: I0712 17:53:14.919381 2559 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"whereabouts-flatfile-configmap\" (UniqueName: \"kubernetes.io/configmap/370d4e7b-6713-48b2-81ee-f055fe1d5490-whereabouts-flatfile-configmap\") pod \"multus-additional-cni-plugins-9dh8l\" (UID: \"370d4e7b-6713-48b2-81ee-f055fe1d5490\") " pod="openshift-multus/multus-additional-cni-plugins-9dh8l" Jul 12 17:53:14.919869 ip-10-0-129-128 kubenswrapper[2559]: I0712 17:53:14.919401 2559 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"kube-system\"/\"konnectivity-agent\"" Jul 12 17:53:14.919869 ip-10-0-129-128 kubenswrapper[2559]: I0712 17:53:14.919404 2559 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-vx2tt\" (UniqueName: \"kubernetes.io/projected/5841758e-4587-4d16-807c-b4a40b824c1b-kube-api-access-vx2tt\") pod \"iptables-alerter-fql27\" (UID: \"5841758e-4587-4d16-807c-b4a40b824c1b\") " pod="openshift-network-operator/iptables-alerter-fql27" Jul 12 17:53:14.919869 ip-10-0-129-128 kubenswrapper[2559]: I0712 17:53:14.919430 2559 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"cnibin\" (UniqueName: \"kubernetes.io/host-path/370d4e7b-6713-48b2-81ee-f055fe1d5490-cnibin\") pod \"multus-additional-cni-plugins-9dh8l\" (UID: \"370d4e7b-6713-48b2-81ee-f055fe1d5490\") " pod="openshift-multus/multus-additional-cni-plugins-9dh8l" Jul 12 17:53:14.919869 ip-10-0-129-128 kubenswrapper[2559]: I0712 17:53:14.919456 2559 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tuning-conf-dir\" (UniqueName: \"kubernetes.io/host-path/370d4e7b-6713-48b2-81ee-f055fe1d5490-tuning-conf-dir\") pod \"multus-additional-cni-plugins-9dh8l\" (UID: \"370d4e7b-6713-48b2-81ee-f055fe1d5490\") " pod="openshift-multus/multus-additional-cni-plugins-9dh8l" Jul 12 17:53:14.919869 ip-10-0-129-128 kubenswrapper[2559]: I0712 17:53:14.919502 2559 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"iptables-alerter-script\" (UniqueName: \"kubernetes.io/configmap/5841758e-4587-4d16-807c-b4a40b824c1b-iptables-alerter-script\") pod \"iptables-alerter-fql27\" (UID: \"5841758e-4587-4d16-807c-b4a40b824c1b\") " pod="openshift-network-operator/iptables-alerter-fql27" Jul 12 17:53:14.919869 ip-10-0-129-128 kubenswrapper[2559]: I0712 17:53:14.919526 2559 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/41eafd0c-4a0b-48f6-8e30-8e408412d389-tmp-dir\") pod \"node-resolver-fqnvr\" (UID: \"41eafd0c-4a0b-48f6-8e30-8e408412d389\") " pod="openshift-dns/node-resolver-fqnvr" Jul 12 17:53:14.919869 ip-10-0-129-128 kubenswrapper[2559]: I0712 17:53:14.919549 2559 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"system-cni-dir\" (UniqueName: \"kubernetes.io/host-path/370d4e7b-6713-48b2-81ee-f055fe1d5490-system-cni-dir\") pod \"multus-additional-cni-plugins-9dh8l\" (UID: \"370d4e7b-6713-48b2-81ee-f055fe1d5490\") " pod="openshift-multus/multus-additional-cni-plugins-9dh8l" Jul 12 17:53:14.921100 ip-10-0-129-128 kubenswrapper[2559]: I0712 17:53:14.921054 2559 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-image-registry/node-ca-vxx7w" Jul 12 17:53:14.924275 ip-10-0-129-128 kubenswrapper[2559]: I0712 17:53:14.923990 2559 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-image-registry\"/\"kube-root-ca.crt\"" Jul 12 17:53:14.924275 ip-10-0-129-128 kubenswrapper[2559]: I0712 17:53:14.924031 2559 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-image-registry\"/\"openshift-service-ca.crt\"" Jul 12 17:53:14.924275 ip-10-0-129-128 kubenswrapper[2559]: I0712 17:53:14.923990 2559 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-image-registry\"/\"node-ca-dockercfg-w52gl\"" Jul 12 17:53:14.924275 ip-10-0-129-128 kubenswrapper[2559]: I0712 17:53:14.924223 2559 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-image-registry\"/\"image-registry-certificates\"" Jul 12 17:53:14.925669 ip-10-0-129-128 kubenswrapper[2559]: I0712 17:53:14.924742 2559 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ovn-kubernetes/ovnkube-node-q5dw8" Jul 12 17:53:14.929916 ip-10-0-129-128 kubenswrapper[2559]: I0712 17:53:14.927873 2559 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ovn-kubernetes\"/\"openshift-service-ca.crt\"" Jul 12 17:53:14.929916 ip-10-0-129-128 kubenswrapper[2559]: I0712 17:53:14.928004 2559 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ovn-kubernetes\"/\"ovn-node-metrics-cert\"" Jul 12 17:53:14.929916 ip-10-0-129-128 kubenswrapper[2559]: I0712 17:53:14.928305 2559 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ovn-kubernetes\"/\"ovnkube-config\"" Jul 12 17:53:14.929916 ip-10-0-129-128 kubenswrapper[2559]: I0712 17:53:14.928776 2559 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ovn-kubernetes\"/\"ovn-kubernetes-node-dockercfg-zfktj\"" Jul 12 17:53:14.933897 ip-10-0-129-128 kubenswrapper[2559]: I0712 17:53:14.930238 2559 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-m2r7q" Jul 12 17:53:14.933897 ip-10-0-129-128 kubenswrapper[2559]: I0712 17:53:14.931569 2559 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ovn-kubernetes\"/\"ovnkube-script-lib\"" Jul 12 17:53:14.933897 ip-10-0-129-128 kubenswrapper[2559]: I0712 17:53:14.931851 2559 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ovn-kubernetes\"/\"env-overrides\"" Jul 12 17:53:14.933897 ip-10-0-129-128 kubenswrapper[2559]: I0712 17:53:14.932124 2559 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ovn-kubernetes\"/\"kube-root-ca.crt\"" Jul 12 17:53:14.933897 ip-10-0-129-128 kubenswrapper[2559]: I0712 17:53:14.933013 2559 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-cluster-csi-drivers\"/\"openshift-service-ca.crt\"" Jul 12 17:53:14.933897 ip-10-0-129-128 kubenswrapper[2559]: I0712 17:53:14.933856 2559 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-cluster-csi-drivers\"/\"aws-ebs-csi-driver-node-metrics-serving-cert\"" Jul 12 17:53:14.934437 ip-10-0-129-128 kubenswrapper[2559]: I0712 17:53:14.934418 2559 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-cluster-csi-drivers\"/\"kube-root-ca.crt\"" Jul 12 17:53:14.934511 ip-10-0-129-128 kubenswrapper[2559]: I0712 17:53:14.934448 2559 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-cluster-csi-drivers\"/\"aws-ebs-csi-driver-node-sa-dockercfg-jhdfn\"" Jul 12 17:53:14.936557 ip-10-0-129-128 kubenswrapper[2559]: I0712 17:53:14.935834 2559 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-cluster-node-tuning-operator/tuned-66xkf" Jul 12 17:53:14.936557 ip-10-0-129-128 kubenswrapper[2559]: I0712 17:53:14.936533 2559 csr.go:274] "Certificate signing request is approved, waiting to be issued" logger="kubernetes.io/kubelet-serving" csr="csr-t9bvq" Jul 12 17:53:14.938200 ip-10-0-129-128 kubenswrapper[2559]: I0712 17:53:14.938181 2559 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-cluster-node-tuning-operator\"/\"openshift-service-ca.crt\"" Jul 12 17:53:14.938287 ip-10-0-129-128 kubenswrapper[2559]: I0712 17:53:14.938238 2559 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-cluster-node-tuning-operator\"/\"tuned-dockercfg-tvv4p\"" Jul 12 17:53:14.938445 ip-10-0-129-128 kubenswrapper[2559]: I0712 17:53:14.938394 2559 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-cluster-node-tuning-operator\"/\"kube-root-ca.crt\"" Jul 12 17:53:14.944912 ip-10-0-129-128 kubenswrapper[2559]: I0712 17:53:14.944897 2559 csr.go:270] "Certificate signing request is issued" logger="kubernetes.io/kubelet-serving" csr="csr-t9bvq" Jul 12 17:53:15.004827 ip-10-0-129-128 kubenswrapper[2559]: I0712 17:53:15.004802 2559 desired_state_of_world_populator.go:154] "Finished populating initial desired state of world" Jul 12 17:53:15.020035 ip-10-0-129-128 kubenswrapper[2559]: I0712 17:53:15.020005 2559 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-kubelet\" (UniqueName: \"kubernetes.io/host-path/e51bad50-bc74-4e38-8307-ca257b599314-host-kubelet\") pod \"ovnkube-node-q5dw8\" (UID: \"e51bad50-bc74-4e38-8307-ca257b599314\") " pod="openshift-ovn-kubernetes/ovnkube-node-q5dw8" Jul 12 17:53:15.020706 ip-10-0-129-128 kubenswrapper[2559]: I0712 17:53:15.020044 2559 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"multus-daemon-config\" (UniqueName: \"kubernetes.io/configmap/ff97f946-352d-4629-bf96-351edf53ebde-multus-daemon-config\") pod \"multus-c2jnt\" (UID: \"ff97f946-352d-4629-bf96-351edf53ebde\") " pod="openshift-multus/multus-c2jnt" Jul 12 17:53:15.020706 ip-10-0-129-128 kubenswrapper[2559]: I0712 17:53:15.020070 2559 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-bn2rx\" (UniqueName: \"kubernetes.io/projected/e2c3a206-19c0-4d4d-bb98-a99190023ad4-kube-api-access-bn2rx\") pod \"network-check-target-h4bgw\" (UID: \"e2c3a206-19c0-4d4d-bb98-a99190023ad4\") " pod="openshift-network-diagnostics/network-check-target-h4bgw" Jul 12 17:53:15.020706 ip-10-0-129-128 kubenswrapper[2559]: I0712 17:53:15.020098 2559 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"multus-cni-dir\" (UniqueName: \"kubernetes.io/host-path/ff97f946-352d-4629-bf96-351edf53ebde-multus-cni-dir\") pod \"multus-c2jnt\" (UID: \"ff97f946-352d-4629-bf96-351edf53ebde\") " pod="openshift-multus/multus-c2jnt" Jul 12 17:53:15.020706 ip-10-0-129-128 kubenswrapper[2559]: I0712 17:53:15.020121 2559 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-slash\" (UniqueName: \"kubernetes.io/host-path/5841758e-4587-4d16-807c-b4a40b824c1b-host-slash\") pod \"iptables-alerter-fql27\" (UID: \"5841758e-4587-4d16-807c-b4a40b824c1b\") " pod="openshift-network-operator/iptables-alerter-fql27" Jul 12 17:53:15.020706 ip-10-0-129-128 kubenswrapper[2559]: I0712 17:53:15.020137 2559 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-zz8nf\" (UniqueName: \"kubernetes.io/projected/370d4e7b-6713-48b2-81ee-f055fe1d5490-kube-api-access-zz8nf\") pod \"multus-additional-cni-plugins-9dh8l\" (UID: \"370d4e7b-6713-48b2-81ee-f055fe1d5490\") " pod="openshift-multus/multus-additional-cni-plugins-9dh8l" Jul 12 17:53:15.020706 ip-10-0-129-128 kubenswrapper[2559]: I0712 17:53:15.020158 2559 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"ovnkube-script-lib\" (UniqueName: \"kubernetes.io/configmap/e51bad50-bc74-4e38-8307-ca257b599314-ovnkube-script-lib\") pod \"ovnkube-node-q5dw8\" (UID: \"e51bad50-bc74-4e38-8307-ca257b599314\") " pod="openshift-ovn-kubernetes/ovnkube-node-q5dw8" Jul 12 17:53:15.020706 ip-10-0-129-128 kubenswrapper[2559]: I0712 17:53:15.020182 2559 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"os-release\" (UniqueName: \"kubernetes.io/host-path/ff97f946-352d-4629-bf96-351edf53ebde-os-release\") pod \"multus-c2jnt\" (UID: \"ff97f946-352d-4629-bf96-351edf53ebde\") " pod="openshift-multus/multus-c2jnt" Jul 12 17:53:15.020706 ip-10-0-129-128 kubenswrapper[2559]: I0712 17:53:15.020204 2559 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"cni-binary-copy\" (UniqueName: \"kubernetes.io/configmap/ff97f946-352d-4629-bf96-351edf53ebde-cni-binary-copy\") pod \"multus-c2jnt\" (UID: \"ff97f946-352d-4629-bf96-351edf53ebde\") " pod="openshift-multus/multus-c2jnt" Jul 12 17:53:15.020706 ip-10-0-129-128 kubenswrapper[2559]: I0712 17:53:15.020226 2559 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"multus-socket-dir-parent\" (UniqueName: \"kubernetes.io/host-path/ff97f946-352d-4629-bf96-351edf53ebde-multus-socket-dir-parent\") pod \"multus-c2jnt\" (UID: \"ff97f946-352d-4629-bf96-351edf53ebde\") " pod="openshift-multus/multus-c2jnt" Jul 12 17:53:15.020706 ip-10-0-129-128 kubenswrapper[2559]: I0712 17:53:15.020201 2559 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"host-slash\" (UniqueName: \"kubernetes.io/host-path/5841758e-4587-4d16-807c-b4a40b824c1b-host-slash\") pod \"iptables-alerter-fql27\" (UID: \"5841758e-4587-4d16-807c-b4a40b824c1b\") " pod="openshift-network-operator/iptables-alerter-fql27" Jul 12 17:53:15.020706 ip-10-0-129-128 kubenswrapper[2559]: I0712 17:53:15.020259 2559 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"etc-kubernetes\" (UniqueName: \"kubernetes.io/host-path/ff97f946-352d-4629-bf96-351edf53ebde-etc-kubernetes\") pod \"multus-c2jnt\" (UID: \"ff97f946-352d-4629-bf96-351edf53ebde\") " pod="openshift-multus/multus-c2jnt" Jul 12 17:53:15.020706 ip-10-0-129-128 kubenswrapper[2559]: I0712 17:53:15.020308 2559 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"run-openvswitch\" (UniqueName: \"kubernetes.io/host-path/e51bad50-bc74-4e38-8307-ca257b599314-run-openvswitch\") pod \"ovnkube-node-q5dw8\" (UID: \"e51bad50-bc74-4e38-8307-ca257b599314\") " pod="openshift-ovn-kubernetes/ovnkube-node-q5dw8" Jul 12 17:53:15.020706 ip-10-0-129-128 kubenswrapper[2559]: I0712 17:53:15.020325 2559 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"cni-binary-copy\" (UniqueName: \"kubernetes.io/configmap/370d4e7b-6713-48b2-81ee-f055fe1d5490-cni-binary-copy\") pod \"multus-additional-cni-plugins-9dh8l\" (UID: \"370d4e7b-6713-48b2-81ee-f055fe1d5490\") " pod="openshift-multus/multus-additional-cni-plugins-9dh8l" Jul 12 17:53:15.020706 ip-10-0-129-128 kubenswrapper[2559]: I0712 17:53:15.020356 2559 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-slash\" (UniqueName: \"kubernetes.io/host-path/e51bad50-bc74-4e38-8307-ca257b599314-host-slash\") pod \"ovnkube-node-q5dw8\" (UID: \"e51bad50-bc74-4e38-8307-ca257b599314\") " pod="openshift-ovn-kubernetes/ovnkube-node-q5dw8" Jul 12 17:53:15.021323 ip-10-0-129-128 kubenswrapper[2559]: I0712 17:53:15.020371 2559 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-p9sff\" (UniqueName: \"kubernetes.io/projected/e51bad50-bc74-4e38-8307-ca257b599314-kube-api-access-p9sff\") pod \"ovnkube-node-q5dw8\" (UID: \"e51bad50-bc74-4e38-8307-ca257b599314\") " pod="openshift-ovn-kubernetes/ovnkube-node-q5dw8" Jul 12 17:53:15.021323 ip-10-0-129-128 kubenswrapper[2559]: I0712 17:53:15.020408 2559 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-var-lib-cni-multus\" (UniqueName: \"kubernetes.io/host-path/ff97f946-352d-4629-bf96-351edf53ebde-host-var-lib-cni-multus\") pod \"multus-c2jnt\" (UID: \"ff97f946-352d-4629-bf96-351edf53ebde\") " pod="openshift-multus/multus-c2jnt" Jul 12 17:53:15.021323 ip-10-0-129-128 kubenswrapper[2559]: I0712 17:53:15.020431 2559 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-2wqhw\" (UniqueName: \"kubernetes.io/projected/ff97f946-352d-4629-bf96-351edf53ebde-kube-api-access-2wqhw\") pod \"multus-c2jnt\" (UID: \"ff97f946-352d-4629-bf96-351edf53ebde\") " pod="openshift-multus/multus-c2jnt" Jul 12 17:53:15.021323 ip-10-0-129-128 kubenswrapper[2559]: I0712 17:53:15.020476 2559 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"sys-fs\" (UniqueName: \"kubernetes.io/host-path/3e848887-84a9-4599-afc5-c3750c0857a0-sys-fs\") pod \"aws-ebs-csi-driver-node-m2r7q\" (UID: \"3e848887-84a9-4599-afc5-c3750c0857a0\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-m2r7q" Jul 12 17:53:15.021323 ip-10-0-129-128 kubenswrapper[2559]: I0712 17:53:15.020510 2559 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"run-systemd\" (UniqueName: \"kubernetes.io/host-path/e51bad50-bc74-4e38-8307-ca257b599314-run-systemd\") pod \"ovnkube-node-q5dw8\" (UID: \"e51bad50-bc74-4e38-8307-ca257b599314\") " pod="openshift-ovn-kubernetes/ovnkube-node-q5dw8" Jul 12 17:53:15.021323 ip-10-0-129-128 kubenswrapper[2559]: I0712 17:53:15.020542 2559 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-run-k8s-cni-cncf-io\" (UniqueName: \"kubernetes.io/host-path/ff97f946-352d-4629-bf96-351edf53ebde-host-run-k8s-cni-cncf-io\") pod \"multus-c2jnt\" (UID: \"ff97f946-352d-4629-bf96-351edf53ebde\") " pod="openshift-multus/multus-c2jnt" Jul 12 17:53:15.021323 ip-10-0-129-128 kubenswrapper[2559]: I0712 17:53:15.020573 2559 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-var-lib-kubelet\" (UniqueName: \"kubernetes.io/host-path/ff97f946-352d-4629-bf96-351edf53ebde-host-var-lib-kubelet\") pod \"multus-c2jnt\" (UID: \"ff97f946-352d-4629-bf96-351edf53ebde\") " pod="openshift-multus/multus-c2jnt" Jul 12 17:53:15.021323 ip-10-0-129-128 kubenswrapper[2559]: I0712 17:53:15.020601 2559 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"socket-dir\" (UniqueName: \"kubernetes.io/host-path/3e848887-84a9-4599-afc5-c3750c0857a0-socket-dir\") pod \"aws-ebs-csi-driver-node-m2r7q\" (UID: \"3e848887-84a9-4599-afc5-c3750c0857a0\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-m2r7q" Jul 12 17:53:15.021323 ip-10-0-129-128 kubenswrapper[2559]: I0712 17:53:15.020625 2559 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"registration-dir\" (UniqueName: \"kubernetes.io/host-path/3e848887-84a9-4599-afc5-c3750c0857a0-registration-dir\") pod \"aws-ebs-csi-driver-node-m2r7q\" (UID: \"3e848887-84a9-4599-afc5-c3750c0857a0\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-m2r7q" Jul 12 17:53:15.021323 ip-10-0-129-128 kubenswrapper[2559]: I0712 17:53:15.020674 2559 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/41eafd0c-4a0b-48f6-8e30-8e408412d389-tmp-dir\") pod \"node-resolver-fqnvr\" (UID: \"41eafd0c-4a0b-48f6-8e30-8e408412d389\") " pod="openshift-dns/node-resolver-fqnvr" Jul 12 17:53:15.021323 ip-10-0-129-128 kubenswrapper[2559]: I0712 17:53:15.020715 2559 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"konnectivity-ca\" (UniqueName: \"kubernetes.io/configmap/64692aea-0243-4167-a3b6-2a6dc81a8fef-konnectivity-ca\") pod \"konnectivity-agent-g6rx5\" (UID: \"64692aea-0243-4167-a3b6-2a6dc81a8fef\") " pod="kube-system/konnectivity-agent-g6rx5" Jul 12 17:53:15.021323 ip-10-0-129-128 kubenswrapper[2559]: I0712 17:53:15.020818 2559 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-cni-netd\" (UniqueName: \"kubernetes.io/host-path/e51bad50-bc74-4e38-8307-ca257b599314-host-cni-netd\") pod \"ovnkube-node-q5dw8\" (UID: \"e51bad50-bc74-4e38-8307-ca257b599314\") " pod="openshift-ovn-kubernetes/ovnkube-node-q5dw8" Jul 12 17:53:15.021323 ip-10-0-129-128 kubenswrapper[2559]: I0712 17:53:15.020845 2559 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"etc-sysctl-d\" (UniqueName: \"kubernetes.io/host-path/cf65b059-f8a2-48d6-a773-74f51ac6ae5d-etc-sysctl-d\") pod \"tuned-66xkf\" (UID: \"cf65b059-f8a2-48d6-a773-74f51ac6ae5d\") " pod="openshift-cluster-node-tuning-operator/tuned-66xkf" Jul 12 17:53:15.021323 ip-10-0-129-128 kubenswrapper[2559]: I0712 17:53:15.020865 2559 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host\" (UniqueName: \"kubernetes.io/host-path/6d23c7c4-9eb6-483f-baf3-f25e641e2ead-host\") pod \"node-ca-vxx7w\" (UID: \"6d23c7c4-9eb6-483f-baf3-f25e641e2ead\") " pod="openshift-image-registry/node-ca-vxx7w" Jul 12 17:53:15.022029 ip-10-0-129-128 kubenswrapper[2559]: I0712 17:53:15.020890 2559 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"var-lib-openvswitch\" (UniqueName: \"kubernetes.io/host-path/e51bad50-bc74-4e38-8307-ca257b599314-var-lib-openvswitch\") pod \"ovnkube-node-q5dw8\" (UID: \"e51bad50-bc74-4e38-8307-ca257b599314\") " pod="openshift-ovn-kubernetes/ovnkube-node-q5dw8" Jul 12 17:53:15.022029 ip-10-0-129-128 kubenswrapper[2559]: I0712 17:53:15.020914 2559 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"etc-kubernetes\" (UniqueName: \"kubernetes.io/host-path/cf65b059-f8a2-48d6-a773-74f51ac6ae5d-etc-kubernetes\") pod \"tuned-66xkf\" (UID: \"cf65b059-f8a2-48d6-a773-74f51ac6ae5d\") " pod="openshift-cluster-node-tuning-operator/tuned-66xkf" Jul 12 17:53:15.022029 ip-10-0-129-128 kubenswrapper[2559]: I0712 17:53:15.020935 2559 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"etc-systemd\" (UniqueName: \"kubernetes.io/host-path/cf65b059-f8a2-48d6-a773-74f51ac6ae5d-etc-systemd\") pod \"tuned-66xkf\" (UID: \"cf65b059-f8a2-48d6-a773-74f51ac6ae5d\") " pod="openshift-cluster-node-tuning-operator/tuned-66xkf" Jul 12 17:53:15.022029 ip-10-0-129-128 kubenswrapper[2559]: I0712 17:53:15.020956 2559 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tmp\" (UniqueName: \"kubernetes.io/empty-dir/cf65b059-f8a2-48d6-a773-74f51ac6ae5d-tmp\") pod \"tuned-66xkf\" (UID: \"cf65b059-f8a2-48d6-a773-74f51ac6ae5d\") " pod="openshift-cluster-node-tuning-operator/tuned-66xkf" Jul 12 17:53:15.022029 ip-10-0-129-128 kubenswrapper[2559]: I0712 17:53:15.020978 2559 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-run-netns\" (UniqueName: \"kubernetes.io/host-path/ff97f946-352d-4629-bf96-351edf53ebde-host-run-netns\") pod \"multus-c2jnt\" (UID: \"ff97f946-352d-4629-bf96-351edf53ebde\") " pod="openshift-multus/multus-c2jnt" Jul 12 17:53:15.022029 ip-10-0-129-128 kubenswrapper[2559]: I0712 17:53:15.020993 2559 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/41eafd0c-4a0b-48f6-8e30-8e408412d389-tmp-dir\") pod \"node-resolver-fqnvr\" (UID: \"41eafd0c-4a0b-48f6-8e30-8e408412d389\") " pod="openshift-dns/node-resolver-fqnvr" Jul 12 17:53:15.022029 ip-10-0-129-128 kubenswrapper[2559]: I0712 17:53:15.021006 2559 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"hosts-file\" (UniqueName: \"kubernetes.io/host-path/41eafd0c-4a0b-48f6-8e30-8e408412d389-hosts-file\") pod \"node-resolver-fqnvr\" (UID: \"41eafd0c-4a0b-48f6-8e30-8e408412d389\") " pod="openshift-dns/node-resolver-fqnvr" Jul 12 17:53:15.022029 ip-10-0-129-128 kubenswrapper[2559]: I0712 17:53:15.021027 2559 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"agent-certs\" (UniqueName: \"kubernetes.io/secret/64692aea-0243-4167-a3b6-2a6dc81a8fef-agent-certs\") pod \"konnectivity-agent-g6rx5\" (UID: \"64692aea-0243-4167-a3b6-2a6dc81a8fef\") " pod="kube-system/konnectivity-agent-g6rx5" Jul 12 17:53:15.022029 ip-10-0-129-128 kubenswrapper[2559]: I0712 17:53:15.021050 2559 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-run-ovn-kubernetes\" (UniqueName: \"kubernetes.io/host-path/e51bad50-bc74-4e38-8307-ca257b599314-host-run-ovn-kubernetes\") pod \"ovnkube-node-q5dw8\" (UID: \"e51bad50-bc74-4e38-8307-ca257b599314\") " pod="openshift-ovn-kubernetes/ovnkube-node-q5dw8" Jul 12 17:53:15.022029 ip-10-0-129-128 kubenswrapper[2559]: I0712 17:53:15.021061 2559 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"hosts-file\" (UniqueName: \"kubernetes.io/host-path/41eafd0c-4a0b-48f6-8e30-8e408412d389-hosts-file\") pod \"node-resolver-fqnvr\" (UID: \"41eafd0c-4a0b-48f6-8e30-8e408412d389\") " pod="openshift-dns/node-resolver-fqnvr" Jul 12 17:53:15.022029 ip-10-0-129-128 kubenswrapper[2559]: I0712 17:53:15.021075 2559 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-var-lib-cni-networks-ovn-kubernetes\" (UniqueName: \"kubernetes.io/host-path/e51bad50-bc74-4e38-8307-ca257b599314-host-var-lib-cni-networks-ovn-kubernetes\") pod \"ovnkube-node-q5dw8\" (UID: \"e51bad50-bc74-4e38-8307-ca257b599314\") " pod="openshift-ovn-kubernetes/ovnkube-node-q5dw8" Jul 12 17:53:15.022029 ip-10-0-129-128 kubenswrapper[2559]: I0712 17:53:15.021099 2559 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"ovnkube-config\" (UniqueName: \"kubernetes.io/configmap/e51bad50-bc74-4e38-8307-ca257b599314-ovnkube-config\") pod \"ovnkube-node-q5dw8\" (UID: \"e51bad50-bc74-4e38-8307-ca257b599314\") " pod="openshift-ovn-kubernetes/ovnkube-node-q5dw8" Jul 12 17:53:15.022029 ip-10-0-129-128 kubenswrapper[2559]: I0712 17:53:15.021123 2559 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"run\" (UniqueName: \"kubernetes.io/host-path/cf65b059-f8a2-48d6-a773-74f51ac6ae5d-run\") pod \"tuned-66xkf\" (UID: \"cf65b059-f8a2-48d6-a773-74f51ac6ae5d\") " pod="openshift-cluster-node-tuning-operator/tuned-66xkf" Jul 12 17:53:15.022029 ip-10-0-129-128 kubenswrapper[2559]: I0712 17:53:15.021148 2559 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-vx2tt\" (UniqueName: \"kubernetes.io/projected/5841758e-4587-4d16-807c-b4a40b824c1b-kube-api-access-vx2tt\") pod \"iptables-alerter-fql27\" (UID: \"5841758e-4587-4d16-807c-b4a40b824c1b\") " pod="openshift-network-operator/iptables-alerter-fql27" Jul 12 17:53:15.022611 ip-10-0-129-128 kubenswrapper[2559]: I0712 17:53:15.021169 2559 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"cnibin\" (UniqueName: \"kubernetes.io/host-path/370d4e7b-6713-48b2-81ee-f055fe1d5490-cnibin\") pod \"multus-additional-cni-plugins-9dh8l\" (UID: \"370d4e7b-6713-48b2-81ee-f055fe1d5490\") " pod="openshift-multus/multus-additional-cni-plugins-9dh8l" Jul 12 17:53:15.022611 ip-10-0-129-128 kubenswrapper[2559]: I0712 17:53:15.021191 2559 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"etc-tuned\" (UniqueName: \"kubernetes.io/empty-dir/cf65b059-f8a2-48d6-a773-74f51ac6ae5d-etc-tuned\") pod \"tuned-66xkf\" (UID: \"cf65b059-f8a2-48d6-a773-74f51ac6ae5d\") " pod="openshift-cluster-node-tuning-operator/tuned-66xkf" Jul 12 17:53:15.022611 ip-10-0-129-128 kubenswrapper[2559]: I0712 17:53:15.021218 2559 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kubelet-config\" (UniqueName: \"kubernetes.io/host-path/6ab3daa1-3376-4f53-8106-d6d7417a327e-kubelet-config\") pod \"global-pull-secret-syncer-mpqvv\" (UID: \"6ab3daa1-3376-4f53-8106-d6d7417a327e\") " pod="kube-system/global-pull-secret-syncer-mpqvv" Jul 12 17:53:15.022611 ip-10-0-129-128 kubenswrapper[2559]: I0712 17:53:15.021219 2559 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"cnibin\" (UniqueName: \"kubernetes.io/host-path/370d4e7b-6713-48b2-81ee-f055fe1d5490-cnibin\") pod \"multus-additional-cni-plugins-9dh8l\" (UID: \"370d4e7b-6713-48b2-81ee-f055fe1d5490\") " pod="openshift-multus/multus-additional-cni-plugins-9dh8l" Jul 12 17:53:15.022611 ip-10-0-129-128 kubenswrapper[2559]: I0712 17:53:15.021243 2559 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-dhv25\" (UniqueName: \"kubernetes.io/projected/3e848887-84a9-4599-afc5-c3750c0857a0-kube-api-access-dhv25\") pod \"aws-ebs-csi-driver-node-m2r7q\" (UID: \"3e848887-84a9-4599-afc5-c3750c0857a0\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-m2r7q" Jul 12 17:53:15.022611 ip-10-0-129-128 kubenswrapper[2559]: I0712 17:53:15.021266 2559 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"etc-selinux\" (UniqueName: \"kubernetes.io/host-path/3e848887-84a9-4599-afc5-c3750c0857a0-etc-selinux\") pod \"aws-ebs-csi-driver-node-m2r7q\" (UID: \"3e848887-84a9-4599-afc5-c3750c0857a0\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-m2r7q" Jul 12 17:53:15.022611 ip-10-0-129-128 kubenswrapper[2559]: I0712 17:53:15.021286 2559 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-l54gm\" (UniqueName: \"kubernetes.io/projected/41eafd0c-4a0b-48f6-8e30-8e408412d389-kube-api-access-l54gm\") pod \"node-resolver-fqnvr\" (UID: \"41eafd0c-4a0b-48f6-8e30-8e408412d389\") " pod="openshift-dns/node-resolver-fqnvr" Jul 12 17:53:15.022611 ip-10-0-129-128 kubenswrapper[2559]: I0712 17:53:15.021365 2559 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"os-release\" (UniqueName: \"kubernetes.io/host-path/370d4e7b-6713-48b2-81ee-f055fe1d5490-os-release\") pod \"multus-additional-cni-plugins-9dh8l\" (UID: \"370d4e7b-6713-48b2-81ee-f055fe1d5490\") " pod="openshift-multus/multus-additional-cni-plugins-9dh8l" Jul 12 17:53:15.022611 ip-10-0-129-128 kubenswrapper[2559]: I0712 17:53:15.021402 2559 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"log-socket\" (UniqueName: \"kubernetes.io/host-path/e51bad50-bc74-4e38-8307-ca257b599314-log-socket\") pod \"ovnkube-node-q5dw8\" (UID: \"e51bad50-bc74-4e38-8307-ca257b599314\") " pod="openshift-ovn-kubernetes/ovnkube-node-q5dw8" Jul 12 17:53:15.022611 ip-10-0-129-128 kubenswrapper[2559]: I0712 17:53:15.021424 2559 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-cni-bin\" (UniqueName: \"kubernetes.io/host-path/e51bad50-bc74-4e38-8307-ca257b599314-host-cni-bin\") pod \"ovnkube-node-q5dw8\" (UID: \"e51bad50-bc74-4e38-8307-ca257b599314\") " pod="openshift-ovn-kubernetes/ovnkube-node-q5dw8" Jul 12 17:53:15.022611 ip-10-0-129-128 kubenswrapper[2559]: I0712 17:53:15.021445 2559 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host\" (UniqueName: \"kubernetes.io/host-path/cf65b059-f8a2-48d6-a773-74f51ac6ae5d-host\") pod \"tuned-66xkf\" (UID: \"cf65b059-f8a2-48d6-a773-74f51ac6ae5d\") " pod="openshift-cluster-node-tuning-operator/tuned-66xkf" Jul 12 17:53:15.022611 ip-10-0-129-128 kubenswrapper[2559]: I0712 17:53:15.021468 2559 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-var-lib-cni-bin\" (UniqueName: \"kubernetes.io/host-path/ff97f946-352d-4629-bf96-351edf53ebde-host-var-lib-cni-bin\") pod \"multus-c2jnt\" (UID: \"ff97f946-352d-4629-bf96-351edf53ebde\") " pod="openshift-multus/multus-c2jnt" Jul 12 17:53:15.022611 ip-10-0-129-128 kubenswrapper[2559]: I0712 17:53:15.021490 2559 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"dbus\" (UniqueName: \"kubernetes.io/host-path/6ab3daa1-3376-4f53-8106-d6d7417a327e-dbus\") pod \"global-pull-secret-syncer-mpqvv\" (UID: \"6ab3daa1-3376-4f53-8106-d6d7417a327e\") " pod="kube-system/global-pull-secret-syncer-mpqvv" Jul 12 17:53:15.022611 ip-10-0-129-128 kubenswrapper[2559]: I0712 17:53:15.021512 2559 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-run-netns\" (UniqueName: \"kubernetes.io/host-path/e51bad50-bc74-4e38-8307-ca257b599314-host-run-netns\") pod \"ovnkube-node-q5dw8\" (UID: \"e51bad50-bc74-4e38-8307-ca257b599314\") " pod="openshift-ovn-kubernetes/ovnkube-node-q5dw8" Jul 12 17:53:15.023111 ip-10-0-129-128 kubenswrapper[2559]: I0712 17:53:15.021517 2559 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"cni-binary-copy\" (UniqueName: \"kubernetes.io/configmap/370d4e7b-6713-48b2-81ee-f055fe1d5490-cni-binary-copy\") pod \"multus-additional-cni-plugins-9dh8l\" (UID: \"370d4e7b-6713-48b2-81ee-f055fe1d5490\") " pod="openshift-multus/multus-additional-cni-plugins-9dh8l" Jul 12 17:53:15.023111 ip-10-0-129-128 kubenswrapper[2559]: I0712 17:53:15.021539 2559 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"whereabouts-flatfile-configmap\" (UniqueName: \"kubernetes.io/configmap/370d4e7b-6713-48b2-81ee-f055fe1d5490-whereabouts-flatfile-configmap\") pod \"multus-additional-cni-plugins-9dh8l\" (UID: \"370d4e7b-6713-48b2-81ee-f055fe1d5490\") " pod="openshift-multus/multus-additional-cni-plugins-9dh8l" Jul 12 17:53:15.023111 ip-10-0-129-128 kubenswrapper[2559]: I0712 17:53:15.021566 2559 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"node-log\" (UniqueName: \"kubernetes.io/host-path/e51bad50-bc74-4e38-8307-ca257b599314-node-log\") pod \"ovnkube-node-q5dw8\" (UID: \"e51bad50-bc74-4e38-8307-ca257b599314\") " pod="openshift-ovn-kubernetes/ovnkube-node-q5dw8" Jul 12 17:53:15.023111 ip-10-0-129-128 kubenswrapper[2559]: I0712 17:53:15.021590 2559 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"var-lib-kubelet\" (UniqueName: \"kubernetes.io/host-path/cf65b059-f8a2-48d6-a773-74f51ac6ae5d-var-lib-kubelet\") pod \"tuned-66xkf\" (UID: \"cf65b059-f8a2-48d6-a773-74f51ac6ae5d\") " pod="openshift-cluster-node-tuning-operator/tuned-66xkf" Jul 12 17:53:15.023111 ip-10-0-129-128 kubenswrapper[2559]: I0712 17:53:15.021614 2559 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"hostroot\" (UniqueName: \"kubernetes.io/host-path/ff97f946-352d-4629-bf96-351edf53ebde-hostroot\") pod \"multus-c2jnt\" (UID: \"ff97f946-352d-4629-bf96-351edf53ebde\") " pod="openshift-multus/multus-c2jnt" Jul 12 17:53:15.023111 ip-10-0-129-128 kubenswrapper[2559]: I0712 17:53:15.021630 2559 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"os-release\" (UniqueName: \"kubernetes.io/host-path/370d4e7b-6713-48b2-81ee-f055fe1d5490-os-release\") pod \"multus-additional-cni-plugins-9dh8l\" (UID: \"370d4e7b-6713-48b2-81ee-f055fe1d5490\") " pod="openshift-multus/multus-additional-cni-plugins-9dh8l" Jul 12 17:53:15.023111 ip-10-0-129-128 kubenswrapper[2559]: I0712 17:53:15.021660 2559 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-26mv6\" (UniqueName: \"kubernetes.io/projected/6f65bad8-9ec7-4a51-8557-74b3f79ef76c-kube-api-access-26mv6\") pod \"network-metrics-daemon-qtkvt\" (UID: \"6f65bad8-9ec7-4a51-8557-74b3f79ef76c\") " pod="openshift-multus/network-metrics-daemon-qtkvt" Jul 12 17:53:15.023111 ip-10-0-129-128 kubenswrapper[2559]: I0712 17:53:15.021692 2559 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"serviceca\" (UniqueName: \"kubernetes.io/configmap/6d23c7c4-9eb6-483f-baf3-f25e641e2ead-serviceca\") pod \"node-ca-vxx7w\" (UID: \"6d23c7c4-9eb6-483f-baf3-f25e641e2ead\") " pod="openshift-image-registry/node-ca-vxx7w" Jul 12 17:53:15.023111 ip-10-0-129-128 kubenswrapper[2559]: I0712 17:53:15.021721 2559 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-mqz8x\" (UniqueName: \"kubernetes.io/projected/6d23c7c4-9eb6-483f-baf3-f25e641e2ead-kube-api-access-mqz8x\") pod \"node-ca-vxx7w\" (UID: \"6d23c7c4-9eb6-483f-baf3-f25e641e2ead\") " pod="openshift-image-registry/node-ca-vxx7w" Jul 12 17:53:15.023111 ip-10-0-129-128 kubenswrapper[2559]: I0712 17:53:15.021752 2559 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"systemd-units\" (UniqueName: \"kubernetes.io/host-path/e51bad50-bc74-4e38-8307-ca257b599314-systemd-units\") pod \"ovnkube-node-q5dw8\" (UID: \"e51bad50-bc74-4e38-8307-ca257b599314\") " pod="openshift-ovn-kubernetes/ovnkube-node-q5dw8" Jul 12 17:53:15.023111 ip-10-0-129-128 kubenswrapper[2559]: I0712 17:53:15.021768 2559 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"run-ovn\" (UniqueName: \"kubernetes.io/host-path/e51bad50-bc74-4e38-8307-ca257b599314-run-ovn\") pod \"ovnkube-node-q5dw8\" (UID: \"e51bad50-bc74-4e38-8307-ca257b599314\") " pod="openshift-ovn-kubernetes/ovnkube-node-q5dw8" Jul 12 17:53:15.023111 ip-10-0-129-128 kubenswrapper[2559]: I0712 17:53:15.021792 2559 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"system-cni-dir\" (UniqueName: \"kubernetes.io/host-path/ff97f946-352d-4629-bf96-351edf53ebde-system-cni-dir\") pod \"multus-c2jnt\" (UID: \"ff97f946-352d-4629-bf96-351edf53ebde\") " pod="openshift-multus/multus-c2jnt" Jul 12 17:53:15.023111 ip-10-0-129-128 kubenswrapper[2559]: I0712 17:53:15.021811 2559 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/6f65bad8-9ec7-4a51-8557-74b3f79ef76c-metrics-certs\") pod \"network-metrics-daemon-qtkvt\" (UID: \"6f65bad8-9ec7-4a51-8557-74b3f79ef76c\") " pod="openshift-multus/network-metrics-daemon-qtkvt" Jul 12 17:53:15.023111 ip-10-0-129-128 kubenswrapper[2559]: I0712 17:53:15.021825 2559 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"original-pull-secret\" (UniqueName: \"kubernetes.io/secret/6ab3daa1-3376-4f53-8106-d6d7417a327e-original-pull-secret\") pod \"global-pull-secret-syncer-mpqvv\" (UID: \"6ab3daa1-3376-4f53-8106-d6d7417a327e\") " pod="kube-system/global-pull-secret-syncer-mpqvv" Jul 12 17:53:15.023606 ip-10-0-129-128 kubenswrapper[2559]: I0712 17:53:15.021840 2559 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kubelet-dir\" (UniqueName: \"kubernetes.io/host-path/3e848887-84a9-4599-afc5-c3750c0857a0-kubelet-dir\") pod \"aws-ebs-csi-driver-node-m2r7q\" (UID: \"3e848887-84a9-4599-afc5-c3750c0857a0\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-m2r7q" Jul 12 17:53:15.023606 ip-10-0-129-128 kubenswrapper[2559]: I0712 17:53:15.021864 2559 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tuning-conf-dir\" (UniqueName: \"kubernetes.io/host-path/370d4e7b-6713-48b2-81ee-f055fe1d5490-tuning-conf-dir\") pod \"multus-additional-cni-plugins-9dh8l\" (UID: \"370d4e7b-6713-48b2-81ee-f055fe1d5490\") " pod="openshift-multus/multus-additional-cni-plugins-9dh8l" Jul 12 17:53:15.023606 ip-10-0-129-128 kubenswrapper[2559]: I0712 17:53:15.021887 2559 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"iptables-alerter-script\" (UniqueName: \"kubernetes.io/configmap/5841758e-4587-4d16-807c-b4a40b824c1b-iptables-alerter-script\") pod \"iptables-alerter-fql27\" (UID: \"5841758e-4587-4d16-807c-b4a40b824c1b\") " pod="openshift-network-operator/iptables-alerter-fql27" Jul 12 17:53:15.023606 ip-10-0-129-128 kubenswrapper[2559]: I0712 17:53:15.021904 2559 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"system-cni-dir\" (UniqueName: \"kubernetes.io/host-path/370d4e7b-6713-48b2-81ee-f055fe1d5490-system-cni-dir\") pod \"multus-additional-cni-plugins-9dh8l\" (UID: \"370d4e7b-6713-48b2-81ee-f055fe1d5490\") " pod="openshift-multus/multus-additional-cni-plugins-9dh8l" Jul 12 17:53:15.023606 ip-10-0-129-128 kubenswrapper[2559]: I0712 17:53:15.021919 2559 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"env-overrides\" (UniqueName: \"kubernetes.io/configmap/e51bad50-bc74-4e38-8307-ca257b599314-env-overrides\") pod \"ovnkube-node-q5dw8\" (UID: \"e51bad50-bc74-4e38-8307-ca257b599314\") " pod="openshift-ovn-kubernetes/ovnkube-node-q5dw8" Jul 12 17:53:15.023606 ip-10-0-129-128 kubenswrapper[2559]: I0712 17:53:15.021937 2559 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"etc-sysconfig\" (UniqueName: \"kubernetes.io/host-path/cf65b059-f8a2-48d6-a773-74f51ac6ae5d-etc-sysconfig\") pod \"tuned-66xkf\" (UID: \"cf65b059-f8a2-48d6-a773-74f51ac6ae5d\") " pod="openshift-cluster-node-tuning-operator/tuned-66xkf" Jul 12 17:53:15.023606 ip-10-0-129-128 kubenswrapper[2559]: I0712 17:53:15.021959 2559 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"etc-sysctl-conf\" (UniqueName: \"kubernetes.io/host-path/cf65b059-f8a2-48d6-a773-74f51ac6ae5d-etc-sysctl-conf\") pod \"tuned-66xkf\" (UID: \"cf65b059-f8a2-48d6-a773-74f51ac6ae5d\") " pod="openshift-cluster-node-tuning-operator/tuned-66xkf" Jul 12 17:53:15.023606 ip-10-0-129-128 kubenswrapper[2559]: I0712 17:53:15.021988 2559 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"lib-modules\" (UniqueName: \"kubernetes.io/host-path/cf65b059-f8a2-48d6-a773-74f51ac6ae5d-lib-modules\") pod \"tuned-66xkf\" (UID: \"cf65b059-f8a2-48d6-a773-74f51ac6ae5d\") " pod="openshift-cluster-node-tuning-operator/tuned-66xkf" Jul 12 17:53:15.023606 ip-10-0-129-128 kubenswrapper[2559]: I0712 17:53:15.022010 2559 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-q7sxg\" (UniqueName: \"kubernetes.io/projected/cf65b059-f8a2-48d6-a773-74f51ac6ae5d-kube-api-access-q7sxg\") pod \"tuned-66xkf\" (UID: \"cf65b059-f8a2-48d6-a773-74f51ac6ae5d\") " pod="openshift-cluster-node-tuning-operator/tuned-66xkf" Jul 12 17:53:15.023606 ip-10-0-129-128 kubenswrapper[2559]: I0712 17:53:15.022035 2559 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"cni-sysctl-allowlist\" (UniqueName: \"kubernetes.io/configmap/370d4e7b-6713-48b2-81ee-f055fe1d5490-cni-sysctl-allowlist\") pod \"multus-additional-cni-plugins-9dh8l\" (UID: \"370d4e7b-6713-48b2-81ee-f055fe1d5490\") " pod="openshift-multus/multus-additional-cni-plugins-9dh8l" Jul 12 17:53:15.023606 ip-10-0-129-128 kubenswrapper[2559]: I0712 17:53:15.022059 2559 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"sys\" (UniqueName: \"kubernetes.io/host-path/cf65b059-f8a2-48d6-a773-74f51ac6ae5d-sys\") pod \"tuned-66xkf\" (UID: \"cf65b059-f8a2-48d6-a773-74f51ac6ae5d\") " pod="openshift-cluster-node-tuning-operator/tuned-66xkf" Jul 12 17:53:15.023606 ip-10-0-129-128 kubenswrapper[2559]: I0712 17:53:15.022082 2559 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"etc-openvswitch\" (UniqueName: \"kubernetes.io/host-path/e51bad50-bc74-4e38-8307-ca257b599314-etc-openvswitch\") pod \"ovnkube-node-q5dw8\" (UID: \"e51bad50-bc74-4e38-8307-ca257b599314\") " pod="openshift-ovn-kubernetes/ovnkube-node-q5dw8" Jul 12 17:53:15.023606 ip-10-0-129-128 kubenswrapper[2559]: I0712 17:53:15.022107 2559 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"ovn-node-metrics-cert\" (UniqueName: \"kubernetes.io/secret/e51bad50-bc74-4e38-8307-ca257b599314-ovn-node-metrics-cert\") pod \"ovnkube-node-q5dw8\" (UID: \"e51bad50-bc74-4e38-8307-ca257b599314\") " pod="openshift-ovn-kubernetes/ovnkube-node-q5dw8" Jul 12 17:53:15.024178 ip-10-0-129-128 kubenswrapper[2559]: I0712 17:53:15.022107 2559 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"whereabouts-flatfile-configmap\" (UniqueName: \"kubernetes.io/configmap/370d4e7b-6713-48b2-81ee-f055fe1d5490-whereabouts-flatfile-configmap\") pod \"multus-additional-cni-plugins-9dh8l\" (UID: \"370d4e7b-6713-48b2-81ee-f055fe1d5490\") " pod="openshift-multus/multus-additional-cni-plugins-9dh8l" Jul 12 17:53:15.024178 ip-10-0-129-128 kubenswrapper[2559]: I0712 17:53:15.022131 2559 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"cnibin\" (UniqueName: \"kubernetes.io/host-path/ff97f946-352d-4629-bf96-351edf53ebde-cnibin\") pod \"multus-c2jnt\" (UID: \"ff97f946-352d-4629-bf96-351edf53ebde\") " pod="openshift-multus/multus-c2jnt" Jul 12 17:53:15.024178 ip-10-0-129-128 kubenswrapper[2559]: I0712 17:53:15.022155 2559 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"multus-conf-dir\" (UniqueName: \"kubernetes.io/host-path/ff97f946-352d-4629-bf96-351edf53ebde-multus-conf-dir\") pod \"multus-c2jnt\" (UID: \"ff97f946-352d-4629-bf96-351edf53ebde\") " pod="openshift-multus/multus-c2jnt" Jul 12 17:53:15.024178 ip-10-0-129-128 kubenswrapper[2559]: I0712 17:53:15.022180 2559 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-run-multus-certs\" (UniqueName: \"kubernetes.io/host-path/ff97f946-352d-4629-bf96-351edf53ebde-host-run-multus-certs\") pod \"multus-c2jnt\" (UID: \"ff97f946-352d-4629-bf96-351edf53ebde\") " pod="openshift-multus/multus-c2jnt" Jul 12 17:53:15.024178 ip-10-0-129-128 kubenswrapper[2559]: I0712 17:53:15.022178 2559 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"system-cni-dir\" (UniqueName: \"kubernetes.io/host-path/370d4e7b-6713-48b2-81ee-f055fe1d5490-system-cni-dir\") pod \"multus-additional-cni-plugins-9dh8l\" (UID: \"370d4e7b-6713-48b2-81ee-f055fe1d5490\") " pod="openshift-multus/multus-additional-cni-plugins-9dh8l" Jul 12 17:53:15.024178 ip-10-0-129-128 kubenswrapper[2559]: I0712 17:53:15.022218 2559 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"etc-modprobe-d\" (UniqueName: \"kubernetes.io/host-path/cf65b059-f8a2-48d6-a773-74f51ac6ae5d-etc-modprobe-d\") pod \"tuned-66xkf\" (UID: \"cf65b059-f8a2-48d6-a773-74f51ac6ae5d\") " pod="openshift-cluster-node-tuning-operator/tuned-66xkf" Jul 12 17:53:15.024178 ip-10-0-129-128 kubenswrapper[2559]: I0712 17:53:15.022280 2559 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"device-dir\" (UniqueName: \"kubernetes.io/host-path/3e848887-84a9-4599-afc5-c3750c0857a0-device-dir\") pod \"aws-ebs-csi-driver-node-m2r7q\" (UID: \"3e848887-84a9-4599-afc5-c3750c0857a0\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-m2r7q" Jul 12 17:53:15.024178 ip-10-0-129-128 kubenswrapper[2559]: I0712 17:53:15.022413 2559 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"iptables-alerter-script\" (UniqueName: \"kubernetes.io/configmap/5841758e-4587-4d16-807c-b4a40b824c1b-iptables-alerter-script\") pod \"iptables-alerter-fql27\" (UID: \"5841758e-4587-4d16-807c-b4a40b824c1b\") " pod="openshift-network-operator/iptables-alerter-fql27" Jul 12 17:53:15.024178 ip-10-0-129-128 kubenswrapper[2559]: I0712 17:53:15.022582 2559 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"tuning-conf-dir\" (UniqueName: \"kubernetes.io/host-path/370d4e7b-6713-48b2-81ee-f055fe1d5490-tuning-conf-dir\") pod \"multus-additional-cni-plugins-9dh8l\" (UID: \"370d4e7b-6713-48b2-81ee-f055fe1d5490\") " pod="openshift-multus/multus-additional-cni-plugins-9dh8l" Jul 12 17:53:15.024178 ip-10-0-129-128 kubenswrapper[2559]: I0712 17:53:15.022699 2559 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"cni-sysctl-allowlist\" (UniqueName: \"kubernetes.io/configmap/370d4e7b-6713-48b2-81ee-f055fe1d5490-cni-sysctl-allowlist\") pod \"multus-additional-cni-plugins-9dh8l\" (UID: \"370d4e7b-6713-48b2-81ee-f055fe1d5490\") " pod="openshift-multus/multus-additional-cni-plugins-9dh8l" Jul 12 17:53:15.027253 ip-10-0-129-128 kubenswrapper[2559]: I0712 17:53:15.027228 2559 swap_util.go:74] "error creating dir to test if tmpfs noswap is enabled. Assuming not supported" mount path="" error="stat /var/lib/kubelet/plugins/kubernetes.io/empty-dir: no such file or directory" Jul 12 17:53:15.031494 ip-10-0-129-128 kubenswrapper[2559]: I0712 17:53:15.031446 2559 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-vx2tt\" (UniqueName: \"kubernetes.io/projected/5841758e-4587-4d16-807c-b4a40b824c1b-kube-api-access-vx2tt\") pod \"iptables-alerter-fql27\" (UID: \"5841758e-4587-4d16-807c-b4a40b824c1b\") " pod="openshift-network-operator/iptables-alerter-fql27" Jul 12 17:53:15.031595 ip-10-0-129-128 kubenswrapper[2559]: I0712 17:53:15.031494 2559 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-zz8nf\" (UniqueName: \"kubernetes.io/projected/370d4e7b-6713-48b2-81ee-f055fe1d5490-kube-api-access-zz8nf\") pod \"multus-additional-cni-plugins-9dh8l\" (UID: \"370d4e7b-6713-48b2-81ee-f055fe1d5490\") " pod="openshift-multus/multus-additional-cni-plugins-9dh8l" Jul 12 17:53:15.032500 ip-10-0-129-128 kubenswrapper[2559]: I0712 17:53:15.032478 2559 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-l54gm\" (UniqueName: \"kubernetes.io/projected/41eafd0c-4a0b-48f6-8e30-8e408412d389-kube-api-access-l54gm\") pod \"node-resolver-fqnvr\" (UID: \"41eafd0c-4a0b-48f6-8e30-8e408412d389\") " pod="openshift-dns/node-resolver-fqnvr" Jul 12 17:53:15.033451 ip-10-0-129-128 kubenswrapper[2559]: I0712 17:53:15.033405 2559 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-129-128.ec2.internal" event={"ID":"862cce2888b36ce6c9c6601a2547ab41","Type":"ContainerStarted","Data":"60ac41c3b6c2ebc2c09802f521dae9426aaab25d711049f38d311b23926f97b8"} Jul 12 17:53:15.034891 ip-10-0-129-128 kubenswrapper[2559]: I0712 17:53:15.034868 2559 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kube-system/kube-apiserver-proxy-ip-10-0-129-128.ec2.internal" event={"ID":"89625644b4c452349901306bb1be9b95","Type":"ContainerStarted","Data":"b21ddbce83ff6ecccf906e84f76462a21d3f412e75be1f0fb292e1cd9101fdbc"} Jul 12 17:53:15.122829 ip-10-0-129-128 kubenswrapper[2559]: I0712 17:53:15.122790 2559 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"original-pull-secret\" (UniqueName: \"kubernetes.io/secret/6ab3daa1-3376-4f53-8106-d6d7417a327e-original-pull-secret\") pod \"global-pull-secret-syncer-mpqvv\" (UID: \"6ab3daa1-3376-4f53-8106-d6d7417a327e\") " pod="kube-system/global-pull-secret-syncer-mpqvv" Jul 12 17:53:15.122976 ip-10-0-129-128 kubenswrapper[2559]: I0712 17:53:15.122838 2559 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kubelet-dir\" (UniqueName: \"kubernetes.io/host-path/3e848887-84a9-4599-afc5-c3750c0857a0-kubelet-dir\") pod \"aws-ebs-csi-driver-node-m2r7q\" (UID: \"3e848887-84a9-4599-afc5-c3750c0857a0\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-m2r7q" Jul 12 17:53:15.122976 ip-10-0-129-128 kubenswrapper[2559]: I0712 17:53:15.122867 2559 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"env-overrides\" (UniqueName: \"kubernetes.io/configmap/e51bad50-bc74-4e38-8307-ca257b599314-env-overrides\") pod \"ovnkube-node-q5dw8\" (UID: \"e51bad50-bc74-4e38-8307-ca257b599314\") " pod="openshift-ovn-kubernetes/ovnkube-node-q5dw8" Jul 12 17:53:15.122976 ip-10-0-129-128 kubenswrapper[2559]: I0712 17:53:15.122891 2559 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"etc-sysconfig\" (UniqueName: \"kubernetes.io/host-path/cf65b059-f8a2-48d6-a773-74f51ac6ae5d-etc-sysconfig\") pod \"tuned-66xkf\" (UID: \"cf65b059-f8a2-48d6-a773-74f51ac6ae5d\") " pod="openshift-cluster-node-tuning-operator/tuned-66xkf" Jul 12 17:53:15.122976 ip-10-0-129-128 kubenswrapper[2559]: I0712 17:53:15.122913 2559 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"etc-sysctl-conf\" (UniqueName: \"kubernetes.io/host-path/cf65b059-f8a2-48d6-a773-74f51ac6ae5d-etc-sysctl-conf\") pod \"tuned-66xkf\" (UID: \"cf65b059-f8a2-48d6-a773-74f51ac6ae5d\") " pod="openshift-cluster-node-tuning-operator/tuned-66xkf" Jul 12 17:53:15.122976 ip-10-0-129-128 kubenswrapper[2559]: I0712 17:53:15.122934 2559 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"lib-modules\" (UniqueName: \"kubernetes.io/host-path/cf65b059-f8a2-48d6-a773-74f51ac6ae5d-lib-modules\") pod \"tuned-66xkf\" (UID: \"cf65b059-f8a2-48d6-a773-74f51ac6ae5d\") " pod="openshift-cluster-node-tuning-operator/tuned-66xkf" Jul 12 17:53:15.122976 ip-10-0-129-128 kubenswrapper[2559]: E0712 17:53:15.122943 2559 secret.go:189] Couldn't get secret kube-system/original-pull-secret: object "kube-system"/"original-pull-secret" not registered Jul 12 17:53:15.123271 ip-10-0-129-128 kubenswrapper[2559]: I0712 17:53:15.123012 2559 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kubelet-dir\" (UniqueName: \"kubernetes.io/host-path/3e848887-84a9-4599-afc5-c3750c0857a0-kubelet-dir\") pod \"aws-ebs-csi-driver-node-m2r7q\" (UID: \"3e848887-84a9-4599-afc5-c3750c0857a0\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-m2r7q" Jul 12 17:53:15.123271 ip-10-0-129-128 kubenswrapper[2559]: E0712 17:53:15.123039 2559 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/6ab3daa1-3376-4f53-8106-d6d7417a327e-original-pull-secret podName:6ab3daa1-3376-4f53-8106-d6d7417a327e nodeName:}" failed. No retries permitted until 2026-07-12 17:53:15.62300125 +0000 UTC m=+3.117069172 (durationBeforeRetry 500ms). Error: MountVolume.SetUp failed for volume "original-pull-secret" (UniqueName: "kubernetes.io/secret/6ab3daa1-3376-4f53-8106-d6d7417a327e-original-pull-secret") pod "global-pull-secret-syncer-mpqvv" (UID: "6ab3daa1-3376-4f53-8106-d6d7417a327e") : object "kube-system"/"original-pull-secret" not registered Jul 12 17:53:15.123271 ip-10-0-129-128 kubenswrapper[2559]: I0712 17:53:15.123069 2559 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"lib-modules\" (UniqueName: \"kubernetes.io/host-path/cf65b059-f8a2-48d6-a773-74f51ac6ae5d-lib-modules\") pod \"tuned-66xkf\" (UID: \"cf65b059-f8a2-48d6-a773-74f51ac6ae5d\") " pod="openshift-cluster-node-tuning-operator/tuned-66xkf" Jul 12 17:53:15.123271 ip-10-0-129-128 kubenswrapper[2559]: I0712 17:53:15.123098 2559 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"etc-sysconfig\" (UniqueName: \"kubernetes.io/host-path/cf65b059-f8a2-48d6-a773-74f51ac6ae5d-etc-sysconfig\") pod \"tuned-66xkf\" (UID: \"cf65b059-f8a2-48d6-a773-74f51ac6ae5d\") " pod="openshift-cluster-node-tuning-operator/tuned-66xkf" Jul 12 17:53:15.123271 ip-10-0-129-128 kubenswrapper[2559]: I0712 17:53:15.123105 2559 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-q7sxg\" (UniqueName: \"kubernetes.io/projected/cf65b059-f8a2-48d6-a773-74f51ac6ae5d-kube-api-access-q7sxg\") pod \"tuned-66xkf\" (UID: \"cf65b059-f8a2-48d6-a773-74f51ac6ae5d\") " pod="openshift-cluster-node-tuning-operator/tuned-66xkf" Jul 12 17:53:15.123271 ip-10-0-129-128 kubenswrapper[2559]: I0712 17:53:15.123155 2559 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"sys\" (UniqueName: \"kubernetes.io/host-path/cf65b059-f8a2-48d6-a773-74f51ac6ae5d-sys\") pod \"tuned-66xkf\" (UID: \"cf65b059-f8a2-48d6-a773-74f51ac6ae5d\") " pod="openshift-cluster-node-tuning-operator/tuned-66xkf" Jul 12 17:53:15.123271 ip-10-0-129-128 kubenswrapper[2559]: I0712 17:53:15.123173 2559 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"etc-openvswitch\" (UniqueName: \"kubernetes.io/host-path/e51bad50-bc74-4e38-8307-ca257b599314-etc-openvswitch\") pod \"ovnkube-node-q5dw8\" (UID: \"e51bad50-bc74-4e38-8307-ca257b599314\") " pod="openshift-ovn-kubernetes/ovnkube-node-q5dw8" Jul 12 17:53:15.123271 ip-10-0-129-128 kubenswrapper[2559]: I0712 17:53:15.123185 2559 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"etc-sysctl-conf\" (UniqueName: \"kubernetes.io/host-path/cf65b059-f8a2-48d6-a773-74f51ac6ae5d-etc-sysctl-conf\") pod \"tuned-66xkf\" (UID: \"cf65b059-f8a2-48d6-a773-74f51ac6ae5d\") " pod="openshift-cluster-node-tuning-operator/tuned-66xkf" Jul 12 17:53:15.123271 ip-10-0-129-128 kubenswrapper[2559]: I0712 17:53:15.123196 2559 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"ovn-node-metrics-cert\" (UniqueName: \"kubernetes.io/secret/e51bad50-bc74-4e38-8307-ca257b599314-ovn-node-metrics-cert\") pod \"ovnkube-node-q5dw8\" (UID: \"e51bad50-bc74-4e38-8307-ca257b599314\") " pod="openshift-ovn-kubernetes/ovnkube-node-q5dw8" Jul 12 17:53:15.123271 ip-10-0-129-128 kubenswrapper[2559]: I0712 17:53:15.123223 2559 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"cnibin\" (UniqueName: \"kubernetes.io/host-path/ff97f946-352d-4629-bf96-351edf53ebde-cnibin\") pod \"multus-c2jnt\" (UID: \"ff97f946-352d-4629-bf96-351edf53ebde\") " pod="openshift-multus/multus-c2jnt" Jul 12 17:53:15.123271 ip-10-0-129-128 kubenswrapper[2559]: I0712 17:53:15.123224 2559 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"sys\" (UniqueName: \"kubernetes.io/host-path/cf65b059-f8a2-48d6-a773-74f51ac6ae5d-sys\") pod \"tuned-66xkf\" (UID: \"cf65b059-f8a2-48d6-a773-74f51ac6ae5d\") " pod="openshift-cluster-node-tuning-operator/tuned-66xkf" Jul 12 17:53:15.123271 ip-10-0-129-128 kubenswrapper[2559]: I0712 17:53:15.123233 2559 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"etc-openvswitch\" (UniqueName: \"kubernetes.io/host-path/e51bad50-bc74-4e38-8307-ca257b599314-etc-openvswitch\") pod \"ovnkube-node-q5dw8\" (UID: \"e51bad50-bc74-4e38-8307-ca257b599314\") " pod="openshift-ovn-kubernetes/ovnkube-node-q5dw8" Jul 12 17:53:15.123271 ip-10-0-129-128 kubenswrapper[2559]: I0712 17:53:15.123245 2559 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"multus-conf-dir\" (UniqueName: \"kubernetes.io/host-path/ff97f946-352d-4629-bf96-351edf53ebde-multus-conf-dir\") pod \"multus-c2jnt\" (UID: \"ff97f946-352d-4629-bf96-351edf53ebde\") " pod="openshift-multus/multus-c2jnt" Jul 12 17:53:15.123271 ip-10-0-129-128 kubenswrapper[2559]: I0712 17:53:15.123269 2559 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-run-multus-certs\" (UniqueName: \"kubernetes.io/host-path/ff97f946-352d-4629-bf96-351edf53ebde-host-run-multus-certs\") pod \"multus-c2jnt\" (UID: \"ff97f946-352d-4629-bf96-351edf53ebde\") " pod="openshift-multus/multus-c2jnt" Jul 12 17:53:15.123891 ip-10-0-129-128 kubenswrapper[2559]: I0712 17:53:15.123275 2559 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"cnibin\" (UniqueName: \"kubernetes.io/host-path/ff97f946-352d-4629-bf96-351edf53ebde-cnibin\") pod \"multus-c2jnt\" (UID: \"ff97f946-352d-4629-bf96-351edf53ebde\") " pod="openshift-multus/multus-c2jnt" Jul 12 17:53:15.123891 ip-10-0-129-128 kubenswrapper[2559]: I0712 17:53:15.123303 2559 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"etc-modprobe-d\" (UniqueName: \"kubernetes.io/host-path/cf65b059-f8a2-48d6-a773-74f51ac6ae5d-etc-modprobe-d\") pod \"tuned-66xkf\" (UID: \"cf65b059-f8a2-48d6-a773-74f51ac6ae5d\") " pod="openshift-cluster-node-tuning-operator/tuned-66xkf" Jul 12 17:53:15.123891 ip-10-0-129-128 kubenswrapper[2559]: I0712 17:53:15.123329 2559 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"device-dir\" (UniqueName: \"kubernetes.io/host-path/3e848887-84a9-4599-afc5-c3750c0857a0-device-dir\") pod \"aws-ebs-csi-driver-node-m2r7q\" (UID: \"3e848887-84a9-4599-afc5-c3750c0857a0\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-m2r7q" Jul 12 17:53:15.123891 ip-10-0-129-128 kubenswrapper[2559]: I0712 17:53:15.123352 2559 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-kubelet\" (UniqueName: \"kubernetes.io/host-path/e51bad50-bc74-4e38-8307-ca257b599314-host-kubelet\") pod \"ovnkube-node-q5dw8\" (UID: \"e51bad50-bc74-4e38-8307-ca257b599314\") " pod="openshift-ovn-kubernetes/ovnkube-node-q5dw8" Jul 12 17:53:15.123891 ip-10-0-129-128 kubenswrapper[2559]: I0712 17:53:15.123341 2559 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"host-run-multus-certs\" (UniqueName: \"kubernetes.io/host-path/ff97f946-352d-4629-bf96-351edf53ebde-host-run-multus-certs\") pod \"multus-c2jnt\" (UID: \"ff97f946-352d-4629-bf96-351edf53ebde\") " pod="openshift-multus/multus-c2jnt" Jul 12 17:53:15.123891 ip-10-0-129-128 kubenswrapper[2559]: I0712 17:53:15.123391 2559 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"multus-conf-dir\" (UniqueName: \"kubernetes.io/host-path/ff97f946-352d-4629-bf96-351edf53ebde-multus-conf-dir\") pod \"multus-c2jnt\" (UID: \"ff97f946-352d-4629-bf96-351edf53ebde\") " pod="openshift-multus/multus-c2jnt" Jul 12 17:53:15.123891 ip-10-0-129-128 kubenswrapper[2559]: I0712 17:53:15.123409 2559 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"multus-daemon-config\" (UniqueName: \"kubernetes.io/configmap/ff97f946-352d-4629-bf96-351edf53ebde-multus-daemon-config\") pod \"multus-c2jnt\" (UID: \"ff97f946-352d-4629-bf96-351edf53ebde\") " pod="openshift-multus/multus-c2jnt" Jul 12 17:53:15.123891 ip-10-0-129-128 kubenswrapper[2559]: I0712 17:53:15.123407 2559 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"host-kubelet\" (UniqueName: \"kubernetes.io/host-path/e51bad50-bc74-4e38-8307-ca257b599314-host-kubelet\") pod \"ovnkube-node-q5dw8\" (UID: \"e51bad50-bc74-4e38-8307-ca257b599314\") " pod="openshift-ovn-kubernetes/ovnkube-node-q5dw8" Jul 12 17:53:15.123891 ip-10-0-129-128 kubenswrapper[2559]: I0712 17:53:15.123432 2559 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-bn2rx\" (UniqueName: \"kubernetes.io/projected/e2c3a206-19c0-4d4d-bb98-a99190023ad4-kube-api-access-bn2rx\") pod \"network-check-target-h4bgw\" (UID: \"e2c3a206-19c0-4d4d-bb98-a99190023ad4\") " pod="openshift-network-diagnostics/network-check-target-h4bgw" Jul 12 17:53:15.123891 ip-10-0-129-128 kubenswrapper[2559]: I0712 17:53:15.123494 2559 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"etc-modprobe-d\" (UniqueName: \"kubernetes.io/host-path/cf65b059-f8a2-48d6-a773-74f51ac6ae5d-etc-modprobe-d\") pod \"tuned-66xkf\" (UID: \"cf65b059-f8a2-48d6-a773-74f51ac6ae5d\") " pod="openshift-cluster-node-tuning-operator/tuned-66xkf" Jul 12 17:53:15.123891 ip-10-0-129-128 kubenswrapper[2559]: I0712 17:53:15.123517 2559 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"device-dir\" (UniqueName: \"kubernetes.io/host-path/3e848887-84a9-4599-afc5-c3750c0857a0-device-dir\") pod \"aws-ebs-csi-driver-node-m2r7q\" (UID: \"3e848887-84a9-4599-afc5-c3750c0857a0\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-m2r7q" Jul 12 17:53:15.123891 ip-10-0-129-128 kubenswrapper[2559]: I0712 17:53:15.123749 2559 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"multus-cni-dir\" (UniqueName: \"kubernetes.io/host-path/ff97f946-352d-4629-bf96-351edf53ebde-multus-cni-dir\") pod \"multus-c2jnt\" (UID: \"ff97f946-352d-4629-bf96-351edf53ebde\") " pod="openshift-multus/multus-c2jnt" Jul 12 17:53:15.123891 ip-10-0-129-128 kubenswrapper[2559]: I0712 17:53:15.123782 2559 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"ovnkube-script-lib\" (UniqueName: \"kubernetes.io/configmap/e51bad50-bc74-4e38-8307-ca257b599314-ovnkube-script-lib\") pod \"ovnkube-node-q5dw8\" (UID: \"e51bad50-bc74-4e38-8307-ca257b599314\") " pod="openshift-ovn-kubernetes/ovnkube-node-q5dw8" Jul 12 17:53:15.123891 ip-10-0-129-128 kubenswrapper[2559]: I0712 17:53:15.123804 2559 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"os-release\" (UniqueName: \"kubernetes.io/host-path/ff97f946-352d-4629-bf96-351edf53ebde-os-release\") pod \"multus-c2jnt\" (UID: \"ff97f946-352d-4629-bf96-351edf53ebde\") " pod="openshift-multus/multus-c2jnt" Jul 12 17:53:15.123891 ip-10-0-129-128 kubenswrapper[2559]: I0712 17:53:15.123825 2559 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"cni-binary-copy\" (UniqueName: \"kubernetes.io/configmap/ff97f946-352d-4629-bf96-351edf53ebde-cni-binary-copy\") pod \"multus-c2jnt\" (UID: \"ff97f946-352d-4629-bf96-351edf53ebde\") " pod="openshift-multus/multus-c2jnt" Jul 12 17:53:15.124572 ip-10-0-129-128 kubenswrapper[2559]: I0712 17:53:15.123849 2559 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"multus-socket-dir-parent\" (UniqueName: \"kubernetes.io/host-path/ff97f946-352d-4629-bf96-351edf53ebde-multus-socket-dir-parent\") pod \"multus-c2jnt\" (UID: \"ff97f946-352d-4629-bf96-351edf53ebde\") " pod="openshift-multus/multus-c2jnt" Jul 12 17:53:15.124572 ip-10-0-129-128 kubenswrapper[2559]: I0712 17:53:15.123871 2559 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"etc-kubernetes\" (UniqueName: \"kubernetes.io/host-path/ff97f946-352d-4629-bf96-351edf53ebde-etc-kubernetes\") pod \"multus-c2jnt\" (UID: \"ff97f946-352d-4629-bf96-351edf53ebde\") " pod="openshift-multus/multus-c2jnt" Jul 12 17:53:15.124572 ip-10-0-129-128 kubenswrapper[2559]: I0712 17:53:15.123873 2559 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"env-overrides\" (UniqueName: \"kubernetes.io/configmap/e51bad50-bc74-4e38-8307-ca257b599314-env-overrides\") pod \"ovnkube-node-q5dw8\" (UID: \"e51bad50-bc74-4e38-8307-ca257b599314\") " pod="openshift-ovn-kubernetes/ovnkube-node-q5dw8" Jul 12 17:53:15.124572 ip-10-0-129-128 kubenswrapper[2559]: I0712 17:53:15.123893 2559 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"run-openvswitch\" (UniqueName: \"kubernetes.io/host-path/e51bad50-bc74-4e38-8307-ca257b599314-run-openvswitch\") pod \"ovnkube-node-q5dw8\" (UID: \"e51bad50-bc74-4e38-8307-ca257b599314\") " pod="openshift-ovn-kubernetes/ovnkube-node-q5dw8" Jul 12 17:53:15.124572 ip-10-0-129-128 kubenswrapper[2559]: I0712 17:53:15.123938 2559 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"run-openvswitch\" (UniqueName: \"kubernetes.io/host-path/e51bad50-bc74-4e38-8307-ca257b599314-run-openvswitch\") pod \"ovnkube-node-q5dw8\" (UID: \"e51bad50-bc74-4e38-8307-ca257b599314\") " pod="openshift-ovn-kubernetes/ovnkube-node-q5dw8" Jul 12 17:53:15.124572 ip-10-0-129-128 kubenswrapper[2559]: I0712 17:53:15.123944 2559 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"multus-cni-dir\" (UniqueName: \"kubernetes.io/host-path/ff97f946-352d-4629-bf96-351edf53ebde-multus-cni-dir\") pod \"multus-c2jnt\" (UID: \"ff97f946-352d-4629-bf96-351edf53ebde\") " pod="openshift-multus/multus-c2jnt" Jul 12 17:53:15.124572 ip-10-0-129-128 kubenswrapper[2559]: I0712 17:53:15.123940 2559 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-slash\" (UniqueName: \"kubernetes.io/host-path/e51bad50-bc74-4e38-8307-ca257b599314-host-slash\") pod \"ovnkube-node-q5dw8\" (UID: \"e51bad50-bc74-4e38-8307-ca257b599314\") " pod="openshift-ovn-kubernetes/ovnkube-node-q5dw8" Jul 12 17:53:15.124572 ip-10-0-129-128 kubenswrapper[2559]: I0712 17:53:15.123974 2559 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"host-slash\" (UniqueName: \"kubernetes.io/host-path/e51bad50-bc74-4e38-8307-ca257b599314-host-slash\") pod \"ovnkube-node-q5dw8\" (UID: \"e51bad50-bc74-4e38-8307-ca257b599314\") " pod="openshift-ovn-kubernetes/ovnkube-node-q5dw8" Jul 12 17:53:15.124572 ip-10-0-129-128 kubenswrapper[2559]: I0712 17:53:15.123976 2559 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-p9sff\" (UniqueName: \"kubernetes.io/projected/e51bad50-bc74-4e38-8307-ca257b599314-kube-api-access-p9sff\") pod \"ovnkube-node-q5dw8\" (UID: \"e51bad50-bc74-4e38-8307-ca257b599314\") " pod="openshift-ovn-kubernetes/ovnkube-node-q5dw8" Jul 12 17:53:15.124572 ip-10-0-129-128 kubenswrapper[2559]: I0712 17:53:15.124012 2559 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-var-lib-cni-multus\" (UniqueName: \"kubernetes.io/host-path/ff97f946-352d-4629-bf96-351edf53ebde-host-var-lib-cni-multus\") pod \"multus-c2jnt\" (UID: \"ff97f946-352d-4629-bf96-351edf53ebde\") " pod="openshift-multus/multus-c2jnt" Jul 12 17:53:15.124572 ip-10-0-129-128 kubenswrapper[2559]: I0712 17:53:15.124035 2559 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-2wqhw\" (UniqueName: \"kubernetes.io/projected/ff97f946-352d-4629-bf96-351edf53ebde-kube-api-access-2wqhw\") pod \"multus-c2jnt\" (UID: \"ff97f946-352d-4629-bf96-351edf53ebde\") " pod="openshift-multus/multus-c2jnt" Jul 12 17:53:15.124572 ip-10-0-129-128 kubenswrapper[2559]: I0712 17:53:15.124058 2559 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"sys-fs\" (UniqueName: \"kubernetes.io/host-path/3e848887-84a9-4599-afc5-c3750c0857a0-sys-fs\") pod \"aws-ebs-csi-driver-node-m2r7q\" (UID: \"3e848887-84a9-4599-afc5-c3750c0857a0\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-m2r7q" Jul 12 17:53:15.124572 ip-10-0-129-128 kubenswrapper[2559]: I0712 17:53:15.124078 2559 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"run-systemd\" (UniqueName: \"kubernetes.io/host-path/e51bad50-bc74-4e38-8307-ca257b599314-run-systemd\") pod \"ovnkube-node-q5dw8\" (UID: \"e51bad50-bc74-4e38-8307-ca257b599314\") " pod="openshift-ovn-kubernetes/ovnkube-node-q5dw8" Jul 12 17:53:15.124572 ip-10-0-129-128 kubenswrapper[2559]: I0712 17:53:15.124100 2559 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-run-k8s-cni-cncf-io\" (UniqueName: \"kubernetes.io/host-path/ff97f946-352d-4629-bf96-351edf53ebde-host-run-k8s-cni-cncf-io\") pod \"multus-c2jnt\" (UID: \"ff97f946-352d-4629-bf96-351edf53ebde\") " pod="openshift-multus/multus-c2jnt" Jul 12 17:53:15.124572 ip-10-0-129-128 kubenswrapper[2559]: I0712 17:53:15.124124 2559 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-var-lib-kubelet\" (UniqueName: \"kubernetes.io/host-path/ff97f946-352d-4629-bf96-351edf53ebde-host-var-lib-kubelet\") pod \"multus-c2jnt\" (UID: \"ff97f946-352d-4629-bf96-351edf53ebde\") " pod="openshift-multus/multus-c2jnt" Jul 12 17:53:15.125334 ip-10-0-129-128 kubenswrapper[2559]: I0712 17:53:15.124153 2559 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"socket-dir\" (UniqueName: \"kubernetes.io/host-path/3e848887-84a9-4599-afc5-c3750c0857a0-socket-dir\") pod \"aws-ebs-csi-driver-node-m2r7q\" (UID: \"3e848887-84a9-4599-afc5-c3750c0857a0\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-m2r7q" Jul 12 17:53:15.125334 ip-10-0-129-128 kubenswrapper[2559]: I0712 17:53:15.124177 2559 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"registration-dir\" (UniqueName: \"kubernetes.io/host-path/3e848887-84a9-4599-afc5-c3750c0857a0-registration-dir\") pod \"aws-ebs-csi-driver-node-m2r7q\" (UID: \"3e848887-84a9-4599-afc5-c3750c0857a0\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-m2r7q" Jul 12 17:53:15.125334 ip-10-0-129-128 kubenswrapper[2559]: I0712 17:53:15.124201 2559 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"konnectivity-ca\" (UniqueName: \"kubernetes.io/configmap/64692aea-0243-4167-a3b6-2a6dc81a8fef-konnectivity-ca\") pod \"konnectivity-agent-g6rx5\" (UID: \"64692aea-0243-4167-a3b6-2a6dc81a8fef\") " pod="kube-system/konnectivity-agent-g6rx5" Jul 12 17:53:15.125334 ip-10-0-129-128 kubenswrapper[2559]: I0712 17:53:15.124225 2559 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-cni-netd\" (UniqueName: \"kubernetes.io/host-path/e51bad50-bc74-4e38-8307-ca257b599314-host-cni-netd\") pod \"ovnkube-node-q5dw8\" (UID: \"e51bad50-bc74-4e38-8307-ca257b599314\") " pod="openshift-ovn-kubernetes/ovnkube-node-q5dw8" Jul 12 17:53:15.125334 ip-10-0-129-128 kubenswrapper[2559]: I0712 17:53:15.124233 2559 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"os-release\" (UniqueName: \"kubernetes.io/host-path/ff97f946-352d-4629-bf96-351edf53ebde-os-release\") pod \"multus-c2jnt\" (UID: \"ff97f946-352d-4629-bf96-351edf53ebde\") " pod="openshift-multus/multus-c2jnt" Jul 12 17:53:15.125334 ip-10-0-129-128 kubenswrapper[2559]: I0712 17:53:15.124246 2559 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"etc-sysctl-d\" (UniqueName: \"kubernetes.io/host-path/cf65b059-f8a2-48d6-a773-74f51ac6ae5d-etc-sysctl-d\") pod \"tuned-66xkf\" (UID: \"cf65b059-f8a2-48d6-a773-74f51ac6ae5d\") " pod="openshift-cluster-node-tuning-operator/tuned-66xkf" Jul 12 17:53:15.125334 ip-10-0-129-128 kubenswrapper[2559]: I0712 17:53:15.124245 2559 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"etc-kubernetes\" (UniqueName: \"kubernetes.io/host-path/ff97f946-352d-4629-bf96-351edf53ebde-etc-kubernetes\") pod \"multus-c2jnt\" (UID: \"ff97f946-352d-4629-bf96-351edf53ebde\") " pod="openshift-multus/multus-c2jnt" Jul 12 17:53:15.125334 ip-10-0-129-128 kubenswrapper[2559]: I0712 17:53:15.124265 2559 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"host-var-lib-cni-multus\" (UniqueName: \"kubernetes.io/host-path/ff97f946-352d-4629-bf96-351edf53ebde-host-var-lib-cni-multus\") pod \"multus-c2jnt\" (UID: \"ff97f946-352d-4629-bf96-351edf53ebde\") " pod="openshift-multus/multus-c2jnt" Jul 12 17:53:15.125334 ip-10-0-129-128 kubenswrapper[2559]: I0712 17:53:15.124202 2559 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"multus-socket-dir-parent\" (UniqueName: \"kubernetes.io/host-path/ff97f946-352d-4629-bf96-351edf53ebde-multus-socket-dir-parent\") pod \"multus-c2jnt\" (UID: \"ff97f946-352d-4629-bf96-351edf53ebde\") " pod="openshift-multus/multus-c2jnt" Jul 12 17:53:15.125334 ip-10-0-129-128 kubenswrapper[2559]: I0712 17:53:15.124338 2559 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host\" (UniqueName: \"kubernetes.io/host-path/6d23c7c4-9eb6-483f-baf3-f25e641e2ead-host\") pod \"node-ca-vxx7w\" (UID: \"6d23c7c4-9eb6-483f-baf3-f25e641e2ead\") " pod="openshift-image-registry/node-ca-vxx7w" Jul 12 17:53:15.125334 ip-10-0-129-128 kubenswrapper[2559]: I0712 17:53:15.124235 2559 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"host-run-k8s-cni-cncf-io\" (UniqueName: \"kubernetes.io/host-path/ff97f946-352d-4629-bf96-351edf53ebde-host-run-k8s-cni-cncf-io\") pod \"multus-c2jnt\" (UID: \"ff97f946-352d-4629-bf96-351edf53ebde\") " pod="openshift-multus/multus-c2jnt" Jul 12 17:53:15.125334 ip-10-0-129-128 kubenswrapper[2559]: I0712 17:53:15.124388 2559 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"host\" (UniqueName: \"kubernetes.io/host-path/6d23c7c4-9eb6-483f-baf3-f25e641e2ead-host\") pod \"node-ca-vxx7w\" (UID: \"6d23c7c4-9eb6-483f-baf3-f25e641e2ead\") " pod="openshift-image-registry/node-ca-vxx7w" Jul 12 17:53:15.125334 ip-10-0-129-128 kubenswrapper[2559]: I0712 17:53:15.124417 2559 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"sys-fs\" (UniqueName: \"kubernetes.io/host-path/3e848887-84a9-4599-afc5-c3750c0857a0-sys-fs\") pod \"aws-ebs-csi-driver-node-m2r7q\" (UID: \"3e848887-84a9-4599-afc5-c3750c0857a0\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-m2r7q" Jul 12 17:53:15.125334 ip-10-0-129-128 kubenswrapper[2559]: I0712 17:53:15.124516 2559 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"ovnkube-script-lib\" (UniqueName: \"kubernetes.io/configmap/e51bad50-bc74-4e38-8307-ca257b599314-ovnkube-script-lib\") pod \"ovnkube-node-q5dw8\" (UID: \"e51bad50-bc74-4e38-8307-ca257b599314\") " pod="openshift-ovn-kubernetes/ovnkube-node-q5dw8" Jul 12 17:53:15.125334 ip-10-0-129-128 kubenswrapper[2559]: I0712 17:53:15.124579 2559 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"host-var-lib-kubelet\" (UniqueName: \"kubernetes.io/host-path/ff97f946-352d-4629-bf96-351edf53ebde-host-var-lib-kubelet\") pod \"multus-c2jnt\" (UID: \"ff97f946-352d-4629-bf96-351edf53ebde\") " pod="openshift-multus/multus-c2jnt" Jul 12 17:53:15.126137 ip-10-0-129-128 kubenswrapper[2559]: I0712 17:53:15.124602 2559 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"etc-sysctl-d\" (UniqueName: \"kubernetes.io/host-path/cf65b059-f8a2-48d6-a773-74f51ac6ae5d-etc-sysctl-d\") pod \"tuned-66xkf\" (UID: \"cf65b059-f8a2-48d6-a773-74f51ac6ae5d\") " pod="openshift-cluster-node-tuning-operator/tuned-66xkf" Jul 12 17:53:15.126137 ip-10-0-129-128 kubenswrapper[2559]: I0712 17:53:15.124620 2559 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"run-systemd\" (UniqueName: \"kubernetes.io/host-path/e51bad50-bc74-4e38-8307-ca257b599314-run-systemd\") pod \"ovnkube-node-q5dw8\" (UID: \"e51bad50-bc74-4e38-8307-ca257b599314\") " pod="openshift-ovn-kubernetes/ovnkube-node-q5dw8" Jul 12 17:53:15.126137 ip-10-0-129-128 kubenswrapper[2559]: I0712 17:53:15.124759 2559 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"host-cni-netd\" (UniqueName: \"kubernetes.io/host-path/e51bad50-bc74-4e38-8307-ca257b599314-host-cni-netd\") pod \"ovnkube-node-q5dw8\" (UID: \"e51bad50-bc74-4e38-8307-ca257b599314\") " pod="openshift-ovn-kubernetes/ovnkube-node-q5dw8" Jul 12 17:53:15.126137 ip-10-0-129-128 kubenswrapper[2559]: I0712 17:53:15.124796 2559 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"socket-dir\" (UniqueName: \"kubernetes.io/host-path/3e848887-84a9-4599-afc5-c3750c0857a0-socket-dir\") pod \"aws-ebs-csi-driver-node-m2r7q\" (UID: \"3e848887-84a9-4599-afc5-c3750c0857a0\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-m2r7q" Jul 12 17:53:15.126137 ip-10-0-129-128 kubenswrapper[2559]: I0712 17:53:15.124812 2559 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"registration-dir\" (UniqueName: \"kubernetes.io/host-path/3e848887-84a9-4599-afc5-c3750c0857a0-registration-dir\") pod \"aws-ebs-csi-driver-node-m2r7q\" (UID: \"3e848887-84a9-4599-afc5-c3750c0857a0\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-m2r7q" Jul 12 17:53:15.126137 ip-10-0-129-128 kubenswrapper[2559]: I0712 17:53:15.124831 2559 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"var-lib-openvswitch\" (UniqueName: \"kubernetes.io/host-path/e51bad50-bc74-4e38-8307-ca257b599314-var-lib-openvswitch\") pod \"ovnkube-node-q5dw8\" (UID: \"e51bad50-bc74-4e38-8307-ca257b599314\") " pod="openshift-ovn-kubernetes/ovnkube-node-q5dw8" Jul 12 17:53:15.126137 ip-10-0-129-128 kubenswrapper[2559]: I0712 17:53:15.124849 2559 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"etc-kubernetes\" (UniqueName: \"kubernetes.io/host-path/cf65b059-f8a2-48d6-a773-74f51ac6ae5d-etc-kubernetes\") pod \"tuned-66xkf\" (UID: \"cf65b059-f8a2-48d6-a773-74f51ac6ae5d\") " pod="openshift-cluster-node-tuning-operator/tuned-66xkf" Jul 12 17:53:15.126137 ip-10-0-129-128 kubenswrapper[2559]: I0712 17:53:15.124869 2559 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"etc-systemd\" (UniqueName: \"kubernetes.io/host-path/cf65b059-f8a2-48d6-a773-74f51ac6ae5d-etc-systemd\") pod \"tuned-66xkf\" (UID: \"cf65b059-f8a2-48d6-a773-74f51ac6ae5d\") " pod="openshift-cluster-node-tuning-operator/tuned-66xkf" Jul 12 17:53:15.126137 ip-10-0-129-128 kubenswrapper[2559]: I0712 17:53:15.124884 2559 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tmp\" (UniqueName: \"kubernetes.io/empty-dir/cf65b059-f8a2-48d6-a773-74f51ac6ae5d-tmp\") pod \"tuned-66xkf\" (UID: \"cf65b059-f8a2-48d6-a773-74f51ac6ae5d\") " pod="openshift-cluster-node-tuning-operator/tuned-66xkf" Jul 12 17:53:15.126137 ip-10-0-129-128 kubenswrapper[2559]: I0712 17:53:15.124910 2559 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"cni-binary-copy\" (UniqueName: \"kubernetes.io/configmap/ff97f946-352d-4629-bf96-351edf53ebde-cni-binary-copy\") pod \"multus-c2jnt\" (UID: \"ff97f946-352d-4629-bf96-351edf53ebde\") " pod="openshift-multus/multus-c2jnt" Jul 12 17:53:15.126137 ip-10-0-129-128 kubenswrapper[2559]: I0712 17:53:15.124951 2559 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"multus-daemon-config\" (UniqueName: \"kubernetes.io/configmap/ff97f946-352d-4629-bf96-351edf53ebde-multus-daemon-config\") pod \"multus-c2jnt\" (UID: \"ff97f946-352d-4629-bf96-351edf53ebde\") " pod="openshift-multus/multus-c2jnt" Jul 12 17:53:15.126137 ip-10-0-129-128 kubenswrapper[2559]: I0712 17:53:15.124967 2559 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"var-lib-openvswitch\" (UniqueName: \"kubernetes.io/host-path/e51bad50-bc74-4e38-8307-ca257b599314-var-lib-openvswitch\") pod \"ovnkube-node-q5dw8\" (UID: \"e51bad50-bc74-4e38-8307-ca257b599314\") " pod="openshift-ovn-kubernetes/ovnkube-node-q5dw8" Jul 12 17:53:15.126137 ip-10-0-129-128 kubenswrapper[2559]: I0712 17:53:15.124971 2559 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-run-netns\" (UniqueName: \"kubernetes.io/host-path/ff97f946-352d-4629-bf96-351edf53ebde-host-run-netns\") pod \"multus-c2jnt\" (UID: \"ff97f946-352d-4629-bf96-351edf53ebde\") " pod="openshift-multus/multus-c2jnt" Jul 12 17:53:15.126137 ip-10-0-129-128 kubenswrapper[2559]: I0712 17:53:15.125009 2559 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"host-run-netns\" (UniqueName: \"kubernetes.io/host-path/ff97f946-352d-4629-bf96-351edf53ebde-host-run-netns\") pod \"multus-c2jnt\" (UID: \"ff97f946-352d-4629-bf96-351edf53ebde\") " pod="openshift-multus/multus-c2jnt" Jul 12 17:53:15.126137 ip-10-0-129-128 kubenswrapper[2559]: I0712 17:53:15.125011 2559 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"agent-certs\" (UniqueName: \"kubernetes.io/secret/64692aea-0243-4167-a3b6-2a6dc81a8fef-agent-certs\") pod \"konnectivity-agent-g6rx5\" (UID: \"64692aea-0243-4167-a3b6-2a6dc81a8fef\") " pod="kube-system/konnectivity-agent-g6rx5" Jul 12 17:53:15.126909 ip-10-0-129-128 kubenswrapper[2559]: I0712 17:53:15.125039 2559 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-run-ovn-kubernetes\" (UniqueName: \"kubernetes.io/host-path/e51bad50-bc74-4e38-8307-ca257b599314-host-run-ovn-kubernetes\") pod \"ovnkube-node-q5dw8\" (UID: \"e51bad50-bc74-4e38-8307-ca257b599314\") " pod="openshift-ovn-kubernetes/ovnkube-node-q5dw8" Jul 12 17:53:15.126909 ip-10-0-129-128 kubenswrapper[2559]: I0712 17:53:15.125068 2559 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-var-lib-cni-networks-ovn-kubernetes\" (UniqueName: \"kubernetes.io/host-path/e51bad50-bc74-4e38-8307-ca257b599314-host-var-lib-cni-networks-ovn-kubernetes\") pod \"ovnkube-node-q5dw8\" (UID: \"e51bad50-bc74-4e38-8307-ca257b599314\") " pod="openshift-ovn-kubernetes/ovnkube-node-q5dw8" Jul 12 17:53:15.126909 ip-10-0-129-128 kubenswrapper[2559]: I0712 17:53:15.125094 2559 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"ovnkube-config\" (UniqueName: \"kubernetes.io/configmap/e51bad50-bc74-4e38-8307-ca257b599314-ovnkube-config\") pod \"ovnkube-node-q5dw8\" (UID: \"e51bad50-bc74-4e38-8307-ca257b599314\") " pod="openshift-ovn-kubernetes/ovnkube-node-q5dw8" Jul 12 17:53:15.126909 ip-10-0-129-128 kubenswrapper[2559]: I0712 17:53:15.125116 2559 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"host-run-ovn-kubernetes\" (UniqueName: \"kubernetes.io/host-path/e51bad50-bc74-4e38-8307-ca257b599314-host-run-ovn-kubernetes\") pod \"ovnkube-node-q5dw8\" (UID: \"e51bad50-bc74-4e38-8307-ca257b599314\") " pod="openshift-ovn-kubernetes/ovnkube-node-q5dw8" Jul 12 17:53:15.126909 ip-10-0-129-128 kubenswrapper[2559]: I0712 17:53:15.125012 2559 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"etc-systemd\" (UniqueName: \"kubernetes.io/host-path/cf65b059-f8a2-48d6-a773-74f51ac6ae5d-etc-systemd\") pod \"tuned-66xkf\" (UID: \"cf65b059-f8a2-48d6-a773-74f51ac6ae5d\") " pod="openshift-cluster-node-tuning-operator/tuned-66xkf" Jul 12 17:53:15.126909 ip-10-0-129-128 kubenswrapper[2559]: I0712 17:53:15.125118 2559 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"run\" (UniqueName: \"kubernetes.io/host-path/cf65b059-f8a2-48d6-a773-74f51ac6ae5d-run\") pod \"tuned-66xkf\" (UID: \"cf65b059-f8a2-48d6-a773-74f51ac6ae5d\") " pod="openshift-cluster-node-tuning-operator/tuned-66xkf" Jul 12 17:53:15.126909 ip-10-0-129-128 kubenswrapper[2559]: I0712 17:53:15.125153 2559 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"run\" (UniqueName: \"kubernetes.io/host-path/cf65b059-f8a2-48d6-a773-74f51ac6ae5d-run\") pod \"tuned-66xkf\" (UID: \"cf65b059-f8a2-48d6-a773-74f51ac6ae5d\") " pod="openshift-cluster-node-tuning-operator/tuned-66xkf" Jul 12 17:53:15.126909 ip-10-0-129-128 kubenswrapper[2559]: I0712 17:53:15.125168 2559 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"host-var-lib-cni-networks-ovn-kubernetes\" (UniqueName: \"kubernetes.io/host-path/e51bad50-bc74-4e38-8307-ca257b599314-host-var-lib-cni-networks-ovn-kubernetes\") pod \"ovnkube-node-q5dw8\" (UID: \"e51bad50-bc74-4e38-8307-ca257b599314\") " pod="openshift-ovn-kubernetes/ovnkube-node-q5dw8" Jul 12 17:53:15.126909 ip-10-0-129-128 kubenswrapper[2559]: I0712 17:53:15.125170 2559 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"etc-tuned\" (UniqueName: \"kubernetes.io/empty-dir/cf65b059-f8a2-48d6-a773-74f51ac6ae5d-etc-tuned\") pod \"tuned-66xkf\" (UID: \"cf65b059-f8a2-48d6-a773-74f51ac6ae5d\") " pod="openshift-cluster-node-tuning-operator/tuned-66xkf" Jul 12 17:53:15.126909 ip-10-0-129-128 kubenswrapper[2559]: I0712 17:53:15.125182 2559 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"konnectivity-ca\" (UniqueName: \"kubernetes.io/configmap/64692aea-0243-4167-a3b6-2a6dc81a8fef-konnectivity-ca\") pod \"konnectivity-agent-g6rx5\" (UID: \"64692aea-0243-4167-a3b6-2a6dc81a8fef\") " pod="kube-system/konnectivity-agent-g6rx5" Jul 12 17:53:15.126909 ip-10-0-129-128 kubenswrapper[2559]: I0712 17:53:15.125207 2559 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kubelet-config\" (UniqueName: \"kubernetes.io/host-path/6ab3daa1-3376-4f53-8106-d6d7417a327e-kubelet-config\") pod \"global-pull-secret-syncer-mpqvv\" (UID: \"6ab3daa1-3376-4f53-8106-d6d7417a327e\") " pod="kube-system/global-pull-secret-syncer-mpqvv" Jul 12 17:53:15.126909 ip-10-0-129-128 kubenswrapper[2559]: I0712 17:53:15.124953 2559 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"etc-kubernetes\" (UniqueName: \"kubernetes.io/host-path/cf65b059-f8a2-48d6-a773-74f51ac6ae5d-etc-kubernetes\") pod \"tuned-66xkf\" (UID: \"cf65b059-f8a2-48d6-a773-74f51ac6ae5d\") " pod="openshift-cluster-node-tuning-operator/tuned-66xkf" Jul 12 17:53:15.126909 ip-10-0-129-128 kubenswrapper[2559]: I0712 17:53:15.125232 2559 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-dhv25\" (UniqueName: \"kubernetes.io/projected/3e848887-84a9-4599-afc5-c3750c0857a0-kube-api-access-dhv25\") pod \"aws-ebs-csi-driver-node-m2r7q\" (UID: \"3e848887-84a9-4599-afc5-c3750c0857a0\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-m2r7q" Jul 12 17:53:15.126909 ip-10-0-129-128 kubenswrapper[2559]: I0712 17:53:15.125255 2559 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"etc-selinux\" (UniqueName: \"kubernetes.io/host-path/3e848887-84a9-4599-afc5-c3750c0857a0-etc-selinux\") pod \"aws-ebs-csi-driver-node-m2r7q\" (UID: \"3e848887-84a9-4599-afc5-c3750c0857a0\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-m2r7q" Jul 12 17:53:15.127576 ip-10-0-129-128 kubenswrapper[2559]: I0712 17:53:15.125267 2559 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kubelet-config\" (UniqueName: \"kubernetes.io/host-path/6ab3daa1-3376-4f53-8106-d6d7417a327e-kubelet-config\") pod \"global-pull-secret-syncer-mpqvv\" (UID: \"6ab3daa1-3376-4f53-8106-d6d7417a327e\") " pod="kube-system/global-pull-secret-syncer-mpqvv" Jul 12 17:53:15.127576 ip-10-0-129-128 kubenswrapper[2559]: I0712 17:53:15.125278 2559 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"log-socket\" (UniqueName: \"kubernetes.io/host-path/e51bad50-bc74-4e38-8307-ca257b599314-log-socket\") pod \"ovnkube-node-q5dw8\" (UID: \"e51bad50-bc74-4e38-8307-ca257b599314\") " pod="openshift-ovn-kubernetes/ovnkube-node-q5dw8" Jul 12 17:53:15.127576 ip-10-0-129-128 kubenswrapper[2559]: I0712 17:53:15.125303 2559 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-cni-bin\" (UniqueName: \"kubernetes.io/host-path/e51bad50-bc74-4e38-8307-ca257b599314-host-cni-bin\") pod \"ovnkube-node-q5dw8\" (UID: \"e51bad50-bc74-4e38-8307-ca257b599314\") " pod="openshift-ovn-kubernetes/ovnkube-node-q5dw8" Jul 12 17:53:15.127576 ip-10-0-129-128 kubenswrapper[2559]: I0712 17:53:15.125328 2559 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host\" (UniqueName: \"kubernetes.io/host-path/cf65b059-f8a2-48d6-a773-74f51ac6ae5d-host\") pod \"tuned-66xkf\" (UID: \"cf65b059-f8a2-48d6-a773-74f51ac6ae5d\") " pod="openshift-cluster-node-tuning-operator/tuned-66xkf" Jul 12 17:53:15.127576 ip-10-0-129-128 kubenswrapper[2559]: I0712 17:53:15.125351 2559 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-var-lib-cni-bin\" (UniqueName: \"kubernetes.io/host-path/ff97f946-352d-4629-bf96-351edf53ebde-host-var-lib-cni-bin\") pod \"multus-c2jnt\" (UID: \"ff97f946-352d-4629-bf96-351edf53ebde\") " pod="openshift-multus/multus-c2jnt" Jul 12 17:53:15.127576 ip-10-0-129-128 kubenswrapper[2559]: I0712 17:53:15.125377 2559 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"dbus\" (UniqueName: \"kubernetes.io/host-path/6ab3daa1-3376-4f53-8106-d6d7417a327e-dbus\") pod \"global-pull-secret-syncer-mpqvv\" (UID: \"6ab3daa1-3376-4f53-8106-d6d7417a327e\") " pod="kube-system/global-pull-secret-syncer-mpqvv" Jul 12 17:53:15.127576 ip-10-0-129-128 kubenswrapper[2559]: I0712 17:53:15.125378 2559 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"etc-selinux\" (UniqueName: \"kubernetes.io/host-path/3e848887-84a9-4599-afc5-c3750c0857a0-etc-selinux\") pod \"aws-ebs-csi-driver-node-m2r7q\" (UID: \"3e848887-84a9-4599-afc5-c3750c0857a0\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-m2r7q" Jul 12 17:53:15.127576 ip-10-0-129-128 kubenswrapper[2559]: I0712 17:53:15.125401 2559 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-run-netns\" (UniqueName: \"kubernetes.io/host-path/e51bad50-bc74-4e38-8307-ca257b599314-host-run-netns\") pod \"ovnkube-node-q5dw8\" (UID: \"e51bad50-bc74-4e38-8307-ca257b599314\") " pod="openshift-ovn-kubernetes/ovnkube-node-q5dw8" Jul 12 17:53:15.127576 ip-10-0-129-128 kubenswrapper[2559]: I0712 17:53:15.125407 2559 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"host-cni-bin\" (UniqueName: \"kubernetes.io/host-path/e51bad50-bc74-4e38-8307-ca257b599314-host-cni-bin\") pod \"ovnkube-node-q5dw8\" (UID: \"e51bad50-bc74-4e38-8307-ca257b599314\") " pod="openshift-ovn-kubernetes/ovnkube-node-q5dw8" Jul 12 17:53:15.127576 ip-10-0-129-128 kubenswrapper[2559]: I0712 17:53:15.125425 2559 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"node-log\" (UniqueName: \"kubernetes.io/host-path/e51bad50-bc74-4e38-8307-ca257b599314-node-log\") pod \"ovnkube-node-q5dw8\" (UID: \"e51bad50-bc74-4e38-8307-ca257b599314\") " pod="openshift-ovn-kubernetes/ovnkube-node-q5dw8" Jul 12 17:53:15.127576 ip-10-0-129-128 kubenswrapper[2559]: I0712 17:53:15.125435 2559 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"host-var-lib-cni-bin\" (UniqueName: \"kubernetes.io/host-path/ff97f946-352d-4629-bf96-351edf53ebde-host-var-lib-cni-bin\") pod \"multus-c2jnt\" (UID: \"ff97f946-352d-4629-bf96-351edf53ebde\") " pod="openshift-multus/multus-c2jnt" Jul 12 17:53:15.127576 ip-10-0-129-128 kubenswrapper[2559]: I0712 17:53:15.125450 2559 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"var-lib-kubelet\" (UniqueName: \"kubernetes.io/host-path/cf65b059-f8a2-48d6-a773-74f51ac6ae5d-var-lib-kubelet\") pod \"tuned-66xkf\" (UID: \"cf65b059-f8a2-48d6-a773-74f51ac6ae5d\") " pod="openshift-cluster-node-tuning-operator/tuned-66xkf" Jul 12 17:53:15.127576 ip-10-0-129-128 kubenswrapper[2559]: I0712 17:53:15.125458 2559 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"log-socket\" (UniqueName: \"kubernetes.io/host-path/e51bad50-bc74-4e38-8307-ca257b599314-log-socket\") pod \"ovnkube-node-q5dw8\" (UID: \"e51bad50-bc74-4e38-8307-ca257b599314\") " pod="openshift-ovn-kubernetes/ovnkube-node-q5dw8" Jul 12 17:53:15.127576 ip-10-0-129-128 kubenswrapper[2559]: I0712 17:53:15.125487 2559 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"host\" (UniqueName: \"kubernetes.io/host-path/cf65b059-f8a2-48d6-a773-74f51ac6ae5d-host\") pod \"tuned-66xkf\" (UID: \"cf65b059-f8a2-48d6-a773-74f51ac6ae5d\") " pod="openshift-cluster-node-tuning-operator/tuned-66xkf" Jul 12 17:53:15.127576 ip-10-0-129-128 kubenswrapper[2559]: I0712 17:53:15.125490 2559 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"host-run-netns\" (UniqueName: \"kubernetes.io/host-path/e51bad50-bc74-4e38-8307-ca257b599314-host-run-netns\") pod \"ovnkube-node-q5dw8\" (UID: \"e51bad50-bc74-4e38-8307-ca257b599314\") " pod="openshift-ovn-kubernetes/ovnkube-node-q5dw8" Jul 12 17:53:15.128232 ip-10-0-129-128 kubenswrapper[2559]: I0712 17:53:15.125507 2559 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"var-lib-kubelet\" (UniqueName: \"kubernetes.io/host-path/cf65b059-f8a2-48d6-a773-74f51ac6ae5d-var-lib-kubelet\") pod \"tuned-66xkf\" (UID: \"cf65b059-f8a2-48d6-a773-74f51ac6ae5d\") " pod="openshift-cluster-node-tuning-operator/tuned-66xkf" Jul 12 17:53:15.128232 ip-10-0-129-128 kubenswrapper[2559]: I0712 17:53:15.125511 2559 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"hostroot\" (UniqueName: \"kubernetes.io/host-path/ff97f946-352d-4629-bf96-351edf53ebde-hostroot\") pod \"multus-c2jnt\" (UID: \"ff97f946-352d-4629-bf96-351edf53ebde\") " pod="openshift-multus/multus-c2jnt" Jul 12 17:53:15.128232 ip-10-0-129-128 kubenswrapper[2559]: I0712 17:53:15.125519 2559 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"dbus\" (UniqueName: \"kubernetes.io/host-path/6ab3daa1-3376-4f53-8106-d6d7417a327e-dbus\") pod \"global-pull-secret-syncer-mpqvv\" (UID: \"6ab3daa1-3376-4f53-8106-d6d7417a327e\") " pod="kube-system/global-pull-secret-syncer-mpqvv" Jul 12 17:53:15.128232 ip-10-0-129-128 kubenswrapper[2559]: I0712 17:53:15.125537 2559 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"node-log\" (UniqueName: \"kubernetes.io/host-path/e51bad50-bc74-4e38-8307-ca257b599314-node-log\") pod \"ovnkube-node-q5dw8\" (UID: \"e51bad50-bc74-4e38-8307-ca257b599314\") " pod="openshift-ovn-kubernetes/ovnkube-node-q5dw8" Jul 12 17:53:15.128232 ip-10-0-129-128 kubenswrapper[2559]: I0712 17:53:15.125544 2559 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"hostroot\" (UniqueName: \"kubernetes.io/host-path/ff97f946-352d-4629-bf96-351edf53ebde-hostroot\") pod \"multus-c2jnt\" (UID: \"ff97f946-352d-4629-bf96-351edf53ebde\") " pod="openshift-multus/multus-c2jnt" Jul 12 17:53:15.128232 ip-10-0-129-128 kubenswrapper[2559]: I0712 17:53:15.125545 2559 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-26mv6\" (UniqueName: \"kubernetes.io/projected/6f65bad8-9ec7-4a51-8557-74b3f79ef76c-kube-api-access-26mv6\") pod \"network-metrics-daemon-qtkvt\" (UID: \"6f65bad8-9ec7-4a51-8557-74b3f79ef76c\") " pod="openshift-multus/network-metrics-daemon-qtkvt" Jul 12 17:53:15.128232 ip-10-0-129-128 kubenswrapper[2559]: I0712 17:53:15.125576 2559 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"serviceca\" (UniqueName: \"kubernetes.io/configmap/6d23c7c4-9eb6-483f-baf3-f25e641e2ead-serviceca\") pod \"node-ca-vxx7w\" (UID: \"6d23c7c4-9eb6-483f-baf3-f25e641e2ead\") " pod="openshift-image-registry/node-ca-vxx7w" Jul 12 17:53:15.128232 ip-10-0-129-128 kubenswrapper[2559]: I0712 17:53:15.125593 2559 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"ovnkube-config\" (UniqueName: \"kubernetes.io/configmap/e51bad50-bc74-4e38-8307-ca257b599314-ovnkube-config\") pod \"ovnkube-node-q5dw8\" (UID: \"e51bad50-bc74-4e38-8307-ca257b599314\") " pod="openshift-ovn-kubernetes/ovnkube-node-q5dw8" Jul 12 17:53:15.128232 ip-10-0-129-128 kubenswrapper[2559]: I0712 17:53:15.125600 2559 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-mqz8x\" (UniqueName: \"kubernetes.io/projected/6d23c7c4-9eb6-483f-baf3-f25e641e2ead-kube-api-access-mqz8x\") pod \"node-ca-vxx7w\" (UID: \"6d23c7c4-9eb6-483f-baf3-f25e641e2ead\") " pod="openshift-image-registry/node-ca-vxx7w" Jul 12 17:53:15.128232 ip-10-0-129-128 kubenswrapper[2559]: I0712 17:53:15.125626 2559 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"systemd-units\" (UniqueName: \"kubernetes.io/host-path/e51bad50-bc74-4e38-8307-ca257b599314-systemd-units\") pod \"ovnkube-node-q5dw8\" (UID: \"e51bad50-bc74-4e38-8307-ca257b599314\") " pod="openshift-ovn-kubernetes/ovnkube-node-q5dw8" Jul 12 17:53:15.128232 ip-10-0-129-128 kubenswrapper[2559]: I0712 17:53:15.125678 2559 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"run-ovn\" (UniqueName: \"kubernetes.io/host-path/e51bad50-bc74-4e38-8307-ca257b599314-run-ovn\") pod \"ovnkube-node-q5dw8\" (UID: \"e51bad50-bc74-4e38-8307-ca257b599314\") " pod="openshift-ovn-kubernetes/ovnkube-node-q5dw8" Jul 12 17:53:15.128232 ip-10-0-129-128 kubenswrapper[2559]: I0712 17:53:15.125704 2559 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"system-cni-dir\" (UniqueName: \"kubernetes.io/host-path/ff97f946-352d-4629-bf96-351edf53ebde-system-cni-dir\") pod \"multus-c2jnt\" (UID: \"ff97f946-352d-4629-bf96-351edf53ebde\") " pod="openshift-multus/multus-c2jnt" Jul 12 17:53:15.128232 ip-10-0-129-128 kubenswrapper[2559]: I0712 17:53:15.125711 2559 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"systemd-units\" (UniqueName: \"kubernetes.io/host-path/e51bad50-bc74-4e38-8307-ca257b599314-systemd-units\") pod \"ovnkube-node-q5dw8\" (UID: \"e51bad50-bc74-4e38-8307-ca257b599314\") " pod="openshift-ovn-kubernetes/ovnkube-node-q5dw8" Jul 12 17:53:15.128232 ip-10-0-129-128 kubenswrapper[2559]: I0712 17:53:15.125728 2559 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/6f65bad8-9ec7-4a51-8557-74b3f79ef76c-metrics-certs\") pod \"network-metrics-daemon-qtkvt\" (UID: \"6f65bad8-9ec7-4a51-8557-74b3f79ef76c\") " pod="openshift-multus/network-metrics-daemon-qtkvt" Jul 12 17:53:15.128232 ip-10-0-129-128 kubenswrapper[2559]: I0712 17:53:15.125757 2559 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"run-ovn\" (UniqueName: \"kubernetes.io/host-path/e51bad50-bc74-4e38-8307-ca257b599314-run-ovn\") pod \"ovnkube-node-q5dw8\" (UID: \"e51bad50-bc74-4e38-8307-ca257b599314\") " pod="openshift-ovn-kubernetes/ovnkube-node-q5dw8" Jul 12 17:53:15.128876 ip-10-0-129-128 kubenswrapper[2559]: I0712 17:53:15.125798 2559 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"system-cni-dir\" (UniqueName: \"kubernetes.io/host-path/ff97f946-352d-4629-bf96-351edf53ebde-system-cni-dir\") pod \"multus-c2jnt\" (UID: \"ff97f946-352d-4629-bf96-351edf53ebde\") " pod="openshift-multus/multus-c2jnt" Jul 12 17:53:15.128876 ip-10-0-129-128 kubenswrapper[2559]: E0712 17:53:15.125832 2559 secret.go:189] Couldn't get secret openshift-multus/metrics-daemon-secret: object "openshift-multus"/"metrics-daemon-secret" not registered Jul 12 17:53:15.128876 ip-10-0-129-128 kubenswrapper[2559]: E0712 17:53:15.125913 2559 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/6f65bad8-9ec7-4a51-8557-74b3f79ef76c-metrics-certs podName:6f65bad8-9ec7-4a51-8557-74b3f79ef76c nodeName:}" failed. No retries permitted until 2026-07-12 17:53:15.625884504 +0000 UTC m=+3.119952418 (durationBeforeRetry 500ms). Error: MountVolume.SetUp failed for volume "metrics-certs" (UniqueName: "kubernetes.io/secret/6f65bad8-9ec7-4a51-8557-74b3f79ef76c-metrics-certs") pod "network-metrics-daemon-qtkvt" (UID: "6f65bad8-9ec7-4a51-8557-74b3f79ef76c") : object "openshift-multus"/"metrics-daemon-secret" not registered Jul 12 17:53:15.128876 ip-10-0-129-128 kubenswrapper[2559]: I0712 17:53:15.126017 2559 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"serviceca\" (UniqueName: \"kubernetes.io/configmap/6d23c7c4-9eb6-483f-baf3-f25e641e2ead-serviceca\") pod \"node-ca-vxx7w\" (UID: \"6d23c7c4-9eb6-483f-baf3-f25e641e2ead\") " pod="openshift-image-registry/node-ca-vxx7w" Jul 12 17:53:15.128876 ip-10-0-129-128 kubenswrapper[2559]: I0712 17:53:15.126617 2559 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"ovn-node-metrics-cert\" (UniqueName: \"kubernetes.io/secret/e51bad50-bc74-4e38-8307-ca257b599314-ovn-node-metrics-cert\") pod \"ovnkube-node-q5dw8\" (UID: \"e51bad50-bc74-4e38-8307-ca257b599314\") " pod="openshift-ovn-kubernetes/ovnkube-node-q5dw8" Jul 12 17:53:15.128876 ip-10-0-129-128 kubenswrapper[2559]: I0712 17:53:15.127858 2559 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"etc-tuned\" (UniqueName: \"kubernetes.io/empty-dir/cf65b059-f8a2-48d6-a773-74f51ac6ae5d-etc-tuned\") pod \"tuned-66xkf\" (UID: \"cf65b059-f8a2-48d6-a773-74f51ac6ae5d\") " pod="openshift-cluster-node-tuning-operator/tuned-66xkf" Jul 12 17:53:15.128876 ip-10-0-129-128 kubenswrapper[2559]: I0712 17:53:15.127896 2559 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"tmp\" (UniqueName: \"kubernetes.io/empty-dir/cf65b059-f8a2-48d6-a773-74f51ac6ae5d-tmp\") pod \"tuned-66xkf\" (UID: \"cf65b059-f8a2-48d6-a773-74f51ac6ae5d\") " pod="openshift-cluster-node-tuning-operator/tuned-66xkf" Jul 12 17:53:15.128876 ip-10-0-129-128 kubenswrapper[2559]: I0712 17:53:15.128494 2559 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"agent-certs\" (UniqueName: \"kubernetes.io/secret/64692aea-0243-4167-a3b6-2a6dc81a8fef-agent-certs\") pod \"konnectivity-agent-g6rx5\" (UID: \"64692aea-0243-4167-a3b6-2a6dc81a8fef\") " pod="kube-system/konnectivity-agent-g6rx5" Jul 12 17:53:15.130762 ip-10-0-129-128 kubenswrapper[2559]: E0712 17:53:15.130739 2559 projected.go:291] Couldn't get configMap openshift-network-diagnostics/kube-root-ca.crt: object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered Jul 12 17:53:15.130901 ip-10-0-129-128 kubenswrapper[2559]: E0712 17:53:15.130767 2559 projected.go:291] Couldn't get configMap openshift-network-diagnostics/openshift-service-ca.crt: object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered Jul 12 17:53:15.130901 ip-10-0-129-128 kubenswrapper[2559]: E0712 17:53:15.130781 2559 projected.go:196] Error preparing data for projected volume kube-api-access-bn2rx for pod openshift-network-diagnostics/network-check-target-h4bgw: [object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered, object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered] Jul 12 17:53:15.130901 ip-10-0-129-128 kubenswrapper[2559]: E0712 17:53:15.130845 2559 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/projected/e2c3a206-19c0-4d4d-bb98-a99190023ad4-kube-api-access-bn2rx podName:e2c3a206-19c0-4d4d-bb98-a99190023ad4 nodeName:}" failed. No retries permitted until 2026-07-12 17:53:15.630828302 +0000 UTC m=+3.124896230 (durationBeforeRetry 500ms). Error: MountVolume.SetUp failed for volume "kube-api-access-bn2rx" (UniqueName: "kubernetes.io/projected/e2c3a206-19c0-4d4d-bb98-a99190023ad4-kube-api-access-bn2rx") pod "network-check-target-h4bgw" (UID: "e2c3a206-19c0-4d4d-bb98-a99190023ad4") : [object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered, object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered] Jul 12 17:53:15.133027 ip-10-0-129-128 kubenswrapper[2559]: I0712 17:53:15.133001 2559 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-q7sxg\" (UniqueName: \"kubernetes.io/projected/cf65b059-f8a2-48d6-a773-74f51ac6ae5d-kube-api-access-q7sxg\") pod \"tuned-66xkf\" (UID: \"cf65b059-f8a2-48d6-a773-74f51ac6ae5d\") " pod="openshift-cluster-node-tuning-operator/tuned-66xkf" Jul 12 17:53:15.133690 ip-10-0-129-128 kubenswrapper[2559]: I0712 17:53:15.133626 2559 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-2wqhw\" (UniqueName: \"kubernetes.io/projected/ff97f946-352d-4629-bf96-351edf53ebde-kube-api-access-2wqhw\") pod \"multus-c2jnt\" (UID: \"ff97f946-352d-4629-bf96-351edf53ebde\") " pod="openshift-multus/multus-c2jnt" Jul 12 17:53:15.133821 ip-10-0-129-128 kubenswrapper[2559]: I0712 17:53:15.133799 2559 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-p9sff\" (UniqueName: \"kubernetes.io/projected/e51bad50-bc74-4e38-8307-ca257b599314-kube-api-access-p9sff\") pod \"ovnkube-node-q5dw8\" (UID: \"e51bad50-bc74-4e38-8307-ca257b599314\") " pod="openshift-ovn-kubernetes/ovnkube-node-q5dw8" Jul 12 17:53:15.134795 ip-10-0-129-128 kubenswrapper[2559]: I0712 17:53:15.134778 2559 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-dhv25\" (UniqueName: \"kubernetes.io/projected/3e848887-84a9-4599-afc5-c3750c0857a0-kube-api-access-dhv25\") pod \"aws-ebs-csi-driver-node-m2r7q\" (UID: \"3e848887-84a9-4599-afc5-c3750c0857a0\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-m2r7q" Jul 12 17:53:15.136578 ip-10-0-129-128 kubenswrapper[2559]: I0712 17:53:15.136558 2559 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-mqz8x\" (UniqueName: \"kubernetes.io/projected/6d23c7c4-9eb6-483f-baf3-f25e641e2ead-kube-api-access-mqz8x\") pod \"node-ca-vxx7w\" (UID: \"6d23c7c4-9eb6-483f-baf3-f25e641e2ead\") " pod="openshift-image-registry/node-ca-vxx7w" Jul 12 17:53:15.137441 ip-10-0-129-128 kubenswrapper[2559]: I0712 17:53:15.137423 2559 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-26mv6\" (UniqueName: \"kubernetes.io/projected/6f65bad8-9ec7-4a51-8557-74b3f79ef76c-kube-api-access-26mv6\") pod \"network-metrics-daemon-qtkvt\" (UID: \"6f65bad8-9ec7-4a51-8557-74b3f79ef76c\") " pod="openshift-multus/network-metrics-daemon-qtkvt" Jul 12 17:53:15.218434 ip-10-0-129-128 kubenswrapper[2559]: I0712 17:53:15.218342 2559 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-operator/iptables-alerter-fql27" Jul 12 17:53:15.227205 ip-10-0-129-128 kubenswrapper[2559]: I0712 17:53:15.227183 2559 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-dns/node-resolver-fqnvr" Jul 12 17:53:15.238925 ip-10-0-129-128 kubenswrapper[2559]: I0712 17:53:15.238904 2559 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/multus-additional-cni-plugins-9dh8l" Jul 12 17:53:15.246719 ip-10-0-129-128 kubenswrapper[2559]: I0712 17:53:15.246696 2559 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/multus-c2jnt" Jul 12 17:53:15.257544 ip-10-0-129-128 kubenswrapper[2559]: I0712 17:53:15.257503 2559 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/konnectivity-agent-g6rx5" Jul 12 17:53:15.265321 ip-10-0-129-128 kubenswrapper[2559]: I0712 17:53:15.265295 2559 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-image-registry/node-ca-vxx7w" Jul 12 17:53:15.273282 ip-10-0-129-128 kubenswrapper[2559]: I0712 17:53:15.273262 2559 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ovn-kubernetes/ovnkube-node-q5dw8" Jul 12 17:53:15.285753 ip-10-0-129-128 kubenswrapper[2559]: I0712 17:53:15.285733 2559 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-m2r7q" Jul 12 17:53:15.291477 ip-10-0-129-128 kubenswrapper[2559]: I0712 17:53:15.291460 2559 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-cluster-node-tuning-operator/tuned-66xkf" Jul 12 17:53:15.629841 ip-10-0-129-128 kubenswrapper[2559]: I0712 17:53:15.629756 2559 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/6f65bad8-9ec7-4a51-8557-74b3f79ef76c-metrics-certs\") pod \"network-metrics-daemon-qtkvt\" (UID: \"6f65bad8-9ec7-4a51-8557-74b3f79ef76c\") " pod="openshift-multus/network-metrics-daemon-qtkvt" Jul 12 17:53:15.629841 ip-10-0-129-128 kubenswrapper[2559]: I0712 17:53:15.629811 2559 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"original-pull-secret\" (UniqueName: \"kubernetes.io/secret/6ab3daa1-3376-4f53-8106-d6d7417a327e-original-pull-secret\") pod \"global-pull-secret-syncer-mpqvv\" (UID: \"6ab3daa1-3376-4f53-8106-d6d7417a327e\") " pod="kube-system/global-pull-secret-syncer-mpqvv" Jul 12 17:53:15.630037 ip-10-0-129-128 kubenswrapper[2559]: E0712 17:53:15.629931 2559 secret.go:189] Couldn't get secret openshift-multus/metrics-daemon-secret: object "openshift-multus"/"metrics-daemon-secret" not registered Jul 12 17:53:15.630037 ip-10-0-129-128 kubenswrapper[2559]: E0712 17:53:15.629996 2559 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/6f65bad8-9ec7-4a51-8557-74b3f79ef76c-metrics-certs podName:6f65bad8-9ec7-4a51-8557-74b3f79ef76c nodeName:}" failed. No retries permitted until 2026-07-12 17:53:16.629978281 +0000 UTC m=+4.124046203 (durationBeforeRetry 1s). Error: MountVolume.SetUp failed for volume "metrics-certs" (UniqueName: "kubernetes.io/secret/6f65bad8-9ec7-4a51-8557-74b3f79ef76c-metrics-certs") pod "network-metrics-daemon-qtkvt" (UID: "6f65bad8-9ec7-4a51-8557-74b3f79ef76c") : object "openshift-multus"/"metrics-daemon-secret" not registered Jul 12 17:53:15.630037 ip-10-0-129-128 kubenswrapper[2559]: E0712 17:53:15.630027 2559 secret.go:189] Couldn't get secret kube-system/original-pull-secret: object "kube-system"/"original-pull-secret" not registered Jul 12 17:53:15.630189 ip-10-0-129-128 kubenswrapper[2559]: E0712 17:53:15.630080 2559 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/6ab3daa1-3376-4f53-8106-d6d7417a327e-original-pull-secret podName:6ab3daa1-3376-4f53-8106-d6d7417a327e nodeName:}" failed. No retries permitted until 2026-07-12 17:53:16.630062944 +0000 UTC m=+4.124130855 (durationBeforeRetry 1s). Error: MountVolume.SetUp failed for volume "original-pull-secret" (UniqueName: "kubernetes.io/secret/6ab3daa1-3376-4f53-8106-d6d7417a327e-original-pull-secret") pod "global-pull-secret-syncer-mpqvv" (UID: "6ab3daa1-3376-4f53-8106-d6d7417a327e") : object "kube-system"/"original-pull-secret" not registered Jul 12 17:53:15.730802 ip-10-0-129-128 kubenswrapper[2559]: I0712 17:53:15.730770 2559 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-bn2rx\" (UniqueName: \"kubernetes.io/projected/e2c3a206-19c0-4d4d-bb98-a99190023ad4-kube-api-access-bn2rx\") pod \"network-check-target-h4bgw\" (UID: \"e2c3a206-19c0-4d4d-bb98-a99190023ad4\") " pod="openshift-network-diagnostics/network-check-target-h4bgw" Jul 12 17:53:15.730972 ip-10-0-129-128 kubenswrapper[2559]: E0712 17:53:15.730909 2559 projected.go:291] Couldn't get configMap openshift-network-diagnostics/kube-root-ca.crt: object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered Jul 12 17:53:15.730972 ip-10-0-129-128 kubenswrapper[2559]: E0712 17:53:15.730929 2559 projected.go:291] Couldn't get configMap openshift-network-diagnostics/openshift-service-ca.crt: object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered Jul 12 17:53:15.730972 ip-10-0-129-128 kubenswrapper[2559]: E0712 17:53:15.730938 2559 projected.go:196] Error preparing data for projected volume kube-api-access-bn2rx for pod openshift-network-diagnostics/network-check-target-h4bgw: [object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered, object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered] Jul 12 17:53:15.731144 ip-10-0-129-128 kubenswrapper[2559]: E0712 17:53:15.730995 2559 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/projected/e2c3a206-19c0-4d4d-bb98-a99190023ad4-kube-api-access-bn2rx podName:e2c3a206-19c0-4d4d-bb98-a99190023ad4 nodeName:}" failed. No retries permitted until 2026-07-12 17:53:16.730974126 +0000 UTC m=+4.225042035 (durationBeforeRetry 1s). Error: MountVolume.SetUp failed for volume "kube-api-access-bn2rx" (UniqueName: "kubernetes.io/projected/e2c3a206-19c0-4d4d-bb98-a99190023ad4-kube-api-access-bn2rx") pod "network-check-target-h4bgw" (UID: "e2c3a206-19c0-4d4d-bb98-a99190023ad4") : [object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered, object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered] Jul 12 17:53:15.948320 ip-10-0-129-128 kubenswrapper[2559]: I0712 17:53:15.948227 2559 certificate_manager.go:715] "Certificate rotation deadline determined" logger="kubernetes.io/kubelet-serving" expiration="2028-07-11 17:48:14 +0000 UTC" deadline="2028-01-30 06:26:14.075230584 +0000 UTC" Jul 12 17:53:15.948320 ip-10-0-129-128 kubenswrapper[2559]: I0712 17:53:15.948268 2559 certificate_manager.go:431] "Waiting for next certificate rotation" logger="kubernetes.io/kubelet-serving" sleep="13596h32m58.1269666s" Jul 12 17:53:16.028840 ip-10-0-129-128 kubenswrapper[2559]: I0712 17:53:16.028813 2559 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-mpqvv" Jul 12 17:53:16.029395 ip-10-0-129-128 kubenswrapper[2559]: E0712 17:53:16.028939 2559 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-mpqvv" podUID="6ab3daa1-3376-4f53-8106-d6d7417a327e" Jul 12 17:53:16.175457 ip-10-0-129-128 kubenswrapper[2559]: W0712 17:53:16.175425 2559 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podcf65b059_f8a2_48d6_a773_74f51ac6ae5d.slice/crio-528d7dbfdc90533e163810243ea8d70c941c3427a3e6b3d7064b860667c1bc3e WatchSource:0}: Error finding container 528d7dbfdc90533e163810243ea8d70c941c3427a3e6b3d7064b860667c1bc3e: Status 404 returned error can't find the container with id 528d7dbfdc90533e163810243ea8d70c941c3427a3e6b3d7064b860667c1bc3e Jul 12 17:53:16.182374 ip-10-0-129-128 kubenswrapper[2559]: W0712 17:53:16.182348 2559 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod3e848887_84a9_4599_afc5_c3750c0857a0.slice/crio-20141c8d01bac5498f48662718c05e9c104ee35c8d07a4a6f6b679c4a25c1515 WatchSource:0}: Error finding container 20141c8d01bac5498f48662718c05e9c104ee35c8d07a4a6f6b679c4a25c1515: Status 404 returned error can't find the container with id 20141c8d01bac5498f48662718c05e9c104ee35c8d07a4a6f6b679c4a25c1515 Jul 12 17:53:16.183081 ip-10-0-129-128 kubenswrapper[2559]: W0712 17:53:16.183059 2559 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod64692aea_0243_4167_a3b6_2a6dc81a8fef.slice/crio-d617f1e08f1ceeabbc17c3dbc1acdb0cad7fa9353f0b757f34620425b078bbf2 WatchSource:0}: Error finding container d617f1e08f1ceeabbc17c3dbc1acdb0cad7fa9353f0b757f34620425b078bbf2: Status 404 returned error can't find the container with id d617f1e08f1ceeabbc17c3dbc1acdb0cad7fa9353f0b757f34620425b078bbf2 Jul 12 17:53:16.185085 ip-10-0-129-128 kubenswrapper[2559]: W0712 17:53:16.185061 2559 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod41eafd0c_4a0b_48f6_8e30_8e408412d389.slice/crio-06a669f3856ecf6907cc20943b47615a7e2a2b0295c57a708f017d6afe2a1551 WatchSource:0}: Error finding container 06a669f3856ecf6907cc20943b47615a7e2a2b0295c57a708f017d6afe2a1551: Status 404 returned error can't find the container with id 06a669f3856ecf6907cc20943b47615a7e2a2b0295c57a708f017d6afe2a1551 Jul 12 17:53:16.186435 ip-10-0-129-128 kubenswrapper[2559]: W0712 17:53:16.185775 2559 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod5841758e_4587_4d16_807c_b4a40b824c1b.slice/crio-d8617c16db10b472ec45484844b92b9da20d793ec33d643a71466a15782865e9 WatchSource:0}: Error finding container d8617c16db10b472ec45484844b92b9da20d793ec33d643a71466a15782865e9: Status 404 returned error can't find the container with id d8617c16db10b472ec45484844b92b9da20d793ec33d643a71466a15782865e9 Jul 12 17:53:16.638980 ip-10-0-129-128 kubenswrapper[2559]: I0712 17:53:16.638626 2559 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/6f65bad8-9ec7-4a51-8557-74b3f79ef76c-metrics-certs\") pod \"network-metrics-daemon-qtkvt\" (UID: \"6f65bad8-9ec7-4a51-8557-74b3f79ef76c\") " pod="openshift-multus/network-metrics-daemon-qtkvt" Jul 12 17:53:16.638980 ip-10-0-129-128 kubenswrapper[2559]: I0712 17:53:16.638933 2559 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"original-pull-secret\" (UniqueName: \"kubernetes.io/secret/6ab3daa1-3376-4f53-8106-d6d7417a327e-original-pull-secret\") pod \"global-pull-secret-syncer-mpqvv\" (UID: \"6ab3daa1-3376-4f53-8106-d6d7417a327e\") " pod="kube-system/global-pull-secret-syncer-mpqvv" Jul 12 17:53:16.639179 ip-10-0-129-128 kubenswrapper[2559]: E0712 17:53:16.638791 2559 secret.go:189] Couldn't get secret openshift-multus/metrics-daemon-secret: object "openshift-multus"/"metrics-daemon-secret" not registered Jul 12 17:53:16.639179 ip-10-0-129-128 kubenswrapper[2559]: E0712 17:53:16.639130 2559 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/6f65bad8-9ec7-4a51-8557-74b3f79ef76c-metrics-certs podName:6f65bad8-9ec7-4a51-8557-74b3f79ef76c nodeName:}" failed. No retries permitted until 2026-07-12 17:53:18.639108698 +0000 UTC m=+6.133176610 (durationBeforeRetry 2s). Error: MountVolume.SetUp failed for volume "metrics-certs" (UniqueName: "kubernetes.io/secret/6f65bad8-9ec7-4a51-8557-74b3f79ef76c-metrics-certs") pod "network-metrics-daemon-qtkvt" (UID: "6f65bad8-9ec7-4a51-8557-74b3f79ef76c") : object "openshift-multus"/"metrics-daemon-secret" not registered Jul 12 17:53:16.639411 ip-10-0-129-128 kubenswrapper[2559]: E0712 17:53:16.639062 2559 secret.go:189] Couldn't get secret kube-system/original-pull-secret: object "kube-system"/"original-pull-secret" not registered Jul 12 17:53:16.639411 ip-10-0-129-128 kubenswrapper[2559]: E0712 17:53:16.639373 2559 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/6ab3daa1-3376-4f53-8106-d6d7417a327e-original-pull-secret podName:6ab3daa1-3376-4f53-8106-d6d7417a327e nodeName:}" failed. No retries permitted until 2026-07-12 17:53:18.639352206 +0000 UTC m=+6.133420118 (durationBeforeRetry 2s). Error: MountVolume.SetUp failed for volume "original-pull-secret" (UniqueName: "kubernetes.io/secret/6ab3daa1-3376-4f53-8106-d6d7417a327e-original-pull-secret") pod "global-pull-secret-syncer-mpqvv" (UID: "6ab3daa1-3376-4f53-8106-d6d7417a327e") : object "kube-system"/"original-pull-secret" not registered Jul 12 17:53:16.740397 ip-10-0-129-128 kubenswrapper[2559]: I0712 17:53:16.739788 2559 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-bn2rx\" (UniqueName: \"kubernetes.io/projected/e2c3a206-19c0-4d4d-bb98-a99190023ad4-kube-api-access-bn2rx\") pod \"network-check-target-h4bgw\" (UID: \"e2c3a206-19c0-4d4d-bb98-a99190023ad4\") " pod="openshift-network-diagnostics/network-check-target-h4bgw" Jul 12 17:53:16.740397 ip-10-0-129-128 kubenswrapper[2559]: E0712 17:53:16.739981 2559 projected.go:291] Couldn't get configMap openshift-network-diagnostics/kube-root-ca.crt: object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered Jul 12 17:53:16.740397 ip-10-0-129-128 kubenswrapper[2559]: E0712 17:53:16.740003 2559 projected.go:291] Couldn't get configMap openshift-network-diagnostics/openshift-service-ca.crt: object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered Jul 12 17:53:16.740397 ip-10-0-129-128 kubenswrapper[2559]: E0712 17:53:16.740015 2559 projected.go:196] Error preparing data for projected volume kube-api-access-bn2rx for pod openshift-network-diagnostics/network-check-target-h4bgw: [object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered, object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered] Jul 12 17:53:16.740397 ip-10-0-129-128 kubenswrapper[2559]: E0712 17:53:16.740070 2559 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/projected/e2c3a206-19c0-4d4d-bb98-a99190023ad4-kube-api-access-bn2rx podName:e2c3a206-19c0-4d4d-bb98-a99190023ad4 nodeName:}" failed. No retries permitted until 2026-07-12 17:53:18.740052455 +0000 UTC m=+6.234120376 (durationBeforeRetry 2s). Error: MountVolume.SetUp failed for volume "kube-api-access-bn2rx" (UniqueName: "kubernetes.io/projected/e2c3a206-19c0-4d4d-bb98-a99190023ad4-kube-api-access-bn2rx") pod "network-check-target-h4bgw" (UID: "e2c3a206-19c0-4d4d-bb98-a99190023ad4") : [object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered, object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered] Jul 12 17:53:16.948854 ip-10-0-129-128 kubenswrapper[2559]: I0712 17:53:16.948725 2559 certificate_manager.go:715] "Certificate rotation deadline determined" logger="kubernetes.io/kubelet-serving" expiration="2028-07-11 17:48:14 +0000 UTC" deadline="2028-03-14 06:34:04.959675095 +0000 UTC" Jul 12 17:53:16.948854 ip-10-0-129-128 kubenswrapper[2559]: I0712 17:53:16.948783 2559 certificate_manager.go:431] "Waiting for next certificate rotation" logger="kubernetes.io/kubelet-serving" sleep="14652h40m48.010896383s" Jul 12 17:53:17.029277 ip-10-0-129-128 kubenswrapper[2559]: I0712 17:53:17.028763 2559 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-qtkvt" Jul 12 17:53:17.029277 ip-10-0-129-128 kubenswrapper[2559]: I0712 17:53:17.028763 2559 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-h4bgw" Jul 12 17:53:17.029277 ip-10-0-129-128 kubenswrapper[2559]: E0712 17:53:17.028895 2559 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-qtkvt" podUID="6f65bad8-9ec7-4a51-8557-74b3f79ef76c" Jul 12 17:53:17.029277 ip-10-0-129-128 kubenswrapper[2559]: E0712 17:53:17.028965 2559 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-h4bgw" podUID="e2c3a206-19c0-4d4d-bb98-a99190023ad4" Jul 12 17:53:17.052632 ip-10-0-129-128 kubenswrapper[2559]: I0712 17:53:17.052545 2559 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-network-operator/iptables-alerter-fql27" event={"ID":"5841758e-4587-4d16-807c-b4a40b824c1b","Type":"ContainerStarted","Data":"d8617c16db10b472ec45484844b92b9da20d793ec33d643a71466a15782865e9"} Jul 12 17:53:17.057197 ip-10-0-129-128 kubenswrapper[2559]: I0712 17:53:17.057163 2559 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-dns/node-resolver-fqnvr" event={"ID":"41eafd0c-4a0b-48f6-8e30-8e408412d389","Type":"ContainerStarted","Data":"06a669f3856ecf6907cc20943b47615a7e2a2b0295c57a708f017d6afe2a1551"} Jul 12 17:53:17.063200 ip-10-0-129-128 kubenswrapper[2559]: I0712 17:53:17.063174 2559 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kube-system/konnectivity-agent-g6rx5" event={"ID":"64692aea-0243-4167-a3b6-2a6dc81a8fef","Type":"ContainerStarted","Data":"d617f1e08f1ceeabbc17c3dbc1acdb0cad7fa9353f0b757f34620425b078bbf2"} Jul 12 17:53:17.066564 ip-10-0-129-128 kubenswrapper[2559]: I0712 17:53:17.066535 2559 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-multus/multus-c2jnt" event={"ID":"ff97f946-352d-4629-bf96-351edf53ebde","Type":"ContainerStarted","Data":"02a02511bb91ee7e521f307ff978f3e605bc877b581f958d4d53d65c5f3be485"} Jul 12 17:53:17.071338 ip-10-0-129-128 kubenswrapper[2559]: I0712 17:53:17.071290 2559 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-cluster-node-tuning-operator/tuned-66xkf" event={"ID":"cf65b059-f8a2-48d6-a773-74f51ac6ae5d","Type":"ContainerStarted","Data":"528d7dbfdc90533e163810243ea8d70c941c3427a3e6b3d7064b860667c1bc3e"} Jul 12 17:53:17.080686 ip-10-0-129-128 kubenswrapper[2559]: I0712 17:53:17.080155 2559 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kube-system/kube-apiserver-proxy-ip-10-0-129-128.ec2.internal" event={"ID":"89625644b4c452349901306bb1be9b95","Type":"ContainerStarted","Data":"3997aebbd2d0b90610e5c2187b4141ac0af9b23d0d84a639cf306ab74ed52267"} Jul 12 17:53:17.090124 ip-10-0-129-128 kubenswrapper[2559]: I0712 17:53:17.090097 2559 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-m2r7q" event={"ID":"3e848887-84a9-4599-afc5-c3750c0857a0","Type":"ContainerStarted","Data":"20141c8d01bac5498f48662718c05e9c104ee35c8d07a4a6f6b679c4a25c1515"} Jul 12 17:53:17.108118 ip-10-0-129-128 kubenswrapper[2559]: I0712 17:53:17.108086 2559 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-multus/multus-additional-cni-plugins-9dh8l" event={"ID":"370d4e7b-6713-48b2-81ee-f055fe1d5490","Type":"ContainerStarted","Data":"8cb892eb901ca55d0111dcffe37b23f8492f9bd5b7ed7125e2fda8a588993736"} Jul 12 17:53:17.118163 ip-10-0-129-128 kubenswrapper[2559]: I0712 17:53:17.118132 2559 generic.go:356] "Generic (PLEG): container finished" podID="862cce2888b36ce6c9c6601a2547ab41" containerID="c4c037a315dfc69328cb1e687c6a7804acd0bd4ae169b6ba64cf372721225639" exitCode=0 Jul 12 17:53:17.118285 ip-10-0-129-128 kubenswrapper[2559]: I0712 17:53:17.118215 2559 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-129-128.ec2.internal" event={"ID":"862cce2888b36ce6c9c6601a2547ab41","Type":"ContainerDied","Data":"c4c037a315dfc69328cb1e687c6a7804acd0bd4ae169b6ba64cf372721225639"} Jul 12 17:53:17.125516 ip-10-0-129-128 kubenswrapper[2559]: I0712 17:53:17.125404 2559 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ovn-kubernetes/ovnkube-node-q5dw8" event={"ID":"e51bad50-bc74-4e38-8307-ca257b599314","Type":"ContainerStarted","Data":"d06e305a78078ae1bb46bfb21c37118c244df90b93a456e6f50942a37524f3fd"} Jul 12 17:53:17.127626 ip-10-0-129-128 kubenswrapper[2559]: I0712 17:53:17.127599 2559 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-image-registry/node-ca-vxx7w" event={"ID":"6d23c7c4-9eb6-483f-baf3-f25e641e2ead","Type":"ContainerStarted","Data":"acb60d9c3b7eb7224b1ce6a874f12aa7007b582fa170d45ca2a2a022dc176b01"} Jul 12 17:53:17.135222 ip-10-0-129-128 kubenswrapper[2559]: I0712 17:53:17.135085 2559 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="kube-system/kube-apiserver-proxy-ip-10-0-129-128.ec2.internal" podStartSLOduration=3.135066646 podStartE2EDuration="3.135066646s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=true podCreationTimestamp="2026-07-12 17:53:14 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-12 17:53:17.09630206 +0000 UTC m=+4.590369991" watchObservedRunningTime="2026-07-12 17:53:17.135066646 +0000 UTC m=+4.629134579" Jul 12 17:53:18.028293 ip-10-0-129-128 kubenswrapper[2559]: I0712 17:53:18.028219 2559 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-mpqvv" Jul 12 17:53:18.028445 ip-10-0-129-128 kubenswrapper[2559]: E0712 17:53:18.028352 2559 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-mpqvv" podUID="6ab3daa1-3376-4f53-8106-d6d7417a327e" Jul 12 17:53:18.159831 ip-10-0-129-128 kubenswrapper[2559]: I0712 17:53:18.159344 2559 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-129-128.ec2.internal" event={"ID":"862cce2888b36ce6c9c6601a2547ab41","Type":"ContainerStarted","Data":"dee65ca91b56121c8a494e54c28e413fefc265b37a120902c10bede270b126c8"} Jul 12 17:53:18.657489 ip-10-0-129-128 kubenswrapper[2559]: I0712 17:53:18.656929 2559 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/6f65bad8-9ec7-4a51-8557-74b3f79ef76c-metrics-certs\") pod \"network-metrics-daemon-qtkvt\" (UID: \"6f65bad8-9ec7-4a51-8557-74b3f79ef76c\") " pod="openshift-multus/network-metrics-daemon-qtkvt" Jul 12 17:53:18.657489 ip-10-0-129-128 kubenswrapper[2559]: I0712 17:53:18.656977 2559 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"original-pull-secret\" (UniqueName: \"kubernetes.io/secret/6ab3daa1-3376-4f53-8106-d6d7417a327e-original-pull-secret\") pod \"global-pull-secret-syncer-mpqvv\" (UID: \"6ab3daa1-3376-4f53-8106-d6d7417a327e\") " pod="kube-system/global-pull-secret-syncer-mpqvv" Jul 12 17:53:18.657489 ip-10-0-129-128 kubenswrapper[2559]: E0712 17:53:18.657134 2559 secret.go:189] Couldn't get secret kube-system/original-pull-secret: object "kube-system"/"original-pull-secret" not registered Jul 12 17:53:18.657489 ip-10-0-129-128 kubenswrapper[2559]: E0712 17:53:18.657184 2559 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/6ab3daa1-3376-4f53-8106-d6d7417a327e-original-pull-secret podName:6ab3daa1-3376-4f53-8106-d6d7417a327e nodeName:}" failed. No retries permitted until 2026-07-12 17:53:22.657172254 +0000 UTC m=+10.151240163 (durationBeforeRetry 4s). Error: MountVolume.SetUp failed for volume "original-pull-secret" (UniqueName: "kubernetes.io/secret/6ab3daa1-3376-4f53-8106-d6d7417a327e-original-pull-secret") pod "global-pull-secret-syncer-mpqvv" (UID: "6ab3daa1-3376-4f53-8106-d6d7417a327e") : object "kube-system"/"original-pull-secret" not registered Jul 12 17:53:18.657489 ip-10-0-129-128 kubenswrapper[2559]: E0712 17:53:18.657227 2559 secret.go:189] Couldn't get secret openshift-multus/metrics-daemon-secret: object "openshift-multus"/"metrics-daemon-secret" not registered Jul 12 17:53:18.657489 ip-10-0-129-128 kubenswrapper[2559]: E0712 17:53:18.657247 2559 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/6f65bad8-9ec7-4a51-8557-74b3f79ef76c-metrics-certs podName:6f65bad8-9ec7-4a51-8557-74b3f79ef76c nodeName:}" failed. No retries permitted until 2026-07-12 17:53:22.657240532 +0000 UTC m=+10.151308440 (durationBeforeRetry 4s). Error: MountVolume.SetUp failed for volume "metrics-certs" (UniqueName: "kubernetes.io/secret/6f65bad8-9ec7-4a51-8557-74b3f79ef76c-metrics-certs") pod "network-metrics-daemon-qtkvt" (UID: "6f65bad8-9ec7-4a51-8557-74b3f79ef76c") : object "openshift-multus"/"metrics-daemon-secret" not registered Jul 12 17:53:18.757651 ip-10-0-129-128 kubenswrapper[2559]: I0712 17:53:18.757558 2559 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-bn2rx\" (UniqueName: \"kubernetes.io/projected/e2c3a206-19c0-4d4d-bb98-a99190023ad4-kube-api-access-bn2rx\") pod \"network-check-target-h4bgw\" (UID: \"e2c3a206-19c0-4d4d-bb98-a99190023ad4\") " pod="openshift-network-diagnostics/network-check-target-h4bgw" Jul 12 17:53:18.757826 ip-10-0-129-128 kubenswrapper[2559]: E0712 17:53:18.757762 2559 projected.go:291] Couldn't get configMap openshift-network-diagnostics/kube-root-ca.crt: object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered Jul 12 17:53:18.757826 ip-10-0-129-128 kubenswrapper[2559]: E0712 17:53:18.757787 2559 projected.go:291] Couldn't get configMap openshift-network-diagnostics/openshift-service-ca.crt: object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered Jul 12 17:53:18.757826 ip-10-0-129-128 kubenswrapper[2559]: E0712 17:53:18.757801 2559 projected.go:196] Error preparing data for projected volume kube-api-access-bn2rx for pod openshift-network-diagnostics/network-check-target-h4bgw: [object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered, object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered] Jul 12 17:53:18.757986 ip-10-0-129-128 kubenswrapper[2559]: E0712 17:53:18.757860 2559 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/projected/e2c3a206-19c0-4d4d-bb98-a99190023ad4-kube-api-access-bn2rx podName:e2c3a206-19c0-4d4d-bb98-a99190023ad4 nodeName:}" failed. No retries permitted until 2026-07-12 17:53:22.757841247 +0000 UTC m=+10.251909160 (durationBeforeRetry 4s). Error: MountVolume.SetUp failed for volume "kube-api-access-bn2rx" (UniqueName: "kubernetes.io/projected/e2c3a206-19c0-4d4d-bb98-a99190023ad4-kube-api-access-bn2rx") pod "network-check-target-h4bgw" (UID: "e2c3a206-19c0-4d4d-bb98-a99190023ad4") : [object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered, object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered] Jul 12 17:53:19.030949 ip-10-0-129-128 kubenswrapper[2559]: I0712 17:53:19.030868 2559 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-qtkvt" Jul 12 17:53:19.031099 ip-10-0-129-128 kubenswrapper[2559]: E0712 17:53:19.031017 2559 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-qtkvt" podUID="6f65bad8-9ec7-4a51-8557-74b3f79ef76c" Jul 12 17:53:19.031378 ip-10-0-129-128 kubenswrapper[2559]: I0712 17:53:19.031356 2559 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-h4bgw" Jul 12 17:53:19.031470 ip-10-0-129-128 kubenswrapper[2559]: E0712 17:53:19.031451 2559 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-h4bgw" podUID="e2c3a206-19c0-4d4d-bb98-a99190023ad4" Jul 12 17:53:20.029437 ip-10-0-129-128 kubenswrapper[2559]: I0712 17:53:20.028931 2559 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-mpqvv" Jul 12 17:53:20.029437 ip-10-0-129-128 kubenswrapper[2559]: E0712 17:53:20.029072 2559 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-mpqvv" podUID="6ab3daa1-3376-4f53-8106-d6d7417a327e" Jul 12 17:53:21.028559 ip-10-0-129-128 kubenswrapper[2559]: I0712 17:53:21.028464 2559 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-qtkvt" Jul 12 17:53:21.028559 ip-10-0-129-128 kubenswrapper[2559]: I0712 17:53:21.028497 2559 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-h4bgw" Jul 12 17:53:21.028794 ip-10-0-129-128 kubenswrapper[2559]: E0712 17:53:21.028608 2559 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-qtkvt" podUID="6f65bad8-9ec7-4a51-8557-74b3f79ef76c" Jul 12 17:53:21.029056 ip-10-0-129-128 kubenswrapper[2559]: E0712 17:53:21.029026 2559 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-h4bgw" podUID="e2c3a206-19c0-4d4d-bb98-a99190023ad4" Jul 12 17:53:22.028115 ip-10-0-129-128 kubenswrapper[2559]: I0712 17:53:22.028084 2559 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-mpqvv" Jul 12 17:53:22.028572 ip-10-0-129-128 kubenswrapper[2559]: E0712 17:53:22.028225 2559 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-mpqvv" podUID="6ab3daa1-3376-4f53-8106-d6d7417a327e" Jul 12 17:53:22.687574 ip-10-0-129-128 kubenswrapper[2559]: I0712 17:53:22.687528 2559 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/6f65bad8-9ec7-4a51-8557-74b3f79ef76c-metrics-certs\") pod \"network-metrics-daemon-qtkvt\" (UID: \"6f65bad8-9ec7-4a51-8557-74b3f79ef76c\") " pod="openshift-multus/network-metrics-daemon-qtkvt" Jul 12 17:53:22.687773 ip-10-0-129-128 kubenswrapper[2559]: I0712 17:53:22.687583 2559 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"original-pull-secret\" (UniqueName: \"kubernetes.io/secret/6ab3daa1-3376-4f53-8106-d6d7417a327e-original-pull-secret\") pod \"global-pull-secret-syncer-mpqvv\" (UID: \"6ab3daa1-3376-4f53-8106-d6d7417a327e\") " pod="kube-system/global-pull-secret-syncer-mpqvv" Jul 12 17:53:22.687773 ip-10-0-129-128 kubenswrapper[2559]: E0712 17:53:22.687721 2559 secret.go:189] Couldn't get secret openshift-multus/metrics-daemon-secret: object "openshift-multus"/"metrics-daemon-secret" not registered Jul 12 17:53:22.687893 ip-10-0-129-128 kubenswrapper[2559]: E0712 17:53:22.687724 2559 secret.go:189] Couldn't get secret kube-system/original-pull-secret: object "kube-system"/"original-pull-secret" not registered Jul 12 17:53:22.687893 ip-10-0-129-128 kubenswrapper[2559]: E0712 17:53:22.687797 2559 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/6f65bad8-9ec7-4a51-8557-74b3f79ef76c-metrics-certs podName:6f65bad8-9ec7-4a51-8557-74b3f79ef76c nodeName:}" failed. No retries permitted until 2026-07-12 17:53:30.687774004 +0000 UTC m=+18.181841935 (durationBeforeRetry 8s). Error: MountVolume.SetUp failed for volume "metrics-certs" (UniqueName: "kubernetes.io/secret/6f65bad8-9ec7-4a51-8557-74b3f79ef76c-metrics-certs") pod "network-metrics-daemon-qtkvt" (UID: "6f65bad8-9ec7-4a51-8557-74b3f79ef76c") : object "openshift-multus"/"metrics-daemon-secret" not registered Jul 12 17:53:22.687893 ip-10-0-129-128 kubenswrapper[2559]: E0712 17:53:22.687827 2559 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/6ab3daa1-3376-4f53-8106-d6d7417a327e-original-pull-secret podName:6ab3daa1-3376-4f53-8106-d6d7417a327e nodeName:}" failed. No retries permitted until 2026-07-12 17:53:30.687810859 +0000 UTC m=+18.181878788 (durationBeforeRetry 8s). Error: MountVolume.SetUp failed for volume "original-pull-secret" (UniqueName: "kubernetes.io/secret/6ab3daa1-3376-4f53-8106-d6d7417a327e-original-pull-secret") pod "global-pull-secret-syncer-mpqvv" (UID: "6ab3daa1-3376-4f53-8106-d6d7417a327e") : object "kube-system"/"original-pull-secret" not registered Jul 12 17:53:22.788405 ip-10-0-129-128 kubenswrapper[2559]: I0712 17:53:22.788369 2559 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-bn2rx\" (UniqueName: \"kubernetes.io/projected/e2c3a206-19c0-4d4d-bb98-a99190023ad4-kube-api-access-bn2rx\") pod \"network-check-target-h4bgw\" (UID: \"e2c3a206-19c0-4d4d-bb98-a99190023ad4\") " pod="openshift-network-diagnostics/network-check-target-h4bgw" Jul 12 17:53:22.788574 ip-10-0-129-128 kubenswrapper[2559]: E0712 17:53:22.788526 2559 projected.go:291] Couldn't get configMap openshift-network-diagnostics/kube-root-ca.crt: object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered Jul 12 17:53:22.788574 ip-10-0-129-128 kubenswrapper[2559]: E0712 17:53:22.788547 2559 projected.go:291] Couldn't get configMap openshift-network-diagnostics/openshift-service-ca.crt: object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered Jul 12 17:53:22.788574 ip-10-0-129-128 kubenswrapper[2559]: E0712 17:53:22.788561 2559 projected.go:196] Error preparing data for projected volume kube-api-access-bn2rx for pod openshift-network-diagnostics/network-check-target-h4bgw: [object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered, object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered] Jul 12 17:53:22.788757 ip-10-0-129-128 kubenswrapper[2559]: E0712 17:53:22.788624 2559 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/projected/e2c3a206-19c0-4d4d-bb98-a99190023ad4-kube-api-access-bn2rx podName:e2c3a206-19c0-4d4d-bb98-a99190023ad4 nodeName:}" failed. No retries permitted until 2026-07-12 17:53:30.788604496 +0000 UTC m=+18.282672408 (durationBeforeRetry 8s). Error: MountVolume.SetUp failed for volume "kube-api-access-bn2rx" (UniqueName: "kubernetes.io/projected/e2c3a206-19c0-4d4d-bb98-a99190023ad4-kube-api-access-bn2rx") pod "network-check-target-h4bgw" (UID: "e2c3a206-19c0-4d4d-bb98-a99190023ad4") : [object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered, object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered] Jul 12 17:53:23.032293 ip-10-0-129-128 kubenswrapper[2559]: I0712 17:53:23.031548 2559 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-qtkvt" Jul 12 17:53:23.032293 ip-10-0-129-128 kubenswrapper[2559]: E0712 17:53:23.031679 2559 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-qtkvt" podUID="6f65bad8-9ec7-4a51-8557-74b3f79ef76c" Jul 12 17:53:23.032293 ip-10-0-129-128 kubenswrapper[2559]: I0712 17:53:23.032022 2559 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-h4bgw" Jul 12 17:53:23.032293 ip-10-0-129-128 kubenswrapper[2559]: E0712 17:53:23.032107 2559 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-h4bgw" podUID="e2c3a206-19c0-4d4d-bb98-a99190023ad4" Jul 12 17:53:24.028857 ip-10-0-129-128 kubenswrapper[2559]: I0712 17:53:24.028816 2559 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-mpqvv" Jul 12 17:53:24.029092 ip-10-0-129-128 kubenswrapper[2559]: E0712 17:53:24.028958 2559 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-mpqvv" podUID="6ab3daa1-3376-4f53-8106-d6d7417a327e" Jul 12 17:53:25.028101 ip-10-0-129-128 kubenswrapper[2559]: I0712 17:53:25.028064 2559 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-qtkvt" Jul 12 17:53:25.028562 ip-10-0-129-128 kubenswrapper[2559]: E0712 17:53:25.028209 2559 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-qtkvt" podUID="6f65bad8-9ec7-4a51-8557-74b3f79ef76c" Jul 12 17:53:25.028734 ip-10-0-129-128 kubenswrapper[2559]: I0712 17:53:25.028709 2559 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-h4bgw" Jul 12 17:53:25.028827 ip-10-0-129-128 kubenswrapper[2559]: E0712 17:53:25.028807 2559 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-h4bgw" podUID="e2c3a206-19c0-4d4d-bb98-a99190023ad4" Jul 12 17:53:26.028390 ip-10-0-129-128 kubenswrapper[2559]: I0712 17:53:26.028358 2559 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-mpqvv" Jul 12 17:53:26.028761 ip-10-0-129-128 kubenswrapper[2559]: E0712 17:53:26.028474 2559 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-mpqvv" podUID="6ab3daa1-3376-4f53-8106-d6d7417a327e" Jul 12 17:53:27.028743 ip-10-0-129-128 kubenswrapper[2559]: I0712 17:53:27.028725 2559 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-qtkvt" Jul 12 17:53:27.029037 ip-10-0-129-128 kubenswrapper[2559]: I0712 17:53:27.028725 2559 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-h4bgw" Jul 12 17:53:27.029037 ip-10-0-129-128 kubenswrapper[2559]: E0712 17:53:27.028834 2559 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-qtkvt" podUID="6f65bad8-9ec7-4a51-8557-74b3f79ef76c" Jul 12 17:53:27.029037 ip-10-0-129-128 kubenswrapper[2559]: E0712 17:53:27.028914 2559 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-h4bgw" podUID="e2c3a206-19c0-4d4d-bb98-a99190023ad4" Jul 12 17:53:28.029063 ip-10-0-129-128 kubenswrapper[2559]: I0712 17:53:28.029025 2559 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-mpqvv" Jul 12 17:53:28.029492 ip-10-0-129-128 kubenswrapper[2559]: E0712 17:53:28.029141 2559 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-mpqvv" podUID="6ab3daa1-3376-4f53-8106-d6d7417a327e" Jul 12 17:53:29.028290 ip-10-0-129-128 kubenswrapper[2559]: I0712 17:53:29.028252 2559 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-qtkvt" Jul 12 17:53:29.028290 ip-10-0-129-128 kubenswrapper[2559]: I0712 17:53:29.028296 2559 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-h4bgw" Jul 12 17:53:29.028523 ip-10-0-129-128 kubenswrapper[2559]: E0712 17:53:29.028400 2559 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-qtkvt" podUID="6f65bad8-9ec7-4a51-8557-74b3f79ef76c" Jul 12 17:53:29.028575 ip-10-0-129-128 kubenswrapper[2559]: E0712 17:53:29.028543 2559 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-h4bgw" podUID="e2c3a206-19c0-4d4d-bb98-a99190023ad4" Jul 12 17:53:30.028252 ip-10-0-129-128 kubenswrapper[2559]: I0712 17:53:30.028210 2559 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-mpqvv" Jul 12 17:53:30.028707 ip-10-0-129-128 kubenswrapper[2559]: E0712 17:53:30.028353 2559 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-mpqvv" podUID="6ab3daa1-3376-4f53-8106-d6d7417a327e" Jul 12 17:53:30.751185 ip-10-0-129-128 kubenswrapper[2559]: I0712 17:53:30.751155 2559 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/6f65bad8-9ec7-4a51-8557-74b3f79ef76c-metrics-certs\") pod \"network-metrics-daemon-qtkvt\" (UID: \"6f65bad8-9ec7-4a51-8557-74b3f79ef76c\") " pod="openshift-multus/network-metrics-daemon-qtkvt" Jul 12 17:53:30.751185 ip-10-0-129-128 kubenswrapper[2559]: I0712 17:53:30.751192 2559 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"original-pull-secret\" (UniqueName: \"kubernetes.io/secret/6ab3daa1-3376-4f53-8106-d6d7417a327e-original-pull-secret\") pod \"global-pull-secret-syncer-mpqvv\" (UID: \"6ab3daa1-3376-4f53-8106-d6d7417a327e\") " pod="kube-system/global-pull-secret-syncer-mpqvv" Jul 12 17:53:30.751401 ip-10-0-129-128 kubenswrapper[2559]: E0712 17:53:30.751308 2559 secret.go:189] Couldn't get secret kube-system/original-pull-secret: object "kube-system"/"original-pull-secret" not registered Jul 12 17:53:30.751401 ip-10-0-129-128 kubenswrapper[2559]: E0712 17:53:30.751312 2559 secret.go:189] Couldn't get secret openshift-multus/metrics-daemon-secret: object "openshift-multus"/"metrics-daemon-secret" not registered Jul 12 17:53:30.751401 ip-10-0-129-128 kubenswrapper[2559]: E0712 17:53:30.751373 2559 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/6ab3daa1-3376-4f53-8106-d6d7417a327e-original-pull-secret podName:6ab3daa1-3376-4f53-8106-d6d7417a327e nodeName:}" failed. No retries permitted until 2026-07-12 17:53:46.751354802 +0000 UTC m=+34.245422712 (durationBeforeRetry 16s). Error: MountVolume.SetUp failed for volume "original-pull-secret" (UniqueName: "kubernetes.io/secret/6ab3daa1-3376-4f53-8106-d6d7417a327e-original-pull-secret") pod "global-pull-secret-syncer-mpqvv" (UID: "6ab3daa1-3376-4f53-8106-d6d7417a327e") : object "kube-system"/"original-pull-secret" not registered Jul 12 17:53:30.751401 ip-10-0-129-128 kubenswrapper[2559]: E0712 17:53:30.751393 2559 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/6f65bad8-9ec7-4a51-8557-74b3f79ef76c-metrics-certs podName:6f65bad8-9ec7-4a51-8557-74b3f79ef76c nodeName:}" failed. No retries permitted until 2026-07-12 17:53:46.751382769 +0000 UTC m=+34.245450682 (durationBeforeRetry 16s). Error: MountVolume.SetUp failed for volume "metrics-certs" (UniqueName: "kubernetes.io/secret/6f65bad8-9ec7-4a51-8557-74b3f79ef76c-metrics-certs") pod "network-metrics-daemon-qtkvt" (UID: "6f65bad8-9ec7-4a51-8557-74b3f79ef76c") : object "openshift-multus"/"metrics-daemon-secret" not registered Jul 12 17:53:30.852096 ip-10-0-129-128 kubenswrapper[2559]: I0712 17:53:30.852064 2559 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-bn2rx\" (UniqueName: \"kubernetes.io/projected/e2c3a206-19c0-4d4d-bb98-a99190023ad4-kube-api-access-bn2rx\") pod \"network-check-target-h4bgw\" (UID: \"e2c3a206-19c0-4d4d-bb98-a99190023ad4\") " pod="openshift-network-diagnostics/network-check-target-h4bgw" Jul 12 17:53:30.852268 ip-10-0-129-128 kubenswrapper[2559]: E0712 17:53:30.852248 2559 projected.go:291] Couldn't get configMap openshift-network-diagnostics/kube-root-ca.crt: object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered Jul 12 17:53:30.852310 ip-10-0-129-128 kubenswrapper[2559]: E0712 17:53:30.852274 2559 projected.go:291] Couldn't get configMap openshift-network-diagnostics/openshift-service-ca.crt: object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered Jul 12 17:53:30.852310 ip-10-0-129-128 kubenswrapper[2559]: E0712 17:53:30.852284 2559 projected.go:196] Error preparing data for projected volume kube-api-access-bn2rx for pod openshift-network-diagnostics/network-check-target-h4bgw: [object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered, object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered] Jul 12 17:53:30.852399 ip-10-0-129-128 kubenswrapper[2559]: E0712 17:53:30.852338 2559 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/projected/e2c3a206-19c0-4d4d-bb98-a99190023ad4-kube-api-access-bn2rx podName:e2c3a206-19c0-4d4d-bb98-a99190023ad4 nodeName:}" failed. No retries permitted until 2026-07-12 17:53:46.852324268 +0000 UTC m=+34.346392176 (durationBeforeRetry 16s). Error: MountVolume.SetUp failed for volume "kube-api-access-bn2rx" (UniqueName: "kubernetes.io/projected/e2c3a206-19c0-4d4d-bb98-a99190023ad4-kube-api-access-bn2rx") pod "network-check-target-h4bgw" (UID: "e2c3a206-19c0-4d4d-bb98-a99190023ad4") : [object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered, object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered] Jul 12 17:53:31.028889 ip-10-0-129-128 kubenswrapper[2559]: I0712 17:53:31.028802 2559 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-h4bgw" Jul 12 17:53:31.029276 ip-10-0-129-128 kubenswrapper[2559]: I0712 17:53:31.028804 2559 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-qtkvt" Jul 12 17:53:31.029276 ip-10-0-129-128 kubenswrapper[2559]: E0712 17:53:31.028927 2559 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-h4bgw" podUID="e2c3a206-19c0-4d4d-bb98-a99190023ad4" Jul 12 17:53:31.029276 ip-10-0-129-128 kubenswrapper[2559]: E0712 17:53:31.029044 2559 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-qtkvt" podUID="6f65bad8-9ec7-4a51-8557-74b3f79ef76c" Jul 12 17:53:32.028169 ip-10-0-129-128 kubenswrapper[2559]: I0712 17:53:32.028128 2559 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-mpqvv" Jul 12 17:53:32.028336 ip-10-0-129-128 kubenswrapper[2559]: E0712 17:53:32.028255 2559 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-mpqvv" podUID="6ab3daa1-3376-4f53-8106-d6d7417a327e" Jul 12 17:53:33.029097 ip-10-0-129-128 kubenswrapper[2559]: I0712 17:53:33.029016 2559 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-qtkvt" Jul 12 17:53:33.029537 ip-10-0-129-128 kubenswrapper[2559]: E0712 17:53:33.029135 2559 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-qtkvt" podUID="6f65bad8-9ec7-4a51-8557-74b3f79ef76c" Jul 12 17:53:33.029537 ip-10-0-129-128 kubenswrapper[2559]: I0712 17:53:33.029175 2559 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-h4bgw" Jul 12 17:53:33.029537 ip-10-0-129-128 kubenswrapper[2559]: E0712 17:53:33.029242 2559 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-h4bgw" podUID="e2c3a206-19c0-4d4d-bb98-a99190023ad4" Jul 12 17:53:34.029001 ip-10-0-129-128 kubenswrapper[2559]: I0712 17:53:34.028967 2559 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-mpqvv" Jul 12 17:53:34.029194 ip-10-0-129-128 kubenswrapper[2559]: E0712 17:53:34.029099 2559 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-mpqvv" podUID="6ab3daa1-3376-4f53-8106-d6d7417a327e" Jul 12 17:53:35.029555 ip-10-0-129-128 kubenswrapper[2559]: I0712 17:53:35.028920 2559 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-qtkvt" Jul 12 17:53:35.029555 ip-10-0-129-128 kubenswrapper[2559]: I0712 17:53:35.029250 2559 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-h4bgw" Jul 12 17:53:35.029555 ip-10-0-129-128 kubenswrapper[2559]: E0712 17:53:35.029246 2559 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-qtkvt" podUID="6f65bad8-9ec7-4a51-8557-74b3f79ef76c" Jul 12 17:53:35.029555 ip-10-0-129-128 kubenswrapper[2559]: E0712 17:53:35.029330 2559 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-h4bgw" podUID="e2c3a206-19c0-4d4d-bb98-a99190023ad4" Jul 12 17:53:35.195026 ip-10-0-129-128 kubenswrapper[2559]: I0712 17:53:35.194990 2559 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-dns/node-resolver-fqnvr" event={"ID":"41eafd0c-4a0b-48f6-8e30-8e408412d389","Type":"ContainerStarted","Data":"fc95136fdd7fdccb94728530ddf416192aa6e93984f7819e8d80b2aa97814de1"} Jul 12 17:53:35.198260 ip-10-0-129-128 kubenswrapper[2559]: I0712 17:53:35.198230 2559 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kube-system/konnectivity-agent-g6rx5" event={"ID":"64692aea-0243-4167-a3b6-2a6dc81a8fef","Type":"ContainerStarted","Data":"b0f7c55fa18026c0bf5b2a842ccecbea6b3d56d089d3b1aa9a43903be79d8ac2"} Jul 12 17:53:35.200317 ip-10-0-129-128 kubenswrapper[2559]: I0712 17:53:35.200289 2559 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-multus/multus-c2jnt" event={"ID":"ff97f946-352d-4629-bf96-351edf53ebde","Type":"ContainerStarted","Data":"72729c0155a2e32c88ed0f71e020fecfa9c5107802f2e4ac25e997e732624570"} Jul 12 17:53:35.202237 ip-10-0-129-128 kubenswrapper[2559]: I0712 17:53:35.201943 2559 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-cluster-node-tuning-operator/tuned-66xkf" event={"ID":"cf65b059-f8a2-48d6-a773-74f51ac6ae5d","Type":"ContainerStarted","Data":"2623c807cbeb95f3ef0eb4e49681dea38b07e474e2ed9f04fc7a55a9a224f864"} Jul 12 17:53:35.203467 ip-10-0-129-128 kubenswrapper[2559]: I0712 17:53:35.203442 2559 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-m2r7q" event={"ID":"3e848887-84a9-4599-afc5-c3750c0857a0","Type":"ContainerStarted","Data":"55aa5e63997f8f0fca8e6fb2796bdeafdad489a72301cfc94be5baa8c63e80b7"} Jul 12 17:53:35.205276 ip-10-0-129-128 kubenswrapper[2559]: I0712 17:53:35.205080 2559 generic.go:356] "Generic (PLEG): container finished" podID="370d4e7b-6713-48b2-81ee-f055fe1d5490" containerID="940c32452e57e54c41688cdd9381b5ddd0d98cf7d27e1c31e49936c82596758c" exitCode=0 Jul 12 17:53:35.205276 ip-10-0-129-128 kubenswrapper[2559]: I0712 17:53:35.205159 2559 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-multus/multus-additional-cni-plugins-9dh8l" event={"ID":"370d4e7b-6713-48b2-81ee-f055fe1d5490","Type":"ContainerDied","Data":"940c32452e57e54c41688cdd9381b5ddd0d98cf7d27e1c31e49936c82596758c"} Jul 12 17:53:35.207121 ip-10-0-129-128 kubenswrapper[2559]: I0712 17:53:35.207106 2559 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ovn-kubernetes_ovnkube-node-q5dw8_e51bad50-bc74-4e38-8307-ca257b599314/ovn-acl-logging/0.log" Jul 12 17:53:35.207423 ip-10-0-129-128 kubenswrapper[2559]: I0712 17:53:35.207404 2559 generic.go:356] "Generic (PLEG): container finished" podID="e51bad50-bc74-4e38-8307-ca257b599314" containerID="748f49c843d400251e359f833789d2d460aa7729609977562eb2977a7ab7ff25" exitCode=1 Jul 12 17:53:35.207501 ip-10-0-129-128 kubenswrapper[2559]: I0712 17:53:35.207464 2559 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ovn-kubernetes/ovnkube-node-q5dw8" event={"ID":"e51bad50-bc74-4e38-8307-ca257b599314","Type":"ContainerStarted","Data":"72c9c031c83e4bb55114d8cb56fcd35d24cc2e6cdee7e64819f605bfe73f1490"} Jul 12 17:53:35.207548 ip-10-0-129-128 kubenswrapper[2559]: I0712 17:53:35.207510 2559 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ovn-kubernetes/ovnkube-node-q5dw8" event={"ID":"e51bad50-bc74-4e38-8307-ca257b599314","Type":"ContainerDied","Data":"748f49c843d400251e359f833789d2d460aa7729609977562eb2977a7ab7ff25"} Jul 12 17:53:35.207603 ip-10-0-129-128 kubenswrapper[2559]: I0712 17:53:35.207553 2559 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ovn-kubernetes/ovnkube-node-q5dw8" event={"ID":"e51bad50-bc74-4e38-8307-ca257b599314","Type":"ContainerStarted","Data":"c6d55a93bd51ad9a6920b4767694a906c93130e3a2bcbde5326e95e1d67153b2"} Jul 12 17:53:35.208714 ip-10-0-129-128 kubenswrapper[2559]: I0712 17:53:35.208694 2559 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-image-registry/node-ca-vxx7w" event={"ID":"6d23c7c4-9eb6-483f-baf3-f25e641e2ead","Type":"ContainerStarted","Data":"97e5e5e530b1b4316103e7ad9102df9c98470be14f3391cb2c35661d318c24a3"} Jul 12 17:53:35.211101 ip-10-0-129-128 kubenswrapper[2559]: I0712 17:53:35.211063 2559 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-dns/node-resolver-fqnvr" podStartSLOduration=3.555844846 podStartE2EDuration="22.211052054s" totalImagesPullingTime="18.655207208s" totalInitContainerRuntime="0s" isStatefulPod=true podCreationTimestamp="2026-07-12 17:53:13 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-12 17:53:35.210286788 +0000 UTC m=+22.704354715" watchObservedRunningTime="2026-07-12 17:53:35.211052054 +0000 UTC m=+22.705119984" Jul 12 17:53:35.211203 ip-10-0-129-128 kubenswrapper[2559]: I0712 17:53:35.211140 2559 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-129-128.ec2.internal" podStartSLOduration=21.211135376 podStartE2EDuration="21.211135376s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=true podCreationTimestamp="2026-07-12 17:53:14 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-12 17:53:18.175280283 +0000 UTC m=+5.669348214" watchObservedRunningTime="2026-07-12 17:53:35.211135376 +0000 UTC m=+22.705203310" Jul 12 17:53:35.224887 ip-10-0-129-128 kubenswrapper[2559]: I0712 17:53:35.224854 2559 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="kube-system/konnectivity-agent-g6rx5" podStartSLOduration=7.727171704 podStartE2EDuration="22.22484072s" totalImagesPullingTime="14.497669016s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-12 17:53:13 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-12 17:53:35.224413606 +0000 UTC m=+22.718481537" watchObservedRunningTime="2026-07-12 17:53:35.22484072 +0000 UTC m=+22.718908650" Jul 12 17:53:35.240940 ip-10-0-129-128 kubenswrapper[2559]: I0712 17:53:35.240897 2559 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-image-registry/node-ca-vxx7w" podStartSLOduration=3.582258962 podStartE2EDuration="22.240883536s" totalImagesPullingTime="18.658624574s" totalInitContainerRuntime="0s" isStatefulPod=true podCreationTimestamp="2026-07-12 17:53:13 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-12 17:53:35.240549945 +0000 UTC m=+22.734617875" watchObservedRunningTime="2026-07-12 17:53:35.240883536 +0000 UTC m=+22.734951465" Jul 12 17:53:35.257464 ip-10-0-129-128 kubenswrapper[2559]: I0712 17:53:35.257422 2559 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-multus/multus-c2jnt" podStartSLOduration=3.534127433 podStartE2EDuration="22.257408035s" totalImagesPullingTime="18.723280602s" totalInitContainerRuntime="0s" isStatefulPod=true podCreationTimestamp="2026-07-12 17:53:13 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-12 17:53:35.256853654 +0000 UTC m=+22.750921583" watchObservedRunningTime="2026-07-12 17:53:35.257408035 +0000 UTC m=+22.751475965" Jul 12 17:53:35.275507 ip-10-0-129-128 kubenswrapper[2559]: I0712 17:53:35.275468 2559 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-cluster-node-tuning-operator/tuned-66xkf" podStartSLOduration=3.594819973 podStartE2EDuration="22.275455781s" totalImagesPullingTime="18.680635808s" totalInitContainerRuntime="0s" isStatefulPod=true podCreationTimestamp="2026-07-12 17:53:13 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-12 17:53:35.275038676 +0000 UTC m=+22.769106607" watchObservedRunningTime="2026-07-12 17:53:35.275455781 +0000 UTC m=+22.769523710" Jul 12 17:53:36.028184 ip-10-0-129-128 kubenswrapper[2559]: I0712 17:53:36.028159 2559 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-mpqvv" Jul 12 17:53:36.028399 ip-10-0-129-128 kubenswrapper[2559]: E0712 17:53:36.028325 2559 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-mpqvv" podUID="6ab3daa1-3376-4f53-8106-d6d7417a327e" Jul 12 17:53:36.214231 ip-10-0-129-128 kubenswrapper[2559]: I0712 17:53:36.214204 2559 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ovn-kubernetes_ovnkube-node-q5dw8_e51bad50-bc74-4e38-8307-ca257b599314/ovn-acl-logging/0.log" Jul 12 17:53:36.214907 ip-10-0-129-128 kubenswrapper[2559]: I0712 17:53:36.214634 2559 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ovn-kubernetes/ovnkube-node-q5dw8" event={"ID":"e51bad50-bc74-4e38-8307-ca257b599314","Type":"ContainerStarted","Data":"001a11601be180faf2f7aed57ef54a07f9573137f2b73545df3a372b14cbdf15"} Jul 12 17:53:36.214907 ip-10-0-129-128 kubenswrapper[2559]: I0712 17:53:36.214711 2559 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ovn-kubernetes/ovnkube-node-q5dw8" event={"ID":"e51bad50-bc74-4e38-8307-ca257b599314","Type":"ContainerStarted","Data":"1a20777d668941d12fba4164b109cb55e4412c2290e1cde3089f8d3ff50c354b"} Jul 12 17:53:36.214907 ip-10-0-129-128 kubenswrapper[2559]: I0712 17:53:36.214760 2559 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ovn-kubernetes/ovnkube-node-q5dw8" event={"ID":"e51bad50-bc74-4e38-8307-ca257b599314","Type":"ContainerStarted","Data":"8aaf72819b40666c55ed9022e09a4d59671e96a0f24c8604bd4e04e88128e153"} Jul 12 17:53:36.216262 ip-10-0-129-128 kubenswrapper[2559]: I0712 17:53:36.216209 2559 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-network-operator/iptables-alerter-fql27" event={"ID":"5841758e-4587-4d16-807c-b4a40b824c1b","Type":"ContainerStarted","Data":"f391e6f0c6cb3f390ea35cbf7d0d6873e88b967dd2c8cc2c85fb0b3a38d46bbc"} Jul 12 17:53:36.231085 ip-10-0-129-128 kubenswrapper[2559]: I0712 17:53:36.231032 2559 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-network-operator/iptables-alerter-fql27" podStartSLOduration=4.573987838 podStartE2EDuration="23.231016331s" totalImagesPullingTime="18.657028493s" totalInitContainerRuntime="0s" isStatefulPod=true podCreationTimestamp="2026-07-12 17:53:13 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-12 17:53:36.230337714 +0000 UTC m=+23.724405644" watchObservedRunningTime="2026-07-12 17:53:36.231016331 +0000 UTC m=+23.725084262" Jul 12 17:53:36.441436 ip-10-0-129-128 kubenswrapper[2559]: I0712 17:53:36.441408 2559 plugin_watcher.go:194] "Adding socket path or updating timestamp to desired state cache" path="/var/lib/kubelet/plugins_registry/ebs.csi.aws.com-reg.sock" Jul 12 17:53:37.028902 ip-10-0-129-128 kubenswrapper[2559]: I0712 17:53:37.028869 2559 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-qtkvt" Jul 12 17:53:37.029280 ip-10-0-129-128 kubenswrapper[2559]: E0712 17:53:37.029006 2559 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-qtkvt" podUID="6f65bad8-9ec7-4a51-8557-74b3f79ef76c" Jul 12 17:53:37.029355 ip-10-0-129-128 kubenswrapper[2559]: I0712 17:53:37.029278 2559 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-h4bgw" Jul 12 17:53:37.029411 ip-10-0-129-128 kubenswrapper[2559]: E0712 17:53:37.029367 2559 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-h4bgw" podUID="e2c3a206-19c0-4d4d-bb98-a99190023ad4" Jul 12 17:53:37.038300 ip-10-0-129-128 kubenswrapper[2559]: I0712 17:53:37.038205 2559 reconciler.go:161] "OperationExecutor.RegisterPlugin started" plugin={"SocketPath":"/var/lib/kubelet/plugins_registry/ebs.csi.aws.com-reg.sock","Timestamp":"2026-07-12T17:53:36.441426341Z","UUID":"1318af9f-4911-4fad-b834-ab333b5a46e8","Handler":null,"Name":"","Endpoint":""} Jul 12 17:53:37.040192 ip-10-0-129-128 kubenswrapper[2559]: I0712 17:53:37.040172 2559 csi_plugin.go:106] kubernetes.io/csi: Trying to validate a new CSI Driver with name: ebs.csi.aws.com endpoint: /var/lib/kubelet/plugins/ebs.csi.aws.com/csi.sock versions: 1.0.0 Jul 12 17:53:37.040344 ip-10-0-129-128 kubenswrapper[2559]: I0712 17:53:37.040213 2559 csi_plugin.go:119] kubernetes.io/csi: Register new plugin with name: ebs.csi.aws.com at endpoint: /var/lib/kubelet/plugins/ebs.csi.aws.com/csi.sock Jul 12 17:53:37.219992 ip-10-0-129-128 kubenswrapper[2559]: I0712 17:53:37.219961 2559 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-m2r7q" event={"ID":"3e848887-84a9-4599-afc5-c3750c0857a0","Type":"ContainerStarted","Data":"8bf9034eee97186a7168d19f8c9b812a2e13fb2f081d87b6fe8f99b7397b9f3e"} Jul 12 17:53:37.803008 ip-10-0-129-128 kubenswrapper[2559]: I0712 17:53:37.802744 2559 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="kube-system/konnectivity-agent-g6rx5" Jul 12 17:53:37.803464 ip-10-0-129-128 kubenswrapper[2559]: I0712 17:53:37.803277 2559 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="kube-system/konnectivity-agent-g6rx5" Jul 12 17:53:38.028925 ip-10-0-129-128 kubenswrapper[2559]: I0712 17:53:38.028894 2559 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-mpqvv" Jul 12 17:53:38.029136 ip-10-0-129-128 kubenswrapper[2559]: E0712 17:53:38.028998 2559 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-mpqvv" podUID="6ab3daa1-3376-4f53-8106-d6d7417a327e" Jul 12 17:53:38.224114 ip-10-0-129-128 kubenswrapper[2559]: I0712 17:53:38.224028 2559 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-m2r7q" event={"ID":"3e848887-84a9-4599-afc5-c3750c0857a0","Type":"ContainerStarted","Data":"0f5da610a43c6a7db6bad60b908e9feae08ac02fcca4835220b8dd8925485aae"} Jul 12 17:53:38.227485 ip-10-0-129-128 kubenswrapper[2559]: I0712 17:53:38.227463 2559 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ovn-kubernetes_ovnkube-node-q5dw8_e51bad50-bc74-4e38-8307-ca257b599314/ovn-acl-logging/0.log" Jul 12 17:53:38.227888 ip-10-0-129-128 kubenswrapper[2559]: I0712 17:53:38.227852 2559 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ovn-kubernetes/ovnkube-node-q5dw8" event={"ID":"e51bad50-bc74-4e38-8307-ca257b599314","Type":"ContainerStarted","Data":"6c88a28fb5653f7b51fcecdd1baa98cffc8374034ca58a66ae63d469aa332807"} Jul 12 17:53:38.241881 ip-10-0-129-128 kubenswrapper[2559]: I0712 17:53:38.241837 2559 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-m2r7q" podStartSLOduration=4.117321838 podStartE2EDuration="25.241818514s" totalImagesPullingTime="21.124496676s" totalInitContainerRuntime="0s" isStatefulPod=true podCreationTimestamp="2026-07-12 17:53:13 +0000 UTC" imagePullSessionsCount=3 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-12 17:53:38.241394828 +0000 UTC m=+25.735462758" watchObservedRunningTime="2026-07-12 17:53:38.241818514 +0000 UTC m=+25.735886445" Jul 12 17:53:38.936455 ip-10-0-129-128 kubenswrapper[2559]: I0712 17:53:38.936420 2559 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="kube-system/konnectivity-agent-g6rx5" Jul 12 17:53:38.937198 ip-10-0-129-128 kubenswrapper[2559]: I0712 17:53:38.937179 2559 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="kube-system/konnectivity-agent-g6rx5" Jul 12 17:53:39.028308 ip-10-0-129-128 kubenswrapper[2559]: I0712 17:53:39.028275 2559 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-qtkvt" Jul 12 17:53:39.028477 ip-10-0-129-128 kubenswrapper[2559]: I0712 17:53:39.028275 2559 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-h4bgw" Jul 12 17:53:39.028477 ip-10-0-129-128 kubenswrapper[2559]: E0712 17:53:39.028423 2559 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-qtkvt" podUID="6f65bad8-9ec7-4a51-8557-74b3f79ef76c" Jul 12 17:53:39.028581 ip-10-0-129-128 kubenswrapper[2559]: E0712 17:53:39.028489 2559 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-h4bgw" podUID="e2c3a206-19c0-4d4d-bb98-a99190023ad4" Jul 12 17:53:40.028431 ip-10-0-129-128 kubenswrapper[2559]: I0712 17:53:40.028285 2559 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-mpqvv" Jul 12 17:53:40.028996 ip-10-0-129-128 kubenswrapper[2559]: E0712 17:53:40.028522 2559 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-mpqvv" podUID="6ab3daa1-3376-4f53-8106-d6d7417a327e" Jul 12 17:53:40.235434 ip-10-0-129-128 kubenswrapper[2559]: I0712 17:53:40.235348 2559 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ovn-kubernetes_ovnkube-node-q5dw8_e51bad50-bc74-4e38-8307-ca257b599314/ovn-acl-logging/0.log" Jul 12 17:53:40.235797 ip-10-0-129-128 kubenswrapper[2559]: I0712 17:53:40.235774 2559 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ovn-kubernetes/ovnkube-node-q5dw8" event={"ID":"e51bad50-bc74-4e38-8307-ca257b599314","Type":"ContainerStarted","Data":"92962c70734196c652d3395daa4070c68f4aa2668037821b0976389186b66ed3"} Jul 12 17:53:40.236002 ip-10-0-129-128 kubenswrapper[2559]: I0712 17:53:40.235984 2559 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ovn-kubernetes/ovnkube-node-q5dw8" Jul 12 17:53:40.236198 ip-10-0-129-128 kubenswrapper[2559]: I0712 17:53:40.236182 2559 scope.go:117] "RemoveContainer" containerID="748f49c843d400251e359f833789d2d460aa7729609977562eb2977a7ab7ff25" Jul 12 17:53:40.237792 ip-10-0-129-128 kubenswrapper[2559]: I0712 17:53:40.237768 2559 generic.go:356] "Generic (PLEG): container finished" podID="370d4e7b-6713-48b2-81ee-f055fe1d5490" containerID="35920f16feeed36ce2f34580d8ccfd3a5019bf3ca32a684fc316c3f366906f6f" exitCode=0 Jul 12 17:53:40.237892 ip-10-0-129-128 kubenswrapper[2559]: I0712 17:53:40.237810 2559 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-multus/multus-additional-cni-plugins-9dh8l" event={"ID":"370d4e7b-6713-48b2-81ee-f055fe1d5490","Type":"ContainerDied","Data":"35920f16feeed36ce2f34580d8ccfd3a5019bf3ca32a684fc316c3f366906f6f"} Jul 12 17:53:40.253518 ip-10-0-129-128 kubenswrapper[2559]: I0712 17:53:40.253500 2559 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ovn-kubernetes/ovnkube-node-q5dw8" Jul 12 17:53:41.029084 ip-10-0-129-128 kubenswrapper[2559]: I0712 17:53:41.029049 2559 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-qtkvt" Jul 12 17:53:41.029584 ip-10-0-129-128 kubenswrapper[2559]: E0712 17:53:41.029177 2559 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-qtkvt" podUID="6f65bad8-9ec7-4a51-8557-74b3f79ef76c" Jul 12 17:53:41.029584 ip-10-0-129-128 kubenswrapper[2559]: I0712 17:53:41.029225 2559 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-h4bgw" Jul 12 17:53:41.029584 ip-10-0-129-128 kubenswrapper[2559]: E0712 17:53:41.029337 2559 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-h4bgw" podUID="e2c3a206-19c0-4d4d-bb98-a99190023ad4" Jul 12 17:53:41.245654 ip-10-0-129-128 kubenswrapper[2559]: I0712 17:53:41.245618 2559 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ovn-kubernetes_ovnkube-node-q5dw8_e51bad50-bc74-4e38-8307-ca257b599314/ovn-acl-logging/0.log" Jul 12 17:53:41.246011 ip-10-0-129-128 kubenswrapper[2559]: I0712 17:53:41.245988 2559 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ovn-kubernetes/ovnkube-node-q5dw8" event={"ID":"e51bad50-bc74-4e38-8307-ca257b599314","Type":"ContainerStarted","Data":"158fd5c0922cc182d96e542ad472344924488a56d5727404917ea9d449384a17"} Jul 12 17:53:41.246135 ip-10-0-129-128 kubenswrapper[2559]: I0712 17:53:41.246124 2559 prober_manager.go:312] "Failed to trigger a manual run" probe="Readiness" Jul 12 17:53:41.246388 ip-10-0-129-128 kubenswrapper[2559]: I0712 17:53:41.246353 2559 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ovn-kubernetes/ovnkube-node-q5dw8" Jul 12 17:53:41.260929 ip-10-0-129-128 kubenswrapper[2559]: I0712 17:53:41.260905 2559 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ovn-kubernetes/ovnkube-node-q5dw8" Jul 12 17:53:41.273827 ip-10-0-129-128 kubenswrapper[2559]: I0712 17:53:41.273788 2559 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-ovn-kubernetes/ovnkube-node-q5dw8" podStartSLOduration=9.551209434 podStartE2EDuration="28.273774992s" totalImagesPullingTime="18.722565558s" totalInitContainerRuntime="0s" isStatefulPod=true podCreationTimestamp="2026-07-12 17:53:13 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-12 17:53:41.273483093 +0000 UTC m=+28.767551023" watchObservedRunningTime="2026-07-12 17:53:41.273774992 +0000 UTC m=+28.767842922" Jul 12 17:53:41.687358 ip-10-0-129-128 kubenswrapper[2559]: I0712 17:53:41.687327 2559 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ovn-kubernetes/ovnkube-node-q5dw8" Jul 12 17:53:41.883634 ip-10-0-129-128 kubenswrapper[2559]: I0712 17:53:41.883601 2559 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kube-system/global-pull-secret-syncer-mpqvv"] Jul 12 17:53:41.883847 ip-10-0-129-128 kubenswrapper[2559]: I0712 17:53:41.883746 2559 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-mpqvv" Jul 12 17:53:41.883890 ip-10-0-129-128 kubenswrapper[2559]: E0712 17:53:41.883840 2559 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-mpqvv" podUID="6ab3daa1-3376-4f53-8106-d6d7417a327e" Jul 12 17:53:41.886803 ip-10-0-129-128 kubenswrapper[2559]: I0712 17:53:41.886776 2559 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-network-diagnostics/network-check-target-h4bgw"] Jul 12 17:53:41.886971 ip-10-0-129-128 kubenswrapper[2559]: I0712 17:53:41.886869 2559 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-h4bgw" Jul 12 17:53:41.886971 ip-10-0-129-128 kubenswrapper[2559]: E0712 17:53:41.886951 2559 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-h4bgw" podUID="e2c3a206-19c0-4d4d-bb98-a99190023ad4" Jul 12 17:53:41.887447 ip-10-0-129-128 kubenswrapper[2559]: I0712 17:53:41.887423 2559 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-multus/network-metrics-daemon-qtkvt"] Jul 12 17:53:41.887559 ip-10-0-129-128 kubenswrapper[2559]: I0712 17:53:41.887531 2559 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-qtkvt" Jul 12 17:53:41.887664 ip-10-0-129-128 kubenswrapper[2559]: E0712 17:53:41.887630 2559 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-qtkvt" podUID="6f65bad8-9ec7-4a51-8557-74b3f79ef76c" Jul 12 17:53:42.250251 ip-10-0-129-128 kubenswrapper[2559]: I0712 17:53:42.250167 2559 generic.go:356] "Generic (PLEG): container finished" podID="370d4e7b-6713-48b2-81ee-f055fe1d5490" containerID="b891946ceb91a3aa8b69fa4732f1e41afa8efcf668e586cb5a22233d9b129f0b" exitCode=0 Jul 12 17:53:42.250728 ip-10-0-129-128 kubenswrapper[2559]: I0712 17:53:42.250242 2559 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-multus/multus-additional-cni-plugins-9dh8l" event={"ID":"370d4e7b-6713-48b2-81ee-f055fe1d5490","Type":"ContainerDied","Data":"b891946ceb91a3aa8b69fa4732f1e41afa8efcf668e586cb5a22233d9b129f0b"} Jul 12 17:53:43.028800 ip-10-0-129-128 kubenswrapper[2559]: I0712 17:53:43.028764 2559 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-mpqvv" Jul 12 17:53:43.029063 ip-10-0-129-128 kubenswrapper[2559]: E0712 17:53:43.028878 2559 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-mpqvv" podUID="6ab3daa1-3376-4f53-8106-d6d7417a327e" Jul 12 17:53:44.029092 ip-10-0-129-128 kubenswrapper[2559]: I0712 17:53:44.028929 2559 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-qtkvt" Jul 12 17:53:44.029436 ip-10-0-129-128 kubenswrapper[2559]: I0712 17:53:44.028929 2559 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-h4bgw" Jul 12 17:53:44.029436 ip-10-0-129-128 kubenswrapper[2559]: E0712 17:53:44.029179 2559 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-qtkvt" podUID="6f65bad8-9ec7-4a51-8557-74b3f79ef76c" Jul 12 17:53:44.029436 ip-10-0-129-128 kubenswrapper[2559]: E0712 17:53:44.029227 2559 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-h4bgw" podUID="e2c3a206-19c0-4d4d-bb98-a99190023ad4" Jul 12 17:53:44.258448 ip-10-0-129-128 kubenswrapper[2559]: I0712 17:53:44.258414 2559 generic.go:356] "Generic (PLEG): container finished" podID="370d4e7b-6713-48b2-81ee-f055fe1d5490" containerID="e25f0450a6b84a9ccbd3acfc64675cbf0b7bafbf9c460127ca4e139fdc684b93" exitCode=0 Jul 12 17:53:44.258624 ip-10-0-129-128 kubenswrapper[2559]: I0712 17:53:44.258458 2559 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-multus/multus-additional-cni-plugins-9dh8l" event={"ID":"370d4e7b-6713-48b2-81ee-f055fe1d5490","Type":"ContainerDied","Data":"e25f0450a6b84a9ccbd3acfc64675cbf0b7bafbf9c460127ca4e139fdc684b93"} Jul 12 17:53:45.029111 ip-10-0-129-128 kubenswrapper[2559]: I0712 17:53:45.029075 2559 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-mpqvv" Jul 12 17:53:45.029731 ip-10-0-129-128 kubenswrapper[2559]: E0712 17:53:45.029220 2559 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-mpqvv" podUID="6ab3daa1-3376-4f53-8106-d6d7417a327e" Jul 12 17:53:46.028200 ip-10-0-129-128 kubenswrapper[2559]: I0712 17:53:46.028161 2559 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-h4bgw" Jul 12 17:53:46.028397 ip-10-0-129-128 kubenswrapper[2559]: E0712 17:53:46.028298 2559 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-h4bgw" podUID="e2c3a206-19c0-4d4d-bb98-a99190023ad4" Jul 12 17:53:46.028397 ip-10-0-129-128 kubenswrapper[2559]: I0712 17:53:46.028349 2559 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-qtkvt" Jul 12 17:53:46.028512 ip-10-0-129-128 kubenswrapper[2559]: E0712 17:53:46.028464 2559 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-qtkvt" podUID="6f65bad8-9ec7-4a51-8557-74b3f79ef76c" Jul 12 17:53:46.781908 ip-10-0-129-128 kubenswrapper[2559]: I0712 17:53:46.781863 2559 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/6f65bad8-9ec7-4a51-8557-74b3f79ef76c-metrics-certs\") pod \"network-metrics-daemon-qtkvt\" (UID: \"6f65bad8-9ec7-4a51-8557-74b3f79ef76c\") " pod="openshift-multus/network-metrics-daemon-qtkvt" Jul 12 17:53:46.782353 ip-10-0-129-128 kubenswrapper[2559]: I0712 17:53:46.781925 2559 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"original-pull-secret\" (UniqueName: \"kubernetes.io/secret/6ab3daa1-3376-4f53-8106-d6d7417a327e-original-pull-secret\") pod \"global-pull-secret-syncer-mpqvv\" (UID: \"6ab3daa1-3376-4f53-8106-d6d7417a327e\") " pod="kube-system/global-pull-secret-syncer-mpqvv" Jul 12 17:53:46.782353 ip-10-0-129-128 kubenswrapper[2559]: E0712 17:53:46.781993 2559 secret.go:189] Couldn't get secret openshift-multus/metrics-daemon-secret: object "openshift-multus"/"metrics-daemon-secret" not registered Jul 12 17:53:46.782353 ip-10-0-129-128 kubenswrapper[2559]: E0712 17:53:46.782056 2559 secret.go:189] Couldn't get secret kube-system/original-pull-secret: object "kube-system"/"original-pull-secret" not registered Jul 12 17:53:46.782353 ip-10-0-129-128 kubenswrapper[2559]: E0712 17:53:46.782072 2559 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/6f65bad8-9ec7-4a51-8557-74b3f79ef76c-metrics-certs podName:6f65bad8-9ec7-4a51-8557-74b3f79ef76c nodeName:}" failed. No retries permitted until 2026-07-12 17:54:18.782049398 +0000 UTC m=+66.276117305 (durationBeforeRetry 32s). Error: MountVolume.SetUp failed for volume "metrics-certs" (UniqueName: "kubernetes.io/secret/6f65bad8-9ec7-4a51-8557-74b3f79ef76c-metrics-certs") pod "network-metrics-daemon-qtkvt" (UID: "6f65bad8-9ec7-4a51-8557-74b3f79ef76c") : object "openshift-multus"/"metrics-daemon-secret" not registered Jul 12 17:53:46.782353 ip-10-0-129-128 kubenswrapper[2559]: E0712 17:53:46.782107 2559 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/6ab3daa1-3376-4f53-8106-d6d7417a327e-original-pull-secret podName:6ab3daa1-3376-4f53-8106-d6d7417a327e nodeName:}" failed. No retries permitted until 2026-07-12 17:54:18.782092036 +0000 UTC m=+66.276159949 (durationBeforeRetry 32s). Error: MountVolume.SetUp failed for volume "original-pull-secret" (UniqueName: "kubernetes.io/secret/6ab3daa1-3376-4f53-8106-d6d7417a327e-original-pull-secret") pod "global-pull-secret-syncer-mpqvv" (UID: "6ab3daa1-3376-4f53-8106-d6d7417a327e") : object "kube-system"/"original-pull-secret" not registered Jul 12 17:53:46.882708 ip-10-0-129-128 kubenswrapper[2559]: I0712 17:53:46.882616 2559 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-bn2rx\" (UniqueName: \"kubernetes.io/projected/e2c3a206-19c0-4d4d-bb98-a99190023ad4-kube-api-access-bn2rx\") pod \"network-check-target-h4bgw\" (UID: \"e2c3a206-19c0-4d4d-bb98-a99190023ad4\") " pod="openshift-network-diagnostics/network-check-target-h4bgw" Jul 12 17:53:46.882830 ip-10-0-129-128 kubenswrapper[2559]: E0712 17:53:46.882765 2559 projected.go:291] Couldn't get configMap openshift-network-diagnostics/kube-root-ca.crt: object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered Jul 12 17:53:46.882830 ip-10-0-129-128 kubenswrapper[2559]: E0712 17:53:46.882789 2559 projected.go:291] Couldn't get configMap openshift-network-diagnostics/openshift-service-ca.crt: object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered Jul 12 17:53:46.882830 ip-10-0-129-128 kubenswrapper[2559]: E0712 17:53:46.882802 2559 projected.go:196] Error preparing data for projected volume kube-api-access-bn2rx for pod openshift-network-diagnostics/network-check-target-h4bgw: [object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered, object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered] Jul 12 17:53:46.882928 ip-10-0-129-128 kubenswrapper[2559]: E0712 17:53:46.882860 2559 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/projected/e2c3a206-19c0-4d4d-bb98-a99190023ad4-kube-api-access-bn2rx podName:e2c3a206-19c0-4d4d-bb98-a99190023ad4 nodeName:}" failed. No retries permitted until 2026-07-12 17:54:18.882847921 +0000 UTC m=+66.376915829 (durationBeforeRetry 32s). Error: MountVolume.SetUp failed for volume "kube-api-access-bn2rx" (UniqueName: "kubernetes.io/projected/e2c3a206-19c0-4d4d-bb98-a99190023ad4-kube-api-access-bn2rx") pod "network-check-target-h4bgw" (UID: "e2c3a206-19c0-4d4d-bb98-a99190023ad4") : [object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered, object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered] Jul 12 17:53:47.031792 ip-10-0-129-128 kubenswrapper[2559]: I0712 17:53:47.031761 2559 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-mpqvv" Jul 12 17:53:47.031965 ip-10-0-129-128 kubenswrapper[2559]: E0712 17:53:47.031884 2559 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-mpqvv" podUID="6ab3daa1-3376-4f53-8106-d6d7417a327e" Jul 12 17:53:48.028767 ip-10-0-129-128 kubenswrapper[2559]: I0712 17:53:48.028721 2559 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-h4bgw" Jul 12 17:53:48.029406 ip-10-0-129-128 kubenswrapper[2559]: I0712 17:53:48.028736 2559 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-qtkvt" Jul 12 17:53:48.029406 ip-10-0-129-128 kubenswrapper[2559]: E0712 17:53:48.028856 2559 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-h4bgw" podUID="e2c3a206-19c0-4d4d-bb98-a99190023ad4" Jul 12 17:53:48.029406 ip-10-0-129-128 kubenswrapper[2559]: E0712 17:53:48.028912 2559 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-qtkvt" podUID="6f65bad8-9ec7-4a51-8557-74b3f79ef76c" Jul 12 17:53:48.826130 ip-10-0-129-128 kubenswrapper[2559]: I0712 17:53:48.826106 2559 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-129-128.ec2.internal" event="NodeReady" Jul 12 17:53:48.826256 ip-10-0-129-128 kubenswrapper[2559]: I0712 17:53:48.826244 2559 kubelet_node_status.go:489] "Fast updating node status as it just became ready" Jul 12 17:53:48.858400 ip-10-0-129-128 kubenswrapper[2559]: I0712 17:53:48.858364 2559 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-cluster-samples-operator/cluster-samples-operator-65db858894-996qw"] Jul 12 17:53:48.860060 ip-10-0-129-128 kubenswrapper[2559]: I0712 17:53:48.860045 2559 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-cluster-samples-operator/cluster-samples-operator-65db858894-996qw" Jul 12 17:53:48.861976 ip-10-0-129-128 kubenswrapper[2559]: I0712 17:53:48.861946 2559 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-cluster-storage-operator/volume-data-source-validator-78495597f6-dnhgq"] Jul 12 17:53:48.863230 ip-10-0-129-128 kubenswrapper[2559]: I0712 17:53:48.863202 2559 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-cluster-samples-operator\"/\"openshift-service-ca.crt\"" Jul 12 17:53:48.863452 ip-10-0-129-128 kubenswrapper[2559]: I0712 17:53:48.863353 2559 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-cluster-samples-operator\"/\"samples-operator-tls\"" Jul 12 17:53:48.863452 ip-10-0-129-128 kubenswrapper[2559]: I0712 17:53:48.863364 2559 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-cluster-samples-operator\"/\"cluster-samples-operator-dockercfg-vwpq5\"" Jul 12 17:53:48.863697 ip-10-0-129-128 kubenswrapper[2559]: I0712 17:53:48.863473 2559 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-cluster-samples-operator\"/\"kube-root-ca.crt\"" Jul 12 17:53:48.864132 ip-10-0-129-128 kubenswrapper[2559]: I0712 17:53:48.864112 2559 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-console-operator/console-operator-7866497494-wlk78"] Jul 12 17:53:48.864260 ip-10-0-129-128 kubenswrapper[2559]: I0712 17:53:48.864237 2559 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-cluster-storage-operator/volume-data-source-validator-78495597f6-dnhgq" Jul 12 17:53:48.865578 ip-10-0-129-128 kubenswrapper[2559]: I0712 17:53:48.865552 2559 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-service-ca-operator/service-ca-operator-7695fc6c8c-cnb6s"] Jul 12 17:53:48.865714 ip-10-0-129-128 kubenswrapper[2559]: I0712 17:53:48.865700 2559 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-console-operator/console-operator-7866497494-wlk78" Jul 12 17:53:48.866962 ip-10-0-129-128 kubenswrapper[2559]: I0712 17:53:48.866940 2559 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-cluster-storage-operator\"/\"openshift-service-ca.crt\"" Jul 12 17:53:48.867421 ip-10-0-129-128 kubenswrapper[2559]: I0712 17:53:48.867404 2559 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-cluster-storage-operator\"/\"kube-root-ca.crt\"" Jul 12 17:53:48.869937 ip-10-0-129-128 kubenswrapper[2559]: I0712 17:53:48.867516 2559 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-cluster-storage-operator\"/\"volume-data-source-validator-dockercfg-k7v2f\"" Jul 12 17:53:48.869937 ip-10-0-129-128 kubenswrapper[2559]: I0712 17:53:48.867420 2559 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-image-registry/image-registry-85b88cb476-kzl6q"] Jul 12 17:53:48.869937 ip-10-0-129-128 kubenswrapper[2559]: I0712 17:53:48.867498 2559 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-service-ca-operator/service-ca-operator-7695fc6c8c-cnb6s" Jul 12 17:53:48.869937 ip-10-0-129-128 kubenswrapper[2559]: I0712 17:53:48.869209 2559 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-console-operator\"/\"console-operator-dockercfg-bp5gf\"" Jul 12 17:53:48.869937 ip-10-0-129-128 kubenswrapper[2559]: I0712 17:53:48.869214 2559 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-console-operator\"/\"kube-root-ca.crt\"" Jul 12 17:53:48.870377 ip-10-0-129-128 kubenswrapper[2559]: I0712 17:53:48.870047 2559 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-console-operator\"/\"console-operator-config\"" Jul 12 17:53:48.870377 ip-10-0-129-128 kubenswrapper[2559]: I0712 17:53:48.870056 2559 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-console-operator\"/\"serving-cert\"" Jul 12 17:53:48.871726 ip-10-0-129-128 kubenswrapper[2559]: I0712 17:53:48.870996 2559 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-console-operator\"/\"openshift-service-ca.crt\"" Jul 12 17:53:48.872578 ip-10-0-129-128 kubenswrapper[2559]: I0712 17:53:48.872162 2559 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-service-ca-operator\"/\"service-ca-operator-dockercfg-qhwl6\"" Jul 12 17:53:48.872705 ip-10-0-129-128 kubenswrapper[2559]: I0712 17:53:48.872687 2559 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-kube-storage-version-migrator-operator/kube-storage-version-migrator-operator-799cb7f958-rfppz"] Jul 12 17:53:48.872869 ip-10-0-129-128 kubenswrapper[2559]: I0712 17:53:48.872821 2559 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-image-registry/image-registry-85b88cb476-kzl6q" Jul 12 17:53:48.873561 ip-10-0-129-128 kubenswrapper[2559]: I0712 17:53:48.873541 2559 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-service-ca-operator\"/\"openshift-service-ca.crt\"" Jul 12 17:53:48.874848 ip-10-0-129-128 kubenswrapper[2559]: I0712 17:53:48.874349 2559 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-service-ca-operator\"/\"kube-root-ca.crt\"" Jul 12 17:53:48.874848 ip-10-0-129-128 kubenswrapper[2559]: I0712 17:53:48.874553 2559 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-service-ca-operator\"/\"serving-cert\"" Jul 12 17:53:48.874848 ip-10-0-129-128 kubenswrapper[2559]: I0712 17:53:48.874720 2559 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-service-ca-operator\"/\"service-ca-operator-config\"" Jul 12 17:53:48.875625 ip-10-0-129-128 kubenswrapper[2559]: I0712 17:53:48.875459 2559 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-cluster-samples-operator/cluster-samples-operator-65db858894-996qw"] Jul 12 17:53:48.875625 ip-10-0-129-128 kubenswrapper[2559]: I0712 17:53:48.875596 2559 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-kube-storage-version-migrator-operator/kube-storage-version-migrator-operator-799cb7f958-rfppz" Jul 12 17:53:48.876845 ip-10-0-129-128 kubenswrapper[2559]: I0712 17:53:48.876233 2559 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-image-registry\"/\"image-registry-tls\"" Jul 12 17:53:48.876845 ip-10-0-129-128 kubenswrapper[2559]: I0712 17:53:48.876320 2559 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-image-registry\"/\"registry-dockercfg-qczrr\"" Jul 12 17:53:48.876845 ip-10-0-129-128 kubenswrapper[2559]: I0712 17:53:48.876466 2559 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-image-registry\"/\"installation-pull-secrets\"" Jul 12 17:53:48.876845 ip-10-0-129-128 kubenswrapper[2559]: I0712 17:53:48.876770 2559 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-image-registry\"/\"image-registry-private-configuration\"" Jul 12 17:53:48.877109 ip-10-0-129-128 kubenswrapper[2559]: I0712 17:53:48.877079 2559 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-console-operator\"/\"trusted-ca\"" Jul 12 17:53:48.878234 ip-10-0-129-128 kubenswrapper[2559]: I0712 17:53:48.878214 2559 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-kube-storage-version-migrator-operator\"/\"openshift-service-ca.crt\"" Jul 12 17:53:48.878681 ip-10-0-129-128 kubenswrapper[2559]: I0712 17:53:48.878462 2559 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-kube-storage-version-migrator-operator\"/\"kube-storage-version-migrator-operator-dockercfg-jvkp8\"" Jul 12 17:53:48.882709 ip-10-0-129-128 kubenswrapper[2559]: I0712 17:53:48.881548 2559 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-network-diagnostics/network-check-source-869dcfcdcd-whn86"] Jul 12 17:53:48.885070 ip-10-0-129-128 kubenswrapper[2559]: I0712 17:53:48.884554 2559 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["open-cluster-management-agent-addon/cluster-proxy-proxy-agent-7fdc8d7ffb-7hrbq"] Jul 12 17:53:48.885070 ip-10-0-129-128 kubenswrapper[2559]: I0712 17:53:48.885038 2559 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-source-869dcfcdcd-whn86" Jul 12 17:53:48.886501 ip-10-0-129-128 kubenswrapper[2559]: I0712 17:53:48.886340 2559 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-ingress-canary/ingress-canary-fdmcz"] Jul 12 17:53:48.887093 ip-10-0-129-128 kubenswrapper[2559]: I0712 17:53:48.886771 2559 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-kube-storage-version-migrator-operator\"/\"kube-root-ca.crt\"" Jul 12 17:53:48.887497 ip-10-0-129-128 kubenswrapper[2559]: I0712 17:53:48.887460 2559 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-kube-storage-version-migrator-operator\"/\"config\"" Jul 12 17:53:48.888252 ip-10-0-129-128 kubenswrapper[2559]: I0712 17:53:48.887765 2559 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-kube-storage-version-migrator-operator\"/\"serving-cert\"" Jul 12 17:53:48.888366 ip-10-0-129-128 kubenswrapper[2559]: I0712 17:53:48.888323 2559 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-service-ca-operator/service-ca-operator-7695fc6c8c-cnb6s"] Jul 12 17:53:48.888366 ip-10-0-129-128 kubenswrapper[2559]: I0712 17:53:48.888355 2559 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["open-cluster-management-agent-addon/klusterlet-addon-workmgr-74d7bc54f4-k7lv7"] Jul 12 17:53:48.888366 ip-10-0-129-128 kubenswrapper[2559]: I0712 17:53:48.888356 2559 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-image-registry\"/\"trusted-ca\"" Jul 12 17:53:48.888783 ip-10-0-129-128 kubenswrapper[2559]: I0712 17:53:48.888576 2559 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="open-cluster-management-agent-addon/cluster-proxy-proxy-agent-7fdc8d7ffb-7hrbq" Jul 12 17:53:48.889164 ip-10-0-129-128 kubenswrapper[2559]: I0712 17:53:48.889143 2559 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress-canary/ingress-canary-fdmcz" Jul 12 17:53:48.890351 ip-10-0-129-128 kubenswrapper[2559]: I0712 17:53:48.890130 2559 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-network-diagnostics\"/\"openshift-service-ca.crt\"" Jul 12 17:53:48.894270 ip-10-0-129-128 kubenswrapper[2559]: I0712 17:53:48.891176 2559 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-network-diagnostics\"/\"kube-root-ca.crt\"" Jul 12 17:53:48.894270 ip-10-0-129-128 kubenswrapper[2559]: I0712 17:53:48.891254 2559 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-console-operator/console-operator-7866497494-wlk78"] Jul 12 17:53:48.894270 ip-10-0-129-128 kubenswrapper[2559]: I0712 17:53:48.891285 2559 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-cluster-storage-operator/volume-data-source-validator-78495597f6-dnhgq"] Jul 12 17:53:48.894270 ip-10-0-129-128 kubenswrapper[2559]: I0712 17:53:48.891306 2559 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["open-cluster-management-agent-addon/managed-serviceaccount-addon-agent-7c88559b8b-qk5pk"] Jul 12 17:53:48.894270 ip-10-0-129-128 kubenswrapper[2559]: I0712 17:53:48.891411 2559 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-network-diagnostics\"/\"network-diagnostics-dockercfg-gzrzk\"" Jul 12 17:53:48.894270 ip-10-0-129-128 kubenswrapper[2559]: I0712 17:53:48.891981 2559 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"open-cluster-management-agent-addon\"/\"kube-root-ca.crt\"" Jul 12 17:53:48.894270 ip-10-0-129-128 kubenswrapper[2559]: I0712 17:53:48.892177 2559 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"open-cluster-management-agent-addon\"/\"cluster-proxy-service-proxy-server-certificates\"" Jul 12 17:53:48.894270 ip-10-0-129-128 kubenswrapper[2559]: I0712 17:53:48.892453 2559 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"open-cluster-management-agent-addon\"/\"cluster-proxy-ca\"" Jul 12 17:53:48.894270 ip-10-0-129-128 kubenswrapper[2559]: I0712 17:53:48.892458 2559 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="open-cluster-management-agent-addon/klusterlet-addon-workmgr-74d7bc54f4-k7lv7" Jul 12 17:53:48.894270 ip-10-0-129-128 kubenswrapper[2559]: I0712 17:53:48.892679 2559 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"open-cluster-management-agent-addon\"/\"cluster-proxy-open-cluster-management.io-proxy-agent-signer-client-cert\"" Jul 12 17:53:48.894270 ip-10-0-129-128 kubenswrapper[2559]: I0712 17:53:48.893071 2559 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ingress-canary\"/\"openshift-service-ca.crt\"" Jul 12 17:53:48.894270 ip-10-0-129-128 kubenswrapper[2559]: I0712 17:53:48.893259 2559 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress-canary\"/\"default-dockercfg-59fr9\"" Jul 12 17:53:48.894270 ip-10-0-129-128 kubenswrapper[2559]: I0712 17:53:48.893324 2559 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress-canary\"/\"canary-serving-cert\"" Jul 12 17:53:48.894270 ip-10-0-129-128 kubenswrapper[2559]: I0712 17:53:48.893496 2559 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-dns/dns-default-wvr9g"] Jul 12 17:53:48.894270 ip-10-0-129-128 kubenswrapper[2559]: I0712 17:53:48.893740 2559 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="open-cluster-management-agent-addon/managed-serviceaccount-addon-agent-7c88559b8b-qk5pk" Jul 12 17:53:48.895289 ip-10-0-129-128 kubenswrapper[2559]: I0712 17:53:48.894862 2559 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"open-cluster-management-agent-addon\"/\"cluster-proxy-hub-kubeconfig\"" Jul 12 17:53:48.895896 ip-10-0-129-128 kubenswrapper[2559]: I0712 17:53:48.895816 2559 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-kube-storage-version-migrator-operator/kube-storage-version-migrator-operator-799cb7f958-rfppz"] Jul 12 17:53:48.895896 ip-10-0-129-128 kubenswrapper[2559]: I0712 17:53:48.895862 2559 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-image-registry/image-registry-85b88cb476-kzl6q"] Jul 12 17:53:48.897584 ip-10-0-129-128 kubenswrapper[2559]: I0712 17:53:48.896469 2559 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"open-cluster-management-agent-addon\"/\"openshift-service-ca.crt\"" Jul 12 17:53:48.897584 ip-10-0-129-128 kubenswrapper[2559]: I0712 17:53:48.896742 2559 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"open-cluster-management-agent-addon\"/\"work-manager-hub-kubeconfig\"" Jul 12 17:53:48.898667 ip-10-0-129-128 kubenswrapper[2559]: I0712 17:53:48.897858 2559 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"open-cluster-management-agent-addon\"/\"open-cluster-management-image-pull-credentials\"" Jul 12 17:53:48.898667 ip-10-0-129-128 kubenswrapper[2559]: I0712 17:53:48.897984 2559 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"open-cluster-management-agent-addon\"/\"managed-serviceaccount-dockercfg-gmdvn\"" Jul 12 17:53:48.899670 ip-10-0-129-128 kubenswrapper[2559]: I0712 17:53:48.898856 2559 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"open-cluster-management-agent-addon\"/\"managed-serviceaccount-hub-kubeconfig\"" Jul 12 17:53:48.899670 ip-10-0-129-128 kubenswrapper[2559]: I0712 17:53:48.899373 2559 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ingress-canary\"/\"kube-root-ca.crt\"" Jul 12 17:53:48.902664 ip-10-0-129-128 kubenswrapper[2559]: I0712 17:53:48.902384 2559 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-network-diagnostics/network-check-source-869dcfcdcd-whn86"] Jul 12 17:53:48.902664 ip-10-0-129-128 kubenswrapper[2559]: I0712 17:53:48.902420 2559 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress-canary/ingress-canary-fdmcz"] Jul 12 17:53:48.902664 ip-10-0-129-128 kubenswrapper[2559]: I0712 17:53:48.902447 2559 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["open-cluster-management-agent-addon/managed-serviceaccount-addon-agent-7c88559b8b-qk5pk"] Jul 12 17:53:48.902664 ip-10-0-129-128 kubenswrapper[2559]: I0712 17:53:48.902529 2559 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-dns/dns-default-wvr9g" Jul 12 17:53:48.903410 ip-10-0-129-128 kubenswrapper[2559]: I0712 17:53:48.902933 2559 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["open-cluster-management-agent-addon/cluster-proxy-proxy-agent-7fdc8d7ffb-7hrbq"] Jul 12 17:53:48.905568 ip-10-0-129-128 kubenswrapper[2559]: I0712 17:53:48.905209 2559 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-dns\"/\"dns-default-metrics-tls\"" Jul 12 17:53:48.905568 ip-10-0-129-128 kubenswrapper[2559]: I0712 17:53:48.905261 2559 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-dns/dns-default-wvr9g"] Jul 12 17:53:48.905568 ip-10-0-129-128 kubenswrapper[2559]: I0712 17:53:48.905387 2559 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-dns\"/\"dns-dockercfg-xtknx\"" Jul 12 17:53:48.905568 ip-10-0-129-128 kubenswrapper[2559]: I0712 17:53:48.905461 2559 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-dns\"/\"dns-default\"" Jul 12 17:53:48.911363 ip-10-0-129-128 kubenswrapper[2559]: I0712 17:53:48.911343 2559 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["open-cluster-management-agent-addon/klusterlet-addon-workmgr-74d7bc54f4-k7lv7"] Jul 12 17:53:49.001567 ip-10-0-129-128 kubenswrapper[2559]: I0712 17:53:49.001543 2559 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"config\" (UniqueName: \"kubernetes.io/configmap/e73254f8-d204-4dd4-a86f-512fc5d17ebe-config\") pod \"kube-storage-version-migrator-operator-799cb7f958-rfppz\" (UID: \"e73254f8-d204-4dd4-a86f-512fc5d17ebe\") " pod="openshift-kube-storage-version-migrator-operator/kube-storage-version-migrator-operator-799cb7f958-rfppz" Jul 12 17:53:49.001733 ip-10-0-129-128 kubenswrapper[2559]: I0712 17:53:49.001579 2559 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"cert\" (UniqueName: \"kubernetes.io/secret/98f3df53-0008-483f-bb5f-0a5718df3920-cert\") pod \"ingress-canary-fdmcz\" (UID: \"98f3df53-0008-483f-bb5f-0a5718df3920\") " pod="openshift-ingress-canary/ingress-canary-fdmcz" Jul 12 17:53:49.001733 ip-10-0-129-128 kubenswrapper[2559]: I0712 17:53:49.001610 2559 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"registry-tls\" (UniqueName: \"kubernetes.io/projected/6c8669ba-68cf-4ec6-81d4-1c5842a3d34a-registry-tls\") pod \"image-registry-85b88cb476-kzl6q\" (UID: \"6c8669ba-68cf-4ec6-81d4-1c5842a3d34a\") " pod="openshift-image-registry/image-registry-85b88cb476-kzl6q" Jul 12 17:53:49.001733 ip-10-0-129-128 kubenswrapper[2559]: I0712 17:53:49.001687 2559 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"config\" (UniqueName: \"kubernetes.io/configmap/f1d46430-bb60-4470-b8d8-eacd44f4f6b1-config\") pod \"console-operator-7866497494-wlk78\" (UID: \"f1d46430-bb60-4470-b8d8-eacd44f4f6b1\") " pod="openshift-console-operator/console-operator-7866497494-wlk78" Jul 12 17:53:49.001854 ip-10-0-129-128 kubenswrapper[2559]: I0712 17:53:49.001740 2559 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"hub-kubeconfig\" (UniqueName: \"kubernetes.io/secret/4727caec-6a00-4f71-836f-93b328b67d8e-hub-kubeconfig\") pod \"managed-serviceaccount-addon-agent-7c88559b8b-qk5pk\" (UID: \"4727caec-6a00-4f71-836f-93b328b67d8e\") " pod="open-cluster-management-agent-addon/managed-serviceaccount-addon-agent-7c88559b8b-qk5pk" Jul 12 17:53:49.001854 ip-10-0-129-128 kubenswrapper[2559]: I0712 17:53:49.001794 2559 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tmp\" (UniqueName: \"kubernetes.io/empty-dir/27899582-4f2a-4b54-a4f5-0f6167d16c1f-tmp\") pod \"klusterlet-addon-workmgr-74d7bc54f4-k7lv7\" (UID: \"27899582-4f2a-4b54-a4f5-0f6167d16c1f\") " pod="open-cluster-management-agent-addon/klusterlet-addon-workmgr-74d7bc54f4-k7lv7" Jul 12 17:53:49.001854 ip-10-0-129-128 kubenswrapper[2559]: I0712 17:53:49.001828 2559 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"klusterlet-config\" (UniqueName: \"kubernetes.io/secret/27899582-4f2a-4b54-a4f5-0f6167d16c1f-klusterlet-config\") pod \"klusterlet-addon-workmgr-74d7bc54f4-k7lv7\" (UID: \"27899582-4f2a-4b54-a4f5-0f6167d16c1f\") " pod="open-cluster-management-agent-addon/klusterlet-addon-workmgr-74d7bc54f4-k7lv7" Jul 12 17:53:49.001854 ip-10-0-129-128 kubenswrapper[2559]: I0712 17:53:49.001847 2559 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"installation-pull-secrets\" (UniqueName: \"kubernetes.io/secret/6c8669ba-68cf-4ec6-81d4-1c5842a3d34a-installation-pull-secrets\") pod \"image-registry-85b88cb476-kzl6q\" (UID: \"6c8669ba-68cf-4ec6-81d4-1c5842a3d34a\") " pod="openshift-image-registry/image-registry-85b88cb476-kzl6q" Jul 12 17:53:49.002020 ip-10-0-129-128 kubenswrapper[2559]: I0712 17:53:49.001862 2559 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-lkq5d\" (UniqueName: \"kubernetes.io/projected/7a1ef2ea-470b-4dfa-b70c-30bec8f2d908-kube-api-access-lkq5d\") pod \"cluster-proxy-proxy-agent-7fdc8d7ffb-7hrbq\" (UID: \"7a1ef2ea-470b-4dfa-b70c-30bec8f2d908\") " pod="open-cluster-management-agent-addon/cluster-proxy-proxy-agent-7fdc8d7ffb-7hrbq" Jul 12 17:53:49.002020 ip-10-0-129-128 kubenswrapper[2559]: I0712 17:53:49.001878 2559 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"metrics-tls\" (UniqueName: \"kubernetes.io/secret/15d24b76-d54a-409b-91e4-6b9e4f9ac7b2-metrics-tls\") pod \"dns-default-wvr9g\" (UID: \"15d24b76-d54a-409b-91e4-6b9e4f9ac7b2\") " pod="openshift-dns/dns-default-wvr9g" Jul 12 17:53:49.002020 ip-10-0-129-128 kubenswrapper[2559]: I0712 17:53:49.001896 2559 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"hub-kubeconfig\" (UniqueName: \"kubernetes.io/secret/7a1ef2ea-470b-4dfa-b70c-30bec8f2d908-hub-kubeconfig\") pod \"cluster-proxy-proxy-agent-7fdc8d7ffb-7hrbq\" (UID: \"7a1ef2ea-470b-4dfa-b70c-30bec8f2d908\") " pod="open-cluster-management-agent-addon/cluster-proxy-proxy-agent-7fdc8d7ffb-7hrbq" Jul 12 17:53:49.002020 ip-10-0-129-128 kubenswrapper[2559]: I0712 17:53:49.001988 2559 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"trusted-ca\" (UniqueName: \"kubernetes.io/configmap/f1d46430-bb60-4470-b8d8-eacd44f4f6b1-trusted-ca\") pod \"console-operator-7866497494-wlk78\" (UID: \"f1d46430-bb60-4470-b8d8-eacd44f4f6b1\") " pod="openshift-console-operator/console-operator-7866497494-wlk78" Jul 12 17:53:49.002169 ip-10-0-129-128 kubenswrapper[2559]: I0712 17:53:49.002028 2559 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"config-volume\" (UniqueName: \"kubernetes.io/configmap/15d24b76-d54a-409b-91e4-6b9e4f9ac7b2-config-volume\") pod \"dns-default-wvr9g\" (UID: \"15d24b76-d54a-409b-91e4-6b9e4f9ac7b2\") " pod="openshift-dns/dns-default-wvr9g" Jul 12 17:53:49.002169 ip-10-0-129-128 kubenswrapper[2559]: I0712 17:53:49.002055 2559 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"bound-sa-token\" (UniqueName: \"kubernetes.io/projected/6c8669ba-68cf-4ec6-81d4-1c5842a3d34a-bound-sa-token\") pod \"image-registry-85b88cb476-kzl6q\" (UID: \"6c8669ba-68cf-4ec6-81d4-1c5842a3d34a\") " pod="openshift-image-registry/image-registry-85b88cb476-kzl6q" Jul 12 17:53:49.002169 ip-10-0-129-128 kubenswrapper[2559]: I0712 17:53:49.002114 2559 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-rvs58\" (UniqueName: \"kubernetes.io/projected/356ce79c-8163-4cdc-b93e-fb248a89d42c-kube-api-access-rvs58\") pod \"service-ca-operator-7695fc6c8c-cnb6s\" (UID: \"356ce79c-8163-4cdc-b93e-fb248a89d42c\") " pod="openshift-service-ca-operator/service-ca-operator-7695fc6c8c-cnb6s" Jul 12 17:53:49.002169 ip-10-0-129-128 kubenswrapper[2559]: I0712 17:53:49.002144 2559 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"registry-certificates\" (UniqueName: \"kubernetes.io/configmap/6c8669ba-68cf-4ec6-81d4-1c5842a3d34a-registry-certificates\") pod \"image-registry-85b88cb476-kzl6q\" (UID: \"6c8669ba-68cf-4ec6-81d4-1c5842a3d34a\") " pod="openshift-image-registry/image-registry-85b88cb476-kzl6q" Jul 12 17:53:49.002295 ip-10-0-129-128 kubenswrapper[2559]: I0712 17:53:49.002172 2559 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"trusted-ca\" (UniqueName: \"kubernetes.io/configmap/6c8669ba-68cf-4ec6-81d4-1c5842a3d34a-trusted-ca\") pod \"image-registry-85b88cb476-kzl6q\" (UID: \"6c8669ba-68cf-4ec6-81d4-1c5842a3d34a\") " pod="openshift-image-registry/image-registry-85b88cb476-kzl6q" Jul 12 17:53:49.002295 ip-10-0-129-128 kubenswrapper[2559]: I0712 17:53:49.002191 2559 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-4xhmm\" (UniqueName: \"kubernetes.io/projected/e73254f8-d204-4dd4-a86f-512fc5d17ebe-kube-api-access-4xhmm\") pod \"kube-storage-version-migrator-operator-799cb7f958-rfppz\" (UID: \"e73254f8-d204-4dd4-a86f-512fc5d17ebe\") " pod="openshift-kube-storage-version-migrator-operator/kube-storage-version-migrator-operator-799cb7f958-rfppz" Jul 12 17:53:49.002295 ip-10-0-129-128 kubenswrapper[2559]: I0712 17:53:49.002206 2559 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-9mzs5\" (UniqueName: \"kubernetes.io/projected/f1d46430-bb60-4470-b8d8-eacd44f4f6b1-kube-api-access-9mzs5\") pod \"console-operator-7866497494-wlk78\" (UID: \"f1d46430-bb60-4470-b8d8-eacd44f4f6b1\") " pod="openshift-console-operator/console-operator-7866497494-wlk78" Jul 12 17:53:49.002295 ip-10-0-129-128 kubenswrapper[2559]: I0712 17:53:49.002225 2559 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"serving-cert\" (UniqueName: \"kubernetes.io/secret/e73254f8-d204-4dd4-a86f-512fc5d17ebe-serving-cert\") pod \"kube-storage-version-migrator-operator-799cb7f958-rfppz\" (UID: \"e73254f8-d204-4dd4-a86f-512fc5d17ebe\") " pod="openshift-kube-storage-version-migrator-operator/kube-storage-version-migrator-operator-799cb7f958-rfppz" Jul 12 17:53:49.002295 ip-10-0-129-128 kubenswrapper[2559]: I0712 17:53:49.002246 2559 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"ca-trust-extracted\" (UniqueName: \"kubernetes.io/empty-dir/6c8669ba-68cf-4ec6-81d4-1c5842a3d34a-ca-trust-extracted\") pod \"image-registry-85b88cb476-kzl6q\" (UID: \"6c8669ba-68cf-4ec6-81d4-1c5842a3d34a\") " pod="openshift-image-registry/image-registry-85b88cb476-kzl6q" Jul 12 17:53:49.002295 ip-10-0-129-128 kubenswrapper[2559]: I0712 17:53:49.002261 2559 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"serving-cert\" (UniqueName: \"kubernetes.io/secret/f1d46430-bb60-4470-b8d8-eacd44f4f6b1-serving-cert\") pod \"console-operator-7866497494-wlk78\" (UID: \"f1d46430-bb60-4470-b8d8-eacd44f4f6b1\") " pod="openshift-console-operator/console-operator-7866497494-wlk78" Jul 12 17:53:49.002295 ip-10-0-129-128 kubenswrapper[2559]: I0712 17:53:49.002279 2559 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"hub\" (UniqueName: \"kubernetes.io/secret/7a1ef2ea-470b-4dfa-b70c-30bec8f2d908-hub\") pod \"cluster-proxy-proxy-agent-7fdc8d7ffb-7hrbq\" (UID: \"7a1ef2ea-470b-4dfa-b70c-30bec8f2d908\") " pod="open-cluster-management-agent-addon/cluster-proxy-proxy-agent-7fdc8d7ffb-7hrbq" Jul 12 17:53:49.002578 ip-10-0-129-128 kubenswrapper[2559]: I0712 17:53:49.002314 2559 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"config\" (UniqueName: \"kubernetes.io/configmap/356ce79c-8163-4cdc-b93e-fb248a89d42c-config\") pod \"service-ca-operator-7695fc6c8c-cnb6s\" (UID: \"356ce79c-8163-4cdc-b93e-fb248a89d42c\") " pod="openshift-service-ca-operator/service-ca-operator-7695fc6c8c-cnb6s" Jul 12 17:53:49.002578 ip-10-0-129-128 kubenswrapper[2559]: I0712 17:53:49.002343 2559 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"additional-service-ca\" (UniqueName: \"kubernetes.io/configmap/7a1ef2ea-470b-4dfa-b70c-30bec8f2d908-additional-service-ca\") pod \"cluster-proxy-proxy-agent-7fdc8d7ffb-7hrbq\" (UID: \"7a1ef2ea-470b-4dfa-b70c-30bec8f2d908\") " pod="open-cluster-management-agent-addon/cluster-proxy-proxy-agent-7fdc8d7ffb-7hrbq" Jul 12 17:53:49.002578 ip-10-0-129-128 kubenswrapper[2559]: I0712 17:53:49.002417 2559 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-qcdjb\" (UniqueName: \"kubernetes.io/projected/27899582-4f2a-4b54-a4f5-0f6167d16c1f-kube-api-access-qcdjb\") pod \"klusterlet-addon-workmgr-74d7bc54f4-k7lv7\" (UID: \"27899582-4f2a-4b54-a4f5-0f6167d16c1f\") " pod="open-cluster-management-agent-addon/klusterlet-addon-workmgr-74d7bc54f4-k7lv7" Jul 12 17:53:49.002578 ip-10-0-129-128 kubenswrapper[2559]: I0712 17:53:49.002448 2559 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"samples-operator-tls\" (UniqueName: \"kubernetes.io/secret/a91aea93-bc91-4394-b212-c7699e8593e2-samples-operator-tls\") pod \"cluster-samples-operator-65db858894-996qw\" (UID: \"a91aea93-bc91-4394-b212-c7699e8593e2\") " pod="openshift-cluster-samples-operator/cluster-samples-operator-65db858894-996qw" Jul 12 17:53:49.002578 ip-10-0-129-128 kubenswrapper[2559]: I0712 17:53:49.002515 2559 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/15d24b76-d54a-409b-91e4-6b9e4f9ac7b2-tmp-dir\") pod \"dns-default-wvr9g\" (UID: \"15d24b76-d54a-409b-91e4-6b9e4f9ac7b2\") " pod="openshift-dns/dns-default-wvr9g" Jul 12 17:53:49.002794 ip-10-0-129-128 kubenswrapper[2559]: I0712 17:53:49.002584 2559 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-m54kw\" (UniqueName: \"kubernetes.io/projected/358e2b76-e0cb-45cb-b0ac-9360910b4e05-kube-api-access-m54kw\") pod \"network-check-source-869dcfcdcd-whn86\" (UID: \"358e2b76-e0cb-45cb-b0ac-9360910b4e05\") " pod="openshift-network-diagnostics/network-check-source-869dcfcdcd-whn86" Jul 12 17:53:49.002794 ip-10-0-129-128 kubenswrapper[2559]: I0712 17:53:49.002622 2559 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-q4mg9\" (UniqueName: \"kubernetes.io/projected/a91aea93-bc91-4394-b212-c7699e8593e2-kube-api-access-q4mg9\") pod \"cluster-samples-operator-65db858894-996qw\" (UID: \"a91aea93-bc91-4394-b212-c7699e8593e2\") " pod="openshift-cluster-samples-operator/cluster-samples-operator-65db858894-996qw" Jul 12 17:53:49.002794 ip-10-0-129-128 kubenswrapper[2559]: I0712 17:53:49.002671 2559 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"image-registry-private-configuration\" (UniqueName: \"kubernetes.io/secret/6c8669ba-68cf-4ec6-81d4-1c5842a3d34a-image-registry-private-configuration\") pod \"image-registry-85b88cb476-kzl6q\" (UID: \"6c8669ba-68cf-4ec6-81d4-1c5842a3d34a\") " pod="openshift-image-registry/image-registry-85b88cb476-kzl6q" Jul 12 17:53:49.002794 ip-10-0-129-128 kubenswrapper[2559]: I0712 17:53:49.002698 2559 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"ca\" (UniqueName: \"kubernetes.io/secret/7a1ef2ea-470b-4dfa-b70c-30bec8f2d908-ca\") pod \"cluster-proxy-proxy-agent-7fdc8d7ffb-7hrbq\" (UID: \"7a1ef2ea-470b-4dfa-b70c-30bec8f2d908\") " pod="open-cluster-management-agent-addon/cluster-proxy-proxy-agent-7fdc8d7ffb-7hrbq" Jul 12 17:53:49.002794 ip-10-0-129-128 kubenswrapper[2559]: I0712 17:53:49.002723 2559 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-tkbbx\" (UniqueName: \"kubernetes.io/projected/4727caec-6a00-4f71-836f-93b328b67d8e-kube-api-access-tkbbx\") pod \"managed-serviceaccount-addon-agent-7c88559b8b-qk5pk\" (UID: \"4727caec-6a00-4f71-836f-93b328b67d8e\") " pod="open-cluster-management-agent-addon/managed-serviceaccount-addon-agent-7c88559b8b-qk5pk" Jul 12 17:53:49.002794 ip-10-0-129-128 kubenswrapper[2559]: I0712 17:53:49.002771 2559 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"service-proxy-server-cert\" (UniqueName: \"kubernetes.io/secret/7a1ef2ea-470b-4dfa-b70c-30bec8f2d908-service-proxy-server-cert\") pod \"cluster-proxy-proxy-agent-7fdc8d7ffb-7hrbq\" (UID: \"7a1ef2ea-470b-4dfa-b70c-30bec8f2d908\") " pod="open-cluster-management-agent-addon/cluster-proxy-proxy-agent-7fdc8d7ffb-7hrbq" Jul 12 17:53:49.003044 ip-10-0-129-128 kubenswrapper[2559]: I0712 17:53:49.002799 2559 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"serving-cert\" (UniqueName: \"kubernetes.io/secret/356ce79c-8163-4cdc-b93e-fb248a89d42c-serving-cert\") pod \"service-ca-operator-7695fc6c8c-cnb6s\" (UID: \"356ce79c-8163-4cdc-b93e-fb248a89d42c\") " pod="openshift-service-ca-operator/service-ca-operator-7695fc6c8c-cnb6s" Jul 12 17:53:49.003044 ip-10-0-129-128 kubenswrapper[2559]: I0712 17:53:49.002831 2559 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-69wwf\" (UniqueName: \"kubernetes.io/projected/ab3f4928-9f96-49df-ad17-e6a7fe6f7df3-kube-api-access-69wwf\") pod \"volume-data-source-validator-78495597f6-dnhgq\" (UID: \"ab3f4928-9f96-49df-ad17-e6a7fe6f7df3\") " pod="openshift-cluster-storage-operator/volume-data-source-validator-78495597f6-dnhgq" Jul 12 17:53:49.003044 ip-10-0-129-128 kubenswrapper[2559]: I0712 17:53:49.002857 2559 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-jzfmv\" (UniqueName: \"kubernetes.io/projected/15d24b76-d54a-409b-91e4-6b9e4f9ac7b2-kube-api-access-jzfmv\") pod \"dns-default-wvr9g\" (UID: \"15d24b76-d54a-409b-91e4-6b9e4f9ac7b2\") " pod="openshift-dns/dns-default-wvr9g" Jul 12 17:53:49.003044 ip-10-0-129-128 kubenswrapper[2559]: I0712 17:53:49.002883 2559 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-f79q5\" (UniqueName: \"kubernetes.io/projected/98f3df53-0008-483f-bb5f-0a5718df3920-kube-api-access-f79q5\") pod \"ingress-canary-fdmcz\" (UID: \"98f3df53-0008-483f-bb5f-0a5718df3920\") " pod="openshift-ingress-canary/ingress-canary-fdmcz" Jul 12 17:53:49.003044 ip-10-0-129-128 kubenswrapper[2559]: I0712 17:53:49.002906 2559 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-4x6bh\" (UniqueName: \"kubernetes.io/projected/6c8669ba-68cf-4ec6-81d4-1c5842a3d34a-kube-api-access-4x6bh\") pod \"image-registry-85b88cb476-kzl6q\" (UID: \"6c8669ba-68cf-4ec6-81d4-1c5842a3d34a\") " pod="openshift-image-registry/image-registry-85b88cb476-kzl6q" Jul 12 17:53:49.020020 ip-10-0-129-128 kubenswrapper[2559]: I0712 17:53:49.019992 2559 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-g6rx5_64692aea-0243-4167-a3b6-2a6dc81a8fef/konnectivity-agent/0.log" limit=1024 Jul 12 17:53:49.028148 ip-10-0-129-128 kubenswrapper[2559]: I0712 17:53:49.028126 2559 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-mpqvv" Jul 12 17:53:49.030737 ip-10-0-129-128 kubenswrapper[2559]: I0712 17:53:49.030701 2559 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"kube-system\"/\"original-pull-secret\"" Jul 12 17:53:49.103693 ip-10-0-129-128 kubenswrapper[2559]: I0712 17:53:49.103635 2559 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"registry-tls\" (UniqueName: \"kubernetes.io/projected/6c8669ba-68cf-4ec6-81d4-1c5842a3d34a-registry-tls\") pod \"image-registry-85b88cb476-kzl6q\" (UID: \"6c8669ba-68cf-4ec6-81d4-1c5842a3d34a\") " pod="openshift-image-registry/image-registry-85b88cb476-kzl6q" Jul 12 17:53:49.103693 ip-10-0-129-128 kubenswrapper[2559]: I0712 17:53:49.103691 2559 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"config\" (UniqueName: \"kubernetes.io/configmap/f1d46430-bb60-4470-b8d8-eacd44f4f6b1-config\") pod \"console-operator-7866497494-wlk78\" (UID: \"f1d46430-bb60-4470-b8d8-eacd44f4f6b1\") " pod="openshift-console-operator/console-operator-7866497494-wlk78" Jul 12 17:53:49.103934 ip-10-0-129-128 kubenswrapper[2559]: I0712 17:53:49.103721 2559 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"hub-kubeconfig\" (UniqueName: \"kubernetes.io/secret/4727caec-6a00-4f71-836f-93b328b67d8e-hub-kubeconfig\") pod \"managed-serviceaccount-addon-agent-7c88559b8b-qk5pk\" (UID: \"4727caec-6a00-4f71-836f-93b328b67d8e\") " pod="open-cluster-management-agent-addon/managed-serviceaccount-addon-agent-7c88559b8b-qk5pk" Jul 12 17:53:49.103934 ip-10-0-129-128 kubenswrapper[2559]: I0712 17:53:49.103739 2559 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tmp\" (UniqueName: \"kubernetes.io/empty-dir/27899582-4f2a-4b54-a4f5-0f6167d16c1f-tmp\") pod \"klusterlet-addon-workmgr-74d7bc54f4-k7lv7\" (UID: \"27899582-4f2a-4b54-a4f5-0f6167d16c1f\") " pod="open-cluster-management-agent-addon/klusterlet-addon-workmgr-74d7bc54f4-k7lv7" Jul 12 17:53:49.103934 ip-10-0-129-128 kubenswrapper[2559]: I0712 17:53:49.103763 2559 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"klusterlet-config\" (UniqueName: \"kubernetes.io/secret/27899582-4f2a-4b54-a4f5-0f6167d16c1f-klusterlet-config\") pod \"klusterlet-addon-workmgr-74d7bc54f4-k7lv7\" (UID: \"27899582-4f2a-4b54-a4f5-0f6167d16c1f\") " pod="open-cluster-management-agent-addon/klusterlet-addon-workmgr-74d7bc54f4-k7lv7" Jul 12 17:53:49.103934 ip-10-0-129-128 kubenswrapper[2559]: I0712 17:53:49.103779 2559 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"installation-pull-secrets\" (UniqueName: \"kubernetes.io/secret/6c8669ba-68cf-4ec6-81d4-1c5842a3d34a-installation-pull-secrets\") pod \"image-registry-85b88cb476-kzl6q\" (UID: \"6c8669ba-68cf-4ec6-81d4-1c5842a3d34a\") " pod="openshift-image-registry/image-registry-85b88cb476-kzl6q" Jul 12 17:53:49.103934 ip-10-0-129-128 kubenswrapper[2559]: E0712 17:53:49.103785 2559 projected.go:266] Couldn't get secret openshift-image-registry/image-registry-tls: secret "image-registry-tls" not found Jul 12 17:53:49.103934 ip-10-0-129-128 kubenswrapper[2559]: E0712 17:53:49.103806 2559 projected.go:196] Error preparing data for projected volume registry-tls for pod openshift-image-registry/image-registry-85b88cb476-kzl6q: secret "image-registry-tls" not found Jul 12 17:53:49.103934 ip-10-0-129-128 kubenswrapper[2559]: E0712 17:53:49.103866 2559 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/projected/6c8669ba-68cf-4ec6-81d4-1c5842a3d34a-registry-tls podName:6c8669ba-68cf-4ec6-81d4-1c5842a3d34a nodeName:}" failed. No retries permitted until 2026-07-12 17:53:49.603847098 +0000 UTC m=+37.097915086 (durationBeforeRetry 500ms). Error: MountVolume.SetUp failed for volume "registry-tls" (UniqueName: "kubernetes.io/projected/6c8669ba-68cf-4ec6-81d4-1c5842a3d34a-registry-tls") pod "image-registry-85b88cb476-kzl6q" (UID: "6c8669ba-68cf-4ec6-81d4-1c5842a3d34a") : secret "image-registry-tls" not found Jul 12 17:53:49.103934 ip-10-0-129-128 kubenswrapper[2559]: I0712 17:53:49.103794 2559 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-lkq5d\" (UniqueName: \"kubernetes.io/projected/7a1ef2ea-470b-4dfa-b70c-30bec8f2d908-kube-api-access-lkq5d\") pod \"cluster-proxy-proxy-agent-7fdc8d7ffb-7hrbq\" (UID: \"7a1ef2ea-470b-4dfa-b70c-30bec8f2d908\") " pod="open-cluster-management-agent-addon/cluster-proxy-proxy-agent-7fdc8d7ffb-7hrbq" Jul 12 17:53:49.104325 ip-10-0-129-128 kubenswrapper[2559]: I0712 17:53:49.103956 2559 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-tls\" (UniqueName: \"kubernetes.io/secret/15d24b76-d54a-409b-91e4-6b9e4f9ac7b2-metrics-tls\") pod \"dns-default-wvr9g\" (UID: \"15d24b76-d54a-409b-91e4-6b9e4f9ac7b2\") " pod="openshift-dns/dns-default-wvr9g" Jul 12 17:53:49.104325 ip-10-0-129-128 kubenswrapper[2559]: I0712 17:53:49.103988 2559 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"hub-kubeconfig\" (UniqueName: \"kubernetes.io/secret/7a1ef2ea-470b-4dfa-b70c-30bec8f2d908-hub-kubeconfig\") pod \"cluster-proxy-proxy-agent-7fdc8d7ffb-7hrbq\" (UID: \"7a1ef2ea-470b-4dfa-b70c-30bec8f2d908\") " pod="open-cluster-management-agent-addon/cluster-proxy-proxy-agent-7fdc8d7ffb-7hrbq" Jul 12 17:53:49.104325 ip-10-0-129-128 kubenswrapper[2559]: E0712 17:53:49.104090 2559 secret.go:189] Couldn't get secret openshift-dns/dns-default-metrics-tls: secret "dns-default-metrics-tls" not found Jul 12 17:53:49.104325 ip-10-0-129-128 kubenswrapper[2559]: E0712 17:53:49.104175 2559 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/15d24b76-d54a-409b-91e4-6b9e4f9ac7b2-metrics-tls podName:15d24b76-d54a-409b-91e4-6b9e4f9ac7b2 nodeName:}" failed. No retries permitted until 2026-07-12 17:53:49.604132309 +0000 UTC m=+37.098200231 (durationBeforeRetry 500ms). Error: MountVolume.SetUp failed for volume "metrics-tls" (UniqueName: "kubernetes.io/secret/15d24b76-d54a-409b-91e4-6b9e4f9ac7b2-metrics-tls") pod "dns-default-wvr9g" (UID: "15d24b76-d54a-409b-91e4-6b9e4f9ac7b2") : secret "dns-default-metrics-tls" not found Jul 12 17:53:49.104325 ip-10-0-129-128 kubenswrapper[2559]: I0712 17:53:49.104212 2559 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"trusted-ca\" (UniqueName: \"kubernetes.io/configmap/f1d46430-bb60-4470-b8d8-eacd44f4f6b1-trusted-ca\") pod \"console-operator-7866497494-wlk78\" (UID: \"f1d46430-bb60-4470-b8d8-eacd44f4f6b1\") " pod="openshift-console-operator/console-operator-7866497494-wlk78" Jul 12 17:53:49.104325 ip-10-0-129-128 kubenswrapper[2559]: I0712 17:53:49.104239 2559 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"config-volume\" (UniqueName: \"kubernetes.io/configmap/15d24b76-d54a-409b-91e4-6b9e4f9ac7b2-config-volume\") pod \"dns-default-wvr9g\" (UID: \"15d24b76-d54a-409b-91e4-6b9e4f9ac7b2\") " pod="openshift-dns/dns-default-wvr9g" Jul 12 17:53:49.104325 ip-10-0-129-128 kubenswrapper[2559]: I0712 17:53:49.104260 2559 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"bound-sa-token\" (UniqueName: \"kubernetes.io/projected/6c8669ba-68cf-4ec6-81d4-1c5842a3d34a-bound-sa-token\") pod \"image-registry-85b88cb476-kzl6q\" (UID: \"6c8669ba-68cf-4ec6-81d4-1c5842a3d34a\") " pod="openshift-image-registry/image-registry-85b88cb476-kzl6q" Jul 12 17:53:49.104325 ip-10-0-129-128 kubenswrapper[2559]: I0712 17:53:49.104292 2559 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-rvs58\" (UniqueName: \"kubernetes.io/projected/356ce79c-8163-4cdc-b93e-fb248a89d42c-kube-api-access-rvs58\") pod \"service-ca-operator-7695fc6c8c-cnb6s\" (UID: \"356ce79c-8163-4cdc-b93e-fb248a89d42c\") " pod="openshift-service-ca-operator/service-ca-operator-7695fc6c8c-cnb6s" Jul 12 17:53:49.104325 ip-10-0-129-128 kubenswrapper[2559]: I0712 17:53:49.104319 2559 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"registry-certificates\" (UniqueName: \"kubernetes.io/configmap/6c8669ba-68cf-4ec6-81d4-1c5842a3d34a-registry-certificates\") pod \"image-registry-85b88cb476-kzl6q\" (UID: \"6c8669ba-68cf-4ec6-81d4-1c5842a3d34a\") " pod="openshift-image-registry/image-registry-85b88cb476-kzl6q" Jul 12 17:53:49.104798 ip-10-0-129-128 kubenswrapper[2559]: I0712 17:53:49.104343 2559 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"trusted-ca\" (UniqueName: \"kubernetes.io/configmap/6c8669ba-68cf-4ec6-81d4-1c5842a3d34a-trusted-ca\") pod \"image-registry-85b88cb476-kzl6q\" (UID: \"6c8669ba-68cf-4ec6-81d4-1c5842a3d34a\") " pod="openshift-image-registry/image-registry-85b88cb476-kzl6q" Jul 12 17:53:49.104798 ip-10-0-129-128 kubenswrapper[2559]: I0712 17:53:49.104368 2559 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-4xhmm\" (UniqueName: \"kubernetes.io/projected/e73254f8-d204-4dd4-a86f-512fc5d17ebe-kube-api-access-4xhmm\") pod \"kube-storage-version-migrator-operator-799cb7f958-rfppz\" (UID: \"e73254f8-d204-4dd4-a86f-512fc5d17ebe\") " pod="openshift-kube-storage-version-migrator-operator/kube-storage-version-migrator-operator-799cb7f958-rfppz" Jul 12 17:53:49.104798 ip-10-0-129-128 kubenswrapper[2559]: I0712 17:53:49.104390 2559 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-9mzs5\" (UniqueName: \"kubernetes.io/projected/f1d46430-bb60-4470-b8d8-eacd44f4f6b1-kube-api-access-9mzs5\") pod \"console-operator-7866497494-wlk78\" (UID: \"f1d46430-bb60-4470-b8d8-eacd44f4f6b1\") " pod="openshift-console-operator/console-operator-7866497494-wlk78" Jul 12 17:53:49.104798 ip-10-0-129-128 kubenswrapper[2559]: I0712 17:53:49.104415 2559 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"serving-cert\" (UniqueName: \"kubernetes.io/secret/e73254f8-d204-4dd4-a86f-512fc5d17ebe-serving-cert\") pod \"kube-storage-version-migrator-operator-799cb7f958-rfppz\" (UID: \"e73254f8-d204-4dd4-a86f-512fc5d17ebe\") " pod="openshift-kube-storage-version-migrator-operator/kube-storage-version-migrator-operator-799cb7f958-rfppz" Jul 12 17:53:49.104798 ip-10-0-129-128 kubenswrapper[2559]: I0712 17:53:49.104443 2559 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"ca-trust-extracted\" (UniqueName: \"kubernetes.io/empty-dir/6c8669ba-68cf-4ec6-81d4-1c5842a3d34a-ca-trust-extracted\") pod \"image-registry-85b88cb476-kzl6q\" (UID: \"6c8669ba-68cf-4ec6-81d4-1c5842a3d34a\") " pod="openshift-image-registry/image-registry-85b88cb476-kzl6q" Jul 12 17:53:49.104798 ip-10-0-129-128 kubenswrapper[2559]: I0712 17:53:49.104472 2559 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"serving-cert\" (UniqueName: \"kubernetes.io/secret/f1d46430-bb60-4470-b8d8-eacd44f4f6b1-serving-cert\") pod \"console-operator-7866497494-wlk78\" (UID: \"f1d46430-bb60-4470-b8d8-eacd44f4f6b1\") " pod="openshift-console-operator/console-operator-7866497494-wlk78" Jul 12 17:53:49.104798 ip-10-0-129-128 kubenswrapper[2559]: I0712 17:53:49.104496 2559 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"hub\" (UniqueName: \"kubernetes.io/secret/7a1ef2ea-470b-4dfa-b70c-30bec8f2d908-hub\") pod \"cluster-proxy-proxy-agent-7fdc8d7ffb-7hrbq\" (UID: \"7a1ef2ea-470b-4dfa-b70c-30bec8f2d908\") " pod="open-cluster-management-agent-addon/cluster-proxy-proxy-agent-7fdc8d7ffb-7hrbq" Jul 12 17:53:49.104798 ip-10-0-129-128 kubenswrapper[2559]: I0712 17:53:49.104517 2559 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"config\" (UniqueName: \"kubernetes.io/configmap/356ce79c-8163-4cdc-b93e-fb248a89d42c-config\") pod \"service-ca-operator-7695fc6c8c-cnb6s\" (UID: \"356ce79c-8163-4cdc-b93e-fb248a89d42c\") " pod="openshift-service-ca-operator/service-ca-operator-7695fc6c8c-cnb6s" Jul 12 17:53:49.104798 ip-10-0-129-128 kubenswrapper[2559]: I0712 17:53:49.104211 2559 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"tmp\" (UniqueName: \"kubernetes.io/empty-dir/27899582-4f2a-4b54-a4f5-0f6167d16c1f-tmp\") pod \"klusterlet-addon-workmgr-74d7bc54f4-k7lv7\" (UID: \"27899582-4f2a-4b54-a4f5-0f6167d16c1f\") " pod="open-cluster-management-agent-addon/klusterlet-addon-workmgr-74d7bc54f4-k7lv7" Jul 12 17:53:49.106271 ip-10-0-129-128 kubenswrapper[2559]: I0712 17:53:49.106063 2559 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"trusted-ca\" (UniqueName: \"kubernetes.io/configmap/f1d46430-bb60-4470-b8d8-eacd44f4f6b1-trusted-ca\") pod \"console-operator-7866497494-wlk78\" (UID: \"f1d46430-bb60-4470-b8d8-eacd44f4f6b1\") " pod="openshift-console-operator/console-operator-7866497494-wlk78" Jul 12 17:53:49.106271 ip-10-0-129-128 kubenswrapper[2559]: I0712 17:53:49.106071 2559 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"trusted-ca\" (UniqueName: \"kubernetes.io/configmap/6c8669ba-68cf-4ec6-81d4-1c5842a3d34a-trusted-ca\") pod \"image-registry-85b88cb476-kzl6q\" (UID: \"6c8669ba-68cf-4ec6-81d4-1c5842a3d34a\") " pod="openshift-image-registry/image-registry-85b88cb476-kzl6q" Jul 12 17:53:49.106510 ip-10-0-129-128 kubenswrapper[2559]: I0712 17:53:49.106487 2559 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"config\" (UniqueName: \"kubernetes.io/configmap/356ce79c-8163-4cdc-b93e-fb248a89d42c-config\") pod \"service-ca-operator-7695fc6c8c-cnb6s\" (UID: \"356ce79c-8163-4cdc-b93e-fb248a89d42c\") " pod="openshift-service-ca-operator/service-ca-operator-7695fc6c8c-cnb6s" Jul 12 17:53:49.106743 ip-10-0-129-128 kubenswrapper[2559]: I0712 17:53:49.106711 2559 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"config\" (UniqueName: \"kubernetes.io/configmap/f1d46430-bb60-4470-b8d8-eacd44f4f6b1-config\") pod \"console-operator-7866497494-wlk78\" (UID: \"f1d46430-bb60-4470-b8d8-eacd44f4f6b1\") " pod="openshift-console-operator/console-operator-7866497494-wlk78" Jul 12 17:53:49.107329 ip-10-0-129-128 kubenswrapper[2559]: I0712 17:53:49.107306 2559 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"registry-certificates\" (UniqueName: \"kubernetes.io/configmap/6c8669ba-68cf-4ec6-81d4-1c5842a3d34a-registry-certificates\") pod \"image-registry-85b88cb476-kzl6q\" (UID: \"6c8669ba-68cf-4ec6-81d4-1c5842a3d34a\") " pod="openshift-image-registry/image-registry-85b88cb476-kzl6q" Jul 12 17:53:49.107473 ip-10-0-129-128 kubenswrapper[2559]: I0712 17:53:49.107370 2559 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"additional-service-ca\" (UniqueName: \"kubernetes.io/configmap/7a1ef2ea-470b-4dfa-b70c-30bec8f2d908-additional-service-ca\") pod \"cluster-proxy-proxy-agent-7fdc8d7ffb-7hrbq\" (UID: \"7a1ef2ea-470b-4dfa-b70c-30bec8f2d908\") " pod="open-cluster-management-agent-addon/cluster-proxy-proxy-agent-7fdc8d7ffb-7hrbq" Jul 12 17:53:49.107473 ip-10-0-129-128 kubenswrapper[2559]: I0712 17:53:49.107440 2559 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-qcdjb\" (UniqueName: \"kubernetes.io/projected/27899582-4f2a-4b54-a4f5-0f6167d16c1f-kube-api-access-qcdjb\") pod \"klusterlet-addon-workmgr-74d7bc54f4-k7lv7\" (UID: \"27899582-4f2a-4b54-a4f5-0f6167d16c1f\") " pod="open-cluster-management-agent-addon/klusterlet-addon-workmgr-74d7bc54f4-k7lv7" Jul 12 17:53:49.107580 ip-10-0-129-128 kubenswrapper[2559]: I0712 17:53:49.107476 2559 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"samples-operator-tls\" (UniqueName: \"kubernetes.io/secret/a91aea93-bc91-4394-b212-c7699e8593e2-samples-operator-tls\") pod \"cluster-samples-operator-65db858894-996qw\" (UID: \"a91aea93-bc91-4394-b212-c7699e8593e2\") " pod="openshift-cluster-samples-operator/cluster-samples-operator-65db858894-996qw" Jul 12 17:53:49.107580 ip-10-0-129-128 kubenswrapper[2559]: I0712 17:53:49.107517 2559 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/15d24b76-d54a-409b-91e4-6b9e4f9ac7b2-tmp-dir\") pod \"dns-default-wvr9g\" (UID: \"15d24b76-d54a-409b-91e4-6b9e4f9ac7b2\") " pod="openshift-dns/dns-default-wvr9g" Jul 12 17:53:49.107580 ip-10-0-129-128 kubenswrapper[2559]: I0712 17:53:49.107567 2559 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-m54kw\" (UniqueName: \"kubernetes.io/projected/358e2b76-e0cb-45cb-b0ac-9360910b4e05-kube-api-access-m54kw\") pod \"network-check-source-869dcfcdcd-whn86\" (UID: \"358e2b76-e0cb-45cb-b0ac-9360910b4e05\") " pod="openshift-network-diagnostics/network-check-source-869dcfcdcd-whn86" Jul 12 17:53:49.107780 ip-10-0-129-128 kubenswrapper[2559]: I0712 17:53:49.107606 2559 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-q4mg9\" (UniqueName: \"kubernetes.io/projected/a91aea93-bc91-4394-b212-c7699e8593e2-kube-api-access-q4mg9\") pod \"cluster-samples-operator-65db858894-996qw\" (UID: \"a91aea93-bc91-4394-b212-c7699e8593e2\") " pod="openshift-cluster-samples-operator/cluster-samples-operator-65db858894-996qw" Jul 12 17:53:49.107780 ip-10-0-129-128 kubenswrapper[2559]: I0712 17:53:49.107663 2559 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"image-registry-private-configuration\" (UniqueName: \"kubernetes.io/secret/6c8669ba-68cf-4ec6-81d4-1c5842a3d34a-image-registry-private-configuration\") pod \"image-registry-85b88cb476-kzl6q\" (UID: \"6c8669ba-68cf-4ec6-81d4-1c5842a3d34a\") " pod="openshift-image-registry/image-registry-85b88cb476-kzl6q" Jul 12 17:53:49.107780 ip-10-0-129-128 kubenswrapper[2559]: I0712 17:53:49.107692 2559 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"ca\" (UniqueName: \"kubernetes.io/secret/7a1ef2ea-470b-4dfa-b70c-30bec8f2d908-ca\") pod \"cluster-proxy-proxy-agent-7fdc8d7ffb-7hrbq\" (UID: \"7a1ef2ea-470b-4dfa-b70c-30bec8f2d908\") " pod="open-cluster-management-agent-addon/cluster-proxy-proxy-agent-7fdc8d7ffb-7hrbq" Jul 12 17:53:49.107780 ip-10-0-129-128 kubenswrapper[2559]: I0712 17:53:49.107725 2559 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-tkbbx\" (UniqueName: \"kubernetes.io/projected/4727caec-6a00-4f71-836f-93b328b67d8e-kube-api-access-tkbbx\") pod \"managed-serviceaccount-addon-agent-7c88559b8b-qk5pk\" (UID: \"4727caec-6a00-4f71-836f-93b328b67d8e\") " pod="open-cluster-management-agent-addon/managed-serviceaccount-addon-agent-7c88559b8b-qk5pk" Jul 12 17:53:49.107780 ip-10-0-129-128 kubenswrapper[2559]: I0712 17:53:49.107769 2559 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"service-proxy-server-cert\" (UniqueName: \"kubernetes.io/secret/7a1ef2ea-470b-4dfa-b70c-30bec8f2d908-service-proxy-server-cert\") pod \"cluster-proxy-proxy-agent-7fdc8d7ffb-7hrbq\" (UID: \"7a1ef2ea-470b-4dfa-b70c-30bec8f2d908\") " pod="open-cluster-management-agent-addon/cluster-proxy-proxy-agent-7fdc8d7ffb-7hrbq" Jul 12 17:53:49.108017 ip-10-0-129-128 kubenswrapper[2559]: I0712 17:53:49.107805 2559 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"serving-cert\" (UniqueName: \"kubernetes.io/secret/356ce79c-8163-4cdc-b93e-fb248a89d42c-serving-cert\") pod \"service-ca-operator-7695fc6c8c-cnb6s\" (UID: \"356ce79c-8163-4cdc-b93e-fb248a89d42c\") " pod="openshift-service-ca-operator/service-ca-operator-7695fc6c8c-cnb6s" Jul 12 17:53:49.108017 ip-10-0-129-128 kubenswrapper[2559]: I0712 17:53:49.107842 2559 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-69wwf\" (UniqueName: \"kubernetes.io/projected/ab3f4928-9f96-49df-ad17-e6a7fe6f7df3-kube-api-access-69wwf\") pod \"volume-data-source-validator-78495597f6-dnhgq\" (UID: \"ab3f4928-9f96-49df-ad17-e6a7fe6f7df3\") " pod="openshift-cluster-storage-operator/volume-data-source-validator-78495597f6-dnhgq" Jul 12 17:53:49.108017 ip-10-0-129-128 kubenswrapper[2559]: I0712 17:53:49.107871 2559 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-jzfmv\" (UniqueName: \"kubernetes.io/projected/15d24b76-d54a-409b-91e4-6b9e4f9ac7b2-kube-api-access-jzfmv\") pod \"dns-default-wvr9g\" (UID: \"15d24b76-d54a-409b-91e4-6b9e4f9ac7b2\") " pod="openshift-dns/dns-default-wvr9g" Jul 12 17:53:49.108017 ip-10-0-129-128 kubenswrapper[2559]: I0712 17:53:49.107910 2559 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-f79q5\" (UniqueName: \"kubernetes.io/projected/98f3df53-0008-483f-bb5f-0a5718df3920-kube-api-access-f79q5\") pod \"ingress-canary-fdmcz\" (UID: \"98f3df53-0008-483f-bb5f-0a5718df3920\") " pod="openshift-ingress-canary/ingress-canary-fdmcz" Jul 12 17:53:49.108017 ip-10-0-129-128 kubenswrapper[2559]: I0712 17:53:49.107943 2559 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-4x6bh\" (UniqueName: \"kubernetes.io/projected/6c8669ba-68cf-4ec6-81d4-1c5842a3d34a-kube-api-access-4x6bh\") pod \"image-registry-85b88cb476-kzl6q\" (UID: \"6c8669ba-68cf-4ec6-81d4-1c5842a3d34a\") " pod="openshift-image-registry/image-registry-85b88cb476-kzl6q" Jul 12 17:53:49.108017 ip-10-0-129-128 kubenswrapper[2559]: I0712 17:53:49.107998 2559 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"config\" (UniqueName: \"kubernetes.io/configmap/e73254f8-d204-4dd4-a86f-512fc5d17ebe-config\") pod \"kube-storage-version-migrator-operator-799cb7f958-rfppz\" (UID: \"e73254f8-d204-4dd4-a86f-512fc5d17ebe\") " pod="openshift-kube-storage-version-migrator-operator/kube-storage-version-migrator-operator-799cb7f958-rfppz" Jul 12 17:53:49.108287 ip-10-0-129-128 kubenswrapper[2559]: I0712 17:53:49.108030 2559 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"cert\" (UniqueName: \"kubernetes.io/secret/98f3df53-0008-483f-bb5f-0a5718df3920-cert\") pod \"ingress-canary-fdmcz\" (UID: \"98f3df53-0008-483f-bb5f-0a5718df3920\") " pod="openshift-ingress-canary/ingress-canary-fdmcz" Jul 12 17:53:49.108781 ip-10-0-129-128 kubenswrapper[2559]: I0712 17:53:49.108760 2559 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"config-volume\" (UniqueName: \"kubernetes.io/configmap/15d24b76-d54a-409b-91e4-6b9e4f9ac7b2-config-volume\") pod \"dns-default-wvr9g\" (UID: \"15d24b76-d54a-409b-91e4-6b9e4f9ac7b2\") " pod="openshift-dns/dns-default-wvr9g" Jul 12 17:53:49.108977 ip-10-0-129-128 kubenswrapper[2559]: I0712 17:53:49.108943 2559 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"ca-trust-extracted\" (UniqueName: \"kubernetes.io/empty-dir/6c8669ba-68cf-4ec6-81d4-1c5842a3d34a-ca-trust-extracted\") pod \"image-registry-85b88cb476-kzl6q\" (UID: \"6c8669ba-68cf-4ec6-81d4-1c5842a3d34a\") " pod="openshift-image-registry/image-registry-85b88cb476-kzl6q" Jul 12 17:53:49.109156 ip-10-0-129-128 kubenswrapper[2559]: I0712 17:53:49.109124 2559 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"klusterlet-config\" (UniqueName: \"kubernetes.io/secret/27899582-4f2a-4b54-a4f5-0f6167d16c1f-klusterlet-config\") pod \"klusterlet-addon-workmgr-74d7bc54f4-k7lv7\" (UID: \"27899582-4f2a-4b54-a4f5-0f6167d16c1f\") " pod="open-cluster-management-agent-addon/klusterlet-addon-workmgr-74d7bc54f4-k7lv7" Jul 12 17:53:49.109314 ip-10-0-129-128 kubenswrapper[2559]: I0712 17:53:49.109289 2559 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"hub-kubeconfig\" (UniqueName: \"kubernetes.io/secret/7a1ef2ea-470b-4dfa-b70c-30bec8f2d908-hub-kubeconfig\") pod \"cluster-proxy-proxy-agent-7fdc8d7ffb-7hrbq\" (UID: \"7a1ef2ea-470b-4dfa-b70c-30bec8f2d908\") " pod="open-cluster-management-agent-addon/cluster-proxy-proxy-agent-7fdc8d7ffb-7hrbq" Jul 12 17:53:49.110976 ip-10-0-129-128 kubenswrapper[2559]: I0712 17:53:49.110383 2559 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"installation-pull-secrets\" (UniqueName: \"kubernetes.io/secret/6c8669ba-68cf-4ec6-81d4-1c5842a3d34a-installation-pull-secrets\") pod \"image-registry-85b88cb476-kzl6q\" (UID: \"6c8669ba-68cf-4ec6-81d4-1c5842a3d34a\") " pod="openshift-image-registry/image-registry-85b88cb476-kzl6q" Jul 12 17:53:49.111805 ip-10-0-129-128 kubenswrapper[2559]: I0712 17:53:49.111309 2559 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"additional-service-ca\" (UniqueName: \"kubernetes.io/configmap/7a1ef2ea-470b-4dfa-b70c-30bec8f2d908-additional-service-ca\") pod \"cluster-proxy-proxy-agent-7fdc8d7ffb-7hrbq\" (UID: \"7a1ef2ea-470b-4dfa-b70c-30bec8f2d908\") " pod="open-cluster-management-agent-addon/cluster-proxy-proxy-agent-7fdc8d7ffb-7hrbq" Jul 12 17:53:49.112073 ip-10-0-129-128 kubenswrapper[2559]: E0712 17:53:49.112054 2559 secret.go:189] Couldn't get secret openshift-cluster-samples-operator/samples-operator-tls: secret "samples-operator-tls" not found Jul 12 17:53:49.112163 ip-10-0-129-128 kubenswrapper[2559]: E0712 17:53:49.112126 2559 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/a91aea93-bc91-4394-b212-c7699e8593e2-samples-operator-tls podName:a91aea93-bc91-4394-b212-c7699e8593e2 nodeName:}" failed. No retries permitted until 2026-07-12 17:53:49.612108522 +0000 UTC m=+37.106176443 (durationBeforeRetry 500ms). Error: MountVolume.SetUp failed for volume "samples-operator-tls" (UniqueName: "kubernetes.io/secret/a91aea93-bc91-4394-b212-c7699e8593e2-samples-operator-tls") pod "cluster-samples-operator-65db858894-996qw" (UID: "a91aea93-bc91-4394-b212-c7699e8593e2") : secret "samples-operator-tls" not found Jul 12 17:53:49.112163 ip-10-0-129-128 kubenswrapper[2559]: I0712 17:53:49.112121 2559 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"cert\" (UniqueName: \"kubernetes.io/secret/98f3df53-0008-483f-bb5f-0a5718df3920-cert\") pod \"ingress-canary-fdmcz\" (UID: \"98f3df53-0008-483f-bb5f-0a5718df3920\") " pod="openshift-ingress-canary/ingress-canary-fdmcz" Jul 12 17:53:49.112428 ip-10-0-129-128 kubenswrapper[2559]: I0712 17:53:49.112407 2559 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/15d24b76-d54a-409b-91e4-6b9e4f9ac7b2-tmp-dir\") pod \"dns-default-wvr9g\" (UID: \"15d24b76-d54a-409b-91e4-6b9e4f9ac7b2\") " pod="openshift-dns/dns-default-wvr9g" Jul 12 17:53:49.113699 ip-10-0-129-128 kubenswrapper[2559]: I0712 17:53:49.113267 2559 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"image-registry-private-configuration\" (UniqueName: \"kubernetes.io/secret/6c8669ba-68cf-4ec6-81d4-1c5842a3d34a-image-registry-private-configuration\") pod \"image-registry-85b88cb476-kzl6q\" (UID: \"6c8669ba-68cf-4ec6-81d4-1c5842a3d34a\") " pod="openshift-image-registry/image-registry-85b88cb476-kzl6q" Jul 12 17:53:49.113699 ip-10-0-129-128 kubenswrapper[2559]: I0712 17:53:49.113404 2559 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"serving-cert\" (UniqueName: \"kubernetes.io/secret/e73254f8-d204-4dd4-a86f-512fc5d17ebe-serving-cert\") pod \"kube-storage-version-migrator-operator-799cb7f958-rfppz\" (UID: \"e73254f8-d204-4dd4-a86f-512fc5d17ebe\") " pod="openshift-kube-storage-version-migrator-operator/kube-storage-version-migrator-operator-799cb7f958-rfppz" Jul 12 17:53:49.114543 ip-10-0-129-128 kubenswrapper[2559]: I0712 17:53:49.113993 2559 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"serving-cert\" (UniqueName: \"kubernetes.io/secret/f1d46430-bb60-4470-b8d8-eacd44f4f6b1-serving-cert\") pod \"console-operator-7866497494-wlk78\" (UID: \"f1d46430-bb60-4470-b8d8-eacd44f4f6b1\") " pod="openshift-console-operator/console-operator-7866497494-wlk78" Jul 12 17:53:49.114543 ip-10-0-129-128 kubenswrapper[2559]: I0712 17:53:49.114068 2559 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"config\" (UniqueName: \"kubernetes.io/configmap/e73254f8-d204-4dd4-a86f-512fc5d17ebe-config\") pod \"kube-storage-version-migrator-operator-799cb7f958-rfppz\" (UID: \"e73254f8-d204-4dd4-a86f-512fc5d17ebe\") " pod="openshift-kube-storage-version-migrator-operator/kube-storage-version-migrator-operator-799cb7f958-rfppz" Jul 12 17:53:49.114543 ip-10-0-129-128 kubenswrapper[2559]: I0712 17:53:49.114538 2559 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"hub\" (UniqueName: \"kubernetes.io/secret/7a1ef2ea-470b-4dfa-b70c-30bec8f2d908-hub\") pod \"cluster-proxy-proxy-agent-7fdc8d7ffb-7hrbq\" (UID: \"7a1ef2ea-470b-4dfa-b70c-30bec8f2d908\") " pod="open-cluster-management-agent-addon/cluster-proxy-proxy-agent-7fdc8d7ffb-7hrbq" Jul 12 17:53:49.115663 ip-10-0-129-128 kubenswrapper[2559]: I0712 17:53:49.114996 2559 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"service-proxy-server-cert\" (UniqueName: \"kubernetes.io/secret/7a1ef2ea-470b-4dfa-b70c-30bec8f2d908-service-proxy-server-cert\") pod \"cluster-proxy-proxy-agent-7fdc8d7ffb-7hrbq\" (UID: \"7a1ef2ea-470b-4dfa-b70c-30bec8f2d908\") " pod="open-cluster-management-agent-addon/cluster-proxy-proxy-agent-7fdc8d7ffb-7hrbq" Jul 12 17:53:49.115663 ip-10-0-129-128 kubenswrapper[2559]: I0712 17:53:49.115181 2559 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"ca\" (UniqueName: \"kubernetes.io/secret/7a1ef2ea-470b-4dfa-b70c-30bec8f2d908-ca\") pod \"cluster-proxy-proxy-agent-7fdc8d7ffb-7hrbq\" (UID: \"7a1ef2ea-470b-4dfa-b70c-30bec8f2d908\") " pod="open-cluster-management-agent-addon/cluster-proxy-proxy-agent-7fdc8d7ffb-7hrbq" Jul 12 17:53:49.115663 ip-10-0-129-128 kubenswrapper[2559]: I0712 17:53:49.115495 2559 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"serving-cert\" (UniqueName: \"kubernetes.io/secret/356ce79c-8163-4cdc-b93e-fb248a89d42c-serving-cert\") pod \"service-ca-operator-7695fc6c8c-cnb6s\" (UID: \"356ce79c-8163-4cdc-b93e-fb248a89d42c\") " pod="openshift-service-ca-operator/service-ca-operator-7695fc6c8c-cnb6s" Jul 12 17:53:49.116070 ip-10-0-129-128 kubenswrapper[2559]: I0712 17:53:49.116049 2559 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-lkq5d\" (UniqueName: \"kubernetes.io/projected/7a1ef2ea-470b-4dfa-b70c-30bec8f2d908-kube-api-access-lkq5d\") pod \"cluster-proxy-proxy-agent-7fdc8d7ffb-7hrbq\" (UID: \"7a1ef2ea-470b-4dfa-b70c-30bec8f2d908\") " pod="open-cluster-management-agent-addon/cluster-proxy-proxy-agent-7fdc8d7ffb-7hrbq" Jul 12 17:53:49.116793 ip-10-0-129-128 kubenswrapper[2559]: I0712 17:53:49.116771 2559 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"hub-kubeconfig\" (UniqueName: \"kubernetes.io/secret/4727caec-6a00-4f71-836f-93b328b67d8e-hub-kubeconfig\") pod \"managed-serviceaccount-addon-agent-7c88559b8b-qk5pk\" (UID: \"4727caec-6a00-4f71-836f-93b328b67d8e\") " pod="open-cluster-management-agent-addon/managed-serviceaccount-addon-agent-7c88559b8b-qk5pk" Jul 12 17:53:49.117807 ip-10-0-129-128 kubenswrapper[2559]: I0712 17:53:49.117774 2559 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-rvs58\" (UniqueName: \"kubernetes.io/projected/356ce79c-8163-4cdc-b93e-fb248a89d42c-kube-api-access-rvs58\") pod \"service-ca-operator-7695fc6c8c-cnb6s\" (UID: \"356ce79c-8163-4cdc-b93e-fb248a89d42c\") " pod="openshift-service-ca-operator/service-ca-operator-7695fc6c8c-cnb6s" Jul 12 17:53:49.117874 ip-10-0-129-128 kubenswrapper[2559]: I0712 17:53:49.117802 2559 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-9mzs5\" (UniqueName: \"kubernetes.io/projected/f1d46430-bb60-4470-b8d8-eacd44f4f6b1-kube-api-access-9mzs5\") pod \"console-operator-7866497494-wlk78\" (UID: \"f1d46430-bb60-4470-b8d8-eacd44f4f6b1\") " pod="openshift-console-operator/console-operator-7866497494-wlk78" Jul 12 17:53:49.126332 ip-10-0-129-128 kubenswrapper[2559]: I0712 17:53:49.125595 2559 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-4xhmm\" (UniqueName: \"kubernetes.io/projected/e73254f8-d204-4dd4-a86f-512fc5d17ebe-kube-api-access-4xhmm\") pod \"kube-storage-version-migrator-operator-799cb7f958-rfppz\" (UID: \"e73254f8-d204-4dd4-a86f-512fc5d17ebe\") " pod="openshift-kube-storage-version-migrator-operator/kube-storage-version-migrator-operator-799cb7f958-rfppz" Jul 12 17:53:49.126332 ip-10-0-129-128 kubenswrapper[2559]: I0712 17:53:49.125902 2559 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-tkbbx\" (UniqueName: \"kubernetes.io/projected/4727caec-6a00-4f71-836f-93b328b67d8e-kube-api-access-tkbbx\") pod \"managed-serviceaccount-addon-agent-7c88559b8b-qk5pk\" (UID: \"4727caec-6a00-4f71-836f-93b328b67d8e\") " pod="open-cluster-management-agent-addon/managed-serviceaccount-addon-agent-7c88559b8b-qk5pk" Jul 12 17:53:49.126332 ip-10-0-129-128 kubenswrapper[2559]: I0712 17:53:49.126295 2559 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"bound-sa-token\" (UniqueName: \"kubernetes.io/projected/6c8669ba-68cf-4ec6-81d4-1c5842a3d34a-bound-sa-token\") pod \"image-registry-85b88cb476-kzl6q\" (UID: \"6c8669ba-68cf-4ec6-81d4-1c5842a3d34a\") " pod="openshift-image-registry/image-registry-85b88cb476-kzl6q" Jul 12 17:53:49.126566 ip-10-0-129-128 kubenswrapper[2559]: I0712 17:53:49.126340 2559 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-m54kw\" (UniqueName: \"kubernetes.io/projected/358e2b76-e0cb-45cb-b0ac-9360910b4e05-kube-api-access-m54kw\") pod \"network-check-source-869dcfcdcd-whn86\" (UID: \"358e2b76-e0cb-45cb-b0ac-9360910b4e05\") " pod="openshift-network-diagnostics/network-check-source-869dcfcdcd-whn86" Jul 12 17:53:49.126566 ip-10-0-129-128 kubenswrapper[2559]: I0712 17:53:49.126396 2559 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-jzfmv\" (UniqueName: \"kubernetes.io/projected/15d24b76-d54a-409b-91e4-6b9e4f9ac7b2-kube-api-access-jzfmv\") pod \"dns-default-wvr9g\" (UID: \"15d24b76-d54a-409b-91e4-6b9e4f9ac7b2\") " pod="openshift-dns/dns-default-wvr9g" Jul 12 17:53:49.126566 ip-10-0-129-128 kubenswrapper[2559]: I0712 17:53:49.126408 2559 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-f79q5\" (UniqueName: \"kubernetes.io/projected/98f3df53-0008-483f-bb5f-0a5718df3920-kube-api-access-f79q5\") pod \"ingress-canary-fdmcz\" (UID: \"98f3df53-0008-483f-bb5f-0a5718df3920\") " pod="openshift-ingress-canary/ingress-canary-fdmcz" Jul 12 17:53:49.126751 ip-10-0-129-128 kubenswrapper[2559]: I0712 17:53:49.126731 2559 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-69wwf\" (UniqueName: \"kubernetes.io/projected/ab3f4928-9f96-49df-ad17-e6a7fe6f7df3-kube-api-access-69wwf\") pod \"volume-data-source-validator-78495597f6-dnhgq\" (UID: \"ab3f4928-9f96-49df-ad17-e6a7fe6f7df3\") " pod="openshift-cluster-storage-operator/volume-data-source-validator-78495597f6-dnhgq" Jul 12 17:53:49.127608 ip-10-0-129-128 kubenswrapper[2559]: I0712 17:53:49.127578 2559 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-q4mg9\" (UniqueName: \"kubernetes.io/projected/a91aea93-bc91-4394-b212-c7699e8593e2-kube-api-access-q4mg9\") pod \"cluster-samples-operator-65db858894-996qw\" (UID: \"a91aea93-bc91-4394-b212-c7699e8593e2\") " pod="openshift-cluster-samples-operator/cluster-samples-operator-65db858894-996qw" Jul 12 17:53:49.127812 ip-10-0-129-128 kubenswrapper[2559]: I0712 17:53:49.127795 2559 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-4x6bh\" (UniqueName: \"kubernetes.io/projected/6c8669ba-68cf-4ec6-81d4-1c5842a3d34a-kube-api-access-4x6bh\") pod \"image-registry-85b88cb476-kzl6q\" (UID: \"6c8669ba-68cf-4ec6-81d4-1c5842a3d34a\") " pod="openshift-image-registry/image-registry-85b88cb476-kzl6q" Jul 12 17:53:49.128003 ip-10-0-129-128 kubenswrapper[2559]: I0712 17:53:49.127987 2559 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-qcdjb\" (UniqueName: \"kubernetes.io/projected/27899582-4f2a-4b54-a4f5-0f6167d16c1f-kube-api-access-qcdjb\") pod \"klusterlet-addon-workmgr-74d7bc54f4-k7lv7\" (UID: \"27899582-4f2a-4b54-a4f5-0f6167d16c1f\") " pod="open-cluster-management-agent-addon/klusterlet-addon-workmgr-74d7bc54f4-k7lv7" Jul 12 17:53:49.176785 ip-10-0-129-128 kubenswrapper[2559]: I0712 17:53:49.176757 2559 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-network-console/networking-console-plugin-6cf856758c-7ghw7"] Jul 12 17:53:49.179505 ip-10-0-129-128 kubenswrapper[2559]: I0712 17:53:49.179477 2559 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-console/networking-console-plugin-6cf856758c-7ghw7" Jul 12 17:53:49.182088 ip-10-0-129-128 kubenswrapper[2559]: I0712 17:53:49.182066 2559 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-network-console\"/\"networking-console-plugin-cert\"" Jul 12 17:53:49.182195 ip-10-0-129-128 kubenswrapper[2559]: I0712 17:53:49.182072 2559 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-network-console\"/\"networking-console-plugin\"" Jul 12 17:53:49.182439 ip-10-0-129-128 kubenswrapper[2559]: I0712 17:53:49.182422 2559 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-network-console\"/\"default-dockercfg-7lxn6\"" Jul 12 17:53:49.187357 ip-10-0-129-128 kubenswrapper[2559]: I0712 17:53:49.187337 2559 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-network-console/networking-console-plugin-6cf856758c-7ghw7"] Jul 12 17:53:49.191207 ip-10-0-129-128 kubenswrapper[2559]: I0712 17:53:49.191189 2559 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-cluster-storage-operator/volume-data-source-validator-78495597f6-dnhgq" Jul 12 17:53:49.256597 ip-10-0-129-128 kubenswrapper[2559]: I0712 17:53:49.256438 2559 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-console-operator/console-operator-7866497494-wlk78" Jul 12 17:53:49.261515 ip-10-0-129-128 kubenswrapper[2559]: I0712 17:53:49.261446 2559 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-service-ca-operator/service-ca-operator-7695fc6c8c-cnb6s" Jul 12 17:53:49.272873 ip-10-0-129-128 kubenswrapper[2559]: I0712 17:53:49.272759 2559 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-kube-storage-version-migrator-operator/kube-storage-version-migrator-operator-799cb7f958-rfppz" Jul 12 17:53:49.275571 ip-10-0-129-128 kubenswrapper[2559]: I0712 17:53:49.275513 2559 generic.go:356] "Generic (PLEG): container finished" podID="370d4e7b-6713-48b2-81ee-f055fe1d5490" containerID="1fc3c7b71a8834ac3540a4adce9de1402918f47d88bf6f6a92871589d476246f" exitCode=0 Jul 12 17:53:49.275571 ip-10-0-129-128 kubenswrapper[2559]: I0712 17:53:49.275552 2559 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-multus/multus-additional-cni-plugins-9dh8l" event={"ID":"370d4e7b-6713-48b2-81ee-f055fe1d5490","Type":"ContainerDied","Data":"1fc3c7b71a8834ac3540a4adce9de1402918f47d88bf6f6a92871589d476246f"} Jul 12 17:53:49.279722 ip-10-0-129-128 kubenswrapper[2559]: I0712 17:53:49.279009 2559 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-source-869dcfcdcd-whn86" Jul 12 17:53:49.300198 ip-10-0-129-128 kubenswrapper[2559]: I0712 17:53:49.299992 2559 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="open-cluster-management-agent-addon/cluster-proxy-proxy-agent-7fdc8d7ffb-7hrbq" Jul 12 17:53:49.305990 ip-10-0-129-128 kubenswrapper[2559]: I0712 17:53:49.305929 2559 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress-canary/ingress-canary-fdmcz" Jul 12 17:53:49.310958 ip-10-0-129-128 kubenswrapper[2559]: I0712 17:53:49.310670 2559 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"networking-console-plugin-cert\" (UniqueName: \"kubernetes.io/secret/2edbd7ab-1773-4fbf-af70-0b2cfd5cc642-networking-console-plugin-cert\") pod \"networking-console-plugin-6cf856758c-7ghw7\" (UID: \"2edbd7ab-1773-4fbf-af70-0b2cfd5cc642\") " pod="openshift-network-console/networking-console-plugin-6cf856758c-7ghw7" Jul 12 17:53:49.310958 ip-10-0-129-128 kubenswrapper[2559]: I0712 17:53:49.310733 2559 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"nginx-conf\" (UniqueName: \"kubernetes.io/configmap/2edbd7ab-1773-4fbf-af70-0b2cfd5cc642-nginx-conf\") pod \"networking-console-plugin-6cf856758c-7ghw7\" (UID: \"2edbd7ab-1773-4fbf-af70-0b2cfd5cc642\") " pod="openshift-network-console/networking-console-plugin-6cf856758c-7ghw7" Jul 12 17:53:49.313075 ip-10-0-129-128 kubenswrapper[2559]: I0712 17:53:49.312822 2559 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="open-cluster-management-agent-addon/klusterlet-addon-workmgr-74d7bc54f4-k7lv7" Jul 12 17:53:49.320511 ip-10-0-129-128 kubenswrapper[2559]: I0712 17:53:49.320216 2559 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="open-cluster-management-agent-addon/managed-serviceaccount-addon-agent-7c88559b8b-qk5pk" Jul 12 17:53:49.362541 ip-10-0-129-128 kubenswrapper[2559]: I0712 17:53:49.361988 2559 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-cluster-storage-operator/volume-data-source-validator-78495597f6-dnhgq"] Jul 12 17:53:49.421160 ip-10-0-129-128 kubenswrapper[2559]: I0712 17:53:49.412914 2559 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"nginx-conf\" (UniqueName: \"kubernetes.io/configmap/2edbd7ab-1773-4fbf-af70-0b2cfd5cc642-nginx-conf\") pod \"networking-console-plugin-6cf856758c-7ghw7\" (UID: \"2edbd7ab-1773-4fbf-af70-0b2cfd5cc642\") " pod="openshift-network-console/networking-console-plugin-6cf856758c-7ghw7" Jul 12 17:53:49.421160 ip-10-0-129-128 kubenswrapper[2559]: I0712 17:53:49.413134 2559 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"networking-console-plugin-cert\" (UniqueName: \"kubernetes.io/secret/2edbd7ab-1773-4fbf-af70-0b2cfd5cc642-networking-console-plugin-cert\") pod \"networking-console-plugin-6cf856758c-7ghw7\" (UID: \"2edbd7ab-1773-4fbf-af70-0b2cfd5cc642\") " pod="openshift-network-console/networking-console-plugin-6cf856758c-7ghw7" Jul 12 17:53:49.421160 ip-10-0-129-128 kubenswrapper[2559]: I0712 17:53:49.414544 2559 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"nginx-conf\" (UniqueName: \"kubernetes.io/configmap/2edbd7ab-1773-4fbf-af70-0b2cfd5cc642-nginx-conf\") pod \"networking-console-plugin-6cf856758c-7ghw7\" (UID: \"2edbd7ab-1773-4fbf-af70-0b2cfd5cc642\") " pod="openshift-network-console/networking-console-plugin-6cf856758c-7ghw7" Jul 12 17:53:49.421160 ip-10-0-129-128 kubenswrapper[2559]: E0712 17:53:49.415147 2559 secret.go:189] Couldn't get secret openshift-network-console/networking-console-plugin-cert: secret "networking-console-plugin-cert" not found Jul 12 17:53:49.421160 ip-10-0-129-128 kubenswrapper[2559]: E0712 17:53:49.415206 2559 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/2edbd7ab-1773-4fbf-af70-0b2cfd5cc642-networking-console-plugin-cert podName:2edbd7ab-1773-4fbf-af70-0b2cfd5cc642 nodeName:}" failed. No retries permitted until 2026-07-12 17:53:49.915187095 +0000 UTC m=+37.409255008 (durationBeforeRetry 500ms). Error: MountVolume.SetUp failed for volume "networking-console-plugin-cert" (UniqueName: "kubernetes.io/secret/2edbd7ab-1773-4fbf-af70-0b2cfd5cc642-networking-console-plugin-cert") pod "networking-console-plugin-6cf856758c-7ghw7" (UID: "2edbd7ab-1773-4fbf-af70-0b2cfd5cc642") : secret "networking-console-plugin-cert" not found Jul 12 17:53:49.508788 ip-10-0-129-128 kubenswrapper[2559]: I0712 17:53:49.508571 2559 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-service-ca-operator/service-ca-operator-7695fc6c8c-cnb6s"] Jul 12 17:53:49.513808 ip-10-0-129-128 kubenswrapper[2559]: I0712 17:53:49.511007 2559 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-console-operator/console-operator-7866497494-wlk78"] Jul 12 17:53:49.532917 ip-10-0-129-128 kubenswrapper[2559]: I0712 17:53:49.530527 2559 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-kube-storage-version-migrator-operator/kube-storage-version-migrator-operator-799cb7f958-rfppz"] Jul 12 17:53:49.582564 ip-10-0-129-128 kubenswrapper[2559]: W0712 17:53:49.582527 2559 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pode73254f8_d204_4dd4_a86f_512fc5d17ebe.slice/crio-80970c1b7280b2e225bb600b11f96cbcdacfc6a72f303f87cfd4401cdc1ae3e6 WatchSource:0}: Error finding container 80970c1b7280b2e225bb600b11f96cbcdacfc6a72f303f87cfd4401cdc1ae3e6: Status 404 returned error can't find the container with id 80970c1b7280b2e225bb600b11f96cbcdacfc6a72f303f87cfd4401cdc1ae3e6 Jul 12 17:53:49.587807 ip-10-0-129-128 kubenswrapper[2559]: I0712 17:53:49.587773 2559 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress-canary/ingress-canary-fdmcz"] Jul 12 17:53:49.588679 ip-10-0-129-128 kubenswrapper[2559]: I0712 17:53:49.588618 2559 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-network-diagnostics/network-check-source-869dcfcdcd-whn86"] Jul 12 17:53:49.593173 ip-10-0-129-128 kubenswrapper[2559]: I0712 17:53:49.593148 2559 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["open-cluster-management-agent-addon/cluster-proxy-proxy-agent-7fdc8d7ffb-7hrbq"] Jul 12 17:53:49.609289 ip-10-0-129-128 kubenswrapper[2559]: I0712 17:53:49.609263 2559 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["open-cluster-management-agent-addon/managed-serviceaccount-addon-agent-7c88559b8b-qk5pk"] Jul 12 17:53:49.619920 ip-10-0-129-128 kubenswrapper[2559]: I0712 17:53:49.619892 2559 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["open-cluster-management-agent-addon/klusterlet-addon-workmgr-74d7bc54f4-k7lv7"] Jul 12 17:53:49.620198 ip-10-0-129-128 kubenswrapper[2559]: I0712 17:53:49.620180 2559 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"samples-operator-tls\" (UniqueName: \"kubernetes.io/secret/a91aea93-bc91-4394-b212-c7699e8593e2-samples-operator-tls\") pod \"cluster-samples-operator-65db858894-996qw\" (UID: \"a91aea93-bc91-4394-b212-c7699e8593e2\") " pod="openshift-cluster-samples-operator/cluster-samples-operator-65db858894-996qw" Jul 12 17:53:49.620252 ip-10-0-129-128 kubenswrapper[2559]: I0712 17:53:49.620239 2559 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"registry-tls\" (UniqueName: \"kubernetes.io/projected/6c8669ba-68cf-4ec6-81d4-1c5842a3d34a-registry-tls\") pod \"image-registry-85b88cb476-kzl6q\" (UID: \"6c8669ba-68cf-4ec6-81d4-1c5842a3d34a\") " pod="openshift-image-registry/image-registry-85b88cb476-kzl6q" Jul 12 17:53:49.620300 ip-10-0-129-128 kubenswrapper[2559]: I0712 17:53:49.620285 2559 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-tls\" (UniqueName: \"kubernetes.io/secret/15d24b76-d54a-409b-91e4-6b9e4f9ac7b2-metrics-tls\") pod \"dns-default-wvr9g\" (UID: \"15d24b76-d54a-409b-91e4-6b9e4f9ac7b2\") " pod="openshift-dns/dns-default-wvr9g" Jul 12 17:53:49.620344 ip-10-0-129-128 kubenswrapper[2559]: E0712 17:53:49.620332 2559 secret.go:189] Couldn't get secret openshift-cluster-samples-operator/samples-operator-tls: secret "samples-operator-tls" not found Jul 12 17:53:49.620435 ip-10-0-129-128 kubenswrapper[2559]: E0712 17:53:49.620335 2559 projected.go:266] Couldn't get secret openshift-image-registry/image-registry-tls: secret "image-registry-tls" not found Jul 12 17:53:49.620435 ip-10-0-129-128 kubenswrapper[2559]: E0712 17:53:49.620405 2559 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/a91aea93-bc91-4394-b212-c7699e8593e2-samples-operator-tls podName:a91aea93-bc91-4394-b212-c7699e8593e2 nodeName:}" failed. No retries permitted until 2026-07-12 17:53:50.620379891 +0000 UTC m=+38.114447817 (durationBeforeRetry 1s). Error: MountVolume.SetUp failed for volume "samples-operator-tls" (UniqueName: "kubernetes.io/secret/a91aea93-bc91-4394-b212-c7699e8593e2-samples-operator-tls") pod "cluster-samples-operator-65db858894-996qw" (UID: "a91aea93-bc91-4394-b212-c7699e8593e2") : secret "samples-operator-tls" not found Jul 12 17:53:49.620435 ip-10-0-129-128 kubenswrapper[2559]: E0712 17:53:49.620408 2559 secret.go:189] Couldn't get secret openshift-dns/dns-default-metrics-tls: secret "dns-default-metrics-tls" not found Jul 12 17:53:49.620632 ip-10-0-129-128 kubenswrapper[2559]: E0712 17:53:49.620460 2559 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/15d24b76-d54a-409b-91e4-6b9e4f9ac7b2-metrics-tls podName:15d24b76-d54a-409b-91e4-6b9e4f9ac7b2 nodeName:}" failed. No retries permitted until 2026-07-12 17:53:50.620442936 +0000 UTC m=+38.114510845 (durationBeforeRetry 1s). Error: MountVolume.SetUp failed for volume "metrics-tls" (UniqueName: "kubernetes.io/secret/15d24b76-d54a-409b-91e4-6b9e4f9ac7b2-metrics-tls") pod "dns-default-wvr9g" (UID: "15d24b76-d54a-409b-91e4-6b9e4f9ac7b2") : secret "dns-default-metrics-tls" not found Jul 12 17:53:49.620632 ip-10-0-129-128 kubenswrapper[2559]: E0712 17:53:49.620407 2559 projected.go:196] Error preparing data for projected volume registry-tls for pod openshift-image-registry/image-registry-85b88cb476-kzl6q: secret "image-registry-tls" not found Jul 12 17:53:49.620632 ip-10-0-129-128 kubenswrapper[2559]: E0712 17:53:49.620502 2559 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/projected/6c8669ba-68cf-4ec6-81d4-1c5842a3d34a-registry-tls podName:6c8669ba-68cf-4ec6-81d4-1c5842a3d34a nodeName:}" failed. No retries permitted until 2026-07-12 17:53:50.620492831 +0000 UTC m=+38.114560744 (durationBeforeRetry 1s). Error: MountVolume.SetUp failed for volume "registry-tls" (UniqueName: "kubernetes.io/projected/6c8669ba-68cf-4ec6-81d4-1c5842a3d34a-registry-tls") pod "image-registry-85b88cb476-kzl6q" (UID: "6c8669ba-68cf-4ec6-81d4-1c5842a3d34a") : secret "image-registry-tls" not found Jul 12 17:53:49.622767 ip-10-0-129-128 kubenswrapper[2559]: W0712 17:53:49.622749 2559 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-besteffort.slice/kubepods-besteffort-pod4727caec_6a00_4f71_836f_93b328b67d8e.slice/crio-4c39d3391bf8bca0ec739a1b47f70631707c3dc093fe9d6fc1c44bcd3edde94b WatchSource:0}: Error finding container 4c39d3391bf8bca0ec739a1b47f70631707c3dc093fe9d6fc1c44bcd3edde94b: Status 404 returned error can't find the container with id 4c39d3391bf8bca0ec739a1b47f70631707c3dc093fe9d6fc1c44bcd3edde94b Jul 12 17:53:49.623842 ip-10-0-129-128 kubenswrapper[2559]: W0712 17:53:49.623824 2559 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-besteffort.slice/kubepods-besteffort-pod7a1ef2ea_470b_4dfa_b70c_30bec8f2d908.slice/crio-f8a25c726c7113cd682e6f8a8269839e61f1b468f183c173ff7e920e8a9a486d WatchSource:0}: Error finding container f8a25c726c7113cd682e6f8a8269839e61f1b468f183c173ff7e920e8a9a486d: Status 404 returned error can't find the container with id f8a25c726c7113cd682e6f8a8269839e61f1b468f183c173ff7e920e8a9a486d Jul 12 17:53:49.629339 ip-10-0-129-128 kubenswrapper[2559]: W0712 17:53:49.629224 2559 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod98f3df53_0008_483f_bb5f_0a5718df3920.slice/crio-a7b0c7555bf07fb410c0c4d8cce00a117e22d8fc0b3eacf43cbd244e41350354 WatchSource:0}: Error finding container a7b0c7555bf07fb410c0c4d8cce00a117e22d8fc0b3eacf43cbd244e41350354: Status 404 returned error can't find the container with id a7b0c7555bf07fb410c0c4d8cce00a117e22d8fc0b3eacf43cbd244e41350354 Jul 12 17:53:49.922543 ip-10-0-129-128 kubenswrapper[2559]: I0712 17:53:49.922436 2559 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"networking-console-plugin-cert\" (UniqueName: \"kubernetes.io/secret/2edbd7ab-1773-4fbf-af70-0b2cfd5cc642-networking-console-plugin-cert\") pod \"networking-console-plugin-6cf856758c-7ghw7\" (UID: \"2edbd7ab-1773-4fbf-af70-0b2cfd5cc642\") " pod="openshift-network-console/networking-console-plugin-6cf856758c-7ghw7" Jul 12 17:53:49.922735 ip-10-0-129-128 kubenswrapper[2559]: E0712 17:53:49.922589 2559 secret.go:189] Couldn't get secret openshift-network-console/networking-console-plugin-cert: secret "networking-console-plugin-cert" not found Jul 12 17:53:49.922735 ip-10-0-129-128 kubenswrapper[2559]: E0712 17:53:49.922670 2559 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/2edbd7ab-1773-4fbf-af70-0b2cfd5cc642-networking-console-plugin-cert podName:2edbd7ab-1773-4fbf-af70-0b2cfd5cc642 nodeName:}" failed. No retries permitted until 2026-07-12 17:53:50.922652223 +0000 UTC m=+38.416720144 (durationBeforeRetry 1s). Error: MountVolume.SetUp failed for volume "networking-console-plugin-cert" (UniqueName: "kubernetes.io/secret/2edbd7ab-1773-4fbf-af70-0b2cfd5cc642-networking-console-plugin-cert") pod "networking-console-plugin-6cf856758c-7ghw7" (UID: "2edbd7ab-1773-4fbf-af70-0b2cfd5cc642") : secret "networking-console-plugin-cert" not found Jul 12 17:53:50.028791 ip-10-0-129-128 kubenswrapper[2559]: I0712 17:53:50.028755 2559 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-h4bgw" Jul 12 17:53:50.029870 ip-10-0-129-128 kubenswrapper[2559]: I0712 17:53:50.029357 2559 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-qtkvt" Jul 12 17:53:50.032700 ip-10-0-129-128 kubenswrapper[2559]: I0712 17:53:50.032156 2559 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-network-diagnostics\"/\"default-dockercfg-h9zfm\"" Jul 12 17:53:50.032700 ip-10-0-129-128 kubenswrapper[2559]: I0712 17:53:50.032377 2559 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-multus\"/\"metrics-daemon-sa-dockercfg-84rf9\"" Jul 12 17:53:50.032700 ip-10-0-129-128 kubenswrapper[2559]: I0712 17:53:50.032532 2559 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-multus\"/\"metrics-daemon-secret\"" Jul 12 17:53:50.286050 ip-10-0-129-128 kubenswrapper[2559]: I0712 17:53:50.285978 2559 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-cluster-storage-operator/volume-data-source-validator-78495597f6-dnhgq" event={"ID":"ab3f4928-9f96-49df-ad17-e6a7fe6f7df3","Type":"ContainerStarted","Data":"fe878d045cf7a4c7f277e96fb77279be7d95ca7df9d2883d6c39f893c32e03e9"} Jul 12 17:53:50.299206 ip-10-0-129-128 kubenswrapper[2559]: I0712 17:53:50.299170 2559 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-network-diagnostics/network-check-source-869dcfcdcd-whn86" event={"ID":"358e2b76-e0cb-45cb-b0ac-9360910b4e05","Type":"ContainerStarted","Data":"4fd641ab5193e65fc667bcac000d4d4edf4b56d1d9982504dc914a7252da3fd8"} Jul 12 17:53:50.321740 ip-10-0-129-128 kubenswrapper[2559]: I0712 17:53:50.321707 2559 generic.go:356] "Generic (PLEG): container finished" podID="370d4e7b-6713-48b2-81ee-f055fe1d5490" containerID="0a47d9da370ee718c475828048687720c6171d9d96a8d369e22052cb501d8ce4" exitCode=0 Jul 12 17:53:50.321909 ip-10-0-129-128 kubenswrapper[2559]: I0712 17:53:50.321835 2559 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-multus/multus-additional-cni-plugins-9dh8l" event={"ID":"370d4e7b-6713-48b2-81ee-f055fe1d5490","Type":"ContainerDied","Data":"0a47d9da370ee718c475828048687720c6171d9d96a8d369e22052cb501d8ce4"} Jul 12 17:53:50.337905 ip-10-0-129-128 kubenswrapper[2559]: I0712 17:53:50.337825 2559 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="open-cluster-management-agent-addon/managed-serviceaccount-addon-agent-7c88559b8b-qk5pk" event={"ID":"4727caec-6a00-4f71-836f-93b328b67d8e","Type":"ContainerStarted","Data":"4c39d3391bf8bca0ec739a1b47f70631707c3dc093fe9d6fc1c44bcd3edde94b"} Jul 12 17:53:50.346099 ip-10-0-129-128 kubenswrapper[2559]: I0712 17:53:50.346021 2559 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="open-cluster-management-agent-addon/klusterlet-addon-workmgr-74d7bc54f4-k7lv7" event={"ID":"27899582-4f2a-4b54-a4f5-0f6167d16c1f","Type":"ContainerStarted","Data":"04ebbf3c5ac0dfabe5ad5c2a5adc7d08097cb66695d520fa0c6c584cfca19005"} Jul 12 17:53:50.357866 ip-10-0-129-128 kubenswrapper[2559]: I0712 17:53:50.357821 2559 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-kube-storage-version-migrator-operator/kube-storage-version-migrator-operator-799cb7f958-rfppz" event={"ID":"e73254f8-d204-4dd4-a86f-512fc5d17ebe","Type":"ContainerStarted","Data":"80970c1b7280b2e225bb600b11f96cbcdacfc6a72f303f87cfd4401cdc1ae3e6"} Jul 12 17:53:50.373823 ip-10-0-129-128 kubenswrapper[2559]: I0712 17:53:50.372141 2559 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-service-ca-operator/service-ca-operator-7695fc6c8c-cnb6s" event={"ID":"356ce79c-8163-4cdc-b93e-fb248a89d42c","Type":"ContainerStarted","Data":"6a460c910ac2e027192f97594712ee2cd3b7f234f0f02667f0bf5b1a76c76619"} Jul 12 17:53:50.382661 ip-10-0-129-128 kubenswrapper[2559]: I0712 17:53:50.382584 2559 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress-canary/ingress-canary-fdmcz" event={"ID":"98f3df53-0008-483f-bb5f-0a5718df3920","Type":"ContainerStarted","Data":"a7b0c7555bf07fb410c0c4d8cce00a117e22d8fc0b3eacf43cbd244e41350354"} Jul 12 17:53:50.398808 ip-10-0-129-128 kubenswrapper[2559]: I0712 17:53:50.398772 2559 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="open-cluster-management-agent-addon/cluster-proxy-proxy-agent-7fdc8d7ffb-7hrbq" event={"ID":"7a1ef2ea-470b-4dfa-b70c-30bec8f2d908","Type":"ContainerStarted","Data":"f8a25c726c7113cd682e6f8a8269839e61f1b468f183c173ff7e920e8a9a486d"} Jul 12 17:53:50.424087 ip-10-0-129-128 kubenswrapper[2559]: I0712 17:53:50.424048 2559 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-console-operator/console-operator-7866497494-wlk78" event={"ID":"f1d46430-bb60-4470-b8d8-eacd44f4f6b1","Type":"ContainerStarted","Data":"19e2b143a43242fd985237ed4dd2905336a413ceb4bfb58df16f99daf779a544"} Jul 12 17:53:50.629932 ip-10-0-129-128 kubenswrapper[2559]: I0712 17:53:50.628976 2559 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"samples-operator-tls\" (UniqueName: \"kubernetes.io/secret/a91aea93-bc91-4394-b212-c7699e8593e2-samples-operator-tls\") pod \"cluster-samples-operator-65db858894-996qw\" (UID: \"a91aea93-bc91-4394-b212-c7699e8593e2\") " pod="openshift-cluster-samples-operator/cluster-samples-operator-65db858894-996qw" Jul 12 17:53:50.629932 ip-10-0-129-128 kubenswrapper[2559]: I0712 17:53:50.629073 2559 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"registry-tls\" (UniqueName: \"kubernetes.io/projected/6c8669ba-68cf-4ec6-81d4-1c5842a3d34a-registry-tls\") pod \"image-registry-85b88cb476-kzl6q\" (UID: \"6c8669ba-68cf-4ec6-81d4-1c5842a3d34a\") " pod="openshift-image-registry/image-registry-85b88cb476-kzl6q" Jul 12 17:53:50.629932 ip-10-0-129-128 kubenswrapper[2559]: I0712 17:53:50.629122 2559 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-tls\" (UniqueName: \"kubernetes.io/secret/15d24b76-d54a-409b-91e4-6b9e4f9ac7b2-metrics-tls\") pod \"dns-default-wvr9g\" (UID: \"15d24b76-d54a-409b-91e4-6b9e4f9ac7b2\") " pod="openshift-dns/dns-default-wvr9g" Jul 12 17:53:50.629932 ip-10-0-129-128 kubenswrapper[2559]: E0712 17:53:50.629259 2559 secret.go:189] Couldn't get secret openshift-dns/dns-default-metrics-tls: secret "dns-default-metrics-tls" not found Jul 12 17:53:50.629932 ip-10-0-129-128 kubenswrapper[2559]: E0712 17:53:50.629324 2559 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/15d24b76-d54a-409b-91e4-6b9e4f9ac7b2-metrics-tls podName:15d24b76-d54a-409b-91e4-6b9e4f9ac7b2 nodeName:}" failed. No retries permitted until 2026-07-12 17:53:52.629304358 +0000 UTC m=+40.123372270 (durationBeforeRetry 2s). Error: MountVolume.SetUp failed for volume "metrics-tls" (UniqueName: "kubernetes.io/secret/15d24b76-d54a-409b-91e4-6b9e4f9ac7b2-metrics-tls") pod "dns-default-wvr9g" (UID: "15d24b76-d54a-409b-91e4-6b9e4f9ac7b2") : secret "dns-default-metrics-tls" not found Jul 12 17:53:50.629932 ip-10-0-129-128 kubenswrapper[2559]: E0712 17:53:50.629393 2559 secret.go:189] Couldn't get secret openshift-cluster-samples-operator/samples-operator-tls: secret "samples-operator-tls" not found Jul 12 17:53:50.629932 ip-10-0-129-128 kubenswrapper[2559]: E0712 17:53:50.629428 2559 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/a91aea93-bc91-4394-b212-c7699e8593e2-samples-operator-tls podName:a91aea93-bc91-4394-b212-c7699e8593e2 nodeName:}" failed. No retries permitted until 2026-07-12 17:53:52.629416259 +0000 UTC m=+40.123484172 (durationBeforeRetry 2s). Error: MountVolume.SetUp failed for volume "samples-operator-tls" (UniqueName: "kubernetes.io/secret/a91aea93-bc91-4394-b212-c7699e8593e2-samples-operator-tls") pod "cluster-samples-operator-65db858894-996qw" (UID: "a91aea93-bc91-4394-b212-c7699e8593e2") : secret "samples-operator-tls" not found Jul 12 17:53:50.629932 ip-10-0-129-128 kubenswrapper[2559]: E0712 17:53:50.629510 2559 projected.go:266] Couldn't get secret openshift-image-registry/image-registry-tls: secret "image-registry-tls" not found Jul 12 17:53:50.629932 ip-10-0-129-128 kubenswrapper[2559]: E0712 17:53:50.629521 2559 projected.go:196] Error preparing data for projected volume registry-tls for pod openshift-image-registry/image-registry-85b88cb476-kzl6q: secret "image-registry-tls" not found Jul 12 17:53:50.629932 ip-10-0-129-128 kubenswrapper[2559]: E0712 17:53:50.629552 2559 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/projected/6c8669ba-68cf-4ec6-81d4-1c5842a3d34a-registry-tls podName:6c8669ba-68cf-4ec6-81d4-1c5842a3d34a nodeName:}" failed. No retries permitted until 2026-07-12 17:53:52.629541455 +0000 UTC m=+40.123609365 (durationBeforeRetry 2s). Error: MountVolume.SetUp failed for volume "registry-tls" (UniqueName: "kubernetes.io/projected/6c8669ba-68cf-4ec6-81d4-1c5842a3d34a-registry-tls") pod "image-registry-85b88cb476-kzl6q" (UID: "6c8669ba-68cf-4ec6-81d4-1c5842a3d34a") : secret "image-registry-tls" not found Jul 12 17:53:50.932793 ip-10-0-129-128 kubenswrapper[2559]: I0712 17:53:50.932135 2559 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"networking-console-plugin-cert\" (UniqueName: \"kubernetes.io/secret/2edbd7ab-1773-4fbf-af70-0b2cfd5cc642-networking-console-plugin-cert\") pod \"networking-console-plugin-6cf856758c-7ghw7\" (UID: \"2edbd7ab-1773-4fbf-af70-0b2cfd5cc642\") " pod="openshift-network-console/networking-console-plugin-6cf856758c-7ghw7" Jul 12 17:53:50.932793 ip-10-0-129-128 kubenswrapper[2559]: E0712 17:53:50.932298 2559 secret.go:189] Couldn't get secret openshift-network-console/networking-console-plugin-cert: secret "networking-console-plugin-cert" not found Jul 12 17:53:50.932793 ip-10-0-129-128 kubenswrapper[2559]: E0712 17:53:50.932365 2559 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/2edbd7ab-1773-4fbf-af70-0b2cfd5cc642-networking-console-plugin-cert podName:2edbd7ab-1773-4fbf-af70-0b2cfd5cc642 nodeName:}" failed. No retries permitted until 2026-07-12 17:53:52.932345264 +0000 UTC m=+40.426413195 (durationBeforeRetry 2s). Error: MountVolume.SetUp failed for volume "networking-console-plugin-cert" (UniqueName: "kubernetes.io/secret/2edbd7ab-1773-4fbf-af70-0b2cfd5cc642-networking-console-plugin-cert") pod "networking-console-plugin-6cf856758c-7ghw7" (UID: "2edbd7ab-1773-4fbf-af70-0b2cfd5cc642") : secret "networking-console-plugin-cert" not found Jul 12 17:53:51.444004 ip-10-0-129-128 kubenswrapper[2559]: I0712 17:53:51.442855 2559 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-multus/multus-additional-cni-plugins-9dh8l" event={"ID":"370d4e7b-6713-48b2-81ee-f055fe1d5490","Type":"ContainerStarted","Data":"227f5d317eff66adefa0ecc46ca40d6a549b0f3c16072e96c9a75cf3e01e8437"} Jul 12 17:53:52.651428 ip-10-0-129-128 kubenswrapper[2559]: I0712 17:53:52.649131 2559 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"registry-tls\" (UniqueName: \"kubernetes.io/projected/6c8669ba-68cf-4ec6-81d4-1c5842a3d34a-registry-tls\") pod \"image-registry-85b88cb476-kzl6q\" (UID: \"6c8669ba-68cf-4ec6-81d4-1c5842a3d34a\") " pod="openshift-image-registry/image-registry-85b88cb476-kzl6q" Jul 12 17:53:52.651428 ip-10-0-129-128 kubenswrapper[2559]: I0712 17:53:52.649223 2559 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-tls\" (UniqueName: \"kubernetes.io/secret/15d24b76-d54a-409b-91e4-6b9e4f9ac7b2-metrics-tls\") pod \"dns-default-wvr9g\" (UID: \"15d24b76-d54a-409b-91e4-6b9e4f9ac7b2\") " pod="openshift-dns/dns-default-wvr9g" Jul 12 17:53:52.651428 ip-10-0-129-128 kubenswrapper[2559]: I0712 17:53:52.649325 2559 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"samples-operator-tls\" (UniqueName: \"kubernetes.io/secret/a91aea93-bc91-4394-b212-c7699e8593e2-samples-operator-tls\") pod \"cluster-samples-operator-65db858894-996qw\" (UID: \"a91aea93-bc91-4394-b212-c7699e8593e2\") " pod="openshift-cluster-samples-operator/cluster-samples-operator-65db858894-996qw" Jul 12 17:53:52.651428 ip-10-0-129-128 kubenswrapper[2559]: E0712 17:53:52.649863 2559 projected.go:266] Couldn't get secret openshift-image-registry/image-registry-tls: secret "image-registry-tls" not found Jul 12 17:53:52.651428 ip-10-0-129-128 kubenswrapper[2559]: E0712 17:53:52.649881 2559 projected.go:196] Error preparing data for projected volume registry-tls for pod openshift-image-registry/image-registry-85b88cb476-kzl6q: secret "image-registry-tls" not found Jul 12 17:53:52.651428 ip-10-0-129-128 kubenswrapper[2559]: E0712 17:53:52.649935 2559 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/projected/6c8669ba-68cf-4ec6-81d4-1c5842a3d34a-registry-tls podName:6c8669ba-68cf-4ec6-81d4-1c5842a3d34a nodeName:}" failed. No retries permitted until 2026-07-12 17:53:56.649916906 +0000 UTC m=+44.143984828 (durationBeforeRetry 4s). Error: MountVolume.SetUp failed for volume "registry-tls" (UniqueName: "kubernetes.io/projected/6c8669ba-68cf-4ec6-81d4-1c5842a3d34a-registry-tls") pod "image-registry-85b88cb476-kzl6q" (UID: "6c8669ba-68cf-4ec6-81d4-1c5842a3d34a") : secret "image-registry-tls" not found Jul 12 17:53:52.651428 ip-10-0-129-128 kubenswrapper[2559]: E0712 17:53:52.650382 2559 secret.go:189] Couldn't get secret openshift-dns/dns-default-metrics-tls: secret "dns-default-metrics-tls" not found Jul 12 17:53:52.651428 ip-10-0-129-128 kubenswrapper[2559]: E0712 17:53:52.650430 2559 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/15d24b76-d54a-409b-91e4-6b9e4f9ac7b2-metrics-tls podName:15d24b76-d54a-409b-91e4-6b9e4f9ac7b2 nodeName:}" failed. No retries permitted until 2026-07-12 17:53:56.650414919 +0000 UTC m=+44.144482836 (durationBeforeRetry 4s). Error: MountVolume.SetUp failed for volume "metrics-tls" (UniqueName: "kubernetes.io/secret/15d24b76-d54a-409b-91e4-6b9e4f9ac7b2-metrics-tls") pod "dns-default-wvr9g" (UID: "15d24b76-d54a-409b-91e4-6b9e4f9ac7b2") : secret "dns-default-metrics-tls" not found Jul 12 17:53:52.651428 ip-10-0-129-128 kubenswrapper[2559]: E0712 17:53:52.651086 2559 secret.go:189] Couldn't get secret openshift-cluster-samples-operator/samples-operator-tls: secret "samples-operator-tls" not found Jul 12 17:53:52.651428 ip-10-0-129-128 kubenswrapper[2559]: E0712 17:53:52.651130 2559 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/a91aea93-bc91-4394-b212-c7699e8593e2-samples-operator-tls podName:a91aea93-bc91-4394-b212-c7699e8593e2 nodeName:}" failed. No retries permitted until 2026-07-12 17:53:56.651115761 +0000 UTC m=+44.145183682 (durationBeforeRetry 4s). Error: MountVolume.SetUp failed for volume "samples-operator-tls" (UniqueName: "kubernetes.io/secret/a91aea93-bc91-4394-b212-c7699e8593e2-samples-operator-tls") pod "cluster-samples-operator-65db858894-996qw" (UID: "a91aea93-bc91-4394-b212-c7699e8593e2") : secret "samples-operator-tls" not found Jul 12 17:53:52.953713 ip-10-0-129-128 kubenswrapper[2559]: I0712 17:53:52.953194 2559 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"networking-console-plugin-cert\" (UniqueName: \"kubernetes.io/secret/2edbd7ab-1773-4fbf-af70-0b2cfd5cc642-networking-console-plugin-cert\") pod \"networking-console-plugin-6cf856758c-7ghw7\" (UID: \"2edbd7ab-1773-4fbf-af70-0b2cfd5cc642\") " pod="openshift-network-console/networking-console-plugin-6cf856758c-7ghw7" Jul 12 17:53:52.953713 ip-10-0-129-128 kubenswrapper[2559]: E0712 17:53:52.953358 2559 secret.go:189] Couldn't get secret openshift-network-console/networking-console-plugin-cert: secret "networking-console-plugin-cert" not found Jul 12 17:53:52.953713 ip-10-0-129-128 kubenswrapper[2559]: E0712 17:53:52.953421 2559 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/2edbd7ab-1773-4fbf-af70-0b2cfd5cc642-networking-console-plugin-cert podName:2edbd7ab-1773-4fbf-af70-0b2cfd5cc642 nodeName:}" failed. No retries permitted until 2026-07-12 17:53:56.953401999 +0000 UTC m=+44.447469909 (durationBeforeRetry 4s). Error: MountVolume.SetUp failed for volume "networking-console-plugin-cert" (UniqueName: "kubernetes.io/secret/2edbd7ab-1773-4fbf-af70-0b2cfd5cc642-networking-console-plugin-cert") pod "networking-console-plugin-6cf856758c-7ghw7" (UID: "2edbd7ab-1773-4fbf-af70-0b2cfd5cc642") : secret "networking-console-plugin-cert" not found Jul 12 17:53:56.684310 ip-10-0-129-128 kubenswrapper[2559]: I0712 17:53:56.684276 2559 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-tls\" (UniqueName: \"kubernetes.io/secret/15d24b76-d54a-409b-91e4-6b9e4f9ac7b2-metrics-tls\") pod \"dns-default-wvr9g\" (UID: \"15d24b76-d54a-409b-91e4-6b9e4f9ac7b2\") " pod="openshift-dns/dns-default-wvr9g" Jul 12 17:53:56.684688 ip-10-0-129-128 kubenswrapper[2559]: I0712 17:53:56.684371 2559 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"samples-operator-tls\" (UniqueName: \"kubernetes.io/secret/a91aea93-bc91-4394-b212-c7699e8593e2-samples-operator-tls\") pod \"cluster-samples-operator-65db858894-996qw\" (UID: \"a91aea93-bc91-4394-b212-c7699e8593e2\") " pod="openshift-cluster-samples-operator/cluster-samples-operator-65db858894-996qw" Jul 12 17:53:56.684688 ip-10-0-129-128 kubenswrapper[2559]: E0712 17:53:56.684434 2559 secret.go:189] Couldn't get secret openshift-dns/dns-default-metrics-tls: secret "dns-default-metrics-tls" not found Jul 12 17:53:56.684688 ip-10-0-129-128 kubenswrapper[2559]: I0712 17:53:56.684442 2559 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"registry-tls\" (UniqueName: \"kubernetes.io/projected/6c8669ba-68cf-4ec6-81d4-1c5842a3d34a-registry-tls\") pod \"image-registry-85b88cb476-kzl6q\" (UID: \"6c8669ba-68cf-4ec6-81d4-1c5842a3d34a\") " pod="openshift-image-registry/image-registry-85b88cb476-kzl6q" Jul 12 17:53:56.684688 ip-10-0-129-128 kubenswrapper[2559]: E0712 17:53:56.684504 2559 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/15d24b76-d54a-409b-91e4-6b9e4f9ac7b2-metrics-tls podName:15d24b76-d54a-409b-91e4-6b9e4f9ac7b2 nodeName:}" failed. No retries permitted until 2026-07-12 17:54:04.684482608 +0000 UTC m=+52.178550539 (durationBeforeRetry 8s). Error: MountVolume.SetUp failed for volume "metrics-tls" (UniqueName: "kubernetes.io/secret/15d24b76-d54a-409b-91e4-6b9e4f9ac7b2-metrics-tls") pod "dns-default-wvr9g" (UID: "15d24b76-d54a-409b-91e4-6b9e4f9ac7b2") : secret "dns-default-metrics-tls" not found Jul 12 17:53:56.684688 ip-10-0-129-128 kubenswrapper[2559]: E0712 17:53:56.684544 2559 projected.go:266] Couldn't get secret openshift-image-registry/image-registry-tls: secret "image-registry-tls" not found Jul 12 17:53:56.684688 ip-10-0-129-128 kubenswrapper[2559]: E0712 17:53:56.684544 2559 secret.go:189] Couldn't get secret openshift-cluster-samples-operator/samples-operator-tls: secret "samples-operator-tls" not found Jul 12 17:53:56.684688 ip-10-0-129-128 kubenswrapper[2559]: E0712 17:53:56.684628 2559 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/a91aea93-bc91-4394-b212-c7699e8593e2-samples-operator-tls podName:a91aea93-bc91-4394-b212-c7699e8593e2 nodeName:}" failed. No retries permitted until 2026-07-12 17:54:04.684595389 +0000 UTC m=+52.178663312 (durationBeforeRetry 8s). Error: MountVolume.SetUp failed for volume "samples-operator-tls" (UniqueName: "kubernetes.io/secret/a91aea93-bc91-4394-b212-c7699e8593e2-samples-operator-tls") pod "cluster-samples-operator-65db858894-996qw" (UID: "a91aea93-bc91-4394-b212-c7699e8593e2") : secret "samples-operator-tls" not found Jul 12 17:53:56.684688 ip-10-0-129-128 kubenswrapper[2559]: E0712 17:53:56.684556 2559 projected.go:196] Error preparing data for projected volume registry-tls for pod openshift-image-registry/image-registry-85b88cb476-kzl6q: secret "image-registry-tls" not found Jul 12 17:53:56.684984 ip-10-0-129-128 kubenswrapper[2559]: E0712 17:53:56.684719 2559 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/projected/6c8669ba-68cf-4ec6-81d4-1c5842a3d34a-registry-tls podName:6c8669ba-68cf-4ec6-81d4-1c5842a3d34a nodeName:}" failed. No retries permitted until 2026-07-12 17:54:04.684700948 +0000 UTC m=+52.178768883 (durationBeforeRetry 8s). Error: MountVolume.SetUp failed for volume "registry-tls" (UniqueName: "kubernetes.io/projected/6c8669ba-68cf-4ec6-81d4-1c5842a3d34a-registry-tls") pod "image-registry-85b88cb476-kzl6q" (UID: "6c8669ba-68cf-4ec6-81d4-1c5842a3d34a") : secret "image-registry-tls" not found Jul 12 17:53:56.987596 ip-10-0-129-128 kubenswrapper[2559]: I0712 17:53:56.987506 2559 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"networking-console-plugin-cert\" (UniqueName: \"kubernetes.io/secret/2edbd7ab-1773-4fbf-af70-0b2cfd5cc642-networking-console-plugin-cert\") pod \"networking-console-plugin-6cf856758c-7ghw7\" (UID: \"2edbd7ab-1773-4fbf-af70-0b2cfd5cc642\") " pod="openshift-network-console/networking-console-plugin-6cf856758c-7ghw7" Jul 12 17:53:56.987771 ip-10-0-129-128 kubenswrapper[2559]: E0712 17:53:56.987654 2559 secret.go:189] Couldn't get secret openshift-network-console/networking-console-plugin-cert: secret "networking-console-plugin-cert" not found Jul 12 17:53:56.987771 ip-10-0-129-128 kubenswrapper[2559]: E0712 17:53:56.987731 2559 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/2edbd7ab-1773-4fbf-af70-0b2cfd5cc642-networking-console-plugin-cert podName:2edbd7ab-1773-4fbf-af70-0b2cfd5cc642 nodeName:}" failed. No retries permitted until 2026-07-12 17:54:04.987709382 +0000 UTC m=+52.481777313 (durationBeforeRetry 8s). Error: MountVolume.SetUp failed for volume "networking-console-plugin-cert" (UniqueName: "kubernetes.io/secret/2edbd7ab-1773-4fbf-af70-0b2cfd5cc642-networking-console-plugin-cert") pod "networking-console-plugin-6cf856758c-7ghw7" (UID: "2edbd7ab-1773-4fbf-af70-0b2cfd5cc642") : secret "networking-console-plugin-cert" not found Jul 12 17:54:03.480878 ip-10-0-129-128 kubenswrapper[2559]: I0712 17:54:03.480840 2559 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress-canary/ingress-canary-fdmcz" event={"ID":"98f3df53-0008-483f-bb5f-0a5718df3920","Type":"ContainerStarted","Data":"4d25b938af520e754be6d545088fe1d33758e726d6f114784c210037d16b5dfb"} Jul 12 17:54:03.483140 ip-10-0-129-128 kubenswrapper[2559]: I0712 17:54:03.483110 2559 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="open-cluster-management-agent-addon/cluster-proxy-proxy-agent-7fdc8d7ffb-7hrbq" event={"ID":"7a1ef2ea-470b-4dfa-b70c-30bec8f2d908","Type":"ContainerStarted","Data":"b018de6ca7b48ea5fd4a85113e0bed76f987d11422249dfc5a4385dd8c5c32bc"} Jul 12 17:54:03.483297 ip-10-0-129-128 kubenswrapper[2559]: I0712 17:54:03.483149 2559 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="open-cluster-management-agent-addon/cluster-proxy-proxy-agent-7fdc8d7ffb-7hrbq" event={"ID":"7a1ef2ea-470b-4dfa-b70c-30bec8f2d908","Type":"ContainerStarted","Data":"61118d238ebdc283bf6ed40726c1131f616f0e71393016f61dfc7265b20eb5c0"} Jul 12 17:54:03.483297 ip-10-0-129-128 kubenswrapper[2559]: I0712 17:54:03.483167 2559 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="open-cluster-management-agent-addon/cluster-proxy-proxy-agent-7fdc8d7ffb-7hrbq" event={"ID":"7a1ef2ea-470b-4dfa-b70c-30bec8f2d908","Type":"ContainerStarted","Data":"8c1506c20c384e5167d69dd0550b15b6c5e1d7f699ef0986c6c82c9f8b5fbefa"} Jul 12 17:54:03.484864 ip-10-0-129-128 kubenswrapper[2559]: I0712 17:54:03.484838 2559 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-console-operator_console-operator-7866497494-wlk78_f1d46430-bb60-4470-b8d8-eacd44f4f6b1/console-operator/0.log" Jul 12 17:54:03.484956 ip-10-0-129-128 kubenswrapper[2559]: I0712 17:54:03.484880 2559 generic.go:356] "Generic (PLEG): container finished" podID="f1d46430-bb60-4470-b8d8-eacd44f4f6b1" containerID="f8c5a617f4d032837fe52cab46c8b6ca1e0d898875569d94c50627ae90d2a0b1" exitCode=255 Jul 12 17:54:03.484956 ip-10-0-129-128 kubenswrapper[2559]: I0712 17:54:03.484952 2559 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-console-operator/console-operator-7866497494-wlk78" event={"ID":"f1d46430-bb60-4470-b8d8-eacd44f4f6b1","Type":"ContainerDied","Data":"f8c5a617f4d032837fe52cab46c8b6ca1e0d898875569d94c50627ae90d2a0b1"} Jul 12 17:54:03.485147 ip-10-0-129-128 kubenswrapper[2559]: I0712 17:54:03.485125 2559 scope.go:117] "RemoveContainer" containerID="f8c5a617f4d032837fe52cab46c8b6ca1e0d898875569d94c50627ae90d2a0b1" Jul 12 17:54:03.487259 ip-10-0-129-128 kubenswrapper[2559]: I0712 17:54:03.487237 2559 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-cluster-storage-operator/volume-data-source-validator-78495597f6-dnhgq" event={"ID":"ab3f4928-9f96-49df-ad17-e6a7fe6f7df3","Type":"ContainerStarted","Data":"36d0ad9427a072ae0ea6254f15c52f9fa3edf19ff5045d929d28ce74538d8eb8"} Jul 12 17:54:03.488796 ip-10-0-129-128 kubenswrapper[2559]: I0712 17:54:03.488774 2559 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-network-diagnostics/network-check-source-869dcfcdcd-whn86" event={"ID":"358e2b76-e0cb-45cb-b0ac-9360910b4e05","Type":"ContainerStarted","Data":"83e32a2916a361b5b52001c042a8027af8e55f6393b5132b17717aaf2612b406"} Jul 12 17:54:03.490214 ip-10-0-129-128 kubenswrapper[2559]: I0712 17:54:03.490195 2559 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="open-cluster-management-agent-addon/managed-serviceaccount-addon-agent-7c88559b8b-qk5pk" event={"ID":"4727caec-6a00-4f71-836f-93b328b67d8e","Type":"ContainerStarted","Data":"b3d12f7bc82ac08abb8693efc48df241a74bbe8a69377c8007caf1a7a0c4a8e2"} Jul 12 17:54:03.491814 ip-10-0-129-128 kubenswrapper[2559]: I0712 17:54:03.491790 2559 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="open-cluster-management-agent-addon/klusterlet-addon-workmgr-74d7bc54f4-k7lv7" event={"ID":"27899582-4f2a-4b54-a4f5-0f6167d16c1f","Type":"ContainerStarted","Data":"d2f14e0d4a53eb292d291b799cb58424f799a2979db02714a456c1f7db5238e9"} Jul 12 17:54:03.491954 ip-10-0-129-128 kubenswrapper[2559]: I0712 17:54:03.491938 2559 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="open-cluster-management-agent-addon/klusterlet-addon-workmgr-74d7bc54f4-k7lv7" Jul 12 17:54:03.493285 ip-10-0-129-128 kubenswrapper[2559]: I0712 17:54:03.493261 2559 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-kube-storage-version-migrator-operator/kube-storage-version-migrator-operator-799cb7f958-rfppz" event={"ID":"e73254f8-d204-4dd4-a86f-512fc5d17ebe","Type":"ContainerStarted","Data":"69ac0c9604c3700711bc3aeed4b7900649f422bb5f1e5f6113469877a25e0e2b"} Jul 12 17:54:03.493842 ip-10-0-129-128 kubenswrapper[2559]: I0712 17:54:03.493826 2559 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="open-cluster-management-agent-addon/klusterlet-addon-workmgr-74d7bc54f4-k7lv7" Jul 12 17:54:03.494871 ip-10-0-129-128 kubenswrapper[2559]: I0712 17:54:03.494849 2559 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-service-ca-operator/service-ca-operator-7695fc6c8c-cnb6s" event={"ID":"356ce79c-8163-4cdc-b93e-fb248a89d42c","Type":"ContainerStarted","Data":"56b7d85f57c318c7eb44aad18d958951e7714a675127206752214da5ca7e03ce"} Jul 12 17:54:03.498867 ip-10-0-129-128 kubenswrapper[2559]: I0712 17:54:03.498820 2559 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-ingress-canary/ingress-canary-fdmcz" podStartSLOduration=2.627386975 podStartE2EDuration="15.498804703s" totalImagesPullingTime="12.871417728s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-12 17:53:48 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-12 17:54:03.497747643 +0000 UTC m=+50.991815573" watchObservedRunningTime="2026-07-12 17:54:03.498804703 +0000 UTC m=+50.992872614" Jul 12 17:54:03.498953 ip-10-0-129-128 kubenswrapper[2559]: I0712 17:54:03.498929 2559 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-multus/multus-additional-cni-plugins-9dh8l" podStartSLOduration=20.302509592 podStartE2EDuration="50.498921332s" totalImagesPullingTime="30.19641174s" totalInitContainerRuntime="0s" isStatefulPod=true podCreationTimestamp="2026-07-12 17:53:13 +0000 UTC" imagePullSessionsCount=5 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-12 17:53:51.472354551 +0000 UTC m=+38.966422482" watchObservedRunningTime="2026-07-12 17:54:03.498921332 +0000 UTC m=+50.992989244" Jul 12 17:54:03.531107 ip-10-0-129-128 kubenswrapper[2559]: I0712 17:54:03.531064 2559 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-kube-storage-version-migrator-operator/kube-storage-version-migrator-operator-799cb7f958-rfppz" podStartSLOduration=32.613299637 podStartE2EDuration="45.531048396s" totalImagesPullingTime="12.917748759s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-12 17:53:18 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-12 17:54:03.530720019 +0000 UTC m=+51.024787950" watchObservedRunningTime="2026-07-12 17:54:03.531048396 +0000 UTC m=+51.025116326" Jul 12 17:54:03.531782 ip-10-0-129-128 kubenswrapper[2559]: I0712 17:54:03.531750 2559 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-cluster-storage-operator/volume-data-source-validator-78495597f6-dnhgq" podStartSLOduration=38.62819772 podStartE2EDuration="49.53173964s" totalImagesPullingTime="10.90354192s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-12 17:53:14 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-12 17:54:03.514554113 +0000 UTC m=+51.008622045" watchObservedRunningTime="2026-07-12 17:54:03.53173964 +0000 UTC m=+51.025807569" Jul 12 17:54:03.545954 ip-10-0-129-128 kubenswrapper[2559]: I0712 17:54:03.545899 2559 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="open-cluster-management-agent-addon/klusterlet-addon-workmgr-74d7bc54f4-k7lv7" podStartSLOduration=18.653878298 podStartE2EDuration="31.545879192s" totalImagesPullingTime="12.892000894s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-12 17:53:32 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-12 17:54:03.545135689 +0000 UTC m=+51.039203621" watchObservedRunningTime="2026-07-12 17:54:03.545879192 +0000 UTC m=+51.039947123" Jul 12 17:54:03.582628 ip-10-0-129-128 kubenswrapper[2559]: I0712 17:54:03.582585 2559 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-service-ca-operator/service-ca-operator-7695fc6c8c-cnb6s" podStartSLOduration=32.614895757 podStartE2EDuration="45.582569492s" totalImagesPullingTime="12.967673735s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-12 17:53:18 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-12 17:54:03.582428959 +0000 UTC m=+51.076496888" watchObservedRunningTime="2026-07-12 17:54:03.582569492 +0000 UTC m=+51.076637421" Jul 12 17:54:03.583131 ip-10-0-129-128 kubenswrapper[2559]: I0712 17:54:03.583099 2559 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="open-cluster-management-agent-addon/managed-serviceaccount-addon-agent-7c88559b8b-qk5pk" podStartSLOduration=18.707468775 podStartE2EDuration="31.583089296s" totalImagesPullingTime="12.875620521s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-12 17:53:32 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-12 17:54:03.565597107 +0000 UTC m=+51.059665036" watchObservedRunningTime="2026-07-12 17:54:03.583089296 +0000 UTC m=+51.077157228" Jul 12 17:54:03.624872 ip-10-0-129-128 kubenswrapper[2559]: I0712 17:54:03.624816 2559 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="open-cluster-management-agent-addon/cluster-proxy-proxy-agent-7fdc8d7ffb-7hrbq" podStartSLOduration=18.751915322 podStartE2EDuration="31.624795047s" totalImagesPullingTime="12.872879725s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-12 17:53:32 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-12 17:54:03.624398199 +0000 UTC m=+51.118466129" watchObservedRunningTime="2026-07-12 17:54:03.624795047 +0000 UTC m=+51.118862978" Jul 12 17:54:03.626420 ip-10-0-129-128 kubenswrapper[2559]: I0712 17:54:03.626363 2559 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-network-diagnostics/network-check-source-869dcfcdcd-whn86" podStartSLOduration=28.75358043 podStartE2EDuration="41.626346176s" totalImagesPullingTime="12.872765746s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-12 17:53:22 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-12 17:54:03.600878866 +0000 UTC m=+51.094946798" watchObservedRunningTime="2026-07-12 17:54:03.626346176 +0000 UTC m=+51.120414111" Jul 12 17:54:04.499822 ip-10-0-129-128 kubenswrapper[2559]: I0712 17:54:04.499786 2559 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-console-operator_console-operator-7866497494-wlk78_f1d46430-bb60-4470-b8d8-eacd44f4f6b1/console-operator/1.log" Jul 12 17:54:04.500276 ip-10-0-129-128 kubenswrapper[2559]: I0712 17:54:04.500220 2559 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-console-operator_console-operator-7866497494-wlk78_f1d46430-bb60-4470-b8d8-eacd44f4f6b1/console-operator/0.log" Jul 12 17:54:04.500276 ip-10-0-129-128 kubenswrapper[2559]: I0712 17:54:04.500262 2559 generic.go:356] "Generic (PLEG): container finished" podID="f1d46430-bb60-4470-b8d8-eacd44f4f6b1" containerID="91af80d35f960f5b789c9ae523998edb77001be5e0c500472f418a28e4e015af" exitCode=255 Jul 12 17:54:04.500384 ip-10-0-129-128 kubenswrapper[2559]: I0712 17:54:04.500356 2559 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-console-operator/console-operator-7866497494-wlk78" event={"ID":"f1d46430-bb60-4470-b8d8-eacd44f4f6b1","Type":"ContainerDied","Data":"91af80d35f960f5b789c9ae523998edb77001be5e0c500472f418a28e4e015af"} Jul 12 17:54:04.500457 ip-10-0-129-128 kubenswrapper[2559]: I0712 17:54:04.500414 2559 scope.go:117] "RemoveContainer" containerID="f8c5a617f4d032837fe52cab46c8b6ca1e0d898875569d94c50627ae90d2a0b1" Jul 12 17:54:04.500679 ip-10-0-129-128 kubenswrapper[2559]: I0712 17:54:04.500629 2559 scope.go:117] "RemoveContainer" containerID="91af80d35f960f5b789c9ae523998edb77001be5e0c500472f418a28e4e015af" Jul 12 17:54:04.500850 ip-10-0-129-128 kubenswrapper[2559]: E0712 17:54:04.500827 2559 pod_workers.go:1324] "Error syncing pod, skipping" err="failed to \"StartContainer\" for \"console-operator\" with CrashLoopBackOff: \"back-off 10s restarting failed container=console-operator pod=console-operator-7866497494-wlk78_openshift-console-operator(f1d46430-bb60-4470-b8d8-eacd44f4f6b1)\"" pod="openshift-console-operator/console-operator-7866497494-wlk78" podUID="f1d46430-bb60-4470-b8d8-eacd44f4f6b1" Jul 12 17:54:04.753067 ip-10-0-129-128 kubenswrapper[2559]: I0712 17:54:04.752977 2559 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-tls\" (UniqueName: \"kubernetes.io/secret/15d24b76-d54a-409b-91e4-6b9e4f9ac7b2-metrics-tls\") pod \"dns-default-wvr9g\" (UID: \"15d24b76-d54a-409b-91e4-6b9e4f9ac7b2\") " pod="openshift-dns/dns-default-wvr9g" Jul 12 17:54:04.753234 ip-10-0-129-128 kubenswrapper[2559]: I0712 17:54:04.753071 2559 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"samples-operator-tls\" (UniqueName: \"kubernetes.io/secret/a91aea93-bc91-4394-b212-c7699e8593e2-samples-operator-tls\") pod \"cluster-samples-operator-65db858894-996qw\" (UID: \"a91aea93-bc91-4394-b212-c7699e8593e2\") " pod="openshift-cluster-samples-operator/cluster-samples-operator-65db858894-996qw" Jul 12 17:54:04.753234 ip-10-0-129-128 kubenswrapper[2559]: E0712 17:54:04.753114 2559 secret.go:189] Couldn't get secret openshift-dns/dns-default-metrics-tls: secret "dns-default-metrics-tls" not found Jul 12 17:54:04.753234 ip-10-0-129-128 kubenswrapper[2559]: I0712 17:54:04.753142 2559 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"registry-tls\" (UniqueName: \"kubernetes.io/projected/6c8669ba-68cf-4ec6-81d4-1c5842a3d34a-registry-tls\") pod \"image-registry-85b88cb476-kzl6q\" (UID: \"6c8669ba-68cf-4ec6-81d4-1c5842a3d34a\") " pod="openshift-image-registry/image-registry-85b88cb476-kzl6q" Jul 12 17:54:04.753234 ip-10-0-129-128 kubenswrapper[2559]: E0712 17:54:04.753181 2559 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/15d24b76-d54a-409b-91e4-6b9e4f9ac7b2-metrics-tls podName:15d24b76-d54a-409b-91e4-6b9e4f9ac7b2 nodeName:}" failed. No retries permitted until 2026-07-12 17:54:20.753164995 +0000 UTC m=+68.247232903 (durationBeforeRetry 16s). Error: MountVolume.SetUp failed for volume "metrics-tls" (UniqueName: "kubernetes.io/secret/15d24b76-d54a-409b-91e4-6b9e4f9ac7b2-metrics-tls") pod "dns-default-wvr9g" (UID: "15d24b76-d54a-409b-91e4-6b9e4f9ac7b2") : secret "dns-default-metrics-tls" not found Jul 12 17:54:04.753234 ip-10-0-129-128 kubenswrapper[2559]: E0712 17:54:04.753217 2559 secret.go:189] Couldn't get secret openshift-cluster-samples-operator/samples-operator-tls: secret "samples-operator-tls" not found Jul 12 17:54:04.753234 ip-10-0-129-128 kubenswrapper[2559]: E0712 17:54:04.753232 2559 projected.go:266] Couldn't get secret openshift-image-registry/image-registry-tls: secret "image-registry-tls" not found Jul 12 17:54:04.753466 ip-10-0-129-128 kubenswrapper[2559]: E0712 17:54:04.753247 2559 projected.go:196] Error preparing data for projected volume registry-tls for pod openshift-image-registry/image-registry-85b88cb476-kzl6q: secret "image-registry-tls" not found Jul 12 17:54:04.753466 ip-10-0-129-128 kubenswrapper[2559]: E0712 17:54:04.753281 2559 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/a91aea93-bc91-4394-b212-c7699e8593e2-samples-operator-tls podName:a91aea93-bc91-4394-b212-c7699e8593e2 nodeName:}" failed. No retries permitted until 2026-07-12 17:54:20.753264004 +0000 UTC m=+68.247331912 (durationBeforeRetry 16s). Error: MountVolume.SetUp failed for volume "samples-operator-tls" (UniqueName: "kubernetes.io/secret/a91aea93-bc91-4394-b212-c7699e8593e2-samples-operator-tls") pod "cluster-samples-operator-65db858894-996qw" (UID: "a91aea93-bc91-4394-b212-c7699e8593e2") : secret "samples-operator-tls" not found Jul 12 17:54:04.753466 ip-10-0-129-128 kubenswrapper[2559]: E0712 17:54:04.753302 2559 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/projected/6c8669ba-68cf-4ec6-81d4-1c5842a3d34a-registry-tls podName:6c8669ba-68cf-4ec6-81d4-1c5842a3d34a nodeName:}" failed. No retries permitted until 2026-07-12 17:54:20.753291352 +0000 UTC m=+68.247359267 (durationBeforeRetry 16s). Error: MountVolume.SetUp failed for volume "registry-tls" (UniqueName: "kubernetes.io/projected/6c8669ba-68cf-4ec6-81d4-1c5842a3d34a-registry-tls") pod "image-registry-85b88cb476-kzl6q" (UID: "6c8669ba-68cf-4ec6-81d4-1c5842a3d34a") : secret "image-registry-tls" not found Jul 12 17:54:05.055717 ip-10-0-129-128 kubenswrapper[2559]: I0712 17:54:05.055682 2559 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"networking-console-plugin-cert\" (UniqueName: \"kubernetes.io/secret/2edbd7ab-1773-4fbf-af70-0b2cfd5cc642-networking-console-plugin-cert\") pod \"networking-console-plugin-6cf856758c-7ghw7\" (UID: \"2edbd7ab-1773-4fbf-af70-0b2cfd5cc642\") " pod="openshift-network-console/networking-console-plugin-6cf856758c-7ghw7" Jul 12 17:54:05.055876 ip-10-0-129-128 kubenswrapper[2559]: E0712 17:54:05.055832 2559 secret.go:189] Couldn't get secret openshift-network-console/networking-console-plugin-cert: secret "networking-console-plugin-cert" not found Jul 12 17:54:05.055927 ip-10-0-129-128 kubenswrapper[2559]: E0712 17:54:05.055916 2559 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/2edbd7ab-1773-4fbf-af70-0b2cfd5cc642-networking-console-plugin-cert podName:2edbd7ab-1773-4fbf-af70-0b2cfd5cc642 nodeName:}" failed. No retries permitted until 2026-07-12 17:54:21.055897541 +0000 UTC m=+68.549965450 (durationBeforeRetry 16s). Error: MountVolume.SetUp failed for volume "networking-console-plugin-cert" (UniqueName: "kubernetes.io/secret/2edbd7ab-1773-4fbf-af70-0b2cfd5cc642-networking-console-plugin-cert") pod "networking-console-plugin-6cf856758c-7ghw7" (UID: "2edbd7ab-1773-4fbf-af70-0b2cfd5cc642") : secret "networking-console-plugin-cert" not found Jul 12 17:54:05.504401 ip-10-0-129-128 kubenswrapper[2559]: I0712 17:54:05.504379 2559 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-console-operator_console-operator-7866497494-wlk78_f1d46430-bb60-4470-b8d8-eacd44f4f6b1/console-operator/1.log" Jul 12 17:54:05.504805 ip-10-0-129-128 kubenswrapper[2559]: I0712 17:54:05.504723 2559 scope.go:117] "RemoveContainer" containerID="91af80d35f960f5b789c9ae523998edb77001be5e0c500472f418a28e4e015af" Jul 12 17:54:05.504898 ip-10-0-129-128 kubenswrapper[2559]: E0712 17:54:05.504879 2559 pod_workers.go:1324] "Error syncing pod, skipping" err="failed to \"StartContainer\" for \"console-operator\" with CrashLoopBackOff: \"back-off 10s restarting failed container=console-operator pod=console-operator-7866497494-wlk78_openshift-console-operator(f1d46430-bb60-4470-b8d8-eacd44f4f6b1)\"" pod="openshift-console-operator/console-operator-7866497494-wlk78" podUID="f1d46430-bb60-4470-b8d8-eacd44f4f6b1" Jul 12 17:54:09.256817 ip-10-0-129-128 kubenswrapper[2559]: I0712 17:54:09.256774 2559 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-console-operator/console-operator-7866497494-wlk78" Jul 12 17:54:09.257263 ip-10-0-129-128 kubenswrapper[2559]: I0712 17:54:09.256861 2559 kubelet.go:2654] "SyncLoop (probe)" probe="liveness" status="unhealthy" pod="openshift-console-operator/console-operator-7866497494-wlk78" Jul 12 17:54:09.257263 ip-10-0-129-128 kubenswrapper[2559]: I0712 17:54:09.257210 2559 scope.go:117] "RemoveContainer" containerID="91af80d35f960f5b789c9ae523998edb77001be5e0c500472f418a28e4e015af" Jul 12 17:54:09.257398 ip-10-0-129-128 kubenswrapper[2559]: E0712 17:54:09.257374 2559 pod_workers.go:1324] "Error syncing pod, skipping" err="failed to \"StartContainer\" for \"console-operator\" with CrashLoopBackOff: \"back-off 10s restarting failed container=console-operator pod=console-operator-7866497494-wlk78_openshift-console-operator(f1d46430-bb60-4470-b8d8-eacd44f4f6b1)\"" pod="openshift-console-operator/console-operator-7866497494-wlk78" podUID="f1d46430-bb60-4470-b8d8-eacd44f4f6b1" Jul 12 17:54:13.272842 ip-10-0-129-128 kubenswrapper[2559]: I0712 17:54:13.272807 2559 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ovn-kubernetes/ovnkube-node-q5dw8" Jul 12 17:54:18.869614 ip-10-0-129-128 kubenswrapper[2559]: I0712 17:54:18.869575 2559 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"original-pull-secret\" (UniqueName: \"kubernetes.io/secret/6ab3daa1-3376-4f53-8106-d6d7417a327e-original-pull-secret\") pod \"global-pull-secret-syncer-mpqvv\" (UID: \"6ab3daa1-3376-4f53-8106-d6d7417a327e\") " pod="kube-system/global-pull-secret-syncer-mpqvv" Jul 12 17:54:18.870166 ip-10-0-129-128 kubenswrapper[2559]: I0712 17:54:18.869670 2559 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/6f65bad8-9ec7-4a51-8557-74b3f79ef76c-metrics-certs\") pod \"network-metrics-daemon-qtkvt\" (UID: \"6f65bad8-9ec7-4a51-8557-74b3f79ef76c\") " pod="openshift-multus/network-metrics-daemon-qtkvt" Jul 12 17:54:18.871928 ip-10-0-129-128 kubenswrapper[2559]: I0712 17:54:18.871901 2559 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-multus\"/\"metrics-daemon-secret\"" Jul 12 17:54:18.872069 ip-10-0-129-128 kubenswrapper[2559]: I0712 17:54:18.872051 2559 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"kube-system\"/\"original-pull-secret\"" Jul 12 17:54:18.880145 ip-10-0-129-128 kubenswrapper[2559]: E0712 17:54:18.880121 2559 secret.go:189] Couldn't get secret openshift-multus/metrics-daemon-secret: secret "metrics-daemon-secret" not found Jul 12 17:54:18.880242 ip-10-0-129-128 kubenswrapper[2559]: E0712 17:54:18.880192 2559 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/6f65bad8-9ec7-4a51-8557-74b3f79ef76c-metrics-certs podName:6f65bad8-9ec7-4a51-8557-74b3f79ef76c nodeName:}" failed. No retries permitted until 2026-07-12 17:55:22.880170102 +0000 UTC m=+130.374238025 (durationBeforeRetry 1m4s). Error: MountVolume.SetUp failed for volume "metrics-certs" (UniqueName: "kubernetes.io/secret/6f65bad8-9ec7-4a51-8557-74b3f79ef76c-metrics-certs") pod "network-metrics-daemon-qtkvt" (UID: "6f65bad8-9ec7-4a51-8557-74b3f79ef76c") : secret "metrics-daemon-secret" not found Jul 12 17:54:18.883878 ip-10-0-129-128 kubenswrapper[2559]: I0712 17:54:18.883859 2559 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"original-pull-secret\" (UniqueName: \"kubernetes.io/secret/6ab3daa1-3376-4f53-8106-d6d7417a327e-original-pull-secret\") pod \"global-pull-secret-syncer-mpqvv\" (UID: \"6ab3daa1-3376-4f53-8106-d6d7417a327e\") " pod="kube-system/global-pull-secret-syncer-mpqvv" Jul 12 17:54:18.970559 ip-10-0-129-128 kubenswrapper[2559]: I0712 17:54:18.970513 2559 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-bn2rx\" (UniqueName: \"kubernetes.io/projected/e2c3a206-19c0-4d4d-bb98-a99190023ad4-kube-api-access-bn2rx\") pod \"network-check-target-h4bgw\" (UID: \"e2c3a206-19c0-4d4d-bb98-a99190023ad4\") " pod="openshift-network-diagnostics/network-check-target-h4bgw" Jul 12 17:54:18.972998 ip-10-0-129-128 kubenswrapper[2559]: I0712 17:54:18.972969 2559 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-bn2rx\" (UniqueName: \"kubernetes.io/projected/e2c3a206-19c0-4d4d-bb98-a99190023ad4-kube-api-access-bn2rx\") pod \"network-check-target-h4bgw\" (UID: \"e2c3a206-19c0-4d4d-bb98-a99190023ad4\") " pod="openshift-network-diagnostics/network-check-target-h4bgw" Jul 12 17:54:19.039631 ip-10-0-129-128 kubenswrapper[2559]: I0712 17:54:19.039599 2559 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-mpqvv" Jul 12 17:54:19.148428 ip-10-0-129-128 kubenswrapper[2559]: I0712 17:54:19.148351 2559 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-network-diagnostics\"/\"default-dockercfg-h9zfm\"" Jul 12 17:54:19.156414 ip-10-0-129-128 kubenswrapper[2559]: I0712 17:54:19.156392 2559 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-h4bgw" Jul 12 17:54:19.160276 ip-10-0-129-128 kubenswrapper[2559]: I0712 17:54:19.160244 2559 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kube-system/global-pull-secret-syncer-mpqvv"] Jul 12 17:54:19.163044 ip-10-0-129-128 kubenswrapper[2559]: W0712 17:54:19.163014 2559 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod6ab3daa1_3376_4f53_8106_d6d7417a327e.slice/crio-3ef5c6250e7f02206e9f8b0d60b018076372558e43b6de4a0c9bbcf749a2f025 WatchSource:0}: Error finding container 3ef5c6250e7f02206e9f8b0d60b018076372558e43b6de4a0c9bbcf749a2f025: Status 404 returned error can't find the container with id 3ef5c6250e7f02206e9f8b0d60b018076372558e43b6de4a0c9bbcf749a2f025 Jul 12 17:54:19.277538 ip-10-0-129-128 kubenswrapper[2559]: I0712 17:54:19.277508 2559 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-network-diagnostics/network-check-target-h4bgw"] Jul 12 17:54:19.280201 ip-10-0-129-128 kubenswrapper[2559]: W0712 17:54:19.280173 2559 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pode2c3a206_19c0_4d4d_bb98_a99190023ad4.slice/crio-32625ee7211464b8f3dd07c3861dd11db57c369cbe4e48b7c5be43f38a960c31 WatchSource:0}: Error finding container 32625ee7211464b8f3dd07c3861dd11db57c369cbe4e48b7c5be43f38a960c31: Status 404 returned error can't find the container with id 32625ee7211464b8f3dd07c3861dd11db57c369cbe4e48b7c5be43f38a960c31 Jul 12 17:54:19.550749 ip-10-0-129-128 kubenswrapper[2559]: I0712 17:54:19.550707 2559 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-network-diagnostics/network-check-target-h4bgw" event={"ID":"e2c3a206-19c0-4d4d-bb98-a99190023ad4","Type":"ContainerStarted","Data":"276cc54d7646342f89ff0dbf011e4f737683cabfd96ea19544e7a3102bd968ab"} Jul 12 17:54:19.550966 ip-10-0-129-128 kubenswrapper[2559]: I0712 17:54:19.550773 2559 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-network-diagnostics/network-check-target-h4bgw" event={"ID":"e2c3a206-19c0-4d4d-bb98-a99190023ad4","Type":"ContainerStarted","Data":"32625ee7211464b8f3dd07c3861dd11db57c369cbe4e48b7c5be43f38a960c31"} Jul 12 17:54:19.550966 ip-10-0-129-128 kubenswrapper[2559]: I0712 17:54:19.550799 2559 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-network-diagnostics/network-check-target-h4bgw" Jul 12 17:54:19.551863 ip-10-0-129-128 kubenswrapper[2559]: I0712 17:54:19.551836 2559 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kube-system/global-pull-secret-syncer-mpqvv" event={"ID":"6ab3daa1-3376-4f53-8106-d6d7417a327e","Type":"ContainerStarted","Data":"3ef5c6250e7f02206e9f8b0d60b018076372558e43b6de4a0c9bbcf749a2f025"} Jul 12 17:54:19.566334 ip-10-0-129-128 kubenswrapper[2559]: I0712 17:54:19.566287 2559 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-network-diagnostics/network-check-target-h4bgw" podStartSLOduration=66.566273083 podStartE2EDuration="1m6.566273083s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-12 17:53:13 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-12 17:54:19.566114443 +0000 UTC m=+67.060182374" watchObservedRunningTime="2026-07-12 17:54:19.566273083 +0000 UTC m=+67.060341042" Jul 12 17:54:20.788195 ip-10-0-129-128 kubenswrapper[2559]: I0712 17:54:20.788159 2559 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"registry-tls\" (UniqueName: \"kubernetes.io/projected/6c8669ba-68cf-4ec6-81d4-1c5842a3d34a-registry-tls\") pod \"image-registry-85b88cb476-kzl6q\" (UID: \"6c8669ba-68cf-4ec6-81d4-1c5842a3d34a\") " pod="openshift-image-registry/image-registry-85b88cb476-kzl6q" Jul 12 17:54:20.788195 ip-10-0-129-128 kubenswrapper[2559]: I0712 17:54:20.788226 2559 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-tls\" (UniqueName: \"kubernetes.io/secret/15d24b76-d54a-409b-91e4-6b9e4f9ac7b2-metrics-tls\") pod \"dns-default-wvr9g\" (UID: \"15d24b76-d54a-409b-91e4-6b9e4f9ac7b2\") " pod="openshift-dns/dns-default-wvr9g" Jul 12 17:54:20.789041 ip-10-0-129-128 kubenswrapper[2559]: I0712 17:54:20.788297 2559 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"samples-operator-tls\" (UniqueName: \"kubernetes.io/secret/a91aea93-bc91-4394-b212-c7699e8593e2-samples-operator-tls\") pod \"cluster-samples-operator-65db858894-996qw\" (UID: \"a91aea93-bc91-4394-b212-c7699e8593e2\") " pod="openshift-cluster-samples-operator/cluster-samples-operator-65db858894-996qw" Jul 12 17:54:20.791147 ip-10-0-129-128 kubenswrapper[2559]: I0712 17:54:20.791123 2559 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"metrics-tls\" (UniqueName: \"kubernetes.io/secret/15d24b76-d54a-409b-91e4-6b9e4f9ac7b2-metrics-tls\") pod \"dns-default-wvr9g\" (UID: \"15d24b76-d54a-409b-91e4-6b9e4f9ac7b2\") " pod="openshift-dns/dns-default-wvr9g" Jul 12 17:54:20.791240 ip-10-0-129-128 kubenswrapper[2559]: I0712 17:54:20.791123 2559 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"registry-tls\" (UniqueName: \"kubernetes.io/projected/6c8669ba-68cf-4ec6-81d4-1c5842a3d34a-registry-tls\") pod \"image-registry-85b88cb476-kzl6q\" (UID: \"6c8669ba-68cf-4ec6-81d4-1c5842a3d34a\") " pod="openshift-image-registry/image-registry-85b88cb476-kzl6q" Jul 12 17:54:20.791289 ip-10-0-129-128 kubenswrapper[2559]: I0712 17:54:20.791252 2559 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"samples-operator-tls\" (UniqueName: \"kubernetes.io/secret/a91aea93-bc91-4394-b212-c7699e8593e2-samples-operator-tls\") pod \"cluster-samples-operator-65db858894-996qw\" (UID: \"a91aea93-bc91-4394-b212-c7699e8593e2\") " pod="openshift-cluster-samples-operator/cluster-samples-operator-65db858894-996qw" Jul 12 17:54:20.837582 ip-10-0-129-128 kubenswrapper[2559]: I0712 17:54:20.837552 2559 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-dns\"/\"dns-dockercfg-xtknx\"" Jul 12 17:54:20.845695 ip-10-0-129-128 kubenswrapper[2559]: I0712 17:54:20.845669 2559 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-dns/dns-default-wvr9g" Jul 12 17:54:20.977869 ip-10-0-129-128 kubenswrapper[2559]: I0712 17:54:20.977843 2559 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-cluster-samples-operator\"/\"cluster-samples-operator-dockercfg-vwpq5\"" Jul 12 17:54:20.986000 ip-10-0-129-128 kubenswrapper[2559]: I0712 17:54:20.985976 2559 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-cluster-samples-operator/cluster-samples-operator-65db858894-996qw" Jul 12 17:54:21.001214 ip-10-0-129-128 kubenswrapper[2559]: I0712 17:54:21.001194 2559 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-dns/dns-default-wvr9g"] Jul 12 17:54:21.003358 ip-10-0-129-128 kubenswrapper[2559]: W0712 17:54:21.003334 2559 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod15d24b76_d54a_409b_91e4_6b9e4f9ac7b2.slice/crio-e886a7b6532d42cbb225375bcb053c394a6137fe8751d7231f3173a7cd9b9a57 WatchSource:0}: Error finding container e886a7b6532d42cbb225375bcb053c394a6137fe8751d7231f3173a7cd9b9a57: Status 404 returned error can't find the container with id e886a7b6532d42cbb225375bcb053c394a6137fe8751d7231f3173a7cd9b9a57 Jul 12 17:54:21.070684 ip-10-0-129-128 kubenswrapper[2559]: I0712 17:54:21.070660 2559 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-image-registry\"/\"registry-dockercfg-qczrr\"" Jul 12 17:54:21.078792 ip-10-0-129-128 kubenswrapper[2559]: I0712 17:54:21.078769 2559 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-image-registry/image-registry-85b88cb476-kzl6q" Jul 12 17:54:21.090806 ip-10-0-129-128 kubenswrapper[2559]: I0712 17:54:21.090345 2559 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"networking-console-plugin-cert\" (UniqueName: \"kubernetes.io/secret/2edbd7ab-1773-4fbf-af70-0b2cfd5cc642-networking-console-plugin-cert\") pod \"networking-console-plugin-6cf856758c-7ghw7\" (UID: \"2edbd7ab-1773-4fbf-af70-0b2cfd5cc642\") " pod="openshift-network-console/networking-console-plugin-6cf856758c-7ghw7" Jul 12 17:54:21.095140 ip-10-0-129-128 kubenswrapper[2559]: I0712 17:54:21.095110 2559 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"networking-console-plugin-cert\" (UniqueName: \"kubernetes.io/secret/2edbd7ab-1773-4fbf-af70-0b2cfd5cc642-networking-console-plugin-cert\") pod \"networking-console-plugin-6cf856758c-7ghw7\" (UID: \"2edbd7ab-1773-4fbf-af70-0b2cfd5cc642\") " pod="openshift-network-console/networking-console-plugin-6cf856758c-7ghw7" Jul 12 17:54:21.138569 ip-10-0-129-128 kubenswrapper[2559]: I0712 17:54:21.138537 2559 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-cluster-samples-operator/cluster-samples-operator-65db858894-996qw"] Jul 12 17:54:21.240416 ip-10-0-129-128 kubenswrapper[2559]: I0712 17:54:21.240389 2559 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-image-registry/image-registry-85b88cb476-kzl6q"] Jul 12 17:54:21.242692 ip-10-0-129-128 kubenswrapper[2559]: W0712 17:54:21.242666 2559 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod6c8669ba_68cf_4ec6_81d4_1c5842a3d34a.slice/crio-ae25c7de1487e2c0d221759e0ecde4d08b4c3a792aa3edcde47112de448cc3eb WatchSource:0}: Error finding container ae25c7de1487e2c0d221759e0ecde4d08b4c3a792aa3edcde47112de448cc3eb: Status 404 returned error can't find the container with id ae25c7de1487e2c0d221759e0ecde4d08b4c3a792aa3edcde47112de448cc3eb Jul 12 17:54:21.291890 ip-10-0-129-128 kubenswrapper[2559]: I0712 17:54:21.291854 2559 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-network-console\"/\"default-dockercfg-7lxn6\"" Jul 12 17:54:21.299949 ip-10-0-129-128 kubenswrapper[2559]: I0712 17:54:21.299892 2559 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-console/networking-console-plugin-6cf856758c-7ghw7" Jul 12 17:54:21.465115 ip-10-0-129-128 kubenswrapper[2559]: I0712 17:54:21.465031 2559 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-network-console/networking-console-plugin-6cf856758c-7ghw7"] Jul 12 17:54:21.496907 ip-10-0-129-128 kubenswrapper[2559]: W0712 17:54:21.496742 2559 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod2edbd7ab_1773_4fbf_af70_0b2cfd5cc642.slice/crio-b7200023a5042e42226dc54de2498546b99bd40f3f31e5bef4bf33818ff1efd9 WatchSource:0}: Error finding container b7200023a5042e42226dc54de2498546b99bd40f3f31e5bef4bf33818ff1efd9: Status 404 returned error can't find the container with id b7200023a5042e42226dc54de2498546b99bd40f3f31e5bef4bf33818ff1efd9 Jul 12 17:54:21.562526 ip-10-0-129-128 kubenswrapper[2559]: I0712 17:54:21.562439 2559 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-cluster-samples-operator/cluster-samples-operator-65db858894-996qw" event={"ID":"a91aea93-bc91-4394-b212-c7699e8593e2","Type":"ContainerStarted","Data":"9872634503b83530b3295891b1c7400050f8ceb263d6e94f64fc6117c66af3c6"} Jul 12 17:54:21.565168 ip-10-0-129-128 kubenswrapper[2559]: I0712 17:54:21.565101 2559 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-network-console/networking-console-plugin-6cf856758c-7ghw7" event={"ID":"2edbd7ab-1773-4fbf-af70-0b2cfd5cc642","Type":"ContainerStarted","Data":"b7200023a5042e42226dc54de2498546b99bd40f3f31e5bef4bf33818ff1efd9"} Jul 12 17:54:21.567146 ip-10-0-129-128 kubenswrapper[2559]: I0712 17:54:21.567115 2559 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-image-registry/image-registry-85b88cb476-kzl6q" event={"ID":"6c8669ba-68cf-4ec6-81d4-1c5842a3d34a","Type":"ContainerStarted","Data":"349c1851f2ac9b82c550189863b6ab7f1614b101d4756b812aeea84d2a63953b"} Jul 12 17:54:21.567253 ip-10-0-129-128 kubenswrapper[2559]: I0712 17:54:21.567160 2559 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-image-registry/image-registry-85b88cb476-kzl6q" event={"ID":"6c8669ba-68cf-4ec6-81d4-1c5842a3d34a","Type":"ContainerStarted","Data":"ae25c7de1487e2c0d221759e0ecde4d08b4c3a792aa3edcde47112de448cc3eb"} Jul 12 17:54:21.567463 ip-10-0-129-128 kubenswrapper[2559]: I0712 17:54:21.567414 2559 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-image-registry/image-registry-85b88cb476-kzl6q" Jul 12 17:54:21.569048 ip-10-0-129-128 kubenswrapper[2559]: I0712 17:54:21.569024 2559 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-dns/dns-default-wvr9g" event={"ID":"15d24b76-d54a-409b-91e4-6b9e4f9ac7b2","Type":"ContainerStarted","Data":"e886a7b6532d42cbb225375bcb053c394a6137fe8751d7231f3173a7cd9b9a57"} Jul 12 17:54:25.028678 ip-10-0-129-128 kubenswrapper[2559]: I0712 17:54:25.028626 2559 scope.go:117] "RemoveContainer" containerID="91af80d35f960f5b789c9ae523998edb77001be5e0c500472f418a28e4e015af" Jul 12 17:54:25.585517 ip-10-0-129-128 kubenswrapper[2559]: I0712 17:54:25.585487 2559 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-console-operator_console-operator-7866497494-wlk78_f1d46430-bb60-4470-b8d8-eacd44f4f6b1/console-operator/1.log" Jul 12 17:54:25.585718 ip-10-0-129-128 kubenswrapper[2559]: I0712 17:54:25.585582 2559 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-console-operator/console-operator-7866497494-wlk78" event={"ID":"f1d46430-bb60-4470-b8d8-eacd44f4f6b1","Type":"ContainerStarted","Data":"14756876508f79fa57880592c961ee1237f3bbcca115cfcc2e748411a02810c8"} Jul 12 17:54:25.585951 ip-10-0-129-128 kubenswrapper[2559]: I0712 17:54:25.585933 2559 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-console-operator/console-operator-7866497494-wlk78" Jul 12 17:54:25.587466 ip-10-0-129-128 kubenswrapper[2559]: I0712 17:54:25.587438 2559 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-cluster-samples-operator/cluster-samples-operator-65db858894-996qw" event={"ID":"a91aea93-bc91-4394-b212-c7699e8593e2","Type":"ContainerStarted","Data":"871ad3c61ae32fa0620a8ad32d0ca742f7dfacd2cb850b201a30d1996068a543"} Jul 12 17:54:25.587568 ip-10-0-129-128 kubenswrapper[2559]: I0712 17:54:25.587489 2559 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-cluster-samples-operator/cluster-samples-operator-65db858894-996qw" event={"ID":"a91aea93-bc91-4394-b212-c7699e8593e2","Type":"ContainerStarted","Data":"5fb3dd3a2399c2193cacfdc946433f1ab884f61c61a5b74e3f7dd696a04a6d93"} Jul 12 17:54:25.588786 ip-10-0-129-128 kubenswrapper[2559]: I0712 17:54:25.588760 2559 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kube-system/global-pull-secret-syncer-mpqvv" event={"ID":"6ab3daa1-3376-4f53-8106-d6d7417a327e","Type":"ContainerStarted","Data":"aba034a8bf0e14dc510f9c9339239ae4d186a13a2f6af8fd1c17911b555fcf39"} Jul 12 17:54:25.590186 ip-10-0-129-128 kubenswrapper[2559]: I0712 17:54:25.590158 2559 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-network-console/networking-console-plugin-6cf856758c-7ghw7" event={"ID":"2edbd7ab-1773-4fbf-af70-0b2cfd5cc642","Type":"ContainerStarted","Data":"259dc9bbb2070ef4bf2109b5cf31a22a1e4a400d6cc350817de24e5bd0c42441"} Jul 12 17:54:25.591732 ip-10-0-129-128 kubenswrapper[2559]: I0712 17:54:25.591713 2559 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-dns/dns-default-wvr9g" event={"ID":"15d24b76-d54a-409b-91e4-6b9e4f9ac7b2","Type":"ContainerStarted","Data":"6a4e1d4a376b12385d9e7771061855da6e8e35c9817cdb29c5268740ccda6671"} Jul 12 17:54:25.591835 ip-10-0-129-128 kubenswrapper[2559]: I0712 17:54:25.591745 2559 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-dns/dns-default-wvr9g" event={"ID":"15d24b76-d54a-409b-91e4-6b9e4f9ac7b2","Type":"ContainerStarted","Data":"3effd90e27763749f73a93f9a227d41b9186378d85f5e88657c1285de75b8940"} Jul 12 17:54:25.591895 ip-10-0-129-128 kubenswrapper[2559]: I0712 17:54:25.591844 2559 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-dns/dns-default-wvr9g" Jul 12 17:54:25.602115 ip-10-0-129-128 kubenswrapper[2559]: I0712 17:54:25.602069 2559 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-console-operator/console-operator-7866497494-wlk78" podStartSLOduration=57.628365374 podStartE2EDuration="1m10.602057831s" totalImagesPullingTime="12.973692457s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-12 17:53:15 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-12 17:54:25.601495162 +0000 UTC m=+73.095563105" watchObservedRunningTime="2026-07-12 17:54:25.602057831 +0000 UTC m=+73.096125760" Jul 12 17:54:25.602212 ip-10-0-129-128 kubenswrapper[2559]: I0712 17:54:25.602174 2559 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-image-registry/image-registry-85b88cb476-kzl6q" podStartSLOduration=63.602164966 podStartE2EDuration="1m3.602164966s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-12 17:53:22 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-12 17:54:21.589693475 +0000 UTC m=+69.083761407" watchObservedRunningTime="2026-07-12 17:54:25.602164966 +0000 UTC m=+73.096232945" Jul 12 17:54:25.615112 ip-10-0-129-128 kubenswrapper[2559]: I0712 17:54:25.615071 2559 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="kube-system/global-pull-secret-syncer-mpqvv" podStartSLOduration=67.063689195 podStartE2EDuration="1m12.615064129s" totalImagesPullingTime="5.551374934s" totalInitContainerRuntime="0s" isStatefulPod=true podCreationTimestamp="2026-07-12 17:53:13 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-12 17:54:25.614829924 +0000 UTC m=+73.108897855" watchObservedRunningTime="2026-07-12 17:54:25.615064129 +0000 UTC m=+73.109132037" Jul 12 17:54:25.630192 ip-10-0-129-128 kubenswrapper[2559]: I0712 17:54:25.630156 2559 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-dns/dns-default-wvr9g" podStartSLOduration=33.91894844 podStartE2EDuration="37.630149858s" totalImagesPullingTime="3.711201418s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-12 17:53:48 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-12 17:54:25.630000024 +0000 UTC m=+73.124067954" watchObservedRunningTime="2026-07-12 17:54:25.630149858 +0000 UTC m=+73.124217788" Jul 12 17:54:25.644670 ip-10-0-129-128 kubenswrapper[2559]: I0712 17:54:25.644609 2559 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-network-console/networking-console-plugin-6cf856758c-7ghw7" podStartSLOduration=33.426680284 podStartE2EDuration="36.644592361s" totalImagesPullingTime="3.217912077s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-12 17:53:49 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-12 17:54:25.644013498 +0000 UTC m=+73.138081422" watchObservedRunningTime="2026-07-12 17:54:25.644592361 +0000 UTC m=+73.138660296" Jul 12 17:54:25.658452 ip-10-0-129-128 kubenswrapper[2559]: I0712 17:54:25.658413 2559 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-cluster-samples-operator/cluster-samples-operator-65db858894-996qw" podStartSLOduration=69.131026381 podStartE2EDuration="1m12.658388895s" totalImagesPullingTime="3.527362514s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-12 17:53:13 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-12 17:54:25.658016618 +0000 UTC m=+73.152084549" watchObservedRunningTime="2026-07-12 17:54:25.658388895 +0000 UTC m=+73.152456828" Jul 12 17:54:26.387764 ip-10-0-129-128 kubenswrapper[2559]: I0712 17:54:26.387734 2559 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-console-operator/console-operator-7866497494-wlk78" Jul 12 17:54:29.667945 ip-10-0-129-128 kubenswrapper[2559]: I0712 17:54:29.667906 2559 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-image-registry/image-registry-85b88cb476-kzl6q"] Jul 12 17:54:29.776485 ip-10-0-129-128 kubenswrapper[2559]: I0712 17:54:29.776452 2559 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-image-registry/image-registry-77677cbfcd-2dbbt"] Jul 12 17:54:29.810489 ip-10-0-129-128 kubenswrapper[2559]: I0712 17:54:29.810458 2559 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-image-registry/image-registry-77677cbfcd-2dbbt"] Jul 12 17:54:29.810693 ip-10-0-129-128 kubenswrapper[2559]: I0712 17:54:29.810586 2559 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-image-registry/image-registry-77677cbfcd-2dbbt" Jul 12 17:54:29.965909 ip-10-0-129-128 kubenswrapper[2559]: I0712 17:54:29.965830 2559 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"installation-pull-secrets\" (UniqueName: \"kubernetes.io/secret/0611924a-325e-4d7b-b47a-493b936fa67f-installation-pull-secrets\") pod \"image-registry-77677cbfcd-2dbbt\" (UID: \"0611924a-325e-4d7b-b47a-493b936fa67f\") " pod="openshift-image-registry/image-registry-77677cbfcd-2dbbt" Jul 12 17:54:29.965909 ip-10-0-129-128 kubenswrapper[2559]: I0712 17:54:29.965866 2559 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"trusted-ca\" (UniqueName: \"kubernetes.io/configmap/0611924a-325e-4d7b-b47a-493b936fa67f-trusted-ca\") pod \"image-registry-77677cbfcd-2dbbt\" (UID: \"0611924a-325e-4d7b-b47a-493b936fa67f\") " pod="openshift-image-registry/image-registry-77677cbfcd-2dbbt" Jul 12 17:54:29.966107 ip-10-0-129-128 kubenswrapper[2559]: I0712 17:54:29.965971 2559 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"image-registry-private-configuration\" (UniqueName: \"kubernetes.io/secret/0611924a-325e-4d7b-b47a-493b936fa67f-image-registry-private-configuration\") pod \"image-registry-77677cbfcd-2dbbt\" (UID: \"0611924a-325e-4d7b-b47a-493b936fa67f\") " pod="openshift-image-registry/image-registry-77677cbfcd-2dbbt" Jul 12 17:54:29.966107 ip-10-0-129-128 kubenswrapper[2559]: I0712 17:54:29.966003 2559 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"ca-trust-extracted\" (UniqueName: \"kubernetes.io/empty-dir/0611924a-325e-4d7b-b47a-493b936fa67f-ca-trust-extracted\") pod \"image-registry-77677cbfcd-2dbbt\" (UID: \"0611924a-325e-4d7b-b47a-493b936fa67f\") " pod="openshift-image-registry/image-registry-77677cbfcd-2dbbt" Jul 12 17:54:29.966107 ip-10-0-129-128 kubenswrapper[2559]: I0712 17:54:29.966026 2559 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"bound-sa-token\" (UniqueName: \"kubernetes.io/projected/0611924a-325e-4d7b-b47a-493b936fa67f-bound-sa-token\") pod \"image-registry-77677cbfcd-2dbbt\" (UID: \"0611924a-325e-4d7b-b47a-493b936fa67f\") " pod="openshift-image-registry/image-registry-77677cbfcd-2dbbt" Jul 12 17:54:29.966107 ip-10-0-129-128 kubenswrapper[2559]: I0712 17:54:29.966097 2559 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"registry-tls\" (UniqueName: \"kubernetes.io/projected/0611924a-325e-4d7b-b47a-493b936fa67f-registry-tls\") pod \"image-registry-77677cbfcd-2dbbt\" (UID: \"0611924a-325e-4d7b-b47a-493b936fa67f\") " pod="openshift-image-registry/image-registry-77677cbfcd-2dbbt" Jul 12 17:54:29.966239 ip-10-0-129-128 kubenswrapper[2559]: I0712 17:54:29.966131 2559 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"registry-certificates\" (UniqueName: \"kubernetes.io/configmap/0611924a-325e-4d7b-b47a-493b936fa67f-registry-certificates\") pod \"image-registry-77677cbfcd-2dbbt\" (UID: \"0611924a-325e-4d7b-b47a-493b936fa67f\") " pod="openshift-image-registry/image-registry-77677cbfcd-2dbbt" Jul 12 17:54:29.966239 ip-10-0-129-128 kubenswrapper[2559]: I0712 17:54:29.966148 2559 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-csvdt\" (UniqueName: \"kubernetes.io/projected/0611924a-325e-4d7b-b47a-493b936fa67f-kube-api-access-csvdt\") pod \"image-registry-77677cbfcd-2dbbt\" (UID: \"0611924a-325e-4d7b-b47a-493b936fa67f\") " pod="openshift-image-registry/image-registry-77677cbfcd-2dbbt" Jul 12 17:54:30.067106 ip-10-0-129-128 kubenswrapper[2559]: I0712 17:54:30.067070 2559 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"bound-sa-token\" (UniqueName: \"kubernetes.io/projected/0611924a-325e-4d7b-b47a-493b936fa67f-bound-sa-token\") pod \"image-registry-77677cbfcd-2dbbt\" (UID: \"0611924a-325e-4d7b-b47a-493b936fa67f\") " pod="openshift-image-registry/image-registry-77677cbfcd-2dbbt" Jul 12 17:54:30.067320 ip-10-0-129-128 kubenswrapper[2559]: I0712 17:54:30.067117 2559 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"registry-tls\" (UniqueName: \"kubernetes.io/projected/0611924a-325e-4d7b-b47a-493b936fa67f-registry-tls\") pod \"image-registry-77677cbfcd-2dbbt\" (UID: \"0611924a-325e-4d7b-b47a-493b936fa67f\") " pod="openshift-image-registry/image-registry-77677cbfcd-2dbbt" Jul 12 17:54:30.067320 ip-10-0-129-128 kubenswrapper[2559]: I0712 17:54:30.067134 2559 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"registry-certificates\" (UniqueName: \"kubernetes.io/configmap/0611924a-325e-4d7b-b47a-493b936fa67f-registry-certificates\") pod \"image-registry-77677cbfcd-2dbbt\" (UID: \"0611924a-325e-4d7b-b47a-493b936fa67f\") " pod="openshift-image-registry/image-registry-77677cbfcd-2dbbt" Jul 12 17:54:30.067320 ip-10-0-129-128 kubenswrapper[2559]: I0712 17:54:30.067149 2559 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-csvdt\" (UniqueName: \"kubernetes.io/projected/0611924a-325e-4d7b-b47a-493b936fa67f-kube-api-access-csvdt\") pod \"image-registry-77677cbfcd-2dbbt\" (UID: \"0611924a-325e-4d7b-b47a-493b936fa67f\") " pod="openshift-image-registry/image-registry-77677cbfcd-2dbbt" Jul 12 17:54:30.067320 ip-10-0-129-128 kubenswrapper[2559]: I0712 17:54:30.067270 2559 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"installation-pull-secrets\" (UniqueName: \"kubernetes.io/secret/0611924a-325e-4d7b-b47a-493b936fa67f-installation-pull-secrets\") pod \"image-registry-77677cbfcd-2dbbt\" (UID: \"0611924a-325e-4d7b-b47a-493b936fa67f\") " pod="openshift-image-registry/image-registry-77677cbfcd-2dbbt" Jul 12 17:54:30.067320 ip-10-0-129-128 kubenswrapper[2559]: I0712 17:54:30.067312 2559 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"trusted-ca\" (UniqueName: \"kubernetes.io/configmap/0611924a-325e-4d7b-b47a-493b936fa67f-trusted-ca\") pod \"image-registry-77677cbfcd-2dbbt\" (UID: \"0611924a-325e-4d7b-b47a-493b936fa67f\") " pod="openshift-image-registry/image-registry-77677cbfcd-2dbbt" Jul 12 17:54:30.067557 ip-10-0-129-128 kubenswrapper[2559]: I0712 17:54:30.067382 2559 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"image-registry-private-configuration\" (UniqueName: \"kubernetes.io/secret/0611924a-325e-4d7b-b47a-493b936fa67f-image-registry-private-configuration\") pod \"image-registry-77677cbfcd-2dbbt\" (UID: \"0611924a-325e-4d7b-b47a-493b936fa67f\") " pod="openshift-image-registry/image-registry-77677cbfcd-2dbbt" Jul 12 17:54:30.067557 ip-10-0-129-128 kubenswrapper[2559]: I0712 17:54:30.067411 2559 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"ca-trust-extracted\" (UniqueName: \"kubernetes.io/empty-dir/0611924a-325e-4d7b-b47a-493b936fa67f-ca-trust-extracted\") pod \"image-registry-77677cbfcd-2dbbt\" (UID: \"0611924a-325e-4d7b-b47a-493b936fa67f\") " pod="openshift-image-registry/image-registry-77677cbfcd-2dbbt" Jul 12 17:54:30.067804 ip-10-0-129-128 kubenswrapper[2559]: I0712 17:54:30.067777 2559 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"ca-trust-extracted\" (UniqueName: \"kubernetes.io/empty-dir/0611924a-325e-4d7b-b47a-493b936fa67f-ca-trust-extracted\") pod \"image-registry-77677cbfcd-2dbbt\" (UID: \"0611924a-325e-4d7b-b47a-493b936fa67f\") " pod="openshift-image-registry/image-registry-77677cbfcd-2dbbt" Jul 12 17:54:30.068060 ip-10-0-129-128 kubenswrapper[2559]: I0712 17:54:30.068030 2559 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"registry-certificates\" (UniqueName: \"kubernetes.io/configmap/0611924a-325e-4d7b-b47a-493b936fa67f-registry-certificates\") pod \"image-registry-77677cbfcd-2dbbt\" (UID: \"0611924a-325e-4d7b-b47a-493b936fa67f\") " pod="openshift-image-registry/image-registry-77677cbfcd-2dbbt" Jul 12 17:54:30.068259 ip-10-0-129-128 kubenswrapper[2559]: I0712 17:54:30.068239 2559 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"trusted-ca\" (UniqueName: \"kubernetes.io/configmap/0611924a-325e-4d7b-b47a-493b936fa67f-trusted-ca\") pod \"image-registry-77677cbfcd-2dbbt\" (UID: \"0611924a-325e-4d7b-b47a-493b936fa67f\") " pod="openshift-image-registry/image-registry-77677cbfcd-2dbbt" Jul 12 17:54:30.069960 ip-10-0-129-128 kubenswrapper[2559]: I0712 17:54:30.069934 2559 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"image-registry-private-configuration\" (UniqueName: \"kubernetes.io/secret/0611924a-325e-4d7b-b47a-493b936fa67f-image-registry-private-configuration\") pod \"image-registry-77677cbfcd-2dbbt\" (UID: \"0611924a-325e-4d7b-b47a-493b936fa67f\") " pod="openshift-image-registry/image-registry-77677cbfcd-2dbbt" Jul 12 17:54:30.070186 ip-10-0-129-128 kubenswrapper[2559]: I0712 17:54:30.070162 2559 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"installation-pull-secrets\" (UniqueName: \"kubernetes.io/secret/0611924a-325e-4d7b-b47a-493b936fa67f-installation-pull-secrets\") pod \"image-registry-77677cbfcd-2dbbt\" (UID: \"0611924a-325e-4d7b-b47a-493b936fa67f\") " pod="openshift-image-registry/image-registry-77677cbfcd-2dbbt" Jul 12 17:54:30.070259 ip-10-0-129-128 kubenswrapper[2559]: I0712 17:54:30.070245 2559 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"registry-tls\" (UniqueName: \"kubernetes.io/projected/0611924a-325e-4d7b-b47a-493b936fa67f-registry-tls\") pod \"image-registry-77677cbfcd-2dbbt\" (UID: \"0611924a-325e-4d7b-b47a-493b936fa67f\") " pod="openshift-image-registry/image-registry-77677cbfcd-2dbbt" Jul 12 17:54:30.074974 ip-10-0-129-128 kubenswrapper[2559]: I0712 17:54:30.074947 2559 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"bound-sa-token\" (UniqueName: \"kubernetes.io/projected/0611924a-325e-4d7b-b47a-493b936fa67f-bound-sa-token\") pod \"image-registry-77677cbfcd-2dbbt\" (UID: \"0611924a-325e-4d7b-b47a-493b936fa67f\") " pod="openshift-image-registry/image-registry-77677cbfcd-2dbbt" Jul 12 17:54:30.075261 ip-10-0-129-128 kubenswrapper[2559]: I0712 17:54:30.075241 2559 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-csvdt\" (UniqueName: \"kubernetes.io/projected/0611924a-325e-4d7b-b47a-493b936fa67f-kube-api-access-csvdt\") pod \"image-registry-77677cbfcd-2dbbt\" (UID: \"0611924a-325e-4d7b-b47a-493b936fa67f\") " pod="openshift-image-registry/image-registry-77677cbfcd-2dbbt" Jul 12 17:54:30.120380 ip-10-0-129-128 kubenswrapper[2559]: I0712 17:54:30.120346 2559 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-image-registry/image-registry-77677cbfcd-2dbbt" Jul 12 17:54:30.283280 ip-10-0-129-128 kubenswrapper[2559]: I0712 17:54:30.283255 2559 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-image-registry/image-registry-77677cbfcd-2dbbt"] Jul 12 17:54:30.285781 ip-10-0-129-128 kubenswrapper[2559]: W0712 17:54:30.285756 2559 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod0611924a_325e_4d7b_b47a_493b936fa67f.slice/crio-d7e90514a55e45e736ea063974e14ff4929624ffd5091ad8781e8c5cae66bdd5 WatchSource:0}: Error finding container d7e90514a55e45e736ea063974e14ff4929624ffd5091ad8781e8c5cae66bdd5: Status 404 returned error can't find the container with id d7e90514a55e45e736ea063974e14ff4929624ffd5091ad8781e8c5cae66bdd5 Jul 12 17:54:30.610336 ip-10-0-129-128 kubenswrapper[2559]: I0712 17:54:30.610293 2559 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-image-registry/image-registry-77677cbfcd-2dbbt" event={"ID":"0611924a-325e-4d7b-b47a-493b936fa67f","Type":"ContainerStarted","Data":"6e06f5058d6080d47e2f958930e1177e8fe5bf2efcc95956e03f2d10b5b4f61b"} Jul 12 17:54:30.610531 ip-10-0-129-128 kubenswrapper[2559]: I0712 17:54:30.610352 2559 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-image-registry/image-registry-77677cbfcd-2dbbt" event={"ID":"0611924a-325e-4d7b-b47a-493b936fa67f","Type":"ContainerStarted","Data":"d7e90514a55e45e736ea063974e14ff4929624ffd5091ad8781e8c5cae66bdd5"} Jul 12 17:54:30.610531 ip-10-0-129-128 kubenswrapper[2559]: I0712 17:54:30.610436 2559 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-image-registry/image-registry-77677cbfcd-2dbbt" Jul 12 17:54:30.627222 ip-10-0-129-128 kubenswrapper[2559]: I0712 17:54:30.627165 2559 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-image-registry/image-registry-77677cbfcd-2dbbt" podStartSLOduration=1.627148263 podStartE2EDuration="1.627148263s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-12 17:54:29 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-12 17:54:30.626360841 +0000 UTC m=+78.120428919" watchObservedRunningTime="2026-07-12 17:54:30.627148263 +0000 UTC m=+78.121216193" Jul 12 17:54:34.623237 ip-10-0-129-128 kubenswrapper[2559]: I0712 17:54:34.623206 2559 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-insights/insights-runtime-extractor-tnhcr"] Jul 12 17:54:34.638322 ip-10-0-129-128 kubenswrapper[2559]: I0712 17:54:34.638295 2559 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-insights/insights-runtime-extractor-tnhcr" Jul 12 17:54:34.640768 ip-10-0-129-128 kubenswrapper[2559]: I0712 17:54:34.640743 2559 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-insights\"/\"kube-rbac-proxy\"" Jul 12 17:54:34.640905 ip-10-0-129-128 kubenswrapper[2559]: I0712 17:54:34.640761 2559 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-insights\"/\"insights-runtime-extractor-tls\"" Jul 12 17:54:34.640905 ip-10-0-129-128 kubenswrapper[2559]: I0712 17:54:34.640885 2559 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-insights/insights-runtime-extractor-tnhcr"] Jul 12 17:54:34.641095 ip-10-0-129-128 kubenswrapper[2559]: I0712 17:54:34.641078 2559 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-insights\"/\"openshift-service-ca.crt\"" Jul 12 17:54:34.641168 ip-10-0-129-128 kubenswrapper[2559]: I0712 17:54:34.641150 2559 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-insights\"/\"kube-root-ca.crt\"" Jul 12 17:54:34.641858 ip-10-0-129-128 kubenswrapper[2559]: I0712 17:54:34.641845 2559 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-insights\"/\"insights-runtime-extractor-sa-dockercfg-q8r2n\"" Jul 12 17:54:34.805711 ip-10-0-129-128 kubenswrapper[2559]: I0712 17:54:34.805669 2559 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"crio-socket\" (UniqueName: \"kubernetes.io/host-path/9868d941-2fea-4f26-9802-46788f17ca08-crio-socket\") pod \"insights-runtime-extractor-tnhcr\" (UID: \"9868d941-2fea-4f26-9802-46788f17ca08\") " pod="openshift-insights/insights-runtime-extractor-tnhcr" Jul 12 17:54:34.805872 ip-10-0-129-128 kubenswrapper[2559]: I0712 17:54:34.805726 2559 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-rbac-proxy-cm\" (UniqueName: \"kubernetes.io/configmap/9868d941-2fea-4f26-9802-46788f17ca08-kube-rbac-proxy-cm\") pod \"insights-runtime-extractor-tnhcr\" (UID: \"9868d941-2fea-4f26-9802-46788f17ca08\") " pod="openshift-insights/insights-runtime-extractor-tnhcr" Jul 12 17:54:34.805872 ip-10-0-129-128 kubenswrapper[2559]: I0712 17:54:34.805814 2559 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-87j5m\" (UniqueName: \"kubernetes.io/projected/9868d941-2fea-4f26-9802-46788f17ca08-kube-api-access-87j5m\") pod \"insights-runtime-extractor-tnhcr\" (UID: \"9868d941-2fea-4f26-9802-46788f17ca08\") " pod="openshift-insights/insights-runtime-extractor-tnhcr" Jul 12 17:54:34.805962 ip-10-0-129-128 kubenswrapper[2559]: I0712 17:54:34.805852 2559 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"insights-runtime-extractor-tls\" (UniqueName: \"kubernetes.io/secret/9868d941-2fea-4f26-9802-46788f17ca08-insights-runtime-extractor-tls\") pod \"insights-runtime-extractor-tnhcr\" (UID: \"9868d941-2fea-4f26-9802-46788f17ca08\") " pod="openshift-insights/insights-runtime-extractor-tnhcr" Jul 12 17:54:34.805962 ip-10-0-129-128 kubenswrapper[2559]: I0712 17:54:34.805901 2559 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"data-volume\" (UniqueName: \"kubernetes.io/empty-dir/9868d941-2fea-4f26-9802-46788f17ca08-data-volume\") pod \"insights-runtime-extractor-tnhcr\" (UID: \"9868d941-2fea-4f26-9802-46788f17ca08\") " pod="openshift-insights/insights-runtime-extractor-tnhcr" Jul 12 17:54:34.906886 ip-10-0-129-128 kubenswrapper[2559]: I0712 17:54:34.906801 2559 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-87j5m\" (UniqueName: \"kubernetes.io/projected/9868d941-2fea-4f26-9802-46788f17ca08-kube-api-access-87j5m\") pod \"insights-runtime-extractor-tnhcr\" (UID: \"9868d941-2fea-4f26-9802-46788f17ca08\") " pod="openshift-insights/insights-runtime-extractor-tnhcr" Jul 12 17:54:34.906886 ip-10-0-129-128 kubenswrapper[2559]: I0712 17:54:34.906846 2559 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"insights-runtime-extractor-tls\" (UniqueName: \"kubernetes.io/secret/9868d941-2fea-4f26-9802-46788f17ca08-insights-runtime-extractor-tls\") pod \"insights-runtime-extractor-tnhcr\" (UID: \"9868d941-2fea-4f26-9802-46788f17ca08\") " pod="openshift-insights/insights-runtime-extractor-tnhcr" Jul 12 17:54:34.906886 ip-10-0-129-128 kubenswrapper[2559]: I0712 17:54:34.906870 2559 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"data-volume\" (UniqueName: \"kubernetes.io/empty-dir/9868d941-2fea-4f26-9802-46788f17ca08-data-volume\") pod \"insights-runtime-extractor-tnhcr\" (UID: \"9868d941-2fea-4f26-9802-46788f17ca08\") " pod="openshift-insights/insights-runtime-extractor-tnhcr" Jul 12 17:54:34.907133 ip-10-0-129-128 kubenswrapper[2559]: I0712 17:54:34.906910 2559 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"crio-socket\" (UniqueName: \"kubernetes.io/host-path/9868d941-2fea-4f26-9802-46788f17ca08-crio-socket\") pod \"insights-runtime-extractor-tnhcr\" (UID: \"9868d941-2fea-4f26-9802-46788f17ca08\") " pod="openshift-insights/insights-runtime-extractor-tnhcr" Jul 12 17:54:34.907133 ip-10-0-129-128 kubenswrapper[2559]: I0712 17:54:34.906940 2559 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-rbac-proxy-cm\" (UniqueName: \"kubernetes.io/configmap/9868d941-2fea-4f26-9802-46788f17ca08-kube-rbac-proxy-cm\") pod \"insights-runtime-extractor-tnhcr\" (UID: \"9868d941-2fea-4f26-9802-46788f17ca08\") " pod="openshift-insights/insights-runtime-extractor-tnhcr" Jul 12 17:54:34.907133 ip-10-0-129-128 kubenswrapper[2559]: E0712 17:54:34.906956 2559 secret.go:189] Couldn't get secret openshift-insights/insights-runtime-extractor-tls: secret "insights-runtime-extractor-tls" not found Jul 12 17:54:34.907133 ip-10-0-129-128 kubenswrapper[2559]: E0712 17:54:34.906996 2559 configmap.go:193] Couldn't get configMap openshift-insights/kube-rbac-proxy: configmap "kube-rbac-proxy" not found Jul 12 17:54:34.907133 ip-10-0-129-128 kubenswrapper[2559]: E0712 17:54:34.907037 2559 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/9868d941-2fea-4f26-9802-46788f17ca08-insights-runtime-extractor-tls podName:9868d941-2fea-4f26-9802-46788f17ca08 nodeName:}" failed. No retries permitted until 2026-07-12 17:54:35.407014529 +0000 UTC m=+82.901082444 (durationBeforeRetry 500ms). Error: MountVolume.SetUp failed for volume "insights-runtime-extractor-tls" (UniqueName: "kubernetes.io/secret/9868d941-2fea-4f26-9802-46788f17ca08-insights-runtime-extractor-tls") pod "insights-runtime-extractor-tnhcr" (UID: "9868d941-2fea-4f26-9802-46788f17ca08") : secret "insights-runtime-extractor-tls" not found Jul 12 17:54:34.907133 ip-10-0-129-128 kubenswrapper[2559]: I0712 17:54:34.907037 2559 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"crio-socket\" (UniqueName: \"kubernetes.io/host-path/9868d941-2fea-4f26-9802-46788f17ca08-crio-socket\") pod \"insights-runtime-extractor-tnhcr\" (UID: \"9868d941-2fea-4f26-9802-46788f17ca08\") " pod="openshift-insights/insights-runtime-extractor-tnhcr" Jul 12 17:54:34.907133 ip-10-0-129-128 kubenswrapper[2559]: E0712 17:54:34.907057 2559 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/configmap/9868d941-2fea-4f26-9802-46788f17ca08-kube-rbac-proxy-cm podName:9868d941-2fea-4f26-9802-46788f17ca08 nodeName:}" failed. No retries permitted until 2026-07-12 17:54:35.407048076 +0000 UTC m=+82.901115984 (durationBeforeRetry 500ms). Error: MountVolume.SetUp failed for volume "kube-rbac-proxy-cm" (UniqueName: "kubernetes.io/configmap/9868d941-2fea-4f26-9802-46788f17ca08-kube-rbac-proxy-cm") pod "insights-runtime-extractor-tnhcr" (UID: "9868d941-2fea-4f26-9802-46788f17ca08") : configmap "kube-rbac-proxy" not found Jul 12 17:54:34.907353 ip-10-0-129-128 kubenswrapper[2559]: I0712 17:54:34.907299 2559 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"data-volume\" (UniqueName: \"kubernetes.io/empty-dir/9868d941-2fea-4f26-9802-46788f17ca08-data-volume\") pod \"insights-runtime-extractor-tnhcr\" (UID: \"9868d941-2fea-4f26-9802-46788f17ca08\") " pod="openshift-insights/insights-runtime-extractor-tnhcr" Jul 12 17:54:34.919995 ip-10-0-129-128 kubenswrapper[2559]: I0712 17:54:34.919946 2559 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-87j5m\" (UniqueName: \"kubernetes.io/projected/9868d941-2fea-4f26-9802-46788f17ca08-kube-api-access-87j5m\") pod \"insights-runtime-extractor-tnhcr\" (UID: \"9868d941-2fea-4f26-9802-46788f17ca08\") " pod="openshift-insights/insights-runtime-extractor-tnhcr" Jul 12 17:54:35.412488 ip-10-0-129-128 kubenswrapper[2559]: I0712 17:54:35.412363 2559 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-rbac-proxy-cm\" (UniqueName: \"kubernetes.io/configmap/9868d941-2fea-4f26-9802-46788f17ca08-kube-rbac-proxy-cm\") pod \"insights-runtime-extractor-tnhcr\" (UID: \"9868d941-2fea-4f26-9802-46788f17ca08\") " pod="openshift-insights/insights-runtime-extractor-tnhcr" Jul 12 17:54:35.412488 ip-10-0-129-128 kubenswrapper[2559]: I0712 17:54:35.412414 2559 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"insights-runtime-extractor-tls\" (UniqueName: \"kubernetes.io/secret/9868d941-2fea-4f26-9802-46788f17ca08-insights-runtime-extractor-tls\") pod \"insights-runtime-extractor-tnhcr\" (UID: \"9868d941-2fea-4f26-9802-46788f17ca08\") " pod="openshift-insights/insights-runtime-extractor-tnhcr" Jul 12 17:54:35.412721 ip-10-0-129-128 kubenswrapper[2559]: E0712 17:54:35.412491 2559 configmap.go:193] Couldn't get configMap openshift-insights/kube-rbac-proxy: configmap "kube-rbac-proxy" not found Jul 12 17:54:35.412721 ip-10-0-129-128 kubenswrapper[2559]: E0712 17:54:35.412508 2559 secret.go:189] Couldn't get secret openshift-insights/insights-runtime-extractor-tls: secret "insights-runtime-extractor-tls" not found Jul 12 17:54:35.412721 ip-10-0-129-128 kubenswrapper[2559]: E0712 17:54:35.412553 2559 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/configmap/9868d941-2fea-4f26-9802-46788f17ca08-kube-rbac-proxy-cm podName:9868d941-2fea-4f26-9802-46788f17ca08 nodeName:}" failed. No retries permitted until 2026-07-12 17:54:36.412539299 +0000 UTC m=+83.906607211 (durationBeforeRetry 1s). Error: MountVolume.SetUp failed for volume "kube-rbac-proxy-cm" (UniqueName: "kubernetes.io/configmap/9868d941-2fea-4f26-9802-46788f17ca08-kube-rbac-proxy-cm") pod "insights-runtime-extractor-tnhcr" (UID: "9868d941-2fea-4f26-9802-46788f17ca08") : configmap "kube-rbac-proxy" not found Jul 12 17:54:35.412721 ip-10-0-129-128 kubenswrapper[2559]: E0712 17:54:35.412579 2559 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/9868d941-2fea-4f26-9802-46788f17ca08-insights-runtime-extractor-tls podName:9868d941-2fea-4f26-9802-46788f17ca08 nodeName:}" failed. No retries permitted until 2026-07-12 17:54:36.412564165 +0000 UTC m=+83.906632079 (durationBeforeRetry 1s). Error: MountVolume.SetUp failed for volume "insights-runtime-extractor-tls" (UniqueName: "kubernetes.io/secret/9868d941-2fea-4f26-9802-46788f17ca08-insights-runtime-extractor-tls") pod "insights-runtime-extractor-tnhcr" (UID: "9868d941-2fea-4f26-9802-46788f17ca08") : secret "insights-runtime-extractor-tls" not found Jul 12 17:54:35.598158 ip-10-0-129-128 kubenswrapper[2559]: I0712 17:54:35.598131 2559 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-dns/dns-default-wvr9g" Jul 12 17:54:35.639003 ip-10-0-129-128 kubenswrapper[2559]: I0712 17:54:35.638971 2559 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-dns_dns-default-wvr9g_15d24b76-d54a-409b-91e4-6b9e4f9ac7b2/dns/0.log" Jul 12 17:54:35.837482 ip-10-0-129-128 kubenswrapper[2559]: I0712 17:54:35.837457 2559 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-dns_dns-default-wvr9g_15d24b76-d54a-409b-91e4-6b9e4f9ac7b2/kube-rbac-proxy/0.log" Jul 12 17:54:36.421964 ip-10-0-129-128 kubenswrapper[2559]: I0712 17:54:36.421923 2559 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-rbac-proxy-cm\" (UniqueName: \"kubernetes.io/configmap/9868d941-2fea-4f26-9802-46788f17ca08-kube-rbac-proxy-cm\") pod \"insights-runtime-extractor-tnhcr\" (UID: \"9868d941-2fea-4f26-9802-46788f17ca08\") " pod="openshift-insights/insights-runtime-extractor-tnhcr" Jul 12 17:54:36.422162 ip-10-0-129-128 kubenswrapper[2559]: I0712 17:54:36.421989 2559 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"insights-runtime-extractor-tls\" (UniqueName: \"kubernetes.io/secret/9868d941-2fea-4f26-9802-46788f17ca08-insights-runtime-extractor-tls\") pod \"insights-runtime-extractor-tnhcr\" (UID: \"9868d941-2fea-4f26-9802-46788f17ca08\") " pod="openshift-insights/insights-runtime-extractor-tnhcr" Jul 12 17:54:36.422162 ip-10-0-129-128 kubenswrapper[2559]: E0712 17:54:36.422062 2559 configmap.go:193] Couldn't get configMap openshift-insights/kube-rbac-proxy: configmap "kube-rbac-proxy" not found Jul 12 17:54:36.422162 ip-10-0-129-128 kubenswrapper[2559]: E0712 17:54:36.422117 2559 secret.go:189] Couldn't get secret openshift-insights/insights-runtime-extractor-tls: secret "insights-runtime-extractor-tls" not found Jul 12 17:54:36.422162 ip-10-0-129-128 kubenswrapper[2559]: E0712 17:54:36.422133 2559 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/configmap/9868d941-2fea-4f26-9802-46788f17ca08-kube-rbac-proxy-cm podName:9868d941-2fea-4f26-9802-46788f17ca08 nodeName:}" failed. No retries permitted until 2026-07-12 17:54:38.422114096 +0000 UTC m=+85.916182021 (durationBeforeRetry 2s). Error: MountVolume.SetUp failed for volume "kube-rbac-proxy-cm" (UniqueName: "kubernetes.io/configmap/9868d941-2fea-4f26-9802-46788f17ca08-kube-rbac-proxy-cm") pod "insights-runtime-extractor-tnhcr" (UID: "9868d941-2fea-4f26-9802-46788f17ca08") : configmap "kube-rbac-proxy" not found Jul 12 17:54:36.422162 ip-10-0-129-128 kubenswrapper[2559]: E0712 17:54:36.422153 2559 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/9868d941-2fea-4f26-9802-46788f17ca08-insights-runtime-extractor-tls podName:9868d941-2fea-4f26-9802-46788f17ca08 nodeName:}" failed. No retries permitted until 2026-07-12 17:54:38.422141334 +0000 UTC m=+85.916209251 (durationBeforeRetry 2s). Error: MountVolume.SetUp failed for volume "insights-runtime-extractor-tls" (UniqueName: "kubernetes.io/secret/9868d941-2fea-4f26-9802-46788f17ca08-insights-runtime-extractor-tls") pod "insights-runtime-extractor-tnhcr" (UID: "9868d941-2fea-4f26-9802-46788f17ca08") : secret "insights-runtime-extractor-tls" not found Jul 12 17:54:36.437656 ip-10-0-129-128 kubenswrapper[2559]: I0712 17:54:36.437612 2559 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-dns_node-resolver-fqnvr_41eafd0c-4a0b-48f6-8e30-8e408412d389/dns-node-resolver/0.log" Jul 12 17:54:36.641806 ip-10-0-129-128 kubenswrapper[2559]: I0712 17:54:36.641775 2559 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-image-registry_image-registry-77677cbfcd-2dbbt_0611924a-325e-4d7b-b47a-493b936fa67f/registry/0.log" Jul 12 17:54:37.038042 ip-10-0-129-128 kubenswrapper[2559]: I0712 17:54:37.038004 2559 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-image-registry_image-registry-85b88cb476-kzl6q_6c8669ba-68cf-4ec6-81d4-1c5842a3d34a/registry/0.log" Jul 12 17:54:37.638467 ip-10-0-129-128 kubenswrapper[2559]: I0712 17:54:37.638431 2559 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-image-registry_node-ca-vxx7w_6d23c7c4-9eb6-483f-baf3-f25e641e2ead/node-ca/0.log" Jul 12 17:54:38.037154 ip-10-0-129-128 kubenswrapper[2559]: I0712 17:54:38.037124 2559 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ingress-canary_ingress-canary-fdmcz_98f3df53-0008-483f-bb5f-0a5718df3920/serve-healthcheck-canary/0.log" Jul 12 17:54:38.440344 ip-10-0-129-128 kubenswrapper[2559]: I0712 17:54:38.440263 2559 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-rbac-proxy-cm\" (UniqueName: \"kubernetes.io/configmap/9868d941-2fea-4f26-9802-46788f17ca08-kube-rbac-proxy-cm\") pod \"insights-runtime-extractor-tnhcr\" (UID: \"9868d941-2fea-4f26-9802-46788f17ca08\") " pod="openshift-insights/insights-runtime-extractor-tnhcr" Jul 12 17:54:38.440344 ip-10-0-129-128 kubenswrapper[2559]: I0712 17:54:38.440315 2559 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"insights-runtime-extractor-tls\" (UniqueName: \"kubernetes.io/secret/9868d941-2fea-4f26-9802-46788f17ca08-insights-runtime-extractor-tls\") pod \"insights-runtime-extractor-tnhcr\" (UID: \"9868d941-2fea-4f26-9802-46788f17ca08\") " pod="openshift-insights/insights-runtime-extractor-tnhcr" Jul 12 17:54:38.440565 ip-10-0-129-128 kubenswrapper[2559]: E0712 17:54:38.440389 2559 configmap.go:193] Couldn't get configMap openshift-insights/kube-rbac-proxy: configmap "kube-rbac-proxy" not found Jul 12 17:54:38.440565 ip-10-0-129-128 kubenswrapper[2559]: E0712 17:54:38.440465 2559 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/configmap/9868d941-2fea-4f26-9802-46788f17ca08-kube-rbac-proxy-cm podName:9868d941-2fea-4f26-9802-46788f17ca08 nodeName:}" failed. No retries permitted until 2026-07-12 17:54:42.440446436 +0000 UTC m=+89.934514343 (durationBeforeRetry 4s). Error: MountVolume.SetUp failed for volume "kube-rbac-proxy-cm" (UniqueName: "kubernetes.io/configmap/9868d941-2fea-4f26-9802-46788f17ca08-kube-rbac-proxy-cm") pod "insights-runtime-extractor-tnhcr" (UID: "9868d941-2fea-4f26-9802-46788f17ca08") : configmap "kube-rbac-proxy" not found Jul 12 17:54:38.440565 ip-10-0-129-128 kubenswrapper[2559]: E0712 17:54:38.440405 2559 secret.go:189] Couldn't get secret openshift-insights/insights-runtime-extractor-tls: secret "insights-runtime-extractor-tls" not found Jul 12 17:54:38.440565 ip-10-0-129-128 kubenswrapper[2559]: E0712 17:54:38.440519 2559 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/9868d941-2fea-4f26-9802-46788f17ca08-insights-runtime-extractor-tls podName:9868d941-2fea-4f26-9802-46788f17ca08 nodeName:}" failed. No retries permitted until 2026-07-12 17:54:42.440507977 +0000 UTC m=+89.934575885 (durationBeforeRetry 4s). Error: MountVolume.SetUp failed for volume "insights-runtime-extractor-tls" (UniqueName: "kubernetes.io/secret/9868d941-2fea-4f26-9802-46788f17ca08-insights-runtime-extractor-tls") pod "insights-runtime-extractor-tnhcr" (UID: "9868d941-2fea-4f26-9802-46788f17ca08") : secret "insights-runtime-extractor-tls" not found Jul 12 17:54:39.674778 ip-10-0-129-128 kubenswrapper[2559]: I0712 17:54:39.674748 2559 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-image-registry/image-registry-85b88cb476-kzl6q" Jul 12 17:54:42.470555 ip-10-0-129-128 kubenswrapper[2559]: I0712 17:54:42.470512 2559 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"insights-runtime-extractor-tls\" (UniqueName: \"kubernetes.io/secret/9868d941-2fea-4f26-9802-46788f17ca08-insights-runtime-extractor-tls\") pod \"insights-runtime-extractor-tnhcr\" (UID: \"9868d941-2fea-4f26-9802-46788f17ca08\") " pod="openshift-insights/insights-runtime-extractor-tnhcr" Jul 12 17:54:42.470989 ip-10-0-129-128 kubenswrapper[2559]: I0712 17:54:42.470628 2559 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-rbac-proxy-cm\" (UniqueName: \"kubernetes.io/configmap/9868d941-2fea-4f26-9802-46788f17ca08-kube-rbac-proxy-cm\") pod \"insights-runtime-extractor-tnhcr\" (UID: \"9868d941-2fea-4f26-9802-46788f17ca08\") " pod="openshift-insights/insights-runtime-extractor-tnhcr" Jul 12 17:54:42.470989 ip-10-0-129-128 kubenswrapper[2559]: E0712 17:54:42.470689 2559 secret.go:189] Couldn't get secret openshift-insights/insights-runtime-extractor-tls: secret "insights-runtime-extractor-tls" not found Jul 12 17:54:42.470989 ip-10-0-129-128 kubenswrapper[2559]: E0712 17:54:42.470742 2559 configmap.go:193] Couldn't get configMap openshift-insights/kube-rbac-proxy: configmap "kube-rbac-proxy" not found Jul 12 17:54:42.470989 ip-10-0-129-128 kubenswrapper[2559]: E0712 17:54:42.470780 2559 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/9868d941-2fea-4f26-9802-46788f17ca08-insights-runtime-extractor-tls podName:9868d941-2fea-4f26-9802-46788f17ca08 nodeName:}" failed. No retries permitted until 2026-07-12 17:54:50.470763526 +0000 UTC m=+97.964831437 (durationBeforeRetry 8s). Error: MountVolume.SetUp failed for volume "insights-runtime-extractor-tls" (UniqueName: "kubernetes.io/secret/9868d941-2fea-4f26-9802-46788f17ca08-insights-runtime-extractor-tls") pod "insights-runtime-extractor-tnhcr" (UID: "9868d941-2fea-4f26-9802-46788f17ca08") : secret "insights-runtime-extractor-tls" not found Jul 12 17:54:42.470989 ip-10-0-129-128 kubenswrapper[2559]: E0712 17:54:42.470796 2559 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/configmap/9868d941-2fea-4f26-9802-46788f17ca08-kube-rbac-proxy-cm podName:9868d941-2fea-4f26-9802-46788f17ca08 nodeName:}" failed. No retries permitted until 2026-07-12 17:54:50.470788762 +0000 UTC m=+97.964856670 (durationBeforeRetry 8s). Error: MountVolume.SetUp failed for volume "kube-rbac-proxy-cm" (UniqueName: "kubernetes.io/configmap/9868d941-2fea-4f26-9802-46788f17ca08-kube-rbac-proxy-cm") pod "insights-runtime-extractor-tnhcr" (UID: "9868d941-2fea-4f26-9802-46788f17ca08") : configmap "kube-rbac-proxy" not found Jul 12 17:54:42.689906 ip-10-0-129-128 kubenswrapper[2559]: I0712 17:54:42.689873 2559 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-monitoring/node-exporter-mt8np"] Jul 12 17:54:42.692825 ip-10-0-129-128 kubenswrapper[2559]: I0712 17:54:42.692801 2559 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-monitoring/node-exporter-mt8np" Jul 12 17:54:42.695132 ip-10-0-129-128 kubenswrapper[2559]: I0712 17:54:42.695105 2559 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"node-exporter-dockercfg-wl5tq\"" Jul 12 17:54:42.695350 ip-10-0-129-128 kubenswrapper[2559]: I0712 17:54:42.695333 2559 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"node-exporter-kube-rbac-proxy-config\"" Jul 12 17:54:42.695421 ip-10-0-129-128 kubenswrapper[2559]: I0712 17:54:42.695349 2559 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-monitoring\"/\"metrics-client-ca\"" Jul 12 17:54:42.696318 ip-10-0-129-128 kubenswrapper[2559]: I0712 17:54:42.696297 2559 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"node-exporter-tls\"" Jul 12 17:54:42.696318 ip-10-0-129-128 kubenswrapper[2559]: I0712 17:54:42.696308 2559 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-monitoring\"/\"node-exporter-accelerators-collector-config\"" Jul 12 17:54:42.696435 ip-10-0-129-128 kubenswrapper[2559]: I0712 17:54:42.696333 2559 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-monitoring\"/\"kube-root-ca.crt\"" Jul 12 17:54:42.696435 ip-10-0-129-128 kubenswrapper[2559]: I0712 17:54:42.696336 2559 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-monitoring\"/\"openshift-service-ca.crt\"" Jul 12 17:54:42.873807 ip-10-0-129-128 kubenswrapper[2559]: I0712 17:54:42.873769 2559 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"node-exporter-wtmp\" (UniqueName: \"kubernetes.io/host-path/e3bd7aa5-e639-434e-9068-979d50d94274-node-exporter-wtmp\") pod \"node-exporter-mt8np\" (UID: \"e3bd7aa5-e639-434e-9068-979d50d94274\") " pod="openshift-monitoring/node-exporter-mt8np" Jul 12 17:54:42.873979 ip-10-0-129-128 kubenswrapper[2559]: I0712 17:54:42.873811 2559 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"node-exporter-textfile\" (UniqueName: \"kubernetes.io/empty-dir/e3bd7aa5-e639-434e-9068-979d50d94274-node-exporter-textfile\") pod \"node-exporter-mt8np\" (UID: \"e3bd7aa5-e639-434e-9068-979d50d94274\") " pod="openshift-monitoring/node-exporter-mt8np" Jul 12 17:54:42.873979 ip-10-0-129-128 kubenswrapper[2559]: I0712 17:54:42.873895 2559 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"metrics-client-ca\" (UniqueName: \"kubernetes.io/configmap/e3bd7aa5-e639-434e-9068-979d50d94274-metrics-client-ca\") pod \"node-exporter-mt8np\" (UID: \"e3bd7aa5-e639-434e-9068-979d50d94274\") " pod="openshift-monitoring/node-exporter-mt8np" Jul 12 17:54:42.873979 ip-10-0-129-128 kubenswrapper[2559]: I0712 17:54:42.873929 2559 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"root\" (UniqueName: \"kubernetes.io/host-path/e3bd7aa5-e639-434e-9068-979d50d94274-root\") pod \"node-exporter-mt8np\" (UID: \"e3bd7aa5-e639-434e-9068-979d50d94274\") " pod="openshift-monitoring/node-exporter-mt8np" Jul 12 17:54:42.873979 ip-10-0-129-128 kubenswrapper[2559]: I0712 17:54:42.873948 2559 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"node-exporter-kube-rbac-proxy-config\" (UniqueName: \"kubernetes.io/secret/e3bd7aa5-e639-434e-9068-979d50d94274-node-exporter-kube-rbac-proxy-config\") pod \"node-exporter-mt8np\" (UID: \"e3bd7aa5-e639-434e-9068-979d50d94274\") " pod="openshift-monitoring/node-exporter-mt8np" Jul 12 17:54:42.874110 ip-10-0-129-128 kubenswrapper[2559]: I0712 17:54:42.873985 2559 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"node-exporter-accelerators-collector-config\" (UniqueName: \"kubernetes.io/configmap/e3bd7aa5-e639-434e-9068-979d50d94274-node-exporter-accelerators-collector-config\") pod \"node-exporter-mt8np\" (UID: \"e3bd7aa5-e639-434e-9068-979d50d94274\") " pod="openshift-monitoring/node-exporter-mt8np" Jul 12 17:54:42.874110 ip-10-0-129-128 kubenswrapper[2559]: I0712 17:54:42.874015 2559 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-9zl88\" (UniqueName: \"kubernetes.io/projected/e3bd7aa5-e639-434e-9068-979d50d94274-kube-api-access-9zl88\") pod \"node-exporter-mt8np\" (UID: \"e3bd7aa5-e639-434e-9068-979d50d94274\") " pod="openshift-monitoring/node-exporter-mt8np" Jul 12 17:54:42.874110 ip-10-0-129-128 kubenswrapper[2559]: I0712 17:54:42.874075 2559 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"node-exporter-tls\" (UniqueName: \"kubernetes.io/secret/e3bd7aa5-e639-434e-9068-979d50d94274-node-exporter-tls\") pod \"node-exporter-mt8np\" (UID: \"e3bd7aa5-e639-434e-9068-979d50d94274\") " pod="openshift-monitoring/node-exporter-mt8np" Jul 12 17:54:42.874110 ip-10-0-129-128 kubenswrapper[2559]: I0712 17:54:42.874106 2559 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"sys\" (UniqueName: \"kubernetes.io/host-path/e3bd7aa5-e639-434e-9068-979d50d94274-sys\") pod \"node-exporter-mt8np\" (UID: \"e3bd7aa5-e639-434e-9068-979d50d94274\") " pod="openshift-monitoring/node-exporter-mt8np" Jul 12 17:54:42.974844 ip-10-0-129-128 kubenswrapper[2559]: I0712 17:54:42.974810 2559 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"node-exporter-accelerators-collector-config\" (UniqueName: \"kubernetes.io/configmap/e3bd7aa5-e639-434e-9068-979d50d94274-node-exporter-accelerators-collector-config\") pod \"node-exporter-mt8np\" (UID: \"e3bd7aa5-e639-434e-9068-979d50d94274\") " pod="openshift-monitoring/node-exporter-mt8np" Jul 12 17:54:42.974990 ip-10-0-129-128 kubenswrapper[2559]: I0712 17:54:42.974855 2559 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-9zl88\" (UniqueName: \"kubernetes.io/projected/e3bd7aa5-e639-434e-9068-979d50d94274-kube-api-access-9zl88\") pod \"node-exporter-mt8np\" (UID: \"e3bd7aa5-e639-434e-9068-979d50d94274\") " pod="openshift-monitoring/node-exporter-mt8np" Jul 12 17:54:42.974990 ip-10-0-129-128 kubenswrapper[2559]: I0712 17:54:42.974895 2559 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"node-exporter-tls\" (UniqueName: \"kubernetes.io/secret/e3bd7aa5-e639-434e-9068-979d50d94274-node-exporter-tls\") pod \"node-exporter-mt8np\" (UID: \"e3bd7aa5-e639-434e-9068-979d50d94274\") " pod="openshift-monitoring/node-exporter-mt8np" Jul 12 17:54:42.974990 ip-10-0-129-128 kubenswrapper[2559]: I0712 17:54:42.974923 2559 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"sys\" (UniqueName: \"kubernetes.io/host-path/e3bd7aa5-e639-434e-9068-979d50d94274-sys\") pod \"node-exporter-mt8np\" (UID: \"e3bd7aa5-e639-434e-9068-979d50d94274\") " pod="openshift-monitoring/node-exporter-mt8np" Jul 12 17:54:42.974990 ip-10-0-129-128 kubenswrapper[2559]: I0712 17:54:42.974942 2559 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"node-exporter-wtmp\" (UniqueName: \"kubernetes.io/host-path/e3bd7aa5-e639-434e-9068-979d50d94274-node-exporter-wtmp\") pod \"node-exporter-mt8np\" (UID: \"e3bd7aa5-e639-434e-9068-979d50d94274\") " pod="openshift-monitoring/node-exporter-mt8np" Jul 12 17:54:42.974990 ip-10-0-129-128 kubenswrapper[2559]: I0712 17:54:42.974962 2559 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"node-exporter-textfile\" (UniqueName: \"kubernetes.io/empty-dir/e3bd7aa5-e639-434e-9068-979d50d94274-node-exporter-textfile\") pod \"node-exporter-mt8np\" (UID: \"e3bd7aa5-e639-434e-9068-979d50d94274\") " pod="openshift-monitoring/node-exporter-mt8np" Jul 12 17:54:42.975245 ip-10-0-129-128 kubenswrapper[2559]: I0712 17:54:42.975002 2559 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-client-ca\" (UniqueName: \"kubernetes.io/configmap/e3bd7aa5-e639-434e-9068-979d50d94274-metrics-client-ca\") pod \"node-exporter-mt8np\" (UID: \"e3bd7aa5-e639-434e-9068-979d50d94274\") " pod="openshift-monitoring/node-exporter-mt8np" Jul 12 17:54:42.975245 ip-10-0-129-128 kubenswrapper[2559]: I0712 17:54:42.975027 2559 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"root\" (UniqueName: \"kubernetes.io/host-path/e3bd7aa5-e639-434e-9068-979d50d94274-root\") pod \"node-exporter-mt8np\" (UID: \"e3bd7aa5-e639-434e-9068-979d50d94274\") " pod="openshift-monitoring/node-exporter-mt8np" Jul 12 17:54:42.975245 ip-10-0-129-128 kubenswrapper[2559]: I0712 17:54:42.975029 2559 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"sys\" (UniqueName: \"kubernetes.io/host-path/e3bd7aa5-e639-434e-9068-979d50d94274-sys\") pod \"node-exporter-mt8np\" (UID: \"e3bd7aa5-e639-434e-9068-979d50d94274\") " pod="openshift-monitoring/node-exporter-mt8np" Jul 12 17:54:42.975245 ip-10-0-129-128 kubenswrapper[2559]: I0712 17:54:42.975046 2559 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"node-exporter-kube-rbac-proxy-config\" (UniqueName: \"kubernetes.io/secret/e3bd7aa5-e639-434e-9068-979d50d94274-node-exporter-kube-rbac-proxy-config\") pod \"node-exporter-mt8np\" (UID: \"e3bd7aa5-e639-434e-9068-979d50d94274\") " pod="openshift-monitoring/node-exporter-mt8np" Jul 12 17:54:42.975245 ip-10-0-129-128 kubenswrapper[2559]: I0712 17:54:42.975119 2559 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"node-exporter-wtmp\" (UniqueName: \"kubernetes.io/host-path/e3bd7aa5-e639-434e-9068-979d50d94274-node-exporter-wtmp\") pod \"node-exporter-mt8np\" (UID: \"e3bd7aa5-e639-434e-9068-979d50d94274\") " pod="openshift-monitoring/node-exporter-mt8np" Jul 12 17:54:42.975245 ip-10-0-129-128 kubenswrapper[2559]: I0712 17:54:42.975237 2559 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"root\" (UniqueName: \"kubernetes.io/host-path/e3bd7aa5-e639-434e-9068-979d50d94274-root\") pod \"node-exporter-mt8np\" (UID: \"e3bd7aa5-e639-434e-9068-979d50d94274\") " pod="openshift-monitoring/node-exporter-mt8np" Jul 12 17:54:42.975502 ip-10-0-129-128 kubenswrapper[2559]: I0712 17:54:42.975452 2559 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"node-exporter-accelerators-collector-config\" (UniqueName: \"kubernetes.io/configmap/e3bd7aa5-e639-434e-9068-979d50d94274-node-exporter-accelerators-collector-config\") pod \"node-exporter-mt8np\" (UID: \"e3bd7aa5-e639-434e-9068-979d50d94274\") " pod="openshift-monitoring/node-exporter-mt8np" Jul 12 17:54:42.975585 ip-10-0-129-128 kubenswrapper[2559]: I0712 17:54:42.975560 2559 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"node-exporter-textfile\" (UniqueName: \"kubernetes.io/empty-dir/e3bd7aa5-e639-434e-9068-979d50d94274-node-exporter-textfile\") pod \"node-exporter-mt8np\" (UID: \"e3bd7aa5-e639-434e-9068-979d50d94274\") " pod="openshift-monitoring/node-exporter-mt8np" Jul 12 17:54:42.975773 ip-10-0-129-128 kubenswrapper[2559]: I0712 17:54:42.975754 2559 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"metrics-client-ca\" (UniqueName: \"kubernetes.io/configmap/e3bd7aa5-e639-434e-9068-979d50d94274-metrics-client-ca\") pod \"node-exporter-mt8np\" (UID: \"e3bd7aa5-e639-434e-9068-979d50d94274\") " pod="openshift-monitoring/node-exporter-mt8np" Jul 12 17:54:42.977241 ip-10-0-129-128 kubenswrapper[2559]: I0712 17:54:42.977220 2559 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"node-exporter-kube-rbac-proxy-config\" (UniqueName: \"kubernetes.io/secret/e3bd7aa5-e639-434e-9068-979d50d94274-node-exporter-kube-rbac-proxy-config\") pod \"node-exporter-mt8np\" (UID: \"e3bd7aa5-e639-434e-9068-979d50d94274\") " pod="openshift-monitoring/node-exporter-mt8np" Jul 12 17:54:42.977325 ip-10-0-129-128 kubenswrapper[2559]: I0712 17:54:42.977285 2559 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"node-exporter-tls\" (UniqueName: \"kubernetes.io/secret/e3bd7aa5-e639-434e-9068-979d50d94274-node-exporter-tls\") pod \"node-exporter-mt8np\" (UID: \"e3bd7aa5-e639-434e-9068-979d50d94274\") " pod="openshift-monitoring/node-exporter-mt8np" Jul 12 17:54:42.981893 ip-10-0-129-128 kubenswrapper[2559]: I0712 17:54:42.981872 2559 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-9zl88\" (UniqueName: \"kubernetes.io/projected/e3bd7aa5-e639-434e-9068-979d50d94274-kube-api-access-9zl88\") pod \"node-exporter-mt8np\" (UID: \"e3bd7aa5-e639-434e-9068-979d50d94274\") " pod="openshift-monitoring/node-exporter-mt8np" Jul 12 17:54:43.003451 ip-10-0-129-128 kubenswrapper[2559]: I0712 17:54:43.003433 2559 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-monitoring/node-exporter-mt8np" Jul 12 17:54:43.013849 ip-10-0-129-128 kubenswrapper[2559]: W0712 17:54:43.013825 2559 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pode3bd7aa5_e639_434e_9068_979d50d94274.slice/crio-5d7f9a576808920035a55d997cdc6cfa491ba01ce467db881179feed1e3717a4 WatchSource:0}: Error finding container 5d7f9a576808920035a55d997cdc6cfa491ba01ce467db881179feed1e3717a4: Status 404 returned error can't find the container with id 5d7f9a576808920035a55d997cdc6cfa491ba01ce467db881179feed1e3717a4 Jul 12 17:54:43.667808 ip-10-0-129-128 kubenswrapper[2559]: I0712 17:54:43.667775 2559 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/node-exporter-mt8np" event={"ID":"e3bd7aa5-e639-434e-9068-979d50d94274","Type":"ContainerStarted","Data":"5d7f9a576808920035a55d997cdc6cfa491ba01ce467db881179feed1e3717a4"} Jul 12 17:54:44.672454 ip-10-0-129-128 kubenswrapper[2559]: I0712 17:54:44.672422 2559 generic.go:356] "Generic (PLEG): container finished" podID="e3bd7aa5-e639-434e-9068-979d50d94274" containerID="3d4e75a3385b7171d5d41561b0979435d4acda6ccd2e5976cd65a27657854311" exitCode=0 Jul 12 17:54:44.672858 ip-10-0-129-128 kubenswrapper[2559]: I0712 17:54:44.672482 2559 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/node-exporter-mt8np" event={"ID":"e3bd7aa5-e639-434e-9068-979d50d94274","Type":"ContainerDied","Data":"3d4e75a3385b7171d5d41561b0979435d4acda6ccd2e5976cd65a27657854311"} Jul 12 17:54:45.680117 ip-10-0-129-128 kubenswrapper[2559]: I0712 17:54:45.680077 2559 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/node-exporter-mt8np" event={"ID":"e3bd7aa5-e639-434e-9068-979d50d94274","Type":"ContainerStarted","Data":"ac12de68fa98c4010e44a7477b901af1e89faaa549206338fe728ce20835def8"} Jul 12 17:54:45.680492 ip-10-0-129-128 kubenswrapper[2559]: I0712 17:54:45.680137 2559 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/node-exporter-mt8np" event={"ID":"e3bd7aa5-e639-434e-9068-979d50d94274","Type":"ContainerStarted","Data":"579f0ae9e7f04fb26fd8f93cdf468ec79728f3088a6cd581318faae238165a7b"} Jul 12 17:54:45.703845 ip-10-0-129-128 kubenswrapper[2559]: I0712 17:54:45.703801 2559 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-monitoring/node-exporter-mt8np" podStartSLOduration=2.72374227 podStartE2EDuration="3.703786931s" totalImagesPullingTime="914.352533ms" totalInitContainerRuntime="65.692128ms" isStatefulPod=true podCreationTimestamp="2026-07-12 17:54:42 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-12 17:54:45.701573405 +0000 UTC m=+93.195641335" watchObservedRunningTime="2026-07-12 17:54:45.703786931 +0000 UTC m=+93.197854861" Jul 12 17:54:50.536296 ip-10-0-129-128 kubenswrapper[2559]: I0712 17:54:50.536259 2559 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-rbac-proxy-cm\" (UniqueName: \"kubernetes.io/configmap/9868d941-2fea-4f26-9802-46788f17ca08-kube-rbac-proxy-cm\") pod \"insights-runtime-extractor-tnhcr\" (UID: \"9868d941-2fea-4f26-9802-46788f17ca08\") " pod="openshift-insights/insights-runtime-extractor-tnhcr" Jul 12 17:54:50.536775 ip-10-0-129-128 kubenswrapper[2559]: I0712 17:54:50.536309 2559 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"insights-runtime-extractor-tls\" (UniqueName: \"kubernetes.io/secret/9868d941-2fea-4f26-9802-46788f17ca08-insights-runtime-extractor-tls\") pod \"insights-runtime-extractor-tnhcr\" (UID: \"9868d941-2fea-4f26-9802-46788f17ca08\") " pod="openshift-insights/insights-runtime-extractor-tnhcr" Jul 12 17:54:50.536819 ip-10-0-129-128 kubenswrapper[2559]: I0712 17:54:50.536786 2559 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-rbac-proxy-cm\" (UniqueName: \"kubernetes.io/configmap/9868d941-2fea-4f26-9802-46788f17ca08-kube-rbac-proxy-cm\") pod \"insights-runtime-extractor-tnhcr\" (UID: \"9868d941-2fea-4f26-9802-46788f17ca08\") " pod="openshift-insights/insights-runtime-extractor-tnhcr" Jul 12 17:54:50.538559 ip-10-0-129-128 kubenswrapper[2559]: I0712 17:54:50.538543 2559 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"insights-runtime-extractor-tls\" (UniqueName: \"kubernetes.io/secret/9868d941-2fea-4f26-9802-46788f17ca08-insights-runtime-extractor-tls\") pod \"insights-runtime-extractor-tnhcr\" (UID: \"9868d941-2fea-4f26-9802-46788f17ca08\") " pod="openshift-insights/insights-runtime-extractor-tnhcr" Jul 12 17:54:50.549688 ip-10-0-129-128 kubenswrapper[2559]: I0712 17:54:50.549667 2559 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-insights/insights-runtime-extractor-tnhcr" Jul 12 17:54:50.558837 ip-10-0-129-128 kubenswrapper[2559]: I0712 17:54:50.558819 2559 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-network-diagnostics/network-check-target-h4bgw" Jul 12 17:54:50.677740 ip-10-0-129-128 kubenswrapper[2559]: I0712 17:54:50.677716 2559 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-insights/insights-runtime-extractor-tnhcr"] Jul 12 17:54:50.679860 ip-10-0-129-128 kubenswrapper[2559]: W0712 17:54:50.679828 2559 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod9868d941_2fea_4f26_9802_46788f17ca08.slice/crio-de427172dd39cb30a4e66eefe26b47439b52b3dfbb9d615fe96e030cb49add9a WatchSource:0}: Error finding container de427172dd39cb30a4e66eefe26b47439b52b3dfbb9d615fe96e030cb49add9a: Status 404 returned error can't find the container with id de427172dd39cb30a4e66eefe26b47439b52b3dfbb9d615fe96e030cb49add9a Jul 12 17:54:50.698341 ip-10-0-129-128 kubenswrapper[2559]: I0712 17:54:50.698312 2559 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-insights/insights-runtime-extractor-tnhcr" event={"ID":"9868d941-2fea-4f26-9802-46788f17ca08","Type":"ContainerStarted","Data":"de427172dd39cb30a4e66eefe26b47439b52b3dfbb9d615fe96e030cb49add9a"} Jul 12 17:54:51.619506 ip-10-0-129-128 kubenswrapper[2559]: I0712 17:54:51.619473 2559 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-image-registry/image-registry-77677cbfcd-2dbbt" Jul 12 17:54:54.689595 ip-10-0-129-128 kubenswrapper[2559]: I0712 17:54:54.689530 2559 kuberuntime_container.go:873] "Killing container with a grace period" pod="openshift-image-registry/image-registry-85b88cb476-kzl6q" podUID="6c8669ba-68cf-4ec6-81d4-1c5842a3d34a" containerName="registry" containerID="cri-o://349c1851f2ac9b82c550189863b6ab7f1614b101d4756b812aeea84d2a63953b" gracePeriod=30 Jul 12 17:54:54.980862 ip-10-0-129-128 kubenswrapper[2559]: I0712 17:54:54.980831 2559 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-image-registry/image-registry-85b88cb476-kzl6q" Jul 12 17:54:55.072654 ip-10-0-129-128 kubenswrapper[2559]: I0712 17:54:55.072619 2559 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"bound-sa-token\" (UniqueName: \"kubernetes.io/projected/6c8669ba-68cf-4ec6-81d4-1c5842a3d34a-bound-sa-token\") pod \"6c8669ba-68cf-4ec6-81d4-1c5842a3d34a\" (UID: \"6c8669ba-68cf-4ec6-81d4-1c5842a3d34a\") " Jul 12 17:54:55.072815 ip-10-0-129-128 kubenswrapper[2559]: I0712 17:54:55.072707 2559 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-4x6bh\" (UniqueName: \"kubernetes.io/projected/6c8669ba-68cf-4ec6-81d4-1c5842a3d34a-kube-api-access-4x6bh\") pod \"6c8669ba-68cf-4ec6-81d4-1c5842a3d34a\" (UID: \"6c8669ba-68cf-4ec6-81d4-1c5842a3d34a\") " Jul 12 17:54:55.072815 ip-10-0-129-128 kubenswrapper[2559]: I0712 17:54:55.072758 2559 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"registry-certificates\" (UniqueName: \"kubernetes.io/configmap/6c8669ba-68cf-4ec6-81d4-1c5842a3d34a-registry-certificates\") pod \"6c8669ba-68cf-4ec6-81d4-1c5842a3d34a\" (UID: \"6c8669ba-68cf-4ec6-81d4-1c5842a3d34a\") " Jul 12 17:54:55.072815 ip-10-0-129-128 kubenswrapper[2559]: I0712 17:54:55.072784 2559 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"image-registry-private-configuration\" (UniqueName: \"kubernetes.io/secret/6c8669ba-68cf-4ec6-81d4-1c5842a3d34a-image-registry-private-configuration\") pod \"6c8669ba-68cf-4ec6-81d4-1c5842a3d34a\" (UID: \"6c8669ba-68cf-4ec6-81d4-1c5842a3d34a\") " Jul 12 17:54:55.073032 ip-10-0-129-128 kubenswrapper[2559]: I0712 17:54:55.072814 2559 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"installation-pull-secrets\" (UniqueName: \"kubernetes.io/secret/6c8669ba-68cf-4ec6-81d4-1c5842a3d34a-installation-pull-secrets\") pod \"6c8669ba-68cf-4ec6-81d4-1c5842a3d34a\" (UID: \"6c8669ba-68cf-4ec6-81d4-1c5842a3d34a\") " Jul 12 17:54:55.073032 ip-10-0-129-128 kubenswrapper[2559]: I0712 17:54:55.072842 2559 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"trusted-ca\" (UniqueName: \"kubernetes.io/configmap/6c8669ba-68cf-4ec6-81d4-1c5842a3d34a-trusted-ca\") pod \"6c8669ba-68cf-4ec6-81d4-1c5842a3d34a\" (UID: \"6c8669ba-68cf-4ec6-81d4-1c5842a3d34a\") " Jul 12 17:54:55.073032 ip-10-0-129-128 kubenswrapper[2559]: I0712 17:54:55.072890 2559 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"registry-tls\" (UniqueName: \"kubernetes.io/projected/6c8669ba-68cf-4ec6-81d4-1c5842a3d34a-registry-tls\") pod \"6c8669ba-68cf-4ec6-81d4-1c5842a3d34a\" (UID: \"6c8669ba-68cf-4ec6-81d4-1c5842a3d34a\") " Jul 12 17:54:55.073032 ip-10-0-129-128 kubenswrapper[2559]: I0712 17:54:55.072916 2559 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"ca-trust-extracted\" (UniqueName: \"kubernetes.io/empty-dir/6c8669ba-68cf-4ec6-81d4-1c5842a3d34a-ca-trust-extracted\") pod \"6c8669ba-68cf-4ec6-81d4-1c5842a3d34a\" (UID: \"6c8669ba-68cf-4ec6-81d4-1c5842a3d34a\") " Jul 12 17:54:55.073257 ip-10-0-129-128 kubenswrapper[2559]: I0712 17:54:55.073227 2559 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/6c8669ba-68cf-4ec6-81d4-1c5842a3d34a-registry-certificates" (OuterVolumeSpecName: "registry-certificates") pod "6c8669ba-68cf-4ec6-81d4-1c5842a3d34a" (UID: "6c8669ba-68cf-4ec6-81d4-1c5842a3d34a"). InnerVolumeSpecName "registry-certificates". PluginName "kubernetes.io/configmap", VolumeGIDValue "" Jul 12 17:54:55.073478 ip-10-0-129-128 kubenswrapper[2559]: I0712 17:54:55.073436 2559 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/6c8669ba-68cf-4ec6-81d4-1c5842a3d34a-trusted-ca" (OuterVolumeSpecName: "trusted-ca") pod "6c8669ba-68cf-4ec6-81d4-1c5842a3d34a" (UID: "6c8669ba-68cf-4ec6-81d4-1c5842a3d34a"). InnerVolumeSpecName "trusted-ca". PluginName "kubernetes.io/configmap", VolumeGIDValue "" Jul 12 17:54:55.075328 ip-10-0-129-128 kubenswrapper[2559]: I0712 17:54:55.075297 2559 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/6c8669ba-68cf-4ec6-81d4-1c5842a3d34a-image-registry-private-configuration" (OuterVolumeSpecName: "image-registry-private-configuration") pod "6c8669ba-68cf-4ec6-81d4-1c5842a3d34a" (UID: "6c8669ba-68cf-4ec6-81d4-1c5842a3d34a"). InnerVolumeSpecName "image-registry-private-configuration". PluginName "kubernetes.io/secret", VolumeGIDValue "" Jul 12 17:54:55.075517 ip-10-0-129-128 kubenswrapper[2559]: I0712 17:54:55.075493 2559 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/6c8669ba-68cf-4ec6-81d4-1c5842a3d34a-registry-tls" (OuterVolumeSpecName: "registry-tls") pod "6c8669ba-68cf-4ec6-81d4-1c5842a3d34a" (UID: "6c8669ba-68cf-4ec6-81d4-1c5842a3d34a"). InnerVolumeSpecName "registry-tls". PluginName "kubernetes.io/projected", VolumeGIDValue "" Jul 12 17:54:55.075606 ip-10-0-129-128 kubenswrapper[2559]: I0712 17:54:55.075540 2559 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/6c8669ba-68cf-4ec6-81d4-1c5842a3d34a-installation-pull-secrets" (OuterVolumeSpecName: "installation-pull-secrets") pod "6c8669ba-68cf-4ec6-81d4-1c5842a3d34a" (UID: "6c8669ba-68cf-4ec6-81d4-1c5842a3d34a"). InnerVolumeSpecName "installation-pull-secrets". PluginName "kubernetes.io/secret", VolumeGIDValue "" Jul 12 17:54:55.075827 ip-10-0-129-128 kubenswrapper[2559]: I0712 17:54:55.075655 2559 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/6c8669ba-68cf-4ec6-81d4-1c5842a3d34a-bound-sa-token" (OuterVolumeSpecName: "bound-sa-token") pod "6c8669ba-68cf-4ec6-81d4-1c5842a3d34a" (UID: "6c8669ba-68cf-4ec6-81d4-1c5842a3d34a"). InnerVolumeSpecName "bound-sa-token". PluginName "kubernetes.io/projected", VolumeGIDValue "" Jul 12 17:54:55.075827 ip-10-0-129-128 kubenswrapper[2559]: I0712 17:54:55.075770 2559 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/6c8669ba-68cf-4ec6-81d4-1c5842a3d34a-kube-api-access-4x6bh" (OuterVolumeSpecName: "kube-api-access-4x6bh") pod "6c8669ba-68cf-4ec6-81d4-1c5842a3d34a" (UID: "6c8669ba-68cf-4ec6-81d4-1c5842a3d34a"). InnerVolumeSpecName "kube-api-access-4x6bh". PluginName "kubernetes.io/projected", VolumeGIDValue "" Jul 12 17:54:55.082070 ip-10-0-129-128 kubenswrapper[2559]: I0712 17:54:55.082047 2559 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/6c8669ba-68cf-4ec6-81d4-1c5842a3d34a-ca-trust-extracted" (OuterVolumeSpecName: "ca-trust-extracted") pod "6c8669ba-68cf-4ec6-81d4-1c5842a3d34a" (UID: "6c8669ba-68cf-4ec6-81d4-1c5842a3d34a"). InnerVolumeSpecName "ca-trust-extracted". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Jul 12 17:54:55.174245 ip-10-0-129-128 kubenswrapper[2559]: I0712 17:54:55.174215 2559 reconciler_common.go:299] "Volume detached for volume \"trusted-ca\" (UniqueName: \"kubernetes.io/configmap/6c8669ba-68cf-4ec6-81d4-1c5842a3d34a-trusted-ca\") on node \"ip-10-0-129-128.ec2.internal\" DevicePath \"\"" Jul 12 17:54:55.174245 ip-10-0-129-128 kubenswrapper[2559]: I0712 17:54:55.174242 2559 reconciler_common.go:299] "Volume detached for volume \"registry-tls\" (UniqueName: \"kubernetes.io/projected/6c8669ba-68cf-4ec6-81d4-1c5842a3d34a-registry-tls\") on node \"ip-10-0-129-128.ec2.internal\" DevicePath \"\"" Jul 12 17:54:55.174432 ip-10-0-129-128 kubenswrapper[2559]: I0712 17:54:55.174255 2559 reconciler_common.go:299] "Volume detached for volume \"ca-trust-extracted\" (UniqueName: \"kubernetes.io/empty-dir/6c8669ba-68cf-4ec6-81d4-1c5842a3d34a-ca-trust-extracted\") on node \"ip-10-0-129-128.ec2.internal\" DevicePath \"\"" Jul 12 17:54:55.174432 ip-10-0-129-128 kubenswrapper[2559]: I0712 17:54:55.174265 2559 reconciler_common.go:299] "Volume detached for volume \"bound-sa-token\" (UniqueName: \"kubernetes.io/projected/6c8669ba-68cf-4ec6-81d4-1c5842a3d34a-bound-sa-token\") on node \"ip-10-0-129-128.ec2.internal\" DevicePath \"\"" Jul 12 17:54:55.174432 ip-10-0-129-128 kubenswrapper[2559]: I0712 17:54:55.174273 2559 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-4x6bh\" (UniqueName: \"kubernetes.io/projected/6c8669ba-68cf-4ec6-81d4-1c5842a3d34a-kube-api-access-4x6bh\") on node \"ip-10-0-129-128.ec2.internal\" DevicePath \"\"" Jul 12 17:54:55.174432 ip-10-0-129-128 kubenswrapper[2559]: I0712 17:54:55.174281 2559 reconciler_common.go:299] "Volume detached for volume \"registry-certificates\" (UniqueName: \"kubernetes.io/configmap/6c8669ba-68cf-4ec6-81d4-1c5842a3d34a-registry-certificates\") on node \"ip-10-0-129-128.ec2.internal\" DevicePath \"\"" Jul 12 17:54:55.174432 ip-10-0-129-128 kubenswrapper[2559]: I0712 17:54:55.174290 2559 reconciler_common.go:299] "Volume detached for volume \"image-registry-private-configuration\" (UniqueName: \"kubernetes.io/secret/6c8669ba-68cf-4ec6-81d4-1c5842a3d34a-image-registry-private-configuration\") on node \"ip-10-0-129-128.ec2.internal\" DevicePath \"\"" Jul 12 17:54:55.174432 ip-10-0-129-128 kubenswrapper[2559]: I0712 17:54:55.174301 2559 reconciler_common.go:299] "Volume detached for volume \"installation-pull-secrets\" (UniqueName: \"kubernetes.io/secret/6c8669ba-68cf-4ec6-81d4-1c5842a3d34a-installation-pull-secrets\") on node \"ip-10-0-129-128.ec2.internal\" DevicePath \"\"" Jul 12 17:54:55.722796 ip-10-0-129-128 kubenswrapper[2559]: I0712 17:54:55.722761 2559 generic.go:356] "Generic (PLEG): container finished" podID="6c8669ba-68cf-4ec6-81d4-1c5842a3d34a" containerID="349c1851f2ac9b82c550189863b6ab7f1614b101d4756b812aeea84d2a63953b" exitCode=0 Jul 12 17:54:55.723257 ip-10-0-129-128 kubenswrapper[2559]: I0712 17:54:55.722810 2559 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-image-registry/image-registry-85b88cb476-kzl6q" Jul 12 17:54:55.723257 ip-10-0-129-128 kubenswrapper[2559]: I0712 17:54:55.722846 2559 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-image-registry/image-registry-85b88cb476-kzl6q" event={"ID":"6c8669ba-68cf-4ec6-81d4-1c5842a3d34a","Type":"ContainerDied","Data":"349c1851f2ac9b82c550189863b6ab7f1614b101d4756b812aeea84d2a63953b"} Jul 12 17:54:55.723257 ip-10-0-129-128 kubenswrapper[2559]: I0712 17:54:55.722912 2559 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-image-registry/image-registry-85b88cb476-kzl6q" event={"ID":"6c8669ba-68cf-4ec6-81d4-1c5842a3d34a","Type":"ContainerDied","Data":"ae25c7de1487e2c0d221759e0ecde4d08b4c3a792aa3edcde47112de448cc3eb"} Jul 12 17:54:55.723257 ip-10-0-129-128 kubenswrapper[2559]: I0712 17:54:55.722945 2559 scope.go:117] "RemoveContainer" containerID="349c1851f2ac9b82c550189863b6ab7f1614b101d4756b812aeea84d2a63953b" Jul 12 17:54:55.724695 ip-10-0-129-128 kubenswrapper[2559]: I0712 17:54:55.724672 2559 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-insights/insights-runtime-extractor-tnhcr" event={"ID":"9868d941-2fea-4f26-9802-46788f17ca08","Type":"ContainerStarted","Data":"7039ff2adb7f10aef902b5f5d3e1979c03c811feca4b9f7b0a7c332904ac2e0f"} Jul 12 17:54:55.744685 ip-10-0-129-128 kubenswrapper[2559]: I0712 17:54:55.744662 2559 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-image-registry/image-registry-85b88cb476-kzl6q"] Jul 12 17:54:55.748072 ip-10-0-129-128 kubenswrapper[2559]: I0712 17:54:55.748049 2559 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["openshift-image-registry/image-registry-85b88cb476-kzl6q"] Jul 12 17:54:55.875952 ip-10-0-129-128 kubenswrapper[2559]: I0712 17:54:55.875925 2559 scope.go:117] "RemoveContainer" containerID="349c1851f2ac9b82c550189863b6ab7f1614b101d4756b812aeea84d2a63953b" Jul 12 17:54:55.876328 ip-10-0-129-128 kubenswrapper[2559]: E0712 17:54:55.876302 2559 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"349c1851f2ac9b82c550189863b6ab7f1614b101d4756b812aeea84d2a63953b\": container with ID starting with 349c1851f2ac9b82c550189863b6ab7f1614b101d4756b812aeea84d2a63953b not found: ID does not exist" containerID="349c1851f2ac9b82c550189863b6ab7f1614b101d4756b812aeea84d2a63953b" Jul 12 17:54:55.876409 ip-10-0-129-128 kubenswrapper[2559]: I0712 17:54:55.876345 2559 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"349c1851f2ac9b82c550189863b6ab7f1614b101d4756b812aeea84d2a63953b"} err="failed to get container status \"349c1851f2ac9b82c550189863b6ab7f1614b101d4756b812aeea84d2a63953b\": rpc error: code = NotFound desc = could not find container \"349c1851f2ac9b82c550189863b6ab7f1614b101d4756b812aeea84d2a63953b\": container with ID starting with 349c1851f2ac9b82c550189863b6ab7f1614b101d4756b812aeea84d2a63953b not found: ID does not exist" Jul 12 17:54:56.731965 ip-10-0-129-128 kubenswrapper[2559]: I0712 17:54:56.731913 2559 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-insights/insights-runtime-extractor-tnhcr" event={"ID":"9868d941-2fea-4f26-9802-46788f17ca08","Type":"ContainerStarted","Data":"178ae1eaaaa460016b4e817ba1c6ff7d343058ad082c4c34f737dd09c859e9bd"} Jul 12 17:54:57.034728 ip-10-0-129-128 kubenswrapper[2559]: I0712 17:54:57.034692 2559 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="6c8669ba-68cf-4ec6-81d4-1c5842a3d34a" path="/var/lib/kubelet/pods/6c8669ba-68cf-4ec6-81d4-1c5842a3d34a/volumes" Jul 12 17:54:58.741291 ip-10-0-129-128 kubenswrapper[2559]: I0712 17:54:58.741253 2559 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-insights/insights-runtime-extractor-tnhcr" event={"ID":"9868d941-2fea-4f26-9802-46788f17ca08","Type":"ContainerStarted","Data":"c308dd82b03213b0209d2505c1fea4f49b06502ea828c0fcdcbf57976eb70f30"} Jul 12 17:54:58.761086 ip-10-0-129-128 kubenswrapper[2559]: I0712 17:54:58.761037 2559 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-insights/insights-runtime-extractor-tnhcr" podStartSLOduration=17.11059879 podStartE2EDuration="24.761021561s" totalImagesPullingTime="7.650422771s" totalInitContainerRuntime="0s" isStatefulPod=true podCreationTimestamp="2026-07-12 17:54:34 +0000 UTC" imagePullSessionsCount=3 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-12 17:54:58.759854576 +0000 UTC m=+106.253922506" watchObservedRunningTime="2026-07-12 17:54:58.761021561 +0000 UTC m=+106.255089490" Jul 12 17:55:09.782432 ip-10-0-129-128 kubenswrapper[2559]: I0712 17:55:09.782395 2559 generic.go:356] "Generic (PLEG): container finished" podID="356ce79c-8163-4cdc-b93e-fb248a89d42c" containerID="56b7d85f57c318c7eb44aad18d958951e7714a675127206752214da5ca7e03ce" exitCode=0 Jul 12 17:55:09.782900 ip-10-0-129-128 kubenswrapper[2559]: I0712 17:55:09.782455 2559 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-service-ca-operator/service-ca-operator-7695fc6c8c-cnb6s" event={"ID":"356ce79c-8163-4cdc-b93e-fb248a89d42c","Type":"ContainerDied","Data":"56b7d85f57c318c7eb44aad18d958951e7714a675127206752214da5ca7e03ce"} Jul 12 17:55:09.782900 ip-10-0-129-128 kubenswrapper[2559]: I0712 17:55:09.782744 2559 scope.go:117] "RemoveContainer" containerID="56b7d85f57c318c7eb44aad18d958951e7714a675127206752214da5ca7e03ce" Jul 12 17:55:10.787757 ip-10-0-129-128 kubenswrapper[2559]: I0712 17:55:10.787721 2559 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-service-ca-operator/service-ca-operator-7695fc6c8c-cnb6s" event={"ID":"356ce79c-8163-4cdc-b93e-fb248a89d42c","Type":"ContainerStarted","Data":"c43d715240af544925d0392e5fbed7534994b4f3a2d7e24547e2ba650ed6cf46"} Jul 12 17:55:18.817763 ip-10-0-129-128 kubenswrapper[2559]: I0712 17:55:18.817727 2559 generic.go:356] "Generic (PLEG): container finished" podID="e73254f8-d204-4dd4-a86f-512fc5d17ebe" containerID="69ac0c9604c3700711bc3aeed4b7900649f422bb5f1e5f6113469877a25e0e2b" exitCode=0 Jul 12 17:55:18.818150 ip-10-0-129-128 kubenswrapper[2559]: I0712 17:55:18.817762 2559 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-kube-storage-version-migrator-operator/kube-storage-version-migrator-operator-799cb7f958-rfppz" event={"ID":"e73254f8-d204-4dd4-a86f-512fc5d17ebe","Type":"ContainerDied","Data":"69ac0c9604c3700711bc3aeed4b7900649f422bb5f1e5f6113469877a25e0e2b"} Jul 12 17:55:18.818150 ip-10-0-129-128 kubenswrapper[2559]: I0712 17:55:18.818104 2559 scope.go:117] "RemoveContainer" containerID="69ac0c9604c3700711bc3aeed4b7900649f422bb5f1e5f6113469877a25e0e2b" Jul 12 17:55:19.303326 ip-10-0-129-128 kubenswrapper[2559]: I0712 17:55:19.303282 2559 prober.go:120] "Probe failed" probeType="Liveness" pod="open-cluster-management-agent-addon/cluster-proxy-proxy-agent-7fdc8d7ffb-7hrbq" podUID="7a1ef2ea-470b-4dfa-b70c-30bec8f2d908" containerName="service-proxy" probeResult="failure" output="HTTP probe failed with statuscode: 500" Jul 12 17:55:19.827922 ip-10-0-129-128 kubenswrapper[2559]: I0712 17:55:19.827884 2559 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-kube-storage-version-migrator-operator/kube-storage-version-migrator-operator-799cb7f958-rfppz" event={"ID":"e73254f8-d204-4dd4-a86f-512fc5d17ebe","Type":"ContainerStarted","Data":"f8f545947af5039e367e154d22cdf347038087fb7186d0cc7dcb3b79915183f7"} Jul 12 17:55:22.898689 ip-10-0-129-128 kubenswrapper[2559]: I0712 17:55:22.898627 2559 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/6f65bad8-9ec7-4a51-8557-74b3f79ef76c-metrics-certs\") pod \"network-metrics-daemon-qtkvt\" (UID: \"6f65bad8-9ec7-4a51-8557-74b3f79ef76c\") " pod="openshift-multus/network-metrics-daemon-qtkvt" Jul 12 17:55:22.900905 ip-10-0-129-128 kubenswrapper[2559]: I0712 17:55:22.900875 2559 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/6f65bad8-9ec7-4a51-8557-74b3f79ef76c-metrics-certs\") pod \"network-metrics-daemon-qtkvt\" (UID: \"6f65bad8-9ec7-4a51-8557-74b3f79ef76c\") " pod="openshift-multus/network-metrics-daemon-qtkvt" Jul 12 17:55:23.059850 ip-10-0-129-128 kubenswrapper[2559]: I0712 17:55:23.059822 2559 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-multus\"/\"metrics-daemon-sa-dockercfg-84rf9\"" Jul 12 17:55:23.068514 ip-10-0-129-128 kubenswrapper[2559]: I0712 17:55:23.068488 2559 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-qtkvt" Jul 12 17:55:23.192888 ip-10-0-129-128 kubenswrapper[2559]: I0712 17:55:23.192866 2559 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-multus/network-metrics-daemon-qtkvt"] Jul 12 17:55:23.195533 ip-10-0-129-128 kubenswrapper[2559]: W0712 17:55:23.195503 2559 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod6f65bad8_9ec7_4a51_8557_74b3f79ef76c.slice/crio-43a9d493d8cb103a0b5e8370bc80eb9412c04ce42f2b04744a924e900c1a2d75 WatchSource:0}: Error finding container 43a9d493d8cb103a0b5e8370bc80eb9412c04ce42f2b04744a924e900c1a2d75: Status 404 returned error can't find the container with id 43a9d493d8cb103a0b5e8370bc80eb9412c04ce42f2b04744a924e900c1a2d75 Jul 12 17:55:23.842006 ip-10-0-129-128 kubenswrapper[2559]: I0712 17:55:23.841967 2559 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-multus/network-metrics-daemon-qtkvt" event={"ID":"6f65bad8-9ec7-4a51-8557-74b3f79ef76c","Type":"ContainerStarted","Data":"43a9d493d8cb103a0b5e8370bc80eb9412c04ce42f2b04744a924e900c1a2d75"} Jul 12 17:55:24.847151 ip-10-0-129-128 kubenswrapper[2559]: I0712 17:55:24.847112 2559 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-multus/network-metrics-daemon-qtkvt" event={"ID":"6f65bad8-9ec7-4a51-8557-74b3f79ef76c","Type":"ContainerStarted","Data":"a3e5895ab03c97fdde3fff7d583e0bd7854bd07ea045a0638cb7ad3b05259b27"} Jul 12 17:55:24.847623 ip-10-0-129-128 kubenswrapper[2559]: I0712 17:55:24.847219 2559 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-multus/network-metrics-daemon-qtkvt" event={"ID":"6f65bad8-9ec7-4a51-8557-74b3f79ef76c","Type":"ContainerStarted","Data":"d1537992d0bcc09469354df9d317c32a3421d168bcd109407a955528ff14f0f1"} Jul 12 17:55:24.863163 ip-10-0-129-128 kubenswrapper[2559]: I0712 17:55:24.863112 2559 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-multus/network-metrics-daemon-qtkvt" podStartSLOduration=130.635557951 podStartE2EDuration="2m11.863095604s" totalImagesPullingTime="1.227537653s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-12 17:53:13 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-12 17:55:24.862091082 +0000 UTC m=+132.356159012" watchObservedRunningTime="2026-07-12 17:55:24.863095604 +0000 UTC m=+132.357163533" Jul 12 17:55:29.302882 ip-10-0-129-128 kubenswrapper[2559]: I0712 17:55:29.302847 2559 prober.go:120] "Probe failed" probeType="Liveness" pod="open-cluster-management-agent-addon/cluster-proxy-proxy-agent-7fdc8d7ffb-7hrbq" podUID="7a1ef2ea-470b-4dfa-b70c-30bec8f2d908" containerName="service-proxy" probeResult="failure" output="HTTP probe failed with statuscode: 500" Jul 12 17:55:39.303124 ip-10-0-129-128 kubenswrapper[2559]: I0712 17:55:39.303081 2559 prober.go:120] "Probe failed" probeType="Liveness" pod="open-cluster-management-agent-addon/cluster-proxy-proxy-agent-7fdc8d7ffb-7hrbq" podUID="7a1ef2ea-470b-4dfa-b70c-30bec8f2d908" containerName="service-proxy" probeResult="failure" output="HTTP probe failed with statuscode: 500" Jul 12 17:55:39.303577 ip-10-0-129-128 kubenswrapper[2559]: I0712 17:55:39.303175 2559 kubelet.go:2654] "SyncLoop (probe)" probe="liveness" status="unhealthy" pod="open-cluster-management-agent-addon/cluster-proxy-proxy-agent-7fdc8d7ffb-7hrbq" Jul 12 17:55:39.303773 ip-10-0-129-128 kubenswrapper[2559]: I0712 17:55:39.303752 2559 kuberuntime_manager.go:1192] "Message for Container of pod" containerName="service-proxy" containerStatusID={"Type":"cri-o","ID":"b018de6ca7b48ea5fd4a85113e0bed76f987d11422249dfc5a4385dd8c5c32bc"} pod="open-cluster-management-agent-addon/cluster-proxy-proxy-agent-7fdc8d7ffb-7hrbq" containerMessage="Container service-proxy failed liveness probe, will be restarted" Jul 12 17:55:39.303815 ip-10-0-129-128 kubenswrapper[2559]: I0712 17:55:39.303796 2559 kuberuntime_container.go:873] "Killing container with a grace period" pod="open-cluster-management-agent-addon/cluster-proxy-proxy-agent-7fdc8d7ffb-7hrbq" podUID="7a1ef2ea-470b-4dfa-b70c-30bec8f2d908" containerName="service-proxy" containerID="cri-o://b018de6ca7b48ea5fd4a85113e0bed76f987d11422249dfc5a4385dd8c5c32bc" gracePeriod=30 Jul 12 17:55:39.903471 ip-10-0-129-128 kubenswrapper[2559]: I0712 17:55:39.903432 2559 generic.go:356] "Generic (PLEG): container finished" podID="7a1ef2ea-470b-4dfa-b70c-30bec8f2d908" containerID="b018de6ca7b48ea5fd4a85113e0bed76f987d11422249dfc5a4385dd8c5c32bc" exitCode=2 Jul 12 17:55:39.903693 ip-10-0-129-128 kubenswrapper[2559]: I0712 17:55:39.903505 2559 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="open-cluster-management-agent-addon/cluster-proxy-proxy-agent-7fdc8d7ffb-7hrbq" event={"ID":"7a1ef2ea-470b-4dfa-b70c-30bec8f2d908","Type":"ContainerDied","Data":"b018de6ca7b48ea5fd4a85113e0bed76f987d11422249dfc5a4385dd8c5c32bc"} Jul 12 17:55:39.903693 ip-10-0-129-128 kubenswrapper[2559]: I0712 17:55:39.903550 2559 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="open-cluster-management-agent-addon/cluster-proxy-proxy-agent-7fdc8d7ffb-7hrbq" event={"ID":"7a1ef2ea-470b-4dfa-b70c-30bec8f2d908","Type":"ContainerStarted","Data":"99f86880e8d8524534259e6afbb795b11caa43d7674873a266cdbc5c5faf3dc8"} Jul 12 17:58:12.913778 ip-10-0-129-128 kubenswrapper[2559]: I0712 17:58:12.913752 2559 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-console-operator_console-operator-7866497494-wlk78_f1d46430-bb60-4470-b8d8-eacd44f4f6b1/console-operator/1.log" Jul 12 17:58:12.916182 ip-10-0-129-128 kubenswrapper[2559]: I0712 17:58:12.916154 2559 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ovn-kubernetes_ovnkube-node-q5dw8_e51bad50-bc74-4e38-8307-ca257b599314/ovn-acl-logging/0.log" Jul 12 17:58:12.917040 ip-10-0-129-128 kubenswrapper[2559]: I0712 17:58:12.917019 2559 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-console-operator_console-operator-7866497494-wlk78_f1d46430-bb60-4470-b8d8-eacd44f4f6b1/console-operator/1.log" Jul 12 17:58:12.920144 ip-10-0-129-128 kubenswrapper[2559]: I0712 17:58:12.920122 2559 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ovn-kubernetes_ovnkube-node-q5dw8_e51bad50-bc74-4e38-8307-ca257b599314/ovn-acl-logging/0.log" Jul 12 17:58:12.923991 ip-10-0-129-128 kubenswrapper[2559]: I0712 17:58:12.923970 2559 kubelet.go:1648] "Image garbage collection succeeded" Jul 12 17:59:12.353882 ip-10-0-129-128 kubenswrapper[2559]: I0712 17:59:12.353800 2559 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["opendatahub/opendatahub-operator-controller-manager-5dbd4cc77c-dphpk"] Jul 12 17:59:12.354285 ip-10-0-129-128 kubenswrapper[2559]: I0712 17:59:12.354072 2559 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="6c8669ba-68cf-4ec6-81d4-1c5842a3d34a" containerName="registry" Jul 12 17:59:12.354285 ip-10-0-129-128 kubenswrapper[2559]: I0712 17:59:12.354083 2559 state_mem.go:107] "Deleted CPUSet assignment" podUID="6c8669ba-68cf-4ec6-81d4-1c5842a3d34a" containerName="registry" Jul 12 17:59:12.354285 ip-10-0-129-128 kubenswrapper[2559]: I0712 17:59:12.354136 2559 memory_manager.go:368] "RemoveStaleState removing state" podUID="6c8669ba-68cf-4ec6-81d4-1c5842a3d34a" containerName="registry" Jul 12 17:59:12.356882 ip-10-0-129-128 kubenswrapper[2559]: I0712 17:59:12.356865 2559 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="opendatahub/opendatahub-operator-controller-manager-5dbd4cc77c-dphpk" Jul 12 17:59:12.360676 ip-10-0-129-128 kubenswrapper[2559]: I0712 17:59:12.360633 2559 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"opendatahub\"/\"opendatahub-operator-controller-manager-service-cert\"" Jul 12 17:59:12.360808 ip-10-0-129-128 kubenswrapper[2559]: I0712 17:59:12.360742 2559 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"opendatahub\"/\"opendatahub-operator-controller-webhook-cert\"" Jul 12 17:59:12.360808 ip-10-0-129-128 kubenswrapper[2559]: I0712 17:59:12.360788 2559 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"opendatahub\"/\"kube-root-ca.crt\"" Jul 12 17:59:12.360973 ip-10-0-129-128 kubenswrapper[2559]: I0712 17:59:12.360960 2559 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"opendatahub\"/\"openshift-service-ca.crt\"" Jul 12 17:59:12.361015 ip-10-0-129-128 kubenswrapper[2559]: I0712 17:59:12.360999 2559 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"opendatahub\"/\"opendatahub-operator-controller-manager-dockercfg-m5hgb\"" Jul 12 17:59:12.371185 ip-10-0-129-128 kubenswrapper[2559]: I0712 17:59:12.371157 2559 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["opendatahub/opendatahub-operator-controller-manager-5dbd4cc77c-dphpk"] Jul 12 17:59:12.496413 ip-10-0-129-128 kubenswrapper[2559]: I0712 17:59:12.496372 2559 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"webhook-cert\" (UniqueName: \"kubernetes.io/secret/e845af50-76b4-4945-b406-f7fb65e0f475-webhook-cert\") pod \"opendatahub-operator-controller-manager-5dbd4cc77c-dphpk\" (UID: \"e845af50-76b4-4945-b406-f7fb65e0f475\") " pod="opendatahub/opendatahub-operator-controller-manager-5dbd4cc77c-dphpk" Jul 12 17:59:12.496413 ip-10-0-129-128 kubenswrapper[2559]: I0712 17:59:12.496415 2559 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"manifests\" (UniqueName: \"kubernetes.io/empty-dir/e845af50-76b4-4945-b406-f7fb65e0f475-manifests\") pod \"opendatahub-operator-controller-manager-5dbd4cc77c-dphpk\" (UID: \"e845af50-76b4-4945-b406-f7fb65e0f475\") " pod="opendatahub/opendatahub-operator-controller-manager-5dbd4cc77c-dphpk" Jul 12 17:59:12.496614 ip-10-0-129-128 kubenswrapper[2559]: I0712 17:59:12.496439 2559 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tmp\" (UniqueName: \"kubernetes.io/empty-dir/e845af50-76b4-4945-b406-f7fb65e0f475-tmp\") pod \"opendatahub-operator-controller-manager-5dbd4cc77c-dphpk\" (UID: \"e845af50-76b4-4945-b406-f7fb65e0f475\") " pod="opendatahub/opendatahub-operator-controller-manager-5dbd4cc77c-dphpk" Jul 12 17:59:12.496614 ip-10-0-129-128 kubenswrapper[2559]: I0712 17:59:12.496525 2559 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"apiservice-cert\" (UniqueName: \"kubernetes.io/secret/e845af50-76b4-4945-b406-f7fb65e0f475-apiservice-cert\") pod \"opendatahub-operator-controller-manager-5dbd4cc77c-dphpk\" (UID: \"e845af50-76b4-4945-b406-f7fb65e0f475\") " pod="opendatahub/opendatahub-operator-controller-manager-5dbd4cc77c-dphpk" Jul 12 17:59:12.496614 ip-10-0-129-128 kubenswrapper[2559]: I0712 17:59:12.496561 2559 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-gnbcm\" (UniqueName: \"kubernetes.io/projected/e845af50-76b4-4945-b406-f7fb65e0f475-kube-api-access-gnbcm\") pod \"opendatahub-operator-controller-manager-5dbd4cc77c-dphpk\" (UID: \"e845af50-76b4-4945-b406-f7fb65e0f475\") " pod="opendatahub/opendatahub-operator-controller-manager-5dbd4cc77c-dphpk" Jul 12 17:59:12.597785 ip-10-0-129-128 kubenswrapper[2559]: I0712 17:59:12.597739 2559 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"apiservice-cert\" (UniqueName: \"kubernetes.io/secret/e845af50-76b4-4945-b406-f7fb65e0f475-apiservice-cert\") pod \"opendatahub-operator-controller-manager-5dbd4cc77c-dphpk\" (UID: \"e845af50-76b4-4945-b406-f7fb65e0f475\") " pod="opendatahub/opendatahub-operator-controller-manager-5dbd4cc77c-dphpk" Jul 12 17:59:12.597785 ip-10-0-129-128 kubenswrapper[2559]: I0712 17:59:12.597787 2559 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-gnbcm\" (UniqueName: \"kubernetes.io/projected/e845af50-76b4-4945-b406-f7fb65e0f475-kube-api-access-gnbcm\") pod \"opendatahub-operator-controller-manager-5dbd4cc77c-dphpk\" (UID: \"e845af50-76b4-4945-b406-f7fb65e0f475\") " pod="opendatahub/opendatahub-operator-controller-manager-5dbd4cc77c-dphpk" Jul 12 17:59:12.598028 ip-10-0-129-128 kubenswrapper[2559]: I0712 17:59:12.597832 2559 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"webhook-cert\" (UniqueName: \"kubernetes.io/secret/e845af50-76b4-4945-b406-f7fb65e0f475-webhook-cert\") pod \"opendatahub-operator-controller-manager-5dbd4cc77c-dphpk\" (UID: \"e845af50-76b4-4945-b406-f7fb65e0f475\") " pod="opendatahub/opendatahub-operator-controller-manager-5dbd4cc77c-dphpk" Jul 12 17:59:12.598028 ip-10-0-129-128 kubenswrapper[2559]: I0712 17:59:12.597852 2559 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"manifests\" (UniqueName: \"kubernetes.io/empty-dir/e845af50-76b4-4945-b406-f7fb65e0f475-manifests\") pod \"opendatahub-operator-controller-manager-5dbd4cc77c-dphpk\" (UID: \"e845af50-76b4-4945-b406-f7fb65e0f475\") " pod="opendatahub/opendatahub-operator-controller-manager-5dbd4cc77c-dphpk" Jul 12 17:59:12.598028 ip-10-0-129-128 kubenswrapper[2559]: I0712 17:59:12.597875 2559 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tmp\" (UniqueName: \"kubernetes.io/empty-dir/e845af50-76b4-4945-b406-f7fb65e0f475-tmp\") pod \"opendatahub-operator-controller-manager-5dbd4cc77c-dphpk\" (UID: \"e845af50-76b4-4945-b406-f7fb65e0f475\") " pod="opendatahub/opendatahub-operator-controller-manager-5dbd4cc77c-dphpk" Jul 12 17:59:12.598343 ip-10-0-129-128 kubenswrapper[2559]: I0712 17:59:12.598319 2559 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"manifests\" (UniqueName: \"kubernetes.io/empty-dir/e845af50-76b4-4945-b406-f7fb65e0f475-manifests\") pod \"opendatahub-operator-controller-manager-5dbd4cc77c-dphpk\" (UID: \"e845af50-76b4-4945-b406-f7fb65e0f475\") " pod="opendatahub/opendatahub-operator-controller-manager-5dbd4cc77c-dphpk" Jul 12 17:59:12.600626 ip-10-0-129-128 kubenswrapper[2559]: I0712 17:59:12.600600 2559 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"tmp\" (UniqueName: \"kubernetes.io/empty-dir/e845af50-76b4-4945-b406-f7fb65e0f475-tmp\") pod \"opendatahub-operator-controller-manager-5dbd4cc77c-dphpk\" (UID: \"e845af50-76b4-4945-b406-f7fb65e0f475\") " pod="opendatahub/opendatahub-operator-controller-manager-5dbd4cc77c-dphpk" Jul 12 17:59:12.600754 ip-10-0-129-128 kubenswrapper[2559]: I0712 17:59:12.600735 2559 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"apiservice-cert\" (UniqueName: \"kubernetes.io/secret/e845af50-76b4-4945-b406-f7fb65e0f475-apiservice-cert\") pod \"opendatahub-operator-controller-manager-5dbd4cc77c-dphpk\" (UID: \"e845af50-76b4-4945-b406-f7fb65e0f475\") " pod="opendatahub/opendatahub-operator-controller-manager-5dbd4cc77c-dphpk" Jul 12 17:59:12.600754 ip-10-0-129-128 kubenswrapper[2559]: I0712 17:59:12.600748 2559 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"webhook-cert\" (UniqueName: \"kubernetes.io/secret/e845af50-76b4-4945-b406-f7fb65e0f475-webhook-cert\") pod \"opendatahub-operator-controller-manager-5dbd4cc77c-dphpk\" (UID: \"e845af50-76b4-4945-b406-f7fb65e0f475\") " pod="opendatahub/opendatahub-operator-controller-manager-5dbd4cc77c-dphpk" Jul 12 17:59:12.607750 ip-10-0-129-128 kubenswrapper[2559]: I0712 17:59:12.607685 2559 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-gnbcm\" (UniqueName: \"kubernetes.io/projected/e845af50-76b4-4945-b406-f7fb65e0f475-kube-api-access-gnbcm\") pod \"opendatahub-operator-controller-manager-5dbd4cc77c-dphpk\" (UID: \"e845af50-76b4-4945-b406-f7fb65e0f475\") " pod="opendatahub/opendatahub-operator-controller-manager-5dbd4cc77c-dphpk" Jul 12 17:59:12.672002 ip-10-0-129-128 kubenswrapper[2559]: I0712 17:59:12.671966 2559 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="opendatahub/opendatahub-operator-controller-manager-5dbd4cc77c-dphpk" Jul 12 17:59:12.803688 ip-10-0-129-128 kubenswrapper[2559]: I0712 17:59:12.803657 2559 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["opendatahub/opendatahub-operator-controller-manager-5dbd4cc77c-dphpk"] Jul 12 17:59:12.806621 ip-10-0-129-128 kubenswrapper[2559]: W0712 17:59:12.806591 2559 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pode845af50_76b4_4945_b406_f7fb65e0f475.slice/crio-974d4785080ad4e89133204926dd94a7ea6ce4bebb17a72771e236c2f9dd0dca WatchSource:0}: Error finding container 974d4785080ad4e89133204926dd94a7ea6ce4bebb17a72771e236c2f9dd0dca: Status 404 returned error can't find the container with id 974d4785080ad4e89133204926dd94a7ea6ce4bebb17a72771e236c2f9dd0dca Jul 12 17:59:12.808368 ip-10-0-129-128 kubenswrapper[2559]: I0712 17:59:12.808351 2559 provider.go:93] Refreshing cache for provider: *credentialprovider.defaultDockerConfigProvider Jul 12 17:59:13.667850 ip-10-0-129-128 kubenswrapper[2559]: I0712 17:59:13.667811 2559 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="opendatahub/opendatahub-operator-controller-manager-5dbd4cc77c-dphpk" event={"ID":"e845af50-76b4-4945-b406-f7fb65e0f475","Type":"ContainerStarted","Data":"974d4785080ad4e89133204926dd94a7ea6ce4bebb17a72771e236c2f9dd0dca"} Jul 12 17:59:16.684017 ip-10-0-129-128 kubenswrapper[2559]: I0712 17:59:16.683972 2559 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="opendatahub/opendatahub-operator-controller-manager-5dbd4cc77c-dphpk" event={"ID":"e845af50-76b4-4945-b406-f7fb65e0f475","Type":"ContainerStarted","Data":"116518c4cf056aee8774282905e885f66aa83c569a942b1144cd9064d4940df7"} Jul 12 17:59:17.689254 ip-10-0-129-128 kubenswrapper[2559]: I0712 17:59:17.689209 2559 generic.go:356] "Generic (PLEG): container finished" podID="e845af50-76b4-4945-b406-f7fb65e0f475" containerID="116518c4cf056aee8774282905e885f66aa83c569a942b1144cd9064d4940df7" exitCode=0 Jul 12 17:59:17.689676 ip-10-0-129-128 kubenswrapper[2559]: I0712 17:59:17.689266 2559 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="opendatahub/opendatahub-operator-controller-manager-5dbd4cc77c-dphpk" event={"ID":"e845af50-76b4-4945-b406-f7fb65e0f475","Type":"ContainerDied","Data":"116518c4cf056aee8774282905e885f66aa83c569a942b1144cd9064d4940df7"} Jul 12 17:59:18.696061 ip-10-0-129-128 kubenswrapper[2559]: I0712 17:59:18.696025 2559 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="opendatahub/opendatahub-operator-controller-manager-5dbd4cc77c-dphpk" event={"ID":"e845af50-76b4-4945-b406-f7fb65e0f475","Type":"ContainerStarted","Data":"8937b4ac492f409a1dc92af56452cba82c4c2084dda62184da0ad0bbe47948bc"} Jul 12 17:59:18.696489 ip-10-0-129-128 kubenswrapper[2559]: I0712 17:59:18.696268 2559 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="opendatahub/opendatahub-operator-controller-manager-5dbd4cc77c-dphpk" Jul 12 17:59:18.717289 ip-10-0-129-128 kubenswrapper[2559]: I0712 17:59:18.717241 2559 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="opendatahub/opendatahub-operator-controller-manager-5dbd4cc77c-dphpk" podStartSLOduration=2.334264864 podStartE2EDuration="6.717225301s" totalImagesPullingTime="3.313512245s" totalInitContainerRuntime="1.069448192s" isStatefulPod=false podCreationTimestamp="2026-07-12 17:59:12 +0000 UTC" imagePullSessionsCount=2 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-12 17:59:18.715902712 +0000 UTC m=+366.209970642" watchObservedRunningTime="2026-07-12 17:59:18.717225301 +0000 UTC m=+366.211293231" Jul 12 17:59:29.702616 ip-10-0-129-128 kubenswrapper[2559]: I0712 17:59:29.702583 2559 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="opendatahub/opendatahub-operator-controller-manager-5dbd4cc77c-dphpk" Jul 12 17:59:32.077840 ip-10-0-129-128 kubenswrapper[2559]: I0712 17:59:32.077804 2559 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-ingress/kube-auth-proxy-795d67df6d-knt8z"] Jul 12 17:59:32.086814 ip-10-0-129-128 kubenswrapper[2559]: I0712 17:59:32.086789 2559 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/kube-auth-proxy-795d67df6d-knt8z" Jul 12 17:59:32.089435 ip-10-0-129-128 kubenswrapper[2559]: I0712 17:59:32.089410 2559 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress\"/\"kube-auth-proxy-creds\"" Jul 12 17:59:32.089546 ip-10-0-129-128 kubenswrapper[2559]: I0712 17:59:32.089514 2559 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ingress\"/\"kube-root-ca.crt\"" Jul 12 17:59:32.090553 ip-10-0-129-128 kubenswrapper[2559]: I0712 17:59:32.090531 2559 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress\"/\"kube-auth-proxy-tls\"" Jul 12 17:59:32.090671 ip-10-0-129-128 kubenswrapper[2559]: I0712 17:59:32.090573 2559 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress\"/\"kube-auth-proxy-dockercfg-lqljj\"" Jul 12 17:59:32.090671 ip-10-0-129-128 kubenswrapper[2559]: I0712 17:59:32.090573 2559 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ingress\"/\"openshift-service-ca.crt\"" Jul 12 17:59:32.096029 ip-10-0-129-128 kubenswrapper[2559]: I0712 17:59:32.096009 2559 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/kube-auth-proxy-795d67df6d-knt8z"] Jul 12 17:59:32.149272 ip-10-0-129-128 kubenswrapper[2559]: I0712 17:59:32.149230 2559 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/77be4045-2796-4d09-b161-95ad4a170b23-tls-certs\") pod \"kube-auth-proxy-795d67df6d-knt8z\" (UID: \"77be4045-2796-4d09-b161-95ad4a170b23\") " pod="openshift-ingress/kube-auth-proxy-795d67df6d-knt8z" Jul 12 17:59:32.149616 ip-10-0-129-128 kubenswrapper[2559]: I0712 17:59:32.149287 2559 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tmp\" (UniqueName: \"kubernetes.io/empty-dir/77be4045-2796-4d09-b161-95ad4a170b23-tmp\") pod \"kube-auth-proxy-795d67df6d-knt8z\" (UID: \"77be4045-2796-4d09-b161-95ad4a170b23\") " pod="openshift-ingress/kube-auth-proxy-795d67df6d-knt8z" Jul 12 17:59:32.149616 ip-10-0-129-128 kubenswrapper[2559]: I0712 17:59:32.149318 2559 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-4bbzs\" (UniqueName: \"kubernetes.io/projected/77be4045-2796-4d09-b161-95ad4a170b23-kube-api-access-4bbzs\") pod \"kube-auth-proxy-795d67df6d-knt8z\" (UID: \"77be4045-2796-4d09-b161-95ad4a170b23\") " pod="openshift-ingress/kube-auth-proxy-795d67df6d-knt8z" Jul 12 17:59:32.250357 ip-10-0-129-128 kubenswrapper[2559]: I0712 17:59:32.250319 2559 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-4bbzs\" (UniqueName: \"kubernetes.io/projected/77be4045-2796-4d09-b161-95ad4a170b23-kube-api-access-4bbzs\") pod \"kube-auth-proxy-795d67df6d-knt8z\" (UID: \"77be4045-2796-4d09-b161-95ad4a170b23\") " pod="openshift-ingress/kube-auth-proxy-795d67df6d-knt8z" Jul 12 17:59:32.250524 ip-10-0-129-128 kubenswrapper[2559]: I0712 17:59:32.250372 2559 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/77be4045-2796-4d09-b161-95ad4a170b23-tls-certs\") pod \"kube-auth-proxy-795d67df6d-knt8z\" (UID: \"77be4045-2796-4d09-b161-95ad4a170b23\") " pod="openshift-ingress/kube-auth-proxy-795d67df6d-knt8z" Jul 12 17:59:32.250524 ip-10-0-129-128 kubenswrapper[2559]: I0712 17:59:32.250409 2559 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tmp\" (UniqueName: \"kubernetes.io/empty-dir/77be4045-2796-4d09-b161-95ad4a170b23-tmp\") pod \"kube-auth-proxy-795d67df6d-knt8z\" (UID: \"77be4045-2796-4d09-b161-95ad4a170b23\") " pod="openshift-ingress/kube-auth-proxy-795d67df6d-knt8z" Jul 12 17:59:32.252631 ip-10-0-129-128 kubenswrapper[2559]: I0712 17:59:32.252611 2559 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"tmp\" (UniqueName: \"kubernetes.io/empty-dir/77be4045-2796-4d09-b161-95ad4a170b23-tmp\") pod \"kube-auth-proxy-795d67df6d-knt8z\" (UID: \"77be4045-2796-4d09-b161-95ad4a170b23\") " pod="openshift-ingress/kube-auth-proxy-795d67df6d-knt8z" Jul 12 17:59:32.252868 ip-10-0-129-128 kubenswrapper[2559]: I0712 17:59:32.252849 2559 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/77be4045-2796-4d09-b161-95ad4a170b23-tls-certs\") pod \"kube-auth-proxy-795d67df6d-knt8z\" (UID: \"77be4045-2796-4d09-b161-95ad4a170b23\") " pod="openshift-ingress/kube-auth-proxy-795d67df6d-knt8z" Jul 12 17:59:32.271856 ip-10-0-129-128 kubenswrapper[2559]: I0712 17:59:32.271828 2559 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-4bbzs\" (UniqueName: \"kubernetes.io/projected/77be4045-2796-4d09-b161-95ad4a170b23-kube-api-access-4bbzs\") pod \"kube-auth-proxy-795d67df6d-knt8z\" (UID: \"77be4045-2796-4d09-b161-95ad4a170b23\") " pod="openshift-ingress/kube-auth-proxy-795d67df6d-knt8z" Jul 12 17:59:32.304525 ip-10-0-129-128 kubenswrapper[2559]: I0712 17:59:32.304494 2559 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["opendatahub/odh-model-controller-7998c67dc4-tdm5v"] Jul 12 17:59:32.306986 ip-10-0-129-128 kubenswrapper[2559]: I0712 17:59:32.306971 2559 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="opendatahub/odh-model-controller-7998c67dc4-tdm5v" Jul 12 17:59:32.310286 ip-10-0-129-128 kubenswrapper[2559]: I0712 17:59:32.310267 2559 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"opendatahub\"/\"odh-model-controller-dockercfg-rnj9v\"" Jul 12 17:59:32.310451 ip-10-0-129-128 kubenswrapper[2559]: I0712 17:59:32.310438 2559 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"opendatahub\"/\"odh-model-controller-webhook-cert\"" Jul 12 17:59:32.321065 ip-10-0-129-128 kubenswrapper[2559]: I0712 17:59:32.321044 2559 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["opendatahub/odh-model-controller-7998c67dc4-tdm5v"] Jul 12 17:59:32.350848 ip-10-0-129-128 kubenswrapper[2559]: I0712 17:59:32.350768 2559 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"cert\" (UniqueName: \"kubernetes.io/secret/5315b422-0354-44c7-be09-903e5cb60ca8-cert\") pod \"odh-model-controller-7998c67dc4-tdm5v\" (UID: \"5315b422-0354-44c7-be09-903e5cb60ca8\") " pod="opendatahub/odh-model-controller-7998c67dc4-tdm5v" Jul 12 17:59:32.350848 ip-10-0-129-128 kubenswrapper[2559]: I0712 17:59:32.350820 2559 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-qgfwd\" (UniqueName: \"kubernetes.io/projected/5315b422-0354-44c7-be09-903e5cb60ca8-kube-api-access-qgfwd\") pod \"odh-model-controller-7998c67dc4-tdm5v\" (UID: \"5315b422-0354-44c7-be09-903e5cb60ca8\") " pod="opendatahub/odh-model-controller-7998c67dc4-tdm5v" Jul 12 17:59:32.396989 ip-10-0-129-128 kubenswrapper[2559]: I0712 17:59:32.396947 2559 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/kube-auth-proxy-795d67df6d-knt8z" Jul 12 17:59:32.451592 ip-10-0-129-128 kubenswrapper[2559]: I0712 17:59:32.451557 2559 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"cert\" (UniqueName: \"kubernetes.io/secret/5315b422-0354-44c7-be09-903e5cb60ca8-cert\") pod \"odh-model-controller-7998c67dc4-tdm5v\" (UID: \"5315b422-0354-44c7-be09-903e5cb60ca8\") " pod="opendatahub/odh-model-controller-7998c67dc4-tdm5v" Jul 12 17:59:32.451777 ip-10-0-129-128 kubenswrapper[2559]: I0712 17:59:32.451672 2559 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-qgfwd\" (UniqueName: \"kubernetes.io/projected/5315b422-0354-44c7-be09-903e5cb60ca8-kube-api-access-qgfwd\") pod \"odh-model-controller-7998c67dc4-tdm5v\" (UID: \"5315b422-0354-44c7-be09-903e5cb60ca8\") " pod="opendatahub/odh-model-controller-7998c67dc4-tdm5v" Jul 12 17:59:32.451777 ip-10-0-129-128 kubenswrapper[2559]: E0712 17:59:32.451721 2559 secret.go:189] Couldn't get secret opendatahub/odh-model-controller-webhook-cert: secret "odh-model-controller-webhook-cert" not found Jul 12 17:59:32.451893 ip-10-0-129-128 kubenswrapper[2559]: E0712 17:59:32.451791 2559 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/5315b422-0354-44c7-be09-903e5cb60ca8-cert podName:5315b422-0354-44c7-be09-903e5cb60ca8 nodeName:}" failed. No retries permitted until 2026-07-12 17:59:32.951775224 +0000 UTC m=+380.445843132 (durationBeforeRetry 500ms). Error: MountVolume.SetUp failed for volume "cert" (UniqueName: "kubernetes.io/secret/5315b422-0354-44c7-be09-903e5cb60ca8-cert") pod "odh-model-controller-7998c67dc4-tdm5v" (UID: "5315b422-0354-44c7-be09-903e5cb60ca8") : secret "odh-model-controller-webhook-cert" not found Jul 12 17:59:32.461925 ip-10-0-129-128 kubenswrapper[2559]: I0712 17:59:32.461860 2559 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-qgfwd\" (UniqueName: \"kubernetes.io/projected/5315b422-0354-44c7-be09-903e5cb60ca8-kube-api-access-qgfwd\") pod \"odh-model-controller-7998c67dc4-tdm5v\" (UID: \"5315b422-0354-44c7-be09-903e5cb60ca8\") " pod="opendatahub/odh-model-controller-7998c67dc4-tdm5v" Jul 12 17:59:32.530764 ip-10-0-129-128 kubenswrapper[2559]: I0712 17:59:32.530708 2559 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/kube-auth-proxy-795d67df6d-knt8z"] Jul 12 17:59:32.533455 ip-10-0-129-128 kubenswrapper[2559]: W0712 17:59:32.533426 2559 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod77be4045_2796_4d09_b161_95ad4a170b23.slice/crio-005f28e06d298239bd5433ff64f2d03ba7d48fd835d538391d772fdb737b0287 WatchSource:0}: Error finding container 005f28e06d298239bd5433ff64f2d03ba7d48fd835d538391d772fdb737b0287: Status 404 returned error can't find the container with id 005f28e06d298239bd5433ff64f2d03ba7d48fd835d538391d772fdb737b0287 Jul 12 17:59:32.751032 ip-10-0-129-128 kubenswrapper[2559]: I0712 17:59:32.750948 2559 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/kube-auth-proxy-795d67df6d-knt8z" event={"ID":"77be4045-2796-4d09-b161-95ad4a170b23","Type":"ContainerStarted","Data":"005f28e06d298239bd5433ff64f2d03ba7d48fd835d538391d772fdb737b0287"} Jul 12 17:59:32.954827 ip-10-0-129-128 kubenswrapper[2559]: I0712 17:59:32.954790 2559 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"cert\" (UniqueName: \"kubernetes.io/secret/5315b422-0354-44c7-be09-903e5cb60ca8-cert\") pod \"odh-model-controller-7998c67dc4-tdm5v\" (UID: \"5315b422-0354-44c7-be09-903e5cb60ca8\") " pod="opendatahub/odh-model-controller-7998c67dc4-tdm5v" Jul 12 17:59:32.957113 ip-10-0-129-128 kubenswrapper[2559]: I0712 17:59:32.957090 2559 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"cert\" (UniqueName: \"kubernetes.io/secret/5315b422-0354-44c7-be09-903e5cb60ca8-cert\") pod \"odh-model-controller-7998c67dc4-tdm5v\" (UID: \"5315b422-0354-44c7-be09-903e5cb60ca8\") " pod="opendatahub/odh-model-controller-7998c67dc4-tdm5v" Jul 12 17:59:33.217170 ip-10-0-129-128 kubenswrapper[2559]: I0712 17:59:33.217138 2559 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="opendatahub/odh-model-controller-7998c67dc4-tdm5v" Jul 12 17:59:33.233502 ip-10-0-129-128 kubenswrapper[2559]: I0712 17:59:33.233421 2559 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["opendatahub/model-serving-api-54d89bd5fc-c46wr"] Jul 12 17:59:33.236557 ip-10-0-129-128 kubenswrapper[2559]: I0712 17:59:33.236528 2559 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="opendatahub/model-serving-api-54d89bd5fc-c46wr" Jul 12 17:59:33.239037 ip-10-0-129-128 kubenswrapper[2559]: I0712 17:59:33.239007 2559 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"opendatahub\"/\"model-serving-api-tls\"" Jul 12 17:59:33.245082 ip-10-0-129-128 kubenswrapper[2559]: I0712 17:59:33.245058 2559 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["opendatahub/model-serving-api-54d89bd5fc-c46wr"] Jul 12 17:59:33.257417 ip-10-0-129-128 kubenswrapper[2559]: I0712 17:59:33.257333 2559 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-8mwsh\" (UniqueName: \"kubernetes.io/projected/ddbd25e2-7012-4650-b0ee-53d2854a2c8f-kube-api-access-8mwsh\") pod \"model-serving-api-54d89bd5fc-c46wr\" (UID: \"ddbd25e2-7012-4650-b0ee-53d2854a2c8f\") " pod="opendatahub/model-serving-api-54d89bd5fc-c46wr" Jul 12 17:59:33.257571 ip-10-0-129-128 kubenswrapper[2559]: I0712 17:59:33.257470 2559 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/ddbd25e2-7012-4650-b0ee-53d2854a2c8f-tls-certs\") pod \"model-serving-api-54d89bd5fc-c46wr\" (UID: \"ddbd25e2-7012-4650-b0ee-53d2854a2c8f\") " pod="opendatahub/model-serving-api-54d89bd5fc-c46wr" Jul 12 17:59:33.358677 ip-10-0-129-128 kubenswrapper[2559]: I0712 17:59:33.358357 2559 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-8mwsh\" (UniqueName: \"kubernetes.io/projected/ddbd25e2-7012-4650-b0ee-53d2854a2c8f-kube-api-access-8mwsh\") pod \"model-serving-api-54d89bd5fc-c46wr\" (UID: \"ddbd25e2-7012-4650-b0ee-53d2854a2c8f\") " pod="opendatahub/model-serving-api-54d89bd5fc-c46wr" Jul 12 17:59:33.358677 ip-10-0-129-128 kubenswrapper[2559]: I0712 17:59:33.358464 2559 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/ddbd25e2-7012-4650-b0ee-53d2854a2c8f-tls-certs\") pod \"model-serving-api-54d89bd5fc-c46wr\" (UID: \"ddbd25e2-7012-4650-b0ee-53d2854a2c8f\") " pod="opendatahub/model-serving-api-54d89bd5fc-c46wr" Jul 12 17:59:33.361175 ip-10-0-129-128 kubenswrapper[2559]: I0712 17:59:33.361150 2559 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/ddbd25e2-7012-4650-b0ee-53d2854a2c8f-tls-certs\") pod \"model-serving-api-54d89bd5fc-c46wr\" (UID: \"ddbd25e2-7012-4650-b0ee-53d2854a2c8f\") " pod="opendatahub/model-serving-api-54d89bd5fc-c46wr" Jul 12 17:59:33.367624 ip-10-0-129-128 kubenswrapper[2559]: I0712 17:59:33.367589 2559 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["opendatahub/odh-model-controller-7998c67dc4-tdm5v"] Jul 12 17:59:33.370871 ip-10-0-129-128 kubenswrapper[2559]: W0712 17:59:33.370842 2559 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod5315b422_0354_44c7_be09_903e5cb60ca8.slice/crio-40ecfaea567107e980f92f6c7fa2d2bd22b7b52f01a4a778344cda53ed22d514 WatchSource:0}: Error finding container 40ecfaea567107e980f92f6c7fa2d2bd22b7b52f01a4a778344cda53ed22d514: Status 404 returned error can't find the container with id 40ecfaea567107e980f92f6c7fa2d2bd22b7b52f01a4a778344cda53ed22d514 Jul 12 17:59:33.373973 ip-10-0-129-128 kubenswrapper[2559]: I0712 17:59:33.373947 2559 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-8mwsh\" (UniqueName: \"kubernetes.io/projected/ddbd25e2-7012-4650-b0ee-53d2854a2c8f-kube-api-access-8mwsh\") pod \"model-serving-api-54d89bd5fc-c46wr\" (UID: \"ddbd25e2-7012-4650-b0ee-53d2854a2c8f\") " pod="opendatahub/model-serving-api-54d89bd5fc-c46wr" Jul 12 17:59:33.378194 ip-10-0-129-128 kubenswrapper[2559]: I0712 17:59:33.378172 2559 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-ingress/istiod-openshift-gateway-7574459656-6zg5p"] Jul 12 17:59:33.384413 ip-10-0-129-128 kubenswrapper[2559]: I0712 17:59:33.382252 2559 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/istiod-openshift-gateway-7574459656-6zg5p" Jul 12 17:59:33.385579 ip-10-0-129-128 kubenswrapper[2559]: I0712 17:59:33.385076 2559 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress\"/\"istio-kubeconfig\"" Jul 12 17:59:33.385946 ip-10-0-129-128 kubenswrapper[2559]: I0712 17:59:33.385782 2559 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress\"/\"istiod-tls\"" Jul 12 17:59:33.385946 ip-10-0-129-128 kubenswrapper[2559]: I0712 17:59:33.385864 2559 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ingress\"/\"openshift-gw-ca-root-cert\"" Jul 12 17:59:33.385946 ip-10-0-129-128 kubenswrapper[2559]: I0712 17:59:33.385888 2559 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress\"/\"istiod-openshift-gateway-dockercfg-7rjg7\"" Jul 12 17:59:33.386124 ip-10-0-129-128 kubenswrapper[2559]: I0712 17:59:33.385868 2559 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress\"/\"cacerts\"" Jul 12 17:59:33.397459 ip-10-0-129-128 kubenswrapper[2559]: I0712 17:59:33.397436 2559 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/istiod-openshift-gateway-7574459656-6zg5p"] Jul 12 17:59:33.459386 ip-10-0-129-128 kubenswrapper[2559]: I0712 17:59:33.459350 2559 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/7decb9cf-de84-4357-ae0b-a1530031a3af-istio-token\") pod \"istiod-openshift-gateway-7574459656-6zg5p\" (UID: \"7decb9cf-de84-4357-ae0b-a1530031a3af\") " pod="openshift-ingress/istiod-openshift-gateway-7574459656-6zg5p" Jul 12 17:59:33.459574 ip-10-0-129-128 kubenswrapper[2559]: I0712 17:59:33.459404 2559 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-8sgkz\" (UniqueName: \"kubernetes.io/projected/7decb9cf-de84-4357-ae0b-a1530031a3af-kube-api-access-8sgkz\") pod \"istiod-openshift-gateway-7574459656-6zg5p\" (UID: \"7decb9cf-de84-4357-ae0b-a1530031a3af\") " pod="openshift-ingress/istiod-openshift-gateway-7574459656-6zg5p" Jul 12 17:59:33.459574 ip-10-0-129-128 kubenswrapper[2559]: I0712 17:59:33.459455 2559 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"local-certs\" (UniqueName: \"kubernetes.io/empty-dir/7decb9cf-de84-4357-ae0b-a1530031a3af-local-certs\") pod \"istiod-openshift-gateway-7574459656-6zg5p\" (UID: \"7decb9cf-de84-4357-ae0b-a1530031a3af\") " pod="openshift-ingress/istiod-openshift-gateway-7574459656-6zg5p" Jul 12 17:59:33.459574 ip-10-0-129-128 kubenswrapper[2559]: I0712 17:59:33.459520 2559 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"cacerts\" (UniqueName: \"kubernetes.io/secret/7decb9cf-de84-4357-ae0b-a1530031a3af-cacerts\") pod \"istiod-openshift-gateway-7574459656-6zg5p\" (UID: \"7decb9cf-de84-4357-ae0b-a1530031a3af\") " pod="openshift-ingress/istiod-openshift-gateway-7574459656-6zg5p" Jul 12 17:59:33.459574 ip-10-0-129-128 kubenswrapper[2559]: I0712 17:59:33.459547 2559 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-csr-ca-configmap\" (UniqueName: \"kubernetes.io/configmap/7decb9cf-de84-4357-ae0b-a1530031a3af-istio-csr-ca-configmap\") pod \"istiod-openshift-gateway-7574459656-6zg5p\" (UID: \"7decb9cf-de84-4357-ae0b-a1530031a3af\") " pod="openshift-ingress/istiod-openshift-gateway-7574459656-6zg5p" Jul 12 17:59:33.459861 ip-10-0-129-128 kubenswrapper[2559]: I0712 17:59:33.459580 2559 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-kubeconfig\" (UniqueName: \"kubernetes.io/secret/7decb9cf-de84-4357-ae0b-a1530031a3af-istio-kubeconfig\") pod \"istiod-openshift-gateway-7574459656-6zg5p\" (UID: \"7decb9cf-de84-4357-ae0b-a1530031a3af\") " pod="openshift-ingress/istiod-openshift-gateway-7574459656-6zg5p" Jul 12 17:59:33.459861 ip-10-0-129-128 kubenswrapper[2559]: I0712 17:59:33.459614 2559 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-csr-dns-cert\" (UniqueName: \"kubernetes.io/secret/7decb9cf-de84-4357-ae0b-a1530031a3af-istio-csr-dns-cert\") pod \"istiod-openshift-gateway-7574459656-6zg5p\" (UID: \"7decb9cf-de84-4357-ae0b-a1530031a3af\") " pod="openshift-ingress/istiod-openshift-gateway-7574459656-6zg5p" Jul 12 17:59:33.551712 ip-10-0-129-128 kubenswrapper[2559]: I0712 17:59:33.551679 2559 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="opendatahub/model-serving-api-54d89bd5fc-c46wr" Jul 12 17:59:33.561051 ip-10-0-129-128 kubenswrapper[2559]: I0712 17:59:33.561017 2559 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-kubeconfig\" (UniqueName: \"kubernetes.io/secret/7decb9cf-de84-4357-ae0b-a1530031a3af-istio-kubeconfig\") pod \"istiod-openshift-gateway-7574459656-6zg5p\" (UID: \"7decb9cf-de84-4357-ae0b-a1530031a3af\") " pod="openshift-ingress/istiod-openshift-gateway-7574459656-6zg5p" Jul 12 17:59:33.561192 ip-10-0-129-128 kubenswrapper[2559]: I0712 17:59:33.561072 2559 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-csr-dns-cert\" (UniqueName: \"kubernetes.io/secret/7decb9cf-de84-4357-ae0b-a1530031a3af-istio-csr-dns-cert\") pod \"istiod-openshift-gateway-7574459656-6zg5p\" (UID: \"7decb9cf-de84-4357-ae0b-a1530031a3af\") " pod="openshift-ingress/istiod-openshift-gateway-7574459656-6zg5p" Jul 12 17:59:33.561192 ip-10-0-129-128 kubenswrapper[2559]: I0712 17:59:33.561108 2559 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/7decb9cf-de84-4357-ae0b-a1530031a3af-istio-token\") pod \"istiod-openshift-gateway-7574459656-6zg5p\" (UID: \"7decb9cf-de84-4357-ae0b-a1530031a3af\") " pod="openshift-ingress/istiod-openshift-gateway-7574459656-6zg5p" Jul 12 17:59:33.561192 ip-10-0-129-128 kubenswrapper[2559]: I0712 17:59:33.561139 2559 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-8sgkz\" (UniqueName: \"kubernetes.io/projected/7decb9cf-de84-4357-ae0b-a1530031a3af-kube-api-access-8sgkz\") pod \"istiod-openshift-gateway-7574459656-6zg5p\" (UID: \"7decb9cf-de84-4357-ae0b-a1530031a3af\") " pod="openshift-ingress/istiod-openshift-gateway-7574459656-6zg5p" Jul 12 17:59:33.561348 ip-10-0-129-128 kubenswrapper[2559]: I0712 17:59:33.561205 2559 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"local-certs\" (UniqueName: \"kubernetes.io/empty-dir/7decb9cf-de84-4357-ae0b-a1530031a3af-local-certs\") pod \"istiod-openshift-gateway-7574459656-6zg5p\" (UID: \"7decb9cf-de84-4357-ae0b-a1530031a3af\") " pod="openshift-ingress/istiod-openshift-gateway-7574459656-6zg5p" Jul 12 17:59:33.561348 ip-10-0-129-128 kubenswrapper[2559]: I0712 17:59:33.561255 2559 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"cacerts\" (UniqueName: \"kubernetes.io/secret/7decb9cf-de84-4357-ae0b-a1530031a3af-cacerts\") pod \"istiod-openshift-gateway-7574459656-6zg5p\" (UID: \"7decb9cf-de84-4357-ae0b-a1530031a3af\") " pod="openshift-ingress/istiod-openshift-gateway-7574459656-6zg5p" Jul 12 17:59:33.561348 ip-10-0-129-128 kubenswrapper[2559]: I0712 17:59:33.561280 2559 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-csr-ca-configmap\" (UniqueName: \"kubernetes.io/configmap/7decb9cf-de84-4357-ae0b-a1530031a3af-istio-csr-ca-configmap\") pod \"istiod-openshift-gateway-7574459656-6zg5p\" (UID: \"7decb9cf-de84-4357-ae0b-a1530031a3af\") " pod="openshift-ingress/istiod-openshift-gateway-7574459656-6zg5p" Jul 12 17:59:33.562015 ip-10-0-129-128 kubenswrapper[2559]: I0712 17:59:33.561981 2559 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istio-csr-ca-configmap\" (UniqueName: \"kubernetes.io/configmap/7decb9cf-de84-4357-ae0b-a1530031a3af-istio-csr-ca-configmap\") pod \"istiod-openshift-gateway-7574459656-6zg5p\" (UID: \"7decb9cf-de84-4357-ae0b-a1530031a3af\") " pod="openshift-ingress/istiod-openshift-gateway-7574459656-6zg5p" Jul 12 17:59:33.564352 ip-10-0-129-128 kubenswrapper[2559]: I0712 17:59:33.564326 2559 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istio-kubeconfig\" (UniqueName: \"kubernetes.io/secret/7decb9cf-de84-4357-ae0b-a1530031a3af-istio-kubeconfig\") pod \"istiod-openshift-gateway-7574459656-6zg5p\" (UID: \"7decb9cf-de84-4357-ae0b-a1530031a3af\") " pod="openshift-ingress/istiod-openshift-gateway-7574459656-6zg5p" Jul 12 17:59:33.564485 ip-10-0-129-128 kubenswrapper[2559]: I0712 17:59:33.564462 2559 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"cacerts\" (UniqueName: \"kubernetes.io/secret/7decb9cf-de84-4357-ae0b-a1530031a3af-cacerts\") pod \"istiod-openshift-gateway-7574459656-6zg5p\" (UID: \"7decb9cf-de84-4357-ae0b-a1530031a3af\") " pod="openshift-ingress/istiod-openshift-gateway-7574459656-6zg5p" Jul 12 17:59:33.564764 ip-10-0-129-128 kubenswrapper[2559]: I0712 17:59:33.564744 2559 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"local-certs\" (UniqueName: \"kubernetes.io/empty-dir/7decb9cf-de84-4357-ae0b-a1530031a3af-local-certs\") pod \"istiod-openshift-gateway-7574459656-6zg5p\" (UID: \"7decb9cf-de84-4357-ae0b-a1530031a3af\") " pod="openshift-ingress/istiod-openshift-gateway-7574459656-6zg5p" Jul 12 17:59:33.564856 ip-10-0-129-128 kubenswrapper[2559]: I0712 17:59:33.564819 2559 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istio-csr-dns-cert\" (UniqueName: \"kubernetes.io/secret/7decb9cf-de84-4357-ae0b-a1530031a3af-istio-csr-dns-cert\") pod \"istiod-openshift-gateway-7574459656-6zg5p\" (UID: \"7decb9cf-de84-4357-ae0b-a1530031a3af\") " pod="openshift-ingress/istiod-openshift-gateway-7574459656-6zg5p" Jul 12 17:59:33.569266 ip-10-0-129-128 kubenswrapper[2559]: I0712 17:59:33.569239 2559 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/7decb9cf-de84-4357-ae0b-a1530031a3af-istio-token\") pod \"istiod-openshift-gateway-7574459656-6zg5p\" (UID: \"7decb9cf-de84-4357-ae0b-a1530031a3af\") " pod="openshift-ingress/istiod-openshift-gateway-7574459656-6zg5p" Jul 12 17:59:33.570052 ip-10-0-129-128 kubenswrapper[2559]: I0712 17:59:33.570026 2559 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-8sgkz\" (UniqueName: \"kubernetes.io/projected/7decb9cf-de84-4357-ae0b-a1530031a3af-kube-api-access-8sgkz\") pod \"istiod-openshift-gateway-7574459656-6zg5p\" (UID: \"7decb9cf-de84-4357-ae0b-a1530031a3af\") " pod="openshift-ingress/istiod-openshift-gateway-7574459656-6zg5p" Jul 12 17:59:33.697345 ip-10-0-129-128 kubenswrapper[2559]: I0712 17:59:33.697310 2559 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["opendatahub/model-serving-api-54d89bd5fc-c46wr"] Jul 12 17:59:33.697489 ip-10-0-129-128 kubenswrapper[2559]: I0712 17:59:33.697412 2559 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/istiod-openshift-gateway-7574459656-6zg5p" Jul 12 17:59:33.699846 ip-10-0-129-128 kubenswrapper[2559]: W0712 17:59:33.699818 2559 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podddbd25e2_7012_4650_b0ee_53d2854a2c8f.slice/crio-10467ae3e04a3c2f0b5f5ea58a3e30b6d9733371dc76402b550715ebe55e3263 WatchSource:0}: Error finding container 10467ae3e04a3c2f0b5f5ea58a3e30b6d9733371dc76402b550715ebe55e3263: Status 404 returned error can't find the container with id 10467ae3e04a3c2f0b5f5ea58a3e30b6d9733371dc76402b550715ebe55e3263 Jul 12 17:59:33.759797 ip-10-0-129-128 kubenswrapper[2559]: I0712 17:59:33.759749 2559 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="opendatahub/model-serving-api-54d89bd5fc-c46wr" event={"ID":"ddbd25e2-7012-4650-b0ee-53d2854a2c8f","Type":"ContainerStarted","Data":"10467ae3e04a3c2f0b5f5ea58a3e30b6d9733371dc76402b550715ebe55e3263"} Jul 12 17:59:33.764489 ip-10-0-129-128 kubenswrapper[2559]: I0712 17:59:33.764463 2559 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="opendatahub/odh-model-controller-7998c67dc4-tdm5v" event={"ID":"5315b422-0354-44c7-be09-903e5cb60ca8","Type":"ContainerStarted","Data":"40ecfaea567107e980f92f6c7fa2d2bd22b7b52f01a4a778344cda53ed22d514"} Jul 12 17:59:33.847920 ip-10-0-129-128 kubenswrapper[2559]: I0712 17:59:33.847897 2559 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/istiod-openshift-gateway-7574459656-6zg5p"] Jul 12 17:59:33.850982 ip-10-0-129-128 kubenswrapper[2559]: W0712 17:59:33.850918 2559 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod7decb9cf_de84_4357_ae0b_a1530031a3af.slice/crio-d3c545ec9e2d3fd347bbb4864096bf03f0c0af73acf837e9078f8aaed45cedeb WatchSource:0}: Error finding container d3c545ec9e2d3fd347bbb4864096bf03f0c0af73acf837e9078f8aaed45cedeb: Status 404 returned error can't find the container with id d3c545ec9e2d3fd347bbb4864096bf03f0c0af73acf837e9078f8aaed45cedeb Jul 12 17:59:34.772906 ip-10-0-129-128 kubenswrapper[2559]: I0712 17:59:34.772850 2559 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/istiod-openshift-gateway-7574459656-6zg5p" event={"ID":"7decb9cf-de84-4357-ae0b-a1530031a3af","Type":"ContainerStarted","Data":"d3c545ec9e2d3fd347bbb4864096bf03f0c0af73acf837e9078f8aaed45cedeb"} Jul 12 17:59:37.978223 ip-10-0-129-128 kubenswrapper[2559]: I0712 17:59:37.978189 2559 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236228Ki","pods":"250"} Jul 12 17:59:37.978585 ip-10-0-129-128 kubenswrapper[2559]: I0712 17:59:37.978269 2559 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236228Ki","pods":"250"} Jul 12 17:59:38.796106 ip-10-0-129-128 kubenswrapper[2559]: I0712 17:59:38.796065 2559 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/kube-auth-proxy-795d67df6d-knt8z" event={"ID":"77be4045-2796-4d09-b161-95ad4a170b23","Type":"ContainerStarted","Data":"8f0db901e5ed327b18d23fce3f243478710ed12a0addba5293bfb3df91f4067f"} Jul 12 17:59:38.799239 ip-10-0-129-128 kubenswrapper[2559]: I0712 17:59:38.799207 2559 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="opendatahub/model-serving-api-54d89bd5fc-c46wr" event={"ID":"ddbd25e2-7012-4650-b0ee-53d2854a2c8f","Type":"ContainerStarted","Data":"a896de0a8403ae209b51d1884f868895148e68ec8eef6a1b9c7a7a20fbbaa8b1"} Jul 12 17:59:38.799419 ip-10-0-129-128 kubenswrapper[2559]: I0712 17:59:38.799289 2559 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="opendatahub/model-serving-api-54d89bd5fc-c46wr" Jul 12 17:59:38.801161 ip-10-0-129-128 kubenswrapper[2559]: I0712 17:59:38.801130 2559 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="opendatahub/odh-model-controller-7998c67dc4-tdm5v" event={"ID":"5315b422-0354-44c7-be09-903e5cb60ca8","Type":"ContainerStarted","Data":"880a7f433c886aab9640faa04cd41f33554a0a842ba02ccfaef549244b9bdf85"} Jul 12 17:59:38.801345 ip-10-0-129-128 kubenswrapper[2559]: I0712 17:59:38.801323 2559 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="opendatahub/odh-model-controller-7998c67dc4-tdm5v" Jul 12 17:59:38.803425 ip-10-0-129-128 kubenswrapper[2559]: I0712 17:59:38.803396 2559 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/istiod-openshift-gateway-7574459656-6zg5p" event={"ID":"7decb9cf-de84-4357-ae0b-a1530031a3af","Type":"ContainerStarted","Data":"f23556fc4f0c9b1ba45458787a26a9356df40cbfc5fb64b2b465850ad49d9373"} Jul 12 17:59:38.803940 ip-10-0-129-128 kubenswrapper[2559]: I0712 17:59:38.803917 2559 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/istiod-openshift-gateway-7574459656-6zg5p" Jul 12 17:59:38.823095 ip-10-0-129-128 kubenswrapper[2559]: I0712 17:59:38.823030 2559 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-ingress/kube-auth-proxy-795d67df6d-knt8z" podStartSLOduration=1.441743314 podStartE2EDuration="6.823009354s" totalImagesPullingTime="5.38126604s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-12 17:59:32 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-12 17:59:38.82252819 +0000 UTC m=+386.316596120" watchObservedRunningTime="2026-07-12 17:59:38.823009354 +0000 UTC m=+386.317077285" Jul 12 17:59:38.877233 ip-10-0-129-128 kubenswrapper[2559]: I0712 17:59:38.877153 2559 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-ingress/istiod-openshift-gateway-7574459656-6zg5p" podStartSLOduration=1.752326475 podStartE2EDuration="5.877113272s" totalImagesPullingTime="4.124786797s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-12 17:59:33 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-12 17:59:38.856096529 +0000 UTC m=+386.350164473" watchObservedRunningTime="2026-07-12 17:59:38.877113272 +0000 UTC m=+386.371181202" Jul 12 17:59:38.878753 ip-10-0-129-128 kubenswrapper[2559]: I0712 17:59:38.878679 2559 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="opendatahub/model-serving-api-54d89bd5fc-c46wr" podStartSLOduration=1.663397061 podStartE2EDuration="5.878665067s" totalImagesPullingTime="4.215268006s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-12 17:59:33 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-12 17:59:38.876434396 +0000 UTC m=+386.370502326" watchObservedRunningTime="2026-07-12 17:59:38.878665067 +0000 UTC m=+386.372732998" Jul 12 17:59:38.898580 ip-10-0-129-128 kubenswrapper[2559]: I0712 17:59:38.898246 2559 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="opendatahub/odh-model-controller-7998c67dc4-tdm5v" podStartSLOduration=2.34270825 podStartE2EDuration="6.898224925s" totalImagesPullingTime="4.555516675s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-12 17:59:32 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-12 17:59:38.895876023 +0000 UTC m=+386.389943954" watchObservedRunningTime="2026-07-12 17:59:38.898224925 +0000 UTC m=+386.392292855" Jul 12 17:59:39.458016 ip-10-0-129-128 kubenswrapper[2559]: I0712 17:59:39.457979 2559 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-ingress/data-science-gateway-data-science-gateway-class-6576859cf49q7gm"] Jul 12 17:59:39.461531 ip-10-0-129-128 kubenswrapper[2559]: I0712 17:59:39.461508 2559 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/data-science-gateway-data-science-gateway-class-6576859cf49q7gm" Jul 12 17:59:39.463982 ip-10-0-129-128 kubenswrapper[2559]: I0712 17:59:39.463962 2559 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress\"/\"data-science-gateway-data-science-gateway-class-dockercfg-4szmx\"" Jul 12 17:59:39.476252 ip-10-0-129-128 kubenswrapper[2559]: I0712 17:59:39.476223 2559 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/data-science-gateway-data-science-gateway-class-6576859cf49q7gm"] Jul 12 17:59:39.515661 ip-10-0-129-128 kubenswrapper[2559]: I0712 17:59:39.515598 2559 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/6b94673d-3ac1-4706-b8eb-aaa44bb04be3-istiod-ca-cert\") pod \"data-science-gateway-data-science-gateway-class-6576859cf49q7gm\" (UID: \"6b94673d-3ac1-4706-b8eb-aaa44bb04be3\") " pod="openshift-ingress/data-science-gateway-data-science-gateway-class-6576859cf49q7gm" Jul 12 17:59:39.515802 ip-10-0-129-128 kubenswrapper[2559]: I0712 17:59:39.515673 2559 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-8zwmp\" (UniqueName: \"kubernetes.io/projected/6b94673d-3ac1-4706-b8eb-aaa44bb04be3-kube-api-access-8zwmp\") pod \"data-science-gateway-data-science-gateway-class-6576859cf49q7gm\" (UID: \"6b94673d-3ac1-4706-b8eb-aaa44bb04be3\") " pod="openshift-ingress/data-science-gateway-data-science-gateway-class-6576859cf49q7gm" Jul 12 17:59:39.515802 ip-10-0-129-128 kubenswrapper[2559]: I0712 17:59:39.515724 2559 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/6b94673d-3ac1-4706-b8eb-aaa44bb04be3-workload-socket\") pod \"data-science-gateway-data-science-gateway-class-6576859cf49q7gm\" (UID: \"6b94673d-3ac1-4706-b8eb-aaa44bb04be3\") " pod="openshift-ingress/data-science-gateway-data-science-gateway-class-6576859cf49q7gm" Jul 12 17:59:39.515802 ip-10-0-129-128 kubenswrapper[2559]: I0712 17:59:39.515747 2559 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/6b94673d-3ac1-4706-b8eb-aaa44bb04be3-workload-certs\") pod \"data-science-gateway-data-science-gateway-class-6576859cf49q7gm\" (UID: \"6b94673d-3ac1-4706-b8eb-aaa44bb04be3\") " pod="openshift-ingress/data-science-gateway-data-science-gateway-class-6576859cf49q7gm" Jul 12 17:59:39.515802 ip-10-0-129-128 kubenswrapper[2559]: I0712 17:59:39.515772 2559 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/6b94673d-3ac1-4706-b8eb-aaa44bb04be3-istio-podinfo\") pod \"data-science-gateway-data-science-gateway-class-6576859cf49q7gm\" (UID: \"6b94673d-3ac1-4706-b8eb-aaa44bb04be3\") " pod="openshift-ingress/data-science-gateway-data-science-gateway-class-6576859cf49q7gm" Jul 12 17:59:39.515802 ip-10-0-129-128 kubenswrapper[2559]: I0712 17:59:39.515796 2559 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/6b94673d-3ac1-4706-b8eb-aaa44bb04be3-credential-socket\") pod \"data-science-gateway-data-science-gateway-class-6576859cf49q7gm\" (UID: \"6b94673d-3ac1-4706-b8eb-aaa44bb04be3\") " pod="openshift-ingress/data-science-gateway-data-science-gateway-class-6576859cf49q7gm" Jul 12 17:59:39.515989 ip-10-0-129-128 kubenswrapper[2559]: I0712 17:59:39.515864 2559 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/6b94673d-3ac1-4706-b8eb-aaa44bb04be3-istio-token\") pod \"data-science-gateway-data-science-gateway-class-6576859cf49q7gm\" (UID: \"6b94673d-3ac1-4706-b8eb-aaa44bb04be3\") " pod="openshift-ingress/data-science-gateway-data-science-gateway-class-6576859cf49q7gm" Jul 12 17:59:39.515989 ip-10-0-129-128 kubenswrapper[2559]: I0712 17:59:39.515914 2559 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/6b94673d-3ac1-4706-b8eb-aaa44bb04be3-istio-envoy\") pod \"data-science-gateway-data-science-gateway-class-6576859cf49q7gm\" (UID: \"6b94673d-3ac1-4706-b8eb-aaa44bb04be3\") " pod="openshift-ingress/data-science-gateway-data-science-gateway-class-6576859cf49q7gm" Jul 12 17:59:39.515989 ip-10-0-129-128 kubenswrapper[2559]: I0712 17:59:39.515965 2559 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/6b94673d-3ac1-4706-b8eb-aaa44bb04be3-istio-data\") pod \"data-science-gateway-data-science-gateway-class-6576859cf49q7gm\" (UID: \"6b94673d-3ac1-4706-b8eb-aaa44bb04be3\") " pod="openshift-ingress/data-science-gateway-data-science-gateway-class-6576859cf49q7gm" Jul 12 17:59:39.616694 ip-10-0-129-128 kubenswrapper[2559]: I0712 17:59:39.616630 2559 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/6b94673d-3ac1-4706-b8eb-aaa44bb04be3-istiod-ca-cert\") pod \"data-science-gateway-data-science-gateway-class-6576859cf49q7gm\" (UID: \"6b94673d-3ac1-4706-b8eb-aaa44bb04be3\") " pod="openshift-ingress/data-science-gateway-data-science-gateway-class-6576859cf49q7gm" Jul 12 17:59:39.616875 ip-10-0-129-128 kubenswrapper[2559]: I0712 17:59:39.616703 2559 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-8zwmp\" (UniqueName: \"kubernetes.io/projected/6b94673d-3ac1-4706-b8eb-aaa44bb04be3-kube-api-access-8zwmp\") pod \"data-science-gateway-data-science-gateway-class-6576859cf49q7gm\" (UID: \"6b94673d-3ac1-4706-b8eb-aaa44bb04be3\") " pod="openshift-ingress/data-science-gateway-data-science-gateway-class-6576859cf49q7gm" Jul 12 17:59:39.616875 ip-10-0-129-128 kubenswrapper[2559]: I0712 17:59:39.616739 2559 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/6b94673d-3ac1-4706-b8eb-aaa44bb04be3-workload-socket\") pod \"data-science-gateway-data-science-gateway-class-6576859cf49q7gm\" (UID: \"6b94673d-3ac1-4706-b8eb-aaa44bb04be3\") " pod="openshift-ingress/data-science-gateway-data-science-gateway-class-6576859cf49q7gm" Jul 12 17:59:39.616875 ip-10-0-129-128 kubenswrapper[2559]: I0712 17:59:39.616755 2559 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/6b94673d-3ac1-4706-b8eb-aaa44bb04be3-workload-certs\") pod \"data-science-gateway-data-science-gateway-class-6576859cf49q7gm\" (UID: \"6b94673d-3ac1-4706-b8eb-aaa44bb04be3\") " pod="openshift-ingress/data-science-gateway-data-science-gateway-class-6576859cf49q7gm" Jul 12 17:59:39.616875 ip-10-0-129-128 kubenswrapper[2559]: I0712 17:59:39.616772 2559 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/6b94673d-3ac1-4706-b8eb-aaa44bb04be3-istio-podinfo\") pod \"data-science-gateway-data-science-gateway-class-6576859cf49q7gm\" (UID: \"6b94673d-3ac1-4706-b8eb-aaa44bb04be3\") " pod="openshift-ingress/data-science-gateway-data-science-gateway-class-6576859cf49q7gm" Jul 12 17:59:39.616875 ip-10-0-129-128 kubenswrapper[2559]: I0712 17:59:39.616797 2559 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/6b94673d-3ac1-4706-b8eb-aaa44bb04be3-credential-socket\") pod \"data-science-gateway-data-science-gateway-class-6576859cf49q7gm\" (UID: \"6b94673d-3ac1-4706-b8eb-aaa44bb04be3\") " pod="openshift-ingress/data-science-gateway-data-science-gateway-class-6576859cf49q7gm" Jul 12 17:59:39.616875 ip-10-0-129-128 kubenswrapper[2559]: I0712 17:59:39.616825 2559 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/6b94673d-3ac1-4706-b8eb-aaa44bb04be3-istio-token\") pod \"data-science-gateway-data-science-gateway-class-6576859cf49q7gm\" (UID: \"6b94673d-3ac1-4706-b8eb-aaa44bb04be3\") " pod="openshift-ingress/data-science-gateway-data-science-gateway-class-6576859cf49q7gm" Jul 12 17:59:39.616875 ip-10-0-129-128 kubenswrapper[2559]: I0712 17:59:39.616847 2559 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/6b94673d-3ac1-4706-b8eb-aaa44bb04be3-istio-envoy\") pod \"data-science-gateway-data-science-gateway-class-6576859cf49q7gm\" (UID: \"6b94673d-3ac1-4706-b8eb-aaa44bb04be3\") " pod="openshift-ingress/data-science-gateway-data-science-gateway-class-6576859cf49q7gm" Jul 12 17:59:39.617232 ip-10-0-129-128 kubenswrapper[2559]: I0712 17:59:39.616902 2559 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/6b94673d-3ac1-4706-b8eb-aaa44bb04be3-istio-data\") pod \"data-science-gateway-data-science-gateway-class-6576859cf49q7gm\" (UID: \"6b94673d-3ac1-4706-b8eb-aaa44bb04be3\") " pod="openshift-ingress/data-science-gateway-data-science-gateway-class-6576859cf49q7gm" Jul 12 17:59:39.617232 ip-10-0-129-128 kubenswrapper[2559]: I0712 17:59:39.617185 2559 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/6b94673d-3ac1-4706-b8eb-aaa44bb04be3-workload-socket\") pod \"data-science-gateway-data-science-gateway-class-6576859cf49q7gm\" (UID: \"6b94673d-3ac1-4706-b8eb-aaa44bb04be3\") " pod="openshift-ingress/data-science-gateway-data-science-gateway-class-6576859cf49q7gm" Jul 12 17:59:39.617338 ip-10-0-129-128 kubenswrapper[2559]: I0712 17:59:39.617265 2559 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/6b94673d-3ac1-4706-b8eb-aaa44bb04be3-workload-certs\") pod \"data-science-gateway-data-science-gateway-class-6576859cf49q7gm\" (UID: \"6b94673d-3ac1-4706-b8eb-aaa44bb04be3\") " pod="openshift-ingress/data-science-gateway-data-science-gateway-class-6576859cf49q7gm" Jul 12 17:59:39.617448 ip-10-0-129-128 kubenswrapper[2559]: I0712 17:59:39.617424 2559 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/6b94673d-3ac1-4706-b8eb-aaa44bb04be3-credential-socket\") pod \"data-science-gateway-data-science-gateway-class-6576859cf49q7gm\" (UID: \"6b94673d-3ac1-4706-b8eb-aaa44bb04be3\") " pod="openshift-ingress/data-science-gateway-data-science-gateway-class-6576859cf49q7gm" Jul 12 17:59:39.617512 ip-10-0-129-128 kubenswrapper[2559]: I0712 17:59:39.617441 2559 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/6b94673d-3ac1-4706-b8eb-aaa44bb04be3-istiod-ca-cert\") pod \"data-science-gateway-data-science-gateway-class-6576859cf49q7gm\" (UID: \"6b94673d-3ac1-4706-b8eb-aaa44bb04be3\") " pod="openshift-ingress/data-science-gateway-data-science-gateway-class-6576859cf49q7gm" Jul 12 17:59:39.617512 ip-10-0-129-128 kubenswrapper[2559]: I0712 17:59:39.617493 2559 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/6b94673d-3ac1-4706-b8eb-aaa44bb04be3-istio-data\") pod \"data-science-gateway-data-science-gateway-class-6576859cf49q7gm\" (UID: \"6b94673d-3ac1-4706-b8eb-aaa44bb04be3\") " pod="openshift-ingress/data-science-gateway-data-science-gateway-class-6576859cf49q7gm" Jul 12 17:59:39.619518 ip-10-0-129-128 kubenswrapper[2559]: I0712 17:59:39.619495 2559 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/6b94673d-3ac1-4706-b8eb-aaa44bb04be3-istio-envoy\") pod \"data-science-gateway-data-science-gateway-class-6576859cf49q7gm\" (UID: \"6b94673d-3ac1-4706-b8eb-aaa44bb04be3\") " pod="openshift-ingress/data-science-gateway-data-science-gateway-class-6576859cf49q7gm" Jul 12 17:59:39.619651 ip-10-0-129-128 kubenswrapper[2559]: I0712 17:59:39.619617 2559 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/6b94673d-3ac1-4706-b8eb-aaa44bb04be3-istio-podinfo\") pod \"data-science-gateway-data-science-gateway-class-6576859cf49q7gm\" (UID: \"6b94673d-3ac1-4706-b8eb-aaa44bb04be3\") " pod="openshift-ingress/data-science-gateway-data-science-gateway-class-6576859cf49q7gm" Jul 12 17:59:39.624961 ip-10-0-129-128 kubenswrapper[2559]: I0712 17:59:39.624936 2559 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/6b94673d-3ac1-4706-b8eb-aaa44bb04be3-istio-token\") pod \"data-science-gateway-data-science-gateway-class-6576859cf49q7gm\" (UID: \"6b94673d-3ac1-4706-b8eb-aaa44bb04be3\") " pod="openshift-ingress/data-science-gateway-data-science-gateway-class-6576859cf49q7gm" Jul 12 17:59:39.625228 ip-10-0-129-128 kubenswrapper[2559]: I0712 17:59:39.625206 2559 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-8zwmp\" (UniqueName: \"kubernetes.io/projected/6b94673d-3ac1-4706-b8eb-aaa44bb04be3-kube-api-access-8zwmp\") pod \"data-science-gateway-data-science-gateway-class-6576859cf49q7gm\" (UID: \"6b94673d-3ac1-4706-b8eb-aaa44bb04be3\") " pod="openshift-ingress/data-science-gateway-data-science-gateway-class-6576859cf49q7gm" Jul 12 17:59:39.772108 ip-10-0-129-128 kubenswrapper[2559]: I0712 17:59:39.772077 2559 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/data-science-gateway-data-science-gateway-class-6576859cf49q7gm" Jul 12 17:59:39.809699 ip-10-0-129-128 kubenswrapper[2559]: I0712 17:59:39.809622 2559 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/istiod-openshift-gateway-7574459656-6zg5p" Jul 12 17:59:39.922428 ip-10-0-129-128 kubenswrapper[2559]: I0712 17:59:39.922401 2559 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/data-science-gateway-data-science-gateway-class-6576859cf49q7gm"] Jul 12 17:59:39.924492 ip-10-0-129-128 kubenswrapper[2559]: W0712 17:59:39.924467 2559 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod6b94673d_3ac1_4706_b8eb_aaa44bb04be3.slice/crio-e1c04d6e0957d88c0cb17eb26ed9338236e4ab1c242de91502dfde0fb9d72162 WatchSource:0}: Error finding container e1c04d6e0957d88c0cb17eb26ed9338236e4ab1c242de91502dfde0fb9d72162: Status 404 returned error can't find the container with id e1c04d6e0957d88c0cb17eb26ed9338236e4ab1c242de91502dfde0fb9d72162 Jul 12 17:59:40.814442 ip-10-0-129-128 kubenswrapper[2559]: I0712 17:59:40.814401 2559 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/data-science-gateway-data-science-gateway-class-6576859cf49q7gm" event={"ID":"6b94673d-3ac1-4706-b8eb-aaa44bb04be3","Type":"ContainerStarted","Data":"e1c04d6e0957d88c0cb17eb26ed9338236e4ab1c242de91502dfde0fb9d72162"} Jul 12 17:59:41.992134 ip-10-0-129-128 kubenswrapper[2559]: I0712 17:59:41.992093 2559 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["opendatahub/kserve-controller-manager-d6f4b65b8-zm6bp"] Jul 12 17:59:41.999215 ip-10-0-129-128 kubenswrapper[2559]: I0712 17:59:41.999186 2559 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="opendatahub/kserve-controller-manager-d6f4b65b8-zm6bp" Jul 12 17:59:42.003631 ip-10-0-129-128 kubenswrapper[2559]: I0712 17:59:42.003606 2559 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"opendatahub\"/\"kserve-webhook-server-cert\"" Jul 12 17:59:42.004462 ip-10-0-129-128 kubenswrapper[2559]: I0712 17:59:42.004440 2559 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"opendatahub\"/\"kserve-controller-manager-dockercfg-m5tch\"" Jul 12 17:59:42.006275 ip-10-0-129-128 kubenswrapper[2559]: I0712 17:59:42.006223 2559 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["opendatahub/kserve-controller-manager-d6f4b65b8-zm6bp"] Jul 12 17:59:42.039072 ip-10-0-129-128 kubenswrapper[2559]: I0712 17:59:42.038850 2559 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"cert\" (UniqueName: \"kubernetes.io/secret/402923e6-af3f-48fc-ae8e-f6cd828a0e73-cert\") pod \"kserve-controller-manager-d6f4b65b8-zm6bp\" (UID: \"402923e6-af3f-48fc-ae8e-f6cd828a0e73\") " pod="opendatahub/kserve-controller-manager-d6f4b65b8-zm6bp" Jul 12 17:59:42.039072 ip-10-0-129-128 kubenswrapper[2559]: I0712 17:59:42.038998 2559 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-z6t29\" (UniqueName: \"kubernetes.io/projected/402923e6-af3f-48fc-ae8e-f6cd828a0e73-kube-api-access-z6t29\") pod \"kserve-controller-manager-d6f4b65b8-zm6bp\" (UID: \"402923e6-af3f-48fc-ae8e-f6cd828a0e73\") " pod="opendatahub/kserve-controller-manager-d6f4b65b8-zm6bp" Jul 12 17:59:42.139887 ip-10-0-129-128 kubenswrapper[2559]: I0712 17:59:42.139852 2559 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"cert\" (UniqueName: \"kubernetes.io/secret/402923e6-af3f-48fc-ae8e-f6cd828a0e73-cert\") pod \"kserve-controller-manager-d6f4b65b8-zm6bp\" (UID: \"402923e6-af3f-48fc-ae8e-f6cd828a0e73\") " pod="opendatahub/kserve-controller-manager-d6f4b65b8-zm6bp" Jul 12 17:59:42.140083 ip-10-0-129-128 kubenswrapper[2559]: I0712 17:59:42.139973 2559 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-z6t29\" (UniqueName: \"kubernetes.io/projected/402923e6-af3f-48fc-ae8e-f6cd828a0e73-kube-api-access-z6t29\") pod \"kserve-controller-manager-d6f4b65b8-zm6bp\" (UID: \"402923e6-af3f-48fc-ae8e-f6cd828a0e73\") " pod="opendatahub/kserve-controller-manager-d6f4b65b8-zm6bp" Jul 12 17:59:42.140083 ip-10-0-129-128 kubenswrapper[2559]: E0712 17:59:42.140022 2559 secret.go:189] Couldn't get secret opendatahub/kserve-webhook-server-cert: secret "kserve-webhook-server-cert" not found Jul 12 17:59:42.140197 ip-10-0-129-128 kubenswrapper[2559]: E0712 17:59:42.140102 2559 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/402923e6-af3f-48fc-ae8e-f6cd828a0e73-cert podName:402923e6-af3f-48fc-ae8e-f6cd828a0e73 nodeName:}" failed. No retries permitted until 2026-07-12 17:59:42.640079561 +0000 UTC m=+390.134147488 (durationBeforeRetry 500ms). Error: MountVolume.SetUp failed for volume "cert" (UniqueName: "kubernetes.io/secret/402923e6-af3f-48fc-ae8e-f6cd828a0e73-cert") pod "kserve-controller-manager-d6f4b65b8-zm6bp" (UID: "402923e6-af3f-48fc-ae8e-f6cd828a0e73") : secret "kserve-webhook-server-cert" not found Jul 12 17:59:42.153992 ip-10-0-129-128 kubenswrapper[2559]: I0712 17:59:42.153960 2559 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-z6t29\" (UniqueName: \"kubernetes.io/projected/402923e6-af3f-48fc-ae8e-f6cd828a0e73-kube-api-access-z6t29\") pod \"kserve-controller-manager-d6f4b65b8-zm6bp\" (UID: \"402923e6-af3f-48fc-ae8e-f6cd828a0e73\") " pod="opendatahub/kserve-controller-manager-d6f4b65b8-zm6bp" Jul 12 17:59:42.500699 ip-10-0-129-128 kubenswrapper[2559]: I0712 17:59:42.500667 2559 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236228Ki","pods":"250"} Jul 12 17:59:42.500788 ip-10-0-129-128 kubenswrapper[2559]: I0712 17:59:42.500740 2559 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236228Ki","pods":"250"} Jul 12 17:59:42.500788 ip-10-0-129-128 kubenswrapper[2559]: I0712 17:59:42.500775 2559 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236228Ki","pods":"250"} Jul 12 17:59:42.644632 ip-10-0-129-128 kubenswrapper[2559]: I0712 17:59:42.644596 2559 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"cert\" (UniqueName: \"kubernetes.io/secret/402923e6-af3f-48fc-ae8e-f6cd828a0e73-cert\") pod \"kserve-controller-manager-d6f4b65b8-zm6bp\" (UID: \"402923e6-af3f-48fc-ae8e-f6cd828a0e73\") " pod="opendatahub/kserve-controller-manager-d6f4b65b8-zm6bp" Jul 12 17:59:42.647094 ip-10-0-129-128 kubenswrapper[2559]: I0712 17:59:42.647067 2559 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"cert\" (UniqueName: \"kubernetes.io/secret/402923e6-af3f-48fc-ae8e-f6cd828a0e73-cert\") pod \"kserve-controller-manager-d6f4b65b8-zm6bp\" (UID: \"402923e6-af3f-48fc-ae8e-f6cd828a0e73\") " pod="opendatahub/kserve-controller-manager-d6f4b65b8-zm6bp" Jul 12 17:59:42.826769 ip-10-0-129-128 kubenswrapper[2559]: I0712 17:59:42.826729 2559 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/data-science-gateway-data-science-gateway-class-6576859cf49q7gm" event={"ID":"6b94673d-3ac1-4706-b8eb-aaa44bb04be3","Type":"ContainerStarted","Data":"06a2b1442dca22d95a990f0bc8950caab11929614b14da534692c9788c7d6dc6"} Jul 12 17:59:42.861943 ip-10-0-129-128 kubenswrapper[2559]: I0712 17:59:42.861880 2559 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-ingress/data-science-gateway-data-science-gateway-class-6576859cf49q7gm" podStartSLOduration=1.288431202 podStartE2EDuration="3.861858896s" totalImagesPullingTime="2.573427694s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-12 17:59:39 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-12 17:59:42.861203047 +0000 UTC m=+390.355270978" watchObservedRunningTime="2026-07-12 17:59:42.861858896 +0000 UTC m=+390.355927039" Jul 12 17:59:42.912604 ip-10-0-129-128 kubenswrapper[2559]: I0712 17:59:42.912564 2559 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="opendatahub/kserve-controller-manager-d6f4b65b8-zm6bp" Jul 12 17:59:43.063718 ip-10-0-129-128 kubenswrapper[2559]: I0712 17:59:43.063693 2559 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["opendatahub/kserve-controller-manager-d6f4b65b8-zm6bp"] Jul 12 17:59:43.065380 ip-10-0-129-128 kubenswrapper[2559]: W0712 17:59:43.065353 2559 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod402923e6_af3f_48fc_ae8e_f6cd828a0e73.slice/crio-76bbb4db435d025a718422b53d99b90a97b5ef506b3ed5be4542ffc9811ff6ac WatchSource:0}: Error finding container 76bbb4db435d025a718422b53d99b90a97b5ef506b3ed5be4542ffc9811ff6ac: Status 404 returned error can't find the container with id 76bbb4db435d025a718422b53d99b90a97b5ef506b3ed5be4542ffc9811ff6ac Jul 12 17:59:43.521834 ip-10-0-129-128 kubenswrapper[2559]: I0712 17:59:43.521800 2559 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-ingress/istiod-openshift-gateway-7574459656-6zg5p"] Jul 12 17:59:43.522038 ip-10-0-129-128 kubenswrapper[2559]: I0712 17:59:43.522015 2559 kuberuntime_container.go:873] "Killing container with a grace period" pod="openshift-ingress/istiod-openshift-gateway-7574459656-6zg5p" podUID="7decb9cf-de84-4357-ae0b-a1530031a3af" containerName="discovery" containerID="cri-o://f23556fc4f0c9b1ba45458787a26a9356df40cbfc5fb64b2b465850ad49d9373" gracePeriod=30 Jul 12 17:59:43.772763 ip-10-0-129-128 kubenswrapper[2559]: I0712 17:59:43.772691 2559 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="openshift-ingress/data-science-gateway-data-science-gateway-class-6576859cf49q7gm" Jul 12 17:59:43.777155 ip-10-0-129-128 kubenswrapper[2559]: I0712 17:59:43.777130 2559 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/istiod-openshift-gateway-7574459656-6zg5p" Jul 12 17:59:43.778136 ip-10-0-129-128 kubenswrapper[2559]: I0712 17:59:43.778115 2559 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="openshift-ingress/data-science-gateway-data-science-gateway-class-6576859cf49q7gm" Jul 12 17:59:43.836226 ip-10-0-129-128 kubenswrapper[2559]: I0712 17:59:43.835754 2559 generic.go:356] "Generic (PLEG): container finished" podID="7decb9cf-de84-4357-ae0b-a1530031a3af" containerID="f23556fc4f0c9b1ba45458787a26a9356df40cbfc5fb64b2b465850ad49d9373" exitCode=0 Jul 12 17:59:43.836226 ip-10-0-129-128 kubenswrapper[2559]: I0712 17:59:43.835865 2559 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/istiod-openshift-gateway-7574459656-6zg5p" event={"ID":"7decb9cf-de84-4357-ae0b-a1530031a3af","Type":"ContainerDied","Data":"f23556fc4f0c9b1ba45458787a26a9356df40cbfc5fb64b2b465850ad49d9373"} Jul 12 17:59:43.836226 ip-10-0-129-128 kubenswrapper[2559]: I0712 17:59:43.835910 2559 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/istiod-openshift-gateway-7574459656-6zg5p" event={"ID":"7decb9cf-de84-4357-ae0b-a1530031a3af","Type":"ContainerDied","Data":"d3c545ec9e2d3fd347bbb4864096bf03f0c0af73acf837e9078f8aaed45cedeb"} Jul 12 17:59:43.836226 ip-10-0-129-128 kubenswrapper[2559]: I0712 17:59:43.835937 2559 scope.go:117] "RemoveContainer" containerID="f23556fc4f0c9b1ba45458787a26a9356df40cbfc5fb64b2b465850ad49d9373" Jul 12 17:59:43.836226 ip-10-0-129-128 kubenswrapper[2559]: I0712 17:59:43.836080 2559 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/istiod-openshift-gateway-7574459656-6zg5p" Jul 12 17:59:43.841893 ip-10-0-129-128 kubenswrapper[2559]: I0712 17:59:43.841547 2559 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="opendatahub/kserve-controller-manager-d6f4b65b8-zm6bp" event={"ID":"402923e6-af3f-48fc-ae8e-f6cd828a0e73","Type":"ContainerStarted","Data":"76bbb4db435d025a718422b53d99b90a97b5ef506b3ed5be4542ffc9811ff6ac"} Jul 12 17:59:43.841893 ip-10-0-129-128 kubenswrapper[2559]: I0712 17:59:43.841605 2559 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/data-science-gateway-data-science-gateway-class-6576859cf49q7gm" Jul 12 17:59:43.843060 ip-10-0-129-128 kubenswrapper[2559]: I0712 17:59:43.843034 2559 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/data-science-gateway-data-science-gateway-class-6576859cf49q7gm" Jul 12 17:59:43.849699 ip-10-0-129-128 kubenswrapper[2559]: I0712 17:59:43.849679 2559 scope.go:117] "RemoveContainer" containerID="f23556fc4f0c9b1ba45458787a26a9356df40cbfc5fb64b2b465850ad49d9373" Jul 12 17:59:43.849943 ip-10-0-129-128 kubenswrapper[2559]: E0712 17:59:43.849922 2559 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"f23556fc4f0c9b1ba45458787a26a9356df40cbfc5fb64b2b465850ad49d9373\": container with ID starting with f23556fc4f0c9b1ba45458787a26a9356df40cbfc5fb64b2b465850ad49d9373 not found: ID does not exist" containerID="f23556fc4f0c9b1ba45458787a26a9356df40cbfc5fb64b2b465850ad49d9373" Jul 12 17:59:43.850001 ip-10-0-129-128 kubenswrapper[2559]: I0712 17:59:43.849955 2559 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"f23556fc4f0c9b1ba45458787a26a9356df40cbfc5fb64b2b465850ad49d9373"} err="failed to get container status \"f23556fc4f0c9b1ba45458787a26a9356df40cbfc5fb64b2b465850ad49d9373\": rpc error: code = NotFound desc = could not find container \"f23556fc4f0c9b1ba45458787a26a9356df40cbfc5fb64b2b465850ad49d9373\": container with ID starting with f23556fc4f0c9b1ba45458787a26a9356df40cbfc5fb64b2b465850ad49d9373 not found: ID does not exist" Jul 12 17:59:43.854203 ip-10-0-129-128 kubenswrapper[2559]: I0712 17:59:43.854186 2559 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"local-certs\" (UniqueName: \"kubernetes.io/empty-dir/7decb9cf-de84-4357-ae0b-a1530031a3af-local-certs\") pod \"7decb9cf-de84-4357-ae0b-a1530031a3af\" (UID: \"7decb9cf-de84-4357-ae0b-a1530031a3af\") " Jul 12 17:59:43.854269 ip-10-0-129-128 kubenswrapper[2559]: I0712 17:59:43.854253 2559 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"cacerts\" (UniqueName: \"kubernetes.io/secret/7decb9cf-de84-4357-ae0b-a1530031a3af-cacerts\") pod \"7decb9cf-de84-4357-ae0b-a1530031a3af\" (UID: \"7decb9cf-de84-4357-ae0b-a1530031a3af\") " Jul 12 17:59:43.854325 ip-10-0-129-128 kubenswrapper[2559]: I0712 17:59:43.854312 2559 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/7decb9cf-de84-4357-ae0b-a1530031a3af-istio-token\") pod \"7decb9cf-de84-4357-ae0b-a1530031a3af\" (UID: \"7decb9cf-de84-4357-ae0b-a1530031a3af\") " Jul 12 17:59:43.854361 ip-10-0-129-128 kubenswrapper[2559]: I0712 17:59:43.854348 2559 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istio-csr-ca-configmap\" (UniqueName: \"kubernetes.io/configmap/7decb9cf-de84-4357-ae0b-a1530031a3af-istio-csr-ca-configmap\") pod \"7decb9cf-de84-4357-ae0b-a1530031a3af\" (UID: \"7decb9cf-de84-4357-ae0b-a1530031a3af\") " Jul 12 17:59:43.854392 ip-10-0-129-128 kubenswrapper[2559]: I0712 17:59:43.854376 2559 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istio-csr-dns-cert\" (UniqueName: \"kubernetes.io/secret/7decb9cf-de84-4357-ae0b-a1530031a3af-istio-csr-dns-cert\") pod \"7decb9cf-de84-4357-ae0b-a1530031a3af\" (UID: \"7decb9cf-de84-4357-ae0b-a1530031a3af\") " Jul 12 17:59:43.854439 ip-10-0-129-128 kubenswrapper[2559]: I0712 17:59:43.854407 2559 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istio-kubeconfig\" (UniqueName: \"kubernetes.io/secret/7decb9cf-de84-4357-ae0b-a1530031a3af-istio-kubeconfig\") pod \"7decb9cf-de84-4357-ae0b-a1530031a3af\" (UID: \"7decb9cf-de84-4357-ae0b-a1530031a3af\") " Jul 12 17:59:43.854595 ip-10-0-129-128 kubenswrapper[2559]: I0712 17:59:43.854568 2559 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-8sgkz\" (UniqueName: \"kubernetes.io/projected/7decb9cf-de84-4357-ae0b-a1530031a3af-kube-api-access-8sgkz\") pod \"7decb9cf-de84-4357-ae0b-a1530031a3af\" (UID: \"7decb9cf-de84-4357-ae0b-a1530031a3af\") " Jul 12 17:59:43.854847 ip-10-0-129-128 kubenswrapper[2559]: I0712 17:59:43.854818 2559 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/7decb9cf-de84-4357-ae0b-a1530031a3af-istio-csr-ca-configmap" (OuterVolumeSpecName: "istio-csr-ca-configmap") pod "7decb9cf-de84-4357-ae0b-a1530031a3af" (UID: "7decb9cf-de84-4357-ae0b-a1530031a3af"). InnerVolumeSpecName "istio-csr-ca-configmap". PluginName "kubernetes.io/configmap", VolumeGIDValue "" Jul 12 17:59:43.854937 ip-10-0-129-128 kubenswrapper[2559]: I0712 17:59:43.854918 2559 reconciler_common.go:299] "Volume detached for volume \"istio-csr-ca-configmap\" (UniqueName: \"kubernetes.io/configmap/7decb9cf-de84-4357-ae0b-a1530031a3af-istio-csr-ca-configmap\") on node \"ip-10-0-129-128.ec2.internal\" DevicePath \"\"" Jul 12 17:59:43.857256 ip-10-0-129-128 kubenswrapper[2559]: I0712 17:59:43.857226 2559 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/7decb9cf-de84-4357-ae0b-a1530031a3af-istio-kubeconfig" (OuterVolumeSpecName: "istio-kubeconfig") pod "7decb9cf-de84-4357-ae0b-a1530031a3af" (UID: "7decb9cf-de84-4357-ae0b-a1530031a3af"). InnerVolumeSpecName "istio-kubeconfig". PluginName "kubernetes.io/secret", VolumeGIDValue "" Jul 12 17:59:43.857256 ip-10-0-129-128 kubenswrapper[2559]: I0712 17:59:43.857235 2559 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/7decb9cf-de84-4357-ae0b-a1530031a3af-local-certs" (OuterVolumeSpecName: "local-certs") pod "7decb9cf-de84-4357-ae0b-a1530031a3af" (UID: "7decb9cf-de84-4357-ae0b-a1530031a3af"). InnerVolumeSpecName "local-certs". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Jul 12 17:59:43.857552 ip-10-0-129-128 kubenswrapper[2559]: I0712 17:59:43.857535 2559 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/7decb9cf-de84-4357-ae0b-a1530031a3af-cacerts" (OuterVolumeSpecName: "cacerts") pod "7decb9cf-de84-4357-ae0b-a1530031a3af" (UID: "7decb9cf-de84-4357-ae0b-a1530031a3af"). InnerVolumeSpecName "cacerts". PluginName "kubernetes.io/secret", VolumeGIDValue "" Jul 12 17:59:43.857627 ip-10-0-129-128 kubenswrapper[2559]: I0712 17:59:43.857568 2559 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/7decb9cf-de84-4357-ae0b-a1530031a3af-kube-api-access-8sgkz" (OuterVolumeSpecName: "kube-api-access-8sgkz") pod "7decb9cf-de84-4357-ae0b-a1530031a3af" (UID: "7decb9cf-de84-4357-ae0b-a1530031a3af"). InnerVolumeSpecName "kube-api-access-8sgkz". PluginName "kubernetes.io/projected", VolumeGIDValue "" Jul 12 17:59:43.857994 ip-10-0-129-128 kubenswrapper[2559]: I0712 17:59:43.857951 2559 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/7decb9cf-de84-4357-ae0b-a1530031a3af-istio-csr-dns-cert" (OuterVolumeSpecName: "istio-csr-dns-cert") pod "7decb9cf-de84-4357-ae0b-a1530031a3af" (UID: "7decb9cf-de84-4357-ae0b-a1530031a3af"). InnerVolumeSpecName "istio-csr-dns-cert". PluginName "kubernetes.io/secret", VolumeGIDValue "" Jul 12 17:59:43.857994 ip-10-0-129-128 kubenswrapper[2559]: I0712 17:59:43.857968 2559 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/7decb9cf-de84-4357-ae0b-a1530031a3af-istio-token" (OuterVolumeSpecName: "istio-token") pod "7decb9cf-de84-4357-ae0b-a1530031a3af" (UID: "7decb9cf-de84-4357-ae0b-a1530031a3af"). InnerVolumeSpecName "istio-token". PluginName "kubernetes.io/projected", VolumeGIDValue "" Jul 12 17:59:43.955541 ip-10-0-129-128 kubenswrapper[2559]: I0712 17:59:43.955498 2559 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-8sgkz\" (UniqueName: \"kubernetes.io/projected/7decb9cf-de84-4357-ae0b-a1530031a3af-kube-api-access-8sgkz\") on node \"ip-10-0-129-128.ec2.internal\" DevicePath \"\"" Jul 12 17:59:43.955541 ip-10-0-129-128 kubenswrapper[2559]: I0712 17:59:43.955537 2559 reconciler_common.go:299] "Volume detached for volume \"local-certs\" (UniqueName: \"kubernetes.io/empty-dir/7decb9cf-de84-4357-ae0b-a1530031a3af-local-certs\") on node \"ip-10-0-129-128.ec2.internal\" DevicePath \"\"" Jul 12 17:59:43.955541 ip-10-0-129-128 kubenswrapper[2559]: I0712 17:59:43.955553 2559 reconciler_common.go:299] "Volume detached for volume \"cacerts\" (UniqueName: \"kubernetes.io/secret/7decb9cf-de84-4357-ae0b-a1530031a3af-cacerts\") on node \"ip-10-0-129-128.ec2.internal\" DevicePath \"\"" Jul 12 17:59:43.955880 ip-10-0-129-128 kubenswrapper[2559]: I0712 17:59:43.955565 2559 reconciler_common.go:299] "Volume detached for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/7decb9cf-de84-4357-ae0b-a1530031a3af-istio-token\") on node \"ip-10-0-129-128.ec2.internal\" DevicePath \"\"" Jul 12 17:59:43.955880 ip-10-0-129-128 kubenswrapper[2559]: I0712 17:59:43.955580 2559 reconciler_common.go:299] "Volume detached for volume \"istio-csr-dns-cert\" (UniqueName: \"kubernetes.io/secret/7decb9cf-de84-4357-ae0b-a1530031a3af-istio-csr-dns-cert\") on node \"ip-10-0-129-128.ec2.internal\" DevicePath \"\"" Jul 12 17:59:43.955880 ip-10-0-129-128 kubenswrapper[2559]: I0712 17:59:43.955592 2559 reconciler_common.go:299] "Volume detached for volume \"istio-kubeconfig\" (UniqueName: \"kubernetes.io/secret/7decb9cf-de84-4357-ae0b-a1530031a3af-istio-kubeconfig\") on node \"ip-10-0-129-128.ec2.internal\" DevicePath \"\"" Jul 12 17:59:44.161348 ip-10-0-129-128 kubenswrapper[2559]: I0712 17:59:44.161316 2559 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-ingress/istiod-openshift-gateway-7574459656-6zg5p"] Jul 12 17:59:44.168237 ip-10-0-129-128 kubenswrapper[2559]: I0712 17:59:44.168204 2559 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["openshift-ingress/istiod-openshift-gateway-7574459656-6zg5p"] Jul 12 17:59:45.037068 ip-10-0-129-128 kubenswrapper[2559]: I0712 17:59:45.037029 2559 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="7decb9cf-de84-4357-ae0b-a1530031a3af" path="/var/lib/kubelet/pods/7decb9cf-de84-4357-ae0b-a1530031a3af/volumes" Jul 12 17:59:45.853323 ip-10-0-129-128 kubenswrapper[2559]: I0712 17:59:45.853288 2559 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="opendatahub/kserve-controller-manager-d6f4b65b8-zm6bp" event={"ID":"402923e6-af3f-48fc-ae8e-f6cd828a0e73","Type":"ContainerStarted","Data":"0584c80f01857b1d127339be7fd0fc98afcef032e5b8db0374830abe799d9e5e"} Jul 12 17:59:45.853812 ip-10-0-129-128 kubenswrapper[2559]: I0712 17:59:45.853365 2559 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="opendatahub/kserve-controller-manager-d6f4b65b8-zm6bp" Jul 12 17:59:45.870090 ip-10-0-129-128 kubenswrapper[2559]: I0712 17:59:45.870043 2559 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="opendatahub/kserve-controller-manager-d6f4b65b8-zm6bp" podStartSLOduration=2.579096217 podStartE2EDuration="4.870028661s" totalImagesPullingTime="2.290932444s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-12 17:59:41 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-12 17:59:45.868760529 +0000 UTC m=+393.362828460" watchObservedRunningTime="2026-07-12 17:59:45.870028661 +0000 UTC m=+393.364096591" Jul 12 17:59:49.810366 ip-10-0-129-128 kubenswrapper[2559]: I0712 17:59:49.810334 2559 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="opendatahub/odh-model-controller-7998c67dc4-tdm5v" Jul 12 17:59:49.812300 ip-10-0-129-128 kubenswrapper[2559]: I0712 17:59:49.812280 2559 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="opendatahub/model-serving-api-54d89bd5fc-c46wr" Jul 12 18:00:16.862887 ip-10-0-129-128 kubenswrapper[2559]: I0712 18:00:16.862855 2559 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="opendatahub/kserve-controller-manager-d6f4b65b8-zm6bp" Jul 12 18:00:33.067437 ip-10-0-129-128 kubenswrapper[2559]: I0712 18:00:33.067403 2559 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["kuadrant-system/kuadrant-operator-catalog-pdq46"] Jul 12 18:00:33.067831 ip-10-0-129-128 kubenswrapper[2559]: I0712 18:00:33.067710 2559 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="7decb9cf-de84-4357-ae0b-a1530031a3af" containerName="discovery" Jul 12 18:00:33.067831 ip-10-0-129-128 kubenswrapper[2559]: I0712 18:00:33.067721 2559 state_mem.go:107] "Deleted CPUSet assignment" podUID="7decb9cf-de84-4357-ae0b-a1530031a3af" containerName="discovery" Jul 12 18:00:33.067831 ip-10-0-129-128 kubenswrapper[2559]: I0712 18:00:33.067785 2559 memory_manager.go:368] "RemoveStaleState removing state" podUID="7decb9cf-de84-4357-ae0b-a1530031a3af" containerName="discovery" Jul 12 18:00:33.077814 ip-10-0-129-128 kubenswrapper[2559]: I0712 18:00:33.077789 2559 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/kuadrant-operator-catalog-pdq46" Jul 12 18:00:33.080543 ip-10-0-129-128 kubenswrapper[2559]: I0712 18:00:33.080498 2559 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"kuadrant-system\"/\"kube-root-ca.crt\"" Jul 12 18:00:33.081067 ip-10-0-129-128 kubenswrapper[2559]: I0712 18:00:33.081042 2559 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/kuadrant-operator-catalog-pdq46"] Jul 12 18:00:33.081249 ip-10-0-129-128 kubenswrapper[2559]: I0712 18:00:33.081120 2559 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"kuadrant-system\"/\"kuadrant-operator-catalog-dockercfg-nkxfm\"" Jul 12 18:00:33.081249 ip-10-0-129-128 kubenswrapper[2559]: I0712 18:00:33.081122 2559 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"kuadrant-system\"/\"openshift-service-ca.crt\"" Jul 12 18:00:33.161919 ip-10-0-129-128 kubenswrapper[2559]: I0712 18:00:33.161883 2559 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-gbrz8\" (UniqueName: \"kubernetes.io/projected/67938a8a-5688-4459-9d5c-f3c232fc5bb6-kube-api-access-gbrz8\") pod \"kuadrant-operator-catalog-pdq46\" (UID: \"67938a8a-5688-4459-9d5c-f3c232fc5bb6\") " pod="kuadrant-system/kuadrant-operator-catalog-pdq46" Jul 12 18:00:33.262533 ip-10-0-129-128 kubenswrapper[2559]: I0712 18:00:33.262490 2559 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-gbrz8\" (UniqueName: \"kubernetes.io/projected/67938a8a-5688-4459-9d5c-f3c232fc5bb6-kube-api-access-gbrz8\") pod \"kuadrant-operator-catalog-pdq46\" (UID: \"67938a8a-5688-4459-9d5c-f3c232fc5bb6\") " pod="kuadrant-system/kuadrant-operator-catalog-pdq46" Jul 12 18:00:33.270785 ip-10-0-129-128 kubenswrapper[2559]: I0712 18:00:33.270754 2559 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-gbrz8\" (UniqueName: \"kubernetes.io/projected/67938a8a-5688-4459-9d5c-f3c232fc5bb6-kube-api-access-gbrz8\") pod \"kuadrant-operator-catalog-pdq46\" (UID: \"67938a8a-5688-4459-9d5c-f3c232fc5bb6\") " pod="kuadrant-system/kuadrant-operator-catalog-pdq46" Jul 12 18:00:33.388879 ip-10-0-129-128 kubenswrapper[2559]: I0712 18:00:33.388795 2559 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/kuadrant-operator-catalog-pdq46" Jul 12 18:00:33.430774 ip-10-0-129-128 kubenswrapper[2559]: I0712 18:00:33.430688 2559 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["kuadrant-system/kuadrant-operator-catalog-pdq46"] Jul 12 18:00:33.522117 ip-10-0-129-128 kubenswrapper[2559]: I0712 18:00:33.522085 2559 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["kuadrant-system/kuadrant-operator-catalog-pdq46"] Jul 12 18:00:33.525783 ip-10-0-129-128 kubenswrapper[2559]: W0712 18:00:33.525753 2559 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod67938a8a_5688_4459_9d5c_f3c232fc5bb6.slice/crio-c912a383cfaaead9e67782cf0a9f0776854494c82d43ec0bf277c5afec38da9d WatchSource:0}: Error finding container c912a383cfaaead9e67782cf0a9f0776854494c82d43ec0bf277c5afec38da9d: Status 404 returned error can't find the container with id c912a383cfaaead9e67782cf0a9f0776854494c82d43ec0bf277c5afec38da9d Jul 12 18:00:33.641028 ip-10-0-129-128 kubenswrapper[2559]: I0712 18:00:33.640936 2559 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["kuadrant-system/kuadrant-operator-catalog-gnh7d"] Jul 12 18:00:33.643713 ip-10-0-129-128 kubenswrapper[2559]: I0712 18:00:33.643688 2559 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/kuadrant-operator-catalog-gnh7d" Jul 12 18:00:33.652050 ip-10-0-129-128 kubenswrapper[2559]: I0712 18:00:33.652019 2559 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/kuadrant-operator-catalog-gnh7d"] Jul 12 18:00:33.666540 ip-10-0-129-128 kubenswrapper[2559]: I0712 18:00:33.666507 2559 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-cgmrh\" (UniqueName: \"kubernetes.io/projected/c03827ed-4ed1-49b1-9ff7-0e20fbeb0722-kube-api-access-cgmrh\") pod \"kuadrant-operator-catalog-gnh7d\" (UID: \"c03827ed-4ed1-49b1-9ff7-0e20fbeb0722\") " pod="kuadrant-system/kuadrant-operator-catalog-gnh7d" Jul 12 18:00:33.767833 ip-10-0-129-128 kubenswrapper[2559]: I0712 18:00:33.767793 2559 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-cgmrh\" (UniqueName: \"kubernetes.io/projected/c03827ed-4ed1-49b1-9ff7-0e20fbeb0722-kube-api-access-cgmrh\") pod \"kuadrant-operator-catalog-gnh7d\" (UID: \"c03827ed-4ed1-49b1-9ff7-0e20fbeb0722\") " pod="kuadrant-system/kuadrant-operator-catalog-gnh7d" Jul 12 18:00:33.779299 ip-10-0-129-128 kubenswrapper[2559]: I0712 18:00:33.779277 2559 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-cgmrh\" (UniqueName: \"kubernetes.io/projected/c03827ed-4ed1-49b1-9ff7-0e20fbeb0722-kube-api-access-cgmrh\") pod \"kuadrant-operator-catalog-gnh7d\" (UID: \"c03827ed-4ed1-49b1-9ff7-0e20fbeb0722\") " pod="kuadrant-system/kuadrant-operator-catalog-gnh7d" Jul 12 18:00:33.953689 ip-10-0-129-128 kubenswrapper[2559]: I0712 18:00:33.953586 2559 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/kuadrant-operator-catalog-gnh7d" Jul 12 18:00:34.058947 ip-10-0-129-128 kubenswrapper[2559]: I0712 18:00:34.058870 2559 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/kuadrant-operator-catalog-pdq46" event={"ID":"67938a8a-5688-4459-9d5c-f3c232fc5bb6","Type":"ContainerStarted","Data":"c912a383cfaaead9e67782cf0a9f0776854494c82d43ec0bf277c5afec38da9d"} Jul 12 18:00:34.088592 ip-10-0-129-128 kubenswrapper[2559]: I0712 18:00:34.088564 2559 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/kuadrant-operator-catalog-gnh7d"] Jul 12 18:00:34.114298 ip-10-0-129-128 kubenswrapper[2559]: W0712 18:00:34.114263 2559 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podc03827ed_4ed1_49b1_9ff7_0e20fbeb0722.slice/crio-d1f3728ee1fde5256de47d3187162f94b0a709d2c04acfd7446890d470cbc973 WatchSource:0}: Error finding container d1f3728ee1fde5256de47d3187162f94b0a709d2c04acfd7446890d470cbc973: Status 404 returned error can't find the container with id d1f3728ee1fde5256de47d3187162f94b0a709d2c04acfd7446890d470cbc973 Jul 12 18:00:35.065712 ip-10-0-129-128 kubenswrapper[2559]: I0712 18:00:35.065677 2559 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/kuadrant-operator-catalog-gnh7d" event={"ID":"c03827ed-4ed1-49b1-9ff7-0e20fbeb0722","Type":"ContainerStarted","Data":"d1f3728ee1fde5256de47d3187162f94b0a709d2c04acfd7446890d470cbc973"} Jul 12 18:00:36.071406 ip-10-0-129-128 kubenswrapper[2559]: I0712 18:00:36.071349 2559 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/kuadrant-operator-catalog-pdq46" event={"ID":"67938a8a-5688-4459-9d5c-f3c232fc5bb6","Type":"ContainerStarted","Data":"4ba3ad9770b5f17a9fcf2dbc058bbf385904ba668dcfde12d5b52a265a634212"} Jul 12 18:00:36.071406 ip-10-0-129-128 kubenswrapper[2559]: I0712 18:00:36.071368 2559 kuberuntime_container.go:873] "Killing container with a grace period" pod="kuadrant-system/kuadrant-operator-catalog-pdq46" podUID="67938a8a-5688-4459-9d5c-f3c232fc5bb6" containerName="registry-server" containerID="cri-o://4ba3ad9770b5f17a9fcf2dbc058bbf385904ba668dcfde12d5b52a265a634212" gracePeriod=2 Jul 12 18:00:36.072913 ip-10-0-129-128 kubenswrapper[2559]: I0712 18:00:36.072878 2559 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/kuadrant-operator-catalog-gnh7d" event={"ID":"c03827ed-4ed1-49b1-9ff7-0e20fbeb0722","Type":"ContainerStarted","Data":"94aa5240e44063999a6ceb4a416437962bdc5c8ed9640ada0b984d45e15ac85c"} Jul 12 18:00:36.086627 ip-10-0-129-128 kubenswrapper[2559]: I0712 18:00:36.086584 2559 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="kuadrant-system/kuadrant-operator-catalog-pdq46" podStartSLOduration=0.960575056 podStartE2EDuration="3.086568609s" totalImagesPullingTime="2.125993553s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-12 18:00:33 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-12 18:00:36.085414896 +0000 UTC m=+443.579482839" watchObservedRunningTime="2026-07-12 18:00:36.086568609 +0000 UTC m=+443.580636553" Jul 12 18:00:36.099764 ip-10-0-129-128 kubenswrapper[2559]: I0712 18:00:36.099715 2559 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="kuadrant-system/kuadrant-operator-catalog-gnh7d" podStartSLOduration=1.552834016 podStartE2EDuration="3.099698552s" totalImagesPullingTime="1.546864536s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-12 18:00:33 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-12 18:00:36.098768753 +0000 UTC m=+443.592836683" watchObservedRunningTime="2026-07-12 18:00:36.099698552 +0000 UTC m=+443.593766482" Jul 12 18:00:36.311698 ip-10-0-129-128 kubenswrapper[2559]: I0712 18:00:36.311674 2559 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/kuadrant-operator-catalog-pdq46" Jul 12 18:00:36.395734 ip-10-0-129-128 kubenswrapper[2559]: I0712 18:00:36.395631 2559 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-gbrz8\" (UniqueName: \"kubernetes.io/projected/67938a8a-5688-4459-9d5c-f3c232fc5bb6-kube-api-access-gbrz8\") pod \"67938a8a-5688-4459-9d5c-f3c232fc5bb6\" (UID: \"67938a8a-5688-4459-9d5c-f3c232fc5bb6\") " Jul 12 18:00:36.397776 ip-10-0-129-128 kubenswrapper[2559]: I0712 18:00:36.397749 2559 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/67938a8a-5688-4459-9d5c-f3c232fc5bb6-kube-api-access-gbrz8" (OuterVolumeSpecName: "kube-api-access-gbrz8") pod "67938a8a-5688-4459-9d5c-f3c232fc5bb6" (UID: "67938a8a-5688-4459-9d5c-f3c232fc5bb6"). InnerVolumeSpecName "kube-api-access-gbrz8". PluginName "kubernetes.io/projected", VolumeGIDValue "" Jul 12 18:00:36.496441 ip-10-0-129-128 kubenswrapper[2559]: I0712 18:00:36.496397 2559 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-gbrz8\" (UniqueName: \"kubernetes.io/projected/67938a8a-5688-4459-9d5c-f3c232fc5bb6-kube-api-access-gbrz8\") on node \"ip-10-0-129-128.ec2.internal\" DevicePath \"\"" Jul 12 18:00:37.078236 ip-10-0-129-128 kubenswrapper[2559]: I0712 18:00:37.078200 2559 generic.go:356] "Generic (PLEG): container finished" podID="67938a8a-5688-4459-9d5c-f3c232fc5bb6" containerID="4ba3ad9770b5f17a9fcf2dbc058bbf385904ba668dcfde12d5b52a265a634212" exitCode=0 Jul 12 18:00:37.078737 ip-10-0-129-128 kubenswrapper[2559]: I0712 18:00:37.078250 2559 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/kuadrant-operator-catalog-pdq46" Jul 12 18:00:37.078737 ip-10-0-129-128 kubenswrapper[2559]: I0712 18:00:37.078289 2559 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/kuadrant-operator-catalog-pdq46" event={"ID":"67938a8a-5688-4459-9d5c-f3c232fc5bb6","Type":"ContainerDied","Data":"4ba3ad9770b5f17a9fcf2dbc058bbf385904ba668dcfde12d5b52a265a634212"} Jul 12 18:00:37.078737 ip-10-0-129-128 kubenswrapper[2559]: I0712 18:00:37.078334 2559 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/kuadrant-operator-catalog-pdq46" event={"ID":"67938a8a-5688-4459-9d5c-f3c232fc5bb6","Type":"ContainerDied","Data":"c912a383cfaaead9e67782cf0a9f0776854494c82d43ec0bf277c5afec38da9d"} Jul 12 18:00:37.078737 ip-10-0-129-128 kubenswrapper[2559]: I0712 18:00:37.078353 2559 scope.go:117] "RemoveContainer" containerID="4ba3ad9770b5f17a9fcf2dbc058bbf385904ba668dcfde12d5b52a265a634212" Jul 12 18:00:37.087059 ip-10-0-129-128 kubenswrapper[2559]: I0712 18:00:37.087041 2559 scope.go:117] "RemoveContainer" containerID="4ba3ad9770b5f17a9fcf2dbc058bbf385904ba668dcfde12d5b52a265a634212" Jul 12 18:00:37.087360 ip-10-0-129-128 kubenswrapper[2559]: E0712 18:00:37.087337 2559 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"4ba3ad9770b5f17a9fcf2dbc058bbf385904ba668dcfde12d5b52a265a634212\": container with ID starting with 4ba3ad9770b5f17a9fcf2dbc058bbf385904ba668dcfde12d5b52a265a634212 not found: ID does not exist" containerID="4ba3ad9770b5f17a9fcf2dbc058bbf385904ba668dcfde12d5b52a265a634212" Jul 12 18:00:37.087416 ip-10-0-129-128 kubenswrapper[2559]: I0712 18:00:37.087363 2559 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"4ba3ad9770b5f17a9fcf2dbc058bbf385904ba668dcfde12d5b52a265a634212"} err="failed to get container status \"4ba3ad9770b5f17a9fcf2dbc058bbf385904ba668dcfde12d5b52a265a634212\": rpc error: code = NotFound desc = could not find container \"4ba3ad9770b5f17a9fcf2dbc058bbf385904ba668dcfde12d5b52a265a634212\": container with ID starting with 4ba3ad9770b5f17a9fcf2dbc058bbf385904ba668dcfde12d5b52a265a634212 not found: ID does not exist" Jul 12 18:00:37.095815 ip-10-0-129-128 kubenswrapper[2559]: I0712 18:00:37.095793 2559 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["kuadrant-system/kuadrant-operator-catalog-pdq46"] Jul 12 18:00:37.099705 ip-10-0-129-128 kubenswrapper[2559]: I0712 18:00:37.099683 2559 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["kuadrant-system/kuadrant-operator-catalog-pdq46"] Jul 12 18:00:39.033550 ip-10-0-129-128 kubenswrapper[2559]: I0712 18:00:39.033515 2559 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="67938a8a-5688-4459-9d5c-f3c232fc5bb6" path="/var/lib/kubelet/pods/67938a8a-5688-4459-9d5c-f3c232fc5bb6/volumes" Jul 12 18:00:43.954626 ip-10-0-129-128 kubenswrapper[2559]: I0712 18:00:43.954589 2559 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="kuadrant-system/kuadrant-operator-catalog-gnh7d" Jul 12 18:00:43.955235 ip-10-0-129-128 kubenswrapper[2559]: I0712 18:00:43.954700 2559 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="kuadrant-system/kuadrant-operator-catalog-gnh7d" Jul 12 18:00:43.977598 ip-10-0-129-128 kubenswrapper[2559]: I0712 18:00:43.977572 2559 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="kuadrant-system/kuadrant-operator-catalog-gnh7d" Jul 12 18:00:44.130291 ip-10-0-129-128 kubenswrapper[2559]: I0712 18:00:44.130266 2559 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="kuadrant-system/kuadrant-operator-catalog-gnh7d" Jul 12 18:01:05.187267 ip-10-0-129-128 kubenswrapper[2559]: I0712 18:01:05.187228 2559 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["kuadrant-system/dns-operator-controller-manager-864dd78c4c-zwc7p"] Jul 12 18:01:05.187661 ip-10-0-129-128 kubenswrapper[2559]: I0712 18:01:05.187574 2559 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="67938a8a-5688-4459-9d5c-f3c232fc5bb6" containerName="registry-server" Jul 12 18:01:05.187661 ip-10-0-129-128 kubenswrapper[2559]: I0712 18:01:05.187586 2559 state_mem.go:107] "Deleted CPUSet assignment" podUID="67938a8a-5688-4459-9d5c-f3c232fc5bb6" containerName="registry-server" Jul 12 18:01:05.187661 ip-10-0-129-128 kubenswrapper[2559]: I0712 18:01:05.187635 2559 memory_manager.go:368] "RemoveStaleState removing state" podUID="67938a8a-5688-4459-9d5c-f3c232fc5bb6" containerName="registry-server" Jul 12 18:01:05.191778 ip-10-0-129-128 kubenswrapper[2559]: I0712 18:01:05.191758 2559 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/dns-operator-controller-manager-864dd78c4c-zwc7p" Jul 12 18:01:05.197537 ip-10-0-129-128 kubenswrapper[2559]: I0712 18:01:05.197511 2559 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"kuadrant-system\"/\"dns-operator-controller-manager-dockercfg-r662s\"" Jul 12 18:01:05.197681 ip-10-0-129-128 kubenswrapper[2559]: I0712 18:01:05.197547 2559 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"kuadrant-system\"/\"dns-operator-controller-env\"" Jul 12 18:01:05.206933 ip-10-0-129-128 kubenswrapper[2559]: I0712 18:01:05.206901 2559 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/dns-operator-controller-manager-864dd78c4c-zwc7p"] Jul 12 18:01:05.344566 ip-10-0-129-128 kubenswrapper[2559]: I0712 18:01:05.344532 2559 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-pjfv9\" (UniqueName: \"kubernetes.io/projected/e8c1fa35-12cf-4ac7-8f0d-0413e0eba07f-kube-api-access-pjfv9\") pod \"dns-operator-controller-manager-864dd78c4c-zwc7p\" (UID: \"e8c1fa35-12cf-4ac7-8f0d-0413e0eba07f\") " pod="kuadrant-system/dns-operator-controller-manager-864dd78c4c-zwc7p" Jul 12 18:01:05.445227 ip-10-0-129-128 kubenswrapper[2559]: I0712 18:01:05.445139 2559 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-pjfv9\" (UniqueName: \"kubernetes.io/projected/e8c1fa35-12cf-4ac7-8f0d-0413e0eba07f-kube-api-access-pjfv9\") pod \"dns-operator-controller-manager-864dd78c4c-zwc7p\" (UID: \"e8c1fa35-12cf-4ac7-8f0d-0413e0eba07f\") " pod="kuadrant-system/dns-operator-controller-manager-864dd78c4c-zwc7p" Jul 12 18:01:05.456598 ip-10-0-129-128 kubenswrapper[2559]: I0712 18:01:05.456560 2559 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-pjfv9\" (UniqueName: \"kubernetes.io/projected/e8c1fa35-12cf-4ac7-8f0d-0413e0eba07f-kube-api-access-pjfv9\") pod \"dns-operator-controller-manager-864dd78c4c-zwc7p\" (UID: \"e8c1fa35-12cf-4ac7-8f0d-0413e0eba07f\") " pod="kuadrant-system/dns-operator-controller-manager-864dd78c4c-zwc7p" Jul 12 18:01:05.502338 ip-10-0-129-128 kubenswrapper[2559]: I0712 18:01:05.502298 2559 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/dns-operator-controller-manager-864dd78c4c-zwc7p" Jul 12 18:01:05.635562 ip-10-0-129-128 kubenswrapper[2559]: I0712 18:01:05.635539 2559 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/dns-operator-controller-manager-864dd78c4c-zwc7p"] Jul 12 18:01:05.638068 ip-10-0-129-128 kubenswrapper[2559]: W0712 18:01:05.638042 2559 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pode8c1fa35_12cf_4ac7_8f0d_0413e0eba07f.slice/crio-0331e5cd7f01079dd85ec623af80723b5808e8862f3910dff1d40d46c12f74a3 WatchSource:0}: Error finding container 0331e5cd7f01079dd85ec623af80723b5808e8862f3910dff1d40d46c12f74a3: Status 404 returned error can't find the container with id 0331e5cd7f01079dd85ec623af80723b5808e8862f3910dff1d40d46c12f74a3 Jul 12 18:01:06.202461 ip-10-0-129-128 kubenswrapper[2559]: I0712 18:01:06.202430 2559 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/dns-operator-controller-manager-864dd78c4c-zwc7p" event={"ID":"e8c1fa35-12cf-4ac7-8f0d-0413e0eba07f","Type":"ContainerStarted","Data":"0331e5cd7f01079dd85ec623af80723b5808e8862f3910dff1d40d46c12f74a3"} Jul 12 18:01:08.214194 ip-10-0-129-128 kubenswrapper[2559]: I0712 18:01:08.214157 2559 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/dns-operator-controller-manager-864dd78c4c-zwc7p" event={"ID":"e8c1fa35-12cf-4ac7-8f0d-0413e0eba07f","Type":"ContainerStarted","Data":"9aa7e0c9ddc33f75b0c14fb7d1f888ce949f04a3273898cc352acb617d389f10"} Jul 12 18:01:08.214589 ip-10-0-129-128 kubenswrapper[2559]: I0712 18:01:08.214297 2559 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="kuadrant-system/dns-operator-controller-manager-864dd78c4c-zwc7p" Jul 12 18:01:08.231123 ip-10-0-129-128 kubenswrapper[2559]: I0712 18:01:08.231070 2559 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="kuadrant-system/dns-operator-controller-manager-864dd78c4c-zwc7p" podStartSLOduration=1.204340752 podStartE2EDuration="3.231054933s" totalImagesPullingTime="2.026714181s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-12 18:01:05 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-12 18:01:08.229566706 +0000 UTC m=+475.723634638" watchObservedRunningTime="2026-07-12 18:01:08.231054933 +0000 UTC m=+475.725122913" Jul 12 18:01:08.575306 ip-10-0-129-128 kubenswrapper[2559]: I0712 18:01:08.575268 2559 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["kuadrant-system/limitador-operator-controller-manager-859d7b78bd-cf5jh"] Jul 12 18:01:08.578738 ip-10-0-129-128 kubenswrapper[2559]: I0712 18:01:08.578719 2559 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/limitador-operator-controller-manager-859d7b78bd-cf5jh" Jul 12 18:01:08.581135 ip-10-0-129-128 kubenswrapper[2559]: I0712 18:01:08.581117 2559 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"kuadrant-system\"/\"limitador-operator-controller-manager-dockercfg-6595j\"" Jul 12 18:01:08.585860 ip-10-0-129-128 kubenswrapper[2559]: I0712 18:01:08.585837 2559 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-operator-controller-manager-859d7b78bd-cf5jh"] Jul 12 18:01:08.672372 ip-10-0-129-128 kubenswrapper[2559]: I0712 18:01:08.672337 2559 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-snwf5\" (UniqueName: \"kubernetes.io/projected/c737107f-73bd-4457-b865-dd7ea610b596-kube-api-access-snwf5\") pod \"limitador-operator-controller-manager-859d7b78bd-cf5jh\" (UID: \"c737107f-73bd-4457-b865-dd7ea610b596\") " pod="kuadrant-system/limitador-operator-controller-manager-859d7b78bd-cf5jh" Jul 12 18:01:08.772984 ip-10-0-129-128 kubenswrapper[2559]: I0712 18:01:08.772946 2559 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-snwf5\" (UniqueName: \"kubernetes.io/projected/c737107f-73bd-4457-b865-dd7ea610b596-kube-api-access-snwf5\") pod \"limitador-operator-controller-manager-859d7b78bd-cf5jh\" (UID: \"c737107f-73bd-4457-b865-dd7ea610b596\") " pod="kuadrant-system/limitador-operator-controller-manager-859d7b78bd-cf5jh" Jul 12 18:01:08.785139 ip-10-0-129-128 kubenswrapper[2559]: I0712 18:01:08.785117 2559 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-snwf5\" (UniqueName: \"kubernetes.io/projected/c737107f-73bd-4457-b865-dd7ea610b596-kube-api-access-snwf5\") pod \"limitador-operator-controller-manager-859d7b78bd-cf5jh\" (UID: \"c737107f-73bd-4457-b865-dd7ea610b596\") " pod="kuadrant-system/limitador-operator-controller-manager-859d7b78bd-cf5jh" Jul 12 18:01:08.889574 ip-10-0-129-128 kubenswrapper[2559]: I0712 18:01:08.889488 2559 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/limitador-operator-controller-manager-859d7b78bd-cf5jh" Jul 12 18:01:09.019338 ip-10-0-129-128 kubenswrapper[2559]: I0712 18:01:09.019312 2559 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-operator-controller-manager-859d7b78bd-cf5jh"] Jul 12 18:01:09.021996 ip-10-0-129-128 kubenswrapper[2559]: W0712 18:01:09.021969 2559 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podc737107f_73bd_4457_b865_dd7ea610b596.slice/crio-c55e62c937e129345d40ab096413b08faaabddd722bfaee3845d32aeaf34f416 WatchSource:0}: Error finding container c55e62c937e129345d40ab096413b08faaabddd722bfaee3845d32aeaf34f416: Status 404 returned error can't find the container with id c55e62c937e129345d40ab096413b08faaabddd722bfaee3845d32aeaf34f416 Jul 12 18:01:09.219629 ip-10-0-129-128 kubenswrapper[2559]: I0712 18:01:09.219535 2559 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/limitador-operator-controller-manager-859d7b78bd-cf5jh" event={"ID":"c737107f-73bd-4457-b865-dd7ea610b596","Type":"ContainerStarted","Data":"c55e62c937e129345d40ab096413b08faaabddd722bfaee3845d32aeaf34f416"} Jul 12 18:01:11.231128 ip-10-0-129-128 kubenswrapper[2559]: I0712 18:01:11.231092 2559 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/limitador-operator-controller-manager-859d7b78bd-cf5jh" event={"ID":"c737107f-73bd-4457-b865-dd7ea610b596","Type":"ContainerStarted","Data":"8a637af9431dec95bfd9125e8ca3795a8d3c1ce55cb33f24c89ecf5e194d28be"} Jul 12 18:01:11.231537 ip-10-0-129-128 kubenswrapper[2559]: I0712 18:01:11.231226 2559 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="kuadrant-system/limitador-operator-controller-manager-859d7b78bd-cf5jh" Jul 12 18:01:11.248614 ip-10-0-129-128 kubenswrapper[2559]: I0712 18:01:11.248565 2559 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="kuadrant-system/limitador-operator-controller-manager-859d7b78bd-cf5jh" podStartSLOduration=1.560063822 podStartE2EDuration="3.248547481s" totalImagesPullingTime="1.688483659s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-12 18:01:08 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-12 18:01:11.24786226 +0000 UTC m=+478.741930190" watchObservedRunningTime="2026-07-12 18:01:11.248547481 +0000 UTC m=+478.742615411" Jul 12 18:01:12.416206 ip-10-0-129-128 kubenswrapper[2559]: I0712 18:01:12.416171 2559 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["kuadrant-system/authorino-operator-75cc8d5cb7-85b85"] Jul 12 18:01:12.419740 ip-10-0-129-128 kubenswrapper[2559]: I0712 18:01:12.419721 2559 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/authorino-operator-75cc8d5cb7-85b85" Jul 12 18:01:12.422779 ip-10-0-129-128 kubenswrapper[2559]: I0712 18:01:12.422756 2559 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"kuadrant-system\"/\"authorino-operator-dockercfg-cdxzr\"" Jul 12 18:01:12.430357 ip-10-0-129-128 kubenswrapper[2559]: I0712 18:01:12.430335 2559 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/authorino-operator-75cc8d5cb7-85b85"] Jul 12 18:01:12.507094 ip-10-0-129-128 kubenswrapper[2559]: I0712 18:01:12.507050 2559 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-j4ffm\" (UniqueName: \"kubernetes.io/projected/e41e01dc-8b6c-499c-8402-58ed840ff0de-kube-api-access-j4ffm\") pod \"authorino-operator-75cc8d5cb7-85b85\" (UID: \"e41e01dc-8b6c-499c-8402-58ed840ff0de\") " pod="kuadrant-system/authorino-operator-75cc8d5cb7-85b85" Jul 12 18:01:12.608292 ip-10-0-129-128 kubenswrapper[2559]: I0712 18:01:12.608258 2559 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-j4ffm\" (UniqueName: \"kubernetes.io/projected/e41e01dc-8b6c-499c-8402-58ed840ff0de-kube-api-access-j4ffm\") pod \"authorino-operator-75cc8d5cb7-85b85\" (UID: \"e41e01dc-8b6c-499c-8402-58ed840ff0de\") " pod="kuadrant-system/authorino-operator-75cc8d5cb7-85b85" Jul 12 18:01:12.625919 ip-10-0-129-128 kubenswrapper[2559]: I0712 18:01:12.625884 2559 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-j4ffm\" (UniqueName: \"kubernetes.io/projected/e41e01dc-8b6c-499c-8402-58ed840ff0de-kube-api-access-j4ffm\") pod \"authorino-operator-75cc8d5cb7-85b85\" (UID: \"e41e01dc-8b6c-499c-8402-58ed840ff0de\") " pod="kuadrant-system/authorino-operator-75cc8d5cb7-85b85" Jul 12 18:01:12.730283 ip-10-0-129-128 kubenswrapper[2559]: I0712 18:01:12.730196 2559 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/authorino-operator-75cc8d5cb7-85b85" Jul 12 18:01:12.865580 ip-10-0-129-128 kubenswrapper[2559]: I0712 18:01:12.865556 2559 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/authorino-operator-75cc8d5cb7-85b85"] Jul 12 18:01:12.867529 ip-10-0-129-128 kubenswrapper[2559]: W0712 18:01:12.867499 2559 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pode41e01dc_8b6c_499c_8402_58ed840ff0de.slice/crio-cebbc4c08f5086098be35d4340caf00d7c8c2c1f83aa627bd6025361c3e0c478 WatchSource:0}: Error finding container cebbc4c08f5086098be35d4340caf00d7c8c2c1f83aa627bd6025361c3e0c478: Status 404 returned error can't find the container with id cebbc4c08f5086098be35d4340caf00d7c8c2c1f83aa627bd6025361c3e0c478 Jul 12 18:01:13.240778 ip-10-0-129-128 kubenswrapper[2559]: I0712 18:01:13.240740 2559 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/authorino-operator-75cc8d5cb7-85b85" event={"ID":"e41e01dc-8b6c-499c-8402-58ed840ff0de","Type":"ContainerStarted","Data":"cebbc4c08f5086098be35d4340caf00d7c8c2c1f83aa627bd6025361c3e0c478"} Jul 12 18:01:15.251903 ip-10-0-129-128 kubenswrapper[2559]: I0712 18:01:15.251812 2559 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/authorino-operator-75cc8d5cb7-85b85" event={"ID":"e41e01dc-8b6c-499c-8402-58ed840ff0de","Type":"ContainerStarted","Data":"c88712959701a9f9c3e5dfc6583261487f97a28d2dde44ade283bee042d12089"} Jul 12 18:01:15.252292 ip-10-0-129-128 kubenswrapper[2559]: I0712 18:01:15.251907 2559 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="kuadrant-system/authorino-operator-75cc8d5cb7-85b85" Jul 12 18:01:15.268526 ip-10-0-129-128 kubenswrapper[2559]: I0712 18:01:15.268470 2559 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="kuadrant-system/authorino-operator-75cc8d5cb7-85b85" podStartSLOduration=1.171214888 podStartE2EDuration="3.268454164s" totalImagesPullingTime="2.097239276s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-12 18:01:12 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-12 18:01:15.266786342 +0000 UTC m=+482.760854273" watchObservedRunningTime="2026-07-12 18:01:15.268454164 +0000 UTC m=+482.762522093" Jul 12 18:01:17.184406 ip-10-0-129-128 kubenswrapper[2559]: I0712 18:01:17.184374 2559 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["kuadrant-system/kuadrant-operator-controller-manager-8684c9ff49-xhgm6"] Jul 12 18:01:17.187939 ip-10-0-129-128 kubenswrapper[2559]: I0712 18:01:17.187917 2559 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/kuadrant-operator-controller-manager-8684c9ff49-xhgm6" Jul 12 18:01:17.190604 ip-10-0-129-128 kubenswrapper[2559]: I0712 18:01:17.190588 2559 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"kuadrant-system\"/\"kuadrant-operator-controller-manager-dockercfg-tjgw4\"" Jul 12 18:01:17.202303 ip-10-0-129-128 kubenswrapper[2559]: I0712 18:01:17.202284 2559 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/kuadrant-operator-controller-manager-8684c9ff49-xhgm6"] Jul 12 18:01:17.350410 ip-10-0-129-128 kubenswrapper[2559]: I0712 18:01:17.350371 2559 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-hm6bm\" (UniqueName: \"kubernetes.io/projected/fb27f716-105a-4e58-823b-0a2cbaa2e2ab-kube-api-access-hm6bm\") pod \"kuadrant-operator-controller-manager-8684c9ff49-xhgm6\" (UID: \"fb27f716-105a-4e58-823b-0a2cbaa2e2ab\") " pod="kuadrant-system/kuadrant-operator-controller-manager-8684c9ff49-xhgm6" Jul 12 18:01:17.350590 ip-10-0-129-128 kubenswrapper[2559]: I0712 18:01:17.350434 2559 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"extensions-socket-volume\" (UniqueName: \"kubernetes.io/empty-dir/fb27f716-105a-4e58-823b-0a2cbaa2e2ab-extensions-socket-volume\") pod \"kuadrant-operator-controller-manager-8684c9ff49-xhgm6\" (UID: \"fb27f716-105a-4e58-823b-0a2cbaa2e2ab\") " pod="kuadrant-system/kuadrant-operator-controller-manager-8684c9ff49-xhgm6" Jul 12 18:01:17.451895 ip-10-0-129-128 kubenswrapper[2559]: I0712 18:01:17.451809 2559 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-hm6bm\" (UniqueName: \"kubernetes.io/projected/fb27f716-105a-4e58-823b-0a2cbaa2e2ab-kube-api-access-hm6bm\") pod \"kuadrant-operator-controller-manager-8684c9ff49-xhgm6\" (UID: \"fb27f716-105a-4e58-823b-0a2cbaa2e2ab\") " pod="kuadrant-system/kuadrant-operator-controller-manager-8684c9ff49-xhgm6" Jul 12 18:01:17.451895 ip-10-0-129-128 kubenswrapper[2559]: I0712 18:01:17.451866 2559 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"extensions-socket-volume\" (UniqueName: \"kubernetes.io/empty-dir/fb27f716-105a-4e58-823b-0a2cbaa2e2ab-extensions-socket-volume\") pod \"kuadrant-operator-controller-manager-8684c9ff49-xhgm6\" (UID: \"fb27f716-105a-4e58-823b-0a2cbaa2e2ab\") " pod="kuadrant-system/kuadrant-operator-controller-manager-8684c9ff49-xhgm6" Jul 12 18:01:17.452240 ip-10-0-129-128 kubenswrapper[2559]: I0712 18:01:17.452223 2559 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"extensions-socket-volume\" (UniqueName: \"kubernetes.io/empty-dir/fb27f716-105a-4e58-823b-0a2cbaa2e2ab-extensions-socket-volume\") pod \"kuadrant-operator-controller-manager-8684c9ff49-xhgm6\" (UID: \"fb27f716-105a-4e58-823b-0a2cbaa2e2ab\") " pod="kuadrant-system/kuadrant-operator-controller-manager-8684c9ff49-xhgm6" Jul 12 18:01:17.459790 ip-10-0-129-128 kubenswrapper[2559]: I0712 18:01:17.459765 2559 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-hm6bm\" (UniqueName: \"kubernetes.io/projected/fb27f716-105a-4e58-823b-0a2cbaa2e2ab-kube-api-access-hm6bm\") pod \"kuadrant-operator-controller-manager-8684c9ff49-xhgm6\" (UID: \"fb27f716-105a-4e58-823b-0a2cbaa2e2ab\") " pod="kuadrant-system/kuadrant-operator-controller-manager-8684c9ff49-xhgm6" Jul 12 18:01:17.498277 ip-10-0-129-128 kubenswrapper[2559]: I0712 18:01:17.498239 2559 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/kuadrant-operator-controller-manager-8684c9ff49-xhgm6" Jul 12 18:01:17.638763 ip-10-0-129-128 kubenswrapper[2559]: I0712 18:01:17.638728 2559 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/kuadrant-operator-controller-manager-8684c9ff49-xhgm6"] Jul 12 18:01:17.641863 ip-10-0-129-128 kubenswrapper[2559]: W0712 18:01:17.641833 2559 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podfb27f716_105a_4e58_823b_0a2cbaa2e2ab.slice/crio-ca492f5890d310c3ca1810225595f3cbda6bddf0a38ce1f410b25669b0408f42 WatchSource:0}: Error finding container ca492f5890d310c3ca1810225595f3cbda6bddf0a38ce1f410b25669b0408f42: Status 404 returned error can't find the container with id ca492f5890d310c3ca1810225595f3cbda6bddf0a38ce1f410b25669b0408f42 Jul 12 18:01:18.266591 ip-10-0-129-128 kubenswrapper[2559]: I0712 18:01:18.266552 2559 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/kuadrant-operator-controller-manager-8684c9ff49-xhgm6" event={"ID":"fb27f716-105a-4e58-823b-0a2cbaa2e2ab","Type":"ContainerStarted","Data":"ca492f5890d310c3ca1810225595f3cbda6bddf0a38ce1f410b25669b0408f42"} Jul 12 18:01:19.222310 ip-10-0-129-128 kubenswrapper[2559]: I0712 18:01:19.222269 2559 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="kuadrant-system/dns-operator-controller-manager-864dd78c4c-zwc7p" Jul 12 18:01:22.238134 ip-10-0-129-128 kubenswrapper[2559]: I0712 18:01:22.238096 2559 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="kuadrant-system/limitador-operator-controller-manager-859d7b78bd-cf5jh" Jul 12 18:01:24.299207 ip-10-0-129-128 kubenswrapper[2559]: I0712 18:01:24.299167 2559 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/kuadrant-operator-controller-manager-8684c9ff49-xhgm6" event={"ID":"fb27f716-105a-4e58-823b-0a2cbaa2e2ab","Type":"ContainerStarted","Data":"c1133526206c209f66a4583793a195fba0090cac62489c43a601aab8cd226713"} Jul 12 18:01:24.299604 ip-10-0-129-128 kubenswrapper[2559]: I0712 18:01:24.299288 2559 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="kuadrant-system/kuadrant-operator-controller-manager-8684c9ff49-xhgm6" Jul 12 18:01:24.318396 ip-10-0-129-128 kubenswrapper[2559]: I0712 18:01:24.318347 2559 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="kuadrant-system/kuadrant-operator-controller-manager-8684c9ff49-xhgm6" podStartSLOduration=1.585685051 podStartE2EDuration="7.318332139s" totalImagesPullingTime="5.732647088s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-12 18:01:17 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-12 18:01:24.316053859 +0000 UTC m=+491.810121801" watchObservedRunningTime="2026-07-12 18:01:24.318332139 +0000 UTC m=+491.812400068" Jul 12 18:01:26.259277 ip-10-0-129-128 kubenswrapper[2559]: I0712 18:01:26.259241 2559 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="kuadrant-system/authorino-operator-75cc8d5cb7-85b85" Jul 12 18:01:35.305882 ip-10-0-129-128 kubenswrapper[2559]: I0712 18:01:35.305846 2559 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="kuadrant-system/kuadrant-operator-controller-manager-8684c9ff49-xhgm6" Jul 12 18:01:36.963022 ip-10-0-129-128 kubenswrapper[2559]: I0712 18:01:36.962961 2559 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["kuadrant-system/kuadrant-operator-controller-manager-8684c9ff49-xhgm6"] Jul 12 18:01:36.963472 ip-10-0-129-128 kubenswrapper[2559]: I0712 18:01:36.963164 2559 kuberuntime_container.go:873] "Killing container with a grace period" pod="kuadrant-system/kuadrant-operator-controller-manager-8684c9ff49-xhgm6" podUID="fb27f716-105a-4e58-823b-0a2cbaa2e2ab" containerName="manager" containerID="cri-o://c1133526206c209f66a4583793a195fba0090cac62489c43a601aab8cd226713" gracePeriod=2 Jul 12 18:01:36.976881 ip-10-0-129-128 kubenswrapper[2559]: I0712 18:01:36.976855 2559 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["kuadrant-system/kuadrant-operator-controller-manager-8684c9ff49-xhgm6"] Jul 12 18:01:36.978865 ip-10-0-129-128 kubenswrapper[2559]: E0712 18:01:36.978827 2559 status_manager.go:1041] "Failed to update status for pod" err="failed to patch status \"{\\\"metadata\\\":{\\\"uid\\\":\\\"fb27f716-105a-4e58-823b-0a2cbaa2e2ab\\\"},\\\"status\\\":{\\\"$setElementOrder/conditions\\\":[{\\\"type\\\":\\\"PodReadyToStartContainers\\\"},{\\\"type\\\":\\\"Initialized\\\"},{\\\"type\\\":\\\"Ready\\\"},{\\\"type\\\":\\\"ContainersReady\\\"},{\\\"type\\\":\\\"PodScheduled\\\"}],\\\"conditions\\\":[{\\\"observedGeneration\\\":2,\\\"type\\\":\\\"PodReadyToStartContainers\\\"},{\\\"observedGeneration\\\":2,\\\"type\\\":\\\"Initialized\\\"},{\\\"observedGeneration\\\":2,\\\"type\\\":\\\"Ready\\\"},{\\\"observedGeneration\\\":2,\\\"type\\\":\\\"ContainersReady\\\"},{\\\"observedGeneration\\\":2,\\\"type\\\":\\\"PodScheduled\\\"}],\\\"observedGeneration\\\":2}}\" for pod \"kuadrant-system\"/\"kuadrant-operator-controller-manager-8684c9ff49-xhgm6\": pods \"kuadrant-operator-controller-manager-8684c9ff49-xhgm6\" not found" pod="kuadrant-system/kuadrant-operator-controller-manager-8684c9ff49-xhgm6" Jul 12 18:01:36.985967 ip-10-0-129-128 kubenswrapper[2559]: I0712 18:01:36.985941 2559 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["kuadrant-system/kuadrant-operator-controller-manager-8684c9ff49-zwsvz"] Jul 12 18:01:36.986361 ip-10-0-129-128 kubenswrapper[2559]: I0712 18:01:36.986345 2559 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="fb27f716-105a-4e58-823b-0a2cbaa2e2ab" containerName="manager" Jul 12 18:01:36.986457 ip-10-0-129-128 kubenswrapper[2559]: I0712 18:01:36.986364 2559 state_mem.go:107] "Deleted CPUSet assignment" podUID="fb27f716-105a-4e58-823b-0a2cbaa2e2ab" containerName="manager" Jul 12 18:01:36.986457 ip-10-0-129-128 kubenswrapper[2559]: I0712 18:01:36.986452 2559 memory_manager.go:368] "RemoveStaleState removing state" podUID="fb27f716-105a-4e58-823b-0a2cbaa2e2ab" containerName="manager" Jul 12 18:01:36.989495 ip-10-0-129-128 kubenswrapper[2559]: I0712 18:01:36.989469 2559 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["kuadrant-system/limitador-operator-controller-manager-859d7b78bd-cf5jh"] Jul 12 18:01:36.989604 ip-10-0-129-128 kubenswrapper[2559]: I0712 18:01:36.989584 2559 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/kuadrant-operator-controller-manager-8684c9ff49-zwsvz" Jul 12 18:01:36.989761 ip-10-0-129-128 kubenswrapper[2559]: I0712 18:01:36.989737 2559 kuberuntime_container.go:873] "Killing container with a grace period" pod="kuadrant-system/limitador-operator-controller-manager-859d7b78bd-cf5jh" podUID="c737107f-73bd-4457-b865-dd7ea610b596" containerName="manager" containerID="cri-o://8a637af9431dec95bfd9125e8ca3795a8d3c1ce55cb33f24c89ecf5e194d28be" gracePeriod=2 Jul 12 18:01:36.998627 ip-10-0-129-128 kubenswrapper[2559]: I0712 18:01:36.998599 2559 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["kuadrant-system/limitador-operator-controller-manager-859d7b78bd-cf5jh"] Jul 12 18:01:37.000002 ip-10-0-129-128 kubenswrapper[2559]: E0712 18:01:36.999970 2559 status_manager.go:1041] "Failed to update status for pod" err="failed to patch status \"{\\\"metadata\\\":{\\\"uid\\\":\\\"c737107f-73bd-4457-b865-dd7ea610b596\\\"},\\\"status\\\":{\\\"$setElementOrder/conditions\\\":[{\\\"type\\\":\\\"PodReadyToStartContainers\\\"},{\\\"type\\\":\\\"Initialized\\\"},{\\\"type\\\":\\\"Ready\\\"},{\\\"type\\\":\\\"ContainersReady\\\"},{\\\"type\\\":\\\"PodScheduled\\\"}],\\\"conditions\\\":[{\\\"observedGeneration\\\":2,\\\"type\\\":\\\"PodReadyToStartContainers\\\"},{\\\"observedGeneration\\\":2,\\\"type\\\":\\\"Initialized\\\"},{\\\"observedGeneration\\\":2,\\\"type\\\":\\\"Ready\\\"},{\\\"observedGeneration\\\":2,\\\"type\\\":\\\"ContainersReady\\\"},{\\\"observedGeneration\\\":2,\\\"type\\\":\\\"PodScheduled\\\"}],\\\"observedGeneration\\\":2}}\" for pod \"kuadrant-system\"/\"limitador-operator-controller-manager-859d7b78bd-cf5jh\": pods \"limitador-operator-controller-manager-859d7b78bd-cf5jh\" not found" pod="kuadrant-system/limitador-operator-controller-manager-859d7b78bd-cf5jh" Jul 12 18:01:37.004055 ip-10-0-129-128 kubenswrapper[2559]: E0712 18:01:37.004030 2559 status_manager.go:1018] "Failed to get status for pod" err="pods \"kuadrant-operator-controller-manager-8684c9ff49-xhgm6\" is forbidden: User \"system:node:ip-10-0-129-128.ec2.internal\" cannot get resource \"pods\" in API group \"\" in the namespace \"kuadrant-system\": no relationship found between node 'ip-10-0-129-128.ec2.internal' and this object" podUID="fb27f716-105a-4e58-823b-0a2cbaa2e2ab" pod="kuadrant-system/kuadrant-operator-controller-manager-8684c9ff49-xhgm6" Jul 12 18:01:37.006750 ip-10-0-129-128 kubenswrapper[2559]: E0712 18:01:37.006712 2559 status_manager.go:1018] "Failed to get status for pod" err="pods \"kuadrant-operator-controller-manager-8684c9ff49-xhgm6\" is forbidden: User \"system:node:ip-10-0-129-128.ec2.internal\" cannot get resource \"pods\" in API group \"\" in the namespace \"kuadrant-system\": no relationship found between node 'ip-10-0-129-128.ec2.internal' and this object" podUID="fb27f716-105a-4e58-823b-0a2cbaa2e2ab" pod="kuadrant-system/kuadrant-operator-controller-manager-8684c9ff49-xhgm6" Jul 12 18:01:37.007600 ip-10-0-129-128 kubenswrapper[2559]: I0712 18:01:37.007577 2559 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/kuadrant-operator-controller-manager-8684c9ff49-zwsvz"] Jul 12 18:01:37.018789 ip-10-0-129-128 kubenswrapper[2559]: I0712 18:01:37.018764 2559 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["kuadrant-system/limitador-operator-controller-manager-859d7b78bd-jh5jl"] Jul 12 18:01:37.019085 ip-10-0-129-128 kubenswrapper[2559]: I0712 18:01:37.019063 2559 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"extensions-socket-volume\" (UniqueName: \"kubernetes.io/empty-dir/55a8c35f-e1f2-4f32-ac07-eab40b2db860-extensions-socket-volume\") pod \"kuadrant-operator-controller-manager-8684c9ff49-zwsvz\" (UID: \"55a8c35f-e1f2-4f32-ac07-eab40b2db860\") " pod="kuadrant-system/kuadrant-operator-controller-manager-8684c9ff49-zwsvz" Jul 12 18:01:37.019195 ip-10-0-129-128 kubenswrapper[2559]: I0712 18:01:37.019116 2559 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-7n82x\" (UniqueName: \"kubernetes.io/projected/55a8c35f-e1f2-4f32-ac07-eab40b2db860-kube-api-access-7n82x\") pod \"kuadrant-operator-controller-manager-8684c9ff49-zwsvz\" (UID: \"55a8c35f-e1f2-4f32-ac07-eab40b2db860\") " pod="kuadrant-system/kuadrant-operator-controller-manager-8684c9ff49-zwsvz" Jul 12 18:01:37.019195 ip-10-0-129-128 kubenswrapper[2559]: I0712 18:01:37.019134 2559 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="c737107f-73bd-4457-b865-dd7ea610b596" containerName="manager" Jul 12 18:01:37.019195 ip-10-0-129-128 kubenswrapper[2559]: I0712 18:01:37.019148 2559 state_mem.go:107] "Deleted CPUSet assignment" podUID="c737107f-73bd-4457-b865-dd7ea610b596" containerName="manager" Jul 12 18:01:37.019359 ip-10-0-129-128 kubenswrapper[2559]: I0712 18:01:37.019217 2559 memory_manager.go:368] "RemoveStaleState removing state" podUID="c737107f-73bd-4457-b865-dd7ea610b596" containerName="manager" Jul 12 18:01:37.022224 ip-10-0-129-128 kubenswrapper[2559]: I0712 18:01:37.022204 2559 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/limitador-operator-controller-manager-859d7b78bd-jh5jl" Jul 12 18:01:37.037577 ip-10-0-129-128 kubenswrapper[2559]: E0712 18:01:37.037539 2559 status_manager.go:1018] "Failed to get status for pod" err="pods \"limitador-operator-controller-manager-859d7b78bd-cf5jh\" is forbidden: User \"system:node:ip-10-0-129-128.ec2.internal\" cannot get resource \"pods\" in API group \"\" in the namespace \"kuadrant-system\": no relationship found between node 'ip-10-0-129-128.ec2.internal' and this object" podUID="c737107f-73bd-4457-b865-dd7ea610b596" pod="kuadrant-system/limitador-operator-controller-manager-859d7b78bd-cf5jh" Jul 12 18:01:37.038550 ip-10-0-129-128 kubenswrapper[2559]: I0712 18:01:37.038528 2559 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-operator-controller-manager-859d7b78bd-jh5jl"] Jul 12 18:01:37.120108 ip-10-0-129-128 kubenswrapper[2559]: I0712 18:01:37.120077 2559 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"extensions-socket-volume\" (UniqueName: \"kubernetes.io/empty-dir/55a8c35f-e1f2-4f32-ac07-eab40b2db860-extensions-socket-volume\") pod \"kuadrant-operator-controller-manager-8684c9ff49-zwsvz\" (UID: \"55a8c35f-e1f2-4f32-ac07-eab40b2db860\") " pod="kuadrant-system/kuadrant-operator-controller-manager-8684c9ff49-zwsvz" Jul 12 18:01:37.120221 ip-10-0-129-128 kubenswrapper[2559]: I0712 18:01:37.120133 2559 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-7n82x\" (UniqueName: \"kubernetes.io/projected/55a8c35f-e1f2-4f32-ac07-eab40b2db860-kube-api-access-7n82x\") pod \"kuadrant-operator-controller-manager-8684c9ff49-zwsvz\" (UID: \"55a8c35f-e1f2-4f32-ac07-eab40b2db860\") " pod="kuadrant-system/kuadrant-operator-controller-manager-8684c9ff49-zwsvz" Jul 12 18:01:37.120221 ip-10-0-129-128 kubenswrapper[2559]: I0712 18:01:37.120161 2559 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-qmqtz\" (UniqueName: \"kubernetes.io/projected/07c44f20-f5a7-46ac-9534-3ebf12e0ebb9-kube-api-access-qmqtz\") pod \"limitador-operator-controller-manager-859d7b78bd-jh5jl\" (UID: \"07c44f20-f5a7-46ac-9534-3ebf12e0ebb9\") " pod="kuadrant-system/limitador-operator-controller-manager-859d7b78bd-jh5jl" Jul 12 18:01:37.120490 ip-10-0-129-128 kubenswrapper[2559]: I0712 18:01:37.120467 2559 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"extensions-socket-volume\" (UniqueName: \"kubernetes.io/empty-dir/55a8c35f-e1f2-4f32-ac07-eab40b2db860-extensions-socket-volume\") pod \"kuadrant-operator-controller-manager-8684c9ff49-zwsvz\" (UID: \"55a8c35f-e1f2-4f32-ac07-eab40b2db860\") " pod="kuadrant-system/kuadrant-operator-controller-manager-8684c9ff49-zwsvz" Jul 12 18:01:37.153537 ip-10-0-129-128 kubenswrapper[2559]: I0712 18:01:37.153457 2559 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-7n82x\" (UniqueName: \"kubernetes.io/projected/55a8c35f-e1f2-4f32-ac07-eab40b2db860-kube-api-access-7n82x\") pod \"kuadrant-operator-controller-manager-8684c9ff49-zwsvz\" (UID: \"55a8c35f-e1f2-4f32-ac07-eab40b2db860\") " pod="kuadrant-system/kuadrant-operator-controller-manager-8684c9ff49-zwsvz" Jul 12 18:01:37.220936 ip-10-0-129-128 kubenswrapper[2559]: I0712 18:01:37.220906 2559 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-qmqtz\" (UniqueName: \"kubernetes.io/projected/07c44f20-f5a7-46ac-9534-3ebf12e0ebb9-kube-api-access-qmqtz\") pod \"limitador-operator-controller-manager-859d7b78bd-jh5jl\" (UID: \"07c44f20-f5a7-46ac-9534-3ebf12e0ebb9\") " pod="kuadrant-system/limitador-operator-controller-manager-859d7b78bd-jh5jl" Jul 12 18:01:37.235071 ip-10-0-129-128 kubenswrapper[2559]: I0712 18:01:37.235044 2559 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-qmqtz\" (UniqueName: \"kubernetes.io/projected/07c44f20-f5a7-46ac-9534-3ebf12e0ebb9-kube-api-access-qmqtz\") pod \"limitador-operator-controller-manager-859d7b78bd-jh5jl\" (UID: \"07c44f20-f5a7-46ac-9534-3ebf12e0ebb9\") " pod="kuadrant-system/limitador-operator-controller-manager-859d7b78bd-jh5jl" Jul 12 18:01:37.241016 ip-10-0-129-128 kubenswrapper[2559]: I0712 18:01:37.240994 2559 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/limitador-operator-controller-manager-859d7b78bd-cf5jh" Jul 12 18:01:37.243021 ip-10-0-129-128 kubenswrapper[2559]: I0712 18:01:37.243005 2559 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/kuadrant-operator-controller-manager-8684c9ff49-xhgm6" Jul 12 18:01:37.321456 ip-10-0-129-128 kubenswrapper[2559]: I0712 18:01:37.321426 2559 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-hm6bm\" (UniqueName: \"kubernetes.io/projected/fb27f716-105a-4e58-823b-0a2cbaa2e2ab-kube-api-access-hm6bm\") pod \"fb27f716-105a-4e58-823b-0a2cbaa2e2ab\" (UID: \"fb27f716-105a-4e58-823b-0a2cbaa2e2ab\") " Jul 12 18:01:37.321679 ip-10-0-129-128 kubenswrapper[2559]: I0712 18:01:37.321477 2559 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-snwf5\" (UniqueName: \"kubernetes.io/projected/c737107f-73bd-4457-b865-dd7ea610b596-kube-api-access-snwf5\") pod \"c737107f-73bd-4457-b865-dd7ea610b596\" (UID: \"c737107f-73bd-4457-b865-dd7ea610b596\") " Jul 12 18:01:37.321679 ip-10-0-129-128 kubenswrapper[2559]: I0712 18:01:37.321505 2559 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"extensions-socket-volume\" (UniqueName: \"kubernetes.io/empty-dir/fb27f716-105a-4e58-823b-0a2cbaa2e2ab-extensions-socket-volume\") pod \"fb27f716-105a-4e58-823b-0a2cbaa2e2ab\" (UID: \"fb27f716-105a-4e58-823b-0a2cbaa2e2ab\") " Jul 12 18:01:37.321998 ip-10-0-129-128 kubenswrapper[2559]: I0712 18:01:37.321974 2559 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/fb27f716-105a-4e58-823b-0a2cbaa2e2ab-extensions-socket-volume" (OuterVolumeSpecName: "extensions-socket-volume") pod "fb27f716-105a-4e58-823b-0a2cbaa2e2ab" (UID: "fb27f716-105a-4e58-823b-0a2cbaa2e2ab"). InnerVolumeSpecName "extensions-socket-volume". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Jul 12 18:01:37.323590 ip-10-0-129-128 kubenswrapper[2559]: I0712 18:01:37.323567 2559 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/c737107f-73bd-4457-b865-dd7ea610b596-kube-api-access-snwf5" (OuterVolumeSpecName: "kube-api-access-snwf5") pod "c737107f-73bd-4457-b865-dd7ea610b596" (UID: "c737107f-73bd-4457-b865-dd7ea610b596"). InnerVolumeSpecName "kube-api-access-snwf5". PluginName "kubernetes.io/projected", VolumeGIDValue "" Jul 12 18:01:37.323674 ip-10-0-129-128 kubenswrapper[2559]: I0712 18:01:37.323594 2559 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/fb27f716-105a-4e58-823b-0a2cbaa2e2ab-kube-api-access-hm6bm" (OuterVolumeSpecName: "kube-api-access-hm6bm") pod "fb27f716-105a-4e58-823b-0a2cbaa2e2ab" (UID: "fb27f716-105a-4e58-823b-0a2cbaa2e2ab"). InnerVolumeSpecName "kube-api-access-hm6bm". PluginName "kubernetes.io/projected", VolumeGIDValue "" Jul 12 18:01:37.359084 ip-10-0-129-128 kubenswrapper[2559]: I0712 18:01:37.359049 2559 generic.go:356] "Generic (PLEG): container finished" podID="fb27f716-105a-4e58-823b-0a2cbaa2e2ab" containerID="c1133526206c209f66a4583793a195fba0090cac62489c43a601aab8cd226713" exitCode=0 Jul 12 18:01:37.359244 ip-10-0-129-128 kubenswrapper[2559]: I0712 18:01:37.359098 2559 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/kuadrant-operator-controller-manager-8684c9ff49-xhgm6" Jul 12 18:01:37.359244 ip-10-0-129-128 kubenswrapper[2559]: I0712 18:01:37.359156 2559 scope.go:117] "RemoveContainer" containerID="c1133526206c209f66a4583793a195fba0090cac62489c43a601aab8cd226713" Jul 12 18:01:37.360486 ip-10-0-129-128 kubenswrapper[2559]: I0712 18:01:37.360463 2559 generic.go:356] "Generic (PLEG): container finished" podID="c737107f-73bd-4457-b865-dd7ea610b596" containerID="8a637af9431dec95bfd9125e8ca3795a8d3c1ce55cb33f24c89ecf5e194d28be" exitCode=0 Jul 12 18:01:37.360566 ip-10-0-129-128 kubenswrapper[2559]: I0712 18:01:37.360519 2559 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/limitador-operator-controller-manager-859d7b78bd-cf5jh" Jul 12 18:01:37.367730 ip-10-0-129-128 kubenswrapper[2559]: I0712 18:01:37.367713 2559 scope.go:117] "RemoveContainer" containerID="c1133526206c209f66a4583793a195fba0090cac62489c43a601aab8cd226713" Jul 12 18:01:37.368033 ip-10-0-129-128 kubenswrapper[2559]: E0712 18:01:37.368009 2559 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"c1133526206c209f66a4583793a195fba0090cac62489c43a601aab8cd226713\": container with ID starting with c1133526206c209f66a4583793a195fba0090cac62489c43a601aab8cd226713 not found: ID does not exist" containerID="c1133526206c209f66a4583793a195fba0090cac62489c43a601aab8cd226713" Jul 12 18:01:37.368085 ip-10-0-129-128 kubenswrapper[2559]: I0712 18:01:37.368046 2559 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"c1133526206c209f66a4583793a195fba0090cac62489c43a601aab8cd226713"} err="failed to get container status \"c1133526206c209f66a4583793a195fba0090cac62489c43a601aab8cd226713\": rpc error: code = NotFound desc = could not find container \"c1133526206c209f66a4583793a195fba0090cac62489c43a601aab8cd226713\": container with ID starting with c1133526206c209f66a4583793a195fba0090cac62489c43a601aab8cd226713 not found: ID does not exist" Jul 12 18:01:37.368085 ip-10-0-129-128 kubenswrapper[2559]: I0712 18:01:37.368073 2559 scope.go:117] "RemoveContainer" containerID="8a637af9431dec95bfd9125e8ca3795a8d3c1ce55cb33f24c89ecf5e194d28be" Jul 12 18:01:37.376428 ip-10-0-129-128 kubenswrapper[2559]: I0712 18:01:37.376412 2559 scope.go:117] "RemoveContainer" containerID="8a637af9431dec95bfd9125e8ca3795a8d3c1ce55cb33f24c89ecf5e194d28be" Jul 12 18:01:37.376758 ip-10-0-129-128 kubenswrapper[2559]: E0712 18:01:37.376736 2559 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"8a637af9431dec95bfd9125e8ca3795a8d3c1ce55cb33f24c89ecf5e194d28be\": container with ID starting with 8a637af9431dec95bfd9125e8ca3795a8d3c1ce55cb33f24c89ecf5e194d28be not found: ID does not exist" containerID="8a637af9431dec95bfd9125e8ca3795a8d3c1ce55cb33f24c89ecf5e194d28be" Jul 12 18:01:37.376827 ip-10-0-129-128 kubenswrapper[2559]: I0712 18:01:37.376767 2559 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"8a637af9431dec95bfd9125e8ca3795a8d3c1ce55cb33f24c89ecf5e194d28be"} err="failed to get container status \"8a637af9431dec95bfd9125e8ca3795a8d3c1ce55cb33f24c89ecf5e194d28be\": rpc error: code = NotFound desc = could not find container \"8a637af9431dec95bfd9125e8ca3795a8d3c1ce55cb33f24c89ecf5e194d28be\": container with ID starting with 8a637af9431dec95bfd9125e8ca3795a8d3c1ce55cb33f24c89ecf5e194d28be not found: ID does not exist" Jul 12 18:01:37.406212 ip-10-0-129-128 kubenswrapper[2559]: I0712 18:01:37.406186 2559 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/kuadrant-operator-controller-manager-8684c9ff49-zwsvz" Jul 12 18:01:37.413254 ip-10-0-129-128 kubenswrapper[2559]: I0712 18:01:37.413232 2559 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/limitador-operator-controller-manager-859d7b78bd-jh5jl" Jul 12 18:01:37.422518 ip-10-0-129-128 kubenswrapper[2559]: I0712 18:01:37.422494 2559 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-hm6bm\" (UniqueName: \"kubernetes.io/projected/fb27f716-105a-4e58-823b-0a2cbaa2e2ab-kube-api-access-hm6bm\") on node \"ip-10-0-129-128.ec2.internal\" DevicePath \"\"" Jul 12 18:01:37.422603 ip-10-0-129-128 kubenswrapper[2559]: I0712 18:01:37.422523 2559 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-snwf5\" (UniqueName: \"kubernetes.io/projected/c737107f-73bd-4457-b865-dd7ea610b596-kube-api-access-snwf5\") on node \"ip-10-0-129-128.ec2.internal\" DevicePath \"\"" Jul 12 18:01:37.422603 ip-10-0-129-128 kubenswrapper[2559]: I0712 18:01:37.422540 2559 reconciler_common.go:299] "Volume detached for volume \"extensions-socket-volume\" (UniqueName: \"kubernetes.io/empty-dir/fb27f716-105a-4e58-823b-0a2cbaa2e2ab-extensions-socket-volume\") on node \"ip-10-0-129-128.ec2.internal\" DevicePath \"\"" Jul 12 18:01:37.560100 ip-10-0-129-128 kubenswrapper[2559]: I0712 18:01:37.560032 2559 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/kuadrant-operator-controller-manager-8684c9ff49-zwsvz"] Jul 12 18:01:37.563085 ip-10-0-129-128 kubenswrapper[2559]: W0712 18:01:37.563059 2559 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod55a8c35f_e1f2_4f32_ac07_eab40b2db860.slice/crio-b59babf9112de9a8af4ea5bf439ecffabc19dd60115457d31eed2a24682cd13e WatchSource:0}: Error finding container b59babf9112de9a8af4ea5bf439ecffabc19dd60115457d31eed2a24682cd13e: Status 404 returned error can't find the container with id b59babf9112de9a8af4ea5bf439ecffabc19dd60115457d31eed2a24682cd13e Jul 12 18:01:37.571951 ip-10-0-129-128 kubenswrapper[2559]: I0712 18:01:37.571872 2559 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-operator-controller-manager-859d7b78bd-jh5jl"] Jul 12 18:01:37.574393 ip-10-0-129-128 kubenswrapper[2559]: W0712 18:01:37.574351 2559 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod07c44f20_f5a7_46ac_9534_3ebf12e0ebb9.slice/crio-61f68c178fa9806e82587607aba889710b5a451bd127a88122c17a30105decca WatchSource:0}: Error finding container 61f68c178fa9806e82587607aba889710b5a451bd127a88122c17a30105decca: Status 404 returned error can't find the container with id 61f68c178fa9806e82587607aba889710b5a451bd127a88122c17a30105decca Jul 12 18:01:38.366824 ip-10-0-129-128 kubenswrapper[2559]: I0712 18:01:38.366789 2559 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/limitador-operator-controller-manager-859d7b78bd-jh5jl" event={"ID":"07c44f20-f5a7-46ac-9534-3ebf12e0ebb9","Type":"ContainerStarted","Data":"6f7e81640e0bca7db2a39efd11fc328a4bd5e4ec661a82aa517f7831b9928670"} Jul 12 18:01:38.367245 ip-10-0-129-128 kubenswrapper[2559]: I0712 18:01:38.366844 2559 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/limitador-operator-controller-manager-859d7b78bd-jh5jl" event={"ID":"07c44f20-f5a7-46ac-9534-3ebf12e0ebb9","Type":"ContainerStarted","Data":"61f68c178fa9806e82587607aba889710b5a451bd127a88122c17a30105decca"} Jul 12 18:01:38.367245 ip-10-0-129-128 kubenswrapper[2559]: I0712 18:01:38.366875 2559 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="kuadrant-system/limitador-operator-controller-manager-859d7b78bd-jh5jl" Jul 12 18:01:38.368535 ip-10-0-129-128 kubenswrapper[2559]: I0712 18:01:38.368510 2559 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/kuadrant-operator-controller-manager-8684c9ff49-zwsvz" event={"ID":"55a8c35f-e1f2-4f32-ac07-eab40b2db860","Type":"ContainerStarted","Data":"3727712eec43a10a113227db3b3295e280d648d6a4f68482be3b7893ee2ec382"} Jul 12 18:01:38.368702 ip-10-0-129-128 kubenswrapper[2559]: I0712 18:01:38.368554 2559 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/kuadrant-operator-controller-manager-8684c9ff49-zwsvz" event={"ID":"55a8c35f-e1f2-4f32-ac07-eab40b2db860","Type":"ContainerStarted","Data":"b59babf9112de9a8af4ea5bf439ecffabc19dd60115457d31eed2a24682cd13e"} Jul 12 18:01:38.368702 ip-10-0-129-128 kubenswrapper[2559]: I0712 18:01:38.368585 2559 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="kuadrant-system/kuadrant-operator-controller-manager-8684c9ff49-zwsvz" Jul 12 18:01:38.404031 ip-10-0-129-128 kubenswrapper[2559]: I0712 18:01:38.403978 2559 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="kuadrant-system/limitador-operator-controller-manager-859d7b78bd-jh5jl" podStartSLOduration=2.403961214 podStartE2EDuration="2.403961214s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-12 18:01:36 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-12 18:01:38.403398865 +0000 UTC m=+505.897466797" watchObservedRunningTime="2026-07-12 18:01:38.403961214 +0000 UTC m=+505.898029144" Jul 12 18:01:38.426039 ip-10-0-129-128 kubenswrapper[2559]: I0712 18:01:38.422306 2559 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="kuadrant-system/kuadrant-operator-controller-manager-8684c9ff49-zwsvz" podStartSLOduration=2.422270061 podStartE2EDuration="2.422270061s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-12 18:01:36 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-12 18:01:38.418954085 +0000 UTC m=+505.913022009" watchObservedRunningTime="2026-07-12 18:01:38.422270061 +0000 UTC m=+505.916337991" Jul 12 18:01:39.033473 ip-10-0-129-128 kubenswrapper[2559]: I0712 18:01:39.033437 2559 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="c737107f-73bd-4457-b865-dd7ea610b596" path="/var/lib/kubelet/pods/c737107f-73bd-4457-b865-dd7ea610b596/volumes" Jul 12 18:01:39.033806 ip-10-0-129-128 kubenswrapper[2559]: I0712 18:01:39.033792 2559 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="fb27f716-105a-4e58-823b-0a2cbaa2e2ab" path="/var/lib/kubelet/pods/fb27f716-105a-4e58-823b-0a2cbaa2e2ab/volumes" Jul 12 18:01:49.377594 ip-10-0-129-128 kubenswrapper[2559]: I0712 18:01:49.377557 2559 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="kuadrant-system/limitador-operator-controller-manager-859d7b78bd-jh5jl" Jul 12 18:01:49.378204 ip-10-0-129-128 kubenswrapper[2559]: I0712 18:01:49.377635 2559 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="kuadrant-system/kuadrant-operator-controller-manager-8684c9ff49-zwsvz" Jul 12 18:01:53.060028 ip-10-0-129-128 kubenswrapper[2559]: I0712 18:01:53.059996 2559 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["kuadrant-system/kuadrant-operator-controller-manager-8684c9ff49-zwsvz"] Jul 12 18:01:53.060407 ip-10-0-129-128 kubenswrapper[2559]: I0712 18:01:53.060167 2559 kuberuntime_container.go:873] "Killing container with a grace period" pod="kuadrant-system/kuadrant-operator-controller-manager-8684c9ff49-zwsvz" podUID="55a8c35f-e1f2-4f32-ac07-eab40b2db860" containerName="manager" containerID="cri-o://3727712eec43a10a113227db3b3295e280d648d6a4f68482be3b7893ee2ec382" gracePeriod=10 Jul 12 18:01:53.317372 ip-10-0-129-128 kubenswrapper[2559]: I0712 18:01:53.317308 2559 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/kuadrant-operator-controller-manager-8684c9ff49-zwsvz" Jul 12 18:01:53.344324 ip-10-0-129-128 kubenswrapper[2559]: I0712 18:01:53.344291 2559 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["kuadrant-system/kuadrant-operator-controller-manager-86878c769f-vbnpm"] Jul 12 18:01:53.344655 ip-10-0-129-128 kubenswrapper[2559]: I0712 18:01:53.344631 2559 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="55a8c35f-e1f2-4f32-ac07-eab40b2db860" containerName="manager" Jul 12 18:01:53.344704 ip-10-0-129-128 kubenswrapper[2559]: I0712 18:01:53.344661 2559 state_mem.go:107] "Deleted CPUSet assignment" podUID="55a8c35f-e1f2-4f32-ac07-eab40b2db860" containerName="manager" Jul 12 18:01:53.344752 ip-10-0-129-128 kubenswrapper[2559]: I0712 18:01:53.344743 2559 memory_manager.go:368] "RemoveStaleState removing state" podUID="55a8c35f-e1f2-4f32-ac07-eab40b2db860" containerName="manager" Jul 12 18:01:53.347525 ip-10-0-129-128 kubenswrapper[2559]: I0712 18:01:53.347510 2559 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/kuadrant-operator-controller-manager-86878c769f-vbnpm" Jul 12 18:01:53.382025 ip-10-0-129-128 kubenswrapper[2559]: I0712 18:01:53.381993 2559 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/kuadrant-operator-controller-manager-86878c769f-vbnpm"] Jul 12 18:01:53.439826 ip-10-0-129-128 kubenswrapper[2559]: I0712 18:01:53.439794 2559 generic.go:356] "Generic (PLEG): container finished" podID="55a8c35f-e1f2-4f32-ac07-eab40b2db860" containerID="3727712eec43a10a113227db3b3295e280d648d6a4f68482be3b7893ee2ec382" exitCode=0 Jul 12 18:01:53.439995 ip-10-0-129-128 kubenswrapper[2559]: I0712 18:01:53.439836 2559 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/kuadrant-operator-controller-manager-8684c9ff49-zwsvz" Jul 12 18:01:53.439995 ip-10-0-129-128 kubenswrapper[2559]: I0712 18:01:53.439884 2559 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/kuadrant-operator-controller-manager-8684c9ff49-zwsvz" event={"ID":"55a8c35f-e1f2-4f32-ac07-eab40b2db860","Type":"ContainerDied","Data":"3727712eec43a10a113227db3b3295e280d648d6a4f68482be3b7893ee2ec382"} Jul 12 18:01:53.439995 ip-10-0-129-128 kubenswrapper[2559]: I0712 18:01:53.439948 2559 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/kuadrant-operator-controller-manager-8684c9ff49-zwsvz" event={"ID":"55a8c35f-e1f2-4f32-ac07-eab40b2db860","Type":"ContainerDied","Data":"b59babf9112de9a8af4ea5bf439ecffabc19dd60115457d31eed2a24682cd13e"} Jul 12 18:01:53.439995 ip-10-0-129-128 kubenswrapper[2559]: I0712 18:01:53.439972 2559 scope.go:117] "RemoveContainer" containerID="3727712eec43a10a113227db3b3295e280d648d6a4f68482be3b7893ee2ec382" Jul 12 18:01:53.448707 ip-10-0-129-128 kubenswrapper[2559]: I0712 18:01:53.448690 2559 scope.go:117] "RemoveContainer" containerID="3727712eec43a10a113227db3b3295e280d648d6a4f68482be3b7893ee2ec382" Jul 12 18:01:53.448978 ip-10-0-129-128 kubenswrapper[2559]: E0712 18:01:53.448961 2559 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"3727712eec43a10a113227db3b3295e280d648d6a4f68482be3b7893ee2ec382\": container with ID starting with 3727712eec43a10a113227db3b3295e280d648d6a4f68482be3b7893ee2ec382 not found: ID does not exist" containerID="3727712eec43a10a113227db3b3295e280d648d6a4f68482be3b7893ee2ec382" Jul 12 18:01:53.449022 ip-10-0-129-128 kubenswrapper[2559]: I0712 18:01:53.448991 2559 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"3727712eec43a10a113227db3b3295e280d648d6a4f68482be3b7893ee2ec382"} err="failed to get container status \"3727712eec43a10a113227db3b3295e280d648d6a4f68482be3b7893ee2ec382\": rpc error: code = NotFound desc = could not find container \"3727712eec43a10a113227db3b3295e280d648d6a4f68482be3b7893ee2ec382\": container with ID starting with 3727712eec43a10a113227db3b3295e280d648d6a4f68482be3b7893ee2ec382 not found: ID does not exist" Jul 12 18:01:53.464800 ip-10-0-129-128 kubenswrapper[2559]: I0712 18:01:53.464775 2559 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"extensions-socket-volume\" (UniqueName: \"kubernetes.io/empty-dir/55a8c35f-e1f2-4f32-ac07-eab40b2db860-extensions-socket-volume\") pod \"55a8c35f-e1f2-4f32-ac07-eab40b2db860\" (UID: \"55a8c35f-e1f2-4f32-ac07-eab40b2db860\") " Jul 12 18:01:53.464897 ip-10-0-129-128 kubenswrapper[2559]: I0712 18:01:53.464811 2559 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-7n82x\" (UniqueName: \"kubernetes.io/projected/55a8c35f-e1f2-4f32-ac07-eab40b2db860-kube-api-access-7n82x\") pod \"55a8c35f-e1f2-4f32-ac07-eab40b2db860\" (UID: \"55a8c35f-e1f2-4f32-ac07-eab40b2db860\") " Jul 12 18:01:53.464897 ip-10-0-129-128 kubenswrapper[2559]: I0712 18:01:53.464889 2559 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"extensions-socket-volume\" (UniqueName: \"kubernetes.io/empty-dir/38a75322-152d-48f3-90a0-e91a9f28137a-extensions-socket-volume\") pod \"kuadrant-operator-controller-manager-86878c769f-vbnpm\" (UID: \"38a75322-152d-48f3-90a0-e91a9f28137a\") " pod="kuadrant-system/kuadrant-operator-controller-manager-86878c769f-vbnpm" Jul 12 18:01:53.464976 ip-10-0-129-128 kubenswrapper[2559]: I0712 18:01:53.464963 2559 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-lzf6l\" (UniqueName: \"kubernetes.io/projected/38a75322-152d-48f3-90a0-e91a9f28137a-kube-api-access-lzf6l\") pod \"kuadrant-operator-controller-manager-86878c769f-vbnpm\" (UID: \"38a75322-152d-48f3-90a0-e91a9f28137a\") " pod="kuadrant-system/kuadrant-operator-controller-manager-86878c769f-vbnpm" Jul 12 18:01:53.465120 ip-10-0-129-128 kubenswrapper[2559]: I0712 18:01:53.465099 2559 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/55a8c35f-e1f2-4f32-ac07-eab40b2db860-extensions-socket-volume" (OuterVolumeSpecName: "extensions-socket-volume") pod "55a8c35f-e1f2-4f32-ac07-eab40b2db860" (UID: "55a8c35f-e1f2-4f32-ac07-eab40b2db860"). InnerVolumeSpecName "extensions-socket-volume". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Jul 12 18:01:53.466856 ip-10-0-129-128 kubenswrapper[2559]: I0712 18:01:53.466834 2559 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/55a8c35f-e1f2-4f32-ac07-eab40b2db860-kube-api-access-7n82x" (OuterVolumeSpecName: "kube-api-access-7n82x") pod "55a8c35f-e1f2-4f32-ac07-eab40b2db860" (UID: "55a8c35f-e1f2-4f32-ac07-eab40b2db860"). InnerVolumeSpecName "kube-api-access-7n82x". PluginName "kubernetes.io/projected", VolumeGIDValue "" Jul 12 18:01:53.565616 ip-10-0-129-128 kubenswrapper[2559]: I0712 18:01:53.565575 2559 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"extensions-socket-volume\" (UniqueName: \"kubernetes.io/empty-dir/38a75322-152d-48f3-90a0-e91a9f28137a-extensions-socket-volume\") pod \"kuadrant-operator-controller-manager-86878c769f-vbnpm\" (UID: \"38a75322-152d-48f3-90a0-e91a9f28137a\") " pod="kuadrant-system/kuadrant-operator-controller-manager-86878c769f-vbnpm" Jul 12 18:01:53.565791 ip-10-0-129-128 kubenswrapper[2559]: I0712 18:01:53.565674 2559 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-lzf6l\" (UniqueName: \"kubernetes.io/projected/38a75322-152d-48f3-90a0-e91a9f28137a-kube-api-access-lzf6l\") pod \"kuadrant-operator-controller-manager-86878c769f-vbnpm\" (UID: \"38a75322-152d-48f3-90a0-e91a9f28137a\") " pod="kuadrant-system/kuadrant-operator-controller-manager-86878c769f-vbnpm" Jul 12 18:01:53.565791 ip-10-0-129-128 kubenswrapper[2559]: I0712 18:01:53.565713 2559 reconciler_common.go:299] "Volume detached for volume \"extensions-socket-volume\" (UniqueName: \"kubernetes.io/empty-dir/55a8c35f-e1f2-4f32-ac07-eab40b2db860-extensions-socket-volume\") on node \"ip-10-0-129-128.ec2.internal\" DevicePath \"\"" Jul 12 18:01:53.565791 ip-10-0-129-128 kubenswrapper[2559]: I0712 18:01:53.565723 2559 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-7n82x\" (UniqueName: \"kubernetes.io/projected/55a8c35f-e1f2-4f32-ac07-eab40b2db860-kube-api-access-7n82x\") on node \"ip-10-0-129-128.ec2.internal\" DevicePath \"\"" Jul 12 18:01:53.565975 ip-10-0-129-128 kubenswrapper[2559]: I0712 18:01:53.565953 2559 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"extensions-socket-volume\" (UniqueName: \"kubernetes.io/empty-dir/38a75322-152d-48f3-90a0-e91a9f28137a-extensions-socket-volume\") pod \"kuadrant-operator-controller-manager-86878c769f-vbnpm\" (UID: \"38a75322-152d-48f3-90a0-e91a9f28137a\") " pod="kuadrant-system/kuadrant-operator-controller-manager-86878c769f-vbnpm" Jul 12 18:01:53.576306 ip-10-0-129-128 kubenswrapper[2559]: I0712 18:01:53.576237 2559 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-lzf6l\" (UniqueName: \"kubernetes.io/projected/38a75322-152d-48f3-90a0-e91a9f28137a-kube-api-access-lzf6l\") pod \"kuadrant-operator-controller-manager-86878c769f-vbnpm\" (UID: \"38a75322-152d-48f3-90a0-e91a9f28137a\") " pod="kuadrant-system/kuadrant-operator-controller-manager-86878c769f-vbnpm" Jul 12 18:01:53.657183 ip-10-0-129-128 kubenswrapper[2559]: I0712 18:01:53.657149 2559 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/kuadrant-operator-controller-manager-86878c769f-vbnpm" Jul 12 18:01:53.762936 ip-10-0-129-128 kubenswrapper[2559]: I0712 18:01:53.762903 2559 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["kuadrant-system/kuadrant-operator-controller-manager-8684c9ff49-zwsvz"] Jul 12 18:01:53.767213 ip-10-0-129-128 kubenswrapper[2559]: I0712 18:01:53.767187 2559 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["kuadrant-system/kuadrant-operator-controller-manager-8684c9ff49-zwsvz"] Jul 12 18:01:53.820084 ip-10-0-129-128 kubenswrapper[2559]: I0712 18:01:53.820060 2559 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/kuadrant-operator-controller-manager-86878c769f-vbnpm"] Jul 12 18:01:53.822387 ip-10-0-129-128 kubenswrapper[2559]: W0712 18:01:53.822357 2559 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod38a75322_152d_48f3_90a0_e91a9f28137a.slice/crio-fd17727ec9c5506afd53b30196351ba9e5d894d090d432ac385032bbca8c4059 WatchSource:0}: Error finding container fd17727ec9c5506afd53b30196351ba9e5d894d090d432ac385032bbca8c4059: Status 404 returned error can't find the container with id fd17727ec9c5506afd53b30196351ba9e5d894d090d432ac385032bbca8c4059 Jul 12 18:01:54.447777 ip-10-0-129-128 kubenswrapper[2559]: I0712 18:01:54.447739 2559 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/kuadrant-operator-controller-manager-86878c769f-vbnpm" event={"ID":"38a75322-152d-48f3-90a0-e91a9f28137a","Type":"ContainerStarted","Data":"53c4dc728e27dee8ab1afd513e54ac6f73c05ab8825fc3c02b4d270bcef074bf"} Jul 12 18:01:54.448194 ip-10-0-129-128 kubenswrapper[2559]: I0712 18:01:54.447799 2559 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/kuadrant-operator-controller-manager-86878c769f-vbnpm" event={"ID":"38a75322-152d-48f3-90a0-e91a9f28137a","Type":"ContainerStarted","Data":"fd17727ec9c5506afd53b30196351ba9e5d894d090d432ac385032bbca8c4059"} Jul 12 18:01:54.448194 ip-10-0-129-128 kubenswrapper[2559]: I0712 18:01:54.447860 2559 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="kuadrant-system/kuadrant-operator-controller-manager-86878c769f-vbnpm" Jul 12 18:01:54.467789 ip-10-0-129-128 kubenswrapper[2559]: I0712 18:01:54.467738 2559 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="kuadrant-system/kuadrant-operator-controller-manager-86878c769f-vbnpm" podStartSLOduration=1.467723258 podStartE2EDuration="1.467723258s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-12 18:01:53 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-12 18:01:54.4660709 +0000 UTC m=+521.960138830" watchObservedRunningTime="2026-07-12 18:01:54.467723258 +0000 UTC m=+521.961791188" Jul 12 18:01:55.033850 ip-10-0-129-128 kubenswrapper[2559]: I0712 18:01:55.033819 2559 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="55a8c35f-e1f2-4f32-ac07-eab40b2db860" path="/var/lib/kubelet/pods/55a8c35f-e1f2-4f32-ac07-eab40b2db860/volumes" Jul 12 18:02:05.455373 ip-10-0-129-128 kubenswrapper[2559]: I0712 18:02:05.455291 2559 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="kuadrant-system/kuadrant-operator-controller-manager-86878c769f-vbnpm" Jul 12 18:02:09.449704 ip-10-0-129-128 kubenswrapper[2559]: I0712 18:02:09.449673 2559 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-ingress/maas-default-gateway-openshift-default-56654dcf7-vxfhz"] Jul 12 18:02:09.453247 ip-10-0-129-128 kubenswrapper[2559]: I0712 18:02:09.453228 2559 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/maas-default-gateway-openshift-default-56654dcf7-vxfhz" Jul 12 18:02:09.455677 ip-10-0-129-128 kubenswrapper[2559]: I0712 18:02:09.455633 2559 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress\"/\"maas-default-gateway-openshift-default-dockercfg-fnrms\"" Jul 12 18:02:09.465711 ip-10-0-129-128 kubenswrapper[2559]: I0712 18:02:09.465615 2559 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/maas-default-gateway-openshift-default-56654dcf7-vxfhz"] Jul 12 18:02:09.599347 ip-10-0-129-128 kubenswrapper[2559]: I0712 18:02:09.599311 2559 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/37f0b65a-a3ee-4aee-a490-3c9888b7475f-workload-socket\") pod \"maas-default-gateway-openshift-default-56654dcf7-vxfhz\" (UID: \"37f0b65a-a3ee-4aee-a490-3c9888b7475f\") " pod="openshift-ingress/maas-default-gateway-openshift-default-56654dcf7-vxfhz" Jul 12 18:02:09.599347 ip-10-0-129-128 kubenswrapper[2559]: I0712 18:02:09.599352 2559 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/37f0b65a-a3ee-4aee-a490-3c9888b7475f-istio-data\") pod \"maas-default-gateway-openshift-default-56654dcf7-vxfhz\" (UID: \"37f0b65a-a3ee-4aee-a490-3c9888b7475f\") " pod="openshift-ingress/maas-default-gateway-openshift-default-56654dcf7-vxfhz" Jul 12 18:02:09.599547 ip-10-0-129-128 kubenswrapper[2559]: I0712 18:02:09.599376 2559 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/37f0b65a-a3ee-4aee-a490-3c9888b7475f-istio-podinfo\") pod \"maas-default-gateway-openshift-default-56654dcf7-vxfhz\" (UID: \"37f0b65a-a3ee-4aee-a490-3c9888b7475f\") " pod="openshift-ingress/maas-default-gateway-openshift-default-56654dcf7-vxfhz" Jul 12 18:02:09.599547 ip-10-0-129-128 kubenswrapper[2559]: I0712 18:02:09.599419 2559 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/37f0b65a-a3ee-4aee-a490-3c9888b7475f-istiod-ca-cert\") pod \"maas-default-gateway-openshift-default-56654dcf7-vxfhz\" (UID: \"37f0b65a-a3ee-4aee-a490-3c9888b7475f\") " pod="openshift-ingress/maas-default-gateway-openshift-default-56654dcf7-vxfhz" Jul 12 18:02:09.599547 ip-10-0-129-128 kubenswrapper[2559]: I0712 18:02:09.599477 2559 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/37f0b65a-a3ee-4aee-a490-3c9888b7475f-workload-certs\") pod \"maas-default-gateway-openshift-default-56654dcf7-vxfhz\" (UID: \"37f0b65a-a3ee-4aee-a490-3c9888b7475f\") " pod="openshift-ingress/maas-default-gateway-openshift-default-56654dcf7-vxfhz" Jul 12 18:02:09.599547 ip-10-0-129-128 kubenswrapper[2559]: I0712 18:02:09.599510 2559 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/37f0b65a-a3ee-4aee-a490-3c9888b7475f-credential-socket\") pod \"maas-default-gateway-openshift-default-56654dcf7-vxfhz\" (UID: \"37f0b65a-a3ee-4aee-a490-3c9888b7475f\") " pod="openshift-ingress/maas-default-gateway-openshift-default-56654dcf7-vxfhz" Jul 12 18:02:09.599547 ip-10-0-129-128 kubenswrapper[2559]: I0712 18:02:09.599526 2559 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/37f0b65a-a3ee-4aee-a490-3c9888b7475f-istio-envoy\") pod \"maas-default-gateway-openshift-default-56654dcf7-vxfhz\" (UID: \"37f0b65a-a3ee-4aee-a490-3c9888b7475f\") " pod="openshift-ingress/maas-default-gateway-openshift-default-56654dcf7-vxfhz" Jul 12 18:02:09.599738 ip-10-0-129-128 kubenswrapper[2559]: I0712 18:02:09.599549 2559 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-t7qwn\" (UniqueName: \"kubernetes.io/projected/37f0b65a-a3ee-4aee-a490-3c9888b7475f-kube-api-access-t7qwn\") pod \"maas-default-gateway-openshift-default-56654dcf7-vxfhz\" (UID: \"37f0b65a-a3ee-4aee-a490-3c9888b7475f\") " pod="openshift-ingress/maas-default-gateway-openshift-default-56654dcf7-vxfhz" Jul 12 18:02:09.599738 ip-10-0-129-128 kubenswrapper[2559]: I0712 18:02:09.599582 2559 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/37f0b65a-a3ee-4aee-a490-3c9888b7475f-istio-token\") pod \"maas-default-gateway-openshift-default-56654dcf7-vxfhz\" (UID: \"37f0b65a-a3ee-4aee-a490-3c9888b7475f\") " pod="openshift-ingress/maas-default-gateway-openshift-default-56654dcf7-vxfhz" Jul 12 18:02:09.700780 ip-10-0-129-128 kubenswrapper[2559]: I0712 18:02:09.700675 2559 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/37f0b65a-a3ee-4aee-a490-3c9888b7475f-workload-socket\") pod \"maas-default-gateway-openshift-default-56654dcf7-vxfhz\" (UID: \"37f0b65a-a3ee-4aee-a490-3c9888b7475f\") " pod="openshift-ingress/maas-default-gateway-openshift-default-56654dcf7-vxfhz" Jul 12 18:02:09.700780 ip-10-0-129-128 kubenswrapper[2559]: I0712 18:02:09.700724 2559 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/37f0b65a-a3ee-4aee-a490-3c9888b7475f-istio-data\") pod \"maas-default-gateway-openshift-default-56654dcf7-vxfhz\" (UID: \"37f0b65a-a3ee-4aee-a490-3c9888b7475f\") " pod="openshift-ingress/maas-default-gateway-openshift-default-56654dcf7-vxfhz" Jul 12 18:02:09.700780 ip-10-0-129-128 kubenswrapper[2559]: I0712 18:02:09.700761 2559 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/37f0b65a-a3ee-4aee-a490-3c9888b7475f-istio-podinfo\") pod \"maas-default-gateway-openshift-default-56654dcf7-vxfhz\" (UID: \"37f0b65a-a3ee-4aee-a490-3c9888b7475f\") " pod="openshift-ingress/maas-default-gateway-openshift-default-56654dcf7-vxfhz" Jul 12 18:02:09.701056 ip-10-0-129-128 kubenswrapper[2559]: I0712 18:02:09.700787 2559 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/37f0b65a-a3ee-4aee-a490-3c9888b7475f-istiod-ca-cert\") pod \"maas-default-gateway-openshift-default-56654dcf7-vxfhz\" (UID: \"37f0b65a-a3ee-4aee-a490-3c9888b7475f\") " pod="openshift-ingress/maas-default-gateway-openshift-default-56654dcf7-vxfhz" Jul 12 18:02:09.701056 ip-10-0-129-128 kubenswrapper[2559]: I0712 18:02:09.700831 2559 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/37f0b65a-a3ee-4aee-a490-3c9888b7475f-workload-certs\") pod \"maas-default-gateway-openshift-default-56654dcf7-vxfhz\" (UID: \"37f0b65a-a3ee-4aee-a490-3c9888b7475f\") " pod="openshift-ingress/maas-default-gateway-openshift-default-56654dcf7-vxfhz" Jul 12 18:02:09.701056 ip-10-0-129-128 kubenswrapper[2559]: I0712 18:02:09.700875 2559 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/37f0b65a-a3ee-4aee-a490-3c9888b7475f-credential-socket\") pod \"maas-default-gateway-openshift-default-56654dcf7-vxfhz\" (UID: \"37f0b65a-a3ee-4aee-a490-3c9888b7475f\") " pod="openshift-ingress/maas-default-gateway-openshift-default-56654dcf7-vxfhz" Jul 12 18:02:09.701056 ip-10-0-129-128 kubenswrapper[2559]: I0712 18:02:09.700897 2559 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/37f0b65a-a3ee-4aee-a490-3c9888b7475f-istio-envoy\") pod \"maas-default-gateway-openshift-default-56654dcf7-vxfhz\" (UID: \"37f0b65a-a3ee-4aee-a490-3c9888b7475f\") " pod="openshift-ingress/maas-default-gateway-openshift-default-56654dcf7-vxfhz" Jul 12 18:02:09.701056 ip-10-0-129-128 kubenswrapper[2559]: I0712 18:02:09.700922 2559 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-t7qwn\" (UniqueName: \"kubernetes.io/projected/37f0b65a-a3ee-4aee-a490-3c9888b7475f-kube-api-access-t7qwn\") pod \"maas-default-gateway-openshift-default-56654dcf7-vxfhz\" (UID: \"37f0b65a-a3ee-4aee-a490-3c9888b7475f\") " pod="openshift-ingress/maas-default-gateway-openshift-default-56654dcf7-vxfhz" Jul 12 18:02:09.701056 ip-10-0-129-128 kubenswrapper[2559]: I0712 18:02:09.700966 2559 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/37f0b65a-a3ee-4aee-a490-3c9888b7475f-istio-token\") pod \"maas-default-gateway-openshift-default-56654dcf7-vxfhz\" (UID: \"37f0b65a-a3ee-4aee-a490-3c9888b7475f\") " pod="openshift-ingress/maas-default-gateway-openshift-default-56654dcf7-vxfhz" Jul 12 18:02:09.701367 ip-10-0-129-128 kubenswrapper[2559]: I0712 18:02:09.701158 2559 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/37f0b65a-a3ee-4aee-a490-3c9888b7475f-workload-socket\") pod \"maas-default-gateway-openshift-default-56654dcf7-vxfhz\" (UID: \"37f0b65a-a3ee-4aee-a490-3c9888b7475f\") " pod="openshift-ingress/maas-default-gateway-openshift-default-56654dcf7-vxfhz" Jul 12 18:02:09.701519 ip-10-0-129-128 kubenswrapper[2559]: I0712 18:02:09.701489 2559 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/37f0b65a-a3ee-4aee-a490-3c9888b7475f-credential-socket\") pod \"maas-default-gateway-openshift-default-56654dcf7-vxfhz\" (UID: \"37f0b65a-a3ee-4aee-a490-3c9888b7475f\") " pod="openshift-ingress/maas-default-gateway-openshift-default-56654dcf7-vxfhz" Jul 12 18:02:09.701658 ip-10-0-129-128 kubenswrapper[2559]: I0712 18:02:09.701518 2559 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/37f0b65a-a3ee-4aee-a490-3c9888b7475f-istio-data\") pod \"maas-default-gateway-openshift-default-56654dcf7-vxfhz\" (UID: \"37f0b65a-a3ee-4aee-a490-3c9888b7475f\") " pod="openshift-ingress/maas-default-gateway-openshift-default-56654dcf7-vxfhz" Jul 12 18:02:09.701733 ip-10-0-129-128 kubenswrapper[2559]: I0712 18:02:09.701678 2559 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/37f0b65a-a3ee-4aee-a490-3c9888b7475f-workload-certs\") pod \"maas-default-gateway-openshift-default-56654dcf7-vxfhz\" (UID: \"37f0b65a-a3ee-4aee-a490-3c9888b7475f\") " pod="openshift-ingress/maas-default-gateway-openshift-default-56654dcf7-vxfhz" Jul 12 18:02:09.702061 ip-10-0-129-128 kubenswrapper[2559]: I0712 18:02:09.702037 2559 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/37f0b65a-a3ee-4aee-a490-3c9888b7475f-istiod-ca-cert\") pod \"maas-default-gateway-openshift-default-56654dcf7-vxfhz\" (UID: \"37f0b65a-a3ee-4aee-a490-3c9888b7475f\") " pod="openshift-ingress/maas-default-gateway-openshift-default-56654dcf7-vxfhz" Jul 12 18:02:09.703340 ip-10-0-129-128 kubenswrapper[2559]: I0712 18:02:09.703317 2559 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/37f0b65a-a3ee-4aee-a490-3c9888b7475f-istio-envoy\") pod \"maas-default-gateway-openshift-default-56654dcf7-vxfhz\" (UID: \"37f0b65a-a3ee-4aee-a490-3c9888b7475f\") " pod="openshift-ingress/maas-default-gateway-openshift-default-56654dcf7-vxfhz" Jul 12 18:02:09.703441 ip-10-0-129-128 kubenswrapper[2559]: I0712 18:02:09.703419 2559 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/37f0b65a-a3ee-4aee-a490-3c9888b7475f-istio-podinfo\") pod \"maas-default-gateway-openshift-default-56654dcf7-vxfhz\" (UID: \"37f0b65a-a3ee-4aee-a490-3c9888b7475f\") " pod="openshift-ingress/maas-default-gateway-openshift-default-56654dcf7-vxfhz" Jul 12 18:02:09.708542 ip-10-0-129-128 kubenswrapper[2559]: I0712 18:02:09.708517 2559 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/37f0b65a-a3ee-4aee-a490-3c9888b7475f-istio-token\") pod \"maas-default-gateway-openshift-default-56654dcf7-vxfhz\" (UID: \"37f0b65a-a3ee-4aee-a490-3c9888b7475f\") " pod="openshift-ingress/maas-default-gateway-openshift-default-56654dcf7-vxfhz" Jul 12 18:02:09.708542 ip-10-0-129-128 kubenswrapper[2559]: I0712 18:02:09.708533 2559 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-t7qwn\" (UniqueName: \"kubernetes.io/projected/37f0b65a-a3ee-4aee-a490-3c9888b7475f-kube-api-access-t7qwn\") pod \"maas-default-gateway-openshift-default-56654dcf7-vxfhz\" (UID: \"37f0b65a-a3ee-4aee-a490-3c9888b7475f\") " pod="openshift-ingress/maas-default-gateway-openshift-default-56654dcf7-vxfhz" Jul 12 18:02:09.764300 ip-10-0-129-128 kubenswrapper[2559]: I0712 18:02:09.764258 2559 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/maas-default-gateway-openshift-default-56654dcf7-vxfhz" Jul 12 18:02:09.903985 ip-10-0-129-128 kubenswrapper[2559]: I0712 18:02:09.903958 2559 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/maas-default-gateway-openshift-default-56654dcf7-vxfhz"] Jul 12 18:02:09.905607 ip-10-0-129-128 kubenswrapper[2559]: W0712 18:02:09.905582 2559 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod37f0b65a_a3ee_4aee_a490_3c9888b7475f.slice/crio-a094de5454dfe3c4d39253916658579b0ce5e1ee8b0fe1a584ff44ecdf2b8f7a WatchSource:0}: Error finding container a094de5454dfe3c4d39253916658579b0ce5e1ee8b0fe1a584ff44ecdf2b8f7a: Status 404 returned error can't find the container with id a094de5454dfe3c4d39253916658579b0ce5e1ee8b0fe1a584ff44ecdf2b8f7a Jul 12 18:02:09.907695 ip-10-0-129-128 kubenswrapper[2559]: I0712 18:02:09.907663 2559 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236228Ki","pods":"250"} Jul 12 18:02:09.907787 ip-10-0-129-128 kubenswrapper[2559]: I0712 18:02:09.907746 2559 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236228Ki","pods":"250"} Jul 12 18:02:09.907852 ip-10-0-129-128 kubenswrapper[2559]: I0712 18:02:09.907793 2559 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236228Ki","pods":"250"} Jul 12 18:02:10.531013 ip-10-0-129-128 kubenswrapper[2559]: I0712 18:02:10.530977 2559 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/maas-default-gateway-openshift-default-56654dcf7-vxfhz" event={"ID":"37f0b65a-a3ee-4aee-a490-3c9888b7475f","Type":"ContainerStarted","Data":"b8f2e372668b19b65a86c0035b54517eeedb99611123396347490a53c2c81560"} Jul 12 18:02:10.531497 ip-10-0-129-128 kubenswrapper[2559]: I0712 18:02:10.531027 2559 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/maas-default-gateway-openshift-default-56654dcf7-vxfhz" event={"ID":"37f0b65a-a3ee-4aee-a490-3c9888b7475f","Type":"ContainerStarted","Data":"a094de5454dfe3c4d39253916658579b0ce5e1ee8b0fe1a584ff44ecdf2b8f7a"} Jul 12 18:02:10.554000 ip-10-0-129-128 kubenswrapper[2559]: I0712 18:02:10.553942 2559 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-ingress/maas-default-gateway-openshift-default-56654dcf7-vxfhz" podStartSLOduration=1.553926355 podStartE2EDuration="1.553926355s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-12 18:02:09 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-12 18:02:10.551436201 +0000 UTC m=+538.045504130" watchObservedRunningTime="2026-07-12 18:02:10.553926355 +0000 UTC m=+538.047994285" Jul 12 18:02:10.765039 ip-10-0-129-128 kubenswrapper[2559]: I0712 18:02:10.765003 2559 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="openshift-ingress/maas-default-gateway-openshift-default-56654dcf7-vxfhz" Jul 12 18:02:10.769667 ip-10-0-129-128 kubenswrapper[2559]: I0712 18:02:10.769630 2559 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="openshift-ingress/maas-default-gateway-openshift-default-56654dcf7-vxfhz" Jul 12 18:02:11.536389 ip-10-0-129-128 kubenswrapper[2559]: I0712 18:02:11.536357 2559 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/maas-default-gateway-openshift-default-56654dcf7-vxfhz" Jul 12 18:02:11.537344 ip-10-0-129-128 kubenswrapper[2559]: I0712 18:02:11.537325 2559 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/maas-default-gateway-openshift-default-56654dcf7-vxfhz" Jul 12 18:02:23.892074 ip-10-0-129-128 kubenswrapper[2559]: I0712 18:02:23.892036 2559 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-pwsvh"] Jul 12 18:02:23.897248 ip-10-0-129-128 kubenswrapper[2559]: I0712 18:02:23.897223 2559 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/limitador-limitador-54d6d988d5-pwsvh" Jul 12 18:02:23.899958 ip-10-0-129-128 kubenswrapper[2559]: I0712 18:02:23.899933 2559 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"kuadrant-system\"/\"limitador-limits-config-limitador\"" Jul 12 18:02:23.900088 ip-10-0-129-128 kubenswrapper[2559]: I0712 18:02:23.900004 2559 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"kuadrant-system\"/\"default-dockercfg-6bjmk\"" Jul 12 18:02:23.905715 ip-10-0-129-128 kubenswrapper[2559]: I0712 18:02:23.905691 2559 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-pwsvh"] Jul 12 18:02:23.920881 ip-10-0-129-128 kubenswrapper[2559]: I0712 18:02:23.920850 2559 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"config-file\" (UniqueName: \"kubernetes.io/configmap/645f2a0a-1588-47a1-a7c9-ab7eaeec4326-config-file\") pod \"limitador-limitador-54d6d988d5-pwsvh\" (UID: \"645f2a0a-1588-47a1-a7c9-ab7eaeec4326\") " pod="kuadrant-system/limitador-limitador-54d6d988d5-pwsvh" Jul 12 18:02:23.921002 ip-10-0-129-128 kubenswrapper[2559]: I0712 18:02:23.920919 2559 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-gg6gh\" (UniqueName: \"kubernetes.io/projected/645f2a0a-1588-47a1-a7c9-ab7eaeec4326-kube-api-access-gg6gh\") pod \"limitador-limitador-54d6d988d5-pwsvh\" (UID: \"645f2a0a-1588-47a1-a7c9-ab7eaeec4326\") " pod="kuadrant-system/limitador-limitador-54d6d988d5-pwsvh" Jul 12 18:02:23.933184 ip-10-0-129-128 kubenswrapper[2559]: I0712 18:02:23.933159 2559 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-pwsvh"] Jul 12 18:02:24.022011 ip-10-0-129-128 kubenswrapper[2559]: I0712 18:02:24.021977 2559 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-gg6gh\" (UniqueName: \"kubernetes.io/projected/645f2a0a-1588-47a1-a7c9-ab7eaeec4326-kube-api-access-gg6gh\") pod \"limitador-limitador-54d6d988d5-pwsvh\" (UID: \"645f2a0a-1588-47a1-a7c9-ab7eaeec4326\") " pod="kuadrant-system/limitador-limitador-54d6d988d5-pwsvh" Jul 12 18:02:24.022218 ip-10-0-129-128 kubenswrapper[2559]: I0712 18:02:24.022084 2559 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"config-file\" (UniqueName: \"kubernetes.io/configmap/645f2a0a-1588-47a1-a7c9-ab7eaeec4326-config-file\") pod \"limitador-limitador-54d6d988d5-pwsvh\" (UID: \"645f2a0a-1588-47a1-a7c9-ab7eaeec4326\") " pod="kuadrant-system/limitador-limitador-54d6d988d5-pwsvh" Jul 12 18:02:24.022737 ip-10-0-129-128 kubenswrapper[2559]: I0712 18:02:24.022713 2559 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"config-file\" (UniqueName: \"kubernetes.io/configmap/645f2a0a-1588-47a1-a7c9-ab7eaeec4326-config-file\") pod \"limitador-limitador-54d6d988d5-pwsvh\" (UID: \"645f2a0a-1588-47a1-a7c9-ab7eaeec4326\") " pod="kuadrant-system/limitador-limitador-54d6d988d5-pwsvh" Jul 12 18:02:24.029895 ip-10-0-129-128 kubenswrapper[2559]: I0712 18:02:24.029865 2559 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-gg6gh\" (UniqueName: \"kubernetes.io/projected/645f2a0a-1588-47a1-a7c9-ab7eaeec4326-kube-api-access-gg6gh\") pod \"limitador-limitador-54d6d988d5-pwsvh\" (UID: \"645f2a0a-1588-47a1-a7c9-ab7eaeec4326\") " pod="kuadrant-system/limitador-limitador-54d6d988d5-pwsvh" Jul 12 18:02:24.209271 ip-10-0-129-128 kubenswrapper[2559]: I0712 18:02:24.209188 2559 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/limitador-limitador-54d6d988d5-pwsvh" Jul 12 18:02:24.341363 ip-10-0-129-128 kubenswrapper[2559]: I0712 18:02:24.341294 2559 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-pwsvh"] Jul 12 18:02:24.343696 ip-10-0-129-128 kubenswrapper[2559]: W0712 18:02:24.343667 2559 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod645f2a0a_1588_47a1_a7c9_ab7eaeec4326.slice/crio-187601aedcc09e34d54cb21c2c0ca2662606fb4221ba63ec4480afe6bff8858f WatchSource:0}: Error finding container 187601aedcc09e34d54cb21c2c0ca2662606fb4221ba63ec4480afe6bff8858f: Status 404 returned error can't find the container with id 187601aedcc09e34d54cb21c2c0ca2662606fb4221ba63ec4480afe6bff8858f Jul 12 18:02:24.598454 ip-10-0-129-128 kubenswrapper[2559]: I0712 18:02:24.598420 2559 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/limitador-limitador-54d6d988d5-pwsvh" event={"ID":"645f2a0a-1588-47a1-a7c9-ab7eaeec4326","Type":"ContainerStarted","Data":"187601aedcc09e34d54cb21c2c0ca2662606fb4221ba63ec4480afe6bff8858f"} Jul 12 18:02:27.619002 ip-10-0-129-128 kubenswrapper[2559]: I0712 18:02:27.618964 2559 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/limitador-limitador-54d6d988d5-pwsvh" event={"ID":"645f2a0a-1588-47a1-a7c9-ab7eaeec4326","Type":"ContainerStarted","Data":"4468205c2a9e887346174614ed500300e744efc9e209aeb27c0a26e8e10e29df"} Jul 12 18:02:27.619392 ip-10-0-129-128 kubenswrapper[2559]: I0712 18:02:27.619105 2559 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="kuadrant-system/limitador-limitador-54d6d988d5-pwsvh" Jul 12 18:02:27.636593 ip-10-0-129-128 kubenswrapper[2559]: I0712 18:02:27.636538 2559 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="kuadrant-system/limitador-limitador-54d6d988d5-pwsvh" podStartSLOduration=2.258475478 podStartE2EDuration="4.636521994s" totalImagesPullingTime="2.378046516s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-12 18:02:23 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-12 18:02:27.634814141 +0000 UTC m=+555.128882075" watchObservedRunningTime="2026-07-12 18:02:27.636521994 +0000 UTC m=+555.130589924" Jul 12 18:02:38.625034 ip-10-0-129-128 kubenswrapper[2559]: I0712 18:02:38.625005 2559 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="kuadrant-system/limitador-limitador-54d6d988d5-pwsvh" Jul 12 18:02:56.275628 ip-10-0-129-128 kubenswrapper[2559]: I0712 18:02:56.275589 2559 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["kuadrant-system/authorino-5bb7b5df65-nvkvk"] Jul 12 18:02:56.284111 ip-10-0-129-128 kubenswrapper[2559]: I0712 18:02:56.284083 2559 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/authorino-5bb7b5df65-nvkvk" Jul 12 18:02:56.285367 ip-10-0-129-128 kubenswrapper[2559]: I0712 18:02:56.285336 2559 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/authorino-5bb7b5df65-nvkvk"] Jul 12 18:02:56.287672 ip-10-0-129-128 kubenswrapper[2559]: I0712 18:02:56.287627 2559 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"kuadrant-system\"/\"authorino-authorino-dockercfg-bm6zs\"" Jul 12 18:02:56.287790 ip-10-0-129-128 kubenswrapper[2559]: I0712 18:02:56.287699 2559 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"kuadrant-system\"/\"authorino-server-cert\"" Jul 12 18:02:56.403489 ip-10-0-129-128 kubenswrapper[2559]: I0712 18:02:56.403456 2559 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-gn9hk\" (UniqueName: \"kubernetes.io/projected/3a9b6543-4bd9-43ff-bcea-94566d038ab7-kube-api-access-gn9hk\") pod \"authorino-5bb7b5df65-nvkvk\" (UID: \"3a9b6543-4bd9-43ff-bcea-94566d038ab7\") " pod="kuadrant-system/authorino-5bb7b5df65-nvkvk" Jul 12 18:02:56.403679 ip-10-0-129-128 kubenswrapper[2559]: I0712 18:02:56.403501 2559 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tls-cert\" (UniqueName: \"kubernetes.io/secret/3a9b6543-4bd9-43ff-bcea-94566d038ab7-tls-cert\") pod \"authorino-5bb7b5df65-nvkvk\" (UID: \"3a9b6543-4bd9-43ff-bcea-94566d038ab7\") " pod="kuadrant-system/authorino-5bb7b5df65-nvkvk" Jul 12 18:02:56.504363 ip-10-0-129-128 kubenswrapper[2559]: I0712 18:02:56.504317 2559 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-gn9hk\" (UniqueName: \"kubernetes.io/projected/3a9b6543-4bd9-43ff-bcea-94566d038ab7-kube-api-access-gn9hk\") pod \"authorino-5bb7b5df65-nvkvk\" (UID: \"3a9b6543-4bd9-43ff-bcea-94566d038ab7\") " pod="kuadrant-system/authorino-5bb7b5df65-nvkvk" Jul 12 18:02:56.504363 ip-10-0-129-128 kubenswrapper[2559]: I0712 18:02:56.504390 2559 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tls-cert\" (UniqueName: \"kubernetes.io/secret/3a9b6543-4bd9-43ff-bcea-94566d038ab7-tls-cert\") pod \"authorino-5bb7b5df65-nvkvk\" (UID: \"3a9b6543-4bd9-43ff-bcea-94566d038ab7\") " pod="kuadrant-system/authorino-5bb7b5df65-nvkvk" Jul 12 18:02:56.507337 ip-10-0-129-128 kubenswrapper[2559]: I0712 18:02:56.507310 2559 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"tls-cert\" (UniqueName: \"kubernetes.io/secret/3a9b6543-4bd9-43ff-bcea-94566d038ab7-tls-cert\") pod \"authorino-5bb7b5df65-nvkvk\" (UID: \"3a9b6543-4bd9-43ff-bcea-94566d038ab7\") " pod="kuadrant-system/authorino-5bb7b5df65-nvkvk" Jul 12 18:02:56.513221 ip-10-0-129-128 kubenswrapper[2559]: I0712 18:02:56.513196 2559 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-gn9hk\" (UniqueName: \"kubernetes.io/projected/3a9b6543-4bd9-43ff-bcea-94566d038ab7-kube-api-access-gn9hk\") pod \"authorino-5bb7b5df65-nvkvk\" (UID: \"3a9b6543-4bd9-43ff-bcea-94566d038ab7\") " pod="kuadrant-system/authorino-5bb7b5df65-nvkvk" Jul 12 18:02:56.594736 ip-10-0-129-128 kubenswrapper[2559]: I0712 18:02:56.594627 2559 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/authorino-5bb7b5df65-nvkvk" Jul 12 18:02:56.728317 ip-10-0-129-128 kubenswrapper[2559]: I0712 18:02:56.728293 2559 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/authorino-5bb7b5df65-nvkvk"] Jul 12 18:02:56.729965 ip-10-0-129-128 kubenswrapper[2559]: W0712 18:02:56.729934 2559 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-besteffort.slice/kubepods-besteffort-pod3a9b6543_4bd9_43ff_bcea_94566d038ab7.slice/crio-ca65640101c46110fb9cfdee67a79eb2e111a7cf0d9405fa0535855d9e5d8d1f WatchSource:0}: Error finding container ca65640101c46110fb9cfdee67a79eb2e111a7cf0d9405fa0535855d9e5d8d1f: Status 404 returned error can't find the container with id ca65640101c46110fb9cfdee67a79eb2e111a7cf0d9405fa0535855d9e5d8d1f Jul 12 18:02:56.757701 ip-10-0-129-128 kubenswrapper[2559]: I0712 18:02:56.757673 2559 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/authorino-5bb7b5df65-nvkvk" event={"ID":"3a9b6543-4bd9-43ff-bcea-94566d038ab7","Type":"ContainerStarted","Data":"ca65640101c46110fb9cfdee67a79eb2e111a7cf0d9405fa0535855d9e5d8d1f"} Jul 12 18:02:59.776457 ip-10-0-129-128 kubenswrapper[2559]: I0712 18:02:59.776421 2559 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/authorino-5bb7b5df65-nvkvk" event={"ID":"3a9b6543-4bd9-43ff-bcea-94566d038ab7","Type":"ContainerStarted","Data":"e0d862ce5ff51b193ccbf7b8c872250160b04a22528b0442f16bbab48c3db529"} Jul 12 18:02:59.790787 ip-10-0-129-128 kubenswrapper[2559]: I0712 18:02:59.790740 2559 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="kuadrant-system/authorino-5bb7b5df65-nvkvk" podStartSLOduration=0.910243613 podStartE2EDuration="3.790723752s" totalImagesPullingTime="2.880480139s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-12 18:02:56 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-12 18:02:59.789886536 +0000 UTC m=+587.283954490" watchObservedRunningTime="2026-07-12 18:02:59.790723752 +0000 UTC m=+587.284791681" Jul 12 18:03:12.944572 ip-10-0-129-128 kubenswrapper[2559]: I0712 18:03:12.944547 2559 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-console-operator_console-operator-7866497494-wlk78_f1d46430-bb60-4470-b8d8-eacd44f4f6b1/console-operator/1.log" Jul 12 18:03:12.946084 ip-10-0-129-128 kubenswrapper[2559]: I0712 18:03:12.946062 2559 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-console-operator_console-operator-7866497494-wlk78_f1d46430-bb60-4470-b8d8-eacd44f4f6b1/console-operator/1.log" Jul 12 18:03:12.947041 ip-10-0-129-128 kubenswrapper[2559]: I0712 18:03:12.947024 2559 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ovn-kubernetes_ovnkube-node-q5dw8_e51bad50-bc74-4e38-8307-ca257b599314/ovn-acl-logging/0.log" Jul 12 18:03:12.948500 ip-10-0-129-128 kubenswrapper[2559]: I0712 18:03:12.948482 2559 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ovn-kubernetes_ovnkube-node-q5dw8_e51bad50-bc74-4e38-8307-ca257b599314/ovn-acl-logging/0.log" Jul 12 18:03:41.126215 ip-10-0-129-128 kubenswrapper[2559]: I0712 18:03:41.126175 2559 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-pwsvh"] Jul 12 18:03:48.999282 ip-10-0-129-128 kubenswrapper[2559]: I0712 18:03:48.999195 2559 generic.go:356] "Generic (PLEG): container finished" podID="37f0b65a-a3ee-4aee-a490-3c9888b7475f" containerID="b8f2e372668b19b65a86c0035b54517eeedb99611123396347490a53c2c81560" exitCode=0 Jul 12 18:03:48.999282 ip-10-0-129-128 kubenswrapper[2559]: I0712 18:03:48.999245 2559 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/maas-default-gateway-openshift-default-56654dcf7-vxfhz" event={"ID":"37f0b65a-a3ee-4aee-a490-3c9888b7475f","Type":"ContainerDied","Data":"b8f2e372668b19b65a86c0035b54517eeedb99611123396347490a53c2c81560"} Jul 12 18:03:48.999729 ip-10-0-129-128 kubenswrapper[2559]: I0712 18:03:48.999664 2559 scope.go:117] "RemoveContainer" containerID="b8f2e372668b19b65a86c0035b54517eeedb99611123396347490a53c2c81560" Jul 12 18:03:49.000726 ip-10-0-129-128 kubenswrapper[2559]: I0712 18:03:49.000700 2559 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236228Ki","pods":"250"} Jul 12 18:03:49.000791 ip-10-0-129-128 kubenswrapper[2559]: I0712 18:03:49.000765 2559 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236228Ki","pods":"250"} Jul 12 18:03:49.000830 ip-10-0-129-128 kubenswrapper[2559]: I0712 18:03:49.000796 2559 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236228Ki","pods":"250"} Jul 12 18:03:50.006138 ip-10-0-129-128 kubenswrapper[2559]: I0712 18:03:50.006104 2559 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/maas-default-gateway-openshift-default-56654dcf7-vxfhz" event={"ID":"37f0b65a-a3ee-4aee-a490-3c9888b7475f","Type":"ContainerStarted","Data":"dcca51053b6f3ee20adcf94f043daa8dde778a18c47790e48b8843b0906dcd68"} Jul 12 18:03:50.765351 ip-10-0-129-128 kubenswrapper[2559]: I0712 18:03:50.765314 2559 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="openshift-ingress/maas-default-gateway-openshift-default-56654dcf7-vxfhz" Jul 12 18:03:50.765528 ip-10-0-129-128 kubenswrapper[2559]: I0712 18:03:50.765505 2559 patch_prober.go:28] interesting pod/maas-default-gateway-openshift-default-56654dcf7-vxfhz container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.132.0.37:15021/healthz/ready\": dial tcp 10.132.0.37:15021: connect: connection refused" start-of-body= Jul 12 18:03:50.765573 ip-10-0-129-128 kubenswrapper[2559]: I0712 18:03:50.765557 2559 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/maas-default-gateway-openshift-default-56654dcf7-vxfhz" podUID="37f0b65a-a3ee-4aee-a490-3c9888b7475f" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.132.0.37:15021/healthz/ready\": dial tcp 10.132.0.37:15021: connect: connection refused" Jul 12 18:03:51.764778 ip-10-0-129-128 kubenswrapper[2559]: I0712 18:03:51.764748 2559 patch_prober.go:28] interesting pod/maas-default-gateway-openshift-default-56654dcf7-vxfhz container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.132.0.37:15021/healthz/ready\": dial tcp 10.132.0.37:15021: connect: connection refused" start-of-body= Jul 12 18:03:51.765136 ip-10-0-129-128 kubenswrapper[2559]: I0712 18:03:51.764808 2559 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/maas-default-gateway-openshift-default-56654dcf7-vxfhz" podUID="37f0b65a-a3ee-4aee-a490-3c9888b7475f" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.132.0.37:15021/healthz/ready\": dial tcp 10.132.0.37:15021: connect: connection refused" Jul 12 18:03:52.765555 ip-10-0-129-128 kubenswrapper[2559]: I0712 18:03:52.765507 2559 patch_prober.go:28] interesting pod/maas-default-gateway-openshift-default-56654dcf7-vxfhz container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.132.0.37:15021/healthz/ready\": dial tcp 10.132.0.37:15021: connect: connection refused" start-of-body= Jul 12 18:03:52.766075 ip-10-0-129-128 kubenswrapper[2559]: I0712 18:03:52.765569 2559 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/maas-default-gateway-openshift-default-56654dcf7-vxfhz" podUID="37f0b65a-a3ee-4aee-a490-3c9888b7475f" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.132.0.37:15021/healthz/ready\": dial tcp 10.132.0.37:15021: connect: connection refused" Jul 12 18:03:53.764719 ip-10-0-129-128 kubenswrapper[2559]: I0712 18:03:53.764676 2559 patch_prober.go:28] interesting pod/maas-default-gateway-openshift-default-56654dcf7-vxfhz container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.132.0.37:15021/healthz/ready\": dial tcp 10.132.0.37:15021: connect: connection refused" start-of-body= Jul 12 18:03:53.764911 ip-10-0-129-128 kubenswrapper[2559]: I0712 18:03:53.764737 2559 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/maas-default-gateway-openshift-default-56654dcf7-vxfhz" podUID="37f0b65a-a3ee-4aee-a490-3c9888b7475f" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.132.0.37:15021/healthz/ready\": dial tcp 10.132.0.37:15021: connect: connection refused" Jul 12 18:03:54.768971 ip-10-0-129-128 kubenswrapper[2559]: I0712 18:03:54.768938 2559 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="openshift-ingress/maas-default-gateway-openshift-default-56654dcf7-vxfhz" Jul 12 18:03:54.769343 ip-10-0-129-128 kubenswrapper[2559]: I0712 18:03:54.769297 2559 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/maas-default-gateway-openshift-default-56654dcf7-vxfhz" Jul 12 18:03:54.769971 ip-10-0-129-128 kubenswrapper[2559]: I0712 18:03:54.769953 2559 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/maas-default-gateway-openshift-default-56654dcf7-vxfhz" Jul 12 18:04:04.303701 ip-10-0-129-128 kubenswrapper[2559]: I0712 18:04:04.303661 2559 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-pwsvh"] Jul 12 18:04:10.790554 ip-10-0-129-128 kubenswrapper[2559]: I0712 18:04:10.790517 2559 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-pwsvh"] Jul 12 18:04:18.292020 ip-10-0-129-128 kubenswrapper[2559]: I0712 18:04:18.291979 2559 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-pwsvh"] Jul 12 18:04:30.183101 ip-10-0-129-128 kubenswrapper[2559]: I0712 18:04:30.183059 2559 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-pwsvh"] Jul 12 18:04:38.783242 ip-10-0-129-128 kubenswrapper[2559]: I0712 18:04:38.783202 2559 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-pwsvh"] Jul 12 18:04:54.686175 ip-10-0-129-128 kubenswrapper[2559]: I0712 18:04:54.686135 2559 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-pwsvh"] Jul 12 18:06:15.000394 ip-10-0-129-128 kubenswrapper[2559]: I0712 18:06:15.000358 2559 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["kuadrant-system/authorino-7497fcb44d-xbwrz"] Jul 12 18:06:15.003688 ip-10-0-129-128 kubenswrapper[2559]: I0712 18:06:15.003663 2559 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/authorino-7497fcb44d-xbwrz" Jul 12 18:06:15.011099 ip-10-0-129-128 kubenswrapper[2559]: I0712 18:06:15.011071 2559 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/authorino-7497fcb44d-xbwrz"] Jul 12 18:06:15.126635 ip-10-0-129-128 kubenswrapper[2559]: I0712 18:06:15.126603 2559 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tls-cert\" (UniqueName: \"kubernetes.io/secret/411b22c6-e8d8-4930-827f-c2c94d231237-tls-cert\") pod \"authorino-7497fcb44d-xbwrz\" (UID: \"411b22c6-e8d8-4930-827f-c2c94d231237\") " pod="kuadrant-system/authorino-7497fcb44d-xbwrz" Jul 12 18:06:15.126819 ip-10-0-129-128 kubenswrapper[2559]: I0712 18:06:15.126666 2559 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-rbnbk\" (UniqueName: \"kubernetes.io/projected/411b22c6-e8d8-4930-827f-c2c94d231237-kube-api-access-rbnbk\") pod \"authorino-7497fcb44d-xbwrz\" (UID: \"411b22c6-e8d8-4930-827f-c2c94d231237\") " pod="kuadrant-system/authorino-7497fcb44d-xbwrz" Jul 12 18:06:15.227708 ip-10-0-129-128 kubenswrapper[2559]: I0712 18:06:15.227664 2559 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tls-cert\" (UniqueName: \"kubernetes.io/secret/411b22c6-e8d8-4930-827f-c2c94d231237-tls-cert\") pod \"authorino-7497fcb44d-xbwrz\" (UID: \"411b22c6-e8d8-4930-827f-c2c94d231237\") " pod="kuadrant-system/authorino-7497fcb44d-xbwrz" Jul 12 18:06:15.227708 ip-10-0-129-128 kubenswrapper[2559]: I0712 18:06:15.227708 2559 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-rbnbk\" (UniqueName: \"kubernetes.io/projected/411b22c6-e8d8-4930-827f-c2c94d231237-kube-api-access-rbnbk\") pod \"authorino-7497fcb44d-xbwrz\" (UID: \"411b22c6-e8d8-4930-827f-c2c94d231237\") " pod="kuadrant-system/authorino-7497fcb44d-xbwrz" Jul 12 18:06:15.230280 ip-10-0-129-128 kubenswrapper[2559]: I0712 18:06:15.230247 2559 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"tls-cert\" (UniqueName: \"kubernetes.io/secret/411b22c6-e8d8-4930-827f-c2c94d231237-tls-cert\") pod \"authorino-7497fcb44d-xbwrz\" (UID: \"411b22c6-e8d8-4930-827f-c2c94d231237\") " pod="kuadrant-system/authorino-7497fcb44d-xbwrz" Jul 12 18:06:15.239398 ip-10-0-129-128 kubenswrapper[2559]: I0712 18:06:15.239372 2559 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-rbnbk\" (UniqueName: \"kubernetes.io/projected/411b22c6-e8d8-4930-827f-c2c94d231237-kube-api-access-rbnbk\") pod \"authorino-7497fcb44d-xbwrz\" (UID: \"411b22c6-e8d8-4930-827f-c2c94d231237\") " pod="kuadrant-system/authorino-7497fcb44d-xbwrz" Jul 12 18:06:15.313542 ip-10-0-129-128 kubenswrapper[2559]: I0712 18:06:15.313505 2559 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/authorino-7497fcb44d-xbwrz" Jul 12 18:06:15.451544 ip-10-0-129-128 kubenswrapper[2559]: I0712 18:06:15.451512 2559 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/authorino-7497fcb44d-xbwrz"] Jul 12 18:06:15.453825 ip-10-0-129-128 kubenswrapper[2559]: W0712 18:06:15.453787 2559 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-besteffort.slice/kubepods-besteffort-pod411b22c6_e8d8_4930_827f_c2c94d231237.slice/crio-d1d3aa9f65644ffeea6cef5bc6fb61d5374fe47438123ebcbc450d960e30e036 WatchSource:0}: Error finding container d1d3aa9f65644ffeea6cef5bc6fb61d5374fe47438123ebcbc450d960e30e036: Status 404 returned error can't find the container with id d1d3aa9f65644ffeea6cef5bc6fb61d5374fe47438123ebcbc450d960e30e036 Jul 12 18:06:15.455179 ip-10-0-129-128 kubenswrapper[2559]: I0712 18:06:15.455159 2559 provider.go:93] Refreshing cache for provider: *credentialprovider.defaultDockerConfigProvider Jul 12 18:06:15.688713 ip-10-0-129-128 kubenswrapper[2559]: I0712 18:06:15.688599 2559 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/authorino-7497fcb44d-xbwrz" event={"ID":"411b22c6-e8d8-4930-827f-c2c94d231237","Type":"ContainerStarted","Data":"d1d3aa9f65644ffeea6cef5bc6fb61d5374fe47438123ebcbc450d960e30e036"} Jul 12 18:06:16.697243 ip-10-0-129-128 kubenswrapper[2559]: I0712 18:06:16.697205 2559 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/authorino-7497fcb44d-xbwrz" event={"ID":"411b22c6-e8d8-4930-827f-c2c94d231237","Type":"ContainerStarted","Data":"78fd1d15a09b64fe14cf5f8b126e263bf6fe8963479f9ca33b32a681480e21ca"} Jul 12 18:06:16.714896 ip-10-0-129-128 kubenswrapper[2559]: I0712 18:06:16.714836 2559 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="kuadrant-system/authorino-7497fcb44d-xbwrz" podStartSLOduration=2.3473309540000002 podStartE2EDuration="2.714813478s" totalImagesPullingTime="367.482524ms" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-12 18:06:14 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-12 18:06:16.714184249 +0000 UTC m=+784.208252181" watchObservedRunningTime="2026-07-12 18:06:16.714813478 +0000 UTC m=+784.208881409" Jul 12 18:06:16.743737 ip-10-0-129-128 kubenswrapper[2559]: I0712 18:06:16.743704 2559 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["kuadrant-system/authorino-5bb7b5df65-nvkvk"] Jul 12 18:06:16.743943 ip-10-0-129-128 kubenswrapper[2559]: I0712 18:06:16.743909 2559 kuberuntime_container.go:873] "Killing container with a grace period" pod="kuadrant-system/authorino-5bb7b5df65-nvkvk" podUID="3a9b6543-4bd9-43ff-bcea-94566d038ab7" containerName="authorino" containerID="cri-o://e0d862ce5ff51b193ccbf7b8c872250160b04a22528b0442f16bbab48c3db529" gracePeriod=30 Jul 12 18:06:17.008759 ip-10-0-129-128 kubenswrapper[2559]: I0712 18:06:17.008732 2559 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/authorino-5bb7b5df65-nvkvk" Jul 12 18:06:17.147739 ip-10-0-129-128 kubenswrapper[2559]: I0712 18:06:17.147707 2559 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"tls-cert\" (UniqueName: \"kubernetes.io/secret/3a9b6543-4bd9-43ff-bcea-94566d038ab7-tls-cert\") pod \"3a9b6543-4bd9-43ff-bcea-94566d038ab7\" (UID: \"3a9b6543-4bd9-43ff-bcea-94566d038ab7\") " Jul 12 18:06:17.147926 ip-10-0-129-128 kubenswrapper[2559]: I0712 18:06:17.147840 2559 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-gn9hk\" (UniqueName: \"kubernetes.io/projected/3a9b6543-4bd9-43ff-bcea-94566d038ab7-kube-api-access-gn9hk\") pod \"3a9b6543-4bd9-43ff-bcea-94566d038ab7\" (UID: \"3a9b6543-4bd9-43ff-bcea-94566d038ab7\") " Jul 12 18:06:17.149947 ip-10-0-129-128 kubenswrapper[2559]: I0712 18:06:17.149907 2559 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/3a9b6543-4bd9-43ff-bcea-94566d038ab7-kube-api-access-gn9hk" (OuterVolumeSpecName: "kube-api-access-gn9hk") pod "3a9b6543-4bd9-43ff-bcea-94566d038ab7" (UID: "3a9b6543-4bd9-43ff-bcea-94566d038ab7"). InnerVolumeSpecName "kube-api-access-gn9hk". PluginName "kubernetes.io/projected", VolumeGIDValue "" Jul 12 18:06:17.158767 ip-10-0-129-128 kubenswrapper[2559]: I0712 18:06:17.158734 2559 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/3a9b6543-4bd9-43ff-bcea-94566d038ab7-tls-cert" (OuterVolumeSpecName: "tls-cert") pod "3a9b6543-4bd9-43ff-bcea-94566d038ab7" (UID: "3a9b6543-4bd9-43ff-bcea-94566d038ab7"). InnerVolumeSpecName "tls-cert". PluginName "kubernetes.io/secret", VolumeGIDValue "" Jul 12 18:06:17.248988 ip-10-0-129-128 kubenswrapper[2559]: I0712 18:06:17.248907 2559 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-gn9hk\" (UniqueName: \"kubernetes.io/projected/3a9b6543-4bd9-43ff-bcea-94566d038ab7-kube-api-access-gn9hk\") on node \"ip-10-0-129-128.ec2.internal\" DevicePath \"\"" Jul 12 18:06:17.248988 ip-10-0-129-128 kubenswrapper[2559]: I0712 18:06:17.248937 2559 reconciler_common.go:299] "Volume detached for volume \"tls-cert\" (UniqueName: \"kubernetes.io/secret/3a9b6543-4bd9-43ff-bcea-94566d038ab7-tls-cert\") on node \"ip-10-0-129-128.ec2.internal\" DevicePath \"\"" Jul 12 18:06:17.703702 ip-10-0-129-128 kubenswrapper[2559]: I0712 18:06:17.703633 2559 generic.go:356] "Generic (PLEG): container finished" podID="3a9b6543-4bd9-43ff-bcea-94566d038ab7" containerID="e0d862ce5ff51b193ccbf7b8c872250160b04a22528b0442f16bbab48c3db529" exitCode=0 Jul 12 18:06:17.703702 ip-10-0-129-128 kubenswrapper[2559]: I0712 18:06:17.703698 2559 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/authorino-5bb7b5df65-nvkvk" event={"ID":"3a9b6543-4bd9-43ff-bcea-94566d038ab7","Type":"ContainerDied","Data":"e0d862ce5ff51b193ccbf7b8c872250160b04a22528b0442f16bbab48c3db529"} Jul 12 18:06:17.704206 ip-10-0-129-128 kubenswrapper[2559]: I0712 18:06:17.703732 2559 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/authorino-5bb7b5df65-nvkvk" event={"ID":"3a9b6543-4bd9-43ff-bcea-94566d038ab7","Type":"ContainerDied","Data":"ca65640101c46110fb9cfdee67a79eb2e111a7cf0d9405fa0535855d9e5d8d1f"} Jul 12 18:06:17.704206 ip-10-0-129-128 kubenswrapper[2559]: I0712 18:06:17.703748 2559 scope.go:117] "RemoveContainer" containerID="e0d862ce5ff51b193ccbf7b8c872250160b04a22528b0442f16bbab48c3db529" Jul 12 18:06:17.704206 ip-10-0-129-128 kubenswrapper[2559]: I0712 18:06:17.703681 2559 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/authorino-5bb7b5df65-nvkvk" Jul 12 18:06:17.713554 ip-10-0-129-128 kubenswrapper[2559]: I0712 18:06:17.713538 2559 scope.go:117] "RemoveContainer" containerID="e0d862ce5ff51b193ccbf7b8c872250160b04a22528b0442f16bbab48c3db529" Jul 12 18:06:17.713855 ip-10-0-129-128 kubenswrapper[2559]: E0712 18:06:17.713833 2559 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"e0d862ce5ff51b193ccbf7b8c872250160b04a22528b0442f16bbab48c3db529\": container with ID starting with e0d862ce5ff51b193ccbf7b8c872250160b04a22528b0442f16bbab48c3db529 not found: ID does not exist" containerID="e0d862ce5ff51b193ccbf7b8c872250160b04a22528b0442f16bbab48c3db529" Jul 12 18:06:17.713976 ip-10-0-129-128 kubenswrapper[2559]: I0712 18:06:17.713859 2559 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"e0d862ce5ff51b193ccbf7b8c872250160b04a22528b0442f16bbab48c3db529"} err="failed to get container status \"e0d862ce5ff51b193ccbf7b8c872250160b04a22528b0442f16bbab48c3db529\": rpc error: code = NotFound desc = could not find container \"e0d862ce5ff51b193ccbf7b8c872250160b04a22528b0442f16bbab48c3db529\": container with ID starting with e0d862ce5ff51b193ccbf7b8c872250160b04a22528b0442f16bbab48c3db529 not found: ID does not exist" Jul 12 18:06:17.731194 ip-10-0-129-128 kubenswrapper[2559]: I0712 18:06:17.731167 2559 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["kuadrant-system/authorino-5bb7b5df65-nvkvk"] Jul 12 18:06:17.744425 ip-10-0-129-128 kubenswrapper[2559]: I0712 18:06:17.744386 2559 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["kuadrant-system/authorino-5bb7b5df65-nvkvk"] Jul 12 18:06:19.034276 ip-10-0-129-128 kubenswrapper[2559]: I0712 18:06:19.034242 2559 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="3a9b6543-4bd9-43ff-bcea-94566d038ab7" path="/var/lib/kubelet/pods/3a9b6543-4bd9-43ff-bcea-94566d038ab7/volumes" Jul 12 18:06:43.889897 ip-10-0-129-128 kubenswrapper[2559]: I0712 18:06:43.889861 2559 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-pwsvh"] Jul 12 18:06:53.386531 ip-10-0-129-128 kubenswrapper[2559]: I0712 18:06:53.386493 2559 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-pwsvh"] Jul 12 18:07:04.795762 ip-10-0-129-128 kubenswrapper[2559]: I0712 18:07:04.795724 2559 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-pwsvh"] Jul 12 18:07:12.883591 ip-10-0-129-128 kubenswrapper[2559]: I0712 18:07:12.883551 2559 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-pwsvh"] Jul 12 18:07:24.588255 ip-10-0-129-128 kubenswrapper[2559]: I0712 18:07:24.588216 2559 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-pwsvh"] Jul 12 18:07:53.188954 ip-10-0-129-128 kubenswrapper[2559]: I0712 18:07:53.188920 2559 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-pwsvh"] Jul 12 18:08:04.485200 ip-10-0-129-128 kubenswrapper[2559]: I0712 18:08:04.485164 2559 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-pwsvh"] Jul 12 18:08:12.973169 ip-10-0-129-128 kubenswrapper[2559]: I0712 18:08:12.973085 2559 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-console-operator_console-operator-7866497494-wlk78_f1d46430-bb60-4470-b8d8-eacd44f4f6b1/console-operator/1.log" Jul 12 18:08:12.973607 ip-10-0-129-128 kubenswrapper[2559]: I0712 18:08:12.973253 2559 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-console-operator_console-operator-7866497494-wlk78_f1d46430-bb60-4470-b8d8-eacd44f4f6b1/console-operator/1.log" Jul 12 18:08:12.975836 ip-10-0-129-128 kubenswrapper[2559]: I0712 18:08:12.975816 2559 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ovn-kubernetes_ovnkube-node-q5dw8_e51bad50-bc74-4e38-8307-ca257b599314/ovn-acl-logging/0.log" Jul 12 18:08:12.975939 ip-10-0-129-128 kubenswrapper[2559]: I0712 18:08:12.975846 2559 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ovn-kubernetes_ovnkube-node-q5dw8_e51bad50-bc74-4e38-8307-ca257b599314/ovn-acl-logging/0.log" Jul 12 18:08:43.789974 ip-10-0-129-128 kubenswrapper[2559]: I0712 18:08:43.789941 2559 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-pwsvh"] Jul 12 18:08:52.989085 ip-10-0-129-128 kubenswrapper[2559]: I0712 18:08:52.989047 2559 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-pwsvh"] Jul 12 18:09:01.794966 ip-10-0-129-128 kubenswrapper[2559]: I0712 18:09:01.794925 2559 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-pwsvh"] Jul 12 18:09:12.886390 ip-10-0-129-128 kubenswrapper[2559]: I0712 18:09:12.886352 2559 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-pwsvh"] Jul 12 18:09:22.782458 ip-10-0-129-128 kubenswrapper[2559]: I0712 18:09:22.782421 2559 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-pwsvh"] Jul 12 18:09:34.597154 ip-10-0-129-128 kubenswrapper[2559]: I0712 18:09:34.597113 2559 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-pwsvh"] Jul 12 18:09:44.989882 ip-10-0-129-128 kubenswrapper[2559]: I0712 18:09:44.989842 2559 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-pwsvh"] Jul 12 18:09:54.887489 ip-10-0-129-128 kubenswrapper[2559]: I0712 18:09:54.887451 2559 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-pwsvh"] Jul 12 18:10:17.186357 ip-10-0-129-128 kubenswrapper[2559]: I0712 18:10:17.186321 2559 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-pwsvh"] Jul 12 18:10:29.185482 ip-10-0-129-128 kubenswrapper[2559]: I0712 18:10:29.185443 2559 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-pwsvh"] Jul 12 18:10:52.488253 ip-10-0-129-128 kubenswrapper[2559]: I0712 18:10:52.488218 2559 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-pwsvh"] Jul 12 18:11:03.483156 ip-10-0-129-128 kubenswrapper[2559]: I0712 18:11:03.483115 2559 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-pwsvh"] Jul 12 18:11:19.685871 ip-10-0-129-128 kubenswrapper[2559]: I0712 18:11:19.685787 2559 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-pwsvh"] Jul 12 18:11:30.688731 ip-10-0-129-128 kubenswrapper[2559]: I0712 18:11:30.688691 2559 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-pwsvh"] Jul 12 18:11:40.990815 ip-10-0-129-128 kubenswrapper[2559]: I0712 18:11:40.990775 2559 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-pwsvh"] Jul 12 18:11:50.289621 ip-10-0-129-128 kubenswrapper[2559]: I0712 18:11:50.289586 2559 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-pwsvh"] Jul 12 18:11:59.884321 ip-10-0-129-128 kubenswrapper[2559]: I0712 18:11:59.884285 2559 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-pwsvh"] Jul 12 18:12:24.413929 ip-10-0-129-128 kubenswrapper[2559]: I0712 18:12:24.413891 2559 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["kuadrant-system/kuadrant-operator-controller-manager-86878c769f-vbnpm"] Jul 12 18:12:24.414533 ip-10-0-129-128 kubenswrapper[2559]: I0712 18:12:24.414093 2559 kuberuntime_container.go:873] "Killing container with a grace period" pod="kuadrant-system/kuadrant-operator-controller-manager-86878c769f-vbnpm" podUID="38a75322-152d-48f3-90a0-e91a9f28137a" containerName="manager" containerID="cri-o://53c4dc728e27dee8ab1afd513e54ac6f73c05ab8825fc3c02b4d270bcef074bf" gracePeriod=10 Jul 12 18:12:27.980787 ip-10-0-129-128 kubenswrapper[2559]: I0712 18:12:27.980721 2559 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/kuadrant-operator-controller-manager-86878c769f-vbnpm" Jul 12 18:12:28.061722 ip-10-0-129-128 kubenswrapper[2559]: I0712 18:12:28.061689 2559 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-lzf6l\" (UniqueName: \"kubernetes.io/projected/38a75322-152d-48f3-90a0-e91a9f28137a-kube-api-access-lzf6l\") pod \"38a75322-152d-48f3-90a0-e91a9f28137a\" (UID: \"38a75322-152d-48f3-90a0-e91a9f28137a\") " Jul 12 18:12:28.061904 ip-10-0-129-128 kubenswrapper[2559]: I0712 18:12:28.061779 2559 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"extensions-socket-volume\" (UniqueName: \"kubernetes.io/empty-dir/38a75322-152d-48f3-90a0-e91a9f28137a-extensions-socket-volume\") pod \"38a75322-152d-48f3-90a0-e91a9f28137a\" (UID: \"38a75322-152d-48f3-90a0-e91a9f28137a\") " Jul 12 18:12:28.062158 ip-10-0-129-128 kubenswrapper[2559]: I0712 18:12:28.062131 2559 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/38a75322-152d-48f3-90a0-e91a9f28137a-extensions-socket-volume" (OuterVolumeSpecName: "extensions-socket-volume") pod "38a75322-152d-48f3-90a0-e91a9f28137a" (UID: "38a75322-152d-48f3-90a0-e91a9f28137a"). InnerVolumeSpecName "extensions-socket-volume". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Jul 12 18:12:28.063754 ip-10-0-129-128 kubenswrapper[2559]: I0712 18:12:28.063729 2559 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/38a75322-152d-48f3-90a0-e91a9f28137a-kube-api-access-lzf6l" (OuterVolumeSpecName: "kube-api-access-lzf6l") pod "38a75322-152d-48f3-90a0-e91a9f28137a" (UID: "38a75322-152d-48f3-90a0-e91a9f28137a"). InnerVolumeSpecName "kube-api-access-lzf6l". PluginName "kubernetes.io/projected", VolumeGIDValue "" Jul 12 18:12:28.163313 ip-10-0-129-128 kubenswrapper[2559]: I0712 18:12:28.163277 2559 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-lzf6l\" (UniqueName: \"kubernetes.io/projected/38a75322-152d-48f3-90a0-e91a9f28137a-kube-api-access-lzf6l\") on node \"ip-10-0-129-128.ec2.internal\" DevicePath \"\"" Jul 12 18:12:28.163313 ip-10-0-129-128 kubenswrapper[2559]: I0712 18:12:28.163309 2559 reconciler_common.go:299] "Volume detached for volume \"extensions-socket-volume\" (UniqueName: \"kubernetes.io/empty-dir/38a75322-152d-48f3-90a0-e91a9f28137a-extensions-socket-volume\") on node \"ip-10-0-129-128.ec2.internal\" DevicePath \"\"" Jul 12 18:12:28.510231 ip-10-0-129-128 kubenswrapper[2559]: I0712 18:12:28.510198 2559 generic.go:356] "Generic (PLEG): container finished" podID="38a75322-152d-48f3-90a0-e91a9f28137a" containerID="53c4dc728e27dee8ab1afd513e54ac6f73c05ab8825fc3c02b4d270bcef074bf" exitCode=0 Jul 12 18:12:28.510405 ip-10-0-129-128 kubenswrapper[2559]: I0712 18:12:28.510240 2559 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/kuadrant-operator-controller-manager-86878c769f-vbnpm" Jul 12 18:12:28.510405 ip-10-0-129-128 kubenswrapper[2559]: I0712 18:12:28.510283 2559 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/kuadrant-operator-controller-manager-86878c769f-vbnpm" event={"ID":"38a75322-152d-48f3-90a0-e91a9f28137a","Type":"ContainerDied","Data":"53c4dc728e27dee8ab1afd513e54ac6f73c05ab8825fc3c02b4d270bcef074bf"} Jul 12 18:12:28.510405 ip-10-0-129-128 kubenswrapper[2559]: I0712 18:12:28.510332 2559 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/kuadrant-operator-controller-manager-86878c769f-vbnpm" event={"ID":"38a75322-152d-48f3-90a0-e91a9f28137a","Type":"ContainerDied","Data":"fd17727ec9c5506afd53b30196351ba9e5d894d090d432ac385032bbca8c4059"} Jul 12 18:12:28.510405 ip-10-0-129-128 kubenswrapper[2559]: I0712 18:12:28.510354 2559 scope.go:117] "RemoveContainer" containerID="53c4dc728e27dee8ab1afd513e54ac6f73c05ab8825fc3c02b4d270bcef074bf" Jul 12 18:12:28.526570 ip-10-0-129-128 kubenswrapper[2559]: I0712 18:12:28.526544 2559 scope.go:117] "RemoveContainer" containerID="53c4dc728e27dee8ab1afd513e54ac6f73c05ab8825fc3c02b4d270bcef074bf" Jul 12 18:12:28.526990 ip-10-0-129-128 kubenswrapper[2559]: E0712 18:12:28.526957 2559 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"53c4dc728e27dee8ab1afd513e54ac6f73c05ab8825fc3c02b4d270bcef074bf\": container with ID starting with 53c4dc728e27dee8ab1afd513e54ac6f73c05ab8825fc3c02b4d270bcef074bf not found: ID does not exist" containerID="53c4dc728e27dee8ab1afd513e54ac6f73c05ab8825fc3c02b4d270bcef074bf" Jul 12 18:12:28.527073 ip-10-0-129-128 kubenswrapper[2559]: I0712 18:12:28.526998 2559 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"53c4dc728e27dee8ab1afd513e54ac6f73c05ab8825fc3c02b4d270bcef074bf"} err="failed to get container status \"53c4dc728e27dee8ab1afd513e54ac6f73c05ab8825fc3c02b4d270bcef074bf\": rpc error: code = NotFound desc = could not find container \"53c4dc728e27dee8ab1afd513e54ac6f73c05ab8825fc3c02b4d270bcef074bf\": container with ID starting with 53c4dc728e27dee8ab1afd513e54ac6f73c05ab8825fc3c02b4d270bcef074bf not found: ID does not exist" Jul 12 18:12:28.536199 ip-10-0-129-128 kubenswrapper[2559]: I0712 18:12:28.536176 2559 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["kuadrant-system/kuadrant-operator-controller-manager-86878c769f-vbnpm"] Jul 12 18:12:28.540838 ip-10-0-129-128 kubenswrapper[2559]: I0712 18:12:28.540817 2559 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["kuadrant-system/kuadrant-operator-controller-manager-86878c769f-vbnpm"] Jul 12 18:12:29.034101 ip-10-0-129-128 kubenswrapper[2559]: I0712 18:12:29.034068 2559 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="38a75322-152d-48f3-90a0-e91a9f28137a" path="/var/lib/kubelet/pods/38a75322-152d-48f3-90a0-e91a9f28137a/volumes" Jul 12 18:13:13.001073 ip-10-0-129-128 kubenswrapper[2559]: I0712 18:13:13.001034 2559 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-console-operator_console-operator-7866497494-wlk78_f1d46430-bb60-4470-b8d8-eacd44f4f6b1/console-operator/1.log" Jul 12 18:13:13.001659 ip-10-0-129-128 kubenswrapper[2559]: I0712 18:13:13.001602 2559 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-console-operator_console-operator-7866497494-wlk78_f1d46430-bb60-4470-b8d8-eacd44f4f6b1/console-operator/1.log" Jul 12 18:13:13.003378 ip-10-0-129-128 kubenswrapper[2559]: I0712 18:13:13.003358 2559 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ovn-kubernetes_ovnkube-node-q5dw8_e51bad50-bc74-4e38-8307-ca257b599314/ovn-acl-logging/0.log" Jul 12 18:13:13.003982 ip-10-0-129-128 kubenswrapper[2559]: I0712 18:13:13.003956 2559 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ovn-kubernetes_ovnkube-node-q5dw8_e51bad50-bc74-4e38-8307-ca257b599314/ovn-acl-logging/0.log" Jul 12 18:13:25.581089 ip-10-0-129-128 kubenswrapper[2559]: I0712 18:13:25.581053 2559 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["kuadrant-system/kuadrant-operator-controller-manager-86878c769f-q7zgh"] Jul 12 18:13:25.581534 ip-10-0-129-128 kubenswrapper[2559]: I0712 18:13:25.581412 2559 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="3a9b6543-4bd9-43ff-bcea-94566d038ab7" containerName="authorino" Jul 12 18:13:25.581534 ip-10-0-129-128 kubenswrapper[2559]: I0712 18:13:25.581422 2559 state_mem.go:107] "Deleted CPUSet assignment" podUID="3a9b6543-4bd9-43ff-bcea-94566d038ab7" containerName="authorino" Jul 12 18:13:25.581534 ip-10-0-129-128 kubenswrapper[2559]: I0712 18:13:25.581440 2559 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="38a75322-152d-48f3-90a0-e91a9f28137a" containerName="manager" Jul 12 18:13:25.581534 ip-10-0-129-128 kubenswrapper[2559]: I0712 18:13:25.581446 2559 state_mem.go:107] "Deleted CPUSet assignment" podUID="38a75322-152d-48f3-90a0-e91a9f28137a" containerName="manager" Jul 12 18:13:25.581534 ip-10-0-129-128 kubenswrapper[2559]: I0712 18:13:25.581507 2559 memory_manager.go:368] "RemoveStaleState removing state" podUID="38a75322-152d-48f3-90a0-e91a9f28137a" containerName="manager" Jul 12 18:13:25.581534 ip-10-0-129-128 kubenswrapper[2559]: I0712 18:13:25.581516 2559 memory_manager.go:368] "RemoveStaleState removing state" podUID="3a9b6543-4bd9-43ff-bcea-94566d038ab7" containerName="authorino" Jul 12 18:13:25.585609 ip-10-0-129-128 kubenswrapper[2559]: I0712 18:13:25.585592 2559 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/kuadrant-operator-controller-manager-86878c769f-q7zgh" Jul 12 18:13:25.589401 ip-10-0-129-128 kubenswrapper[2559]: I0712 18:13:25.589380 2559 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"kuadrant-system\"/\"kuadrant-operator-controller-manager-dockercfg-tjgw4\"" Jul 12 18:13:25.604918 ip-10-0-129-128 kubenswrapper[2559]: I0712 18:13:25.604893 2559 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/kuadrant-operator-controller-manager-86878c769f-q7zgh"] Jul 12 18:13:25.636468 ip-10-0-129-128 kubenswrapper[2559]: I0712 18:13:25.636437 2559 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"extensions-socket-volume\" (UniqueName: \"kubernetes.io/empty-dir/af76461c-8878-47bb-bae3-bc2d8eb02f76-extensions-socket-volume\") pod \"kuadrant-operator-controller-manager-86878c769f-q7zgh\" (UID: \"af76461c-8878-47bb-bae3-bc2d8eb02f76\") " pod="kuadrant-system/kuadrant-operator-controller-manager-86878c769f-q7zgh" Jul 12 18:13:25.636576 ip-10-0-129-128 kubenswrapper[2559]: I0712 18:13:25.636478 2559 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-k54mc\" (UniqueName: \"kubernetes.io/projected/af76461c-8878-47bb-bae3-bc2d8eb02f76-kube-api-access-k54mc\") pod \"kuadrant-operator-controller-manager-86878c769f-q7zgh\" (UID: \"af76461c-8878-47bb-bae3-bc2d8eb02f76\") " pod="kuadrant-system/kuadrant-operator-controller-manager-86878c769f-q7zgh" Jul 12 18:13:25.737213 ip-10-0-129-128 kubenswrapper[2559]: I0712 18:13:25.737178 2559 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"extensions-socket-volume\" (UniqueName: \"kubernetes.io/empty-dir/af76461c-8878-47bb-bae3-bc2d8eb02f76-extensions-socket-volume\") pod \"kuadrant-operator-controller-manager-86878c769f-q7zgh\" (UID: \"af76461c-8878-47bb-bae3-bc2d8eb02f76\") " pod="kuadrant-system/kuadrant-operator-controller-manager-86878c769f-q7zgh" Jul 12 18:13:25.737213 ip-10-0-129-128 kubenswrapper[2559]: I0712 18:13:25.737219 2559 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-k54mc\" (UniqueName: \"kubernetes.io/projected/af76461c-8878-47bb-bae3-bc2d8eb02f76-kube-api-access-k54mc\") pod \"kuadrant-operator-controller-manager-86878c769f-q7zgh\" (UID: \"af76461c-8878-47bb-bae3-bc2d8eb02f76\") " pod="kuadrant-system/kuadrant-operator-controller-manager-86878c769f-q7zgh" Jul 12 18:13:25.737565 ip-10-0-129-128 kubenswrapper[2559]: I0712 18:13:25.737545 2559 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"extensions-socket-volume\" (UniqueName: \"kubernetes.io/empty-dir/af76461c-8878-47bb-bae3-bc2d8eb02f76-extensions-socket-volume\") pod \"kuadrant-operator-controller-manager-86878c769f-q7zgh\" (UID: \"af76461c-8878-47bb-bae3-bc2d8eb02f76\") " pod="kuadrant-system/kuadrant-operator-controller-manager-86878c769f-q7zgh" Jul 12 18:13:25.745012 ip-10-0-129-128 kubenswrapper[2559]: I0712 18:13:25.744993 2559 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-k54mc\" (UniqueName: \"kubernetes.io/projected/af76461c-8878-47bb-bae3-bc2d8eb02f76-kube-api-access-k54mc\") pod \"kuadrant-operator-controller-manager-86878c769f-q7zgh\" (UID: \"af76461c-8878-47bb-bae3-bc2d8eb02f76\") " pod="kuadrant-system/kuadrant-operator-controller-manager-86878c769f-q7zgh" Jul 12 18:13:25.896905 ip-10-0-129-128 kubenswrapper[2559]: I0712 18:13:25.896820 2559 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/kuadrant-operator-controller-manager-86878c769f-q7zgh" Jul 12 18:13:26.037786 ip-10-0-129-128 kubenswrapper[2559]: I0712 18:13:26.037763 2559 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/kuadrant-operator-controller-manager-86878c769f-q7zgh"] Jul 12 18:13:26.039672 ip-10-0-129-128 kubenswrapper[2559]: W0712 18:13:26.039628 2559 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podaf76461c_8878_47bb_bae3_bc2d8eb02f76.slice/crio-71c17b3cd5d3c8fbfb72faad62e0d0f95a81d46d6f05712f7b776c134efdc116 WatchSource:0}: Error finding container 71c17b3cd5d3c8fbfb72faad62e0d0f95a81d46d6f05712f7b776c134efdc116: Status 404 returned error can't find the container with id 71c17b3cd5d3c8fbfb72faad62e0d0f95a81d46d6f05712f7b776c134efdc116 Jul 12 18:13:26.791136 ip-10-0-129-128 kubenswrapper[2559]: I0712 18:13:26.791099 2559 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/kuadrant-operator-controller-manager-86878c769f-q7zgh" event={"ID":"af76461c-8878-47bb-bae3-bc2d8eb02f76","Type":"ContainerStarted","Data":"7c4c2efc4b184e95016290280a292a44de1f9bcf95784ea943d14e97d4dc40bd"} Jul 12 18:13:26.791544 ip-10-0-129-128 kubenswrapper[2559]: I0712 18:13:26.791150 2559 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/kuadrant-operator-controller-manager-86878c769f-q7zgh" event={"ID":"af76461c-8878-47bb-bae3-bc2d8eb02f76","Type":"ContainerStarted","Data":"71c17b3cd5d3c8fbfb72faad62e0d0f95a81d46d6f05712f7b776c134efdc116"} Jul 12 18:13:26.791544 ip-10-0-129-128 kubenswrapper[2559]: I0712 18:13:26.791230 2559 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="kuadrant-system/kuadrant-operator-controller-manager-86878c769f-q7zgh" Jul 12 18:13:26.811816 ip-10-0-129-128 kubenswrapper[2559]: I0712 18:13:26.811768 2559 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="kuadrant-system/kuadrant-operator-controller-manager-86878c769f-q7zgh" podStartSLOduration=1.8117513870000002 podStartE2EDuration="1.811751387s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-12 18:13:25 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-12 18:13:26.80979252 +0000 UTC m=+1214.303860441" watchObservedRunningTime="2026-07-12 18:13:26.811751387 +0000 UTC m=+1214.305819316" Jul 12 18:13:37.798400 ip-10-0-129-128 kubenswrapper[2559]: I0712 18:13:37.798368 2559 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="kuadrant-system/kuadrant-operator-controller-manager-86878c769f-q7zgh" Jul 12 18:13:48.888628 ip-10-0-129-128 kubenswrapper[2559]: I0712 18:13:48.888589 2559 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-pwsvh"] Jul 12 18:13:58.093272 ip-10-0-129-128 kubenswrapper[2559]: I0712 18:13:58.093231 2559 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-pwsvh"] Jul 12 18:14:11.982926 ip-10-0-129-128 kubenswrapper[2559]: I0712 18:14:11.982841 2559 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-pwsvh"] Jul 12 18:14:21.184526 ip-10-0-129-128 kubenswrapper[2559]: I0712 18:14:21.184487 2559 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-pwsvh"] Jul 12 18:14:30.989055 ip-10-0-129-128 kubenswrapper[2559]: I0712 18:14:30.989017 2559 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-pwsvh"] Jul 12 18:14:39.387535 ip-10-0-129-128 kubenswrapper[2559]: I0712 18:14:39.387495 2559 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-pwsvh"] Jul 12 18:14:50.696536 ip-10-0-129-128 kubenswrapper[2559]: I0712 18:14:50.696500 2559 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-pwsvh"] Jul 12 18:15:01.989391 ip-10-0-129-128 kubenswrapper[2559]: I0712 18:15:01.989357 2559 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-pwsvh"] Jul 12 18:15:10.988463 ip-10-0-129-128 kubenswrapper[2559]: I0712 18:15:10.988424 2559 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-pwsvh"] Jul 12 18:15:22.284848 ip-10-0-129-128 kubenswrapper[2559]: I0712 18:15:22.284807 2559 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-pwsvh"] Jul 12 18:15:33.182846 ip-10-0-129-128 kubenswrapper[2559]: I0712 18:15:33.182805 2559 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-pwsvh"] Jul 12 18:15:40.791253 ip-10-0-129-128 kubenswrapper[2559]: I0712 18:15:40.791210 2559 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-pwsvh"] Jul 12 18:15:58.590288 ip-10-0-129-128 kubenswrapper[2559]: I0712 18:15:58.590251 2559 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-pwsvh"] Jul 12 18:16:07.994530 ip-10-0-129-128 kubenswrapper[2559]: I0712 18:16:07.994483 2559 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-pwsvh"] Jul 12 18:16:18.195585 ip-10-0-129-128 kubenswrapper[2559]: I0712 18:16:18.191955 2559 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-pwsvh"] Jul 12 18:16:27.691169 ip-10-0-129-128 kubenswrapper[2559]: I0712 18:16:27.691121 2559 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-pwsvh"] Jul 12 18:16:45.287577 ip-10-0-129-128 kubenswrapper[2559]: I0712 18:16:45.287540 2559 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-pwsvh"] Jul 12 18:16:56.992422 ip-10-0-129-128 kubenswrapper[2559]: I0712 18:16:56.992379 2559 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-pwsvh"] Jul 12 18:17:09.786663 ip-10-0-129-128 kubenswrapper[2559]: I0712 18:17:09.786561 2559 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-pwsvh"] Jul 12 18:17:20.195576 ip-10-0-129-128 kubenswrapper[2559]: I0712 18:17:20.195530 2559 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-pwsvh"] Jul 12 18:17:29.983969 ip-10-0-129-128 kubenswrapper[2559]: I0712 18:17:29.983933 2559 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-pwsvh"] Jul 12 18:17:39.584057 ip-10-0-129-128 kubenswrapper[2559]: I0712 18:17:39.584019 2559 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-pwsvh"] Jul 12 18:17:53.386817 ip-10-0-129-128 kubenswrapper[2559]: I0712 18:17:53.386770 2559 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-pwsvh"] Jul 12 18:18:02.894860 ip-10-0-129-128 kubenswrapper[2559]: I0712 18:18:02.894823 2559 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-pwsvh"] Jul 12 18:18:12.692279 ip-10-0-129-128 kubenswrapper[2559]: I0712 18:18:12.692234 2559 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-pwsvh"] Jul 12 18:18:13.029725 ip-10-0-129-128 kubenswrapper[2559]: I0712 18:18:13.029692 2559 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-console-operator_console-operator-7866497494-wlk78_f1d46430-bb60-4470-b8d8-eacd44f4f6b1/console-operator/1.log" Jul 12 18:18:13.032107 ip-10-0-129-128 kubenswrapper[2559]: I0712 18:18:13.032073 2559 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-console-operator_console-operator-7866497494-wlk78_f1d46430-bb60-4470-b8d8-eacd44f4f6b1/console-operator/1.log" Jul 12 18:18:13.033871 ip-10-0-129-128 kubenswrapper[2559]: I0712 18:18:13.033850 2559 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ovn-kubernetes_ovnkube-node-q5dw8_e51bad50-bc74-4e38-8307-ca257b599314/ovn-acl-logging/0.log" Jul 12 18:18:13.035459 ip-10-0-129-128 kubenswrapper[2559]: I0712 18:18:13.035435 2559 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ovn-kubernetes_ovnkube-node-q5dw8_e51bad50-bc74-4e38-8307-ca257b599314/ovn-acl-logging/0.log" Jul 12 18:18:25.684651 ip-10-0-129-128 kubenswrapper[2559]: I0712 18:18:25.684603 2559 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-pwsvh"] Jul 12 18:18:35.086239 ip-10-0-129-128 kubenswrapper[2559]: I0712 18:18:35.086195 2559 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-pwsvh"] Jul 12 18:18:42.082894 ip-10-0-129-128 kubenswrapper[2559]: I0712 18:18:42.082858 2559 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-pwsvh"] Jul 12 18:18:58.691991 ip-10-0-129-128 kubenswrapper[2559]: I0712 18:18:58.691945 2559 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-pwsvh"] Jul 12 18:19:11.492634 ip-10-0-129-128 kubenswrapper[2559]: I0712 18:19:11.492590 2559 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-pwsvh"] Jul 12 18:19:22.686399 ip-10-0-129-128 kubenswrapper[2559]: I0712 18:19:22.686362 2559 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-pwsvh"] Jul 12 18:19:31.995436 ip-10-0-129-128 kubenswrapper[2559]: I0712 18:19:31.995398 2559 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-pwsvh"] Jul 12 18:19:52.587947 ip-10-0-129-128 kubenswrapper[2559]: I0712 18:19:52.587903 2559 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-pwsvh"] Jul 12 18:19:58.141065 ip-10-0-129-128 kubenswrapper[2559]: I0712 18:19:58.141028 2559 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-ingress/e2e-ait-6a5892af-openshift-default-5679c8fbb7-s6p7w"] Jul 12 18:19:58.145125 ip-10-0-129-128 kubenswrapper[2559]: I0712 18:19:58.145102 2559 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-ait-6a5892af-openshift-default-5679c8fbb7-s6p7w" Jul 12 18:19:58.147598 ip-10-0-129-128 kubenswrapper[2559]: I0712 18:19:58.147575 2559 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress\"/\"e2e-ait-6a5892af-openshift-default-dockercfg-lxw8g\"" Jul 12 18:19:58.166139 ip-10-0-129-128 kubenswrapper[2559]: I0712 18:19:58.166108 2559 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/e2e-ait-6a5892af-openshift-default-5679c8fbb7-s6p7w"] Jul 12 18:19:58.214723 ip-10-0-129-128 kubenswrapper[2559]: I0712 18:19:58.214672 2559 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/bf113811-c3e1-4d88-b784-7e9fc71af4e3-workload-socket\") pod \"e2e-ait-6a5892af-openshift-default-5679c8fbb7-s6p7w\" (UID: \"bf113811-c3e1-4d88-b784-7e9fc71af4e3\") " pod="openshift-ingress/e2e-ait-6a5892af-openshift-default-5679c8fbb7-s6p7w" Jul 12 18:19:58.214723 ip-10-0-129-128 kubenswrapper[2559]: I0712 18:19:58.214725 2559 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/bf113811-c3e1-4d88-b784-7e9fc71af4e3-istio-data\") pod \"e2e-ait-6a5892af-openshift-default-5679c8fbb7-s6p7w\" (UID: \"bf113811-c3e1-4d88-b784-7e9fc71af4e3\") " pod="openshift-ingress/e2e-ait-6a5892af-openshift-default-5679c8fbb7-s6p7w" Jul 12 18:19:58.214953 ip-10-0-129-128 kubenswrapper[2559]: I0712 18:19:58.214815 2559 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/bf113811-c3e1-4d88-b784-7e9fc71af4e3-istio-token\") pod \"e2e-ait-6a5892af-openshift-default-5679c8fbb7-s6p7w\" (UID: \"bf113811-c3e1-4d88-b784-7e9fc71af4e3\") " pod="openshift-ingress/e2e-ait-6a5892af-openshift-default-5679c8fbb7-s6p7w" Jul 12 18:19:58.214953 ip-10-0-129-128 kubenswrapper[2559]: I0712 18:19:58.214865 2559 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/bf113811-c3e1-4d88-b784-7e9fc71af4e3-credential-socket\") pod \"e2e-ait-6a5892af-openshift-default-5679c8fbb7-s6p7w\" (UID: \"bf113811-c3e1-4d88-b784-7e9fc71af4e3\") " pod="openshift-ingress/e2e-ait-6a5892af-openshift-default-5679c8fbb7-s6p7w" Jul 12 18:19:58.215062 ip-10-0-129-128 kubenswrapper[2559]: I0712 18:19:58.214948 2559 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/bf113811-c3e1-4d88-b784-7e9fc71af4e3-workload-certs\") pod \"e2e-ait-6a5892af-openshift-default-5679c8fbb7-s6p7w\" (UID: \"bf113811-c3e1-4d88-b784-7e9fc71af4e3\") " pod="openshift-ingress/e2e-ait-6a5892af-openshift-default-5679c8fbb7-s6p7w" Jul 12 18:19:58.215062 ip-10-0-129-128 kubenswrapper[2559]: I0712 18:19:58.214994 2559 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/bf113811-c3e1-4d88-b784-7e9fc71af4e3-istio-podinfo\") pod \"e2e-ait-6a5892af-openshift-default-5679c8fbb7-s6p7w\" (UID: \"bf113811-c3e1-4d88-b784-7e9fc71af4e3\") " pod="openshift-ingress/e2e-ait-6a5892af-openshift-default-5679c8fbb7-s6p7w" Jul 12 18:19:58.215062 ip-10-0-129-128 kubenswrapper[2559]: I0712 18:19:58.215026 2559 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/bf113811-c3e1-4d88-b784-7e9fc71af4e3-istiod-ca-cert\") pod \"e2e-ait-6a5892af-openshift-default-5679c8fbb7-s6p7w\" (UID: \"bf113811-c3e1-4d88-b784-7e9fc71af4e3\") " pod="openshift-ingress/e2e-ait-6a5892af-openshift-default-5679c8fbb7-s6p7w" Jul 12 18:19:58.215062 ip-10-0-129-128 kubenswrapper[2559]: I0712 18:19:58.215051 2559 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-jnst7\" (UniqueName: \"kubernetes.io/projected/bf113811-c3e1-4d88-b784-7e9fc71af4e3-kube-api-access-jnst7\") pod \"e2e-ait-6a5892af-openshift-default-5679c8fbb7-s6p7w\" (UID: \"bf113811-c3e1-4d88-b784-7e9fc71af4e3\") " pod="openshift-ingress/e2e-ait-6a5892af-openshift-default-5679c8fbb7-s6p7w" Jul 12 18:19:58.215229 ip-10-0-129-128 kubenswrapper[2559]: I0712 18:19:58.215108 2559 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/bf113811-c3e1-4d88-b784-7e9fc71af4e3-istio-envoy\") pod \"e2e-ait-6a5892af-openshift-default-5679c8fbb7-s6p7w\" (UID: \"bf113811-c3e1-4d88-b784-7e9fc71af4e3\") " pod="openshift-ingress/e2e-ait-6a5892af-openshift-default-5679c8fbb7-s6p7w" Jul 12 18:19:58.316056 ip-10-0-129-128 kubenswrapper[2559]: I0712 18:19:58.316024 2559 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/bf113811-c3e1-4d88-b784-7e9fc71af4e3-istio-token\") pod \"e2e-ait-6a5892af-openshift-default-5679c8fbb7-s6p7w\" (UID: \"bf113811-c3e1-4d88-b784-7e9fc71af4e3\") " pod="openshift-ingress/e2e-ait-6a5892af-openshift-default-5679c8fbb7-s6p7w" Jul 12 18:19:58.316241 ip-10-0-129-128 kubenswrapper[2559]: I0712 18:19:58.316073 2559 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/bf113811-c3e1-4d88-b784-7e9fc71af4e3-credential-socket\") pod \"e2e-ait-6a5892af-openshift-default-5679c8fbb7-s6p7w\" (UID: \"bf113811-c3e1-4d88-b784-7e9fc71af4e3\") " pod="openshift-ingress/e2e-ait-6a5892af-openshift-default-5679c8fbb7-s6p7w" Jul 12 18:19:58.316241 ip-10-0-129-128 kubenswrapper[2559]: I0712 18:19:58.316101 2559 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/bf113811-c3e1-4d88-b784-7e9fc71af4e3-workload-certs\") pod \"e2e-ait-6a5892af-openshift-default-5679c8fbb7-s6p7w\" (UID: \"bf113811-c3e1-4d88-b784-7e9fc71af4e3\") " pod="openshift-ingress/e2e-ait-6a5892af-openshift-default-5679c8fbb7-s6p7w" Jul 12 18:19:58.316241 ip-10-0-129-128 kubenswrapper[2559]: I0712 18:19:58.316130 2559 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/bf113811-c3e1-4d88-b784-7e9fc71af4e3-istio-podinfo\") pod \"e2e-ait-6a5892af-openshift-default-5679c8fbb7-s6p7w\" (UID: \"bf113811-c3e1-4d88-b784-7e9fc71af4e3\") " pod="openshift-ingress/e2e-ait-6a5892af-openshift-default-5679c8fbb7-s6p7w" Jul 12 18:19:58.316241 ip-10-0-129-128 kubenswrapper[2559]: I0712 18:19:58.316148 2559 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/bf113811-c3e1-4d88-b784-7e9fc71af4e3-istiod-ca-cert\") pod \"e2e-ait-6a5892af-openshift-default-5679c8fbb7-s6p7w\" (UID: \"bf113811-c3e1-4d88-b784-7e9fc71af4e3\") " pod="openshift-ingress/e2e-ait-6a5892af-openshift-default-5679c8fbb7-s6p7w" Jul 12 18:19:58.316241 ip-10-0-129-128 kubenswrapper[2559]: I0712 18:19:58.316163 2559 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-jnst7\" (UniqueName: \"kubernetes.io/projected/bf113811-c3e1-4d88-b784-7e9fc71af4e3-kube-api-access-jnst7\") pod \"e2e-ait-6a5892af-openshift-default-5679c8fbb7-s6p7w\" (UID: \"bf113811-c3e1-4d88-b784-7e9fc71af4e3\") " pod="openshift-ingress/e2e-ait-6a5892af-openshift-default-5679c8fbb7-s6p7w" Jul 12 18:19:58.316516 ip-10-0-129-128 kubenswrapper[2559]: I0712 18:19:58.316325 2559 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/bf113811-c3e1-4d88-b784-7e9fc71af4e3-istio-envoy\") pod \"e2e-ait-6a5892af-openshift-default-5679c8fbb7-s6p7w\" (UID: \"bf113811-c3e1-4d88-b784-7e9fc71af4e3\") " pod="openshift-ingress/e2e-ait-6a5892af-openshift-default-5679c8fbb7-s6p7w" Jul 12 18:19:58.316516 ip-10-0-129-128 kubenswrapper[2559]: I0712 18:19:58.316384 2559 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/bf113811-c3e1-4d88-b784-7e9fc71af4e3-workload-socket\") pod \"e2e-ait-6a5892af-openshift-default-5679c8fbb7-s6p7w\" (UID: \"bf113811-c3e1-4d88-b784-7e9fc71af4e3\") " pod="openshift-ingress/e2e-ait-6a5892af-openshift-default-5679c8fbb7-s6p7w" Jul 12 18:19:58.316516 ip-10-0-129-128 kubenswrapper[2559]: I0712 18:19:58.316415 2559 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/bf113811-c3e1-4d88-b784-7e9fc71af4e3-istio-data\") pod \"e2e-ait-6a5892af-openshift-default-5679c8fbb7-s6p7w\" (UID: \"bf113811-c3e1-4d88-b784-7e9fc71af4e3\") " pod="openshift-ingress/e2e-ait-6a5892af-openshift-default-5679c8fbb7-s6p7w" Jul 12 18:19:58.316706 ip-10-0-129-128 kubenswrapper[2559]: I0712 18:19:58.316590 2559 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/bf113811-c3e1-4d88-b784-7e9fc71af4e3-credential-socket\") pod \"e2e-ait-6a5892af-openshift-default-5679c8fbb7-s6p7w\" (UID: \"bf113811-c3e1-4d88-b784-7e9fc71af4e3\") " pod="openshift-ingress/e2e-ait-6a5892af-openshift-default-5679c8fbb7-s6p7w" Jul 12 18:19:58.316706 ip-10-0-129-128 kubenswrapper[2559]: I0712 18:19:58.316615 2559 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/bf113811-c3e1-4d88-b784-7e9fc71af4e3-workload-certs\") pod \"e2e-ait-6a5892af-openshift-default-5679c8fbb7-s6p7w\" (UID: \"bf113811-c3e1-4d88-b784-7e9fc71af4e3\") " pod="openshift-ingress/e2e-ait-6a5892af-openshift-default-5679c8fbb7-s6p7w" Jul 12 18:19:58.316864 ip-10-0-129-128 kubenswrapper[2559]: I0712 18:19:58.316842 2559 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/bf113811-c3e1-4d88-b784-7e9fc71af4e3-istio-data\") pod \"e2e-ait-6a5892af-openshift-default-5679c8fbb7-s6p7w\" (UID: \"bf113811-c3e1-4d88-b784-7e9fc71af4e3\") " pod="openshift-ingress/e2e-ait-6a5892af-openshift-default-5679c8fbb7-s6p7w" Jul 12 18:19:58.316938 ip-10-0-129-128 kubenswrapper[2559]: I0712 18:19:58.316867 2559 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/bf113811-c3e1-4d88-b784-7e9fc71af4e3-workload-socket\") pod \"e2e-ait-6a5892af-openshift-default-5679c8fbb7-s6p7w\" (UID: \"bf113811-c3e1-4d88-b784-7e9fc71af4e3\") " pod="openshift-ingress/e2e-ait-6a5892af-openshift-default-5679c8fbb7-s6p7w" Jul 12 18:19:58.317104 ip-10-0-129-128 kubenswrapper[2559]: I0712 18:19:58.317087 2559 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/bf113811-c3e1-4d88-b784-7e9fc71af4e3-istiod-ca-cert\") pod \"e2e-ait-6a5892af-openshift-default-5679c8fbb7-s6p7w\" (UID: \"bf113811-c3e1-4d88-b784-7e9fc71af4e3\") " pod="openshift-ingress/e2e-ait-6a5892af-openshift-default-5679c8fbb7-s6p7w" Jul 12 18:19:58.318648 ip-10-0-129-128 kubenswrapper[2559]: I0712 18:19:58.318616 2559 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/bf113811-c3e1-4d88-b784-7e9fc71af4e3-istio-envoy\") pod \"e2e-ait-6a5892af-openshift-default-5679c8fbb7-s6p7w\" (UID: \"bf113811-c3e1-4d88-b784-7e9fc71af4e3\") " pod="openshift-ingress/e2e-ait-6a5892af-openshift-default-5679c8fbb7-s6p7w" Jul 12 18:19:58.319095 ip-10-0-129-128 kubenswrapper[2559]: I0712 18:19:58.319078 2559 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/bf113811-c3e1-4d88-b784-7e9fc71af4e3-istio-podinfo\") pod \"e2e-ait-6a5892af-openshift-default-5679c8fbb7-s6p7w\" (UID: \"bf113811-c3e1-4d88-b784-7e9fc71af4e3\") " pod="openshift-ingress/e2e-ait-6a5892af-openshift-default-5679c8fbb7-s6p7w" Jul 12 18:19:58.324512 ip-10-0-129-128 kubenswrapper[2559]: I0712 18:19:58.324490 2559 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-jnst7\" (UniqueName: \"kubernetes.io/projected/bf113811-c3e1-4d88-b784-7e9fc71af4e3-kube-api-access-jnst7\") pod \"e2e-ait-6a5892af-openshift-default-5679c8fbb7-s6p7w\" (UID: \"bf113811-c3e1-4d88-b784-7e9fc71af4e3\") " pod="openshift-ingress/e2e-ait-6a5892af-openshift-default-5679c8fbb7-s6p7w" Jul 12 18:19:58.325022 ip-10-0-129-128 kubenswrapper[2559]: I0712 18:19:58.325004 2559 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/bf113811-c3e1-4d88-b784-7e9fc71af4e3-istio-token\") pod \"e2e-ait-6a5892af-openshift-default-5679c8fbb7-s6p7w\" (UID: \"bf113811-c3e1-4d88-b784-7e9fc71af4e3\") " pod="openshift-ingress/e2e-ait-6a5892af-openshift-default-5679c8fbb7-s6p7w" Jul 12 18:19:58.455863 ip-10-0-129-128 kubenswrapper[2559]: I0712 18:19:58.455768 2559 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-ait-6a5892af-openshift-default-5679c8fbb7-s6p7w" Jul 12 18:19:58.593505 ip-10-0-129-128 kubenswrapper[2559]: I0712 18:19:58.593479 2559 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/e2e-ait-6a5892af-openshift-default-5679c8fbb7-s6p7w"] Jul 12 18:19:58.595553 ip-10-0-129-128 kubenswrapper[2559]: W0712 18:19:58.595529 2559 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podbf113811_c3e1_4d88_b784_7e9fc71af4e3.slice/crio-ee86e7b3f98a8f1a1648dfb6e9166cb1d120f45525a4b16f027b81638c796933 WatchSource:0}: Error finding container ee86e7b3f98a8f1a1648dfb6e9166cb1d120f45525a4b16f027b81638c796933: Status 404 returned error can't find the container with id ee86e7b3f98a8f1a1648dfb6e9166cb1d120f45525a4b16f027b81638c796933 Jul 12 18:19:58.597626 ip-10-0-129-128 kubenswrapper[2559]: I0712 18:19:58.597591 2559 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236228Ki","pods":"250"} Jul 12 18:19:58.597741 ip-10-0-129-128 kubenswrapper[2559]: I0712 18:19:58.597703 2559 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236228Ki","pods":"250"} Jul 12 18:19:58.597804 ip-10-0-129-128 kubenswrapper[2559]: I0712 18:19:58.597751 2559 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236228Ki","pods":"250"} Jul 12 18:19:58.744122 ip-10-0-129-128 kubenswrapper[2559]: I0712 18:19:58.744033 2559 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-ait-6a5892af-openshift-default-5679c8fbb7-s6p7w" event={"ID":"bf113811-c3e1-4d88-b784-7e9fc71af4e3","Type":"ContainerStarted","Data":"abbf5d806b905abf44034d50bea303233707cb865890a37ca0c9b922f68dc2f1"} Jul 12 18:19:58.744122 ip-10-0-129-128 kubenswrapper[2559]: I0712 18:19:58.744095 2559 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-ait-6a5892af-openshift-default-5679c8fbb7-s6p7w" event={"ID":"bf113811-c3e1-4d88-b784-7e9fc71af4e3","Type":"ContainerStarted","Data":"ee86e7b3f98a8f1a1648dfb6e9166cb1d120f45525a4b16f027b81638c796933"} Jul 12 18:19:58.763196 ip-10-0-129-128 kubenswrapper[2559]: I0712 18:19:58.763128 2559 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-ingress/e2e-ait-6a5892af-openshift-default-5679c8fbb7-s6p7w" podStartSLOduration=0.76310587 podStartE2EDuration="763.10587ms" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-12 18:19:58 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-12 18:19:58.762240918 +0000 UTC m=+1606.256308981" watchObservedRunningTime="2026-07-12 18:19:58.76310587 +0000 UTC m=+1606.257173800" Jul 12 18:19:59.456654 ip-10-0-129-128 kubenswrapper[2559]: I0712 18:19:59.456614 2559 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="openshift-ingress/e2e-ait-6a5892af-openshift-default-5679c8fbb7-s6p7w" Jul 12 18:19:59.461576 ip-10-0-129-128 kubenswrapper[2559]: I0712 18:19:59.461547 2559 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="openshift-ingress/e2e-ait-6a5892af-openshift-default-5679c8fbb7-s6p7w" Jul 12 18:19:59.749594 ip-10-0-129-128 kubenswrapper[2559]: I0712 18:19:59.749508 2559 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/e2e-ait-6a5892af-openshift-default-5679c8fbb7-s6p7w" Jul 12 18:19:59.750431 ip-10-0-129-128 kubenswrapper[2559]: I0712 18:19:59.750409 2559 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/e2e-ait-6a5892af-openshift-default-5679c8fbb7-s6p7w" Jul 12 18:20:02.786089 ip-10-0-129-128 kubenswrapper[2559]: I0712 18:20:02.786054 2559 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-pwsvh"] Jul 12 18:20:10.808501 ip-10-0-129-128 kubenswrapper[2559]: I0712 18:20:10.808466 2559 generic.go:356] "Generic (PLEG): container finished" podID="bf113811-c3e1-4d88-b784-7e9fc71af4e3" containerID="abbf5d806b905abf44034d50bea303233707cb865890a37ca0c9b922f68dc2f1" exitCode=0 Jul 12 18:20:10.808950 ip-10-0-129-128 kubenswrapper[2559]: I0712 18:20:10.808539 2559 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-ait-6a5892af-openshift-default-5679c8fbb7-s6p7w" event={"ID":"bf113811-c3e1-4d88-b784-7e9fc71af4e3","Type":"ContainerDied","Data":"abbf5d806b905abf44034d50bea303233707cb865890a37ca0c9b922f68dc2f1"} Jul 12 18:20:10.808994 ip-10-0-129-128 kubenswrapper[2559]: I0712 18:20:10.808953 2559 scope.go:117] "RemoveContainer" containerID="abbf5d806b905abf44034d50bea303233707cb865890a37ca0c9b922f68dc2f1" Jul 12 18:20:10.809960 ip-10-0-129-128 kubenswrapper[2559]: I0712 18:20:10.809937 2559 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236228Ki","pods":"250"} Jul 12 18:20:10.810032 ip-10-0-129-128 kubenswrapper[2559]: I0712 18:20:10.809999 2559 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236228Ki","pods":"250"} Jul 12 18:20:10.810103 ip-10-0-129-128 kubenswrapper[2559]: I0712 18:20:10.810032 2559 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236228Ki","pods":"250"} Jul 12 18:20:11.815619 ip-10-0-129-128 kubenswrapper[2559]: I0712 18:20:11.815583 2559 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-ait-6a5892af-openshift-default-5679c8fbb7-s6p7w" event={"ID":"bf113811-c3e1-4d88-b784-7e9fc71af4e3","Type":"ContainerStarted","Data":"be240d47eb30c326c212bd32481ac37a0bf17c30d1843d9cb17c375e3761e342"} Jul 12 18:20:12.456307 ip-10-0-129-128 kubenswrapper[2559]: I0712 18:20:12.456265 2559 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="openshift-ingress/e2e-ait-6a5892af-openshift-default-5679c8fbb7-s6p7w" Jul 12 18:20:12.457400 ip-10-0-129-128 kubenswrapper[2559]: I0712 18:20:12.457377 2559 patch_prober.go:28] interesting pod/e2e-ait-6a5892af-openshift-default-5679c8fbb7-s6p7w container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.132.0.42:15021/healthz/ready\": dial tcp 10.132.0.42:15021: connect: connection refused" start-of-body= Jul 12 18:20:12.457505 ip-10-0-129-128 kubenswrapper[2559]: I0712 18:20:12.457432 2559 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-ait-6a5892af-openshift-default-5679c8fbb7-s6p7w" podUID="bf113811-c3e1-4d88-b784-7e9fc71af4e3" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.132.0.42:15021/healthz/ready\": dial tcp 10.132.0.42:15021: connect: connection refused" Jul 12 18:20:13.456423 ip-10-0-129-128 kubenswrapper[2559]: I0712 18:20:13.456380 2559 patch_prober.go:28] interesting pod/e2e-ait-6a5892af-openshift-default-5679c8fbb7-s6p7w container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.132.0.42:15021/healthz/ready\": dial tcp 10.132.0.42:15021: connect: connection refused" start-of-body= Jul 12 18:20:13.456938 ip-10-0-129-128 kubenswrapper[2559]: I0712 18:20:13.456443 2559 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-ait-6a5892af-openshift-default-5679c8fbb7-s6p7w" podUID="bf113811-c3e1-4d88-b784-7e9fc71af4e3" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.132.0.42:15021/healthz/ready\": dial tcp 10.132.0.42:15021: connect: connection refused" Jul 12 18:20:14.456570 ip-10-0-129-128 kubenswrapper[2559]: I0712 18:20:14.456532 2559 patch_prober.go:28] interesting pod/e2e-ait-6a5892af-openshift-default-5679c8fbb7-s6p7w container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.132.0.42:15021/healthz/ready\": dial tcp 10.132.0.42:15021: connect: connection refused" start-of-body= Jul 12 18:20:14.457012 ip-10-0-129-128 kubenswrapper[2559]: I0712 18:20:14.456590 2559 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-ait-6a5892af-openshift-default-5679c8fbb7-s6p7w" podUID="bf113811-c3e1-4d88-b784-7e9fc71af4e3" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.132.0.42:15021/healthz/ready\": dial tcp 10.132.0.42:15021: connect: connection refused" Jul 12 18:20:14.750540 ip-10-0-129-128 kubenswrapper[2559]: I0712 18:20:14.749662 2559 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/e2e-ait-6a5892af-openshift-default-5679c8fbb7-s6p7w" Jul 12 18:20:15.456415 ip-10-0-129-128 kubenswrapper[2559]: I0712 18:20:15.456378 2559 patch_prober.go:28] interesting pod/e2e-ait-6a5892af-openshift-default-5679c8fbb7-s6p7w container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.132.0.42:15021/healthz/ready\": dial tcp 10.132.0.42:15021: connect: connection refused" start-of-body= Jul 12 18:20:15.456632 ip-10-0-129-128 kubenswrapper[2559]: I0712 18:20:15.456437 2559 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-ait-6a5892af-openshift-default-5679c8fbb7-s6p7w" podUID="bf113811-c3e1-4d88-b784-7e9fc71af4e3" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.132.0.42:15021/healthz/ready\": dial tcp 10.132.0.42:15021: connect: connection refused" Jul 12 18:20:16.460318 ip-10-0-129-128 kubenswrapper[2559]: I0712 18:20:16.460286 2559 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="openshift-ingress/e2e-ait-6a5892af-openshift-default-5679c8fbb7-s6p7w" Jul 12 18:20:16.461395 ip-10-0-129-128 kubenswrapper[2559]: I0712 18:20:16.461372 2559 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/e2e-ait-6a5892af-openshift-default-5679c8fbb7-s6p7w" Jul 12 18:23:04.442831 ip-10-0-129-128 kubenswrapper[2559]: I0712 18:23:04.442790 2559 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-ingress/e2e-ait-6a5892af-openshift-default-5679c8fbb7-s6p7w"] Jul 12 18:23:04.443792 ip-10-0-129-128 kubenswrapper[2559]: I0712 18:23:04.443078 2559 kuberuntime_container.go:873] "Killing container with a grace period" pod="openshift-ingress/e2e-ait-6a5892af-openshift-default-5679c8fbb7-s6p7w" podUID="bf113811-c3e1-4d88-b784-7e9fc71af4e3" containerName="istio-proxy" containerID="cri-o://be240d47eb30c326c212bd32481ac37a0bf17c30d1843d9cb17c375e3761e342" gracePeriod=30 Jul 12 18:23:07.291139 ip-10-0-129-128 kubenswrapper[2559]: I0712 18:23:07.291051 2559 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-pwsvh"] Jul 12 18:23:09.661573 ip-10-0-129-128 kubenswrapper[2559]: I0712 18:23:09.661540 2559 generic.go:356] "Generic (PLEG): container finished" podID="bf113811-c3e1-4d88-b784-7e9fc71af4e3" containerID="be240d47eb30c326c212bd32481ac37a0bf17c30d1843d9cb17c375e3761e342" exitCode=0 Jul 12 18:23:09.661983 ip-10-0-129-128 kubenswrapper[2559]: I0712 18:23:09.661619 2559 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-ait-6a5892af-openshift-default-5679c8fbb7-s6p7w" event={"ID":"bf113811-c3e1-4d88-b784-7e9fc71af4e3","Type":"ContainerDied","Data":"be240d47eb30c326c212bd32481ac37a0bf17c30d1843d9cb17c375e3761e342"} Jul 12 18:23:09.661983 ip-10-0-129-128 kubenswrapper[2559]: I0712 18:23:09.661696 2559 scope.go:117] "RemoveContainer" containerID="abbf5d806b905abf44034d50bea303233707cb865890a37ca0c9b922f68dc2f1" Jul 12 18:23:09.702147 ip-10-0-129-128 kubenswrapper[2559]: I0712 18:23:09.702123 2559 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-ait-6a5892af-openshift-default-5679c8fbb7-s6p7w" Jul 12 18:23:09.742445 ip-10-0-129-128 kubenswrapper[2559]: I0712 18:23:09.742361 2559 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/bf113811-c3e1-4d88-b784-7e9fc71af4e3-workload-socket\") pod \"bf113811-c3e1-4d88-b784-7e9fc71af4e3\" (UID: \"bf113811-c3e1-4d88-b784-7e9fc71af4e3\") " Jul 12 18:23:09.742445 ip-10-0-129-128 kubenswrapper[2559]: I0712 18:23:09.742407 2559 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/bf113811-c3e1-4d88-b784-7e9fc71af4e3-istio-envoy\") pod \"bf113811-c3e1-4d88-b784-7e9fc71af4e3\" (UID: \"bf113811-c3e1-4d88-b784-7e9fc71af4e3\") " Jul 12 18:23:09.742445 ip-10-0-129-128 kubenswrapper[2559]: I0712 18:23:09.742430 2559 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/bf113811-c3e1-4d88-b784-7e9fc71af4e3-istio-token\") pod \"bf113811-c3e1-4d88-b784-7e9fc71af4e3\" (UID: \"bf113811-c3e1-4d88-b784-7e9fc71af4e3\") " Jul 12 18:23:09.742860 ip-10-0-129-128 kubenswrapper[2559]: I0712 18:23:09.742470 2559 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/bf113811-c3e1-4d88-b784-7e9fc71af4e3-istiod-ca-cert\") pod \"bf113811-c3e1-4d88-b784-7e9fc71af4e3\" (UID: \"bf113811-c3e1-4d88-b784-7e9fc71af4e3\") " Jul 12 18:23:09.742860 ip-10-0-129-128 kubenswrapper[2559]: I0712 18:23:09.742507 2559 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/bf113811-c3e1-4d88-b784-7e9fc71af4e3-workload-certs\") pod \"bf113811-c3e1-4d88-b784-7e9fc71af4e3\" (UID: \"bf113811-c3e1-4d88-b784-7e9fc71af4e3\") " Jul 12 18:23:09.742860 ip-10-0-129-128 kubenswrapper[2559]: I0712 18:23:09.742537 2559 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/bf113811-c3e1-4d88-b784-7e9fc71af4e3-credential-socket\") pod \"bf113811-c3e1-4d88-b784-7e9fc71af4e3\" (UID: \"bf113811-c3e1-4d88-b784-7e9fc71af4e3\") " Jul 12 18:23:09.742860 ip-10-0-129-128 kubenswrapper[2559]: I0712 18:23:09.742566 2559 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-jnst7\" (UniqueName: \"kubernetes.io/projected/bf113811-c3e1-4d88-b784-7e9fc71af4e3-kube-api-access-jnst7\") pod \"bf113811-c3e1-4d88-b784-7e9fc71af4e3\" (UID: \"bf113811-c3e1-4d88-b784-7e9fc71af4e3\") " Jul 12 18:23:09.742860 ip-10-0-129-128 kubenswrapper[2559]: I0712 18:23:09.742611 2559 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/bf113811-c3e1-4d88-b784-7e9fc71af4e3-istio-podinfo\") pod \"bf113811-c3e1-4d88-b784-7e9fc71af4e3\" (UID: \"bf113811-c3e1-4d88-b784-7e9fc71af4e3\") " Jul 12 18:23:09.742860 ip-10-0-129-128 kubenswrapper[2559]: I0712 18:23:09.742677 2559 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/bf113811-c3e1-4d88-b784-7e9fc71af4e3-istio-data\") pod \"bf113811-c3e1-4d88-b784-7e9fc71af4e3\" (UID: \"bf113811-c3e1-4d88-b784-7e9fc71af4e3\") " Jul 12 18:23:09.742860 ip-10-0-129-128 kubenswrapper[2559]: I0712 18:23:09.742712 2559 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/bf113811-c3e1-4d88-b784-7e9fc71af4e3-workload-socket" (OuterVolumeSpecName: "workload-socket") pod "bf113811-c3e1-4d88-b784-7e9fc71af4e3" (UID: "bf113811-c3e1-4d88-b784-7e9fc71af4e3"). InnerVolumeSpecName "workload-socket". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Jul 12 18:23:09.743270 ip-10-0-129-128 kubenswrapper[2559]: I0712 18:23:09.742997 2559 reconciler_common.go:299] "Volume detached for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/bf113811-c3e1-4d88-b784-7e9fc71af4e3-workload-socket\") on node \"ip-10-0-129-128.ec2.internal\" DevicePath \"\"" Jul 12 18:23:09.743270 ip-10-0-129-128 kubenswrapper[2559]: I0712 18:23:09.743040 2559 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/bf113811-c3e1-4d88-b784-7e9fc71af4e3-credential-socket" (OuterVolumeSpecName: "credential-socket") pod "bf113811-c3e1-4d88-b784-7e9fc71af4e3" (UID: "bf113811-c3e1-4d88-b784-7e9fc71af4e3"). InnerVolumeSpecName "credential-socket". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Jul 12 18:23:09.743270 ip-10-0-129-128 kubenswrapper[2559]: I0712 18:23:09.743255 2559 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/bf113811-c3e1-4d88-b784-7e9fc71af4e3-workload-certs" (OuterVolumeSpecName: "workload-certs") pod "bf113811-c3e1-4d88-b784-7e9fc71af4e3" (UID: "bf113811-c3e1-4d88-b784-7e9fc71af4e3"). InnerVolumeSpecName "workload-certs". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Jul 12 18:23:09.743433 ip-10-0-129-128 kubenswrapper[2559]: I0712 18:23:09.743311 2559 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/bf113811-c3e1-4d88-b784-7e9fc71af4e3-istiod-ca-cert" (OuterVolumeSpecName: "istiod-ca-cert") pod "bf113811-c3e1-4d88-b784-7e9fc71af4e3" (UID: "bf113811-c3e1-4d88-b784-7e9fc71af4e3"). InnerVolumeSpecName "istiod-ca-cert". PluginName "kubernetes.io/configmap", VolumeGIDValue "" Jul 12 18:23:09.743954 ip-10-0-129-128 kubenswrapper[2559]: I0712 18:23:09.743916 2559 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/bf113811-c3e1-4d88-b784-7e9fc71af4e3-istio-data" (OuterVolumeSpecName: "istio-data") pod "bf113811-c3e1-4d88-b784-7e9fc71af4e3" (UID: "bf113811-c3e1-4d88-b784-7e9fc71af4e3"). InnerVolumeSpecName "istio-data". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Jul 12 18:23:09.745085 ip-10-0-129-128 kubenswrapper[2559]: I0712 18:23:09.745059 2559 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/bf113811-c3e1-4d88-b784-7e9fc71af4e3-istio-envoy" (OuterVolumeSpecName: "istio-envoy") pod "bf113811-c3e1-4d88-b784-7e9fc71af4e3" (UID: "bf113811-c3e1-4d88-b784-7e9fc71af4e3"). InnerVolumeSpecName "istio-envoy". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Jul 12 18:23:09.745204 ip-10-0-129-128 kubenswrapper[2559]: I0712 18:23:09.745117 2559 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/bf113811-c3e1-4d88-b784-7e9fc71af4e3-istio-token" (OuterVolumeSpecName: "istio-token") pod "bf113811-c3e1-4d88-b784-7e9fc71af4e3" (UID: "bf113811-c3e1-4d88-b784-7e9fc71af4e3"). InnerVolumeSpecName "istio-token". PluginName "kubernetes.io/projected", VolumeGIDValue "" Jul 12 18:23:09.745319 ip-10-0-129-128 kubenswrapper[2559]: I0712 18:23:09.745296 2559 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/downward-api/bf113811-c3e1-4d88-b784-7e9fc71af4e3-istio-podinfo" (OuterVolumeSpecName: "istio-podinfo") pod "bf113811-c3e1-4d88-b784-7e9fc71af4e3" (UID: "bf113811-c3e1-4d88-b784-7e9fc71af4e3"). InnerVolumeSpecName "istio-podinfo". PluginName "kubernetes.io/downward-api", VolumeGIDValue "" Jul 12 18:23:09.745498 ip-10-0-129-128 kubenswrapper[2559]: I0712 18:23:09.745471 2559 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/bf113811-c3e1-4d88-b784-7e9fc71af4e3-kube-api-access-jnst7" (OuterVolumeSpecName: "kube-api-access-jnst7") pod "bf113811-c3e1-4d88-b784-7e9fc71af4e3" (UID: "bf113811-c3e1-4d88-b784-7e9fc71af4e3"). InnerVolumeSpecName "kube-api-access-jnst7". PluginName "kubernetes.io/projected", VolumeGIDValue "" Jul 12 18:23:09.843659 ip-10-0-129-128 kubenswrapper[2559]: I0712 18:23:09.843618 2559 reconciler_common.go:299] "Volume detached for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/bf113811-c3e1-4d88-b784-7e9fc71af4e3-workload-certs\") on node \"ip-10-0-129-128.ec2.internal\" DevicePath \"\"" Jul 12 18:23:09.843824 ip-10-0-129-128 kubenswrapper[2559]: I0712 18:23:09.843669 2559 reconciler_common.go:299] "Volume detached for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/bf113811-c3e1-4d88-b784-7e9fc71af4e3-credential-socket\") on node \"ip-10-0-129-128.ec2.internal\" DevicePath \"\"" Jul 12 18:23:09.843824 ip-10-0-129-128 kubenswrapper[2559]: I0712 18:23:09.843680 2559 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-jnst7\" (UniqueName: \"kubernetes.io/projected/bf113811-c3e1-4d88-b784-7e9fc71af4e3-kube-api-access-jnst7\") on node \"ip-10-0-129-128.ec2.internal\" DevicePath \"\"" Jul 12 18:23:09.843824 ip-10-0-129-128 kubenswrapper[2559]: I0712 18:23:09.843689 2559 reconciler_common.go:299] "Volume detached for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/bf113811-c3e1-4d88-b784-7e9fc71af4e3-istio-podinfo\") on node \"ip-10-0-129-128.ec2.internal\" DevicePath \"\"" Jul 12 18:23:09.843824 ip-10-0-129-128 kubenswrapper[2559]: I0712 18:23:09.843701 2559 reconciler_common.go:299] "Volume detached for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/bf113811-c3e1-4d88-b784-7e9fc71af4e3-istio-data\") on node \"ip-10-0-129-128.ec2.internal\" DevicePath \"\"" Jul 12 18:23:09.843824 ip-10-0-129-128 kubenswrapper[2559]: I0712 18:23:09.843709 2559 reconciler_common.go:299] "Volume detached for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/bf113811-c3e1-4d88-b784-7e9fc71af4e3-istio-envoy\") on node \"ip-10-0-129-128.ec2.internal\" DevicePath \"\"" Jul 12 18:23:09.843824 ip-10-0-129-128 kubenswrapper[2559]: I0712 18:23:09.843717 2559 reconciler_common.go:299] "Volume detached for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/bf113811-c3e1-4d88-b784-7e9fc71af4e3-istio-token\") on node \"ip-10-0-129-128.ec2.internal\" DevicePath \"\"" Jul 12 18:23:09.843824 ip-10-0-129-128 kubenswrapper[2559]: I0712 18:23:09.843727 2559 reconciler_common.go:299] "Volume detached for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/bf113811-c3e1-4d88-b784-7e9fc71af4e3-istiod-ca-cert\") on node \"ip-10-0-129-128.ec2.internal\" DevicePath \"\"" Jul 12 18:23:10.667467 ip-10-0-129-128 kubenswrapper[2559]: I0712 18:23:10.667430 2559 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-ait-6a5892af-openshift-default-5679c8fbb7-s6p7w" Jul 12 18:23:10.667930 ip-10-0-129-128 kubenswrapper[2559]: I0712 18:23:10.667429 2559 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-ait-6a5892af-openshift-default-5679c8fbb7-s6p7w" event={"ID":"bf113811-c3e1-4d88-b784-7e9fc71af4e3","Type":"ContainerDied","Data":"ee86e7b3f98a8f1a1648dfb6e9166cb1d120f45525a4b16f027b81638c796933"} Jul 12 18:23:10.667930 ip-10-0-129-128 kubenswrapper[2559]: I0712 18:23:10.667570 2559 scope.go:117] "RemoveContainer" containerID="be240d47eb30c326c212bd32481ac37a0bf17c30d1843d9cb17c375e3761e342" Jul 12 18:23:10.703141 ip-10-0-129-128 kubenswrapper[2559]: I0712 18:23:10.703113 2559 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-ingress/e2e-ait-6a5892af-openshift-default-5679c8fbb7-s6p7w"] Jul 12 18:23:10.710627 ip-10-0-129-128 kubenswrapper[2559]: I0712 18:23:10.710600 2559 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["openshift-ingress/e2e-ait-6a5892af-openshift-default-5679c8fbb7-s6p7w"] Jul 12 18:23:11.034826 ip-10-0-129-128 kubenswrapper[2559]: I0712 18:23:11.034793 2559 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="bf113811-c3e1-4d88-b784-7e9fc71af4e3" path="/var/lib/kubelet/pods/bf113811-c3e1-4d88-b784-7e9fc71af4e3/volumes" Jul 12 18:23:13.059833 ip-10-0-129-128 kubenswrapper[2559]: I0712 18:23:13.059804 2559 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-console-operator_console-operator-7866497494-wlk78_f1d46430-bb60-4470-b8d8-eacd44f4f6b1/console-operator/1.log" Jul 12 18:23:13.062099 ip-10-0-129-128 kubenswrapper[2559]: I0712 18:23:13.062075 2559 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ovn-kubernetes_ovnkube-node-q5dw8_e51bad50-bc74-4e38-8307-ca257b599314/ovn-acl-logging/0.log" Jul 12 18:23:13.062099 ip-10-0-129-128 kubenswrapper[2559]: I0712 18:23:13.062083 2559 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-console-operator_console-operator-7866497494-wlk78_f1d46430-bb60-4470-b8d8-eacd44f4f6b1/console-operator/1.log" Jul 12 18:23:13.064703 ip-10-0-129-128 kubenswrapper[2559]: I0712 18:23:13.064684 2559 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ovn-kubernetes_ovnkube-node-q5dw8_e51bad50-bc74-4e38-8307-ca257b599314/ovn-acl-logging/0.log" Jul 12 18:23:16.607725 ip-10-0-129-128 kubenswrapper[2559]: I0712 18:23:16.607666 2559 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-ingress/e2e-migrate-ad0567c9-gw-openshift-default-58c847b895-lhr67"] Jul 12 18:23:16.608120 ip-10-0-129-128 kubenswrapper[2559]: I0712 18:23:16.608015 2559 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="bf113811-c3e1-4d88-b784-7e9fc71af4e3" containerName="istio-proxy" Jul 12 18:23:16.608120 ip-10-0-129-128 kubenswrapper[2559]: I0712 18:23:16.608025 2559 state_mem.go:107] "Deleted CPUSet assignment" podUID="bf113811-c3e1-4d88-b784-7e9fc71af4e3" containerName="istio-proxy" Jul 12 18:23:16.608120 ip-10-0-129-128 kubenswrapper[2559]: I0712 18:23:16.608094 2559 memory_manager.go:368] "RemoveStaleState removing state" podUID="bf113811-c3e1-4d88-b784-7e9fc71af4e3" containerName="istio-proxy" Jul 12 18:23:16.608120 ip-10-0-129-128 kubenswrapper[2559]: I0712 18:23:16.608103 2559 memory_manager.go:368] "RemoveStaleState removing state" podUID="bf113811-c3e1-4d88-b784-7e9fc71af4e3" containerName="istio-proxy" Jul 12 18:23:16.611008 ip-10-0-129-128 kubenswrapper[2559]: I0712 18:23:16.610985 2559 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-migrate-ad0567c9-gw-openshift-default-58c847b895-lhr67" Jul 12 18:23:16.613386 ip-10-0-129-128 kubenswrapper[2559]: I0712 18:23:16.613365 2559 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress\"/\"e2e-migrate-ad0567c9-gw-openshift-default-dockercfg-5c5pj\"" Jul 12 18:23:16.629689 ip-10-0-129-128 kubenswrapper[2559]: I0712 18:23:16.629660 2559 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/e2e-migrate-ad0567c9-gw-openshift-default-58c847b895-lhr67"] Jul 12 18:23:16.702551 ip-10-0-129-128 kubenswrapper[2559]: I0712 18:23:16.702511 2559 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/c1e7a10f-b3fd-4386-910b-114f73ff2017-workload-socket\") pod \"e2e-migrate-ad0567c9-gw-openshift-default-58c847b895-lhr67\" (UID: \"c1e7a10f-b3fd-4386-910b-114f73ff2017\") " pod="openshift-ingress/e2e-migrate-ad0567c9-gw-openshift-default-58c847b895-lhr67" Jul 12 18:23:16.702770 ip-10-0-129-128 kubenswrapper[2559]: I0712 18:23:16.702560 2559 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/c1e7a10f-b3fd-4386-910b-114f73ff2017-istio-podinfo\") pod \"e2e-migrate-ad0567c9-gw-openshift-default-58c847b895-lhr67\" (UID: \"c1e7a10f-b3fd-4386-910b-114f73ff2017\") " pod="openshift-ingress/e2e-migrate-ad0567c9-gw-openshift-default-58c847b895-lhr67" Jul 12 18:23:16.702770 ip-10-0-129-128 kubenswrapper[2559]: I0712 18:23:16.702590 2559 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/c1e7a10f-b3fd-4386-910b-114f73ff2017-istio-token\") pod \"e2e-migrate-ad0567c9-gw-openshift-default-58c847b895-lhr67\" (UID: \"c1e7a10f-b3fd-4386-910b-114f73ff2017\") " pod="openshift-ingress/e2e-migrate-ad0567c9-gw-openshift-default-58c847b895-lhr67" Jul 12 18:23:16.702770 ip-10-0-129-128 kubenswrapper[2559]: I0712 18:23:16.702633 2559 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/c1e7a10f-b3fd-4386-910b-114f73ff2017-istio-data\") pod \"e2e-migrate-ad0567c9-gw-openshift-default-58c847b895-lhr67\" (UID: \"c1e7a10f-b3fd-4386-910b-114f73ff2017\") " pod="openshift-ingress/e2e-migrate-ad0567c9-gw-openshift-default-58c847b895-lhr67" Jul 12 18:23:16.702770 ip-10-0-129-128 kubenswrapper[2559]: I0712 18:23:16.702700 2559 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/c1e7a10f-b3fd-4386-910b-114f73ff2017-workload-certs\") pod \"e2e-migrate-ad0567c9-gw-openshift-default-58c847b895-lhr67\" (UID: \"c1e7a10f-b3fd-4386-910b-114f73ff2017\") " pod="openshift-ingress/e2e-migrate-ad0567c9-gw-openshift-default-58c847b895-lhr67" Jul 12 18:23:16.702770 ip-10-0-129-128 kubenswrapper[2559]: I0712 18:23:16.702735 2559 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/c1e7a10f-b3fd-4386-910b-114f73ff2017-credential-socket\") pod \"e2e-migrate-ad0567c9-gw-openshift-default-58c847b895-lhr67\" (UID: \"c1e7a10f-b3fd-4386-910b-114f73ff2017\") " pod="openshift-ingress/e2e-migrate-ad0567c9-gw-openshift-default-58c847b895-lhr67" Jul 12 18:23:16.702770 ip-10-0-129-128 kubenswrapper[2559]: I0712 18:23:16.702750 2559 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/c1e7a10f-b3fd-4386-910b-114f73ff2017-istiod-ca-cert\") pod \"e2e-migrate-ad0567c9-gw-openshift-default-58c847b895-lhr67\" (UID: \"c1e7a10f-b3fd-4386-910b-114f73ff2017\") " pod="openshift-ingress/e2e-migrate-ad0567c9-gw-openshift-default-58c847b895-lhr67" Jul 12 18:23:16.702770 ip-10-0-129-128 kubenswrapper[2559]: I0712 18:23:16.702768 2559 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/c1e7a10f-b3fd-4386-910b-114f73ff2017-istio-envoy\") pod \"e2e-migrate-ad0567c9-gw-openshift-default-58c847b895-lhr67\" (UID: \"c1e7a10f-b3fd-4386-910b-114f73ff2017\") " pod="openshift-ingress/e2e-migrate-ad0567c9-gw-openshift-default-58c847b895-lhr67" Jul 12 18:23:16.703010 ip-10-0-129-128 kubenswrapper[2559]: I0712 18:23:16.702834 2559 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-k6m9w\" (UniqueName: \"kubernetes.io/projected/c1e7a10f-b3fd-4386-910b-114f73ff2017-kube-api-access-k6m9w\") pod \"e2e-migrate-ad0567c9-gw-openshift-default-58c847b895-lhr67\" (UID: \"c1e7a10f-b3fd-4386-910b-114f73ff2017\") " pod="openshift-ingress/e2e-migrate-ad0567c9-gw-openshift-default-58c847b895-lhr67" Jul 12 18:23:16.804049 ip-10-0-129-128 kubenswrapper[2559]: I0712 18:23:16.804006 2559 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-k6m9w\" (UniqueName: \"kubernetes.io/projected/c1e7a10f-b3fd-4386-910b-114f73ff2017-kube-api-access-k6m9w\") pod \"e2e-migrate-ad0567c9-gw-openshift-default-58c847b895-lhr67\" (UID: \"c1e7a10f-b3fd-4386-910b-114f73ff2017\") " pod="openshift-ingress/e2e-migrate-ad0567c9-gw-openshift-default-58c847b895-lhr67" Jul 12 18:23:16.804252 ip-10-0-129-128 kubenswrapper[2559]: I0712 18:23:16.804068 2559 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/c1e7a10f-b3fd-4386-910b-114f73ff2017-workload-socket\") pod \"e2e-migrate-ad0567c9-gw-openshift-default-58c847b895-lhr67\" (UID: \"c1e7a10f-b3fd-4386-910b-114f73ff2017\") " pod="openshift-ingress/e2e-migrate-ad0567c9-gw-openshift-default-58c847b895-lhr67" Jul 12 18:23:16.804252 ip-10-0-129-128 kubenswrapper[2559]: I0712 18:23:16.804088 2559 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/c1e7a10f-b3fd-4386-910b-114f73ff2017-istio-podinfo\") pod \"e2e-migrate-ad0567c9-gw-openshift-default-58c847b895-lhr67\" (UID: \"c1e7a10f-b3fd-4386-910b-114f73ff2017\") " pod="openshift-ingress/e2e-migrate-ad0567c9-gw-openshift-default-58c847b895-lhr67" Jul 12 18:23:16.804252 ip-10-0-129-128 kubenswrapper[2559]: I0712 18:23:16.804105 2559 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/c1e7a10f-b3fd-4386-910b-114f73ff2017-istio-token\") pod \"e2e-migrate-ad0567c9-gw-openshift-default-58c847b895-lhr67\" (UID: \"c1e7a10f-b3fd-4386-910b-114f73ff2017\") " pod="openshift-ingress/e2e-migrate-ad0567c9-gw-openshift-default-58c847b895-lhr67" Jul 12 18:23:16.804410 ip-10-0-129-128 kubenswrapper[2559]: I0712 18:23:16.804333 2559 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/c1e7a10f-b3fd-4386-910b-114f73ff2017-istio-data\") pod \"e2e-migrate-ad0567c9-gw-openshift-default-58c847b895-lhr67\" (UID: \"c1e7a10f-b3fd-4386-910b-114f73ff2017\") " pod="openshift-ingress/e2e-migrate-ad0567c9-gw-openshift-default-58c847b895-lhr67" Jul 12 18:23:16.804410 ip-10-0-129-128 kubenswrapper[2559]: I0712 18:23:16.804375 2559 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/c1e7a10f-b3fd-4386-910b-114f73ff2017-workload-certs\") pod \"e2e-migrate-ad0567c9-gw-openshift-default-58c847b895-lhr67\" (UID: \"c1e7a10f-b3fd-4386-910b-114f73ff2017\") " pod="openshift-ingress/e2e-migrate-ad0567c9-gw-openshift-default-58c847b895-lhr67" Jul 12 18:23:16.804496 ip-10-0-129-128 kubenswrapper[2559]: I0712 18:23:16.804422 2559 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/c1e7a10f-b3fd-4386-910b-114f73ff2017-credential-socket\") pod \"e2e-migrate-ad0567c9-gw-openshift-default-58c847b895-lhr67\" (UID: \"c1e7a10f-b3fd-4386-910b-114f73ff2017\") " pod="openshift-ingress/e2e-migrate-ad0567c9-gw-openshift-default-58c847b895-lhr67" Jul 12 18:23:16.804496 ip-10-0-129-128 kubenswrapper[2559]: I0712 18:23:16.804433 2559 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/c1e7a10f-b3fd-4386-910b-114f73ff2017-workload-socket\") pod \"e2e-migrate-ad0567c9-gw-openshift-default-58c847b895-lhr67\" (UID: \"c1e7a10f-b3fd-4386-910b-114f73ff2017\") " pod="openshift-ingress/e2e-migrate-ad0567c9-gw-openshift-default-58c847b895-lhr67" Jul 12 18:23:16.804496 ip-10-0-129-128 kubenswrapper[2559]: I0712 18:23:16.804449 2559 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/c1e7a10f-b3fd-4386-910b-114f73ff2017-istiod-ca-cert\") pod \"e2e-migrate-ad0567c9-gw-openshift-default-58c847b895-lhr67\" (UID: \"c1e7a10f-b3fd-4386-910b-114f73ff2017\") " pod="openshift-ingress/e2e-migrate-ad0567c9-gw-openshift-default-58c847b895-lhr67" Jul 12 18:23:16.804682 ip-10-0-129-128 kubenswrapper[2559]: I0712 18:23:16.804519 2559 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/c1e7a10f-b3fd-4386-910b-114f73ff2017-istio-envoy\") pod \"e2e-migrate-ad0567c9-gw-openshift-default-58c847b895-lhr67\" (UID: \"c1e7a10f-b3fd-4386-910b-114f73ff2017\") " pod="openshift-ingress/e2e-migrate-ad0567c9-gw-openshift-default-58c847b895-lhr67" Jul 12 18:23:16.804749 ip-10-0-129-128 kubenswrapper[2559]: I0712 18:23:16.804721 2559 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/c1e7a10f-b3fd-4386-910b-114f73ff2017-istio-data\") pod \"e2e-migrate-ad0567c9-gw-openshift-default-58c847b895-lhr67\" (UID: \"c1e7a10f-b3fd-4386-910b-114f73ff2017\") " pod="openshift-ingress/e2e-migrate-ad0567c9-gw-openshift-default-58c847b895-lhr67" Jul 12 18:23:16.804811 ip-10-0-129-128 kubenswrapper[2559]: I0712 18:23:16.804756 2559 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/c1e7a10f-b3fd-4386-910b-114f73ff2017-workload-certs\") pod \"e2e-migrate-ad0567c9-gw-openshift-default-58c847b895-lhr67\" (UID: \"c1e7a10f-b3fd-4386-910b-114f73ff2017\") " pod="openshift-ingress/e2e-migrate-ad0567c9-gw-openshift-default-58c847b895-lhr67" Jul 12 18:23:16.804866 ip-10-0-129-128 kubenswrapper[2559]: I0712 18:23:16.804819 2559 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/c1e7a10f-b3fd-4386-910b-114f73ff2017-credential-socket\") pod \"e2e-migrate-ad0567c9-gw-openshift-default-58c847b895-lhr67\" (UID: \"c1e7a10f-b3fd-4386-910b-114f73ff2017\") " pod="openshift-ingress/e2e-migrate-ad0567c9-gw-openshift-default-58c847b895-lhr67" Jul 12 18:23:16.805043 ip-10-0-129-128 kubenswrapper[2559]: I0712 18:23:16.805026 2559 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/c1e7a10f-b3fd-4386-910b-114f73ff2017-istiod-ca-cert\") pod \"e2e-migrate-ad0567c9-gw-openshift-default-58c847b895-lhr67\" (UID: \"c1e7a10f-b3fd-4386-910b-114f73ff2017\") " pod="openshift-ingress/e2e-migrate-ad0567c9-gw-openshift-default-58c847b895-lhr67" Jul 12 18:23:16.806557 ip-10-0-129-128 kubenswrapper[2559]: I0712 18:23:16.806538 2559 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/c1e7a10f-b3fd-4386-910b-114f73ff2017-istio-podinfo\") pod \"e2e-migrate-ad0567c9-gw-openshift-default-58c847b895-lhr67\" (UID: \"c1e7a10f-b3fd-4386-910b-114f73ff2017\") " pod="openshift-ingress/e2e-migrate-ad0567c9-gw-openshift-default-58c847b895-lhr67" Jul 12 18:23:16.806729 ip-10-0-129-128 kubenswrapper[2559]: I0712 18:23:16.806708 2559 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/c1e7a10f-b3fd-4386-910b-114f73ff2017-istio-envoy\") pod \"e2e-migrate-ad0567c9-gw-openshift-default-58c847b895-lhr67\" (UID: \"c1e7a10f-b3fd-4386-910b-114f73ff2017\") " pod="openshift-ingress/e2e-migrate-ad0567c9-gw-openshift-default-58c847b895-lhr67" Jul 12 18:23:16.811698 ip-10-0-129-128 kubenswrapper[2559]: I0712 18:23:16.811667 2559 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-k6m9w\" (UniqueName: \"kubernetes.io/projected/c1e7a10f-b3fd-4386-910b-114f73ff2017-kube-api-access-k6m9w\") pod \"e2e-migrate-ad0567c9-gw-openshift-default-58c847b895-lhr67\" (UID: \"c1e7a10f-b3fd-4386-910b-114f73ff2017\") " pod="openshift-ingress/e2e-migrate-ad0567c9-gw-openshift-default-58c847b895-lhr67" Jul 12 18:23:16.811809 ip-10-0-129-128 kubenswrapper[2559]: I0712 18:23:16.811781 2559 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/c1e7a10f-b3fd-4386-910b-114f73ff2017-istio-token\") pod \"e2e-migrate-ad0567c9-gw-openshift-default-58c847b895-lhr67\" (UID: \"c1e7a10f-b3fd-4386-910b-114f73ff2017\") " pod="openshift-ingress/e2e-migrate-ad0567c9-gw-openshift-default-58c847b895-lhr67" Jul 12 18:23:16.922706 ip-10-0-129-128 kubenswrapper[2559]: I0712 18:23:16.922601 2559 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-migrate-ad0567c9-gw-openshift-default-58c847b895-lhr67" Jul 12 18:23:17.064677 ip-10-0-129-128 kubenswrapper[2559]: I0712 18:23:17.064633 2559 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/e2e-migrate-ad0567c9-gw-openshift-default-58c847b895-lhr67"] Jul 12 18:23:17.066672 ip-10-0-129-128 kubenswrapper[2559]: W0712 18:23:17.066624 2559 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podc1e7a10f_b3fd_4386_910b_114f73ff2017.slice/crio-b349205e7d5164f332b51fb14c4075ed235ff5409be8653fb15033780877e12e WatchSource:0}: Error finding container b349205e7d5164f332b51fb14c4075ed235ff5409be8653fb15033780877e12e: Status 404 returned error can't find the container with id b349205e7d5164f332b51fb14c4075ed235ff5409be8653fb15033780877e12e Jul 12 18:23:17.069100 ip-10-0-129-128 kubenswrapper[2559]: I0712 18:23:17.069069 2559 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236228Ki","pods":"250"} Jul 12 18:23:17.069193 ip-10-0-129-128 kubenswrapper[2559]: I0712 18:23:17.069135 2559 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236228Ki","pods":"250"} Jul 12 18:23:17.069193 ip-10-0-129-128 kubenswrapper[2559]: I0712 18:23:17.069170 2559 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236228Ki","pods":"250"} Jul 12 18:23:17.703708 ip-10-0-129-128 kubenswrapper[2559]: I0712 18:23:17.703672 2559 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-migrate-ad0567c9-gw-openshift-default-58c847b895-lhr67" event={"ID":"c1e7a10f-b3fd-4386-910b-114f73ff2017","Type":"ContainerStarted","Data":"500525e7b99cf0378273e7254a7284d68ce46fa8d3c1d1f003baa3260f58f83c"} Jul 12 18:23:17.704106 ip-10-0-129-128 kubenswrapper[2559]: I0712 18:23:17.703754 2559 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-migrate-ad0567c9-gw-openshift-default-58c847b895-lhr67" event={"ID":"c1e7a10f-b3fd-4386-910b-114f73ff2017","Type":"ContainerStarted","Data":"b349205e7d5164f332b51fb14c4075ed235ff5409be8653fb15033780877e12e"} Jul 12 18:23:17.725221 ip-10-0-129-128 kubenswrapper[2559]: I0712 18:23:17.725170 2559 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-ingress/e2e-migrate-ad0567c9-gw-openshift-default-58c847b895-lhr67" podStartSLOduration=1.725154527 podStartE2EDuration="1.725154527s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-12 18:23:16 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-12 18:23:17.724185006 +0000 UTC m=+1805.218252938" watchObservedRunningTime="2026-07-12 18:23:17.725154527 +0000 UTC m=+1805.219222457" Jul 12 18:23:17.922770 ip-10-0-129-128 kubenswrapper[2559]: I0712 18:23:17.922733 2559 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="openshift-ingress/e2e-migrate-ad0567c9-gw-openshift-default-58c847b895-lhr67" Jul 12 18:23:18.927858 ip-10-0-129-128 kubenswrapper[2559]: I0712 18:23:18.927810 2559 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="openshift-ingress/e2e-migrate-ad0567c9-gw-openshift-default-58c847b895-lhr67" Jul 12 18:23:19.714957 ip-10-0-129-128 kubenswrapper[2559]: I0712 18:23:19.714928 2559 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/e2e-migrate-ad0567c9-gw-openshift-default-58c847b895-lhr67" Jul 12 18:23:19.716147 ip-10-0-129-128 kubenswrapper[2559]: I0712 18:23:19.716128 2559 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/e2e-migrate-ad0567c9-gw-openshift-default-58c847b895-lhr67" Jul 12 18:23:21.582906 ip-10-0-129-128 kubenswrapper[2559]: I0712 18:23:21.582867 2559 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-pwsvh"] Jul 12 18:23:29.772846 ip-10-0-129-128 kubenswrapper[2559]: I0712 18:23:29.772810 2559 generic.go:356] "Generic (PLEG): container finished" podID="c1e7a10f-b3fd-4386-910b-114f73ff2017" containerID="500525e7b99cf0378273e7254a7284d68ce46fa8d3c1d1f003baa3260f58f83c" exitCode=0 Jul 12 18:23:29.773388 ip-10-0-129-128 kubenswrapper[2559]: I0712 18:23:29.772897 2559 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-migrate-ad0567c9-gw-openshift-default-58c847b895-lhr67" event={"ID":"c1e7a10f-b3fd-4386-910b-114f73ff2017","Type":"ContainerDied","Data":"500525e7b99cf0378273e7254a7284d68ce46fa8d3c1d1f003baa3260f58f83c"} Jul 12 18:23:29.773388 ip-10-0-129-128 kubenswrapper[2559]: I0712 18:23:29.773381 2559 scope.go:117] "RemoveContainer" containerID="500525e7b99cf0378273e7254a7284d68ce46fa8d3c1d1f003baa3260f58f83c" Jul 12 18:23:29.774361 ip-10-0-129-128 kubenswrapper[2559]: I0712 18:23:29.774334 2559 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236228Ki","pods":"250"} Jul 12 18:23:29.774465 ip-10-0-129-128 kubenswrapper[2559]: I0712 18:23:29.774417 2559 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236228Ki","pods":"250"} Jul 12 18:23:29.774541 ip-10-0-129-128 kubenswrapper[2559]: I0712 18:23:29.774462 2559 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236228Ki","pods":"250"} Jul 12 18:23:30.779623 ip-10-0-129-128 kubenswrapper[2559]: I0712 18:23:30.779590 2559 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-migrate-ad0567c9-gw-openshift-default-58c847b895-lhr67" event={"ID":"c1e7a10f-b3fd-4386-910b-114f73ff2017","Type":"ContainerStarted","Data":"90998206c0ee26e1dca45d0909c304fa874850ea33a9d9b83760b1ac438e6b98"} Jul 12 18:23:30.923229 ip-10-0-129-128 kubenswrapper[2559]: I0712 18:23:30.923194 2559 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="openshift-ingress/e2e-migrate-ad0567c9-gw-openshift-default-58c847b895-lhr67" Jul 12 18:23:30.924304 ip-10-0-129-128 kubenswrapper[2559]: I0712 18:23:30.924278 2559 patch_prober.go:28] interesting pod/e2e-migrate-ad0567c9-gw-openshift-default-58c847b895-lhr67 container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.132.0.43:15021/healthz/ready\": dial tcp 10.132.0.43:15021: connect: connection refused" start-of-body= Jul 12 18:23:30.924365 ip-10-0-129-128 kubenswrapper[2559]: I0712 18:23:30.924331 2559 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-migrate-ad0567c9-gw-openshift-default-58c847b895-lhr67" podUID="c1e7a10f-b3fd-4386-910b-114f73ff2017" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.132.0.43:15021/healthz/ready\": dial tcp 10.132.0.43:15021: connect: connection refused" Jul 12 18:23:31.923692 ip-10-0-129-128 kubenswrapper[2559]: I0712 18:23:31.923624 2559 patch_prober.go:28] interesting pod/e2e-migrate-ad0567c9-gw-openshift-default-58c847b895-lhr67 container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.132.0.43:15021/healthz/ready\": dial tcp 10.132.0.43:15021: connect: connection refused" start-of-body= Jul 12 18:23:31.924099 ip-10-0-129-128 kubenswrapper[2559]: I0712 18:23:31.923717 2559 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-migrate-ad0567c9-gw-openshift-default-58c847b895-lhr67" podUID="c1e7a10f-b3fd-4386-910b-114f73ff2017" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.132.0.43:15021/healthz/ready\": dial tcp 10.132.0.43:15021: connect: connection refused" Jul 12 18:23:32.923723 ip-10-0-129-128 kubenswrapper[2559]: I0712 18:23:32.923687 2559 patch_prober.go:28] interesting pod/e2e-migrate-ad0567c9-gw-openshift-default-58c847b895-lhr67 container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.132.0.43:15021/healthz/ready\": dial tcp 10.132.0.43:15021: connect: connection refused" start-of-body= Jul 12 18:23:32.924124 ip-10-0-129-128 kubenswrapper[2559]: I0712 18:23:32.923744 2559 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-migrate-ad0567c9-gw-openshift-default-58c847b895-lhr67" podUID="c1e7a10f-b3fd-4386-910b-114f73ff2017" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.132.0.43:15021/healthz/ready\": dial tcp 10.132.0.43:15021: connect: connection refused" Jul 12 18:23:33.924086 ip-10-0-129-128 kubenswrapper[2559]: I0712 18:23:33.924041 2559 patch_prober.go:28] interesting pod/e2e-migrate-ad0567c9-gw-openshift-default-58c847b895-lhr67 container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.132.0.43:15021/healthz/ready\": dial tcp 10.132.0.43:15021: connect: connection refused" start-of-body= Jul 12 18:23:33.924495 ip-10-0-129-128 kubenswrapper[2559]: I0712 18:23:33.924103 2559 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-migrate-ad0567c9-gw-openshift-default-58c847b895-lhr67" podUID="c1e7a10f-b3fd-4386-910b-114f73ff2017" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.132.0.43:15021/healthz/ready\": dial tcp 10.132.0.43:15021: connect: connection refused" Jul 12 18:23:34.715339 ip-10-0-129-128 kubenswrapper[2559]: I0712 18:23:34.715290 2559 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/e2e-migrate-ad0567c9-gw-openshift-default-58c847b895-lhr67" Jul 12 18:23:34.927034 ip-10-0-129-128 kubenswrapper[2559]: I0712 18:23:34.927000 2559 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="openshift-ingress/e2e-migrate-ad0567c9-gw-openshift-default-58c847b895-lhr67" Jul 12 18:23:34.928188 ip-10-0-129-128 kubenswrapper[2559]: I0712 18:23:34.928167 2559 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/e2e-migrate-ad0567c9-gw-openshift-default-58c847b895-lhr67" Jul 12 18:24:22.483117 ip-10-0-129-128 kubenswrapper[2559]: I0712 18:24:22.483082 2559 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-ingress/e2e-migrate-ad0567c9-gw-openshift-default-58c847b895-lhr67"] Jul 12 18:24:22.483588 ip-10-0-129-128 kubenswrapper[2559]: I0712 18:24:22.483310 2559 kuberuntime_container.go:873] "Killing container with a grace period" pod="openshift-ingress/e2e-migrate-ad0567c9-gw-openshift-default-58c847b895-lhr67" podUID="c1e7a10f-b3fd-4386-910b-114f73ff2017" containerName="istio-proxy" containerID="cri-o://90998206c0ee26e1dca45d0909c304fa874850ea33a9d9b83760b1ac438e6b98" gracePeriod=30 Jul 12 18:24:25.783218 ip-10-0-129-128 kubenswrapper[2559]: I0712 18:24:25.783182 2559 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-pwsvh"] Jul 12 18:24:27.748206 ip-10-0-129-128 kubenswrapper[2559]: I0712 18:24:27.748172 2559 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-migrate-ad0567c9-gw-openshift-default-58c847b895-lhr67" Jul 12 18:24:27.839569 ip-10-0-129-128 kubenswrapper[2559]: I0712 18:24:27.839535 2559 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/c1e7a10f-b3fd-4386-910b-114f73ff2017-workload-certs\") pod \"c1e7a10f-b3fd-4386-910b-114f73ff2017\" (UID: \"c1e7a10f-b3fd-4386-910b-114f73ff2017\") " Jul 12 18:24:27.839569 ip-10-0-129-128 kubenswrapper[2559]: I0712 18:24:27.839570 2559 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/c1e7a10f-b3fd-4386-910b-114f73ff2017-istiod-ca-cert\") pod \"c1e7a10f-b3fd-4386-910b-114f73ff2017\" (UID: \"c1e7a10f-b3fd-4386-910b-114f73ff2017\") " Jul 12 18:24:27.839810 ip-10-0-129-128 kubenswrapper[2559]: I0712 18:24:27.839591 2559 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/c1e7a10f-b3fd-4386-910b-114f73ff2017-istio-envoy\") pod \"c1e7a10f-b3fd-4386-910b-114f73ff2017\" (UID: \"c1e7a10f-b3fd-4386-910b-114f73ff2017\") " Jul 12 18:24:27.839810 ip-10-0-129-128 kubenswrapper[2559]: I0712 18:24:27.839617 2559 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/c1e7a10f-b3fd-4386-910b-114f73ff2017-workload-socket\") pod \"c1e7a10f-b3fd-4386-910b-114f73ff2017\" (UID: \"c1e7a10f-b3fd-4386-910b-114f73ff2017\") " Jul 12 18:24:27.839810 ip-10-0-129-128 kubenswrapper[2559]: I0712 18:24:27.839763 2559 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-k6m9w\" (UniqueName: \"kubernetes.io/projected/c1e7a10f-b3fd-4386-910b-114f73ff2017-kube-api-access-k6m9w\") pod \"c1e7a10f-b3fd-4386-910b-114f73ff2017\" (UID: \"c1e7a10f-b3fd-4386-910b-114f73ff2017\") " Jul 12 18:24:27.839966 ip-10-0-129-128 kubenswrapper[2559]: I0712 18:24:27.839864 2559 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/c1e7a10f-b3fd-4386-910b-114f73ff2017-workload-certs" (OuterVolumeSpecName: "workload-certs") pod "c1e7a10f-b3fd-4386-910b-114f73ff2017" (UID: "c1e7a10f-b3fd-4386-910b-114f73ff2017"). InnerVolumeSpecName "workload-certs". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Jul 12 18:24:27.839966 ip-10-0-129-128 kubenswrapper[2559]: I0712 18:24:27.839890 2559 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/c1e7a10f-b3fd-4386-910b-114f73ff2017-istio-data\") pod \"c1e7a10f-b3fd-4386-910b-114f73ff2017\" (UID: \"c1e7a10f-b3fd-4386-910b-114f73ff2017\") " Jul 12 18:24:27.840063 ip-10-0-129-128 kubenswrapper[2559]: I0712 18:24:27.839961 2559 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/c1e7a10f-b3fd-4386-910b-114f73ff2017-workload-socket" (OuterVolumeSpecName: "workload-socket") pod "c1e7a10f-b3fd-4386-910b-114f73ff2017" (UID: "c1e7a10f-b3fd-4386-910b-114f73ff2017"). InnerVolumeSpecName "workload-socket". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Jul 12 18:24:27.840063 ip-10-0-129-128 kubenswrapper[2559]: I0712 18:24:27.839923 2559 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/c1e7a10f-b3fd-4386-910b-114f73ff2017-credential-socket\") pod \"c1e7a10f-b3fd-4386-910b-114f73ff2017\" (UID: \"c1e7a10f-b3fd-4386-910b-114f73ff2017\") " Jul 12 18:24:27.840167 ip-10-0-129-128 kubenswrapper[2559]: I0712 18:24:27.840056 2559 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/c1e7a10f-b3fd-4386-910b-114f73ff2017-istio-podinfo\") pod \"c1e7a10f-b3fd-4386-910b-114f73ff2017\" (UID: \"c1e7a10f-b3fd-4386-910b-114f73ff2017\") " Jul 12 18:24:27.840167 ip-10-0-129-128 kubenswrapper[2559]: I0712 18:24:27.840124 2559 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/c1e7a10f-b3fd-4386-910b-114f73ff2017-istio-token\") pod \"c1e7a10f-b3fd-4386-910b-114f73ff2017\" (UID: \"c1e7a10f-b3fd-4386-910b-114f73ff2017\") " Jul 12 18:24:27.840263 ip-10-0-129-128 kubenswrapper[2559]: I0712 18:24:27.840200 2559 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/c1e7a10f-b3fd-4386-910b-114f73ff2017-credential-socket" (OuterVolumeSpecName: "credential-socket") pod "c1e7a10f-b3fd-4386-910b-114f73ff2017" (UID: "c1e7a10f-b3fd-4386-910b-114f73ff2017"). InnerVolumeSpecName "credential-socket". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Jul 12 18:24:27.840436 ip-10-0-129-128 kubenswrapper[2559]: I0712 18:24:27.840416 2559 reconciler_common.go:299] "Volume detached for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/c1e7a10f-b3fd-4386-910b-114f73ff2017-workload-socket\") on node \"ip-10-0-129-128.ec2.internal\" DevicePath \"\"" Jul 12 18:24:27.840788 ip-10-0-129-128 kubenswrapper[2559]: I0712 18:24:27.840771 2559 reconciler_common.go:299] "Volume detached for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/c1e7a10f-b3fd-4386-910b-114f73ff2017-credential-socket\") on node \"ip-10-0-129-128.ec2.internal\" DevicePath \"\"" Jul 12 18:24:27.840967 ip-10-0-129-128 kubenswrapper[2559]: I0712 18:24:27.840947 2559 reconciler_common.go:299] "Volume detached for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/c1e7a10f-b3fd-4386-910b-114f73ff2017-workload-certs\") on node \"ip-10-0-129-128.ec2.internal\" DevicePath \"\"" Jul 12 18:24:27.841123 ip-10-0-129-128 kubenswrapper[2559]: I0712 18:24:27.840677 2559 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/c1e7a10f-b3fd-4386-910b-114f73ff2017-istio-data" (OuterVolumeSpecName: "istio-data") pod "c1e7a10f-b3fd-4386-910b-114f73ff2017" (UID: "c1e7a10f-b3fd-4386-910b-114f73ff2017"). InnerVolumeSpecName "istio-data". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Jul 12 18:24:27.841259 ip-10-0-129-128 kubenswrapper[2559]: I0712 18:24:27.840778 2559 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/c1e7a10f-b3fd-4386-910b-114f73ff2017-istiod-ca-cert" (OuterVolumeSpecName: "istiod-ca-cert") pod "c1e7a10f-b3fd-4386-910b-114f73ff2017" (UID: "c1e7a10f-b3fd-4386-910b-114f73ff2017"). InnerVolumeSpecName "istiod-ca-cert". PluginName "kubernetes.io/configmap", VolumeGIDValue "" Jul 12 18:24:27.842742 ip-10-0-129-128 kubenswrapper[2559]: I0712 18:24:27.842705 2559 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/c1e7a10f-b3fd-4386-910b-114f73ff2017-istio-token" (OuterVolumeSpecName: "istio-token") pod "c1e7a10f-b3fd-4386-910b-114f73ff2017" (UID: "c1e7a10f-b3fd-4386-910b-114f73ff2017"). InnerVolumeSpecName "istio-token". PluginName "kubernetes.io/projected", VolumeGIDValue "" Jul 12 18:24:27.843154 ip-10-0-129-128 kubenswrapper[2559]: I0712 18:24:27.843115 2559 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/c1e7a10f-b3fd-4386-910b-114f73ff2017-istio-envoy" (OuterVolumeSpecName: "istio-envoy") pod "c1e7a10f-b3fd-4386-910b-114f73ff2017" (UID: "c1e7a10f-b3fd-4386-910b-114f73ff2017"). InnerVolumeSpecName "istio-envoy". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Jul 12 18:24:27.843154 ip-10-0-129-128 kubenswrapper[2559]: I0712 18:24:27.843130 2559 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/c1e7a10f-b3fd-4386-910b-114f73ff2017-kube-api-access-k6m9w" (OuterVolumeSpecName: "kube-api-access-k6m9w") pod "c1e7a10f-b3fd-4386-910b-114f73ff2017" (UID: "c1e7a10f-b3fd-4386-910b-114f73ff2017"). InnerVolumeSpecName "kube-api-access-k6m9w". PluginName "kubernetes.io/projected", VolumeGIDValue "" Jul 12 18:24:27.843154 ip-10-0-129-128 kubenswrapper[2559]: I0712 18:24:27.843126 2559 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/downward-api/c1e7a10f-b3fd-4386-910b-114f73ff2017-istio-podinfo" (OuterVolumeSpecName: "istio-podinfo") pod "c1e7a10f-b3fd-4386-910b-114f73ff2017" (UID: "c1e7a10f-b3fd-4386-910b-114f73ff2017"). InnerVolumeSpecName "istio-podinfo". PluginName "kubernetes.io/downward-api", VolumeGIDValue "" Jul 12 18:24:27.941958 ip-10-0-129-128 kubenswrapper[2559]: I0712 18:24:27.941921 2559 reconciler_common.go:299] "Volume detached for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/c1e7a10f-b3fd-4386-910b-114f73ff2017-istiod-ca-cert\") on node \"ip-10-0-129-128.ec2.internal\" DevicePath \"\"" Jul 12 18:24:27.941958 ip-10-0-129-128 kubenswrapper[2559]: I0712 18:24:27.941953 2559 reconciler_common.go:299] "Volume detached for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/c1e7a10f-b3fd-4386-910b-114f73ff2017-istio-envoy\") on node \"ip-10-0-129-128.ec2.internal\" DevicePath \"\"" Jul 12 18:24:27.941958 ip-10-0-129-128 kubenswrapper[2559]: I0712 18:24:27.941962 2559 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-k6m9w\" (UniqueName: \"kubernetes.io/projected/c1e7a10f-b3fd-4386-910b-114f73ff2017-kube-api-access-k6m9w\") on node \"ip-10-0-129-128.ec2.internal\" DevicePath \"\"" Jul 12 18:24:27.942196 ip-10-0-129-128 kubenswrapper[2559]: I0712 18:24:27.941971 2559 reconciler_common.go:299] "Volume detached for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/c1e7a10f-b3fd-4386-910b-114f73ff2017-istio-data\") on node \"ip-10-0-129-128.ec2.internal\" DevicePath \"\"" Jul 12 18:24:27.942196 ip-10-0-129-128 kubenswrapper[2559]: I0712 18:24:27.941979 2559 reconciler_common.go:299] "Volume detached for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/c1e7a10f-b3fd-4386-910b-114f73ff2017-istio-podinfo\") on node \"ip-10-0-129-128.ec2.internal\" DevicePath \"\"" Jul 12 18:24:27.942196 ip-10-0-129-128 kubenswrapper[2559]: I0712 18:24:27.941987 2559 reconciler_common.go:299] "Volume detached for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/c1e7a10f-b3fd-4386-910b-114f73ff2017-istio-token\") on node \"ip-10-0-129-128.ec2.internal\" DevicePath \"\"" Jul 12 18:24:28.059346 ip-10-0-129-128 kubenswrapper[2559]: I0712 18:24:28.059256 2559 generic.go:356] "Generic (PLEG): container finished" podID="c1e7a10f-b3fd-4386-910b-114f73ff2017" containerID="90998206c0ee26e1dca45d0909c304fa874850ea33a9d9b83760b1ac438e6b98" exitCode=0 Jul 12 18:24:28.059346 ip-10-0-129-128 kubenswrapper[2559]: I0712 18:24:28.059301 2559 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-migrate-ad0567c9-gw-openshift-default-58c847b895-lhr67" event={"ID":"c1e7a10f-b3fd-4386-910b-114f73ff2017","Type":"ContainerDied","Data":"90998206c0ee26e1dca45d0909c304fa874850ea33a9d9b83760b1ac438e6b98"} Jul 12 18:24:28.059346 ip-10-0-129-128 kubenswrapper[2559]: I0712 18:24:28.059324 2559 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-migrate-ad0567c9-gw-openshift-default-58c847b895-lhr67" Jul 12 18:24:28.059562 ip-10-0-129-128 kubenswrapper[2559]: I0712 18:24:28.059354 2559 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-migrate-ad0567c9-gw-openshift-default-58c847b895-lhr67" event={"ID":"c1e7a10f-b3fd-4386-910b-114f73ff2017","Type":"ContainerDied","Data":"b349205e7d5164f332b51fb14c4075ed235ff5409be8653fb15033780877e12e"} Jul 12 18:24:28.059562 ip-10-0-129-128 kubenswrapper[2559]: I0712 18:24:28.059385 2559 scope.go:117] "RemoveContainer" containerID="90998206c0ee26e1dca45d0909c304fa874850ea33a9d9b83760b1ac438e6b98" Jul 12 18:24:28.068951 ip-10-0-129-128 kubenswrapper[2559]: I0712 18:24:28.068931 2559 scope.go:117] "RemoveContainer" containerID="500525e7b99cf0378273e7254a7284d68ce46fa8d3c1d1f003baa3260f58f83c" Jul 12 18:24:28.077175 ip-10-0-129-128 kubenswrapper[2559]: I0712 18:24:28.077154 2559 scope.go:117] "RemoveContainer" containerID="90998206c0ee26e1dca45d0909c304fa874850ea33a9d9b83760b1ac438e6b98" Jul 12 18:24:28.077444 ip-10-0-129-128 kubenswrapper[2559]: E0712 18:24:28.077424 2559 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"90998206c0ee26e1dca45d0909c304fa874850ea33a9d9b83760b1ac438e6b98\": container with ID starting with 90998206c0ee26e1dca45d0909c304fa874850ea33a9d9b83760b1ac438e6b98 not found: ID does not exist" containerID="90998206c0ee26e1dca45d0909c304fa874850ea33a9d9b83760b1ac438e6b98" Jul 12 18:24:28.077514 ip-10-0-129-128 kubenswrapper[2559]: I0712 18:24:28.077459 2559 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"90998206c0ee26e1dca45d0909c304fa874850ea33a9d9b83760b1ac438e6b98"} err="failed to get container status \"90998206c0ee26e1dca45d0909c304fa874850ea33a9d9b83760b1ac438e6b98\": rpc error: code = NotFound desc = could not find container \"90998206c0ee26e1dca45d0909c304fa874850ea33a9d9b83760b1ac438e6b98\": container with ID starting with 90998206c0ee26e1dca45d0909c304fa874850ea33a9d9b83760b1ac438e6b98 not found: ID does not exist" Jul 12 18:24:28.077514 ip-10-0-129-128 kubenswrapper[2559]: I0712 18:24:28.077484 2559 scope.go:117] "RemoveContainer" containerID="500525e7b99cf0378273e7254a7284d68ce46fa8d3c1d1f003baa3260f58f83c" Jul 12 18:24:28.077775 ip-10-0-129-128 kubenswrapper[2559]: E0712 18:24:28.077756 2559 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"500525e7b99cf0378273e7254a7284d68ce46fa8d3c1d1f003baa3260f58f83c\": container with ID starting with 500525e7b99cf0378273e7254a7284d68ce46fa8d3c1d1f003baa3260f58f83c not found: ID does not exist" containerID="500525e7b99cf0378273e7254a7284d68ce46fa8d3c1d1f003baa3260f58f83c" Jul 12 18:24:28.077834 ip-10-0-129-128 kubenswrapper[2559]: I0712 18:24:28.077786 2559 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"500525e7b99cf0378273e7254a7284d68ce46fa8d3c1d1f003baa3260f58f83c"} err="failed to get container status \"500525e7b99cf0378273e7254a7284d68ce46fa8d3c1d1f003baa3260f58f83c\": rpc error: code = NotFound desc = could not find container \"500525e7b99cf0378273e7254a7284d68ce46fa8d3c1d1f003baa3260f58f83c\": container with ID starting with 500525e7b99cf0378273e7254a7284d68ce46fa8d3c1d1f003baa3260f58f83c not found: ID does not exist" Jul 12 18:24:28.086524 ip-10-0-129-128 kubenswrapper[2559]: I0712 18:24:28.086489 2559 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-ingress/e2e-migrate-ad0567c9-gw-openshift-default-58c847b895-lhr67"] Jul 12 18:24:28.092060 ip-10-0-129-128 kubenswrapper[2559]: I0712 18:24:28.092038 2559 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["openshift-ingress/e2e-migrate-ad0567c9-gw-openshift-default-58c847b895-lhr67"] Jul 12 18:24:29.036445 ip-10-0-129-128 kubenswrapper[2559]: I0712 18:24:29.036403 2559 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="c1e7a10f-b3fd-4386-910b-114f73ff2017" path="/var/lib/kubelet/pods/c1e7a10f-b3fd-4386-910b-114f73ff2017/volumes" Jul 12 18:24:34.473341 ip-10-0-129-128 kubenswrapper[2559]: I0712 18:24:34.473293 2559 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-ingress/e2e-ait-67bdeceb-openshift-default-5dff59848c-w2jbt"] Jul 12 18:24:34.473937 ip-10-0-129-128 kubenswrapper[2559]: I0712 18:24:34.473916 2559 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="bf113811-c3e1-4d88-b784-7e9fc71af4e3" containerName="istio-proxy" Jul 12 18:24:34.473937 ip-10-0-129-128 kubenswrapper[2559]: I0712 18:24:34.473939 2559 state_mem.go:107] "Deleted CPUSet assignment" podUID="bf113811-c3e1-4d88-b784-7e9fc71af4e3" containerName="istio-proxy" Jul 12 18:24:34.474084 ip-10-0-129-128 kubenswrapper[2559]: I0712 18:24:34.473952 2559 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="c1e7a10f-b3fd-4386-910b-114f73ff2017" containerName="istio-proxy" Jul 12 18:24:34.474084 ip-10-0-129-128 kubenswrapper[2559]: I0712 18:24:34.473961 2559 state_mem.go:107] "Deleted CPUSet assignment" podUID="c1e7a10f-b3fd-4386-910b-114f73ff2017" containerName="istio-proxy" Jul 12 18:24:34.474084 ip-10-0-129-128 kubenswrapper[2559]: I0712 18:24:34.473984 2559 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="c1e7a10f-b3fd-4386-910b-114f73ff2017" containerName="istio-proxy" Jul 12 18:24:34.474084 ip-10-0-129-128 kubenswrapper[2559]: I0712 18:24:34.473992 2559 state_mem.go:107] "Deleted CPUSet assignment" podUID="c1e7a10f-b3fd-4386-910b-114f73ff2017" containerName="istio-proxy" Jul 12 18:24:34.474295 ip-10-0-129-128 kubenswrapper[2559]: I0712 18:24:34.474097 2559 memory_manager.go:368] "RemoveStaleState removing state" podUID="c1e7a10f-b3fd-4386-910b-114f73ff2017" containerName="istio-proxy" Jul 12 18:24:34.479272 ip-10-0-129-128 kubenswrapper[2559]: I0712 18:24:34.479080 2559 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-ait-67bdeceb-openshift-default-5dff59848c-w2jbt" Jul 12 18:24:34.481478 ip-10-0-129-128 kubenswrapper[2559]: I0712 18:24:34.481459 2559 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress\"/\"e2e-ait-67bdeceb-openshift-default-dockercfg-zxvjs\"" Jul 12 18:24:34.489358 ip-10-0-129-128 kubenswrapper[2559]: I0712 18:24:34.489329 2559 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/e2e-ait-67bdeceb-openshift-default-5dff59848c-w2jbt"] Jul 12 18:24:34.502027 ip-10-0-129-128 kubenswrapper[2559]: I0712 18:24:34.502004 2559 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/090b769a-3586-4fee-9090-33b5200ab66b-istio-data\") pod \"e2e-ait-67bdeceb-openshift-default-5dff59848c-w2jbt\" (UID: \"090b769a-3586-4fee-9090-33b5200ab66b\") " pod="openshift-ingress/e2e-ait-67bdeceb-openshift-default-5dff59848c-w2jbt" Jul 12 18:24:34.502133 ip-10-0-129-128 kubenswrapper[2559]: I0712 18:24:34.502034 2559 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-xtmpm\" (UniqueName: \"kubernetes.io/projected/090b769a-3586-4fee-9090-33b5200ab66b-kube-api-access-xtmpm\") pod \"e2e-ait-67bdeceb-openshift-default-5dff59848c-w2jbt\" (UID: \"090b769a-3586-4fee-9090-33b5200ab66b\") " pod="openshift-ingress/e2e-ait-67bdeceb-openshift-default-5dff59848c-w2jbt" Jul 12 18:24:34.502133 ip-10-0-129-128 kubenswrapper[2559]: I0712 18:24:34.502055 2559 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/090b769a-3586-4fee-9090-33b5200ab66b-workload-certs\") pod \"e2e-ait-67bdeceb-openshift-default-5dff59848c-w2jbt\" (UID: \"090b769a-3586-4fee-9090-33b5200ab66b\") " pod="openshift-ingress/e2e-ait-67bdeceb-openshift-default-5dff59848c-w2jbt" Jul 12 18:24:34.502133 ip-10-0-129-128 kubenswrapper[2559]: I0712 18:24:34.502098 2559 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/090b769a-3586-4fee-9090-33b5200ab66b-istiod-ca-cert\") pod \"e2e-ait-67bdeceb-openshift-default-5dff59848c-w2jbt\" (UID: \"090b769a-3586-4fee-9090-33b5200ab66b\") " pod="openshift-ingress/e2e-ait-67bdeceb-openshift-default-5dff59848c-w2jbt" Jul 12 18:24:34.502281 ip-10-0-129-128 kubenswrapper[2559]: I0712 18:24:34.502169 2559 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/090b769a-3586-4fee-9090-33b5200ab66b-credential-socket\") pod \"e2e-ait-67bdeceb-openshift-default-5dff59848c-w2jbt\" (UID: \"090b769a-3586-4fee-9090-33b5200ab66b\") " pod="openshift-ingress/e2e-ait-67bdeceb-openshift-default-5dff59848c-w2jbt" Jul 12 18:24:34.502281 ip-10-0-129-128 kubenswrapper[2559]: I0712 18:24:34.502212 2559 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/090b769a-3586-4fee-9090-33b5200ab66b-istio-token\") pod \"e2e-ait-67bdeceb-openshift-default-5dff59848c-w2jbt\" (UID: \"090b769a-3586-4fee-9090-33b5200ab66b\") " pod="openshift-ingress/e2e-ait-67bdeceb-openshift-default-5dff59848c-w2jbt" Jul 12 18:24:34.502281 ip-10-0-129-128 kubenswrapper[2559]: I0712 18:24:34.502273 2559 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/090b769a-3586-4fee-9090-33b5200ab66b-istio-podinfo\") pod \"e2e-ait-67bdeceb-openshift-default-5dff59848c-w2jbt\" (UID: \"090b769a-3586-4fee-9090-33b5200ab66b\") " pod="openshift-ingress/e2e-ait-67bdeceb-openshift-default-5dff59848c-w2jbt" Jul 12 18:24:34.502389 ip-10-0-129-128 kubenswrapper[2559]: I0712 18:24:34.502308 2559 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/090b769a-3586-4fee-9090-33b5200ab66b-istio-envoy\") pod \"e2e-ait-67bdeceb-openshift-default-5dff59848c-w2jbt\" (UID: \"090b769a-3586-4fee-9090-33b5200ab66b\") " pod="openshift-ingress/e2e-ait-67bdeceb-openshift-default-5dff59848c-w2jbt" Jul 12 18:24:34.502389 ip-10-0-129-128 kubenswrapper[2559]: I0712 18:24:34.502328 2559 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/090b769a-3586-4fee-9090-33b5200ab66b-workload-socket\") pod \"e2e-ait-67bdeceb-openshift-default-5dff59848c-w2jbt\" (UID: \"090b769a-3586-4fee-9090-33b5200ab66b\") " pod="openshift-ingress/e2e-ait-67bdeceb-openshift-default-5dff59848c-w2jbt" Jul 12 18:24:34.603316 ip-10-0-129-128 kubenswrapper[2559]: I0712 18:24:34.603286 2559 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/090b769a-3586-4fee-9090-33b5200ab66b-istio-podinfo\") pod \"e2e-ait-67bdeceb-openshift-default-5dff59848c-w2jbt\" (UID: \"090b769a-3586-4fee-9090-33b5200ab66b\") " pod="openshift-ingress/e2e-ait-67bdeceb-openshift-default-5dff59848c-w2jbt" Jul 12 18:24:34.603517 ip-10-0-129-128 kubenswrapper[2559]: I0712 18:24:34.603329 2559 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/090b769a-3586-4fee-9090-33b5200ab66b-istio-envoy\") pod \"e2e-ait-67bdeceb-openshift-default-5dff59848c-w2jbt\" (UID: \"090b769a-3586-4fee-9090-33b5200ab66b\") " pod="openshift-ingress/e2e-ait-67bdeceb-openshift-default-5dff59848c-w2jbt" Jul 12 18:24:34.603517 ip-10-0-129-128 kubenswrapper[2559]: I0712 18:24:34.603356 2559 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/090b769a-3586-4fee-9090-33b5200ab66b-workload-socket\") pod \"e2e-ait-67bdeceb-openshift-default-5dff59848c-w2jbt\" (UID: \"090b769a-3586-4fee-9090-33b5200ab66b\") " pod="openshift-ingress/e2e-ait-67bdeceb-openshift-default-5dff59848c-w2jbt" Jul 12 18:24:34.603517 ip-10-0-129-128 kubenswrapper[2559]: I0712 18:24:34.603387 2559 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/090b769a-3586-4fee-9090-33b5200ab66b-istio-data\") pod \"e2e-ait-67bdeceb-openshift-default-5dff59848c-w2jbt\" (UID: \"090b769a-3586-4fee-9090-33b5200ab66b\") " pod="openshift-ingress/e2e-ait-67bdeceb-openshift-default-5dff59848c-w2jbt" Jul 12 18:24:34.603517 ip-10-0-129-128 kubenswrapper[2559]: I0712 18:24:34.603440 2559 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-xtmpm\" (UniqueName: \"kubernetes.io/projected/090b769a-3586-4fee-9090-33b5200ab66b-kube-api-access-xtmpm\") pod \"e2e-ait-67bdeceb-openshift-default-5dff59848c-w2jbt\" (UID: \"090b769a-3586-4fee-9090-33b5200ab66b\") " pod="openshift-ingress/e2e-ait-67bdeceb-openshift-default-5dff59848c-w2jbt" Jul 12 18:24:34.603517 ip-10-0-129-128 kubenswrapper[2559]: I0712 18:24:34.603490 2559 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/090b769a-3586-4fee-9090-33b5200ab66b-workload-certs\") pod \"e2e-ait-67bdeceb-openshift-default-5dff59848c-w2jbt\" (UID: \"090b769a-3586-4fee-9090-33b5200ab66b\") " pod="openshift-ingress/e2e-ait-67bdeceb-openshift-default-5dff59848c-w2jbt" Jul 12 18:24:34.603801 ip-10-0-129-128 kubenswrapper[2559]: I0712 18:24:34.603717 2559 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/090b769a-3586-4fee-9090-33b5200ab66b-istiod-ca-cert\") pod \"e2e-ait-67bdeceb-openshift-default-5dff59848c-w2jbt\" (UID: \"090b769a-3586-4fee-9090-33b5200ab66b\") " pod="openshift-ingress/e2e-ait-67bdeceb-openshift-default-5dff59848c-w2jbt" Jul 12 18:24:34.603801 ip-10-0-129-128 kubenswrapper[2559]: I0712 18:24:34.603770 2559 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/090b769a-3586-4fee-9090-33b5200ab66b-credential-socket\") pod \"e2e-ait-67bdeceb-openshift-default-5dff59848c-w2jbt\" (UID: \"090b769a-3586-4fee-9090-33b5200ab66b\") " pod="openshift-ingress/e2e-ait-67bdeceb-openshift-default-5dff59848c-w2jbt" Jul 12 18:24:34.603917 ip-10-0-129-128 kubenswrapper[2559]: I0712 18:24:34.603819 2559 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/090b769a-3586-4fee-9090-33b5200ab66b-istio-token\") pod \"e2e-ait-67bdeceb-openshift-default-5dff59848c-w2jbt\" (UID: \"090b769a-3586-4fee-9090-33b5200ab66b\") " pod="openshift-ingress/e2e-ait-67bdeceb-openshift-default-5dff59848c-w2jbt" Jul 12 18:24:34.603917 ip-10-0-129-128 kubenswrapper[2559]: I0712 18:24:34.603835 2559 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/090b769a-3586-4fee-9090-33b5200ab66b-istio-data\") pod \"e2e-ait-67bdeceb-openshift-default-5dff59848c-w2jbt\" (UID: \"090b769a-3586-4fee-9090-33b5200ab66b\") " pod="openshift-ingress/e2e-ait-67bdeceb-openshift-default-5dff59848c-w2jbt" Jul 12 18:24:34.603917 ip-10-0-129-128 kubenswrapper[2559]: I0712 18:24:34.603843 2559 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/090b769a-3586-4fee-9090-33b5200ab66b-workload-socket\") pod \"e2e-ait-67bdeceb-openshift-default-5dff59848c-w2jbt\" (UID: \"090b769a-3586-4fee-9090-33b5200ab66b\") " pod="openshift-ingress/e2e-ait-67bdeceb-openshift-default-5dff59848c-w2jbt" Jul 12 18:24:34.604062 ip-10-0-129-128 kubenswrapper[2559]: I0712 18:24:34.604040 2559 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/090b769a-3586-4fee-9090-33b5200ab66b-workload-certs\") pod \"e2e-ait-67bdeceb-openshift-default-5dff59848c-w2jbt\" (UID: \"090b769a-3586-4fee-9090-33b5200ab66b\") " pod="openshift-ingress/e2e-ait-67bdeceb-openshift-default-5dff59848c-w2jbt" Jul 12 18:24:34.604144 ip-10-0-129-128 kubenswrapper[2559]: I0712 18:24:34.604110 2559 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/090b769a-3586-4fee-9090-33b5200ab66b-credential-socket\") pod \"e2e-ait-67bdeceb-openshift-default-5dff59848c-w2jbt\" (UID: \"090b769a-3586-4fee-9090-33b5200ab66b\") " pod="openshift-ingress/e2e-ait-67bdeceb-openshift-default-5dff59848c-w2jbt" Jul 12 18:24:34.604351 ip-10-0-129-128 kubenswrapper[2559]: I0712 18:24:34.604330 2559 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/090b769a-3586-4fee-9090-33b5200ab66b-istiod-ca-cert\") pod \"e2e-ait-67bdeceb-openshift-default-5dff59848c-w2jbt\" (UID: \"090b769a-3586-4fee-9090-33b5200ab66b\") " pod="openshift-ingress/e2e-ait-67bdeceb-openshift-default-5dff59848c-w2jbt" Jul 12 18:24:34.605702 ip-10-0-129-128 kubenswrapper[2559]: I0712 18:24:34.605680 2559 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/090b769a-3586-4fee-9090-33b5200ab66b-istio-envoy\") pod \"e2e-ait-67bdeceb-openshift-default-5dff59848c-w2jbt\" (UID: \"090b769a-3586-4fee-9090-33b5200ab66b\") " pod="openshift-ingress/e2e-ait-67bdeceb-openshift-default-5dff59848c-w2jbt" Jul 12 18:24:34.605928 ip-10-0-129-128 kubenswrapper[2559]: I0712 18:24:34.605910 2559 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/090b769a-3586-4fee-9090-33b5200ab66b-istio-podinfo\") pod \"e2e-ait-67bdeceb-openshift-default-5dff59848c-w2jbt\" (UID: \"090b769a-3586-4fee-9090-33b5200ab66b\") " pod="openshift-ingress/e2e-ait-67bdeceb-openshift-default-5dff59848c-w2jbt" Jul 12 18:24:34.611204 ip-10-0-129-128 kubenswrapper[2559]: I0712 18:24:34.611180 2559 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/090b769a-3586-4fee-9090-33b5200ab66b-istio-token\") pod \"e2e-ait-67bdeceb-openshift-default-5dff59848c-w2jbt\" (UID: \"090b769a-3586-4fee-9090-33b5200ab66b\") " pod="openshift-ingress/e2e-ait-67bdeceb-openshift-default-5dff59848c-w2jbt" Jul 12 18:24:34.612070 ip-10-0-129-128 kubenswrapper[2559]: I0712 18:24:34.612047 2559 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-xtmpm\" (UniqueName: \"kubernetes.io/projected/090b769a-3586-4fee-9090-33b5200ab66b-kube-api-access-xtmpm\") pod \"e2e-ait-67bdeceb-openshift-default-5dff59848c-w2jbt\" (UID: \"090b769a-3586-4fee-9090-33b5200ab66b\") " pod="openshift-ingress/e2e-ait-67bdeceb-openshift-default-5dff59848c-w2jbt" Jul 12 18:24:34.793230 ip-10-0-129-128 kubenswrapper[2559]: I0712 18:24:34.793195 2559 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-ait-67bdeceb-openshift-default-5dff59848c-w2jbt" Jul 12 18:24:34.926023 ip-10-0-129-128 kubenswrapper[2559]: I0712 18:24:34.925997 2559 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/e2e-ait-67bdeceb-openshift-default-5dff59848c-w2jbt"] Jul 12 18:24:34.928271 ip-10-0-129-128 kubenswrapper[2559]: W0712 18:24:34.928235 2559 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod090b769a_3586_4fee_9090_33b5200ab66b.slice/crio-864f8e04fd00597c1f4f4de2d9c6903fb4f09cb64248b48b5ef3c824c36e6f99 WatchSource:0}: Error finding container 864f8e04fd00597c1f4f4de2d9c6903fb4f09cb64248b48b5ef3c824c36e6f99: Status 404 returned error can't find the container with id 864f8e04fd00597c1f4f4de2d9c6903fb4f09cb64248b48b5ef3c824c36e6f99 Jul 12 18:24:34.930410 ip-10-0-129-128 kubenswrapper[2559]: I0712 18:24:34.930374 2559 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236228Ki","pods":"250"} Jul 12 18:24:34.930553 ip-10-0-129-128 kubenswrapper[2559]: I0712 18:24:34.930440 2559 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236228Ki","pods":"250"} Jul 12 18:24:34.930553 ip-10-0-129-128 kubenswrapper[2559]: I0712 18:24:34.930467 2559 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236228Ki","pods":"250"} Jul 12 18:24:35.095154 ip-10-0-129-128 kubenswrapper[2559]: I0712 18:24:35.095058 2559 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-ait-67bdeceb-openshift-default-5dff59848c-w2jbt" event={"ID":"090b769a-3586-4fee-9090-33b5200ab66b","Type":"ContainerStarted","Data":"ff3d26d7500312eafc153b7c62fa87d55f92762cea8f1572c6c1d333af31f85e"} Jul 12 18:24:35.095154 ip-10-0-129-128 kubenswrapper[2559]: I0712 18:24:35.095116 2559 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-ait-67bdeceb-openshift-default-5dff59848c-w2jbt" event={"ID":"090b769a-3586-4fee-9090-33b5200ab66b","Type":"ContainerStarted","Data":"864f8e04fd00597c1f4f4de2d9c6903fb4f09cb64248b48b5ef3c824c36e6f99"} Jul 12 18:24:35.114346 ip-10-0-129-128 kubenswrapper[2559]: I0712 18:24:35.114284 2559 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-ingress/e2e-ait-67bdeceb-openshift-default-5dff59848c-w2jbt" podStartSLOduration=1.114266187 podStartE2EDuration="1.114266187s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-12 18:24:34 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-12 18:24:35.113195355 +0000 UTC m=+1882.607263288" watchObservedRunningTime="2026-07-12 18:24:35.114266187 +0000 UTC m=+1882.608334118" Jul 12 18:24:35.793503 ip-10-0-129-128 kubenswrapper[2559]: I0712 18:24:35.793471 2559 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="openshift-ingress/e2e-ait-67bdeceb-openshift-default-5dff59848c-w2jbt" Jul 12 18:24:35.798386 ip-10-0-129-128 kubenswrapper[2559]: I0712 18:24:35.798363 2559 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="openshift-ingress/e2e-ait-67bdeceb-openshift-default-5dff59848c-w2jbt" Jul 12 18:24:36.100235 ip-10-0-129-128 kubenswrapper[2559]: I0712 18:24:36.100149 2559 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/e2e-ait-67bdeceb-openshift-default-5dff59848c-w2jbt" Jul 12 18:24:36.101218 ip-10-0-129-128 kubenswrapper[2559]: I0712 18:24:36.101198 2559 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/e2e-ait-67bdeceb-openshift-default-5dff59848c-w2jbt" Jul 12 18:24:39.387813 ip-10-0-129-128 kubenswrapper[2559]: I0712 18:24:39.387774 2559 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-pwsvh"] Jul 12 18:24:48.180953 ip-10-0-129-128 kubenswrapper[2559]: I0712 18:24:48.180921 2559 generic.go:356] "Generic (PLEG): container finished" podID="090b769a-3586-4fee-9090-33b5200ab66b" containerID="ff3d26d7500312eafc153b7c62fa87d55f92762cea8f1572c6c1d333af31f85e" exitCode=0 Jul 12 18:24:48.181379 ip-10-0-129-128 kubenswrapper[2559]: I0712 18:24:48.181001 2559 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-ait-67bdeceb-openshift-default-5dff59848c-w2jbt" event={"ID":"090b769a-3586-4fee-9090-33b5200ab66b","Type":"ContainerDied","Data":"ff3d26d7500312eafc153b7c62fa87d55f92762cea8f1572c6c1d333af31f85e"} Jul 12 18:24:48.181425 ip-10-0-129-128 kubenswrapper[2559]: I0712 18:24:48.181378 2559 scope.go:117] "RemoveContainer" containerID="ff3d26d7500312eafc153b7c62fa87d55f92762cea8f1572c6c1d333af31f85e" Jul 12 18:24:48.182474 ip-10-0-129-128 kubenswrapper[2559]: I0712 18:24:48.182448 2559 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236228Ki","pods":"250"} Jul 12 18:24:48.182566 ip-10-0-129-128 kubenswrapper[2559]: I0712 18:24:48.182528 2559 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236228Ki","pods":"250"} Jul 12 18:24:48.182566 ip-10-0-129-128 kubenswrapper[2559]: I0712 18:24:48.182562 2559 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236228Ki","pods":"250"} Jul 12 18:24:49.188550 ip-10-0-129-128 kubenswrapper[2559]: I0712 18:24:49.188513 2559 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-ait-67bdeceb-openshift-default-5dff59848c-w2jbt" event={"ID":"090b769a-3586-4fee-9090-33b5200ab66b","Type":"ContainerStarted","Data":"87ff8976b7a1dfd6e8766f5e9743438ae5f644ea8a0960f8b45f6393aa408721"} Jul 12 18:24:49.794033 ip-10-0-129-128 kubenswrapper[2559]: I0712 18:24:49.793996 2559 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="openshift-ingress/e2e-ait-67bdeceb-openshift-default-5dff59848c-w2jbt" Jul 12 18:24:49.795018 ip-10-0-129-128 kubenswrapper[2559]: I0712 18:24:49.794989 2559 patch_prober.go:28] interesting pod/e2e-ait-67bdeceb-openshift-default-5dff59848c-w2jbt container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.132.0.44:15021/healthz/ready\": dial tcp 10.132.0.44:15021: connect: connection refused" start-of-body= Jul 12 18:24:49.795161 ip-10-0-129-128 kubenswrapper[2559]: I0712 18:24:49.795059 2559 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-ait-67bdeceb-openshift-default-5dff59848c-w2jbt" podUID="090b769a-3586-4fee-9090-33b5200ab66b" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.132.0.44:15021/healthz/ready\": dial tcp 10.132.0.44:15021: connect: connection refused" Jul 12 18:24:50.793988 ip-10-0-129-128 kubenswrapper[2559]: I0712 18:24:50.793944 2559 patch_prober.go:28] interesting pod/e2e-ait-67bdeceb-openshift-default-5dff59848c-w2jbt container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.132.0.44:15021/healthz/ready\": dial tcp 10.132.0.44:15021: connect: connection refused" start-of-body= Jul 12 18:24:50.794458 ip-10-0-129-128 kubenswrapper[2559]: I0712 18:24:50.794007 2559 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-ait-67bdeceb-openshift-default-5dff59848c-w2jbt" podUID="090b769a-3586-4fee-9090-33b5200ab66b" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.132.0.44:15021/healthz/ready\": dial tcp 10.132.0.44:15021: connect: connection refused" Jul 12 18:24:51.100677 ip-10-0-129-128 kubenswrapper[2559]: I0712 18:24:51.100564 2559 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/e2e-ait-67bdeceb-openshift-default-5dff59848c-w2jbt" Jul 12 18:24:51.794311 ip-10-0-129-128 kubenswrapper[2559]: I0712 18:24:51.794277 2559 patch_prober.go:28] interesting pod/e2e-ait-67bdeceb-openshift-default-5dff59848c-w2jbt container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.132.0.44:15021/healthz/ready\": dial tcp 10.132.0.44:15021: connect: connection refused" start-of-body= Jul 12 18:24:51.794718 ip-10-0-129-128 kubenswrapper[2559]: I0712 18:24:51.794331 2559 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-ait-67bdeceb-openshift-default-5dff59848c-w2jbt" podUID="090b769a-3586-4fee-9090-33b5200ab66b" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.132.0.44:15021/healthz/ready\": dial tcp 10.132.0.44:15021: connect: connection refused" Jul 12 18:24:52.793742 ip-10-0-129-128 kubenswrapper[2559]: I0712 18:24:52.793705 2559 patch_prober.go:28] interesting pod/e2e-ait-67bdeceb-openshift-default-5dff59848c-w2jbt container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.132.0.44:15021/healthz/ready\": dial tcp 10.132.0.44:15021: connect: connection refused" start-of-body= Jul 12 18:24:52.793934 ip-10-0-129-128 kubenswrapper[2559]: I0712 18:24:52.793790 2559 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-ait-67bdeceb-openshift-default-5dff59848c-w2jbt" podUID="090b769a-3586-4fee-9090-33b5200ab66b" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.132.0.44:15021/healthz/ready\": dial tcp 10.132.0.44:15021: connect: connection refused" Jul 12 18:24:53.798080 ip-10-0-129-128 kubenswrapper[2559]: I0712 18:24:53.798048 2559 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="openshift-ingress/e2e-ait-67bdeceb-openshift-default-5dff59848c-w2jbt" Jul 12 18:24:53.799123 ip-10-0-129-128 kubenswrapper[2559]: I0712 18:24:53.799100 2559 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/e2e-ait-67bdeceb-openshift-default-5dff59848c-w2jbt" Jul 12 18:28:13.095255 ip-10-0-129-128 kubenswrapper[2559]: I0712 18:28:13.095223 2559 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-console-operator_console-operator-7866497494-wlk78_f1d46430-bb60-4470-b8d8-eacd44f4f6b1/console-operator/1.log" Jul 12 18:28:13.097864 ip-10-0-129-128 kubenswrapper[2559]: I0712 18:28:13.097840 2559 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ovn-kubernetes_ovnkube-node-q5dw8_e51bad50-bc74-4e38-8307-ca257b599314/ovn-acl-logging/0.log" Jul 12 18:28:13.098104 ip-10-0-129-128 kubenswrapper[2559]: I0712 18:28:13.098083 2559 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-console-operator_console-operator-7866497494-wlk78_f1d46430-bb60-4470-b8d8-eacd44f4f6b1/console-operator/1.log" Jul 12 18:28:13.100665 ip-10-0-129-128 kubenswrapper[2559]: I0712 18:28:13.100632 2559 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ovn-kubernetes_ovnkube-node-q5dw8_e51bad50-bc74-4e38-8307-ca257b599314/ovn-acl-logging/0.log" Jul 12 18:30:40.937358 ip-10-0-129-128 kubenswrapper[2559]: I0712 18:30:40.937317 2559 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-ingress/e2e-ait-67bdeceb-openshift-default-5dff59848c-w2jbt"] Jul 12 18:30:40.939870 ip-10-0-129-128 kubenswrapper[2559]: I0712 18:30:40.937622 2559 kuberuntime_container.go:873] "Killing container with a grace period" pod="openshift-ingress/e2e-ait-67bdeceb-openshift-default-5dff59848c-w2jbt" podUID="090b769a-3586-4fee-9090-33b5200ab66b" containerName="istio-proxy" containerID="cri-o://87ff8976b7a1dfd6e8766f5e9743438ae5f644ea8a0960f8b45f6393aa408721" gracePeriod=30 Jul 12 18:30:43.985266 ip-10-0-129-128 kubenswrapper[2559]: I0712 18:30:43.985228 2559 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-pwsvh"] Jul 12 18:30:46.199542 ip-10-0-129-128 kubenswrapper[2559]: I0712 18:30:46.199475 2559 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-ait-67bdeceb-openshift-default-5dff59848c-w2jbt" Jul 12 18:30:46.283807 ip-10-0-129-128 kubenswrapper[2559]: I0712 18:30:46.283766 2559 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/090b769a-3586-4fee-9090-33b5200ab66b-credential-socket\") pod \"090b769a-3586-4fee-9090-33b5200ab66b\" (UID: \"090b769a-3586-4fee-9090-33b5200ab66b\") " Jul 12 18:30:46.283807 ip-10-0-129-128 kubenswrapper[2559]: I0712 18:30:46.283809 2559 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/090b769a-3586-4fee-9090-33b5200ab66b-workload-socket\") pod \"090b769a-3586-4fee-9090-33b5200ab66b\" (UID: \"090b769a-3586-4fee-9090-33b5200ab66b\") " Jul 12 18:30:46.284041 ip-10-0-129-128 kubenswrapper[2559]: I0712 18:30:46.283848 2559 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/090b769a-3586-4fee-9090-33b5200ab66b-istiod-ca-cert\") pod \"090b769a-3586-4fee-9090-33b5200ab66b\" (UID: \"090b769a-3586-4fee-9090-33b5200ab66b\") " Jul 12 18:30:46.284041 ip-10-0-129-128 kubenswrapper[2559]: I0712 18:30:46.283881 2559 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-xtmpm\" (UniqueName: \"kubernetes.io/projected/090b769a-3586-4fee-9090-33b5200ab66b-kube-api-access-xtmpm\") pod \"090b769a-3586-4fee-9090-33b5200ab66b\" (UID: \"090b769a-3586-4fee-9090-33b5200ab66b\") " Jul 12 18:30:46.284041 ip-10-0-129-128 kubenswrapper[2559]: I0712 18:30:46.283906 2559 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/090b769a-3586-4fee-9090-33b5200ab66b-istio-podinfo\") pod \"090b769a-3586-4fee-9090-33b5200ab66b\" (UID: \"090b769a-3586-4fee-9090-33b5200ab66b\") " Jul 12 18:30:46.284041 ip-10-0-129-128 kubenswrapper[2559]: I0712 18:30:46.283935 2559 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/090b769a-3586-4fee-9090-33b5200ab66b-workload-certs\") pod \"090b769a-3586-4fee-9090-33b5200ab66b\" (UID: \"090b769a-3586-4fee-9090-33b5200ab66b\") " Jul 12 18:30:46.284041 ip-10-0-129-128 kubenswrapper[2559]: I0712 18:30:46.283972 2559 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/090b769a-3586-4fee-9090-33b5200ab66b-istio-token\") pod \"090b769a-3586-4fee-9090-33b5200ab66b\" (UID: \"090b769a-3586-4fee-9090-33b5200ab66b\") " Jul 12 18:30:46.284041 ip-10-0-129-128 kubenswrapper[2559]: I0712 18:30:46.284034 2559 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/090b769a-3586-4fee-9090-33b5200ab66b-istio-envoy\") pod \"090b769a-3586-4fee-9090-33b5200ab66b\" (UID: \"090b769a-3586-4fee-9090-33b5200ab66b\") " Jul 12 18:30:46.284361 ip-10-0-129-128 kubenswrapper[2559]: I0712 18:30:46.284043 2559 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/090b769a-3586-4fee-9090-33b5200ab66b-credential-socket" (OuterVolumeSpecName: "credential-socket") pod "090b769a-3586-4fee-9090-33b5200ab66b" (UID: "090b769a-3586-4fee-9090-33b5200ab66b"). InnerVolumeSpecName "credential-socket". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Jul 12 18:30:46.284361 ip-10-0-129-128 kubenswrapper[2559]: I0712 18:30:46.284077 2559 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/090b769a-3586-4fee-9090-33b5200ab66b-istio-data\") pod \"090b769a-3586-4fee-9090-33b5200ab66b\" (UID: \"090b769a-3586-4fee-9090-33b5200ab66b\") " Jul 12 18:30:46.284361 ip-10-0-129-128 kubenswrapper[2559]: I0712 18:30:46.284089 2559 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/090b769a-3586-4fee-9090-33b5200ab66b-workload-socket" (OuterVolumeSpecName: "workload-socket") pod "090b769a-3586-4fee-9090-33b5200ab66b" (UID: "090b769a-3586-4fee-9090-33b5200ab66b"). InnerVolumeSpecName "workload-socket". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Jul 12 18:30:46.284361 ip-10-0-129-128 kubenswrapper[2559]: I0712 18:30:46.284238 2559 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/090b769a-3586-4fee-9090-33b5200ab66b-workload-certs" (OuterVolumeSpecName: "workload-certs") pod "090b769a-3586-4fee-9090-33b5200ab66b" (UID: "090b769a-3586-4fee-9090-33b5200ab66b"). InnerVolumeSpecName "workload-certs". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Jul 12 18:30:46.284361 ip-10-0-129-128 kubenswrapper[2559]: I0712 18:30:46.284320 2559 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/090b769a-3586-4fee-9090-33b5200ab66b-istiod-ca-cert" (OuterVolumeSpecName: "istiod-ca-cert") pod "090b769a-3586-4fee-9090-33b5200ab66b" (UID: "090b769a-3586-4fee-9090-33b5200ab66b"). InnerVolumeSpecName "istiod-ca-cert". PluginName "kubernetes.io/configmap", VolumeGIDValue "" Jul 12 18:30:46.284361 ip-10-0-129-128 kubenswrapper[2559]: I0712 18:30:46.284347 2559 reconciler_common.go:299] "Volume detached for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/090b769a-3586-4fee-9090-33b5200ab66b-credential-socket\") on node \"ip-10-0-129-128.ec2.internal\" DevicePath \"\"" Jul 12 18:30:46.284717 ip-10-0-129-128 kubenswrapper[2559]: I0712 18:30:46.284364 2559 reconciler_common.go:299] "Volume detached for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/090b769a-3586-4fee-9090-33b5200ab66b-workload-socket\") on node \"ip-10-0-129-128.ec2.internal\" DevicePath \"\"" Jul 12 18:30:46.284717 ip-10-0-129-128 kubenswrapper[2559]: I0712 18:30:46.284378 2559 reconciler_common.go:299] "Volume detached for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/090b769a-3586-4fee-9090-33b5200ab66b-workload-certs\") on node \"ip-10-0-129-128.ec2.internal\" DevicePath \"\"" Jul 12 18:30:46.285327 ip-10-0-129-128 kubenswrapper[2559]: I0712 18:30:46.285294 2559 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/090b769a-3586-4fee-9090-33b5200ab66b-istio-data" (OuterVolumeSpecName: "istio-data") pod "090b769a-3586-4fee-9090-33b5200ab66b" (UID: "090b769a-3586-4fee-9090-33b5200ab66b"). InnerVolumeSpecName "istio-data". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Jul 12 18:30:46.286309 ip-10-0-129-128 kubenswrapper[2559]: I0712 18:30:46.286273 2559 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/downward-api/090b769a-3586-4fee-9090-33b5200ab66b-istio-podinfo" (OuterVolumeSpecName: "istio-podinfo") pod "090b769a-3586-4fee-9090-33b5200ab66b" (UID: "090b769a-3586-4fee-9090-33b5200ab66b"). InnerVolumeSpecName "istio-podinfo". PluginName "kubernetes.io/downward-api", VolumeGIDValue "" Jul 12 18:30:46.286438 ip-10-0-129-128 kubenswrapper[2559]: I0712 18:30:46.286415 2559 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/090b769a-3586-4fee-9090-33b5200ab66b-istio-envoy" (OuterVolumeSpecName: "istio-envoy") pod "090b769a-3586-4fee-9090-33b5200ab66b" (UID: "090b769a-3586-4fee-9090-33b5200ab66b"). InnerVolumeSpecName "istio-envoy". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Jul 12 18:30:46.286573 ip-10-0-129-128 kubenswrapper[2559]: I0712 18:30:46.286552 2559 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/090b769a-3586-4fee-9090-33b5200ab66b-istio-token" (OuterVolumeSpecName: "istio-token") pod "090b769a-3586-4fee-9090-33b5200ab66b" (UID: "090b769a-3586-4fee-9090-33b5200ab66b"). InnerVolumeSpecName "istio-token". PluginName "kubernetes.io/projected", VolumeGIDValue "" Jul 12 18:30:46.286749 ip-10-0-129-128 kubenswrapper[2559]: I0712 18:30:46.286606 2559 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/090b769a-3586-4fee-9090-33b5200ab66b-kube-api-access-xtmpm" (OuterVolumeSpecName: "kube-api-access-xtmpm") pod "090b769a-3586-4fee-9090-33b5200ab66b" (UID: "090b769a-3586-4fee-9090-33b5200ab66b"). InnerVolumeSpecName "kube-api-access-xtmpm". PluginName "kubernetes.io/projected", VolumeGIDValue "" Jul 12 18:30:46.384946 ip-10-0-129-128 kubenswrapper[2559]: I0712 18:30:46.384909 2559 reconciler_common.go:299] "Volume detached for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/090b769a-3586-4fee-9090-33b5200ab66b-istiod-ca-cert\") on node \"ip-10-0-129-128.ec2.internal\" DevicePath \"\"" Jul 12 18:30:46.384946 ip-10-0-129-128 kubenswrapper[2559]: I0712 18:30:46.384946 2559 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-xtmpm\" (UniqueName: \"kubernetes.io/projected/090b769a-3586-4fee-9090-33b5200ab66b-kube-api-access-xtmpm\") on node \"ip-10-0-129-128.ec2.internal\" DevicePath \"\"" Jul 12 18:30:46.384946 ip-10-0-129-128 kubenswrapper[2559]: I0712 18:30:46.384962 2559 reconciler_common.go:299] "Volume detached for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/090b769a-3586-4fee-9090-33b5200ab66b-istio-podinfo\") on node \"ip-10-0-129-128.ec2.internal\" DevicePath \"\"" Jul 12 18:30:46.385236 ip-10-0-129-128 kubenswrapper[2559]: I0712 18:30:46.384975 2559 reconciler_common.go:299] "Volume detached for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/090b769a-3586-4fee-9090-33b5200ab66b-istio-token\") on node \"ip-10-0-129-128.ec2.internal\" DevicePath \"\"" Jul 12 18:30:46.385236 ip-10-0-129-128 kubenswrapper[2559]: I0712 18:30:46.384989 2559 reconciler_common.go:299] "Volume detached for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/090b769a-3586-4fee-9090-33b5200ab66b-istio-envoy\") on node \"ip-10-0-129-128.ec2.internal\" DevicePath \"\"" Jul 12 18:30:46.385236 ip-10-0-129-128 kubenswrapper[2559]: I0712 18:30:46.385000 2559 reconciler_common.go:299] "Volume detached for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/090b769a-3586-4fee-9090-33b5200ab66b-istio-data\") on node \"ip-10-0-129-128.ec2.internal\" DevicePath \"\"" Jul 12 18:30:46.925316 ip-10-0-129-128 kubenswrapper[2559]: I0712 18:30:46.925275 2559 generic.go:356] "Generic (PLEG): container finished" podID="090b769a-3586-4fee-9090-33b5200ab66b" containerID="87ff8976b7a1dfd6e8766f5e9743438ae5f644ea8a0960f8b45f6393aa408721" exitCode=0 Jul 12 18:30:46.925492 ip-10-0-129-128 kubenswrapper[2559]: I0712 18:30:46.925329 2559 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-ait-67bdeceb-openshift-default-5dff59848c-w2jbt" Jul 12 18:30:46.925492 ip-10-0-129-128 kubenswrapper[2559]: I0712 18:30:46.925324 2559 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-ait-67bdeceb-openshift-default-5dff59848c-w2jbt" event={"ID":"090b769a-3586-4fee-9090-33b5200ab66b","Type":"ContainerDied","Data":"87ff8976b7a1dfd6e8766f5e9743438ae5f644ea8a0960f8b45f6393aa408721"} Jul 12 18:30:46.925492 ip-10-0-129-128 kubenswrapper[2559]: I0712 18:30:46.925440 2559 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-ait-67bdeceb-openshift-default-5dff59848c-w2jbt" event={"ID":"090b769a-3586-4fee-9090-33b5200ab66b","Type":"ContainerDied","Data":"864f8e04fd00597c1f4f4de2d9c6903fb4f09cb64248b48b5ef3c824c36e6f99"} Jul 12 18:30:46.925492 ip-10-0-129-128 kubenswrapper[2559]: I0712 18:30:46.925475 2559 scope.go:117] "RemoveContainer" containerID="87ff8976b7a1dfd6e8766f5e9743438ae5f644ea8a0960f8b45f6393aa408721" Jul 12 18:30:46.935667 ip-10-0-129-128 kubenswrapper[2559]: I0712 18:30:46.935620 2559 scope.go:117] "RemoveContainer" containerID="ff3d26d7500312eafc153b7c62fa87d55f92762cea8f1572c6c1d333af31f85e" Jul 12 18:30:46.943878 ip-10-0-129-128 kubenswrapper[2559]: I0712 18:30:46.943861 2559 scope.go:117] "RemoveContainer" containerID="87ff8976b7a1dfd6e8766f5e9743438ae5f644ea8a0960f8b45f6393aa408721" Jul 12 18:30:46.944160 ip-10-0-129-128 kubenswrapper[2559]: E0712 18:30:46.944142 2559 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"87ff8976b7a1dfd6e8766f5e9743438ae5f644ea8a0960f8b45f6393aa408721\": container with ID starting with 87ff8976b7a1dfd6e8766f5e9743438ae5f644ea8a0960f8b45f6393aa408721 not found: ID does not exist" containerID="87ff8976b7a1dfd6e8766f5e9743438ae5f644ea8a0960f8b45f6393aa408721" Jul 12 18:30:46.944205 ip-10-0-129-128 kubenswrapper[2559]: I0712 18:30:46.944170 2559 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"87ff8976b7a1dfd6e8766f5e9743438ae5f644ea8a0960f8b45f6393aa408721"} err="failed to get container status \"87ff8976b7a1dfd6e8766f5e9743438ae5f644ea8a0960f8b45f6393aa408721\": rpc error: code = NotFound desc = could not find container \"87ff8976b7a1dfd6e8766f5e9743438ae5f644ea8a0960f8b45f6393aa408721\": container with ID starting with 87ff8976b7a1dfd6e8766f5e9743438ae5f644ea8a0960f8b45f6393aa408721 not found: ID does not exist" Jul 12 18:30:46.944205 ip-10-0-129-128 kubenswrapper[2559]: I0712 18:30:46.944189 2559 scope.go:117] "RemoveContainer" containerID="ff3d26d7500312eafc153b7c62fa87d55f92762cea8f1572c6c1d333af31f85e" Jul 12 18:30:46.944505 ip-10-0-129-128 kubenswrapper[2559]: E0712 18:30:46.944480 2559 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"ff3d26d7500312eafc153b7c62fa87d55f92762cea8f1572c6c1d333af31f85e\": container with ID starting with ff3d26d7500312eafc153b7c62fa87d55f92762cea8f1572c6c1d333af31f85e not found: ID does not exist" containerID="ff3d26d7500312eafc153b7c62fa87d55f92762cea8f1572c6c1d333af31f85e" Jul 12 18:30:46.944611 ip-10-0-129-128 kubenswrapper[2559]: I0712 18:30:46.944512 2559 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"ff3d26d7500312eafc153b7c62fa87d55f92762cea8f1572c6c1d333af31f85e"} err="failed to get container status \"ff3d26d7500312eafc153b7c62fa87d55f92762cea8f1572c6c1d333af31f85e\": rpc error: code = NotFound desc = could not find container \"ff3d26d7500312eafc153b7c62fa87d55f92762cea8f1572c6c1d333af31f85e\": container with ID starting with ff3d26d7500312eafc153b7c62fa87d55f92762cea8f1572c6c1d333af31f85e not found: ID does not exist" Jul 12 18:30:46.946880 ip-10-0-129-128 kubenswrapper[2559]: I0712 18:30:46.946854 2559 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-ingress/e2e-ait-67bdeceb-openshift-default-5dff59848c-w2jbt"] Jul 12 18:30:46.948883 ip-10-0-129-128 kubenswrapper[2559]: I0712 18:30:46.948864 2559 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["openshift-ingress/e2e-ait-67bdeceb-openshift-default-5dff59848c-w2jbt"] Jul 12 18:30:47.036052 ip-10-0-129-128 kubenswrapper[2559]: I0712 18:30:47.036016 2559 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="090b769a-3586-4fee-9090-33b5200ab66b" path="/var/lib/kubelet/pods/090b769a-3586-4fee-9090-33b5200ab66b/volumes" Jul 12 18:30:52.614140 ip-10-0-129-128 kubenswrapper[2559]: I0712 18:30:52.614109 2559 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-ingress/e2e-derive-d19d5af8-openshift-default-84f845d559-lr8v2"] Jul 12 18:30:52.614542 ip-10-0-129-128 kubenswrapper[2559]: I0712 18:30:52.614456 2559 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="090b769a-3586-4fee-9090-33b5200ab66b" containerName="istio-proxy" Jul 12 18:30:52.614542 ip-10-0-129-128 kubenswrapper[2559]: I0712 18:30:52.614467 2559 state_mem.go:107] "Deleted CPUSet assignment" podUID="090b769a-3586-4fee-9090-33b5200ab66b" containerName="istio-proxy" Jul 12 18:30:52.614542 ip-10-0-129-128 kubenswrapper[2559]: I0712 18:30:52.614477 2559 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="090b769a-3586-4fee-9090-33b5200ab66b" containerName="istio-proxy" Jul 12 18:30:52.614542 ip-10-0-129-128 kubenswrapper[2559]: I0712 18:30:52.614482 2559 state_mem.go:107] "Deleted CPUSet assignment" podUID="090b769a-3586-4fee-9090-33b5200ab66b" containerName="istio-proxy" Jul 12 18:30:52.614691 ip-10-0-129-128 kubenswrapper[2559]: I0712 18:30:52.614580 2559 memory_manager.go:368] "RemoveStaleState removing state" podUID="c1e7a10f-b3fd-4386-910b-114f73ff2017" containerName="istio-proxy" Jul 12 18:30:52.614691 ip-10-0-129-128 kubenswrapper[2559]: I0712 18:30:52.614590 2559 memory_manager.go:368] "RemoveStaleState removing state" podUID="090b769a-3586-4fee-9090-33b5200ab66b" containerName="istio-proxy" Jul 12 18:30:52.614691 ip-10-0-129-128 kubenswrapper[2559]: I0712 18:30:52.614596 2559 memory_manager.go:368] "RemoveStaleState removing state" podUID="090b769a-3586-4fee-9090-33b5200ab66b" containerName="istio-proxy" Jul 12 18:30:52.617588 ip-10-0-129-128 kubenswrapper[2559]: I0712 18:30:52.617571 2559 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-derive-d19d5af8-openshift-default-84f845d559-lr8v2" Jul 12 18:30:52.620563 ip-10-0-129-128 kubenswrapper[2559]: I0712 18:30:52.620544 2559 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress\"/\"e2e-derive-d19d5af8-openshift-default-dockercfg-5j8rl\"" Jul 12 18:30:52.628922 ip-10-0-129-128 kubenswrapper[2559]: I0712 18:30:52.628902 2559 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/e2e-derive-d19d5af8-openshift-default-84f845d559-lr8v2"] Jul 12 18:30:52.642588 ip-10-0-129-128 kubenswrapper[2559]: I0712 18:30:52.642552 2559 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-mz2mt\" (UniqueName: \"kubernetes.io/projected/9f172362-788e-4ce3-a328-f967e623de5f-kube-api-access-mz2mt\") pod \"e2e-derive-d19d5af8-openshift-default-84f845d559-lr8v2\" (UID: \"9f172362-788e-4ce3-a328-f967e623de5f\") " pod="openshift-ingress/e2e-derive-d19d5af8-openshift-default-84f845d559-lr8v2" Jul 12 18:30:52.642740 ip-10-0-129-128 kubenswrapper[2559]: I0712 18:30:52.642716 2559 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/9f172362-788e-4ce3-a328-f967e623de5f-workload-socket\") pod \"e2e-derive-d19d5af8-openshift-default-84f845d559-lr8v2\" (UID: \"9f172362-788e-4ce3-a328-f967e623de5f\") " pod="openshift-ingress/e2e-derive-d19d5af8-openshift-default-84f845d559-lr8v2" Jul 12 18:30:52.642782 ip-10-0-129-128 kubenswrapper[2559]: I0712 18:30:52.642755 2559 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/9f172362-788e-4ce3-a328-f967e623de5f-credential-socket\") pod \"e2e-derive-d19d5af8-openshift-default-84f845d559-lr8v2\" (UID: \"9f172362-788e-4ce3-a328-f967e623de5f\") " pod="openshift-ingress/e2e-derive-d19d5af8-openshift-default-84f845d559-lr8v2" Jul 12 18:30:52.642819 ip-10-0-129-128 kubenswrapper[2559]: I0712 18:30:52.642781 2559 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/9f172362-788e-4ce3-a328-f967e623de5f-workload-certs\") pod \"e2e-derive-d19d5af8-openshift-default-84f845d559-lr8v2\" (UID: \"9f172362-788e-4ce3-a328-f967e623de5f\") " pod="openshift-ingress/e2e-derive-d19d5af8-openshift-default-84f845d559-lr8v2" Jul 12 18:30:52.642868 ip-10-0-129-128 kubenswrapper[2559]: I0712 18:30:52.642846 2559 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/9f172362-788e-4ce3-a328-f967e623de5f-istiod-ca-cert\") pod \"e2e-derive-d19d5af8-openshift-default-84f845d559-lr8v2\" (UID: \"9f172362-788e-4ce3-a328-f967e623de5f\") " pod="openshift-ingress/e2e-derive-d19d5af8-openshift-default-84f845d559-lr8v2" Jul 12 18:30:52.642934 ip-10-0-129-128 kubenswrapper[2559]: I0712 18:30:52.642916 2559 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/9f172362-788e-4ce3-a328-f967e623de5f-istio-podinfo\") pod \"e2e-derive-d19d5af8-openshift-default-84f845d559-lr8v2\" (UID: \"9f172362-788e-4ce3-a328-f967e623de5f\") " pod="openshift-ingress/e2e-derive-d19d5af8-openshift-default-84f845d559-lr8v2" Jul 12 18:30:52.642976 ip-10-0-129-128 kubenswrapper[2559]: I0712 18:30:52.642947 2559 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/9f172362-788e-4ce3-a328-f967e623de5f-istio-envoy\") pod \"e2e-derive-d19d5af8-openshift-default-84f845d559-lr8v2\" (UID: \"9f172362-788e-4ce3-a328-f967e623de5f\") " pod="openshift-ingress/e2e-derive-d19d5af8-openshift-default-84f845d559-lr8v2" Jul 12 18:30:52.642976 ip-10-0-129-128 kubenswrapper[2559]: I0712 18:30:52.642969 2559 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/9f172362-788e-4ce3-a328-f967e623de5f-istio-token\") pod \"e2e-derive-d19d5af8-openshift-default-84f845d559-lr8v2\" (UID: \"9f172362-788e-4ce3-a328-f967e623de5f\") " pod="openshift-ingress/e2e-derive-d19d5af8-openshift-default-84f845d559-lr8v2" Jul 12 18:30:52.643039 ip-10-0-129-128 kubenswrapper[2559]: I0712 18:30:52.642991 2559 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/9f172362-788e-4ce3-a328-f967e623de5f-istio-data\") pod \"e2e-derive-d19d5af8-openshift-default-84f845d559-lr8v2\" (UID: \"9f172362-788e-4ce3-a328-f967e623de5f\") " pod="openshift-ingress/e2e-derive-d19d5af8-openshift-default-84f845d559-lr8v2" Jul 12 18:30:52.744210 ip-10-0-129-128 kubenswrapper[2559]: I0712 18:30:52.744179 2559 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/9f172362-788e-4ce3-a328-f967e623de5f-workload-socket\") pod \"e2e-derive-d19d5af8-openshift-default-84f845d559-lr8v2\" (UID: \"9f172362-788e-4ce3-a328-f967e623de5f\") " pod="openshift-ingress/e2e-derive-d19d5af8-openshift-default-84f845d559-lr8v2" Jul 12 18:30:52.744210 ip-10-0-129-128 kubenswrapper[2559]: I0712 18:30:52.744220 2559 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/9f172362-788e-4ce3-a328-f967e623de5f-credential-socket\") pod \"e2e-derive-d19d5af8-openshift-default-84f845d559-lr8v2\" (UID: \"9f172362-788e-4ce3-a328-f967e623de5f\") " pod="openshift-ingress/e2e-derive-d19d5af8-openshift-default-84f845d559-lr8v2" Jul 12 18:30:52.744488 ip-10-0-129-128 kubenswrapper[2559]: I0712 18:30:52.744244 2559 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/9f172362-788e-4ce3-a328-f967e623de5f-workload-certs\") pod \"e2e-derive-d19d5af8-openshift-default-84f845d559-lr8v2\" (UID: \"9f172362-788e-4ce3-a328-f967e623de5f\") " pod="openshift-ingress/e2e-derive-d19d5af8-openshift-default-84f845d559-lr8v2" Jul 12 18:30:52.744488 ip-10-0-129-128 kubenswrapper[2559]: I0712 18:30:52.744269 2559 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/9f172362-788e-4ce3-a328-f967e623de5f-istiod-ca-cert\") pod \"e2e-derive-d19d5af8-openshift-default-84f845d559-lr8v2\" (UID: \"9f172362-788e-4ce3-a328-f967e623de5f\") " pod="openshift-ingress/e2e-derive-d19d5af8-openshift-default-84f845d559-lr8v2" Jul 12 18:30:52.744488 ip-10-0-129-128 kubenswrapper[2559]: I0712 18:30:52.744310 2559 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/9f172362-788e-4ce3-a328-f967e623de5f-istio-podinfo\") pod \"e2e-derive-d19d5af8-openshift-default-84f845d559-lr8v2\" (UID: \"9f172362-788e-4ce3-a328-f967e623de5f\") " pod="openshift-ingress/e2e-derive-d19d5af8-openshift-default-84f845d559-lr8v2" Jul 12 18:30:52.744488 ip-10-0-129-128 kubenswrapper[2559]: I0712 18:30:52.744337 2559 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/9f172362-788e-4ce3-a328-f967e623de5f-istio-envoy\") pod \"e2e-derive-d19d5af8-openshift-default-84f845d559-lr8v2\" (UID: \"9f172362-788e-4ce3-a328-f967e623de5f\") " pod="openshift-ingress/e2e-derive-d19d5af8-openshift-default-84f845d559-lr8v2" Jul 12 18:30:52.744488 ip-10-0-129-128 kubenswrapper[2559]: I0712 18:30:52.744360 2559 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/9f172362-788e-4ce3-a328-f967e623de5f-istio-token\") pod \"e2e-derive-d19d5af8-openshift-default-84f845d559-lr8v2\" (UID: \"9f172362-788e-4ce3-a328-f967e623de5f\") " pod="openshift-ingress/e2e-derive-d19d5af8-openshift-default-84f845d559-lr8v2" Jul 12 18:30:52.744488 ip-10-0-129-128 kubenswrapper[2559]: I0712 18:30:52.744387 2559 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/9f172362-788e-4ce3-a328-f967e623de5f-istio-data\") pod \"e2e-derive-d19d5af8-openshift-default-84f845d559-lr8v2\" (UID: \"9f172362-788e-4ce3-a328-f967e623de5f\") " pod="openshift-ingress/e2e-derive-d19d5af8-openshift-default-84f845d559-lr8v2" Jul 12 18:30:52.744488 ip-10-0-129-128 kubenswrapper[2559]: I0712 18:30:52.744447 2559 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-mz2mt\" (UniqueName: \"kubernetes.io/projected/9f172362-788e-4ce3-a328-f967e623de5f-kube-api-access-mz2mt\") pod \"e2e-derive-d19d5af8-openshift-default-84f845d559-lr8v2\" (UID: \"9f172362-788e-4ce3-a328-f967e623de5f\") " pod="openshift-ingress/e2e-derive-d19d5af8-openshift-default-84f845d559-lr8v2" Jul 12 18:30:52.744860 ip-10-0-129-128 kubenswrapper[2559]: I0712 18:30:52.744632 2559 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/9f172362-788e-4ce3-a328-f967e623de5f-workload-socket\") pod \"e2e-derive-d19d5af8-openshift-default-84f845d559-lr8v2\" (UID: \"9f172362-788e-4ce3-a328-f967e623de5f\") " pod="openshift-ingress/e2e-derive-d19d5af8-openshift-default-84f845d559-lr8v2" Jul 12 18:30:52.744860 ip-10-0-129-128 kubenswrapper[2559]: I0712 18:30:52.744702 2559 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/9f172362-788e-4ce3-a328-f967e623de5f-workload-certs\") pod \"e2e-derive-d19d5af8-openshift-default-84f845d559-lr8v2\" (UID: \"9f172362-788e-4ce3-a328-f967e623de5f\") " pod="openshift-ingress/e2e-derive-d19d5af8-openshift-default-84f845d559-lr8v2" Jul 12 18:30:52.744860 ip-10-0-129-128 kubenswrapper[2559]: I0712 18:30:52.744732 2559 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/9f172362-788e-4ce3-a328-f967e623de5f-credential-socket\") pod \"e2e-derive-d19d5af8-openshift-default-84f845d559-lr8v2\" (UID: \"9f172362-788e-4ce3-a328-f967e623de5f\") " pod="openshift-ingress/e2e-derive-d19d5af8-openshift-default-84f845d559-lr8v2" Jul 12 18:30:52.745039 ip-10-0-129-128 kubenswrapper[2559]: I0712 18:30:52.745016 2559 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/9f172362-788e-4ce3-a328-f967e623de5f-istio-data\") pod \"e2e-derive-d19d5af8-openshift-default-84f845d559-lr8v2\" (UID: \"9f172362-788e-4ce3-a328-f967e623de5f\") " pod="openshift-ingress/e2e-derive-d19d5af8-openshift-default-84f845d559-lr8v2" Jul 12 18:30:52.745039 ip-10-0-129-128 kubenswrapper[2559]: I0712 18:30:52.745030 2559 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/9f172362-788e-4ce3-a328-f967e623de5f-istiod-ca-cert\") pod \"e2e-derive-d19d5af8-openshift-default-84f845d559-lr8v2\" (UID: \"9f172362-788e-4ce3-a328-f967e623de5f\") " pod="openshift-ingress/e2e-derive-d19d5af8-openshift-default-84f845d559-lr8v2" Jul 12 18:30:52.746818 ip-10-0-129-128 kubenswrapper[2559]: I0712 18:30:52.746797 2559 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/9f172362-788e-4ce3-a328-f967e623de5f-istio-envoy\") pod \"e2e-derive-d19d5af8-openshift-default-84f845d559-lr8v2\" (UID: \"9f172362-788e-4ce3-a328-f967e623de5f\") " pod="openshift-ingress/e2e-derive-d19d5af8-openshift-default-84f845d559-lr8v2" Jul 12 18:30:52.746818 ip-10-0-129-128 kubenswrapper[2559]: I0712 18:30:52.746808 2559 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/9f172362-788e-4ce3-a328-f967e623de5f-istio-podinfo\") pod \"e2e-derive-d19d5af8-openshift-default-84f845d559-lr8v2\" (UID: \"9f172362-788e-4ce3-a328-f967e623de5f\") " pod="openshift-ingress/e2e-derive-d19d5af8-openshift-default-84f845d559-lr8v2" Jul 12 18:30:52.752288 ip-10-0-129-128 kubenswrapper[2559]: I0712 18:30:52.752261 2559 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/9f172362-788e-4ce3-a328-f967e623de5f-istio-token\") pod \"e2e-derive-d19d5af8-openshift-default-84f845d559-lr8v2\" (UID: \"9f172362-788e-4ce3-a328-f967e623de5f\") " pod="openshift-ingress/e2e-derive-d19d5af8-openshift-default-84f845d559-lr8v2" Jul 12 18:30:52.752455 ip-10-0-129-128 kubenswrapper[2559]: I0712 18:30:52.752437 2559 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-mz2mt\" (UniqueName: \"kubernetes.io/projected/9f172362-788e-4ce3-a328-f967e623de5f-kube-api-access-mz2mt\") pod \"e2e-derive-d19d5af8-openshift-default-84f845d559-lr8v2\" (UID: \"9f172362-788e-4ce3-a328-f967e623de5f\") " pod="openshift-ingress/e2e-derive-d19d5af8-openshift-default-84f845d559-lr8v2" Jul 12 18:30:52.929431 ip-10-0-129-128 kubenswrapper[2559]: I0712 18:30:52.929337 2559 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-derive-d19d5af8-openshift-default-84f845d559-lr8v2" Jul 12 18:30:53.066984 ip-10-0-129-128 kubenswrapper[2559]: I0712 18:30:53.066953 2559 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/e2e-derive-d19d5af8-openshift-default-84f845d559-lr8v2"] Jul 12 18:30:53.069784 ip-10-0-129-128 kubenswrapper[2559]: W0712 18:30:53.069754 2559 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod9f172362_788e_4ce3_a328_f967e623de5f.slice/crio-cadc11b3322946e6ae3080aa3fd0358cf63a8ed5610825f0ed2462435bfe7475 WatchSource:0}: Error finding container cadc11b3322946e6ae3080aa3fd0358cf63a8ed5610825f0ed2462435bfe7475: Status 404 returned error can't find the container with id cadc11b3322946e6ae3080aa3fd0358cf63a8ed5610825f0ed2462435bfe7475 Jul 12 18:30:53.071906 ip-10-0-129-128 kubenswrapper[2559]: I0712 18:30:53.071877 2559 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236228Ki","pods":"250"} Jul 12 18:30:53.071963 ip-10-0-129-128 kubenswrapper[2559]: I0712 18:30:53.071943 2559 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236228Ki","pods":"250"} Jul 12 18:30:53.072004 ip-10-0-129-128 kubenswrapper[2559]: I0712 18:30:53.071968 2559 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236228Ki","pods":"250"} Jul 12 18:30:53.967593 ip-10-0-129-128 kubenswrapper[2559]: I0712 18:30:53.967552 2559 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-derive-d19d5af8-openshift-default-84f845d559-lr8v2" event={"ID":"9f172362-788e-4ce3-a328-f967e623de5f","Type":"ContainerStarted","Data":"3112562a5e25afa41f44ec7a7aebc573d8c871dd12d7cb4dacaf8b2cbbf3c21c"} Jul 12 18:30:53.968006 ip-10-0-129-128 kubenswrapper[2559]: I0712 18:30:53.967653 2559 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-derive-d19d5af8-openshift-default-84f845d559-lr8v2" event={"ID":"9f172362-788e-4ce3-a328-f967e623de5f","Type":"ContainerStarted","Data":"cadc11b3322946e6ae3080aa3fd0358cf63a8ed5610825f0ed2462435bfe7475"} Jul 12 18:30:53.988243 ip-10-0-129-128 kubenswrapper[2559]: I0712 18:30:53.988194 2559 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-ingress/e2e-derive-d19d5af8-openshift-default-84f845d559-lr8v2" podStartSLOduration=1.988177498 podStartE2EDuration="1.988177498s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-12 18:30:52 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-12 18:30:53.98663286 +0000 UTC m=+2261.480700790" watchObservedRunningTime="2026-07-12 18:30:53.988177498 +0000 UTC m=+2261.482245427" Jul 12 18:30:54.929634 ip-10-0-129-128 kubenswrapper[2559]: I0712 18:30:54.929592 2559 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="openshift-ingress/e2e-derive-d19d5af8-openshift-default-84f845d559-lr8v2" Jul 12 18:30:54.934840 ip-10-0-129-128 kubenswrapper[2559]: I0712 18:30:54.934810 2559 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="openshift-ingress/e2e-derive-d19d5af8-openshift-default-84f845d559-lr8v2" Jul 12 18:30:54.973080 ip-10-0-129-128 kubenswrapper[2559]: I0712 18:30:54.973048 2559 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/e2e-derive-d19d5af8-openshift-default-84f845d559-lr8v2" Jul 12 18:30:54.973974 ip-10-0-129-128 kubenswrapper[2559]: I0712 18:30:54.973951 2559 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/e2e-derive-d19d5af8-openshift-default-84f845d559-lr8v2" Jul 12 18:30:57.495026 ip-10-0-129-128 kubenswrapper[2559]: I0712 18:30:57.494992 2559 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-pwsvh"] Jul 12 18:31:06.029652 ip-10-0-129-128 kubenswrapper[2559]: I0712 18:31:06.029606 2559 generic.go:356] "Generic (PLEG): container finished" podID="9f172362-788e-4ce3-a328-f967e623de5f" containerID="3112562a5e25afa41f44ec7a7aebc573d8c871dd12d7cb4dacaf8b2cbbf3c21c" exitCode=0 Jul 12 18:31:06.030060 ip-10-0-129-128 kubenswrapper[2559]: I0712 18:31:06.029683 2559 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-derive-d19d5af8-openshift-default-84f845d559-lr8v2" event={"ID":"9f172362-788e-4ce3-a328-f967e623de5f","Type":"ContainerDied","Data":"3112562a5e25afa41f44ec7a7aebc573d8c871dd12d7cb4dacaf8b2cbbf3c21c"} Jul 12 18:31:06.030147 ip-10-0-129-128 kubenswrapper[2559]: I0712 18:31:06.030063 2559 scope.go:117] "RemoveContainer" containerID="3112562a5e25afa41f44ec7a7aebc573d8c871dd12d7cb4dacaf8b2cbbf3c21c" Jul 12 18:31:06.031153 ip-10-0-129-128 kubenswrapper[2559]: I0712 18:31:06.031127 2559 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236228Ki","pods":"250"} Jul 12 18:31:06.031255 ip-10-0-129-128 kubenswrapper[2559]: I0712 18:31:06.031187 2559 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236228Ki","pods":"250"} Jul 12 18:31:06.031255 ip-10-0-129-128 kubenswrapper[2559]: I0712 18:31:06.031220 2559 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236228Ki","pods":"250"} Jul 12 18:31:07.040561 ip-10-0-129-128 kubenswrapper[2559]: I0712 18:31:07.040513 2559 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-derive-d19d5af8-openshift-default-84f845d559-lr8v2" event={"ID":"9f172362-788e-4ce3-a328-f967e623de5f","Type":"ContainerStarted","Data":"b80051f9b5f7cd579312fb93a6506ff565d646bc9196c5933c55250747875560"} Jul 12 18:31:07.930278 ip-10-0-129-128 kubenswrapper[2559]: I0712 18:31:07.930237 2559 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="openshift-ingress/e2e-derive-d19d5af8-openshift-default-84f845d559-lr8v2" Jul 12 18:31:07.931166 ip-10-0-129-128 kubenswrapper[2559]: I0712 18:31:07.931129 2559 patch_prober.go:28] interesting pod/e2e-derive-d19d5af8-openshift-default-84f845d559-lr8v2 container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.132.0.45:15021/healthz/ready\": dial tcp 10.132.0.45:15021: connect: connection refused" start-of-body= Jul 12 18:31:07.931317 ip-10-0-129-128 kubenswrapper[2559]: I0712 18:31:07.931190 2559 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-derive-d19d5af8-openshift-default-84f845d559-lr8v2" podUID="9f172362-788e-4ce3-a328-f967e623de5f" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.132.0.45:15021/healthz/ready\": dial tcp 10.132.0.45:15021: connect: connection refused" Jul 12 18:31:08.930237 ip-10-0-129-128 kubenswrapper[2559]: I0712 18:31:08.930190 2559 patch_prober.go:28] interesting pod/e2e-derive-d19d5af8-openshift-default-84f845d559-lr8v2 container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.132.0.45:15021/healthz/ready\": dial tcp 10.132.0.45:15021: connect: connection refused" start-of-body= Jul 12 18:31:08.930751 ip-10-0-129-128 kubenswrapper[2559]: I0712 18:31:08.930269 2559 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-derive-d19d5af8-openshift-default-84f845d559-lr8v2" podUID="9f172362-788e-4ce3-a328-f967e623de5f" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.132.0.45:15021/healthz/ready\": dial tcp 10.132.0.45:15021: connect: connection refused" Jul 12 18:31:09.929827 ip-10-0-129-128 kubenswrapper[2559]: I0712 18:31:09.929790 2559 patch_prober.go:28] interesting pod/e2e-derive-d19d5af8-openshift-default-84f845d559-lr8v2 container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.132.0.45:15021/healthz/ready\": dial tcp 10.132.0.45:15021: connect: connection refused" start-of-body= Jul 12 18:31:09.930089 ip-10-0-129-128 kubenswrapper[2559]: I0712 18:31:09.930057 2559 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-derive-d19d5af8-openshift-default-84f845d559-lr8v2" podUID="9f172362-788e-4ce3-a328-f967e623de5f" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.132.0.45:15021/healthz/ready\": dial tcp 10.132.0.45:15021: connect: connection refused" Jul 12 18:31:09.973915 ip-10-0-129-128 kubenswrapper[2559]: I0712 18:31:09.973880 2559 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/e2e-derive-d19d5af8-openshift-default-84f845d559-lr8v2" Jul 12 18:31:10.933687 ip-10-0-129-128 kubenswrapper[2559]: I0712 18:31:10.933631 2559 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="openshift-ingress/e2e-derive-d19d5af8-openshift-default-84f845d559-lr8v2" Jul 12 18:31:10.934775 ip-10-0-129-128 kubenswrapper[2559]: I0712 18:31:10.934750 2559 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/e2e-derive-d19d5af8-openshift-default-84f845d559-lr8v2" Jul 12 18:33:13.123983 ip-10-0-129-128 kubenswrapper[2559]: I0712 18:33:13.123949 2559 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-console-operator_console-operator-7866497494-wlk78_f1d46430-bb60-4470-b8d8-eacd44f4f6b1/console-operator/1.log" Jul 12 18:33:13.126354 ip-10-0-129-128 kubenswrapper[2559]: I0712 18:33:13.126332 2559 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ovn-kubernetes_ovnkube-node-q5dw8_e51bad50-bc74-4e38-8307-ca257b599314/ovn-acl-logging/0.log" Jul 12 18:33:13.127572 ip-10-0-129-128 kubenswrapper[2559]: I0712 18:33:13.127550 2559 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-console-operator_console-operator-7866497494-wlk78_f1d46430-bb60-4470-b8d8-eacd44f4f6b1/console-operator/1.log" Jul 12 18:33:13.130162 ip-10-0-129-128 kubenswrapper[2559]: I0712 18:33:13.130145 2559 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ovn-kubernetes_ovnkube-node-q5dw8_e51bad50-bc74-4e38-8307-ca257b599314/ovn-acl-logging/0.log" Jul 12 18:33:58.303961 ip-10-0-129-128 kubenswrapper[2559]: I0712 18:33:58.303929 2559 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-ingress/e2e-derive-d19d5af8-openshift-default-84f845d559-lr8v2"] Jul 12 18:33:58.304492 ip-10-0-129-128 kubenswrapper[2559]: I0712 18:33:58.304141 2559 kuberuntime_container.go:873] "Killing container with a grace period" pod="openshift-ingress/e2e-derive-d19d5af8-openshift-default-84f845d559-lr8v2" podUID="9f172362-788e-4ce3-a328-f967e623de5f" containerName="istio-proxy" containerID="cri-o://b80051f9b5f7cd579312fb93a6506ff565d646bc9196c5933c55250747875560" gracePeriod=30 Jul 12 18:34:01.389968 ip-10-0-129-128 kubenswrapper[2559]: I0712 18:34:01.389934 2559 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-pwsvh"] Jul 12 18:34:03.565385 ip-10-0-129-128 kubenswrapper[2559]: I0712 18:34:03.565323 2559 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-derive-d19d5af8-openshift-default-84f845d559-lr8v2" Jul 12 18:34:03.611827 ip-10-0-129-128 kubenswrapper[2559]: I0712 18:34:03.611792 2559 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/9f172362-788e-4ce3-a328-f967e623de5f-istiod-ca-cert\") pod \"9f172362-788e-4ce3-a328-f967e623de5f\" (UID: \"9f172362-788e-4ce3-a328-f967e623de5f\") " Jul 12 18:34:03.611827 ip-10-0-129-128 kubenswrapper[2559]: I0712 18:34:03.611826 2559 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/9f172362-788e-4ce3-a328-f967e623de5f-istio-data\") pod \"9f172362-788e-4ce3-a328-f967e623de5f\" (UID: \"9f172362-788e-4ce3-a328-f967e623de5f\") " Jul 12 18:34:03.612074 ip-10-0-129-128 kubenswrapper[2559]: I0712 18:34:03.611871 2559 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/9f172362-788e-4ce3-a328-f967e623de5f-workload-socket\") pod \"9f172362-788e-4ce3-a328-f967e623de5f\" (UID: \"9f172362-788e-4ce3-a328-f967e623de5f\") " Jul 12 18:34:03.612074 ip-10-0-129-128 kubenswrapper[2559]: I0712 18:34:03.611887 2559 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/9f172362-788e-4ce3-a328-f967e623de5f-istio-token\") pod \"9f172362-788e-4ce3-a328-f967e623de5f\" (UID: \"9f172362-788e-4ce3-a328-f967e623de5f\") " Jul 12 18:34:03.612074 ip-10-0-129-128 kubenswrapper[2559]: I0712 18:34:03.611932 2559 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-mz2mt\" (UniqueName: \"kubernetes.io/projected/9f172362-788e-4ce3-a328-f967e623de5f-kube-api-access-mz2mt\") pod \"9f172362-788e-4ce3-a328-f967e623de5f\" (UID: \"9f172362-788e-4ce3-a328-f967e623de5f\") " Jul 12 18:34:03.612074 ip-10-0-129-128 kubenswrapper[2559]: I0712 18:34:03.611955 2559 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/9f172362-788e-4ce3-a328-f967e623de5f-istio-podinfo\") pod \"9f172362-788e-4ce3-a328-f967e623de5f\" (UID: \"9f172362-788e-4ce3-a328-f967e623de5f\") " Jul 12 18:34:03.612074 ip-10-0-129-128 kubenswrapper[2559]: I0712 18:34:03.611980 2559 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/9f172362-788e-4ce3-a328-f967e623de5f-workload-certs\") pod \"9f172362-788e-4ce3-a328-f967e623de5f\" (UID: \"9f172362-788e-4ce3-a328-f967e623de5f\") " Jul 12 18:34:03.612074 ip-10-0-129-128 kubenswrapper[2559]: I0712 18:34:03.612062 2559 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/9f172362-788e-4ce3-a328-f967e623de5f-credential-socket\") pod \"9f172362-788e-4ce3-a328-f967e623de5f\" (UID: \"9f172362-788e-4ce3-a328-f967e623de5f\") " Jul 12 18:34:03.612410 ip-10-0-129-128 kubenswrapper[2559]: I0712 18:34:03.612091 2559 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/9f172362-788e-4ce3-a328-f967e623de5f-istio-envoy\") pod \"9f172362-788e-4ce3-a328-f967e623de5f\" (UID: \"9f172362-788e-4ce3-a328-f967e623de5f\") " Jul 12 18:34:03.612410 ip-10-0-129-128 kubenswrapper[2559]: I0712 18:34:03.612150 2559 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/9f172362-788e-4ce3-a328-f967e623de5f-istiod-ca-cert" (OuterVolumeSpecName: "istiod-ca-cert") pod "9f172362-788e-4ce3-a328-f967e623de5f" (UID: "9f172362-788e-4ce3-a328-f967e623de5f"). InnerVolumeSpecName "istiod-ca-cert". PluginName "kubernetes.io/configmap", VolumeGIDValue "" Jul 12 18:34:03.612410 ip-10-0-129-128 kubenswrapper[2559]: I0712 18:34:03.612238 2559 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/9f172362-788e-4ce3-a328-f967e623de5f-workload-socket" (OuterVolumeSpecName: "workload-socket") pod "9f172362-788e-4ce3-a328-f967e623de5f" (UID: "9f172362-788e-4ce3-a328-f967e623de5f"). InnerVolumeSpecName "workload-socket". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Jul 12 18:34:03.612574 ip-10-0-129-128 kubenswrapper[2559]: I0712 18:34:03.612406 2559 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/9f172362-788e-4ce3-a328-f967e623de5f-workload-certs" (OuterVolumeSpecName: "workload-certs") pod "9f172362-788e-4ce3-a328-f967e623de5f" (UID: "9f172362-788e-4ce3-a328-f967e623de5f"). InnerVolumeSpecName "workload-certs". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Jul 12 18:34:03.612574 ip-10-0-129-128 kubenswrapper[2559]: I0712 18:34:03.612426 2559 reconciler_common.go:299] "Volume detached for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/9f172362-788e-4ce3-a328-f967e623de5f-istiod-ca-cert\") on node \"ip-10-0-129-128.ec2.internal\" DevicePath \"\"" Jul 12 18:34:03.612574 ip-10-0-129-128 kubenswrapper[2559]: I0712 18:34:03.612447 2559 reconciler_common.go:299] "Volume detached for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/9f172362-788e-4ce3-a328-f967e623de5f-workload-socket\") on node \"ip-10-0-129-128.ec2.internal\" DevicePath \"\"" Jul 12 18:34:03.612807 ip-10-0-129-128 kubenswrapper[2559]: I0712 18:34:03.612671 2559 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/9f172362-788e-4ce3-a328-f967e623de5f-credential-socket" (OuterVolumeSpecName: "credential-socket") pod "9f172362-788e-4ce3-a328-f967e623de5f" (UID: "9f172362-788e-4ce3-a328-f967e623de5f"). InnerVolumeSpecName "credential-socket". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Jul 12 18:34:03.612919 ip-10-0-129-128 kubenswrapper[2559]: I0712 18:34:03.612893 2559 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/9f172362-788e-4ce3-a328-f967e623de5f-istio-data" (OuterVolumeSpecName: "istio-data") pod "9f172362-788e-4ce3-a328-f967e623de5f" (UID: "9f172362-788e-4ce3-a328-f967e623de5f"). InnerVolumeSpecName "istio-data". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Jul 12 18:34:03.614354 ip-10-0-129-128 kubenswrapper[2559]: I0712 18:34:03.614315 2559 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/9f172362-788e-4ce3-a328-f967e623de5f-istio-envoy" (OuterVolumeSpecName: "istio-envoy") pod "9f172362-788e-4ce3-a328-f967e623de5f" (UID: "9f172362-788e-4ce3-a328-f967e623de5f"). InnerVolumeSpecName "istio-envoy". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Jul 12 18:34:03.614616 ip-10-0-129-128 kubenswrapper[2559]: I0712 18:34:03.614596 2559 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/9f172362-788e-4ce3-a328-f967e623de5f-istio-token" (OuterVolumeSpecName: "istio-token") pod "9f172362-788e-4ce3-a328-f967e623de5f" (UID: "9f172362-788e-4ce3-a328-f967e623de5f"). InnerVolumeSpecName "istio-token". PluginName "kubernetes.io/projected", VolumeGIDValue "" Jul 12 18:34:03.614800 ip-10-0-129-128 kubenswrapper[2559]: I0712 18:34:03.614781 2559 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/downward-api/9f172362-788e-4ce3-a328-f967e623de5f-istio-podinfo" (OuterVolumeSpecName: "istio-podinfo") pod "9f172362-788e-4ce3-a328-f967e623de5f" (UID: "9f172362-788e-4ce3-a328-f967e623de5f"). InnerVolumeSpecName "istio-podinfo". PluginName "kubernetes.io/downward-api", VolumeGIDValue "" Jul 12 18:34:03.615064 ip-10-0-129-128 kubenswrapper[2559]: I0712 18:34:03.615043 2559 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/9f172362-788e-4ce3-a328-f967e623de5f-kube-api-access-mz2mt" (OuterVolumeSpecName: "kube-api-access-mz2mt") pod "9f172362-788e-4ce3-a328-f967e623de5f" (UID: "9f172362-788e-4ce3-a328-f967e623de5f"). InnerVolumeSpecName "kube-api-access-mz2mt". PluginName "kubernetes.io/projected", VolumeGIDValue "" Jul 12 18:34:03.712992 ip-10-0-129-128 kubenswrapper[2559]: I0712 18:34:03.712960 2559 reconciler_common.go:299] "Volume detached for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/9f172362-788e-4ce3-a328-f967e623de5f-credential-socket\") on node \"ip-10-0-129-128.ec2.internal\" DevicePath \"\"" Jul 12 18:34:03.712992 ip-10-0-129-128 kubenswrapper[2559]: I0712 18:34:03.712987 2559 reconciler_common.go:299] "Volume detached for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/9f172362-788e-4ce3-a328-f967e623de5f-istio-envoy\") on node \"ip-10-0-129-128.ec2.internal\" DevicePath \"\"" Jul 12 18:34:03.712992 ip-10-0-129-128 kubenswrapper[2559]: I0712 18:34:03.712996 2559 reconciler_common.go:299] "Volume detached for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/9f172362-788e-4ce3-a328-f967e623de5f-istio-data\") on node \"ip-10-0-129-128.ec2.internal\" DevicePath \"\"" Jul 12 18:34:03.713210 ip-10-0-129-128 kubenswrapper[2559]: I0712 18:34:03.713006 2559 reconciler_common.go:299] "Volume detached for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/9f172362-788e-4ce3-a328-f967e623de5f-istio-token\") on node \"ip-10-0-129-128.ec2.internal\" DevicePath \"\"" Jul 12 18:34:03.713210 ip-10-0-129-128 kubenswrapper[2559]: I0712 18:34:03.713013 2559 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-mz2mt\" (UniqueName: \"kubernetes.io/projected/9f172362-788e-4ce3-a328-f967e623de5f-kube-api-access-mz2mt\") on node \"ip-10-0-129-128.ec2.internal\" DevicePath \"\"" Jul 12 18:34:03.713210 ip-10-0-129-128 kubenswrapper[2559]: I0712 18:34:03.713022 2559 reconciler_common.go:299] "Volume detached for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/9f172362-788e-4ce3-a328-f967e623de5f-istio-podinfo\") on node \"ip-10-0-129-128.ec2.internal\" DevicePath \"\"" Jul 12 18:34:03.713210 ip-10-0-129-128 kubenswrapper[2559]: I0712 18:34:03.713030 2559 reconciler_common.go:299] "Volume detached for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/9f172362-788e-4ce3-a328-f967e623de5f-workload-certs\") on node \"ip-10-0-129-128.ec2.internal\" DevicePath \"\"" Jul 12 18:34:03.890245 ip-10-0-129-128 kubenswrapper[2559]: I0712 18:34:03.890155 2559 generic.go:356] "Generic (PLEG): container finished" podID="9f172362-788e-4ce3-a328-f967e623de5f" containerID="b80051f9b5f7cd579312fb93a6506ff565d646bc9196c5933c55250747875560" exitCode=0 Jul 12 18:34:03.890245 ip-10-0-129-128 kubenswrapper[2559]: I0712 18:34:03.890203 2559 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-derive-d19d5af8-openshift-default-84f845d559-lr8v2" Jul 12 18:34:03.890431 ip-10-0-129-128 kubenswrapper[2559]: I0712 18:34:03.890238 2559 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-derive-d19d5af8-openshift-default-84f845d559-lr8v2" event={"ID":"9f172362-788e-4ce3-a328-f967e623de5f","Type":"ContainerDied","Data":"b80051f9b5f7cd579312fb93a6506ff565d646bc9196c5933c55250747875560"} Jul 12 18:34:03.890431 ip-10-0-129-128 kubenswrapper[2559]: I0712 18:34:03.890306 2559 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-derive-d19d5af8-openshift-default-84f845d559-lr8v2" event={"ID":"9f172362-788e-4ce3-a328-f967e623de5f","Type":"ContainerDied","Data":"cadc11b3322946e6ae3080aa3fd0358cf63a8ed5610825f0ed2462435bfe7475"} Jul 12 18:34:03.890431 ip-10-0-129-128 kubenswrapper[2559]: I0712 18:34:03.890331 2559 scope.go:117] "RemoveContainer" containerID="b80051f9b5f7cd579312fb93a6506ff565d646bc9196c5933c55250747875560" Jul 12 18:34:03.901358 ip-10-0-129-128 kubenswrapper[2559]: I0712 18:34:03.901341 2559 scope.go:117] "RemoveContainer" containerID="3112562a5e25afa41f44ec7a7aebc573d8c871dd12d7cb4dacaf8b2cbbf3c21c" Jul 12 18:34:03.909395 ip-10-0-129-128 kubenswrapper[2559]: I0712 18:34:03.909380 2559 scope.go:117] "RemoveContainer" containerID="b80051f9b5f7cd579312fb93a6506ff565d646bc9196c5933c55250747875560" Jul 12 18:34:03.909666 ip-10-0-129-128 kubenswrapper[2559]: E0712 18:34:03.909630 2559 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"b80051f9b5f7cd579312fb93a6506ff565d646bc9196c5933c55250747875560\": container with ID starting with b80051f9b5f7cd579312fb93a6506ff565d646bc9196c5933c55250747875560 not found: ID does not exist" containerID="b80051f9b5f7cd579312fb93a6506ff565d646bc9196c5933c55250747875560" Jul 12 18:34:03.909723 ip-10-0-129-128 kubenswrapper[2559]: I0712 18:34:03.909678 2559 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"b80051f9b5f7cd579312fb93a6506ff565d646bc9196c5933c55250747875560"} err="failed to get container status \"b80051f9b5f7cd579312fb93a6506ff565d646bc9196c5933c55250747875560\": rpc error: code = NotFound desc = could not find container \"b80051f9b5f7cd579312fb93a6506ff565d646bc9196c5933c55250747875560\": container with ID starting with b80051f9b5f7cd579312fb93a6506ff565d646bc9196c5933c55250747875560 not found: ID does not exist" Jul 12 18:34:03.909723 ip-10-0-129-128 kubenswrapper[2559]: I0712 18:34:03.909701 2559 scope.go:117] "RemoveContainer" containerID="3112562a5e25afa41f44ec7a7aebc573d8c871dd12d7cb4dacaf8b2cbbf3c21c" Jul 12 18:34:03.909941 ip-10-0-129-128 kubenswrapper[2559]: E0712 18:34:03.909921 2559 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"3112562a5e25afa41f44ec7a7aebc573d8c871dd12d7cb4dacaf8b2cbbf3c21c\": container with ID starting with 3112562a5e25afa41f44ec7a7aebc573d8c871dd12d7cb4dacaf8b2cbbf3c21c not found: ID does not exist" containerID="3112562a5e25afa41f44ec7a7aebc573d8c871dd12d7cb4dacaf8b2cbbf3c21c" Jul 12 18:34:03.909996 ip-10-0-129-128 kubenswrapper[2559]: I0712 18:34:03.909949 2559 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"3112562a5e25afa41f44ec7a7aebc573d8c871dd12d7cb4dacaf8b2cbbf3c21c"} err="failed to get container status \"3112562a5e25afa41f44ec7a7aebc573d8c871dd12d7cb4dacaf8b2cbbf3c21c\": rpc error: code = NotFound desc = could not find container \"3112562a5e25afa41f44ec7a7aebc573d8c871dd12d7cb4dacaf8b2cbbf3c21c\": container with ID starting with 3112562a5e25afa41f44ec7a7aebc573d8c871dd12d7cb4dacaf8b2cbbf3c21c not found: ID does not exist" Jul 12 18:34:03.914457 ip-10-0-129-128 kubenswrapper[2559]: I0712 18:34:03.914432 2559 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-ingress/e2e-derive-d19d5af8-openshift-default-84f845d559-lr8v2"] Jul 12 18:34:03.916339 ip-10-0-129-128 kubenswrapper[2559]: I0712 18:34:03.916320 2559 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["openshift-ingress/e2e-derive-d19d5af8-openshift-default-84f845d559-lr8v2"] Jul 12 18:34:05.038177 ip-10-0-129-128 kubenswrapper[2559]: I0712 18:34:05.038136 2559 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="9f172362-788e-4ce3-a328-f967e623de5f" path="/var/lib/kubelet/pods/9f172362-788e-4ce3-a328-f967e623de5f/volumes" Jul 12 18:35:33.589438 ip-10-0-129-128 kubenswrapper[2559]: I0712 18:35:33.589349 2559 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-pwsvh"] Jul 12 18:35:43.230867 ip-10-0-129-128 kubenswrapper[2559]: I0712 18:35:43.230824 2559 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-pwsvh"] Jul 12 18:35:44.803499 ip-10-0-129-128 kubenswrapper[2559]: I0712 18:35:44.803467 2559 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-ingress/e2e-mt-70a9c475-openshift-default-958b5846c-cdq9h"] Jul 12 18:35:44.803901 ip-10-0-129-128 kubenswrapper[2559]: I0712 18:35:44.803887 2559 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="9f172362-788e-4ce3-a328-f967e623de5f" containerName="istio-proxy" Jul 12 18:35:44.803960 ip-10-0-129-128 kubenswrapper[2559]: I0712 18:35:44.803903 2559 state_mem.go:107] "Deleted CPUSet assignment" podUID="9f172362-788e-4ce3-a328-f967e623de5f" containerName="istio-proxy" Jul 12 18:35:44.803999 ip-10-0-129-128 kubenswrapper[2559]: I0712 18:35:44.803983 2559 memory_manager.go:368] "RemoveStaleState removing state" podUID="9f172362-788e-4ce3-a328-f967e623de5f" containerName="istio-proxy" Jul 12 18:35:44.807035 ip-10-0-129-128 kubenswrapper[2559]: I0712 18:35:44.807018 2559 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-mt-70a9c475-openshift-default-958b5846c-cdq9h" Jul 12 18:35:44.809734 ip-10-0-129-128 kubenswrapper[2559]: I0712 18:35:44.809712 2559 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress\"/\"e2e-mt-70a9c475-openshift-default-dockercfg-qj7lk\"" Jul 12 18:35:44.831222 ip-10-0-129-128 kubenswrapper[2559]: I0712 18:35:44.831190 2559 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/e2e-mt-70a9c475-openshift-default-958b5846c-cdq9h"] Jul 12 18:35:44.893808 ip-10-0-129-128 kubenswrapper[2559]: I0712 18:35:44.893758 2559 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/0efa7dab-49ea-4a0e-bf42-95867a0c144f-credential-socket\") pod \"e2e-mt-70a9c475-openshift-default-958b5846c-cdq9h\" (UID: \"0efa7dab-49ea-4a0e-bf42-95867a0c144f\") " pod="openshift-ingress/e2e-mt-70a9c475-openshift-default-958b5846c-cdq9h" Jul 12 18:35:44.893987 ip-10-0-129-128 kubenswrapper[2559]: I0712 18:35:44.893827 2559 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/0efa7dab-49ea-4a0e-bf42-95867a0c144f-istio-envoy\") pod \"e2e-mt-70a9c475-openshift-default-958b5846c-cdq9h\" (UID: \"0efa7dab-49ea-4a0e-bf42-95867a0c144f\") " pod="openshift-ingress/e2e-mt-70a9c475-openshift-default-958b5846c-cdq9h" Jul 12 18:35:44.893987 ip-10-0-129-128 kubenswrapper[2559]: I0712 18:35:44.893882 2559 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/0efa7dab-49ea-4a0e-bf42-95867a0c144f-istiod-ca-cert\") pod \"e2e-mt-70a9c475-openshift-default-958b5846c-cdq9h\" (UID: \"0efa7dab-49ea-4a0e-bf42-95867a0c144f\") " pod="openshift-ingress/e2e-mt-70a9c475-openshift-default-958b5846c-cdq9h" Jul 12 18:35:44.893987 ip-10-0-129-128 kubenswrapper[2559]: I0712 18:35:44.893923 2559 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/0efa7dab-49ea-4a0e-bf42-95867a0c144f-istio-podinfo\") pod \"e2e-mt-70a9c475-openshift-default-958b5846c-cdq9h\" (UID: \"0efa7dab-49ea-4a0e-bf42-95867a0c144f\") " pod="openshift-ingress/e2e-mt-70a9c475-openshift-default-958b5846c-cdq9h" Jul 12 18:35:44.893987 ip-10-0-129-128 kubenswrapper[2559]: I0712 18:35:44.893956 2559 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/0efa7dab-49ea-4a0e-bf42-95867a0c144f-istio-data\") pod \"e2e-mt-70a9c475-openshift-default-958b5846c-cdq9h\" (UID: \"0efa7dab-49ea-4a0e-bf42-95867a0c144f\") " pod="openshift-ingress/e2e-mt-70a9c475-openshift-default-958b5846c-cdq9h" Jul 12 18:35:44.893987 ip-10-0-129-128 kubenswrapper[2559]: I0712 18:35:44.893982 2559 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/0efa7dab-49ea-4a0e-bf42-95867a0c144f-workload-socket\") pod \"e2e-mt-70a9c475-openshift-default-958b5846c-cdq9h\" (UID: \"0efa7dab-49ea-4a0e-bf42-95867a0c144f\") " pod="openshift-ingress/e2e-mt-70a9c475-openshift-default-958b5846c-cdq9h" Jul 12 18:35:44.894226 ip-10-0-129-128 kubenswrapper[2559]: I0712 18:35:44.894035 2559 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/0efa7dab-49ea-4a0e-bf42-95867a0c144f-istio-token\") pod \"e2e-mt-70a9c475-openshift-default-958b5846c-cdq9h\" (UID: \"0efa7dab-49ea-4a0e-bf42-95867a0c144f\") " pod="openshift-ingress/e2e-mt-70a9c475-openshift-default-958b5846c-cdq9h" Jul 12 18:35:44.894226 ip-10-0-129-128 kubenswrapper[2559]: I0712 18:35:44.894102 2559 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-k9jf6\" (UniqueName: \"kubernetes.io/projected/0efa7dab-49ea-4a0e-bf42-95867a0c144f-kube-api-access-k9jf6\") pod \"e2e-mt-70a9c475-openshift-default-958b5846c-cdq9h\" (UID: \"0efa7dab-49ea-4a0e-bf42-95867a0c144f\") " pod="openshift-ingress/e2e-mt-70a9c475-openshift-default-958b5846c-cdq9h" Jul 12 18:35:44.894226 ip-10-0-129-128 kubenswrapper[2559]: I0712 18:35:44.894157 2559 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/0efa7dab-49ea-4a0e-bf42-95867a0c144f-workload-certs\") pod \"e2e-mt-70a9c475-openshift-default-958b5846c-cdq9h\" (UID: \"0efa7dab-49ea-4a0e-bf42-95867a0c144f\") " pod="openshift-ingress/e2e-mt-70a9c475-openshift-default-958b5846c-cdq9h" Jul 12 18:35:44.995335 ip-10-0-129-128 kubenswrapper[2559]: I0712 18:35:44.995291 2559 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/0efa7dab-49ea-4a0e-bf42-95867a0c144f-istiod-ca-cert\") pod \"e2e-mt-70a9c475-openshift-default-958b5846c-cdq9h\" (UID: \"0efa7dab-49ea-4a0e-bf42-95867a0c144f\") " pod="openshift-ingress/e2e-mt-70a9c475-openshift-default-958b5846c-cdq9h" Jul 12 18:35:44.995545 ip-10-0-129-128 kubenswrapper[2559]: I0712 18:35:44.995351 2559 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/0efa7dab-49ea-4a0e-bf42-95867a0c144f-istio-podinfo\") pod \"e2e-mt-70a9c475-openshift-default-958b5846c-cdq9h\" (UID: \"0efa7dab-49ea-4a0e-bf42-95867a0c144f\") " pod="openshift-ingress/e2e-mt-70a9c475-openshift-default-958b5846c-cdq9h" Jul 12 18:35:44.995545 ip-10-0-129-128 kubenswrapper[2559]: I0712 18:35:44.995381 2559 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/0efa7dab-49ea-4a0e-bf42-95867a0c144f-istio-data\") pod \"e2e-mt-70a9c475-openshift-default-958b5846c-cdq9h\" (UID: \"0efa7dab-49ea-4a0e-bf42-95867a0c144f\") " pod="openshift-ingress/e2e-mt-70a9c475-openshift-default-958b5846c-cdq9h" Jul 12 18:35:44.995545 ip-10-0-129-128 kubenswrapper[2559]: I0712 18:35:44.995406 2559 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/0efa7dab-49ea-4a0e-bf42-95867a0c144f-workload-socket\") pod \"e2e-mt-70a9c475-openshift-default-958b5846c-cdq9h\" (UID: \"0efa7dab-49ea-4a0e-bf42-95867a0c144f\") " pod="openshift-ingress/e2e-mt-70a9c475-openshift-default-958b5846c-cdq9h" Jul 12 18:35:44.995545 ip-10-0-129-128 kubenswrapper[2559]: I0712 18:35:44.995455 2559 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/0efa7dab-49ea-4a0e-bf42-95867a0c144f-istio-token\") pod \"e2e-mt-70a9c475-openshift-default-958b5846c-cdq9h\" (UID: \"0efa7dab-49ea-4a0e-bf42-95867a0c144f\") " pod="openshift-ingress/e2e-mt-70a9c475-openshift-default-958b5846c-cdq9h" Jul 12 18:35:44.995545 ip-10-0-129-128 kubenswrapper[2559]: I0712 18:35:44.995476 2559 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-k9jf6\" (UniqueName: \"kubernetes.io/projected/0efa7dab-49ea-4a0e-bf42-95867a0c144f-kube-api-access-k9jf6\") pod \"e2e-mt-70a9c475-openshift-default-958b5846c-cdq9h\" (UID: \"0efa7dab-49ea-4a0e-bf42-95867a0c144f\") " pod="openshift-ingress/e2e-mt-70a9c475-openshift-default-958b5846c-cdq9h" Jul 12 18:35:44.995545 ip-10-0-129-128 kubenswrapper[2559]: I0712 18:35:44.995503 2559 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/0efa7dab-49ea-4a0e-bf42-95867a0c144f-workload-certs\") pod \"e2e-mt-70a9c475-openshift-default-958b5846c-cdq9h\" (UID: \"0efa7dab-49ea-4a0e-bf42-95867a0c144f\") " pod="openshift-ingress/e2e-mt-70a9c475-openshift-default-958b5846c-cdq9h" Jul 12 18:35:44.995545 ip-10-0-129-128 kubenswrapper[2559]: I0712 18:35:44.995532 2559 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/0efa7dab-49ea-4a0e-bf42-95867a0c144f-credential-socket\") pod \"e2e-mt-70a9c475-openshift-default-958b5846c-cdq9h\" (UID: \"0efa7dab-49ea-4a0e-bf42-95867a0c144f\") " pod="openshift-ingress/e2e-mt-70a9c475-openshift-default-958b5846c-cdq9h" Jul 12 18:35:44.995972 ip-10-0-129-128 kubenswrapper[2559]: I0712 18:35:44.995571 2559 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/0efa7dab-49ea-4a0e-bf42-95867a0c144f-istio-envoy\") pod \"e2e-mt-70a9c475-openshift-default-958b5846c-cdq9h\" (UID: \"0efa7dab-49ea-4a0e-bf42-95867a0c144f\") " pod="openshift-ingress/e2e-mt-70a9c475-openshift-default-958b5846c-cdq9h" Jul 12 18:35:44.995972 ip-10-0-129-128 kubenswrapper[2559]: I0712 18:35:44.995889 2559 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/0efa7dab-49ea-4a0e-bf42-95867a0c144f-istio-data\") pod \"e2e-mt-70a9c475-openshift-default-958b5846c-cdq9h\" (UID: \"0efa7dab-49ea-4a0e-bf42-95867a0c144f\") " pod="openshift-ingress/e2e-mt-70a9c475-openshift-default-958b5846c-cdq9h" Jul 12 18:35:44.995972 ip-10-0-129-128 kubenswrapper[2559]: I0712 18:35:44.995951 2559 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/0efa7dab-49ea-4a0e-bf42-95867a0c144f-workload-socket\") pod \"e2e-mt-70a9c475-openshift-default-958b5846c-cdq9h\" (UID: \"0efa7dab-49ea-4a0e-bf42-95867a0c144f\") " pod="openshift-ingress/e2e-mt-70a9c475-openshift-default-958b5846c-cdq9h" Jul 12 18:35:44.996136 ip-10-0-129-128 kubenswrapper[2559]: I0712 18:35:44.996092 2559 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/0efa7dab-49ea-4a0e-bf42-95867a0c144f-credential-socket\") pod \"e2e-mt-70a9c475-openshift-default-958b5846c-cdq9h\" (UID: \"0efa7dab-49ea-4a0e-bf42-95867a0c144f\") " pod="openshift-ingress/e2e-mt-70a9c475-openshift-default-958b5846c-cdq9h" Jul 12 18:35:44.996194 ip-10-0-129-128 kubenswrapper[2559]: I0712 18:35:44.996173 2559 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/0efa7dab-49ea-4a0e-bf42-95867a0c144f-istiod-ca-cert\") pod \"e2e-mt-70a9c475-openshift-default-958b5846c-cdq9h\" (UID: \"0efa7dab-49ea-4a0e-bf42-95867a0c144f\") " pod="openshift-ingress/e2e-mt-70a9c475-openshift-default-958b5846c-cdq9h" Jul 12 18:35:44.996272 ip-10-0-129-128 kubenswrapper[2559]: I0712 18:35:44.996250 2559 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/0efa7dab-49ea-4a0e-bf42-95867a0c144f-workload-certs\") pod \"e2e-mt-70a9c475-openshift-default-958b5846c-cdq9h\" (UID: \"0efa7dab-49ea-4a0e-bf42-95867a0c144f\") " pod="openshift-ingress/e2e-mt-70a9c475-openshift-default-958b5846c-cdq9h" Jul 12 18:35:44.997958 ip-10-0-129-128 kubenswrapper[2559]: I0712 18:35:44.997934 2559 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/0efa7dab-49ea-4a0e-bf42-95867a0c144f-istio-podinfo\") pod \"e2e-mt-70a9c475-openshift-default-958b5846c-cdq9h\" (UID: \"0efa7dab-49ea-4a0e-bf42-95867a0c144f\") " pod="openshift-ingress/e2e-mt-70a9c475-openshift-default-958b5846c-cdq9h" Jul 12 18:35:44.998051 ip-10-0-129-128 kubenswrapper[2559]: I0712 18:35:44.997968 2559 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/0efa7dab-49ea-4a0e-bf42-95867a0c144f-istio-envoy\") pod \"e2e-mt-70a9c475-openshift-default-958b5846c-cdq9h\" (UID: \"0efa7dab-49ea-4a0e-bf42-95867a0c144f\") " pod="openshift-ingress/e2e-mt-70a9c475-openshift-default-958b5846c-cdq9h" Jul 12 18:35:45.003746 ip-10-0-129-128 kubenswrapper[2559]: I0712 18:35:45.003726 2559 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-k9jf6\" (UniqueName: \"kubernetes.io/projected/0efa7dab-49ea-4a0e-bf42-95867a0c144f-kube-api-access-k9jf6\") pod \"e2e-mt-70a9c475-openshift-default-958b5846c-cdq9h\" (UID: \"0efa7dab-49ea-4a0e-bf42-95867a0c144f\") " pod="openshift-ingress/e2e-mt-70a9c475-openshift-default-958b5846c-cdq9h" Jul 12 18:35:45.003868 ip-10-0-129-128 kubenswrapper[2559]: I0712 18:35:45.003803 2559 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/0efa7dab-49ea-4a0e-bf42-95867a0c144f-istio-token\") pod \"e2e-mt-70a9c475-openshift-default-958b5846c-cdq9h\" (UID: \"0efa7dab-49ea-4a0e-bf42-95867a0c144f\") " pod="openshift-ingress/e2e-mt-70a9c475-openshift-default-958b5846c-cdq9h" Jul 12 18:35:45.121147 ip-10-0-129-128 kubenswrapper[2559]: I0712 18:35:45.121013 2559 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-mt-70a9c475-openshift-default-958b5846c-cdq9h" Jul 12 18:35:45.268692 ip-10-0-129-128 kubenswrapper[2559]: I0712 18:35:45.268665 2559 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/e2e-mt-70a9c475-openshift-default-958b5846c-cdq9h"] Jul 12 18:35:45.270691 ip-10-0-129-128 kubenswrapper[2559]: W0712 18:35:45.270661 2559 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod0efa7dab_49ea_4a0e_bf42_95867a0c144f.slice/crio-c4a0941da052a4c484b614bcaf75f37d09d186a63c2abe870d5655ff25b304a7 WatchSource:0}: Error finding container c4a0941da052a4c484b614bcaf75f37d09d186a63c2abe870d5655ff25b304a7: Status 404 returned error can't find the container with id c4a0941da052a4c484b614bcaf75f37d09d186a63c2abe870d5655ff25b304a7 Jul 12 18:35:45.272861 ip-10-0-129-128 kubenswrapper[2559]: I0712 18:35:45.272830 2559 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236228Ki","pods":"250"} Jul 12 18:35:45.272963 ip-10-0-129-128 kubenswrapper[2559]: I0712 18:35:45.272896 2559 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236228Ki","pods":"250"} Jul 12 18:35:45.272963 ip-10-0-129-128 kubenswrapper[2559]: I0712 18:35:45.272931 2559 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236228Ki","pods":"250"} Jul 12 18:35:45.392446 ip-10-0-129-128 kubenswrapper[2559]: I0712 18:35:45.392346 2559 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-mt-70a9c475-openshift-default-958b5846c-cdq9h" event={"ID":"0efa7dab-49ea-4a0e-bf42-95867a0c144f","Type":"ContainerStarted","Data":"e4e47e4380c24391b363d57c1dab7ec4e41f665b77b21fd5c16b548815daf69f"} Jul 12 18:35:45.392446 ip-10-0-129-128 kubenswrapper[2559]: I0712 18:35:45.392427 2559 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-mt-70a9c475-openshift-default-958b5846c-cdq9h" event={"ID":"0efa7dab-49ea-4a0e-bf42-95867a0c144f","Type":"ContainerStarted","Data":"c4a0941da052a4c484b614bcaf75f37d09d186a63c2abe870d5655ff25b304a7"} Jul 12 18:35:45.412733 ip-10-0-129-128 kubenswrapper[2559]: I0712 18:35:45.412677 2559 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-ingress/e2e-mt-70a9c475-openshift-default-958b5846c-cdq9h" podStartSLOduration=1.4126616410000001 podStartE2EDuration="1.412661641s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-12 18:35:44 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-12 18:35:45.410492981 +0000 UTC m=+2552.904560911" watchObservedRunningTime="2026-07-12 18:35:45.412661641 +0000 UTC m=+2552.906729562" Jul 12 18:35:46.121754 ip-10-0-129-128 kubenswrapper[2559]: I0712 18:35:46.121628 2559 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="openshift-ingress/e2e-mt-70a9c475-openshift-default-958b5846c-cdq9h" Jul 12 18:35:46.126792 ip-10-0-129-128 kubenswrapper[2559]: I0712 18:35:46.126730 2559 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="openshift-ingress/e2e-mt-70a9c475-openshift-default-958b5846c-cdq9h" Jul 12 18:35:46.397589 ip-10-0-129-128 kubenswrapper[2559]: I0712 18:35:46.397495 2559 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/e2e-mt-70a9c475-openshift-default-958b5846c-cdq9h" Jul 12 18:35:46.398491 ip-10-0-129-128 kubenswrapper[2559]: I0712 18:35:46.398465 2559 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/e2e-mt-70a9c475-openshift-default-958b5846c-cdq9h" Jul 12 18:35:54.199539 ip-10-0-129-128 kubenswrapper[2559]: I0712 18:35:54.199504 2559 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-pwsvh"] Jul 12 18:36:02.114209 ip-10-0-129-128 kubenswrapper[2559]: I0712 18:36:02.113512 2559 patch_prober.go:28] interesting pod/e2e-mt-70a9c475-openshift-default-958b5846c-cdq9h container/istio-proxy namespace/openshift-ingress: Readiness probe status=failure output="Get \"http://10.132.0.46:15021/healthz/ready\": read tcp 10.132.0.2:58028->10.132.0.46:15021: read: connection reset by peer" start-of-body= Jul 12 18:36:02.114209 ip-10-0-129-128 kubenswrapper[2559]: I0712 18:36:02.113604 2559 prober.go:120] "Probe failed" probeType="Readiness" pod="openshift-ingress/e2e-mt-70a9c475-openshift-default-958b5846c-cdq9h" podUID="0efa7dab-49ea-4a0e-bf42-95867a0c144f" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.132.0.46:15021/healthz/ready\": read tcp 10.132.0.2:58028->10.132.0.46:15021: read: connection reset by peer" Jul 12 18:36:02.487041 ip-10-0-129-128 kubenswrapper[2559]: I0712 18:36:02.486939 2559 generic.go:356] "Generic (PLEG): container finished" podID="0efa7dab-49ea-4a0e-bf42-95867a0c144f" containerID="e4e47e4380c24391b363d57c1dab7ec4e41f665b77b21fd5c16b548815daf69f" exitCode=0 Jul 12 18:36:02.487041 ip-10-0-129-128 kubenswrapper[2559]: I0712 18:36:02.486981 2559 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-mt-70a9c475-openshift-default-958b5846c-cdq9h" event={"ID":"0efa7dab-49ea-4a0e-bf42-95867a0c144f","Type":"ContainerDied","Data":"e4e47e4380c24391b363d57c1dab7ec4e41f665b77b21fd5c16b548815daf69f"} Jul 12 18:36:02.487359 ip-10-0-129-128 kubenswrapper[2559]: I0712 18:36:02.487345 2559 scope.go:117] "RemoveContainer" containerID="e4e47e4380c24391b363d57c1dab7ec4e41f665b77b21fd5c16b548815daf69f" Jul 12 18:36:02.488481 ip-10-0-129-128 kubenswrapper[2559]: I0712 18:36:02.488447 2559 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236228Ki","pods":"250"} Jul 12 18:36:02.488563 ip-10-0-129-128 kubenswrapper[2559]: I0712 18:36:02.488526 2559 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236228Ki","pods":"250"} Jul 12 18:36:02.488605 ip-10-0-129-128 kubenswrapper[2559]: I0712 18:36:02.488561 2559 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236228Ki","pods":"250"} Jul 12 18:36:03.493552 ip-10-0-129-128 kubenswrapper[2559]: I0712 18:36:03.493517 2559 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-mt-70a9c475-openshift-default-958b5846c-cdq9h" event={"ID":"0efa7dab-49ea-4a0e-bf42-95867a0c144f","Type":"ContainerStarted","Data":"67fd16b4c1755c677228df49d3442ee2fc55db97795a461204fdd7df6f6f725f"} Jul 12 18:36:04.121576 ip-10-0-129-128 kubenswrapper[2559]: I0712 18:36:04.121538 2559 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="openshift-ingress/e2e-mt-70a9c475-openshift-default-958b5846c-cdq9h" Jul 12 18:36:04.121808 ip-10-0-129-128 kubenswrapper[2559]: I0712 18:36:04.121782 2559 patch_prober.go:28] interesting pod/e2e-mt-70a9c475-openshift-default-958b5846c-cdq9h container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.132.0.46:15021/healthz/ready\": dial tcp 10.132.0.46:15021: connect: connection refused" start-of-body= Jul 12 18:36:04.121874 ip-10-0-129-128 kubenswrapper[2559]: I0712 18:36:04.121840 2559 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-mt-70a9c475-openshift-default-958b5846c-cdq9h" podUID="0efa7dab-49ea-4a0e-bf42-95867a0c144f" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.132.0.46:15021/healthz/ready\": dial tcp 10.132.0.46:15021: connect: connection refused" Jul 12 18:36:05.121433 ip-10-0-129-128 kubenswrapper[2559]: I0712 18:36:05.121386 2559 patch_prober.go:28] interesting pod/e2e-mt-70a9c475-openshift-default-958b5846c-cdq9h container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.132.0.46:15021/healthz/ready\": dial tcp 10.132.0.46:15021: connect: connection refused" start-of-body= Jul 12 18:36:05.121969 ip-10-0-129-128 kubenswrapper[2559]: I0712 18:36:05.121455 2559 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-mt-70a9c475-openshift-default-958b5846c-cdq9h" podUID="0efa7dab-49ea-4a0e-bf42-95867a0c144f" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.132.0.46:15021/healthz/ready\": dial tcp 10.132.0.46:15021: connect: connection refused" Jul 12 18:36:06.122085 ip-10-0-129-128 kubenswrapper[2559]: I0712 18:36:06.122051 2559 patch_prober.go:28] interesting pod/e2e-mt-70a9c475-openshift-default-958b5846c-cdq9h container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.132.0.46:15021/healthz/ready\": dial tcp 10.132.0.46:15021: connect: connection refused" start-of-body= Jul 12 18:36:06.122479 ip-10-0-129-128 kubenswrapper[2559]: I0712 18:36:06.122122 2559 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-mt-70a9c475-openshift-default-958b5846c-cdq9h" podUID="0efa7dab-49ea-4a0e-bf42-95867a0c144f" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.132.0.46:15021/healthz/ready\": dial tcp 10.132.0.46:15021: connect: connection refused" Jul 12 18:36:07.121560 ip-10-0-129-128 kubenswrapper[2559]: I0712 18:36:07.121518 2559 patch_prober.go:28] interesting pod/e2e-mt-70a9c475-openshift-default-958b5846c-cdq9h container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.132.0.46:15021/healthz/ready\": dial tcp 10.132.0.46:15021: connect: connection refused" start-of-body= Jul 12 18:36:07.121769 ip-10-0-129-128 kubenswrapper[2559]: I0712 18:36:07.121601 2559 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-mt-70a9c475-openshift-default-958b5846c-cdq9h" podUID="0efa7dab-49ea-4a0e-bf42-95867a0c144f" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.132.0.46:15021/healthz/ready\": dial tcp 10.132.0.46:15021: connect: connection refused" Jul 12 18:36:08.125761 ip-10-0-129-128 kubenswrapper[2559]: I0712 18:36:08.125724 2559 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="openshift-ingress/e2e-mt-70a9c475-openshift-default-958b5846c-cdq9h" Jul 12 18:36:08.126230 ip-10-0-129-128 kubenswrapper[2559]: I0712 18:36:08.125999 2559 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/e2e-mt-70a9c475-openshift-default-958b5846c-cdq9h" Jul 12 18:36:08.126740 ip-10-0-129-128 kubenswrapper[2559]: I0712 18:36:08.126720 2559 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/e2e-mt-70a9c475-openshift-default-958b5846c-cdq9h" Jul 12 18:38:13.152909 ip-10-0-129-128 kubenswrapper[2559]: I0712 18:38:13.152818 2559 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-console-operator_console-operator-7866497494-wlk78_f1d46430-bb60-4470-b8d8-eacd44f4f6b1/console-operator/1.log" Jul 12 18:38:13.155201 ip-10-0-129-128 kubenswrapper[2559]: I0712 18:38:13.155177 2559 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ovn-kubernetes_ovnkube-node-q5dw8_e51bad50-bc74-4e38-8307-ca257b599314/ovn-acl-logging/0.log" Jul 12 18:38:13.158449 ip-10-0-129-128 kubenswrapper[2559]: I0712 18:38:13.158428 2559 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-console-operator_console-operator-7866497494-wlk78_f1d46430-bb60-4470-b8d8-eacd44f4f6b1/console-operator/1.log" Jul 12 18:38:13.160903 ip-10-0-129-128 kubenswrapper[2559]: I0712 18:38:13.160884 2559 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ovn-kubernetes_ovnkube-node-q5dw8_e51bad50-bc74-4e38-8307-ca257b599314/ovn-acl-logging/0.log" Jul 12 18:39:07.979399 ip-10-0-129-128 kubenswrapper[2559]: I0712 18:39:07.979359 2559 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-pwsvh"] Jul 12 18:39:11.409046 ip-10-0-129-128 kubenswrapper[2559]: I0712 18:39:11.409008 2559 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-ingress/e2e-mt-70a9c475-openshift-default-958b5846c-cdq9h"] Jul 12 18:39:11.409460 ip-10-0-129-128 kubenswrapper[2559]: I0712 18:39:11.409277 2559 kuberuntime_container.go:873] "Killing container with a grace period" pod="openshift-ingress/e2e-mt-70a9c475-openshift-default-958b5846c-cdq9h" podUID="0efa7dab-49ea-4a0e-bf42-95867a0c144f" containerName="istio-proxy" containerID="cri-o://67fd16b4c1755c677228df49d3442ee2fc55db97795a461204fdd7df6f6f725f" gracePeriod=30 Jul 12 18:39:16.667217 ip-10-0-129-128 kubenswrapper[2559]: I0712 18:39:16.667190 2559 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-mt-70a9c475-openshift-default-958b5846c-cdq9h" Jul 12 18:39:16.837537 ip-10-0-129-128 kubenswrapper[2559]: I0712 18:39:16.837496 2559 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/0efa7dab-49ea-4a0e-bf42-95867a0c144f-istio-data\") pod \"0efa7dab-49ea-4a0e-bf42-95867a0c144f\" (UID: \"0efa7dab-49ea-4a0e-bf42-95867a0c144f\") " Jul 12 18:39:16.837537 ip-10-0-129-128 kubenswrapper[2559]: I0712 18:39:16.837541 2559 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-k9jf6\" (UniqueName: \"kubernetes.io/projected/0efa7dab-49ea-4a0e-bf42-95867a0c144f-kube-api-access-k9jf6\") pod \"0efa7dab-49ea-4a0e-bf42-95867a0c144f\" (UID: \"0efa7dab-49ea-4a0e-bf42-95867a0c144f\") " Jul 12 18:39:16.837847 ip-10-0-129-128 kubenswrapper[2559]: I0712 18:39:16.837596 2559 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/0efa7dab-49ea-4a0e-bf42-95867a0c144f-istio-token\") pod \"0efa7dab-49ea-4a0e-bf42-95867a0c144f\" (UID: \"0efa7dab-49ea-4a0e-bf42-95867a0c144f\") " Jul 12 18:39:16.837847 ip-10-0-129-128 kubenswrapper[2559]: I0712 18:39:16.837661 2559 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/0efa7dab-49ea-4a0e-bf42-95867a0c144f-workload-certs\") pod \"0efa7dab-49ea-4a0e-bf42-95867a0c144f\" (UID: \"0efa7dab-49ea-4a0e-bf42-95867a0c144f\") " Jul 12 18:39:16.837847 ip-10-0-129-128 kubenswrapper[2559]: I0712 18:39:16.837824 2559 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/0efa7dab-49ea-4a0e-bf42-95867a0c144f-credential-socket\") pod \"0efa7dab-49ea-4a0e-bf42-95867a0c144f\" (UID: \"0efa7dab-49ea-4a0e-bf42-95867a0c144f\") " Jul 12 18:39:16.838006 ip-10-0-129-128 kubenswrapper[2559]: I0712 18:39:16.837855 2559 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/0efa7dab-49ea-4a0e-bf42-95867a0c144f-istiod-ca-cert\") pod \"0efa7dab-49ea-4a0e-bf42-95867a0c144f\" (UID: \"0efa7dab-49ea-4a0e-bf42-95867a0c144f\") " Jul 12 18:39:16.838060 ip-10-0-129-128 kubenswrapper[2559]: I0712 18:39:16.838031 2559 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/0efa7dab-49ea-4a0e-bf42-95867a0c144f-workload-certs" (OuterVolumeSpecName: "workload-certs") pod "0efa7dab-49ea-4a0e-bf42-95867a0c144f" (UID: "0efa7dab-49ea-4a0e-bf42-95867a0c144f"). InnerVolumeSpecName "workload-certs". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Jul 12 18:39:16.838116 ip-10-0-129-128 kubenswrapper[2559]: I0712 18:39:16.838070 2559 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/0efa7dab-49ea-4a0e-bf42-95867a0c144f-credential-socket" (OuterVolumeSpecName: "credential-socket") pod "0efa7dab-49ea-4a0e-bf42-95867a0c144f" (UID: "0efa7dab-49ea-4a0e-bf42-95867a0c144f"). InnerVolumeSpecName "credential-socket". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Jul 12 18:39:16.838227 ip-10-0-129-128 kubenswrapper[2559]: I0712 18:39:16.838201 2559 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/0efa7dab-49ea-4a0e-bf42-95867a0c144f-workload-socket\") pod \"0efa7dab-49ea-4a0e-bf42-95867a0c144f\" (UID: \"0efa7dab-49ea-4a0e-bf42-95867a0c144f\") " Jul 12 18:39:16.838360 ip-10-0-129-128 kubenswrapper[2559]: I0712 18:39:16.838294 2559 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/0efa7dab-49ea-4a0e-bf42-95867a0c144f-istio-envoy\") pod \"0efa7dab-49ea-4a0e-bf42-95867a0c144f\" (UID: \"0efa7dab-49ea-4a0e-bf42-95867a0c144f\") " Jul 12 18:39:16.838360 ip-10-0-129-128 kubenswrapper[2559]: I0712 18:39:16.838322 2559 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/0efa7dab-49ea-4a0e-bf42-95867a0c144f-istio-podinfo\") pod \"0efa7dab-49ea-4a0e-bf42-95867a0c144f\" (UID: \"0efa7dab-49ea-4a0e-bf42-95867a0c144f\") " Jul 12 18:39:16.838360 ip-10-0-129-128 kubenswrapper[2559]: I0712 18:39:16.838333 2559 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/0efa7dab-49ea-4a0e-bf42-95867a0c144f-workload-socket" (OuterVolumeSpecName: "workload-socket") pod "0efa7dab-49ea-4a0e-bf42-95867a0c144f" (UID: "0efa7dab-49ea-4a0e-bf42-95867a0c144f"). InnerVolumeSpecName "workload-socket". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Jul 12 18:39:16.838522 ip-10-0-129-128 kubenswrapper[2559]: I0712 18:39:16.838268 2559 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/0efa7dab-49ea-4a0e-bf42-95867a0c144f-istiod-ca-cert" (OuterVolumeSpecName: "istiod-ca-cert") pod "0efa7dab-49ea-4a0e-bf42-95867a0c144f" (UID: "0efa7dab-49ea-4a0e-bf42-95867a0c144f"). InnerVolumeSpecName "istiod-ca-cert". PluginName "kubernetes.io/configmap", VolumeGIDValue "" Jul 12 18:39:16.838522 ip-10-0-129-128 kubenswrapper[2559]: I0712 18:39:16.838371 2559 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/0efa7dab-49ea-4a0e-bf42-95867a0c144f-istio-data" (OuterVolumeSpecName: "istio-data") pod "0efa7dab-49ea-4a0e-bf42-95867a0c144f" (UID: "0efa7dab-49ea-4a0e-bf42-95867a0c144f"). InnerVolumeSpecName "istio-data". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Jul 12 18:39:16.838626 ip-10-0-129-128 kubenswrapper[2559]: I0712 18:39:16.838609 2559 reconciler_common.go:299] "Volume detached for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/0efa7dab-49ea-4a0e-bf42-95867a0c144f-istio-data\") on node \"ip-10-0-129-128.ec2.internal\" DevicePath \"\"" Jul 12 18:39:16.838705 ip-10-0-129-128 kubenswrapper[2559]: I0712 18:39:16.838630 2559 reconciler_common.go:299] "Volume detached for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/0efa7dab-49ea-4a0e-bf42-95867a0c144f-workload-certs\") on node \"ip-10-0-129-128.ec2.internal\" DevicePath \"\"" Jul 12 18:39:16.838705 ip-10-0-129-128 kubenswrapper[2559]: I0712 18:39:16.838667 2559 reconciler_common.go:299] "Volume detached for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/0efa7dab-49ea-4a0e-bf42-95867a0c144f-credential-socket\") on node \"ip-10-0-129-128.ec2.internal\" DevicePath \"\"" Jul 12 18:39:16.838705 ip-10-0-129-128 kubenswrapper[2559]: I0712 18:39:16.838684 2559 reconciler_common.go:299] "Volume detached for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/0efa7dab-49ea-4a0e-bf42-95867a0c144f-istiod-ca-cert\") on node \"ip-10-0-129-128.ec2.internal\" DevicePath \"\"" Jul 12 18:39:16.838705 ip-10-0-129-128 kubenswrapper[2559]: I0712 18:39:16.838696 2559 reconciler_common.go:299] "Volume detached for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/0efa7dab-49ea-4a0e-bf42-95867a0c144f-workload-socket\") on node \"ip-10-0-129-128.ec2.internal\" DevicePath \"\"" Jul 12 18:39:16.840016 ip-10-0-129-128 kubenswrapper[2559]: I0712 18:39:16.839989 2559 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/0efa7dab-49ea-4a0e-bf42-95867a0c144f-kube-api-access-k9jf6" (OuterVolumeSpecName: "kube-api-access-k9jf6") pod "0efa7dab-49ea-4a0e-bf42-95867a0c144f" (UID: "0efa7dab-49ea-4a0e-bf42-95867a0c144f"). InnerVolumeSpecName "kube-api-access-k9jf6". PluginName "kubernetes.io/projected", VolumeGIDValue "" Jul 12 18:39:16.840197 ip-10-0-129-128 kubenswrapper[2559]: I0712 18:39:16.840178 2559 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/0efa7dab-49ea-4a0e-bf42-95867a0c144f-istio-token" (OuterVolumeSpecName: "istio-token") pod "0efa7dab-49ea-4a0e-bf42-95867a0c144f" (UID: "0efa7dab-49ea-4a0e-bf42-95867a0c144f"). InnerVolumeSpecName "istio-token". PluginName "kubernetes.io/projected", VolumeGIDValue "" Jul 12 18:39:16.840261 ip-10-0-129-128 kubenswrapper[2559]: I0712 18:39:16.840239 2559 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/0efa7dab-49ea-4a0e-bf42-95867a0c144f-istio-envoy" (OuterVolumeSpecName: "istio-envoy") pod "0efa7dab-49ea-4a0e-bf42-95867a0c144f" (UID: "0efa7dab-49ea-4a0e-bf42-95867a0c144f"). InnerVolumeSpecName "istio-envoy". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Jul 12 18:39:16.840362 ip-10-0-129-128 kubenswrapper[2559]: I0712 18:39:16.840343 2559 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/downward-api/0efa7dab-49ea-4a0e-bf42-95867a0c144f-istio-podinfo" (OuterVolumeSpecName: "istio-podinfo") pod "0efa7dab-49ea-4a0e-bf42-95867a0c144f" (UID: "0efa7dab-49ea-4a0e-bf42-95867a0c144f"). InnerVolumeSpecName "istio-podinfo". PluginName "kubernetes.io/downward-api", VolumeGIDValue "" Jul 12 18:39:16.940145 ip-10-0-129-128 kubenswrapper[2559]: I0712 18:39:16.940109 2559 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-k9jf6\" (UniqueName: \"kubernetes.io/projected/0efa7dab-49ea-4a0e-bf42-95867a0c144f-kube-api-access-k9jf6\") on node \"ip-10-0-129-128.ec2.internal\" DevicePath \"\"" Jul 12 18:39:16.940145 ip-10-0-129-128 kubenswrapper[2559]: I0712 18:39:16.940140 2559 reconciler_common.go:299] "Volume detached for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/0efa7dab-49ea-4a0e-bf42-95867a0c144f-istio-token\") on node \"ip-10-0-129-128.ec2.internal\" DevicePath \"\"" Jul 12 18:39:16.940145 ip-10-0-129-128 kubenswrapper[2559]: I0712 18:39:16.940149 2559 reconciler_common.go:299] "Volume detached for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/0efa7dab-49ea-4a0e-bf42-95867a0c144f-istio-envoy\") on node \"ip-10-0-129-128.ec2.internal\" DevicePath \"\"" Jul 12 18:39:16.940385 ip-10-0-129-128 kubenswrapper[2559]: I0712 18:39:16.940159 2559 reconciler_common.go:299] "Volume detached for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/0efa7dab-49ea-4a0e-bf42-95867a0c144f-istio-podinfo\") on node \"ip-10-0-129-128.ec2.internal\" DevicePath \"\"" Jul 12 18:39:17.409040 ip-10-0-129-128 kubenswrapper[2559]: I0712 18:39:17.409003 2559 generic.go:356] "Generic (PLEG): container finished" podID="0efa7dab-49ea-4a0e-bf42-95867a0c144f" containerID="67fd16b4c1755c677228df49d3442ee2fc55db97795a461204fdd7df6f6f725f" exitCode=0 Jul 12 18:39:17.409210 ip-10-0-129-128 kubenswrapper[2559]: I0712 18:39:17.409052 2559 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-mt-70a9c475-openshift-default-958b5846c-cdq9h" Jul 12 18:39:17.409210 ip-10-0-129-128 kubenswrapper[2559]: I0712 18:39:17.409060 2559 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-mt-70a9c475-openshift-default-958b5846c-cdq9h" event={"ID":"0efa7dab-49ea-4a0e-bf42-95867a0c144f","Type":"ContainerDied","Data":"67fd16b4c1755c677228df49d3442ee2fc55db97795a461204fdd7df6f6f725f"} Jul 12 18:39:17.409210 ip-10-0-129-128 kubenswrapper[2559]: I0712 18:39:17.409105 2559 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-mt-70a9c475-openshift-default-958b5846c-cdq9h" event={"ID":"0efa7dab-49ea-4a0e-bf42-95867a0c144f","Type":"ContainerDied","Data":"c4a0941da052a4c484b614bcaf75f37d09d186a63c2abe870d5655ff25b304a7"} Jul 12 18:39:17.409210 ip-10-0-129-128 kubenswrapper[2559]: I0712 18:39:17.409127 2559 scope.go:117] "RemoveContainer" containerID="67fd16b4c1755c677228df49d3442ee2fc55db97795a461204fdd7df6f6f725f" Jul 12 18:39:17.418227 ip-10-0-129-128 kubenswrapper[2559]: I0712 18:39:17.418209 2559 scope.go:117] "RemoveContainer" containerID="e4e47e4380c24391b363d57c1dab7ec4e41f665b77b21fd5c16b548815daf69f" Jul 12 18:39:17.426968 ip-10-0-129-128 kubenswrapper[2559]: I0712 18:39:17.426932 2559 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-ingress/e2e-mt-70a9c475-openshift-default-958b5846c-cdq9h"] Jul 12 18:39:17.427473 ip-10-0-129-128 kubenswrapper[2559]: I0712 18:39:17.427454 2559 scope.go:117] "RemoveContainer" containerID="67fd16b4c1755c677228df49d3442ee2fc55db97795a461204fdd7df6f6f725f" Jul 12 18:39:17.427768 ip-10-0-129-128 kubenswrapper[2559]: E0712 18:39:17.427739 2559 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"67fd16b4c1755c677228df49d3442ee2fc55db97795a461204fdd7df6f6f725f\": container with ID starting with 67fd16b4c1755c677228df49d3442ee2fc55db97795a461204fdd7df6f6f725f not found: ID does not exist" containerID="67fd16b4c1755c677228df49d3442ee2fc55db97795a461204fdd7df6f6f725f" Jul 12 18:39:17.427872 ip-10-0-129-128 kubenswrapper[2559]: I0712 18:39:17.427781 2559 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"67fd16b4c1755c677228df49d3442ee2fc55db97795a461204fdd7df6f6f725f"} err="failed to get container status \"67fd16b4c1755c677228df49d3442ee2fc55db97795a461204fdd7df6f6f725f\": rpc error: code = NotFound desc = could not find container \"67fd16b4c1755c677228df49d3442ee2fc55db97795a461204fdd7df6f6f725f\": container with ID starting with 67fd16b4c1755c677228df49d3442ee2fc55db97795a461204fdd7df6f6f725f not found: ID does not exist" Jul 12 18:39:17.427872 ip-10-0-129-128 kubenswrapper[2559]: I0712 18:39:17.427798 2559 scope.go:117] "RemoveContainer" containerID="e4e47e4380c24391b363d57c1dab7ec4e41f665b77b21fd5c16b548815daf69f" Jul 12 18:39:17.428074 ip-10-0-129-128 kubenswrapper[2559]: E0712 18:39:17.428055 2559 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"e4e47e4380c24391b363d57c1dab7ec4e41f665b77b21fd5c16b548815daf69f\": container with ID starting with e4e47e4380c24391b363d57c1dab7ec4e41f665b77b21fd5c16b548815daf69f not found: ID does not exist" containerID="e4e47e4380c24391b363d57c1dab7ec4e41f665b77b21fd5c16b548815daf69f" Jul 12 18:39:17.428120 ip-10-0-129-128 kubenswrapper[2559]: I0712 18:39:17.428082 2559 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"e4e47e4380c24391b363d57c1dab7ec4e41f665b77b21fd5c16b548815daf69f"} err="failed to get container status \"e4e47e4380c24391b363d57c1dab7ec4e41f665b77b21fd5c16b548815daf69f\": rpc error: code = NotFound desc = could not find container \"e4e47e4380c24391b363d57c1dab7ec4e41f665b77b21fd5c16b548815daf69f\": container with ID starting with e4e47e4380c24391b363d57c1dab7ec4e41f665b77b21fd5c16b548815daf69f not found: ID does not exist" Jul 12 18:39:17.431805 ip-10-0-129-128 kubenswrapper[2559]: I0712 18:39:17.431786 2559 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["openshift-ingress/e2e-mt-70a9c475-openshift-default-958b5846c-cdq9h"] Jul 12 18:39:19.034901 ip-10-0-129-128 kubenswrapper[2559]: I0712 18:39:19.034868 2559 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="0efa7dab-49ea-4a0e-bf42-95867a0c144f" path="/var/lib/kubelet/pods/0efa7dab-49ea-4a0e-bf42-95867a0c144f/volumes" Jul 12 18:39:30.923713 ip-10-0-129-128 kubenswrapper[2559]: I0712 18:39:30.923679 2559 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-ingress/e2e-mt-833acb39-openshift-default-858cc74ff-cn9hn"] Jul 12 18:39:30.924089 ip-10-0-129-128 kubenswrapper[2559]: I0712 18:39:30.924038 2559 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="9f172362-788e-4ce3-a328-f967e623de5f" containerName="istio-proxy" Jul 12 18:39:30.924089 ip-10-0-129-128 kubenswrapper[2559]: I0712 18:39:30.924049 2559 state_mem.go:107] "Deleted CPUSet assignment" podUID="9f172362-788e-4ce3-a328-f967e623de5f" containerName="istio-proxy" Jul 12 18:39:30.924089 ip-10-0-129-128 kubenswrapper[2559]: I0712 18:39:30.924070 2559 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="0efa7dab-49ea-4a0e-bf42-95867a0c144f" containerName="istio-proxy" Jul 12 18:39:30.924089 ip-10-0-129-128 kubenswrapper[2559]: I0712 18:39:30.924076 2559 state_mem.go:107] "Deleted CPUSet assignment" podUID="0efa7dab-49ea-4a0e-bf42-95867a0c144f" containerName="istio-proxy" Jul 12 18:39:30.924235 ip-10-0-129-128 kubenswrapper[2559]: I0712 18:39:30.924138 2559 memory_manager.go:368] "RemoveStaleState removing state" podUID="0efa7dab-49ea-4a0e-bf42-95867a0c144f" containerName="istio-proxy" Jul 12 18:39:30.924235 ip-10-0-129-128 kubenswrapper[2559]: I0712 18:39:30.924147 2559 memory_manager.go:368] "RemoveStaleState removing state" podUID="0efa7dab-49ea-4a0e-bf42-95867a0c144f" containerName="istio-proxy" Jul 12 18:39:30.924235 ip-10-0-129-128 kubenswrapper[2559]: I0712 18:39:30.924154 2559 memory_manager.go:368] "RemoveStaleState removing state" podUID="9f172362-788e-4ce3-a328-f967e623de5f" containerName="istio-proxy" Jul 12 18:39:30.927020 ip-10-0-129-128 kubenswrapper[2559]: I0712 18:39:30.927002 2559 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-mt-833acb39-openshift-default-858cc74ff-cn9hn" Jul 12 18:39:30.929924 ip-10-0-129-128 kubenswrapper[2559]: I0712 18:39:30.929897 2559 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress\"/\"e2e-mt-833acb39-openshift-default-dockercfg-tmz7z\"" Jul 12 18:39:30.942382 ip-10-0-129-128 kubenswrapper[2559]: I0712 18:39:30.942360 2559 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/e2e-mt-833acb39-openshift-default-858cc74ff-cn9hn"] Jul 12 18:39:30.946654 ip-10-0-129-128 kubenswrapper[2559]: I0712 18:39:30.946609 2559 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/65e98e30-049f-455b-ae99-a10476c9df1e-istiod-ca-cert\") pod \"e2e-mt-833acb39-openshift-default-858cc74ff-cn9hn\" (UID: \"65e98e30-049f-455b-ae99-a10476c9df1e\") " pod="openshift-ingress/e2e-mt-833acb39-openshift-default-858cc74ff-cn9hn" Jul 12 18:39:30.946790 ip-10-0-129-128 kubenswrapper[2559]: I0712 18:39:30.946664 2559 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/65e98e30-049f-455b-ae99-a10476c9df1e-istio-envoy\") pod \"e2e-mt-833acb39-openshift-default-858cc74ff-cn9hn\" (UID: \"65e98e30-049f-455b-ae99-a10476c9df1e\") " pod="openshift-ingress/e2e-mt-833acb39-openshift-default-858cc74ff-cn9hn" Jul 12 18:39:30.946790 ip-10-0-129-128 kubenswrapper[2559]: I0712 18:39:30.946703 2559 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/65e98e30-049f-455b-ae99-a10476c9df1e-istio-podinfo\") pod \"e2e-mt-833acb39-openshift-default-858cc74ff-cn9hn\" (UID: \"65e98e30-049f-455b-ae99-a10476c9df1e\") " pod="openshift-ingress/e2e-mt-833acb39-openshift-default-858cc74ff-cn9hn" Jul 12 18:39:30.946915 ip-10-0-129-128 kubenswrapper[2559]: I0712 18:39:30.946779 2559 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/65e98e30-049f-455b-ae99-a10476c9df1e-workload-certs\") pod \"e2e-mt-833acb39-openshift-default-858cc74ff-cn9hn\" (UID: \"65e98e30-049f-455b-ae99-a10476c9df1e\") " pod="openshift-ingress/e2e-mt-833acb39-openshift-default-858cc74ff-cn9hn" Jul 12 18:39:30.946915 ip-10-0-129-128 kubenswrapper[2559]: I0712 18:39:30.946819 2559 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/65e98e30-049f-455b-ae99-a10476c9df1e-istio-token\") pod \"e2e-mt-833acb39-openshift-default-858cc74ff-cn9hn\" (UID: \"65e98e30-049f-455b-ae99-a10476c9df1e\") " pod="openshift-ingress/e2e-mt-833acb39-openshift-default-858cc74ff-cn9hn" Jul 12 18:39:30.946915 ip-10-0-129-128 kubenswrapper[2559]: I0712 18:39:30.946877 2559 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/65e98e30-049f-455b-ae99-a10476c9df1e-istio-data\") pod \"e2e-mt-833acb39-openshift-default-858cc74ff-cn9hn\" (UID: \"65e98e30-049f-455b-ae99-a10476c9df1e\") " pod="openshift-ingress/e2e-mt-833acb39-openshift-default-858cc74ff-cn9hn" Jul 12 18:39:30.947082 ip-10-0-129-128 kubenswrapper[2559]: I0712 18:39:30.946924 2559 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/65e98e30-049f-455b-ae99-a10476c9df1e-credential-socket\") pod \"e2e-mt-833acb39-openshift-default-858cc74ff-cn9hn\" (UID: \"65e98e30-049f-455b-ae99-a10476c9df1e\") " pod="openshift-ingress/e2e-mt-833acb39-openshift-default-858cc74ff-cn9hn" Jul 12 18:39:30.947082 ip-10-0-129-128 kubenswrapper[2559]: I0712 18:39:30.946955 2559 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-mzc7n\" (UniqueName: \"kubernetes.io/projected/65e98e30-049f-455b-ae99-a10476c9df1e-kube-api-access-mzc7n\") pod \"e2e-mt-833acb39-openshift-default-858cc74ff-cn9hn\" (UID: \"65e98e30-049f-455b-ae99-a10476c9df1e\") " pod="openshift-ingress/e2e-mt-833acb39-openshift-default-858cc74ff-cn9hn" Jul 12 18:39:30.947082 ip-10-0-129-128 kubenswrapper[2559]: I0712 18:39:30.947029 2559 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/65e98e30-049f-455b-ae99-a10476c9df1e-workload-socket\") pod \"e2e-mt-833acb39-openshift-default-858cc74ff-cn9hn\" (UID: \"65e98e30-049f-455b-ae99-a10476c9df1e\") " pod="openshift-ingress/e2e-mt-833acb39-openshift-default-858cc74ff-cn9hn" Jul 12 18:39:31.048224 ip-10-0-129-128 kubenswrapper[2559]: I0712 18:39:31.048189 2559 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/65e98e30-049f-455b-ae99-a10476c9df1e-istiod-ca-cert\") pod \"e2e-mt-833acb39-openshift-default-858cc74ff-cn9hn\" (UID: \"65e98e30-049f-455b-ae99-a10476c9df1e\") " pod="openshift-ingress/e2e-mt-833acb39-openshift-default-858cc74ff-cn9hn" Jul 12 18:39:31.048428 ip-10-0-129-128 kubenswrapper[2559]: I0712 18:39:31.048230 2559 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/65e98e30-049f-455b-ae99-a10476c9df1e-istio-envoy\") pod \"e2e-mt-833acb39-openshift-default-858cc74ff-cn9hn\" (UID: \"65e98e30-049f-455b-ae99-a10476c9df1e\") " pod="openshift-ingress/e2e-mt-833acb39-openshift-default-858cc74ff-cn9hn" Jul 12 18:39:31.048428 ip-10-0-129-128 kubenswrapper[2559]: I0712 18:39:31.048261 2559 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/65e98e30-049f-455b-ae99-a10476c9df1e-istio-podinfo\") pod \"e2e-mt-833acb39-openshift-default-858cc74ff-cn9hn\" (UID: \"65e98e30-049f-455b-ae99-a10476c9df1e\") " pod="openshift-ingress/e2e-mt-833acb39-openshift-default-858cc74ff-cn9hn" Jul 12 18:39:31.048428 ip-10-0-129-128 kubenswrapper[2559]: I0712 18:39:31.048297 2559 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/65e98e30-049f-455b-ae99-a10476c9df1e-workload-certs\") pod \"e2e-mt-833acb39-openshift-default-858cc74ff-cn9hn\" (UID: \"65e98e30-049f-455b-ae99-a10476c9df1e\") " pod="openshift-ingress/e2e-mt-833acb39-openshift-default-858cc74ff-cn9hn" Jul 12 18:39:31.048428 ip-10-0-129-128 kubenswrapper[2559]: I0712 18:39:31.048317 2559 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/65e98e30-049f-455b-ae99-a10476c9df1e-istio-token\") pod \"e2e-mt-833acb39-openshift-default-858cc74ff-cn9hn\" (UID: \"65e98e30-049f-455b-ae99-a10476c9df1e\") " pod="openshift-ingress/e2e-mt-833acb39-openshift-default-858cc74ff-cn9hn" Jul 12 18:39:31.048428 ip-10-0-129-128 kubenswrapper[2559]: I0712 18:39:31.048341 2559 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/65e98e30-049f-455b-ae99-a10476c9df1e-istio-data\") pod \"e2e-mt-833acb39-openshift-default-858cc74ff-cn9hn\" (UID: \"65e98e30-049f-455b-ae99-a10476c9df1e\") " pod="openshift-ingress/e2e-mt-833acb39-openshift-default-858cc74ff-cn9hn" Jul 12 18:39:31.048428 ip-10-0-129-128 kubenswrapper[2559]: I0712 18:39:31.048386 2559 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/65e98e30-049f-455b-ae99-a10476c9df1e-credential-socket\") pod \"e2e-mt-833acb39-openshift-default-858cc74ff-cn9hn\" (UID: \"65e98e30-049f-455b-ae99-a10476c9df1e\") " pod="openshift-ingress/e2e-mt-833acb39-openshift-default-858cc74ff-cn9hn" Jul 12 18:39:31.048428 ip-10-0-129-128 kubenswrapper[2559]: I0712 18:39:31.048414 2559 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-mzc7n\" (UniqueName: \"kubernetes.io/projected/65e98e30-049f-455b-ae99-a10476c9df1e-kube-api-access-mzc7n\") pod \"e2e-mt-833acb39-openshift-default-858cc74ff-cn9hn\" (UID: \"65e98e30-049f-455b-ae99-a10476c9df1e\") " pod="openshift-ingress/e2e-mt-833acb39-openshift-default-858cc74ff-cn9hn" Jul 12 18:39:31.048847 ip-10-0-129-128 kubenswrapper[2559]: I0712 18:39:31.048484 2559 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/65e98e30-049f-455b-ae99-a10476c9df1e-workload-socket\") pod \"e2e-mt-833acb39-openshift-default-858cc74ff-cn9hn\" (UID: \"65e98e30-049f-455b-ae99-a10476c9df1e\") " pod="openshift-ingress/e2e-mt-833acb39-openshift-default-858cc74ff-cn9hn" Jul 12 18:39:31.048847 ip-10-0-129-128 kubenswrapper[2559]: I0712 18:39:31.048777 2559 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/65e98e30-049f-455b-ae99-a10476c9df1e-workload-certs\") pod \"e2e-mt-833acb39-openshift-default-858cc74ff-cn9hn\" (UID: \"65e98e30-049f-455b-ae99-a10476c9df1e\") " pod="openshift-ingress/e2e-mt-833acb39-openshift-default-858cc74ff-cn9hn" Jul 12 18:39:31.048847 ip-10-0-129-128 kubenswrapper[2559]: I0712 18:39:31.048811 2559 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/65e98e30-049f-455b-ae99-a10476c9df1e-credential-socket\") pod \"e2e-mt-833acb39-openshift-default-858cc74ff-cn9hn\" (UID: \"65e98e30-049f-455b-ae99-a10476c9df1e\") " pod="openshift-ingress/e2e-mt-833acb39-openshift-default-858cc74ff-cn9hn" Jul 12 18:39:31.048992 ip-10-0-129-128 kubenswrapper[2559]: I0712 18:39:31.048867 2559 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/65e98e30-049f-455b-ae99-a10476c9df1e-workload-socket\") pod \"e2e-mt-833acb39-openshift-default-858cc74ff-cn9hn\" (UID: \"65e98e30-049f-455b-ae99-a10476c9df1e\") " pod="openshift-ingress/e2e-mt-833acb39-openshift-default-858cc74ff-cn9hn" Jul 12 18:39:31.049078 ip-10-0-129-128 kubenswrapper[2559]: I0712 18:39:31.049053 2559 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/65e98e30-049f-455b-ae99-a10476c9df1e-istio-data\") pod \"e2e-mt-833acb39-openshift-default-858cc74ff-cn9hn\" (UID: \"65e98e30-049f-455b-ae99-a10476c9df1e\") " pod="openshift-ingress/e2e-mt-833acb39-openshift-default-858cc74ff-cn9hn" Jul 12 18:39:31.049116 ip-10-0-129-128 kubenswrapper[2559]: I0712 18:39:31.049075 2559 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/65e98e30-049f-455b-ae99-a10476c9df1e-istiod-ca-cert\") pod \"e2e-mt-833acb39-openshift-default-858cc74ff-cn9hn\" (UID: \"65e98e30-049f-455b-ae99-a10476c9df1e\") " pod="openshift-ingress/e2e-mt-833acb39-openshift-default-858cc74ff-cn9hn" Jul 12 18:39:31.050737 ip-10-0-129-128 kubenswrapper[2559]: I0712 18:39:31.050708 2559 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/65e98e30-049f-455b-ae99-a10476c9df1e-istio-envoy\") pod \"e2e-mt-833acb39-openshift-default-858cc74ff-cn9hn\" (UID: \"65e98e30-049f-455b-ae99-a10476c9df1e\") " pod="openshift-ingress/e2e-mt-833acb39-openshift-default-858cc74ff-cn9hn" Jul 12 18:39:31.050970 ip-10-0-129-128 kubenswrapper[2559]: I0712 18:39:31.050952 2559 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/65e98e30-049f-455b-ae99-a10476c9df1e-istio-podinfo\") pod \"e2e-mt-833acb39-openshift-default-858cc74ff-cn9hn\" (UID: \"65e98e30-049f-455b-ae99-a10476c9df1e\") " pod="openshift-ingress/e2e-mt-833acb39-openshift-default-858cc74ff-cn9hn" Jul 12 18:39:31.056456 ip-10-0-129-128 kubenswrapper[2559]: I0712 18:39:31.056425 2559 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/65e98e30-049f-455b-ae99-a10476c9df1e-istio-token\") pod \"e2e-mt-833acb39-openshift-default-858cc74ff-cn9hn\" (UID: \"65e98e30-049f-455b-ae99-a10476c9df1e\") " pod="openshift-ingress/e2e-mt-833acb39-openshift-default-858cc74ff-cn9hn" Jul 12 18:39:31.056889 ip-10-0-129-128 kubenswrapper[2559]: I0712 18:39:31.056871 2559 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-mzc7n\" (UniqueName: \"kubernetes.io/projected/65e98e30-049f-455b-ae99-a10476c9df1e-kube-api-access-mzc7n\") pod \"e2e-mt-833acb39-openshift-default-858cc74ff-cn9hn\" (UID: \"65e98e30-049f-455b-ae99-a10476c9df1e\") " pod="openshift-ingress/e2e-mt-833acb39-openshift-default-858cc74ff-cn9hn" Jul 12 18:39:31.238622 ip-10-0-129-128 kubenswrapper[2559]: I0712 18:39:31.238529 2559 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-mt-833acb39-openshift-default-858cc74ff-cn9hn" Jul 12 18:39:31.388690 ip-10-0-129-128 kubenswrapper[2559]: I0712 18:39:31.388657 2559 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/e2e-mt-833acb39-openshift-default-858cc74ff-cn9hn"] Jul 12 18:39:31.390859 ip-10-0-129-128 kubenswrapper[2559]: W0712 18:39:31.390829 2559 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod65e98e30_049f_455b_ae99_a10476c9df1e.slice/crio-5a99bd9884ab5c0d2a08aec71d16f37f87483254a87fd6a89e317b9bde071d01 WatchSource:0}: Error finding container 5a99bd9884ab5c0d2a08aec71d16f37f87483254a87fd6a89e317b9bde071d01: Status 404 returned error can't find the container with id 5a99bd9884ab5c0d2a08aec71d16f37f87483254a87fd6a89e317b9bde071d01 Jul 12 18:39:31.393007 ip-10-0-129-128 kubenswrapper[2559]: I0712 18:39:31.392974 2559 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236228Ki","pods":"250"} Jul 12 18:39:31.393124 ip-10-0-129-128 kubenswrapper[2559]: I0712 18:39:31.393039 2559 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236228Ki","pods":"250"} Jul 12 18:39:31.393124 ip-10-0-129-128 kubenswrapper[2559]: I0712 18:39:31.393070 2559 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236228Ki","pods":"250"} Jul 12 18:39:31.489491 ip-10-0-129-128 kubenswrapper[2559]: I0712 18:39:31.489397 2559 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-mt-833acb39-openshift-default-858cc74ff-cn9hn" event={"ID":"65e98e30-049f-455b-ae99-a10476c9df1e","Type":"ContainerStarted","Data":"5a99bd9884ab5c0d2a08aec71d16f37f87483254a87fd6a89e317b9bde071d01"} Jul 12 18:39:32.496650 ip-10-0-129-128 kubenswrapper[2559]: I0712 18:39:32.496599 2559 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-mt-833acb39-openshift-default-858cc74ff-cn9hn" event={"ID":"65e98e30-049f-455b-ae99-a10476c9df1e","Type":"ContainerStarted","Data":"74fc067c520c4ebce2a582fc1f6af86af9608b79d26aba115cb7bfce0e51f0c3"} Jul 12 18:39:32.517995 ip-10-0-129-128 kubenswrapper[2559]: I0712 18:39:32.517943 2559 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-ingress/e2e-mt-833acb39-openshift-default-858cc74ff-cn9hn" podStartSLOduration=2.517925538 podStartE2EDuration="2.517925538s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-12 18:39:30 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-12 18:39:32.515256563 +0000 UTC m=+2780.009324490" watchObservedRunningTime="2026-07-12 18:39:32.517925538 +0000 UTC m=+2780.011993467" Jul 12 18:39:33.239526 ip-10-0-129-128 kubenswrapper[2559]: I0712 18:39:33.239483 2559 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="openshift-ingress/e2e-mt-833acb39-openshift-default-858cc74ff-cn9hn" Jul 12 18:39:33.244430 ip-10-0-129-128 kubenswrapper[2559]: I0712 18:39:33.244405 2559 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="openshift-ingress/e2e-mt-833acb39-openshift-default-858cc74ff-cn9hn" Jul 12 18:39:33.501841 ip-10-0-129-128 kubenswrapper[2559]: I0712 18:39:33.501757 2559 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/e2e-mt-833acb39-openshift-default-858cc74ff-cn9hn" Jul 12 18:39:33.502818 ip-10-0-129-128 kubenswrapper[2559]: I0712 18:39:33.502794 2559 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/e2e-mt-833acb39-openshift-default-858cc74ff-cn9hn" Jul 12 18:39:39.291196 ip-10-0-129-128 kubenswrapper[2559]: I0712 18:39:39.291111 2559 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-pwsvh"] Jul 12 18:39:47.579660 ip-10-0-129-128 kubenswrapper[2559]: I0712 18:39:47.579551 2559 generic.go:356] "Generic (PLEG): container finished" podID="65e98e30-049f-455b-ae99-a10476c9df1e" containerID="74fc067c520c4ebce2a582fc1f6af86af9608b79d26aba115cb7bfce0e51f0c3" exitCode=0 Jul 12 18:39:47.579660 ip-10-0-129-128 kubenswrapper[2559]: I0712 18:39:47.579624 2559 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-mt-833acb39-openshift-default-858cc74ff-cn9hn" event={"ID":"65e98e30-049f-455b-ae99-a10476c9df1e","Type":"ContainerDied","Data":"74fc067c520c4ebce2a582fc1f6af86af9608b79d26aba115cb7bfce0e51f0c3"} Jul 12 18:39:47.580113 ip-10-0-129-128 kubenswrapper[2559]: I0712 18:39:47.580053 2559 scope.go:117] "RemoveContainer" containerID="74fc067c520c4ebce2a582fc1f6af86af9608b79d26aba115cb7bfce0e51f0c3" Jul 12 18:39:47.581115 ip-10-0-129-128 kubenswrapper[2559]: I0712 18:39:47.581087 2559 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236228Ki","pods":"250"} Jul 12 18:39:47.581234 ip-10-0-129-128 kubenswrapper[2559]: I0712 18:39:47.581159 2559 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236228Ki","pods":"250"} Jul 12 18:39:47.581234 ip-10-0-129-128 kubenswrapper[2559]: I0712 18:39:47.581196 2559 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236228Ki","pods":"250"} Jul 12 18:39:48.502577 ip-10-0-129-128 kubenswrapper[2559]: I0712 18:39:48.502546 2559 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/e2e-mt-833acb39-openshift-default-858cc74ff-cn9hn" Jul 12 18:39:48.586304 ip-10-0-129-128 kubenswrapper[2559]: I0712 18:39:48.586263 2559 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-mt-833acb39-openshift-default-858cc74ff-cn9hn" event={"ID":"65e98e30-049f-455b-ae99-a10476c9df1e","Type":"ContainerStarted","Data":"88f34ccaf90ef1dc2cdf20d1d0ad8f19e9b19c822750f61e9af0303daecb5c90"} Jul 12 18:39:49.239981 ip-10-0-129-128 kubenswrapper[2559]: I0712 18:39:49.239950 2559 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="openshift-ingress/e2e-mt-833acb39-openshift-default-858cc74ff-cn9hn" Jul 12 18:39:49.240306 ip-10-0-129-128 kubenswrapper[2559]: I0712 18:39:49.240285 2559 patch_prober.go:28] interesting pod/e2e-mt-833acb39-openshift-default-858cc74ff-cn9hn container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.132.0.47:15021/healthz/ready\": dial tcp 10.132.0.47:15021: connect: connection refused" start-of-body= Jul 12 18:39:49.240457 ip-10-0-129-128 kubenswrapper[2559]: I0712 18:39:49.240432 2559 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-mt-833acb39-openshift-default-858cc74ff-cn9hn" podUID="65e98e30-049f-455b-ae99-a10476c9df1e" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.132.0.47:15021/healthz/ready\": dial tcp 10.132.0.47:15021: connect: connection refused" Jul 12 18:39:50.239766 ip-10-0-129-128 kubenswrapper[2559]: I0712 18:39:50.239732 2559 patch_prober.go:28] interesting pod/e2e-mt-833acb39-openshift-default-858cc74ff-cn9hn container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.132.0.47:15021/healthz/ready\": dial tcp 10.132.0.47:15021: connect: connection refused" start-of-body= Jul 12 18:39:50.240368 ip-10-0-129-128 kubenswrapper[2559]: I0712 18:39:50.240327 2559 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-mt-833acb39-openshift-default-858cc74ff-cn9hn" podUID="65e98e30-049f-455b-ae99-a10476c9df1e" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.132.0.47:15021/healthz/ready\": dial tcp 10.132.0.47:15021: connect: connection refused" Jul 12 18:39:51.239735 ip-10-0-129-128 kubenswrapper[2559]: I0712 18:39:51.239682 2559 patch_prober.go:28] interesting pod/e2e-mt-833acb39-openshift-default-858cc74ff-cn9hn container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.132.0.47:15021/healthz/ready\": dial tcp 10.132.0.47:15021: connect: connection refused" start-of-body= Jul 12 18:39:51.239969 ip-10-0-129-128 kubenswrapper[2559]: I0712 18:39:51.239758 2559 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-mt-833acb39-openshift-default-858cc74ff-cn9hn" podUID="65e98e30-049f-455b-ae99-a10476c9df1e" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.132.0.47:15021/healthz/ready\": dial tcp 10.132.0.47:15021: connect: connection refused" Jul 12 18:39:52.239394 ip-10-0-129-128 kubenswrapper[2559]: I0712 18:39:52.239354 2559 patch_prober.go:28] interesting pod/e2e-mt-833acb39-openshift-default-858cc74ff-cn9hn container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.132.0.47:15021/healthz/ready\": dial tcp 10.132.0.47:15021: connect: connection refused" start-of-body= Jul 12 18:39:52.239723 ip-10-0-129-128 kubenswrapper[2559]: I0712 18:39:52.239691 2559 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-mt-833acb39-openshift-default-858cc74ff-cn9hn" podUID="65e98e30-049f-455b-ae99-a10476c9df1e" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.132.0.47:15021/healthz/ready\": dial tcp 10.132.0.47:15021: connect: connection refused" Jul 12 18:39:53.242921 ip-10-0-129-128 kubenswrapper[2559]: I0712 18:39:53.242890 2559 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="openshift-ingress/e2e-mt-833acb39-openshift-default-858cc74ff-cn9hn" Jul 12 18:39:53.243308 ip-10-0-129-128 kubenswrapper[2559]: I0712 18:39:53.243173 2559 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/e2e-mt-833acb39-openshift-default-858cc74ff-cn9hn" Jul 12 18:39:53.243781 ip-10-0-129-128 kubenswrapper[2559]: I0712 18:39:53.243762 2559 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/e2e-mt-833acb39-openshift-default-858cc74ff-cn9hn" Jul 12 18:39:57.779016 ip-10-0-129-128 kubenswrapper[2559]: I0712 18:39:57.778983 2559 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-pwsvh"] Jul 12 18:40:19.288049 ip-10-0-129-128 kubenswrapper[2559]: I0712 18:40:19.288013 2559 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-pwsvh"] Jul 12 18:43:13.188199 ip-10-0-129-128 kubenswrapper[2559]: I0712 18:43:13.188168 2559 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-console-operator_console-operator-7866497494-wlk78_f1d46430-bb60-4470-b8d8-eacd44f4f6b1/console-operator/1.log" Jul 12 18:43:13.190673 ip-10-0-129-128 kubenswrapper[2559]: I0712 18:43:13.190628 2559 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ovn-kubernetes_ovnkube-node-q5dw8_e51bad50-bc74-4e38-8307-ca257b599314/ovn-acl-logging/0.log" Jul 12 18:43:13.195667 ip-10-0-129-128 kubenswrapper[2559]: I0712 18:43:13.195628 2559 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-console-operator_console-operator-7866497494-wlk78_f1d46430-bb60-4470-b8d8-eacd44f4f6b1/console-operator/1.log" Jul 12 18:43:13.197837 ip-10-0-129-128 kubenswrapper[2559]: I0712 18:43:13.197818 2559 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ovn-kubernetes_ovnkube-node-q5dw8_e51bad50-bc74-4e38-8307-ca257b599314/ovn-acl-logging/0.log" Jul 12 18:46:28.884355 ip-10-0-129-128 kubenswrapper[2559]: I0712 18:46:28.884314 2559 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-pwsvh"] Jul 12 18:47:12.184372 ip-10-0-129-128 kubenswrapper[2559]: I0712 18:47:12.184295 2559 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-pwsvh"] Jul 12 18:47:37.373266 ip-10-0-129-128 kubenswrapper[2559]: I0712 18:47:37.373226 2559 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-ingress/e2e-mt-833acb39-openshift-default-858cc74ff-cn9hn"] Jul 12 18:47:37.373817 ip-10-0-129-128 kubenswrapper[2559]: I0712 18:47:37.373480 2559 kuberuntime_container.go:873] "Killing container with a grace period" pod="openshift-ingress/e2e-mt-833acb39-openshift-default-858cc74ff-cn9hn" podUID="65e98e30-049f-455b-ae99-a10476c9df1e" containerName="istio-proxy" containerID="cri-o://88f34ccaf90ef1dc2cdf20d1d0ad8f19e9b19c822750f61e9af0303daecb5c90" gracePeriod=30 Jul 12 18:47:38.246242 ip-10-0-129-128 kubenswrapper[2559]: I0712 18:47:38.246201 2559 patch_prober.go:28] interesting pod/e2e-mt-833acb39-openshift-default-858cc74ff-cn9hn container/istio-proxy namespace/openshift-ingress: Readiness probe status=failure output="HTTP probe failed with statuscode: 503" start-of-body= Jul 12 18:47:38.246405 ip-10-0-129-128 kubenswrapper[2559]: I0712 18:47:38.246274 2559 prober.go:120] "Probe failed" probeType="Readiness" pod="openshift-ingress/e2e-mt-833acb39-openshift-default-858cc74ff-cn9hn" podUID="65e98e30-049f-455b-ae99-a10476c9df1e" containerName="istio-proxy" probeResult="failure" output="HTTP probe failed with statuscode: 503" Jul 12 18:47:41.696757 ip-10-0-129-128 kubenswrapper[2559]: I0712 18:47:41.696723 2559 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-pwsvh"] Jul 12 18:47:42.635683 ip-10-0-129-128 kubenswrapper[2559]: I0712 18:47:42.635655 2559 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-mt-833acb39-openshift-default-858cc74ff-cn9hn" Jul 12 18:47:42.727388 ip-10-0-129-128 kubenswrapper[2559]: I0712 18:47:42.727293 2559 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/65e98e30-049f-455b-ae99-a10476c9df1e-istio-token\") pod \"65e98e30-049f-455b-ae99-a10476c9df1e\" (UID: \"65e98e30-049f-455b-ae99-a10476c9df1e\") " Jul 12 18:47:42.727388 ip-10-0-129-128 kubenswrapper[2559]: I0712 18:47:42.727330 2559 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/65e98e30-049f-455b-ae99-a10476c9df1e-workload-socket\") pod \"65e98e30-049f-455b-ae99-a10476c9df1e\" (UID: \"65e98e30-049f-455b-ae99-a10476c9df1e\") " Jul 12 18:47:42.727388 ip-10-0-129-128 kubenswrapper[2559]: I0712 18:47:42.727388 2559 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/65e98e30-049f-455b-ae99-a10476c9df1e-credential-socket\") pod \"65e98e30-049f-455b-ae99-a10476c9df1e\" (UID: \"65e98e30-049f-455b-ae99-a10476c9df1e\") " Jul 12 18:47:42.727985 ip-10-0-129-128 kubenswrapper[2559]: I0712 18:47:42.727415 2559 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/65e98e30-049f-455b-ae99-a10476c9df1e-istiod-ca-cert\") pod \"65e98e30-049f-455b-ae99-a10476c9df1e\" (UID: \"65e98e30-049f-455b-ae99-a10476c9df1e\") " Jul 12 18:47:42.727985 ip-10-0-129-128 kubenswrapper[2559]: I0712 18:47:42.727434 2559 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/65e98e30-049f-455b-ae99-a10476c9df1e-istio-podinfo\") pod \"65e98e30-049f-455b-ae99-a10476c9df1e\" (UID: \"65e98e30-049f-455b-ae99-a10476c9df1e\") " Jul 12 18:47:42.727985 ip-10-0-129-128 kubenswrapper[2559]: I0712 18:47:42.727454 2559 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/65e98e30-049f-455b-ae99-a10476c9df1e-workload-certs\") pod \"65e98e30-049f-455b-ae99-a10476c9df1e\" (UID: \"65e98e30-049f-455b-ae99-a10476c9df1e\") " Jul 12 18:47:42.727985 ip-10-0-129-128 kubenswrapper[2559]: I0712 18:47:42.727485 2559 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/65e98e30-049f-455b-ae99-a10476c9df1e-istio-data\") pod \"65e98e30-049f-455b-ae99-a10476c9df1e\" (UID: \"65e98e30-049f-455b-ae99-a10476c9df1e\") " Jul 12 18:47:42.727985 ip-10-0-129-128 kubenswrapper[2559]: I0712 18:47:42.727545 2559 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-mzc7n\" (UniqueName: \"kubernetes.io/projected/65e98e30-049f-455b-ae99-a10476c9df1e-kube-api-access-mzc7n\") pod \"65e98e30-049f-455b-ae99-a10476c9df1e\" (UID: \"65e98e30-049f-455b-ae99-a10476c9df1e\") " Jul 12 18:47:42.727985 ip-10-0-129-128 kubenswrapper[2559]: I0712 18:47:42.727591 2559 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/65e98e30-049f-455b-ae99-a10476c9df1e-istio-envoy\") pod \"65e98e30-049f-455b-ae99-a10476c9df1e\" (UID: \"65e98e30-049f-455b-ae99-a10476c9df1e\") " Jul 12 18:47:42.727985 ip-10-0-129-128 kubenswrapper[2559]: I0712 18:47:42.727773 2559 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/65e98e30-049f-455b-ae99-a10476c9df1e-workload-socket" (OuterVolumeSpecName: "workload-socket") pod "65e98e30-049f-455b-ae99-a10476c9df1e" (UID: "65e98e30-049f-455b-ae99-a10476c9df1e"). InnerVolumeSpecName "workload-socket". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Jul 12 18:47:42.727985 ip-10-0-129-128 kubenswrapper[2559]: I0712 18:47:42.727817 2559 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/65e98e30-049f-455b-ae99-a10476c9df1e-workload-certs" (OuterVolumeSpecName: "workload-certs") pod "65e98e30-049f-455b-ae99-a10476c9df1e" (UID: "65e98e30-049f-455b-ae99-a10476c9df1e"). InnerVolumeSpecName "workload-certs". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Jul 12 18:47:42.727985 ip-10-0-129-128 kubenswrapper[2559]: I0712 18:47:42.727929 2559 reconciler_common.go:299] "Volume detached for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/65e98e30-049f-455b-ae99-a10476c9df1e-workload-socket\") on node \"ip-10-0-129-128.ec2.internal\" DevicePath \"\"" Jul 12 18:47:42.727985 ip-10-0-129-128 kubenswrapper[2559]: I0712 18:47:42.727946 2559 reconciler_common.go:299] "Volume detached for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/65e98e30-049f-455b-ae99-a10476c9df1e-workload-certs\") on node \"ip-10-0-129-128.ec2.internal\" DevicePath \"\"" Jul 12 18:47:42.728589 ip-10-0-129-128 kubenswrapper[2559]: I0712 18:47:42.728531 2559 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/65e98e30-049f-455b-ae99-a10476c9df1e-istiod-ca-cert" (OuterVolumeSpecName: "istiod-ca-cert") pod "65e98e30-049f-455b-ae99-a10476c9df1e" (UID: "65e98e30-049f-455b-ae99-a10476c9df1e"). InnerVolumeSpecName "istiod-ca-cert". PluginName "kubernetes.io/configmap", VolumeGIDValue "" Jul 12 18:47:42.728751 ip-10-0-129-128 kubenswrapper[2559]: I0712 18:47:42.728600 2559 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/65e98e30-049f-455b-ae99-a10476c9df1e-istio-data" (OuterVolumeSpecName: "istio-data") pod "65e98e30-049f-455b-ae99-a10476c9df1e" (UID: "65e98e30-049f-455b-ae99-a10476c9df1e"). InnerVolumeSpecName "istio-data". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Jul 12 18:47:42.728751 ip-10-0-129-128 kubenswrapper[2559]: I0712 18:47:42.728732 2559 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/65e98e30-049f-455b-ae99-a10476c9df1e-credential-socket" (OuterVolumeSpecName: "credential-socket") pod "65e98e30-049f-455b-ae99-a10476c9df1e" (UID: "65e98e30-049f-455b-ae99-a10476c9df1e"). InnerVolumeSpecName "credential-socket". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Jul 12 18:47:42.730306 ip-10-0-129-128 kubenswrapper[2559]: I0712 18:47:42.730275 2559 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/65e98e30-049f-455b-ae99-a10476c9df1e-istio-envoy" (OuterVolumeSpecName: "istio-envoy") pod "65e98e30-049f-455b-ae99-a10476c9df1e" (UID: "65e98e30-049f-455b-ae99-a10476c9df1e"). InnerVolumeSpecName "istio-envoy". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Jul 12 18:47:42.730441 ip-10-0-129-128 kubenswrapper[2559]: I0712 18:47:42.730325 2559 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/65e98e30-049f-455b-ae99-a10476c9df1e-istio-token" (OuterVolumeSpecName: "istio-token") pod "65e98e30-049f-455b-ae99-a10476c9df1e" (UID: "65e98e30-049f-455b-ae99-a10476c9df1e"). InnerVolumeSpecName "istio-token". PluginName "kubernetes.io/projected", VolumeGIDValue "" Jul 12 18:47:42.730441 ip-10-0-129-128 kubenswrapper[2559]: I0712 18:47:42.730398 2559 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/65e98e30-049f-455b-ae99-a10476c9df1e-kube-api-access-mzc7n" (OuterVolumeSpecName: "kube-api-access-mzc7n") pod "65e98e30-049f-455b-ae99-a10476c9df1e" (UID: "65e98e30-049f-455b-ae99-a10476c9df1e"). InnerVolumeSpecName "kube-api-access-mzc7n". PluginName "kubernetes.io/projected", VolumeGIDValue "" Jul 12 18:47:42.730899 ip-10-0-129-128 kubenswrapper[2559]: I0712 18:47:42.730866 2559 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/downward-api/65e98e30-049f-455b-ae99-a10476c9df1e-istio-podinfo" (OuterVolumeSpecName: "istio-podinfo") pod "65e98e30-049f-455b-ae99-a10476c9df1e" (UID: "65e98e30-049f-455b-ae99-a10476c9df1e"). InnerVolumeSpecName "istio-podinfo". PluginName "kubernetes.io/downward-api", VolumeGIDValue "" Jul 12 18:47:42.829199 ip-10-0-129-128 kubenswrapper[2559]: I0712 18:47:42.829160 2559 reconciler_common.go:299] "Volume detached for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/65e98e30-049f-455b-ae99-a10476c9df1e-istio-data\") on node \"ip-10-0-129-128.ec2.internal\" DevicePath \"\"" Jul 12 18:47:42.829199 ip-10-0-129-128 kubenswrapper[2559]: I0712 18:47:42.829192 2559 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-mzc7n\" (UniqueName: \"kubernetes.io/projected/65e98e30-049f-455b-ae99-a10476c9df1e-kube-api-access-mzc7n\") on node \"ip-10-0-129-128.ec2.internal\" DevicePath \"\"" Jul 12 18:47:42.829199 ip-10-0-129-128 kubenswrapper[2559]: I0712 18:47:42.829203 2559 reconciler_common.go:299] "Volume detached for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/65e98e30-049f-455b-ae99-a10476c9df1e-istio-envoy\") on node \"ip-10-0-129-128.ec2.internal\" DevicePath \"\"" Jul 12 18:47:42.829199 ip-10-0-129-128 kubenswrapper[2559]: I0712 18:47:42.829211 2559 reconciler_common.go:299] "Volume detached for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/65e98e30-049f-455b-ae99-a10476c9df1e-istio-token\") on node \"ip-10-0-129-128.ec2.internal\" DevicePath \"\"" Jul 12 18:47:42.829582 ip-10-0-129-128 kubenswrapper[2559]: I0712 18:47:42.829219 2559 reconciler_common.go:299] "Volume detached for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/65e98e30-049f-455b-ae99-a10476c9df1e-credential-socket\") on node \"ip-10-0-129-128.ec2.internal\" DevicePath \"\"" Jul 12 18:47:42.829582 ip-10-0-129-128 kubenswrapper[2559]: I0712 18:47:42.829227 2559 reconciler_common.go:299] "Volume detached for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/65e98e30-049f-455b-ae99-a10476c9df1e-istiod-ca-cert\") on node \"ip-10-0-129-128.ec2.internal\" DevicePath \"\"" Jul 12 18:47:42.829582 ip-10-0-129-128 kubenswrapper[2559]: I0712 18:47:42.829235 2559 reconciler_common.go:299] "Volume detached for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/65e98e30-049f-455b-ae99-a10476c9df1e-istio-podinfo\") on node \"ip-10-0-129-128.ec2.internal\" DevicePath \"\"" Jul 12 18:47:42.872804 ip-10-0-129-128 kubenswrapper[2559]: I0712 18:47:42.872766 2559 generic.go:356] "Generic (PLEG): container finished" podID="65e98e30-049f-455b-ae99-a10476c9df1e" containerID="88f34ccaf90ef1dc2cdf20d1d0ad8f19e9b19c822750f61e9af0303daecb5c90" exitCode=0 Jul 12 18:47:42.873003 ip-10-0-129-128 kubenswrapper[2559]: I0712 18:47:42.872819 2559 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-mt-833acb39-openshift-default-858cc74ff-cn9hn" Jul 12 18:47:42.873003 ip-10-0-129-128 kubenswrapper[2559]: I0712 18:47:42.872850 2559 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-mt-833acb39-openshift-default-858cc74ff-cn9hn" event={"ID":"65e98e30-049f-455b-ae99-a10476c9df1e","Type":"ContainerDied","Data":"88f34ccaf90ef1dc2cdf20d1d0ad8f19e9b19c822750f61e9af0303daecb5c90"} Jul 12 18:47:42.873003 ip-10-0-129-128 kubenswrapper[2559]: I0712 18:47:42.872946 2559 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-mt-833acb39-openshift-default-858cc74ff-cn9hn" event={"ID":"65e98e30-049f-455b-ae99-a10476c9df1e","Type":"ContainerDied","Data":"5a99bd9884ab5c0d2a08aec71d16f37f87483254a87fd6a89e317b9bde071d01"} Jul 12 18:47:42.873003 ip-10-0-129-128 kubenswrapper[2559]: I0712 18:47:42.872978 2559 scope.go:117] "RemoveContainer" containerID="88f34ccaf90ef1dc2cdf20d1d0ad8f19e9b19c822750f61e9af0303daecb5c90" Jul 12 18:47:42.884557 ip-10-0-129-128 kubenswrapper[2559]: I0712 18:47:42.884534 2559 scope.go:117] "RemoveContainer" containerID="74fc067c520c4ebce2a582fc1f6af86af9608b79d26aba115cb7bfce0e51f0c3" Jul 12 18:47:42.893092 ip-10-0-129-128 kubenswrapper[2559]: I0712 18:47:42.892998 2559 scope.go:117] "RemoveContainer" containerID="88f34ccaf90ef1dc2cdf20d1d0ad8f19e9b19c822750f61e9af0303daecb5c90" Jul 12 18:47:42.893289 ip-10-0-129-128 kubenswrapper[2559]: E0712 18:47:42.893269 2559 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"88f34ccaf90ef1dc2cdf20d1d0ad8f19e9b19c822750f61e9af0303daecb5c90\": container with ID starting with 88f34ccaf90ef1dc2cdf20d1d0ad8f19e9b19c822750f61e9af0303daecb5c90 not found: ID does not exist" containerID="88f34ccaf90ef1dc2cdf20d1d0ad8f19e9b19c822750f61e9af0303daecb5c90" Jul 12 18:47:42.893370 ip-10-0-129-128 kubenswrapper[2559]: I0712 18:47:42.893302 2559 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"88f34ccaf90ef1dc2cdf20d1d0ad8f19e9b19c822750f61e9af0303daecb5c90"} err="failed to get container status \"88f34ccaf90ef1dc2cdf20d1d0ad8f19e9b19c822750f61e9af0303daecb5c90\": rpc error: code = NotFound desc = could not find container \"88f34ccaf90ef1dc2cdf20d1d0ad8f19e9b19c822750f61e9af0303daecb5c90\": container with ID starting with 88f34ccaf90ef1dc2cdf20d1d0ad8f19e9b19c822750f61e9af0303daecb5c90 not found: ID does not exist" Jul 12 18:47:42.893370 ip-10-0-129-128 kubenswrapper[2559]: I0712 18:47:42.893325 2559 scope.go:117] "RemoveContainer" containerID="74fc067c520c4ebce2a582fc1f6af86af9608b79d26aba115cb7bfce0e51f0c3" Jul 12 18:47:42.893619 ip-10-0-129-128 kubenswrapper[2559]: E0712 18:47:42.893597 2559 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"74fc067c520c4ebce2a582fc1f6af86af9608b79d26aba115cb7bfce0e51f0c3\": container with ID starting with 74fc067c520c4ebce2a582fc1f6af86af9608b79d26aba115cb7bfce0e51f0c3 not found: ID does not exist" containerID="74fc067c520c4ebce2a582fc1f6af86af9608b79d26aba115cb7bfce0e51f0c3" Jul 12 18:47:42.893722 ip-10-0-129-128 kubenswrapper[2559]: I0712 18:47:42.893657 2559 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"74fc067c520c4ebce2a582fc1f6af86af9608b79d26aba115cb7bfce0e51f0c3"} err="failed to get container status \"74fc067c520c4ebce2a582fc1f6af86af9608b79d26aba115cb7bfce0e51f0c3\": rpc error: code = NotFound desc = could not find container \"74fc067c520c4ebce2a582fc1f6af86af9608b79d26aba115cb7bfce0e51f0c3\": container with ID starting with 74fc067c520c4ebce2a582fc1f6af86af9608b79d26aba115cb7bfce0e51f0c3 not found: ID does not exist" Jul 12 18:47:42.894393 ip-10-0-129-128 kubenswrapper[2559]: I0712 18:47:42.894369 2559 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-ingress/e2e-mt-833acb39-openshift-default-858cc74ff-cn9hn"] Jul 12 18:47:42.897916 ip-10-0-129-128 kubenswrapper[2559]: I0712 18:47:42.897890 2559 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["openshift-ingress/e2e-mt-833acb39-openshift-default-858cc74ff-cn9hn"] Jul 12 18:47:43.034883 ip-10-0-129-128 kubenswrapper[2559]: I0712 18:47:43.034851 2559 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="65e98e30-049f-455b-ae99-a10476c9df1e" path="/var/lib/kubelet/pods/65e98e30-049f-455b-ae99-a10476c9df1e/volumes" Jul 12 18:47:55.285957 ip-10-0-129-128 kubenswrapper[2559]: I0712 18:47:55.285921 2559 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-pwsvh"] Jul 12 18:48:13.215446 ip-10-0-129-128 kubenswrapper[2559]: I0712 18:48:13.215416 2559 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-console-operator_console-operator-7866497494-wlk78_f1d46430-bb60-4470-b8d8-eacd44f4f6b1/console-operator/1.log" Jul 12 18:48:13.217736 ip-10-0-129-128 kubenswrapper[2559]: I0712 18:48:13.217711 2559 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ovn-kubernetes_ovnkube-node-q5dw8_e51bad50-bc74-4e38-8307-ca257b599314/ovn-acl-logging/0.log" Jul 12 18:48:13.223319 ip-10-0-129-128 kubenswrapper[2559]: I0712 18:48:13.223300 2559 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-console-operator_console-operator-7866497494-wlk78_f1d46430-bb60-4470-b8d8-eacd44f4f6b1/console-operator/1.log" Jul 12 18:48:13.225560 ip-10-0-129-128 kubenswrapper[2559]: I0712 18:48:13.225543 2559 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ovn-kubernetes_ovnkube-node-q5dw8_e51bad50-bc74-4e38-8307-ca257b599314/ovn-acl-logging/0.log" Jul 12 18:48:31.553191 ip-10-0-129-128 kubenswrapper[2559]: I0712 18:48:31.553157 2559 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-ingress/e2e-inf-a-5beeb3-openshift-default-587f95df85-pm8t6"] Jul 12 18:48:31.553715 ip-10-0-129-128 kubenswrapper[2559]: I0712 18:48:31.553576 2559 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="65e98e30-049f-455b-ae99-a10476c9df1e" containerName="istio-proxy" Jul 12 18:48:31.553715 ip-10-0-129-128 kubenswrapper[2559]: I0712 18:48:31.553593 2559 state_mem.go:107] "Deleted CPUSet assignment" podUID="65e98e30-049f-455b-ae99-a10476c9df1e" containerName="istio-proxy" Jul 12 18:48:31.553715 ip-10-0-129-128 kubenswrapper[2559]: I0712 18:48:31.553610 2559 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="0efa7dab-49ea-4a0e-bf42-95867a0c144f" containerName="istio-proxy" Jul 12 18:48:31.553715 ip-10-0-129-128 kubenswrapper[2559]: I0712 18:48:31.553619 2559 state_mem.go:107] "Deleted CPUSet assignment" podUID="0efa7dab-49ea-4a0e-bf42-95867a0c144f" containerName="istio-proxy" Jul 12 18:48:31.553715 ip-10-0-129-128 kubenswrapper[2559]: I0712 18:48:31.553630 2559 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="65e98e30-049f-455b-ae99-a10476c9df1e" containerName="istio-proxy" Jul 12 18:48:31.553715 ip-10-0-129-128 kubenswrapper[2559]: I0712 18:48:31.553652 2559 state_mem.go:107] "Deleted CPUSet assignment" podUID="65e98e30-049f-455b-ae99-a10476c9df1e" containerName="istio-proxy" Jul 12 18:48:31.554053 ip-10-0-129-128 kubenswrapper[2559]: I0712 18:48:31.553736 2559 memory_manager.go:368] "RemoveStaleState removing state" podUID="65e98e30-049f-455b-ae99-a10476c9df1e" containerName="istio-proxy" Jul 12 18:48:31.554053 ip-10-0-129-128 kubenswrapper[2559]: I0712 18:48:31.553749 2559 memory_manager.go:368] "RemoveStaleState removing state" podUID="65e98e30-049f-455b-ae99-a10476c9df1e" containerName="istio-proxy" Jul 12 18:48:31.557114 ip-10-0-129-128 kubenswrapper[2559]: I0712 18:48:31.557094 2559 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-inf-a-5beeb3-openshift-default-587f95df85-pm8t6" Jul 12 18:48:31.559570 ip-10-0-129-128 kubenswrapper[2559]: I0712 18:48:31.559551 2559 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress\"/\"e2e-inf-a-5beeb3-openshift-default-dockercfg-x6nhq\"" Jul 12 18:48:31.572259 ip-10-0-129-128 kubenswrapper[2559]: I0712 18:48:31.572235 2559 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/e2e-inf-a-5beeb3-openshift-default-587f95df85-pm8t6"] Jul 12 18:48:31.635229 ip-10-0-129-128 kubenswrapper[2559]: I0712 18:48:31.635197 2559 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/e444c91f-2e77-453d-b905-178bb96e46d5-workload-socket\") pod \"e2e-inf-a-5beeb3-openshift-default-587f95df85-pm8t6\" (UID: \"e444c91f-2e77-453d-b905-178bb96e46d5\") " pod="openshift-ingress/e2e-inf-a-5beeb3-openshift-default-587f95df85-pm8t6" Jul 12 18:48:31.635365 ip-10-0-129-128 kubenswrapper[2559]: I0712 18:48:31.635253 2559 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/e444c91f-2e77-453d-b905-178bb96e46d5-workload-certs\") pod \"e2e-inf-a-5beeb3-openshift-default-587f95df85-pm8t6\" (UID: \"e444c91f-2e77-453d-b905-178bb96e46d5\") " pod="openshift-ingress/e2e-inf-a-5beeb3-openshift-default-587f95df85-pm8t6" Jul 12 18:48:31.635365 ip-10-0-129-128 kubenswrapper[2559]: I0712 18:48:31.635297 2559 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/e444c91f-2e77-453d-b905-178bb96e46d5-istio-data\") pod \"e2e-inf-a-5beeb3-openshift-default-587f95df85-pm8t6\" (UID: \"e444c91f-2e77-453d-b905-178bb96e46d5\") " pod="openshift-ingress/e2e-inf-a-5beeb3-openshift-default-587f95df85-pm8t6" Jul 12 18:48:31.635365 ip-10-0-129-128 kubenswrapper[2559]: I0712 18:48:31.635343 2559 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/e444c91f-2e77-453d-b905-178bb96e46d5-istio-token\") pod \"e2e-inf-a-5beeb3-openshift-default-587f95df85-pm8t6\" (UID: \"e444c91f-2e77-453d-b905-178bb96e46d5\") " pod="openshift-ingress/e2e-inf-a-5beeb3-openshift-default-587f95df85-pm8t6" Jul 12 18:48:31.635475 ip-10-0-129-128 kubenswrapper[2559]: I0712 18:48:31.635370 2559 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/e444c91f-2e77-453d-b905-178bb96e46d5-istio-envoy\") pod \"e2e-inf-a-5beeb3-openshift-default-587f95df85-pm8t6\" (UID: \"e444c91f-2e77-453d-b905-178bb96e46d5\") " pod="openshift-ingress/e2e-inf-a-5beeb3-openshift-default-587f95df85-pm8t6" Jul 12 18:48:31.635475 ip-10-0-129-128 kubenswrapper[2559]: I0712 18:48:31.635417 2559 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/e444c91f-2e77-453d-b905-178bb96e46d5-credential-socket\") pod \"e2e-inf-a-5beeb3-openshift-default-587f95df85-pm8t6\" (UID: \"e444c91f-2e77-453d-b905-178bb96e46d5\") " pod="openshift-ingress/e2e-inf-a-5beeb3-openshift-default-587f95df85-pm8t6" Jul 12 18:48:31.635475 ip-10-0-129-128 kubenswrapper[2559]: I0712 18:48:31.635438 2559 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/e444c91f-2e77-453d-b905-178bb96e46d5-istio-podinfo\") pod \"e2e-inf-a-5beeb3-openshift-default-587f95df85-pm8t6\" (UID: \"e444c91f-2e77-453d-b905-178bb96e46d5\") " pod="openshift-ingress/e2e-inf-a-5beeb3-openshift-default-587f95df85-pm8t6" Jul 12 18:48:31.635475 ip-10-0-129-128 kubenswrapper[2559]: I0712 18:48:31.635457 2559 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/e444c91f-2e77-453d-b905-178bb96e46d5-istiod-ca-cert\") pod \"e2e-inf-a-5beeb3-openshift-default-587f95df85-pm8t6\" (UID: \"e444c91f-2e77-453d-b905-178bb96e46d5\") " pod="openshift-ingress/e2e-inf-a-5beeb3-openshift-default-587f95df85-pm8t6" Jul 12 18:48:31.635613 ip-10-0-129-128 kubenswrapper[2559]: I0712 18:48:31.635476 2559 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-kkwrd\" (UniqueName: \"kubernetes.io/projected/e444c91f-2e77-453d-b905-178bb96e46d5-kube-api-access-kkwrd\") pod \"e2e-inf-a-5beeb3-openshift-default-587f95df85-pm8t6\" (UID: \"e444c91f-2e77-453d-b905-178bb96e46d5\") " pod="openshift-ingress/e2e-inf-a-5beeb3-openshift-default-587f95df85-pm8t6" Jul 12 18:48:31.736320 ip-10-0-129-128 kubenswrapper[2559]: I0712 18:48:31.736287 2559 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/e444c91f-2e77-453d-b905-178bb96e46d5-credential-socket\") pod \"e2e-inf-a-5beeb3-openshift-default-587f95df85-pm8t6\" (UID: \"e444c91f-2e77-453d-b905-178bb96e46d5\") " pod="openshift-ingress/e2e-inf-a-5beeb3-openshift-default-587f95df85-pm8t6" Jul 12 18:48:31.736320 ip-10-0-129-128 kubenswrapper[2559]: I0712 18:48:31.736323 2559 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/e444c91f-2e77-453d-b905-178bb96e46d5-istio-podinfo\") pod \"e2e-inf-a-5beeb3-openshift-default-587f95df85-pm8t6\" (UID: \"e444c91f-2e77-453d-b905-178bb96e46d5\") " pod="openshift-ingress/e2e-inf-a-5beeb3-openshift-default-587f95df85-pm8t6" Jul 12 18:48:31.736585 ip-10-0-129-128 kubenswrapper[2559]: I0712 18:48:31.736344 2559 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/e444c91f-2e77-453d-b905-178bb96e46d5-istiod-ca-cert\") pod \"e2e-inf-a-5beeb3-openshift-default-587f95df85-pm8t6\" (UID: \"e444c91f-2e77-453d-b905-178bb96e46d5\") " pod="openshift-ingress/e2e-inf-a-5beeb3-openshift-default-587f95df85-pm8t6" Jul 12 18:48:31.736585 ip-10-0-129-128 kubenswrapper[2559]: I0712 18:48:31.736371 2559 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-kkwrd\" (UniqueName: \"kubernetes.io/projected/e444c91f-2e77-453d-b905-178bb96e46d5-kube-api-access-kkwrd\") pod \"e2e-inf-a-5beeb3-openshift-default-587f95df85-pm8t6\" (UID: \"e444c91f-2e77-453d-b905-178bb96e46d5\") " pod="openshift-ingress/e2e-inf-a-5beeb3-openshift-default-587f95df85-pm8t6" Jul 12 18:48:31.736585 ip-10-0-129-128 kubenswrapper[2559]: I0712 18:48:31.736417 2559 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/e444c91f-2e77-453d-b905-178bb96e46d5-workload-socket\") pod \"e2e-inf-a-5beeb3-openshift-default-587f95df85-pm8t6\" (UID: \"e444c91f-2e77-453d-b905-178bb96e46d5\") " pod="openshift-ingress/e2e-inf-a-5beeb3-openshift-default-587f95df85-pm8t6" Jul 12 18:48:31.736585 ip-10-0-129-128 kubenswrapper[2559]: I0712 18:48:31.736472 2559 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/e444c91f-2e77-453d-b905-178bb96e46d5-workload-certs\") pod \"e2e-inf-a-5beeb3-openshift-default-587f95df85-pm8t6\" (UID: \"e444c91f-2e77-453d-b905-178bb96e46d5\") " pod="openshift-ingress/e2e-inf-a-5beeb3-openshift-default-587f95df85-pm8t6" Jul 12 18:48:31.736585 ip-10-0-129-128 kubenswrapper[2559]: I0712 18:48:31.736496 2559 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/e444c91f-2e77-453d-b905-178bb96e46d5-istio-data\") pod \"e2e-inf-a-5beeb3-openshift-default-587f95df85-pm8t6\" (UID: \"e444c91f-2e77-453d-b905-178bb96e46d5\") " pod="openshift-ingress/e2e-inf-a-5beeb3-openshift-default-587f95df85-pm8t6" Jul 12 18:48:31.736585 ip-10-0-129-128 kubenswrapper[2559]: I0712 18:48:31.736545 2559 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/e444c91f-2e77-453d-b905-178bb96e46d5-istio-token\") pod \"e2e-inf-a-5beeb3-openshift-default-587f95df85-pm8t6\" (UID: \"e444c91f-2e77-453d-b905-178bb96e46d5\") " pod="openshift-ingress/e2e-inf-a-5beeb3-openshift-default-587f95df85-pm8t6" Jul 12 18:48:31.736585 ip-10-0-129-128 kubenswrapper[2559]: I0712 18:48:31.736576 2559 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/e444c91f-2e77-453d-b905-178bb96e46d5-istio-envoy\") pod \"e2e-inf-a-5beeb3-openshift-default-587f95df85-pm8t6\" (UID: \"e444c91f-2e77-453d-b905-178bb96e46d5\") " pod="openshift-ingress/e2e-inf-a-5beeb3-openshift-default-587f95df85-pm8t6" Jul 12 18:48:31.736992 ip-10-0-129-128 kubenswrapper[2559]: I0712 18:48:31.736789 2559 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/e444c91f-2e77-453d-b905-178bb96e46d5-credential-socket\") pod \"e2e-inf-a-5beeb3-openshift-default-587f95df85-pm8t6\" (UID: \"e444c91f-2e77-453d-b905-178bb96e46d5\") " pod="openshift-ingress/e2e-inf-a-5beeb3-openshift-default-587f95df85-pm8t6" Jul 12 18:48:31.736992 ip-10-0-129-128 kubenswrapper[2559]: I0712 18:48:31.736928 2559 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/e444c91f-2e77-453d-b905-178bb96e46d5-workload-socket\") pod \"e2e-inf-a-5beeb3-openshift-default-587f95df85-pm8t6\" (UID: \"e444c91f-2e77-453d-b905-178bb96e46d5\") " pod="openshift-ingress/e2e-inf-a-5beeb3-openshift-default-587f95df85-pm8t6" Jul 12 18:48:31.737212 ip-10-0-129-128 kubenswrapper[2559]: I0712 18:48:31.737193 2559 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/e444c91f-2e77-453d-b905-178bb96e46d5-istio-data\") pod \"e2e-inf-a-5beeb3-openshift-default-587f95df85-pm8t6\" (UID: \"e444c91f-2e77-453d-b905-178bb96e46d5\") " pod="openshift-ingress/e2e-inf-a-5beeb3-openshift-default-587f95df85-pm8t6" Jul 12 18:48:31.737268 ip-10-0-129-128 kubenswrapper[2559]: I0712 18:48:31.737234 2559 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/e444c91f-2e77-453d-b905-178bb96e46d5-workload-certs\") pod \"e2e-inf-a-5beeb3-openshift-default-587f95df85-pm8t6\" (UID: \"e444c91f-2e77-453d-b905-178bb96e46d5\") " pod="openshift-ingress/e2e-inf-a-5beeb3-openshift-default-587f95df85-pm8t6" Jul 12 18:48:31.737331 ip-10-0-129-128 kubenswrapper[2559]: I0712 18:48:31.737292 2559 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/e444c91f-2e77-453d-b905-178bb96e46d5-istiod-ca-cert\") pod \"e2e-inf-a-5beeb3-openshift-default-587f95df85-pm8t6\" (UID: \"e444c91f-2e77-453d-b905-178bb96e46d5\") " pod="openshift-ingress/e2e-inf-a-5beeb3-openshift-default-587f95df85-pm8t6" Jul 12 18:48:31.739040 ip-10-0-129-128 kubenswrapper[2559]: I0712 18:48:31.739019 2559 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/e444c91f-2e77-453d-b905-178bb96e46d5-istio-podinfo\") pod \"e2e-inf-a-5beeb3-openshift-default-587f95df85-pm8t6\" (UID: \"e444c91f-2e77-453d-b905-178bb96e46d5\") " pod="openshift-ingress/e2e-inf-a-5beeb3-openshift-default-587f95df85-pm8t6" Jul 12 18:48:31.739141 ip-10-0-129-128 kubenswrapper[2559]: I0712 18:48:31.739046 2559 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/e444c91f-2e77-453d-b905-178bb96e46d5-istio-envoy\") pod \"e2e-inf-a-5beeb3-openshift-default-587f95df85-pm8t6\" (UID: \"e444c91f-2e77-453d-b905-178bb96e46d5\") " pod="openshift-ingress/e2e-inf-a-5beeb3-openshift-default-587f95df85-pm8t6" Jul 12 18:48:31.744032 ip-10-0-129-128 kubenswrapper[2559]: I0712 18:48:31.744010 2559 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/e444c91f-2e77-453d-b905-178bb96e46d5-istio-token\") pod \"e2e-inf-a-5beeb3-openshift-default-587f95df85-pm8t6\" (UID: \"e444c91f-2e77-453d-b905-178bb96e46d5\") " pod="openshift-ingress/e2e-inf-a-5beeb3-openshift-default-587f95df85-pm8t6" Jul 12 18:48:31.744201 ip-10-0-129-128 kubenswrapper[2559]: I0712 18:48:31.744182 2559 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-kkwrd\" (UniqueName: \"kubernetes.io/projected/e444c91f-2e77-453d-b905-178bb96e46d5-kube-api-access-kkwrd\") pod \"e2e-inf-a-5beeb3-openshift-default-587f95df85-pm8t6\" (UID: \"e444c91f-2e77-453d-b905-178bb96e46d5\") " pod="openshift-ingress/e2e-inf-a-5beeb3-openshift-default-587f95df85-pm8t6" Jul 12 18:48:31.867204 ip-10-0-129-128 kubenswrapper[2559]: I0712 18:48:31.867111 2559 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-inf-a-5beeb3-openshift-default-587f95df85-pm8t6" Jul 12 18:48:32.001138 ip-10-0-129-128 kubenswrapper[2559]: I0712 18:48:32.001097 2559 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/e2e-inf-a-5beeb3-openshift-default-587f95df85-pm8t6"] Jul 12 18:48:32.003390 ip-10-0-129-128 kubenswrapper[2559]: W0712 18:48:32.003363 2559 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pode444c91f_2e77_453d_b905_178bb96e46d5.slice/crio-f95cc3dbc05975d7d80b2cfbcf173a1c2f830aad02ee994f6891e2bb2fe7293f WatchSource:0}: Error finding container f95cc3dbc05975d7d80b2cfbcf173a1c2f830aad02ee994f6891e2bb2fe7293f: Status 404 returned error can't find the container with id f95cc3dbc05975d7d80b2cfbcf173a1c2f830aad02ee994f6891e2bb2fe7293f Jul 12 18:48:32.005516 ip-10-0-129-128 kubenswrapper[2559]: I0712 18:48:32.005475 2559 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236228Ki","pods":"250"} Jul 12 18:48:32.005604 ip-10-0-129-128 kubenswrapper[2559]: I0712 18:48:32.005538 2559 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236228Ki","pods":"250"} Jul 12 18:48:32.005604 ip-10-0-129-128 kubenswrapper[2559]: I0712 18:48:32.005570 2559 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236228Ki","pods":"250"} Jul 12 18:48:32.107679 ip-10-0-129-128 kubenswrapper[2559]: I0712 18:48:32.107633 2559 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-inf-a-5beeb3-openshift-default-587f95df85-pm8t6" event={"ID":"e444c91f-2e77-453d-b905-178bb96e46d5","Type":"ContainerStarted","Data":"9b23a6fef8d328398755d83da48ea985a461ef395cd8f7991bd02e4657f5995b"} Jul 12 18:48:32.107816 ip-10-0-129-128 kubenswrapper[2559]: I0712 18:48:32.107709 2559 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-inf-a-5beeb3-openshift-default-587f95df85-pm8t6" event={"ID":"e444c91f-2e77-453d-b905-178bb96e46d5","Type":"ContainerStarted","Data":"f95cc3dbc05975d7d80b2cfbcf173a1c2f830aad02ee994f6891e2bb2fe7293f"} Jul 12 18:48:32.126599 ip-10-0-129-128 kubenswrapper[2559]: I0712 18:48:32.126497 2559 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-ingress/e2e-inf-a-5beeb3-openshift-default-587f95df85-pm8t6" podStartSLOduration=1.126465549 podStartE2EDuration="1.126465549s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-12 18:48:31 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-12 18:48:32.125274273 +0000 UTC m=+3319.619342203" watchObservedRunningTime="2026-07-12 18:48:32.126465549 +0000 UTC m=+3319.620533486" Jul 12 18:48:32.867464 ip-10-0-129-128 kubenswrapper[2559]: I0712 18:48:32.867430 2559 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="openshift-ingress/e2e-inf-a-5beeb3-openshift-default-587f95df85-pm8t6" Jul 12 18:48:33.873018 ip-10-0-129-128 kubenswrapper[2559]: I0712 18:48:33.872984 2559 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="openshift-ingress/e2e-inf-a-5beeb3-openshift-default-587f95df85-pm8t6" Jul 12 18:48:34.118312 ip-10-0-129-128 kubenswrapper[2559]: I0712 18:48:34.118279 2559 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/e2e-inf-a-5beeb3-openshift-default-587f95df85-pm8t6" Jul 12 18:48:34.119464 ip-10-0-129-128 kubenswrapper[2559]: I0712 18:48:34.119445 2559 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/e2e-inf-a-5beeb3-openshift-default-587f95df85-pm8t6" Jul 12 18:48:39.069092 ip-10-0-129-128 kubenswrapper[2559]: I0712 18:48:39.069006 2559 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-ingress/e2e-inf-b-d2f6a5-openshift-default-7558f8b88c-qnw6s"] Jul 12 18:48:39.072735 ip-10-0-129-128 kubenswrapper[2559]: I0712 18:48:39.072704 2559 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-inf-b-d2f6a5-openshift-default-7558f8b88c-qnw6s" Jul 12 18:48:39.075220 ip-10-0-129-128 kubenswrapper[2559]: I0712 18:48:39.075201 2559 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress\"/\"e2e-inf-b-d2f6a5-openshift-default-dockercfg-fh6kj\"" Jul 12 18:48:39.084132 ip-10-0-129-128 kubenswrapper[2559]: I0712 18:48:39.084109 2559 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/e2e-inf-b-d2f6a5-openshift-default-7558f8b88c-qnw6s"] Jul 12 18:48:39.106247 ip-10-0-129-128 kubenswrapper[2559]: I0712 18:48:39.106212 2559 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/9dba6483-775d-4b0c-b35d-48d700327eda-istiod-ca-cert\") pod \"e2e-inf-b-d2f6a5-openshift-default-7558f8b88c-qnw6s\" (UID: \"9dba6483-775d-4b0c-b35d-48d700327eda\") " pod="openshift-ingress/e2e-inf-b-d2f6a5-openshift-default-7558f8b88c-qnw6s" Jul 12 18:48:39.106375 ip-10-0-129-128 kubenswrapper[2559]: I0712 18:48:39.106257 2559 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/9dba6483-775d-4b0c-b35d-48d700327eda-istio-data\") pod \"e2e-inf-b-d2f6a5-openshift-default-7558f8b88c-qnw6s\" (UID: \"9dba6483-775d-4b0c-b35d-48d700327eda\") " pod="openshift-ingress/e2e-inf-b-d2f6a5-openshift-default-7558f8b88c-qnw6s" Jul 12 18:48:39.106375 ip-10-0-129-128 kubenswrapper[2559]: I0712 18:48:39.106276 2559 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/9dba6483-775d-4b0c-b35d-48d700327eda-istio-envoy\") pod \"e2e-inf-b-d2f6a5-openshift-default-7558f8b88c-qnw6s\" (UID: \"9dba6483-775d-4b0c-b35d-48d700327eda\") " pod="openshift-ingress/e2e-inf-b-d2f6a5-openshift-default-7558f8b88c-qnw6s" Jul 12 18:48:39.106463 ip-10-0-129-128 kubenswrapper[2559]: I0712 18:48:39.106367 2559 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/9dba6483-775d-4b0c-b35d-48d700327eda-istio-podinfo\") pod \"e2e-inf-b-d2f6a5-openshift-default-7558f8b88c-qnw6s\" (UID: \"9dba6483-775d-4b0c-b35d-48d700327eda\") " pod="openshift-ingress/e2e-inf-b-d2f6a5-openshift-default-7558f8b88c-qnw6s" Jul 12 18:48:39.106463 ip-10-0-129-128 kubenswrapper[2559]: I0712 18:48:39.106393 2559 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-b6l8n\" (UniqueName: \"kubernetes.io/projected/9dba6483-775d-4b0c-b35d-48d700327eda-kube-api-access-b6l8n\") pod \"e2e-inf-b-d2f6a5-openshift-default-7558f8b88c-qnw6s\" (UID: \"9dba6483-775d-4b0c-b35d-48d700327eda\") " pod="openshift-ingress/e2e-inf-b-d2f6a5-openshift-default-7558f8b88c-qnw6s" Jul 12 18:48:39.106568 ip-10-0-129-128 kubenswrapper[2559]: I0712 18:48:39.106480 2559 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/9dba6483-775d-4b0c-b35d-48d700327eda-workload-certs\") pod \"e2e-inf-b-d2f6a5-openshift-default-7558f8b88c-qnw6s\" (UID: \"9dba6483-775d-4b0c-b35d-48d700327eda\") " pod="openshift-ingress/e2e-inf-b-d2f6a5-openshift-default-7558f8b88c-qnw6s" Jul 12 18:48:39.106568 ip-10-0-129-128 kubenswrapper[2559]: I0712 18:48:39.106527 2559 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/9dba6483-775d-4b0c-b35d-48d700327eda-workload-socket\") pod \"e2e-inf-b-d2f6a5-openshift-default-7558f8b88c-qnw6s\" (UID: \"9dba6483-775d-4b0c-b35d-48d700327eda\") " pod="openshift-ingress/e2e-inf-b-d2f6a5-openshift-default-7558f8b88c-qnw6s" Jul 12 18:48:39.106568 ip-10-0-129-128 kubenswrapper[2559]: I0712 18:48:39.106560 2559 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/9dba6483-775d-4b0c-b35d-48d700327eda-istio-token\") pod \"e2e-inf-b-d2f6a5-openshift-default-7558f8b88c-qnw6s\" (UID: \"9dba6483-775d-4b0c-b35d-48d700327eda\") " pod="openshift-ingress/e2e-inf-b-d2f6a5-openshift-default-7558f8b88c-qnw6s" Jul 12 18:48:39.106758 ip-10-0-129-128 kubenswrapper[2559]: I0712 18:48:39.106670 2559 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/9dba6483-775d-4b0c-b35d-48d700327eda-credential-socket\") pod \"e2e-inf-b-d2f6a5-openshift-default-7558f8b88c-qnw6s\" (UID: \"9dba6483-775d-4b0c-b35d-48d700327eda\") " pod="openshift-ingress/e2e-inf-b-d2f6a5-openshift-default-7558f8b88c-qnw6s" Jul 12 18:48:39.207997 ip-10-0-129-128 kubenswrapper[2559]: I0712 18:48:39.207956 2559 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/9dba6483-775d-4b0c-b35d-48d700327eda-workload-socket\") pod \"e2e-inf-b-d2f6a5-openshift-default-7558f8b88c-qnw6s\" (UID: \"9dba6483-775d-4b0c-b35d-48d700327eda\") " pod="openshift-ingress/e2e-inf-b-d2f6a5-openshift-default-7558f8b88c-qnw6s" Jul 12 18:48:39.208169 ip-10-0-129-128 kubenswrapper[2559]: I0712 18:48:39.208019 2559 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/9dba6483-775d-4b0c-b35d-48d700327eda-istio-token\") pod \"e2e-inf-b-d2f6a5-openshift-default-7558f8b88c-qnw6s\" (UID: \"9dba6483-775d-4b0c-b35d-48d700327eda\") " pod="openshift-ingress/e2e-inf-b-d2f6a5-openshift-default-7558f8b88c-qnw6s" Jul 12 18:48:39.208169 ip-10-0-129-128 kubenswrapper[2559]: I0712 18:48:39.208064 2559 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/9dba6483-775d-4b0c-b35d-48d700327eda-credential-socket\") pod \"e2e-inf-b-d2f6a5-openshift-default-7558f8b88c-qnw6s\" (UID: \"9dba6483-775d-4b0c-b35d-48d700327eda\") " pod="openshift-ingress/e2e-inf-b-d2f6a5-openshift-default-7558f8b88c-qnw6s" Jul 12 18:48:39.208169 ip-10-0-129-128 kubenswrapper[2559]: I0712 18:48:39.208096 2559 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/9dba6483-775d-4b0c-b35d-48d700327eda-istiod-ca-cert\") pod \"e2e-inf-b-d2f6a5-openshift-default-7558f8b88c-qnw6s\" (UID: \"9dba6483-775d-4b0c-b35d-48d700327eda\") " pod="openshift-ingress/e2e-inf-b-d2f6a5-openshift-default-7558f8b88c-qnw6s" Jul 12 18:48:39.208169 ip-10-0-129-128 kubenswrapper[2559]: I0712 18:48:39.208141 2559 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/9dba6483-775d-4b0c-b35d-48d700327eda-istio-data\") pod \"e2e-inf-b-d2f6a5-openshift-default-7558f8b88c-qnw6s\" (UID: \"9dba6483-775d-4b0c-b35d-48d700327eda\") " pod="openshift-ingress/e2e-inf-b-d2f6a5-openshift-default-7558f8b88c-qnw6s" Jul 12 18:48:39.208385 ip-10-0-129-128 kubenswrapper[2559]: I0712 18:48:39.208170 2559 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/9dba6483-775d-4b0c-b35d-48d700327eda-istio-envoy\") pod \"e2e-inf-b-d2f6a5-openshift-default-7558f8b88c-qnw6s\" (UID: \"9dba6483-775d-4b0c-b35d-48d700327eda\") " pod="openshift-ingress/e2e-inf-b-d2f6a5-openshift-default-7558f8b88c-qnw6s" Jul 12 18:48:39.208385 ip-10-0-129-128 kubenswrapper[2559]: I0712 18:48:39.208227 2559 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/9dba6483-775d-4b0c-b35d-48d700327eda-istio-podinfo\") pod \"e2e-inf-b-d2f6a5-openshift-default-7558f8b88c-qnw6s\" (UID: \"9dba6483-775d-4b0c-b35d-48d700327eda\") " pod="openshift-ingress/e2e-inf-b-d2f6a5-openshift-default-7558f8b88c-qnw6s" Jul 12 18:48:39.208385 ip-10-0-129-128 kubenswrapper[2559]: I0712 18:48:39.208239 2559 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/9dba6483-775d-4b0c-b35d-48d700327eda-workload-socket\") pod \"e2e-inf-b-d2f6a5-openshift-default-7558f8b88c-qnw6s\" (UID: \"9dba6483-775d-4b0c-b35d-48d700327eda\") " pod="openshift-ingress/e2e-inf-b-d2f6a5-openshift-default-7558f8b88c-qnw6s" Jul 12 18:48:39.208385 ip-10-0-129-128 kubenswrapper[2559]: I0712 18:48:39.208255 2559 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-b6l8n\" (UniqueName: \"kubernetes.io/projected/9dba6483-775d-4b0c-b35d-48d700327eda-kube-api-access-b6l8n\") pod \"e2e-inf-b-d2f6a5-openshift-default-7558f8b88c-qnw6s\" (UID: \"9dba6483-775d-4b0c-b35d-48d700327eda\") " pod="openshift-ingress/e2e-inf-b-d2f6a5-openshift-default-7558f8b88c-qnw6s" Jul 12 18:48:39.208385 ip-10-0-129-128 kubenswrapper[2559]: I0712 18:48:39.208333 2559 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/9dba6483-775d-4b0c-b35d-48d700327eda-workload-certs\") pod \"e2e-inf-b-d2f6a5-openshift-default-7558f8b88c-qnw6s\" (UID: \"9dba6483-775d-4b0c-b35d-48d700327eda\") " pod="openshift-ingress/e2e-inf-b-d2f6a5-openshift-default-7558f8b88c-qnw6s" Jul 12 18:48:39.208385 ip-10-0-129-128 kubenswrapper[2559]: I0712 18:48:39.208375 2559 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/9dba6483-775d-4b0c-b35d-48d700327eda-credential-socket\") pod \"e2e-inf-b-d2f6a5-openshift-default-7558f8b88c-qnw6s\" (UID: \"9dba6483-775d-4b0c-b35d-48d700327eda\") " pod="openshift-ingress/e2e-inf-b-d2f6a5-openshift-default-7558f8b88c-qnw6s" Jul 12 18:48:39.208711 ip-10-0-129-128 kubenswrapper[2559]: I0712 18:48:39.208558 2559 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/9dba6483-775d-4b0c-b35d-48d700327eda-istio-data\") pod \"e2e-inf-b-d2f6a5-openshift-default-7558f8b88c-qnw6s\" (UID: \"9dba6483-775d-4b0c-b35d-48d700327eda\") " pod="openshift-ingress/e2e-inf-b-d2f6a5-openshift-default-7558f8b88c-qnw6s" Jul 12 18:48:39.208711 ip-10-0-129-128 kubenswrapper[2559]: I0712 18:48:39.208673 2559 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/9dba6483-775d-4b0c-b35d-48d700327eda-workload-certs\") pod \"e2e-inf-b-d2f6a5-openshift-default-7558f8b88c-qnw6s\" (UID: \"9dba6483-775d-4b0c-b35d-48d700327eda\") " pod="openshift-ingress/e2e-inf-b-d2f6a5-openshift-default-7558f8b88c-qnw6s" Jul 12 18:48:39.208886 ip-10-0-129-128 kubenswrapper[2559]: I0712 18:48:39.208868 2559 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/9dba6483-775d-4b0c-b35d-48d700327eda-istiod-ca-cert\") pod \"e2e-inf-b-d2f6a5-openshift-default-7558f8b88c-qnw6s\" (UID: \"9dba6483-775d-4b0c-b35d-48d700327eda\") " pod="openshift-ingress/e2e-inf-b-d2f6a5-openshift-default-7558f8b88c-qnw6s" Jul 12 18:48:39.210359 ip-10-0-129-128 kubenswrapper[2559]: I0712 18:48:39.210338 2559 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/9dba6483-775d-4b0c-b35d-48d700327eda-istio-envoy\") pod \"e2e-inf-b-d2f6a5-openshift-default-7558f8b88c-qnw6s\" (UID: \"9dba6483-775d-4b0c-b35d-48d700327eda\") " pod="openshift-ingress/e2e-inf-b-d2f6a5-openshift-default-7558f8b88c-qnw6s" Jul 12 18:48:39.210632 ip-10-0-129-128 kubenswrapper[2559]: I0712 18:48:39.210615 2559 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/9dba6483-775d-4b0c-b35d-48d700327eda-istio-podinfo\") pod \"e2e-inf-b-d2f6a5-openshift-default-7558f8b88c-qnw6s\" (UID: \"9dba6483-775d-4b0c-b35d-48d700327eda\") " pod="openshift-ingress/e2e-inf-b-d2f6a5-openshift-default-7558f8b88c-qnw6s" Jul 12 18:48:39.216206 ip-10-0-129-128 kubenswrapper[2559]: I0712 18:48:39.216179 2559 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/9dba6483-775d-4b0c-b35d-48d700327eda-istio-token\") pod \"e2e-inf-b-d2f6a5-openshift-default-7558f8b88c-qnw6s\" (UID: \"9dba6483-775d-4b0c-b35d-48d700327eda\") " pod="openshift-ingress/e2e-inf-b-d2f6a5-openshift-default-7558f8b88c-qnw6s" Jul 12 18:48:39.216381 ip-10-0-129-128 kubenswrapper[2559]: I0712 18:48:39.216362 2559 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-b6l8n\" (UniqueName: \"kubernetes.io/projected/9dba6483-775d-4b0c-b35d-48d700327eda-kube-api-access-b6l8n\") pod \"e2e-inf-b-d2f6a5-openshift-default-7558f8b88c-qnw6s\" (UID: \"9dba6483-775d-4b0c-b35d-48d700327eda\") " pod="openshift-ingress/e2e-inf-b-d2f6a5-openshift-default-7558f8b88c-qnw6s" Jul 12 18:48:39.385883 ip-10-0-129-128 kubenswrapper[2559]: I0712 18:48:39.385791 2559 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-inf-b-d2f6a5-openshift-default-7558f8b88c-qnw6s" Jul 12 18:48:39.531492 ip-10-0-129-128 kubenswrapper[2559]: I0712 18:48:39.531445 2559 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/e2e-inf-b-d2f6a5-openshift-default-7558f8b88c-qnw6s"] Jul 12 18:48:39.535494 ip-10-0-129-128 kubenswrapper[2559]: W0712 18:48:39.535460 2559 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod9dba6483_775d_4b0c_b35d_48d700327eda.slice/crio-a4abd19cfbd13296b3c7d5262a751ebe97e12762c64ac97a891791f963232b4e WatchSource:0}: Error finding container a4abd19cfbd13296b3c7d5262a751ebe97e12762c64ac97a891791f963232b4e: Status 404 returned error can't find the container with id a4abd19cfbd13296b3c7d5262a751ebe97e12762c64ac97a891791f963232b4e Jul 12 18:48:39.537724 ip-10-0-129-128 kubenswrapper[2559]: I0712 18:48:39.537692 2559 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236228Ki","pods":"250"} Jul 12 18:48:39.537816 ip-10-0-129-128 kubenswrapper[2559]: I0712 18:48:39.537752 2559 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236228Ki","pods":"250"} Jul 12 18:48:39.537816 ip-10-0-129-128 kubenswrapper[2559]: I0712 18:48:39.537778 2559 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236228Ki","pods":"250"} Jul 12 18:48:40.147950 ip-10-0-129-128 kubenswrapper[2559]: I0712 18:48:40.147907 2559 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-inf-b-d2f6a5-openshift-default-7558f8b88c-qnw6s" event={"ID":"9dba6483-775d-4b0c-b35d-48d700327eda","Type":"ContainerStarted","Data":"9b5021488d772b4da9e19bb94621ca1eba182007d44f0e354a0ae6cb8704c8d8"} Jul 12 18:48:40.148341 ip-10-0-129-128 kubenswrapper[2559]: I0712 18:48:40.147967 2559 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-inf-b-d2f6a5-openshift-default-7558f8b88c-qnw6s" event={"ID":"9dba6483-775d-4b0c-b35d-48d700327eda","Type":"ContainerStarted","Data":"a4abd19cfbd13296b3c7d5262a751ebe97e12762c64ac97a891791f963232b4e"} Jul 12 18:48:40.167624 ip-10-0-129-128 kubenswrapper[2559]: I0712 18:48:40.167577 2559 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-ingress/e2e-inf-b-d2f6a5-openshift-default-7558f8b88c-qnw6s" podStartSLOduration=1.167561495 podStartE2EDuration="1.167561495s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-12 18:48:39 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-12 18:48:40.166136949 +0000 UTC m=+3327.660204890" watchObservedRunningTime="2026-07-12 18:48:40.167561495 +0000 UTC m=+3327.661629479" Jul 12 18:48:40.386135 ip-10-0-129-128 kubenswrapper[2559]: I0712 18:48:40.386105 2559 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="openshift-ingress/e2e-inf-b-d2f6a5-openshift-default-7558f8b88c-qnw6s" Jul 12 18:48:40.391131 ip-10-0-129-128 kubenswrapper[2559]: I0712 18:48:40.391106 2559 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="openshift-ingress/e2e-inf-b-d2f6a5-openshift-default-7558f8b88c-qnw6s" Jul 12 18:48:41.152353 ip-10-0-129-128 kubenswrapper[2559]: I0712 18:48:41.152324 2559 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/e2e-inf-b-d2f6a5-openshift-default-7558f8b88c-qnw6s" Jul 12 18:48:41.153330 ip-10-0-129-128 kubenswrapper[2559]: I0712 18:48:41.153309 2559 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/e2e-inf-b-d2f6a5-openshift-default-7558f8b88c-qnw6s" Jul 12 18:48:41.485144 ip-10-0-129-128 kubenswrapper[2559]: I0712 18:48:41.485057 2559 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-pwsvh"] Jul 12 18:48:49.120033 ip-10-0-129-128 kubenswrapper[2559]: I0712 18:48:49.119996 2559 patch_prober.go:28] interesting pod/e2e-inf-a-5beeb3-openshift-default-587f95df85-pm8t6 container/istio-proxy namespace/openshift-ingress: Readiness probe status=failure output="Get \"http://10.132.0.48:15021/healthz/ready\": dial tcp 10.132.0.48:15021: connect: connection refused" start-of-body= Jul 12 18:48:49.120477 ip-10-0-129-128 kubenswrapper[2559]: I0712 18:48:49.120068 2559 prober.go:120] "Probe failed" probeType="Readiness" pod="openshift-ingress/e2e-inf-a-5beeb3-openshift-default-587f95df85-pm8t6" podUID="e444c91f-2e77-453d-b905-178bb96e46d5" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.132.0.48:15021/healthz/ready\": dial tcp 10.132.0.48:15021: connect: connection refused" Jul 12 18:48:49.196404 ip-10-0-129-128 kubenswrapper[2559]: I0712 18:48:49.196329 2559 generic.go:356] "Generic (PLEG): container finished" podID="e444c91f-2e77-453d-b905-178bb96e46d5" containerID="9b23a6fef8d328398755d83da48ea985a461ef395cd8f7991bd02e4657f5995b" exitCode=0 Jul 12 18:48:49.196404 ip-10-0-129-128 kubenswrapper[2559]: I0712 18:48:49.196393 2559 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-inf-a-5beeb3-openshift-default-587f95df85-pm8t6" event={"ID":"e444c91f-2e77-453d-b905-178bb96e46d5","Type":"ContainerDied","Data":"9b23a6fef8d328398755d83da48ea985a461ef395cd8f7991bd02e4657f5995b"} Jul 12 18:48:49.196803 ip-10-0-129-128 kubenswrapper[2559]: I0712 18:48:49.196788 2559 scope.go:117] "RemoveContainer" containerID="9b23a6fef8d328398755d83da48ea985a461ef395cd8f7991bd02e4657f5995b" Jul 12 18:48:49.197888 ip-10-0-129-128 kubenswrapper[2559]: I0712 18:48:49.197857 2559 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236228Ki","pods":"250"} Jul 12 18:48:49.197957 ip-10-0-129-128 kubenswrapper[2559]: I0712 18:48:49.197931 2559 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236228Ki","pods":"250"} Jul 12 18:48:49.197993 ip-10-0-129-128 kubenswrapper[2559]: I0712 18:48:49.197962 2559 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236228Ki","pods":"250"} Jul 12 18:48:50.203696 ip-10-0-129-128 kubenswrapper[2559]: I0712 18:48:50.203653 2559 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-inf-a-5beeb3-openshift-default-587f95df85-pm8t6" event={"ID":"e444c91f-2e77-453d-b905-178bb96e46d5","Type":"ContainerStarted","Data":"577669c1e15874cd8f723aed395e1161a40277391f621fac3839015d4f1f40f8"} Jul 12 18:48:50.867489 ip-10-0-129-128 kubenswrapper[2559]: I0712 18:48:50.867456 2559 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="openshift-ingress/e2e-inf-a-5beeb3-openshift-default-587f95df85-pm8t6" Jul 12 18:48:50.867677 ip-10-0-129-128 kubenswrapper[2559]: I0712 18:48:50.867624 2559 patch_prober.go:28] interesting pod/e2e-inf-a-5beeb3-openshift-default-587f95df85-pm8t6 container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.132.0.48:15021/healthz/ready\": dial tcp 10.132.0.48:15021: connect: connection refused" start-of-body= Jul 12 18:48:50.867749 ip-10-0-129-128 kubenswrapper[2559]: I0712 18:48:50.867703 2559 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-inf-a-5beeb3-openshift-default-587f95df85-pm8t6" podUID="e444c91f-2e77-453d-b905-178bb96e46d5" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.132.0.48:15021/healthz/ready\": dial tcp 10.132.0.48:15021: connect: connection refused" Jul 12 18:48:51.867976 ip-10-0-129-128 kubenswrapper[2559]: I0712 18:48:51.867935 2559 patch_prober.go:28] interesting pod/e2e-inf-a-5beeb3-openshift-default-587f95df85-pm8t6 container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.132.0.48:15021/healthz/ready\": dial tcp 10.132.0.48:15021: connect: connection refused" start-of-body= Jul 12 18:48:51.868554 ip-10-0-129-128 kubenswrapper[2559]: I0712 18:48:51.868522 2559 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-inf-a-5beeb3-openshift-default-587f95df85-pm8t6" podUID="e444c91f-2e77-453d-b905-178bb96e46d5" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.132.0.48:15021/healthz/ready\": dial tcp 10.132.0.48:15021: connect: connection refused" Jul 12 18:48:52.867592 ip-10-0-129-128 kubenswrapper[2559]: I0712 18:48:52.867545 2559 patch_prober.go:28] interesting pod/e2e-inf-a-5beeb3-openshift-default-587f95df85-pm8t6 container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.132.0.48:15021/healthz/ready\": dial tcp 10.132.0.48:15021: connect: connection refused" start-of-body= Jul 12 18:48:52.867835 ip-10-0-129-128 kubenswrapper[2559]: I0712 18:48:52.867673 2559 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-inf-a-5beeb3-openshift-default-587f95df85-pm8t6" podUID="e444c91f-2e77-453d-b905-178bb96e46d5" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.132.0.48:15021/healthz/ready\": dial tcp 10.132.0.48:15021: connect: connection refused" Jul 12 18:48:53.868550 ip-10-0-129-128 kubenswrapper[2559]: I0712 18:48:53.868510 2559 patch_prober.go:28] interesting pod/e2e-inf-a-5beeb3-openshift-default-587f95df85-pm8t6 container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.132.0.48:15021/healthz/ready\": dial tcp 10.132.0.48:15021: connect: connection refused" start-of-body= Jul 12 18:48:53.869551 ip-10-0-129-128 kubenswrapper[2559]: I0712 18:48:53.869145 2559 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-inf-a-5beeb3-openshift-default-587f95df85-pm8t6" podUID="e444c91f-2e77-453d-b905-178bb96e46d5" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.132.0.48:15021/healthz/ready\": dial tcp 10.132.0.48:15021: connect: connection refused" Jul 12 18:48:54.871965 ip-10-0-129-128 kubenswrapper[2559]: I0712 18:48:54.871937 2559 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="openshift-ingress/e2e-inf-a-5beeb3-openshift-default-587f95df85-pm8t6" Jul 12 18:48:54.872346 ip-10-0-129-128 kubenswrapper[2559]: I0712 18:48:54.872295 2559 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/e2e-inf-a-5beeb3-openshift-default-587f95df85-pm8t6" Jul 12 18:48:54.873133 ip-10-0-129-128 kubenswrapper[2559]: I0712 18:48:54.873115 2559 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/e2e-inf-a-5beeb3-openshift-default-587f95df85-pm8t6" Jul 12 18:48:55.986187 ip-10-0-129-128 kubenswrapper[2559]: I0712 18:48:55.986150 2559 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-pwsvh"] Jul 12 18:48:57.194821 ip-10-0-129-128 kubenswrapper[2559]: I0712 18:48:57.194777 2559 patch_prober.go:28] interesting pod/e2e-inf-b-d2f6a5-openshift-default-7558f8b88c-qnw6s container/istio-proxy namespace/openshift-ingress: Readiness probe status=failure output="Get \"http://10.132.0.49:15021/healthz/ready\": context deadline exceeded (Client.Timeout exceeded while awaiting headers)" start-of-body= Jul 12 18:48:57.195229 ip-10-0-129-128 kubenswrapper[2559]: I0712 18:48:57.194851 2559 prober.go:120] "Probe failed" probeType="Readiness" pod="openshift-ingress/e2e-inf-b-d2f6a5-openshift-default-7558f8b88c-qnw6s" podUID="9dba6483-775d-4b0c-b35d-48d700327eda" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.132.0.49:15021/healthz/ready\": context deadline exceeded (Client.Timeout exceeded while awaiting headers)" Jul 12 18:48:58.248291 ip-10-0-129-128 kubenswrapper[2559]: I0712 18:48:58.248204 2559 generic.go:356] "Generic (PLEG): container finished" podID="9dba6483-775d-4b0c-b35d-48d700327eda" containerID="9b5021488d772b4da9e19bb94621ca1eba182007d44f0e354a0ae6cb8704c8d8" exitCode=0 Jul 12 18:48:58.248808 ip-10-0-129-128 kubenswrapper[2559]: I0712 18:48:58.248282 2559 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-inf-b-d2f6a5-openshift-default-7558f8b88c-qnw6s" event={"ID":"9dba6483-775d-4b0c-b35d-48d700327eda","Type":"ContainerDied","Data":"9b5021488d772b4da9e19bb94621ca1eba182007d44f0e354a0ae6cb8704c8d8"} Jul 12 18:48:58.248808 ip-10-0-129-128 kubenswrapper[2559]: I0712 18:48:58.248795 2559 scope.go:117] "RemoveContainer" containerID="9b5021488d772b4da9e19bb94621ca1eba182007d44f0e354a0ae6cb8704c8d8" Jul 12 18:48:58.249833 ip-10-0-129-128 kubenswrapper[2559]: I0712 18:48:58.249807 2559 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236228Ki","pods":"250"} Jul 12 18:48:58.249933 ip-10-0-129-128 kubenswrapper[2559]: I0712 18:48:58.249873 2559 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236228Ki","pods":"250"} Jul 12 18:48:58.249933 ip-10-0-129-128 kubenswrapper[2559]: I0712 18:48:58.249902 2559 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236228Ki","pods":"250"} Jul 12 18:48:59.259195 ip-10-0-129-128 kubenswrapper[2559]: I0712 18:48:59.259158 2559 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-inf-b-d2f6a5-openshift-default-7558f8b88c-qnw6s" event={"ID":"9dba6483-775d-4b0c-b35d-48d700327eda","Type":"ContainerStarted","Data":"1469c666f16266ce0a1911f89d1e1b841903c8e6e14b6233f517df378964f00c"} Jul 12 18:48:59.386461 ip-10-0-129-128 kubenswrapper[2559]: I0712 18:48:59.386425 2559 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="openshift-ingress/e2e-inf-b-d2f6a5-openshift-default-7558f8b88c-qnw6s" Jul 12 18:48:59.386721 ip-10-0-129-128 kubenswrapper[2559]: I0712 18:48:59.386699 2559 patch_prober.go:28] interesting pod/e2e-inf-b-d2f6a5-openshift-default-7558f8b88c-qnw6s container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.132.0.49:15021/healthz/ready\": dial tcp 10.132.0.49:15021: connect: connection refused" start-of-body= Jul 12 18:48:59.386779 ip-10-0-129-128 kubenswrapper[2559]: I0712 18:48:59.386744 2559 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-inf-b-d2f6a5-openshift-default-7558f8b88c-qnw6s" podUID="9dba6483-775d-4b0c-b35d-48d700327eda" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.132.0.49:15021/healthz/ready\": dial tcp 10.132.0.49:15021: connect: connection refused" Jul 12 18:49:00.387247 ip-10-0-129-128 kubenswrapper[2559]: I0712 18:49:00.387205 2559 patch_prober.go:28] interesting pod/e2e-inf-b-d2f6a5-openshift-default-7558f8b88c-qnw6s container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.132.0.49:15021/healthz/ready\": dial tcp 10.132.0.49:15021: connect: connection refused" start-of-body= Jul 12 18:49:00.387713 ip-10-0-129-128 kubenswrapper[2559]: I0712 18:49:00.387266 2559 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-inf-b-d2f6a5-openshift-default-7558f8b88c-qnw6s" podUID="9dba6483-775d-4b0c-b35d-48d700327eda" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.132.0.49:15021/healthz/ready\": dial tcp 10.132.0.49:15021: connect: connection refused" Jul 12 18:49:01.386612 ip-10-0-129-128 kubenswrapper[2559]: I0712 18:49:01.386573 2559 patch_prober.go:28] interesting pod/e2e-inf-b-d2f6a5-openshift-default-7558f8b88c-qnw6s container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.132.0.49:15021/healthz/ready\": dial tcp 10.132.0.49:15021: connect: connection refused" start-of-body= Jul 12 18:49:01.386821 ip-10-0-129-128 kubenswrapper[2559]: I0712 18:49:01.386627 2559 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-inf-b-d2f6a5-openshift-default-7558f8b88c-qnw6s" podUID="9dba6483-775d-4b0c-b35d-48d700327eda" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.132.0.49:15021/healthz/ready\": dial tcp 10.132.0.49:15021: connect: connection refused" Jul 12 18:49:02.386289 ip-10-0-129-128 kubenswrapper[2559]: I0712 18:49:02.386253 2559 patch_prober.go:28] interesting pod/e2e-inf-b-d2f6a5-openshift-default-7558f8b88c-qnw6s container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.132.0.49:15021/healthz/ready\": dial tcp 10.132.0.49:15021: connect: connection refused" start-of-body= Jul 12 18:49:02.386740 ip-10-0-129-128 kubenswrapper[2559]: I0712 18:49:02.386309 2559 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-inf-b-d2f6a5-openshift-default-7558f8b88c-qnw6s" podUID="9dba6483-775d-4b0c-b35d-48d700327eda" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.132.0.49:15021/healthz/ready\": dial tcp 10.132.0.49:15021: connect: connection refused" Jul 12 18:49:03.389833 ip-10-0-129-128 kubenswrapper[2559]: I0712 18:49:03.389803 2559 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="openshift-ingress/e2e-inf-b-d2f6a5-openshift-default-7558f8b88c-qnw6s" Jul 12 18:49:03.390321 ip-10-0-129-128 kubenswrapper[2559]: I0712 18:49:03.390158 2559 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/e2e-inf-b-d2f6a5-openshift-default-7558f8b88c-qnw6s" Jul 12 18:49:03.390892 ip-10-0-129-128 kubenswrapper[2559]: I0712 18:49:03.390875 2559 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/e2e-inf-b-d2f6a5-openshift-default-7558f8b88c-qnw6s" Jul 12 18:49:11.290916 ip-10-0-129-128 kubenswrapper[2559]: I0712 18:49:11.290885 2559 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-pwsvh"] Jul 12 18:49:27.992315 ip-10-0-129-128 kubenswrapper[2559]: I0712 18:49:27.992276 2559 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-pwsvh"] Jul 12 18:49:52.783484 ip-10-0-129-128 kubenswrapper[2559]: I0712 18:49:52.783450 2559 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-pwsvh"] Jul 12 18:52:55.591077 ip-10-0-129-128 kubenswrapper[2559]: I0712 18:52:55.591046 2559 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-pwsvh"] Jul 12 18:52:57.816115 ip-10-0-129-128 kubenswrapper[2559]: I0712 18:52:57.816081 2559 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-ingress/e2e-inf-a-5beeb3-openshift-default-587f95df85-pm8t6"] Jul 12 18:52:57.816532 ip-10-0-129-128 kubenswrapper[2559]: I0712 18:52:57.816323 2559 kuberuntime_container.go:873] "Killing container with a grace period" pod="openshift-ingress/e2e-inf-a-5beeb3-openshift-default-587f95df85-pm8t6" podUID="e444c91f-2e77-453d-b905-178bb96e46d5" containerName="istio-proxy" containerID="cri-o://577669c1e15874cd8f723aed395e1161a40277391f621fac3839015d4f1f40f8" gracePeriod=30 Jul 12 18:53:02.849079 ip-10-0-129-128 kubenswrapper[2559]: E0712 18:53:02.849049 2559 cadvisor_stats_provider.go:567] "Partial failure issuing cadvisor.ContainerInfoV2" err="partial failures: [\"/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pode444c91f_2e77_453d_b905_178bb96e46d5.slice/crio-577669c1e15874cd8f723aed395e1161a40277391f621fac3839015d4f1f40f8.scope\": RecentStats: unable to find data in memory cache]" Jul 12 18:53:03.072022 ip-10-0-129-128 kubenswrapper[2559]: I0712 18:53:03.071998 2559 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-inf-a-5beeb3-openshift-default-587f95df85-pm8t6" Jul 12 18:53:03.125108 ip-10-0-129-128 kubenswrapper[2559]: I0712 18:53:03.125020 2559 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/e444c91f-2e77-453d-b905-178bb96e46d5-workload-socket\") pod \"e444c91f-2e77-453d-b905-178bb96e46d5\" (UID: \"e444c91f-2e77-453d-b905-178bb96e46d5\") " Jul 12 18:53:03.125108 ip-10-0-129-128 kubenswrapper[2559]: I0712 18:53:03.125054 2559 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/e444c91f-2e77-453d-b905-178bb96e46d5-istiod-ca-cert\") pod \"e444c91f-2e77-453d-b905-178bb96e46d5\" (UID: \"e444c91f-2e77-453d-b905-178bb96e46d5\") " Jul 12 18:53:03.125108 ip-10-0-129-128 kubenswrapper[2559]: I0712 18:53:03.125084 2559 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/e444c91f-2e77-453d-b905-178bb96e46d5-workload-certs\") pod \"e444c91f-2e77-453d-b905-178bb96e46d5\" (UID: \"e444c91f-2e77-453d-b905-178bb96e46d5\") " Jul 12 18:53:03.125108 ip-10-0-129-128 kubenswrapper[2559]: I0712 18:53:03.125105 2559 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/e444c91f-2e77-453d-b905-178bb96e46d5-istio-envoy\") pod \"e444c91f-2e77-453d-b905-178bb96e46d5\" (UID: \"e444c91f-2e77-453d-b905-178bb96e46d5\") " Jul 12 18:53:03.125448 ip-10-0-129-128 kubenswrapper[2559]: I0712 18:53:03.125149 2559 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/e444c91f-2e77-453d-b905-178bb96e46d5-istio-data\") pod \"e444c91f-2e77-453d-b905-178bb96e46d5\" (UID: \"e444c91f-2e77-453d-b905-178bb96e46d5\") " Jul 12 18:53:03.125448 ip-10-0-129-128 kubenswrapper[2559]: I0712 18:53:03.125198 2559 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-kkwrd\" (UniqueName: \"kubernetes.io/projected/e444c91f-2e77-453d-b905-178bb96e46d5-kube-api-access-kkwrd\") pod \"e444c91f-2e77-453d-b905-178bb96e46d5\" (UID: \"e444c91f-2e77-453d-b905-178bb96e46d5\") " Jul 12 18:53:03.125448 ip-10-0-129-128 kubenswrapper[2559]: I0712 18:53:03.125243 2559 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/e444c91f-2e77-453d-b905-178bb96e46d5-istio-token\") pod \"e444c91f-2e77-453d-b905-178bb96e46d5\" (UID: \"e444c91f-2e77-453d-b905-178bb96e46d5\") " Jul 12 18:53:03.125448 ip-10-0-129-128 kubenswrapper[2559]: I0712 18:53:03.125272 2559 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/e444c91f-2e77-453d-b905-178bb96e46d5-credential-socket\") pod \"e444c91f-2e77-453d-b905-178bb96e46d5\" (UID: \"e444c91f-2e77-453d-b905-178bb96e46d5\") " Jul 12 18:53:03.125448 ip-10-0-129-128 kubenswrapper[2559]: I0712 18:53:03.125299 2559 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/e444c91f-2e77-453d-b905-178bb96e46d5-istio-podinfo\") pod \"e444c91f-2e77-453d-b905-178bb96e46d5\" (UID: \"e444c91f-2e77-453d-b905-178bb96e46d5\") " Jul 12 18:53:03.125448 ip-10-0-129-128 kubenswrapper[2559]: I0712 18:53:03.125310 2559 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/e444c91f-2e77-453d-b905-178bb96e46d5-workload-socket" (OuterVolumeSpecName: "workload-socket") pod "e444c91f-2e77-453d-b905-178bb96e46d5" (UID: "e444c91f-2e77-453d-b905-178bb96e46d5"). InnerVolumeSpecName "workload-socket". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Jul 12 18:53:03.125448 ip-10-0-129-128 kubenswrapper[2559]: I0712 18:53:03.125428 2559 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/e444c91f-2e77-453d-b905-178bb96e46d5-workload-certs" (OuterVolumeSpecName: "workload-certs") pod "e444c91f-2e77-453d-b905-178bb96e46d5" (UID: "e444c91f-2e77-453d-b905-178bb96e46d5"). InnerVolumeSpecName "workload-certs". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Jul 12 18:53:03.125809 ip-10-0-129-128 kubenswrapper[2559]: I0712 18:53:03.125593 2559 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/e444c91f-2e77-453d-b905-178bb96e46d5-credential-socket" (OuterVolumeSpecName: "credential-socket") pod "e444c91f-2e77-453d-b905-178bb96e46d5" (UID: "e444c91f-2e77-453d-b905-178bb96e46d5"). InnerVolumeSpecName "credential-socket". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Jul 12 18:53:03.125809 ip-10-0-129-128 kubenswrapper[2559]: I0712 18:53:03.125504 2559 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/e444c91f-2e77-453d-b905-178bb96e46d5-istiod-ca-cert" (OuterVolumeSpecName: "istiod-ca-cert") pod "e444c91f-2e77-453d-b905-178bb96e46d5" (UID: "e444c91f-2e77-453d-b905-178bb96e46d5"). InnerVolumeSpecName "istiod-ca-cert". PluginName "kubernetes.io/configmap", VolumeGIDValue "" Jul 12 18:53:03.125809 ip-10-0-129-128 kubenswrapper[2559]: I0712 18:53:03.125668 2559 reconciler_common.go:299] "Volume detached for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/e444c91f-2e77-453d-b905-178bb96e46d5-workload-socket\") on node \"ip-10-0-129-128.ec2.internal\" DevicePath \"\"" Jul 12 18:53:03.125809 ip-10-0-129-128 kubenswrapper[2559]: I0712 18:53:03.125690 2559 reconciler_common.go:299] "Volume detached for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/e444c91f-2e77-453d-b905-178bb96e46d5-workload-certs\") on node \"ip-10-0-129-128.ec2.internal\" DevicePath \"\"" Jul 12 18:53:03.126203 ip-10-0-129-128 kubenswrapper[2559]: I0712 18:53:03.126178 2559 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/e444c91f-2e77-453d-b905-178bb96e46d5-istio-data" (OuterVolumeSpecName: "istio-data") pod "e444c91f-2e77-453d-b905-178bb96e46d5" (UID: "e444c91f-2e77-453d-b905-178bb96e46d5"). InnerVolumeSpecName "istio-data". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Jul 12 18:53:03.127611 ip-10-0-129-128 kubenswrapper[2559]: I0712 18:53:03.127583 2559 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/e444c91f-2e77-453d-b905-178bb96e46d5-kube-api-access-kkwrd" (OuterVolumeSpecName: "kube-api-access-kkwrd") pod "e444c91f-2e77-453d-b905-178bb96e46d5" (UID: "e444c91f-2e77-453d-b905-178bb96e46d5"). InnerVolumeSpecName "kube-api-access-kkwrd". PluginName "kubernetes.io/projected", VolumeGIDValue "" Jul 12 18:53:03.127726 ip-10-0-129-128 kubenswrapper[2559]: I0712 18:53:03.127586 2559 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/e444c91f-2e77-453d-b905-178bb96e46d5-istio-envoy" (OuterVolumeSpecName: "istio-envoy") pod "e444c91f-2e77-453d-b905-178bb96e46d5" (UID: "e444c91f-2e77-453d-b905-178bb96e46d5"). InnerVolumeSpecName "istio-envoy". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Jul 12 18:53:03.127934 ip-10-0-129-128 kubenswrapper[2559]: I0712 18:53:03.127903 2559 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/downward-api/e444c91f-2e77-453d-b905-178bb96e46d5-istio-podinfo" (OuterVolumeSpecName: "istio-podinfo") pod "e444c91f-2e77-453d-b905-178bb96e46d5" (UID: "e444c91f-2e77-453d-b905-178bb96e46d5"). InnerVolumeSpecName "istio-podinfo". PluginName "kubernetes.io/downward-api", VolumeGIDValue "" Jul 12 18:53:03.127934 ip-10-0-129-128 kubenswrapper[2559]: I0712 18:53:03.127915 2559 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/e444c91f-2e77-453d-b905-178bb96e46d5-istio-token" (OuterVolumeSpecName: "istio-token") pod "e444c91f-2e77-453d-b905-178bb96e46d5" (UID: "e444c91f-2e77-453d-b905-178bb96e46d5"). InnerVolumeSpecName "istio-token". PluginName "kubernetes.io/projected", VolumeGIDValue "" Jul 12 18:53:03.226559 ip-10-0-129-128 kubenswrapper[2559]: I0712 18:53:03.226516 2559 reconciler_common.go:299] "Volume detached for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/e444c91f-2e77-453d-b905-178bb96e46d5-istio-envoy\") on node \"ip-10-0-129-128.ec2.internal\" DevicePath \"\"" Jul 12 18:53:03.226559 ip-10-0-129-128 kubenswrapper[2559]: I0712 18:53:03.226551 2559 reconciler_common.go:299] "Volume detached for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/e444c91f-2e77-453d-b905-178bb96e46d5-istio-data\") on node \"ip-10-0-129-128.ec2.internal\" DevicePath \"\"" Jul 12 18:53:03.226559 ip-10-0-129-128 kubenswrapper[2559]: I0712 18:53:03.226562 2559 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-kkwrd\" (UniqueName: \"kubernetes.io/projected/e444c91f-2e77-453d-b905-178bb96e46d5-kube-api-access-kkwrd\") on node \"ip-10-0-129-128.ec2.internal\" DevicePath \"\"" Jul 12 18:53:03.226883 ip-10-0-129-128 kubenswrapper[2559]: I0712 18:53:03.226572 2559 reconciler_common.go:299] "Volume detached for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/e444c91f-2e77-453d-b905-178bb96e46d5-istio-token\") on node \"ip-10-0-129-128.ec2.internal\" DevicePath \"\"" Jul 12 18:53:03.226883 ip-10-0-129-128 kubenswrapper[2559]: I0712 18:53:03.226581 2559 reconciler_common.go:299] "Volume detached for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/e444c91f-2e77-453d-b905-178bb96e46d5-credential-socket\") on node \"ip-10-0-129-128.ec2.internal\" DevicePath \"\"" Jul 12 18:53:03.226883 ip-10-0-129-128 kubenswrapper[2559]: I0712 18:53:03.226589 2559 reconciler_common.go:299] "Volume detached for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/e444c91f-2e77-453d-b905-178bb96e46d5-istio-podinfo\") on node \"ip-10-0-129-128.ec2.internal\" DevicePath \"\"" Jul 12 18:53:03.226883 ip-10-0-129-128 kubenswrapper[2559]: I0712 18:53:03.226597 2559 reconciler_common.go:299] "Volume detached for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/e444c91f-2e77-453d-b905-178bb96e46d5-istiod-ca-cert\") on node \"ip-10-0-129-128.ec2.internal\" DevicePath \"\"" Jul 12 18:53:03.446511 ip-10-0-129-128 kubenswrapper[2559]: I0712 18:53:03.446416 2559 generic.go:356] "Generic (PLEG): container finished" podID="e444c91f-2e77-453d-b905-178bb96e46d5" containerID="577669c1e15874cd8f723aed395e1161a40277391f621fac3839015d4f1f40f8" exitCode=0 Jul 12 18:53:03.446511 ip-10-0-129-128 kubenswrapper[2559]: I0712 18:53:03.446462 2559 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-inf-a-5beeb3-openshift-default-587f95df85-pm8t6" Jul 12 18:53:03.446511 ip-10-0-129-128 kubenswrapper[2559]: I0712 18:53:03.446486 2559 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-inf-a-5beeb3-openshift-default-587f95df85-pm8t6" event={"ID":"e444c91f-2e77-453d-b905-178bb96e46d5","Type":"ContainerDied","Data":"577669c1e15874cd8f723aed395e1161a40277391f621fac3839015d4f1f40f8"} Jul 12 18:53:03.446795 ip-10-0-129-128 kubenswrapper[2559]: I0712 18:53:03.446538 2559 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-inf-a-5beeb3-openshift-default-587f95df85-pm8t6" event={"ID":"e444c91f-2e77-453d-b905-178bb96e46d5","Type":"ContainerDied","Data":"f95cc3dbc05975d7d80b2cfbcf173a1c2f830aad02ee994f6891e2bb2fe7293f"} Jul 12 18:53:03.446795 ip-10-0-129-128 kubenswrapper[2559]: I0712 18:53:03.446560 2559 scope.go:117] "RemoveContainer" containerID="577669c1e15874cd8f723aed395e1161a40277391f621fac3839015d4f1f40f8" Jul 12 18:53:03.456681 ip-10-0-129-128 kubenswrapper[2559]: I0712 18:53:03.456658 2559 scope.go:117] "RemoveContainer" containerID="9b23a6fef8d328398755d83da48ea985a461ef395cd8f7991bd02e4657f5995b" Jul 12 18:53:03.464854 ip-10-0-129-128 kubenswrapper[2559]: I0712 18:53:03.464839 2559 scope.go:117] "RemoveContainer" containerID="577669c1e15874cd8f723aed395e1161a40277391f621fac3839015d4f1f40f8" Jul 12 18:53:03.465100 ip-10-0-129-128 kubenswrapper[2559]: E0712 18:53:03.465082 2559 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"577669c1e15874cd8f723aed395e1161a40277391f621fac3839015d4f1f40f8\": container with ID starting with 577669c1e15874cd8f723aed395e1161a40277391f621fac3839015d4f1f40f8 not found: ID does not exist" containerID="577669c1e15874cd8f723aed395e1161a40277391f621fac3839015d4f1f40f8" Jul 12 18:53:03.465138 ip-10-0-129-128 kubenswrapper[2559]: I0712 18:53:03.465108 2559 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"577669c1e15874cd8f723aed395e1161a40277391f621fac3839015d4f1f40f8"} err="failed to get container status \"577669c1e15874cd8f723aed395e1161a40277391f621fac3839015d4f1f40f8\": rpc error: code = NotFound desc = could not find container \"577669c1e15874cd8f723aed395e1161a40277391f621fac3839015d4f1f40f8\": container with ID starting with 577669c1e15874cd8f723aed395e1161a40277391f621fac3839015d4f1f40f8 not found: ID does not exist" Jul 12 18:53:03.465138 ip-10-0-129-128 kubenswrapper[2559]: I0712 18:53:03.465125 2559 scope.go:117] "RemoveContainer" containerID="9b23a6fef8d328398755d83da48ea985a461ef395cd8f7991bd02e4657f5995b" Jul 12 18:53:03.465366 ip-10-0-129-128 kubenswrapper[2559]: E0712 18:53:03.465348 2559 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"9b23a6fef8d328398755d83da48ea985a461ef395cd8f7991bd02e4657f5995b\": container with ID starting with 9b23a6fef8d328398755d83da48ea985a461ef395cd8f7991bd02e4657f5995b not found: ID does not exist" containerID="9b23a6fef8d328398755d83da48ea985a461ef395cd8f7991bd02e4657f5995b" Jul 12 18:53:03.465402 ip-10-0-129-128 kubenswrapper[2559]: I0712 18:53:03.465373 2559 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"9b23a6fef8d328398755d83da48ea985a461ef395cd8f7991bd02e4657f5995b"} err="failed to get container status \"9b23a6fef8d328398755d83da48ea985a461ef395cd8f7991bd02e4657f5995b\": rpc error: code = NotFound desc = could not find container \"9b23a6fef8d328398755d83da48ea985a461ef395cd8f7991bd02e4657f5995b\": container with ID starting with 9b23a6fef8d328398755d83da48ea985a461ef395cd8f7991bd02e4657f5995b not found: ID does not exist" Jul 12 18:53:03.475668 ip-10-0-129-128 kubenswrapper[2559]: I0712 18:53:03.474315 2559 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-ingress/e2e-inf-a-5beeb3-openshift-default-587f95df85-pm8t6"] Jul 12 18:53:03.480081 ip-10-0-129-128 kubenswrapper[2559]: I0712 18:53:03.480057 2559 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["openshift-ingress/e2e-inf-a-5beeb3-openshift-default-587f95df85-pm8t6"] Jul 12 18:53:05.035245 ip-10-0-129-128 kubenswrapper[2559]: I0712 18:53:05.035211 2559 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="e444c91f-2e77-453d-b905-178bb96e46d5" path="/var/lib/kubelet/pods/e444c91f-2e77-453d-b905-178bb96e46d5/volumes" Jul 12 18:53:13.243476 ip-10-0-129-128 kubenswrapper[2559]: I0712 18:53:13.243448 2559 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-console-operator_console-operator-7866497494-wlk78_f1d46430-bb60-4470-b8d8-eacd44f4f6b1/console-operator/1.log" Jul 12 18:53:13.245934 ip-10-0-129-128 kubenswrapper[2559]: I0712 18:53:13.245909 2559 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ovn-kubernetes_ovnkube-node-q5dw8_e51bad50-bc74-4e38-8307-ca257b599314/ovn-acl-logging/0.log" Jul 12 18:53:13.252906 ip-10-0-129-128 kubenswrapper[2559]: I0712 18:53:13.252886 2559 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-console-operator_console-operator-7866497494-wlk78_f1d46430-bb60-4470-b8d8-eacd44f4f6b1/console-operator/1.log" Jul 12 18:53:13.255288 ip-10-0-129-128 kubenswrapper[2559]: I0712 18:53:13.255270 2559 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ovn-kubernetes_ovnkube-node-q5dw8_e51bad50-bc74-4e38-8307-ca257b599314/ovn-acl-logging/0.log" Jul 12 18:56:07.409619 ip-10-0-129-128 kubenswrapper[2559]: I0712 18:56:07.409540 2559 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-pwsvh"] Jul 12 18:56:10.405735 ip-10-0-129-128 kubenswrapper[2559]: I0712 18:56:10.405695 2559 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-ingress/e2e-inf-b-d2f6a5-openshift-default-7558f8b88c-qnw6s"] Jul 12 18:56:10.406172 ip-10-0-129-128 kubenswrapper[2559]: I0712 18:56:10.405932 2559 kuberuntime_container.go:873] "Killing container with a grace period" pod="openshift-ingress/e2e-inf-b-d2f6a5-openshift-default-7558f8b88c-qnw6s" podUID="9dba6483-775d-4b0c-b35d-48d700327eda" containerName="istio-proxy" containerID="cri-o://1469c666f16266ce0a1911f89d1e1b841903c8e6e14b6233f517df378964f00c" gracePeriod=30 Jul 12 18:56:11.272862 ip-10-0-129-128 kubenswrapper[2559]: I0712 18:56:11.272829 2559 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-ingress/e2e-api-a-045df0-openshift-default-cfcdff7b5-x8p7w"] Jul 12 18:56:11.273207 ip-10-0-129-128 kubenswrapper[2559]: I0712 18:56:11.273195 2559 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="e444c91f-2e77-453d-b905-178bb96e46d5" containerName="istio-proxy" Jul 12 18:56:11.273258 ip-10-0-129-128 kubenswrapper[2559]: I0712 18:56:11.273208 2559 state_mem.go:107] "Deleted CPUSet assignment" podUID="e444c91f-2e77-453d-b905-178bb96e46d5" containerName="istio-proxy" Jul 12 18:56:11.273258 ip-10-0-129-128 kubenswrapper[2559]: I0712 18:56:11.273217 2559 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="e444c91f-2e77-453d-b905-178bb96e46d5" containerName="istio-proxy" Jul 12 18:56:11.273258 ip-10-0-129-128 kubenswrapper[2559]: I0712 18:56:11.273223 2559 state_mem.go:107] "Deleted CPUSet assignment" podUID="e444c91f-2e77-453d-b905-178bb96e46d5" containerName="istio-proxy" Jul 12 18:56:11.273355 ip-10-0-129-128 kubenswrapper[2559]: I0712 18:56:11.273283 2559 memory_manager.go:368] "RemoveStaleState removing state" podUID="e444c91f-2e77-453d-b905-178bb96e46d5" containerName="istio-proxy" Jul 12 18:56:11.273355 ip-10-0-129-128 kubenswrapper[2559]: I0712 18:56:11.273292 2559 memory_manager.go:368] "RemoveStaleState removing state" podUID="e444c91f-2e77-453d-b905-178bb96e46d5" containerName="istio-proxy" Jul 12 18:56:11.276144 ip-10-0-129-128 kubenswrapper[2559]: I0712 18:56:11.276127 2559 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-api-a-045df0-openshift-default-cfcdff7b5-x8p7w" Jul 12 18:56:11.278551 ip-10-0-129-128 kubenswrapper[2559]: I0712 18:56:11.278526 2559 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress\"/\"e2e-api-a-045df0-openshift-default-dockercfg-xgmkv\"" Jul 12 18:56:11.286284 ip-10-0-129-128 kubenswrapper[2559]: I0712 18:56:11.286259 2559 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/e2e-api-a-045df0-openshift-default-cfcdff7b5-x8p7w"] Jul 12 18:56:11.385574 ip-10-0-129-128 kubenswrapper[2559]: I0712 18:56:11.385536 2559 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/6572fd4a-74f5-4571-acf1-5d69bebd6773-istio-token\") pod \"e2e-api-a-045df0-openshift-default-cfcdff7b5-x8p7w\" (UID: \"6572fd4a-74f5-4571-acf1-5d69bebd6773\") " pod="openshift-ingress/e2e-api-a-045df0-openshift-default-cfcdff7b5-x8p7w" Jul 12 18:56:11.385792 ip-10-0-129-128 kubenswrapper[2559]: I0712 18:56:11.385574 2559 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/6572fd4a-74f5-4571-acf1-5d69bebd6773-istiod-ca-cert\") pod \"e2e-api-a-045df0-openshift-default-cfcdff7b5-x8p7w\" (UID: \"6572fd4a-74f5-4571-acf1-5d69bebd6773\") " pod="openshift-ingress/e2e-api-a-045df0-openshift-default-cfcdff7b5-x8p7w" Jul 12 18:56:11.385792 ip-10-0-129-128 kubenswrapper[2559]: I0712 18:56:11.385613 2559 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/6572fd4a-74f5-4571-acf1-5d69bebd6773-istio-podinfo\") pod \"e2e-api-a-045df0-openshift-default-cfcdff7b5-x8p7w\" (UID: \"6572fd4a-74f5-4571-acf1-5d69bebd6773\") " pod="openshift-ingress/e2e-api-a-045df0-openshift-default-cfcdff7b5-x8p7w" Jul 12 18:56:11.385792 ip-10-0-129-128 kubenswrapper[2559]: I0712 18:56:11.385690 2559 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/6572fd4a-74f5-4571-acf1-5d69bebd6773-istio-data\") pod \"e2e-api-a-045df0-openshift-default-cfcdff7b5-x8p7w\" (UID: \"6572fd4a-74f5-4571-acf1-5d69bebd6773\") " pod="openshift-ingress/e2e-api-a-045df0-openshift-default-cfcdff7b5-x8p7w" Jul 12 18:56:11.385792 ip-10-0-129-128 kubenswrapper[2559]: I0712 18:56:11.385723 2559 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/6572fd4a-74f5-4571-acf1-5d69bebd6773-credential-socket\") pod \"e2e-api-a-045df0-openshift-default-cfcdff7b5-x8p7w\" (UID: \"6572fd4a-74f5-4571-acf1-5d69bebd6773\") " pod="openshift-ingress/e2e-api-a-045df0-openshift-default-cfcdff7b5-x8p7w" Jul 12 18:56:11.385792 ip-10-0-129-128 kubenswrapper[2559]: I0712 18:56:11.385748 2559 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-wtnnd\" (UniqueName: \"kubernetes.io/projected/6572fd4a-74f5-4571-acf1-5d69bebd6773-kube-api-access-wtnnd\") pod \"e2e-api-a-045df0-openshift-default-cfcdff7b5-x8p7w\" (UID: \"6572fd4a-74f5-4571-acf1-5d69bebd6773\") " pod="openshift-ingress/e2e-api-a-045df0-openshift-default-cfcdff7b5-x8p7w" Jul 12 18:56:11.385792 ip-10-0-129-128 kubenswrapper[2559]: I0712 18:56:11.385782 2559 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/6572fd4a-74f5-4571-acf1-5d69bebd6773-workload-socket\") pod \"e2e-api-a-045df0-openshift-default-cfcdff7b5-x8p7w\" (UID: \"6572fd4a-74f5-4571-acf1-5d69bebd6773\") " pod="openshift-ingress/e2e-api-a-045df0-openshift-default-cfcdff7b5-x8p7w" Jul 12 18:56:11.386066 ip-10-0-129-128 kubenswrapper[2559]: I0712 18:56:11.385832 2559 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/6572fd4a-74f5-4571-acf1-5d69bebd6773-istio-envoy\") pod \"e2e-api-a-045df0-openshift-default-cfcdff7b5-x8p7w\" (UID: \"6572fd4a-74f5-4571-acf1-5d69bebd6773\") " pod="openshift-ingress/e2e-api-a-045df0-openshift-default-cfcdff7b5-x8p7w" Jul 12 18:56:11.386066 ip-10-0-129-128 kubenswrapper[2559]: I0712 18:56:11.385886 2559 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/6572fd4a-74f5-4571-acf1-5d69bebd6773-workload-certs\") pod \"e2e-api-a-045df0-openshift-default-cfcdff7b5-x8p7w\" (UID: \"6572fd4a-74f5-4571-acf1-5d69bebd6773\") " pod="openshift-ingress/e2e-api-a-045df0-openshift-default-cfcdff7b5-x8p7w" Jul 12 18:56:11.486667 ip-10-0-129-128 kubenswrapper[2559]: I0712 18:56:11.486608 2559 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/6572fd4a-74f5-4571-acf1-5d69bebd6773-istio-token\") pod \"e2e-api-a-045df0-openshift-default-cfcdff7b5-x8p7w\" (UID: \"6572fd4a-74f5-4571-acf1-5d69bebd6773\") " pod="openshift-ingress/e2e-api-a-045df0-openshift-default-cfcdff7b5-x8p7w" Jul 12 18:56:11.487146 ip-10-0-129-128 kubenswrapper[2559]: I0712 18:56:11.486678 2559 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/6572fd4a-74f5-4571-acf1-5d69bebd6773-istiod-ca-cert\") pod \"e2e-api-a-045df0-openshift-default-cfcdff7b5-x8p7w\" (UID: \"6572fd4a-74f5-4571-acf1-5d69bebd6773\") " pod="openshift-ingress/e2e-api-a-045df0-openshift-default-cfcdff7b5-x8p7w" Jul 12 18:56:11.487146 ip-10-0-129-128 kubenswrapper[2559]: I0712 18:56:11.486709 2559 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/6572fd4a-74f5-4571-acf1-5d69bebd6773-istio-podinfo\") pod \"e2e-api-a-045df0-openshift-default-cfcdff7b5-x8p7w\" (UID: \"6572fd4a-74f5-4571-acf1-5d69bebd6773\") " pod="openshift-ingress/e2e-api-a-045df0-openshift-default-cfcdff7b5-x8p7w" Jul 12 18:56:11.487146 ip-10-0-129-128 kubenswrapper[2559]: I0712 18:56:11.486767 2559 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/6572fd4a-74f5-4571-acf1-5d69bebd6773-istio-data\") pod \"e2e-api-a-045df0-openshift-default-cfcdff7b5-x8p7w\" (UID: \"6572fd4a-74f5-4571-acf1-5d69bebd6773\") " pod="openshift-ingress/e2e-api-a-045df0-openshift-default-cfcdff7b5-x8p7w" Jul 12 18:56:11.487146 ip-10-0-129-128 kubenswrapper[2559]: I0712 18:56:11.486809 2559 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/6572fd4a-74f5-4571-acf1-5d69bebd6773-credential-socket\") pod \"e2e-api-a-045df0-openshift-default-cfcdff7b5-x8p7w\" (UID: \"6572fd4a-74f5-4571-acf1-5d69bebd6773\") " pod="openshift-ingress/e2e-api-a-045df0-openshift-default-cfcdff7b5-x8p7w" Jul 12 18:56:11.487146 ip-10-0-129-128 kubenswrapper[2559]: I0712 18:56:11.486836 2559 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-wtnnd\" (UniqueName: \"kubernetes.io/projected/6572fd4a-74f5-4571-acf1-5d69bebd6773-kube-api-access-wtnnd\") pod \"e2e-api-a-045df0-openshift-default-cfcdff7b5-x8p7w\" (UID: \"6572fd4a-74f5-4571-acf1-5d69bebd6773\") " pod="openshift-ingress/e2e-api-a-045df0-openshift-default-cfcdff7b5-x8p7w" Jul 12 18:56:11.487146 ip-10-0-129-128 kubenswrapper[2559]: I0712 18:56:11.486873 2559 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/6572fd4a-74f5-4571-acf1-5d69bebd6773-workload-socket\") pod \"e2e-api-a-045df0-openshift-default-cfcdff7b5-x8p7w\" (UID: \"6572fd4a-74f5-4571-acf1-5d69bebd6773\") " pod="openshift-ingress/e2e-api-a-045df0-openshift-default-cfcdff7b5-x8p7w" Jul 12 18:56:11.487146 ip-10-0-129-128 kubenswrapper[2559]: I0712 18:56:11.486977 2559 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/6572fd4a-74f5-4571-acf1-5d69bebd6773-istio-envoy\") pod \"e2e-api-a-045df0-openshift-default-cfcdff7b5-x8p7w\" (UID: \"6572fd4a-74f5-4571-acf1-5d69bebd6773\") " pod="openshift-ingress/e2e-api-a-045df0-openshift-default-cfcdff7b5-x8p7w" Jul 12 18:56:11.487146 ip-10-0-129-128 kubenswrapper[2559]: I0712 18:56:11.487061 2559 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/6572fd4a-74f5-4571-acf1-5d69bebd6773-workload-certs\") pod \"e2e-api-a-045df0-openshift-default-cfcdff7b5-x8p7w\" (UID: \"6572fd4a-74f5-4571-acf1-5d69bebd6773\") " pod="openshift-ingress/e2e-api-a-045df0-openshift-default-cfcdff7b5-x8p7w" Jul 12 18:56:11.487510 ip-10-0-129-128 kubenswrapper[2559]: I0712 18:56:11.487210 2559 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/6572fd4a-74f5-4571-acf1-5d69bebd6773-istio-data\") pod \"e2e-api-a-045df0-openshift-default-cfcdff7b5-x8p7w\" (UID: \"6572fd4a-74f5-4571-acf1-5d69bebd6773\") " pod="openshift-ingress/e2e-api-a-045df0-openshift-default-cfcdff7b5-x8p7w" Jul 12 18:56:11.487510 ip-10-0-129-128 kubenswrapper[2559]: I0712 18:56:11.487341 2559 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/6572fd4a-74f5-4571-acf1-5d69bebd6773-workload-socket\") pod \"e2e-api-a-045df0-openshift-default-cfcdff7b5-x8p7w\" (UID: \"6572fd4a-74f5-4571-acf1-5d69bebd6773\") " pod="openshift-ingress/e2e-api-a-045df0-openshift-default-cfcdff7b5-x8p7w" Jul 12 18:56:11.487510 ip-10-0-129-128 kubenswrapper[2559]: I0712 18:56:11.487415 2559 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/6572fd4a-74f5-4571-acf1-5d69bebd6773-workload-certs\") pod \"e2e-api-a-045df0-openshift-default-cfcdff7b5-x8p7w\" (UID: \"6572fd4a-74f5-4571-acf1-5d69bebd6773\") " pod="openshift-ingress/e2e-api-a-045df0-openshift-default-cfcdff7b5-x8p7w" Jul 12 18:56:11.487510 ip-10-0-129-128 kubenswrapper[2559]: I0712 18:56:11.487427 2559 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/6572fd4a-74f5-4571-acf1-5d69bebd6773-credential-socket\") pod \"e2e-api-a-045df0-openshift-default-cfcdff7b5-x8p7w\" (UID: \"6572fd4a-74f5-4571-acf1-5d69bebd6773\") " pod="openshift-ingress/e2e-api-a-045df0-openshift-default-cfcdff7b5-x8p7w" Jul 12 18:56:11.487667 ip-10-0-129-128 kubenswrapper[2559]: I0712 18:56:11.487534 2559 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/6572fd4a-74f5-4571-acf1-5d69bebd6773-istiod-ca-cert\") pod \"e2e-api-a-045df0-openshift-default-cfcdff7b5-x8p7w\" (UID: \"6572fd4a-74f5-4571-acf1-5d69bebd6773\") " pod="openshift-ingress/e2e-api-a-045df0-openshift-default-cfcdff7b5-x8p7w" Jul 12 18:56:11.489244 ip-10-0-129-128 kubenswrapper[2559]: I0712 18:56:11.489213 2559 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/6572fd4a-74f5-4571-acf1-5d69bebd6773-istio-envoy\") pod \"e2e-api-a-045df0-openshift-default-cfcdff7b5-x8p7w\" (UID: \"6572fd4a-74f5-4571-acf1-5d69bebd6773\") " pod="openshift-ingress/e2e-api-a-045df0-openshift-default-cfcdff7b5-x8p7w" Jul 12 18:56:11.489486 ip-10-0-129-128 kubenswrapper[2559]: I0712 18:56:11.489467 2559 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/6572fd4a-74f5-4571-acf1-5d69bebd6773-istio-podinfo\") pod \"e2e-api-a-045df0-openshift-default-cfcdff7b5-x8p7w\" (UID: \"6572fd4a-74f5-4571-acf1-5d69bebd6773\") " pod="openshift-ingress/e2e-api-a-045df0-openshift-default-cfcdff7b5-x8p7w" Jul 12 18:56:11.494826 ip-10-0-129-128 kubenswrapper[2559]: I0712 18:56:11.494799 2559 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/6572fd4a-74f5-4571-acf1-5d69bebd6773-istio-token\") pod \"e2e-api-a-045df0-openshift-default-cfcdff7b5-x8p7w\" (UID: \"6572fd4a-74f5-4571-acf1-5d69bebd6773\") " pod="openshift-ingress/e2e-api-a-045df0-openshift-default-cfcdff7b5-x8p7w" Jul 12 18:56:11.494973 ip-10-0-129-128 kubenswrapper[2559]: I0712 18:56:11.494956 2559 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-wtnnd\" (UniqueName: \"kubernetes.io/projected/6572fd4a-74f5-4571-acf1-5d69bebd6773-kube-api-access-wtnnd\") pod \"e2e-api-a-045df0-openshift-default-cfcdff7b5-x8p7w\" (UID: \"6572fd4a-74f5-4571-acf1-5d69bebd6773\") " pod="openshift-ingress/e2e-api-a-045df0-openshift-default-cfcdff7b5-x8p7w" Jul 12 18:56:11.587074 ip-10-0-129-128 kubenswrapper[2559]: I0712 18:56:11.586971 2559 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-api-a-045df0-openshift-default-cfcdff7b5-x8p7w" Jul 12 18:56:11.723867 ip-10-0-129-128 kubenswrapper[2559]: I0712 18:56:11.723837 2559 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/e2e-api-a-045df0-openshift-default-cfcdff7b5-x8p7w"] Jul 12 18:56:11.725293 ip-10-0-129-128 kubenswrapper[2559]: W0712 18:56:11.725270 2559 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod6572fd4a_74f5_4571_acf1_5d69bebd6773.slice/crio-e4c2478cf0bed116a52751db8a6946e8d5759a23e85cde850ff6e487ec80f770 WatchSource:0}: Error finding container e4c2478cf0bed116a52751db8a6946e8d5759a23e85cde850ff6e487ec80f770: Status 404 returned error can't find the container with id e4c2478cf0bed116a52751db8a6946e8d5759a23e85cde850ff6e487ec80f770 Jul 12 18:56:11.727438 ip-10-0-129-128 kubenswrapper[2559]: I0712 18:56:11.727412 2559 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236228Ki","pods":"250"} Jul 12 18:56:11.727513 ip-10-0-129-128 kubenswrapper[2559]: I0712 18:56:11.727471 2559 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236228Ki","pods":"250"} Jul 12 18:56:11.727554 ip-10-0-129-128 kubenswrapper[2559]: I0712 18:56:11.727519 2559 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236228Ki","pods":"250"} Jul 12 18:56:12.358224 ip-10-0-129-128 kubenswrapper[2559]: I0712 18:56:12.358174 2559 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-api-a-045df0-openshift-default-cfcdff7b5-x8p7w" event={"ID":"6572fd4a-74f5-4571-acf1-5d69bebd6773","Type":"ContainerStarted","Data":"d40b60eb3247c3e1095ca431fbff1990500eb52f3a2d7ea86253cd3427972cd1"} Jul 12 18:56:12.358410 ip-10-0-129-128 kubenswrapper[2559]: I0712 18:56:12.358242 2559 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-api-a-045df0-openshift-default-cfcdff7b5-x8p7w" event={"ID":"6572fd4a-74f5-4571-acf1-5d69bebd6773","Type":"ContainerStarted","Data":"e4c2478cf0bed116a52751db8a6946e8d5759a23e85cde850ff6e487ec80f770"} Jul 12 18:56:12.389945 ip-10-0-129-128 kubenswrapper[2559]: I0712 18:56:12.389896 2559 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-ingress/e2e-api-a-045df0-openshift-default-cfcdff7b5-x8p7w" podStartSLOduration=1.389880225 podStartE2EDuration="1.389880225s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-12 18:56:11 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-12 18:56:12.387442788 +0000 UTC m=+3779.881510718" watchObservedRunningTime="2026-07-12 18:56:12.389880225 +0000 UTC m=+3779.883948155" Jul 12 18:56:12.587746 ip-10-0-129-128 kubenswrapper[2559]: I0712 18:56:12.587703 2559 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="openshift-ingress/e2e-api-a-045df0-openshift-default-cfcdff7b5-x8p7w" Jul 12 18:56:12.593002 ip-10-0-129-128 kubenswrapper[2559]: I0712 18:56:12.592978 2559 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="openshift-ingress/e2e-api-a-045df0-openshift-default-cfcdff7b5-x8p7w" Jul 12 18:56:13.364298 ip-10-0-129-128 kubenswrapper[2559]: I0712 18:56:13.364265 2559 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/e2e-api-a-045df0-openshift-default-cfcdff7b5-x8p7w" Jul 12 18:56:13.365259 ip-10-0-129-128 kubenswrapper[2559]: I0712 18:56:13.365236 2559 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/e2e-api-a-045df0-openshift-default-cfcdff7b5-x8p7w" Jul 12 18:56:15.663187 ip-10-0-129-128 kubenswrapper[2559]: I0712 18:56:15.663158 2559 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-inf-b-d2f6a5-openshift-default-7558f8b88c-qnw6s" Jul 12 18:56:15.828220 ip-10-0-129-128 kubenswrapper[2559]: I0712 18:56:15.828176 2559 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/9dba6483-775d-4b0c-b35d-48d700327eda-istiod-ca-cert\") pod \"9dba6483-775d-4b0c-b35d-48d700327eda\" (UID: \"9dba6483-775d-4b0c-b35d-48d700327eda\") " Jul 12 18:56:15.828441 ip-10-0-129-128 kubenswrapper[2559]: I0712 18:56:15.828231 2559 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/9dba6483-775d-4b0c-b35d-48d700327eda-workload-socket\") pod \"9dba6483-775d-4b0c-b35d-48d700327eda\" (UID: \"9dba6483-775d-4b0c-b35d-48d700327eda\") " Jul 12 18:56:15.828441 ip-10-0-129-128 kubenswrapper[2559]: I0712 18:56:15.828258 2559 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/9dba6483-775d-4b0c-b35d-48d700327eda-istio-token\") pod \"9dba6483-775d-4b0c-b35d-48d700327eda\" (UID: \"9dba6483-775d-4b0c-b35d-48d700327eda\") " Jul 12 18:56:15.828441 ip-10-0-129-128 kubenswrapper[2559]: I0712 18:56:15.828302 2559 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/9dba6483-775d-4b0c-b35d-48d700327eda-istio-data\") pod \"9dba6483-775d-4b0c-b35d-48d700327eda\" (UID: \"9dba6483-775d-4b0c-b35d-48d700327eda\") " Jul 12 18:56:15.828441 ip-10-0-129-128 kubenswrapper[2559]: I0712 18:56:15.828344 2559 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/9dba6483-775d-4b0c-b35d-48d700327eda-istio-envoy\") pod \"9dba6483-775d-4b0c-b35d-48d700327eda\" (UID: \"9dba6483-775d-4b0c-b35d-48d700327eda\") " Jul 12 18:56:15.828441 ip-10-0-129-128 kubenswrapper[2559]: I0712 18:56:15.828387 2559 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/9dba6483-775d-4b0c-b35d-48d700327eda-credential-socket\") pod \"9dba6483-775d-4b0c-b35d-48d700327eda\" (UID: \"9dba6483-775d-4b0c-b35d-48d700327eda\") " Jul 12 18:56:15.828441 ip-10-0-129-128 kubenswrapper[2559]: I0712 18:56:15.828433 2559 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/9dba6483-775d-4b0c-b35d-48d700327eda-istio-podinfo\") pod \"9dba6483-775d-4b0c-b35d-48d700327eda\" (UID: \"9dba6483-775d-4b0c-b35d-48d700327eda\") " Jul 12 18:56:15.828803 ip-10-0-129-128 kubenswrapper[2559]: I0712 18:56:15.828474 2559 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/9dba6483-775d-4b0c-b35d-48d700327eda-workload-certs\") pod \"9dba6483-775d-4b0c-b35d-48d700327eda\" (UID: \"9dba6483-775d-4b0c-b35d-48d700327eda\") " Jul 12 18:56:15.828803 ip-10-0-129-128 kubenswrapper[2559]: I0712 18:56:15.828540 2559 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-b6l8n\" (UniqueName: \"kubernetes.io/projected/9dba6483-775d-4b0c-b35d-48d700327eda-kube-api-access-b6l8n\") pod \"9dba6483-775d-4b0c-b35d-48d700327eda\" (UID: \"9dba6483-775d-4b0c-b35d-48d700327eda\") " Jul 12 18:56:15.828917 ip-10-0-129-128 kubenswrapper[2559]: I0712 18:56:15.828827 2559 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/9dba6483-775d-4b0c-b35d-48d700327eda-istiod-ca-cert" (OuterVolumeSpecName: "istiod-ca-cert") pod "9dba6483-775d-4b0c-b35d-48d700327eda" (UID: "9dba6483-775d-4b0c-b35d-48d700327eda"). InnerVolumeSpecName "istiod-ca-cert". PluginName "kubernetes.io/configmap", VolumeGIDValue "" Jul 12 18:56:15.828917 ip-10-0-129-128 kubenswrapper[2559]: I0712 18:56:15.828829 2559 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/9dba6483-775d-4b0c-b35d-48d700327eda-workload-socket" (OuterVolumeSpecName: "workload-socket") pod "9dba6483-775d-4b0c-b35d-48d700327eda" (UID: "9dba6483-775d-4b0c-b35d-48d700327eda"). InnerVolumeSpecName "workload-socket". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Jul 12 18:56:15.829096 ip-10-0-129-128 kubenswrapper[2559]: I0712 18:56:15.829074 2559 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/9dba6483-775d-4b0c-b35d-48d700327eda-workload-certs" (OuterVolumeSpecName: "workload-certs") pod "9dba6483-775d-4b0c-b35d-48d700327eda" (UID: "9dba6483-775d-4b0c-b35d-48d700327eda"). InnerVolumeSpecName "workload-certs". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Jul 12 18:56:15.829159 ip-10-0-129-128 kubenswrapper[2559]: I0712 18:56:15.829093 2559 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/9dba6483-775d-4b0c-b35d-48d700327eda-credential-socket" (OuterVolumeSpecName: "credential-socket") pod "9dba6483-775d-4b0c-b35d-48d700327eda" (UID: "9dba6483-775d-4b0c-b35d-48d700327eda"). InnerVolumeSpecName "credential-socket". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Jul 12 18:56:15.829585 ip-10-0-129-128 kubenswrapper[2559]: I0712 18:56:15.829556 2559 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/9dba6483-775d-4b0c-b35d-48d700327eda-istio-data" (OuterVolumeSpecName: "istio-data") pod "9dba6483-775d-4b0c-b35d-48d700327eda" (UID: "9dba6483-775d-4b0c-b35d-48d700327eda"). InnerVolumeSpecName "istio-data". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Jul 12 18:56:15.831012 ip-10-0-129-128 kubenswrapper[2559]: I0712 18:56:15.830986 2559 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/9dba6483-775d-4b0c-b35d-48d700327eda-istio-envoy" (OuterVolumeSpecName: "istio-envoy") pod "9dba6483-775d-4b0c-b35d-48d700327eda" (UID: "9dba6483-775d-4b0c-b35d-48d700327eda"). InnerVolumeSpecName "istio-envoy". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Jul 12 18:56:15.831222 ip-10-0-129-128 kubenswrapper[2559]: I0712 18:56:15.831196 2559 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/9dba6483-775d-4b0c-b35d-48d700327eda-istio-token" (OuterVolumeSpecName: "istio-token") pod "9dba6483-775d-4b0c-b35d-48d700327eda" (UID: "9dba6483-775d-4b0c-b35d-48d700327eda"). InnerVolumeSpecName "istio-token". PluginName "kubernetes.io/projected", VolumeGIDValue "" Jul 12 18:56:15.831293 ip-10-0-129-128 kubenswrapper[2559]: I0712 18:56:15.831224 2559 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/downward-api/9dba6483-775d-4b0c-b35d-48d700327eda-istio-podinfo" (OuterVolumeSpecName: "istio-podinfo") pod "9dba6483-775d-4b0c-b35d-48d700327eda" (UID: "9dba6483-775d-4b0c-b35d-48d700327eda"). InnerVolumeSpecName "istio-podinfo". PluginName "kubernetes.io/downward-api", VolumeGIDValue "" Jul 12 18:56:15.831293 ip-10-0-129-128 kubenswrapper[2559]: I0712 18:56:15.831204 2559 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/9dba6483-775d-4b0c-b35d-48d700327eda-kube-api-access-b6l8n" (OuterVolumeSpecName: "kube-api-access-b6l8n") pod "9dba6483-775d-4b0c-b35d-48d700327eda" (UID: "9dba6483-775d-4b0c-b35d-48d700327eda"). InnerVolumeSpecName "kube-api-access-b6l8n". PluginName "kubernetes.io/projected", VolumeGIDValue "" Jul 12 18:56:15.930171 ip-10-0-129-128 kubenswrapper[2559]: I0712 18:56:15.930136 2559 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-b6l8n\" (UniqueName: \"kubernetes.io/projected/9dba6483-775d-4b0c-b35d-48d700327eda-kube-api-access-b6l8n\") on node \"ip-10-0-129-128.ec2.internal\" DevicePath \"\"" Jul 12 18:56:15.930171 ip-10-0-129-128 kubenswrapper[2559]: I0712 18:56:15.930164 2559 reconciler_common.go:299] "Volume detached for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/9dba6483-775d-4b0c-b35d-48d700327eda-istiod-ca-cert\") on node \"ip-10-0-129-128.ec2.internal\" DevicePath \"\"" Jul 12 18:56:15.930171 ip-10-0-129-128 kubenswrapper[2559]: I0712 18:56:15.930175 2559 reconciler_common.go:299] "Volume detached for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/9dba6483-775d-4b0c-b35d-48d700327eda-workload-socket\") on node \"ip-10-0-129-128.ec2.internal\" DevicePath \"\"" Jul 12 18:56:15.930415 ip-10-0-129-128 kubenswrapper[2559]: I0712 18:56:15.930184 2559 reconciler_common.go:299] "Volume detached for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/9dba6483-775d-4b0c-b35d-48d700327eda-istio-token\") on node \"ip-10-0-129-128.ec2.internal\" DevicePath \"\"" Jul 12 18:56:15.930415 ip-10-0-129-128 kubenswrapper[2559]: I0712 18:56:15.930194 2559 reconciler_common.go:299] "Volume detached for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/9dba6483-775d-4b0c-b35d-48d700327eda-istio-data\") on node \"ip-10-0-129-128.ec2.internal\" DevicePath \"\"" Jul 12 18:56:15.930415 ip-10-0-129-128 kubenswrapper[2559]: I0712 18:56:15.930203 2559 reconciler_common.go:299] "Volume detached for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/9dba6483-775d-4b0c-b35d-48d700327eda-istio-envoy\") on node \"ip-10-0-129-128.ec2.internal\" DevicePath \"\"" Jul 12 18:56:15.930415 ip-10-0-129-128 kubenswrapper[2559]: I0712 18:56:15.930210 2559 reconciler_common.go:299] "Volume detached for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/9dba6483-775d-4b0c-b35d-48d700327eda-credential-socket\") on node \"ip-10-0-129-128.ec2.internal\" DevicePath \"\"" Jul 12 18:56:15.930415 ip-10-0-129-128 kubenswrapper[2559]: I0712 18:56:15.930217 2559 reconciler_common.go:299] "Volume detached for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/9dba6483-775d-4b0c-b35d-48d700327eda-istio-podinfo\") on node \"ip-10-0-129-128.ec2.internal\" DevicePath \"\"" Jul 12 18:56:15.930415 ip-10-0-129-128 kubenswrapper[2559]: I0712 18:56:15.930225 2559 reconciler_common.go:299] "Volume detached for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/9dba6483-775d-4b0c-b35d-48d700327eda-workload-certs\") on node \"ip-10-0-129-128.ec2.internal\" DevicePath \"\"" Jul 12 18:56:16.380292 ip-10-0-129-128 kubenswrapper[2559]: I0712 18:56:16.380258 2559 generic.go:356] "Generic (PLEG): container finished" podID="9dba6483-775d-4b0c-b35d-48d700327eda" containerID="1469c666f16266ce0a1911f89d1e1b841903c8e6e14b6233f517df378964f00c" exitCode=0 Jul 12 18:56:16.380467 ip-10-0-129-128 kubenswrapper[2559]: I0712 18:56:16.380307 2559 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-inf-b-d2f6a5-openshift-default-7558f8b88c-qnw6s" Jul 12 18:56:16.380467 ip-10-0-129-128 kubenswrapper[2559]: I0712 18:56:16.380342 2559 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-inf-b-d2f6a5-openshift-default-7558f8b88c-qnw6s" event={"ID":"9dba6483-775d-4b0c-b35d-48d700327eda","Type":"ContainerDied","Data":"1469c666f16266ce0a1911f89d1e1b841903c8e6e14b6233f517df378964f00c"} Jul 12 18:56:16.380467 ip-10-0-129-128 kubenswrapper[2559]: I0712 18:56:16.380397 2559 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-inf-b-d2f6a5-openshift-default-7558f8b88c-qnw6s" event={"ID":"9dba6483-775d-4b0c-b35d-48d700327eda","Type":"ContainerDied","Data":"a4abd19cfbd13296b3c7d5262a751ebe97e12762c64ac97a891791f963232b4e"} Jul 12 18:56:16.380467 ip-10-0-129-128 kubenswrapper[2559]: I0712 18:56:16.380419 2559 scope.go:117] "RemoveContainer" containerID="1469c666f16266ce0a1911f89d1e1b841903c8e6e14b6233f517df378964f00c" Jul 12 18:56:16.390270 ip-10-0-129-128 kubenswrapper[2559]: I0712 18:56:16.390252 2559 scope.go:117] "RemoveContainer" containerID="9b5021488d772b4da9e19bb94621ca1eba182007d44f0e354a0ae6cb8704c8d8" Jul 12 18:56:16.398329 ip-10-0-129-128 kubenswrapper[2559]: I0712 18:56:16.398310 2559 scope.go:117] "RemoveContainer" containerID="1469c666f16266ce0a1911f89d1e1b841903c8e6e14b6233f517df378964f00c" Jul 12 18:56:16.398621 ip-10-0-129-128 kubenswrapper[2559]: E0712 18:56:16.398602 2559 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"1469c666f16266ce0a1911f89d1e1b841903c8e6e14b6233f517df378964f00c\": container with ID starting with 1469c666f16266ce0a1911f89d1e1b841903c8e6e14b6233f517df378964f00c not found: ID does not exist" containerID="1469c666f16266ce0a1911f89d1e1b841903c8e6e14b6233f517df378964f00c" Jul 12 18:56:16.398692 ip-10-0-129-128 kubenswrapper[2559]: I0712 18:56:16.398630 2559 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"1469c666f16266ce0a1911f89d1e1b841903c8e6e14b6233f517df378964f00c"} err="failed to get container status \"1469c666f16266ce0a1911f89d1e1b841903c8e6e14b6233f517df378964f00c\": rpc error: code = NotFound desc = could not find container \"1469c666f16266ce0a1911f89d1e1b841903c8e6e14b6233f517df378964f00c\": container with ID starting with 1469c666f16266ce0a1911f89d1e1b841903c8e6e14b6233f517df378964f00c not found: ID does not exist" Jul 12 18:56:16.398692 ip-10-0-129-128 kubenswrapper[2559]: I0712 18:56:16.398667 2559 scope.go:117] "RemoveContainer" containerID="9b5021488d772b4da9e19bb94621ca1eba182007d44f0e354a0ae6cb8704c8d8" Jul 12 18:56:16.398919 ip-10-0-129-128 kubenswrapper[2559]: E0712 18:56:16.398901 2559 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"9b5021488d772b4da9e19bb94621ca1eba182007d44f0e354a0ae6cb8704c8d8\": container with ID starting with 9b5021488d772b4da9e19bb94621ca1eba182007d44f0e354a0ae6cb8704c8d8 not found: ID does not exist" containerID="9b5021488d772b4da9e19bb94621ca1eba182007d44f0e354a0ae6cb8704c8d8" Jul 12 18:56:16.398969 ip-10-0-129-128 kubenswrapper[2559]: I0712 18:56:16.398924 2559 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"9b5021488d772b4da9e19bb94621ca1eba182007d44f0e354a0ae6cb8704c8d8"} err="failed to get container status \"9b5021488d772b4da9e19bb94621ca1eba182007d44f0e354a0ae6cb8704c8d8\": rpc error: code = NotFound desc = could not find container \"9b5021488d772b4da9e19bb94621ca1eba182007d44f0e354a0ae6cb8704c8d8\": container with ID starting with 9b5021488d772b4da9e19bb94621ca1eba182007d44f0e354a0ae6cb8704c8d8 not found: ID does not exist" Jul 12 18:56:16.406884 ip-10-0-129-128 kubenswrapper[2559]: I0712 18:56:16.406859 2559 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-ingress/e2e-inf-b-d2f6a5-openshift-default-7558f8b88c-qnw6s"] Jul 12 18:56:16.412541 ip-10-0-129-128 kubenswrapper[2559]: I0712 18:56:16.412521 2559 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["openshift-ingress/e2e-inf-b-d2f6a5-openshift-default-7558f8b88c-qnw6s"] Jul 12 18:56:17.035023 ip-10-0-129-128 kubenswrapper[2559]: I0712 18:56:17.034991 2559 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="9dba6483-775d-4b0c-b35d-48d700327eda" path="/var/lib/kubelet/pods/9dba6483-775d-4b0c-b35d-48d700327eda/volumes" Jul 12 18:56:18.692272 ip-10-0-129-128 kubenswrapper[2559]: I0712 18:56:18.692238 2559 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-pwsvh"] Jul 12 18:56:19.521294 ip-10-0-129-128 kubenswrapper[2559]: I0712 18:56:19.521262 2559 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-ingress/e2e-api-b-a249e9-openshift-default-7bd58f74f-s7jfn"] Jul 12 18:56:19.521738 ip-10-0-129-128 kubenswrapper[2559]: I0712 18:56:19.521721 2559 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="9dba6483-775d-4b0c-b35d-48d700327eda" containerName="istio-proxy" Jul 12 18:56:19.521830 ip-10-0-129-128 kubenswrapper[2559]: I0712 18:56:19.521740 2559 state_mem.go:107] "Deleted CPUSet assignment" podUID="9dba6483-775d-4b0c-b35d-48d700327eda" containerName="istio-proxy" Jul 12 18:56:19.521830 ip-10-0-129-128 kubenswrapper[2559]: I0712 18:56:19.521751 2559 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="9dba6483-775d-4b0c-b35d-48d700327eda" containerName="istio-proxy" Jul 12 18:56:19.521830 ip-10-0-129-128 kubenswrapper[2559]: I0712 18:56:19.521759 2559 state_mem.go:107] "Deleted CPUSet assignment" podUID="9dba6483-775d-4b0c-b35d-48d700327eda" containerName="istio-proxy" Jul 12 18:56:19.521999 ip-10-0-129-128 kubenswrapper[2559]: I0712 18:56:19.521866 2559 memory_manager.go:368] "RemoveStaleState removing state" podUID="9dba6483-775d-4b0c-b35d-48d700327eda" containerName="istio-proxy" Jul 12 18:56:19.521999 ip-10-0-129-128 kubenswrapper[2559]: I0712 18:56:19.521879 2559 memory_manager.go:368] "RemoveStaleState removing state" podUID="9dba6483-775d-4b0c-b35d-48d700327eda" containerName="istio-proxy" Jul 12 18:56:19.526231 ip-10-0-129-128 kubenswrapper[2559]: I0712 18:56:19.526209 2559 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-api-b-a249e9-openshift-default-7bd58f74f-s7jfn" Jul 12 18:56:19.528728 ip-10-0-129-128 kubenswrapper[2559]: I0712 18:56:19.528704 2559 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress\"/\"e2e-api-b-a249e9-openshift-default-dockercfg-657nj\"" Jul 12 18:56:19.537571 ip-10-0-129-128 kubenswrapper[2559]: I0712 18:56:19.537546 2559 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/e2e-api-b-a249e9-openshift-default-7bd58f74f-s7jfn"] Jul 12 18:56:19.562787 ip-10-0-129-128 kubenswrapper[2559]: I0712 18:56:19.562756 2559 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/6770b17c-88fb-45af-8337-00e99f7ecf19-credential-socket\") pod \"e2e-api-b-a249e9-openshift-default-7bd58f74f-s7jfn\" (UID: \"6770b17c-88fb-45af-8337-00e99f7ecf19\") " pod="openshift-ingress/e2e-api-b-a249e9-openshift-default-7bd58f74f-s7jfn" Jul 12 18:56:19.562940 ip-10-0-129-128 kubenswrapper[2559]: I0712 18:56:19.562803 2559 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/6770b17c-88fb-45af-8337-00e99f7ecf19-istio-data\") pod \"e2e-api-b-a249e9-openshift-default-7bd58f74f-s7jfn\" (UID: \"6770b17c-88fb-45af-8337-00e99f7ecf19\") " pod="openshift-ingress/e2e-api-b-a249e9-openshift-default-7bd58f74f-s7jfn" Jul 12 18:56:19.562940 ip-10-0-129-128 kubenswrapper[2559]: I0712 18:56:19.562876 2559 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/6770b17c-88fb-45af-8337-00e99f7ecf19-istio-podinfo\") pod \"e2e-api-b-a249e9-openshift-default-7bd58f74f-s7jfn\" (UID: \"6770b17c-88fb-45af-8337-00e99f7ecf19\") " pod="openshift-ingress/e2e-api-b-a249e9-openshift-default-7bd58f74f-s7jfn" Jul 12 18:56:19.562940 ip-10-0-129-128 kubenswrapper[2559]: I0712 18:56:19.562907 2559 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/6770b17c-88fb-45af-8337-00e99f7ecf19-workload-socket\") pod \"e2e-api-b-a249e9-openshift-default-7bd58f74f-s7jfn\" (UID: \"6770b17c-88fb-45af-8337-00e99f7ecf19\") " pod="openshift-ingress/e2e-api-b-a249e9-openshift-default-7bd58f74f-s7jfn" Jul 12 18:56:19.562940 ip-10-0-129-128 kubenswrapper[2559]: I0712 18:56:19.562938 2559 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/6770b17c-88fb-45af-8337-00e99f7ecf19-workload-certs\") pod \"e2e-api-b-a249e9-openshift-default-7bd58f74f-s7jfn\" (UID: \"6770b17c-88fb-45af-8337-00e99f7ecf19\") " pod="openshift-ingress/e2e-api-b-a249e9-openshift-default-7bd58f74f-s7jfn" Jul 12 18:56:19.563103 ip-10-0-129-128 kubenswrapper[2559]: I0712 18:56:19.562967 2559 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/6770b17c-88fb-45af-8337-00e99f7ecf19-istiod-ca-cert\") pod \"e2e-api-b-a249e9-openshift-default-7bd58f74f-s7jfn\" (UID: \"6770b17c-88fb-45af-8337-00e99f7ecf19\") " pod="openshift-ingress/e2e-api-b-a249e9-openshift-default-7bd58f74f-s7jfn" Jul 12 18:56:19.563103 ip-10-0-129-128 kubenswrapper[2559]: I0712 18:56:19.563034 2559 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/6770b17c-88fb-45af-8337-00e99f7ecf19-istio-token\") pod \"e2e-api-b-a249e9-openshift-default-7bd58f74f-s7jfn\" (UID: \"6770b17c-88fb-45af-8337-00e99f7ecf19\") " pod="openshift-ingress/e2e-api-b-a249e9-openshift-default-7bd58f74f-s7jfn" Jul 12 18:56:19.563103 ip-10-0-129-128 kubenswrapper[2559]: I0712 18:56:19.563058 2559 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-xtchv\" (UniqueName: \"kubernetes.io/projected/6770b17c-88fb-45af-8337-00e99f7ecf19-kube-api-access-xtchv\") pod \"e2e-api-b-a249e9-openshift-default-7bd58f74f-s7jfn\" (UID: \"6770b17c-88fb-45af-8337-00e99f7ecf19\") " pod="openshift-ingress/e2e-api-b-a249e9-openshift-default-7bd58f74f-s7jfn" Jul 12 18:56:19.563216 ip-10-0-129-128 kubenswrapper[2559]: I0712 18:56:19.563106 2559 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/6770b17c-88fb-45af-8337-00e99f7ecf19-istio-envoy\") pod \"e2e-api-b-a249e9-openshift-default-7bd58f74f-s7jfn\" (UID: \"6770b17c-88fb-45af-8337-00e99f7ecf19\") " pod="openshift-ingress/e2e-api-b-a249e9-openshift-default-7bd58f74f-s7jfn" Jul 12 18:56:19.664325 ip-10-0-129-128 kubenswrapper[2559]: I0712 18:56:19.664284 2559 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/6770b17c-88fb-45af-8337-00e99f7ecf19-istiod-ca-cert\") pod \"e2e-api-b-a249e9-openshift-default-7bd58f74f-s7jfn\" (UID: \"6770b17c-88fb-45af-8337-00e99f7ecf19\") " pod="openshift-ingress/e2e-api-b-a249e9-openshift-default-7bd58f74f-s7jfn" Jul 12 18:56:19.664520 ip-10-0-129-128 kubenswrapper[2559]: I0712 18:56:19.664335 2559 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/6770b17c-88fb-45af-8337-00e99f7ecf19-istio-token\") pod \"e2e-api-b-a249e9-openshift-default-7bd58f74f-s7jfn\" (UID: \"6770b17c-88fb-45af-8337-00e99f7ecf19\") " pod="openshift-ingress/e2e-api-b-a249e9-openshift-default-7bd58f74f-s7jfn" Jul 12 18:56:19.664520 ip-10-0-129-128 kubenswrapper[2559]: I0712 18:56:19.664396 2559 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-xtchv\" (UniqueName: \"kubernetes.io/projected/6770b17c-88fb-45af-8337-00e99f7ecf19-kube-api-access-xtchv\") pod \"e2e-api-b-a249e9-openshift-default-7bd58f74f-s7jfn\" (UID: \"6770b17c-88fb-45af-8337-00e99f7ecf19\") " pod="openshift-ingress/e2e-api-b-a249e9-openshift-default-7bd58f74f-s7jfn" Jul 12 18:56:19.664520 ip-10-0-129-128 kubenswrapper[2559]: I0712 18:56:19.664435 2559 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/6770b17c-88fb-45af-8337-00e99f7ecf19-istio-envoy\") pod \"e2e-api-b-a249e9-openshift-default-7bd58f74f-s7jfn\" (UID: \"6770b17c-88fb-45af-8337-00e99f7ecf19\") " pod="openshift-ingress/e2e-api-b-a249e9-openshift-default-7bd58f74f-s7jfn" Jul 12 18:56:19.664520 ip-10-0-129-128 kubenswrapper[2559]: I0712 18:56:19.664486 2559 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/6770b17c-88fb-45af-8337-00e99f7ecf19-credential-socket\") pod \"e2e-api-b-a249e9-openshift-default-7bd58f74f-s7jfn\" (UID: \"6770b17c-88fb-45af-8337-00e99f7ecf19\") " pod="openshift-ingress/e2e-api-b-a249e9-openshift-default-7bd58f74f-s7jfn" Jul 12 18:56:19.664747 ip-10-0-129-128 kubenswrapper[2559]: I0712 18:56:19.664525 2559 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/6770b17c-88fb-45af-8337-00e99f7ecf19-istio-data\") pod \"e2e-api-b-a249e9-openshift-default-7bd58f74f-s7jfn\" (UID: \"6770b17c-88fb-45af-8337-00e99f7ecf19\") " pod="openshift-ingress/e2e-api-b-a249e9-openshift-default-7bd58f74f-s7jfn" Jul 12 18:56:19.664747 ip-10-0-129-128 kubenswrapper[2559]: I0712 18:56:19.664577 2559 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/6770b17c-88fb-45af-8337-00e99f7ecf19-istio-podinfo\") pod \"e2e-api-b-a249e9-openshift-default-7bd58f74f-s7jfn\" (UID: \"6770b17c-88fb-45af-8337-00e99f7ecf19\") " pod="openshift-ingress/e2e-api-b-a249e9-openshift-default-7bd58f74f-s7jfn" Jul 12 18:56:19.664747 ip-10-0-129-128 kubenswrapper[2559]: I0712 18:56:19.664606 2559 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/6770b17c-88fb-45af-8337-00e99f7ecf19-workload-socket\") pod \"e2e-api-b-a249e9-openshift-default-7bd58f74f-s7jfn\" (UID: \"6770b17c-88fb-45af-8337-00e99f7ecf19\") " pod="openshift-ingress/e2e-api-b-a249e9-openshift-default-7bd58f74f-s7jfn" Jul 12 18:56:19.664747 ip-10-0-129-128 kubenswrapper[2559]: I0712 18:56:19.664724 2559 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/6770b17c-88fb-45af-8337-00e99f7ecf19-workload-certs\") pod \"e2e-api-b-a249e9-openshift-default-7bd58f74f-s7jfn\" (UID: \"6770b17c-88fb-45af-8337-00e99f7ecf19\") " pod="openshift-ingress/e2e-api-b-a249e9-openshift-default-7bd58f74f-s7jfn" Jul 12 18:56:19.664985 ip-10-0-129-128 kubenswrapper[2559]: I0712 18:56:19.664898 2559 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/6770b17c-88fb-45af-8337-00e99f7ecf19-credential-socket\") pod \"e2e-api-b-a249e9-openshift-default-7bd58f74f-s7jfn\" (UID: \"6770b17c-88fb-45af-8337-00e99f7ecf19\") " pod="openshift-ingress/e2e-api-b-a249e9-openshift-default-7bd58f74f-s7jfn" Jul 12 18:56:19.665042 ip-10-0-129-128 kubenswrapper[2559]: I0712 18:56:19.664985 2559 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/6770b17c-88fb-45af-8337-00e99f7ecf19-istio-data\") pod \"e2e-api-b-a249e9-openshift-default-7bd58f74f-s7jfn\" (UID: \"6770b17c-88fb-45af-8337-00e99f7ecf19\") " pod="openshift-ingress/e2e-api-b-a249e9-openshift-default-7bd58f74f-s7jfn" Jul 12 18:56:19.665097 ip-10-0-129-128 kubenswrapper[2559]: I0712 18:56:19.665074 2559 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/6770b17c-88fb-45af-8337-00e99f7ecf19-workload-certs\") pod \"e2e-api-b-a249e9-openshift-default-7bd58f74f-s7jfn\" (UID: \"6770b17c-88fb-45af-8337-00e99f7ecf19\") " pod="openshift-ingress/e2e-api-b-a249e9-openshift-default-7bd58f74f-s7jfn" Jul 12 18:56:19.665139 ip-10-0-129-128 kubenswrapper[2559]: I0712 18:56:19.665086 2559 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/6770b17c-88fb-45af-8337-00e99f7ecf19-workload-socket\") pod \"e2e-api-b-a249e9-openshift-default-7bd58f74f-s7jfn\" (UID: \"6770b17c-88fb-45af-8337-00e99f7ecf19\") " pod="openshift-ingress/e2e-api-b-a249e9-openshift-default-7bd58f74f-s7jfn" Jul 12 18:56:19.665183 ip-10-0-129-128 kubenswrapper[2559]: I0712 18:56:19.665157 2559 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/6770b17c-88fb-45af-8337-00e99f7ecf19-istiod-ca-cert\") pod \"e2e-api-b-a249e9-openshift-default-7bd58f74f-s7jfn\" (UID: \"6770b17c-88fb-45af-8337-00e99f7ecf19\") " pod="openshift-ingress/e2e-api-b-a249e9-openshift-default-7bd58f74f-s7jfn" Jul 12 18:56:19.666785 ip-10-0-129-128 kubenswrapper[2559]: I0712 18:56:19.666761 2559 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/6770b17c-88fb-45af-8337-00e99f7ecf19-istio-envoy\") pod \"e2e-api-b-a249e9-openshift-default-7bd58f74f-s7jfn\" (UID: \"6770b17c-88fb-45af-8337-00e99f7ecf19\") " pod="openshift-ingress/e2e-api-b-a249e9-openshift-default-7bd58f74f-s7jfn" Jul 12 18:56:19.667097 ip-10-0-129-128 kubenswrapper[2559]: I0712 18:56:19.667077 2559 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/6770b17c-88fb-45af-8337-00e99f7ecf19-istio-podinfo\") pod \"e2e-api-b-a249e9-openshift-default-7bd58f74f-s7jfn\" (UID: \"6770b17c-88fb-45af-8337-00e99f7ecf19\") " pod="openshift-ingress/e2e-api-b-a249e9-openshift-default-7bd58f74f-s7jfn" Jul 12 18:56:19.675609 ip-10-0-129-128 kubenswrapper[2559]: I0712 18:56:19.675552 2559 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/6770b17c-88fb-45af-8337-00e99f7ecf19-istio-token\") pod \"e2e-api-b-a249e9-openshift-default-7bd58f74f-s7jfn\" (UID: \"6770b17c-88fb-45af-8337-00e99f7ecf19\") " pod="openshift-ingress/e2e-api-b-a249e9-openshift-default-7bd58f74f-s7jfn" Jul 12 18:56:19.676119 ip-10-0-129-128 kubenswrapper[2559]: I0712 18:56:19.676067 2559 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-xtchv\" (UniqueName: \"kubernetes.io/projected/6770b17c-88fb-45af-8337-00e99f7ecf19-kube-api-access-xtchv\") pod \"e2e-api-b-a249e9-openshift-default-7bd58f74f-s7jfn\" (UID: \"6770b17c-88fb-45af-8337-00e99f7ecf19\") " pod="openshift-ingress/e2e-api-b-a249e9-openshift-default-7bd58f74f-s7jfn" Jul 12 18:56:19.838073 ip-10-0-129-128 kubenswrapper[2559]: I0712 18:56:19.837971 2559 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-api-b-a249e9-openshift-default-7bd58f74f-s7jfn" Jul 12 18:56:19.983716 ip-10-0-129-128 kubenswrapper[2559]: I0712 18:56:19.983229 2559 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/e2e-api-b-a249e9-openshift-default-7bd58f74f-s7jfn"] Jul 12 18:56:19.997166 ip-10-0-129-128 kubenswrapper[2559]: W0712 18:56:19.997104 2559 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod6770b17c_88fb_45af_8337_00e99f7ecf19.slice/crio-dc983ccb299fab2fd605cc93547325052b963e15ee2f504c2fe363a29e908a46 WatchSource:0}: Error finding container dc983ccb299fab2fd605cc93547325052b963e15ee2f504c2fe363a29e908a46: Status 404 returned error can't find the container with id dc983ccb299fab2fd605cc93547325052b963e15ee2f504c2fe363a29e908a46 Jul 12 18:56:19.999458 ip-10-0-129-128 kubenswrapper[2559]: I0712 18:56:19.999427 2559 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236228Ki","pods":"250"} Jul 12 18:56:19.999531 ip-10-0-129-128 kubenswrapper[2559]: I0712 18:56:19.999496 2559 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236228Ki","pods":"250"} Jul 12 18:56:19.999531 ip-10-0-129-128 kubenswrapper[2559]: I0712 18:56:19.999525 2559 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236228Ki","pods":"250"} Jul 12 18:56:20.401941 ip-10-0-129-128 kubenswrapper[2559]: I0712 18:56:20.401906 2559 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-api-b-a249e9-openshift-default-7bd58f74f-s7jfn" event={"ID":"6770b17c-88fb-45af-8337-00e99f7ecf19","Type":"ContainerStarted","Data":"6e3ece87bc50b4a78cd7af041c44227e1d8b8dbbad6fee3391dbccd9546ed55a"} Jul 12 18:56:20.402126 ip-10-0-129-128 kubenswrapper[2559]: I0712 18:56:20.401958 2559 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-api-b-a249e9-openshift-default-7bd58f74f-s7jfn" event={"ID":"6770b17c-88fb-45af-8337-00e99f7ecf19","Type":"ContainerStarted","Data":"dc983ccb299fab2fd605cc93547325052b963e15ee2f504c2fe363a29e908a46"} Jul 12 18:56:20.420037 ip-10-0-129-128 kubenswrapper[2559]: I0712 18:56:20.419988 2559 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-ingress/e2e-api-b-a249e9-openshift-default-7bd58f74f-s7jfn" podStartSLOduration=1.41997146 podStartE2EDuration="1.41997146s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-12 18:56:19 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-12 18:56:20.4187581 +0000 UTC m=+3787.912826055" watchObservedRunningTime="2026-07-12 18:56:20.41997146 +0000 UTC m=+3787.914039391" Jul 12 18:56:20.839190 ip-10-0-129-128 kubenswrapper[2559]: I0712 18:56:20.838400 2559 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="openshift-ingress/e2e-api-b-a249e9-openshift-default-7bd58f74f-s7jfn" Jul 12 18:56:21.855829 ip-10-0-129-128 kubenswrapper[2559]: I0712 18:56:21.855790 2559 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="openshift-ingress/e2e-api-b-a249e9-openshift-default-7bd58f74f-s7jfn" Jul 12 18:56:22.411660 ip-10-0-129-128 kubenswrapper[2559]: I0712 18:56:22.411621 2559 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/e2e-api-b-a249e9-openshift-default-7bd58f74f-s7jfn" Jul 12 18:56:22.412658 ip-10-0-129-128 kubenswrapper[2559]: I0712 18:56:22.412619 2559 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/e2e-api-b-a249e9-openshift-default-7bd58f74f-s7jfn" Jul 12 18:56:27.441758 ip-10-0-129-128 kubenswrapper[2559]: I0712 18:56:27.441628 2559 generic.go:356] "Generic (PLEG): container finished" podID="6572fd4a-74f5-4571-acf1-5d69bebd6773" containerID="d40b60eb3247c3e1095ca431fbff1990500eb52f3a2d7ea86253cd3427972cd1" exitCode=0 Jul 12 18:56:27.442284 ip-10-0-129-128 kubenswrapper[2559]: I0712 18:56:27.441793 2559 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-api-a-045df0-openshift-default-cfcdff7b5-x8p7w" event={"ID":"6572fd4a-74f5-4571-acf1-5d69bebd6773","Type":"ContainerDied","Data":"d40b60eb3247c3e1095ca431fbff1990500eb52f3a2d7ea86253cd3427972cd1"} Jul 12 18:56:27.442385 ip-10-0-129-128 kubenswrapper[2559]: I0712 18:56:27.442280 2559 scope.go:117] "RemoveContainer" containerID="d40b60eb3247c3e1095ca431fbff1990500eb52f3a2d7ea86253cd3427972cd1" Jul 12 18:56:27.443549 ip-10-0-129-128 kubenswrapper[2559]: I0712 18:56:27.443485 2559 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236228Ki","pods":"250"} Jul 12 18:56:27.443619 ip-10-0-129-128 kubenswrapper[2559]: I0712 18:56:27.443574 2559 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236228Ki","pods":"250"} Jul 12 18:56:27.443711 ip-10-0-129-128 kubenswrapper[2559]: I0712 18:56:27.443621 2559 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236228Ki","pods":"250"} Jul 12 18:56:28.364910 ip-10-0-129-128 kubenswrapper[2559]: I0712 18:56:28.364874 2559 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/e2e-api-a-045df0-openshift-default-cfcdff7b5-x8p7w" Jul 12 18:56:28.453260 ip-10-0-129-128 kubenswrapper[2559]: I0712 18:56:28.453225 2559 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-api-a-045df0-openshift-default-cfcdff7b5-x8p7w" event={"ID":"6572fd4a-74f5-4571-acf1-5d69bebd6773","Type":"ContainerStarted","Data":"ade918f555da67287a7f8891bb9ccb675bd6399301e41bcd62284fec66460f3f"} Jul 12 18:56:28.587360 ip-10-0-129-128 kubenswrapper[2559]: I0712 18:56:28.587322 2559 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="openshift-ingress/e2e-api-a-045df0-openshift-default-cfcdff7b5-x8p7w" Jul 12 18:56:28.588656 ip-10-0-129-128 kubenswrapper[2559]: I0712 18:56:28.588613 2559 patch_prober.go:28] interesting pod/e2e-api-a-045df0-openshift-default-cfcdff7b5-x8p7w container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.132.0.50:15021/healthz/ready\": dial tcp 10.132.0.50:15021: connect: connection refused" start-of-body= Jul 12 18:56:28.588790 ip-10-0-129-128 kubenswrapper[2559]: I0712 18:56:28.588702 2559 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-api-a-045df0-openshift-default-cfcdff7b5-x8p7w" podUID="6572fd4a-74f5-4571-acf1-5d69bebd6773" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.132.0.50:15021/healthz/ready\": dial tcp 10.132.0.50:15021: connect: connection refused" Jul 12 18:56:29.588162 ip-10-0-129-128 kubenswrapper[2559]: I0712 18:56:29.588122 2559 patch_prober.go:28] interesting pod/e2e-api-a-045df0-openshift-default-cfcdff7b5-x8p7w container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.132.0.50:15021/healthz/ready\": dial tcp 10.132.0.50:15021: connect: connection refused" start-of-body= Jul 12 18:56:29.588595 ip-10-0-129-128 kubenswrapper[2559]: I0712 18:56:29.588197 2559 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-api-a-045df0-openshift-default-cfcdff7b5-x8p7w" podUID="6572fd4a-74f5-4571-acf1-5d69bebd6773" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.132.0.50:15021/healthz/ready\": dial tcp 10.132.0.50:15021: connect: connection refused" Jul 12 18:56:30.588628 ip-10-0-129-128 kubenswrapper[2559]: I0712 18:56:30.588590 2559 patch_prober.go:28] interesting pod/e2e-api-a-045df0-openshift-default-cfcdff7b5-x8p7w container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.132.0.50:15021/healthz/ready\": dial tcp 10.132.0.50:15021: connect: connection refused" start-of-body= Jul 12 18:56:30.589179 ip-10-0-129-128 kubenswrapper[2559]: I0712 18:56:30.588669 2559 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-api-a-045df0-openshift-default-cfcdff7b5-x8p7w" podUID="6572fd4a-74f5-4571-acf1-5d69bebd6773" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.132.0.50:15021/healthz/ready\": dial tcp 10.132.0.50:15021: connect: connection refused" Jul 12 18:56:31.587733 ip-10-0-129-128 kubenswrapper[2559]: I0712 18:56:31.587690 2559 patch_prober.go:28] interesting pod/e2e-api-a-045df0-openshift-default-cfcdff7b5-x8p7w container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.132.0.50:15021/healthz/ready\": dial tcp 10.132.0.50:15021: connect: connection refused" start-of-body= Jul 12 18:56:31.587906 ip-10-0-129-128 kubenswrapper[2559]: I0712 18:56:31.587760 2559 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-api-a-045df0-openshift-default-cfcdff7b5-x8p7w" podUID="6572fd4a-74f5-4571-acf1-5d69bebd6773" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.132.0.50:15021/healthz/ready\": dial tcp 10.132.0.50:15021: connect: connection refused" Jul 12 18:56:32.588354 ip-10-0-129-128 kubenswrapper[2559]: I0712 18:56:32.588310 2559 patch_prober.go:28] interesting pod/e2e-api-a-045df0-openshift-default-cfcdff7b5-x8p7w container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.132.0.50:15021/healthz/ready\": dial tcp 10.132.0.50:15021: connect: connection refused" start-of-body= Jul 12 18:56:32.588794 ip-10-0-129-128 kubenswrapper[2559]: I0712 18:56:32.588381 2559 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-api-a-045df0-openshift-default-cfcdff7b5-x8p7w" podUID="6572fd4a-74f5-4571-acf1-5d69bebd6773" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.132.0.50:15021/healthz/ready\": dial tcp 10.132.0.50:15021: connect: connection refused" Jul 12 18:56:33.591584 ip-10-0-129-128 kubenswrapper[2559]: I0712 18:56:33.591552 2559 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="openshift-ingress/e2e-api-a-045df0-openshift-default-cfcdff7b5-x8p7w" Jul 12 18:56:33.592081 ip-10-0-129-128 kubenswrapper[2559]: I0712 18:56:33.591947 2559 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/e2e-api-a-045df0-openshift-default-cfcdff7b5-x8p7w" Jul 12 18:56:33.592740 ip-10-0-129-128 kubenswrapper[2559]: I0712 18:56:33.592719 2559 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/e2e-api-a-045df0-openshift-default-cfcdff7b5-x8p7w" Jul 12 18:56:35.499073 ip-10-0-129-128 kubenswrapper[2559]: I0712 18:56:35.499040 2559 generic.go:356] "Generic (PLEG): container finished" podID="6770b17c-88fb-45af-8337-00e99f7ecf19" containerID="6e3ece87bc50b4a78cd7af041c44227e1d8b8dbbad6fee3391dbccd9546ed55a" exitCode=0 Jul 12 18:56:35.499499 ip-10-0-129-128 kubenswrapper[2559]: I0712 18:56:35.499114 2559 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-api-b-a249e9-openshift-default-7bd58f74f-s7jfn" event={"ID":"6770b17c-88fb-45af-8337-00e99f7ecf19","Type":"ContainerDied","Data":"6e3ece87bc50b4a78cd7af041c44227e1d8b8dbbad6fee3391dbccd9546ed55a"} Jul 12 18:56:35.499546 ip-10-0-129-128 kubenswrapper[2559]: I0712 18:56:35.499498 2559 scope.go:117] "RemoveContainer" containerID="6e3ece87bc50b4a78cd7af041c44227e1d8b8dbbad6fee3391dbccd9546ed55a" Jul 12 18:56:35.500544 ip-10-0-129-128 kubenswrapper[2559]: I0712 18:56:35.500515 2559 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236228Ki","pods":"250"} Jul 12 18:56:35.500669 ip-10-0-129-128 kubenswrapper[2559]: I0712 18:56:35.500581 2559 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236228Ki","pods":"250"} Jul 12 18:56:35.500669 ip-10-0-129-128 kubenswrapper[2559]: I0712 18:56:35.500620 2559 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236228Ki","pods":"250"} Jul 12 18:56:35.984469 ip-10-0-129-128 kubenswrapper[2559]: I0712 18:56:35.984431 2559 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-pwsvh"] Jul 12 18:56:36.513462 ip-10-0-129-128 kubenswrapper[2559]: I0712 18:56:36.513426 2559 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-api-b-a249e9-openshift-default-7bd58f74f-s7jfn" event={"ID":"6770b17c-88fb-45af-8337-00e99f7ecf19","Type":"ContainerStarted","Data":"07624368555ce63e142839573b80949c0a47d75eab633b32881d1ffd6d107911"} Jul 12 18:56:36.839890 ip-10-0-129-128 kubenswrapper[2559]: I0712 18:56:36.839791 2559 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="openshift-ingress/e2e-api-b-a249e9-openshift-default-7bd58f74f-s7jfn" Jul 12 18:56:36.840232 ip-10-0-129-128 kubenswrapper[2559]: I0712 18:56:36.840204 2559 patch_prober.go:28] interesting pod/e2e-api-b-a249e9-openshift-default-7bd58f74f-s7jfn container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.132.0.51:15021/healthz/ready\": dial tcp 10.132.0.51:15021: connect: connection refused" start-of-body= Jul 12 18:56:36.840399 ip-10-0-129-128 kubenswrapper[2559]: I0712 18:56:36.840373 2559 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-api-b-a249e9-openshift-default-7bd58f74f-s7jfn" podUID="6770b17c-88fb-45af-8337-00e99f7ecf19" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.132.0.51:15021/healthz/ready\": dial tcp 10.132.0.51:15021: connect: connection refused" Jul 12 18:56:37.412103 ip-10-0-129-128 kubenswrapper[2559]: I0712 18:56:37.412063 2559 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/e2e-api-b-a249e9-openshift-default-7bd58f74f-s7jfn" Jul 12 18:56:37.839146 ip-10-0-129-128 kubenswrapper[2559]: I0712 18:56:37.838673 2559 patch_prober.go:28] interesting pod/e2e-api-b-a249e9-openshift-default-7bd58f74f-s7jfn container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.132.0.51:15021/healthz/ready\": dial tcp 10.132.0.51:15021: connect: connection refused" start-of-body= Jul 12 18:56:37.839146 ip-10-0-129-128 kubenswrapper[2559]: I0712 18:56:37.838740 2559 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-api-b-a249e9-openshift-default-7bd58f74f-s7jfn" podUID="6770b17c-88fb-45af-8337-00e99f7ecf19" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.132.0.51:15021/healthz/ready\": dial tcp 10.132.0.51:15021: connect: connection refused" Jul 12 18:56:38.839577 ip-10-0-129-128 kubenswrapper[2559]: I0712 18:56:38.839540 2559 patch_prober.go:28] interesting pod/e2e-api-b-a249e9-openshift-default-7bd58f74f-s7jfn container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.132.0.51:15021/healthz/ready\": dial tcp 10.132.0.51:15021: connect: connection refused" start-of-body= Jul 12 18:56:38.840737 ip-10-0-129-128 kubenswrapper[2559]: I0712 18:56:38.840299 2559 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-api-b-a249e9-openshift-default-7bd58f74f-s7jfn" podUID="6770b17c-88fb-45af-8337-00e99f7ecf19" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.132.0.51:15021/healthz/ready\": dial tcp 10.132.0.51:15021: connect: connection refused" Jul 12 18:56:39.839407 ip-10-0-129-128 kubenswrapper[2559]: I0712 18:56:39.839369 2559 patch_prober.go:28] interesting pod/e2e-api-b-a249e9-openshift-default-7bd58f74f-s7jfn container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.132.0.51:15021/healthz/ready\": dial tcp 10.132.0.51:15021: connect: connection refused" start-of-body= Jul 12 18:56:39.839733 ip-10-0-129-128 kubenswrapper[2559]: I0712 18:56:39.839703 2559 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-api-b-a249e9-openshift-default-7bd58f74f-s7jfn" podUID="6770b17c-88fb-45af-8337-00e99f7ecf19" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.132.0.51:15021/healthz/ready\": dial tcp 10.132.0.51:15021: connect: connection refused" Jul 12 18:56:40.842587 ip-10-0-129-128 kubenswrapper[2559]: I0712 18:56:40.842557 2559 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="openshift-ingress/e2e-api-b-a249e9-openshift-default-7bd58f74f-s7jfn" Jul 12 18:56:40.843560 ip-10-0-129-128 kubenswrapper[2559]: I0712 18:56:40.843538 2559 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/e2e-api-b-a249e9-openshift-default-7bd58f74f-s7jfn" Jul 12 18:58:13.275965 ip-10-0-129-128 kubenswrapper[2559]: I0712 18:58:13.275934 2559 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-console-operator_console-operator-7866497494-wlk78_f1d46430-bb60-4470-b8d8-eacd44f4f6b1/console-operator/1.log" Jul 12 18:58:13.278682 ip-10-0-129-128 kubenswrapper[2559]: I0712 18:58:13.278661 2559 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ovn-kubernetes_ovnkube-node-q5dw8_e51bad50-bc74-4e38-8307-ca257b599314/ovn-acl-logging/0.log" Jul 12 18:58:13.287762 ip-10-0-129-128 kubenswrapper[2559]: I0712 18:58:13.287740 2559 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-console-operator_console-operator-7866497494-wlk78_f1d46430-bb60-4470-b8d8-eacd44f4f6b1/console-operator/1.log" Jul 12 18:58:13.289838 ip-10-0-129-128 kubenswrapper[2559]: I0712 18:58:13.289821 2559 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ovn-kubernetes_ovnkube-node-q5dw8_e51bad50-bc74-4e38-8307-ca257b599314/ovn-acl-logging/0.log" Jul 12 18:59:44.286394 ip-10-0-129-128 kubenswrapper[2559]: I0712 18:59:44.286356 2559 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-pwsvh"] Jul 12 18:59:46.918425 ip-10-0-129-128 kubenswrapper[2559]: I0712 18:59:46.918388 2559 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-ingress/e2e-api-a-045df0-openshift-default-cfcdff7b5-x8p7w"] Jul 12 18:59:46.918893 ip-10-0-129-128 kubenswrapper[2559]: I0712 18:59:46.918673 2559 kuberuntime_container.go:873] "Killing container with a grace period" pod="openshift-ingress/e2e-api-a-045df0-openshift-default-cfcdff7b5-x8p7w" podUID="6572fd4a-74f5-4571-acf1-5d69bebd6773" containerName="istio-proxy" containerID="cri-o://ade918f555da67287a7f8891bb9ccb675bd6399301e41bcd62284fec66460f3f" gracePeriod=30 Jul 12 18:59:48.594404 ip-10-0-129-128 kubenswrapper[2559]: I0712 18:59:48.594365 2559 patch_prober.go:28] interesting pod/e2e-api-a-045df0-openshift-default-cfcdff7b5-x8p7w container/istio-proxy namespace/openshift-ingress: Readiness probe status=failure output="HTTP probe failed with statuscode: 503" start-of-body= Jul 12 18:59:48.594824 ip-10-0-129-128 kubenswrapper[2559]: I0712 18:59:48.594422 2559 prober.go:120] "Probe failed" probeType="Readiness" pod="openshift-ingress/e2e-api-a-045df0-openshift-default-cfcdff7b5-x8p7w" podUID="6572fd4a-74f5-4571-acf1-5d69bebd6773" containerName="istio-proxy" probeResult="failure" output="HTTP probe failed with statuscode: 503" Jul 12 18:59:52.173309 ip-10-0-129-128 kubenswrapper[2559]: I0712 18:59:52.173285 2559 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-api-a-045df0-openshift-default-cfcdff7b5-x8p7w" Jul 12 18:59:52.279108 ip-10-0-129-128 kubenswrapper[2559]: I0712 18:59:52.279072 2559 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/6572fd4a-74f5-4571-acf1-5d69bebd6773-credential-socket\") pod \"6572fd4a-74f5-4571-acf1-5d69bebd6773\" (UID: \"6572fd4a-74f5-4571-acf1-5d69bebd6773\") " Jul 12 18:59:52.279287 ip-10-0-129-128 kubenswrapper[2559]: I0712 18:59:52.279123 2559 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-wtnnd\" (UniqueName: \"kubernetes.io/projected/6572fd4a-74f5-4571-acf1-5d69bebd6773-kube-api-access-wtnnd\") pod \"6572fd4a-74f5-4571-acf1-5d69bebd6773\" (UID: \"6572fd4a-74f5-4571-acf1-5d69bebd6773\") " Jul 12 18:59:52.279287 ip-10-0-129-128 kubenswrapper[2559]: I0712 18:59:52.279144 2559 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/6572fd4a-74f5-4571-acf1-5d69bebd6773-istio-envoy\") pod \"6572fd4a-74f5-4571-acf1-5d69bebd6773\" (UID: \"6572fd4a-74f5-4571-acf1-5d69bebd6773\") " Jul 12 18:59:52.279287 ip-10-0-129-128 kubenswrapper[2559]: I0712 18:59:52.279167 2559 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/6572fd4a-74f5-4571-acf1-5d69bebd6773-workload-certs\") pod \"6572fd4a-74f5-4571-acf1-5d69bebd6773\" (UID: \"6572fd4a-74f5-4571-acf1-5d69bebd6773\") " Jul 12 18:59:52.279287 ip-10-0-129-128 kubenswrapper[2559]: I0712 18:59:52.279203 2559 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/6572fd4a-74f5-4571-acf1-5d69bebd6773-istio-data\") pod \"6572fd4a-74f5-4571-acf1-5d69bebd6773\" (UID: \"6572fd4a-74f5-4571-acf1-5d69bebd6773\") " Jul 12 18:59:52.279287 ip-10-0-129-128 kubenswrapper[2559]: I0712 18:59:52.279252 2559 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/6572fd4a-74f5-4571-acf1-5d69bebd6773-istio-token\") pod \"6572fd4a-74f5-4571-acf1-5d69bebd6773\" (UID: \"6572fd4a-74f5-4571-acf1-5d69bebd6773\") " Jul 12 18:59:52.279534 ip-10-0-129-128 kubenswrapper[2559]: I0712 18:59:52.279289 2559 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/6572fd4a-74f5-4571-acf1-5d69bebd6773-istiod-ca-cert\") pod \"6572fd4a-74f5-4571-acf1-5d69bebd6773\" (UID: \"6572fd4a-74f5-4571-acf1-5d69bebd6773\") " Jul 12 18:59:52.279534 ip-10-0-129-128 kubenswrapper[2559]: I0712 18:59:52.279311 2559 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/6572fd4a-74f5-4571-acf1-5d69bebd6773-istio-podinfo\") pod \"6572fd4a-74f5-4571-acf1-5d69bebd6773\" (UID: \"6572fd4a-74f5-4571-acf1-5d69bebd6773\") " Jul 12 18:59:52.279534 ip-10-0-129-128 kubenswrapper[2559]: I0712 18:59:52.279352 2559 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/6572fd4a-74f5-4571-acf1-5d69bebd6773-workload-socket\") pod \"6572fd4a-74f5-4571-acf1-5d69bebd6773\" (UID: \"6572fd4a-74f5-4571-acf1-5d69bebd6773\") " Jul 12 18:59:52.279534 ip-10-0-129-128 kubenswrapper[2559]: I0712 18:59:52.279381 2559 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/6572fd4a-74f5-4571-acf1-5d69bebd6773-credential-socket" (OuterVolumeSpecName: "credential-socket") pod "6572fd4a-74f5-4571-acf1-5d69bebd6773" (UID: "6572fd4a-74f5-4571-acf1-5d69bebd6773"). InnerVolumeSpecName "credential-socket". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Jul 12 18:59:52.279742 ip-10-0-129-128 kubenswrapper[2559]: I0712 18:59:52.279712 2559 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/6572fd4a-74f5-4571-acf1-5d69bebd6773-workload-socket" (OuterVolumeSpecName: "workload-socket") pod "6572fd4a-74f5-4571-acf1-5d69bebd6773" (UID: "6572fd4a-74f5-4571-acf1-5d69bebd6773"). InnerVolumeSpecName "workload-socket". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Jul 12 18:59:52.279876 ip-10-0-129-128 kubenswrapper[2559]: I0712 18:59:52.279845 2559 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/6572fd4a-74f5-4571-acf1-5d69bebd6773-istiod-ca-cert" (OuterVolumeSpecName: "istiod-ca-cert") pod "6572fd4a-74f5-4571-acf1-5d69bebd6773" (UID: "6572fd4a-74f5-4571-acf1-5d69bebd6773"). InnerVolumeSpecName "istiod-ca-cert". PluginName "kubernetes.io/configmap", VolumeGIDValue "" Jul 12 18:59:52.279998 ip-10-0-129-128 kubenswrapper[2559]: I0712 18:59:52.279945 2559 reconciler_common.go:299] "Volume detached for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/6572fd4a-74f5-4571-acf1-5d69bebd6773-credential-socket\") on node \"ip-10-0-129-128.ec2.internal\" DevicePath \"\"" Jul 12 18:59:52.279998 ip-10-0-129-128 kubenswrapper[2559]: I0712 18:59:52.279966 2559 reconciler_common.go:299] "Volume detached for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/6572fd4a-74f5-4571-acf1-5d69bebd6773-istiod-ca-cert\") on node \"ip-10-0-129-128.ec2.internal\" DevicePath \"\"" Jul 12 18:59:52.279998 ip-10-0-129-128 kubenswrapper[2559]: I0712 18:59:52.279982 2559 reconciler_common.go:299] "Volume detached for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/6572fd4a-74f5-4571-acf1-5d69bebd6773-workload-socket\") on node \"ip-10-0-129-128.ec2.internal\" DevicePath \"\"" Jul 12 18:59:52.280273 ip-10-0-129-128 kubenswrapper[2559]: I0712 18:59:52.280242 2559 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/6572fd4a-74f5-4571-acf1-5d69bebd6773-istio-data" (OuterVolumeSpecName: "istio-data") pod "6572fd4a-74f5-4571-acf1-5d69bebd6773" (UID: "6572fd4a-74f5-4571-acf1-5d69bebd6773"). InnerVolumeSpecName "istio-data". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Jul 12 18:59:52.280345 ip-10-0-129-128 kubenswrapper[2559]: I0712 18:59:52.280281 2559 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/6572fd4a-74f5-4571-acf1-5d69bebd6773-workload-certs" (OuterVolumeSpecName: "workload-certs") pod "6572fd4a-74f5-4571-acf1-5d69bebd6773" (UID: "6572fd4a-74f5-4571-acf1-5d69bebd6773"). InnerVolumeSpecName "workload-certs". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Jul 12 18:59:52.281444 ip-10-0-129-128 kubenswrapper[2559]: I0712 18:59:52.281417 2559 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/6572fd4a-74f5-4571-acf1-5d69bebd6773-kube-api-access-wtnnd" (OuterVolumeSpecName: "kube-api-access-wtnnd") pod "6572fd4a-74f5-4571-acf1-5d69bebd6773" (UID: "6572fd4a-74f5-4571-acf1-5d69bebd6773"). InnerVolumeSpecName "kube-api-access-wtnnd". PluginName "kubernetes.io/projected", VolumeGIDValue "" Jul 12 18:59:52.281538 ip-10-0-129-128 kubenswrapper[2559]: I0712 18:59:52.281501 2559 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/6572fd4a-74f5-4571-acf1-5d69bebd6773-istio-envoy" (OuterVolumeSpecName: "istio-envoy") pod "6572fd4a-74f5-4571-acf1-5d69bebd6773" (UID: "6572fd4a-74f5-4571-acf1-5d69bebd6773"). InnerVolumeSpecName "istio-envoy". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Jul 12 18:59:52.281538 ip-10-0-129-128 kubenswrapper[2559]: I0712 18:59:52.281517 2559 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/6572fd4a-74f5-4571-acf1-5d69bebd6773-istio-token" (OuterVolumeSpecName: "istio-token") pod "6572fd4a-74f5-4571-acf1-5d69bebd6773" (UID: "6572fd4a-74f5-4571-acf1-5d69bebd6773"). InnerVolumeSpecName "istio-token". PluginName "kubernetes.io/projected", VolumeGIDValue "" Jul 12 18:59:52.281825 ip-10-0-129-128 kubenswrapper[2559]: I0712 18:59:52.281807 2559 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/downward-api/6572fd4a-74f5-4571-acf1-5d69bebd6773-istio-podinfo" (OuterVolumeSpecName: "istio-podinfo") pod "6572fd4a-74f5-4571-acf1-5d69bebd6773" (UID: "6572fd4a-74f5-4571-acf1-5d69bebd6773"). InnerVolumeSpecName "istio-podinfo". PluginName "kubernetes.io/downward-api", VolumeGIDValue "" Jul 12 18:59:52.380561 ip-10-0-129-128 kubenswrapper[2559]: I0712 18:59:52.380524 2559 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-wtnnd\" (UniqueName: \"kubernetes.io/projected/6572fd4a-74f5-4571-acf1-5d69bebd6773-kube-api-access-wtnnd\") on node \"ip-10-0-129-128.ec2.internal\" DevicePath \"\"" Jul 12 18:59:52.380561 ip-10-0-129-128 kubenswrapper[2559]: I0712 18:59:52.380555 2559 reconciler_common.go:299] "Volume detached for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/6572fd4a-74f5-4571-acf1-5d69bebd6773-istio-envoy\") on node \"ip-10-0-129-128.ec2.internal\" DevicePath \"\"" Jul 12 18:59:52.380561 ip-10-0-129-128 kubenswrapper[2559]: I0712 18:59:52.380566 2559 reconciler_common.go:299] "Volume detached for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/6572fd4a-74f5-4571-acf1-5d69bebd6773-workload-certs\") on node \"ip-10-0-129-128.ec2.internal\" DevicePath \"\"" Jul 12 18:59:52.380842 ip-10-0-129-128 kubenswrapper[2559]: I0712 18:59:52.380574 2559 reconciler_common.go:299] "Volume detached for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/6572fd4a-74f5-4571-acf1-5d69bebd6773-istio-data\") on node \"ip-10-0-129-128.ec2.internal\" DevicePath \"\"" Jul 12 18:59:52.380842 ip-10-0-129-128 kubenswrapper[2559]: I0712 18:59:52.380582 2559 reconciler_common.go:299] "Volume detached for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/6572fd4a-74f5-4571-acf1-5d69bebd6773-istio-token\") on node \"ip-10-0-129-128.ec2.internal\" DevicePath \"\"" Jul 12 18:59:52.380842 ip-10-0-129-128 kubenswrapper[2559]: I0712 18:59:52.380589 2559 reconciler_common.go:299] "Volume detached for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/6572fd4a-74f5-4571-acf1-5d69bebd6773-istio-podinfo\") on node \"ip-10-0-129-128.ec2.internal\" DevicePath \"\"" Jul 12 18:59:52.467285 ip-10-0-129-128 kubenswrapper[2559]: I0712 18:59:52.467254 2559 generic.go:356] "Generic (PLEG): container finished" podID="6572fd4a-74f5-4571-acf1-5d69bebd6773" containerID="ade918f555da67287a7f8891bb9ccb675bd6399301e41bcd62284fec66460f3f" exitCode=0 Jul 12 18:59:52.467465 ip-10-0-129-128 kubenswrapper[2559]: I0712 18:59:52.467303 2559 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-api-a-045df0-openshift-default-cfcdff7b5-x8p7w" Jul 12 18:59:52.467465 ip-10-0-129-128 kubenswrapper[2559]: I0712 18:59:52.467309 2559 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-api-a-045df0-openshift-default-cfcdff7b5-x8p7w" event={"ID":"6572fd4a-74f5-4571-acf1-5d69bebd6773","Type":"ContainerDied","Data":"ade918f555da67287a7f8891bb9ccb675bd6399301e41bcd62284fec66460f3f"} Jul 12 18:59:52.467465 ip-10-0-129-128 kubenswrapper[2559]: I0712 18:59:52.467359 2559 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-api-a-045df0-openshift-default-cfcdff7b5-x8p7w" event={"ID":"6572fd4a-74f5-4571-acf1-5d69bebd6773","Type":"ContainerDied","Data":"e4c2478cf0bed116a52751db8a6946e8d5759a23e85cde850ff6e487ec80f770"} Jul 12 18:59:52.467465 ip-10-0-129-128 kubenswrapper[2559]: I0712 18:59:52.467382 2559 scope.go:117] "RemoveContainer" containerID="ade918f555da67287a7f8891bb9ccb675bd6399301e41bcd62284fec66460f3f" Jul 12 18:59:52.477757 ip-10-0-129-128 kubenswrapper[2559]: I0712 18:59:52.477741 2559 scope.go:117] "RemoveContainer" containerID="d40b60eb3247c3e1095ca431fbff1990500eb52f3a2d7ea86253cd3427972cd1" Jul 12 18:59:52.488453 ip-10-0-129-128 kubenswrapper[2559]: I0712 18:59:52.488429 2559 scope.go:117] "RemoveContainer" containerID="ade918f555da67287a7f8891bb9ccb675bd6399301e41bcd62284fec66460f3f" Jul 12 18:59:52.488709 ip-10-0-129-128 kubenswrapper[2559]: E0712 18:59:52.488686 2559 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"ade918f555da67287a7f8891bb9ccb675bd6399301e41bcd62284fec66460f3f\": container with ID starting with ade918f555da67287a7f8891bb9ccb675bd6399301e41bcd62284fec66460f3f not found: ID does not exist" containerID="ade918f555da67287a7f8891bb9ccb675bd6399301e41bcd62284fec66460f3f" Jul 12 18:59:52.488812 ip-10-0-129-128 kubenswrapper[2559]: I0712 18:59:52.488716 2559 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"ade918f555da67287a7f8891bb9ccb675bd6399301e41bcd62284fec66460f3f"} err="failed to get container status \"ade918f555da67287a7f8891bb9ccb675bd6399301e41bcd62284fec66460f3f\": rpc error: code = NotFound desc = could not find container \"ade918f555da67287a7f8891bb9ccb675bd6399301e41bcd62284fec66460f3f\": container with ID starting with ade918f555da67287a7f8891bb9ccb675bd6399301e41bcd62284fec66460f3f not found: ID does not exist" Jul 12 18:59:52.488812 ip-10-0-129-128 kubenswrapper[2559]: I0712 18:59:52.488732 2559 scope.go:117] "RemoveContainer" containerID="d40b60eb3247c3e1095ca431fbff1990500eb52f3a2d7ea86253cd3427972cd1" Jul 12 18:59:52.488965 ip-10-0-129-128 kubenswrapper[2559]: E0712 18:59:52.488948 2559 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"d40b60eb3247c3e1095ca431fbff1990500eb52f3a2d7ea86253cd3427972cd1\": container with ID starting with d40b60eb3247c3e1095ca431fbff1990500eb52f3a2d7ea86253cd3427972cd1 not found: ID does not exist" containerID="d40b60eb3247c3e1095ca431fbff1990500eb52f3a2d7ea86253cd3427972cd1" Jul 12 18:59:52.489033 ip-10-0-129-128 kubenswrapper[2559]: I0712 18:59:52.489006 2559 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"d40b60eb3247c3e1095ca431fbff1990500eb52f3a2d7ea86253cd3427972cd1"} err="failed to get container status \"d40b60eb3247c3e1095ca431fbff1990500eb52f3a2d7ea86253cd3427972cd1\": rpc error: code = NotFound desc = could not find container \"d40b60eb3247c3e1095ca431fbff1990500eb52f3a2d7ea86253cd3427972cd1\": container with ID starting with d40b60eb3247c3e1095ca431fbff1990500eb52f3a2d7ea86253cd3427972cd1 not found: ID does not exist" Jul 12 18:59:52.489701 ip-10-0-129-128 kubenswrapper[2559]: I0712 18:59:52.489590 2559 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-ingress/e2e-api-a-045df0-openshift-default-cfcdff7b5-x8p7w"] Jul 12 18:59:52.491432 ip-10-0-129-128 kubenswrapper[2559]: I0712 18:59:52.491410 2559 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["openshift-ingress/e2e-api-a-045df0-openshift-default-cfcdff7b5-x8p7w"] Jul 12 18:59:53.037434 ip-10-0-129-128 kubenswrapper[2559]: I0712 18:59:53.037401 2559 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="6572fd4a-74f5-4571-acf1-5d69bebd6773" path="/var/lib/kubelet/pods/6572fd4a-74f5-4571-acf1-5d69bebd6773/volumes" Jul 12 19:02:56.289682 ip-10-0-129-128 kubenswrapper[2559]: I0712 19:02:56.289634 2559 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-pwsvh"] Jul 12 19:02:59.071906 ip-10-0-129-128 kubenswrapper[2559]: I0712 19:02:59.071865 2559 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-ingress/e2e-api-b-a249e9-openshift-default-7bd58f74f-s7jfn"] Jul 12 19:02:59.072412 ip-10-0-129-128 kubenswrapper[2559]: I0712 19:02:59.072192 2559 kuberuntime_container.go:873] "Killing container with a grace period" pod="openshift-ingress/e2e-api-b-a249e9-openshift-default-7bd58f74f-s7jfn" podUID="6770b17c-88fb-45af-8337-00e99f7ecf19" containerName="istio-proxy" containerID="cri-o://07624368555ce63e142839573b80949c0a47d75eab633b32881d1ffd6d107911" gracePeriod=30 Jul 12 19:02:59.963804 ip-10-0-129-128 kubenswrapper[2559]: I0712 19:02:59.963768 2559 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-ingress/e2e-shared-a-e6a009-openshift-default-64dfb96979-vgx8n"] Jul 12 19:02:59.964238 ip-10-0-129-128 kubenswrapper[2559]: I0712 19:02:59.964219 2559 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="6572fd4a-74f5-4571-acf1-5d69bebd6773" containerName="istio-proxy" Jul 12 19:02:59.964336 ip-10-0-129-128 kubenswrapper[2559]: I0712 19:02:59.964242 2559 state_mem.go:107] "Deleted CPUSet assignment" podUID="6572fd4a-74f5-4571-acf1-5d69bebd6773" containerName="istio-proxy" Jul 12 19:02:59.964401 ip-10-0-129-128 kubenswrapper[2559]: I0712 19:02:59.964353 2559 memory_manager.go:368] "RemoveStaleState removing state" podUID="6572fd4a-74f5-4571-acf1-5d69bebd6773" containerName="istio-proxy" Jul 12 19:02:59.967407 ip-10-0-129-128 kubenswrapper[2559]: I0712 19:02:59.967386 2559 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-shared-a-e6a009-openshift-default-64dfb96979-vgx8n" Jul 12 19:02:59.969965 ip-10-0-129-128 kubenswrapper[2559]: I0712 19:02:59.969948 2559 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress\"/\"e2e-shared-a-e6a009-openshift-default-dockercfg-hngfx\"" Jul 12 19:02:59.978678 ip-10-0-129-128 kubenswrapper[2559]: I0712 19:02:59.978654 2559 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/e2e-shared-a-e6a009-openshift-default-64dfb96979-vgx8n"] Jul 12 19:03:00.032225 ip-10-0-129-128 kubenswrapper[2559]: I0712 19:03:00.032180 2559 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/fbbdd92f-9c45-4177-9527-9ff6999e4589-istiod-ca-cert\") pod \"e2e-shared-a-e6a009-openshift-default-64dfb96979-vgx8n\" (UID: \"fbbdd92f-9c45-4177-9527-9ff6999e4589\") " pod="openshift-ingress/e2e-shared-a-e6a009-openshift-default-64dfb96979-vgx8n" Jul 12 19:03:00.032225 ip-10-0-129-128 kubenswrapper[2559]: I0712 19:03:00.032222 2559 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-dvh2d\" (UniqueName: \"kubernetes.io/projected/fbbdd92f-9c45-4177-9527-9ff6999e4589-kube-api-access-dvh2d\") pod \"e2e-shared-a-e6a009-openshift-default-64dfb96979-vgx8n\" (UID: \"fbbdd92f-9c45-4177-9527-9ff6999e4589\") " pod="openshift-ingress/e2e-shared-a-e6a009-openshift-default-64dfb96979-vgx8n" Jul 12 19:03:00.032459 ip-10-0-129-128 kubenswrapper[2559]: I0712 19:03:00.032263 2559 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/fbbdd92f-9c45-4177-9527-9ff6999e4589-istio-podinfo\") pod \"e2e-shared-a-e6a009-openshift-default-64dfb96979-vgx8n\" (UID: \"fbbdd92f-9c45-4177-9527-9ff6999e4589\") " pod="openshift-ingress/e2e-shared-a-e6a009-openshift-default-64dfb96979-vgx8n" Jul 12 19:03:00.032459 ip-10-0-129-128 kubenswrapper[2559]: I0712 19:03:00.032305 2559 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/fbbdd92f-9c45-4177-9527-9ff6999e4589-workload-socket\") pod \"e2e-shared-a-e6a009-openshift-default-64dfb96979-vgx8n\" (UID: \"fbbdd92f-9c45-4177-9527-9ff6999e4589\") " pod="openshift-ingress/e2e-shared-a-e6a009-openshift-default-64dfb96979-vgx8n" Jul 12 19:03:00.032459 ip-10-0-129-128 kubenswrapper[2559]: I0712 19:03:00.032328 2559 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/fbbdd92f-9c45-4177-9527-9ff6999e4589-istio-envoy\") pod \"e2e-shared-a-e6a009-openshift-default-64dfb96979-vgx8n\" (UID: \"fbbdd92f-9c45-4177-9527-9ff6999e4589\") " pod="openshift-ingress/e2e-shared-a-e6a009-openshift-default-64dfb96979-vgx8n" Jul 12 19:03:00.032459 ip-10-0-129-128 kubenswrapper[2559]: I0712 19:03:00.032356 2559 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/fbbdd92f-9c45-4177-9527-9ff6999e4589-workload-certs\") pod \"e2e-shared-a-e6a009-openshift-default-64dfb96979-vgx8n\" (UID: \"fbbdd92f-9c45-4177-9527-9ff6999e4589\") " pod="openshift-ingress/e2e-shared-a-e6a009-openshift-default-64dfb96979-vgx8n" Jul 12 19:03:00.032459 ip-10-0-129-128 kubenswrapper[2559]: I0712 19:03:00.032371 2559 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/fbbdd92f-9c45-4177-9527-9ff6999e4589-istio-token\") pod \"e2e-shared-a-e6a009-openshift-default-64dfb96979-vgx8n\" (UID: \"fbbdd92f-9c45-4177-9527-9ff6999e4589\") " pod="openshift-ingress/e2e-shared-a-e6a009-openshift-default-64dfb96979-vgx8n" Jul 12 19:03:00.032636 ip-10-0-129-128 kubenswrapper[2559]: I0712 19:03:00.032473 2559 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/fbbdd92f-9c45-4177-9527-9ff6999e4589-istio-data\") pod \"e2e-shared-a-e6a009-openshift-default-64dfb96979-vgx8n\" (UID: \"fbbdd92f-9c45-4177-9527-9ff6999e4589\") " pod="openshift-ingress/e2e-shared-a-e6a009-openshift-default-64dfb96979-vgx8n" Jul 12 19:03:00.032636 ip-10-0-129-128 kubenswrapper[2559]: I0712 19:03:00.032531 2559 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/fbbdd92f-9c45-4177-9527-9ff6999e4589-credential-socket\") pod \"e2e-shared-a-e6a009-openshift-default-64dfb96979-vgx8n\" (UID: \"fbbdd92f-9c45-4177-9527-9ff6999e4589\") " pod="openshift-ingress/e2e-shared-a-e6a009-openshift-default-64dfb96979-vgx8n" Jul 12 19:03:00.133030 ip-10-0-129-128 kubenswrapper[2559]: I0712 19:03:00.132988 2559 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/fbbdd92f-9c45-4177-9527-9ff6999e4589-credential-socket\") pod \"e2e-shared-a-e6a009-openshift-default-64dfb96979-vgx8n\" (UID: \"fbbdd92f-9c45-4177-9527-9ff6999e4589\") " pod="openshift-ingress/e2e-shared-a-e6a009-openshift-default-64dfb96979-vgx8n" Jul 12 19:03:00.133518 ip-10-0-129-128 kubenswrapper[2559]: I0712 19:03:00.133044 2559 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/fbbdd92f-9c45-4177-9527-9ff6999e4589-istiod-ca-cert\") pod \"e2e-shared-a-e6a009-openshift-default-64dfb96979-vgx8n\" (UID: \"fbbdd92f-9c45-4177-9527-9ff6999e4589\") " pod="openshift-ingress/e2e-shared-a-e6a009-openshift-default-64dfb96979-vgx8n" Jul 12 19:03:00.133518 ip-10-0-129-128 kubenswrapper[2559]: I0712 19:03:00.133069 2559 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-dvh2d\" (UniqueName: \"kubernetes.io/projected/fbbdd92f-9c45-4177-9527-9ff6999e4589-kube-api-access-dvh2d\") pod \"e2e-shared-a-e6a009-openshift-default-64dfb96979-vgx8n\" (UID: \"fbbdd92f-9c45-4177-9527-9ff6999e4589\") " pod="openshift-ingress/e2e-shared-a-e6a009-openshift-default-64dfb96979-vgx8n" Jul 12 19:03:00.133518 ip-10-0-129-128 kubenswrapper[2559]: I0712 19:03:00.133127 2559 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/fbbdd92f-9c45-4177-9527-9ff6999e4589-istio-podinfo\") pod \"e2e-shared-a-e6a009-openshift-default-64dfb96979-vgx8n\" (UID: \"fbbdd92f-9c45-4177-9527-9ff6999e4589\") " pod="openshift-ingress/e2e-shared-a-e6a009-openshift-default-64dfb96979-vgx8n" Jul 12 19:03:00.133518 ip-10-0-129-128 kubenswrapper[2559]: I0712 19:03:00.133319 2559 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/fbbdd92f-9c45-4177-9527-9ff6999e4589-workload-socket\") pod \"e2e-shared-a-e6a009-openshift-default-64dfb96979-vgx8n\" (UID: \"fbbdd92f-9c45-4177-9527-9ff6999e4589\") " pod="openshift-ingress/e2e-shared-a-e6a009-openshift-default-64dfb96979-vgx8n" Jul 12 19:03:00.133518 ip-10-0-129-128 kubenswrapper[2559]: I0712 19:03:00.133366 2559 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/fbbdd92f-9c45-4177-9527-9ff6999e4589-istio-envoy\") pod \"e2e-shared-a-e6a009-openshift-default-64dfb96979-vgx8n\" (UID: \"fbbdd92f-9c45-4177-9527-9ff6999e4589\") " pod="openshift-ingress/e2e-shared-a-e6a009-openshift-default-64dfb96979-vgx8n" Jul 12 19:03:00.133518 ip-10-0-129-128 kubenswrapper[2559]: I0712 19:03:00.133422 2559 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/fbbdd92f-9c45-4177-9527-9ff6999e4589-workload-certs\") pod \"e2e-shared-a-e6a009-openshift-default-64dfb96979-vgx8n\" (UID: \"fbbdd92f-9c45-4177-9527-9ff6999e4589\") " pod="openshift-ingress/e2e-shared-a-e6a009-openshift-default-64dfb96979-vgx8n" Jul 12 19:03:00.133518 ip-10-0-129-128 kubenswrapper[2559]: I0712 19:03:00.133447 2559 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/fbbdd92f-9c45-4177-9527-9ff6999e4589-istio-token\") pod \"e2e-shared-a-e6a009-openshift-default-64dfb96979-vgx8n\" (UID: \"fbbdd92f-9c45-4177-9527-9ff6999e4589\") " pod="openshift-ingress/e2e-shared-a-e6a009-openshift-default-64dfb96979-vgx8n" Jul 12 19:03:00.133518 ip-10-0-129-128 kubenswrapper[2559]: I0712 19:03:00.133467 2559 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/fbbdd92f-9c45-4177-9527-9ff6999e4589-credential-socket\") pod \"e2e-shared-a-e6a009-openshift-default-64dfb96979-vgx8n\" (UID: \"fbbdd92f-9c45-4177-9527-9ff6999e4589\") " pod="openshift-ingress/e2e-shared-a-e6a009-openshift-default-64dfb96979-vgx8n" Jul 12 19:03:00.133974 ip-10-0-129-128 kubenswrapper[2559]: I0712 19:03:00.133524 2559 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/fbbdd92f-9c45-4177-9527-9ff6999e4589-istio-data\") pod \"e2e-shared-a-e6a009-openshift-default-64dfb96979-vgx8n\" (UID: \"fbbdd92f-9c45-4177-9527-9ff6999e4589\") " pod="openshift-ingress/e2e-shared-a-e6a009-openshift-default-64dfb96979-vgx8n" Jul 12 19:03:00.133974 ip-10-0-129-128 kubenswrapper[2559]: I0712 19:03:00.133775 2559 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/fbbdd92f-9c45-4177-9527-9ff6999e4589-workload-certs\") pod \"e2e-shared-a-e6a009-openshift-default-64dfb96979-vgx8n\" (UID: \"fbbdd92f-9c45-4177-9527-9ff6999e4589\") " pod="openshift-ingress/e2e-shared-a-e6a009-openshift-default-64dfb96979-vgx8n" Jul 12 19:03:00.133974 ip-10-0-129-128 kubenswrapper[2559]: I0712 19:03:00.133875 2559 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/fbbdd92f-9c45-4177-9527-9ff6999e4589-istio-data\") pod \"e2e-shared-a-e6a009-openshift-default-64dfb96979-vgx8n\" (UID: \"fbbdd92f-9c45-4177-9527-9ff6999e4589\") " pod="openshift-ingress/e2e-shared-a-e6a009-openshift-default-64dfb96979-vgx8n" Jul 12 19:03:00.134140 ip-10-0-129-128 kubenswrapper[2559]: I0712 19:03:00.133970 2559 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/fbbdd92f-9c45-4177-9527-9ff6999e4589-istiod-ca-cert\") pod \"e2e-shared-a-e6a009-openshift-default-64dfb96979-vgx8n\" (UID: \"fbbdd92f-9c45-4177-9527-9ff6999e4589\") " pod="openshift-ingress/e2e-shared-a-e6a009-openshift-default-64dfb96979-vgx8n" Jul 12 19:03:00.134140 ip-10-0-129-128 kubenswrapper[2559]: I0712 19:03:00.134019 2559 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/fbbdd92f-9c45-4177-9527-9ff6999e4589-workload-socket\") pod \"e2e-shared-a-e6a009-openshift-default-64dfb96979-vgx8n\" (UID: \"fbbdd92f-9c45-4177-9527-9ff6999e4589\") " pod="openshift-ingress/e2e-shared-a-e6a009-openshift-default-64dfb96979-vgx8n" Jul 12 19:03:00.135858 ip-10-0-129-128 kubenswrapper[2559]: I0712 19:03:00.135828 2559 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/fbbdd92f-9c45-4177-9527-9ff6999e4589-istio-envoy\") pod \"e2e-shared-a-e6a009-openshift-default-64dfb96979-vgx8n\" (UID: \"fbbdd92f-9c45-4177-9527-9ff6999e4589\") " pod="openshift-ingress/e2e-shared-a-e6a009-openshift-default-64dfb96979-vgx8n" Jul 12 19:03:00.136492 ip-10-0-129-128 kubenswrapper[2559]: I0712 19:03:00.136472 2559 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/fbbdd92f-9c45-4177-9527-9ff6999e4589-istio-podinfo\") pod \"e2e-shared-a-e6a009-openshift-default-64dfb96979-vgx8n\" (UID: \"fbbdd92f-9c45-4177-9527-9ff6999e4589\") " pod="openshift-ingress/e2e-shared-a-e6a009-openshift-default-64dfb96979-vgx8n" Jul 12 19:03:00.141320 ip-10-0-129-128 kubenswrapper[2559]: I0712 19:03:00.141297 2559 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/fbbdd92f-9c45-4177-9527-9ff6999e4589-istio-token\") pod \"e2e-shared-a-e6a009-openshift-default-64dfb96979-vgx8n\" (UID: \"fbbdd92f-9c45-4177-9527-9ff6999e4589\") " pod="openshift-ingress/e2e-shared-a-e6a009-openshift-default-64dfb96979-vgx8n" Jul 12 19:03:00.141558 ip-10-0-129-128 kubenswrapper[2559]: I0712 19:03:00.141538 2559 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-dvh2d\" (UniqueName: \"kubernetes.io/projected/fbbdd92f-9c45-4177-9527-9ff6999e4589-kube-api-access-dvh2d\") pod \"e2e-shared-a-e6a009-openshift-default-64dfb96979-vgx8n\" (UID: \"fbbdd92f-9c45-4177-9527-9ff6999e4589\") " pod="openshift-ingress/e2e-shared-a-e6a009-openshift-default-64dfb96979-vgx8n" Jul 12 19:03:00.278240 ip-10-0-129-128 kubenswrapper[2559]: I0712 19:03:00.278210 2559 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-shared-a-e6a009-openshift-default-64dfb96979-vgx8n" Jul 12 19:03:00.414471 ip-10-0-129-128 kubenswrapper[2559]: I0712 19:03:00.414444 2559 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/e2e-shared-a-e6a009-openshift-default-64dfb96979-vgx8n"] Jul 12 19:03:00.416527 ip-10-0-129-128 kubenswrapper[2559]: W0712 19:03:00.416503 2559 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podfbbdd92f_9c45_4177_9527_9ff6999e4589.slice/crio-30051032c954cc7fbd6557320f6216314d4fd25ba895b8c8329a523680143135 WatchSource:0}: Error finding container 30051032c954cc7fbd6557320f6216314d4fd25ba895b8c8329a523680143135: Status 404 returned error can't find the container with id 30051032c954cc7fbd6557320f6216314d4fd25ba895b8c8329a523680143135 Jul 12 19:03:00.418722 ip-10-0-129-128 kubenswrapper[2559]: I0712 19:03:00.418696 2559 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236228Ki","pods":"250"} Jul 12 19:03:00.418803 ip-10-0-129-128 kubenswrapper[2559]: I0712 19:03:00.418756 2559 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236228Ki","pods":"250"} Jul 12 19:03:00.418803 ip-10-0-129-128 kubenswrapper[2559]: I0712 19:03:00.418782 2559 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236228Ki","pods":"250"} Jul 12 19:03:01.364199 ip-10-0-129-128 kubenswrapper[2559]: I0712 19:03:01.364164 2559 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-shared-a-e6a009-openshift-default-64dfb96979-vgx8n" event={"ID":"fbbdd92f-9c45-4177-9527-9ff6999e4589","Type":"ContainerStarted","Data":"0ebc7347097ca115ea7802c36c2f847bcea4ed1c4c4583c5d53d08321a065e96"} Jul 12 19:03:01.364581 ip-10-0-129-128 kubenswrapper[2559]: I0712 19:03:01.364219 2559 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-shared-a-e6a009-openshift-default-64dfb96979-vgx8n" event={"ID":"fbbdd92f-9c45-4177-9527-9ff6999e4589","Type":"ContainerStarted","Data":"30051032c954cc7fbd6557320f6216314d4fd25ba895b8c8329a523680143135"} Jul 12 19:03:01.383796 ip-10-0-129-128 kubenswrapper[2559]: I0712 19:03:01.383752 2559 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-ingress/e2e-shared-a-e6a009-openshift-default-64dfb96979-vgx8n" podStartSLOduration=2.383737151 podStartE2EDuration="2.383737151s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-12 19:02:59 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-12 19:03:01.381508369 +0000 UTC m=+4188.875576299" watchObservedRunningTime="2026-07-12 19:03:01.383737151 +0000 UTC m=+4188.877805081" Jul 12 19:03:02.279266 ip-10-0-129-128 kubenswrapper[2559]: I0712 19:03:02.279222 2559 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="openshift-ingress/e2e-shared-a-e6a009-openshift-default-64dfb96979-vgx8n" Jul 12 19:03:02.284160 ip-10-0-129-128 kubenswrapper[2559]: I0712 19:03:02.284132 2559 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="openshift-ingress/e2e-shared-a-e6a009-openshift-default-64dfb96979-vgx8n" Jul 12 19:03:02.370075 ip-10-0-129-128 kubenswrapper[2559]: I0712 19:03:02.370034 2559 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/e2e-shared-a-e6a009-openshift-default-64dfb96979-vgx8n" Jul 12 19:03:02.371089 ip-10-0-129-128 kubenswrapper[2559]: I0712 19:03:02.371068 2559 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/e2e-shared-a-e6a009-openshift-default-64dfb96979-vgx8n" Jul 12 19:03:04.319535 ip-10-0-129-128 kubenswrapper[2559]: I0712 19:03:04.319507 2559 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-api-b-a249e9-openshift-default-7bd58f74f-s7jfn" Jul 12 19:03:04.372046 ip-10-0-129-128 kubenswrapper[2559]: I0712 19:03:04.371957 2559 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/6770b17c-88fb-45af-8337-00e99f7ecf19-workload-socket\") pod \"6770b17c-88fb-45af-8337-00e99f7ecf19\" (UID: \"6770b17c-88fb-45af-8337-00e99f7ecf19\") " Jul 12 19:03:04.372046 ip-10-0-129-128 kubenswrapper[2559]: I0712 19:03:04.371999 2559 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/6770b17c-88fb-45af-8337-00e99f7ecf19-istio-token\") pod \"6770b17c-88fb-45af-8337-00e99f7ecf19\" (UID: \"6770b17c-88fb-45af-8337-00e99f7ecf19\") " Jul 12 19:03:04.372046 ip-10-0-129-128 kubenswrapper[2559]: I0712 19:03:04.372034 2559 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/6770b17c-88fb-45af-8337-00e99f7ecf19-workload-certs\") pod \"6770b17c-88fb-45af-8337-00e99f7ecf19\" (UID: \"6770b17c-88fb-45af-8337-00e99f7ecf19\") " Jul 12 19:03:04.372351 ip-10-0-129-128 kubenswrapper[2559]: I0712 19:03:04.372055 2559 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/6770b17c-88fb-45af-8337-00e99f7ecf19-istiod-ca-cert\") pod \"6770b17c-88fb-45af-8337-00e99f7ecf19\" (UID: \"6770b17c-88fb-45af-8337-00e99f7ecf19\") " Jul 12 19:03:04.372351 ip-10-0-129-128 kubenswrapper[2559]: I0712 19:03:04.372080 2559 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/6770b17c-88fb-45af-8337-00e99f7ecf19-istio-envoy\") pod \"6770b17c-88fb-45af-8337-00e99f7ecf19\" (UID: \"6770b17c-88fb-45af-8337-00e99f7ecf19\") " Jul 12 19:03:04.372351 ip-10-0-129-128 kubenswrapper[2559]: I0712 19:03:04.372111 2559 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/6770b17c-88fb-45af-8337-00e99f7ecf19-credential-socket\") pod \"6770b17c-88fb-45af-8337-00e99f7ecf19\" (UID: \"6770b17c-88fb-45af-8337-00e99f7ecf19\") " Jul 12 19:03:04.372351 ip-10-0-129-128 kubenswrapper[2559]: I0712 19:03:04.372136 2559 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/6770b17c-88fb-45af-8337-00e99f7ecf19-istio-data\") pod \"6770b17c-88fb-45af-8337-00e99f7ecf19\" (UID: \"6770b17c-88fb-45af-8337-00e99f7ecf19\") " Jul 12 19:03:04.372351 ip-10-0-129-128 kubenswrapper[2559]: I0712 19:03:04.372204 2559 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/6770b17c-88fb-45af-8337-00e99f7ecf19-workload-socket" (OuterVolumeSpecName: "workload-socket") pod "6770b17c-88fb-45af-8337-00e99f7ecf19" (UID: "6770b17c-88fb-45af-8337-00e99f7ecf19"). InnerVolumeSpecName "workload-socket". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Jul 12 19:03:04.372351 ip-10-0-129-128 kubenswrapper[2559]: I0712 19:03:04.372272 2559 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/6770b17c-88fb-45af-8337-00e99f7ecf19-istio-podinfo\") pod \"6770b17c-88fb-45af-8337-00e99f7ecf19\" (UID: \"6770b17c-88fb-45af-8337-00e99f7ecf19\") " Jul 12 19:03:04.372351 ip-10-0-129-128 kubenswrapper[2559]: I0712 19:03:04.372298 2559 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-xtchv\" (UniqueName: \"kubernetes.io/projected/6770b17c-88fb-45af-8337-00e99f7ecf19-kube-api-access-xtchv\") pod \"6770b17c-88fb-45af-8337-00e99f7ecf19\" (UID: \"6770b17c-88fb-45af-8337-00e99f7ecf19\") " Jul 12 19:03:04.372877 ip-10-0-129-128 kubenswrapper[2559]: I0712 19:03:04.372407 2559 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/6770b17c-88fb-45af-8337-00e99f7ecf19-workload-certs" (OuterVolumeSpecName: "workload-certs") pod "6770b17c-88fb-45af-8337-00e99f7ecf19" (UID: "6770b17c-88fb-45af-8337-00e99f7ecf19"). InnerVolumeSpecName "workload-certs". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Jul 12 19:03:04.372877 ip-10-0-129-128 kubenswrapper[2559]: I0712 19:03:04.372601 2559 reconciler_common.go:299] "Volume detached for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/6770b17c-88fb-45af-8337-00e99f7ecf19-workload-socket\") on node \"ip-10-0-129-128.ec2.internal\" DevicePath \"\"" Jul 12 19:03:04.372877 ip-10-0-129-128 kubenswrapper[2559]: I0712 19:03:04.372619 2559 reconciler_common.go:299] "Volume detached for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/6770b17c-88fb-45af-8337-00e99f7ecf19-workload-certs\") on node \"ip-10-0-129-128.ec2.internal\" DevicePath \"\"" Jul 12 19:03:04.373224 ip-10-0-129-128 kubenswrapper[2559]: I0712 19:03:04.373095 2559 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/6770b17c-88fb-45af-8337-00e99f7ecf19-istio-data" (OuterVolumeSpecName: "istio-data") pod "6770b17c-88fb-45af-8337-00e99f7ecf19" (UID: "6770b17c-88fb-45af-8337-00e99f7ecf19"). InnerVolumeSpecName "istio-data". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Jul 12 19:03:04.373516 ip-10-0-129-128 kubenswrapper[2559]: I0712 19:03:04.373492 2559 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/6770b17c-88fb-45af-8337-00e99f7ecf19-istiod-ca-cert" (OuterVolumeSpecName: "istiod-ca-cert") pod "6770b17c-88fb-45af-8337-00e99f7ecf19" (UID: "6770b17c-88fb-45af-8337-00e99f7ecf19"). InnerVolumeSpecName "istiod-ca-cert". PluginName "kubernetes.io/configmap", VolumeGIDValue "" Jul 12 19:03:04.375014 ip-10-0-129-128 kubenswrapper[2559]: I0712 19:03:04.374988 2559 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/6770b17c-88fb-45af-8337-00e99f7ecf19-credential-socket" (OuterVolumeSpecName: "credential-socket") pod "6770b17c-88fb-45af-8337-00e99f7ecf19" (UID: "6770b17c-88fb-45af-8337-00e99f7ecf19"). InnerVolumeSpecName "credential-socket". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Jul 12 19:03:04.375204 ip-10-0-129-128 kubenswrapper[2559]: I0712 19:03:04.375183 2559 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/6770b17c-88fb-45af-8337-00e99f7ecf19-istio-token" (OuterVolumeSpecName: "istio-token") pod "6770b17c-88fb-45af-8337-00e99f7ecf19" (UID: "6770b17c-88fb-45af-8337-00e99f7ecf19"). InnerVolumeSpecName "istio-token". PluginName "kubernetes.io/projected", VolumeGIDValue "" Jul 12 19:03:04.375609 ip-10-0-129-128 kubenswrapper[2559]: I0712 19:03:04.375585 2559 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/6770b17c-88fb-45af-8337-00e99f7ecf19-kube-api-access-xtchv" (OuterVolumeSpecName: "kube-api-access-xtchv") pod "6770b17c-88fb-45af-8337-00e99f7ecf19" (UID: "6770b17c-88fb-45af-8337-00e99f7ecf19"). InnerVolumeSpecName "kube-api-access-xtchv". PluginName "kubernetes.io/projected", VolumeGIDValue "" Jul 12 19:03:04.375709 ip-10-0-129-128 kubenswrapper[2559]: I0712 19:03:04.375607 2559 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/downward-api/6770b17c-88fb-45af-8337-00e99f7ecf19-istio-podinfo" (OuterVolumeSpecName: "istio-podinfo") pod "6770b17c-88fb-45af-8337-00e99f7ecf19" (UID: "6770b17c-88fb-45af-8337-00e99f7ecf19"). InnerVolumeSpecName "istio-podinfo". PluginName "kubernetes.io/downward-api", VolumeGIDValue "" Jul 12 19:03:04.375875 ip-10-0-129-128 kubenswrapper[2559]: I0712 19:03:04.375853 2559 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/6770b17c-88fb-45af-8337-00e99f7ecf19-istio-envoy" (OuterVolumeSpecName: "istio-envoy") pod "6770b17c-88fb-45af-8337-00e99f7ecf19" (UID: "6770b17c-88fb-45af-8337-00e99f7ecf19"). InnerVolumeSpecName "istio-envoy". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Jul 12 19:03:04.381421 ip-10-0-129-128 kubenswrapper[2559]: I0712 19:03:04.381384 2559 generic.go:356] "Generic (PLEG): container finished" podID="6770b17c-88fb-45af-8337-00e99f7ecf19" containerID="07624368555ce63e142839573b80949c0a47d75eab633b32881d1ffd6d107911" exitCode=0 Jul 12 19:03:04.381512 ip-10-0-129-128 kubenswrapper[2559]: I0712 19:03:04.381441 2559 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-api-b-a249e9-openshift-default-7bd58f74f-s7jfn" Jul 12 19:03:04.381512 ip-10-0-129-128 kubenswrapper[2559]: I0712 19:03:04.381461 2559 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-api-b-a249e9-openshift-default-7bd58f74f-s7jfn" event={"ID":"6770b17c-88fb-45af-8337-00e99f7ecf19","Type":"ContainerDied","Data":"07624368555ce63e142839573b80949c0a47d75eab633b32881d1ffd6d107911"} Jul 12 19:03:04.381597 ip-10-0-129-128 kubenswrapper[2559]: I0712 19:03:04.381547 2559 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-api-b-a249e9-openshift-default-7bd58f74f-s7jfn" event={"ID":"6770b17c-88fb-45af-8337-00e99f7ecf19","Type":"ContainerDied","Data":"dc983ccb299fab2fd605cc93547325052b963e15ee2f504c2fe363a29e908a46"} Jul 12 19:03:04.381597 ip-10-0-129-128 kubenswrapper[2559]: I0712 19:03:04.381577 2559 scope.go:117] "RemoveContainer" containerID="07624368555ce63e142839573b80949c0a47d75eab633b32881d1ffd6d107911" Jul 12 19:03:04.396069 ip-10-0-129-128 kubenswrapper[2559]: I0712 19:03:04.396047 2559 scope.go:117] "RemoveContainer" containerID="6e3ece87bc50b4a78cd7af041c44227e1d8b8dbbad6fee3391dbccd9546ed55a" Jul 12 19:03:04.404106 ip-10-0-129-128 kubenswrapper[2559]: I0712 19:03:04.404090 2559 scope.go:117] "RemoveContainer" containerID="07624368555ce63e142839573b80949c0a47d75eab633b32881d1ffd6d107911" Jul 12 19:03:04.404375 ip-10-0-129-128 kubenswrapper[2559]: E0712 19:03:04.404356 2559 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"07624368555ce63e142839573b80949c0a47d75eab633b32881d1ffd6d107911\": container with ID starting with 07624368555ce63e142839573b80949c0a47d75eab633b32881d1ffd6d107911 not found: ID does not exist" containerID="07624368555ce63e142839573b80949c0a47d75eab633b32881d1ffd6d107911" Jul 12 19:03:04.404437 ip-10-0-129-128 kubenswrapper[2559]: I0712 19:03:04.404384 2559 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"07624368555ce63e142839573b80949c0a47d75eab633b32881d1ffd6d107911"} err="failed to get container status \"07624368555ce63e142839573b80949c0a47d75eab633b32881d1ffd6d107911\": rpc error: code = NotFound desc = could not find container \"07624368555ce63e142839573b80949c0a47d75eab633b32881d1ffd6d107911\": container with ID starting with 07624368555ce63e142839573b80949c0a47d75eab633b32881d1ffd6d107911 not found: ID does not exist" Jul 12 19:03:04.404437 ip-10-0-129-128 kubenswrapper[2559]: I0712 19:03:04.404404 2559 scope.go:117] "RemoveContainer" containerID="6e3ece87bc50b4a78cd7af041c44227e1d8b8dbbad6fee3391dbccd9546ed55a" Jul 12 19:03:04.404694 ip-10-0-129-128 kubenswrapper[2559]: E0712 19:03:04.404631 2559 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"6e3ece87bc50b4a78cd7af041c44227e1d8b8dbbad6fee3391dbccd9546ed55a\": container with ID starting with 6e3ece87bc50b4a78cd7af041c44227e1d8b8dbbad6fee3391dbccd9546ed55a not found: ID does not exist" containerID="6e3ece87bc50b4a78cd7af041c44227e1d8b8dbbad6fee3391dbccd9546ed55a" Jul 12 19:03:04.404694 ip-10-0-129-128 kubenswrapper[2559]: I0712 19:03:04.404686 2559 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"6e3ece87bc50b4a78cd7af041c44227e1d8b8dbbad6fee3391dbccd9546ed55a"} err="failed to get container status \"6e3ece87bc50b4a78cd7af041c44227e1d8b8dbbad6fee3391dbccd9546ed55a\": rpc error: code = NotFound desc = could not find container \"6e3ece87bc50b4a78cd7af041c44227e1d8b8dbbad6fee3391dbccd9546ed55a\": container with ID starting with 6e3ece87bc50b4a78cd7af041c44227e1d8b8dbbad6fee3391dbccd9546ed55a not found: ID does not exist" Jul 12 19:03:04.406261 ip-10-0-129-128 kubenswrapper[2559]: I0712 19:03:04.406241 2559 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-ingress/e2e-api-b-a249e9-openshift-default-7bd58f74f-s7jfn"] Jul 12 19:03:04.409700 ip-10-0-129-128 kubenswrapper[2559]: I0712 19:03:04.409681 2559 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["openshift-ingress/e2e-api-b-a249e9-openshift-default-7bd58f74f-s7jfn"] Jul 12 19:03:04.473462 ip-10-0-129-128 kubenswrapper[2559]: I0712 19:03:04.473428 2559 reconciler_common.go:299] "Volume detached for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/6770b17c-88fb-45af-8337-00e99f7ecf19-istio-token\") on node \"ip-10-0-129-128.ec2.internal\" DevicePath \"\"" Jul 12 19:03:04.473462 ip-10-0-129-128 kubenswrapper[2559]: I0712 19:03:04.473458 2559 reconciler_common.go:299] "Volume detached for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/6770b17c-88fb-45af-8337-00e99f7ecf19-istiod-ca-cert\") on node \"ip-10-0-129-128.ec2.internal\" DevicePath \"\"" Jul 12 19:03:04.473462 ip-10-0-129-128 kubenswrapper[2559]: I0712 19:03:04.473467 2559 reconciler_common.go:299] "Volume detached for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/6770b17c-88fb-45af-8337-00e99f7ecf19-istio-envoy\") on node \"ip-10-0-129-128.ec2.internal\" DevicePath \"\"" Jul 12 19:03:04.473709 ip-10-0-129-128 kubenswrapper[2559]: I0712 19:03:04.473475 2559 reconciler_common.go:299] "Volume detached for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/6770b17c-88fb-45af-8337-00e99f7ecf19-credential-socket\") on node \"ip-10-0-129-128.ec2.internal\" DevicePath \"\"" Jul 12 19:03:04.473709 ip-10-0-129-128 kubenswrapper[2559]: I0712 19:03:04.473488 2559 reconciler_common.go:299] "Volume detached for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/6770b17c-88fb-45af-8337-00e99f7ecf19-istio-data\") on node \"ip-10-0-129-128.ec2.internal\" DevicePath \"\"" Jul 12 19:03:04.473709 ip-10-0-129-128 kubenswrapper[2559]: I0712 19:03:04.473503 2559 reconciler_common.go:299] "Volume detached for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/6770b17c-88fb-45af-8337-00e99f7ecf19-istio-podinfo\") on node \"ip-10-0-129-128.ec2.internal\" DevicePath \"\"" Jul 12 19:03:04.473709 ip-10-0-129-128 kubenswrapper[2559]: I0712 19:03:04.473515 2559 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-xtchv\" (UniqueName: \"kubernetes.io/projected/6770b17c-88fb-45af-8337-00e99f7ecf19-kube-api-access-xtchv\") on node \"ip-10-0-129-128.ec2.internal\" DevicePath \"\"" Jul 12 19:03:05.037906 ip-10-0-129-128 kubenswrapper[2559]: I0712 19:03:05.037873 2559 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="6770b17c-88fb-45af-8337-00e99f7ecf19" path="/var/lib/kubelet/pods/6770b17c-88fb-45af-8337-00e99f7ecf19/volumes" Jul 12 19:03:07.893368 ip-10-0-129-128 kubenswrapper[2559]: I0712 19:03:07.893327 2559 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-pwsvh"] Jul 12 19:03:08.360441 ip-10-0-129-128 kubenswrapper[2559]: I0712 19:03:08.360408 2559 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-ingress/e2e-shared-b-53dad9-openshift-default-5d59cb9c4c-nnkps"] Jul 12 19:03:08.360811 ip-10-0-129-128 kubenswrapper[2559]: I0712 19:03:08.360795 2559 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="6770b17c-88fb-45af-8337-00e99f7ecf19" containerName="istio-proxy" Jul 12 19:03:08.360889 ip-10-0-129-128 kubenswrapper[2559]: I0712 19:03:08.360811 2559 state_mem.go:107] "Deleted CPUSet assignment" podUID="6770b17c-88fb-45af-8337-00e99f7ecf19" containerName="istio-proxy" Jul 12 19:03:08.360889 ip-10-0-129-128 kubenswrapper[2559]: I0712 19:03:08.360832 2559 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="6572fd4a-74f5-4571-acf1-5d69bebd6773" containerName="istio-proxy" Jul 12 19:03:08.360889 ip-10-0-129-128 kubenswrapper[2559]: I0712 19:03:08.360838 2559 state_mem.go:107] "Deleted CPUSet assignment" podUID="6572fd4a-74f5-4571-acf1-5d69bebd6773" containerName="istio-proxy" Jul 12 19:03:08.361046 ip-10-0-129-128 kubenswrapper[2559]: I0712 19:03:08.360893 2559 memory_manager.go:368] "RemoveStaleState removing state" podUID="6770b17c-88fb-45af-8337-00e99f7ecf19" containerName="istio-proxy" Jul 12 19:03:08.361046 ip-10-0-129-128 kubenswrapper[2559]: I0712 19:03:08.360902 2559 memory_manager.go:368] "RemoveStaleState removing state" podUID="6572fd4a-74f5-4571-acf1-5d69bebd6773" containerName="istio-proxy" Jul 12 19:03:08.365285 ip-10-0-129-128 kubenswrapper[2559]: I0712 19:03:08.365260 2559 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-shared-b-53dad9-openshift-default-5d59cb9c4c-nnkps" Jul 12 19:03:08.367563 ip-10-0-129-128 kubenswrapper[2559]: I0712 19:03:08.367542 2559 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress\"/\"e2e-shared-b-53dad9-openshift-default-dockercfg-59ccz\"" Jul 12 19:03:08.375292 ip-10-0-129-128 kubenswrapper[2559]: I0712 19:03:08.375272 2559 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/e2e-shared-b-53dad9-openshift-default-5d59cb9c4c-nnkps"] Jul 12 19:03:08.409654 ip-10-0-129-128 kubenswrapper[2559]: I0712 19:03:08.409607 2559 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/747c5adc-afa3-4f4c-85bc-e3181fb60ac4-workload-socket\") pod \"e2e-shared-b-53dad9-openshift-default-5d59cb9c4c-nnkps\" (UID: \"747c5adc-afa3-4f4c-85bc-e3181fb60ac4\") " pod="openshift-ingress/e2e-shared-b-53dad9-openshift-default-5d59cb9c4c-nnkps" Jul 12 19:03:08.409819 ip-10-0-129-128 kubenswrapper[2559]: I0712 19:03:08.409675 2559 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/747c5adc-afa3-4f4c-85bc-e3181fb60ac4-credential-socket\") pod \"e2e-shared-b-53dad9-openshift-default-5d59cb9c4c-nnkps\" (UID: \"747c5adc-afa3-4f4c-85bc-e3181fb60ac4\") " pod="openshift-ingress/e2e-shared-b-53dad9-openshift-default-5d59cb9c4c-nnkps" Jul 12 19:03:08.409819 ip-10-0-129-128 kubenswrapper[2559]: I0712 19:03:08.409704 2559 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/747c5adc-afa3-4f4c-85bc-e3181fb60ac4-istiod-ca-cert\") pod \"e2e-shared-b-53dad9-openshift-default-5d59cb9c4c-nnkps\" (UID: \"747c5adc-afa3-4f4c-85bc-e3181fb60ac4\") " pod="openshift-ingress/e2e-shared-b-53dad9-openshift-default-5d59cb9c4c-nnkps" Jul 12 19:03:08.409819 ip-10-0-129-128 kubenswrapper[2559]: I0712 19:03:08.409733 2559 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/747c5adc-afa3-4f4c-85bc-e3181fb60ac4-workload-certs\") pod \"e2e-shared-b-53dad9-openshift-default-5d59cb9c4c-nnkps\" (UID: \"747c5adc-afa3-4f4c-85bc-e3181fb60ac4\") " pod="openshift-ingress/e2e-shared-b-53dad9-openshift-default-5d59cb9c4c-nnkps" Jul 12 19:03:08.409819 ip-10-0-129-128 kubenswrapper[2559]: I0712 19:03:08.409748 2559 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-z56sw\" (UniqueName: \"kubernetes.io/projected/747c5adc-afa3-4f4c-85bc-e3181fb60ac4-kube-api-access-z56sw\") pod \"e2e-shared-b-53dad9-openshift-default-5d59cb9c4c-nnkps\" (UID: \"747c5adc-afa3-4f4c-85bc-e3181fb60ac4\") " pod="openshift-ingress/e2e-shared-b-53dad9-openshift-default-5d59cb9c4c-nnkps" Jul 12 19:03:08.409819 ip-10-0-129-128 kubenswrapper[2559]: I0712 19:03:08.409801 2559 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/747c5adc-afa3-4f4c-85bc-e3181fb60ac4-istio-token\") pod \"e2e-shared-b-53dad9-openshift-default-5d59cb9c4c-nnkps\" (UID: \"747c5adc-afa3-4f4c-85bc-e3181fb60ac4\") " pod="openshift-ingress/e2e-shared-b-53dad9-openshift-default-5d59cb9c4c-nnkps" Jul 12 19:03:08.410048 ip-10-0-129-128 kubenswrapper[2559]: I0712 19:03:08.409839 2559 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/747c5adc-afa3-4f4c-85bc-e3181fb60ac4-istio-envoy\") pod \"e2e-shared-b-53dad9-openshift-default-5d59cb9c4c-nnkps\" (UID: \"747c5adc-afa3-4f4c-85bc-e3181fb60ac4\") " pod="openshift-ingress/e2e-shared-b-53dad9-openshift-default-5d59cb9c4c-nnkps" Jul 12 19:03:08.410048 ip-10-0-129-128 kubenswrapper[2559]: I0712 19:03:08.409860 2559 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/747c5adc-afa3-4f4c-85bc-e3181fb60ac4-istio-podinfo\") pod \"e2e-shared-b-53dad9-openshift-default-5d59cb9c4c-nnkps\" (UID: \"747c5adc-afa3-4f4c-85bc-e3181fb60ac4\") " pod="openshift-ingress/e2e-shared-b-53dad9-openshift-default-5d59cb9c4c-nnkps" Jul 12 19:03:08.410048 ip-10-0-129-128 kubenswrapper[2559]: I0712 19:03:08.409888 2559 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/747c5adc-afa3-4f4c-85bc-e3181fb60ac4-istio-data\") pod \"e2e-shared-b-53dad9-openshift-default-5d59cb9c4c-nnkps\" (UID: \"747c5adc-afa3-4f4c-85bc-e3181fb60ac4\") " pod="openshift-ingress/e2e-shared-b-53dad9-openshift-default-5d59cb9c4c-nnkps" Jul 12 19:03:08.510772 ip-10-0-129-128 kubenswrapper[2559]: I0712 19:03:08.510732 2559 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/747c5adc-afa3-4f4c-85bc-e3181fb60ac4-credential-socket\") pod \"e2e-shared-b-53dad9-openshift-default-5d59cb9c4c-nnkps\" (UID: \"747c5adc-afa3-4f4c-85bc-e3181fb60ac4\") " pod="openshift-ingress/e2e-shared-b-53dad9-openshift-default-5d59cb9c4c-nnkps" Jul 12 19:03:08.510772 ip-10-0-129-128 kubenswrapper[2559]: I0712 19:03:08.510773 2559 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/747c5adc-afa3-4f4c-85bc-e3181fb60ac4-istiod-ca-cert\") pod \"e2e-shared-b-53dad9-openshift-default-5d59cb9c4c-nnkps\" (UID: \"747c5adc-afa3-4f4c-85bc-e3181fb60ac4\") " pod="openshift-ingress/e2e-shared-b-53dad9-openshift-default-5d59cb9c4c-nnkps" Jul 12 19:03:08.511020 ip-10-0-129-128 kubenswrapper[2559]: I0712 19:03:08.510817 2559 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/747c5adc-afa3-4f4c-85bc-e3181fb60ac4-workload-certs\") pod \"e2e-shared-b-53dad9-openshift-default-5d59cb9c4c-nnkps\" (UID: \"747c5adc-afa3-4f4c-85bc-e3181fb60ac4\") " pod="openshift-ingress/e2e-shared-b-53dad9-openshift-default-5d59cb9c4c-nnkps" Jul 12 19:03:08.511020 ip-10-0-129-128 kubenswrapper[2559]: I0712 19:03:08.510841 2559 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-z56sw\" (UniqueName: \"kubernetes.io/projected/747c5adc-afa3-4f4c-85bc-e3181fb60ac4-kube-api-access-z56sw\") pod \"e2e-shared-b-53dad9-openshift-default-5d59cb9c4c-nnkps\" (UID: \"747c5adc-afa3-4f4c-85bc-e3181fb60ac4\") " pod="openshift-ingress/e2e-shared-b-53dad9-openshift-default-5d59cb9c4c-nnkps" Jul 12 19:03:08.511020 ip-10-0-129-128 kubenswrapper[2559]: I0712 19:03:08.510869 2559 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/747c5adc-afa3-4f4c-85bc-e3181fb60ac4-istio-token\") pod \"e2e-shared-b-53dad9-openshift-default-5d59cb9c4c-nnkps\" (UID: \"747c5adc-afa3-4f4c-85bc-e3181fb60ac4\") " pod="openshift-ingress/e2e-shared-b-53dad9-openshift-default-5d59cb9c4c-nnkps" Jul 12 19:03:08.511020 ip-10-0-129-128 kubenswrapper[2559]: I0712 19:03:08.510921 2559 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/747c5adc-afa3-4f4c-85bc-e3181fb60ac4-istio-envoy\") pod \"e2e-shared-b-53dad9-openshift-default-5d59cb9c4c-nnkps\" (UID: \"747c5adc-afa3-4f4c-85bc-e3181fb60ac4\") " pod="openshift-ingress/e2e-shared-b-53dad9-openshift-default-5d59cb9c4c-nnkps" Jul 12 19:03:08.511020 ip-10-0-129-128 kubenswrapper[2559]: I0712 19:03:08.510951 2559 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/747c5adc-afa3-4f4c-85bc-e3181fb60ac4-istio-podinfo\") pod \"e2e-shared-b-53dad9-openshift-default-5d59cb9c4c-nnkps\" (UID: \"747c5adc-afa3-4f4c-85bc-e3181fb60ac4\") " pod="openshift-ingress/e2e-shared-b-53dad9-openshift-default-5d59cb9c4c-nnkps" Jul 12 19:03:08.511020 ip-10-0-129-128 kubenswrapper[2559]: I0712 19:03:08.510995 2559 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/747c5adc-afa3-4f4c-85bc-e3181fb60ac4-istio-data\") pod \"e2e-shared-b-53dad9-openshift-default-5d59cb9c4c-nnkps\" (UID: \"747c5adc-afa3-4f4c-85bc-e3181fb60ac4\") " pod="openshift-ingress/e2e-shared-b-53dad9-openshift-default-5d59cb9c4c-nnkps" Jul 12 19:03:08.511360 ip-10-0-129-128 kubenswrapper[2559]: I0712 19:03:08.511031 2559 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/747c5adc-afa3-4f4c-85bc-e3181fb60ac4-workload-socket\") pod \"e2e-shared-b-53dad9-openshift-default-5d59cb9c4c-nnkps\" (UID: \"747c5adc-afa3-4f4c-85bc-e3181fb60ac4\") " pod="openshift-ingress/e2e-shared-b-53dad9-openshift-default-5d59cb9c4c-nnkps" Jul 12 19:03:08.511360 ip-10-0-129-128 kubenswrapper[2559]: I0712 19:03:08.511197 2559 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/747c5adc-afa3-4f4c-85bc-e3181fb60ac4-credential-socket\") pod \"e2e-shared-b-53dad9-openshift-default-5d59cb9c4c-nnkps\" (UID: \"747c5adc-afa3-4f4c-85bc-e3181fb60ac4\") " pod="openshift-ingress/e2e-shared-b-53dad9-openshift-default-5d59cb9c4c-nnkps" Jul 12 19:03:08.511360 ip-10-0-129-128 kubenswrapper[2559]: I0712 19:03:08.511318 2559 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/747c5adc-afa3-4f4c-85bc-e3181fb60ac4-workload-socket\") pod \"e2e-shared-b-53dad9-openshift-default-5d59cb9c4c-nnkps\" (UID: \"747c5adc-afa3-4f4c-85bc-e3181fb60ac4\") " pod="openshift-ingress/e2e-shared-b-53dad9-openshift-default-5d59cb9c4c-nnkps" Jul 12 19:03:08.511488 ip-10-0-129-128 kubenswrapper[2559]: I0712 19:03:08.511412 2559 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/747c5adc-afa3-4f4c-85bc-e3181fb60ac4-workload-certs\") pod \"e2e-shared-b-53dad9-openshift-default-5d59cb9c4c-nnkps\" (UID: \"747c5adc-afa3-4f4c-85bc-e3181fb60ac4\") " pod="openshift-ingress/e2e-shared-b-53dad9-openshift-default-5d59cb9c4c-nnkps" Jul 12 19:03:08.511488 ip-10-0-129-128 kubenswrapper[2559]: I0712 19:03:08.511476 2559 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/747c5adc-afa3-4f4c-85bc-e3181fb60ac4-istio-data\") pod \"e2e-shared-b-53dad9-openshift-default-5d59cb9c4c-nnkps\" (UID: \"747c5adc-afa3-4f4c-85bc-e3181fb60ac4\") " pod="openshift-ingress/e2e-shared-b-53dad9-openshift-default-5d59cb9c4c-nnkps" Jul 12 19:03:08.511579 ip-10-0-129-128 kubenswrapper[2559]: I0712 19:03:08.511562 2559 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/747c5adc-afa3-4f4c-85bc-e3181fb60ac4-istiod-ca-cert\") pod \"e2e-shared-b-53dad9-openshift-default-5d59cb9c4c-nnkps\" (UID: \"747c5adc-afa3-4f4c-85bc-e3181fb60ac4\") " pod="openshift-ingress/e2e-shared-b-53dad9-openshift-default-5d59cb9c4c-nnkps" Jul 12 19:03:08.513299 ip-10-0-129-128 kubenswrapper[2559]: I0712 19:03:08.513279 2559 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/747c5adc-afa3-4f4c-85bc-e3181fb60ac4-istio-envoy\") pod \"e2e-shared-b-53dad9-openshift-default-5d59cb9c4c-nnkps\" (UID: \"747c5adc-afa3-4f4c-85bc-e3181fb60ac4\") " pod="openshift-ingress/e2e-shared-b-53dad9-openshift-default-5d59cb9c4c-nnkps" Jul 12 19:03:08.513364 ip-10-0-129-128 kubenswrapper[2559]: I0712 19:03:08.513312 2559 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/747c5adc-afa3-4f4c-85bc-e3181fb60ac4-istio-podinfo\") pod \"e2e-shared-b-53dad9-openshift-default-5d59cb9c4c-nnkps\" (UID: \"747c5adc-afa3-4f4c-85bc-e3181fb60ac4\") " pod="openshift-ingress/e2e-shared-b-53dad9-openshift-default-5d59cb9c4c-nnkps" Jul 12 19:03:08.518216 ip-10-0-129-128 kubenswrapper[2559]: I0712 19:03:08.518183 2559 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/747c5adc-afa3-4f4c-85bc-e3181fb60ac4-istio-token\") pod \"e2e-shared-b-53dad9-openshift-default-5d59cb9c4c-nnkps\" (UID: \"747c5adc-afa3-4f4c-85bc-e3181fb60ac4\") " pod="openshift-ingress/e2e-shared-b-53dad9-openshift-default-5d59cb9c4c-nnkps" Jul 12 19:03:08.518411 ip-10-0-129-128 kubenswrapper[2559]: I0712 19:03:08.518395 2559 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-z56sw\" (UniqueName: \"kubernetes.io/projected/747c5adc-afa3-4f4c-85bc-e3181fb60ac4-kube-api-access-z56sw\") pod \"e2e-shared-b-53dad9-openshift-default-5d59cb9c4c-nnkps\" (UID: \"747c5adc-afa3-4f4c-85bc-e3181fb60ac4\") " pod="openshift-ingress/e2e-shared-b-53dad9-openshift-default-5d59cb9c4c-nnkps" Jul 12 19:03:08.676557 ip-10-0-129-128 kubenswrapper[2559]: I0712 19:03:08.676464 2559 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-shared-b-53dad9-openshift-default-5d59cb9c4c-nnkps" Jul 12 19:03:08.817084 ip-10-0-129-128 kubenswrapper[2559]: I0712 19:03:08.817060 2559 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/e2e-shared-b-53dad9-openshift-default-5d59cb9c4c-nnkps"] Jul 12 19:03:08.818705 ip-10-0-129-128 kubenswrapper[2559]: W0712 19:03:08.818678 2559 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod747c5adc_afa3_4f4c_85bc_e3181fb60ac4.slice/crio-ac8f3d64219ede4bff9973812c424b3c4e8469b6c22fc56a3b739d8addac31ff WatchSource:0}: Error finding container ac8f3d64219ede4bff9973812c424b3c4e8469b6c22fc56a3b739d8addac31ff: Status 404 returned error can't find the container with id ac8f3d64219ede4bff9973812c424b3c4e8469b6c22fc56a3b739d8addac31ff Jul 12 19:03:08.820900 ip-10-0-129-128 kubenswrapper[2559]: I0712 19:03:08.820867 2559 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236228Ki","pods":"250"} Jul 12 19:03:08.820997 ip-10-0-129-128 kubenswrapper[2559]: I0712 19:03:08.820931 2559 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236228Ki","pods":"250"} Jul 12 19:03:08.820997 ip-10-0-129-128 kubenswrapper[2559]: I0712 19:03:08.820956 2559 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236228Ki","pods":"250"} Jul 12 19:03:09.408563 ip-10-0-129-128 kubenswrapper[2559]: I0712 19:03:09.408522 2559 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-shared-b-53dad9-openshift-default-5d59cb9c4c-nnkps" event={"ID":"747c5adc-afa3-4f4c-85bc-e3181fb60ac4","Type":"ContainerStarted","Data":"be1da2d2c5579643b608ef46ecbadf2273cf794909e23a7fe715234537efc34e"} Jul 12 19:03:09.409027 ip-10-0-129-128 kubenswrapper[2559]: I0712 19:03:09.408591 2559 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-shared-b-53dad9-openshift-default-5d59cb9c4c-nnkps" event={"ID":"747c5adc-afa3-4f4c-85bc-e3181fb60ac4","Type":"ContainerStarted","Data":"ac8f3d64219ede4bff9973812c424b3c4e8469b6c22fc56a3b739d8addac31ff"} Jul 12 19:03:09.677787 ip-10-0-129-128 kubenswrapper[2559]: I0712 19:03:09.677682 2559 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="openshift-ingress/e2e-shared-b-53dad9-openshift-default-5d59cb9c4c-nnkps" Jul 12 19:03:09.682768 ip-10-0-129-128 kubenswrapper[2559]: I0712 19:03:09.682746 2559 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="openshift-ingress/e2e-shared-b-53dad9-openshift-default-5d59cb9c4c-nnkps" Jul 12 19:03:09.702187 ip-10-0-129-128 kubenswrapper[2559]: I0712 19:03:09.702144 2559 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-ingress/e2e-shared-b-53dad9-openshift-default-5d59cb9c4c-nnkps" podStartSLOduration=1.702128596 podStartE2EDuration="1.702128596s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-12 19:03:08 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-12 19:03:09.434317473 +0000 UTC m=+4196.928385403" watchObservedRunningTime="2026-07-12 19:03:09.702128596 +0000 UTC m=+4197.196196526" Jul 12 19:03:10.413577 ip-10-0-129-128 kubenswrapper[2559]: I0712 19:03:10.413538 2559 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/e2e-shared-b-53dad9-openshift-default-5d59cb9c4c-nnkps" Jul 12 19:03:10.414690 ip-10-0-129-128 kubenswrapper[2559]: I0712 19:03:10.414666 2559 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/e2e-shared-b-53dad9-openshift-default-5d59cb9c4c-nnkps" Jul 12 19:03:13.305625 ip-10-0-129-128 kubenswrapper[2559]: I0712 19:03:13.305591 2559 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-console-operator_console-operator-7866497494-wlk78_f1d46430-bb60-4470-b8d8-eacd44f4f6b1/console-operator/1.log" Jul 12 19:03:13.308578 ip-10-0-129-128 kubenswrapper[2559]: I0712 19:03:13.308551 2559 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ovn-kubernetes_ovnkube-node-q5dw8_e51bad50-bc74-4e38-8307-ca257b599314/ovn-acl-logging/0.log" Jul 12 19:03:13.323340 ip-10-0-129-128 kubenswrapper[2559]: I0712 19:03:13.323311 2559 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-console-operator_console-operator-7866497494-wlk78_f1d46430-bb60-4470-b8d8-eacd44f4f6b1/console-operator/1.log" Jul 12 19:03:13.325827 ip-10-0-129-128 kubenswrapper[2559]: I0712 19:03:13.325809 2559 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ovn-kubernetes_ovnkube-node-q5dw8_e51bad50-bc74-4e38-8307-ca257b599314/ovn-acl-logging/0.log" Jul 12 19:03:16.446290 ip-10-0-129-128 kubenswrapper[2559]: I0712 19:03:16.446244 2559 generic.go:356] "Generic (PLEG): container finished" podID="fbbdd92f-9c45-4177-9527-9ff6999e4589" containerID="0ebc7347097ca115ea7802c36c2f847bcea4ed1c4c4583c5d53d08321a065e96" exitCode=0 Jul 12 19:03:16.446888 ip-10-0-129-128 kubenswrapper[2559]: I0712 19:03:16.446367 2559 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-shared-a-e6a009-openshift-default-64dfb96979-vgx8n" event={"ID":"fbbdd92f-9c45-4177-9527-9ff6999e4589","Type":"ContainerDied","Data":"0ebc7347097ca115ea7802c36c2f847bcea4ed1c4c4583c5d53d08321a065e96"} Jul 12 19:03:16.446888 ip-10-0-129-128 kubenswrapper[2559]: I0712 19:03:16.446868 2559 scope.go:117] "RemoveContainer" containerID="0ebc7347097ca115ea7802c36c2f847bcea4ed1c4c4583c5d53d08321a065e96" Jul 12 19:03:16.451774 ip-10-0-129-128 kubenswrapper[2559]: I0712 19:03:16.451730 2559 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236228Ki","pods":"250"} Jul 12 19:03:16.451896 ip-10-0-129-128 kubenswrapper[2559]: I0712 19:03:16.451833 2559 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236228Ki","pods":"250"} Jul 12 19:03:16.451896 ip-10-0-129-128 kubenswrapper[2559]: I0712 19:03:16.451881 2559 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236228Ki","pods":"250"} Jul 12 19:03:17.370361 ip-10-0-129-128 kubenswrapper[2559]: I0712 19:03:17.370330 2559 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/e2e-shared-a-e6a009-openshift-default-64dfb96979-vgx8n" Jul 12 19:03:17.458104 ip-10-0-129-128 kubenswrapper[2559]: I0712 19:03:17.457337 2559 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-shared-a-e6a009-openshift-default-64dfb96979-vgx8n" event={"ID":"fbbdd92f-9c45-4177-9527-9ff6999e4589","Type":"ContainerStarted","Data":"470aa5319e6bc714193236dd860402144dde5a440926b85db8c01fce7b8747c4"} Jul 12 19:03:18.279327 ip-10-0-129-128 kubenswrapper[2559]: I0712 19:03:18.279282 2559 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="openshift-ingress/e2e-shared-a-e6a009-openshift-default-64dfb96979-vgx8n" Jul 12 19:03:18.281046 ip-10-0-129-128 kubenswrapper[2559]: I0712 19:03:18.281018 2559 patch_prober.go:28] interesting pod/e2e-shared-a-e6a009-openshift-default-64dfb96979-vgx8n container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.132.0.52:15021/healthz/ready\": dial tcp 10.132.0.52:15021: connect: connection refused" start-of-body= Jul 12 19:03:18.281209 ip-10-0-129-128 kubenswrapper[2559]: I0712 19:03:18.281070 2559 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-shared-a-e6a009-openshift-default-64dfb96979-vgx8n" podUID="fbbdd92f-9c45-4177-9527-9ff6999e4589" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.132.0.52:15021/healthz/ready\": dial tcp 10.132.0.52:15021: connect: connection refused" Jul 12 19:03:19.279578 ip-10-0-129-128 kubenswrapper[2559]: I0712 19:03:19.279539 2559 patch_prober.go:28] interesting pod/e2e-shared-a-e6a009-openshift-default-64dfb96979-vgx8n container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.132.0.52:15021/healthz/ready\": dial tcp 10.132.0.52:15021: connect: connection refused" start-of-body= Jul 12 19:03:19.280198 ip-10-0-129-128 kubenswrapper[2559]: I0712 19:03:19.280164 2559 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-shared-a-e6a009-openshift-default-64dfb96979-vgx8n" podUID="fbbdd92f-9c45-4177-9527-9ff6999e4589" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.132.0.52:15021/healthz/ready\": dial tcp 10.132.0.52:15021: connect: connection refused" Jul 12 19:03:20.278749 ip-10-0-129-128 kubenswrapper[2559]: I0712 19:03:20.278716 2559 patch_prober.go:28] interesting pod/e2e-shared-a-e6a009-openshift-default-64dfb96979-vgx8n container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.132.0.52:15021/healthz/ready\": dial tcp 10.132.0.52:15021: connect: connection refused" start-of-body= Jul 12 19:03:20.278994 ip-10-0-129-128 kubenswrapper[2559]: I0712 19:03:20.278778 2559 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-shared-a-e6a009-openshift-default-64dfb96979-vgx8n" podUID="fbbdd92f-9c45-4177-9527-9ff6999e4589" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.132.0.52:15021/healthz/ready\": dial tcp 10.132.0.52:15021: connect: connection refused" Jul 12 19:03:21.279460 ip-10-0-129-128 kubenswrapper[2559]: I0712 19:03:21.279429 2559 patch_prober.go:28] interesting pod/e2e-shared-a-e6a009-openshift-default-64dfb96979-vgx8n container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.132.0.52:15021/healthz/ready\": dial tcp 10.132.0.52:15021: connect: connection refused" start-of-body= Jul 12 19:03:21.279940 ip-10-0-129-128 kubenswrapper[2559]: I0712 19:03:21.279488 2559 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-shared-a-e6a009-openshift-default-64dfb96979-vgx8n" podUID="fbbdd92f-9c45-4177-9527-9ff6999e4589" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.132.0.52:15021/healthz/ready\": dial tcp 10.132.0.52:15021: connect: connection refused" Jul 12 19:03:22.283142 ip-10-0-129-128 kubenswrapper[2559]: I0712 19:03:22.283110 2559 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="openshift-ingress/e2e-shared-a-e6a009-openshift-default-64dfb96979-vgx8n" Jul 12 19:03:22.283537 ip-10-0-129-128 kubenswrapper[2559]: I0712 19:03:22.283396 2559 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/e2e-shared-a-e6a009-openshift-default-64dfb96979-vgx8n" Jul 12 19:03:22.284243 ip-10-0-129-128 kubenswrapper[2559]: I0712 19:03:22.284223 2559 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/e2e-shared-a-e6a009-openshift-default-64dfb96979-vgx8n" Jul 12 19:03:23.495263 ip-10-0-129-128 kubenswrapper[2559]: I0712 19:03:23.495230 2559 generic.go:356] "Generic (PLEG): container finished" podID="747c5adc-afa3-4f4c-85bc-e3181fb60ac4" containerID="be1da2d2c5579643b608ef46ecbadf2273cf794909e23a7fe715234537efc34e" exitCode=0 Jul 12 19:03:23.495706 ip-10-0-129-128 kubenswrapper[2559]: I0712 19:03:23.495304 2559 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-shared-b-53dad9-openshift-default-5d59cb9c4c-nnkps" event={"ID":"747c5adc-afa3-4f4c-85bc-e3181fb60ac4","Type":"ContainerDied","Data":"be1da2d2c5579643b608ef46ecbadf2273cf794909e23a7fe715234537efc34e"} Jul 12 19:03:23.495804 ip-10-0-129-128 kubenswrapper[2559]: I0712 19:03:23.495786 2559 scope.go:117] "RemoveContainer" containerID="be1da2d2c5579643b608ef46ecbadf2273cf794909e23a7fe715234537efc34e" Jul 12 19:03:23.496729 ip-10-0-129-128 kubenswrapper[2559]: I0712 19:03:23.496703 2559 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236228Ki","pods":"250"} Jul 12 19:03:23.496790 ip-10-0-129-128 kubenswrapper[2559]: I0712 19:03:23.496768 2559 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236228Ki","pods":"250"} Jul 12 19:03:23.496826 ip-10-0-129-128 kubenswrapper[2559]: I0712 19:03:23.496799 2559 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236228Ki","pods":"250"} Jul 12 19:03:24.386623 ip-10-0-129-128 kubenswrapper[2559]: I0712 19:03:24.386587 2559 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-pwsvh"] Jul 12 19:03:24.502121 ip-10-0-129-128 kubenswrapper[2559]: I0712 19:03:24.502085 2559 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-shared-b-53dad9-openshift-default-5d59cb9c4c-nnkps" event={"ID":"747c5adc-afa3-4f4c-85bc-e3181fb60ac4","Type":"ContainerStarted","Data":"7cb54c0ee85eac84cfaf8284f4645cbeafa3c5f1d38d33013ea699c11e4e78d9"} Jul 12 19:03:24.677683 ip-10-0-129-128 kubenswrapper[2559]: I0712 19:03:24.677563 2559 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="openshift-ingress/e2e-shared-b-53dad9-openshift-default-5d59cb9c4c-nnkps" Jul 12 19:03:24.678901 ip-10-0-129-128 kubenswrapper[2559]: I0712 19:03:24.678870 2559 patch_prober.go:28] interesting pod/e2e-shared-b-53dad9-openshift-default-5d59cb9c4c-nnkps container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.132.0.53:15021/healthz/ready\": dial tcp 10.132.0.53:15021: connect: connection refused" start-of-body= Jul 12 19:03:24.679028 ip-10-0-129-128 kubenswrapper[2559]: I0712 19:03:24.678932 2559 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-shared-b-53dad9-openshift-default-5d59cb9c4c-nnkps" podUID="747c5adc-afa3-4f4c-85bc-e3181fb60ac4" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.132.0.53:15021/healthz/ready\": dial tcp 10.132.0.53:15021: connect: connection refused" Jul 12 19:03:25.413692 ip-10-0-129-128 kubenswrapper[2559]: I0712 19:03:25.413655 2559 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/e2e-shared-b-53dad9-openshift-default-5d59cb9c4c-nnkps" Jul 12 19:03:25.677671 ip-10-0-129-128 kubenswrapper[2559]: I0712 19:03:25.677559 2559 patch_prober.go:28] interesting pod/e2e-shared-b-53dad9-openshift-default-5d59cb9c4c-nnkps container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.132.0.53:15021/healthz/ready\": dial tcp 10.132.0.53:15021: connect: connection refused" start-of-body= Jul 12 19:03:25.677671 ip-10-0-129-128 kubenswrapper[2559]: I0712 19:03:25.677623 2559 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-shared-b-53dad9-openshift-default-5d59cb9c4c-nnkps" podUID="747c5adc-afa3-4f4c-85bc-e3181fb60ac4" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.132.0.53:15021/healthz/ready\": dial tcp 10.132.0.53:15021: connect: connection refused" Jul 12 19:03:26.677178 ip-10-0-129-128 kubenswrapper[2559]: I0712 19:03:26.677142 2559 patch_prober.go:28] interesting pod/e2e-shared-b-53dad9-openshift-default-5d59cb9c4c-nnkps container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.132.0.53:15021/healthz/ready\": dial tcp 10.132.0.53:15021: connect: connection refused" start-of-body= Jul 12 19:03:26.677405 ip-10-0-129-128 kubenswrapper[2559]: I0712 19:03:26.677208 2559 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-shared-b-53dad9-openshift-default-5d59cb9c4c-nnkps" podUID="747c5adc-afa3-4f4c-85bc-e3181fb60ac4" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.132.0.53:15021/healthz/ready\": dial tcp 10.132.0.53:15021: connect: connection refused" Jul 12 19:03:27.676934 ip-10-0-129-128 kubenswrapper[2559]: I0712 19:03:27.676900 2559 patch_prober.go:28] interesting pod/e2e-shared-b-53dad9-openshift-default-5d59cb9c4c-nnkps container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.132.0.53:15021/healthz/ready\": dial tcp 10.132.0.53:15021: connect: connection refused" start-of-body= Jul 12 19:03:27.677389 ip-10-0-129-128 kubenswrapper[2559]: I0712 19:03:27.676954 2559 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-shared-b-53dad9-openshift-default-5d59cb9c4c-nnkps" podUID="747c5adc-afa3-4f4c-85bc-e3181fb60ac4" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.132.0.53:15021/healthz/ready\": dial tcp 10.132.0.53:15021: connect: connection refused" Jul 12 19:03:28.681087 ip-10-0-129-128 kubenswrapper[2559]: I0712 19:03:28.681056 2559 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="openshift-ingress/e2e-shared-b-53dad9-openshift-default-5d59cb9c4c-nnkps" Jul 12 19:03:28.682163 ip-10-0-129-128 kubenswrapper[2559]: I0712 19:03:28.682142 2559 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/e2e-shared-b-53dad9-openshift-default-5d59cb9c4c-nnkps" Jul 12 19:03:34.987371 ip-10-0-129-128 kubenswrapper[2559]: I0712 19:03:34.987331 2559 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-pwsvh"] Jul 12 19:03:49.788530 ip-10-0-129-128 kubenswrapper[2559]: I0712 19:03:49.788495 2559 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-pwsvh"] Jul 12 19:04:03.708574 ip-10-0-129-128 kubenswrapper[2559]: I0712 19:04:03.708537 2559 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-pwsvh"] Jul 12 19:04:23.680916 ip-10-0-129-128 kubenswrapper[2559]: I0712 19:04:23.680879 2559 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-pwsvh"] Jul 12 19:04:40.077655 ip-10-0-129-128 kubenswrapper[2559]: I0712 19:04:40.077610 2559 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-pwsvh"] Jul 12 19:04:58.689461 ip-10-0-129-128 kubenswrapper[2559]: I0712 19:04:58.689428 2559 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-pwsvh"] Jul 12 19:05:17.685864 ip-10-0-129-128 kubenswrapper[2559]: I0712 19:05:17.685768 2559 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-pwsvh"] Jul 12 19:05:34.591148 ip-10-0-129-128 kubenswrapper[2559]: I0712 19:05:34.591111 2559 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-pwsvh"] Jul 12 19:05:54.090018 ip-10-0-129-128 kubenswrapper[2559]: I0712 19:05:54.089980 2559 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-pwsvh"] Jul 12 19:06:14.383683 ip-10-0-129-128 kubenswrapper[2559]: I0712 19:06:14.383650 2559 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-pwsvh"] Jul 12 19:06:37.912286 ip-10-0-129-128 kubenswrapper[2559]: I0712 19:06:37.912061 2559 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-pwsvh"] Jul 12 19:06:58.284758 ip-10-0-129-128 kubenswrapper[2559]: I0712 19:06:58.284720 2559 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-pwsvh"] Jul 12 19:07:19.282805 ip-10-0-129-128 kubenswrapper[2559]: I0712 19:07:19.282769 2559 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-pwsvh"] Jul 12 19:07:35.488522 ip-10-0-129-128 kubenswrapper[2559]: I0712 19:07:35.488480 2559 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-pwsvh"] Jul 12 19:07:56.489163 ip-10-0-129-128 kubenswrapper[2559]: I0712 19:07:56.489127 2559 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-pwsvh"] Jul 12 19:08:12.090695 ip-10-0-129-128 kubenswrapper[2559]: I0712 19:08:12.090587 2559 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-pwsvh"] Jul 12 19:08:13.356547 ip-10-0-129-128 kubenswrapper[2559]: I0712 19:08:13.356516 2559 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-console-operator_console-operator-7866497494-wlk78_f1d46430-bb60-4470-b8d8-eacd44f4f6b1/console-operator/1.log" Jul 12 19:08:13.359409 ip-10-0-129-128 kubenswrapper[2559]: I0712 19:08:13.359385 2559 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ovn-kubernetes_ovnkube-node-q5dw8_e51bad50-bc74-4e38-8307-ca257b599314/ovn-acl-logging/0.log" Jul 12 19:08:13.367367 ip-10-0-129-128 kubenswrapper[2559]: I0712 19:08:13.367347 2559 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-console-operator_console-operator-7866497494-wlk78_f1d46430-bb60-4470-b8d8-eacd44f4f6b1/console-operator/1.log" Jul 12 19:08:13.370349 ip-10-0-129-128 kubenswrapper[2559]: I0712 19:08:13.370333 2559 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ovn-kubernetes_ovnkube-node-q5dw8_e51bad50-bc74-4e38-8307-ca257b599314/ovn-acl-logging/0.log" Jul 12 19:08:33.483818 ip-10-0-129-128 kubenswrapper[2559]: I0712 19:08:33.483783 2559 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-pwsvh"] Jul 12 19:08:48.697838 ip-10-0-129-128 kubenswrapper[2559]: I0712 19:08:48.697791 2559 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-pwsvh"] Jul 12 19:11:40.085428 ip-10-0-129-128 kubenswrapper[2559]: I0712 19:11:40.085386 2559 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-pwsvh"] Jul 12 19:11:42.333614 ip-10-0-129-128 kubenswrapper[2559]: I0712 19:11:42.333574 2559 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-ingress/e2e-shared-a-e6a009-openshift-default-64dfb96979-vgx8n"] Jul 12 19:11:42.334134 ip-10-0-129-128 kubenswrapper[2559]: I0712 19:11:42.333886 2559 kuberuntime_container.go:873] "Killing container with a grace period" pod="openshift-ingress/e2e-shared-a-e6a009-openshift-default-64dfb96979-vgx8n" podUID="fbbdd92f-9c45-4177-9527-9ff6999e4589" containerName="istio-proxy" containerID="cri-o://470aa5319e6bc714193236dd860402144dde5a440926b85db8c01fce7b8747c4" gracePeriod=30 Jul 12 19:11:47.596071 ip-10-0-129-128 kubenswrapper[2559]: I0712 19:11:47.596013 2559 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-shared-a-e6a009-openshift-default-64dfb96979-vgx8n" Jul 12 19:11:47.684510 ip-10-0-129-128 kubenswrapper[2559]: I0712 19:11:47.684471 2559 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/fbbdd92f-9c45-4177-9527-9ff6999e4589-istio-podinfo\") pod \"fbbdd92f-9c45-4177-9527-9ff6999e4589\" (UID: \"fbbdd92f-9c45-4177-9527-9ff6999e4589\") " Jul 12 19:11:47.684747 ip-10-0-129-128 kubenswrapper[2559]: I0712 19:11:47.684516 2559 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/fbbdd92f-9c45-4177-9527-9ff6999e4589-istio-data\") pod \"fbbdd92f-9c45-4177-9527-9ff6999e4589\" (UID: \"fbbdd92f-9c45-4177-9527-9ff6999e4589\") " Jul 12 19:11:47.684747 ip-10-0-129-128 kubenswrapper[2559]: I0712 19:11:47.684584 2559 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/fbbdd92f-9c45-4177-9527-9ff6999e4589-workload-socket\") pod \"fbbdd92f-9c45-4177-9527-9ff6999e4589\" (UID: \"fbbdd92f-9c45-4177-9527-9ff6999e4589\") " Jul 12 19:11:47.684747 ip-10-0-129-128 kubenswrapper[2559]: I0712 19:11:47.684623 2559 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/fbbdd92f-9c45-4177-9527-9ff6999e4589-istio-envoy\") pod \"fbbdd92f-9c45-4177-9527-9ff6999e4589\" (UID: \"fbbdd92f-9c45-4177-9527-9ff6999e4589\") " Jul 12 19:11:47.684747 ip-10-0-129-128 kubenswrapper[2559]: I0712 19:11:47.684656 2559 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/fbbdd92f-9c45-4177-9527-9ff6999e4589-credential-socket\") pod \"fbbdd92f-9c45-4177-9527-9ff6999e4589\" (UID: \"fbbdd92f-9c45-4177-9527-9ff6999e4589\") " Jul 12 19:11:47.684747 ip-10-0-129-128 kubenswrapper[2559]: I0712 19:11:47.684726 2559 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/fbbdd92f-9c45-4177-9527-9ff6999e4589-istiod-ca-cert\") pod \"fbbdd92f-9c45-4177-9527-9ff6999e4589\" (UID: \"fbbdd92f-9c45-4177-9527-9ff6999e4589\") " Jul 12 19:11:47.685022 ip-10-0-129-128 kubenswrapper[2559]: I0712 19:11:47.684753 2559 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-dvh2d\" (UniqueName: \"kubernetes.io/projected/fbbdd92f-9c45-4177-9527-9ff6999e4589-kube-api-access-dvh2d\") pod \"fbbdd92f-9c45-4177-9527-9ff6999e4589\" (UID: \"fbbdd92f-9c45-4177-9527-9ff6999e4589\") " Jul 12 19:11:47.685022 ip-10-0-129-128 kubenswrapper[2559]: I0712 19:11:47.684801 2559 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/fbbdd92f-9c45-4177-9527-9ff6999e4589-workload-certs\") pod \"fbbdd92f-9c45-4177-9527-9ff6999e4589\" (UID: \"fbbdd92f-9c45-4177-9527-9ff6999e4589\") " Jul 12 19:11:47.685022 ip-10-0-129-128 kubenswrapper[2559]: I0712 19:11:47.684829 2559 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/fbbdd92f-9c45-4177-9527-9ff6999e4589-istio-token\") pod \"fbbdd92f-9c45-4177-9527-9ff6999e4589\" (UID: \"fbbdd92f-9c45-4177-9527-9ff6999e4589\") " Jul 12 19:11:47.685022 ip-10-0-129-128 kubenswrapper[2559]: I0712 19:11:47.684902 2559 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/fbbdd92f-9c45-4177-9527-9ff6999e4589-workload-socket" (OuterVolumeSpecName: "workload-socket") pod "fbbdd92f-9c45-4177-9527-9ff6999e4589" (UID: "fbbdd92f-9c45-4177-9527-9ff6999e4589"). InnerVolumeSpecName "workload-socket". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Jul 12 19:11:47.685022 ip-10-0-129-128 kubenswrapper[2559]: I0712 19:11:47.685004 2559 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/fbbdd92f-9c45-4177-9527-9ff6999e4589-credential-socket" (OuterVolumeSpecName: "credential-socket") pod "fbbdd92f-9c45-4177-9527-9ff6999e4589" (UID: "fbbdd92f-9c45-4177-9527-9ff6999e4589"). InnerVolumeSpecName "credential-socket". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Jul 12 19:11:47.685277 ip-10-0-129-128 kubenswrapper[2559]: I0712 19:11:47.685103 2559 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/fbbdd92f-9c45-4177-9527-9ff6999e4589-istiod-ca-cert" (OuterVolumeSpecName: "istiod-ca-cert") pod "fbbdd92f-9c45-4177-9527-9ff6999e4589" (UID: "fbbdd92f-9c45-4177-9527-9ff6999e4589"). InnerVolumeSpecName "istiod-ca-cert". PluginName "kubernetes.io/configmap", VolumeGIDValue "" Jul 12 19:11:47.685277 ip-10-0-129-128 kubenswrapper[2559]: I0712 19:11:47.685203 2559 reconciler_common.go:299] "Volume detached for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/fbbdd92f-9c45-4177-9527-9ff6999e4589-workload-socket\") on node \"ip-10-0-129-128.ec2.internal\" DevicePath \"\"" Jul 12 19:11:47.685277 ip-10-0-129-128 kubenswrapper[2559]: I0712 19:11:47.685223 2559 reconciler_common.go:299] "Volume detached for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/fbbdd92f-9c45-4177-9527-9ff6999e4589-credential-socket\") on node \"ip-10-0-129-128.ec2.internal\" DevicePath \"\"" Jul 12 19:11:47.685277 ip-10-0-129-128 kubenswrapper[2559]: I0712 19:11:47.685237 2559 reconciler_common.go:299] "Volume detached for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/fbbdd92f-9c45-4177-9527-9ff6999e4589-istiod-ca-cert\") on node \"ip-10-0-129-128.ec2.internal\" DevicePath \"\"" Jul 12 19:11:47.685470 ip-10-0-129-128 kubenswrapper[2559]: I0712 19:11:47.685312 2559 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/fbbdd92f-9c45-4177-9527-9ff6999e4589-workload-certs" (OuterVolumeSpecName: "workload-certs") pod "fbbdd92f-9c45-4177-9527-9ff6999e4589" (UID: "fbbdd92f-9c45-4177-9527-9ff6999e4589"). InnerVolumeSpecName "workload-certs". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Jul 12 19:11:47.685551 ip-10-0-129-128 kubenswrapper[2559]: I0712 19:11:47.685523 2559 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/fbbdd92f-9c45-4177-9527-9ff6999e4589-istio-data" (OuterVolumeSpecName: "istio-data") pod "fbbdd92f-9c45-4177-9527-9ff6999e4589" (UID: "fbbdd92f-9c45-4177-9527-9ff6999e4589"). InnerVolumeSpecName "istio-data". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Jul 12 19:11:47.686772 ip-10-0-129-128 kubenswrapper[2559]: I0712 19:11:47.686747 2559 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/downward-api/fbbdd92f-9c45-4177-9527-9ff6999e4589-istio-podinfo" (OuterVolumeSpecName: "istio-podinfo") pod "fbbdd92f-9c45-4177-9527-9ff6999e4589" (UID: "fbbdd92f-9c45-4177-9527-9ff6999e4589"). InnerVolumeSpecName "istio-podinfo". PluginName "kubernetes.io/downward-api", VolumeGIDValue "" Jul 12 19:11:47.686923 ip-10-0-129-128 kubenswrapper[2559]: I0712 19:11:47.686900 2559 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/fbbdd92f-9c45-4177-9527-9ff6999e4589-istio-envoy" (OuterVolumeSpecName: "istio-envoy") pod "fbbdd92f-9c45-4177-9527-9ff6999e4589" (UID: "fbbdd92f-9c45-4177-9527-9ff6999e4589"). InnerVolumeSpecName "istio-envoy". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Jul 12 19:11:47.687296 ip-10-0-129-128 kubenswrapper[2559]: I0712 19:11:47.687275 2559 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/fbbdd92f-9c45-4177-9527-9ff6999e4589-istio-token" (OuterVolumeSpecName: "istio-token") pod "fbbdd92f-9c45-4177-9527-9ff6999e4589" (UID: "fbbdd92f-9c45-4177-9527-9ff6999e4589"). InnerVolumeSpecName "istio-token". PluginName "kubernetes.io/projected", VolumeGIDValue "" Jul 12 19:11:47.687394 ip-10-0-129-128 kubenswrapper[2559]: I0712 19:11:47.687294 2559 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/fbbdd92f-9c45-4177-9527-9ff6999e4589-kube-api-access-dvh2d" (OuterVolumeSpecName: "kube-api-access-dvh2d") pod "fbbdd92f-9c45-4177-9527-9ff6999e4589" (UID: "fbbdd92f-9c45-4177-9527-9ff6999e4589"). InnerVolumeSpecName "kube-api-access-dvh2d". PluginName "kubernetes.io/projected", VolumeGIDValue "" Jul 12 19:11:47.786471 ip-10-0-129-128 kubenswrapper[2559]: I0712 19:11:47.786425 2559 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-dvh2d\" (UniqueName: \"kubernetes.io/projected/fbbdd92f-9c45-4177-9527-9ff6999e4589-kube-api-access-dvh2d\") on node \"ip-10-0-129-128.ec2.internal\" DevicePath \"\"" Jul 12 19:11:47.786471 ip-10-0-129-128 kubenswrapper[2559]: I0712 19:11:47.786466 2559 reconciler_common.go:299] "Volume detached for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/fbbdd92f-9c45-4177-9527-9ff6999e4589-workload-certs\") on node \"ip-10-0-129-128.ec2.internal\" DevicePath \"\"" Jul 12 19:11:47.786471 ip-10-0-129-128 kubenswrapper[2559]: I0712 19:11:47.786475 2559 reconciler_common.go:299] "Volume detached for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/fbbdd92f-9c45-4177-9527-9ff6999e4589-istio-token\") on node \"ip-10-0-129-128.ec2.internal\" DevicePath \"\"" Jul 12 19:11:47.786471 ip-10-0-129-128 kubenswrapper[2559]: I0712 19:11:47.786484 2559 reconciler_common.go:299] "Volume detached for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/fbbdd92f-9c45-4177-9527-9ff6999e4589-istio-podinfo\") on node \"ip-10-0-129-128.ec2.internal\" DevicePath \"\"" Jul 12 19:11:47.786471 ip-10-0-129-128 kubenswrapper[2559]: I0712 19:11:47.786492 2559 reconciler_common.go:299] "Volume detached for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/fbbdd92f-9c45-4177-9527-9ff6999e4589-istio-data\") on node \"ip-10-0-129-128.ec2.internal\" DevicePath \"\"" Jul 12 19:11:47.786820 ip-10-0-129-128 kubenswrapper[2559]: I0712 19:11:47.786500 2559 reconciler_common.go:299] "Volume detached for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/fbbdd92f-9c45-4177-9527-9ff6999e4589-istio-envoy\") on node \"ip-10-0-129-128.ec2.internal\" DevicePath \"\"" Jul 12 19:11:48.015630 ip-10-0-129-128 kubenswrapper[2559]: I0712 19:11:48.015592 2559 generic.go:356] "Generic (PLEG): container finished" podID="fbbdd92f-9c45-4177-9527-9ff6999e4589" containerID="470aa5319e6bc714193236dd860402144dde5a440926b85db8c01fce7b8747c4" exitCode=0 Jul 12 19:11:48.015831 ip-10-0-129-128 kubenswrapper[2559]: I0712 19:11:48.015667 2559 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-shared-a-e6a009-openshift-default-64dfb96979-vgx8n" Jul 12 19:11:48.015831 ip-10-0-129-128 kubenswrapper[2559]: I0712 19:11:48.015675 2559 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-shared-a-e6a009-openshift-default-64dfb96979-vgx8n" event={"ID":"fbbdd92f-9c45-4177-9527-9ff6999e4589","Type":"ContainerDied","Data":"470aa5319e6bc714193236dd860402144dde5a440926b85db8c01fce7b8747c4"} Jul 12 19:11:48.015831 ip-10-0-129-128 kubenswrapper[2559]: I0712 19:11:48.015726 2559 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-shared-a-e6a009-openshift-default-64dfb96979-vgx8n" event={"ID":"fbbdd92f-9c45-4177-9527-9ff6999e4589","Type":"ContainerDied","Data":"30051032c954cc7fbd6557320f6216314d4fd25ba895b8c8329a523680143135"} Jul 12 19:11:48.015831 ip-10-0-129-128 kubenswrapper[2559]: I0712 19:11:48.015748 2559 scope.go:117] "RemoveContainer" containerID="470aa5319e6bc714193236dd860402144dde5a440926b85db8c01fce7b8747c4" Jul 12 19:11:48.025446 ip-10-0-129-128 kubenswrapper[2559]: I0712 19:11:48.025422 2559 scope.go:117] "RemoveContainer" containerID="0ebc7347097ca115ea7802c36c2f847bcea4ed1c4c4583c5d53d08321a065e96" Jul 12 19:11:48.035480 ip-10-0-129-128 kubenswrapper[2559]: I0712 19:11:48.035456 2559 scope.go:117] "RemoveContainer" containerID="470aa5319e6bc714193236dd860402144dde5a440926b85db8c01fce7b8747c4" Jul 12 19:11:48.035963 ip-10-0-129-128 kubenswrapper[2559]: E0712 19:11:48.035890 2559 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"470aa5319e6bc714193236dd860402144dde5a440926b85db8c01fce7b8747c4\": container with ID starting with 470aa5319e6bc714193236dd860402144dde5a440926b85db8c01fce7b8747c4 not found: ID does not exist" containerID="470aa5319e6bc714193236dd860402144dde5a440926b85db8c01fce7b8747c4" Jul 12 19:11:48.035963 ip-10-0-129-128 kubenswrapper[2559]: I0712 19:11:48.035926 2559 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"470aa5319e6bc714193236dd860402144dde5a440926b85db8c01fce7b8747c4"} err="failed to get container status \"470aa5319e6bc714193236dd860402144dde5a440926b85db8c01fce7b8747c4\": rpc error: code = NotFound desc = could not find container \"470aa5319e6bc714193236dd860402144dde5a440926b85db8c01fce7b8747c4\": container with ID starting with 470aa5319e6bc714193236dd860402144dde5a440926b85db8c01fce7b8747c4 not found: ID does not exist" Jul 12 19:11:48.035963 ip-10-0-129-128 kubenswrapper[2559]: I0712 19:11:48.035951 2559 scope.go:117] "RemoveContainer" containerID="0ebc7347097ca115ea7802c36c2f847bcea4ed1c4c4583c5d53d08321a065e96" Jul 12 19:11:48.036239 ip-10-0-129-128 kubenswrapper[2559]: E0712 19:11:48.036213 2559 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"0ebc7347097ca115ea7802c36c2f847bcea4ed1c4c4583c5d53d08321a065e96\": container with ID starting with 0ebc7347097ca115ea7802c36c2f847bcea4ed1c4c4583c5d53d08321a065e96 not found: ID does not exist" containerID="0ebc7347097ca115ea7802c36c2f847bcea4ed1c4c4583c5d53d08321a065e96" Jul 12 19:11:48.036299 ip-10-0-129-128 kubenswrapper[2559]: I0712 19:11:48.036247 2559 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"0ebc7347097ca115ea7802c36c2f847bcea4ed1c4c4583c5d53d08321a065e96"} err="failed to get container status \"0ebc7347097ca115ea7802c36c2f847bcea4ed1c4c4583c5d53d08321a065e96\": rpc error: code = NotFound desc = could not find container \"0ebc7347097ca115ea7802c36c2f847bcea4ed1c4c4583c5d53d08321a065e96\": container with ID starting with 0ebc7347097ca115ea7802c36c2f847bcea4ed1c4c4583c5d53d08321a065e96 not found: ID does not exist" Jul 12 19:11:48.037577 ip-10-0-129-128 kubenswrapper[2559]: I0712 19:11:48.037555 2559 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-ingress/e2e-shared-a-e6a009-openshift-default-64dfb96979-vgx8n"] Jul 12 19:11:48.040918 ip-10-0-129-128 kubenswrapper[2559]: I0712 19:11:48.040893 2559 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["openshift-ingress/e2e-shared-a-e6a009-openshift-default-64dfb96979-vgx8n"] Jul 12 19:11:49.033907 ip-10-0-129-128 kubenswrapper[2559]: I0712 19:11:49.033867 2559 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="fbbdd92f-9c45-4177-9527-9ff6999e4589" path="/var/lib/kubelet/pods/fbbdd92f-9c45-4177-9527-9ff6999e4589/volumes" Jul 12 19:13:13.386121 ip-10-0-129-128 kubenswrapper[2559]: I0712 19:13:13.386090 2559 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-console-operator_console-operator-7866497494-wlk78_f1d46430-bb60-4470-b8d8-eacd44f4f6b1/console-operator/1.log" Jul 12 19:13:13.388848 ip-10-0-129-128 kubenswrapper[2559]: I0712 19:13:13.388826 2559 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ovn-kubernetes_ovnkube-node-q5dw8_e51bad50-bc74-4e38-8307-ca257b599314/ovn-acl-logging/0.log" Jul 12 19:13:13.403511 ip-10-0-129-128 kubenswrapper[2559]: I0712 19:13:13.403489 2559 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-console-operator_console-operator-7866497494-wlk78_f1d46430-bb60-4470-b8d8-eacd44f4f6b1/console-operator/1.log" Jul 12 19:13:13.405957 ip-10-0-129-128 kubenswrapper[2559]: I0712 19:13:13.405939 2559 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ovn-kubernetes_ovnkube-node-q5dw8_e51bad50-bc74-4e38-8307-ca257b599314/ovn-acl-logging/0.log" Jul 12 19:14:51.682045 ip-10-0-129-128 kubenswrapper[2559]: I0712 19:14:51.682007 2559 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-pwsvh"] Jul 12 19:14:54.876749 ip-10-0-129-128 kubenswrapper[2559]: I0712 19:14:54.876716 2559 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-ingress/e2e-shared-b-53dad9-openshift-default-5d59cb9c4c-nnkps"] Jul 12 19:14:54.877174 ip-10-0-129-128 kubenswrapper[2559]: I0712 19:14:54.876918 2559 kuberuntime_container.go:873] "Killing container with a grace period" pod="openshift-ingress/e2e-shared-b-53dad9-openshift-default-5d59cb9c4c-nnkps" podUID="747c5adc-afa3-4f4c-85bc-e3181fb60ac4" containerName="istio-proxy" containerID="cri-o://7cb54c0ee85eac84cfaf8284f4645cbeafa3c5f1d38d33013ea699c11e4e78d9" gracePeriod=30 Jul 12 19:14:55.525209 ip-10-0-129-128 kubenswrapper[2559]: I0712 19:14:55.525171 2559 log.go:25] "Finished parsing log file" path="/var/log/pods/kuadrant-system_authorino-7497fcb44d-xbwrz_411b22c6-e8d8-4930-827f-c2c94d231237/authorino/0.log" Jul 12 19:14:58.683484 ip-10-0-129-128 kubenswrapper[2559]: I0712 19:14:58.683441 2559 patch_prober.go:28] interesting pod/e2e-shared-b-53dad9-openshift-default-5d59cb9c4c-nnkps container/istio-proxy namespace/openshift-ingress: Readiness probe status=failure output="HTTP probe failed with statuscode: 503" start-of-body= Jul 12 19:14:58.683858 ip-10-0-129-128 kubenswrapper[2559]: I0712 19:14:58.683527 2559 prober.go:120] "Probe failed" probeType="Readiness" pod="openshift-ingress/e2e-shared-b-53dad9-openshift-default-5d59cb9c4c-nnkps" podUID="747c5adc-afa3-4f4c-85bc-e3181fb60ac4" containerName="istio-proxy" probeResult="failure" output="HTTP probe failed with statuscode: 503" Jul 12 19:15:00.143505 ip-10-0-129-128 kubenswrapper[2559]: I0712 19:15:00.143444 2559 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-shared-b-53dad9-openshift-default-5d59cb9c4c-nnkps" Jul 12 19:15:00.236134 ip-10-0-129-128 kubenswrapper[2559]: I0712 19:15:00.236104 2559 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-z56sw\" (UniqueName: \"kubernetes.io/projected/747c5adc-afa3-4f4c-85bc-e3181fb60ac4-kube-api-access-z56sw\") pod \"747c5adc-afa3-4f4c-85bc-e3181fb60ac4\" (UID: \"747c5adc-afa3-4f4c-85bc-e3181fb60ac4\") " Jul 12 19:15:00.236399 ip-10-0-129-128 kubenswrapper[2559]: I0712 19:15:00.236141 2559 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/747c5adc-afa3-4f4c-85bc-e3181fb60ac4-istio-data\") pod \"747c5adc-afa3-4f4c-85bc-e3181fb60ac4\" (UID: \"747c5adc-afa3-4f4c-85bc-e3181fb60ac4\") " Jul 12 19:15:00.236399 ip-10-0-129-128 kubenswrapper[2559]: I0712 19:15:00.236172 2559 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/747c5adc-afa3-4f4c-85bc-e3181fb60ac4-credential-socket\") pod \"747c5adc-afa3-4f4c-85bc-e3181fb60ac4\" (UID: \"747c5adc-afa3-4f4c-85bc-e3181fb60ac4\") " Jul 12 19:15:00.236399 ip-10-0-129-128 kubenswrapper[2559]: I0712 19:15:00.236199 2559 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/747c5adc-afa3-4f4c-85bc-e3181fb60ac4-istio-envoy\") pod \"747c5adc-afa3-4f4c-85bc-e3181fb60ac4\" (UID: \"747c5adc-afa3-4f4c-85bc-e3181fb60ac4\") " Jul 12 19:15:00.236399 ip-10-0-129-128 kubenswrapper[2559]: I0712 19:15:00.236260 2559 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/747c5adc-afa3-4f4c-85bc-e3181fb60ac4-workload-socket\") pod \"747c5adc-afa3-4f4c-85bc-e3181fb60ac4\" (UID: \"747c5adc-afa3-4f4c-85bc-e3181fb60ac4\") " Jul 12 19:15:00.236399 ip-10-0-129-128 kubenswrapper[2559]: I0712 19:15:00.236301 2559 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/747c5adc-afa3-4f4c-85bc-e3181fb60ac4-istiod-ca-cert\") pod \"747c5adc-afa3-4f4c-85bc-e3181fb60ac4\" (UID: \"747c5adc-afa3-4f4c-85bc-e3181fb60ac4\") " Jul 12 19:15:00.236399 ip-10-0-129-128 kubenswrapper[2559]: I0712 19:15:00.236332 2559 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/747c5adc-afa3-4f4c-85bc-e3181fb60ac4-istio-podinfo\") pod \"747c5adc-afa3-4f4c-85bc-e3181fb60ac4\" (UID: \"747c5adc-afa3-4f4c-85bc-e3181fb60ac4\") " Jul 12 19:15:00.236399 ip-10-0-129-128 kubenswrapper[2559]: I0712 19:15:00.236383 2559 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/747c5adc-afa3-4f4c-85bc-e3181fb60ac4-workload-certs\") pod \"747c5adc-afa3-4f4c-85bc-e3181fb60ac4\" (UID: \"747c5adc-afa3-4f4c-85bc-e3181fb60ac4\") " Jul 12 19:15:00.236827 ip-10-0-129-128 kubenswrapper[2559]: I0712 19:15:00.236422 2559 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/747c5adc-afa3-4f4c-85bc-e3181fb60ac4-istio-token\") pod \"747c5adc-afa3-4f4c-85bc-e3181fb60ac4\" (UID: \"747c5adc-afa3-4f4c-85bc-e3181fb60ac4\") " Jul 12 19:15:00.236827 ip-10-0-129-128 kubenswrapper[2559]: I0712 19:15:00.236569 2559 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/747c5adc-afa3-4f4c-85bc-e3181fb60ac4-credential-socket" (OuterVolumeSpecName: "credential-socket") pod "747c5adc-afa3-4f4c-85bc-e3181fb60ac4" (UID: "747c5adc-afa3-4f4c-85bc-e3181fb60ac4"). InnerVolumeSpecName "credential-socket". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Jul 12 19:15:00.236827 ip-10-0-129-128 kubenswrapper[2559]: I0712 19:15:00.236737 2559 reconciler_common.go:299] "Volume detached for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/747c5adc-afa3-4f4c-85bc-e3181fb60ac4-credential-socket\") on node \"ip-10-0-129-128.ec2.internal\" DevicePath \"\"" Jul 12 19:15:00.237105 ip-10-0-129-128 kubenswrapper[2559]: I0712 19:15:00.237081 2559 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/747c5adc-afa3-4f4c-85bc-e3181fb60ac4-istiod-ca-cert" (OuterVolumeSpecName: "istiod-ca-cert") pod "747c5adc-afa3-4f4c-85bc-e3181fb60ac4" (UID: "747c5adc-afa3-4f4c-85bc-e3181fb60ac4"). InnerVolumeSpecName "istiod-ca-cert". PluginName "kubernetes.io/configmap", VolumeGIDValue "" Jul 12 19:15:00.237430 ip-10-0-129-128 kubenswrapper[2559]: I0712 19:15:00.237273 2559 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/747c5adc-afa3-4f4c-85bc-e3181fb60ac4-istio-data" (OuterVolumeSpecName: "istio-data") pod "747c5adc-afa3-4f4c-85bc-e3181fb60ac4" (UID: "747c5adc-afa3-4f4c-85bc-e3181fb60ac4"). InnerVolumeSpecName "istio-data". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Jul 12 19:15:00.237430 ip-10-0-129-128 kubenswrapper[2559]: I0712 19:15:00.237324 2559 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/747c5adc-afa3-4f4c-85bc-e3181fb60ac4-workload-socket" (OuterVolumeSpecName: "workload-socket") pod "747c5adc-afa3-4f4c-85bc-e3181fb60ac4" (UID: "747c5adc-afa3-4f4c-85bc-e3181fb60ac4"). InnerVolumeSpecName "workload-socket". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Jul 12 19:15:00.237430 ip-10-0-129-128 kubenswrapper[2559]: I0712 19:15:00.237393 2559 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/747c5adc-afa3-4f4c-85bc-e3181fb60ac4-workload-certs" (OuterVolumeSpecName: "workload-certs") pod "747c5adc-afa3-4f4c-85bc-e3181fb60ac4" (UID: "747c5adc-afa3-4f4c-85bc-e3181fb60ac4"). InnerVolumeSpecName "workload-certs". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Jul 12 19:15:00.238953 ip-10-0-129-128 kubenswrapper[2559]: I0712 19:15:00.238900 2559 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/747c5adc-afa3-4f4c-85bc-e3181fb60ac4-kube-api-access-z56sw" (OuterVolumeSpecName: "kube-api-access-z56sw") pod "747c5adc-afa3-4f4c-85bc-e3181fb60ac4" (UID: "747c5adc-afa3-4f4c-85bc-e3181fb60ac4"). InnerVolumeSpecName "kube-api-access-z56sw". PluginName "kubernetes.io/projected", VolumeGIDValue "" Jul 12 19:15:00.239065 ip-10-0-129-128 kubenswrapper[2559]: I0712 19:15:00.238957 2559 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/747c5adc-afa3-4f4c-85bc-e3181fb60ac4-istio-token" (OuterVolumeSpecName: "istio-token") pod "747c5adc-afa3-4f4c-85bc-e3181fb60ac4" (UID: "747c5adc-afa3-4f4c-85bc-e3181fb60ac4"). InnerVolumeSpecName "istio-token". PluginName "kubernetes.io/projected", VolumeGIDValue "" Jul 12 19:15:00.239161 ip-10-0-129-128 kubenswrapper[2559]: I0712 19:15:00.239116 2559 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/747c5adc-afa3-4f4c-85bc-e3181fb60ac4-istio-envoy" (OuterVolumeSpecName: "istio-envoy") pod "747c5adc-afa3-4f4c-85bc-e3181fb60ac4" (UID: "747c5adc-afa3-4f4c-85bc-e3181fb60ac4"). InnerVolumeSpecName "istio-envoy". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Jul 12 19:15:00.239392 ip-10-0-129-128 kubenswrapper[2559]: I0712 19:15:00.239374 2559 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/downward-api/747c5adc-afa3-4f4c-85bc-e3181fb60ac4-istio-podinfo" (OuterVolumeSpecName: "istio-podinfo") pod "747c5adc-afa3-4f4c-85bc-e3181fb60ac4" (UID: "747c5adc-afa3-4f4c-85bc-e3181fb60ac4"). InnerVolumeSpecName "istio-podinfo". PluginName "kubernetes.io/downward-api", VolumeGIDValue "" Jul 12 19:15:00.271614 ip-10-0-129-128 kubenswrapper[2559]: I0712 19:15:00.271587 2559 log.go:25] "Finished parsing log file" path="/var/log/pods/opendatahub_kserve-controller-manager-d6f4b65b8-zm6bp_402923e6-af3f-48fc-ae8e-f6cd828a0e73/manager/0.log" Jul 12 19:15:00.337123 ip-10-0-129-128 kubenswrapper[2559]: I0712 19:15:00.337090 2559 reconciler_common.go:299] "Volume detached for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/747c5adc-afa3-4f4c-85bc-e3181fb60ac4-workload-certs\") on node \"ip-10-0-129-128.ec2.internal\" DevicePath \"\"" Jul 12 19:15:00.337123 ip-10-0-129-128 kubenswrapper[2559]: I0712 19:15:00.337115 2559 reconciler_common.go:299] "Volume detached for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/747c5adc-afa3-4f4c-85bc-e3181fb60ac4-istio-token\") on node \"ip-10-0-129-128.ec2.internal\" DevicePath \"\"" Jul 12 19:15:00.337123 ip-10-0-129-128 kubenswrapper[2559]: I0712 19:15:00.337125 2559 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-z56sw\" (UniqueName: \"kubernetes.io/projected/747c5adc-afa3-4f4c-85bc-e3181fb60ac4-kube-api-access-z56sw\") on node \"ip-10-0-129-128.ec2.internal\" DevicePath \"\"" Jul 12 19:15:00.337423 ip-10-0-129-128 kubenswrapper[2559]: I0712 19:15:00.337136 2559 reconciler_common.go:299] "Volume detached for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/747c5adc-afa3-4f4c-85bc-e3181fb60ac4-istio-data\") on node \"ip-10-0-129-128.ec2.internal\" DevicePath \"\"" Jul 12 19:15:00.337423 ip-10-0-129-128 kubenswrapper[2559]: I0712 19:15:00.337145 2559 reconciler_common.go:299] "Volume detached for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/747c5adc-afa3-4f4c-85bc-e3181fb60ac4-istio-envoy\") on node \"ip-10-0-129-128.ec2.internal\" DevicePath \"\"" Jul 12 19:15:00.337423 ip-10-0-129-128 kubenswrapper[2559]: I0712 19:15:00.337153 2559 reconciler_common.go:299] "Volume detached for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/747c5adc-afa3-4f4c-85bc-e3181fb60ac4-workload-socket\") on node \"ip-10-0-129-128.ec2.internal\" DevicePath \"\"" Jul 12 19:15:00.337423 ip-10-0-129-128 kubenswrapper[2559]: I0712 19:15:00.337161 2559 reconciler_common.go:299] "Volume detached for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/747c5adc-afa3-4f4c-85bc-e3181fb60ac4-istiod-ca-cert\") on node \"ip-10-0-129-128.ec2.internal\" DevicePath \"\"" Jul 12 19:15:00.337423 ip-10-0-129-128 kubenswrapper[2559]: I0712 19:15:00.337169 2559 reconciler_common.go:299] "Volume detached for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/747c5adc-afa3-4f4c-85bc-e3181fb60ac4-istio-podinfo\") on node \"ip-10-0-129-128.ec2.internal\" DevicePath \"\"" Jul 12 19:15:00.619826 ip-10-0-129-128 kubenswrapper[2559]: I0712 19:15:00.619798 2559 log.go:25] "Finished parsing log file" path="/var/log/pods/opendatahub_model-serving-api-54d89bd5fc-c46wr_ddbd25e2-7012-4650-b0ee-53d2854a2c8f/server/0.log" Jul 12 19:15:00.740349 ip-10-0-129-128 kubenswrapper[2559]: I0712 19:15:00.740319 2559 log.go:25] "Finished parsing log file" path="/var/log/pods/opendatahub_odh-model-controller-7998c67dc4-tdm5v_5315b422-0354-44c7-be09-903e5cb60ca8/manager/0.log" Jul 12 19:15:00.856341 ip-10-0-129-128 kubenswrapper[2559]: I0712 19:15:00.856316 2559 log.go:25] "Finished parsing log file" path="/var/log/pods/opendatahub_opendatahub-operator-controller-manager-5dbd4cc77c-dphpk_e845af50-76b4-4945-b406-f7fb65e0f475/copy-manifests/0.log" Jul 12 19:15:00.862398 ip-10-0-129-128 kubenswrapper[2559]: I0712 19:15:00.862373 2559 log.go:25] "Finished parsing log file" path="/var/log/pods/opendatahub_opendatahub-operator-controller-manager-5dbd4cc77c-dphpk_e845af50-76b4-4945-b406-f7fb65e0f475/manager/0.log" Jul 12 19:15:00.945427 ip-10-0-129-128 kubenswrapper[2559]: I0712 19:15:00.945339 2559 generic.go:356] "Generic (PLEG): container finished" podID="747c5adc-afa3-4f4c-85bc-e3181fb60ac4" containerID="7cb54c0ee85eac84cfaf8284f4645cbeafa3c5f1d38d33013ea699c11e4e78d9" exitCode=0 Jul 12 19:15:00.945427 ip-10-0-129-128 kubenswrapper[2559]: I0712 19:15:00.945382 2559 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-shared-b-53dad9-openshift-default-5d59cb9c4c-nnkps" Jul 12 19:15:00.945617 ip-10-0-129-128 kubenswrapper[2559]: I0712 19:15:00.945424 2559 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-shared-b-53dad9-openshift-default-5d59cb9c4c-nnkps" event={"ID":"747c5adc-afa3-4f4c-85bc-e3181fb60ac4","Type":"ContainerDied","Data":"7cb54c0ee85eac84cfaf8284f4645cbeafa3c5f1d38d33013ea699c11e4e78d9"} Jul 12 19:15:00.945617 ip-10-0-129-128 kubenswrapper[2559]: I0712 19:15:00.945489 2559 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-shared-b-53dad9-openshift-default-5d59cb9c4c-nnkps" event={"ID":"747c5adc-afa3-4f4c-85bc-e3181fb60ac4","Type":"ContainerDied","Data":"ac8f3d64219ede4bff9973812c424b3c4e8469b6c22fc56a3b739d8addac31ff"} Jul 12 19:15:00.945617 ip-10-0-129-128 kubenswrapper[2559]: I0712 19:15:00.945526 2559 scope.go:117] "RemoveContainer" containerID="7cb54c0ee85eac84cfaf8284f4645cbeafa3c5f1d38d33013ea699c11e4e78d9" Jul 12 19:15:00.958030 ip-10-0-129-128 kubenswrapper[2559]: I0712 19:15:00.958002 2559 scope.go:117] "RemoveContainer" containerID="be1da2d2c5579643b608ef46ecbadf2273cf794909e23a7fe715234537efc34e" Jul 12 19:15:00.967143 ip-10-0-129-128 kubenswrapper[2559]: I0712 19:15:00.967122 2559 scope.go:117] "RemoveContainer" containerID="7cb54c0ee85eac84cfaf8284f4645cbeafa3c5f1d38d33013ea699c11e4e78d9" Jul 12 19:15:00.967462 ip-10-0-129-128 kubenswrapper[2559]: E0712 19:15:00.967428 2559 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"7cb54c0ee85eac84cfaf8284f4645cbeafa3c5f1d38d33013ea699c11e4e78d9\": container with ID starting with 7cb54c0ee85eac84cfaf8284f4645cbeafa3c5f1d38d33013ea699c11e4e78d9 not found: ID does not exist" containerID="7cb54c0ee85eac84cfaf8284f4645cbeafa3c5f1d38d33013ea699c11e4e78d9" Jul 12 19:15:00.967564 ip-10-0-129-128 kubenswrapper[2559]: I0712 19:15:00.967477 2559 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"7cb54c0ee85eac84cfaf8284f4645cbeafa3c5f1d38d33013ea699c11e4e78d9"} err="failed to get container status \"7cb54c0ee85eac84cfaf8284f4645cbeafa3c5f1d38d33013ea699c11e4e78d9\": rpc error: code = NotFound desc = could not find container \"7cb54c0ee85eac84cfaf8284f4645cbeafa3c5f1d38d33013ea699c11e4e78d9\": container with ID starting with 7cb54c0ee85eac84cfaf8284f4645cbeafa3c5f1d38d33013ea699c11e4e78d9 not found: ID does not exist" Jul 12 19:15:00.967564 ip-10-0-129-128 kubenswrapper[2559]: I0712 19:15:00.967503 2559 scope.go:117] "RemoveContainer" containerID="be1da2d2c5579643b608ef46ecbadf2273cf794909e23a7fe715234537efc34e" Jul 12 19:15:00.967854 ip-10-0-129-128 kubenswrapper[2559]: E0712 19:15:00.967819 2559 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"be1da2d2c5579643b608ef46ecbadf2273cf794909e23a7fe715234537efc34e\": container with ID starting with be1da2d2c5579643b608ef46ecbadf2273cf794909e23a7fe715234537efc34e not found: ID does not exist" containerID="be1da2d2c5579643b608ef46ecbadf2273cf794909e23a7fe715234537efc34e" Jul 12 19:15:00.967957 ip-10-0-129-128 kubenswrapper[2559]: I0712 19:15:00.967857 2559 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"be1da2d2c5579643b608ef46ecbadf2273cf794909e23a7fe715234537efc34e"} err="failed to get container status \"be1da2d2c5579643b608ef46ecbadf2273cf794909e23a7fe715234537efc34e\": rpc error: code = NotFound desc = could not find container \"be1da2d2c5579643b608ef46ecbadf2273cf794909e23a7fe715234537efc34e\": container with ID starting with be1da2d2c5579643b608ef46ecbadf2273cf794909e23a7fe715234537efc34e not found: ID does not exist" Jul 12 19:15:00.970177 ip-10-0-129-128 kubenswrapper[2559]: I0712 19:15:00.970155 2559 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-ingress/e2e-shared-b-53dad9-openshift-default-5d59cb9c4c-nnkps"] Jul 12 19:15:00.973275 ip-10-0-129-128 kubenswrapper[2559]: I0712 19:15:00.973256 2559 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["openshift-ingress/e2e-shared-b-53dad9-openshift-default-5d59cb9c4c-nnkps"] Jul 12 19:15:01.037255 ip-10-0-129-128 kubenswrapper[2559]: I0712 19:15:01.037228 2559 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="747c5adc-afa3-4f4c-85bc-e3181fb60ac4" path="/var/lib/kubelet/pods/747c5adc-afa3-4f4c-85bc-e3181fb60ac4/volumes" Jul 12 19:15:02.850329 ip-10-0-129-128 kubenswrapper[2559]: I0712 19:15:02.850281 2559 log.go:25] "Finished parsing log file" path="/var/log/pods/kuadrant-system_authorino-7497fcb44d-xbwrz_411b22c6-e8d8-4930-827f-c2c94d231237/authorino/0.log" Jul 12 19:15:02.960751 ip-10-0-129-128 kubenswrapper[2559]: I0712 19:15:02.960717 2559 log.go:25] "Finished parsing log file" path="/var/log/pods/kuadrant-system_authorino-operator-75cc8d5cb7-85b85_e41e01dc-8b6c-499c-8402-58ed840ff0de/manager/0.log" Jul 12 19:15:03.074109 ip-10-0-129-128 kubenswrapper[2559]: I0712 19:15:03.074076 2559 log.go:25] "Finished parsing log file" path="/var/log/pods/kuadrant-system_dns-operator-controller-manager-864dd78c4c-zwc7p_e8c1fa35-12cf-4ac7-8f0d-0413e0eba07f/manager/0.log" Jul 12 19:15:03.304368 ip-10-0-129-128 kubenswrapper[2559]: I0712 19:15:03.304333 2559 log.go:25] "Finished parsing log file" path="/var/log/pods/kuadrant-system_kuadrant-operator-catalog-gnh7d_c03827ed-4ed1-49b1-9ff7-0e20fbeb0722/registry-server/0.log" Jul 12 19:15:03.421328 ip-10-0-129-128 kubenswrapper[2559]: I0712 19:15:03.421288 2559 log.go:25] "Finished parsing log file" path="/var/log/pods/kuadrant-system_kuadrant-operator-controller-manager-86878c769f-q7zgh_af76461c-8878-47bb-bae3-bc2d8eb02f76/manager/0.log" Jul 12 19:15:03.534679 ip-10-0-129-128 kubenswrapper[2559]: I0712 19:15:03.534631 2559 log.go:25] "Finished parsing log file" path="/var/log/pods/kuadrant-system_limitador-limitador-54d6d988d5-pwsvh_645f2a0a-1588-47a1-a7c9-ab7eaeec4326/limitador/0.log" Jul 12 19:15:03.653306 ip-10-0-129-128 kubenswrapper[2559]: I0712 19:15:03.653225 2559 log.go:25] "Finished parsing log file" path="/var/log/pods/kuadrant-system_limitador-operator-controller-manager-859d7b78bd-jh5jl_07c44f20-f5a7-46ac-9534-3ebf12e0ebb9/manager/0.log" Jul 12 19:15:03.995269 ip-10-0-129-128 kubenswrapper[2559]: I0712 19:15:03.995183 2559 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ingress_data-science-gateway-data-science-gateway-class-6576859cf49q7gm_6b94673d-3ac1-4706-b8eb-aaa44bb04be3/istio-proxy/0.log" Jul 12 19:15:04.225355 ip-10-0-129-128 kubenswrapper[2559]: I0712 19:15:04.225323 2559 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ingress_kube-auth-proxy-795d67df6d-knt8z_77be4045-2796-4d09-b161-95ad4a170b23/kube-auth-proxy/0.log" Jul 12 19:15:04.442627 ip-10-0-129-128 kubenswrapper[2559]: I0712 19:15:04.442597 2559 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ingress_maas-default-gateway-openshift-default-56654dcf7-vxfhz_37f0b65a-a3ee-4aee-a490-3c9888b7475f/istio-proxy/1.log" Jul 12 19:15:09.872262 ip-10-0-129-128 kubenswrapper[2559]: I0712 19:15:09.872229 2559 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-must-gather-h56g7/must-gather-qq6rt"] Jul 12 19:15:09.872676 ip-10-0-129-128 kubenswrapper[2559]: I0712 19:15:09.872571 2559 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="747c5adc-afa3-4f4c-85bc-e3181fb60ac4" containerName="istio-proxy" Jul 12 19:15:09.872676 ip-10-0-129-128 kubenswrapper[2559]: I0712 19:15:09.872582 2559 state_mem.go:107] "Deleted CPUSet assignment" podUID="747c5adc-afa3-4f4c-85bc-e3181fb60ac4" containerName="istio-proxy" Jul 12 19:15:09.872676 ip-10-0-129-128 kubenswrapper[2559]: I0712 19:15:09.872592 2559 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="747c5adc-afa3-4f4c-85bc-e3181fb60ac4" containerName="istio-proxy" Jul 12 19:15:09.872676 ip-10-0-129-128 kubenswrapper[2559]: I0712 19:15:09.872600 2559 state_mem.go:107] "Deleted CPUSet assignment" podUID="747c5adc-afa3-4f4c-85bc-e3181fb60ac4" containerName="istio-proxy" Jul 12 19:15:09.872676 ip-10-0-129-128 kubenswrapper[2559]: I0712 19:15:09.872617 2559 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="fbbdd92f-9c45-4177-9527-9ff6999e4589" containerName="istio-proxy" Jul 12 19:15:09.872676 ip-10-0-129-128 kubenswrapper[2559]: I0712 19:15:09.872622 2559 state_mem.go:107] "Deleted CPUSet assignment" podUID="fbbdd92f-9c45-4177-9527-9ff6999e4589" containerName="istio-proxy" Jul 12 19:15:09.872676 ip-10-0-129-128 kubenswrapper[2559]: I0712 19:15:09.872632 2559 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="6770b17c-88fb-45af-8337-00e99f7ecf19" containerName="istio-proxy" Jul 12 19:15:09.872676 ip-10-0-129-128 kubenswrapper[2559]: I0712 19:15:09.872661 2559 state_mem.go:107] "Deleted CPUSet assignment" podUID="6770b17c-88fb-45af-8337-00e99f7ecf19" containerName="istio-proxy" Jul 12 19:15:09.872676 ip-10-0-129-128 kubenswrapper[2559]: I0712 19:15:09.872670 2559 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="fbbdd92f-9c45-4177-9527-9ff6999e4589" containerName="istio-proxy" Jul 12 19:15:09.872676 ip-10-0-129-128 kubenswrapper[2559]: I0712 19:15:09.872675 2559 state_mem.go:107] "Deleted CPUSet assignment" podUID="fbbdd92f-9c45-4177-9527-9ff6999e4589" containerName="istio-proxy" Jul 12 19:15:09.873041 ip-10-0-129-128 kubenswrapper[2559]: I0712 19:15:09.872733 2559 memory_manager.go:368] "RemoveStaleState removing state" podUID="fbbdd92f-9c45-4177-9527-9ff6999e4589" containerName="istio-proxy" Jul 12 19:15:09.873041 ip-10-0-129-128 kubenswrapper[2559]: I0712 19:15:09.872740 2559 memory_manager.go:368] "RemoveStaleState removing state" podUID="747c5adc-afa3-4f4c-85bc-e3181fb60ac4" containerName="istio-proxy" Jul 12 19:15:09.873041 ip-10-0-129-128 kubenswrapper[2559]: I0712 19:15:09.872748 2559 memory_manager.go:368] "RemoveStaleState removing state" podUID="fbbdd92f-9c45-4177-9527-9ff6999e4589" containerName="istio-proxy" Jul 12 19:15:09.873041 ip-10-0-129-128 kubenswrapper[2559]: I0712 19:15:09.872753 2559 memory_manager.go:368] "RemoveStaleState removing state" podUID="6770b17c-88fb-45af-8337-00e99f7ecf19" containerName="istio-proxy" Jul 12 19:15:09.873041 ip-10-0-129-128 kubenswrapper[2559]: I0712 19:15:09.872861 2559 memory_manager.go:368] "RemoveStaleState removing state" podUID="747c5adc-afa3-4f4c-85bc-e3181fb60ac4" containerName="istio-proxy" Jul 12 19:15:09.875850 ip-10-0-129-128 kubenswrapper[2559]: I0712 19:15:09.875832 2559 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-must-gather-h56g7/must-gather-qq6rt" Jul 12 19:15:09.878384 ip-10-0-129-128 kubenswrapper[2559]: I0712 19:15:09.878363 2559 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-must-gather-h56g7\"/\"kube-root-ca.crt\"" Jul 12 19:15:09.878723 ip-10-0-129-128 kubenswrapper[2559]: I0712 19:15:09.878700 2559 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-must-gather-h56g7\"/\"openshift-service-ca.crt\"" Jul 12 19:15:09.888591 ip-10-0-129-128 kubenswrapper[2559]: I0712 19:15:09.888571 2559 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-must-gather-h56g7/must-gather-qq6rt"] Jul 12 19:15:09.917936 ip-10-0-129-128 kubenswrapper[2559]: I0712 19:15:09.917907 2559 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"must-gather-output\" (UniqueName: \"kubernetes.io/empty-dir/9d5aecea-b748-43bc-9759-15a73a1acf0e-must-gather-output\") pod \"must-gather-qq6rt\" (UID: \"9d5aecea-b748-43bc-9759-15a73a1acf0e\") " pod="openshift-must-gather-h56g7/must-gather-qq6rt" Jul 12 19:15:09.918063 ip-10-0-129-128 kubenswrapper[2559]: I0712 19:15:09.917966 2559 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-xhhp9\" (UniqueName: \"kubernetes.io/projected/9d5aecea-b748-43bc-9759-15a73a1acf0e-kube-api-access-xhhp9\") pod \"must-gather-qq6rt\" (UID: \"9d5aecea-b748-43bc-9759-15a73a1acf0e\") " pod="openshift-must-gather-h56g7/must-gather-qq6rt" Jul 12 19:15:10.019268 ip-10-0-129-128 kubenswrapper[2559]: I0712 19:15:10.019232 2559 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"must-gather-output\" (UniqueName: \"kubernetes.io/empty-dir/9d5aecea-b748-43bc-9759-15a73a1acf0e-must-gather-output\") pod \"must-gather-qq6rt\" (UID: \"9d5aecea-b748-43bc-9759-15a73a1acf0e\") " pod="openshift-must-gather-h56g7/must-gather-qq6rt" Jul 12 19:15:10.019457 ip-10-0-129-128 kubenswrapper[2559]: I0712 19:15:10.019290 2559 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-xhhp9\" (UniqueName: \"kubernetes.io/projected/9d5aecea-b748-43bc-9759-15a73a1acf0e-kube-api-access-xhhp9\") pod \"must-gather-qq6rt\" (UID: \"9d5aecea-b748-43bc-9759-15a73a1acf0e\") " pod="openshift-must-gather-h56g7/must-gather-qq6rt" Jul 12 19:15:10.019592 ip-10-0-129-128 kubenswrapper[2559]: I0712 19:15:10.019560 2559 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"must-gather-output\" (UniqueName: \"kubernetes.io/empty-dir/9d5aecea-b748-43bc-9759-15a73a1acf0e-must-gather-output\") pod \"must-gather-qq6rt\" (UID: \"9d5aecea-b748-43bc-9759-15a73a1acf0e\") " pod="openshift-must-gather-h56g7/must-gather-qq6rt" Jul 12 19:15:10.026892 ip-10-0-129-128 kubenswrapper[2559]: I0712 19:15:10.026867 2559 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-xhhp9\" (UniqueName: \"kubernetes.io/projected/9d5aecea-b748-43bc-9759-15a73a1acf0e-kube-api-access-xhhp9\") pod \"must-gather-qq6rt\" (UID: \"9d5aecea-b748-43bc-9759-15a73a1acf0e\") " pod="openshift-must-gather-h56g7/must-gather-qq6rt" Jul 12 19:15:10.185142 ip-10-0-129-128 kubenswrapper[2559]: I0712 19:15:10.185055 2559 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-must-gather-h56g7/must-gather-qq6rt" Jul 12 19:15:10.318962 ip-10-0-129-128 kubenswrapper[2559]: I0712 19:15:10.318938 2559 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-must-gather-h56g7/must-gather-qq6rt"] Jul 12 19:15:10.320913 ip-10-0-129-128 kubenswrapper[2559]: W0712 19:15:10.320880 2559 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-besteffort.slice/kubepods-besteffort-pod9d5aecea_b748_43bc_9759_15a73a1acf0e.slice/crio-68f5c971f6d2282d55147e04ff791f1c75b0109cb30efc763ed307f6c9ee707b WatchSource:0}: Error finding container 68f5c971f6d2282d55147e04ff791f1c75b0109cb30efc763ed307f6c9ee707b: Status 404 returned error can't find the container with id 68f5c971f6d2282d55147e04ff791f1c75b0109cb30efc763ed307f6c9ee707b Jul 12 19:15:10.322720 ip-10-0-129-128 kubenswrapper[2559]: I0712 19:15:10.322701 2559 provider.go:93] Refreshing cache for provider: *credentialprovider.defaultDockerConfigProvider Jul 12 19:15:10.994822 ip-10-0-129-128 kubenswrapper[2559]: I0712 19:15:10.994784 2559 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-must-gather-h56g7/must-gather-qq6rt" event={"ID":"9d5aecea-b748-43bc-9759-15a73a1acf0e","Type":"ContainerStarted","Data":"68f5c971f6d2282d55147e04ff791f1c75b0109cb30efc763ed307f6c9ee707b"} Jul 12 19:15:12.002784 ip-10-0-129-128 kubenswrapper[2559]: I0712 19:15:12.002748 2559 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-must-gather-h56g7/must-gather-qq6rt" event={"ID":"9d5aecea-b748-43bc-9759-15a73a1acf0e","Type":"ContainerStarted","Data":"e3ccc367f6437cc3305763daecce05210fb8883d9df4e69fed8e53b012773d7c"} Jul 12 19:15:12.003257 ip-10-0-129-128 kubenswrapper[2559]: I0712 19:15:12.002797 2559 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-must-gather-h56g7/must-gather-qq6rt" event={"ID":"9d5aecea-b748-43bc-9759-15a73a1acf0e","Type":"ContainerStarted","Data":"576e2bb242e1693ebbc2bfdc1a478346ba35dd279a4544f77cc065c72ecb019b"} Jul 12 19:15:12.018601 ip-10-0-129-128 kubenswrapper[2559]: I0712 19:15:12.018538 2559 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-must-gather-h56g7/must-gather-qq6rt" podStartSLOduration=2.173683452 podStartE2EDuration="3.018515036s" totalImagesPullingTime="844.831584ms" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-12 19:15:09 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-12 19:15:12.016535148 +0000 UTC m=+4919.510603078" watchObservedRunningTime="2026-07-12 19:15:12.018515036 +0000 UTC m=+4919.512582967" Jul 12 19:15:12.762944 ip-10-0-129-128 kubenswrapper[2559]: I0712 19:15:12.762908 2559 log.go:25] "Finished parsing log file" path="/var/log/pods/kube-system_global-pull-secret-syncer-mpqvv_6ab3daa1-3376-4f53-8106-d6d7417a327e/global-pull-secret-syncer/0.log" Jul 12 19:15:12.871781 ip-10-0-129-128 kubenswrapper[2559]: I0712 19:15:12.871744 2559 log.go:25] "Finished parsing log file" path="/var/log/pods/kube-system_konnectivity-agent-g6rx5_64692aea-0243-4167-a3b6-2a6dc81a8fef/konnectivity-agent/0.log" Jul 12 19:15:12.932872 ip-10-0-129-128 kubenswrapper[2559]: I0712 19:15:12.932839 2559 log.go:25] "Finished parsing log file" path="/var/log/pods/kube-system_kube-apiserver-proxy-ip-10-0-129-128.ec2.internal_89625644b4c452349901306bb1be9b95/haproxy/0.log" Jul 12 19:15:18.201671 ip-10-0-129-128 kubenswrapper[2559]: I0712 19:15:18.201612 2559 log.go:25] "Finished parsing log file" path="/var/log/pods/kuadrant-system_authorino-7497fcb44d-xbwrz_411b22c6-e8d8-4930-827f-c2c94d231237/authorino/0.log" Jul 12 19:15:18.230445 ip-10-0-129-128 kubenswrapper[2559]: I0712 19:15:18.230325 2559 log.go:25] "Finished parsing log file" path="/var/log/pods/kuadrant-system_authorino-operator-75cc8d5cb7-85b85_e41e01dc-8b6c-499c-8402-58ed840ff0de/manager/0.log" Jul 12 19:15:18.270520 ip-10-0-129-128 kubenswrapper[2559]: I0712 19:15:18.270248 2559 log.go:25] "Finished parsing log file" path="/var/log/pods/kuadrant-system_dns-operator-controller-manager-864dd78c4c-zwc7p_e8c1fa35-12cf-4ac7-8f0d-0413e0eba07f/manager/0.log" Jul 12 19:15:18.351520 ip-10-0-129-128 kubenswrapper[2559]: I0712 19:15:18.351476 2559 log.go:25] "Finished parsing log file" path="/var/log/pods/kuadrant-system_kuadrant-operator-catalog-gnh7d_c03827ed-4ed1-49b1-9ff7-0e20fbeb0722/registry-server/0.log" Jul 12 19:15:18.505551 ip-10-0-129-128 kubenswrapper[2559]: I0712 19:15:18.505297 2559 log.go:25] "Finished parsing log file" path="/var/log/pods/kuadrant-system_kuadrant-operator-controller-manager-86878c769f-q7zgh_af76461c-8878-47bb-bae3-bc2d8eb02f76/manager/0.log" Jul 12 19:15:18.536212 ip-10-0-129-128 kubenswrapper[2559]: I0712 19:15:18.536170 2559 log.go:25] "Finished parsing log file" path="/var/log/pods/kuadrant-system_limitador-limitador-54d6d988d5-pwsvh_645f2a0a-1588-47a1-a7c9-ab7eaeec4326/limitador/0.log" Jul 12 19:15:18.622733 ip-10-0-129-128 kubenswrapper[2559]: I0712 19:15:18.622696 2559 log.go:25] "Finished parsing log file" path="/var/log/pods/kuadrant-system_limitador-operator-controller-manager-859d7b78bd-jh5jl_07c44f20-f5a7-46ac-9534-3ebf12e0ebb9/manager/0.log" Jul 12 19:15:20.857911 ip-10-0-129-128 kubenswrapper[2559]: I0712 19:15:20.857883 2559 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_node-exporter-mt8np_e3bd7aa5-e639-434e-9068-979d50d94274/node-exporter/0.log" Jul 12 19:15:20.880328 ip-10-0-129-128 kubenswrapper[2559]: I0712 19:15:20.880274 2559 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_node-exporter-mt8np_e3bd7aa5-e639-434e-9068-979d50d94274/kube-rbac-proxy/0.log" Jul 12 19:15:20.902232 ip-10-0-129-128 kubenswrapper[2559]: I0712 19:15:20.902202 2559 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_node-exporter-mt8np_e3bd7aa5-e639-434e-9068-979d50d94274/init-textfile/0.log" Jul 12 19:15:22.615714 ip-10-0-129-128 kubenswrapper[2559]: I0712 19:15:22.615683 2559 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-network-console_networking-console-plugin-6cf856758c-7ghw7_2edbd7ab-1773-4fbf-af70-0b2cfd5cc642/networking-console-plugin/0.log" Jul 12 19:15:22.888976 ip-10-0-129-128 kubenswrapper[2559]: I0712 19:15:22.888867 2559 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-must-gather-h56g7/perf-node-gather-daemonset-vpm5h"] Jul 12 19:15:22.894878 ip-10-0-129-128 kubenswrapper[2559]: I0712 19:15:22.894844 2559 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-must-gather-h56g7/perf-node-gather-daemonset-vpm5h" Jul 12 19:15:22.897706 ip-10-0-129-128 kubenswrapper[2559]: I0712 19:15:22.897677 2559 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-must-gather-h56g7\"/\"default-dockercfg-p4lkn\"" Jul 12 19:15:22.902420 ip-10-0-129-128 kubenswrapper[2559]: I0712 19:15:22.902374 2559 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-must-gather-h56g7/perf-node-gather-daemonset-vpm5h"] Jul 12 19:15:22.955368 ip-10-0-129-128 kubenswrapper[2559]: I0712 19:15:22.955333 2559 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-ccnw2\" (UniqueName: \"kubernetes.io/projected/3dc81f21-9b13-4523-a5b8-7f07ef54d08a-kube-api-access-ccnw2\") pod \"perf-node-gather-daemonset-vpm5h\" (UID: \"3dc81f21-9b13-4523-a5b8-7f07ef54d08a\") " pod="openshift-must-gather-h56g7/perf-node-gather-daemonset-vpm5h" Jul 12 19:15:22.955544 ip-10-0-129-128 kubenswrapper[2559]: I0712 19:15:22.955430 2559 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"sys\" (UniqueName: \"kubernetes.io/host-path/3dc81f21-9b13-4523-a5b8-7f07ef54d08a-sys\") pod \"perf-node-gather-daemonset-vpm5h\" (UID: \"3dc81f21-9b13-4523-a5b8-7f07ef54d08a\") " pod="openshift-must-gather-h56g7/perf-node-gather-daemonset-vpm5h" Jul 12 19:15:22.955544 ip-10-0-129-128 kubenswrapper[2559]: I0712 19:15:22.955511 2559 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"lib-modules\" (UniqueName: \"kubernetes.io/host-path/3dc81f21-9b13-4523-a5b8-7f07ef54d08a-lib-modules\") pod \"perf-node-gather-daemonset-vpm5h\" (UID: \"3dc81f21-9b13-4523-a5b8-7f07ef54d08a\") " pod="openshift-must-gather-h56g7/perf-node-gather-daemonset-vpm5h" Jul 12 19:15:22.955688 ip-10-0-129-128 kubenswrapper[2559]: I0712 19:15:22.955546 2559 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"proc\" (UniqueName: \"kubernetes.io/host-path/3dc81f21-9b13-4523-a5b8-7f07ef54d08a-proc\") pod \"perf-node-gather-daemonset-vpm5h\" (UID: \"3dc81f21-9b13-4523-a5b8-7f07ef54d08a\") " pod="openshift-must-gather-h56g7/perf-node-gather-daemonset-vpm5h" Jul 12 19:15:22.955688 ip-10-0-129-128 kubenswrapper[2559]: I0712 19:15:22.955576 2559 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"podres\" (UniqueName: \"kubernetes.io/host-path/3dc81f21-9b13-4523-a5b8-7f07ef54d08a-podres\") pod \"perf-node-gather-daemonset-vpm5h\" (UID: \"3dc81f21-9b13-4523-a5b8-7f07ef54d08a\") " pod="openshift-must-gather-h56g7/perf-node-gather-daemonset-vpm5h" Jul 12 19:15:23.056752 ip-10-0-129-128 kubenswrapper[2559]: I0712 19:15:23.056719 2559 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-ccnw2\" (UniqueName: \"kubernetes.io/projected/3dc81f21-9b13-4523-a5b8-7f07ef54d08a-kube-api-access-ccnw2\") pod \"perf-node-gather-daemonset-vpm5h\" (UID: \"3dc81f21-9b13-4523-a5b8-7f07ef54d08a\") " pod="openshift-must-gather-h56g7/perf-node-gather-daemonset-vpm5h" Jul 12 19:15:23.056931 ip-10-0-129-128 kubenswrapper[2559]: I0712 19:15:23.056768 2559 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"sys\" (UniqueName: \"kubernetes.io/host-path/3dc81f21-9b13-4523-a5b8-7f07ef54d08a-sys\") pod \"perf-node-gather-daemonset-vpm5h\" (UID: \"3dc81f21-9b13-4523-a5b8-7f07ef54d08a\") " pod="openshift-must-gather-h56g7/perf-node-gather-daemonset-vpm5h" Jul 12 19:15:23.056931 ip-10-0-129-128 kubenswrapper[2559]: I0712 19:15:23.056810 2559 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"lib-modules\" (UniqueName: \"kubernetes.io/host-path/3dc81f21-9b13-4523-a5b8-7f07ef54d08a-lib-modules\") pod \"perf-node-gather-daemonset-vpm5h\" (UID: \"3dc81f21-9b13-4523-a5b8-7f07ef54d08a\") " pod="openshift-must-gather-h56g7/perf-node-gather-daemonset-vpm5h" Jul 12 19:15:23.056931 ip-10-0-129-128 kubenswrapper[2559]: I0712 19:15:23.056839 2559 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"proc\" (UniqueName: \"kubernetes.io/host-path/3dc81f21-9b13-4523-a5b8-7f07ef54d08a-proc\") pod \"perf-node-gather-daemonset-vpm5h\" (UID: \"3dc81f21-9b13-4523-a5b8-7f07ef54d08a\") " pod="openshift-must-gather-h56g7/perf-node-gather-daemonset-vpm5h" Jul 12 19:15:23.056931 ip-10-0-129-128 kubenswrapper[2559]: I0712 19:15:23.056863 2559 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"podres\" (UniqueName: \"kubernetes.io/host-path/3dc81f21-9b13-4523-a5b8-7f07ef54d08a-podres\") pod \"perf-node-gather-daemonset-vpm5h\" (UID: \"3dc81f21-9b13-4523-a5b8-7f07ef54d08a\") " pod="openshift-must-gather-h56g7/perf-node-gather-daemonset-vpm5h" Jul 12 19:15:23.057094 ip-10-0-129-128 kubenswrapper[2559]: I0712 19:15:23.056927 2559 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"sys\" (UniqueName: \"kubernetes.io/host-path/3dc81f21-9b13-4523-a5b8-7f07ef54d08a-sys\") pod \"perf-node-gather-daemonset-vpm5h\" (UID: \"3dc81f21-9b13-4523-a5b8-7f07ef54d08a\") " pod="openshift-must-gather-h56g7/perf-node-gather-daemonset-vpm5h" Jul 12 19:15:23.057094 ip-10-0-129-128 kubenswrapper[2559]: I0712 19:15:23.056947 2559 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"proc\" (UniqueName: \"kubernetes.io/host-path/3dc81f21-9b13-4523-a5b8-7f07ef54d08a-proc\") pod \"perf-node-gather-daemonset-vpm5h\" (UID: \"3dc81f21-9b13-4523-a5b8-7f07ef54d08a\") " pod="openshift-must-gather-h56g7/perf-node-gather-daemonset-vpm5h" Jul 12 19:15:23.057094 ip-10-0-129-128 kubenswrapper[2559]: I0712 19:15:23.057044 2559 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"podres\" (UniqueName: \"kubernetes.io/host-path/3dc81f21-9b13-4523-a5b8-7f07ef54d08a-podres\") pod \"perf-node-gather-daemonset-vpm5h\" (UID: \"3dc81f21-9b13-4523-a5b8-7f07ef54d08a\") " pod="openshift-must-gather-h56g7/perf-node-gather-daemonset-vpm5h" Jul 12 19:15:23.057214 ip-10-0-129-128 kubenswrapper[2559]: I0712 19:15:23.057134 2559 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"lib-modules\" (UniqueName: \"kubernetes.io/host-path/3dc81f21-9b13-4523-a5b8-7f07ef54d08a-lib-modules\") pod \"perf-node-gather-daemonset-vpm5h\" (UID: \"3dc81f21-9b13-4523-a5b8-7f07ef54d08a\") " pod="openshift-must-gather-h56g7/perf-node-gather-daemonset-vpm5h" Jul 12 19:15:23.065967 ip-10-0-129-128 kubenswrapper[2559]: I0712 19:15:23.065939 2559 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-ccnw2\" (UniqueName: \"kubernetes.io/projected/3dc81f21-9b13-4523-a5b8-7f07ef54d08a-kube-api-access-ccnw2\") pod \"perf-node-gather-daemonset-vpm5h\" (UID: \"3dc81f21-9b13-4523-a5b8-7f07ef54d08a\") " pod="openshift-must-gather-h56g7/perf-node-gather-daemonset-vpm5h" Jul 12 19:15:23.213277 ip-10-0-129-128 kubenswrapper[2559]: I0712 19:15:23.213183 2559 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-must-gather-h56g7/perf-node-gather-daemonset-vpm5h" Jul 12 19:15:23.292014 ip-10-0-129-128 kubenswrapper[2559]: I0712 19:15:23.291958 2559 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-console-operator_console-operator-7866497494-wlk78_f1d46430-bb60-4470-b8d8-eacd44f4f6b1/console-operator/1.log" Jul 12 19:15:23.301127 ip-10-0-129-128 kubenswrapper[2559]: I0712 19:15:23.301088 2559 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-console-operator_console-operator-7866497494-wlk78_f1d46430-bb60-4470-b8d8-eacd44f4f6b1/console-operator/2.log" Jul 12 19:15:23.400043 ip-10-0-129-128 kubenswrapper[2559]: I0712 19:15:23.397554 2559 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-must-gather-h56g7/perf-node-gather-daemonset-vpm5h"] Jul 12 19:15:24.117667 ip-10-0-129-128 kubenswrapper[2559]: I0712 19:15:24.116176 2559 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-must-gather-h56g7/perf-node-gather-daemonset-vpm5h" event={"ID":"3dc81f21-9b13-4523-a5b8-7f07ef54d08a","Type":"ContainerStarted","Data":"6362c93ce8758a3a94e3911c774298611eab6aab38688656f6fc57110e4237e0"} Jul 12 19:15:24.117667 ip-10-0-129-128 kubenswrapper[2559]: I0712 19:15:24.116243 2559 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-must-gather-h56g7/perf-node-gather-daemonset-vpm5h" event={"ID":"3dc81f21-9b13-4523-a5b8-7f07ef54d08a","Type":"ContainerStarted","Data":"4dcf24199e5a2573ad5ec49ee45d13e5fe101633e424ee14e4013f704d37ef39"} Jul 12 19:15:24.117667 ip-10-0-129-128 kubenswrapper[2559]: I0712 19:15:24.116719 2559 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-must-gather-h56g7/perf-node-gather-daemonset-vpm5h" Jul 12 19:15:24.133277 ip-10-0-129-128 kubenswrapper[2559]: I0712 19:15:24.133205 2559 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-must-gather-h56g7/perf-node-gather-daemonset-vpm5h" podStartSLOduration=2.1331847 podStartE2EDuration="2.1331847s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=true podCreationTimestamp="2026-07-12 19:15:22 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-12 19:15:24.132949507 +0000 UTC m=+4931.627017438" watchObservedRunningTime="2026-07-12 19:15:24.1331847 +0000 UTC m=+4931.627252632" Jul 12 19:15:24.443694 ip-10-0-129-128 kubenswrapper[2559]: I0712 19:15:24.443589 2559 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-cluster-storage-operator_volume-data-source-validator-78495597f6-dnhgq_ab3f4928-9f96-49df-ad17-e6a7fe6f7df3/volume-data-source-validator/0.log" Jul 12 19:15:25.543604 ip-10-0-129-128 kubenswrapper[2559]: I0712 19:15:25.543573 2559 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-dns_dns-default-wvr9g_15d24b76-d54a-409b-91e4-6b9e4f9ac7b2/dns/0.log" Jul 12 19:15:25.563962 ip-10-0-129-128 kubenswrapper[2559]: I0712 19:15:25.563936 2559 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-dns_dns-default-wvr9g_15d24b76-d54a-409b-91e4-6b9e4f9ac7b2/kube-rbac-proxy/0.log" Jul 12 19:15:25.639241 ip-10-0-129-128 kubenswrapper[2559]: I0712 19:15:25.639212 2559 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-dns_node-resolver-fqnvr_41eafd0c-4a0b-48f6-8e30-8e408412d389/dns-node-resolver/0.log" Jul 12 19:15:26.174875 ip-10-0-129-128 kubenswrapper[2559]: I0712 19:15:26.174844 2559 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-image-registry_image-registry-77677cbfcd-2dbbt_0611924a-325e-4d7b-b47a-493b936fa67f/registry/0.log" Jul 12 19:15:26.242692 ip-10-0-129-128 kubenswrapper[2559]: I0712 19:15:26.242663 2559 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-image-registry_node-ca-vxx7w_6d23c7c4-9eb6-483f-baf3-f25e641e2ead/node-ca/0.log" Jul 12 19:15:27.185242 ip-10-0-129-128 kubenswrapper[2559]: I0712 19:15:27.185215 2559 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ingress_data-science-gateway-data-science-gateway-class-6576859cf49q7gm_6b94673d-3ac1-4706-b8eb-aaa44bb04be3/istio-proxy/0.log" Jul 12 19:15:27.428035 ip-10-0-129-128 kubenswrapper[2559]: I0712 19:15:27.428008 2559 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ingress_kube-auth-proxy-795d67df6d-knt8z_77be4045-2796-4d09-b161-95ad4a170b23/kube-auth-proxy/0.log" Jul 12 19:15:27.478044 ip-10-0-129-128 kubenswrapper[2559]: I0712 19:15:27.477932 2559 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ingress_maas-default-gateway-openshift-default-56654dcf7-vxfhz_37f0b65a-a3ee-4aee-a490-3c9888b7475f/istio-proxy/0.log" Jul 12 19:15:27.522865 ip-10-0-129-128 kubenswrapper[2559]: I0712 19:15:27.522824 2559 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ingress_maas-default-gateway-openshift-default-56654dcf7-vxfhz_37f0b65a-a3ee-4aee-a490-3c9888b7475f/istio-proxy/1.log" Jul 12 19:15:28.269460 ip-10-0-129-128 kubenswrapper[2559]: I0712 19:15:28.269433 2559 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ingress-canary_ingress-canary-fdmcz_98f3df53-0008-483f-bb5f-0a5718df3920/serve-healthcheck-canary/0.log" Jul 12 19:15:28.935445 ip-10-0-129-128 kubenswrapper[2559]: I0712 19:15:28.935416 2559 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-insights_insights-runtime-extractor-tnhcr_9868d941-2fea-4f26-9802-46788f17ca08/kube-rbac-proxy/0.log" Jul 12 19:15:28.957002 ip-10-0-129-128 kubenswrapper[2559]: I0712 19:15:28.956960 2559 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-insights_insights-runtime-extractor-tnhcr_9868d941-2fea-4f26-9802-46788f17ca08/exporter/0.log" Jul 12 19:15:28.979247 ip-10-0-129-128 kubenswrapper[2559]: I0712 19:15:28.979217 2559 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-insights_insights-runtime-extractor-tnhcr_9868d941-2fea-4f26-9802-46788f17ca08/extractor/0.log" Jul 12 19:15:30.137001 ip-10-0-129-128 kubenswrapper[2559]: I0712 19:15:30.136975 2559 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-must-gather-h56g7/perf-node-gather-daemonset-vpm5h" Jul 12 19:15:31.045375 ip-10-0-129-128 kubenswrapper[2559]: I0712 19:15:31.045341 2559 log.go:25] "Finished parsing log file" path="/var/log/pods/opendatahub_kserve-controller-manager-d6f4b65b8-zm6bp_402923e6-af3f-48fc-ae8e-f6cd828a0e73/manager/0.log" Jul 12 19:15:31.303110 ip-10-0-129-128 kubenswrapper[2559]: I0712 19:15:31.303034 2559 log.go:25] "Finished parsing log file" path="/var/log/pods/opendatahub_model-serving-api-54d89bd5fc-c46wr_ddbd25e2-7012-4650-b0ee-53d2854a2c8f/server/0.log" Jul 12 19:15:31.392396 ip-10-0-129-128 kubenswrapper[2559]: I0712 19:15:31.392367 2559 log.go:25] "Finished parsing log file" path="/var/log/pods/opendatahub_odh-model-controller-7998c67dc4-tdm5v_5315b422-0354-44c7-be09-903e5cb60ca8/manager/0.log" Jul 12 19:15:31.425089 ip-10-0-129-128 kubenswrapper[2559]: I0712 19:15:31.425056 2559 log.go:25] "Finished parsing log file" path="/var/log/pods/opendatahub_opendatahub-operator-controller-manager-5dbd4cc77c-dphpk_e845af50-76b4-4945-b406-f7fb65e0f475/manager/0.log" Jul 12 19:15:31.447719 ip-10-0-129-128 kubenswrapper[2559]: I0712 19:15:31.447687 2559 log.go:25] "Finished parsing log file" path="/var/log/pods/opendatahub_opendatahub-operator-controller-manager-5dbd4cc77c-dphpk_e845af50-76b4-4945-b406-f7fb65e0f475/copy-manifests/0.log" Jul 12 19:15:33.246731 ip-10-0-129-128 kubenswrapper[2559]: I0712 19:15:33.246703 2559 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ingress_data-science-gateway-data-science-gateway-class-6576859cf49q7gm_6b94673d-3ac1-4706-b8eb-aaa44bb04be3/istio-proxy/0.log" Jul 12 19:15:33.459271 ip-10-0-129-128 kubenswrapper[2559]: I0712 19:15:33.459186 2559 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ingress_kube-auth-proxy-795d67df6d-knt8z_77be4045-2796-4d09-b161-95ad4a170b23/kube-auth-proxy/0.log" Jul 12 19:15:33.488916 ip-10-0-129-128 kubenswrapper[2559]: I0712 19:15:33.488886 2559 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ingress_maas-default-gateway-openshift-default-56654dcf7-vxfhz_37f0b65a-a3ee-4aee-a490-3c9888b7475f/istio-proxy/0.log" Jul 12 19:15:33.524076 ip-10-0-129-128 kubenswrapper[2559]: I0712 19:15:33.524044 2559 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ingress_maas-default-gateway-openshift-default-56654dcf7-vxfhz_37f0b65a-a3ee-4aee-a490-3c9888b7475f/istio-proxy/1.log" Jul 12 19:15:34.792864 ip-10-0-129-128 kubenswrapper[2559]: I0712 19:15:34.792780 2559 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ingress_data-science-gateway-data-science-gateway-class-6576859cf49q7gm_6b94673d-3ac1-4706-b8eb-aaa44bb04be3/istio-proxy/0.log" Jul 12 19:15:35.004380 ip-10-0-129-128 kubenswrapper[2559]: I0712 19:15:35.004342 2559 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ingress_kube-auth-proxy-795d67df6d-knt8z_77be4045-2796-4d09-b161-95ad4a170b23/kube-auth-proxy/0.log" Jul 12 19:15:35.044400 ip-10-0-129-128 kubenswrapper[2559]: I0712 19:15:35.044331 2559 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ingress_maas-default-gateway-openshift-default-56654dcf7-vxfhz_37f0b65a-a3ee-4aee-a490-3c9888b7475f/istio-proxy/0.log" Jul 12 19:15:35.086303 ip-10-0-129-128 kubenswrapper[2559]: I0712 19:15:35.086256 2559 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ingress_maas-default-gateway-openshift-default-56654dcf7-vxfhz_37f0b65a-a3ee-4aee-a490-3c9888b7475f/istio-proxy/1.log" Jul 12 19:15:38.373452 ip-10-0-129-128 kubenswrapper[2559]: I0712 19:15:38.373422 2559 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-kube-storage-version-migrator-operator_kube-storage-version-migrator-operator-799cb7f958-rfppz_e73254f8-d204-4dd4-a86f-512fc5d17ebe/kube-storage-version-migrator-operator/1.log" Jul 12 19:15:38.374254 ip-10-0-129-128 kubenswrapper[2559]: I0712 19:15:38.374232 2559 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-kube-storage-version-migrator-operator_kube-storage-version-migrator-operator-799cb7f958-rfppz_e73254f8-d204-4dd4-a86f-512fc5d17ebe/kube-storage-version-migrator-operator/0.log" Jul 12 19:15:39.372918 ip-10-0-129-128 kubenswrapper[2559]: I0712 19:15:39.372890 2559 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-multus_multus-additional-cni-plugins-9dh8l_370d4e7b-6713-48b2-81ee-f055fe1d5490/kube-multus-additional-cni-plugins/0.log" Jul 12 19:15:39.395616 ip-10-0-129-128 kubenswrapper[2559]: I0712 19:15:39.395591 2559 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-multus_multus-additional-cni-plugins-9dh8l_370d4e7b-6713-48b2-81ee-f055fe1d5490/egress-router-binary-copy/0.log" Jul 12 19:15:39.417099 ip-10-0-129-128 kubenswrapper[2559]: I0712 19:15:39.417074 2559 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-multus_multus-additional-cni-plugins-9dh8l_370d4e7b-6713-48b2-81ee-f055fe1d5490/cni-plugins/0.log" Jul 12 19:15:39.438800 ip-10-0-129-128 kubenswrapper[2559]: I0712 19:15:39.438770 2559 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-multus_multus-additional-cni-plugins-9dh8l_370d4e7b-6713-48b2-81ee-f055fe1d5490/bond-cni-plugin/0.log" Jul 12 19:15:39.460494 ip-10-0-129-128 kubenswrapper[2559]: I0712 19:15:39.460470 2559 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-multus_multus-additional-cni-plugins-9dh8l_370d4e7b-6713-48b2-81ee-f055fe1d5490/routeoverride-cni/0.log" Jul 12 19:15:39.481635 ip-10-0-129-128 kubenswrapper[2559]: I0712 19:15:39.481607 2559 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-multus_multus-additional-cni-plugins-9dh8l_370d4e7b-6713-48b2-81ee-f055fe1d5490/whereabouts-cni-bincopy/0.log" Jul 12 19:15:39.502949 ip-10-0-129-128 kubenswrapper[2559]: I0712 19:15:39.502925 2559 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-multus_multus-additional-cni-plugins-9dh8l_370d4e7b-6713-48b2-81ee-f055fe1d5490/whereabouts-cni/0.log" Jul 12 19:15:39.831582 ip-10-0-129-128 kubenswrapper[2559]: I0712 19:15:39.831532 2559 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-multus_multus-c2jnt_ff97f946-352d-4629-bf96-351edf53ebde/kube-multus/0.log" Jul 12 19:15:39.974553 ip-10-0-129-128 kubenswrapper[2559]: I0712 19:15:39.974520 2559 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-multus_network-metrics-daemon-qtkvt_6f65bad8-9ec7-4a51-8557-74b3f79ef76c/network-metrics-daemon/0.log" Jul 12 19:15:39.997419 ip-10-0-129-128 kubenswrapper[2559]: I0712 19:15:39.997394 2559 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-multus_network-metrics-daemon-qtkvt_6f65bad8-9ec7-4a51-8557-74b3f79ef76c/kube-rbac-proxy/0.log"